2024-04-25T09:45:45.433 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T09:45:45.433 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} --archive /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032 --name neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --owner scheduled_neesingh@teuthology -- /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032/orig.config.yaml 2024-04-25T09:45:45.478 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:45:45.565 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673032' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec kclient: syntax: v2 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1124 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHluDo6QtwvzcKbpG/7dyEJny733MCqqFUZqaO9C1fWAsFyzJ8U8bOVwcgDfWprkYOScQd2DLcXurCjmRVmyFVI= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGKr7J+T0OU30IhNcwztV/zctpDacBJ35Xj7sgY6Ohcl6SDNmEAtm1BkaSjGZsElp2DlMS6CfRqsCYwADzD+aqI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_09:36:50 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T09:45:45.565 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa; will attempt to use it 2024-04-25T09:45:45.566 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa/tasks 2024-04-25T09:45:45.566 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T09:45:45.568 INFO:teuthology.task.internal:Checking packages... 2024-04-25T09:45:45.594 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T09:45:45.594 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T09:45:45.595 INFO:teuthology.packaging:ref: None 2024-04-25T09:45:45.595 INFO:teuthology.packaging:tag: None 2024-04-25T09:45:45.595 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T09:45:45.595 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:45:45.595 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T09:45:45.735 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628.g97ae6cc9 2024-04-25T09:45:45.737 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T09:45:45.743 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T09:45:45.744 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T09:45:45.784 INFO:teuthology.task.internal:Saving configuration 2024-04-25T09:45:45.800 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T09:45:45.808 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:45:45.830 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:40:28.438969', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHluDo6QtwvzcKbpG/7dyEJny733MCqqFUZqaO9C1fWAsFyzJ8U8bOVwcgDfWprkYOScQd2DLcXurCjmRVmyFVI='} 2024-04-25T09:45:45.851 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:40:28.440936', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGKr7J+T0OU30IhNcwztV/zctpDacBJ35Xj7sgY6Ohcl6SDNmEAtm1BkaSjGZsElp2DlMS6CfRqsCYwADzD+aqI='} 2024-04-25T09:45:45.851 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T09:45:45.857 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T09:45:45.858 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T09:45:45.858 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T09:45:45.942 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0e61b52160>, signals=[15]) 2024-04-25T09:45:45.943 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T09:45:45.949 INFO:teuthology.task.internal:Opening connections... 2024-04-25T09:45:45.949 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:45:45.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:45:46.027 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T09:45:46.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:45:46.099 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T09:45:46.107 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-04-25T09:45:46.125 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-04-25T09:45:46.125 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-04-25T09:45:46.182 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2024-04-25T09:45:46.182 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2024-04-25T09:45:46.182 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2024-04-25T09:45:46.182 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:45:46.182 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2024-04-25T09:45:46.182 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:45:46.182 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:45:46.182 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2024-04-25T09:45:46.182 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:45:46.182 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:45:46.183 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:45:46.183 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:45:46.183 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:45:46.183 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:45:46.183 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-04-25T09:45:46.206 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-04-25T09:45:46.223 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-04-25T09:45:46.224 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-04-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-04-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-04-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-04-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-04-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-04-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:45:46.282 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-04-25T09:45:46.314 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T09:45:46.324 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T09:45:46.351 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T09:45:46.351 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:45:46.353 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:45:46.370 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T09:45:46.375 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T09:45:46.375 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:45:46.411 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:45:46.522 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T09:45:46.577 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T09:45:46.591 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T09:45:46.592 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-25T09:45:46.592 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-25T09:45:46.592 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:45:46.593 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:45:46.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:45:46.609 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-04-25T09:45:46.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:45:46.610 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-04-25T09:45:46.666 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:45:46.666 INFO:teuthology.task.kernel:Running kernel on smithi139: 5.14.0-437.el9.x86_64 2024-04-25T09:45:46.666 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum install -y kernel 2024-04-25T09:45:46.667 INFO:teuthology.orchestra.run.smithi150.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:45:46.667 INFO:teuthology.task.kernel:Running kernel on smithi150: 5.14.0-437.el9.x86_64 2024-04-25T09:45:46.668 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum install -y kernel 2024-04-25T09:45:48.889 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T09:45:49.487 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 7.6 MB/s | 8.0 MB 00:01 2024-04-25T09:45:51.297 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-25T09:45:51.432 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-25T09:45:55.405 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-25T09:45:55.916 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 6.8 MB/s | 6.2 MB 00:00 2024-04-25T09:45:57.039 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T09:45:57.566 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-25T09:45:58.310 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T09:45:59.120 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T09:46:03.390 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T09:46:04.284 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 8.0 kB/s | 1.7 kB 00:00 2024-04-25T09:46:05.247 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T09:46:05.247 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T09:46:05.341 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-25T09:46:05.342 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-04-25T09:46:05.342 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-25T09:46:05.432 DEBUG:teuthology.orchestra.run.smithi150:> echo no | sudo yum reinstall kernel || true 2024-04-25T09:46:05.933 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 09:46:03 AM UTC. 2024-04-25T09:46:06.062 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-25T09:46:06.069 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-25T09:46:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-04-25T09:46:06.069 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-25T09:46:06.070 INFO:teuthology.orchestra.run.smithi150.stdout:Reinstalling: 2024-04-25T09:46:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:46:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:46:06.070 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-25T09:46:06.070 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-25T09:46:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:46:06.070 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 255 k 2024-04-25T09:46:06.070 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 0 2024-04-25T09:46:06.070 INFO:teuthology.orchestra.run.smithi150.stderr:Operation aborted. 2024-04-25T09:46:06.115 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T09:46:06.115 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T09:46:06.126 INFO:teuthology.orchestra.run.smithi150.stdout:Is this ok [y/N]: 2024-04-25T09:46:06.126 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum reinstall -y kernel || true 2024-04-25T09:46:06.223 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-04-25T09:46:06.224 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-04-25T09:46:06.224 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-04-25T09:46:06.364 DEBUG:teuthology.orchestra.run.smithi139:> echo no | sudo yum reinstall kernel || true 2024-04-25T09:46:06.626 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 09:46:03 AM UTC. 2024-04-25T09:46:06.741 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-25T09:46:06.742 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-25T09:46:06.742 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-04-25T09:46:06.742 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-25T09:46:06.742 INFO:teuthology.orchestra.run.smithi150.stdout:Reinstalling: 2024-04-25T09:46:06.742 INFO:teuthology.orchestra.run.smithi150.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:46:06.742 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:46:06.742 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-25T09:46:06.742 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-25T09:46:06.742 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:46:06.743 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 255 k 2024-04-25T09:46:06.743 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 0 2024-04-25T09:46:06.743 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-25T09:46:06.906 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 09:46:04 AM UTC. 2024-04-25T09:46:07.031 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-04-25T09:46:07.038 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-04-25T09:46:07.038 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-04-25T09:46:07.038 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-04-25T09:46:07.038 INFO:teuthology.orchestra.run.smithi139.stdout:Reinstalling: 2024-04-25T09:46:07.039 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:46:07.039 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:46:07.039 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-04-25T09:46:07.039 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-04-25T09:46:07.039 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:46:07.039 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 255 k 2024-04-25T09:46:07.039 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 0 2024-04-25T09:46:07.039 INFO:teuthology.orchestra.run.smithi139.stderr:Operation aborted. 2024-04-25T09:46:07.112 INFO:teuthology.orchestra.run.smithi139.stdout:Is this ok [y/N]: 2024-04-25T09:46:07.113 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum reinstall -y kernel || true 2024-04-25T09:46:07.612 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 09:46:04 AM UTC. 2024-04-25T09:46:07.729 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-04-25T09:46:07.730 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-04-25T09:46:07.730 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-04-25T09:46:07.730 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-04-25T09:46:07.730 INFO:teuthology.orchestra.run.smithi139.stdout:Reinstalling: 2024-04-25T09:46:07.730 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:46:07.730 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:46:07.730 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-04-25T09:46:07.730 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-04-25T09:46:07.730 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:46:07.731 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 255 k 2024-04-25T09:46:07.731 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 0 2024-04-25T09:46:07.731 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-04-25T09:46:07.834 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-5.14.0-437.el9.x86_64.rpm 458 kB/s | 255 kB 00:00 2024-04-25T09:46:07.835 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:46:07.835 INFO:teuthology.orchestra.run.smithi150.stdout:Total 233 kB/s | 255 kB 00:01 2024-04-25T09:46:07.851 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-25T09:46:08.029 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-25T09:46:08.029 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-25T09:46:08.034 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-25T09:46:08.035 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-25T09:46:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-25T09:46:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:46:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:46:08.469 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-437.el9.x86_64.rpm 555 kB/s | 255 kB 00:00 2024-04-25T09:46:08.469 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:46:08.469 INFO:teuthology.orchestra.run.smithi139.stdout:Total 345 kB/s | 255 kB 00:00 2024-04-25T09:46:08.490 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-04-25T09:46:08.691 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-04-25T09:46:08.691 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-04-25T09:46:08.697 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-04-25T09:46:08.697 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-04-25T09:46:08.826 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-04-25T09:46:08.890 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:46:08.890 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:46:08.903 INFO:teuthology.orchestra.run.smithi139.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:46:08.903 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:46:09.543 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:46:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:46:09.544 INFO:teuthology.orchestra.run.smithi150.stdout:Reinstalled: 2024-04-25T09:46:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T09:46:09.544 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:46:09.544 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-25T09:46:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:46:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:46:09.713 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T09:46:09.760 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T09:46:09.760 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T09:46:09.760 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T09:46:09.760 DEBUG:teuthology.parallel:result is None 2024-04-25T09:46:10.253 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:46:10.253 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:46:10.253 INFO:teuthology.orchestra.run.smithi139.stdout:Reinstalled: 2024-04-25T09:46:10.253 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T09:46:10.253 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:46:10.253 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-04-25T09:46:10.431 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T09:46:10.477 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T09:46:10.478 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T09:46:10.478 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T09:46:10.478 DEBUG:teuthology.parallel:result is None 2024-04-25T09:46:10.478 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T09:46:10.489 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T09:46:10.489 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:46:10.492 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:46:10.509 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T09:46:10.531 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T09:46:10.556 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T09:46:10.556 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:46:10.559 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:46:10.588 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T09:46:10.595 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T09:46:10.595 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:46:10.617 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:46:10.673 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:46:10.689 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:46:10.692 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:46:10.706 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:46:10.707 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T09:46:10.714 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T09:46:10.715 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:46:10.717 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:46:10.742 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T09:46:10.751 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T09:46:10.751 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:46:10.783 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:46:10.799 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:46:10.877 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:46:10.924 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:46:11.008 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:46:11.044 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:46:11.045 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:46:11.108 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:46:11.152 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:46:11.198 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:46:11.282 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:46:11.322 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:46:11.322 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:46:11.387 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-04-25T09:46:11.389 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-04-25T09:46:11.426 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:46:11.460 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:46:11.846 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T09:46:11.854 INFO:teuthology.task.internal:Starting timer... 2024-04-25T09:46:11.855 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T09:46:11.892 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T09:46:11.917 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2024-04-25T09:46:11.964 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2024-04-25T09:46:11.965 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-04-25T09:46:12.026 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-04-25T09:46:12.028 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T09:46:12.028 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2024-04-25T09:46:12.051 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2024-04-25T09:46:12.052 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-04-25T09:46:12.083 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-04-25T09:46:12.083 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi139.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive'} 2024-04-25T09:46:12.083 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:46:12.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:46:12.116 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:46:12.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:46:12.148 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T09:46:12.148 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:46:12.180 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:46:12.215 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T09:46:12.225 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T09:46:12.230 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T09:46:12.355 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T09:46:12.371 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T09:46:12.373 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi139.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T09:49:40.349 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi139.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2024-04-25T09:49:40.351 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-04-25T09:49:40.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:49:40.429 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-04-25T09:49:40.505 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-04-25T09:49:40.506 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-25T09:49:40.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:49:40.578 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-04-25T09:49:40.618 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-25T09:49:40.618 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T09:49:40.629 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T09:49:40.630 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:49:40.630 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:49:40.633 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:49:40.633 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:49:40.659 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T09:49:40.672 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T09:49:40.687 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T09:49:40.695 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2024-04-25T09:49:40.700 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T09:49:40.705 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2024-04-25T09:49:40.718 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T09:49:40.723 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-04-25T09:49:40.731 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T09:49:40.733 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-04-25T09:49:40.746 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T09:49:40.759 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T09:49:40.788 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:49:40.790 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:49:40.791 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-04-25T09:49:40.791 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:49:40.791 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:49:40.791 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:49:40.791 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:49:40.814 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:49:40.817 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:49:40.817 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-25T09:49:40.818 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:49:40.818 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:49:40.818 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:49:40.818 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:49:40.819 INFO:teuthology.run_tasks:Running task install... 2024-04-25T09:49:40.828 DEBUG:teuthology.task.install:project ceph 2024-04-25T09:49:40.828 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T09:49:40.828 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T09:49:40.828 INFO:teuthology.task.install:Using flavor: default 2024-04-25T09:49:40.834 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T09:49:40.834 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T09:49:40.835 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T09:49:40.837 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:49:40.839 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T09:49:40.839 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:49:40.988 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T09:49:40.988 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T09:49:40.992 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T09:49:40.992 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T09:49:41.292 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T09:49:41.292 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:49:41.292 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T09:49:41.317 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T09:49:41.318 DEBUG:teuthology.orchestra.run.smithi139:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T09:49:41.341 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T09:49:41.341 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:49:41.341 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T09:49:41.368 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T09:49:41.368 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T09:49:41.382 DEBUG:teuthology.orchestra.run.smithi139:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T09:49:41.434 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T09:49:41.455 DEBUG:teuthology.orchestra.run.smithi139:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T09:49:41.508 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T09:49:41.518 INFO:teuthology.orchestra.run.smithi139.stdout:check_obsoletes = 1 2024-04-25T09:49:41.519 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum clean all 2024-04-25T09:49:41.570 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-04-25T09:49:41.572 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-04-25T09:49:41.742 INFO:teuthology.orchestra.run.smithi139.stdout:48 files removed 2024-04-25T09:49:41.770 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T09:49:41.799 INFO:teuthology.orchestra.run.smithi150.stdout:48 files removed 2024-04-25T09:49:41.826 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T09:49:42.830 INFO:teuthology.orchestra.run.smithi139.stdout:ceph packages for x86_64 102 kB/s | 82 kB 00:00 2024-04-25T09:49:42.887 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 102 kB/s | 82 kB 00:00 2024-04-25T09:49:43.497 INFO:teuthology.orchestra.run.smithi139.stdout:ceph noarch packages 16 kB/s | 10 kB 00:00 2024-04-25T09:49:43.596 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 15 kB/s | 10 kB 00:00 2024-04-25T09:49:44.248 INFO:teuthology.orchestra.run.smithi139.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-04-25T09:49:44.305 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-04-25T09:49:45.147 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T09:49:45.398 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 8.0 MB 00:01 2024-04-25T09:49:47.149 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-04-25T09:49:47.640 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-25T09:49:51.125 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-25T09:49:51.957 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.2 MB 00:00 2024-04-25T09:49:52.751 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-25T09:49:53.283 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 16 kB 00:00 2024-04-25T09:49:56.038 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 21 MB 00:03 2024-04-25T09:50:01.102 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T09:50:03.269 INFO:teuthology.orchestra.run.smithi139.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:50:03.270 INFO:teuthology.orchestra.run.smithi139.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:50:03.281 INFO:teuthology.orchestra.run.smithi139.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T09:50:03.282 INFO:teuthology.orchestra.run.smithi139.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T09:50:03.283 INFO:teuthology.orchestra.run.smithi139.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T09:50:03.284 INFO:teuthology.orchestra.run.smithi139.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T09:50:03.285 INFO:teuthology.orchestra.run.smithi139.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T09:50:03.288 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T09:50:03.288 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T09:50:03.350 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-04-25T09:50:03.359 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-04-25T09:50:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-04-25T09:50:03.359 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-04-25T09:50:03.359 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-04-25T09:50:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T09:50:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T09:50:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T09:50:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T09:50:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T09:50:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T09:50:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T09:50:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T09:50:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T09:50:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T09:50:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T09:50:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T09:50:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T09:50:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T09:50:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T09:50:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T09:50:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T09:50:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T09:50:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T09:50:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T09:50:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T09:50:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T09:50:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T09:50:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T09:50:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T09:50:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T09:50:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T09:50:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T09:50:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T09:50:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T09:50:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T09:50:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T09:50:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T09:50:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T09:50:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:50:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T09:50:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T09:50:03.363 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrading: 2024-04-25T09:50:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T09:50:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:50:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T09:50:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T09:50:03.363 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-04-25T09:50:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T09:50:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T09:50:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T09:50:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T09:50:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T09:50:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T09:50:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T09:50:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T09:50:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T09:50:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T09:50:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T09:50:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T09:50:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T09:50:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T09:50:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T09:50:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T09:50:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T09:50:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T09:50:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T09:50:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T09:50:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T09:50:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T09:50:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T09:50:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T09:50:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T09:50:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T09:50:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T09:50:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T09:50:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T09:50:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T09:50:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T09:50:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T09:50:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T09:50:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T09:50:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T09:50:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T09:50:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T09:50:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T09:50:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T09:50:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T09:50:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T09:50:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T09:50:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T09:50:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T09:50:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T09:50:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T09:50:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T09:50:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T09:50:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T09:50:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T09:50:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T09:50:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T09:50:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T09:50:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T09:50:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T09:50:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T09:50:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T09:50:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T09:50:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T09:50:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T09:50:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T09:50:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T09:50:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T09:50:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T09:50:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T09:50:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T09:50:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T09:50:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T09:50:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T09:50:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T09:50:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T09:50:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T09:50:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T09:50:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T09:50:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T09:50:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T09:50:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T09:50:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T09:50:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T09:50:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T09:50:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T09:50:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T09:50:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T09:50:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T09:50:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T09:50:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T09:50:03.373 INFO:teuthology.orchestra.run.smithi139.stdout:Installing weak dependencies: 2024-04-25T09:50:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T09:50:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:50:03.373 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-04-25T09:50:03.373 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-04-25T09:50:03.373 INFO:teuthology.orchestra.run.smithi139.stdout:Install 148 Packages 2024-04-25T09:50:03.373 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrade 4 Packages 2024-04-25T09:50:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:50:03.374 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 215 M 2024-04-25T09:50:03.374 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-04-25T09:50:04.420 INFO:teuthology.orchestra.run.smithi139.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 32 kB/s | 6.6 kB 00:00 2024-04-25T09:50:05.261 INFO:teuthology.orchestra.run.smithi139.stdout:(2/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 5.1 MB/s | 5.3 MB 00:01 2024-04-25T09:50:05.345 INFO:teuthology.orchestra.run.smithi139.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 1.7 MB/s | 145 kB 00:00 2024-04-25T09:50:05.420 INFO:teuthology.orchestra.run.smithi139.stdout:(4/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 886 kB/s | 887 kB 00:01 2024-04-25T09:50:05.696 INFO:teuthology.orchestra.run.smithi139.stdout:(5/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 6.1 MB/s | 2.1 MB 00:00 2024-04-25T09:50:06.382 INFO:teuthology.orchestra.run.smithi139.stdout:(6/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 6.9 MB/s | 4.7 MB 00:00 2024-04-25T09:50:06.421 INFO:teuthology.orchestra.run.smithi139.stdout:(7/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 1.5 MB/s | 1.5 MB 00:01 2024-04-25T09:50:08.229 INFO:teuthology.orchestra.run.smithi139.stdout:(8/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x86 9.1 MB/s | 17 MB 00:01 2024-04-25T09:50:08.296 INFO:teuthology.orchestra.run.smithi139.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 380 kB/s | 25 kB 00:00 2024-04-25T09:50:08.580 INFO:teuthology.orchestra.run.smithi139.stdout:(10/152): ceph-common-19.0.0-1628.g97ae6cc9.el9 4.6 MB/s | 20 MB 00:04 2024-04-25T09:50:08.738 INFO:teuthology.orchestra.run.smithi139.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 1.4 MB/s | 220 kB 00:00 2024-04-25T09:50:08.830 INFO:teuthology.orchestra.run.smithi139.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 296 kB/s | 27 kB 00:00 2024-04-25T09:50:08.914 INFO:teuthology.orchestra.run.smithi139.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 375 kB/s | 31 kB 00:00 2024-04-25T09:50:09.156 INFO:teuthology.orchestra.run.smithi139.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 3.0 MB/s | 744 kB 00:00 2024-04-25T09:50:09.258 INFO:teuthology.orchestra.run.smithi139.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 1.6 MB/s | 163 kB 00:00 2024-04-25T09:50:09.348 INFO:teuthology.orchestra.run.smithi139.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.3 MB/s | 127 kB 00:00 2024-04-25T09:50:09.540 INFO:teuthology.orchestra.run.smithi139.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 2.6 MB/s | 501 kB 00:00 2024-04-25T09:50:10.366 INFO:teuthology.orchestra.run.smithi139.stdout:(18/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 6.2 MB/s | 5.1 MB 00:00 2024-04-25T09:50:10.466 INFO:teuthology.orchestra.run.smithi139.stdout:(19/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 2.4 MB/s | 9.8 MB 00:04 2024-04-25T09:50:10.485 INFO:teuthology.orchestra.run.smithi139.stdout:(20/152): python3-ceph-argparse-19.0.0-1628.g97 386 kB/s | 45 kB 00:00 2024-04-25T09:50:10.658 INFO:teuthology.orchestra.run.smithi139.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 926 kB/s | 162 kB 00:00 2024-04-25T09:50:10.692 INFO:teuthology.orchestra.run.smithi139.stdout:(22/152): python3-ceph-common-19.0.0-1628.g97ae 582 kB/s | 131 kB 00:00 2024-04-25T09:50:10.775 INFO:teuthology.orchestra.run.smithi139.stdout:(23/152): python3-rados-19.0.0-1628.g97ae6cc9.e 2.7 MB/s | 321 kB 00:00 2024-04-25T09:50:10.933 INFO:teuthology.orchestra.run.smithi139.stdout:(24/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 1.2 MB/s | 298 kB 00:00 2024-04-25T09:50:11.026 INFO:teuthology.orchestra.run.smithi139.stdout:(25/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 396 kB/s | 99 kB 00:00 2024-04-25T09:50:11.334 INFO:teuthology.orchestra.run.smithi139.stdout:(26/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 213 kB/s | 85 kB 00:00 2024-04-25T09:50:11.509 INFO:teuthology.orchestra.run.smithi139.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 974 kB/s | 170 kB 00:00 2024-04-25T09:50:11.610 INFO:teuthology.orchestra.run.smithi139.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 314 kB/s | 31 kB 00:00 2024-04-25T09:50:11.710 INFO:teuthology.orchestra.run.smithi139.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.3 MB/s | 138 kB 00:00 2024-04-25T09:50:12.010 INFO:teuthology.orchestra.run.smithi139.stdout:(30/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 3.1 MB/s | 3.1 MB 00:00 2024-04-25T09:50:12.569 INFO:teuthology.orchestra.run.smithi139.stdout:(31/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 2.1 MB/s | 1.8 MB 00:00 2024-04-25T09:50:12.694 INFO:teuthology.orchestra.run.smithi139.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 1.9 MB/s | 248 kB 00:00 2024-04-25T09:50:12.802 INFO:teuthology.orchestra.run.smithi139.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 456 kB/s | 49 kB 00:00 2024-04-25T09:50:12.905 INFO:teuthology.orchestra.run.smithi139.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 150 kB/s | 15 kB 00:00 2024-04-25T09:50:13.036 INFO:teuthology.orchestra.run.smithi139.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 2.0 MB/s | 264 kB 00:00 2024-04-25T09:50:13.537 INFO:teuthology.orchestra.run.smithi139.stdout:(36/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 9.0 MB/s | 47 MB 00:05 2024-04-25T09:50:13.570 INFO:teuthology.orchestra.run.smithi139.stdout:(37/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 1.4 MB/s | 748 kB 00:00 2024-04-25T09:50:13.637 INFO:teuthology.orchestra.run.smithi139.stdout:(38/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 182 kB/s | 18 kB 00:00 2024-04-25T09:50:13.763 INFO:teuthology.orchestra.run.smithi139.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 418 kB/s | 80 kB 00:00 2024-04-25T09:50:13.854 INFO:teuthology.orchestra.run.smithi139.stdout:(40/152): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-04-25T09:50:13.880 INFO:teuthology.orchestra.run.smithi139.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-04-25T09:50:13.955 INFO:teuthology.orchestra.run.smithi139.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-04-25T09:50:14.014 INFO:teuthology.orchestra.run.smithi139.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T09:50:14.047 INFO:teuthology.orchestra.run.smithi139.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 7.3 MB/s | 1.2 MB 00:00 2024-04-25T09:50:14.073 INFO:teuthology.orchestra.run.smithi139.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T09:50:14.098 INFO:teuthology.orchestra.run.smithi139.stdout:(46/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-04-25T09:50:14.123 INFO:teuthology.orchestra.run.smithi139.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-25T09:50:14.148 INFO:teuthology.orchestra.run.smithi139.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T09:50:14.248 INFO:teuthology.orchestra.run.smithi139.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-04-25T09:50:14.307 INFO:teuthology.orchestra.run.smithi139.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-04-25T09:50:14.332 INFO:teuthology.orchestra.run.smithi139.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 258 kB/s | 21 kB 00:00 2024-04-25T09:50:14.357 INFO:teuthology.orchestra.run.smithi139.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-04-25T09:50:14.408 INFO:teuthology.orchestra.run.smithi139.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-04-25T09:50:14.441 INFO:teuthology.orchestra.run.smithi139.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-25T09:50:14.483 INFO:teuthology.orchestra.run.smithi139.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 243 kB/s | 18 kB 00:00 2024-04-25T09:50:14.508 INFO:teuthology.orchestra.run.smithi139.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-04-25T09:50:14.567 INFO:teuthology.orchestra.run.smithi139.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-25T09:50:14.592 INFO:teuthology.orchestra.run.smithi139.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 542 kB/s | 45 kB 00:00 2024-04-25T09:50:14.726 INFO:teuthology.orchestra.run.smithi139.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.1 MB/s | 3.0 MB 00:00 2024-04-25T09:50:14.759 INFO:teuthology.orchestra.run.smithi139.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-04-25T09:50:14.786 INFO:teuthology.orchestra.run.smithi139.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 3.0 MB/s | 582 kB 00:00 2024-04-25T09:50:14.810 INFO:teuthology.orchestra.run.smithi139.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 304 kB/s | 25 kB 00:00 2024-04-25T09:50:14.835 INFO:teuthology.orchestra.run.smithi139.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-04-25T09:50:14.885 INFO:teuthology.orchestra.run.smithi139.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 707 kB/s | 35 kB 00:00 2024-04-25T09:50:14.910 INFO:teuthology.orchestra.run.smithi139.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-25T09:50:14.936 INFO:teuthology.orchestra.run.smithi139.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-25T09:50:15.145 INFO:teuthology.orchestra.run.smithi139.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-25T09:50:15.311 INFO:teuthology.orchestra.run.smithi139.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-04-25T09:50:15.370 INFO:teuthology.orchestra.run.smithi139.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-04-25T09:50:15.420 INFO:teuthology.orchestra.run.smithi139.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 3.2 MB/s | 911 kB 00:00 2024-04-25T09:50:15.456 INFO:teuthology.orchestra.run.smithi139.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 925 kB/s | 133 kB 00:00 2024-04-25T09:50:15.523 INFO:teuthology.orchestra.run.smithi139.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T09:50:15.598 INFO:teuthology.orchestra.run.smithi139.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.7 MB/s | 1.0 MB 00:00 2024-04-25T09:50:15.682 INFO:teuthology.orchestra.run.smithi139.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-25T09:50:15.774 INFO:teuthology.orchestra.run.smithi139.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-04-25T09:50:15.824 INFO:teuthology.orchestra.run.smithi139.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 516 kB/s | 26 kB 00:00 2024-04-25T09:50:15.849 INFO:teuthology.orchestra.run.smithi139.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-04-25T09:50:15.883 INFO:teuthology.orchestra.run.smithi139.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-04-25T09:50:15.925 INFO:teuthology.orchestra.run.smithi139.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T09:50:15.941 INFO:teuthology.orchestra.run.smithi139.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 595 kB/s | 35 kB 00:00 2024-04-25T09:50:16.025 INFO:teuthology.orchestra.run.smithi139.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-04-25T09:50:16.075 INFO:teuthology.orchestra.run.smithi139.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-04-25T09:50:16.117 INFO:teuthology.orchestra.run.smithi139.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-25T09:50:16.142 INFO:teuthology.orchestra.run.smithi139.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-25T09:50:16.193 INFO:teuthology.orchestra.run.smithi139.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-04-25T09:50:16.243 INFO:teuthology.orchestra.run.smithi139.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-25T09:50:16.285 INFO:teuthology.orchestra.run.smithi139.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-25T09:50:16.377 INFO:teuthology.orchestra.run.smithi139.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-04-25T09:50:16.402 INFO:teuthology.orchestra.run.smithi139.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 437 kB/s | 51 kB 00:00 2024-04-25T09:50:16.427 INFO:teuthology.orchestra.run.smithi139.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-04-25T09:50:16.452 INFO:teuthology.orchestra.run.smithi139.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 713 kB/s | 54 kB 00:00 2024-04-25T09:50:16.503 INFO:teuthology.orchestra.run.smithi139.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-04-25T09:50:16.545 INFO:teuthology.orchestra.run.smithi139.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 481 kB/s | 44 kB 00:00 2024-04-25T09:50:16.595 INFO:teuthology.orchestra.run.smithi139.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-04-25T09:50:16.620 INFO:teuthology.orchestra.run.smithi139.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 478 kB/s | 36 kB 00:00 2024-04-25T09:50:16.662 INFO:teuthology.orchestra.run.smithi139.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-04-25T09:50:16.779 INFO:teuthology.orchestra.run.smithi139.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.9 MB/s | 305 kB 00:00 2024-04-25T09:50:16.805 INFO:teuthology.orchestra.run.smithi139.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 407 kB/s | 58 kB 00:00 2024-04-25T09:50:16.855 INFO:teuthology.orchestra.run.smithi139.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-25T09:50:16.880 INFO:teuthology.orchestra.run.smithi139.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-04-25T09:50:16.947 INFO:teuthology.orchestra.run.smithi139.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 487 kB/s | 45 kB 00:00 2024-04-25T09:50:16.973 INFO:teuthology.orchestra.run.smithi139.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-04-25T09:50:17.231 INFO:teuthology.orchestra.run.smithi139.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-25T09:50:17.273 INFO:teuthology.orchestra.run.smithi139.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-04-25T09:50:17.410 INFO:teuthology.orchestra.run.smithi139.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-04-25T09:50:17.457 INFO:teuthology.orchestra.run.smithi139.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-04-25T09:50:17.507 INFO:teuthology.orchestra.run.smithi139.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-04-25T09:50:17.557 INFO:teuthology.orchestra.run.smithi139.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 975 kB/s | 49 kB 00:00 2024-04-25T09:50:19.108 INFO:teuthology.orchestra.run.smithi139.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 43 kB/s | 67 kB 00:01 2024-04-25T09:50:19.249 INFO:teuthology.orchestra.run.smithi139.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.4 MB/s | 4.4 MB 00:01 2024-04-25T09:50:19.274 INFO:teuthology.orchestra.run.smithi139.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 907 kB/s | 151 kB 00:00 2024-04-25T09:50:19.316 INFO:teuthology.orchestra.run.smithi139.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-04-25T09:50:19.858 INFO:teuthology.orchestra.run.smithi139.stdout:(113/152): abseil-cpp-20211102.0-3.el9.x86_64.r 190 kB/s | 548 kB 00:02 2024-04-25T09:50:19.883 INFO:teuthology.orchestra.run.smithi139.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 77 kB/s | 43 kB 00:00 2024-04-25T09:50:19.909 INFO:teuthology.orchestra.run.smithi139.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 645 kB/s | 32 kB 00:00 2024-04-25T09:50:19.943 INFO:teuthology.orchestra.run.smithi139.stdout:(116/152): python3-asyncssh-2.13.2-5.el9.noarch 821 kB/s | 548 kB 00:00 2024-04-25T09:50:19.968 INFO:teuthology.orchestra.run.smithi139.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-04-25T09:50:19.993 INFO:teuthology.orchestra.run.smithi139.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-25T09:50:20.026 INFO:teuthology.orchestra.run.smithi139.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-25T09:50:20.777 INFO:teuthology.orchestra.run.smithi139.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 375 kB/s | 303 kB 00:00 2024-04-25T09:50:20.801 INFO:teuthology.orchestra.run.smithi139.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 44 kB/s | 35 kB 00:00 2024-04-25T09:50:20.827 INFO:teuthology.orchestra.run.smithi139.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 257 kB/s | 206 kB 00:00 2024-04-25T09:50:21.227 INFO:teuthology.orchestra.run.smithi139.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 27 kB/s | 11 kB 00:00 2024-04-25T09:50:21.252 INFO:teuthology.orchestra.run.smithi139.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 321 kB/s | 144 kB 00:00 2024-04-25T09:50:21.278 INFO:teuthology.orchestra.run.smithi139.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-04-25T09:50:21.303 INFO:teuthology.orchestra.run.smithi139.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-04-25T09:50:21.328 INFO:teuthology.orchestra.run.smithi139.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-04-25T09:50:21.353 INFO:teuthology.orchestra.run.smithi139.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-04-25T09:50:21.404 INFO:teuthology.orchestra.run.smithi139.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-04-25T09:50:21.421 INFO:teuthology.orchestra.run.smithi139.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 692 kB/s | 46 kB 00:00 2024-04-25T09:50:21.463 INFO:teuthology.orchestra.run.smithi139.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-04-25T09:50:21.488 INFO:teuthology.orchestra.run.smithi139.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 867 kB/s | 58 kB 00:00 2024-04-25T09:50:21.505 INFO:teuthology.orchestra.run.smithi139.stdout:(133/152): python3-more-itertools-8.12.0-2.el9. 783 kB/s | 79 kB 00:00 2024-04-25T09:50:21.530 INFO:teuthology.orchestra.run.smithi139.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-25T09:50:21.548 INFO:teuthology.orchestra.run.smithi139.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 278 kB/s | 16 kB 00:00 2024-04-25T09:50:21.573 INFO:teuthology.orchestra.run.smithi139.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-25T09:50:21.590 INFO:teuthology.orchestra.run.smithi139.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 736 kB/s | 31 kB 00:00 2024-04-25T09:50:21.631 INFO:teuthology.orchestra.run.smithi139.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-04-25T09:50:21.673 INFO:teuthology.orchestra.run.smithi139.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 862 kB/s | 36 kB 00:00 2024-04-25T09:50:21.715 INFO:teuthology.orchestra.run.smithi139.stdout:(140/152): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-04-25T09:50:21.774 INFO:teuthology.orchestra.run.smithi139.stdout:(141/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-25T09:50:21.824 INFO:teuthology.orchestra.run.smithi139.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-25T09:50:22.016 INFO:teuthology.orchestra.run.smithi139.stdout:(143/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.2 MB/s | 427 kB 00:00 2024-04-25T09:50:22.058 INFO:teuthology.orchestra.run.smithi139.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 481 kB/s | 20 kB 00:00 2024-04-25T09:50:22.100 INFO:teuthology.orchestra.run.smithi139.stdout:(145/152): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-04-25T09:50:22.208 INFO:teuthology.orchestra.run.smithi139.stdout:(146/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T09:50:23.084 INFO:teuthology.orchestra.run.smithi139.stdout:(147/152): librados2-19.0.0-1628.g97ae6cc9.el9. 3.9 MB/s | 3.4 MB 00:00 2024-04-25T09:50:23.692 INFO:teuthology.orchestra.run.smithi139.stdout:(148/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 5.2 MB/s | 3.1 MB 00:00 2024-04-25T09:50:23.717 INFO:teuthology.orchestra.run.smithi139.stdout:(149/152): python3-pyperclip-1.8.2-2.el9.noarch 13 kB/s | 28 kB 00:02 2024-04-25T09:50:23.743 INFO:teuthology.orchestra.run.smithi139.stdout:(150/152): python3-routes-2.5.1-5.el9.noarch.rp 87 kB/s | 188 kB 00:02 2024-04-25T09:50:23.984 INFO:teuthology.orchestra.run.smithi139.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 4.6 MB/s | 1.3 MB 00:00 2024-04-25T09:50:24.068 INFO:teuthology.orchestra.run.smithi139.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 2.4 MB 00:00 2024-04-25T09:50:24.070 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:50:24.070 INFO:teuthology.orchestra.run.smithi139.stdout:Total 10 MB/s | 215 MB 00:20 2024-04-25T09:50:25.179 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-04-25T09:50:25.321 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-04-25T09:50:25.322 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-04-25T09:50:27.143 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-04-25T09:50:27.143 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-04-25T09:50:33.114 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-04-25T09:50:33.230 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T09:50:33.462 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T09:50:33.524 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T09:50:33.565 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T09:50:33.590 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T09:50:33.634 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T09:50:33.962 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T09:50:34.057 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:50:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:50:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:50:34.290 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:50:34.320 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:50:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T09:50:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T09:50:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:50:34.647 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:50:34.671 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:50:34.717 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:50:34.738 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:50:34.763 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:50:34.914 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:50:34.947 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:50:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T09:50:35.149 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T09:50:35.504 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T09:50:35.546 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T09:50:35.590 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T09:50:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T09:50:35.650 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T09:50:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T09:50:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T09:50:35.803 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T09:50:35.846 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T09:50:35.923 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T09:50:35.957 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T09:50:36.000 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T09:50:36.029 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T09:50:36.099 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T09:50:36.144 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T09:50:36.218 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T09:50:36.364 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T09:50:36.422 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T09:50:36.955 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T09:50:37.042 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T09:50:37.070 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T09:50:37.122 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T09:50:37.160 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T09:50:37.201 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T09:50:37.356 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T09:50:37.421 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T09:50:37.476 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T09:50:37.549 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T09:50:37.580 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T09:50:37.616 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T09:50:37.663 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T09:50:37.699 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T09:50:37.742 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T09:50:37.784 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T09:50:37.920 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T09:50:37.973 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T09:50:38.003 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T09:50:38.045 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T09:50:38.139 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T09:50:38.186 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T09:50:38.226 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T09:50:38.252 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T09:50:38.276 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T09:50:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T09:50:38.478 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T09:50:38.509 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T09:50:38.536 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T09:50:38.571 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T09:50:38.610 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T09:50:38.641 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T09:50:38.671 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T09:50:38.706 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T09:50:38.739 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T09:50:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T09:50:38.815 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T09:50:38.901 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T09:50:38.945 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T09:50:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T09:50:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T09:50:39.122 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T09:50:39.178 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T09:50:39.905 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T09:50:39.967 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T09:50:40.001 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T09:50:40.119 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T09:50:40.175 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T09:50:40.273 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T09:50:40.320 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T09:50:40.500 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T09:50:40.525 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T09:50:40.603 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T09:50:41.259 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T09:50:41.366 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T09:50:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T09:50:42.679 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T09:50:42.707 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T09:50:42.740 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T09:50:43.045 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T09:50:43.086 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:50:43.159 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:50:43.190 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T09:50:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T09:50:43.682 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T09:50:43.714 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:50:43.785 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:50:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T09:50:46.025 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:50:46.061 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:50:46.604 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:50:46.648 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:50:46.737 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:50:46.737 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T09:50:46.737 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T09:50:46.737 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:50:46.766 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:50:56.105 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 342 kB/s | 21 MB 01:02 2024-04-25T09:50:58.067 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:50:58.067 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /sys 2024-04-25T09:50:58.067 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /proc 2024-04-25T09:50:58.067 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /mnt 2024-04-25T09:50:58.067 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /var/tmp 2024-04-25T09:50:58.067 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /home 2024-04-25T09:50:58.067 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /root 2024-04-25T09:50:58.067 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /tmp 2024-04-25T09:50:58.067 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:50:58.234 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:50:58.250 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:50:58.250 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:50:58.250 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T09:50:58.250 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:50:59.737 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:50:59.764 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:50:59.764 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:50:59.764 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T09:50:59.764 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:50:59.764 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:50:59.764 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:50:59.971 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:50:59.994 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:50:59.994 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:50:59.994 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T09:50:59.994 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:50:59.994 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:50:59.994 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:51:00.430 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:51:00.453 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:51:00.453 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:00.453 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T09:51:00.453 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:51:00.453 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:51:00.453 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:51:00.560 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T09:51:00.608 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T09:51:00.664 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T09:51:00.849 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T09:51:00.897 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T09:51:00.976 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T09:51:01.063 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:51:01.203 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:51:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T09:51:01.263 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-25T09:51:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T09:51:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T09:51:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:51:01.738 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:51:01.769 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T09:51:01.830 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T09:51:01.861 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:51:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:51:02.120 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:51:03.080 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:51:03.112 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:51:03.214 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:51:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T09:51:03.389 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:51:03.390 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:51:03.400 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T09:51:03.401 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T09:51:03.402 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T09:51:03.402 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T09:51:03.404 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T09:51:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:51:03.406 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T09:51:03.407 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T09:51:03.429 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:51:03.429 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:03.429 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T09:51:03.429 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:51:03.429 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:51:03.429 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:51:03.467 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-25T09:51:03.473 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:51:03.476 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-25T09:51:03.476 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-25T09:51:03.476 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T09:51:03.477 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T09:51:03.478 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-04-25T09:51:03.479 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T09:51:03.480 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T09:51:03.481 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T09:51:03.482 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T09:51:03.482 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T09:51:03.482 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T09:51:03.482 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T09:51:03.482 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T09:51:03.482 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T09:51:03.482 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T09:51:03.482 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T09:51:03.482 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T09:51:03.483 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T09:51:03.484 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T09:51:03.485 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T09:51:03.486 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T09:51:03.487 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout:Install 148 Packages 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 4 Packages 2024-04-25T09:51:03.488 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:51:03.489 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:51:03.489 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 215 M 2024-04-25T09:51:03.489 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-25T09:51:03.581 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T09:51:03.630 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T09:51:04.331 INFO:teuthology.orchestra.run.smithi150.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 35 kB/s | 6.6 kB 00:00 2024-04-25T09:51:04.480 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T09:51:04.509 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:51:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:51:04.532 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:04.532 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T09:51:04.532 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:51:04.532 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:51:04.532 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:51:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:51:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:51:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:04.599 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T09:51:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:51:04.606 INFO:teuthology.orchestra.run.smithi150.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 3.2 MB/s | 887 kB 00:00 2024-04-25T09:51:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:51:04.666 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:51:04.666 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:04.666 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T09:51:04.666 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:51:04.667 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:51:04.667 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:51:04.681 INFO:teuthology.orchestra.run.smithi150.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 1.9 MB/s | 145 kB 00:00 2024-04-25T09:51:04.969 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:51:04.992 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:51:04.992 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:04.992 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T09:51:04.992 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:51:04.992 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:51:04.992 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:51:05.048 INFO:teuthology.orchestra.run.smithi150.stdout:(4/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 5.9 MB/s | 5.3 MB 00:00 2024-04-25T09:51:05.107 INFO:teuthology.orchestra.run.smithi150.stdout:(5/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 5.0 MB/s | 2.1 MB 00:00 2024-04-25T09:51:05.274 INFO:teuthology.orchestra.run.smithi150.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 6.6 MB/s | 1.5 MB 00:00 2024-04-25T09:51:06.124 INFO:teuthology.orchestra.run.smithi150.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 4.6 MB/s | 4.7 MB 00:01 2024-04-25T09:51:07.057 INFO:teuthology.orchestra.run.smithi150.stdout:(8/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 6.8 MB/s | 20 MB 00:02 2024-04-25T09:51:07.116 INFO:teuthology.orchestra.run.smithi150.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 433 kB/s | 25 kB 00:00 2024-04-25T09:51:07.275 INFO:teuthology.orchestra.run.smithi150.stdout:(10/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 8.4 MB/s | 17 MB 00:02 2024-04-25T09:51:07.341 INFO:teuthology.orchestra.run.smithi150.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 3.2 MB/s | 220 kB 00:00 2024-04-25T09:51:07.408 INFO:teuthology.orchestra.run.smithi150.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 405 kB/s | 27 kB 00:00 2024-04-25T09:51:07.467 INFO:teuthology.orchestra.run.smithi150.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 535 kB/s | 31 kB 00:00 2024-04-25T09:51:07.592 INFO:teuthology.orchestra.run.smithi150.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 5.8 MB/s | 744 kB 00:00 2024-04-25T09:51:07.667 INFO:teuthology.orchestra.run.smithi150.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 2.1 MB/s | 163 kB 00:00 2024-04-25T09:51:07.735 INFO:teuthology.orchestra.run.smithi150.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.9 MB/s | 127 kB 00:00 2024-04-25T09:51:07.810 INFO:teuthology.orchestra.run.smithi150.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 6.5 MB/s | 501 kB 00:00 2024-04-25T09:51:07.935 INFO:teuthology.orchestra.run.smithi150.stdout:(18/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 5.4 MB/s | 9.8 MB 00:01 2024-04-25T09:51:08.010 INFO:teuthology.orchestra.run.smithi150.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 604 kB/s | 45 kB 00:00 2024-04-25T09:51:08.077 INFO:teuthology.orchestra.run.smithi150.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.9 MB/s | 131 kB 00:00 2024-04-25T09:51:08.144 INFO:teuthology.orchestra.run.smithi150.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 2.4 MB/s | 162 kB 00:00 2024-04-25T09:51:08.236 INFO:teuthology.orchestra.run.smithi150.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 3.4 MB/s | 321 kB 00:00 2024-04-25T09:51:08.312 INFO:teuthology.orchestra.run.smithi150.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 3.9 MB/s | 298 kB 00:00 2024-04-25T09:51:08.387 INFO:teuthology.orchestra.run.smithi150.stdout:(24/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 8.9 MB/s | 5.1 MB 00:00 2024-04-25T09:51:08.404 INFO:teuthology.orchestra.run.smithi150.stdout:(25/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.0 MB/s | 99 kB 00:00 2024-04-25T09:51:08.446 INFO:teuthology.orchestra.run.smithi150.stdout:(26/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-25T09:51:08.530 INFO:teuthology.orchestra.run.smithi150.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-25T09:51:08.597 INFO:teuthology.orchestra.run.smithi150.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 471 kB/s | 31 kB 00:00 2024-04-25T09:51:08.680 INFO:teuthology.orchestra.run.smithi150.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.6 MB/s | 138 kB 00:00 2024-04-25T09:51:08.822 INFO:teuthology.orchestra.run.smithi150.stdout:(30/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 7.4 MB/s | 3.1 MB 00:00 2024-04-25T09:51:08.939 INFO:teuthology.orchestra.run.smithi150.stdout:(31/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 6.9 MB/s | 1.8 MB 00:00 2024-04-25T09:51:09.023 INFO:teuthology.orchestra.run.smithi150.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 2.9 MB/s | 248 kB 00:00 2024-04-25T09:51:09.082 INFO:teuthology.orchestra.run.smithi150.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 845 kB/s | 49 kB 00:00 2024-04-25T09:51:09.140 INFO:teuthology.orchestra.run.smithi150.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 262 kB/s | 15 kB 00:00 2024-04-25T09:51:09.216 INFO:teuthology.orchestra.run.smithi150.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 3.4 MB/s | 264 kB 00:00 2024-04-25T09:51:09.366 INFO:teuthology.orchestra.run.smithi150.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 4.9 MB/s | 748 kB 00:00 2024-04-25T09:51:09.432 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T09:51:09.466 INFO:teuthology.orchestra.run.smithi150.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 182 kB/s | 18 kB 00:00 2024-04-25T09:51:09.516 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T09:51:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T09:51:09.586 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T09:51:09.614 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T09:51:09.675 INFO:teuthology.orchestra.run.smithi150.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 8.7 MB/s | 7.4 MB 00:00 2024-04-25T09:51:09.700 INFO:teuthology.orchestra.run.smithi150.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 344 kB/s | 80 kB 00:00 2024-04-25T09:51:09.802 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T09:51:09.843 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T09:51:09.850 INFO:teuthology.orchestra.run.smithi150.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-25T09:51:09.864 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T09:51:10.017 INFO:teuthology.orchestra.run.smithi150.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-04-25T09:51:10.459 INFO:teuthology.orchestra.run.smithi150.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-04-25T09:51:10.470 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:51:10.484 INFO:teuthology.orchestra.run.smithi150.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 992 kB/s | 803 kB 00:00 2024-04-25T09:51:10.520 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:51:10.534 INFO:teuthology.orchestra.run.smithi150.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-25T09:51:10.541 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:51:10.585 INFO:teuthology.orchestra.run.smithi150.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-25T09:51:10.610 INFO:teuthology.orchestra.run.smithi150.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-25T09:51:10.752 INFO:teuthology.orchestra.run.smithi150.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-04-25T09:51:10.936 INFO:teuthology.orchestra.run.smithi150.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-04-25T09:51:10.978 INFO:teuthology.orchestra.run.smithi150.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-25T09:51:11.028 INFO:teuthology.orchestra.run.smithi150.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-25T09:51:11.376 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T09:51:11.449 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T09:51:11.453 INFO:teuthology.orchestra.run.smithi150.stdout:(51/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 11 MB/s | 47 MB 00:04 2024-04-25T09:51:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T09:51:11.495 INFO:teuthology.orchestra.run.smithi150.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-04-25T09:51:11.521 INFO:teuthology.orchestra.run.smithi150.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 637 kB/s | 313 kB 00:00 2024-04-25T09:51:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T09:51:11.579 INFO:teuthology.orchestra.run.smithi150.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-04-25T09:51:11.605 INFO:teuthology.orchestra.run.smithi150.stdout:(55/152): flexiblas-3.0.4-8.el9.x86_64.rpm 217 kB/s | 33 kB 00:00 2024-04-25T09:51:11.611 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T09:51:11.611 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:51:11.647 INFO:teuthology.orchestra.run.smithi150.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-04-25T09:51:11.696 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:51:11.728 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:51:11.728 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:51:11.730 INFO:teuthology.orchestra.run.smithi150.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 545 kB/s | 45 kB 00:00 2024-04-25T09:51:11.755 INFO:teuthology.orchestra.run.smithi150.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-04-25T09:51:11.816 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:51:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:51:11.997 INFO:teuthology.orchestra.run.smithi150.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-04-25T09:51:12.031 INFO:teuthology.orchestra.run.smithi150.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 2.1 MB/s | 582 kB 00:00 2024-04-25T09:51:12.056 INFO:teuthology.orchestra.run.smithi150.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-25T09:51:12.090 INFO:teuthology.orchestra.run.smithi150.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-04-25T09:51:12.173 INFO:teuthology.orchestra.run.smithi150.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-25T09:51:12.207 INFO:teuthology.orchestra.run.smithi150.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-04-25T09:51:12.232 INFO:teuthology.orchestra.run.smithi150.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 602 kB/s | 35 kB 00:00 2024-04-25T09:51:12.283 INFO:teuthology.orchestra.run.smithi150.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-04-25T09:51:12.616 INFO:teuthology.orchestra.run.smithi150.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 4.8 MB/s | 1.6 MB 00:00 2024-04-25T09:51:12.778 INFO:teuthology.orchestra.run.smithi150.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-04-25T09:51:12.833 INFO:teuthology.orchestra.run.smithi150.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-25T09:51:13.092 INFO:teuthology.orchestra.run.smithi150.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-04-25T09:51:13.150 INFO:teuthology.orchestra.run.smithi150.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.0 MB/s | 4.7 MB 00:00 2024-04-25T09:51:13.201 INFO:teuthology.orchestra.run.smithi150.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-25T09:51:13.234 INFO:teuthology.orchestra.run.smithi150.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-04-25T09:51:13.285 INFO:teuthology.orchestra.run.smithi150.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-25T09:51:13.360 INFO:teuthology.orchestra.run.smithi150.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-25T09:51:13.385 INFO:teuthology.orchestra.run.smithi150.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-25T09:51:13.410 INFO:teuthology.orchestra.run.smithi150.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-25T09:51:13.435 INFO:teuthology.orchestra.run.smithi150.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-25T09:51:13.452 INFO:teuthology.orchestra.run.smithi150.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T09:51:13.478 INFO:teuthology.orchestra.run.smithi150.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 827 kB/s | 35 kB 00:00 2024-04-25T09:51:13.528 INFO:teuthology.orchestra.run.smithi150.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-25T09:51:13.578 INFO:teuthology.orchestra.run.smithi150.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T09:51:13.629 INFO:teuthology.orchestra.run.smithi150.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T09:51:13.679 INFO:teuthology.orchestra.run.smithi150.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-04-25T09:51:13.755 INFO:teuthology.orchestra.run.smithi150.stdout:(85/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-25T09:51:13.780 INFO:teuthology.orchestra.run.smithi150.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-25T09:51:13.855 INFO:teuthology.orchestra.run.smithi150.stdout:(87/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-25T09:51:13.905 INFO:teuthology.orchestra.run.smithi150.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-25T09:51:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 681 kB/s | 51 kB 00:00 2024-04-25T09:51:13.956 INFO:teuthology.orchestra.run.smithi150.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-04-25T09:51:13.981 INFO:teuthology.orchestra.run.smithi150.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-04-25T09:51:14.023 INFO:teuthology.orchestra.run.smithi150.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-04-25T09:51:14.048 INFO:teuthology.orchestra.run.smithi150.stdout:(93/152): python3-toml-0.10.2-6.el9.noarch.rpm 625 kB/s | 42 kB 00:00 2024-04-25T09:51:14.073 INFO:teuthology.orchestra.run.smithi150.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T09:51:14.098 INFO:teuthology.orchestra.run.smithi150.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-04-25T09:51:14.124 INFO:teuthology.orchestra.run.smithi150.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-04-25T09:51:14.149 INFO:teuthology.orchestra.run.smithi150.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-25T09:51:14.174 INFO:teuthology.orchestra.run.smithi150.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T09:51:14.200 INFO:teuthology.orchestra.run.smithi150.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T09:51:14.225 INFO:teuthology.orchestra.run.smithi150.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T09:51:14.250 INFO:teuthology.orchestra.run.smithi150.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-04-25T09:51:14.309 INFO:teuthology.orchestra.run.smithi150.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-04-25T09:51:14.359 INFO:teuthology.orchestra.run.smithi150.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-04-25T09:51:14.635 INFO:teuthology.orchestra.run.smithi150.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-25T09:51:14.735 INFO:teuthology.orchestra.run.smithi150.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-04-25T09:51:15.010 INFO:teuthology.orchestra.run.smithi150.stdout:(106/152): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T09:51:15.060 INFO:teuthology.orchestra.run.smithi150.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-04-25T09:51:15.110 INFO:teuthology.orchestra.run.smithi150.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 973 kB/s | 49 kB 00:00 2024-04-25T09:51:15.152 INFO:teuthology.orchestra.run.smithi150.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-25T09:51:15.194 INFO:teuthology.orchestra.run.smithi150.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-04-25T09:51:15.245 INFO:teuthology.orchestra.run.smithi150.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 16 MB/s | 838 kB 00:00 2024-04-25T09:51:15.297 INFO:teuthology.orchestra.run.smithi150.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 10 MB/s | 548 kB 00:00 2024-04-25T09:51:15.348 INFO:teuthology.orchestra.run.smithi150.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 870 kB/s | 43 kB 00:00 2024-04-25T09:51:15.390 INFO:teuthology.orchestra.run.smithi150.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 775 kB/s | 32 kB 00:00 2024-04-25T09:51:15.431 INFO:teuthology.orchestra.run.smithi150.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 360 kB/s | 15 kB 00:00 2024-04-25T09:51:15.481 INFO:teuthology.orchestra.run.smithi150.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-25T09:51:15.532 INFO:teuthology.orchestra.run.smithi150.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-25T09:51:15.582 INFO:teuthology.orchestra.run.smithi150.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-04-25T09:51:15.624 INFO:teuthology.orchestra.run.smithi150.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 848 kB/s | 35 kB 00:00 2024-04-25T09:51:15.674 INFO:teuthology.orchestra.run.smithi150.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-25T09:51:15.749 INFO:teuthology.orchestra.run.smithi150.stdout:(121/152): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2024-04-25T09:51:15.791 INFO:teuthology.orchestra.run.smithi150.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-04-25T09:51:15.842 INFO:teuthology.orchestra.run.smithi150.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-04-25T09:51:15.892 INFO:teuthology.orchestra.run.smithi150.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-04-25T09:51:15.942 INFO:teuthology.orchestra.run.smithi150.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-04-25T09:51:15.993 INFO:teuthology.orchestra.run.smithi150.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 390 kB/s | 19 kB 00:00 2024-04-25T09:51:16.043 INFO:teuthology.orchestra.run.smithi150.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-04-25T09:51:16.101 INFO:teuthology.orchestra.run.smithi150.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-04-25T09:51:16.126 INFO:teuthology.orchestra.run.smithi150.stdout:(129/152): abseil-cpp-20211102.0-3.el9.x86_64.r 302 kB/s | 548 kB 00:01 2024-04-25T09:51:16.152 INFO:teuthology.orchestra.run.smithi150.stdout:(130/152): gperftools-libs-2.9.1-2.el9.x86_64.r 172 kB/s | 309 kB 00:01 2024-04-25T09:51:16.177 INFO:teuthology.orchestra.run.smithi150.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 612 kB/s | 46 kB 00:00 2024-04-25T09:51:16.202 INFO:teuthology.orchestra.run.smithi150.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-25T09:51:16.228 INFO:teuthology.orchestra.run.smithi150.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-04-25T09:51:16.253 INFO:teuthology.orchestra.run.smithi150.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-25T09:51:16.278 INFO:teuthology.orchestra.run.smithi150.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-25T09:51:16.303 INFO:teuthology.orchestra.run.smithi150.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-04-25T09:51:16.329 INFO:teuthology.orchestra.run.smithi150.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-04-25T09:51:16.354 INFO:teuthology.orchestra.run.smithi150.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-25T09:51:16.379 INFO:teuthology.orchestra.run.smithi150.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-25T09:51:16.404 INFO:teuthology.orchestra.run.smithi150.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-04-25T09:51:16.431 INFO:teuthology.orchestra.run.smithi150.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-04-25T09:51:16.455 INFO:teuthology.orchestra.run.smithi150.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-04-25T09:51:16.489 INFO:teuthology.orchestra.run.smithi150.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-25T09:51:16.539 INFO:teuthology.orchestra.run.smithi150.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 401 kB/s | 20 kB 00:00 2024-04-25T09:51:16.589 INFO:teuthology.orchestra.run.smithi150.stdout:(145/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T09:51:16.659 INFO:teuthology.orchestra.run.smithi150.stdout:(146/152): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-04-25T09:51:17.023 INFO:teuthology.orchestra.run.smithi150.stdout:(147/152): librados2-19.0.0-1628.g97ae6cc9.el9. 9.2 MB/s | 3.4 MB 00:00 2024-04-25T09:51:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:51:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T09:51:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T09:51:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T09:51:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T09:51:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T09:51:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T09:51:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T09:51:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:51:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:51:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:51:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T09:51:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T09:51:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:51:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T09:51:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T09:51:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T09:51:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T09:51:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T09:51:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T09:51:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T09:51:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T09:51:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T09:51:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T09:51:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T09:51:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T09:51:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T09:51:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T09:51:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T09:51:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T09:51:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T09:51:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T09:51:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T09:51:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T09:51:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T09:51:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T09:51:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T09:51:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T09:51:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T09:51:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T09:51:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:51:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:51:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T09:51:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T09:51:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T09:51:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T09:51:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T09:51:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T09:51:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T09:51:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T09:51:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T09:51:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T09:51:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T09:51:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T09:51:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T09:51:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T09:51:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T09:51:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T09:51:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T09:51:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T09:51:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T09:51:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T09:51:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:51:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T09:51:17.182 INFO:teuthology.orchestra.run.smithi150.stdout:(148/152): python3-webob-1.8.7-6.el9.noarch.rpm 295 kB/s | 230 kB 00:00 2024-04-25T09:51:17.207 INFO:teuthology.orchestra.run.smithi150.stdout:(149/152): python3-werkzeug-2.0.3-3.el9.1.noarc 568 kB/s | 427 kB 00:00 2024-04-25T09:51:17.475 INFO:teuthology.orchestra.run.smithi150.stdout:(150/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-04-25T09:51:17.607 INFO:teuthology.orchestra.run.smithi150.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 3.2 MB/s | 1.3 MB 00:00 2024-04-25T09:51:18.033 INFO:teuthology.orchestra.run.smithi150.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.9 MB/s | 2.4 MB 00:00 2024-04-25T09:51:18.035 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:51:18.035 INFO:teuthology.orchestra.run.smithi150.stdout:Total 15 MB/s | 215 MB 00:14 2024-04-25T09:51:19.095 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-25T09:51:19.240 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-25T09:51:19.240 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-25T09:51:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:51:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:51:19.250 INFO:teuthology.orchestra.run.smithi139.stdout:Upgraded: 2024-04-25T09:51:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T09:51:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T09:51:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T09:51:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T09:51:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T09:51:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T09:51:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T09:51:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T09:51:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T09:51:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:51:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T09:51:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T09:51:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T09:51:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T09:51:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T09:51:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T09:51:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T09:51:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:51:19.262 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-04-25T09:51:19.639 DEBUG:teuthology.parallel:result is None 2024-04-25T09:51:21.050 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-25T09:51:21.050 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-25T09:51:27.013 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-25T09:51:27.111 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T09:51:27.150 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T09:51:27.212 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T09:51:27.245 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T09:51:27.270 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T09:51:27.313 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T09:51:27.644 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T09:51:27.763 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:51:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:51:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:51:28.040 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:51:28.077 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:51:28.226 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T09:51:28.274 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T09:51:28.302 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:51:28.398 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:51:28.419 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:51:28.465 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:51:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:51:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:51:28.662 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:51:28.695 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:51:28.803 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T09:51:28.898 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T09:51:28.928 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T09:51:28.962 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T09:51:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T09:51:29.078 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T09:51:29.121 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T09:51:29.154 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T09:51:29.182 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T09:51:29.251 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T09:51:29.304 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T09:51:29.335 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T09:51:29.370 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T09:51:29.446 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T09:51:29.481 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T09:51:29.524 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T09:51:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T09:51:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T09:51:29.667 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T09:51:29.742 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T09:51:29.874 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T09:51:29.928 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T09:51:30.455 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T09:51:30.532 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T09:51:30.561 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T09:51:30.603 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T09:51:30.641 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T09:51:30.683 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T09:51:30.832 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T09:51:30.886 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T09:51:30.941 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T09:51:31.014 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T09:51:31.045 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T09:51:31.082 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T09:51:31.128 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T09:51:31.164 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T09:51:31.207 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T09:51:31.249 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T09:51:31.387 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T09:51:31.430 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T09:51:31.460 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T09:51:31.502 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T09:51:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T09:51:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T09:51:31.675 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T09:51:31.700 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T09:51:31.725 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T09:51:31.800 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T09:51:31.927 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T09:51:31.965 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T09:51:31.993 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T09:51:32.028 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T09:51:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T09:51:32.098 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T09:51:32.128 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T09:51:32.179 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T09:51:32.212 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T09:51:32.254 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T09:51:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T09:51:32.375 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T09:51:32.427 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T09:51:32.458 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T09:51:32.505 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T09:51:32.604 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T09:51:32.651 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T09:51:33.379 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T09:51:33.424 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T09:51:33.457 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T09:51:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T09:51:33.615 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T09:51:33.702 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T09:51:33.734 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T09:51:33.906 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T09:51:33.932 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T09:51:34.010 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T09:51:34.659 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T09:51:34.780 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T09:51:36.001 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T09:51:36.086 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T09:51:36.114 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T09:51:36.147 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T09:51:36.462 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T09:51:36.510 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:51:36.601 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:51:36.631 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T09:51:36.667 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T09:51:37.126 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T09:51:37.154 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:51:37.209 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:51:37.231 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T09:51:39.394 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:51:39.428 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:51:39.970 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:51:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:51:40.081 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:51:40.082 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T09:51:40.082 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T09:51:40.082 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:51:40.116 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:51:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:51:51.644 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-04-25T09:51:51.644 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-04-25T09:51:51.644 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-04-25T09:51:51.644 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-04-25T09:51:51.644 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-04-25T09:51:51.644 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-04-25T09:51:51.645 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-04-25T09:51:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:51:51.815 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:51:51.832 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:51:51.832 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:51.833 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T09:51:51.833 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:51:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:51:53.374 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:51:53.374 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:53.374 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T09:51:53.374 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:51:53.374 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:51:53.374 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:51:53.582 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:51:53.604 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:51:53.604 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:53.604 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T09:51:53.604 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:51:53.604 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:51:53.605 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:51:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:51:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:51:54.072 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:54.072 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T09:51:54.072 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:51:54.072 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:51:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:51:54.180 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T09:51:54.237 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T09:51:54.292 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T09:51:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T09:51:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T09:51:54.605 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T09:51:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:51:54.832 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:51:54.876 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T09:51:54.907 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T09:51:54.935 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T09:51:55.370 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:51:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:51:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T09:51:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T09:51:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:51:55.761 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:51:55.799 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:51:56.776 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:51:56.808 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:51:56.909 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:51:57.073 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T09:51:57.110 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:51:57.133 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:51:57.134 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:57.134 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T09:51:57.134 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:51:57.134 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:51:57.134 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:51:57.178 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:51:57.188 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:51:57.287 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T09:51:57.334 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T09:51:58.206 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T09:51:58.239 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:51:58.261 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:51:58.261 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:58.261 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T09:51:58.261 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:51:58.261 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:51:58.262 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:51:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:51:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:51:58.337 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:58.337 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T09:51:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:51:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:51:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:51:58.412 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:58.412 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T09:51:58.413 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:51:58.413 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:51:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:51:58.724 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:51:58.747 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:51:58.747 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:51:58.747 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T09:51:58.747 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:51:58.747 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:51:58.747 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:03.236 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T09:52:03.277 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T09:52:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T09:52:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T09:52:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T09:52:03.580 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T09:52:03.623 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T09:52:03.653 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T09:52:04.251 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:52:04.293 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:52:04.312 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:52:05.160 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T09:52:05.256 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T09:52:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T09:52:05.378 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T09:52:05.418 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T09:52:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:52:05.502 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:52:05.527 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:52:05.527 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:52:05.620 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:52:05.693 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:52:10.868 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:52:10.868 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T09:52:10.868 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T09:52:10.869 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T09:52:10.869 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T09:52:10.869 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T09:52:10.869 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T09:52:10.869 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T09:52:10.869 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:52:10.869 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:52:10.869 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:52:10.869 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T09:52:10.869 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T09:52:10.869 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:52:10.870 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T09:52:10.870 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T09:52:10.870 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T09:52:10.870 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T09:52:10.870 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T09:52:10.870 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T09:52:10.870 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T09:52:10.870 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T09:52:10.870 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T09:52:10.871 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T09:52:10.871 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T09:52:10.871 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T09:52:10.871 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T09:52:10.871 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T09:52:10.871 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T09:52:10.871 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T09:52:10.871 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T09:52:10.884 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T09:52:10.884 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T09:52:10.884 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T09:52:10.885 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T09:52:10.885 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T09:52:10.885 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T09:52:10.885 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T09:52:10.885 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T09:52:10.885 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T09:52:10.885 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T09:52:10.885 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T09:52:10.885 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T09:52:10.885 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T09:52:10.885 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T09:52:10.886 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T09:52:10.886 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T09:52:10.886 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T09:52:10.886 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T09:52:10.886 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T09:52:10.886 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T09:52:10.886 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T09:52:10.886 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T09:52:10.886 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T09:52:10.886 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T09:52:10.887 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T09:52:10.887 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T09:52:10.887 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T09:52:10.887 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T09:52:10.887 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T09:52:10.887 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T09:52:10.887 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T09:52:10.887 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T09:52:10.887 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T09:52:10.887 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T09:52:10.887 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T09:52:10.887 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T09:52:10.888 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T09:52:10.889 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T09:52:10.889 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T09:52:10.889 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T09:52:10.889 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T09:52:10.889 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T09:52:10.889 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T09:52:10.889 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T09:52:10.889 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T09:52:10.889 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T09:52:10.889 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T09:52:10.889 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T09:52:10.890 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T09:52:10.890 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T09:52:10.890 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T09:52:10.890 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T09:52:10.890 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T09:52:10.890 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T09:52:10.890 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T09:52:10.890 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T09:52:10.890 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T09:52:10.890 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T09:52:10.890 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T09:52:10.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T09:52:10.892 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T09:52:10.893 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T09:52:10.894 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T09:52:10.894 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T09:52:10.894 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T09:52:10.894 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T09:52:10.894 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T09:52:10.894 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T09:52:10.894 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T09:52:10.894 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T09:52:10.894 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T09:52:10.894 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T09:52:10.894 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T09:52:10.894 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T09:52:10.895 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T09:52:10.895 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:52:10.895 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T09:52:13.023 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:52:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:13.024 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-04-25T09:52:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T09:52:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T09:52:13.024 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-25T09:52:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T09:52:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T09:52:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T09:52:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T09:52:13.024 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:13.025 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T09:52:13.026 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T09:52:13.027 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T09:52:13.028 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T09:52:13.029 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T09:52:13.030 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T09:52:13.032 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T09:52:13.033 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T09:52:13.034 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:52:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T09:52:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T09:52:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T09:52:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T09:52:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T09:52:13.035 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T09:52:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T09:52:13.036 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:13.036 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-25T09:52:13.410 DEBUG:teuthology.parallel:result is None 2024-04-25T09:52:13.412 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:52:13.556 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T09:52:13.582 INFO:teuthology.orchestra.run.smithi139.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:52:13.583 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:52:13.583 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T09:52:13.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:52:13.730 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T09:52:13.751 INFO:teuthology.orchestra.run.smithi150.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:52:13.751 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:52:13.751 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T09:52:13.753 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T09:52:13.753 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:13.753 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T09:52:13.784 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:13.784 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T09:52:13.817 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T09:52:13.817 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:13.818 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T09:52:13.851 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T09:52:13.917 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:13.917 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T09:52:13.941 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T09:52:14.004 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T09:52:14.005 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:14.005 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T09:52:14.030 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T09:52:14.095 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:14.095 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T09:52:14.119 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T09:52:14.182 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T09:52:14.182 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:14.182 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T09:52:14.205 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T09:52:14.270 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:14.270 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T09:52:14.294 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T09:52:14.357 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T09:52:14.446 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T09:52:14.446 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /var/log/ceph 2024-04-25T09:52:14.449 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /var/log/ceph 2024-04-25T09:52:14.474 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T09:52:14.474 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T09:52:14.517 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T09:52:14.541 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T09:52:14.541 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T09:52:14.581 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T09:52:14.612 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T09:52:14.612 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T09:52:14.612 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T09:52:14.612 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:14.613 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T09:52:14.658 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T09:52:14.767 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T09:52:14.805 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:14.805 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T09:52:14.821 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T09:52:14.930 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T09:52:14.973 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T09:52:14.974 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T09:52:14.974 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673032', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1124, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi139.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHluDo6QtwvzcKbpG/7dyEJny733MCqqFUZqaO9C1fWAsFyzJ8U8bOVwcgDfWprkYOScQd2DLcXurCjmRVmyFVI=', 'smithi150.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGKr7J+T0OU30IhNcwztV/zctpDacBJ35Xj7sgY6Ohcl6SDNmEAtm1BkaSjGZsElp2DlMS6CfRqsCYwADzD+aqI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_09:36:50', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T09:52:14.974 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T09:52:14.977 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T09:52:15.023 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T09:52:15.031 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T09:52:15.087 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:15.087 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T09:52:15.108 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T09:52:15.108 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_1 2024-04-25T09:52:15.163 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T09:52:15.163 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:15.163 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T09:52:15.163 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:15.163 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:15.163 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-04-25 09:51:12.259031302 +0000 2024-04-25T09:52:15.163 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-25 09:49:21.399904356 +0000 2024-04-25T09:52:15.163 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-25 09:49:21.399904356 +0000 2024-04-25T09:52:15.163 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-25 09:49:21.399904356 +0000 2024-04-25T09:52:15.163 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T09:52:15.225 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-04-25T09:52:15.225 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-04-25T09:52:15.225 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.00019583 s, 2.6 MB/s 2024-04-25T09:52:15.226 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T09:52:15.282 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_2 2024-04-25T09:52:15.338 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T09:52:15.338 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:15.338 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T09:52:15.338 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:15.338 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:15.338 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-04-25 09:51:12.259031302 +0000 2024-04-25T09:52:15.338 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-25 09:49:21.758895490 +0000 2024-04-25T09:52:15.338 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-25 09:49:21.758895490 +0000 2024-04-25T09:52:15.338 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-25 09:49:21.758895490 +0000 2024-04-25T09:52:15.339 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T09:52:15.400 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-04-25T09:52:15.400 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-04-25T09:52:15.400 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000186508 s, 2.7 MB/s 2024-04-25T09:52:15.401 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T09:52:15.457 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_3 2024-04-25T09:52:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T09:52:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:15.512 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T09:52:15.512 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:15.512 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:15.512 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-04-25 09:51:12.260031280 +0000 2024-04-25T09:52:15.512 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-25 09:49:22.122886500 +0000 2024-04-25T09:52:15.512 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-25 09:49:22.122886500 +0000 2024-04-25T09:52:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-25 09:49:22.122886500 +0000 2024-04-25T09:52:15.512 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T09:52:15.574 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-04-25T09:52:15.574 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-04-25T09:52:15.574 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000193368 s, 2.6 MB/s 2024-04-25T09:52:15.576 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T09:52:15.632 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_4 2024-04-25T09:52:15.687 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T09:52:15.687 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:15.687 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T09:52:15.688 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:15.688 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:15.688 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-04-25 09:51:12.260031280 +0000 2024-04-25T09:52:15.688 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-25 09:49:22.455878276 +0000 2024-04-25T09:52:15.688 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-25 09:49:22.455878276 +0000 2024-04-25T09:52:15.688 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-25 09:49:22.455878276 +0000 2024-04-25T09:52:15.688 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T09:52:15.750 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-04-25T09:52:15.750 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-04-25T09:52:15.750 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000235279 s, 2.2 MB/s 2024-04-25T09:52:15.751 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T09:52:15.807 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:15.807 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:15.807 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T09:52:15.822 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T09:52:15.823 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-04-25T09:52:15.879 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T09:52:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:15.880 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T09:52:15.880 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:15.880 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:15.880 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-25 09:52:06.088822060 +0000 2024-04-25T09:52:15.880 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-25 09:49:21.405944458 +0000 2024-04-25T09:52:15.880 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-25 09:49:21.405944458 +0000 2024-04-25T09:52:15.880 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-25 09:49:21.405944458 +0000 2024-04-25T09:52:15.881 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T09:52:15.942 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-25T09:52:15.942 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-25T09:52:15.942 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000214536 s, 2.4 MB/s 2024-04-25T09:52:15.944 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T09:52:16.000 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-04-25T09:52:16.055 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T09:52:16.055 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:16.055 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T09:52:16.055 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:16.056 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:16.056 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-25 09:52:06.088822060 +0000 2024-04-25T09:52:16.056 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-25 09:49:21.741937414 +0000 2024-04-25T09:52:16.056 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-25 09:49:21.741937414 +0000 2024-04-25T09:52:16.056 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-25 09:49:21.741937414 +0000 2024-04-25T09:52:16.056 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T09:52:16.117 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-25T09:52:16.117 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-25T09:52:16.118 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000218481 s, 2.3 MB/s 2024-04-25T09:52:16.119 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T09:52:16.174 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-04-25T09:52:16.229 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T09:52:16.229 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:16.229 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T09:52:16.229 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:16.229 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:16.229 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-25 09:52:06.088822060 +0000 2024-04-25T09:52:16.229 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-25 09:49:22.082930264 +0000 2024-04-25T09:52:16.230 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-25 09:49:22.082930264 +0000 2024-04-25T09:52:16.230 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-25 09:49:22.082930264 +0000 2024-04-25T09:52:16.230 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T09:52:16.295 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-25T09:52:16.295 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-25T09:52:16.295 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000219505 s, 2.3 MB/s 2024-04-25T09:52:16.295 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T09:52:16.351 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-04-25T09:52:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T09:52:16.406 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:52:16.406 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T09:52:16.406 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:52:16.406 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:52:16.406 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-25 09:52:06.089822041 +0000 2024-04-25T09:52:16.406 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-25 09:49:22.423923115 +0000 2024-04-25T09:52:16.407 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-25 09:49:22.423923115 +0000 2024-04-25T09:52:16.407 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-25 09:49:22.423923115 +0000 2024-04-25T09:52:16.407 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T09:52:16.468 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-25T09:52:16.468 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-25T09:52:16.469 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000190628 s, 2.7 MB/s 2024-04-25T09:52:16.469 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T09:52:16.525 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:16.526 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi139.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T09:52:16.526 INFO:tasks.ceph:Generating config... 2024-04-25T09:52:16.527 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T09:52:16.527 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T09:52:16.527 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T09:52:16.527 INFO:tasks.ceph:[client] pid file = 2024-04-25T09:52:16.527 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T09:52:16.527 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T09:52:16.527 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T09:52:16.528 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T09:52:16.529 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T09:52:16.529 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T09:52:16.530 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T09:52:16.530 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T09:52:16.591 INFO:teuthology.orchestra.run.smithi139.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T09:52:16.593 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T09:52:16.636 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T09:52:16.702 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.139'), ('mon.b', '172.21.15.150'), ('mon.c', '[v2:172.21.15.150:3301,v1:172.21.15.150:6790]')] 2024-04-25T09:52:16.702 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.139,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T09:52:16.703 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:16.704 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T09:52:16.757 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.139 --add b 172.21.15.150 --addv c '[v2:172.21.15.150:3301,v1:172.21.15.150:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:16.841 INFO:teuthology.orchestra.run.smithi139.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:16.841 INFO:teuthology.orchestra.run.smithi139.stdout:monmaptool: generated fsid 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T09:52:16.841 INFO:teuthology.orchestra.run.smithi139.stdout:setting min_mon_release = quincy 2024-04-25T09:52:16.841 INFO:teuthology.orchestra.run.smithi139.stdout:epoch 0 2024-04-25T09:52:16.842 INFO:teuthology.orchestra.run.smithi139.stdout:fsid 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T09:52:16.842 INFO:teuthology.orchestra.run.smithi139.stdout:last_changed 2024-04-25T09:52:16.841039+0000 2024-04-25T09:52:16.842 INFO:teuthology.orchestra.run.smithi139.stdout:created 2024-04-25T09:52:16.841039+0000 2024-04-25T09:52:16.842 INFO:teuthology.orchestra.run.smithi139.stdout:min_mon_release 17 (quincy) 2024-04-25T09:52:16.842 INFO:teuthology.orchestra.run.smithi139.stdout:election_strategy: 1 2024-04-25T09:52:16.842 INFO:teuthology.orchestra.run.smithi139.stdout:0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-04-25T09:52:16.842 INFO:teuthology.orchestra.run.smithi139.stdout:1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-25T09:52:16.842 INFO:teuthology.orchestra.run.smithi139.stdout:2: [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] mon.c 2024-04-25T09:52:16.842 INFO:teuthology.orchestra.run.smithi139.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T09:52:16.843 DEBUG:teuthology.orchestra.run.smithi139:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T09:52:16.858 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 922739a8-edcf-4ded-8206-dbdb23e9fea4... 2024-04-25T09:52:16.859 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:52:16.900 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:52:16.938 INFO:teuthology.orchestra.run.smithi139.stdout:[global] 2024-04-25T09:52:16.938 INFO:teuthology.orchestra.run.smithi139.stdout: chdir = "" 2024-04-25T09:52:16.938 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:52:16.938 INFO:teuthology.orchestra.run.smithi139.stdout: auth supported = cephx 2024-04-25T09:52:16.938 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.938 INFO:teuthology.orchestra.run.smithi139.stdout: filestore xattr use omap = true 2024-04-25T09:52:16.938 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.938 INFO:teuthology.orchestra.run.smithi139.stdout: mon clock drift allowed = 1.000 2024-04-25T09:52:16.938 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: osd crush chooseleaf type = 0 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: auth debug = true 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on old message = true 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on bug = true 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: mon pg warn max object skew = 0 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default size = 2 2024-04-25T09:52:16.939 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow primary affinity = true 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow pg remap = true 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on no sortbitwise = false 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on too few osds = false 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: mon_allow_pool_size_one = true 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: osd default data pool replay window = 5 2024-04-25T09:52:16.940 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: mon allow pool delete = true 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: mon cluster log file level = debug 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: debug asserts on shutdown = true 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: mon health detail to clog = false 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: mon host = "172.21.15.139,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on skipped message = False 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: fsid = 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout:[osd] 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: osd journal size = 100 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub load threshold = 5.0 2024-04-25T09:52:16.941 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub max interval = 600 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: osd recover clone overlap = true 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: osd recovery max chunk = 1048576 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug shutdown = true 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug op order = true 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug verify stray on activate = true 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug trim objects = true 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: osd open classes on start = true 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug pg log writeout = true 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.942 INFO:teuthology.orchestra.run.smithi139.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: osd map max advance = 10 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: journal zero on create = true 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: filestore apply finisher threads = 3 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: bdev debug aio = true 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug misdirected ops = true 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore block size = 96636764160 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore fsck on mount = True 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluefs = 20 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluestore = 20 2024-04-25T09:52:16.943 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: debug osd = 20 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: debug rocksdb = 10 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd full ratio = 0.9 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: osd objectstore = bluestore 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr] 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: debug mgr = 20 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-04-25T09:52:16.944 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-04-25T09:52:16.945 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:52:16.945 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:52:16.945 INFO:teuthology.orchestra.run.smithi139.stdout: mgr/telemetry/nag = false 2024-04-25T09:52:16.945 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.945 INFO:teuthology.orchestra.run.smithi139.stdout:[mon] 2024-04-25T09:52:16.945 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T09:52:16.945 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-04-25T09:52:16.945 INFO:teuthology.orchestra.run.smithi139.stdout: debug paxos = 20 2024-04-25T09:52:16.945 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-04-25T09:52:16.946 INFO:teuthology.orchestra.run.smithi139.stdout: mon data avail warn = 5 2024-04-25T09:52:16.946 INFO:teuthology.orchestra.run.smithi139.stdout: mon mgr mkfs grace = 240 2024-04-25T09:52:16.946 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:52:16.946 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd reporter subtree level = osd 2024-04-25T09:52:16.946 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd prime pg temp = true 2024-04-25T09:52:16.946 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:52:16.946 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.946 INFO:teuthology.orchestra.run.smithi139.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: # 1m isn't quite enough 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: mon op complaint time = 120 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:16.947 INFO:teuthology.orchestra.run.smithi139.stdout:[client] 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout: rgw cache enabled = true 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable ops log = true 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable usage log = true 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout: client mount timeout = 600 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout: debug client = 20 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = "" 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.a] 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.b] 2024-04-25T09:52:16.948 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.c] 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout:[mds] 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds = 20 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds balancer = 20 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug frag = True 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug scatterstat = True 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout: mds op complaint time = 180 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout: mds verify scatter = True 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-04-25T09:52:16.949 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-04-25T09:52:16.950 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-04-25T09:52:16.950 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-04-25T09:52:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2024-04-25T09:52:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:52:16.950 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2024-04-25T09:52:16.951 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:52:16.952 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.139,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on skipped message = False 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T09:52:16.953 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2024-04-25T09:52:16.954 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug trim objects = true 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.955 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 20 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 20 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 10 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2024-04-25T09:52:16.956 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-25T09:52:16.957 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:52:16.958 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: mon op complaint time = 120 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:52:16.959 INFO:teuthology.orchestra.run.smithi150.stdout: client mount timeout = 600 2024-04-25T09:52:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2024-04-25T09:52:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T09:52:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2024-04-25T09:52:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2024-04-25T09:52:16.961 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2024-04-25T09:52:16.961 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2024-04-25T09:52:16.961 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2024-04-25T09:52:16.961 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2024-04-25T09:52:16.961 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2024-04-25T09:52:16.961 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2024-04-25T09:52:16.961 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2024-04-25T09:52:16.962 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2024-04-25T09:52:16.962 INFO:teuthology.orchestra.run.smithi150.stdout:[mds] 2024-04-25T09:52:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds = 20 2024-04-25T09:52:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds balancer = 20 2024-04-25T09:52:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T09:52:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug frag = True 2024-04-25T09:52:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug scatterstat = True 2024-04-25T09:52:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: mds op complaint time = 180 2024-04-25T09:52:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: mds verify scatter = True 2024-04-25T09:52:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2024-04-25T09:52:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2024-04-25T09:52:16.962 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2024-04-25T09:52:16.964 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T09:52:16.964 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T09:52:17.055 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T09:52:17.055 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:17.055 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T09:52:17.070 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:17.070 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T09:52:17.124 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:52:17.124 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:17.124 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T09:52:17.124 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T09:52:17.196 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:17.196 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:17.251 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T09:52:17.251 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:17.251 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T09:52:17.251 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T09:52:17.285 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:17.285 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:17.340 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T09:52:17.340 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T09:52:17.340 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T09:52:17.388 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T09:52:17.390 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T09:52:17.459 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T09:52:17.461 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T09:52:17.461 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T09:52:17.509 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T09:52:17.512 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T09:52:17.535 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T09:52:17.623 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T09:52:17.625 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T09:52:17.648 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T09:52:17.696 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T09:52:17.699 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T09:52:17.723 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T09:52:17.811 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T09:52:17.813 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T09:52:17.837 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T09:52:17.837 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T09:52:17.877 INFO:teuthology.orchestra.run.smithi139.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T09:52:17.889 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T09:52:17.968 INFO:teuthology.orchestra.run.smithi139.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T09:52:17.979 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T09:52:18.018 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T09:52:18.030 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T09:52:18.109 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T09:52:18.121 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T09:52:18.121 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi139.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T09:52:18.121 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T09:52:18.144 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:18.145 INFO:tasks.ceph:role: osd.0 2024-04-25T09:52:18.145 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:52:18.145 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T09:52:18.215 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:18.215 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:18.216 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:18.216 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:18.216 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:18.216 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:18.216 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:18.216 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:18.216 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:18.216 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:18.718 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-04-25T09:52:18.720 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-04-25T09:52:18.720 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T09:52:18.854 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T09:52:18.883 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T09:52:18.947 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:18.947 INFO:tasks.ceph:role: osd.1 2024-04-25T09:52:18.947 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:52:18.947 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T09:52:19.018 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:19.019 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:19.019 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:19.019 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:19.019 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:19.019 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:19.019 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:19.019 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:19.019 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:19.019 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:19.482 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-04-25T09:52:19.483 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-04-25T09:52:19.484 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T09:52:19.579 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T09:52:19.608 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T09:52:19.672 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:19.672 INFO:tasks.ceph:role: osd.2 2024-04-25T09:52:19.672 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:52:19.672 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T09:52:19.743 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:19.744 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:19.744 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:19.744 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:19.744 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:19.744 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:19.744 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:19.744 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:19.744 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:19.744 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:20.234 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-04-25T09:52:20.236 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-04-25T09:52:20.236 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T09:52:20.332 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T09:52:20.362 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T09:52:20.427 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:20.427 INFO:tasks.ceph:role: osd.3 2024-04-25T09:52:20.427 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:52:20.427 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T09:52:20.498 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:20.498 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:20.498 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:20.498 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:20.498 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:20.499 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:20.499 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:20.499 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:20.499 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:20.499 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:21.012 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-04-25T09:52:21.014 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-04-25T09:52:21.015 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T09:52:21.132 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T09:52:21.162 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:21.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:21.249+0000 7f8878a10640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T09:52:21.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:21.249+0000 7f8878a10640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T09:52:21.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:21.249+0000 7f8878a10640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T09:52:21.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:21.249+0000 7f8878a10640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T09:52:21.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:21.249+0000 7f8878a10640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T09:52:21.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:21.250+0000 7f8878a10640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T09:52:24.102 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T09:52:24.128 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:24.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:24.212+0000 7f96058fd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T09:52:24.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:24.212+0000 7f96058fd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T09:52:24.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:24.212+0000 7f96058fd640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T09:52:24.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:24.212+0000 7f96058fd640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T09:52:24.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:24.212+0000 7f96058fd640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T09:52:24.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:24.213+0000 7f96058fd640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T09:52:27.062 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T09:52:27.089 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:27.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:27.175+0000 7fe1c61cb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T09:52:27.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:27.176+0000 7fe1c61cb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T09:52:27.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:27.176+0000 7fe1c61cb640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T09:52:27.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:27.176+0000 7fe1c61cb640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T09:52:27.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:27.176+0000 7fe1c61cb640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T09:52:27.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:27.176+0000 7fe1c61cb640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T09:52:30.028 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T09:52:30.056 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:30.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:30.143+0000 7f8ae6c0c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T09:52:30.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:30.143+0000 7f8ae6c0c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T09:52:30.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:30.143+0000 7f8ae6c0c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T09:52:30.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:30.143+0000 7f8ae6c0c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T09:52:30.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:30.143+0000 7f8ae6c0c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T09:52:30.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:30.144+0000 7f8ae6c0c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T09:52:32.996 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T09:52:33.024 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T09:52:33.053 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:33.053 INFO:tasks.ceph:role: osd.4 2024-04-25T09:52:33.053 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T09:52:33.053 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T09:52:33.128 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:33.128 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:33.128 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:33.128 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:33.128 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:33.129 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:33.129 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:33.129 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:33.129 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:33.129 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:33.639 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-04-25T09:52:33.641 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-04-25T09:52:33.642 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T09:52:33.786 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T09:52:33.818 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T09:52:33.882 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:33.882 INFO:tasks.ceph:role: osd.5 2024-04-25T09:52:33.882 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T09:52:33.882 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T09:52:33.954 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:33.954 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:33.954 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:33.954 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:33.954 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:33.954 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:33.954 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:33.955 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:33.955 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:33.955 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:34.396 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-04-25T09:52:34.399 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-04-25T09:52:34.399 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T09:52:34.491 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T09:52:34.519 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T09:52:34.582 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:34.583 INFO:tasks.ceph:role: osd.6 2024-04-25T09:52:34.583 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T09:52:34.583 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T09:52:34.653 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:34.654 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:34.654 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:34.654 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:34.654 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:34.654 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:35.143 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-04-25T09:52:35.144 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-04-25T09:52:35.144 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T09:52:35.271 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T09:52:35.299 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T09:52:35.363 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:52:35.363 INFO:tasks.ceph:role: osd.7 2024-04-25T09:52:35.363 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T09:52:35.363 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T09:52:35.434 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:52:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:52:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:52:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:52:35.435 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:52:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:52:35.435 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:52:35.435 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:52:35.435 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:52:35.435 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:52:35.960 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-04-25T09:52:35.962 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-04-25T09:52:35.962 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T09:52:36.091 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T09:52:36.121 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:36.215+0000 7f319d015640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T09:52:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:36.215+0000 7f319d015640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T09:52:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:36.216+0000 7f319d015640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T09:52:36.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:36.216+0000 7f319d015640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T09:52:36.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:36.216+0000 7f319d015640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T09:52:36.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:36.216+0000 7f319d015640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T09:52:39.067 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T09:52:39.094 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:39.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:39.179+0000 7fbcb4086640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T09:52:39.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:39.180+0000 7fbcb4086640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T09:52:39.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:39.180+0000 7fbcb4086640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T09:52:39.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:39.180+0000 7fbcb4086640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T09:52:39.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:39.180+0000 7fbcb4086640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T09:52:39.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:39.180+0000 7fbcb4086640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T09:52:42.037 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T09:52:42.064 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:42.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:42.151+0000 7faa498f0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T09:52:42.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:42.151+0000 7faa498f0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T09:52:42.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:42.151+0000 7faa498f0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T09:52:42.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:42.151+0000 7faa498f0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T09:52:42.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:42.151+0000 7faa498f0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T09:52:42.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:42.152+0000 7faa498f0640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T09:52:44.976 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:52:44.979 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:52:45.033 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T09:52:45.058 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:45.139+0000 7f3abf40c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T09:52:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:45.140+0000 7f3abf40c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T09:52:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:45.140+0000 7f3abf40c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T09:52:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:45.140+0000 7f3abf40c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T09:52:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:45.140+0000 7f3abf40c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T09:52:45.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:45.140+0000 7f3abf40c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T09:52:47.996 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T09:52:48.065 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T09:52:48.065 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:48.065 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T09:52:48.095 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:48.095 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T09:52:48.162 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:48.162 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T09:52:48.228 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:48.228 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T09:52:48.293 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:48.293 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T09:52:48.358 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:48.358 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T09:52:48.423 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:48.423 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T09:52:48.488 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:48.488 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T09:52:48.516 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:48.517 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T09:52:48.583 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:48.583 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T09:52:48.650 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:48.650 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T09:52:48.715 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:48.715 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T09:52:48.779 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:48.780 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T09:52:48.844 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:48.844 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T09:52:48.908 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:48.909 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T09:52:48.925 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:48.926 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T09:52:48.982 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:48.982 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T09:52:48.997 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:48.997 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T09:52:49.052 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T09:52:49.053 DEBUG:teuthology.orchestra.run.smithi139:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T09:52:49.055 DEBUG:teuthology.orchestra.run.smithi150:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T09:52:49.096 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr.y] 2024-04-25T09:52:49.096 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmuJ4bFxAAw6a53AgThU/ut+Mk5j0gqQ== 2024-04-25T09:52:49.096 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2024-04-25T09:52:49.096 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypm0xdaHhAA6nupxCTQhucepW2iEWNUoQ== 2024-04-25T09:52:49.096 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2024-04-25T09:52:49.096 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmvYgaJRAA/YIE6ralp2WuTK993D7H0Q== 2024-04-25T09:52:49.096 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.0] 2024-04-25T09:52:49.096 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDVJypmG7TpDhAAo9CKuZBw8BsQgB2F+BR5gw== 2024-04-25T09:52:49.097 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.1] 2024-04-25T09:52:49.097 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDYJypmgR62DBAAotfnvS47WDunP9Ic03gA6Q== 2024-04-25T09:52:49.097 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.2] 2024-04-25T09:52:49.097 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDbJypmzyqGChAA6wrfIdqpiLm3RO36oorihQ== 2024-04-25T09:52:49.097 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.3] 2024-04-25T09:52:49.097 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDeJypmt72cCBAAgcLrFC3MUU7wlFySAfLWig== 2024-04-25T09:52:49.097 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr.x] 2024-04-25T09:52:49.097 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmt+RVGxAAi1djVoqnoYCDMzJ9A0dmNA== 2024-04-25T09:52:49.097 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2024-04-25T09:52:49.097 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmB9GDKRAA/lmU4OiKO5dFgayYx09GQg== 2024-04-25T09:52:49.097 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2024-04-25T09:52:49.098 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmg49WMBAAlPY9ZSeOoV296xSdl/m1gg== 2024-04-25T09:52:49.098 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.4] 2024-04-25T09:52:49.098 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDkJypmxJLpDBAAvROZ9AvYdWMqOALEKIAFCw== 2024-04-25T09:52:49.098 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.5] 2024-04-25T09:52:49.098 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDnJypmTPXEChAA20BghY39PGw6zFyu9yWbBQ== 2024-04-25T09:52:49.098 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.6] 2024-04-25T09:52:49.098 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDqJypmaBoYCRAAy9eHgUI/WTkE2XdT1RKLsw== 2024-04-25T09:52:49.098 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.7] 2024-04-25T09:52:49.099 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDtJypmViJlCBAANJSDWVHVbWR03Y1SMDH++Q== 2024-04-25T09:52:49.099 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2024-04-25T09:52:49.099 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg== 2024-04-25T09:52:49.099 INFO:teuthology.orchestra.run.smithi139.stdout:[client.2] 2024-04-25T09:52:49.099 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmQs2wORAArzyV70nGDX4486LlFjM70Q== 2024-04-25T09:52:49.099 INFO:teuthology.orchestra.run.smithi139.stdout:[client.1] 2024-04-25T09:52:49.099 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDSJypmOncYARAA6gYjZbRDkhqFX0mpSDbH5Q== 2024-04-25T09:52:49.099 INFO:teuthology.orchestra.run.smithi139.stdout:[client.3] 2024-04-25T09:52:49.100 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDSJypm90x9BhAA41rZPs0O5mwmuHy1bkPCvw== 2024-04-25T09:52:49.114 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.y] 2024-04-25T09:52:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDRJypmuJ4bFxAAw6a53AgThU/ut+Mk5j0gqQ== 2024-04-25T09:52:49.115 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2024-04-25T09:52:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDRJypm0xdaHhAA6nupxCTQhucepW2iEWNUoQ== 2024-04-25T09:52:49.115 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2024-04-25T09:52:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDRJypmvYgaJRAA/YIE6ralp2WuTK993D7H0Q== 2024-04-25T09:52:49.115 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.0] 2024-04-25T09:52:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDVJypmG7TpDhAAo9CKuZBw8BsQgB2F+BR5gw== 2024-04-25T09:52:49.115 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.1] 2024-04-25T09:52:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDYJypmgR62DBAAotfnvS47WDunP9Ic03gA6Q== 2024-04-25T09:52:49.115 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.2] 2024-04-25T09:52:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDbJypmzyqGChAA6wrfIdqpiLm3RO36oorihQ== 2024-04-25T09:52:49.115 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.3] 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDeJypmt72cCBAAgcLrFC3MUU7wlFySAfLWig== 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.x] 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDRJypmt+RVGxAAi1djVoqnoYCDMzJ9A0dmNA== 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDRJypmB9GDKRAA/lmU4OiKO5dFgayYx09GQg== 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDRJypmg49WMBAAlPY9ZSeOoV296xSdl/m1gg== 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.4] 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDkJypmxJLpDBAAvROZ9AvYdWMqOALEKIAFCw== 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.5] 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDnJypmTPXEChAA20BghY39PGw6zFyu9yWbBQ== 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.6] 2024-04-25T09:52:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDqJypmaBoYCRAAy9eHgUI/WTkE2XdT1RKLsw== 2024-04-25T09:52:49.117 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.7] 2024-04-25T09:52:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDtJypmViJlCBAANJSDWVHVbWR03Y1SMDH++Q== 2024-04-25T09:52:49.117 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2024-04-25T09:52:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg== 2024-04-25T09:52:49.117 INFO:teuthology.orchestra.run.smithi150.stdout:[client.2] 2024-04-25T09:52:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDRJypmQs2wORAArzyV70nGDX4486LlFjM70Q== 2024-04-25T09:52:49.117 INFO:teuthology.orchestra.run.smithi150.stdout:[client.1] 2024-04-25T09:52:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDSJypmOncYARAA6gYjZbRDkhqFX0mpSDbH5Q== 2024-04-25T09:52:49.117 INFO:teuthology.orchestra.run.smithi150.stdout:[client.3] 2024-04-25T09:52:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDSJypm90x9BhAA41rZPs0O5mwmuHy1bkPCvw== 2024-04-25T09:52:49.119 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:52:49.144 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:52:49.208 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:49.211 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:49.262 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:49.265 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:49.316 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.318 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.366 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.369 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.417 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.419 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.468 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.471 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.519 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:52:49.522 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:52:49.571 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:49.574 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:49.626 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:49.628 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:52:49.679 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.682 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.735 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.737 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.785 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.788 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.837 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.840 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:52:49.888 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:49.890 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:49.941 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:49.943 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:49.995 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:49.997 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:50.049 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:50.051 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:52:50.103 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T09:52:50.103 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T09:52:50.126 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T09:52:50.219 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T09:52:50.243 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T09:52:50.269 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T09:52:50.363 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T09:52:50.387 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T09:52:50.450 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T09:52:50.543 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T09:52:50.566 DEBUG:teuthology.orchestra.run.smithi139:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:50.568 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:52:50.621 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T09:52:50.621 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T09:52:50.621 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T09:52:50.624 INFO:tasks.ceph.mon.a:Started 2024-04-25T09:52:50.624 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T09:52:50.624 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T09:52:50.664 INFO:tasks.ceph.mon.b:Started 2024-04-25T09:52:50.664 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T09:52:50.664 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T09:52:50.666 INFO:tasks.ceph.mon.c:Started 2024-04-25T09:52:50.666 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T09:52:50.667 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T09:52:50.667 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T09:52:50.669 INFO:tasks.ceph.mgr.y:Started 2024-04-25T09:52:50.669 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T09:52:50.669 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T09:52:50.671 INFO:tasks.ceph.mgr.x:Started 2024-04-25T09:52:50.671 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T09:52:50.671 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T09:52:50.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.765+0000 7ff40a7b0640 1 Processor -- start 2024-04-25T09:52:50.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.765+0000 7ff40a7b0640 1 -- start start 2024-04-25T09:52:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff40a7b0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff40415a870 0x7ff4040b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff40a7b0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff4040b4190 0x7ff4040d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff40a7b0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4040b45f0 con 0x7ff404070260 2024-04-25T09:52:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff40a7b0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4040d1020 con 0x7ff4040b4190 2024-04-25T09:52:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff40a7b0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4040b5190 con 0x7ff40415a870 2024-04-25T09:52:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff4037fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff4040b4190 0x7ff4040d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff408d4d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff404070260 0x7ff4040ce610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:36412/0 (socket says 172.21.15.139:36412) 2024-04-25T09:52:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff408d4d640 1 -- 172.21.15.139:0/2898660375 learned_addr learned my addr 172.21.15.139:0/2898660375 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:52:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff4037fe640 1 -- 172.21.15.139:0/2898660375 >> v2:172.21.15.139:3300/0 conn(0x7ff4040b4190 msgr2=0x7ff4040d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T09:52:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff4037fe640 1 -- 172.21.15.139:0/2898660375 >> v2:172.21.15.139:3300/0 conn(0x7ff4040b4190 msgr2=0x7ff4040d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T09:52:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff4037fe640 1 --2- 172.21.15.139:0/2898660375 >> v2:172.21.15.139:3300/0 conn(0x7ff4040b4190 0x7ff4040d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T09:52:50.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff4037fe640 1 --2- 172.21.15.139:0/2898660375 >> v2:172.21.15.139:3300/0 conn(0x7ff4040b4190 0x7ff4040d05f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T09:52:50.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff403fff640 1 --2- 172.21.15.139:0/2898660375 >> v2:172.21.15.150:3300/0 conn(0x7ff40415a870 0x7ff4040b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:50.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff403fff640 1 -- 172.21.15.139:0/2898660375 >> v2:172.21.15.150:3300/0 conn(0x7ff40415a870 msgr2=0x7ff4040b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T09:52:50.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff403fff640 1 -- 172.21.15.139:0/2898660375 >> v2:172.21.15.150:3300/0 conn(0x7ff40415a870 msgr2=0x7ff4040b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T09:52:50.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff403fff640 1 --2- 172.21.15.139:0/2898660375 >> v2:172.21.15.150:3300/0 conn(0x7ff40415a870 0x7ff4040b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T09:52:50.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.766+0000 7ff403fff640 1 --2- 172.21.15.139:0/2898660375 >> v2:172.21.15.150:3300/0 conn(0x7ff40415a870 0x7ff4040b5620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T09:52:50.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240857431 0 0) 0x7ff4040b45f0 con 0x7ff404070260 2024-04-25T09:52:50.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3ec003520 con 0x7ff404070260 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2937155834 0 0) 0x7ff3ec003520 con 0x7ff404070260 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 >> v2:172.21.15.150:3300/0 conn(0x7ff40415a870 msgr2=0x7ff4040b5620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 --2- 172.21.15.139:0/2898660375 >> v2:172.21.15.150:3300/0 conn(0x7ff40415a870 0x7ff4040b5620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 >> v2:172.21.15.139:3300/0 conn(0x7ff4040b4190 msgr2=0x7ff4040d05f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 --2- 172.21.15.139:0/2898660375 >> v2:172.21.15.139:3300/0 conn(0x7ff4040b4190 0x7ff4040d05f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4040d0d40 con 0x7ff404070260 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7ff3fc0019f0 con 0x7ff404070260 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 >> v1:172.21.15.139:6789/0 conn(0x7ff404070260 legacy=0x7ff4040ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 --2- 172.21.15.139:0/2898660375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3ec0039b0 0x7ff3ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 --2- 172.21.15.139:0/2898660375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ec004800 0x7ff3ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 --2- 172.21.15.139:0/2898660375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3ec005230 0x7ff3ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040d1020 con 0x7ff3ec004800 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040b5190 con 0x7ff3ec0039b0 2024-04-25T09:52:50.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec00c210 con 0x7ff3ec005230 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff4037fe640 1 --2- 172.21.15.139:0/2898660375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ec004800 0x7ff3ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff403fff640 1 --2- 172.21.15.139:0/2898660375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3ec0039b0 0x7ff3ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.922+0000 7ff408d4d640 1 --2- 172.21.15.139:0/2898660375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3ec005230 0x7ff3ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.923+0000 7ff4037fe640 1 -- 172.21.15.139:0/2898660375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3ec005230 msgr2=0x7ff3ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.923+0000 7ff4037fe640 1 --2- 172.21.15.139:0/2898660375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3ec005230 0x7ff3ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.923+0000 7ff4037fe640 1 -- 172.21.15.139:0/2898660375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3ec0039b0 msgr2=0x7ff3ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.923+0000 7ff4037fe640 1 --2- 172.21.15.139:0/2898660375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3ec0039b0 0x7ff3ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.923+0000 7ff4037fe640 1 -- 172.21.15.139:0/2898660375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec003520 con 0x7ff3ec004800 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.923+0000 7ff408d4d640 1 -- 172.21.15.139:0/2898660375 reap_dead start 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.923+0000 7ff403fff640 1 --2- 172.21.15.139:0/2898660375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3ec0039b0 0x7ff3ec003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.923+0000 7ff4037fe640 1 --2- 172.21.15.139:0/2898660375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ec004800 0x7ff3ec004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800b9e0 tx=0x7ff3ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ea010174295d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.924+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8030070 con 0x7ff3ec004800 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.924+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f802ada0 con 0x7ff3ec004800 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.924+0000 7ff402ffd640 1 -- 172.21.15.139:0/2898660375 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f800b5e0 con 0x7ff3ec004800 2024-04-25T09:52:50.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.924+0000 7ff40a7b0640 1 -- 172.21.15.139:0/2898660375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ec004800 msgr2=0x7ff3ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:50.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.924+0000 7ff40a7b0640 1 --2- 172.21.15.139:0/2898660375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ec004800 0x7ff3ec004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800b9e0 tx=0x7ff3ec005830 comp rx=0 tx=0).stop 2024-04-25T09:52:50.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.924+0000 7ff40a7b0640 1 -- 172.21.15.139:0/2898660375 shutdown_connections 2024-04-25T09:52:50.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.924+0000 7ff40a7b0640 1 --2- 172.21.15.139:0/2898660375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ec004800 0x7ff3ec004c80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:50.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.924+0000 7ff40a7b0640 1 -- 172.21.15.139:0/2898660375 >> 172.21.15.139:0/2898660375 conn(0x7ff4040b23f0 msgr2=0x7ff4040b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:50.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.924+0000 7ff40a7b0640 1 -- 172.21.15.139:0/2898660375 shutdown_connections 2024-04-25T09:52:50.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.924+0000 7ff40a7b0640 1 -- 172.21.15.139:0/2898660375 wait complete. 2024-04-25T09:52:50.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.926+0000 7ff40a7b0640 1 Processor -- start 2024-04-25T09:52:50.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.926+0000 7ff40a7b0640 1 -- start start 2024-04-25T09:52:50.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.926+0000 7ff40a7b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3ec004800 0x7ff40414b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:50.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.926+0000 7ff40a7b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff404152150 0x7ff40414b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.926+0000 7ff40a7b0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff40414be90 0x7ff4041586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.926+0000 7ff40a7b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040b5620 con 0x7ff404152150 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.926+0000 7ff40a7b0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040b4e20 con 0x7ff40414be90 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.926+0000 7ff40a7b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040b57a0 con 0x7ff3ec004800 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff4037fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff404152150 0x7ff40414b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff4037fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff404152150 0x7ff40414b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58018/0 (socket says 172.21.15.139:58018) 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff4037fe640 1 -- 172.21.15.139:0/3357439396 learned_addr learned my addr 172.21.15.139:0/3357439396 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff403fff640 1 --2- 172.21.15.139:0/3357439396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3ec004800 0x7ff40414b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff408d4d640 1 --2- 172.21.15.139:0/3357439396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff40414be90 0x7ff4041586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff4037fe640 1 -- 172.21.15.139:0/3357439396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3ec004800 msgr2=0x7ff40414b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff4037fe640 1 --2- 172.21.15.139:0/3357439396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3ec004800 0x7ff40414b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff4037fe640 1 -- 172.21.15.139:0/3357439396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff40414be90 msgr2=0x7ff4041586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff4037fe640 1 --2- 172.21.15.139:0/3357439396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff40414be90 0x7ff4041586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff4037fe640 1 -- 172.21.15.139:0/3357439396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4040c9b10 con 0x7ff404152150 2024-04-25T09:52:50.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff4037fe640 1 --2- 172.21.15.139:0/3357439396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff404152150 0x7ff40414b950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800bb00 tx=0x7ff3f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:50.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff400ff9640 1 -- 172.21.15.139:0/3357439396 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8043020 con 0x7ff404152150 2024-04-25T09:52:50.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff400ff9640 1 -- 172.21.15.139:0/3357439396 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f803f680 con 0x7ff404152150 2024-04-25T09:52:50.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff40a7b0640 1 -- 172.21.15.139:0/3357439396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff404152af0 con 0x7ff404152150 2024-04-25T09:52:50.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff400ff9640 1 -- 172.21.15.139:0/3357439396 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f800b5e0 con 0x7ff404152150 2024-04-25T09:52:50.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.927+0000 7ff40a7b0640 1 -- 172.21.15.139:0/3357439396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4041530b0 con 0x7ff404152150 2024-04-25T09:52:50.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.928+0000 7ff400ff9640 1 -- 172.21.15.139:0/3357439396 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff3f8046020 con 0x7ff404152150 2024-04-25T09:52:50.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.928+0000 7ff400ff9640 1 -- 172.21.15.139:0/3357439396 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff3f803dcf0 con 0x7ff404152150 2024-04-25T09:52:50.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.928+0000 7ff40a7b0640 1 -- 172.21.15.139:0/3357439396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff404074520 con 0x7ff404152150 2024-04-25T09:52:50.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.930+0000 7ff400ff9640 1 -- 172.21.15.139:0/3357439396 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff3f804d030 con 0x7ff404152150 2024-04-25T09:52:50.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.997+0000 7ff40a7b0640 1 -- 172.21.15.139:0/3357439396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff404155a00 con 0x7ff404152150 2024-04-25T09:52:51.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.998+0000 7ff400ff9640 1 -- 172.21.15.139:0/3357439396 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff3f803d440 con 0x7ff404152150 2024-04-25T09:52:51.001 INFO:teuthology.orchestra.run.smithi139.stderr:adjusted tunables profile to default 2024-04-25T09:52:51.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.999+0000 7ff40a7b0640 1 -- 172.21.15.139:0/3357439396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff404152150 msgr2=0x7ff40414b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.999+0000 7ff40a7b0640 1 --2- 172.21.15.139:0/3357439396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff404152150 0x7ff40414b950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800bb00 tx=0x7ff3f8004730 comp rx=0 tx=0).stop 2024-04-25T09:52:51.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.999+0000 7ff40a7b0640 1 -- 172.21.15.139:0/3357439396 shutdown_connections 2024-04-25T09:52:51.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.999+0000 7ff40a7b0640 1 --2- 172.21.15.139:0/3357439396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff404152150 0x7ff40414b950 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.999+0000 7ff40a7b0640 1 --2- 172.21.15.139:0/3357439396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff40414be90 0x7ff4041586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.999+0000 7ff40a7b0640 1 --2- 172.21.15.139:0/3357439396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3ec004800 0x7ff40414b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.999+0000 7ff40a7b0640 1 -- 172.21.15.139:0/3357439396 >> 172.21.15.139:0/3357439396 conn(0x7ff4040b23f0 msgr2=0x7ff4040b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:51.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.999+0000 7ff40a7b0640 1 -- 172.21.15.139:0/3357439396 shutdown_connections 2024-04-25T09:52:51.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:50.999+0000 7ff40a7b0640 1 -- 172.21.15.139:0/3357439396 wait complete. 2024-04-25T09:52:51.013 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T09:52:51.013 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T09:52:51.013 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:51.013 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T09:52:51.033 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:51.031+0000 7f26c3819040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T09:52:51.036 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:51.037 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T09:52:51.049 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:51.047+0000 7f7b52f30040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T09:52:51.100 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:51.100 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T09:52:51.164 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:52:51.164 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T09:52:51.200 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:51.198+0000 7f7b52f30040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:52:51.203 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:51.202+0000 7f26c3819040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:52:51.226 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:51.226 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T09:52:51.250 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:51.250 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T09:52:51.255 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:51.253+0000 7f7b52f30040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T09:52:51.259 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:51.257+0000 7f26c3819040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T09:52:51.278 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:51.278 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T09:52:51.342 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:52:51.343 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T09:52:51.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 4af65dcf-4971-4314-8195-7e9e848ba2e2 0 2024-04-25T09:52:51.501 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:51.499+0000 7f26c3819040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T09:52:51.508 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:51.507+0000 7f7b52f30040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T09:52:51.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.543+0000 7f9f9282c640 1 Processor -- start 2024-04-25T09:52:51.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.543+0000 7f9f9282c640 1 -- start start 2024-04-25T09:52:51.546 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:51.544+0000 7f26c3819040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T09:52:51.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.543+0000 7f9f9282c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9f8c06b270 0x7f9f8c0704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.543+0000 7f9f9282c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c072d60 0x7f9f8c167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.543+0000 7f9f9282c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c15de10 con 0x7f9f8c15da30 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.544+0000 7f9f9282c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f8c073210 con 0x7f9f8c06b270 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.544+0000 7f9f9282c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0cd540 con 0x7f9f8c072d60 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.544+0000 7f9f8bfff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9f8c15da30 0x7f9f8c06ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:57768/0 (socket says 172.21.15.150:57768) 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.544+0000 7f9f90dc9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c072d60 0x7f9f8c167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.544+0000 7f9f8bfff640 1 -- 172.21.15.150:0/3743636436 learned_addr learned my addr 172.21.15.150:0/3743636436 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.544+0000 7f9f8b7fe640 1 --2- 172.21.15.150:0/3743636436 >> v2:172.21.15.139:3300/0 conn(0x7f9f8c06b270 0x7f9f8c0704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.544+0000 7f9f8affd640 1 -- 172.21.15.150:0/3743636436 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608076495 0 0) 0x7f9f8c15de10 con 0x7f9f8c15da30 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.544+0000 7f9f90dc9640 1 -- 172.21.15.150:0/3743636436 >> v2:172.21.15.139:3300/0 conn(0x7f9f8c06b270 msgr2=0x7f9f8c0704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.544+0000 7f9f90dc9640 1 --2- 172.21.15.150:0/3743636436 >> v2:172.21.15.139:3300/0 conn(0x7f9f8c06b270 0x7f9f8c0704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.544+0000 7f9f90dc9640 1 -- 172.21.15.150:0/3743636436 >> v1:172.21.15.150:6789/0 conn(0x7f9f8c15da30 legacy=0x7f9f8c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.544+0000 7f9f90dc9640 1 -- 172.21.15.150:0/3743636436 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c0d1470 con 0x7f9f8c072d60 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.545+0000 7f9f90dc9640 1 --2- 172.21.15.150:0/3743636436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c072d60 0x7f9f8c167550 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009d90 tx=0x7f9f8402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c64713a7af9f8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:51.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.545+0000 7f9f8affd640 1 -- 172.21.15.150:0/3743636436 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84007740 con 0x7f9f8c072d60 2024-04-25T09:52:51.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.545+0000 7f9f8affd640 1 -- 172.21.15.150:0/3743636436 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f840078e0 con 0x7f9f8c072d60 2024-04-25T09:52:51.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.545+0000 7f9f8affd640 1 -- 172.21.15.150:0/3743636436 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84007cb0 con 0x7f9f8c072d60 2024-04-25T09:52:51.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.545+0000 7f9f9282c640 1 -- 172.21.15.150:0/3743636436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c072d60 msgr2=0x7f9f8c167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.545+0000 7f9f9282c640 1 --2- 172.21.15.150:0/3743636436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c072d60 0x7f9f8c167550 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009d90 tx=0x7f9f8402a660 comp rx=0 tx=0).stop 2024-04-25T09:52:51.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.545+0000 7f9f9282c640 1 -- 172.21.15.150:0/3743636436 shutdown_connections 2024-04-25T09:52:51.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.545+0000 7f9f9282c640 1 --2- 172.21.15.150:0/3743636436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c072d60 0x7f9f8c167550 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.545+0000 7f9f9282c640 1 --2- 172.21.15.150:0/3743636436 >> v2:172.21.15.139:3300/0 conn(0x7f9f8c06b270 0x7f9f8c0704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.545+0000 7f9f9282c640 1 -- 172.21.15.150:0/3743636436 >> 172.21.15.150:0/3743636436 conn(0x7f9f8c0b23f0 msgr2=0x7f9f8c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:51.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.546+0000 7f9f9282c640 1 -- 172.21.15.150:0/3743636436 shutdown_connections 2024-04-25T09:52:51.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.546+0000 7f9f9282c640 1 -- 172.21.15.150:0/3743636436 wait complete. 2024-04-25T09:52:51.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.547+0000 7f9f9282c640 1 Processor -- start 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.547+0000 7f9f9282c640 1 -- start start 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f9282c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c06b270 0x7f9f8c142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f9282c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f8c072d60 0x7f9f8c0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f9282c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f8c15da30 0x7f9f8c15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f9282c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0b3b00 con 0x7f9f8c072d60 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f9282c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0b3de0 con 0x7f9f8c15da30 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f9282c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0b42a0 con 0x7f9f8c06b270 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f8bfff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c06b270 0x7f9f8c142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f90dc9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f8c15da30 0x7f9f8c15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f8bfff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c06b270 0x7f9f8c142f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:54948/0 (socket says 172.21.15.150:54948) 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f8b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f8c072d60 0x7f9f8c0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f8bfff640 1 -- 172.21.15.150:0/2395751308 learned_addr learned my addr 172.21.15.150:0/2395751308 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T09:52:51.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f8b7fe640 1 -- 172.21.15.150:0/2395751308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c06b270 msgr2=0x7f9f8c142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f8b7fe640 1 --2- 172.21.15.150:0/2395751308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c06b270 0x7f9f8c142f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f8b7fe640 1 -- 172.21.15.150:0/2395751308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f8c15da30 msgr2=0x7f9f8c15c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f8b7fe640 1 --2- 172.21.15.150:0/2395751308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f8c15da30 0x7f9f8c15c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f8b7fe640 1 -- 172.21.15.150:0/2395751308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c145c70 con 0x7f9f8c072d60 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f90dc9640 1 --2- 172.21.15.150:0/2395751308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f8c15da30 0x7f9f8c15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f8bfff640 1 --2- 172.21.15.150:0/2395751308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c06b270 0x7f9f8c142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f8b7fe640 1 --2- 172.21.15.150:0/2395751308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f8c072d60 0x7f9f8c0cc1f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000e870 tx=0x7f9f8000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f88ff9640 1 -- 172.21.15.150:0/2395751308 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f80017070 con 0x7f9f8c072d60 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f88ff9640 1 -- 172.21.15.150:0/2395751308 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8000ee60 con 0x7f9f8c072d60 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f9282c640 1 -- 172.21.15.150:0/2395751308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8c15f770 con 0x7f9f8c072d60 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.548+0000 7f9f88ff9640 1 -- 172.21.15.150:0/2395751308 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8000cb60 con 0x7f9f8c072d60 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.549+0000 7f9f9282c640 1 -- 172.21.15.150:0/2395751308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c15fd30 con 0x7f9f8c072d60 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.549+0000 7f9f88ff9640 1 -- 172.21.15.150:0/2395751308 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9f8001a070 con 0x7f9f8c072d60 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.549+0000 7f9f88ff9640 1 -- 172.21.15.150:0/2395751308 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9f80013470 con 0x7f9f8c072d60 2024-04-25T09:52:51.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.549+0000 7f9f9282c640 1 -- 172.21.15.150:0/2395751308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8c074520 con 0x7f9f8c072d60 2024-04-25T09:52:51.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.553+0000 7f9f88ff9640 1 -- 172.21.15.150:0/2395751308 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9f8000cda0 con 0x7f9f8c072d60 2024-04-25T09:52:51.558 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:51.556+0000 7f7b52f30040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T09:52:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.623+0000 7f9f9282c640 1 -- 172.21.15.150:0/2395751308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4af65dcf-4971-4314-8195-7e9e848ba2e2", "id": 0} v 0) v1 -- 0x7f9f8c0b27d0 con 0x7f9f8c072d60 2024-04-25T09:52:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.625+0000 7f9f88ff9640 1 -- 172.21.15.150:0/2395751308 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4af65dcf-4971-4314-8195-7e9e848ba2e2", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9f80005990 con 0x7f9f8c072d60 2024-04-25T09:52:51.628 INFO:teuthology.orchestra.run.smithi150.stdout:0 2024-04-25T09:52:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.626+0000 7f9f9282c640 1 -- 172.21.15.150:0/2395751308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f8c072d60 msgr2=0x7f9f8c0cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.626+0000 7f9f9282c640 1 --2- 172.21.15.150:0/2395751308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f8c072d60 0x7f9f8c0cc1f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000e870 tx=0x7f9f8000ed30 comp rx=0 tx=0).stop 2024-04-25T09:52:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.626+0000 7f9f9282c640 1 -- 172.21.15.150:0/2395751308 shutdown_connections 2024-04-25T09:52:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.626+0000 7f9f9282c640 1 --2- 172.21.15.150:0/2395751308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f8c072d60 0x7f9f8c0cc1f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.626+0000 7f9f9282c640 1 --2- 172.21.15.150:0/2395751308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f8c15da30 0x7f9f8c15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.626+0000 7f9f9282c640 1 --2- 172.21.15.150:0/2395751308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f8c06b270 0x7f9f8c142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.626+0000 7f9f9282c640 1 -- 172.21.15.150:0/2395751308 >> 172.21.15.150:0/2395751308 conn(0x7f9f8c0b23f0 msgr2=0x7f9f8c0b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.626+0000 7f9f9282c640 1 -- 172.21.15.150:0/2395751308 shutdown_connections 2024-04-25T09:52:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.626+0000 7f9f9282c640 1 -- 172.21.15.150:0/2395751308 wait complete. 2024-04-25T09:52:51.639 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 42562e83-3222-41c7-a37d-cbe67b25096f 1 2024-04-25T09:52:51.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.735+0000 7fdd622dd640 1 Processor -- start 2024-04-25T09:52:51.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.735+0000 7fdd622dd640 1 -- start start 2024-04-25T09:52:51.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.736+0000 7fdd622dd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdd5c06b270 0x7fdd5c0704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.736+0000 7fdd622dd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5c0731d0 con 0x7fdd5c15da30 2024-04-25T09:52:51.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.736+0000 7fdd622dd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5c15de10 con 0x7fdd5c072d60 2024-04-25T09:52:51.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.736+0000 7fdd622dd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c0cd540 con 0x7fdd5c06b270 2024-04-25T09:52:51.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.736+0000 7fdd5b7fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdd5c06b270 0x7fdd5c0704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.736+0000 7fdd6087a640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fdd5c072d60 0x7fdd5c167550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:57782/0 (socket says 172.21.15.150:57782) 2024-04-25T09:52:51.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.736+0000 7fdd6087a640 1 -- 172.21.15.150:0/106479467 learned_addr learned my addr 172.21.15.150:0/106479467 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-25T09:52:51.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.736+0000 7fdd5b7fe640 1 -- 172.21.15.150:0/106479467 >> v1:172.21.15.150:6789/0 conn(0x7fdd5c072d60 legacy=0x7fdd5c167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.736+0000 7fdd5b7fe640 1 -- 172.21.15.150:0/106479467 >> v1:172.21.15.139:6789/0 conn(0x7fdd5c15da30 legacy=0x7fdd5c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.736+0000 7fdd5b7fe640 1 -- 172.21.15.150:0/106479467 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c0d1470 con 0x7fdd5c06b270 2024-04-25T09:52:51.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.737+0000 7fdd5b7fe640 1 --2- 172.21.15.150:0/106479467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5c06b270 0x7fdd5c0704a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0cd6c0 tx=0x7fdd5002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97179512be49178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:51.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.737+0000 7fdd5affd640 1 -- 172.21.15.150:0/106479467 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50037070 con 0x7fdd5c06b270 2024-04-25T09:52:51.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.737+0000 7fdd5affd640 1 -- 172.21.15.150:0/106479467 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50032440 con 0x7fdd5c06b270 2024-04-25T09:52:51.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.737+0000 7fdd5affd640 1 -- 172.21.15.150:0/106479467 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50032810 con 0x7fdd5c06b270 2024-04-25T09:52:51.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.737+0000 7fdd622dd640 1 -- 172.21.15.150:0/106479467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5c06b270 msgr2=0x7fdd5c0704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.737+0000 7fdd622dd640 1 --2- 172.21.15.150:0/106479467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5c06b270 0x7fdd5c0704a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0cd6c0 tx=0x7fdd5002a210 comp rx=0 tx=0).stop 2024-04-25T09:52:51.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.737+0000 7fdd622dd640 1 -- 172.21.15.150:0/106479467 shutdown_connections 2024-04-25T09:52:51.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.737+0000 7fdd622dd640 1 --2- 172.21.15.150:0/106479467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5c06b270 0x7fdd5c0704a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.737+0000 7fdd622dd640 1 -- 172.21.15.150:0/106479467 >> 172.21.15.150:0/106479467 conn(0x7fdd5c0b23f0 msgr2=0x7fdd5c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:51.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.738+0000 7fdd622dd640 1 -- 172.21.15.150:0/106479467 shutdown_connections 2024-04-25T09:52:51.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.738+0000 7fdd622dd640 1 -- 172.21.15.150:0/106479467 wait complete. 2024-04-25T09:52:51.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.739+0000 7fdd622dd640 1 Processor -- start 2024-04-25T09:52:51.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.739+0000 7fdd622dd640 1 -- start start 2024-04-25T09:52:51.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.739+0000 7fdd622dd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd5c06b270 0x7fdd5c0b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd622dd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5c072d60 0x7fdd5c151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd622dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd5c15da30 0x7fdd5c156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd622dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0b3b00 con 0x7fdd5c15da30 2024-04-25T09:52:51.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd622dd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0b3de0 con 0x7fdd5c072d60 2024-04-25T09:52:51.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd622dd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0b42a0 con 0x7fdd5c06b270 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd5b7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5c072d60 0x7fdd5c151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd5b7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5c072d60 0x7fdd5c151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51224/0 (socket says 172.21.15.150:51224) 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd5bfff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd5c06b270 0x7fdd5c0b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd5b7fe640 1 -- 172.21.15.150:0/459744235 learned_addr learned my addr 172.21.15.150:0/459744235 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd6087a640 1 --2- 172.21.15.150:0/459744235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd5c15da30 0x7fdd5c156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd5b7fe640 1 -- 172.21.15.150:0/459744235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd5c06b270 msgr2=0x7fdd5c0b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd5b7fe640 1 --2- 172.21.15.150:0/459744235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd5c06b270 0x7fdd5c0b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd5b7fe640 1 -- 172.21.15.150:0/459744235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd5c15da30 msgr2=0x7fdd5c156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd5b7fe640 1 --2- 172.21.15.150:0/459744235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd5c15da30 0x7fdd5c156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd5b7fe640 1 -- 172.21.15.150:0/459744235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c152390 con 0x7fdd5c072d60 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd5bfff640 1 --2- 172.21.15.150:0/459744235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd5c06b270 0x7fdd5c0b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd5b7fe640 1 --2- 172.21.15.150:0/459744235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5c072d60 0x7fdd5c151e50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdd50000c00 tx=0x7fdd500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd58ff9640 1 -- 172.21.15.150:0/459744235 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50037040 con 0x7fdd5c072d60 2024-04-25T09:52:51.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd58ff9640 1 -- 172.21.15.150:0/459744235 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50042070 con 0x7fdd5c072d60 2024-04-25T09:52:51.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd58ff9640 1 -- 172.21.15.150:0/459744235 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50030a50 con 0x7fdd5c072d60 2024-04-25T09:52:51.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd622dd640 1 -- 172.21.15.150:0/459744235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c152850 con 0x7fdd5c072d60 2024-04-25T09:52:51.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.740+0000 7fdd622dd640 1 -- 172.21.15.150:0/459744235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c160660 con 0x7fdd5c072d60 2024-04-25T09:52:51.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.741+0000 7fdd58ff9640 1 -- 172.21.15.150:0/459744235 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdd50030bf0 con 0x7fdd5c072d60 2024-04-25T09:52:51.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.741+0000 7fdd58ff9640 1 -- 172.21.15.150:0/459744235 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fdd5003d7f0 con 0x7fdd5c072d60 2024-04-25T09:52:51.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.741+0000 7fdd622dd640 1 -- 172.21.15.150:0/459744235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c074520 con 0x7fdd5c072d60 2024-04-25T09:52:51.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.743+0000 7fdd58ff9640 1 -- 172.21.15.150:0/459744235 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdd5c152390 con 0x7fdd5c072d60 2024-04-25T09:52:51.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.815+0000 7fdd622dd640 1 -- 172.21.15.150:0/459744235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "42562e83-3222-41c7-a37d-cbe67b25096f", "id": 1} v 0) v1 -- 0x7fdd5c0cf750 con 0x7fdd5c072d60 2024-04-25T09:52:51.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.824+0000 7fdd58ff9640 1 -- 172.21.15.150:0/459744235 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "42562e83-3222-41c7-a37d-cbe67b25096f", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdd50047c30 con 0x7fdd5c072d60 2024-04-25T09:52:51.827 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-04-25T09:52:51.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.825+0000 7fdd622dd640 1 -- 172.21.15.150:0/459744235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5c072d60 msgr2=0x7fdd5c151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.825+0000 7fdd622dd640 1 --2- 172.21.15.150:0/459744235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5c072d60 0x7fdd5c151e50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdd50000c00 tx=0x7fdd500046c0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.825+0000 7fdd622dd640 1 -- 172.21.15.150:0/459744235 shutdown_connections 2024-04-25T09:52:51.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.825+0000 7fdd622dd640 1 --2- 172.21.15.150:0/459744235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd5c15da30 0x7fdd5c156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.825+0000 7fdd622dd640 1 --2- 172.21.15.150:0/459744235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd5c072d60 0x7fdd5c151e50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.825+0000 7fdd622dd640 1 --2- 172.21.15.150:0/459744235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd5c06b270 0x7fdd5c0b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.825+0000 7fdd622dd640 1 -- 172.21.15.150:0/459744235 >> 172.21.15.150:0/459744235 conn(0x7fdd5c0b23f0 msgr2=0x7fdd5c15b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:51.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.825+0000 7fdd622dd640 1 -- 172.21.15.150:0/459744235 shutdown_connections 2024-04-25T09:52:51.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.825+0000 7fdd622dd640 1 -- 172.21.15.150:0/459744235 wait complete. 2024-04-25T09:52:51.838 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 69e1d487-9327-46e1-adf8-9e5b5a16a81e 2 2024-04-25T09:52:51.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.934+0000 7f50bfbce640 1 Processor -- start 2024-04-25T09:52:51.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.934+0000 7f50bfbce640 1 -- start start 2024-04-25T09:52:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bfbce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b815da30 0x7f50b80704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bfbce640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f50b8072d60 0x7f50b806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bfbce640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b80731a0 con 0x7f50b806b0a0 2024-04-25T09:52:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bfbce640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b80cd540 con 0x7f50b8072d60 2024-04-25T09:52:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bfbce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b80b3de0 con 0x7f50b815da30 2024-04-25T09:52:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bd96a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b815da30 0x7f50b80704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bd169640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f50b8072d60 0x7f50b806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bd96a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b815da30 0x7f50b80704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:54954/0 (socket says 172.21.15.150:54954) 2024-04-25T09:52:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bd169640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f50b8072d60 0x7f50b806ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51230/0 (socket says 172.21.15.150:51230) 2024-04-25T09:52:51.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bd96a640 1 -- 172.21.15.150:0/955650704 learned_addr learned my addr 172.21.15.150:0/955650704 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T09:52:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bd169640 1 -- 172.21.15.150:0/955650704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b815da30 msgr2=0x7f50b80704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bd169640 1 --2- 172.21.15.150:0/955650704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b815da30 0x7f50b80704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bd169640 1 -- 172.21.15.150:0/955650704 >> v1:172.21.15.150:6789/0 conn(0x7f50b806b0a0 legacy=0x7f50b80d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bd169640 1 -- 172.21.15.150:0/955650704 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b815de10 con 0x7f50b8072d60 2024-04-25T09:52:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.935+0000 7f50bd96a640 1 --2- 172.21.15.150:0/955650704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b815da30 0x7f50b80704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.936+0000 7f50bd169640 1 --2- 172.21.15.150:0/955650704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b8072d60 0x7f50b806ab60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f50a80097a0 tx=0x7f50a802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5850095afbeda67d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.936+0000 7f50bc968640 1 -- 172.21.15.150:0/955650704 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8037070 con 0x7f50b8072d60 2024-04-25T09:52:51.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.936+0000 7f50bc968640 1 -- 172.21.15.150:0/955650704 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a8032440 con 0x7f50b8072d60 2024-04-25T09:52:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.936+0000 7f50bc968640 1 -- 172.21.15.150:0/955650704 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8032810 con 0x7f50b8072d60 2024-04-25T09:52:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.937+0000 7f50bfbce640 1 -- 172.21.15.150:0/955650704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b8072d60 msgr2=0x7f50b806ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.937+0000 7f50bfbce640 1 --2- 172.21.15.150:0/955650704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b8072d60 0x7f50b806ab60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f50a80097a0 tx=0x7f50a802a180 comp rx=0 tx=0).stop 2024-04-25T09:52:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.937+0000 7f50bfbce640 1 -- 172.21.15.150:0/955650704 shutdown_connections 2024-04-25T09:52:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.937+0000 7f50bfbce640 1 --2- 172.21.15.150:0/955650704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b8072d60 0x7f50b806ab60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.937+0000 7f50bfbce640 1 --2- 172.21.15.150:0/955650704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b815da30 0x7f50b80704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.937+0000 7f50bfbce640 1 -- 172.21.15.150:0/955650704 >> 172.21.15.150:0/955650704 conn(0x7f50b80b23f0 msgr2=0x7f50b80b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.937+0000 7f50bfbce640 1 -- 172.21.15.150:0/955650704 shutdown_connections 2024-04-25T09:52:51.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.937+0000 7f50bfbce640 1 -- 172.21.15.150:0/955650704 wait complete. 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.938+0000 7f50bfbce640 1 Processor -- start 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.938+0000 7f50bfbce640 1 -- start start 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bfbce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b806b0a0 0x7f50b80cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bfbce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b8072d60 0x7f50b80c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bfbce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b815da30 0x7f50b81544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bfbce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b80b42a0 con 0x7f50b815da30 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bfbce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b80b3b00 con 0x7f50b806b0a0 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bfbce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b80b4580 con 0x7f50b8072d60 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bd96a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b806b0a0 0x7f50b80cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bd169640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b8072d60 0x7f50b80c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bd96a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b806b0a0 0x7f50b80cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51232/0 (socket says 172.21.15.150:51232) 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bd96a640 1 -- 172.21.15.150:0/1123898067 learned_addr learned my addr 172.21.15.150:0/1123898067 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50be16b640 1 --2- 172.21.15.150:0/1123898067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b815da30 0x7f50b81544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:51.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bd169640 1 -- 172.21.15.150:0/1123898067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b806b0a0 msgr2=0x7f50b80cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bd169640 1 --2- 172.21.15.150:0/1123898067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b806b0a0 0x7f50b80cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bd169640 1 -- 172.21.15.150:0/1123898067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b815da30 msgr2=0x7f50b81544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bd169640 1 --2- 172.21.15.150:0/1123898067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b815da30 0x7f50b81544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bd169640 1 -- 172.21.15.150:0/1123898067 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b815f470 con 0x7f50b8072d60 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bd96a640 1 --2- 172.21.15.150:0/1123898067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b806b0a0 0x7f50b80cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50be16b640 1 --2- 172.21.15.150:0/1123898067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b815da30 0x7f50b81544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50bd169640 1 --2- 172.21.15.150:0/1123898067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b8072d60 0x7f50b80c9b90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f50a8031000 tx=0x7f50a80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.939+0000 7f50ae7fc640 1 -- 172.21.15.150:0/1123898067 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8041030 con 0x7f50b8072d60 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.940+0000 7f50ae7fc640 1 -- 172.21.15.150:0/1123898067 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a8030500 con 0x7f50b8072d60 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.940+0000 7f50ae7fc640 1 -- 172.21.15.150:0/1123898067 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8037040 con 0x7f50b8072d60 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.940+0000 7f50bfbce640 1 -- 172.21.15.150:0/1123898067 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b80ca0d0 con 0x7f50b8072d60 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.940+0000 7f50bfbce640 1 -- 172.21.15.150:0/1123898067 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b80ca930 con 0x7f50b8072d60 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.940+0000 7f50ae7fc640 1 -- 172.21.15.150:0/1123898067 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f50a8044070 con 0x7f50b8072d60 2024-04-25T09:52:51.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.940+0000 7f50ae7fc640 1 -- 172.21.15.150:0/1123898067 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f50a803da20 con 0x7f50b8072d60 2024-04-25T09:52:51.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.941+0000 7f50bfbce640 1 -- 172.21.15.150:0/1123898067 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b8074520 con 0x7f50b8072d60 2024-04-25T09:52:51.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:51.945+0000 7f50ae7fc640 1 -- 172.21.15.150:0/1123898067 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f50a803dd00 con 0x7f50b8072d60 2024-04-25T09:52:52.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.018+0000 7f50bfbce640 1 -- 172.21.15.150:0/1123898067 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd new", "uuid": "69e1d487-9327-46e1-adf8-9e5b5a16a81e", "id": 2} v 0) v1 -- 0x7f50b80b28d0 con 0x7f50b8072d60 2024-04-25T09:52:52.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.023+0000 7f50ae7fc640 1 -- 172.21.15.150:0/1123898067 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "69e1d487-9327-46e1-adf8-9e5b5a16a81e", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f50a803d600 con 0x7f50b8072d60 2024-04-25T09:52:52.026 INFO:teuthology.orchestra.run.smithi150.stdout:2 2024-04-25T09:52:52.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.024+0000 7f50bfbce640 1 -- 172.21.15.150:0/1123898067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b8072d60 msgr2=0x7f50b80c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.024+0000 7f50bfbce640 1 --2- 172.21.15.150:0/1123898067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b8072d60 0x7f50b80c9b90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f50a8031000 tx=0x7f50a80046c0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.024+0000 7f50bfbce640 1 -- 172.21.15.150:0/1123898067 shutdown_connections 2024-04-25T09:52:52.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.024+0000 7f50bfbce640 1 --2- 172.21.15.150:0/1123898067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b815da30 0x7f50b81544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.024+0000 7f50bfbce640 1 --2- 172.21.15.150:0/1123898067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50b8072d60 0x7f50b80c9b90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.024+0000 7f50bfbce640 1 --2- 172.21.15.150:0/1123898067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50b806b0a0 0x7f50b80cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.024+0000 7f50bfbce640 1 -- 172.21.15.150:0/1123898067 >> 172.21.15.150:0/1123898067 conn(0x7f50b80b23f0 msgr2=0x7f50b8154da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:52.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.024+0000 7f50bfbce640 1 -- 172.21.15.150:0/1123898067 shutdown_connections 2024-04-25T09:52:52.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.024+0000 7f50bfbce640 1 -- 172.21.15.150:0/1123898067 wait complete. 2024-04-25T09:52:52.037 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new b3e8830f-b9fe-4afc-8aeb-b4b1417ab368 3 2024-04-25T09:52:52.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.132+0000 7f9a8cd53640 1 Processor -- start 2024-04-25T09:52:52.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.132+0000 7f9a8cd53640 1 -- start start 2024-04-25T09:52:52.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8cd53640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9a88072db0 0x7f9a880cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8cd53640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a880cdd30 con 0x7f9a88159530 2024-04-25T09:52:52.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8cd53640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a880ce010 con 0x7f9a88070930 2024-04-25T09:52:52.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8cd53640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a880cdaf0 con 0x7f9a88072db0 2024-04-25T09:52:52.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a85d9b640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9a88070930 0x7f9a880726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:57798/0 (socket says 172.21.15.150:57798) 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a85d9b640 1 -- 172.21.15.150:0/3035874685 learned_addr learned my addr 172.21.15.150:0/3035874685 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a86d9d640 1 --2- 172.21.15.150:0/3035874685 >> v2:172.21.15.139:3300/0 conn(0x7f9a88072db0 0x7f9a880cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534590127 0 0) 0x7f9a880ce010 con 0x7f9a88070930 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a70003520 con 0x7f9a88070930 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 <== mon.1 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 165038747 0 0) 0x7f9a70003520 con 0x7f9a88070930 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 >> v2:172.21.15.139:3300/0 conn(0x7f9a88072db0 msgr2=0x7f9a880cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8559a640 1 --2- 172.21.15.150:0/3035874685 >> v2:172.21.15.139:3300/0 conn(0x7f9a88072db0 0x7f9a880cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 >> v1:172.21.15.139:6789/0 conn(0x7f9a88159530 legacy=0x7f9a880cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8806fce0 con 0x7f9a88070930 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 <== mon.1 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f9a78001670 con 0x7f9a88070930 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.133+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 >> v1:172.21.15.150:6789/0 conn(0x7f9a88070930 legacy=0x7f9a880726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a8559a640 1 --2- 172.21.15.150:0/3035874685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a700039b0 0x7f9a70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a8559a640 1 --2- 172.21.15.150:0/3035874685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a70004800 0x7f9a70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a8559a640 1 --2- 172.21.15.150:0/3035874685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a700051c0 0x7f9a7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880cdaf0 con 0x7f9a70004800 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7000c060 con 0x7f9a700039b0 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7000c260 con 0x7f9a700051c0 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a8659c640 1 --2- 172.21.15.150:0/3035874685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a700039b0 0x7f9a70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a86d9d640 1 --2- 172.21.15.150:0/3035874685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a700051c0 0x7f9a7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a85d9b640 1 --2- 172.21.15.150:0/3035874685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a70004800 0x7f9a70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a8659c640 1 -- 172.21.15.150:0/3035874685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a700051c0 msgr2=0x7f9a7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a8659c640 1 --2- 172.21.15.150:0/3035874685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a700051c0 0x7f9a7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a8659c640 1 -- 172.21.15.150:0/3035874685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a70004800 msgr2=0x7f9a70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a8659c640 1 --2- 172.21.15.150:0/3035874685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a70004800 0x7f9a70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a8659c640 1 -- 172.21.15.150:0/3035874685 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a70003520 con 0x7f9a700039b0 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.134+0000 7f9a86d9d640 1 -- 172.21.15.150:0/3035874685 reap_dead start 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.135+0000 7f9a8659c640 1 --2- 172.21.15.150:0/3035874685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a700039b0 0x7f9a70003dd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c001750 tx=0x7f9a70005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d30ec60791c222d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.135+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c001370 con 0x7f9a700039b0 2024-04-25T09:52:52.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.135+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c0052d0 con 0x7f9a700039b0 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.135+0000 7f9a8559a640 1 -- 172.21.15.150:0/3035874685 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c02abe0 con 0x7f9a700039b0 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.135+0000 7f9a8cd53640 1 -- 172.21.15.150:0/3035874685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a700039b0 msgr2=0x7f9a70003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.135+0000 7f9a8cd53640 1 --2- 172.21.15.150:0/3035874685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a700039b0 0x7f9a70003dd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c001750 tx=0x7f9a70005830 comp rx=0 tx=0).stop 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.136+0000 7f9a8cd53640 1 -- 172.21.15.150:0/3035874685 shutdown_connections 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.136+0000 7f9a8cd53640 1 --2- 172.21.15.150:0/3035874685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a700039b0 0x7f9a70003dd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.136+0000 7f9a8cd53640 1 -- 172.21.15.150:0/3035874685 >> 172.21.15.150:0/3035874685 conn(0x7f9a880b23f0 msgr2=0x7f9a880b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.136+0000 7f9a8cd53640 1 -- 172.21.15.150:0/3035874685 shutdown_connections 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.136+0000 7f9a8cd53640 1 -- 172.21.15.150:0/3035874685 wait complete. 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.137+0000 7f9a8cd53640 1 Processor -- start 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.137+0000 7f9a8cd53640 1 -- start start 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a8cd53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a700039b0 0x7f9a881f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a8cd53640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a881e4d40 0x7f9a881569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a8cd53640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a880b0cd0 0x7f9a880b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a8cd53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880704e0 con 0x7f9a700039b0 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a8cd53640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8806fa90 con 0x7f9a880b0cd0 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a8cd53640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8815bda0 con 0x7f9a881e4d40 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a86d9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a880b0cd0 0x7f9a880b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a85d9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a881e4d40 0x7f9a881569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a86d9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a880b0cd0 0x7f9a880b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51260/0 (socket says 172.21.15.150:51260) 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a85d9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a881e4d40 0x7f9a881569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:54970/0 (socket says 172.21.15.150:54970) 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a86d9d640 1 -- 172.21.15.150:0/4276617000 learned_addr learned my addr 172.21.15.150:0/4276617000 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a8659c640 1 --2- 172.21.15.150:0/4276617000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a700039b0 0x7f9a881f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a86d9d640 1 -- 172.21.15.150:0/4276617000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a881e4d40 msgr2=0x7f9a881569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a86d9d640 1 --2- 172.21.15.150:0/4276617000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a881e4d40 0x7f9a881569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a86d9d640 1 -- 172.21.15.150:0/4276617000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a700039b0 msgr2=0x7f9a881f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a86d9d640 1 --2- 172.21.15.150:0/4276617000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a700039b0 0x7f9a881f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a86d9d640 1 -- 172.21.15.150:0/4276617000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a881f13f0 con 0x7f9a880b0cd0 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a8659c640 1 --2- 172.21.15.150:0/4276617000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a700039b0 0x7f9a881f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:52.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a85d9b640 1 --2- 172.21.15.150:0/4276617000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a881e4d40 0x7f9a881569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a86d9d640 1 --2- 172.21.15.150:0/4276617000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a880b0cd0 0x7f9a880b1180 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9a880d16a0 tx=0x7f9a8000f1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a76ffd640 1 -- 172.21.15.150:0/4276617000 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8000fce0 con 0x7f9a880b0cd0 2024-04-25T09:52:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a76ffd640 1 -- 172.21.15.150:0/4276617000 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8001d070 con 0x7f9a880b0cd0 2024-04-25T09:52:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a8cd53640 1 -- 172.21.15.150:0/4276617000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a88157340 con 0x7f9a880b0cd0 2024-04-25T09:52:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a76ffd640 1 -- 172.21.15.150:0/4276617000 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80015670 con 0x7f9a880b0cd0 2024-04-25T09:52:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.138+0000 7f9a8cd53640 1 -- 172.21.15.150:0/4276617000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a88157900 con 0x7f9a880b0cd0 2024-04-25T09:52:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.139+0000 7f9a76ffd640 1 -- 172.21.15.150:0/4276617000 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9a80015810 con 0x7f9a880b0cd0 2024-04-25T09:52:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.139+0000 7f9a76ffd640 1 -- 172.21.15.150:0/4276617000 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f9a800213c0 con 0x7f9a880b0cd0 2024-04-25T09:52:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.139+0000 7f9a8cd53640 1 -- 172.21.15.150:0/4276617000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a88074520 con 0x7f9a880b0cd0 2024-04-25T09:52:52.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.143+0000 7f9a76ffd640 1 -- 172.21.15.150:0/4276617000 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9a80010040 con 0x7f9a880b0cd0 2024-04-25T09:52:52.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.215+0000 7f9a8cd53640 1 -- 172.21.15.150:0/4276617000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b3e8830f-b9fe-4afc-8aeb-b4b1417ab368", "id": 3} v 0) v1 -- 0x7f9a88072480 con 0x7f9a880b0cd0 2024-04-25T09:52:52.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.220+0000 7f9a76ffd640 1 -- 172.21.15.150:0/4276617000 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b3e8830f-b9fe-4afc-8aeb-b4b1417ab368", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9a80015d10 con 0x7f9a880b0cd0 2024-04-25T09:52:52.222 INFO:teuthology.orchestra.run.smithi150.stdout:3 2024-04-25T09:52:52.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.221+0000 7f9a8cd53640 1 -- 172.21.15.150:0/4276617000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a880b0cd0 msgr2=0x7f9a880b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.221+0000 7f9a8cd53640 1 --2- 172.21.15.150:0/4276617000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a880b0cd0 0x7f9a880b1180 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9a880d16a0 tx=0x7f9a8000f1b0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.221+0000 7f9a8cd53640 1 -- 172.21.15.150:0/4276617000 shutdown_connections 2024-04-25T09:52:52.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.221+0000 7f9a8cd53640 1 --2- 172.21.15.150:0/4276617000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a880b0cd0 0x7f9a880b1180 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.221+0000 7f9a8cd53640 1 --2- 172.21.15.150:0/4276617000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a881e4d40 0x7f9a881569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.221+0000 7f9a8cd53640 1 --2- 172.21.15.150:0/4276617000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a700039b0 0x7f9a881f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.221+0000 7f9a8cd53640 1 -- 172.21.15.150:0/4276617000 >> 172.21.15.150:0/4276617000 conn(0x7f9a880b23f0 msgr2=0x7f9a8815cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:52.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.221+0000 7f9a8cd53640 1 -- 172.21.15.150:0/4276617000 shutdown_connections 2024-04-25T09:52:52.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.221+0000 7f9a8cd53640 1 -- 172.21.15.150:0/4276617000 wait complete. 2024-04-25T09:52:52.230 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:52.229+0000 7f26c3819040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T09:52:52.233 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 7bc3028f-8684-4634-91f8-fab6676f7c75 4 2024-04-25T09:52:52.316 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:52.314+0000 7f26c3819040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T09:52:52.327 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:52.326+0000 7f7b52f30040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T09:52:52.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.329+0000 7f28c726f640 1 Processor -- start 2024-04-25T09:52:52.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.329+0000 7f28c726f640 1 -- start start 2024-04-25T09:52:52.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.330+0000 7f28c726f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f28c00719d0 0x7f28c00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.330+0000 7f28c726f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c00b1100 con 0x7f28c00712f0 2024-04-25T09:52:52.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.330+0000 7f28c726f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c00ce2c0 con 0x7f28c0145250 2024-04-25T09:52:52.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.330+0000 7f28c726f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c00ceed0 con 0x7f28c00719d0 2024-04-25T09:52:52.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.330+0000 7f28c500b640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f28c0145250 0x7f28c0070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:57814/0 (socket says 172.21.15.150:57814) 2024-04-25T09:52:52.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.330+0000 7f28c500b640 1 -- 172.21.15.150:0/1635518882 learned_addr learned my addr 172.21.15.150:0/1635518882 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-25T09:52:52.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.330+0000 7f28c580c640 1 --2- 172.21.15.150:0/1635518882 >> v2:172.21.15.139:3300/0 conn(0x7f28c00719d0 0x7f28c00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.330+0000 7f28b7fff640 1 -- 172.21.15.150:0/1635518882 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987885229 0 0) 0x7f28c00ce2c0 con 0x7f28c0145250 2024-04-25T09:52:52.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.330+0000 7f28b7fff640 1 -- 172.21.15.150:0/1635518882 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28ac003520 con 0x7f28c0145250 2024-04-25T09:52:52.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.330+0000 7f28c580c640 1 -- 172.21.15.150:0/1635518882 >> v1:172.21.15.150:6789/0 conn(0x7f28c0145250 legacy=0x7f28c0070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.330+0000 7f28c580c640 1 -- 172.21.15.150:0/1635518882 >> v1:172.21.15.139:6789/0 conn(0x7f28c00712f0 legacy=0x7f28c00b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.330+0000 7f28c580c640 1 -- 172.21.15.150:0/1635518882 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c00ce5a0 con 0x7f28c00719d0 2024-04-25T09:52:52.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.331+0000 7f28c580c640 1 --2- 172.21.15.150:0/1635518882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c00719d0 0x7f28c00b4d90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009d90 tx=0x7f28b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93ce8c0b6b8d5fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:52.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.331+0000 7f28b7fff640 1 -- 172.21.15.150:0/1635518882 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8007740 con 0x7f28c00719d0 2024-04-25T09:52:52.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.331+0000 7f28b7fff640 1 -- 172.21.15.150:0/1635518882 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b80078e0 con 0x7f28c00719d0 2024-04-25T09:52:52.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.331+0000 7f28b7fff640 1 -- 172.21.15.150:0/1635518882 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8007cb0 con 0x7f28c00719d0 2024-04-25T09:52:52.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.332+0000 7f28c726f640 1 -- 172.21.15.150:0/1635518882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c00719d0 msgr2=0x7f28c00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.332+0000 7f28c726f640 1 --2- 172.21.15.150:0/1635518882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c00719d0 0x7f28c00b4d90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009d90 tx=0x7f28b802a660 comp rx=0 tx=0).stop 2024-04-25T09:52:52.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.332+0000 7f28c726f640 1 -- 172.21.15.150:0/1635518882 shutdown_connections 2024-04-25T09:52:52.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.332+0000 7f28c726f640 1 --2- 172.21.15.150:0/1635518882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c00719d0 0x7f28c00b4d90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.332+0000 7f28c726f640 1 -- 172.21.15.150:0/1635518882 >> 172.21.15.150:0/1635518882 conn(0x7f28c006f460 msgr2=0x7f28c006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:52.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.332+0000 7f28c726f640 1 -- 172.21.15.150:0/1635518882 shutdown_connections 2024-04-25T09:52:52.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.332+0000 7f28c726f640 1 -- 172.21.15.150:0/1635518882 wait complete. 2024-04-25T09:52:52.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.333+0000 7f28c726f640 1 Processor -- start 2024-04-25T09:52:52.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.333+0000 7f28c726f640 1 -- start start 2024-04-25T09:52:52.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c726f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28c00712f0 0x7f28c015bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c726f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00719d0 0x7f28c01e6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c726f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c0145250 0x7f28c013ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c726f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c0145630 con 0x7f28c0145250 2024-04-25T09:52:52.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c726f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00ceb60 con 0x7f28c00719d0 2024-04-25T09:52:52.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c726f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00d11a0 con 0x7f28c00712f0 2024-04-25T09:52:52.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c480a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00719d0 0x7f28c01e6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c480a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00719d0 0x7f28c01e6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51264/0 (socket says 172.21.15.150:51264) 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c500b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28c00712f0 0x7f28c015bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c480a640 1 -- 172.21.15.150:0/978373585 learned_addr learned my addr 172.21.15.150:0/978373585 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c580c640 1 --2- 172.21.15.150:0/978373585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c0145250 0x7f28c013ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c480a640 1 -- 172.21.15.150:0/978373585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28c00712f0 msgr2=0x7f28c015bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c480a640 1 --2- 172.21.15.150:0/978373585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28c00712f0 0x7f28c015bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c480a640 1 -- 172.21.15.150:0/978373585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c0145250 msgr2=0x7f28c013ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c480a640 1 --2- 172.21.15.150:0/978373585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c0145250 0x7f28c013ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c480a640 1 -- 172.21.15.150:0/978373585 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c01e4120 con 0x7f28c00719d0 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c500b640 1 --2- 172.21.15.150:0/978373585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28c00712f0 0x7f28c015bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c480a640 1 --2- 172.21.15.150:0/978373585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00719d0 0x7f28c01e6640 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f28b000c400 tx=0x7f28b000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28b5ffb640 1 -- 172.21.15.150:0/978373585 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b00052d0 con 0x7f28c00719d0 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28b5ffb640 1 -- 172.21.15.150:0/978373585 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b0005490 con 0x7f28c00719d0 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28c726f640 1 -- 172.21.15.150:0/978373585 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c01e4400 con 0x7f28c00719d0 2024-04-25T09:52:52.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.334+0000 7f28b5ffb640 1 -- 172.21.15.150:0/978373585 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0007260 con 0x7f28c00719d0 2024-04-25T09:52:52.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.335+0000 7f28b5ffb640 1 -- 172.21.15.150:0/978373585 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f28b0007400 con 0x7f28c00719d0 2024-04-25T09:52:52.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.335+0000 7f28c726f640 1 -- 172.21.15.150:0/978373585 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c01e49c0 con 0x7f28c00719d0 2024-04-25T09:52:52.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.335+0000 7f28b5ffb640 1 -- 172.21.15.150:0/978373585 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f28b0006ce0 con 0x7f28c00719d0 2024-04-25T09:52:52.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.336+0000 7f28c726f640 1 -- 172.21.15.150:0/978373585 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c01e4bc0 con 0x7f28c00719d0 2024-04-25T09:52:52.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.339+0000 7f28b5ffb640 1 -- 172.21.15.150:0/978373585 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f28c01e4bc0 con 0x7f28c00719d0 2024-04-25T09:52:52.358 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:52.357+0000 7f26c3819040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T09:52:52.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.411+0000 7f28c726f640 1 -- 172.21.15.150:0/978373585 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7bc3028f-8684-4634-91f8-fab6676f7c75", "id": 4} v 0) v1 -- 0x7f28c0074520 con 0x7f28c00719d0 2024-04-25T09:52:52.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.417+0000 7f28b5ffb640 1 -- 172.21.15.150:0/978373585 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7bc3028f-8684-4634-91f8-fab6676f7c75", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f28b00035f0 con 0x7f28c00719d0 2024-04-25T09:52:52.419 INFO:teuthology.orchestra.run.smithi150.stdout:4 2024-04-25T09:52:52.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.418+0000 7f28c726f640 1 -- 172.21.15.150:0/978373585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00719d0 msgr2=0x7f28c01e6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.418+0000 7f28c726f640 1 --2- 172.21.15.150:0/978373585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00719d0 0x7f28c01e6640 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f28b000c400 tx=0x7f28b000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.418+0000 7f28c726f640 1 -- 172.21.15.150:0/978373585 shutdown_connections 2024-04-25T09:52:52.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.418+0000 7f28c726f640 1 --2- 172.21.15.150:0/978373585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c0145250 0x7f28c013ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.418+0000 7f28c726f640 1 --2- 172.21.15.150:0/978373585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28c00719d0 0x7f28c01e6640 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.418+0000 7f28c726f640 1 --2- 172.21.15.150:0/978373585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28c00712f0 0x7f28c015bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.418+0000 7f28c726f640 1 -- 172.21.15.150:0/978373585 >> 172.21.15.150:0/978373585 conn(0x7f28c006f460 msgr2=0x7f28c013f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:52.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.418+0000 7f28c726f640 1 -- 172.21.15.150:0/978373585 shutdown_connections 2024-04-25T09:52:52.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.418+0000 7f28c726f640 1 -- 172.21.15.150:0/978373585 wait complete. 2024-04-25T09:52:52.421 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:52.420+0000 7f7b52f30040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T09:52:52.431 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 19cf45c1-b7da-4aa8-9033-76a51df3ee24 5 2024-04-25T09:52:52.467 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:52.466+0000 7f7b52f30040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T09:52:52.520 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:52.518+0000 7f26c3819040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T09:52:52.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.526+0000 7f31dc22a640 1 Processor -- start 2024-04-25T09:52:52.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.527+0000 7f31dc22a640 1 -- start start 2024-04-25T09:52:52.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.527+0000 7f31dc22a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f31d415d9e0 0x7f31d40668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.527+0000 7f31dc22a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40b4960 0x7f31d406bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.527+0000 7f31dc22a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f31d41616d0 0x7f31d4161b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.527+0000 7f31dc22a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d4162080 con 0x7f31d415d9e0 2024-04-25T09:52:52.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.527+0000 7f31dc22a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d40b34f0 con 0x7f31d41616d0 2024-04-25T09:52:52.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.527+0000 7f31dc22a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d406c490 con 0x7f31d40b4960 2024-04-25T09:52:52.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.528+0000 7f31da7c7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f31d41616d0 0x7f31d4161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.528+0000 7f31d97c5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40b4960 0x7f31d406bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.528+0000 7f31da7c7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f31d41616d0 0x7f31d4161b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51270/0 (socket says 172.21.15.150:51270) 2024-04-25T09:52:52.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.528+0000 7f31da7c7640 1 -- 172.21.15.150:0/3866625369 learned_addr learned my addr 172.21.15.150:0/3866625369 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T09:52:52.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.528+0000 7f31d9fc6640 1 --2- 172.21.15.150:0/3866625369 >> v2:172.21.15.139:3300/0 conn(0x7f31d415d9e0 0x7f31d40668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.528+0000 7f31d97c5640 1 -- 172.21.15.150:0/3866625369 >> v2:172.21.15.150:3300/0 conn(0x7f31d41616d0 msgr2=0x7f31d4161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:52.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.528+0000 7f31d97c5640 1 --2- 172.21.15.150:0/3866625369 >> v2:172.21.15.150:3300/0 conn(0x7f31d41616d0 0x7f31d4161b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.528+0000 7f31d97c5640 1 -- 172.21.15.150:0/3866625369 >> v2:172.21.15.139:3300/0 conn(0x7f31d415d9e0 msgr2=0x7f31d40668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:52.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.528+0000 7f31d97c5640 1 --2- 172.21.15.150:0/3866625369 >> v2:172.21.15.139:3300/0 conn(0x7f31d415d9e0 0x7f31d40668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.528+0000 7f31d97c5640 1 -- 172.21.15.150:0/3866625369 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d40b4ed0 con 0x7f31d40b4960 2024-04-25T09:52:52.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.528+0000 7f31da7c7640 1 --2- 172.21.15.150:0/3866625369 >> v2:172.21.15.150:3300/0 conn(0x7f31d41616d0 0x7f31d4161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:52.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.529+0000 7f31d97c5640 1 --2- 172.21.15.150:0/3866625369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40b4960 0x7f31d406bf50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f31d406ff50 tx=0x7f31c402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d68471bb487f3f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:52.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.529+0000 7f31d8fc4640 1 -- 172.21.15.150:0/3866625369 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4037070 con 0x7f31d40b4960 2024-04-25T09:52:52.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.529+0000 7f31d8fc4640 1 -- 172.21.15.150:0/3866625369 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c402ae50 con 0x7f31d40b4960 2024-04-25T09:52:52.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.529+0000 7f31d8fc4640 1 -- 172.21.15.150:0/3866625369 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4032660 con 0x7f31d40b4960 2024-04-25T09:52:52.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.529+0000 7f31dc22a640 1 -- 172.21.15.150:0/3866625369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40b4960 msgr2=0x7f31d406bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.529+0000 7f31dc22a640 1 --2- 172.21.15.150:0/3866625369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40b4960 0x7f31d406bf50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f31d406ff50 tx=0x7f31c402a100 comp rx=0 tx=0).stop 2024-04-25T09:52:52.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.529+0000 7f31dc22a640 1 -- 172.21.15.150:0/3866625369 shutdown_connections 2024-04-25T09:52:52.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.529+0000 7f31dc22a640 1 --2- 172.21.15.150:0/3866625369 >> v2:172.21.15.150:3300/0 conn(0x7f31d41616d0 0x7f31d4161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.529+0000 7f31dc22a640 1 --2- 172.21.15.150:0/3866625369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d40b4960 0x7f31d406bf50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.529+0000 7f31dc22a640 1 --2- 172.21.15.150:0/3866625369 >> v2:172.21.15.139:3300/0 conn(0x7f31d415d9e0 0x7f31d40668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.529+0000 7f31dc22a640 1 -- 172.21.15.150:0/3866625369 >> 172.21.15.150:0/3866625369 conn(0x7f31d40cc820 msgr2=0x7f31d40ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:52.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.530+0000 7f31dc22a640 1 -- 172.21.15.150:0/3866625369 shutdown_connections 2024-04-25T09:52:52.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.530+0000 7f31dc22a640 1 -- 172.21.15.150:0/3866625369 wait complete. 2024-04-25T09:52:52.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.531+0000 7f31dc22a640 1 Processor -- start 2024-04-25T09:52:52.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.531+0000 7f31dc22a640 1 -- start start 2024-04-25T09:52:52.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.531+0000 7f31dc22a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31d40b4960 0x7f31d414c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.531+0000 7f31dc22a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d415d9e0 0x7f31d415cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.531+0000 7f31dc22a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d41616d0 0x7f31d4147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.531+0000 7f31dc22a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4162430 con 0x7f31d40b4960 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.531+0000 7f31dc22a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d40b3210 con 0x7f31d41616d0 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.531+0000 7f31dc22a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d415ddc0 con 0x7f31d415d9e0 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31d97c5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d415d9e0 0x7f31d415cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31da7c7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d41616d0 0x7f31d4147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31da7c7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d41616d0 0x7f31d4147540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51282/0 (socket says 172.21.15.150:51282) 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31d97c5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d415d9e0 0x7f31d415cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:54986/0 (socket says 172.21.15.150:54986) 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31da7c7640 1 -- 172.21.15.150:0/1039206585 learned_addr learned my addr 172.21.15.150:0/1039206585 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31d9fc6640 1 --2- 172.21.15.150:0/1039206585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31d40b4960 0x7f31d414c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31d97c5640 1 -- 172.21.15.150:0/1039206585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d41616d0 msgr2=0x7f31d4147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31d97c5640 1 --2- 172.21.15.150:0/1039206585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d41616d0 0x7f31d4147540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31d97c5640 1 -- 172.21.15.150:0/1039206585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31d40b4960 msgr2=0x7f31d414c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31d97c5640 1 --2- 172.21.15.150:0/1039206585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31d40b4960 0x7f31d414c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31d97c5640 1 -- 172.21.15.150:0/1039206585 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d4146780 con 0x7f31d415d9e0 2024-04-25T09:52:52.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31da7c7640 1 --2- 172.21.15.150:0/1039206585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d41616d0 0x7f31d4147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:52.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31d97c5640 1 --2- 172.21.15.150:0/1039206585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d415d9e0 0x7f31d415cae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f31c4031000 tx=0x7f31c40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:52.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31ca7fc640 1 -- 172.21.15.150:0/1039206585 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4041070 con 0x7f31d415d9e0 2024-04-25T09:52:52.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31ca7fc640 1 -- 172.21.15.150:0/1039206585 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c403c460 con 0x7f31d415d9e0 2024-04-25T09:52:52.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31dc22a640 1 -- 172.21.15.150:0/1039206585 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d4146a60 con 0x7f31d415d9e0 2024-04-25T09:52:52.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31ca7fc640 1 -- 172.21.15.150:0/1039206585 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4037040 con 0x7f31d415d9e0 2024-04-25T09:52:52.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31dc22a640 1 -- 172.21.15.150:0/1039206585 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d4147020 con 0x7f31d415d9e0 2024-04-25T09:52:52.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31ca7fc640 1 -- 172.21.15.150:0/1039206585 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f31c403a070 con 0x7f31d415d9e0 2024-04-25T09:52:52.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.532+0000 7f31ca7fc640 1 -- 172.21.15.150:0/1039206585 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f31c4045b10 con 0x7f31d415d9e0 2024-04-25T09:52:52.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.533+0000 7f31dc22a640 1 -- 172.21.15.150:0/1039206585 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d4074520 con 0x7f31d415d9e0 2024-04-25T09:52:52.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.535+0000 7f31ca7fc640 1 -- 172.21.15.150:0/1039206585 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f31c4045590 con 0x7f31d415d9e0 2024-04-25T09:52:52.584 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:52.582+0000 7f26c3819040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T09:52:52.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.607+0000 7f31dc22a640 1 -- 172.21.15.150:0/1039206585 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd new", "uuid": "19cf45c1-b7da-4aa8-9033-76a51df3ee24", "id": 5} v 0) v1 -- 0x7f31d40cd6c0 con 0x7f31d415d9e0 2024-04-25T09:52:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.613+0000 7f31ca7fc640 1 -- 172.21.15.150:0/1039206585 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "19cf45c1-b7da-4aa8-9033-76a51df3ee24", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f31c4045770 con 0x7f31d415d9e0 2024-04-25T09:52:52.615 INFO:teuthology.orchestra.run.smithi150.stdout:5 2024-04-25T09:52:52.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.614+0000 7f31dc22a640 1 -- 172.21.15.150:0/1039206585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d415d9e0 msgr2=0x7f31d415cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.614+0000 7f31dc22a640 1 --2- 172.21.15.150:0/1039206585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d415d9e0 0x7f31d415cae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f31c4031000 tx=0x7f31c40046c0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.614+0000 7f31dc22a640 1 -- 172.21.15.150:0/1039206585 shutdown_connections 2024-04-25T09:52:52.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.614+0000 7f31dc22a640 1 --2- 172.21.15.150:0/1039206585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31d41616d0 0x7f31d4147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.614+0000 7f31dc22a640 1 --2- 172.21.15.150:0/1039206585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31d415d9e0 0x7f31d415cae0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.614+0000 7f31dc22a640 1 --2- 172.21.15.150:0/1039206585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31d40b4960 0x7f31d414c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.614+0000 7f31dc22a640 1 -- 172.21.15.150:0/1039206585 >> 172.21.15.150:0/1039206585 conn(0x7f31d40cc820 msgr2=0x7f31d40aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:52.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.614+0000 7f31dc22a640 1 -- 172.21.15.150:0/1039206585 shutdown_connections 2024-04-25T09:52:52.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.614+0000 7f31dc22a640 1 -- 172.21.15.150:0/1039206585 wait complete. 2024-04-25T09:52:52.626 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 9aaf5762-215d-4714-ace8-980aeecb6330 6 2024-04-25T09:52:52.636 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:52.634+0000 7f26c3819040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T09:52:52.643 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:52.641+0000 7f7b52f30040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T09:52:52.703 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:52.701+0000 7f7b52f30040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T09:52:52.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.738+0000 7f1951412640 1 Processor -- start 2024-04-25T09:52:52.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.738+0000 7f1951412640 1 -- start start 2024-04-25T09:52:52.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.739+0000 7f1951412640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f194c0b5620 0x7f194c0b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.739+0000 7f1951412640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c0b5a20 con 0x7f194c15a870 2024-04-25T09:52:52.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.739+0000 7f1951412640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c0d0d40 con 0x7f194c070970 2024-04-25T09:52:52.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.739+0000 7f1951412640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c0d1020 con 0x7f194c0b5620 2024-04-25T09:52:52.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.739+0000 7f194a7fc640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f194c070970 0x7f194c0ce610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:57824/0 (socket says 172.21.15.150:57824) 2024-04-25T09:52:52.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.739+0000 7f194a7fc640 1 -- 172.21.15.150:0/2526929761 learned_addr learned my addr 172.21.15.150:0/2526929761 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-04-25T09:52:52.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.739+0000 7f194b7fe640 1 --2- 172.21.15.150:0/2526929761 >> v2:172.21.15.139:3300/0 conn(0x7f194c0b5620 0x7f194c0b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.739+0000 7f1949ffb640 1 -- 172.21.15.150:0/2526929761 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631084559 0 0) 0x7f194c0d0d40 con 0x7f194c070970 2024-04-25T09:52:52.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.739+0000 7f1949ffb640 1 -- 172.21.15.150:0/2526929761 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1934003520 con 0x7f194c070970 2024-04-25T09:52:52.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.740+0000 7f194b7fe640 1 -- 172.21.15.150:0/2526929761 >> v1:172.21.15.150:6789/0 conn(0x7f194c070970 legacy=0x7f194c0ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.740+0000 7f194b7fe640 1 -- 172.21.15.150:0/2526929761 >> v1:172.21.15.139:6789/0 conn(0x7f194c15a870 legacy=0x7f194c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.740+0000 7f1949ffb640 1 -- 172.21.15.150:0/2526929761 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985264918 0 0) 0x7f194c0b5a20 con 0x7f194c15a870 2024-04-25T09:52:52.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.740+0000 7f194b7fe640 1 -- 172.21.15.150:0/2526929761 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0b4b40 con 0x7f194c0b5620 2024-04-25T09:52:52.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.740+0000 7f194b7fe640 1 --2- 172.21.15.150:0/2526929761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c0b5620 0x7f194c0b4190 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1944009d90 tx=0x7f194402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65547f0e8772d5e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:52.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.740+0000 7f1949ffb640 1 -- 172.21.15.150:0/2526929761 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944007740 con 0x7f194c0b5620 2024-04-25T09:52:52.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.740+0000 7f1949ffb640 1 -- 172.21.15.150:0/2526929761 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19440078e0 con 0x7f194c0b5620 2024-04-25T09:52:52.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.740+0000 7f1949ffb640 1 -- 172.21.15.150:0/2526929761 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944007cb0 con 0x7f194c0b5620 2024-04-25T09:52:52.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.741+0000 7f1951412640 1 -- 172.21.15.150:0/2526929761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c0b5620 msgr2=0x7f194c0b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.741+0000 7f1951412640 1 --2- 172.21.15.150:0/2526929761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c0b5620 0x7f194c0b4190 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1944009d90 tx=0x7f194402a660 comp rx=0 tx=0).stop 2024-04-25T09:52:52.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.741+0000 7f1951412640 1 -- 172.21.15.150:0/2526929761 shutdown_connections 2024-04-25T09:52:52.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.741+0000 7f1951412640 1 --2- 172.21.15.150:0/2526929761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c0b5620 0x7f194c0b4190 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.741+0000 7f1951412640 1 -- 172.21.15.150:0/2526929761 >> 172.21.15.150:0/2526929761 conn(0x7f194c0b23f0 msgr2=0x7f194c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:52.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.741+0000 7f1951412640 1 -- 172.21.15.150:0/2526929761 shutdown_connections 2024-04-25T09:52:52.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.741+0000 7f1951412640 1 -- 172.21.15.150:0/2526929761 wait complete. 2024-04-25T09:52:52.749 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.742+0000 7f1951412640 1 Processor -- start 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.743+0000 7f1951412640 1 -- start start 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.743+0000 7f1951412640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c070970 0x7f194c14cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.743+0000 7f1951412640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f194c0b5620 0x7f194c159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.743+0000 7f1951412640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f194c15a870 0x7f194c159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.743+0000 7f1951412640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0d05f0 con 0x7f194c070970 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.743+0000 7f1951412640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0b5190 con 0x7f194c15a870 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.743+0000 7f1951412640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0d0770 con 0x7f194c0b5620 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.743+0000 7f194a7fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f194c0b5620 0x7f194c159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.743+0000 7f194b7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f194c15a870 0x7f194c159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.743+0000 7f194a7fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f194c0b5620 0x7f194c159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:54996/0 (socket says 172.21.15.150:54996) 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.743+0000 7f194a7fc640 1 -- 172.21.15.150:0/3679525684 learned_addr learned my addr 172.21.15.150:0/3679525684 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.743+0000 7f194affd640 1 --2- 172.21.15.150:0/3679525684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c070970 0x7f194c14cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f194b7fe640 1 -- 172.21.15.150:0/3679525684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f194c0b5620 msgr2=0x7f194c159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f194b7fe640 1 --2- 172.21.15.150:0/3679525684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f194c0b5620 0x7f194c159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f194b7fe640 1 -- 172.21.15.150:0/3679525684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c070970 msgr2=0x7f194c14cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f194b7fe640 1 --2- 172.21.15.150:0/3679525684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c070970 0x7f194c14cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f194b7fe640 1 -- 172.21.15.150:0/3679525684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0c9ba0 con 0x7f194c15a870 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f194a7fc640 1 --2- 172.21.15.150:0/3679525684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f194c0b5620 0x7f194c159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f194b7fe640 1 --2- 172.21.15.150:0/3679525684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f194c15a870 0x7f194c159b50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1944031000 tx=0x7f1944004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f19337fe640 1 -- 172.21.15.150:0/3679525684 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944040070 con 0x7f194c15a870 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f19337fe640 1 -- 172.21.15.150:0/3679525684 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1944031bc0 con 0x7f194c15a870 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f1951412640 1 -- 172.21.15.150:0/3679525684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c0c9e80 con 0x7f194c15a870 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f19337fe640 1 -- 172.21.15.150:0/3679525684 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944009450 con 0x7f194c15a870 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f1951412640 1 -- 172.21.15.150:0/3679525684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c0ca440 con 0x7f194c15a870 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f19337fe640 1 -- 172.21.15.150:0/3679525684 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1944039040 con 0x7f194c15a870 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.744+0000 7f19337fe640 1 -- 172.21.15.150:0/3679525684 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f1944044b80 con 0x7f194c15a870 2024-04-25T09:52:52.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.745+0000 7f1951412640 1 -- 172.21.15.150:0/3679525684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c074520 con 0x7f194c15a870 2024-04-25T09:52:52.752 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:52.744+0000 7f26c3819040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:52:52.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.747+0000 7f19337fe640 1 -- 172.21.15.150:0/3679525684 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1944044dd0 con 0x7f194c15a870 2024-04-25T09:52:52.759 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:52.757+0000 7f7b52f30040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T09:52:52.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.820+0000 7f1951412640 1 -- 172.21.15.150:0/3679525684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9aaf5762-215d-4714-ace8-980aeecb6330", "id": 6} v 0) v1 -- 0x7f194c14d8e0 con 0x7f194c15a870 2024-04-25T09:52:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.825+0000 7f19337fe640 1 -- 172.21.15.150:0/3679525684 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9aaf5762-215d-4714-ace8-980aeecb6330", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1944044550 con 0x7f194c15a870 2024-04-25T09:52:52.827 INFO:teuthology.orchestra.run.smithi150.stdout:6 2024-04-25T09:52:52.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.826+0000 7f1951412640 1 -- 172.21.15.150:0/3679525684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f194c15a870 msgr2=0x7f194c159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.826+0000 7f1951412640 1 --2- 172.21.15.150:0/3679525684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f194c15a870 0x7f194c159b50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1944031000 tx=0x7f1944004a40 comp rx=0 tx=0).stop 2024-04-25T09:52:52.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.826+0000 7f1951412640 1 -- 172.21.15.150:0/3679525684 shutdown_connections 2024-04-25T09:52:52.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.826+0000 7f1951412640 1 --2- 172.21.15.150:0/3679525684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f194c15a870 0x7f194c159b50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.826+0000 7f1951412640 1 --2- 172.21.15.150:0/3679525684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f194c0b5620 0x7f194c159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.826+0000 7f1951412640 1 --2- 172.21.15.150:0/3679525684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f194c070970 0x7f194c14cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.826+0000 7f1951412640 1 -- 172.21.15.150:0/3679525684 >> 172.21.15.150:0/3679525684 conn(0x7f194c0b23f0 msgr2=0x7f194c0cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.826+0000 7f1951412640 1 -- 172.21.15.150:0/3679525684 shutdown_connections 2024-04-25T09:52:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.826+0000 7f1951412640 1 -- 172.21.15.150:0/3679525684 wait complete. 2024-04-25T09:52:52.839 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 62755a3e-bdec-4e96-8e7a-a608b17acac4 7 2024-04-25T09:52:52.877 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:52.875+0000 7f7b52f30040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:52:52.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.934+0000 7f63e9513640 1 Processor -- start 2024-04-25T09:52:52.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.935+0000 7f63e9513640 1 -- start start 2024-04-25T09:52:52.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.935+0000 7f63e9513640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f63e415dbf0 0x7f63e41617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.935+0000 7f63e9513640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e40b9370 0x7f63e40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.935+0000 7f63e9513640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f63e40cec80 0x7f63e406c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.935+0000 7f63e9513640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e40d1ee0 con 0x7f63e415dbf0 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.935+0000 7f63e9513640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e4161dc0 con 0x7f63e40cec80 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.935+0000 7f63e9513640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e41620a0 con 0x7f63e40b9370 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.936+0000 7f63e37fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f63e40cec80 0x7f63e406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.936+0000 7f63e27fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e40b9370 0x7f63e40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.936+0000 7f63e37fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f63e40cec80 0x7f63e406c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51306/0 (socket says 172.21.15.150:51306) 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.936+0000 7f63e27fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e40b9370 0x7f63e40ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:55004/0 (socket says 172.21.15.150:55004) 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.936+0000 7f63e37fe640 1 -- 172.21.15.150:0/5520968 learned_addr learned my addr 172.21.15.150:0/5520968 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.936+0000 7f63e2ffd640 1 --2- 172.21.15.150:0/5520968 >> v2:172.21.15.139:3300/0 conn(0x7f63e415dbf0 0x7f63e41617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.936+0000 7f63e37fe640 1 -- 172.21.15.150:0/5520968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e40b9370 msgr2=0x7f63e40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.936+0000 7f63e37fe640 1 --2- 172.21.15.150:0/5520968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e40b9370 0x7f63e40ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.936+0000 7f63e37fe640 1 -- 172.21.15.150:0/5520968 >> v2:172.21.15.139:3300/0 conn(0x7f63e415dbf0 msgr2=0x7f63e41617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.936+0000 7f63e37fe640 1 --2- 172.21.15.150:0/5520968 >> v2:172.21.15.139:3300/0 conn(0x7f63e415dbf0 0x7f63e41617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.936+0000 7f63e37fe640 1 -- 172.21.15.150:0/5520968 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e4162380 con 0x7f63e40cec80 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.936+0000 7f63e27fc640 1 --2- 172.21.15.150:0/5520968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e40b9370 0x7f63e40ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:52.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e37fe640 1 --2- 172.21.15.150:0/5520968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e40cec80 0x7f63e406c210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63e408c170 tx=0x7f63dc02a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd677de035581a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:52.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e1ffb640 1 -- 172.21.15.150:0/5520968 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc036070 con 0x7f63e40cec80 2024-04-25T09:52:52.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e1ffb640 1 -- 172.21.15.150:0/5520968 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc031600 con 0x7f63e40cec80 2024-04-25T09:52:52.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e1ffb640 1 -- 172.21.15.150:0/5520968 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0319d0 con 0x7f63e40cec80 2024-04-25T09:52:52.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e9513640 1 -- 172.21.15.150:0/5520968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e40cec80 msgr2=0x7f63e406c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e9513640 1 --2- 172.21.15.150:0/5520968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e40cec80 0x7f63e406c210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63e408c170 tx=0x7f63dc02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e9513640 1 -- 172.21.15.150:0/5520968 shutdown_connections 2024-04-25T09:52:52.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e9513640 1 --2- 172.21.15.150:0/5520968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e40cec80 0x7f63e406c210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e9513640 1 --2- 172.21.15.150:0/5520968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e40b9370 0x7f63e40ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e9513640 1 --2- 172.21.15.150:0/5520968 >> v2:172.21.15.139:3300/0 conn(0x7f63e415dbf0 0x7f63e41617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e9513640 1 -- 172.21.15.150:0/5520968 >> 172.21.15.150:0/5520968 conn(0x7f63e40b5480 msgr2=0x7f63e40b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:52.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e9513640 1 -- 172.21.15.150:0/5520968 shutdown_connections 2024-04-25T09:52:52.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.937+0000 7f63e9513640 1 -- 172.21.15.150:0/5520968 wait complete. 2024-04-25T09:52:52.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.939+0000 7f63e9513640 1 Processor -- start 2024-04-25T09:52:52.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.939+0000 7f63e9513640 1 -- start start 2024-04-25T09:52:52.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.939+0000 7f63e9513640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63e40b9370 0x7f63e415b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.939+0000 7f63e9513640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e40cec80 0x7f63e406c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.939+0000 7f63e9513640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e415dbf0 0x7f63e4159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:52.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.939+0000 7f63e9513640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e415dfd0 con 0x7f63e40b9370 2024-04-25T09:52:52.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.939+0000 7f63e9513640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e40b97d0 con 0x7f63e40cec80 2024-04-25T09:52:52.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.939+0000 7f63e9513640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4159400 con 0x7f63e415dbf0 2024-04-25T09:52:52.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e27fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e40cec80 0x7f63e406c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e37fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e415dbf0 0x7f63e4159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63e40b9370 0x7f63e415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e27fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e40cec80 0x7f63e406c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51310/0 (socket says 172.21.15.150:51310) 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63e40b9370 0x7f63e415b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.150:35862/0 (socket says 172.21.15.150:35862) 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e27fc640 1 -- 172.21.15.150:0/2156117754 learned_addr learned my addr 172.21.15.150:0/2156117754 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e27fc640 1 -- 172.21.15.150:0/2156117754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e415dbf0 msgr2=0x7f63e4159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e27fc640 1 --2- 172.21.15.150:0/2156117754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e415dbf0 0x7f63e4159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e27fc640 1 -- 172.21.15.150:0/2156117754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63e40b9370 msgr2=0x7f63e415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e27fc640 1 --2- 172.21.15.150:0/2156117754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63e40b9370 0x7f63e415b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e27fc640 1 -- 172.21.15.150:0/2156117754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e415bb40 con 0x7f63e40cec80 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e27fc640 1 --2- 172.21.15.150:0/2156117754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e40cec80 0x7f63e406c3b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63d400e8e0 tx=0x7f63d400eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63cb7fe640 1 -- 172.21.15.150:0/2156117754 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4017070 con 0x7f63e40cec80 2024-04-25T09:52:52.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63cb7fe640 1 -- 172.21.15.150:0/2156117754 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d400c940 con 0x7f63e40cec80 2024-04-25T09:52:52.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e9513640 1 -- 172.21.15.150:0/2156117754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e415be20 con 0x7f63e40cec80 2024-04-25T09:52:52.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63cb7fe640 1 -- 172.21.15.150:0/2156117754 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d400cd10 con 0x7f63e40cec80 2024-04-25T09:52:52.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63e9513640 1 -- 172.21.15.150:0/2156117754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e415c6c0 con 0x7f63e40cec80 2024-04-25T09:52:52.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63cb7fe640 1 -- 172.21.15.150:0/2156117754 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f63d401a070 con 0x7f63e40cec80 2024-04-25T09:52:52.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.940+0000 7f63cb7fe640 1 -- 172.21.15.150:0/2156117754 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f63d401cab0 con 0x7f63e40cec80 2024-04-25T09:52:52.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.941+0000 7f63e9513640 1 -- 172.21.15.150:0/2156117754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e415bff0 con 0x7f63e40cec80 2024-04-25T09:52:52.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:52.945+0000 7f63cb7fe640 1 -- 172.21.15.150:0/2156117754 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f63d401c450 con 0x7f63e40cec80 2024-04-25T09:52:53.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:53.017+0000 7f63e9513640 1 -- 172.21.15.150:0/2156117754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "62755a3e-bdec-4e96-8e7a-a608b17acac4", "id": 7} v 0) v1 -- 0x7f63e4074520 con 0x7f63e40cec80 2024-04-25T09:52:53.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:53.026+0000 7f63cb7fe640 1 -- 172.21.15.150:0/2156117754 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "62755a3e-bdec-4e96-8e7a-a608b17acac4", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f63d401c630 con 0x7f63e40cec80 2024-04-25T09:52:53.027 INFO:teuthology.orchestra.run.smithi150.stdout:7 2024-04-25T09:52:53.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:53.026+0000 7f63e9513640 1 -- 172.21.15.150:0/2156117754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e40cec80 msgr2=0x7f63e406c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:53.026+0000 7f63e9513640 1 --2- 172.21.15.150:0/2156117754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e40cec80 0x7f63e406c3b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63d400e8e0 tx=0x7f63d400eda0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:53.026+0000 7f63e9513640 1 -- 172.21.15.150:0/2156117754 shutdown_connections 2024-04-25T09:52:53.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:53.027+0000 7f63e9513640 1 --2- 172.21.15.150:0/2156117754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63e415dbf0 0x7f63e4159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:53.027+0000 7f63e9513640 1 --2- 172.21.15.150:0/2156117754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e40cec80 0x7f63e406c3b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:53.027+0000 7f63e9513640 1 --2- 172.21.15.150:0/2156117754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63e40b9370 0x7f63e415b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:53.027+0000 7f63e9513640 1 -- 172.21.15.150:0/2156117754 >> 172.21.15.150:0/2156117754 conn(0x7f63e40b5480 msgr2=0x7f63e40c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:53.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:53.027+0000 7f63e9513640 1 -- 172.21.15.150:0/2156117754 shutdown_connections 2024-04-25T09:52:53.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T09:52:53.027+0000 7f63e9513640 1 -- 172.21.15.150:0/2156117754 wait complete. 2024-04-25T09:52:53.039 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T09:52:53.039 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T09:52:53.041 INFO:tasks.ceph.osd.0:Started 2024-04-25T09:52:53.042 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T09:52:53.042 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T09:52:53.044 INFO:tasks.ceph.osd.1:Started 2024-04-25T09:52:53.044 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T09:52:53.044 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T09:52:53.046 INFO:tasks.ceph.osd.2:Started 2024-04-25T09:52:53.046 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T09:52:53.046 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T09:52:53.048 INFO:tasks.ceph.osd.3:Started 2024-04-25T09:52:53.049 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T09:52:53.049 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T09:52:53.082 INFO:tasks.ceph.osd.4:Started 2024-04-25T09:52:53.082 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T09:52:53.082 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T09:52:53.084 INFO:tasks.ceph.osd.5:Started 2024-04-25T09:52:53.085 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T09:52:53.085 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T09:52:53.087 INFO:tasks.ceph.osd.6:Started 2024-04-25T09:52:53.087 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T09:52:53.087 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T09:52:53.089 INFO:tasks.ceph.osd.7:Started 2024-04-25T09:52:53.090 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:52:53.158 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:53.156+0000 7f26c3819040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T09:52:53.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.195+0000 7f16cca91640 1 Processor -- start 2024-04-25T09:52:53.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.195+0000 7f16cca91640 1 -- start start 2024-04-25T09:52:53.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.196+0000 7f16cca91640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c815dc60 0x7f16c80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:53.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.196+0000 7f16cca91640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f16c8161850 0x7f16c80ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.196+0000 7f16cca91640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c80cf250 con 0x7f16c80d0970 2024-04-25T09:52:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.196+0000 7f16cca91640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c80ce780 con 0x7f16c8161850 2024-04-25T09:52:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.196+0000 7f16cca91640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c80d3730 con 0x7f16c815dc60 2024-04-25T09:52:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.196+0000 7f16c5d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f16c8161850 0x7f16c80ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.196+0000 7f16c5d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f16c8161850 0x7f16c80ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58046/0 (socket says 172.21.15.139:58046) 2024-04-25T09:52:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.197+0000 7f16c5d9b640 1 -- 172.21.15.139:0/2346671865 learned_addr learned my addr 172.21.15.139:0/2346671865 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:52:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.196+0000 7f16c6d9d640 1 --1- 172.21.15.139:0/2346671865 >> v1:172.21.15.139:6789/0 conn(0x7f16c80d0970 0x7f16c80d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:36446/0 (socket says 172.21.15.139:36446) 2024-04-25T09:52:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.197+0000 7f16c659c640 1 --2- 172.21.15.139:0/2346671865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c815dc60 0x7f16c80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.197+0000 7f16c5d9b640 1 -- 172.21.15.139:0/2346671865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c815dc60 msgr2=0x7f16c80b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.197+0000 7f16c5d9b640 1 --2- 172.21.15.139:0/2346671865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c815dc60 0x7f16c80b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.197+0000 7f16c5d9b640 1 -- 172.21.15.139:0/2346671865 >> v1:172.21.15.139:6789/0 conn(0x7f16c80d0970 legacy=0x7f16c80d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.197+0000 7f16c5d9b640 1 -- 172.21.15.139:0/2346671865 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c80ce9b0 con 0x7f16c8161850 2024-04-25T09:52:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.197+0000 7f16c559a640 1 -- 172.21.15.139:0/2346671865 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074886946 0 0) 0x7f16c0001490 con 0x7f16c80d0970 2024-04-25T09:52:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.197+0000 7f16c659c640 1 --2- 172.21.15.139:0/2346671865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c815dc60 0x7f16c80b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.197+0000 7f16c5d9b640 1 --2- 172.21.15.139:0/2346671865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16c8161850 0x7f16c80ce240 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f16b80098a0 tx=0x7f16b802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b029940d32ca34a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.198+0000 7f16c559a640 1 -- 172.21.15.139:0/2346671865 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8037070 con 0x7f16c8161850 2024-04-25T09:52:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.198+0000 7f16c559a640 1 -- 172.21.15.139:0/2346671865 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8032440 con 0x7f16c8161850 2024-04-25T09:52:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.198+0000 7f16c559a640 1 -- 172.21.15.139:0/2346671865 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8032810 con 0x7f16c8161850 2024-04-25T09:52:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.198+0000 7f16cca91640 1 -- 172.21.15.139:0/2346671865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16c8161850 msgr2=0x7f16c80ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.198+0000 7f16cca91640 1 --2- 172.21.15.139:0/2346671865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16c8161850 0x7f16c80ce240 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f16b80098a0 tx=0x7f16b802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.198+0000 7f16cca91640 1 -- 172.21.15.139:0/2346671865 shutdown_connections 2024-04-25T09:52:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.198+0000 7f16cca91640 1 --2- 172.21.15.139:0/2346671865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16c8161850 0x7f16c80ce240 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.198+0000 7f16cca91640 1 --2- 172.21.15.139:0/2346671865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c815dc60 0x7f16c80b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.198+0000 7f16cca91640 1 -- 172.21.15.139:0/2346671865 >> 172.21.15.139:0/2346671865 conn(0x7f16c80b5910 msgr2=0x7f16c80b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:53.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.198+0000 7f16cca91640 1 -- 172.21.15.139:0/2346671865 shutdown_connections 2024-04-25T09:52:53.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.198+0000 7f16cca91640 1 -- 172.21.15.139:0/2346671865 wait complete. 2024-04-25T09:52:53.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.200+0000 7f16cca91640 1 Processor -- start 2024-04-25T09:52:53.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.200+0000 7f16cca91640 1 -- start start 2024-04-25T09:52:53.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.200+0000 7f16cca91640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c80d0970 0x7f16c81f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:53.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.200+0000 7f16cca91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16c815dc60 0x7f16c80c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:53.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.200+0000 7f16cca91640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16c8161850 0x7f16c815e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:53.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.200+0000 7f16cca91640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c80d1fc0 con 0x7f16c815dc60 2024-04-25T09:52:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.200+0000 7f16cca91640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c80cef70 con 0x7f16c8161850 2024-04-25T09:52:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.200+0000 7f16cca91640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c815e040 con 0x7f16c80d0970 2024-04-25T09:52:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16c5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16c815dc60 0x7f16c80c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16c5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16c815dc60 0x7f16c80c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58062/0 (socket says 172.21.15.139:58062) 2024-04-25T09:52:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16c5d9b640 1 -- 172.21.15.139:0/275165410 learned_addr learned my addr 172.21.15.139:0/275165410 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:52:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16c659c640 1 --2- 172.21.15.139:0/275165410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c80d0970 0x7f16c81f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16c6d9d640 1 --2- 172.21.15.139:0/275165410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16c8161850 0x7f16c815e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16c5d9b640 1 -- 172.21.15.139:0/275165410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c80d0970 msgr2=0x7f16c81f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16c5d9b640 1 --2- 172.21.15.139:0/275165410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c80d0970 0x7f16c81f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16c5d9b640 1 -- 172.21.15.139:0/275165410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16c8161850 msgr2=0x7f16c815e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16c5d9b640 1 --2- 172.21.15.139:0/275165410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16c8161850 0x7f16c815e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16c5d9b640 1 -- 172.21.15.139:0/275165410 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c8159070 con 0x7f16c815dc60 2024-04-25T09:52:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16c659c640 1 --2- 172.21.15.139:0/275165410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c80d0970 0x7f16c81f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:52:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16c5d9b640 1 --2- 172.21.15.139:0/275165410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16c815dc60 0x7f16c80c7510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f16b80098a0 tx=0x7f16b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16b6ffd640 1 -- 172.21.15.139:0/275165410 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8037040 con 0x7f16c815dc60 2024-04-25T09:52:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16b6ffd640 1 -- 172.21.15.139:0/275165410 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8042070 con 0x7f16c815dc60 2024-04-25T09:52:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.201+0000 7f16cca91640 1 -- 172.21.15.139:0/275165410 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c8159350 con 0x7f16c815dc60 2024-04-25T09:52:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.202+0000 7f16b6ffd640 1 -- 172.21.15.139:0/275165410 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8030880 con 0x7f16c815dc60 2024-04-25T09:52:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.202+0000 7f16cca91640 1 -- 172.21.15.139:0/275165410 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c8159910 con 0x7f16c815dc60 2024-04-25T09:52:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.202+0000 7f16b6ffd640 1 -- 172.21.15.139:0/275165410 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f16b8030a20 con 0x7f16c815dc60 2024-04-25T09:52:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.202+0000 7f16b6ffd640 1 -- 172.21.15.139:0/275165410 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f16b8047b30 con 0x7f16c815dc60 2024-04-25T09:52:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.202+0000 7f16cca91640 1 -- 172.21.15.139:0/275165410 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c8074520 con 0x7f16c815dc60 2024-04-25T09:52:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.205+0000 7f16b6ffd640 1 -- 172.21.15.139:0/275165410 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f16b8030d10 con 0x7f16c815dc60 2024-04-25T09:52:53.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.264+0000 7f16cca91640 1 -- 172.21.15.139:0/275165410 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16c80d16f0 con 0x7f16c815dc60 2024-04-25T09:52:53.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.265+0000 7f16b6ffd640 1 -- 172.21.15.139:0/275165410 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f16b8047460 con 0x7f16c815dc60 2024-04-25T09:52:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:52:53.267 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":10,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:52:53.019402+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:52:53.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.265+0000 7f16cca91640 1 -- 172.21.15.139:0/275165410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16c815dc60 msgr2=0x7f16c80c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:53.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.265+0000 7f16cca91640 1 --2- 172.21.15.139:0/275165410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16c815dc60 0x7f16c80c7510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f16b80098a0 tx=0x7f16b80046c0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.265+0000 7f16cca91640 1 -- 172.21.15.139:0/275165410 shutdown_connections 2024-04-25T09:52:53.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.265+0000 7f16cca91640 1 --2- 172.21.15.139:0/275165410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16c815dc60 0x7f16c80c7510 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.265+0000 7f16cca91640 1 --2- 172.21.15.139:0/275165410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16c8161850 0x7f16c815e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.265+0000 7f16cca91640 1 --2- 172.21.15.139:0/275165410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c80d0970 0x7f16c81f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:53.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.266+0000 7f16cca91640 1 -- 172.21.15.139:0/275165410 >> 172.21.15.139:0/275165410 conn(0x7f16c80b5910 msgr2=0x7f16c815c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:53.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.266+0000 7f16cca91640 1 -- 172.21.15.139:0/275165410 shutdown_connections 2024-04-25T09:52:53.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:52:53.266+0000 7f16cca91640 1 -- 172.21.15.139:0/275165410 wait complete. 2024-04-25T09:52:53.278 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T09:52:53.278 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T09:52:53.305 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:53.303+0000 7f7b52f30040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T09:52:53.420 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:53.418+0000 7f26c3819040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T09:52:53.472 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:53.470+0000 7f26c3819040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T09:52:53.562 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:53.560+0000 7f7b52f30040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T09:52:53.585 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:53.583+0000 7f26c3819040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T09:52:53.615 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:53.613+0000 7f7b52f30040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T09:52:53.657 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-04-25T09:52:53.655+0000 7f0be4413640 -1 Falling back to public interface 2024-04-25T09:52:53.659 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-04-25T09:52:53.657+0000 7fa685bfc640 -1 Falling back to public interface 2024-04-25T09:52:53.660 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-04-25T09:52:53.658+0000 7f210fa0e640 -1 Falling back to public interface 2024-04-25T09:52:53.660 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-04-25T09:52:53.658+0000 7f210e31d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6813/0: (98) Address already in use 2024-04-25T09:52:53.665 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-04-25T09:52:53.664+0000 7fc09d01f640 -1 Falling back to public interface 2024-04-25T09:52:53.688 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-04-25T09:52:53.686+0000 7fe18a620640 -1 Falling back to public interface 2024-04-25T09:52:53.699 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-04-25T09:52:53.697+0000 7f1673fd4640 -1 Falling back to public interface 2024-04-25T09:52:53.700 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-04-25T09:52:53.697+0000 7fe22820d640 -1 Falling back to public interface 2024-04-25T09:52:53.700 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-04-25T09:52:53.698+0000 7f1672f83640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2024-04-25T09:52:53.714 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-04-25T09:52:53.712+0000 7f1046f4e640 -1 Falling back to public interface 2024-04-25T09:52:53.733 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:53.731+0000 7f7b52f30040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T09:52:53.865 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:53.863+0000 7f26c3819040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T09:52:53.911 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:53.910+0000 7f26c3819040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T09:52:54.015 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:54.013+0000 7f7b52f30040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T09:52:54.063 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:54.061+0000 7f7b52f30040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T09:52:54.066 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:54.064+0000 7f26c3819040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T09:52:54.084 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T09:52:54.173 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.171+0000 7fe91c4e1640 1 Processor -- start 2024-04-25T09:52:54.173 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.171+0000 7fe91c4e1640 1 -- start start 2024-04-25T09:52:54.174 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe91c4e1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9140cdb80 0x7fe91406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:54.174 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe91c4e1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe91406b0a0 0x7fe9140cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:54.174 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe91c4e1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe91415bff0 con 0x7fe9140cb1f0 2024-04-25T09:52:54.175 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe91c4e1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9140cb9e0 con 0x7fe91406b0a0 2024-04-25T09:52:54.175 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe91c4e1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91415c920 con 0x7fe9140cdb80 2024-04-25T09:52:54.175 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe919a7c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe91406b0a0 0x7fe9140cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:54.175 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe91aa7e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe9140cb1f0 0x7fe91406fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:36452/0 (socket says 172.21.15.139:36452) 2024-04-25T09:52:54.175 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe91aa7e640 1 -- 172.21.15.139:0/3144829129 learned_addr learned my addr 172.21.15.139:0/3144829129 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:52:54.175 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe91a27d640 1 --2- 172.21.15.139:0/3144829129 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9140cdb80 0x7fe91406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:54.175 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe919a7c640 1 -- 172.21.15.139:0/3144829129 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9140cdb80 msgr2=0x7fe91406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:52:54.175 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe919a7c640 1 --2- 172.21.15.139:0/3144829129 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9140cdb80 0x7fe91406ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:54.175 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe919a7c640 1 -- 172.21.15.139:0/3144829129 >> v1:172.21.15.139:6789/0 conn(0x7fe9140cb1f0 legacy=0x7fe91406fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:54.175 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe91927b640 1 -- 172.21.15.139:0/3144829129 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707350244 0 0) 0x7fe91415bff0 con 0x7fe9140cb1f0 2024-04-25T09:52:54.175 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.172+0000 7fe919a7c640 1 -- 172.21.15.139:0/3144829129 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91415c2d0 con 0x7fe91406b0a0 2024-04-25T09:52:54.177 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.173+0000 7fe919a7c640 1 --2- 172.21.15.139:0/3144829129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91406b0a0 0x7fe9140cacb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe9100097a0 tx=0x7fe91002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bdef85b279d71f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:54.177 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.174+0000 7fe91927b640 1 -- 172.21.15.139:0/3144829129 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910037070 con 0x7fe91406b0a0 2024-04-25T09:52:54.177 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.174+0000 7fe91927b640 1 -- 172.21.15.139:0/3144829129 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe910032440 con 0x7fe91406b0a0 2024-04-25T09:52:54.177 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.174+0000 7fe91927b640 1 -- 172.21.15.139:0/3144829129 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910032810 con 0x7fe91406b0a0 2024-04-25T09:52:54.177 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.174+0000 7fe91c4e1640 1 -- 172.21.15.139:0/3144829129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91406b0a0 msgr2=0x7fe9140cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:54.177 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.174+0000 7fe91c4e1640 1 --2- 172.21.15.139:0/3144829129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91406b0a0 0x7fe9140cacb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe9100097a0 tx=0x7fe91002a180 comp rx=0 tx=0).stop 2024-04-25T09:52:54.177 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.174+0000 7fe91c4e1640 1 -- 172.21.15.139:0/3144829129 shutdown_connections 2024-04-25T09:52:54.177 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.174+0000 7fe91c4e1640 1 --2- 172.21.15.139:0/3144829129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91406b0a0 0x7fe9140cacb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:54.177 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.174+0000 7fe91c4e1640 1 --2- 172.21.15.139:0/3144829129 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9140cdb80 0x7fe91406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:54.177 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.174+0000 7fe91c4e1640 1 -- 172.21.15.139:0/3144829129 >> 172.21.15.139:0/3144829129 conn(0x7fe9140b2330 msgr2=0x7fe9140b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:54.177 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.174+0000 7fe91c4e1640 1 -- 172.21.15.139:0/3144829129 shutdown_connections 2024-04-25T09:52:54.177 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.174+0000 7fe91c4e1640 1 -- 172.21.15.139:0/3144829129 wait complete. 2024-04-25T09:52:54.180 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.175+0000 7fe91c4e1640 1 Processor -- start 2024-04-25T09:52:54.180 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.176+0000 7fe91c4e1640 1 -- start start 2024-04-25T09:52:54.180 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.176+0000 7fe91c4e1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91406b0a0 0x7fe9141540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:54.180 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.176+0000 7fe91c4e1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9140cb1f0 0x7fe914151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:54.180 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.176+0000 7fe91c4e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9140cdb80 0x7fe914156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:52:54.180 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.176+0000 7fe91c4e1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140721f0 con 0x7fe9140cdb80 2024-04-25T09:52:54.180 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.176+0000 7fe91c4e1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91406b4e0 con 0x7fe91406b0a0 2024-04-25T09:52:54.180 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.176+0000 7fe91c4e1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140cb660 con 0x7fe9140cb1f0 2024-04-25T09:52:54.180 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.176+0000 7fe91aa7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9140cdb80 0x7fe914156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:54.180 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.176+0000 7fe91aa7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9140cdb80 0x7fe914156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58080/0 (socket says 172.21.15.139:58080) 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.176+0000 7fe91aa7e640 1 -- 172.21.15.139:0/2526913005 learned_addr learned my addr 172.21.15.139:0/2526913005 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.176+0000 7fe919a7c640 1 --2- 172.21.15.139:0/2526913005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9140cb1f0 0x7fe914151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.176+0000 7fe91a27d640 1 --2- 172.21.15.139:0/2526913005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91406b0a0 0x7fe9141540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe91aa7e640 1 -- 172.21.15.139:0/2526913005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9140cb1f0 msgr2=0x7fe914151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe91aa7e640 1 --2- 172.21.15.139:0/2526913005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9140cb1f0 0x7fe914151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe91aa7e640 1 -- 172.21.15.139:0/2526913005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91406b0a0 msgr2=0x7fe9141540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe91aa7e640 1 --2- 172.21.15.139:0/2526913005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91406b0a0 0x7fe9141540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe91aa7e640 1 -- 172.21.15.139:0/2526913005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe914153070 con 0x7fe9140cdb80 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe91aa7e640 1 --2- 172.21.15.139:0/2526913005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9140cdb80 0x7fe914156870 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00c700 tx=0x7fe90c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe906ffd640 1 -- 172.21.15.139:0/2526913005 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c018070 con 0x7fe9140cdb80 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe906ffd640 1 -- 172.21.15.139:0/2526913005 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90c00ce10 con 0x7fe9140cdb80 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe91c4e1640 1 -- 172.21.15.139:0/2526913005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9141532f0 con 0x7fe9140cdb80 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe906ffd640 1 -- 172.21.15.139:0/2526913005 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c00a880 con 0x7fe9140cdb80 2024-04-25T09:52:54.181 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe91c4e1640 1 -- 172.21.15.139:0/2526913005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9141538b0 con 0x7fe9140cdb80 2024-04-25T09:52:54.182 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe906ffd640 1 -- 172.21.15.139:0/2526913005 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe90c01b070 con 0x7fe9140cdb80 2024-04-25T09:52:54.182 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.177+0000 7fe906ffd640 1 -- 172.21.15.139:0/2526913005 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fe90c007840 con 0x7fe9140cdb80 2024-04-25T09:52:54.182 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.178+0000 7fe91c4e1640 1 -- 172.21.15.139:0/2526913005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91404ef20 con 0x7fe9140cdb80 2024-04-25T09:52:54.182 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.180+0000 7fe906ffd640 1 -- 172.21.15.139:0/2526913005 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe914153070 con 0x7fe9140cdb80 2024-04-25T09:52:54.208 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:54.206+0000 7f26c3819040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T09:52:54.220 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:54.218+0000 7f7b52f30040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T09:52:54.242 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.241+0000 7fe91c4e1640 1 -- 172.21.15.139:0/2526913005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe91406ad00 con 0x7fe9140cdb80 2024-04-25T09:52:54.243 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.241+0000 7fe906ffd640 1 -- 172.21.15.139:0/2526913005 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fe90c0070d0 con 0x7fe9140cdb80 2024-04-25T09:52:54.243 INFO:teuthology.misc.health.smithi139.stdout: 2024-04-25T09:52:54.243 INFO:teuthology.misc.health.smithi139.stdout:{"epoch":10,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:52:53.019402+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:52:54.244 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.242+0000 7fe91c4e1640 1 -- 172.21.15.139:0/2526913005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9140cdb80 msgr2=0x7fe914156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:52:54.244 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.242+0000 7fe91c4e1640 1 --2- 172.21.15.139:0/2526913005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9140cdb80 0x7fe914156870 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00c700 tx=0x7fe90c00cbc0 comp rx=0 tx=0).stop 2024-04-25T09:52:54.244 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.242+0000 7fe91c4e1640 1 -- 172.21.15.139:0/2526913005 shutdown_connections 2024-04-25T09:52:54.245 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.242+0000 7fe91c4e1640 1 --2- 172.21.15.139:0/2526913005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9140cdb80 0x7fe914156870 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:54.245 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.242+0000 7fe91c4e1640 1 --2- 172.21.15.139:0/2526913005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9140cb1f0 0x7fe914151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:54.245 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.242+0000 7fe91c4e1640 1 --2- 172.21.15.139:0/2526913005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe91406b0a0 0x7fe9141540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:52:54.245 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.242+0000 7fe91c4e1640 1 -- 172.21.15.139:0/2526913005 >> 172.21.15.139:0/2526913005 conn(0x7fe9140b2330 msgr2=0x7fe91415fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:52:54.245 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.242+0000 7fe91c4e1640 1 -- 172.21.15.139:0/2526913005 shutdown_connections 2024-04-25T09:52:54.245 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:52:54.242+0000 7fe91c4e1640 1 -- 172.21.15.139:0/2526913005 wait complete. 2024-04-25T09:52:54.253 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T09:52:54.259 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:54.258+0000 7f26c3819040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T09:52:54.328 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:54.327+0000 7f26c3819040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T09:52:54.358 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:54.356+0000 7f7b52f30040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T09:52:54.373 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:54.371+0000 7f26c3819040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T09:52:54.407 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:54.405+0000 7f7b52f30040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T09:52:54.475 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:54.474+0000 7f7b52f30040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T09:52:54.522 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:54.520+0000 7f7b52f30040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T09:52:54.843 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:54.841+0000 7f26c3819040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T09:52:54.956 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:54.954+0000 7f26c3819040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T09:52:55.000 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:54.998+0000 7f7b52f30040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T09:52:55.117 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-04-25T09:52:55.115+0000 7f7b52f30040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T09:52:55.970 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:55.968+0000 7f26640bb640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T09:52:55.970 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-04-25T09:52:55.969+0000 7f26640bb640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T09:52:57.126 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-04-25T09:52:57.124+0000 7f210fa0e640 -1 osd.1 0 log_to_monitors true 2024-04-25T09:52:57.181 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-04-25T09:52:57.179+0000 7fa685bfc640 -1 osd.0 0 log_to_monitors true 2024-04-25T09:52:57.185 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-04-25T09:52:57.184+0000 7f0be4413640 -1 osd.2 0 log_to_monitors true 2024-04-25T09:52:57.194 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-04-25T09:52:57.193+0000 7fc09d01f640 -1 osd.3 0 log_to_monitors true 2024-04-25T09:52:57.196 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-04-25T09:52:57.194+0000 7fe18a620640 -1 osd.4 0 log_to_monitors true 2024-04-25T09:52:57.199 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-04-25T09:52:57.198+0000 7fe22820d640 -1 osd.7 0 log_to_monitors true 2024-04-25T09:52:57.201 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-04-25T09:52:57.199+0000 7f1673fd4640 -1 osd.6 0 log_to_monitors true 2024-04-25T09:52:57.225 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-04-25T09:52:57.223+0000 7f1046f4e640 -1 osd.5 0 log_to_monitors true 2024-04-25T09:53:01.012 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-04-25T09:53:01.009+0000 7fa681b92640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T09:53:01.028 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-04-25T09:53:01.026+0000 7f210bb18640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T09:53:01.032 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-04-25T09:53:01.031+0000 7fa6799b2640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:53:01.045 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-04-25T09:53:01.044+0000 7f2103124640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:53:01.061 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:01.065 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-04-25T09:53:01.063+0000 7fe186826640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T09:53:01.065 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-04-25T09:53:01.063+0000 7fc098fb5640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T09:53:01.067 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-04-25T09:53:01.066+0000 7f0bdfd04640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T09:53:01.078 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-04-25T09:53:01.077+0000 7fc090dd5640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:53:01.080 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-04-25T09:53:01.079+0000 7f0bd7b24640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:53:01.084 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-04-25T09:53:01.082+0000 7fe223b98640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T09:53:01.090 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-04-25T09:53:01.089+0000 7fe17de32640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:53:01.093 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-04-25T09:53:01.092+0000 7f1042ee4640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T09:53:01.097 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-04-25T09:53:01.096+0000 7fe21b9b8640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:53:01.105 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-04-25T09:53:01.104+0000 7f103ad04640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:53:01.116 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-04-25T09:53:01.115+0000 7f166ff6a640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T09:53:01.131 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-04-25T09:53:01.129+0000 7f1667d8a640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:53:01.154 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.152+0000 7fcab2b7e640 1 Processor -- start 2024-04-25T09:53:01.154 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.152+0000 7fcab2b7e640 1 -- start start 2024-04-25T09:53:01.156 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab2b7e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcaac15d860 0x7fcaac0b9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:01.156 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab2b7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcaac0d1040 0x7fcaac0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:01.156 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab2b7e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaac1617e0 con 0x7fcaac15dc40 2024-04-25T09:53:01.156 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab2b7e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaac161a80 con 0x7fcaac15d860 2024-04-25T09:53:01.156 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab2b7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac161d60 con 0x7fcaac0d1040 2024-04-25T09:53:01.156 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab111b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcaac0d1040 0x7fcaac0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:01.156 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab091a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcaac15d860 0x7fcaac0b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:01.156 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcaabfff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fcaac15dc40 0x7fcaac0d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:44550/0 (socket says 172.21.15.139:44550) 2024-04-25T09:53:01.156 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab111b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcaac0d1040 0x7fcaac0d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:40628/0 (socket says 172.21.15.139:40628) 2024-04-25T09:53:01.156 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab091a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcaac15d860 0x7fcaac0b9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:39638/0 (socket says 172.21.15.139:39638) 2024-04-25T09:53:01.157 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab111b640 1 -- 172.21.15.139:0/1129673518 learned_addr learned my addr 172.21.15.139:0/1129673518 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:01.157 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab111b640 1 -- 172.21.15.139:0/1129673518 >> v2:172.21.15.150:3300/0 conn(0x7fcaac15d860 msgr2=0x7fcaac0b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:01.157 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab111b640 1 --2- 172.21.15.139:0/1129673518 >> v2:172.21.15.150:3300/0 conn(0x7fcaac15d860 0x7fcaac0b9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.157 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab111b640 1 -- 172.21.15.139:0/1129673518 >> v1:172.21.15.150:6789/0 conn(0x7fcaac15dc40 legacy=0x7fcaac0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.157 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab111b640 1 -- 172.21.15.139:0/1129673518 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac162040 con 0x7fcaac0d1040 2024-04-25T09:53:01.157 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.153+0000 7fcab091a640 1 --2- 172.21.15.139:0/1129673518 >> v2:172.21.15.150:3300/0 conn(0x7fcaac15d860 0x7fcaac0b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:01.157 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.154+0000 7fcab111b640 1 --2- 172.21.15.139:0/1129673518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcaac0d1040 0x7fcaac0d1bf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009e50 tx=0x7fcaa402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17372e0d728491de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:01.157 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.154+0000 7fcaab7fe640 1 -- 172.21.15.139:0/1129673518 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4036070 con 0x7fcaac0d1040 2024-04-25T09:53:01.157 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.154+0000 7fcaab7fe640 1 -- 172.21.15.139:0/1129673518 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa4031600 con 0x7fcaac0d1040 2024-04-25T09:53:01.157 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.154+0000 7fcaab7fe640 1 -- 172.21.15.139:0/1129673518 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40319d0 con 0x7fcaac0d1040 2024-04-25T09:53:01.158 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.155+0000 7fcab2b7e640 1 -- 172.21.15.139:0/1129673518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcaac0d1040 msgr2=0x7fcaac0d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.158 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.155+0000 7fcab2b7e640 1 --2- 172.21.15.139:0/1129673518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcaac0d1040 0x7fcaac0d1bf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009e50 tx=0x7fcaa402a680 comp rx=0 tx=0).stop 2024-04-25T09:53:01.158 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.155+0000 7fcab2b7e640 1 -- 172.21.15.139:0/1129673518 shutdown_connections 2024-04-25T09:53:01.158 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.155+0000 7fcab2b7e640 1 --2- 172.21.15.139:0/1129673518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcaac0d1040 0x7fcaac0d1bf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.159 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.155+0000 7fcab2b7e640 1 --2- 172.21.15.139:0/1129673518 >> v2:172.21.15.150:3300/0 conn(0x7fcaac15d860 0x7fcaac0b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.159 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.155+0000 7fcab2b7e640 1 -- 172.21.15.139:0/1129673518 >> 172.21.15.139:0/1129673518 conn(0x7fcaac06c440 msgr2=0x7fcaac06c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:01.159 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.155+0000 7fcab2b7e640 1 -- 172.21.15.139:0/1129673518 shutdown_connections 2024-04-25T09:53:01.159 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.155+0000 7fcab2b7e640 1 -- 172.21.15.139:0/1129673518 wait complete. 2024-04-25T09:53:01.159 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.156+0000 7fcab2b7e640 1 Processor -- start 2024-04-25T09:53:01.159 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.156+0000 7fcab2b7e640 1 -- start start 2024-04-25T09:53:01.162 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.157+0000 7fcab2b7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcaac0d1040 0x7fcaac1f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:01.162 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.157+0000 7fcab2b7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcaac15d860 0x7fcaac152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:01.162 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.157+0000 7fcab2b7e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaac15dc40 0x7fcaac1f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:01.162 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.157+0000 7fcab2b7e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0b99c0 con 0x7fcaac15d860 2024-04-25T09:53:01.162 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.157+0000 7fcab2b7e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0b9840 con 0x7fcaac15dc40 2024-04-25T09:53:01.162 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.157+0000 7fcab2b7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac0b2c60 con 0x7fcaac0d1040 2024-04-25T09:53:01.162 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.157+0000 7fcaabfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcaac15d860 0x7fcaac152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:01.162 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.157+0000 7fcaabfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcaac15d860 0x7fcaac152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34186/0 (socket says 172.21.15.139:34186) 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.157+0000 7fcaabfff640 1 -- 172.21.15.139:0/498061867 learned_addr learned my addr 172.21.15.139:0/498061867 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.157+0000 7fcab091a640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcaac0d1040 0x7fcaac1f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.157+0000 7fcab111b640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaac15dc40 0x7fcaac1f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.158+0000 7fcaabfff640 1 -- 172.21.15.139:0/498061867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcaac0d1040 msgr2=0x7fcaac1f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.158+0000 7fcaabfff640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcaac0d1040 0x7fcaac1f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.158+0000 7fcaabfff640 1 -- 172.21.15.139:0/498061867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaac15dc40 msgr2=0x7fcaac1f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.158+0000 7fcaabfff640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaac15dc40 0x7fcaac1f6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.158+0000 7fcaabfff640 1 -- 172.21.15.139:0/498061867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac0ccbe0 con 0x7fcaac15d860 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.158+0000 7fcaabfff640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcaac15d860 0x7fcaac152a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fca9800c430 tx=0x7fca9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.158+0000 7fcaa97fa640 1 -- 172.21.15.139:0/498061867 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca980074e0 con 0x7fcaac15d860 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.158+0000 7fcaa97fa640 1 -- 172.21.15.139:0/498061867 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca98007680 con 0x7fcaac15d860 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.158+0000 7fcab2b7e640 1 -- 172.21.15.139:0/498061867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaac0ccec0 con 0x7fcaac15d860 2024-04-25T09:53:01.163 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.158+0000 7fcaa97fa640 1 -- 172.21.15.139:0/498061867 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98004a00 con 0x7fcaac15d860 2024-04-25T09:53:01.164 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.158+0000 7fcab2b7e640 1 -- 172.21.15.139:0/498061867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaac0cd480 con 0x7fcaac15d860 2024-04-25T09:53:01.164 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.159+0000 7fcab2b7e640 1 -- 172.21.15.139:0/498061867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaac14c690 con 0x7fcaac15d860 2024-04-25T09:53:01.164 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.159+0000 7fcaa97fa640 1 -- 172.21.15.139:0/498061867 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fca9801a020 con 0x7fcaac15d860 2024-04-25T09:53:01.164 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.160+0000 7fcaa97fa640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca9006b440 0x7fca9006d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:01.164 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.160+0000 7fcab091a640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca9006b440 0x7fca9006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:01.164 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.160+0000 7fcaa97fa640 1 -- 172.21.15.139:0/498061867 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fca98088950 con 0x7fcaac15d860 2024-04-25T09:53:01.164 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.160+0000 7fcab091a640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca9006b440 0x7fca9006d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009560 tx=0x7fcaa0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:01.166 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.164+0000 7fcaa97fa640 1 -- 172.21.15.139:0/498061867 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcaac14c690 con 0x7fcaac15d860 2024-04-25T09:53:01.336 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.334+0000 7fcab2b7e640 1 -- 172.21.15.139:0/498061867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcaac04ef20 con 0x7fcaac15d860 2024-04-25T09:53:01.337 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.335+0000 7fcaa97fa640 1 -- 172.21.15.139:0/498061867 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fca98058080 con 0x7fcaac15d860 2024-04-25T09:53:01.337 INFO:teuthology.misc.health.smithi139.stdout: 2024-04-25T09:53:01.337 INFO:teuthology.misc.health.smithi139.stdout:{"epoch":12,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:52:58.970344+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:01.339 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.337+0000 7fcab2b7e640 1 -- 172.21.15.139:0/498061867 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca9006b440 msgr2=0x7fca9006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.339 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.337+0000 7fcab2b7e640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca9006b440 0x7fca9006d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009560 tx=0x7fcaa0009190 comp rx=0 tx=0).stop 2024-04-25T09:53:01.339 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.337+0000 7fcab2b7e640 1 -- 172.21.15.139:0/498061867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcaac15d860 msgr2=0x7fcaac152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:01.339 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.337+0000 7fcab2b7e640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcaac15d860 0x7fcaac152a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fca9800c430 tx=0x7fca9800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.339 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.337+0000 7fcab2b7e640 1 -- 172.21.15.139:0/498061867 shutdown_connections 2024-04-25T09:53:01.339 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.337+0000 7fcab2b7e640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcaac15d860 0x7fcaac152a30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.339 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.337+0000 7fcab2b7e640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca9006b440 0x7fca9006d900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.339 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.337+0000 7fcab2b7e640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaac15dc40 0x7fcaac1f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.340 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.337+0000 7fcab2b7e640 1 --2- 172.21.15.139:0/498061867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcaac0d1040 0x7fcaac1f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:01.340 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.337+0000 7fcab2b7e640 1 -- 172.21.15.139:0/498061867 >> 172.21.15.139:0/498061867 conn(0x7fcaac06c440 msgr2=0x7fcaac0b77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:01.340 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.337+0000 7fcab2b7e640 1 -- 172.21.15.139:0/498061867 shutdown_connections 2024-04-25T09:53:01.340 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:01.337+0000 7fcab2b7e640 1 -- 172.21.15.139:0/498061867 wait complete. 2024-04-25T09:53:01.348 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T09:53:08.156 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:08.256 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.254+0000 7f82775ce640 1 Processor -- start 2024-04-25T09:53:08.256 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.254+0000 7f82775ce640 1 -- start start 2024-04-25T09:53:08.263 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.255+0000 7f82775ce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8270161180 0x7f827006cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.263 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.255+0000 7f82775ce640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827006be10 con 0x7f82700d05b0 2024-04-25T09:53:08.263 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.255+0000 7f82775ce640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827006c0f0 con 0x7f827009a110 2024-04-25T09:53:08.263 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.255+0000 7f82775ce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827006c3d0 con 0x7f8270161180 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.255+0000 7f8274b69640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f82700d05b0 0x7f8270099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56024/0 (socket says 172.21.15.139:56024) 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.255+0000 7f8274b69640 1 -- 172.21.15.139:0/3494263763 learned_addr learned my addr 172.21.15.139:0/3494263763 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.255+0000 7f827536a640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8270161180 0x7f827006cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225424413 0 0) 0x7f827006be10 con 0x7f82700d05b0 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f825c003520 con 0x7f82700d05b0 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716369226 0 0) 0x7f827006c0f0 con 0x7f827009a110 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f827006be10 con 0x7f827009a110 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1225594892 0 0) 0x7f825c003520 con 0x7f82700d05b0 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8270161180 msgr2=0x7f827006cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8270161180 0x7f827006cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 >> v1:172.21.15.150:6789/0 conn(0x7f827009a110 legacy=0x7f82700cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827006c6b0 con 0x7f82700d05b0 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f8260001670 con 0x7f82700d05b0 2024-04-25T09:53:08.264 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f827536a640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8270161180 0x7f827006cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 >> v1:172.21.15.139:6789/0 conn(0x7f82700d05b0 legacy=0x7f8270099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f825c003b70 0x7f825c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8270161180 0x7f825c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f825c004d40 0x7f825c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827006c3d0 con 0x7f825c004d40 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c00ba70 con 0x7f825c003b70 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c00bbf0 con 0x7f8270161180 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f827536a640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f825c003b70 0x7f825c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8275b6b640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f825c004d40 0x7f825c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8274b69640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8270161180 0x7f825c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8275b6b640 1 -- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8270161180 msgr2=0x7f825c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8275b6b640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8270161180 0x7f825c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8275b6b640 1 -- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f825c003b70 msgr2=0x7f825c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:08.265 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8275b6b640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f825c003b70 0x7f825c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.266 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f8275b6b640 1 -- 172.21.15.139:0/3494263763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c003520 con 0x7f825c004d40 2024-04-25T09:53:08.266 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.256+0000 7f827536a640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f825c003b70 0x7f825c003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:08.266 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.257+0000 7f8275b6b640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f825c004d40 0x7f825c0051c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82680018d0 tx=0x7f825c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc883ee377940af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:08.266 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.257+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82680014f0 con 0x7f825c004d40 2024-04-25T09:53:08.266 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.257+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82680052d0 con 0x7f825c004d40 2024-04-25T09:53:08.266 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.257+0000 7f8267fff640 1 -- 172.21.15.139:0/3494263763 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826802ac60 con 0x7f825c004d40 2024-04-25T09:53:08.266 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.258+0000 7f82775ce640 1 -- 172.21.15.139:0/3494263763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f825c004d40 msgr2=0x7f825c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.266 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.258+0000 7f82775ce640 1 --2- 172.21.15.139:0/3494263763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f825c004d40 0x7f825c0051c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82680018d0 tx=0x7f825c005830 comp rx=0 tx=0).stop 2024-04-25T09:53:08.266 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.258+0000 7f8275b6b640 1 -- 172.21.15.139:0/3494263763 reap_dead start 2024-04-25T09:53:08.266 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.258+0000 7f82775ce640 1 -- 172.21.15.139:0/3494263763 shutdown_connections 2024-04-25T09:53:08.267 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.258+0000 7f82775ce640 1 -- 172.21.15.139:0/3494263763 >> 172.21.15.139:0/3494263763 conn(0x7f82700b32e0 msgr2=0x7f82700b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:08.267 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.258+0000 7f82775ce640 1 -- 172.21.15.139:0/3494263763 shutdown_connections 2024-04-25T09:53:08.267 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.258+0000 7f82775ce640 1 -- 172.21.15.139:0/3494263763 wait complete. 2024-04-25T09:53:08.267 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.259+0000 7f82775ce640 1 Processor -- start 2024-04-25T09:53:08.267 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.259+0000 7f82775ce640 1 -- start start 2024-04-25T09:53:08.267 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f82775ce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f825c004d40 0x7f827014fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.267 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f82775ce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827015f140 0x7f827015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.267 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f82775ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82700c67c0 0x7f82700c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.267 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f82775ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82700d0b90 con 0x7f82700c67c0 2024-04-25T09:53:08.268 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f82775ce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82700d0a10 con 0x7f827015f140 2024-04-25T09:53:08.268 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f82775ce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82700cc550 con 0x7f825c004d40 2024-04-25T09:53:08.268 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f8275b6b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82700c67c0 0x7f82700c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.268 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f8275b6b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82700c67c0 0x7f82700c6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44064/0 (socket says 172.21.15.139:44064) 2024-04-25T09:53:08.268 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f8275b6b640 1 -- 172.21.15.139:0/2500499217 learned_addr learned my addr 172.21.15.139:0/2500499217 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:08.268 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f827536a640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f825c004d40 0x7f827014fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.268 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f8274b69640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827015f140 0x7f827015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.269 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f8275b6b640 1 -- 172.21.15.139:0/2500499217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f825c004d40 msgr2=0x7f827014fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.269 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f8275b6b640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f825c004d40 0x7f827014fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.269 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f8275b6b640 1 -- 172.21.15.139:0/2500499217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827015f140 msgr2=0x7f827015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.269 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f8275b6b640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827015f140 0x7f827015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.269 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.260+0000 7f8275b6b640 1 -- 172.21.15.139:0/2500499217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82700c5720 con 0x7f82700c67c0 2024-04-25T09:53:08.269 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.261+0000 7f8275b6b640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82700c67c0 0x7f82700c6c40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f826802a730 tx=0x7f82680012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:08.269 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.261+0000 7f8265ffb640 1 -- 172.21.15.139:0/2500499217 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268037040 con 0x7f82700c67c0 2024-04-25T09:53:08.269 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.261+0000 7f8265ffb640 1 -- 172.21.15.139:0/2500499217 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8268040070 con 0x7f82700c67c0 2024-04-25T09:53:08.269 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.261+0000 7f82775ce640 1 -- 172.21.15.139:0/2500499217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82700c5a00 con 0x7f82700c67c0 2024-04-25T09:53:08.269 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.261+0000 7f8265ffb640 1 -- 172.21.15.139:0/2500499217 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c009de0 con 0x7f82700c67c0 2024-04-25T09:53:08.269 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.261+0000 7f82775ce640 1 -- 172.21.15.139:0/2500499217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82700c5fc0 con 0x7f82700c67c0 2024-04-25T09:53:08.270 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.262+0000 7f82775ce640 1 -- 172.21.15.139:0/2500499217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827004ef20 con 0x7f82700c67c0 2024-04-25T09:53:08.270 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.262+0000 7f8265ffb640 1 -- 172.21.15.139:0/2500499217 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82680052d0 con 0x7f82700c67c0 2024-04-25T09:53:08.270 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.263+0000 7f8265ffb640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f825406b500 0x7f825406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.270 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.263+0000 7f8265ffb640 1 -- 172.21.15.139:0/2500499217 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f82680a9520 con 0x7f82700c67c0 2024-04-25T09:53:08.270 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.263+0000 7f827536a640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f825406b500 0x7f825406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.271 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.263+0000 7f827536a640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f825406b500 0x7f825406d9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f826c009bd0 tx=0x7f826c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:08.271 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.267+0000 7f8265ffb640 1 -- 172.21.15.139:0/2500499217 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8268078810 con 0x7f82700c67c0 2024-04-25T09:53:08.439 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.437+0000 7f82775ce640 1 -- 172.21.15.139:0/2500499217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f827015fae0 con 0x7f82700c67c0 2024-04-25T09:53:08.440 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.438+0000 7f8265ffb640 1 -- 172.21.15.139:0/2500499217 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12932 (secure 0 0 0) 0x7f8268078110 con 0x7f82700c67c0 2024-04-25T09:53:08.441 INFO:teuthology.misc.health.smithi139.stdout: 2024-04-25T09:53:08.441 INFO:teuthology.misc.health.smithi139.stdout:{"epoch":16,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:05.033853+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:08.442 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.440+0000 7f82775ce640 1 -- 172.21.15.139:0/2500499217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f825406b500 msgr2=0x7f825406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.443 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.440+0000 7f82775ce640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f825406b500 0x7f825406d9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f826c009bd0 tx=0x7f826c00b040 comp rx=0 tx=0).stop 2024-04-25T09:53:08.443 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.440+0000 7f82775ce640 1 -- 172.21.15.139:0/2500499217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82700c67c0 msgr2=0x7f82700c6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.443 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.440+0000 7f82775ce640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82700c67c0 0x7f82700c6c40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f826802a730 tx=0x7f82680012d0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.443 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.440+0000 7f82775ce640 1 -- 172.21.15.139:0/2500499217 shutdown_connections 2024-04-25T09:53:08.443 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.440+0000 7f82775ce640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82700c67c0 0x7f82700c6c40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.443 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.440+0000 7f82775ce640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f825406b500 0x7f825406d9c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.443 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.440+0000 7f82775ce640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827015f140 0x7f827015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.443 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.440+0000 7f82775ce640 1 --2- 172.21.15.139:0/2500499217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f825c004d40 0x7f827014fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.443 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.440+0000 7f82775ce640 1 -- 172.21.15.139:0/2500499217 >> 172.21.15.139:0/2500499217 conn(0x7f82700b32e0 msgr2=0x7f8270070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:08.443 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.440+0000 7f82775ce640 1 -- 172.21.15.139:0/2500499217 shutdown_connections 2024-04-25T09:53:08.444 INFO:teuthology.misc.health.smithi139.stderr:2024-04-25T09:53:08.440+0000 7f82775ce640 1 -- 172.21.15.139:0/2500499217 wait complete. 2024-04-25T09:53:08.452 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T09:53:08.452 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T09:53:08.452 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:53:08.452 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:53:08.455 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:53:08.455 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:53:08.455 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:53:08.457 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:53:08.457 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:53:08.457 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:53:08.460 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:53:08.460 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:53:08.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:53:08.463 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:53:08.463 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T09:53:08.463 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:08.526 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T09:53:08.527 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-04-25T09:53:08.527 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T09:53:08.525+0000 7f6b16ff1640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-25T09:53:08.527 INFO:tasks.ceph.mds.c.smithi139.stderr:2024-04-25T09:53:08.525+0000 7f2ca3e8e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-25T09:53:08.527 INFO:tasks.ceph.mds.c.smithi139.stderr:2024-04-25T09:53:08.525+0000 7f2ca3e8e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2024-04-25T09:53:08.544 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-04-25T09:53:08.544 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-04-25T09:53:08.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.563+0000 7fa8d889e640 1 Processor -- start 2024-04-25T09:53:08.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.563+0000 7fa8d889e640 1 -- start start 2024-04-25T09:53:08.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.564+0000 7fa8d889e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa8d40b4310 0x7fa8d406e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.564+0000 7fa8d889e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d40b4bf0 con 0x7fa8d406c280 2024-04-25T09:53:08.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.564+0000 7fa8d889e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d40b4ed0 con 0x7fa8d415bab0 2024-04-25T09:53:08.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.564+0000 7fa8d889e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d4070270 con 0x7fa8d40b4310 2024-04-25T09:53:08.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.564+0000 7fa8cbfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa8d406c280 0x7fa8d40b2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56044/0 (socket says 172.21.15.139:56044) 2024-04-25T09:53:08.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.564+0000 7fa8cbfff640 1 -- 172.21.15.139:0/612592432 learned_addr learned my addr 172.21.15.139:0/612592432 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:08.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.564+0000 7fa8d2d9d640 1 --2- 172.21.15.139:0/612592432 >> v2:172.21.15.150:3300/0 conn(0x7fa8d40b4310 0x7fa8d406e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.564+0000 7fa8d1d9b640 1 -- 172.21.15.139:0/612592432 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876199303 0 0) 0x7fa8d40b4bf0 con 0x7fa8d406c280 2024-04-25T09:53:08.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.564+0000 7fa8d1d9b640 1 -- 172.21.15.139:0/612592432 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8c0003520 con 0x7fa8d406c280 2024-04-25T09:53:08.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.564+0000 7fa8d2d9d640 1 -- 172.21.15.139:0/612592432 >> v1:172.21.15.150:6789/0 conn(0x7fa8d415bab0 legacy=0x7fa8d406bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.564+0000 7fa8d2d9d640 1 -- 172.21.15.139:0/612592432 >> v1:172.21.15.139:6789/0 conn(0x7fa8d406c280 legacy=0x7fa8d40b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.564+0000 7fa8d2d9d640 1 -- 172.21.15.139:0/612592432 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d40af130 con 0x7fa8d40b4310 2024-04-25T09:53:08.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.565+0000 7fa8d2d9d640 1 --2- 172.21.15.139:0/612592432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d40b4310 0x7fa8d406e6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009d90 tx=0x7fa8cc02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4773319099e4cb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:08.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.565+0000 7fa8d1d9b640 1 -- 172.21.15.139:0/612592432 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc007740 con 0x7fa8d40b4310 2024-04-25T09:53:08.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.565+0000 7fa8d1d9b640 1 -- 172.21.15.139:0/612592432 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc0078e0 con 0x7fa8d40b4310 2024-04-25T09:53:08.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.565+0000 7fa8d1d9b640 1 -- 172.21.15.139:0/612592432 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc007cb0 con 0x7fa8d40b4310 2024-04-25T09:53:08.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.566+0000 7fa8d889e640 1 -- 172.21.15.139:0/612592432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d40b4310 msgr2=0x7fa8d406e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.566+0000 7fa8d889e640 1 --2- 172.21.15.139:0/612592432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d40b4310 0x7fa8d406e6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009d90 tx=0x7fa8cc02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.566+0000 7fa8d889e640 1 -- 172.21.15.139:0/612592432 shutdown_connections 2024-04-25T09:53:08.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.566+0000 7fa8d889e640 1 --2- 172.21.15.139:0/612592432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d40b4310 0x7fa8d406e6a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.566+0000 7fa8d889e640 1 -- 172.21.15.139:0/612592432 >> 172.21.15.139:0/612592432 conn(0x7fa8d4071a00 msgr2=0x7fa8d4071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:08.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.566+0000 7fa8d889e640 1 -- 172.21.15.139:0/612592432 shutdown_connections 2024-04-25T09:53:08.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.566+0000 7fa8d889e640 1 -- 172.21.15.139:0/612592432 wait complete. 2024-04-25T09:53:08.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.567+0000 7fa8d889e640 1 Processor -- start 2024-04-25T09:53:08.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.567+0000 7fa8d889e640 1 -- start start 2024-04-25T09:53:08.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8d889e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8d406c280 0x7fa8d40c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8d889e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8d40b4310 0x7fa8d414a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8d889e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d415bab0 0x7fa8d40c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8d889e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d40b35d0 con 0x7fa8d40b4310 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8d889e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d40b3450 con 0x7fa8d415bab0 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8d889e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d40b3750 con 0x7fa8d406c280 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8cbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8d40b4310 0x7fa8d414a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8cbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8d40b4310 0x7fa8d414a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44102/0 (socket says 172.21.15.139:44102) 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8cbfff640 1 -- 172.21.15.139:0/3482185869 learned_addr learned my addr 172.21.15.139:0/3482185869 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8d259c640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8d406c280 0x7fa8d40c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8d2d9d640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d415bab0 0x7fa8d40c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8cbfff640 1 -- 172.21.15.139:0/3482185869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8d406c280 msgr2=0x7fa8d40c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8cbfff640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8d406c280 0x7fa8d40c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8cbfff640 1 -- 172.21.15.139:0/3482185869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d415bab0 msgr2=0x7fa8d40c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8cbfff640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d415bab0 0x7fa8d40c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8cbfff640 1 -- 172.21.15.139:0/3482185869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d40c6d80 con 0x7fa8d40b4310 2024-04-25T09:53:08.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8d259c640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8d406c280 0x7fa8d40c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:08.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.568+0000 7fa8cbfff640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8d40b4310 0x7fa8d414a710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c500 tx=0x7fa8bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:08.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.569+0000 7fa8caffd640 1 -- 172.21.15.139:0/3482185869 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc017070 con 0x7fa8d40b4310 2024-04-25T09:53:08.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.569+0000 7fa8caffd640 1 -- 172.21.15.139:0/3482185869 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc005ce0 con 0x7fa8d40b4310 2024-04-25T09:53:08.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.569+0000 7fa8d889e640 1 -- 172.21.15.139:0/3482185869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d414b530 con 0x7fa8d40b4310 2024-04-25T09:53:08.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.569+0000 7fa8caffd640 1 -- 172.21.15.139:0/3482185869 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc007330 con 0x7fa8d40b4310 2024-04-25T09:53:08.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.569+0000 7fa8d889e640 1 -- 172.21.15.139:0/3482185869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d414baf0 con 0x7fa8d40b4310 2024-04-25T09:53:08.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.570+0000 7fa8d889e640 1 -- 172.21.15.139:0/3482185869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d4074520 con 0x7fa8d40b4310 2024-04-25T09:53:08.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.570+0000 7fa8caffd640 1 -- 172.21.15.139:0/3482185869 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8bc01a070 con 0x7fa8d40b4310 2024-04-25T09:53:08.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.571+0000 7fa8caffd640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa8b406b4d0 0x7fa8b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.571+0000 7fa8d259c640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa8b406b4d0 0x7fa8b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.571+0000 7fa8caffd640 1 -- 172.21.15.139:0/3482185869 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa8bc0892b0 con 0x7fa8d40b4310 2024-04-25T09:53:08.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.571+0000 7fa8d259c640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa8b406b4d0 0x7fa8b406d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4001660 tx=0x7fa8c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:08.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.575+0000 7fa8caffd640 1 -- 172.21.15.139:0/3482185869 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8bc0585d0 con 0x7fa8d40b4310 2024-04-25T09:53:08.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.746+0000 7fa8d889e640 1 -- 172.21.15.139:0/3482185869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8d415d700 con 0x7fa8d40b4310 2024-04-25T09:53:08.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.747+0000 7fa8caffd640 1 -- 172.21.15.139:0/3482185869 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12932 (secure 0 0 0) 0x7fa8bc0890d0 con 0x7fa8d40b4310 2024-04-25T09:53:08.749 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:08.749 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":16,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:05.033853+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:08.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.749+0000 7fa8d889e640 1 -- 172.21.15.139:0/3482185869 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa8b406b4d0 msgr2=0x7fa8b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.749+0000 7fa8d889e640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa8b406b4d0 0x7fa8b406d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4001660 tx=0x7fa8c4009040 comp rx=0 tx=0).stop 2024-04-25T09:53:08.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.749+0000 7fa8d889e640 1 -- 172.21.15.139:0/3482185869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8d40b4310 msgr2=0x7fa8d414a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.749+0000 7fa8d889e640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8d40b4310 0x7fa8d414a710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c500 tx=0x7fa8bc00c9c0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.749+0000 7fa8d889e640 1 -- 172.21.15.139:0/3482185869 shutdown_connections 2024-04-25T09:53:08.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.749+0000 7fa8d889e640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8d40b4310 0x7fa8d414a710 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.749+0000 7fa8d889e640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa8b406b4d0 0x7fa8b406d990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.749+0000 7fa8d889e640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8d415bab0 0x7fa8d40c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.749+0000 7fa8d889e640 1 --2- 172.21.15.139:0/3482185869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8d406c280 0x7fa8d40c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.749+0000 7fa8d889e640 1 -- 172.21.15.139:0/3482185869 >> 172.21.15.139:0/3482185869 conn(0x7fa8d4071a00 msgr2=0x7fa8d40b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:08.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.749+0000 7fa8d889e640 1 -- 172.21.15.139:0/3482185869 shutdown_connections 2024-04-25T09:53:08.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.749+0000 7fa8d889e640 1 -- 172.21.15.139:0/3482185869 wait complete. 2024-04-25T09:53:08.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:53:08.763 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:08.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.861+0000 7f0fc16b3640 1 Processor -- start 2024-04-25T09:53:08.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.861+0000 7f0fc16b3640 1 -- start start 2024-04-25T09:53:08.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.862+0000 7f0fc16b3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0fbc15db40 0x7f0fbc0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.862+0000 7f0fc16b3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0fbc0b5880 0x7f0fbc06bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.862+0000 7f0fc16b3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fbc070f50 con 0x7f0fbc0b4bc0 2024-04-25T09:53:08.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.862+0000 7f0fc16b3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fbc071230 con 0x7f0fbc15db40 2024-04-25T09:53:08.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.862+0000 7f0fc16b3640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fbc071510 con 0x7f0fbc0b5880 2024-04-25T09:53:08.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.862+0000 7f0fbaffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0fbc15db40 0x7f0fbc0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.862+0000 7f0fbaffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0fbc15db40 0x7f0fbc0b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44104/0 (socket says 172.21.15.139:44104) 2024-04-25T09:53:08.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.862+0000 7f0fbaffd640 1 -- 172.21.15.139:0/499678826 learned_addr learned my addr 172.21.15.139:0/499678826 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:08.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.862+0000 7f0fbb7fe640 1 --2- 172.21.15.139:0/499678826 >> v2:172.21.15.150:3300/0 conn(0x7f0fbc0b5880 0x7f0fbc06bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.863+0000 7f0fbaffd640 1 -- 172.21.15.139:0/499678826 >> v2:172.21.15.150:3300/0 conn(0x7f0fbc0b5880 msgr2=0x7f0fbc06bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:08.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.863+0000 7f0fbaffd640 1 --2- 172.21.15.139:0/499678826 >> v2:172.21.15.150:3300/0 conn(0x7f0fbc0b5880 0x7f0fbc06bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.863+0000 7f0fbaffd640 1 -- 172.21.15.139:0/499678826 >> v1:172.21.15.150:6789/0 conn(0x7f0fbc0b4bc0 legacy=0x7f0fbc0b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:08.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.863+0000 7f0fbaffd640 1 -- 172.21.15.139:0/499678826 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc0717f0 con 0x7f0fbc15db40 2024-04-25T09:53:08.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.863+0000 7f0fbaffd640 1 --2- 172.21.15.139:0/499678826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fbc15db40 0x7f0fbc0b4680 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0b68f0 tx=0x7f0fac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dca1ba5689304bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:08.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.863+0000 7f0fb9ffb640 1 -- 172.21.15.139:0/499678826 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac036070 con 0x7f0fbc15db40 2024-04-25T09:53:08.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.863+0000 7f0fb9ffb640 1 -- 172.21.15.139:0/499678826 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac031600 con 0x7f0fbc15db40 2024-04-25T09:53:08.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.863+0000 7f0fb9ffb640 1 -- 172.21.15.139:0/499678826 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0319d0 con 0x7f0fbc15db40 2024-04-25T09:53:08.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.864+0000 7f0fc16b3640 1 -- 172.21.15.139:0/499678826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fbc15db40 msgr2=0x7f0fbc0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.864+0000 7f0fc16b3640 1 --2- 172.21.15.139:0/499678826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fbc15db40 0x7f0fbc0b4680 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0b68f0 tx=0x7f0fac02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.864+0000 7f0fc16b3640 1 -- 172.21.15.139:0/499678826 shutdown_connections 2024-04-25T09:53:08.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.864+0000 7f0fc16b3640 1 --2- 172.21.15.139:0/499678826 >> v2:172.21.15.150:3300/0 conn(0x7f0fbc0b5880 0x7f0fbc06bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.864+0000 7f0fc16b3640 1 --2- 172.21.15.139:0/499678826 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fbc15db40 0x7f0fbc0b4680 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.864+0000 7f0fc16b3640 1 -- 172.21.15.139:0/499678826 >> 172.21.15.139:0/499678826 conn(0x7f0fbc06f460 msgr2=0x7f0fbc06f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:08.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.864+0000 7f0fc16b3640 1 -- 172.21.15.139:0/499678826 shutdown_connections 2024-04-25T09:53:08.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.864+0000 7f0fc16b3640 1 -- 172.21.15.139:0/499678826 wait complete. 2024-04-25T09:53:08.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.865+0000 7f0fc16b3640 1 Processor -- start 2024-04-25T09:53:08.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.865+0000 7f0fc16b3640 1 -- start start 2024-04-25T09:53:08.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fc16b3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fbc0b4bc0 0x7f0fbc1f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fc16b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fbc0b5880 0x7f0fbc15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fc16b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fbc15db40 0x7f0fbc15c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fc16b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc06c490 con 0x7f0fbc0b5880 2024-04-25T09:53:08.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fc16b3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc070be0 con 0x7f0fbc0b4bc0 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fc16b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc06c610 con 0x7f0fbc15db40 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fba7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fbc0b5880 0x7f0fbc15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fba7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fbc0b5880 0x7f0fbc15be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44110/0 (socket says 172.21.15.139:44110) 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fba7fc640 1 -- 172.21.15.139:0/3714862038 learned_addr learned my addr 172.21.15.139:0/3714862038 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fbb7fe640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fbc15db40 0x7f0fbc15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fbaffd640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fbc0b4bc0 0x7f0fbc1f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fba7fc640 1 -- 172.21.15.139:0/3714862038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fbc15db40 msgr2=0x7f0fbc15c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fba7fc640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fbc15db40 0x7f0fbc15c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fba7fc640 1 -- 172.21.15.139:0/3714862038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fbc0b4bc0 msgr2=0x7f0fbc1f1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fba7fc640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fbc0b4bc0 0x7f0fbc1f1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fba7fc640 1 -- 172.21.15.139:0/3714862038 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc1e8370 con 0x7f0fbc0b5880 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fbaffd640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fbc0b4bc0 0x7f0fbc1f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.866+0000 7f0fbb7fe640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fbc15db40 0x7f0fbc15c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:08.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.867+0000 7f0fba7fc640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fbc0b5880 0x7f0fbc15be40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c430 tx=0x7f0fb000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:08.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.867+0000 7f0fa37fe640 1 -- 172.21.15.139:0/3714862038 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0017070 con 0x7f0fbc0b5880 2024-04-25T09:53:08.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.867+0000 7f0fa37fe640 1 -- 172.21.15.139:0/3714862038 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb00035f0 con 0x7f0fbc0b5880 2024-04-25T09:53:08.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.867+0000 7f0fc16b3640 1 -- 172.21.15.139:0/3714862038 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fbc1e8650 con 0x7f0fbc0b5880 2024-04-25T09:53:08.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.867+0000 7f0fa37fe640 1 -- 172.21.15.139:0/3714862038 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00039e0 con 0x7f0fbc0b5880 2024-04-25T09:53:08.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.867+0000 7f0fc16b3640 1 -- 172.21.15.139:0/3714862038 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fbc1e8c10 con 0x7f0fbc0b5880 2024-04-25T09:53:08.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.868+0000 7f0fc16b3640 1 -- 172.21.15.139:0/3714862038 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fbc074520 con 0x7f0fbc0b5880 2024-04-25T09:53:08.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.868+0000 7f0fa37fe640 1 -- 172.21.15.139:0/3714862038 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fb001a070 con 0x7f0fbc0b5880 2024-04-25T09:53:08.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.868+0000 7f0fa37fe640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fa806b4d0 0x7f0fa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:08.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.869+0000 7f0fa37fe640 1 -- 172.21.15.139:0/3714862038 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0fb0089200 con 0x7f0fbc0b5880 2024-04-25T09:53:08.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.869+0000 7f0fbaffd640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fa806b4d0 0x7f0fa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:08.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.869+0000 7f0fbaffd640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fa806b4d0 0x7f0fa806d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0fac033000 tx=0x7f0fac009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:08.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:08.873+0000 7f0fa37fe640 1 -- 172.21.15.139:0/3714862038 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0fb0057d80 con 0x7f0fbc0b5880 2024-04-25T09:53:09.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.045+0000 7f0fc16b3640 1 -- 172.21.15.139:0/3714862038 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0fbc0701f0 con 0x7f0fbc0b5880 2024-04-25T09:53:09.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.045+0000 7f0fa37fe640 1 -- 172.21.15.139:0/3714862038 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0fb005be20 con 0x7f0fbc0b5880 2024-04-25T09:53:09.047 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:53:09.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.047+0000 7f0fc16b3640 1 -- 172.21.15.139:0/3714862038 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fa806b4d0 msgr2=0x7f0fa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:09.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.047+0000 7f0fc16b3640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fa806b4d0 0x7f0fa806d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0fac033000 tx=0x7f0fac009000 comp rx=0 tx=0).stop 2024-04-25T09:53:09.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.047+0000 7f0fc16b3640 1 -- 172.21.15.139:0/3714862038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fbc0b5880 msgr2=0x7f0fbc15be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:09.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.047+0000 7f0fc16b3640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fbc0b5880 0x7f0fbc15be40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c430 tx=0x7f0fb000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.047+0000 7f0fc16b3640 1 -- 172.21.15.139:0/3714862038 shutdown_connections 2024-04-25T09:53:09.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.047+0000 7f0fc16b3640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fbc0b5880 0x7f0fbc15be40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.047+0000 7f0fc16b3640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fa806b4d0 0x7f0fa806d990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.047+0000 7f0fc16b3640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fbc15db40 0x7f0fbc15c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.047+0000 7f0fc16b3640 1 --2- 172.21.15.139:0/3714862038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fbc0b4bc0 0x7f0fbc1f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.047+0000 7f0fc16b3640 1 -- 172.21.15.139:0/3714862038 >> 172.21.15.139:0/3714862038 conn(0x7f0fbc06f460 msgr2=0x7f0fbc06c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:09.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.047+0000 7f0fc16b3640 1 -- 172.21.15.139:0/3714862038 shutdown_connections 2024-04-25T09:53:09.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.047+0000 7f0fc16b3640 1 -- 172.21.15.139:0/3714862038 wait complete. 2024-04-25T09:53:09.060 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:09.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.155+0000 7f017141f640 1 Processor -- start 2024-04-25T09:53:09.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.155+0000 7f017141f640 1 -- start start 2024-04-25T09:53:09.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f017141f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f016c15dc60 0x7f016c0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:09.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f017141f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f016c161850 0x7f016c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:09.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f017141f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f016c0d0970 0x7f016c0d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:09.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f017141f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016c0ce780 con 0x7f016c0d0970 2024-04-25T09:53:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f017141f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016c0d3730 con 0x7f016c15dc60 2024-04-25T09:53:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f017141f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0cf250 con 0x7f016c161850 2024-04-25T09:53:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f016b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f016c0d0970 0x7f016c0d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f016b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f016c0d0970 0x7f016c0d0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44126/0 (socket says 172.21.15.139:44126) 2024-04-25T09:53:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f016b7fe640 1 -- 172.21.15.139:0/2837448541 learned_addr learned my addr 172.21.15.139:0/2837448541 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f0163fff640 1 --2- 172.21.15.139:0/2837448541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f016c161850 0x7f016c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:09.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f016affd640 1 --2- 172.21.15.139:0/2837448541 >> v2:172.21.15.150:3300/0 conn(0x7f016c15dc60 0x7f016c0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:09.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f016b7fe640 1 -- 172.21.15.139:0/2837448541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f016c161850 msgr2=0x7f016c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:09.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f016b7fe640 1 --2- 172.21.15.139:0/2837448541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f016c161850 0x7f016c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f016b7fe640 1 -- 172.21.15.139:0/2837448541 >> v2:172.21.15.150:3300/0 conn(0x7f016c15dc60 msgr2=0x7f016c0b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:09.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f016b7fe640 1 --2- 172.21.15.139:0/2837448541 >> v2:172.21.15.150:3300/0 conn(0x7f016c15dc60 0x7f016c0b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f016b7fe640 1 -- 172.21.15.139:0/2837448541 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0ce9b0 con 0x7f016c0d0970 2024-04-25T09:53:09.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.156+0000 7f0163fff640 1 --2- 172.21.15.139:0/2837448541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f016c161850 0x7f016c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:09.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.157+0000 7f016b7fe640 1 --2- 172.21.15.139:0/2837448541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c0d0970 0x7f016c0d0e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f016c08c680 tx=0x7f016402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326857d1771a4e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:09.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.157+0000 7f016a7fc640 1 -- 172.21.15.139:0/2837448541 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164036070 con 0x7f016c0d0970 2024-04-25T09:53:09.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.157+0000 7f016a7fc640 1 -- 172.21.15.139:0/2837448541 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0164031600 con 0x7f016c0d0970 2024-04-25T09:53:09.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.157+0000 7f016a7fc640 1 -- 172.21.15.139:0/2837448541 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01640319d0 con 0x7f016c0d0970 2024-04-25T09:53:09.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.157+0000 7f017141f640 1 -- 172.21.15.139:0/2837448541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c0d0970 msgr2=0x7f016c0d0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:09.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.157+0000 7f017141f640 1 --2- 172.21.15.139:0/2837448541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c0d0970 0x7f016c0d0e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f016c08c680 tx=0x7f016402a6c0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.157+0000 7f017141f640 1 -- 172.21.15.139:0/2837448541 shutdown_connections 2024-04-25T09:53:09.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.157+0000 7f017141f640 1 --2- 172.21.15.139:0/2837448541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c0d0970 0x7f016c0d0e00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.157+0000 7f017141f640 1 --2- 172.21.15.139:0/2837448541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f016c161850 0x7f016c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.157+0000 7f017141f640 1 --2- 172.21.15.139:0/2837448541 >> v2:172.21.15.150:3300/0 conn(0x7f016c15dc60 0x7f016c0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.157+0000 7f017141f640 1 -- 172.21.15.139:0/2837448541 >> 172.21.15.139:0/2837448541 conn(0x7f016c0b5910 msgr2=0x7f016c0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:09.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.157+0000 7f017141f640 1 -- 172.21.15.139:0/2837448541 shutdown_connections 2024-04-25T09:53:09.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.158+0000 7f017141f640 1 -- 172.21.15.139:0/2837448541 wait complete. 2024-04-25T09:53:09.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.159+0000 7f017141f640 1 Processor -- start 2024-04-25T09:53:09.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.159+0000 7f017141f640 1 -- start start 2024-04-25T09:53:09.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.159+0000 7f017141f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f016c0d0970 0x7f016c14ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:09.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.159+0000 7f017141f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f016c15dc60 0x7f016c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:09.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.159+0000 7f017141f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c161850 0x7f016c0cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:09.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.159+0000 7f017141f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0d1fc0 con 0x7f016c161850 2024-04-25T09:53:09.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.159+0000 7f017141f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0cef70 con 0x7f016c15dc60 2024-04-25T09:53:09.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.159+0000 7f017141f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c15e040 con 0x7f016c0d0970 2024-04-25T09:53:09.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.159+0000 7f016b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c161850 0x7f016c0cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:09.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.159+0000 7f016b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c161850 0x7f016c0cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44142/0 (socket says 172.21.15.139:44142) 2024-04-25T09:53:09.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.159+0000 7f016b7fe640 1 -- 172.21.15.139:0/3213175867 learned_addr learned my addr 172.21.15.139:0/3213175867 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:09.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f016affd640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f016c0d0970 0x7f016c14ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:09.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f016b7fe640 1 -- 172.21.15.139:0/3213175867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f016c0d0970 msgr2=0x7f016c14ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:09.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f0163fff640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f016c15dc60 0x7f016c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:09.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f016b7fe640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f016c0d0970 0x7f016c14ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f016b7fe640 1 -- 172.21.15.139:0/3213175867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f016c15dc60 msgr2=0x7f016c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:09.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f016b7fe640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f016c15dc60 0x7f016c0c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f016b7fe640 1 -- 172.21.15.139:0/3213175867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0cbda0 con 0x7f016c161850 2024-04-25T09:53:09.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f016b7fe640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c161850 0x7f016c0cc9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0164009e50 tx=0x7f0164004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:09.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f01637fe640 1 -- 172.21.15.139:0/3213175867 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164041020 con 0x7f016c161850 2024-04-25T09:53:09.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f01637fe640 1 -- 172.21.15.139:0/3213175867 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0164007780 con 0x7f016c161850 2024-04-25T09:53:09.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f017141f640 1 -- 172.21.15.139:0/3213175867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c0cad00 con 0x7f016c161850 2024-04-25T09:53:09.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f01637fe640 1 -- 172.21.15.139:0/3213175867 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164036040 con 0x7f016c161850 2024-04-25T09:53:09.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.160+0000 7f017141f640 1 -- 172.21.15.139:0/3213175867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c0cb2c0 con 0x7f016c161850 2024-04-25T09:53:09.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.161+0000 7f017141f640 1 -- 172.21.15.139:0/3213175867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016c074520 con 0x7f016c161850 2024-04-25T09:53:09.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.161+0000 7f01637fe640 1 -- 172.21.15.139:0/3213175867 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0164044020 con 0x7f016c161850 2024-04-25T09:53:09.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.162+0000 7f01637fe640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f014c06b4d0 0x7f014c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:09.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.162+0000 7f01637fe640 1 -- 172.21.15.139:0/3213175867 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f01640aaca0 con 0x7f016c161850 2024-04-25T09:53:09.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.162+0000 7f016affd640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f014c06b4d0 0x7f014c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:09.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.162+0000 7f016affd640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f014c06b4d0 0x7f014c06d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f015c006fd0 tx=0x7f015c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:09.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.166+0000 7f01637fe640 1 -- 172.21.15.139:0/3213175867 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f016404b030 con 0x7f016c161850 2024-04-25T09:53:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.373+0000 7f017141f640 1 -- 172.21.15.139:0/3213175867 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f016c161160 con 0x7f016c161850 2024-04-25T09:53:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.373+0000 7f01637fe640 1 -- 172.21.15.139:0/3213175867 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f0164079f30 con 0x7f016c161850 2024-04-25T09:53:09.376 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:09.376 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/2751714076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2751714076},{"type":"v1","addr":"172.21.15.139:6834","nonce":2751714076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T09:53:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 2 2024-04-25T09:53:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.375+0000 7f017141f640 1 -- 172.21.15.139:0/3213175867 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f014c06b4d0 msgr2=0x7f014c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.375+0000 7f017141f640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f014c06b4d0 0x7f014c06d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f015c006fd0 tx=0x7f015c008040 comp rx=0 tx=0).stop 2024-04-25T09:53:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.375+0000 7f017141f640 1 -- 172.21.15.139:0/3213175867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c161850 msgr2=0x7f016c0cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:09.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.375+0000 7f017141f640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c161850 0x7f016c0cc9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0164009e50 tx=0x7f0164004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:09.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.375+0000 7f017141f640 1 -- 172.21.15.139:0/3213175867 shutdown_connections 2024-04-25T09:53:09.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.375+0000 7f017141f640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f016c161850 0x7f016c0cc9c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.375+0000 7f017141f640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f014c06b4d0 0x7f014c06d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.375+0000 7f017141f640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f016c15dc60 0x7f016c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.375+0000 7f017141f640 1 --2- 172.21.15.139:0/3213175867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f016c0d0970 0x7f016c14ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:09.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.376+0000 7f017141f640 1 -- 172.21.15.139:0/3213175867 >> 172.21.15.139:0/3213175867 conn(0x7f016c0b5910 msgr2=0x7f016c0cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:09.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.376+0000 7f017141f640 1 -- 172.21.15.139:0/3213175867 shutdown_connections 2024-04-25T09:53:09.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:09.376+0000 7f017141f640 1 -- 172.21.15.139:0/3213175867 wait complete. 2024-04-25T09:53:11.389 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.495+0000 7f7aea25d640 1 Processor -- start 2024-04-25T09:53:11.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.495+0000 7f7aea25d640 1 -- start start 2024-04-25T09:53:11.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.495+0000 7f7aea25d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7ae4155e30 0x7f7ae415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.495+0000 7f7aea25d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7ae40b1e50 0x7f7ae415c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.495+0000 7f7aea25d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae415c880 con 0x7f7ae415bdc0 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.495+0000 7f7aea25d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae406d6f0 con 0x7f7ae4155e30 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.495+0000 7f7aea25d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae415cb60 con 0x7f7ae40b1e50 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.496+0000 7f7ae37fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7ae4155e30 0x7f7ae415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.496+0000 7f7ae2ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f7ae415bdc0 0x7f7ae40b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56046/0 (socket says 172.21.15.139:56046) 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.496+0000 7f7ae37fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7ae4155e30 0x7f7ae415cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44148/0 (socket says 172.21.15.139:44148) 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.496+0000 7f7ae2ffd640 1 -- 172.21.15.139:0/2874976955 learned_addr learned my addr 172.21.15.139:0/2874976955 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.496+0000 7f7ae3fff640 1 --2- 172.21.15.139:0/2874976955 >> v2:172.21.15.150:3300/0 conn(0x7f7ae40b1e50 0x7f7ae415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.496+0000 7f7ae37fe640 1 -- 172.21.15.139:0/2874976955 >> v2:172.21.15.150:3300/0 conn(0x7f7ae40b1e50 msgr2=0x7f7ae415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.496+0000 7f7ae37fe640 1 --2- 172.21.15.139:0/2874976955 >> v2:172.21.15.150:3300/0 conn(0x7f7ae40b1e50 0x7f7ae415c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.496+0000 7f7ae37fe640 1 -- 172.21.15.139:0/2874976955 >> v1:172.21.15.139:6789/0 conn(0x7f7ae415bdc0 legacy=0x7f7ae40b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.496+0000 7f7ae37fe640 1 -- 172.21.15.139:0/2874976955 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae406d380 con 0x7f7ae4155e30 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.496+0000 7f7ae27fc640 1 -- 172.21.15.139:0/2874976955 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1548015402 0 0) 0x7f7ae415c880 con 0x7f7ae415bdc0 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.497+0000 7f7ae37fe640 1 --2- 172.21.15.139:0/2874976955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae4155e30 0x7f7ae415cf80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7ae406f600 tx=0x7f7ad402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89d585161b0069e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.497+0000 7f7ae27fc640 1 -- 172.21.15.139:0/2874976955 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4037070 con 0x7f7ae4155e30 2024-04-25T09:53:11.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.497+0000 7f7ae27fc640 1 -- 172.21.15.139:0/2874976955 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad4032440 con 0x7f7ae4155e30 2024-04-25T09:53:11.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.497+0000 7f7ae27fc640 1 -- 172.21.15.139:0/2874976955 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4032810 con 0x7f7ae4155e30 2024-04-25T09:53:11.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.497+0000 7f7aea25d640 1 -- 172.21.15.139:0/2874976955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae4155e30 msgr2=0x7f7ae415cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.497+0000 7f7aea25d640 1 --2- 172.21.15.139:0/2874976955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae4155e30 0x7f7ae415cf80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7ae406f600 tx=0x7f7ad402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.497+0000 7f7aea25d640 1 -- 172.21.15.139:0/2874976955 shutdown_connections 2024-04-25T09:53:11.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.497+0000 7f7aea25d640 1 --2- 172.21.15.139:0/2874976955 >> v2:172.21.15.150:3300/0 conn(0x7f7ae40b1e50 0x7f7ae415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.497+0000 7f7aea25d640 1 --2- 172.21.15.139:0/2874976955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae4155e30 0x7f7ae415cf80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.497+0000 7f7aea25d640 1 -- 172.21.15.139:0/2874976955 >> 172.21.15.139:0/2874976955 conn(0x7f7ae4066f20 msgr2=0x7f7ae406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:11.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.497+0000 7f7aea25d640 1 -- 172.21.15.139:0/2874976955 shutdown_connections 2024-04-25T09:53:11.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.497+0000 7f7aea25d640 1 -- 172.21.15.139:0/2874976955 wait complete. 2024-04-25T09:53:11.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.499+0000 7f7aea25d640 1 Processor -- start 2024-04-25T09:53:11.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.499+0000 7f7aea25d640 1 -- start start 2024-04-25T09:53:11.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.499+0000 7f7aea25d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ae40b1e50 0x7f7ae414e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.499+0000 7f7aea25d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae4155e30 0x7f7ae4153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.499+0000 7f7aea25d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ae415bdc0 0x7f7ae41540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.499+0000 7f7aea25d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40af260 con 0x7f7ae4155e30 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.499+0000 7f7aea25d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40af0e0 con 0x7f7ae415bdc0 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.499+0000 7f7aea25d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae415d4c0 con 0x7f7ae40b1e50 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7ae2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae4155e30 0x7f7ae4153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7ae2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae4155e30 0x7f7ae4153bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44150/0 (socket says 172.21.15.139:44150) 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7ae2ffd640 1 -- 172.21.15.139:0/4026230985 learned_addr learned my addr 172.21.15.139:0/4026230985 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7ae3fff640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ae415bdc0 0x7f7ae41540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7ae37fe640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ae40b1e50 0x7f7ae414e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7ae2ffd640 1 -- 172.21.15.139:0/4026230985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ae40b1e50 msgr2=0x7f7ae414e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7ae2ffd640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ae40b1e50 0x7f7ae414e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7ae2ffd640 1 -- 172.21.15.139:0/4026230985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ae415bdc0 msgr2=0x7f7ae41540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7ae2ffd640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ae415bdc0 0x7f7ae41540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7ae2ffd640 1 -- 172.21.15.139:0/4026230985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae41f46e0 con 0x7f7ae4155e30 2024-04-25T09:53:11.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7ae2ffd640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae4155e30 0x7f7ae4153bb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c500 tx=0x7f7ad800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:11.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7acbfff640 1 -- 172.21.15.139:0/4026230985 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8018070 con 0x7f7ae4155e30 2024-04-25T09:53:11.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7acbfff640 1 -- 172.21.15.139:0/4026230985 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad8007580 con 0x7f7ae4155e30 2024-04-25T09:53:11.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7acbfff640 1 -- 172.21.15.139:0/4026230985 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8004a20 con 0x7f7ae4155e30 2024-04-25T09:53:11.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7aea25d640 1 -- 172.21.15.139:0/4026230985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae41f3690 con 0x7f7ae4155e30 2024-04-25T09:53:11.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.500+0000 7f7aea25d640 1 -- 172.21.15.139:0/4026230985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae41f3c00 con 0x7f7ae4155e30 2024-04-25T09:53:11.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.501+0000 7f7aea25d640 1 -- 172.21.15.139:0/4026230985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae4074520 con 0x7f7ae4155e30 2024-04-25T09:53:11.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.502+0000 7f7acbfff640 1 -- 172.21.15.139:0/4026230985 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ad801b070 con 0x7f7ae4155e30 2024-04-25T09:53:11.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.502+0000 7f7acbfff640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ad006b4d0 0x7f7ad006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.502+0000 7f7ae37fe640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ad006b4d0 0x7f7ad006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.502+0000 7f7acbfff640 1 -- 172.21.15.139:0/4026230985 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7ad80890a0 con 0x7f7ae4155e30 2024-04-25T09:53:11.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.503+0000 7f7ae37fe640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ad006b4d0 0x7f7ad006d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4034000 tx=0x7f7ad402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:11.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.506+0000 7f7acbfff640 1 -- 172.21.15.139:0/4026230985 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ad8057cb0 con 0x7f7ae4155e30 2024-04-25T09:53:11.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.717+0000 7f7aea25d640 1 -- 172.21.15.139:0/4026230985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ae406b830 con 0x7f7ae4155e30 2024-04-25T09:53:11.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.717+0000 7f7acbfff640 1 -- 172.21.15.139:0/4026230985 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7f7ad805bd50 con 0x7f7ae4155e30 2024-04-25T09:53:11.719 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:11.720 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/892226922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":892226922},{"type":"v1","addr":"172.21.15.150:6834","nonce":892226922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/582019435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":582019435},{"type":"v1","addr":"172.21.15.139:6837","nonce":582019435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/2751714076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2751714076},{"type":"v1","addr":"172.21.15.139:6834","nonce":2751714076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2160557452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2160557452},{"type":"v1","addr":"172.21.15.150:6835","nonce":2160557452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T09:53:11.720 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 3 2024-04-25T09:53:11.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.719+0000 7f7aea25d640 1 -- 172.21.15.139:0/4026230985 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ad006b4d0 msgr2=0x7f7ad006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.719+0000 7f7aea25d640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ad006b4d0 0x7f7ad006d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4034000 tx=0x7f7ad402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:11.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.719+0000 7f7aea25d640 1 -- 172.21.15.139:0/4026230985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae4155e30 msgr2=0x7f7ae4153bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.719+0000 7f7aea25d640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae4155e30 0x7f7ae4153bb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c500 tx=0x7f7ad800c9c0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.719+0000 7f7aea25d640 1 -- 172.21.15.139:0/4026230985 shutdown_connections 2024-04-25T09:53:11.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.719+0000 7f7aea25d640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae4155e30 0x7f7ae4153bb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.719+0000 7f7aea25d640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ad006b4d0 0x7f7ad006d990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.719+0000 7f7aea25d640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ae415bdc0 0x7f7ae41540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.719+0000 7f7aea25d640 1 --2- 172.21.15.139:0/4026230985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ae40b1e50 0x7f7ae414e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.719+0000 7f7aea25d640 1 -- 172.21.15.139:0/4026230985 >> 172.21.15.139:0/4026230985 conn(0x7f7ae4066f20 msgr2=0x7f7ae406e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:11.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.720+0000 7f7aea25d640 1 -- 172.21.15.139:0/4026230985 shutdown_connections 2024-04-25T09:53:11.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.720+0000 7f7aea25d640 1 -- 172.21.15.139:0/4026230985 wait complete. 2024-04-25T09:53:11.732 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:11.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.827+0000 7fc74ed42640 1 Processor -- start 2024-04-25T09:53:11.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.827+0000 7fc74ed42640 1 -- start start 2024-04-25T09:53:11.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.827+0000 7fc74ed42640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc7481611e0 0x7fc7480b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.827+0000 7fc74ed42640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7480cea00 0x7fc74806c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.827+0000 7fc74ed42640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7480d4740 con 0x7fc7480cee60 2024-04-25T09:53:11.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.827+0000 7fc74ed42640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7480d1fa0 con 0x7fc7481611e0 2024-04-25T09:53:11.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.827+0000 7fc74ed42640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7480d2560 con 0x7fc7480cea00 2024-04-25T09:53:11.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.828+0000 7fc74d2df640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc7480cee60 0x7fc7480d29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56056/0 (socket says 172.21.15.139:56056) 2024-04-25T09:53:11.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.828+0000 7fc74d2df640 1 -- 172.21.15.139:0/1497993921 learned_addr learned my addr 172.21.15.139:0/1497993921 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:11.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.828+0000 7fc747fff640 1 --2- 172.21.15.139:0/1497993921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7480cea00 0x7fc74806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.828+0000 7fc74cade640 1 --2- 172.21.15.139:0/1497993921 >> v2:172.21.15.150:3300/0 conn(0x7fc7481611e0 0x7fc7480b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.828+0000 7fc7477fe640 1 -- 172.21.15.139:0/1497993921 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426626174 0 0) 0x7fc7480d4740 con 0x7fc7480cee60 2024-04-25T09:53:11.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.828+0000 7fc7477fe640 1 -- 172.21.15.139:0/1497993921 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc734003520 con 0x7fc7480cee60 2024-04-25T09:53:11.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.828+0000 7fc747fff640 1 -- 172.21.15.139:0/1497993921 >> v2:172.21.15.150:3300/0 conn(0x7fc7481611e0 msgr2=0x7fc7480b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:11.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.828+0000 7fc747fff640 1 --2- 172.21.15.139:0/1497993921 >> v2:172.21.15.150:3300/0 conn(0x7fc7481611e0 0x7fc7480b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.828+0000 7fc747fff640 1 -- 172.21.15.139:0/1497993921 >> v1:172.21.15.139:6789/0 conn(0x7fc7480cee60 legacy=0x7fc7480d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.828+0000 7fc747fff640 1 -- 172.21.15.139:0/1497993921 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7480b2770 con 0x7fc7480cea00 2024-04-25T09:53:11.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.829+0000 7fc747fff640 1 --2- 172.21.15.139:0/1497993921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7480cea00 0x7fc74806c070 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc7380097d0 tx=0x7fc73802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12db3f0ba4a370d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:11.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.829+0000 7fc7477fe640 1 -- 172.21.15.139:0/1497993921 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738037070 con 0x7fc7480cea00 2024-04-25T09:53:11.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.829+0000 7fc7477fe640 1 -- 172.21.15.139:0/1497993921 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc738032440 con 0x7fc7480cea00 2024-04-25T09:53:11.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.829+0000 7fc7477fe640 1 -- 172.21.15.139:0/1497993921 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7380327f0 con 0x7fc7480cea00 2024-04-25T09:53:11.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.830+0000 7fc74ed42640 1 -- 172.21.15.139:0/1497993921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7480cea00 msgr2=0x7fc74806c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.830+0000 7fc74ed42640 1 --2- 172.21.15.139:0/1497993921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7480cea00 0x7fc74806c070 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc7380097d0 tx=0x7fc73802a1c0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.830+0000 7fc74ed42640 1 -- 172.21.15.139:0/1497993921 shutdown_connections 2024-04-25T09:53:11.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.830+0000 7fc74ed42640 1 --2- 172.21.15.139:0/1497993921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7480cea00 0x7fc74806c070 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.830+0000 7fc74ed42640 1 --2- 172.21.15.139:0/1497993921 >> v2:172.21.15.150:3300/0 conn(0x7fc7481611e0 0x7fc7480b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.830+0000 7fc74ed42640 1 -- 172.21.15.139:0/1497993921 >> 172.21.15.139:0/1497993921 conn(0x7fc7480b5b00 msgr2=0x7fc7480d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:11.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.830+0000 7fc74ed42640 1 -- 172.21.15.139:0/1497993921 shutdown_connections 2024-04-25T09:53:11.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.830+0000 7fc74ed42640 1 -- 172.21.15.139:0/1497993921 wait complete. 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.831+0000 7fc74ed42640 1 Processor -- start 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.831+0000 7fc74ed42640 1 -- start start 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc74ed42640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7480cea00 0x7fc748154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc74ed42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7480cee60 0x7fc74815d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc74ed42640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7481611e0 0x7fc748161c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc74ed42640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7480d21f0 con 0x7fc7480cee60 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc74ed42640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7480b2d30 con 0x7fc7480cea00 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc74ed42640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7481615c0 con 0x7fc7481611e0 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc747fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7480cee60 0x7fc74815d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc747fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7480cee60 0x7fc74815d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44158/0 (socket says 172.21.15.139:44158) 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc747fff640 1 -- 172.21.15.139:0/3795361520 learned_addr learned my addr 172.21.15.139:0/3795361520 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc74d2df640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7481611e0 0x7fc748161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc74cade640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7480cea00 0x7fc748154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc747fff640 1 -- 172.21.15.139:0/3795361520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7481611e0 msgr2=0x7fc748161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc747fff640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7481611e0 0x7fc748161c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc747fff640 1 -- 172.21.15.139:0/3795361520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7480cea00 msgr2=0x7fc748154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:11.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc747fff640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7480cea00 0x7fc748154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:11.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc747fff640 1 -- 172.21.15.139:0/3795361520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74815e1d0 con 0x7fc7480cee60 2024-04-25T09:53:11.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc747fff640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7480cee60 0x7fc74815d7f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc738000c00 tx=0x7fc7380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:11.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc7457fa640 1 -- 172.21.15.139:0/3795361520 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738037040 con 0x7fc7480cee60 2024-04-25T09:53:11.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc7457fa640 1 -- 172.21.15.139:0/3795361520 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7380044c0 con 0x7fc7480cee60 2024-04-25T09:53:11.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc74ed42640 1 -- 172.21.15.139:0/3795361520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7481f6eb0 con 0x7fc7480cee60 2024-04-25T09:53:11.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc7457fa640 1 -- 172.21.15.139:0/3795361520 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7380325f0 con 0x7fc7480cee60 2024-04-25T09:53:11.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.832+0000 7fc74ed42640 1 -- 172.21.15.139:0/3795361520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7481f7470 con 0x7fc7480cee60 2024-04-25T09:53:11.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.833+0000 7fc74ed42640 1 -- 172.21.15.139:0/3795361520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc748074520 con 0x7fc7480cee60 2024-04-25T09:53:11.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.834+0000 7fc7457fa640 1 -- 172.21.15.139:0/3795361520 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc738044020 con 0x7fc7480cee60 2024-04-25T09:53:11.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.834+0000 7fc7457fa640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc72806b4d0 0x7fc72806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:11.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.834+0000 7fc74cade640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc72806b4d0 0x7fc72806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:11.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.834+0000 7fc7457fa640 1 -- 172.21.15.139:0/3795361520 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc7380a9e30 con 0x7fc7480cee60 2024-04-25T09:53:11.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.835+0000 7fc74cade640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc72806b4d0 0x7fc72806d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7400073f0 tx=0x7fc74000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:11.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:11.838+0000 7fc7457fa640 1 -- 172.21.15.139:0/3795361520 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc73804b020 con 0x7fc7480cee60 2024-04-25T09:53:12.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.010+0000 7fc74ed42640 1 -- 172.21.15.139:0/3795361520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc74806ceb0 con 0x7fc7480cee60 2024-04-25T09:53:12.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.011+0000 7fc7457fa640 1 -- 172.21.15.139:0/3795361520 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12932 (secure 0 0 0) 0x7fc738079000 con 0x7fc7480cee60 2024-04-25T09:53:12.013 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:12.013 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":16,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:05.033853+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:12.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.013+0000 7fc74ed42640 1 -- 172.21.15.139:0/3795361520 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc72806b4d0 msgr2=0x7fc72806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.013+0000 7fc74ed42640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc72806b4d0 0x7fc72806d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7400073f0 tx=0x7fc74000b040 comp rx=0 tx=0).stop 2024-04-25T09:53:12.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.013+0000 7fc74ed42640 1 -- 172.21.15.139:0/3795361520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7480cee60 msgr2=0x7fc74815d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.013+0000 7fc74ed42640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7480cee60 0x7fc74815d7f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc738000c00 tx=0x7fc7380046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.013+0000 7fc74ed42640 1 -- 172.21.15.139:0/3795361520 shutdown_connections 2024-04-25T09:53:12.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.013+0000 7fc74ed42640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7480cee60 0x7fc74815d7f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.013+0000 7fc74ed42640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc72806b4d0 0x7fc72806d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.013+0000 7fc74ed42640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7481611e0 0x7fc748161c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.013+0000 7fc74ed42640 1 --2- 172.21.15.139:0/3795361520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7480cea00 0x7fc748154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.013+0000 7fc74ed42640 1 -- 172.21.15.139:0/3795361520 >> 172.21.15.139:0/3795361520 conn(0x7fc7480b5b00 msgr2=0x7fc7480b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:12.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.013+0000 7fc74ed42640 1 -- 172.21.15.139:0/3795361520 shutdown_connections 2024-04-25T09:53:12.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.013+0000 7fc74ed42640 1 -- 172.21.15.139:0/3795361520 wait complete. 2024-04-25T09:53:12.026 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:53:12.026 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:12.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.124+0000 7fc313396640 1 Processor -- start 2024-04-25T09:53:12.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.124+0000 7fc313396640 1 -- start start 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc313396640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc30c15dbf0 0x7fc30c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc313396640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc30c0b9370 0x7fc30c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc313396640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc30c0cec80 0x7fc30c06c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc313396640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30c0d1ee0 con 0x7fc30c0cec80 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc313396640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30c161dc0 con 0x7fc30c0b9370 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc313396640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c1620a0 con 0x7fc30c15dbf0 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc311933640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc30c0cec80 0x7fc30c06c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc311933640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc30c0cec80 0x7fc30c06c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44168/0 (socket says 172.21.15.139:44168) 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc311933640 1 -- 172.21.15.139:0/2000338971 learned_addr learned my addr 172.21.15.139:0/2000338971 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc310931640 1 --2- 172.21.15.139:0/2000338971 >> v2:172.21.15.150:3300/0 conn(0x7fc30c0b9370 0x7fc30c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc311132640 1 --2- 172.21.15.139:0/2000338971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc30c15dbf0 0x7fc30c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc311933640 1 -- 172.21.15.139:0/2000338971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc30c15dbf0 msgr2=0x7fc30c1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc311933640 1 --2- 172.21.15.139:0/2000338971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc30c15dbf0 0x7fc30c1617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc311933640 1 -- 172.21.15.139:0/2000338971 >> v2:172.21.15.150:3300/0 conn(0x7fc30c0b9370 msgr2=0x7fc30c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:12.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc311933640 1 --2- 172.21.15.139:0/2000338971 >> v2:172.21.15.150:3300/0 conn(0x7fc30c0b9370 0x7fc30c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.125+0000 7fc311933640 1 -- 172.21.15.139:0/2000338971 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c162380 con 0x7fc30c0cec80 2024-04-25T09:53:12.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.126+0000 7fc311933640 1 --2- 172.21.15.139:0/2000338971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc30c0cec80 0x7fc30c06c210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc30c068310 tx=0x7fc30802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6877b47576a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:12.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.126+0000 7fc303fff640 1 -- 172.21.15.139:0/2000338971 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308036070 con 0x7fc30c0cec80 2024-04-25T09:53:12.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.126+0000 7fc303fff640 1 -- 172.21.15.139:0/2000338971 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc308031600 con 0x7fc30c0cec80 2024-04-25T09:53:12.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.126+0000 7fc303fff640 1 -- 172.21.15.139:0/2000338971 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3080319d0 con 0x7fc30c0cec80 2024-04-25T09:53:12.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.127+0000 7fc313396640 1 -- 172.21.15.139:0/2000338971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc30c0cec80 msgr2=0x7fc30c06c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.127+0000 7fc313396640 1 --2- 172.21.15.139:0/2000338971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc30c0cec80 0x7fc30c06c210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc30c068310 tx=0x7fc30802a6a0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.127+0000 7fc313396640 1 -- 172.21.15.139:0/2000338971 shutdown_connections 2024-04-25T09:53:12.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.127+0000 7fc313396640 1 --2- 172.21.15.139:0/2000338971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc30c0cec80 0x7fc30c06c210 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.127+0000 7fc313396640 1 --2- 172.21.15.139:0/2000338971 >> v2:172.21.15.150:3300/0 conn(0x7fc30c0b9370 0x7fc30c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.127+0000 7fc313396640 1 --2- 172.21.15.139:0/2000338971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc30c15dbf0 0x7fc30c1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.127+0000 7fc313396640 1 -- 172.21.15.139:0/2000338971 >> 172.21.15.139:0/2000338971 conn(0x7fc30c0b5480 msgr2=0x7fc30c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.127+0000 7fc313396640 1 -- 172.21.15.139:0/2000338971 shutdown_connections 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.127+0000 7fc313396640 1 -- 172.21.15.139:0/2000338971 wait complete. 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.128+0000 7fc313396640 1 Processor -- start 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.128+0000 7fc313396640 1 -- start start 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc313396640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc30c0b9370 0x7fc30c0b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc313396640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc30c0cec80 0x7fc30c0b7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc313396640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc30c15dbf0 0x7fc30c14a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc313396640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c15dfd0 con 0x7fc30c0cec80 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc313396640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c0b97d0 con 0x7fc30c15dbf0 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc313396640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c14b5d0 con 0x7fc30c0b9370 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc310931640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc30c0cec80 0x7fc30c0b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc310931640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc30c0cec80 0x7fc30c0b7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44178/0 (socket says 172.21.15.139:44178) 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc310931640 1 -- 172.21.15.139:0/794735774 learned_addr learned my addr 172.21.15.139:0/794735774 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc311132640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc30c0b9370 0x7fc30c0b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc311933640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc30c15dbf0 0x7fc30c14a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc310931640 1 -- 172.21.15.139:0/794735774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc30c0b9370 msgr2=0x7fc30c0b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc310931640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc30c0b9370 0x7fc30c0b6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc310931640 1 -- 172.21.15.139:0/794735774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc30c15dbf0 msgr2=0x7fc30c14a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc310931640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc30c15dbf0 0x7fc30c14a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc310931640 1 -- 172.21.15.139:0/794735774 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c159a70 con 0x7fc30c0cec80 2024-04-25T09:53:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc310931640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc30c0cec80 0x7fc30c0b7010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00d800 tx=0x7fc2fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc301ffb640 1 -- 172.21.15.139:0/794735774 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc018030 con 0x7fc30c0cec80 2024-04-25T09:53:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc301ffb640 1 -- 172.21.15.139:0/794735774 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc0096a0 con 0x7fc30c0cec80 2024-04-25T09:53:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc313396640 1 -- 172.21.15.139:0/794735774 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30c159d50 con 0x7fc30c0cec80 2024-04-25T09:53:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.129+0000 7fc301ffb640 1 -- 172.21.15.139:0/794735774 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc009870 con 0x7fc30c0cec80 2024-04-25T09:53:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.130+0000 7fc313396640 1 -- 172.21.15.139:0/794735774 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30c15a310 con 0x7fc30c0cec80 2024-04-25T09:53:12.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.130+0000 7fc313396640 1 -- 172.21.15.139:0/794735774 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30c074520 con 0x7fc30c0cec80 2024-04-25T09:53:12.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.131+0000 7fc301ffb640 1 -- 172.21.15.139:0/794735774 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2fc01b070 con 0x7fc30c0cec80 2024-04-25T09:53:12.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.131+0000 7fc301ffb640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc2f006b500 0x7fc2f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.131+0000 7fc311132640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc2f006b500 0x7fc2f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.131+0000 7fc301ffb640 1 -- 172.21.15.139:0/794735774 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc2fc08a120 con 0x7fc30c0cec80 2024-04-25T09:53:12.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.132+0000 7fc311132640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc2f006b500 0x7fc2f006d9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc304006fd0 tx=0x7fc304008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:12.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.135+0000 7fc301ffb640 1 -- 172.21.15.139:0/794735774 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2fc0592f0 con 0x7fc30c0cec80 2024-04-25T09:53:12.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.306+0000 7fc313396640 1 -- 172.21.15.139:0/794735774 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc30c0d1680 con 0x7fc30c0cec80 2024-04-25T09:53:12.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.306+0000 7fc301ffb640 1 -- 172.21.15.139:0/794735774 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc2fc058bf0 con 0x7fc30c0cec80 2024-04-25T09:53:12.308 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:53:12.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.308+0000 7fc313396640 1 -- 172.21.15.139:0/794735774 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc2f006b500 msgr2=0x7fc2f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.308+0000 7fc313396640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc2f006b500 0x7fc2f006d9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc304006fd0 tx=0x7fc304008040 comp rx=0 tx=0).stop 2024-04-25T09:53:12.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.308+0000 7fc313396640 1 -- 172.21.15.139:0/794735774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc30c0cec80 msgr2=0x7fc30c0b7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.308+0000 7fc313396640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc30c0cec80 0x7fc30c0b7010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00d800 tx=0x7fc2fc00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.308+0000 7fc313396640 1 -- 172.21.15.139:0/794735774 shutdown_connections 2024-04-25T09:53:12.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.308+0000 7fc313396640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc30c0cec80 0x7fc30c0b7010 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.308+0000 7fc313396640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc2f006b500 0x7fc2f006d9c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.308+0000 7fc313396640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc30c15dbf0 0x7fc30c14a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.308+0000 7fc313396640 1 --2- 172.21.15.139:0/794735774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc30c0b9370 0x7fc30c0b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.309+0000 7fc313396640 1 -- 172.21.15.139:0/794735774 >> 172.21.15.139:0/794735774 conn(0x7fc30c0b5480 msgr2=0x7fc30c0c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:12.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.309+0000 7fc313396640 1 -- 172.21.15.139:0/794735774 shutdown_connections 2024-04-25T09:53:12.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.309+0000 7fc313396640 1 -- 172.21.15.139:0/794735774 wait complete. 2024-04-25T09:53:12.321 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:53:12.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.416+0000 7fd0bc1e2640 1 Processor -- start 2024-04-25T09:53:12.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.416+0000 7fd0bc1e2640 1 -- start start 2024-04-25T09:53:12.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0bc1e2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd0b40b6af0 0x7fd0b406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0bc1e2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd0b406c5b0 0x7fd0b40d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0bc1e2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b40cf110 con 0x7fd0b41611e0 2024-04-25T09:53:12.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0bc1e2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b40d1fa0 con 0x7fd0b406c5b0 2024-04-25T09:53:12.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0bc1e2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b40d2560 con 0x7fd0b40b6af0 2024-04-25T09:53:12.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0ba77f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd0b406c5b0 0x7fd0b40d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0ba77f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd0b406c5b0 0x7fd0b40d29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44194/0 (socket says 172.21.15.139:44194) 2024-04-25T09:53:12.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0ba77f640 1 -- 172.21.15.139:0/3626619894 learned_addr learned my addr 172.21.15.139:0/3626619894 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:12.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0b977d640 1 --2- 172.21.15.139:0/3626619894 >> v2:172.21.15.150:3300/0 conn(0x7fd0b40b6af0 0x7fd0b406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0ba77f640 1 -- 172.21.15.139:0/3626619894 >> v2:172.21.15.150:3300/0 conn(0x7fd0b40b6af0 msgr2=0x7fd0b406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:12.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0ba77f640 1 --2- 172.21.15.139:0/3626619894 >> v2:172.21.15.150:3300/0 conn(0x7fd0b40b6af0 0x7fd0b406c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0ba77f640 1 -- 172.21.15.139:0/3626619894 >> v1:172.21.15.150:6789/0 conn(0x7fd0b41611e0 legacy=0x7fd0b40cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.417+0000 7fd0ba77f640 1 -- 172.21.15.139:0/3626619894 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b40b6f50 con 0x7fd0b406c5b0 2024-04-25T09:53:12.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.418+0000 7fd0ba77f640 1 --2- 172.21.15.139:0/3626619894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0b406c5b0 0x7fd0b40d29f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40b3410 tx=0x7fd0ac02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180d7461d67224a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:12.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.418+0000 7fd0b8f7c640 1 -- 172.21.15.139:0/3626619894 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac036070 con 0x7fd0b406c5b0 2024-04-25T09:53:12.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.418+0000 7fd0b8f7c640 1 -- 172.21.15.139:0/3626619894 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac031600 con 0x7fd0b406c5b0 2024-04-25T09:53:12.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.418+0000 7fd0b8f7c640 1 -- 172.21.15.139:0/3626619894 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0319d0 con 0x7fd0b406c5b0 2024-04-25T09:53:12.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.419+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/3626619894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0b406c5b0 msgr2=0x7fd0b40d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.419+0000 7fd0bc1e2640 1 --2- 172.21.15.139:0/3626619894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0b406c5b0 0x7fd0b40d29f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40b3410 tx=0x7fd0ac02a730 comp rx=0 tx=0).stop 2024-04-25T09:53:12.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.419+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/3626619894 shutdown_connections 2024-04-25T09:53:12.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.419+0000 7fd0bc1e2640 1 --2- 172.21.15.139:0/3626619894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0b406c5b0 0x7fd0b40d29f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.419+0000 7fd0bc1e2640 1 --2- 172.21.15.139:0/3626619894 >> v2:172.21.15.150:3300/0 conn(0x7fd0b40b6af0 0x7fd0b406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.419+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/3626619894 >> 172.21.15.139:0/3626619894 conn(0x7fd0b40b5b00 msgr2=0x7fd0b40d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:12.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.419+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/3626619894 shutdown_connections 2024-04-25T09:53:12.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.419+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/3626619894 wait complete. 2024-04-25T09:53:12.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.420+0000 7fd0bc1e2640 1 Processor -- start 2024-04-25T09:53:12.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.420+0000 7fd0bc1e2640 1 -- start start 2024-04-25T09:53:12.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.420+0000 7fd0bc1e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0b406c5b0 0x7fd0b40abed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.420+0000 7fd0bc1e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0b40b6af0 0x7fd0b4151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.420+0000 7fd0bc1e2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0b41611e0 0x7fd0b415f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.420+0000 7fd0bc1e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40b2a50 con 0x7fd0b406c5b0 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.420+0000 7fd0bc1e2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40b2770 con 0x7fd0b41611e0 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.420+0000 7fd0bc1e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40b2d30 con 0x7fd0b40b6af0 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0b9f7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0b406c5b0 0x7fd0b40abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0b9f7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0b406c5b0 0x7fd0b40abed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44196/0 (socket says 172.21.15.139:44196) 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0b9f7e640 1 -- 172.21.15.139:0/2471125371 learned_addr learned my addr 172.21.15.139:0/2471125371 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0b977d640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0b40b6af0 0x7fd0b4151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0ba77f640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0b41611e0 0x7fd0b415f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0b9f7e640 1 -- 172.21.15.139:0/2471125371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0b40b6af0 msgr2=0x7fd0b4151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0b9f7e640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0b40b6af0 0x7fd0b4151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0b9f7e640 1 -- 172.21.15.139:0/2471125371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0b41611e0 msgr2=0x7fd0b415f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0b9f7e640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0b41611e0 0x7fd0b415f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0b9f7e640 1 -- 172.21.15.139:0/2471125371 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b40aae70 con 0x7fd0b406c5b0 2024-04-25T09:53:12.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0ba77f640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0b41611e0 0x7fd0b415f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:12.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0b9f7e640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0b406c5b0 0x7fd0b40abed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000c3f0 tx=0x7fd0b000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:12.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0aa7fc640 1 -- 172.21.15.139:0/2471125371 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b00052d0 con 0x7fd0b406c5b0 2024-04-25T09:53:12.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0aa7fc640 1 -- 172.21.15.139:0/2471125371 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b0005490 con 0x7fd0b406c5b0 2024-04-25T09:53:12.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/2471125371 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b40ab110 con 0x7fd0b406c5b0 2024-04-25T09:53:12.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0aa7fc640 1 -- 172.21.15.139:0/2471125371 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0007260 con 0x7fd0b406c5b0 2024-04-25T09:53:12.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.421+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/2471125371 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b40ab6d0 con 0x7fd0b406c5b0 2024-04-25T09:53:12.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.422+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/2471125371 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b4074520 con 0x7fd0b406c5b0 2024-04-25T09:53:12.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.423+0000 7fd0aa7fc640 1 -- 172.21.15.139:0/2471125371 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0b0007400 con 0x7fd0b406c5b0 2024-04-25T09:53:12.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.423+0000 7fd0aa7fc640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd09806b4d0 0x7fd09806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.423+0000 7fd0aa7fc640 1 -- 172.21.15.139:0/2471125371 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd0b0089250 con 0x7fd0b406c5b0 2024-04-25T09:53:12.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.423+0000 7fd0b977d640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd09806b4d0 0x7fd09806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.423+0000 7fd0b977d640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd09806b4d0 0x7fd09806d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40ad590 tx=0x7fd0a4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:12.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.427+0000 7fd0aa7fc640 1 -- 172.21.15.139:0/2471125371 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0b0057e60 con 0x7fd0b406c5b0 2024-04-25T09:53:12.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.599+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/2471125371 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd0b4160a00 con 0x7fd0b406c5b0 2024-04-25T09:53:12.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.599+0000 7fd0aa7fc640 1 -- 172.21.15.139:0/2471125371 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd0b005ce20 con 0x7fd0b406c5b0 2024-04-25T09:53:12.601 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:12.601 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-04-25T09:53:12.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:53:12.602 INFO:teuthology.orchestra.run.smithi139.stdout: "poolnum": 1, 2024-04-25T09:53:12.602 INFO:teuthology.orchestra.run.smithi139.stdout: "poolname": ".mgr" 2024-04-25T09:53:12.602 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:53:12.602 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-04-25T09:53:12.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.601+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/2471125371 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd09806b4d0 msgr2=0x7fd09806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.602+0000 7fd0bc1e2640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd09806b4d0 0x7fd09806d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40ad590 tx=0x7fd0a4009190 comp rx=0 tx=0).stop 2024-04-25T09:53:12.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.602+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/2471125371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0b406c5b0 msgr2=0x7fd0b40abed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.602+0000 7fd0bc1e2640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0b406c5b0 0x7fd0b40abed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000c3f0 tx=0x7fd0b000c8b0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.602+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/2471125371 shutdown_connections 2024-04-25T09:53:12.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.602+0000 7fd0bc1e2640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd09806b4d0 0x7fd09806d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.602+0000 7fd0bc1e2640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0b41611e0 0x7fd0b415f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.602+0000 7fd0bc1e2640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0b40b6af0 0x7fd0b4151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.602+0000 7fd0bc1e2640 1 --2- 172.21.15.139:0/2471125371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0b406c5b0 0x7fd0b40abed0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.602+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/2471125371 >> 172.21.15.139:0/2471125371 conn(0x7fd0b40b5b00 msgr2=0x7fd0b40b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:12.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.602+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/2471125371 shutdown_connections 2024-04-25T09:53:12.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.602+0000 7fd0bc1e2640 1 -- 172.21.15.139:0/2471125371 wait complete. 2024-04-25T09:53:12.614 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:53:12.614 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:53:12.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.713+0000 7f44c433b640 1 Processor -- start 2024-04-25T09:53:12.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.713+0000 7f44c433b640 1 -- start start 2024-04-25T09:53:12.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.714+0000 7f44c433b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f44bc145230 0x7f44bc15be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.714+0000 7f44c433b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f44bc15c3d0 0x7f44bc0711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.714+0000 7f44c433b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc15c960 con 0x7f44bc071700 2024-04-25T09:53:12.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.714+0000 7f44c433b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44bc071b30 con 0x7f44bc15c3d0 2024-04-25T09:53:12.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.714+0000 7f44c433b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44bc15cc40 con 0x7f44bc145230 2024-04-25T09:53:12.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.714+0000 7f44c18d6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f44bc15c3d0 0x7f44bc0711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.714+0000 7f44c28d8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f44bc071700 0x7f44bc06d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56066/0 (socket says 172.21.15.139:56066) 2024-04-25T09:53:12.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.714+0000 7f44c28d8640 1 -- 172.21.15.139:0/1778075434 learned_addr learned my addr 172.21.15.139:0/1778075434 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:12.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.714+0000 7f44c20d7640 1 --2- 172.21.15.139:0/1778075434 >> v2:172.21.15.150:3300/0 conn(0x7f44bc145230 0x7f44bc15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.714+0000 7f44c18d6640 1 -- 172.21.15.139:0/1778075434 >> v2:172.21.15.150:3300/0 conn(0x7f44bc145230 msgr2=0x7f44bc15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:12.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.714+0000 7f44c18d6640 1 --2- 172.21.15.139:0/1778075434 >> v2:172.21.15.150:3300/0 conn(0x7f44bc145230 0x7f44bc15be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.714+0000 7f44c18d6640 1 -- 172.21.15.139:0/1778075434 >> v1:172.21.15.139:6789/0 conn(0x7f44bc071700 legacy=0x7f44bc06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.715+0000 7f44c18d6640 1 -- 172.21.15.139:0/1778075434 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc15d230 con 0x7f44bc15c3d0 2024-04-25T09:53:12.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.715+0000 7f44c10d5640 1 -- 172.21.15.139:0/1778075434 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602905482 0 0) 0x7f44bc15c960 con 0x7f44bc071700 2024-04-25T09:53:12.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.715+0000 7f44c18d6640 1 --2- 172.21.15.139:0/1778075434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44bc15c3d0 0x7f44bc0711c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0b4990 tx=0x7f44ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baff57aab7aea56f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:12.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.715+0000 7f44c10d5640 1 -- 172.21.15.139:0/1778075434 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac037070 con 0x7f44bc15c3d0 2024-04-25T09:53:12.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.715+0000 7f44c10d5640 1 -- 172.21.15.139:0/1778075434 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac032440 con 0x7f44bc15c3d0 2024-04-25T09:53:12.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.715+0000 7f44c10d5640 1 -- 172.21.15.139:0/1778075434 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac032810 con 0x7f44bc15c3d0 2024-04-25T09:53:12.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.716+0000 7f44c433b640 1 -- 172.21.15.139:0/1778075434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44bc15c3d0 msgr2=0x7f44bc0711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.716+0000 7f44c433b640 1 --2- 172.21.15.139:0/1778075434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44bc15c3d0 0x7f44bc0711c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0b4990 tx=0x7f44ac02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.716+0000 7f44c433b640 1 -- 172.21.15.139:0/1778075434 shutdown_connections 2024-04-25T09:53:12.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.716+0000 7f44c433b640 1 --2- 172.21.15.139:0/1778075434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44bc15c3d0 0x7f44bc0711c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.716+0000 7f44c433b640 1 --2- 172.21.15.139:0/1778075434 >> v2:172.21.15.150:3300/0 conn(0x7f44bc145230 0x7f44bc15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.716+0000 7f44c433b640 1 -- 172.21.15.139:0/1778075434 >> 172.21.15.139:0/1778075434 conn(0x7f44bc067000 msgr2=0x7f44bc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:12.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.716+0000 7f44c433b640 1 -- 172.21.15.139:0/1778075434 shutdown_connections 2024-04-25T09:53:12.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.716+0000 7f44c433b640 1 -- 172.21.15.139:0/1778075434 wait complete. 2024-04-25T09:53:12.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.717+0000 7f44c433b640 1 Processor -- start 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.717+0000 7f44c433b640 1 -- start start 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c433b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44bc071700 0x7f44bc15b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c433b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44bc145230 0x7f44bc0c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c433b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44bc15c3d0 0x7f44bc0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c433b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc15ea40 con 0x7f44bc145230 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c433b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc06e4b0 con 0x7f44bc071700 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c433b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc15ebc0 con 0x7f44bc15c3d0 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c18d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44bc145230 0x7f44bc0c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c18d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44bc145230 0x7f44bc0c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44218/0 (socket says 172.21.15.139:44218) 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c18d6640 1 -- 172.21.15.139:0/2898794947 learned_addr learned my addr 172.21.15.139:0/2898794947 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c28d8640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44bc15c3d0 0x7f44bc0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c20d7640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44bc071700 0x7f44bc15b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c18d6640 1 -- 172.21.15.139:0/2898794947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44bc15c3d0 msgr2=0x7f44bc0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c18d6640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44bc15c3d0 0x7f44bc0cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c18d6640 1 -- 172.21.15.139:0/2898794947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44bc071700 msgr2=0x7f44bc15b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c18d6640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44bc071700 0x7f44bc15b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c18d6640 1 -- 172.21.15.139:0/2898794947 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc0aa700 con 0x7f44bc145230 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.718+0000 7f44c28d8640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44bc15c3d0 0x7f44bc0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.719+0000 7f44c18d6640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44bc145230 0x7f44bc0c4560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009890 tx=0x7f44ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.719+0000 7f44b2ffd640 1 -- 172.21.15.139:0/2898794947 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac037040 con 0x7f44bc145230 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.719+0000 7f44b2ffd640 1 -- 172.21.15.139:0/2898794947 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac042070 con 0x7f44bc145230 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.719+0000 7f44c433b640 1 -- 172.21.15.139:0/2898794947 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44bc0aa9e0 con 0x7f44bc145230 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.719+0000 7f44b2ffd640 1 -- 172.21.15.139:0/2898794947 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac030bd0 con 0x7f44bc145230 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.719+0000 7f44c433b640 1 -- 172.21.15.139:0/2898794947 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc0aafa0 con 0x7f44bc145230 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.720+0000 7f44c433b640 1 -- 172.21.15.139:0/2898794947 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc074520 con 0x7f44bc145230 2024-04-25T09:53:12.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.720+0000 7f44b2ffd640 1 -- 172.21.15.139:0/2898794947 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44ac047460 con 0x7f44bc145230 2024-04-25T09:53:12.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.720+0000 7f44b2ffd640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f44a006b4d0 0x7f44a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:12.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.721+0000 7f44b2ffd640 1 -- 172.21.15.139:0/2898794947 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f44ac0aaa00 con 0x7f44bc145230 2024-04-25T09:53:12.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.721+0000 7f44c20d7640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f44a006b4d0 0x7f44a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:12.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.721+0000 7f44c20d7640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f44a006b4d0 0x7f44a006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009f30 tx=0x7f44b800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:12.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.725+0000 7f44b2ffd640 1 -- 172.21.15.139:0/2898794947 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44ac032e20 con 0x7f44bc145230 2024-04-25T09:53:12.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:12.895+0000 7f44c433b640 1 -- 172.21.15.139:0/2898794947 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f44bc15ef90 con 0x7f44bc145230 2024-04-25T09:53:13.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.686+0000 7f44b2ffd640 1 -- 172.21.15.139:0/2898794947 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f44ac079c60 con 0x7f44bc145230 2024-04-25T09:53:13.688 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' created 2024-04-25T09:53:13.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.689+0000 7f44c433b640 1 -- 172.21.15.139:0/2898794947 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f44a006b4d0 msgr2=0x7f44a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.689+0000 7f44c433b640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f44a006b4d0 0x7f44a006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009f30 tx=0x7f44b800b040 comp rx=0 tx=0).stop 2024-04-25T09:53:13.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.689+0000 7f44c433b640 1 -- 172.21.15.139:0/2898794947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44bc145230 msgr2=0x7f44bc0c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.689+0000 7f44c433b640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44bc145230 0x7f44bc0c4560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009890 tx=0x7f44ac0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.689+0000 7f44c433b640 1 -- 172.21.15.139:0/2898794947 shutdown_connections 2024-04-25T09:53:13.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.689+0000 7f44c433b640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44bc145230 0x7f44bc0c4560 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.689+0000 7f44c433b640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f44a006b4d0 0x7f44a006d990 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009f30 tx=0x7f44b800b040 comp rx=0 tx=0).stop 2024-04-25T09:53:13.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.689+0000 7f44c433b640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44bc15c3d0 0x7f44bc0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.689+0000 7f44c433b640 1 --2- 172.21.15.139:0/2898794947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44bc071700 0x7f44bc15b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.689+0000 7f44c433b640 1 -- 172.21.15.139:0/2898794947 >> 172.21.15.139:0/2898794947 conn(0x7f44bc067000 msgr2=0x7f44bc1443d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:13.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.690+0000 7f44c433b640 1 -- 172.21.15.139:0/2898794947 shutdown_connections 2024-04-25T09:53:13.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.690+0000 7f44c433b640 1 -- 172.21.15.139:0/2898794947 wait complete. 2024-04-25T09:53:13.708 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T09:53:13.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.813+0000 7fb84675c640 1 Processor -- start 2024-04-25T09:53:13.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.813+0000 7fb84675c640 1 -- start start 2024-04-25T09:53:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.813+0000 7fb84675c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb8400cd610 0x7fb8400d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.813+0000 7fb84675c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb840161fd0 0x7fb8400d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.813+0000 7fb84675c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb840099ca0 con 0x7fb84009ad40 2024-04-25T09:53:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.813+0000 7fb84675c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb840099a50 con 0x7fb8400cd610 2024-04-25T09:53:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.813+0000 7fb84675c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb840099f80 con 0x7fb840161fd0 2024-04-25T09:53:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.814+0000 7fb83f7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb84009ad40 0x7fb8401618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56072/0 (socket says 172.21.15.139:56072) 2024-04-25T09:53:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.814+0000 7fb83f7fe640 1 -- 172.21.15.139:0/3786317624 learned_addr learned my addr 172.21.15.139:0/3786317624 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:13.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.814+0000 7fb844cf9640 1 --2- 172.21.15.139:0/3786317624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb840161fd0 0x7fb8400d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.814+0000 7fb83ffff640 1 --2- 172.21.15.139:0/3786317624 >> v2:172.21.15.150:3300/0 conn(0x7fb8400cd610 0x7fb8400d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.814+0000 7fb83effd640 1 -- 172.21.15.139:0/3786317624 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118542255 0 0) 0x7fb840099ca0 con 0x7fb84009ad40 2024-04-25T09:53:13.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.814+0000 7fb83effd640 1 -- 172.21.15.139:0/3786317624 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb828003520 con 0x7fb84009ad40 2024-04-25T09:53:13.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.814+0000 7fb83ffff640 1 -- 172.21.15.139:0/3786317624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb840161fd0 msgr2=0x7fb8400d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:13.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.814+0000 7fb83ffff640 1 --2- 172.21.15.139:0/3786317624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb840161fd0 0x7fb8400d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.814+0000 7fb83effd640 1 -- 172.21.15.139:0/3786317624 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 547546523 0 0) 0x7fb828003520 con 0x7fb84009ad40 2024-04-25T09:53:13.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.814+0000 7fb83ffff640 1 -- 172.21.15.139:0/3786317624 >> v1:172.21.15.139:6789/0 conn(0x7fb84009ad40 legacy=0x7fb8401618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.814+0000 7fb83ffff640 1 -- 172.21.15.139:0/3786317624 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84009a260 con 0x7fb8400cd610 2024-04-25T09:53:13.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.814+0000 7fb844cf9640 1 --2- 172.21.15.139:0/3786317624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb840161fd0 0x7fb8400d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:13.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.815+0000 7fb83ffff640 1 --2- 172.21.15.139:0/3786317624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8400cd610 0x7fb8400d0370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb8300097a0 tx=0x7fb83002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a26133f02daf8720 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:13.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.815+0000 7fb83effd640 1 -- 172.21.15.139:0/3786317624 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb830037070 con 0x7fb8400cd610 2024-04-25T09:53:13.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.815+0000 7fb83effd640 1 -- 172.21.15.139:0/3786317624 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83002ac30 con 0x7fb8400cd610 2024-04-25T09:53:13.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.815+0000 7fb83effd640 1 -- 172.21.15.139:0/3786317624 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834001670 con 0x7fb8400cd610 2024-04-25T09:53:13.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.815+0000 7fb84675c640 1 -- 172.21.15.139:0/3786317624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8400cd610 msgr2=0x7fb8400d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.815+0000 7fb84675c640 1 --2- 172.21.15.139:0/3786317624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8400cd610 0x7fb8400d0370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb8300097a0 tx=0x7fb83002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.816+0000 7fb84675c640 1 -- 172.21.15.139:0/3786317624 shutdown_connections 2024-04-25T09:53:13.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.816+0000 7fb84675c640 1 --2- 172.21.15.139:0/3786317624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb840161fd0 0x7fb8400d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.816+0000 7fb84675c640 1 --2- 172.21.15.139:0/3786317624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8400cd610 0x7fb8400d0370 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.816+0000 7fb84675c640 1 -- 172.21.15.139:0/3786317624 >> 172.21.15.139:0/3786317624 conn(0x7fb8400b31f0 msgr2=0x7fb8400b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:13.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.816+0000 7fb84675c640 1 -- 172.21.15.139:0/3786317624 shutdown_connections 2024-04-25T09:53:13.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.816+0000 7fb84675c640 1 -- 172.21.15.139:0/3786317624 wait complete. 2024-04-25T09:53:13.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.817+0000 7fb84675c640 1 Processor -- start 2024-04-25T09:53:13.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.817+0000 7fb84675c640 1 -- start start 2024-04-25T09:53:13.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb84675c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb84009ad40 0x7fb8400c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb84675c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400cd610 0x7fb84015c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb84675c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb840161fd0 0x7fb8401f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb84675c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400cd9f0 con 0x7fb8400cd610 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb84675c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84009a820 con 0x7fb840161fd0 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb84675c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400cdb70 con 0x7fb84009ad40 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400cd610 0x7fb84015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400cd610 0x7fb84015c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44234/0 (socket says 172.21.15.139:44234) 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83f7fe640 1 -- 172.21.15.139:0/4184235977 learned_addr learned my addr 172.21.15.139:0/4184235977 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83ffff640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb84009ad40 0x7fb8400c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb844cf9640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb840161fd0 0x7fb8401f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83f7fe640 1 -- 172.21.15.139:0/4184235977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb84009ad40 msgr2=0x7fb8400c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83f7fe640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb84009ad40 0x7fb8400c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83f7fe640 1 -- 172.21.15.139:0/4184235977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb840161fd0 msgr2=0x7fb8401f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83f7fe640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb840161fd0 0x7fb8401f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83f7fe640 1 -- 172.21.15.139:0/4184235977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8400c8030 con 0x7fb8400cd610 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83ffff640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb84009ad40 0x7fb8400c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb844cf9640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb840161fd0 0x7fb8401f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:13.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83f7fe640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400cd610 0x7fb84015c800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb83400c780 tx=0x7fb83400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:13.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83cff9640 1 -- 172.21.15.139:0/4184235977 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834017070 con 0x7fb8400cd610 2024-04-25T09:53:13.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83cff9640 1 -- 172.21.15.139:0/4184235977 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb834005ad0 con 0x7fb8400cd610 2024-04-25T09:53:13.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb84675c640 1 -- 172.21.15.139:0/4184235977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8401e9330 con 0x7fb8400cd610 2024-04-25T09:53:13.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb83cff9640 1 -- 172.21.15.139:0/4184235977 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834007070 con 0x7fb8400cd610 2024-04-25T09:53:13.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.818+0000 7fb84675c640 1 -- 172.21.15.139:0/4184235977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8401e98f0 con 0x7fb8400cd610 2024-04-25T09:53:13.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.819+0000 7fb84675c640 1 -- 172.21.15.139:0/4184235977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb840074520 con 0x7fb8400cd610 2024-04-25T09:53:13.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.820+0000 7fb83cff9640 1 -- 172.21.15.139:0/4184235977 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb83401a070 con 0x7fb8400cd610 2024-04-25T09:53:13.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.820+0000 7fb83cff9640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb82c06b4d0 0x7fb82c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:13.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.820+0000 7fb83cff9640 1 -- 172.21.15.139:0/4184235977 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb8340895f0 con 0x7fb8400cd610 2024-04-25T09:53:13.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.820+0000 7fb83ffff640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb82c06b4d0 0x7fb82c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:13.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.820+0000 7fb83ffff640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb82c06b4d0 0x7fb82c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb830034000 tx=0x7fb83002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:13.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.824+0000 7fb83cff9640 1 -- 172.21.15.139:0/4184235977 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8340585d0 con 0x7fb8400cd610 2024-04-25T09:53:13.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:13.996+0000 7fb84675c640 1 -- 172.21.15.139:0/4184235977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fb8400b3f70 con 0x7fb8400cd610 2024-04-25T09:53:14.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.689+0000 7fb83cff9640 1 -- 172.21.15.139:0/4184235977 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fb834057ed0 con 0x7fb8400cd610 2024-04-25T09:53:14.691 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' created 2024-04-25T09:53:14.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.692+0000 7fb84675c640 1 -- 172.21.15.139:0/4184235977 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb82c06b4d0 msgr2=0x7fb82c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.692+0000 7fb84675c640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb82c06b4d0 0x7fb82c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb830034000 tx=0x7fb83002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:14.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.692+0000 7fb84675c640 1 -- 172.21.15.139:0/4184235977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400cd610 msgr2=0x7fb84015c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.692+0000 7fb84675c640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400cd610 0x7fb84015c800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb83400c780 tx=0x7fb83400cc40 comp rx=0 tx=0).stop 2024-04-25T09:53:14.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.692+0000 7fb84675c640 1 -- 172.21.15.139:0/4184235977 shutdown_connections 2024-04-25T09:53:14.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.692+0000 7fb84675c640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400cd610 0x7fb84015c800 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.692+0000 7fb84675c640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb82c06b4d0 0x7fb82c06d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.692+0000 7fb84675c640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb840161fd0 0x7fb8401f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.692+0000 7fb84675c640 1 --2- 172.21.15.139:0/4184235977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb84009ad40 0x7fb8400c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.692+0000 7fb84675c640 1 -- 172.21.15.139:0/4184235977 >> 172.21.15.139:0/4184235977 conn(0x7fb8400b31f0 msgr2=0x7fb8400b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:14.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.692+0000 7fb84675c640 1 -- 172.21.15.139:0/4184235977 shutdown_connections 2024-04-25T09:53:14.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.692+0000 7fb84675c640 1 -- 172.21.15.139:0/4184235977 wait complete. 2024-04-25T09:53:14.721 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:53:14.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.822+0000 7f06e066b640 1 Processor -- start 2024-04-25T09:53:14.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.822+0000 7f06e066b640 1 -- start start 2024-04-25T09:53:14.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06e066b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f06d815dbf0 0x7f06d81617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06e066b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06d80cee50 0x7f06d80d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06e066b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d8161dc0 con 0x7f06d80b9370 2024-04-25T09:53:14.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06e066b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d80d1ee0 con 0x7f06d815dbf0 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06e066b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d81620a0 con 0x7f06d80cee50 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06de407640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f06d815dbf0 0x7f06d81617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06de407640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f06d815dbf0 0x7f06d81617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44242/0 (socket says 172.21.15.139:44242) 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06ddc06640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f06d80b9370 0x7f06d80ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56080/0 (socket says 172.21.15.139:56080) 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06de407640 1 -- 172.21.15.139:0/3876005705 learned_addr learned my addr 172.21.15.139:0/3876005705 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06dec08640 1 --2- 172.21.15.139:0/3876005705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06d80cee50 0x7f06d80d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06de407640 1 -- 172.21.15.139:0/3876005705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06d80cee50 msgr2=0x7f06d80d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06de407640 1 --2- 172.21.15.139:0/3876005705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06d80cee50 0x7f06d80d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06de407640 1 -- 172.21.15.139:0/3876005705 >> v1:172.21.15.139:6789/0 conn(0x7f06d80b9370 legacy=0x7f06d80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06de407640 1 -- 172.21.15.139:0/3876005705 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d8162380 con 0x7f06d815dbf0 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.823+0000 7f06dd405640 1 -- 172.21.15.139:0/3876005705 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345953049 0 0) 0x7f06d8161dc0 con 0x7f06d80b9370 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.824+0000 7f06de407640 1 --2- 172.21.15.139:0/3876005705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d815dbf0 0x7f06d81617e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009870 tx=0x7f06d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cee2166b7f987f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.824+0000 7f06dd405640 1 -- 172.21.15.139:0/3876005705 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4037070 con 0x7f06d815dbf0 2024-04-25T09:53:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.824+0000 7f06dd405640 1 -- 172.21.15.139:0/3876005705 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d4032440 con 0x7f06d815dbf0 2024-04-25T09:53:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.824+0000 7f06dd405640 1 -- 172.21.15.139:0/3876005705 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4032810 con 0x7f06d815dbf0 2024-04-25T09:53:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.824+0000 7f06e066b640 1 -- 172.21.15.139:0/3876005705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d815dbf0 msgr2=0x7f06d81617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.824+0000 7f06e066b640 1 --2- 172.21.15.139:0/3876005705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d815dbf0 0x7f06d81617e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009870 tx=0x7f06d402a270 comp rx=0 tx=0).stop 2024-04-25T09:53:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.824+0000 7f06e066b640 1 -- 172.21.15.139:0/3876005705 shutdown_connections 2024-04-25T09:53:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.824+0000 7f06e066b640 1 --2- 172.21.15.139:0/3876005705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06d80cee50 0x7f06d80d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.824+0000 7f06e066b640 1 --2- 172.21.15.139:0/3876005705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d815dbf0 0x7f06d81617e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.824+0000 7f06e066b640 1 -- 172.21.15.139:0/3876005705 >> 172.21.15.139:0/3876005705 conn(0x7f06d80b5480 msgr2=0x7f06d80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.824+0000 7f06e066b640 1 -- 172.21.15.139:0/3876005705 shutdown_connections 2024-04-25T09:53:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.824+0000 7f06e066b640 1 -- 172.21.15.139:0/3876005705 wait complete. 2024-04-25T09:53:14.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06e066b640 1 Processor -- start 2024-04-25T09:53:14.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06e066b640 1 -- start start 2024-04-25T09:53:14.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06e066b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06d80b9370 0x7f06d8151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06e066b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06d80cee50 0x7f06d80cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06e066b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d815dbf0 0x7f06d80cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06e066b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d815dfd0 con 0x7f06d815dbf0 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06e066b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80b97d0 con 0x7f06d80b9370 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06e066b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d8153ee0 con 0x7f06d80cee50 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06dec08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d815dbf0 0x7f06d80cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06dec08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d815dbf0 0x7f06d80cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44248/0 (socket says 172.21.15.139:44248) 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06dec08640 1 -- 172.21.15.139:0/1002073337 learned_addr learned my addr 172.21.15.139:0/1002073337 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06de407640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06d80b9370 0x7f06d8151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.826+0000 7f06ddc06640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06d80cee50 0x7f06d80cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.827+0000 7f06dec08640 1 -- 172.21.15.139:0/1002073337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06d80cee50 msgr2=0x7f06d80cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.827+0000 7f06dec08640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06d80cee50 0x7f06d80cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.827+0000 7f06dec08640 1 -- 172.21.15.139:0/1002073337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06d80b9370 msgr2=0x7f06d8151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.827+0000 7f06dec08640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06d80b9370 0x7f06d8151e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.827+0000 7f06dec08640 1 -- 172.21.15.139:0/1002073337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d81f43e0 con 0x7f06d815dbf0 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.827+0000 7f06dec08640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d815dbf0 0x7f06d80cc510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f06d000cd20 tx=0x7f06d000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.827+0000 7f06ceffd640 1 -- 172.21.15.139:0/1002073337 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d0003b90 con 0x7f06d815dbf0 2024-04-25T09:53:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.827+0000 7f06ceffd640 1 -- 172.21.15.139:0/1002073337 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d0003d30 con 0x7f06d815dbf0 2024-04-25T09:53:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.827+0000 7f06e066b640 1 -- 172.21.15.139:0/1002073337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d81541a0 con 0x7f06d815dbf0 2024-04-25T09:53:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.827+0000 7f06ceffd640 1 -- 172.21.15.139:0/1002073337 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d0004240 con 0x7f06d815dbf0 2024-04-25T09:53:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.827+0000 7f06e066b640 1 -- 172.21.15.139:0/1002073337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d81523a0 con 0x7f06d815dbf0 2024-04-25T09:53:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.828+0000 7f06e066b640 1 -- 172.21.15.139:0/1002073337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d8074520 con 0x7f06d815dbf0 2024-04-25T09:53:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.828+0000 7f06ceffd640 1 -- 172.21.15.139:0/1002073337 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06d00043e0 con 0x7f06d815dbf0 2024-04-25T09:53:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.829+0000 7f06ceffd640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f06bc06b4d0 0x7f06bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.829+0000 7f06ceffd640 1 -- 172.21.15.139:0/1002073337 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f06d008aae0 con 0x7f06d815dbf0 2024-04-25T09:53:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.829+0000 7f06de407640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f06bc06b4d0 0x7f06bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.829+0000 7f06de407640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f06bc06b4d0 0x7f06bc06d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f06d4034000 tx=0x7f06d402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:14.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:14.833+0000 7f06ceffd640 1 -- 172.21.15.139:0/1002073337 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06d0059400 con 0x7f06d815dbf0 2024-04-25T09:53:15.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.040+0000 7f06e066b640 1 -- 172.21.15.139:0/1002073337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f06d80ca2d0 con 0x7f06d815dbf0 2024-04-25T09:53:15.051 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:15.053 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:15.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.702+0000 7f06ceffd640 1 -- 172.21.15.139:0/1002073337 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f06d005d4a0 con 0x7f06d815dbf0 2024-04-25T09:53:15.704 INFO:teuthology.orchestra.run.smithi139.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T09:53:15.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.706+0000 7f06e066b640 1 -- 172.21.15.139:0/1002073337 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f06bc06b4d0 msgr2=0x7f06bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.706+0000 7f06e066b640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f06bc06b4d0 0x7f06bc06d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f06d4034000 tx=0x7f06d402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:15.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.706+0000 7f06e066b640 1 -- 172.21.15.139:0/1002073337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d815dbf0 msgr2=0x7f06d80cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.706+0000 7f06e066b640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d815dbf0 0x7f06d80cc510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f06d000cd20 tx=0x7f06d000a660 comp rx=0 tx=0).stop 2024-04-25T09:53:15.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.706+0000 7f06e066b640 1 -- 172.21.15.139:0/1002073337 shutdown_connections 2024-04-25T09:53:15.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.706+0000 7f06e066b640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06d815dbf0 0x7f06d80cc510 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.706+0000 7f06e066b640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f06bc06b4d0 0x7f06bc06d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.706+0000 7f06e066b640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06d80cee50 0x7f06d80cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.706+0000 7f06e066b640 1 --2- 172.21.15.139:0/1002073337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06d80b9370 0x7f06d8151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.706+0000 7f06e066b640 1 -- 172.21.15.139:0/1002073337 >> 172.21.15.139:0/1002073337 conn(0x7f06d80b5480 msgr2=0x7f06d80d2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:15.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.706+0000 7f06e066b640 1 -- 172.21.15.139:0/1002073337 shutdown_connections 2024-04-25T09:53:15.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.707+0000 7f06e066b640 1 -- 172.21.15.139:0/1002073337 wait complete. 2024-04-25T09:53:15.737 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T09:53:15.738 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T09:53:15.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.839+0000 7f3f76734640 1 Processor -- start 2024-04-25T09:53:15.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.839+0000 7f3f76734640 1 -- start start 2024-04-25T09:53:15.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f76734640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3f7015dbf0 0x7f3f701617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f76734640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3f700cee50 0x7f3f700d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f76734640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f70161dc0 con 0x7f3f700b9370 2024-04-25T09:53:15.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f76734640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f700d1ee0 con 0x7f3f700cee50 2024-04-25T09:53:15.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f76734640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f701620a0 con 0x7f3f7015dbf0 2024-04-25T09:53:15.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f6f7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3f700b9370 0x7f3f700ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56094/0 (socket says 172.21.15.139:56094) 2024-04-25T09:53:15.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f74cd1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3f700cee50 0x7f3f700d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f6f7fe640 1 -- 172.21.15.139:0/812092425 learned_addr learned my addr 172.21.15.139:0/812092425 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:15.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f6ffff640 1 --2- 172.21.15.139:0/812092425 >> v2:172.21.15.150:3300/0 conn(0x7f3f7015dbf0 0x7f3f701617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f74cd1640 1 -- 172.21.15.139:0/812092425 >> v2:172.21.15.150:3300/0 conn(0x7f3f7015dbf0 msgr2=0x7f3f701617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:15.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f74cd1640 1 --2- 172.21.15.139:0/812092425 >> v2:172.21.15.150:3300/0 conn(0x7f3f7015dbf0 0x7f3f701617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f74cd1640 1 -- 172.21.15.139:0/812092425 >> v1:172.21.15.139:6789/0 conn(0x7f3f700b9370 legacy=0x7f3f700ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f74cd1640 1 -- 172.21.15.139:0/812092425 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f70162380 con 0x7f3f700cee50 2024-04-25T09:53:15.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.840+0000 7f3f6effd640 1 -- 172.21.15.139:0/812092425 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445264788 0 0) 0x7f3f70161dc0 con 0x7f3f700b9370 2024-04-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.841+0000 7f3f74cd1640 1 --2- 172.21.15.139:0/812092425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f700cee50 0x7f3f700d2370 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3f7006d330 tx=0x7f3f6802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409a4eb722e3aec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.841+0000 7f3f6effd640 1 -- 172.21.15.139:0/812092425 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68007740 con 0x7f3f700cee50 2024-04-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.841+0000 7f3f6effd640 1 -- 172.21.15.139:0/812092425 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f680078e0 con 0x7f3f700cee50 2024-04-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.841+0000 7f3f6effd640 1 -- 172.21.15.139:0/812092425 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68007cb0 con 0x7f3f700cee50 2024-04-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.841+0000 7f3f76734640 1 -- 172.21.15.139:0/812092425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f700cee50 msgr2=0x7f3f700d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.841+0000 7f3f76734640 1 --2- 172.21.15.139:0/812092425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f700cee50 0x7f3f700d2370 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3f7006d330 tx=0x7f3f6802a680 comp rx=0 tx=0).stop 2024-04-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.841+0000 7f3f76734640 1 -- 172.21.15.139:0/812092425 shutdown_connections 2024-04-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.841+0000 7f3f76734640 1 --2- 172.21.15.139:0/812092425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f700cee50 0x7f3f700d2370 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.841+0000 7f3f76734640 1 --2- 172.21.15.139:0/812092425 >> v2:172.21.15.150:3300/0 conn(0x7f3f7015dbf0 0x7f3f701617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.841+0000 7f3f76734640 1 -- 172.21.15.139:0/812092425 >> 172.21.15.139:0/812092425 conn(0x7f3f700b5480 msgr2=0x7f3f700b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.841+0000 7f3f76734640 1 -- 172.21.15.139:0/812092425 shutdown_connections 2024-04-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.841+0000 7f3f76734640 1 -- 172.21.15.139:0/812092425 wait complete. 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.842+0000 7f3f76734640 1 Processor -- start 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.843+0000 7f3f76734640 1 -- start start 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.843+0000 7f3f76734640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f700b9370 0x7f3f7015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.843+0000 7f3f76734640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f700cee50 0x7f3f700d3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.843+0000 7f3f76734640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f7015dbf0 0x7f3f700d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.843+0000 7f3f76734640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7015dfd0 con 0x7f3f700b9370 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.843+0000 7f3f76734640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f700b97d0 con 0x7f3f700cee50 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.843+0000 7f3f76734640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7015fb30 con 0x7f3f7015dbf0 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.843+0000 7f3f6ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f700b9370 0x7f3f7015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.843+0000 7f3f6ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f700b9370 0x7f3f7015ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44260/0 (socket says 172.21.15.139:44260) 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.843+0000 7f3f6ffff640 1 -- 172.21.15.139:0/1907432855 learned_addr learned my addr 172.21.15.139:0/1907432855 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f6f7fe640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f700cee50 0x7f3f700d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f74cd1640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f7015dbf0 0x7f3f700d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f6ffff640 1 -- 172.21.15.139:0/1907432855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f7015dbf0 msgr2=0x7f3f700d3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f6ffff640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f7015dbf0 0x7f3f700d3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f6ffff640 1 -- 172.21.15.139:0/1907432855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f700cee50 msgr2=0x7f3f700d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f6ffff640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f700cee50 0x7f3f700d3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f6ffff640 1 -- 172.21.15.139:0/1907432855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7015f850 con 0x7f3f700b9370 2024-04-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f74cd1640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f7015dbf0 0x7f3f700d3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f6ffff640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f700b9370 0x7f3f7015ce30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000d830 tx=0x7f3f6000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f6cff9640 1 -- 172.21.15.139:0/1907432855 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000b6c0 con 0x7f3f700b9370 2024-04-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f6cff9640 1 -- 172.21.15.139:0/1907432855 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f60019070 con 0x7f3f700b9370 2024-04-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f76734640 1 -- 172.21.15.139:0/1907432855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f70150730 con 0x7f3f700b9370 2024-04-25T09:53:15.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f6cff9640 1 -- 172.21.15.139:0/1907432855 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000ba80 con 0x7f3f700b9370 2024-04-25T09:53:15.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.844+0000 7f3f76734640 1 -- 172.21.15.139:0/1907432855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f70150cf0 con 0x7f3f700b9370 2024-04-25T09:53:15.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.845+0000 7f3f76734640 1 -- 172.21.15.139:0/1907432855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f70074520 con 0x7f3f700b9370 2024-04-25T09:53:15.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.845+0000 7f3f6cff9640 1 -- 172.21.15.139:0/1907432855 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f6001e460 con 0x7f3f700b9370 2024-04-25T09:53:15.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.846+0000 7f3f6cff9640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f5c06b500 0x7f3f5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:15.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.846+0000 7f3f6f7fe640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f5c06b500 0x7f3f5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:15.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.846+0000 7f3f6cff9640 1 -- 172.21.15.139:0/1907432855 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f6008a800 con 0x7f3f700b9370 2024-04-25T09:53:15.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.846+0000 7f3f6f7fe640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f5c06b500 0x7f3f5c06d9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f7015e390 tx=0x7f3f64007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:15.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:15.850+0000 7f3f6cff9640 1 -- 172.21.15.139:0/1907432855 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f60059680 con 0x7f3f700b9370 2024-04-25T09:53:16.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.030+0000 7f3f76734640 1 -- 172.21.15.139:0/1907432855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f3f700d1680 con 0x7f3f700b9370 2024-04-25T09:53:16.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.719+0000 7f3f6cff9640 1 -- 172.21.15.139:0/1907432855 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f3f60058f80 con 0x7f3f700b9370 2024-04-25T09:53:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.723+0000 7f3f76734640 1 -- 172.21.15.139:0/1907432855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f5c06b500 msgr2=0x7f3f5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.723+0000 7f3f76734640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f5c06b500 0x7f3f5c06d9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f7015e390 tx=0x7f3f64007de0 comp rx=0 tx=0).stop 2024-04-25T09:53:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.723+0000 7f3f76734640 1 -- 172.21.15.139:0/1907432855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f700b9370 msgr2=0x7f3f7015ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.723+0000 7f3f76734640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f700b9370 0x7f3f7015ce30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000d830 tx=0x7f3f6000dcf0 comp rx=0 tx=0).stop 2024-04-25T09:53:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.723+0000 7f3f76734640 1 -- 172.21.15.139:0/1907432855 shutdown_connections 2024-04-25T09:53:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.723+0000 7f3f76734640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f5c06b500 0x7f3f5c06d9c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:16.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.723+0000 7f3f76734640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f7015dbf0 0x7f3f700d3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:16.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.723+0000 7f3f76734640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f700cee50 0x7f3f700d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:16.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.723+0000 7f3f76734640 1 --2- 172.21.15.139:0/1907432855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f700b9370 0x7f3f7015ce30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:16.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.723+0000 7f3f76734640 1 -- 172.21.15.139:0/1907432855 >> 172.21.15.139:0/1907432855 conn(0x7f3f700b5480 msgr2=0x7f3f700b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:16.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.723+0000 7f3f76734640 1 -- 172.21.15.139:0/1907432855 shutdown_connections 2024-04-25T09:53:16.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.723+0000 7f3f76734640 1 -- 172.21.15.139:0/1907432855 wait complete. 2024-04-25T09:53:16.737 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:53:16.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.841+0000 7f7bf41f5640 1 Processor -- start 2024-04-25T09:53:16.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.841+0000 7f7bf41f5640 1 -- start start 2024-04-25T09:53:16.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.842+0000 7f7bf41f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bec15dbd0 0x7f7bec1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:16.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.842+0000 7f7bf41f5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7bec0cee50 0x7f7bec0d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:16.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.842+0000 7f7bf41f5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bec161e70 con 0x7f7bec0b9390 2024-04-25T09:53:16.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.842+0000 7f7bf41f5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bec0d1f00 con 0x7f7bec0cee50 2024-04-25T09:53:16.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.842+0000 7f7bf41f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec162710 con 0x7f7bec15dbd0 2024-04-25T09:53:16.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.842+0000 7f7bf2792640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7bec0cee50 0x7f7bec0d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:16.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.842+0000 7f7bf2792640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7bec0cee50 0x7f7bec0d2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59850/0 (socket says 172.21.15.139:59850) 2024-04-25T09:53:16.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.842+0000 7f7bf2792640 1 -- 172.21.15.139:0/3427874910 learned_addr learned my addr 172.21.15.139:0/3427874910 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:16.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.842+0000 7f7bf1f91640 1 --2- 172.21.15.139:0/3427874910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bec15dbd0 0x7f7bec1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:16.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.842+0000 7f7bf2792640 1 -- 172.21.15.139:0/3427874910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bec15dbd0 msgr2=0x7f7bec1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:16.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.842+0000 7f7bf2792640 1 --2- 172.21.15.139:0/3427874910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bec15dbd0 0x7f7bec1617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:16.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.843+0000 7f7bf2792640 1 -- 172.21.15.139:0/3427874910 >> v1:172.21.15.150:6789/0 conn(0x7f7bec0b9390 legacy=0x7f7bec0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:16.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.843+0000 7f7bf2792640 1 -- 172.21.15.139:0/3427874910 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec162150 con 0x7f7bec0cee50 2024-04-25T09:53:16.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.843+0000 7f7bf2792640 1 --2- 172.21.15.139:0/3427874910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bec0cee50 0x7f7bec0d2390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009e50 tx=0x7f7be402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2fb02c6af11a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:16.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.843+0000 7f7bf0f8f640 1 -- 172.21.15.139:0/3427874910 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4036070 con 0x7f7bec0cee50 2024-04-25T09:53:16.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.843+0000 7f7bf0f8f640 1 -- 172.21.15.139:0/3427874910 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be4031600 con 0x7f7bec0cee50 2024-04-25T09:53:16.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.843+0000 7f7bf0f8f640 1 -- 172.21.15.139:0/3427874910 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be40319d0 con 0x7f7bec0cee50 2024-04-25T09:53:16.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.844+0000 7f7bf41f5640 1 -- 172.21.15.139:0/3427874910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bec0cee50 msgr2=0x7f7bec0d2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:16.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.844+0000 7f7bf41f5640 1 --2- 172.21.15.139:0/3427874910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bec0cee50 0x7f7bec0d2390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009e50 tx=0x7f7be402a6c0 comp rx=0 tx=0).stop 2024-04-25T09:53:16.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.844+0000 7f7bf41f5640 1 -- 172.21.15.139:0/3427874910 shutdown_connections 2024-04-25T09:53:16.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.844+0000 7f7bf41f5640 1 --2- 172.21.15.139:0/3427874910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bec0cee50 0x7f7bec0d2390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:16.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.844+0000 7f7bf41f5640 1 --2- 172.21.15.139:0/3427874910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bec15dbd0 0x7f7bec1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:16.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.844+0000 7f7bf41f5640 1 -- 172.21.15.139:0/3427874910 >> 172.21.15.139:0/3427874910 conn(0x7f7bec0b5480 msgr2=0x7f7bec0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:16.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.844+0000 7f7bf41f5640 1 -- 172.21.15.139:0/3427874910 shutdown_connections 2024-04-25T09:53:16.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.844+0000 7f7bf41f5640 1 -- 172.21.15.139:0/3427874910 wait complete. 2024-04-25T09:53:16.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.845+0000 7f7bf41f5640 1 Processor -- start 2024-04-25T09:53:16.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.845+0000 7f7bf41f5640 1 -- start start 2024-04-25T09:53:16.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf41f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bec0b9390 0x7f7bec153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:16.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf41f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bec0cee50 0x7f7bec15fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:16.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf41f5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bec15dbd0 0x7f7bec157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf41f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0b97b0 con 0x7f7bec0cee50 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf41f5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec15dfb0 con 0x7f7bec15dbd0 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf41f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec153b60 con 0x7f7bec0b9390 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf1790640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bec0cee50 0x7f7bec15fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf1790640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bec0cee50 0x7f7bec15fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59856/0 (socket says 172.21.15.139:59856) 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf1790640 1 -- 172.21.15.139:0/1176830832 learned_addr learned my addr 172.21.15.139:0/1176830832 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf1f91640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bec0b9390 0x7f7bec153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf2792640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bec15dbd0 0x7f7bec157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf1790640 1 -- 172.21.15.139:0/1176830832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bec0b9390 msgr2=0x7f7bec153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf1790640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bec0b9390 0x7f7bec153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf1790640 1 -- 172.21.15.139:0/1176830832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bec15dbd0 msgr2=0x7f7bec157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf1790640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bec15dbd0 0x7f7bec157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf1790640 1 -- 172.21.15.139:0/1176830832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec1606d0 con 0x7f7bec0cee50 2024-04-25T09:53:16.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf1790640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bec0cee50 0x7f7bec15fd80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c430 tx=0x7f7be800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:16.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7be27fc640 1 -- 172.21.15.139:0/1176830832 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8011790 con 0x7f7bec0cee50 2024-04-25T09:53:16.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7be27fc640 1 -- 172.21.15.139:0/1176830832 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be8017070 con 0x7f7bec0cee50 2024-04-25T09:53:16.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7be27fc640 1 -- 172.21.15.139:0/1176830832 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8011bb0 con 0x7f7bec0cee50 2024-04-25T09:53:16.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.846+0000 7f7bf41f5640 1 -- 172.21.15.139:0/1176830832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bec152b40 con 0x7f7bec0cee50 2024-04-25T09:53:16.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.847+0000 7f7bf41f5640 1 -- 172.21.15.139:0/1176830832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec1602c0 con 0x7f7bec0cee50 2024-04-25T09:53:16.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.847+0000 7f7bf41f5640 1 -- 172.21.15.139:0/1176830832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bec15d430 con 0x7f7bec0cee50 2024-04-25T09:53:16.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.848+0000 7f7be27fc640 1 -- 172.21.15.139:0/1176830832 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7be801c460 con 0x7f7bec0cee50 2024-04-25T09:53:16.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.848+0000 7f7be27fc640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7bd006b4d0 0x7f7bd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:16.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.848+0000 7f7bf1f91640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7bd006b4d0 0x7f7bd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:16.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.849+0000 7f7be27fc640 1 -- 172.21.15.139:0/1176830832 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6007+0+0 (secure 0 0 0) 0x7f7be808a020 con 0x7f7bec0cee50 2024-04-25T09:53:16.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.849+0000 7f7bf1f91640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7bd006b4d0 0x7f7bd006d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0079a0 tx=0x7f7bdc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:16.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:16.853+0000 7f7be27fc640 1 -- 172.21.15.139:0/1176830832 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7bec15d430 con 0x7f7bec0cee50 2024-04-25T09:53:17.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:17.025+0000 7f7bf41f5640 1 -- 172.21.15.139:0/1176830832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7bec074520 con 0x7f7bec0cee50 2024-04-25T09:53:18.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.732+0000 7f7be27fc640 1 -- 172.21.15.139:0/1176830832 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f7be80587b0 con 0x7f7bec0cee50 2024-04-25T09:53:18.734 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data_ec' created 2024-04-25T09:53:18.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.735+0000 7f7bf41f5640 1 -- 172.21.15.139:0/1176830832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7bd006b4d0 msgr2=0x7f7bd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.735+0000 7f7bf41f5640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7bd006b4d0 0x7f7bd006d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0079a0 tx=0x7f7bdc008040 comp rx=0 tx=0).stop 2024-04-25T09:53:18.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.735+0000 7f7bf41f5640 1 -- 172.21.15.139:0/1176830832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bec0cee50 msgr2=0x7f7bec15fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.735+0000 7f7bf41f5640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bec0cee50 0x7f7bec15fd80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c430 tx=0x7f7be800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.735+0000 7f7bf41f5640 1 -- 172.21.15.139:0/1176830832 shutdown_connections 2024-04-25T09:53:18.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.735+0000 7f7bf41f5640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bec0cee50 0x7f7bec15fd80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.735+0000 7f7bf41f5640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7bd006b4d0 0x7f7bd006d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.736+0000 7f7bf41f5640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bec15dbd0 0x7f7bec157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.736+0000 7f7bf41f5640 1 --2- 172.21.15.139:0/1176830832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bec0b9390 0x7f7bec153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.736+0000 7f7bf41f5640 1 -- 172.21.15.139:0/1176830832 >> 172.21.15.139:0/1176830832 conn(0x7f7bec0b5480 msgr2=0x7f7bec0d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:18.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.736+0000 7f7bf41f5640 1 -- 172.21.15.139:0/1176830832 shutdown_connections 2024-04-25T09:53:18.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.736+0000 7f7bf41f5640 1 -- 172.21.15.139:0/1176830832 wait complete. 2024-04-25T09:53:18.757 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T09:53:18.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.861+0000 7fae72977640 1 Processor -- start 2024-04-25T09:53:18.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.861+0000 7fae72977640 1 -- start start 2024-04-25T09:53:18.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.862+0000 7fae72977640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae6c0cd610 0x7fae6c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.862+0000 7fae72977640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae6c09ad40 0x7fae6c0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.862+0000 7fae72977640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6c099ca0 con 0x7fae6c1618c0 2024-04-25T09:53:18.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.862+0000 7fae72977640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6c099a50 con 0x7fae6c0cd610 2024-04-25T09:53:18.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.862+0000 7fae72977640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c099f80 con 0x7fae6c09ad40 2024-04-25T09:53:18.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.862+0000 7fae6bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae6c0cd610 0x7fae6c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.862+0000 7fae6bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae6c0cd610 0x7fae6c0d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59870/0 (socket says 172.21.15.139:59870) 2024-04-25T09:53:18.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.862+0000 7fae6bfff640 1 -- 172.21.15.139:0/2411319679 learned_addr learned my addr 172.21.15.139:0/2411319679 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:18.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.863+0000 7fae6b7fe640 1 --2- 172.21.15.139:0/2411319679 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae6c09ad40 0x7fae6c0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.863+0000 7fae6bfff640 1 -- 172.21.15.139:0/2411319679 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae6c09ad40 msgr2=0x7fae6c0d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:18.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.863+0000 7fae6bfff640 1 --2- 172.21.15.139:0/2411319679 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae6c09ad40 0x7fae6c0d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.863+0000 7fae6bfff640 1 -- 172.21.15.139:0/2411319679 >> v1:172.21.15.150:6789/0 conn(0x7fae6c1618c0 legacy=0x7fae6c06be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:18.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.863+0000 7fae6bfff640 1 -- 172.21.15.139:0/2411319679 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c09a260 con 0x7fae6c0cd610 2024-04-25T09:53:18.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.863+0000 7fae6b7fe640 1 --2- 172.21.15.139:0/2411319679 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae6c09ad40 0x7fae6c0d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:18.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.863+0000 7fae6bfff640 1 --2- 172.21.15.139:0/2411319679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae6c0cd610 0x7fae6c0d0370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009870 tx=0x7fae5c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61dd3504fc148f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:18.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.863+0000 7fae6affd640 1 -- 172.21.15.139:0/2411319679 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c037070 con 0x7fae6c0cd610 2024-04-25T09:53:18.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.863+0000 7fae6affd640 1 -- 172.21.15.139:0/2411319679 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c032440 con 0x7fae6c0cd610 2024-04-25T09:53:18.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.863+0000 7fae6affd640 1 -- 172.21.15.139:0/2411319679 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c032810 con 0x7fae6c0cd610 2024-04-25T09:53:18.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.864+0000 7fae72977640 1 -- 172.21.15.139:0/2411319679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae6c0cd610 msgr2=0x7fae6c0d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.864+0000 7fae72977640 1 --2- 172.21.15.139:0/2411319679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae6c0cd610 0x7fae6c0d0370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009870 tx=0x7fae5c02a2a0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.864+0000 7fae72977640 1 -- 172.21.15.139:0/2411319679 shutdown_connections 2024-04-25T09:53:18.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.864+0000 7fae72977640 1 --2- 172.21.15.139:0/2411319679 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae6c09ad40 0x7fae6c0d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.864+0000 7fae72977640 1 --2- 172.21.15.139:0/2411319679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae6c0cd610 0x7fae6c0d0370 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.864+0000 7fae72977640 1 -- 172.21.15.139:0/2411319679 >> 172.21.15.139:0/2411319679 conn(0x7fae6c0b31f0 msgr2=0x7fae6c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:18.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.864+0000 7fae72977640 1 -- 172.21.15.139:0/2411319679 shutdown_connections 2024-04-25T09:53:18.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.864+0000 7fae72977640 1 -- 172.21.15.139:0/2411319679 wait complete. 2024-04-25T09:53:18.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.865+0000 7fae72977640 1 Processor -- start 2024-04-25T09:53:18.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.865+0000 7fae72977640 1 -- start start 2024-04-25T09:53:18.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae72977640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae6c09ad40 0x7fae6c14ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae72977640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae6c0cd610 0x7fae6c1f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae72977640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae6c1618c0 0x7fae6c1f7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae72977640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c0cd9f0 con 0x7fae6c0cd610 2024-04-25T09:53:18.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae72977640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c09a820 con 0x7fae6c1618c0 2024-04-25T09:53:18.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae72977640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c0cdb70 con 0x7fae6c09ad40 2024-04-25T09:53:18.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae6b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae6c0cd610 0x7fae6c1f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae6b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae6c0cd610 0x7fae6c1f7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59878/0 (socket says 172.21.15.139:59878) 2024-04-25T09:53:18.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae6b7fe640 1 -- 172.21.15.139:0/1501527299 learned_addr learned my addr 172.21.15.139:0/1501527299 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:18.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae6bfff640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae6c09ad40 0x7fae6c14ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae70f14640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae6c1618c0 0x7fae6c1f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae6b7fe640 1 -- 172.21.15.139:0/1501527299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae6c09ad40 msgr2=0x7fae6c14ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae6b7fe640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae6c09ad40 0x7fae6c14ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae6b7fe640 1 -- 172.21.15.139:0/1501527299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae6c1618c0 msgr2=0x7fae6c1f7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:18.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae6b7fe640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae6c1618c0 0x7fae6c1f7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:18.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae6b7fe640 1 -- 172.21.15.139:0/1501527299 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c157480 con 0x7fae6c0cd610 2024-04-25T09:53:18.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.866+0000 7fae70f14640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae6c1618c0 0x7fae6c1f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:18.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.867+0000 7fae6b7fe640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae6c0cd610 0x7fae6c1f7230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae6000c7f0 tx=0x7fae6000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:18.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.867+0000 7fae68ff9640 1 -- 172.21.15.139:0/1501527299 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60015070 con 0x7fae6c0cd610 2024-04-25T09:53:18.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.867+0000 7fae68ff9640 1 -- 172.21.15.139:0/1501527299 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae60007a30 con 0x7fae6c0cd610 2024-04-25T09:53:18.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.867+0000 7fae72977640 1 -- 172.21.15.139:0/1501527299 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6c1f7d80 con 0x7fae6c0cd610 2024-04-25T09:53:18.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.867+0000 7fae68ff9640 1 -- 172.21.15.139:0/1501527299 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60007e00 con 0x7fae6c0cd610 2024-04-25T09:53:18.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.867+0000 7fae72977640 1 -- 172.21.15.139:0/1501527299 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6c157a20 con 0x7fae6c0cd610 2024-04-25T09:53:18.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.868+0000 7fae72977640 1 -- 172.21.15.139:0/1501527299 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6c157c20 con 0x7fae6c0cd610 2024-04-25T09:53:18.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.868+0000 7fae68ff9640 1 -- 172.21.15.139:0/1501527299 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae60018070 con 0x7fae6c0cd610 2024-04-25T09:53:18.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.869+0000 7fae68ff9640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae5806b4d0 0x7fae5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:18.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.869+0000 7fae68ff9640 1 -- 172.21.15.139:0/1501527299 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fae6008a3e0 con 0x7fae6c0cd610 2024-04-25T09:53:18.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.869+0000 7fae6bfff640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae5806b4d0 0x7fae5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:18.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.869+0000 7fae6bfff640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae5806b4d0 0x7fae5806d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fae5c034000 tx=0x7fae5c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:18.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:18.873+0000 7fae68ff9640 1 -- 172.21.15.139:0/1501527299 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae6c157c20 con 0x7fae6c0cd610 2024-04-25T09:53:19.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.044+0000 7fae72977640 1 -- 172.21.15.139:0/1501527299 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fae6c074520 con 0x7fae6c0cd610 2024-04-25T09:53:19.540 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-04-25T09:53:19.538+0000 7f9641822640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:53:19.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.737+0000 7fae68ff9640 1 -- 172.21.15.139:0/1501527299 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7fae60058fd0 con 0x7fae6c0cd610 2024-04-25T09:53:19.739 INFO:teuthology.orchestra.run.smithi139.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T09:53:19.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.743+0000 7fae72977640 1 -- 172.21.15.139:0/1501527299 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae5806b4d0 msgr2=0x7fae5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.743+0000 7fae72977640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae5806b4d0 0x7fae5806d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fae5c034000 tx=0x7fae5c02e000 comp rx=0 tx=0).stop 2024-04-25T09:53:19.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.743+0000 7fae72977640 1 -- 172.21.15.139:0/1501527299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae6c0cd610 msgr2=0x7fae6c1f7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.743+0000 7fae72977640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae6c0cd610 0x7fae6c1f7230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae6000c7f0 tx=0x7fae6000cbf0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.743+0000 7fae72977640 1 -- 172.21.15.139:0/1501527299 shutdown_connections 2024-04-25T09:53:19.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.743+0000 7fae72977640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae6c0cd610 0x7fae6c1f7230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.743+0000 7fae72977640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae5806b4d0 0x7fae5806d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.743+0000 7fae72977640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae6c1618c0 0x7fae6c1f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.743+0000 7fae72977640 1 --2- 172.21.15.139:0/1501527299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae6c09ad40 0x7fae6c14ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.743+0000 7fae72977640 1 -- 172.21.15.139:0/1501527299 >> 172.21.15.139:0/1501527299 conn(0x7fae6c0b31f0 msgr2=0x7fae6c06c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:19.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.748+0000 7fae72977640 1 -- 172.21.15.139:0/1501527299 shutdown_connections 2024-04-25T09:53:19.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.750+0000 7fae72977640 1 -- 172.21.15.139:0/1501527299 wait complete. 2024-04-25T09:53:19.774 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T09:53:19.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.874+0000 7fb608aac640 1 Processor -- start 2024-04-25T09:53:19.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.874+0000 7fb608aac640 1 -- start start 2024-04-25T09:53:19.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb608aac640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb60415f8c0 0x7fb60409a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb608aac640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb60415fca0 0x7fb6040cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb608aac640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb604099bf0 con 0x7fb6040d0220 2024-04-25T09:53:19.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb608aac640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6040d06c0 con 0x7fb60415f8c0 2024-04-25T09:53:19.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb608aac640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb604099ed0 con 0x7fb60415fca0 2024-04-25T09:53:19.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb60259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb60415f8c0 0x7fb60409a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb602d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb6040d0220 0x7fb60415b560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38736/0 (socket says 172.21.15.139:38736) 2024-04-25T09:53:19.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb602d9d640 1 -- 172.21.15.139:0/1291113257 learned_addr learned my addr 172.21.15.139:0/1291113257 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:19.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb5fbfff640 1 --2- 172.21.15.139:0/1291113257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb60415fca0 0x7fb6040cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb60259c640 1 -- 172.21.15.139:0/1291113257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb60415fca0 msgr2=0x7fb6040cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb60259c640 1 --2- 172.21.15.139:0/1291113257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb60415fca0 0x7fb6040cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb60259c640 1 -- 172.21.15.139:0/1291113257 >> v1:172.21.15.139:6789/0 conn(0x7fb6040d0220 legacy=0x7fb60415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb60259c640 1 -- 172.21.15.139:0/1291113257 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60409a1b0 con 0x7fb60415f8c0 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb601d9b640 1 -- 172.21.15.139:0/1291113257 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883269016 0 0) 0x7fb604099bf0 con 0x7fb6040d0220 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.875+0000 7fb5fbfff640 1 --2- 172.21.15.139:0/1291113257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb60415fca0 0x7fb6040cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.876+0000 7fb60259c640 1 --2- 172.21.15.139:0/1291113257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb60415f8c0 0x7fb60409a9b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40097a0 tx=0x7fb5f402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234e2585b1cad09a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.876+0000 7fb601d9b640 1 -- 172.21.15.139:0/1291113257 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4037070 con 0x7fb60415f8c0 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.876+0000 7fb601d9b640 1 -- 172.21.15.139:0/1291113257 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f4032440 con 0x7fb60415f8c0 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.876+0000 7fb601d9b640 1 -- 172.21.15.139:0/1291113257 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4032810 con 0x7fb60415f8c0 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.877+0000 7fb608aac640 1 -- 172.21.15.139:0/1291113257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb60415f8c0 msgr2=0x7fb60409a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.877+0000 7fb608aac640 1 --2- 172.21.15.139:0/1291113257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb60415f8c0 0x7fb60409a9b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40097a0 tx=0x7fb5f402a210 comp rx=0 tx=0).stop 2024-04-25T09:53:19.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.877+0000 7fb608aac640 1 -- 172.21.15.139:0/1291113257 shutdown_connections 2024-04-25T09:53:19.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.877+0000 7fb608aac640 1 --2- 172.21.15.139:0/1291113257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb60415fca0 0x7fb6040cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.877+0000 7fb608aac640 1 --2- 172.21.15.139:0/1291113257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb60415f8c0 0x7fb60409a9b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.877+0000 7fb608aac640 1 -- 172.21.15.139:0/1291113257 >> 172.21.15.139:0/1291113257 conn(0x7fb6040b31f0 msgr2=0x7fb6040b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:19.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.877+0000 7fb608aac640 1 -- 172.21.15.139:0/1291113257 shutdown_connections 2024-04-25T09:53:19.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.877+0000 7fb608aac640 1 -- 172.21.15.139:0/1291113257 wait complete. 2024-04-25T09:53:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.878+0000 7fb608aac640 1 Processor -- start 2024-04-25T09:53:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.878+0000 7fb608aac640 1 -- start start 2024-04-25T09:53:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb608aac640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6040d0220 0x7fb6040c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb608aac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb60415f8c0 0x7fb6041f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb608aac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb60415fca0 0x7fb6040ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb608aac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6041602d0 con 0x7fb60415f8c0 2024-04-25T09:53:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb608aac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60409aef0 con 0x7fb60415fca0 2024-04-25T09:53:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb608aac640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb604160450 con 0x7fb6040d0220 2024-04-25T09:53:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb5fbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb60415f8c0 0x7fb6041f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb5fbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb60415f8c0 0x7fb6041f0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59896/0 (socket says 172.21.15.139:59896) 2024-04-25T09:53:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb5fbfff640 1 -- 172.21.15.139:0/3404514963 learned_addr learned my addr 172.21.15.139:0/3404514963 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:19.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb60259c640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6040d0220 0x7fb6040c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb602d9d640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb60415fca0 0x7fb6040ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb5fbfff640 1 -- 172.21.15.139:0/3404514963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6040d0220 msgr2=0x7fb6040c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb5fbfff640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6040d0220 0x7fb6040c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb5fbfff640 1 -- 172.21.15.139:0/3404514963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb60415fca0 msgr2=0x7fb6040ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb5fbfff640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb60415fca0 0x7fb6040ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb5fbfff640 1 -- 172.21.15.139:0/3404514963 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb604158560 con 0x7fb60415f8c0 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb60259c640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6040d0220 0x7fb6040c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb602d9d640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb60415fca0 0x7fb6040ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb5fbfff640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb60415f8c0 0x7fb6041f0ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00d7d0 tx=0x7fb5ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb5faffd640 1 -- 172.21.15.139:0/3404514963 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec004030 con 0x7fb60415f8c0 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb5faffd640 1 -- 172.21.15.139:0/3404514963 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec0041d0 con 0x7fb60415f8c0 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb608aac640 1 -- 172.21.15.139:0/3404514963 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6040b7170 con 0x7fb60415f8c0 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.879+0000 7fb5faffd640 1 -- 172.21.15.139:0/3404514963 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0096a0 con 0x7fb60415f8c0 2024-04-25T09:53:19.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.880+0000 7fb608aac640 1 -- 172.21.15.139:0/3404514963 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6040b4230 con 0x7fb60415f8c0 2024-04-25T09:53:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.881+0000 7fb608aac640 1 -- 172.21.15.139:0/3404514963 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb604074520 con 0x7fb60415f8c0 2024-04-25T09:53:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.881+0000 7fb5faffd640 1 -- 172.21.15.139:0/3404514963 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5ec009840 con 0x7fb60415f8c0 2024-04-25T09:53:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.881+0000 7fb5faffd640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb5e406b4d0 0x7fb5e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.881+0000 7fb5faffd640 1 -- 172.21.15.139:0/3404514963 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb5ec08a720 con 0x7fb60415f8c0 2024-04-25T09:53:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.881+0000 7fb60259c640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb5e406b4d0 0x7fb5e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:19.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.882+0000 7fb60259c640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb5e406b4d0 0x7fb5e406d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4034000 tx=0x7fb5f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:19.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:19.886+0000 7fb5faffd640 1 -- 172.21.15.139:0/3404514963 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5ec059310 con 0x7fb60415f8c0 2024-04-25T09:53:20.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.099+0000 7fb608aac640 1 -- 172.21.15.139:0/3404514963 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fb6040cdc30 con 0x7fb60415f8c0 2024-04-25T09:53:20.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.743+0000 7fb5faffd640 1 -- 172.21.15.139:0/3404514963 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fb5ec058c10 con 0x7fb60415f8c0 2024-04-25T09:53:20.745 INFO:teuthology.orchestra.run.smithi139.stderr:added data pool 4 to fsmap 2024-04-25T09:53:20.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.746+0000 7fb608aac640 1 -- 172.21.15.139:0/3404514963 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb5e406b4d0 msgr2=0x7fb5e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.746+0000 7fb608aac640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb5e406b4d0 0x7fb5e406d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4034000 tx=0x7fb5f402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:20.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.746+0000 7fb608aac640 1 -- 172.21.15.139:0/3404514963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb60415f8c0 msgr2=0x7fb6041f0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.746+0000 7fb608aac640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb60415f8c0 0x7fb6041f0ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00d7d0 tx=0x7fb5ec00dc90 comp rx=0 tx=0).stop 2024-04-25T09:53:20.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.746+0000 7fb608aac640 1 -- 172.21.15.139:0/3404514963 shutdown_connections 2024-04-25T09:53:20.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.746+0000 7fb608aac640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb60415f8c0 0x7fb6041f0ad0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.746+0000 7fb608aac640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb5e406b4d0 0x7fb5e406d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.746+0000 7fb608aac640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb60415fca0 0x7fb6040ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.746+0000 7fb608aac640 1 --2- 172.21.15.139:0/3404514963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6040d0220 0x7fb6040c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.746+0000 7fb608aac640 1 -- 172.21.15.139:0/3404514963 >> 172.21.15.139:0/3404514963 conn(0x7fb6040b31f0 msgr2=0x7fb60415e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:20.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.746+0000 7fb608aac640 1 -- 172.21.15.139:0/3404514963 shutdown_connections 2024-04-25T09:53:20.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.746+0000 7fb608aac640 1 -- 172.21.15.139:0/3404514963 wait complete. 2024-04-25T09:53:20.759 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:20.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.853+0000 7fc4ef3c1640 1 Processor -- start 2024-04-25T09:53:20.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.854+0000 7fc4ef3c1640 1 -- start start 2024-04-25T09:53:20.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.854+0000 7fc4ef3c1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc4e80d0730 0x7fc4e80cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.854+0000 7fc4ef3c1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e80ccb70 con 0x7fc4e80d0b90 2024-04-25T09:53:20.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.854+0000 7fc4ef3c1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e80cf970 con 0x7fc4e80b0750 2024-04-25T09:53:20.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.854+0000 7fc4ef3c1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e80cfc50 con 0x7fc4e80d0730 2024-04-25T09:53:20.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.855+0000 7fc4ed95e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc4e80d0b90 0x7fc4e80cc380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38744/0 (socket says 172.21.15.139:38744) 2024-04-25T09:53:20.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.855+0000 7fc4ed95e640 1 -- 172.21.15.139:0/3378829357 learned_addr learned my addr 172.21.15.139:0/3378829357 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:20.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.855+0000 7fc4ec95c640 1 --2- 172.21.15.139:0/3378829357 >> v2:172.21.15.150:3300/0 conn(0x7fc4e80d0730 0x7fc4e80cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.855+0000 7fc4dffff640 1 -- 172.21.15.139:0/3378829357 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717811008 0 0) 0x7fc4e80ccb70 con 0x7fc4e80d0b90 2024-04-25T09:53:20.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.855+0000 7fc4dffff640 1 -- 172.21.15.139:0/3378829357 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4d4003520 con 0x7fc4e80d0b90 2024-04-25T09:53:20.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.855+0000 7fc4ec95c640 1 -- 172.21.15.139:0/3378829357 >> v1:172.21.15.150:6789/0 conn(0x7fc4e80b0750 legacy=0x7fc4e80b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.855+0000 7fc4ec95c640 1 -- 172.21.15.139:0/3378829357 >> v1:172.21.15.139:6789/0 conn(0x7fc4e80d0b90 legacy=0x7fc4e80cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.855+0000 7fc4ec95c640 1 -- 172.21.15.139:0/3378829357 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80cff30 con 0x7fc4e80d0730 2024-04-25T09:53:20.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.856+0000 7fc4ec95c640 1 --2- 172.21.15.139:0/3378829357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4e80d0730 0x7fc4e80cbe40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80097a0 tx=0x7fc4d802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=626dc35d1eb4bd17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:20.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.856+0000 7fc4dffff640 1 -- 172.21.15.139:0/3378829357 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8037070 con 0x7fc4e80d0730 2024-04-25T09:53:20.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.856+0000 7fc4dffff640 1 -- 172.21.15.139:0/3378829357 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d8032440 con 0x7fc4e80d0730 2024-04-25T09:53:20.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.856+0000 7fc4dffff640 1 -- 172.21.15.139:0/3378829357 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8032810 con 0x7fc4e80d0730 2024-04-25T09:53:20.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.856+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3378829357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4e80d0730 msgr2=0x7fc4e80cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.856+0000 7fc4ef3c1640 1 --2- 172.21.15.139:0/3378829357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4e80d0730 0x7fc4e80cbe40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80097a0 tx=0x7fc4d802a230 comp rx=0 tx=0).stop 2024-04-25T09:53:20.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.856+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3378829357 shutdown_connections 2024-04-25T09:53:20.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.856+0000 7fc4ef3c1640 1 --2- 172.21.15.139:0/3378829357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4e80d0730 0x7fc4e80cbe40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.856+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3378829357 >> 172.21.15.139:0/3378829357 conn(0x7fc4e80b3380 msgr2=0x7fc4e80b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:20.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.856+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3378829357 shutdown_connections 2024-04-25T09:53:20.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.856+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3378829357 wait complete. 2024-04-25T09:53:20.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.857+0000 7fc4ef3c1640 1 Processor -- start 2024-04-25T09:53:20.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.858+0000 7fc4ef3c1640 1 -- start start 2024-04-25T09:53:20.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.858+0000 7fc4ef3c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4e80b0750 0x7fc4e80afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.858+0000 7fc4ef3c1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4e80d0730 0x7fc4e8150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.858+0000 7fc4ef3c1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4e806c9a0 0x7fc4e80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.858+0000 7fc4ef3c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e815be40 con 0x7fc4e80b0750 2024-04-25T09:53:20.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.858+0000 7fc4ef3c1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e815bcc0 con 0x7fc4e806c9a0 2024-04-25T09:53:20.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.858+0000 7fc4ef3c1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80b4780 con 0x7fc4e80d0730 2024-04-25T09:53:20.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.858+0000 7fc4ed15d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4e80b0750 0x7fc4e80afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.858+0000 7fc4ed15d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4e80b0750 0x7fc4e80afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59908/0 (socket says 172.21.15.139:59908) 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.858+0000 7fc4ed15d640 1 -- 172.21.15.139:0/3015904598 learned_addr learned my addr 172.21.15.139:0/3015904598 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.858+0000 7fc4ec95c640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4e80d0730 0x7fc4e8150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.858+0000 7fc4ed95e640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4e806c9a0 0x7fc4e80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.859+0000 7fc4ed15d640 1 -- 172.21.15.139:0/3015904598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4e80d0730 msgr2=0x7fc4e8150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.859+0000 7fc4ed15d640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4e80d0730 0x7fc4e8150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.859+0000 7fc4ed15d640 1 -- 172.21.15.139:0/3015904598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4e806c9a0 msgr2=0x7fc4e80caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.859+0000 7fc4ed15d640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4e806c9a0 0x7fc4e80caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.859+0000 7fc4ed15d640 1 -- 172.21.15.139:0/3015904598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80b0390 con 0x7fc4e80b0750 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.859+0000 7fc4ed15d640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4e80b0750 0x7fc4e80afdb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c480 tx=0x7fc4e400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.859+0000 7fc4ddffb640 1 -- 172.21.15.139:0/3015904598 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40052d0 con 0x7fc4e80b0750 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.859+0000 7fc4ddffb640 1 -- 172.21.15.139:0/3015904598 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e4005470 con 0x7fc4e80b0750 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.859+0000 7fc4ddffb640 1 -- 172.21.15.139:0/3015904598 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4012870 con 0x7fc4e80b0750 2024-04-25T09:53:20.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.859+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3015904598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e806bec0 con 0x7fc4e80b0750 2024-04-25T09:53:20.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.859+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3015904598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e806dbc0 con 0x7fc4e80b0750 2024-04-25T09:53:20.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.860+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3015904598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e8074520 con 0x7fc4e80b0750 2024-04-25T09:53:20.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.861+0000 7fc4ddffb640 1 -- 172.21.15.139:0/3015904598 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4e4012a10 con 0x7fc4e80b0750 2024-04-25T09:53:20.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.861+0000 7fc4ddffb640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc4cc06b4d0 0x7fc4cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:20.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.861+0000 7fc4ddffb640 1 -- 172.21.15.139:0/3015904598 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc4e408a190 con 0x7fc4e80b0750 2024-04-25T09:53:20.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.861+0000 7fc4ec95c640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc4cc06b4d0 0x7fc4cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:20.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.861+0000 7fc4ec95c640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc4cc06b4d0 0x7fc4cc06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8034000 tx=0x7fc4d802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:20.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:20.865+0000 7fc4ddffb640 1 -- 172.21.15.139:0/3015904598 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4e4058d60 con 0x7fc4e80b0750 2024-04-25T09:53:21.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.073+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3015904598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4e8070030 con 0x7fc4e80b0750 2024-04-25T09:53:21.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.073+0000 7fc4ddffb640 1 -- 172.21.15.139:0/3015904598 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fc4e4058660 con 0x7fc4e80b0750 2024-04-25T09:53:21.076 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:21.076 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/892226922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":892226922},{"type":"v1","addr":"172.21.15.150:6834","nonce":892226922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/582019435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":582019435},{"type":"v1","addr":"172.21.15.139:6837","nonce":582019435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/2751714076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2751714076},{"type":"v1","addr":"172.21.15.139:6834","nonce":2751714076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:15.042067+0000","modified":"2024-04-25T09:53:20.100914+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2160557452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2160557452},{"type":"v1","addr":"172.21.15.150:6835","nonce":2160557452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T09:53:21.076 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 7 2024-04-25T09:53:21.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.075+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3015904598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc4cc06b4d0 msgr2=0x7fc4cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.075+0000 7fc4ef3c1640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc4cc06b4d0 0x7fc4cc06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8034000 tx=0x7fc4d802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:21.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.075+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3015904598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4e80b0750 msgr2=0x7fc4e80afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.075+0000 7fc4ef3c1640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4e80b0750 0x7fc4e80afdb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c480 tx=0x7fc4e400c940 comp rx=0 tx=0).stop 2024-04-25T09:53:21.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.075+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3015904598 shutdown_connections 2024-04-25T09:53:21.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.075+0000 7fc4ef3c1640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc4cc06b4d0 0x7fc4cc06d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.075+0000 7fc4ef3c1640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4e806c9a0 0x7fc4e80caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.075+0000 7fc4ef3c1640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4e80d0730 0x7fc4e8150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.075+0000 7fc4ef3c1640 1 --2- 172.21.15.139:0/3015904598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4e80b0750 0x7fc4e80afdb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.075+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3015904598 >> 172.21.15.139:0/3015904598 conn(0x7fc4e80b3380 msgr2=0x7fc4e806f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:21.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.075+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3015904598 shutdown_connections 2024-04-25T09:53:21.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.075+0000 7fc4ef3c1640 1 -- 172.21.15.139:0/3015904598 wait complete. 2024-04-25T09:53:21.088 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:21.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.182+0000 7f15f295c640 1 Processor -- start 2024-04-25T09:53:21.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.182+0000 7f15f295c640 1 -- start start 2024-04-25T09:53:21.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.183+0000 7f15f295c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec06be40 0x7f15ec0acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.183+0000 7f15f295c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f15ec15dd30 0x7f15ec0b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.183+0000 7f15f295c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ec0ad2d0 con 0x7f15ec06c220 2024-04-25T09:53:21.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.183+0000 7f15f295c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ec0b2de0 con 0x7f15ec15dd30 2024-04-25T09:53:21.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.183+0000 7f15f295c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec0b30c0 con 0x7f15ec06be40 2024-04-25T09:53:21.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.183+0000 7f15eb7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f15ec06c220 0x7f15ec15d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38748/0 (socket says 172.21.15.139:38748) 2024-04-25T09:53:21.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.183+0000 7f15eb7fe640 1 -- 172.21.15.139:0/4082196687 learned_addr learned my addr 172.21.15.139:0/4082196687 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15ebfff640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec06be40 0x7f15ec0acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15f0ef9640 1 --2- 172.21.15.139:0/4082196687 >> v2:172.21.15.150:3300/0 conn(0x7f15ec15dd30 0x7f15ec0b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285143380 0 0) 0x7f15ec0ad2d0 con 0x7f15ec06c220 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15d4003520 con 0x7f15ec06c220 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1126661608 0 0) 0x7f15d4003520 con 0x7f15ec06c220 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec06be40 msgr2=0x7f15ec0acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec06be40 0x7f15ec0acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 >> v2:172.21.15.150:3300/0 conn(0x7f15ec15dd30 msgr2=0x7f15ec0b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 --2- 172.21.15.139:0/4082196687 >> v2:172.21.15.150:3300/0 conn(0x7f15ec15dd30 0x7f15ec0b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec0b33a0 con 0x7f15ec06c220 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f15e0001670 con 0x7f15ec06c220 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 >> v1:172.21.15.139:6789/0 conn(0x7f15ec06c220 legacy=0x7f15ec15d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d40039b0 0x7f15d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15d4004800 0x7f15d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec06be40 0x7f15d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec0b2de0 con 0x7f15d4004800 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec0b30c0 con 0x7f15d40039b0 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d400b9c0 con 0x7f15ec06be40 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eb7fe640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15d4004800 0x7f15d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15ebfff640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d40039b0 0x7f15d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15f0ef9640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec06be40 0x7f15d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eb7fe640 1 -- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec06be40 msgr2=0x7f15d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eb7fe640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec06be40 0x7f15d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eb7fe640 1 -- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d40039b0 msgr2=0x7f15d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eb7fe640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d40039b0 0x7f15d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15eb7fe640 1 -- 172.21.15.139:0/4082196687 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d4003520 con 0x7f15d4004800 2024-04-25T09:53:21.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15ebfff640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d40039b0 0x7f15d4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:21.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.184+0000 7f15f0ef9640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec06be40 0x7f15d40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:21.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.185+0000 7f15eb7fe640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15d4004800 0x7f15d4004c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007fd0 tx=0x7f15d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72864a9152fbc047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.185+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0007d50 con 0x7f15d4004800 2024-04-25T09:53:21.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.185+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e0005ad0 con 0x7f15d4004800 2024-04-25T09:53:21.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.185+0000 7f15eaffd640 1 -- 172.21.15.139:0/4082196687 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0007070 con 0x7f15d4004800 2024-04-25T09:53:21.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.186+0000 7f15f295c640 1 -- 172.21.15.139:0/4082196687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15d4004800 msgr2=0x7f15d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.186+0000 7f15f295c640 1 --2- 172.21.15.139:0/4082196687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15d4004800 0x7f15d4004c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007fd0 tx=0x7f15d4005830 comp rx=0 tx=0).stop 2024-04-25T09:53:21.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.186+0000 7f15f0ef9640 1 -- 172.21.15.139:0/4082196687 reap_dead start 2024-04-25T09:53:21.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.186+0000 7f15f295c640 1 -- 172.21.15.139:0/4082196687 shutdown_connections 2024-04-25T09:53:21.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.186+0000 7f15f295c640 1 -- 172.21.15.139:0/4082196687 >> 172.21.15.139:0/4082196687 conn(0x7f15ec071a00 msgr2=0x7f15ec071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:21.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.186+0000 7f15f295c640 1 -- 172.21.15.139:0/4082196687 shutdown_connections 2024-04-25T09:53:21.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.186+0000 7f15f295c640 1 -- 172.21.15.139:0/4082196687 wait complete. 2024-04-25T09:53:21.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.187+0000 7f15f295c640 1 Processor -- start 2024-04-25T09:53:21.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.187+0000 7f15f295c640 1 -- start start 2024-04-25T09:53:21.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f295c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d4004800 0x7f15ec14f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f295c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec152890 0x7f15ec1603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f295c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ec15b610 0x7f15ec1608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f295c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec06d9d0 con 0x7f15ec15b610 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f295c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec0b3960 con 0x7f15d4004800 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f295c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec072520 con 0x7f15ec152890 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f0ef9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ec15b610 0x7f15ec1608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f0ef9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ec15b610 0x7f15ec1608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59936/0 (socket says 172.21.15.139:59936) 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f0ef9640 1 -- 172.21.15.139:0/4030544082 learned_addr learned my addr 172.21.15.139:0/4030544082 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15ebfff640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d4004800 0x7f15ec14f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15eb7fe640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec152890 0x7f15ec1603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f0ef9640 1 -- 172.21.15.139:0/4030544082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec152890 msgr2=0x7f15ec1603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f0ef9640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec152890 0x7f15ec1603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f0ef9640 1 -- 172.21.15.139:0/4030544082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d4004800 msgr2=0x7f15ec14f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f0ef9640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d4004800 0x7f15ec14f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f0ef9640 1 -- 172.21.15.139:0/4030544082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec15a570 con 0x7f15ec15b610 2024-04-25T09:53:21.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f0ef9640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ec15b610 0x7f15ec1608f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f15ec15e110 tx=0x7f15e4010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15e8ff9640 1 -- 172.21.15.139:0/4030544082 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4003990 con 0x7f15ec15b610 2024-04-25T09:53:21.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15e8ff9640 1 -- 172.21.15.139:0/4030544082 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e4003b50 con 0x7f15ec15b610 2024-04-25T09:53:21.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f295c640 1 -- 172.21.15.139:0/4030544082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec15a850 con 0x7f15ec15b610 2024-04-25T09:53:21.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15e8ff9640 1 -- 172.21.15.139:0/4030544082 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4004020 con 0x7f15ec15b610 2024-04-25T09:53:21.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.188+0000 7f15f295c640 1 -- 172.21.15.139:0/4030544082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec15ae10 con 0x7f15ec15b610 2024-04-25T09:53:21.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.189+0000 7f15f295c640 1 -- 172.21.15.139:0/4030544082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ec074520 con 0x7f15ec15b610 2024-04-25T09:53:21.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.190+0000 7f15e8ff9640 1 -- 172.21.15.139:0/4030544082 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15e401e030 con 0x7f15ec15b610 2024-04-25T09:53:21.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.190+0000 7f15e8ff9640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f15d806b4d0 0x7f15d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.190+0000 7f15e8ff9640 1 -- 172.21.15.139:0/4030544082 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f15e408ada0 con 0x7f15ec15b610 2024-04-25T09:53:21.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.190+0000 7f15ebfff640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f15d806b4d0 0x7f15d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.191+0000 7f15ebfff640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f15d806b4d0 0x7f15d806d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0098b0 tx=0x7f15dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.195+0000 7f15e8ff9640 1 -- 172.21.15.139:0/4030544082 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15e4025020 con 0x7f15ec15b610 2024-04-25T09:53:21.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.370+0000 7f15f295c640 1 -- 172.21.15.139:0/4030544082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15ec0703c0 con 0x7f15ec15b610 2024-04-25T09:53:21.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.371+0000 7f15e8ff9640 1 -- 172.21.15.139:0/4030544082 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f15e4059970 con 0x7f15ec15b610 2024-04-25T09:53:21.374 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:21.374 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:21.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.373+0000 7f15f295c640 1 -- 172.21.15.139:0/4030544082 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f15d806b4d0 msgr2=0x7f15d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.373+0000 7f15f295c640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f15d806b4d0 0x7f15d806d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0098b0 tx=0x7f15dc008040 comp rx=0 tx=0).stop 2024-04-25T09:53:21.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.373+0000 7f15f295c640 1 -- 172.21.15.139:0/4030544082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ec15b610 msgr2=0x7f15ec1608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.373+0000 7f15f295c640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ec15b610 0x7f15ec1608f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f15ec15e110 tx=0x7f15e4010590 comp rx=0 tx=0).stop 2024-04-25T09:53:21.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.373+0000 7f15f295c640 1 -- 172.21.15.139:0/4030544082 shutdown_connections 2024-04-25T09:53:21.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.373+0000 7f15f295c640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15ec15b610 0x7f15ec1608f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.373+0000 7f15f295c640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f15d806b4d0 0x7f15d806d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.373+0000 7f15f295c640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15ec152890 0x7f15ec1603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.373+0000 7f15f295c640 1 --2- 172.21.15.139:0/4030544082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15d4004800 0x7f15ec14f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.373+0000 7f15f295c640 1 -- 172.21.15.139:0/4030544082 >> 172.21.15.139:0/4030544082 conn(0x7f15ec071a00 msgr2=0x7f15ec0afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:21.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.374+0000 7f15f295c640 1 -- 172.21.15.139:0/4030544082 shutdown_connections 2024-04-25T09:53:21.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.374+0000 7f15f295c640 1 -- 172.21.15.139:0/4030544082 wait complete. 2024-04-25T09:53:21.386 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:21.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.481+0000 7fc11588f640 1 Processor -- start 2024-04-25T09:53:21.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.481+0000 7fc11588f640 1 -- start start 2024-04-25T09:53:21.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc11588f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc1100d1080 0x7fc1100b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc11588f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc110161850 0x7fc1100ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc11588f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1100cf250 con 0x7fc11015dc60 2024-04-25T09:53:21.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc11588f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1100ce780 con 0x7fc110161850 2024-04-25T09:53:21.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc11588f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1100d3730 con 0x7fc1100d1080 2024-04-25T09:53:21.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc10f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc110161850 0x7fc1100ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc10f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc110161850 0x7fc1100ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59948/0 (socket says 172.21.15.139:59948) 2024-04-25T09:53:21.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc10f7fe640 1 -- 172.21.15.139:0/634128196 learned_addr learned my addr 172.21.15.139:0/634128196 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:21.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc10e7fc640 1 --2- 172.21.15.139:0/634128196 >> v2:172.21.15.150:3300/0 conn(0x7fc1100d1080 0x7fc1100b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc10f7fe640 1 -- 172.21.15.139:0/634128196 >> v2:172.21.15.150:3300/0 conn(0x7fc1100d1080 msgr2=0x7fc1100b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:21.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc10f7fe640 1 --2- 172.21.15.139:0/634128196 >> v2:172.21.15.150:3300/0 conn(0x7fc1100d1080 0x7fc1100b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc10f7fe640 1 -- 172.21.15.139:0/634128196 >> v1:172.21.15.150:6789/0 conn(0x7fc11015dc60 legacy=0x7fc1100d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:21.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc10f7fe640 1 -- 172.21.15.139:0/634128196 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1100ce9b0 con 0x7fc110161850 2024-04-25T09:53:21.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.482+0000 7fc10e7fc640 1 --2- 172.21.15.139:0/634128196 >> v2:172.21.15.150:3300/0 conn(0x7fc1100d1080 0x7fc1100b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:21.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.483+0000 7fc10f7fe640 1 --2- 172.21.15.139:0/634128196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc110161850 0x7fc1100ce240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc110068750 tx=0x7fc10802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840fcc5f42280377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.483+0000 7fc10dffb640 1 -- 172.21.15.139:0/634128196 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108036070 con 0x7fc110161850 2024-04-25T09:53:21.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.483+0000 7fc10dffb640 1 -- 172.21.15.139:0/634128196 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc108031600 con 0x7fc110161850 2024-04-25T09:53:21.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.483+0000 7fc10dffb640 1 -- 172.21.15.139:0/634128196 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1080319d0 con 0x7fc110161850 2024-04-25T09:53:21.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.484+0000 7fc11588f640 1 -- 172.21.15.139:0/634128196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc110161850 msgr2=0x7fc1100ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.484+0000 7fc11588f640 1 --2- 172.21.15.139:0/634128196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc110161850 0x7fc1100ce240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc110068750 tx=0x7fc10802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.484+0000 7fc11588f640 1 -- 172.21.15.139:0/634128196 shutdown_connections 2024-04-25T09:53:21.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.484+0000 7fc11588f640 1 --2- 172.21.15.139:0/634128196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc110161850 0x7fc1100ce240 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.484+0000 7fc11588f640 1 --2- 172.21.15.139:0/634128196 >> v2:172.21.15.150:3300/0 conn(0x7fc1100d1080 0x7fc1100b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.484+0000 7fc11588f640 1 -- 172.21.15.139:0/634128196 >> 172.21.15.139:0/634128196 conn(0x7fc1100b5910 msgr2=0x7fc1100b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.484+0000 7fc11588f640 1 -- 172.21.15.139:0/634128196 shutdown_connections 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.484+0000 7fc11588f640 1 -- 172.21.15.139:0/634128196 wait complete. 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.485+0000 7fc11588f640 1 Processor -- start 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.485+0000 7fc11588f640 1 -- start start 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc11588f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1100d1080 0x7fc1100d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc11588f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11015dc60 0x7fc11014db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc11588f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc110161850 0x7fc11014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc11588f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1100d1fc0 con 0x7fc11015dc60 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc11588f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1100cef70 con 0x7fc1100d1080 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc11588f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11015e040 con 0x7fc110161850 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc10e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11015dc60 0x7fc11014db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc10e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11015dc60 0x7fc11014db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59962/0 (socket says 172.21.15.139:59962) 2024-04-25T09:53:21.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc10e7fc640 1 -- 172.21.15.139:0/61559763 learned_addr learned my addr 172.21.15.139:0/61559763 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:21.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc10effd640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1100d1080 0x7fc1100d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc10f7fe640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc110161850 0x7fc11014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc10e7fc640 1 -- 172.21.15.139:0/61559763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc110161850 msgr2=0x7fc11014e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc10e7fc640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc110161850 0x7fc11014e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc10e7fc640 1 -- 172.21.15.139:0/61559763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1100d1080 msgr2=0x7fc1100d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc10e7fc640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1100d1080 0x7fc1100d3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc10e7fc640 1 -- 172.21.15.139:0/61559763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1100c8600 con 0x7fc11015dc60 2024-04-25T09:53:21.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc10e7fc640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11015dc60 0x7fc11014db40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc10400c7f0 tx=0x7fc10400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc0f7fff640 1 -- 172.21.15.139:0/61559763 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104015070 con 0x7fc11015dc60 2024-04-25T09:53:21.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc0f7fff640 1 -- 172.21.15.139:0/61559763 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc104007820 con 0x7fc11015dc60 2024-04-25T09:53:21.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc0f7fff640 1 -- 172.21.15.139:0/61559763 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104007bd0 con 0x7fc11015dc60 2024-04-25T09:53:21.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.486+0000 7fc11588f640 1 -- 172.21.15.139:0/61559763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1100c88e0 con 0x7fc11015dc60 2024-04-25T09:53:21.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.487+0000 7fc11588f640 1 -- 172.21.15.139:0/61559763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1100c8ea0 con 0x7fc11015dc60 2024-04-25T09:53:21.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.487+0000 7fc11588f640 1 -- 172.21.15.139:0/61559763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc110074520 con 0x7fc11015dc60 2024-04-25T09:53:21.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.488+0000 7fc0f7fff640 1 -- 172.21.15.139:0/61559763 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc104018070 con 0x7fc11015dc60 2024-04-25T09:53:21.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.488+0000 7fc0f7fff640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.488+0000 7fc10effd640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.488+0000 7fc0f7fff640 1 -- 172.21.15.139:0/61559763 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc10408a5c0 con 0x7fc11015dc60 2024-04-25T09:53:21.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.489+0000 7fc10effd640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc100004f60 tx=0x7fc100007e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.492+0000 7fc0f7fff640 1 -- 172.21.15.139:0/61559763 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc104058ab0 con 0x7fc11015dc60 2024-04-25T09:53:21.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.663+0000 7fc11588f640 1 -- 172.21.15.139:0/61559763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1100d16f0 con 0x7fc11015dc60 2024-04-25T09:53:21.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.664+0000 7fc0f7fff640 1 -- 172.21.15.139:0/61559763 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7fc10405cb50 con 0x7fc11015dc60 2024-04-25T09:53:21.666 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:21.667 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.666+0000 7fc11588f640 1 -- 172.21.15.139:0/61559763 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0fc06b4d0 msgr2=0x7fc0fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.666+0000 7fc11588f640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc100004f60 tx=0x7fc100007e30 comp rx=0 tx=0).stop 2024-04-25T09:53:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.666+0000 7fc11588f640 1 -- 172.21.15.139:0/61559763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11015dc60 msgr2=0x7fc11014db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.666+0000 7fc11588f640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11015dc60 0x7fc11014db40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc10400c7f0 tx=0x7fc10400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.666+0000 7fc11588f640 1 -- 172.21.15.139:0/61559763 shutdown_connections 2024-04-25T09:53:21.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.666+0000 7fc11588f640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11015dc60 0x7fc11014db40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.666+0000 7fc11588f640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.666+0000 7fc11588f640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc110161850 0x7fc11014e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.666+0000 7fc11588f640 1 --2- 172.21.15.139:0/61559763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1100d1080 0x7fc1100d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.666+0000 7fc11588f640 1 -- 172.21.15.139:0/61559763 >> 172.21.15.139:0/61559763 conn(0x7fc1100b5910 msgr2=0x7fc1100cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:21.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.666+0000 7fc11588f640 1 -- 172.21.15.139:0/61559763 shutdown_connections 2024-04-25T09:53:21.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.666+0000 7fc11588f640 1 -- 172.21.15.139:0/61559763 wait complete. 2024-04-25T09:53:21.700 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:21.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.796+0000 7f44a9768640 1 Processor -- start 2024-04-25T09:53:21.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.796+0000 7f44a9768640 1 -- start start 2024-04-25T09:53:21.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.797+0000 7f44a9768640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f44a40d1080 0x7f44a40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.797+0000 7f44a9768640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a40cf250 con 0x7f44a415dc60 2024-04-25T09:53:21.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.797+0000 7f44a9768640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a40ce9b0 con 0x7f44a4161850 2024-04-25T09:53:21.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.797+0000 7f44a9768640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a40d3730 con 0x7f44a40d1080 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.797+0000 7f449bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f44a40d1080 0x7f44a40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.797+0000 7f44a2ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f44a415dc60 0x7f44a40d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38764/0 (socket says 172.21.15.139:38764) 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.797+0000 7f44a2ffd640 1 -- 172.21.15.139:0/3911868099 learned_addr learned my addr 172.21.15.139:0/3911868099 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.797+0000 7f449bfff640 1 -- 172.21.15.139:0/3911868099 >> v1:172.21.15.150:6789/0 conn(0x7f44a4161850 legacy=0x7f44a40d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.797+0000 7f449bfff640 1 -- 172.21.15.139:0/3911868099 >> v1:172.21.15.139:6789/0 conn(0x7f44a415dc60 legacy=0x7f44a40d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.797+0000 7f449bfff640 1 -- 172.21.15.139:0/3911868099 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a40cec90 con 0x7f44a40d1080 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.797+0000 7f44a27fc640 1 -- 172.21.15.139:0/3911868099 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362855500 0 0) 0x7f4494001110 con 0x7f44a415dc60 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.798+0000 7f449bfff640 1 --2- 172.21.15.139:0/3911868099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a40d1080 0x7f44a40b6c10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f448c0097a0 tx=0x7f448c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9fff88f00565d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.798+0000 7f44a27fc640 1 -- 172.21.15.139:0/3911868099 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c037070 con 0x7f44a40d1080 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.798+0000 7f44a27fc640 1 -- 172.21.15.139:0/3911868099 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c032440 con 0x7f44a40d1080 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.798+0000 7f44a27fc640 1 -- 172.21.15.139:0/3911868099 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c032810 con 0x7f44a40d1080 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.798+0000 7f44a9768640 1 -- 172.21.15.139:0/3911868099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a40d1080 msgr2=0x7f44a40b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.798+0000 7f44a9768640 1 --2- 172.21.15.139:0/3911868099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a40d1080 0x7f44a40b6c10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f448c0097a0 tx=0x7f448c02a230 comp rx=0 tx=0).stop 2024-04-25T09:53:21.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.798+0000 7f44a9768640 1 -- 172.21.15.139:0/3911868099 shutdown_connections 2024-04-25T09:53:21.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.798+0000 7f44a9768640 1 --2- 172.21.15.139:0/3911868099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a40d1080 0x7f44a40b6c10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.798+0000 7f44a9768640 1 -- 172.21.15.139:0/3911868099 >> 172.21.15.139:0/3911868099 conn(0x7f44a40b5910 msgr2=0x7f44a40b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:21.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.798+0000 7f44a9768640 1 -- 172.21.15.139:0/3911868099 shutdown_connections 2024-04-25T09:53:21.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.798+0000 7f44a9768640 1 -- 172.21.15.139:0/3911868099 wait complete. 2024-04-25T09:53:21.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.800+0000 7f44a9768640 1 Processor -- start 2024-04-25T09:53:21.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.800+0000 7f44a9768640 1 -- start start 2024-04-25T09:53:21.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.800+0000 7f44a9768640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a40d1080 0x7f44a4160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.800+0000 7f44a9768640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a415dc60 0x7f44a4153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.800+0000 7f44a9768640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a4161850 0x7f44a415f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.800+0000 7f44a9768640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a415e040 con 0x7f44a415dc60 2024-04-25T09:53:21.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.800+0000 7f44a9768640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40d1fc0 con 0x7f44a40d1080 2024-04-25T09:53:21.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.800+0000 7f44a9768640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a415fad0 con 0x7f44a4161850 2024-04-25T09:53:21.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.800+0000 7f449bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a415dc60 0x7f44a4153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.800+0000 7f449bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a415dc60 0x7f44a4153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59984/0 (socket says 172.21.15.139:59984) 2024-04-25T09:53:21.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.800+0000 7f449bfff640 1 -- 172.21.15.139:0/2185512568 learned_addr learned my addr 172.21.15.139:0/2185512568 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:21.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.801+0000 7f449bfff640 1 -- 172.21.15.139:0/2185512568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a4161850 msgr2=0x7f44a415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.801+0000 7f449bfff640 1 --2- 172.21.15.139:0/2185512568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a4161850 0x7f44a415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.801+0000 7f449bfff640 1 -- 172.21.15.139:0/2185512568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a40d1080 msgr2=0x7f44a4160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.801+0000 7f449bfff640 1 --2- 172.21.15.139:0/2185512568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a40d1080 0x7f44a4160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.801+0000 7f449bfff640 1 -- 172.21.15.139:0/2185512568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a415fd00 con 0x7f44a415dc60 2024-04-25T09:53:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.801+0000 7f449bfff640 1 --2- 172.21.15.139:0/2185512568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a415dc60 0x7f44a4153550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f448c0097a0 tx=0x7f448c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.801+0000 7f449b7fe640 1 -- 172.21.15.139:0/2185512568 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c037050 con 0x7f44a415dc60 2024-04-25T09:53:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.801+0000 7f449b7fe640 1 -- 172.21.15.139:0/2185512568 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c038040 con 0x7f44a415dc60 2024-04-25T09:53:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.801+0000 7f44a9768640 1 -- 172.21.15.139:0/2185512568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a415ffe0 con 0x7f44a415dc60 2024-04-25T09:53:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.801+0000 7f449b7fe640 1 -- 172.21.15.139:0/2185512568 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c004490 con 0x7f44a415dc60 2024-04-25T09:53:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.801+0000 7f44a9768640 1 -- 172.21.15.139:0/2185512568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a41605a0 con 0x7f44a415dc60 2024-04-25T09:53:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.802+0000 7f44a9768640 1 -- 172.21.15.139:0/2185512568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a4074520 con 0x7f44a415dc60 2024-04-25T09:53:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.802+0000 7f449b7fe640 1 -- 172.21.15.139:0/2185512568 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f448c030900 con 0x7f44a415dc60 2024-04-25T09:53:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.803+0000 7f449b7fe640 1 --2- 172.21.15.139:0/2185512568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f448406b500 0x7f448406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.803+0000 7f449b7fe640 1 -- 172.21.15.139:0/2185512568 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f448c0ab210 con 0x7f44a415dc60 2024-04-25T09:53:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.803+0000 7f44a2ffd640 1 --2- 172.21.15.139:0/2185512568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f448406b500 0x7f448406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.803+0000 7f44a2ffd640 1 --2- 172.21.15.139:0/2185512568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f448406b500 0x7f448406d9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f44940013b0 tx=0x7f4494001300 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:21.807+0000 7f449b7fe640 1 -- 172.21.15.139:0/2185512568 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f448c079e70 con 0x7f44a415dc60 2024-04-25T09:53:22.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.011+0000 7f44a9768640 1 -- 172.21.15.139:0/2185512568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44a40d16f0 con 0x7f44a415dc60 2024-04-25T09:53:22.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.011+0000 7f449b7fe640 1 -- 172.21.15.139:0/2185512568 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f448c079770 con 0x7f44a415dc60 2024-04-25T09:53:22.013 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:22.014 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/892226922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":892226922},{"type":"v1","addr":"172.21.15.150:6834","nonce":892226922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/582019435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":582019435},{"type":"v1","addr":"172.21.15.139:6837","nonce":582019435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/2751714076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2751714076},{"type":"v1","addr":"172.21.15.139:6834","nonce":2751714076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:15.042067+0000","modified":"2024-04-25T09:53:20.100914+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2160557452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2160557452},{"type":"v1","addr":"172.21.15.150:6835","nonce":2160557452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T09:53:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 7 2024-04-25T09:53:22.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.013+0000 7f44a9768640 1 -- 172.21.15.139:0/2185512568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f448406b500 msgr2=0x7f448406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.013+0000 7f44a9768640 1 --2- 172.21.15.139:0/2185512568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f448406b500 0x7f448406d9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f44940013b0 tx=0x7f4494001300 comp rx=0 tx=0).stop 2024-04-25T09:53:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.013+0000 7f44a9768640 1 -- 172.21.15.139:0/2185512568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a415dc60 msgr2=0x7f44a4153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.013+0000 7f44a9768640 1 --2- 172.21.15.139:0/2185512568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a415dc60 0x7f44a4153550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f448c0097a0 tx=0x7f448c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.013+0000 7f44a9768640 1 -- 172.21.15.139:0/2185512568 shutdown_connections 2024-04-25T09:53:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.013+0000 7f44a9768640 1 --2- 172.21.15.139:0/2185512568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a415dc60 0x7f44a4153550 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.013+0000 7f44a9768640 1 --2- 172.21.15.139:0/2185512568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f448406b500 0x7f448406d9c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.013+0000 7f44a9768640 1 --2- 172.21.15.139:0/2185512568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a4161850 0x7f44a415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.013+0000 7f44a9768640 1 --2- 172.21.15.139:0/2185512568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a40d1080 0x7f44a4160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.013+0000 7f44a9768640 1 -- 172.21.15.139:0/2185512568 >> 172.21.15.139:0/2185512568 conn(0x7f44a40b5910 msgr2=0x7f44a406cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.013+0000 7f44a9768640 1 -- 172.21.15.139:0/2185512568 shutdown_connections 2024-04-25T09:53:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.013+0000 7f44a9768640 1 -- 172.21.15.139:0/2185512568 wait complete. 2024-04-25T09:53:22.027 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.122+0000 7fa737b7e640 1 Processor -- start 2024-04-25T09:53:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.123+0000 7fa737b7e640 1 -- start start 2024-04-25T09:53:22.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.123+0000 7fa737b7e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa73015ac40 0x7fa73006e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.123+0000 7fa737b7e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa73006e850 0x7fa7300b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.123+0000 7fa737b7e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7300af6d0 con 0x7fa73015a860 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.123+0000 7fa737b7e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73006eca0 con 0x7fa73006e850 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.123+0000 7fa737b7e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7300b02e0 con 0x7fa73015ac40 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.124+0000 7fa73611b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa73006e850 0x7fa7300b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.124+0000 7fa73611b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa73006e850 0x7fa7300b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59998/0 (socket says 172.21.15.139:59998) 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.124+0000 7fa73591a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa73015a860 0x7fa7300cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38772/0 (socket says 172.21.15.139:38772) 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.124+0000 7fa73611b640 1 -- 172.21.15.139:0/3298294546 learned_addr learned my addr 172.21.15.139:0/3298294546 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.124+0000 7fa73611b640 1 -- 172.21.15.139:0/3298294546 >> v2:172.21.15.150:3300/0 conn(0x7fa73015ac40 msgr2=0x7fa73006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.124+0000 7fa73611b640 1 --2- 172.21.15.139:0/3298294546 >> v2:172.21.15.150:3300/0 conn(0x7fa73015ac40 0x7fa73006e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.124+0000 7fa73611b640 1 -- 172.21.15.139:0/3298294546 >> v1:172.21.15.139:6789/0 conn(0x7fa73015a860 legacy=0x7fa7300cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.124+0000 7fa734918640 1 -- 172.21.15.139:0/3298294546 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165799587 0 0) 0x7fa7300af6d0 con 0x7fa73015a860 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.124+0000 7fa73611b640 1 -- 172.21.15.139:0/3298294546 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7300af9b0 con 0x7fa73006e850 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.125+0000 7fa73611b640 1 --2- 172.21.15.139:0/3298294546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa73006e850 0x7fa7300b0770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa7300b2610 tx=0x7fa72c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b35d33188a31706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.125+0000 7fa734918640 1 -- 172.21.15.139:0/3298294546 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c037070 con 0x7fa73006e850 2024-04-25T09:53:22.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.125+0000 7fa734918640 1 -- 172.21.15.139:0/3298294546 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c032440 con 0x7fa73006e850 2024-04-25T09:53:22.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.125+0000 7fa734918640 1 -- 172.21.15.139:0/3298294546 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c032810 con 0x7fa73006e850 2024-04-25T09:53:22.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.125+0000 7fa737b7e640 1 -- 172.21.15.139:0/3298294546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa73006e850 msgr2=0x7fa7300b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.125+0000 7fa737b7e640 1 --2- 172.21.15.139:0/3298294546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa73006e850 0x7fa7300b0770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa7300b2610 tx=0x7fa72c02a270 comp rx=0 tx=0).stop 2024-04-25T09:53:22.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.125+0000 7fa737b7e640 1 -- 172.21.15.139:0/3298294546 shutdown_connections 2024-04-25T09:53:22.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.125+0000 7fa737b7e640 1 --2- 172.21.15.139:0/3298294546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa73006e850 0x7fa7300b0770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.125+0000 7fa737b7e640 1 --2- 172.21.15.139:0/3298294546 >> v2:172.21.15.150:3300/0 conn(0x7fa73015ac40 0x7fa73006e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.125+0000 7fa737b7e640 1 -- 172.21.15.139:0/3298294546 >> 172.21.15.139:0/3298294546 conn(0x7fa730066f20 msgr2=0x7fa73006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:22.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.125+0000 7fa737b7e640 1 -- 172.21.15.139:0/3298294546 shutdown_connections 2024-04-25T09:53:22.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.125+0000 7fa737b7e640 1 -- 172.21.15.139:0/3298294546 wait complete. 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.126+0000 7fa737b7e640 1 Processor -- start 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.127+0000 7fa737b7e640 1 -- start start 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.127+0000 7fa737b7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa73006e850 0x7fa73014d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.127+0000 7fa737b7e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa73015a860 0x7fa7300b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.127+0000 7fa737b7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa73015ac40 0x7fa7301537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.127+0000 7fa737b7e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300cd3c0 con 0x7fa73006e850 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.127+0000 7fa737b7e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300aff70 con 0x7fa73015a860 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.127+0000 7fa737b7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300af2a0 con 0x7fa73015ac40 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.127+0000 7fa73591a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa73006e850 0x7fa73014d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.127+0000 7fa73591a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa73006e850 0x7fa73014d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60006/0 (socket says 172.21.15.139:60006) 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.127+0000 7fa73591a640 1 -- 172.21.15.139:0/3892941817 learned_addr learned my addr 172.21.15.139:0/3892941817 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.128+0000 7fa73591a640 1 -- 172.21.15.139:0/3892941817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa73015ac40 msgr2=0x7fa7301537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.128+0000 7fa73591a640 1 --2- 172.21.15.139:0/3892941817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa73015ac40 0x7fa7301537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.128+0000 7fa73591a640 1 -- 172.21.15.139:0/3892941817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa73015a860 msgr2=0x7fa7300b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.128+0000 7fa73591a640 1 --2- 172.21.15.139:0/3892941817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa73015a860 0x7fa7300b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.128+0000 7fa73591a640 1 -- 172.21.15.139:0/3892941817 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7300b3620 con 0x7fa73006e850 2024-04-25T09:53:22.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.128+0000 7fa73591a640 1 --2- 172.21.15.139:0/3892941817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa73006e850 0x7fa73014d1b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa72800c890 tx=0x7fa72800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.128+0000 7fa7267fc640 1 -- 172.21.15.139:0/3892941817 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728018070 con 0x7fa73006e850 2024-04-25T09:53:22.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.128+0000 7fa7267fc640 1 -- 172.21.15.139:0/3892941817 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72800a640 con 0x7fa73006e850 2024-04-25T09:53:22.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.128+0000 7fa7267fc640 1 -- 172.21.15.139:0/3892941817 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72800aa30 con 0x7fa73006e850 2024-04-25T09:53:22.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.128+0000 7fa737b7e640 1 -- 172.21.15.139:0/3892941817 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7300b3900 con 0x7fa73006e850 2024-04-25T09:53:22.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.128+0000 7fa737b7e640 1 -- 172.21.15.139:0/3892941817 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7300b3ec0 con 0x7fa73006e850 2024-04-25T09:53:22.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.129+0000 7fa737b7e640 1 -- 172.21.15.139:0/3892941817 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa730074520 con 0x7fa73006e850 2024-04-25T09:53:22.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.129+0000 7fa7267fc640 1 -- 172.21.15.139:0/3892941817 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa72801b070 con 0x7fa73006e850 2024-04-25T09:53:22.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.130+0000 7fa7267fc640 1 --2- 172.21.15.139:0/3892941817 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa71406b4d0 0x7fa71406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.130+0000 7fa7267fc640 1 -- 172.21.15.139:0/3892941817 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa728089c30 con 0x7fa73006e850 2024-04-25T09:53:22.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.130+0000 7fa735119640 1 --2- 172.21.15.139:0/3892941817 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa71406b4d0 0x7fa71406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.130+0000 7fa735119640 1 --2- 172.21.15.139:0/3892941817 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa71406b4d0 0x7fa71406d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7300cca50 tx=0x7fa720006c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.134+0000 7fa7267fc640 1 -- 172.21.15.139:0/3892941817 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa728058770 con 0x7fa73006e850 2024-04-25T09:53:22.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.304+0000 7fa737b7e640 1 -- 172.21.15.139:0/3892941817 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa73006de60 con 0x7fa73006e850 2024-04-25T09:53:22.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.306+0000 7fa7267fc640 1 -- 172.21.15.139:0/3892941817 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7fa728058070 con 0x7fa73006e850 2024-04-25T09:53:22.308 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:22.308 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:22.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.308+0000 7fa737b7e640 1 -- 172.21.15.139:0/3892941817 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa71406b4d0 msgr2=0x7fa71406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.308+0000 7fa737b7e640 1 --2- 172.21.15.139:0/3892941817 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa71406b4d0 0x7fa71406d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7300cca50 tx=0x7fa720006c90 comp rx=0 tx=0).stop 2024-04-25T09:53:22.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.308+0000 7fa737b7e640 1 -- 172.21.15.139:0/3892941817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa73006e850 msgr2=0x7fa73014d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.308+0000 7fa737b7e640 1 --2- 172.21.15.139:0/3892941817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa73006e850 0x7fa73014d1b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa72800c890 tx=0x7fa72800cd50 comp rx=0 tx=0).stop 2024-04-25T09:53:22.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.308+0000 7fa737b7e640 1 -- 172.21.15.139:0/3892941817 shutdown_connections 2024-04-25T09:53:22.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.308+0000 7fa737b7e640 1 --2- 172.21.15.139:0/3892941817 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa71406b4d0 0x7fa71406d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.308+0000 7fa737b7e640 1 --2- 172.21.15.139:0/3892941817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa73015ac40 0x7fa7301537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.308+0000 7fa737b7e640 1 --2- 172.21.15.139:0/3892941817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa73015a860 0x7fa7300b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.308+0000 7fa737b7e640 1 --2- 172.21.15.139:0/3892941817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa73006e850 0x7fa73014d1b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.308+0000 7fa737b7e640 1 -- 172.21.15.139:0/3892941817 >> 172.21.15.139:0/3892941817 conn(0x7fa730066f20 msgr2=0x7fa730158000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:22.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.308+0000 7fa737b7e640 1 -- 172.21.15.139:0/3892941817 shutdown_connections 2024-04-25T09:53:22.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.308+0000 7fa737b7e640 1 -- 172.21.15.139:0/3892941817 wait complete. 2024-04-25T09:53:22.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:53:15.042067+0000', 'modified': '2024-04-25T09:53:20.100914+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6835/2160557452', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6833', 'nonce': 2160557452}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 2160557452}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-04-25T09:53:22.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:53:22.724 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T09:53:22.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.820+0000 7f191256a640 1 Processor -- start 2024-04-25T09:53:22.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.820+0000 7f191256a640 1 -- start start 2024-04-25T09:53:22.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f191256a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f190c15dbf0 0x7f190c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f191256a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f190c0b9370 0x7f190c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f191256a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c161dc0 con 0x7f190c0cec80 2024-04-25T09:53:22.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f191256a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190c0d1ee0 con 0x7f190c15dbf0 2024-04-25T09:53:22.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f191256a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190c1620a0 con 0x7f190c0b9370 2024-04-25T09:53:22.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f190bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f190c15dbf0 0x7f190c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f190bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f190c15dbf0 0x7f190c1617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60012/0 (socket says 172.21.15.139:60012) 2024-04-25T09:53:22.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f190bfff640 1 -- 172.21.15.139:0/1773961962 learned_addr learned my addr 172.21.15.139:0/1773961962 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:22.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f190b7fe640 1 --2- 172.21.15.139:0/1773961962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f190c0b9370 0x7f190c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f190bfff640 1 -- 172.21.15.139:0/1773961962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f190c0b9370 msgr2=0x7f190c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f190bfff640 1 --2- 172.21.15.139:0/1773961962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f190c0b9370 0x7f190c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f190bfff640 1 -- 172.21.15.139:0/1773961962 >> v1:172.21.15.150:6789/0 conn(0x7f190c0cec80 legacy=0x7f190c06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.821+0000 7f190bfff640 1 -- 172.21.15.139:0/1773961962 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c162380 con 0x7f190c15dbf0 2024-04-25T09:53:22.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.822+0000 7f190bfff640 1 --2- 172.21.15.139:0/1773961962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f190c15dbf0 0x7f190c1617e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009870 tx=0x7f18fc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37354fcdbd40377a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.822+0000 7f190affd640 1 -- 172.21.15.139:0/1773961962 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc037070 con 0x7f190c15dbf0 2024-04-25T09:53:22.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.822+0000 7f190affd640 1 -- 172.21.15.139:0/1773961962 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc032440 con 0x7f190c15dbf0 2024-04-25T09:53:22.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.822+0000 7f190affd640 1 -- 172.21.15.139:0/1773961962 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc032810 con 0x7f190c15dbf0 2024-04-25T09:53:22.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.823+0000 7f191256a640 1 -- 172.21.15.139:0/1773961962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f190c15dbf0 msgr2=0x7f190c1617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.823+0000 7f191256a640 1 --2- 172.21.15.139:0/1773961962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f190c15dbf0 0x7f190c1617e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009870 tx=0x7f18fc02a210 comp rx=0 tx=0).stop 2024-04-25T09:53:22.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.823+0000 7f191256a640 1 -- 172.21.15.139:0/1773961962 shutdown_connections 2024-04-25T09:53:22.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.823+0000 7f191256a640 1 --2- 172.21.15.139:0/1773961962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f190c0b9370 0x7f190c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.823+0000 7f191256a640 1 --2- 172.21.15.139:0/1773961962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f190c15dbf0 0x7f190c1617e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.823+0000 7f191256a640 1 -- 172.21.15.139:0/1773961962 >> 172.21.15.139:0/1773961962 conn(0x7f190c0b5480 msgr2=0x7f190c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:22.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.823+0000 7f191256a640 1 -- 172.21.15.139:0/1773961962 shutdown_connections 2024-04-25T09:53:22.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.823+0000 7f191256a640 1 -- 172.21.15.139:0/1773961962 wait complete. 2024-04-25T09:53:22.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.824+0000 7f191256a640 1 Processor -- start 2024-04-25T09:53:22.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.824+0000 7f191256a640 1 -- start start 2024-04-25T09:53:22.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f191256a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f190c0b9370 0x7f190c0abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f191256a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f190c0cec80 0x7f190c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f191256a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f190c15dbf0 0x7f190c14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f191256a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c15dfd0 con 0x7f190c0b9370 2024-04-25T09:53:22.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f191256a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0b97d0 con 0x7f190c0cec80 2024-04-25T09:53:22.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f191256a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0ab6d0 con 0x7f190c15dbf0 2024-04-25T09:53:22.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f190bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f190c0b9370 0x7f190c0abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f190bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f190c0b9370 0x7f190c0abb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60024/0 (socket says 172.21.15.139:60024) 2024-04-25T09:53:22.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f190bfff640 1 -- 172.21.15.139:0/698171336 learned_addr learned my addr 172.21.15.139:0/698171336 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f1910b07640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f190c15dbf0 0x7f190c14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f190b7fe640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f190c0cec80 0x7f190c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f190bfff640 1 -- 172.21.15.139:0/698171336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f190c15dbf0 msgr2=0x7f190c14ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f190bfff640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f190c15dbf0 0x7f190c14ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f190bfff640 1 -- 172.21.15.139:0/698171336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f190c0cec80 msgr2=0x7f190c0b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f190bfff640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f190c0cec80 0x7f190c0b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f190bfff640 1 -- 172.21.15.139:0/698171336 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c0aaac0 con 0x7f190c0b9370 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.825+0000 7f190bfff640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f190c0b9370 0x7f190c0abb60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009870 tx=0x7f18fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.826+0000 7f1908ff9640 1 -- 172.21.15.139:0/698171336 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc041070 con 0x7f190c0b9370 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.826+0000 7f1908ff9640 1 -- 172.21.15.139:0/698171336 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc03c890 con 0x7f190c0b9370 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.826+0000 7f191256a640 1 -- 172.21.15.139:0/698171336 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190c0aada0 con 0x7f190c0b9370 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.826+0000 7f1908ff9640 1 -- 172.21.15.139:0/698171336 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc037040 con 0x7f190c0b9370 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.826+0000 7f191256a640 1 -- 172.21.15.139:0/698171336 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c0ab360 con 0x7f190c0b9370 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.827+0000 7f1908ff9640 1 -- 172.21.15.139:0/698171336 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18fc044070 con 0x7f190c0b9370 2024-04-25T09:53:22.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.828+0000 7f1908ff9640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f18f806b4d0 0x7f18f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.828+0000 7f190b7fe640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f18f806b4d0 0x7f18f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.828+0000 7f1908ff9640 1 -- 172.21.15.139:0/698171336 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f18fc0aaf10 con 0x7f190c0b9370 2024-04-25T09:53:22.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.828+0000 7f190b7fe640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f18f806b4d0 0x7f18f806d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f190c0b5d60 tx=0x7f1900008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.829+0000 7f191525f740 1 Processor -- start 2024-04-25T09:53:22.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.829+0000 7f191525f740 1 -- start start 2024-04-25T09:53:22.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.829+0000 7f191525f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7b83230 0x55a0f7b83610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.829+0000 7f191525f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 0x55a0f7ba1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.829+0000 7f191525f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b58f10 0x55a0f7b9f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.829+0000 7f191525f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f7abba10 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.829+0000 7f191525f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f7a64b30 con 0x55a0f7b58f10 2024-04-25T09:53:22.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.829+0000 7f191525f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0f7ab7c50 con 0x55a0f7b83230 2024-04-25T09:53:22.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f190b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 0x55a0f7ba1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f190b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 0x55a0f7ba1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60036/0 (socket says 172.21.15.139:60036) 2024-04-25T09:53:22.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f190b7fe640 1 -- 172.21.15.139:0/268720461 learned_addr learned my addr 172.21.15.139:0/268720461 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:22.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f190bfff640 1 --2- 172.21.15.139:0/268720461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7b83230 0x55a0f7b83610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f190b7fe640 1 -- 172.21.15.139:0/268720461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7b83230 msgr2=0x55a0f7b83610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f190b7fe640 1 --2- 172.21.15.139:0/268720461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7b83230 0x55a0f7b83610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f190b7fe640 1 -- 172.21.15.139:0/268720461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b58f10 msgr2=0x55a0f7b9f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f1910b07640 1 --2- 172.21.15.139:0/268720461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b58f10 0x55a0f7b9f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f190b7fe640 1 --2- 172.21.15.139:0/268720461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b58f10 0x55a0f7b9f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f190b7fe640 1 -- 172.21.15.139:0/268720461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0f7a79e10 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f190bfff640 1 --2- 172.21.15.139:0/268720461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7b83230 0x55a0f7b83610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f190b7fe640 1 --2- 172.21.15.139:0/268720461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 0x55a0f7ba1020 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1900010a30 tx=0x7f1900013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa66589a20ef267e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f18f1ffb640 1 -- 172.21.15.139:0/268720461 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190001c050 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f18f1ffb640 1 -- 172.21.15.139:0/268720461 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1900026070 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.830+0000 7f18f1ffb640 1 -- 172.21.15.139:0/268720461 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900016e10 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.831+0000 7f191525f740 1 -- 172.21.15.139:0/268720461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 msgr2=0x55a0f7ba1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.831+0000 7f191525f740 1 --2- 172.21.15.139:0/268720461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 0x55a0f7ba1020 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1900010a30 tx=0x7f1900013000 comp rx=0 tx=0).stop 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.831+0000 7f191525f740 1 -- 172.21.15.139:0/268720461 shutdown_connections 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.831+0000 7f191525f740 1 --2- 172.21.15.139:0/268720461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 0x55a0f7ba1020 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.831+0000 7f191525f740 1 --2- 172.21.15.139:0/268720461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b58f10 0x55a0f7b9f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.831+0000 7f191525f740 1 --2- 172.21.15.139:0/268720461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7b83230 0x55a0f7b83610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.831+0000 7f191525f740 1 -- 172.21.15.139:0/268720461 >> 172.21.15.139:0/268720461 conn(0x55a0f7b55840 msgr2=0x55a0f7b9e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:22.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.831+0000 7f191525f740 1 -- 172.21.15.139:0/268720461 shutdown_connections 2024-04-25T09:53:22.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.831+0000 7f191525f740 1 -- 172.21.15.139:0/268720461 wait complete. 2024-04-25T09:53:22.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.831+0000 7f191525f740 1 -- start start 2024-04-25T09:53:22.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f191525f740 1 Processor -- start 2024-04-25T09:53:22.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f191525f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b83230 0x55a0f7b7f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f191525f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7ba0ad0 0x55a0f7baed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f191525f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b96490 0x55a0f7b96b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f191525f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f7b59cb0 con 0x55a0f7b83230 2024-04-25T09:53:22.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f191525f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f7a780c0 con 0x55a0f7b96490 2024-04-25T09:53:22.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f191525f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0f7b53770 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f190bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b83230 0x55a0f7b7f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f191525f740 10 client.0 _ll_register_callbacks cb 0x55a0f7a6bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:53:22.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f190bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b83230 0x55a0f7b7f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60038/0 (socket says 172.21.15.139:60038) 2024-04-25T09:53:22.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f190bfff640 1 -- 172.21.15.139:0/1532991868 learned_addr learned my addr 172.21.15.139:0/1532991868 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:22.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f190b7fe640 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7ba0ad0 0x55a0f7baed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.832+0000 7f1910b07640 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b96490 0x55a0f7b96b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f190bfff640 1 -- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7ba0ad0 msgr2=0x55a0f7baed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f190bfff640 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7ba0ad0 0x55a0f7baed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f190bfff640 1 -- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b96490 msgr2=0x55a0f7b96b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f190bfff640 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b96490 0x55a0f7b96b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f190bfff640 1 -- 172.21.15.139:0/1532991868 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0f77d02d0 con 0x55a0f7b83230 2024-04-25T09:53:22.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f1910b07640 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b96490 0x55a0f7b96b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:22.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f190bfff640 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b83230 0x55a0f7b7f940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f18fc006b10 tx=0x7f18fc078f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242ad0c35d34c9c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f18eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:53:22.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f18eaffd640 1 -- 172.21.15.139:0/1532991868 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc04b040 con 0x55a0f7b83230 2024-04-25T09:53:22.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f18eaffd640 1 -- 172.21.15.139:0/1532991868 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f18fc079a10 con 0x55a0f7b83230 2024-04-25T09:53:22.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f191525f740 1 -- 172.21.15.139:0/1532991868 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1900026070 con 0x55a0f7b83230 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f18eaffd640 1 -- 172.21.15.139:0/1532991868 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18fc081e60 con 0x55a0f7b83230 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f191525f740 10 client.4446 fetch_fsmap learned FSMap version 7 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f191525f740 1 -- 172.21.15.139:0/1532991868 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55a0f789a6b0 con 0x55a0f7b83230 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f18eaffd640 1 -- 172.21.15.139:0/1532991868 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1900026070 con 0x55a0f7b83230 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f18eaffd640 1 -- 172.21.15.139:0/1532991868 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f18fc081780 con 0x55a0f7b83230 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f191525f740 10 client.4446 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f191525f740 10 client.4446 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f191525f740 20 client.4446 populate_metadata read hostname 'smithi139' 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f191525f740 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x55a0f7b84eb0 0x55a0f7b85290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f191525f740 4 client.4446 mds_command: new command op to 4299 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.833+0000 7f191525f740 1 -- 172.21.15.139:0/1532991868 --> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0f7b1a920 con 0x55a0f7b84eb0 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.834+0000 7f190b7fe640 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x55a0f7b84eb0 0x55a0f7b85290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.834+0000 7f190b7fe640 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x55a0f7b84eb0 0x55a0f7b85290 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37ad26b532dccd60 server_cookie=282a8beb61e6a113 in_seq=0 out_seq=0 2024-04-25T09:53:22.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.834+0000 7f18eaffd640 10 client.4446 ms_handle_connect on v2:172.21.15.150:6833/2160557452 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f18eaffd640 1 -- 172.21.15.139:0/1532991868 <== mds.0 v2:172.21.15.150:6833/2160557452 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a0f7b1a920 con 0x55a0f7b84eb0 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f18eaffd640 10 client.4446 handle_command_reply: tid=0 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f191525f740 1 client.4446 shutdown 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f191525f740 1 -- 172.21.15.139:0/1532991868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b83230 msgr2=0x55a0f7b7f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f191525f740 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b83230 0x55a0f7b7f940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f18fc006b10 tx=0x7f18fc078f10 comp rx=0 tx=0).stop 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f191525f740 1 -- 172.21.15.139:0/1532991868 shutdown_connections 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f191525f740 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x55a0f7b84eb0 0x55a0f7b85290 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f191525f740 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b96490 0x55a0f7b96b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f191525f740 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7ba0ad0 0x55a0f7baed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f18eaffd640 0 client.4446 ms_handle_reset on v2:172.21.15.150:6833/2160557452 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f191525f740 1 --2- 172.21.15.139:0/1532991868 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b83230 0x55a0f7b7f940 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f191525f740 1 -- 172.21.15.139:0/1532991868 >> 172.21.15.139:0/1532991868 conn(0x55a0f7b58f10 msgr2=0x55a0f7b822b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f191525f740 1 -- 172.21.15.139:0/1532991868 shutdown_connections 2024-04-25T09:53:22.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f191525f740 1 -- 172.21.15.139:0/1532991868 wait complete. 2024-04-25T09:53:22.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.835+0000 7f191525f740 20 client.4446 trim_cache size 0 max 16384 2024-04-25T09:53:22.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.850+0000 7f191525f740 1 Processor -- start 2024-04-25T09:53:22.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.850+0000 7f191525f740 1 -- start start 2024-04-25T09:53:22.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f191525f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b83230 0x55a0f7b9cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f191525f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b96490 0x55a0f7b9d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f191525f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7ba0ad0 0x55a0f7baa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f191525f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c201350 con 0x55a0f7b96490 2024-04-25T09:53:22.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f191525f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c2004f0 con 0x55a0f7b83230 2024-04-25T09:53:22.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f191525f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190c15a760 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f190b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b96490 0x55a0f7b9d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f190b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b96490 0x55a0f7b9d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60048/0 (socket says 172.21.15.139:60048) 2024-04-25T09:53:22.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f190b7fe640 1 -- 172.21.15.139:0/3263394888 learned_addr learned my addr 172.21.15.139:0/3263394888 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:22.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f190b7fe640 1 -- 172.21.15.139:0/3263394888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7ba0ad0 msgr2=0x55a0f7baa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f190b7fe640 1 --2- 172.21.15.139:0/3263394888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7ba0ad0 0x55a0f7baa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f190b7fe640 1 -- 172.21.15.139:0/3263394888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b83230 msgr2=0x55a0f7b9cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f190b7fe640 1 --2- 172.21.15.139:0/3263394888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b83230 0x55a0f7b9cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.851+0000 7f190b7fe640 1 -- 172.21.15.139:0/3263394888 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0f7b82e70 con 0x55a0f7b96490 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f190b7fe640 1 --2- 172.21.15.139:0/3263394888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b96490 0x55a0f7b9d420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1900016fd0 tx=0x7f1900015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb58350aeabce1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f18f17fa640 1 -- 172.21.15.139:0/3263394888 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190001c040 con 0x55a0f7b96490 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f18f17fa640 1 -- 172.21.15.139:0/3263394888 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1900026070 con 0x55a0f7b96490 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f18f17fa640 1 -- 172.21.15.139:0/3263394888 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900005c00 con 0x55a0f7b96490 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f191525f740 1 -- 172.21.15.139:0/3263394888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b96490 msgr2=0x55a0f7b9d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f191525f740 1 --2- 172.21.15.139:0/3263394888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b96490 0x55a0f7b9d420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1900016fd0 tx=0x7f1900015000 comp rx=0 tx=0).stop 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f191525f740 1 -- 172.21.15.139:0/3263394888 shutdown_connections 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f191525f740 1 --2- 172.21.15.139:0/3263394888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7b96490 0x55a0f7b9d420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f191525f740 1 --2- 172.21.15.139:0/3263394888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7ba0ad0 0x55a0f7baa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f191525f740 1 --2- 172.21.15.139:0/3263394888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b83230 0x55a0f7b9cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f191525f740 1 -- 172.21.15.139:0/3263394888 >> 172.21.15.139:0/3263394888 conn(0x55a0f7b58f10 msgr2=0x55a0f7b7fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:22.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f191525f740 1 -- 172.21.15.139:0/3263394888 shutdown_connections 2024-04-25T09:53:22.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.852+0000 7f191525f740 1 -- 172.21.15.139:0/3263394888 wait complete. 2024-04-25T09:53:22.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.853+0000 7f191525f740 1 -- start start 2024-04-25T09:53:22.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.853+0000 7f191525f740 1 Processor -- start 2024-04-25T09:53:22.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f191525f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b83230 0x55a0f7b85bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f191525f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7b96490 0x55a0f7c08940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f191525f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 0x55a0f7b9cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f191525f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f7b84280 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f191525f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c1ffef0 con 0x55a0f7b83230 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f191525f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0f7baf240 con 0x55a0f7b96490 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f191525f740 10 client.0 _ll_register_callbacks cb 0x55a0f7b53c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f1910b07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 0x55a0f7b9cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f1910b07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 0x55a0f7b9cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60062/0 (socket says 172.21.15.139:60062) 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f1910b07640 1 -- 172.21.15.139:0/468988667 learned_addr learned my addr 172.21.15.139:0/468988667 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f190b7fe640 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7b96490 0x55a0f7c08940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f190bfff640 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b83230 0x55a0f7b85bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f1910b07640 1 -- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7b96490 msgr2=0x55a0f7c08940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f1910b07640 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7b96490 0x55a0f7c08940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f1910b07640 1 -- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b83230 msgr2=0x55a0f7b85bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f1910b07640 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b83230 0x55a0f7b85bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f1910b07640 1 -- 172.21.15.139:0/468988667 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0f7b54740 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f190bfff640 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b83230 0x55a0f7b85bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f1910b07640 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 0x55a0f7b9cee0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55a0f7b9e010 tx=0x7f19040126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ff20e56b97031b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f18eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f18eaffd640 1 -- 172.21.15.139:0/468988667 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904006d60 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f18eaffd640 1 -- 172.21.15.139:0/468988667 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1904016480 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f191525f740 1 -- 172.21.15.139:0/468988667 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1900026070 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.854+0000 7f18eaffd640 1 -- 172.21.15.139:0/468988667 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1904022050 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f191525f740 10 client.4452 fetch_fsmap learned FSMap version 7 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f191525f740 1 -- 172.21.15.139:0/468988667 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55a0f7c08e80 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f18eaffd640 1 -- 172.21.15.139:0/468988667 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1900026070 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f18eaffd640 1 -- 172.21.15.139:0/468988667 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f1904025020 con 0x55a0f7ba0ad0 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f191525f740 10 client.4452 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f191525f740 10 client.4452 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:53:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f191525f740 20 client.4452 populate_metadata read hostname 'smithi139' 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f191525f740 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x55a0f7c0d130 0x55a0f7b524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f191525f740 4 client.4452 mds_command: new command op to 4299 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f191525f740 1 -- 172.21.15.139:0/468988667 --> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a0f7ac8f50 con 0x55a0f7c0d130 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f190bfff640 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x55a0f7c0d130 0x55a0f7b524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f18eaffd640 10 client.4452 ms_handle_connect on v2:172.21.15.150:6833/2160557452 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.855+0000 7f190bfff640 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x55a0f7c0d130 0x55a0f7b524a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fea8c3f7f95a68 server_cookie=ff4bd1c618470ee3 in_seq=0 out_seq=0 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f18eaffd640 1 -- 172.21.15.139:0/468988667 <== mds.0 v2:172.21.15.150:6833/2160557452 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+317 (crc 0 0 0) 0x55a0f7ac8f50 con 0x55a0f7c0d130 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f18eaffd640 10 client.4452 handle_command_reply: tid=0 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191525f740 1 client.4452 shutdown 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191525f740 1 -- 172.21.15.139:0/468988667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 msgr2=0x55a0f7b9cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191525f740 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 0x55a0f7b9cee0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55a0f7b9e010 tx=0x7f19040126f0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191525f740 1 -- 172.21.15.139:0/468988667 shutdown_connections 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191525f740 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x55a0f7c0d130 0x55a0f7b524a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191525f740 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0f7ba0ad0 0x55a0f7b9cee0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f18eaffd640 0 client.4452 ms_handle_reset on v2:172.21.15.150:6833/2160557452 2024-04-25T09:53:22.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191525f740 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0f7b96490 0x55a0f7c08940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191525f740 1 --2- 172.21.15.139:0/468988667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0f7b83230 0x55a0f7b85bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191525f740 1 -- 172.21.15.139:0/468988667 >> 172.21.15.139:0/468988667 conn(0x55a0f7b58f10 msgr2=0x55a0f7b82270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:22.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191525f740 1 -- 172.21.15.139:0/468988667 shutdown_connections 2024-04-25T09:53:22.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191525f740 1 -- 172.21.15.139:0/468988667 wait complete. 2024-04-25T09:53:22.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191525f740 20 client.4452 trim_cache size 0 max 16384 2024-04-25T09:53:22.865 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:53:22.865 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T09:53:22.865 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T09:53:22.865 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 4299, 2024-04-25T09:53:22.865 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T09:53:22.866 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T09:53:22.866 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T09:53:22.866 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 7.152669339, 2024-04-25T09:53:22.866 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 7, 2024-04-25T09:53:22.866 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 24, 2024-04-25T09:53:22.866 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T09:53:22.866 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 14.311707927 2024-04-25T09:53:22.866 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:53:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191256a640 1 -- 172.21.15.139:0/698171336 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f18f806b4d0 msgr2=0x7f18f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191256a640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f18f806b4d0 0x7f18f806d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f190c0b5d60 tx=0x7f1900008040 comp rx=0 tx=0).stop 2024-04-25T09:53:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191256a640 1 -- 172.21.15.139:0/698171336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f190c0b9370 msgr2=0x7f190c0abb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191256a640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f190c0b9370 0x7f190c0abb60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009870 tx=0x7f18fc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191256a640 1 -- 172.21.15.139:0/698171336 shutdown_connections 2024-04-25T09:53:22.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191256a640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f18f806b4d0 0x7f18f806d990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191256a640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f190c15dbf0 0x7f190c14ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191256a640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f190c0cec80 0x7f190c0b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191256a640 1 --2- 172.21.15.139:0/698171336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f190c0b9370 0x7f190c0abb60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.856+0000 7f191256a640 1 -- 172.21.15.139:0/698171336 >> 172.21.15.139:0/698171336 conn(0x7f190c0b5480 msgr2=0x7f190c0d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:22.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.857+0000 7f191256a640 1 -- 172.21.15.139:0/698171336 shutdown_connections 2024-04-25T09:53:22.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.857+0000 7f191256a640 1 -- 172.21.15.139:0/698171336 wait complete. 2024-04-25T09:53:22.869 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:53:22.869 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:22.870 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:53:22.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.963+0000 7f6873fee640 1 Processor -- start 2024-04-25T09:53:22.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.964+0000 7f6873fee640 1 -- start start 2024-04-25T09:53:22.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.964+0000 7f6873fee640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f686c15aa00 0x7f686c0b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.964+0000 7f6873fee640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f686c072280 0x7f686c072730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.964+0000 7f6873fee640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686c15e500 con 0x7f686c0b5110 2024-04-25T09:53:22.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.964+0000 7f6873fee640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686c0b5c20 con 0x7f686c072280 2024-04-25T09:53:22.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.964+0000 7f6873fee640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686c0b45d0 con 0x7f686c15aa00 2024-04-25T09:53:22.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.965+0000 7f687258b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f686c072280 0x7f686c072730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.965+0000 7f687258b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f686c072280 0x7f686c072730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60070/0 (socket says 172.21.15.139:60070) 2024-04-25T09:53:22.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.965+0000 7f687258b640 1 -- 172.21.15.139:0/1911427157 learned_addr learned my addr 172.21.15.139:0/1911427157 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:22.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.965+0000 7f6871d8a640 1 --2- 172.21.15.139:0/1911427157 >> v2:172.21.15.150:3300/0 conn(0x7f686c15aa00 0x7f686c0b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.965+0000 7f687258b640 1 -- 172.21.15.139:0/1911427157 >> v2:172.21.15.150:3300/0 conn(0x7f686c15aa00 msgr2=0x7f686c0b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.965+0000 7f687258b640 1 --2- 172.21.15.139:0/1911427157 >> v2:172.21.15.150:3300/0 conn(0x7f686c15aa00 0x7f686c0b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.965+0000 7f687258b640 1 -- 172.21.15.139:0/1911427157 >> v1:172.21.15.150:6789/0 conn(0x7f686c0b5110 legacy=0x7f686c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.965+0000 7f687258b640 1 -- 172.21.15.139:0/1911427157 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c0b5660 con 0x7f686c072280 2024-04-25T09:53:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.965+0000 7f687258b640 1 --2- 172.21.15.139:0/1911427157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f686c072280 0x7f686c072730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f686400d150 tx=0x7f686402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a08dd372fc4cbe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.966+0000 7f6870d88640 1 -- 172.21.15.139:0/1911427157 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864039070 con 0x7f686c072280 2024-04-25T09:53:22.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.966+0000 7f6870d88640 1 -- 172.21.15.139:0/1911427157 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6864030050 con 0x7f686c072280 2024-04-25T09:53:22.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.966+0000 7f6870d88640 1 -- 172.21.15.139:0/1911427157 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864009820 con 0x7f686c072280 2024-04-25T09:53:22.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.966+0000 7f6873fee640 1 -- 172.21.15.139:0/1911427157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f686c072280 msgr2=0x7f686c072730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.966+0000 7f6873fee640 1 --2- 172.21.15.139:0/1911427157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f686c072280 0x7f686c072730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f686400d150 tx=0x7f686402c890 comp rx=0 tx=0).stop 2024-04-25T09:53:22.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.966+0000 7f6873fee640 1 -- 172.21.15.139:0/1911427157 shutdown_connections 2024-04-25T09:53:22.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.966+0000 7f6873fee640 1 --2- 172.21.15.139:0/1911427157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f686c072280 0x7f686c072730 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.966+0000 7f6873fee640 1 --2- 172.21.15.139:0/1911427157 >> v2:172.21.15.150:3300/0 conn(0x7f686c15aa00 0x7f686c0b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.966+0000 7f6873fee640 1 -- 172.21.15.139:0/1911427157 >> 172.21.15.139:0/1911427157 conn(0x7f686c06e280 msgr2=0x7f686c06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:22.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.966+0000 7f6873fee640 1 -- 172.21.15.139:0/1911427157 shutdown_connections 2024-04-25T09:53:22.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.966+0000 7f6873fee640 1 -- 172.21.15.139:0/1911427157 wait complete. 2024-04-25T09:53:22.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.967+0000 7f6873fee640 1 Processor -- start 2024-04-25T09:53:22.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.967+0000 7f6873fee640 1 -- start start 2024-04-25T09:53:22.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.968+0000 7f6873fee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f686c072280 0x7f686c0cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.968+0000 7f6873fee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f686c0b5110 0x7f686c157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.968+0000 7f6873fee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f686c157b90 0x7f686c15d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.968+0000 7f6873fee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0ce610 con 0x7f686c072280 2024-04-25T09:53:22.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.968+0000 7f6873fee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c15e7a0 con 0x7f686c157b90 2024-04-25T09:53:22.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.968+0000 7f6873fee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0ce790 con 0x7f686c0b5110 2024-04-25T09:53:22.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.968+0000 7f6871d8a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f686c072280 0x7f686c0cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.968+0000 7f6871d8a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f686c072280 0x7f686c0cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60084/0 (socket says 172.21.15.139:60084) 2024-04-25T09:53:22.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.968+0000 7f6871d8a640 1 -- 172.21.15.139:0/2756702065 learned_addr learned my addr 172.21.15.139:0/2756702065 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:22.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.968+0000 7f687258b640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f686c157b90 0x7f686c15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.968+0000 7f6871589640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f686c0b5110 0x7f686c157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.969+0000 7f6871d8a640 1 -- 172.21.15.139:0/2756702065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f686c0b5110 msgr2=0x7f686c157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.969+0000 7f6871d8a640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f686c0b5110 0x7f686c157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.969+0000 7f6871d8a640 1 -- 172.21.15.139:0/2756702065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f686c157b90 msgr2=0x7f686c15d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:22.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.969+0000 7f6871d8a640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f686c157b90 0x7f686c15d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:22.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.969+0000 7f6871d8a640 1 -- 172.21.15.139:0/2756702065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c0cc540 con 0x7f686c072280 2024-04-25T09:53:22.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.969+0000 7f6871d8a640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f686c072280 0x7f686c0cc000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f686800c820 tx=0x7f686800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.969+0000 7f68627fc640 1 -- 172.21.15.139:0/2756702065 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868015070 con 0x7f686c072280 2024-04-25T09:53:22.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.969+0000 7f68627fc640 1 -- 172.21.15.139:0/2756702065 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6868004020 con 0x7f686c072280 2024-04-25T09:53:22.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.969+0000 7f6873fee640 1 -- 172.21.15.139:0/2756702065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c158090 con 0x7f686c072280 2024-04-25T09:53:22.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.969+0000 7f68627fc640 1 -- 172.21.15.139:0/2756702065 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68680043f0 con 0x7f686c072280 2024-04-25T09:53:22.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.969+0000 7f6873fee640 1 -- 172.21.15.139:0/2756702065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c158650 con 0x7f686c072280 2024-04-25T09:53:22.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.970+0000 7f6873fee640 1 -- 172.21.15.139:0/2756702065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686c06e990 con 0x7f686c072280 2024-04-25T09:53:22.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.970+0000 7f68627fc640 1 -- 172.21.15.139:0/2756702065 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6868007830 con 0x7f686c072280 2024-04-25T09:53:22.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.970+0000 7f68627fc640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f685006b4d0 0x7f685006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:22.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.971+0000 7f68627fc640 1 -- 172.21.15.139:0/2756702065 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f686808a1c0 con 0x7f686c072280 2024-04-25T09:53:22.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.971+0000 7f6871589640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f685006b4d0 0x7f685006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:22.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.971+0000 7f6871589640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f685006b4d0 0x7f685006d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f686c1f4110 tx=0x7f685c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:22.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:22.975+0000 7f68627fc640 1 -- 172.21.15.139:0/2756702065 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f686c06e990 con 0x7f686c072280 2024-04-25T09:53:23.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.182+0000 7f6873fee640 1 -- 172.21.15.139:0/2756702065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f686c074520 con 0x7f686c072280 2024-04-25T09:53:23.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.182+0000 7f68627fc640 1 -- 172.21.15.139:0/2756702065 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f6868058740 con 0x7f686c072280 2024-04-25T09:53:23.184 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:53:23.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.184+0000 7f6873fee640 1 -- 172.21.15.139:0/2756702065 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f685006b4d0 msgr2=0x7f685006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:23.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.184+0000 7f6873fee640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f685006b4d0 0x7f685006d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f686c1f4110 tx=0x7f685c009040 comp rx=0 tx=0).stop 2024-04-25T09:53:23.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.184+0000 7f6873fee640 1 -- 172.21.15.139:0/2756702065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f686c072280 msgr2=0x7f686c0cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:23.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.184+0000 7f6873fee640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f686c072280 0x7f686c0cc000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f686800c820 tx=0x7f686800cce0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.184+0000 7f6873fee640 1 -- 172.21.15.139:0/2756702065 shutdown_connections 2024-04-25T09:53:23.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.184+0000 7f6873fee640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f685006b4d0 0x7f685006d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.184+0000 7f6873fee640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f686c157b90 0x7f686c15d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.184+0000 7f6873fee640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f686c0b5110 0x7f686c157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.184+0000 7f6873fee640 1 --2- 172.21.15.139:0/2756702065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f686c072280 0x7f686c0cc000 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:23.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.184+0000 7f6873fee640 1 -- 172.21.15.139:0/2756702065 >> 172.21.15.139:0/2756702065 conn(0x7f686c06e280 msgr2=0x7f686c0b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:23.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.184+0000 7f6873fee640 1 -- 172.21.15.139:0/2756702065 shutdown_connections 2024-04-25T09:53:23.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:23.184+0000 7f6873fee640 1 -- 172.21.15.139:0/2756702065 wait complete. 2024-04-25T09:53:23.196 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:53:23.196 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:53:23.196 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T09:53:23.197 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T09:53:23.197 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:23.197 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:53:23.197 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:53:23.197 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:53:23.197 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T09:53:23.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:23.197 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T09:53:23.212 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:23.213 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:53:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:53:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42695sec preferred_lft 42695sec 2024-04-25T09:53:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T09:53:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:23.214 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T09:53:23.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:23.215 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:23.215 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:23.215 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add name ceph-brx type bridge 2024-04-25T09:53:23.215 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr flush dev ceph-brx 2024-04-25T09:53:23.215 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set ceph-brx up 2024-04-25T09:53:23.215 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T09:53:23.215 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:23.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:23.348 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:53:23.372 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:53:23.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:23.374 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T09:53:23.437 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T09:53:23.438 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:53:23.438 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:53:23.438 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:53:23.438 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:53:23.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:23.438 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:23.438 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:23.438 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:53:23.439 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:53:23.439 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:53:23.439 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:23.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:23.660 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:53:23.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:23.675 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list-id 2024-04-25T09:53:23.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:23.730 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:23.730 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:23.730 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:53:23.730 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T09:53:23.730 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:23.806 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T09:53:23.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:23.806 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:23.806 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:23.806 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T09:53:23.806 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T09:53:23.807 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T09:53:23.807 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T09:53:23.807 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T09:53:23.807 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:23.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:23.932 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:23.932 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:23.932 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T09:53:23.932 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T09:53:23.932 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:23.967 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T09:53:23.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:23.968 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:24.024 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T09:53:24.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:24.024 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:24.078 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-04-25T09:53:24.143 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:53:24.199 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-04-25T09:53:24.199 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-04-25T09:53:24.199 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-04-25T09:53:24.199 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-04-25T09:53:24.199 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-04-25T09:53:24.199 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/selinux 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:53:24.200 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:53:24.201 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:53:24.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:24.202 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-04-25T09:53:24.255 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:53:24.255 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T09:53:24.297 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-04-25T09:53:24.320 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:53:24.375 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-04-25T09:53:24.375 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-04-25T09:53:24.375 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-04-25T09:53:24.375 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-04-25T09:53:24.375 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-04-25T09:53:24.375 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-04-25T09:53:24.375 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-04-25T09:53:24.375 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-04-25T09:53:24.375 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/selinux 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:53:24.376 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:53:24.377 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:53:24.377 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-04-25T09:53:24.377 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-04-25T09:53:24.377 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:53:24.377 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:53:24.377 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:53:24.377 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:53:24.377 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-04-25T09:53:24.377 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:53:24.377 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:53:24.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:24.378 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-04-25T09:53:24.386 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi139.stderr:2024-04-25T09:53:24.384+0000 7fb57e2ec480 -1 init, newargv = 0x5566decba020 newargc=15 2024-04-25T09:53:24.387 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi139.stderr:ceph-fuse[41688]: starting ceph client 2024-04-25T09:53:24.392 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi139.stderr:ceph-fuse[41688]: starting fuse 2024-04-25T09:53:24.402 INFO:teuthology.orchestra.run.smithi139.stdout:43 2024-04-25T09:53:24.402 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T09:53:24.402 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:53:24.402 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-04-25T09:53:24.402 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-04-25T09:53:24.403 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T09:53:24.403 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-04-25T09:53:24.403 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:53:24.403 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-04-25T09:53:24.403 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T09:53:24.403 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-04-25T09:53:24.403 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T09:53:24.403 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:53:24.403 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-04-25T09:53:24.403 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.admin")) 2024-04-25T09:53:24.404 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T09:53:24.519 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.admin.41688.asok 2024-04-25T09:53:24.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:53:24.530 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41688.asok 2024-04-25T09:53:24.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:24.530 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41688.asok status 2024-04-25T09:53:24.662 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "admin", 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "41688", 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 4464, 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-04-25T09:53:24.663 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 4464 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 203265368 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 203265368 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:53:24.664 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.4464 192.168.0.1:0/203265368", 2024-04-25T09:53:24.665 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/203265368", 2024-04-25T09:53:24.665 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-04-25T09:53:24.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 7, 2024-04-25T09:53:24.665 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 24, 2024-04-25T09:53:24.665 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:53:24.665 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-04-25T09:53:24.665 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-04-25T09:53:24.665 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:53:24.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:24.670 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-04-25T09:53:24.670 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:24.686 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-04-25T09:53:24.686 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:24.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:24.687 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:24.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:24.751 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T09:53:24.820 INFO:teuthology.orchestra.run.smithi139.stdout:# file: . 2024-04-25T09:53:24.820 INFO:teuthology.orchestra.run.smithi139.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T09:53:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:24.821 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:53:24.821 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:24.877 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:25.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.013+0000 7f0085d89640 1 Processor -- start 2024-04-25T09:53:25.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.013+0000 7f0085d89640 1 -- start start 2024-04-25T09:53:25.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f0085d89640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f008015efb0 0x7f00800d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f0085d89640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f00800cf900 0x7f008006c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f0085d89640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f008006cd80 0x7f0080099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f0085d89640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00800d0a80 con 0x7f00800cf900 2024-04-25T09:53:25.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f0085d89640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00800cfe70 con 0x7f008015efb0 2024-04-25T09:53:25.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f0085d89640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00800d0150 con 0x7f008006cd80 2024-04-25T09:53:25.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f007effd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f00800cf900 0x7f008006c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f007effd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f00800cf900 0x7f008006c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60094/0 (socket says 172.21.15.139:60094) 2024-04-25T09:53:25.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f007effd640 1 -- 172.21.15.139:0/3364081519 learned_addr learned my addr 172.21.15.139:0/3364081519 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:25.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f007ffff640 1 --2- 172.21.15.139:0/3364081519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f008006cd80 0x7f0080099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f007f7fe640 1 --2- 172.21.15.139:0/3364081519 >> v2:172.21.15.150:3300/0 conn(0x7f008015efb0 0x7f00800d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f007effd640 1 -- 172.21.15.139:0/3364081519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f008006cd80 msgr2=0x7f0080099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.014+0000 7f007effd640 1 --2- 172.21.15.139:0/3364081519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f008006cd80 0x7f0080099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.015+0000 7f007effd640 1 -- 172.21.15.139:0/3364081519 >> v2:172.21.15.150:3300/0 conn(0x7f008015efb0 msgr2=0x7f00800d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.015+0000 7f007effd640 1 --2- 172.21.15.139:0/3364081519 >> v2:172.21.15.150:3300/0 conn(0x7f008015efb0 0x7f00800d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.015+0000 7f007effd640 1 -- 172.21.15.139:0/3364081519 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00800d0430 con 0x7f00800cf900 2024-04-25T09:53:25.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.015+0000 7f007ffff640 1 --2- 172.21.15.139:0/3364081519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f008006cd80 0x7f0080099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.015+0000 7f007f7fe640 1 --2- 172.21.15.139:0/3364081519 >> v2:172.21.15.150:3300/0 conn(0x7f008015efb0 0x7f00800d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.015+0000 7f007effd640 1 --2- 172.21.15.139:0/3364081519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00800cf900 0x7f008006c840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0080072c10 tx=0x7f007402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ed2b93bdde1d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.015+0000 7f007e7fc640 1 -- 172.21.15.139:0/3364081519 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074037070 con 0x7f00800cf900 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.015+0000 7f007e7fc640 1 -- 172.21.15.139:0/3364081519 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0074032440 con 0x7f00800cf900 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.015+0000 7f007e7fc640 1 -- 172.21.15.139:0/3364081519 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074032810 con 0x7f00800cf900 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.016+0000 7f0085d89640 1 -- 172.21.15.139:0/3364081519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00800cf900 msgr2=0x7f008006c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.016+0000 7f0085d89640 1 --2- 172.21.15.139:0/3364081519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00800cf900 0x7f008006c840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0080072c10 tx=0x7f007402a140 comp rx=0 tx=0).stop 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.016+0000 7f0085d89640 1 -- 172.21.15.139:0/3364081519 shutdown_connections 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.016+0000 7f0085d89640 1 --2- 172.21.15.139:0/3364081519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f008006cd80 0x7f0080099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.016+0000 7f0085d89640 1 --2- 172.21.15.139:0/3364081519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00800cf900 0x7f008006c840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.016+0000 7f0085d89640 1 --2- 172.21.15.139:0/3364081519 >> v2:172.21.15.150:3300/0 conn(0x7f008015efb0 0x7f00800d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.016+0000 7f0085d89640 1 -- 172.21.15.139:0/3364081519 >> 172.21.15.139:0/3364081519 conn(0x7f00800d1790 msgr2=0x7f00800d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:25.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.016+0000 7f0085d89640 1 -- 172.21.15.139:0/3364081519 shutdown_connections 2024-04-25T09:53:25.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.016+0000 7f0085d89640 1 -- 172.21.15.139:0/3364081519 wait complete. 2024-04-25T09:53:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.017+0000 7f0085d89640 1 Processor -- start 2024-04-25T09:53:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.017+0000 7f0085d89640 1 -- start start 2024-04-25T09:53:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f0085d89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f008006cd80 0x7f00801f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f0085d89640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00800cf900 0x7f00800b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f0085d89640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f008015efb0 0x7f00800c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f0085d89640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008009a110 con 0x7f008006cd80 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f0085d89640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0080099f90 con 0x7f008015efb0 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f0085d89640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008009a290 con 0x7f00800cf900 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f007f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f008006cd80 0x7f00801f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f007f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f008006cd80 0x7f00801f53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60106/0 (socket says 172.21.15.139:60106) 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f007f7fe640 1 -- 172.21.15.139:0/2216053894 learned_addr learned my addr 172.21.15.139:0/2216053894 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f007effd640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00800cf900 0x7f00800b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f007ffff640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f008015efb0 0x7f00800c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f007f7fe640 1 -- 172.21.15.139:0/2216053894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00800cf900 msgr2=0x7f00800b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f007f7fe640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00800cf900 0x7f00800b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f007f7fe640 1 -- 172.21.15.139:0/2216053894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f008015efb0 msgr2=0x7f00800c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f007f7fe640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f008015efb0 0x7f00800c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f007f7fe640 1 -- 172.21.15.139:0/2216053894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00801f5900 con 0x7f008006cd80 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f007effd640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00800cf900 0x7f00800b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.018+0000 7f007f7fe640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f008006cd80 0x7f00801f53c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f007000c7f0 tx=0x7f007000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.019+0000 7f0067fff640 1 -- 172.21.15.139:0/2216053894 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070015070 con 0x7f008006cd80 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.019+0000 7f0067fff640 1 -- 172.21.15.139:0/2216053894 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0070007820 con 0x7f008006cd80 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.019+0000 7f0085d89640 1 -- 172.21.15.139:0/2216053894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00800b4950 con 0x7f008006cd80 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.019+0000 7f0067fff640 1 -- 172.21.15.139:0/2216053894 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070007bd0 con 0x7f008006cd80 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.019+0000 7f0085d89640 1 -- 172.21.15.139:0/2216053894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00800b4f10 con 0x7f008006cd80 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.020+0000 7f0085d89640 1 -- 172.21.15.139:0/2216053894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00801f4270 con 0x7f008006cd80 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.020+0000 7f0067fff640 1 -- 172.21.15.139:0/2216053894 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0070018070 con 0x7f008006cd80 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.020+0000 7f0067fff640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f006c06b4d0 0x7f006c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.021+0000 7f0067fff640 1 -- 172.21.15.139:0/2216053894 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f007008a5e0 con 0x7f008006cd80 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.021+0000 7f007effd640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f006c06b4d0 0x7f006c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.021+0000 7f007effd640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f006c06b4d0 0x7f006c06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0074034000 tx=0x7f007402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.025+0000 7f0067fff640 1 -- 172.21.15.139:0/2216053894 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00700591b0 con 0x7f008006cd80 2024-04-25T09:53:25.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.197+0000 7f0085d89640 1 -- 172.21.15.139:0/2216053894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0080074520 con 0x7f008006cd80 2024-04-25T09:53:25.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.199+0000 7f0067fff640 1 -- 172.21.15.139:0/2216053894 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f0070058ab0 con 0x7f008006cd80 2024-04-25T09:53:25.201 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:25.201 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:25.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.200+0000 7f0085d89640 1 -- 172.21.15.139:0/2216053894 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f006c06b4d0 msgr2=0x7f006c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.200+0000 7f0085d89640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f006c06b4d0 0x7f006c06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0074034000 tx=0x7f007402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:25.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.200+0000 7f0085d89640 1 -- 172.21.15.139:0/2216053894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f008006cd80 msgr2=0x7f00801f53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.200+0000 7f0085d89640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f008006cd80 0x7f00801f53c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f007000c7f0 tx=0x7f007000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.201+0000 7f0085d89640 1 -- 172.21.15.139:0/2216053894 shutdown_connections 2024-04-25T09:53:25.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.201+0000 7f0085d89640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f006c06b4d0 0x7f006c06d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.201+0000 7f0085d89640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f008015efb0 0x7f00800c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.201+0000 7f0085d89640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00800cf900 0x7f00800b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.201+0000 7f0085d89640 1 --2- 172.21.15.139:0/2216053894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f008006cd80 0x7f00801f53c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.201+0000 7f0085d89640 1 -- 172.21.15.139:0/2216053894 >> 172.21.15.139:0/2216053894 conn(0x7f00800d1790 msgr2=0x7f008015e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:25.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.201+0000 7f0085d89640 1 -- 172.21.15.139:0/2216053894 shutdown_connections 2024-04-25T09:53:25.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.201+0000 7f0085d89640 1 -- 172.21.15.139:0/2216053894 wait complete. 2024-04-25T09:53:25.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:25.214 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.312+0000 7f7c0415a640 1 Processor -- start 2024-04-25T09:53:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.312+0000 7f7c0415a640 1 -- start start 2024-04-25T09:53:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c0415a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7bfc15dbf0 0x7f7bfc1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c0415a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7bfc0b9370 0x7f7bfc0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c0415a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bfc0cec80 0x7f7bfc06c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c0415a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bfc0d1ee0 con 0x7f7bfc15dbf0 2024-04-25T09:53:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c0415a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bfc161dc0 con 0x7f7bfc0b9370 2024-04-25T09:53:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c0415a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc1620a0 con 0x7f7bfc0cec80 2024-04-25T09:53:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c01ef6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7bfc15dbf0 0x7f7bfc1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c01ef6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7bfc15dbf0 0x7f7bfc1617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60112/0 (socket says 172.21.15.139:60112) 2024-04-25T09:53:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c01ef6640 1 -- 172.21.15.139:0/2804066009 learned_addr learned my addr 172.21.15.139:0/2804066009 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c026f7640 1 --2- 172.21.15.139:0/2804066009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bfc0cec80 0x7f7bfc06c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c016f5640 1 --2- 172.21.15.139:0/2804066009 >> v2:172.21.15.150:3300/0 conn(0x7f7bfc0b9370 0x7f7bfc0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c01ef6640 1 -- 172.21.15.139:0/2804066009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bfc0cec80 msgr2=0x7f7bfc06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c01ef6640 1 --2- 172.21.15.139:0/2804066009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bfc0cec80 0x7f7bfc06c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c01ef6640 1 -- 172.21.15.139:0/2804066009 >> v2:172.21.15.150:3300/0 conn(0x7f7bfc0b9370 msgr2=0x7f7bfc0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c01ef6640 1 --2- 172.21.15.139:0/2804066009 >> v2:172.21.15.150:3300/0 conn(0x7f7bfc0b9370 0x7f7bfc0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.313+0000 7f7c01ef6640 1 -- 172.21.15.139:0/2804066009 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc162380 con 0x7f7bfc15dbf0 2024-04-25T09:53:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.314+0000 7f7c01ef6640 1 --2- 172.21.15.139:0/2804066009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bfc15dbf0 0x7f7bfc1617e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc08c420 tx=0x7f7bf802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcf163a423d1d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.314+0000 7f7c00ef4640 1 -- 172.21.15.139:0/2804066009 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8037070 con 0x7f7bfc15dbf0 2024-04-25T09:53:25.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.314+0000 7f7c00ef4640 1 -- 172.21.15.139:0/2804066009 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf8032440 con 0x7f7bfc15dbf0 2024-04-25T09:53:25.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.314+0000 7f7c00ef4640 1 -- 172.21.15.139:0/2804066009 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8032810 con 0x7f7bfc15dbf0 2024-04-25T09:53:25.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.314+0000 7f7c0415a640 1 -- 172.21.15.139:0/2804066009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bfc15dbf0 msgr2=0x7f7bfc1617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.314+0000 7f7c0415a640 1 --2- 172.21.15.139:0/2804066009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bfc15dbf0 0x7f7bfc1617e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc08c420 tx=0x7f7bf802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.314+0000 7f7c0415a640 1 -- 172.21.15.139:0/2804066009 shutdown_connections 2024-04-25T09:53:25.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.314+0000 7f7c0415a640 1 --2- 172.21.15.139:0/2804066009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bfc0cec80 0x7f7bfc06c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.314+0000 7f7c0415a640 1 --2- 172.21.15.139:0/2804066009 >> v2:172.21.15.150:3300/0 conn(0x7f7bfc0b9370 0x7f7bfc0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.314+0000 7f7c0415a640 1 --2- 172.21.15.139:0/2804066009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bfc15dbf0 0x7f7bfc1617e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.314+0000 7f7c0415a640 1 -- 172.21.15.139:0/2804066009 >> 172.21.15.139:0/2804066009 conn(0x7f7bfc0b5480 msgr2=0x7f7bfc0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:25.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.315+0000 7f7c0415a640 1 -- 172.21.15.139:0/2804066009 shutdown_connections 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.315+0000 7f7c0415a640 1 -- 172.21.15.139:0/2804066009 wait complete. 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.316+0000 7f7c0415a640 1 Processor -- start 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.316+0000 7f7c0415a640 1 -- start start 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.316+0000 7f7c0415a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bfc0b9370 0x7f7bfc1f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.316+0000 7f7c0415a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bfc0cec80 0x7f7bfc0ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.316+0000 7f7c0415a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bfc15dbf0 0x7f7bfc157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.316+0000 7f7c0415a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc15dfd0 con 0x7f7bfc0cec80 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.316+0000 7f7c0415a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc0b97d0 con 0x7f7bfc0b9370 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.316+0000 7f7c0415a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc0aa610 con 0x7f7bfc15dbf0 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.316+0000 7f7c016f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bfc0cec80 0x7f7bfc0ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7c016f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bfc0cec80 0x7f7bfc0ab490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60116/0 (socket says 172.21.15.139:60116) 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7c016f5640 1 -- 172.21.15.139:0/4089192174 learned_addr learned my addr 172.21.15.139:0/4089192174 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7c026f7640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bfc15dbf0 0x7f7bfc157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7c01ef6640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bfc0b9370 0x7f7bfc1f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7c016f5640 1 -- 172.21.15.139:0/4089192174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bfc15dbf0 msgr2=0x7f7bfc157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7c016f5640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bfc15dbf0 0x7f7bfc157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7c016f5640 1 -- 172.21.15.139:0/4089192174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bfc0b9370 msgr2=0x7f7bfc1f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7c016f5640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bfc0b9370 0x7f7bfc1f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7c016f5640 1 -- 172.21.15.139:0/4089192174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc1f5c50 con 0x7f7bfc0cec80 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7c016f5640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bfc0cec80 0x7f7bfc0ab490 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00e8a0 tx=0x7f7bec00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7bf27fc640 1 -- 172.21.15.139:0/4089192174 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec017070 con 0x7f7bfc0cec80 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7bf27fc640 1 -- 172.21.15.139:0/4089192174 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bec00ee90 con 0x7f7bfc0cec80 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7c0415a640 1 -- 172.21.15.139:0/4089192174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc1f5f30 con 0x7f7bfc0cec80 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7bf27fc640 1 -- 172.21.15.139:0/4089192174 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec00cb80 con 0x7f7bfc0cec80 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.317+0000 7f7c0415a640 1 -- 172.21.15.139:0/4089192174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc1f64f0 con 0x7f7bfc0cec80 2024-04-25T09:53:25.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.318+0000 7f7c0415a640 1 -- 172.21.15.139:0/4089192174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bfc074520 con 0x7f7bfc0cec80 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.319+0000 7f7bf27fc640 1 -- 172.21.15.139:0/4089192174 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7bec01a070 con 0x7f7bfc0cec80 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.319+0000 7f7bf27fc640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7be006b4d0 0x7f7be006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.319+0000 7f7bf27fc640 1 -- 172.21.15.139:0/4089192174 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7bec08a650 con 0x7f7bfc0cec80 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.319+0000 7f7c01ef6640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7be006b4d0 0x7f7be006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.319+0000 7f7c01ef6640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7be006b4d0 0x7f7be006d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8034000 tx=0x7f7bf802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.323+0000 7f7bf27fc640 1 -- 172.21.15.139:0/4089192174 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7bec058b40 con 0x7f7bfc0cec80 2024-04-25T09:53:25.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.492+0000 7f7c0415a640 1 -- 172.21.15.139:0/4089192174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bfc0d1680 con 0x7f7bfc0cec80 2024-04-25T09:53:25.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.493+0000 7f7bf27fc640 1 -- 172.21.15.139:0/4089192174 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bec05cbe0 con 0x7f7bfc0cec80 2024-04-25T09:53:25.496 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:53:25.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.494+0000 7f7c0415a640 1 -- 172.21.15.139:0/4089192174 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7be006b4d0 msgr2=0x7f7be006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.494+0000 7f7c0415a640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7be006b4d0 0x7f7be006d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8034000 tx=0x7f7bf802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:25.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.494+0000 7f7c0415a640 1 -- 172.21.15.139:0/4089192174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bfc0cec80 msgr2=0x7f7bfc0ab490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.494+0000 7f7c0415a640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bfc0cec80 0x7f7bfc0ab490 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00e8a0 tx=0x7f7bec00ed60 comp rx=0 tx=0).stop 2024-04-25T09:53:25.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.494+0000 7f7c0415a640 1 -- 172.21.15.139:0/4089192174 shutdown_connections 2024-04-25T09:53:25.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.494+0000 7f7c0415a640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bfc0cec80 0x7f7bfc0ab490 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.494+0000 7f7c0415a640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7be006b4d0 0x7f7be006d990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.494+0000 7f7c0415a640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bfc15dbf0 0x7f7bfc157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.494+0000 7f7c0415a640 1 --2- 172.21.15.139:0/4089192174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bfc0b9370 0x7f7bfc1f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.494+0000 7f7c0415a640 1 -- 172.21.15.139:0/4089192174 >> 172.21.15.139:0/4089192174 conn(0x7f7bfc0b5480 msgr2=0x7f7bfc0caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:25.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.495+0000 7f7c0415a640 1 -- 172.21.15.139:0/4089192174 shutdown_connections 2024-04-25T09:53:25.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.495+0000 7f7c0415a640 1 -- 172.21.15.139:0/4089192174 wait complete. 2024-04-25T09:53:25.508 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:25.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.606+0000 7f7cc92f5640 1 Processor -- start 2024-04-25T09:53:25.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.606+0000 7f7cc92f5640 1 -- start start 2024-04-25T09:53:25.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.606+0000 7f7cc92f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40b93b0 0x7f7cc40ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.606+0000 7f7cc92f5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7cc40ced30 0x7f7cc40d2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.606+0000 7f7cc92f5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc40d1e60 con 0x7f7cc415dba0 2024-04-25T09:53:25.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.606+0000 7f7cc92f5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc40cf180 con 0x7f7cc40ced30 2024-04-25T09:53:25.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.606+0000 7f7cc92f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40d21b0 con 0x7f7cc40b93b0 2024-04-25T09:53:25.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc2ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f7cc415dba0 0x7f7cc4161800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38780/0 (socket says 172.21.15.139:38780) 2024-04-25T09:53:25.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc2ffd640 1 -- 172.21.15.139:0/775766250 learned_addr learned my addr 172.21.15.139:0/775766250 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:25.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc27fc640 1 --2- 172.21.15.139:0/775766250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40b93b0 0x7f7cc40ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc37fe640 1 --2- 172.21.15.139:0/775766250 >> v2:172.21.15.150:3300/0 conn(0x7f7cc40ced30 0x7f7cc40d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197082279 0 0) 0x7f7cc40d1e60 con 0x7f7cc415dba0 2024-04-25T09:53:25.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cac003520 con 0x7f7cc415dba0 2024-04-25T09:53:25.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1793469692 0 0) 0x7f7cac003520 con 0x7f7cc415dba0 2024-04-25T09:53:25.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40b93b0 msgr2=0x7f7cc40ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 --2- 172.21.15.139:0/775766250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40b93b0 0x7f7cc40ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 >> v2:172.21.15.150:3300/0 conn(0x7f7cc40ced30 msgr2=0x7f7cc40d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 --2- 172.21.15.139:0/775766250 >> v2:172.21.15.150:3300/0 conn(0x7f7cc40ced30 0x7f7cc40d2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc4162050 con 0x7f7cc415dba0 2024-04-25T09:53:25.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f7cb4001670 con 0x7f7cc415dba0 2024-04-25T09:53:25.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 >> v1:172.21.15.139:6789/0 conn(0x7f7cc415dba0 legacy=0x7f7cc4161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 --2- 172.21.15.139:0/775766250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac0039b0 0x7f7cac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 --2- 172.21.15.139:0/775766250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cac004800 0x7f7cac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 --2- 172.21.15.139:0/775766250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40b93b0 0x7f7cac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40cf180 con 0x7f7cac004800 2024-04-25T09:53:25.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40d21b0 con 0x7f7cac0039b0 2024-04-25T09:53:25.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac00b9c0 con 0x7f7cc40b93b0 2024-04-25T09:53:25.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc27fc640 1 --2- 172.21.15.139:0/775766250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cac004800 0x7f7cac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc2ffd640 1 --2- 172.21.15.139:0/775766250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac0039b0 0x7f7cac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.607+0000 7f7cc37fe640 1 --2- 172.21.15.139:0/775766250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40b93b0 0x7f7cac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.608+0000 7f7cc27fc640 1 -- 172.21.15.139:0/775766250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40b93b0 msgr2=0x7f7cac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.608+0000 7f7cc27fc640 1 --2- 172.21.15.139:0/775766250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40b93b0 0x7f7cac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.608+0000 7f7cc27fc640 1 -- 172.21.15.139:0/775766250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac0039b0 msgr2=0x7f7cac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:25.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.608+0000 7f7cc27fc640 1 --2- 172.21.15.139:0/775766250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac0039b0 0x7f7cac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.608+0000 7f7cc27fc640 1 -- 172.21.15.139:0/775766250 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac003520 con 0x7f7cac004800 2024-04-25T09:53:25.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.608+0000 7f7cc27fc640 1 --2- 172.21.15.139:0/775766250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cac004800 0x7f7cac004c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009ac0 tx=0x7f7cac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9d2276620f4cde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.608+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8036070 con 0x7f7cac004800 2024-04-25T09:53:25.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.608+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb8031400 con 0x7f7cac004800 2024-04-25T09:53:25.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.608+0000 7f7cc1ffb640 1 -- 172.21.15.139:0/775766250 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8009690 con 0x7f7cac004800 2024-04-25T09:53:25.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.609+0000 7f7cc92f5640 1 -- 172.21.15.139:0/775766250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cac004800 msgr2=0x7f7cac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.609+0000 7f7cc92f5640 1 --2- 172.21.15.139:0/775766250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cac004800 0x7f7cac004c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009ac0 tx=0x7f7cac005830 comp rx=0 tx=0).stop 2024-04-25T09:53:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.609+0000 7f7cc37fe640 1 -- 172.21.15.139:0/775766250 reap_dead start 2024-04-25T09:53:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.609+0000 7f7cc92f5640 1 -- 172.21.15.139:0/775766250 shutdown_connections 2024-04-25T09:53:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.609+0000 7f7cc92f5640 1 -- 172.21.15.139:0/775766250 >> 172.21.15.139:0/775766250 conn(0x7f7cc40d04a0 msgr2=0x7f7cc40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.609+0000 7f7cc92f5640 1 -- 172.21.15.139:0/775766250 shutdown_connections 2024-04-25T09:53:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.609+0000 7f7cc92f5640 1 -- 172.21.15.139:0/775766250 wait complete. 2024-04-25T09:53:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.610+0000 7f7cc92f5640 1 Processor -- start 2024-04-25T09:53:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.610+0000 7f7cc92f5640 1 -- start start 2024-04-25T09:53:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.611+0000 7f7cc92f5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac004800 0x7f7cc414f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.611+0000 7f7cc92f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40cb500 0x7f7cc40b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.611+0000 7f7cc92f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40cb980 0x7f7cc40b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.611+0000 7f7cc92f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40b6bc0 con 0x7f7cc40cb500 2024-04-25T09:53:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.611+0000 7f7cc92f5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc4162610 con 0x7f7cac004800 2024-04-25T09:53:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.611+0000 7f7cc92f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc415df80 con 0x7f7cc40cb980 2024-04-25T09:53:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.611+0000 7f7cc27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40cb500 0x7f7cc40b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.611+0000 7f7cc27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40cb500 0x7f7cc40b75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60126/0 (socket says 172.21.15.139:60126) 2024-04-25T09:53:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.611+0000 7f7cc27fc640 1 -- 172.21.15.139:0/3748756480 learned_addr learned my addr 172.21.15.139:0/3748756480 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.611+0000 7f7cc2ffd640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac004800 0x7f7cc414f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.611+0000 7f7cc37fe640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40cb980 0x7f7cc40b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.612+0000 7f7cc27fc640 1 -- 172.21.15.139:0/3748756480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40cb980 msgr2=0x7f7cc40b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.612+0000 7f7cc27fc640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40cb980 0x7f7cc40b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.612+0000 7f7cc27fc640 1 -- 172.21.15.139:0/3748756480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac004800 msgr2=0x7f7cc414f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.612+0000 7f7cc27fc640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac004800 0x7f7cc414f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.612+0000 7f7cc27fc640 1 -- 172.21.15.139:0/3748756480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc40cbee0 con 0x7f7cc40cb500 2024-04-25T09:53:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.612+0000 7f7cc27fc640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40cb500 0x7f7cc40b75b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009be0 tx=0x7f7cb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.612+0000 7f7cab7fe640 1 -- 172.21.15.139:0/3748756480 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8041020 con 0x7f7cc40cb500 2024-04-25T09:53:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.612+0000 7f7cab7fe640 1 -- 172.21.15.139:0/3748756480 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb803c470 con 0x7f7cc40cb500 2024-04-25T09:53:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.612+0000 7f7cc92f5640 1 -- 172.21.15.139:0/3748756480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc40cc1c0 con 0x7f7cc40cb500 2024-04-25T09:53:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.612+0000 7f7cab7fe640 1 -- 172.21.15.139:0/3748756480 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8036040 con 0x7f7cc40cb500 2024-04-25T09:53:25.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.612+0000 7f7cc92f5640 1 -- 172.21.15.139:0/3748756480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc40cc780 con 0x7f7cc40cb500 2024-04-25T09:53:25.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.613+0000 7f7cc92f5640 1 -- 172.21.15.139:0/3748756480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc40b6e20 con 0x7f7cc40cb500 2024-04-25T09:53:25.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.613+0000 7f7cab7fe640 1 -- 172.21.15.139:0/3748756480 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7cb8044020 con 0x7f7cc40cb500 2024-04-25T09:53:25.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.614+0000 7f7cab7fe640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7cb006b500 0x7f7cb006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.614+0000 7f7cab7fe640 1 -- 172.21.15.139:0/3748756480 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7cb80aa2a0 con 0x7f7cc40cb500 2024-04-25T09:53:25.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.614+0000 7f7cc2ffd640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7cb006b500 0x7f7cb006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.614+0000 7f7cc2ffd640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7cb006b500 0x7f7cb006d9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4160d80 tx=0x7f7cb4007970 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.618+0000 7f7cab7fe640 1 -- 172.21.15.139:0/3748756480 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7cb8078790 con 0x7f7cc40cb500 2024-04-25T09:53:25.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.790+0000 7f7cc92f5640 1 -- 172.21.15.139:0/3748756480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7cc40d1730 con 0x7f7cc40cb500 2024-04-25T09:53:25.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.790+0000 7f7cab7fe640 1 -- 172.21.15.139:0/3748756480 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7cb807c830 con 0x7f7cc40cb500 2024-04-25T09:53:25.792 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:25.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.792+0000 7f7cc92f5640 1 -- 172.21.15.139:0/3748756480 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7cb006b500 msgr2=0x7f7cb006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.792+0000 7f7cc92f5640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7cb006b500 0x7f7cb006d9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4160d80 tx=0x7f7cb4007970 comp rx=0 tx=0).stop 2024-04-25T09:53:25.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.792+0000 7f7cc92f5640 1 -- 172.21.15.139:0/3748756480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40cb500 msgr2=0x7f7cc40b75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.792+0000 7f7cc92f5640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40cb500 0x7f7cc40b75b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009be0 tx=0x7f7cb8004730 comp rx=0 tx=0).stop 2024-04-25T09:53:25.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.792+0000 7f7cc92f5640 1 -- 172.21.15.139:0/3748756480 shutdown_connections 2024-04-25T09:53:25.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.792+0000 7f7cc92f5640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cc40cb500 0x7f7cc40b75b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.792+0000 7f7cc92f5640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7cb006b500 0x7f7cb006d9c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.792+0000 7f7cc92f5640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40cb980 0x7f7cc40b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.792+0000 7f7cc92f5640 1 --2- 172.21.15.139:0/3748756480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac004800 0x7f7cc414f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.792+0000 7f7cc92f5640 1 -- 172.21.15.139:0/3748756480 >> 172.21.15.139:0/3748756480 conn(0x7f7cc40d04a0 msgr2=0x7f7cc406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:25.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.792+0000 7f7cc92f5640 1 -- 172.21.15.139:0/3748756480 shutdown_connections 2024-04-25T09:53:25.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.792+0000 7f7cc92f5640 1 -- 172.21.15.139:0/3748756480 wait complete. 2024-04-25T09:53:25.806 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:25.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.904+0000 7f6eb89ce640 1 Processor -- start 2024-04-25T09:53:25.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.904+0000 7f6eb89ce640 1 -- start start 2024-04-25T09:53:25.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.905+0000 7f6eb89ce640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6eb415cb20 0x7f6eb40cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.905+0000 7f6eb89ce640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb40d02c0 con 0x7f6eb40afc00 2024-04-25T09:53:25.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.905+0000 7f6eb89ce640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb40ccef0 con 0x7f6eb40cfeb0 2024-04-25T09:53:25.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.905+0000 7f6eb89ce640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb40703c0 con 0x7f6eb415cb20 2024-04-25T09:53:25.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.905+0000 7f6ea9d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6eb415cb20 0x7f6eb40cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.905+0000 7f6eb259c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f6eb40afc00 0x7f6eb406fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38794/0 (socket says 172.21.15.139:38794) 2024-04-25T09:53:25.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.905+0000 7f6eb259c640 1 -- 172.21.15.139:0/2300633486 learned_addr learned my addr 172.21.15.139:0/2300633486 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:25.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.905+0000 7f6ea9d9b640 1 -- 172.21.15.139:0/2300633486 >> v1:172.21.15.150:6789/0 conn(0x7f6eb40cfeb0 legacy=0x7f6eb406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.905+0000 7f6ea9d9b640 1 -- 172.21.15.139:0/2300633486 >> v1:172.21.15.139:6789/0 conn(0x7f6eb40afc00 legacy=0x7f6eb406fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.905+0000 7f6ea9d9b640 1 -- 172.21.15.139:0/2300633486 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb40cd1d0 con 0x7f6eb415cb20 2024-04-25T09:53:25.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.905+0000 7f6eb1d9b640 1 -- 172.21.15.139:0/2300633486 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497174912 0 0) 0x7f6eb40d02c0 con 0x7f6eb40afc00 2024-04-25T09:53:25.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.906+0000 7f6ea9d9b640 1 --2- 172.21.15.139:0/2300633486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6eb415cb20 0x7f6eb40cf970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0097a0 tx=0x7f6e9c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387330a7c332c5c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.906+0000 7f6eb1d9b640 1 -- 172.21.15.139:0/2300633486 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c037070 con 0x7f6eb415cb20 2024-04-25T09:53:25.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.906+0000 7f6eb1d9b640 1 -- 172.21.15.139:0/2300633486 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c032440 con 0x7f6eb415cb20 2024-04-25T09:53:25.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.906+0000 7f6eb1d9b640 1 -- 172.21.15.139:0/2300633486 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c02ae60 con 0x7f6eb415cb20 2024-04-25T09:53:25.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.906+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2300633486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6eb415cb20 msgr2=0x7f6eb40cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.906+0000 7f6eb89ce640 1 --2- 172.21.15.139:0/2300633486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6eb415cb20 0x7f6eb40cf970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0097a0 tx=0x7f6e9c02a110 comp rx=0 tx=0).stop 2024-04-25T09:53:25.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.906+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2300633486 shutdown_connections 2024-04-25T09:53:25.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.906+0000 7f6eb89ce640 1 --2- 172.21.15.139:0/2300633486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6eb415cb20 0x7f6eb40cf970 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.906+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2300633486 >> 172.21.15.139:0/2300633486 conn(0x7f6eb40b2230 msgr2=0x7f6eb40b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:25.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.907+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2300633486 shutdown_connections 2024-04-25T09:53:25.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.907+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2300633486 wait complete. 2024-04-25T09:53:25.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.908+0000 7f6eb89ce640 1 Processor -- start 2024-04-25T09:53:25.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.908+0000 7f6eb89ce640 1 -- start start 2024-04-25T09:53:25.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6eb89ce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb40afc00 0x7f6eb40ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6eb89ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6eb40cfeb0 0x7f6eb4148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6eb89ce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb415cb20 0x7f6eb414b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6eb89ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40affe0 con 0x7f6eb40cfeb0 2024-04-25T09:53:25.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6eb89ce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40cd820 con 0x7f6eb40afc00 2024-04-25T09:53:25.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6eb89ce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40cdcb0 con 0x7f6eb415cb20 2024-04-25T09:53:25.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6ea9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6eb40cfeb0 0x7f6eb4148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6ea9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6eb40cfeb0 0x7f6eb4148ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60138/0 (socket says 172.21.15.139:60138) 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6ea9d9b640 1 -- 172.21.15.139:0/2378622975 learned_addr learned my addr 172.21.15.139:0/2378622975 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6eb259c640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb40afc00 0x7f6eb40ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6eb2d9d640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb415cb20 0x7f6eb414b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6ea9d9b640 1 -- 172.21.15.139:0/2378622975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb415cb20 msgr2=0x7f6eb414b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6ea9d9b640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb415cb20 0x7f6eb414b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6ea9d9b640 1 -- 172.21.15.139:0/2378622975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb40afc00 msgr2=0x7f6eb40ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6ea9d9b640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb40afc00 0x7f6eb40ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6ea9d9b640 1 -- 172.21.15.139:0/2378622975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb40aded0 con 0x7f6eb40cfeb0 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6ea9d9b640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6eb40cfeb0 0x7f6eb4148ce0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c035040 tx=0x7f6e9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6eab7fe640 1 -- 172.21.15.139:0/2378622975 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c040020 con 0x7f6eb40cfeb0 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6eab7fe640 1 -- 172.21.15.139:0/2378622975 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c038040 con 0x7f6eb40cfeb0 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2378622975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb40b3690 con 0x7f6eb40cfeb0 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.909+0000 7f6eab7fe640 1 -- 172.21.15.139:0/2378622975 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c037070 con 0x7f6eb40cfeb0 2024-04-25T09:53:25.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.910+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2378622975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb40ae170 con 0x7f6eb40cfeb0 2024-04-25T09:53:25.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.911+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2378622975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb40b3bb0 con 0x7f6eb40cfeb0 2024-04-25T09:53:25.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.911+0000 7f6eab7fe640 1 -- 172.21.15.139:0/2378622975 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e9c043020 con 0x7f6eb40cfeb0 2024-04-25T09:53:25.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.911+0000 7f6eab7fe640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6e9406b4d0 0x7f6e9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:25.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.912+0000 7f6eb259c640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6e9406b4d0 0x7f6e9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:25.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.912+0000 7f6eab7fe640 1 -- 172.21.15.139:0/2378622975 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6e9c0aaab0 con 0x7f6eb40cfeb0 2024-04-25T09:53:25.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.912+0000 7f6eb259c640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6e9406b4d0 0x7f6e9406d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4007f70 tx=0x7f6ea4007f00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:25.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:25.916+0000 7f6eab7fe640 1 -- 172.21.15.139:0/2378622975 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e9c04a030 con 0x7f6eb40cfeb0 2024-04-25T09:53:26.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.089+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2378622975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6eb4066d50 con 0x7f6eb40cfeb0 2024-04-25T09:53:26.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.089+0000 7f6eab7fe640 1 -- 172.21.15.139:0/2378622975 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e9c032e20 con 0x7f6eb40cfeb0 2024-04-25T09:53:26.091 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:53:26.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.091+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2378622975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6e9406b4d0 msgr2=0x7f6e9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.091+0000 7f6eb89ce640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6e9406b4d0 0x7f6e9406d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4007f70 tx=0x7f6ea4007f00 comp rx=0 tx=0).stop 2024-04-25T09:53:26.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.091+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2378622975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6eb40cfeb0 msgr2=0x7f6eb4148ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.091+0000 7f6eb89ce640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6eb40cfeb0 0x7f6eb4148ce0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c035040 tx=0x7f6e9c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.091+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2378622975 shutdown_connections 2024-04-25T09:53:26.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.091+0000 7f6eb89ce640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6eb40cfeb0 0x7f6eb4148ce0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.091+0000 7f6eb89ce640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6e9406b4d0 0x7f6e9406d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.091+0000 7f6eb89ce640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb415cb20 0x7f6eb414b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.091+0000 7f6eb89ce640 1 --2- 172.21.15.139:0/2378622975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb40afc00 0x7f6eb40ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.091+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2378622975 >> 172.21.15.139:0/2378622975 conn(0x7f6eb40b2230 msgr2=0x7f6eb406ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.091+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2378622975 shutdown_connections 2024-04-25T09:53:26.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.091+0000 7f6eb89ce640 1 -- 172.21.15.139:0/2378622975 wait complete. 2024-04-25T09:53:26.104 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:26.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.202+0000 7f4bcd8b5640 1 Processor -- start 2024-04-25T09:53:26.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.202+0000 7f4bcd8b5640 1 -- start start 2024-04-25T09:53:26.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bcd8b5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4bc81617e0 0x7f4bc80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bcd8b5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4bc80cec80 0x7f4bc80d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bcd8b5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc8161dc0 con 0x7f4bc815dbf0 2024-04-25T09:53:26.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bcd8b5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc80cf0d0 con 0x7f4bc81617e0 2024-04-25T09:53:26.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bcd8b5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc80d1ee0 con 0x7f4bc80cec80 2024-04-25T09:53:26.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bc67fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4bc81617e0 0x7f4bc80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bc67fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4bc81617e0 0x7f4bc80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60142/0 (socket says 172.21.15.139:60142) 2024-04-25T09:53:26.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bc67fc640 1 -- 172.21.15.139:0/385381123 learned_addr learned my addr 172.21.15.139:0/385381123 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:26.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bc77fe640 1 --2- 172.21.15.139:0/385381123 >> v2:172.21.15.150:3300/0 conn(0x7f4bc80cec80 0x7f4bc80d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bc67fc640 1 -- 172.21.15.139:0/385381123 >> v2:172.21.15.150:3300/0 conn(0x7f4bc80cec80 msgr2=0x7f4bc80d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bc67fc640 1 --2- 172.21.15.139:0/385381123 >> v2:172.21.15.150:3300/0 conn(0x7f4bc80cec80 0x7f4bc80d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bc67fc640 1 -- 172.21.15.139:0/385381123 >> v1:172.21.15.150:6789/0 conn(0x7f4bc815dbf0 legacy=0x7f4bc80b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.203+0000 7f4bc67fc640 1 -- 172.21.15.139:0/385381123 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc81620a0 con 0x7f4bc81617e0 2024-04-25T09:53:26.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.204+0000 7f4bc67fc640 1 --2- 172.21.15.139:0/385381123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bc81617e0 0x7f4bc80ce740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4bc808ce00 tx=0x7f4bbc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33202aae3d2459af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.204+0000 7f4bc5ffb640 1 -- 172.21.15.139:0/385381123 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc037070 con 0x7f4bc81617e0 2024-04-25T09:53:26.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.204+0000 7f4bc5ffb640 1 -- 172.21.15.139:0/385381123 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc032440 con 0x7f4bc81617e0 2024-04-25T09:53:26.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.204+0000 7f4bc5ffb640 1 -- 172.21.15.139:0/385381123 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc032810 con 0x7f4bc81617e0 2024-04-25T09:53:26.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.205+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/385381123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bc81617e0 msgr2=0x7f4bc80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.205+0000 7f4bcd8b5640 1 --2- 172.21.15.139:0/385381123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bc81617e0 0x7f4bc80ce740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4bc808ce00 tx=0x7f4bbc02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.205+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/385381123 shutdown_connections 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.205+0000 7f4bcd8b5640 1 --2- 172.21.15.139:0/385381123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bc81617e0 0x7f4bc80ce740 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.205+0000 7f4bcd8b5640 1 --2- 172.21.15.139:0/385381123 >> v2:172.21.15.150:3300/0 conn(0x7f4bc80cec80 0x7f4bc80d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.205+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/385381123 >> 172.21.15.139:0/385381123 conn(0x7f4bc80b5480 msgr2=0x7f4bc80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.205+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/385381123 shutdown_connections 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.205+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/385381123 wait complete. 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.206+0000 7f4bcd8b5640 1 Processor -- start 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.206+0000 7f4bcd8b5640 1 -- start start 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bcd8b5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bc80cec80 0x7f4bc8148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bcd8b5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bc815dbf0 0x7f4bc8153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bcd8b5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc81617e0 0x7f4bc8153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bcd8b5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc815dfd0 con 0x7f4bc815dbf0 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bcd8b5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc8162660 con 0x7f4bc81617e0 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bcd8b5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc815e6f0 con 0x7f4bc80cec80 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bc67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bc815dbf0 0x7f4bc8153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bc67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bc815dbf0 0x7f4bc8153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60150/0 (socket says 172.21.15.139:60150) 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bc67fc640 1 -- 172.21.15.139:0/2364568665 learned_addr learned my addr 172.21.15.139:0/2364568665 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bc6ffd640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bc80cec80 0x7f4bc8148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bc77fe640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc81617e0 0x7f4bc8153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bc67fc640 1 -- 172.21.15.139:0/2364568665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bc80cec80 msgr2=0x7f4bc8148b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bc67fc640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bc80cec80 0x7f4bc8148b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bc67fc640 1 -- 172.21.15.139:0/2364568665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc81617e0 msgr2=0x7f4bc8153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bc67fc640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc81617e0 0x7f4bc8153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bc67fc640 1 -- 172.21.15.139:0/2364568665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8152f10 con 0x7f4bc815dbf0 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.207+0000 7f4bc77fe640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc81617e0 0x7f4bc8153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:26.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.208+0000 7f4bc67fc640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bc815dbf0 0x7f4bc8153640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc000c00 tx=0x7f4bbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.208+0000 7f4baffff640 1 -- 172.21.15.139:0/2364568665 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc037040 con 0x7f4bc815dbf0 2024-04-25T09:53:26.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.208+0000 7f4baffff640 1 -- 172.21.15.139:0/2364568665 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc042070 con 0x7f4bc815dbf0 2024-04-25T09:53:26.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.208+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/2364568665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc81547e0 con 0x7f4bc815dbf0 2024-04-25T09:53:26.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.208+0000 7f4baffff640 1 -- 172.21.15.139:0/2364568665 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc030880 con 0x7f4bc815dbf0 2024-04-25T09:53:26.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.208+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/2364568665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc8154500 con 0x7f4bc815dbf0 2024-04-25T09:53:26.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.209+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/2364568665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc80cf8e0 con 0x7f4bc815dbf0 2024-04-25T09:53:26.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.209+0000 7f4baffff640 1 -- 172.21.15.139:0/2364568665 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bbc030a20 con 0x7f4bc815dbf0 2024-04-25T09:53:26.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.209+0000 7f4baffff640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4bb406b4d0 0x7f4bb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.209+0000 7f4baffff640 1 -- 172.21.15.139:0/2364568665 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4bbc0aae50 con 0x7f4bc815dbf0 2024-04-25T09:53:26.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.209+0000 7f4bc6ffd640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4bb406b4d0 0x7f4bb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.210+0000 7f4bc6ffd640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4bb406b4d0 0x7f4bb406d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8001640 tx=0x7f4bb8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.214+0000 7f4baffff640 1 -- 172.21.15.139:0/2364568665 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4bc80cf8e0 con 0x7f4bc815dbf0 2024-04-25T09:53:26.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.381+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/2364568665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4bc8074520 con 0x7f4bc815dbf0 2024-04-25T09:53:26.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.382+0000 7f4baffff640 1 -- 172.21.15.139:0/2364568665 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4bc8152f10 con 0x7f4bc815dbf0 2024-04-25T09:53:26.384 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:26.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.384+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/2364568665 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4bb406b4d0 msgr2=0x7f4bb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.384+0000 7f4bcd8b5640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4bb406b4d0 0x7f4bb406d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8001640 tx=0x7f4bb8009040 comp rx=0 tx=0).stop 2024-04-25T09:53:26.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.384+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/2364568665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bc815dbf0 msgr2=0x7f4bc8153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.384+0000 7f4bcd8b5640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bc815dbf0 0x7f4bc8153640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc000c00 tx=0x7f4bbc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.384+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/2364568665 shutdown_connections 2024-04-25T09:53:26.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.384+0000 7f4bcd8b5640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4bc815dbf0 0x7f4bc8153640 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.384+0000 7f4bcd8b5640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4bb406b4d0 0x7f4bb406d990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.384+0000 7f4bcd8b5640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc81617e0 0x7f4bc8153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.384+0000 7f4bcd8b5640 1 --2- 172.21.15.139:0/2364568665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bc80cec80 0x7f4bc8148b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.384+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/2364568665 >> 172.21.15.139:0/2364568665 conn(0x7f4bc80b5480 msgr2=0x7f4bc80d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.384+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/2364568665 shutdown_connections 2024-04-25T09:53:26.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.384+0000 7f4bcd8b5640 1 -- 172.21.15.139:0/2364568665 wait complete. 2024-04-25T09:53:26.397 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:53:26.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.492+0000 7fbd9469c640 1 Processor -- start 2024-04-25T09:53:26.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.492+0000 7fbd9469c640 1 -- start start 2024-04-25T09:53:26.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.493+0000 7fbd9469c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd8c070970 0x7fbd8c0b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.493+0000 7fbd9469c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c0b4600 con 0x7fbd8c0b4190 2024-04-25T09:53:26.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.493+0000 7fbd9469c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c0d0d40 con 0x7fbd8c15a870 2024-04-25T09:53:26.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.493+0000 7fbd9469c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c0d1020 con 0x7fbd8c070970 2024-04-25T09:53:26.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd92c39640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbd8c0b4190 0x7fbd8c0ce610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46352/0 (socket says 172.21.15.139:46352) 2024-04-25T09:53:26.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd92c39640 1 -- 172.21.15.139:0/762631497 learned_addr learned my addr 172.21.15.139:0/762631497 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:26.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91c37640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd8c070970 0x7fbd8c0b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455722361 0 0) 0x7fbd8c0b4600 con 0x7fbd8c0b4190 2024-04-25T09:53:26.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd78003520 con 0x7fbd8c0b4190 2024-04-25T09:53:26.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 906753598 0 0) 0x7fbd78003520 con 0x7fbd8c0b4190 2024-04-25T09:53:26.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd8c070970 msgr2=0x7fbd8c0b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd8c070970 0x7fbd8c0b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 >> v1:172.21.15.150:6789/0 conn(0x7fbd8c15a870 legacy=0x7fbd8c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c0b4b40 con 0x7fbd8c0b4190 2024-04-25T09:53:26.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fbd840019f0 con 0x7fbd8c0b4190 2024-04-25T09:53:26.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 >> v1:172.21.15.139:6789/0 conn(0x7fbd8c0b4190 legacy=0x7fbd8c0ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd8c070970 0x7fbd780039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd78003f60 0x7fbd78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd78004d40 0x7fbd780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c0d1020 con 0x7fbd78003f60 2024-04-25T09:53:26.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7800bc00 con 0x7fbd78004d40 2024-04-25T09:53:26.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7800be00 con 0x7fbd8c070970 2024-04-25T09:53:26.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91c37640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd78003f60 0x7fbd78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd92438640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd8c070970 0x7fbd780039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd92c39640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd78004d40 0x7fbd780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91c37640 1 -- 172.21.15.139:0/762631497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd8c070970 msgr2=0x7fbd780039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91c37640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd8c070970 0x7fbd780039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91c37640 1 -- 172.21.15.139:0/762631497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd78004d40 msgr2=0x7fbd780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:26.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91c37640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd78004d40 0x7fbd780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd91c37640 1 -- 172.21.15.139:0/762631497 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd78003520 con 0x7fbd78003f60 2024-04-25T09:53:26.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.494+0000 7fbd92c39640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd78004d40 0x7fbd780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:26.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.495+0000 7fbd91c37640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd78003f60 0x7fbd78004800 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009ac0 tx=0x7fbd78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5d61d5900f2aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.495+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80036070 con 0x7fbd78003f60 2024-04-25T09:53:26.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.495+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd80031400 con 0x7fbd78003f60 2024-04-25T09:53:26.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.495+0000 7fbd91436640 1 -- 172.21.15.139:0/762631497 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80009690 con 0x7fbd78003f60 2024-04-25T09:53:26.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.496+0000 7fbd9469c640 1 -- 172.21.15.139:0/762631497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd78003f60 msgr2=0x7fbd78004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.496+0000 7fbd9469c640 1 --2- 172.21.15.139:0/762631497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd78003f60 0x7fbd78004800 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009ac0 tx=0x7fbd78005830 comp rx=0 tx=0).stop 2024-04-25T09:53:26.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.496+0000 7fbd92c39640 1 -- 172.21.15.139:0/762631497 reap_dead start 2024-04-25T09:53:26.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.496+0000 7fbd9469c640 1 -- 172.21.15.139:0/762631497 shutdown_connections 2024-04-25T09:53:26.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.496+0000 7fbd9469c640 1 -- 172.21.15.139:0/762631497 >> 172.21.15.139:0/762631497 conn(0x7fbd8c0b23f0 msgr2=0x7fbd8c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.496+0000 7fbd9469c640 1 -- 172.21.15.139:0/762631497 shutdown_connections 2024-04-25T09:53:26.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.496+0000 7fbd9469c640 1 -- 172.21.15.139:0/762631497 wait complete. 2024-04-25T09:53:26.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.497+0000 7fbd9469c640 1 Processor -- start 2024-04-25T09:53:26.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.497+0000 7fbd9469c640 1 -- start start 2024-04-25T09:53:26.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd9469c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd78003f60 0x7fbd8c14cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd9469c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd8c14f5c0 0x7fbd8c153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd9469c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd8c14fa20 0x7fbd8c1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd9469c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c0b5620 con 0x7fbd8c14fa20 2024-04-25T09:53:26.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd9469c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c0b5190 con 0x7fbd8c14f5c0 2024-04-25T09:53:26.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd9469c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c0b57a0 con 0x7fbd78003f60 2024-04-25T09:53:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd92c39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd8c14fa20 0x7fbd8c1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd92c39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd8c14fa20 0x7fbd8c1540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58454/0 (socket says 172.21.15.139:58454) 2024-04-25T09:53:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd92c39640 1 -- 172.21.15.139:0/3163530290 learned_addr learned my addr 172.21.15.139:0/3163530290 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd92438640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd78003f60 0x7fbd8c14cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd91c37640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd8c14f5c0 0x7fbd8c153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd92c39640 1 -- 172.21.15.139:0/3163530290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd78003f60 msgr2=0x7fbd8c14cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd92c39640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd78003f60 0x7fbd8c14cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd92c39640 1 -- 172.21.15.139:0/3163530290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd8c14f5c0 msgr2=0x7fbd8c153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd92c39640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd8c14f5c0 0x7fbd8c153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd92c39640 1 -- 172.21.15.139:0/3163530290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c14ff60 con 0x7fbd8c14fa20 2024-04-25T09:53:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd91c37640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd8c14f5c0 0x7fbd8c153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd92c39640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd8c14fa20 0x7fbd8c1540c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c06de10 tx=0x7fbd84010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd7effd640 1 -- 172.21.15.139:0/3163530290 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84012450 con 0x7fbd8c14fa20 2024-04-25T09:53:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd7effd640 1 -- 172.21.15.139:0/3163530290 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8401a070 con 0x7fbd8c14fa20 2024-04-25T09:53:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd7effd640 1 -- 172.21.15.139:0/3163530290 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84012850 con 0x7fbd8c14fa20 2024-04-25T09:53:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd9469c640 1 -- 172.21.15.139:0/3163530290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8c150240 con 0x7fbd8c14fa20 2024-04-25T09:53:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.498+0000 7fbd9469c640 1 -- 172.21.15.139:0/3163530290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8c150800 con 0x7fbd8c14fa20 2024-04-25T09:53:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.499+0000 7fbd9469c640 1 -- 172.21.15.139:0/3163530290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd8c074520 con 0x7fbd8c14fa20 2024-04-25T09:53:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.500+0000 7fbd7effd640 1 -- 172.21.15.139:0/3163530290 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd84012c00 con 0x7fbd8c14fa20 2024-04-25T09:53:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.500+0000 7fbd7effd640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbd7006b4d0 0x7fbd7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.500+0000 7fbd7effd640 1 -- 172.21.15.139:0/3163530290 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbd8408cee0 con 0x7fbd8c14fa20 2024-04-25T09:53:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.500+0000 7fbd92438640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbd7006b4d0 0x7fbd7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.501+0000 7fbd92438640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbd7006b4d0 0x7fbd7006d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbd880035f0 tx=0x7fbd88009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.504+0000 7fbd7effd640 1 -- 172.21.15.139:0/3163530290 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd8405b910 con 0x7fbd8c14fa20 2024-04-25T09:53:26.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.685+0000 7fbd9469c640 1 -- 172.21.15.139:0/3163530290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbd8c0728e0 con 0x7fbd8c14fa20 2024-04-25T09:53:26.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.685+0000 7fbd7effd640 1 -- 172.21.15.139:0/3163530290 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbd8405b210 con 0x7fbd8c14fa20 2024-04-25T09:53:26.688 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T09:53:26.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.687+0000 7fbd9469c640 1 -- 172.21.15.139:0/3163530290 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbd7006b4d0 msgr2=0x7fbd7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.687+0000 7fbd9469c640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbd7006b4d0 0x7fbd7006d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbd880035f0 tx=0x7fbd88009040 comp rx=0 tx=0).stop 2024-04-25T09:53:26.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.687+0000 7fbd9469c640 1 -- 172.21.15.139:0/3163530290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd8c14fa20 msgr2=0x7fbd8c1540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.687+0000 7fbd9469c640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd8c14fa20 0x7fbd8c1540c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c06de10 tx=0x7fbd84010740 comp rx=0 tx=0).stop 2024-04-25T09:53:26.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.687+0000 7fbd9469c640 1 -- 172.21.15.139:0/3163530290 shutdown_connections 2024-04-25T09:53:26.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.687+0000 7fbd9469c640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd8c14fa20 0x7fbd8c1540c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.687+0000 7fbd9469c640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbd7006b4d0 0x7fbd7006d990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.687+0000 7fbd9469c640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd8c14f5c0 0x7fbd8c153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.687+0000 7fbd9469c640 1 --2- 172.21.15.139:0/3163530290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd78003f60 0x7fbd8c14cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.687+0000 7fbd9469c640 1 -- 172.21.15.139:0/3163530290 >> 172.21.15.139:0/3163530290 conn(0x7fbd8c0b23f0 msgr2=0x7fbd8c0b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.688+0000 7fbd9469c640 1 -- 172.21.15.139:0/3163530290 shutdown_connections 2024-04-25T09:53:26.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.688+0000 7fbd9469c640 1 -- 172.21.15.139:0/3163530290 wait complete. 2024-04-25T09:53:26.701 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:53:26.701 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:26.717 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.852+0000 7ff9050cb640 1 Processor -- start 2024-04-25T09:53:26.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.852+0000 7ff9050cb640 1 -- start start 2024-04-25T09:53:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.852+0000 7ff9050cb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff90015a860 0x7ff9000cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.852+0000 7ff9050cb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9000af6d0 con 0x7ff90006ea20 2024-04-25T09:53:26.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.852+0000 7ff9050cb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9000af9b0 con 0x7ff9000cd1f0 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.852+0000 7ff9050cb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9000b02e0 con 0x7ff90015a860 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.853+0000 7ff8fed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff90015a860 0x7ff9000cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.853+0000 7ff8ff59e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff90006ea20 0x7ff9000b0770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46366/0 (socket says 172.21.15.139:46366) 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.853+0000 7ff8ff59e640 1 -- 172.21.15.139:0/107740080 learned_addr learned my addr 172.21.15.139:0/107740080 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.853+0000 7ff8fed9d640 1 -- 172.21.15.139:0/107740080 >> v1:172.21.15.150:6789/0 conn(0x7ff9000cd1f0 legacy=0x7ff90006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.853+0000 7ff8fed9d640 1 -- 172.21.15.139:0/107740080 >> v1:172.21.15.139:6789/0 conn(0x7ff90006ea20 legacy=0x7ff9000b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.853+0000 7ff8fe59c640 1 -- 172.21.15.139:0/107740080 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3353302051 0 0) 0x7ff9000af6d0 con 0x7ff90006ea20 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.853+0000 7ff8fed9d640 1 -- 172.21.15.139:0/107740080 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000afc90 con 0x7ff90015a860 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.854+0000 7ff8fed9d640 1 --2- 172.21.15.139:0/107740080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff90015a860 0x7ff9000cccb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00097a0 tx=0x7ff8f002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0623d7621dcde68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.854+0000 7ff8fe59c640 1 -- 172.21.15.139:0/107740080 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0037070 con 0x7ff90015a860 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.854+0000 7ff8fe59c640 1 -- 172.21.15.139:0/107740080 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0032440 con 0x7ff90015a860 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.854+0000 7ff8fe59c640 1 -- 172.21.15.139:0/107740080 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0032810 con 0x7ff90015a860 2024-04-25T09:53:26.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.854+0000 7ff9050cb640 1 -- 172.21.15.139:0/107740080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff90015a860 msgr2=0x7ff9000cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.854+0000 7ff9050cb640 1 --2- 172.21.15.139:0/107740080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff90015a860 0x7ff9000cccb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00097a0 tx=0x7ff8f002a230 comp rx=0 tx=0).stop 2024-04-25T09:53:26.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.854+0000 7ff9050cb640 1 -- 172.21.15.139:0/107740080 shutdown_connections 2024-04-25T09:53:26.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.854+0000 7ff9050cb640 1 --2- 172.21.15.139:0/107740080 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff90015a860 0x7ff9000cccb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.854+0000 7ff9050cb640 1 -- 172.21.15.139:0/107740080 >> 172.21.15.139:0/107740080 conn(0x7ff900066f20 msgr2=0x7ff90006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:26.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.854+0000 7ff9050cb640 1 -- 172.21.15.139:0/107740080 shutdown_connections 2024-04-25T09:53:26.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.854+0000 7ff9050cb640 1 -- 172.21.15.139:0/107740080 wait complete. 2024-04-25T09:53:26.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.856+0000 7ff9050cb640 1 Processor -- start 2024-04-25T09:53:26.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.856+0000 7ff9050cb640 1 -- start start 2024-04-25T09:53:26.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.856+0000 7ff9050cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff90006ea20 0x7ff900158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.856+0000 7ff9050cb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9000cd1f0 0x7ff9000a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.856+0000 7ff9050cb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff90015a860 0x7ff9000add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.856+0000 7ff9050cb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9000af420 con 0x7ff90006ea20 2024-04-25T09:53:26.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.856+0000 7ff9050cb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9000af2a0 con 0x7ff90015a860 2024-04-25T09:53:26.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.856+0000 7ff9050cb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90015ac40 con 0x7ff9000cd1f0 2024-04-25T09:53:26.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.856+0000 7ff8fed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff90006ea20 0x7ff900158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.856+0000 7ff8fed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff90006ea20 0x7ff900158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58464/0 (socket says 172.21.15.139:58464) 2024-04-25T09:53:26.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.856+0000 7ff8fed9d640 1 -- 172.21.15.139:0/2852044525 learned_addr learned my addr 172.21.15.139:0/2852044525 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:26.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff8f7fff640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9000cd1f0 0x7ff9000a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff8ff59e640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff90015a860 0x7ff9000add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff8fed9d640 1 -- 172.21.15.139:0/2852044525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9000cd1f0 msgr2=0x7ff9000a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff8fed9d640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9000cd1f0 0x7ff9000a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff8fed9d640 1 -- 172.21.15.139:0/2852044525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff90015a860 msgr2=0x7ff9000add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:26.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff8fed9d640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff90015a860 0x7ff9000add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:26.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff8fed9d640 1 -- 172.21.15.139:0/2852044525 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000a87e0 con 0x7ff90006ea20 2024-04-25T09:53:26.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff8fed9d640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff90006ea20 0x7ff900158350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00098c0 tx=0x7ff8f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff8f77fe640 1 -- 172.21.15.139:0/2852044525 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0041070 con 0x7ff90006ea20 2024-04-25T09:53:26.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff8f77fe640 1 -- 172.21.15.139:0/2852044525 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f003c930 con 0x7ff90006ea20 2024-04-25T09:53:26.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff9050cb640 1 -- 172.21.15.139:0/2852044525 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9001f2980 con 0x7ff90006ea20 2024-04-25T09:53:26.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff8f77fe640 1 -- 172.21.15.139:0/2852044525 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0037040 con 0x7ff90006ea20 2024-04-25T09:53:26.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.857+0000 7ff9050cb640 1 -- 172.21.15.139:0/2852044525 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9001f2f40 con 0x7ff90006ea20 2024-04-25T09:53:26.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.858+0000 7ff9050cb640 1 -- 172.21.15.139:0/2852044525 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff900074520 con 0x7ff90006ea20 2024-04-25T09:53:26.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.859+0000 7ff8f77fe640 1 -- 172.21.15.139:0/2852044525 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8f0044070 con 0x7ff90006ea20 2024-04-25T09:53:26.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.859+0000 7ff8f77fe640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8e006b4d0 0x7ff8e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:26.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.859+0000 7ff8f77fe640 1 -- 172.21.15.139:0/2852044525 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff8f00ab220 con 0x7ff90006ea20 2024-04-25T09:53:26.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.859+0000 7ff8f7fff640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8e006b4d0 0x7ff8e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:26.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.860+0000 7ff8f7fff640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8e006b4d0 0x7ff8e006d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff9001582e0 tx=0x7ff8e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:26.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:26.863+0000 7ff8f77fe640 1 -- 172.21.15.139:0/2852044525 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9000a87e0 con 0x7ff90006ea20 2024-04-25T09:53:27.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.035+0000 7ff9050cb640 1 -- 172.21.15.139:0/2852044525 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9001f3870 con 0x7ff90006ea20 2024-04-25T09:53:27.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.036+0000 7ff8f77fe640 1 -- 172.21.15.139:0/2852044525 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7ff8f0079830 con 0x7ff90006ea20 2024-04-25T09:53:27.039 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:27.039 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.038+0000 7ff9050cb640 1 -- 172.21.15.139:0/2852044525 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8e006b4d0 msgr2=0x7ff8e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.038+0000 7ff9050cb640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8e006b4d0 0x7ff8e006d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff9001582e0 tx=0x7ff8e8009040 comp rx=0 tx=0).stop 2024-04-25T09:53:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.039+0000 7ff9050cb640 1 -- 172.21.15.139:0/2852044525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff90006ea20 msgr2=0x7ff900158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.039+0000 7ff9050cb640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff90006ea20 0x7ff900158350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00098c0 tx=0x7ff8f00046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.039+0000 7ff9050cb640 1 -- 172.21.15.139:0/2852044525 shutdown_connections 2024-04-25T09:53:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.039+0000 7ff9050cb640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8e006b4d0 0x7ff8e006d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.039+0000 7ff9050cb640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff90015a860 0x7ff9000add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.039+0000 7ff9050cb640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9000cd1f0 0x7ff9000a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.039+0000 7ff9050cb640 1 --2- 172.21.15.139:0/2852044525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff90006ea20 0x7ff900158350 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.039+0000 7ff9050cb640 1 -- 172.21.15.139:0/2852044525 >> 172.21.15.139:0/2852044525 conn(0x7ff900066f20 msgr2=0x7ff9000b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.039+0000 7ff9050cb640 1 -- 172.21.15.139:0/2852044525 shutdown_connections 2024-04-25T09:53:27.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.039+0000 7ff9050cb640 1 -- 172.21.15.139:0/2852044525 wait complete. 2024-04-25T09:53:27.052 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:27.052 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:27.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.151+0000 7f72a7276640 1 Processor -- start 2024-04-25T09:53:27.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.151+0000 7f72a7276640 1 -- start start 2024-04-25T09:53:27.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a7276640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f72a0072db0 0x7f72a006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a7276640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a015f3a0 0x7f72a00cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a7276640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a015bc60 con 0x7f72a015b880 2024-04-25T09:53:27.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a7276640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a0070cd0 con 0x7f72a0072db0 2024-04-25T09:53:27.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a7276640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00cffa0 con 0x7f72a015f3a0 2024-04-25T09:53:27.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a4811640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f72a0072db0 0x7f72a006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a5012640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f72a015b880 0x7f72a00726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46368/0 (socket says 172.21.15.139:46368) 2024-04-25T09:53:27.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a5012640 1 -- 172.21.15.139:0/2424638085 learned_addr learned my addr 172.21.15.139:0/2424638085 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:27.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a5813640 1 --2- 172.21.15.139:0/2424638085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a015f3a0 0x7f72a00cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a4811640 1 -- 172.21.15.139:0/2424638085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a015f3a0 msgr2=0x7f72a00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:27.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a4811640 1 --2- 172.21.15.139:0/2424638085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a015f3a0 0x7f72a00cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a4811640 1 -- 172.21.15.139:0/2424638085 >> v1:172.21.15.139:6789/0 conn(0x7f72a015b880 legacy=0x7f72a00726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f72a4811640 1 -- 172.21.15.139:0/2424638085 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a00cf6b0 con 0x7f72a0072db0 2024-04-25T09:53:27.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.152+0000 7f7297fff640 1 -- 172.21.15.139:0/2424638085 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233266497 0 0) 0x7f72a015bc60 con 0x7f72a015b880 2024-04-25T09:53:27.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.153+0000 7f72a4811640 1 --2- 172.21.15.139:0/2424638085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a0072db0 0x7f72a006fa90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f72a00d0120 tx=0x7f729002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a86c4554189a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.153+0000 7f7297fff640 1 -- 172.21.15.139:0/2424638085 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290037070 con 0x7f72a0072db0 2024-04-25T09:53:27.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.153+0000 7f7297fff640 1 -- 172.21.15.139:0/2424638085 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7290032440 con 0x7f72a0072db0 2024-04-25T09:53:27.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.153+0000 7f7297fff640 1 -- 172.21.15.139:0/2424638085 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290032810 con 0x7f72a0072db0 2024-04-25T09:53:27.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.153+0000 7f72a7276640 1 -- 172.21.15.139:0/2424638085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a0072db0 msgr2=0x7f72a006fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.153+0000 7f72a7276640 1 --2- 172.21.15.139:0/2424638085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a0072db0 0x7f72a006fa90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f72a00d0120 tx=0x7f729002a180 comp rx=0 tx=0).stop 2024-04-25T09:53:27.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.153+0000 7f72a7276640 1 -- 172.21.15.139:0/2424638085 shutdown_connections 2024-04-25T09:53:27.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.153+0000 7f72a7276640 1 --2- 172.21.15.139:0/2424638085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a015f3a0 0x7f72a00cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.153+0000 7f72a7276640 1 --2- 172.21.15.139:0/2424638085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a0072db0 0x7f72a006fa90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.153+0000 7f72a7276640 1 -- 172.21.15.139:0/2424638085 >> 172.21.15.139:0/2424638085 conn(0x7f72a00b23f0 msgr2=0x7f72a00b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.154+0000 7f72a7276640 1 -- 172.21.15.139:0/2424638085 shutdown_connections 2024-04-25T09:53:27.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.154+0000 7f72a7276640 1 -- 172.21.15.139:0/2424638085 wait complete. 2024-04-25T09:53:27.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.155+0000 7f72a7276640 1 Processor -- start 2024-04-25T09:53:27.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.155+0000 7f72a7276640 1 -- start start 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.155+0000 7f72a7276640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a0072db0 0x7f72a0140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.155+0000 7f72a7276640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a015b880 0x7f72a015de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.155+0000 7f72a7276640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a015f3a0 0x7f72a015e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.155+0000 7f72a7276640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00cfc30 con 0x7f72a015b880 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.155+0000 7f72a7276640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00cf950 con 0x7f72a0072db0 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.155+0000 7f72a7276640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a0070610 con 0x7f72a015f3a0 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.155+0000 7f72a4811640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a015b880 0x7f72a015de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.155+0000 7f72a4811640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a015b880 0x7f72a015de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58480/0 (socket says 172.21.15.139:58480) 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.155+0000 7f72a4811640 1 -- 172.21.15.139:0/4008750979 learned_addr learned my addr 172.21.15.139:0/4008750979 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f72a5813640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a015f3a0 0x7f72a015e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f72a5012640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a0072db0 0x7f72a0140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f72a4811640 1 -- 172.21.15.139:0/4008750979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a015f3a0 msgr2=0x7f72a015e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f72a4811640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a015f3a0 0x7f72a015e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f72a4811640 1 -- 172.21.15.139:0/4008750979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a0072db0 msgr2=0x7f72a0140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f72a4811640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a0072db0 0x7f72a0140080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f72a4811640 1 -- 172.21.15.139:0/4008750979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a01f5e10 con 0x7f72a015b880 2024-04-25T09:53:27.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f72a4811640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a015b880 0x7f72a015de10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7290000c00 tx=0x7f72900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f7295ffb640 1 -- 172.21.15.139:0/4008750979 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290037040 con 0x7f72a015b880 2024-04-25T09:53:27.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f7295ffb640 1 -- 172.21.15.139:0/4008750979 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7290042070 con 0x7f72a015b880 2024-04-25T09:53:27.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f72a7276640 1 -- 172.21.15.139:0/4008750979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a01405c0 con 0x7f72a015b880 2024-04-25T09:53:27.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f7295ffb640 1 -- 172.21.15.139:0/4008750979 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290030a50 con 0x7f72a015b880 2024-04-25T09:53:27.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.156+0000 7f72a7276640 1 -- 172.21.15.139:0/4008750979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a01f5330 con 0x7f72a015b880 2024-04-25T09:53:27.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.157+0000 7f72a7276640 1 -- 172.21.15.139:0/4008750979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a0074520 con 0x7f72a015b880 2024-04-25T09:53:27.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.158+0000 7f7295ffb640 1 -- 172.21.15.139:0/4008750979 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7290030bf0 con 0x7f72a015b880 2024-04-25T09:53:27.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.158+0000 7f7295ffb640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f728406b500 0x7f728406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.158+0000 7f7295ffb640 1 -- 172.21.15.139:0/4008750979 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f72900aaf90 con 0x7f72a015b880 2024-04-25T09:53:27.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.158+0000 7f72a5012640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f728406b500 0x7f728406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.158+0000 7f72a5012640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f728406b500 0x7f728406d9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f729c000f60 tx=0x7f729c007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.162+0000 7f7295ffb640 1 -- 172.21.15.139:0/4008750979 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72a01405c0 con 0x7f72a015b880 2024-04-25T09:53:27.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.332+0000 7f72a7276640 1 -- 172.21.15.139:0/4008750979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72a015b0a0 con 0x7f72a015b880 2024-04-25T09:53:27.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.332+0000 7f7295ffb640 1 -- 172.21.15.139:0/4008750979 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7290079c80 con 0x7f72a015b880 2024-04-25T09:53:27.335 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:53:27.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.334+0000 7f72a7276640 1 -- 172.21.15.139:0/4008750979 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f728406b500 msgr2=0x7f728406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.334+0000 7f72a7276640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f728406b500 0x7f728406d9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f729c000f60 tx=0x7f729c007b50 comp rx=0 tx=0).stop 2024-04-25T09:53:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.334+0000 7f72a7276640 1 -- 172.21.15.139:0/4008750979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a015b880 msgr2=0x7f72a015de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.334+0000 7f72a7276640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a015b880 0x7f72a015de10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7290000c00 tx=0x7f72900046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.334+0000 7f72a7276640 1 -- 172.21.15.139:0/4008750979 shutdown_connections 2024-04-25T09:53:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.334+0000 7f72a7276640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72a015b880 0x7f72a015de10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.334+0000 7f72a7276640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f728406b500 0x7f728406d9c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.334+0000 7f72a7276640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a015f3a0 0x7f72a015e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.334+0000 7f72a7276640 1 --2- 172.21.15.139:0/4008750979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a0072db0 0x7f72a0140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.334+0000 7f72a7276640 1 -- 172.21.15.139:0/4008750979 >> 172.21.15.139:0/4008750979 conn(0x7f72a00b23f0 msgr2=0x7f72a00b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.334+0000 7f72a7276640 1 -- 172.21.15.139:0/4008750979 shutdown_connections 2024-04-25T09:53:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.334+0000 7f72a7276640 1 -- 172.21.15.139:0/4008750979 wait complete. 2024-04-25T09:53:27.347 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:27.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.445+0000 7fb5121e0640 1 Processor -- start 2024-04-25T09:53:27.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.445+0000 7fb5121e0640 1 -- start start 2024-04-25T09:53:27.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.445+0000 7fb5121e0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb50c06ea40 0x7fb50c15e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb5121e0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50c0b2a50 con 0x7fb50c0b31a0 2024-04-25T09:53:27.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb5121e0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50c0cd0f0 con 0x7fb50c144090 2024-04-25T09:53:27.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb5121e0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50c0b3610 con 0x7fb50c06ea40 2024-04-25T09:53:27.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50affd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb50c0b31a0 0x7fb50c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46382/0 (socket says 172.21.15.139:46382) 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50affd640 1 -- 172.21.15.139:0/2444080673 learned_addr learned my addr 172.21.15.139:0/2444080673 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50bfff640 1 --2- 172.21.15.139:0/2444080673 >> v2:172.21.15.150:3300/0 conn(0x7fb50c06ea40 0x7fb50c15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298537129 0 0) 0x7fb4fc001110 con 0x7fb50c0b31a0 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4f4003520 con 0x7fb50c0b31a0 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1172347740 0 0) 0x7fb4f4003520 con 0x7fb50c0b31a0 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 >> v2:172.21.15.150:3300/0 conn(0x7fb50c06ea40 msgr2=0x7fb50c15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 --2- 172.21.15.139:0/2444080673 >> v2:172.21.15.150:3300/0 conn(0x7fb50c06ea40 0x7fb50c15e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 >> v1:172.21.15.150:6789/0 conn(0x7fb50c144090 legacy=0x7fb50c0cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c15b090 con 0x7fb50c0b31a0 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fb4fc001840 con 0x7fb50c0b31a0 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 >> v1:172.21.15.139:6789/0 conn(0x7fb50c0b31a0 legacy=0x7fb50c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 --2- 172.21.15.139:0/2444080673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4f40039b0 0x7fb4f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 --2- 172.21.15.139:0/2444080673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4f4004800 0x7fb4f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 --2- 172.21.15.139:0/2444080673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4f40051c0 0x7fb4f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c0b3610 con 0x7fb4f4004800 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f400c160 con 0x7fb4f40039b0 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f400c360 con 0x7fb4f40051c0 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50affd640 1 --2- 172.21.15.139:0/2444080673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4f4004800 0x7fb4f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50b7fe640 1 --2- 172.21.15.139:0/2444080673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4f40039b0 0x7fb4f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.446+0000 7fb50bfff640 1 --2- 172.21.15.139:0/2444080673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4f40051c0 0x7fb4f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.447+0000 7fb50affd640 1 -- 172.21.15.139:0/2444080673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4f40051c0 msgr2=0x7fb4f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.447+0000 7fb50affd640 1 --2- 172.21.15.139:0/2444080673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4f40051c0 0x7fb4f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.447+0000 7fb50affd640 1 -- 172.21.15.139:0/2444080673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4f40039b0 msgr2=0x7fb4f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.447+0000 7fb50affd640 1 --2- 172.21.15.139:0/2444080673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4f40039b0 0x7fb4f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.447+0000 7fb50affd640 1 -- 172.21.15.139:0/2444080673 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4003520 con 0x7fb4f4004800 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.447+0000 7fb50bfff640 1 -- 172.21.15.139:0/2444080673 reap_dead start 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.447+0000 7fb50affd640 1 --2- 172.21.15.139:0/2444080673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4f4004800 0x7fb4f4004c80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00c4c0 tx=0x7fb4f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31c944a46276fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.447+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc00c0e0 con 0x7fb4f4004800 2024-04-25T09:53:27.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.447+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4fc034070 con 0x7fb4f4004800 2024-04-25T09:53:27.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.447+0000 7fb50a7fc640 1 -- 172.21.15.139:0/2444080673 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc005d70 con 0x7fb4f4004800 2024-04-25T09:53:27.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.448+0000 7fb5121e0640 1 -- 172.21.15.139:0/2444080673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4f4004800 msgr2=0x7fb4f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.448+0000 7fb5121e0640 1 --2- 172.21.15.139:0/2444080673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4f4004800 0x7fb4f4004c80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00c4c0 tx=0x7fb4f4005830 comp rx=0 tx=0).stop 2024-04-25T09:53:27.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.448+0000 7fb5121e0640 1 -- 172.21.15.139:0/2444080673 shutdown_connections 2024-04-25T09:53:27.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.448+0000 7fb5121e0640 1 --2- 172.21.15.139:0/2444080673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4f4004800 0x7fb4f4004c80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.448+0000 7fb5121e0640 1 -- 172.21.15.139:0/2444080673 >> 172.21.15.139:0/2444080673 conn(0x7fb50c067000 msgr2=0x7fb50c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.448+0000 7fb5121e0640 1 -- 172.21.15.139:0/2444080673 shutdown_connections 2024-04-25T09:53:27.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.448+0000 7fb5121e0640 1 -- 172.21.15.139:0/2444080673 wait complete. 2024-04-25T09:53:27.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.449+0000 7fb5121e0640 1 Processor -- start 2024-04-25T09:53:27.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.449+0000 7fb5121e0640 1 -- start start 2024-04-25T09:53:27.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb5121e0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4f4004800 0x7fb50c0ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb5121e0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb50c1f1140 0x7fb50c0ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb5121e0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb50c13ebd0 0x7fb50c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb5121e0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c15b930 con 0x7fb50c13ebd0 2024-04-25T09:53:27.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb5121e0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c15b650 con 0x7fb4f4004800 2024-04-25T09:53:27.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb5121e0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c15bc10 con 0x7fb50c1f1140 2024-04-25T09:53:27.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb50bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb50c13ebd0 0x7fb50c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb50bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb50c13ebd0 0x7fb50c15d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58498/0 (socket says 172.21.15.139:58498) 2024-04-25T09:53:27.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb50bfff640 1 -- 172.21.15.139:0/1796940070 learned_addr learned my addr 172.21.15.139:0/1796940070 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:27.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb50affd640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb50c1f1140 0x7fb50c0ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb50b7fe640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4f4004800 0x7fb50c0ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb50bfff640 1 -- 172.21.15.139:0/1796940070 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb50c1f1140 msgr2=0x7fb50c0ac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb50bfff640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb50c1f1140 0x7fb50c0ac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb50bfff640 1 -- 172.21.15.139:0/1796940070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4f4004800 msgr2=0x7fb50c0ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb50bfff640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4f4004800 0x7fb50c0ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb50bfff640 1 -- 172.21.15.139:0/1796940070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c0aca90 con 0x7fb50c13ebd0 2024-04-25T09:53:27.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.450+0000 7fb50b7fe640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4f4004800 0x7fb50c0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:27.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.451+0000 7fb50bfff640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb50c13ebd0 0x7fb50c15d060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0b30f0 tx=0x7fb50400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.451+0000 7fb4f3fff640 1 -- 172.21.15.139:0/1796940070 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504011e90 con 0x7fb50c13ebd0 2024-04-25T09:53:27.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.451+0000 7fb4f3fff640 1 -- 172.21.15.139:0/1796940070 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50401c070 con 0x7fb50c13ebd0 2024-04-25T09:53:27.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.451+0000 7fb4f3fff640 1 -- 172.21.15.139:0/1796940070 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504003bd0 con 0x7fb50c13ebd0 2024-04-25T09:53:27.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.451+0000 7fb5121e0640 1 -- 172.21.15.139:0/1796940070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c0acd70 con 0x7fb50c13ebd0 2024-04-25T09:53:27.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.451+0000 7fb5121e0640 1 -- 172.21.15.139:0/1796940070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c1f0940 con 0x7fb50c13ebd0 2024-04-25T09:53:27.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.452+0000 7fb5121e0640 1 -- 172.21.15.139:0/1796940070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50c1f0450 con 0x7fb50c13ebd0 2024-04-25T09:53:27.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.452+0000 7fb4f3fff640 1 -- 172.21.15.139:0/1796940070 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb504004030 con 0x7fb50c13ebd0 2024-04-25T09:53:27.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.453+0000 7fb4f3fff640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb4f806b670 0x7fb4f806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.453+0000 7fb4f3fff640 1 -- 172.21.15.139:0/1796940070 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb50408af80 con 0x7fb50c13ebd0 2024-04-25T09:53:27.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.453+0000 7fb50b7fe640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb4f806b670 0x7fb4f806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.453+0000 7fb50b7fe640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb4f806b670 0x7fb4f806db30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb500000c00 tx=0x7fb500009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.457+0000 7fb4f3fff640 1 -- 172.21.15.139:0/1796940070 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb504010040 con 0x7fb50c13ebd0 2024-04-25T09:53:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.624+0000 7fb5121e0640 1 -- 172.21.15.139:0/1796940070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb50c074520 con 0x7fb50c13ebd0 2024-04-25T09:53:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.624+0000 7fb4f3fff640 1 -- 172.21.15.139:0/1796940070 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb504059ac0 con 0x7fb50c13ebd0 2024-04-25T09:53:27.627 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:27.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.626+0000 7fb5121e0640 1 -- 172.21.15.139:0/1796940070 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb4f806b670 msgr2=0x7fb4f806db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.626+0000 7fb5121e0640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb4f806b670 0x7fb4f806db30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb500000c00 tx=0x7fb500009040 comp rx=0 tx=0).stop 2024-04-25T09:53:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.626+0000 7fb5121e0640 1 -- 172.21.15.139:0/1796940070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb50c13ebd0 msgr2=0x7fb50c15d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.626+0000 7fb5121e0640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb50c13ebd0 0x7fb50c15d060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0b30f0 tx=0x7fb50400f580 comp rx=0 tx=0).stop 2024-04-25T09:53:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.626+0000 7fb5121e0640 1 -- 172.21.15.139:0/1796940070 shutdown_connections 2024-04-25T09:53:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.626+0000 7fb5121e0640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb50c13ebd0 0x7fb50c15d060 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.626+0000 7fb5121e0640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb4f806b670 0x7fb4f806db30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.626+0000 7fb5121e0640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb50c1f1140 0x7fb50c0ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.626+0000 7fb5121e0640 1 --2- 172.21.15.139:0/1796940070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4f4004800 0x7fb50c0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.626+0000 7fb5121e0640 1 -- 172.21.15.139:0/1796940070 >> 172.21.15.139:0/1796940070 conn(0x7fb50c067000 msgr2=0x7fb50c0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.626+0000 7fb5121e0640 1 -- 172.21.15.139:0/1796940070 shutdown_connections 2024-04-25T09:53:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.626+0000 7fb5121e0640 1 -- 172.21.15.139:0/1796940070 wait complete. 2024-04-25T09:53:27.639 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:27.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.735+0000 7fa49de15640 1 Processor -- start 2024-04-25T09:53:27.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.736+0000 7fa49de15640 1 -- start start 2024-04-25T09:53:27.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.736+0000 7fa49de15640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa4980b4960 0x7fa4980668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.736+0000 7fa49de15640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa4981616d0 0x7fa498161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.736+0000 7fa49de15640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa498162030 con 0x7fa49815d9e0 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.736+0000 7fa49de15640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4980b34f0 con 0x7fa4981616d0 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.736+0000 7fa49de15640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4980b4ed0 con 0x7fa4980b4960 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.736+0000 7fa497fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa4981616d0 0x7fa498161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.736+0000 7fa4977fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa49815d9e0 0x7fa49806bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46386/0 (socket says 172.21.15.139:46386) 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.737+0000 7fa4977fe640 1 -- 172.21.15.139:0/4013040106 learned_addr learned my addr 172.21.15.139:0/4013040106 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.737+0000 7fa497fff640 1 -- 172.21.15.139:0/4013040106 >> v2:172.21.15.150:3300/0 conn(0x7fa4980b4960 msgr2=0x7fa4980668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.737+0000 7fa497fff640 1 --2- 172.21.15.139:0/4013040106 >> v2:172.21.15.150:3300/0 conn(0x7fa4980b4960 0x7fa4980668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.737+0000 7fa497fff640 1 -- 172.21.15.139:0/4013040106 >> v1:172.21.15.139:6789/0 conn(0x7fa49815d9e0 legacy=0x7fa49806bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.737+0000 7fa496ffd640 1 -- 172.21.15.139:0/4013040106 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377985876 0 0) 0x7fa498162030 con 0x7fa49815d9e0 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.737+0000 7fa497fff640 1 -- 172.21.15.139:0/4013040106 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980b2f30 con 0x7fa4981616d0 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.737+0000 7fa497fff640 1 --2- 172.21.15.139:0/4013040106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4981616d0 0x7fa498161af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa4980673d0 tx=0x7fa49002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5caf1e8c69a432 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.738+0000 7fa496ffd640 1 -- 172.21.15.139:0/4013040106 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490007740 con 0x7fa4981616d0 2024-04-25T09:53:27.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.738+0000 7fa496ffd640 1 -- 172.21.15.139:0/4013040106 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4900078e0 con 0x7fa4981616d0 2024-04-25T09:53:27.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.738+0000 7fa496ffd640 1 -- 172.21.15.139:0/4013040106 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490007cb0 con 0x7fa4981616d0 2024-04-25T09:53:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.738+0000 7fa49de15640 1 -- 172.21.15.139:0/4013040106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4981616d0 msgr2=0x7fa498161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.738+0000 7fa49de15640 1 --2- 172.21.15.139:0/4013040106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4981616d0 0x7fa498161af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa4980673d0 tx=0x7fa49002a680 comp rx=0 tx=0).stop 2024-04-25T09:53:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.738+0000 7fa49de15640 1 -- 172.21.15.139:0/4013040106 shutdown_connections 2024-04-25T09:53:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.738+0000 7fa49de15640 1 --2- 172.21.15.139:0/4013040106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4981616d0 0x7fa498161af0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.738+0000 7fa49de15640 1 --2- 172.21.15.139:0/4013040106 >> v2:172.21.15.150:3300/0 conn(0x7fa4980b4960 0x7fa4980668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.738+0000 7fa49de15640 1 -- 172.21.15.139:0/4013040106 >> 172.21.15.139:0/4013040106 conn(0x7fa4980cc820 msgr2=0x7fa4980ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.738+0000 7fa49de15640 1 -- 172.21.15.139:0/4013040106 shutdown_connections 2024-04-25T09:53:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.738+0000 7fa49de15640 1 -- 172.21.15.139:0/4013040106 wait complete. 2024-04-25T09:53:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.739+0000 7fa49de15640 1 Processor -- start 2024-04-25T09:53:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.740+0000 7fa49de15640 1 -- start start 2024-04-25T09:53:27.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.740+0000 7fa49de15640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4980b4960 0x7fa498157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.740+0000 7fa49de15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa49815d9e0 0x7fa498160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.740+0000 7fa49de15640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4981616d0 0x7fa49806add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.740+0000 7fa49de15640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49815ddc0 con 0x7fa49815d9e0 2024-04-25T09:53:27.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.740+0000 7fa49de15640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa498162430 con 0x7fa4980b4960 2024-04-25T09:53:27.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.740+0000 7fa49de15640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49815df40 con 0x7fa4981616d0 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.740+0000 7fa48effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa49815d9e0 0x7fa498160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.740+0000 7fa48effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa49815d9e0 0x7fa498160850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58520/0 (socket says 172.21.15.139:58520) 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.740+0000 7fa48effd640 1 -- 172.21.15.139:0/2070306855 learned_addr learned my addr 172.21.15.139:0/2070306855 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa497fff640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4981616d0 0x7fa49806add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa4977fe640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4980b4960 0x7fa498157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa48effd640 1 -- 172.21.15.139:0/2070306855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4981616d0 msgr2=0x7fa49806add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa48effd640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4981616d0 0x7fa49806add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa48effd640 1 -- 172.21.15.139:0/2070306855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4980b4960 msgr2=0x7fa498157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa48effd640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4980b4960 0x7fa498157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa48effd640 1 -- 172.21.15.139:0/2070306855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa498155780 con 0x7fa49815d9e0 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa48effd640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa49815d9e0 0x7fa498160850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa48000b4a0 tx=0x7fa48000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa494ff9640 1 -- 172.21.15.139:0/2070306855 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480014070 con 0x7fa49815d9e0 2024-04-25T09:53:27.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa494ff9640 1 -- 172.21.15.139:0/2070306855 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa480010610 con 0x7fa49815d9e0 2024-04-25T09:53:27.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa49de15640 1 -- 172.21.15.139:0/2070306855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa498155a60 con 0x7fa49815d9e0 2024-04-25T09:53:27.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa494ff9640 1 -- 172.21.15.139:0/2070306855 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480010a00 con 0x7fa49815d9e0 2024-04-25T09:53:27.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.741+0000 7fa49de15640 1 -- 172.21.15.139:0/2070306855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa498156020 con 0x7fa49815d9e0 2024-04-25T09:53:27.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.742+0000 7fa49de15640 1 -- 172.21.15.139:0/2070306855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa498074520 con 0x7fa49815d9e0 2024-04-25T09:53:27.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.743+0000 7fa494ff9640 1 -- 172.21.15.139:0/2070306855 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa480017070 con 0x7fa49815d9e0 2024-04-25T09:53:27.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.743+0000 7fa494ff9640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa47806b4d0 0x7fa47806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:27.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.743+0000 7fa494ff9640 1 -- 172.21.15.139:0/2070306855 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa480089ef0 con 0x7fa49815d9e0 2024-04-25T09:53:27.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.743+0000 7fa4977fe640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa47806b4d0 0x7fa47806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:27.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.743+0000 7fa4977fe640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa47806b4d0 0x7fa47806d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa488000f60 tx=0x7fa488007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:27.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.747+0000 7fa494ff9640 1 -- 172.21.15.139:0/2070306855 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa480058ac0 con 0x7fa49815d9e0 2024-04-25T09:53:27.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.922+0000 7fa49de15640 1 -- 172.21.15.139:0/2070306855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa49815d0b0 con 0x7fa49815d9e0 2024-04-25T09:53:27.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.923+0000 7fa494ff9640 1 -- 172.21.15.139:0/2070306855 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4800583c0 con 0x7fa49815d9e0 2024-04-25T09:53:27.926 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:53:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.925+0000 7fa49de15640 1 -- 172.21.15.139:0/2070306855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa47806b4d0 msgr2=0x7fa47806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.925+0000 7fa49de15640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa47806b4d0 0x7fa47806d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa488000f60 tx=0x7fa488007b50 comp rx=0 tx=0).stop 2024-04-25T09:53:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.925+0000 7fa49de15640 1 -- 172.21.15.139:0/2070306855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa49815d9e0 msgr2=0x7fa498160850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.925+0000 7fa49de15640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa49815d9e0 0x7fa498160850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa48000b4a0 tx=0x7fa48000b960 comp rx=0 tx=0).stop 2024-04-25T09:53:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.925+0000 7fa49de15640 1 -- 172.21.15.139:0/2070306855 shutdown_connections 2024-04-25T09:53:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.925+0000 7fa49de15640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa49815d9e0 0x7fa498160850 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.925+0000 7fa49de15640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa47806b4d0 0x7fa47806d990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.925+0000 7fa49de15640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4981616d0 0x7fa49806add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.925+0000 7fa49de15640 1 --2- 172.21.15.139:0/2070306855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4980b4960 0x7fa498157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:27.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.925+0000 7fa49de15640 1 -- 172.21.15.139:0/2070306855 >> 172.21.15.139:0/2070306855 conn(0x7fa4980cc820 msgr2=0x7fa4980afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:27.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.925+0000 7fa49de15640 1 -- 172.21.15.139:0/2070306855 shutdown_connections 2024-04-25T09:53:27.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:27.925+0000 7fa49de15640 1 -- 172.21.15.139:0/2070306855 wait complete. 2024-04-25T09:53:27.939 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:28.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.036+0000 7f1493a5d640 1 Processor -- start 2024-04-25T09:53:28.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.036+0000 7f1493a5d640 1 -- start start 2024-04-25T09:53:28.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.037+0000 7f1493a5d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f148c1617e0 0x7f148c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.037+0000 7f1493a5d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f148c0cec80 0x7f148c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.037+0000 7f1493a5d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f148c161dc0 con 0x7f148c15dbf0 2024-04-25T09:53:28.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.037+0000 7f1493a5d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148c0d1ee0 con 0x7f148c1617e0 2024-04-25T09:53:28.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.037+0000 7f1493a5d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148c1620a0 con 0x7f148c0cec80 2024-04-25T09:53:28.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.037+0000 7f1490ff8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f148c1617e0 0x7f148c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.037+0000 7f14917f9640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f148c15dbf0 0x7f148c0b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46400/0 (socket says 172.21.15.139:46400) 2024-04-25T09:53:28.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.037+0000 7f14917f9640 1 -- 172.21.15.139:0/2449370328 learned_addr learned my addr 172.21.15.139:0/2449370328 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:28.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.037+0000 7f1491ffa640 1 --2- 172.21.15.139:0/2449370328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f148c0cec80 0x7f148c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.038+0000 7f1490ff8640 1 -- 172.21.15.139:0/2449370328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f148c0cec80 msgr2=0x7f148c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:28.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.038+0000 7f1490ff8640 1 --2- 172.21.15.139:0/2449370328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f148c0cec80 0x7f148c0d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.038+0000 7f1490ff8640 1 -- 172.21.15.139:0/2449370328 >> v1:172.21.15.139:6789/0 conn(0x7f148c15dbf0 legacy=0x7f148c0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.038+0000 7f1483fff640 1 -- 172.21.15.139:0/2449370328 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270981747 0 0) 0x7f148c161dc0 con 0x7f148c15dbf0 2024-04-25T09:53:28.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.038+0000 7f1490ff8640 1 -- 172.21.15.139:0/2449370328 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c162380 con 0x7f148c1617e0 2024-04-25T09:53:28.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.038+0000 7f1490ff8640 1 --2- 172.21.15.139:0/2449370328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f148c1617e0 0x7f148c0ce740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f147c0097a0 tx=0x7f147c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935d060db2ff4cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.038+0000 7f1483fff640 1 -- 172.21.15.139:0/2449370328 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c037070 con 0x7f148c1617e0 2024-04-25T09:53:28.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.038+0000 7f1483fff640 1 -- 172.21.15.139:0/2449370328 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c02ae50 con 0x7f148c1617e0 2024-04-25T09:53:28.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.038+0000 7f1483fff640 1 -- 172.21.15.139:0/2449370328 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c032660 con 0x7f148c1617e0 2024-04-25T09:53:28.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.039+0000 7f1493a5d640 1 -- 172.21.15.139:0/2449370328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f148c1617e0 msgr2=0x7f148c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.039+0000 7f1493a5d640 1 --2- 172.21.15.139:0/2449370328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f148c1617e0 0x7f148c0ce740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f147c0097a0 tx=0x7f147c02a100 comp rx=0 tx=0).stop 2024-04-25T09:53:28.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.039+0000 7f1493a5d640 1 -- 172.21.15.139:0/2449370328 shutdown_connections 2024-04-25T09:53:28.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.039+0000 7f1493a5d640 1 --2- 172.21.15.139:0/2449370328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f148c0cec80 0x7f148c0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.039+0000 7f1493a5d640 1 --2- 172.21.15.139:0/2449370328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f148c1617e0 0x7f148c0ce740 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.039+0000 7f1493a5d640 1 -- 172.21.15.139:0/2449370328 >> 172.21.15.139:0/2449370328 conn(0x7f148c0b5480 msgr2=0x7f148c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:28.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.039+0000 7f1493a5d640 1 -- 172.21.15.139:0/2449370328 shutdown_connections 2024-04-25T09:53:28.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.039+0000 7f1493a5d640 1 -- 172.21.15.139:0/2449370328 wait complete. 2024-04-25T09:53:28.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.040+0000 7f1493a5d640 1 Processor -- start 2024-04-25T09:53:28.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.040+0000 7f1493a5d640 1 -- start start 2024-04-25T09:53:28.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1493a5d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f148c0cec80 0x7f148c0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1493a5d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f148c15dbf0 0x7f148c160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1493a5d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f148c1617e0 0x7f148c1605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1493a5d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c15dfd0 con 0x7f148c1617e0 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1493a5d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0d28b0 con 0x7f148c0cec80 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1493a5d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148c160af0 con 0x7f148c15dbf0 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1491ffa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f148c1617e0 0x7f148c1605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1491ffa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f148c1617e0 0x7f148c1605b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58530/0 (socket says 172.21.15.139:58530) 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1491ffa640 1 -- 172.21.15.139:0/3332326986 learned_addr learned my addr 172.21.15.139:0/3332326986 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1490ff8640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f148c15dbf0 0x7f148c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f14917f9640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f148c0cec80 0x7f148c0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1491ffa640 1 -- 172.21.15.139:0/3332326986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f148c15dbf0 msgr2=0x7f148c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1491ffa640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f148c15dbf0 0x7f148c160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1491ffa640 1 -- 172.21.15.139:0/3332326986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f148c0cec80 msgr2=0x7f148c0c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1491ffa640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f148c0cec80 0x7f148c0c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f1491ffa640 1 -- 172.21.15.139:0/3332326986 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c14f640 con 0x7f148c1617e0 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.041+0000 7f14917f9640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f148c0cec80 0x7f148c0c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.042+0000 7f1491ffa640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f148c1617e0 0x7f148c1605b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1484007e40 tx=0x7f148400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.042+0000 7f1481ffb640 1 -- 172.21.15.139:0/3332326986 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484014620 con 0x7f148c1617e0 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.042+0000 7f1481ffb640 1 -- 172.21.15.139:0/3332326986 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1484019070 con 0x7f148c1617e0 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.042+0000 7f1481ffb640 1 -- 172.21.15.139:0/3332326986 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484014a20 con 0x7f148c1617e0 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.042+0000 7f1493a5d640 1 -- 172.21.15.139:0/3332326986 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c14f920 con 0x7f148c1617e0 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.042+0000 7f1493a5d640 1 -- 172.21.15.139:0/3332326986 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c0c93b0 con 0x7f148c1617e0 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.043+0000 7f1493a5d640 1 -- 172.21.15.139:0/3332326986 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148c074520 con 0x7f148c1617e0 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.043+0000 7f1481ffb640 1 -- 172.21.15.139:0/3332326986 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1484014bc0 con 0x7f148c1617e0 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.044+0000 7f1481ffb640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f147006b4d0 0x7f147006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.044+0000 7f1481ffb640 1 -- 172.21.15.139:0/3332326986 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f148408ad40 con 0x7f148c1617e0 2024-04-25T09:53:28.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.044+0000 7f14917f9640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f147006b4d0 0x7f147006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.044+0000 7f14917f9640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f147006b4d0 0x7f147006d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1488000f60 tx=0x7f1488007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.048+0000 7f1481ffb640 1 -- 172.21.15.139:0/3332326986 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1484059910 con 0x7f148c1617e0 2024-04-25T09:53:28.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.217+0000 7f1493a5d640 1 -- 172.21.15.139:0/3332326986 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f148c0cd480 con 0x7f148c1617e0 2024-04-25T09:53:28.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.218+0000 7f1481ffb640 1 -- 172.21.15.139:0/3332326986 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f148c0cd480 con 0x7f148c1617e0 2024-04-25T09:53:28.221 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:28.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.220+0000 7f1493a5d640 1 -- 172.21.15.139:0/3332326986 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f147006b4d0 msgr2=0x7f147006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.220+0000 7f1493a5d640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f147006b4d0 0x7f147006d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1488000f60 tx=0x7f1488007c10 comp rx=0 tx=0).stop 2024-04-25T09:53:28.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.220+0000 7f1493a5d640 1 -- 172.21.15.139:0/3332326986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f148c1617e0 msgr2=0x7f148c1605b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.220+0000 7f1493a5d640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f148c1617e0 0x7f148c1605b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1484007e40 tx=0x7f148400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.220+0000 7f1493a5d640 1 -- 172.21.15.139:0/3332326986 shutdown_connections 2024-04-25T09:53:28.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.220+0000 7f1493a5d640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f148c1617e0 0x7f148c1605b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.220+0000 7f1493a5d640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f147006b4d0 0x7f147006d990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.220+0000 7f1493a5d640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f148c15dbf0 0x7f148c160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.220+0000 7f1493a5d640 1 --2- 172.21.15.139:0/3332326986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f148c0cec80 0x7f148c0c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.220+0000 7f1493a5d640 1 -- 172.21.15.139:0/3332326986 >> 172.21.15.139:0/3332326986 conn(0x7f148c0b5480 msgr2=0x7f148c0d2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:28.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.220+0000 7f1493a5d640 1 -- 172.21.15.139:0/3332326986 shutdown_connections 2024-04-25T09:53:28.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.220+0000 7f1493a5d640 1 -- 172.21.15.139:0/3332326986 wait complete. 2024-04-25T09:53:28.234 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:53:28.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.329+0000 7fd457934640 1 Processor -- start 2024-04-25T09:53:28.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.330+0000 7fd457934640 1 -- start start 2024-04-25T09:53:28.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.330+0000 7fd457934640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd45015db40 0x7fd4500b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.330+0000 7fd457934640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd4500b4bc0 0x7fd4500b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.330+0000 7fd457934640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd450070f50 con 0x7fd4500b56b0 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.330+0000 7fd457934640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd450071230 con 0x7fd45015db40 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.330+0000 7fd457934640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd450071510 con 0x7fd4500b4bc0 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.330+0000 7fd4556d0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd45015db40 0x7fd4500b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.330+0000 7fd4556d0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd45015db40 0x7fd4500b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58532/0 (socket says 172.21.15.139:58532) 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.330+0000 7fd4556d0640 1 -- 172.21.15.139:0/1042973734 learned_addr learned my addr 172.21.15.139:0/1042973734 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.331+0000 7fd454ecf640 1 --2- 172.21.15.139:0/1042973734 >> v2:172.21.15.150:3300/0 conn(0x7fd4500b4bc0 0x7fd4500b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.331+0000 7fd4556d0640 1 -- 172.21.15.139:0/1042973734 >> v2:172.21.15.150:3300/0 conn(0x7fd4500b4bc0 msgr2=0x7fd4500b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.331+0000 7fd4556d0640 1 --2- 172.21.15.139:0/1042973734 >> v2:172.21.15.150:3300/0 conn(0x7fd4500b4bc0 0x7fd4500b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.331+0000 7fd4556d0640 1 -- 172.21.15.139:0/1042973734 >> v1:172.21.15.150:6789/0 conn(0x7fd4500b56b0 legacy=0x7fd4501615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.331+0000 7fd4556d0640 1 -- 172.21.15.139:0/1042973734 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4500717f0 con 0x7fd45015db40 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.331+0000 7fd454ecf640 1 --2- 172.21.15.139:0/1042973734 >> v2:172.21.15.150:3300/0 conn(0x7fd4500b4bc0 0x7fd4500b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.331+0000 7fd4556d0640 1 --2- 172.21.15.139:0/1042973734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd45015db40 0x7fd4500b4680 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd45006bb40 tx=0x7fd44c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f667556e9a9a0c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.331+0000 7fd447fff640 1 -- 172.21.15.139:0/1042973734 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c037070 con 0x7fd45015db40 2024-04-25T09:53:28.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.332+0000 7fd447fff640 1 -- 172.21.15.139:0/1042973734 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c032440 con 0x7fd45015db40 2024-04-25T09:53:28.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.332+0000 7fd447fff640 1 -- 172.21.15.139:0/1042973734 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c032810 con 0x7fd45015db40 2024-04-25T09:53:28.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.332+0000 7fd457934640 1 -- 172.21.15.139:0/1042973734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd45015db40 msgr2=0x7fd4500b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.332+0000 7fd457934640 1 --2- 172.21.15.139:0/1042973734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd45015db40 0x7fd4500b4680 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd45006bb40 tx=0x7fd44c02a270 comp rx=0 tx=0).stop 2024-04-25T09:53:28.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.332+0000 7fd457934640 1 -- 172.21.15.139:0/1042973734 shutdown_connections 2024-04-25T09:53:28.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.332+0000 7fd457934640 1 --2- 172.21.15.139:0/1042973734 >> v2:172.21.15.150:3300/0 conn(0x7fd4500b4bc0 0x7fd4500b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.332+0000 7fd457934640 1 --2- 172.21.15.139:0/1042973734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd45015db40 0x7fd4500b4680 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.332+0000 7fd457934640 1 -- 172.21.15.139:0/1042973734 >> 172.21.15.139:0/1042973734 conn(0x7fd45006f460 msgr2=0x7fd45006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.332+0000 7fd457934640 1 -- 172.21.15.139:0/1042973734 shutdown_connections 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.332+0000 7fd457934640 1 -- 172.21.15.139:0/1042973734 wait complete. 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.333+0000 7fd457934640 1 Processor -- start 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.333+0000 7fd457934640 1 -- start start 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd457934640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4500b4bc0 0x7fd4501f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd457934640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4500b56b0 0x7fd4500b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd457934640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd45015db40 0x7fd4501518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd457934640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45006cf60 con 0x7fd4500b4bc0 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd457934640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd450070be0 con 0x7fd4500b56b0 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd457934640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45006d0e0 con 0x7fd45015db40 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd4556d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4500b4bc0 0x7fd4501f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd4556d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4500b4bc0 0x7fd4501f3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58536/0 (socket says 172.21.15.139:58536) 2024-04-25T09:53:28.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd4556d0640 1 -- 172.21.15.139:0/3215121672 learned_addr learned my addr 172.21.15.139:0/3215121672 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd454ecf640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4500b56b0 0x7fd4500b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd455ed1640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd45015db40 0x7fd4501518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd4556d0640 1 -- 172.21.15.139:0/3215121672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd45015db40 msgr2=0x7fd4501518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd4556d0640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd45015db40 0x7fd4501518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd4556d0640 1 -- 172.21.15.139:0/3215121672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4500b56b0 msgr2=0x7fd4500b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd4556d0640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4500b56b0 0x7fd4500b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd4556d0640 1 -- 172.21.15.139:0/3215121672 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4500ce520 con 0x7fd4500b4bc0 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd455ed1640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd45015db40 0x7fd4501518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.334+0000 7fd454ecf640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4500b56b0 0x7fd4500b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.335+0000 7fd4556d0640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4500b4bc0 0x7fd4501f3fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd44c02a2b0 tx=0x7fd44c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.335+0000 7fd445ffb640 1 -- 172.21.15.139:0/3215121672 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c041020 con 0x7fd4500b4bc0 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.335+0000 7fd445ffb640 1 -- 172.21.15.139:0/3215121672 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c0306c0 con 0x7fd4500b4bc0 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.335+0000 7fd457934640 1 -- 172.21.15.139:0/3215121672 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4500ce800 con 0x7fd4500b4bc0 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.335+0000 7fd445ffb640 1 -- 172.21.15.139:0/3215121672 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c037040 con 0x7fd4500b4bc0 2024-04-25T09:53:28.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.335+0000 7fd457934640 1 -- 172.21.15.139:0/3215121672 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4500cedc0 con 0x7fd4500b4bc0 2024-04-25T09:53:28.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.336+0000 7fd457934640 1 -- 172.21.15.139:0/3215121672 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd450074520 con 0x7fd4500b4bc0 2024-04-25T09:53:28.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.336+0000 7fd445ffb640 1 -- 172.21.15.139:0/3215121672 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd44c044020 con 0x7fd4500b4bc0 2024-04-25T09:53:28.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.337+0000 7fd445ffb640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd43406b4d0 0x7fd43406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.337+0000 7fd445ffb640 1 -- 172.21.15.139:0/3215121672 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd44c0aad50 con 0x7fd4500b4bc0 2024-04-25T09:53:28.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.337+0000 7fd454ecf640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd43406b4d0 0x7fd43406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.337+0000 7fd454ecf640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd43406b4d0 0x7fd43406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd450162190 tx=0x7fd440008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.341+0000 7fd445ffb640 1 -- 172.21.15.139:0/3215121672 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd44c0ae030 con 0x7fd4500b4bc0 2024-04-25T09:53:28.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.523+0000 7fd457934640 1 -- 172.21.15.139:0/3215121672 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd4501528d0 con 0x7fd4500b4bc0 2024-04-25T09:53:28.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.524+0000 7fd445ffb640 1 -- 172.21.15.139:0/3215121672 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd44c04b030 con 0x7fd4500b4bc0 2024-04-25T09:53:28.527 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T09:53:28.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.526+0000 7fd457934640 1 -- 172.21.15.139:0/3215121672 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd43406b4d0 msgr2=0x7fd43406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.526+0000 7fd457934640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd43406b4d0 0x7fd43406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd450162190 tx=0x7fd440008040 comp rx=0 tx=0).stop 2024-04-25T09:53:28.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.526+0000 7fd457934640 1 -- 172.21.15.139:0/3215121672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4500b4bc0 msgr2=0x7fd4501f3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.526+0000 7fd457934640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4500b4bc0 0x7fd4501f3fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd44c02a2b0 tx=0x7fd44c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.526+0000 7fd457934640 1 -- 172.21.15.139:0/3215121672 shutdown_connections 2024-04-25T09:53:28.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.526+0000 7fd457934640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd43406b4d0 0x7fd43406d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.526+0000 7fd457934640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd45015db40 0x7fd4501518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.526+0000 7fd457934640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4500b56b0 0x7fd4500b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.526+0000 7fd457934640 1 --2- 172.21.15.139:0/3215121672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4500b4bc0 0x7fd4501f3fe0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.526+0000 7fd457934640 1 -- 172.21.15.139:0/3215121672 >> 172.21.15.139:0/3215121672 conn(0x7fd45006f460 msgr2=0x7fd450073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:28.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.526+0000 7fd457934640 1 -- 172.21.15.139:0/3215121672 shutdown_connections 2024-04-25T09:53:28.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.526+0000 7fd457934640 1 -- 172.21.15.139:0/3215121672 wait complete. 2024-04-25T09:53:28.539 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-04-25T09:53:28.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:28.540 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:53:28.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi139.stderr:ceph-fuse[41688]: fuse finished with error 0 and tester_r 0 2024-04-25T09:53:28.563 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:53:28.578 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:53:28.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:28.578 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T09:53:28.636 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:28.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.770+0000 7fba09b9e640 1 Processor -- start 2024-04-25T09:53:28.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.770+0000 7fba09b9e640 1 -- start start 2024-04-25T09:53:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.771+0000 7fba09b9e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba041611e0 0x7fba040b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.771+0000 7fba09b9e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fba040cea00 0x7fba0406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.771+0000 7fba09b9e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fba040cee60 0x7fba0406c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.771+0000 7fba09b9e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba040d1fa0 con 0x7fba040cea00 2024-04-25T09:53:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.771+0000 7fba09b9e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba040d2560 con 0x7fba040cee60 2024-04-25T09:53:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.771+0000 7fba09b9e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba040d4740 con 0x7fba041611e0 2024-04-25T09:53:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.771+0000 7fb9fbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fba040cea00 0x7fba0406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.771+0000 7fb9fbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fba040cea00 0x7fba0406c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58540/0 (socket says 172.21.15.139:58540) 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.771+0000 7fb9fbfff640 1 -- 172.21.15.139:0/683476449 learned_addr learned my addr 172.21.15.139:0/683476449 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.772+0000 7fba037fe640 1 --2- 172.21.15.139:0/683476449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba041611e0 0x7fba040b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.772+0000 7fba03fff640 1 --2- 172.21.15.139:0/683476449 >> v2:172.21.15.150:3300/0 conn(0x7fba040cee60 0x7fba0406c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.772+0000 7fb9fbfff640 1 -- 172.21.15.139:0/683476449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba041611e0 msgr2=0x7fba040b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.772+0000 7fb9fbfff640 1 --2- 172.21.15.139:0/683476449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba041611e0 0x7fba040b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.772+0000 7fb9fbfff640 1 -- 172.21.15.139:0/683476449 >> v2:172.21.15.150:3300/0 conn(0x7fba040cee60 msgr2=0x7fba0406c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.772+0000 7fb9fbfff640 1 --2- 172.21.15.139:0/683476449 >> v2:172.21.15.150:3300/0 conn(0x7fba040cee60 0x7fba0406c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.772+0000 7fb9fbfff640 1 -- 172.21.15.139:0/683476449 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba040b2770 con 0x7fba040cea00 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.772+0000 7fb9fbfff640 1 --2- 172.21.15.139:0/683476449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba040cea00 0x7fba0406c070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009870 tx=0x7fb9ec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d93ca7e4d62bb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.772+0000 7fba02ffd640 1 -- 172.21.15.139:0/683476449 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec037070 con 0x7fba040cea00 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.772+0000 7fba02ffd640 1 -- 172.21.15.139:0/683476449 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec032440 con 0x7fba040cea00 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.772+0000 7fba02ffd640 1 -- 172.21.15.139:0/683476449 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec032810 con 0x7fba040cea00 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.773+0000 7fba09b9e640 1 -- 172.21.15.139:0/683476449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba040cea00 msgr2=0x7fba0406c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.773+0000 7fba09b9e640 1 --2- 172.21.15.139:0/683476449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba040cea00 0x7fba0406c070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009870 tx=0x7fb9ec02a210 comp rx=0 tx=0).stop 2024-04-25T09:53:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.773+0000 7fba09b9e640 1 -- 172.21.15.139:0/683476449 shutdown_connections 2024-04-25T09:53:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.773+0000 7fba09b9e640 1 --2- 172.21.15.139:0/683476449 >> v2:172.21.15.150:3300/0 conn(0x7fba040cee60 0x7fba0406c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.773+0000 7fba09b9e640 1 --2- 172.21.15.139:0/683476449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba040cea00 0x7fba0406c070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.773+0000 7fba09b9e640 1 --2- 172.21.15.139:0/683476449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba041611e0 0x7fba040b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.773+0000 7fba09b9e640 1 -- 172.21.15.139:0/683476449 >> 172.21.15.139:0/683476449 conn(0x7fba040b5b00 msgr2=0x7fba040d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.773+0000 7fba09b9e640 1 -- 172.21.15.139:0/683476449 shutdown_connections 2024-04-25T09:53:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.773+0000 7fba09b9e640 1 -- 172.21.15.139:0/683476449 wait complete. 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.774+0000 7fba09b9e640 1 Processor -- start 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.774+0000 7fba09b9e640 1 -- start start 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba09b9e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba040cea00 0x7fba040c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba09b9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba040cee60 0x7fba040b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba09b9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba041611e0 0x7fba040b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba09b9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba040d21f0 con 0x7fba041611e0 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba09b9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba040b2d30 con 0x7fba040cee60 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba09b9e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba041615c0 con 0x7fba040cea00 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba03fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba041611e0 0x7fba040b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba03fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba041611e0 0x7fba040b4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58552/0 (socket says 172.21.15.139:58552) 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba03fff640 1 -- 172.21.15.139:0/1463157032 learned_addr learned my addr 172.21.15.139:0/1463157032 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba037fe640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba040cea00 0x7fba040c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fb9fbfff640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba040cee60 0x7fba040b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba03fff640 1 -- 172.21.15.139:0/1463157032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba040cea00 msgr2=0x7fba040c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba03fff640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba040cea00 0x7fba040c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba03fff640 1 -- 172.21.15.139:0/1463157032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba040cee60 msgr2=0x7fba040b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba03fff640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba040cee60 0x7fba040b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fba03fff640 1 -- 172.21.15.139:0/1463157032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba040c7740 con 0x7fba041611e0 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.775+0000 7fb9fbfff640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba040cee60 0x7fba040b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.776+0000 7fba03fff640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba041611e0 0x7fba040b4a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc007e90 tx=0x7fb9fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.776+0000 7fba00ff9640 1 -- 172.21.15.139:0/1463157032 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc014420 con 0x7fba041611e0 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.776+0000 7fba00ff9640 1 -- 172.21.15.139:0/1463157032 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc019070 con 0x7fba041611e0 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.776+0000 7fba09b9e640 1 -- 172.21.15.139:0/1463157032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba040c7a20 con 0x7fba041611e0 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.776+0000 7fba00ff9640 1 -- 172.21.15.139:0/1463157032 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc014820 con 0x7fba041611e0 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.776+0000 7fba09b9e640 1 -- 172.21.15.139:0/1463157032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba040c7fe0 con 0x7fba041611e0 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.777+0000 7fba09b9e640 1 -- 172.21.15.139:0/1463157032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba04074520 con 0x7fba041611e0 2024-04-25T09:53:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.777+0000 7fba00ff9640 1 -- 172.21.15.139:0/1463157032 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9fc0149c0 con 0x7fba041611e0 2024-04-25T09:53:28.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.777+0000 7fba00ff9640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb9e406b4d0 0x7fb9e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:28.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.778+0000 7fba00ff9640 1 -- 172.21.15.139:0/1463157032 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb9fc08aca0 con 0x7fba041611e0 2024-04-25T09:53:28.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.778+0000 7fba037fe640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb9e406b4d0 0x7fb9e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:28.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.778+0000 7fba037fe640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb9e406b4d0 0x7fb9e406d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40079a0 tx=0x7fb9f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:28.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.782+0000 7fba00ff9640 1 -- 172.21.15.139:0/1463157032 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9fc059870 con 0x7fba041611e0 2024-04-25T09:53:28.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.951+0000 7fba09b9e640 1 -- 172.21.15.139:0/1463157032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba0406ceb0 con 0x7fba041611e0 2024-04-25T09:53:28.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.952+0000 7fba00ff9640 1 -- 172.21.15.139:0/1463157032 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7fb9fc059170 con 0x7fba041611e0 2024-04-25T09:53:28.956 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:28.956 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:28.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.954+0000 7fba09b9e640 1 -- 172.21.15.139:0/1463157032 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb9e406b4d0 msgr2=0x7fb9e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.954+0000 7fba09b9e640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb9e406b4d0 0x7fb9e406d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40079a0 tx=0x7fb9f4008040 comp rx=0 tx=0).stop 2024-04-25T09:53:28.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.954+0000 7fba09b9e640 1 -- 172.21.15.139:0/1463157032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba041611e0 msgr2=0x7fba040b4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:28.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.954+0000 7fba09b9e640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba041611e0 0x7fba040b4a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc007e90 tx=0x7fb9fc00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.954+0000 7fba09b9e640 1 -- 172.21.15.139:0/1463157032 shutdown_connections 2024-04-25T09:53:28.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.954+0000 7fba09b9e640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fba041611e0 0x7fba040b4a70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.954+0000 7fba09b9e640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb9e406b4d0 0x7fb9e406d990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.954+0000 7fba09b9e640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba040cee60 0x7fba040b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.954+0000 7fba09b9e640 1 --2- 172.21.15.139:0/1463157032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba040cea00 0x7fba040c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:28.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.954+0000 7fba09b9e640 1 -- 172.21.15.139:0/1463157032 >> 172.21.15.139:0/1463157032 conn(0x7fba040b5b00 msgr2=0x7fba0406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:28.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.954+0000 7fba09b9e640 1 -- 172.21.15.139:0/1463157032 shutdown_connections 2024-04-25T09:53:28.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:28.954+0000 7fba09b9e640 1 -- 172.21.15.139:0/1463157032 wait complete. 2024-04-25T09:53:28.968 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:29.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.061+0000 7f85574cb640 1 Processor -- start 2024-04-25T09:53:29.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.062+0000 7f85574cb640 1 -- start start 2024-04-25T09:53:29.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.062+0000 7f85574cb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f855015efb0 0x7f85500d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.062+0000 7f85574cb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8550099a50 0x7f8550099e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.062+0000 7f85574cb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85500cfe70 con 0x7f85500cf900 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.062+0000 7f85574cb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85500d0a80 con 0x7f855015efb0 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.062+0000 7f85574cb640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855009a3c0 con 0x7f8550099a50 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.063+0000 7f8555267640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f855015efb0 0x7f85500d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.063+0000 7f8554a66640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f85500cf900 0x7f855006c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46416/0 (socket says 172.21.15.139:46416) 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.063+0000 7f8554a66640 1 -- 172.21.15.139:0/1775593792 learned_addr learned my addr 172.21.15.139:0/1775593792 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.063+0000 7f8555a68640 1 --2- 172.21.15.139:0/1775593792 >> v2:172.21.15.150:3300/0 conn(0x7f8550099a50 0x7f8550099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.063+0000 7f8555267640 1 -- 172.21.15.139:0/1775593792 >> v2:172.21.15.150:3300/0 conn(0x7f8550099a50 msgr2=0x7f8550099e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.063+0000 7f8555267640 1 --2- 172.21.15.139:0/1775593792 >> v2:172.21.15.150:3300/0 conn(0x7f8550099a50 0x7f8550099e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.063+0000 7f8555267640 1 -- 172.21.15.139:0/1775593792 >> v1:172.21.15.139:6789/0 conn(0x7f85500cf900 legacy=0x7f855006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.063+0000 7f8547fff640 1 -- 172.21.15.139:0/1775593792 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958913506 0 0) 0x7f85500cfe70 con 0x7f85500cf900 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.063+0000 7f8555267640 1 -- 172.21.15.139:0/1775593792 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85500d0150 con 0x7f855015efb0 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.064+0000 7f8555267640 1 --2- 172.21.15.139:0/1775593792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f855015efb0 0x7f85500d0f10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8550072a90 tx=0x7f854002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b382edb7e5b4d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.064+0000 7f8547fff640 1 -- 172.21.15.139:0/1775593792 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540037070 con 0x7f855015efb0 2024-04-25T09:53:29.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.064+0000 7f8547fff640 1 -- 172.21.15.139:0/1775593792 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8540032440 con 0x7f855015efb0 2024-04-25T09:53:29.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.064+0000 7f8547fff640 1 -- 172.21.15.139:0/1775593792 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854002ae60 con 0x7f855015efb0 2024-04-25T09:53:29.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.064+0000 7f85574cb640 1 -- 172.21.15.139:0/1775593792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f855015efb0 msgr2=0x7f85500d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.064+0000 7f85574cb640 1 --2- 172.21.15.139:0/1775593792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f855015efb0 0x7f85500d0f10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8550072a90 tx=0x7f854002a110 comp rx=0 tx=0).stop 2024-04-25T09:53:29.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.064+0000 7f85574cb640 1 -- 172.21.15.139:0/1775593792 shutdown_connections 2024-04-25T09:53:29.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.064+0000 7f85574cb640 1 --2- 172.21.15.139:0/1775593792 >> v2:172.21.15.150:3300/0 conn(0x7f8550099a50 0x7f8550099e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.064+0000 7f85574cb640 1 --2- 172.21.15.139:0/1775593792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f855015efb0 0x7f85500d0f10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.064+0000 7f85574cb640 1 -- 172.21.15.139:0/1775593792 >> 172.21.15.139:0/1775593792 conn(0x7f85500d1790 msgr2=0x7f85500d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:29.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.064+0000 7f85574cb640 1 -- 172.21.15.139:0/1775593792 shutdown_connections 2024-04-25T09:53:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.064+0000 7f85574cb640 1 -- 172.21.15.139:0/1775593792 wait complete. 2024-04-25T09:53:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.065+0000 7f85574cb640 1 Processor -- start 2024-04-25T09:53:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.065+0000 7f85574cb640 1 -- start start 2024-04-25T09:53:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f85574cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8550099a50 0x7f85500b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f85574cb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85500cf900 0x7f85500b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f85574cb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f855015efb0 0x7f85500b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f85574cb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855006cf50 con 0x7f8550099a50 2024-04-25T09:53:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f85574cb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500d0710 con 0x7f855015efb0 2024-04-25T09:53:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f85574cb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500cbde0 con 0x7f85500cf900 2024-04-25T09:53:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f8555267640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8550099a50 0x7f85500b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f8555267640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8550099a50 0x7f85500b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58576/0 (socket says 172.21.15.139:58576) 2024-04-25T09:53:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f8555267640 1 -- 172.21.15.139:0/1103021048 learned_addr learned my addr 172.21.15.139:0/1103021048 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f8554a66640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85500cf900 0x7f85500b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f8555267640 1 -- 172.21.15.139:0/1103021048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85500cf900 msgr2=0x7f85500b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f8555267640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85500cf900 0x7f85500b59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.066+0000 7f8555a68640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f855015efb0 0x7f85500b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.067+0000 7f8555267640 1 -- 172.21.15.139:0/1103021048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f855015efb0 msgr2=0x7f85500b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.067+0000 7f8555267640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f855015efb0 0x7f85500b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.067+0000 7f8555267640 1 -- 172.21.15.139:0/1103021048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85500b4f50 con 0x7f8550099a50 2024-04-25T09:53:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.067+0000 7f8555267640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8550099a50 0x7f85500b6920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8540000c00 tx=0x7f85400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.067+0000 7f8545ffb640 1 -- 172.21.15.139:0/1103021048 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540037040 con 0x7f8550099a50 2024-04-25T09:53:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.067+0000 7f8545ffb640 1 -- 172.21.15.139:0/1103021048 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8540042070 con 0x7f8550099a50 2024-04-25T09:53:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.067+0000 7f85574cb640 1 -- 172.21.15.139:0/1103021048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85500b5230 con 0x7f8550099a50 2024-04-25T09:53:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.067+0000 7f8545ffb640 1 -- 172.21.15.139:0/1103021048 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540030bd0 con 0x7f8550099a50 2024-04-25T09:53:29.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.067+0000 7f85574cb640 1 -- 172.21.15.139:0/1103021048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85500c5700 con 0x7f8550099a50 2024-04-25T09:53:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.068+0000 7f85574cb640 1 -- 172.21.15.139:0/1103021048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8550074520 con 0x7f8550099a50 2024-04-25T09:53:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.068+0000 7f8545ffb640 1 -- 172.21.15.139:0/1103021048 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8540047460 con 0x7f8550099a50 2024-04-25T09:53:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.069+0000 7f8545ffb640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f853406b4d0 0x7f853406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.069+0000 7f8545ffb640 1 -- 172.21.15.139:0/1103021048 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f85400ab120 con 0x7f8550099a50 2024-04-25T09:53:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.069+0000 7f8554a66640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f853406b4d0 0x7f853406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.069+0000 7f8554a66640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f853406b4d0 0x7f853406d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f854c007fd0 tx=0x7f854c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.073+0000 7f8545ffb640 1 -- 172.21.15.139:0/1103021048 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8540032e20 con 0x7f8550099a50 2024-04-25T09:53:29.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.243+0000 7f85574cb640 1 -- 172.21.15.139:0/1103021048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f855009ae50 con 0x7f8550099a50 2024-04-25T09:53:29.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.245+0000 7f8545ffb640 1 -- 172.21.15.139:0/1103021048 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f8540079c60 con 0x7f8550099a50 2024-04-25T09:53:29.248 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:29.248 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:29.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.247+0000 7f85574cb640 1 -- 172.21.15.139:0/1103021048 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f853406b4d0 msgr2=0x7f853406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.247+0000 7f85574cb640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f853406b4d0 0x7f853406d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f854c007fd0 tx=0x7f854c009040 comp rx=0 tx=0).stop 2024-04-25T09:53:29.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.247+0000 7f85574cb640 1 -- 172.21.15.139:0/1103021048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8550099a50 msgr2=0x7f85500b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.247+0000 7f85574cb640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8550099a50 0x7f85500b6920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8540000c00 tx=0x7f85400046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.247+0000 7f85574cb640 1 -- 172.21.15.139:0/1103021048 shutdown_connections 2024-04-25T09:53:29.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.247+0000 7f85574cb640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f853406b4d0 0x7f853406d990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.247+0000 7f85574cb640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f855015efb0 0x7f85500b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.247+0000 7f85574cb640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85500cf900 0x7f85500b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.247+0000 7f85574cb640 1 --2- 172.21.15.139:0/1103021048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8550099a50 0x7f85500b6920 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.247+0000 7f85574cb640 1 -- 172.21.15.139:0/1103021048 >> 172.21.15.139:0/1103021048 conn(0x7f85500d1790 msgr2=0x7f855015dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:29.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.247+0000 7f85574cb640 1 -- 172.21.15.139:0/1103021048 shutdown_connections 2024-04-25T09:53:29.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.247+0000 7f85574cb640 1 -- 172.21.15.139:0/1103021048 wait complete. 2024-04-25T09:53:29.261 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:53:29.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.358+0000 7f3b38a10640 1 Processor -- start 2024-04-25T09:53:29.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.358+0000 7f3b38a10640 1 -- start start 2024-04-25T09:53:29.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.358+0000 7f3b38a10640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b340af8a0 0x7f3b340b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b38a10640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b340c6cc0 con 0x7f3b340afc80 2024-04-25T09:53:29.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b38a10640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b340c6fa0 con 0x7f3b3406b700 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b38a10640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340c7280 con 0x7f3b340af8a0 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b29d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3b340afc80 0x7f3b340cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46428/0 (socket says 172.21.15.139:46428) 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b29d9b640 1 -- 172.21.15.139:0/936045763 learned_addr learned my addr 172.21.15.139:0/936045763 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b3259c640 1 --2- 172.21.15.139:0/936045763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b340af8a0 0x7f3b340b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373615882 0 0) 0x7f3b340c6cc0 con 0x7f3b340afc80 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b20003520 con 0x7f3b340afc80 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609822738 0 0) 0x7f3b340c6fa0 con 0x7f3b3406b700 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b340c6cc0 con 0x7f3b3406b700 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2104484671 0 0) 0x7f3b20003520 con 0x7f3b340afc80 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b340af8a0 msgr2=0x7f3b340b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 --2- 172.21.15.139:0/936045763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b340af8a0 0x7f3b340b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 >> v1:172.21.15.150:6789/0 conn(0x7f3b3406b700 legacy=0x7f3b3406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b340cd090 con 0x7f3b340afc80 2024-04-25T09:53:29.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f3b1c0015f0 con 0x7f3b340afc80 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 >> v1:172.21.15.139:6789/0 conn(0x7f3b340afc80 legacy=0x7f3b340cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 --2- 172.21.15.139:0/936045763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b340af8a0 0x7f3b20003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 --2- 172.21.15.139:0/936045763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b20004120 0x7f3b20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 --2- 172.21.15.139:0/936045763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20004d40 0x7f3b200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340c7280 con 0x7f3b20004120 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2000bb00 con 0x7f3b20004d40 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2000bd00 con 0x7f3b340af8a0 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b29d9b640 1 --2- 172.21.15.139:0/936045763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b20004120 0x7f3b20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b3259c640 1 --2- 172.21.15.139:0/936045763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b340af8a0 0x7f3b20003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.359+0000 7f3b32d9d640 1 --2- 172.21.15.139:0/936045763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20004d40 0x7f3b200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.360+0000 7f3b29d9b640 1 -- 172.21.15.139:0/936045763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b340af8a0 msgr2=0x7f3b20003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.360+0000 7f3b29d9b640 1 --2- 172.21.15.139:0/936045763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b340af8a0 0x7f3b20003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.360+0000 7f3b29d9b640 1 -- 172.21.15.139:0/936045763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20004d40 msgr2=0x7f3b200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.360+0000 7f3b29d9b640 1 --2- 172.21.15.139:0/936045763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20004d40 0x7f3b200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.360+0000 7f3b29d9b640 1 -- 172.21.15.139:0/936045763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b20003520 con 0x7f3b20004120 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.360+0000 7f3b29d9b640 1 --2- 172.21.15.139:0/936045763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b20004120 0x7f3b20004800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c007f40 tx=0x7f3b20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943cc7d7f3d7b009 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.360+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c007b60 con 0x7f3b20004120 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.361+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c005b90 con 0x7f3b20004120 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.361+0000 7f3b31d9b640 1 -- 172.21.15.139:0/936045763 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c007070 con 0x7f3b20004120 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.361+0000 7f3b38a10640 1 -- 172.21.15.139:0/936045763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b20004120 msgr2=0x7f3b20004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.361+0000 7f3b38a10640 1 --2- 172.21.15.139:0/936045763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b20004120 0x7f3b20004800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c007f40 tx=0x7f3b20005830 comp rx=0 tx=0).stop 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.361+0000 7f3b32d9d640 1 -- 172.21.15.139:0/936045763 reap_dead start 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.361+0000 7f3b38a10640 1 -- 172.21.15.139:0/936045763 shutdown_connections 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.361+0000 7f3b38a10640 1 -- 172.21.15.139:0/936045763 >> 172.21.15.139:0/936045763 conn(0x7f3b340b2230 msgr2=0x7f3b340b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.361+0000 7f3b38a10640 1 -- 172.21.15.139:0/936045763 shutdown_connections 2024-04-25T09:53:29.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.361+0000 7f3b38a10640 1 -- 172.21.15.139:0/936045763 wait complete. 2024-04-25T09:53:29.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.363+0000 7f3b38a10640 1 Processor -- start 2024-04-25T09:53:29.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.363+0000 7f3b38a10640 1 -- start start 2024-04-25T09:53:29.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.363+0000 7f3b38a10640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20004120 0x7f3b3414aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.363+0000 7f3b38a10640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b3415de60 0x7f3b3415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.363+0000 7f3b38a10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b3414d570 0x7f3b3414d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.363+0000 7f3b38a10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3406b3f0 con 0x7f3b3414d570 2024-04-25T09:53:29.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.363+0000 7f3b38a10640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3406b270 con 0x7f3b20004120 2024-04-25T09:53:29.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.363+0000 7f3b38a10640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340b2d50 con 0x7f3b3415de60 2024-04-25T09:53:29.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.363+0000 7f3b32d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b3414d570 0x7f3b3414d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.363+0000 7f3b32d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b3414d570 0x7f3b3414d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58588/0 (socket says 172.21.15.139:58588) 2024-04-25T09:53:29.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.363+0000 7f3b32d9d640 1 -- 172.21.15.139:0/1685468632 learned_addr learned my addr 172.21.15.139:0/1685468632 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:29.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b3259c640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20004120 0x7f3b3414aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b29d9b640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b3415de60 0x7f3b3415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b32d9d640 1 -- 172.21.15.139:0/1685468632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b3415de60 msgr2=0x7f3b3415e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b32d9d640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b3415de60 0x7f3b3415e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b32d9d640 1 -- 172.21.15.139:0/1685468632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20004120 msgr2=0x7f3b3414aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b32d9d640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20004120 0x7f3b3414aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b32d9d640 1 -- 172.21.15.139:0/1685468632 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3414e290 con 0x7f3b3414d570 2024-04-25T09:53:29.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b29d9b640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b3415de60 0x7f3b3415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:29.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b3259c640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20004120 0x7f3b3414aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:29.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b32d9d640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b3414d570 0x7f3b3414d9f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3b3406f060 tx=0x7f3b2c00ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b2b7fe640 1 -- 172.21.15.139:0/1685468632 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c019070 con 0x7f3b3414d570 2024-04-25T09:53:29.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b2b7fe640 1 -- 172.21.15.139:0/1685468632 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c006e20 con 0x7f3b3414d570 2024-04-25T09:53:29.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b38a10640 1 -- 172.21.15.139:0/1685468632 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3414b6e0 con 0x7f3b3414d570 2024-04-25T09:53:29.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b2b7fe640 1 -- 172.21.15.139:0/1685468632 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c003b90 con 0x7f3b3414d570 2024-04-25T09:53:29.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.364+0000 7f3b38a10640 1 -- 172.21.15.139:0/1685468632 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3414df30 con 0x7f3b3414d570 2024-04-25T09:53:29.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.365+0000 7f3b38a10640 1 -- 172.21.15.139:0/1685468632 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b34074520 con 0x7f3b3414d570 2024-04-25T09:53:29.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.365+0000 7f3b2b7fe640 1 -- 172.21.15.139:0/1685468632 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b2c01c070 con 0x7f3b3414d570 2024-04-25T09:53:29.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.366+0000 7f3b2b7fe640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b1406b500 0x7f3b1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.366+0000 7f3b2b7fe640 1 -- 172.21.15.139:0/1685468632 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3b2c0898d0 con 0x7f3b3414d570 2024-04-25T09:53:29.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.366+0000 7f3b3259c640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b1406b500 0x7f3b1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.366+0000 7f3b3259c640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b1406b500 0x7f3b1406d9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b340ccb60 tx=0x7f3b24008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.370+0000 7f3b2b7fe640 1 -- 172.21.15.139:0/1685468632 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b2c058410 con 0x7f3b3414d570 2024-04-25T09:53:29.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.576+0000 7f3b38a10640 1 -- 172.21.15.139:0/1685468632 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3b340b27e0 con 0x7f3b3414d570 2024-04-25T09:53:29.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.760+0000 7f3b2b7fe640 1 -- 172.21.15.139:0/1685468632 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3b2c057d10 con 0x7f3b3414d570 2024-04-25T09:53:29.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.763+0000 7f3b38a10640 1 -- 172.21.15.139:0/1685468632 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b1406b500 msgr2=0x7f3b1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.763+0000 7f3b38a10640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b1406b500 0x7f3b1406d9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b340ccb60 tx=0x7f3b24008040 comp rx=0 tx=0).stop 2024-04-25T09:53:29.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.763+0000 7f3b38a10640 1 -- 172.21.15.139:0/1685468632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b3414d570 msgr2=0x7f3b3414d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.763+0000 7f3b38a10640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b3414d570 0x7f3b3414d9f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3b3406f060 tx=0x7f3b2c00ff70 comp rx=0 tx=0).stop 2024-04-25T09:53:29.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.763+0000 7f3b38a10640 1 -- 172.21.15.139:0/1685468632 shutdown_connections 2024-04-25T09:53:29.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.763+0000 7f3b38a10640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b3414d570 0x7f3b3414d9f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.763+0000 7f3b38a10640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b1406b500 0x7f3b1406d9c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.763+0000 7f3b38a10640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b3415de60 0x7f3b3415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.763+0000 7f3b38a10640 1 --2- 172.21.15.139:0/1685468632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b20004120 0x7f3b3414aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.763+0000 7f3b38a10640 1 -- 172.21.15.139:0/1685468632 >> 172.21.15.139:0/1685468632 conn(0x7f3b340b2230 msgr2=0x7f3b340ae070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:29.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.763+0000 7f3b38a10640 1 -- 172.21.15.139:0/1685468632 shutdown_connections 2024-04-25T09:53:29.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.763+0000 7f3b38a10640 1 -- 172.21.15.139:0/1685468632 wait complete. 2024-04-25T09:53:29.776 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T09:53:29.777 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:53:29.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.874+0000 7f3f66222640 1 Processor -- start 2024-04-25T09:53:29.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.874+0000 7f3f66222640 1 -- start start 2024-04-25T09:53:29.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.874+0000 7f3f66222640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f60072db0 0x7f3f600cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.874+0000 7f3f66222640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f600cdd30 con 0x7f3f60159530 2024-04-25T09:53:29.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.874+0000 7f3f66222640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f600ce010 con 0x7f3f60070930 2024-04-25T09:53:29.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.874+0000 7f3f66222640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f600cdaf0 con 0x7f3f60072db0 2024-04-25T09:53:29.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.875+0000 7f3f5f7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3f60159530 0x7f3f600cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46442/0 (socket says 172.21.15.139:46442) 2024-04-25T09:53:29.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.875+0000 7f3f5f7fe640 1 -- 172.21.15.139:0/2447593386 learned_addr learned my addr 172.21.15.139:0/2447593386 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:29.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.875+0000 7f3f5ffff640 1 --2- 172.21.15.139:0/2447593386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f60072db0 0x7f3f600cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.875+0000 7f3f5effd640 1 -- 172.21.15.139:0/2447593386 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039298512 0 0) 0x7f3f600cdd30 con 0x7f3f60159530 2024-04-25T09:53:29.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.875+0000 7f3f5effd640 1 -- 172.21.15.139:0/2447593386 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f4c003520 con 0x7f3f60159530 2024-04-25T09:53:29.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.875+0000 7f3f5ffff640 1 -- 172.21.15.139:0/2447593386 >> v1:172.21.15.150:6789/0 conn(0x7f3f60070930 legacy=0x7f3f600726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.875+0000 7f3f5ffff640 1 -- 172.21.15.139:0/2447593386 >> v1:172.21.15.139:6789/0 conn(0x7f3f60159530 legacy=0x7f3f600cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.875+0000 7f3f5effd640 1 -- 172.21.15.139:0/2447593386 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819899759 0 0) 0x7f3f600ce010 con 0x7f3f60070930 2024-04-25T09:53:29.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.875+0000 7f3f5ffff640 1 -- 172.21.15.139:0/2447593386 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6006fce0 con 0x7f3f60072db0 2024-04-25T09:53:29.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.876+0000 7f3f5ffff640 1 --2- 172.21.15.139:0/2447593386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f60072db0 0x7f3f600cf6b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009dc0 tx=0x7f3f5802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=673c511a34f171a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.876+0000 7f3f5effd640 1 -- 172.21.15.139:0/2447593386 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58007740 con 0x7f3f60072db0 2024-04-25T09:53:29.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.876+0000 7f3f5effd640 1 -- 172.21.15.139:0/2447593386 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f580078e0 con 0x7f3f60072db0 2024-04-25T09:53:29.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.876+0000 7f3f5effd640 1 -- 172.21.15.139:0/2447593386 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58007c90 con 0x7f3f60072db0 2024-04-25T09:53:29.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.876+0000 7f3f66222640 1 -- 172.21.15.139:0/2447593386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f60072db0 msgr2=0x7f3f600cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.876+0000 7f3f66222640 1 --2- 172.21.15.139:0/2447593386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f60072db0 0x7f3f600cf6b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009dc0 tx=0x7f3f5802a6d0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.877+0000 7f3f66222640 1 -- 172.21.15.139:0/2447593386 shutdown_connections 2024-04-25T09:53:29.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.877+0000 7f3f66222640 1 --2- 172.21.15.139:0/2447593386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f60072db0 0x7f3f600cf6b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.877+0000 7f3f66222640 1 -- 172.21.15.139:0/2447593386 >> 172.21.15.139:0/2447593386 conn(0x7f3f600b23f0 msgr2=0x7f3f600b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:29.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.877+0000 7f3f66222640 1 -- 172.21.15.139:0/2447593386 shutdown_connections 2024-04-25T09:53:29.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.877+0000 7f3f66222640 1 -- 172.21.15.139:0/2447593386 wait complete. 2024-04-25T09:53:29.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.878+0000 7f3f66222640 1 Processor -- start 2024-04-25T09:53:29.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.878+0000 7f3f66222640 1 -- start start 2024-04-25T09:53:29.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.878+0000 7f3f66222640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f60070930 0x7f3f60145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.878+0000 7f3f66222640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f60072db0 0x7f3f601f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.878+0000 7f3f66222640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f60159530 0x7f3f6015d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.878+0000 7f3f66222640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f600704e0 con 0x7f3f60072db0 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.878+0000 7f3f66222640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6006fa90 con 0x7f3f60159530 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.878+0000 7f3f66222640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6015bda0 con 0x7f3f60070930 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.878+0000 7f3f57fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f60072db0 0x7f3f601f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f57fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f60072db0 0x7f3f601f5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58602/0 (socket says 172.21.15.139:58602) 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f57fff640 1 -- 172.21.15.139:0/3263641905 learned_addr learned my addr 172.21.15.139:0/3263641905 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f5f7fe640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f60070930 0x7f3f60145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f5ffff640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f60159530 0x7f3f6015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f57fff640 1 -- 172.21.15.139:0/3263641905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f60070930 msgr2=0x7f3f60145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f57fff640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f60070930 0x7f3f60145460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f57fff640 1 -- 172.21.15.139:0/3263641905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f60159530 msgr2=0x7f3f6015d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f57fff640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f60159530 0x7f3f6015d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:29.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f57fff640 1 -- 172.21.15.139:0/3263641905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f601459a0 con 0x7f3f60072db0 2024-04-25T09:53:29.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f57fff640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f60072db0 0x7f3f601f5020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800c450 tx=0x7f3f4800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f5cff9640 1 -- 172.21.15.139:0/3263641905 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48017070 con 0x7f3f60072db0 2024-04-25T09:53:29.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f5cff9640 1 -- 172.21.15.139:0/3263641905 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48005480 con 0x7f3f60072db0 2024-04-25T09:53:29.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f66222640 1 -- 172.21.15.139:0/3263641905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f60147f00 con 0x7f3f60072db0 2024-04-25T09:53:29.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f5cff9640 1 -- 172.21.15.139:0/3263641905 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48003830 con 0x7f3f60072db0 2024-04-25T09:53:29.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.879+0000 7f3f66222640 1 -- 172.21.15.139:0/3263641905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f601484c0 con 0x7f3f60072db0 2024-04-25T09:53:29.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.880+0000 7f3f66222640 1 -- 172.21.15.139:0/3263641905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f60074520 con 0x7f3f60072db0 2024-04-25T09:53:29.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.881+0000 7f3f5cff9640 1 -- 172.21.15.139:0/3263641905 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f4801a070 con 0x7f3f60072db0 2024-04-25T09:53:29.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.881+0000 7f3f5cff9640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f4006b4d0 0x7f3f4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:29.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.881+0000 7f3f5cff9640 1 -- 172.21.15.139:0/3263641905 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3f48089840 con 0x7f3f60072db0 2024-04-25T09:53:29.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.881+0000 7f3f5f7fe640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f4006b4d0 0x7f3f4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:29.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.882+0000 7f3f5f7fe640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f4006b4d0 0x7f3f4006d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f50007fd0 tx=0x7f3f50009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:29.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:29.885+0000 7f3f5cff9640 1 -- 172.21.15.139:0/3263641905 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f48058380 con 0x7f3f60072db0 2024-04-25T09:53:30.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.092+0000 7f3f66222640 1 -- 172.21.15.139:0/3263641905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3f60072480 con 0x7f3f60072db0 2024-04-25T09:53:30.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.762+0000 7f3f5cff9640 1 -- 172.21.15.139:0/3263641905 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3f4805e080 con 0x7f3f60072db0 2024-04-25T09:53:30.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.764+0000 7f3f66222640 1 -- 172.21.15.139:0/3263641905 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f4006b4d0 msgr2=0x7f3f4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.764+0000 7f3f66222640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f4006b4d0 0x7f3f4006d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f50007fd0 tx=0x7f3f50009040 comp rx=0 tx=0).stop 2024-04-25T09:53:30.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.764+0000 7f3f66222640 1 -- 172.21.15.139:0/3263641905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f60072db0 msgr2=0x7f3f601f5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.764+0000 7f3f66222640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f60072db0 0x7f3f601f5020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800c450 tx=0x7f3f4800c910 comp rx=0 tx=0).stop 2024-04-25T09:53:30.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.764+0000 7f3f66222640 1 -- 172.21.15.139:0/3263641905 shutdown_connections 2024-04-25T09:53:30.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.764+0000 7f3f66222640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f60072db0 0x7f3f601f5020 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.764+0000 7f3f66222640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f4006b4d0 0x7f3f4006d990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.764+0000 7f3f66222640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f60159530 0x7f3f6015d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.764+0000 7f3f66222640 1 --2- 172.21.15.139:0/3263641905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f60070930 0x7f3f60145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.764+0000 7f3f66222640 1 -- 172.21.15.139:0/3263641905 >> 172.21.15.139:0/3263641905 conn(0x7f3f600b23f0 msgr2=0x7f3f60157910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:30.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.764+0000 7f3f66222640 1 -- 172.21.15.139:0/3263641905 shutdown_connections 2024-04-25T09:53:30.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.764+0000 7f3f66222640 1 -- 172.21.15.139:0/3263641905 wait complete. 2024-04-25T09:53:30.780 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:53:30.780 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:30.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.877+0000 7f20c8b22640 1 Processor -- start 2024-04-25T09:53:30.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.877+0000 7f20c8b22640 1 -- start start 2024-04-25T09:53:30.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.878+0000 7f20c8b22640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c4158240 0x7f20c40d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.878+0000 7f20c8b22640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f20c40d1860 0x7f20c40cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.878+0000 7f20c8b22640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c40cbf80 con 0x7f20c40cfba0 2024-04-25T09:53:30.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.878+0000 7f20c8b22640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c4158620 con 0x7f20c40d1860 2024-04-25T09:53:30.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.878+0000 7f20c8b22640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40cce10 con 0x7f20c4158240 2024-04-25T09:53:30.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.878+0000 7f20c1d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f20c40d1860 0x7f20c40cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.878+0000 7f20c2d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f20c40cfba0 0x7f20c41616d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46452/0 (socket says 172.21.15.139:46452) 2024-04-25T09:53:30.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.878+0000 7f20c1d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f20c40d1860 0x7f20c40cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58614/0 (socket says 172.21.15.139:58614) 2024-04-25T09:53:30.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.878+0000 7f20c2d9d640 1 -- 172.21.15.139:0/3006712162 learned_addr learned my addr 172.21.15.139:0/3006712162 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:30.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.879+0000 7f20c259c640 1 --2- 172.21.15.139:0/3006712162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c4158240 0x7f20c40d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.879+0000 7f20c1d9b640 1 -- 172.21.15.139:0/3006712162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c4158240 msgr2=0x7f20c40d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:30.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.879+0000 7f20c1d9b640 1 --2- 172.21.15.139:0/3006712162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c4158240 0x7f20c40d1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.879+0000 7f20c1d9b640 1 -- 172.21.15.139:0/3006712162 >> v1:172.21.15.139:6789/0 conn(0x7f20c40cfba0 legacy=0x7f20c41616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.879+0000 7f20c1d9b640 1 -- 172.21.15.139:0/3006712162 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40ceb50 con 0x7f20c40d1860 2024-04-25T09:53:30.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.879+0000 7f20c159a640 1 -- 172.21.15.139:0/3006712162 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886637525 0 0) 0x7f20bc001110 con 0x7f20c40cfba0 2024-04-25T09:53:30.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.879+0000 7f20c259c640 1 --2- 172.21.15.139:0/3006712162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c4158240 0x7f20c40d1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:30.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.879+0000 7f20c1d9b640 1 --2- 172.21.15.139:0/3006712162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20c40d1860 0x7f20c40cf660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f20b80097a0 tx=0x7f20b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c062445be767a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:30.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.879+0000 7f20c159a640 1 -- 172.21.15.139:0/3006712162 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8037070 con 0x7f20c40d1860 2024-04-25T09:53:30.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.879+0000 7f20c159a640 1 -- 172.21.15.139:0/3006712162 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b8032440 con 0x7f20c40d1860 2024-04-25T09:53:30.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.879+0000 7f20c159a640 1 -- 172.21.15.139:0/3006712162 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8032810 con 0x7f20c40d1860 2024-04-25T09:53:30.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.880+0000 7f20c8b22640 1 -- 172.21.15.139:0/3006712162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20c40d1860 msgr2=0x7f20c40cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.880+0000 7f20c8b22640 1 --2- 172.21.15.139:0/3006712162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20c40d1860 0x7f20c40cf660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f20b80097a0 tx=0x7f20b802a180 comp rx=0 tx=0).stop 2024-04-25T09:53:30.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.880+0000 7f20c8b22640 1 -- 172.21.15.139:0/3006712162 shutdown_connections 2024-04-25T09:53:30.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.880+0000 7f20c8b22640 1 --2- 172.21.15.139:0/3006712162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20c40d1860 0x7f20c40cf660 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.880+0000 7f20c8b22640 1 --2- 172.21.15.139:0/3006712162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c4158240 0x7f20c40d1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.880+0000 7f20c8b22640 1 -- 172.21.15.139:0/3006712162 >> 172.21.15.139:0/3006712162 conn(0x7f20c40b3380 msgr2=0x7f20c40b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:30.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.880+0000 7f20c8b22640 1 -- 172.21.15.139:0/3006712162 shutdown_connections 2024-04-25T09:53:30.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.880+0000 7f20c8b22640 1 -- 172.21.15.139:0/3006712162 wait complete. 2024-04-25T09:53:30.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.881+0000 7f20c8b22640 1 Processor -- start 2024-04-25T09:53:30.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.881+0000 7f20c8b22640 1 -- start start 2024-04-25T09:53:30.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.882+0000 7f20c8b22640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c40cfba0 0x7f20c4156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.882+0000 7f20c8b22640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20c40d1860 0x7f20c40b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.882+0000 7f20c8b22640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c4158240 0x7f20c406ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.882+0000 7f20c8b22640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40cc4e0 con 0x7f20c40d1860 2024-04-25T09:53:30.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.882+0000 7f20c8b22640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40cc200 con 0x7f20c4158240 2024-04-25T09:53:30.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.882+0000 7f20c8b22640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40cc7c0 con 0x7f20c40cfba0 2024-04-25T09:53:30.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.882+0000 7f20c1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20c40d1860 0x7f20c40b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.882+0000 7f20c1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20c40d1860 0x7f20c40b5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58626/0 (socket says 172.21.15.139:58626) 2024-04-25T09:53:30.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.882+0000 7f20c1d9b640 1 -- 172.21.15.139:0/2919894270 learned_addr learned my addr 172.21.15.139:0/2919894270 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:30.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20c1d9b640 1 -- 172.21.15.139:0/2919894270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c40cfba0 msgr2=0x7f20c4156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20c259c640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c40cfba0 0x7f20c4156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20c1d9b640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c40cfba0 0x7f20c4156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20c1d9b640 1 -- 172.21.15.139:0/2919894270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c4158240 msgr2=0x7f20c406ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:30.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20c2d9d640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c4158240 0x7f20c406ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20c1d9b640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c4158240 0x7f20c406ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:30.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20c1d9b640 1 -- 172.21.15.139:0/2919894270 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40b5750 con 0x7f20c40d1860 2024-04-25T09:53:30.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20c1d9b640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20c40d1860 0x7f20c40b5210 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f20b80098c0 tx=0x7f20b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:30.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20b2ffd640 1 -- 172.21.15.139:0/2919894270 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8037040 con 0x7f20c40d1860 2024-04-25T09:53:30.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20b2ffd640 1 -- 172.21.15.139:0/2919894270 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b8030770 con 0x7f20c40d1860 2024-04-25T09:53:30.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20c8b22640 1 -- 172.21.15.139:0/2919894270 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c40c7070 con 0x7f20c40d1860 2024-04-25T09:53:30.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20b2ffd640 1 -- 172.21.15.139:0/2919894270 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8030b40 con 0x7f20c40d1860 2024-04-25T09:53:30.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.883+0000 7f20c8b22640 1 -- 172.21.15.139:0/2919894270 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c40c7630 con 0x7f20c40d1860 2024-04-25T09:53:30.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.884+0000 7f20c8b22640 1 -- 172.21.15.139:0/2919894270 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c4074520 con 0x7f20c40d1860 2024-04-25T09:53:30.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.884+0000 7f20b2ffd640 1 -- 172.21.15.139:0/2919894270 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20b8044020 con 0x7f20c40d1860 2024-04-25T09:53:30.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.885+0000 7f20b2ffd640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f20a406b4d0 0x7f20a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:30.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.885+0000 7f20c259c640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f20a406b4d0 0x7f20a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:30.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.885+0000 7f20b2ffd640 1 -- 172.21.15.139:0/2919894270 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f20b80aab90 con 0x7f20c40d1860 2024-04-25T09:53:30.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.885+0000 7f20c259c640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f20a406b4d0 0x7f20a406d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f20b4008ee0 tx=0x7f20b4008430 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:30.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:30.889+0000 7f20b2ffd640 1 -- 172.21.15.139:0/2919894270 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20c40b5750 con 0x7f20c40d1860 2024-04-25T09:53:31.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.096+0000 7f20c8b22640 1 -- 172.21.15.139:0/2919894270 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20c406e4b0 con 0x7f20c40d1860 2024-04-25T09:53:31.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.097+0000 7f20b2ffd640 1 -- 172.21.15.139:0/2919894270 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f20b8032e20 con 0x7f20c40d1860 2024-04-25T09:53:31.100 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:31.100 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/892226922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":892226922},{"type":"v1","addr":"172.21.15.150:6834","nonce":892226922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/582019435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":582019435},{"type":"v1","addr":"172.21.15.139:6837","nonce":582019435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/2751714076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2751714076},{"type":"v1","addr":"172.21.15.139:6834","nonce":2751714076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:15.042067+0000","modified":"2024-04-25T09:53:30.094430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2160557452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2160557452},{"type":"v1","addr":"172.21.15.150:6835","nonce":2160557452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:53:31.100 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 9 2024-04-25T09:53:31.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.099+0000 7f20c8b22640 1 -- 172.21.15.139:0/2919894270 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f20a406b4d0 msgr2=0x7f20a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.099+0000 7f20c8b22640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f20a406b4d0 0x7f20a406d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f20b4008ee0 tx=0x7f20b4008430 comp rx=0 tx=0).stop 2024-04-25T09:53:31.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.099+0000 7f20c8b22640 1 -- 172.21.15.139:0/2919894270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20c40d1860 msgr2=0x7f20c40b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.099+0000 7f20c8b22640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20c40d1860 0x7f20c40b5210 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f20b80098c0 tx=0x7f20b80046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.099+0000 7f20c8b22640 1 -- 172.21.15.139:0/2919894270 shutdown_connections 2024-04-25T09:53:31.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.099+0000 7f20c8b22640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20c40d1860 0x7f20c40b5210 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.099+0000 7f20c8b22640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f20a406b4d0 0x7f20a406d990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.099+0000 7f20c8b22640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c4158240 0x7f20c406ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.099+0000 7f20c8b22640 1 --2- 172.21.15.139:0/2919894270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20c40cfba0 0x7f20c4156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.099+0000 7f20c8b22640 1 -- 172.21.15.139:0/2919894270 >> 172.21.15.139:0/2919894270 conn(0x7f20c40b3380 msgr2=0x7f20c40cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:31.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.099+0000 7f20c8b22640 1 -- 172.21.15.139:0/2919894270 shutdown_connections 2024-04-25T09:53:31.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.099+0000 7f20c8b22640 1 -- 172.21.15.139:0/2919894270 wait complete. 2024-04-25T09:53:31.113 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:31.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.206+0000 7f285aa9e640 1 Processor -- start 2024-04-25T09:53:31.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.206+0000 7f285aa9e640 1 -- start start 2024-04-25T09:53:31.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.207+0000 7f285aa9e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f285406afc0 0x7f28540d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.207+0000 7f285aa9e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2854070a40 con 0x7f285415b950 2024-04-25T09:53:31.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.207+0000 7f285aa9e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285406fc80 con 0x7f285406ab60 2024-04-25T09:53:31.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.207+0000 7f285aa9e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2854073170 con 0x7f285406afc0 2024-04-25T09:53:31.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.207+0000 7f285883a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f285415b950 0x7f2854072a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46460/0 (socket says 172.21.15.139:46460) 2024-04-25T09:53:31.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.207+0000 7f285883a640 1 -- 172.21.15.139:0/1768903966 learned_addr learned my addr 172.21.15.139:0/1768903966 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.207+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278084489 0 0) 0x7f2854070a40 con 0x7f285415b950 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.207+0000 7f285903b640 1 --2- 172.21.15.139:0/1768903966 >> v2:172.21.15.150:3300/0 conn(0x7f285406afc0 0x7f28540d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.207+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f283c003520 con 0x7f285415b950 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4071412932 0 0) 0x7f283c003520 con 0x7f285415b950 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 >> v2:172.21.15.150:3300/0 conn(0x7f285406afc0 msgr2=0x7f28540d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 --2- 172.21.15.139:0/1768903966 >> v2:172.21.15.150:3300/0 conn(0x7f285406afc0 0x7f28540d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 >> v1:172.21.15.150:6789/0 conn(0x7f285406ab60 legacy=0x7f28540cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285406ff60 con 0x7f285415b950 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f2844001670 con 0x7f285415b950 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 >> v1:172.21.15.139:6789/0 conn(0x7f285415b950 legacy=0x7f2854072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 --2- 172.21.15.139:0/1768903966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f283c0039b0 0x7f283c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 --2- 172.21.15.139:0/1768903966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f283c004800 0x7f283c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 --2- 172.21.15.139:0/1768903966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 0x7f283c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2854073170 con 0x7f283c0051c0 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c00c060 con 0x7f283c004800 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283c00c260 con 0x7f283c0039b0 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f285883a640 1 --2- 172.21.15.139:0/1768903966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f283c0039b0 0x7f283c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f285903b640 1 --2- 172.21.15.139:0/1768903966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 0x7f283c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f2853fff640 1 --2- 172.21.15.139:0/1768903966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f283c004800 0x7f283c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f285903b640 1 -- 172.21.15.139:0/1768903966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f283c0039b0 msgr2=0x7f283c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f285903b640 1 --2- 172.21.15.139:0/1768903966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f283c0039b0 0x7f283c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f285903b640 1 -- 172.21.15.139:0/1768903966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f283c004800 msgr2=0x7f283c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f285903b640 1 --2- 172.21.15.139:0/1768903966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f283c004800 0x7f283c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.208+0000 7f285903b640 1 -- 172.21.15.139:0/1768903966 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c003520 con 0x7f283c0051c0 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.209+0000 7f285903b640 1 -- 172.21.15.139:0/1768903966 reap_dead start 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.209+0000 7f285903b640 1 --2- 172.21.15.139:0/1768903966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 0x7f283c00b9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f284c009fc0 tx=0x7f283c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f8dad86fef6ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.209+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c035020 con 0x7f283c0051c0 2024-04-25T09:53:31.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.209+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c003bd0 con 0x7f283c0051c0 2024-04-25T09:53:31.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.209+0000 7f28537fe640 1 -- 172.21.15.139:0/1768903966 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c009c10 con 0x7f283c0051c0 2024-04-25T09:53:31.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.209+0000 7f285aa9e640 1 -- 172.21.15.139:0/1768903966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 msgr2=0x7f283c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.209+0000 7f285aa9e640 1 --2- 172.21.15.139:0/1768903966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 0x7f283c00b9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f284c009fc0 tx=0x7f283c005830 comp rx=0 tx=0).stop 2024-04-25T09:53:31.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.210+0000 7f285aa9e640 1 -- 172.21.15.139:0/1768903966 shutdown_connections 2024-04-25T09:53:31.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.210+0000 7f285aa9e640 1 --2- 172.21.15.139:0/1768903966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 0x7f283c00b9c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.210+0000 7f285aa9e640 1 -- 172.21.15.139:0/1768903966 >> 172.21.15.139:0/1768903966 conn(0x7f28540b23f0 msgr2=0x7f28540b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:31.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.210+0000 7f285aa9e640 1 -- 172.21.15.139:0/1768903966 shutdown_connections 2024-04-25T09:53:31.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.210+0000 7f285aa9e640 1 -- 172.21.15.139:0/1768903966 wait complete. 2024-04-25T09:53:31.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.211+0000 7f285aa9e640 1 Processor -- start 2024-04-25T09:53:31.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.211+0000 7f285aa9e640 1 -- start start 2024-04-25T09:53:31.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285aa9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 0x7f285415f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285aa9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28541457d0 0x7f2854145c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285aa9e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28540ae710 0x7f28540aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285aa9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28540cdce0 con 0x7f283c0051c0 2024-04-25T09:53:31.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285aa9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28540cda00 con 0x7f28541457d0 2024-04-25T09:53:31.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285aa9e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28540cdfc0 con 0x7f28540ae710 2024-04-25T09:53:31.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285883a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 0x7f285415f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285883a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 0x7f285415f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58644/0 (socket says 172.21.15.139:58644) 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285883a640 1 -- 172.21.15.139:0/3399525873 learned_addr learned my addr 172.21.15.139:0/3399525873 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f2853fff640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28541457d0 0x7f2854145c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285903b640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28540ae710 0x7f28540aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285883a640 1 -- 172.21.15.139:0/3399525873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28540ae710 msgr2=0x7f28540aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285883a640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28540ae710 0x7f28540aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285883a640 1 -- 172.21.15.139:0/3399525873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28541457d0 msgr2=0x7f2854145c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285883a640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28541457d0 0x7f2854145c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285883a640 1 -- 172.21.15.139:0/3399525873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2854146170 con 0x7f283c0051c0 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285883a640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 0x7f285415f510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f28440072f0 tx=0x7f28440076f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f28517fa640 1 -- 172.21.15.139:0/3399525873 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284401a070 con 0x7f283c0051c0 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f28517fa640 1 -- 172.21.15.139:0/3399525873 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284400f940 con 0x7f283c0051c0 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285aa9e640 1 -- 172.21.15.139:0/3399525873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28540a8550 con 0x7f283c0051c0 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f28517fa640 1 -- 172.21.15.139:0/3399525873 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400fd10 con 0x7f283c0051c0 2024-04-25T09:53:31.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.212+0000 7f285aa9e640 1 -- 172.21.15.139:0/3399525873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28540a8b10 con 0x7f283c0051c0 2024-04-25T09:53:31.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.213+0000 7f285aa9e640 1 -- 172.21.15.139:0/3399525873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2854074520 con 0x7f283c0051c0 2024-04-25T09:53:31.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.214+0000 7f28517fa640 1 -- 172.21.15.139:0/3399525873 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f284401e3f0 con 0x7f283c0051c0 2024-04-25T09:53:31.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.214+0000 7f28517fa640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f284006b500 0x7f284006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.214+0000 7f28517fa640 1 -- 172.21.15.139:0/3399525873 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2844089ce0 con 0x7f283c0051c0 2024-04-25T09:53:31.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.214+0000 7f2853fff640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f284006b500 0x7f284006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.215+0000 7f2853fff640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f284006b500 0x7f284006d9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2848000c00 tx=0x7f2848009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.218+0000 7f28517fa640 1 -- 172.21.15.139:0/3399525873 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2844058140 con 0x7f283c0051c0 2024-04-25T09:53:31.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.389+0000 7f285aa9e640 1 -- 172.21.15.139:0/3399525873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28540afd10 con 0x7f283c0051c0 2024-04-25T09:53:31.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.390+0000 7f28517fa640 1 -- 172.21.15.139:0/3399525873 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f284405c1e0 con 0x7f283c0051c0 2024-04-25T09:53:31.393 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:31.394 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:31.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.392+0000 7f285aa9e640 1 -- 172.21.15.139:0/3399525873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f284006b500 msgr2=0x7f284006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.392+0000 7f285aa9e640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f284006b500 0x7f284006d9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2848000c00 tx=0x7f2848009040 comp rx=0 tx=0).stop 2024-04-25T09:53:31.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.392+0000 7f285aa9e640 1 -- 172.21.15.139:0/3399525873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 msgr2=0x7f285415f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.392+0000 7f285aa9e640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 0x7f285415f510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f28440072f0 tx=0x7f28440076f0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.392+0000 7f285aa9e640 1 -- 172.21.15.139:0/3399525873 shutdown_connections 2024-04-25T09:53:31.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.392+0000 7f285aa9e640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f284006b500 0x7f284006d9c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.392+0000 7f285aa9e640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28540ae710 0x7f28540aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.392+0000 7f285aa9e640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28541457d0 0x7f2854145c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.392+0000 7f285aa9e640 1 --2- 172.21.15.139:0/3399525873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f283c0051c0 0x7f285415f510 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.392+0000 7f285aa9e640 1 -- 172.21.15.139:0/3399525873 >> 172.21.15.139:0/3399525873 conn(0x7f28540b23f0 msgr2=0x7f28540b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:31.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.392+0000 7f285aa9e640 1 -- 172.21.15.139:0/3399525873 shutdown_connections 2024-04-25T09:53:31.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.392+0000 7f285aa9e640 1 -- 172.21.15.139:0/3399525873 wait complete. 2024-04-25T09:53:31.407 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:53:31.407 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:53:31.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.501+0000 7f1e51bb7640 1 Processor -- start 2024-04-25T09:53:31.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.501+0000 7f1e51bb7640 1 -- start start 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.502+0000 7f1e51bb7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1e4c1611e0 0x7f1e4c0b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.502+0000 7f1e51bb7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1e4c0cea00 0x7f1e4c06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.502+0000 7f1e51bb7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e4c0cee60 0x7f1e4c06c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.502+0000 7f1e51bb7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4c0d1fa0 con 0x7f1e4c0cea00 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.502+0000 7f1e51bb7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4c0d2560 con 0x7f1e4c1611e0 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.502+0000 7f1e51bb7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c0d4740 con 0x7f1e4c0cee60 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.502+0000 7f1e4affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1e4c0cea00 0x7f1e4c06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.502+0000 7f1e4affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1e4c0cea00 0x7f1e4c06c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58656/0 (socket says 172.21.15.139:58656) 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.502+0000 7f1e4affd640 1 -- 172.21.15.139:0/2464337068 learned_addr learned my addr 172.21.15.139:0/2464337068 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.502+0000 7f1e4b7fe640 1 --2- 172.21.15.139:0/2464337068 >> v2:172.21.15.150:3300/0 conn(0x7f1e4c1611e0 0x7f1e4c0b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.502+0000 7f1e4bfff640 1 --2- 172.21.15.139:0/2464337068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e4c0cee60 0x7f1e4c06c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.502+0000 7f1e4affd640 1 -- 172.21.15.139:0/2464337068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e4c0cee60 msgr2=0x7f1e4c06c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:31.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.503+0000 7f1e4affd640 1 --2- 172.21.15.139:0/2464337068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e4c0cee60 0x7f1e4c06c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.503+0000 7f1e4affd640 1 -- 172.21.15.139:0/2464337068 >> v2:172.21.15.150:3300/0 conn(0x7f1e4c1611e0 msgr2=0x7f1e4c0b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.503+0000 7f1e4affd640 1 --2- 172.21.15.139:0/2464337068 >> v2:172.21.15.150:3300/0 conn(0x7f1e4c1611e0 0x7f1e4c0b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.503+0000 7f1e4affd640 1 -- 172.21.15.139:0/2464337068 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c0b2770 con 0x7f1e4c0cea00 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.503+0000 7f1e4affd640 1 --2- 172.21.15.139:0/2464337068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c0cea00 0x7f1e4c06c070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c08c420 tx=0x7f1e4002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6412dd3894a98aba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.503+0000 7f1e4a7fc640 1 -- 172.21.15.139:0/2464337068 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40037070 con 0x7f1e4c0cea00 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.503+0000 7f1e4a7fc640 1 -- 172.21.15.139:0/2464337068 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e40032440 con 0x7f1e4c0cea00 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.503+0000 7f1e4a7fc640 1 -- 172.21.15.139:0/2464337068 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40032810 con 0x7f1e4c0cea00 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.504+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2464337068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c0cea00 msgr2=0x7f1e4c06c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.504+0000 7f1e51bb7640 1 --2- 172.21.15.139:0/2464337068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c0cea00 0x7f1e4c06c070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c08c420 tx=0x7f1e4002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.504+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2464337068 shutdown_connections 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.504+0000 7f1e51bb7640 1 --2- 172.21.15.139:0/2464337068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e4c0cee60 0x7f1e4c06c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.504+0000 7f1e51bb7640 1 --2- 172.21.15.139:0/2464337068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c0cea00 0x7f1e4c06c070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.504+0000 7f1e51bb7640 1 --2- 172.21.15.139:0/2464337068 >> v2:172.21.15.150:3300/0 conn(0x7f1e4c1611e0 0x7f1e4c0b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.504+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2464337068 >> 172.21.15.139:0/2464337068 conn(0x7f1e4c0b5b00 msgr2=0x7f1e4c0d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:31.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.504+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2464337068 shutdown_connections 2024-04-25T09:53:31.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.504+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2464337068 wait complete. 2024-04-25T09:53:31.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.505+0000 7f1e51bb7640 1 Processor -- start 2024-04-25T09:53:31.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.505+0000 7f1e51bb7640 1 -- start start 2024-04-25T09:53:31.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.506+0000 7f1e51bb7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e4c0cea00 0x7f1e4c14acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.506+0000 7f1e51bb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c0cee60 0x7f1e4c15b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.506+0000 7f1e51bb7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e4c1611e0 0x7f1e4c15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.506+0000 7f1e51bb7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c0d21f0 con 0x7f1e4c0cee60 2024-04-25T09:53:31.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.506+0000 7f1e51bb7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c0b2d30 con 0x7f1e4c1611e0 2024-04-25T09:53:31.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.506+0000 7f1e51bb7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c1615c0 con 0x7f1e4c0cea00 2024-04-25T09:53:31.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.506+0000 7f1e4affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c0cee60 0x7f1e4c15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.506+0000 7f1e4affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c0cee60 0x7f1e4c15b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58664/0 (socket says 172.21.15.139:58664) 2024-04-25T09:53:31.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.506+0000 7f1e4affd640 1 -- 172.21.15.139:0/2043642010 learned_addr learned my addr 172.21.15.139:0/2043642010 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:31.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.506+0000 7f1e4b7fe640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e4c0cea00 0x7f1e4c14acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.506+0000 7f1e4bfff640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e4c1611e0 0x7f1e4c15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.507+0000 7f1e4affd640 1 -- 172.21.15.139:0/2043642010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e4c0cea00 msgr2=0x7f1e4c14acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.507+0000 7f1e4affd640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e4c0cea00 0x7f1e4c14acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.507+0000 7f1e4affd640 1 -- 172.21.15.139:0/2043642010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e4c1611e0 msgr2=0x7f1e4c15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.507+0000 7f1e4affd640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e4c1611e0 0x7f1e4c15bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.507+0000 7f1e4affd640 1 -- 172.21.15.139:0/2043642010 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c1560c0 con 0x7f1e4c0cee60 2024-04-25T09:53:31.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.507+0000 7f1e4affd640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c0cee60 0x7f1e4c15b760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e400097c0 tx=0x7f1e400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.507+0000 7f1e33fff640 1 -- 172.21.15.139:0/2043642010 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40041020 con 0x7f1e4c0cee60 2024-04-25T09:53:31.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.507+0000 7f1e33fff640 1 -- 172.21.15.139:0/2043642010 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4003d620 con 0x7f1e4c0cee60 2024-04-25T09:53:31.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.507+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2043642010 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c1563a0 con 0x7f1e4c0cee60 2024-04-25T09:53:31.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.507+0000 7f1e33fff640 1 -- 172.21.15.139:0/2043642010 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40037040 con 0x7f1e4c0cee60 2024-04-25T09:53:31.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.507+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2043642010 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4c156c40 con 0x7f1e4c0cee60 2024-04-25T09:53:31.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.508+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2043642010 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4c074520 con 0x7f1e4c0cee60 2024-04-25T09:53:31.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.508+0000 7f1e33fff640 1 -- 172.21.15.139:0/2043642010 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e40044020 con 0x7f1e4c0cee60 2024-04-25T09:53:31.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.509+0000 7f1e33fff640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1e3806b4d0 0x7f1e3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.509+0000 7f1e33fff640 1 -- 172.21.15.139:0/2043642010 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1e400aa9a0 con 0x7f1e4c0cee60 2024-04-25T09:53:31.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.509+0000 7f1e4b7fe640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1e3806b4d0 0x7f1e3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.509+0000 7f1e4b7fe640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1e3806b4d0 0x7f1e3806d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0cd790 tx=0x7f1e3c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.513+0000 7f1e33fff640 1 -- 172.21.15.139:0/2043642010 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e4004b030 con 0x7f1e4c0cee60 2024-04-25T09:53:31.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.681+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2043642010 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e4c06ceb0 con 0x7f1e3806b4d0 2024-04-25T09:53:31.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.689+0000 7f1e33fff640 1 -- 172.21.15.139:0/2043642010 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f1e4c06ceb0 con 0x7f1e3806b4d0 2024-04-25T09:53:31.693 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:31.699 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:53:31.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.692+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2043642010 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1e3806b4d0 msgr2=0x7f1e3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.692+0000 7f1e51bb7640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1e3806b4d0 0x7f1e3806d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0cd790 tx=0x7f1e3c009190 comp rx=0 tx=0).stop 2024-04-25T09:53:31.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.692+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2043642010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c0cee60 msgr2=0x7f1e4c15b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.692+0000 7f1e51bb7640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c0cee60 0x7f1e4c15b760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e400097c0 tx=0x7f1e400046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.692+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2043642010 shutdown_connections 2024-04-25T09:53:31.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.692+0000 7f1e51bb7640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e4c0cee60 0x7f1e4c15b760 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.692+0000 7f1e51bb7640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1e3806b4d0 0x7f1e3806d990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.692+0000 7f1e51bb7640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e4c1611e0 0x7f1e4c15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.692+0000 7f1e51bb7640 1 --2- 172.21.15.139:0/2043642010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e4c0cea00 0x7f1e4c14acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.692+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2043642010 >> 172.21.15.139:0/2043642010 conn(0x7f1e4c0b5b00 msgr2=0x7f1e4c0b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:31.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.693+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2043642010 shutdown_connections 2024-04-25T09:53:31.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.693+0000 7f1e51bb7640 1 -- 172.21.15.139:0/2043642010 wait complete. 2024-04-25T09:53:31.707 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-04-25T09:53:30.970463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3284992,"data_stored":1991782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.230503"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302246+0000","last_change":"2024-04-25T09:53:14.712100+0000","last_active":"2024-04-25T09:53:21.302246+0000","last_peered":"2024-04-25T09:53:21.302246+0000","last_clean":"2024-04-25T09:53:21.302246+0000","last_became_active":"2024-04-25T09:53:14.711827+0000","last_became_peered":"2024-04-25T09:53:14.711827+0000","last_unstale":"2024-04-25T09:53:21.302246+0000","last_undegraded":"2024-04-25T09:53:21.302246+0000","last_fullsized":"2024-04-25T09:53:21.302246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:57.295010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302544+0000","last_change":"2024-04-25T09:53:15.723415+0000","last_active":"2024-04-25T09:53:21.302544+0000","last_peered":"2024-04-25T09:53:21.302544+0000","last_clean":"2024-04-25T09:53:21.302544+0000","last_became_active":"2024-04-25T09:53:15.723126+0000","last_became_peered":"2024-04-25T09:53:15.723126+0000","last_unstale":"2024-04-25T09:53:21.302544+0000","last_undegraded":"2024-04-25T09:53:21.302544+0000","last_fullsized":"2024-04-25T09:53:21.302544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:53.040580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303119+0000","last_change":"2024-04-25T09:53:19.938169+0000","last_active":"2024-04-25T09:53:21.303119+0000","last_peered":"2024-04-25T09:53:21.303119+0000","last_clean":"2024-04-25T09:53:21.303119+0000","last_became_active":"2024-04-25T09:53:19.937831+0000","last_became_peered":"2024-04-25T09:53:19.937831+0000","last_unstale":"2024-04-25T09:53:21.303119+0000","last_undegraded":"2024-04-25T09:53:21.303119+0000","last_fullsized":"2024-04-25T09:53:21.303119+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:05.087194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302754+0000","last_change":"2024-04-25T09:53:14.708452+0000","last_active":"2024-04-25T09:53:21.302754+0000","last_peered":"2024-04-25T09:53:21.302754+0000","last_clean":"2024-04-25T09:53:21.302754+0000","last_became_active":"2024-04-25T09:53:14.708168+0000","last_became_peered":"2024-04-25T09:53:14.708168+0000","last_unstale":"2024-04-25T09:53:21.302754+0000","last_undegraded":"2024-04-25T09:53:21.302754+0000","last_fullsized":"2024-04-25T09:53:21.302754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:08.699272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301824+0000","last_change":"2024-04-25T09:53:15.731605+0000","last_active":"2024-04-25T09:53:21.301824+0000","last_peered":"2024-04-25T09:53:21.301824+0000","last_clean":"2024-04-25T09:53:21.301824+0000","last_became_active":"2024-04-25T09:53:15.731254+0000","last_became_peered":"2024-04-25T09:53:15.731254+0000","last_unstale":"2024-04-25T09:53:21.301824+0000","last_undegraded":"2024-04-25T09:53:21.301824+0000","last_fullsized":"2024-04-25T09:53:21.301824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742162+0000","last_change":"2024-04-25T09:53:19.756305+0000","last_active":"2024-04-25T09:53:20.742162+0000","last_peered":"2024-04-25T09:53:20.742162+0000","last_clean":"2024-04-25T09:53:20.742162+0000","last_became_active":"2024-04-25T09:53:19.755760+0000","last_became_peered":"2024-04-25T09:53:19.755760+0000","last_unstale":"2024-04-25T09:53:20.742162+0000","last_undegraded":"2024-04-25T09:53:20.742162+0000","last_fullsized":"2024-04-25T09:53:20.742162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:45.103063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303011+0000","last_change":"2024-04-25T09:53:14.713424+0000","last_active":"2024-04-25T09:53:21.303011+0000","last_peered":"2024-04-25T09:53:21.303011+0000","last_clean":"2024-04-25T09:53:21.303011+0000","last_became_active":"2024-04-25T09:53:14.711564+0000","last_became_peered":"2024-04-25T09:53:14.711564+0000","last_unstale":"2024-04-25T09:53:21.303011+0000","last_undegraded":"2024-04-25T09:53:21.303011+0000","last_fullsized":"2024-04-25T09:53:21.303011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:02.525834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302665+0000","last_change":"2024-04-25T09:53:15.737042+0000","last_active":"2024-04-25T09:53:21.302665+0000","last_peered":"2024-04-25T09:53:21.302665+0000","last_clean":"2024-04-25T09:53:21.302665+0000","last_became_active":"2024-04-25T09:53:15.736770+0000","last_became_peered":"2024-04-25T09:53:15.736770+0000","last_unstale":"2024-04-25T09:53:21.302665+0000","last_undegraded":"2024-04-25T09:53:21.302665+0000","last_fullsized":"2024-04-25T09:53:21.302665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:57.316572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742503+0000","last_change":"2024-04-25T09:53:19.761920+0000","last_active":"2024-04-25T09:53:20.742503+0000","last_peered":"2024-04-25T09:53:20.742503+0000","last_clean":"2024-04-25T09:53:20.742503+0000","last_became_active":"2024-04-25T09:53:19.761083+0000","last_became_peered":"2024-04-25T09:53:19.761083+0000","last_unstale":"2024-04-25T09:53:20.742503+0000","last_undegraded":"2024-04-25T09:53:20.742503+0000","last_fullsized":"2024-04-25T09:53:20.742503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:57.080146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301800+0000","last_change":"2024-04-25T09:53:14.713667+0000","last_active":"2024-04-25T09:53:21.301800+0000","last_peered":"2024-04-25T09:53:21.301800+0000","last_clean":"2024-04-25T09:53:21.301800+0000","last_became_active":"2024-04-25T09:53:14.713327+0000","last_became_peered":"2024-04-25T09:53:14.713327+0000","last_unstale":"2024-04-25T09:53:21.301800+0000","last_undegraded":"2024-04-25T09:53:21.301800+0000","last_fullsized":"2024-04-25T09:53:21.301800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:26.183785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302970+0000","last_change":"2024-04-25T09:53:15.737782+0000","last_active":"2024-04-25T09:53:21.302970+0000","last_peered":"2024-04-25T09:53:21.302970+0000","last_clean":"2024-04-25T09:53:21.302970+0000","last_became_active":"2024-04-25T09:53:15.737531+0000","last_became_peered":"2024-04-25T09:53:15.737531+0000","last_unstale":"2024-04-25T09:53:21.302970+0000","last_undegraded":"2024-04-25T09:53:21.302970+0000","last_fullsized":"2024-04-25T09:53:21.302970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:55.029099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742249+0000","last_change":"2024-04-25T09:53:19.754738+0000","last_active":"2024-04-25T09:53:20.742249+0000","last_peered":"2024-04-25T09:53:20.742249+0000","last_clean":"2024-04-25T09:53:20.742249+0000","last_became_active":"2024-04-25T09:53:19.754248+0000","last_became_peered":"2024-04-25T09:53:19.754248+0000","last_unstale":"2024-04-25T09:53:20.742249+0000","last_undegraded":"2024-04-25T09:53:20.742249+0000","last_fullsized":"2024-04-25T09:53:20.742249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:55.483582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302991+0000","last_change":"2024-04-25T09:53:15.729322+0000","last_active":"2024-04-25T09:53:21.302991+0000","last_peered":"2024-04-25T09:53:21.302991+0000","last_clean":"2024-04-25T09:53:21.302991+0000","last_became_active":"2024-04-25T09:53:15.729037+0000","last_became_peered":"2024-04-25T09:53:15.729037+0000","last_unstale":"2024-04-25T09:53:21.302991+0000","last_undegraded":"2024-04-25T09:53:21.302991+0000","last_fullsized":"2024-04-25T09:53:21.302991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302405+0000","last_change":"2024-04-25T09:53:14.712891+0000","last_active":"2024-04-25T09:53:21.302405+0000","last_peered":"2024-04-25T09:53:21.302405+0000","last_clean":"2024-04-25T09:53:21.302405+0000","last_became_active":"2024-04-25T09:53:14.712375+0000","last_became_peered":"2024-04-25T09:53:14.712375+0000","last_unstale":"2024-04-25T09:53:21.302405+0000","last_undegraded":"2024-04-25T09:53:21.302405+0000","last_fullsized":"2024-04-25T09:53:21.302405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:58.733069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743015+0000","last_change":"2024-04-25T09:53:19.762022+0000","last_active":"2024-04-25T09:53:20.743015+0000","last_peered":"2024-04-25T09:53:20.743015+0000","last_clean":"2024-04-25T09:53:20.743015+0000","last_became_active":"2024-04-25T09:53:19.761624+0000","last_became_peered":"2024-04-25T09:53:19.761624+0000","last_unstale":"2024-04-25T09:53:20.743015+0000","last_undegraded":"2024-04-25T09:53:20.743015+0000","last_fullsized":"2024-04-25T09:53:20.743015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:01.356138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302197+0000","last_change":"2024-04-25T09:53:15.720406+0000","last_active":"2024-04-25T09:53:21.302197+0000","last_peered":"2024-04-25T09:53:21.302197+0000","last_clean":"2024-04-25T09:53:21.302197+0000","last_became_active":"2024-04-25T09:53:15.720140+0000","last_became_peered":"2024-04-25T09:53:15.720140+0000","last_unstale":"2024-04-25T09:53:21.302197+0000","last_undegraded":"2024-04-25T09:53:21.302197+0000","last_fullsized":"2024-04-25T09:53:21.302197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302882+0000","last_change":"2024-04-25T09:53:14.705015+0000","last_active":"2024-04-25T09:53:21.302882+0000","last_peered":"2024-04-25T09:53:21.302882+0000","last_clean":"2024-04-25T09:53:21.302882+0000","last_became_active":"2024-04-25T09:53:14.704705+0000","last_became_peered":"2024-04-25T09:53:14.704705+0000","last_unstale":"2024-04-25T09:53:21.302882+0000","last_undegraded":"2024-04-25T09:53:21.302882+0000","last_fullsized":"2024-04-25T09:53:21.302882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743231+0000","last_change":"2024-04-25T09:53:19.761275+0000","last_active":"2024-04-25T09:53:20.743231+0000","last_peered":"2024-04-25T09:53:20.743231+0000","last_clean":"2024-04-25T09:53:20.743231+0000","last_became_active":"2024-04-25T09:53:19.760859+0000","last_became_peered":"2024-04-25T09:53:19.760859+0000","last_unstale":"2024-04-25T09:53:20.743231+0000","last_undegraded":"2024-04-25T09:53:20.743231+0000","last_fullsized":"2024-04-25T09:53:20.743231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:20.608892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301802+0000","last_change":"2024-04-25T09:53:15.726544+0000","last_active":"2024-04-25T09:53:21.301802+0000","last_peered":"2024-04-25T09:53:21.301802+0000","last_clean":"2024-04-25T09:53:21.301802+0000","last_became_active":"2024-04-25T09:53:15.726267+0000","last_became_peered":"2024-04-25T09:53:15.726267+0000","last_unstale":"2024-04-25T09:53:21.301802+0000","last_undegraded":"2024-04-25T09:53:21.301802+0000","last_fullsized":"2024-04-25T09:53:21.301802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301803+0000","last_change":"2024-04-25T09:53:14.712292+0000","last_active":"2024-04-25T09:53:21.301803+0000","last_peered":"2024-04-25T09:53:21.301803+0000","last_clean":"2024-04-25T09:53:21.301803+0000","last_became_active":"2024-04-25T09:53:14.711702+0000","last_became_peered":"2024-04-25T09:53:14.711702+0000","last_unstale":"2024-04-25T09:53:21.301803+0000","last_undegraded":"2024-04-25T09:53:21.301803+0000","last_fullsized":"2024-04-25T09:53:21.301803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303737+0000","last_change":"2024-04-25T09:53:19.937513+0000","last_active":"2024-04-25T09:53:21.303737+0000","last_peered":"2024-04-25T09:53:21.303737+0000","last_clean":"2024-04-25T09:53:21.303737+0000","last_became_active":"2024-04-25T09:53:19.937076+0000","last_became_peered":"2024-04-25T09:53:19.937076+0000","last_unstale":"2024-04-25T09:53:21.303737+0000","last_undegraded":"2024-04-25T09:53:21.303737+0000","last_fullsized":"2024-04-25T09:53:21.303737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:50.372524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742399+0000","last_change":"2024-04-25T09:53:15.717251+0000","last_active":"2024-04-25T09:53:20.742399+0000","last_peered":"2024-04-25T09:53:20.742399+0000","last_clean":"2024-04-25T09:53:20.742399+0000","last_became_active":"2024-04-25T09:53:15.716981+0000","last_became_peered":"2024-04-25T09:53:15.716981+0000","last_unstale":"2024-04-25T09:53:20.742399+0000","last_undegraded":"2024-04-25T09:53:20.742399+0000","last_fullsized":"2024-04-25T09:53:20.742399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:08.082240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301576+0000","last_change":"2024-04-25T09:53:14.711486+0000","last_active":"2024-04-25T09:53:21.301576+0000","last_peered":"2024-04-25T09:53:21.301576+0000","last_clean":"2024-04-25T09:53:21.301576+0000","last_became_active":"2024-04-25T09:53:14.711090+0000","last_became_peered":"2024-04-25T09:53:14.711090+0000","last_unstale":"2024-04-25T09:53:21.301576+0000","last_undegraded":"2024-04-25T09:53:21.301576+0000","last_fullsized":"2024-04-25T09:53:21.301576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742532+0000","last_change":"2024-04-25T09:53:19.756221+0000","last_active":"2024-04-25T09:53:20.742532+0000","last_peered":"2024-04-25T09:53:20.742532+0000","last_clean":"2024-04-25T09:53:20.742532+0000","last_became_active":"2024-04-25T09:53:19.755565+0000","last_became_peered":"2024-04-25T09:53:19.755565+0000","last_unstale":"2024-04-25T09:53:20.742532+0000","last_undegraded":"2024-04-25T09:53:20.742532+0000","last_fullsized":"2024-04-25T09:53:20.742532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:03.262744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302070+0000","last_change":"2024-04-25T09:53:15.720035+0000","last_active":"2024-04-25T09:53:21.302070+0000","last_peered":"2024-04-25T09:53:21.302070+0000","last_clean":"2024-04-25T09:53:21.302070+0000","last_became_active":"2024-04-25T09:53:15.719506+0000","last_became_peered":"2024-04-25T09:53:15.719506+0000","last_unstale":"2024-04-25T09:53:21.302070+0000","last_undegraded":"2024-04-25T09:53:21.302070+0000","last_fullsized":"2024-04-25T09:53:21.302070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303882+0000","last_change":"2024-04-25T09:53:14.714287+0000","last_active":"2024-04-25T09:53:21.303882+0000","last_peered":"2024-04-25T09:53:21.303882+0000","last_clean":"2024-04-25T09:53:21.303882+0000","last_became_active":"2024-04-25T09:53:14.714026+0000","last_became_peered":"2024-04-25T09:53:14.714026+0000","last_unstale":"2024-04-25T09:53:21.303882+0000","last_undegraded":"2024-04-25T09:53:21.303882+0000","last_fullsized":"2024-04-25T09:53:21.303882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742315+0000","last_change":"2024-04-25T09:53:19.934699+0000","last_active":"2024-04-25T09:53:20.742315+0000","last_peered":"2024-04-25T09:53:20.742315+0000","last_clean":"2024-04-25T09:53:20.742315+0000","last_became_active":"2024-04-25T09:53:19.934296+0000","last_became_peered":"2024-04-25T09:53:19.934296+0000","last_unstale":"2024-04-25T09:53:20.742315+0000","last_undegraded":"2024-04-25T09:53:20.742315+0000","last_fullsized":"2024-04-25T09:53:20.742315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:04.247738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302840+0000","last_change":"2024-04-25T09:53:15.735174+0000","last_active":"2024-04-25T09:53:21.302840+0000","last_peered":"2024-04-25T09:53:21.302840+0000","last_clean":"2024-04-25T09:53:21.302840+0000","last_became_active":"2024-04-25T09:53:15.734811+0000","last_became_peered":"2024-04-25T09:53:15.734811+0000","last_unstale":"2024-04-25T09:53:21.302840+0000","last_undegraded":"2024-04-25T09:53:21.302840+0000","last_fullsized":"2024-04-25T09:53:21.302840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:53.172005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303710+0000","last_change":"2024-04-25T09:53:14.707243+0000","last_active":"2024-04-25T09:53:21.303710+0000","last_peered":"2024-04-25T09:53:21.303710+0000","last_clean":"2024-04-25T09:53:21.303710+0000","last_became_active":"2024-04-25T09:53:14.706608+0000","last_became_peered":"2024-04-25T09:53:14.706608+0000","last_unstale":"2024-04-25T09:53:21.303710+0000","last_undegraded":"2024-04-25T09:53:21.303710+0000","last_fullsized":"2024-04-25T09:53:21.303710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:49.257995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301370+0000","last_change":"2024-04-25T09:53:19.938687+0000","last_active":"2024-04-25T09:53:21.301370+0000","last_peered":"2024-04-25T09:53:21.301370+0000","last_clean":"2024-04-25T09:53:21.301370+0000","last_became_active":"2024-04-25T09:53:19.938312+0000","last_became_peered":"2024-04-25T09:53:19.938312+0000","last_unstale":"2024-04-25T09:53:21.301370+0000","last_undegraded":"2024-04-25T09:53:21.301370+0000","last_fullsized":"2024-04-25T09:53:21.301370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:11.692337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302904+0000","last_change":"2024-04-25T09:53:15.724325+0000","last_active":"2024-04-25T09:53:21.302904+0000","last_peered":"2024-04-25T09:53:21.302904+0000","last_clean":"2024-04-25T09:53:21.302904+0000","last_became_active":"2024-04-25T09:53:15.724024+0000","last_became_peered":"2024-04-25T09:53:15.724024+0000","last_unstale":"2024-04-25T09:53:21.302904+0000","last_undegraded":"2024-04-25T09:53:21.302904+0000","last_fullsized":"2024-04-25T09:53:21.302904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303201+0000","last_change":"2024-04-25T09:53:14.712119+0000","last_active":"2024-04-25T09:53:21.303201+0000","last_peered":"2024-04-25T09:53:21.303201+0000","last_clean":"2024-04-25T09:53:21.303201+0000","last_became_active":"2024-04-25T09:53:14.711462+0000","last_became_peered":"2024-04-25T09:53:14.711462+0000","last_unstale":"2024-04-25T09:53:21.303201+0000","last_undegraded":"2024-04-25T09:53:21.303201+0000","last_fullsized":"2024-04-25T09:53:21.303201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:39.622554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302329+0000","last_change":"2024-04-25T09:53:19.934786+0000","last_active":"2024-04-25T09:53:21.302329+0000","last_peered":"2024-04-25T09:53:21.302329+0000","last_clean":"2024-04-25T09:53:21.302329+0000","last_became_active":"2024-04-25T09:53:19.934417+0000","last_became_peered":"2024-04-25T09:53:19.934417+0000","last_unstale":"2024-04-25T09:53:21.302329+0000","last_undegraded":"2024-04-25T09:53:21.302329+0000","last_fullsized":"2024-04-25T09:53:21.302329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:50.372524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303078+0000","last_change":"2024-04-25T09:53:15.731430+0000","last_active":"2024-04-25T09:53:21.303078+0000","last_peered":"2024-04-25T09:53:21.303078+0000","last_clean":"2024-04-25T09:53:21.303078+0000","last_became_active":"2024-04-25T09:53:15.731152+0000","last_became_peered":"2024-04-25T09:53:15.731152+0000","last_unstale":"2024-04-25T09:53:21.303078+0000","last_undegraded":"2024-04-25T09:53:21.303078+0000","last_fullsized":"2024-04-25T09:53:21.303078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:46.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302987+0000","last_change":"2024-04-25T09:53:14.708418+0000","last_active":"2024-04-25T09:53:21.302987+0000","last_peered":"2024-04-25T09:53:21.302987+0000","last_clean":"2024-04-25T09:53:21.302987+0000","last_became_active":"2024-04-25T09:53:14.708161+0000","last_became_peered":"2024-04-25T09:53:14.708161+0000","last_unstale":"2024-04-25T09:53:21.302987+0000","last_undegraded":"2024-04-25T09:53:21.302987+0000","last_fullsized":"2024-04-25T09:53:21.302987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:58.733069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302535+0000","last_change":"2024-04-25T09:53:19.758749+0000","last_active":"2024-04-25T09:53:21.302535+0000","last_peered":"2024-04-25T09:53:21.302535+0000","last_clean":"2024-04-25T09:53:21.302535+0000","last_became_active":"2024-04-25T09:53:19.757529+0000","last_became_peered":"2024-04-25T09:53:19.757529+0000","last_unstale":"2024-04-25T09:53:21.302535+0000","last_undegraded":"2024-04-25T09:53:21.302535+0000","last_fullsized":"2024-04-25T09:53:21.302535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:06.760703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742905+0000","last_change":"2024-04-25T09:53:15.726160+0000","last_active":"2024-04-25T09:53:20.742905+0000","last_peered":"2024-04-25T09:53:20.742905+0000","last_clean":"2024-04-25T09:53:20.742905+0000","last_became_active":"2024-04-25T09:53:15.725846+0000","last_became_peered":"2024-04-25T09:53:15.725846+0000","last_unstale":"2024-04-25T09:53:20.742905+0000","last_undegraded":"2024-04-25T09:53:20.742905+0000","last_fullsized":"2024-04-25T09:53:20.742905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.262190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302405+0000","last_change":"2024-04-25T09:53:14.712606+0000","last_active":"2024-04-25T09:53:21.302405+0000","last_peered":"2024-04-25T09:53:21.302405+0000","last_clean":"2024-04-25T09:53:21.302405+0000","last_became_active":"2024-04-25T09:53:14.712327+0000","last_became_peered":"2024-04-25T09:53:14.712327+0000","last_unstale":"2024-04-25T09:53:21.302405+0000","last_undegraded":"2024-04-25T09:53:21.302405+0000","last_fullsized":"2024-04-25T09:53:21.302405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:49.257995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302949+0000","last_change":"2024-04-25T09:53:19.757694+0000","last_active":"2024-04-25T09:53:21.302949+0000","last_peered":"2024-04-25T09:53:21.302949+0000","last_clean":"2024-04-25T09:53:21.302949+0000","last_became_active":"2024-04-25T09:53:19.757293+0000","last_became_peered":"2024-04-25T09:53:19.757293+0000","last_unstale":"2024-04-25T09:53:21.302949+0000","last_undegraded":"2024-04-25T09:53:21.302949+0000","last_fullsized":"2024-04-25T09:53:21.302949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:12.794746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302692+0000","last_change":"2024-04-25T09:53:15.725158+0000","last_active":"2024-04-25T09:53:21.302692+0000","last_peered":"2024-04-25T09:53:21.302692+0000","last_clean":"2024-04-25T09:53:21.302692+0000","last_became_active":"2024-04-25T09:53:15.724890+0000","last_became_peered":"2024-04-25T09:53:15.724890+0000","last_unstale":"2024-04-25T09:53:21.302692+0000","last_undegraded":"2024-04-25T09:53:21.302692+0000","last_fullsized":"2024-04-25T09:53:21.302692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741185+0000","last_change":"2024-04-25T09:53:14.703999+0000","last_active":"2024-04-25T09:53:20.741185+0000","last_peered":"2024-04-25T09:53:20.741185+0000","last_clean":"2024-04-25T09:53:20.741185+0000","last_became_active":"2024-04-25T09:53:14.703714+0000","last_became_peered":"2024-04-25T09:53:14.703714+0000","last_unstale":"2024-04-25T09:53:20.741185+0000","last_undegraded":"2024-04-25T09:53:20.741185+0000","last_fullsized":"2024-04-25T09:53:20.741185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303908+0000","last_change":"2024-04-25T09:53:19.756345+0000","last_active":"2024-04-25T09:53:21.303908+0000","last_peered":"2024-04-25T09:53:21.303908+0000","last_clean":"2024-04-25T09:53:21.303908+0000","last_became_active":"2024-04-25T09:53:19.755915+0000","last_became_peered":"2024-04-25T09:53:19.755915+0000","last_unstale":"2024-04-25T09:53:21.303908+0000","last_undegraded":"2024-04-25T09:53:21.303908+0000","last_fullsized":"2024-04-25T09:53:21.303908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:54.084750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301435+0000","last_change":"2024-04-25T09:53:15.723323+0000","last_active":"2024-04-25T09:53:21.301435+0000","last_peered":"2024-04-25T09:53:21.301435+0000","last_clean":"2024-04-25T09:53:21.301435+0000","last_became_active":"2024-04-25T09:53:15.723019+0000","last_became_peered":"2024-04-25T09:53:15.723019+0000","last_unstale":"2024-04-25T09:53:21.301435+0000","last_undegraded":"2024-04-25T09:53:21.301435+0000","last_fullsized":"2024-04-25T09:53:21.301435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:43.951510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301698+0000","last_change":"2024-04-25T09:53:14.712642+0000","last_active":"2024-04-25T09:53:21.301698+0000","last_peered":"2024-04-25T09:53:21.301698+0000","last_clean":"2024-04-25T09:53:21.301698+0000","last_became_active":"2024-04-25T09:53:14.711778+0000","last_became_peered":"2024-04-25T09:53:14.711778+0000","last_unstale":"2024-04-25T09:53:21.301698+0000","last_undegraded":"2024-04-25T09:53:21.301698+0000","last_fullsized":"2024-04-25T09:53:21.301698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:49.257995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301888+0000","last_change":"2024-04-25T09:53:19.934697+0000","last_active":"2024-04-25T09:53:21.301888+0000","last_peered":"2024-04-25T09:53:21.301888+0000","last_clean":"2024-04-25T09:53:21.301888+0000","last_became_active":"2024-04-25T09:53:19.934259+0000","last_became_peered":"2024-04-25T09:53:19.934259+0000","last_unstale":"2024-04-25T09:53:21.301888+0000","last_undegraded":"2024-04-25T09:53:21.301888+0000","last_fullsized":"2024-04-25T09:53:21.301888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:12.794746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301861+0000","last_change":"2024-04-25T09:53:15.731434+0000","last_active":"2024-04-25T09:53:21.301861+0000","last_peered":"2024-04-25T09:53:21.301861+0000","last_clean":"2024-04-25T09:53:21.301861+0000","last_became_active":"2024-04-25T09:53:15.731139+0000","last_became_peered":"2024-04-25T09:53:15.731139+0000","last_unstale":"2024-04-25T09:53:21.301861+0000","last_undegraded":"2024-04-25T09:53:21.301861+0000","last_fullsized":"2024-04-25T09:53:21.301861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302366+0000","last_change":"2024-04-25T09:53:14.706419+0000","last_active":"2024-04-25T09:53:21.302366+0000","last_peered":"2024-04-25T09:53:21.302366+0000","last_clean":"2024-04-25T09:53:21.302366+0000","last_became_active":"2024-04-25T09:53:14.706020+0000","last_became_peered":"2024-04-25T09:53:14.706020+0000","last_unstale":"2024-04-25T09:53:21.302366+0000","last_undegraded":"2024-04-25T09:53:21.302366+0000","last_fullsized":"2024-04-25T09:53:21.302366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301803+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:53:21.301803+0000","last_peered":"2024-04-25T09:53:21.301803+0000","last_clean":"2024-04-25T09:53:21.301803+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:53:21.301803+0000","last_undegraded":"2024-04-25T09:53:21.301803+0000","last_fullsized":"2024-04-25T09:53:21.301803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303580+0000","last_change":"2024-04-25T09:53:19.937575+0000","last_active":"2024-04-25T09:53:21.303580+0000","last_peered":"2024-04-25T09:53:21.303580+0000","last_clean":"2024-04-25T09:53:21.303580+0000","last_became_active":"2024-04-25T09:53:19.937138+0000","last_became_peered":"2024-04-25T09:53:19.937138+0000","last_unstale":"2024-04-25T09:53:21.303580+0000","last_undegraded":"2024-04-25T09:53:21.303580+0000","last_fullsized":"2024-04-25T09:53:21.303580+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:01.356138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302839+0000","last_change":"2024-04-25T09:53:15.733512+0000","last_active":"2024-04-25T09:53:21.302839+0000","last_peered":"2024-04-25T09:53:21.302839+0000","last_clean":"2024-04-25T09:53:21.302839+0000","last_became_active":"2024-04-25T09:53:15.733224+0000","last_became_peered":"2024-04-25T09:53:15.733224+0000","last_unstale":"2024-04-25T09:53:21.302839+0000","last_undegraded":"2024-04-25T09:53:21.302839+0000","last_fullsized":"2024-04-25T09:53:21.302839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:57.316572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741420+0000","last_change":"2024-04-25T09:53:14.708846+0000","last_active":"2024-04-25T09:53:20.741420+0000","last_peered":"2024-04-25T09:53:20.741420+0000","last_clean":"2024-04-25T09:53:20.741420+0000","last_became_active":"2024-04-25T09:53:14.708587+0000","last_became_peered":"2024-04-25T09:53:14.708587+0000","last_unstale":"2024-04-25T09:53:20.741420+0000","last_undegraded":"2024-04-25T09:53:20.741420+0000","last_fullsized":"2024-04-25T09:53:20.741420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302923+0000","last_change":"2024-04-25T09:53:19.934818+0000","last_active":"2024-04-25T09:53:21.302923+0000","last_peered":"2024-04-25T09:53:21.302923+0000","last_clean":"2024-04-25T09:53:21.302923+0000","last_became_active":"2024-04-25T09:53:19.934403+0000","last_became_peered":"2024-04-25T09:53:19.934403+0000","last_unstale":"2024-04-25T09:53:21.302923+0000","last_undegraded":"2024-04-25T09:53:21.302923+0000","last_fullsized":"2024-04-25T09:53:21.302923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:57.211571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302766+0000","last_change":"2024-04-25T09:53:15.732566+0000","last_active":"2024-04-25T09:53:21.302766+0000","last_peered":"2024-04-25T09:53:21.302766+0000","last_clean":"2024-04-25T09:53:21.302766+0000","last_became_active":"2024-04-25T09:53:15.731920+0000","last_became_peered":"2024-04-25T09:53:15.731920+0000","last_unstale":"2024-04-25T09:53:21.302766+0000","last_undegraded":"2024-04-25T09:53:21.302766+0000","last_fullsized":"2024-04-25T09:53:21.302766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:53.172005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302249+0000","last_change":"2024-04-25T09:53:14.703268+0000","last_active":"2024-04-25T09:53:21.302249+0000","last_peered":"2024-04-25T09:53:21.302249+0000","last_clean":"2024-04-25T09:53:21.302249+0000","last_became_active":"2024-04-25T09:53:14.702989+0000","last_became_peered":"2024-04-25T09:53:14.702989+0000","last_unstale":"2024-04-25T09:53:21.302249+0000","last_undegraded":"2024-04-25T09:53:21.302249+0000","last_fullsized":"2024-04-25T09:53:21.302249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302491+0000","last_change":"2024-04-25T09:53:19.762847+0000","last_active":"2024-04-25T09:53:21.302491+0000","last_peered":"2024-04-25T09:53:21.302491+0000","last_clean":"2024-04-25T09:53:21.302491+0000","last_became_active":"2024-04-25T09:53:19.762336+0000","last_became_peered":"2024-04-25T09:53:19.762336+0000","last_unstale":"2024-04-25T09:53:21.302491+0000","last_undegraded":"2024-04-25T09:53:21.302491+0000","last_fullsized":"2024-04-25T09:53:21.302491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:54.084750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303390+0000","last_change":"2024-04-25T09:53:15.732927+0000","last_active":"2024-04-25T09:53:21.303390+0000","last_peered":"2024-04-25T09:53:21.303390+0000","last_clean":"2024-04-25T09:53:21.303390+0000","last_became_active":"2024-04-25T09:53:15.732640+0000","last_became_peered":"2024-04-25T09:53:15.732640+0000","last_unstale":"2024-04-25T09:53:21.303390+0000","last_undegraded":"2024-04-25T09:53:21.303390+0000","last_fullsized":"2024-04-25T09:53:21.303390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:27.493283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741538+0000","last_change":"2024-04-25T09:53:14.708266+0000","last_active":"2024-04-25T09:53:20.741538+0000","last_peered":"2024-04-25T09:53:20.741538+0000","last_clean":"2024-04-25T09:53:20.741538+0000","last_became_active":"2024-04-25T09:53:14.707980+0000","last_became_peered":"2024-04-25T09:53:14.707980+0000","last_unstale":"2024-04-25T09:53:20.741538+0000","last_undegraded":"2024-04-25T09:53:20.741538+0000","last_fullsized":"2024-04-25T09:53:20.741538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301745+0000","last_change":"2024-04-25T09:53:19.934601+0000","last_active":"2024-04-25T09:53:21.301745+0000","last_peered":"2024-04-25T09:53:21.301745+0000","last_clean":"2024-04-25T09:53:21.301745+0000","last_became_active":"2024-04-25T09:53:19.934185+0000","last_became_peered":"2024-04-25T09:53:19.934185+0000","last_unstale":"2024-04-25T09:53:21.301745+0000","last_undegraded":"2024-04-25T09:53:21.301745+0000","last_fullsized":"2024-04-25T09:53:21.301745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:05.087194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302564+0000","last_change":"2024-04-25T09:53:15.732497+0000","last_active":"2024-04-25T09:53:21.302564+0000","last_peered":"2024-04-25T09:53:21.302564+0000","last_clean":"2024-04-25T09:53:21.302564+0000","last_became_active":"2024-04-25T09:53:15.732229+0000","last_became_peered":"2024-04-25T09:53:15.732229+0000","last_unstale":"2024-04-25T09:53:21.302564+0000","last_undegraded":"2024-04-25T09:53:21.302564+0000","last_fullsized":"2024-04-25T09:53:21.302564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:08.755180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742750+0000","last_change":"2024-04-25T09:53:14.708424+0000","last_active":"2024-04-25T09:53:20.742750+0000","last_peered":"2024-04-25T09:53:20.742750+0000","last_clean":"2024-04-25T09:53:20.742750+0000","last_became_active":"2024-04-25T09:53:14.708139+0000","last_became_peered":"2024-04-25T09:53:14.708139+0000","last_unstale":"2024-04-25T09:53:20.742750+0000","last_undegraded":"2024-04-25T09:53:20.742750+0000","last_fullsized":"2024-04-25T09:53:20.742750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:11.395695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742138+0000","last_change":"2024-04-25T09:53:19.753647+0000","last_active":"2024-04-25T09:53:20.742138+0000","last_peered":"2024-04-25T09:53:20.742138+0000","last_clean":"2024-04-25T09:53:20.742138+0000","last_became_active":"2024-04-25T09:53:19.753091+0000","last_became_peered":"2024-04-25T09:53:19.753091+0000","last_unstale":"2024-04-25T09:53:20.742138+0000","last_undegraded":"2024-04-25T09:53:20.742138+0000","last_fullsized":"2024-04-25T09:53:20.742138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:58.778179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302186+0000","last_change":"2024-04-25T09:53:15.731871+0000","last_active":"2024-04-25T09:53:21.302186+0000","last_peered":"2024-04-25T09:53:21.302186+0000","last_clean":"2024-04-25T09:53:21.302186+0000","last_became_active":"2024-04-25T09:53:15.731615+0000","last_became_peered":"2024-04-25T09:53:15.731615+0000","last_unstale":"2024-04-25T09:53:21.302186+0000","last_undegraded":"2024-04-25T09:53:21.302186+0000","last_fullsized":"2024-04-25T09:53:21.302186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:53.172005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741071+0000","last_change":"2024-04-25T09:53:14.701334+0000","last_active":"2024-04-25T09:53:20.741071+0000","last_peered":"2024-04-25T09:53:20.741071+0000","last_clean":"2024-04-25T09:53:20.741071+0000","last_became_active":"2024-04-25T09:53:14.701062+0000","last_became_peered":"2024-04-25T09:53:14.701062+0000","last_unstale":"2024-04-25T09:53:20.741071+0000","last_undegraded":"2024-04-25T09:53:20.741071+0000","last_fullsized":"2024-04-25T09:53:20.741071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301727+0000","last_change":"2024-04-25T09:53:19.758482+0000","last_active":"2024-04-25T09:53:21.301727+0000","last_peered":"2024-04-25T09:53:21.301727+0000","last_clean":"2024-04-25T09:53:21.301727+0000","last_became_active":"2024-04-25T09:53:19.758036+0000","last_became_peered":"2024-04-25T09:53:19.758036+0000","last_unstale":"2024-04-25T09:53:21.301727+0000","last_undegraded":"2024-04-25T09:53:21.301727+0000","last_fullsized":"2024-04-25T09:53:21.301727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.068666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301688+0000","last_change":"2024-04-25T09:53:15.721989+0000","last_active":"2024-04-25T09:53:21.301688+0000","last_peered":"2024-04-25T09:53:21.301688+0000","last_clean":"2024-04-25T09:53:21.301688+0000","last_became_active":"2024-04-25T09:53:15.721636+0000","last_became_peered":"2024-04-25T09:53:15.721636+0000","last_unstale":"2024-04-25T09:53:21.301688+0000","last_undegraded":"2024-04-25T09:53:21.301688+0000","last_fullsized":"2024-04-25T09:53:21.301688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:39.445622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302694+0000","last_change":"2024-04-25T09:53:14.709691+0000","last_active":"2024-04-25T09:53:21.302694+0000","last_peered":"2024-04-25T09:53:21.302694+0000","last_clean":"2024-04-25T09:53:21.302694+0000","last_became_active":"2024-04-25T09:53:14.709373+0000","last_became_peered":"2024-04-25T09:53:14.709373+0000","last_unstale":"2024-04-25T09:53:21.302694+0000","last_undegraded":"2024-04-25T09:53:21.302694+0000","last_fullsized":"2024-04-25T09:53:21.302694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302719+0000","last_change":"2024-04-25T09:53:19.937412+0000","last_active":"2024-04-25T09:53:21.302719+0000","last_peered":"2024-04-25T09:53:21.302719+0000","last_clean":"2024-04-25T09:53:21.302719+0000","last_became_active":"2024-04-25T09:53:19.937068+0000","last_became_peered":"2024-04-25T09:53:19.937068+0000","last_unstale":"2024-04-25T09:53:21.302719+0000","last_undegraded":"2024-04-25T09:53:21.302719+0000","last_fullsized":"2024-04-25T09:53:21.302719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:18.654048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302364+0000","last_change":"2024-04-25T09:53:15.734287+0000","last_active":"2024-04-25T09:53:21.302364+0000","last_peered":"2024-04-25T09:53:21.302364+0000","last_clean":"2024-04-25T09:53:21.302364+0000","last_became_active":"2024-04-25T09:53:15.733835+0000","last_became_peered":"2024-04-25T09:53:15.733835+0000","last_unstale":"2024-04-25T09:53:21.302364+0000","last_undegraded":"2024-04-25T09:53:21.302364+0000","last_fullsized":"2024-04-25T09:53:21.302364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:55.029099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302389+0000","last_change":"2024-04-25T09:53:14.707323+0000","last_active":"2024-04-25T09:53:21.302389+0000","last_peered":"2024-04-25T09:53:21.302389+0000","last_clean":"2024-04-25T09:53:21.302389+0000","last_became_active":"2024-04-25T09:53:14.707061+0000","last_became_peered":"2024-04-25T09:53:14.707061+0000","last_unstale":"2024-04-25T09:53:21.302389+0000","last_undegraded":"2024-04-25T09:53:21.302389+0000","last_fullsized":"2024-04-25T09:53:21.302389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303418+0000","last_change":"2024-04-25T09:53:19.935315+0000","last_active":"2024-04-25T09:53:21.303418+0000","last_peered":"2024-04-25T09:53:21.303418+0000","last_clean":"2024-04-25T09:53:21.303418+0000","last_became_active":"2024-04-25T09:53:19.934800+0000","last_became_peered":"2024-04-25T09:53:19.934800+0000","last_unstale":"2024-04-25T09:53:21.303418+0000","last_undegraded":"2024-04-25T09:53:21.303418+0000","last_fullsized":"2024-04-25T09:53:21.303418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:50.137095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302044+0000","last_change":"2024-04-25T09:53:15.729897+0000","last_active":"2024-04-25T09:53:21.302044+0000","last_peered":"2024-04-25T09:53:21.302044+0000","last_clean":"2024-04-25T09:53:21.302044+0000","last_became_active":"2024-04-25T09:53:15.729269+0000","last_became_peered":"2024-04-25T09:53:15.729269+0000","last_unstale":"2024-04-25T09:53:21.302044+0000","last_undegraded":"2024-04-25T09:53:21.302044+0000","last_fullsized":"2024-04-25T09:53:21.302044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:55.387059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302564+0000","last_change":"2024-04-25T09:53:14.709636+0000","last_active":"2024-04-25T09:53:21.302564+0000","last_peered":"2024-04-25T09:53:21.302564+0000","last_clean":"2024-04-25T09:53:21.302564+0000","last_became_active":"2024-04-25T09:53:14.709366+0000","last_became_peered":"2024-04-25T09:53:14.709366+0000","last_unstale":"2024-04-25T09:53:21.302564+0000","last_undegraded":"2024-04-25T09:53:21.302564+0000","last_fullsized":"2024-04-25T09:53:21.302564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:05.869022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303066+0000","last_change":"2024-04-25T09:53:19.937074+0000","last_active":"2024-04-25T09:53:21.303066+0000","last_peered":"2024-04-25T09:53:21.303066+0000","last_clean":"2024-04-25T09:53:21.303066+0000","last_became_active":"2024-04-25T09:53:19.935313+0000","last_became_peered":"2024-04-25T09:53:19.935313+0000","last_unstale":"2024-04-25T09:53:21.303066+0000","last_undegraded":"2024-04-25T09:53:21.303066+0000","last_fullsized":"2024-04-25T09:53:21.303066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:56.116208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302820+0000","last_change":"2024-04-25T09:53:15.726822+0000","last_active":"2024-04-25T09:53:21.302820+0000","last_peered":"2024-04-25T09:53:21.302820+0000","last_clean":"2024-04-25T09:53:21.302820+0000","last_became_active":"2024-04-25T09:53:15.726526+0000","last_became_peered":"2024-04-25T09:53:15.726526+0000","last_unstale":"2024-04-25T09:53:21.302820+0000","last_undegraded":"2024-04-25T09:53:21.302820+0000","last_fullsized":"2024-04-25T09:53:21.302820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:16.569326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:24.390526+0000","last_change":"2024-04-25T09:53:14.707187+0000","last_active":"2024-04-25T09:53:24.390526+0000","last_peered":"2024-04-25T09:53:24.390526+0000","last_clean":"2024-04-25T09:53:24.390526+0000","last_became_active":"2024-04-25T09:53:14.706509+0000","last_became_peered":"2024-04-25T09:53:14.706509+0000","last_unstale":"2024-04-25T09:53:24.390526+0000","last_undegraded":"2024-04-25T09:53:24.390526+0000","last_fullsized":"2024-04-25T09:53:24.390526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302676+0000","last_change":"2024-04-25T09:53:19.758453+0000","last_active":"2024-04-25T09:53:21.302676+0000","last_peered":"2024-04-25T09:53:21.302676+0000","last_clean":"2024-04-25T09:53:21.302676+0000","last_became_active":"2024-04-25T09:53:19.757004+0000","last_became_peered":"2024-04-25T09:53:19.757004+0000","last_unstale":"2024-04-25T09:53:21.302676+0000","last_undegraded":"2024-04-25T09:53:21.302676+0000","last_fullsized":"2024-04-25T09:53:21.302676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.068666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743281+0000","last_change":"2024-04-25T09:53:15.731519+0000","last_active":"2024-04-25T09:53:20.743281+0000","last_peered":"2024-04-25T09:53:20.743281+0000","last_clean":"2024-04-25T09:53:20.743281+0000","last_became_active":"2024-04-25T09:53:15.731037+0000","last_became_peered":"2024-04-25T09:53:15.731037+0000","last_unstale":"2024-04-25T09:53:20.743281+0000","last_undegraded":"2024-04-25T09:53:20.743281+0000","last_fullsized":"2024-04-25T09:53:20.743281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.530029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302151+0000","last_change":"2024-04-25T09:53:14.713398+0000","last_active":"2024-04-25T09:53:21.302151+0000","last_peered":"2024-04-25T09:53:21.302151+0000","last_clean":"2024-04-25T09:53:21.302151+0000","last_became_active":"2024-04-25T09:53:14.713125+0000","last_became_peered":"2024-04-25T09:53:14.713125+0000","last_unstale":"2024-04-25T09:53:21.302151+0000","last_undegraded":"2024-04-25T09:53:21.302151+0000","last_fullsized":"2024-04-25T09:53:21.302151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742729+0000","last_change":"2024-04-25T09:53:19.762998+0000","last_active":"2024-04-25T09:53:20.742729+0000","last_peered":"2024-04-25T09:53:20.742729+0000","last_clean":"2024-04-25T09:53:20.742729+0000","last_became_active":"2024-04-25T09:53:19.762550+0000","last_became_peered":"2024-04-25T09:53:19.762550+0000","last_unstale":"2024-04-25T09:53:20.742729+0000","last_undegraded":"2024-04-25T09:53:20.742729+0000","last_fullsized":"2024-04-25T09:53:20.742729+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:57.211571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302003+0000","last_change":"2024-04-25T09:53:15.731707+0000","last_active":"2024-04-25T09:53:21.302003+0000","last_peered":"2024-04-25T09:53:21.302003+0000","last_clean":"2024-04-25T09:53:21.302003+0000","last_became_active":"2024-04-25T09:53:15.731303+0000","last_became_peered":"2024-04-25T09:53:15.731303+0000","last_unstale":"2024-04-25T09:53:21.302003+0000","last_undegraded":"2024-04-25T09:53:21.302003+0000","last_fullsized":"2024-04-25T09:53:21.302003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302319+0000","last_change":"2024-04-25T09:53:14.714447+0000","last_active":"2024-04-25T09:53:21.302319+0000","last_peered":"2024-04-25T09:53:21.302319+0000","last_clean":"2024-04-25T09:53:21.302319+0000","last_became_active":"2024-04-25T09:53:14.714155+0000","last_became_peered":"2024-04-25T09:53:14.714155+0000","last_unstale":"2024-04-25T09:53:21.302319+0000","last_undegraded":"2024-04-25T09:53:21.302319+0000","last_fullsized":"2024-04-25T09:53:21.302319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301664+0000","last_change":"2024-04-25T09:53:19.761513+0000","last_active":"2024-04-25T09:53:21.301664+0000","last_peered":"2024-04-25T09:53:21.301664+0000","last_clean":"2024-04-25T09:53:21.301664+0000","last_became_active":"2024-04-25T09:53:19.761037+0000","last_became_peered":"2024-04-25T09:53:19.761037+0000","last_unstale":"2024-04-25T09:53:21.301664+0000","last_undegraded":"2024-04-25T09:53:21.301664+0000","last_fullsized":"2024-04-25T09:53:21.301664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:31.532850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302218+0000","last_change":"2024-04-25T09:53:15.742482+0000","last_active":"2024-04-25T09:53:21.302218+0000","last_peered":"2024-04-25T09:53:21.302218+0000","last_clean":"2024-04-25T09:53:21.302218+0000","last_became_active":"2024-04-25T09:53:15.742170+0000","last_became_peered":"2024-04-25T09:53:15.742170+0000","last_unstale":"2024-04-25T09:53:21.302218+0000","last_undegraded":"2024-04-25T09:53:21.302218+0000","last_fullsized":"2024-04-25T09:53:21.302218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:16.569326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303779+0000","last_change":"2024-04-25T09:53:14.714376+0000","last_active":"2024-04-25T09:53:21.303779+0000","last_peered":"2024-04-25T09:53:21.303779+0000","last_clean":"2024-04-25T09:53:21.303779+0000","last_became_active":"2024-04-25T09:53:14.714116+0000","last_became_peered":"2024-04-25T09:53:14.714116+0000","last_unstale":"2024-04-25T09:53:21.303779+0000","last_undegraded":"2024-04-25T09:53:21.303779+0000","last_fullsized":"2024-04-25T09:53:21.303779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303146+0000","last_change":"2024-04-25T09:53:19.938076+0000","last_active":"2024-04-25T09:53:21.303146+0000","last_peered":"2024-04-25T09:53:21.303146+0000","last_clean":"2024-04-25T09:53:21.303146+0000","last_became_active":"2024-04-25T09:53:19.937704+0000","last_became_peered":"2024-04-25T09:53:19.937704+0000","last_unstale":"2024-04-25T09:53:21.303146+0000","last_undegraded":"2024-04-25T09:53:21.303146+0000","last_fullsized":"2024-04-25T09:53:21.303146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:56.116208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301567+0000","last_change":"2024-04-25T09:53:19.936514+0000","last_active":"2024-04-25T09:53:21.301567+0000","last_peered":"2024-04-25T09:53:21.301567+0000","last_clean":"2024-04-25T09:53:21.301567+0000","last_became_active":"2024-04-25T09:53:19.936095+0000","last_became_peered":"2024-04-25T09:53:19.936095+0000","last_unstale":"2024-04-25T09:53:21.301567+0000","last_undegraded":"2024-04-25T09:53:21.301567+0000","last_fullsized":"2024-04-25T09:53:21.301567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:06.760703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301623+0000","last_change":"2024-04-25T09:53:15.726524+0000","last_active":"2024-04-25T09:53:21.301623+0000","last_peered":"2024-04-25T09:53:21.301623+0000","last_clean":"2024-04-25T09:53:21.301623+0000","last_became_active":"2024-04-25T09:53:15.726160+0000","last_became_peered":"2024-04-25T09:53:15.726160+0000","last_unstale":"2024-04-25T09:53:21.301623+0000","last_undegraded":"2024-04-25T09:53:21.301623+0000","last_fullsized":"2024-04-25T09:53:21.301623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302003+0000","last_change":"2024-04-25T09:53:14.704589+0000","last_active":"2024-04-25T09:53:21.302003+0000","last_peered":"2024-04-25T09:53:21.302003+0000","last_clean":"2024-04-25T09:53:21.302003+0000","last_became_active":"2024-04-25T09:53:14.704228+0000","last_became_peered":"2024-04-25T09:53:14.704228+0000","last_unstale":"2024-04-25T09:53:21.302003+0000","last_undegraded":"2024-04-25T09:53:21.302003+0000","last_fullsized":"2024-04-25T09:53:21.302003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303831+0000","last_change":"2024-04-25T09:53:19.761677+0000","last_active":"2024-04-25T09:53:21.303831+0000","last_peered":"2024-04-25T09:53:21.303831+0000","last_clean":"2024-04-25T09:53:21.303831+0000","last_became_active":"2024-04-25T09:53:19.761276+0000","last_became_peered":"2024-04-25T09:53:19.761276+0000","last_unstale":"2024-04-25T09:53:21.303831+0000","last_undegraded":"2024-04-25T09:53:21.303831+0000","last_fullsized":"2024-04-25T09:53:21.303831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:20.608892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301714+0000","last_change":"2024-04-25T09:53:15.722486+0000","last_active":"2024-04-25T09:53:21.301714+0000","last_peered":"2024-04-25T09:53:21.301714+0000","last_clean":"2024-04-25T09:53:21.301714+0000","last_became_active":"2024-04-25T09:53:15.722225+0000","last_became_peered":"2024-04-25T09:53:15.722225+0000","last_unstale":"2024-04-25T09:53:21.301714+0000","last_undegraded":"2024-04-25T09:53:21.301714+0000","last_fullsized":"2024-04-25T09:53:21.301714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303275+0000","last_change":"2024-04-25T09:53:14.707399+0000","last_active":"2024-04-25T09:53:21.303275+0000","last_peered":"2024-04-25T09:53:21.303275+0000","last_clean":"2024-04-25T09:53:21.303275+0000","last_became_active":"2024-04-25T09:53:14.707078+0000","last_became_peered":"2024-04-25T09:53:14.707078+0000","last_unstale":"2024-04-25T09:53:21.303275+0000","last_undegraded":"2024-04-25T09:53:21.303275+0000","last_fullsized":"2024-04-25T09:53:21.303275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:57.295010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303180+0000","last_change":"2024-04-25T09:53:14.711893+0000","last_active":"2024-04-25T09:53:21.303180+0000","last_peered":"2024-04-25T09:53:21.303180+0000","last_clean":"2024-04-25T09:53:21.303180+0000","last_became_active":"2024-04-25T09:53:14.711513+0000","last_became_peered":"2024-04-25T09:53:14.711513+0000","last_unstale":"2024-04-25T09:53:21.303180+0000","last_undegraded":"2024-04-25T09:53:21.303180+0000","last_fullsized":"2024-04-25T09:53:21.303180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302544+0000","last_change":"2024-04-25T09:53:15.728536+0000","last_active":"2024-04-25T09:53:21.302544+0000","last_peered":"2024-04-25T09:53:21.302544+0000","last_clean":"2024-04-25T09:53:21.302544+0000","last_became_active":"2024-04-25T09:53:15.728235+0000","last_became_peered":"2024-04-25T09:53:15.728235+0000","last_unstale":"2024-04-25T09:53:21.302544+0000","last_undegraded":"2024-04-25T09:53:21.302544+0000","last_fullsized":"2024-04-25T09:53:21.302544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741367+0000","last_change":"2024-04-25T09:53:19.764626+0000","last_active":"2024-04-25T09:53:20.741367+0000","last_peered":"2024-04-25T09:53:20.741367+0000","last_clean":"2024-04-25T09:53:20.741367+0000","last_became_active":"2024-04-25T09:53:19.763954+0000","last_became_peered":"2024-04-25T09:53:19.763954+0000","last_unstale":"2024-04-25T09:53:20.741367+0000","last_undegraded":"2024-04-25T09:53:20.741367+0000","last_fullsized":"2024-04-25T09:53:20.741367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:47.991077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302285+0000","last_change":"2024-04-25T09:53:14.712000+0000","last_active":"2024-04-25T09:53:21.302285+0000","last_peered":"2024-04-25T09:53:21.302285+0000","last_clean":"2024-04-25T09:53:21.302285+0000","last_became_active":"2024-04-25T09:53:14.711712+0000","last_became_peered":"2024-04-25T09:53:14.711712+0000","last_unstale":"2024-04-25T09:53:21.302285+0000","last_undegraded":"2024-04-25T09:53:21.302285+0000","last_fullsized":"2024-04-25T09:53:21.302285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742400+0000","last_change":"2024-04-25T09:53:15.722357+0000","last_active":"2024-04-25T09:53:20.742400+0000","last_peered":"2024-04-25T09:53:20.742400+0000","last_clean":"2024-04-25T09:53:20.742400+0000","last_became_active":"2024-04-25T09:53:15.722082+0000","last_became_peered":"2024-04-25T09:53:15.722082+0000","last_unstale":"2024-04-25T09:53:20.742400+0000","last_undegraded":"2024-04-25T09:53:20.742400+0000","last_fullsized":"2024-04-25T09:53:20.742400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:39.270721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303146+0000","last_change":"2024-04-25T09:53:19.759080+0000","last_active":"2024-04-25T09:53:21.303146+0000","last_peered":"2024-04-25T09:53:21.303146+0000","last_clean":"2024-04-25T09:53:21.303146+0000","last_became_active":"2024-04-25T09:53:19.758705+0000","last_became_peered":"2024-04-25T09:53:19.758705+0000","last_unstale":"2024-04-25T09:53:21.303146+0000","last_undegraded":"2024-04-25T09:53:21.303146+0000","last_fullsized":"2024-04-25T09:53:21.303146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:54.228875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301484+0000","last_change":"2024-04-25T09:53:15.723965+0000","last_active":"2024-04-25T09:53:21.301484+0000","last_peered":"2024-04-25T09:53:21.301484+0000","last_clean":"2024-04-25T09:53:21.301484+0000","last_became_active":"2024-04-25T09:53:15.723660+0000","last_became_peered":"2024-04-25T09:53:15.723660+0000","last_unstale":"2024-04-25T09:53:21.301484+0000","last_undegraded":"2024-04-25T09:53:21.301484+0000","last_fullsized":"2024-04-25T09:53:21.301484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741222+0000","last_change":"2024-04-25T09:53:15.734494+0000","last_active":"2024-04-25T09:53:20.741222+0000","last_peered":"2024-04-25T09:53:20.741222+0000","last_clean":"2024-04-25T09:53:20.741222+0000","last_became_active":"2024-04-25T09:53:15.734204+0000","last_became_peered":"2024-04-25T09:53:15.734204+0000","last_unstale":"2024-04-25T09:53:20.741222+0000","last_undegraded":"2024-04-25T09:53:20.741222+0000","last_fullsized":"2024-04-25T09:53:20.741222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.530029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302296+0000","last_change":"2024-04-25T09:53:15.731317+0000","last_active":"2024-04-25T09:53:21.302296+0000","last_peered":"2024-04-25T09:53:21.302296+0000","last_clean":"2024-04-25T09:53:21.302296+0000","last_became_active":"2024-04-25T09:53:15.731022+0000","last_became_peered":"2024-04-25T09:53:15.731022+0000","last_unstale":"2024-04-25T09:53:21.302296+0000","last_undegraded":"2024-04-25T09:53:21.302296+0000","last_fullsized":"2024-04-25T09:53:21.302296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:02.721136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302472+0000","last_change":"2024-04-25T09:53:15.720872+0000","last_active":"2024-04-25T09:53:21.302472+0000","last_peered":"2024-04-25T09:53:21.302472+0000","last_clean":"2024-04-25T09:53:21.302472+0000","last_became_active":"2024-04-25T09:53:15.720242+0000","last_became_peered":"2024-04-25T09:53:15.720242+0000","last_unstale":"2024-04-25T09:53:21.302472+0000","last_undegraded":"2024-04-25T09:53:21.302472+0000","last_fullsized":"2024-04-25T09:53:21.302472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303531+0000","last_change":"2024-04-25T09:53:15.728500+0000","last_active":"2024-04-25T09:53:21.303531+0000","last_peered":"2024-04-25T09:53:21.303531+0000","last_clean":"2024-04-25T09:53:21.303531+0000","last_became_active":"2024-04-25T09:53:15.726998+0000","last_became_peered":"2024-04-25T09:53:15.726998+0000","last_unstale":"2024-04-25T09:53:21.303531+0000","last_undegraded":"2024-04-25T09:53:21.303531+0000","last_fullsized":"2024-04-25T09:53:21.303531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:00.208172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301991+0000","last_change":"2024-04-25T09:53:15.726337+0000","last_active":"2024-04-25T09:53:21.301991+0000","last_peered":"2024-04-25T09:53:21.301991+0000","last_clean":"2024-04-25T09:53:21.301991+0000","last_became_active":"2024-04-25T09:53:15.726072+0000","last_became_peered":"2024-04-25T09:53:15.726072+0000","last_unstale":"2024-04-25T09:53:21.301991+0000","last_undegraded":"2024-04-25T09:53:21.301991+0000","last_fullsized":"2024-04-25T09:53:21.301991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303810+0000","last_change":"2024-04-25T09:53:15.735200+0000","last_active":"2024-04-25T09:53:21.303810+0000","last_peered":"2024-04-25T09:53:21.303810+0000","last_clean":"2024-04-25T09:53:21.303810+0000","last_became_active":"2024-04-25T09:53:15.734903+0000","last_became_peered":"2024-04-25T09:53:15.734903+0000","last_unstale":"2024-04-25T09:53:21.303810+0000","last_undegraded":"2024-04-25T09:53:21.303810+0000","last_fullsized":"2024-04-25T09:53:21.303810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302760+0000","last_change":"2024-04-25T09:53:15.726664+0000","last_active":"2024-04-25T09:53:21.302760+0000","last_peered":"2024-04-25T09:53:21.302760+0000","last_clean":"2024-04-25T09:53:21.302760+0000","last_became_active":"2024-04-25T09:53:15.726362+0000","last_became_peered":"2024-04-25T09:53:15.726362+0000","last_unstale":"2024-04-25T09:53:21.302760+0000","last_undegraded":"2024-04-25T09:53:21.302760+0000","last_fullsized":"2024-04-25T09:53:21.302760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:59.223177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301824+0000","last_change":"2024-04-25T09:53:15.731610+0000","last_active":"2024-04-25T09:53:21.301824+0000","last_peered":"2024-04-25T09:53:21.301824+0000","last_clean":"2024-04-25T09:53:21.301824+0000","last_became_active":"2024-04-25T09:53:15.731289+0000","last_became_peered":"2024-04-25T09:53:15.731289+0000","last_unstale":"2024-04-25T09:53:21.301824+0000","last_undegraded":"2024-04-25T09:53:21.301824+0000","last_fullsized":"2024-04-25T09:53:21.301824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303221+0000","last_change":"2024-04-25T09:53:15.726434+0000","last_active":"2024-04-25T09:53:21.303221+0000","last_peered":"2024-04-25T09:53:21.303221+0000","last_clean":"2024-04-25T09:53:21.303221+0000","last_became_active":"2024-04-25T09:53:15.726160+0000","last_became_peered":"2024-04-25T09:53:15.726160+0000","last_unstale":"2024-04-25T09:53:21.303221+0000","last_undegraded":"2024-04-25T09:53:21.303221+0000","last_fullsized":"2024-04-25T09:53:21.303221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:39.445622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302185+0000","last_change":"2024-04-25T09:53:15.742414+0000","last_active":"2024-04-25T09:53:21.302185+0000","last_peered":"2024-04-25T09:53:21.302185+0000","last_clean":"2024-04-25T09:53:21.302185+0000","last_became_active":"2024-04-25T09:53:15.742099+0000","last_became_peered":"2024-04-25T09:53:15.742099+0000","last_unstale":"2024-04-25T09:53:21.302185+0000","last_undegraded":"2024-04-25T09:53:21.302185+0000","last_fullsized":"2024-04-25T09:53:21.302185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:53.040580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302379+0000","last_change":"2024-04-25T09:53:15.729969+0000","last_active":"2024-04-25T09:53:21.302379+0000","last_peered":"2024-04-25T09:53:21.302379+0000","last_clean":"2024-04-25T09:53:21.302379+0000","last_became_active":"2024-04-25T09:53:15.729608+0000","last_became_peered":"2024-04-25T09:53:15.729608+0000","last_unstale":"2024-04-25T09:53:21.302379+0000","last_undegraded":"2024-04-25T09:53:21.302379+0000","last_fullsized":"2024-04-25T09:53:21.302379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303562+0000","last_change":"2024-04-25T09:53:15.730114+0000","last_active":"2024-04-25T09:53:21.303562+0000","last_peered":"2024-04-25T09:53:21.303562+0000","last_clean":"2024-04-25T09:53:21.303562+0000","last_became_active":"2024-04-25T09:53:15.729664+0000","last_became_peered":"2024-04-25T09:53:15.729664+0000","last_unstale":"2024-04-25T09:53:21.303562+0000","last_undegraded":"2024-04-25T09:53:21.303562+0000","last_fullsized":"2024-04-25T09:53:21.303562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303105+0000","last_change":"2024-04-25T09:53:15.730458+0000","last_active":"2024-04-25T09:53:21.303105+0000","last_peered":"2024-04-25T09:53:21.303105+0000","last_clean":"2024-04-25T09:53:21.303105+0000","last_became_active":"2024-04-25T09:53:15.730193+0000","last_became_peered":"2024-04-25T09:53:15.730193+0000","last_unstale":"2024-04-25T09:53:21.303105+0000","last_undegraded":"2024-04-25T09:53:21.303105+0000","last_fullsized":"2024-04-25T09:53:21.303105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:46.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741462+0000","last_change":"2024-04-25T09:53:15.723126+0000","last_active":"2024-04-25T09:53:20.741462+0000","last_peered":"2024-04-25T09:53:20.741462+0000","last_clean":"2024-04-25T09:53:20.741462+0000","last_became_active":"2024-04-25T09:53:15.722834+0000","last_became_peered":"2024-04-25T09:53:15.722834+0000","last_unstale":"2024-04-25T09:53:20.741462+0000","last_undegraded":"2024-04-25T09:53:20.741462+0000","last_fullsized":"2024-04-25T09:53:20.741462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:40.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303240+0000","last_change":"2024-04-25T09:53:15.742492+0000","last_active":"2024-04-25T09:53:21.303240+0000","last_peered":"2024-04-25T09:53:21.303240+0000","last_clean":"2024-04-25T09:53:21.303240+0000","last_became_active":"2024-04-25T09:53:15.742132+0000","last_became_peered":"2024-04-25T09:53:15.742132+0000","last_unstale":"2024-04-25T09:53:21.303240+0000","last_undegraded":"2024-04-25T09:53:21.303240+0000","last_fullsized":"2024-04-25T09:53:21.303240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:08.755180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301867+0000","last_change":"2024-04-25T09:53:15.729450+0000","last_active":"2024-04-25T09:53:21.301867+0000","last_peered":"2024-04-25T09:53:21.301867+0000","last_clean":"2024-04-25T09:53:21.301867+0000","last_became_active":"2024-04-25T09:53:15.729157+0000","last_became_peered":"2024-04-25T09:53:15.729157+0000","last_unstale":"2024-04-25T09:53:21.301867+0000","last_undegraded":"2024-04-25T09:53:21.301867+0000","last_fullsized":"2024-04-25T09:53:21.301867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301716+0000","last_change":"2024-04-25T09:53:15.727922+0000","last_active":"2024-04-25T09:53:21.301716+0000","last_peered":"2024-04-25T09:53:21.301716+0000","last_clean":"2024-04-25T09:53:21.301716+0000","last_became_active":"2024-04-25T09:53:15.727626+0000","last_became_peered":"2024-04-25T09:53:15.727626+0000","last_unstale":"2024-04-25T09:53:21.301716+0000","last_undegraded":"2024-04-25T09:53:21.301716+0000","last_fullsized":"2024-04-25T09:53:21.301716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301947+0000","last_change":"2024-04-25T09:53:15.721374+0000","last_active":"2024-04-25T09:53:21.301947+0000","last_peered":"2024-04-25T09:53:21.301947+0000","last_clean":"2024-04-25T09:53:21.301947+0000","last_became_active":"2024-04-25T09:53:15.719489+0000","last_became_peered":"2024-04-25T09:53:15.719489+0000","last_unstale":"2024-04-25T09:53:21.301947+0000","last_undegraded":"2024-04-25T09:53:21.301947+0000","last_fullsized":"2024-04-25T09:53:21.301947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:53.990141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302112+0000","last_change":"2024-04-25T09:53:15.723353+0000","last_active":"2024-04-25T09:53:21.302112+0000","last_peered":"2024-04-25T09:53:21.302112+0000","last_clean":"2024-04-25T09:53:21.302112+0000","last_became_active":"2024-04-25T09:53:15.723060+0000","last_became_peered":"2024-04-25T09:53:15.723060+0000","last_unstale":"2024-04-25T09:53:21.302112+0000","last_undegraded":"2024-04-25T09:53:21.302112+0000","last_fullsized":"2024-04-25T09:53:21.302112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743264+0000","last_change":"2024-04-25T09:53:15.731294+0000","last_active":"2024-04-25T09:53:20.743264+0000","last_peered":"2024-04-25T09:53:20.743264+0000","last_clean":"2024-04-25T09:53:20.743264+0000","last_became_active":"2024-04-25T09:53:15.730556+0000","last_became_peered":"2024-04-25T09:53:15.730556+0000","last_unstale":"2024-04-25T09:53:20.743264+0000","last_undegraded":"2024-04-25T09:53:20.743264+0000","last_fullsized":"2024-04-25T09:53:20.743264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:59.737264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303649+0000","last_change":"2024-04-25T09:53:15.737541+0000","last_active":"2024-04-25T09:53:21.303649+0000","last_peered":"2024-04-25T09:53:21.303649+0000","last_clean":"2024-04-25T09:53:21.303649+0000","last_became_active":"2024-04-25T09:53:15.737262+0000","last_became_peered":"2024-04-25T09:53:15.737262+0000","last_unstale":"2024-04-25T09:53:21.303649+0000","last_undegraded":"2024-04-25T09:53:21.303649+0000","last_fullsized":"2024-04-25T09:53:21.303649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303812+0000","last_change":"2024-04-25T09:53:15.730915+0000","last_active":"2024-04-25T09:53:21.303812+0000","last_peered":"2024-04-25T09:53:21.303812+0000","last_clean":"2024-04-25T09:53:21.303812+0000","last_became_active":"2024-04-25T09:53:15.730657+0000","last_became_peered":"2024-04-25T09:53:15.730657+0000","last_unstale":"2024-04-25T09:53:21.303812+0000","last_undegraded":"2024-04-25T09:53:21.303812+0000","last_fullsized":"2024-04-25T09:53:21.303812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303315+0000","last_change":"2024-04-25T09:53:15.733524+0000","last_active":"2024-04-25T09:53:21.303315+0000","last_peered":"2024-04-25T09:53:21.303315+0000","last_clean":"2024-04-25T09:53:21.303315+0000","last_became_active":"2024-04-25T09:53:15.733244+0000","last_became_peered":"2024-04-25T09:53:15.733244+0000","last_unstale":"2024-04-25T09:53:21.303315+0000","last_undegraded":"2024-04-25T09:53:21.303315+0000","last_fullsized":"2024-04-25T09:53:21.303315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:07.652771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301474+0000","last_change":"2024-04-25T09:53:15.731461+0000","last_active":"2024-04-25T09:53:21.301474+0000","last_peered":"2024-04-25T09:53:21.301474+0000","last_clean":"2024-04-25T09:53:21.301474+0000","last_became_active":"2024-04-25T09:53:15.731091+0000","last_became_peered":"2024-04-25T09:53:15.731091+0000","last_unstale":"2024-04-25T09:53:21.301474+0000","last_undegraded":"2024-04-25T09:53:21.301474+0000","last_fullsized":"2024-04-25T09:53:21.301474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302158+0000","last_change":"2024-04-25T09:53:15.724502+0000","last_active":"2024-04-25T09:53:21.302158+0000","last_peered":"2024-04-25T09:53:21.302158+0000","last_clean":"2024-04-25T09:53:21.302158+0000","last_became_active":"2024-04-25T09:53:15.724218+0000","last_became_peered":"2024-04-25T09:53:15.724218+0000","last_unstale":"2024-04-25T09:53:21.302158+0000","last_undegraded":"2024-04-25T09:53:21.302158+0000","last_fullsized":"2024-04-25T09:53:21.302158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:59.223177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302094+0000","last_change":"2024-04-25T09:53:15.734479+0000","last_active":"2024-04-25T09:53:21.302094+0000","last_peered":"2024-04-25T09:53:21.302094+0000","last_clean":"2024-04-25T09:53:21.302094+0000","last_became_active":"2024-04-25T09:53:15.734127+0000","last_became_peered":"2024-04-25T09:53:15.734127+0000","last_unstale":"2024-04-25T09:53:21.302094+0000","last_undegraded":"2024-04-25T09:53:21.302094+0000","last_fullsized":"2024-04-25T09:53:21.302094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:00.208172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302254+0000","last_change":"2024-04-25T09:53:15.731043+0000","last_active":"2024-04-25T09:53:21.302254+0000","last_peered":"2024-04-25T09:53:21.302254+0000","last_clean":"2024-04-25T09:53:21.302254+0000","last_became_active":"2024-04-25T09:53:15.730779+0000","last_became_peered":"2024-04-25T09:53:15.730779+0000","last_unstale":"2024-04-25T09:53:21.302254+0000","last_undegraded":"2024-04-25T09:53:21.302254+0000","last_fullsized":"2024-04-25T09:53:21.302254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:02.721136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302662+0000","last_change":"2024-04-25T09:53:15.732810+0000","last_active":"2024-04-25T09:53:21.302662+0000","last_peered":"2024-04-25T09:53:21.302662+0000","last_clean":"2024-04-25T09:53:21.302662+0000","last_became_active":"2024-04-25T09:53:15.732270+0000","last_became_peered":"2024-04-25T09:53:15.732270+0000","last_unstale":"2024-04-25T09:53:21.302662+0000","last_undegraded":"2024-04-25T09:53:21.302662+0000","last_fullsized":"2024-04-25T09:53:21.302662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:59.223177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301627+0000","last_change":"2024-04-25T09:53:15.713610+0000","last_active":"2024-04-25T09:53:21.301627+0000","last_peered":"2024-04-25T09:53:21.301627+0000","last_clean":"2024-04-25T09:53:21.301627+0000","last_became_active":"2024-04-25T09:53:15.713342+0000","last_became_peered":"2024-04-25T09:53:15.713342+0000","last_unstale":"2024-04-25T09:53:21.301627+0000","last_undegraded":"2024-04-25T09:53:21.301627+0000","last_fullsized":"2024-04-25T09:53:21.301627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:39.445622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303622+0000","last_change":"2024-04-25T09:53:15.723089+0000","last_active":"2024-04-25T09:53:21.303622+0000","last_peered":"2024-04-25T09:53:21.303622+0000","last_clean":"2024-04-25T09:53:21.303622+0000","last_became_active":"2024-04-25T09:53:15.722840+0000","last_became_peered":"2024-04-25T09:53:15.722840+0000","last_unstale":"2024-04-25T09:53:21.303622+0000","last_undegraded":"2024-04-25T09:53:21.303622+0000","last_fullsized":"2024-04-25T09:53:21.303622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303201+0000","last_change":"2024-04-25T09:53:15.734984+0000","last_active":"2024-04-25T09:53:21.303201+0000","last_peered":"2024-04-25T09:53:21.303201+0000","last_clean":"2024-04-25T09:53:21.303201+0000","last_became_active":"2024-04-25T09:53:15.734344+0000","last_became_peered":"2024-04-25T09:53:15.734344+0000","last_unstale":"2024-04-25T09:53:21.303201+0000","last_undegraded":"2024-04-25T09:53:21.303201+0000","last_fullsized":"2024-04-25T09:53:21.303201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:50.045183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:53:31.712 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:53:31.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.842+0000 7f88093e7640 1 Processor -- start 2024-04-25T09:53:31.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.842+0000 7f88093e7640 1 -- start start 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f88093e7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415dc60 0x7f88040b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f88093e7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8804161850 0x7f88040ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f88093e7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88040cf250 con 0x7f88040d0970 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f88093e7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88040ce780 con 0x7f8804161850 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f88093e7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88040d3730 con 0x7f880415dc60 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f8802ffd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415dc60 0x7f88040b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f88027fc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8804161850 0x7f88040ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f88027fc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8804161850 0x7f88040ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:45744/0 (socket says 172.21.15.139:45744) 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f8802ffd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415dc60 0x7f88040b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57002/0 (socket says 172.21.15.139:57002) 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f88027fc640 1 -- 172.21.15.139:0/3839441298 learned_addr learned my addr 172.21.15.139:0/3839441298 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f88027fc640 1 -- 172.21.15.139:0/3839441298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415dc60 msgr2=0x7f88040b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:31.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f88027fc640 1 --2- 172.21.15.139:0/3839441298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415dc60 0x7f88040b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f88027fc640 1 -- 172.21.15.139:0/3839441298 >> v1:172.21.15.150:6789/0 conn(0x7f88040d0970 legacy=0x7f88040d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f88027fc640 1 -- 172.21.15.139:0/3839441298 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88040ce9b0 con 0x7f8804161850 2024-04-25T09:53:31.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f8801ffb640 1 -- 172.21.15.139:0/3839441298 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911959941 0 0) 0x7f87fc001110 con 0x7f88040d0970 2024-04-25T09:53:31.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.843+0000 7f8802ffd640 1 --2- 172.21.15.139:0/3839441298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415dc60 0x7f88040b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:31.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.844+0000 7f88027fc640 1 --2- 172.21.15.139:0/3839441298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8804161850 0x7f88040ce240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f87f80098a0 tx=0x7f87f802a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfbcb20b7baa2bb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.844+0000 7f8801ffb640 1 -- 172.21.15.139:0/3839441298 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8037070 con 0x7f8804161850 2024-04-25T09:53:31.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.844+0000 7f8801ffb640 1 -- 172.21.15.139:0/3839441298 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f8032440 con 0x7f8804161850 2024-04-25T09:53:31.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.844+0000 7f8801ffb640 1 -- 172.21.15.139:0/3839441298 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8032810 con 0x7f8804161850 2024-04-25T09:53:31.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.845+0000 7f88093e7640 1 -- 172.21.15.139:0/3839441298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8804161850 msgr2=0x7f88040ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.845+0000 7f88093e7640 1 --2- 172.21.15.139:0/3839441298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8804161850 0x7f88040ce240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f87f80098a0 tx=0x7f87f802a290 comp rx=0 tx=0).stop 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.845+0000 7f88093e7640 1 -- 172.21.15.139:0/3839441298 shutdown_connections 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.845+0000 7f88093e7640 1 --2- 172.21.15.139:0/3839441298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8804161850 0x7f88040ce240 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.845+0000 7f88093e7640 1 --2- 172.21.15.139:0/3839441298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415dc60 0x7f88040b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.845+0000 7f88093e7640 1 -- 172.21.15.139:0/3839441298 >> 172.21.15.139:0/3839441298 conn(0x7f88040b5910 msgr2=0x7f88040b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.845+0000 7f88093e7640 1 -- 172.21.15.139:0/3839441298 shutdown_connections 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.845+0000 7f88093e7640 1 -- 172.21.15.139:0/3839441298 wait complete. 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.846+0000 7f88093e7640 1 Processor -- start 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.846+0000 7f88093e7640 1 -- start start 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f88093e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88040d0970 0x7f8804149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f88093e7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415dc60 0x7f8804153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f88093e7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8804161850 0x7f8804153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f88093e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88040d1fc0 con 0x7f88040d0970 2024-04-25T09:53:31.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f88093e7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88040cef70 con 0x7f8804161850 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f88093e7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880415e040 con 0x7f880415dc60 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f8802ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88040d0970 0x7f8804149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f8802ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88040d0970 0x7f8804149d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58668/0 (socket says 172.21.15.139:58668) 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f8802ffd640 1 -- 172.21.15.139:0/1482035924 learned_addr learned my addr 172.21.15.139:0/1482035924 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f88027fc640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415dc60 0x7f8804153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f88037fe640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8804161850 0x7f8804153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f8802ffd640 1 -- 172.21.15.139:0/1482035924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415dc60 msgr2=0x7f8804153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f8802ffd640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415dc60 0x7f8804153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f8802ffd640 1 -- 172.21.15.139:0/1482035924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8804161850 msgr2=0x7f8804153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f8802ffd640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8804161850 0x7f8804153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.847+0000 7f8802ffd640 1 -- 172.21.15.139:0/1482035924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88040cfd70 con 0x7f88040d0970 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.848+0000 7f8802ffd640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88040d0970 0x7f8804149d50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f87f400efc0 tx=0x7f87f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.848+0000 7f87eb7fe640 1 -- 172.21.15.139:0/1482035924 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4018020 con 0x7f88040d0970 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.848+0000 7f87eb7fe640 1 -- 172.21.15.139:0/1482035924 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f4014680 con 0x7f88040d0970 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.848+0000 7f88093e7640 1 -- 172.21.15.139:0/1482035924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880415c170 con 0x7f88040d0970 2024-04-25T09:53:31.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.848+0000 7f87eb7fe640 1 -- 172.21.15.139:0/1482035924 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4014a50 con 0x7f88040d0970 2024-04-25T09:53:31.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.848+0000 7f88093e7640 1 -- 172.21.15.139:0/1482035924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880414a290 con 0x7f88040d0970 2024-04-25T09:53:31.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.849+0000 7f88093e7640 1 -- 172.21.15.139:0/1482035924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8804074520 con 0x7f88040d0970 2024-04-25T09:53:31.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.849+0000 7f87eb7fe640 1 -- 172.21.15.139:0/1482035924 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87f4005d10 con 0x7f88040d0970 2024-04-25T09:53:31.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.850+0000 7f87eb7fe640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f87f006b4d0 0x7f87f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:31.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.850+0000 7f88027fc640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f87f006b4d0 0x7f87f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:31.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.850+0000 7f87eb7fe640 1 -- 172.21.15.139:0/1482035924 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f87f408b230 con 0x7f88040d0970 2024-04-25T09:53:31.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.850+0000 7f88027fc640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f87f006b4d0 0x7f87f006d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f87f8034000 tx=0x7f87f802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:31.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:31.854+0000 7f87eb7fe640 1 -- 172.21.15.139:0/1482035924 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87f4022030 con 0x7f88040d0970 2024-04-25T09:53:32.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.021+0000 7f88093e7640 1 -- 172.21.15.139:0/1482035924 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88040d0050 con 0x7f87f006b4d0 2024-04-25T09:53:32.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.028+0000 7f87eb7fe640 1 -- 172.21.15.139:0/1482035924 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f87f8004340 con 0x7f87f006b4d0 2024-04-25T09:53:32.032 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:32.037 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:53:32.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.031+0000 7f88093e7640 1 -- 172.21.15.139:0/1482035924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f87f006b4d0 msgr2=0x7f87f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.031+0000 7f88093e7640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f87f006b4d0 0x7f87f006d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f87f8034000 tx=0x7f87f802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:32.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.031+0000 7f88093e7640 1 -- 172.21.15.139:0/1482035924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88040d0970 msgr2=0x7f8804149d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.031+0000 7f88093e7640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88040d0970 0x7f8804149d50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f87f400efc0 tx=0x7f87f400c490 comp rx=0 tx=0).stop 2024-04-25T09:53:32.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.031+0000 7f88093e7640 1 -- 172.21.15.139:0/1482035924 shutdown_connections 2024-04-25T09:53:32.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.031+0000 7f88093e7640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f87f006b4d0 0x7f87f006d990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.031+0000 7f88093e7640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8804161850 0x7f8804153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.031+0000 7f88093e7640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f880415dc60 0x7f8804153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.031+0000 7f88093e7640 1 --2- 172.21.15.139:0/1482035924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88040d0970 0x7f8804149d50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.031+0000 7f88093e7640 1 -- 172.21.15.139:0/1482035924 >> 172.21.15.139:0/1482035924 conn(0x7f88040b5910 msgr2=0x7f8804157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:32.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.032+0000 7f88093e7640 1 -- 172.21.15.139:0/1482035924 shutdown_connections 2024-04-25T09:53:32.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.032+0000 7f88093e7640 1 -- 172.21.15.139:0/1482035924 wait complete. 2024-04-25T09:53:32.045 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-04-25T09:53:30.970463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3284992,"data_stored":1991782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.230503"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302246+0000","last_change":"2024-04-25T09:53:14.712100+0000","last_active":"2024-04-25T09:53:21.302246+0000","last_peered":"2024-04-25T09:53:21.302246+0000","last_clean":"2024-04-25T09:53:21.302246+0000","last_became_active":"2024-04-25T09:53:14.711827+0000","last_became_peered":"2024-04-25T09:53:14.711827+0000","last_unstale":"2024-04-25T09:53:21.302246+0000","last_undegraded":"2024-04-25T09:53:21.302246+0000","last_fullsized":"2024-04-25T09:53:21.302246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:57.295010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302544+0000","last_change":"2024-04-25T09:53:15.723415+0000","last_active":"2024-04-25T09:53:21.302544+0000","last_peered":"2024-04-25T09:53:21.302544+0000","last_clean":"2024-04-25T09:53:21.302544+0000","last_became_active":"2024-04-25T09:53:15.723126+0000","last_became_peered":"2024-04-25T09:53:15.723126+0000","last_unstale":"2024-04-25T09:53:21.302544+0000","last_undegraded":"2024-04-25T09:53:21.302544+0000","last_fullsized":"2024-04-25T09:53:21.302544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:53.040580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303119+0000","last_change":"2024-04-25T09:53:19.938169+0000","last_active":"2024-04-25T09:53:21.303119+0000","last_peered":"2024-04-25T09:53:21.303119+0000","last_clean":"2024-04-25T09:53:21.303119+0000","last_became_active":"2024-04-25T09:53:19.937831+0000","last_became_peered":"2024-04-25T09:53:19.937831+0000","last_unstale":"2024-04-25T09:53:21.303119+0000","last_undegraded":"2024-04-25T09:53:21.303119+0000","last_fullsized":"2024-04-25T09:53:21.303119+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:05.087194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302754+0000","last_change":"2024-04-25T09:53:14.708452+0000","last_active":"2024-04-25T09:53:21.302754+0000","last_peered":"2024-04-25T09:53:21.302754+0000","last_clean":"2024-04-25T09:53:21.302754+0000","last_became_active":"2024-04-25T09:53:14.708168+0000","last_became_peered":"2024-04-25T09:53:14.708168+0000","last_unstale":"2024-04-25T09:53:21.302754+0000","last_undegraded":"2024-04-25T09:53:21.302754+0000","last_fullsized":"2024-04-25T09:53:21.302754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:08.699272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301824+0000","last_change":"2024-04-25T09:53:15.731605+0000","last_active":"2024-04-25T09:53:21.301824+0000","last_peered":"2024-04-25T09:53:21.301824+0000","last_clean":"2024-04-25T09:53:21.301824+0000","last_became_active":"2024-04-25T09:53:15.731254+0000","last_became_peered":"2024-04-25T09:53:15.731254+0000","last_unstale":"2024-04-25T09:53:21.301824+0000","last_undegraded":"2024-04-25T09:53:21.301824+0000","last_fullsized":"2024-04-25T09:53:21.301824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742162+0000","last_change":"2024-04-25T09:53:19.756305+0000","last_active":"2024-04-25T09:53:20.742162+0000","last_peered":"2024-04-25T09:53:20.742162+0000","last_clean":"2024-04-25T09:53:20.742162+0000","last_became_active":"2024-04-25T09:53:19.755760+0000","last_became_peered":"2024-04-25T09:53:19.755760+0000","last_unstale":"2024-04-25T09:53:20.742162+0000","last_undegraded":"2024-04-25T09:53:20.742162+0000","last_fullsized":"2024-04-25T09:53:20.742162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:45.103063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303011+0000","last_change":"2024-04-25T09:53:14.713424+0000","last_active":"2024-04-25T09:53:21.303011+0000","last_peered":"2024-04-25T09:53:21.303011+0000","last_clean":"2024-04-25T09:53:21.303011+0000","last_became_active":"2024-04-25T09:53:14.711564+0000","last_became_peered":"2024-04-25T09:53:14.711564+0000","last_unstale":"2024-04-25T09:53:21.303011+0000","last_undegraded":"2024-04-25T09:53:21.303011+0000","last_fullsized":"2024-04-25T09:53:21.303011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:02.525834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302665+0000","last_change":"2024-04-25T09:53:15.737042+0000","last_active":"2024-04-25T09:53:21.302665+0000","last_peered":"2024-04-25T09:53:21.302665+0000","last_clean":"2024-04-25T09:53:21.302665+0000","last_became_active":"2024-04-25T09:53:15.736770+0000","last_became_peered":"2024-04-25T09:53:15.736770+0000","last_unstale":"2024-04-25T09:53:21.302665+0000","last_undegraded":"2024-04-25T09:53:21.302665+0000","last_fullsized":"2024-04-25T09:53:21.302665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:57.316572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742503+0000","last_change":"2024-04-25T09:53:19.761920+0000","last_active":"2024-04-25T09:53:20.742503+0000","last_peered":"2024-04-25T09:53:20.742503+0000","last_clean":"2024-04-25T09:53:20.742503+0000","last_became_active":"2024-04-25T09:53:19.761083+0000","last_became_peered":"2024-04-25T09:53:19.761083+0000","last_unstale":"2024-04-25T09:53:20.742503+0000","last_undegraded":"2024-04-25T09:53:20.742503+0000","last_fullsized":"2024-04-25T09:53:20.742503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:57.080146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301800+0000","last_change":"2024-04-25T09:53:14.713667+0000","last_active":"2024-04-25T09:53:21.301800+0000","last_peered":"2024-04-25T09:53:21.301800+0000","last_clean":"2024-04-25T09:53:21.301800+0000","last_became_active":"2024-04-25T09:53:14.713327+0000","last_became_peered":"2024-04-25T09:53:14.713327+0000","last_unstale":"2024-04-25T09:53:21.301800+0000","last_undegraded":"2024-04-25T09:53:21.301800+0000","last_fullsized":"2024-04-25T09:53:21.301800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:26.183785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302970+0000","last_change":"2024-04-25T09:53:15.737782+0000","last_active":"2024-04-25T09:53:21.302970+0000","last_peered":"2024-04-25T09:53:21.302970+0000","last_clean":"2024-04-25T09:53:21.302970+0000","last_became_active":"2024-04-25T09:53:15.737531+0000","last_became_peered":"2024-04-25T09:53:15.737531+0000","last_unstale":"2024-04-25T09:53:21.302970+0000","last_undegraded":"2024-04-25T09:53:21.302970+0000","last_fullsized":"2024-04-25T09:53:21.302970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:55.029099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742249+0000","last_change":"2024-04-25T09:53:19.754738+0000","last_active":"2024-04-25T09:53:20.742249+0000","last_peered":"2024-04-25T09:53:20.742249+0000","last_clean":"2024-04-25T09:53:20.742249+0000","last_became_active":"2024-04-25T09:53:19.754248+0000","last_became_peered":"2024-04-25T09:53:19.754248+0000","last_unstale":"2024-04-25T09:53:20.742249+0000","last_undegraded":"2024-04-25T09:53:20.742249+0000","last_fullsized":"2024-04-25T09:53:20.742249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:55.483582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302991+0000","last_change":"2024-04-25T09:53:15.729322+0000","last_active":"2024-04-25T09:53:21.302991+0000","last_peered":"2024-04-25T09:53:21.302991+0000","last_clean":"2024-04-25T09:53:21.302991+0000","last_became_active":"2024-04-25T09:53:15.729037+0000","last_became_peered":"2024-04-25T09:53:15.729037+0000","last_unstale":"2024-04-25T09:53:21.302991+0000","last_undegraded":"2024-04-25T09:53:21.302991+0000","last_fullsized":"2024-04-25T09:53:21.302991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302405+0000","last_change":"2024-04-25T09:53:14.712891+0000","last_active":"2024-04-25T09:53:21.302405+0000","last_peered":"2024-04-25T09:53:21.302405+0000","last_clean":"2024-04-25T09:53:21.302405+0000","last_became_active":"2024-04-25T09:53:14.712375+0000","last_became_peered":"2024-04-25T09:53:14.712375+0000","last_unstale":"2024-04-25T09:53:21.302405+0000","last_undegraded":"2024-04-25T09:53:21.302405+0000","last_fullsized":"2024-04-25T09:53:21.302405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:58.733069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743015+0000","last_change":"2024-04-25T09:53:19.762022+0000","last_active":"2024-04-25T09:53:20.743015+0000","last_peered":"2024-04-25T09:53:20.743015+0000","last_clean":"2024-04-25T09:53:20.743015+0000","last_became_active":"2024-04-25T09:53:19.761624+0000","last_became_peered":"2024-04-25T09:53:19.761624+0000","last_unstale":"2024-04-25T09:53:20.743015+0000","last_undegraded":"2024-04-25T09:53:20.743015+0000","last_fullsized":"2024-04-25T09:53:20.743015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:01.356138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302197+0000","last_change":"2024-04-25T09:53:15.720406+0000","last_active":"2024-04-25T09:53:21.302197+0000","last_peered":"2024-04-25T09:53:21.302197+0000","last_clean":"2024-04-25T09:53:21.302197+0000","last_became_active":"2024-04-25T09:53:15.720140+0000","last_became_peered":"2024-04-25T09:53:15.720140+0000","last_unstale":"2024-04-25T09:53:21.302197+0000","last_undegraded":"2024-04-25T09:53:21.302197+0000","last_fullsized":"2024-04-25T09:53:21.302197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302882+0000","last_change":"2024-04-25T09:53:14.705015+0000","last_active":"2024-04-25T09:53:21.302882+0000","last_peered":"2024-04-25T09:53:21.302882+0000","last_clean":"2024-04-25T09:53:21.302882+0000","last_became_active":"2024-04-25T09:53:14.704705+0000","last_became_peered":"2024-04-25T09:53:14.704705+0000","last_unstale":"2024-04-25T09:53:21.302882+0000","last_undegraded":"2024-04-25T09:53:21.302882+0000","last_fullsized":"2024-04-25T09:53:21.302882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743231+0000","last_change":"2024-04-25T09:53:19.761275+0000","last_active":"2024-04-25T09:53:20.743231+0000","last_peered":"2024-04-25T09:53:20.743231+0000","last_clean":"2024-04-25T09:53:20.743231+0000","last_became_active":"2024-04-25T09:53:19.760859+0000","last_became_peered":"2024-04-25T09:53:19.760859+0000","last_unstale":"2024-04-25T09:53:20.743231+0000","last_undegraded":"2024-04-25T09:53:20.743231+0000","last_fullsized":"2024-04-25T09:53:20.743231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:20.608892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301802+0000","last_change":"2024-04-25T09:53:15.726544+0000","last_active":"2024-04-25T09:53:21.301802+0000","last_peered":"2024-04-25T09:53:21.301802+0000","last_clean":"2024-04-25T09:53:21.301802+0000","last_became_active":"2024-04-25T09:53:15.726267+0000","last_became_peered":"2024-04-25T09:53:15.726267+0000","last_unstale":"2024-04-25T09:53:21.301802+0000","last_undegraded":"2024-04-25T09:53:21.301802+0000","last_fullsized":"2024-04-25T09:53:21.301802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301803+0000","last_change":"2024-04-25T09:53:14.712292+0000","last_active":"2024-04-25T09:53:21.301803+0000","last_peered":"2024-04-25T09:53:21.301803+0000","last_clean":"2024-04-25T09:53:21.301803+0000","last_became_active":"2024-04-25T09:53:14.711702+0000","last_became_peered":"2024-04-25T09:53:14.711702+0000","last_unstale":"2024-04-25T09:53:21.301803+0000","last_undegraded":"2024-04-25T09:53:21.301803+0000","last_fullsized":"2024-04-25T09:53:21.301803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303737+0000","last_change":"2024-04-25T09:53:19.937513+0000","last_active":"2024-04-25T09:53:21.303737+0000","last_peered":"2024-04-25T09:53:21.303737+0000","last_clean":"2024-04-25T09:53:21.303737+0000","last_became_active":"2024-04-25T09:53:19.937076+0000","last_became_peered":"2024-04-25T09:53:19.937076+0000","last_unstale":"2024-04-25T09:53:21.303737+0000","last_undegraded":"2024-04-25T09:53:21.303737+0000","last_fullsized":"2024-04-25T09:53:21.303737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:50.372524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742399+0000","last_change":"2024-04-25T09:53:15.717251+0000","last_active":"2024-04-25T09:53:20.742399+0000","last_peered":"2024-04-25T09:53:20.742399+0000","last_clean":"2024-04-25T09:53:20.742399+0000","last_became_active":"2024-04-25T09:53:15.716981+0000","last_became_peered":"2024-04-25T09:53:15.716981+0000","last_unstale":"2024-04-25T09:53:20.742399+0000","last_undegraded":"2024-04-25T09:53:20.742399+0000","last_fullsized":"2024-04-25T09:53:20.742399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:08.082240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301576+0000","last_change":"2024-04-25T09:53:14.711486+0000","last_active":"2024-04-25T09:53:21.301576+0000","last_peered":"2024-04-25T09:53:21.301576+0000","last_clean":"2024-04-25T09:53:21.301576+0000","last_became_active":"2024-04-25T09:53:14.711090+0000","last_became_peered":"2024-04-25T09:53:14.711090+0000","last_unstale":"2024-04-25T09:53:21.301576+0000","last_undegraded":"2024-04-25T09:53:21.301576+0000","last_fullsized":"2024-04-25T09:53:21.301576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742532+0000","last_change":"2024-04-25T09:53:19.756221+0000","last_active":"2024-04-25T09:53:20.742532+0000","last_peered":"2024-04-25T09:53:20.742532+0000","last_clean":"2024-04-25T09:53:20.742532+0000","last_became_active":"2024-04-25T09:53:19.755565+0000","last_became_peered":"2024-04-25T09:53:19.755565+0000","last_unstale":"2024-04-25T09:53:20.742532+0000","last_undegraded":"2024-04-25T09:53:20.742532+0000","last_fullsized":"2024-04-25T09:53:20.742532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:03.262744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302070+0000","last_change":"2024-04-25T09:53:15.720035+0000","last_active":"2024-04-25T09:53:21.302070+0000","last_peered":"2024-04-25T09:53:21.302070+0000","last_clean":"2024-04-25T09:53:21.302070+0000","last_became_active":"2024-04-25T09:53:15.719506+0000","last_became_peered":"2024-04-25T09:53:15.719506+0000","last_unstale":"2024-04-25T09:53:21.302070+0000","last_undegraded":"2024-04-25T09:53:21.302070+0000","last_fullsized":"2024-04-25T09:53:21.302070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303882+0000","last_change":"2024-04-25T09:53:14.714287+0000","last_active":"2024-04-25T09:53:21.303882+0000","last_peered":"2024-04-25T09:53:21.303882+0000","last_clean":"2024-04-25T09:53:21.303882+0000","last_became_active":"2024-04-25T09:53:14.714026+0000","last_became_peered":"2024-04-25T09:53:14.714026+0000","last_unstale":"2024-04-25T09:53:21.303882+0000","last_undegraded":"2024-04-25T09:53:21.303882+0000","last_fullsized":"2024-04-25T09:53:21.303882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742315+0000","last_change":"2024-04-25T09:53:19.934699+0000","last_active":"2024-04-25T09:53:20.742315+0000","last_peered":"2024-04-25T09:53:20.742315+0000","last_clean":"2024-04-25T09:53:20.742315+0000","last_became_active":"2024-04-25T09:53:19.934296+0000","last_became_peered":"2024-04-25T09:53:19.934296+0000","last_unstale":"2024-04-25T09:53:20.742315+0000","last_undegraded":"2024-04-25T09:53:20.742315+0000","last_fullsized":"2024-04-25T09:53:20.742315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:04.247738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302840+0000","last_change":"2024-04-25T09:53:15.735174+0000","last_active":"2024-04-25T09:53:21.302840+0000","last_peered":"2024-04-25T09:53:21.302840+0000","last_clean":"2024-04-25T09:53:21.302840+0000","last_became_active":"2024-04-25T09:53:15.734811+0000","last_became_peered":"2024-04-25T09:53:15.734811+0000","last_unstale":"2024-04-25T09:53:21.302840+0000","last_undegraded":"2024-04-25T09:53:21.302840+0000","last_fullsized":"2024-04-25T09:53:21.302840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:53.172005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303710+0000","last_change":"2024-04-25T09:53:14.707243+0000","last_active":"2024-04-25T09:53:21.303710+0000","last_peered":"2024-04-25T09:53:21.303710+0000","last_clean":"2024-04-25T09:53:21.303710+0000","last_became_active":"2024-04-25T09:53:14.706608+0000","last_became_peered":"2024-04-25T09:53:14.706608+0000","last_unstale":"2024-04-25T09:53:21.303710+0000","last_undegraded":"2024-04-25T09:53:21.303710+0000","last_fullsized":"2024-04-25T09:53:21.303710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:49.257995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301370+0000","last_change":"2024-04-25T09:53:19.938687+0000","last_active":"2024-04-25T09:53:21.301370+0000","last_peered":"2024-04-25T09:53:21.301370+0000","last_clean":"2024-04-25T09:53:21.301370+0000","last_became_active":"2024-04-25T09:53:19.938312+0000","last_became_peered":"2024-04-25T09:53:19.938312+0000","last_unstale":"2024-04-25T09:53:21.301370+0000","last_undegraded":"2024-04-25T09:53:21.301370+0000","last_fullsized":"2024-04-25T09:53:21.301370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:11.692337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302904+0000","last_change":"2024-04-25T09:53:15.724325+0000","last_active":"2024-04-25T09:53:21.302904+0000","last_peered":"2024-04-25T09:53:21.302904+0000","last_clean":"2024-04-25T09:53:21.302904+0000","last_became_active":"2024-04-25T09:53:15.724024+0000","last_became_peered":"2024-04-25T09:53:15.724024+0000","last_unstale":"2024-04-25T09:53:21.302904+0000","last_undegraded":"2024-04-25T09:53:21.302904+0000","last_fullsized":"2024-04-25T09:53:21.302904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303201+0000","last_change":"2024-04-25T09:53:14.712119+0000","last_active":"2024-04-25T09:53:21.303201+0000","last_peered":"2024-04-25T09:53:21.303201+0000","last_clean":"2024-04-25T09:53:21.303201+0000","last_became_active":"2024-04-25T09:53:14.711462+0000","last_became_peered":"2024-04-25T09:53:14.711462+0000","last_unstale":"2024-04-25T09:53:21.303201+0000","last_undegraded":"2024-04-25T09:53:21.303201+0000","last_fullsized":"2024-04-25T09:53:21.303201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:39.622554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302329+0000","last_change":"2024-04-25T09:53:19.934786+0000","last_active":"2024-04-25T09:53:21.302329+0000","last_peered":"2024-04-25T09:53:21.302329+0000","last_clean":"2024-04-25T09:53:21.302329+0000","last_became_active":"2024-04-25T09:53:19.934417+0000","last_became_peered":"2024-04-25T09:53:19.934417+0000","last_unstale":"2024-04-25T09:53:21.302329+0000","last_undegraded":"2024-04-25T09:53:21.302329+0000","last_fullsized":"2024-04-25T09:53:21.302329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:50.372524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303078+0000","last_change":"2024-04-25T09:53:15.731430+0000","last_active":"2024-04-25T09:53:21.303078+0000","last_peered":"2024-04-25T09:53:21.303078+0000","last_clean":"2024-04-25T09:53:21.303078+0000","last_became_active":"2024-04-25T09:53:15.731152+0000","last_became_peered":"2024-04-25T09:53:15.731152+0000","last_unstale":"2024-04-25T09:53:21.303078+0000","last_undegraded":"2024-04-25T09:53:21.303078+0000","last_fullsized":"2024-04-25T09:53:21.303078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:46.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302987+0000","last_change":"2024-04-25T09:53:14.708418+0000","last_active":"2024-04-25T09:53:21.302987+0000","last_peered":"2024-04-25T09:53:21.302987+0000","last_clean":"2024-04-25T09:53:21.302987+0000","last_became_active":"2024-04-25T09:53:14.708161+0000","last_became_peered":"2024-04-25T09:53:14.708161+0000","last_unstale":"2024-04-25T09:53:21.302987+0000","last_undegraded":"2024-04-25T09:53:21.302987+0000","last_fullsized":"2024-04-25T09:53:21.302987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:58.733069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302535+0000","last_change":"2024-04-25T09:53:19.758749+0000","last_active":"2024-04-25T09:53:21.302535+0000","last_peered":"2024-04-25T09:53:21.302535+0000","last_clean":"2024-04-25T09:53:21.302535+0000","last_became_active":"2024-04-25T09:53:19.757529+0000","last_became_peered":"2024-04-25T09:53:19.757529+0000","last_unstale":"2024-04-25T09:53:21.302535+0000","last_undegraded":"2024-04-25T09:53:21.302535+0000","last_fullsized":"2024-04-25T09:53:21.302535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:06.760703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742905+0000","last_change":"2024-04-25T09:53:15.726160+0000","last_active":"2024-04-25T09:53:20.742905+0000","last_peered":"2024-04-25T09:53:20.742905+0000","last_clean":"2024-04-25T09:53:20.742905+0000","last_became_active":"2024-04-25T09:53:15.725846+0000","last_became_peered":"2024-04-25T09:53:15.725846+0000","last_unstale":"2024-04-25T09:53:20.742905+0000","last_undegraded":"2024-04-25T09:53:20.742905+0000","last_fullsized":"2024-04-25T09:53:20.742905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.262190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302405+0000","last_change":"2024-04-25T09:53:14.712606+0000","last_active":"2024-04-25T09:53:21.302405+0000","last_peered":"2024-04-25T09:53:21.302405+0000","last_clean":"2024-04-25T09:53:21.302405+0000","last_became_active":"2024-04-25T09:53:14.712327+0000","last_became_peered":"2024-04-25T09:53:14.712327+0000","last_unstale":"2024-04-25T09:53:21.302405+0000","last_undegraded":"2024-04-25T09:53:21.302405+0000","last_fullsized":"2024-04-25T09:53:21.302405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:49.257995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302949+0000","last_change":"2024-04-25T09:53:19.757694+0000","last_active":"2024-04-25T09:53:21.302949+0000","last_peered":"2024-04-25T09:53:21.302949+0000","last_clean":"2024-04-25T09:53:21.302949+0000","last_became_active":"2024-04-25T09:53:19.757293+0000","last_became_peered":"2024-04-25T09:53:19.757293+0000","last_unstale":"2024-04-25T09:53:21.302949+0000","last_undegraded":"2024-04-25T09:53:21.302949+0000","last_fullsized":"2024-04-25T09:53:21.302949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:12.794746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302692+0000","last_change":"2024-04-25T09:53:15.725158+0000","last_active":"2024-04-25T09:53:21.302692+0000","last_peered":"2024-04-25T09:53:21.302692+0000","last_clean":"2024-04-25T09:53:21.302692+0000","last_became_active":"2024-04-25T09:53:15.724890+0000","last_became_peered":"2024-04-25T09:53:15.724890+0000","last_unstale":"2024-04-25T09:53:21.302692+0000","last_undegraded":"2024-04-25T09:53:21.302692+0000","last_fullsized":"2024-04-25T09:53:21.302692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741185+0000","last_change":"2024-04-25T09:53:14.703999+0000","last_active":"2024-04-25T09:53:20.741185+0000","last_peered":"2024-04-25T09:53:20.741185+0000","last_clean":"2024-04-25T09:53:20.741185+0000","last_became_active":"2024-04-25T09:53:14.703714+0000","last_became_peered":"2024-04-25T09:53:14.703714+0000","last_unstale":"2024-04-25T09:53:20.741185+0000","last_undegraded":"2024-04-25T09:53:20.741185+0000","last_fullsized":"2024-04-25T09:53:20.741185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303908+0000","last_change":"2024-04-25T09:53:19.756345+0000","last_active":"2024-04-25T09:53:21.303908+0000","last_peered":"2024-04-25T09:53:21.303908+0000","last_clean":"2024-04-25T09:53:21.303908+0000","last_became_active":"2024-04-25T09:53:19.755915+0000","last_became_peered":"2024-04-25T09:53:19.755915+0000","last_unstale":"2024-04-25T09:53:21.303908+0000","last_undegraded":"2024-04-25T09:53:21.303908+0000","last_fullsized":"2024-04-25T09:53:21.303908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:54.084750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301435+0000","last_change":"2024-04-25T09:53:15.723323+0000","last_active":"2024-04-25T09:53:21.301435+0000","last_peered":"2024-04-25T09:53:21.301435+0000","last_clean":"2024-04-25T09:53:21.301435+0000","last_became_active":"2024-04-25T09:53:15.723019+0000","last_became_peered":"2024-04-25T09:53:15.723019+0000","last_unstale":"2024-04-25T09:53:21.301435+0000","last_undegraded":"2024-04-25T09:53:21.301435+0000","last_fullsized":"2024-04-25T09:53:21.301435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:43.951510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301698+0000","last_change":"2024-04-25T09:53:14.712642+0000","last_active":"2024-04-25T09:53:21.301698+0000","last_peered":"2024-04-25T09:53:21.301698+0000","last_clean":"2024-04-25T09:53:21.301698+0000","last_became_active":"2024-04-25T09:53:14.711778+0000","last_became_peered":"2024-04-25T09:53:14.711778+0000","last_unstale":"2024-04-25T09:53:21.301698+0000","last_undegraded":"2024-04-25T09:53:21.301698+0000","last_fullsized":"2024-04-25T09:53:21.301698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:49.257995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301888+0000","last_change":"2024-04-25T09:53:19.934697+0000","last_active":"2024-04-25T09:53:21.301888+0000","last_peered":"2024-04-25T09:53:21.301888+0000","last_clean":"2024-04-25T09:53:21.301888+0000","last_became_active":"2024-04-25T09:53:19.934259+0000","last_became_peered":"2024-04-25T09:53:19.934259+0000","last_unstale":"2024-04-25T09:53:21.301888+0000","last_undegraded":"2024-04-25T09:53:21.301888+0000","last_fullsized":"2024-04-25T09:53:21.301888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:12.794746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301861+0000","last_change":"2024-04-25T09:53:15.731434+0000","last_active":"2024-04-25T09:53:21.301861+0000","last_peered":"2024-04-25T09:53:21.301861+0000","last_clean":"2024-04-25T09:53:21.301861+0000","last_became_active":"2024-04-25T09:53:15.731139+0000","last_became_peered":"2024-04-25T09:53:15.731139+0000","last_unstale":"2024-04-25T09:53:21.301861+0000","last_undegraded":"2024-04-25T09:53:21.301861+0000","last_fullsized":"2024-04-25T09:53:21.301861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302366+0000","last_change":"2024-04-25T09:53:14.706419+0000","last_active":"2024-04-25T09:53:21.302366+0000","last_peered":"2024-04-25T09:53:21.302366+0000","last_clean":"2024-04-25T09:53:21.302366+0000","last_became_active":"2024-04-25T09:53:14.706020+0000","last_became_peered":"2024-04-25T09:53:14.706020+0000","last_unstale":"2024-04-25T09:53:21.302366+0000","last_undegraded":"2024-04-25T09:53:21.302366+0000","last_fullsized":"2024-04-25T09:53:21.302366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301803+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:53:21.301803+0000","last_peered":"2024-04-25T09:53:21.301803+0000","last_clean":"2024-04-25T09:53:21.301803+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:53:21.301803+0000","last_undegraded":"2024-04-25T09:53:21.301803+0000","last_fullsized":"2024-04-25T09:53:21.301803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303580+0000","last_change":"2024-04-25T09:53:19.937575+0000","last_active":"2024-04-25T09:53:21.303580+0000","last_peered":"2024-04-25T09:53:21.303580+0000","last_clean":"2024-04-25T09:53:21.303580+0000","last_became_active":"2024-04-25T09:53:19.937138+0000","last_became_peered":"2024-04-25T09:53:19.937138+0000","last_unstale":"2024-04-25T09:53:21.303580+0000","last_undegraded":"2024-04-25T09:53:21.303580+0000","last_fullsized":"2024-04-25T09:53:21.303580+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:01.356138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302839+0000","last_change":"2024-04-25T09:53:15.733512+0000","last_active":"2024-04-25T09:53:21.302839+0000","last_peered":"2024-04-25T09:53:21.302839+0000","last_clean":"2024-04-25T09:53:21.302839+0000","last_became_active":"2024-04-25T09:53:15.733224+0000","last_became_peered":"2024-04-25T09:53:15.733224+0000","last_unstale":"2024-04-25T09:53:21.302839+0000","last_undegraded":"2024-04-25T09:53:21.302839+0000","last_fullsized":"2024-04-25T09:53:21.302839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:57.316572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741420+0000","last_change":"2024-04-25T09:53:14.708846+0000","last_active":"2024-04-25T09:53:20.741420+0000","last_peered":"2024-04-25T09:53:20.741420+0000","last_clean":"2024-04-25T09:53:20.741420+0000","last_became_active":"2024-04-25T09:53:14.708587+0000","last_became_peered":"2024-04-25T09:53:14.708587+0000","last_unstale":"2024-04-25T09:53:20.741420+0000","last_undegraded":"2024-04-25T09:53:20.741420+0000","last_fullsized":"2024-04-25T09:53:20.741420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302923+0000","last_change":"2024-04-25T09:53:19.934818+0000","last_active":"2024-04-25T09:53:21.302923+0000","last_peered":"2024-04-25T09:53:21.302923+0000","last_clean":"2024-04-25T09:53:21.302923+0000","last_became_active":"2024-04-25T09:53:19.934403+0000","last_became_peered":"2024-04-25T09:53:19.934403+0000","last_unstale":"2024-04-25T09:53:21.302923+0000","last_undegraded":"2024-04-25T09:53:21.302923+0000","last_fullsized":"2024-04-25T09:53:21.302923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:57.211571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302766+0000","last_change":"2024-04-25T09:53:15.732566+0000","last_active":"2024-04-25T09:53:21.302766+0000","last_peered":"2024-04-25T09:53:21.302766+0000","last_clean":"2024-04-25T09:53:21.302766+0000","last_became_active":"2024-04-25T09:53:15.731920+0000","last_became_peered":"2024-04-25T09:53:15.731920+0000","last_unstale":"2024-04-25T09:53:21.302766+0000","last_undegraded":"2024-04-25T09:53:21.302766+0000","last_fullsized":"2024-04-25T09:53:21.302766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:53.172005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302249+0000","last_change":"2024-04-25T09:53:14.703268+0000","last_active":"2024-04-25T09:53:21.302249+0000","last_peered":"2024-04-25T09:53:21.302249+0000","last_clean":"2024-04-25T09:53:21.302249+0000","last_became_active":"2024-04-25T09:53:14.702989+0000","last_became_peered":"2024-04-25T09:53:14.702989+0000","last_unstale":"2024-04-25T09:53:21.302249+0000","last_undegraded":"2024-04-25T09:53:21.302249+0000","last_fullsized":"2024-04-25T09:53:21.302249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302491+0000","last_change":"2024-04-25T09:53:19.762847+0000","last_active":"2024-04-25T09:53:21.302491+0000","last_peered":"2024-04-25T09:53:21.302491+0000","last_clean":"2024-04-25T09:53:21.302491+0000","last_became_active":"2024-04-25T09:53:19.762336+0000","last_became_peered":"2024-04-25T09:53:19.762336+0000","last_unstale":"2024-04-25T09:53:21.302491+0000","last_undegraded":"2024-04-25T09:53:21.302491+0000","last_fullsized":"2024-04-25T09:53:21.302491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:54.084750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303390+0000","last_change":"2024-04-25T09:53:15.732927+0000","last_active":"2024-04-25T09:53:21.303390+0000","last_peered":"2024-04-25T09:53:21.303390+0000","last_clean":"2024-04-25T09:53:21.303390+0000","last_became_active":"2024-04-25T09:53:15.732640+0000","last_became_peered":"2024-04-25T09:53:15.732640+0000","last_unstale":"2024-04-25T09:53:21.303390+0000","last_undegraded":"2024-04-25T09:53:21.303390+0000","last_fullsized":"2024-04-25T09:53:21.303390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:27.493283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741538+0000","last_change":"2024-04-25T09:53:14.708266+0000","last_active":"2024-04-25T09:53:20.741538+0000","last_peered":"2024-04-25T09:53:20.741538+0000","last_clean":"2024-04-25T09:53:20.741538+0000","last_became_active":"2024-04-25T09:53:14.707980+0000","last_became_peered":"2024-04-25T09:53:14.707980+0000","last_unstale":"2024-04-25T09:53:20.741538+0000","last_undegraded":"2024-04-25T09:53:20.741538+0000","last_fullsized":"2024-04-25T09:53:20.741538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301745+0000","last_change":"2024-04-25T09:53:19.934601+0000","last_active":"2024-04-25T09:53:21.301745+0000","last_peered":"2024-04-25T09:53:21.301745+0000","last_clean":"2024-04-25T09:53:21.301745+0000","last_became_active":"2024-04-25T09:53:19.934185+0000","last_became_peered":"2024-04-25T09:53:19.934185+0000","last_unstale":"2024-04-25T09:53:21.301745+0000","last_undegraded":"2024-04-25T09:53:21.301745+0000","last_fullsized":"2024-04-25T09:53:21.301745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:05.087194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302564+0000","last_change":"2024-04-25T09:53:15.732497+0000","last_active":"2024-04-25T09:53:21.302564+0000","last_peered":"2024-04-25T09:53:21.302564+0000","last_clean":"2024-04-25T09:53:21.302564+0000","last_became_active":"2024-04-25T09:53:15.732229+0000","last_became_peered":"2024-04-25T09:53:15.732229+0000","last_unstale":"2024-04-25T09:53:21.302564+0000","last_undegraded":"2024-04-25T09:53:21.302564+0000","last_fullsized":"2024-04-25T09:53:21.302564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:08.755180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742750+0000","last_change":"2024-04-25T09:53:14.708424+0000","last_active":"2024-04-25T09:53:20.742750+0000","last_peered":"2024-04-25T09:53:20.742750+0000","last_clean":"2024-04-25T09:53:20.742750+0000","last_became_active":"2024-04-25T09:53:14.708139+0000","last_became_peered":"2024-04-25T09:53:14.708139+0000","last_unstale":"2024-04-25T09:53:20.742750+0000","last_undegraded":"2024-04-25T09:53:20.742750+0000","last_fullsized":"2024-04-25T09:53:20.742750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:11.395695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742138+0000","last_change":"2024-04-25T09:53:19.753647+0000","last_active":"2024-04-25T09:53:20.742138+0000","last_peered":"2024-04-25T09:53:20.742138+0000","last_clean":"2024-04-25T09:53:20.742138+0000","last_became_active":"2024-04-25T09:53:19.753091+0000","last_became_peered":"2024-04-25T09:53:19.753091+0000","last_unstale":"2024-04-25T09:53:20.742138+0000","last_undegraded":"2024-04-25T09:53:20.742138+0000","last_fullsized":"2024-04-25T09:53:20.742138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:58.778179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302186+0000","last_change":"2024-04-25T09:53:15.731871+0000","last_active":"2024-04-25T09:53:21.302186+0000","last_peered":"2024-04-25T09:53:21.302186+0000","last_clean":"2024-04-25T09:53:21.302186+0000","last_became_active":"2024-04-25T09:53:15.731615+0000","last_became_peered":"2024-04-25T09:53:15.731615+0000","last_unstale":"2024-04-25T09:53:21.302186+0000","last_undegraded":"2024-04-25T09:53:21.302186+0000","last_fullsized":"2024-04-25T09:53:21.302186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:53.172005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741071+0000","last_change":"2024-04-25T09:53:14.701334+0000","last_active":"2024-04-25T09:53:20.741071+0000","last_peered":"2024-04-25T09:53:20.741071+0000","last_clean":"2024-04-25T09:53:20.741071+0000","last_became_active":"2024-04-25T09:53:14.701062+0000","last_became_peered":"2024-04-25T09:53:14.701062+0000","last_unstale":"2024-04-25T09:53:20.741071+0000","last_undegraded":"2024-04-25T09:53:20.741071+0000","last_fullsized":"2024-04-25T09:53:20.741071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301727+0000","last_change":"2024-04-25T09:53:19.758482+0000","last_active":"2024-04-25T09:53:21.301727+0000","last_peered":"2024-04-25T09:53:21.301727+0000","last_clean":"2024-04-25T09:53:21.301727+0000","last_became_active":"2024-04-25T09:53:19.758036+0000","last_became_peered":"2024-04-25T09:53:19.758036+0000","last_unstale":"2024-04-25T09:53:21.301727+0000","last_undegraded":"2024-04-25T09:53:21.301727+0000","last_fullsized":"2024-04-25T09:53:21.301727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.068666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301688+0000","last_change":"2024-04-25T09:53:15.721989+0000","last_active":"2024-04-25T09:53:21.301688+0000","last_peered":"2024-04-25T09:53:21.301688+0000","last_clean":"2024-04-25T09:53:21.301688+0000","last_became_active":"2024-04-25T09:53:15.721636+0000","last_became_peered":"2024-04-25T09:53:15.721636+0000","last_unstale":"2024-04-25T09:53:21.301688+0000","last_undegraded":"2024-04-25T09:53:21.301688+0000","last_fullsized":"2024-04-25T09:53:21.301688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:39.445622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302694+0000","last_change":"2024-04-25T09:53:14.709691+0000","last_active":"2024-04-25T09:53:21.302694+0000","last_peered":"2024-04-25T09:53:21.302694+0000","last_clean":"2024-04-25T09:53:21.302694+0000","last_became_active":"2024-04-25T09:53:14.709373+0000","last_became_peered":"2024-04-25T09:53:14.709373+0000","last_unstale":"2024-04-25T09:53:21.302694+0000","last_undegraded":"2024-04-25T09:53:21.302694+0000","last_fullsized":"2024-04-25T09:53:21.302694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302719+0000","last_change":"2024-04-25T09:53:19.937412+0000","last_active":"2024-04-25T09:53:21.302719+0000","last_peered":"2024-04-25T09:53:21.302719+0000","last_clean":"2024-04-25T09:53:21.302719+0000","last_became_active":"2024-04-25T09:53:19.937068+0000","last_became_peered":"2024-04-25T09:53:19.937068+0000","last_unstale":"2024-04-25T09:53:21.302719+0000","last_undegraded":"2024-04-25T09:53:21.302719+0000","last_fullsized":"2024-04-25T09:53:21.302719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:18.654048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302364+0000","last_change":"2024-04-25T09:53:15.734287+0000","last_active":"2024-04-25T09:53:21.302364+0000","last_peered":"2024-04-25T09:53:21.302364+0000","last_clean":"2024-04-25T09:53:21.302364+0000","last_became_active":"2024-04-25T09:53:15.733835+0000","last_became_peered":"2024-04-25T09:53:15.733835+0000","last_unstale":"2024-04-25T09:53:21.302364+0000","last_undegraded":"2024-04-25T09:53:21.302364+0000","last_fullsized":"2024-04-25T09:53:21.302364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:55.029099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302389+0000","last_change":"2024-04-25T09:53:14.707323+0000","last_active":"2024-04-25T09:53:21.302389+0000","last_peered":"2024-04-25T09:53:21.302389+0000","last_clean":"2024-04-25T09:53:21.302389+0000","last_became_active":"2024-04-25T09:53:14.707061+0000","last_became_peered":"2024-04-25T09:53:14.707061+0000","last_unstale":"2024-04-25T09:53:21.302389+0000","last_undegraded":"2024-04-25T09:53:21.302389+0000","last_fullsized":"2024-04-25T09:53:21.302389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303418+0000","last_change":"2024-04-25T09:53:19.935315+0000","last_active":"2024-04-25T09:53:21.303418+0000","last_peered":"2024-04-25T09:53:21.303418+0000","last_clean":"2024-04-25T09:53:21.303418+0000","last_became_active":"2024-04-25T09:53:19.934800+0000","last_became_peered":"2024-04-25T09:53:19.934800+0000","last_unstale":"2024-04-25T09:53:21.303418+0000","last_undegraded":"2024-04-25T09:53:21.303418+0000","last_fullsized":"2024-04-25T09:53:21.303418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:50.137095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302044+0000","last_change":"2024-04-25T09:53:15.729897+0000","last_active":"2024-04-25T09:53:21.302044+0000","last_peered":"2024-04-25T09:53:21.302044+0000","last_clean":"2024-04-25T09:53:21.302044+0000","last_became_active":"2024-04-25T09:53:15.729269+0000","last_became_peered":"2024-04-25T09:53:15.729269+0000","last_unstale":"2024-04-25T09:53:21.302044+0000","last_undegraded":"2024-04-25T09:53:21.302044+0000","last_fullsized":"2024-04-25T09:53:21.302044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:55.387059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302564+0000","last_change":"2024-04-25T09:53:14.709636+0000","last_active":"2024-04-25T09:53:21.302564+0000","last_peered":"2024-04-25T09:53:21.302564+0000","last_clean":"2024-04-25T09:53:21.302564+0000","last_became_active":"2024-04-25T09:53:14.709366+0000","last_became_peered":"2024-04-25T09:53:14.709366+0000","last_unstale":"2024-04-25T09:53:21.302564+0000","last_undegraded":"2024-04-25T09:53:21.302564+0000","last_fullsized":"2024-04-25T09:53:21.302564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:05.869022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303066+0000","last_change":"2024-04-25T09:53:19.937074+0000","last_active":"2024-04-25T09:53:21.303066+0000","last_peered":"2024-04-25T09:53:21.303066+0000","last_clean":"2024-04-25T09:53:21.303066+0000","last_became_active":"2024-04-25T09:53:19.935313+0000","last_became_peered":"2024-04-25T09:53:19.935313+0000","last_unstale":"2024-04-25T09:53:21.303066+0000","last_undegraded":"2024-04-25T09:53:21.303066+0000","last_fullsized":"2024-04-25T09:53:21.303066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:56.116208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302820+0000","last_change":"2024-04-25T09:53:15.726822+0000","last_active":"2024-04-25T09:53:21.302820+0000","last_peered":"2024-04-25T09:53:21.302820+0000","last_clean":"2024-04-25T09:53:21.302820+0000","last_became_active":"2024-04-25T09:53:15.726526+0000","last_became_peered":"2024-04-25T09:53:15.726526+0000","last_unstale":"2024-04-25T09:53:21.302820+0000","last_undegraded":"2024-04-25T09:53:21.302820+0000","last_fullsized":"2024-04-25T09:53:21.302820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:16.569326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:24.390526+0000","last_change":"2024-04-25T09:53:14.707187+0000","last_active":"2024-04-25T09:53:24.390526+0000","last_peered":"2024-04-25T09:53:24.390526+0000","last_clean":"2024-04-25T09:53:24.390526+0000","last_became_active":"2024-04-25T09:53:14.706509+0000","last_became_peered":"2024-04-25T09:53:14.706509+0000","last_unstale":"2024-04-25T09:53:24.390526+0000","last_undegraded":"2024-04-25T09:53:24.390526+0000","last_fullsized":"2024-04-25T09:53:24.390526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302676+0000","last_change":"2024-04-25T09:53:19.758453+0000","last_active":"2024-04-25T09:53:21.302676+0000","last_peered":"2024-04-25T09:53:21.302676+0000","last_clean":"2024-04-25T09:53:21.302676+0000","last_became_active":"2024-04-25T09:53:19.757004+0000","last_became_peered":"2024-04-25T09:53:19.757004+0000","last_unstale":"2024-04-25T09:53:21.302676+0000","last_undegraded":"2024-04-25T09:53:21.302676+0000","last_fullsized":"2024-04-25T09:53:21.302676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.068666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743281+0000","last_change":"2024-04-25T09:53:15.731519+0000","last_active":"2024-04-25T09:53:20.743281+0000","last_peered":"2024-04-25T09:53:20.743281+0000","last_clean":"2024-04-25T09:53:20.743281+0000","last_became_active":"2024-04-25T09:53:15.731037+0000","last_became_peered":"2024-04-25T09:53:15.731037+0000","last_unstale":"2024-04-25T09:53:20.743281+0000","last_undegraded":"2024-04-25T09:53:20.743281+0000","last_fullsized":"2024-04-25T09:53:20.743281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.530029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302151+0000","last_change":"2024-04-25T09:53:14.713398+0000","last_active":"2024-04-25T09:53:21.302151+0000","last_peered":"2024-04-25T09:53:21.302151+0000","last_clean":"2024-04-25T09:53:21.302151+0000","last_became_active":"2024-04-25T09:53:14.713125+0000","last_became_peered":"2024-04-25T09:53:14.713125+0000","last_unstale":"2024-04-25T09:53:21.302151+0000","last_undegraded":"2024-04-25T09:53:21.302151+0000","last_fullsized":"2024-04-25T09:53:21.302151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742729+0000","last_change":"2024-04-25T09:53:19.762998+0000","last_active":"2024-04-25T09:53:20.742729+0000","last_peered":"2024-04-25T09:53:20.742729+0000","last_clean":"2024-04-25T09:53:20.742729+0000","last_became_active":"2024-04-25T09:53:19.762550+0000","last_became_peered":"2024-04-25T09:53:19.762550+0000","last_unstale":"2024-04-25T09:53:20.742729+0000","last_undegraded":"2024-04-25T09:53:20.742729+0000","last_fullsized":"2024-04-25T09:53:20.742729+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:57.211571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302003+0000","last_change":"2024-04-25T09:53:15.731707+0000","last_active":"2024-04-25T09:53:21.302003+0000","last_peered":"2024-04-25T09:53:21.302003+0000","last_clean":"2024-04-25T09:53:21.302003+0000","last_became_active":"2024-04-25T09:53:15.731303+0000","last_became_peered":"2024-04-25T09:53:15.731303+0000","last_unstale":"2024-04-25T09:53:21.302003+0000","last_undegraded":"2024-04-25T09:53:21.302003+0000","last_fullsized":"2024-04-25T09:53:21.302003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302319+0000","last_change":"2024-04-25T09:53:14.714447+0000","last_active":"2024-04-25T09:53:21.302319+0000","last_peered":"2024-04-25T09:53:21.302319+0000","last_clean":"2024-04-25T09:53:21.302319+0000","last_became_active":"2024-04-25T09:53:14.714155+0000","last_became_peered":"2024-04-25T09:53:14.714155+0000","last_unstale":"2024-04-25T09:53:21.302319+0000","last_undegraded":"2024-04-25T09:53:21.302319+0000","last_fullsized":"2024-04-25T09:53:21.302319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301664+0000","last_change":"2024-04-25T09:53:19.761513+0000","last_active":"2024-04-25T09:53:21.301664+0000","last_peered":"2024-04-25T09:53:21.301664+0000","last_clean":"2024-04-25T09:53:21.301664+0000","last_became_active":"2024-04-25T09:53:19.761037+0000","last_became_peered":"2024-04-25T09:53:19.761037+0000","last_unstale":"2024-04-25T09:53:21.301664+0000","last_undegraded":"2024-04-25T09:53:21.301664+0000","last_fullsized":"2024-04-25T09:53:21.301664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:31.532850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302218+0000","last_change":"2024-04-25T09:53:15.742482+0000","last_active":"2024-04-25T09:53:21.302218+0000","last_peered":"2024-04-25T09:53:21.302218+0000","last_clean":"2024-04-25T09:53:21.302218+0000","last_became_active":"2024-04-25T09:53:15.742170+0000","last_became_peered":"2024-04-25T09:53:15.742170+0000","last_unstale":"2024-04-25T09:53:21.302218+0000","last_undegraded":"2024-04-25T09:53:21.302218+0000","last_fullsized":"2024-04-25T09:53:21.302218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:16.569326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303779+0000","last_change":"2024-04-25T09:53:14.714376+0000","last_active":"2024-04-25T09:53:21.303779+0000","last_peered":"2024-04-25T09:53:21.303779+0000","last_clean":"2024-04-25T09:53:21.303779+0000","last_became_active":"2024-04-25T09:53:14.714116+0000","last_became_peered":"2024-04-25T09:53:14.714116+0000","last_unstale":"2024-04-25T09:53:21.303779+0000","last_undegraded":"2024-04-25T09:53:21.303779+0000","last_fullsized":"2024-04-25T09:53:21.303779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303146+0000","last_change":"2024-04-25T09:53:19.938076+0000","last_active":"2024-04-25T09:53:21.303146+0000","last_peered":"2024-04-25T09:53:21.303146+0000","last_clean":"2024-04-25T09:53:21.303146+0000","last_became_active":"2024-04-25T09:53:19.937704+0000","last_became_peered":"2024-04-25T09:53:19.937704+0000","last_unstale":"2024-04-25T09:53:21.303146+0000","last_undegraded":"2024-04-25T09:53:21.303146+0000","last_fullsized":"2024-04-25T09:53:21.303146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:56.116208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301567+0000","last_change":"2024-04-25T09:53:19.936514+0000","last_active":"2024-04-25T09:53:21.301567+0000","last_peered":"2024-04-25T09:53:21.301567+0000","last_clean":"2024-04-25T09:53:21.301567+0000","last_became_active":"2024-04-25T09:53:19.936095+0000","last_became_peered":"2024-04-25T09:53:19.936095+0000","last_unstale":"2024-04-25T09:53:21.301567+0000","last_undegraded":"2024-04-25T09:53:21.301567+0000","last_fullsized":"2024-04-25T09:53:21.301567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:06.760703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301623+0000","last_change":"2024-04-25T09:53:15.726524+0000","last_active":"2024-04-25T09:53:21.301623+0000","last_peered":"2024-04-25T09:53:21.301623+0000","last_clean":"2024-04-25T09:53:21.301623+0000","last_became_active":"2024-04-25T09:53:15.726160+0000","last_became_peered":"2024-04-25T09:53:15.726160+0000","last_unstale":"2024-04-25T09:53:21.301623+0000","last_undegraded":"2024-04-25T09:53:21.301623+0000","last_fullsized":"2024-04-25T09:53:21.301623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302003+0000","last_change":"2024-04-25T09:53:14.704589+0000","last_active":"2024-04-25T09:53:21.302003+0000","last_peered":"2024-04-25T09:53:21.302003+0000","last_clean":"2024-04-25T09:53:21.302003+0000","last_became_active":"2024-04-25T09:53:14.704228+0000","last_became_peered":"2024-04-25T09:53:14.704228+0000","last_unstale":"2024-04-25T09:53:21.302003+0000","last_undegraded":"2024-04-25T09:53:21.302003+0000","last_fullsized":"2024-04-25T09:53:21.302003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303831+0000","last_change":"2024-04-25T09:53:19.761677+0000","last_active":"2024-04-25T09:53:21.303831+0000","last_peered":"2024-04-25T09:53:21.303831+0000","last_clean":"2024-04-25T09:53:21.303831+0000","last_became_active":"2024-04-25T09:53:19.761276+0000","last_became_peered":"2024-04-25T09:53:19.761276+0000","last_unstale":"2024-04-25T09:53:21.303831+0000","last_undegraded":"2024-04-25T09:53:21.303831+0000","last_fullsized":"2024-04-25T09:53:21.303831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:20.608892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301714+0000","last_change":"2024-04-25T09:53:15.722486+0000","last_active":"2024-04-25T09:53:21.301714+0000","last_peered":"2024-04-25T09:53:21.301714+0000","last_clean":"2024-04-25T09:53:21.301714+0000","last_became_active":"2024-04-25T09:53:15.722225+0000","last_became_peered":"2024-04-25T09:53:15.722225+0000","last_unstale":"2024-04-25T09:53:21.301714+0000","last_undegraded":"2024-04-25T09:53:21.301714+0000","last_fullsized":"2024-04-25T09:53:21.301714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303275+0000","last_change":"2024-04-25T09:53:14.707399+0000","last_active":"2024-04-25T09:53:21.303275+0000","last_peered":"2024-04-25T09:53:21.303275+0000","last_clean":"2024-04-25T09:53:21.303275+0000","last_became_active":"2024-04-25T09:53:14.707078+0000","last_became_peered":"2024-04-25T09:53:14.707078+0000","last_unstale":"2024-04-25T09:53:21.303275+0000","last_undegraded":"2024-04-25T09:53:21.303275+0000","last_fullsized":"2024-04-25T09:53:21.303275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:57.295010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303180+0000","last_change":"2024-04-25T09:53:14.711893+0000","last_active":"2024-04-25T09:53:21.303180+0000","last_peered":"2024-04-25T09:53:21.303180+0000","last_clean":"2024-04-25T09:53:21.303180+0000","last_became_active":"2024-04-25T09:53:14.711513+0000","last_became_peered":"2024-04-25T09:53:14.711513+0000","last_unstale":"2024-04-25T09:53:21.303180+0000","last_undegraded":"2024-04-25T09:53:21.303180+0000","last_fullsized":"2024-04-25T09:53:21.303180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302544+0000","last_change":"2024-04-25T09:53:15.728536+0000","last_active":"2024-04-25T09:53:21.302544+0000","last_peered":"2024-04-25T09:53:21.302544+0000","last_clean":"2024-04-25T09:53:21.302544+0000","last_became_active":"2024-04-25T09:53:15.728235+0000","last_became_peered":"2024-04-25T09:53:15.728235+0000","last_unstale":"2024-04-25T09:53:21.302544+0000","last_undegraded":"2024-04-25T09:53:21.302544+0000","last_fullsized":"2024-04-25T09:53:21.302544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741367+0000","last_change":"2024-04-25T09:53:19.764626+0000","last_active":"2024-04-25T09:53:20.741367+0000","last_peered":"2024-04-25T09:53:20.741367+0000","last_clean":"2024-04-25T09:53:20.741367+0000","last_became_active":"2024-04-25T09:53:19.763954+0000","last_became_peered":"2024-04-25T09:53:19.763954+0000","last_unstale":"2024-04-25T09:53:20.741367+0000","last_undegraded":"2024-04-25T09:53:20.741367+0000","last_fullsized":"2024-04-25T09:53:20.741367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:47.991077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302285+0000","last_change":"2024-04-25T09:53:14.712000+0000","last_active":"2024-04-25T09:53:21.302285+0000","last_peered":"2024-04-25T09:53:21.302285+0000","last_clean":"2024-04-25T09:53:21.302285+0000","last_became_active":"2024-04-25T09:53:14.711712+0000","last_became_peered":"2024-04-25T09:53:14.711712+0000","last_unstale":"2024-04-25T09:53:21.302285+0000","last_undegraded":"2024-04-25T09:53:21.302285+0000","last_fullsized":"2024-04-25T09:53:21.302285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742400+0000","last_change":"2024-04-25T09:53:15.722357+0000","last_active":"2024-04-25T09:53:20.742400+0000","last_peered":"2024-04-25T09:53:20.742400+0000","last_clean":"2024-04-25T09:53:20.742400+0000","last_became_active":"2024-04-25T09:53:15.722082+0000","last_became_peered":"2024-04-25T09:53:15.722082+0000","last_unstale":"2024-04-25T09:53:20.742400+0000","last_undegraded":"2024-04-25T09:53:20.742400+0000","last_fullsized":"2024-04-25T09:53:20.742400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:39.270721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303146+0000","last_change":"2024-04-25T09:53:19.759080+0000","last_active":"2024-04-25T09:53:21.303146+0000","last_peered":"2024-04-25T09:53:21.303146+0000","last_clean":"2024-04-25T09:53:21.303146+0000","last_became_active":"2024-04-25T09:53:19.758705+0000","last_became_peered":"2024-04-25T09:53:19.758705+0000","last_unstale":"2024-04-25T09:53:21.303146+0000","last_undegraded":"2024-04-25T09:53:21.303146+0000","last_fullsized":"2024-04-25T09:53:21.303146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:54.228875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301484+0000","last_change":"2024-04-25T09:53:15.723965+0000","last_active":"2024-04-25T09:53:21.301484+0000","last_peered":"2024-04-25T09:53:21.301484+0000","last_clean":"2024-04-25T09:53:21.301484+0000","last_became_active":"2024-04-25T09:53:15.723660+0000","last_became_peered":"2024-04-25T09:53:15.723660+0000","last_unstale":"2024-04-25T09:53:21.301484+0000","last_undegraded":"2024-04-25T09:53:21.301484+0000","last_fullsized":"2024-04-25T09:53:21.301484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741222+0000","last_change":"2024-04-25T09:53:15.734494+0000","last_active":"2024-04-25T09:53:20.741222+0000","last_peered":"2024-04-25T09:53:20.741222+0000","last_clean":"2024-04-25T09:53:20.741222+0000","last_became_active":"2024-04-25T09:53:15.734204+0000","last_became_peered":"2024-04-25T09:53:15.734204+0000","last_unstale":"2024-04-25T09:53:20.741222+0000","last_undegraded":"2024-04-25T09:53:20.741222+0000","last_fullsized":"2024-04-25T09:53:20.741222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.530029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302296+0000","last_change":"2024-04-25T09:53:15.731317+0000","last_active":"2024-04-25T09:53:21.302296+0000","last_peered":"2024-04-25T09:53:21.302296+0000","last_clean":"2024-04-25T09:53:21.302296+0000","last_became_active":"2024-04-25T09:53:15.731022+0000","last_became_peered":"2024-04-25T09:53:15.731022+0000","last_unstale":"2024-04-25T09:53:21.302296+0000","last_undegraded":"2024-04-25T09:53:21.302296+0000","last_fullsized":"2024-04-25T09:53:21.302296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:02.721136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302472+0000","last_change":"2024-04-25T09:53:15.720872+0000","last_active":"2024-04-25T09:53:21.302472+0000","last_peered":"2024-04-25T09:53:21.302472+0000","last_clean":"2024-04-25T09:53:21.302472+0000","last_became_active":"2024-04-25T09:53:15.720242+0000","last_became_peered":"2024-04-25T09:53:15.720242+0000","last_unstale":"2024-04-25T09:53:21.302472+0000","last_undegraded":"2024-04-25T09:53:21.302472+0000","last_fullsized":"2024-04-25T09:53:21.302472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303531+0000","last_change":"2024-04-25T09:53:15.728500+0000","last_active":"2024-04-25T09:53:21.303531+0000","last_peered":"2024-04-25T09:53:21.303531+0000","last_clean":"2024-04-25T09:53:21.303531+0000","last_became_active":"2024-04-25T09:53:15.726998+0000","last_became_peered":"2024-04-25T09:53:15.726998+0000","last_unstale":"2024-04-25T09:53:21.303531+0000","last_undegraded":"2024-04-25T09:53:21.303531+0000","last_fullsized":"2024-04-25T09:53:21.303531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:00.208172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301991+0000","last_change":"2024-04-25T09:53:15.726337+0000","last_active":"2024-04-25T09:53:21.301991+0000","last_peered":"2024-04-25T09:53:21.301991+0000","last_clean":"2024-04-25T09:53:21.301991+0000","last_became_active":"2024-04-25T09:53:15.726072+0000","last_became_peered":"2024-04-25T09:53:15.726072+0000","last_unstale":"2024-04-25T09:53:21.301991+0000","last_undegraded":"2024-04-25T09:53:21.301991+0000","last_fullsized":"2024-04-25T09:53:21.301991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303810+0000","last_change":"2024-04-25T09:53:15.735200+0000","last_active":"2024-04-25T09:53:21.303810+0000","last_peered":"2024-04-25T09:53:21.303810+0000","last_clean":"2024-04-25T09:53:21.303810+0000","last_became_active":"2024-04-25T09:53:15.734903+0000","last_became_peered":"2024-04-25T09:53:15.734903+0000","last_unstale":"2024-04-25T09:53:21.303810+0000","last_undegraded":"2024-04-25T09:53:21.303810+0000","last_fullsized":"2024-04-25T09:53:21.303810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302760+0000","last_change":"2024-04-25T09:53:15.726664+0000","last_active":"2024-04-25T09:53:21.302760+0000","last_peered":"2024-04-25T09:53:21.302760+0000","last_clean":"2024-04-25T09:53:21.302760+0000","last_became_active":"2024-04-25T09:53:15.726362+0000","last_became_peered":"2024-04-25T09:53:15.726362+0000","last_unstale":"2024-04-25T09:53:21.302760+0000","last_undegraded":"2024-04-25T09:53:21.302760+0000","last_fullsized":"2024-04-25T09:53:21.302760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:59.223177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301824+0000","last_change":"2024-04-25T09:53:15.731610+0000","last_active":"2024-04-25T09:53:21.301824+0000","last_peered":"2024-04-25T09:53:21.301824+0000","last_clean":"2024-04-25T09:53:21.301824+0000","last_became_active":"2024-04-25T09:53:15.731289+0000","last_became_peered":"2024-04-25T09:53:15.731289+0000","last_unstale":"2024-04-25T09:53:21.301824+0000","last_undegraded":"2024-04-25T09:53:21.301824+0000","last_fullsized":"2024-04-25T09:53:21.301824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303221+0000","last_change":"2024-04-25T09:53:15.726434+0000","last_active":"2024-04-25T09:53:21.303221+0000","last_peered":"2024-04-25T09:53:21.303221+0000","last_clean":"2024-04-25T09:53:21.303221+0000","last_became_active":"2024-04-25T09:53:15.726160+0000","last_became_peered":"2024-04-25T09:53:15.726160+0000","last_unstale":"2024-04-25T09:53:21.303221+0000","last_undegraded":"2024-04-25T09:53:21.303221+0000","last_fullsized":"2024-04-25T09:53:21.303221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:39.445622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302185+0000","last_change":"2024-04-25T09:53:15.742414+0000","last_active":"2024-04-25T09:53:21.302185+0000","last_peered":"2024-04-25T09:53:21.302185+0000","last_clean":"2024-04-25T09:53:21.302185+0000","last_became_active":"2024-04-25T09:53:15.742099+0000","last_became_peered":"2024-04-25T09:53:15.742099+0000","last_unstale":"2024-04-25T09:53:21.302185+0000","last_undegraded":"2024-04-25T09:53:21.302185+0000","last_fullsized":"2024-04-25T09:53:21.302185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:53.040580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302379+0000","last_change":"2024-04-25T09:53:15.729969+0000","last_active":"2024-04-25T09:53:21.302379+0000","last_peered":"2024-04-25T09:53:21.302379+0000","last_clean":"2024-04-25T09:53:21.302379+0000","last_became_active":"2024-04-25T09:53:15.729608+0000","last_became_peered":"2024-04-25T09:53:15.729608+0000","last_unstale":"2024-04-25T09:53:21.302379+0000","last_undegraded":"2024-04-25T09:53:21.302379+0000","last_fullsized":"2024-04-25T09:53:21.302379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303562+0000","last_change":"2024-04-25T09:53:15.730114+0000","last_active":"2024-04-25T09:53:21.303562+0000","last_peered":"2024-04-25T09:53:21.303562+0000","last_clean":"2024-04-25T09:53:21.303562+0000","last_became_active":"2024-04-25T09:53:15.729664+0000","last_became_peered":"2024-04-25T09:53:15.729664+0000","last_unstale":"2024-04-25T09:53:21.303562+0000","last_undegraded":"2024-04-25T09:53:21.303562+0000","last_fullsized":"2024-04-25T09:53:21.303562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303105+0000","last_change":"2024-04-25T09:53:15.730458+0000","last_active":"2024-04-25T09:53:21.303105+0000","last_peered":"2024-04-25T09:53:21.303105+0000","last_clean":"2024-04-25T09:53:21.303105+0000","last_became_active":"2024-04-25T09:53:15.730193+0000","last_became_peered":"2024-04-25T09:53:15.730193+0000","last_unstale":"2024-04-25T09:53:21.303105+0000","last_undegraded":"2024-04-25T09:53:21.303105+0000","last_fullsized":"2024-04-25T09:53:21.303105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:46.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741462+0000","last_change":"2024-04-25T09:53:15.723126+0000","last_active":"2024-04-25T09:53:20.741462+0000","last_peered":"2024-04-25T09:53:20.741462+0000","last_clean":"2024-04-25T09:53:20.741462+0000","last_became_active":"2024-04-25T09:53:15.722834+0000","last_became_peered":"2024-04-25T09:53:15.722834+0000","last_unstale":"2024-04-25T09:53:20.741462+0000","last_undegraded":"2024-04-25T09:53:20.741462+0000","last_fullsized":"2024-04-25T09:53:20.741462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:40.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303240+0000","last_change":"2024-04-25T09:53:15.742492+0000","last_active":"2024-04-25T09:53:21.303240+0000","last_peered":"2024-04-25T09:53:21.303240+0000","last_clean":"2024-04-25T09:53:21.303240+0000","last_became_active":"2024-04-25T09:53:15.742132+0000","last_became_peered":"2024-04-25T09:53:15.742132+0000","last_unstale":"2024-04-25T09:53:21.303240+0000","last_undegraded":"2024-04-25T09:53:21.303240+0000","last_fullsized":"2024-04-25T09:53:21.303240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:08.755180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301867+0000","last_change":"2024-04-25T09:53:15.729450+0000","last_active":"2024-04-25T09:53:21.301867+0000","last_peered":"2024-04-25T09:53:21.301867+0000","last_clean":"2024-04-25T09:53:21.301867+0000","last_became_active":"2024-04-25T09:53:15.729157+0000","last_became_peered":"2024-04-25T09:53:15.729157+0000","last_unstale":"2024-04-25T09:53:21.301867+0000","last_undegraded":"2024-04-25T09:53:21.301867+0000","last_fullsized":"2024-04-25T09:53:21.301867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301716+0000","last_change":"2024-04-25T09:53:15.727922+0000","last_active":"2024-04-25T09:53:21.301716+0000","last_peered":"2024-04-25T09:53:21.301716+0000","last_clean":"2024-04-25T09:53:21.301716+0000","last_became_active":"2024-04-25T09:53:15.727626+0000","last_became_peered":"2024-04-25T09:53:15.727626+0000","last_unstale":"2024-04-25T09:53:21.301716+0000","last_undegraded":"2024-04-25T09:53:21.301716+0000","last_fullsized":"2024-04-25T09:53:21.301716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301947+0000","last_change":"2024-04-25T09:53:15.721374+0000","last_active":"2024-04-25T09:53:21.301947+0000","last_peered":"2024-04-25T09:53:21.301947+0000","last_clean":"2024-04-25T09:53:21.301947+0000","last_became_active":"2024-04-25T09:53:15.719489+0000","last_became_peered":"2024-04-25T09:53:15.719489+0000","last_unstale":"2024-04-25T09:53:21.301947+0000","last_undegraded":"2024-04-25T09:53:21.301947+0000","last_fullsized":"2024-04-25T09:53:21.301947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:53.990141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302112+0000","last_change":"2024-04-25T09:53:15.723353+0000","last_active":"2024-04-25T09:53:21.302112+0000","last_peered":"2024-04-25T09:53:21.302112+0000","last_clean":"2024-04-25T09:53:21.302112+0000","last_became_active":"2024-04-25T09:53:15.723060+0000","last_became_peered":"2024-04-25T09:53:15.723060+0000","last_unstale":"2024-04-25T09:53:21.302112+0000","last_undegraded":"2024-04-25T09:53:21.302112+0000","last_fullsized":"2024-04-25T09:53:21.302112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743264+0000","last_change":"2024-04-25T09:53:15.731294+0000","last_active":"2024-04-25T09:53:20.743264+0000","last_peered":"2024-04-25T09:53:20.743264+0000","last_clean":"2024-04-25T09:53:20.743264+0000","last_became_active":"2024-04-25T09:53:15.730556+0000","last_became_peered":"2024-04-25T09:53:15.730556+0000","last_unstale":"2024-04-25T09:53:20.743264+0000","last_undegraded":"2024-04-25T09:53:20.743264+0000","last_fullsized":"2024-04-25T09:53:20.743264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:59.737264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303649+0000","last_change":"2024-04-25T09:53:15.737541+0000","last_active":"2024-04-25T09:53:21.303649+0000","last_peered":"2024-04-25T09:53:21.303649+0000","last_clean":"2024-04-25T09:53:21.303649+0000","last_became_active":"2024-04-25T09:53:15.737262+0000","last_became_peered":"2024-04-25T09:53:15.737262+0000","last_unstale":"2024-04-25T09:53:21.303649+0000","last_undegraded":"2024-04-25T09:53:21.303649+0000","last_fullsized":"2024-04-25T09:53:21.303649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303812+0000","last_change":"2024-04-25T09:53:15.730915+0000","last_active":"2024-04-25T09:53:21.303812+0000","last_peered":"2024-04-25T09:53:21.303812+0000","last_clean":"2024-04-25T09:53:21.303812+0000","last_became_active":"2024-04-25T09:53:15.730657+0000","last_became_peered":"2024-04-25T09:53:15.730657+0000","last_unstale":"2024-04-25T09:53:21.303812+0000","last_undegraded":"2024-04-25T09:53:21.303812+0000","last_fullsized":"2024-04-25T09:53:21.303812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303315+0000","last_change":"2024-04-25T09:53:15.733524+0000","last_active":"2024-04-25T09:53:21.303315+0000","last_peered":"2024-04-25T09:53:21.303315+0000","last_clean":"2024-04-25T09:53:21.303315+0000","last_became_active":"2024-04-25T09:53:15.733244+0000","last_became_peered":"2024-04-25T09:53:15.733244+0000","last_unstale":"2024-04-25T09:53:21.303315+0000","last_undegraded":"2024-04-25T09:53:21.303315+0000","last_fullsized":"2024-04-25T09:53:21.303315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:07.652771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301474+0000","last_change":"2024-04-25T09:53:15.731461+0000","last_active":"2024-04-25T09:53:21.301474+0000","last_peered":"2024-04-25T09:53:21.301474+0000","last_clean":"2024-04-25T09:53:21.301474+0000","last_became_active":"2024-04-25T09:53:15.731091+0000","last_became_peered":"2024-04-25T09:53:15.731091+0000","last_unstale":"2024-04-25T09:53:21.301474+0000","last_undegraded":"2024-04-25T09:53:21.301474+0000","last_fullsized":"2024-04-25T09:53:21.301474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302158+0000","last_change":"2024-04-25T09:53:15.724502+0000","last_active":"2024-04-25T09:53:21.302158+0000","last_peered":"2024-04-25T09:53:21.302158+0000","last_clean":"2024-04-25T09:53:21.302158+0000","last_became_active":"2024-04-25T09:53:15.724218+0000","last_became_peered":"2024-04-25T09:53:15.724218+0000","last_unstale":"2024-04-25T09:53:21.302158+0000","last_undegraded":"2024-04-25T09:53:21.302158+0000","last_fullsized":"2024-04-25T09:53:21.302158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:59.223177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302094+0000","last_change":"2024-04-25T09:53:15.734479+0000","last_active":"2024-04-25T09:53:21.302094+0000","last_peered":"2024-04-25T09:53:21.302094+0000","last_clean":"2024-04-25T09:53:21.302094+0000","last_became_active":"2024-04-25T09:53:15.734127+0000","last_became_peered":"2024-04-25T09:53:15.734127+0000","last_unstale":"2024-04-25T09:53:21.302094+0000","last_undegraded":"2024-04-25T09:53:21.302094+0000","last_fullsized":"2024-04-25T09:53:21.302094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:00.208172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302254+0000","last_change":"2024-04-25T09:53:15.731043+0000","last_active":"2024-04-25T09:53:21.302254+0000","last_peered":"2024-04-25T09:53:21.302254+0000","last_clean":"2024-04-25T09:53:21.302254+0000","last_became_active":"2024-04-25T09:53:15.730779+0000","last_became_peered":"2024-04-25T09:53:15.730779+0000","last_unstale":"2024-04-25T09:53:21.302254+0000","last_undegraded":"2024-04-25T09:53:21.302254+0000","last_fullsized":"2024-04-25T09:53:21.302254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:02.721136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302662+0000","last_change":"2024-04-25T09:53:15.732810+0000","last_active":"2024-04-25T09:53:21.302662+0000","last_peered":"2024-04-25T09:53:21.302662+0000","last_clean":"2024-04-25T09:53:21.302662+0000","last_became_active":"2024-04-25T09:53:15.732270+0000","last_became_peered":"2024-04-25T09:53:15.732270+0000","last_unstale":"2024-04-25T09:53:21.302662+0000","last_undegraded":"2024-04-25T09:53:21.302662+0000","last_fullsized":"2024-04-25T09:53:21.302662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:59.223177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301627+0000","last_change":"2024-04-25T09:53:15.713610+0000","last_active":"2024-04-25T09:53:21.301627+0000","last_peered":"2024-04-25T09:53:21.301627+0000","last_clean":"2024-04-25T09:53:21.301627+0000","last_became_active":"2024-04-25T09:53:15.713342+0000","last_became_peered":"2024-04-25T09:53:15.713342+0000","last_unstale":"2024-04-25T09:53:21.301627+0000","last_undegraded":"2024-04-25T09:53:21.301627+0000","last_fullsized":"2024-04-25T09:53:21.301627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:39.445622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303622+0000","last_change":"2024-04-25T09:53:15.723089+0000","last_active":"2024-04-25T09:53:21.303622+0000","last_peered":"2024-04-25T09:53:21.303622+0000","last_clean":"2024-04-25T09:53:21.303622+0000","last_became_active":"2024-04-25T09:53:15.722840+0000","last_became_peered":"2024-04-25T09:53:15.722840+0000","last_unstale":"2024-04-25T09:53:21.303622+0000","last_undegraded":"2024-04-25T09:53:21.303622+0000","last_fullsized":"2024-04-25T09:53:21.303622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303201+0000","last_change":"2024-04-25T09:53:15.734984+0000","last_active":"2024-04-25T09:53:21.303201+0000","last_peered":"2024-04-25T09:53:21.303201+0000","last_clean":"2024-04-25T09:53:21.303201+0000","last_became_active":"2024-04-25T09:53:15.734344+0000","last_became_peered":"2024-04-25T09:53:15.734344+0000","last_unstale":"2024-04-25T09:53:21.303201+0000","last_undegraded":"2024-04-25T09:53:21.303201+0000","last_fullsized":"2024-04-25T09:53:21.303201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:50.045183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:53:32.050 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:53:32.050 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T09:53:32.050 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T09:53:32.051 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T09:53:32.051 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T09:53:32.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.179+0000 7fdf5855c640 1 Processor -- start 2024-04-25T09:53:32.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.179+0000 7fdf5855c640 1 -- start start 2024-04-25T09:53:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.180+0000 7fdf5855c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf501611e0 0x7fdf500b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.180+0000 7fdf5855c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf500d4740 con 0x7fdf5006c780 2024-04-25T09:53:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.180+0000 7fdf5855c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf500b2770 con 0x7fdf500cea00 2024-04-25T09:53:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.180+0000 7fdf5855c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500d1fa0 con 0x7fdf501611e0 2024-04-25T09:53:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.181+0000 7fdf56af9640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdf5006c780 0x7fdf500d29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46464/0 (socket says 172.21.15.139:46464) 2024-04-25T09:53:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.181+0000 7fdf56af9640 1 -- 172.21.15.139:0/522980402 learned_addr learned my addr 172.21.15.139:0/522980402 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.181+0000 7fdf562f8640 1 --2- 172.21.15.139:0/522980402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf501611e0 0x7fdf500b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.181+0000 7fdf552f6640 1 -- 172.21.15.139:0/522980402 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311526296 0 0) 0x7fdf500d4740 con 0x7fdf5006c780 2024-04-25T09:53:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.181+0000 7fdf552f6640 1 -- 172.21.15.139:0/522980402 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf3c003520 con 0x7fdf5006c780 2024-04-25T09:53:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.181+0000 7fdf552f6640 1 -- 172.21.15.139:0/522980402 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688550017 0 0) 0x7fdf500b2770 con 0x7fdf500cea00 2024-04-25T09:53:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.181+0000 7fdf552f6640 1 -- 172.21.15.139:0/522980402 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf500d4740 con 0x7fdf500cea00 2024-04-25T09:53:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.181+0000 7fdf562f8640 1 -- 172.21.15.139:0/522980402 >> v1:172.21.15.150:6789/0 conn(0x7fdf500cea00 legacy=0x7fdf5006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.181+0000 7fdf562f8640 1 -- 172.21.15.139:0/522980402 >> v1:172.21.15.139:6789/0 conn(0x7fdf5006c780 legacy=0x7fdf500d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.181+0000 7fdf562f8640 1 -- 172.21.15.139:0/522980402 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf500b2a50 con 0x7fdf501611e0 2024-04-25T09:53:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.182+0000 7fdf562f8640 1 --2- 172.21.15.139:0/522980402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf501611e0 0x7fdf500b6af0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009dc0 tx=0x7fdf4802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5986a598c7aa8ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.182+0000 7fdf552f6640 1 -- 172.21.15.139:0/522980402 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48007740 con 0x7fdf501611e0 2024-04-25T09:53:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.182+0000 7fdf552f6640 1 -- 172.21.15.139:0/522980402 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf480078e0 con 0x7fdf501611e0 2024-04-25T09:53:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.182+0000 7fdf552f6640 1 -- 172.21.15.139:0/522980402 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48007cb0 con 0x7fdf501611e0 2024-04-25T09:53:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.182+0000 7fdf5855c640 1 -- 172.21.15.139:0/522980402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf501611e0 msgr2=0x7fdf500b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.182+0000 7fdf5855c640 1 --2- 172.21.15.139:0/522980402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf501611e0 0x7fdf500b6af0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009dc0 tx=0x7fdf4802a6d0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.183+0000 7fdf5855c640 1 -- 172.21.15.139:0/522980402 shutdown_connections 2024-04-25T09:53:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.183+0000 7fdf5855c640 1 --2- 172.21.15.139:0/522980402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf501611e0 0x7fdf500b6af0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.183+0000 7fdf5855c640 1 -- 172.21.15.139:0/522980402 >> 172.21.15.139:0/522980402 conn(0x7fdf500b5b00 msgr2=0x7fdf500d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.183+0000 7fdf5855c640 1 -- 172.21.15.139:0/522980402 shutdown_connections 2024-04-25T09:53:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.183+0000 7fdf5855c640 1 -- 172.21.15.139:0/522980402 wait complete. 2024-04-25T09:53:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.184+0000 7fdf5855c640 1 Processor -- start 2024-04-25T09:53:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.184+0000 7fdf5855c640 1 -- start start 2024-04-25T09:53:32.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.184+0000 7fdf5855c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5006c780 0x7fdf500c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.184+0000 7fdf5855c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf500cea00 0x7fdf500c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.184+0000 7fdf5855c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf501611e0 0x7fdf50157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.184+0000 7fdf5855c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500d21f0 con 0x7fdf500cea00 2024-04-25T09:53:32.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.184+0000 7fdf5855c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500d2560 con 0x7fdf5006c780 2024-04-25T09:53:32.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.184+0000 7fdf5855c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf501615c0 con 0x7fdf501611e0 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.184+0000 7fdf55af7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf500cea00 0x7fdf500c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf55af7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf500cea00 0x7fdf500c9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58672/0 (socket says 172.21.15.139:58672) 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf55af7640 1 -- 172.21.15.139:0/3004958157 learned_addr learned my addr 172.21.15.139:0/3004958157 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf55af7640 1 -- 172.21.15.139:0/3004958157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf501611e0 msgr2=0x7fdf50157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf562f8640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5006c780 0x7fdf500c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf56af9640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf501611e0 0x7fdf50157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf55af7640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf501611e0 0x7fdf50157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf55af7640 1 -- 172.21.15.139:0/3004958157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5006c780 msgr2=0x7fdf500c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf55af7640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5006c780 0x7fdf500c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf55af7640 1 -- 172.21.15.139:0/3004958157 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5015fad0 con 0x7fdf500cea00 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf562f8640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5006c780 0x7fdf500c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf56af9640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf501611e0 0x7fdf50157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf55af7640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf500cea00 0x7fdf500c9870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000c500 tx=0x7fdf4000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf46ffd640 1 -- 172.21.15.139:0/3004958157 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf400052d0 con 0x7fdf500cea00 2024-04-25T09:53:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf46ffd640 1 -- 172.21.15.139:0/3004958157 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf40005490 con 0x7fdf500cea00 2024-04-25T09:53:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf5855c640 1 -- 172.21.15.139:0/3004958157 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf50158470 con 0x7fdf500cea00 2024-04-25T09:53:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf46ffd640 1 -- 172.21.15.139:0/3004958157 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40012870 con 0x7fdf500cea00 2024-04-25T09:53:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.185+0000 7fdf5855c640 1 -- 172.21.15.139:0/3004958157 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf500c7db0 con 0x7fdf500cea00 2024-04-25T09:53:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.186+0000 7fdf5855c640 1 -- 172.21.15.139:0/3004958157 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf50074520 con 0x7fdf500cea00 2024-04-25T09:53:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.187+0000 7fdf46ffd640 1 -- 172.21.15.139:0/3004958157 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf40012a10 con 0x7fdf500cea00 2024-04-25T09:53:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.187+0000 7fdf46ffd640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf3406b4d0 0x7fdf3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.187+0000 7fdf46ffd640 1 -- 172.21.15.139:0/3004958157 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdf4008a230 con 0x7fdf500cea00 2024-04-25T09:53:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.187+0000 7fdf562f8640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf3406b4d0 0x7fdf3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.187+0000 7fdf562f8640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf3406b4d0 0x7fdf3406d990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdf48034000 tx=0x7fdf4802f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.191+0000 7fdf46ffd640 1 -- 172.21.15.139:0/3004958157 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf40058d80 con 0x7fdf500cea00 2024-04-25T09:53:32.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.405+0000 7fdf5855c640 1 -- 172.21.15.139:0/3004958157 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fdf5006ceb0 con 0x7fdf500cea00 2024-04-25T09:53:32.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.408+0000 7fdf46ffd640 1 -- 172.21.15.139:0/3004958157 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171788 (secure 0 0 0) 0x7fdf40058680 con 0x7fdf500cea00 2024-04-25T09:53:32.412 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:32.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.411+0000 7fdf5855c640 1 -- 172.21.15.139:0/3004958157 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf3406b4d0 msgr2=0x7fdf3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.411+0000 7fdf5855c640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf3406b4d0 0x7fdf3406d990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdf48034000 tx=0x7fdf4802f000 comp rx=0 tx=0).stop 2024-04-25T09:53:32.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.411+0000 7fdf5855c640 1 -- 172.21.15.139:0/3004958157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf500cea00 msgr2=0x7fdf500c9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.411+0000 7fdf5855c640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf500cea00 0x7fdf500c9870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000c500 tx=0x7fdf4000c9c0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.411+0000 7fdf5855c640 1 -- 172.21.15.139:0/3004958157 shutdown_connections 2024-04-25T09:53:32.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.411+0000 7fdf5855c640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf500cea00 0x7fdf500c9870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.411+0000 7fdf5855c640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf3406b4d0 0x7fdf3406d990 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.411+0000 7fdf5855c640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf501611e0 0x7fdf50157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.411+0000 7fdf5855c640 1 --2- 172.21.15.139:0/3004958157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf5006c780 0x7fdf500c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.411+0000 7fdf5855c640 1 -- 172.21.15.139:0/3004958157 >> 172.21.15.139:0/3004958157 conn(0x7fdf500b5b00 msgr2=0x7fdf500b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:32.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.411+0000 7fdf5855c640 1 -- 172.21.15.139:0/3004958157 shutdown_connections 2024-04-25T09:53:32.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.411+0000 7fdf5855c640 1 -- 172.21.15.139:0/3004958157 wait complete. 2024-04-25T09:53:32.425 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":839667050},{"type":"v1","addr":"172.21.15.139:6833","nonce":839667050}]},"active_addr":"172.21.15.139:6833/839667050","active_change":"2024-04-25T09:52:54.957626+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":658440282}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":1857393173}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":3678953864}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":3702663853}]}]} 2024-04-25T09:53:32.428 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T09:53:32.428 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T09:53:32.428 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:32.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.523+0000 7fb65142a640 1 Processor -- start 2024-04-25T09:53:32.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.523+0000 7fb65142a640 1 -- start start 2024-04-25T09:53:32.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.523+0000 7fb65142a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb64c15dc60 0x7fb64c0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.523+0000 7fb65142a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb64c161850 0x7fb64c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.523+0000 7fb65142a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb64c0d0970 0x7fb64c0d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.523+0000 7fb65142a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64c0ce780 con 0x7fb64c161850 2024-04-25T09:53:32.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.523+0000 7fb65142a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64c0d3730 con 0x7fb64c15dc60 2024-04-25T09:53:32.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.523+0000 7fb65142a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0cf250 con 0x7fb64c0d0970 2024-04-25T09:53:32.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.524+0000 7fb64a7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb64c161850 0x7fb64c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.524+0000 7fb64a7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb64c161850 0x7fb64c0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58676/0 (socket says 172.21.15.139:58676) 2024-04-25T09:53:32.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.524+0000 7fb64a7fc640 1 -- 172.21.15.139:0/3418827057 learned_addr learned my addr 172.21.15.139:0/3418827057 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:32.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.524+0000 7fb64affd640 1 --2- 172.21.15.139:0/3418827057 >> v2:172.21.15.150:3300/0 conn(0x7fb64c15dc60 0x7fb64c0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.524+0000 7fb64b7fe640 1 --2- 172.21.15.139:0/3418827057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb64c0d0970 0x7fb64c0d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.524+0000 7fb64a7fc640 1 -- 172.21.15.139:0/3418827057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb64c0d0970 msgr2=0x7fb64c0d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:32.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.524+0000 7fb64a7fc640 1 --2- 172.21.15.139:0/3418827057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb64c0d0970 0x7fb64c0d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.524+0000 7fb64a7fc640 1 -- 172.21.15.139:0/3418827057 >> v2:172.21.15.150:3300/0 conn(0x7fb64c15dc60 msgr2=0x7fb64c0b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:32.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.524+0000 7fb64a7fc640 1 --2- 172.21.15.139:0/3418827057 >> v2:172.21.15.150:3300/0 conn(0x7fb64c15dc60 0x7fb64c0b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.524+0000 7fb64a7fc640 1 -- 172.21.15.139:0/3418827057 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c0ce9b0 con 0x7fb64c161850 2024-04-25T09:53:32.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb64a7fc640 1 --2- 172.21.15.139:0/3418827057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c161850 0x7fb64c0ce240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb64c08c680 tx=0x7fb64002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e171a9c054df2cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb649ffb640 1 -- 172.21.15.139:0/3418827057 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb640037070 con 0x7fb64c161850 2024-04-25T09:53:32.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb649ffb640 1 -- 172.21.15.139:0/3418827057 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb640032440 con 0x7fb64c161850 2024-04-25T09:53:32.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb649ffb640 1 -- 172.21.15.139:0/3418827057 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb640032810 con 0x7fb64c161850 2024-04-25T09:53:32.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb65142a640 1 -- 172.21.15.139:0/3418827057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c161850 msgr2=0x7fb64c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb65142a640 1 --2- 172.21.15.139:0/3418827057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c161850 0x7fb64c0ce240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb64c08c680 tx=0x7fb64002a210 comp rx=0 tx=0).stop 2024-04-25T09:53:32.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb65142a640 1 -- 172.21.15.139:0/3418827057 shutdown_connections 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb65142a640 1 --2- 172.21.15.139:0/3418827057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb64c0d0970 0x7fb64c0d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb65142a640 1 --2- 172.21.15.139:0/3418827057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c161850 0x7fb64c0ce240 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb65142a640 1 --2- 172.21.15.139:0/3418827057 >> v2:172.21.15.150:3300/0 conn(0x7fb64c15dc60 0x7fb64c0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb65142a640 1 -- 172.21.15.139:0/3418827057 >> 172.21.15.139:0/3418827057 conn(0x7fb64c0b5910 msgr2=0x7fb64c0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb65142a640 1 -- 172.21.15.139:0/3418827057 shutdown_connections 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.525+0000 7fb65142a640 1 -- 172.21.15.139:0/3418827057 wait complete. 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.527+0000 7fb65142a640 1 Processor -- start 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.527+0000 7fb65142a640 1 -- start start 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.527+0000 7fb65142a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c0d0970 0x7fb64c0c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.527+0000 7fb65142a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb64c15dc60 0x7fb64c15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.527+0000 7fb65142a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb64c161850 0x7fb64c1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.527+0000 7fb65142a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0d1fc0 con 0x7fb64c0d0970 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.527+0000 7fb65142a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0cef70 con 0x7fb64c161850 2024-04-25T09:53:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.527+0000 7fb65142a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c15e040 con 0x7fb64c15dc60 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb64affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c0d0970 0x7fb64c0c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb64affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c0d0970 0x7fb64c0c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58680/0 (socket says 172.21.15.139:58680) 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb64affd640 1 -- 172.21.15.139:0/4104186879 learned_addr learned my addr 172.21.15.139:0/4104186879 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb64a7fc640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb64c15dc60 0x7fb64c15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb64b7fe640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb64c161850 0x7fb64c1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb64affd640 1 -- 172.21.15.139:0/4104186879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb64c15dc60 msgr2=0x7fb64c15e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb64affd640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb64c15dc60 0x7fb64c15e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb64affd640 1 -- 172.21.15.139:0/4104186879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb64c161850 msgr2=0x7fb64c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb64affd640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb64c161850 0x7fb64c1563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb64affd640 1 -- 172.21.15.139:0/4104186879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c160ec0 con 0x7fb64c0d0970 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb64a7fc640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb64c15dc60 0x7fb64c15e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb64affd640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c0d0970 0x7fb64c0c7740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00d620 tx=0x7fb63c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb6337fe640 1 -- 172.21.15.139:0/4104186879 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00b620 con 0x7fb64c0d0970 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb6337fe640 1 -- 172.21.15.139:0/4104186879 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c019070 con 0x7fb64c0d0970 2024-04-25T09:53:32.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb65142a640 1 -- 172.21.15.139:0/4104186879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64c1558d0 con 0x7fb64c0d0970 2024-04-25T09:53:32.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb6337fe640 1 -- 172.21.15.139:0/4104186879 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00ba00 con 0x7fb64c0d0970 2024-04-25T09:53:32.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.528+0000 7fb65142a640 1 -- 172.21.15.139:0/4104186879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64c155e90 con 0x7fb64c0d0970 2024-04-25T09:53:32.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.529+0000 7fb65142a640 1 -- 172.21.15.139:0/4104186879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64c074520 con 0x7fb64c0d0970 2024-04-25T09:53:32.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.530+0000 7fb6337fe640 1 -- 172.21.15.139:0/4104186879 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb63c00bba0 con 0x7fb64c0d0970 2024-04-25T09:53:32.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.530+0000 7fb6337fe640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb63806b4d0 0x7fb63806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.530+0000 7fb6337fe640 1 -- 172.21.15.139:0/4104186879 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb63c08a8d0 con 0x7fb64c0d0970 2024-04-25T09:53:32.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.530+0000 7fb64a7fc640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb63806b4d0 0x7fb63806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.530+0000 7fb64a7fc640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb63806b4d0 0x7fb63806d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb640034000 tx=0x7fb64002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.534+0000 7fb6337fe640 1 -- 172.21.15.139:0/4104186879 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb63c0594a0 con 0x7fb64c0d0970 2024-04-25T09:53:32.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.707+0000 7fb65142a640 1 -- 172.21.15.139:0/4104186879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb64c0d16f0 con 0x7fb64c0d0970 2024-04-25T09:53:32.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.708+0000 7fb6337fe640 1 -- 172.21.15.139:0/4104186879 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7fb63c058da0 con 0x7fb64c0d0970 2024-04-25T09:53:32.712 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:32.712 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:32.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.710+0000 7fb65142a640 1 -- 172.21.15.139:0/4104186879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb63806b4d0 msgr2=0x7fb63806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.710+0000 7fb65142a640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb63806b4d0 0x7fb63806d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb640034000 tx=0x7fb64002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:32.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.711+0000 7fb65142a640 1 -- 172.21.15.139:0/4104186879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c0d0970 msgr2=0x7fb64c0c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.711+0000 7fb65142a640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c0d0970 0x7fb64c0c7740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00d620 tx=0x7fb63c00dae0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.711+0000 7fb65142a640 1 -- 172.21.15.139:0/4104186879 shutdown_connections 2024-04-25T09:53:32.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.711+0000 7fb65142a640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb63806b4d0 0x7fb63806d990 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.711+0000 7fb65142a640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb64c161850 0x7fb64c1563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.711+0000 7fb65142a640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb64c15dc60 0x7fb64c15e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.711+0000 7fb65142a640 1 --2- 172.21.15.139:0/4104186879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb64c0d0970 0x7fb64c0c7740 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.711+0000 7fb65142a640 1 -- 172.21.15.139:0/4104186879 >> 172.21.15.139:0/4104186879 conn(0x7fb64c0b5910 msgr2=0x7fb64c15c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:32.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.711+0000 7fb65142a640 1 -- 172.21.15.139:0/4104186879 shutdown_connections 2024-04-25T09:53:32.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.711+0000 7fb65142a640 1 -- 172.21.15.139:0/4104186879 wait complete. 2024-04-25T09:53:32.725 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T09:53:32.725 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:32.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.819+0000 7f992c088640 1 Processor -- start 2024-04-25T09:53:32.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.820+0000 7f992c088640 1 -- start start 2024-04-25T09:53:32.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.820+0000 7f992c088640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f992415dc60 0x7f99240b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f992c088640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9924161850 0x7f99240ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f992c088640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99240cf250 con 0x7f99240d0970 2024-04-25T09:53:32.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f992c088640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99240ce780 con 0x7f9924161850 2024-04-25T09:53:32.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f992c088640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99240d3730 con 0x7f992415dc60 2024-04-25T09:53:32.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f9929623640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9924161850 0x7f99240ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f992a625640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f99240d0970 0x7f99240d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46478/0 (socket says 172.21.15.139:46478) 2024-04-25T09:53:32.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f992a625640 1 -- 172.21.15.139:0/3518117211 learned_addr learned my addr 172.21.15.139:0/3518117211 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:32.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f9929e24640 1 --2- 172.21.15.139:0/3518117211 >> v2:172.21.15.150:3300/0 conn(0x7f992415dc60 0x7f99240b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f9929623640 1 -- 172.21.15.139:0/3518117211 >> v2:172.21.15.150:3300/0 conn(0x7f992415dc60 msgr2=0x7f99240b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:32.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f9929623640 1 --2- 172.21.15.139:0/3518117211 >> v2:172.21.15.150:3300/0 conn(0x7f992415dc60 0x7f99240b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f9929623640 1 -- 172.21.15.139:0/3518117211 >> v1:172.21.15.139:6789/0 conn(0x7f99240d0970 legacy=0x7f99240d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f9929623640 1 -- 172.21.15.139:0/3518117211 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99240ce9b0 con 0x7f9924161850 2024-04-25T09:53:32.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f9928e22640 1 -- 172.21.15.139:0/3518117211 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574084426 0 0) 0x7f991c001490 con 0x7f99240d0970 2024-04-25T09:53:32.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.821+0000 7f9929e24640 1 --2- 172.21.15.139:0/3518117211 >> v2:172.21.15.150:3300/0 conn(0x7f992415dc60 0x7f99240b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:32.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.822+0000 7f9929623640 1 --2- 172.21.15.139:0/3518117211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924161850 0x7f99240ce240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f992408c680 tx=0x7f992002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d963b76f5db48435 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.822+0000 7f9928e22640 1 -- 172.21.15.139:0/3518117211 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920037070 con 0x7f9924161850 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.822+0000 7f9928e22640 1 -- 172.21.15.139:0/3518117211 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9920032440 con 0x7f9924161850 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.822+0000 7f9928e22640 1 -- 172.21.15.139:0/3518117211 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920032810 con 0x7f9924161850 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.822+0000 7f992c088640 1 -- 172.21.15.139:0/3518117211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924161850 msgr2=0x7f99240ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.822+0000 7f992c088640 1 --2- 172.21.15.139:0/3518117211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924161850 0x7f99240ce240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f992408c680 tx=0x7f992002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.822+0000 7f992c088640 1 -- 172.21.15.139:0/3518117211 shutdown_connections 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.822+0000 7f992c088640 1 --2- 172.21.15.139:0/3518117211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9924161850 0x7f99240ce240 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.822+0000 7f992c088640 1 --2- 172.21.15.139:0/3518117211 >> v2:172.21.15.150:3300/0 conn(0x7f992415dc60 0x7f99240b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.822+0000 7f992c088640 1 -- 172.21.15.139:0/3518117211 >> 172.21.15.139:0/3518117211 conn(0x7f99240b5910 msgr2=0x7f99240b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.823+0000 7f992c088640 1 -- 172.21.15.139:0/3518117211 shutdown_connections 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.823+0000 7f992c088640 1 -- 172.21.15.139:0/3518117211 wait complete. 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.824+0000 7f992c088640 1 Processor -- start 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.824+0000 7f992c088640 1 -- start start 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.824+0000 7f992c088640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99240d0970 0x7f9924150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.824+0000 7f992c088640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f992415dc60 0x7f99240c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.824+0000 7f992c088640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9924161850 0x7f99241e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.824+0000 7f992c088640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99240d1fc0 con 0x7f99240d0970 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.824+0000 7f992c088640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99240cef70 con 0x7f9924161850 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.824+0000 7f992c088640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992415e040 con 0x7f992415dc60 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.824+0000 7f9929e24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99240d0970 0x7f9924150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.824+0000 7f9929e24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99240d0970 0x7f9924150780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58698/0 (socket says 172.21.15.139:58698) 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.824+0000 7f9929e24640 1 -- 172.21.15.139:0/3158370201 learned_addr learned my addr 172.21.15.139:0/3158370201 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.825+0000 7f9929e24640 1 -- 172.21.15.139:0/3158370201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f992415dc60 msgr2=0x7f99240c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.825+0000 7f9929e24640 1 --2- 172.21.15.139:0/3158370201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f992415dc60 0x7f99240c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.825+0000 7f9929e24640 1 -- 172.21.15.139:0/3158370201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9924161850 msgr2=0x7f99241e7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.825+0000 7f9929e24640 1 --2- 172.21.15.139:0/3158370201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9924161850 0x7f99241e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.825+0000 7f9929e24640 1 -- 172.21.15.139:0/3158370201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9924158380 con 0x7f99240d0970 2024-04-25T09:53:32.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.825+0000 7f9929e24640 1 --2- 172.21.15.139:0/3158370201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99240d0970 0x7f9924150780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f991400e7e0 tx=0x7f991400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.825+0000 7f991a7fc640 1 -- 172.21.15.139:0/3158370201 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991400f040 con 0x7f99240d0970 2024-04-25T09:53:32.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.825+0000 7f991a7fc640 1 -- 172.21.15.139:0/3158370201 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9914004030 con 0x7f99240d0970 2024-04-25T09:53:32.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.825+0000 7f992c088640 1 -- 172.21.15.139:0/3158370201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99241586c0 con 0x7f99240d0970 2024-04-25T09:53:32.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.825+0000 7f991a7fc640 1 -- 172.21.15.139:0/3158370201 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914004400 con 0x7f99240d0970 2024-04-25T09:53:32.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.825+0000 7f992c088640 1 -- 172.21.15.139:0/3158370201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9924158f00 con 0x7f99240d0970 2024-04-25T09:53:32.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.826+0000 7f992c088640 1 -- 172.21.15.139:0/3158370201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9924074520 con 0x7f99240d0970 2024-04-25T09:53:32.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.826+0000 7f991a7fc640 1 -- 172.21.15.139:0/3158370201 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f991400c670 con 0x7f99240d0970 2024-04-25T09:53:32.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.827+0000 7f991a7fc640 1 --2- 172.21.15.139:0/3158370201 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f990806b4d0 0x7f990806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:32.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.827+0000 7f991a7fc640 1 -- 172.21.15.139:0/3158370201 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f991408a380 con 0x7f99240d0970 2024-04-25T09:53:32.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.827+0000 7f9929623640 1 --2- 172.21.15.139:0/3158370201 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f990806b4d0 0x7f990806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:32.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.827+0000 7f9929623640 1 --2- 172.21.15.139:0/3158370201 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f990806b4d0 0x7f990806d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9920034000 tx=0x7f992002c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:32.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:32.831+0000 7f991a7fc640 1 -- 172.21.15.139:0/3158370201 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99241586c0 con 0x7f99240d0970 2024-04-25T09:53:33.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.005+0000 7f992c088640 1 -- 172.21.15.139:0/3158370201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99240d16f0 con 0x7f99240d0970 2024-04-25T09:53:33.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.006+0000 7f991a7fc640 1 -- 172.21.15.139:0/3158370201 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f9914017020 con 0x7f99240d0970 2024-04-25T09:53:33.009 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:33.009 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:33.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.008+0000 7f992c088640 1 -- 172.21.15.139:0/3158370201 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f990806b4d0 msgr2=0x7f990806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.008+0000 7f992c088640 1 --2- 172.21.15.139:0/3158370201 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f990806b4d0 0x7f990806d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9920034000 tx=0x7f992002c000 comp rx=0 tx=0).stop 2024-04-25T09:53:33.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.008+0000 7f992c088640 1 -- 172.21.15.139:0/3158370201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99240d0970 msgr2=0x7f9924150780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.008+0000 7f992c088640 1 --2- 172.21.15.139:0/3158370201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99240d0970 0x7f9924150780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f991400e7e0 tx=0x7f991400eca0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.008+0000 7f992c088640 1 -- 172.21.15.139:0/3158370201 shutdown_connections 2024-04-25T09:53:33.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.008+0000 7f992c088640 1 --2- 172.21.15.139:0/3158370201 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f990806b4d0 0x7f990806d990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.008+0000 7f992c088640 1 --2- 172.21.15.139:0/3158370201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9924161850 0x7f99241e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.008+0000 7f992c088640 1 --2- 172.21.15.139:0/3158370201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f992415dc60 0x7f99240c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.008+0000 7f992c088640 1 --2- 172.21.15.139:0/3158370201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99240d0970 0x7f9924150780 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.008+0000 7f992c088640 1 -- 172.21.15.139:0/3158370201 >> 172.21.15.139:0/3158370201 conn(0x7f99240b5910 msgr2=0x7f992415c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.008+0000 7f992c088640 1 -- 172.21.15.139:0/3158370201 shutdown_connections 2024-04-25T09:53:33.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.008+0000 7f992c088640 1 -- 172.21.15.139:0/3158370201 wait complete. 2024-04-25T09:53:33.022 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T09:53:33.023 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T09:53:33.023 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T09:53:33.024 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T09:53:33.024 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T09:53:33.024 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T09:53:33.025 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T09:53:33.025 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T09:53:33.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.184+0000 7f8ed9815640 1 Processor -- start 2024-04-25T09:53:33.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.184+0000 7f8ed9815640 1 -- start start 2024-04-25T09:53:33.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.185+0000 7f8ed9815640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8ecc0df720 0x7f8ecc054d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.185+0000 7f8ed9815640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ecc053c60 con 0x7f8ecc03ad80 2024-04-25T09:53:33.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.185+0000 7f8ed9815640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ecc053f40 con 0x7f8ecc0500c0 2024-04-25T09:53:33.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.185+0000 7f8ed9815640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ecc054220 con 0x7f8ecc0df720 2024-04-25T09:53:33.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.186+0000 7f8ed2ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8ecc0df720 0x7f8ecc054d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.186+0000 7f8ed27fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8ecc03ad80 0x7f8ecc053430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46492/0 (socket says 172.21.15.139:46492) 2024-04-25T09:53:33.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.186+0000 7f8ed27fc640 1 -- 172.21.15.139:0/1114122538 learned_addr learned my addr 172.21.15.139:0/1114122538 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:33.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.186+0000 7f8ed2ffd640 1 -- 172.21.15.139:0/1114122538 >> v1:172.21.15.150:6789/0 conn(0x7f8ecc0500c0 legacy=0x7f8ecc0504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.186+0000 7f8ed2ffd640 1 -- 172.21.15.139:0/1114122538 >> v1:172.21.15.139:6789/0 conn(0x7f8ecc03ad80 legacy=0x7f8ecc053430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.186+0000 7f8ed2ffd640 1 -- 172.21.15.139:0/1114122538 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc054500 con 0x7f8ecc0df720 2024-04-25T09:53:33.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.186+0000 7f8ed1ffb640 1 -- 172.21.15.139:0/1114122538 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369320588 0 0) 0x7f8ecc053c60 con 0x7f8ecc03ad80 2024-04-25T09:53:33.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.187+0000 7f8ed2ffd640 1 --2- 172.21.15.139:0/1114122538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ecc0df720 0x7f8ecc054d00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40097a0 tx=0x7f8ec402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f1796da6f27a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.188+0000 7f8ed1ffb640 1 -- 172.21.15.139:0/1114122538 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4037070 con 0x7f8ecc0df720 2024-04-25T09:53:33.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.188+0000 7f8ed1ffb640 1 -- 172.21.15.139:0/1114122538 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec4032440 con 0x7f8ecc0df720 2024-04-25T09:53:33.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.188+0000 7f8ed1ffb640 1 -- 172.21.15.139:0/1114122538 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4032810 con 0x7f8ecc0df720 2024-04-25T09:53:33.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.188+0000 7f1c6a2f5640 1 Processor -- start 2024-04-25T09:53:33.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.188+0000 7f1c6a2f5640 1 -- start start 2024-04-25T09:53:33.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f6f4d74e640 1 Processor -- start 2024-04-25T09:53:33.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f8ed9815640 1 -- 172.21.15.139:0/1114122538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ecc0df720 msgr2=0x7f8ecc054d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f8ed9815640 1 --2- 172.21.15.139:0/1114122538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ecc0df720 0x7f8ecc054d00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40097a0 tx=0x7f8ec402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f1c6a2f5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1c5c0eaca0 0x7f1c5c0526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f1c6a2f5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1c5c03b0e0 0x7f1c5c0483f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f1c6a2f5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c03a320 con 0x7f1c5c0eb080 2024-04-25T09:53:33.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f1c6a2f5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c052bf0 con 0x7f1c5c0eaca0 2024-04-25T09:53:33.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f1c6a2f5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c03a0d0 con 0x7f1c5c03b0e0 2024-04-25T09:53:33.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f8ed9815640 1 -- 172.21.15.139:0/1114122538 shutdown_connections 2024-04-25T09:53:33.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7fc64d65d640 1 Processor -- start 2024-04-25T09:53:33.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f1c63fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1c5c0eaca0 0x7f1c5c0526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f1c63fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1c5c0eaca0 0x7f1c5c0526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58728/0 (socket says 172.21.15.139:58728) 2024-04-25T09:53:33.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f1c637fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1c5c03b0e0 0x7f1c5c0483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f1c63fff640 1 -- 172.21.15.139:0/1831397822 learned_addr learned my addr 172.21.15.139:0/1831397822 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.189+0000 7f6f4d74e640 1 -- start start 2024-04-25T09:53:33.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.190+0000 7f6f4d74e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f4806d330 0x7f6f481626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.190+0000 7f6f4d74e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f48165b40 con 0x7f6f4806d710 2024-04-25T09:53:33.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.190+0000 7f6f4d74e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f48165e20 con 0x7f6f48144550 2024-04-25T09:53:33.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.190+0000 7f6f4d74e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f481450d0 con 0x7f6f4806d330 2024-04-25T09:53:33.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.190+0000 7f8ed9815640 1 --2- 172.21.15.139:0/1114122538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ecc0df720 0x7f8ecc054d00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.190+0000 7f8ed9815640 1 -- 172.21.15.139:0/1114122538 >> 172.21.15.139:0/1114122538 conn(0x7f8ecc055380 msgr2=0x7f8ecc055790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7f6f477fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f6f4806d710 0x7f6f48162bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46508/0 (socket says 172.21.15.139:46508) 2024-04-25T09:53:33.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7f6f477fe640 1 -- 172.21.15.139:0/550702702 learned_addr learned my addr 172.21.15.139:0/550702702 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:33.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.190+0000 7f1c63fff640 1 -- 172.21.15.139:0/1831397822 >> v2:172.21.15.150:3300/0 conn(0x7f1c5c03b0e0 msgr2=0x7f1c5c0483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7f6f47fff640 1 --2- 172.21.15.139:0/550702702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f4806d330 0x7f6f481626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.190+0000 7f1c63fff640 1 --2- 172.21.15.139:0/1831397822 >> v2:172.21.15.150:3300/0 conn(0x7f1c5c03b0e0 0x7f1c5c0483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.190+0000 7f1c63fff640 1 -- 172.21.15.139:0/1831397822 >> v1:172.21.15.139:6789/0 conn(0x7f1c5c0eb080 legacy=0x7f1c5c035090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.190+0000 7f1c63fff640 1 -- 172.21.15.139:0/1831397822 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c03a600 con 0x7f1c5c0eaca0 2024-04-25T09:53:33.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.190+0000 7fc64d65d640 1 -- start start 2024-04-25T09:53:33.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7fc64d65d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc6480b93b0 0x7fc6480ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7fc64d65d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc6480ced30 0x7fc6480d2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7fc64d65d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6480cf140 con 0x7fc64815dba0 2024-04-25T09:53:33.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7fc64d65d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6480d21b0 con 0x7fc6480b93b0 2024-04-25T09:53:33.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7fc64d65d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6480d1e60 con 0x7fc6480ced30 2024-04-25T09:53:33.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7fc6467fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc6480b93b0 0x7fc6480ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7fa3b445a640 1 Processor -- start 2024-04-25T09:53:33.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7fa3b445a640 1 -- start start 2024-04-25T09:53:33.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7f6f46ffd640 1 -- 172.21.15.139:0/550702702 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740722935 0 0) 0x7f6f48165e20 con 0x7f6f48144550 2024-04-25T09:53:33.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7f6f46ffd640 1 -- 172.21.15.139:0/550702702 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f34003520 con 0x7f6f48144550 2024-04-25T09:53:33.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7f6f46ffd640 1 -- 172.21.15.139:0/550702702 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898177357 0 0) 0x7f6f48165b40 con 0x7f6f4806d710 2024-04-25T09:53:33.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7f6f46ffd640 1 -- 172.21.15.139:0/550702702 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f48165e20 con 0x7f6f4806d710 2024-04-25T09:53:33.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7f1c63fff640 1 --2- 172.21.15.139:0/1831397822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c0eaca0 0x7f1c5c0526b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1c6402d370 tx=0x7f1c640a4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d88aab7d8dfdde9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7f6f46ffd640 1 -- 172.21.15.139:0/550702702 <== mon.1 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 225998076 0 0) 0x7f6f34003520 con 0x7f6f48144550 2024-04-25T09:53:33.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7fc6467fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc6480b93b0 0x7fc6480ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58732/0 (socket says 172.21.15.139:58732) 2024-04-25T09:53:33.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7fc6467fc640 1 -- 172.21.15.139:0/1467324027 learned_addr learned my addr 172.21.15.139:0/1467324027 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.191+0000 7fc6477fe640 1 --2- 172.21.15.139:0/1467324027 >> v2:172.21.15.150:3300/0 conn(0x7fc6480ced30 0x7fc6480d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7f6f47fff640 1 -- 172.21.15.139:0/550702702 >> v1:172.21.15.150:6789/0 conn(0x7f6f48144550 legacy=0x7f6f481449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7f765a35c640 1 Processor -- start 2024-04-25T09:53:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7f6f47fff640 1 -- 172.21.15.139:0/550702702 >> v1:172.21.15.139:6789/0 conn(0x7f6f4806d710 legacy=0x7f6f48162bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7f6f47fff640 1 -- 172.21.15.139:0/550702702 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f48166100 con 0x7f6f4806d330 2024-04-25T09:53:33.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7f8ed9815640 1 -- 172.21.15.139:0/1114122538 shutdown_connections 2024-04-25T09:53:33.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7f765a35c640 1 -- start start 2024-04-25T09:53:33.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7fc6477fe640 1 -- 172.21.15.139:0/1467324027 >> v2:172.21.15.139:3300/0 conn(0x7fc6480b93b0 msgr2=0x7fc6480ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7fc6477fe640 1 --2- 172.21.15.139:0/1467324027 >> v2:172.21.15.139:3300/0 conn(0x7fc6480b93b0 0x7fc6480ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7fc6477fe640 1 -- 172.21.15.139:0/1467324027 >> v1:172.21.15.150:6789/0 conn(0x7fc64815dba0 legacy=0x7fc648161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7fc6477fe640 1 -- 172.21.15.139:0/1467324027 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc648162050 con 0x7fc6480ced30 2024-04-25T09:53:33.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7fa3b445a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa3ac154090 0x7fa3ac0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7f8ed9815640 1 -- 172.21.15.139:0/1114122538 wait complete. 2024-04-25T09:53:33.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.192+0000 7f1c62ffd640 1 -- 172.21.15.139:0/1831397822 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640b2070 con 0x7f1c5c0eaca0 2024-04-25T09:53:33.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7f1c62ffd640 1 -- 172.21.15.139:0/1831397822 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c640a5ce0 con 0x7f1c5c0eaca0 2024-04-25T09:53:33.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7fa3b445a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa3ac0711c0 0x7fa3ac071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7fa3b445a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ac154470 con 0x7fa3ac0b2660 2024-04-25T09:53:33.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7fa3b21f6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa3ac154090 0x7fa3ac0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7fa3b21f6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa3ac154090 0x7fa3ac0b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58734/0 (socket says 172.21.15.139:58734) 2024-04-25T09:53:33.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7f79ad5b0640 1 Processor -- start 2024-04-25T09:53:33.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7fa3b21f6640 1 -- 172.21.15.139:0/498279114 learned_addr learned my addr 172.21.15.139:0/498279114 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7f79ad5b0640 1 -- start start 2024-04-25T09:53:33.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7fa3b19f5640 1 --2- 172.21.15.139:0/498279114 >> v2:172.21.15.150:3300/0 conn(0x7fa3ac0711c0 0x7fa3ac071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7f6f47fff640 1 --2- 172.21.15.139:0/550702702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f4806d330 0x7f6f481626b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009dc0 tx=0x7f6f4002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f57f9c2f43d4cc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7fa3b445a640 1 -- 172.21.15.139:0/498279114 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ac0b4b40 con 0x7fa3ac154090 2024-04-25T09:53:33.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7fa3b445a640 1 -- 172.21.15.139:0/498279114 --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ac0b4170 con 0x7fa3ac0711c0 2024-04-25T09:53:33.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602229643 0 0) 0x7fa3ac154470 con 0x7fa3ac0b2660 2024-04-25T09:53:33.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7f765a35c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f76540ce900 0x7f76540b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7f765a35c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f76540b25d0 0x7f76540b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7f765a35c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76540b1c10 con 0x7f765406dbd0 2024-04-25T09:53:33.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7f765a35c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76540b19c0 con 0x7f76540b25d0 2024-04-25T09:53:33.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7f765a35c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76540b1ef0 con 0x7f76540ce900 2024-04-25T09:53:33.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7f6f46ffd640 1 -- 172.21.15.139:0/550702702 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f30001670 con 0x7f6f4806d330 2024-04-25T09:53:33.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f6f46ffd640 1 -- 172.21.15.139:0/550702702 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f40007540 con 0x7f6f4806d330 2024-04-25T09:53:33.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa398003500 con 0x7fa3ac0b2660 2024-04-25T09:53:33.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/1831397822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c0eaca0 msgr2=0x7f1c5c0526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.193+0000 7f1c6a2f5640 1 --2- 172.21.15.139:0/1831397822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c0eaca0 0x7f1c5c0526b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1c6402d370 tx=0x7f1c640a4f90 comp rx=0 tx=0).stop 2024-04-25T09:53:33.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/1831397822 shutdown_connections 2024-04-25T09:53:33.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f1c6a2f5640 1 --2- 172.21.15.139:0/1831397822 >> v2:172.21.15.150:3300/0 conn(0x7f1c5c03b0e0 0x7f1c5c0483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f1c6a2f5640 1 --2- 172.21.15.139:0/1831397822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c0eaca0 0x7f1c5c0526b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/1831397822 >> 172.21.15.139:0/1831397822 conn(0x7f1c5c01e6f0 msgr2=0x7f1c5c055420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/1831397822 shutdown_connections 2024-04-25T09:53:33.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/1831397822 wait complete. 2024-04-25T09:53:33.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 <== mon.1 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 122532481 0 0) 0x7fa398003500 con 0x7fa3ac0b2660 2024-04-25T09:53:33.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 >> v2:172.21.15.150:3300/0 conn(0x7fa3ac0711c0 msgr2=0x7fa3ac071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 --2- 172.21.15.139:0/498279114 >> v2:172.21.15.150:3300/0 conn(0x7fa3ac0711c0 0x7fa3ac071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 >> v2:172.21.15.139:3300/0 conn(0x7fa3ac154090 msgr2=0x7fa3ac0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 --2- 172.21.15.139:0/498279114 >> v2:172.21.15.139:3300/0 conn(0x7fa3ac154090 0x7fa3ac0b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0b3390 con 0x7fa3ac0b2660 2024-04-25T09:53:33.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 <== mon.1 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fa3a40019f0 con 0x7fa3ac0b2660 2024-04-25T09:53:33.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 >> v1:172.21.15.150:6789/0 conn(0x7fa3ac0b2660 legacy=0x7fa3ac0b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 --2- 172.21.15.139:0/498279114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa398003cd0 0x7fa3980060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 --2- 172.21.15.139:0/498279114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa398006650 0x7fa398008ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 --2- 172.21.15.139:0/498279114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3980091d0 0x7fa39800b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0b4b40 con 0x7fa398003cd0 2024-04-25T09:53:33.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0b4170 con 0x7fa398006650 2024-04-25T09:53:33.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39800bff0 con 0x7fa3980091d0 2024-04-25T09:53:33.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f07b67cf640 1 Processor -- start 2024-04-25T09:53:33.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f07b67cf640 1 -- start start 2024-04-25T09:53:33.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f76588f9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f76540b25d0 0x7f76540b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f76588f9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f76540b25d0 0x7f76540b2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58740/0 (socket says 172.21.15.139:58740) 2024-04-25T09:53:33.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f76588f9640 1 -- 172.21.15.139:0/1492693777 learned_addr learned my addr 172.21.15.139:0/1492693777 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fc6477fe640 1 --2- 172.21.15.139:0/1467324027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6480ced30 0x7fc6480d2640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc6480b9150 tx=0x7fc63c02a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4afe123d30c7d027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f8ed9815640 1 Processor -- start 2024-04-25T09:53:33.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f6f46ffd640 1 -- 172.21.15.139:0/550702702 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f400078f0 con 0x7f6f4806d330 2024-04-25T09:53:33.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7fa3b29f7640 1 --2- 172.21.15.139:0/498279114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3980091d0 0x7fa39800b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.194+0000 7f76537fe640 1 --2- 172.21.15.139:0/1492693777 >> v2:172.21.15.150:3300/0 conn(0x7f76540ce900 0x7f76540b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7fa3b19f5640 1 --2- 172.21.15.139:0/498279114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa398006650 0x7fa398008ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f6f4d74e640 1 -- 172.21.15.139:0/550702702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f4806d330 msgr2=0x7f6f481626b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f6f4d74e640 1 --2- 172.21.15.139:0/550702702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f4806d330 0x7f6f481626b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009dc0 tx=0x7f6f4002a680 comp rx=0 tx=0).stop 2024-04-25T09:53:33.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7fc645ffb640 1 -- 172.21.15.139:0/1467324027 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c037070 con 0x7fc6480ced30 2024-04-25T09:53:33.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7fc645ffb640 1 -- 172.21.15.139:0/1467324027 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63c032440 con 0x7fc6480ced30 2024-04-25T09:53:33.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7fc645ffb640 1 -- 172.21.15.139:0/1467324027 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c032810 con 0x7fc6480ced30 2024-04-25T09:53:33.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f6f4d74e640 1 -- 172.21.15.139:0/550702702 shutdown_connections 2024-04-25T09:53:33.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f6f4d74e640 1 --2- 172.21.15.139:0/550702702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f4806d330 0x7f6f481626b0 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009dc0 tx=0x7f6f4002a680 comp rx=0 tx=0).stop 2024-04-25T09:53:33.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f6f4d74e640 1 -- 172.21.15.139:0/550702702 >> 172.21.15.139:0/550702702 conn(0x7f6f48066f20 msgr2=0x7f6f4806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f79ad5b0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f79a815a760 0x7f79a80aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7fa3b29f7640 1 -- 172.21.15.139:0/498279114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa398006650 msgr2=0x7fa398008ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7fa3b29f7640 1 --2- 172.21.15.139:0/498279114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa398006650 0x7fa398008ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7fa3b29f7640 1 -- 172.21.15.139:0/498279114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa398003cd0 msgr2=0x7fa3980060c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7fa3b29f7640 1 --2- 172.21.15.139:0/498279114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa398003cd0 0x7fa3980060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7fa3b29f7640 1 -- 172.21.15.139:0/498279114 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0b3390 con 0x7fa3980091d0 2024-04-25T09:53:33.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f76588f9640 1 -- 172.21.15.139:0/1492693777 >> v2:172.21.15.150:3300/0 conn(0x7f76540ce900 msgr2=0x7f76540b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f76588f9640 1 --2- 172.21.15.139:0/1492693777 >> v2:172.21.15.150:3300/0 conn(0x7f76540ce900 0x7f76540b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f76588f9640 1 -- 172.21.15.139:0/1492693777 >> v1:172.21.15.150:6789/0 conn(0x7f765406dbd0 legacy=0x7f76541596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f76588f9640 1 -- 172.21.15.139:0/1492693777 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76540b42c0 con 0x7f76540b25d0 2024-04-25T09:53:33.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f79ad5b0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f79a80726c0 0x7f79a80cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f6f4d74e640 1 -- 172.21.15.139:0/550702702 shutdown_connections 2024-04-25T09:53:33.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f6f4d74e640 1 -- 172.21.15.139:0/550702702 wait complete. 2024-04-25T09:53:33.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f79ad5b0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a806e860 con 0x7f79a80b4e70 2024-04-25T09:53:33.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f79ad5b0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a806eb40 con 0x7f79a80726c0 2024-04-25T09:53:33.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f79ad5b0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a806ee20 con 0x7f79a815a760 2024-04-25T09:53:33.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f79a7fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f79a815a760 0x7f79a80aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f79a7fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f79a815a760 0x7f79a80aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:45866/0 (socket says 172.21.15.139:45866) 2024-04-25T09:53:33.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f79a7fff640 1 -- 172.21.15.139:0/3917356785 learned_addr learned my addr 172.21.15.139:0/3917356785 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f1c6a2f5640 1 Processor -- start 2024-04-25T09:53:33.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f07b67cf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07b0165920 0x7f07b0165d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7fa3b29f7640 1 -- 172.21.15.139:0/498279114 reap_dead start 2024-04-25T09:53:33.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f79acdaf640 1 --2- 172.21.15.139:0/3917356785 >> v2:172.21.15.139:3300/0 conn(0x7f79a80726c0 0x7f79a80cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f07affff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07b0165920 0x7f07b0165d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f79a7fff640 1 -- 172.21.15.139:0/3917356785 >> v2:172.21.15.139:3300/0 conn(0x7f79a80726c0 msgr2=0x7f79a80cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f79a7fff640 1 --2- 172.21.15.139:0/3917356785 >> v2:172.21.15.139:3300/0 conn(0x7f79a80726c0 0x7f79a80cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f76588f9640 1 --2- 172.21.15.139:0/1492693777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76540b25d0 0x7f76540b2a40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7654070f60 tx=0x7f764402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11250e2aeaf8bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.195+0000 7f8ed9815640 1 -- start start 2024-04-25T09:53:33.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f79a7fff640 1 -- 172.21.15.139:0/3917356785 >> v1:172.21.15.150:6789/0 conn(0x7f79a80b4e70 legacy=0x7f79a8071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f79a7fff640 1 -- 172.21.15.139:0/3917356785 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a806f100 con 0x7f79a815a760 2024-04-25T09:53:33.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f07affff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07b0165920 0x7f07b0165d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57094/0 (socket says 172.21.15.139:57094) 2024-04-25T09:53:33.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f1c6a2f5640 1 -- start start 2024-04-25T09:53:33.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7fa3b29f7640 1 --2- 172.21.15.139:0/498279114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3980091d0 0x7fa39800b690 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400e7c0 tx=0x7fa3980037f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a1cb6c11dd3f000 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f7652ffd640 1 -- 172.21.15.139:0/1492693777 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644037070 con 0x7f76540b25d0 2024-04-25T09:53:33.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f7652ffd640 1 -- 172.21.15.139:0/1492693777 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7644032440 con 0x7f76540b25d0 2024-04-25T09:53:33.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f7652ffd640 1 -- 172.21.15.139:0/1492693777 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644032810 con 0x7f76540b25d0 2024-04-25T09:53:33.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.196+0000 7f07affff640 1 -- 172.21.15.139:0/1748730974 learned_addr learned my addr 172.21.15.139:0/1748730974 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07b67cf640 1 --2- 172.21.15.139:0/1748730974 >> v2:172.21.15.150:3300/0 conn(0x7f07b01621b0 0x7f07b0162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07b67cf640 1 -- 172.21.15.139:0/1748730974 --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b00cedb0 con 0x7f07b00b65a0 2024-04-25T09:53:33.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f6f4d74e640 1 Processor -- start 2024-04-25T09:53:33.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07b4d6c640 1 --2- 172.21.15.139:0/1748730974 >> v2:172.21.15.150:3300/0 conn(0x7f07b01621b0 0x7f07b0162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7fc64d65d640 1 -- 172.21.15.139:0/1467324027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6480ced30 msgr2=0x7fc6480d2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7fc64d65d640 1 --2- 172.21.15.139:0/1467324027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6480ced30 0x7fc6480d2640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc6480b9150 tx=0x7fc63c02a160 comp rx=0 tx=0).stop 2024-04-25T09:53:33.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07b67cf640 1 -- 172.21.15.139:0/1748730974 --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b00cf890 con 0x7f07b01621b0 2024-04-25T09:53:33.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f8ed9815640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc03ad80 0x7f8ecc0d95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f6f4d74e640 1 -- start start 2024-04-25T09:53:33.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f1c6a2f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c03b0e0 0x7f1c5c0dd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a400e390 con 0x7fa3980091d0 2024-04-25T09:53:33.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07b67cf640 1 -- 172.21.15.139:0/1748730974 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00b8e10 con 0x7f07b0165920 2024-04-25T09:53:33.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237118523 0 0) 0x7f07b00cedb0 con 0x7f07b00b65a0 2024-04-25T09:53:33.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0798003520 con 0x7f07b00b65a0 2024-04-25T09:53:33.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a402ddf0 con 0x7fa3980091d0 2024-04-25T09:53:33.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f8ed9815640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc0500c0 0x7f8ecc047da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f8ed9815640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ecc0df720 0x7f8ecc168730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f8ed9815640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc006410 con 0x7f8ecc0df720 2024-04-25T09:53:33.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f8ed9815640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc006290 con 0x7f8ecc03ad80 2024-04-25T09:53:33.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f8ed9815640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc006590 con 0x7f8ecc0500c0 2024-04-25T09:53:33.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f8ed27fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc0500c0 0x7f8ecc047da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f8ed27fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc0500c0 0x7f8ecc047da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57106/0 (socket says 172.21.15.139:57106) 2024-04-25T09:53:33.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f8ed27fc640 1 -- 172.21.15.139:0/2101483707 learned_addr learned my addr 172.21.15.139:0/2101483707 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f8ed2ffd640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc03ad80 0x7f8ecc0d95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f1c63fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c03b0e0 0x7f1c5c0dd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f1c63fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c03b0e0 0x7f1c5c0dd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58770/0 (socket says 172.21.15.139:58770) 2024-04-25T09:53:33.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f79a7fff640 1 --2- 172.21.15.139:0/3917356785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79a815a760 0x7f79a80aefe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79a8072b00 tx=0x7f799802c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6537fe87e97099bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 <== mon.1 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 743013025 0 0) 0x7f0798003520 con 0x7f07b00b65a0 2024-04-25T09:53:33.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07b0165920 msgr2=0x7f07b0165d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 --2- 172.21.15.139:0/1748730974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07b0165920 0x7f07b0165d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 >> v2:172.21.15.150:3300/0 conn(0x7f07b01621b0 msgr2=0x7f07b0162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 --2- 172.21.15.139:0/1748730974 >> v2:172.21.15.150:3300/0 conn(0x7f07b01621b0 0x7f07b0162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b00cfae0 con 0x7f07b00b65a0 2024-04-25T09:53:33.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 <== mon.1 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f07a4001670 con 0x7f07b00b65a0 2024-04-25T09:53:33.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f8ed37fe640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ecc0df720 0x7f8ecc168730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 >> v1:172.21.15.150:6789/0 conn(0x7f07b00b65a0 legacy=0x7f07b0166240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f1c63fff640 1 -- 172.21.15.139:0/352335834 learned_addr learned my addr 172.21.15.139:0/352335834 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 --2- 172.21.15.139:0/1748730974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07980039b0 0x7f0798003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 --2- 172.21.15.139:0/1748730974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07980043a0 0x7f0798004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f1c6a2f5640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c5c0eaca0 0x7f1c5c0de850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f765a35c640 1 -- 172.21.15.139:0/1492693777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76540b25d0 msgr2=0x7f76540b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f765a35c640 1 --2- 172.21.15.139:0/1492693777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76540b25d0 0x7f76540b2a40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7654070f60 tx=0x7f764402a270 comp rx=0 tx=0).stop 2024-04-25T09:53:33.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 --2- 172.21.15.139:0/1748730974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0798005040 0x7f079800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7fa3b11f4640 1 -- 172.21.15.139:0/498279114 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4035630 con 0x7fa3980091d0 2024-04-25T09:53:33.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f79a6ffd640 1 -- 172.21.15.139:0/3917356785 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998031710 con 0x7f79a815a760 2024-04-25T09:53:33.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00cf890 con 0x7f07980043a0 2024-04-25T09:53:33.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f765a35c640 1 -- 172.21.15.139:0/1492693777 shutdown_connections 2024-04-25T09:53:33.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f765a35c640 1 --2- 172.21.15.139:0/1492693777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76540b25d0 0x7f76540b2a40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f765a35c640 1 --2- 172.21.15.139:0/1492693777 >> v2:172.21.15.150:3300/0 conn(0x7f76540ce900 0x7f76540b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f765a35c640 1 -- 172.21.15.139:0/1492693777 >> 172.21.15.139:0/1492693777 conn(0x7f765406d0e0 msgr2=0x7f765406d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7fc64d65d640 1 -- 172.21.15.139:0/1467324027 shutdown_connections 2024-04-25T09:53:33.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7fc64d65d640 1 --2- 172.21.15.139:0/1467324027 >> v2:172.21.15.139:3300/0 conn(0x7fc6480b93b0 0x7fc6480ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7fc64d65d640 1 --2- 172.21.15.139:0/1467324027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6480ced30 0x7fc6480d2640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7fc64d65d640 1 -- 172.21.15.139:0/1467324027 >> 172.21.15.139:0/1467324027 conn(0x7fc6480d04a0 msgr2=0x7fc6480d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f765a35c640 1 -- 172.21.15.139:0/1492693777 shutdown_connections 2024-04-25T09:53:33.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f79a6ffd640 1 -- 172.21.15.139:0/3917356785 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79980318b0 con 0x7f79a815a760 2024-04-25T09:53:33.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f79a6ffd640 1 -- 172.21.15.139:0/3917356785 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998031c80 con 0x7f79a815a760 2024-04-25T09:53:33.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00b8e10 con 0x7f0798005040 2024-04-25T09:53:33.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f765a35c640 1 -- 172.21.15.139:0/1492693777 wait complete. 2024-04-25T09:53:33.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.197+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079800bfe0 con 0x7f07980039b0 2024-04-25T09:53:33.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f07af7fe640 1 --2- 172.21.15.139:0/1748730974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07980043a0 0x7f0798004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f07b4d6c640 1 --2- 172.21.15.139:0/1748730974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0798005040 0x7f079800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f8ed2ffd640 1 -- 172.21.15.139:0/2101483707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc0500c0 msgr2=0x7f8ecc047da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f8ed2ffd640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc0500c0 0x7f8ecc047da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f8ed2ffd640 1 -- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ecc0df720 msgr2=0x7f8ecc168730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f8ed2ffd640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ecc0df720 0x7f8ecc168730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f8ed2ffd640 1 -- 172.21.15.139:0/2101483707 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc0e0bd0 con 0x7f8ecc03ad80 2024-04-25T09:53:33.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7fa3b445a640 1 -- 172.21.15.139:0/498279114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3980091d0 msgr2=0x7fa39800b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7fa3b445a640 1 --2- 172.21.15.139:0/498279114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3980091d0 0x7fa39800b690 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400e7c0 tx=0x7fa3980037f0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f6f4d74e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f4806d710 0x7f6f4813d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f6f4d74e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f48144550 0x7f6f480ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7fa3b445a640 1 -- 172.21.15.139:0/498279114 shutdown_connections 2024-04-25T09:53:33.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f6f4d74e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f480cea40 0x7f6f48149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f1c6a2f5640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c5c0eb080 0x7f1c5c0ded90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c056000 con 0x7f1c5c03b0e0 2024-04-25T09:53:33.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c03ac50 con 0x7f1c5c0eb080 2024-04-25T09:53:33.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c056180 con 0x7f1c5c0eaca0 2024-04-25T09:53:33.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7fa3b445a640 1 --2- 172.21.15.139:0/498279114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3980091d0 0x7fa39800b690 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7fa3b445a640 1 -- 172.21.15.139:0/498279114 >> 172.21.15.139:0/498279114 conn(0x7fa3ac067000 msgr2=0x7fa3ac06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f1c637fe640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c5c0eaca0 0x7f1c5c0de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f4d74e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f48166470 con 0x7f6f480cea40 2024-04-25T09:53:33.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f4d74e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f481658f0 con 0x7f6f48144550 2024-04-25T09:53:33.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f4d74e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f48147950 con 0x7f6f4806d710 2024-04-25T09:53:33.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f79ad5b0640 1 -- 172.21.15.139:0/3917356785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79a815a760 msgr2=0x7f79a80aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f4cf4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f480cea40 0x7f6f48149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f4cf4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f480cea40 0x7f6f48149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58776/0 (socket says 172.21.15.139:58776) 2024-04-25T09:53:33.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f4cf4d640 1 -- 172.21.15.139:0/2243590930 learned_addr learned my addr 172.21.15.139:0/2243590930 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f79ad5b0640 1 --2- 172.21.15.139:0/3917356785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79a815a760 0x7f79a80aefe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79a8072b00 tx=0x7f799802c7b0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f47fff640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f4806d710 0x7f6f4813d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f07b4d6c640 1 -- 172.21.15.139:0/1748730974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07980039b0 msgr2=0x7f0798003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:53:33.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f07b4d6c640 1 --2- 172.21.15.139:0/1748730974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07980039b0 0x7f0798003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f07b4d6c640 1 -- 172.21.15.139:0/1748730974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07980043a0 msgr2=0x7f0798004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f07b4d6c640 1 --2- 172.21.15.139:0/1748730974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07980043a0 0x7f0798004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7f07b4d6c640 1 -- 172.21.15.139:0/1748730974 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b00cfae0 con 0x7f0798005040 2024-04-25T09:53:33.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f477fe640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f48144550 0x7f6f480ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f1c68892640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c5c0eb080 0x7f1c5c0ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f1c63fff640 1 -- 172.21.15.139:0/352335834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c5c0eaca0 msgr2=0x7f1c5c0de850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f1c63fff640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c5c0eaca0 0x7f1c5c0de850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f1c63fff640 1 -- 172.21.15.139:0/352335834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c5c0eb080 msgr2=0x7f1c5c0ded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f1c63fff640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c5c0eb080 0x7f1c5c0ded90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f1c63fff640 1 -- 172.21.15.139:0/352335834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c0ddef0 con 0x7f1c5c03b0e0 2024-04-25T09:53:33.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7fc64d65d640 1 -- 172.21.15.139:0/1467324027 shutdown_connections 2024-04-25T09:53:33.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f8ed2ffd640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc03ad80 0x7f8ecc0d95e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4031000 tx=0x7f8ec40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7fa3b445a640 1 -- 172.21.15.139:0/498279114 shutdown_connections 2024-04-25T09:53:33.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f79ad5b0640 1 -- 172.21.15.139:0/3917356785 shutdown_connections 2024-04-25T09:53:33.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f79ad5b0640 1 --2- 172.21.15.139:0/3917356785 >> v2:172.21.15.139:3300/0 conn(0x7f79a80726c0 0x7f79a80cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f79ad5b0640 1 --2- 172.21.15.139:0/3917356785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79a815a760 0x7f79a80aefe0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7fa3b445a640 1 -- 172.21.15.139:0/498279114 wait complete. 2024-04-25T09:53:33.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f4cf4d640 1 -- 172.21.15.139:0/2243590930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f4806d710 msgr2=0x7f6f4813d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f4cf4d640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f4806d710 0x7f6f4813d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f4cf4d640 1 -- 172.21.15.139:0/2243590930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f48144550 msgr2=0x7f6f480ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f4cf4d640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f48144550 0x7f6f480ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f6f4cf4d640 1 -- 172.21.15.139:0/2243590930 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f48149ec0 con 0x7f6f480cea40 2024-04-25T09:53:33.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f79ad5b0640 1 -- 172.21.15.139:0/3917356785 >> 172.21.15.139:0/3917356785 conn(0x7f79a80b22f0 msgr2=0x7f79a80718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.198+0000 7fc64d65d640 1 -- 172.21.15.139:0/1467324027 wait complete. 2024-04-25T09:53:33.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f1c63fff640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c03b0e0 0x7f1c5c0dd9b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1c640690b0 tx=0x7f1c640741f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f765a35c640 1 Processor -- start 2024-04-25T09:53:33.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f8ebbfff640 1 -- 172.21.15.139:0/2101483707 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4037040 con 0x7f8ecc03ad80 2024-04-25T09:53:33.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.199+0000 7f79ad5b0640 1 -- 172.21.15.139:0/3917356785 shutdown_connections 2024-04-25T09:53:33.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f8ebbfff640 1 -- 172.21.15.139:0/2101483707 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec4004270 con 0x7f8ecc03ad80 2024-04-25T09:53:33.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f8ebbfff640 1 -- 172.21.15.139:0/2101483707 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4006e10 con 0x7f8ecc03ad80 2024-04-25T09:53:33.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f07b4d6c640 1 --2- 172.21.15.139:0/1748730974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0798005040 0x7f079800b9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009f90 tx=0x7f0798005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8390b95b96b1b972 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f8ed9815640 1 -- 172.21.15.139:0/2101483707 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc0e0eb0 con 0x7f8ecc03ad80 2024-04-25T09:53:33.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f6f4cf4d640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f480cea40 0x7f6f48149040 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c450 tx=0x7f6f3000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f8ed9815640 1 -- 172.21.15.139:0/2101483707 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc0e1470 con 0x7f8ecc03ad80 2024-04-25T09:53:33.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f6f44ff9640 1 -- 172.21.15.139:0/2243590930 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f300035f0 con 0x7f6f480cea40 2024-04-25T09:53:33.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7fc64d65d640 1 Processor -- start 2024-04-25T09:53:33.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f765a35c640 1 -- start start 2024-04-25T09:53:33.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f6f44ff9640 1 -- 172.21.15.139:0/2243590930 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f30003790 con 0x7f6f480cea40 2024-04-25T09:53:33.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f6f4d74e640 1 -- 172.21.15.139:0/2243590930 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4814a1a0 con 0x7f6f480cea40 2024-04-25T09:53:33.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f6f4d74e640 1 -- 172.21.15.139:0/2243590930 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4813ebf0 con 0x7f6f480cea40 2024-04-25T09:53:33.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7fc64d65d640 1 -- start start 2024-04-25T09:53:33.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f6f44ff9640 1 -- 172.21.15.139:0/2243590930 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f30003b80 con 0x7f6f480cea40 2024-04-25T09:53:33.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f765a35c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f765406dbd0 0x7f765413b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7fc64d65d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6480b93b0 0x7fc648158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7fc64d65d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6480ced30 0x7fc648160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7fc64d65d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc64815dba0 0x7fc648160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7fc64d65d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480b6bc0 con 0x7fc6480ced30 2024-04-25T09:53:33.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7fc64d65d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc648162610 con 0x7fc6480b93b0 2024-04-25T09:53:33.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7fc64d65d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64815df80 con 0x7fc64815dba0 2024-04-25T09:53:33.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f1c60ff9640 1 -- 172.21.15.139:0/352335834 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640b2050 con 0x7f1c5c03b0e0 2024-04-25T09:53:33.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c050f00 con 0x7f1c5c03b0e0 2024-04-25T09:53:33.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c0514c0 con 0x7f1c5c03b0e0 2024-04-25T09:53:33.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f1c60ff9640 1 -- 172.21.15.139:0/352335834 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c640b3040 con 0x7f1c5c03b0e0 2024-04-25T09:53:33.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f1c60ff9640 1 -- 172.21.15.139:0/352335834 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640b78d0 con 0x7f1c5c03b0e0 2024-04-25T09:53:33.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f765a35c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76540b25d0 0x7f76540a9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f765a35c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76540ce900 0x7f7654143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8009c10 con 0x7f0798005040 2024-04-25T09:53:33.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7fc6477fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc64815dba0 0x7fc648160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a8036070 con 0x7f0798005040 2024-04-25T09:53:33.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f765a35c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76540b3310 con 0x7f765406dbd0 2024-04-25T09:53:33.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f07aeffd640 1 -- 172.21.15.139:0/1748730974 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8009c10 con 0x7f0798005040 2024-04-25T09:53:33.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f765a35c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76540b3190 con 0x7f76540ce900 2024-04-25T09:53:33.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f765a35c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765415d3b0 con 0x7f76540b25d0 2024-04-25T09:53:33.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f76537fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76540b25d0 0x7f76540a9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7fc6477fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc64815dba0 0x7fc648160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57154/0 (socket says 172.21.15.139:57154) 2024-04-25T09:53:33.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7fc6477fe640 1 -- 172.21.15.139:0/4072941933 learned_addr learned my addr 172.21.15.139:0/4072941933 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7f8ebbfff640 1 -- 172.21.15.139:0/2101483707 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ec403a070 con 0x7f8ecc03ad80 2024-04-25T09:53:33.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.200+0000 7f79ad5b0640 1 -- 172.21.15.139:0/3917356785 wait complete. 2024-04-25T09:53:33.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7fa3b445a640 1 Processor -- start 2024-04-25T09:53:33.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7fa3b445a640 1 -- start start 2024-04-25T09:53:33.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.201+0000 7f76537fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76540b25d0 0x7f76540a9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57140/0 (socket says 172.21.15.139:57140) 2024-04-25T09:53:33.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7f76537fe640 1 -- 172.21.15.139:0/2140928542 learned_addr learned my addr 172.21.15.139:0/2140928542 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7f76537fe640 1 -- 172.21.15.139:0/2140928542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76540ce900 msgr2=0x7f7654143a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7f76537fe640 1 --2- 172.21.15.139:0/2140928542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76540ce900 0x7f7654143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7f76537fe640 1 -- 172.21.15.139:0/2140928542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f765406dbd0 msgr2=0x7f765413b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7f76537fe640 1 --2- 172.21.15.139:0/2140928542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f765406dbd0 0x7f765413b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7f76537fe640 1 -- 172.21.15.139:0/2140928542 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76540aa680 con 0x7f76540b25d0 2024-04-25T09:53:33.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7f76537fe640 1 --2- 172.21.15.139:0/2140928542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76540b25d0 0x7f76540a9ce0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f764800d5f0 tx=0x7f764800dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7f07b67cf640 1 -- 172.21.15.139:0/1748730974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0798005040 msgr2=0x7f079800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7f6f4d74e640 1 -- 172.21.15.139:0/2243590930 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f2c001030 con 0x7f6f480cea40 2024-04-25T09:53:33.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fc6467fc640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6480ced30 0x7fc648160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7f07b67cf640 1 --2- 172.21.15.139:0/1748730974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0798005040 0x7f079800b9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009f90 tx=0x7f0798005830 comp rx=0 tx=0).stop 2024-04-25T09:53:33.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.202+0000 7f07b4d6c640 1 -- 172.21.15.139:0/1748730974 reap_dead start 2024-04-25T09:53:33.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7f8ebbfff640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ec006b5a0 0x7f8ec006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7f07b67cf640 1 -- 172.21.15.139:0/1748730974 shutdown_connections 2024-04-25T09:53:33.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7f07b67cf640 1 -- 172.21.15.139:0/1748730974 >> 172.21.15.139:0/1748730974 conn(0x7f07b00d3750 msgr2=0x7f07b00d3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7f07b67cf640 1 -- 172.21.15.139:0/1748730974 shutdown_connections 2024-04-25T09:53:33.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7f8ed27fc640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ec006b5a0 0x7f8ec006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7f8ed9815640 1 -- 172.21.15.139:0/2101483707 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ecc005810 con 0x7f8ecc03ad80 2024-04-25T09:53:33.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fc6467fc640 1 -- 172.21.15.139:0/4072941933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc64815dba0 msgr2=0x7fc648160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fc6467fc640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc64815dba0 0x7fc648160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fc6467fc640 1 -- 172.21.15.139:0/4072941933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6480b93b0 msgr2=0x7fc648158090 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T09:53:33.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fc6467fc640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6480b93b0 0x7fc648158090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fc6467fc640 1 -- 172.21.15.139:0/4072941933 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6481585d0 con 0x7fc6480ced30 2024-04-25T09:53:33.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fa3b445a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3980091d0 0x7fa3ac1557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fc6477fe640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc64815dba0 0x7fc648160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:33.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7f79ad5b0640 1 Processor -- start 2024-04-25T09:53:33.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7f07b67cf640 1 -- 172.21.15.139:0/1748730974 wait complete. 2024-04-25T09:53:33.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fa3b445a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3ac155d00 0x7fa3ac0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fc6467fc640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6480ced30 0x7fc648160730 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc64000efc0 tx=0x7fc64000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c44001030 con 0x7f1c5c03b0e0 2024-04-25T09:53:33.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7f79ad5b0640 1 -- start start 2024-04-25T09:53:33.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fa3b445a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3ac0cdfd0 0x7fa3ac142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fa3b21f6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3980091d0 0x7fa3ac1557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7fa3b21f6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3980091d0 0x7fa3ac1557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57158/0 (socket says 172.21.15.139:57158) 2024-04-25T09:53:33.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7fa3b19f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3ac155d00 0x7fa3ac0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7fa3b19f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3ac155d00 0x7fa3ac0cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58804/0 (socket says 172.21.15.139:58804) 2024-04-25T09:53:33.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7fa3b21f6640 1 -- 172.21.15.139:0/306973907 learned_addr learned my addr 172.21.15.139:0/306973907 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.203+0000 7f7650ff9640 1 -- 172.21.15.139:0/2140928542 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480125f0 con 0x7f76540b25d0 2024-04-25T09:53:33.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac06f2b0 con 0x7fa3ac155d00 2024-04-25T09:53:33.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0b3e00 con 0x7fa3ac0cdfd0 2024-04-25T09:53:33.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac06f430 con 0x7fa3980091d0 2024-04-25T09:53:33.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7fc62f7fe640 1 -- 172.21.15.139:0/4072941933 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64000cda0 con 0x7fc6480ced30 2024-04-25T09:53:33.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7fc62f7fe640 1 -- 172.21.15.139:0/4072941933 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc640019070 con 0x7fc6480ced30 2024-04-25T09:53:33.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7f8ed27fc640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ec006b5a0 0x7f8ec006da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8007fd0 tx=0x7f8ec8007f60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7f8ebbfff640 1 -- 172.21.15.139:0/2101483707 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8ec40ab0a0 con 0x7f8ecc03ad80 2024-04-25T09:53:33.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7f765a35c640 1 -- 172.21.15.139:0/2140928542 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76540aa960 con 0x7f76540b25d0 2024-04-25T09:53:33.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7f765a35c640 1 -- 172.21.15.139:0/2140928542 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76540aaf20 con 0x7f76540b25d0 2024-04-25T09:53:33.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7f7650ff9640 1 -- 172.21.15.139:0/2140928542 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648019070 con 0x7f76540b25d0 2024-04-25T09:53:33.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7f7650ff9640 1 -- 172.21.15.139:0/2140928542 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480129b0 con 0x7f76540b25d0 2024-04-25T09:53:33.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7f8ebbfff640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x7f8ec0074ca0 0x7f8ec00770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7f1c60ff9640 1 -- 172.21.15.139:0/352335834 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c640c0020 con 0x7f1c5c03b0e0 2024-04-25T09:53:33.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7fa3b21f6640 1 -- 172.21.15.139:0/306973907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3ac0cdfd0 msgr2=0x7fa3ac142ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T09:53:33.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7fa3b21f6640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3ac0cdfd0 0x7fa3ac142ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.204+0000 7fa3b21f6640 1 -- 172.21.15.139:0/306973907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3ac155d00 msgr2=0x7fa3ac0cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f79ad5b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79a80726c0 0x7f79a814f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7fa3b21f6640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3ac155d00 0x7fa3ac0cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f79ad5b0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79a80b4e70 0x7f79a809aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f79ad5b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79a815a760 0x7f79a80ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f79ad5b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a80ccbd0 con 0x7f79a815a760 2024-04-25T09:53:33.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f79ad5b0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a80adab0 con 0x7f79a80b4e70 2024-04-25T09:53:33.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f79ad5b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a80ccd50 con 0x7f79a80726c0 2024-04-25T09:53:33.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f8ed37fe640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x7f8ec0074ca0 0x7f8ec00770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f8ebbfff640 1 -- 172.21.15.139:0/2101483707 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ec40067f0 con 0x7f8ec0074ca0 2024-04-25T09:53:33.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f8ebbfff640 1 -- 172.21.15.139:0/2101483707 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ec404a070 con 0x7f8ecc03ad80 2024-04-25T09:53:33.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7fa3b21f6640 1 -- 172.21.15.139:0/306973907 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac1561a0 con 0x7fa3980091d0 2024-04-25T09:53:33.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7fa3b19f5640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3ac155d00 0x7fa3ac0cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:33.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f1c60ff9640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1c4806b5a0 0x7f1c4806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f6f44ff9640 1 -- 172.21.15.139:0/2243590930 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f3001b400 con 0x7f6f480cea40 2024-04-25T09:53:33.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7fc62f7fe640 1 -- 172.21.15.139:0/4072941933 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640014680 con 0x7fc6480ced30 2024-04-25T09:53:33.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f79a77fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79a80b4e70 0x7f79a809aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f79acdaf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79a815a760 0x7f79a80ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f79acdaf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79a815a760 0x7f79a80ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58812/0 (socket says 172.21.15.139:58812) 2024-04-25T09:53:33.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f79a77fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79a80b4e70 0x7f79a809aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:45920/0 (socket says 172.21.15.139:45920) 2024-04-25T09:53:33.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f79acdaf640 1 -- 172.21.15.139:0/893371537 learned_addr learned my addr 172.21.15.139:0/893371537 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f79a7fff640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79a80726c0 0x7f79a814f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7fc64d65d640 1 -- 172.21.15.139:0/4072941933 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64815ed60 con 0x7fc6480ced30 2024-04-25T09:53:33.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f1c60ff9640 1 -- 172.21.15.139:0/352335834 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1c64125320 con 0x7f1c5c03b0e0 2024-04-25T09:53:33.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7f07b67cf640 1 Processor -- start 2024-04-25T09:53:33.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7fc64d65d640 1 -- 172.21.15.139:0/4072941933 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64815f2e0 con 0x7fc6480ced30 2024-04-25T09:53:33.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.205+0000 7fa3b21f6640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3980091d0 0x7fa3ac1557c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00f950 tx=0x7fa39c00fe10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f1c60ff9640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x7f1c48074ca0 0x7f1c480770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f1c60ff9640 1 -- 172.21.15.139:0/352335834 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c480777b0 con 0x7f1c48074ca0 2024-04-25T09:53:33.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f07b67cf640 1 -- start start 2024-04-25T09:53:33.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f79a77fe640 1 -- 172.21.15.139:0/893371537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79a80726c0 msgr2=0x7f79a814f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f79a77fe640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79a80726c0 0x7f79a814f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f79a77fe640 1 -- 172.21.15.139:0/893371537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79a815a760 msgr2=0x7f79a80ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f1c68892640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x7f1c48074ca0 0x7f1c480770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f79a77fe640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79a815a760 0x7f79a80ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f79a77fe640 1 -- 172.21.15.139:0/893371537 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a815bf40 con 0x7f79a80b4e70 2024-04-25T09:53:33.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f79acdaf640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79a815a760 0x7f79a80ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:33.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f7650ff9640 1 -- 172.21.15.139:0/2140928542 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7648012b50 con 0x7f76540b25d0 2024-04-25T09:53:33.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f1c68892640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x7f1c48074ca0 0x7f1c480770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7fc62d7fa640 1 -- 172.21.15.139:0/4072941933 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc6480d1730 con 0x7fc6480ced30 2024-04-25T09:53:33.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7fc62f7fe640 1 -- 172.21.15.139:0/4072941933 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc640005d10 con 0x7fc6480ced30 2024-04-25T09:53:33.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f8ed37fe640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x7f8ec0074ca0 0x7f8ec00770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f6f44ff9640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6f2806b4d0 0x7f6f2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f6f44ff9640 1 -- 172.21.15.139:0/2243590930 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6f3008a180 con 0x7f6f480cea40 2024-04-25T09:53:33.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.206+0000 7f6f44ff9640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x7f6f28074c50 0x7f6f28077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f6f44ff9640 1 -- 172.21.15.139:0/2243590930 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f30020e50 con 0x7f6f28074c50 2024-04-25T09:53:33.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f6f44ff9640 1 -- 172.21.15.139:0/2243590930 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f30091050 con 0x7f6f480cea40 2024-04-25T09:53:33.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f6f477fe640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x7f6f28074c50 0x7f6f28077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f6f477fe640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x7f6f28074c50 0x7f6f28077090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f1c637fe640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1c4806b5a0 0x7f1c4806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7f1c637fe640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1c4806b5a0 0x7f1c4806da60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0e4910 tx=0x7f1c58008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f79a77fe640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79a80b4e70 0x7f79a809aa60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f799c00c500 tx=0x7f799c00c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f79a4ff9640 1 -- 172.21.15.139:0/893371537 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c011680 con 0x7f79a80b4e70 2024-04-25T09:53:33.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f79a4ff9640 1 -- 172.21.15.139:0/893371537 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c015070 con 0x7f79a80b4e70 2024-04-25T09:53:33.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f79a4ff9640 1 -- 172.21.15.139:0/893371537 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c011a80 con 0x7f79a80b4e70 2024-04-25T09:53:33.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f79ad5b0640 1 -- 172.21.15.139:0/893371537 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a815ac70 con 0x7f79a80b4e70 2024-04-25T09:53:33.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f79ad5b0640 1 -- 172.21.15.139:0/893371537 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a815b230 con 0x7f79a80b4e70 2024-04-25T09:53:33.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7fa3a2ffd640 1 -- 172.21.15.139:0/306973907 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c017070 con 0x7fa3980091d0 2024-04-25T09:53:33.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f07b67cf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0798005040 0x7f07b0160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f07b67cf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07b00b7dd0 0x7f07b00b8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f07b67cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07b00cd260 0x7f07b00cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f07af7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07b00b7dd0 0x7f07b00b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f07af7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07b00b7dd0 0x7f07b00b8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57174/0 (socket says 172.21.15.139:57174) 2024-04-25T09:53:33.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f07af7fe640 1 -- 172.21.15.139:0/1519611921 learned_addr learned my addr 172.21.15.139:0/1519611921 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00b92a0 con 0x7f07b00cd260 2024-04-25T09:53:33.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00b8aa0 con 0x7f0798005040 2024-04-25T09:53:33.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b0165d00 con 0x7f07b00b7dd0 2024-04-25T09:53:33.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f7650ff9640 1 --2- 172.21.15.139:0/2140928542 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f764006b5a0 0x7f764006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f7650ff9640 1 -- 172.21.15.139:0/2140928542 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f764808a960 con 0x7f76540b25d0 2024-04-25T09:53:33.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.207+0000 7f7653fff640 1 --2- 172.21.15.139:0/2140928542 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f764006b5a0 0x7f764006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7f7653fff640 1 --2- 172.21.15.139:0/2140928542 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f764006b5a0 0x7f764006da60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7654159da0 tx=0x7f764c007cd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7fa3a2ffd640 1 -- 172.21.15.139:0/306973907 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c00dc50 con 0x7fa3980091d0 2024-04-25T09:53:33.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7f765a35c640 1 --2- 172.21.15.139:0/2140928542 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x7f7634001650 0x7f7634003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7f765a35c640 1 -- 172.21.15.139:0/2140928542 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7634006bb0 con 0x7f7634001650 2024-04-25T09:53:33.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7fa3a2ffd640 1 -- 172.21.15.139:0/306973907 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c012460 con 0x7fa3980091d0 2024-04-25T09:53:33.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7f07af7fe640 1 -- 172.21.15.139:0/1519611921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0798005040 msgr2=0x7f07b0160a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T09:53:33.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7f07af7fe640 1 --2- 172.21.15.139:0/1519611921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0798005040 0x7f07b0160a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7f07af7fe640 1 -- 172.21.15.139:0/1519611921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07b00cd260 msgr2=0x7f07b00cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7f07af7fe640 1 --2- 172.21.15.139:0/1519611921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07b00cd260 0x7f07b00cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7f07af7fe640 1 -- 172.21.15.139:0/1519611921 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b01fb630 con 0x7f07b00b7dd0 2024-04-25T09:53:33.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7f07af7fe640 1 --2- 172.21.15.139:0/1519611921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07b00b7dd0 0x7f07b00b8250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f07a400fbe0 tx=0x7f07a400d520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7fc62f7fe640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc63406b5a0 0x7fc63406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ac156480 con 0x7fa3980091d0 2024-04-25T09:53:33.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ac156a40 con 0x7fa3980091d0 2024-04-25T09:53:33.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.208+0000 7f1c60ff9640 1 -- 172.21.15.139:0/352335834 <== osd.3 v2:172.21.15.139:6824/1040931291 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1c480777b0 con 0x7f1c48074ca0 2024-04-25T09:53:33.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7fc62f7fe640 1 -- 172.21.15.139:0/4072941933 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc64008aad0 con 0x7fc6480ced30 2024-04-25T09:53:33.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7f1c60ff9640 1 -- 172.21.15.139:0/352335834 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c6412c050 con 0x7f1c5c03b0e0 2024-04-25T09:53:33.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7f07acff9640 1 -- 172.21.15.139:0/1519611921 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400dbd0 con 0x7f07b00b7dd0 2024-04-25T09:53:33.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7f07acff9640 1 -- 172.21.15.139:0/1519611921 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a4018070 con 0x7f07b00b7dd0 2024-04-25T09:53:33.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7f07acff9640 1 -- 172.21.15.139:0/1519611921 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a40050d0 con 0x7f07b00b7dd0 2024-04-25T09:53:33.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b01fb910 con 0x7f07b00b7dd0 2024-04-25T09:53:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7f76588f9640 1 --2- 172.21.15.139:0/2140928542 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x7f7634001650 0x7f7634003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7f6f47fff640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6f2806b4d0 0x7f6f2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7f6f44ff9640 1 -- 172.21.15.139:0/2243590930 <== osd.2 v2:172.21.15.139:6800/1982336775 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6f30020e50 con 0x7f6f28074c50 2024-04-25T09:53:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b01fbed0 con 0x7f07b00b7dd0 2024-04-25T09:53:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7f8ebbfff640 1 -- 172.21.15.139:0/2101483707 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8ec40067f0 con 0x7f8ec0074ca0 2024-04-25T09:53:33.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7f79ad5b0640 1 -- 172.21.15.139:0/893371537 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f79a814fc10 con 0x7f79a80b4e70 2024-04-25T09:53:33.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7f79a4ff9640 1 -- 172.21.15.139:0/893371537 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f799c011c20 con 0x7f79a80b4e70 2024-04-25T09:53:33.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.210+0000 7f6f47fff640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6f2806b4d0 0x7f6f2806d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6f40034000 tx=0x7f6f4002f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.210+0000 7f76588f9640 1 --2- 172.21.15.139:0/2140928542 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x7f7634001650 0x7f7634003b10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7fc646ffd640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc63406b5a0 0x7fc63406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.209+0000 7fc646ffd640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc63406b5a0 0x7fc63406da60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc6380095f0 tx=0x7fc638006b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.210+0000 7fc62f7fe640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x7fc634074ca0 0x7fc6340770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.210+0000 7fc62f7fe640 1 -- 172.21.15.139:0/4072941933 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc640019070 con 0x7fc634074ca0 2024-04-25T09:53:33.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.210+0000 7fc6477fe640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x7fc634074ca0 0x7fc6340770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.210+0000 7fc62f7fe640 1 -- 172.21.15.139:0/4072941933 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc640055190 con 0x7fc6480ced30 2024-04-25T09:53:33.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.211+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa3ac156e20 con 0x7fa3980091d0 2024-04-25T09:53:33.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.210+0000 7f79a4ff9640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f799406b5a0 0x7f799406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.210+0000 7f79a4ff9640 1 -- 172.21.15.139:0/893371537 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f799c08e0b0 con 0x7f79a80b4e70 2024-04-25T09:53:33.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.210+0000 7f79a4ff9640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x7f7994074ca0 0x7f79940770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.210+0000 7f79a4ff9640 1 -- 172.21.15.139:0/893371537 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f799c01ce50 con 0x7f7994074ca0 2024-04-25T09:53:33.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.211+0000 7f79a4ff9640 1 -- 172.21.15.139:0/893371537 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f799c095050 con 0x7f79a80b4e70 2024-04-25T09:53:33.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.211+0000 7f79acdaf640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x7f7994074ca0 0x7f79940770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.211+0000 7f7650ff9640 1 -- 172.21.15.139:0/2140928542 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7634006bb0 con 0x7f7634001650 2024-04-25T09:53:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.211+0000 7f79a7fff640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f799406b5a0 0x7f799406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.211+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f07b0074520 con 0x7f07b00b7dd0 2024-04-25T09:53:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.211+0000 7fc6477fe640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x7fc634074ca0 0x7fc6340770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.211+0000 7f79acdaf640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x7f7994074ca0 0x7f79940770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.212+0000 7f79a7fff640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f799406b5a0 0x7f799406da60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7998036000 tx=0x7f7998008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.212+0000 7fa3a2ffd640 1 -- 172.21.15.139:0/306973907 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa39c01a070 con 0x7fa3980091d0 2024-04-25T09:53:33.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.212+0000 7f79a4ff9640 1 -- 172.21.15.139:0/893371537 <== osd.4 v2:172.21.15.150:6800/2810531857 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f799c01ce50 con 0x7f7994074ca0 2024-04-25T09:53:33.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.212+0000 7fa3a2ffd640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa39006b520 0x7fa39006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.212+0000 7fc62f7fe640 1 -- 172.21.15.139:0/4072941933 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc640019070 con 0x7fc634074ca0 2024-04-25T09:53:33.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.212+0000 7fa3a2ffd640 1 -- 172.21.15.139:0/306973907 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa39c08a7e0 con 0x7fa3980091d0 2024-04-25T09:53:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.213+0000 7fa3a2ffd640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x7fa390074ca0 0x7fa3900770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.213+0000 7fa3a2ffd640 1 -- 172.21.15.139:0/306973907 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3900777b0 con 0x7fa390074ca0 2024-04-25T09:53:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.213+0000 7fa3a2ffd640 1 -- 172.21.15.139:0/306973907 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa39c091050 con 0x7fa3980091d0 2024-04-25T09:53:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.213+0000 7fa3b19f5640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa39006b520 0x7fa39006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.213+0000 7fa3b19f5640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa39006b520 0x7fa39006d9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac1f24b0 tx=0x7fa3a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.212+0000 7f07acff9640 1 -- 172.21.15.139:0/1519611921 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07a4005270 con 0x7f07b00b7dd0 2024-04-25T09:53:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.212+0000 7f07acff9640 1 --2- 172.21.15.139:0/1519611921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f079c06b5d0 0x7f079c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.212+0000 7f07acff9640 1 -- 172.21.15.139:0/1519611921 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f07a408a6c0 con 0x7f07b00b7dd0 2024-04-25T09:53:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.213+0000 7f07affff640 1 --2- 172.21.15.139:0/1519611921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f079c06b5d0 0x7f079c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.213+0000 7f07affff640 1 --2- 172.21.15.139:0/1519611921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f079c06b5d0 0x7f079c06da90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07b0074e60 tx=0x7f07a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.213+0000 7f07acff9640 1 --2- 172.21.15.139:0/1519611921 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x7f079c074d50 0x7f079c077190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.213+0000 7f07acff9640 1 -- 172.21.15.139:0/1519611921 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07a400f020 con 0x7f079c074d50 2024-04-25T09:53:33.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.213+0000 7f07acff9640 1 -- 172.21.15.139:0/1519611921 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a4091050 con 0x7f07b00b7dd0 2024-04-25T09:53:33.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.214+0000 7f07b4d6c640 1 --2- 172.21.15.139:0/1519611921 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x7f079c074d50 0x7f079c077190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.214+0000 7fa3b29f7640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x7fa390074ca0 0x7fa3900770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.214+0000 7fa3b29f7640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x7fa390074ca0 0x7fa3900770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.214+0000 7f07b4d6c640 1 --2- 172.21.15.139:0/1519611921 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x7f079c074d50 0x7f079c077190 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.215+0000 7fa3a2ffd640 1 -- 172.21.15.139:0/306973907 <== osd.5 v2:172.21.15.150:6824/2184499879 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa3900777b0 con 0x7fa390074ca0 2024-04-25T09:53:33.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.215+0000 7f07acff9640 1 -- 172.21.15.139:0/1519611921 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f07a400f020 con 0x7f079c074d50 2024-04-25T09:53:33.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.242+0000 7f6f4d74e640 1 -- 172.21.15.139:0/2243590930 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6f2c002dd0 con 0x7f6f28074c50 2024-04-25T09:53:33.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.242+0000 7f8ed9815640 1 -- 172.21.15.139:0/2101483707 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8ecc008330 con 0x7f8ec0074ca0 2024-04-25T09:53:33.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.242+0000 7f8ebbfff640 1 -- 172.21.15.139:0/2101483707 <== osd.1 v2:172.21.15.139:6814/315640110 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8ecc008330 con 0x7f8ec0074ca0 2024-04-25T09:53:33.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.243+0000 7f8ed9815640 1 -- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x7f8ec0074ca0 msgr2=0x7f8ec00770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.243+0000 7f6f44ff9640 1 -- 172.21.15.139:0/2243590930 <== osd.2 v2:172.21.15.139:6800/1982336775 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6f2c002dd0 con 0x7f6f28074c50 2024-04-25T09:53:33.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.243+0000 7f8ed9815640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x7f8ec0074ca0 0x7f8ec00770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.243+0000 7f8ed9815640 1 -- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ec006b5a0 msgr2=0x7f8ec006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.243+0000 7f8ed9815640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ec006b5a0 0x7f8ec006da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8007fd0 tx=0x7f8ec8007f60 comp rx=0 tx=0).stop 2024-04-25T09:53:33.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.243+0000 7f8ed9815640 1 -- 172.21.15.139:0/2101483707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc03ad80 msgr2=0x7f8ecc0d95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.243+0000 7f8ed9815640 1 --2- 172.21.15.139:0/2101483707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc03ad80 0x7f8ecc0d95e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4031000 tx=0x7f8ec40046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f6f4d74e640 1 -- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x7f6f28074c50 msgr2=0x7f6f28077090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f8ed37fe640 1 -- 172.21.15.139:0/2101483707 reap_dead start 2024-04-25T09:53:33.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f6f4d74e640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x7f6f28074c50 0x7f6f28077090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c44002dd0 con 0x7f1c48074ca0 2024-04-25T09:53:33.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f8ed9815640 1 -- 172.21.15.139:0/2101483707 shutdown_connections 2024-04-25T09:53:33.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f8ed9815640 1 -- 172.21.15.139:0/2101483707 >> 172.21.15.139:0/2101483707 conn(0x7f8ecc055380 msgr2=0x7f8ecc007800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f6f4d74e640 1 -- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6f2806b4d0 msgr2=0x7f6f2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f6f4d74e640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6f2806b4d0 0x7f6f2806d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6f40034000 tx=0x7f6f4002f000 comp rx=0 tx=0).stop 2024-04-25T09:53:33.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f6f4d74e640 1 -- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f480cea40 msgr2=0x7f6f48149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f6f4d74e640 1 --2- 172.21.15.139:0/2243590930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f480cea40 0x7f6f48149040 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c450 tx=0x7f6f3000c910 comp rx=0 tx=0).stop 2024-04-25T09:53:33.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f6f4cf4d640 1 -- 172.21.15.139:0/2243590930 reap_dead start 2024-04-25T09:53:33.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f6f4d74e640 1 -- 172.21.15.139:0/2243590930 shutdown_connections 2024-04-25T09:53:33.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f6f4d74e640 1 -- 172.21.15.139:0/2243590930 >> 172.21.15.139:0/2243590930 conn(0x7f6f48066f20 msgr2=0x7f6f48163930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f8ed9815640 1 -- 172.21.15.139:0/2101483707 shutdown_connections 2024-04-25T09:53:33.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f8ed9815640 1 -- 172.21.15.139:0/2101483707 wait complete. 2024-04-25T09:53:33.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f765a35c640 1 -- 172.21.15.139:0/2140928542 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7634005c40 con 0x7f7634001650 2024-04-25T09:53:33.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7fc62d7fa640 1 -- 172.21.15.139:0/4072941933 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc6480b8f30 con 0x7fc634074ca0 2024-04-25T09:53:33.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.244+0000 7f6f4d74e640 1 -- 172.21.15.139:0/2243590930 shutdown_connections 2024-04-25T09:53:33.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.245+0000 7f6f4d74e640 1 -- 172.21.15.139:0/2243590930 wait complete. 2024-04-25T09:53:33.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.245+0000 7f1c60ff9640 1 -- 172.21.15.139:0/352335834 <== osd.3 v2:172.21.15.139:6824/1040931291 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1c44002dd0 con 0x7f1c48074ca0 2024-04-25T09:53:33.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.245+0000 7fc62f7fe640 1 -- 172.21.15.139:0/4072941933 <== osd.0 v2:172.21.15.139:6808/1167038101 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc6480b8f30 con 0x7fc634074ca0 2024-04-25T09:53:33.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.245+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x7f1c48074ca0 msgr2=0x7f1c480770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.245+0000 7f1c6a2f5640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x7f1c48074ca0 0x7f1c480770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.245+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1c4806b5a0 msgr2=0x7f1c4806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.245+0000 7f1c6a2f5640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1c4806b5a0 0x7f1c4806da60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0e4910 tx=0x7f1c58008040 comp rx=0 tx=0).stop 2024-04-25T09:53:33.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.245+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c03b0e0 msgr2=0x7f1c5c0dd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.245+0000 7f1c6a2f5640 1 --2- 172.21.15.139:0/352335834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c03b0e0 0x7f1c5c0dd9b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1c640690b0 tx=0x7f1c640741f0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.245+0000 7f1c68892640 1 -- 172.21.15.139:0/352335834 reap_dead start 2024-04-25T09:53:33.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.245+0000 7f79ad5b0640 1 -- 172.21.15.139:0/893371537 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f79a8074520 con 0x7f7994074ca0 2024-04-25T09:53:33.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 shutdown_connections 2024-04-25T09:53:33.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 >> 172.21.15.139:0/352335834 conn(0x7f1c5c01e6f0 msgr2=0x7f1c5c0e78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7fc62d7fa640 1 -- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x7fc634074ca0 msgr2=0x7fc6340770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7fc62d7fa640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x7fc634074ca0 0x7fc6340770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7f7650ff9640 1 -- 172.21.15.139:0/2140928542 <== osd.7 v2:172.21.15.150:6808/2122423536 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7634005c40 con 0x7f7634001650 2024-04-25T09:53:33.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 shutdown_connections 2024-04-25T09:53:33.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7f1c6a2f5640 1 -- 172.21.15.139:0/352335834 wait complete. 2024-04-25T09:53:33.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7fc62d7fa640 1 -- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc63406b5a0 msgr2=0x7fc63406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7fc62d7fa640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc63406b5a0 0x7fc63406da60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc6380095f0 tx=0x7fc638006b50 comp rx=0 tx=0).stop 2024-04-25T09:53:33.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7fc62d7fa640 1 -- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6480ced30 msgr2=0x7fc648160730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7fc62d7fa640 1 --2- 172.21.15.139:0/4072941933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6480ced30 0x7fc648160730 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc64000efc0 tx=0x7fc64000c490 comp rx=0 tx=0).stop 2024-04-25T09:53:33.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7fc6477fe640 1 -- 172.21.15.139:0/4072941933 reap_dead start 2024-04-25T09:53:33.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7fc62d7fa640 1 -- 172.21.15.139:0/4072941933 shutdown_connections 2024-04-25T09:53:33.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.246+0000 7fc62d7fa640 1 -- 172.21.15.139:0/4072941933 >> 172.21.15.139:0/4072941933 conn(0x7fc6480d04a0 msgr2=0x7fc6480b7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7f79a4ff9640 1 -- 172.21.15.139:0/893371537 <== osd.4 v2:172.21.15.150:6800/2810531857 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f79a8074520 con 0x7f7994074ca0 2024-04-25T09:53:33.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7f765a35c640 1 -- 172.21.15.139:0/2140928542 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x7f7634001650 msgr2=0x7f7634003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7f765a35c640 1 --2- 172.21.15.139:0/2140928542 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x7f7634001650 0x7f7634003b10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7f765a35c640 1 -- 172.21.15.139:0/2140928542 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f764006b5a0 msgr2=0x7f764006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7f765a35c640 1 --2- 172.21.15.139:0/2140928542 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f764006b5a0 0x7f764006da60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7654159da0 tx=0x7f764c007cd0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7f765a35c640 1 -- 172.21.15.139:0/2140928542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76540b25d0 msgr2=0x7f76540a9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7f765a35c640 1 --2- 172.21.15.139:0/2140928542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76540b25d0 0x7f76540a9ce0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f764800d5f0 tx=0x7f764800dab0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7f76588f9640 1 -- 172.21.15.139:0/2140928542 reap_dead start 2024-04-25T09:53:33.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7f765a35c640 1 -- 172.21.15.139:0/2140928542 shutdown_connections 2024-04-25T09:53:33.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7f765a35c640 1 -- 172.21.15.139:0/2140928542 >> 172.21.15.139:0/2140928542 conn(0x7f765406d0e0 msgr2=0x7f7654071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7fc62d7fa640 1 -- 172.21.15.139:0/4072941933 shutdown_connections 2024-04-25T09:53:33.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7fc62d7fa640 1 -- 172.21.15.139:0/4072941933 wait complete. 2024-04-25T09:53:33.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7f798e7fc640 1 -- 172.21.15.139:0/893371537 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x7f7994074ca0 msgr2=0x7f79940770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.247+0000 7f798e7fc640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x7f7994074ca0 0x7f79940770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.248+0000 7f798e7fc640 1 -- 172.21.15.139:0/893371537 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f799406b5a0 msgr2=0x7f799406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.248+0000 7f798e7fc640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f799406b5a0 0x7f799406da60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7998036000 tx=0x7f7998008040 comp rx=0 tx=0).stop 2024-04-25T09:53:33.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.248+0000 7f798e7fc640 1 -- 172.21.15.139:0/893371537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79a80b4e70 msgr2=0x7f79a809aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.248+0000 7f798e7fc640 1 --2- 172.21.15.139:0/893371537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79a80b4e70 0x7f79a809aa60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f799c00c500 tx=0x7f799c00c9c0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.248+0000 7f79acdaf640 1 -- 172.21.15.139:0/893371537 reap_dead start 2024-04-25T09:53:33.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.248+0000 7f765a35c640 1 -- 172.21.15.139:0/2140928542 shutdown_connections 2024-04-25T09:53:33.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.248+0000 7f798e7fc640 1 -- 172.21.15.139:0/893371537 shutdown_connections 2024-04-25T09:53:33.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.248+0000 7f798e7fc640 1 -- 172.21.15.139:0/893371537 >> 172.21.15.139:0/893371537 conn(0x7f79a80b22f0 msgr2=0x7f79a8159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.248+0000 7f765a35c640 1 -- 172.21.15.139:0/2140928542 wait complete. 2024-04-25T09:53:33.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.248+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f07b00d4660 con 0x7f079c074d50 2024-04-25T09:53:33.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.248+0000 7f798e7fc640 1 -- 172.21.15.139:0/893371537 shutdown_connections 2024-04-25T09:53:33.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.249+0000 7f798e7fc640 1 -- 172.21.15.139:0/893371537 wait complete. 2024-04-25T09:53:33.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.249+0000 7f07acff9640 1 -- 172.21.15.139:0/1519611921 <== osd.6 v2:172.21.15.150:6809/3369628189 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f07b00d4660 con 0x7f079c074d50 2024-04-25T09:53:33.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.249+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa3ac074520 con 0x7fa390074ca0 2024-04-25T09:53:33.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.249+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x7f079c074d50 msgr2=0x7f079c077190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.250+0000 7fa3a2ffd640 1 -- 172.21.15.139:0/306973907 <== osd.5 v2:172.21.15.150:6824/2184499879 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa3ac074520 con 0x7fa390074ca0 2024-04-25T09:53:33.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.249+0000 7f07b67cf640 1 --2- 172.21.15.139:0/1519611921 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x7f079c074d50 0x7f079c077190 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.250+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f079c06b5d0 msgr2=0x7f079c06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.250+0000 7f07b67cf640 1 --2- 172.21.15.139:0/1519611921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f079c06b5d0 0x7f079c06da90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07b0074e60 tx=0x7f07a0008040 comp rx=0 tx=0).stop 2024-04-25T09:53:33.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.250+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07b00b7dd0 msgr2=0x7f07b00b8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.250+0000 7f07b67cf640 1 --2- 172.21.15.139:0/1519611921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07b00b7dd0 0x7f07b00b8250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f07a400fbe0 tx=0x7f07a400d520 comp rx=0 tx=0).stop 2024-04-25T09:53:33.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.250+0000 7f07b4d6c640 1 -- 172.21.15.139:0/1519611921 reap_dead start 2024-04-25T09:53:33.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.250+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 shutdown_connections 2024-04-25T09:53:33.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.250+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 >> 172.21.15.139:0/1519611921 conn(0x7f07b00d3750 msgr2=0x7f07b0161830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.250+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x7fa390074ca0 msgr2=0x7fa3900770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.250+0000 7fa3b445a640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x7fa390074ca0 0x7fa3900770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.251+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 shutdown_connections 2024-04-25T09:53:33.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.251+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa39006b520 msgr2=0x7fa39006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.251+0000 7f07b67cf640 1 -- 172.21.15.139:0/1519611921 wait complete. 2024-04-25T09:53:33.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.251+0000 7fa3b445a640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa39006b520 0x7fa39006d9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac1f24b0 tx=0x7fa3a8008040 comp rx=0 tx=0).stop 2024-04-25T09:53:33.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.251+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3980091d0 msgr2=0x7fa3ac1557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.251+0000 7fa3b445a640 1 --2- 172.21.15.139:0/306973907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3980091d0 0x7fa3ac1557c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00f950 tx=0x7fa39c00fe10 comp rx=0 tx=0).stop 2024-04-25T09:53:33.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.251+0000 7fa3b29f7640 1 -- 172.21.15.139:0/306973907 reap_dead start 2024-04-25T09:53:33.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.251+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 shutdown_connections 2024-04-25T09:53:33.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.251+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 >> 172.21.15.139:0/306973907 conn(0x7fa3ac067000 msgr2=0x7fa3ac0afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.252+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 shutdown_connections 2024-04-25T09:53:33.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.252+0000 7fa3b445a640 1 -- 172.21.15.139:0/306973907 wait complete. 2024-04-25T09:53:33.314 INFO:teuthology.orchestra.run.smithi139.stdout:55834574857 2024-04-25T09:53:33.315 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T09:53:33.315 INFO:teuthology.orchestra.run.smithi139.stdout:55834574859 2024-04-25T09:53:33.316 INFO:teuthology.orchestra.run.smithi139.stdout:55834574858 2024-04-25T09:53:33.316 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T09:53:33.317 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T09:53:33.318 INFO:teuthology.orchestra.run.smithi139.stdout:55834574858 2024-04-25T09:53:33.318 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T09:53:33.319 INFO:teuthology.orchestra.run.smithi139.stdout:55834574858 2024-04-25T09:53:33.319 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T09:53:33.320 INFO:teuthology.orchestra.run.smithi139.stdout:55834574858 2024-04-25T09:53:33.320 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T09:53:33.321 INFO:teuthology.orchestra.run.smithi139.stdout:55834574858 2024-04-25T09:53:33.321 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T09:53:33.322 INFO:teuthology.orchestra.run.smithi139.stdout:55834574857 2024-04-25T09:53:33.322 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T09:53:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.521+0000 7f3eae4b1640 1 Processor -- start 2024-04-25T09:53:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.521+0000 7f3eae4b1640 1 -- start start 2024-04-25T09:53:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.522+0000 7f3eae4b1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea80b1da0 0x7f3ea8150720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.522+0000 7f3eae4b1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ea806e720 0x7f3ea806eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.522+0000 7f3eae4b1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea8072680 con 0x7f3ea80b25b0 2024-04-25T09:53:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.522+0000 7f3eae4b1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea806f0d0 con 0x7f3ea806e720 2024-04-25T09:53:33.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.522+0000 7f3eae4b1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea8072960 con 0x7f3ea80b1da0 2024-04-25T09:53:33.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.522+0000 7f7f9b251640 1 Processor -- start 2024-04-25T09:53:33.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.523+0000 7f3eaccae640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f3ea80b25b0 0x7f3ea80718e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:37432/0 (socket says 172.21.15.139:37432) 2024-04-25T09:53:33.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.523+0000 7f3eaccae640 1 -- 172.21.15.139:0/3428031399 learned_addr learned my addr 172.21.15.139:0/3428031399 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:33.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.523+0000 7f3ead4af640 1 --2- 172.21.15.139:0/3428031399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea80b1da0 0x7f3ea8150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.523+0000 7f3eadcb0640 1 --2- 172.21.15.139:0/3428031399 >> v2:172.21.15.150:3300/0 conn(0x7f3ea806e720 0x7f3ea806eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.523+0000 7f3e9ffff640 1 -- 172.21.15.139:0/3428031399 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470966571 0 0) 0x7f3ea8072680 con 0x7f3ea80b25b0 2024-04-25T09:53:33.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.523+0000 7f3e9ffff640 1 -- 172.21.15.139:0/3428031399 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e94003520 con 0x7f3ea80b25b0 2024-04-25T09:53:33.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f3e9ffff640 1 -- 172.21.15.139:0/3428031399 <== mon.1 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 914174261 0 0) 0x7f3e94003520 con 0x7f3ea80b25b0 2024-04-25T09:53:33.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f3ead4af640 1 -- 172.21.15.139:0/3428031399 >> v2:172.21.15.150:3300/0 conn(0x7f3ea806e720 msgr2=0x7f3ea806eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f3ead4af640 1 --2- 172.21.15.139:0/3428031399 >> v2:172.21.15.150:3300/0 conn(0x7f3ea806e720 0x7f3ea806eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f3ead4af640 1 -- 172.21.15.139:0/3428031399 >> v1:172.21.15.150:6789/0 conn(0x7f3ea80b25b0 legacy=0x7f3ea80718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f29bf390640 1 Processor -- start 2024-04-25T09:53:33.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.522+0000 7f7f9b251640 1 -- start start 2024-04-25T09:53:33.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f7f9b251640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7f8c04fc00 0x7f7f8c051760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f7f9b251640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f8c0548a0 0x7f7f8c0f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f7f9b251640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c048510 con 0x7f7f8c008180 2024-04-25T09:53:33.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f7f9b251640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8c0487b0 con 0x7f7f8c04fc00 2024-04-25T09:53:33.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f7f9b251640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c048a70 con 0x7f7f8c0548a0 2024-04-25T09:53:33.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.525+0000 7f3ead4af640 1 -- 172.21.15.139:0/3428031399 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea80cb410 con 0x7f3ea80b1da0 2024-04-25T09:53:33.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f9fa08cc640 1 Processor -- start 2024-04-25T09:53:33.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f9fa08cc640 1 -- start start 2024-04-25T09:53:33.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.525+0000 7f7f99a4e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f7f8c008180 0x7f7f8c04ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46520/0 (socket says 172.21.15.139:46520) 2024-04-25T09:53:33.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.525+0000 7f7f99a4e640 1 -- 172.21.15.139:0/4093971842 learned_addr learned my addr 172.21.15.139:0/4093971842 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:33.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.525+0000 7f7f9a24f640 1 --2- 172.21.15.139:0/4093971842 >> v2:172.21.15.150:3300/0 conn(0x7f7f8c04fc00 0x7f7f8c051760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.525+0000 7fb63886a640 1 Processor -- start 2024-04-25T09:53:33.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.525+0000 7fb63886a640 1 -- start start 2024-04-25T09:53:33.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.525+0000 7f9fa08cc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9f9c1449f0 0x7f9f9c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.524+0000 7f29bf390640 1 -- start start 2024-04-25T09:53:33.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.525+0000 7f9f9ad9d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9f9c1449f0 0x7f9f9c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.525+0000 7f63408cd640 1 Processor -- start 2024-04-25T09:53:33.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.525+0000 7f4deab06640 1 Processor -- start 2024-04-25T09:53:33.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.525+0000 7f9f9ad9d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9f9c1449f0 0x7f9f9c06d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:45936/0 (socket says 172.21.15.139:45936) 2024-04-25T09:53:33.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f29bf390640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29b00eacf0 0x7f29b004fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f29bf390640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f29b004ffc0 0x7f29b0037bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f29bf390640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f29b0038100 0x7f29b0038590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f9f9a59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9f9c144dd0 0x7f9f9c162500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46528/0 (socket says 172.21.15.139:46528) 2024-04-25T09:53:33.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f29bf390640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b0054320 con 0x7f29b004ffc0 2024-04-25T09:53:33.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f29bf390640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b0053710 con 0x7f29b0038100 2024-04-25T09:53:33.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f29bf390640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00539f0 con 0x7f29b00eacf0 2024-04-25T09:53:33.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f4deab06640 1 -- start start 2024-04-25T09:53:33.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f9fa08cc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9c165a60 con 0x7f9f9c144dd0 2024-04-25T09:53:33.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f7f9aa50640 1 --2- 172.21.15.139:0/4093971842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f8c0548a0 0x7f7f8c0f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f9f9ad9d640 1 -- 172.21.15.139:0/2641854691 learned_addr learned my addr 172.21.15.139:0/2641854691 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f29bd12c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29b00eacf0 0x7f29b004fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f29bd12c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29b00eacf0 0x7f29b004fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57216/0 (socket says 172.21.15.139:57216) 2024-04-25T09:53:33.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f29bd12c640 1 -- 172.21.15.139:0/2300735838 learned_addr learned my addr 172.21.15.139:0/2300735838 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f3ead4af640 1 --2- 172.21.15.139:0/3428031399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea80b1da0 0x7f3ea8150720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40097a0 tx=0x7f3ea402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2ab16be521b2517 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f1f87f89640 1 Processor -- start 2024-04-25T09:53:33.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f9fa08cc640 1 -- 172.21.15.139:0/2641854691 --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9c165d40 con 0x7f9f9c162c10 2024-04-25T09:53:33.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f9fa08cc640 1 -- 172.21.15.139:0/2641854691 --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c166020 con 0x7f9f9c1449f0 2024-04-25T09:53:33.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f29bc92b640 1 --2- 172.21.15.139:0/2300735838 >> v2:172.21.15.139:3300/0 conn(0x7f29b004ffc0 0x7f29b0037bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7fb63886a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb62c0e7630 0x7fb62c054550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f29bd92d640 1 --2- 172.21.15.139:0/2300735838 >> v2:172.21.15.150:3300/0 conn(0x7f29b0038100 0x7f29b0038590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7fb63886a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb62c04fbd0 0x7fb62c0063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f7f9a24f640 1 -- 172.21.15.139:0/4093971842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f8c0548a0 msgr2=0x7f7f8c0f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f7f9a24f640 1 --2- 172.21.15.139:0/4093971842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f8c0548a0 0x7f7f8c0f1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f7f9a24f640 1 -- 172.21.15.139:0/4093971842 >> v1:172.21.15.139:6789/0 conn(0x7f7f8c008180 legacy=0x7f7f8c04ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f9f99d9b640 1 -- 172.21.15.139:0/2641854691 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783800979 0 0) 0x7f9f9c165a60 con 0x7f9f9c144dd0 2024-04-25T09:53:33.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f9f99d9b640 1 -- 172.21.15.139:0/2641854691 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f84003520 con 0x7f9f9c144dd0 2024-04-25T09:53:33.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f9f99d9b640 1 -- 172.21.15.139:0/2641854691 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105039600 0 0) 0x7f9f9c165d40 con 0x7f9f9c162c10 2024-04-25T09:53:33.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f9f99d9b640 1 -- 172.21.15.139:0/2641854691 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f9c165a60 con 0x7f9f9c162c10 2024-04-25T09:53:33.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f3e9ffff640 1 -- 172.21.15.139:0/3428031399 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4037070 con 0x7f3ea80b1da0 2024-04-25T09:53:33.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f3e9ffff640 1 -- 172.21.15.139:0/3428031399 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea402ac50 con 0x7f3ea80b1da0 2024-04-25T09:53:33.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f3e9ffff640 1 -- 172.21.15.139:0/3428031399 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0001a10 con 0x7f3ea80b1da0 2024-04-25T09:53:33.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f7f9a24f640 1 -- 172.21.15.139:0/4093971842 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c053090 con 0x7f7f8c04fc00 2024-04-25T09:53:33.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7fb63886a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62c050210 con 0x7fb62c054b20 2024-04-25T09:53:33.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7fb63886a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62c053960 con 0x7fb62c0e7630 2024-04-25T09:53:33.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7fb63886a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62c054200 con 0x7fb62c04fbd0 2024-04-25T09:53:33.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7fb63259c640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fb62c054b20 0x7fb62c04f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:37446/0 (socket says 172.21.15.139:37446) 2024-04-25T09:53:33.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7fb63259c640 1 -- 172.21.15.139:0/3815485050 learned_addr learned my addr 172.21.15.139:0/3815485050 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:33.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.525+0000 7f63408cd640 1 -- start start 2024-04-25T09:53:33.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f63408cd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f633c15fb60 0x7f633c0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f63408cd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f633c071d20 0x7f633c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f63408cd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633c1624d0 con 0x7f633c0b2aa0 2024-04-25T09:53:33.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f63408cd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633c1630c0 con 0x7f633c071d20 2024-04-25T09:53:33.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f63408cd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c162790 con 0x7f633c15fb60 2024-04-25T09:53:33.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7fb632d9d640 1 --2- 172.21.15.139:0/3815485050 >> v2:172.21.15.139:3300/0 conn(0x7fb62c0e7630 0x7fb62c054550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f4deab06640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ddc0e6e10 0x7f4ddc0e7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f4deab06640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ddc007330 con 0x7f4ddc057af0 2024-04-25T09:53:33.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f4deab06640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ddc007610 con 0x7f4ddc0e33e0 2024-04-25T09:53:33.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f4deab06640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc0e77b0 con 0x7f4ddc0e6e10 2024-04-25T09:53:33.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.526+0000 7f1f87f89640 1 -- start start 2024-04-25T09:53:33.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f633ad9d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f633c071d20 0x7f633c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f633ad9d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f633c071d20 0x7f633c06d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:45966/0 (socket says 172.21.15.139:45966) 2024-04-25T09:53:33.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f633ad9d640 1 -- 172.21.15.139:0/3320085730 learned_addr learned my addr 172.21.15.139:0/3320085730 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f29bd12c640 1 -- 172.21.15.139:0/2300735838 >> v2:172.21.15.150:3300/0 conn(0x7f29b0038100 msgr2=0x7f29b0038590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f29bd12c640 1 --2- 172.21.15.139:0/2300735838 >> v2:172.21.15.150:3300/0 conn(0x7f29b0038100 0x7f29b0038590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f29bd12c640 1 -- 172.21.15.139:0/2300735838 >> v2:172.21.15.139:3300/0 conn(0x7f29b004ffc0 msgr2=0x7f29b0037bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f29bd12c640 1 --2- 172.21.15.139:0/2300735838 >> v2:172.21.15.139:3300/0 conn(0x7f29b004ffc0 0x7f29b0037bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f29bd12c640 1 -- 172.21.15.139:0/2300735838 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b0053cd0 con 0x7f29b00eacf0 2024-04-25T09:53:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f29bc92b640 1 --2- 172.21.15.139:0/2300735838 >> v2:172.21.15.139:3300/0 conn(0x7f29b004ffc0 0x7f29b0037bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f29bd92d640 1 --2- 172.21.15.139:0/2300735838 >> v2:172.21.15.150:3300/0 conn(0x7f29b0038100 0x7f29b0038590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f29bd12c640 1 --2- 172.21.15.139:0/2300735838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29b00eacf0 0x7f29b004fa80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29b802d370 tx=0x7f29b80a5140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71f079d2fddbed1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f9f99d9b640 1 -- 172.21.15.139:0/2641854691 <== mon.1 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 983898239 0 0) 0x7f9f9c165a60 con 0x7f9f9c162c10 2024-04-25T09:53:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f9f9ad9d640 1 -- 172.21.15.139:0/2641854691 >> v1:172.21.15.150:6789/0 conn(0x7f9f9c162c10 legacy=0x7f9f9c0b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f9f9ad9d640 1 -- 172.21.15.139:0/2641854691 >> v1:172.21.15.139:6789/0 conn(0x7f9f9c144dd0 legacy=0x7f9f9c162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.527+0000 7f9f9ad9d640 1 -- 172.21.15.139:0/2641854691 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c166300 con 0x7f9f9c1449f0 2024-04-25T09:53:33.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f633a59c640 1 --2- 172.21.15.139:0/3320085730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f633c15fb60 0x7f633c0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f4dea305640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4ddc057af0 0x7f4ddc057f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46540/0 (socket says 172.21.15.139:46540) 2024-04-25T09:53:33.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f4dea305640 1 -- 172.21.15.139:0/1107750938 learned_addr learned my addr 172.21.15.139:0/1107750938 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:33.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f4de9b04640 1 --1- 172.21.15.139:0/1107750938 >> v1:172.21.15.150:6789/0 conn(0x7f4ddc0e33e0 0x7f4ddc006890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:37462/0 (socket says 172.21.15.139:37462) 2024-04-25T09:53:33.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f4de9303640 1 --2- 172.21.15.139:0/1107750938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ddc0e6e10 0x7f4ddc0e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f1f87f89640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1f800711c0 0x7f1f8006f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f1f87f89640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f800b25f0 0x7f1f80159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f1f87f89640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8006f760 con 0x7f1f801542a0 2024-04-25T09:53:33.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f1f87f89640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8006fa40 con 0x7f1f800711c0 2024-04-25T09:53:33.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f1f87f89640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f800cdc60 con 0x7f1f800b25f0 2024-04-25T09:53:33.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f9f9ad9d640 1 --2- 172.21.15.139:0/2641854691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f9c1449f0 0x7f9f9c06d0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0097f0 tx=0x7f9f8c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a4ecf0a4ee4932 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f633ad9d640 1 -- 172.21.15.139:0/3320085730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f633c15fb60 msgr2=0x7f633c0b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f633ad9d640 1 --2- 172.21.15.139:0/3320085730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f633c15fb60 0x7f633c0b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f633ad9d640 1 -- 172.21.15.139:0/3320085730 >> v1:172.21.15.150:6789/0 conn(0x7f633c0b2aa0 legacy=0x7f633c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f633ad9d640 1 -- 172.21.15.139:0/3320085730 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c162a70 con 0x7f633c071d20 2024-04-25T09:53:33.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931565388 0 0) 0x7f4ddc007330 con 0x7f4ddc057af0 2024-04-25T09:53:33.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dd0003520 con 0x7f4ddc057af0 2024-04-25T09:53:33.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f1f85524640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1f800711c0 0x7f1f8006f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f1f86526640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f800b25f0 0x7f1f80159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f1f85524640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1f800711c0 0x7f1f8006f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:45982/0 (socket says 172.21.15.139:45982) 2024-04-25T09:53:33.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f1f85524640 1 -- 172.21.15.139:0/1032840590 learned_addr learned my addr 172.21.15.139:0/1032840590 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7fb63359e640 1 --2- 172.21.15.139:0/3815485050 >> v2:172.21.15.150:3300/0 conn(0x7fb62c04fbd0 0x7fb62c0063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7fb632d9d640 1 -- 172.21.15.139:0/3815485050 >> v2:172.21.15.150:3300/0 conn(0x7fb62c04fbd0 msgr2=0x7fb62c0063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7fb632d9d640 1 --2- 172.21.15.139:0/3815485050 >> v2:172.21.15.150:3300/0 conn(0x7fb62c04fbd0 0x7fb62c0063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7fb632d9d640 1 -- 172.21.15.139:0/3815485050 >> v1:172.21.15.150:6789/0 conn(0x7fb62c054b20 legacy=0x7fb62c04f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7fb632d9d640 1 -- 172.21.15.139:0/3815485050 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c0485b0 con 0x7fb62c0e7630 2024-04-25T09:53:33.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f7f9a24f640 1 --2- 172.21.15.139:0/4093971842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c04fc00 0x7f7f8c051760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7f840097a0 tx=0x7f7f8402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a08ac6616ba3ace7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f9f99d9b640 1 -- 172.21.15.139:0/2641854691 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c037070 con 0x7f9f9c1449f0 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3428031399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea80b1da0 msgr2=0x7f3ea8150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f3eae4b1640 1 --2- 172.21.15.139:0/3428031399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea80b1da0 0x7f3ea8150720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40097a0 tx=0x7f3ea402a1c0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2146951591 0 0) 0x7f4dd0003520 con 0x7f4ddc057af0 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ddc0e6e10 msgr2=0x7f4ddc0e7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f4de8b02640 1 --2- 172.21.15.139:0/1107750938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ddc0e6e10 0x7f4ddc0e7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 >> v1:172.21.15.150:6789/0 conn(0x7f4ddc0e33e0 legacy=0x7f4ddc006890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc0572f0 con 0x7f4ddc057af0 2024-04-25T09:53:33.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3428031399 shutdown_connections 2024-04-25T09:53:33.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f3eae4b1640 1 --2- 172.21.15.139:0/3428031399 >> v2:172.21.15.150:3300/0 conn(0x7f3ea806e720 0x7f3ea806eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f3eae4b1640 1 --2- 172.21.15.139:0/3428031399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea80b1da0 0x7f3ea8150720 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3428031399 >> 172.21.15.139:0/3428031399 conn(0x7f3ea80b12b0 msgr2=0x7f3ea80b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f4de40627f0 con 0x7f4ddc057af0 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f9f99d9b640 1 -- 172.21.15.139:0/2641854691 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c02ad40 con 0x7f9f9c1449f0 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.528+0000 7f9f99d9b640 1 -- 172.21.15.139:0/2641854691 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c037070 con 0x7f9f9c1449f0 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f1f85524640 1 -- 172.21.15.139:0/1032840590 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f800b25f0 msgr2=0x7f1f80159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f1f85524640 1 --2- 172.21.15.139:0/1032840590 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f800b25f0 0x7f1f80159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f1f85524640 1 -- 172.21.15.139:0/1032840590 >> v1:172.21.15.150:6789/0 conn(0x7f1f801542a0 legacy=0x7f1f8006ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f1f85524640 1 -- 172.21.15.139:0/1032840590 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f800cdf40 con 0x7f1f800711c0 2024-04-25T09:53:33.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f7f9924d640 1 -- 172.21.15.139:0/4093971842 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84037070 con 0x7f7f8c04fc00 2024-04-25T09:53:33.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f7f9924d640 1 -- 172.21.15.139:0/4093971842 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f84032440 con 0x7f7f8c04fc00 2024-04-25T09:53:33.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f7f9924d640 1 -- 172.21.15.139:0/4093971842 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84032850 con 0x7f7f8c04fc00 2024-04-25T09:53:33.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 >> v1:172.21.15.139:6789/0 conn(0x7f4ddc057af0 legacy=0x7f4ddc057f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de8b02640 1 --2- 172.21.15.139:0/1107750938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dd00039e0 0x7f4dd0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de8b02640 1 --2- 172.21.15.139:0/1107750938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dd0004800 0x7f4dd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de8b02640 1 --2- 172.21.15.139:0/1107750938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ddc0e6e10 0x7f4dd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd000bb00 con 0x7f4dd0004800 2024-04-25T09:53:33.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd000bc80 con 0x7f4dd00039e0 2024-04-25T09:53:33.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd000be50 con 0x7f4ddc0e6e10 2024-04-25T09:53:33.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3428031399 shutdown_connections 2024-04-25T09:53:33.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f1f86526640 1 --2- 172.21.15.139:0/1032840590 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f800b25f0 0x7f1f80159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f633ad9d640 1 --2- 172.21.15.139:0/3320085730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633c071d20 0x7f633c06d670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6334009e50 tx=0x7f633402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e6ad379551aab40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f29affff640 1 -- 172.21.15.139:0/2300735838 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b80b2070 con 0x7f29b00eacf0 2024-04-25T09:53:33.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de9303640 1 --2- 172.21.15.139:0/1107750938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dd0004800 0x7f4dd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de9b04640 1 --2- 172.21.15.139:0/1107750938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dd00039e0 0x7f4dd0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4dea305640 1 --2- 172.21.15.139:0/1107750938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ddc0e6e10 0x7f4dd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f29affff640 1 -- 172.21.15.139:0/2300735838 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b80ad440 con 0x7f29b00eacf0 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f29affff640 1 -- 172.21.15.139:0/2300735838 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b80ad8b0 con 0x7f29b00eacf0 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f9fa08cc640 1 -- 172.21.15.139:0/2641854691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f9c1449f0 msgr2=0x7f9f9c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f9fa08cc640 1 --2- 172.21.15.139:0/2641854691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f9c1449f0 0x7f9f9c06d0c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0097f0 tx=0x7f9f8c02a230 comp rx=0 tx=0).stop 2024-04-25T09:53:33.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f633959a640 1 -- 172.21.15.139:0/3320085730 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334036070 con 0x7f633c071d20 2024-04-25T09:53:33.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3428031399 wait complete. 2024-04-25T09:53:33.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f9fa08cc640 1 -- 172.21.15.139:0/2641854691 shutdown_connections 2024-04-25T09:53:33.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f9fa08cc640 1 --2- 172.21.15.139:0/2641854691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f9c1449f0 0x7f9f9c06d0c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f9fa08cc640 1 -- 172.21.15.139:0/2641854691 >> 172.21.15.139:0/2641854691 conn(0x7f9f9c066f20 msgr2=0x7f9f9c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f9fa08cc640 1 -- 172.21.15.139:0/2641854691 shutdown_connections 2024-04-25T09:53:33.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f1f85524640 1 --2- 172.21.15.139:0/1032840590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f800711c0 0x7f1f8006f160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009870 tx=0x7f1f7c02a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c609bf0f3b9942f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f633959a640 1 -- 172.21.15.139:0/3320085730 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6334031600 con 0x7f633c071d20 2024-04-25T09:53:33.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f633959a640 1 -- 172.21.15.139:0/3320085730 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340319d0 con 0x7f633c071d20 2024-04-25T09:53:33.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f7f9b251640 1 -- 172.21.15.139:0/4093971842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c04fc00 msgr2=0x7f7f8c051760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f7f9b251640 1 --2- 172.21.15.139:0/4093971842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c04fc00 0x7f7f8c051760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7f840097a0 tx=0x7f7f8402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de9303640 1 -- 172.21.15.139:0/1107750938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ddc0e6e10 msgr2=0x7f4dd00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de9303640 1 --2- 172.21.15.139:0/1107750938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ddc0e6e10 0x7f4dd00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7f4de9303640 1 -- 172.21.15.139:0/1107750938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dd00039e0 msgr2=0x7f4dd0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:33.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f4de9303640 1 --2- 172.21.15.139:0/1107750938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dd00039e0 0x7f4dd0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f4de9303640 1 -- 172.21.15.139:0/1107750938 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd0003520 con 0x7f4dd0004800 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f7f9b251640 1 -- 172.21.15.139:0/4093971842 shutdown_connections 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f7f9b251640 1 --2- 172.21.15.139:0/4093971842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f8c0548a0 0x7f7f8c0f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f7f9b251640 1 --2- 172.21.15.139:0/4093971842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c04fc00 0x7f7f8c051760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f7f9b251640 1 -- 172.21.15.139:0/4093971842 >> 172.21.15.139:0/4093971842 conn(0x7f7f8c055340 msgr2=0x7f7f8c055750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.529+0000 7fb632d9d640 1 --2- 172.21.15.139:0/3815485050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb62c0e7630 0x7fb62c054550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb62c048000 tx=0x7fb62402a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868a3efb2232fb1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7fb631d9b640 1 -- 172.21.15.139:0/3815485050 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624037070 con 0x7fb62c0e7630 2024-04-25T09:53:33.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7fb631d9b640 1 -- 172.21.15.139:0/3815485050 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb624032440 con 0x7fb62c0e7630 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7fb631d9b640 1 -- 172.21.15.139:0/3815485050 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624032810 con 0x7fb62c0e7630 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f9fa08cc640 1 -- 172.21.15.139:0/2641854691 wait complete. 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f7f9b251640 1 -- 172.21.15.139:0/4093971842 shutdown_connections 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f63408cd640 1 -- 172.21.15.139:0/3320085730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633c071d20 msgr2=0x7f633c06d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f63408cd640 1 --2- 172.21.15.139:0/3320085730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633c071d20 0x7f633c06d670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6334009e50 tx=0x7f633402a710 comp rx=0 tx=0).stop 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.530+0000 7f7f9b251640 1 -- 172.21.15.139:0/4093971842 wait complete. 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f29bf390640 1 -- 172.21.15.139:0/2300735838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29b00eacf0 msgr2=0x7f29b004fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f29bf390640 1 --2- 172.21.15.139:0/2300735838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29b00eacf0 0x7f29b004fa80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29b802d370 tx=0x7f29b80a5140 comp rx=0 tx=0).stop 2024-04-25T09:53:33.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f4de9303640 1 --2- 172.21.15.139:0/1107750938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dd0004800 0x7f4dd0004c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4de000ba00 tx=0x7f4dd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3315fabf03e2540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f29bf390640 1 -- 172.21.15.139:0/2300735838 shutdown_connections 2024-04-25T09:53:33.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f29bf390640 1 --2- 172.21.15.139:0/2300735838 >> v2:172.21.15.150:3300/0 conn(0x7f29b0038100 0x7f29b0038590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f29bf390640 1 --2- 172.21.15.139:0/2300735838 >> v2:172.21.15.139:3300/0 conn(0x7f29b004ffc0 0x7f29b0037bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f29bf390640 1 --2- 172.21.15.139:0/2300735838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29b00eacf0 0x7f29b004fa80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f29bf390640 1 -- 172.21.15.139:0/2300735838 >> 172.21.15.139:0/2300735838 conn(0x7f29b0009fa0 msgr2=0x7f29b000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f63408cd640 1 -- 172.21.15.139:0/3320085730 shutdown_connections 2024-04-25T09:53:33.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f63408cd640 1 --2- 172.21.15.139:0/3320085730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633c071d20 0x7f633c06d670 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f63408cd640 1 --2- 172.21.15.139:0/3320085730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f633c15fb60 0x7f633c0b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f63408cd640 1 -- 172.21.15.139:0/3320085730 >> 172.21.15.139:0/3320085730 conn(0x7f633c0b17a0 msgr2=0x7f633c0b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f3eae4b1640 1 Processor -- start 2024-04-25T09:53:33.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f29bf390640 1 -- 172.21.15.139:0/2300735838 shutdown_connections 2024-04-25T09:53:33.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f63408cd640 1 -- 172.21.15.139:0/3320085730 shutdown_connections 2024-04-25T09:53:33.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7fb63886a640 1 -- 172.21.15.139:0/3815485050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb62c0e7630 msgr2=0x7fb62c054550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7fb63886a640 1 --2- 172.21.15.139:0/3815485050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb62c0e7630 0x7fb62c054550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb62c048000 tx=0x7fb62402a370 comp rx=0 tx=0).stop 2024-04-25T09:53:33.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7fb63886a640 1 -- 172.21.15.139:0/3815485050 shutdown_connections 2024-04-25T09:53:33.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7fb63886a640 1 --2- 172.21.15.139:0/3815485050 >> v2:172.21.15.150:3300/0 conn(0x7fb62c04fbd0 0x7fb62c0063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7fb63886a640 1 --2- 172.21.15.139:0/3815485050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb62c0e7630 0x7fb62c054550 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb62c048000 tx=0x7fb62402a370 comp rx=0 tx=0).stop 2024-04-25T09:53:33.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7fb63886a640 1 -- 172.21.15.139:0/3815485050 >> 172.21.15.139:0/3815485050 conn(0x7fb62c00a3f0 msgr2=0x7fb62c00a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7fb63886a640 1 -- 172.21.15.139:0/3815485050 shutdown_connections 2024-04-25T09:53:33.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0030070 con 0x7f4dd0004800 2024-04-25T09:53:33.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de002ad80 con 0x7f4dd0004800 2024-04-25T09:53:33.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f4de8b02640 1 -- 172.21.15.139:0/1107750938 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000b600 con 0x7f4dd0004800 2024-04-25T09:53:33.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f63408cd640 1 -- 172.21.15.139:0/3320085730 wait complete. 2024-04-25T09:53:33.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f9fa08cc640 1 Processor -- start 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f7f9b251640 1 Processor -- start 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f9fa08cc640 1 -- start start 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f4deab06640 1 -- 172.21.15.139:0/1107750938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dd0004800 msgr2=0x7f4dd0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f4deab06640 1 --2- 172.21.15.139:0/1107750938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dd0004800 0x7f4dd0004c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4de000ba00 tx=0x7f4dd0005830 comp rx=0 tx=0).stop 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f7f9b251640 1 -- start start 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f4dea305640 1 -- 172.21.15.139:0/1107750938 reap_dead start 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f4deab06640 1 -- 172.21.15.139:0/1107750938 shutdown_connections 2024-04-25T09:53:33.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f4deab06640 1 -- 172.21.15.139:0/1107750938 >> 172.21.15.139:0/1107750938 conn(0x7f4ddc03aa20 msgr2=0x7f4ddc03ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f3eae4b1640 1 -- start start 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f3eae4b1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea806e720 0x7f3ea81faec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f3eae4b1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ea80b1da0 0x7f3ea815d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f3eae4b1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea80b25b0 0x7f3ea815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f3eae4b1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea8071ff0 con 0x7f3ea80b25b0 2024-04-25T09:53:33.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f3eae4b1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea806f480 con 0x7f3ea80b1da0 2024-04-25T09:53:33.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f3eae4b1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea8158370 con 0x7f3ea806e720 2024-04-25T09:53:33.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7f29bf390640 1 -- 172.21.15.139:0/2300735838 wait complete. 2024-04-25T09:53:33.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.531+0000 7fb63886a640 1 -- 172.21.15.139:0/3815485050 wait complete. 2024-04-25T09:53:33.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f4deab06640 1 -- 172.21.15.139:0/1107750938 shutdown_connections 2024-04-25T09:53:33.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f4deab06640 1 -- 172.21.15.139:0/1107750938 wait complete. 2024-04-25T09:53:33.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f63408cd640 1 Processor -- start 2024-04-25T09:53:33.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f63408cd640 1 -- start start 2024-04-25T09:53:33.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f9fa08cc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f9c1449f0 0x7f9f9c141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f7f9b251640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f8c008180 0x7f7f8c04d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7fb63886a640 1 Processor -- start 2024-04-25T09:53:33.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f7f9b251640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c04fc00 0x7f7f8c0d9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f29bf390640 1 Processor -- start 2024-04-25T09:53:33.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f29bf390640 1 -- start start 2024-04-25T09:53:33.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f7f9b251640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f8c0548a0 0x7f7f8c044910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f7f9a24f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f8c008180 0x7f7f8c04d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f7f9a24f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f8c008180 0x7f7f8c04d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57262/0 (socket says 172.21.15.139:57262) 2024-04-25T09:53:33.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f7f9a24f640 1 -- 172.21.15.139:0/902254777 learned_addr learned my addr 172.21.15.139:0/902254777 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0f16a0 con 0x7f7f8c0548a0 2024-04-25T09:53:33.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0506f0 con 0x7f7f8c04fc00 2024-04-25T09:53:33.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0f1820 con 0x7f7f8c008180 2024-04-25T09:53:33.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f7f9aa50640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f8c0548a0 0x7f7f8c044910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f1f84d23640 1 -- 172.21.15.139:0/1032840590 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c037070 con 0x7f1f800711c0 2024-04-25T09:53:33.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f1f84d23640 1 -- 172.21.15.139:0/1032840590 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c032640 con 0x7f1f800711c0 2024-04-25T09:53:33.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.532+0000 7f1f84d23640 1 -- 172.21.15.139:0/1032840590 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c032a10 con 0x7f1f800711c0 2024-04-25T09:53:33.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f9f9ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f9c1449f0 0x7f9f9c141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f9f9ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f9c1449f0 0x7f9f9c141190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58884/0 (socket says 172.21.15.139:58884) 2024-04-25T09:53:33.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f7f99a4e640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c04fc00 0x7f7f8c0d9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f9f9ad9d640 1 -- 172.21.15.139:0/970411386 learned_addr learned my addr 172.21.15.139:0/970411386 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f9fa08cc640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f9c144dd0 0x7f9f9c164780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f7f9a24f640 1 -- 172.21.15.139:0/902254777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c04fc00 msgr2=0x7f7f8c0d9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f1f87f89640 1 -- 172.21.15.139:0/1032840590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f800711c0 msgr2=0x7f1f8006f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f1f87f89640 1 --2- 172.21.15.139:0/1032840590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f800711c0 0x7f1f8006f160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009870 tx=0x7f1f7c02a2f0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f9fa08cc640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f9c162c10 0x7f9f9c164cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f7f9a24f640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c04fc00 0x7f7f8c0d9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f7f9a24f640 1 -- 172.21.15.139:0/902254777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f8c0548a0 msgr2=0x7f7f8c044910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f7f9a24f640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f8c0548a0 0x7f7f8c044910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f7f9a24f640 1 -- 172.21.15.139:0/902254777 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c0e1230 con 0x7f7f8c008180 2024-04-25T09:53:33.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f3ead4af640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea806e720 0x7f3ea81faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f3ead4af640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea806e720 0x7f3ea81faec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57254/0 (socket says 172.21.15.139:57254) 2024-04-25T09:53:33.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7f3ead4af640 1 -- 172.21.15.139:0/3010784255 learned_addr learned my addr 172.21.15.139:0/3010784255 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f3eadcb0640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea80b25b0 0x7f3ea815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f9f9a59c640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f9c144dd0 0x7f9f9c164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c06d780 con 0x7f9f9c1449f0 2024-04-25T09:53:33.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f9f9b59e640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f9c162c10 0x7f9f9c164cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f7f9a24f640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f8c008180 0x7f7f8c04d300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f840096f0 tx=0x7f7f840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f3eaccae640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ea80b1da0 0x7f3ea815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c06d600 con 0x7f9f9c162c10 2024-04-25T09:53:33.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c06d900 con 0x7f9f9c144dd0 2024-04-25T09:53:33.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f1f87f89640 1 -- 172.21.15.139:0/1032840590 shutdown_connections 2024-04-25T09:53:33.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f1f87f89640 1 --2- 172.21.15.139:0/1032840590 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f800b25f0 0x7f1f80159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f1f87f89640 1 --2- 172.21.15.139:0/1032840590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f800711c0 0x7f1f8006f160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f1f87f89640 1 -- 172.21.15.139:0/1032840590 >> 172.21.15.139:0/1032840590 conn(0x7f1f8006af80 msgr2=0x7f1f8006b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f29bf390640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29b0038100 0x7f29b0032e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f29bf390640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29b004ffc0 0x7f29b0048730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f29bf390640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29b00eacf0 0x7f29b0048c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f29bf390640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b0054930 con 0x7f29b004ffc0 2024-04-25T09:53:33.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f29bf390640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00547b0 con 0x7f29b00eacf0 2024-04-25T09:53:33.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f29bf390640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b004f540 con 0x7f29b0038100 2024-04-25T09:53:33.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.534+0000 7f29bc92b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29b004ffc0 0x7f29b0048730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f29bc92b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29b004ffc0 0x7f29b0048730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58894/0 (socket says 172.21.15.139:58894) 2024-04-25T09:53:33.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f29bc92b640 1 -- 172.21.15.139:0/1816752337 learned_addr learned my addr 172.21.15.139:0/1816752337 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.533+0000 7fb63886a640 1 -- start start 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7fb63886a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb62c04fbd0 0x7fb62c174a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7fb63886a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb62c054b20 0x7fb62c007440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7fb63886a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0e0dc0 0x7fb62c1717b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f3ead4af640 1 -- 172.21.15.139:0/3010784255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ea80b1da0 msgr2=0x7f3ea815d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f3ead4af640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ea80b1da0 0x7f3ea815d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f3ead4af640 1 -- 172.21.15.139:0/3010784255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea80b25b0 msgr2=0x7f3ea815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f3ead4af640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea80b25b0 0x7f3ea815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f3ead4af640 1 -- 172.21.15.139:0/3010784255 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea8158090 con 0x7f3ea806e720 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f1f87f89640 1 -- 172.21.15.139:0/1032840590 shutdown_connections 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7fb63886a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0e7a10 con 0x7fb62c054b20 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7fb63886a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c053e90 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f7f8affd640 1 -- 172.21.15.139:0/902254777 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84042020 con 0x7f7f8c008180 2024-04-25T09:53:33.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f7f8affd640 1 -- 172.21.15.139:0/902254777 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f840308f0 con 0x7f7f8c008180 2024-04-25T09:53:33.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7fb63359e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0e0dc0 0x7fb62c1717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7fb63359e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0e0dc0 0x7fb62c1717b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:46046/0 (socket says 172.21.15.139:46046) 2024-04-25T09:53:33.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7fb63359e640 1 -- 172.21.15.139:0/529022313 learned_addr learned my addr 172.21.15.139:0/529022313 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f7f8affd640 1 -- 172.21.15.139:0/902254777 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84037040 con 0x7f7f8c008180 2024-04-25T09:53:33.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c04da30 con 0x7f7f8c008180 2024-04-25T09:53:33.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c04dff0 con 0x7f7f8c008180 2024-04-25T09:53:33.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7fb632d9d640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb62c04fbd0 0x7fb62c174a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7fb63886a640 1 -- 172.21.15.139:0/529022313 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c048260 con 0x7fb62c04fbd0 2024-04-25T09:53:33.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f4deab06640 1 Processor -- start 2024-04-25T09:53:33.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f63408cd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f633c071d20 0x7f633c13a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f3ead4af640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea806e720 0x7f3ea81faec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40098c0 tx=0x7f3ea40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f63408cd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633c0b2aa0 0x7f633c1613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f63408cd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f633c15fb60 0x7f633c13aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f4deab06640 1 -- start start 2024-04-25T09:53:33.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f63408cd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0b3080 con 0x7f633c071d20 2024-04-25T09:53:33.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f63408cd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0b2f00 con 0x7f633c0b2aa0 2024-04-25T09:53:33.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f63408cd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c06dbb0 con 0x7f633c15fb60 2024-04-25T09:53:33.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f29bd12c640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29b0038100 0x7f29b0032e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f3e9dffb640 1 -- 172.21.15.139:0/3010784255 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4037030 con 0x7f3ea806e720 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f3e9dffb640 1 -- 172.21.15.139:0/3010784255 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea4041070 con 0x7f3ea806e720 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f9f9b59e640 1 -- 172.21.15.139:0/970411386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f9c144dd0 msgr2=0x7f9f9c164780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f9f9b59e640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f9c144dd0 0x7f9f9c164780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f9f9b59e640 1 -- 172.21.15.139:0/970411386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f9c1449f0 msgr2=0x7f9f9c141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f9f9b59e640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f9c1449f0 0x7f9f9c141190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f9f9b59e640 1 -- 172.21.15.139:0/970411386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c1f8590 con 0x7f9f9c162c10 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f9f9a59c640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f9c144dd0 0x7f9f9c164780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f9f9b59e640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f9c162c10 0x7f9f9c164cc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400cac0 tx=0x7f9f9400cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f9f837fe640 1 -- 172.21.15.139:0/970411386 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94003d80 con 0x7f9f9c162c10 2024-04-25T09:53:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f9f837fe640 1 -- 172.21.15.139:0/970411386 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9400a700 con 0x7f9f9c162c10 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c141b20 con 0x7f9f9c162c10 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f9f837fe640 1 -- 172.21.15.139:0/970411386 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400aad0 con 0x7f9f9c162c10 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c1420e0 con 0x7f9f9c162c10 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f1f87f89640 1 -- 172.21.15.139:0/1032840590 wait complete. 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f1f87f89640 1 Processor -- start 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f29bd92d640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29b00eacf0 0x7f29b0048c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f29bc92b640 1 -- 172.21.15.139:0/1816752337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29b0038100 msgr2=0x7f29b0032e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f29bc92b640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29b0038100 0x7f29b0032e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f29bc92b640 1 -- 172.21.15.139:0/1816752337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29b00eacf0 msgr2=0x7f29b0048c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f29bc92b640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29b00eacf0 0x7f29b0048c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.535+0000 7f29bc92b640 1 -- 172.21.15.139:0/1816752337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b00d6140 con 0x7f29b004ffc0 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f29bc92b640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29b004ffc0 0x7f29b0048730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f29b400ebe0 tx=0x7f29b400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f633a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f633c071d20 0x7f633c13a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f633ad9d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f633c15fb60 0x7f633c13aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f633a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f633c071d20 0x7f633c13a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58914/0 (socket says 172.21.15.139:58914) 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f633a59c640 1 -- 172.21.15.139:0/1427042034 learned_addr learned my addr 172.21.15.139:0/1427042034 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f6339d9b640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633c0b2aa0 0x7f633c1613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f633a59c640 1 -- 172.21.15.139:0/1427042034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f633c15fb60 msgr2=0x7f633c13aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f633a59c640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f633c15fb60 0x7f633c13aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f633a59c640 1 -- 172.21.15.139:0/1427042034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633c0b2aa0 msgr2=0x7f633c1613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f633a59c640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633c0b2aa0 0x7f633c1613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7f633a59c640 1 -- 172.21.15.139:0/1427042034 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c13b980 con 0x7f633c071d20 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f633a59c640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f633c071d20 0x7f633c13a800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f632c00d620 tx=0x7f632c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7fb63259c640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb62c054b20 0x7fb62c007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7fb63359e640 1 -- 172.21.15.139:0/529022313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb62c04fbd0 msgr2=0x7fb62c174a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7fb63359e640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb62c04fbd0 0x7fb62c174a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7fb63359e640 1 -- 172.21.15.139:0/529022313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb62c054b20 msgr2=0x7fb62c007440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7fb63359e640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb62c054b20 0x7fb62c007440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7fb63359e640 1 -- 172.21.15.139:0/529022313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c007bb0 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7fb63259c640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb62c054b20 0x7fb62c007440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7fb63359e640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0e0dc0 0x7fb62c1717b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb634069730 tx=0x7fb634068690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7fb61b7fe640 1 -- 172.21.15.139:0/529022313 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634073d90 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f4deab06640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ddc0d73b0 0x7f4ddc0d98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f4deab06640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ddc0d7df0 0x7f4ddc0dfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f4deab06640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ddc0e0210 0x7f4ddc04ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f4deab06640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc0e7b80 con 0x7f4ddc0d7df0 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f4deab06640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc03c0b0 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f4deab06640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc006fa0 con 0x7f4ddc0e0210 2024-04-25T09:53:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f1f87f89640 1 -- start start 2024-04-25T09:53:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f7f8affd640 1 -- 172.21.15.139:0/902254777 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f84045020 con 0x7f7f8c008180 2024-04-25T09:53:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8c04f700 con 0x7f7f8c008180 2024-04-25T09:53:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.536+0000 7fb61b7fe640 1 -- 172.21.15.139:0/529022313 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63408e070 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7fb63886a640 1 -- 172.21.15.139:0/529022313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62c0e3eb0 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7fb63886a640 1 -- 172.21.15.139:0/529022313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62c0e4470 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7fb61b7fe640 1 -- 172.21.15.139:0/529022313 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634073d90 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f7f8affd640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f7806b5a0 0x7f7f7806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3010784255 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea8155020 con 0x7f3ea806e720 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3010784255 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea81555e0 con 0x7f3ea806e720 2024-04-25T09:53:33.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.537+0000 7f3e9dffb640 1 -- 172.21.15.139:0/3010784255 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea403c440 con 0x7f3ea806e720 2024-04-25T09:53:33.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f632affd640 1 -- 172.21.15.139:0/1427042034 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00b620 con 0x7f633c071d20 2024-04-25T09:53:33.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f632affd640 1 -- 172.21.15.139:0/1427042034 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c019070 con 0x7f633c071d20 2024-04-25T09:53:33.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f632affd640 1 -- 172.21.15.139:0/1427042034 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00b9e0 con 0x7f633c071d20 2024-04-25T09:53:33.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f63408cd640 1 -- 172.21.15.139:0/1427042034 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c139a40 con 0x7f633c071d20 2024-04-25T09:53:33.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f7f8affd640 1 -- 172.21.15.139:0/902254777 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7f840ab0b0 con 0x7f7f8c008180 2024-04-25T09:53:33.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f1f87f89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f800711c0 0x7f1f80157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f1f87f89640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f800b25f0 0x7f1f80165520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f1f87f89640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f801542a0 0x7f1f8013f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f1f87f89640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8015a680 con 0x7f1f800711c0 2024-04-25T09:53:33.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f29adffb640 1 -- 172.21.15.139:0/1816752337 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4017070 con 0x7f29b004ffc0 2024-04-25T09:53:33.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f29adffb640 1 -- 172.21.15.139:0/1816752337 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b400cc30 con 0x7f29b004ffc0 2024-04-25T09:53:33.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f29adffb640 1 -- 172.21.15.139:0/1816752337 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4013470 con 0x7f29b004ffc0 2024-04-25T09:53:33.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f29bf390640 1 -- 172.21.15.139:0/1816752337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b00d6420 con 0x7f29b004ffc0 2024-04-25T09:53:33.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f29bf390640 1 -- 172.21.15.139:0/1816752337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b0033350 con 0x7f29b004ffc0 2024-04-25T09:53:33.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f4de9b04640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ddc0d73b0 0x7f4ddc0d98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f4de9b04640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ddc0d73b0 0x7f4ddc0d98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:46058/0 (socket says 172.21.15.139:46058) 2024-04-25T09:53:33.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f4de9b04640 1 -- 172.21.15.139:0/676807312 learned_addr learned my addr 172.21.15.139:0/676807312 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f4de9303640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ddc0d7df0 0x7f4ddc0dfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f1f87f89640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8015a500 con 0x7f1f801542a0 2024-04-25T09:53:33.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7fb63886a640 1 -- 172.21.15.139:0/529022313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62c008e10 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f4de9b04640 1 -- 172.21.15.139:0/676807312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ddc0e0210 msgr2=0x7f4ddc04ff70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T09:53:33.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f4de9b04640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ddc0e0210 0x7f4ddc04ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f4de9b04640 1 -- 172.21.15.139:0/676807312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ddc0d7df0 msgr2=0x7f4ddc0dfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f4de9b04640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ddc0d7df0 0x7f4ddc0dfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f4de9b04640 1 -- 172.21.15.139:0/676807312 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc0db790 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f4de9303640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ddc0d7df0 0x7f4ddc0dfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:33.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f1f87f89640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8015a800 con 0x7f1f800b25f0 2024-04-25T09:53:33.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f7f99a4e640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f7806b5a0 0x7f7f7806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.538+0000 7f1f85524640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f800b25f0 0x7f1f80165520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f4de9b04640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ddc0d73b0 0x7f4ddc0d98b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400e7e0 tx=0x7f4dd400eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f1f85524640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f800b25f0 0x7f1f80165520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57322/0 (socket says 172.21.15.139:57322) 2024-04-25T09:53:33.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f1f85524640 1 -- 172.21.15.139:0/1108621797 learned_addr learned my addr 172.21.15.139:0/1108621797 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:33.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f1f85d25640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f800711c0 0x7f1f80157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f1f86526640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f801542a0 0x7f1f8013f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f63408cd640 1 -- 172.21.15.139:0/1427042034 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c13a000 con 0x7f633c071d20 2024-04-25T09:53:33.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c074520 con 0x7f9f9c162c10 2024-04-25T09:53:33.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7f29bf390640 1 -- 172.21.15.139:0/1816752337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b0005380 con 0x7f29b004ffc0 2024-04-25T09:53:33.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.539+0000 7f4dda7fc640 1 -- 172.21.15.139:0/676807312 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd400c6c0 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7f4deab06640 1 -- 172.21.15.139:0/676807312 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ddc0d85d0 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7f4deab06640 1 -- 172.21.15.139:0/676807312 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc0d8b90 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7f1f86526640 1 -- 172.21.15.139:0/1108621797 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f800b25f0 msgr2=0x7f1f80165520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7f1f86526640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f800b25f0 0x7f1f80165520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7f1f86526640 1 -- 172.21.15.139:0/1108621797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f800711c0 msgr2=0x7f1f80157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7f1f86526640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f800711c0 0x7f1f80157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7f1f86526640 1 -- 172.21.15.139:0/1108621797 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8014d1c0 con 0x7f1f801542a0 2024-04-25T09:53:33.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7f4dda7fc640 1 -- 172.21.15.139:0/676807312 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd4019070 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7f4dda7fc640 1 -- 172.21.15.139:0/676807312 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd400caa0 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7f1f86526640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f801542a0 0x7f1f8013f320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f78007ef0 tx=0x7f1f7800d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7f7f99a4e640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f7806b5a0 0x7f7f7806da60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7f940695f0 tx=0x7f7f9406e040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7fb61b7fe640 1 -- 172.21.15.139:0/529022313 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6340923d0 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.540+0000 7fb61b7fe640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb62006b900 0x7fb62006ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3010784255 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea80b44d0 con 0x7f3ea806e720 2024-04-25T09:53:33.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f63408cd640 1 -- 172.21.15.139:0/1427042034 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c074520 con 0x7f633c071d20 2024-04-25T09:53:33.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f632affd640 1 -- 172.21.15.139:0/1427042034 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f632c00bb80 con 0x7f633c071d20 2024-04-25T09:53:33.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7fb61b7fe640 1 -- 172.21.15.139:0/529022313 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb634067f20 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f1f767fc640 1 -- 172.21.15.139:0/1108621797 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7801a070 con 0x7f1f801542a0 2024-04-25T09:53:33.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f1f767fc640 1 -- 172.21.15.139:0/1108621797 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f78003b90 con 0x7f1f801542a0 2024-04-25T09:53:33.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f1f767fc640 1 -- 172.21.15.139:0/1108621797 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78004030 con 0x7f1f801542a0 2024-04-25T09:53:33.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f9f837fe640 1 -- 172.21.15.139:0/970411386 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f9400ac70 con 0x7f9f9c162c10 2024-04-25T09:53:33.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f4deab06640 1 -- 172.21.15.139:0/676807312 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dcc005290 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f1f87f89640 1 -- 172.21.15.139:0/1108621797 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8014d4a0 con 0x7f1f801542a0 2024-04-25T09:53:33.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f1f87f89640 1 -- 172.21.15.139:0/1108621797 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8014da60 con 0x7f1f801542a0 2024-04-25T09:53:33.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f632affd640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f631c06b520 0x7f631c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f632affd640 1 -- 172.21.15.139:0/1427042034 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f632c08a8f0 con 0x7f633c071d20 2024-04-25T09:53:33.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.542+0000 7f6339d9b640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f631c06b520 0x7f631c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.542+0000 7fb632d9d640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb62006b900 0x7fb62006ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.542+0000 7f9f837fe640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9f8806b4d0 0x7f9f8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.542+0000 7f4dda7fc640 1 -- 172.21.15.139:0/676807312 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4dd400cc40 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.542+0000 7f9f837fe640 1 -- 172.21.15.139:0/970411386 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9f9408a4e0 con 0x7f9f9c162c10 2024-04-25T09:53:33.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.541+0000 7f29adffb640 1 -- 172.21.15.139:0/1816752337 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29b401a070 con 0x7f29b004ffc0 2024-04-25T09:53:33.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.542+0000 7f29adffb640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f299c06b5a0 0x7f299c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.542+0000 7f29adffb640 1 -- 172.21.15.139:0/1816752337 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f29b408a8b0 con 0x7f29b004ffc0 2024-04-25T09:53:33.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.542+0000 7f9f9ad9d640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9f8806b4d0 0x7f9f8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.542+0000 7f29bd12c640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f299c06b5a0 0x7f299c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.542+0000 7f4dda7fc640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4dc806b550 0x7f4dc806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.542+0000 7f6339d9b640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f631c06b520 0x7f631c06d9e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f633c0cd090 tx=0x7f6330009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.543+0000 7f4dda7fc640 1 -- 172.21.15.139:0/676807312 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4dd408a580 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.543+0000 7f4de9303640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4dc806b550 0x7f4dc806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.543+0000 7f29bd12c640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f299c06b5a0 0x7f299c06da60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f29b0053b70 tx=0x7f29b80a9000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.543+0000 7f9f9ad9d640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9f8806b4d0 0x7f9f8806d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c000c00 tx=0x7f9f8c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.543+0000 7fb632d9d640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb62006b900 0x7fb62006ddc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb624034000 tx=0x7fb62402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.543+0000 7f4de9303640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4dc806b550 0x7f4dc806da10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00057e0 tx=0x7f4de0030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.543+0000 7f1f87f89640 1 -- 172.21.15.139:0/1108621797 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8014dda0 con 0x7f1f801542a0 2024-04-25T09:53:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.545+0000 7f7f8affd640 1 -- 172.21.15.139:0/902254777 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f8c04f700 con 0x7f7f8c008180 2024-04-25T09:53:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.546+0000 7f3e9dffb640 1 -- 172.21.15.139:0/3010784255 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ea403c5e0 con 0x7f3ea806e720 2024-04-25T09:53:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.547+0000 7f3e9dffb640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e8c06b5a0 0x7f3e8c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.547+0000 7f3e9dffb640 1 -- 172.21.15.139:0/3010784255 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3ea40ab110 con 0x7f3ea806e720 2024-04-25T09:53:33.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.547+0000 7f3eaccae640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e8c06b5a0 0x7f3e8c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.547+0000 7f632affd640 1 -- 172.21.15.139:0/1427042034 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f632c059420 con 0x7f633c071d20 2024-04-25T09:53:33.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.547+0000 7f3eaccae640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e8c06b5a0 0x7f3e8c06da60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0001b80 tx=0x7f3ea0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.552+0000 7fb61b7fe640 1 -- 172.21.15.139:0/529022313 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6340d26c0 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.555+0000 7f3e9dffb640 1 -- 172.21.15.139:0/3010784255 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ea80b44d0 con 0x7f3ea806e720 2024-04-25T09:53:33.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.556+0000 7f29adffb640 1 -- 172.21.15.139:0/1816752337 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29b4059340 con 0x7f29b004ffc0 2024-04-25T09:53:33.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.558+0000 7f9f837fe640 1 -- 172.21.15.139:0/970411386 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f940589c0 con 0x7f9f9c162c10 2024-04-25T09:53:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.564+0000 7f1f767fc640 1 -- 172.21.15.139:0/1108621797 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f78005d10 con 0x7f1f801542a0 2024-04-25T09:53:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.564+0000 7f4dda7fc640 1 -- 172.21.15.139:0/676807312 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4dd40590a0 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.564+0000 7f1f767fc640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1f6406b520 0x7f1f6406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:33.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.564+0000 7f1f767fc640 1 -- 172.21.15.139:0/1108621797 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1f7808af00 con 0x7f1f801542a0 2024-04-25T09:53:33.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.565+0000 7f1f85d25640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1f6406b520 0x7f1f6406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:33.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.565+0000 7f1f85d25640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1f6406b520 0x7f1f6406d9e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1f801fae90 tx=0x7f1f70009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:33.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.567+0000 7f1f767fc640 1 -- 172.21.15.139:0/1108621797 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f8014dda0 con 0x7f1f801542a0 2024-04-25T09:53:33.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.945+0000 7f63408cd640 1 -- 172.21.15.139:0/1427042034 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f633c0b24e0 con 0x7f633c071d20 2024-04-25T09:53:33.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.945+0000 7f632affd640 1 -- 172.21.15.139:0/1427042034 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f632c058d20 con 0x7f633c071d20 2024-04-25T09:53:33.949 INFO:teuthology.orchestra.run.smithi139.stdout:55834574856 2024-04-25T09:53:33.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.949+0000 7f6328ff9640 1 -- 172.21.15.139:0/1427042034 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f631c06b520 msgr2=0x7f631c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.949+0000 7f6328ff9640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f631c06b520 0x7f631c06d9e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f633c0cd090 tx=0x7f6330009040 comp rx=0 tx=0).stop 2024-04-25T09:53:33.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.949+0000 7f6328ff9640 1 -- 172.21.15.139:0/1427042034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f633c071d20 msgr2=0x7f633c13a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.949+0000 7f6328ff9640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f633c071d20 0x7f633c13a800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f632c00d620 tx=0x7f632c00dae0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.950+0000 7f6328ff9640 1 -- 172.21.15.139:0/1427042034 shutdown_connections 2024-04-25T09:53:33.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.950+0000 7f6328ff9640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f631c06b520 0x7f631c06d9e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.950+0000 7f6328ff9640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f633c15fb60 0x7f633c13aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.950+0000 7f6328ff9640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f633c0b2aa0 0x7f633c1613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.950+0000 7f6328ff9640 1 --2- 172.21.15.139:0/1427042034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f633c071d20 0x7f633c13a800 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.950+0000 7f6328ff9640 1 -- 172.21.15.139:0/1427042034 >> 172.21.15.139:0/1427042034 conn(0x7f633c0b17a0 msgr2=0x7f633c14dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.951+0000 7f6328ff9640 1 -- 172.21.15.139:0/1427042034 shutdown_connections 2024-04-25T09:53:33.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.951+0000 7f6328ff9640 1 -- 172.21.15.139:0/1427042034 wait complete. 2024-04-25T09:53:33.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.956+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7f8c0057d0 con 0x7f7f8c008180 2024-04-25T09:53:33.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.957+0000 7f7f8affd640 1 -- 172.21.15.139:0/902254777 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f8404c030 con 0x7f7f8c008180 2024-04-25T09:53:33.960 INFO:teuthology.orchestra.run.smithi139.stdout:55834574856 2024-04-25T09:53:33.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.957+0000 7fb63886a640 1 -- 172.21.15.139:0/529022313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb62c0057d0 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.959+0000 7fb61b7fe640 1 -- 172.21.15.139:0/529022313 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6340d6760 con 0x7fb62c0e0dc0 2024-04-25T09:53:33.962 INFO:teuthology.orchestra.run.smithi139.stdout:55834574856 2024-04-25T09:53:33.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.960+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f7806b5a0 msgr2=0x7f7f7806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.960+0000 7f7f9b251640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f7806b5a0 0x7f7f7806da60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7f940695f0 tx=0x7f7f9406e040 comp rx=0 tx=0).stop 2024-04-25T09:53:33.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.961+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f8c008180 msgr2=0x7f7f8c04d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.961+0000 7f7f9b251640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f8c008180 0x7f7f8c04d300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f840096f0 tx=0x7f7f840046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.961+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 shutdown_connections 2024-04-25T09:53:33.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.961+0000 7f7f9b251640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f8c0548a0 0x7f7f8c044910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.961+0000 7f7f9b251640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f7806b5a0 0x7f7f7806da60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.961+0000 7f7f9b251640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c04fc00 0x7f7f8c0d9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.961+0000 7f7f9b251640 1 --2- 172.21.15.139:0/902254777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f8c008180 0x7f7f8c04d300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.961+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 >> 172.21.15.139:0/902254777 conn(0x7f7f8c055340 msgr2=0x7f7f8c0e7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.961+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 shutdown_connections 2024-04-25T09:53:33.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.961+0000 7f7f9b251640 1 -- 172.21.15.139:0/902254777 wait complete. 2024-04-25T09:53:33.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.962+0000 7fb63886a640 1 -- 172.21.15.139:0/529022313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb62006b900 msgr2=0x7fb62006ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.962+0000 7fb63886a640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb62006b900 0x7fb62006ddc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb624034000 tx=0x7fb62402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:33.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.962+0000 7fb63886a640 1 -- 172.21.15.139:0/529022313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0e0dc0 msgr2=0x7fb62c1717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.962+0000 7fb63886a640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0e0dc0 0x7fb62c1717b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb634069730 tx=0x7fb634068690 comp rx=0 tx=0).stop 2024-04-25T09:53:33.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.963+0000 7fb63886a640 1 -- 172.21.15.139:0/529022313 shutdown_connections 2024-04-25T09:53:33.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.963+0000 7fb63886a640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb62c054b20 0x7fb62c007440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.963+0000 7fb63886a640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb62006b900 0x7fb62006ddc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.963+0000 7fb63886a640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0e0dc0 0x7fb62c1717b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.963+0000 7fb63886a640 1 --2- 172.21.15.139:0/529022313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb62c04fbd0 0x7fb62c174a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.963+0000 7fb63886a640 1 -- 172.21.15.139:0/529022313 >> 172.21.15.139:0/529022313 conn(0x7fb62c00a3f0 msgr2=0x7fb62c0e85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.963+0000 7fb63886a640 1 -- 172.21.15.139:0/529022313 shutdown_connections 2024-04-25T09:53:33.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.963+0000 7fb63886a640 1 -- 172.21.15.139:0/529022313 wait complete. 2024-04-25T09:53:33.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.966+0000 7f29bf390640 1 -- 172.21.15.139:0/1816752337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f29b00478c0 con 0x7f29b004ffc0 2024-04-25T09:53:33.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.967+0000 7f29adffb640 1 -- 172.21.15.139:0/1816752337 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29b4058c40 con 0x7f29b004ffc0 2024-04-25T09:53:33.971 INFO:teuthology.orchestra.run.smithi139.stdout:55834574856 2024-04-25T09:53:33.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.7 2024-04-25T09:53:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.971+0000 7f29bf390640 1 -- 172.21.15.139:0/1816752337 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f299c06b5a0 msgr2=0x7f299c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.971+0000 7f29bf390640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f299c06b5a0 0x7f299c06da60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f29b0053b70 tx=0x7f29b80a9000 comp rx=0 tx=0).stop 2024-04-25T09:53:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.971+0000 7f29bf390640 1 -- 172.21.15.139:0/1816752337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29b004ffc0 msgr2=0x7f29b0048730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.971+0000 7f29bf390640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29b004ffc0 0x7f29b0048730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f29b400ebe0 tx=0x7f29b400c520 comp rx=0 tx=0).stop 2024-04-25T09:53:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.971+0000 7f29bf390640 1 -- 172.21.15.139:0/1816752337 shutdown_connections 2024-04-25T09:53:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.971+0000 7f29bf390640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29b004ffc0 0x7f29b0048730 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.971+0000 7f29bf390640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f299c06b5a0 0x7f299c06da60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.971+0000 7f29bf390640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29b00eacf0 0x7f29b0048c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.971+0000 7f29bf390640 1 --2- 172.21.15.139:0/1816752337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29b0038100 0x7f29b0032e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.971+0000 7f29bf390640 1 -- 172.21.15.139:0/1816752337 >> 172.21.15.139:0/1816752337 conn(0x7f29b0009fa0 msgr2=0x7f29b004db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.971+0000 7f29bf390640 1 -- 172.21.15.139:0/1816752337 shutdown_connections 2024-04-25T09:53:33.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.971+0000 7f29bf390640 1 -- 172.21.15.139:0/1816752337 wait complete. 2024-04-25T09:53:33.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.973+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3010784255 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3ea8067510 con 0x7f3ea806e720 2024-04-25T09:53:33.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.974+0000 7f3e9dffb640 1 -- 172.21.15.139:0/3010784255 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ea8067510 con 0x7f3ea806e720 2024-04-25T09:53:33.977 INFO:teuthology.orchestra.run.smithi139.stdout:55834574855 2024-04-25T09:53:33.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.976+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9f9c158fb0 con 0x7f9f9c162c10 2024-04-25T09:53:33.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.976+0000 7f9f837fe640 1 -- 172.21.15.139:0/970411386 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9f9405ca60 con 0x7f9f9c162c10 2024-04-25T09:53:33.979 INFO:teuthology.orchestra.run.smithi139.stdout:55834574856 2024-04-25T09:53:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.977+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3010784255 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e8c06b5a0 msgr2=0x7f3e8c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.977+0000 7f3eae4b1640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e8c06b5a0 0x7f3e8c06da60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0001b80 tx=0x7f3ea0009040 comp rx=0 tx=0).stop 2024-04-25T09:53:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.977+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3010784255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea806e720 msgr2=0x7f3ea81faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.977+0000 7f3eae4b1640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea806e720 0x7f3ea81faec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40098c0 tx=0x7f3ea40046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.977+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3010784255 shutdown_connections 2024-04-25T09:53:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.977+0000 7f3eae4b1640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ea80b25b0 0x7f3ea815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.977+0000 7f3eae4b1640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e8c06b5a0 0x7f3e8c06da60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.977+0000 7f3eae4b1640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ea80b1da0 0x7f3ea815d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.977+0000 7f3eae4b1640 1 --2- 172.21.15.139:0/3010784255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ea806e720 0x7f3ea81faec0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.977+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3010784255 >> 172.21.15.139:0/3010784255 conn(0x7f3ea80b12b0 msgr2=0x7f3ea80b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.977+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3010784255 shutdown_connections 2024-04-25T09:53:33.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.978+0000 7f3eae4b1640 1 -- 172.21.15.139:0/3010784255 wait complete. 2024-04-25T09:53:33.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.4 2024-04-25T09:53:33.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.980+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9f8806b4d0 msgr2=0x7f9f8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.980+0000 7f9fa08cc640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9f8806b4d0 0x7f9f8806d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c000c00 tx=0x7f9f8c02e000 comp rx=0 tx=0).stop 2024-04-25T09:53:33.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.980+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f9c162c10 msgr2=0x7f9f9c164cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.980+0000 7f9fa08cc640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f9c162c10 0x7f9f9c164cc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400cac0 tx=0x7f9f9400cf80 comp rx=0 tx=0).stop 2024-04-25T09:53:33.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.6 2024-04-25T09:53:33.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.980+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 shutdown_connections 2024-04-25T09:53:33.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.980+0000 7f9fa08cc640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9f8806b4d0 0x7f9f8806d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.980+0000 7f9fa08cc640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f9c162c10 0x7f9f9c164cc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.980+0000 7f9fa08cc640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f9c144dd0 0x7f9f9c164780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.980+0000 7f9fa08cc640 1 --2- 172.21.15.139:0/970411386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f9c1449f0 0x7f9f9c141190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.980+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 >> 172.21.15.139:0/970411386 conn(0x7f9f9c066f20 msgr2=0x7f9f9c1469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.981+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 shutdown_connections 2024-04-25T09:53:33.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.981+0000 7f9fa08cc640 1 -- 172.21.15.139:0/970411386 wait complete. 2024-04-25T09:53:33.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.982+0000 7f4deab06640 1 -- 172.21.15.139:0/676807312 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4dcc005580 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.983+0000 7f4dda7fc640 1 -- 172.21.15.139:0/676807312 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4dd40589a0 con 0x7f4ddc0d73b0 2024-04-25T09:53:33.986 INFO:teuthology.orchestra.run.smithi139.stdout:55834574855 2024-04-25T09:53:33.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.983+0000 7f1f87f89640 1 -- 172.21.15.139:0/1108621797 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1f800af230 con 0x7f1f801542a0 2024-04-25T09:53:33.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.983+0000 7f1f767fc640 1 -- 172.21.15.139:0/1108621797 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f78010070 con 0x7f1f801542a0 2024-04-25T09:53:33.987 INFO:teuthology.orchestra.run.smithi139.stdout:55834574857 2024-04-25T09:53:33.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.985+0000 7f4deab06640 1 -- 172.21.15.139:0/676807312 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4dc806b550 msgr2=0x7f4dc806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.985+0000 7f4deab06640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4dc806b550 0x7f4dc806da10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00057e0 tx=0x7f4de0030000 comp rx=0 tx=0).stop 2024-04-25T09:53:33.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.985+0000 7f4deab06640 1 -- 172.21.15.139:0/676807312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ddc0d73b0 msgr2=0x7f4ddc0d98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.985+0000 7f4deab06640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ddc0d73b0 0x7f4ddc0d98b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400e7e0 tx=0x7f4dd400eca0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f4deab06640 1 -- 172.21.15.139:0/676807312 shutdown_connections 2024-04-25T09:53:33.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f4deab06640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ddc0d7df0 0x7f4ddc0dfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f4deab06640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4dc806b550 0x7f4dc806da10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f4deab06640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ddc0e0210 0x7f4ddc04ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f4deab06640 1 --2- 172.21.15.139:0/676807312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ddc0d73b0 0x7f4ddc0d98b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f4deab06640 1 -- 172.21.15.139:0/676807312 >> 172.21.15.139:0/676807312 conn(0x7f4ddc03aa20 msgr2=0x7f4ddc03ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f4deab06640 1 -- 172.21.15.139:0/676807312 shutdown_connections 2024-04-25T09:53:33.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f4deab06640 1 -- 172.21.15.139:0/676807312 wait complete. 2024-04-25T09:53:33.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f1f87f89640 1 -- 172.21.15.139:0/1108621797 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1f6406b520 msgr2=0x7f1f6406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f1f87f89640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1f6406b520 0x7f1f6406d9e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1f801fae90 tx=0x7f1f70009040 comp rx=0 tx=0).stop 2024-04-25T09:53:33.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f1f87f89640 1 -- 172.21.15.139:0/1108621797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f801542a0 msgr2=0x7f1f8013f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:33.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f1f87f89640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f801542a0 0x7f1f8013f320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f78007ef0 tx=0x7f1f7800d4a0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f1f87f89640 1 -- 172.21.15.139:0/1108621797 shutdown_connections 2024-04-25T09:53:33.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f1f87f89640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1f6406b520 0x7f1f6406d9e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f1f87f89640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f801542a0 0x7f1f8013f320 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f1f87f89640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f800b25f0 0x7f1f80165520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f1f87f89640 1 --2- 172.21.15.139:0/1108621797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f800711c0 0x7f1f80157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:33.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f1f87f89640 1 -- 172.21.15.139:0/1108621797 >> 172.21.15.139:0/1108621797 conn(0x7f1f8006af80 msgr2=0x7f1f800ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:33.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f1f87f89640 1 -- 172.21.15.139:0/1108621797 shutdown_connections 2024-04-25T09:53:33.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:33.986+0000 7f1f87f89640 1 -- 172.21.15.139:0/1108621797 wait complete. 2024-04-25T09:53:33.994 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.0 2024-04-25T09:53:33.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.1 2024-04-25T09:53:34.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.3 2024-04-25T09:53:34.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.5 2024-04-25T09:53:34.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.2 2024-04-25T09:53:34.973 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T09:53:34.983 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T09:53:34.985 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T09:53:34.996 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T09:53:34.997 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T09:53:35.002 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T09:53:35.004 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T09:53:35.005 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T09:53:35.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.127+0000 7fd4c98d2640 1 Processor -- start 2024-04-25T09:53:35.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.127+0000 7fd4c98d2640 1 -- start start 2024-04-25T09:53:35.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.128+0000 7fd4c98d2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd4bc0e7530 0x7fd4bc03a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.128+0000 7fd4c98d2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc056060 0x7fd4bc036120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.128+0000 7fd4c98d2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd4bc00dc60 0x7fd4bc00e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.128+0000 7fd4c98d2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4bc0516d0 con 0x7fd4bc0e7530 2024-04-25T09:53:35.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.128+0000 7fd4c98d2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4bc0553f0 con 0x7fd4bc00dc60 2024-04-25T09:53:35.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.128+0000 7fd4c98d2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc0556d0 con 0x7fd4bc056060 2024-04-25T09:53:35.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.129+0000 7fd4c90d1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd4bc00dc60 0x7fd4bc00e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.129+0000 7fd4c90d1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd4bc00dc60 0x7fd4bc00e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:46080/0 (socket says 172.21.15.139:46080) 2024-04-25T09:53:35.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.129+0000 7fd4c90d1640 1 -- 172.21.15.139:0/1557645090 learned_addr learned my addr 172.21.15.139:0/1557645090 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.129+0000 7fd4c3fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc056060 0x7fd4bc036120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.129+0000 7fd4c88d0640 1 --2- 172.21.15.139:0/1557645090 >> v2:172.21.15.139:3300/0 conn(0x7fd4bc0e7530 0x7fd4bc03a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.130+0000 7fd4c88d0640 1 -- 172.21.15.139:0/1557645090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc056060 msgr2=0x7fd4bc036120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.130+0000 7fd4c88d0640 1 --2- 172.21.15.139:0/1557645090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc056060 0x7fd4bc036120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.130+0000 7fd4c88d0640 1 -- 172.21.15.139:0/1557645090 >> v2:172.21.15.150:3300/0 conn(0x7fd4bc00dc60 msgr2=0x7fd4bc00e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.130+0000 7fd4c88d0640 1 --2- 172.21.15.139:0/1557645090 >> v2:172.21.15.150:3300/0 conn(0x7fd4bc00dc60 0x7fd4bc00e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.130+0000 7fd4c88d0640 1 -- 172.21.15.139:0/1557645090 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc0559b0 con 0x7fd4bc0e7530 2024-04-25T09:53:35.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.131+0000 7fd4c88d0640 1 --2- 172.21.15.139:0/1557645090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4bc0e7530 0x7fd4bc03a740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd4c404fcd0 tx=0x7fd4c40a4fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccf00cc8141af79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.132+0000 7fd4c37fe640 1 -- 172.21.15.139:0/1557645090 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40b2070 con 0x7fd4bc0e7530 2024-04-25T09:53:35.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.132+0000 7fd4c37fe640 1 -- 172.21.15.139:0/1557645090 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c40a5e30 con 0x7fd4bc0e7530 2024-04-25T09:53:35.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.132+0000 7fd4c37fe640 1 -- 172.21.15.139:0/1557645090 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40ad640 con 0x7fd4bc0e7530 2024-04-25T09:53:35.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.133+0000 7fd4c98d2640 1 -- 172.21.15.139:0/1557645090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4bc0e7530 msgr2=0x7fd4bc03a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.133+0000 7fd4c98d2640 1 --2- 172.21.15.139:0/1557645090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4bc0e7530 0x7fd4bc03a740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd4c404fcd0 tx=0x7fd4c40a4fe0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.134+0000 7fd4c98d2640 1 -- 172.21.15.139:0/1557645090 shutdown_connections 2024-04-25T09:53:35.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.134+0000 7fd4c98d2640 1 --2- 172.21.15.139:0/1557645090 >> v2:172.21.15.150:3300/0 conn(0x7fd4bc00dc60 0x7fd4bc00e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.134+0000 7fd4c98d2640 1 --2- 172.21.15.139:0/1557645090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc056060 0x7fd4bc036120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.134+0000 7fd4c98d2640 1 --2- 172.21.15.139:0/1557645090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4bc0e7530 0x7fd4bc03a740 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.134+0000 7fd4c98d2640 1 -- 172.21.15.139:0/1557645090 >> 172.21.15.139:0/1557645090 conn(0x7fd4bc056ce0 msgr2=0x7fd4bc0570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.134+0000 7fd4c98d2640 1 -- 172.21.15.139:0/1557645090 shutdown_connections 2024-04-25T09:53:35.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.134+0000 7fd4c98d2640 1 -- 172.21.15.139:0/1557645090 wait complete. 2024-04-25T09:53:35.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.136+0000 7fd4c98d2640 1 Processor -- start 2024-04-25T09:53:35.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.136+0000 7fd4c98d2640 1 -- start start 2024-04-25T09:53:35.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.138+0000 7fd4c98d2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4bc00dc60 0x7fd4bc00a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.138+0000 7fd4c98d2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4bc056060 0x7fd4bc038930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.138+0000 7fd4c98d2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc0e7530 0x7fd4bc03a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.138+0000 7fd4c3fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4bc056060 0x7fd4bc038930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.138+0000 7fd4c3fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4bc056060 0x7fd4bc038930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:46090/0 (socket says 172.21.15.139:46090) 2024-04-25T09:53:35.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.138+0000 7fd4c88d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4bc00dc60 0x7fd4bc00a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.139+0000 7fd4c88d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4bc00dc60 0x7fd4bc00a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58940/0 (socket says 172.21.15.139:58940) 2024-04-25T09:53:35.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.139+0000 7fd4c98d2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc0eb130 con 0x7fd4bc00dc60 2024-04-25T09:53:35.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.139+0000 7fd4c98d2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc0e7910 con 0x7fd4bc056060 2024-04-25T09:53:35.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.139+0000 7fd4c98d2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc0eb2b0 con 0x7fd4bc0e7530 2024-04-25T09:53:35.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.139+0000 7fd4c3fff640 1 -- 172.21.15.139:0/3781464649 learned_addr learned my addr 172.21.15.139:0/3781464649 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.139+0000 7fd4c90d1640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc0e7530 0x7fd4bc03a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.139+0000 7fd4c90d1640 1 -- 172.21.15.139:0/3781464649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4bc056060 msgr2=0x7fd4bc038930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.139+0000 7fd4c90d1640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4bc056060 0x7fd4bc038930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.139+0000 7fd4c90d1640 1 -- 172.21.15.139:0/3781464649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4bc00dc60 msgr2=0x7fd4bc00a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.140+0000 7fd4c90d1640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4bc00dc60 0x7fd4bc00a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.140+0000 7fd4c90d1640 1 -- 172.21.15.139:0/3781464649 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc0cf340 con 0x7fd4bc0e7530 2024-04-25T09:53:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.140+0000 7fd4c88d0640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4bc00dc60 0x7fd4bc00a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.140+0000 7fd4c90d1640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc0e7530 0x7fd4bc03a700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800e840 tx=0x7fd4b800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.140+0000 7fd4c17fa640 1 -- 172.21.15.139:0/3781464649 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8017070 con 0x7fd4bc0e7530 2024-04-25T09:53:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.140+0000 7fd4c17fa640 1 -- 172.21.15.139:0/3781464649 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b800c940 con 0x7fd4bc0e7530 2024-04-25T09:53:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.140+0000 7fd4c17fa640 1 -- 172.21.15.139:0/3781464649 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b800cd10 con 0x7fd4bc0e7530 2024-04-25T09:53:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.140+0000 7fd4c98d2640 1 -- 172.21.15.139:0/3781464649 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4bc0cf620 con 0x7fd4bc0e7530 2024-04-25T09:53:35.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.141+0000 7fd4c98d2640 1 -- 172.21.15.139:0/3781464649 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4bc0cfbe0 con 0x7fd4bc0e7530 2024-04-25T09:53:35.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.143+0000 7fd4c17fa640 1 -- 172.21.15.139:0/3781464649 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4b801a070 con 0x7fd4bc0e7530 2024-04-25T09:53:35.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.144+0000 7fd4c17fa640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd4b406b610 0x7fd4b406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.144+0000 7fd4c17fa640 1 -- 172.21.15.139:0/3781464649 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd4b808a720 con 0x7fd4bc0e7530 2024-04-25T09:53:35.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.144+0000 7fd4c88d0640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd4b406b610 0x7fd4b406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.144+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3781464649 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4bc0ed5b0 con 0x7fd4bc0e7530 2024-04-25T09:53:35.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.145+0000 7fd4c88d0640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd4b406b610 0x7fd4b406dad0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00d540 tx=0x7fd4c40a9000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.150+0000 7fd4c17fa640 1 -- 172.21.15.139:0/3781464649 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4bc0ed5b0 con 0x7fd4bc0e7530 2024-04-25T09:53:35.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.187+0000 7f7b20d9e640 1 Processor -- start 2024-04-25T09:53:35.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.187+0000 7f7b20d9e640 1 -- start start 2024-04-25T09:53:35.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.188+0000 7f7b20d9e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7b1c0af7f0 0x7f7b1c0b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.188+0000 7f7b20d9e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1c0afbd0 con 0x7f7b1c16a5c0 2024-04-25T09:53:35.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.188+0000 7f7b20d9e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1c0afe40 con 0x7f7b1c0b1590 2024-04-25T09:53:35.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.188+0000 7f7b20d9e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1c16aa50 con 0x7f7b1c0af7f0 2024-04-25T09:53:35.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.189+0000 7f7b1ad9d640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f7b1c0b1590 0x7f7b1c0b1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:37478/0 (socket says 172.21.15.139:37478) 2024-04-25T09:53:35.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.189+0000 7f7b1ad9d640 1 -- 172.21.15.139:0/3686538158 learned_addr learned my addr 172.21.15.139:0/3686538158 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:35.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.189+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125529729 0 0) 0x7f7b1c0afe40 con 0x7f7b1c0b1590 2024-04-25T09:53:35.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.189+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b0400bd30 con 0x7f7b1c0b1590 2024-04-25T09:53:35.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.189+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361277182 0 0) 0x7f7b1c0afbd0 con 0x7f7b1c16a5c0 2024-04-25T09:53:35.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.189+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b1c0afe40 con 0x7f7b1c16a5c0 2024-04-25T09:53:35.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f7b1a59c640 1 --2- 172.21.15.139:0/3686538158 >> v2:172.21.15.150:3300/0 conn(0x7f7b1c0af7f0 0x7f7b1c0b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 <== mon.1 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3777092562 0 0) 0x7f7b0400bd30 con 0x7f7b1c0b1590 2024-04-25T09:53:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 >> v2:172.21.15.150:3300/0 conn(0x7f7b1c0af7f0 msgr2=0x7f7b1c0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f7b1959a640 1 --2- 172.21.15.139:0/3686538158 >> v2:172.21.15.150:3300/0 conn(0x7f7b1c0af7f0 0x7f7b1c0b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 >> v1:172.21.15.139:6789/0 conn(0x7f7b1c16a5c0 legacy=0x7f7b1c16b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0b0120 con 0x7f7b1c0b1590 2024-04-25T09:53:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 <== mon.1 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f7b140019f0 con 0x7f7b1c0b1590 2024-04-25T09:53:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 >> v1:172.21.15.150:6789/0 conn(0x7f7b1c0b1590 legacy=0x7f7b1c0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f7b1959a640 1 --2- 172.21.15.139:0/3686538158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0400c350 0x7f7b0400c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f7b1959a640 1 --2- 172.21.15.139:0/3686538158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0400d010 0x7f7b0400d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f7b1a59c640 1 --2- 172.21.15.139:0/3686538158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0400c350 0x7f7b0400c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f7b1959a640 1 --2- 172.21.15.139:0/3686538158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b0400d9d0 0x7f7b040141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c16aa50 con 0x7f7b0400d010 2024-04-25T09:53:35.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04014820 con 0x7f7b0400c350 2024-04-25T09:53:35.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04014a20 con 0x7f7b0400d9d0 2024-04-25T09:53:35.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.189+0000 7f6cadda7640 1 Processor -- start 2024-04-25T09:53:35.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f6cadda7640 1 -- start start 2024-04-25T09:53:35.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7fb44923c640 1 Processor -- start 2024-04-25T09:53:35.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7fb44923c640 1 -- start start 2024-04-25T09:53:35.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f2df1dd5640 1 Processor -- start 2024-04-25T09:53:35.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f7b19d9b640 1 --2- 172.21.15.139:0/3686538158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0400d010 0x7f7b0400d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f7b1ad9d640 1 --2- 172.21.15.139:0/3686538158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b0400d9d0 0x7f7b040141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f6cadda7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca00065e0 0x7f6ca0006a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f6cadda7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6ca0006f80 0x7f6ca0007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f6cadda7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca003d020 con 0x7f6ca0006200 2024-04-25T09:53:35.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f6cadda7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca0057a70 con 0x7f6ca0006f80 2024-04-25T09:53:35.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f6cadda7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0057d50 con 0x7f6ca00065e0 2024-04-25T09:53:35.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f7b1a59c640 1 -- 172.21.15.139:0/3686538158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b0400d9d0 msgr2=0x7f7b040141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f7b1a59c640 1 --2- 172.21.15.139:0/3686538158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b0400d9d0 0x7f7b040141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f7b1a59c640 1 -- 172.21.15.139:0/3686538158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0400d010 msgr2=0x7f7b0400d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f7b1a59c640 1 --2- 172.21.15.139:0/3686538158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0400d010 0x7f7b0400d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f3b28ab3640 1 Processor -- start 2024-04-25T09:53:35.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f7b1a59c640 1 -- 172.21.15.139:0/3686538158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0400bd30 con 0x7f7b0400c350 2024-04-25T09:53:35.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f6cad5a6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6ca0006f80 0x7f6ca0007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f6cad5a6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6ca0006f80 0x7f6ca0007410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58956/0 (socket says 172.21.15.139:58956) 2024-04-25T09:53:35.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f6cad5a6640 1 -- 172.21.15.139:0/455022911 learned_addr learned my addr 172.21.15.139:0/455022911 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.190+0000 7f2df1dd5640 1 -- start start 2024-04-25T09:53:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f2df1dd5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2de40df410 0x7f2de4053250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f2df1dd5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de4006ae0 con 0x7f2de404fa80 2024-04-25T09:53:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f2df1dd5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de4006dc0 con 0x7f2de40392c0 2024-04-25T09:53:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f2df1dd5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de4006890 con 0x7f2de40df410 2024-04-25T09:53:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f610759e640 1 Processor -- start 2024-04-25T09:53:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7f2debfff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f2de40392c0 0x7f2de4053790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:37486/0 (socket says 172.21.15.139:37486) 2024-04-25T09:53:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7f2debfff640 1 -- 172.21.15.139:0/453864863 learned_addr learned my addr 172.21.15.139:0/453864863 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7fb44923c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb43c0df210 0x7fb43c038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7f2deaffd640 1 --2- 172.21.15.139:0/453864863 >> v2:172.21.15.150:3300/0 conn(0x7f2de40df410 0x7f2de4053250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7f7b1ad9d640 1 -- 172.21.15.139:0/3686538158 reap_dead start 2024-04-25T09:53:35.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7f6cad5a6640 1 -- 172.21.15.139:0/455022911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca00065e0 msgr2=0x7f6ca0006a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:53:35.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7f6cad5a6640 1 --2- 172.21.15.139:0/455022911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca00065e0 0x7f6ca0006a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7f6cad5a6640 1 -- 172.21.15.139:0/455022911 >> v1:172.21.15.139:6789/0 conn(0x7f6ca0006200 legacy=0x7f6ca0056fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.191+0000 7f3b28ab3640 1 -- start start 2024-04-25T09:53:35.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7fb44923c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb43c050190 0x7fb43c006a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7fb443fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb43c0df210 0x7fb43c038070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7f2dea7fc640 1 -- 172.21.15.139:0/453864863 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616921675 0 0) 0x7f2de4006dc0 con 0x7f2de40392c0 2024-04-25T09:53:35.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7fb448a3b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb43c050190 0x7fb43c006a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7fb443fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb43c0df210 0x7fb43c038070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57376/0 (socket says 172.21.15.139:57376) 2024-04-25T09:53:35.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7fb443fff640 1 -- 172.21.15.139:0/1121984439 learned_addr learned my addr 172.21.15.139:0/1121984439 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7fb44923c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43c0544b0 con 0x7fb43c053c60 2024-04-25T09:53:35.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7fb4437fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb43c053c60 0x7fb43c04fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46576/0 (socket says 172.21.15.139:46576) 2024-04-25T09:53:35.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7fb44923c640 1 -- 172.21.15.139:0/1121984439 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43c0541d0 con 0x7fb43c050190 2024-04-25T09:53:35.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7f2dea7fc640 1 -- 172.21.15.139:0/453864863 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ddc003520 con 0x7f2de40392c0 2024-04-25T09:53:35.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7fb44923c640 1 -- 172.21.15.139:0/1121984439 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c005f20 con 0x7fb43c0df210 2024-04-25T09:53:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7ff3ed51c640 1 Processor -- start 2024-04-25T09:53:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f2deaffd640 1 -- 172.21.15.139:0/453864863 >> v1:172.21.15.150:6789/0 conn(0x7f2de40392c0 legacy=0x7f2de4053790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f2deaffd640 1 -- 172.21.15.139:0/453864863 >> v1:172.21.15.139:6789/0 conn(0x7f2de404fa80 legacy=0x7f2de404fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f2deaffd640 1 -- 172.21.15.139:0/453864863 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de40070a0 con 0x7f2de40df410 2024-04-25T09:53:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7fb448a3b640 1 -- 172.21.15.139:0/1121984439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb43c0df210 msgr2=0x7fb43c038070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7fb448a3b640 1 --2- 172.21.15.139:0/1121984439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb43c0df210 0x7fb43c038070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7fb448a3b640 1 -- 172.21.15.139:0/1121984439 >> v1:172.21.15.139:6789/0 conn(0x7fb43c053c60 legacy=0x7fb43c04fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7fb448a3b640 1 -- 172.21.15.139:0/1121984439 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c006200 con 0x7fb43c050190 2024-04-25T09:53:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.192+0000 7f610759e640 1 -- start start 2024-04-25T09:53:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f610759e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61080b2810 0x7f61080d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7ff3ed51c640 1 -- start start 2024-04-25T09:53:35.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f6cad5a6640 1 -- 172.21.15.139:0/455022911 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca0058030 con 0x7f6ca0006f80 2024-04-25T09:53:35.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f610759e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f61080d0590 0x7f61080d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f3b28ab3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3b2415dbf0 0x7f3b241617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f610659c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61080b2810 0x7f61080d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f610659c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61080b2810 0x7f61080d2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57396/0 (socket says 172.21.15.139:57396) 2024-04-25T09:53:35.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f6105d9b640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6108166570 0x7f61080cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:37498/0 (socket says 172.21.15.139:37498) 2024-04-25T09:53:35.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f3b28ab3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b24161dc0 con 0x7f3b240cee50 2024-04-25T09:53:35.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f610659c640 1 -- 172.21.15.139:0/3002657188 learned_addr learned my addr 172.21.15.139:0/3002657188 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f3b28ab3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b241620a0 con 0x7f3b240b9370 2024-04-25T09:53:35.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610759e640 1 -- 172.21.15.139:0/3002657188 --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61080ce500 con 0x7f6108166570 2024-04-25T09:53:35.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610759e640 1 -- 172.21.15.139:0/3002657188 --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61080d0f20 con 0x7f61080d0590 2024-04-25T09:53:35.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.193+0000 7f3b28ab3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b240d1ee0 con 0x7f3b2415dbf0 2024-04-25T09:53:35.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f3b2259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3b2415dbf0 0x7f3b241617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f3b2259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3b2415dbf0 0x7f3b241617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58980/0 (socket says 172.21.15.139:58980) 2024-04-25T09:53:35.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f3b2259c640 1 -- 172.21.15.139:0/4289678362 learned_addr learned my addr 172.21.15.139:0/4289678362 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f7b1a59c640 1 --2- 172.21.15.139:0/3686538158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0400c350 0x7f7b0400c730 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000ba10 tx=0x7f7b0400e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1104e2aa6b69a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610759e640 1 -- 172.21.15.139:0/3002657188 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61080d30c0 con 0x7f61080b2810 2024-04-25T09:53:35.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f2deaffd640 1 --2- 172.21.15.139:0/453864863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de40df410 0x7f2de4053250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009720 tx=0x7f2dd802c1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a9c007ea580066 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f3b2259c640 1 -- 172.21.15.139:0/4289678362 >> v1:172.21.15.150:6789/0 conn(0x7f3b240b9370 legacy=0x7f3b240ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10030070 con 0x7f7b0400c350 2024-04-25T09:53:35.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1002add0 con 0x7f7b0400c350 2024-04-25T09:53:35.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f7b1959a640 1 -- 172.21.15.139:0/3686538158 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1000b5e0 con 0x7f7b0400c350 2024-04-25T09:53:35.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f6106d9d640 1 --2- 172.21.15.139:0/3002657188 >> v2:172.21.15.150:3300/0 conn(0x7f61080d0590 0x7f61080d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f3b2259c640 1 -- 172.21.15.139:0/4289678362 >> v1:172.21.15.139:6789/0 conn(0x7f3b240cee50 legacy=0x7f3b240d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f3b2259c640 1 -- 172.21.15.139:0/4289678362 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b24162380 con 0x7f3b2415dbf0 2024-04-25T09:53:35.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7fb448a3b640 1 --2- 172.21.15.139:0/1121984439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c050190 0x7fb43c006a00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb444000b60 tx=0x7fb4440a4fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53b78f11217b3e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286135470 0 0) 0x7f61080ce500 con 0x7f6108166570 2024-04-25T09:53:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60f0003520 con 0x7f6108166570 2024-04-25T09:53:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 <== mon.1 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 271155233 0 0) 0x7f60f0003520 con 0x7f6108166570 2024-04-25T09:53:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61080b2810 msgr2=0x7f61080d2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 --2- 172.21.15.139:0/3002657188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61080b2810 0x7f61080d2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 >> v2:172.21.15.150:3300/0 conn(0x7f61080d0590 msgr2=0x7f61080d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 --2- 172.21.15.139:0/3002657188 >> v2:172.21.15.150:3300/0 conn(0x7f61080d0590 0x7f61080d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61080d33a0 con 0x7f6108166570 2024-04-25T09:53:35.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 <== mon.1 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f60fc001670 con 0x7f6108166570 2024-04-25T09:53:35.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 >> v1:172.21.15.150:6789/0 conn(0x7f6108166570 legacy=0x7f61080cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7fb442ffd640 1 -- 172.21.15.139:0/1121984439 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4440b2070 con 0x7fb43c050190 2024-04-25T09:53:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7ff3ed51c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff3e8161d30 0x7ff3e8166590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7ff3ed51c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e806ecb0 0x7ff3e80b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7ff3ed51c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff3e80cc3a0 0x7ff3e80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7ff3ed51c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e80ccd50 con 0x7ff3e80cc3a0 2024-04-25T09:53:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7ff3ed51c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e806d930 con 0x7ff3e8161d30 2024-04-25T09:53:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7ff3ed51c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80ce3c0 con 0x7ff3e806ecb0 2024-04-25T09:53:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7fb442ffd640 1 -- 172.21.15.139:0/1121984439 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4440a5df0 con 0x7fb43c050190 2024-04-25T09:53:35.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f2dea7fc640 1 -- 172.21.15.139:0/453864863 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8039070 con 0x7f2de40df410 2024-04-25T09:53:35.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7fb442ffd640 1 -- 172.21.15.139:0/1121984439 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4440b2070 con 0x7fb43c050190 2024-04-25T09:53:35.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 --2- 172.21.15.139:0/3002657188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f00039b0 0x7f60f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f2dea7fc640 1 -- 172.21.15.139:0/453864863 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd802cc40 con 0x7f2de40df410 2024-04-25T09:53:35.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7ff3e77fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e806ecb0 0x7ff3e80b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7ff3e77fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e806ecb0 0x7ff3e80b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57406/0 (socket says 172.21.15.139:57406) 2024-04-25T09:53:35.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7ff3e77fe640 1 -- 172.21.15.139:0/1701867556 learned_addr learned my addr 172.21.15.139:0/1701867556 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f6cad5a6640 1 --2- 172.21.15.139:0/455022911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca0006f80 0x7f6ca0007410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6ca802d370 tx=0x7f6ca80a5450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0c928ffea57178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7ff3e7fff640 1 --2- 172.21.15.139:0/1701867556 >> v2:172.21.15.150:3300/0 conn(0x7ff3e8161d30 0x7ff3e8166590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7ff3ecd1b640 1 --2- 172.21.15.139:0/1701867556 >> v2:172.21.15.139:3300/0 conn(0x7ff3e80cc3a0 0x7ff3e80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 --2- 172.21.15.139:0/3002657188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60f0004800 0x7f60f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.194+0000 7f610559a640 1 --2- 172.21.15.139:0/3002657188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60f00051c0 0x7f60f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61080d0f20 con 0x7f60f00051c0 2024-04-25T09:53:35.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61080d30c0 con 0x7f60f00039b0 2024-04-25T09:53:35.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f000bf00 con 0x7f60f0004800 2024-04-25T09:53:35.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f6106d9d640 1 --2- 172.21.15.139:0/3002657188 >> v2:172.21.15.150:3300/0 conn(0x7f61080d0590 0x7f61080d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:35.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f6106d9d640 1 --2- 172.21.15.139:0/3002657188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60f00051c0 0x7f60f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f610659c640 1 --2- 172.21.15.139:0/3002657188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61080b2810 0x7f61080d2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:35.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f6ca77fe640 1 -- 172.21.15.139:0/455022911 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80b1070 con 0x7f6ca0006f80 2024-04-25T09:53:35.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f610659c640 1 --2- 172.21.15.139:0/3002657188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f00039b0 0x7f60f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f6ca77fe640 1 -- 172.21.15.139:0/455022911 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca80ac640 con 0x7f6ca0006f80 2024-04-25T09:53:35.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f3b2259c640 1 --2- 172.21.15.139:0/4289678362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b2415dbf0 0x7f3b241617e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009870 tx=0x7f3b1802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee627075480d213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f2dea7fc640 1 -- 172.21.15.139:0/453864863 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0627f0 con 0x7f2de40df410 2024-04-25T09:53:35.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.195+0000 7f6ca77fe640 1 -- 172.21.15.139:0/455022911 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80acab0 con 0x7f6ca0006f80 2024-04-25T09:53:35.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f3b2159a640 1 -- 172.21.15.139:0/4289678362 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18037070 con 0x7f3b2415dbf0 2024-04-25T09:53:35.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f2df1dd5640 1 -- 172.21.15.139:0/453864863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de40df410 msgr2=0x7f2de4053250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f2df1dd5640 1 --2- 172.21.15.139:0/453864863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de40df410 0x7f2de4053250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009720 tx=0x7f2dd802c1b0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7fb44923c640 1 -- 172.21.15.139:0/1121984439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c050190 msgr2=0x7fb43c006a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f6106d9d640 1 -- 172.21.15.139:0/3002657188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60f0004800 msgr2=0x7f60f0004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:53:35.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f6106d9d640 1 --2- 172.21.15.139:0/3002657188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60f0004800 0x7f60f0004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f6106d9d640 1 -- 172.21.15.139:0/3002657188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f00039b0 msgr2=0x7f60f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f6106d9d640 1 --2- 172.21.15.139:0/3002657188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f00039b0 0x7f60f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f6106d9d640 1 -- 172.21.15.139:0/3002657188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61080d33a0 con 0x7f60f00051c0 2024-04-25T09:53:35.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7fb44923c640 1 --2- 172.21.15.139:0/1121984439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c050190 0x7fb43c006a00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb444000b60 tx=0x7fb4440a4fa0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f3b2159a640 1 -- 172.21.15.139:0/4289678362 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b18032440 con 0x7f3b2415dbf0 2024-04-25T09:53:35.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f2df1dd5640 1 -- 172.21.15.139:0/453864863 shutdown_connections 2024-04-25T09:53:35.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f2df1dd5640 1 --2- 172.21.15.139:0/453864863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de40df410 0x7f2de4053250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7ff3e77fe640 1 -- 172.21.15.139:0/1701867556 >> v2:172.21.15.150:3300/0 conn(0x7ff3e8161d30 msgr2=0x7ff3e8166590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7ff3e77fe640 1 --2- 172.21.15.139:0/1701867556 >> v2:172.21.15.150:3300/0 conn(0x7ff3e8161d30 0x7ff3e8166590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7ff3e77fe640 1 -- 172.21.15.139:0/1701867556 >> v2:172.21.15.139:3300/0 conn(0x7ff3e80cc3a0 msgr2=0x7ff3e80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:35.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7ff3e77fe640 1 --2- 172.21.15.139:0/1701867556 >> v2:172.21.15.139:3300/0 conn(0x7ff3e80cc3a0 0x7ff3e80cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7ff3e77fe640 1 -- 172.21.15.139:0/1701867556 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e80ce6a0 con 0x7ff3e806ecb0 2024-04-25T09:53:35.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f2df1dd5640 1 -- 172.21.15.139:0/453864863 >> 172.21.15.139:0/453864863 conn(0x7f2de4009fa0 msgr2=0x7f2de400a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7fb44923c640 1 -- 172.21.15.139:0/1121984439 shutdown_connections 2024-04-25T09:53:35.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7fb44923c640 1 --2- 172.21.15.139:0/1121984439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c050190 0x7fb43c006a00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7fb44923c640 1 --2- 172.21.15.139:0/1121984439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb43c0df210 0x7fb43c038070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7fb44923c640 1 -- 172.21.15.139:0/1121984439 >> 172.21.15.139:0/1121984439 conn(0x7fb43c009fa0 msgr2=0x7fb43c00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7fb44923c640 1 -- 172.21.15.139:0/1121984439 shutdown_connections 2024-04-25T09:53:35.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f7b20d9e640 1 -- 172.21.15.139:0/3686538158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0400c350 msgr2=0x7f7b0400c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f7b20d9e640 1 --2- 172.21.15.139:0/3686538158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0400c350 0x7f7b0400c730 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000ba10 tx=0x7f7b0400e040 comp rx=0 tx=0).stop 2024-04-25T09:53:35.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f7b20d9e640 1 -- 172.21.15.139:0/3686538158 shutdown_connections 2024-04-25T09:53:35.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f7b20d9e640 1 --2- 172.21.15.139:0/3686538158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b0400c350 0x7f7b0400c730 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000ba10 tx=0x7f7b0400e040 comp rx=0 tx=0).stop 2024-04-25T09:53:35.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f7b20d9e640 1 -- 172.21.15.139:0/3686538158 >> 172.21.15.139:0/3686538158 conn(0x7f7b1c06d0e0 msgr2=0x7f7b1c06d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f6106d9d640 1 --2- 172.21.15.139:0/3002657188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60f00051c0 0x7f60f000b9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f610000e380 tx=0x7f60f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cdebd4394b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6100009dc0 con 0x7f60f00051c0 2024-04-25T09:53:35.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6100003b90 con 0x7f60f00051c0 2024-04-25T09:53:35.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f610559a640 1 -- 172.21.15.139:0/3002657188 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6100004020 con 0x7f60f00051c0 2024-04-25T09:53:35.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f6cadda7640 1 -- 172.21.15.139:0/455022911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca0006f80 msgr2=0x7f6ca0007410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f6cadda7640 1 --2- 172.21.15.139:0/455022911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca0006f80 0x7f6ca0007410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6ca802d370 tx=0x7f6ca80a5450 comp rx=0 tx=0).stop 2024-04-25T09:53:35.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f6cadda7640 1 -- 172.21.15.139:0/455022911 shutdown_connections 2024-04-25T09:53:35.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f6cadda7640 1 --2- 172.21.15.139:0/455022911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca0006f80 0x7f6ca0007410 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f6cadda7640 1 --2- 172.21.15.139:0/455022911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca00065e0 0x7f6ca0006a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f6cadda7640 1 -- 172.21.15.139:0/455022911 >> 172.21.15.139:0/455022911 conn(0x7f6ca0058ee0 msgr2=0x7f6ca00592f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f6cadda7640 1 -- 172.21.15.139:0/455022911 shutdown_connections 2024-04-25T09:53:35.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f3b2159a640 1 -- 172.21.15.139:0/4289678362 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18032810 con 0x7f3b2415dbf0 2024-04-25T09:53:35.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f3b28ab3640 1 -- 172.21.15.139:0/4289678362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b2415dbf0 msgr2=0x7f3b241617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f3b28ab3640 1 --2- 172.21.15.139:0/4289678362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b2415dbf0 0x7f3b241617e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009870 tx=0x7f3b1802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f3b28ab3640 1 -- 172.21.15.139:0/4289678362 shutdown_connections 2024-04-25T09:53:35.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f3b28ab3640 1 --2- 172.21.15.139:0/4289678362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b2415dbf0 0x7f3b241617e0 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009870 tx=0x7f3b1802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f3b28ab3640 1 -- 172.21.15.139:0/4289678362 >> 172.21.15.139:0/4289678362 conn(0x7f3b240b5480 msgr2=0x7f3b240b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7fb44923c640 1 -- 172.21.15.139:0/1121984439 wait complete. 2024-04-25T09:53:35.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f2df1dd5640 1 -- 172.21.15.139:0/453864863 shutdown_connections 2024-04-25T09:53:35.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.196+0000 7f2df1dd5640 1 -- 172.21.15.139:0/453864863 wait complete. 2024-04-25T09:53:35.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f3b28ab3640 1 -- 172.21.15.139:0/4289678362 shutdown_connections 2024-04-25T09:53:35.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7ff3e77fe640 1 --2- 172.21.15.139:0/1701867556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e806ecb0 0x7ff3e80b3080 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80cd060 tx=0x7ff3d802a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e965bd4887489200 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.198+0000 7ff3e6ffd640 1 -- 172.21.15.139:0/1701867556 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8037070 con 0x7ff3e806ecb0 2024-04-25T09:53:35.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f7b20d9e640 1 -- 172.21.15.139:0/3686538158 shutdown_connections 2024-04-25T09:53:35.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f7b20d9e640 1 -- 172.21.15.139:0/3686538158 wait complete. 2024-04-25T09:53:35.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f6cadda7640 1 -- 172.21.15.139:0/455022911 wait complete. 2024-04-25T09:53:35.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.198+0000 7f2df1dd5640 1 Processor -- start 2024-04-25T09:53:35.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.198+0000 7f6cadda7640 1 Processor -- start 2024-04-25T09:53:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.198+0000 7f610759e640 1 -- 172.21.15.139:0/3002657188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60f00051c0 msgr2=0x7f60f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.198+0000 7f610759e640 1 --2- 172.21.15.139:0/3002657188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60f00051c0 0x7f60f000b9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f610000e380 tx=0x7f60f0005830 comp rx=0 tx=0).stop 2024-04-25T09:53:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.198+0000 7f6106d9d640 1 -- 172.21.15.139:0/3002657188 reap_dead start 2024-04-25T09:53:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.198+0000 7f610759e640 1 -- 172.21.15.139:0/3002657188 shutdown_connections 2024-04-25T09:53:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.198+0000 7f610759e640 1 -- 172.21.15.139:0/3002657188 >> 172.21.15.139:0/3002657188 conn(0x7f610806bf60 msgr2=0x7f610806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.198+0000 7f6cadda7640 1 -- start start 2024-04-25T09:53:35.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.198+0000 7fb44923c640 1 Processor -- start 2024-04-25T09:53:35.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.199+0000 7ff3e6ffd640 1 -- 172.21.15.139:0/1701867556 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d802ae00 con 0x7ff3e806ecb0 2024-04-25T09:53:35.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.199+0000 7f2df1dd5640 1 -- start start 2024-04-25T09:53:35.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.199+0000 7fb44923c640 1 -- start start 2024-04-25T09:53:35.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.199+0000 7f7b20d9e640 1 Processor -- start 2024-04-25T09:53:35.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.199+0000 7f7b20d9e640 1 -- start start 2024-04-25T09:53:35.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.197+0000 7f3b28ab3640 1 -- 172.21.15.139:0/4289678362 wait complete. 2024-04-25T09:53:35.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.199+0000 7ff3e6ffd640 1 -- 172.21.15.139:0/1701867556 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8032660 con 0x7ff3e806ecb0 2024-04-25T09:53:35.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.199+0000 7f3b28ab3640 1 Processor -- start 2024-04-25T09:53:35.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.199+0000 7f3b28ab3640 1 -- start start 2024-04-25T09:53:35.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7ff3ed51c640 1 -- 172.21.15.139:0/1701867556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e806ecb0 msgr2=0x7ff3e80b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7ff3ed51c640 1 --2- 172.21.15.139:0/1701867556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e806ecb0 0x7ff3e80b3080 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80cd060 tx=0x7ff3d802a0b0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7ff3ed51c640 1 -- 172.21.15.139:0/1701867556 shutdown_connections 2024-04-25T09:53:35.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7ff3ed51c640 1 --2- 172.21.15.139:0/1701867556 >> v2:172.21.15.139:3300/0 conn(0x7ff3e80cc3a0 0x7ff3e80cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7ff3ed51c640 1 --2- 172.21.15.139:0/1701867556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e806ecb0 0x7ff3e80b3080 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7ff3ed51c640 1 --2- 172.21.15.139:0/1701867556 >> v2:172.21.15.150:3300/0 conn(0x7ff3e8161d30 0x7ff3e8166590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7ff3ed51c640 1 -- 172.21.15.139:0/1701867556 >> 172.21.15.139:0/1701867556 conn(0x7ff3e8067030 msgr2=0x7ff3e80b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7ff3ed51c640 1 -- 172.21.15.139:0/1701867556 shutdown_connections 2024-04-25T09:53:35.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7ff3ed51c640 1 -- 172.21.15.139:0/1701867556 wait complete. 2024-04-25T09:53:35.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f6cadda7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca0006200 0x7f6ca004e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f2df1dd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40392c0 0x7f2de4178bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f7b20d9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b1c168f00 0x7f7b1c0ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f7b20d9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b1c204210 0x7f7b1c0a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f6cadda7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca00065e0 0x7f6ca004e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f2df1dd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de404fa80 0x7f2de40e0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f6cadda7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca0006f80 0x7f6ca00d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7fb44923c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c050190 0x7fb43c02d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f7b20d9e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b1c0ad840 0x7f7b1c0adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f7b20d9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c0b09c0 con 0x7f7b1c168f00 2024-04-25T09:53:35.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f7b20d9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c0b06e0 con 0x7f7b1c204210 2024-04-25T09:53:35.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f7b20d9e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c0b0ee0 con 0x7f7b1c0ad840 2024-04-25T09:53:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f610759e640 1 -- 172.21.15.139:0/3002657188 shutdown_connections 2024-04-25T09:53:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f7b1a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b1c168f00 0x7f7b1c0ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7fb44923c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb43c053c60 0x7fb43c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2df1dd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2de40df410 0x7f2de40e07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7fb443fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c050190 0x7fb43c02d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7fb443fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c050190 0x7fb43c02d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59028/0 (socket says 172.21.15.139:59028) 2024-04-25T09:53:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7fb443fff640 1 -- 172.21.15.139:0/36932350 learned_addr learned my addr 172.21.15.139:0/36932350 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f7b19d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b1c204210 0x7f7b1c0a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f7b19d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b1c204210 0x7f7b1c0a8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:46170/0 (socket says 172.21.15.139:46170) 2024-04-25T09:53:35.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f7b19d9b640 1 -- 172.21.15.139:0/844086375 learned_addr learned my addr 172.21.15.139:0/844086375 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7fb4437fe640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb43c053c60 0x7fb43c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2deaffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de404fa80 0x7f2de40e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2deaffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de404fa80 0x7f2de40e0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:46172/0 (socket says 172.21.15.139:46172) 2024-04-25T09:53:35.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2deaffd640 1 -- 172.21.15.139:0/3780046533 learned_addr learned my addr 172.21.15.139:0/3780046533 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7fb44923c640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb43c0df210 0x7fb43c0cff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f7b1ad9d640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b1c0ad840 0x7f7b1c0adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4052cb0 con 0x7f2de40df410 2024-04-25T09:53:35.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4052b30 con 0x7f2de404fa80 2024-04-25T09:53:35.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f3b28ab3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b240b9370 0x7f3b2415b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de400aa90 con 0x7f2de40392c0 2024-04-25T09:53:35.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c037a80 con 0x7fb43c050190 2024-04-25T09:53:35.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c037900 con 0x7fb43c0df210 2024-04-25T09:53:35.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0df5f0 con 0x7fb43c053c60 2024-04-25T09:53:35.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f3b28ab3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b240cee50 0x7f3b241f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7fb448a3b640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb43c0df210 0x7fb43c0cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.200+0000 7f610759e640 1 -- 172.21.15.139:0/3002657188 wait complete. 2024-04-25T09:53:35.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f3b2259c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b240b9370 0x7f3b2415b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f3b2259c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b240b9370 0x7f3b2415b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57448/0 (socket says 172.21.15.139:57448) 2024-04-25T09:53:35.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f3b2259c640 1 -- 172.21.15.139:0/3216914499 learned_addr learned my addr 172.21.15.139:0/3216914499 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2deb7fe640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40392c0 0x7f2de4178bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f3b28ab3640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b241f3eb0 0x7f3b24158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2415dfd0 con 0x7f3b240cee50 2024-04-25T09:53:35.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b240b97d0 con 0x7f3b241f3eb0 2024-04-25T09:53:35.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b241f43c0 con 0x7f3b240b9370 2024-04-25T09:53:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f7b1ad9d640 1 -- 172.21.15.139:0/844086375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b1c204210 msgr2=0x7f7b1c0a8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f7b1ad9d640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b1c204210 0x7f7b1c0a8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f7b1ad9d640 1 -- 172.21.15.139:0/844086375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b1c168f00 msgr2=0x7f7b1c0ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f7b1ad9d640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b1c168f00 0x7f7b1c0ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f7b1ad9d640 1 -- 172.21.15.139:0/844086375 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0ab2c0 con 0x7f7b1c0ad840 2024-04-25T09:53:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7ff3ed51c640 1 Processor -- start 2024-04-25T09:53:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f3b22d9d640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b241f3eb0 0x7f3b24158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2deaffd640 1 -- 172.21.15.139:0/3780046533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40392c0 msgr2=0x7f2de4178bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2deaffd640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40392c0 0x7f2de4178bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2deaffd640 1 -- 172.21.15.139:0/3780046533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2de40df410 msgr2=0x7f2de40e07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2deaffd640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2de40df410 0x7f2de40e07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f2deaffd640 1 -- 172.21.15.139:0/3780046533 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de404e890 con 0x7f2de404fa80 2024-04-25T09:53:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f6cadda7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca00e3520 con 0x7f6ca0006f80 2024-04-25T09:53:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f6cadda7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca00585f0 con 0x7f6ca0006200 2024-04-25T09:53:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f6cadda7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca003edd0 con 0x7f6ca00065e0 2024-04-25T09:53:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.201+0000 7f6cad5a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca0006f80 0x7f6ca00d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f6ca7fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca00065e0 0x7f6ca004e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f6cacda5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca0006200 0x7f6ca004e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f6ca7fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca00065e0 0x7f6ca004e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57420/0 (socket says 172.21.15.139:57420) 2024-04-25T09:53:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f6ca7fff640 1 -- 172.21.15.139:0/2549447221 learned_addr learned my addr 172.21.15.139:0/2549447221 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f3b21d9b640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b240cee50 0x7f3b241f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7fb4437fe640 1 -- 172.21.15.139:0/36932350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb43c0df210 msgr2=0x7fb43c0cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7fb4437fe640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb43c0df210 0x7fb43c0cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7fb4437fe640 1 -- 172.21.15.139:0/36932350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c050190 msgr2=0x7fb43c02d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7fb4437fe640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c050190 0x7fb43c02d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7fb4437fe640 1 -- 172.21.15.139:0/36932350 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c02b500 con 0x7fb43c053c60 2024-04-25T09:53:35.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7ff3ed51c640 1 -- start start 2024-04-25T09:53:35.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f2deaffd640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de404fa80 0x7f2de40e0270 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8000c00 tx=0x7f2dd80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f6ca7fff640 1 -- 172.21.15.139:0/2549447221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca0006200 msgr2=0x7f6ca004e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f6ca7fff640 1 --2- 172.21.15.139:0/2549447221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca0006200 0x7f6ca004e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f6ca7fff640 1 -- 172.21.15.139:0/2549447221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca0006f80 msgr2=0x7f6ca00d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f6ca7fff640 1 --2- 172.21.15.139:0/2549447221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca0006f80 0x7f6ca00d4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7fb4437fe640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb43c053c60 0x7fb43c0cf9d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb43000e700 tx=0x7fb43000ebc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f6ca7fff640 1 -- 172.21.15.139:0/2549447221 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca00d9d00 con 0x7f6ca00065e0 2024-04-25T09:53:35.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f7b1ad9d640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b1c0ad840 0x7f7b1c0adcc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c16a9a0 tx=0x7f7b14005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f2dd7fff640 1 -- 172.21.15.139:0/3780046533 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8039070 con 0x7f2de404fa80 2024-04-25T09:53:35.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f7b0affd640 1 -- 172.21.15.139:0/844086375 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b14008800 con 0x7f7b1c0ad840 2024-04-25T09:53:35.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7fb440ff9640 1 -- 172.21.15.139:0/36932350 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43001a070 con 0x7fb43c053c60 2024-04-25T09:53:35.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f2dd7fff640 1 -- 172.21.15.139:0/3780046533 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd803a040 con 0x7f2de404fa80 2024-04-25T09:53:35.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f610759e640 1 Processor -- start 2024-04-25T09:53:35.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f7b20d9e640 1 -- 172.21.15.139:0/844086375 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1c0ab5a0 con 0x7f7b1c0ad840 2024-04-25T09:53:35.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f7b0affd640 1 -- 172.21.15.139:0/844086375 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b140089a0 con 0x7f7b1c0ad840 2024-04-25T09:53:35.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f7b0affd640 1 -- 172.21.15.139:0/844086375 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b140143f0 con 0x7f7b1c0ad840 2024-04-25T09:53:35.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f3b2259c640 1 -- 172.21.15.139:0/3216914499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b241f3eb0 msgr2=0x7f3b24158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f3b2259c640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b241f3eb0 0x7f3b24158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f3b2259c640 1 -- 172.21.15.139:0/3216914499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b240cee50 msgr2=0x7f3b241f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f3b2259c640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b240cee50 0x7f3b241f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.202+0000 7f3b2259c640 1 -- 172.21.15.139:0/3216914499 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b241f46a0 con 0x7f3b240b9370 2024-04-25T09:53:35.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7fb440ff9640 1 -- 172.21.15.139:0/36932350 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb430006bf0 con 0x7fb43c053c60 2024-04-25T09:53:35.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7fb440ff9640 1 -- 172.21.15.139:0/36932350 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43000c640 con 0x7fb43c053c60 2024-04-25T09:53:35.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7f7b20d9e640 1 -- 172.21.15.139:0/844086375 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1c0ae200 con 0x7f7b1c0ad840 2024-04-25T09:53:35.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c02b7a0 con 0x7fb43c053c60 2024-04-25T09:53:35.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43c02bd60 con 0x7fb43c053c60 2024-04-25T09:53:35.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7f6ca7fff640 1 --2- 172.21.15.139:0/2549447221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca00065e0 0x7f6ca004e600 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00b450 tx=0x7f6c9c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7f3b2259c640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b240b9370 0x7f3b2415b5e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009870 tx=0x7f3b180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7f3b12ffd640 1 -- 172.21.15.139:0/3216914499 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18041070 con 0x7f3b240b9370 2024-04-25T09:53:35.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7ff3ed51c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3e80cc3a0 0x7ff3e81f8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7ff3ed51c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3e8161d30 0x7ff3e8159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7ff3ed51c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e8162fa0 0x7ff3e80c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7f2dd7fff640 1 -- 172.21.15.139:0/3780046533 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8004450 con 0x7f2de404fa80 2024-04-25T09:53:35.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7ff3ed51c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e8162110 con 0x7ff3e80cc3a0 2024-04-25T09:53:35.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7ff3ed51c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e806f110 con 0x7ff3e8161d30 2024-04-25T09:53:35.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de4185ee0 con 0x7f2de404fa80 2024-04-25T09:53:35.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de41864a0 con 0x7f2de404fa80 2024-04-25T09:53:35.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7f610759e640 1 -- start start 2024-04-25T09:53:35.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.204+0000 7f610759e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f00051c0 0x7f61081f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.204+0000 7f610759e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f610815d6c0 0x7f61080c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.204+0000 7f610759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6108163650 0x7f6108153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.204+0000 7f6105d9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f610815d6c0 0x7f61080c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.204+0000 7f6105d9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f610815d6c0 0x7f61080c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57472/0 (socket says 172.21.15.139:57472) 2024-04-25T09:53:35.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.204+0000 7f6105d9b640 1 -- 172.21.15.139:0/2056325491 learned_addr learned my addr 172.21.15.139:0/2056325491 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.204+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61080b2bf0 con 0x7f6108163650 2024-04-25T09:53:35.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.204+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61080b7cc0 con 0x7f60f00051c0 2024-04-25T09:53:35.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.204+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61080b7600 con 0x7f610815d6c0 2024-04-25T09:53:35.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b241f4980 con 0x7f3b240b9370 2024-04-25T09:53:35.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.203+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b24159dd0 con 0x7f3b240b9370 2024-04-25T09:53:35.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.204+0000 7f3b12ffd640 1 -- 172.21.15.139:0/3216914499 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b18032440 con 0x7f3b240b9370 2024-04-25T09:53:35.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f610659c640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f00051c0 0x7f61081f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f6106d9d640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6108163650 0x7f6108153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f6105d9b640 1 -- 172.21.15.139:0/2056325491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f00051c0 msgr2=0x7f61081f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f6105d9b640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f00051c0 0x7f61081f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f6105d9b640 1 -- 172.21.15.139:0/2056325491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6108163650 msgr2=0x7f6108153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f6105d9b640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6108163650 0x7f6108153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f6105d9b640 1 -- 172.21.15.139:0/2056325491 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61080ca510 con 0x7f610815d6c0 2024-04-25T09:53:35.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f610659c640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f00051c0 0x7f61081f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:35.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f6106d9d640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6108163650 0x7f6108153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:35.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7f6105d9b640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f610815d6c0 0x7f61080c9e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00c8f0 tx=0x7f60fc00cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f7b0affd640 1 -- 172.21.15.139:0/844086375 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b1401b020 con 0x7f7b1c0ad840 2024-04-25T09:53:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f6ca57fa640 1 -- 172.21.15.139:0/2549447221 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c015020 con 0x7f6ca00065e0 2024-04-25T09:53:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f6ca57fa640 1 -- 172.21.15.139:0/2549447221 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c0118b0 con 0x7f6ca00065e0 2024-04-25T09:53:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f6cadda7640 1 -- 172.21.15.139:0/2549447221 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca00d9fe0 con 0x7f6ca00065e0 2024-04-25T09:53:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f6cadda7640 1 -- 172.21.15.139:0/2549447221 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca0038fe0 con 0x7f6ca00065e0 2024-04-25T09:53:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f3b12ffd640 1 -- 172.21.15.139:0/3216914499 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18037040 con 0x7f3b240b9370 2024-04-25T09:53:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b08005290 con 0x7f3b240b9370 2024-04-25T09:53:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb424005290 con 0x7fb43c053c60 2024-04-25T09:53:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.205+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de40089c0 con 0x7f2de404fa80 2024-04-25T09:53:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7fb440ff9640 1 -- 172.21.15.139:0/36932350 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb43001d070 con 0x7fb43c053c60 2024-04-25T09:53:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7ff3e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3e80cc3a0 0x7ff3e81f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7ff3e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3e80cc3a0 0x7ff3e81f8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59054/0 (socket says 172.21.15.139:59054) 2024-04-25T09:53:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7ff3e7fff640 1 -- 172.21.15.139:0/3116045638 learned_addr learned my addr 172.21.15.139:0/3116045638 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7ff3ecd1b640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e8162fa0 0x7ff3e80c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7ff3ed51c640 1 -- 172.21.15.139:0/3116045638 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e8162290 con 0x7ff3e8162fa0 2024-04-25T09:53:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7f2dd7fff640 1 -- 172.21.15.139:0/3780046533 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2dd8003610 con 0x7f2de404fa80 2024-04-25T09:53:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7f3b12ffd640 1 -- 172.21.15.139:0/3216914499 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b18044070 con 0x7f3b240b9370 2024-04-25T09:53:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7f6ca57fa640 1 -- 172.21.15.139:0/2549447221 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c011ca0 con 0x7f6ca00065e0 2024-04-25T09:53:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7fb440ff9640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb42806b5a0 0x7fb42806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7ff3e77fe640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3e8161d30 0x7ff3e8159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7fb440ff9640 1 -- 172.21.15.139:0/36932350 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb43008b8f0 con 0x7fb43c053c60 2024-04-25T09:53:35.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7f7b0affd640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7afc06b5a0 0x7f7afc06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.206+0000 7f7b0affd640 1 -- 172.21.15.139:0/844086375 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7b1408a3f0 con 0x7f7b1c0ad840 2024-04-25T09:53:35.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7f7b20d9e640 1 -- 172.21.15.139:0/844086375 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b00005290 con 0x7f7b1c0ad840 2024-04-25T09:53:35.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7f7b1a59c640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7afc06b5a0 0x7f7afc06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7f2dd7fff640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2dd006b5a0 0x7f2dd006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7f3b12ffd640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b0406b4d0 0x7f3b0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7f2dd7fff640 1 -- 172.21.15.139:0/3780046533 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2dd80abb40 con 0x7f2de404fa80 2024-04-25T09:53:35.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7f6cadda7640 1 -- 172.21.15.139:0/2549447221 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c88005290 con 0x7f6ca00065e0 2024-04-25T09:53:35.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7f3b12ffd640 1 -- 172.21.15.139:0/3216914499 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3b180aaee0 con 0x7f3b240b9370 2024-04-25T09:53:35.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7f3b21d9b640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b0406b4d0 0x7f3b0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7f2deb7fe640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2dd006b5a0 0x7f2dd006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7fb443fff640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb42806b5a0 0x7fb42806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7f7b1a59c640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7afc06b5a0 0x7f7afc06da60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000b9e0 tx=0x7f7b10030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7ff3ecd1b640 1 -- 172.21.15.139:0/3116045638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3e8161d30 msgr2=0x7ff3e8159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7ff3ecd1b640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3e8161d30 0x7ff3e8159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7ff3ecd1b640 1 -- 172.21.15.139:0/3116045638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3e80cc3a0 msgr2=0x7ff3e81f8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7ff3ecd1b640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3e80cc3a0 0x7ff3e81f8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.207+0000 7ff3ecd1b640 1 -- 172.21.15.139:0/3116045638 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e815a700 con 0x7ff3e8162fa0 2024-04-25T09:53:35.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.208+0000 7ff3ecd1b640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e8162fa0 0x7ff3e80c4930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000eed0 tx=0x7ff3e000b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.208+0000 7f2deb7fe640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2dd006b5a0 0x7f2dd006da60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2de0001860 tx=0x7f2de0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.208+0000 7f6ca57fa640 1 -- 172.21.15.139:0/2549447221 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c9c018020 con 0x7f6ca00065e0 2024-04-25T09:53:35.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.208+0000 7fb443fff640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb42806b5a0 0x7fb42806da60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb4380094a0 tx=0x7fb438009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.208+0000 7f3b21d9b640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b0406b4d0 0x7f3b0406d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3b24203f00 tx=0x7f3b1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.208+0000 7f6ca57fa640 1 --2- 172.21.15.139:0/2549447221 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6c8c06b520 0x7f6c8c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.208+0000 7f6ca57fa640 1 -- 172.21.15.139:0/2549447221 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6c9c08a470 con 0x7f6ca00065e0 2024-04-25T09:53:35.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.208+0000 7f6cacda5640 1 --2- 172.21.15.139:0/2549447221 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6c8c06b520 0x7f6c8c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.209+0000 7f6cacda5640 1 --2- 172.21.15.139:0/2549447221 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6c8c06b520 0x7f6c8c06d9e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ca004df50 tx=0x7f6c94009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.215+0000 7f2dd7fff640 1 -- 172.21.15.139:0/3780046533 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2dd807a660 con 0x7f2de404fa80 2024-04-25T09:53:35.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.215+0000 7f60f6ffd640 1 -- 172.21.15.139:0/2056325491 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc018070 con 0x7f610815d6c0 2024-04-25T09:53:35.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.215+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61080b4020 con 0x7f610815d6c0 2024-04-25T09:53:35.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.215+0000 7fb440ff9640 1 -- 172.21.15.139:0/36932350 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb430059dc0 con 0x7fb43c053c60 2024-04-25T09:53:35.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.215+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61080b45e0 con 0x7f610815d6c0 2024-04-25T09:53:35.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.217+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6108074520 con 0x7f610815d6c0 2024-04-25T09:53:35.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.220+0000 7f6ca57fa640 1 -- 172.21.15.139:0/2549447221 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c9c01f020 con 0x7f6ca00065e0 2024-04-25T09:53:35.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.225+0000 7f7b0affd640 1 -- 172.21.15.139:0/844086375 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b14058f10 con 0x7f7b1c0ad840 2024-04-25T09:53:35.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.228+0000 7ff3e4ff9640 1 -- 172.21.15.139:0/3116045638 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0018020 con 0x7ff3e8162fa0 2024-04-25T09:53:35.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.228+0000 7f60f6ffd640 1 -- 172.21.15.139:0/2056325491 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60fc0074e0 con 0x7f610815d6c0 2024-04-25T09:53:35.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.228+0000 7ff3e4ff9640 1 -- 172.21.15.139:0/3116045638 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e0014400 con 0x7ff3e8162fa0 2024-04-25T09:53:35.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.228+0000 7f3b12ffd640 1 -- 172.21.15.139:0/3216914499 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b180793c0 con 0x7f3b240b9370 2024-04-25T09:53:35.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.228+0000 7ff3ed51c640 1 -- 172.21.15.139:0/3116045638 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e815a9e0 con 0x7ff3e8162fa0 2024-04-25T09:53:35.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.228+0000 7ff3ed51c640 1 -- 172.21.15.139:0/3116045638 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e815afa0 con 0x7ff3e8162fa0 2024-04-25T09:53:35.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.229+0000 7f60f6ffd640 1 -- 172.21.15.139:0/2056325491 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc022060 con 0x7f610815d6c0 2024-04-25T09:53:35.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.229+0000 7ff3e4ff9640 1 -- 172.21.15.139:0/3116045638 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0009e60 con 0x7ff3e8162fa0 2024-04-25T09:53:35.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.229+0000 7ff3ce7fc640 1 -- 172.21.15.139:0/3116045638 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e8074520 con 0x7ff3e8162fa0 2024-04-25T09:53:35.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.230+0000 7f60f6ffd640 1 -- 172.21.15.139:0/2056325491 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60fc01b070 con 0x7f610815d6c0 2024-04-25T09:53:35.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.231+0000 7f60f6ffd640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f60e806b4d0 0x7f60e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.231+0000 7f60f6ffd640 1 -- 172.21.15.139:0/2056325491 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f60fc08a210 con 0x7f610815d6c0 2024-04-25T09:53:35.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.231+0000 7f610659c640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f60e806b4d0 0x7f60e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.232+0000 7f610659c640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f60e806b4d0 0x7f60e806d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f61080d1e90 tx=0x7f60f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.232+0000 7f60f6ffd640 1 -- 172.21.15.139:0/2056325491 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60fc058650 con 0x7f610815d6c0 2024-04-25T09:53:35.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.234+0000 7ff3e4ff9640 1 -- 172.21.15.139:0/3116045638 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3e0005d10 con 0x7ff3e8162fa0 2024-04-25T09:53:35.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.234+0000 7ff3e4ff9640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3d406b5a0 0x7ff3d406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:35.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.235+0000 7ff3e4ff9640 1 -- 172.21.15.139:0/3116045638 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff3e008b320 con 0x7ff3e8162fa0 2024-04-25T09:53:35.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.235+0000 7ff3e7fff640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3d406b5a0 0x7ff3d406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:35.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.236+0000 7ff3e7fff640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3d406b5a0 0x7ff3d406da60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009c00 tx=0x7ff3dc009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:35.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.237+0000 7ff3e4ff9640 1 -- 172.21.15.139:0/3116045638 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3e0022030 con 0x7ff3e8162fa0 2024-04-25T09:53:35.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.532+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3781464649 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fd4bc005390 con 0x7fd4bc0e7530 2024-04-25T09:53:35.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.532+0000 7fd4c17fa640 1 -- 172.21.15.139:0/3781464649 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd4b8058c00 con 0x7fd4bc0e7530 2024-04-25T09:53:35.535 INFO:teuthology.orchestra.run.smithi139.stdout:55834574856 2024-04-25T09:53:35.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.535+0000 7fd4c98d2640 1 -- 172.21.15.139:0/3781464649 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd4b406b610 msgr2=0x7fd4b406dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.535+0000 7fd4c98d2640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd4b406b610 0x7fd4b406dad0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00d540 tx=0x7fd4c40a9000 comp rx=0 tx=0).stop 2024-04-25T09:53:35.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.536+0000 7fd4c98d2640 1 -- 172.21.15.139:0/3781464649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc0e7530 msgr2=0x7fd4bc03a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.536+0000 7fd4c98d2640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc0e7530 0x7fd4bc03a700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800e840 tx=0x7fd4b800ed00 comp rx=0 tx=0).stop 2024-04-25T09:53:35.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.536+0000 7fd4c98d2640 1 -- 172.21.15.139:0/3781464649 shutdown_connections 2024-04-25T09:53:35.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.536+0000 7fd4c98d2640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd4b406b610 0x7fd4b406dad0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.536+0000 7fd4c98d2640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4bc0e7530 0x7fd4bc03a700 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.536+0000 7fd4c98d2640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4bc056060 0x7fd4bc038930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.536+0000 7fd4c98d2640 1 --2- 172.21.15.139:0/3781464649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4bc00dc60 0x7fd4bc00a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.536+0000 7fd4c98d2640 1 -- 172.21.15.139:0/3781464649 >> 172.21.15.139:0/3781464649 conn(0x7fd4bc056ce0 msgr2=0x7fd4bc0501f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.537+0000 7fd4c98d2640 1 -- 172.21.15.139:0/3781464649 shutdown_connections 2024-04-25T09:53:35.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.537+0000 7fd4c98d2640 1 -- 172.21.15.139:0/3781464649 wait complete. 2024-04-25T09:53:35.557 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.7 2024-04-25T09:53:35.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.583+0000 7f6cadda7640 1 -- 172.21.15.139:0/2549447221 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6c88005580 con 0x7f6ca00065e0 2024-04-25T09:53:35.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.583+0000 7f6ca57fa640 1 -- 172.21.15.139:0/2549447221 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c9c00d040 con 0x7f6ca00065e0 2024-04-25T09:53:35.586 INFO:teuthology.orchestra.run.smithi139.stdout:55834574855 2024-04-25T09:53:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.585+0000 7f6cadda7640 1 -- 172.21.15.139:0/2549447221 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6c8c06b520 msgr2=0x7f6c8c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.585+0000 7f6cadda7640 1 --2- 172.21.15.139:0/2549447221 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6c8c06b520 0x7f6c8c06d9e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ca004df50 tx=0x7f6c94009040 comp rx=0 tx=0).stop 2024-04-25T09:53:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.585+0000 7f6cadda7640 1 -- 172.21.15.139:0/2549447221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca00065e0 msgr2=0x7f6ca004e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.585+0000 7f6cadda7640 1 --2- 172.21.15.139:0/2549447221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca00065e0 0x7f6ca004e600 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00b450 tx=0x7f6c9c00b910 comp rx=0 tx=0).stop 2024-04-25T09:53:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.586+0000 7f6cadda7640 1 -- 172.21.15.139:0/2549447221 shutdown_connections 2024-04-25T09:53:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.586+0000 7f6cadda7640 1 --2- 172.21.15.139:0/2549447221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca0006f80 0x7f6ca00d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.586+0000 7f6cadda7640 1 --2- 172.21.15.139:0/2549447221 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6c8c06b520 0x7f6c8c06d9e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.586+0000 7f6cadda7640 1 --2- 172.21.15.139:0/2549447221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca00065e0 0x7f6ca004e600 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.586+0000 7f6cadda7640 1 --2- 172.21.15.139:0/2549447221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca0006200 0x7f6ca004e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.586+0000 7f6cadda7640 1 -- 172.21.15.139:0/2549447221 >> 172.21.15.139:0/2549447221 conn(0x7f6ca0058ee0 msgr2=0x7f6ca00d65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.588+0000 7f6cadda7640 1 -- 172.21.15.139:0/2549447221 shutdown_connections 2024-04-25T09:53:35.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.588+0000 7f6cadda7640 1 -- 172.21.15.139:0/2549447221 wait complete. 2024-04-25T09:53:35.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.1 2024-04-25T09:53:35.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.615+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2de40089c0 con 0x7f2de404fa80 2024-04-25T09:53:35.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.616+0000 7f2dd7fff640 1 -- 172.21.15.139:0/3780046533 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2dd80a9e20 con 0x7f2de404fa80 2024-04-25T09:53:35.619 INFO:teuthology.orchestra.run.smithi139.stdout:55834574857 2024-04-25T09:53:35.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb424005580 con 0x7fb43c053c60 2024-04-25T09:53:35.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2dd006b5a0 msgr2=0x7f2dd006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7f2df1dd5640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2dd006b5a0 0x7f2dd006da60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2de0001860 tx=0x7f2de0009040 comp rx=0 tx=0).stop 2024-04-25T09:53:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de404fa80 msgr2=0x7f2de40e0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7f2df1dd5640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de404fa80 0x7f2de40e0270 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8000c00 tx=0x7f2dd80046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 shutdown_connections 2024-04-25T09:53:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7f2df1dd5640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2de40df410 0x7f2de40e07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7f2df1dd5640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2dd006b5a0 0x7f2dd006da60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7f2df1dd5640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de404fa80 0x7f2de40e0270 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7f2df1dd5640 1 --2- 172.21.15.139:0/3780046533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40392c0 0x7f2de4178bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 >> 172.21.15.139:0/3780046533 conn(0x7f2de4009fa0 msgr2=0x7f2de4052e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 shutdown_connections 2024-04-25T09:53:35.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7f2df1dd5640 1 -- 172.21.15.139:0/3780046533 wait complete. 2024-04-25T09:53:35.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.618+0000 7fb440ff9640 1 -- 172.21.15.139:0/36932350 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb43005de60 con 0x7fb43c053c60 2024-04-25T09:53:35.624 INFO:teuthology.orchestra.run.smithi139.stdout:55834574855 2024-04-25T09:53:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.622+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb42806b5a0 msgr2=0x7fb42806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.622+0000 7fb44923c640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb42806b5a0 0x7fb42806da60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb4380094a0 tx=0x7fb438009240 comp rx=0 tx=0).stop 2024-04-25T09:53:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.622+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb43c053c60 msgr2=0x7fb43c0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.622+0000 7fb44923c640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb43c053c60 0x7fb43c0cf9d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb43000e700 tx=0x7fb43000ebc0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.622+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 shutdown_connections 2024-04-25T09:53:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.622+0000 7fb44923c640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb42806b5a0 0x7fb42806da60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.622+0000 7fb44923c640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb43c0df210 0x7fb43c0cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.622+0000 7fb44923c640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb43c053c60 0x7fb43c0cf9d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.622+0000 7fb44923c640 1 --2- 172.21.15.139:0/36932350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb43c050190 0x7fb43c02d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.622+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 >> 172.21.15.139:0/36932350 conn(0x7fb43c009fa0 msgr2=0x7fb43c036e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.622+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 shutdown_connections 2024-04-25T09:53:35.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.622+0000 7fb44923c640 1 -- 172.21.15.139:0/36932350 wait complete. 2024-04-25T09:53:35.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.2 2024-04-25T09:53:35.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.632+0000 7f7b20d9e640 1 -- 172.21.15.139:0/844086375 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7b00005580 con 0x7f7b1c0ad840 2024-04-25T09:53:35.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.632+0000 7f7b0affd640 1 -- 172.21.15.139:0/844086375 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b14058810 con 0x7f7b1c0ad840 2024-04-25T09:53:35.635 INFO:teuthology.orchestra.run.smithi139.stdout:55834574856 2024-04-25T09:53:35.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.634+0000 7f7b20d9e640 1 -- 172.21.15.139:0/844086375 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7afc06b5a0 msgr2=0x7f7afc06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.634+0000 7f7b20d9e640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7afc06b5a0 0x7f7afc06da60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000b9e0 tx=0x7f7b10030000 comp rx=0 tx=0).stop 2024-04-25T09:53:35.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.634+0000 7f7b20d9e640 1 -- 172.21.15.139:0/844086375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b1c0ad840 msgr2=0x7f7b1c0adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.634+0000 7f7b20d9e640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b1c0ad840 0x7f7b1c0adcc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c16a9a0 tx=0x7f7b14005f30 comp rx=0 tx=0).stop 2024-04-25T09:53:35.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.634+0000 7f7b20d9e640 1 -- 172.21.15.139:0/844086375 shutdown_connections 2024-04-25T09:53:35.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.634+0000 7f7b20d9e640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7afc06b5a0 0x7f7afc06da60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.634+0000 7f7b20d9e640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b1c0ad840 0x7f7b1c0adcc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.634+0000 7f7b20d9e640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b1c204210 0x7f7b1c0a8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.634+0000 7f7b20d9e640 1 --2- 172.21.15.139:0/844086375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b1c168f00 0x7f7b1c0ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.634+0000 7f7b20d9e640 1 -- 172.21.15.139:0/844086375 >> 172.21.15.139:0/844086375 conn(0x7f7b1c06d0e0 msgr2=0x7f7b1c0cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.634+0000 7f7b20d9e640 1 -- 172.21.15.139:0/844086375 shutdown_connections 2024-04-25T09:53:35.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.634+0000 7f7b20d9e640 1 -- 172.21.15.139:0/844086375 wait complete. 2024-04-25T09:53:35.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.638+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f61080d2160 con 0x7f610815d6c0 2024-04-25T09:53:35.642 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.5 2024-04-25T09:53:35.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.639+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3b08005580 con 0x7f3b240b9370 2024-04-25T09:53:35.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.639+0000 7f60f6ffd640 1 -- 172.21.15.139:0/2056325491 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f60fc05c6f0 con 0x7f610815d6c0 2024-04-25T09:53:35.643 INFO:teuthology.orchestra.run.smithi139.stdout:55834574856 2024-04-25T09:53:35.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.639+0000 7f3b12ffd640 1 -- 172.21.15.139:0/3216914499 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3b1807d460 con 0x7f3b240b9370 2024-04-25T09:53:35.644 INFO:teuthology.orchestra.run.smithi139.stdout:55834574856 2024-04-25T09:53:35.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.641+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f60e806b4d0 msgr2=0x7f60e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.641+0000 7f610759e640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f60e806b4d0 0x7f60e806d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f61080d1e90 tx=0x7f60f8008040 comp rx=0 tx=0).stop 2024-04-25T09:53:35.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.641+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f610815d6c0 msgr2=0x7f61080c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.641+0000 7f610759e640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f610815d6c0 0x7f61080c9e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00c8f0 tx=0x7f60fc00cdb0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.641+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 shutdown_connections 2024-04-25T09:53:35.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.641+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b0406b4d0 msgr2=0x7f3b0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.641+0000 7f3b28ab3640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b0406b4d0 0x7f3b0406d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3b24203f00 tx=0x7f3b1c009040 comp rx=0 tx=0).stop 2024-04-25T09:53:35.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.641+0000 7f610759e640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6108163650 0x7f6108153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f610759e640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f60e806b4d0 0x7f60e806d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.641+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b240b9370 msgr2=0x7f3b2415b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.641+0000 7f3b28ab3640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b240b9370 0x7f3b2415b5e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009870 tx=0x7f3b180046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f610759e640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f610815d6c0 0x7f61080c9e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f610759e640 1 --2- 172.21.15.139:0/2056325491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f00051c0 0x7f61081f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 >> 172.21.15.139:0/2056325491 conn(0x7f610806bf60 msgr2=0x7f61080cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 shutdown_connections 2024-04-25T09:53:35.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f3b28ab3640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b240cee50 0x7f3b241f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f3b28ab3640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b0406b4d0 0x7f3b0406d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 shutdown_connections 2024-04-25T09:53:35.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f3b28ab3640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b241f3eb0 0x7f3b24158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f3b28ab3640 1 --2- 172.21.15.139:0/3216914499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b240b9370 0x7f3b2415b5e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 >> 172.21.15.139:0/3216914499 conn(0x7f3b240b5480 msgr2=0x7f3b240cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f610759e640 1 -- 172.21.15.139:0/2056325491 wait complete. 2024-04-25T09:53:35.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 shutdown_connections 2024-04-25T09:53:35.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.642+0000 7f3b28ab3640 1 -- 172.21.15.139:0/3216914499 wait complete. 2024-04-25T09:53:35.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.645+0000 7ff3ce7fc640 1 -- 172.21.15.139:0/3116045638 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff3e80cbee0 con 0x7ff3e8162fa0 2024-04-25T09:53:35.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.646+0000 7ff3e4ff9640 1 -- 172.21.15.139:0/3116045638 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff3e008b0e0 con 0x7ff3e8162fa0 2024-04-25T09:53:35.649 INFO:teuthology.orchestra.run.smithi139.stdout:55834574856 2024-04-25T09:53:35.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.6 2024-04-25T09:53:35.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.648+0000 7ff3ce7fc640 1 -- 172.21.15.139:0/3116045638 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3d406b5a0 msgr2=0x7ff3d406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.648+0000 7ff3ce7fc640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3d406b5a0 0x7ff3d406da60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009c00 tx=0x7ff3dc009190 comp rx=0 tx=0).stop 2024-04-25T09:53:35.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.648+0000 7ff3ce7fc640 1 -- 172.21.15.139:0/3116045638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e8162fa0 msgr2=0x7ff3e80c4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:35.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.648+0000 7ff3ce7fc640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e8162fa0 0x7ff3e80c4930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000eed0 tx=0x7ff3e000b480 comp rx=0 tx=0).stop 2024-04-25T09:53:35.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.648+0000 7ff3ce7fc640 1 -- 172.21.15.139:0/3116045638 shutdown_connections 2024-04-25T09:53:35.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.648+0000 7ff3ce7fc640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3d406b5a0 0x7ff3d406da60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.648+0000 7ff3ce7fc640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e8162fa0 0x7ff3e80c4930 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.648+0000 7ff3ce7fc640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3e8161d30 0x7ff3e8159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.648+0000 7ff3ce7fc640 1 --2- 172.21.15.139:0/3116045638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3e80cc3a0 0x7ff3e81f8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:35.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.648+0000 7ff3ce7fc640 1 -- 172.21.15.139:0/3116045638 >> 172.21.15.139:0/3116045638 conn(0x7ff3e806ecb0 msgr2=0x7ff3e806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:35.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.648+0000 7ff3ce7fc640 1 -- 172.21.15.139:0/3116045638 shutdown_connections 2024-04-25T09:53:35.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:35.648+0000 7ff3ce7fc640 1 -- 172.21.15.139:0/3116045638 wait complete. 2024-04-25T09:53:35.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.4 2024-04-25T09:53:35.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.3 2024-04-25T09:53:35.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.0 2024-04-25T09:53:36.558 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T09:53:36.603 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T09:53:36.636 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T09:53:36.643 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T09:53:36.652 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T09:53:36.665 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T09:53:36.666 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T09:53:36.666 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T09:53:36.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.669+0000 7ff707e76640 1 Processor -- start 2024-04-25T09:53:36.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.669+0000 7ff707e76640 1 -- start start 2024-04-25T09:53:36.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.670+0000 7ff707e76640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff70015d9e0 0x7ff7000668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.670+0000 7ff707e76640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7001616d0 0x7ff700161b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.670+0000 7ff707e76640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7000b34f0 con 0x7ff7000b4960 2024-04-25T09:53:36.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.670+0000 7ff707e76640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff700162080 con 0x7ff70015d9e0 2024-04-25T09:53:36.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.670+0000 7ff707e76640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000b4ed0 con 0x7ff7001616d0 2024-04-25T09:53:36.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.670+0000 7ff706413640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7001616d0 0x7ff700161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.670+0000 7ff705c12640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff70015d9e0 0x7ff7000668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.670+0000 7ff706413640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7001616d0 0x7ff700161b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:54994/0 (socket says 172.21.15.139:54994) 2024-04-25T09:53:36.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.670+0000 7ff706413640 1 -- 172.21.15.139:0/886568377 learned_addr learned my addr 172.21.15.139:0/886568377 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.671+0000 7ff706413640 1 -- 172.21.15.139:0/886568377 >> v2:172.21.15.150:3300/0 conn(0x7ff70015d9e0 msgr2=0x7ff7000668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:36.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.671+0000 7ff706413640 1 --2- 172.21.15.139:0/886568377 >> v2:172.21.15.150:3300/0 conn(0x7ff70015d9e0 0x7ff7000668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.671+0000 7ff706413640 1 -- 172.21.15.139:0/886568377 >> v1:172.21.15.150:6789/0 conn(0x7ff7000b4960 legacy=0x7ff70006bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.671+0000 7ff706413640 1 -- 172.21.15.139:0/886568377 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000b2f30 con 0x7ff7001616d0 2024-04-25T09:53:36.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.672+0000 7ff706413640 1 --2- 172.21.15.139:0/886568377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7001616d0 0x7ff700161b40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009e50 tx=0x7ff6f802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b9082164a9161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.672+0000 7ff704c10640 1 -- 172.21.15.139:0/886568377 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8036070 con 0x7ff7001616d0 2024-04-25T09:53:36.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.672+0000 7ff704c10640 1 -- 172.21.15.139:0/886568377 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f8031600 con 0x7ff7001616d0 2024-04-25T09:53:36.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.672+0000 7ff704c10640 1 -- 172.21.15.139:0/886568377 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80319d0 con 0x7ff7001616d0 2024-04-25T09:53:36.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.673+0000 7ff707e76640 1 -- 172.21.15.139:0/886568377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7001616d0 msgr2=0x7ff700161b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.673+0000 7ff707e76640 1 --2- 172.21.15.139:0/886568377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7001616d0 0x7ff700161b40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009e50 tx=0x7ff6f802a710 comp rx=0 tx=0).stop 2024-04-25T09:53:36.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.673+0000 7ff707e76640 1 -- 172.21.15.139:0/886568377 shutdown_connections 2024-04-25T09:53:36.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.673+0000 7ff707e76640 1 --2- 172.21.15.139:0/886568377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7001616d0 0x7ff700161b40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.673+0000 7ff707e76640 1 --2- 172.21.15.139:0/886568377 >> v2:172.21.15.150:3300/0 conn(0x7ff70015d9e0 0x7ff7000668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.673+0000 7ff707e76640 1 -- 172.21.15.139:0/886568377 >> 172.21.15.139:0/886568377 conn(0x7ff7000cc820 msgr2=0x7ff7000ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:36.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.674+0000 7ff707e76640 1 -- 172.21.15.139:0/886568377 shutdown_connections 2024-04-25T09:53:36.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.674+0000 7ff707e76640 1 -- 172.21.15.139:0/886568377 wait complete. 2024-04-25T09:53:36.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.676+0000 7ff707e76640 1 Processor -- start 2024-04-25T09:53:36.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.676+0000 7ff707e76640 1 -- start start 2024-04-25T09:53:36.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.677+0000 7ff707e76640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7000b4960 0x7ff700158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.678+0000 7ff707e76640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff70015d9e0 0x7ff700149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.678+0000 7ff705c12640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7000b4960 0x7ff700158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.678+0000 7ff705c12640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7000b4960 0x7ff700158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:55002/0 (socket says 172.21.15.139:55002) 2024-04-25T09:53:36.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.678+0000 7ff705411640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff70015d9e0 0x7ff700149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.678+0000 7ff705411640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff70015d9e0 0x7ff700149620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:36748/0 (socket says 172.21.15.139:36748) 2024-04-25T09:53:36.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.678+0000 7ff705411640 1 -- 172.21.15.139:0/3854039778 learned_addr learned my addr 172.21.15.139:0/3854039778 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.678+0000 7ff707e76640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7001616d0 0x7ff7000ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.679+0000 7ff706413640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7001616d0 0x7ff7000ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.679+0000 7ff707e76640 1 -- 172.21.15.139:0/3854039778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70015ddc0 con 0x7ff7001616d0 2024-04-25T09:53:36.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.679+0000 7ff707e76640 1 -- 172.21.15.139:0/3854039778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff700162430 con 0x7ff70015d9e0 2024-04-25T09:53:36.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.679+0000 7ff707e76640 1 -- 172.21.15.139:0/3854039778 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70015df40 con 0x7ff7000b4960 2024-04-25T09:53:36.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.680+0000 7ff705411640 1 -- 172.21.15.139:0/3854039778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7000b4960 msgr2=0x7ff700158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.680+0000 7ff705411640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7000b4960 0x7ff700158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.680+0000 7ff705411640 1 -- 172.21.15.139:0/3854039778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7001616d0 msgr2=0x7ff7000ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.680+0000 7ff705411640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7001616d0 0x7ff7000ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.680+0000 7ff705411640 1 -- 172.21.15.139:0/3854039778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70015a000 con 0x7ff70015d9e0 2024-04-25T09:53:36.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.680+0000 7ff706413640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7001616d0 0x7ff7000ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:36.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.681+0000 7ff705c12640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7000b4960 0x7ff700158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:36.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.681+0000 7ff705411640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff70015d9e0 0x7ff700149620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000c430 tx=0x7ff6f000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.681+0000 7ff6f67fc640 1 -- 172.21.15.139:0/3854039778 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f00052d0 con 0x7ff70015d9e0 2024-04-25T09:53:36.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.681+0000 7ff6f67fc640 1 -- 172.21.15.139:0/3854039778 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f0005470 con 0x7ff70015d9e0 2024-04-25T09:53:36.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.681+0000 7ff6f67fc640 1 -- 172.21.15.139:0/3854039778 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0007240 con 0x7ff70015d9e0 2024-04-25T09:53:36.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.681+0000 7ff707e76640 1 -- 172.21.15.139:0/3854039778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff700157b80 con 0x7ff70015d9e0 2024-04-25T09:53:36.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.683+0000 7ff707e76640 1 -- 172.21.15.139:0/3854039778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff700157e20 con 0x7ff70015d9e0 2024-04-25T09:53:36.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.685+0000 7ff707e76640 1 -- 172.21.15.139:0/3854039778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff700074520 con 0x7ff70015d9e0 2024-04-25T09:53:36.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.687+0000 7ff6f67fc640 1 -- 172.21.15.139:0/3854039778 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6f00073e0 con 0x7ff70015d9e0 2024-04-25T09:53:36.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.688+0000 7ff6f67fc640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff6e406b5a0 0x7ff6e406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.688+0000 7ff6f67fc640 1 -- 172.21.15.139:0/3854039778 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff6f0089960 con 0x7ff70015d9e0 2024-04-25T09:53:36.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.688+0000 7ff705c12640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff6e406b5a0 0x7ff6e406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.689+0000 7ff705c12640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff6e406b5a0 0x7ff6e406da60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009c10 tx=0x7ff6fc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.692+0000 7ff6f67fc640 1 -- 172.21.15.139:0/3854039778 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6f0057ec0 con 0x7ff70015d9e0 2024-04-25T09:53:36.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.733+0000 7f3a5dc9a640 1 Processor -- start 2024-04-25T09:53:36.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.733+0000 7f3a5dc9a640 1 -- start start 2024-04-25T09:53:36.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.734+0000 7f3a5dc9a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3a50048c30 0x7f3a50049060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.735+0000 7f3a5dc9a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a5003aa10 0x7f3a5003ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.735+0000 7f3a5dc9a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a50009560 con 0x7f3a50009180 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.735+0000 7f3a57fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3a50048c30 0x7f3a50049060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.735+0000 7f3a57fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3a50048c30 0x7f3a50049060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:36756/0 (socket says 172.21.15.139:36756) 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.735+0000 7f3a57fff640 1 -- 172.21.15.139:0/2255316114 learned_addr learned my addr 172.21.15.139:0/2255316114 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.735+0000 7f3a5d499640 1 --2- 172.21.15.139:0/2255316114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a5003aa10 0x7f3a5003ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.735+0000 7f3a5dc9a640 1 -- 172.21.15.139:0/2255316114 --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a500495a0 con 0x7f3a50048c30 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.735+0000 7f3a5dc9a640 1 -- 172.21.15.139:0/2255316114 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50035960 con 0x7f3a5003aa10 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.735+0000 7f3a577fe640 1 -- 172.21.15.139:0/2255316114 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125959396 0 0) 0x7f3a50009560 con 0x7f3a50009180 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.735+0000 7f3a577fe640 1 -- 172.21.15.139:0/2255316114 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a48003520 con 0x7f3a50009180 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.736+0000 7f3a5d499640 1 -- 172.21.15.139:0/2255316114 >> v2:172.21.15.150:3300/0 conn(0x7f3a50048c30 msgr2=0x7f3a50049060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.736+0000 7f3a5d499640 1 --2- 172.21.15.139:0/2255316114 >> v2:172.21.15.150:3300/0 conn(0x7f3a50048c30 0x7f3a50049060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.736+0000 7f3a5d499640 1 -- 172.21.15.139:0/2255316114 >> v1:172.21.15.150:6789/0 conn(0x7f3a50009180 legacy=0x7f3a50052ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.736+0000 7f3a5d499640 1 -- 172.21.15.139:0/2255316114 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a50035c20 con 0x7f3a5003aa10 2024-04-25T09:53:36.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.737+0000 7f3a5d499640 1 --2- 172.21.15.139:0/2255316114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a5003aa10 0x7f3a5003ae60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a4401a7c0 tx=0x7f3a4403b1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00e847371fc77bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.738+0000 7f3a577fe640 1 -- 172.21.15.139:0/2255316114 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a44048070 con 0x7f3a5003aa10 2024-04-25T09:53:36.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.738+0000 7f3a577fe640 1 -- 172.21.15.139:0/2255316114 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a44043440 con 0x7f3a5003aa10 2024-04-25T09:53:36.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.738+0000 7f3a577fe640 1 -- 172.21.15.139:0/2255316114 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a44043850 con 0x7f3a5003aa10 2024-04-25T09:53:36.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.739+0000 7f3a5dc9a640 1 -- 172.21.15.139:0/2255316114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a5003aa10 msgr2=0x7f3a5003ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.739+0000 7f3a5dc9a640 1 --2- 172.21.15.139:0/2255316114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a5003aa10 0x7f3a5003ae60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a4401a7c0 tx=0x7f3a4403b1e0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.739+0000 7f3a5dc9a640 1 -- 172.21.15.139:0/2255316114 shutdown_connections 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.739+0000 7f3a5dc9a640 1 --2- 172.21.15.139:0/2255316114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a5003aa10 0x7f3a5003ae60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.739+0000 7f3a5dc9a640 1 --2- 172.21.15.139:0/2255316114 >> v2:172.21.15.150:3300/0 conn(0x7f3a50048c30 0x7f3a50049060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.739+0000 7f3a5dc9a640 1 -- 172.21.15.139:0/2255316114 >> 172.21.15.139:0/2255316114 conn(0x7f3a5001ef30 msgr2=0x7f3a50055c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.739+0000 7f3a5dc9a640 1 -- 172.21.15.139:0/2255316114 shutdown_connections 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.739+0000 7f3a5dc9a640 1 -- 172.21.15.139:0/2255316114 wait complete. 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.741+0000 7f3a5dc9a640 1 Processor -- start 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.741+0000 7f3a5dc9a640 1 -- start start 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.742+0000 7f3a5dc9a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a50009180 0x7f3a5017c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a5dc9a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a5003aa10 0x7f3a50037790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a5dc9a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a50048c30 0x7f3a50049a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a5dc9a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500364c0 con 0x7f3a50009180 2024-04-25T09:53:36.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a5dc9a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500361e0 con 0x7f3a5003aa10 2024-04-25T09:53:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a5dc9a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500367a0 con 0x7f3a50048c30 2024-04-25T09:53:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a5cc98640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a50009180 0x7f3a5017c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a5cc98640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a50009180 0x7f3a5017c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40200/0 (socket says 172.21.15.139:40200) 2024-04-25T09:53:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a5cc98640 1 -- 172.21.15.139:0/686780305 learned_addr learned my addr 172.21.15.139:0/686780305 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a57fff640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a5003aa10 0x7f3a50037790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a57fff640 1 -- 172.21.15.139:0/686780305 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a50048c30 msgr2=0x7f3a50049a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T09:53:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a57fff640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a50048c30 0x7f3a50049a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a57fff640 1 -- 172.21.15.139:0/686780305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a50009180 msgr2=0x7f3a5017c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a57fff640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a50009180 0x7f3a5017c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.743+0000 7f3a57fff640 1 -- 172.21.15.139:0/686780305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a50033a70 con 0x7f3a5003aa10 2024-04-25T09:53:36.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.744+0000 7f3a57fff640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a5003aa10 0x7f3a50037790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a58068260 tx=0x7f3a580682c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.745+0000 7f3a557fa640 1 -- 172.21.15.139:0/686780305 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58086070 con 0x7f3a5003aa10 2024-04-25T09:53:36.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.745+0000 7f3a557fa640 1 -- 172.21.15.139:0/686780305 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a58073c40 con 0x7f3a5003aa10 2024-04-25T09:53:36.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.745+0000 7f3a557fa640 1 -- 172.21.15.139:0/686780305 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58081630 con 0x7f3a5003aa10 2024-04-25T09:53:36.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.745+0000 7f3a5dc9a640 1 -- 172.21.15.139:0/686780305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5019a6e0 con 0x7f3a5003aa10 2024-04-25T09:53:36.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.745+0000 7f3a5dc9a640 1 -- 172.21.15.139:0/686780305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5019aca0 con 0x7f3a5003aa10 2024-04-25T09:53:36.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.748+0000 7f3a42ffd640 1 -- 172.21.15.139:0/686780305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a38005290 con 0x7f3a5003aa10 2024-04-25T09:53:36.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.749+0000 7f3a557fa640 1 -- 172.21.15.139:0/686780305 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a58089070 con 0x7f3a5003aa10 2024-04-25T09:53:36.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.749+0000 7f3a557fa640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a4806ba40 0x7f3a4806df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.750+0000 7f3a557fa640 1 -- 172.21.15.139:0/686780305 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3a580f37e0 con 0x7f3a5003aa10 2024-04-25T09:53:36.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.751+0000 7f3a5cc98640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a4806ba40 0x7f3a4806df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.751+0000 7f3a5cc98640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a4806ba40 0x7f3a4806df00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c007fd0 tx=0x7f3a4c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.754+0000 7f3a557fa640 1 -- 172.21.15.139:0/686780305 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a5807e070 con 0x7f3a5003aa10 2024-04-25T09:53:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.806+0000 7fc53145d640 1 Processor -- start 2024-04-25T09:53:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.803+0000 7f85d6189640 1 Processor -- start 2024-04-25T09:53:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.803+0000 7f85d6189640 1 -- start start 2024-04-25T09:53:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.804+0000 7f85d6189640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85d0154090 0x7f85d00b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.804+0000 7f85d6189640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f85d00b2d70 0x7f85d0071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.804+0000 7f85d6189640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d0154470 con 0x7f85d00711c0 2024-04-25T09:53:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.804+0000 7f85d6189640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d00b4b40 con 0x7f85d00b2d70 2024-04-25T09:53:36.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.804+0000 7f85d6189640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d00b4170 con 0x7f85d0154090 2024-04-25T09:53:36.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.805+0000 7f85cffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f85d00b2d70 0x7f85d0071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.805+0000 7f85cffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f85d00b2d70 0x7f85d0071640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40214/0 (socket says 172.21.15.139:40214) 2024-04-25T09:53:36.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.805+0000 7f85cffff640 1 -- 172.21.15.139:0/3789297124 learned_addr learned my addr 172.21.15.139:0/3789297124 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.805+0000 7f85cffff640 1 -- 172.21.15.139:0/3789297124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85d0154090 msgr2=0x7f85d00b4600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:53:36.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.805+0000 7f85cffff640 1 --2- 172.21.15.139:0/3789297124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85d0154090 0x7f85d00b4600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.805+0000 7f85cffff640 1 -- 172.21.15.139:0/3789297124 >> v1:172.21.15.150:6789/0 conn(0x7f85d00711c0 legacy=0x7f85d00b2660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:53:36.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.805+0000 7f85cffff640 1 -- 172.21.15.139:0/3789297124 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d00b3390 con 0x7f85d00b2d70 2024-04-25T09:53:36.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.807+0000 7f85cffff640 1 --2- 172.21.15.139:0/3789297124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d00b2d70 0x7f85d0071640 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009e50 tx=0x7f85c802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2610528ddaf6efbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.807+0000 7f85ce7fc640 1 -- 172.21.15.139:0/3789297124 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8036070 con 0x7f85d00b2d70 2024-04-25T09:53:36.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.807+0000 7f85ce7fc640 1 -- 172.21.15.139:0/3789297124 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c8031600 con 0x7f85d00b2d70 2024-04-25T09:53:36.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.806+0000 7fc53145d640 1 -- start start 2024-04-25T09:53:36.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.807+0000 7fc53145d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc52c0b25f0 0x7fc52c06ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.807+0000 7fc53145d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c15ab20 0x7fc52c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.807+0000 7fc53145d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c06f4d0 con 0x7fc52c15a030 2024-04-25T09:53:36.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.807+0000 7fc53145d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc52c06f760 con 0x7fc52c0b25f0 2024-04-25T09:53:36.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.807+0000 7fc53145d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c06fa40 con 0x7fc52c15ab20 2024-04-25T09:53:36.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.808+0000 7fc52b7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c15ab20 0x7fc52c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.808+0000 7fc52b7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c15ab20 0x7fc52c06ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:55046/0 (socket says 172.21.15.139:55046) 2024-04-25T09:53:36.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.808+0000 7fc52b7fe640 1 -- 172.21.15.139:0/3177372200 learned_addr learned my addr 172.21.15.139:0/3177372200 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.808+0000 7f85d6189640 1 -- 172.21.15.139:0/3789297124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d00b2d70 msgr2=0x7f85d0071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.808+0000 7f85d6189640 1 --2- 172.21.15.139:0/3789297124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d00b2d70 0x7f85d0071640 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009e50 tx=0x7f85c802a660 comp rx=0 tx=0).stop 2024-04-25T09:53:36.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.809+0000 7f85d6189640 1 -- 172.21.15.139:0/3789297124 shutdown_connections 2024-04-25T09:53:36.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.809+0000 7f85d6189640 1 --2- 172.21.15.139:0/3789297124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d00b2d70 0x7f85d0071640 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.809+0000 7f85d6189640 1 --2- 172.21.15.139:0/3789297124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85d0154090 0x7f85d00b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.809+0000 7fc52a7fc640 1 --2- 172.21.15.139:0/3177372200 >> v2:172.21.15.139:3300/0 conn(0x7fc52c0b25f0 0x7fc52c06ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.809+0000 7f85d6189640 1 -- 172.21.15.139:0/3789297124 >> 172.21.15.139:0/3789297124 conn(0x7f85d0067000 msgr2=0x7f85d006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:36.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.809+0000 7f85d6189640 1 -- 172.21.15.139:0/3789297124 shutdown_connections 2024-04-25T09:53:36.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.809+0000 7fc52a7fc640 1 -- 172.21.15.139:0/3177372200 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c15ab20 msgr2=0x7fc52c06ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:36.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.809+0000 7fc52a7fc640 1 --2- 172.21.15.139:0/3177372200 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c15ab20 0x7fc52c06ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.810+0000 7fc52a7fc640 1 -- 172.21.15.139:0/3177372200 >> v1:172.21.15.150:6789/0 conn(0x7fc52c15a030 legacy=0x7fc52c15a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:36.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.810+0000 7fc52a7fc640 1 -- 172.21.15.139:0/3177372200 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c0cdc60 con 0x7fc52c0b25f0 2024-04-25T09:53:36.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.811+0000 7fc52a7fc640 1 --2- 172.21.15.139:0/3177372200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc52c0b25f0 0x7fc52c06ea50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0097a0 tx=0x7fc51c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0536840469159d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.811+0000 7fc529ffb640 1 -- 172.21.15.139:0/3177372200 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c037070 con 0x7fc52c0b25f0 2024-04-25T09:53:36.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.812+0000 7fc529ffb640 1 -- 172.21.15.139:0/3177372200 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c032440 con 0x7fc52c0b25f0 2024-04-25T09:53:36.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.812+0000 7fc529ffb640 1 -- 172.21.15.139:0/3177372200 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c02ae60 con 0x7fc52c0b25f0 2024-04-25T09:53:36.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7fc53145d640 1 -- 172.21.15.139:0/3177372200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc52c0b25f0 msgr2=0x7fc52c06ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7fc53145d640 1 --2- 172.21.15.139:0/3177372200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc52c0b25f0 0x7fc52c06ea50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0097a0 tx=0x7fc51c02a150 comp rx=0 tx=0).stop 2024-04-25T09:53:36.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7fc53145d640 1 -- 172.21.15.139:0/3177372200 shutdown_connections 2024-04-25T09:53:36.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7fc53145d640 1 --2- 172.21.15.139:0/3177372200 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c15ab20 0x7fc52c06ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7fc53145d640 1 --2- 172.21.15.139:0/3177372200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc52c0b25f0 0x7fc52c06ea50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7fc53145d640 1 -- 172.21.15.139:0/3177372200 >> 172.21.15.139:0/3177372200 conn(0x7fc52c06af80 msgr2=0x7fc52c06b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:36.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.814+0000 7fc53145d640 1 -- 172.21.15.139:0/3177372200 shutdown_connections 2024-04-25T09:53:36.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.809+0000 7f85d6189640 1 -- 172.21.15.139:0/3789297124 wait complete. 2024-04-25T09:53:36.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.811+0000 7f85d6189640 1 Processor -- start 2024-04-25T09:53:36.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.811+0000 7f85d6189640 1 -- start start 2024-04-25T09:53:36.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.812+0000 7f85d6189640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85d00711c0 0x7f85d01579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.812+0000 7f85d6189640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d00b2d70 0x7f85d015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.812+0000 7f85ceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d00b2d70 0x7f85d015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7f85ceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d00b2d70 0x7f85d015c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40222/0 (socket says 172.21.15.139:40222) 2024-04-25T09:53:36.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7f85ceffd640 1 -- 172.21.15.139:0/4211374856 learned_addr learned my addr 172.21.15.139:0/4211374856 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7f85d6189640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d01f6900 0x7f85d015c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7f85d6189640 1 -- 172.21.15.139:0/4211374856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d006f2b0 con 0x7f85d00b2d70 2024-04-25T09:53:36.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7f85d6189640 1 -- 172.21.15.139:0/4211374856 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d00b3e00 con 0x7f85d01f6900 2024-04-25T09:53:36.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7f85d6189640 1 -- 172.21.15.139:0/4211374856 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d006f430 con 0x7f85d00711c0 2024-04-25T09:53:36.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7f85cffff640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d01f6900 0x7f85d015c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.814+0000 7fc53145d640 1 -- 172.21.15.139:0/3177372200 wait complete. 2024-04-25T09:53:36.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.816+0000 7fc53145d640 1 Processor -- start 2024-04-25T09:53:36.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7f85ceffd640 1 -- 172.21.15.139:0/4211374856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85d00711c0 msgr2=0x7f85d01579b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T09:53:36.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7f85ceffd640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85d00711c0 0x7f85d01579b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.816+0000 7fc53145d640 1 -- start start 2024-04-25T09:53:36.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.817+0000 7fc53145d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c0b25f0 0x7fc52c14ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7f85ceffd640 1 -- 172.21.15.139:0/4211374856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d01f6900 msgr2=0x7f85d015c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7f85ceffd640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d01f6900 0x7f85d015c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.813+0000 7f85ceffd640 1 -- 172.21.15.139:0/4211374856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d01f6690 con 0x7f85d00b2d70 2024-04-25T09:53:36.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.817+0000 7fc53145d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc52c15a030 0x7fc52c14c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.817+0000 7fc53145d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc52c15ab20 0x7fc52c1547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.814+0000 7f85ceffd640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d00b2d70 0x7f85d015c430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f85c000b700 tx=0x7f85c000bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.815+0000 7f85b7fff640 1 -- 172.21.15.139:0/4211374856 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0019070 con 0x7f85d00b2d70 2024-04-25T09:53:36.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc53145d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c154330 con 0x7fc52c15ab20 2024-04-25T09:53:36.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc53145d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0cd910 con 0x7fc52c15a030 2024-04-25T09:53:36.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc53145d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c1544b0 con 0x7fc52c0b25f0 2024-04-25T09:53:36.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.815+0000 7f85d6189640 1 -- 172.21.15.139:0/4211374856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d0158140 con 0x7f85d00b2d70 2024-04-25T09:53:36.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.815+0000 7f85d6189640 1 -- 172.21.15.139:0/4211374856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d0158700 con 0x7f85d00b2d70 2024-04-25T09:53:36.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.816+0000 7f85b7fff640 1 -- 172.21.15.139:0/4211374856 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c000c800 con 0x7f85d00b2d70 2024-04-25T09:53:36.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc52affd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c0b25f0 0x7fc52c14ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc52affd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c0b25f0 0x7fc52c14ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:55048/0 (socket says 172.21.15.139:55048) 2024-04-25T09:53:36.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc52affd640 1 -- 172.21.15.139:0/4063297335 learned_addr learned my addr 172.21.15.139:0/4063297335 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.817+0000 7f85b7fff640 1 -- 172.21.15.139:0/4211374856 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c000cbf0 con 0x7f85d00b2d70 2024-04-25T09:53:36.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc52affd640 1 -- 172.21.15.139:0/4063297335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc52c15a030 msgr2=0x7fc52c14c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc52a7fc640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc52c15a030 0x7fc52c14c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc52b7fe640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc52c15ab20 0x7fc52c1547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc52affd640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc52c15a030 0x7fc52c14c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc52affd640 1 -- 172.21.15.139:0/4063297335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc52c15ab20 msgr2=0x7fc52c1547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc52affd640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc52c15ab20 0x7fc52c1547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc52affd640 1 -- 172.21.15.139:0/4063297335 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c14b1e0 con 0x7fc52c0b25f0 2024-04-25T09:53:36.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.819+0000 7f85b5ffb640 1 -- 172.21.15.139:0/4211374856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b0005290 con 0x7f85d00b2d70 2024-04-25T09:53:36.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.818+0000 7fc52b7fe640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc52c15ab20 0x7fc52c1547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:36.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.819+0000 7fc52a7fc640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc52c15a030 0x7fc52c14c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:36.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.819+0000 7fc52affd640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c0b25f0 0x7fc52c14ff40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c430 tx=0x7fc52000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.819+0000 7fc5137fe640 1 -- 172.21.15.139:0/4063297335 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520011790 con 0x7fc52c0b25f0 2024-04-25T09:53:36.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.821+0000 7f85b7fff640 1 -- 172.21.15.139:0/4211374856 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85c001c070 con 0x7f85d00b2d70 2024-04-25T09:53:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.821+0000 7f85b7fff640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f85bc06b5d0 0x7f85bc06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.821+0000 7f85b7fff640 1 -- 172.21.15.139:0/4211374856 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f85c00899c0 con 0x7f85d00b2d70 2024-04-25T09:53:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.819+0000 7fc53145d640 1 -- 172.21.15.139:0/4063297335 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c14b4c0 con 0x7fc52c0b25f0 2024-04-25T09:53:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.819+0000 7fc53145d640 1 -- 172.21.15.139:0/4063297335 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c14ba80 con 0x7fc52c0b25f0 2024-04-25T09:53:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.821+0000 7fc5137fe640 1 -- 172.21.15.139:0/4063297335 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc520017070 con 0x7fc52c0b25f0 2024-04-25T09:53:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.821+0000 7fc5137fe640 1 -- 172.21.15.139:0/4063297335 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520011b70 con 0x7fc52c0b25f0 2024-04-25T09:53:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.822+0000 7fc53145d640 1 -- 172.21.15.139:0/4063297335 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c005290 con 0x7fc52c0b25f0 2024-04-25T09:53:36.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.822+0000 7f85cf7fe640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f85bc06b5d0 0x7f85bc06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.823+0000 7f85cf7fe640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f85bc06b5d0 0x7f85bc06da90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f85d0071e00 tx=0x7f85c4006bd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.824+0000 7fc5137fe640 1 -- 172.21.15.139:0/4063297335 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc520011d10 con 0x7fc52c0b25f0 2024-04-25T09:53:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.825+0000 7fc5137fe640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc51406b6e0 0x7fc51406dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.825+0000 7fc5137fe640 1 -- 172.21.15.139:0/4063297335 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc52008a3e0 con 0x7fc52c0b25f0 2024-04-25T09:53:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.826+0000 7fc52a7fc640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc51406b6e0 0x7fc51406dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.826+0000 7fc52a7fc640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc51406b6e0 0x7fc51406dba0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc51c034000 tx=0x7fc51c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.828+0000 7f85b7fff640 1 -- 172.21.15.139:0/4211374856 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85c00586c0 con 0x7f85d00b2d70 2024-04-25T09:53:36.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.830+0000 7fc5137fe640 1 -- 172.21.15.139:0/4063297335 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc520058f00 con 0x7fc52c0b25f0 2024-04-25T09:53:36.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.838+0000 7f7556d6a640 1 Processor -- start 2024-04-25T09:53:36.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.838+0000 7f7556d6a640 1 -- start start 2024-04-25T09:53:36.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.839+0000 7f7556d6a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75500b25f0 0x7f755006ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f7556d6a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f755015ab20 0x7f755006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f7556d6a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755006f4d0 con 0x7f755015a030 2024-04-25T09:53:36.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f7556d6a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755006f760 con 0x7f755015ab20 2024-04-25T09:53:36.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f7556d6a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755006fa40 con 0x7f75500b25f0 2024-04-25T09:53:36.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.839+0000 7f543376a640 1 Processor -- start 2024-04-25T09:53:36.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f754ffff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75500b25f0 0x7f755006ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f754ffff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75500b25f0 0x7f755006ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:55060/0 (socket says 172.21.15.139:55060) 2024-04-25T09:53:36.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f754ffff640 1 -- 172.21.15.139:0/2877829099 learned_addr learned my addr 172.21.15.139:0/2877829099 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f7555307640 1 --2- 172.21.15.139:0/2877829099 >> v2:172.21.15.150:3300/0 conn(0x7f755015ab20 0x7f755006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f754ffff640 1 -- 172.21.15.139:0/2877829099 >> v2:172.21.15.150:3300/0 conn(0x7f755015ab20 msgr2=0x7f755006ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:36.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f754ffff640 1 --2- 172.21.15.139:0/2877829099 >> v2:172.21.15.150:3300/0 conn(0x7f755015ab20 0x7f755006ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f754ffff640 1 -- 172.21.15.139:0/2877829099 >> v1:172.21.15.150:6789/0 conn(0x7f755015a030 legacy=0x7f755015a410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:53:36.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f754ffff640 1 -- 172.21.15.139:0/2877829099 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75500cdc60 con 0x7f75500b25f0 2024-04-25T09:53:36.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.839+0000 7f543376a640 1 -- start start 2024-04-25T09:53:36.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f543376a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b25f0 0x7f542c162f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f543376a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f542c0711c0 0x7f542c071630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f543376a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c0b3a00 con 0x7f542c0b4240 2024-04-25T09:53:36.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f543376a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c159490 con 0x7f542c0711c0 2024-04-25T09:53:36.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f543376a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c071b70 con 0x7f542c0b25f0 2024-04-25T09:53:36.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f7555307640 1 --2- 172.21.15.139:0/2877829099 >> v2:172.21.15.150:3300/0 conn(0x7f755015ab20 0x7f755006ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:36.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f5430d05640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b25f0 0x7f542c162f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f5431d07640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f542c0711c0 0x7f542c071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.841+0000 7f5431d07640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f542c0711c0 0x7f542c071630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:36792/0 (socket says 172.21.15.139:36792) 2024-04-25T09:53:36.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.841+0000 7f5431d07640 1 -- 172.21.15.139:0/1076937798 learned_addr learned my addr 172.21.15.139:0/1076937798 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7f5430d05640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b25f0 0x7f542c162f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:55064/0 (socket says 172.21.15.139:55064) 2024-04-25T09:53:36.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.841+0000 7fb05203a640 1 Processor -- start 2024-04-25T09:53:36.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.841+0000 7f5431d07640 1 -- 172.21.15.139:0/1076937798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b25f0 msgr2=0x7f542c162f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:36.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.841+0000 7f5431d07640 1 --2- 172.21.15.139:0/1076937798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b25f0 0x7f542c162f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.841+0000 7f5431d07640 1 -- 172.21.15.139:0/1076937798 >> v1:172.21.15.150:6789/0 conn(0x7f542c0b4240 legacy=0x7f542c06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.841+0000 7f5431d07640 1 -- 172.21.15.139:0/1076937798 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c0b2ad0 con 0x7f542c0711c0 2024-04-25T09:53:36.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.840+0000 7fe77a215640 1 Processor -- start 2024-04-25T09:53:36.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.841+0000 7fe77a215640 1 -- start start 2024-04-25T09:53:36.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.841+0000 7fb05203a640 1 -- start start 2024-04-25T09:53:36.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.841+0000 7f754ffff640 1 --2- 172.21.15.139:0/2877829099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75500b25f0 0x7f755006ea50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7544009870 tx=0x7f754402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b0b6466a5ad1ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7f754f7fe640 1 -- 172.21.15.139:0/2877829099 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7544037070 con 0x7f75500b25f0 2024-04-25T09:53:36.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7f754f7fe640 1 -- 172.21.15.139:0/2877829099 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7544032440 con 0x7f75500b25f0 2024-04-25T09:53:36.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7fe77a215640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe77415c310 0x7fe7740719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7f754f7fe640 1 -- 172.21.15.139:0/2877829099 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7544032810 con 0x7f75500b25f0 2024-04-25T09:53:36.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7f5431d07640 1 --2- 172.21.15.139:0/1076937798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f542c0711c0 0x7f542c071630 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5424009e50 tx=0x7f542402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=724fe932ef703f3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7fe7737fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe77415c310 0x7fe7740719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7fe7737fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe77415c310 0x7fe7740719d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40244/0 (socket says 172.21.15.139:40244) 2024-04-25T09:53:36.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7fb05203a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb04c15fb30 0x7fb04c163140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7fb05203a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 0x7fb04c0ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7fb05203a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c166720 con 0x7fb04c165b40 2024-04-25T09:53:36.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7fb05203a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04c0b3240 con 0x7fb04c15fb30 2024-04-25T09:53:36.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7fb05203a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0cea10 con 0x7fb04c0ce070 2024-04-25T09:53:36.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7fe772ffd640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fe774071f10 0x7fe7740b2740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:44462/0 (socket says 172.21.15.139:44462) 2024-04-25T09:53:36.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.842+0000 7fe77a215640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7740b2e50 0x7fe7740ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7fe77a215640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7740b34b0 con 0x7fe774071f10 2024-04-25T09:53:36.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7fe77a215640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7740cde50 con 0x7fe77415c310 2024-04-25T09:53:36.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7fe77a215640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740cd800 con 0x7fe7740b2e50 2024-04-25T09:53:36.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7fe7737fe640 1 -- 172.21.15.139:0/108764082 learned_addr learned my addr 172.21.15.139:0/108764082 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7fb04affd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 0x7fb04c0ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7fb04affd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 0x7fb04c0ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:55072/0 (socket says 172.21.15.139:55072) 2024-04-25T09:53:36.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7fb04affd640 1 -- 172.21.15.139:0/3953017776 learned_addr learned my addr 172.21.15.139:0/3953017776 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7f5423fff640 1 -- 172.21.15.139:0/1076937798 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424036070 con 0x7f542c0711c0 2024-04-25T09:53:36.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7fe7737fe640 1 -- 172.21.15.139:0/108764082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7740b2e50 msgr2=0x7fe7740ce2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:53:36.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7fe7737fe640 1 --2- 172.21.15.139:0/108764082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7740b2e50 0x7fe7740ce2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7fe7737fe640 1 -- 172.21.15.139:0/108764082 >> v1:172.21.15.150:6789/0 conn(0x7fe774071f10 legacy=0x7fe7740b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7fe7737fe640 1 -- 172.21.15.139:0/108764082 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7740cdae0 con 0x7fe77415c310 2024-04-25T09:53:36.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7f7556d6a640 1 -- 172.21.15.139:0/2877829099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75500b25f0 msgr2=0x7f755006ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7f7556d6a640 1 --2- 172.21.15.139:0/2877829099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75500b25f0 0x7f755006ea50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7544009870 tx=0x7f754402a210 comp rx=0 tx=0).stop 2024-04-25T09:53:36.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7f5423fff640 1 -- 172.21.15.139:0/1076937798 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5424031600 con 0x7f542c0711c0 2024-04-25T09:53:36.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7f5423fff640 1 -- 172.21.15.139:0/1076937798 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54240319d0 con 0x7f542c0711c0 2024-04-25T09:53:36.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7f7556d6a640 1 -- 172.21.15.139:0/2877829099 shutdown_connections 2024-04-25T09:53:36.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7f7556d6a640 1 --2- 172.21.15.139:0/2877829099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75500b25f0 0x7f755006ea50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7f7556d6a640 1 --2- 172.21.15.139:0/2877829099 >> v2:172.21.15.150:3300/0 conn(0x7f755015ab20 0x7f755006ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.843+0000 7f7556d6a640 1 -- 172.21.15.139:0/2877829099 >> 172.21.15.139:0/2877829099 conn(0x7f755006af80 msgr2=0x7f755006b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:36.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.844+0000 7f7556d6a640 1 -- 172.21.15.139:0/2877829099 shutdown_connections 2024-04-25T09:53:36.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.844+0000 7fb04b7fe640 1 --2- 172.21.15.139:0/3953017776 >> v2:172.21.15.139:3300/0 conn(0x7fb04c15fb30 0x7fb04c163140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.844+0000 7fb04affd640 1 -- 172.21.15.139:0/3953017776 >> v2:172.21.15.139:3300/0 conn(0x7fb04c15fb30 msgr2=0x7fb04c163140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:36.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.844+0000 7fb04affd640 1 --2- 172.21.15.139:0/3953017776 >> v2:172.21.15.139:3300/0 conn(0x7fb04c15fb30 0x7fb04c163140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.844+0000 7fb04affd640 1 -- 172.21.15.139:0/3953017776 >> v1:172.21.15.150:6789/0 conn(0x7fb04c165b40 legacy=0x7fb04c0d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.844+0000 7fb04affd640 1 -- 172.21.15.139:0/3953017776 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c166a00 con 0x7fb04c0ce070 2024-04-25T09:53:36.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.844+0000 7fe7737fe640 1 --2- 172.21.15.139:0/108764082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe77415c310 0x7fe7740719d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe764009870 tx=0x7fe76402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9eb627cb17441a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.844+0000 7fe7727fc640 1 -- 172.21.15.139:0/108764082 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764037070 con 0x7fe77415c310 2024-04-25T09:53:36.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.844+0000 7f7556d6a640 1 -- 172.21.15.139:0/2877829099 wait complete. 2024-04-25T09:53:36.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.844+0000 7fe7727fc640 1 -- 172.21.15.139:0/108764082 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe764032440 con 0x7fe77415c310 2024-04-25T09:53:36.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.844+0000 7fe7727fc640 1 -- 172.21.15.139:0/108764082 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764032810 con 0x7fe77415c310 2024-04-25T09:53:36.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7f543376a640 1 -- 172.21.15.139:0/1076937798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f542c0711c0 msgr2=0x7f542c071630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7fb04b7fe640 1 --2- 172.21.15.139:0/3953017776 >> v2:172.21.15.139:3300/0 conn(0x7fb04c15fb30 0x7fb04c163140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:36.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7f543376a640 1 --2- 172.21.15.139:0/1076937798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f542c0711c0 0x7f542c071630 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5424009e50 tx=0x7f542402a710 comp rx=0 tx=0).stop 2024-04-25T09:53:36.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7f543376a640 1 -- 172.21.15.139:0/1076937798 shutdown_connections 2024-04-25T09:53:36.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7f543376a640 1 --2- 172.21.15.139:0/1076937798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b25f0 0x7f542c162f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7f543376a640 1 --2- 172.21.15.139:0/1076937798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f542c0711c0 0x7f542c071630 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7f543376a640 1 -- 172.21.15.139:0/1076937798 >> 172.21.15.139:0/1076937798 conn(0x7f542c067000 msgr2=0x7f542c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:36.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7fb04affd640 1 --2- 172.21.15.139:0/3953017776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 0x7fb04c0ce4d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009870 tx=0x7fb03c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ce3dfee55771be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7fb04a7fc640 1 -- 172.21.15.139:0/3953017776 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c037070 con 0x7fb04c0ce070 2024-04-25T09:53:36.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7fb04a7fc640 1 -- 172.21.15.139:0/3953017776 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c02ae10 con 0x7fb04c0ce070 2024-04-25T09:53:36.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7fb04a7fc640 1 -- 172.21.15.139:0/3953017776 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c032660 con 0x7fb04c0ce070 2024-04-25T09:53:36.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7f543376a640 1 -- 172.21.15.139:0/1076937798 shutdown_connections 2024-04-25T09:53:36.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7fe77a215640 1 -- 172.21.15.139:0/108764082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe77415c310 msgr2=0x7fe7740719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7fe77a215640 1 --2- 172.21.15.139:0/108764082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe77415c310 0x7fe7740719d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe764009870 tx=0x7fe76402a1f0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7f7556d6a640 1 Processor -- start 2024-04-25T09:53:36.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7f7556d6a640 1 -- start start 2024-04-25T09:53:36.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.845+0000 7f543376a640 1 -- 172.21.15.139:0/1076937798 wait complete. 2024-04-25T09:53:36.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7fe77a215640 1 -- 172.21.15.139:0/108764082 shutdown_connections 2024-04-25T09:53:36.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7fe77a215640 1 --2- 172.21.15.139:0/108764082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7740b2e50 0x7fe7740ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7fe77a215640 1 --2- 172.21.15.139:0/108764082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe77415c310 0x7fe7740719d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7fe77a215640 1 -- 172.21.15.139:0/108764082 >> 172.21.15.139:0/108764082 conn(0x7fe77406af80 msgr2=0x7fe77406b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:36.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7fe77a215640 1 -- 172.21.15.139:0/108764082 shutdown_connections 2024-04-25T09:53:36.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7fb05203a640 1 -- 172.21.15.139:0/3953017776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 msgr2=0x7fb04c0ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7fb05203a640 1 --2- 172.21.15.139:0/3953017776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 0x7fb04c0ce4d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009870 tx=0x7fb03c02a180 comp rx=0 tx=0).stop 2024-04-25T09:53:36.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7fb05203a640 1 -- 172.21.15.139:0/3953017776 shutdown_connections 2024-04-25T09:53:36.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7fb05203a640 1 --2- 172.21.15.139:0/3953017776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 0x7fb04c0ce4d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7fb05203a640 1 --2- 172.21.15.139:0/3953017776 >> v2:172.21.15.139:3300/0 conn(0x7fb04c15fb30 0x7fb04c163140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7fb05203a640 1 -- 172.21.15.139:0/3953017776 >> 172.21.15.139:0/3953017776 conn(0x7fb04c0d04a0 msgr2=0x7fb04c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:36.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.847+0000 7fb05203a640 1 -- 172.21.15.139:0/3953017776 shutdown_connections 2024-04-25T09:53:36.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.847+0000 7fb05203a640 1 -- 172.21.15.139:0/3953017776 wait complete. 2024-04-25T09:53:36.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.847+0000 7f543376a640 1 Processor -- start 2024-04-25T09:53:36.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.847+0000 7f7556d6a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75500b25f0 0x7f7550154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.847+0000 7f7556d6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f755015a030 0x7f7550155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.847+0000 7f7554b06640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75500b25f0 0x7f7550154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.848+0000 7f7554b06640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75500b25f0 0x7f7550154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:36804/0 (socket says 172.21.15.139:36804) 2024-04-25T09:53:36.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.848+0000 7f7554b06640 1 -- 172.21.15.139:0/473048403 learned_addr learned my addr 172.21.15.139:0/473048403 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.848+0000 7f7556d6a640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f755015ab20 0x7f75500b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.847+0000 7f543376a640 1 -- start start 2024-04-25T09:53:36.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.848+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7550154330 con 0x7f755015a030 2024-04-25T09:53:36.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.848+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75500cd910 con 0x7f75500b25f0 2024-04-25T09:53:36.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.846+0000 7fe77a215640 1 -- 172.21.15.139:0/108764082 wait complete. 2024-04-25T09:53:36.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.848+0000 7fe77a215640 1 Processor -- start 2024-04-25T09:53:36.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.848+0000 7f7555307640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f755015ab20 0x7f75500b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.848+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75501544b0 con 0x7f755015ab20 2024-04-25T09:53:36.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.848+0000 7f543376a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f542c0711c0 0x7f542c14aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.848+0000 7f543376a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f542c0b25f0 0x7f542c15f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.848+0000 7f543376a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b4240 0x7f542c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7f543376a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c06ec70 con 0x7f542c0711c0 2024-04-25T09:53:36.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.848+0000 7f754ffff640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f755015a030 0x7f7550155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7fb05203a640 1 Processor -- start 2024-04-25T09:53:36.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7f7554b06640 1 -- 172.21.15.139:0/473048403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f755015ab20 msgr2=0x7f75500b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7f543376a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0b3090 con 0x7f542c0b25f0 2024-04-25T09:53:36.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7f7554b06640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f755015ab20 0x7f75500b0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7f543376a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c15a5e0 con 0x7f542c0b4240 2024-04-25T09:53:36.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7f5431d07640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b4240 0x7f542c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7f5431d07640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b4240 0x7f542c0b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:55096/0 (socket says 172.21.15.139:55096) 2024-04-25T09:53:36.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7f5431d07640 1 -- 172.21.15.139:0/709934408 learned_addr learned my addr 172.21.15.139:0/709934408 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7f7554b06640 1 -- 172.21.15.139:0/473048403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f755015a030 msgr2=0x7f7550155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7f5430d05640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f542c0b25f0 0x7f542c15f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7f7554b06640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f755015a030 0x7f7550155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7f7554b06640 1 -- 172.21.15.139:0/473048403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7550154c60 con 0x7f75500b25f0 2024-04-25T09:53:36.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7fb05203a640 1 -- start start 2024-04-25T09:53:36.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7f5431d07640 1 -- 172.21.15.139:0/709934408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f542c0b25f0 msgr2=0x7f542c15f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7f5431d07640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f542c0b25f0 0x7f542c15f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7f5431d07640 1 -- 172.21.15.139:0/709934408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f542c0711c0 msgr2=0x7f542c14aa80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T09:53:36.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.849+0000 7fe77a215640 1 -- start start 2024-04-25T09:53:36.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7fe77a215640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe774071f10 0x7fe77415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7fe77a215640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7740b2e50 0x7fe774156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7f7554b06640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75500b25f0 0x7f7550154680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f753c00b450 tx=0x7f753c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7fe77a215640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe77415c310 0x7fe774160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7fe77a215640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77415c870 con 0x7fe7740b2e50 2024-04-25T09:53:36.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7fe77a215640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77415c6f0 con 0x7fe77415c310 2024-04-25T09:53:36.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7fe77a215640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe774163510 con 0x7fe774071f10 2024-04-25T09:53:36.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7f754d7fa640 1 -- 172.21.15.139:0/473048403 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c015020 con 0x7f75500b25f0 2024-04-25T09:53:36.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7fe772ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7740b2e50 0x7fe774156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7fe772ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7740b2e50 0x7fe774156fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40286/0 (socket says 172.21.15.139:40286) 2024-04-25T09:53:36.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7fe772ffd640 1 -- 172.21.15.139:0/1383800895 learned_addr learned my addr 172.21.15.139:0/1383800895 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7fe7737fe640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe774071f10 0x7fe77415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7fe772ffd640 1 -- 172.21.15.139:0/1383800895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe774071f10 msgr2=0x7fe77415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7fe772ffd640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe774071f10 0x7fe77415ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7fe772ffd640 1 -- 172.21.15.139:0/1383800895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe77415c310 msgr2=0x7fe774160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7fe772ffd640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe77415c310 0x7fe774160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7fe772ffd640 1 -- 172.21.15.139:0/1383800895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77415f5f0 con 0x7fe7740b2e50 2024-04-25T09:53:36.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7fe772ffd640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7740b2e50 0x7fe774156fb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00c790 tx=0x7fe76c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7fb05203a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 0x7fb04c06e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7fe76bfff640 1 -- 172.21.15.139:0/1383800895 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c017070 con 0x7fe7740b2e50 2024-04-25T09:53:36.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7550154f40 con 0x7f75500b25f0 2024-04-25T09:53:36.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.850+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7550155500 con 0x7f75500b25f0 2024-04-25T09:53:36.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7f754d7fa640 1 -- 172.21.15.139:0/473048403 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c0118b0 con 0x7f75500b25f0 2024-04-25T09:53:36.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7f754d7fa640 1 -- 172.21.15.139:0/473048403 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c011ca0 con 0x7f75500b25f0 2024-04-25T09:53:36.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7fe76bfff640 1 -- 172.21.15.139:0/1383800895 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76c00a640 con 0x7fe7740b2e50 2024-04-25T09:53:36.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7f5431506640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f542c0711c0 0x7f542c14aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.852+0000 7fb05203a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c15fb30 0x7fb04c06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.852+0000 7fb04b7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 0x7fb04c06e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.852+0000 7fb04b7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 0x7fb04c06e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:55118/0 (socket says 172.21.15.139:55118) 2024-04-25T09:53:36.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.852+0000 7fb04b7fe640 1 -- 172.21.15.139:0/22243047 learned_addr learned my addr 172.21.15.139:0/22243047 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:36.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.852+0000 7fb04affd640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c15fb30 0x7fb04c06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.852+0000 7fb05203a640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb04c165b40 0x7fb04c1ed6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.852+0000 7fb04bfff640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb04c165b40 0x7fb04c1ed6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.852+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c15ff10 con 0x7fb04c165b40 2024-04-25T09:53:36.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.853+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c162940 con 0x7fb04c15fb30 2024-04-25T09:53:36.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.853+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c165fb0 con 0x7fb04c0ce070 2024-04-25T09:53:36.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7fe77a215640 1 -- 172.21.15.139:0/1383800895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77415f930 con 0x7fe7740b2e50 2024-04-25T09:53:36.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.853+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7530005290 con 0x7f75500b25f0 2024-04-25T09:53:36.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.851+0000 7fe77a215640 1 -- 172.21.15.139:0/1383800895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe774159b40 con 0x7fe7740b2e50 2024-04-25T09:53:36.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.853+0000 7f5431d07640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f542c0711c0 0x7f542c14aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.853+0000 7f5431d07640 1 -- 172.21.15.139:0/709934408 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c159820 con 0x7f542c0b4240 2024-04-25T09:53:36.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.853+0000 7f5431506640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f542c0711c0 0x7f542c14aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:36.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.853+0000 7fe77a215640 1 -- 172.21.15.139:0/1383800895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe774159d40 con 0x7fe7740b2e50 2024-04-25T09:53:36.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.853+0000 7fe76bfff640 1 -- 172.21.15.139:0/1383800895 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00a9f0 con 0x7fe7740b2e50 2024-04-25T09:53:36.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.853+0000 7fb04bfff640 1 -- 172.21.15.139:0/22243047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 msgr2=0x7fb04c06e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.853+0000 7fb04bfff640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 0x7fb04c06e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.853+0000 7fb04bfff640 1 -- 172.21.15.139:0/22243047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c15fb30 msgr2=0x7fb04c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:36.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.853+0000 7fb04bfff640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c15fb30 0x7fb04c06e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:36.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.854+0000 7fb04bfff640 1 -- 172.21.15.139:0/22243047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c1ee3f0 con 0x7fb04c165b40 2024-04-25T09:53:36.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.854+0000 7fb04b7fe640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 0x7fb04c06e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:36.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.854+0000 7fb04bfff640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb04c165b40 0x7fb04c1ed6c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb04400caf0 tx=0x7fb044002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.854+0000 7fb033fff640 1 -- 172.21.15.139:0/22243047 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044017070 con 0x7fb04c165b40 2024-04-25T09:53:36.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.854+0000 7f5431d07640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b4240 0x7f542c0b12d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5424009f70 tx=0x7f5424004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.854+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c0703f0 con 0x7fb04c165b40 2024-04-25T09:53:36.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.854+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c14b880 con 0x7fb04c165b40 2024-04-25T09:53:36.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.854+0000 7fb033fff640 1 -- 172.21.15.139:0/22243047 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb044003b70 con 0x7fb04c165b40 2024-04-25T09:53:36.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.854+0000 7fb033fff640 1 -- 172.21.15.139:0/22243047 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400a760 con 0x7fb04c165b40 2024-04-25T09:53:36.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.854+0000 7f754d7fa640 1 -- 172.21.15.139:0/473048403 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f753c018020 con 0x7f75500b25f0 2024-04-25T09:53:36.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.855+0000 7f754d7fa640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f753406b5d0 0x7f753406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.855+0000 7f754d7fa640 1 -- 172.21.15.139:0/473048403 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f753c08b120 con 0x7f75500b25f0 2024-04-25T09:53:36.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.856+0000 7f754ffff640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f753406b5d0 0x7f753406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.855+0000 7f5421ffb640 1 -- 172.21.15.139:0/709934408 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424041020 con 0x7f542c0b4240 2024-04-25T09:53:36.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.855+0000 7f5421ffb640 1 -- 172.21.15.139:0/709934408 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5424008730 con 0x7f542c0b4240 2024-04-25T09:53:36.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.856+0000 7f5421ffb640 1 -- 172.21.15.139:0/709934408 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424036040 con 0x7f542c0b4240 2024-04-25T09:53:36.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.857+0000 7fb033fff640 1 -- 172.21.15.139:0/22243047 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb04401a070 con 0x7fb04c165b40 2024-04-25T09:53:36.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.857+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02c005290 con 0x7fb04c165b40 2024-04-25T09:53:36.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.857+0000 7fb033fff640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb03806b4d0 0x7fb03806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.855+0000 7fe76bfff640 1 -- 172.21.15.139:0/1383800895 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe76c01a070 con 0x7fe7740b2e50 2024-04-25T09:53:36.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.857+0000 7f754ffff640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f753406b5d0 0x7f753406da90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7544034000 tx=0x7f754402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.855+0000 7fe76bfff640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe75406b5d0 0x7fe75406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.856+0000 7fe76bfff640 1 -- 172.21.15.139:0/1383800895 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe76c089ae0 con 0x7fe7740b2e50 2024-04-25T09:53:36.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.856+0000 7fe7737fe640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe75406b5d0 0x7fe75406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.858+0000 7fe7737fe640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe75406b5d0 0x7fe75406da90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe764034000 tx=0x7fe76402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.858+0000 7f543376a640 1 -- 172.21.15.139:0/709934408 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c159b00 con 0x7f542c0b4240 2024-04-25T09:53:36.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.858+0000 7fb04b7fe640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb03806b4d0 0x7fb03806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.858+0000 7f543376a640 1 -- 172.21.15.139:0/709934408 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c15a0c0 con 0x7f542c0b4240 2024-04-25T09:53:36.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.859+0000 7fb04b7fe640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb03806b4d0 0x7fb03806d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb04000f290 tx=0x7fb040008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.859+0000 7fb033fff640 1 -- 172.21.15.139:0/22243047 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb044089b60 con 0x7fb04c165b40 2024-04-25T09:53:36.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.859+0000 7f754d7fa640 1 -- 172.21.15.139:0/473048403 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f753c01f020 con 0x7f75500b25f0 2024-04-25T09:53:36.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.859+0000 7f540f7fe640 1 -- 172.21.15.139:0/709934408 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c06baf0 con 0x7f542c0b4240 2024-04-25T09:53:36.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.860+0000 7f5421ffb640 1 -- 172.21.15.139:0/709934408 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5424044020 con 0x7f542c0b4240 2024-04-25T09:53:36.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.860+0000 7f5421ffb640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f541006b5a0 0x7f541006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:36.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.861+0000 7f5421ffb640 1 -- 172.21.15.139:0/709934408 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f54240ab130 con 0x7f542c0b4240 2024-04-25T09:53:36.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.861+0000 7f5431506640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f541006b5a0 0x7f541006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:36.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.861+0000 7f5431506640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f541006b5a0 0x7f541006da60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5428000f60 tx=0x7f5428009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:36.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.865+0000 7fe76bfff640 1 -- 172.21.15.139:0/1383800895 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe774159d40 con 0x7fe7740b2e50 2024-04-25T09:53:36.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.866+0000 7f5421ffb640 1 -- 172.21.15.139:0/709934408 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f542404b030 con 0x7f542c0b4240 2024-04-25T09:53:36.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:36.874+0000 7fb033fff640 1 -- 172.21.15.139:0/22243047 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb044057fb0 con 0x7fb04c165b40 2024-04-25T09:53:37.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.083+0000 7ff707e76640 1 -- 172.21.15.139:0/3854039778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff7000cd980 con 0x7ff70015d9e0 2024-04-25T09:53:37.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.083+0000 7ff6f67fc640 1 -- 172.21.15.139:0/3854039778 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff6f005bf60 con 0x7ff70015d9e0 2024-04-25T09:53:37.085 INFO:teuthology.orchestra.run.smithi139.stdout:55834574858 2024-04-25T09:53:37.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.087+0000 7ff6e3fff640 1 -- 172.21.15.139:0/3854039778 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff6e406b5a0 msgr2=0x7ff6e406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.087+0000 7ff6e3fff640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff6e406b5a0 0x7ff6e406da60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009c10 tx=0x7ff6fc008040 comp rx=0 tx=0).stop 2024-04-25T09:53:37.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.087+0000 7ff6e3fff640 1 -- 172.21.15.139:0/3854039778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff70015d9e0 msgr2=0x7ff700149620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.087+0000 7ff6e3fff640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff70015d9e0 0x7ff700149620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000c430 tx=0x7ff6f000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.088+0000 7ff6e3fff640 1 -- 172.21.15.139:0/3854039778 shutdown_connections 2024-04-25T09:53:37.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.088+0000 7ff6e3fff640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7001616d0 0x7ff7000ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.088+0000 7ff6e3fff640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff6e406b5a0 0x7ff6e406da60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.088+0000 7ff6e3fff640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff70015d9e0 0x7ff700149620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.088+0000 7ff6e3fff640 1 --2- 172.21.15.139:0/3854039778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7000b4960 0x7ff700158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.088+0000 7ff6e3fff640 1 -- 172.21.15.139:0/3854039778 >> 172.21.15.139:0/3854039778 conn(0x7ff7000cc820 msgr2=0x7ff7000cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.088+0000 7ff6e3fff640 1 -- 172.21.15.139:0/3854039778 shutdown_connections 2024-04-25T09:53:37.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.088+0000 7ff6e3fff640 1 -- 172.21.15.139:0/3854039778 wait complete. 2024-04-25T09:53:37.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-04-25T09:53:37.107 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:37.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.142+0000 7f3a42ffd640 1 -- 172.21.15.139:0/686780305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3a38005580 con 0x7f3a5003aa10 2024-04-25T09:53:37.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.143+0000 7f3a557fa640 1 -- 172.21.15.139:0/686780305 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a580c1cb0 con 0x7f3a5003aa10 2024-04-25T09:53:37.145 INFO:teuthology.orchestra.run.smithi139.stdout:55834574857 2024-04-25T09:53:37.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.146+0000 7f3a42ffd640 1 -- 172.21.15.139:0/686780305 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a4806ba40 msgr2=0x7f3a4806df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.146+0000 7f3a42ffd640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a4806ba40 0x7f3a4806df00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c007fd0 tx=0x7f3a4c009040 comp rx=0 tx=0).stop 2024-04-25T09:53:37.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.146+0000 7f3a42ffd640 1 -- 172.21.15.139:0/686780305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a5003aa10 msgr2=0x7f3a50037790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.146+0000 7f3a42ffd640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a5003aa10 0x7f3a50037790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a58068260 tx=0x7f3a580682c0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.146+0000 7f3a42ffd640 1 -- 172.21.15.139:0/686780305 shutdown_connections 2024-04-25T09:53:37.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.146+0000 7f3a42ffd640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a4806ba40 0x7f3a4806df00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.146+0000 7f3a42ffd640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a50048c30 0x7f3a50049a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.146+0000 7f3a42ffd640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a5003aa10 0x7f3a50037790 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.146+0000 7f3a42ffd640 1 --2- 172.21.15.139:0/686780305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a50009180 0x7f3a5017c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.146+0000 7f3a42ffd640 1 -- 172.21.15.139:0/686780305 >> 172.21.15.139:0/686780305 conn(0x7f3a5001ef30 msgr2=0x7f3a500e11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.146+0000 7f3a42ffd640 1 -- 172.21.15.139:0/686780305 shutdown_connections 2024-04-25T09:53:37.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.146+0000 7f3a42ffd640 1 -- 172.21.15.139:0/686780305 wait complete. 2024-04-25T09:53:37.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.159+0000 7fc53145d640 1 -- 172.21.15.139:0/4063297335 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc50c005580 con 0x7fc52c0b25f0 2024-04-25T09:53:37.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.160+0000 7fc5137fe640 1 -- 172.21.15.139:0/4063297335 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc520058800 con 0x7fc52c0b25f0 2024-04-25T09:53:37.162 INFO:teuthology.orchestra.run.smithi139.stdout:55834574857 2024-04-25T09:53:37.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.162+0000 7fc53145d640 1 -- 172.21.15.139:0/4063297335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc51406b6e0 msgr2=0x7fc51406dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.162+0000 7fc53145d640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc51406b6e0 0x7fc51406dba0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc51c034000 tx=0x7fc51c02e000 comp rx=0 tx=0).stop 2024-04-25T09:53:37.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.162+0000 7fc53145d640 1 -- 172.21.15.139:0/4063297335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c0b25f0 msgr2=0x7fc52c14ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.162+0000 7fc53145d640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c0b25f0 0x7fc52c14ff40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c430 tx=0x7fc52000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.163+0000 7fc53145d640 1 -- 172.21.15.139:0/4063297335 shutdown_connections 2024-04-25T09:53:37.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.163+0000 7fc53145d640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc52c15ab20 0x7fc52c1547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.163+0000 7fc53145d640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc51406b6e0 0x7fc51406dba0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.163+0000 7fc53145d640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc52c15a030 0x7fc52c14c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.163+0000 7fc53145d640 1 --2- 172.21.15.139:0/4063297335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc52c0b25f0 0x7fc52c14ff40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.163+0000 7fc53145d640 1 -- 172.21.15.139:0/4063297335 >> 172.21.15.139:0/4063297335 conn(0x7fc52c06af80 msgr2=0x7fc52c0ad940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.164+0000 7fc53145d640 1 -- 172.21.15.139:0/4063297335 shutdown_connections 2024-04-25T09:53:37.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.164+0000 7fc53145d640 1 -- 172.21.15.139:0/4063297335 wait complete. 2024-04-25T09:53:37.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-04-25T09:53:37.167 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:37.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-04-25T09:53:37.177 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:37.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.207+0000 7f540f7fe640 1 -- 172.21.15.139:0/709934408 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f542c074520 con 0x7f542c0b4240 2024-04-25T09:53:37.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.207+0000 7f5421ffb640 1 -- 172.21.15.139:0/709934408 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5424079c50 con 0x7f542c0b4240 2024-04-25T09:53:37.209 INFO:teuthology.orchestra.run.smithi139.stdout:55834574858 2024-04-25T09:53:37.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.210+0000 7f540f7fe640 1 -- 172.21.15.139:0/709934408 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f541006b5a0 msgr2=0x7f541006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.210+0000 7f540f7fe640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f541006b5a0 0x7f541006da60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5428000f60 tx=0x7f5428009040 comp rx=0 tx=0).stop 2024-04-25T09:53:37.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.210+0000 7f540f7fe640 1 -- 172.21.15.139:0/709934408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b4240 msgr2=0x7f542c0b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.210+0000 7f540f7fe640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b4240 0x7f542c0b12d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5424009f70 tx=0x7f5424004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:37.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.210+0000 7f540f7fe640 1 -- 172.21.15.139:0/709934408 shutdown_connections 2024-04-25T09:53:37.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.210+0000 7f540f7fe640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f541006b5a0 0x7f541006da60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.210+0000 7f540f7fe640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f542c0b4240 0x7f542c0b12d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.210+0000 7f540f7fe640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f542c0b25f0 0x7f542c15f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.210+0000 7f540f7fe640 1 --2- 172.21.15.139:0/709934408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f542c0711c0 0x7f542c14aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.210+0000 7f540f7fe640 1 -- 172.21.15.139:0/709934408 >> 172.21.15.139:0/709934408 conn(0x7f542c067000 msgr2=0x7f542c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.210+0000 7f540f7fe640 1 -- 172.21.15.139:0/709934408 shutdown_connections 2024-04-25T09:53:37.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.210+0000 7f540f7fe640 1 -- 172.21.15.139:0/709934408 wait complete. 2024-04-25T09:53:37.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.217+0000 7f85b5ffb640 1 -- 172.21.15.139:0/4211374856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f85b0005580 con 0x7f85d00b2d70 2024-04-25T09:53:37.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.218+0000 7f85b7fff640 1 -- 172.21.15.139:0/4211374856 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f85c0057fc0 con 0x7f85d00b2d70 2024-04-25T09:53:37.220 INFO:teuthology.orchestra.run.smithi139.stdout:55834574859 2024-04-25T09:53:37.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.220+0000 7f85b5ffb640 1 -- 172.21.15.139:0/4211374856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f85bc06b5d0 msgr2=0x7f85bc06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.220+0000 7f85b5ffb640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f85bc06b5d0 0x7f85bc06da90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f85d0071e00 tx=0x7f85c4006bd0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.220+0000 7f85b5ffb640 1 -- 172.21.15.139:0/4211374856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d00b2d70 msgr2=0x7f85d015c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.220+0000 7f85b5ffb640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d00b2d70 0x7f85d015c430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f85c000b700 tx=0x7f85c000bbc0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.220+0000 7f85b5ffb640 1 -- 172.21.15.139:0/4211374856 shutdown_connections 2024-04-25T09:53:37.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.220+0000 7f85b5ffb640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d00b2d70 0x7f85d015c430 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f85c000b700 tx=0x7f85c000bbc0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.220+0000 7f85b5ffb640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f85bc06b5d0 0x7f85bc06da90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.220+0000 7f85b5ffb640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d01f6900 0x7f85d015c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.220+0000 7f85b5ffb640 1 --2- 172.21.15.139:0/4211374856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85d00711c0 0x7f85d01579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.220+0000 7f85b5ffb640 1 -- 172.21.15.139:0/4211374856 >> 172.21.15.139:0/4211374856 conn(0x7f85d0067000 msgr2=0x7f85d006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.220+0000 7f85b5ffb640 1 -- 172.21.15.139:0/4211374856 shutdown_connections 2024-04-25T09:53:37.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.220+0000 7f85b5ffb640 1 -- 172.21.15.139:0/4211374856 wait complete. 2024-04-25T09:53:37.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-04-25T09:53:37.227 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:37.234 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-04-25T09:53:37.234 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:37.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.244+0000 7fe77a215640 1 -- 172.21.15.139:0/1383800895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe77406df90 con 0x7fe7740b2e50 2024-04-25T09:53:37.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.244+0000 7fe76bfff640 1 -- 172.21.15.139:0/1383800895 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe77415f930 con 0x7fe7740b2e50 2024-04-25T09:53:37.247 INFO:teuthology.orchestra.run.smithi139.stdout:55834574858 2024-04-25T09:53:37.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.246+0000 7fe77a215640 1 -- 172.21.15.139:0/1383800895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe75406b5d0 msgr2=0x7fe75406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.246+0000 7fe77a215640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe75406b5d0 0x7fe75406da90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe764034000 tx=0x7fe76402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:37.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.246+0000 7fe77a215640 1 -- 172.21.15.139:0/1383800895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7740b2e50 msgr2=0x7fe774156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.246+0000 7fe77a215640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7740b2e50 0x7fe774156fb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00c790 tx=0x7fe76c00cc50 comp rx=0 tx=0).stop 2024-04-25T09:53:37.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.246+0000 7fe77a215640 1 -- 172.21.15.139:0/1383800895 shutdown_connections 2024-04-25T09:53:37.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.246+0000 7fe77a215640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7740b2e50 0x7fe774156fb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.246+0000 7fe77a215640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe75406b5d0 0x7fe75406da90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.246+0000 7fe77a215640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe77415c310 0x7fe774160ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.246+0000 7fe77a215640 1 --2- 172.21.15.139:0/1383800895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe774071f10 0x7fe77415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.246+0000 7fe77a215640 1 -- 172.21.15.139:0/1383800895 >> 172.21.15.139:0/1383800895 conn(0x7fe77406af80 msgr2=0x7fe7740b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.246+0000 7fe77a215640 1 -- 172.21.15.139:0/1383800895 shutdown_connections 2024-04-25T09:53:37.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.246+0000 7fe77a215640 1 -- 172.21.15.139:0/1383800895 wait complete. 2024-04-25T09:53:37.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2024-04-25T09:53:37.259 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:37.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.275+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7530005850 con 0x7f75500b25f0 2024-04-25T09:53:37.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.275+0000 7f754d7fa640 1 -- 172.21.15.139:0/473048403 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f753c00d040 con 0x7f75500b25f0 2024-04-25T09:53:37.277 INFO:teuthology.orchestra.run.smithi139.stdout:55834574858 2024-04-25T09:53:37.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.277+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f753406b5d0 msgr2=0x7f753406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.277+0000 7f7556d6a640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f753406b5d0 0x7f753406da90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7544034000 tx=0x7f754402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:37.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.277+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75500b25f0 msgr2=0x7f7550154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.277+0000 7f7556d6a640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75500b25f0 0x7f7550154680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f753c00b450 tx=0x7f753c00b910 comp rx=0 tx=0).stop 2024-04-25T09:53:37.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.277+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 shutdown_connections 2024-04-25T09:53:37.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.277+0000 7f7556d6a640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f755015a030 0x7f7550155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.277+0000 7f7556d6a640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f753406b5d0 0x7f753406da90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.277+0000 7f7556d6a640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f755015ab20 0x7f75500b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.277+0000 7f7556d6a640 1 --2- 172.21.15.139:0/473048403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75500b25f0 0x7f7550154680 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.277+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 >> 172.21.15.139:0/473048403 conn(0x7f755006af80 msgr2=0x7f75500b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.277+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 shutdown_connections 2024-04-25T09:53:37.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.277+0000 7f7556d6a640 1 -- 172.21.15.139:0/473048403 wait complete. 2024-04-25T09:53:37.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.281+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb02c005580 con 0x7fb04c165b40 2024-04-25T09:53:37.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.281+0000 7fb033fff640 1 -- 172.21.15.139:0/22243047 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb04405c050 con 0x7fb04c165b40 2024-04-25T09:53:37.283 INFO:teuthology.orchestra.run.smithi139.stdout:55834574858 2024-04-25T09:53:37.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.283+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb03806b4d0 msgr2=0x7fb03806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.283+0000 7fb05203a640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb03806b4d0 0x7fb03806d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb04000f290 tx=0x7fb040008040 comp rx=0 tx=0).stop 2024-04-25T09:53:37.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.283+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb04c165b40 msgr2=0x7fb04c1ed6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.283+0000 7fb05203a640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb04c165b40 0x7fb04c1ed6c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb04400caf0 tx=0x7fb044002130 comp rx=0 tx=0).stop 2024-04-25T09:53:37.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.283+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 shutdown_connections 2024-04-25T09:53:37.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.283+0000 7fb05203a640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb04c165b40 0x7fb04c1ed6c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.283+0000 7fb05203a640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb03806b4d0 0x7fb03806d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.283+0000 7fb05203a640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb04c15fb30 0x7fb04c06e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.283+0000 7fb05203a640 1 --2- 172.21.15.139:0/22243047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb04c0ce070 0x7fb04c06e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.283+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 >> 172.21.15.139:0/22243047 conn(0x7fb04c0d04a0 msgr2=0x7fb04c0b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.283+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 shutdown_connections 2024-04-25T09:53:37.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.283+0000 7fb05203a640 1 -- 172.21.15.139:0/22243047 wait complete. 2024-04-25T09:53:37.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2024-04-25T09:53:37.291 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:37.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-04-25T09:53:37.297 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:37.297 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T09:53:37.297 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:53:37.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.431+0000 7efce5f5c640 1 Processor -- start 2024-04-25T09:53:37.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.431+0000 7efce5f5c640 1 -- start start 2024-04-25T09:53:37.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.432+0000 7efce5f5c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efce00b6af0 0x7efce006c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.432+0000 7efce5f5c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efce006c5b0 0x7efce00d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.432+0000 7efce5f5c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce00cf110 con 0x7efce01611e0 2024-04-25T09:53:37.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.432+0000 7efce5f5c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce00d1fa0 con 0x7efce00b6af0 2024-04-25T09:53:37.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.432+0000 7efce5f5c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce00d2560 con 0x7efce006c5b0 2024-04-25T09:53:37.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.432+0000 7efcdf7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7efce01611e0 0x7efce00cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48594/0 (socket says 172.21.15.139:48594) 2024-04-25T09:53:37.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.432+0000 7efcdeffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efce00b6af0 0x7efce006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.432+0000 7efcdf7fe640 1 -- 172.21.15.139:0/508552269 learned_addr learned my addr 172.21.15.139:0/508552269 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:37.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.432+0000 7efcdffff640 1 --2- 172.21.15.139:0/508552269 >> v2:172.21.15.150:3300/0 conn(0x7efce006c5b0 0x7efce00d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.432+0000 7efcdeffd640 1 -- 172.21.15.139:0/508552269 >> v2:172.21.15.150:3300/0 conn(0x7efce006c5b0 msgr2=0x7efce00d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:37.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.432+0000 7efcdeffd640 1 --2- 172.21.15.139:0/508552269 >> v2:172.21.15.150:3300/0 conn(0x7efce006c5b0 0x7efce00d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.432+0000 7efcdeffd640 1 -- 172.21.15.139:0/508552269 >> v1:172.21.15.139:6789/0 conn(0x7efce01611e0 legacy=0x7efce00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.433+0000 7efcdeffd640 1 -- 172.21.15.139:0/508552269 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00b6f50 con 0x7efce00b6af0 2024-04-25T09:53:37.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.433+0000 7efcde7fc640 1 -- 172.21.15.139:0/508552269 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460107604 0 0) 0x7efce00cf110 con 0x7efce01611e0 2024-04-25T09:53:37.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.433+0000 7efcdffff640 1 --2- 172.21.15.139:0/508552269 >> v2:172.21.15.150:3300/0 conn(0x7efce006c5b0 0x7efce00d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:37.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.433+0000 7efcdeffd640 1 --2- 172.21.15.139:0/508552269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce00b6af0 0x7efce006c070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efce01629e0 tx=0x7efcd402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2749aa368d65c2da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:37.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.433+0000 7efcde7fc640 1 -- 172.21.15.139:0/508552269 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4037070 con 0x7efce00b6af0 2024-04-25T09:53:37.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.433+0000 7efcde7fc640 1 -- 172.21.15.139:0/508552269 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd4032440 con 0x7efce00b6af0 2024-04-25T09:53:37.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.433+0000 7efcde7fc640 1 -- 172.21.15.139:0/508552269 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4032810 con 0x7efce00b6af0 2024-04-25T09:53:37.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.434+0000 7efce5f5c640 1 -- 172.21.15.139:0/508552269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce00b6af0 msgr2=0x7efce006c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.434+0000 7efce5f5c640 1 --2- 172.21.15.139:0/508552269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce00b6af0 0x7efce006c070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efce01629e0 tx=0x7efcd402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.434+0000 7efce5f5c640 1 -- 172.21.15.139:0/508552269 shutdown_connections 2024-04-25T09:53:37.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.434+0000 7efce5f5c640 1 --2- 172.21.15.139:0/508552269 >> v2:172.21.15.150:3300/0 conn(0x7efce006c5b0 0x7efce00d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.434+0000 7efce5f5c640 1 --2- 172.21.15.139:0/508552269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce00b6af0 0x7efce006c070 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.434+0000 7efce5f5c640 1 -- 172.21.15.139:0/508552269 >> 172.21.15.139:0/508552269 conn(0x7efce00b5b00 msgr2=0x7efce00d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.434+0000 7efce5f5c640 1 -- 172.21.15.139:0/508552269 shutdown_connections 2024-04-25T09:53:37.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.434+0000 7efce5f5c640 1 -- 172.21.15.139:0/508552269 wait complete. 2024-04-25T09:53:37.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.435+0000 7efce5f5c640 1 Processor -- start 2024-04-25T09:53:37.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.435+0000 7efce5f5c640 1 -- start start 2024-04-25T09:53:37.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efce5f5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce006c5b0 0x7efce00b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efce5f5c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efce00b6af0 0x7efce014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efce5f5c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efce01611e0 0x7efce00c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efce5f5c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00b2a50 con 0x7efce006c5b0 2024-04-25T09:53:37.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efce5f5c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00b2770 con 0x7efce01611e0 2024-04-25T09:53:37.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efce5f5c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00b2d30 con 0x7efce00b6af0 2024-04-25T09:53:37.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efcdf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce006c5b0 0x7efce00b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efcdf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce006c5b0 0x7efce00b1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40294/0 (socket says 172.21.15.139:40294) 2024-04-25T09:53:37.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efcdf7fe640 1 -- 172.21.15.139:0/532341912 learned_addr learned my addr 172.21.15.139:0/532341912 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:37.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efcdf7fe640 1 -- 172.21.15.139:0/532341912 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efce00b6af0 msgr2=0x7efce014d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efcdf7fe640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efce00b6af0 0x7efce014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efcdf7fe640 1 -- 172.21.15.139:0/532341912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efce01611e0 msgr2=0x7efce00c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efcdffff640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efce01611e0 0x7efce00c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efcdf7fe640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efce01611e0 0x7efce00c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efcdf7fe640 1 -- 172.21.15.139:0/532341912 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce014db70 con 0x7efce006c5b0 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.436+0000 7efcdffff640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efce01611e0 0x7efce00c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.437+0000 7efcdf7fe640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce006c5b0 0x7efce00b1df0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efcd000c430 tx=0x7efcd000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.437+0000 7efcc7fff640 1 -- 172.21.15.139:0/532341912 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd00074e0 con 0x7efce006c5b0 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.437+0000 7efcc7fff640 1 -- 172.21.15.139:0/532341912 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd00047e0 con 0x7efce006c5b0 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.437+0000 7efce5f5c640 1 -- 172.21.15.139:0/532341912 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce015d0c0 con 0x7efce006c5b0 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.437+0000 7efcc7fff640 1 -- 172.21.15.139:0/532341912 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0004bd0 con 0x7efce006c5b0 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.437+0000 7efce5f5c640 1 -- 172.21.15.139:0/532341912 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce015d680 con 0x7efce006c5b0 2024-04-25T09:53:37.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.438+0000 7efce5f5c640 1 -- 172.21.15.139:0/532341912 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce0074520 con 0x7efce006c5b0 2024-04-25T09:53:37.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.438+0000 7efcc7fff640 1 -- 172.21.15.139:0/532341912 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efcd0011070 con 0x7efce006c5b0 2024-04-25T09:53:37.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.438+0000 7efcc7fff640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efccc06b4d0 0x7efccc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.439+0000 7efcc7fff640 1 -- 172.21.15.139:0/532341912 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efcd008a1c0 con 0x7efce006c5b0 2024-04-25T09:53:37.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.439+0000 7efcdeffd640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efccc06b4d0 0x7efccc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.439+0000 7efcdeffd640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efccc06b4d0 0x7efccc06d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efcd4034000 tx=0x7efcd402c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:37.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.443+0000 7efcc7fff640 1 -- 172.21.15.139:0/532341912 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efcd0058d90 con 0x7efce006c5b0 2024-04-25T09:53:37.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.612+0000 7efce5f5c640 1 -- 172.21.15.139:0/532341912 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efce006ceb0 con 0x7efccc06b4d0 2024-04-25T09:53:37.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.620+0000 7efcc7fff640 1 -- 172.21.15.139:0/532341912 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7efce006ceb0 con 0x7efccc06b4d0 2024-04-25T09:53:37.625 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:37.627 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:53:37.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.623+0000 7efce5f5c640 1 -- 172.21.15.139:0/532341912 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efccc06b4d0 msgr2=0x7efccc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.623+0000 7efce5f5c640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efccc06b4d0 0x7efccc06d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efcd4034000 tx=0x7efcd402c000 comp rx=0 tx=0).stop 2024-04-25T09:53:37.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.623+0000 7efce5f5c640 1 -- 172.21.15.139:0/532341912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce006c5b0 msgr2=0x7efce00b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.623+0000 7efce5f5c640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce006c5b0 0x7efce00b1df0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efcd000c430 tx=0x7efcd000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.623+0000 7efce5f5c640 1 -- 172.21.15.139:0/532341912 shutdown_connections 2024-04-25T09:53:37.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.623+0000 7efce5f5c640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efccc06b4d0 0x7efccc06d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.623+0000 7efce5f5c640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efce01611e0 0x7efce00c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.623+0000 7efce5f5c640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efce00b6af0 0x7efce014d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.623+0000 7efce5f5c640 1 --2- 172.21.15.139:0/532341912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efce006c5b0 0x7efce00b1df0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.623+0000 7efce5f5c640 1 -- 172.21.15.139:0/532341912 >> 172.21.15.139:0/532341912 conn(0x7efce00b5b00 msgr2=0x7efce00cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.623+0000 7efce5f5c640 1 -- 172.21.15.139:0/532341912 shutdown_connections 2024-04-25T09:53:37.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.623+0000 7efce5f5c640 1 -- 172.21.15.139:0/532341912 wait complete. 2024-04-25T09:53:37.637 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-25T09:53:36.971686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002675"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302246+0000","last_change":"2024-04-25T09:53:14.712100+0000","last_active":"2024-04-25T09:53:21.302246+0000","last_peered":"2024-04-25T09:53:21.302246+0000","last_clean":"2024-04-25T09:53:21.302246+0000","last_became_active":"2024-04-25T09:53:14.711827+0000","last_became_peered":"2024-04-25T09:53:14.711827+0000","last_unstale":"2024-04-25T09:53:21.302246+0000","last_undegraded":"2024-04-25T09:53:21.302246+0000","last_fullsized":"2024-04-25T09:53:21.302246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:57.295010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302544+0000","last_change":"2024-04-25T09:53:15.723415+0000","last_active":"2024-04-25T09:53:21.302544+0000","last_peered":"2024-04-25T09:53:21.302544+0000","last_clean":"2024-04-25T09:53:21.302544+0000","last_became_active":"2024-04-25T09:53:15.723126+0000","last_became_peered":"2024-04-25T09:53:15.723126+0000","last_unstale":"2024-04-25T09:53:21.302544+0000","last_undegraded":"2024-04-25T09:53:21.302544+0000","last_fullsized":"2024-04-25T09:53:21.302544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:53.040580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303119+0000","last_change":"2024-04-25T09:53:19.938169+0000","last_active":"2024-04-25T09:53:21.303119+0000","last_peered":"2024-04-25T09:53:21.303119+0000","last_clean":"2024-04-25T09:53:21.303119+0000","last_became_active":"2024-04-25T09:53:19.937831+0000","last_became_peered":"2024-04-25T09:53:19.937831+0000","last_unstale":"2024-04-25T09:53:21.303119+0000","last_undegraded":"2024-04-25T09:53:21.303119+0000","last_fullsized":"2024-04-25T09:53:21.303119+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:05.087194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302754+0000","last_change":"2024-04-25T09:53:14.708452+0000","last_active":"2024-04-25T09:53:21.302754+0000","last_peered":"2024-04-25T09:53:21.302754+0000","last_clean":"2024-04-25T09:53:21.302754+0000","last_became_active":"2024-04-25T09:53:14.708168+0000","last_became_peered":"2024-04-25T09:53:14.708168+0000","last_unstale":"2024-04-25T09:53:21.302754+0000","last_undegraded":"2024-04-25T09:53:21.302754+0000","last_fullsized":"2024-04-25T09:53:21.302754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:08.699272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301824+0000","last_change":"2024-04-25T09:53:15.731605+0000","last_active":"2024-04-25T09:53:21.301824+0000","last_peered":"2024-04-25T09:53:21.301824+0000","last_clean":"2024-04-25T09:53:21.301824+0000","last_became_active":"2024-04-25T09:53:15.731254+0000","last_became_peered":"2024-04-25T09:53:15.731254+0000","last_unstale":"2024-04-25T09:53:21.301824+0000","last_undegraded":"2024-04-25T09:53:21.301824+0000","last_fullsized":"2024-04-25T09:53:21.301824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742162+0000","last_change":"2024-04-25T09:53:19.756305+0000","last_active":"2024-04-25T09:53:20.742162+0000","last_peered":"2024-04-25T09:53:20.742162+0000","last_clean":"2024-04-25T09:53:20.742162+0000","last_became_active":"2024-04-25T09:53:19.755760+0000","last_became_peered":"2024-04-25T09:53:19.755760+0000","last_unstale":"2024-04-25T09:53:20.742162+0000","last_undegraded":"2024-04-25T09:53:20.742162+0000","last_fullsized":"2024-04-25T09:53:20.742162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:45.103063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303011+0000","last_change":"2024-04-25T09:53:14.713424+0000","last_active":"2024-04-25T09:53:21.303011+0000","last_peered":"2024-04-25T09:53:21.303011+0000","last_clean":"2024-04-25T09:53:21.303011+0000","last_became_active":"2024-04-25T09:53:14.711564+0000","last_became_peered":"2024-04-25T09:53:14.711564+0000","last_unstale":"2024-04-25T09:53:21.303011+0000","last_undegraded":"2024-04-25T09:53:21.303011+0000","last_fullsized":"2024-04-25T09:53:21.303011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:02.525834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302665+0000","last_change":"2024-04-25T09:53:15.737042+0000","last_active":"2024-04-25T09:53:21.302665+0000","last_peered":"2024-04-25T09:53:21.302665+0000","last_clean":"2024-04-25T09:53:21.302665+0000","last_became_active":"2024-04-25T09:53:15.736770+0000","last_became_peered":"2024-04-25T09:53:15.736770+0000","last_unstale":"2024-04-25T09:53:21.302665+0000","last_undegraded":"2024-04-25T09:53:21.302665+0000","last_fullsized":"2024-04-25T09:53:21.302665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:57.316572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742503+0000","last_change":"2024-04-25T09:53:19.761920+0000","last_active":"2024-04-25T09:53:20.742503+0000","last_peered":"2024-04-25T09:53:20.742503+0000","last_clean":"2024-04-25T09:53:20.742503+0000","last_became_active":"2024-04-25T09:53:19.761083+0000","last_became_peered":"2024-04-25T09:53:19.761083+0000","last_unstale":"2024-04-25T09:53:20.742503+0000","last_undegraded":"2024-04-25T09:53:20.742503+0000","last_fullsized":"2024-04-25T09:53:20.742503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:57.080146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301800+0000","last_change":"2024-04-25T09:53:14.713667+0000","last_active":"2024-04-25T09:53:21.301800+0000","last_peered":"2024-04-25T09:53:21.301800+0000","last_clean":"2024-04-25T09:53:21.301800+0000","last_became_active":"2024-04-25T09:53:14.713327+0000","last_became_peered":"2024-04-25T09:53:14.713327+0000","last_unstale":"2024-04-25T09:53:21.301800+0000","last_undegraded":"2024-04-25T09:53:21.301800+0000","last_fullsized":"2024-04-25T09:53:21.301800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:26.183785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302970+0000","last_change":"2024-04-25T09:53:15.737782+0000","last_active":"2024-04-25T09:53:21.302970+0000","last_peered":"2024-04-25T09:53:21.302970+0000","last_clean":"2024-04-25T09:53:21.302970+0000","last_became_active":"2024-04-25T09:53:15.737531+0000","last_became_peered":"2024-04-25T09:53:15.737531+0000","last_unstale":"2024-04-25T09:53:21.302970+0000","last_undegraded":"2024-04-25T09:53:21.302970+0000","last_fullsized":"2024-04-25T09:53:21.302970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:55.029099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742249+0000","last_change":"2024-04-25T09:53:19.754738+0000","last_active":"2024-04-25T09:53:20.742249+0000","last_peered":"2024-04-25T09:53:20.742249+0000","last_clean":"2024-04-25T09:53:20.742249+0000","last_became_active":"2024-04-25T09:53:19.754248+0000","last_became_peered":"2024-04-25T09:53:19.754248+0000","last_unstale":"2024-04-25T09:53:20.742249+0000","last_undegraded":"2024-04-25T09:53:20.742249+0000","last_fullsized":"2024-04-25T09:53:20.742249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:55.483582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302991+0000","last_change":"2024-04-25T09:53:15.729322+0000","last_active":"2024-04-25T09:53:21.302991+0000","last_peered":"2024-04-25T09:53:21.302991+0000","last_clean":"2024-04-25T09:53:21.302991+0000","last_became_active":"2024-04-25T09:53:15.729037+0000","last_became_peered":"2024-04-25T09:53:15.729037+0000","last_unstale":"2024-04-25T09:53:21.302991+0000","last_undegraded":"2024-04-25T09:53:21.302991+0000","last_fullsized":"2024-04-25T09:53:21.302991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302405+0000","last_change":"2024-04-25T09:53:14.712891+0000","last_active":"2024-04-25T09:53:21.302405+0000","last_peered":"2024-04-25T09:53:21.302405+0000","last_clean":"2024-04-25T09:53:21.302405+0000","last_became_active":"2024-04-25T09:53:14.712375+0000","last_became_peered":"2024-04-25T09:53:14.712375+0000","last_unstale":"2024-04-25T09:53:21.302405+0000","last_undegraded":"2024-04-25T09:53:21.302405+0000","last_fullsized":"2024-04-25T09:53:21.302405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:58.733069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743015+0000","last_change":"2024-04-25T09:53:19.762022+0000","last_active":"2024-04-25T09:53:20.743015+0000","last_peered":"2024-04-25T09:53:20.743015+0000","last_clean":"2024-04-25T09:53:20.743015+0000","last_became_active":"2024-04-25T09:53:19.761624+0000","last_became_peered":"2024-04-25T09:53:19.761624+0000","last_unstale":"2024-04-25T09:53:20.743015+0000","last_undegraded":"2024-04-25T09:53:20.743015+0000","last_fullsized":"2024-04-25T09:53:20.743015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:01.356138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302197+0000","last_change":"2024-04-25T09:53:15.720406+0000","last_active":"2024-04-25T09:53:21.302197+0000","last_peered":"2024-04-25T09:53:21.302197+0000","last_clean":"2024-04-25T09:53:21.302197+0000","last_became_active":"2024-04-25T09:53:15.720140+0000","last_became_peered":"2024-04-25T09:53:15.720140+0000","last_unstale":"2024-04-25T09:53:21.302197+0000","last_undegraded":"2024-04-25T09:53:21.302197+0000","last_fullsized":"2024-04-25T09:53:21.302197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302882+0000","last_change":"2024-04-25T09:53:14.705015+0000","last_active":"2024-04-25T09:53:21.302882+0000","last_peered":"2024-04-25T09:53:21.302882+0000","last_clean":"2024-04-25T09:53:21.302882+0000","last_became_active":"2024-04-25T09:53:14.704705+0000","last_became_peered":"2024-04-25T09:53:14.704705+0000","last_unstale":"2024-04-25T09:53:21.302882+0000","last_undegraded":"2024-04-25T09:53:21.302882+0000","last_fullsized":"2024-04-25T09:53:21.302882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743231+0000","last_change":"2024-04-25T09:53:19.761275+0000","last_active":"2024-04-25T09:53:20.743231+0000","last_peered":"2024-04-25T09:53:20.743231+0000","last_clean":"2024-04-25T09:53:20.743231+0000","last_became_active":"2024-04-25T09:53:19.760859+0000","last_became_peered":"2024-04-25T09:53:19.760859+0000","last_unstale":"2024-04-25T09:53:20.743231+0000","last_undegraded":"2024-04-25T09:53:20.743231+0000","last_fullsized":"2024-04-25T09:53:20.743231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:20.608892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301802+0000","last_change":"2024-04-25T09:53:15.726544+0000","last_active":"2024-04-25T09:53:21.301802+0000","last_peered":"2024-04-25T09:53:21.301802+0000","last_clean":"2024-04-25T09:53:21.301802+0000","last_became_active":"2024-04-25T09:53:15.726267+0000","last_became_peered":"2024-04-25T09:53:15.726267+0000","last_unstale":"2024-04-25T09:53:21.301802+0000","last_undegraded":"2024-04-25T09:53:21.301802+0000","last_fullsized":"2024-04-25T09:53:21.301802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301803+0000","last_change":"2024-04-25T09:53:14.712292+0000","last_active":"2024-04-25T09:53:21.301803+0000","last_peered":"2024-04-25T09:53:21.301803+0000","last_clean":"2024-04-25T09:53:21.301803+0000","last_became_active":"2024-04-25T09:53:14.711702+0000","last_became_peered":"2024-04-25T09:53:14.711702+0000","last_unstale":"2024-04-25T09:53:21.301803+0000","last_undegraded":"2024-04-25T09:53:21.301803+0000","last_fullsized":"2024-04-25T09:53:21.301803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303737+0000","last_change":"2024-04-25T09:53:19.937513+0000","last_active":"2024-04-25T09:53:21.303737+0000","last_peered":"2024-04-25T09:53:21.303737+0000","last_clean":"2024-04-25T09:53:21.303737+0000","last_became_active":"2024-04-25T09:53:19.937076+0000","last_became_peered":"2024-04-25T09:53:19.937076+0000","last_unstale":"2024-04-25T09:53:21.303737+0000","last_undegraded":"2024-04-25T09:53:21.303737+0000","last_fullsized":"2024-04-25T09:53:21.303737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:50.372524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742399+0000","last_change":"2024-04-25T09:53:15.717251+0000","last_active":"2024-04-25T09:53:20.742399+0000","last_peered":"2024-04-25T09:53:20.742399+0000","last_clean":"2024-04-25T09:53:20.742399+0000","last_became_active":"2024-04-25T09:53:15.716981+0000","last_became_peered":"2024-04-25T09:53:15.716981+0000","last_unstale":"2024-04-25T09:53:20.742399+0000","last_undegraded":"2024-04-25T09:53:20.742399+0000","last_fullsized":"2024-04-25T09:53:20.742399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:08.082240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301576+0000","last_change":"2024-04-25T09:53:14.711486+0000","last_active":"2024-04-25T09:53:21.301576+0000","last_peered":"2024-04-25T09:53:21.301576+0000","last_clean":"2024-04-25T09:53:21.301576+0000","last_became_active":"2024-04-25T09:53:14.711090+0000","last_became_peered":"2024-04-25T09:53:14.711090+0000","last_unstale":"2024-04-25T09:53:21.301576+0000","last_undegraded":"2024-04-25T09:53:21.301576+0000","last_fullsized":"2024-04-25T09:53:21.301576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742532+0000","last_change":"2024-04-25T09:53:19.756221+0000","last_active":"2024-04-25T09:53:20.742532+0000","last_peered":"2024-04-25T09:53:20.742532+0000","last_clean":"2024-04-25T09:53:20.742532+0000","last_became_active":"2024-04-25T09:53:19.755565+0000","last_became_peered":"2024-04-25T09:53:19.755565+0000","last_unstale":"2024-04-25T09:53:20.742532+0000","last_undegraded":"2024-04-25T09:53:20.742532+0000","last_fullsized":"2024-04-25T09:53:20.742532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:03.262744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302070+0000","last_change":"2024-04-25T09:53:15.720035+0000","last_active":"2024-04-25T09:53:21.302070+0000","last_peered":"2024-04-25T09:53:21.302070+0000","last_clean":"2024-04-25T09:53:21.302070+0000","last_became_active":"2024-04-25T09:53:15.719506+0000","last_became_peered":"2024-04-25T09:53:15.719506+0000","last_unstale":"2024-04-25T09:53:21.302070+0000","last_undegraded":"2024-04-25T09:53:21.302070+0000","last_fullsized":"2024-04-25T09:53:21.302070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303882+0000","last_change":"2024-04-25T09:53:14.714287+0000","last_active":"2024-04-25T09:53:21.303882+0000","last_peered":"2024-04-25T09:53:21.303882+0000","last_clean":"2024-04-25T09:53:21.303882+0000","last_became_active":"2024-04-25T09:53:14.714026+0000","last_became_peered":"2024-04-25T09:53:14.714026+0000","last_unstale":"2024-04-25T09:53:21.303882+0000","last_undegraded":"2024-04-25T09:53:21.303882+0000","last_fullsized":"2024-04-25T09:53:21.303882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742315+0000","last_change":"2024-04-25T09:53:19.934699+0000","last_active":"2024-04-25T09:53:20.742315+0000","last_peered":"2024-04-25T09:53:20.742315+0000","last_clean":"2024-04-25T09:53:20.742315+0000","last_became_active":"2024-04-25T09:53:19.934296+0000","last_became_peered":"2024-04-25T09:53:19.934296+0000","last_unstale":"2024-04-25T09:53:20.742315+0000","last_undegraded":"2024-04-25T09:53:20.742315+0000","last_fullsized":"2024-04-25T09:53:20.742315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:04.247738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302840+0000","last_change":"2024-04-25T09:53:15.735174+0000","last_active":"2024-04-25T09:53:21.302840+0000","last_peered":"2024-04-25T09:53:21.302840+0000","last_clean":"2024-04-25T09:53:21.302840+0000","last_became_active":"2024-04-25T09:53:15.734811+0000","last_became_peered":"2024-04-25T09:53:15.734811+0000","last_unstale":"2024-04-25T09:53:21.302840+0000","last_undegraded":"2024-04-25T09:53:21.302840+0000","last_fullsized":"2024-04-25T09:53:21.302840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:53.172005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303710+0000","last_change":"2024-04-25T09:53:14.707243+0000","last_active":"2024-04-25T09:53:21.303710+0000","last_peered":"2024-04-25T09:53:21.303710+0000","last_clean":"2024-04-25T09:53:21.303710+0000","last_became_active":"2024-04-25T09:53:14.706608+0000","last_became_peered":"2024-04-25T09:53:14.706608+0000","last_unstale":"2024-04-25T09:53:21.303710+0000","last_undegraded":"2024-04-25T09:53:21.303710+0000","last_fullsized":"2024-04-25T09:53:21.303710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:49.257995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301370+0000","last_change":"2024-04-25T09:53:19.938687+0000","last_active":"2024-04-25T09:53:21.301370+0000","last_peered":"2024-04-25T09:53:21.301370+0000","last_clean":"2024-04-25T09:53:21.301370+0000","last_became_active":"2024-04-25T09:53:19.938312+0000","last_became_peered":"2024-04-25T09:53:19.938312+0000","last_unstale":"2024-04-25T09:53:21.301370+0000","last_undegraded":"2024-04-25T09:53:21.301370+0000","last_fullsized":"2024-04-25T09:53:21.301370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:11.692337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302904+0000","last_change":"2024-04-25T09:53:15.724325+0000","last_active":"2024-04-25T09:53:21.302904+0000","last_peered":"2024-04-25T09:53:21.302904+0000","last_clean":"2024-04-25T09:53:21.302904+0000","last_became_active":"2024-04-25T09:53:15.724024+0000","last_became_peered":"2024-04-25T09:53:15.724024+0000","last_unstale":"2024-04-25T09:53:21.302904+0000","last_undegraded":"2024-04-25T09:53:21.302904+0000","last_fullsized":"2024-04-25T09:53:21.302904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303201+0000","last_change":"2024-04-25T09:53:14.712119+0000","last_active":"2024-04-25T09:53:21.303201+0000","last_peered":"2024-04-25T09:53:21.303201+0000","last_clean":"2024-04-25T09:53:21.303201+0000","last_became_active":"2024-04-25T09:53:14.711462+0000","last_became_peered":"2024-04-25T09:53:14.711462+0000","last_unstale":"2024-04-25T09:53:21.303201+0000","last_undegraded":"2024-04-25T09:53:21.303201+0000","last_fullsized":"2024-04-25T09:53:21.303201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:39.622554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302329+0000","last_change":"2024-04-25T09:53:19.934786+0000","last_active":"2024-04-25T09:53:21.302329+0000","last_peered":"2024-04-25T09:53:21.302329+0000","last_clean":"2024-04-25T09:53:21.302329+0000","last_became_active":"2024-04-25T09:53:19.934417+0000","last_became_peered":"2024-04-25T09:53:19.934417+0000","last_unstale":"2024-04-25T09:53:21.302329+0000","last_undegraded":"2024-04-25T09:53:21.302329+0000","last_fullsized":"2024-04-25T09:53:21.302329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:50.372524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303078+0000","last_change":"2024-04-25T09:53:15.731430+0000","last_active":"2024-04-25T09:53:21.303078+0000","last_peered":"2024-04-25T09:53:21.303078+0000","last_clean":"2024-04-25T09:53:21.303078+0000","last_became_active":"2024-04-25T09:53:15.731152+0000","last_became_peered":"2024-04-25T09:53:15.731152+0000","last_unstale":"2024-04-25T09:53:21.303078+0000","last_undegraded":"2024-04-25T09:53:21.303078+0000","last_fullsized":"2024-04-25T09:53:21.303078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:46.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302987+0000","last_change":"2024-04-25T09:53:14.708418+0000","last_active":"2024-04-25T09:53:21.302987+0000","last_peered":"2024-04-25T09:53:21.302987+0000","last_clean":"2024-04-25T09:53:21.302987+0000","last_became_active":"2024-04-25T09:53:14.708161+0000","last_became_peered":"2024-04-25T09:53:14.708161+0000","last_unstale":"2024-04-25T09:53:21.302987+0000","last_undegraded":"2024-04-25T09:53:21.302987+0000","last_fullsized":"2024-04-25T09:53:21.302987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:58.733069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302535+0000","last_change":"2024-04-25T09:53:19.758749+0000","last_active":"2024-04-25T09:53:21.302535+0000","last_peered":"2024-04-25T09:53:21.302535+0000","last_clean":"2024-04-25T09:53:21.302535+0000","last_became_active":"2024-04-25T09:53:19.757529+0000","last_became_peered":"2024-04-25T09:53:19.757529+0000","last_unstale":"2024-04-25T09:53:21.302535+0000","last_undegraded":"2024-04-25T09:53:21.302535+0000","last_fullsized":"2024-04-25T09:53:21.302535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:06.760703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742905+0000","last_change":"2024-04-25T09:53:15.726160+0000","last_active":"2024-04-25T09:53:20.742905+0000","last_peered":"2024-04-25T09:53:20.742905+0000","last_clean":"2024-04-25T09:53:20.742905+0000","last_became_active":"2024-04-25T09:53:15.725846+0000","last_became_peered":"2024-04-25T09:53:15.725846+0000","last_unstale":"2024-04-25T09:53:20.742905+0000","last_undegraded":"2024-04-25T09:53:20.742905+0000","last_fullsized":"2024-04-25T09:53:20.742905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.262190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302405+0000","last_change":"2024-04-25T09:53:14.712606+0000","last_active":"2024-04-25T09:53:21.302405+0000","last_peered":"2024-04-25T09:53:21.302405+0000","last_clean":"2024-04-25T09:53:21.302405+0000","last_became_active":"2024-04-25T09:53:14.712327+0000","last_became_peered":"2024-04-25T09:53:14.712327+0000","last_unstale":"2024-04-25T09:53:21.302405+0000","last_undegraded":"2024-04-25T09:53:21.302405+0000","last_fullsized":"2024-04-25T09:53:21.302405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:49.257995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302949+0000","last_change":"2024-04-25T09:53:19.757694+0000","last_active":"2024-04-25T09:53:21.302949+0000","last_peered":"2024-04-25T09:53:21.302949+0000","last_clean":"2024-04-25T09:53:21.302949+0000","last_became_active":"2024-04-25T09:53:19.757293+0000","last_became_peered":"2024-04-25T09:53:19.757293+0000","last_unstale":"2024-04-25T09:53:21.302949+0000","last_undegraded":"2024-04-25T09:53:21.302949+0000","last_fullsized":"2024-04-25T09:53:21.302949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:12.794746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302692+0000","last_change":"2024-04-25T09:53:15.725158+0000","last_active":"2024-04-25T09:53:21.302692+0000","last_peered":"2024-04-25T09:53:21.302692+0000","last_clean":"2024-04-25T09:53:21.302692+0000","last_became_active":"2024-04-25T09:53:15.724890+0000","last_became_peered":"2024-04-25T09:53:15.724890+0000","last_unstale":"2024-04-25T09:53:21.302692+0000","last_undegraded":"2024-04-25T09:53:21.302692+0000","last_fullsized":"2024-04-25T09:53:21.302692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741185+0000","last_change":"2024-04-25T09:53:14.703999+0000","last_active":"2024-04-25T09:53:20.741185+0000","last_peered":"2024-04-25T09:53:20.741185+0000","last_clean":"2024-04-25T09:53:20.741185+0000","last_became_active":"2024-04-25T09:53:14.703714+0000","last_became_peered":"2024-04-25T09:53:14.703714+0000","last_unstale":"2024-04-25T09:53:20.741185+0000","last_undegraded":"2024-04-25T09:53:20.741185+0000","last_fullsized":"2024-04-25T09:53:20.741185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303908+0000","last_change":"2024-04-25T09:53:19.756345+0000","last_active":"2024-04-25T09:53:21.303908+0000","last_peered":"2024-04-25T09:53:21.303908+0000","last_clean":"2024-04-25T09:53:21.303908+0000","last_became_active":"2024-04-25T09:53:19.755915+0000","last_became_peered":"2024-04-25T09:53:19.755915+0000","last_unstale":"2024-04-25T09:53:21.303908+0000","last_undegraded":"2024-04-25T09:53:21.303908+0000","last_fullsized":"2024-04-25T09:53:21.303908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:54.084750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301435+0000","last_change":"2024-04-25T09:53:15.723323+0000","last_active":"2024-04-25T09:53:21.301435+0000","last_peered":"2024-04-25T09:53:21.301435+0000","last_clean":"2024-04-25T09:53:21.301435+0000","last_became_active":"2024-04-25T09:53:15.723019+0000","last_became_peered":"2024-04-25T09:53:15.723019+0000","last_unstale":"2024-04-25T09:53:21.301435+0000","last_undegraded":"2024-04-25T09:53:21.301435+0000","last_fullsized":"2024-04-25T09:53:21.301435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:43.951510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301698+0000","last_change":"2024-04-25T09:53:14.712642+0000","last_active":"2024-04-25T09:53:21.301698+0000","last_peered":"2024-04-25T09:53:21.301698+0000","last_clean":"2024-04-25T09:53:21.301698+0000","last_became_active":"2024-04-25T09:53:14.711778+0000","last_became_peered":"2024-04-25T09:53:14.711778+0000","last_unstale":"2024-04-25T09:53:21.301698+0000","last_undegraded":"2024-04-25T09:53:21.301698+0000","last_fullsized":"2024-04-25T09:53:21.301698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:49.257995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301888+0000","last_change":"2024-04-25T09:53:19.934697+0000","last_active":"2024-04-25T09:53:21.301888+0000","last_peered":"2024-04-25T09:53:21.301888+0000","last_clean":"2024-04-25T09:53:21.301888+0000","last_became_active":"2024-04-25T09:53:19.934259+0000","last_became_peered":"2024-04-25T09:53:19.934259+0000","last_unstale":"2024-04-25T09:53:21.301888+0000","last_undegraded":"2024-04-25T09:53:21.301888+0000","last_fullsized":"2024-04-25T09:53:21.301888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:12.794746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301861+0000","last_change":"2024-04-25T09:53:15.731434+0000","last_active":"2024-04-25T09:53:21.301861+0000","last_peered":"2024-04-25T09:53:21.301861+0000","last_clean":"2024-04-25T09:53:21.301861+0000","last_became_active":"2024-04-25T09:53:15.731139+0000","last_became_peered":"2024-04-25T09:53:15.731139+0000","last_unstale":"2024-04-25T09:53:21.301861+0000","last_undegraded":"2024-04-25T09:53:21.301861+0000","last_fullsized":"2024-04-25T09:53:21.301861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302366+0000","last_change":"2024-04-25T09:53:14.706419+0000","last_active":"2024-04-25T09:53:21.302366+0000","last_peered":"2024-04-25T09:53:21.302366+0000","last_clean":"2024-04-25T09:53:21.302366+0000","last_became_active":"2024-04-25T09:53:14.706020+0000","last_became_peered":"2024-04-25T09:53:14.706020+0000","last_unstale":"2024-04-25T09:53:21.302366+0000","last_undegraded":"2024-04-25T09:53:21.302366+0000","last_fullsized":"2024-04-25T09:53:21.302366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301803+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:53:21.301803+0000","last_peered":"2024-04-25T09:53:21.301803+0000","last_clean":"2024-04-25T09:53:21.301803+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:53:21.301803+0000","last_undegraded":"2024-04-25T09:53:21.301803+0000","last_fullsized":"2024-04-25T09:53:21.301803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303580+0000","last_change":"2024-04-25T09:53:19.937575+0000","last_active":"2024-04-25T09:53:21.303580+0000","last_peered":"2024-04-25T09:53:21.303580+0000","last_clean":"2024-04-25T09:53:21.303580+0000","last_became_active":"2024-04-25T09:53:19.937138+0000","last_became_peered":"2024-04-25T09:53:19.937138+0000","last_unstale":"2024-04-25T09:53:21.303580+0000","last_undegraded":"2024-04-25T09:53:21.303580+0000","last_fullsized":"2024-04-25T09:53:21.303580+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:01.356138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302839+0000","last_change":"2024-04-25T09:53:15.733512+0000","last_active":"2024-04-25T09:53:21.302839+0000","last_peered":"2024-04-25T09:53:21.302839+0000","last_clean":"2024-04-25T09:53:21.302839+0000","last_became_active":"2024-04-25T09:53:15.733224+0000","last_became_peered":"2024-04-25T09:53:15.733224+0000","last_unstale":"2024-04-25T09:53:21.302839+0000","last_undegraded":"2024-04-25T09:53:21.302839+0000","last_fullsized":"2024-04-25T09:53:21.302839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:57.316572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741420+0000","last_change":"2024-04-25T09:53:14.708846+0000","last_active":"2024-04-25T09:53:20.741420+0000","last_peered":"2024-04-25T09:53:20.741420+0000","last_clean":"2024-04-25T09:53:20.741420+0000","last_became_active":"2024-04-25T09:53:14.708587+0000","last_became_peered":"2024-04-25T09:53:14.708587+0000","last_unstale":"2024-04-25T09:53:20.741420+0000","last_undegraded":"2024-04-25T09:53:20.741420+0000","last_fullsized":"2024-04-25T09:53:20.741420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302923+0000","last_change":"2024-04-25T09:53:19.934818+0000","last_active":"2024-04-25T09:53:21.302923+0000","last_peered":"2024-04-25T09:53:21.302923+0000","last_clean":"2024-04-25T09:53:21.302923+0000","last_became_active":"2024-04-25T09:53:19.934403+0000","last_became_peered":"2024-04-25T09:53:19.934403+0000","last_unstale":"2024-04-25T09:53:21.302923+0000","last_undegraded":"2024-04-25T09:53:21.302923+0000","last_fullsized":"2024-04-25T09:53:21.302923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:57.211571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302766+0000","last_change":"2024-04-25T09:53:15.732566+0000","last_active":"2024-04-25T09:53:21.302766+0000","last_peered":"2024-04-25T09:53:21.302766+0000","last_clean":"2024-04-25T09:53:21.302766+0000","last_became_active":"2024-04-25T09:53:15.731920+0000","last_became_peered":"2024-04-25T09:53:15.731920+0000","last_unstale":"2024-04-25T09:53:21.302766+0000","last_undegraded":"2024-04-25T09:53:21.302766+0000","last_fullsized":"2024-04-25T09:53:21.302766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:53.172005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302249+0000","last_change":"2024-04-25T09:53:14.703268+0000","last_active":"2024-04-25T09:53:21.302249+0000","last_peered":"2024-04-25T09:53:21.302249+0000","last_clean":"2024-04-25T09:53:21.302249+0000","last_became_active":"2024-04-25T09:53:14.702989+0000","last_became_peered":"2024-04-25T09:53:14.702989+0000","last_unstale":"2024-04-25T09:53:21.302249+0000","last_undegraded":"2024-04-25T09:53:21.302249+0000","last_fullsized":"2024-04-25T09:53:21.302249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302491+0000","last_change":"2024-04-25T09:53:19.762847+0000","last_active":"2024-04-25T09:53:21.302491+0000","last_peered":"2024-04-25T09:53:21.302491+0000","last_clean":"2024-04-25T09:53:21.302491+0000","last_became_active":"2024-04-25T09:53:19.762336+0000","last_became_peered":"2024-04-25T09:53:19.762336+0000","last_unstale":"2024-04-25T09:53:21.302491+0000","last_undegraded":"2024-04-25T09:53:21.302491+0000","last_fullsized":"2024-04-25T09:53:21.302491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:54.084750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303390+0000","last_change":"2024-04-25T09:53:15.732927+0000","last_active":"2024-04-25T09:53:21.303390+0000","last_peered":"2024-04-25T09:53:21.303390+0000","last_clean":"2024-04-25T09:53:21.303390+0000","last_became_active":"2024-04-25T09:53:15.732640+0000","last_became_peered":"2024-04-25T09:53:15.732640+0000","last_unstale":"2024-04-25T09:53:21.303390+0000","last_undegraded":"2024-04-25T09:53:21.303390+0000","last_fullsized":"2024-04-25T09:53:21.303390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:27.493283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741538+0000","last_change":"2024-04-25T09:53:14.708266+0000","last_active":"2024-04-25T09:53:20.741538+0000","last_peered":"2024-04-25T09:53:20.741538+0000","last_clean":"2024-04-25T09:53:20.741538+0000","last_became_active":"2024-04-25T09:53:14.707980+0000","last_became_peered":"2024-04-25T09:53:14.707980+0000","last_unstale":"2024-04-25T09:53:20.741538+0000","last_undegraded":"2024-04-25T09:53:20.741538+0000","last_fullsized":"2024-04-25T09:53:20.741538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301745+0000","last_change":"2024-04-25T09:53:19.934601+0000","last_active":"2024-04-25T09:53:21.301745+0000","last_peered":"2024-04-25T09:53:21.301745+0000","last_clean":"2024-04-25T09:53:21.301745+0000","last_became_active":"2024-04-25T09:53:19.934185+0000","last_became_peered":"2024-04-25T09:53:19.934185+0000","last_unstale":"2024-04-25T09:53:21.301745+0000","last_undegraded":"2024-04-25T09:53:21.301745+0000","last_fullsized":"2024-04-25T09:53:21.301745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:05.087194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302564+0000","last_change":"2024-04-25T09:53:15.732497+0000","last_active":"2024-04-25T09:53:21.302564+0000","last_peered":"2024-04-25T09:53:21.302564+0000","last_clean":"2024-04-25T09:53:21.302564+0000","last_became_active":"2024-04-25T09:53:15.732229+0000","last_became_peered":"2024-04-25T09:53:15.732229+0000","last_unstale":"2024-04-25T09:53:21.302564+0000","last_undegraded":"2024-04-25T09:53:21.302564+0000","last_fullsized":"2024-04-25T09:53:21.302564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:08.755180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742750+0000","last_change":"2024-04-25T09:53:14.708424+0000","last_active":"2024-04-25T09:53:20.742750+0000","last_peered":"2024-04-25T09:53:20.742750+0000","last_clean":"2024-04-25T09:53:20.742750+0000","last_became_active":"2024-04-25T09:53:14.708139+0000","last_became_peered":"2024-04-25T09:53:14.708139+0000","last_unstale":"2024-04-25T09:53:20.742750+0000","last_undegraded":"2024-04-25T09:53:20.742750+0000","last_fullsized":"2024-04-25T09:53:20.742750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:11.395695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742138+0000","last_change":"2024-04-25T09:53:19.753647+0000","last_active":"2024-04-25T09:53:20.742138+0000","last_peered":"2024-04-25T09:53:20.742138+0000","last_clean":"2024-04-25T09:53:20.742138+0000","last_became_active":"2024-04-25T09:53:19.753091+0000","last_became_peered":"2024-04-25T09:53:19.753091+0000","last_unstale":"2024-04-25T09:53:20.742138+0000","last_undegraded":"2024-04-25T09:53:20.742138+0000","last_fullsized":"2024-04-25T09:53:20.742138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:58.778179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302186+0000","last_change":"2024-04-25T09:53:15.731871+0000","last_active":"2024-04-25T09:53:21.302186+0000","last_peered":"2024-04-25T09:53:21.302186+0000","last_clean":"2024-04-25T09:53:21.302186+0000","last_became_active":"2024-04-25T09:53:15.731615+0000","last_became_peered":"2024-04-25T09:53:15.731615+0000","last_unstale":"2024-04-25T09:53:21.302186+0000","last_undegraded":"2024-04-25T09:53:21.302186+0000","last_fullsized":"2024-04-25T09:53:21.302186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:53.172005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741071+0000","last_change":"2024-04-25T09:53:14.701334+0000","last_active":"2024-04-25T09:53:20.741071+0000","last_peered":"2024-04-25T09:53:20.741071+0000","last_clean":"2024-04-25T09:53:20.741071+0000","last_became_active":"2024-04-25T09:53:14.701062+0000","last_became_peered":"2024-04-25T09:53:14.701062+0000","last_unstale":"2024-04-25T09:53:20.741071+0000","last_undegraded":"2024-04-25T09:53:20.741071+0000","last_fullsized":"2024-04-25T09:53:20.741071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301727+0000","last_change":"2024-04-25T09:53:19.758482+0000","last_active":"2024-04-25T09:53:21.301727+0000","last_peered":"2024-04-25T09:53:21.301727+0000","last_clean":"2024-04-25T09:53:21.301727+0000","last_became_active":"2024-04-25T09:53:19.758036+0000","last_became_peered":"2024-04-25T09:53:19.758036+0000","last_unstale":"2024-04-25T09:53:21.301727+0000","last_undegraded":"2024-04-25T09:53:21.301727+0000","last_fullsized":"2024-04-25T09:53:21.301727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.068666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301688+0000","last_change":"2024-04-25T09:53:15.721989+0000","last_active":"2024-04-25T09:53:21.301688+0000","last_peered":"2024-04-25T09:53:21.301688+0000","last_clean":"2024-04-25T09:53:21.301688+0000","last_became_active":"2024-04-25T09:53:15.721636+0000","last_became_peered":"2024-04-25T09:53:15.721636+0000","last_unstale":"2024-04-25T09:53:21.301688+0000","last_undegraded":"2024-04-25T09:53:21.301688+0000","last_fullsized":"2024-04-25T09:53:21.301688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:39.445622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302694+0000","last_change":"2024-04-25T09:53:14.709691+0000","last_active":"2024-04-25T09:53:21.302694+0000","last_peered":"2024-04-25T09:53:21.302694+0000","last_clean":"2024-04-25T09:53:21.302694+0000","last_became_active":"2024-04-25T09:53:14.709373+0000","last_became_peered":"2024-04-25T09:53:14.709373+0000","last_unstale":"2024-04-25T09:53:21.302694+0000","last_undegraded":"2024-04-25T09:53:21.302694+0000","last_fullsized":"2024-04-25T09:53:21.302694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302719+0000","last_change":"2024-04-25T09:53:19.937412+0000","last_active":"2024-04-25T09:53:21.302719+0000","last_peered":"2024-04-25T09:53:21.302719+0000","last_clean":"2024-04-25T09:53:21.302719+0000","last_became_active":"2024-04-25T09:53:19.937068+0000","last_became_peered":"2024-04-25T09:53:19.937068+0000","last_unstale":"2024-04-25T09:53:21.302719+0000","last_undegraded":"2024-04-25T09:53:21.302719+0000","last_fullsized":"2024-04-25T09:53:21.302719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:18.654048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302364+0000","last_change":"2024-04-25T09:53:15.734287+0000","last_active":"2024-04-25T09:53:21.302364+0000","last_peered":"2024-04-25T09:53:21.302364+0000","last_clean":"2024-04-25T09:53:21.302364+0000","last_became_active":"2024-04-25T09:53:15.733835+0000","last_became_peered":"2024-04-25T09:53:15.733835+0000","last_unstale":"2024-04-25T09:53:21.302364+0000","last_undegraded":"2024-04-25T09:53:21.302364+0000","last_fullsized":"2024-04-25T09:53:21.302364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:55.029099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302389+0000","last_change":"2024-04-25T09:53:14.707323+0000","last_active":"2024-04-25T09:53:21.302389+0000","last_peered":"2024-04-25T09:53:21.302389+0000","last_clean":"2024-04-25T09:53:21.302389+0000","last_became_active":"2024-04-25T09:53:14.707061+0000","last_became_peered":"2024-04-25T09:53:14.707061+0000","last_unstale":"2024-04-25T09:53:21.302389+0000","last_undegraded":"2024-04-25T09:53:21.302389+0000","last_fullsized":"2024-04-25T09:53:21.302389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303418+0000","last_change":"2024-04-25T09:53:19.935315+0000","last_active":"2024-04-25T09:53:21.303418+0000","last_peered":"2024-04-25T09:53:21.303418+0000","last_clean":"2024-04-25T09:53:21.303418+0000","last_became_active":"2024-04-25T09:53:19.934800+0000","last_became_peered":"2024-04-25T09:53:19.934800+0000","last_unstale":"2024-04-25T09:53:21.303418+0000","last_undegraded":"2024-04-25T09:53:21.303418+0000","last_fullsized":"2024-04-25T09:53:21.303418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:50.137095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302044+0000","last_change":"2024-04-25T09:53:15.729897+0000","last_active":"2024-04-25T09:53:21.302044+0000","last_peered":"2024-04-25T09:53:21.302044+0000","last_clean":"2024-04-25T09:53:21.302044+0000","last_became_active":"2024-04-25T09:53:15.729269+0000","last_became_peered":"2024-04-25T09:53:15.729269+0000","last_unstale":"2024-04-25T09:53:21.302044+0000","last_undegraded":"2024-04-25T09:53:21.302044+0000","last_fullsized":"2024-04-25T09:53:21.302044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:55.387059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302564+0000","last_change":"2024-04-25T09:53:14.709636+0000","last_active":"2024-04-25T09:53:21.302564+0000","last_peered":"2024-04-25T09:53:21.302564+0000","last_clean":"2024-04-25T09:53:21.302564+0000","last_became_active":"2024-04-25T09:53:14.709366+0000","last_became_peered":"2024-04-25T09:53:14.709366+0000","last_unstale":"2024-04-25T09:53:21.302564+0000","last_undegraded":"2024-04-25T09:53:21.302564+0000","last_fullsized":"2024-04-25T09:53:21.302564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:05.869022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303066+0000","last_change":"2024-04-25T09:53:19.937074+0000","last_active":"2024-04-25T09:53:21.303066+0000","last_peered":"2024-04-25T09:53:21.303066+0000","last_clean":"2024-04-25T09:53:21.303066+0000","last_became_active":"2024-04-25T09:53:19.935313+0000","last_became_peered":"2024-04-25T09:53:19.935313+0000","last_unstale":"2024-04-25T09:53:21.303066+0000","last_undegraded":"2024-04-25T09:53:21.303066+0000","last_fullsized":"2024-04-25T09:53:21.303066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:56.116208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302820+0000","last_change":"2024-04-25T09:53:15.726822+0000","last_active":"2024-04-25T09:53:21.302820+0000","last_peered":"2024-04-25T09:53:21.302820+0000","last_clean":"2024-04-25T09:53:21.302820+0000","last_became_active":"2024-04-25T09:53:15.726526+0000","last_became_peered":"2024-04-25T09:53:15.726526+0000","last_unstale":"2024-04-25T09:53:21.302820+0000","last_undegraded":"2024-04-25T09:53:21.302820+0000","last_fullsized":"2024-04-25T09:53:21.302820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:16.569326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:24.817228+0000","last_change":"2024-04-25T09:53:14.707187+0000","last_active":"2024-04-25T09:53:24.817228+0000","last_peered":"2024-04-25T09:53:24.817228+0000","last_clean":"2024-04-25T09:53:24.817228+0000","last_became_active":"2024-04-25T09:53:14.706509+0000","last_became_peered":"2024-04-25T09:53:14.706509+0000","last_unstale":"2024-04-25T09:53:24.817228+0000","last_undegraded":"2024-04-25T09:53:24.817228+0000","last_fullsized":"2024-04-25T09:53:24.817228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302676+0000","last_change":"2024-04-25T09:53:19.758453+0000","last_active":"2024-04-25T09:53:21.302676+0000","last_peered":"2024-04-25T09:53:21.302676+0000","last_clean":"2024-04-25T09:53:21.302676+0000","last_became_active":"2024-04-25T09:53:19.757004+0000","last_became_peered":"2024-04-25T09:53:19.757004+0000","last_unstale":"2024-04-25T09:53:21.302676+0000","last_undegraded":"2024-04-25T09:53:21.302676+0000","last_fullsized":"2024-04-25T09:53:21.302676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.068666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743281+0000","last_change":"2024-04-25T09:53:15.731519+0000","last_active":"2024-04-25T09:53:20.743281+0000","last_peered":"2024-04-25T09:53:20.743281+0000","last_clean":"2024-04-25T09:53:20.743281+0000","last_became_active":"2024-04-25T09:53:15.731037+0000","last_became_peered":"2024-04-25T09:53:15.731037+0000","last_unstale":"2024-04-25T09:53:20.743281+0000","last_undegraded":"2024-04-25T09:53:20.743281+0000","last_fullsized":"2024-04-25T09:53:20.743281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.530029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302151+0000","last_change":"2024-04-25T09:53:14.713398+0000","last_active":"2024-04-25T09:53:21.302151+0000","last_peered":"2024-04-25T09:53:21.302151+0000","last_clean":"2024-04-25T09:53:21.302151+0000","last_became_active":"2024-04-25T09:53:14.713125+0000","last_became_peered":"2024-04-25T09:53:14.713125+0000","last_unstale":"2024-04-25T09:53:21.302151+0000","last_undegraded":"2024-04-25T09:53:21.302151+0000","last_fullsized":"2024-04-25T09:53:21.302151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742729+0000","last_change":"2024-04-25T09:53:19.762998+0000","last_active":"2024-04-25T09:53:20.742729+0000","last_peered":"2024-04-25T09:53:20.742729+0000","last_clean":"2024-04-25T09:53:20.742729+0000","last_became_active":"2024-04-25T09:53:19.762550+0000","last_became_peered":"2024-04-25T09:53:19.762550+0000","last_unstale":"2024-04-25T09:53:20.742729+0000","last_undegraded":"2024-04-25T09:53:20.742729+0000","last_fullsized":"2024-04-25T09:53:20.742729+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:57.211571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302003+0000","last_change":"2024-04-25T09:53:15.731707+0000","last_active":"2024-04-25T09:53:21.302003+0000","last_peered":"2024-04-25T09:53:21.302003+0000","last_clean":"2024-04-25T09:53:21.302003+0000","last_became_active":"2024-04-25T09:53:15.731303+0000","last_became_peered":"2024-04-25T09:53:15.731303+0000","last_unstale":"2024-04-25T09:53:21.302003+0000","last_undegraded":"2024-04-25T09:53:21.302003+0000","last_fullsized":"2024-04-25T09:53:21.302003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302319+0000","last_change":"2024-04-25T09:53:14.714447+0000","last_active":"2024-04-25T09:53:21.302319+0000","last_peered":"2024-04-25T09:53:21.302319+0000","last_clean":"2024-04-25T09:53:21.302319+0000","last_became_active":"2024-04-25T09:53:14.714155+0000","last_became_peered":"2024-04-25T09:53:14.714155+0000","last_unstale":"2024-04-25T09:53:21.302319+0000","last_undegraded":"2024-04-25T09:53:21.302319+0000","last_fullsized":"2024-04-25T09:53:21.302319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301664+0000","last_change":"2024-04-25T09:53:19.761513+0000","last_active":"2024-04-25T09:53:21.301664+0000","last_peered":"2024-04-25T09:53:21.301664+0000","last_clean":"2024-04-25T09:53:21.301664+0000","last_became_active":"2024-04-25T09:53:19.761037+0000","last_became_peered":"2024-04-25T09:53:19.761037+0000","last_unstale":"2024-04-25T09:53:21.301664+0000","last_undegraded":"2024-04-25T09:53:21.301664+0000","last_fullsized":"2024-04-25T09:53:21.301664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:31.532850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302218+0000","last_change":"2024-04-25T09:53:15.742482+0000","last_active":"2024-04-25T09:53:21.302218+0000","last_peered":"2024-04-25T09:53:21.302218+0000","last_clean":"2024-04-25T09:53:21.302218+0000","last_became_active":"2024-04-25T09:53:15.742170+0000","last_became_peered":"2024-04-25T09:53:15.742170+0000","last_unstale":"2024-04-25T09:53:21.302218+0000","last_undegraded":"2024-04-25T09:53:21.302218+0000","last_fullsized":"2024-04-25T09:53:21.302218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:16.569326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303779+0000","last_change":"2024-04-25T09:53:14.714376+0000","last_active":"2024-04-25T09:53:21.303779+0000","last_peered":"2024-04-25T09:53:21.303779+0000","last_clean":"2024-04-25T09:53:21.303779+0000","last_became_active":"2024-04-25T09:53:14.714116+0000","last_became_peered":"2024-04-25T09:53:14.714116+0000","last_unstale":"2024-04-25T09:53:21.303779+0000","last_undegraded":"2024-04-25T09:53:21.303779+0000","last_fullsized":"2024-04-25T09:53:21.303779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303146+0000","last_change":"2024-04-25T09:53:19.938076+0000","last_active":"2024-04-25T09:53:21.303146+0000","last_peered":"2024-04-25T09:53:21.303146+0000","last_clean":"2024-04-25T09:53:21.303146+0000","last_became_active":"2024-04-25T09:53:19.937704+0000","last_became_peered":"2024-04-25T09:53:19.937704+0000","last_unstale":"2024-04-25T09:53:21.303146+0000","last_undegraded":"2024-04-25T09:53:21.303146+0000","last_fullsized":"2024-04-25T09:53:21.303146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:56.116208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301567+0000","last_change":"2024-04-25T09:53:19.936514+0000","last_active":"2024-04-25T09:53:21.301567+0000","last_peered":"2024-04-25T09:53:21.301567+0000","last_clean":"2024-04-25T09:53:21.301567+0000","last_became_active":"2024-04-25T09:53:19.936095+0000","last_became_peered":"2024-04-25T09:53:19.936095+0000","last_unstale":"2024-04-25T09:53:21.301567+0000","last_undegraded":"2024-04-25T09:53:21.301567+0000","last_fullsized":"2024-04-25T09:53:21.301567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:06.760703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301623+0000","last_change":"2024-04-25T09:53:15.726524+0000","last_active":"2024-04-25T09:53:21.301623+0000","last_peered":"2024-04-25T09:53:21.301623+0000","last_clean":"2024-04-25T09:53:21.301623+0000","last_became_active":"2024-04-25T09:53:15.726160+0000","last_became_peered":"2024-04-25T09:53:15.726160+0000","last_unstale":"2024-04-25T09:53:21.301623+0000","last_undegraded":"2024-04-25T09:53:21.301623+0000","last_fullsized":"2024-04-25T09:53:21.301623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302003+0000","last_change":"2024-04-25T09:53:14.704589+0000","last_active":"2024-04-25T09:53:21.302003+0000","last_peered":"2024-04-25T09:53:21.302003+0000","last_clean":"2024-04-25T09:53:21.302003+0000","last_became_active":"2024-04-25T09:53:14.704228+0000","last_became_peered":"2024-04-25T09:53:14.704228+0000","last_unstale":"2024-04-25T09:53:21.302003+0000","last_undegraded":"2024-04-25T09:53:21.302003+0000","last_fullsized":"2024-04-25T09:53:21.302003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303831+0000","last_change":"2024-04-25T09:53:19.761677+0000","last_active":"2024-04-25T09:53:21.303831+0000","last_peered":"2024-04-25T09:53:21.303831+0000","last_clean":"2024-04-25T09:53:21.303831+0000","last_became_active":"2024-04-25T09:53:19.761276+0000","last_became_peered":"2024-04-25T09:53:19.761276+0000","last_unstale":"2024-04-25T09:53:21.303831+0000","last_undegraded":"2024-04-25T09:53:21.303831+0000","last_fullsized":"2024-04-25T09:53:21.303831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:20.608892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301714+0000","last_change":"2024-04-25T09:53:15.722486+0000","last_active":"2024-04-25T09:53:21.301714+0000","last_peered":"2024-04-25T09:53:21.301714+0000","last_clean":"2024-04-25T09:53:21.301714+0000","last_became_active":"2024-04-25T09:53:15.722225+0000","last_became_peered":"2024-04-25T09:53:15.722225+0000","last_unstale":"2024-04-25T09:53:21.301714+0000","last_undegraded":"2024-04-25T09:53:21.301714+0000","last_fullsized":"2024-04-25T09:53:21.301714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303275+0000","last_change":"2024-04-25T09:53:14.707399+0000","last_active":"2024-04-25T09:53:21.303275+0000","last_peered":"2024-04-25T09:53:21.303275+0000","last_clean":"2024-04-25T09:53:21.303275+0000","last_became_active":"2024-04-25T09:53:14.707078+0000","last_became_peered":"2024-04-25T09:53:14.707078+0000","last_unstale":"2024-04-25T09:53:21.303275+0000","last_undegraded":"2024-04-25T09:53:21.303275+0000","last_fullsized":"2024-04-25T09:53:21.303275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:57.295010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303180+0000","last_change":"2024-04-25T09:53:14.711893+0000","last_active":"2024-04-25T09:53:21.303180+0000","last_peered":"2024-04-25T09:53:21.303180+0000","last_clean":"2024-04-25T09:53:21.303180+0000","last_became_active":"2024-04-25T09:53:14.711513+0000","last_became_peered":"2024-04-25T09:53:14.711513+0000","last_unstale":"2024-04-25T09:53:21.303180+0000","last_undegraded":"2024-04-25T09:53:21.303180+0000","last_fullsized":"2024-04-25T09:53:21.303180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302544+0000","last_change":"2024-04-25T09:53:15.728536+0000","last_active":"2024-04-25T09:53:21.302544+0000","last_peered":"2024-04-25T09:53:21.302544+0000","last_clean":"2024-04-25T09:53:21.302544+0000","last_became_active":"2024-04-25T09:53:15.728235+0000","last_became_peered":"2024-04-25T09:53:15.728235+0000","last_unstale":"2024-04-25T09:53:21.302544+0000","last_undegraded":"2024-04-25T09:53:21.302544+0000","last_fullsized":"2024-04-25T09:53:21.302544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741367+0000","last_change":"2024-04-25T09:53:19.764626+0000","last_active":"2024-04-25T09:53:20.741367+0000","last_peered":"2024-04-25T09:53:20.741367+0000","last_clean":"2024-04-25T09:53:20.741367+0000","last_became_active":"2024-04-25T09:53:19.763954+0000","last_became_peered":"2024-04-25T09:53:19.763954+0000","last_unstale":"2024-04-25T09:53:20.741367+0000","last_undegraded":"2024-04-25T09:53:20.741367+0000","last_fullsized":"2024-04-25T09:53:20.741367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:47.991077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302285+0000","last_change":"2024-04-25T09:53:14.712000+0000","last_active":"2024-04-25T09:53:21.302285+0000","last_peered":"2024-04-25T09:53:21.302285+0000","last_clean":"2024-04-25T09:53:21.302285+0000","last_became_active":"2024-04-25T09:53:14.711712+0000","last_became_peered":"2024-04-25T09:53:14.711712+0000","last_unstale":"2024-04-25T09:53:21.302285+0000","last_undegraded":"2024-04-25T09:53:21.302285+0000","last_fullsized":"2024-04-25T09:53:21.302285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742400+0000","last_change":"2024-04-25T09:53:15.722357+0000","last_active":"2024-04-25T09:53:20.742400+0000","last_peered":"2024-04-25T09:53:20.742400+0000","last_clean":"2024-04-25T09:53:20.742400+0000","last_became_active":"2024-04-25T09:53:15.722082+0000","last_became_peered":"2024-04-25T09:53:15.722082+0000","last_unstale":"2024-04-25T09:53:20.742400+0000","last_undegraded":"2024-04-25T09:53:20.742400+0000","last_fullsized":"2024-04-25T09:53:20.742400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:39.270721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303146+0000","last_change":"2024-04-25T09:53:19.759080+0000","last_active":"2024-04-25T09:53:21.303146+0000","last_peered":"2024-04-25T09:53:21.303146+0000","last_clean":"2024-04-25T09:53:21.303146+0000","last_became_active":"2024-04-25T09:53:19.758705+0000","last_became_peered":"2024-04-25T09:53:19.758705+0000","last_unstale":"2024-04-25T09:53:21.303146+0000","last_undegraded":"2024-04-25T09:53:21.303146+0000","last_fullsized":"2024-04-25T09:53:21.303146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:54.228875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301484+0000","last_change":"2024-04-25T09:53:15.723965+0000","last_active":"2024-04-25T09:53:21.301484+0000","last_peered":"2024-04-25T09:53:21.301484+0000","last_clean":"2024-04-25T09:53:21.301484+0000","last_became_active":"2024-04-25T09:53:15.723660+0000","last_became_peered":"2024-04-25T09:53:15.723660+0000","last_unstale":"2024-04-25T09:53:21.301484+0000","last_undegraded":"2024-04-25T09:53:21.301484+0000","last_fullsized":"2024-04-25T09:53:21.301484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741222+0000","last_change":"2024-04-25T09:53:15.734494+0000","last_active":"2024-04-25T09:53:20.741222+0000","last_peered":"2024-04-25T09:53:20.741222+0000","last_clean":"2024-04-25T09:53:20.741222+0000","last_became_active":"2024-04-25T09:53:15.734204+0000","last_became_peered":"2024-04-25T09:53:15.734204+0000","last_unstale":"2024-04-25T09:53:20.741222+0000","last_undegraded":"2024-04-25T09:53:20.741222+0000","last_fullsized":"2024-04-25T09:53:20.741222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.530029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302296+0000","last_change":"2024-04-25T09:53:15.731317+0000","last_active":"2024-04-25T09:53:21.302296+0000","last_peered":"2024-04-25T09:53:21.302296+0000","last_clean":"2024-04-25T09:53:21.302296+0000","last_became_active":"2024-04-25T09:53:15.731022+0000","last_became_peered":"2024-04-25T09:53:15.731022+0000","last_unstale":"2024-04-25T09:53:21.302296+0000","last_undegraded":"2024-04-25T09:53:21.302296+0000","last_fullsized":"2024-04-25T09:53:21.302296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:02.721136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302472+0000","last_change":"2024-04-25T09:53:15.720872+0000","last_active":"2024-04-25T09:53:21.302472+0000","last_peered":"2024-04-25T09:53:21.302472+0000","last_clean":"2024-04-25T09:53:21.302472+0000","last_became_active":"2024-04-25T09:53:15.720242+0000","last_became_peered":"2024-04-25T09:53:15.720242+0000","last_unstale":"2024-04-25T09:53:21.302472+0000","last_undegraded":"2024-04-25T09:53:21.302472+0000","last_fullsized":"2024-04-25T09:53:21.302472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303531+0000","last_change":"2024-04-25T09:53:15.728500+0000","last_active":"2024-04-25T09:53:21.303531+0000","last_peered":"2024-04-25T09:53:21.303531+0000","last_clean":"2024-04-25T09:53:21.303531+0000","last_became_active":"2024-04-25T09:53:15.726998+0000","last_became_peered":"2024-04-25T09:53:15.726998+0000","last_unstale":"2024-04-25T09:53:21.303531+0000","last_undegraded":"2024-04-25T09:53:21.303531+0000","last_fullsized":"2024-04-25T09:53:21.303531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:00.208172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301991+0000","last_change":"2024-04-25T09:53:15.726337+0000","last_active":"2024-04-25T09:53:21.301991+0000","last_peered":"2024-04-25T09:53:21.301991+0000","last_clean":"2024-04-25T09:53:21.301991+0000","last_became_active":"2024-04-25T09:53:15.726072+0000","last_became_peered":"2024-04-25T09:53:15.726072+0000","last_unstale":"2024-04-25T09:53:21.301991+0000","last_undegraded":"2024-04-25T09:53:21.301991+0000","last_fullsized":"2024-04-25T09:53:21.301991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303810+0000","last_change":"2024-04-25T09:53:15.735200+0000","last_active":"2024-04-25T09:53:21.303810+0000","last_peered":"2024-04-25T09:53:21.303810+0000","last_clean":"2024-04-25T09:53:21.303810+0000","last_became_active":"2024-04-25T09:53:15.734903+0000","last_became_peered":"2024-04-25T09:53:15.734903+0000","last_unstale":"2024-04-25T09:53:21.303810+0000","last_undegraded":"2024-04-25T09:53:21.303810+0000","last_fullsized":"2024-04-25T09:53:21.303810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302760+0000","last_change":"2024-04-25T09:53:15.726664+0000","last_active":"2024-04-25T09:53:21.302760+0000","last_peered":"2024-04-25T09:53:21.302760+0000","last_clean":"2024-04-25T09:53:21.302760+0000","last_became_active":"2024-04-25T09:53:15.726362+0000","last_became_peered":"2024-04-25T09:53:15.726362+0000","last_unstale":"2024-04-25T09:53:21.302760+0000","last_undegraded":"2024-04-25T09:53:21.302760+0000","last_fullsized":"2024-04-25T09:53:21.302760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:59.223177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301824+0000","last_change":"2024-04-25T09:53:15.731610+0000","last_active":"2024-04-25T09:53:21.301824+0000","last_peered":"2024-04-25T09:53:21.301824+0000","last_clean":"2024-04-25T09:53:21.301824+0000","last_became_active":"2024-04-25T09:53:15.731289+0000","last_became_peered":"2024-04-25T09:53:15.731289+0000","last_unstale":"2024-04-25T09:53:21.301824+0000","last_undegraded":"2024-04-25T09:53:21.301824+0000","last_fullsized":"2024-04-25T09:53:21.301824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303221+0000","last_change":"2024-04-25T09:53:15.726434+0000","last_active":"2024-04-25T09:53:21.303221+0000","last_peered":"2024-04-25T09:53:21.303221+0000","last_clean":"2024-04-25T09:53:21.303221+0000","last_became_active":"2024-04-25T09:53:15.726160+0000","last_became_peered":"2024-04-25T09:53:15.726160+0000","last_unstale":"2024-04-25T09:53:21.303221+0000","last_undegraded":"2024-04-25T09:53:21.303221+0000","last_fullsized":"2024-04-25T09:53:21.303221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:39.445622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302185+0000","last_change":"2024-04-25T09:53:15.742414+0000","last_active":"2024-04-25T09:53:21.302185+0000","last_peered":"2024-04-25T09:53:21.302185+0000","last_clean":"2024-04-25T09:53:21.302185+0000","last_became_active":"2024-04-25T09:53:15.742099+0000","last_became_peered":"2024-04-25T09:53:15.742099+0000","last_unstale":"2024-04-25T09:53:21.302185+0000","last_undegraded":"2024-04-25T09:53:21.302185+0000","last_fullsized":"2024-04-25T09:53:21.302185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:53.040580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302379+0000","last_change":"2024-04-25T09:53:15.729969+0000","last_active":"2024-04-25T09:53:21.302379+0000","last_peered":"2024-04-25T09:53:21.302379+0000","last_clean":"2024-04-25T09:53:21.302379+0000","last_became_active":"2024-04-25T09:53:15.729608+0000","last_became_peered":"2024-04-25T09:53:15.729608+0000","last_unstale":"2024-04-25T09:53:21.302379+0000","last_undegraded":"2024-04-25T09:53:21.302379+0000","last_fullsized":"2024-04-25T09:53:21.302379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303562+0000","last_change":"2024-04-25T09:53:15.730114+0000","last_active":"2024-04-25T09:53:21.303562+0000","last_peered":"2024-04-25T09:53:21.303562+0000","last_clean":"2024-04-25T09:53:21.303562+0000","last_became_active":"2024-04-25T09:53:15.729664+0000","last_became_peered":"2024-04-25T09:53:15.729664+0000","last_unstale":"2024-04-25T09:53:21.303562+0000","last_undegraded":"2024-04-25T09:53:21.303562+0000","last_fullsized":"2024-04-25T09:53:21.303562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303105+0000","last_change":"2024-04-25T09:53:15.730458+0000","last_active":"2024-04-25T09:53:21.303105+0000","last_peered":"2024-04-25T09:53:21.303105+0000","last_clean":"2024-04-25T09:53:21.303105+0000","last_became_active":"2024-04-25T09:53:15.730193+0000","last_became_peered":"2024-04-25T09:53:15.730193+0000","last_unstale":"2024-04-25T09:53:21.303105+0000","last_undegraded":"2024-04-25T09:53:21.303105+0000","last_fullsized":"2024-04-25T09:53:21.303105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:46.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741462+0000","last_change":"2024-04-25T09:53:15.723126+0000","last_active":"2024-04-25T09:53:20.741462+0000","last_peered":"2024-04-25T09:53:20.741462+0000","last_clean":"2024-04-25T09:53:20.741462+0000","last_became_active":"2024-04-25T09:53:15.722834+0000","last_became_peered":"2024-04-25T09:53:15.722834+0000","last_unstale":"2024-04-25T09:53:20.741462+0000","last_undegraded":"2024-04-25T09:53:20.741462+0000","last_fullsized":"2024-04-25T09:53:20.741462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:40.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303240+0000","last_change":"2024-04-25T09:53:15.742492+0000","last_active":"2024-04-25T09:53:21.303240+0000","last_peered":"2024-04-25T09:53:21.303240+0000","last_clean":"2024-04-25T09:53:21.303240+0000","last_became_active":"2024-04-25T09:53:15.742132+0000","last_became_peered":"2024-04-25T09:53:15.742132+0000","last_unstale":"2024-04-25T09:53:21.303240+0000","last_undegraded":"2024-04-25T09:53:21.303240+0000","last_fullsized":"2024-04-25T09:53:21.303240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:08.755180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301867+0000","last_change":"2024-04-25T09:53:15.729450+0000","last_active":"2024-04-25T09:53:21.301867+0000","last_peered":"2024-04-25T09:53:21.301867+0000","last_clean":"2024-04-25T09:53:21.301867+0000","last_became_active":"2024-04-25T09:53:15.729157+0000","last_became_peered":"2024-04-25T09:53:15.729157+0000","last_unstale":"2024-04-25T09:53:21.301867+0000","last_undegraded":"2024-04-25T09:53:21.301867+0000","last_fullsized":"2024-04-25T09:53:21.301867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301716+0000","last_change":"2024-04-25T09:53:15.727922+0000","last_active":"2024-04-25T09:53:21.301716+0000","last_peered":"2024-04-25T09:53:21.301716+0000","last_clean":"2024-04-25T09:53:21.301716+0000","last_became_active":"2024-04-25T09:53:15.727626+0000","last_became_peered":"2024-04-25T09:53:15.727626+0000","last_unstale":"2024-04-25T09:53:21.301716+0000","last_undegraded":"2024-04-25T09:53:21.301716+0000","last_fullsized":"2024-04-25T09:53:21.301716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301947+0000","last_change":"2024-04-25T09:53:15.721374+0000","last_active":"2024-04-25T09:53:21.301947+0000","last_peered":"2024-04-25T09:53:21.301947+0000","last_clean":"2024-04-25T09:53:21.301947+0000","last_became_active":"2024-04-25T09:53:15.719489+0000","last_became_peered":"2024-04-25T09:53:15.719489+0000","last_unstale":"2024-04-25T09:53:21.301947+0000","last_undegraded":"2024-04-25T09:53:21.301947+0000","last_fullsized":"2024-04-25T09:53:21.301947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:53.990141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302112+0000","last_change":"2024-04-25T09:53:15.723353+0000","last_active":"2024-04-25T09:53:21.302112+0000","last_peered":"2024-04-25T09:53:21.302112+0000","last_clean":"2024-04-25T09:53:21.302112+0000","last_became_active":"2024-04-25T09:53:15.723060+0000","last_became_peered":"2024-04-25T09:53:15.723060+0000","last_unstale":"2024-04-25T09:53:21.302112+0000","last_undegraded":"2024-04-25T09:53:21.302112+0000","last_fullsized":"2024-04-25T09:53:21.302112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743264+0000","last_change":"2024-04-25T09:53:15.731294+0000","last_active":"2024-04-25T09:53:20.743264+0000","last_peered":"2024-04-25T09:53:20.743264+0000","last_clean":"2024-04-25T09:53:20.743264+0000","last_became_active":"2024-04-25T09:53:15.730556+0000","last_became_peered":"2024-04-25T09:53:15.730556+0000","last_unstale":"2024-04-25T09:53:20.743264+0000","last_undegraded":"2024-04-25T09:53:20.743264+0000","last_fullsized":"2024-04-25T09:53:20.743264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:59.737264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303649+0000","last_change":"2024-04-25T09:53:15.737541+0000","last_active":"2024-04-25T09:53:21.303649+0000","last_peered":"2024-04-25T09:53:21.303649+0000","last_clean":"2024-04-25T09:53:21.303649+0000","last_became_active":"2024-04-25T09:53:15.737262+0000","last_became_peered":"2024-04-25T09:53:15.737262+0000","last_unstale":"2024-04-25T09:53:21.303649+0000","last_undegraded":"2024-04-25T09:53:21.303649+0000","last_fullsized":"2024-04-25T09:53:21.303649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303812+0000","last_change":"2024-04-25T09:53:15.730915+0000","last_active":"2024-04-25T09:53:21.303812+0000","last_peered":"2024-04-25T09:53:21.303812+0000","last_clean":"2024-04-25T09:53:21.303812+0000","last_became_active":"2024-04-25T09:53:15.730657+0000","last_became_peered":"2024-04-25T09:53:15.730657+0000","last_unstale":"2024-04-25T09:53:21.303812+0000","last_undegraded":"2024-04-25T09:53:21.303812+0000","last_fullsized":"2024-04-25T09:53:21.303812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303315+0000","last_change":"2024-04-25T09:53:15.733524+0000","last_active":"2024-04-25T09:53:21.303315+0000","last_peered":"2024-04-25T09:53:21.303315+0000","last_clean":"2024-04-25T09:53:21.303315+0000","last_became_active":"2024-04-25T09:53:15.733244+0000","last_became_peered":"2024-04-25T09:53:15.733244+0000","last_unstale":"2024-04-25T09:53:21.303315+0000","last_undegraded":"2024-04-25T09:53:21.303315+0000","last_fullsized":"2024-04-25T09:53:21.303315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:07.652771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301474+0000","last_change":"2024-04-25T09:53:15.731461+0000","last_active":"2024-04-25T09:53:21.301474+0000","last_peered":"2024-04-25T09:53:21.301474+0000","last_clean":"2024-04-25T09:53:21.301474+0000","last_became_active":"2024-04-25T09:53:15.731091+0000","last_became_peered":"2024-04-25T09:53:15.731091+0000","last_unstale":"2024-04-25T09:53:21.301474+0000","last_undegraded":"2024-04-25T09:53:21.301474+0000","last_fullsized":"2024-04-25T09:53:21.301474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302158+0000","last_change":"2024-04-25T09:53:15.724502+0000","last_active":"2024-04-25T09:53:21.302158+0000","last_peered":"2024-04-25T09:53:21.302158+0000","last_clean":"2024-04-25T09:53:21.302158+0000","last_became_active":"2024-04-25T09:53:15.724218+0000","last_became_peered":"2024-04-25T09:53:15.724218+0000","last_unstale":"2024-04-25T09:53:21.302158+0000","last_undegraded":"2024-04-25T09:53:21.302158+0000","last_fullsized":"2024-04-25T09:53:21.302158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:59.223177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302094+0000","last_change":"2024-04-25T09:53:15.734479+0000","last_active":"2024-04-25T09:53:21.302094+0000","last_peered":"2024-04-25T09:53:21.302094+0000","last_clean":"2024-04-25T09:53:21.302094+0000","last_became_active":"2024-04-25T09:53:15.734127+0000","last_became_peered":"2024-04-25T09:53:15.734127+0000","last_unstale":"2024-04-25T09:53:21.302094+0000","last_undegraded":"2024-04-25T09:53:21.302094+0000","last_fullsized":"2024-04-25T09:53:21.302094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:00.208172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302254+0000","last_change":"2024-04-25T09:53:15.731043+0000","last_active":"2024-04-25T09:53:21.302254+0000","last_peered":"2024-04-25T09:53:21.302254+0000","last_clean":"2024-04-25T09:53:21.302254+0000","last_became_active":"2024-04-25T09:53:15.730779+0000","last_became_peered":"2024-04-25T09:53:15.730779+0000","last_unstale":"2024-04-25T09:53:21.302254+0000","last_undegraded":"2024-04-25T09:53:21.302254+0000","last_fullsized":"2024-04-25T09:53:21.302254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:02.721136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302662+0000","last_change":"2024-04-25T09:53:15.732810+0000","last_active":"2024-04-25T09:53:21.302662+0000","last_peered":"2024-04-25T09:53:21.302662+0000","last_clean":"2024-04-25T09:53:21.302662+0000","last_became_active":"2024-04-25T09:53:15.732270+0000","last_became_peered":"2024-04-25T09:53:15.732270+0000","last_unstale":"2024-04-25T09:53:21.302662+0000","last_undegraded":"2024-04-25T09:53:21.302662+0000","last_fullsized":"2024-04-25T09:53:21.302662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:59.223177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301627+0000","last_change":"2024-04-25T09:53:15.713610+0000","last_active":"2024-04-25T09:53:21.301627+0000","last_peered":"2024-04-25T09:53:21.301627+0000","last_clean":"2024-04-25T09:53:21.301627+0000","last_became_active":"2024-04-25T09:53:15.713342+0000","last_became_peered":"2024-04-25T09:53:15.713342+0000","last_unstale":"2024-04-25T09:53:21.301627+0000","last_undegraded":"2024-04-25T09:53:21.301627+0000","last_fullsized":"2024-04-25T09:53:21.301627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:39.445622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303622+0000","last_change":"2024-04-25T09:53:15.723089+0000","last_active":"2024-04-25T09:53:21.303622+0000","last_peered":"2024-04-25T09:53:21.303622+0000","last_clean":"2024-04-25T09:53:21.303622+0000","last_became_active":"2024-04-25T09:53:15.722840+0000","last_became_peered":"2024-04-25T09:53:15.722840+0000","last_unstale":"2024-04-25T09:53:21.303622+0000","last_undegraded":"2024-04-25T09:53:21.303622+0000","last_fullsized":"2024-04-25T09:53:21.303622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303201+0000","last_change":"2024-04-25T09:53:15.734984+0000","last_active":"2024-04-25T09:53:21.303201+0000","last_peered":"2024-04-25T09:53:21.303201+0000","last_clean":"2024-04-25T09:53:21.303201+0000","last_became_active":"2024-04-25T09:53:15.734344+0000","last_became_peered":"2024-04-25T09:53:15.734344+0000","last_unstale":"2024-04-25T09:53:21.303201+0000","last_undegraded":"2024-04-25T09:53:21.303201+0000","last_fullsized":"2024-04-25T09:53:21.303201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:50.045183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:53:37.646 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:53:37.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.772+0000 7f5ec4be0640 1 Processor -- start 2024-04-25T09:53:37.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.772+0000 7f5ec4be0640 1 -- start start 2024-04-25T09:53:37.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ec4be0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5ec01611e0 0x7f5ec00b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ec4be0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00cea00 0x7f5ec006c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ec4be0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec00d4740 con 0x7f5ec00cee60 2024-04-25T09:53:37.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ec4be0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec00d1fa0 con 0x7f5ec01611e0 2024-04-25T09:53:37.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ec4be0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00d2560 con 0x7f5ec00cea00 2024-04-25T09:53:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebed9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5ec00cee60 0x7f5ec00d29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48600/0 (socket says 172.21.15.139:48600) 2024-04-25T09:53:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebed9d640 1 -- 172.21.15.139:0/3339314433 learned_addr learned my addr 172.21.15.139:0/3339314433 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebe59c640 1 --2- 172.21.15.139:0/3339314433 >> v2:172.21.15.150:3300/0 conn(0x7f5ec01611e0 0x7f5ec00b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5eb7fff640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00cea00 0x7f5ec006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009938526 0 0) 0x7f5ec00d4740 con 0x7f5ec00cee60 2024-04-25T09:53:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5eac003520 con 0x7f5ec00cee60 2024-04-25T09:53:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 257879264 0 0) 0x7f5eac003520 con 0x7f5ec00cee60 2024-04-25T09:53:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00cea00 msgr2=0x7f5ec006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00cea00 0x7f5ec006c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 >> v2:172.21.15.150:3300/0 conn(0x7f5ec01611e0 msgr2=0x7f5ec00b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 --2- 172.21.15.139:0/3339314433 >> v2:172.21.15.150:3300/0 conn(0x7f5ec01611e0 0x7f5ec00b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec00b2770 con 0x7f5ec00cee60 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f5eb80019f0 con 0x7f5ec00cee60 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebe59c640 1 --2- 172.21.15.139:0/3339314433 >> v2:172.21.15.150:3300/0 conn(0x7f5ec01611e0 0x7f5ec00b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 >> v1:172.21.15.139:6789/0 conn(0x7f5ec00cee60 legacy=0x7f5ec00d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eac0039b0 0x7f5eac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00cea00 0x7f5eac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eac004d40 0x7f5eac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00d1fa0 con 0x7f5eac0039b0 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00d2560 con 0x7f5eac004d40 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.773+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac00b9c0 con 0x7f5ec00cea00 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.774+0000 7f5ebe59c640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eac0039b0 0x7f5eac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.774+0000 7f5eb7fff640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00cea00 0x7f5eac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.774+0000 7f5ebed9d640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eac004d40 0x7f5eac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.774+0000 7f5ebe59c640 1 -- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00cea00 msgr2=0x7f5eac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:37.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.774+0000 7f5ebe59c640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00cea00 0x7f5eac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.774+0000 7f5ebe59c640 1 -- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eac004d40 msgr2=0x7f5eac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:37.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.774+0000 7f5ebe59c640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eac004d40 0x7f5eac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.774+0000 7f5ebe59c640 1 -- 172.21.15.139:0/3339314433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac003520 con 0x7f5eac0039b0 2024-04-25T09:53:37.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.774+0000 7f5eb7fff640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00cea00 0x7f5eac004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:37.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.774+0000 7f5ebed9d640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eac004d40 0x7f5eac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:37.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.774+0000 7f5ebe59c640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eac0039b0 0x7f5eac003dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009c90 tx=0x7f5eac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae2f5ea69727284 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:37.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.775+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0036070 con 0x7f5eac0039b0 2024-04-25T09:53:37.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.775+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb0031400 con 0x7f5eac0039b0 2024-04-25T09:53:37.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.775+0000 7f5ebdd9b640 1 -- 172.21.15.139:0/3339314433 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0009860 con 0x7f5eac0039b0 2024-04-25T09:53:37.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.775+0000 7f5ec4be0640 1 -- 172.21.15.139:0/3339314433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eac0039b0 msgr2=0x7f5eac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.775+0000 7f5ec4be0640 1 --2- 172.21.15.139:0/3339314433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eac0039b0 0x7f5eac003dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009c90 tx=0x7f5eac005830 comp rx=0 tx=0).stop 2024-04-25T09:53:37.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.775+0000 7f5ebed9d640 1 -- 172.21.15.139:0/3339314433 reap_dead start 2024-04-25T09:53:37.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.775+0000 7f5ec4be0640 1 -- 172.21.15.139:0/3339314433 shutdown_connections 2024-04-25T09:53:37.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.775+0000 7f5ec4be0640 1 -- 172.21.15.139:0/3339314433 >> 172.21.15.139:0/3339314433 conn(0x7f5ec00b5b00 msgr2=0x7f5ec00d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.775+0000 7f5ec4be0640 1 -- 172.21.15.139:0/3339314433 shutdown_connections 2024-04-25T09:53:37.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.775+0000 7f5ec4be0640 1 -- 172.21.15.139:0/3339314433 wait complete. 2024-04-25T09:53:37.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.776+0000 7f5ec4be0640 1 Processor -- start 2024-04-25T09:53:37.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.776+0000 7f5ec4be0640 1 -- start start 2024-04-25T09:53:37.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ec4be0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eac0039b0 0x7f5ec0147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ec4be0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec00cb9f0 0x7f5ec00cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ec4be0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00b8780 0x7f5ec00b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ec4be0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00d21f0 con 0x7f5eac0039b0 2024-04-25T09:53:37.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ec4be0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00b2d30 con 0x7f5ec00cb9f0 2024-04-25T09:53:37.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ec4be0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec01615c0 con 0x7f5ec00b8780 2024-04-25T09:53:37.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ebe59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eac0039b0 0x7f5ec0147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ebe59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eac0039b0 0x7f5ec0147b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40308/0 (socket says 172.21.15.139:40308) 2024-04-25T09:53:37.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ebe59c640 1 -- 172.21.15.139:0/1601660217 learned_addr learned my addr 172.21.15.139:0/1601660217 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:37.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ebed9d640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00b8780 0x7f5ec00b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5eb7fff640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec00cb9f0 0x7f5ec00cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ebe59c640 1 -- 172.21.15.139:0/1601660217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00b8780 msgr2=0x7f5ec00b8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ebe59c640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00b8780 0x7f5ec00b8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ebe59c640 1 -- 172.21.15.139:0/1601660217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec00cb9f0 msgr2=0x7f5ec00cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.777+0000 7f5ebe59c640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec00cb9f0 0x7f5ec00cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.778+0000 7f5ebe59c640 1 -- 172.21.15.139:0/1601660217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec015ed20 con 0x7f5eac0039b0 2024-04-25T09:53:37.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.778+0000 7f5ebed9d640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00b8780 0x7f5ec00b8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:37.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.778+0000 7f5eb7fff640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec00cb9f0 0x7f5ec00cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:37.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.778+0000 7f5ebe59c640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eac0039b0 0x7f5ec0147b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009db0 tx=0x7f5eb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:37.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.778+0000 7f5eb6ffd640 1 -- 172.21.15.139:0/1601660217 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0040020 con 0x7f5eac0039b0 2024-04-25T09:53:37.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.778+0000 7f5eb6ffd640 1 -- 172.21.15.139:0/1601660217 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb003b6a0 con 0x7f5eac0039b0 2024-04-25T09:53:37.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.778+0000 7f5ec4be0640 1 -- 172.21.15.139:0/1601660217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec015f000 con 0x7f5eac0039b0 2024-04-25T09:53:37.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.778+0000 7f5eb6ffd640 1 -- 172.21.15.139:0/1601660217 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0036040 con 0x7f5eac0039b0 2024-04-25T09:53:37.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.778+0000 7f5ec4be0640 1 -- 172.21.15.139:0/1601660217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec015f5c0 con 0x7f5eac0039b0 2024-04-25T09:53:37.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.779+0000 7f5ec4be0640 1 -- 172.21.15.139:0/1601660217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec00cfbd0 con 0x7f5eac0039b0 2024-04-25T09:53:37.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.779+0000 7f5eb6ffd640 1 -- 172.21.15.139:0/1601660217 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5eb0043020 con 0x7f5eac0039b0 2024-04-25T09:53:37.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.780+0000 7f5eb6ffd640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5ea006b500 0x7f5ea006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:37.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.780+0000 7f5eb6ffd640 1 -- 172.21.15.139:0/1601660217 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5eb00aaaf0 con 0x7f5eac0039b0 2024-04-25T09:53:37.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.780+0000 7f5eb7fff640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5ea006b500 0x7f5ea006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:37.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.780+0000 7f5eb7fff640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5ea006b500 0x7f5ea006d9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ec01f7130 tx=0x7f5ea8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:37.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.784+0000 7f5eb6ffd640 1 -- 172.21.15.139:0/1601660217 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ec00cfbd0 con 0x7f5eac0039b0 2024-04-25T09:53:37.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.950+0000 7f5ec4be0640 1 -- 172.21.15.139:0/1601660217 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ec0074520 con 0x7f5ea006b500 2024-04-25T09:53:37.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.957+0000 7f5eb6ffd640 1 -- 172.21.15.139:0/1601660217 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f5ec0074520 con 0x7f5ea006b500 2024-04-25T09:53:37.963 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:37.965 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:53:37.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.961+0000 7f5ec4be0640 1 -- 172.21.15.139:0/1601660217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5ea006b500 msgr2=0x7f5ea006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.961+0000 7f5ec4be0640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5ea006b500 0x7f5ea006d9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ec01f7130 tx=0x7f5ea8008040 comp rx=0 tx=0).stop 2024-04-25T09:53:37.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.961+0000 7f5ec4be0640 1 -- 172.21.15.139:0/1601660217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eac0039b0 msgr2=0x7f5ec0147b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:37.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.961+0000 7f5ec4be0640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eac0039b0 0x7f5ec0147b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009db0 tx=0x7f5eb0004730 comp rx=0 tx=0).stop 2024-04-25T09:53:37.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.961+0000 7f5ec4be0640 1 -- 172.21.15.139:0/1601660217 shutdown_connections 2024-04-25T09:53:37.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.961+0000 7f5ec4be0640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5ea006b500 0x7f5ea006d9c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.961+0000 7f5ec4be0640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ec00b8780 0x7f5ec00b8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.961+0000 7f5ec4be0640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ec00cb9f0 0x7f5ec00cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.961+0000 7f5ec4be0640 1 --2- 172.21.15.139:0/1601660217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eac0039b0 0x7f5ec0147b50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:37.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.961+0000 7f5ec4be0640 1 -- 172.21.15.139:0/1601660217 >> 172.21.15.139:0/1601660217 conn(0x7f5ec00b5b00 msgr2=0x7f5ec006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:37.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.961+0000 7f5ec4be0640 1 -- 172.21.15.139:0/1601660217 shutdown_connections 2024-04-25T09:53:37.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:37.961+0000 7f5ec4be0640 1 -- 172.21.15.139:0/1601660217 wait complete. 2024-04-25T09:53:37.974 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-25T09:53:36.971686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002675"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302246+0000","last_change":"2024-04-25T09:53:14.712100+0000","last_active":"2024-04-25T09:53:21.302246+0000","last_peered":"2024-04-25T09:53:21.302246+0000","last_clean":"2024-04-25T09:53:21.302246+0000","last_became_active":"2024-04-25T09:53:14.711827+0000","last_became_peered":"2024-04-25T09:53:14.711827+0000","last_unstale":"2024-04-25T09:53:21.302246+0000","last_undegraded":"2024-04-25T09:53:21.302246+0000","last_fullsized":"2024-04-25T09:53:21.302246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:57.295010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302544+0000","last_change":"2024-04-25T09:53:15.723415+0000","last_active":"2024-04-25T09:53:21.302544+0000","last_peered":"2024-04-25T09:53:21.302544+0000","last_clean":"2024-04-25T09:53:21.302544+0000","last_became_active":"2024-04-25T09:53:15.723126+0000","last_became_peered":"2024-04-25T09:53:15.723126+0000","last_unstale":"2024-04-25T09:53:21.302544+0000","last_undegraded":"2024-04-25T09:53:21.302544+0000","last_fullsized":"2024-04-25T09:53:21.302544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:53.040580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303119+0000","last_change":"2024-04-25T09:53:19.938169+0000","last_active":"2024-04-25T09:53:21.303119+0000","last_peered":"2024-04-25T09:53:21.303119+0000","last_clean":"2024-04-25T09:53:21.303119+0000","last_became_active":"2024-04-25T09:53:19.937831+0000","last_became_peered":"2024-04-25T09:53:19.937831+0000","last_unstale":"2024-04-25T09:53:21.303119+0000","last_undegraded":"2024-04-25T09:53:21.303119+0000","last_fullsized":"2024-04-25T09:53:21.303119+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:05.087194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302754+0000","last_change":"2024-04-25T09:53:14.708452+0000","last_active":"2024-04-25T09:53:21.302754+0000","last_peered":"2024-04-25T09:53:21.302754+0000","last_clean":"2024-04-25T09:53:21.302754+0000","last_became_active":"2024-04-25T09:53:14.708168+0000","last_became_peered":"2024-04-25T09:53:14.708168+0000","last_unstale":"2024-04-25T09:53:21.302754+0000","last_undegraded":"2024-04-25T09:53:21.302754+0000","last_fullsized":"2024-04-25T09:53:21.302754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:08.699272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301824+0000","last_change":"2024-04-25T09:53:15.731605+0000","last_active":"2024-04-25T09:53:21.301824+0000","last_peered":"2024-04-25T09:53:21.301824+0000","last_clean":"2024-04-25T09:53:21.301824+0000","last_became_active":"2024-04-25T09:53:15.731254+0000","last_became_peered":"2024-04-25T09:53:15.731254+0000","last_unstale":"2024-04-25T09:53:21.301824+0000","last_undegraded":"2024-04-25T09:53:21.301824+0000","last_fullsized":"2024-04-25T09:53:21.301824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742162+0000","last_change":"2024-04-25T09:53:19.756305+0000","last_active":"2024-04-25T09:53:20.742162+0000","last_peered":"2024-04-25T09:53:20.742162+0000","last_clean":"2024-04-25T09:53:20.742162+0000","last_became_active":"2024-04-25T09:53:19.755760+0000","last_became_peered":"2024-04-25T09:53:19.755760+0000","last_unstale":"2024-04-25T09:53:20.742162+0000","last_undegraded":"2024-04-25T09:53:20.742162+0000","last_fullsized":"2024-04-25T09:53:20.742162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:45.103063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303011+0000","last_change":"2024-04-25T09:53:14.713424+0000","last_active":"2024-04-25T09:53:21.303011+0000","last_peered":"2024-04-25T09:53:21.303011+0000","last_clean":"2024-04-25T09:53:21.303011+0000","last_became_active":"2024-04-25T09:53:14.711564+0000","last_became_peered":"2024-04-25T09:53:14.711564+0000","last_unstale":"2024-04-25T09:53:21.303011+0000","last_undegraded":"2024-04-25T09:53:21.303011+0000","last_fullsized":"2024-04-25T09:53:21.303011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:02.525834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302665+0000","last_change":"2024-04-25T09:53:15.737042+0000","last_active":"2024-04-25T09:53:21.302665+0000","last_peered":"2024-04-25T09:53:21.302665+0000","last_clean":"2024-04-25T09:53:21.302665+0000","last_became_active":"2024-04-25T09:53:15.736770+0000","last_became_peered":"2024-04-25T09:53:15.736770+0000","last_unstale":"2024-04-25T09:53:21.302665+0000","last_undegraded":"2024-04-25T09:53:21.302665+0000","last_fullsized":"2024-04-25T09:53:21.302665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:57.316572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742503+0000","last_change":"2024-04-25T09:53:19.761920+0000","last_active":"2024-04-25T09:53:20.742503+0000","last_peered":"2024-04-25T09:53:20.742503+0000","last_clean":"2024-04-25T09:53:20.742503+0000","last_became_active":"2024-04-25T09:53:19.761083+0000","last_became_peered":"2024-04-25T09:53:19.761083+0000","last_unstale":"2024-04-25T09:53:20.742503+0000","last_undegraded":"2024-04-25T09:53:20.742503+0000","last_fullsized":"2024-04-25T09:53:20.742503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:57.080146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301800+0000","last_change":"2024-04-25T09:53:14.713667+0000","last_active":"2024-04-25T09:53:21.301800+0000","last_peered":"2024-04-25T09:53:21.301800+0000","last_clean":"2024-04-25T09:53:21.301800+0000","last_became_active":"2024-04-25T09:53:14.713327+0000","last_became_peered":"2024-04-25T09:53:14.713327+0000","last_unstale":"2024-04-25T09:53:21.301800+0000","last_undegraded":"2024-04-25T09:53:21.301800+0000","last_fullsized":"2024-04-25T09:53:21.301800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:26.183785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302970+0000","last_change":"2024-04-25T09:53:15.737782+0000","last_active":"2024-04-25T09:53:21.302970+0000","last_peered":"2024-04-25T09:53:21.302970+0000","last_clean":"2024-04-25T09:53:21.302970+0000","last_became_active":"2024-04-25T09:53:15.737531+0000","last_became_peered":"2024-04-25T09:53:15.737531+0000","last_unstale":"2024-04-25T09:53:21.302970+0000","last_undegraded":"2024-04-25T09:53:21.302970+0000","last_fullsized":"2024-04-25T09:53:21.302970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:55.029099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742249+0000","last_change":"2024-04-25T09:53:19.754738+0000","last_active":"2024-04-25T09:53:20.742249+0000","last_peered":"2024-04-25T09:53:20.742249+0000","last_clean":"2024-04-25T09:53:20.742249+0000","last_became_active":"2024-04-25T09:53:19.754248+0000","last_became_peered":"2024-04-25T09:53:19.754248+0000","last_unstale":"2024-04-25T09:53:20.742249+0000","last_undegraded":"2024-04-25T09:53:20.742249+0000","last_fullsized":"2024-04-25T09:53:20.742249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:55.483582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302991+0000","last_change":"2024-04-25T09:53:15.729322+0000","last_active":"2024-04-25T09:53:21.302991+0000","last_peered":"2024-04-25T09:53:21.302991+0000","last_clean":"2024-04-25T09:53:21.302991+0000","last_became_active":"2024-04-25T09:53:15.729037+0000","last_became_peered":"2024-04-25T09:53:15.729037+0000","last_unstale":"2024-04-25T09:53:21.302991+0000","last_undegraded":"2024-04-25T09:53:21.302991+0000","last_fullsized":"2024-04-25T09:53:21.302991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302405+0000","last_change":"2024-04-25T09:53:14.712891+0000","last_active":"2024-04-25T09:53:21.302405+0000","last_peered":"2024-04-25T09:53:21.302405+0000","last_clean":"2024-04-25T09:53:21.302405+0000","last_became_active":"2024-04-25T09:53:14.712375+0000","last_became_peered":"2024-04-25T09:53:14.712375+0000","last_unstale":"2024-04-25T09:53:21.302405+0000","last_undegraded":"2024-04-25T09:53:21.302405+0000","last_fullsized":"2024-04-25T09:53:21.302405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:58.733069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743015+0000","last_change":"2024-04-25T09:53:19.762022+0000","last_active":"2024-04-25T09:53:20.743015+0000","last_peered":"2024-04-25T09:53:20.743015+0000","last_clean":"2024-04-25T09:53:20.743015+0000","last_became_active":"2024-04-25T09:53:19.761624+0000","last_became_peered":"2024-04-25T09:53:19.761624+0000","last_unstale":"2024-04-25T09:53:20.743015+0000","last_undegraded":"2024-04-25T09:53:20.743015+0000","last_fullsized":"2024-04-25T09:53:20.743015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:01.356138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302197+0000","last_change":"2024-04-25T09:53:15.720406+0000","last_active":"2024-04-25T09:53:21.302197+0000","last_peered":"2024-04-25T09:53:21.302197+0000","last_clean":"2024-04-25T09:53:21.302197+0000","last_became_active":"2024-04-25T09:53:15.720140+0000","last_became_peered":"2024-04-25T09:53:15.720140+0000","last_unstale":"2024-04-25T09:53:21.302197+0000","last_undegraded":"2024-04-25T09:53:21.302197+0000","last_fullsized":"2024-04-25T09:53:21.302197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302882+0000","last_change":"2024-04-25T09:53:14.705015+0000","last_active":"2024-04-25T09:53:21.302882+0000","last_peered":"2024-04-25T09:53:21.302882+0000","last_clean":"2024-04-25T09:53:21.302882+0000","last_became_active":"2024-04-25T09:53:14.704705+0000","last_became_peered":"2024-04-25T09:53:14.704705+0000","last_unstale":"2024-04-25T09:53:21.302882+0000","last_undegraded":"2024-04-25T09:53:21.302882+0000","last_fullsized":"2024-04-25T09:53:21.302882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743231+0000","last_change":"2024-04-25T09:53:19.761275+0000","last_active":"2024-04-25T09:53:20.743231+0000","last_peered":"2024-04-25T09:53:20.743231+0000","last_clean":"2024-04-25T09:53:20.743231+0000","last_became_active":"2024-04-25T09:53:19.760859+0000","last_became_peered":"2024-04-25T09:53:19.760859+0000","last_unstale":"2024-04-25T09:53:20.743231+0000","last_undegraded":"2024-04-25T09:53:20.743231+0000","last_fullsized":"2024-04-25T09:53:20.743231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:20.608892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301802+0000","last_change":"2024-04-25T09:53:15.726544+0000","last_active":"2024-04-25T09:53:21.301802+0000","last_peered":"2024-04-25T09:53:21.301802+0000","last_clean":"2024-04-25T09:53:21.301802+0000","last_became_active":"2024-04-25T09:53:15.726267+0000","last_became_peered":"2024-04-25T09:53:15.726267+0000","last_unstale":"2024-04-25T09:53:21.301802+0000","last_undegraded":"2024-04-25T09:53:21.301802+0000","last_fullsized":"2024-04-25T09:53:21.301802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301803+0000","last_change":"2024-04-25T09:53:14.712292+0000","last_active":"2024-04-25T09:53:21.301803+0000","last_peered":"2024-04-25T09:53:21.301803+0000","last_clean":"2024-04-25T09:53:21.301803+0000","last_became_active":"2024-04-25T09:53:14.711702+0000","last_became_peered":"2024-04-25T09:53:14.711702+0000","last_unstale":"2024-04-25T09:53:21.301803+0000","last_undegraded":"2024-04-25T09:53:21.301803+0000","last_fullsized":"2024-04-25T09:53:21.301803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303737+0000","last_change":"2024-04-25T09:53:19.937513+0000","last_active":"2024-04-25T09:53:21.303737+0000","last_peered":"2024-04-25T09:53:21.303737+0000","last_clean":"2024-04-25T09:53:21.303737+0000","last_became_active":"2024-04-25T09:53:19.937076+0000","last_became_peered":"2024-04-25T09:53:19.937076+0000","last_unstale":"2024-04-25T09:53:21.303737+0000","last_undegraded":"2024-04-25T09:53:21.303737+0000","last_fullsized":"2024-04-25T09:53:21.303737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:50.372524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742399+0000","last_change":"2024-04-25T09:53:15.717251+0000","last_active":"2024-04-25T09:53:20.742399+0000","last_peered":"2024-04-25T09:53:20.742399+0000","last_clean":"2024-04-25T09:53:20.742399+0000","last_became_active":"2024-04-25T09:53:15.716981+0000","last_became_peered":"2024-04-25T09:53:15.716981+0000","last_unstale":"2024-04-25T09:53:20.742399+0000","last_undegraded":"2024-04-25T09:53:20.742399+0000","last_fullsized":"2024-04-25T09:53:20.742399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:08.082240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301576+0000","last_change":"2024-04-25T09:53:14.711486+0000","last_active":"2024-04-25T09:53:21.301576+0000","last_peered":"2024-04-25T09:53:21.301576+0000","last_clean":"2024-04-25T09:53:21.301576+0000","last_became_active":"2024-04-25T09:53:14.711090+0000","last_became_peered":"2024-04-25T09:53:14.711090+0000","last_unstale":"2024-04-25T09:53:21.301576+0000","last_undegraded":"2024-04-25T09:53:21.301576+0000","last_fullsized":"2024-04-25T09:53:21.301576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742532+0000","last_change":"2024-04-25T09:53:19.756221+0000","last_active":"2024-04-25T09:53:20.742532+0000","last_peered":"2024-04-25T09:53:20.742532+0000","last_clean":"2024-04-25T09:53:20.742532+0000","last_became_active":"2024-04-25T09:53:19.755565+0000","last_became_peered":"2024-04-25T09:53:19.755565+0000","last_unstale":"2024-04-25T09:53:20.742532+0000","last_undegraded":"2024-04-25T09:53:20.742532+0000","last_fullsized":"2024-04-25T09:53:20.742532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:03.262744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302070+0000","last_change":"2024-04-25T09:53:15.720035+0000","last_active":"2024-04-25T09:53:21.302070+0000","last_peered":"2024-04-25T09:53:21.302070+0000","last_clean":"2024-04-25T09:53:21.302070+0000","last_became_active":"2024-04-25T09:53:15.719506+0000","last_became_peered":"2024-04-25T09:53:15.719506+0000","last_unstale":"2024-04-25T09:53:21.302070+0000","last_undegraded":"2024-04-25T09:53:21.302070+0000","last_fullsized":"2024-04-25T09:53:21.302070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303882+0000","last_change":"2024-04-25T09:53:14.714287+0000","last_active":"2024-04-25T09:53:21.303882+0000","last_peered":"2024-04-25T09:53:21.303882+0000","last_clean":"2024-04-25T09:53:21.303882+0000","last_became_active":"2024-04-25T09:53:14.714026+0000","last_became_peered":"2024-04-25T09:53:14.714026+0000","last_unstale":"2024-04-25T09:53:21.303882+0000","last_undegraded":"2024-04-25T09:53:21.303882+0000","last_fullsized":"2024-04-25T09:53:21.303882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742315+0000","last_change":"2024-04-25T09:53:19.934699+0000","last_active":"2024-04-25T09:53:20.742315+0000","last_peered":"2024-04-25T09:53:20.742315+0000","last_clean":"2024-04-25T09:53:20.742315+0000","last_became_active":"2024-04-25T09:53:19.934296+0000","last_became_peered":"2024-04-25T09:53:19.934296+0000","last_unstale":"2024-04-25T09:53:20.742315+0000","last_undegraded":"2024-04-25T09:53:20.742315+0000","last_fullsized":"2024-04-25T09:53:20.742315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:04.247738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302840+0000","last_change":"2024-04-25T09:53:15.735174+0000","last_active":"2024-04-25T09:53:21.302840+0000","last_peered":"2024-04-25T09:53:21.302840+0000","last_clean":"2024-04-25T09:53:21.302840+0000","last_became_active":"2024-04-25T09:53:15.734811+0000","last_became_peered":"2024-04-25T09:53:15.734811+0000","last_unstale":"2024-04-25T09:53:21.302840+0000","last_undegraded":"2024-04-25T09:53:21.302840+0000","last_fullsized":"2024-04-25T09:53:21.302840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:53.172005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303710+0000","last_change":"2024-04-25T09:53:14.707243+0000","last_active":"2024-04-25T09:53:21.303710+0000","last_peered":"2024-04-25T09:53:21.303710+0000","last_clean":"2024-04-25T09:53:21.303710+0000","last_became_active":"2024-04-25T09:53:14.706608+0000","last_became_peered":"2024-04-25T09:53:14.706608+0000","last_unstale":"2024-04-25T09:53:21.303710+0000","last_undegraded":"2024-04-25T09:53:21.303710+0000","last_fullsized":"2024-04-25T09:53:21.303710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:49.257995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301370+0000","last_change":"2024-04-25T09:53:19.938687+0000","last_active":"2024-04-25T09:53:21.301370+0000","last_peered":"2024-04-25T09:53:21.301370+0000","last_clean":"2024-04-25T09:53:21.301370+0000","last_became_active":"2024-04-25T09:53:19.938312+0000","last_became_peered":"2024-04-25T09:53:19.938312+0000","last_unstale":"2024-04-25T09:53:21.301370+0000","last_undegraded":"2024-04-25T09:53:21.301370+0000","last_fullsized":"2024-04-25T09:53:21.301370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:11.692337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302904+0000","last_change":"2024-04-25T09:53:15.724325+0000","last_active":"2024-04-25T09:53:21.302904+0000","last_peered":"2024-04-25T09:53:21.302904+0000","last_clean":"2024-04-25T09:53:21.302904+0000","last_became_active":"2024-04-25T09:53:15.724024+0000","last_became_peered":"2024-04-25T09:53:15.724024+0000","last_unstale":"2024-04-25T09:53:21.302904+0000","last_undegraded":"2024-04-25T09:53:21.302904+0000","last_fullsized":"2024-04-25T09:53:21.302904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303201+0000","last_change":"2024-04-25T09:53:14.712119+0000","last_active":"2024-04-25T09:53:21.303201+0000","last_peered":"2024-04-25T09:53:21.303201+0000","last_clean":"2024-04-25T09:53:21.303201+0000","last_became_active":"2024-04-25T09:53:14.711462+0000","last_became_peered":"2024-04-25T09:53:14.711462+0000","last_unstale":"2024-04-25T09:53:21.303201+0000","last_undegraded":"2024-04-25T09:53:21.303201+0000","last_fullsized":"2024-04-25T09:53:21.303201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:39.622554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302329+0000","last_change":"2024-04-25T09:53:19.934786+0000","last_active":"2024-04-25T09:53:21.302329+0000","last_peered":"2024-04-25T09:53:21.302329+0000","last_clean":"2024-04-25T09:53:21.302329+0000","last_became_active":"2024-04-25T09:53:19.934417+0000","last_became_peered":"2024-04-25T09:53:19.934417+0000","last_unstale":"2024-04-25T09:53:21.302329+0000","last_undegraded":"2024-04-25T09:53:21.302329+0000","last_fullsized":"2024-04-25T09:53:21.302329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:50.372524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303078+0000","last_change":"2024-04-25T09:53:15.731430+0000","last_active":"2024-04-25T09:53:21.303078+0000","last_peered":"2024-04-25T09:53:21.303078+0000","last_clean":"2024-04-25T09:53:21.303078+0000","last_became_active":"2024-04-25T09:53:15.731152+0000","last_became_peered":"2024-04-25T09:53:15.731152+0000","last_unstale":"2024-04-25T09:53:21.303078+0000","last_undegraded":"2024-04-25T09:53:21.303078+0000","last_fullsized":"2024-04-25T09:53:21.303078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:46.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302987+0000","last_change":"2024-04-25T09:53:14.708418+0000","last_active":"2024-04-25T09:53:21.302987+0000","last_peered":"2024-04-25T09:53:21.302987+0000","last_clean":"2024-04-25T09:53:21.302987+0000","last_became_active":"2024-04-25T09:53:14.708161+0000","last_became_peered":"2024-04-25T09:53:14.708161+0000","last_unstale":"2024-04-25T09:53:21.302987+0000","last_undegraded":"2024-04-25T09:53:21.302987+0000","last_fullsized":"2024-04-25T09:53:21.302987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:58.733069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302535+0000","last_change":"2024-04-25T09:53:19.758749+0000","last_active":"2024-04-25T09:53:21.302535+0000","last_peered":"2024-04-25T09:53:21.302535+0000","last_clean":"2024-04-25T09:53:21.302535+0000","last_became_active":"2024-04-25T09:53:19.757529+0000","last_became_peered":"2024-04-25T09:53:19.757529+0000","last_unstale":"2024-04-25T09:53:21.302535+0000","last_undegraded":"2024-04-25T09:53:21.302535+0000","last_fullsized":"2024-04-25T09:53:21.302535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:06.760703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742905+0000","last_change":"2024-04-25T09:53:15.726160+0000","last_active":"2024-04-25T09:53:20.742905+0000","last_peered":"2024-04-25T09:53:20.742905+0000","last_clean":"2024-04-25T09:53:20.742905+0000","last_became_active":"2024-04-25T09:53:15.725846+0000","last_became_peered":"2024-04-25T09:53:15.725846+0000","last_unstale":"2024-04-25T09:53:20.742905+0000","last_undegraded":"2024-04-25T09:53:20.742905+0000","last_fullsized":"2024-04-25T09:53:20.742905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.262190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302405+0000","last_change":"2024-04-25T09:53:14.712606+0000","last_active":"2024-04-25T09:53:21.302405+0000","last_peered":"2024-04-25T09:53:21.302405+0000","last_clean":"2024-04-25T09:53:21.302405+0000","last_became_active":"2024-04-25T09:53:14.712327+0000","last_became_peered":"2024-04-25T09:53:14.712327+0000","last_unstale":"2024-04-25T09:53:21.302405+0000","last_undegraded":"2024-04-25T09:53:21.302405+0000","last_fullsized":"2024-04-25T09:53:21.302405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:49.257995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302949+0000","last_change":"2024-04-25T09:53:19.757694+0000","last_active":"2024-04-25T09:53:21.302949+0000","last_peered":"2024-04-25T09:53:21.302949+0000","last_clean":"2024-04-25T09:53:21.302949+0000","last_became_active":"2024-04-25T09:53:19.757293+0000","last_became_peered":"2024-04-25T09:53:19.757293+0000","last_unstale":"2024-04-25T09:53:21.302949+0000","last_undegraded":"2024-04-25T09:53:21.302949+0000","last_fullsized":"2024-04-25T09:53:21.302949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:12.794746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302692+0000","last_change":"2024-04-25T09:53:15.725158+0000","last_active":"2024-04-25T09:53:21.302692+0000","last_peered":"2024-04-25T09:53:21.302692+0000","last_clean":"2024-04-25T09:53:21.302692+0000","last_became_active":"2024-04-25T09:53:15.724890+0000","last_became_peered":"2024-04-25T09:53:15.724890+0000","last_unstale":"2024-04-25T09:53:21.302692+0000","last_undegraded":"2024-04-25T09:53:21.302692+0000","last_fullsized":"2024-04-25T09:53:21.302692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741185+0000","last_change":"2024-04-25T09:53:14.703999+0000","last_active":"2024-04-25T09:53:20.741185+0000","last_peered":"2024-04-25T09:53:20.741185+0000","last_clean":"2024-04-25T09:53:20.741185+0000","last_became_active":"2024-04-25T09:53:14.703714+0000","last_became_peered":"2024-04-25T09:53:14.703714+0000","last_unstale":"2024-04-25T09:53:20.741185+0000","last_undegraded":"2024-04-25T09:53:20.741185+0000","last_fullsized":"2024-04-25T09:53:20.741185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303908+0000","last_change":"2024-04-25T09:53:19.756345+0000","last_active":"2024-04-25T09:53:21.303908+0000","last_peered":"2024-04-25T09:53:21.303908+0000","last_clean":"2024-04-25T09:53:21.303908+0000","last_became_active":"2024-04-25T09:53:19.755915+0000","last_became_peered":"2024-04-25T09:53:19.755915+0000","last_unstale":"2024-04-25T09:53:21.303908+0000","last_undegraded":"2024-04-25T09:53:21.303908+0000","last_fullsized":"2024-04-25T09:53:21.303908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:54.084750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301435+0000","last_change":"2024-04-25T09:53:15.723323+0000","last_active":"2024-04-25T09:53:21.301435+0000","last_peered":"2024-04-25T09:53:21.301435+0000","last_clean":"2024-04-25T09:53:21.301435+0000","last_became_active":"2024-04-25T09:53:15.723019+0000","last_became_peered":"2024-04-25T09:53:15.723019+0000","last_unstale":"2024-04-25T09:53:21.301435+0000","last_undegraded":"2024-04-25T09:53:21.301435+0000","last_fullsized":"2024-04-25T09:53:21.301435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:43.951510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301698+0000","last_change":"2024-04-25T09:53:14.712642+0000","last_active":"2024-04-25T09:53:21.301698+0000","last_peered":"2024-04-25T09:53:21.301698+0000","last_clean":"2024-04-25T09:53:21.301698+0000","last_became_active":"2024-04-25T09:53:14.711778+0000","last_became_peered":"2024-04-25T09:53:14.711778+0000","last_unstale":"2024-04-25T09:53:21.301698+0000","last_undegraded":"2024-04-25T09:53:21.301698+0000","last_fullsized":"2024-04-25T09:53:21.301698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:49.257995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301888+0000","last_change":"2024-04-25T09:53:19.934697+0000","last_active":"2024-04-25T09:53:21.301888+0000","last_peered":"2024-04-25T09:53:21.301888+0000","last_clean":"2024-04-25T09:53:21.301888+0000","last_became_active":"2024-04-25T09:53:19.934259+0000","last_became_peered":"2024-04-25T09:53:19.934259+0000","last_unstale":"2024-04-25T09:53:21.301888+0000","last_undegraded":"2024-04-25T09:53:21.301888+0000","last_fullsized":"2024-04-25T09:53:21.301888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:12.794746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301861+0000","last_change":"2024-04-25T09:53:15.731434+0000","last_active":"2024-04-25T09:53:21.301861+0000","last_peered":"2024-04-25T09:53:21.301861+0000","last_clean":"2024-04-25T09:53:21.301861+0000","last_became_active":"2024-04-25T09:53:15.731139+0000","last_became_peered":"2024-04-25T09:53:15.731139+0000","last_unstale":"2024-04-25T09:53:21.301861+0000","last_undegraded":"2024-04-25T09:53:21.301861+0000","last_fullsized":"2024-04-25T09:53:21.301861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302366+0000","last_change":"2024-04-25T09:53:14.706419+0000","last_active":"2024-04-25T09:53:21.302366+0000","last_peered":"2024-04-25T09:53:21.302366+0000","last_clean":"2024-04-25T09:53:21.302366+0000","last_became_active":"2024-04-25T09:53:14.706020+0000","last_became_peered":"2024-04-25T09:53:14.706020+0000","last_unstale":"2024-04-25T09:53:21.302366+0000","last_undegraded":"2024-04-25T09:53:21.302366+0000","last_fullsized":"2024-04-25T09:53:21.302366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301803+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:53:21.301803+0000","last_peered":"2024-04-25T09:53:21.301803+0000","last_clean":"2024-04-25T09:53:21.301803+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:53:21.301803+0000","last_undegraded":"2024-04-25T09:53:21.301803+0000","last_fullsized":"2024-04-25T09:53:21.301803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303580+0000","last_change":"2024-04-25T09:53:19.937575+0000","last_active":"2024-04-25T09:53:21.303580+0000","last_peered":"2024-04-25T09:53:21.303580+0000","last_clean":"2024-04-25T09:53:21.303580+0000","last_became_active":"2024-04-25T09:53:19.937138+0000","last_became_peered":"2024-04-25T09:53:19.937138+0000","last_unstale":"2024-04-25T09:53:21.303580+0000","last_undegraded":"2024-04-25T09:53:21.303580+0000","last_fullsized":"2024-04-25T09:53:21.303580+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:01.356138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302839+0000","last_change":"2024-04-25T09:53:15.733512+0000","last_active":"2024-04-25T09:53:21.302839+0000","last_peered":"2024-04-25T09:53:21.302839+0000","last_clean":"2024-04-25T09:53:21.302839+0000","last_became_active":"2024-04-25T09:53:15.733224+0000","last_became_peered":"2024-04-25T09:53:15.733224+0000","last_unstale":"2024-04-25T09:53:21.302839+0000","last_undegraded":"2024-04-25T09:53:21.302839+0000","last_fullsized":"2024-04-25T09:53:21.302839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:57.316572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741420+0000","last_change":"2024-04-25T09:53:14.708846+0000","last_active":"2024-04-25T09:53:20.741420+0000","last_peered":"2024-04-25T09:53:20.741420+0000","last_clean":"2024-04-25T09:53:20.741420+0000","last_became_active":"2024-04-25T09:53:14.708587+0000","last_became_peered":"2024-04-25T09:53:14.708587+0000","last_unstale":"2024-04-25T09:53:20.741420+0000","last_undegraded":"2024-04-25T09:53:20.741420+0000","last_fullsized":"2024-04-25T09:53:20.741420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302923+0000","last_change":"2024-04-25T09:53:19.934818+0000","last_active":"2024-04-25T09:53:21.302923+0000","last_peered":"2024-04-25T09:53:21.302923+0000","last_clean":"2024-04-25T09:53:21.302923+0000","last_became_active":"2024-04-25T09:53:19.934403+0000","last_became_peered":"2024-04-25T09:53:19.934403+0000","last_unstale":"2024-04-25T09:53:21.302923+0000","last_undegraded":"2024-04-25T09:53:21.302923+0000","last_fullsized":"2024-04-25T09:53:21.302923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:57.211571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302766+0000","last_change":"2024-04-25T09:53:15.732566+0000","last_active":"2024-04-25T09:53:21.302766+0000","last_peered":"2024-04-25T09:53:21.302766+0000","last_clean":"2024-04-25T09:53:21.302766+0000","last_became_active":"2024-04-25T09:53:15.731920+0000","last_became_peered":"2024-04-25T09:53:15.731920+0000","last_unstale":"2024-04-25T09:53:21.302766+0000","last_undegraded":"2024-04-25T09:53:21.302766+0000","last_fullsized":"2024-04-25T09:53:21.302766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:53.172005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302249+0000","last_change":"2024-04-25T09:53:14.703268+0000","last_active":"2024-04-25T09:53:21.302249+0000","last_peered":"2024-04-25T09:53:21.302249+0000","last_clean":"2024-04-25T09:53:21.302249+0000","last_became_active":"2024-04-25T09:53:14.702989+0000","last_became_peered":"2024-04-25T09:53:14.702989+0000","last_unstale":"2024-04-25T09:53:21.302249+0000","last_undegraded":"2024-04-25T09:53:21.302249+0000","last_fullsized":"2024-04-25T09:53:21.302249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302491+0000","last_change":"2024-04-25T09:53:19.762847+0000","last_active":"2024-04-25T09:53:21.302491+0000","last_peered":"2024-04-25T09:53:21.302491+0000","last_clean":"2024-04-25T09:53:21.302491+0000","last_became_active":"2024-04-25T09:53:19.762336+0000","last_became_peered":"2024-04-25T09:53:19.762336+0000","last_unstale":"2024-04-25T09:53:21.302491+0000","last_undegraded":"2024-04-25T09:53:21.302491+0000","last_fullsized":"2024-04-25T09:53:21.302491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:54.084750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303390+0000","last_change":"2024-04-25T09:53:15.732927+0000","last_active":"2024-04-25T09:53:21.303390+0000","last_peered":"2024-04-25T09:53:21.303390+0000","last_clean":"2024-04-25T09:53:21.303390+0000","last_became_active":"2024-04-25T09:53:15.732640+0000","last_became_peered":"2024-04-25T09:53:15.732640+0000","last_unstale":"2024-04-25T09:53:21.303390+0000","last_undegraded":"2024-04-25T09:53:21.303390+0000","last_fullsized":"2024-04-25T09:53:21.303390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:27.493283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741538+0000","last_change":"2024-04-25T09:53:14.708266+0000","last_active":"2024-04-25T09:53:20.741538+0000","last_peered":"2024-04-25T09:53:20.741538+0000","last_clean":"2024-04-25T09:53:20.741538+0000","last_became_active":"2024-04-25T09:53:14.707980+0000","last_became_peered":"2024-04-25T09:53:14.707980+0000","last_unstale":"2024-04-25T09:53:20.741538+0000","last_undegraded":"2024-04-25T09:53:20.741538+0000","last_fullsized":"2024-04-25T09:53:20.741538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301745+0000","last_change":"2024-04-25T09:53:19.934601+0000","last_active":"2024-04-25T09:53:21.301745+0000","last_peered":"2024-04-25T09:53:21.301745+0000","last_clean":"2024-04-25T09:53:21.301745+0000","last_became_active":"2024-04-25T09:53:19.934185+0000","last_became_peered":"2024-04-25T09:53:19.934185+0000","last_unstale":"2024-04-25T09:53:21.301745+0000","last_undegraded":"2024-04-25T09:53:21.301745+0000","last_fullsized":"2024-04-25T09:53:21.301745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:05.087194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302564+0000","last_change":"2024-04-25T09:53:15.732497+0000","last_active":"2024-04-25T09:53:21.302564+0000","last_peered":"2024-04-25T09:53:21.302564+0000","last_clean":"2024-04-25T09:53:21.302564+0000","last_became_active":"2024-04-25T09:53:15.732229+0000","last_became_peered":"2024-04-25T09:53:15.732229+0000","last_unstale":"2024-04-25T09:53:21.302564+0000","last_undegraded":"2024-04-25T09:53:21.302564+0000","last_fullsized":"2024-04-25T09:53:21.302564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:08.755180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742750+0000","last_change":"2024-04-25T09:53:14.708424+0000","last_active":"2024-04-25T09:53:20.742750+0000","last_peered":"2024-04-25T09:53:20.742750+0000","last_clean":"2024-04-25T09:53:20.742750+0000","last_became_active":"2024-04-25T09:53:14.708139+0000","last_became_peered":"2024-04-25T09:53:14.708139+0000","last_unstale":"2024-04-25T09:53:20.742750+0000","last_undegraded":"2024-04-25T09:53:20.742750+0000","last_fullsized":"2024-04-25T09:53:20.742750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:11.395695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742138+0000","last_change":"2024-04-25T09:53:19.753647+0000","last_active":"2024-04-25T09:53:20.742138+0000","last_peered":"2024-04-25T09:53:20.742138+0000","last_clean":"2024-04-25T09:53:20.742138+0000","last_became_active":"2024-04-25T09:53:19.753091+0000","last_became_peered":"2024-04-25T09:53:19.753091+0000","last_unstale":"2024-04-25T09:53:20.742138+0000","last_undegraded":"2024-04-25T09:53:20.742138+0000","last_fullsized":"2024-04-25T09:53:20.742138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:58.778179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302186+0000","last_change":"2024-04-25T09:53:15.731871+0000","last_active":"2024-04-25T09:53:21.302186+0000","last_peered":"2024-04-25T09:53:21.302186+0000","last_clean":"2024-04-25T09:53:21.302186+0000","last_became_active":"2024-04-25T09:53:15.731615+0000","last_became_peered":"2024-04-25T09:53:15.731615+0000","last_unstale":"2024-04-25T09:53:21.302186+0000","last_undegraded":"2024-04-25T09:53:21.302186+0000","last_fullsized":"2024-04-25T09:53:21.302186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:53.172005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741071+0000","last_change":"2024-04-25T09:53:14.701334+0000","last_active":"2024-04-25T09:53:20.741071+0000","last_peered":"2024-04-25T09:53:20.741071+0000","last_clean":"2024-04-25T09:53:20.741071+0000","last_became_active":"2024-04-25T09:53:14.701062+0000","last_became_peered":"2024-04-25T09:53:14.701062+0000","last_unstale":"2024-04-25T09:53:20.741071+0000","last_undegraded":"2024-04-25T09:53:20.741071+0000","last_fullsized":"2024-04-25T09:53:20.741071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301727+0000","last_change":"2024-04-25T09:53:19.758482+0000","last_active":"2024-04-25T09:53:21.301727+0000","last_peered":"2024-04-25T09:53:21.301727+0000","last_clean":"2024-04-25T09:53:21.301727+0000","last_became_active":"2024-04-25T09:53:19.758036+0000","last_became_peered":"2024-04-25T09:53:19.758036+0000","last_unstale":"2024-04-25T09:53:21.301727+0000","last_undegraded":"2024-04-25T09:53:21.301727+0000","last_fullsized":"2024-04-25T09:53:21.301727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.068666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301688+0000","last_change":"2024-04-25T09:53:15.721989+0000","last_active":"2024-04-25T09:53:21.301688+0000","last_peered":"2024-04-25T09:53:21.301688+0000","last_clean":"2024-04-25T09:53:21.301688+0000","last_became_active":"2024-04-25T09:53:15.721636+0000","last_became_peered":"2024-04-25T09:53:15.721636+0000","last_unstale":"2024-04-25T09:53:21.301688+0000","last_undegraded":"2024-04-25T09:53:21.301688+0000","last_fullsized":"2024-04-25T09:53:21.301688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:39.445622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302694+0000","last_change":"2024-04-25T09:53:14.709691+0000","last_active":"2024-04-25T09:53:21.302694+0000","last_peered":"2024-04-25T09:53:21.302694+0000","last_clean":"2024-04-25T09:53:21.302694+0000","last_became_active":"2024-04-25T09:53:14.709373+0000","last_became_peered":"2024-04-25T09:53:14.709373+0000","last_unstale":"2024-04-25T09:53:21.302694+0000","last_undegraded":"2024-04-25T09:53:21.302694+0000","last_fullsized":"2024-04-25T09:53:21.302694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302719+0000","last_change":"2024-04-25T09:53:19.937412+0000","last_active":"2024-04-25T09:53:21.302719+0000","last_peered":"2024-04-25T09:53:21.302719+0000","last_clean":"2024-04-25T09:53:21.302719+0000","last_became_active":"2024-04-25T09:53:19.937068+0000","last_became_peered":"2024-04-25T09:53:19.937068+0000","last_unstale":"2024-04-25T09:53:21.302719+0000","last_undegraded":"2024-04-25T09:53:21.302719+0000","last_fullsized":"2024-04-25T09:53:21.302719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:18.654048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302364+0000","last_change":"2024-04-25T09:53:15.734287+0000","last_active":"2024-04-25T09:53:21.302364+0000","last_peered":"2024-04-25T09:53:21.302364+0000","last_clean":"2024-04-25T09:53:21.302364+0000","last_became_active":"2024-04-25T09:53:15.733835+0000","last_became_peered":"2024-04-25T09:53:15.733835+0000","last_unstale":"2024-04-25T09:53:21.302364+0000","last_undegraded":"2024-04-25T09:53:21.302364+0000","last_fullsized":"2024-04-25T09:53:21.302364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:55.029099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302389+0000","last_change":"2024-04-25T09:53:14.707323+0000","last_active":"2024-04-25T09:53:21.302389+0000","last_peered":"2024-04-25T09:53:21.302389+0000","last_clean":"2024-04-25T09:53:21.302389+0000","last_became_active":"2024-04-25T09:53:14.707061+0000","last_became_peered":"2024-04-25T09:53:14.707061+0000","last_unstale":"2024-04-25T09:53:21.302389+0000","last_undegraded":"2024-04-25T09:53:21.302389+0000","last_fullsized":"2024-04-25T09:53:21.302389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.266526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303418+0000","last_change":"2024-04-25T09:53:19.935315+0000","last_active":"2024-04-25T09:53:21.303418+0000","last_peered":"2024-04-25T09:53:21.303418+0000","last_clean":"2024-04-25T09:53:21.303418+0000","last_became_active":"2024-04-25T09:53:19.934800+0000","last_became_peered":"2024-04-25T09:53:19.934800+0000","last_unstale":"2024-04-25T09:53:21.303418+0000","last_undegraded":"2024-04-25T09:53:21.303418+0000","last_fullsized":"2024-04-25T09:53:21.303418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:50.137095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302044+0000","last_change":"2024-04-25T09:53:15.729897+0000","last_active":"2024-04-25T09:53:21.302044+0000","last_peered":"2024-04-25T09:53:21.302044+0000","last_clean":"2024-04-25T09:53:21.302044+0000","last_became_active":"2024-04-25T09:53:15.729269+0000","last_became_peered":"2024-04-25T09:53:15.729269+0000","last_unstale":"2024-04-25T09:53:21.302044+0000","last_undegraded":"2024-04-25T09:53:21.302044+0000","last_fullsized":"2024-04-25T09:53:21.302044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:55.387059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302564+0000","last_change":"2024-04-25T09:53:14.709636+0000","last_active":"2024-04-25T09:53:21.302564+0000","last_peered":"2024-04-25T09:53:21.302564+0000","last_clean":"2024-04-25T09:53:21.302564+0000","last_became_active":"2024-04-25T09:53:14.709366+0000","last_became_peered":"2024-04-25T09:53:14.709366+0000","last_unstale":"2024-04-25T09:53:21.302564+0000","last_undegraded":"2024-04-25T09:53:21.302564+0000","last_fullsized":"2024-04-25T09:53:21.302564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:05.869022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303066+0000","last_change":"2024-04-25T09:53:19.937074+0000","last_active":"2024-04-25T09:53:21.303066+0000","last_peered":"2024-04-25T09:53:21.303066+0000","last_clean":"2024-04-25T09:53:21.303066+0000","last_became_active":"2024-04-25T09:53:19.935313+0000","last_became_peered":"2024-04-25T09:53:19.935313+0000","last_unstale":"2024-04-25T09:53:21.303066+0000","last_undegraded":"2024-04-25T09:53:21.303066+0000","last_fullsized":"2024-04-25T09:53:21.303066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:56.116208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302820+0000","last_change":"2024-04-25T09:53:15.726822+0000","last_active":"2024-04-25T09:53:21.302820+0000","last_peered":"2024-04-25T09:53:21.302820+0000","last_clean":"2024-04-25T09:53:21.302820+0000","last_became_active":"2024-04-25T09:53:15.726526+0000","last_became_peered":"2024-04-25T09:53:15.726526+0000","last_unstale":"2024-04-25T09:53:21.302820+0000","last_undegraded":"2024-04-25T09:53:21.302820+0000","last_fullsized":"2024-04-25T09:53:21.302820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:16.569326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:24.817228+0000","last_change":"2024-04-25T09:53:14.707187+0000","last_active":"2024-04-25T09:53:24.817228+0000","last_peered":"2024-04-25T09:53:24.817228+0000","last_clean":"2024-04-25T09:53:24.817228+0000","last_became_active":"2024-04-25T09:53:14.706509+0000","last_became_peered":"2024-04-25T09:53:14.706509+0000","last_unstale":"2024-04-25T09:53:24.817228+0000","last_undegraded":"2024-04-25T09:53:24.817228+0000","last_fullsized":"2024-04-25T09:53:24.817228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302676+0000","last_change":"2024-04-25T09:53:19.758453+0000","last_active":"2024-04-25T09:53:21.302676+0000","last_peered":"2024-04-25T09:53:21.302676+0000","last_clean":"2024-04-25T09:53:21.302676+0000","last_became_active":"2024-04-25T09:53:19.757004+0000","last_became_peered":"2024-04-25T09:53:19.757004+0000","last_unstale":"2024-04-25T09:53:21.302676+0000","last_undegraded":"2024-04-25T09:53:21.302676+0000","last_fullsized":"2024-04-25T09:53:21.302676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.068666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743281+0000","last_change":"2024-04-25T09:53:15.731519+0000","last_active":"2024-04-25T09:53:20.743281+0000","last_peered":"2024-04-25T09:53:20.743281+0000","last_clean":"2024-04-25T09:53:20.743281+0000","last_became_active":"2024-04-25T09:53:15.731037+0000","last_became_peered":"2024-04-25T09:53:15.731037+0000","last_unstale":"2024-04-25T09:53:20.743281+0000","last_undegraded":"2024-04-25T09:53:20.743281+0000","last_fullsized":"2024-04-25T09:53:20.743281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.530029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302151+0000","last_change":"2024-04-25T09:53:14.713398+0000","last_active":"2024-04-25T09:53:21.302151+0000","last_peered":"2024-04-25T09:53:21.302151+0000","last_clean":"2024-04-25T09:53:21.302151+0000","last_became_active":"2024-04-25T09:53:14.713125+0000","last_became_peered":"2024-04-25T09:53:14.713125+0000","last_unstale":"2024-04-25T09:53:21.302151+0000","last_undegraded":"2024-04-25T09:53:21.302151+0000","last_fullsized":"2024-04-25T09:53:21.302151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:38.252466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742729+0000","last_change":"2024-04-25T09:53:19.762998+0000","last_active":"2024-04-25T09:53:20.742729+0000","last_peered":"2024-04-25T09:53:20.742729+0000","last_clean":"2024-04-25T09:53:20.742729+0000","last_became_active":"2024-04-25T09:53:19.762550+0000","last_became_peered":"2024-04-25T09:53:19.762550+0000","last_unstale":"2024-04-25T09:53:20.742729+0000","last_undegraded":"2024-04-25T09:53:20.742729+0000","last_fullsized":"2024-04-25T09:53:20.742729+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:57.211571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302003+0000","last_change":"2024-04-25T09:53:15.731707+0000","last_active":"2024-04-25T09:53:21.302003+0000","last_peered":"2024-04-25T09:53:21.302003+0000","last_clean":"2024-04-25T09:53:21.302003+0000","last_became_active":"2024-04-25T09:53:15.731303+0000","last_became_peered":"2024-04-25T09:53:15.731303+0000","last_unstale":"2024-04-25T09:53:21.302003+0000","last_undegraded":"2024-04-25T09:53:21.302003+0000","last_fullsized":"2024-04-25T09:53:21.302003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302319+0000","last_change":"2024-04-25T09:53:14.714447+0000","last_active":"2024-04-25T09:53:21.302319+0000","last_peered":"2024-04-25T09:53:21.302319+0000","last_clean":"2024-04-25T09:53:21.302319+0000","last_became_active":"2024-04-25T09:53:14.714155+0000","last_became_peered":"2024-04-25T09:53:14.714155+0000","last_unstale":"2024-04-25T09:53:21.302319+0000","last_undegraded":"2024-04-25T09:53:21.302319+0000","last_fullsized":"2024-04-25T09:53:21.302319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301664+0000","last_change":"2024-04-25T09:53:19.761513+0000","last_active":"2024-04-25T09:53:21.301664+0000","last_peered":"2024-04-25T09:53:21.301664+0000","last_clean":"2024-04-25T09:53:21.301664+0000","last_became_active":"2024-04-25T09:53:19.761037+0000","last_became_peered":"2024-04-25T09:53:19.761037+0000","last_unstale":"2024-04-25T09:53:21.301664+0000","last_undegraded":"2024-04-25T09:53:21.301664+0000","last_fullsized":"2024-04-25T09:53:21.301664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:31.532850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302218+0000","last_change":"2024-04-25T09:53:15.742482+0000","last_active":"2024-04-25T09:53:21.302218+0000","last_peered":"2024-04-25T09:53:21.302218+0000","last_clean":"2024-04-25T09:53:21.302218+0000","last_became_active":"2024-04-25T09:53:15.742170+0000","last_became_peered":"2024-04-25T09:53:15.742170+0000","last_unstale":"2024-04-25T09:53:21.302218+0000","last_undegraded":"2024-04-25T09:53:21.302218+0000","last_fullsized":"2024-04-25T09:53:21.302218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:16.569326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303779+0000","last_change":"2024-04-25T09:53:14.714376+0000","last_active":"2024-04-25T09:53:21.303779+0000","last_peered":"2024-04-25T09:53:21.303779+0000","last_clean":"2024-04-25T09:53:21.303779+0000","last_became_active":"2024-04-25T09:53:14.714116+0000","last_became_peered":"2024-04-25T09:53:14.714116+0000","last_unstale":"2024-04-25T09:53:21.303779+0000","last_undegraded":"2024-04-25T09:53:21.303779+0000","last_fullsized":"2024-04-25T09:53:21.303779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303146+0000","last_change":"2024-04-25T09:53:19.938076+0000","last_active":"2024-04-25T09:53:21.303146+0000","last_peered":"2024-04-25T09:53:21.303146+0000","last_clean":"2024-04-25T09:53:21.303146+0000","last_became_active":"2024-04-25T09:53:19.937704+0000","last_became_peered":"2024-04-25T09:53:19.937704+0000","last_unstale":"2024-04-25T09:53:21.303146+0000","last_undegraded":"2024-04-25T09:53:21.303146+0000","last_fullsized":"2024-04-25T09:53:21.303146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:56.116208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301567+0000","last_change":"2024-04-25T09:53:19.936514+0000","last_active":"2024-04-25T09:53:21.301567+0000","last_peered":"2024-04-25T09:53:21.301567+0000","last_clean":"2024-04-25T09:53:21.301567+0000","last_became_active":"2024-04-25T09:53:19.936095+0000","last_became_peered":"2024-04-25T09:53:19.936095+0000","last_unstale":"2024-04-25T09:53:21.301567+0000","last_undegraded":"2024-04-25T09:53:21.301567+0000","last_fullsized":"2024-04-25T09:53:21.301567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:06.760703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301623+0000","last_change":"2024-04-25T09:53:15.726524+0000","last_active":"2024-04-25T09:53:21.301623+0000","last_peered":"2024-04-25T09:53:21.301623+0000","last_clean":"2024-04-25T09:53:21.301623+0000","last_became_active":"2024-04-25T09:53:15.726160+0000","last_became_peered":"2024-04-25T09:53:15.726160+0000","last_unstale":"2024-04-25T09:53:21.301623+0000","last_undegraded":"2024-04-25T09:53:21.301623+0000","last_fullsized":"2024-04-25T09:53:21.301623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302003+0000","last_change":"2024-04-25T09:53:14.704589+0000","last_active":"2024-04-25T09:53:21.302003+0000","last_peered":"2024-04-25T09:53:21.302003+0000","last_clean":"2024-04-25T09:53:21.302003+0000","last_became_active":"2024-04-25T09:53:14.704228+0000","last_became_peered":"2024-04-25T09:53:14.704228+0000","last_unstale":"2024-04-25T09:53:21.302003+0000","last_undegraded":"2024-04-25T09:53:21.302003+0000","last_fullsized":"2024-04-25T09:53:21.302003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:54.843493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303831+0000","last_change":"2024-04-25T09:53:19.761677+0000","last_active":"2024-04-25T09:53:21.303831+0000","last_peered":"2024-04-25T09:53:21.303831+0000","last_clean":"2024-04-25T09:53:21.303831+0000","last_became_active":"2024-04-25T09:53:19.761276+0000","last_became_peered":"2024-04-25T09:53:19.761276+0000","last_unstale":"2024-04-25T09:53:21.303831+0000","last_undegraded":"2024-04-25T09:53:21.303831+0000","last_fullsized":"2024-04-25T09:53:21.303831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:20.608892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301714+0000","last_change":"2024-04-25T09:53:15.722486+0000","last_active":"2024-04-25T09:53:21.301714+0000","last_peered":"2024-04-25T09:53:21.301714+0000","last_clean":"2024-04-25T09:53:21.301714+0000","last_became_active":"2024-04-25T09:53:15.722225+0000","last_became_peered":"2024-04-25T09:53:15.722225+0000","last_unstale":"2024-04-25T09:53:21.301714+0000","last_undegraded":"2024-04-25T09:53:21.301714+0000","last_fullsized":"2024-04-25T09:53:21.301714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303275+0000","last_change":"2024-04-25T09:53:14.707399+0000","last_active":"2024-04-25T09:53:21.303275+0000","last_peered":"2024-04-25T09:53:21.303275+0000","last_clean":"2024-04-25T09:53:21.303275+0000","last_became_active":"2024-04-25T09:53:14.707078+0000","last_became_peered":"2024-04-25T09:53:14.707078+0000","last_unstale":"2024-04-25T09:53:21.303275+0000","last_undegraded":"2024-04-25T09:53:21.303275+0000","last_fullsized":"2024-04-25T09:53:21.303275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:57.295010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303180+0000","last_change":"2024-04-25T09:53:14.711893+0000","last_active":"2024-04-25T09:53:21.303180+0000","last_peered":"2024-04-25T09:53:21.303180+0000","last_clean":"2024-04-25T09:53:21.303180+0000","last_became_active":"2024-04-25T09:53:14.711513+0000","last_became_peered":"2024-04-25T09:53:14.711513+0000","last_unstale":"2024-04-25T09:53:21.303180+0000","last_undegraded":"2024-04-25T09:53:21.303180+0000","last_fullsized":"2024-04-25T09:53:21.303180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.078045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302544+0000","last_change":"2024-04-25T09:53:15.728536+0000","last_active":"2024-04-25T09:53:21.302544+0000","last_peered":"2024-04-25T09:53:21.302544+0000","last_clean":"2024-04-25T09:53:21.302544+0000","last_became_active":"2024-04-25T09:53:15.728235+0000","last_became_peered":"2024-04-25T09:53:15.728235+0000","last_unstale":"2024-04-25T09:53:21.302544+0000","last_undegraded":"2024-04-25T09:53:21.302544+0000","last_fullsized":"2024-04-25T09:53:21.302544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741367+0000","last_change":"2024-04-25T09:53:19.764626+0000","last_active":"2024-04-25T09:53:20.741367+0000","last_peered":"2024-04-25T09:53:20.741367+0000","last_clean":"2024-04-25T09:53:20.741367+0000","last_became_active":"2024-04-25T09:53:19.763954+0000","last_became_peered":"2024-04-25T09:53:19.763954+0000","last_unstale":"2024-04-25T09:53:20.741367+0000","last_undegraded":"2024-04-25T09:53:20.741367+0000","last_fullsized":"2024-04-25T09:53:20.741367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:47.991077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302285+0000","last_change":"2024-04-25T09:53:14.712000+0000","last_active":"2024-04-25T09:53:21.302285+0000","last_peered":"2024-04-25T09:53:21.302285+0000","last_clean":"2024-04-25T09:53:21.302285+0000","last_became_active":"2024-04-25T09:53:14.711712+0000","last_became_peered":"2024-04-25T09:53:14.711712+0000","last_unstale":"2024-04-25T09:53:21.302285+0000","last_undegraded":"2024-04-25T09:53:21.302285+0000","last_fullsized":"2024-04-25T09:53:21.302285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:13.683168+0000","last_clean_scrub_stamp":"2024-04-25T09:53:13.683168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.742400+0000","last_change":"2024-04-25T09:53:15.722357+0000","last_active":"2024-04-25T09:53:20.742400+0000","last_peered":"2024-04-25T09:53:20.742400+0000","last_clean":"2024-04-25T09:53:20.742400+0000","last_became_active":"2024-04-25T09:53:15.722082+0000","last_became_peered":"2024-04-25T09:53:15.722082+0000","last_unstale":"2024-04-25T09:53:20.742400+0000","last_undegraded":"2024-04-25T09:53:20.742400+0000","last_fullsized":"2024-04-25T09:53:20.742400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:39.270721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303146+0000","last_change":"2024-04-25T09:53:19.759080+0000","last_active":"2024-04-25T09:53:21.303146+0000","last_peered":"2024-04-25T09:53:21.303146+0000","last_clean":"2024-04-25T09:53:21.303146+0000","last_became_active":"2024-04-25T09:53:19.758705+0000","last_became_peered":"2024-04-25T09:53:19.758705+0000","last_unstale":"2024-04-25T09:53:21.303146+0000","last_undegraded":"2024-04-25T09:53:21.303146+0000","last_fullsized":"2024-04-25T09:53:21.303146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:18.726929+0000","last_clean_scrub_stamp":"2024-04-25T09:53:18.726929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:54.228875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301484+0000","last_change":"2024-04-25T09:53:15.723965+0000","last_active":"2024-04-25T09:53:21.301484+0000","last_peered":"2024-04-25T09:53:21.301484+0000","last_clean":"2024-04-25T09:53:21.301484+0000","last_became_active":"2024-04-25T09:53:15.723660+0000","last_became_peered":"2024-04-25T09:53:15.723660+0000","last_unstale":"2024-04-25T09:53:21.301484+0000","last_undegraded":"2024-04-25T09:53:21.301484+0000","last_fullsized":"2024-04-25T09:53:21.301484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:04.221889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741222+0000","last_change":"2024-04-25T09:53:15.734494+0000","last_active":"2024-04-25T09:53:20.741222+0000","last_peered":"2024-04-25T09:53:20.741222+0000","last_clean":"2024-04-25T09:53:20.741222+0000","last_became_active":"2024-04-25T09:53:15.734204+0000","last_became_peered":"2024-04-25T09:53:15.734204+0000","last_unstale":"2024-04-25T09:53:20.741222+0000","last_undegraded":"2024-04-25T09:53:20.741222+0000","last_fullsized":"2024-04-25T09:53:20.741222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.530029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302296+0000","last_change":"2024-04-25T09:53:15.731317+0000","last_active":"2024-04-25T09:53:21.302296+0000","last_peered":"2024-04-25T09:53:21.302296+0000","last_clean":"2024-04-25T09:53:21.302296+0000","last_became_active":"2024-04-25T09:53:15.731022+0000","last_became_peered":"2024-04-25T09:53:15.731022+0000","last_unstale":"2024-04-25T09:53:21.302296+0000","last_undegraded":"2024-04-25T09:53:21.302296+0000","last_fullsized":"2024-04-25T09:53:21.302296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:02.721136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302472+0000","last_change":"2024-04-25T09:53:15.720872+0000","last_active":"2024-04-25T09:53:21.302472+0000","last_peered":"2024-04-25T09:53:21.302472+0000","last_clean":"2024-04-25T09:53:21.302472+0000","last_became_active":"2024-04-25T09:53:15.720242+0000","last_became_peered":"2024-04-25T09:53:15.720242+0000","last_unstale":"2024-04-25T09:53:21.302472+0000","last_undegraded":"2024-04-25T09:53:21.302472+0000","last_fullsized":"2024-04-25T09:53:21.302472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303531+0000","last_change":"2024-04-25T09:53:15.728500+0000","last_active":"2024-04-25T09:53:21.303531+0000","last_peered":"2024-04-25T09:53:21.303531+0000","last_clean":"2024-04-25T09:53:21.303531+0000","last_became_active":"2024-04-25T09:53:15.726998+0000","last_became_peered":"2024-04-25T09:53:15.726998+0000","last_unstale":"2024-04-25T09:53:21.303531+0000","last_undegraded":"2024-04-25T09:53:21.303531+0000","last_fullsized":"2024-04-25T09:53:21.303531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:00.208172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301991+0000","last_change":"2024-04-25T09:53:15.726337+0000","last_active":"2024-04-25T09:53:21.301991+0000","last_peered":"2024-04-25T09:53:21.301991+0000","last_clean":"2024-04-25T09:53:21.301991+0000","last_became_active":"2024-04-25T09:53:15.726072+0000","last_became_peered":"2024-04-25T09:53:15.726072+0000","last_unstale":"2024-04-25T09:53:21.301991+0000","last_undegraded":"2024-04-25T09:53:21.301991+0000","last_fullsized":"2024-04-25T09:53:21.301991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303810+0000","last_change":"2024-04-25T09:53:15.735200+0000","last_active":"2024-04-25T09:53:21.303810+0000","last_peered":"2024-04-25T09:53:21.303810+0000","last_clean":"2024-04-25T09:53:21.303810+0000","last_became_active":"2024-04-25T09:53:15.734903+0000","last_became_peered":"2024-04-25T09:53:15.734903+0000","last_unstale":"2024-04-25T09:53:21.303810+0000","last_undegraded":"2024-04-25T09:53:21.303810+0000","last_fullsized":"2024-04-25T09:53:21.303810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302760+0000","last_change":"2024-04-25T09:53:15.726664+0000","last_active":"2024-04-25T09:53:21.302760+0000","last_peered":"2024-04-25T09:53:21.302760+0000","last_clean":"2024-04-25T09:53:21.302760+0000","last_became_active":"2024-04-25T09:53:15.726362+0000","last_became_peered":"2024-04-25T09:53:15.726362+0000","last_unstale":"2024-04-25T09:53:21.302760+0000","last_undegraded":"2024-04-25T09:53:21.302760+0000","last_fullsized":"2024-04-25T09:53:21.302760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:59.223177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301824+0000","last_change":"2024-04-25T09:53:15.731610+0000","last_active":"2024-04-25T09:53:21.301824+0000","last_peered":"2024-04-25T09:53:21.301824+0000","last_clean":"2024-04-25T09:53:21.301824+0000","last_became_active":"2024-04-25T09:53:15.731289+0000","last_became_peered":"2024-04-25T09:53:15.731289+0000","last_unstale":"2024-04-25T09:53:21.301824+0000","last_undegraded":"2024-04-25T09:53:21.301824+0000","last_fullsized":"2024-04-25T09:53:21.301824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303221+0000","last_change":"2024-04-25T09:53:15.726434+0000","last_active":"2024-04-25T09:53:21.303221+0000","last_peered":"2024-04-25T09:53:21.303221+0000","last_clean":"2024-04-25T09:53:21.303221+0000","last_became_active":"2024-04-25T09:53:15.726160+0000","last_became_peered":"2024-04-25T09:53:15.726160+0000","last_unstale":"2024-04-25T09:53:21.303221+0000","last_undegraded":"2024-04-25T09:53:21.303221+0000","last_fullsized":"2024-04-25T09:53:21.303221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:39.445622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302185+0000","last_change":"2024-04-25T09:53:15.742414+0000","last_active":"2024-04-25T09:53:21.302185+0000","last_peered":"2024-04-25T09:53:21.302185+0000","last_clean":"2024-04-25T09:53:21.302185+0000","last_became_active":"2024-04-25T09:53:15.742099+0000","last_became_peered":"2024-04-25T09:53:15.742099+0000","last_unstale":"2024-04-25T09:53:21.302185+0000","last_undegraded":"2024-04-25T09:53:21.302185+0000","last_fullsized":"2024-04-25T09:53:21.302185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:53.040580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302379+0000","last_change":"2024-04-25T09:53:15.729969+0000","last_active":"2024-04-25T09:53:21.302379+0000","last_peered":"2024-04-25T09:53:21.302379+0000","last_clean":"2024-04-25T09:53:21.302379+0000","last_became_active":"2024-04-25T09:53:15.729608+0000","last_became_peered":"2024-04-25T09:53:15.729608+0000","last_unstale":"2024-04-25T09:53:21.302379+0000","last_undegraded":"2024-04-25T09:53:21.302379+0000","last_fullsized":"2024-04-25T09:53:21.302379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303562+0000","last_change":"2024-04-25T09:53:15.730114+0000","last_active":"2024-04-25T09:53:21.303562+0000","last_peered":"2024-04-25T09:53:21.303562+0000","last_clean":"2024-04-25T09:53:21.303562+0000","last_became_active":"2024-04-25T09:53:15.729664+0000","last_became_peered":"2024-04-25T09:53:15.729664+0000","last_unstale":"2024-04-25T09:53:21.303562+0000","last_undegraded":"2024-04-25T09:53:21.303562+0000","last_fullsized":"2024-04-25T09:53:21.303562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303105+0000","last_change":"2024-04-25T09:53:15.730458+0000","last_active":"2024-04-25T09:53:21.303105+0000","last_peered":"2024-04-25T09:53:21.303105+0000","last_clean":"2024-04-25T09:53:21.303105+0000","last_became_active":"2024-04-25T09:53:15.730193+0000","last_became_peered":"2024-04-25T09:53:15.730193+0000","last_unstale":"2024-04-25T09:53:21.303105+0000","last_undegraded":"2024-04-25T09:53:21.303105+0000","last_fullsized":"2024-04-25T09:53:21.303105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:46.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.741462+0000","last_change":"2024-04-25T09:53:15.723126+0000","last_active":"2024-04-25T09:53:20.741462+0000","last_peered":"2024-04-25T09:53:20.741462+0000","last_clean":"2024-04-25T09:53:20.741462+0000","last_became_active":"2024-04-25T09:53:15.722834+0000","last_became_peered":"2024-04-25T09:53:15.722834+0000","last_unstale":"2024-04-25T09:53:20.741462+0000","last_undegraded":"2024-04-25T09:53:20.741462+0000","last_fullsized":"2024-04-25T09:53:20.741462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:40.626749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303240+0000","last_change":"2024-04-25T09:53:15.742492+0000","last_active":"2024-04-25T09:53:21.303240+0000","last_peered":"2024-04-25T09:53:21.303240+0000","last_clean":"2024-04-25T09:53:21.303240+0000","last_became_active":"2024-04-25T09:53:15.742132+0000","last_became_peered":"2024-04-25T09:53:15.742132+0000","last_unstale":"2024-04-25T09:53:21.303240+0000","last_undegraded":"2024-04-25T09:53:21.303240+0000","last_fullsized":"2024-04-25T09:53:21.303240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:08.755180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301867+0000","last_change":"2024-04-25T09:53:15.729450+0000","last_active":"2024-04-25T09:53:21.301867+0000","last_peered":"2024-04-25T09:53:21.301867+0000","last_clean":"2024-04-25T09:53:21.301867+0000","last_became_active":"2024-04-25T09:53:15.729157+0000","last_became_peered":"2024-04-25T09:53:15.729157+0000","last_unstale":"2024-04-25T09:53:21.301867+0000","last_undegraded":"2024-04-25T09:53:21.301867+0000","last_fullsized":"2024-04-25T09:53:21.301867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301716+0000","last_change":"2024-04-25T09:53:15.727922+0000","last_active":"2024-04-25T09:53:21.301716+0000","last_peered":"2024-04-25T09:53:21.301716+0000","last_clean":"2024-04-25T09:53:21.301716+0000","last_became_active":"2024-04-25T09:53:15.727626+0000","last_became_peered":"2024-04-25T09:53:15.727626+0000","last_unstale":"2024-04-25T09:53:21.301716+0000","last_undegraded":"2024-04-25T09:53:21.301716+0000","last_fullsized":"2024-04-25T09:53:21.301716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301947+0000","last_change":"2024-04-25T09:53:15.721374+0000","last_active":"2024-04-25T09:53:21.301947+0000","last_peered":"2024-04-25T09:53:21.301947+0000","last_clean":"2024-04-25T09:53:21.301947+0000","last_became_active":"2024-04-25T09:53:15.719489+0000","last_became_peered":"2024-04-25T09:53:15.719489+0000","last_unstale":"2024-04-25T09:53:21.301947+0000","last_undegraded":"2024-04-25T09:53:21.301947+0000","last_fullsized":"2024-04-25T09:53:21.301947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:53.990141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302112+0000","last_change":"2024-04-25T09:53:15.723353+0000","last_active":"2024-04-25T09:53:21.302112+0000","last_peered":"2024-04-25T09:53:21.302112+0000","last_clean":"2024-04-25T09:53:21.302112+0000","last_became_active":"2024-04-25T09:53:15.723060+0000","last_became_peered":"2024-04-25T09:53:15.723060+0000","last_unstale":"2024-04-25T09:53:21.302112+0000","last_undegraded":"2024-04-25T09:53:21.302112+0000","last_fullsized":"2024-04-25T09:53:21.302112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:25.765295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:20.743264+0000","last_change":"2024-04-25T09:53:15.731294+0000","last_active":"2024-04-25T09:53:20.743264+0000","last_peered":"2024-04-25T09:53:20.743264+0000","last_clean":"2024-04-25T09:53:20.743264+0000","last_became_active":"2024-04-25T09:53:15.730556+0000","last_became_peered":"2024-04-25T09:53:15.730556+0000","last_unstale":"2024-04-25T09:53:20.743264+0000","last_undegraded":"2024-04-25T09:53:20.743264+0000","last_fullsized":"2024-04-25T09:53:20.743264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:59.737264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303649+0000","last_change":"2024-04-25T09:53:15.737541+0000","last_active":"2024-04-25T09:53:21.303649+0000","last_peered":"2024-04-25T09:53:21.303649+0000","last_clean":"2024-04-25T09:53:21.303649+0000","last_became_active":"2024-04-25T09:53:15.737262+0000","last_became_peered":"2024-04-25T09:53:15.737262+0000","last_unstale":"2024-04-25T09:53:21.303649+0000","last_undegraded":"2024-04-25T09:53:21.303649+0000","last_fullsized":"2024-04-25T09:53:21.303649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:54.738612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303812+0000","last_change":"2024-04-25T09:53:15.730915+0000","last_active":"2024-04-25T09:53:21.303812+0000","last_peered":"2024-04-25T09:53:21.303812+0000","last_clean":"2024-04-25T09:53:21.303812+0000","last_became_active":"2024-04-25T09:53:15.730657+0000","last_became_peered":"2024-04-25T09:53:15.730657+0000","last_unstale":"2024-04-25T09:53:21.303812+0000","last_undegraded":"2024-04-25T09:53:21.303812+0000","last_fullsized":"2024-04-25T09:53:21.303812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:51.444016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303315+0000","last_change":"2024-04-25T09:53:15.733524+0000","last_active":"2024-04-25T09:53:21.303315+0000","last_peered":"2024-04-25T09:53:21.303315+0000","last_clean":"2024-04-25T09:53:21.303315+0000","last_became_active":"2024-04-25T09:53:15.733244+0000","last_became_peered":"2024-04-25T09:53:15.733244+0000","last_unstale":"2024-04-25T09:53:21.303315+0000","last_undegraded":"2024-04-25T09:53:21.303315+0000","last_fullsized":"2024-04-25T09:53:21.303315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:07.652771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301474+0000","last_change":"2024-04-25T09:53:15.731461+0000","last_active":"2024-04-25T09:53:21.301474+0000","last_peered":"2024-04-25T09:53:21.301474+0000","last_clean":"2024-04-25T09:53:21.301474+0000","last_became_active":"2024-04-25T09:53:15.731091+0000","last_became_peered":"2024-04-25T09:53:15.731091+0000","last_unstale":"2024-04-25T09:53:21.301474+0000","last_undegraded":"2024-04-25T09:53:21.301474+0000","last_fullsized":"2024-04-25T09:53:21.301474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:53.707984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302158+0000","last_change":"2024-04-25T09:53:15.724502+0000","last_active":"2024-04-25T09:53:21.302158+0000","last_peered":"2024-04-25T09:53:21.302158+0000","last_clean":"2024-04-25T09:53:21.302158+0000","last_became_active":"2024-04-25T09:53:15.724218+0000","last_became_peered":"2024-04-25T09:53:15.724218+0000","last_unstale":"2024-04-25T09:53:21.302158+0000","last_undegraded":"2024-04-25T09:53:21.302158+0000","last_fullsized":"2024-04-25T09:53:21.302158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:59.223177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302094+0000","last_change":"2024-04-25T09:53:15.734479+0000","last_active":"2024-04-25T09:53:21.302094+0000","last_peered":"2024-04-25T09:53:21.302094+0000","last_clean":"2024-04-25T09:53:21.302094+0000","last_became_active":"2024-04-25T09:53:15.734127+0000","last_became_peered":"2024-04-25T09:53:15.734127+0000","last_unstale":"2024-04-25T09:53:21.302094+0000","last_undegraded":"2024-04-25T09:53:21.302094+0000","last_fullsized":"2024-04-25T09:53:21.302094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:00.208172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302254+0000","last_change":"2024-04-25T09:53:15.731043+0000","last_active":"2024-04-25T09:53:21.302254+0000","last_peered":"2024-04-25T09:53:21.302254+0000","last_clean":"2024-04-25T09:53:21.302254+0000","last_became_active":"2024-04-25T09:53:15.730779+0000","last_became_peered":"2024-04-25T09:53:15.730779+0000","last_unstale":"2024-04-25T09:53:21.302254+0000","last_undegraded":"2024-04-25T09:53:21.302254+0000","last_fullsized":"2024-04-25T09:53:21.302254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:02.721136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.302662+0000","last_change":"2024-04-25T09:53:15.732810+0000","last_active":"2024-04-25T09:53:21.302662+0000","last_peered":"2024-04-25T09:53:21.302662+0000","last_clean":"2024-04-25T09:53:21.302662+0000","last_became_active":"2024-04-25T09:53:15.732270+0000","last_became_peered":"2024-04-25T09:53:15.732270+0000","last_unstale":"2024-04-25T09:53:21.302662+0000","last_undegraded":"2024-04-25T09:53:21.302662+0000","last_fullsized":"2024-04-25T09:53:21.302662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:59.223177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.301627+0000","last_change":"2024-04-25T09:53:15.713610+0000","last_active":"2024-04-25T09:53:21.301627+0000","last_peered":"2024-04-25T09:53:21.301627+0000","last_clean":"2024-04-25T09:53:21.301627+0000","last_became_active":"2024-04-25T09:53:15.713342+0000","last_became_peered":"2024-04-25T09:53:15.713342+0000","last_unstale":"2024-04-25T09:53:21.301627+0000","last_undegraded":"2024-04-25T09:53:21.301627+0000","last_fullsized":"2024-04-25T09:53:21.301627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:39.445622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303622+0000","last_change":"2024-04-25T09:53:15.723089+0000","last_active":"2024-04-25T09:53:21.303622+0000","last_peered":"2024-04-25T09:53:21.303622+0000","last_clean":"2024-04-25T09:53:21.303622+0000","last_became_active":"2024-04-25T09:53:15.722840+0000","last_became_peered":"2024-04-25T09:53:15.722840+0000","last_unstale":"2024-04-25T09:53:21.303622+0000","last_undegraded":"2024-04-25T09:53:21.303622+0000","last_fullsized":"2024-04-25T09:53:21.303622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.063496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T09:53:21.303201+0000","last_change":"2024-04-25T09:53:15.734984+0000","last_active":"2024-04-25T09:53:21.303201+0000","last_peered":"2024-04-25T09:53:21.303201+0000","last_clean":"2024-04-25T09:53:21.303201+0000","last_became_active":"2024-04-25T09:53:15.734344+0000","last_became_peered":"2024-04-25T09:53:15.734344+0000","last_unstale":"2024-04-25T09:53:21.303201+0000","last_undegraded":"2024-04-25T09:53:21.303201+0000","last_fullsized":"2024-04-25T09:53:21.303201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:14.687363+0000","last_clean_scrub_stamp":"2024-04-25T09:53:14.687363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:50.045183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:53:37.979 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T09:53:37.979 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T09:53:37.979 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T09:53:37.979 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T09:53:38.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.107+0000 7f183cf29640 1 Processor -- start 2024-04-25T09:53:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.107+0000 7f183cf29640 1 -- start start 2024-04-25T09:53:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.108+0000 7f183cf29640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f18380d0730 0x7f18380cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.108+0000 7f183cf29640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f18380d0b90 0x7f18380cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.108+0000 7f183cf29640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18380cc8c0 con 0x7f18380b0750 2024-04-25T09:53:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.108+0000 7f183cf29640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18380ccb70 con 0x7f18380d0b90 2024-04-25T09:53:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.108+0000 7f183cf29640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18380cf970 con 0x7f18380d0730 2024-04-25T09:53:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.109+0000 7f1836d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f18380d0b90 0x7f18380cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.109+0000 7f1836d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f18380d0b90 0x7f18380cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40314/0 (socket says 172.21.15.139:40314) 2024-04-25T09:53:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.109+0000 7f183659c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f18380b0750 0x7f18380b0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48606/0 (socket says 172.21.15.139:48606) 2024-04-25T09:53:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.109+0000 7f1836d9d640 1 -- 172.21.15.139:0/605552668 learned_addr learned my addr 172.21.15.139:0/605552668 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.109+0000 7f1835d9b640 1 --2- 172.21.15.139:0/605552668 >> v2:172.21.15.150:3300/0 conn(0x7f18380d0730 0x7f18380cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.109+0000 7f1836d9d640 1 -- 172.21.15.139:0/605552668 >> v2:172.21.15.150:3300/0 conn(0x7f18380d0730 msgr2=0x7f18380cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.109+0000 7f1836d9d640 1 --2- 172.21.15.139:0/605552668 >> v2:172.21.15.150:3300/0 conn(0x7f18380d0730 0x7f18380cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.109+0000 7f1836d9d640 1 -- 172.21.15.139:0/605552668 >> v1:172.21.15.139:6789/0 conn(0x7f18380b0750 legacy=0x7f18380b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.109+0000 7f1836d9d640 1 -- 172.21.15.139:0/605552668 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18380cfc50 con 0x7f18380d0b90 2024-04-25T09:53:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.109+0000 7f1836d9d640 1 --2- 172.21.15.139:0/605552668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18380d0b90 0x7f18380cc380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f18380cf310 tx=0x7f182802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b6811f36d6e514 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.110+0000 7f183559a640 1 -- 172.21.15.139:0/605552668 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828037070 con 0x7f18380d0b90 2024-04-25T09:53:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.110+0000 7f183559a640 1 -- 172.21.15.139:0/605552668 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1828032440 con 0x7f18380d0b90 2024-04-25T09:53:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.110+0000 7f183559a640 1 -- 172.21.15.139:0/605552668 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828032810 con 0x7f18380d0b90 2024-04-25T09:53:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.110+0000 7f183cf29640 1 -- 172.21.15.139:0/605552668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18380d0b90 msgr2=0x7f18380cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.110+0000 7f183cf29640 1 --2- 172.21.15.139:0/605552668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18380d0b90 0x7f18380cc380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f18380cf310 tx=0x7f182802a270 comp rx=0 tx=0).stop 2024-04-25T09:53:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.110+0000 7f183cf29640 1 -- 172.21.15.139:0/605552668 shutdown_connections 2024-04-25T09:53:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.110+0000 7f183cf29640 1 --2- 172.21.15.139:0/605552668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18380d0b90 0x7f18380cc380 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.110+0000 7f183cf29640 1 --2- 172.21.15.139:0/605552668 >> v2:172.21.15.150:3300/0 conn(0x7f18380d0730 0x7f18380cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.110+0000 7f183cf29640 1 -- 172.21.15.139:0/605552668 >> 172.21.15.139:0/605552668 conn(0x7f18380b3380 msgr2=0x7f18380b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.110+0000 7f183cf29640 1 -- 172.21.15.139:0/605552668 shutdown_connections 2024-04-25T09:53:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.110+0000 7f183cf29640 1 -- 172.21.15.139:0/605552668 wait complete. 2024-04-25T09:53:38.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.112+0000 7f183cf29640 1 Processor -- start 2024-04-25T09:53:38.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.112+0000 7f183cf29640 1 -- start start 2024-04-25T09:53:38.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.112+0000 7f183cf29640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18380b0750 0x7f18380c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.112+0000 7f183cf29640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18380d0730 0x7f1838153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.112+0000 7f183cf29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1838159ea0 0x7f183815a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.112+0000 7f183cf29640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183815bcc0 con 0x7f1838159ea0 2024-04-25T09:53:38.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.112+0000 7f183cf29640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18380d0210 con 0x7f18380b0750 2024-04-25T09:53:38.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.112+0000 7f183cf29640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183815be40 con 0x7f18380d0730 2024-04-25T09:53:38.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.112+0000 7f1836d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1838159ea0 0x7f183815a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.112+0000 7f1836d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1838159ea0 0x7f183815a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40324/0 (socket says 172.21.15.139:40324) 2024-04-25T09:53:38.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.112+0000 7f1836d9d640 1 -- 172.21.15.139:0/3218097553 learned_addr learned my addr 172.21.15.139:0/3218097553 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:38.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f1835d9b640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18380d0730 0x7f1838153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f183659c640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18380b0750 0x7f18380c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f1836d9d640 1 -- 172.21.15.139:0/3218097553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18380d0730 msgr2=0x7f1838153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f1836d9d640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18380d0730 0x7f1838153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f1836d9d640 1 -- 172.21.15.139:0/3218097553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18380b0750 msgr2=0x7f18380c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f1836d9d640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18380b0750 0x7f18380c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f1836d9d640 1 -- 172.21.15.139:0/3218097553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18381528f0 con 0x7f1838159ea0 2024-04-25T09:53:38.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f1836d9d640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1838159ea0 0x7f183815a330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f18280090b0 tx=0x7f18280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:38.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f182effd640 1 -- 172.21.15.139:0/3218097553 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828037040 con 0x7f1838159ea0 2024-04-25T09:53:38.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f182effd640 1 -- 172.21.15.139:0/3218097553 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1828042070 con 0x7f1838159ea0 2024-04-25T09:53:38.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f183cf29640 1 -- 172.21.15.139:0/3218097553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1838152bd0 con 0x7f1838159ea0 2024-04-25T09:53:38.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f182effd640 1 -- 172.21.15.139:0/3218097553 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828030bd0 con 0x7f1838159ea0 2024-04-25T09:53:38.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.113+0000 7f183cf29640 1 -- 172.21.15.139:0/3218097553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18381533d0 con 0x7f1838159ea0 2024-04-25T09:53:38.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.114+0000 7f183cf29640 1 -- 172.21.15.139:0/3218097553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1838074520 con 0x7f1838159ea0 2024-04-25T09:53:38.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.114+0000 7f182effd640 1 -- 172.21.15.139:0/3218097553 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1828047460 con 0x7f1838159ea0 2024-04-25T09:53:38.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.115+0000 7f182effd640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f181806b4d0 0x7f181806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.115+0000 7f183659c640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f181806b4d0 0x7f181806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.115+0000 7f182effd640 1 -- 172.21.15.139:0/3218097553 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f18280ab140 con 0x7f1838159ea0 2024-04-25T09:53:38.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.115+0000 7f183659c640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f181806b4d0 0x7f181806d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18300012e0 tx=0x7f1830005b70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:38.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.119+0000 7f182effd640 1 -- 172.21.15.139:0/3218097553 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1828079c80 con 0x7f1838159ea0 2024-04-25T09:53:38.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.324+0000 7f183cf29640 1 -- 172.21.15.139:0/3218097553 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f18381596e0 con 0x7f1838159ea0 2024-04-25T09:53:38.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.324+0000 7f182effd640 1 -- 172.21.15.139:0/3218097553 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1828079580 con 0x7f1838159ea0 2024-04-25T09:53:38.327 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:38.327 INFO:teuthology.orchestra.run.smithi139.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T09:53:38.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.326+0000 7f183cf29640 1 -- 172.21.15.139:0/3218097553 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f181806b4d0 msgr2=0x7f181806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.326+0000 7f183cf29640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f181806b4d0 0x7f181806d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18300012e0 tx=0x7f1830005b70 comp rx=0 tx=0).stop 2024-04-25T09:53:38.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.326+0000 7f183cf29640 1 -- 172.21.15.139:0/3218097553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1838159ea0 msgr2=0x7f183815a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.326+0000 7f183cf29640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1838159ea0 0x7f183815a330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f18280090b0 tx=0x7f18280046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.326+0000 7f183cf29640 1 -- 172.21.15.139:0/3218097553 shutdown_connections 2024-04-25T09:53:38.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.326+0000 7f183cf29640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1838159ea0 0x7f183815a330 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.326+0000 7f183cf29640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f181806b4d0 0x7f181806d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.326+0000 7f183cf29640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18380d0730 0x7f1838153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.326+0000 7f183cf29640 1 --2- 172.21.15.139:0/3218097553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18380b0750 0x7f18380c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.326+0000 7f183cf29640 1 -- 172.21.15.139:0/3218097553 >> 172.21.15.139:0/3218097553 conn(0x7f18380b3380 msgr2=0x7f18380b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:38.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.327+0000 7f183cf29640 1 -- 172.21.15.139:0/3218097553 shutdown_connections 2024-04-25T09:53:38.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.327+0000 7f183cf29640 1 -- 172.21.15.139:0/3218097553 wait complete. 2024-04-25T09:53:38.339 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T09:53:38.340 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:38.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.434+0000 7f243e79e640 1 Processor -- start 2024-04-25T09:53:38.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.434+0000 7f243e79e640 1 -- start start 2024-04-25T09:53:38.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f243e79e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f243806be40 0x7f24380acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f243e79e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f243815dd30 0x7f24380b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f243e79e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24380ad2d0 con 0x7f243806c220 2024-04-25T09:53:38.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f243e79e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24380b2de0 con 0x7f243806be40 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f243e79e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24380b30c0 con 0x7f243815dd30 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f2437fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f243806be40 0x7f24380acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f24377fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f243806c220 0x7f243815d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48616/0 (socket says 172.21.15.139:48616) 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f24377fe640 1 -- 172.21.15.139:0/108662321 learned_addr learned my addr 172.21.15.139:0/108662321 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f243cd3b640 1 --2- 172.21.15.139:0/108662321 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f243815dd30 0x7f24380b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f2437fff640 1 -- 172.21.15.139:0/108662321 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f243815dd30 msgr2=0x7f24380b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f2437fff640 1 --2- 172.21.15.139:0/108662321 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f243815dd30 0x7f24380b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f2437fff640 1 -- 172.21.15.139:0/108662321 >> v1:172.21.15.139:6789/0 conn(0x7f243806c220 legacy=0x7f243815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f2436ffd640 1 -- 172.21.15.139:0/108662321 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535139789 0 0) 0x7f24380ad2d0 con 0x7f243806c220 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.435+0000 7f2437fff640 1 -- 172.21.15.139:0/108662321 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24380b33a0 con 0x7f243806be40 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.436+0000 7f2437fff640 1 --2- 172.21.15.139:0/108662321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f243806be40 0x7f24380acd90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f24280097a0 tx=0x7f242802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51eb9ed9c066d3a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.436+0000 7f2436ffd640 1 -- 172.21.15.139:0/108662321 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428037070 con 0x7f243806be40 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.436+0000 7f2436ffd640 1 -- 172.21.15.139:0/108662321 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2428032440 con 0x7f243806be40 2024-04-25T09:53:38.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.436+0000 7f2436ffd640 1 -- 172.21.15.139:0/108662321 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428032810 con 0x7f243806be40 2024-04-25T09:53:38.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.437+0000 7f243e79e640 1 -- 172.21.15.139:0/108662321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f243806be40 msgr2=0x7f24380acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.437+0000 7f243e79e640 1 --2- 172.21.15.139:0/108662321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f243806be40 0x7f24380acd90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f24280097a0 tx=0x7f242802a210 comp rx=0 tx=0).stop 2024-04-25T09:53:38.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.437+0000 7f243e79e640 1 -- 172.21.15.139:0/108662321 shutdown_connections 2024-04-25T09:53:38.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.437+0000 7f243e79e640 1 --2- 172.21.15.139:0/108662321 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f243815dd30 0x7f24380b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.437+0000 7f243e79e640 1 --2- 172.21.15.139:0/108662321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f243806be40 0x7f24380acd90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.437+0000 7f243e79e640 1 -- 172.21.15.139:0/108662321 >> 172.21.15.139:0/108662321 conn(0x7f2438071a00 msgr2=0x7f2438071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.437+0000 7f243e79e640 1 -- 172.21.15.139:0/108662321 shutdown_connections 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.437+0000 7f243e79e640 1 -- 172.21.15.139:0/108662321 wait complete. 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.438+0000 7f243e79e640 1 Processor -- start 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.438+0000 7f243e79e640 1 -- start start 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f243e79e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f243806be40 0x7f243814fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f243e79e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f243806c220 0x7f2438152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f243e79e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f243815dd30 0x7f2438152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f243e79e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243806d9d0 con 0x7f243806c220 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f243e79e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24380b3960 con 0x7f243806be40 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f243e79e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2438072520 con 0x7f243815dd30 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f24377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f243806c220 0x7f2438152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f24377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f243806c220 0x7f2438152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40346/0 (socket says 172.21.15.139:40346) 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f24377fe640 1 -- 172.21.15.139:0/2725623251 learned_addr learned my addr 172.21.15.139:0/2725623251 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f243cd3b640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f243815dd30 0x7f2438152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f2437fff640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f243806be40 0x7f243814fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f24377fe640 1 -- 172.21.15.139:0/2725623251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f243815dd30 msgr2=0x7f2438152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f24377fe640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f243815dd30 0x7f2438152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f24377fe640 1 -- 172.21.15.139:0/2725623251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f243806be40 msgr2=0x7f243814fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f24377fe640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f243806be40 0x7f243814fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f24377fe640 1 -- 172.21.15.139:0/2725623251 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243814b6d0 con 0x7f243806c220 2024-04-25T09:53:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.439+0000 7f24377fe640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f243806c220 0x7f2438152a40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f242c00c4e0 tx=0x7f242c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.440+0000 7f2434ff9640 1 -- 172.21.15.139:0/2725623251 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c018070 con 0x7f243806c220 2024-04-25T09:53:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.440+0000 7f2434ff9640 1 -- 172.21.15.139:0/2725623251 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c007500 con 0x7f243806c220 2024-04-25T09:53:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.440+0000 7f243e79e640 1 -- 172.21.15.139:0/2725623251 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2438159770 con 0x7f243806c220 2024-04-25T09:53:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.440+0000 7f2434ff9640 1 -- 172.21.15.139:0/2725623251 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c004a20 con 0x7f243806c220 2024-04-25T09:53:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.440+0000 7f243e79e640 1 -- 172.21.15.139:0/2725623251 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2438159d30 con 0x7f243806c220 2024-04-25T09:53:38.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.441+0000 7f243e79e640 1 -- 172.21.15.139:0/2725623251 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2438074520 con 0x7f243806c220 2024-04-25T09:53:38.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.441+0000 7f2434ff9640 1 -- 172.21.15.139:0/2725623251 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f242c01b070 con 0x7f243806c220 2024-04-25T09:53:38.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.442+0000 7f2434ff9640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f242406b4d0 0x7f242406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.442+0000 7f2437fff640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f242406b4d0 0x7f242406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.442+0000 7f2434ff9640 1 -- 172.21.15.139:0/2725623251 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f242c0897a0 con 0x7f243806c220 2024-04-25T09:53:38.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.442+0000 7f2437fff640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f242406b4d0 0x7f242406d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2428034000 tx=0x7f242802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.446+0000 7f2434ff9640 1 -- 172.21.15.139:0/2725623251 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f242c057d50 con 0x7f243806c220 2024-04-25T09:53:38.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.618+0000 7f243e79e640 1 -- 172.21.15.139:0/2725623251 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24380703c0 con 0x7f243806c220 2024-04-25T09:53:38.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.619+0000 7f2434ff9640 1 -- 172.21.15.139:0/2725623251 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f242c05bdf0 con 0x7f243806c220 2024-04-25T09:53:38.621 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:38.621 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:38.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.621+0000 7f243e79e640 1 -- 172.21.15.139:0/2725623251 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f242406b4d0 msgr2=0x7f242406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.621+0000 7f243e79e640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f242406b4d0 0x7f242406d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2428034000 tx=0x7f242802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:38.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.621+0000 7f243e79e640 1 -- 172.21.15.139:0/2725623251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f243806c220 msgr2=0x7f2438152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.621+0000 7f243e79e640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f243806c220 0x7f2438152a40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f242c00c4e0 tx=0x7f242c00c9a0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.621+0000 7f243e79e640 1 -- 172.21.15.139:0/2725623251 shutdown_connections 2024-04-25T09:53:38.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.621+0000 7f243e79e640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f243806c220 0x7f2438152a40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.621+0000 7f243e79e640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f242406b4d0 0x7f242406d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.621+0000 7f243e79e640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f243815dd30 0x7f2438152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.621+0000 7f243e79e640 1 --2- 172.21.15.139:0/2725623251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f243806be40 0x7f243814fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.621+0000 7f243e79e640 1 -- 172.21.15.139:0/2725623251 >> 172.21.15.139:0/2725623251 conn(0x7f2438071a00 msgr2=0x7f24380af990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:38.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.621+0000 7f243e79e640 1 -- 172.21.15.139:0/2725623251 shutdown_connections 2024-04-25T09:53:38.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.621+0000 7f243e79e640 1 -- 172.21.15.139:0/2725623251 wait complete. 2024-04-25T09:53:38.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:38.635 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:38.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.734+0000 7ffb322bd640 1 Processor -- start 2024-04-25T09:53:38.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.734+0000 7ffb322bd640 1 -- start start 2024-04-25T09:53:38.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.735+0000 7ffb322bd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ffb2c0cd610 0x7ffb2c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.735+0000 7ffb322bd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2c161fd0 0x7ffb2c0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.735+0000 7ffb322bd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2c099ca0 con 0x7ffb2c09ad40 2024-04-25T09:53:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.735+0000 7ffb322bd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb2c099a50 con 0x7ffb2c0cd610 2024-04-25T09:53:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.735+0000 7ffb322bd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c099f80 con 0x7ffb2c161fd0 2024-04-25T09:53:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.735+0000 7ffb3085a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2c161fd0 0x7ffb2c0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.735+0000 7ffb2bfff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ffb2c0cd610 0x7ffb2c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.735+0000 7ffb3085a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2c161fd0 0x7ffb2c0d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:55192/0 (socket says 172.21.15.139:55192) 2024-04-25T09:53:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.735+0000 7ffb3085a640 1 -- 172.21.15.139:0/3172754204 learned_addr learned my addr 172.21.15.139:0/3172754204 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.735+0000 7ffb2bfff640 1 --2- 172.21.15.139:0/3172754204 >> v2:172.21.15.150:3300/0 conn(0x7ffb2c0cd610 0x7ffb2c0d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:36896/0 (socket says 172.21.15.139:36896) 2024-04-25T09:53:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.736+0000 7ffb3085a640 1 -- 172.21.15.139:0/3172754204 >> v2:172.21.15.150:3300/0 conn(0x7ffb2c0cd610 msgr2=0x7ffb2c0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:38.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.736+0000 7ffb3085a640 1 --2- 172.21.15.139:0/3172754204 >> v2:172.21.15.150:3300/0 conn(0x7ffb2c0cd610 0x7ffb2c0d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.736+0000 7ffb3085a640 1 -- 172.21.15.139:0/3172754204 >> v1:172.21.15.150:6789/0 conn(0x7ffb2c09ad40 legacy=0x7ffb2c1618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.736+0000 7ffb3085a640 1 -- 172.21.15.139:0/3172754204 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c09a260 con 0x7ffb2c161fd0 2024-04-25T09:53:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.736+0000 7ffb3085a640 1 --2- 172.21.15.139:0/3172754204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2c161fd0 0x7ffb2c0d08b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009e50 tx=0x7ffb2402a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e4659d4804e781a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.737+0000 7ffb2affd640 1 -- 172.21.15.139:0/3172754204 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24036070 con 0x7ffb2c161fd0 2024-04-25T09:53:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.737+0000 7ffb2affd640 1 -- 172.21.15.139:0/3172754204 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb24031600 con 0x7ffb2c161fd0 2024-04-25T09:53:38.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.737+0000 7ffb2affd640 1 -- 172.21.15.139:0/3172754204 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb240319d0 con 0x7ffb2c161fd0 2024-04-25T09:53:38.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.737+0000 7ffb322bd640 1 -- 172.21.15.139:0/3172754204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2c161fd0 msgr2=0x7ffb2c0d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.737+0000 7ffb322bd640 1 --2- 172.21.15.139:0/3172754204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2c161fd0 0x7ffb2c0d08b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009e50 tx=0x7ffb2402a7a0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.737+0000 7ffb322bd640 1 -- 172.21.15.139:0/3172754204 shutdown_connections 2024-04-25T09:53:38.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.737+0000 7ffb322bd640 1 --2- 172.21.15.139:0/3172754204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2c161fd0 0x7ffb2c0d08b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.737+0000 7ffb322bd640 1 --2- 172.21.15.139:0/3172754204 >> v2:172.21.15.150:3300/0 conn(0x7ffb2c0cd610 0x7ffb2c0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.737+0000 7ffb322bd640 1 -- 172.21.15.139:0/3172754204 >> 172.21.15.139:0/3172754204 conn(0x7ffb2c0b31f0 msgr2=0x7ffb2c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.737+0000 7ffb322bd640 1 -- 172.21.15.139:0/3172754204 shutdown_connections 2024-04-25T09:53:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.737+0000 7ffb322bd640 1 -- 172.21.15.139:0/3172754204 wait complete. 2024-04-25T09:53:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.738+0000 7ffb322bd640 1 Processor -- start 2024-04-25T09:53:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.739+0000 7ffb322bd640 1 -- start start 2024-04-25T09:53:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.739+0000 7ffb322bd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2c09ad40 0x7ffb2c1f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.739+0000 7ffb322bd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb2c0cd610 0x7ffb2c159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.739+0000 7ffb322bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2c161fd0 0x7ffb2c0b6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.739+0000 7ffb322bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c0cd9f0 con 0x7ffb2c161fd0 2024-04-25T09:53:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.739+0000 7ffb322bd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c09a820 con 0x7ffb2c0cd610 2024-04-25T09:53:38.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.739+0000 7ffb322bd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c0cdb70 con 0x7ffb2c09ad40 2024-04-25T09:53:38.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.739+0000 7ffb3085a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2c161fd0 0x7ffb2c0b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.739+0000 7ffb3085a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2c161fd0 0x7ffb2c0b6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40350/0 (socket says 172.21.15.139:40350) 2024-04-25T09:53:38.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.739+0000 7ffb3085a640 1 -- 172.21.15.139:0/531303861 learned_addr learned my addr 172.21.15.139:0/531303861 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:38.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb2b7fe640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb2c0cd610 0x7ffb2c159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb2bfff640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2c09ad40 0x7ffb2c1f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb3085a640 1 -- 172.21.15.139:0/531303861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2c09ad40 msgr2=0x7ffb2c1f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb3085a640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2c09ad40 0x7ffb2c1f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb3085a640 1 -- 172.21.15.139:0/531303861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb2c0cd610 msgr2=0x7ffb2c159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb3085a640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb2c0cd610 0x7ffb2c159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb3085a640 1 -- 172.21.15.139:0/531303861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c1f4bb0 con 0x7ffb2c161fd0 2024-04-25T09:53:38.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb2b7fe640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb2c0cd610 0x7ffb2c159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:38.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb3085a640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2c161fd0 0x7ffb2c0b6720 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009f70 tx=0x7ffb24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:38.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb28ff9640 1 -- 172.21.15.139:0/531303861 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24041020 con 0x7ffb2c161fd0 2024-04-25T09:53:38.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb28ff9640 1 -- 172.21.15.139:0/531303861 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb24007820 con 0x7ffb2c161fd0 2024-04-25T09:53:38.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb322bd640 1 -- 172.21.15.139:0/531303861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c0b4900 con 0x7ffb2c161fd0 2024-04-25T09:53:38.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb28ff9640 1 -- 172.21.15.139:0/531303861 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24036040 con 0x7ffb2c161fd0 2024-04-25T09:53:38.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.740+0000 7ffb322bd640 1 -- 172.21.15.139:0/531303861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2c0b4ec0 con 0x7ffb2c161fd0 2024-04-25T09:53:38.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.741+0000 7ffb322bd640 1 -- 172.21.15.139:0/531303861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2c074520 con 0x7ffb2c161fd0 2024-04-25T09:53:38.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.741+0000 7ffb28ff9640 1 -- 172.21.15.139:0/531303861 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb24044020 con 0x7ffb2c161fd0 2024-04-25T09:53:38.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.742+0000 7ffb28ff9640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffb1806b4d0 0x7ffb1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:38.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.742+0000 7ffb28ff9640 1 -- 172.21.15.139:0/531303861 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffb240ab130 con 0x7ffb2c161fd0 2024-04-25T09:53:38.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.742+0000 7ffb2bfff640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffb1806b4d0 0x7ffb1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:38.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.742+0000 7ffb2bfff640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffb1806b4d0 0x7ffb1806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009ad0 tx=0x7ffb1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:38.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.746+0000 7ffb28ff9640 1 -- 172.21.15.139:0/531303861 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb2404b030 con 0x7ffb2c161fd0 2024-04-25T09:53:38.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.917+0000 7ffb322bd640 1 -- 172.21.15.139:0/531303861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb2c0b7580 con 0x7ffb2c161fd0 2024-04-25T09:53:38.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.917+0000 7ffb28ff9640 1 -- 172.21.15.139:0/531303861 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb24079c70 con 0x7ffb2c161fd0 2024-04-25T09:53:38.919 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:53:38.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.919+0000 7ffb322bd640 1 -- 172.21.15.139:0/531303861 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffb1806b4d0 msgr2=0x7ffb1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.919+0000 7ffb322bd640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffb1806b4d0 0x7ffb1806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009ad0 tx=0x7ffb1c008040 comp rx=0 tx=0).stop 2024-04-25T09:53:38.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.919+0000 7ffb322bd640 1 -- 172.21.15.139:0/531303861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2c161fd0 msgr2=0x7ffb2c0b6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:38.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.919+0000 7ffb322bd640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2c161fd0 0x7ffb2c0b6720 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009f70 tx=0x7ffb24004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:38.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.919+0000 7ffb322bd640 1 -- 172.21.15.139:0/531303861 shutdown_connections 2024-04-25T09:53:38.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.919+0000 7ffb322bd640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2c161fd0 0x7ffb2c0b6720 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.919+0000 7ffb322bd640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffb1806b4d0 0x7ffb1806d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.919+0000 7ffb322bd640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb2c0cd610 0x7ffb2c159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.919+0000 7ffb322bd640 1 --2- 172.21.15.139:0/531303861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2c09ad40 0x7ffb2c1f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:38.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.919+0000 7ffb322bd640 1 -- 172.21.15.139:0/531303861 >> 172.21.15.139:0/531303861 conn(0x7ffb2c0b31f0 msgr2=0x7ffb2c0b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:38.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.920+0000 7ffb322bd640 1 -- 172.21.15.139:0/531303861 shutdown_connections 2024-04-25T09:53:38.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:38.920+0000 7ffb322bd640 1 -- 172.21.15.139:0/531303861 wait complete. 2024-04-25T09:53:38.933 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:39.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.031+0000 7fadc9acb640 1 Processor -- start 2024-04-25T09:53:39.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.032+0000 7fadc9acb640 1 -- start start 2024-04-25T09:53:39.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.032+0000 7fadc9acb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fadc4070970 0x7fadc40b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.032+0000 7fadc9acb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc40d0d40 con 0x7fadc415a870 2024-04-25T09:53:39.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.032+0000 7fadc9acb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc40b4b40 con 0x7fadc40b4190 2024-04-25T09:53:39.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.032+0000 7fadc9acb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc40d1020 con 0x7fadc4070970 2024-04-25T09:53:39.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.032+0000 7fadbbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fadc4070970 0x7fadc40b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.032+0000 7fadc37fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fadc415a870 0x7fadc4070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48620/0 (socket says 172.21.15.139:48620) 2024-04-25T09:53:39.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.033+0000 7fadc37fe640 1 -- 172.21.15.139:0/642709712 learned_addr learned my addr 172.21.15.139:0/642709712 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:39.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.033+0000 7fadbbfff640 1 -- 172.21.15.139:0/642709712 >> v1:172.21.15.150:6789/0 conn(0x7fadc40b4190 legacy=0x7fadc40ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:39.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.033+0000 7fadbbfff640 1 -- 172.21.15.139:0/642709712 >> v1:172.21.15.139:6789/0 conn(0x7fadc415a870 legacy=0x7fadc4070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.033+0000 7fadbbfff640 1 -- 172.21.15.139:0/642709712 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc40b4e20 con 0x7fadc4070970 2024-04-25T09:53:39.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.033+0000 7fadc2ffd640 1 -- 172.21.15.139:0/642709712 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736868465 0 0) 0x7fadc40d0d40 con 0x7fadc415a870 2024-04-25T09:53:39.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.033+0000 7fadbbfff640 1 --2- 172.21.15.139:0/642709712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadc4070970 0x7fadc40b5620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fadac0097a0 tx=0x7fadac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a96202441d9be0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.034+0000 7fadc2ffd640 1 -- 172.21.15.139:0/642709712 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac037070 con 0x7fadc4070970 2024-04-25T09:53:39.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.034+0000 7fadc2ffd640 1 -- 172.21.15.139:0/642709712 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac02ae50 con 0x7fadc4070970 2024-04-25T09:53:39.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.034+0000 7fadc2ffd640 1 -- 172.21.15.139:0/642709712 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac032660 con 0x7fadc4070970 2024-04-25T09:53:39.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.034+0000 7fadc9acb640 1 -- 172.21.15.139:0/642709712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadc4070970 msgr2=0x7fadc40b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.034+0000 7fadc9acb640 1 --2- 172.21.15.139:0/642709712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadc4070970 0x7fadc40b5620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fadac0097a0 tx=0x7fadac02a140 comp rx=0 tx=0).stop 2024-04-25T09:53:39.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.034+0000 7fadc9acb640 1 -- 172.21.15.139:0/642709712 shutdown_connections 2024-04-25T09:53:39.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.034+0000 7fadc9acb640 1 --2- 172.21.15.139:0/642709712 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadc4070970 0x7fadc40b5620 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.034+0000 7fadc9acb640 1 -- 172.21.15.139:0/642709712 >> 172.21.15.139:0/642709712 conn(0x7fadc40b23f0 msgr2=0x7fadc40b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.034+0000 7fadc9acb640 1 -- 172.21.15.139:0/642709712 shutdown_connections 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.034+0000 7fadc9acb640 1 -- 172.21.15.139:0/642709712 wait complete. 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.035+0000 7fadc9acb640 1 Processor -- start 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.035+0000 7fadc9acb640 1 -- start start 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadc9acb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadc4070970 0x7fadc4144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadc9acb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadc40b4190 0x7fadc41e54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadc9acb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fadc415a870 0x7fadc41e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadc9acb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40d05f0 con 0x7fadc40b4190 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadc9acb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40b5190 con 0x7fadc4070970 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadc9acb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40d0770 con 0x7fadc415a870 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadbbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadc40b4190 0x7fadc41e54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadbbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadc40b4190 0x7fadc41e54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40372/0 (socket says 172.21.15.139:40372) 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadbbfff640 1 -- 172.21.15.139:0/3130137370 learned_addr learned my addr 172.21.15.139:0/3130137370 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadc37fe640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadc4070970 0x7fadc4144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadc3fff640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fadc415a870 0x7fadc41e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadbbfff640 1 -- 172.21.15.139:0/3130137370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fadc415a870 msgr2=0x7fadc41e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadbbfff640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fadc415a870 0x7fadc41e5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.036+0000 7fadbbfff640 1 -- 172.21.15.139:0/3130137370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadc4070970 msgr2=0x7fadc4144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.037+0000 7fadbbfff640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadc4070970 0x7fadc4144030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.037+0000 7fadbbfff640 1 -- 172.21.15.139:0/3130137370 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc41e3d00 con 0x7fadc40b4190 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.037+0000 7fadbbfff640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadc40b4190 0x7fadc41e54d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fadac0097a0 tx=0x7fadac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.037+0000 7fadc0ff9640 1 -- 172.21.15.139:0/3130137370 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac048070 con 0x7fadc40b4190 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.037+0000 7fadc0ff9640 1 -- 172.21.15.139:0/3130137370 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac039600 con 0x7fadc40b4190 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.037+0000 7fadc9acb640 1 -- 172.21.15.139:0/3130137370 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc41e3fe0 con 0x7fadc40b4190 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.037+0000 7fadc0ff9640 1 -- 172.21.15.139:0/3130137370 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac037040 con 0x7fadc40b4190 2024-04-25T09:53:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.037+0000 7fadc9acb640 1 -- 172.21.15.139:0/3130137370 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc41e45a0 con 0x7fadc40b4190 2024-04-25T09:53:39.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.038+0000 7fadc9acb640 1 -- 172.21.15.139:0/3130137370 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc4074520 con 0x7fadc40b4190 2024-04-25T09:53:39.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.038+0000 7fadc0ff9640 1 -- 172.21.15.139:0/3130137370 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadac04b070 con 0x7fadc40b4190 2024-04-25T09:53:39.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.039+0000 7fadc0ff9640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fada406b500 0x7fada406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.039+0000 7fadc0ff9640 1 -- 172.21.15.139:0/3130137370 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fadac0ac360 con 0x7fadc40b4190 2024-04-25T09:53:39.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.039+0000 7fadc37fe640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fada406b500 0x7fada406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.039+0000 7fadc37fe640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fada406b500 0x7fada406d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fadb4000f60 tx=0x7fadb4007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.043+0000 7fadc0ff9640 1 -- 172.21.15.139:0/3130137370 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fadac07ae10 con 0x7fadc40b4190 2024-04-25T09:53:39.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.213+0000 7fadc9acb640 1 -- 172.21.15.139:0/3130137370 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fadc41579b0 con 0x7fadc40b4190 2024-04-25T09:53:39.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.213+0000 7fadc0ff9640 1 -- 172.21.15.139:0/3130137370 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fadac07a710 con 0x7fadc40b4190 2024-04-25T09:53:39.215 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:39.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.215+0000 7fadc9acb640 1 -- 172.21.15.139:0/3130137370 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fada406b500 msgr2=0x7fada406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.215+0000 7fadc9acb640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fada406b500 0x7fada406d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fadb4000f60 tx=0x7fadb4007c00 comp rx=0 tx=0).stop 2024-04-25T09:53:39.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.215+0000 7fadc9acb640 1 -- 172.21.15.139:0/3130137370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadc40b4190 msgr2=0x7fadc41e54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.215+0000 7fadc9acb640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadc40b4190 0x7fadc41e54d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fadac0097a0 tx=0x7fadac004730 comp rx=0 tx=0).stop 2024-04-25T09:53:39.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.215+0000 7fadc9acb640 1 -- 172.21.15.139:0/3130137370 shutdown_connections 2024-04-25T09:53:39.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.215+0000 7fadc9acb640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fadc40b4190 0x7fadc41e54d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.215+0000 7fadc9acb640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fada406b500 0x7fada406d9c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.215+0000 7fadc9acb640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fadc415a870 0x7fadc41e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.215+0000 7fadc9acb640 1 --2- 172.21.15.139:0/3130137370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadc4070970 0x7fadc4144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.215+0000 7fadc9acb640 1 -- 172.21.15.139:0/3130137370 >> 172.21.15.139:0/3130137370 conn(0x7fadc40b23f0 msgr2=0x7fadc40b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:39.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.215+0000 7fadc9acb640 1 -- 172.21.15.139:0/3130137370 shutdown_connections 2024-04-25T09:53:39.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.215+0000 7fadc9acb640 1 -- 172.21.15.139:0/3130137370 wait complete. 2024-04-25T09:53:39.229 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.326+0000 7f9b2cebc640 1 Processor -- start 2024-04-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.326+0000 7f9b2cebc640 1 -- start start 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.327+0000 7f9b2cebc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b2815dbf0 0x7f9b281617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.327+0000 7f9b2cebc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9b280b9370 0x7f9b280ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.327+0000 7f9b2cebc640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b28161dc0 con 0x7f9b280cec80 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.327+0000 7f9b2cebc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b280d1ee0 con 0x7f9b280b9370 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.327+0000 7f9b2cebc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b281620a0 con 0x7f9b2815dbf0 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.327+0000 7f9b25d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9b280b9370 0x7f9b280ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.327+0000 7f9b25d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9b280b9370 0x7f9b280ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40378/0 (socket says 172.21.15.139:40378) 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.327+0000 7f9b25d9b640 1 -- 172.21.15.139:0/219309611 learned_addr learned my addr 172.21.15.139:0/219309611 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.327+0000 7f9b2659c640 1 --2- 172.21.15.139:0/219309611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b2815dbf0 0x7f9b281617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.328+0000 7f9b25d9b640 1 -- 172.21.15.139:0/219309611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b2815dbf0 msgr2=0x7f9b281617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.328+0000 7f9b25d9b640 1 --2- 172.21.15.139:0/219309611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b2815dbf0 0x7f9b281617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.328+0000 7f9b25d9b640 1 -- 172.21.15.139:0/219309611 >> v1:172.21.15.150:6789/0 conn(0x7f9b280cec80 legacy=0x7f9b2806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.328+0000 7f9b25d9b640 1 -- 172.21.15.139:0/219309611 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b28162380 con 0x7f9b280b9370 2024-04-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.328+0000 7f9b25d9b640 1 --2- 172.21.15.139:0/219309611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b280b9370 0x7f9b280ce740 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009870 tx=0x7f9b1c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71da61c32928c865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.328+0000 7f9b2559a640 1 -- 172.21.15.139:0/219309611 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c037070 con 0x7f9b280b9370 2024-04-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.328+0000 7f9b2559a640 1 -- 172.21.15.139:0/219309611 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c032440 con 0x7f9b280b9370 2024-04-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.328+0000 7f9b2559a640 1 -- 172.21.15.139:0/219309611 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c032810 con 0x7f9b280b9370 2024-04-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.329+0000 7f9b2cebc640 1 -- 172.21.15.139:0/219309611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b280b9370 msgr2=0x7f9b280ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.329+0000 7f9b2cebc640 1 --2- 172.21.15.139:0/219309611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b280b9370 0x7f9b280ce740 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009870 tx=0x7f9b1c02a210 comp rx=0 tx=0).stop 2024-04-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.329+0000 7f9b2cebc640 1 -- 172.21.15.139:0/219309611 shutdown_connections 2024-04-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.329+0000 7f9b2cebc640 1 --2- 172.21.15.139:0/219309611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b280b9370 0x7f9b280ce740 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.329+0000 7f9b2cebc640 1 --2- 172.21.15.139:0/219309611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b2815dbf0 0x7f9b281617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.329+0000 7f9b2cebc640 1 -- 172.21.15.139:0/219309611 >> 172.21.15.139:0/219309611 conn(0x7f9b280b5480 msgr2=0x7f9b280b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.329+0000 7f9b2cebc640 1 -- 172.21.15.139:0/219309611 shutdown_connections 2024-04-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.329+0000 7f9b2cebc640 1 -- 172.21.15.139:0/219309611 wait complete. 2024-04-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.330+0000 7f9b2cebc640 1 Processor -- start 2024-04-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.330+0000 7f9b2cebc640 1 -- start start 2024-04-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.331+0000 7f9b2cebc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b280b9370 0x7f9b281f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.331+0000 7f9b2cebc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b280cec80 0x7f9b28158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.331+0000 7f9b2cebc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2815dbf0 0x7f9b280aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.331+0000 7f9b2cebc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2815dfd0 con 0x7f9b2815dbf0 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.331+0000 7f9b2cebc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b280b97d0 con 0x7f9b280b9370 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.331+0000 7f9b2cebc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b281f5010 con 0x7f9b280cec80 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.331+0000 7f9b26d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2815dbf0 0x7f9b280aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.331+0000 7f9b26d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2815dbf0 0x7f9b280aa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40388/0 (socket says 172.21.15.139:40388) 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.331+0000 7f9b26d9d640 1 -- 172.21.15.139:0/1868446574 learned_addr learned my addr 172.21.15.139:0/1868446574 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.331+0000 7f9b25d9b640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b280cec80 0x7f9b28158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.331+0000 7f9b2659c640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b280b9370 0x7f9b281f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.332+0000 7f9b26d9d640 1 -- 172.21.15.139:0/1868446574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b280cec80 msgr2=0x7f9b28158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.332+0000 7f9b26d9d640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b280cec80 0x7f9b28158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.332+0000 7f9b26d9d640 1 -- 172.21.15.139:0/1868446574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b280b9370 msgr2=0x7f9b281f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.332+0000 7f9b26d9d640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b280b9370 0x7f9b281f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.332+0000 7f9b26d9d640 1 -- 172.21.15.139:0/1868446574 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b281f6850 con 0x7f9b2815dbf0 2024-04-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.332+0000 7f9b25d9b640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b280cec80 0x7f9b28158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.332+0000 7f9b26d9d640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2815dbf0 0x7f9b280aa350 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000c440 tx=0x7f9b2000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.332+0000 7f9b16ffd640 1 -- 172.21.15.139:0/1868446574 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20011790 con 0x7f9b2815dbf0 2024-04-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.332+0000 7f9b16ffd640 1 -- 172.21.15.139:0/1868446574 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b20017070 con 0x7f9b2815dbf0 2024-04-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.332+0000 7f9b2cebc640 1 -- 172.21.15.139:0/1868446574 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b281f5270 con 0x7f9b2815dbf0 2024-04-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.332+0000 7f9b16ffd640 1 -- 172.21.15.139:0/1868446574 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20011b70 con 0x7f9b2815dbf0 2024-04-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.332+0000 7f9b2cebc640 1 -- 172.21.15.139:0/1868446574 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b281f5830 con 0x7f9b2815dbf0 2024-04-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.333+0000 7f9b2cebc640 1 -- 172.21.15.139:0/1868446574 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b28074520 con 0x7f9b2815dbf0 2024-04-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.333+0000 7f9b16ffd640 1 -- 172.21.15.139:0/1868446574 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b20011d10 con 0x7f9b2815dbf0 2024-04-25T09:53:39.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.334+0000 7f9b16ffd640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b0806b4d0 0x7f9b0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.334+0000 7f9b16ffd640 1 -- 172.21.15.139:0/1868446574 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9b2008a320 con 0x7f9b2815dbf0 2024-04-25T09:53:39.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.334+0000 7f9b2659c640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b0806b4d0 0x7f9b0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.334+0000 7f9b2659c640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b0806b4d0 0x7f9b0806d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9b180070b0 tx=0x7f9b18007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.338+0000 7f9b16ffd640 1 -- 172.21.15.139:0/1868446574 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b20058e60 con 0x7f9b2815dbf0 2024-04-25T09:53:39.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.508+0000 7f9b2cebc640 1 -- 172.21.15.139:0/1868446574 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9b280d1680 con 0x7f9b2815dbf0 2024-04-25T09:53:39.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.509+0000 7f9b16ffd640 1 -- 172.21.15.139:0/1868446574 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9b20058760 con 0x7f9b2815dbf0 2024-04-25T09:53:39.511 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:53:39.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.510+0000 7f9b2cebc640 1 -- 172.21.15.139:0/1868446574 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b0806b4d0 msgr2=0x7f9b0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.510+0000 7f9b2cebc640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b0806b4d0 0x7f9b0806d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9b180070b0 tx=0x7f9b18007040 comp rx=0 tx=0).stop 2024-04-25T09:53:39.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.510+0000 7f9b2cebc640 1 -- 172.21.15.139:0/1868446574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2815dbf0 msgr2=0x7f9b280aa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.510+0000 7f9b2cebc640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2815dbf0 0x7f9b280aa350 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000c440 tx=0x7f9b2000c900 comp rx=0 tx=0).stop 2024-04-25T09:53:39.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.511+0000 7f9b2cebc640 1 -- 172.21.15.139:0/1868446574 shutdown_connections 2024-04-25T09:53:39.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.511+0000 7f9b2cebc640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2815dbf0 0x7f9b280aa350 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.511+0000 7f9b2cebc640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b0806b4d0 0x7f9b0806d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.511+0000 7f9b2cebc640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b280cec80 0x7f9b28158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.511+0000 7f9b2cebc640 1 --2- 172.21.15.139:0/1868446574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b280b9370 0x7f9b281f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.511+0000 7f9b2cebc640 1 -- 172.21.15.139:0/1868446574 >> 172.21.15.139:0/1868446574 conn(0x7f9b280b5480 msgr2=0x7f9b280b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:39.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.511+0000 7f9b2cebc640 1 -- 172.21.15.139:0/1868446574 shutdown_connections 2024-04-25T09:53:39.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.511+0000 7f9b2cebc640 1 -- 172.21.15.139:0/1868446574 wait complete. 2024-04-25T09:53:39.524 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:39.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.623+0000 7f781a807640 1 Processor -- start 2024-04-25T09:53:39.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.623+0000 7f781a807640 1 -- start start 2024-04-25T09:53:39.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f781a807640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f78140afc00 0x7f781406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f781a807640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78140d0080 0x7f781406e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f781a807640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78140ccef0 con 0x7f781415cb20 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f781a807640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78140703c0 con 0x7f78140afc00 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f781a807640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78140cd820 con 0x7f78140d0080 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f7813fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f78140afc00 0x7f781406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f78137fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f781415cb20 0x7f78140cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48624/0 (socket says 172.21.15.139:48624) 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f78137fe640 1 -- 172.21.15.139:0/777454804 learned_addr learned my addr 172.21.15.139:0/777454804 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f7818da4640 1 --2- 172.21.15.139:0/777454804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78140d0080 0x7f781406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f7812ffd640 1 -- 172.21.15.139:0/777454804 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672510127 0 0) 0x7f78140ccef0 con 0x7f781415cb20 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f7812ffd640 1 -- 172.21.15.139:0/777454804 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77fc003520 con 0x7f781415cb20 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f7813fff640 1 -- 172.21.15.139:0/777454804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78140d0080 msgr2=0x7f781406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f7813fff640 1 --2- 172.21.15.139:0/777454804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78140d0080 0x7f781406e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f7813fff640 1 -- 172.21.15.139:0/777454804 >> v1:172.21.15.139:6789/0 conn(0x7f781415cb20 legacy=0x7f78140cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.624+0000 7f7813fff640 1 -- 172.21.15.139:0/777454804 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78140cd1d0 con 0x7f78140afc00 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.625+0000 7f7813fff640 1 --2- 172.21.15.139:0/777454804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78140afc00 0x7f781406fb20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f78040097a0 tx=0x7f780402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c6963a964b2899 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.625+0000 7f7812ffd640 1 -- 172.21.15.139:0/777454804 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804037070 con 0x7f78140afc00 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.625+0000 7f7812ffd640 1 -- 172.21.15.139:0/777454804 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780402adb0 con 0x7f78140afc00 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.625+0000 7f7812ffd640 1 -- 172.21.15.139:0/777454804 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804032660 con 0x7f78140afc00 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.625+0000 7f781a807640 1 -- 172.21.15.139:0/777454804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78140afc00 msgr2=0x7f781406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.625+0000 7f781a807640 1 --2- 172.21.15.139:0/777454804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78140afc00 0x7f781406fb20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f78040097a0 tx=0x7f780402a060 comp rx=0 tx=0).stop 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.626+0000 7f781a807640 1 -- 172.21.15.139:0/777454804 shutdown_connections 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.626+0000 7f781a807640 1 --2- 172.21.15.139:0/777454804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78140d0080 0x7f781406e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.626+0000 7f781a807640 1 --2- 172.21.15.139:0/777454804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78140afc00 0x7f781406fb20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.626+0000 7f781a807640 1 -- 172.21.15.139:0/777454804 >> 172.21.15.139:0/777454804 conn(0x7f78140b2230 msgr2=0x7f78140b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.626+0000 7f781a807640 1 -- 172.21.15.139:0/777454804 shutdown_connections 2024-04-25T09:53:39.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.626+0000 7f781a807640 1 -- 172.21.15.139:0/777454804 wait complete. 2024-04-25T09:53:39.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.627+0000 7f781a807640 1 Processor -- start 2024-04-25T09:53:39.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.627+0000 7f781a807640 1 -- start start 2024-04-25T09:53:39.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.627+0000 7f781a807640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78140afc00 0x7f781415d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f781a807640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78140d0080 0x7f78141f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f781a807640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f781415cb20 0x7f7814155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f781a807640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7814070060 con 0x7f78140d0080 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f781a807640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78140affe0 con 0x7f781415cb20 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f781a807640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78140cdcb0 con 0x7f78140afc00 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f78137fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78140d0080 0x7f78141f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f78137fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78140d0080 0x7f78141f57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40402/0 (socket says 172.21.15.139:40402) 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f78137fe640 1 -- 172.21.15.139:0/814452841 learned_addr learned my addr 172.21.15.139:0/814452841 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f7813fff640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78140afc00 0x7f781415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f7818da4640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f781415cb20 0x7f7814155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f78137fe640 1 -- 172.21.15.139:0/814452841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78140afc00 msgr2=0x7f781415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f78137fe640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78140afc00 0x7f781415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f78137fe640 1 -- 172.21.15.139:0/814452841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f781415cb20 msgr2=0x7f7814155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f78137fe640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f781415cb20 0x7f7814155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f78137fe640 1 -- 172.21.15.139:0/814452841 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781415d700 con 0x7f78140d0080 2024-04-25T09:53:39.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f7818da4640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f781415cb20 0x7f7814155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:39.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f78137fe640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78140d0080 0x7f78141f57e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f780800c430 tx=0x7f780800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f7810ff9640 1 -- 172.21.15.139:0/814452841 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7808017070 con 0x7f78140d0080 2024-04-25T09:53:39.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f7810ff9640 1 -- 172.21.15.139:0/814452841 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7808005ce0 con 0x7f78140d0080 2024-04-25T09:53:39.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f781a807640 1 -- 172.21.15.139:0/814452841 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781415d9e0 con 0x7f78140d0080 2024-04-25T09:53:39.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f7810ff9640 1 -- 172.21.15.139:0/814452841 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78080072f0 con 0x7f78140d0080 2024-04-25T09:53:39.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.628+0000 7f781a807640 1 -- 172.21.15.139:0/814452841 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781415dfa0 con 0x7f78140d0080 2024-04-25T09:53:39.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.629+0000 7f781a807640 1 -- 172.21.15.139:0/814452841 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7814074520 con 0x7f78140d0080 2024-04-25T09:53:39.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.630+0000 7f7810ff9640 1 -- 172.21.15.139:0/814452841 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f780801a070 con 0x7f78140d0080 2024-04-25T09:53:39.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.630+0000 7f7810ff9640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f780006b4d0 0x7f780006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.630+0000 7f7810ff9640 1 -- 172.21.15.139:0/814452841 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7808089990 con 0x7f78140d0080 2024-04-25T09:53:39.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.630+0000 7f7813fff640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f780006b4d0 0x7f780006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.630+0000 7f7813fff640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f780006b4d0 0x7f780006d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7804034000 tx=0x7f780402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.634+0000 7f7810ff9640 1 -- 172.21.15.139:0/814452841 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7808058560 con 0x7f78140d0080 2024-04-25T09:53:39.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.806+0000 7f781a807640 1 -- 172.21.15.139:0/814452841 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f781406eeb0 con 0x7f78140d0080 2024-04-25T09:53:39.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.807+0000 7f7810ff9640 1 -- 172.21.15.139:0/814452841 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7808057e60 con 0x7f78140d0080 2024-04-25T09:53:39.809 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:39.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.809+0000 7f781a807640 1 -- 172.21.15.139:0/814452841 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f780006b4d0 msgr2=0x7f780006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.809+0000 7f781a807640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f780006b4d0 0x7f780006d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7804034000 tx=0x7f780402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:39.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.809+0000 7f781a807640 1 -- 172.21.15.139:0/814452841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78140d0080 msgr2=0x7f78141f57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.809+0000 7f781a807640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78140d0080 0x7f78141f57e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f780800c430 tx=0x7f780800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.809+0000 7f781a807640 1 -- 172.21.15.139:0/814452841 shutdown_connections 2024-04-25T09:53:39.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.809+0000 7f781a807640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78140d0080 0x7f78141f57e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.809+0000 7f781a807640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f780006b4d0 0x7f780006d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.809+0000 7f781a807640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f781415cb20 0x7f7814155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.809+0000 7f781a807640 1 --2- 172.21.15.139:0/814452841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78140afc00 0x7f781415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.809+0000 7f781a807640 1 -- 172.21.15.139:0/814452841 >> 172.21.15.139:0/814452841 conn(0x7f78140b2230 msgr2=0x7f7814159210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:39.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.809+0000 7f781a807640 1 -- 172.21.15.139:0/814452841 shutdown_connections 2024-04-25T09:53:39.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.809+0000 7f781a807640 1 -- 172.21.15.139:0/814452841 wait complete. 2024-04-25T09:53:39.822 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:39.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.916+0000 7fe96cf7e640 1 Processor -- start 2024-04-25T09:53:39.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.916+0000 7fe96cf7e640 1 -- start start 2024-04-25T09:53:39.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.917+0000 7fe96cf7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe96815efb0 0x7fe9680d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.917+0000 7fe96cf7e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe9680cf900 0x7fe96806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.917+0000 7fe96cf7e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9680cfe70 con 0x7fe96806cd80 2024-04-25T09:53:39.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.917+0000 7fe96cf7e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9680d0a80 con 0x7fe9680cf900 2024-04-25T09:53:39.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.917+0000 7fe96cf7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680d0150 con 0x7fe96815efb0 2024-04-25T09:53:39.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.917+0000 7fe966d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe96806cd80 0x7fe968099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48632/0 (socket says 172.21.15.139:48632) 2024-04-25T09:53:39.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.917+0000 7fe966d9d640 1 -- 172.21.15.139:0/3705020251 learned_addr learned my addr 172.21.15.139:0/3705020251 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.917+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213780460 0 0) 0x7fe9680cfe70 con 0x7fe96806cd80 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.917+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe950003520 con 0x7fe96806cd80 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.917+0000 7fe96659c640 1 --2- 172.21.15.139:0/3705020251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe96815efb0 0x7fe9680d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.917+0000 7fe965d9b640 1 --2- 172.21.15.139:0/3705020251 >> v2:172.21.15.150:3300/0 conn(0x7fe9680cf900 0x7fe96806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 34090305 0 0) 0x7fe950003520 con 0x7fe96806cd80 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe96815efb0 msgr2=0x7fe9680d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 --2- 172.21.15.139:0/3705020251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe96815efb0 0x7fe9680d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 >> v2:172.21.15.150:3300/0 conn(0x7fe9680cf900 msgr2=0x7fe96806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 --2- 172.21.15.139:0/3705020251 >> v2:172.21.15.150:3300/0 conn(0x7fe9680cf900 0x7fe96806c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9680d0430 con 0x7fe96806cd80 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fe9600019f0 con 0x7fe96806cd80 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 >> v1:172.21.15.139:6789/0 conn(0x7fe96806cd80 legacy=0x7fe968099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 --2- 172.21.15.139:0/3705020251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe96815efb0 0x7fe9500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 --2- 172.21.15.139:0/3705020251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe950003f30 0x7fe950004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 --2- 172.21.15.139:0/3705020251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe950004d40 0x7fe9500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680d0a80 con 0x7fe950003f30 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680d0150 con 0x7fe950004d40 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95000ba10 con 0x7fe96815efb0 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe965d9b640 1 --2- 172.21.15.139:0/3705020251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe950003f30 0x7fe950004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe96659c640 1 --2- 172.21.15.139:0/3705020251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe96815efb0 0x7fe9500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe966d9d640 1 --2- 172.21.15.139:0/3705020251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe950004d40 0x7fe9500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe965d9b640 1 -- 172.21.15.139:0/3705020251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe96815efb0 msgr2=0x7fe9500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe965d9b640 1 --2- 172.21.15.139:0/3705020251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe96815efb0 0x7fe9500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe965d9b640 1 -- 172.21.15.139:0/3705020251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe950004d40 msgr2=0x7fe9500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe965d9b640 1 --2- 172.21.15.139:0/3705020251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe950004d40 0x7fe9500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.918+0000 7fe965d9b640 1 -- 172.21.15.139:0/3705020251 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950003520 con 0x7fe950003f30 2024-04-25T09:53:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.919+0000 7fe965d9b640 1 --2- 172.21.15.139:0/3705020251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe950003f30 0x7fe950004800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009af0 tx=0x7fe950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ec0a6c7f92ddc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.919+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c036070 con 0x7fe950003f30 2024-04-25T09:53:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.919+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c031400 con 0x7fe950003f30 2024-04-25T09:53:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.919+0000 7fe96559a640 1 -- 172.21.15.139:0/3705020251 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c009690 con 0x7fe950003f30 2024-04-25T09:53:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.919+0000 7fe96cf7e640 1 -- 172.21.15.139:0/3705020251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe950003f30 msgr2=0x7fe950004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.919+0000 7fe96cf7e640 1 --2- 172.21.15.139:0/3705020251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe950003f30 0x7fe950004800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009af0 tx=0x7fe950005830 comp rx=0 tx=0).stop 2024-04-25T09:53:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.919+0000 7fe966d9d640 1 -- 172.21.15.139:0/3705020251 reap_dead start 2024-04-25T09:53:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.920+0000 7fe96cf7e640 1 -- 172.21.15.139:0/3705020251 shutdown_connections 2024-04-25T09:53:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.920+0000 7fe96cf7e640 1 -- 172.21.15.139:0/3705020251 >> 172.21.15.139:0/3705020251 conn(0x7fe9680d1790 msgr2=0x7fe9680d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.920+0000 7fe96cf7e640 1 -- 172.21.15.139:0/3705020251 shutdown_connections 2024-04-25T09:53:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.920+0000 7fe96cf7e640 1 -- 172.21.15.139:0/3705020251 wait complete. 2024-04-25T09:53:39.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.921+0000 7fe96cf7e640 1 Processor -- start 2024-04-25T09:53:39.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.921+0000 7fe96cf7e640 1 -- start start 2024-04-25T09:53:39.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe96cf7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe950003f30 0x7fe968156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe96cf7e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9681574c0 0x7fe9681f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe96cf7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe96815bc40 0x7fe96815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe96cf7e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96809a2e0 con 0x7fe96815bc40 2024-04-25T09:53:39.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe96cf7e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96809a160 con 0x7fe9681574c0 2024-04-25T09:53:39.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe96cf7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680d0f10 con 0x7fe950003f30 2024-04-25T09:53:39.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe966d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe96815bc40 0x7fe96815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe966d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe96815bc40 0x7fe96815c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40430/0 (socket says 172.21.15.139:40430) 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe966d9d640 1 -- 172.21.15.139:0/2018472840 learned_addr learned my addr 172.21.15.139:0/2018472840 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe96659c640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe950003f30 0x7fe968156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe965d9b640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9681574c0 0x7fe9681f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe966d9d640 1 -- 172.21.15.139:0/2018472840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe950003f30 msgr2=0x7fe968156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe966d9d640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe950003f30 0x7fe968156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe966d9d640 1 -- 172.21.15.139:0/2018472840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9681574c0 msgr2=0x7fe9681f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe966d9d640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9681574c0 0x7fe9681f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe966d9d640 1 -- 172.21.15.139:0/2018472840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9681f4da0 con 0x7fe96815bc40 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe966d9d640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe96815bc40 0x7fe96815c0f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe96806f350 tx=0x7fe960007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe956ffd640 1 -- 172.21.15.139:0/2018472840 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96000abc0 con 0x7fe96815bc40 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe956ffd640 1 -- 172.21.15.139:0/2018472840 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe960018070 con 0x7fe96815bc40 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe96cf7e640 1 -- 172.21.15.139:0/2018472840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9681f5080 con 0x7fe96815bc40 2024-04-25T09:53:39.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe956ffd640 1 -- 172.21.15.139:0/2018472840 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600133f0 con 0x7fe96815bc40 2024-04-25T09:53:39.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.922+0000 7fe96cf7e640 1 -- 172.21.15.139:0/2018472840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9681f5640 con 0x7fe96815bc40 2024-04-25T09:53:39.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.923+0000 7fe96cf7e640 1 -- 172.21.15.139:0/2018472840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe968074520 con 0x7fe96815bc40 2024-04-25T09:53:39.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.924+0000 7fe956ffd640 1 -- 172.21.15.139:0/2018472840 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe960013590 con 0x7fe96815bc40 2024-04-25T09:53:39.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.924+0000 7fe956ffd640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe94806b4d0 0x7fe94806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:39.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.924+0000 7fe956ffd640 1 -- 172.21.15.139:0/2018472840 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe96008ab70 con 0x7fe96815bc40 2024-04-25T09:53:39.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.924+0000 7fe96659c640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe94806b4d0 0x7fe94806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:39.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.924+0000 7fe96659c640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe94806b4d0 0x7fe94806d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe958009a20 tx=0x7fe958008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:39.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:39.928+0000 7fe956ffd640 1 -- 172.21.15.139:0/2018472840 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9600596b0 con 0x7fe96815bc40 2024-04-25T09:53:40.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.136+0000 7fe96cf7e640 1 -- 172.21.15.139:0/2018472840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe96809ae50 con 0x7fe96815bc40 2024-04-25T09:53:40.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.136+0000 7fe956ffd640 1 -- 172.21.15.139:0/2018472840 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fe960058fb0 con 0x7fe96815bc40 2024-04-25T09:53:40.138 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:40.139 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/892226922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":892226922},{"type":"v1","addr":"172.21.15.150:6834","nonce":892226922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/582019435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":582019435},{"type":"v1","addr":"172.21.15.139:6837","nonce":582019435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/2751714076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2751714076},{"type":"v1","addr":"172.21.15.139:6834","nonce":2751714076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:15.042067+0000","modified":"2024-04-25T09:53:30.094430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2160557452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2160557452},{"type":"v1","addr":"172.21.15.150:6835","nonce":2160557452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:53:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 9 2024-04-25T09:53:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.138+0000 7fe96cf7e640 1 -- 172.21.15.139:0/2018472840 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe94806b4d0 msgr2=0x7fe94806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.138+0000 7fe96cf7e640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe94806b4d0 0x7fe94806d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe958009a20 tx=0x7fe958008040 comp rx=0 tx=0).stop 2024-04-25T09:53:40.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.138+0000 7fe96cf7e640 1 -- 172.21.15.139:0/2018472840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe96815bc40 msgr2=0x7fe96815c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.138+0000 7fe96cf7e640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe96815bc40 0x7fe96815c0f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe96806f350 tx=0x7fe960007b20 comp rx=0 tx=0).stop 2024-04-25T09:53:40.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.138+0000 7fe96cf7e640 1 -- 172.21.15.139:0/2018472840 shutdown_connections 2024-04-25T09:53:40.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.138+0000 7fe96cf7e640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe96815bc40 0x7fe96815c0f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.138+0000 7fe96cf7e640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe94806b4d0 0x7fe94806d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.138+0000 7fe96cf7e640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9681574c0 0x7fe9681f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.138+0000 7fe96cf7e640 1 --2- 172.21.15.139:0/2018472840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe950003f30 0x7fe968156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.138+0000 7fe96cf7e640 1 -- 172.21.15.139:0/2018472840 >> 172.21.15.139:0/2018472840 conn(0x7fe9680d1790 msgr2=0x7fe968070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:40.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.138+0000 7fe96cf7e640 1 -- 172.21.15.139:0/2018472840 shutdown_connections 2024-04-25T09:53:40.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.138+0000 7fe96cf7e640 1 -- 172.21.15.139:0/2018472840 wait complete. 2024-04-25T09:53:40.151 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:40.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.246+0000 7fb5cdd6a640 1 Processor -- start 2024-04-25T09:53:40.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.246+0000 7fb5cdd6a640 1 -- start start 2024-04-25T09:53:40.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.246+0000 7fb5cdd6a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb5c815efb0 0x7fb5c80d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.246+0000 7fb5cdd6a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5c80cf900 0x7fb5c806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.246+0000 7fb5cdd6a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c80cfe70 con 0x7fb5c806cd80 2024-04-25T09:53:40.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.246+0000 7fb5cdd6a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c80d0a80 con 0x7fb5c815efb0 2024-04-25T09:53:40.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.246+0000 7fb5cdd6a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80d0150 con 0x7fb5c80cf900 2024-04-25T09:53:40.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.247+0000 7fb5c7fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb5c806cd80 0x7fb5c8099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48644/0 (socket says 172.21.15.139:48644) 2024-04-25T09:53:40.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.247+0000 7fb5c7fff640 1 -- 172.21.15.139:0/978235796 learned_addr learned my addr 172.21.15.139:0/978235796 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:40.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.247+0000 7fb5c6ffd640 1 --2- 172.21.15.139:0/978235796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5c80cf900 0x7fb5c806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.247+0000 7fb5c77fe640 1 --2- 172.21.15.139:0/978235796 >> v2:172.21.15.150:3300/0 conn(0x7fb5c815efb0 0x7fb5c80d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.247+0000 7fb5c67fc640 1 -- 172.21.15.139:0/978235796 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549602967 0 0) 0x7fb5c80cfe70 con 0x7fb5c806cd80 2024-04-25T09:53:40.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.247+0000 7fb5c67fc640 1 -- 172.21.15.139:0/978235796 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b0003520 con 0x7fb5c806cd80 2024-04-25T09:53:40.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.247+0000 7fb5c67fc640 1 -- 172.21.15.139:0/978235796 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2745292205 0 0) 0x7fb5b0003520 con 0x7fb5c806cd80 2024-04-25T09:53:40.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.247+0000 7fb5c6ffd640 1 -- 172.21.15.139:0/978235796 >> v2:172.21.15.150:3300/0 conn(0x7fb5c815efb0 msgr2=0x7fb5c80d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:40.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.247+0000 7fb5c6ffd640 1 --2- 172.21.15.139:0/978235796 >> v2:172.21.15.150:3300/0 conn(0x7fb5c815efb0 0x7fb5c80d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.247+0000 7fb5c6ffd640 1 -- 172.21.15.139:0/978235796 >> v1:172.21.15.139:6789/0 conn(0x7fb5c806cd80 legacy=0x7fb5c8099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.247+0000 7fb5c6ffd640 1 -- 172.21.15.139:0/978235796 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c80d0430 con 0x7fb5c80cf900 2024-04-25T09:53:40.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.248+0000 7fb5c6ffd640 1 --2- 172.21.15.139:0/978235796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5c80cf900 0x7fb5c806c840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80097a0 tx=0x7fb5b802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c57084042617902b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:40.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.248+0000 7fb5c67fc640 1 -- 172.21.15.139:0/978235796 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8037070 con 0x7fb5c80cf900 2024-04-25T09:53:40.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.248+0000 7fb5c67fc640 1 -- 172.21.15.139:0/978235796 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b802ab10 con 0x7fb5c80cf900 2024-04-25T09:53:40.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.248+0000 7fb5c67fc640 1 -- 172.21.15.139:0/978235796 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00019f0 con 0x7fb5c80cf900 2024-04-25T09:53:40.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.249+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/978235796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5c80cf900 msgr2=0x7fb5c806c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.249+0000 7fb5cdd6a640 1 --2- 172.21.15.139:0/978235796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5c80cf900 0x7fb5c806c840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80097a0 tx=0x7fb5b802a080 comp rx=0 tx=0).stop 2024-04-25T09:53:40.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.249+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/978235796 shutdown_connections 2024-04-25T09:53:40.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.249+0000 7fb5cdd6a640 1 --2- 172.21.15.139:0/978235796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5c80cf900 0x7fb5c806c840 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.249+0000 7fb5cdd6a640 1 --2- 172.21.15.139:0/978235796 >> v2:172.21.15.150:3300/0 conn(0x7fb5c815efb0 0x7fb5c80d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.249+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/978235796 >> 172.21.15.139:0/978235796 conn(0x7fb5c80d1790 msgr2=0x7fb5c80d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:40.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.249+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/978235796 shutdown_connections 2024-04-25T09:53:40.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.249+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/978235796 wait complete. 2024-04-25T09:53:40.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.250+0000 7fb5cdd6a640 1 Processor -- start 2024-04-25T09:53:40.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.250+0000 7fb5cdd6a640 1 -- start start 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5cdd6a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5c806cd80 0x7fb5c8154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5cdd6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5c80cf900 0x7fb5c80c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5cdd6a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5c815efb0 0x7fb5c80c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5cdd6a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c809a2e0 con 0x7fb5c80cf900 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5cdd6a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c809a160 con 0x7fb5c815efb0 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5cdd6a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80cbde0 con 0x7fb5c806cd80 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5c6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5c80cf900 0x7fb5c80c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5c6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5c80cf900 0x7fb5c80c64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40434/0 (socket says 172.21.15.139:40434) 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5c6ffd640 1 -- 172.21.15.139:0/2236106407 learned_addr learned my addr 172.21.15.139:0/2236106407 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5c77fe640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5c806cd80 0x7fb5c8154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5c7fff640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5c815efb0 0x7fb5c80c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5c6ffd640 1 -- 172.21.15.139:0/2236106407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5c806cd80 msgr2=0x7fb5c8154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5c6ffd640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5c806cd80 0x7fb5c8154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5c6ffd640 1 -- 172.21.15.139:0/2236106407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5c815efb0 msgr2=0x7fb5c80c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5c6ffd640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5c815efb0 0x7fb5c80c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5c6ffd640 1 -- 172.21.15.139:0/2236106407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c81e98f0 con 0x7fb5c80cf900 2024-04-25T09:53:40.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5c6ffd640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5c80cf900 0x7fb5c80c64b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8035040 tx=0x7fb5b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:40.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5affff640 1 -- 172.21.15.139:0/2236106407 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8041070 con 0x7fb5c80cf900 2024-04-25T09:53:40.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5affff640 1 -- 172.21.15.139:0/2236106407 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b803c660 con 0x7fb5c80cf900 2024-04-25T09:53:40.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/2236106407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c81e9bd0 con 0x7fb5c80cf900 2024-04-25T09:53:40.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.251+0000 7fb5affff640 1 -- 172.21.15.139:0/2236106407 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8037040 con 0x7fb5c80cf900 2024-04-25T09:53:40.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.252+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/2236106407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c81ea190 con 0x7fb5c80cf900 2024-04-25T09:53:40.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.252+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/2236106407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c8074520 con 0x7fb5c80cf900 2024-04-25T09:53:40.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.253+0000 7fb5affff640 1 -- 172.21.15.139:0/2236106407 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5b8044070 con 0x7fb5c80cf900 2024-04-25T09:53:40.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.253+0000 7fb5affff640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb5b406b4d0 0x7fb5b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.253+0000 7fb5affff640 1 -- 172.21.15.139:0/2236106407 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb5b80aaf40 con 0x7fb5c80cf900 2024-04-25T09:53:40.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.253+0000 7fb5c77fe640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb5b406b4d0 0x7fb5b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.254+0000 7fb5c77fe640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb5b406b4d0 0x7fb5b406d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009aa0 tx=0x7fb5bc009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:40.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.257+0000 7fb5affff640 1 -- 172.21.15.139:0/2236106407 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5b80794c0 con 0x7fb5c80cf900 2024-04-25T09:53:40.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.427+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/2236106407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5c809ae50 con 0x7fb5c80cf900 2024-04-25T09:53:40.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.428+0000 7fb5affff640 1 -- 172.21.15.139:0/2236106407 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7fb5b807d560 con 0x7fb5c80cf900 2024-04-25T09:53:40.431 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:40.431 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:40.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.430+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/2236106407 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb5b406b4d0 msgr2=0x7fb5b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.430+0000 7fb5cdd6a640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb5b406b4d0 0x7fb5b406d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009aa0 tx=0x7fb5bc009190 comp rx=0 tx=0).stop 2024-04-25T09:53:40.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.430+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/2236106407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5c80cf900 msgr2=0x7fb5c80c64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.430+0000 7fb5cdd6a640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5c80cf900 0x7fb5c80c64b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8035040 tx=0x7fb5b80046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.430+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/2236106407 shutdown_connections 2024-04-25T09:53:40.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.430+0000 7fb5cdd6a640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5c80cf900 0x7fb5c80c64b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.430+0000 7fb5cdd6a640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb5b406b4d0 0x7fb5b406d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.430+0000 7fb5cdd6a640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5c815efb0 0x7fb5c80c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.430+0000 7fb5cdd6a640 1 --2- 172.21.15.139:0/2236106407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5c806cd80 0x7fb5c8154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.430+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/2236106407 >> 172.21.15.139:0/2236106407 conn(0x7fb5c80d1790 msgr2=0x7fb5c80701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:40.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.431+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/2236106407 shutdown_connections 2024-04-25T09:53:40.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.431+0000 7fb5cdd6a640 1 -- 172.21.15.139:0/2236106407 wait complete. 2024-04-25T09:53:40.444 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:40.445 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:40.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.542+0000 7ffbe99b0640 1 Processor -- start 2024-04-25T09:53:40.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.542+0000 7ffbe99b0640 1 -- start start 2024-04-25T09:53:40.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe99b0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ffbe41617e0 0x7ffbe40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe99b0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe4161dc0 con 0x7ffbe40cec80 2024-04-25T09:53:40.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe99b0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe41620a0 con 0x7ffbe415dbf0 2024-04-25T09:53:40.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe99b0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe40cf0d0 con 0x7ffbe41617e0 2024-04-25T09:53:40.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe37fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ffbe40cec80 0x7ffbe40d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48650/0 (socket says 172.21.15.139:48650) 2024-04-25T09:53:40.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe37fe640 1 -- 172.21.15.139:0/1735014247 learned_addr learned my addr 172.21.15.139:0/1735014247 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:40.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbdbfff640 1 --2- 172.21.15.139:0/1735014247 >> v2:172.21.15.150:3300/0 conn(0x7ffbe41617e0 0x7ffbe40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243477069 0 0) 0x7ffbe4161dc0 con 0x7ffbe40cec80 2024-04-25T09:53:40.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd0003520 con 0x7ffbe40cec80 2024-04-25T09:53:40.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2904583267 0 0) 0x7ffbd0003520 con 0x7ffbe40cec80 2024-04-25T09:53:40.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 >> v2:172.21.15.150:3300/0 conn(0x7ffbe41617e0 msgr2=0x7ffbe40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:40.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe27fc640 1 --2- 172.21.15.139:0/1735014247 >> v2:172.21.15.150:3300/0 conn(0x7ffbe41617e0 0x7ffbe40ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 >> v1:172.21.15.150:6789/0 conn(0x7ffbe415dbf0 legacy=0x7ffbe40b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe4162380 con 0x7ffbe40cec80 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7ffbdc0019f0 con 0x7ffbe40cec80 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 >> v1:172.21.15.139:6789/0 conn(0x7ffbe40cec80 legacy=0x7ffbe40d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.543+0000 7ffbe27fc640 1 --2- 172.21.15.139:0/1735014247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbd00039b0 0x7ffbd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe27fc640 1 --2- 172.21.15.139:0/1735014247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffbd0004800 0x7ffbd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe27fc640 1 --2- 172.21.15.139:0/1735014247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd00051c0 0x7ffbd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40cf0d0 con 0x7ffbd00039b0 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd000c160 con 0x7ffbd00051c0 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd000c360 con 0x7ffbd0004800 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe2ffd640 1 --2- 172.21.15.139:0/1735014247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbd00039b0 0x7ffbd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe37fe640 1 --2- 172.21.15.139:0/1735014247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd00051c0 0x7ffbd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbdbfff640 1 --2- 172.21.15.139:0/1735014247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffbd0004800 0x7ffbd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe2ffd640 1 -- 172.21.15.139:0/1735014247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffbd0004800 msgr2=0x7ffbd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe2ffd640 1 --2- 172.21.15.139:0/1735014247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffbd0004800 0x7ffbd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe2ffd640 1 -- 172.21.15.139:0/1735014247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd00051c0 msgr2=0x7ffbd000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe2ffd640 1 --2- 172.21.15.139:0/1735014247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd00051c0 0x7ffbd000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe37fe640 1 -- 172.21.15.139:0/1735014247 reap_dead start 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.544+0000 7ffbe2ffd640 1 -- 172.21.15.139:0/1735014247 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd0003520 con 0x7ffbd00039b0 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.545+0000 7ffbe2ffd640 1 --2- 172.21.15.139:0/1735014247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbd00039b0 0x7ffbd0003dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4001750 tx=0x7ffbd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96db3cd8cb850803 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.545+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4001370 con 0x7ffbd00039b0 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.545+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd40052d0 con 0x7ffbd00039b0 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.545+0000 7ffbe27fc640 1 -- 172.21.15.139:0/1735014247 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd402abc0 con 0x7ffbd00039b0 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.545+0000 7ffbe99b0640 1 -- 172.21.15.139:0/1735014247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbd00039b0 msgr2=0x7ffbd0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.545+0000 7ffbe99b0640 1 --2- 172.21.15.139:0/1735014247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbd00039b0 0x7ffbd0003dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4001750 tx=0x7ffbd0005830 comp rx=0 tx=0).stop 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.545+0000 7ffbe99b0640 1 -- 172.21.15.139:0/1735014247 shutdown_connections 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.545+0000 7ffbe99b0640 1 --2- 172.21.15.139:0/1735014247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbd00039b0 0x7ffbd0003dd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.545+0000 7ffbe99b0640 1 -- 172.21.15.139:0/1735014247 >> 172.21.15.139:0/1735014247 conn(0x7ffbe40b5480 msgr2=0x7ffbe40b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:40.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.545+0000 7ffbe99b0640 1 -- 172.21.15.139:0/1735014247 shutdown_connections 2024-04-25T09:53:40.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.545+0000 7ffbe99b0640 1 -- 172.21.15.139:0/1735014247 wait complete. 2024-04-25T09:53:40.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.547+0000 7ffbe99b0640 1 Processor -- start 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.547+0000 7ffbe99b0640 1 -- start start 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.547+0000 7ffbe99b0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd00039b0 0x7ffbe41528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.547+0000 7ffbe99b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbe40c9d50 0x7ffbe4151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.547+0000 7ffbe99b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffbe4156ea0 0x7ffbe4157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.547+0000 7ffbe99b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe415dfd0 con 0x7ffbe40c9d50 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.547+0000 7ffbe99b0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40d1ee0 con 0x7ffbd00039b0 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.547+0000 7ffbe99b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe4152eb0 con 0x7ffbe4156ea0 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.547+0000 7ffbdbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbe40c9d50 0x7ffbe4151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.547+0000 7ffbdbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbe40c9d50 0x7ffbe4151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40446/0 (socket says 172.21.15.139:40446) 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.547+0000 7ffbdbfff640 1 -- 172.21.15.139:0/3304834423 learned_addr learned my addr 172.21.15.139:0/3304834423 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbe37fe640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffbe4156ea0 0x7ffbe4157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbe2ffd640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd00039b0 0x7ffbe41528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbdbfff640 1 -- 172.21.15.139:0/3304834423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffbe4156ea0 msgr2=0x7ffbe4157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbdbfff640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffbe4156ea0 0x7ffbe4157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbdbfff640 1 -- 172.21.15.139:0/3304834423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd00039b0 msgr2=0x7ffbe41528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbdbfff640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd00039b0 0x7ffbe41528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbdbfff640 1 -- 172.21.15.139:0/3304834423 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe4153190 con 0x7ffbe40c9d50 2024-04-25T09:53:40.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbe37fe640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffbe4156ea0 0x7ffbe4157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:40.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbdbfff640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbe40c9d50 0x7ffbe4151df0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc011960 tx=0x7ffbcc011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:40.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbe0ff9640 1 -- 172.21.15.139:0/3304834423 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00faf0 con 0x7ffbe40c9d50 2024-04-25T09:53:40.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbe0ff9640 1 -- 172.21.15.139:0/3304834423 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc00fc90 con 0x7ffbe40c9d50 2024-04-25T09:53:40.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbe99b0640 1 -- 172.21.15.139:0/3304834423 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe4153470 con 0x7ffbe40c9d50 2024-04-25T09:53:40.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbe0ff9640 1 -- 172.21.15.139:0/3304834423 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc014440 con 0x7ffbe40c9d50 2024-04-25T09:53:40.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.548+0000 7ffbe99b0640 1 -- 172.21.15.139:0/3304834423 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe41555f0 con 0x7ffbe40c9d50 2024-04-25T09:53:40.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.549+0000 7ffbe99b0640 1 -- 172.21.15.139:0/3304834423 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe4074520 con 0x7ffbe40c9d50 2024-04-25T09:53:40.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.549+0000 7ffbe0ff9640 1 -- 172.21.15.139:0/3304834423 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbcc012040 con 0x7ffbe40c9d50 2024-04-25T09:53:40.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.550+0000 7ffbe0ff9640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffbc406b500 0x7ffbc406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.550+0000 7ffbe2ffd640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffbc406b500 0x7ffbc406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.550+0000 7ffbe0ff9640 1 -- 172.21.15.139:0/3304834423 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffbcc08ab00 con 0x7ffbe40c9d50 2024-04-25T09:53:40.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.550+0000 7ffbe2ffd640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffbc406b500 0x7ffbc406d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4032000 tx=0x7ffbd402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:40.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.554+0000 7ffbe0ff9640 1 -- 172.21.15.139:0/3304834423 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbcc019030 con 0x7ffbe40c9d50 2024-04-25T09:53:40.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.722+0000 7ffbe99b0640 1 -- 172.21.15.139:0/3304834423 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbe40d4530 con 0x7ffbe40c9d50 2024-04-25T09:53:40.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.723+0000 7ffbe0ff9640 1 -- 172.21.15.139:0/3304834423 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffbcc0596d0 con 0x7ffbe40c9d50 2024-04-25T09:53:40.725 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:53:40.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.725+0000 7ffbe99b0640 1 -- 172.21.15.139:0/3304834423 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffbc406b500 msgr2=0x7ffbc406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.725+0000 7ffbe99b0640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffbc406b500 0x7ffbc406d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4032000 tx=0x7ffbd402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:40.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.725+0000 7ffbe99b0640 1 -- 172.21.15.139:0/3304834423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbe40c9d50 msgr2=0x7ffbe4151df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.725+0000 7ffbe99b0640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbe40c9d50 0x7ffbe4151df0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc011960 tx=0x7ffbcc011e20 comp rx=0 tx=0).stop 2024-04-25T09:53:40.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.725+0000 7ffbe99b0640 1 -- 172.21.15.139:0/3304834423 shutdown_connections 2024-04-25T09:53:40.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.725+0000 7ffbe99b0640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbe40c9d50 0x7ffbe4151df0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.725+0000 7ffbe99b0640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffbc406b500 0x7ffbc406d9c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.725+0000 7ffbe99b0640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffbe4156ea0 0x7ffbe4157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.725+0000 7ffbe99b0640 1 --2- 172.21.15.139:0/3304834423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbd00039b0 0x7ffbe41528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.725+0000 7ffbe99b0640 1 -- 172.21.15.139:0/3304834423 >> 172.21.15.139:0/3304834423 conn(0x7ffbe40b5480 msgr2=0x7ffbe40d3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:40.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.725+0000 7ffbe99b0640 1 -- 172.21.15.139:0/3304834423 shutdown_connections 2024-04-25T09:53:40.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.725+0000 7ffbe99b0640 1 -- 172.21.15.139:0/3304834423 wait complete. 2024-04-25T09:53:40.738 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:40.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.837+0000 7f79f929a640 1 Processor -- start 2024-04-25T09:53:40.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.837+0000 7f79f929a640 1 -- start start 2024-04-25T09:53:40.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f929a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f79f415dbf0 0x7f79f41617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f929a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f40cee50 0x7f79f40d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f929a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f4161dc0 con 0x7f79f40b9370 2024-04-25T09:53:40.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f929a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f40d1ee0 con 0x7f79f415dbf0 2024-04-25T09:53:40.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f929a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f41620a0 con 0x7f79f40cee50 2024-04-25T09:53:40.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f27fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f79f40b9370 0x7f79f40ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48656/0 (socket says 172.21.15.139:48656) 2024-04-25T09:53:40.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f27fc640 1 -- 172.21.15.139:0/3566654156 learned_addr learned my addr 172.21.15.139:0/3566654156 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:40.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f37fe640 1 --2- 172.21.15.139:0/3566654156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f40cee50 0x7f79f40d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f2ffd640 1 --2- 172.21.15.139:0/3566654156 >> v2:172.21.15.150:3300/0 conn(0x7f79f415dbf0 0x7f79f41617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f1ffb640 1 -- 172.21.15.139:0/3566654156 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879562023 0 0) 0x7f79f4161dc0 con 0x7f79f40b9370 2024-04-25T09:53:40.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f1ffb640 1 -- 172.21.15.139:0/3566654156 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79dc003520 con 0x7f79f40b9370 2024-04-25T09:53:40.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f2ffd640 1 -- 172.21.15.139:0/3566654156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f40cee50 msgr2=0x7f79f40d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:40.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f2ffd640 1 --2- 172.21.15.139:0/3566654156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f40cee50 0x7f79f40d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f2ffd640 1 -- 172.21.15.139:0/3566654156 >> v1:172.21.15.139:6789/0 conn(0x7f79f40b9370 legacy=0x7f79f40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f2ffd640 1 -- 172.21.15.139:0/3566654156 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f4162380 con 0x7f79f415dbf0 2024-04-25T09:53:40.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.838+0000 7f79f37fe640 1 --2- 172.21.15.139:0/3566654156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f40cee50 0x7f79f40d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:40.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.839+0000 7f79f2ffd640 1 --2- 172.21.15.139:0/3566654156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f415dbf0 0x7f79f41617e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f79e40097a0 tx=0x7f79e402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b73c9e18fc70d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:40.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.839+0000 7f79f1ffb640 1 -- 172.21.15.139:0/3566654156 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4037070 con 0x7f79f415dbf0 2024-04-25T09:53:40.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.839+0000 7f79f1ffb640 1 -- 172.21.15.139:0/3566654156 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e4032440 con 0x7f79f415dbf0 2024-04-25T09:53:40.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.839+0000 7f79f1ffb640 1 -- 172.21.15.139:0/3566654156 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40327f0 con 0x7f79f415dbf0 2024-04-25T09:53:40.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.840+0000 7f79f929a640 1 -- 172.21.15.139:0/3566654156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f415dbf0 msgr2=0x7f79f41617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.840+0000 7f79f929a640 1 --2- 172.21.15.139:0/3566654156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f415dbf0 0x7f79f41617e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f79e40097a0 tx=0x7f79e402a180 comp rx=0 tx=0).stop 2024-04-25T09:53:40.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.840+0000 7f79f929a640 1 -- 172.21.15.139:0/3566654156 shutdown_connections 2024-04-25T09:53:40.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.840+0000 7f79f929a640 1 --2- 172.21.15.139:0/3566654156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f40cee50 0x7f79f40d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.840+0000 7f79f929a640 1 --2- 172.21.15.139:0/3566654156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f415dbf0 0x7f79f41617e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.840+0000 7f79f929a640 1 -- 172.21.15.139:0/3566654156 >> 172.21.15.139:0/3566654156 conn(0x7f79f40b5480 msgr2=0x7f79f40b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:40.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.840+0000 7f79f929a640 1 -- 172.21.15.139:0/3566654156 shutdown_connections 2024-04-25T09:53:40.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.840+0000 7f79f929a640 1 -- 172.21.15.139:0/3566654156 wait complete. 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.841+0000 7f79f929a640 1 Processor -- start 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.841+0000 7f79f929a640 1 -- start start 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f929a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f40b9370 0x7f79f40cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f929a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40cee50 0x7f79f415f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f929a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f415dbf0 0x7f79f415f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f929a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f415dfd0 con 0x7f79f40cee50 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f929a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40b97d0 con 0x7f79f40b9370 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f929a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f415fcf0 con 0x7f79f415dbf0 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40cee50 0x7f79f415f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40cee50 0x7f79f415f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40450/0 (socket says 172.21.15.139:40450) 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f27fc640 1 -- 172.21.15.139:0/791464706 learned_addr learned my addr 172.21.15.139:0/791464706 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f37fe640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f415dbf0 0x7f79f415f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f2ffd640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f40b9370 0x7f79f40cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f27fc640 1 -- 172.21.15.139:0/791464706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f415dbf0 msgr2=0x7f79f415f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f27fc640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f415dbf0 0x7f79f415f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f27fc640 1 -- 172.21.15.139:0/791464706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f40b9370 msgr2=0x7f79f40cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:40.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f27fc640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f40b9370 0x7f79f40cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:40.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.842+0000 7f79f27fc640 1 -- 172.21.15.139:0/791464706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f41f7cd0 con 0x7f79f40cee50 2024-04-25T09:53:40.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.843+0000 7f79f27fc640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40cee50 0x7f79f415f270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f79e800c3c0 tx=0x7f79e800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:40.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.843+0000 7f79db7fe640 1 -- 172.21.15.139:0/791464706 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8017070 con 0x7f79f40cee50 2024-04-25T09:53:40.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.843+0000 7f79db7fe640 1 -- 172.21.15.139:0/791464706 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e8005ce0 con 0x7f79f40cee50 2024-04-25T09:53:40.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.843+0000 7f79f929a640 1 -- 172.21.15.139:0/791464706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f41f3c70 con 0x7f79f40cee50 2024-04-25T09:53:40.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.843+0000 7f79db7fe640 1 -- 172.21.15.139:0/791464706 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80072d0 con 0x7f79f40cee50 2024-04-25T09:53:40.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.843+0000 7f79f929a640 1 -- 172.21.15.139:0/791464706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f41f3920 con 0x7f79f40cee50 2024-04-25T09:53:40.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.844+0000 7f79f929a640 1 -- 172.21.15.139:0/791464706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f41f4060 con 0x7f79f40cee50 2024-04-25T09:53:40.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.844+0000 7f79db7fe640 1 -- 172.21.15.139:0/791464706 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79e801a070 con 0x7f79f40cee50 2024-04-25T09:53:40.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.845+0000 7f79db7fe640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f79e006b4d0 0x7f79e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:40.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.845+0000 7f79f2ffd640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f79e006b4d0 0x7f79e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:40.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.845+0000 7f79db7fe640 1 -- 172.21.15.139:0/791464706 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f79e80897b0 con 0x7f79f40cee50 2024-04-25T09:53:40.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.845+0000 7f79f2ffd640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f79e006b4d0 0x7f79e006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f79e4034000 tx=0x7f79e402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:40.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:40.849+0000 7f79db7fe640 1 -- 172.21.15.139:0/791464706 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79e8057ca0 con 0x7f79f40cee50 2024-04-25T09:53:41.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.021+0000 7f79f929a640 1 -- 172.21.15.139:0/791464706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79f41f3e40 con 0x7f79f40cee50 2024-04-25T09:53:41.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.021+0000 7f79db7fe640 1 -- 172.21.15.139:0/791464706 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79e805bd40 con 0x7f79f40cee50 2024-04-25T09:53:41.027 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:41.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.023+0000 7f79f929a640 1 -- 172.21.15.139:0/791464706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f79e006b4d0 msgr2=0x7f79e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.023+0000 7f79f929a640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f79e006b4d0 0x7f79e006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f79e4034000 tx=0x7f79e402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:41.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.023+0000 7f79f929a640 1 -- 172.21.15.139:0/791464706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40cee50 msgr2=0x7f79f415f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.023+0000 7f79f929a640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40cee50 0x7f79f415f270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f79e800c3c0 tx=0x7f79e800c7c0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.023+0000 7f79f929a640 1 -- 172.21.15.139:0/791464706 shutdown_connections 2024-04-25T09:53:41.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.023+0000 7f79f929a640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79f40cee50 0x7f79f415f270 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.024+0000 7f79f929a640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f79e006b4d0 0x7f79e006d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.024+0000 7f79f929a640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f415dbf0 0x7f79f415f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.024+0000 7f79f929a640 1 --2- 172.21.15.139:0/791464706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f40b9370 0x7f79f40cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.024+0000 7f79f929a640 1 -- 172.21.15.139:0/791464706 >> 172.21.15.139:0/791464706 conn(0x7f79f40b5480 msgr2=0x7f79f40b7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.024+0000 7f79f929a640 1 -- 172.21.15.139:0/791464706 shutdown_connections 2024-04-25T09:53:41.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.024+0000 7f79f929a640 1 -- 172.21.15.139:0/791464706 wait complete. 2024-04-25T09:53:41.037 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:41.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.135+0000 7f4c8897d640 1 Processor -- start 2024-04-25T09:53:41.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.135+0000 7f4c8897d640 1 -- start start 2024-04-25T09:53:41.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c8897d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c840af8a0 0x7f4c840b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c8897d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c840afc80 0x7f4c840cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c8897d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c840c6cc0 con 0x7f4c8406b700 2024-04-25T09:53:41.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c8897d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c840c6fa0 con 0x7f4c840af8a0 2024-04-25T09:53:41.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c8897d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c840c7280 con 0x7f4c840afc80 2024-04-25T09:53:41.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c8259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c840af8a0 0x7f4c840b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c8259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c840af8a0 0x7f4c840b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40466/0 (socket says 172.21.15.139:40466) 2024-04-25T09:53:41.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c8259c640 1 -- 172.21.15.139:0/457591565 learned_addr learned my addr 172.21.15.139:0/457591565 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:41.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c81d9b640 1 --2- 172.21.15.139:0/457591565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c840afc80 0x7f4c840cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c8259c640 1 -- 172.21.15.139:0/457591565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c840afc80 msgr2=0x7f4c840cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:41.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c8259c640 1 --2- 172.21.15.139:0/457591565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c840afc80 0x7f4c840cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c8259c640 1 -- 172.21.15.139:0/457591565 >> v1:172.21.15.150:6789/0 conn(0x7f4c8406b700 legacy=0x7f4c8406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c8259c640 1 -- 172.21.15.139:0/457591565 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c840cd090 con 0x7f4c840af8a0 2024-04-25T09:53:41.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.136+0000 7f4c81d9b640 1 --2- 172.21.15.139:0/457591565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c840afc80 0x7f4c840cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:41.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.137+0000 7f4c8259c640 1 --2- 172.21.15.139:0/457591565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c840af8a0 0x7f4c840b2d50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009870 tx=0x7f4c6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ffcc41bf784361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.137+0000 7f4c8159a640 1 -- 172.21.15.139:0/457591565 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c037070 con 0x7f4c840af8a0 2024-04-25T09:53:41.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.137+0000 7f4c8159a640 1 -- 172.21.15.139:0/457591565 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c032440 con 0x7f4c840af8a0 2024-04-25T09:53:41.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.137+0000 7f4c8159a640 1 -- 172.21.15.139:0/457591565 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c032810 con 0x7f4c840af8a0 2024-04-25T09:53:41.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.138+0000 7f4c8897d640 1 -- 172.21.15.139:0/457591565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c840af8a0 msgr2=0x7f4c840b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.138+0000 7f4c8897d640 1 --2- 172.21.15.139:0/457591565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c840af8a0 0x7f4c840b2d50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009870 tx=0x7f4c6c02a270 comp rx=0 tx=0).stop 2024-04-25T09:53:41.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.138+0000 7f4c8897d640 1 -- 172.21.15.139:0/457591565 shutdown_connections 2024-04-25T09:53:41.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.138+0000 7f4c8897d640 1 --2- 172.21.15.139:0/457591565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c840afc80 0x7f4c840cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.138+0000 7f4c8897d640 1 --2- 172.21.15.139:0/457591565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c840af8a0 0x7f4c840b2d50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.138+0000 7f4c8897d640 1 -- 172.21.15.139:0/457591565 >> 172.21.15.139:0/457591565 conn(0x7f4c840b2230 msgr2=0x7f4c840b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.138+0000 7f4c8897d640 1 -- 172.21.15.139:0/457591565 shutdown_connections 2024-04-25T09:53:41.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.138+0000 7f4c8897d640 1 -- 172.21.15.139:0/457591565 wait complete. 2024-04-25T09:53:41.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.139+0000 7f4c8897d640 1 Processor -- start 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.139+0000 7f4c8897d640 1 -- start start 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c8897d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c8406b700 0x7f4c84158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c8897d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c840af8a0 0x7f4c841f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c8897d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c840afc80 0x7f4c840ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c8897d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8406b270 con 0x7f4c840af8a0 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c8897d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c840cddd0 con 0x7f4c8406b700 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c8897d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8406b3f0 con 0x7f4c840afc80 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c81d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c840af8a0 0x7f4c841f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c81d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c840af8a0 0x7f4c841f2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40482/0 (socket says 172.21.15.139:40482) 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c81d9b640 1 -- 172.21.15.139:0/225714684 learned_addr learned my addr 172.21.15.139:0/225714684 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c8259c640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c8406b700 0x7f4c84158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c82d9d640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c840afc80 0x7f4c840ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c81d9b640 1 -- 172.21.15.139:0/225714684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c840afc80 msgr2=0x7f4c840ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c81d9b640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c840afc80 0x7f4c840ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c81d9b640 1 -- 172.21.15.139:0/225714684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c8406b700 msgr2=0x7f4c84158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c81d9b640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c8406b700 0x7f4c84158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c81d9b640 1 -- 172.21.15.139:0/225714684 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c841f6ac0 con 0x7f4c840af8a0 2024-04-25T09:53:41.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c81d9b640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c840af8a0 0x7f4c841f2c10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d800 tx=0x7f4c7c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c7affd640 1 -- 172.21.15.139:0/225714684 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c004030 con 0x7f4c840af8a0 2024-04-25T09:53:41.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c7affd640 1 -- 172.21.15.139:0/225714684 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7c0041d0 con 0x7f4c840af8a0 2024-04-25T09:53:41.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c8897d640 1 -- 172.21.15.139:0/225714684 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c84156ed0 con 0x7f4c840af8a0 2024-04-25T09:53:41.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c7affd640 1 -- 172.21.15.139:0/225714684 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0096a0 con 0x7f4c840af8a0 2024-04-25T09:53:41.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.140+0000 7f4c8897d640 1 -- 172.21.15.139:0/225714684 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c841f6d60 con 0x7f4c840af8a0 2024-04-25T09:53:41.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.142+0000 7f4c8897d640 1 -- 172.21.15.139:0/225714684 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c84074520 con 0x7f4c840af8a0 2024-04-25T09:53:41.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.142+0000 7f4c7affd640 1 -- 172.21.15.139:0/225714684 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c7c009840 con 0x7f4c840af8a0 2024-04-25T09:53:41.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.142+0000 7f4c7affd640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c6406b4d0 0x7f4c6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.142+0000 7f4c7affd640 1 -- 172.21.15.139:0/225714684 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4c7c08a660 con 0x7f4c840af8a0 2024-04-25T09:53:41.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.142+0000 7f4c8259c640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c6406b4d0 0x7f4c6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.143+0000 7f4c8259c640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c6406b4d0 0x7f4c6406d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c034000 tx=0x7f4c6c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.146+0000 7f4c7affd640 1 -- 172.21.15.139:0/225714684 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c7c058b50 con 0x7f4c840af8a0 2024-04-25T09:53:41.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.314+0000 7f4c8897d640 1 -- 172.21.15.139:0/225714684 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c840008d0 con 0x7f4c840af8a0 2024-04-25T09:53:41.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.315+0000 7f4c7affd640 1 -- 172.21.15.139:0/225714684 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c7c05cbf0 con 0x7f4c840af8a0 2024-04-25T09:53:41.317 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:53:41.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.317+0000 7f4c8897d640 1 -- 172.21.15.139:0/225714684 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c6406b4d0 msgr2=0x7f4c6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.317+0000 7f4c8897d640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c6406b4d0 0x7f4c6406d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c034000 tx=0x7f4c6c02e000 comp rx=0 tx=0).stop 2024-04-25T09:53:41.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.317+0000 7f4c8897d640 1 -- 172.21.15.139:0/225714684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c840af8a0 msgr2=0x7f4c841f2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.317+0000 7f4c8897d640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c840af8a0 0x7f4c841f2c10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d800 tx=0x7f4c7c00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.317+0000 7f4c8897d640 1 -- 172.21.15.139:0/225714684 shutdown_connections 2024-04-25T09:53:41.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.317+0000 7f4c8897d640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c840af8a0 0x7f4c841f2c10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.317+0000 7f4c8897d640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c6406b4d0 0x7f4c6406d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.317+0000 7f4c8897d640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c840afc80 0x7f4c840ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.317+0000 7f4c8897d640 1 --2- 172.21.15.139:0/225714684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c8406b700 0x7f4c84158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.317+0000 7f4c8897d640 1 -- 172.21.15.139:0/225714684 >> 172.21.15.139:0/225714684 conn(0x7f4c840b2230 msgr2=0x7f4c840aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.317+0000 7f4c8897d640 1 -- 172.21.15.139:0/225714684 shutdown_connections 2024-04-25T09:53:41.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.317+0000 7f4c8897d640 1 -- 172.21.15.139:0/225714684 wait complete. 2024-04-25T09:53:41.330 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:41.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.427+0000 7f62e0d9e640 1 Processor -- start 2024-04-25T09:53:41.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.427+0000 7f62e0d9e640 1 -- start start 2024-04-25T09:53:41.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.428+0000 7f62e0d9e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f62dc144090 0x7f62dc0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.428+0000 7f62e0d9e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62dc0b2a50 con 0x7f62dc0cc9e0 2024-04-25T09:53:41.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.428+0000 7f62e0d9e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62dc15b090 con 0x7f62dc06ea40 2024-04-25T09:53:41.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.428+0000 7f62e0d9e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62dc15b370 con 0x7f62dc144090 2024-04-25T09:53:41.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.428+0000 7f62da59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f62dc144090 0x7f62dc0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.428+0000 7f62da59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f62dc144090 0x7f62dc0b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40484/0 (socket says 172.21.15.139:40484) 2024-04-25T09:53:41.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.428+0000 7f62d9d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f62dc0cc9e0 0x7f62dc06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48662/0 (socket says 172.21.15.139:48662) 2024-04-25T09:53:41.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.428+0000 7f62da59c640 1 -- 172.21.15.139:0/289692491 learned_addr learned my addr 172.21.15.139:0/289692491 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:41.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.428+0000 7f62da59c640 1 -- 172.21.15.139:0/289692491 >> v1:172.21.15.150:6789/0 conn(0x7f62dc06ea40 legacy=0x7f62dc15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.428+0000 7f62da59c640 1 -- 172.21.15.139:0/289692491 >> v1:172.21.15.139:6789/0 conn(0x7f62dc0cc9e0 legacy=0x7f62dc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.428+0000 7f62da59c640 1 -- 172.21.15.139:0/289692491 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc15b650 con 0x7f62dc144090 2024-04-25T09:53:41.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.429+0000 7f62da59c640 1 --2- 172.21.15.139:0/289692491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc144090 0x7f62dc0b31a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00d150 tx=0x7f62cc02c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c589a6c78a25a748 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.429+0000 7f62d959a640 1 -- 172.21.15.139:0/289692491 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc009600 con 0x7f62dc144090 2024-04-25T09:53:41.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.429+0000 7f62d959a640 1 -- 172.21.15.139:0/289692491 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc037070 con 0x7f62dc144090 2024-04-25T09:53:41.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.429+0000 7f62d959a640 1 -- 172.21.15.139:0/289692491 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0099c0 con 0x7f62dc144090 2024-04-25T09:53:41.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.430+0000 7f62e0d9e640 1 -- 172.21.15.139:0/289692491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc144090 msgr2=0x7f62dc0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.430+0000 7f62e0d9e640 1 --2- 172.21.15.139:0/289692491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc144090 0x7f62dc0b31a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00d150 tx=0x7f62cc02c930 comp rx=0 tx=0).stop 2024-04-25T09:53:41.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.430+0000 7f62e0d9e640 1 -- 172.21.15.139:0/289692491 shutdown_connections 2024-04-25T09:53:41.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.430+0000 7f62e0d9e640 1 --2- 172.21.15.139:0/289692491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc144090 0x7f62dc0b31a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.430+0000 7f62e0d9e640 1 -- 172.21.15.139:0/289692491 >> 172.21.15.139:0/289692491 conn(0x7f62dc067000 msgr2=0x7f62dc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.430+0000 7f62e0d9e640 1 -- 172.21.15.139:0/289692491 shutdown_connections 2024-04-25T09:53:41.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.430+0000 7f62e0d9e640 1 -- 172.21.15.139:0/289692491 wait complete. 2024-04-25T09:53:41.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.431+0000 7f62e0d9e640 1 Processor -- start 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.431+0000 7f62e0d9e640 1 -- start start 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62e0d9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc06ea40 0x7f62dc0acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62e0d9e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62dc0cc9e0 0x7f62dc1f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62e0d9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62dc144090 0x7f62dc0b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62e0d9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc144470 con 0x7f62dc06ea40 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62e0d9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15bc10 con 0x7f62dc144090 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62e0d9e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15ad80 con 0x7f62dc0cc9e0 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62da59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc06ea40 0x7f62dc0acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62da59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc06ea40 0x7f62dc0acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40488/0 (socket says 172.21.15.139:40488) 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62da59c640 1 -- 172.21.15.139:0/2575023728 learned_addr learned my addr 172.21.15.139:0/2575023728 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62d9d9b640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62dc0cc9e0 0x7f62dc1f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62dad9d640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62dc144090 0x7f62dc0b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62da59c640 1 -- 172.21.15.139:0/2575023728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62dc0cc9e0 msgr2=0x7f62dc1f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62da59c640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62dc0cc9e0 0x7f62dc1f3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62da59c640 1 -- 172.21.15.139:0/2575023728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62dc144090 msgr2=0x7f62dc0b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62da59c640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62dc144090 0x7f62dc0b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62da59c640 1 -- 172.21.15.139:0/2575023728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc15c080 con 0x7f62dc06ea40 2024-04-25T09:53:41.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.432+0000 7f62da59c640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc06ea40 0x7f62dc0acd80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f62cc000f80 tx=0x7f62cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.433+0000 7f62caffd640 1 -- 172.21.15.139:0/2575023728 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc009e40 con 0x7f62dc06ea40 2024-04-25T09:53:41.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.433+0000 7f62caffd640 1 -- 172.21.15.139:0/2575023728 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc032420 con 0x7f62dc06ea40 2024-04-25T09:53:41.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.433+0000 7f62e0d9e640 1 -- 172.21.15.139:0/2575023728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc15c360 con 0x7f62dc06ea40 2024-04-25T09:53:41.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.433+0000 7f62caffd640 1 -- 172.21.15.139:0/2575023728 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0327f0 con 0x7f62dc06ea40 2024-04-25T09:53:41.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.433+0000 7f62e0d9e640 1 -- 172.21.15.139:0/2575023728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc15c920 con 0x7f62dc06ea40 2024-04-25T09:53:41.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.434+0000 7f62e0d9e640 1 -- 172.21.15.139:0/2575023728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc074520 con 0x7f62dc06ea40 2024-04-25T09:53:41.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.434+0000 7f62caffd640 1 -- 172.21.15.139:0/2575023728 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62cc032990 con 0x7f62dc06ea40 2024-04-25T09:53:41.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.434+0000 7f62caffd640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f62bc06b4d0 0x7f62bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.435+0000 7f62d9d9b640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f62bc06b4d0 0x7f62bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.435+0000 7f62caffd640 1 -- 172.21.15.139:0/2575023728 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f62cc0ab410 con 0x7f62dc06ea40 2024-04-25T09:53:41.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.435+0000 7f62d9d9b640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f62bc06b4d0 0x7f62bc06d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0b4370 tx=0x7f62d0007c80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.439+0000 7f62caffd640 1 -- 172.21.15.139:0/2575023728 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62cc079f50 con 0x7f62dc06ea40 2024-04-25T09:53:41.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.607+0000 7f62e0d9e640 1 -- 172.21.15.139:0/2575023728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f62dc0cd0e0 con 0x7f62dc06ea40 2024-04-25T09:53:41.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.608+0000 7f62caffd640 1 -- 172.21.15.139:0/2575023728 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f62cc079850 con 0x7f62dc06ea40 2024-04-25T09:53:41.610 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:41.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.610+0000 7f62e0d9e640 1 -- 172.21.15.139:0/2575023728 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f62bc06b4d0 msgr2=0x7f62bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.610+0000 7f62e0d9e640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f62bc06b4d0 0x7f62bc06d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0b4370 tx=0x7f62d0007c80 comp rx=0 tx=0).stop 2024-04-25T09:53:41.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.610+0000 7f62e0d9e640 1 -- 172.21.15.139:0/2575023728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc06ea40 msgr2=0x7f62dc0acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.610+0000 7f62e0d9e640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc06ea40 0x7f62dc0acd80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f62cc000f80 tx=0x7f62cc004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.610+0000 7f62e0d9e640 1 -- 172.21.15.139:0/2575023728 shutdown_connections 2024-04-25T09:53:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.610+0000 7f62e0d9e640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f62bc06b4d0 0x7f62bc06d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.610+0000 7f62e0d9e640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62dc144090 0x7f62dc0b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.610+0000 7f62e0d9e640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62dc0cc9e0 0x7f62dc1f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.610+0000 7f62e0d9e640 1 --2- 172.21.15.139:0/2575023728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc06ea40 0x7f62dc0acd80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.610+0000 7f62e0d9e640 1 -- 172.21.15.139:0/2575023728 >> 172.21.15.139:0/2575023728 conn(0x7f62dc067000 msgr2=0x7f62dc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.610+0000 7f62e0d9e640 1 -- 172.21.15.139:0/2575023728 shutdown_connections 2024-04-25T09:53:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.610+0000 7f62e0d9e640 1 -- 172.21.15.139:0/2575023728 wait complete. 2024-04-25T09:53:41.623 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:41.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.718+0000 7fcd823d5640 1 Processor -- start 2024-04-25T09:53:41.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.718+0000 7fcd823d5640 1 -- start start 2024-04-25T09:53:41.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.718+0000 7fcd823d5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcd7c15dc60 0x7fcd7c0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.718+0000 7fcd823d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd7c161850 0x7fcd7c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.718+0000 7fcd823d5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcd7c0d0970 0x7fcd7c0d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.718+0000 7fcd823d5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c0ce780 con 0x7fcd7c0d0970 2024-04-25T09:53:41.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.718+0000 7fcd823d5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c0d3730 con 0x7fcd7c15dc60 2024-04-25T09:53:41.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.718+0000 7fcd823d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c0cf250 con 0x7fcd7c161850 2024-04-25T09:53:41.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.719+0000 7fcd80972640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcd7c0d0970 0x7fcd7c0d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.719+0000 7fcd80972640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcd7c0d0970 0x7fcd7c0d0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40490/0 (socket says 172.21.15.139:40490) 2024-04-25T09:53:41.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.719+0000 7fcd80972640 1 -- 172.21.15.139:0/1337506466 learned_addr learned my addr 172.21.15.139:0/1337506466 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:41.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.719+0000 7fcd7b7fe640 1 --2- 172.21.15.139:0/1337506466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd7c161850 0x7fcd7c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.719+0000 7fcd7bfff640 1 --2- 172.21.15.139:0/1337506466 >> v2:172.21.15.150:3300/0 conn(0x7fcd7c15dc60 0x7fcd7c0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.719+0000 7fcd80972640 1 -- 172.21.15.139:0/1337506466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd7c161850 msgr2=0x7fcd7c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.719+0000 7fcd80972640 1 --2- 172.21.15.139:0/1337506466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd7c161850 0x7fcd7c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.719+0000 7fcd80972640 1 -- 172.21.15.139:0/1337506466 >> v2:172.21.15.150:3300/0 conn(0x7fcd7c15dc60 msgr2=0x7fcd7c0b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.719+0000 7fcd80972640 1 --2- 172.21.15.139:0/1337506466 >> v2:172.21.15.150:3300/0 conn(0x7fcd7c15dc60 0x7fcd7c0b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.719+0000 7fcd80972640 1 -- 172.21.15.139:0/1337506466 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c0ce9b0 con 0x7fcd7c0d0970 2024-04-25T09:53:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.719+0000 7fcd7bfff640 1 --2- 172.21.15.139:0/1337506466 >> v2:172.21.15.150:3300/0 conn(0x7fcd7c15dc60 0x7fcd7c0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd80972640 1 --2- 172.21.15.139:0/1337506466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c0d0970 0x7fcd7c0d0e00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c08c680 tx=0x7fcd7402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f178225bc652cc6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd7affd640 1 -- 172.21.15.139:0/1337506466 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74036070 con 0x7fcd7c0d0970 2024-04-25T09:53:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd7affd640 1 -- 172.21.15.139:0/1337506466 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd74031600 con 0x7fcd7c0d0970 2024-04-25T09:53:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd7affd640 1 -- 172.21.15.139:0/1337506466 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd740319d0 con 0x7fcd7c0d0970 2024-04-25T09:53:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd823d5640 1 -- 172.21.15.139:0/1337506466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c0d0970 msgr2=0x7fcd7c0d0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd823d5640 1 --2- 172.21.15.139:0/1337506466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c0d0970 0x7fcd7c0d0e00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c08c680 tx=0x7fcd7402a6c0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd823d5640 1 -- 172.21.15.139:0/1337506466 shutdown_connections 2024-04-25T09:53:41.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd823d5640 1 --2- 172.21.15.139:0/1337506466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c0d0970 0x7fcd7c0d0e00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd823d5640 1 --2- 172.21.15.139:0/1337506466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd7c161850 0x7fcd7c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd823d5640 1 --2- 172.21.15.139:0/1337506466 >> v2:172.21.15.150:3300/0 conn(0x7fcd7c15dc60 0x7fcd7c0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd823d5640 1 -- 172.21.15.139:0/1337506466 >> 172.21.15.139:0/1337506466 conn(0x7fcd7c0b5910 msgr2=0x7fcd7c0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd823d5640 1 -- 172.21.15.139:0/1337506466 shutdown_connections 2024-04-25T09:53:41.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.720+0000 7fcd823d5640 1 -- 172.21.15.139:0/1337506466 wait complete. 2024-04-25T09:53:41.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.721+0000 7fcd823d5640 1 Processor -- start 2024-04-25T09:53:41.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.722+0000 7fcd823d5640 1 -- start start 2024-04-25T09:53:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.722+0000 7fcd823d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd7c0d0970 0x7fcd7c0c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.722+0000 7fcd823d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd7c15dc60 0x7fcd7c1f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.722+0000 7fcd823d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c161850 0x7fcd7c1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.722+0000 7fcd823d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c0d1fc0 con 0x7fcd7c161850 2024-04-25T09:53:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.722+0000 7fcd823d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c0cef70 con 0x7fcd7c15dc60 2024-04-25T09:53:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.722+0000 7fcd823d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c15e040 con 0x7fcd7c0d0970 2024-04-25T09:53:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.722+0000 7fcd80972640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c161850 0x7fcd7c1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd80972640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c161850 0x7fcd7c1592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40502/0 (socket says 172.21.15.139:40502) 2024-04-25T09:53:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd80972640 1 -- 172.21.15.139:0/3058585658 learned_addr learned my addr 172.21.15.139:0/3058585658 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd7bfff640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd7c0d0970 0x7fcd7c0c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd7b7fe640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd7c15dc60 0x7fcd7c1f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd80972640 1 -- 172.21.15.139:0/3058585658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd7c0d0970 msgr2=0x7fcd7c0c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd80972640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd7c0d0970 0x7fcd7c0c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd80972640 1 -- 172.21.15.139:0/3058585658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd7c15dc60 msgr2=0x7fcd7c1f3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd80972640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd7c15dc60 0x7fcd7c1f3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd80972640 1 -- 172.21.15.139:0/3058585658 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c1f4bb0 con 0x7fcd7c161850 2024-04-25T09:53:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd80972640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c161850 0x7fcd7c1592b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009e50 tx=0x7fcd74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd78ff9640 1 -- 172.21.15.139:0/3058585658 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74041020 con 0x7fcd7c161850 2024-04-25T09:53:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd78ff9640 1 -- 172.21.15.139:0/3058585658 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd74007780 con 0x7fcd7c161850 2024-04-25T09:53:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd823d5640 1 -- 172.21.15.139:0/3058585658 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c1f45f0 con 0x7fcd7c161850 2024-04-25T09:53:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd78ff9640 1 -- 172.21.15.139:0/3058585658 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74036040 con 0x7fcd7c161850 2024-04-25T09:53:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.723+0000 7fcd823d5640 1 -- 172.21.15.139:0/3058585658 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c1f4f00 con 0x7fcd7c161850 2024-04-25T09:53:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.724+0000 7fcd823d5640 1 -- 172.21.15.139:0/3058585658 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c074520 con 0x7fcd7c161850 2024-04-25T09:53:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.725+0000 7fcd78ff9640 1 -- 172.21.15.139:0/3058585658 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd74044020 con 0x7fcd7c161850 2024-04-25T09:53:41.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.725+0000 7fcd78ff9640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd6806b4d0 0x7fcd6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:41.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.725+0000 7fcd78ff9640 1 -- 172.21.15.139:0/3058585658 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcd740ab190 con 0x7fcd7c161850 2024-04-25T09:53:41.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.725+0000 7fcd7bfff640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd6806b4d0 0x7fcd6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:41.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.725+0000 7fcd7bfff640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd6806b4d0 0x7fcd6806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c006fd0 tx=0x7fcd6c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:41.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.729+0000 7fcd78ff9640 1 -- 172.21.15.139:0/3058585658 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd7404b030 con 0x7fcd7c161850 2024-04-25T09:53:41.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.936+0000 7fcd823d5640 1 -- 172.21.15.139:0/3058585658 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd7c073d80 con 0x7fcd7c161850 2024-04-25T09:53:41.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.937+0000 7fcd78ff9640 1 -- 172.21.15.139:0/3058585658 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fcd74079e20 con 0x7fcd7c161850 2024-04-25T09:53:41.939 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:41.939 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/892226922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":892226922},{"type":"v1","addr":"172.21.15.150:6834","nonce":892226922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/582019435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":582019435},{"type":"v1","addr":"172.21.15.139:6837","nonce":582019435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/2751714076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2751714076},{"type":"v1","addr":"172.21.15.139:6834","nonce":2751714076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:15.042067+0000","modified":"2024-04-25T09:53:30.094430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2160557452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2160557452},{"type":"v1","addr":"172.21.15.150:6835","nonce":2160557452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:53:41.939 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 9 2024-04-25T09:53:41.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.939+0000 7fcd823d5640 1 -- 172.21.15.139:0/3058585658 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd6806b4d0 msgr2=0x7fcd6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.939+0000 7fcd823d5640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd6806b4d0 0x7fcd6806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c006fd0 tx=0x7fcd6c008040 comp rx=0 tx=0).stop 2024-04-25T09:53:41.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.939+0000 7fcd823d5640 1 -- 172.21.15.139:0/3058585658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c161850 msgr2=0x7fcd7c1592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:41.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.939+0000 7fcd823d5640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c161850 0x7fcd7c1592b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009e50 tx=0x7fcd74004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:41.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.939+0000 7fcd823d5640 1 -- 172.21.15.139:0/3058585658 shutdown_connections 2024-04-25T09:53:41.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.939+0000 7fcd823d5640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c161850 0x7fcd7c1592b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.939+0000 7fcd823d5640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd6806b4d0 0x7fcd6806d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.939+0000 7fcd823d5640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd7c15dc60 0x7fcd7c1f3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.939+0000 7fcd823d5640 1 --2- 172.21.15.139:0/3058585658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd7c0d0970 0x7fcd7c0c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:41.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.939+0000 7fcd823d5640 1 -- 172.21.15.139:0/3058585658 >> 172.21.15.139:0/3058585658 conn(0x7fcd7c0b5910 msgr2=0x7fcd7c15e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:41.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.939+0000 7fcd823d5640 1 -- 172.21.15.139:0/3058585658 shutdown_connections 2024-04-25T09:53:41.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:41.939+0000 7fcd823d5640 1 -- 172.21.15.139:0/3058585658 wait complete. 2024-04-25T09:53:41.952 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.047+0000 7f4caa85d640 1 Processor -- start 2024-04-25T09:53:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.047+0000 7f4caa85d640 1 -- start start 2024-04-25T09:53:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.047+0000 7f4caa85d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4ca406ab60 0x7f4ca406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.047+0000 7f4caa85d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca40d0400 con 0x7f4ca4160e90 2024-04-25T09:53:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.047+0000 7f4caa85d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca40d06e0 con 0x7f4ca40b36f0 2024-04-25T09:53:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.047+0000 7f4caa85d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca40d14a0 con 0x7f4ca406ab60 2024-04-25T09:53:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.048+0000 7f4ca37fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4ca406ab60 0x7f4ca406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.048+0000 7f4ca3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4ca4160e90 0x7f4ca40afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48666/0 (socket says 172.21.15.139:48666) 2024-04-25T09:53:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.048+0000 7f4ca3fff640 1 -- 172.21.15.139:0/2708250271 learned_addr learned my addr 172.21.15.139:0/2708250271 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.048+0000 7f4ca37fe640 1 -- 172.21.15.139:0/2708250271 >> v1:172.21.15.150:6789/0 conn(0x7f4ca40b36f0 legacy=0x7f4ca40b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:42.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.048+0000 7f4ca37fe640 1 -- 172.21.15.139:0/2708250271 >> v1:172.21.15.139:6789/0 conn(0x7f4ca4160e90 legacy=0x7f4ca40afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.048+0000 7f4ca37fe640 1 -- 172.21.15.139:0/2708250271 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca40d09c0 con 0x7f4ca406ab60 2024-04-25T09:53:42.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.048+0000 7f4ca2ffd640 1 -- 172.21.15.139:0/2708250271 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785597804 0 0) 0x7f4ca40d0400 con 0x7f4ca4160e90 2024-04-25T09:53:42.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.048+0000 7f4ca37fe640 1 --2- 172.21.15.139:0/2708250271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ca406ab60 0x7f4ca406aff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0097d0 tx=0x7f4c8c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df367f0568ccf74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.049+0000 7f4ca2ffd640 1 -- 172.21.15.139:0/2708250271 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c037070 con 0x7f4ca406ab60 2024-04-25T09:53:42.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.049+0000 7f4ca2ffd640 1 -- 172.21.15.139:0/2708250271 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c032440 con 0x7f4ca406ab60 2024-04-25T09:53:42.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.049+0000 7f4ca2ffd640 1 -- 172.21.15.139:0/2708250271 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c032810 con 0x7f4ca406ab60 2024-04-25T09:53:42.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.049+0000 7f4caa85d640 1 -- 172.21.15.139:0/2708250271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ca406ab60 msgr2=0x7f4ca406aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.049+0000 7f4caa85d640 1 --2- 172.21.15.139:0/2708250271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ca406ab60 0x7f4ca406aff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0097d0 tx=0x7f4c8c02a180 comp rx=0 tx=0).stop 2024-04-25T09:53:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.049+0000 7f4caa85d640 1 -- 172.21.15.139:0/2708250271 shutdown_connections 2024-04-25T09:53:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.049+0000 7f4caa85d640 1 --2- 172.21.15.139:0/2708250271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ca406ab60 0x7f4ca406aff0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.049+0000 7f4caa85d640 1 -- 172.21.15.139:0/2708250271 >> 172.21.15.139:0/2708250271 conn(0x7f4ca40b23f0 msgr2=0x7f4ca40b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.049+0000 7f4caa85d640 1 -- 172.21.15.139:0/2708250271 shutdown_connections 2024-04-25T09:53:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.049+0000 7f4caa85d640 1 -- 172.21.15.139:0/2708250271 wait complete. 2024-04-25T09:53:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.050+0000 7f4caa85d640 1 Processor -- start 2024-04-25T09:53:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.051+0000 7f4caa85d640 1 -- start start 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.051+0000 7f4caa85d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ca406ab60 0x7f4ca41e7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.051+0000 7f4caa85d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ca40b36f0 0x7f4ca40cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.051+0000 7f4caa85d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ca4160e90 0x7f4ca4144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.051+0000 7f4caa85d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca4161270 con 0x7f4ca406ab60 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.051+0000 7f4caa85d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca40d0f80 con 0x7f4ca4160e90 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.051+0000 7f4caa85d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca41e8290 con 0x7f4ca40b36f0 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.051+0000 7f4ca3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ca406ab60 0x7f4ca41e7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.051+0000 7f4ca3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ca406ab60 0x7f4ca41e7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40510/0 (socket says 172.21.15.139:40510) 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.051+0000 7f4ca3fff640 1 -- 172.21.15.139:0/236051558 learned_addr learned my addr 172.21.15.139:0/236051558 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4ca37fe640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ca40b36f0 0x7f4ca40cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4ca8dfa640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ca4160e90 0x7f4ca4144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4ca3fff640 1 -- 172.21.15.139:0/236051558 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ca40b36f0 msgr2=0x7f4ca40cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4ca3fff640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ca40b36f0 0x7f4ca40cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4ca3fff640 1 -- 172.21.15.139:0/236051558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ca4160e90 msgr2=0x7f4ca4144fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4ca3fff640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ca4160e90 0x7f4ca4144fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4ca3fff640 1 -- 172.21.15.139:0/236051558 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca41e8570 con 0x7f4ca406ab60 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4ca3fff640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ca406ab60 0x7f4ca41e7cb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400c460 tx=0x7f4c9400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4ca0ff9640 1 -- 172.21.15.139:0/236051558 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94018070 con 0x7f4ca406ab60 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4ca0ff9640 1 -- 172.21.15.139:0/236051558 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c94007500 con 0x7f4ca406ab60 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4caa85d640 1 -- 172.21.15.139:0/236051558 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca41e8850 con 0x7f4ca406ab60 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4ca0ff9640 1 -- 172.21.15.139:0/236051558 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94004a20 con 0x7f4ca406ab60 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.052+0000 7f4caa85d640 1 -- 172.21.15.139:0/236051558 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca40cb0c0 con 0x7f4ca406ab60 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.053+0000 7f4caa85d640 1 -- 172.21.15.139:0/236051558 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca4074520 con 0x7f4ca406ab60 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.053+0000 7f4ca0ff9640 1 -- 172.21.15.139:0/236051558 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c9401b070 con 0x7f4ca406ab60 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.054+0000 7f4ca0ff9640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c8406b4d0 0x7f4c8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.054+0000 7f4ca0ff9640 1 -- 172.21.15.139:0/236051558 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4c94089760 con 0x7f4ca406ab60 2024-04-25T09:53:42.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.054+0000 7f4ca37fe640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c8406b4d0 0x7f4c8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.054+0000 7f4ca37fe640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c8406b4d0 0x7f4c8406d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c034000 tx=0x7f4c8c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.058+0000 7f4ca0ff9640 1 -- 172.21.15.139:0/236051558 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c94057c50 con 0x7f4ca406ab60 2024-04-25T09:53:42.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.227+0000 7f4caa85d640 1 -- 172.21.15.139:0/236051558 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ca40728b0 con 0x7f4ca406ab60 2024-04-25T09:53:42.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.228+0000 7f4ca0ff9640 1 -- 172.21.15.139:0/236051558 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f4c9405bcf0 con 0x7f4ca406ab60 2024-04-25T09:53:42.231 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:42.231 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:42.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.230+0000 7f4caa85d640 1 -- 172.21.15.139:0/236051558 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c8406b4d0 msgr2=0x7f4c8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.230+0000 7f4caa85d640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c8406b4d0 0x7f4c8406d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c034000 tx=0x7f4c8c02e000 comp rx=0 tx=0).stop 2024-04-25T09:53:42.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.230+0000 7f4caa85d640 1 -- 172.21.15.139:0/236051558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ca406ab60 msgr2=0x7f4ca41e7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.230+0000 7f4caa85d640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ca406ab60 0x7f4ca41e7cb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400c460 tx=0x7f4c9400c920 comp rx=0 tx=0).stop 2024-04-25T09:53:42.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.230+0000 7f4caa85d640 1 -- 172.21.15.139:0/236051558 shutdown_connections 2024-04-25T09:53:42.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.230+0000 7f4caa85d640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c8406b4d0 0x7f4c8406d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.230+0000 7f4caa85d640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ca4160e90 0x7f4ca4144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.230+0000 7f4caa85d640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ca40b36f0 0x7f4ca40cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.230+0000 7f4caa85d640 1 --2- 172.21.15.139:0/236051558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ca406ab60 0x7f4ca41e7cb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.230+0000 7f4caa85d640 1 -- 172.21.15.139:0/236051558 >> 172.21.15.139:0/236051558 conn(0x7f4ca40b23f0 msgr2=0x7f4ca40b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:42.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.231+0000 7f4caa85d640 1 -- 172.21.15.139:0/236051558 shutdown_connections 2024-04-25T09:53:42.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.231+0000 7f4caa85d640 1 -- 172.21.15.139:0/236051558 wait complete. 2024-04-25T09:53:42.244 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:53:42.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.339+0000 7fbfe2313640 1 Processor -- start 2024-04-25T09:53:42.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.339+0000 7fbfe2313640 1 -- start start 2024-04-25T09:53:42.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfe2313640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfdc0b6af0 0x7fbfdc06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfe2313640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbfdc06c5b0 0x7fbfdc0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfe2313640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfdc0cf110 con 0x7fbfdc1611e0 2024-04-25T09:53:42.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfe2313640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfdc0d1fa0 con 0x7fbfdc06c5b0 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfe2313640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc0d2560 con 0x7fbfdc0b6af0 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfdbfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbfdc1611e0 0x7fbfdc0cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48672/0 (socket says 172.21.15.139:48672) 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfe08b0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbfdc06c5b0 0x7fbfdc0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfdbfff640 1 -- 172.21.15.139:0/147191450 learned_addr learned my addr 172.21.15.139:0/147191450 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfdb7fe640 1 --2- 172.21.15.139:0/147191450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfdc0b6af0 0x7fbfdc06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfe08b0640 1 -- 172.21.15.139:0/147191450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfdc0b6af0 msgr2=0x7fbfdc06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfe08b0640 1 --2- 172.21.15.139:0/147191450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfdc0b6af0 0x7fbfdc06c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfe08b0640 1 -- 172.21.15.139:0/147191450 >> v1:172.21.15.139:6789/0 conn(0x7fbfdc1611e0 legacy=0x7fbfdc0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfe08b0640 1 -- 172.21.15.139:0/147191450 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc0b6f50 con 0x7fbfdc06c5b0 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.340+0000 7fbfdaffd640 1 -- 172.21.15.139:0/147191450 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660143179 0 0) 0x7fbfdc0cf110 con 0x7fbfdc1611e0 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.341+0000 7fbfe08b0640 1 --2- 172.21.15.139:0/147191450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfdc06c5b0 0x7fbfdc0d29f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009d90 tx=0x7fbfd402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8ff5aee057ef5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.341+0000 7fbfdaffd640 1 -- 172.21.15.139:0/147191450 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4007740 con 0x7fbfdc06c5b0 2024-04-25T09:53:42.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.341+0000 7fbfdaffd640 1 -- 172.21.15.139:0/147191450 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd40078e0 con 0x7fbfdc06c5b0 2024-04-25T09:53:42.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.341+0000 7fbfdaffd640 1 -- 172.21.15.139:0/147191450 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4007cb0 con 0x7fbfdc06c5b0 2024-04-25T09:53:42.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.342+0000 7fbfe2313640 1 -- 172.21.15.139:0/147191450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfdc06c5b0 msgr2=0x7fbfdc0d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.342+0000 7fbfe2313640 1 --2- 172.21.15.139:0/147191450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfdc06c5b0 0x7fbfdc0d29f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009d90 tx=0x7fbfd402a660 comp rx=0 tx=0).stop 2024-04-25T09:53:42.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.342+0000 7fbfe2313640 1 -- 172.21.15.139:0/147191450 shutdown_connections 2024-04-25T09:53:42.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.342+0000 7fbfe2313640 1 --2- 172.21.15.139:0/147191450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfdc06c5b0 0x7fbfdc0d29f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.342+0000 7fbfe2313640 1 --2- 172.21.15.139:0/147191450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfdc0b6af0 0x7fbfdc06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.342+0000 7fbfe2313640 1 -- 172.21.15.139:0/147191450 >> 172.21.15.139:0/147191450 conn(0x7fbfdc0b5b00 msgr2=0x7fbfdc0d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:42.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.342+0000 7fbfe2313640 1 -- 172.21.15.139:0/147191450 shutdown_connections 2024-04-25T09:53:42.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.342+0000 7fbfe2313640 1 -- 172.21.15.139:0/147191450 wait complete. 2024-04-25T09:53:42.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.343+0000 7fbfe2313640 1 Processor -- start 2024-04-25T09:53:42.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.343+0000 7fbfe2313640 1 -- start start 2024-04-25T09:53:42.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe2313640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfdc06c5b0 0x7fbfdc157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe2313640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfdc0b6af0 0x7fbfdc1f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe2313640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfdc1611e0 0x7fbfdc1f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe2313640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc0b2a50 con 0x7fbfdc1611e0 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe2313640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc0b2770 con 0x7fbfdc06c5b0 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe2313640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc0b2d30 con 0x7fbfdc0b6af0 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe08b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfdc1611e0 0x7fbfdc1f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe08b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfdc1611e0 0x7fbfdc1f2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40524/0 (socket says 172.21.15.139:40524) 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe08b0640 1 -- 172.21.15.139:0/2531314007 learned_addr learned my addr 172.21.15.139:0/2531314007 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfdb7fe640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfdc0b6af0 0x7fbfdc1f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfdbfff640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfdc06c5b0 0x7fbfdc157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe08b0640 1 -- 172.21.15.139:0/2531314007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfdc0b6af0 msgr2=0x7fbfdc1f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe08b0640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfdc0b6af0 0x7fbfdc1f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe08b0640 1 -- 172.21.15.139:0/2531314007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfdc06c5b0 msgr2=0x7fbfdc157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe08b0640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfdc06c5b0 0x7fbfdc157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe08b0640 1 -- 172.21.15.139:0/2531314007 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc0c78d0 con 0x7fbfdc1611e0 2024-04-25T09:53:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfdb7fe640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfdc0b6af0 0x7fbfdc1f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfe08b0640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfdc1611e0 0x7fbfdc1f2bb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbfd402ac50 tx=0x7fbfd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.344+0000 7fbfd8ff9640 1 -- 172.21.15.139:0/2531314007 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4040070 con 0x7fbfdc1611e0 2024-04-25T09:53:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.345+0000 7fbfd8ff9640 1 -- 172.21.15.139:0/2531314007 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd4031bc0 con 0x7fbfdc1611e0 2024-04-25T09:53:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.345+0000 7fbfe2313640 1 -- 172.21.15.139:0/2531314007 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc0c7b90 con 0x7fbfdc1611e0 2024-04-25T09:53:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.345+0000 7fbfd8ff9640 1 -- 172.21.15.139:0/2531314007 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4045460 con 0x7fbfdc1611e0 2024-04-25T09:53:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.345+0000 7fbfe2313640 1 -- 172.21.15.139:0/2531314007 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc158140 con 0x7fbfdc1611e0 2024-04-25T09:53:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.345+0000 7fbfe2313640 1 -- 172.21.15.139:0/2531314007 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfdc074520 con 0x7fbfdc1611e0 2024-04-25T09:53:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.346+0000 7fbfd8ff9640 1 -- 172.21.15.139:0/2531314007 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfd4039040 con 0x7fbfdc1611e0 2024-04-25T09:53:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.346+0000 7fbfd8ff9640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfc806b500 0x7fbfc806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.346+0000 7fbfd8ff9640 1 -- 172.21.15.139:0/2531314007 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbfd40abe30 con 0x7fbfdc1611e0 2024-04-25T09:53:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.346+0000 7fbfdbfff640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfc806b500 0x7fbfc806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.347+0000 7fbfdbfff640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfc806b500 0x7fbfc806d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc000f60 tx=0x7fbfcc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.350+0000 7fbfd8ff9640 1 -- 172.21.15.139:0/2531314007 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfd4043360 con 0x7fbfdc1611e0 2024-04-25T09:53:42.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.554+0000 7fbfe2313640 1 -- 172.21.15.139:0/2531314007 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfdc06ceb0 con 0x7fbfdc1611e0 2024-04-25T09:53:42.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.554+0000 7fbfd8ff9640 1 -- 172.21.15.139:0/2531314007 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fbfd407a970 con 0x7fbfdc1611e0 2024-04-25T09:53:42.557 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:42.557 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/892226922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":892226922},{"type":"v1","addr":"172.21.15.150:6834","nonce":892226922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/582019435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":582019435},{"type":"v1","addr":"172.21.15.139:6837","nonce":582019435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/2751714076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2751714076},{"type":"v1","addr":"172.21.15.139:6834","nonce":2751714076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:15.042067+0000","modified":"2024-04-25T09:53:30.094430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2160557452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2160557452},{"type":"v1","addr":"172.21.15.150:6835","nonce":2160557452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:53:42.557 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 9 2024-04-25T09:53:42.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.556+0000 7fbfe2313640 1 -- 172.21.15.139:0/2531314007 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfc806b500 msgr2=0x7fbfc806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.556+0000 7fbfe2313640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfc806b500 0x7fbfc806d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc000f60 tx=0x7fbfcc009040 comp rx=0 tx=0).stop 2024-04-25T09:53:42.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.556+0000 7fbfe2313640 1 -- 172.21.15.139:0/2531314007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfdc1611e0 msgr2=0x7fbfdc1f2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:42.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.556+0000 7fbfe2313640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfdc1611e0 0x7fbfdc1f2bb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbfd402ac50 tx=0x7fbfd4004a40 comp rx=0 tx=0).stop 2024-04-25T09:53:42.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.556+0000 7fbfe2313640 1 -- 172.21.15.139:0/2531314007 shutdown_connections 2024-04-25T09:53:42.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.556+0000 7fbfe2313640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfdc1611e0 0x7fbfdc1f2bb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.556+0000 7fbfe2313640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfc806b500 0x7fbfc806d9c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.556+0000 7fbfe2313640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfdc0b6af0 0x7fbfdc1f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.556+0000 7fbfe2313640 1 --2- 172.21.15.139:0/2531314007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfdc06c5b0 0x7fbfdc157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:42.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.556+0000 7fbfe2313640 1 -- 172.21.15.139:0/2531314007 >> 172.21.15.139:0/2531314007 conn(0x7fbfdc0b5b00 msgr2=0x7fbfdc0ce7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:42.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.557+0000 7fbfe2313640 1 -- 172.21.15.139:0/2531314007 shutdown_connections 2024-04-25T09:53:42.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:42.557+0000 7fbfe2313640 1 -- 172.21.15.139:0/2531314007 wait complete. 2024-04-25T09:53:42.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:53:15.042067+0000', 'modified': '2024-04-25T09:53:30.094430+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6835/2160557452', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6833', 'nonce': 2160557452}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 2160557452}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:53:42.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:53:42.973 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T09:53:43.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.071+0000 7f26f05d5640 1 Processor -- start 2024-04-25T09:53:43.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.071+0000 7f26f05d5640 1 -- start start 2024-04-25T09:53:43.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26f05d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e80d0a80 0x7f26e81618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26f05d5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e8099ca0 con 0x7f26e80cd610 2024-04-25T09:53:43.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26f05d5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e8099f80 con 0x7f26e809ad40 2024-04-25T09:53:43.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26f05d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e8099a50 con 0x7f26e80d0a80 2024-04-25T09:53:43.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ee371640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f26e80cd610 0x7f26e80d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48684/0 (socket says 172.21.15.139:48684) 2024-04-25T09:53:43.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ee371640 1 -- 172.21.15.139:0/1865987993 learned_addr learned my addr 172.21.15.139:0/1865987993 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:43.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26edb70640 1 --2- 172.21.15.139:0/1865987993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e80d0a80 0x7f26e81618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197339381 0 0) 0x7f26e8099ca0 con 0x7f26e80cd610 2024-04-25T09:53:43.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26d4003520 con 0x7f26e80cd610 2024-04-25T09:53:43.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774407941 0 0) 0x7f26e8099f80 con 0x7f26e809ad40 2024-04-25T09:53:43.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e8099ca0 con 0x7f26e809ad40 2024-04-25T09:53:43.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3863773181 0 0) 0x7f26d4003520 con 0x7f26e80cd610 2024-04-25T09:53:43.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e80d0a80 msgr2=0x7f26e81618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 --2- 172.21.15.139:0/1865987993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e80d0a80 0x7f26e81618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 >> v1:172.21.15.150:6789/0 conn(0x7f26e809ad40 legacy=0x7f26e80cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e809a260 con 0x7f26e80cd610 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f26e4001670 con 0x7f26e80cd610 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 >> v1:172.21.15.139:6789/0 conn(0x7f26e80cd610 legacy=0x7f26e80d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 --2- 172.21.15.139:0/1865987993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d4003b70 0x7f26d4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 --2- 172.21.15.139:0/1865987993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e80d0a80 0x7f26d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 --2- 172.21.15.139:0/1865987993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26d4004d40 0x7f26d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e8099a50 con 0x7f26d4004d40 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d400ba70 con 0x7f26d4003b70 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d400bc70 con 0x7f26e80d0a80 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26eeb72640 1 --2- 172.21.15.139:0/1865987993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26d4004d40 0x7f26d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26ee371640 1 --2- 172.21.15.139:0/1865987993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d4003b70 0x7f26d4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.072+0000 7f26edb70640 1 --2- 172.21.15.139:0/1865987993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e80d0a80 0x7f26d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.073+0000 7f26eeb72640 1 -- 172.21.15.139:0/1865987993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e80d0a80 msgr2=0x7f26d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.073+0000 7f26eeb72640 1 --2- 172.21.15.139:0/1865987993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e80d0a80 0x7f26d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.073+0000 7f26eeb72640 1 -- 172.21.15.139:0/1865987993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d4003b70 msgr2=0x7f26d4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.073+0000 7f26eeb72640 1 --2- 172.21.15.139:0/1865987993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d4003b70 0x7f26d4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.073+0000 7f26eeb72640 1 -- 172.21.15.139:0/1865987993 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d4003520 con 0x7f26d4004d40 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.073+0000 7f26eeb72640 1 --2- 172.21.15.139:0/1865987993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26d4004d40 0x7f26d40051c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f26e0001fd0 tx=0x7f26d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2a3f0b6279cc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.073+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0001bf0 con 0x7f26d4004d40 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.073+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e0003b70 con 0x7f26d4004d40 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.073+0000 7f26ed36f640 1 -- 172.21.15.139:0/1865987993 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e00050d0 con 0x7f26d4004d40 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.074+0000 7f26f05d5640 1 -- 172.21.15.139:0/1865987993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26d4004d40 msgr2=0x7f26d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.074+0000 7f26f05d5640 1 --2- 172.21.15.139:0/1865987993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26d4004d40 0x7f26d40051c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f26e0001fd0 tx=0x7f26d4005830 comp rx=0 tx=0).stop 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.074+0000 7f26eeb72640 1 -- 172.21.15.139:0/1865987993 reap_dead start 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.074+0000 7f26f05d5640 1 -- 172.21.15.139:0/1865987993 shutdown_connections 2024-04-25T09:53:43.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.074+0000 7f26f05d5640 1 -- 172.21.15.139:0/1865987993 >> 172.21.15.139:0/1865987993 conn(0x7f26e80b31f0 msgr2=0x7f26e80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.074+0000 7f26f05d5640 1 -- 172.21.15.139:0/1865987993 shutdown_connections 2024-04-25T09:53:43.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.074+0000 7f26f05d5640 1 -- 172.21.15.139:0/1865987993 wait complete. 2024-04-25T09:53:43.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.075+0000 7f26f05d5640 1 Processor -- start 2024-04-25T09:53:43.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.075+0000 7f26f05d5640 1 -- start start 2024-04-25T09:53:43.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26f05d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26d4004d40 0x7f26e8157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26f05d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e81591e0 0x7f26e80c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26f05d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e814dd80 0x7f26e80c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26f05d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80cd9f0 con 0x7f26e814dd80 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26f05d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e809a820 con 0x7f26e81591e0 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26f05d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80cdb70 con 0x7f26d4004d40 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26eeb72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e814dd80 0x7f26e80c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26eeb72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e814dd80 0x7f26e80c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40546/0 (socket says 172.21.15.139:40546) 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26eeb72640 1 -- 172.21.15.139:0/181148616 learned_addr learned my addr 172.21.15.139:0/181148616 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26ee371640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26d4004d40 0x7f26e8157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26edb70640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e81591e0 0x7f26e80c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26eeb72640 1 -- 172.21.15.139:0/181148616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26d4004d40 msgr2=0x7f26e8157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26eeb72640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26d4004d40 0x7f26e8157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26eeb72640 1 -- 172.21.15.139:0/181148616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e81591e0 msgr2=0x7f26e80c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26eeb72640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e81591e0 0x7f26e80c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.076+0000 7f26eeb72640 1 -- 172.21.15.139:0/181148616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8158140 con 0x7f26e814dd80 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.077+0000 7f26eeb72640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e814dd80 0x7f26e80c70a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f26e002aab0 tx=0x7f26e0001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.077+0000 7f26deffd640 1 -- 172.21.15.139:0/181148616 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0001bf0 con 0x7f26e814dd80 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.077+0000 7f26deffd640 1 -- 172.21.15.139:0/181148616 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e0043070 con 0x7f26e814dd80 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.077+0000 7f26f05d5640 1 -- 172.21.15.139:0/181148616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e8158420 con 0x7f26e814dd80 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.077+0000 7f26deffd640 1 -- 172.21.15.139:0/181148616 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0005610 con 0x7f26e814dd80 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.077+0000 7f26f05d5640 1 -- 172.21.15.139:0/181148616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e81589e0 con 0x7f26e814dd80 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.078+0000 7f26deffd640 1 -- 172.21.15.139:0/181148616 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26e003a040 con 0x7f26e814dd80 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.079+0000 7f26deffd640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26cc06b4d0 0x7f26cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.079+0000 7f26ee371640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26cc06b4d0 0x7f26cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.079+0000 7f26deffd640 1 -- 172.21.15.139:0/181148616 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f26e00aaa50 con 0x7f26e814dd80 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.079+0000 7f26ee371640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26cc06b4d0 0x7f26cc06d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f26e4001050 tx=0x7f26e4000f80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.080+0000 7f26f33b3740 1 Processor -- start 2024-04-25T09:53:43.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.080+0000 7f26f33b3740 1 -- start start 2024-04-25T09:53:43.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26f33b3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd709230 0x5562cd709610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26f33b3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd727020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26f33b3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd6def10 0x5562cd725190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26f33b3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562cd641a10 con 0x5562cd726ad0 2024-04-25T09:53:43.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26f33b3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562cd5eab30 con 0x5562cd709230 2024-04-25T09:53:43.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26f33b3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562cd63dc50 con 0x5562cd6def10 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26edb70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd727020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26edb70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd727020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40550/0 (socket says 172.21.15.139:40550) 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26edb70640 1 -- 172.21.15.139:0/2756389849 learned_addr learned my addr 172.21.15.139:0/2756389849 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26eeb72640 1 --2- 172.21.15.139:0/2756389849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd6def10 0x5562cd725190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26ee371640 1 --2- 172.21.15.139:0/2756389849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd709230 0x5562cd709610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26edb70640 1 -- 172.21.15.139:0/2756389849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd6def10 msgr2=0x5562cd725190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26edb70640 1 --2- 172.21.15.139:0/2756389849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd6def10 0x5562cd725190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26edb70640 1 -- 172.21.15.139:0/2756389849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd709230 msgr2=0x5562cd709610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26edb70640 1 --2- 172.21.15.139:0/2756389849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd709230 0x5562cd709610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.081+0000 7f26edb70640 1 -- 172.21.15.139:0/2756389849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562cd5ffe10 con 0x5562cd726ad0 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.082+0000 7f26edb70640 1 --2- 172.21.15.139:0/2756389849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd727020 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009c50 tx=0x7f26d8014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605194ab16aa2579 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.082+0000 7f26cbfff640 1 -- 172.21.15.139:0/2756389849 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8022070 con 0x5562cd726ad0 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.082+0000 7f26cbfff640 1 -- 172.21.15.139:0/2756389849 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d801d440 con 0x5562cd726ad0 2024-04-25T09:53:43.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.082+0000 7f26cbfff640 1 -- 172.21.15.139:0/2756389849 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d801d810 con 0x5562cd726ad0 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.082+0000 7f26f33b3740 1 -- 172.21.15.139:0/2756389849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 msgr2=0x5562cd727020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.082+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2756389849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd727020 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009c50 tx=0x7f26d8014780 comp rx=0 tx=0).stop 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.082+0000 7f26f33b3740 1 -- 172.21.15.139:0/2756389849 shutdown_connections 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.082+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2756389849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd727020 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.082+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2756389849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd6def10 0x5562cd725190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.082+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2756389849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd709230 0x5562cd709610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.082+0000 7f26f33b3740 1 -- 172.21.15.139:0/2756389849 >> 172.21.15.139:0/2756389849 conn(0x5562cd6db840 msgr2=0x5562cd724040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.082+0000 7f26f33b3740 1 -- 172.21.15.139:0/2756389849 shutdown_connections 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.083+0000 7f26f33b3740 1 -- 172.21.15.139:0/2756389849 wait complete. 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.083+0000 7f26f33b3740 1 -- start start 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.083+0000 7f26f33b3740 1 Processor -- start 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26f33b3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd709230 0x5562cd70b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26f33b3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd70bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26f33b3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 0x5562cd7072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26f33b3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562cd6dfcb0 con 0x5562cd726ad0 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26f33b3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562cd5fe0c0 con 0x5562cd6f2b60 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26f33b3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562cd6d9770 con 0x5562cd709230 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26f33b3740 10 client.0 _ll_register_callbacks cb 0x5562cd5f1ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26edb70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd70bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26edb70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd70bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40558/0 (socket says 172.21.15.139:40558) 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26edb70640 1 -- 172.21.15.139:0/2942674249 learned_addr learned my addr 172.21.15.139:0/2942674249 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26ee371640 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd709230 0x5562cd70b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26eeb72640 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 0x5562cd7072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26edb70640 1 -- 172.21.15.139:0/2942674249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd709230 msgr2=0x5562cd70b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26edb70640 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd709230 0x5562cd70b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26edb70640 1 -- 172.21.15.139:0/2942674249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 msgr2=0x5562cd7072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26edb70640 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 0x5562cd7072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26edb70640 1 -- 172.21.15.139:0/2942674249 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562cd3562d0 con 0x5562cd726ad0 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26edb70640 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd70bb40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f26d8020040 tx=0x7f26d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c24701527f0874 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.084+0000 7f26c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26c97fa640 1 -- 172.21.15.139:0/2942674249 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d802c020 con 0x5562cd726ad0 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26c97fa640 1 -- 172.21.15.139:0/2942674249 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f26d8028420 con 0x5562cd726ad0 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26f33b3740 1 -- 172.21.15.139:0/2942674249 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26d801d440 con 0x5562cd726ad0 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26c97fa640 1 -- 172.21.15.139:0/2942674249 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26d8022040 con 0x5562cd726ad0 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26f33b3740 10 client.4914 fetch_fsmap learned FSMap version 9 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26f33b3740 1 -- 172.21.15.139:0/2942674249 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x5562cd4206b0 con 0x5562cd726ad0 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26c97fa640 1 -- 172.21.15.139:0/2942674249 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d801d440 con 0x5562cd726ad0 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26c97fa640 1 -- 172.21.15.139:0/2942674249 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f26d8025030 con 0x5562cd726ad0 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26f33b3740 10 client.4914 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26f33b3740 10 client.4914 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:53:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26f33b3740 20 client.4914 populate_metadata read hostname 'smithi139' 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x5562cd70a4f0 0x5562cd70a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26f33b3740 4 client.4914 mds_command: new command op to 4299 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26f33b3740 1 -- 172.21.15.139:0/2942674249 --> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562cd6a0920 con 0x5562cd70a4f0 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.085+0000 7f26ee371640 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x5562cd70a4f0 0x5562cd70a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.086+0000 7f26c97fa640 10 client.4914 ms_handle_connect on v2:172.21.15.150:6833/2160557452 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.086+0000 7f26ee371640 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x5562cd70a4f0 0x5562cd70a8d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed8049b84cd27e96 server_cookie=986616eccb404f60 in_seq=0 out_seq=0 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.086+0000 7f26c97fa640 1 -- 172.21.15.139:0/2942674249 <== mds.0 v2:172.21.15.150:6833/2160557452 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5562cd6a0920 con 0x5562cd70a4f0 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.086+0000 7f26c97fa640 10 client.4914 handle_command_reply: tid=0 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.086+0000 7f26f33b3740 1 client.4914 shutdown 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.086+0000 7f26f33b3740 1 -- 172.21.15.139:0/2942674249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 msgr2=0x5562cd70bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.087+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd70bb40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f26d8020040 tx=0x7f26d80046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.087+0000 7f26f33b3740 1 -- 172.21.15.139:0/2942674249 shutdown_connections 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.087+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x5562cd70a4f0 0x5562cd70a8d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.087+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd70bb40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.087+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 0x5562cd7072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.087+0000 7f26c97fa640 0 client.4914 ms_handle_reset on v2:172.21.15.150:6833/2160557452 2024-04-25T09:53:43.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.087+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2942674249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd709230 0x5562cd70b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.087+0000 7f26f33b3740 1 -- 172.21.15.139:0/2942674249 >> 172.21.15.139:0/2942674249 conn(0x5562cd6def10 msgr2=0x5562cd7275c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.087+0000 7f26f33b3740 1 -- 172.21.15.139:0/2942674249 shutdown_connections 2024-04-25T09:53:43.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.087+0000 7f26f33b3740 1 -- 172.21.15.139:0/2942674249 wait complete. 2024-04-25T09:53:43.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.087+0000 7f26f33b3740 20 client.4914 trim_cache size 0 max 16384 2024-04-25T09:53:43.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.102+0000 7f26f33b3740 1 Processor -- start 2024-04-25T09:53:43.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.103+0000 7f26f33b3740 1 -- start start 2024-04-25T09:53:43.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.103+0000 7f26f33b3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 0x5562cd6f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.103+0000 7f26f33b3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd709230 0x5562cd6f1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.103+0000 7f26f33b3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd726f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.103+0000 7f26f33b3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e8200b60 con 0x5562cd726ad0 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.103+0000 7f26f33b3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80b5c70 con 0x5562cd6f2b60 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.103+0000 7f26f33b3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e8200560 con 0x5562cd709230 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.103+0000 7f26eeb72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd726f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.103+0000 7f26eeb72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd726f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40560/0 (socket says 172.21.15.139:40560) 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.103+0000 7f26eeb72640 1 -- 172.21.15.139:0/1019237005 learned_addr learned my addr 172.21.15.139:0/1019237005 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.103+0000 7f26eeb72640 1 -- 172.21.15.139:0/1019237005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd709230 msgr2=0x5562cd6f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26eeb72640 1 --2- 172.21.15.139:0/1019237005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd709230 0x5562cd6f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26eeb72640 1 -- 172.21.15.139:0/1019237005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 msgr2=0x5562cd6f0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26eeb72640 1 --2- 172.21.15.139:0/1019237005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 0x5562cd6f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26eeb72640 1 -- 172.21.15.139:0/1019237005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562cd70d590 con 0x5562cd726ad0 2024-04-25T09:53:43.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26eeb72640 1 --2- 172.21.15.139:0/1019237005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd726f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f26e0081190 tx=0x7f26e0078910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7f96a2b2f7b36c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26cb7fe640 1 -- 172.21.15.139:0/1019237005 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e00a7070 con 0x5562cd726ad0 2024-04-25T09:53:43.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26cb7fe640 1 -- 172.21.15.139:0/1019237005 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e0078fb0 con 0x5562cd726ad0 2024-04-25T09:53:43.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26cb7fe640 1 -- 172.21.15.139:0/1019237005 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0090030 con 0x5562cd726ad0 2024-04-25T09:53:43.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26f33b3740 1 -- 172.21.15.139:0/1019237005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 msgr2=0x5562cd726f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26f33b3740 1 --2- 172.21.15.139:0/1019237005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd726f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f26e0081190 tx=0x7f26e0078910 comp rx=0 tx=0).stop 2024-04-25T09:53:43.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26f33b3740 1 -- 172.21.15.139:0/1019237005 shutdown_connections 2024-04-25T09:53:43.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26f33b3740 1 --2- 172.21.15.139:0/1019237005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd726ad0 0x5562cd726f80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26f33b3740 1 --2- 172.21.15.139:0/1019237005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd709230 0x5562cd6f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26f33b3740 1 --2- 172.21.15.139:0/1019237005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 0x5562cd6f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26f33b3740 1 -- 172.21.15.139:0/1019237005 >> 172.21.15.139:0/1019237005 conn(0x5562cd6def10 msgr2=0x5562cd6df330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26f33b3740 1 -- 172.21.15.139:0/1019237005 shutdown_connections 2024-04-25T09:53:43.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.104+0000 7f26f33b3740 1 -- 172.21.15.139:0/1019237005 wait complete. 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.105+0000 7f26f33b3740 1 -- start start 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.105+0000 7f26f33b3740 1 Processor -- start 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26f33b3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 0x5562cd721760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26f33b3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd709230 0x5562cd70b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26f33b3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd726ad0 0x5562cd704920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26f33b3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562cd6f3f60 con 0x5562cd709230 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26f33b3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562cd70c9d0 con 0x5562cd6f2b60 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26f33b3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562cd6f36a0 con 0x5562cd726ad0 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26f33b3740 10 client.0 _ll_register_callbacks cb 0x5562cd6d9c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26edb70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd709230 0x5562cd70b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26edb70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd709230 0x5562cd70b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40566/0 (socket says 172.21.15.139:40566) 2024-04-25T09:53:43.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26edb70640 1 -- 172.21.15.139:0/2147639290 learned_addr learned my addr 172.21.15.139:0/2147639290 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26eeb72640 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd726ad0 0x5562cd704920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26ee371640 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 0x5562cd721760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26edb70640 1 -- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd726ad0 msgr2=0x5562cd704920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26edb70640 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd726ad0 0x5562cd704920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26edb70640 1 -- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 msgr2=0x5562cd721760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26edb70640 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 0x5562cd721760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26edb70640 1 -- 172.21.15.139:0/2147639290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562cd70aa70 con 0x5562cd709230 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26ee371640 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 0x5562cd721760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26edb70640 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd709230 0x5562cd70b830 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f26d8005c20 tx=0x7f26d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80622ad3587468c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26c97fa640 1 -- 172.21.15.139:0/2147639290 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d802c040 con 0x5562cd709230 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26c97fa640 1 -- 172.21.15.139:0/2147639290 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f26d803ade0 con 0x5562cd709230 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.106+0000 7f26f33b3740 1 -- 172.21.15.139:0/2147639290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26e0078fb0 con 0x5562cd709230 2024-04-25T09:53:43.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.107+0000 7f26c97fa640 1 -- 172.21.15.139:0/2147639290 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26d801d440 con 0x5562cd709230 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.107+0000 7f26f33b3740 10 client.4920 fetch_fsmap learned FSMap version 9 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.107+0000 7f26f33b3740 1 -- 172.21.15.139:0/2147639290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x5562cd70b370 con 0x5562cd709230 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.107+0000 7f26c97fa640 1 -- 172.21.15.139:0/2147639290 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e0078fb0 con 0x5562cd709230 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.107+0000 7f26c97fa640 1 -- 172.21.15.139:0/2147639290 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f26d8038070 con 0x5562cd709230 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.107+0000 7f26f33b3740 10 client.4920 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.107+0000 7f26f33b3740 10 client.4920 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.107+0000 7f26f33b3740 20 client.4920 populate_metadata read hostname 'smithi139' 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.107+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x5562cd704e60 0x5562cd706260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.107+0000 7f26f33b3740 4 client.4920 mds_command: new command op to 4299 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.107+0000 7f26f33b3740 1 -- 172.21.15.139:0/2147639290 --> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5562cd64ef50 con 0x5562cd704e60 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.107+0000 7f26ee371640 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x5562cd704e60 0x5562cd706260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26c97fa640 10 client.4920 ms_handle_connect on v2:172.21.15.150:6833/2160557452 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26ee371640 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x5562cd704e60 0x5562cd706260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d912f7cf745632c server_cookie=8f05c8cd4d6f9227 in_seq=0 out_seq=0 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26c97fa640 1 -- 172.21.15.139:0/2147639290 <== mds.0 v2:172.21.15.150:6833/2160557452 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5562cd64ef50 con 0x5562cd704e60 2024-04-25T09:53:43.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26c97fa640 10 client.4920 handle_command_reply: tid=0 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26f33b3740 1 client.4920 shutdown 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26f33b3740 1 -- 172.21.15.139:0/2147639290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd709230 msgr2=0x5562cd70b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd709230 0x5562cd70b830 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f26d8005c20 tx=0x7f26d8004730 comp rx=0 tx=0).stop 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26f33b3740 1 -- 172.21.15.139:0/2147639290 shutdown_connections 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:6833/2160557452,v1:172.21.15.150:6835/2160557452] conn(0x5562cd704e60 0x5562cd706260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5562cd709230 0x5562cd70b830 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5562cd726ad0 0x5562cd704920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26c97fa640 0 client.4920 ms_handle_reset on v2:172.21.15.150:6833/2160557452 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26f33b3740 1 --2- 172.21.15.139:0/2147639290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562cd6f2b60 0x5562cd721760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26f33b3740 1 -- 172.21.15.139:0/2147639290 >> 172.21.15.139:0/2147639290 conn(0x5562cd6def10 msgr2=0x5562cd6d84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26f33b3740 1 -- 172.21.15.139:0/2147639290 shutdown_connections 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26f33b3740 1 -- 172.21.15.139:0/2147639290 wait complete. 2024-04-25T09:53:43.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.108+0000 7f26f33b3740 20 client.4920 trim_cache size 0 max 16384 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 4299, 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 27.404837053000001, 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 9, 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 24, 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 34.563877503999997 2024-04-25T09:53:43.117 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.109+0000 7f26f05d5640 1 -- 172.21.15.139:0/181148616 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26cc06b4d0 msgr2=0x7f26cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.109+0000 7f26f05d5640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26cc06b4d0 0x7f26cc06d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f26e4001050 tx=0x7f26e4000f80 comp rx=0 tx=0).stop 2024-04-25T09:53:43.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.109+0000 7f26f05d5640 1 -- 172.21.15.139:0/181148616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e814dd80 msgr2=0x7f26e80c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.109+0000 7f26f05d5640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e814dd80 0x7f26e80c70a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f26e002aab0 tx=0x7f26e0001650 comp rx=0 tx=0).stop 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.109+0000 7f26f05d5640 1 -- 172.21.15.139:0/181148616 shutdown_connections 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.109+0000 7f26f05d5640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e814dd80 0x7f26e80c70a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.109+0000 7f26f05d5640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26cc06b4d0 0x7f26cc06d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.109+0000 7f26f05d5640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e81591e0 0x7f26e80c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.109+0000 7f26f05d5640 1 --2- 172.21.15.139:0/181148616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26d4004d40 0x7f26e8157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.109+0000 7f26f05d5640 1 -- 172.21.15.139:0/181148616 >> 172.21.15.139:0/181148616 conn(0x7f26e80b31f0 msgr2=0x7f26e806f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.109+0000 7f26f05d5640 1 -- 172.21.15.139:0/181148616 shutdown_connections 2024-04-25T09:53:43.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.109+0000 7f26f05d5640 1 -- 172.21.15.139:0/181148616 wait complete. 2024-04-25T09:53:43.122 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T09:53:43.138 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T09:53:43.138 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T09:53:43.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:43.139 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:53:43.154 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T09:53:43.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:43.155 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:53:43.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:43.219 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2024-04-25T09:53:43.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:43.323 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:53:43.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:43.338 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2024-04-25T09:53:43.405 INFO:teuthology.orchestra.run.smithi139.stderr:Cannot find device "ceph-brx" 2024-04-25T09:53:43.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:53:43.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:43.406 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-25T09:53:43.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:43.425 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-04-25T09:53:43.496 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2024-04-25T09:53:43.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:53:43.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:43.497 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-25T09:53:43.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:43.556 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-04-25T09:53:43.623 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2024-04-25T09:53:43.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:53:43.625 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-25T09:53:43.625 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:53:43.626 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:53:43.626 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:53:43.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.720+0000 7fdb4c90a640 1 Processor -- start 2024-04-25T09:53:43.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.720+0000 7fdb4c90a640 1 -- start start 2024-04-25T09:53:43.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.721+0000 7fdb4c90a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4815ac40 0x7fdb4806e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.721+0000 7fdb4c90a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb480af6d0 con 0x7fdb4806e850 2024-04-25T09:53:43.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.721+0000 7fdb4c90a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb480af9b0 con 0x7fdb4815a860 2024-04-25T09:53:43.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.721+0000 7fdb4c90a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb480b02e0 con 0x7fdb4815ac40 2024-04-25T09:53:43.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.721+0000 7fdb46d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdb4806e850 0x7fdb480b0770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48690/0 (socket says 172.21.15.139:48690) 2024-04-25T09:53:43.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.721+0000 7fdb46d9d640 1 -- 172.21.15.139:0/3470050666 learned_addr learned my addr 172.21.15.139:0/3470050666 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:43.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.721+0000 7fdb45d9b640 1 --2- 172.21.15.139:0/3470050666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4815ac40 0x7fdb4806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.722+0000 7fdb4559a640 1 -- 172.21.15.139:0/3470050666 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620968284 0 0) 0x7fdb480af6d0 con 0x7fdb4806e850 2024-04-25T09:53:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.722+0000 7fdb4559a640 1 -- 172.21.15.139:0/3470050666 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb30003520 con 0x7fdb4806e850 2024-04-25T09:53:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.722+0000 7fdb4559a640 1 -- 172.21.15.139:0/3470050666 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82257352 0 0) 0x7fdb480af9b0 con 0x7fdb4815a860 2024-04-25T09:53:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.722+0000 7fdb4559a640 1 -- 172.21.15.139:0/3470050666 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb480af6d0 con 0x7fdb4815a860 2024-04-25T09:53:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.722+0000 7fdb45d9b640 1 -- 172.21.15.139:0/3470050666 >> v1:172.21.15.150:6789/0 conn(0x7fdb4815a860 legacy=0x7fdb480cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.722+0000 7fdb45d9b640 1 -- 172.21.15.139:0/3470050666 >> v1:172.21.15.139:6789/0 conn(0x7fdb4806e850 legacy=0x7fdb480b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.722+0000 7fdb45d9b640 1 -- 172.21.15.139:0/3470050666 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb480afc90 con 0x7fdb4815ac40 2024-04-25T09:53:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.723+0000 7fdb45d9b640 1 --2- 172.21.15.139:0/3470050666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4815ac40 0x7fdb4806e310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0097d0 tx=0x7fdb3c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=507f7fbe833ac1bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.723+0000 7fdb4559a640 1 -- 172.21.15.139:0/3470050666 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c037070 con 0x7fdb4815ac40 2024-04-25T09:53:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.723+0000 7fdb4559a640 1 -- 172.21.15.139:0/3470050666 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c032440 con 0x7fdb4815ac40 2024-04-25T09:53:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.723+0000 7fdb4559a640 1 -- 172.21.15.139:0/3470050666 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c032810 con 0x7fdb4815ac40 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.723+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3470050666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4815ac40 msgr2=0x7fdb4806e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.723+0000 7fdb4c90a640 1 --2- 172.21.15.139:0/3470050666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4815ac40 0x7fdb4806e310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0097d0 tx=0x7fdb3c02a210 comp rx=0 tx=0).stop 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.723+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3470050666 shutdown_connections 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.723+0000 7fdb4c90a640 1 --2- 172.21.15.139:0/3470050666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4815ac40 0x7fdb4806e310 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.723+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3470050666 >> 172.21.15.139:0/3470050666 conn(0x7fdb48066f20 msgr2=0x7fdb4806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.723+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3470050666 shutdown_connections 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.724+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3470050666 wait complete. 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.725+0000 7fdb4c90a640 1 Processor -- start 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.725+0000 7fdb4c90a640 1 -- start start 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb4c90a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb4806e850 0x7fdb48157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb4c90a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4815a860 0x7fdb481f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb4c90a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4815ac40 0x7fdb4814f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb4c90a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb480af2a0 con 0x7fdb4815ac40 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb4c90a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb480cd3c0 con 0x7fdb4806e850 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb4c90a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb480af420 con 0x7fdb4815a860 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb46d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4815ac40 0x7fdb4814f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb46d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4815ac40 0x7fdb4814f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40578/0 (socket says 172.21.15.139:40578) 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb46d9d640 1 -- 172.21.15.139:0/3179998352 learned_addr learned my addr 172.21.15.139:0/3179998352 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb45d9b640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4815a860 0x7fdb481f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb4659c640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb4806e850 0x7fdb48157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb46d9d640 1 -- 172.21.15.139:0/3179998352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4815a860 msgr2=0x7fdb481f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb46d9d640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4815a860 0x7fdb481f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb46d9d640 1 -- 172.21.15.139:0/3179998352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb4806e850 msgr2=0x7fdb48157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb46d9d640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb4806e850 0x7fdb48157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb46d9d640 1 -- 172.21.15.139:0/3179998352 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb480ad410 con 0x7fdb4815ac40 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb46d9d640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4815ac40 0x7fdb4814f010 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000c430 tx=0x7fdb4000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb36ffd640 1 -- 172.21.15.139:0/3179998352 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40017070 con 0x7fdb4815ac40 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb36ffd640 1 -- 172.21.15.139:0/3179998352 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb40005d00 con 0x7fdb4815ac40 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3179998352 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb480ad6b0 con 0x7fdb4815ac40 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb36ffd640 1 -- 172.21.15.139:0/3179998352 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40007310 con 0x7fdb4815ac40 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.726+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3179998352 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb480adc70 con 0x7fdb4815ac40 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.727+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3179998352 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb48074520 con 0x7fdb4815ac40 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.728+0000 7fdb36ffd640 1 -- 172.21.15.139:0/3179998352 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb4001a070 con 0x7fdb4815ac40 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.728+0000 7fdb36ffd640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdb2806b4d0 0x7fdb2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.728+0000 7fdb4659c640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdb2806b4d0 0x7fdb2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:43.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.728+0000 7fdb36ffd640 1 -- 172.21.15.139:0/3179998352 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdb40089920 con 0x7fdb4815ac40 2024-04-25T09:53:43.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.729+0000 7fdb4659c640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdb2806b4d0 0x7fdb2806d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdb38001ea0 tx=0x7fdb38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:43.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.733+0000 7fdb36ffd640 1 -- 172.21.15.139:0/3179998352 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb40057e10 con 0x7fdb4815ac40 2024-04-25T09:53:43.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.940+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3179998352 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdb480ae030 con 0x7fdb4815ac40 2024-04-25T09:53:43.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.941+0000 7fdb36ffd640 1 -- 172.21.15.139:0/3179998352 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fdb4005beb0 con 0x7fdb4815ac40 2024-04-25T09:53:43.946 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:53:43.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.942+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3179998352 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdb2806b4d0 msgr2=0x7fdb2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.942+0000 7fdb4c90a640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdb2806b4d0 0x7fdb2806d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdb38001ea0 tx=0x7fdb38009040 comp rx=0 tx=0).stop 2024-04-25T09:53:43.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.942+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3179998352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4815ac40 msgr2=0x7fdb4814f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:43.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.942+0000 7fdb4c90a640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4815ac40 0x7fdb4814f010 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000c430 tx=0x7fdb4000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.942+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3179998352 shutdown_connections 2024-04-25T09:53:43.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.942+0000 7fdb4c90a640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4815ac40 0x7fdb4814f010 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.942+0000 7fdb4c90a640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdb2806b4d0 0x7fdb2806d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.942+0000 7fdb4c90a640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4815a860 0x7fdb481f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.943+0000 7fdb4c90a640 1 --2- 172.21.15.139:0/3179998352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb4806e850 0x7fdb48157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:43.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.943+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3179998352 >> 172.21.15.139:0/3179998352 conn(0x7fdb48066f20 msgr2=0x7fdb480b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:43.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.943+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3179998352 shutdown_connections 2024-04-25T09:53:43.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:43.943+0000 7fdb4c90a640 1 -- 172.21.15.139:0/3179998352 wait complete. 2024-04-25T09:53:43.956 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:53:43.956 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:53:43.956 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T09:53:43.956 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:53:43.956 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:53:43.956 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:53:43.956 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:53:43.956 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:53:43.956 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:53:43.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:43.957 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T09:53:43.971 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:43.971 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:43.971 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:43.971 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:43.972 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:53:43.973 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42674sec preferred_lft 42674sec 2024-04-25T09:53:43.973 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T09:53:43.973 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:43.973 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T09:53:43.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:43.973 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:43.973 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:43.973 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add name ceph-brx type bridge 2024-04-25T09:53:43.974 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr flush dev ceph-brx 2024-04-25T09:53:43.974 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set ceph-brx up 2024-04-25T09:53:43.974 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T09:53:43.974 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:44.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:44.066 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:53:44.087 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:53:44.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:44.089 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T09:53:44.144 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T09:53:44.144 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:53:44.144 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:53:44.144 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:53:44.144 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:53:44.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:44.145 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:44.145 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:44.145 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:53:44.145 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:53:44.145 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:53:44.145 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:44.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:44.229 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:53:44.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:44.285 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list-id 2024-04-25T09:53:44.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:44.340 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:44.340 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:44.341 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:53:44.341 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T09:53:44.341 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:44.418 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T09:53:44.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:44.418 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:44.419 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:44.419 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T09:53:44.419 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T09:53:44.419 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T09:53:44.419 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T09:53:44.419 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T09:53:44.419 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:44.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:44.525 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:44.526 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:44.526 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T09:53:44.526 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T09:53:44.526 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:44.561 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:44.561 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:53:44.617 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:53:44.617 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:44.617 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:53:44.672 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:53:44.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:44.673 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:44.754 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:44.754 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:44.875 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:44.875 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:44.875 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T09:53:44.875 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T09:53:44.875 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T09:53:44.875 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:53:44.876 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:53:44.876 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:53:44.876 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:53:44.876 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:53:44.877 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:53:44.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:44.878 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:53:44.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:44.941 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:44.999 INFO:teuthology.orchestra.run.smithi139.stdout:client4929 2024-04-25T09:53:44.999 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:53:44.999 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:45.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:45.054 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:45.110 INFO:teuthology.orchestra.run.smithi139.stdout:client4929 2024-04-25T09:53:45.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:45.110 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:45.124 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:45.153 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:client4929 2024-04-25T09:53:45.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:45.172 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/922739a8-edcf-4ded-8206-dbdb23e9fea4.client4929/status') 2024-04-25T09:53:45.274 INFO:teuthology.orchestra.run.smithi139.stdout:instance: client.4929 (3)192.168.0.1:0/4146659583 2024-04-25T09:53:45.274 INFO:teuthology.orchestra.run.smithi139.stdout:blocklisted: false 2024-04-25T09:53:45.274 INFO:teuthology.orchestra.run.smithi139.stderr:0+1 records in 2024-04-25T09:53:45.274 INFO:teuthology.orchestra.run.smithi139.stderr:0+1 records out 2024-04-25T09:53:45.274 INFO:teuthology.orchestra.run.smithi139.stderr:69 bytes copied, 2.3367e-05 s, 3.0 MB/s 2024-04-25T09:53:45.275 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-25T09:53:45.275 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:53:45.276 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T09:53:45.276 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:53:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.413+0000 7f82cc2b5640 1 Processor -- start 2024-04-25T09:53:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.413+0000 7f82cc2b5640 1 -- start start 2024-04-25T09:53:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.414+0000 7f82cc2b5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4159480 0x7f82c40af160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.414+0000 7f82cc2b5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f82c40ad740 0x7f82c406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.414+0000 7f82cc2b5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c4159860 con 0x7f82c415fe50 2024-04-25T09:53:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.414+0000 7f82cc2b5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c41603c0 con 0x7f82c40ad740 2024-04-25T09:53:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.414+0000 7f82cc2b5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c415cdb0 con 0x7f82c4159480 2024-04-25T09:53:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.414+0000 7f82c9850640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f82c415fe50 0x7f82c40ad030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48704/0 (socket says 172.21.15.139:48704) 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.414+0000 7f82c9850640 1 -- 172.21.15.139:0/3402292206 learned_addr learned my addr 172.21.15.139:0/3402292206 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.414+0000 7f82ca852640 1 --2- 172.21.15.139:0/3402292206 >> v2:172.21.15.150:3300/0 conn(0x7f82c40ad740 0x7f82c406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.414+0000 7f82ca051640 1 --2- 172.21.15.139:0/3402292206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4159480 0x7f82c40af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.414+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204145667 0 0) 0x7f82c4159860 con 0x7f82c415fe50 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.414+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82b0003520 con 0x7f82c415fe50 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2675005893 0 0) 0x7f82b0003520 con 0x7f82c415fe50 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4159480 msgr2=0x7f82c40af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 --2- 172.21.15.139:0/3402292206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4159480 0x7f82c40af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 >> v2:172.21.15.150:3300/0 conn(0x7f82c40ad740 msgr2=0x7f82c406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 --2- 172.21.15.139:0/3402292206 >> v2:172.21.15.150:3300/0 conn(0x7f82c40ad740 0x7f82c406ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c415bff0 con 0x7f82c415fe50 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f82b4001670 con 0x7f82c415fe50 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 >> v1:172.21.15.139:6789/0 conn(0x7f82c415fe50 legacy=0x7f82c40ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 --2- 172.21.15.139:0/3402292206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b00039b0 0x7f82b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 --2- 172.21.15.139:0/3402292206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0004800 0x7f82b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 --2- 172.21.15.139:0/3402292206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4159480 0x7f82b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c41603c0 con 0x7f82b00039b0 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c415cdb0 con 0x7f82b0004800 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b000b9c0 con 0x7f82c4159480 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82c9850640 1 --2- 172.21.15.139:0/3402292206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0004800 0x7f82b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82ca051640 1 --2- 172.21.15.139:0/3402292206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b00039b0 0x7f82b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82ca852640 1 --2- 172.21.15.139:0/3402292206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4159480 0x7f82b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82ca051640 1 -- 172.21.15.139:0/3402292206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4159480 msgr2=0x7f82b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82ca051640 1 --2- 172.21.15.139:0/3402292206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4159480 0x7f82b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82ca051640 1 -- 172.21.15.139:0/3402292206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0004800 msgr2=0x7f82b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82ca051640 1 --2- 172.21.15.139:0/3402292206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0004800 0x7f82b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.415+0000 7f82ca051640 1 -- 172.21.15.139:0/3402292206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b0003520 con 0x7f82b00039b0 2024-04-25T09:53:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.416+0000 7f82ca051640 1 --2- 172.21.15.139:0/3402292206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b00039b0 0x7f82b0003dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009ac0 tx=0x7f82b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d50f9b9e95d0d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.416+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc036070 con 0x7f82b00039b0 2024-04-25T09:53:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.416+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc031400 con 0x7f82b00039b0 2024-04-25T09:53:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.416+0000 7f82c904f640 1 -- 172.21.15.139:0/3402292206 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc009690 con 0x7f82b00039b0 2024-04-25T09:53:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.417+0000 7f82cc2b5640 1 -- 172.21.15.139:0/3402292206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b00039b0 msgr2=0x7f82b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.417+0000 7f82cc2b5640 1 --2- 172.21.15.139:0/3402292206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b00039b0 0x7f82b0003dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009ac0 tx=0x7f82b0005830 comp rx=0 tx=0).stop 2024-04-25T09:53:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.417+0000 7f82ca852640 1 -- 172.21.15.139:0/3402292206 reap_dead start 2024-04-25T09:53:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.417+0000 7f82cc2b5640 1 -- 172.21.15.139:0/3402292206 shutdown_connections 2024-04-25T09:53:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.417+0000 7f82cc2b5640 1 -- 172.21.15.139:0/3402292206 >> 172.21.15.139:0/3402292206 conn(0x7f82c40b23d0 msgr2=0x7f82c40cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.417+0000 7f82cc2b5640 1 -- 172.21.15.139:0/3402292206 shutdown_connections 2024-04-25T09:53:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.417+0000 7f82cc2b5640 1 -- 172.21.15.139:0/3402292206 wait complete. 2024-04-25T09:53:45.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.418+0000 7f82cc2b5640 1 Processor -- start 2024-04-25T09:53:45.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.418+0000 7f82cc2b5640 1 -- start start 2024-04-25T09:53:45.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82cc2b5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b00039b0 0x7f82c4154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82cc2b5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4099a50 0x7f82c409b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82cc2b5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82c40b0710 0x7f82c40b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82cc2b5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c415c890 con 0x7f82b00039b0 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82cc2b5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c415c5b0 con 0x7f82c40b0710 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82cc2b5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c406f6f0 con 0x7f82c4099a50 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82ca051640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b00039b0 0x7f82c4154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82ca051640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b00039b0 0x7f82c4154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40598/0 (socket says 172.21.15.139:40598) 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82ca051640 1 -- 172.21.15.139:0/1060554295 learned_addr learned my addr 172.21.15.139:0/1060554295 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82c9850640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4099a50 0x7f82c409b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82ca852640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82c40b0710 0x7f82c40b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82ca051640 1 -- 172.21.15.139:0/1060554295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4099a50 msgr2=0x7f82c409b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82ca051640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4099a50 0x7f82c409b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82ca051640 1 -- 172.21.15.139:0/1060554295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82c40b0710 msgr2=0x7f82c40b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82ca051640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82c40b0710 0x7f82c40b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.419+0000 7f82ca051640 1 -- 172.21.15.139:0/1060554295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c4099fc0 con 0x7f82b00039b0 2024-04-25T09:53:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.420+0000 7f82ca051640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b00039b0 0x7f82c4154780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009ac0 tx=0x7f82bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:45.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.420+0000 7f82baffd640 1 -- 172.21.15.139:0/1060554295 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc040020 con 0x7f82b00039b0 2024-04-25T09:53:45.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.420+0000 7f82baffd640 1 -- 172.21.15.139:0/1060554295 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc003bc0 con 0x7f82b00039b0 2024-04-25T09:53:45.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.420+0000 7f82cc2b5640 1 -- 172.21.15.139:0/1060554295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c409a2a0 con 0x7f82b00039b0 2024-04-25T09:53:45.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.420+0000 7f82baffd640 1 -- 172.21.15.139:0/1060554295 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc036040 con 0x7f82b00039b0 2024-04-25T09:53:45.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.420+0000 7f82cc2b5640 1 -- 172.21.15.139:0/1060554295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c409a860 con 0x7f82b00039b0 2024-04-25T09:53:45.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.421+0000 7f82cc2b5640 1 -- 172.21.15.139:0/1060554295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c4074520 con 0x7f82b00039b0 2024-04-25T09:53:45.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.421+0000 7f82baffd640 1 -- 172.21.15.139:0/1060554295 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82bc043020 con 0x7f82b00039b0 2024-04-25T09:53:45.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.422+0000 7f82baffd640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f82a806b4d0 0x7f82a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:45.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.422+0000 7f82baffd640 1 -- 172.21.15.139:0/1060554295 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f82bc0aa4a0 con 0x7f82b00039b0 2024-04-25T09:53:45.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.422+0000 7f82c9850640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f82a806b4d0 0x7f82a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:45.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.422+0000 7f82c9850640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f82a806b4d0 0x7f82a806d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82c409c8f0 tx=0x7f82b40078f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:45.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.430+0000 7f82baffd640 1 -- 172.21.15.139:0/1060554295 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82bc04a070 con 0x7f82b00039b0 2024-04-25T09:53:45.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.643+0000 7f82cc2b5640 1 -- 172.21.15.139:0/1060554295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f82c40d09c0 con 0x7f82b00039b0 2024-04-25T09:53:45.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.643+0000 7f82baffd640 1 -- 172.21.15.139:0/1060554295 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f82bc0044e0 con 0x7f82b00039b0 2024-04-25T09:53:45.645 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:53:45.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.645+0000 7f82cc2b5640 1 -- 172.21.15.139:0/1060554295 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f82a806b4d0 msgr2=0x7f82a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.645+0000 7f82cc2b5640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f82a806b4d0 0x7f82a806d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82c409c8f0 tx=0x7f82b40078f0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.645+0000 7f82cc2b5640 1 -- 172.21.15.139:0/1060554295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b00039b0 msgr2=0x7f82c4154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:45.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.645+0000 7f82cc2b5640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b00039b0 0x7f82c4154780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009ac0 tx=0x7f82bc004730 comp rx=0 tx=0).stop 2024-04-25T09:53:45.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.645+0000 7f82cc2b5640 1 -- 172.21.15.139:0/1060554295 shutdown_connections 2024-04-25T09:53:45.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.645+0000 7f82cc2b5640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f82a806b4d0 0x7f82a806d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.645+0000 7f82cc2b5640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82c40b0710 0x7f82c40b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.645+0000 7f82cc2b5640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82c4099a50 0x7f82c409b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.645+0000 7f82cc2b5640 1 --2- 172.21.15.139:0/1060554295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82b00039b0 0x7f82c4154780 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:45.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.645+0000 7f82cc2b5640 1 -- 172.21.15.139:0/1060554295 >> 172.21.15.139:0/1060554295 conn(0x7f82c40b23d0 msgr2=0x7f82c4157710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:45.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.645+0000 7f82cc2b5640 1 -- 172.21.15.139:0/1060554295 shutdown_connections 2024-04-25T09:53:45.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:45.645+0000 7f82cc2b5640 1 -- 172.21.15.139:0/1060554295 wait complete. 2024-04-25T09:53:45.659 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:53:45.659 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:53:45.659 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T09:53:45.659 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T09:53:45.659 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T09:53:45.659 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:53:45.660 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:53:45.660 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:53:45.660 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T09:53:45.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:45.660 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T09:53:45.677 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:45.677 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:45.677 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:45.677 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:45.677 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:45.677 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:45.677 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:45.677 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42673sec preferred_lft 42673sec 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:53:45.678 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:45.679 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:53:45.679 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:45.679 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link tentative 2024-04-25T09:53:45.679 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:45.679 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:53:45.679 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:53:45.679 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T09:53:45.679 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:45.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:45.679 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:53:45.740 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:53:45.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:45.741 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T09:53:45.799 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T09:53:45.799 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:53:45.799 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:53:45.799 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:53:45.799 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:53:45.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:45.800 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:45.800 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:45.800 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:53:45.800 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:53:45.800 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:53:45.800 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:45.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:45.884 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:53:45.940 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:53:45.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:45.941 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list-id 2024-04-25T09:53:45.996 INFO:teuthology.orchestra.run.smithi139.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T09:53:45.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:45.996 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:45.996 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:45.996 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:53:45.997 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T09:53:45.997 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:46.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:46.072 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T09:53:46.135 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:46.136 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:46.136 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:46.136 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:46.136 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:46.136 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:46.136 INFO:teuthology.orchestra.run.smithi139.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:53:46.137 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 16:96:55:d7:f6:06 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:53:46.137 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:53:46.137 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:46.137 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::1496:55ff:fed7:f606/64 scope link tentative 2024-04-25T09:53:46.137 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:46.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:46.138 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T09:53:46.201 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:46.201 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:46.201 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:46.201 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:46.202 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:46.202 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:46.202 INFO:teuthology.orchestra.run.smithi139.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:53:46.202 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 16:96:55:d7:f6:06 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:53:46.202 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:53:46.202 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:46.202 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::1496:55ff:fed7:f606/64 scope link tentative 2024-04-25T09:53:46.202 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:46.204 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T09:53:46.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:46.204 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:46.204 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:46.204 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T09:53:46.205 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T09:53:46.205 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T09:53:46.205 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T09:53:46.205 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T09:53:46.205 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:46.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:46.314 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:53:46.314 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:53:46.314 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.1 up 2024-04-25T09:53:46.314 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T09:53:46.314 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:53:46.349 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:46.350 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T09:53:46.403 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:46.403 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T09:53:46.458 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-25T09:53:46.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:46.458 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:46.539 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:46.539 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:46.546 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:46.546 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:46.546 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T09:53:46.547 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 2@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T09:53:46.547 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T09:53:46.547 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T09:53:46.547 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:53:46.547 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:53:46.547 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:53:46.547 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:53:46.549 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:53:46.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:46.549 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T09:53:46.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:46.613 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:46.669 INFO:teuthology.orchestra.run.smithi139.stdout:client4643 2024-04-25T09:53:46.670 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:53:46.670 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:46.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:46.725 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:46.782 INFO:teuthology.orchestra.run.smithi139.stdout:client4643 2024-04-25T09:53:46.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:46.782 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:46.839 INFO:teuthology.orchestra.run.smithi139.stdout:client4643 2024-04-25T09:53:46.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:46.839 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/922739a8-edcf-4ded-8206-dbdb23e9fea4.client4643/status') 2024-04-25T09:53:46.902 INFO:teuthology.orchestra.run.smithi139.stdout:instance: client.4643 (3)172.21.15.139:0/2264244005 2024-04-25T09:53:46.902 INFO:teuthology.orchestra.run.smithi139.stdout:blocklisted: false 2024-04-25T09:53:46.902 INFO:teuthology.orchestra.run.smithi139.stderr:0+1 records in 2024-04-25T09:53:46.903 INFO:teuthology.orchestra.run.smithi139.stderr:0+1 records out 2024-04-25T09:53:46.903 INFO:teuthology.orchestra.run.smithi139.stderr:71 bytes copied, 2.5228e-05 s, 2.8 MB/s 2024-04-25T09:53:46.903 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-25T09:53:46.904 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:53:46.904 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T09:53:46.904 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:53:47.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.039+0000 7fb2a5ff4640 1 Processor -- start 2024-04-25T09:53:47.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.039+0000 7fb2a5ff4640 1 -- start start 2024-04-25T09:53:47.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb2a5ff4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a015ac40 0x7fb2a006e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb2a5ff4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb2a006e850 0x7fb2a00b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb2a5ff4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a00af6d0 con 0x7fb2a015a860 2024-04-25T09:53:47.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb2a5ff4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a00b02e0 con 0x7fb2a006e850 2024-04-25T09:53:47.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb2a5ff4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00af9b0 con 0x7fb2a015ac40 2024-04-25T09:53:47.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb297fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a015ac40 0x7fb2a006e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb29ffff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb2a006e850 0x7fb2a00b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb297fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a015ac40 0x7fb2a006e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:51064/0 (socket says 172.21.15.139:51064) 2024-04-25T09:53:47.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb297fff640 1 -- 172.21.15.139:0/1731169686 learned_addr learned my addr 172.21.15.139:0/1731169686 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:47.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb297fff640 1 -- 172.21.15.139:0/1731169686 >> v2:172.21.15.150:3300/0 conn(0x7fb2a006e850 msgr2=0x7fb2a00b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:47.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb297fff640 1 --2- 172.21.15.139:0/1731169686 >> v2:172.21.15.150:3300/0 conn(0x7fb2a006e850 0x7fb2a00b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb297fff640 1 -- 172.21.15.139:0/1731169686 >> v1:172.21.15.150:6789/0 conn(0x7fb2a015a860 legacy=0x7fb2a00cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.040+0000 7fb297fff640 1 -- 172.21.15.139:0/1731169686 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00afc90 con 0x7fb2a015ac40 2024-04-25T09:53:47.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.041+0000 7fb297fff640 1 --2- 172.21.15.139:0/1731169686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a015ac40 0x7fb2a006e310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb288009870 tx=0x7fb28802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be228a5722c99404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:47.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.041+0000 7fb29effd640 1 -- 172.21.15.139:0/1731169686 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288037070 con 0x7fb2a015ac40 2024-04-25T09:53:47.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.041+0000 7fb29effd640 1 -- 172.21.15.139:0/1731169686 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb288032440 con 0x7fb2a015ac40 2024-04-25T09:53:47.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.041+0000 7fb29effd640 1 -- 172.21.15.139:0/1731169686 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288032810 con 0x7fb2a015ac40 2024-04-25T09:53:47.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.042+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/1731169686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a015ac40 msgr2=0x7fb2a006e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.042+0000 7fb2a5ff4640 1 --2- 172.21.15.139:0/1731169686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a015ac40 0x7fb2a006e310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb288009870 tx=0x7fb28802a270 comp rx=0 tx=0).stop 2024-04-25T09:53:47.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.042+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/1731169686 shutdown_connections 2024-04-25T09:53:47.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.042+0000 7fb2a5ff4640 1 --2- 172.21.15.139:0/1731169686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a015ac40 0x7fb2a006e310 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.042+0000 7fb2a5ff4640 1 --2- 172.21.15.139:0/1731169686 >> v2:172.21.15.150:3300/0 conn(0x7fb2a006e850 0x7fb2a00b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.042+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/1731169686 >> 172.21.15.139:0/1731169686 conn(0x7fb2a0066f20 msgr2=0x7fb2a006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:47.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.042+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/1731169686 shutdown_connections 2024-04-25T09:53:47.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.042+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/1731169686 wait complete. 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.043+0000 7fb2a5ff4640 1 Processor -- start 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.043+0000 7fb2a5ff4640 1 -- start start 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb2a5ff4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a006e850 0x7fb2a00a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb2a5ff4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a015a860 0x7fb2a01f0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb2a5ff4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2a015ac40 0x7fb2a0158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb2a5ff4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00af2a0 con 0x7fb2a015a860 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb2a5ff4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00cd3c0 con 0x7fb2a015ac40 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb2a5ff4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00af420 con 0x7fb2a006e850 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb297fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a015a860 0x7fb2a01f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb297fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a015a860 0x7fb2a01f0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54970/0 (socket says 172.21.15.139:54970) 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb297fff640 1 -- 172.21.15.139:0/209355043 learned_addr learned my addr 172.21.15.139:0/209355043 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb29f7fe640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a006e850 0x7fb2a00a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb29ffff640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2a015ac40 0x7fb2a0158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb297fff640 1 -- 172.21.15.139:0/209355043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a006e850 msgr2=0x7fb2a00a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb297fff640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a006e850 0x7fb2a00a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb297fff640 1 -- 172.21.15.139:0/209355043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2a015ac40 msgr2=0x7fb2a0158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb297fff640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2a015ac40 0x7fb2a0158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb297fff640 1 -- 172.21.15.139:0/209355043 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00a8f30 con 0x7fb2a015a860 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb297fff640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a015a860 0x7fb2a01f0090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb288000c00 tx=0x7fb2880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb29cff9640 1 -- 172.21.15.139:0/209355043 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288037040 con 0x7fb2a015a860 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb29cff9640 1 -- 172.21.15.139:0/209355043 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb288042070 con 0x7fb2a015a860 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.044+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/209355043 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a01580b0 con 0x7fb2a015a860 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.045+0000 7fb29cff9640 1 -- 172.21.15.139:0/209355043 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288030860 con 0x7fb2a015a860 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.045+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/209355043 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a0158670 con 0x7fb2a015a860 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.046+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/209355043 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a0074520 con 0x7fb2a015a860 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.046+0000 7fb29cff9640 1 -- 172.21.15.139:0/209355043 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb288030a00 con 0x7fb2a015a860 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.046+0000 7fb29cff9640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb28006b4d0 0x7fb28006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.046+0000 7fb29cff9640 1 -- 172.21.15.139:0/209355043 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb2880aaca0 con 0x7fb2a015a860 2024-04-25T09:53:47.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.046+0000 7fb29f7fe640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb28006b4d0 0x7fb28006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:47.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.047+0000 7fb29f7fe640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb28006b4d0 0x7fb28006d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb2a01f2420 tx=0x7fb290009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:47.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.051+0000 7fb29cff9640 1 -- 172.21.15.139:0/209355043 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb288079190 con 0x7fb2a015a860 2024-04-25T09:53:47.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.262+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/209355043 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb2a015a080 con 0x7fb2a015a860 2024-04-25T09:53:47.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.263+0000 7fb29cff9640 1 -- 172.21.15.139:0/209355043 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb28807d230 con 0x7fb2a015a860 2024-04-25T09:53:47.265 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:53:47.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.265+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/209355043 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb28006b4d0 msgr2=0x7fb28006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.265+0000 7fb2a5ff4640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb28006b4d0 0x7fb28006d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb2a01f2420 tx=0x7fb290009040 comp rx=0 tx=0).stop 2024-04-25T09:53:47.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.265+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/209355043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a015a860 msgr2=0x7fb2a01f0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:47.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.265+0000 7fb2a5ff4640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a015a860 0x7fb2a01f0090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb288000c00 tx=0x7fb2880046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.265+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/209355043 shutdown_connections 2024-04-25T09:53:47.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.265+0000 7fb2a5ff4640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2a015a860 0x7fb2a01f0090 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.265+0000 7fb2a5ff4640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb28006b4d0 0x7fb28006d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.265+0000 7fb2a5ff4640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2a015ac40 0x7fb2a0158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.265+0000 7fb2a5ff4640 1 --2- 172.21.15.139:0/209355043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a006e850 0x7fb2a00a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:47.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.265+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/209355043 >> 172.21.15.139:0/209355043 conn(0x7fb2a0066f20 msgr2=0x7fb2a00b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:47.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.265+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/209355043 shutdown_connections 2024-04-25T09:53:47.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:47.265+0000 7fb2a5ff4640 1 -- 172.21.15.139:0/209355043 wait complete. 2024-04-25T09:53:47.278 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:53:47.278 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:53:47.278 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-04-25T09:53:47.278 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T09:53:47.278 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T09:53:47.278 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:53:47.278 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:53:47.278 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:53:47.278 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T09:53:47.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:47.279 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-04-25T09:53:47.297 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:47.298 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:47.298 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:47.298 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:47.298 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:47.298 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:47.298 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:47.298 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:47.298 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-04-25T09:53:47.298 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:47.298 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:47.298 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-04-25T09:53:47.299 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:47.299 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:47.299 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:53:47.299 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:47.299 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:53:47.299 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42691sec preferred_lft 42691sec 2024-04-25T09:53:47.299 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::74f2:a893:48f2:5618/64 scope link noprefixroute 2024-04-25T09:53:47.299 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:47.299 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T09:53:47.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:47.300 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-04-25T09:53:47.300 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-25T09:53:47.300 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add name ceph-brx type bridge 2024-04-25T09:53:47.300 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr flush dev ceph-brx 2024-04-25T09:53:47.300 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set ceph-brx up 2024-04-25T09:53:47.300 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T09:53:47.300 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-25T09:53:47.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:47.507 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:53:47.530 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-04-25T09:53:47.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:47.532 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-04-25T09:53:47.601 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-04-25T09:53:47.601 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:53:47.601 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:53:47.601 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:53:47.601 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:53:47.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:47.601 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-04-25T09:53:47.601 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-25T09:53:47.601 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:53:47.602 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:53:47.602 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:53:47.602 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-25T09:53:47.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:47.823 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-25T09:53:47.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:47.838 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2024-04-25T09:53:47.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:47.893 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-04-25T09:53:47.893 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-25T09:53:47.893 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T09:53:47.893 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T09:53:47.893 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-25T09:53:47.970 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T09:53:47.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:47.970 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-04-25T09:53:47.970 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-25T09:53:47.970 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T09:53:47.970 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T09:53:47.971 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T09:53:47.971 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T09:53:47.971 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T09:53:47.971 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-25T09:53:48.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:48.093 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-04-25T09:53:48.093 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-25T09:53:48.093 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.0 up 2024-04-25T09:53:48.093 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T09:53:48.093 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-25T09:53:48.128 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:48.128 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T09:53:48.183 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T09:53:48.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:48.183 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T09:53:48.238 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T09:53:48.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:48.239 DEBUG:teuthology.orchestra.run.smithi150:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:48.331 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:48.331 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:48.439 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:48.439 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:48.439 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T09:53:48.439 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with new device syntax: 1@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T09:53:48.439 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T09:53:48.441 INFO:teuthology.orchestra.run.smithi150.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T09:53:48.441 INFO:teuthology.orchestra.run.smithi150.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:53:48.441 INFO:teuthology.orchestra.run.smithi150.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:53:48.441 INFO:teuthology.orchestra.run.smithi150.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:53:48.441 INFO:teuthology.orchestra.run.smithi150.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:53:48.443 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:53:48.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:48.443 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T09:53:48.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:48.466 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:48.521 INFO:teuthology.orchestra.run.smithi150.stdout:client4652 2024-04-25T09:53:48.522 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:53:48.522 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:48.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:48.577 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:48.632 INFO:teuthology.orchestra.run.smithi150.stdout:client4652 2024-04-25T09:53:48.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:48.633 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:48.688 INFO:teuthology.orchestra.run.smithi150.stdout:client4652 2024-04-25T09:53:48.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:48.688 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/922739a8-edcf-4ded-8206-dbdb23e9fea4.client4652/status') 2024-04-25T09:53:48.751 INFO:teuthology.orchestra.run.smithi150.stdout:instance: client.4652 (3)192.168.0.1:0/3586565765 2024-04-25T09:53:48.751 INFO:teuthology.orchestra.run.smithi150.stdout:blocklisted: false 2024-04-25T09:53:48.751 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records in 2024-04-25T09:53:48.751 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records out 2024-04-25T09:53:48.751 INFO:teuthology.orchestra.run.smithi150.stderr:69 bytes copied, 2.3505e-05 s, 2.9 MB/s 2024-04-25T09:53:48.752 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-25T09:53:48.752 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:53:48.752 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T09:53:48.752 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:53:48.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.860+0000 7fae61df4640 1 Processor -- start 2024-04-25T09:53:48.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.860+0000 7fae61df4640 1 -- start start 2024-04-25T09:53:48.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.861+0000 7fae61df4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fae5c1611e0 0x7fae5c0b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.861+0000 7fae61df4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae5c06c780 0x7fae5c0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.861+0000 7fae61df4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5c0d4740 con 0x7fae5c0cea00 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.861+0000 7fae61df4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5c0d1fa0 con 0x7fae5c06c780 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.861+0000 7fae61df4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5c0d2560 con 0x7fae5c1611e0 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.861+0000 7fae5bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae5c06c780 0x7fae5c0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.861+0000 7fae5bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae5c06c780 0x7fae5c0d29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54982/0 (socket says 172.21.15.139:54982) 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.861+0000 7fae5bfff640 1 -- 172.21.15.139:0/3329283510 learned_addr learned my addr 172.21.15.139:0/3329283510 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.861+0000 7fae5b7fe640 1 --2- 172.21.15.139:0/3329283510 >> v2:172.21.15.150:3300/0 conn(0x7fae5c1611e0 0x7fae5c0b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.862+0000 7fae5bfff640 1 -- 172.21.15.139:0/3329283510 >> v2:172.21.15.150:3300/0 conn(0x7fae5c1611e0 msgr2=0x7fae5c0b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.862+0000 7fae5bfff640 1 --2- 172.21.15.139:0/3329283510 >> v2:172.21.15.150:3300/0 conn(0x7fae5c1611e0 0x7fae5c0b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.862+0000 7fae5bfff640 1 -- 172.21.15.139:0/3329283510 >> v1:172.21.15.150:6789/0 conn(0x7fae5c0cea00 legacy=0x7fae5c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.862+0000 7fae5bfff640 1 -- 172.21.15.139:0/3329283510 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0b2770 con 0x7fae5c06c780 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.862+0000 7fae5bfff640 1 --2- 172.21.15.139:0/3329283510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae5c06c780 0x7fae5c0d29f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fae5c08c420 tx=0x7fae5402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d71e192c962c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.862+0000 7fae5affd640 1 -- 172.21.15.139:0/3329283510 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54036070 con 0x7fae5c06c780 2024-04-25T09:53:48.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.862+0000 7fae5affd640 1 -- 172.21.15.139:0/3329283510 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae54031600 con 0x7fae5c06c780 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.862+0000 7fae5affd640 1 -- 172.21.15.139:0/3329283510 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae540319d0 con 0x7fae5c06c780 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.863+0000 7fae61df4640 1 -- 172.21.15.139:0/3329283510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae5c06c780 msgr2=0x7fae5c0d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.863+0000 7fae61df4640 1 --2- 172.21.15.139:0/3329283510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae5c06c780 0x7fae5c0d29f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fae5c08c420 tx=0x7fae5402a730 comp rx=0 tx=0).stop 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.863+0000 7fae61df4640 1 -- 172.21.15.139:0/3329283510 shutdown_connections 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.863+0000 7fae61df4640 1 --2- 172.21.15.139:0/3329283510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae5c06c780 0x7fae5c0d29f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.863+0000 7fae61df4640 1 --2- 172.21.15.139:0/3329283510 >> v2:172.21.15.150:3300/0 conn(0x7fae5c1611e0 0x7fae5c0b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.863+0000 7fae61df4640 1 -- 172.21.15.139:0/3329283510 >> 172.21.15.139:0/3329283510 conn(0x7fae5c0b5b00 msgr2=0x7fae5c0d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.863+0000 7fae61df4640 1 -- 172.21.15.139:0/3329283510 shutdown_connections 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.863+0000 7fae61df4640 1 -- 172.21.15.139:0/3329283510 wait complete. 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.864+0000 7fae61df4640 1 Processor -- start 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.864+0000 7fae61df4640 1 -- start start 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae61df4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae5c06c780 0x7fae5c0cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae61df4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae5c0cea00 0x7fae5c15a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae61df4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae5c1611e0 0x7fae5c15aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae61df4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0d21f0 con 0x7fae5c06c780 2024-04-25T09:53:48.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae61df4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0b2d30 con 0x7fae5c0cea00 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae61df4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c1615c0 con 0x7fae5c1611e0 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae5b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae5c06c780 0x7fae5c0cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae5b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae5c06c780 0x7fae5c0cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54988/0 (socket says 172.21.15.139:54988) 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae5b7fe640 1 -- 172.21.15.139:0/2116012625 learned_addr learned my addr 172.21.15.139:0/2116012625 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae5bfff640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae5c1611e0 0x7fae5c15aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae5b7fe640 1 -- 172.21.15.139:0/2116012625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae5c1611e0 msgr2=0x7fae5c15aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae53fff640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae5c0cea00 0x7fae5c15a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae5b7fe640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae5c1611e0 0x7fae5c15aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae5b7fe640 1 -- 172.21.15.139:0/2116012625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae5c0cea00 msgr2=0x7fae5c15a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae5b7fe640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae5c0cea00 0x7fae5c15a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.865+0000 7fae5b7fe640 1 -- 172.21.15.139:0/2116012625 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0c74f0 con 0x7fae5c06c780 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.866+0000 7fae5b7fe640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae5c06c780 0x7fae5c0cc6a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e9b0 tx=0x7fae4c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:48.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.866+0000 7fae58ff9640 1 -- 172.21.15.139:0/2116012625 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c017070 con 0x7fae5c06c780 2024-04-25T09:53:48.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.866+0000 7fae58ff9640 1 -- 172.21.15.139:0/2116012625 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c00c9a0 con 0x7fae5c06c780 2024-04-25T09:53:48.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.866+0000 7fae61df4640 1 -- 172.21.15.139:0/2116012625 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c0c7790 con 0x7fae5c06c780 2024-04-25T09:53:48.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.866+0000 7fae58ff9640 1 -- 172.21.15.139:0/2116012625 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c00cd90 con 0x7fae5c06c780 2024-04-25T09:53:48.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.866+0000 7fae61df4640 1 -- 172.21.15.139:0/2116012625 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c0c7d50 con 0x7fae5c06c780 2024-04-25T09:53:48.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.867+0000 7fae61df4640 1 -- 172.21.15.139:0/2116012625 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5c074520 con 0x7fae5c06c780 2024-04-25T09:53:48.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.867+0000 7fae58ff9640 1 -- 172.21.15.139:0/2116012625 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae4c01a070 con 0x7fae5c06c780 2024-04-25T09:53:48.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.868+0000 7fae58ff9640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae3c06b4d0 0x7fae3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:48.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.868+0000 7fae58ff9640 1 -- 172.21.15.139:0/2116012625 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fae4c08a9c0 con 0x7fae5c06c780 2024-04-25T09:53:48.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.868+0000 7fae53fff640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae3c06b4d0 0x7fae3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:48.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.868+0000 7fae53fff640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae3c06b4d0 0x7fae3c06d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae5c15f990 tx=0x7fae44009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:48.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:48.872+0000 7fae58ff9640 1 -- 172.21.15.139:0/2116012625 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae4c059460 con 0x7fae5c06c780 2024-04-25T09:53:49.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.080+0000 7fae61df4640 1 -- 172.21.15.139:0/2116012625 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fae5c06ceb0 con 0x7fae5c06c780 2024-04-25T09:53:49.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.081+0000 7fae58ff9640 1 -- 172.21.15.139:0/2116012625 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fae4c058d60 con 0x7fae5c06c780 2024-04-25T09:53:49.083 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:53:49.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.083+0000 7fae61df4640 1 -- 172.21.15.139:0/2116012625 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae3c06b4d0 msgr2=0x7fae3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.083+0000 7fae61df4640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae3c06b4d0 0x7fae3c06d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae5c15f990 tx=0x7fae44009040 comp rx=0 tx=0).stop 2024-04-25T09:53:49.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.083+0000 7fae61df4640 1 -- 172.21.15.139:0/2116012625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae5c06c780 msgr2=0x7fae5c0cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:49.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.083+0000 7fae61df4640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae5c06c780 0x7fae5c0cc6a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e9b0 tx=0x7fae4c00ee70 comp rx=0 tx=0).stop 2024-04-25T09:53:49.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.083+0000 7fae61df4640 1 -- 172.21.15.139:0/2116012625 shutdown_connections 2024-04-25T09:53:49.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.083+0000 7fae61df4640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae3c06b4d0 0x7fae3c06d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.083+0000 7fae61df4640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae5c1611e0 0x7fae5c15aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.083+0000 7fae61df4640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae5c0cea00 0x7fae5c15a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.083+0000 7fae61df4640 1 --2- 172.21.15.139:0/2116012625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae5c06c780 0x7fae5c0cc6a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.083+0000 7fae61df4640 1 -- 172.21.15.139:0/2116012625 >> 172.21.15.139:0/2116012625 conn(0x7fae5c0b5b00 msgr2=0x7fae5c0b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.083+0000 7fae61df4640 1 -- 172.21.15.139:0/2116012625 shutdown_connections 2024-04-25T09:53:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:49.083+0000 7fae61df4640 1 -- 172.21.15.139:0/2116012625 wait complete. 2024-04-25T09:53:49.095 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:53:49.096 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:53:49.096 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-04-25T09:53:49.096 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T09:53:49.096 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T09:53:49.096 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:53:49.096 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:53:49.096 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:53:49.096 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T09:53:49.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.096 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-04-25T09:53:49.112 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:53:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42689sec preferred_lft 42689sec 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::74f2:a893:48f2:5618/64 scope link noprefixroute 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run.smithi150.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether a6:66:1a:14:ec:20 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::2:1cff:fecd:58e0/64 scope link tentative 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run.smithi150.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether a6:66:1a:14:ec:20 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::a466:1aff:fe14:ec20/64 scope link tentative 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.115 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:53:49.176 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-04-25T09:53:49.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.178 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-04-25T09:53:49.233 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-04-25T09:53:49.233 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:53:49.233 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:53:49.234 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:53:49.234 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:53:49.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.234 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-04-25T09:53:49.234 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-25T09:53:49.234 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:53:49.234 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:53:49.234 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:53:49.234 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-25T09:53:49.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.319 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-25T09:53:49.333 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T09:53:49.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.334 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2024-04-25T09:53:49.389 INFO:teuthology.orchestra.run.smithi150.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T09:53:49.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.389 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-04-25T09:53:49.390 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-25T09:53:49.390 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T09:53:49.390 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T09:53:49.390 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-25T09:53:49.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.464 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T09:53:49.527 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:49.528 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:49.528 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:49.528 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.528 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:49.528 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.528 INFO:teuthology.orchestra.run.smithi150.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:53:49.528 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 8a:e8:af:93:9c:ba brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:53:49.529 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:53:49.529 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.529 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::88e8:afff:fe93:9cba/64 scope link 2024-04-25T09:53:49.529 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.530 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T09:53:49.593 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:53:49.593 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:53:49.593 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:53:49.593 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.593 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-04-25T09:53:49.593 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.593 INFO:teuthology.orchestra.run.smithi150.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:53:49.594 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 8a:e8:af:93:9c:ba brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:53:49.594 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:53:49.594 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.594 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::88e8:afff:fe93:9cba/64 scope link 2024-04-25T09:53:49.594 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:53:49.595 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T09:53:49.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.595 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-04-25T09:53:49.595 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-25T09:53:49.595 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T09:53:49.595 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T09:53:49.596 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T09:53:49.596 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T09:53:49.596 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T09:53:49.596 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-25T09:53:49.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.704 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-04-25T09:53:49.704 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-04-25T09:53:49.705 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.1 up 2024-04-25T09:53:49.705 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T09:53:49.705 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-04-25T09:53:49.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:49.740 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T09:53:49.794 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T09:53:49.795 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:53:49.795 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T09:53:49.850 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-25T09:53:49.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.850 DEBUG:teuthology.orchestra.run.smithi150:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:49.931 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:49.931 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:49.938 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:53:49.938 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:53:49.938 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T09:53:49.938 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with new device syntax: 3@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T09:53:49.938 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T09:53:49.939 INFO:teuthology.orchestra.run.smithi150.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T09:53:49.939 INFO:teuthology.orchestra.run.smithi150.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:53:49.939 INFO:teuthology.orchestra.run.smithi150.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:53:49.939 INFO:teuthology.orchestra.run.smithi150.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:53:49.939 INFO:teuthology.orchestra.run.smithi150.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:53:49.941 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:53:49.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:49.941 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T09:53:50.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:50.005 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:50.061 INFO:teuthology.orchestra.run.smithi150.stdout:client4947 2024-04-25T09:53:50.061 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:53:50.061 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:50.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:50.118 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:50.174 INFO:teuthology.orchestra.run.smithi150.stdout:client4947 2024-04-25T09:53:50.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:50.175 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:53:50.230 INFO:teuthology.orchestra.run.smithi150.stdout:client4947 2024-04-25T09:53:50.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:53:50.231 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/922739a8-edcf-4ded-8206-dbdb23e9fea4.client4947/status') 2024-04-25T09:53:50.294 INFO:teuthology.orchestra.run.smithi150.stdout:instance: client.4947 (3)172.21.15.150:0/1985048427 2024-04-25T09:53:50.294 INFO:teuthology.orchestra.run.smithi150.stdout:blocklisted: false 2024-04-25T09:53:50.294 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records in 2024-04-25T09:53:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records out 2024-04-25T09:53:50.295 INFO:teuthology.orchestra.run.smithi150.stderr:71 bytes copied, 3.8143e-05 s, 1.9 MB/s 2024-04-25T09:53:50.296 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T09:53:50.317 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:50.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.426+0000 7f42b20b3640 1 Processor -- start 2024-04-25T09:53:50.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.426+0000 7f42b20b3640 1 -- start start 2024-04-25T09:53:50.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.426+0000 7f42b20b3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f42ac0b0750 0x7f42ac0b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.426+0000 7f42b20b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac0d0730 0x7f42ac0cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.426+0000 7f42b20b3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ac0b1070 con 0x7f42ac0d0b90 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42b20b3640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ac0ccb70 con 0x7f42ac0b0750 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42b20b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0cf970 con 0x7f42ac0d0730 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42abfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f42ac0d0b90 0x7f42ac0cc380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38376/0 (socket says 172.21.15.139:38376) 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42abfff640 1 -- 172.21.15.139:0/2108023154 learned_addr learned my addr 172.21.15.139:0/2108023154 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42ab7fe640 1 --2- 172.21.15.139:0/2108023154 >> v2:172.21.15.150:3300/0 conn(0x7f42ac0b0750 0x7f42ac0b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795022456 0 0) 0x7f42ac0b1070 con 0x7f42ac0d0b90 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aaffd640 1 --2- 172.21.15.139:0/2108023154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac0d0730 0x7f42ac0cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4294003520 con 0x7f42ac0d0b90 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2524129464 0 0) 0x7f4294003520 con 0x7f42ac0d0b90 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac0d0730 msgr2=0x7f42ac0cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 --2- 172.21.15.139:0/2108023154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac0d0730 0x7f42ac0cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 >> v2:172.21.15.150:3300/0 conn(0x7f42ac0b0750 msgr2=0x7f42ac0b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 --2- 172.21.15.139:0/2108023154 >> v2:172.21.15.150:3300/0 conn(0x7f42ac0b0750 0x7f42ac0b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac0cfc50 con 0x7f42ac0d0b90 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f42a40019f0 con 0x7f42ac0d0b90 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 >> v1:172.21.15.139:6789/0 conn(0x7f42ac0d0b90 legacy=0x7f42ac0cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 --2- 172.21.15.139:0/2108023154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42940039b0 0x7f4294003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 --2- 172.21.15.139:0/2108023154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac0d0730 0x7f4294004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 --2- 172.21.15.139:0/2108023154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4294004d40 0x7f42940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0ccb70 con 0x7f42940039b0 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0cf970 con 0x7f4294004d40 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429400b9c0 con 0x7f42ac0d0730 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42ab7fe640 1 --2- 172.21.15.139:0/2108023154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42940039b0 0x7f4294003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.427+0000 7f42aaffd640 1 --2- 172.21.15.139:0/2108023154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac0d0730 0x7f4294004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.428+0000 7f42abfff640 1 --2- 172.21.15.139:0/2108023154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4294004d40 0x7f42940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.428+0000 7f42ab7fe640 1 -- 172.21.15.139:0/2108023154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac0d0730 msgr2=0x7f4294004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.428+0000 7f42ab7fe640 1 --2- 172.21.15.139:0/2108023154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac0d0730 0x7f4294004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.428+0000 7f42ab7fe640 1 -- 172.21.15.139:0/2108023154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4294004d40 msgr2=0x7f42940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.428+0000 7f42ab7fe640 1 --2- 172.21.15.139:0/2108023154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4294004d40 0x7f42940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.428+0000 7f42ab7fe640 1 -- 172.21.15.139:0/2108023154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4294003520 con 0x7f42940039b0 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.428+0000 7f42ab7fe640 1 --2- 172.21.15.139:0/2108023154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42940039b0 0x7f4294003dd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f429c009ac0 tx=0x7f4294005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698fc8bb17496b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.428+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c036070 con 0x7f42940039b0 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.429+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c031400 con 0x7f42940039b0 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.429+0000 7f42aa7fc640 1 -- 172.21.15.139:0/2108023154 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c009690 con 0x7f42940039b0 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.429+0000 7f42b20b3640 1 -- 172.21.15.139:0/2108023154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42940039b0 msgr2=0x7f4294003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.429+0000 7f42b20b3640 1 --2- 172.21.15.139:0/2108023154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42940039b0 0x7f4294003dd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f429c009ac0 tx=0x7f4294005830 comp rx=0 tx=0).stop 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.429+0000 7f42abfff640 1 -- 172.21.15.139:0/2108023154 reap_dead start 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.429+0000 7f42b20b3640 1 -- 172.21.15.139:0/2108023154 shutdown_connections 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.429+0000 7f42b20b3640 1 -- 172.21.15.139:0/2108023154 >> 172.21.15.139:0/2108023154 conn(0x7f42ac0b3380 msgr2=0x7f42ac0b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:50.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.429+0000 7f42b20b3640 1 -- 172.21.15.139:0/2108023154 shutdown_connections 2024-04-25T09:53:50.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.429+0000 7f42b20b3640 1 -- 172.21.15.139:0/2108023154 wait complete. 2024-04-25T09:53:50.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.430+0000 7f42b20b3640 1 Processor -- start 2024-04-25T09:53:50.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.431+0000 7f42b20b3640 1 -- start start 2024-04-25T09:53:50.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42b20b3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42940039b0 0x7f42ac150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42b20b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac15dbd0 0x7f42ac15e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42b20b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42ac06ccc0 0x7f42ac06d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42b20b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac15bcc0 con 0x7f42ac06ccc0 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42b20b3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0d0210 con 0x7f42940039b0 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42b20b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac15be40 con 0x7f42ac15dbd0 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42abfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42ac06ccc0 0x7f42ac06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42abfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42ac06ccc0 0x7f42ac06d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55012/0 (socket says 172.21.15.139:55012) 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42abfff640 1 -- 172.21.15.139:0/1443779557 learned_addr learned my addr 172.21.15.139:0/1443779557 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42aaffd640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac15dbd0 0x7f42ac15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42ab7fe640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42940039b0 0x7f42ac150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42abfff640 1 -- 172.21.15.139:0/1443779557 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac15dbd0 msgr2=0x7f42ac15e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42abfff640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac15dbd0 0x7f42ac15e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42abfff640 1 -- 172.21.15.139:0/1443779557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42940039b0 msgr2=0x7f42ac150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42abfff640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42940039b0 0x7f42ac150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42abfff640 1 -- 172.21.15.139:0/1443779557 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac15c290 con 0x7f42ac06ccc0 2024-04-25T09:53:50.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42abfff640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42ac06ccc0 0x7f42ac06d170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f42ac0676f0 tx=0x7f42a40079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f4293fff640 1 -- 172.21.15.139:0/1443779557 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a400aac0 con 0x7f42ac06ccc0 2024-04-25T09:53:50.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f4293fff640 1 -- 172.21.15.139:0/1443779557 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a4018070 con 0x7f42ac06ccc0 2024-04-25T09:53:50.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42b20b3640 1 -- 172.21.15.139:0/1443779557 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac15c570 con 0x7f42ac06ccc0 2024-04-25T09:53:50.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f4293fff640 1 -- 172.21.15.139:0/1443779557 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a40133f0 con 0x7f42ac06ccc0 2024-04-25T09:53:50.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.432+0000 7f42b20b3640 1 -- 172.21.15.139:0/1443779557 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac15ebd0 con 0x7f42ac06ccc0 2024-04-25T09:53:50.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.433+0000 7f42b20b3640 1 -- 172.21.15.139:0/1443779557 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ac074520 con 0x7f42ac06ccc0 2024-04-25T09:53:50.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.434+0000 7f4293fff640 1 -- 172.21.15.139:0/1443779557 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42a4013590 con 0x7f42ac06ccc0 2024-04-25T09:53:50.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.434+0000 7f4293fff640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f429806b4d0 0x7f429806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.434+0000 7f4293fff640 1 -- 172.21.15.139:0/1443779557 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f42a408a960 con 0x7f42ac06ccc0 2024-04-25T09:53:50.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.434+0000 7f42ab7fe640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f429806b4d0 0x7f429806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.435+0000 7f42ab7fe640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f429806b4d0 0x7f429806d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f429c033000 tx=0x7f429c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.438+0000 7f4293fff640 1 -- 172.21.15.139:0/1443779557 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42a4059530 con 0x7f42ac06ccc0 2024-04-25T09:53:50.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.612+0000 7f42b20b3640 1 -- 172.21.15.139:0/1443779557 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42ac0b22d0 con 0x7f42ac06ccc0 2024-04-25T09:53:50.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.614+0000 7f4293fff640 1 -- 172.21.15.139:0/1443779557 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f42a4058e30 con 0x7f42ac06ccc0 2024-04-25T09:53:50.616 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:50.616 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:50.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.616+0000 7f42b20b3640 1 -- 172.21.15.139:0/1443779557 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f429806b4d0 msgr2=0x7f429806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.616+0000 7f42b20b3640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f429806b4d0 0x7f429806d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f429c033000 tx=0x7f429c008030 comp rx=0 tx=0).stop 2024-04-25T09:53:50.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.616+0000 7f42b20b3640 1 -- 172.21.15.139:0/1443779557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42ac06ccc0 msgr2=0x7f42ac06d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.616+0000 7f42b20b3640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42ac06ccc0 0x7f42ac06d170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f42ac0676f0 tx=0x7f42a40079d0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.616+0000 7f42b20b3640 1 -- 172.21.15.139:0/1443779557 shutdown_connections 2024-04-25T09:53:50.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.616+0000 7f42b20b3640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42ac06ccc0 0x7f42ac06d170 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.616+0000 7f42b20b3640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f429806b4d0 0x7f429806d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.616+0000 7f42b20b3640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42ac15dbd0 0x7f42ac15e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.616+0000 7f42b20b3640 1 --2- 172.21.15.139:0/1443779557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42940039b0 0x7f42ac150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.616+0000 7f42b20b3640 1 -- 172.21.15.139:0/1443779557 >> 172.21.15.139:0/1443779557 conn(0x7f42ac0b3380 msgr2=0x7f42ac0cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:50.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.616+0000 7f42b20b3640 1 -- 172.21.15.139:0/1443779557 shutdown_connections 2024-04-25T09:53:50.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.616+0000 7f42b20b3640 1 -- 172.21.15.139:0/1443779557 wait complete. 2024-04-25T09:53:50.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:50.629 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:50.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.727+0000 7f89381f9640 1 Processor -- start 2024-04-25T09:53:50.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.727+0000 7f89381f9640 1 -- start start 2024-04-25T09:53:50.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.727+0000 7f89381f9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f89300d0a80 0x7f89301618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.727+0000 7f89381f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893009ad40 0x7f8930161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.727+0000 7f89381f9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8930099ca0 con 0x7f89300cd610 2024-04-25T09:53:50.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.727+0000 7f89381f9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8930099a50 con 0x7f89300d0a80 2024-04-25T09:53:50.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.727+0000 7f89381f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8930099f80 con 0x7f893009ad40 2024-04-25T09:53:50.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.728+0000 7f8935f95640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f89300cd610 0x7f89300d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38386/0 (socket says 172.21.15.139:38386) 2024-04-25T09:53:50.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.728+0000 7f8935f95640 1 -- 172.21.15.139:0/792200821 learned_addr learned my addr 172.21.15.139:0/792200821 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:50.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.728+0000 7f8935794640 1 --2- 172.21.15.139:0/792200821 >> v2:172.21.15.139:3300/0 conn(0x7f89300d0a80 0x7f89301618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.728+0000 7f8936796640 1 --2- 172.21.15.139:0/792200821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893009ad40 0x7f8930161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.728+0000 7f8935794640 1 -- 172.21.15.139:0/792200821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893009ad40 msgr2=0x7f8930161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:50.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.728+0000 7f8935794640 1 --2- 172.21.15.139:0/792200821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893009ad40 0x7f8930161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.728+0000 7f8935794640 1 -- 172.21.15.139:0/792200821 >> v1:172.21.15.139:6789/0 conn(0x7f89300cd610 legacy=0x7f89300d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.728+0000 7f8934f93640 1 -- 172.21.15.139:0/792200821 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832957426 0 0) 0x7f8930099ca0 con 0x7f89300cd610 2024-04-25T09:53:50.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.728+0000 7f8935794640 1 -- 172.21.15.139:0/792200821 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893009a260 con 0x7f89300d0a80 2024-04-25T09:53:50.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.729+0000 7f8935794640 1 --2- 172.21.15.139:0/792200821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89300d0a80 0x7f89301618c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f89200097a0 tx=0x7f892002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ea0b400f6ccddb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.729+0000 7f8934f93640 1 -- 172.21.15.139:0/792200821 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920037070 con 0x7f89300d0a80 2024-04-25T09:53:50.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.729+0000 7f8934f93640 1 -- 172.21.15.139:0/792200821 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8920032440 con 0x7f89300d0a80 2024-04-25T09:53:50.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.729+0000 7f8934f93640 1 -- 172.21.15.139:0/792200821 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920032810 con 0x7f89300d0a80 2024-04-25T09:53:50.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.729+0000 7f89381f9640 1 -- 172.21.15.139:0/792200821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89300d0a80 msgr2=0x7f89301618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.729+0000 7f89381f9640 1 --2- 172.21.15.139:0/792200821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89300d0a80 0x7f89301618c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f89200097a0 tx=0x7f892002a210 comp rx=0 tx=0).stop 2024-04-25T09:53:50.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.729+0000 7f89381f9640 1 -- 172.21.15.139:0/792200821 shutdown_connections 2024-04-25T09:53:50.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.729+0000 7f89381f9640 1 --2- 172.21.15.139:0/792200821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893009ad40 0x7f8930161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.729+0000 7f89381f9640 1 --2- 172.21.15.139:0/792200821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89300d0a80 0x7f89301618c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.729+0000 7f89381f9640 1 -- 172.21.15.139:0/792200821 >> 172.21.15.139:0/792200821 conn(0x7f89300b31f0 msgr2=0x7f89300b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:50.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.729+0000 7f89381f9640 1 -- 172.21.15.139:0/792200821 shutdown_connections 2024-04-25T09:53:50.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.729+0000 7f89381f9640 1 -- 172.21.15.139:0/792200821 wait complete. 2024-04-25T09:53:50.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.731+0000 7f89381f9640 1 Processor -- start 2024-04-25T09:53:50.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.731+0000 7f89381f9640 1 -- start start 2024-04-25T09:53:50.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.731+0000 7f89381f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f893009ad40 0x7f893014ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.731+0000 7f89381f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89300cd610 0x7f89300ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.731+0000 7f89381f9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89300d0a80 0x7f89300ced40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.731+0000 7f89381f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89300cd9f0 con 0x7f893009ad40 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.731+0000 7f89381f9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893009a820 con 0x7f89300d0a80 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.731+0000 7f89381f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89300cdb70 con 0x7f89300cd610 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.731+0000 7f8935f95640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f893009ad40 0x7f893014ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.731+0000 7f8935f95640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f893009ad40 0x7f893014ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55032/0 (socket says 172.21.15.139:55032) 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.731+0000 7f8935f95640 1 -- 172.21.15.139:0/254046584 learned_addr learned my addr 172.21.15.139:0/254046584 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f8936796640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89300d0a80 0x7f89300ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f8935794640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89300cd610 0x7f89300ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f8935f95640 1 -- 172.21.15.139:0/254046584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89300cd610 msgr2=0x7f89300ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f8935f95640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89300cd610 0x7f89300ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f8935f95640 1 -- 172.21.15.139:0/254046584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89300d0a80 msgr2=0x7f89300ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f8935f95640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89300d0a80 0x7f89300ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:50.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f8935f95640 1 -- 172.21.15.139:0/254046584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89300c7be0 con 0x7f893009ad40 2024-04-25T09:53:50.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f8935f95640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f893009ad40 0x7f893014ca00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f892c00c4d0 tx=0x7f892c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f89267fc640 1 -- 172.21.15.139:0/254046584 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c018070 con 0x7f893009ad40 2024-04-25T09:53:50.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f89267fc640 1 -- 172.21.15.139:0/254046584 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c007500 con 0x7f893009ad40 2024-04-25T09:53:50.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f89381f9640 1 -- 172.21.15.139:0/254046584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89300c7ec0 con 0x7f893009ad40 2024-04-25T09:53:50.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f89267fc640 1 -- 172.21.15.139:0/254046584 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c004a20 con 0x7f893009ad40 2024-04-25T09:53:50.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.732+0000 7f89381f9640 1 -- 172.21.15.139:0/254046584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89301e73c0 con 0x7f893009ad40 2024-04-25T09:53:50.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.733+0000 7f89381f9640 1 -- 172.21.15.139:0/254046584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8930074520 con 0x7f893009ad40 2024-04-25T09:53:50.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.733+0000 7f89267fc640 1 -- 172.21.15.139:0/254046584 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f892c01b070 con 0x7f893009ad40 2024-04-25T09:53:50.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.734+0000 7f89267fc640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f891406b500 0x7f891406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:50.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.734+0000 7f8935794640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f891406b500 0x7f891406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:50.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.734+0000 7f89267fc640 1 -- 172.21.15.139:0/254046584 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f892c0897c0 con 0x7f893009ad40 2024-04-25T09:53:50.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.734+0000 7f8935794640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f891406b500 0x7f891406d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8920034000 tx=0x7f892002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:50.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.738+0000 7f89267fc640 1 -- 172.21.15.139:0/254046584 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f892c058390 con 0x7f893009ad40 2024-04-25T09:53:50.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.909+0000 7f89381f9640 1 -- 172.21.15.139:0/254046584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89300b3f70 con 0x7f893009ad40 2024-04-25T09:53:51.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.910+0000 7f89267fc640 1 -- 172.21.15.139:0/254046584 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f892c057c90 con 0x7f893009ad40 2024-04-25T09:53:51.028 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:53:51.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.912+0000 7f89381f9640 1 -- 172.21.15.139:0/254046584 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f891406b500 msgr2=0x7f891406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.912+0000 7f89381f9640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f891406b500 0x7f891406d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8920034000 tx=0x7f892002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:51.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.912+0000 7f89381f9640 1 -- 172.21.15.139:0/254046584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f893009ad40 msgr2=0x7f893014ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.912+0000 7f89381f9640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f893009ad40 0x7f893014ca00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f892c00c4d0 tx=0x7f892c00c990 comp rx=0 tx=0).stop 2024-04-25T09:53:51.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.912+0000 7f89381f9640 1 -- 172.21.15.139:0/254046584 shutdown_connections 2024-04-25T09:53:51.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.912+0000 7f89381f9640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f891406b500 0x7f891406d9c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.912+0000 7f89381f9640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89300d0a80 0x7f89300ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.912+0000 7f89381f9640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89300cd610 0x7f89300ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.912+0000 7f89381f9640 1 --2- 172.21.15.139:0/254046584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f893009ad40 0x7f893014ca00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.912+0000 7f89381f9640 1 -- 172.21.15.139:0/254046584 >> 172.21.15.139:0/254046584 conn(0x7f89300b31f0 msgr2=0x7f89300af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.912+0000 7f89381f9640 1 -- 172.21.15.139:0/254046584 shutdown_connections 2024-04-25T09:53:51.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:50.912+0000 7f89381f9640 1 -- 172.21.15.139:0/254046584 wait complete. 2024-04-25T09:53:51.030 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:51.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.129+0000 7fc0aa99e640 1 Processor -- start 2024-04-25T09:53:51.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.129+0000 7fc0aa99e640 1 -- start start 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.130+0000 7fc0aa99e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc0a415f8c0 0x7fc0a409a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.130+0000 7fc0aa99e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0a415fca0 0x7fc0a40cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.130+0000 7fc0aa99e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a4099bf0 con 0x7fc0a40d0220 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.130+0000 7fc0aa99e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a40d06c0 con 0x7fc0a415f8c0 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.130+0000 7fc0aa99e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a4099ed0 con 0x7fc0a415fca0 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.130+0000 7fc0a3fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc0a415f8c0 0x7fc0a409a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.130+0000 7fc0a37fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0a415fca0 0x7fc0a40cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.130+0000 7fc0a3fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc0a415f8c0 0x7fc0a409a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:38732/0 (socket says 172.21.15.139:38732) 2024-04-25T09:53:51.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.130+0000 7fc0a37fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0a415fca0 0x7fc0a40cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:51150/0 (socket says 172.21.15.139:51150) 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.130+0000 7fc0a3fff640 1 -- 172.21.15.139:0/3819271878 learned_addr learned my addr 172.21.15.139:0/3819271878 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.131+0000 7fc0a3fff640 1 -- 172.21.15.139:0/3819271878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0a415fca0 msgr2=0x7fc0a40cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.131+0000 7fc0a3fff640 1 --2- 172.21.15.139:0/3819271878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0a415fca0 0x7fc0a40cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.131+0000 7fc0a3fff640 1 -- 172.21.15.139:0/3819271878 >> v1:172.21.15.150:6789/0 conn(0x7fc0a40d0220 legacy=0x7fc0a415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.131+0000 7fc0a2ffd640 1 -- 172.21.15.139:0/3819271878 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533035433 0 0) 0x7fc0a4099bf0 con 0x7fc0a40d0220 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.131+0000 7fc0a3fff640 1 -- 172.21.15.139:0/3819271878 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a409a1b0 con 0x7fc0a415f8c0 2024-04-25T09:53:51.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.131+0000 7fc0a37fe640 1 --2- 172.21.15.139:0/3819271878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0a415fca0 0x7fc0a40cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:51.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.131+0000 7fc0a3fff640 1 --2- 172.21.15.139:0/3819271878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a415f8c0 0x7fc0a409a9b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc094009870 tx=0x7fc09402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e930b684557e0f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.132+0000 7fc0a2ffd640 1 -- 172.21.15.139:0/3819271878 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094037070 con 0x7fc0a415f8c0 2024-04-25T09:53:51.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.132+0000 7fc0a2ffd640 1 -- 172.21.15.139:0/3819271878 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09402ae50 con 0x7fc0a415f8c0 2024-04-25T09:53:51.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.132+0000 7fc0a2ffd640 1 -- 172.21.15.139:0/3819271878 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094032860 con 0x7fc0a415f8c0 2024-04-25T09:53:51.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.132+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3819271878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a415f8c0 msgr2=0x7fc0a409a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.132+0000 7fc0aa99e640 1 --2- 172.21.15.139:0/3819271878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a415f8c0 0x7fc0a409a9b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc094009870 tx=0x7fc09402a300 comp rx=0 tx=0).stop 2024-04-25T09:53:51.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.132+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3819271878 shutdown_connections 2024-04-25T09:53:51.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.132+0000 7fc0aa99e640 1 --2- 172.21.15.139:0/3819271878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0a415fca0 0x7fc0a40cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.132+0000 7fc0aa99e640 1 --2- 172.21.15.139:0/3819271878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a415f8c0 0x7fc0a409a9b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.132+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3819271878 >> 172.21.15.139:0/3819271878 conn(0x7fc0a40b31f0 msgr2=0x7fc0a40b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.132+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3819271878 shutdown_connections 2024-04-25T09:53:51.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.132+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3819271878 wait complete. 2024-04-25T09:53:51.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.133+0000 7fc0aa99e640 1 Processor -- start 2024-04-25T09:53:51.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.134+0000 7fc0aa99e640 1 -- start start 2024-04-25T09:53:51.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.134+0000 7fc0aa99e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0a40d0220 0x7fc0a415c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.134+0000 7fc0aa99e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a415f8c0 0x7fc0a40b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.134+0000 7fc0aa99e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0a415fca0 0x7fc0a40b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.134+0000 7fc0aa99e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a41602d0 con 0x7fc0a415fca0 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.134+0000 7fc0aa99e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a409aef0 con 0x7fc0a415f8c0 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.134+0000 7fc0aa99e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a4160450 con 0x7fc0a40d0220 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.134+0000 7fc0a8f3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0a415fca0 0x7fc0a40b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.134+0000 7fc0a8f3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0a415fca0 0x7fc0a40b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55048/0 (socket says 172.21.15.139:55048) 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.134+0000 7fc0a8f3b640 1 -- 172.21.15.139:0/3516251902 learned_addr learned my addr 172.21.15.139:0/3516251902 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.134+0000 7fc0a37fe640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a415f8c0 0x7fc0a40b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.135+0000 7fc0a3fff640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0a40d0220 0x7fc0a415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.135+0000 7fc0a8f3b640 1 -- 172.21.15.139:0/3516251902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0a40d0220 msgr2=0x7fc0a415c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.135+0000 7fc0a8f3b640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0a40d0220 0x7fc0a415c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.135+0000 7fc0a8f3b640 1 -- 172.21.15.139:0/3516251902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a415f8c0 msgr2=0x7fc0a40b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.135+0000 7fc0a8f3b640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a415f8c0 0x7fc0a40b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.135+0000 7fc0a8f3b640 1 -- 172.21.15.139:0/3516251902 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a40b5ae0 con 0x7fc0a415fca0 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.135+0000 7fc0a8f3b640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0a415fca0 0x7fc0a40b5480 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00c760 tx=0x7fc09c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.135+0000 7fc0a0ff9640 1 -- 172.21.15.139:0/3516251902 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c003da0 con 0x7fc0a415fca0 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.135+0000 7fc0a0ff9640 1 -- 172.21.15.139:0/3516251902 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c005070 con 0x7fc0a415fca0 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.135+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3516251902 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a40ae5d0 con 0x7fc0a415fca0 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.135+0000 7fc0a0ff9640 1 -- 172.21.15.139:0/3516251902 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c005460 con 0x7fc0a415fca0 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.135+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3516251902 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a40aeb90 con 0x7fc0a415fca0 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.136+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3516251902 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a4074520 con 0x7fc0a415fca0 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.137+0000 7fc0a0ff9640 1 -- 172.21.15.139:0/3516251902 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc09c005600 con 0x7fc0a415fca0 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.137+0000 7fc0a0ff9640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc09006b4d0 0x7fc09006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.137+0000 7fc0a0ff9640 1 -- 172.21.15.139:0/3516251902 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc09c08a380 con 0x7fc0a415fca0 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.137+0000 7fc0a3fff640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc09006b4d0 0x7fc09006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.137+0000 7fc0a3fff640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc09006b4d0 0x7fc09006d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc094034000 tx=0x7fc09402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.141+0000 7fc0a0ff9640 1 -- 172.21.15.139:0/3516251902 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc09c0587f0 con 0x7fc0a415fca0 2024-04-25T09:53:51.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.310+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3516251902 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc0a40b2250 con 0x7fc0a415fca0 2024-04-25T09:53:51.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.310+0000 7fc0a0ff9640 1 -- 172.21.15.139:0/3516251902 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc09c05c890 con 0x7fc0a415fca0 2024-04-25T09:53:51.313 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.312+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3516251902 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc09006b4d0 msgr2=0x7fc09006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.312+0000 7fc0aa99e640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc09006b4d0 0x7fc09006d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc094034000 tx=0x7fc09402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.312+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3516251902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0a415fca0 msgr2=0x7fc0a40b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.312+0000 7fc0aa99e640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0a415fca0 0x7fc0a40b5480 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00c760 tx=0x7fc09c00cc20 comp rx=0 tx=0).stop 2024-04-25T09:53:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.313+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3516251902 shutdown_connections 2024-04-25T09:53:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.313+0000 7fc0aa99e640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0a415fca0 0x7fc0a40b5480 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.313+0000 7fc0aa99e640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc09006b4d0 0x7fc09006d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.313+0000 7fc0aa99e640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0a415f8c0 0x7fc0a40b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.313+0000 7fc0aa99e640 1 --2- 172.21.15.139:0/3516251902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0a40d0220 0x7fc0a415c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.313+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3516251902 >> 172.21.15.139:0/3516251902 conn(0x7fc0a40b31f0 msgr2=0x7fc0a40c5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.313+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3516251902 shutdown_connections 2024-04-25T09:53:51.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.313+0000 7fc0aa99e640 1 -- 172.21.15.139:0/3516251902 wait complete. 2024-04-25T09:53:51.326 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:51.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.424+0000 7fe310fbe640 1 Processor -- start 2024-04-25T09:53:51.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.424+0000 7fe310fbe640 1 -- start start 2024-04-25T09:53:51.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.425+0000 7fe310fbe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe30c144090 0x7fe30c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.425+0000 7fe310fbe640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c0b2a50 con 0x7fe30c06ea40 2024-04-25T09:53:51.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.425+0000 7fe310fbe640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c15b090 con 0x7fe30c0cc9e0 2024-04-25T09:53:51.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.425+0000 7fe310fbe640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c15b370 con 0x7fe30c144090 2024-04-25T09:53:51.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.425+0000 7fe30a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe30c144090 0x7fe30c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.425+0000 7fe30ad9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe30c06ea40 0x7fe30c15e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38396/0 (socket says 172.21.15.139:38396) 2024-04-25T09:53:51.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.425+0000 7fe30ad9d640 1 -- 172.21.15.139:0/1572589068 learned_addr learned my addr 172.21.15.139:0/1572589068 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:51.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.425+0000 7fe30a59c640 1 -- 172.21.15.139:0/1572589068 >> v1:172.21.15.150:6789/0 conn(0x7fe30c0cc9e0 legacy=0x7fe30c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:51.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.425+0000 7fe30a59c640 1 -- 172.21.15.139:0/1572589068 >> v1:172.21.15.139:6789/0 conn(0x7fe30c06ea40 legacy=0x7fe30c15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.425+0000 7fe30a59c640 1 -- 172.21.15.139:0/1572589068 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c15b650 con 0x7fe30c144090 2024-04-25T09:53:51.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.425+0000 7fe30959a640 1 -- 172.21.15.139:0/1572589068 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165223897 0 0) 0x7fe2fc001110 con 0x7fe30c06ea40 2024-04-25T09:53:51.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.426+0000 7fe30a59c640 1 --2- 172.21.15.139:0/1572589068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe30c144090 0x7fe30c0b31a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe3040097a0 tx=0x7fe30402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76300dfa410cfb63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.426+0000 7fe30959a640 1 -- 172.21.15.139:0/1572589068 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304037070 con 0x7fe30c144090 2024-04-25T09:53:51.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.426+0000 7fe30959a640 1 -- 172.21.15.139:0/1572589068 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30402ae50 con 0x7fe30c144090 2024-04-25T09:53:51.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.426+0000 7fe30959a640 1 -- 172.21.15.139:0/1572589068 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304032660 con 0x7fe30c144090 2024-04-25T09:53:51.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.427+0000 7fe310fbe640 1 -- 172.21.15.139:0/1572589068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe30c144090 msgr2=0x7fe30c0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.427+0000 7fe310fbe640 1 --2- 172.21.15.139:0/1572589068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe30c144090 0x7fe30c0b31a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe3040097a0 tx=0x7fe30402a140 comp rx=0 tx=0).stop 2024-04-25T09:53:51.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.427+0000 7fe310fbe640 1 -- 172.21.15.139:0/1572589068 shutdown_connections 2024-04-25T09:53:51.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.427+0000 7fe310fbe640 1 --2- 172.21.15.139:0/1572589068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe30c144090 0x7fe30c0b31a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.427+0000 7fe310fbe640 1 -- 172.21.15.139:0/1572589068 >> 172.21.15.139:0/1572589068 conn(0x7fe30c067000 msgr2=0x7fe30c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.427+0000 7fe310fbe640 1 -- 172.21.15.139:0/1572589068 shutdown_connections 2024-04-25T09:53:51.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.427+0000 7fe310fbe640 1 -- 172.21.15.139:0/1572589068 wait complete. 2024-04-25T09:53:51.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.428+0000 7fe310fbe640 1 Processor -- start 2024-04-25T09:53:51.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.428+0000 7fe310fbe640 1 -- start start 2024-04-25T09:53:51.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe310fbe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe30c06ea40 0x7fe30c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe310fbe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe30c0cc9e0 0x7fe30c0b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe310fbe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe30c144090 0x7fe30c167150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe310fbe640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c144470 con 0x7fe30c06ea40 2024-04-25T09:53:51.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe310fbe640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c15bc10 con 0x7fe30c144090 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe310fbe640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c15ad80 con 0x7fe30c0cc9e0 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe30a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe30c06ea40 0x7fe30c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe30a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe30c06ea40 0x7fe30c159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55074/0 (socket says 172.21.15.139:55074) 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe30a59c640 1 -- 172.21.15.139:0/1301545000 learned_addr learned my addr 172.21.15.139:0/1301545000 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe30a59c640 1 -- 172.21.15.139:0/1301545000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe30c0cc9e0 msgr2=0x7fe30c0b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe30ad9d640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe30c144090 0x7fe30c167150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe309d9b640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe30c0cc9e0 0x7fe30c0b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe30a59c640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe30c0cc9e0 0x7fe30c0b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe30a59c640 1 -- 172.21.15.139:0/1301545000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe30c144090 msgr2=0x7fe30c167150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe30a59c640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe30c144090 0x7fe30c167150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe30a59c640 1 -- 172.21.15.139:0/1301545000 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c15a2d0 con 0x7fe30c06ea40 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.429+0000 7fe30a59c640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe30c06ea40 0x7fe30c159d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe304000c00 tx=0x7fe3040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.430+0000 7fe2faffd640 1 -- 172.21.15.139:0/1301545000 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304037040 con 0x7fe30c06ea40 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.430+0000 7fe2faffd640 1 -- 172.21.15.139:0/1301545000 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe304004030 con 0x7fe30c06ea40 2024-04-25T09:53:51.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.430+0000 7fe310fbe640 1 -- 172.21.15.139:0/1301545000 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c0adaa0 con 0x7fe30c06ea40 2024-04-25T09:53:51.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.430+0000 7fe2faffd640 1 -- 172.21.15.139:0/1301545000 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304006e10 con 0x7fe30c06ea40 2024-04-25T09:53:51.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.430+0000 7fe310fbe640 1 -- 172.21.15.139:0/1301545000 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c0ae060 con 0x7fe30c06ea40 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.431+0000 7fe310fbe640 1 -- 172.21.15.139:0/1301545000 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c074520 con 0x7fe30c06ea40 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.431+0000 7fe2faffd640 1 -- 172.21.15.139:0/1301545000 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe30403d650 con 0x7fe30c06ea40 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.431+0000 7fe2faffd640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe2ec06b4d0 0x7fe2ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.431+0000 7fe2faffd640 1 -- 172.21.15.139:0/1301545000 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe3040aad10 con 0x7fe30c06ea40 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.431+0000 7fe309d9b640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe2ec06b4d0 0x7fe2ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.432+0000 7fe309d9b640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe2ec06b4d0 0x7fe2ec06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe300007fd0 tx=0x7fe300009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.435+0000 7fe2faffd640 1 -- 172.21.15.139:0/1301545000 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe304042020 con 0x7fe30c06ea40 2024-04-25T09:53:51.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.608+0000 7fe310fbe640 1 -- 172.21.15.139:0/1301545000 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe30c06c1a0 con 0x7fe30c06ea40 2024-04-25T09:53:51.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.608+0000 7fe2faffd640 1 -- 172.21.15.139:0/1301545000 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe304079850 con 0x7fe30c06ea40 2024-04-25T09:53:51.611 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:53:51.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.610+0000 7fe310fbe640 1 -- 172.21.15.139:0/1301545000 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe2ec06b4d0 msgr2=0x7fe2ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.610+0000 7fe310fbe640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe2ec06b4d0 0x7fe2ec06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe300007fd0 tx=0x7fe300009040 comp rx=0 tx=0).stop 2024-04-25T09:53:51.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.610+0000 7fe310fbe640 1 -- 172.21.15.139:0/1301545000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe30c06ea40 msgr2=0x7fe30c159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.610+0000 7fe310fbe640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe30c06ea40 0x7fe30c159d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe304000c00 tx=0x7fe3040046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.610+0000 7fe310fbe640 1 -- 172.21.15.139:0/1301545000 shutdown_connections 2024-04-25T09:53:51.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.611+0000 7fe310fbe640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe2ec06b4d0 0x7fe2ec06d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.611+0000 7fe310fbe640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe30c144090 0x7fe30c167150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.611+0000 7fe310fbe640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe30c0cc9e0 0x7fe30c0b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.611+0000 7fe310fbe640 1 --2- 172.21.15.139:0/1301545000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe30c06ea40 0x7fe30c159d90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.611+0000 7fe310fbe640 1 -- 172.21.15.139:0/1301545000 >> 172.21.15.139:0/1301545000 conn(0x7fe30c067000 msgr2=0x7fe30c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.611+0000 7fe310fbe640 1 -- 172.21.15.139:0/1301545000 shutdown_connections 2024-04-25T09:53:51.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.611+0000 7fe310fbe640 1 -- 172.21.15.139:0/1301545000 wait complete. 2024-04-25T09:53:51.624 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:51.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.721+0000 7f3a35038640 1 Processor -- start 2024-04-25T09:53:51.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.721+0000 7f3a35038640 1 -- start start 2024-04-25T09:53:51.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.722+0000 7f3a35038640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3015dbf0 0x7f3a301617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.722+0000 7f3a35038640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3a300cee50 0x7f3a300d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.722+0000 7f3a35038640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a30161dc0 con 0x7f3a300b9370 2024-04-25T09:53:51.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.722+0000 7f3a35038640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a300d1ee0 con 0x7f3a300cee50 2024-04-25T09:53:51.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.722+0000 7f3a35038640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a301620a0 con 0x7f3a3015dbf0 2024-04-25T09:53:51.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.722+0000 7f3a2f59e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3a300cee50 0x7f3a300d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.722+0000 7f3a2f59e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3a300cee50 0x7f3a300d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55078/0 (socket says 172.21.15.139:55078) 2024-04-25T09:53:51.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.722+0000 7f3a2f59e640 1 -- 172.21.15.139:0/1173174284 learned_addr learned my addr 172.21.15.139:0/1173174284 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:51.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.723+0000 7f3a2ed9d640 1 --2- 172.21.15.139:0/1173174284 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3015dbf0 0x7f3a301617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.723+0000 7f3a2f59e640 1 -- 172.21.15.139:0/1173174284 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3015dbf0 msgr2=0x7f3a301617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:51.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.723+0000 7f3a2f59e640 1 --2- 172.21.15.139:0/1173174284 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3015dbf0 0x7f3a301617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.723+0000 7f3a2f59e640 1 -- 172.21.15.139:0/1173174284 >> v1:172.21.15.150:6789/0 conn(0x7f3a300b9370 legacy=0x7f3a300ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.723+0000 7f3a2f59e640 1 -- 172.21.15.139:0/1173174284 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a30162380 con 0x7f3a300cee50 2024-04-25T09:53:51.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.723+0000 7f3a2f59e640 1 --2- 172.21.15.139:0/1173174284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a300cee50 0x7f3a300d2370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009870 tx=0x7f3a2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360bab2817f5ac4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.723+0000 7f3a2dd9b640 1 -- 172.21.15.139:0/1173174284 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a28037070 con 0x7f3a300cee50 2024-04-25T09:53:51.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.724+0000 7f3a2dd9b640 1 -- 172.21.15.139:0/1173174284 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a28032440 con 0x7f3a300cee50 2024-04-25T09:53:51.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.724+0000 7f3a2dd9b640 1 -- 172.21.15.139:0/1173174284 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a28032810 con 0x7f3a300cee50 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.724+0000 7f3a35038640 1 -- 172.21.15.139:0/1173174284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a300cee50 msgr2=0x7f3a300d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.724+0000 7f3a35038640 1 --2- 172.21.15.139:0/1173174284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a300cee50 0x7f3a300d2370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009870 tx=0x7f3a2802a270 comp rx=0 tx=0).stop 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.724+0000 7f3a35038640 1 -- 172.21.15.139:0/1173174284 shutdown_connections 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.724+0000 7f3a35038640 1 --2- 172.21.15.139:0/1173174284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a300cee50 0x7f3a300d2370 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.724+0000 7f3a35038640 1 --2- 172.21.15.139:0/1173174284 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3015dbf0 0x7f3a301617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.724+0000 7f3a35038640 1 -- 172.21.15.139:0/1173174284 >> 172.21.15.139:0/1173174284 conn(0x7f3a300b5480 msgr2=0x7f3a300b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.724+0000 7f3a35038640 1 -- 172.21.15.139:0/1173174284 shutdown_connections 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.724+0000 7f3a35038640 1 -- 172.21.15.139:0/1173174284 wait complete. 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.725+0000 7f3a35038640 1 Processor -- start 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.726+0000 7f3a35038640 1 -- start start 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.726+0000 7f3a35038640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a300b9370 0x7f3a300c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.726+0000 7f3a35038640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a300cee50 0x7f3a3015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.726+0000 7f3a35038640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3015dbf0 0x7f3a3015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.726+0000 7f3a35038640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3015dfd0 con 0x7f3a300cee50 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.726+0000 7f3a35038640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a300b97d0 con 0x7f3a3015dbf0 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.726+0000 7f3a35038640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a30146820 con 0x7f3a300b9370 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.726+0000 7f3a2e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a300cee50 0x7f3a3015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.726+0000 7f3a2e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a300cee50 0x7f3a3015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55090/0 (socket says 172.21.15.139:55090) 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.726+0000 7f3a2e59c640 1 -- 172.21.15.139:0/1258278217 learned_addr learned my addr 172.21.15.139:0/1258278217 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.726+0000 7f3a2ed9d640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a300b9370 0x7f3a300c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.726+0000 7f3a2f59e640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3015dbf0 0x7f3a3015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.727+0000 7f3a2e59c640 1 -- 172.21.15.139:0/1258278217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a300b9370 msgr2=0x7f3a300c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.727+0000 7f3a2e59c640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a300b9370 0x7f3a300c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.727+0000 7f3a2e59c640 1 -- 172.21.15.139:0/1258278217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3015dbf0 msgr2=0x7f3a3015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.727+0000 7f3a2e59c640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3015dbf0 0x7f3a3015c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.727+0000 7f3a2e59c640 1 -- 172.21.15.139:0/1258278217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3015fe20 con 0x7f3a300cee50 2024-04-25T09:53:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.727+0000 7f3a2f59e640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3015dbf0 0x7f3a3015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:51.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.727+0000 7f3a2e59c640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a300cee50 0x7f3a3015c190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c400 tx=0x7f3a2400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.727+0000 7f3a177fe640 1 -- 172.21.15.139:0/1258278217 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a240052d0 con 0x7f3a300cee50 2024-04-25T09:53:51.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.727+0000 7f3a177fe640 1 -- 172.21.15.139:0/1258278217 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a24005490 con 0x7f3a300cee50 2024-04-25T09:53:51.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.727+0000 7f3a35038640 1 -- 172.21.15.139:0/1258278217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a30146a90 con 0x7f3a300cee50 2024-04-25T09:53:51.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.727+0000 7f3a177fe640 1 -- 172.21.15.139:0/1258278217 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24007260 con 0x7f3a300cee50 2024-04-25T09:53:51.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.727+0000 7f3a35038640 1 -- 172.21.15.139:0/1258278217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a30160100 con 0x7f3a300cee50 2024-04-25T09:53:51.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.728+0000 7f3a35038640 1 -- 172.21.15.139:0/1258278217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a30074520 con 0x7f3a300cee50 2024-04-25T09:53:51.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.728+0000 7f3a177fe640 1 -- 172.21.15.139:0/1258278217 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a24007400 con 0x7f3a300cee50 2024-04-25T09:53:51.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.729+0000 7f3a177fe640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:51.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.729+0000 7f3a177fe640 1 -- 172.21.15.139:0/1258278217 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3a240899a0 con 0x7f3a300cee50 2024-04-25T09:53:51.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.729+0000 7f3a2ed9d640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:51.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.729+0000 7f3a2ed9d640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3a20003aa0 tx=0x7f3a20009830 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.733+0000 7f3a177fe640 1 -- 172.21.15.139:0/1258278217 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a24057e90 con 0x7f3a300cee50 2024-04-25T09:53:51.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.907+0000 7f3a35038640 1 -- 172.21.15.139:0/1258278217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3a300d1680 con 0x7f3a300cee50 2024-04-25T09:53:51.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.908+0000 7f3a177fe640 1 -- 172.21.15.139:0/1258278217 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3a2405bf30 con 0x7f3a300cee50 2024-04-25T09:53:51.910 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:51.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.910+0000 7f3a35038640 1 -- 172.21.15.139:0/1258278217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a1c06b4d0 msgr2=0x7f3a1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.910+0000 7f3a35038640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3a20003aa0 tx=0x7f3a20009830 comp rx=0 tx=0).stop 2024-04-25T09:53:51.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.910+0000 7f3a35038640 1 -- 172.21.15.139:0/1258278217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a300cee50 msgr2=0x7f3a3015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:51.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.910+0000 7f3a35038640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a300cee50 0x7f3a3015c190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c400 tx=0x7f3a2400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.910+0000 7f3a35038640 1 -- 172.21.15.139:0/1258278217 shutdown_connections 2024-04-25T09:53:51.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.910+0000 7f3a35038640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a300cee50 0x7f3a3015c190 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.910+0000 7f3a35038640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.910+0000 7f3a35038640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3015dbf0 0x7f3a3015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.910+0000 7f3a35038640 1 --2- 172.21.15.139:0/1258278217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a300b9370 0x7f3a300c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:51.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.910+0000 7f3a35038640 1 -- 172.21.15.139:0/1258278217 >> 172.21.15.139:0/1258278217 conn(0x7f3a300b5480 msgr2=0x7f3a3006c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:51.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.910+0000 7f3a35038640 1 -- 172.21.15.139:0/1258278217 shutdown_connections 2024-04-25T09:53:51.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:51.910+0000 7f3a35038640 1 -- 172.21.15.139:0/1258278217 wait complete. 2024-04-25T09:53:51.923 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:52.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.018+0000 7f772fbd5640 1 Processor -- start 2024-04-25T09:53:52.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.018+0000 7f772fbd5640 1 -- start start 2024-04-25T09:53:52.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.019+0000 7f772fbd5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f77280b0750 0x7f77280b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.019+0000 7f772fbd5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f77280d0b90 0x7f77280cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.019+0000 7f772fbd5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77280b1070 con 0x7f77280d0730 2024-04-25T09:53:52.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.019+0000 7f772fbd5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77280ccb70 con 0x7f77280d0b90 2024-04-25T09:53:52.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.019+0000 7f772fbd5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77280cf970 con 0x7f77280b0750 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.019+0000 7f772e172640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f77280d0b90 0x7f77280cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.019+0000 7f772e172640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f77280d0b90 0x7f77280cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55106/0 (socket says 172.21.15.139:55106) 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.019+0000 7f772d170640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f77280d0730 0x7f77280cbe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38408/0 (socket says 172.21.15.139:38408) 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.019+0000 7f772e172640 1 -- 172.21.15.139:0/214689439 learned_addr learned my addr 172.21.15.139:0/214689439 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.020+0000 7f772e172640 1 -- 172.21.15.139:0/214689439 >> v2:172.21.15.150:3300/0 conn(0x7f77280b0750 msgr2=0x7f77280b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.020+0000 7f772e172640 1 --2- 172.21.15.139:0/214689439 >> v2:172.21.15.150:3300/0 conn(0x7f77280b0750 0x7f77280b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.020+0000 7f772e172640 1 -- 172.21.15.139:0/214689439 >> v1:172.21.15.139:6789/0 conn(0x7f77280d0730 legacy=0x7f77280cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.020+0000 7f772e172640 1 -- 172.21.15.139:0/214689439 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77280cfc50 con 0x7f77280d0b90 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.020+0000 7f772c96f640 1 -- 172.21.15.139:0/214689439 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899243447 0 0) 0x7f77280b1070 con 0x7f77280d0730 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.020+0000 7f772e172640 1 --2- 172.21.15.139:0/214689439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77280d0b90 0x7f77280cc550 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7728072d20 tx=0x7f772002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3a4281f8e01e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.020+0000 7f772c96f640 1 -- 172.21.15.139:0/214689439 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720037070 con 0x7f77280d0b90 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.021+0000 7f772c96f640 1 -- 172.21.15.139:0/214689439 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7720032440 con 0x7f77280d0b90 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.021+0000 7f772c96f640 1 -- 172.21.15.139:0/214689439 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720032810 con 0x7f77280d0b90 2024-04-25T09:53:52.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.021+0000 7f772fbd5640 1 -- 172.21.15.139:0/214689439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77280d0b90 msgr2=0x7f77280cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.021+0000 7f772fbd5640 1 --2- 172.21.15.139:0/214689439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77280d0b90 0x7f77280cc550 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7728072d20 tx=0x7f772002a270 comp rx=0 tx=0).stop 2024-04-25T09:53:52.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.021+0000 7f772fbd5640 1 -- 172.21.15.139:0/214689439 shutdown_connections 2024-04-25T09:53:52.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.021+0000 7f772fbd5640 1 --2- 172.21.15.139:0/214689439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77280d0b90 0x7f77280cc550 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.021+0000 7f772fbd5640 1 --2- 172.21.15.139:0/214689439 >> v2:172.21.15.150:3300/0 conn(0x7f77280b0750 0x7f77280b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.021+0000 7f772fbd5640 1 -- 172.21.15.139:0/214689439 >> 172.21.15.139:0/214689439 conn(0x7f77280b3380 msgr2=0x7f77280b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.021+0000 7f772fbd5640 1 -- 172.21.15.139:0/214689439 shutdown_connections 2024-04-25T09:53:52.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.021+0000 7f772fbd5640 1 -- 172.21.15.139:0/214689439 wait complete. 2024-04-25T09:53:52.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.022+0000 7f772fbd5640 1 Processor -- start 2024-04-25T09:53:52.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.022+0000 7f772fbd5640 1 -- start start 2024-04-25T09:53:52.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772fbd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77280b0750 0x7f77281f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772fbd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77280d0730 0x7f7728151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772fbd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7728152420 0x7f772815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772fbd5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772815bcc0 con 0x7f77280b0750 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772fbd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77280d0210 con 0x7f7728152420 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772fbd5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772815be40 con 0x7f77280d0730 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772d971640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77280b0750 0x7f77281f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772d971640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77280b0750 0x7f77281f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55108/0 (socket says 172.21.15.139:55108) 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772d971640 1 -- 172.21.15.139:0/4138239716 learned_addr learned my addr 172.21.15.139:0/4138239716 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772e172640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7728152420 0x7f772815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772d170640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77280d0730 0x7f7728151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772d971640 1 -- 172.21.15.139:0/4138239716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77280d0730 msgr2=0x7f7728151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772d971640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77280d0730 0x7f7728151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772d971640 1 -- 172.21.15.139:0/4138239716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7728152420 msgr2=0x7f772815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772d971640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7728152420 0x7f772815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.023+0000 7f772d971640 1 -- 172.21.15.139:0/4138239716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77281f5350 con 0x7f77280b0750 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.024+0000 7f772d971640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77280b0750 0x7f77281f4e10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f772400ba50 tx=0x7f772400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.024+0000 7f771e7fc640 1 -- 172.21.15.139:0/4138239716 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724019070 con 0x7f77280b0750 2024-04-25T09:53:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.024+0000 7f771e7fc640 1 -- 172.21.15.139:0/4138239716 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772400cc00 con 0x7f77280b0750 2024-04-25T09:53:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.024+0000 7f772fbd5640 1 -- 172.21.15.139:0/4138239716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7728152920 con 0x7f77280b0750 2024-04-25T09:53:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.024+0000 7f771e7fc640 1 -- 172.21.15.139:0/4138239716 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724014440 con 0x7f77280b0750 2024-04-25T09:53:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.024+0000 7f772fbd5640 1 -- 172.21.15.139:0/4138239716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77281f48f0 con 0x7f77280b0750 2024-04-25T09:53:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.025+0000 7f772fbd5640 1 -- 172.21.15.139:0/4138239716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7728152f40 con 0x7f77280b0750 2024-04-25T09:53:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.025+0000 7f771e7fc640 1 -- 172.21.15.139:0/4138239716 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f772401c070 con 0x7f77280b0750 2024-04-25T09:53:52.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.026+0000 7f771e7fc640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f770c06b4d0 0x7f770c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.026+0000 7f772d170640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f770c06b4d0 0x7f770c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.026+0000 7f771e7fc640 1 -- 172.21.15.139:0/4138239716 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f772408a220 con 0x7f77280b0750 2024-04-25T09:53:52.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.026+0000 7f772d170640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f770c06b4d0 0x7f770c06d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f772815e080 tx=0x7f7718007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.030+0000 7f771e7fc640 1 -- 172.21.15.139:0/4138239716 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7724058710 con 0x7f77280b0750 2024-04-25T09:53:52.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.201+0000 7f772fbd5640 1 -- 172.21.15.139:0/4138239716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7728074520 con 0x7f77280b0750 2024-04-25T09:53:52.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.202+0000 7f771e7fc640 1 -- 172.21.15.139:0/4138239716 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f772405c7b0 con 0x7f77280b0750 2024-04-25T09:53:52.204 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:52.205 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:52.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.204+0000 7f772fbd5640 1 -- 172.21.15.139:0/4138239716 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f770c06b4d0 msgr2=0x7f770c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.204+0000 7f772fbd5640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f770c06b4d0 0x7f770c06d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f772815e080 tx=0x7f7718007de0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.204+0000 7f772fbd5640 1 -- 172.21.15.139:0/4138239716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77280b0750 msgr2=0x7f77281f4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.204+0000 7f772fbd5640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77280b0750 0x7f77281f4e10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f772400ba50 tx=0x7f772400bf10 comp rx=0 tx=0).stop 2024-04-25T09:53:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.204+0000 7f772fbd5640 1 -- 172.21.15.139:0/4138239716 shutdown_connections 2024-04-25T09:53:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.204+0000 7f772fbd5640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f770c06b4d0 0x7f770c06d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.204+0000 7f772fbd5640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7728152420 0x7f772815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.204+0000 7f772fbd5640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77280d0730 0x7f7728151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.204+0000 7f772fbd5640 1 --2- 172.21.15.139:0/4138239716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77280b0750 0x7f77281f4e10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.204+0000 7f772fbd5640 1 -- 172.21.15.139:0/4138239716 >> 172.21.15.139:0/4138239716 conn(0x7f77280b3380 msgr2=0x7f772806f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.204+0000 7f772fbd5640 1 -- 172.21.15.139:0/4138239716 shutdown_connections 2024-04-25T09:53:52.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.204+0000 7f772fbd5640 1 -- 172.21.15.139:0/4138239716 wait complete. 2024-04-25T09:53:52.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:52.218 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:52.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.315+0000 7fc253ecd640 1 Processor -- start 2024-04-25T09:53:52.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.316+0000 7fc253ecd640 1 -- start start 2024-04-25T09:53:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.316+0000 7fc253ecd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c145250 0x7fc24c070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.316+0000 7fc253ecd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc24c0b5690 0x7fc24c071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.316+0000 7fc253ecd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24c0b1100 con 0x7fc24c0719d0 2024-04-25T09:53:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.316+0000 7fc253ecd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24c0ceed0 con 0x7fc24c0b5690 2024-04-25T09:53:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.316+0000 7fc253ecd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0ce2c0 con 0x7fc24c145250 2024-04-25T09:53:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.317+0000 7fc25246a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc24c0b5690 0x7fc24c071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.317+0000 7fc251c69640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c145250 0x7fc24c070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.317+0000 7fc251c69640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c145250 0x7fc24c070be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:51204/0 (socket says 172.21.15.139:51204) 2024-04-25T09:53:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.317+0000 7fc25246a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc24c0b5690 0x7fc24c071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:38762/0 (socket says 172.21.15.139:38762) 2024-04-25T09:53:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.317+0000 7fc251c69640 1 -- 172.21.15.139:0/1544200365 learned_addr learned my addr 172.21.15.139:0/1544200365 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.317+0000 7fc251c69640 1 -- 172.21.15.139:0/1544200365 >> v2:172.21.15.150:3300/0 conn(0x7fc24c0b5690 msgr2=0x7fc24c071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.317+0000 7fc251c69640 1 --2- 172.21.15.139:0/1544200365 >> v2:172.21.15.150:3300/0 conn(0x7fc24c0b5690 0x7fc24c071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.317+0000 7fc251c69640 1 -- 172.21.15.139:0/1544200365 >> v1:172.21.15.150:6789/0 conn(0x7fc24c0719d0 legacy=0x7fc24c15f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.317+0000 7fc251c69640 1 -- 172.21.15.139:0/1544200365 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c0ce5a0 con 0x7fc24c145250 2024-04-25T09:53:52.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.318+0000 7fc251c69640 1 --2- 172.21.15.139:0/1544200365 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c145250 0x7fc24c070be0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc248009870 tx=0x7fc24802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b122f27c1638765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.318+0000 7fc250c67640 1 -- 172.21.15.139:0/1544200365 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248037070 con 0x7fc24c145250 2024-04-25T09:53:52.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.318+0000 7fc250c67640 1 -- 172.21.15.139:0/1544200365 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc248032440 con 0x7fc24c145250 2024-04-25T09:53:52.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.318+0000 7fc250c67640 1 -- 172.21.15.139:0/1544200365 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248032810 con 0x7fc24c145250 2024-04-25T09:53:52.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.318+0000 7fc253ecd640 1 -- 172.21.15.139:0/1544200365 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c145250 msgr2=0x7fc24c070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.318+0000 7fc253ecd640 1 --2- 172.21.15.139:0/1544200365 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c145250 0x7fc24c070be0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc248009870 tx=0x7fc24802a1c0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.318+0000 7fc253ecd640 1 -- 172.21.15.139:0/1544200365 shutdown_connections 2024-04-25T09:53:52.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.318+0000 7fc253ecd640 1 --2- 172.21.15.139:0/1544200365 >> v2:172.21.15.150:3300/0 conn(0x7fc24c0b5690 0x7fc24c071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.318+0000 7fc253ecd640 1 --2- 172.21.15.139:0/1544200365 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c145250 0x7fc24c070be0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.318+0000 7fc253ecd640 1 -- 172.21.15.139:0/1544200365 >> 172.21.15.139:0/1544200365 conn(0x7fc24c06f460 msgr2=0x7fc24c06f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.319+0000 7fc253ecd640 1 -- 172.21.15.139:0/1544200365 shutdown_connections 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.319+0000 7fc253ecd640 1 -- 172.21.15.139:0/1544200365 wait complete. 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.320+0000 7fc253ecd640 1 Processor -- start 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.320+0000 7fc253ecd640 1 -- start start 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.320+0000 7fc253ecd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc24c0719d0 0x7fc24c1403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.320+0000 7fc253ecd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c0b5690 0x7fc24c0c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.320+0000 7fc253ecd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc24c145250 0x7fc24c0c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.320+0000 7fc253ecd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c145630 con 0x7fc24c145250 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.320+0000 7fc253ecd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0ceb60 con 0x7fc24c0719d0 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.320+0000 7fc253ecd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0d11a0 con 0x7fc24c0b5690 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.320+0000 7fc25246a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc24c145250 0x7fc24c0c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.320+0000 7fc25246a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc24c145250 0x7fc24c0c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55120/0 (socket says 172.21.15.139:55120) 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.320+0000 7fc25246a640 1 -- 172.21.15.139:0/2233974961 learned_addr learned my addr 172.21.15.139:0/2233974961 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc251468640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c0b5690 0x7fc24c0c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc251c69640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc24c0719d0 0x7fc24c1403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc25246a640 1 -- 172.21.15.139:0/2233974961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c0b5690 msgr2=0x7fc24c0c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc25246a640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c0b5690 0x7fc24c0c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc25246a640 1 -- 172.21.15.139:0/2233974961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc24c0719d0 msgr2=0x7fc24c1403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc25246a640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc24c0719d0 0x7fc24c1403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc25246a640 1 -- 172.21.15.139:0/2233974961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c06cfc0 con 0x7fc24c145250 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc251468640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c0b5690 0x7fc24c0c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc251c69640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc24c0719d0 0x7fc24c1403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc25246a640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc24c145250 0x7fc24c0c7b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc24400efc0 tx=0x7fc24400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc2427fc640 1 -- 172.21.15.139:0/2233974961 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244018020 con 0x7fc24c145250 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc2427fc640 1 -- 172.21.15.139:0/2233974961 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc244014480 con 0x7fc24c145250 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc253ecd640 1 -- 172.21.15.139:0/2233974961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24c06d2a0 con 0x7fc24c145250 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc2427fc640 1 -- 172.21.15.139:0/2233974961 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244014870 con 0x7fc24c145250 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.321+0000 7fc253ecd640 1 -- 172.21.15.139:0/2233974961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24c06d860 con 0x7fc24c145250 2024-04-25T09:53:52.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.322+0000 7fc253ecd640 1 -- 172.21.15.139:0/2233974961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24c074520 con 0x7fc24c145250 2024-04-25T09:53:52.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.323+0000 7fc2427fc640 1 -- 172.21.15.139:0/2233974961 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc24401b020 con 0x7fc24c145250 2024-04-25T09:53:52.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.323+0000 7fc2427fc640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc23006b4d0 0x7fc23006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.323+0000 7fc2427fc640 1 -- 172.21.15.139:0/2233974961 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc24408b590 con 0x7fc24c145250 2024-04-25T09:53:52.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.323+0000 7fc251c69640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc23006b4d0 0x7fc23006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.323+0000 7fc251c69640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc23006b4d0 0x7fc23006d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc248034000 tx=0x7fc24802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.327+0000 7fc2427fc640 1 -- 172.21.15.139:0/2233974961 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc244022020 con 0x7fc24c145250 2024-04-25T09:53:52.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.498+0000 7fc253ecd640 1 -- 172.21.15.139:0/2233974961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc24c0701f0 con 0x7fc24c145250 2024-04-25T09:53:52.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.498+0000 7fc2427fc640 1 -- 172.21.15.139:0/2233974961 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc244010040 con 0x7fc24c145250 2024-04-25T09:53:52.500 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:53:52.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.500+0000 7fc253ecd640 1 -- 172.21.15.139:0/2233974961 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc23006b4d0 msgr2=0x7fc23006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.500+0000 7fc253ecd640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc23006b4d0 0x7fc23006d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc248034000 tx=0x7fc24802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:52.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.500+0000 7fc253ecd640 1 -- 172.21.15.139:0/2233974961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc24c145250 msgr2=0x7fc24c0c7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.500+0000 7fc253ecd640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc24c145250 0x7fc24c0c7b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc24400efc0 tx=0x7fc24400c490 comp rx=0 tx=0).stop 2024-04-25T09:53:52.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.500+0000 7fc253ecd640 1 -- 172.21.15.139:0/2233974961 shutdown_connections 2024-04-25T09:53:52.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.500+0000 7fc253ecd640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc24c145250 0x7fc24c0c7b00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.500+0000 7fc253ecd640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc23006b4d0 0x7fc23006d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.500+0000 7fc253ecd640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc24c0b5690 0x7fc24c0c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.500+0000 7fc253ecd640 1 --2- 172.21.15.139:0/2233974961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc24c0719d0 0x7fc24c1403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.500+0000 7fc253ecd640 1 -- 172.21.15.139:0/2233974961 >> 172.21.15.139:0/2233974961 conn(0x7fc24c06f460 msgr2=0x7fc24c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.500+0000 7fc253ecd640 1 -- 172.21.15.139:0/2233974961 shutdown_connections 2024-04-25T09:53:52.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.500+0000 7fc253ecd640 1 -- 172.21.15.139:0/2233974961 wait complete. 2024-04-25T09:53:52.513 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:52.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.610+0000 7f9b4bd6a640 1 Processor -- start 2024-04-25T09:53:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.611+0000 7f9b4bd6a640 1 -- start start 2024-04-25T09:53:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.611+0000 7f9b4bd6a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9b44145250 0x7f9b44070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.611+0000 7f9b4bd6a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b440719d0 0x7f9b44071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.611+0000 7f9b4bd6a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b440b1100 con 0x7f9b440d11a0 2024-04-25T09:53:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.611+0000 7f9b4bd6a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b440ceed0 con 0x7f9b44145250 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.611+0000 7f9b4bd6a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440ce2c0 con 0x7f9b440719d0 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.612+0000 7f9b4a307640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9b440d11a0 0x7f9b4415f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38412/0 (socket says 172.21.15.139:38412) 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.612+0000 7f9b4a307640 1 -- 172.21.15.139:0/2211851674 learned_addr learned my addr 172.21.15.139:0/2211851674 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.612+0000 7f9b49b06640 1 --2- 172.21.15.139:0/2211851674 >> v2:172.21.15.150:3300/0 conn(0x7f9b44145250 0x7f9b44070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.612+0000 7f9b49305640 1 --2- 172.21.15.139:0/2211851674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b440719d0 0x7f9b44071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.612+0000 7f9b48b04640 1 -- 172.21.15.139:0/2211851674 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885322882 0 0) 0x7f9b440b1100 con 0x7f9b440d11a0 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.612+0000 7f9b48b04640 1 -- 172.21.15.139:0/2211851674 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b30003520 con 0x7f9b440d11a0 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.612+0000 7f9b48b04640 1 -- 172.21.15.139:0/2211851674 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1238894470 0 0) 0x7f9b30003520 con 0x7f9b440d11a0 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.612+0000 7f9b49b06640 1 -- 172.21.15.139:0/2211851674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b440719d0 msgr2=0x7f9b44071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.612+0000 7f9b49b06640 1 --2- 172.21.15.139:0/2211851674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b440719d0 0x7f9b44071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.612+0000 7f9b49b06640 1 -- 172.21.15.139:0/2211851674 >> v1:172.21.15.139:6789/0 conn(0x7f9b440d11a0 legacy=0x7f9b4415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.612+0000 7f9b49b06640 1 -- 172.21.15.139:0/2211851674 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b440ce5a0 con 0x7f9b44145250 2024-04-25T09:53:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.612+0000 7f9b49305640 1 --2- 172.21.15.139:0/2211851674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b440719d0 0x7f9b44071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:52.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.613+0000 7f9b49b06640 1 --2- 172.21.15.139:0/2211851674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b44145250 0x7f9b44070be0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009d90 tx=0x7f9b4002a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfdde4574202098 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.613+0000 7f9b48b04640 1 -- 172.21.15.139:0/2211851674 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c001670 con 0x7f9b44145250 2024-04-25T09:53:52.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.613+0000 7f9b48b04640 1 -- 172.21.15.139:0/2211851674 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b40007540 con 0x7f9b44145250 2024-04-25T09:53:52.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.613+0000 7f9b48b04640 1 -- 172.21.15.139:0/2211851674 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b400078f0 con 0x7f9b44145250 2024-04-25T09:53:52.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.613+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2211851674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b44145250 msgr2=0x7f9b44070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.613+0000 7f9b4bd6a640 1 --2- 172.21.15.139:0/2211851674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b44145250 0x7f9b44070be0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009d90 tx=0x7f9b4002a4d0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.614+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2211851674 shutdown_connections 2024-04-25T09:53:52.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.614+0000 7f9b4bd6a640 1 --2- 172.21.15.139:0/2211851674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b440719d0 0x7f9b44071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.614+0000 7f9b4bd6a640 1 --2- 172.21.15.139:0/2211851674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b44145250 0x7f9b44070be0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.614+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2211851674 >> 172.21.15.139:0/2211851674 conn(0x7f9b4406f460 msgr2=0x7f9b4406f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.614+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2211851674 shutdown_connections 2024-04-25T09:53:52.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.614+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2211851674 wait complete. 2024-04-25T09:53:52.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.615+0000 7f9b4bd6a640 1 Processor -- start 2024-04-25T09:53:52.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.615+0000 7f9b4bd6a640 1 -- start start 2024-04-25T09:53:52.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.615+0000 7f9b4bd6a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b440719d0 0x7f9b441f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4bd6a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b440d11a0 0x7f9b440cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4bd6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b44145250 0x7f9b440cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4bd6a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44145630 con 0x7f9b44145250 2024-04-25T09:53:52.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4bd6a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440ceb60 con 0x7f9b440719d0 2024-04-25T09:53:52.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4bd6a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440b5690 con 0x7f9b440d11a0 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4a307640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b44145250 0x7f9b440cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4a307640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b44145250 0x7f9b440cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55134/0 (socket says 172.21.15.139:55134) 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4a307640 1 -- 172.21.15.139:0/2629250952 learned_addr learned my addr 172.21.15.139:0/2629250952 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b49305640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b440d11a0 0x7f9b440cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b49b06640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b440719d0 0x7f9b441f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4a307640 1 -- 172.21.15.139:0/2629250952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b440d11a0 msgr2=0x7f9b440cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4a307640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b440d11a0 0x7f9b440cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4a307640 1 -- 172.21.15.139:0/2629250952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b440719d0 msgr2=0x7f9b441f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4a307640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b440719d0 0x7f9b441f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4a307640 1 -- 172.21.15.139:0/2629250952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b440cb940 con 0x7f9b44145250 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4a307640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b44145250 0x7f9b440cb310 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00c750 tx=0x7f9b3c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b3a7fc640 1 -- 172.21.15.139:0/2629250952 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c017070 con 0x7f9b44145250 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b3a7fc640 1 -- 172.21.15.139:0/2629250952 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c005ad0 con 0x7f9b44145250 2024-04-25T09:53:52.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b3a7fc640 1 -- 172.21.15.139:0/2629250952 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c007070 con 0x7f9b44145250 2024-04-25T09:53:52.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2629250952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b440cbc20 con 0x7f9b44145250 2024-04-25T09:53:52.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.616+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2629250952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b440cc1e0 con 0x7f9b44145250 2024-04-25T09:53:52.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.617+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2629250952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b44074520 con 0x7f9b44145250 2024-04-25T09:53:52.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.618+0000 7f9b3a7fc640 1 -- 172.21.15.139:0/2629250952 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b3c01a070 con 0x7f9b44145250 2024-04-25T09:53:52.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.618+0000 7f9b3a7fc640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b2806b4d0 0x7f9b2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.618+0000 7f9b3a7fc640 1 -- 172.21.15.139:0/2629250952 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9b3c089920 con 0x7f9b44145250 2024-04-25T09:53:52.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.618+0000 7f9b49b06640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b2806b4d0 0x7f9b2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.619+0000 7f9b49b06640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b2806b4d0 0x7f9b2806d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b40034000 tx=0x7f9b4002f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.622+0000 7f9b3a7fc640 1 -- 172.21.15.139:0/2629250952 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b3c057e20 con 0x7f9b44145250 2024-04-25T09:53:52.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.790+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2629250952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b440d0f60 con 0x7f9b44145250 2024-04-25T09:53:52.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.790+0000 7f9b3a7fc640 1 -- 172.21.15.139:0/2629250952 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b3c05bec0 con 0x7f9b44145250 2024-04-25T09:53:52.793 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:52.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.792+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2629250952 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b2806b4d0 msgr2=0x7f9b2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.792+0000 7f9b4bd6a640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b2806b4d0 0x7f9b2806d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b40034000 tx=0x7f9b4002f000 comp rx=0 tx=0).stop 2024-04-25T09:53:52.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.792+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2629250952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b44145250 msgr2=0x7f9b440cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.792+0000 7f9b4bd6a640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b44145250 0x7f9b440cb310 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00c750 tx=0x7f9b3c00cc10 comp rx=0 tx=0).stop 2024-04-25T09:53:52.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.793+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2629250952 shutdown_connections 2024-04-25T09:53:52.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.793+0000 7f9b4bd6a640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b44145250 0x7f9b440cb310 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.793+0000 7f9b4bd6a640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b2806b4d0 0x7f9b2806d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.793+0000 7f9b4bd6a640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b440d11a0 0x7f9b440cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.793+0000 7f9b4bd6a640 1 --2- 172.21.15.139:0/2629250952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b440719d0 0x7f9b441f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.793+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2629250952 >> 172.21.15.139:0/2629250952 conn(0x7f9b4406f460 msgr2=0x7f9b440b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.793+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2629250952 shutdown_connections 2024-04-25T09:53:52.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.793+0000 7f9b4bd6a640 1 -- 172.21.15.139:0/2629250952 wait complete. 2024-04-25T09:53:52.806 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:52.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.905+0000 7fdd0553b640 1 Processor -- start 2024-04-25T09:53:52.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.905+0000 7fdd0553b640 1 -- start start 2024-04-25T09:53:52.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.906+0000 7fdd0553b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdd0015dbf0 0x7fdd001617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.906+0000 7fdd0553b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd00161dc0 con 0x7fdd000cee50 2024-04-25T09:53:52.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.906+0000 7fdd0553b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd001620a0 con 0x7fdd000b9370 2024-04-25T09:53:52.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.906+0000 7fdd0553b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd000d1ee0 con 0x7fdd0015dbf0 2024-04-25T09:53:52.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.906+0000 7fdcfeffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdd0015dbf0 0x7fdd001617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.906+0000 7fdcfeffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdd0015dbf0 0x7fdd001617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55146/0 (socket says 172.21.15.139:55146) 2024-04-25T09:53:52.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.906+0000 7fdcff7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdd000cee50 0x7fdd000d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38418/0 (socket says 172.21.15.139:38418) 2024-04-25T09:53:52.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.906+0000 7fdcfeffd640 1 -- 172.21.15.139:0/658908970 learned_addr learned my addr 172.21.15.139:0/658908970 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:52.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.906+0000 7fdcfeffd640 1 -- 172.21.15.139:0/658908970 >> v1:172.21.15.150:6789/0 conn(0x7fdd000b9370 legacy=0x7fdd000ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:52.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.906+0000 7fdcfeffd640 1 -- 172.21.15.139:0/658908970 >> v1:172.21.15.139:6789/0 conn(0x7fdd000cee50 legacy=0x7fdd000d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.906+0000 7fdcfeffd640 1 -- 172.21.15.139:0/658908970 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd00162380 con 0x7fdd0015dbf0 2024-04-25T09:53:52.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.906+0000 7fdcfdffb640 1 -- 172.21.15.139:0/658908970 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695415161 0 0) 0x7fdd00161dc0 con 0x7fdd000cee50 2024-04-25T09:53:52.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.907+0000 7fdcfeffd640 1 --2- 172.21.15.139:0/658908970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0015dbf0 0x7fdd001617e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009870 tx=0x7fdcf002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ed18bef835a6e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.907+0000 7fdcfdffb640 1 -- 172.21.15.139:0/658908970 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0037070 con 0x7fdd0015dbf0 2024-04-25T09:53:52.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.907+0000 7fdcfdffb640 1 -- 172.21.15.139:0/658908970 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf0032440 con 0x7fdd0015dbf0 2024-04-25T09:53:52.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.907+0000 7fdcfdffb640 1 -- 172.21.15.139:0/658908970 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0032810 con 0x7fdd0015dbf0 2024-04-25T09:53:52.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.908+0000 7fdd0553b640 1 -- 172.21.15.139:0/658908970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0015dbf0 msgr2=0x7fdd001617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.908+0000 7fdd0553b640 1 --2- 172.21.15.139:0/658908970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0015dbf0 0x7fdd001617e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009870 tx=0x7fdcf002a2a0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.908+0000 7fdd0553b640 1 -- 172.21.15.139:0/658908970 shutdown_connections 2024-04-25T09:53:52.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.908+0000 7fdd0553b640 1 --2- 172.21.15.139:0/658908970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0015dbf0 0x7fdd001617e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.908+0000 7fdd0553b640 1 -- 172.21.15.139:0/658908970 >> 172.21.15.139:0/658908970 conn(0x7fdd000b5480 msgr2=0x7fdd000b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:52.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.908+0000 7fdd0553b640 1 -- 172.21.15.139:0/658908970 shutdown_connections 2024-04-25T09:53:52.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.908+0000 7fdd0553b640 1 -- 172.21.15.139:0/658908970 wait complete. 2024-04-25T09:53:52.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.909+0000 7fdd0553b640 1 Processor -- start 2024-04-25T09:53:52.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.909+0000 7fdd0553b640 1 -- start start 2024-04-25T09:53:52.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdd0553b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd000b9370 0x7fdd00157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdd0553b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd000cee50 0x7fdd00160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdd0553b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0015dbf0 0x7fdd00155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdd0553b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0015dfd0 con 0x7fdd0015dbf0 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdd0553b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd000b97d0 con 0x7fdd000cee50 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdd0553b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00156bb0 con 0x7fdd000b9370 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdcff7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0015dbf0 0x7fdd00155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdcff7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0015dbf0 0x7fdd00155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55160/0 (socket says 172.21.15.139:55160) 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdcff7fe640 1 -- 172.21.15.139:0/2137976691 learned_addr learned my addr 172.21.15.139:0/2137976691 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdcfeffd640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd000b9370 0x7fdd00157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdcfe7fc640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd000cee50 0x7fdd00160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdcff7fe640 1 -- 172.21.15.139:0/2137976691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd000b9370 msgr2=0x7fdd00157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdcff7fe640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd000b9370 0x7fdd00157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdcff7fe640 1 -- 172.21.15.139:0/2137976691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd000cee50 msgr2=0x7fdd00160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdcff7fe640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd000cee50 0x7fdd00160a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdcff7fe640 1 -- 172.21.15.139:0/2137976691 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd00156e90 con 0x7fdd0015dbf0 2024-04-25T09:53:52.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdcff7fe640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0015dbf0 0x7fdd00155410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c7b0 tx=0x7fdcf800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdce77fe640 1 -- 172.21.15.139:0/2137976691 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8018070 con 0x7fdd0015dbf0 2024-04-25T09:53:52.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdce77fe640 1 -- 172.21.15.139:0/2137976691 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf800a640 con 0x7fdd0015dbf0 2024-04-25T09:53:52.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdce77fe640 1 -- 172.21.15.139:0/2137976691 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf800aa70 con 0x7fdd0015dbf0 2024-04-25T09:53:52.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdd0553b640 1 -- 172.21.15.139:0/2137976691 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd00157450 con 0x7fdd0015dbf0 2024-04-25T09:53:52.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.910+0000 7fdd0553b640 1 -- 172.21.15.139:0/2137976691 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd00156860 con 0x7fdd0015dbf0 2024-04-25T09:53:52.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.911+0000 7fdd0553b640 1 -- 172.21.15.139:0/2137976691 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd00074520 con 0x7fdd0015dbf0 2024-04-25T09:53:52.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.912+0000 7fdce77fe640 1 -- 172.21.15.139:0/2137976691 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdcf801b070 con 0x7fdd0015dbf0 2024-04-25T09:53:52.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.912+0000 7fdce77fe640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdcec06b4d0 0x7fdcec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:52.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.912+0000 7fdce77fe640 1 -- 172.21.15.139:0/2137976691 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdcf8089c00 con 0x7fdd0015dbf0 2024-04-25T09:53:52.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.912+0000 7fdcfeffd640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdcec06b4d0 0x7fdcec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:52.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.913+0000 7fdcfeffd640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdcec06b4d0 0x7fdcec06d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0034000 tx=0x7fdcf002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:52.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:52.916+0000 7fdce77fe640 1 -- 172.21.15.139:0/2137976691 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdcf8090080 con 0x7fdd0015dbf0 2024-04-25T09:53:53.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.087+0000 7fdd0553b640 1 -- 172.21.15.139:0/2137976691 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd000d1680 con 0x7fdd0015dbf0 2024-04-25T09:53:53.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.087+0000 7fdce77fe640 1 -- 172.21.15.139:0/2137976691 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcf80586b0 con 0x7fdd0015dbf0 2024-04-25T09:53:53.090 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.089+0000 7fdd0553b640 1 -- 172.21.15.139:0/2137976691 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdcec06b4d0 msgr2=0x7fdcec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.089+0000 7fdd0553b640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdcec06b4d0 0x7fdcec06d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0034000 tx=0x7fdcf002e000 comp rx=0 tx=0).stop 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.089+0000 7fdd0553b640 1 -- 172.21.15.139:0/2137976691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0015dbf0 msgr2=0x7fdd00155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.089+0000 7fdd0553b640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0015dbf0 0x7fdd00155410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c7b0 tx=0x7fdcf800cc70 comp rx=0 tx=0).stop 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.089+0000 7fdd0553b640 1 -- 172.21.15.139:0/2137976691 shutdown_connections 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.089+0000 7fdd0553b640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd0015dbf0 0x7fdd00155410 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.089+0000 7fdd0553b640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdcec06b4d0 0x7fdcec06d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.089+0000 7fdd0553b640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd000cee50 0x7fdd00160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.089+0000 7fdd0553b640 1 --2- 172.21.15.139:0/2137976691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd000b9370 0x7fdd00157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.089+0000 7fdd0553b640 1 -- 172.21.15.139:0/2137976691 >> 172.21.15.139:0/2137976691 conn(0x7fdd000b5480 msgr2=0x7fdd000cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.089+0000 7fdd0553b640 1 -- 172.21.15.139:0/2137976691 shutdown_connections 2024-04-25T09:53:53.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.089+0000 7fdd0553b640 1 -- 172.21.15.139:0/2137976691 wait complete. 2024-04-25T09:53:53.102 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:53.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.201+0000 7f6990e4d640 1 Processor -- start 2024-04-25T09:53:53.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.201+0000 7f6990e4d640 1 -- start start 2024-04-25T09:53:53.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6990e4d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f698c144090 0x7f698c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6990e4d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698c0b2a50 con 0x7f698c0cc9e0 2024-04-25T09:53:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6990e4d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698c15b090 con 0x7f698c06ea40 2024-04-25T09:53:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6990e4d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698c15b370 con 0x7f698c144090 2024-04-25T09:53:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6983fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f698c0cc9e0 0x7f698c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38424/0 (socket says 172.21.15.139:38424) 2024-04-25T09:53:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6983fff640 1 -- 172.21.15.139:0/3127703901 learned_addr learned my addr 172.21.15.139:0/3127703901 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f698a59c640 1 --2- 172.21.15.139:0/3127703901 >> v2:172.21.15.150:3300/0 conn(0x7f698c144090 0x7f698c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541893467 0 0) 0x7f6974001110 con 0x7f698c0cc9e0 2024-04-25T09:53:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6978003520 con 0x7f698c0cc9e0 2024-04-25T09:53:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2075113766 0 0) 0x7f6978003520 con 0x7f698c0cc9e0 2024-04-25T09:53:53.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 >> v2:172.21.15.150:3300/0 conn(0x7f698c144090 msgr2=0x7f698c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 --2- 172.21.15.139:0/3127703901 >> v2:172.21.15.150:3300/0 conn(0x7f698c144090 0x7f698c0b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 >> v1:172.21.15.150:6789/0 conn(0x7f698c06ea40 legacy=0x7f698c15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c15b650 con 0x7f698c0cc9e0 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f6974001840 con 0x7f698c0cc9e0 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 >> v1:172.21.15.139:6789/0 conn(0x7f698c0cc9e0 legacy=0x7f698c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 --2- 172.21.15.139:0/3127703901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69780039b0 0x7f6978003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 --2- 172.21.15.139:0/3127703901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978004800 0x7f6978004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 --2- 172.21.15.139:0/3127703901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69780051c0 0x7f697800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c15b370 con 0x7f6978004800 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697800c160 con 0x7f69780039b0 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.202+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697800c360 con 0x7f69780051c0 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.203+0000 7f6983fff640 1 --2- 172.21.15.139:0/3127703901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978004800 0x7f6978004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.203+0000 7f698a59c640 1 --2- 172.21.15.139:0/3127703901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69780039b0 0x7f6978003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.203+0000 7f698ad9d640 1 --2- 172.21.15.139:0/3127703901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69780051c0 0x7f697800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.203+0000 7f6983fff640 1 -- 172.21.15.139:0/3127703901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69780051c0 msgr2=0x7f697800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.203+0000 7f6983fff640 1 --2- 172.21.15.139:0/3127703901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69780051c0 0x7f697800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.203+0000 7f6983fff640 1 -- 172.21.15.139:0/3127703901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69780039b0 msgr2=0x7f6978003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.203+0000 7f6983fff640 1 --2- 172.21.15.139:0/3127703901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69780039b0 0x7f6978003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.203+0000 7f6983fff640 1 -- 172.21.15.139:0/3127703901 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6978003520 con 0x7f6978004800 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.203+0000 7f698ad9d640 1 -- 172.21.15.139:0/3127703901 reap_dead start 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.204+0000 7f6983fff640 1 --2- 172.21.15.139:0/3127703901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978004800 0x7f6978004c80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f697400c4c0 tx=0x7f6978005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b982af8dff9f3486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.204+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697400c0e0 con 0x7f6978004800 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.204+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974034070 con 0x7f6978004800 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.204+0000 7f6989d9b640 1 -- 172.21.15.139:0/3127703901 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974005d70 con 0x7f6978004800 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.204+0000 7f6990e4d640 1 -- 172.21.15.139:0/3127703901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978004800 msgr2=0x7f6978004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.204+0000 7f6990e4d640 1 --2- 172.21.15.139:0/3127703901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978004800 0x7f6978004c80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f697400c4c0 tx=0x7f6978005830 comp rx=0 tx=0).stop 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.204+0000 7f6990e4d640 1 -- 172.21.15.139:0/3127703901 shutdown_connections 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.204+0000 7f6990e4d640 1 --2- 172.21.15.139:0/3127703901 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6978004800 0x7f6978004c80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.204+0000 7f6990e4d640 1 -- 172.21.15.139:0/3127703901 >> 172.21.15.139:0/3127703901 conn(0x7f698c067000 msgr2=0x7f698c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:53.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.204+0000 7f6990e4d640 1 -- 172.21.15.139:0/3127703901 shutdown_connections 2024-04-25T09:53:53.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.204+0000 7f6990e4d640 1 -- 172.21.15.139:0/3127703901 wait complete. 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.206+0000 7f6990e4d640 1 Processor -- start 2024-04-25T09:53:53.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.206+0000 7f6990e4d640 1 -- start start 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.206+0000 7f6990e4d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6978004800 0x7f698c1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.206+0000 7f6990e4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f698c1599d0 0x7f698c1ef320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.206+0000 7f6990e4d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f698c140700 0x7f698c0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.206+0000 7f6990e4d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c144470 con 0x7f698c1599d0 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.206+0000 7f6990e4d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c15bc10 con 0x7f6978004800 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.206+0000 7f6990e4d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698c15ad80 con 0x7f698c140700 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.206+0000 7f6983fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f698c1599d0 0x7f698c1ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.206+0000 7f6983fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f698c1599d0 0x7f698c1ef320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55184/0 (socket says 172.21.15.139:55184) 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.206+0000 7f6983fff640 1 -- 172.21.15.139:0/670305626 learned_addr learned my addr 172.21.15.139:0/670305626 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f698ad9d640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f698c140700 0x7f698c0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f6983fff640 1 -- 172.21.15.139:0/670305626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f698c140700 msgr2=0x7f698c0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f698a59c640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6978004800 0x7f698c1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f6983fff640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f698c140700 0x7f698c0b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f6983fff640 1 -- 172.21.15.139:0/670305626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6978004800 msgr2=0x7f698c1592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f6983fff640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6978004800 0x7f698c1592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f6983fff640 1 -- 172.21.15.139:0/670305626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c15d740 con 0x7f698c1599d0 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f698ad9d640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f698c140700 0x7f698c0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f6983fff640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f698c1599d0 0x7f698c1ef320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6974031040 tx=0x7f698c15b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f6982ffd640 1 -- 172.21.15.139:0/670305626 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974001840 con 0x7f698c1599d0 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f6982ffd640 1 -- 172.21.15.139:0/670305626 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697403f030 con 0x7f698c1599d0 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f6990e4d640 1 -- 172.21.15.139:0/670305626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698c15da20 con 0x7f698c1599d0 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f6982ffd640 1 -- 172.21.15.139:0/670305626 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974007290 con 0x7f698c1599d0 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.207+0000 7f6990e4d640 1 -- 172.21.15.139:0/670305626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698c13e990 con 0x7f698c1599d0 2024-04-25T09:53:53.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.208+0000 7f6990e4d640 1 -- 172.21.15.139:0/670305626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698c074520 con 0x7f698c1599d0 2024-04-25T09:53:53.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.208+0000 7f6982ffd640 1 -- 172.21.15.139:0/670305626 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6974007430 con 0x7f698c1599d0 2024-04-25T09:53:53.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.209+0000 7f6982ffd640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f696c06b500 0x7f696c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.209+0000 7f6982ffd640 1 -- 172.21.15.139:0/670305626 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f69740aab60 con 0x7f698c1599d0 2024-04-25T09:53:53.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.209+0000 7f698a59c640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f696c06b500 0x7f696c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.209+0000 7f698a59c640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f696c06b500 0x7f696c06d9c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f697c00bca0 tx=0x7f697c00b3a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.213+0000 7f6982ffd640 1 -- 172.21.15.139:0/670305626 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6974049030 con 0x7f698c1599d0 2024-04-25T09:53:53.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.382+0000 7f6990e4d640 1 -- 172.21.15.139:0/670305626 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f698c06d310 con 0x7f698c1599d0 2024-04-25T09:53:53.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.382+0000 7f6982ffd640 1 -- 172.21.15.139:0/670305626 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f698c0b2a50 con 0x7f698c1599d0 2024-04-25T09:53:53.384 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:53.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.384+0000 7f6990e4d640 1 -- 172.21.15.139:0/670305626 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f696c06b500 msgr2=0x7f696c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.384+0000 7f6990e4d640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f696c06b500 0x7f696c06d9c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f697c00bca0 tx=0x7f697c00b3a0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.384+0000 7f6990e4d640 1 -- 172.21.15.139:0/670305626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f698c1599d0 msgr2=0x7f698c1ef320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.384+0000 7f6990e4d640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f698c1599d0 0x7f698c1ef320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6974031040 tx=0x7f698c15b820 comp rx=0 tx=0).stop 2024-04-25T09:53:53.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.384+0000 7f6990e4d640 1 -- 172.21.15.139:0/670305626 shutdown_connections 2024-04-25T09:53:53.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.384+0000 7f6990e4d640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f698c1599d0 0x7f698c1ef320 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.384+0000 7f6990e4d640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f696c06b500 0x7f696c06d9c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.384+0000 7f6990e4d640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f698c140700 0x7f698c0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.384+0000 7f6990e4d640 1 --2- 172.21.15.139:0/670305626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6978004800 0x7f698c1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.384+0000 7f6990e4d640 1 -- 172.21.15.139:0/670305626 >> 172.21.15.139:0/670305626 conn(0x7f698c067000 msgr2=0x7f698c06ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:53.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.384+0000 7f6990e4d640 1 -- 172.21.15.139:0/670305626 shutdown_connections 2024-04-25T09:53:53.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.384+0000 7f6990e4d640 1 -- 172.21.15.139:0/670305626 wait complete. 2024-04-25T09:53:53.397 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:53.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.492+0000 7f99a99e7640 1 Processor -- start 2024-04-25T09:53:53.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.492+0000 7f99a99e7640 1 -- start start 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a99e7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f99a415dc60 0x7f99a40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a99e7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99a4161850 0x7f99a40ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a99e7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a40cf250 con 0x7f99a40d0970 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a99e7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a40ce780 con 0x7f99a4161850 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a99e7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a40d3730 con 0x7f99a415dc60 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a27fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99a4161850 0x7f99a40ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a37fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f99a40d0970 0x7f99a40d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38426/0 (socket says 172.21.15.139:38426) 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a27fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99a4161850 0x7f99a40ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55190/0 (socket says 172.21.15.139:55190) 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a37fe640 1 -- 172.21.15.139:0/1230832301 learned_addr learned my addr 172.21.15.139:0/1230832301 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a2ffd640 1 --2- 172.21.15.139:0/1230832301 >> v2:172.21.15.150:3300/0 conn(0x7f99a415dc60 0x7f99a40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a27fc640 1 -- 172.21.15.139:0/1230832301 >> v2:172.21.15.150:3300/0 conn(0x7f99a415dc60 msgr2=0x7f99a40b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a27fc640 1 --2- 172.21.15.139:0/1230832301 >> v2:172.21.15.150:3300/0 conn(0x7f99a415dc60 0x7f99a40b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a27fc640 1 -- 172.21.15.139:0/1230832301 >> v1:172.21.15.139:6789/0 conn(0x7f99a40d0970 legacy=0x7f99a40d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.493+0000 7f99a27fc640 1 -- 172.21.15.139:0/1230832301 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a40ce9b0 con 0x7f99a4161850 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.494+0000 7f99a27fc640 1 --2- 172.21.15.139:0/1230832301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a4161850 0x7f99a40ce240 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f99a408c680 tx=0x7f999802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab35c9a203e4ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.494+0000 7f99a1ffb640 1 -- 172.21.15.139:0/1230832301 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998037070 con 0x7f99a4161850 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.494+0000 7f99a1ffb640 1 -- 172.21.15.139:0/1230832301 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9998032440 con 0x7f99a4161850 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.494+0000 7f99a1ffb640 1 -- 172.21.15.139:0/1230832301 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998032810 con 0x7f99a4161850 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.494+0000 7f99a99e7640 1 -- 172.21.15.139:0/1230832301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a4161850 msgr2=0x7f99a40ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.494+0000 7f99a99e7640 1 --2- 172.21.15.139:0/1230832301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a4161850 0x7f99a40ce240 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f99a408c680 tx=0x7f999802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.495+0000 7f99a99e7640 1 -- 172.21.15.139:0/1230832301 shutdown_connections 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.495+0000 7f99a99e7640 1 --2- 172.21.15.139:0/1230832301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a4161850 0x7f99a40ce240 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.495+0000 7f99a99e7640 1 --2- 172.21.15.139:0/1230832301 >> v2:172.21.15.150:3300/0 conn(0x7f99a415dc60 0x7f99a40b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.495+0000 7f99a99e7640 1 -- 172.21.15.139:0/1230832301 >> 172.21.15.139:0/1230832301 conn(0x7f99a40b5910 msgr2=0x7f99a40b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.495+0000 7f99a99e7640 1 -- 172.21.15.139:0/1230832301 shutdown_connections 2024-04-25T09:53:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.495+0000 7f99a99e7640 1 -- 172.21.15.139:0/1230832301 wait complete. 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.496+0000 7f99a99e7640 1 Processor -- start 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.496+0000 7f99a99e7640 1 -- start start 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a99e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40d0970 0x7f99a40c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a99e7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a415dc60 0x7f99a41f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a99e7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a4161850 0x7f99a40c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a99e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40d1fc0 con 0x7f99a40d0970 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a99e7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40cef70 con 0x7f99a4161850 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a99e7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a415e040 con 0x7f99a415dc60 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40d0970 0x7f99a40c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40d0970 0x7f99a40c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55196/0 (socket says 172.21.15.139:55196) 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a2ffd640 1 -- 172.21.15.139:0/1833840389 learned_addr learned my addr 172.21.15.139:0/1833840389 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a37fe640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a4161850 0x7f99a40c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a27fc640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a415dc60 0x7f99a41f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a2ffd640 1 -- 172.21.15.139:0/1833840389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a415dc60 msgr2=0x7f99a41f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a2ffd640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a415dc60 0x7f99a41f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a2ffd640 1 -- 172.21.15.139:0/1833840389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a4161850 msgr2=0x7f99a40c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a2ffd640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a4161850 0x7f99a40c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a2ffd640 1 -- 172.21.15.139:0/1833840389 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a4159f10 con 0x7f99a40d0970 2024-04-25T09:53:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a27fc640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a415dc60 0x7f99a41f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a37fe640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a4161850 0x7f99a40c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a2ffd640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40d0970 0x7f99a40c6760 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9994007f20 tx=0x7f999400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f998bfff640 1 -- 172.21.15.139:0/1833840389 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994014620 con 0x7f99a40d0970 2024-04-25T09:53:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f998bfff640 1 -- 172.21.15.139:0/1833840389 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9994019070 con 0x7f99a40d0970 2024-04-25T09:53:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a99e7640 1 -- 172.21.15.139:0/1833840389 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a414a3b0 con 0x7f99a40d0970 2024-04-25T09:53:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f998bfff640 1 -- 172.21.15.139:0/1833840389 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994014a00 con 0x7f99a40d0970 2024-04-25T09:53:53.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.497+0000 7f99a99e7640 1 -- 172.21.15.139:0/1833840389 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a415a490 con 0x7f99a40d0970 2024-04-25T09:53:53.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.498+0000 7f99a99e7640 1 -- 172.21.15.139:0/1833840389 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a415ac90 con 0x7f99a40d0970 2024-04-25T09:53:53.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.499+0000 7f998bfff640 1 -- 172.21.15.139:0/1833840389 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9994014ba0 con 0x7f99a40d0970 2024-04-25T09:53:53.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.499+0000 7f998bfff640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f999006b4d0 0x7f999006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.499+0000 7f998bfff640 1 -- 172.21.15.139:0/1833840389 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f999408ae90 con 0x7f99a40d0970 2024-04-25T09:53:53.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.499+0000 7f99a27fc640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f999006b4d0 0x7f999006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.500+0000 7f99a27fc640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f999006b4d0 0x7f999006d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9998034000 tx=0x7f999802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.503+0000 7f998bfff640 1 -- 172.21.15.139:0/1833840389 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99a415ac90 con 0x7f99a40d0970 2024-04-25T09:53:53.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.677+0000 7f99a99e7640 1 -- 172.21.15.139:0/1833840389 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99a4074520 con 0x7f99a40d0970 2024-04-25T09:53:53.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.678+0000 7f998bfff640 1 -- 172.21.15.139:0/1833840389 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f99940599d0 con 0x7f99a40d0970 2024-04-25T09:53:53.680 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:53.680 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:53.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.680+0000 7f99a99e7640 1 -- 172.21.15.139:0/1833840389 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f999006b4d0 msgr2=0x7f999006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.680+0000 7f99a99e7640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f999006b4d0 0x7f999006d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9998034000 tx=0x7f999802e000 comp rx=0 tx=0).stop 2024-04-25T09:53:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.680+0000 7f99a99e7640 1 -- 172.21.15.139:0/1833840389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40d0970 msgr2=0x7f99a40c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.680+0000 7f99a99e7640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40d0970 0x7f99a40c6760 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9994007f20 tx=0x7f999400e570 comp rx=0 tx=0).stop 2024-04-25T09:53:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.680+0000 7f99a99e7640 1 -- 172.21.15.139:0/1833840389 shutdown_connections 2024-04-25T09:53:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.680+0000 7f99a99e7640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f999006b4d0 0x7f999006d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.680+0000 7f99a99e7640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a4161850 0x7f99a40c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.680+0000 7f99a99e7640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a415dc60 0x7f99a41f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.680+0000 7f99a99e7640 1 --2- 172.21.15.139:0/1833840389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40d0970 0x7f99a40c6760 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.680+0000 7f99a99e7640 1 -- 172.21.15.139:0/1833840389 >> 172.21.15.139:0/1833840389 conn(0x7f99a40b5910 msgr2=0x7f99a415be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.680+0000 7f99a99e7640 1 -- 172.21.15.139:0/1833840389 shutdown_connections 2024-04-25T09:53:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.680+0000 7f99a99e7640 1 -- 172.21.15.139:0/1833840389 wait complete. 2024-04-25T09:53:53.694 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:53.694 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:53.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.792+0000 7fdfff4e1640 1 Processor -- start 2024-04-25T09:53:53.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.792+0000 7fdfff4e1640 1 -- start start 2024-04-25T09:53:53.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.793+0000 7fdfff4e1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdff80cd610 0x7fdff80d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.793+0000 7fdfff4e1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdff809ad40 0x7fdff80d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.793+0000 7fdfff4e1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdff81618c0 0x7fdff8161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.793+0000 7fdfff4e1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff8099a50 con 0x7fdff81618c0 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.793+0000 7fdfff4e1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff8099ca0 con 0x7fdff809ad40 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.793+0000 7fdfff4e1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff8099f80 con 0x7fdff80cd610 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.793+0000 7fdffda7e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdff81618c0 0x7fdff8161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.793+0000 7fdffda7e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdff81618c0 0x7fdff8161d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55208/0 (socket says 172.21.15.139:55208) 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.793+0000 7fdffda7e640 1 -- 172.21.15.139:0/3719160478 learned_addr learned my addr 172.21.15.139:0/3719160478 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.793+0000 7fdffda7e640 1 -- 172.21.15.139:0/3719160478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdff80cd610 msgr2=0x7fdff80d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.794+0000 7fdffd27d640 1 --2- 172.21.15.139:0/3719160478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdff80cd610 0x7fdff80d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.794+0000 7fdffca7c640 1 --2- 172.21.15.139:0/3719160478 >> v2:172.21.15.150:3300/0 conn(0x7fdff809ad40 0x7fdff80d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.794+0000 7fdffda7e640 1 --2- 172.21.15.139:0/3719160478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdff80cd610 0x7fdff80d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.794+0000 7fdffda7e640 1 -- 172.21.15.139:0/3719160478 >> v2:172.21.15.150:3300/0 conn(0x7fdff809ad40 msgr2=0x7fdff80d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.794+0000 7fdffda7e640 1 --2- 172.21.15.139:0/3719160478 >> v2:172.21.15.150:3300/0 conn(0x7fdff809ad40 0x7fdff80d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.794+0000 7fdffda7e640 1 -- 172.21.15.139:0/3719160478 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff809a260 con 0x7fdff81618c0 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.794+0000 7fdffda7e640 1 --2- 172.21.15.139:0/3719160478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdff81618c0 0x7fdff8161d50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdff80725f0 tx=0x7fdff002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df375e45db4287b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.794+0000 7fdfeffff640 1 -- 172.21.15.139:0/3719160478 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0036070 con 0x7fdff81618c0 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.794+0000 7fdfeffff640 1 -- 172.21.15.139:0/3719160478 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff0031600 con 0x7fdff81618c0 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.794+0000 7fdfeffff640 1 -- 172.21.15.139:0/3719160478 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff00319d0 con 0x7fdff81618c0 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.795+0000 7fdfff4e1640 1 -- 172.21.15.139:0/3719160478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdff81618c0 msgr2=0x7fdff8161d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.795+0000 7fdfff4e1640 1 --2- 172.21.15.139:0/3719160478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdff81618c0 0x7fdff8161d50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdff80725f0 tx=0x7fdff002a730 comp rx=0 tx=0).stop 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.795+0000 7fdfff4e1640 1 -- 172.21.15.139:0/3719160478 shutdown_connections 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.795+0000 7fdfff4e1640 1 --2- 172.21.15.139:0/3719160478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdff81618c0 0x7fdff8161d50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.795+0000 7fdfff4e1640 1 --2- 172.21.15.139:0/3719160478 >> v2:172.21.15.150:3300/0 conn(0x7fdff809ad40 0x7fdff80d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.795+0000 7fdfff4e1640 1 --2- 172.21.15.139:0/3719160478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdff80cd610 0x7fdff80d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.795+0000 7fdfff4e1640 1 -- 172.21.15.139:0/3719160478 >> 172.21.15.139:0/3719160478 conn(0x7fdff80b31f0 msgr2=0x7fdff80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.795+0000 7fdfff4e1640 1 -- 172.21.15.139:0/3719160478 shutdown_connections 2024-04-25T09:53:53.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.795+0000 7fdfff4e1640 1 -- 172.21.15.139:0/3719160478 wait complete. 2024-04-25T09:53:53.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.796+0000 7fdfff4e1640 1 Processor -- start 2024-04-25T09:53:53.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.796+0000 7fdfff4e1640 1 -- start start 2024-04-25T09:53:53.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdfff4e1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdff809ad40 0x7fdff80ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdfff4e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdff80cd610 0x7fdff8157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdfff4e1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdff81618c0 0x7fdff8157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdfff4e1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80cd9f0 con 0x7fdff80cd610 2024-04-25T09:53:53.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdfff4e1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff809a820 con 0x7fdff81618c0 2024-04-25T09:53:53.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdfff4e1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80cdb70 con 0x7fdff809ad40 2024-04-25T09:53:53.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffca7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdff80cd610 0x7fdff8157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffca7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdff80cd610 0x7fdff8157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55210/0 (socket says 172.21.15.139:55210) 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffca7c640 1 -- 172.21.15.139:0/1787365607 learned_addr learned my addr 172.21.15.139:0/1787365607 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffda7e640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdff81618c0 0x7fdff8157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffd27d640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdff809ad40 0x7fdff80ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffca7c640 1 -- 172.21.15.139:0/1787365607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdff809ad40 msgr2=0x7fdff80ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffca7c640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdff809ad40 0x7fdff80ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffca7c640 1 -- 172.21.15.139:0/1787365607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdff81618c0 msgr2=0x7fdff8157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffca7c640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdff81618c0 0x7fdff8157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffca7c640 1 -- 172.21.15.139:0/1787365607 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff815bb50 con 0x7fdff80cd610 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffda7e640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdff81618c0 0x7fdff8157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffd27d640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdff809ad40 0x7fdff80ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.797+0000 7fdffca7c640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdff80cd610 0x7fdff8157a20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800c7f0 tx=0x7fdfe800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.798+0000 7fdfedffb640 1 -- 172.21.15.139:0/1787365607 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8015070 con 0x7fdff80cd610 2024-04-25T09:53:53.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.798+0000 7fdfedffb640 1 -- 172.21.15.139:0/1787365607 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe8004220 con 0x7fdff80cd610 2024-04-25T09:53:53.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.798+0000 7fdfff4e1640 1 -- 172.21.15.139:0/1787365607 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff8158630 con 0x7fdff80cd610 2024-04-25T09:53:53.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.798+0000 7fdfedffb640 1 -- 172.21.15.139:0/1787365607 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8007620 con 0x7fdff80cd610 2024-04-25T09:53:53.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.798+0000 7fdfff4e1640 1 -- 172.21.15.139:0/1787365607 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff815bdf0 con 0x7fdff80cd610 2024-04-25T09:53:53.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.799+0000 7fdfff4e1640 1 -- 172.21.15.139:0/1787365607 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff8074520 con 0x7fdff80cd610 2024-04-25T09:53:53.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.799+0000 7fdfedffb640 1 -- 172.21.15.139:0/1787365607 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfe80079d0 con 0x7fdff80cd610 2024-04-25T09:53:53.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.799+0000 7fdfedffb640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdfdc06b4d0 0x7fdfdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:53.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.800+0000 7fdffd27d640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdfdc06b4d0 0x7fdfdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:53.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.800+0000 7fdfedffb640 1 -- 172.21.15.139:0/1787365607 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdfe808a620 con 0x7fdff80cd610 2024-04-25T09:53:53.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.800+0000 7fdffd27d640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdfdc06b4d0 0x7fdfdc06d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdff400a9d0 tx=0x7fdff4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:53.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.804+0000 7fdfedffb640 1 -- 172.21.15.139:0/1787365607 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdfe80590d0 con 0x7fdff80cd610 2024-04-25T09:53:53.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.975+0000 7fdfff4e1640 1 -- 172.21.15.139:0/1787365607 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdff80b3f70 con 0x7fdff80cd610 2024-04-25T09:53:53.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.975+0000 7fdfedffb640 1 -- 172.21.15.139:0/1787365607 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdfe80589d0 con 0x7fdff80cd610 2024-04-25T09:53:53.977 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:53:53.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.977+0000 7fdfff4e1640 1 -- 172.21.15.139:0/1787365607 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdfdc06b4d0 msgr2=0x7fdfdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.977+0000 7fdfff4e1640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdfdc06b4d0 0x7fdfdc06d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdff400a9d0 tx=0x7fdff4008040 comp rx=0 tx=0).stop 2024-04-25T09:53:53.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.977+0000 7fdfff4e1640 1 -- 172.21.15.139:0/1787365607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdff80cd610 msgr2=0x7fdff8157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:53.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.977+0000 7fdfff4e1640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdff80cd610 0x7fdff8157a20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800c7f0 tx=0x7fdfe800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.977+0000 7fdfff4e1640 1 -- 172.21.15.139:0/1787365607 shutdown_connections 2024-04-25T09:53:53.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.977+0000 7fdfff4e1640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdff80cd610 0x7fdff8157a20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.977+0000 7fdfff4e1640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdfdc06b4d0 0x7fdfdc06d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.977+0000 7fdfff4e1640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdff81618c0 0x7fdff8157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.977+0000 7fdfff4e1640 1 --2- 172.21.15.139:0/1787365607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdff809ad40 0x7fdff80ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:53.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.977+0000 7fdfff4e1640 1 -- 172.21.15.139:0/1787365607 >> 172.21.15.139:0/1787365607 conn(0x7fdff80b31f0 msgr2=0x7fdff80b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:53.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.977+0000 7fdfff4e1640 1 -- 172.21.15.139:0/1787365607 shutdown_connections 2024-04-25T09:53:53.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:53.978+0000 7fdfff4e1640 1 -- 172.21.15.139:0/1787365607 wait complete. 2024-04-25T09:53:53.990 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:54.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.088+0000 7f143eaac640 1 Processor -- start 2024-04-25T09:53:54.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.089+0000 7f143eaac640 1 -- start start 2024-04-25T09:53:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.089+0000 7f143eaac640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f14380b31a0 0x7f143806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.089+0000 7f143eaac640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14380b2a50 con 0x7f1438144090 2024-04-25T09:53:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.089+0000 7f143eaac640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14380cd0f0 con 0x7f143806e870 2024-04-25T09:53:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.089+0000 7f143eaac640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14380b3610 con 0x7f14380b31a0 2024-04-25T09:53:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.089+0000 7f143c848640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1438144090 0x7f14380cc9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38442/0 (socket says 172.21.15.139:38442) 2024-04-25T09:53:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.089+0000 7f143c848640 1 -- 172.21.15.139:0/1339543049 learned_addr learned my addr 172.21.15.139:0/1339543049 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749333820 0 0) 0x7f142c001110 con 0x7f1438144090 2024-04-25T09:53:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1428003520 con 0x7f1438144090 2024-04-25T09:53:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f1437fff640 1 --2- 172.21.15.139:0/1339543049 >> v2:172.21.15.150:3300/0 conn(0x7f14380b31a0 0x7f143806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 304636492 0 0) 0x7f1428003520 con 0x7f1438144090 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 >> v2:172.21.15.150:3300/0 conn(0x7f14380b31a0 msgr2=0x7f143806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 --2- 172.21.15.139:0/1339543049 >> v2:172.21.15.150:3300/0 conn(0x7f14380b31a0 0x7f143806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 >> v1:172.21.15.150:6789/0 conn(0x7f143806e870 legacy=0x7f143815e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143815b090 con 0x7f1438144090 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f142c001840 con 0x7f1438144090 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 >> v1:172.21.15.139:6789/0 conn(0x7f1438144090 legacy=0x7f14380cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 --2- 172.21.15.139:0/1339543049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14280039b0 0x7f1428003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 --2- 172.21.15.139:0/1339543049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1428004800 0x7f1428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 --2- 172.21.15.139:0/1339543049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14280051c0 0x7f142800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14380b3610 con 0x7f1428004800 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142800c160 con 0x7f14280051c0 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142800c360 con 0x7f14280039b0 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f1437fff640 1 --2- 172.21.15.139:0/1339543049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1428004800 0x7f1428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f143c848640 1 --2- 172.21.15.139:0/1339543049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14280039b0 0x7f1428003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f143d049640 1 --2- 172.21.15.139:0/1339543049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14280051c0 0x7f142800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f1437fff640 1 -- 172.21.15.139:0/1339543049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14280039b0 msgr2=0x7f1428003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f1437fff640 1 --2- 172.21.15.139:0/1339543049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14280039b0 0x7f1428003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f1437fff640 1 -- 172.21.15.139:0/1339543049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14280051c0 msgr2=0x7f142800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f1437fff640 1 --2- 172.21.15.139:0/1339543049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14280051c0 0x7f142800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f143d049640 1 -- 172.21.15.139:0/1339543049 reap_dead start 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f1437fff640 1 -- 172.21.15.139:0/1339543049 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1428003520 con 0x7f1428004800 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.090+0000 7f143c848640 1 --2- 172.21.15.139:0/1339543049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14280039b0 0x7f1428003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.091+0000 7f1437fff640 1 --2- 172.21.15.139:0/1339543049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1428004800 0x7f1428004c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1424009ac0 tx=0x7f1428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77ca0527013ec76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.091+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424036070 con 0x7f1428004800 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.091+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1424031400 con 0x7f1428004800 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.091+0000 7f14377fe640 1 -- 172.21.15.139:0/1339543049 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424009690 con 0x7f1428004800 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.092+0000 7f143eaac640 1 -- 172.21.15.139:0/1339543049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1428004800 msgr2=0x7f1428004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.092+0000 7f143eaac640 1 --2- 172.21.15.139:0/1339543049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1428004800 0x7f1428004c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1424009ac0 tx=0x7f1428005830 comp rx=0 tx=0).stop 2024-04-25T09:53:54.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.092+0000 7f143eaac640 1 -- 172.21.15.139:0/1339543049 shutdown_connections 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.092+0000 7f143eaac640 1 --2- 172.21.15.139:0/1339543049 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1428004800 0x7f1428004c80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.092+0000 7f143eaac640 1 -- 172.21.15.139:0/1339543049 >> 172.21.15.139:0/1339543049 conn(0x7f1438067000 msgr2=0x7f143806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.092+0000 7f143eaac640 1 -- 172.21.15.139:0/1339543049 shutdown_connections 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.092+0000 7f143eaac640 1 -- 172.21.15.139:0/1339543049 wait complete. 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.093+0000 7f143eaac640 1 Processor -- start 2024-04-25T09:53:54.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.093+0000 7f143eaac640 1 -- start start 2024-04-25T09:53:54.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143eaac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1428004800 0x7f1438140d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143eaac640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14381f1630 0x7f14381f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143eaac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14380ad7e0 0x7f14380adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143eaac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143815b650 con 0x7f14380ad7e0 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143eaac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143806eca0 con 0x7f1428004800 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143eaac640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143815b930 con 0x7f14381f1630 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143d049640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14380ad7e0 0x7f14380adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143d049640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14380ad7e0 0x7f14380adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55222/0 (socket says 172.21.15.139:55222) 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143d049640 1 -- 172.21.15.139:0/1170098461 learned_addr learned my addr 172.21.15.139:0/1170098461 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f1437fff640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14381f1630 0x7f14381f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143c848640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1428004800 0x7f1438140d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143d049640 1 -- 172.21.15.139:0/1170098461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14381f1630 msgr2=0x7f14381f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143d049640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14381f1630 0x7f14381f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143d049640 1 -- 172.21.15.139:0/1170098461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1428004800 msgr2=0x7f1438140d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143d049640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1428004800 0x7f1438140d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143d049640 1 -- 172.21.15.139:0/1170098461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1438131e80 con 0x7f14380ad7e0 2024-04-25T09:53:54.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f1437fff640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14381f1630 0x7f14381f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.094+0000 7f143d049640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14380ad7e0 0x7f14380adc60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f143806b8b0 tx=0x7f143000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.095+0000 7f14357fa640 1 -- 172.21.15.139:0/1170098461 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143001a070 con 0x7f14380ad7e0 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.095+0000 7f14357fa640 1 -- 172.21.15.139:0/1170098461 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1430006d60 con 0x7f14380ad7e0 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.095+0000 7f143eaac640 1 -- 172.21.15.139:0/1170098461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1438132160 con 0x7f14380ad7e0 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.095+0000 7f14357fa640 1 -- 172.21.15.139:0/1170098461 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430003bb0 con 0x7f14380ad7e0 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.095+0000 7f143eaac640 1 -- 172.21.15.139:0/1170098461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1438132720 con 0x7f14380ad7e0 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.096+0000 7f143eaac640 1 -- 172.21.15.139:0/1170098461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1438074520 con 0x7f14380ad7e0 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.096+0000 7f14357fa640 1 -- 172.21.15.139:0/1170098461 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f143001d070 con 0x7f14380ad7e0 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.096+0000 7f14357fa640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f141c06b500 0x7f141c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.097+0000 7f14357fa640 1 -- 172.21.15.139:0/1170098461 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1430089ac0 con 0x7f14380ad7e0 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.097+0000 7f143c848640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f141c06b500 0x7f141c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.097+0000 7f143c848640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f141c06b500 0x7f141c06d9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f142c000fd0 tx=0x7f142c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.101+0000 7f14357fa640 1 -- 172.21.15.139:0/1170098461 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1430058600 con 0x7f14380ad7e0 2024-04-25T09:53:54.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.273+0000 7f143eaac640 1 -- 172.21.15.139:0/1170098461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14380ccbd0 con 0x7f14380ad7e0 2024-04-25T09:53:54.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.274+0000 7f14357fa640 1 -- 172.21.15.139:0/1170098461 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1430057f00 con 0x7f14380ad7e0 2024-04-25T09:53:54.276 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:54.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.276+0000 7f143eaac640 1 -- 172.21.15.139:0/1170098461 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f141c06b500 msgr2=0x7f141c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.276+0000 7f143eaac640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f141c06b500 0x7f141c06d9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f142c000fd0 tx=0x7f142c009040 comp rx=0 tx=0).stop 2024-04-25T09:53:54.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.276+0000 7f143eaac640 1 -- 172.21.15.139:0/1170098461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14380ad7e0 msgr2=0x7f14380adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.276+0000 7f143eaac640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14380ad7e0 0x7f14380adc60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f143806b8b0 tx=0x7f143000fc90 comp rx=0 tx=0).stop 2024-04-25T09:53:54.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.276+0000 7f143eaac640 1 -- 172.21.15.139:0/1170098461 shutdown_connections 2024-04-25T09:53:54.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.276+0000 7f143eaac640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14380ad7e0 0x7f14380adc60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.276+0000 7f143eaac640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f141c06b500 0x7f141c06d9c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.276+0000 7f143eaac640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14381f1630 0x7f14381f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.276+0000 7f143eaac640 1 --2- 172.21.15.139:0/1170098461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1428004800 0x7f1438140d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.276+0000 7f143eaac640 1 -- 172.21.15.139:0/1170098461 >> 172.21.15.139:0/1170098461 conn(0x7f1438067000 msgr2=0x7f14380af260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.276+0000 7f143eaac640 1 -- 172.21.15.139:0/1170098461 shutdown_connections 2024-04-25T09:53:54.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.276+0000 7f143eaac640 1 -- 172.21.15.139:0/1170098461 wait complete. 2024-04-25T09:53:54.289 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:54.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.387+0000 7f15c467c640 1 Processor -- start 2024-04-25T09:53:54.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.387+0000 7f15c467c640 1 -- start start 2024-04-25T09:53:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.388+0000 7f15c467c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f15bc0712f0 0x7f15bc0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.388+0000 7f15c467c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15bc0b1100 con 0x7f15bc0719d0 2024-04-25T09:53:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.388+0000 7f15c467c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15bc0ce2c0 con 0x7f15bc145250 2024-04-25T09:53:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.388+0000 7f15c467c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15bc0ceed0 con 0x7f15bc0712f0 2024-04-25T09:53:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.389+0000 7f15c1c17640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f15bc0712f0 0x7f15bc0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.389+0000 7f15c2c19640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f15bc0719d0 0x7f15bc0b4bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38446/0 (socket says 172.21.15.139:38446) 2024-04-25T09:53:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.389+0000 7f15c2c19640 1 -- 172.21.15.139:0/1230653659 learned_addr learned my addr 172.21.15.139:0/1230653659 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.389+0000 7f15c1c17640 1 -- 172.21.15.139:0/1230653659 >> v1:172.21.15.150:6789/0 conn(0x7f15bc145250 legacy=0x7f15bc070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.389+0000 7f15c1c17640 1 -- 172.21.15.139:0/1230653659 >> v1:172.21.15.139:6789/0 conn(0x7f15bc0719d0 legacy=0x7f15bc0b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.389+0000 7f15c1c17640 1 -- 172.21.15.139:0/1230653659 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc0ce5a0 con 0x7f15bc0712f0 2024-04-25T09:53:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.389+0000 7f15c1416640 1 -- 172.21.15.139:0/1230653659 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086828360 0 0) 0x7f15bc0b1100 con 0x7f15bc0719d0 2024-04-25T09:53:54.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.390+0000 7f15c1c17640 1 --2- 172.21.15.139:0/1230653659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15bc0712f0 0x7f15bc0b4680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f15b80097a0 tx=0x7f15b802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a63c47c6e96072 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.390+0000 7f15c1416640 1 -- 172.21.15.139:0/1230653659 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8037070 con 0x7f15bc0712f0 2024-04-25T09:53:54.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.390+0000 7f15c1416640 1 -- 172.21.15.139:0/1230653659 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b8032440 con 0x7f15bc0712f0 2024-04-25T09:53:54.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.390+0000 7f15c1416640 1 -- 172.21.15.139:0/1230653659 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8032810 con 0x7f15bc0712f0 2024-04-25T09:53:54.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.390+0000 7f15c467c640 1 -- 172.21.15.139:0/1230653659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15bc0712f0 msgr2=0x7f15bc0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.390+0000 7f15c467c640 1 --2- 172.21.15.139:0/1230653659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15bc0712f0 0x7f15bc0b4680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f15b80097a0 tx=0x7f15b802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.390+0000 7f15c467c640 1 -- 172.21.15.139:0/1230653659 shutdown_connections 2024-04-25T09:53:54.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.390+0000 7f15c467c640 1 --2- 172.21.15.139:0/1230653659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15bc0712f0 0x7f15bc0b4680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.390+0000 7f15c467c640 1 -- 172.21.15.139:0/1230653659 >> 172.21.15.139:0/1230653659 conn(0x7f15bc06f460 msgr2=0x7f15bc06f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.390+0000 7f15c467c640 1 -- 172.21.15.139:0/1230653659 shutdown_connections 2024-04-25T09:53:54.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.390+0000 7f15c467c640 1 -- 172.21.15.139:0/1230653659 wait complete. 2024-04-25T09:53:54.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.391+0000 7f15c467c640 1 Processor -- start 2024-04-25T09:53:54.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.391+0000 7f15c467c640 1 -- start start 2024-04-25T09:53:54.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.392+0000 7f15c467c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15bc0712f0 0x7f15bc13b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.392+0000 7f15c467c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15bc0719d0 0x7f15bc0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.392+0000 7f15c467c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15bc145250 0x7f15bc0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.392+0000 7f15c467c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc145630 con 0x7f15bc0712f0 2024-04-25T09:53:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.392+0000 7f15c467c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0ceb60 con 0x7f15bc145250 2024-04-25T09:53:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.392+0000 7f15c467c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0d11a0 con 0x7f15bc0719d0 2024-04-25T09:53:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.392+0000 7f15c2418640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15bc0712f0 0x7f15bc13b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.392+0000 7f15c2418640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15bc0712f0 0x7f15bc13b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55244/0 (socket says 172.21.15.139:55244) 2024-04-25T09:53:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.392+0000 7f15c2418640 1 -- 172.21.15.139:0/3787272598 learned_addr learned my addr 172.21.15.139:0/3787272598 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.392+0000 7f15c2418640 1 -- 172.21.15.139:0/3787272598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15bc0719d0 msgr2=0x7f15bc0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.392+0000 7f15c2418640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15bc0719d0 0x7f15bc0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.393+0000 7f15c2418640 1 -- 172.21.15.139:0/3787272598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15bc145250 msgr2=0x7f15bc0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.393+0000 7f15c2c19640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15bc145250 0x7f15bc0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.393+0000 7f15c2418640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15bc145250 0x7f15bc0ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.393+0000 7f15c2418640 1 -- 172.21.15.139:0/3787272598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc0cbe20 con 0x7f15bc0712f0 2024-04-25T09:53:54.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.393+0000 7f15c2418640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15bc0712f0 0x7f15bc13b260 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00c400 tx=0x7f15ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.393+0000 7f15b2ffd640 1 -- 172.21.15.139:0/3787272598 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac017070 con 0x7f15bc0712f0 2024-04-25T09:53:54.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.393+0000 7f15b2ffd640 1 -- 172.21.15.139:0/3787272598 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac0052d0 con 0x7f15bc0712f0 2024-04-25T09:53:54.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.393+0000 7f15c467c640 1 -- 172.21.15.139:0/3787272598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15bc0cc100 con 0x7f15bc0712f0 2024-04-25T09:53:54.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.393+0000 7f15b2ffd640 1 -- 172.21.15.139:0/3787272598 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac003830 con 0x7f15bc0712f0 2024-04-25T09:53:54.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.393+0000 7f15c467c640 1 -- 172.21.15.139:0/3787272598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc0cc6c0 con 0x7f15bc0712f0 2024-04-25T09:53:54.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.394+0000 7f15c467c640 1 -- 172.21.15.139:0/3787272598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15bc074520 con 0x7f15bc0712f0 2024-04-25T09:53:54.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.394+0000 7f15b2ffd640 1 -- 172.21.15.139:0/3787272598 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15ac01a070 con 0x7f15bc0712f0 2024-04-25T09:53:54.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.395+0000 7f15b2ffd640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f15a006b4d0 0x7f15a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.395+0000 7f15b2ffd640 1 -- 172.21.15.139:0/3787272598 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f15ac0898c0 con 0x7f15bc0712f0 2024-04-25T09:53:54.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.395+0000 7f15c1c17640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f15a006b4d0 0x7f15a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.395+0000 7f15c1c17640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f15a006b4d0 0x7f15a006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f15b8034000 tx=0x7f15b802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.399+0000 7f15b2ffd640 1 -- 172.21.15.139:0/3787272598 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15ac058400 con 0x7f15bc0712f0 2024-04-25T09:53:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.567+0000 7f15c467c640 1 -- 172.21.15.139:0/3787272598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15bc0b07d0 con 0x7f15bc0712f0 2024-04-25T09:53:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.568+0000 7f15b2ffd640 1 -- 172.21.15.139:0/3787272598 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15ac057d00 con 0x7f15bc0712f0 2024-04-25T09:53:54.570 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:53:54.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.570+0000 7f15c467c640 1 -- 172.21.15.139:0/3787272598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f15a006b4d0 msgr2=0x7f15a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.570+0000 7f15c467c640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f15a006b4d0 0x7f15a006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f15b8034000 tx=0x7f15b802c000 comp rx=0 tx=0).stop 2024-04-25T09:53:54.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.570+0000 7f15c467c640 1 -- 172.21.15.139:0/3787272598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15bc0712f0 msgr2=0x7f15bc13b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.570+0000 7f15c467c640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15bc0712f0 0x7f15bc13b260 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00c400 tx=0x7f15ac00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.570+0000 7f15c467c640 1 -- 172.21.15.139:0/3787272598 shutdown_connections 2024-04-25T09:53:54.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.570+0000 7f15c467c640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f15a006b4d0 0x7f15a006d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.570+0000 7f15c467c640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15bc145250 0x7f15bc0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.570+0000 7f15c467c640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15bc0719d0 0x7f15bc0cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.570+0000 7f15c467c640 1 --2- 172.21.15.139:0/3787272598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15bc0712f0 0x7f15bc13b260 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.570+0000 7f15c467c640 1 -- 172.21.15.139:0/3787272598 >> 172.21.15.139:0/3787272598 conn(0x7f15bc06f460 msgr2=0x7f15bc070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.570+0000 7f15c467c640 1 -- 172.21.15.139:0/3787272598 shutdown_connections 2024-04-25T09:53:54.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.570+0000 7f15c467c640 1 -- 172.21.15.139:0/3787272598 wait complete. 2024-04-25T09:53:54.583 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:54.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.680+0000 7f2b5f5e2640 1 Processor -- start 2024-04-25T09:53:54.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.680+0000 7f2b5f5e2640 1 -- start start 2024-04-25T09:53:54.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.681+0000 7f2b5f5e2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2b581617e0 0x7f2b580ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.681+0000 7f2b5f5e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b580cec80 0x7f2b580d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.681+0000 7f2b5f5e2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b58161dc0 con 0x7f2b5815dbf0 2024-04-25T09:53:54.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.681+0000 7f2b5f5e2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b580d1ee0 con 0x7f2b581617e0 2024-04-25T09:53:54.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.681+0000 7f2b5f5e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b581620a0 con 0x7f2b580cec80 2024-04-25T09:53:54.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.681+0000 7f2b5cb7d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2b581617e0 0x7f2b580ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.681+0000 7f2b5d37e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2b5815dbf0 0x7f2b580b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38454/0 (socket says 172.21.15.139:38454) 2024-04-25T09:53:54.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.681+0000 7f2b5d37e640 1 -- 172.21.15.139:0/2021889334 learned_addr learned my addr 172.21.15.139:0/2021889334 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.681+0000 7f2b5cb7d640 1 -- 172.21.15.139:0/2021889334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b580cec80 msgr2=0x7f2b580d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.681+0000 7f2b5cb7d640 1 --2- 172.21.15.139:0/2021889334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b580cec80 0x7f2b580d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.681+0000 7f2b5cb7d640 1 -- 172.21.15.139:0/2021889334 >> v1:172.21.15.139:6789/0 conn(0x7f2b5815dbf0 legacy=0x7f2b580b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.681+0000 7f2b5cb7d640 1 -- 172.21.15.139:0/2021889334 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b58162380 con 0x7f2b581617e0 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.682+0000 7f2b5cb7d640 1 --2- 172.21.15.139:0/2021889334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b581617e0 0x7f2b580ce740 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2b480097a0 tx=0x7f2b4802a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e596cfca89f6954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.682+0000 7f2b4ffff640 1 -- 172.21.15.139:0/2021889334 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48037070 con 0x7f2b581617e0 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.682+0000 7f2b4ffff640 1 -- 172.21.15.139:0/2021889334 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4802ac90 con 0x7f2b581617e0 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.682+0000 7f2b4ffff640 1 -- 172.21.15.139:0/2021889334 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48032660 con 0x7f2b581617e0 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.682+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2021889334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b581617e0 msgr2=0x7f2b580ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.682+0000 7f2b5f5e2640 1 --2- 172.21.15.139:0/2021889334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b581617e0 0x7f2b580ce740 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2b480097a0 tx=0x7f2b4802a000 comp rx=0 tx=0).stop 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.682+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2021889334 shutdown_connections 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.682+0000 7f2b5f5e2640 1 --2- 172.21.15.139:0/2021889334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b580cec80 0x7f2b580d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.682+0000 7f2b5f5e2640 1 --2- 172.21.15.139:0/2021889334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b581617e0 0x7f2b580ce740 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.682+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2021889334 >> 172.21.15.139:0/2021889334 conn(0x7f2b580b5480 msgr2=0x7f2b580b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.683+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2021889334 shutdown_connections 2024-04-25T09:53:54.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.683+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2021889334 wait complete. 2024-04-25T09:53:54.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.684+0000 7f2b5f5e2640 1 Processor -- start 2024-04-25T09:53:54.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.684+0000 7f2b5f5e2640 1 -- start start 2024-04-25T09:53:54.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.684+0000 7f2b5f5e2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b580cec80 0x7f2b580c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.684+0000 7f2b5f5e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b5815dbf0 0x7f2b58153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.684+0000 7f2b5f5e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b581617e0 0x7f2b5815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.684+0000 7f2b5f5e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5815dfd0 con 0x7f2b5815dbf0 2024-04-25T09:53:54.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.684+0000 7f2b5f5e2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b580d28b0 con 0x7f2b580cec80 2024-04-25T09:53:54.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.684+0000 7f2b5f5e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5815e380 con 0x7f2b581617e0 2024-04-25T09:53:54.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.684+0000 7f2b5cb7d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b5815dbf0 0x7f2b58153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.684+0000 7f2b5cb7d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b5815dbf0 0x7f2b58153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55264/0 (socket says 172.21.15.139:55264) 2024-04-25T09:53:54.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.684+0000 7f2b5cb7d640 1 -- 172.21.15.139:0/2322066857 learned_addr learned my addr 172.21.15.139:0/2322066857 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:54.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.685+0000 7f2b5cb7d640 1 -- 172.21.15.139:0/2322066857 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b581617e0 msgr2=0x7f2b5815a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.685+0000 7f2b5cb7d640 1 --2- 172.21.15.139:0/2322066857 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b581617e0 0x7f2b5815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.685+0000 7f2b5cb7d640 1 -- 172.21.15.139:0/2322066857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b580cec80 msgr2=0x7f2b580c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.685+0000 7f2b5cb7d640 1 --2- 172.21.15.139:0/2322066857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b580cec80 0x7f2b580c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.685+0000 7f2b5cb7d640 1 -- 172.21.15.139:0/2322066857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5815e660 con 0x7f2b5815dbf0 2024-04-25T09:53:54.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.685+0000 7f2b5cb7d640 1 --2- 172.21.15.139:0/2322066857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b5815dbf0 0x7f2b58153fc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2b480098c0 tx=0x7f2b480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.685+0000 7f2b4dffb640 1 -- 172.21.15.139:0/2322066857 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48037040 con 0x7f2b5815dbf0 2024-04-25T09:53:54.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.685+0000 7f2b4dffb640 1 -- 172.21.15.139:0/2322066857 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b48004230 con 0x7f2b5815dbf0 2024-04-25T09:53:54.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.685+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2322066857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5815e940 con 0x7f2b5815dbf0 2024-04-25T09:53:54.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.685+0000 7f2b4dffb640 1 -- 172.21.15.139:0/2322066857 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48003610 con 0x7f2b5815dbf0 2024-04-25T09:53:54.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.685+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2322066857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5815ef00 con 0x7f2b5815dbf0 2024-04-25T09:53:54.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.686+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2322066857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b58074520 con 0x7f2b5815dbf0 2024-04-25T09:53:54.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.687+0000 7f2b4dffb640 1 -- 172.21.15.139:0/2322066857 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b480037b0 con 0x7f2b5815dbf0 2024-04-25T09:53:54.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.687+0000 7f2b4dffb640 1 --2- 172.21.15.139:0/2322066857 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b3c06b4d0 0x7f2b3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.687+0000 7f2b4dffb640 1 -- 172.21.15.139:0/2322066857 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2b480ab360 con 0x7f2b5815dbf0 2024-04-25T09:53:54.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.687+0000 7f2b5d37e640 1 --2- 172.21.15.139:0/2322066857 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b3c06b4d0 0x7f2b3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.688+0000 7f2b5d37e640 1 --2- 172.21.15.139:0/2322066857 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b3c06b4d0 0x7f2b3c06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2b54000f60 tx=0x7f2b540079e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.691+0000 7f2b4dffb640 1 -- 172.21.15.139:0/2322066857 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b48079f30 con 0x7f2b5815dbf0 2024-04-25T09:53:54.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.863+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2322066857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2b580d1680 con 0x7f2b5815dbf0 2024-04-25T09:53:54.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.863+0000 7f2b4dffb640 1 -- 172.21.15.139:0/2322066857 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2b48079830 con 0x7f2b5815dbf0 2024-04-25T09:53:54.865 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:54.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.865+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2322066857 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b3c06b4d0 msgr2=0x7f2b3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.865+0000 7f2b5f5e2640 1 --2- 172.21.15.139:0/2322066857 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b3c06b4d0 0x7f2b3c06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2b54000f60 tx=0x7f2b540079e0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.865+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2322066857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b5815dbf0 msgr2=0x7f2b58153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.865+0000 7f2b5f5e2640 1 --2- 172.21.15.139:0/2322066857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b5815dbf0 0x7f2b58153fc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2b480098c0 tx=0x7f2b480046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.865+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2322066857 shutdown_connections 2024-04-25T09:53:54.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.865+0000 7f2b5f5e2640 1 --2- 172.21.15.139:0/2322066857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b5815dbf0 0x7f2b58153fc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.865+0000 7f2b5f5e2640 1 --2- 172.21.15.139:0/2322066857 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b3c06b4d0 0x7f2b3c06d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.865+0000 7f2b5f5e2640 1 --2- 172.21.15.139:0/2322066857 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b581617e0 0x7f2b5815a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.865+0000 7f2b5f5e2640 1 --2- 172.21.15.139:0/2322066857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b580cec80 0x7f2b580c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.865+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2322066857 >> 172.21.15.139:0/2322066857 conn(0x7f2b580b5480 msgr2=0x7f2b5806c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.865+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2322066857 shutdown_connections 2024-04-25T09:53:54.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.865+0000 7f2b5f5e2640 1 -- 172.21.15.139:0/2322066857 wait complete. 2024-04-25T09:53:54.878 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:54.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.973+0000 7ff7f4dcf640 1 Processor -- start 2024-04-25T09:53:54.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.973+0000 7ff7f4dcf640 1 -- start start 2024-04-25T09:53:54.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.974+0000 7ff7f4dcf640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff7f00d1860 0x7ff7f00cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.974+0000 7ff7f4dcf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f00cbf80 con 0x7ff7f0158240 2024-04-25T09:53:54.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.974+0000 7ff7f4dcf640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f00cd2a0 con 0x7ff7f00cfba0 2024-04-25T09:53:54.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.974+0000 7ff7f4dcf640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f0158620 con 0x7ff7f00d1860 2024-04-25T09:53:54.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.974+0000 7ff7ee59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff7f0158240 0x7ff7f00d1f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38460/0 (socket says 172.21.15.139:38460) 2024-04-25T09:53:54.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.974+0000 7ff7ee59c640 1 -- 172.21.15.139:0/1063438277 learned_addr learned my addr 172.21.15.139:0/1063438277 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.975+0000 7ff7ed59a640 1 -- 172.21.15.139:0/1063438277 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223898388 0 0) 0x7ff7e8001110 con 0x7ff7f0158240 2024-04-25T09:53:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.975+0000 7ff7ed59a640 1 -- 172.21.15.139:0/1063438277 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d8003520 con 0x7ff7f0158240 2024-04-25T09:53:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.975+0000 7ff7edd9b640 1 --2- 172.21.15.139:0/1063438277 >> v2:172.21.15.150:3300/0 conn(0x7ff7f00d1860 0x7ff7f00cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.975+0000 7ff7edd9b640 1 -- 172.21.15.139:0/1063438277 >> v1:172.21.15.150:6789/0 conn(0x7ff7f00cfba0 legacy=0x7ff7f00cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.975+0000 7ff7edd9b640 1 -- 172.21.15.139:0/1063438277 >> v1:172.21.15.139:6789/0 conn(0x7ff7f0158240 legacy=0x7ff7f00d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.975+0000 7ff7edd9b640 1 -- 172.21.15.139:0/1063438277 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f00cc200 con 0x7ff7f00d1860 2024-04-25T09:53:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.976+0000 7ff7edd9b640 1 --2- 172.21.15.139:0/1063438277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f00d1860 0x7ff7f00cf660 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40097a0 tx=0x7ff7e402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31a091c283015fed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.976+0000 7ff7ed59a640 1 -- 172.21.15.139:0/1063438277 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4037070 con 0x7ff7f00d1860 2024-04-25T09:53:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.976+0000 7ff7ed59a640 1 -- 172.21.15.139:0/1063438277 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e4032440 con 0x7ff7f00d1860 2024-04-25T09:53:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.976+0000 7ff7ed59a640 1 -- 172.21.15.139:0/1063438277 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4032810 con 0x7ff7f00d1860 2024-04-25T09:53:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.977+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/1063438277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f00d1860 msgr2=0x7ff7f00cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.977+0000 7ff7f4dcf640 1 --2- 172.21.15.139:0/1063438277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f00d1860 0x7ff7f00cf660 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40097a0 tx=0x7ff7e402a230 comp rx=0 tx=0).stop 2024-04-25T09:53:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.977+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/1063438277 shutdown_connections 2024-04-25T09:53:54.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.977+0000 7ff7f4dcf640 1 --2- 172.21.15.139:0/1063438277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f00d1860 0x7ff7f00cf660 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.977+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/1063438277 >> 172.21.15.139:0/1063438277 conn(0x7ff7f00b3380 msgr2=0x7ff7f00b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:54.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.977+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/1063438277 shutdown_connections 2024-04-25T09:53:54.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.977+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/1063438277 wait complete. 2024-04-25T09:53:54.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.978+0000 7ff7f4dcf640 1 Processor -- start 2024-04-25T09:53:54.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.978+0000 7ff7f4dcf640 1 -- start start 2024-04-25T09:53:54.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.979+0000 7ff7f4dcf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f00cfba0 0x7ff7f00ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.979+0000 7ff7f4dcf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f00d1860 0x7ff7f00c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.979+0000 7ff7f4dcf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f0158240 0x7ff7f01f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.979+0000 7ff7f4dcf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00cffd0 con 0x7ff7f00cfba0 2024-04-25T09:53:54.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.979+0000 7ff7f4dcf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00cce10 con 0x7ff7f0158240 2024-04-25T09:53:54.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.979+0000 7ff7f4dcf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00cc7c0 con 0x7ff7f00d1860 2024-04-25T09:53:54.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.979+0000 7ff7ee59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f00cfba0 0x7ff7f00ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.979+0000 7ff7ee59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f00cfba0 0x7ff7f00ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55274/0 (socket says 172.21.15.139:55274) 2024-04-25T09:53:54.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.979+0000 7ff7ee59c640 1 -- 172.21.15.139:0/3300480198 learned_addr learned my addr 172.21.15.139:0/3300480198 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.979+0000 7ff7ee59c640 1 -- 172.21.15.139:0/3300480198 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f00d1860 msgr2=0x7ff7f00c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.979+0000 7ff7eed9d640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f0158240 0x7ff7f01f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.979+0000 7ff7edd9b640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f00d1860 0x7ff7f00c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.980+0000 7ff7ee59c640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f00d1860 0x7ff7f00c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.980+0000 7ff7ee59c640 1 -- 172.21.15.139:0/3300480198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f0158240 msgr2=0x7ff7f01f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.980+0000 7ff7ee59c640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f0158240 0x7ff7f01f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.980+0000 7ff7ee59c640 1 -- 172.21.15.139:0/3300480198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f00c92a0 con 0x7ff7f00cfba0 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.980+0000 7ff7eed9d640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f0158240 0x7ff7f01f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.980+0000 7ff7ee59c640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f00cfba0 0x7ff7f00ca340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800c730 tx=0x7ff7e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.980+0000 7ff7deffd640 1 -- 172.21.15.139:0/3300480198 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8017070 con 0x7ff7f00cfba0 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.980+0000 7ff7deffd640 1 -- 172.21.15.139:0/3300480198 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e8005ab0 con 0x7ff7f00cfba0 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.980+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/3300480198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f00c9580 con 0x7ff7f00cfba0 2024-04-25T09:53:54.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.980+0000 7ff7deffd640 1 -- 172.21.15.139:0/3300480198 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8007070 con 0x7ff7f00cfba0 2024-04-25T09:53:54.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.980+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/3300480198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f00c9b40 con 0x7ff7f00cfba0 2024-04-25T09:53:54.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.981+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/3300480198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f0074520 con 0x7ff7f00cfba0 2024-04-25T09:53:54.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.981+0000 7ff7deffd640 1 -- 172.21.15.139:0/3300480198 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7e801a070 con 0x7ff7f00cfba0 2024-04-25T09:53:54.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.982+0000 7ff7deffd640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff7d006b4d0 0x7ff7d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:54.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.982+0000 7ff7deffd640 1 -- 172.21.15.139:0/3300480198 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff7e8089980 con 0x7ff7f00cfba0 2024-04-25T09:53:54.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.982+0000 7ff7edd9b640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff7d006b4d0 0x7ff7d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:54.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.982+0000 7ff7edd9b640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff7d006b4d0 0x7ff7d006d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4034000 tx=0x7ff7e402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:54.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:54.987+0000 7ff7deffd640 1 -- 172.21.15.139:0/3300480198 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7f00cbf80 con 0x7ff7f00cfba0 2024-04-25T09:53:55.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.157+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/3300480198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7f015c480 con 0x7ff7f00cfba0 2024-04-25T09:53:55.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.158+0000 7ff7deffd640 1 -- 172.21.15.139:0/3300480198 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18562 (secure 0 0 0) 0x7ff7e80584d0 con 0x7ff7f00cfba0 2024-04-25T09:53:55.160 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:55.161 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:20.100753+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:55.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.160+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/3300480198 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff7d006b4d0 msgr2=0x7ff7d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.160+0000 7ff7f4dcf640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff7d006b4d0 0x7ff7d006d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4034000 tx=0x7ff7e402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:55.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.160+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/3300480198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f00cfba0 msgr2=0x7ff7f00ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.160+0000 7ff7f4dcf640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f00cfba0 0x7ff7f00ca340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800c730 tx=0x7ff7e800cbf0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.160+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/3300480198 shutdown_connections 2024-04-25T09:53:55.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.160+0000 7ff7f4dcf640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff7d006b4d0 0x7ff7d006d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.160+0000 7ff7f4dcf640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f0158240 0x7ff7f01f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.160+0000 7ff7f4dcf640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f00d1860 0x7ff7f00c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.160+0000 7ff7f4dcf640 1 --2- 172.21.15.139:0/3300480198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7f00cfba0 0x7ff7f00ca340 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.160+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/3300480198 >> 172.21.15.139:0/3300480198 conn(0x7ff7f00b3380 msgr2=0x7ff7f015e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:55.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.160+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/3300480198 shutdown_connections 2024-04-25T09:53:55.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.160+0000 7ff7f4dcf640 1 -- 172.21.15.139:0/3300480198 wait complete. 2024-04-25T09:53:55.174 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:55.174 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:55.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.271+0000 7f8b931f9640 1 Processor -- start 2024-04-25T09:53:55.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.271+0000 7f8b931f9640 1 -- start start 2024-04-25T09:53:55.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.272+0000 7f8b931f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c0af8a0 0x7f8b8c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.272+0000 7f8b931f9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c0c6cc0 con 0x7f8b8c0afc80 2024-04-25T09:53:55.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.272+0000 7f8b931f9640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c0c6fa0 con 0x7f8b8c06b700 2024-04-25T09:53:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.272+0000 7f8b931f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0c7280 con 0x7f8b8c0af8a0 2024-04-25T09:53:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.272+0000 7f8b8bfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8b8c0afc80 0x7f8b8c0cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38474/0 (socket says 172.21.15.139:38474) 2024-04-25T09:53:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.272+0000 7f8b8bfff640 1 -- 172.21.15.139:0/1683572439 learned_addr learned my addr 172.21.15.139:0/1683572439 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b90f95640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c0af8a0 0x7f8b8c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252444245 0 0) 0x7f8b8c0c6cc0 con 0x7f8b8c0afc80 2024-04-25T09:53:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b74003520 con 0x7f8b8c0afc80 2024-04-25T09:53:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 712663144 0 0) 0x7f8b74003520 con 0x7f8b8c0afc80 2024-04-25T09:53:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c0af8a0 msgr2=0x7f8b8c0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c0af8a0 0x7f8b8c0b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 >> v1:172.21.15.150:6789/0 conn(0x7f8b8c06b700 legacy=0x7f8b8c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c0cd090 con 0x7f8b8c0afc80 2024-04-25T09:53:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b90f95640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c0af8a0 0x7f8b8c0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f8b80001670 con 0x7f8b8c0afc80 2024-04-25T09:53:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 >> v1:172.21.15.139:6789/0 conn(0x7f8b8c0afc80 legacy=0x7f8b8c0cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c0af8a0 0x7f8b740039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b74003fd0 0x7f8b74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b74004d40 0x7f8b740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0c7280 con 0x7f8b74003fd0 2024-04-25T09:53:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7400bb20 con 0x7f8b74004d40 2024-04-25T09:53:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7400bd20 con 0x7f8b8c0af8a0 2024-04-25T09:53:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8bfff640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b74003fd0 0x7f8b74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b91796640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b74004d40 0x7f8b740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b90f95640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c0af8a0 0x7f8b740039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8bfff640 1 -- 172.21.15.139:0/1683572439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c0af8a0 msgr2=0x7f8b740039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:55.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8bfff640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c0af8a0 0x7f8b740039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8bfff640 1 -- 172.21.15.139:0/1683572439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b74004d40 msgr2=0x7f8b740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:55.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8bfff640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b74004d40 0x7f8b740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.273+0000 7f8b8bfff640 1 -- 172.21.15.139:0/1683572439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b74003520 con 0x7f8b74003fd0 2024-04-25T09:53:55.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.274+0000 7f8b8bfff640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b74003fd0 0x7f8b74004800 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000c240 tx=0x7f8b74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a77b0816626604 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.274+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80007d10 con 0x7f8b74003fd0 2024-04-25T09:53:55.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.274+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b80005ad0 con 0x7f8b74003fd0 2024-04-25T09:53:55.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.274+0000 7f8b8b7fe640 1 -- 172.21.15.139:0/1683572439 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80007070 con 0x7f8b74003fd0 2024-04-25T09:53:55.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.275+0000 7f8b931f9640 1 -- 172.21.15.139:0/1683572439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b74003fd0 msgr2=0x7f8b74004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.275+0000 7f8b931f9640 1 --2- 172.21.15.139:0/1683572439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b74003fd0 0x7f8b74004800 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000c240 tx=0x7f8b74005830 comp rx=0 tx=0).stop 2024-04-25T09:53:55.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.275+0000 7f8b91796640 1 -- 172.21.15.139:0/1683572439 reap_dead start 2024-04-25T09:53:55.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.275+0000 7f8b931f9640 1 -- 172.21.15.139:0/1683572439 shutdown_connections 2024-04-25T09:53:55.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.275+0000 7f8b931f9640 1 -- 172.21.15.139:0/1683572439 >> 172.21.15.139:0/1683572439 conn(0x7f8b8c0b2230 msgr2=0x7f8b8c0b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:55.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.275+0000 7f8b931f9640 1 -- 172.21.15.139:0/1683572439 shutdown_connections 2024-04-25T09:53:55.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.275+0000 7f8b931f9640 1 -- 172.21.15.139:0/1683572439 wait complete. 2024-04-25T09:53:55.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.276+0000 7f8b931f9640 1 Processor -- start 2024-04-25T09:53:55.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.276+0000 7f8b931f9640 1 -- start start 2024-04-25T09:53:55.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b931f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b74003fd0 0x7f8b8c153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b931f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c159100 0x7f8b8c0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b931f9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b8c152340 0x7f8b8c0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b931f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c06b3f0 con 0x7f8b74003fd0 2024-04-25T09:53:55.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b931f9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c06b270 con 0x7f8b8c152340 2024-04-25T09:53:55.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b931f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0b2d50 con 0x7f8b8c159100 2024-04-25T09:53:55.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b90f95640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b74003fd0 0x7f8b8c153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b90f95640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b74003fd0 0x7f8b8c153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55284/0 (socket says 172.21.15.139:55284) 2024-04-25T09:53:55.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b90f95640 1 -- 172.21.15.139:0/2023910919 learned_addr learned my addr 172.21.15.139:0/2023910919 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:55.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b91796640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b8c152340 0x7f8b8c0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b8bfff640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c159100 0x7f8b8c0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b90f95640 1 -- 172.21.15.139:0/2023910919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c159100 msgr2=0x7f8b8c0b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b90f95640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c159100 0x7f8b8c0b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b90f95640 1 -- 172.21.15.139:0/2023910919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b8c152340 msgr2=0x7f8b8c0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b90f95640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b8c152340 0x7f8b8c0b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b90f95640 1 -- 172.21.15.139:0/2023910919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c152fe0 con 0x7f8b74003fd0 2024-04-25T09:53:55.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b91796640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b8c152340 0x7f8b8c0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:55.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b90f95640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b74003fd0 0x7f8b8c153da0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0115f0 tx=0x7f8b7c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b897fa640 1 -- 172.21.15.139:0/2023910919 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c019070 con 0x7f8b74003fd0 2024-04-25T09:53:55.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b897fa640 1 -- 172.21.15.139:0/2023910919 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7c004220 con 0x7f8b74003fd0 2024-04-25T09:53:55.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b931f9640 1 -- 172.21.15.139:0/2023910919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c1532c0 con 0x7f8b74003fd0 2024-04-25T09:53:55.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b897fa640 1 -- 172.21.15.139:0/2023910919 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c00f040 con 0x7f8b74003fd0 2024-04-25T09:53:55.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.277+0000 7f8b931f9640 1 -- 172.21.15.139:0/2023910919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c1527f0 con 0x7f8b74003fd0 2024-04-25T09:53:55.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.278+0000 7f8b931f9640 1 -- 172.21.15.139:0/2023910919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c074520 con 0x7f8b74003fd0 2024-04-25T09:53:55.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.279+0000 7f8b897fa640 1 -- 172.21.15.139:0/2023910919 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b7c00f6f0 con 0x7f8b74003fd0 2024-04-25T09:53:55.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.279+0000 7f8b897fa640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8b7806b500 0x7f8b7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.279+0000 7f8b897fa640 1 -- 172.21.15.139:0/2023910919 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8b7c08a7a0 con 0x7f8b74003fd0 2024-04-25T09:53:55.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.279+0000 7f8b8bfff640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8b7806b500 0x7f8b7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.280+0000 7f8b8bfff640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8b7806b500 0x7f8b7806d9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8b80032000 tx=0x7f8b80009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.283+0000 7f8b897fa640 1 -- 172.21.15.139:0/2023910919 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b7c059370 con 0x7f8b74003fd0 2024-04-25T09:53:55.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.453+0000 7f8b931f9640 1 -- 172.21.15.139:0/2023910919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b8c15b950 con 0x7f8b74003fd0 2024-04-25T09:53:55.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.453+0000 7f8b897fa640 1 -- 172.21.15.139:0/2023910919 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b7c058c70 con 0x7f8b74003fd0 2024-04-25T09:53:55.455 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:53:55.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.455+0000 7f8b931f9640 1 -- 172.21.15.139:0/2023910919 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8b7806b500 msgr2=0x7f8b7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.455+0000 7f8b931f9640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8b7806b500 0x7f8b7806d9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8b80032000 tx=0x7f8b80009030 comp rx=0 tx=0).stop 2024-04-25T09:53:55.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.455+0000 7f8b931f9640 1 -- 172.21.15.139:0/2023910919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b74003fd0 msgr2=0x7f8b8c153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.455+0000 7f8b931f9640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b74003fd0 0x7f8b8c153da0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0115f0 tx=0x7f8b7c011ab0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.455+0000 7f8b931f9640 1 -- 172.21.15.139:0/2023910919 shutdown_connections 2024-04-25T09:53:55.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.455+0000 7f8b931f9640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8b7806b500 0x7f8b7806d9c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.455+0000 7f8b931f9640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b8c152340 0x7f8b8c0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.455+0000 7f8b931f9640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b8c159100 0x7f8b8c0b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.455+0000 7f8b931f9640 1 --2- 172.21.15.139:0/2023910919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b74003fd0 0x7f8b8c153da0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.455+0000 7f8b931f9640 1 -- 172.21.15.139:0/2023910919 >> 172.21.15.139:0/2023910919 conn(0x7f8b8c0b2230 msgr2=0x7f8b8c0adb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:55.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.455+0000 7f8b931f9640 1 -- 172.21.15.139:0/2023910919 shutdown_connections 2024-04-25T09:53:55.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.455+0000 7f8b931f9640 1 -- 172.21.15.139:0/2023910919 wait complete. 2024-04-25T09:53:55.468 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:55.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.566+0000 7ffb2ec76640 1 Processor -- start 2024-04-25T09:53:55.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.566+0000 7ffb2ec76640 1 -- start start 2024-04-25T09:53:55.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb2ec76640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffb280d0a80 0x7ffb281618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb2ec76640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2809ad40 0x7ffb28161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb2ec76640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb28099ca0 con 0x7ffb280cd610 2024-04-25T09:53:55.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb2ec76640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb28099a50 con 0x7ffb280d0a80 2024-04-25T09:53:55.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb2ec76640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28099f80 con 0x7ffb2809ad40 2024-04-25T09:53:55.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb27fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffb280d0a80 0x7ffb281618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb2ca12640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ffb280cd610 0x7ffb280d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38478/0 (socket says 172.21.15.139:38478) 2024-04-25T09:53:55.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb2ca12640 1 -- 172.21.15.139:0/947407194 learned_addr learned my addr 172.21.15.139:0/947407194 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:55.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb2d213640 1 --2- 172.21.15.139:0/947407194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2809ad40 0x7ffb28161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb27fff640 1 -- 172.21.15.139:0/947407194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2809ad40 msgr2=0x7ffb28161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:55.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb27fff640 1 --2- 172.21.15.139:0/947407194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2809ad40 0x7ffb28161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb27fff640 1 -- 172.21.15.139:0/947407194 >> v1:172.21.15.139:6789/0 conn(0x7ffb280cd610 legacy=0x7ffb280d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb27fff640 1 -- 172.21.15.139:0/947407194 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2809a260 con 0x7ffb280d0a80 2024-04-25T09:53:55.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.567+0000 7ffb277fe640 1 -- 172.21.15.139:0/947407194 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560335158 0 0) 0x7ffb28099ca0 con 0x7ffb280cd610 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.568+0000 7ffb27fff640 1 --2- 172.21.15.139:0/947407194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb280d0a80 0x7ffb281618c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffb140097a0 tx=0x7ffb1402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106d44a029b408c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.568+0000 7ffb277fe640 1 -- 172.21.15.139:0/947407194 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14037070 con 0x7ffb280d0a80 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.568+0000 7ffb277fe640 1 -- 172.21.15.139:0/947407194 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb14032440 con 0x7ffb280d0a80 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.568+0000 7ffb277fe640 1 -- 172.21.15.139:0/947407194 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14032810 con 0x7ffb280d0a80 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.568+0000 7ffb2ec76640 1 -- 172.21.15.139:0/947407194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb280d0a80 msgr2=0x7ffb281618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.568+0000 7ffb2ec76640 1 --2- 172.21.15.139:0/947407194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb280d0a80 0x7ffb281618c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffb140097a0 tx=0x7ffb1402a210 comp rx=0 tx=0).stop 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.568+0000 7ffb2ec76640 1 -- 172.21.15.139:0/947407194 shutdown_connections 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.568+0000 7ffb2ec76640 1 --2- 172.21.15.139:0/947407194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb2809ad40 0x7ffb28161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.568+0000 7ffb2ec76640 1 --2- 172.21.15.139:0/947407194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb280d0a80 0x7ffb281618c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.568+0000 7ffb2ec76640 1 -- 172.21.15.139:0/947407194 >> 172.21.15.139:0/947407194 conn(0x7ffb280b31f0 msgr2=0x7ffb280b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.568+0000 7ffb2ec76640 1 -- 172.21.15.139:0/947407194 shutdown_connections 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.568+0000 7ffb2ec76640 1 -- 172.21.15.139:0/947407194 wait complete. 2024-04-25T09:53:55.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.570+0000 7ffb2ec76640 1 Processor -- start 2024-04-25T09:53:55.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.570+0000 7ffb2ec76640 1 -- start start 2024-04-25T09:53:55.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.570+0000 7ffb2ec76640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2809ad40 0x7ffb2815a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.570+0000 7ffb2ec76640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb280cd610 0x7ffb280c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.570+0000 7ffb2ec76640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb280d0a80 0x7ffb280c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.570+0000 7ffb2ec76640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280cd9f0 con 0x7ffb2809ad40 2024-04-25T09:53:55.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.570+0000 7ffb2ec76640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2809a820 con 0x7ffb280d0a80 2024-04-25T09:53:55.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.570+0000 7ffb2ec76640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280cdb70 con 0x7ffb280cd610 2024-04-25T09:53:55.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb2ca12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2809ad40 0x7ffb2815a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb2ca12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2809ad40 0x7ffb2815a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55290/0 (socket says 172.21.15.139:55290) 2024-04-25T09:53:55.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb2ca12640 1 -- 172.21.15.139:0/1633384238 learned_addr learned my addr 172.21.15.139:0/1633384238 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:55.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb27fff640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb280cd610 0x7ffb280c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb2d213640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb280d0a80 0x7ffb280c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb2ca12640 1 -- 172.21.15.139:0/1633384238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb280cd610 msgr2=0x7ffb280c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb2ca12640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb280cd610 0x7ffb280c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb2ca12640 1 -- 172.21.15.139:0/1633384238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb280d0a80 msgr2=0x7ffb280c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb2ca12640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb280d0a80 0x7ffb280c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb2ca12640 1 -- 172.21.15.139:0/1633384238 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb280c5da0 con 0x7ffb2809ad40 2024-04-25T09:53:55.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb2ca12640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2809ad40 0x7ffb2815a7c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00c500 tx=0x7ffb1c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb257fa640 1 -- 172.21.15.139:0/1633384238 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c018070 con 0x7ffb2809ad40 2024-04-25T09:53:55.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb257fa640 1 -- 172.21.15.139:0/1633384238 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c007580 con 0x7ffb2809ad40 2024-04-25T09:53:55.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb2ec76640 1 -- 172.21.15.139:0/1633384238 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb280c6080 con 0x7ffb2809ad40 2024-04-25T09:53:55.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb257fa640 1 -- 172.21.15.139:0/1633384238 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c004a20 con 0x7ffb2809ad40 2024-04-25T09:53:55.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.571+0000 7ffb2ec76640 1 -- 172.21.15.139:0/1633384238 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb280c6640 con 0x7ffb2809ad40 2024-04-25T09:53:55.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.572+0000 7ffb2ec76640 1 -- 172.21.15.139:0/1633384238 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb28074520 con 0x7ffb2809ad40 2024-04-25T09:53:55.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.573+0000 7ffb257fa640 1 -- 172.21.15.139:0/1633384238 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb1c01b070 con 0x7ffb2809ad40 2024-04-25T09:53:55.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.573+0000 7ffb257fa640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffb0c06b4d0 0x7ffb0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.573+0000 7ffb257fa640 1 -- 172.21.15.139:0/1633384238 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffb1c0897e0 con 0x7ffb2809ad40 2024-04-25T09:53:55.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.573+0000 7ffb27fff640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffb0c06b4d0 0x7ffb0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.573+0000 7ffb27fff640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffb0c06b4d0 0x7ffb0c06d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb14034000 tx=0x7ffb1402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.577+0000 7ffb257fa640 1 -- 172.21.15.139:0/1633384238 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb1c0583b0 con 0x7ffb2809ad40 2024-04-25T09:53:55.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.751+0000 7ffb2ec76640 1 -- 172.21.15.139:0/1633384238 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb280b3f70 con 0x7ffb2809ad40 2024-04-25T09:53:55.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.751+0000 7ffb257fa640 1 -- 172.21.15.139:0/1633384238 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb1c057cb0 con 0x7ffb2809ad40 2024-04-25T09:53:55.754 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:55.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.753+0000 7ffb2ec76640 1 -- 172.21.15.139:0/1633384238 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffb0c06b4d0 msgr2=0x7ffb0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.753+0000 7ffb2ec76640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffb0c06b4d0 0x7ffb0c06d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb14034000 tx=0x7ffb1402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:55.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.753+0000 7ffb2ec76640 1 -- 172.21.15.139:0/1633384238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2809ad40 msgr2=0x7ffb2815a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.753+0000 7ffb2ec76640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2809ad40 0x7ffb2815a7c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00c500 tx=0x7ffb1c00c9c0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.753+0000 7ffb2ec76640 1 -- 172.21.15.139:0/1633384238 shutdown_connections 2024-04-25T09:53:55.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.753+0000 7ffb2ec76640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffb0c06b4d0 0x7ffb0c06d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.753+0000 7ffb2ec76640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb280d0a80 0x7ffb280c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.753+0000 7ffb2ec76640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb280cd610 0x7ffb280c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.753+0000 7ffb2ec76640 1 --2- 172.21.15.139:0/1633384238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb2809ad40 0x7ffb2815a7c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.753+0000 7ffb2ec76640 1 -- 172.21.15.139:0/1633384238 >> 172.21.15.139:0/1633384238 conn(0x7ffb280b31f0 msgr2=0x7ffb280ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:55.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.754+0000 7ffb2ec76640 1 -- 172.21.15.139:0/1633384238 shutdown_connections 2024-04-25T09:53:55.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.754+0000 7ffb2ec76640 1 -- 172.21.15.139:0/1633384238 wait complete. 2024-04-25T09:53:55.768 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:55.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.873+0000 7fc121e88640 1 Processor -- start 2024-04-25T09:53:55.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.873+0000 7fc121e88640 1 -- start start 2024-04-25T09:53:55.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc121e88640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc11c144000 0x7fc11c06f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc121e88640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc11c0b2d70 0x7fc11c071620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc121e88640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c0b33b0 con 0x7fc11c0711c0 2024-04-25T09:53:55.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc121e88640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c071b60 con 0x7fc11c144000 2024-04-25T09:53:55.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc121e88640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c1443e0 con 0x7fc11c0b2d70 2024-04-25T09:53:55.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc11b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc11c144000 0x7fc11c06f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc11affd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc11c0711c0 0x7fc11c0b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38480/0 (socket says 172.21.15.139:38480) 2024-04-25T09:53:55.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc11affd640 1 -- 172.21.15.139:0/2361441880 learned_addr learned my addr 172.21.15.139:0/2361441880 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:55.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc11bfff640 1 --2- 172.21.15.139:0/2361441880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc11c0b2d70 0x7fc11c071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc11b7fe640 1 -- 172.21.15.139:0/2361441880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc11c0b2d70 msgr2=0x7fc11c071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:55.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc11b7fe640 1 --2- 172.21.15.139:0/2361441880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc11c0b2d70 0x7fc11c071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc11b7fe640 1 -- 172.21.15.139:0/2361441880 >> v1:172.21.15.139:6789/0 conn(0x7fc11c0711c0 legacy=0x7fc11c0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc11b7fe640 1 -- 172.21.15.139:0/2361441880 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c06e690 con 0x7fc11c144000 2024-04-25T09:53:55.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.874+0000 7fc11a7fc640 1 -- 172.21.15.139:0/2361441880 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107073696 0 0) 0x7fc11c0b33b0 con 0x7fc11c0711c0 2024-04-25T09:53:55.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.875+0000 7fc11b7fe640 1 --2- 172.21.15.139:0/2361441880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11c144000 0x7fc11c06f170 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc1040097a0 tx=0x7fc10402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09b78eabbd44957 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.875+0000 7fc11a7fc640 1 -- 172.21.15.139:0/2361441880 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104037070 con 0x7fc11c144000 2024-04-25T09:53:55.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.875+0000 7fc11a7fc640 1 -- 172.21.15.139:0/2361441880 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc104032440 con 0x7fc11c144000 2024-04-25T09:53:55.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.875+0000 7fc11a7fc640 1 -- 172.21.15.139:0/2361441880 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104032810 con 0x7fc11c144000 2024-04-25T09:53:55.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.875+0000 7fc121e88640 1 -- 172.21.15.139:0/2361441880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11c144000 msgr2=0x7fc11c06f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.875+0000 7fc121e88640 1 --2- 172.21.15.139:0/2361441880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11c144000 0x7fc11c06f170 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc1040097a0 tx=0x7fc10402a180 comp rx=0 tx=0).stop 2024-04-25T09:53:55.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.876+0000 7fc121e88640 1 -- 172.21.15.139:0/2361441880 shutdown_connections 2024-04-25T09:53:55.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.876+0000 7fc121e88640 1 --2- 172.21.15.139:0/2361441880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc11c0b2d70 0x7fc11c071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.876+0000 7fc121e88640 1 --2- 172.21.15.139:0/2361441880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11c144000 0x7fc11c06f170 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.876+0000 7fc121e88640 1 -- 172.21.15.139:0/2361441880 >> 172.21.15.139:0/2361441880 conn(0x7fc11c067000 msgr2=0x7fc11c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:55.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.876+0000 7fc121e88640 1 -- 172.21.15.139:0/2361441880 shutdown_connections 2024-04-25T09:53:55.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.876+0000 7fc121e88640 1 -- 172.21.15.139:0/2361441880 wait complete. 2024-04-25T09:53:55.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.877+0000 7fc121e88640 1 Processor -- start 2024-04-25T09:53:55.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.877+0000 7fc121e88640 1 -- start start 2024-04-25T09:53:55.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc121e88640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc11c0711c0 0x7fc11c15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc121e88640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11c0b2d70 0x7fc11c145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc121e88640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc11c144000 0x7fc11c146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc121e88640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c06ec50 con 0x7fc11c0b2d70 2024-04-25T09:53:55.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc121e88640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c06e440 con 0x7fc11c144000 2024-04-25T09:53:55.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc121e88640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0add50 con 0x7fc11c0711c0 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11c0b2d70 0x7fc11c145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11c0b2d70 0x7fc11c145bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55306/0 (socket says 172.21.15.139:55306) 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11affd640 1 -- 172.21.15.139:0/1557979331 learned_addr learned my addr 172.21.15.139:0/1557979331 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11b7fe640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc11c0711c0 0x7fc11c15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11bfff640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc11c144000 0x7fc11c146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11affd640 1 -- 172.21.15.139:0/1557979331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc11c0711c0 msgr2=0x7fc11c15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11affd640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc11c0711c0 0x7fc11c15d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11affd640 1 -- 172.21.15.139:0/1557979331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc11c144000 msgr2=0x7fc11c146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11affd640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc11c144000 0x7fc11c146130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11affd640 1 -- 172.21.15.139:0/1557979331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c160790 con 0x7fc11c0b2d70 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11bfff640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc11c144000 0x7fc11c146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11b7fe640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc11c0711c0 0x7fc11c15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc11affd640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11c0b2d70 0x7fc11c145bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c4d0 tx=0x7fc10c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc113fff640 1 -- 172.21.15.139:0/1557979331 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c018070 con 0x7fc11c0b2d70 2024-04-25T09:53:55.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc113fff640 1 -- 172.21.15.139:0/1557979331 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c007500 con 0x7fc11c0b2d70 2024-04-25T09:53:55.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc121e88640 1 -- 172.21.15.139:0/1557979331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c0ced50 con 0x7fc11c0b2d70 2024-04-25T09:53:55.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc113fff640 1 -- 172.21.15.139:0/1557979331 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c004a20 con 0x7fc11c0b2d70 2024-04-25T09:53:55.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.878+0000 7fc121e88640 1 -- 172.21.15.139:0/1557979331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c1f6b40 con 0x7fc11c0b2d70 2024-04-25T09:53:55.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.879+0000 7fc121e88640 1 -- 172.21.15.139:0/1557979331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c074520 con 0x7fc11c0b2d70 2024-04-25T09:53:55.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.880+0000 7fc113fff640 1 -- 172.21.15.139:0/1557979331 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc10c01b070 con 0x7fc11c0b2d70 2024-04-25T09:53:55.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.880+0000 7fc113fff640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:55.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.880+0000 7fc11b7fe640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:55.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.880+0000 7fc113fff640 1 -- 172.21.15.139:0/1557979331 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc10c089920 con 0x7fc11c0b2d70 2024-04-25T09:53:55.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.880+0000 7fc11b7fe640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc104034000 tx=0x7fc10402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:55.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:55.884+0000 7fc113fff640 1 -- 172.21.15.139:0/1557979331 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc11c160790 con 0x7fc11c0b2d70 2024-04-25T09:53:56.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.057+0000 7fc121e88640 1 -- 172.21.15.139:0/1557979331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc11c06c1a0 con 0x7fc11c0b2d70 2024-04-25T09:53:56.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.058+0000 7fc113fff640 1 -- 172.21.15.139:0/1557979331 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc10c0583a0 con 0x7fc11c0b2d70 2024-04-25T09:53:56.060 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:53:56.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.059+0000 7fc121e88640 1 -- 172.21.15.139:0/1557979331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0fc06b4d0 msgr2=0x7fc0fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.059+0000 7fc121e88640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc104034000 tx=0x7fc10402e000 comp rx=0 tx=0).stop 2024-04-25T09:53:56.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.059+0000 7fc121e88640 1 -- 172.21.15.139:0/1557979331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11c0b2d70 msgr2=0x7fc11c145bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.059+0000 7fc121e88640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11c0b2d70 0x7fc11c145bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c4d0 tx=0x7fc10c00c990 comp rx=0 tx=0).stop 2024-04-25T09:53:56.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.059+0000 7fc121e88640 1 -- 172.21.15.139:0/1557979331 shutdown_connections 2024-04-25T09:53:56.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.059+0000 7fc121e88640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc11c0b2d70 0x7fc11c145bf0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.059+0000 7fc121e88640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.059+0000 7fc121e88640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc11c144000 0x7fc11c146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.059+0000 7fc121e88640 1 --2- 172.21.15.139:0/1557979331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc11c0711c0 0x7fc11c15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.059+0000 7fc121e88640 1 -- 172.21.15.139:0/1557979331 >> 172.21.15.139:0/1557979331 conn(0x7fc11c067000 msgr2=0x7fc11c0b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:56.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.060+0000 7fc121e88640 1 -- 172.21.15.139:0/1557979331 shutdown_connections 2024-04-25T09:53:56.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.060+0000 7fc121e88640 1 -- 172.21.15.139:0/1557979331 wait complete. 2024-04-25T09:53:56.073 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:56.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.171+0000 7fed4fb30640 1 Processor -- start 2024-04-25T09:53:56.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.171+0000 7fed4fb30640 1 -- start start 2024-04-25T09:53:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.172+0000 7fed4fb30640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fed480cd610 0x7fed480d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.172+0000 7fed4fb30640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fed48161fd0 0x7fed480d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.172+0000 7fed4fb30640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed48099ca0 con 0x7fed4809ad40 2024-04-25T09:53:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.172+0000 7fed4fb30640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed48099a50 con 0x7fed480cd610 2024-04-25T09:53:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.172+0000 7fed4fb30640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed48099f80 con 0x7fed48161fd0 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.172+0000 7fed4d8cc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fed480cd610 0x7fed480d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.172+0000 7fed4d8cc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fed480cd610 0x7fed480d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55314/0 (socket says 172.21.15.139:55314) 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.172+0000 7fed4d8cc640 1 -- 172.21.15.139:0/272202559 learned_addr learned my addr 172.21.15.139:0/272202559 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.172+0000 7fed4e0cd640 1 --2- 172.21.15.139:0/272202559 >> v2:172.21.15.150:3300/0 conn(0x7fed48161fd0 0x7fed480d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.172+0000 7fed4d8cc640 1 -- 172.21.15.139:0/272202559 >> v2:172.21.15.150:3300/0 conn(0x7fed48161fd0 msgr2=0x7fed480d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.172+0000 7fed4d8cc640 1 --2- 172.21.15.139:0/272202559 >> v2:172.21.15.150:3300/0 conn(0x7fed48161fd0 0x7fed480d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.172+0000 7fed4d8cc640 1 -- 172.21.15.139:0/272202559 >> v1:172.21.15.150:6789/0 conn(0x7fed4809ad40 legacy=0x7fed481618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.173+0000 7fed4d8cc640 1 -- 172.21.15.139:0/272202559 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4809a260 con 0x7fed480cd610 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.173+0000 7fed4d8cc640 1 --2- 172.21.15.139:0/272202559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed480cd610 0x7fed480d0370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fed480b42d0 tx=0x7fed4002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f31b90eb64f748 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.173+0000 7fed4c8ca640 1 -- 172.21.15.139:0/272202559 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40037070 con 0x7fed480cd610 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.173+0000 7fed4c8ca640 1 -- 172.21.15.139:0/272202559 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4002ae50 con 0x7fed480cd610 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.173+0000 7fed4c8ca640 1 -- 172.21.15.139:0/272202559 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40032860 con 0x7fed480cd610 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.174+0000 7fed4fb30640 1 -- 172.21.15.139:0/272202559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed480cd610 msgr2=0x7fed480d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.174+0000 7fed4fb30640 1 --2- 172.21.15.139:0/272202559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed480cd610 0x7fed480d0370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fed480b42d0 tx=0x7fed4002a300 comp rx=0 tx=0).stop 2024-04-25T09:53:56.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.174+0000 7fed4fb30640 1 -- 172.21.15.139:0/272202559 shutdown_connections 2024-04-25T09:53:56.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.174+0000 7fed4fb30640 1 --2- 172.21.15.139:0/272202559 >> v2:172.21.15.150:3300/0 conn(0x7fed48161fd0 0x7fed480d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.174+0000 7fed4fb30640 1 --2- 172.21.15.139:0/272202559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed480cd610 0x7fed480d0370 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.174+0000 7fed4fb30640 1 -- 172.21.15.139:0/272202559 >> 172.21.15.139:0/272202559 conn(0x7fed480b31f0 msgr2=0x7fed480b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:56.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.174+0000 7fed4fb30640 1 -- 172.21.15.139:0/272202559 shutdown_connections 2024-04-25T09:53:56.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.174+0000 7fed4fb30640 1 -- 172.21.15.139:0/272202559 wait complete. 2024-04-25T09:53:56.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.175+0000 7fed4fb30640 1 Processor -- start 2024-04-25T09:53:56.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.175+0000 7fed4fb30640 1 -- start start 2024-04-25T09:53:56.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4fb30640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed4809ad40 0x7fed4814dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4fb30640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed480cd610 0x7fed481e8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4fb30640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed48161fd0 0x7fed480af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4fb30640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed480cd9f0 con 0x7fed48161fd0 2024-04-25T09:53:56.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4fb30640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4809a820 con 0x7fed480cd610 2024-04-25T09:53:56.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4fb30640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed480cdb70 con 0x7fed4809ad40 2024-04-25T09:53:56.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4e0cd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed48161fd0 0x7fed480af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4e0cd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed48161fd0 0x7fed480af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55320/0 (socket says 172.21.15.139:55320) 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4e0cd640 1 -- 172.21.15.139:0/1111281842 learned_addr learned my addr 172.21.15.139:0/1111281842 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4e0cd640 1 -- 172.21.15.139:0/1111281842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed4809ad40 msgr2=0x7fed4814dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4e0cd640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed4809ad40 0x7fed4814dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4e0cd640 1 -- 172.21.15.139:0/1111281842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed480cd610 msgr2=0x7fed481e8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4d0cb640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed480cd610 0x7fed481e8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4e0cd640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed480cd610 0x7fed481e8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4e0cd640 1 -- 172.21.15.139:0/1111281842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed480c53d0 con 0x7fed48161fd0 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed4e0cd640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed48161fd0 0x7fed480af920 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fed44009fd0 tx=0x7fed4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed3e7fc640 1 -- 172.21.15.139:0/1111281842 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44014420 con 0x7fed48161fd0 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.176+0000 7fed3e7fc640 1 -- 172.21.15.139:0/1111281842 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed44019070 con 0x7fed48161fd0 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.177+0000 7fed4fb30640 1 -- 172.21.15.139:0/1111281842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed480c56b0 con 0x7fed48161fd0 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.177+0000 7fed3e7fc640 1 -- 172.21.15.139:0/1111281842 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44014800 con 0x7fed48161fd0 2024-04-25T09:53:56.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.177+0000 7fed4fb30640 1 -- 172.21.15.139:0/1111281842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed480c5c70 con 0x7fed48161fd0 2024-04-25T09:53:56.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.178+0000 7fed4fb30640 1 -- 172.21.15.139:0/1111281842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed48074520 con 0x7fed48161fd0 2024-04-25T09:53:56.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.178+0000 7fed3e7fc640 1 -- 172.21.15.139:0/1111281842 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed440149a0 con 0x7fed48161fd0 2024-04-25T09:53:56.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.178+0000 7fed3e7fc640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed2c06b500 0x7fed2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.178+0000 7fed3e7fc640 1 -- 172.21.15.139:0/1111281842 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fed4408aee0 con 0x7fed48161fd0 2024-04-25T09:53:56.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.178+0000 7fed4d8cc640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed2c06b500 0x7fed2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.179+0000 7fed4d8cc640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed2c06b500 0x7fed2c06d9c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fed40034000 tx=0x7fed4002c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.182+0000 7fed3e7fc640 1 -- 172.21.15.139:0/1111281842 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed440598d0 con 0x7fed48161fd0 2024-04-25T09:53:56.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.353+0000 7fed4fb30640 1 -- 172.21.15.139:0/1111281842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fed480c8560 con 0x7fed48161fd0 2024-04-25T09:53:56.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.354+0000 7fed3e7fc640 1 -- 172.21.15.139:0/1111281842 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fed440591d0 con 0x7fed48161fd0 2024-04-25T09:53:56.356 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:56.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.356+0000 7fed4fb30640 1 -- 172.21.15.139:0/1111281842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed2c06b500 msgr2=0x7fed2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.356+0000 7fed4fb30640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed2c06b500 0x7fed2c06d9c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fed40034000 tx=0x7fed4002c000 comp rx=0 tx=0).stop 2024-04-25T09:53:56.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.356+0000 7fed4fb30640 1 -- 172.21.15.139:0/1111281842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed48161fd0 msgr2=0x7fed480af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.356+0000 7fed4fb30640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed48161fd0 0x7fed480af920 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fed44009fd0 tx=0x7fed4400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.356+0000 7fed4fb30640 1 -- 172.21.15.139:0/1111281842 shutdown_connections 2024-04-25T09:53:56.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.356+0000 7fed4fb30640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed48161fd0 0x7fed480af920 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.356+0000 7fed4fb30640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed2c06b500 0x7fed2c06d9c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.356+0000 7fed4fb30640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed480cd610 0x7fed481e8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.356+0000 7fed4fb30640 1 --2- 172.21.15.139:0/1111281842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed4809ad40 0x7fed4814dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.356+0000 7fed4fb30640 1 -- 172.21.15.139:0/1111281842 >> 172.21.15.139:0/1111281842 conn(0x7fed480b31f0 msgr2=0x7fed481618a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:56.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.356+0000 7fed4fb30640 1 -- 172.21.15.139:0/1111281842 shutdown_connections 2024-04-25T09:53:56.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.356+0000 7fed4fb30640 1 -- 172.21.15.139:0/1111281842 wait complete. 2024-04-25T09:53:56.379 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T09:53:56.380 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:56.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.474+0000 7f9d6f67c640 1 Processor -- start 2024-04-25T09:53:56.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.475+0000 7f9d6f67c640 1 -- start start 2024-04-25T09:53:56.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.475+0000 7f9d6f67c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9d6806c1f0 0x7f9d6815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.475+0000 7f9d6f67c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6815c9c0 0x7f9d680ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.475+0000 7f9d6f67c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9d680ad570 0x7f9d6806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.475+0000 7f9d6f67c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d680af870 con 0x7f9d680ad570 2024-04-25T09:53:56.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.475+0000 7f9d6f67c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6815c530 con 0x7f9d6806c1f0 2024-04-25T09:53:56.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.475+0000 7f9d6f67c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d68160280 con 0x7f9d6815c9c0 2024-04-25T09:53:56.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.476+0000 7f9d6dc19640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9d680ad570 0x7f9d6806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.476+0000 7f9d6dc19640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9d680ad570 0x7f9d6806ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47106/0 (socket says 172.21.15.139:47106) 2024-04-25T09:53:56.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.476+0000 7f9d6dc19640 1 -- 172.21.15.139:0/2858390926 learned_addr learned my addr 172.21.15.139:0/2858390926 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:56.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.476+0000 7f9d6d418640 1 --2- 172.21.15.139:0/2858390926 >> v2:172.21.15.150:3300/0 conn(0x7f9d6806c1f0 0x7f9d6815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.476+0000 7f9d6cc17640 1 --2- 172.21.15.139:0/2858390926 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6815c9c0 0x7f9d680ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.476+0000 7f9d6dc19640 1 -- 172.21.15.139:0/2858390926 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6815c9c0 msgr2=0x7f9d680ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.476+0000 7f9d6dc19640 1 --2- 172.21.15.139:0/2858390926 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6815c9c0 0x7f9d680ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.476+0000 7f9d6dc19640 1 -- 172.21.15.139:0/2858390926 >> v2:172.21.15.150:3300/0 conn(0x7f9d6806c1f0 msgr2=0x7f9d6815fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.476+0000 7f9d6dc19640 1 --2- 172.21.15.139:0/2858390926 >> v2:172.21.15.150:3300/0 conn(0x7f9d6806c1f0 0x7f9d6815fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.476+0000 7f9d6dc19640 1 -- 172.21.15.139:0/2858390926 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d680af590 con 0x7f9d680ad570 2024-04-25T09:53:56.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.477+0000 7f9d6dc19640 1 --2- 172.21.15.139:0/2858390926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680ad570 0x7f9d6806ab60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9d680b51d0 tx=0x7f9d6402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3157a6001d7e8c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.477+0000 7f9d5ffff640 1 -- 172.21.15.139:0/2858390926 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64037070 con 0x7f9d680ad570 2024-04-25T09:53:56.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.477+0000 7f9d5ffff640 1 -- 172.21.15.139:0/2858390926 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d64032440 con 0x7f9d680ad570 2024-04-25T09:53:56.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.477+0000 7f9d5ffff640 1 -- 172.21.15.139:0/2858390926 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64032810 con 0x7f9d680ad570 2024-04-25T09:53:56.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.477+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2858390926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680ad570 msgr2=0x7f9d6806ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.477+0000 7f9d6f67c640 1 --2- 172.21.15.139:0/2858390926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680ad570 0x7f9d6806ab60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9d680b51d0 tx=0x7f9d6402a140 comp rx=0 tx=0).stop 2024-04-25T09:53:56.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.477+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2858390926 shutdown_connections 2024-04-25T09:53:56.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.477+0000 7f9d6f67c640 1 --2- 172.21.15.139:0/2858390926 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680ad570 0x7f9d6806ab60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.477+0000 7f9d6f67c640 1 --2- 172.21.15.139:0/2858390926 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6815c9c0 0x7f9d680ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.477+0000 7f9d6f67c640 1 --2- 172.21.15.139:0/2858390926 >> v2:172.21.15.150:3300/0 conn(0x7f9d6806c1f0 0x7f9d6815fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.477+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2858390926 >> 172.21.15.139:0/2858390926 conn(0x7f9d680b23d0 msgr2=0x7f9d680cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:56.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.478+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2858390926 shutdown_connections 2024-04-25T09:53:56.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.478+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2858390926 wait complete. 2024-04-25T09:53:56.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.479+0000 7f9d6f67c640 1 Processor -- start 2024-04-25T09:53:56.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.479+0000 7f9d6f67c640 1 -- start start 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.479+0000 7f9d6f67c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6806c1f0 0x7f9d680cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.479+0000 7f9d6f67c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680ad570 0x7f9d680ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.479+0000 7f9d6f67c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d6815c9c0 0x7f9d680ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.479+0000 7f9d6f67c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6806f630 con 0x7f9d680ad570 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.479+0000 7f9d6f67c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6815c1c0 con 0x7f9d6815c9c0 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.479+0000 7f9d6f67c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6806f7b0 con 0x7f9d6806c1f0 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.479+0000 7f9d6cc17640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680ad570 0x7f9d680ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.479+0000 7f9d6cc17640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680ad570 0x7f9d680ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47108/0 (socket says 172.21.15.139:47108) 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.479+0000 7f9d6cc17640 1 -- 172.21.15.139:0/2192827483 learned_addr learned my addr 172.21.15.139:0/2192827483 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d6dc19640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d6815c9c0 0x7f9d680ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d6cc17640 1 -- 172.21.15.139:0/2192827483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6806c1f0 msgr2=0x7f9d680cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d6d418640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6806c1f0 0x7f9d680cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d6cc17640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6806c1f0 0x7f9d680cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d6cc17640 1 -- 172.21.15.139:0/2192827483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d6815c9c0 msgr2=0x7f9d680ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d6cc17640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d6815c9c0 0x7f9d680ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d6cc17640 1 -- 172.21.15.139:0/2192827483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d681e5ec0 con 0x7f9d680ad570 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d6dc19640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d6815c9c0 0x7f9d680ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d6d418640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6806c1f0 0x7f9d680cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d6cc17640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680ad570 0x7f9d680ce0f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800e8a0 tx=0x7f9d5800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d5dffb640 1 -- 172.21.15.139:0/2192827483 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58017070 con 0x7f9d680ad570 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d5dffb640 1 -- 172.21.15.139:0/2192827483 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5800c940 con 0x7f9d680ad570 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2192827483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d681e61a0 con 0x7f9d680ad570 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d5dffb640 1 -- 172.21.15.139:0/2192827483 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5800cd30 con 0x7f9d680ad570 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.480+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2192827483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d681e6760 con 0x7f9d680ad570 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.481+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2192827483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d68074520 con 0x7f9d680ad570 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.482+0000 7f9d5dffb640 1 -- 172.21.15.139:0/2192827483 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d5801a070 con 0x7f9d680ad570 2024-04-25T09:53:56.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.482+0000 7f9d5dffb640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9d4c06b4d0 0x7f9d4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.482+0000 7f9d5dffb640 1 -- 172.21.15.139:0/2192827483 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9d5808a830 con 0x7f9d680ad570 2024-04-25T09:53:56.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.482+0000 7f9d6d418640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9d4c06b4d0 0x7f9d4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.482+0000 7f9d6d418640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9d4c06b4d0 0x7f9d4c06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9d60007520 tx=0x7f9d6000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.486+0000 7f9d5dffb640 1 -- 172.21.15.139:0/2192827483 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d580592b0 con 0x7f9d680ad570 2024-04-25T09:53:56.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.657+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2192827483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d6806b6e0 con 0x7f9d680ad570 2024-04-25T09:53:56.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.659+0000 7f9d5dffb640 1 -- 172.21.15.139:0/2192827483 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18951 (secure 0 0 0) 0x7f9d58058bb0 con 0x7f9d680ad570 2024-04-25T09:53:56.661 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:56.661 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":25,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:55.835346+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:56.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.661+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2192827483 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9d4c06b4d0 msgr2=0x7f9d4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.661+0000 7f9d6f67c640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9d4c06b4d0 0x7f9d4c06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9d60007520 tx=0x7f9d6000b040 comp rx=0 tx=0).stop 2024-04-25T09:53:56.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.661+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2192827483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680ad570 msgr2=0x7f9d680ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.661+0000 7f9d6f67c640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680ad570 0x7f9d680ce0f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800e8a0 tx=0x7f9d5800ed60 comp rx=0 tx=0).stop 2024-04-25T09:53:56.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.661+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2192827483 shutdown_connections 2024-04-25T09:53:56.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.661+0000 7f9d6f67c640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680ad570 0x7f9d680ce0f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.661+0000 7f9d6f67c640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9d4c06b4d0 0x7f9d4c06d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.661+0000 7f9d6f67c640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d6815c9c0 0x7f9d680ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.661+0000 7f9d6f67c640 1 --2- 172.21.15.139:0/2192827483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6806c1f0 0x7f9d680cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.661+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2192827483 >> 172.21.15.139:0/2192827483 conn(0x7f9d680b23d0 msgr2=0x7f9d681542e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:56.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.661+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2192827483 shutdown_connections 2024-04-25T09:53:56.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.661+0000 7f9d6f67c640 1 -- 172.21.15.139:0/2192827483 wait complete. 2024-04-25T09:53:56.674 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:56.675 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:56.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.772+0000 7f272f742640 1 Processor -- start 2024-04-25T09:53:56.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.772+0000 7f272f742640 1 -- start start 2024-04-25T09:53:56.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.773+0000 7f272f742640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f27280af8a0 0x7f27280b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.773+0000 7f272f742640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f27280afc80 0x7f27280cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.773+0000 7f272f742640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f272806b700 0x7f272806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.773+0000 7f272f742640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27280c6cc0 con 0x7f27280afc80 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.773+0000 7f272f742640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27280c6fa0 con 0x7f27280af8a0 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.773+0000 7f272f742640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27280c7280 con 0x7f272806b700 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.773+0000 7f272ccdd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f27280afc80 0x7f27280cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.773+0000 7f272ccdd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f27280afc80 0x7f27280cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47120/0 (socket says 172.21.15.139:47120) 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.773+0000 7f272ccdd640 1 -- 172.21.15.139:0/1222184546 learned_addr learned my addr 172.21.15.139:0/1222184546 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.773+0000 7f272dcdf640 1 --2- 172.21.15.139:0/1222184546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f272806b700 0x7f272806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.773+0000 7f272ccdd640 1 -- 172.21.15.139:0/1222184546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f272806b700 msgr2=0x7f272806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.774+0000 7f272d4de640 1 --2- 172.21.15.139:0/1222184546 >> v2:172.21.15.150:3300/0 conn(0x7f27280af8a0 0x7f27280b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.774+0000 7f272ccdd640 1 --2- 172.21.15.139:0/1222184546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f272806b700 0x7f272806ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.774+0000 7f272ccdd640 1 -- 172.21.15.139:0/1222184546 >> v2:172.21.15.150:3300/0 conn(0x7f27280af8a0 msgr2=0x7f27280b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.774+0000 7f272ccdd640 1 --2- 172.21.15.139:0/1222184546 >> v2:172.21.15.150:3300/0 conn(0x7f27280af8a0 0x7f27280b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.774+0000 7f272ccdd640 1 -- 172.21.15.139:0/1222184546 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27280cd090 con 0x7f27280afc80 2024-04-25T09:53:56.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.774+0000 7f272ccdd640 1 --2- 172.21.15.139:0/1222184546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27280afc80 0x7f27280cd890 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2728070aa0 tx=0x7f271802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b7b34efbddfb10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.774+0000 7f271ffff640 1 -- 172.21.15.139:0/1222184546 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718037070 con 0x7f27280afc80 2024-04-25T09:53:56.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.774+0000 7f271ffff640 1 -- 172.21.15.139:0/1222184546 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2718032440 con 0x7f27280afc80 2024-04-25T09:53:56.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.774+0000 7f271ffff640 1 -- 172.21.15.139:0/1222184546 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718032810 con 0x7f27280afc80 2024-04-25T09:53:56.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.775+0000 7f272f742640 1 -- 172.21.15.139:0/1222184546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27280afc80 msgr2=0x7f27280cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.775+0000 7f272f742640 1 --2- 172.21.15.139:0/1222184546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27280afc80 0x7f27280cd890 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2728070aa0 tx=0x7f271802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.775+0000 7f272f742640 1 -- 172.21.15.139:0/1222184546 shutdown_connections 2024-04-25T09:53:56.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.775+0000 7f272f742640 1 --2- 172.21.15.139:0/1222184546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f272806b700 0x7f272806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.775+0000 7f272f742640 1 --2- 172.21.15.139:0/1222184546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27280afc80 0x7f27280cd890 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.775+0000 7f272f742640 1 --2- 172.21.15.139:0/1222184546 >> v2:172.21.15.150:3300/0 conn(0x7f27280af8a0 0x7f27280b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.775+0000 7f272f742640 1 -- 172.21.15.139:0/1222184546 >> 172.21.15.139:0/1222184546 conn(0x7f27280b2230 msgr2=0x7f27280b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:56.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.775+0000 7f272f742640 1 -- 172.21.15.139:0/1222184546 shutdown_connections 2024-04-25T09:53:56.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.775+0000 7f272f742640 1 -- 172.21.15.139:0/1222184546 wait complete. 2024-04-25T09:53:56.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.776+0000 7f272f742640 1 Processor -- start 2024-04-25T09:53:56.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.776+0000 7f272f742640 1 -- start start 2024-04-25T09:53:56.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272f742640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f272806b700 0x7f27280b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272f742640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27280af8a0 0x7f27280b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272f742640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27280afc80 0x7f27280b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272f742640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272806b0a0 con 0x7f272806b700 2024-04-25T09:53:56.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272f742640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27280cddd0 con 0x7f27280af8a0 2024-04-25T09:53:56.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272f742640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272806b220 con 0x7f27280afc80 2024-04-25T09:53:56.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272d4de640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f272806b700 0x7f27280b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272d4de640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f272806b700 0x7f27280b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47134/0 (socket says 172.21.15.139:47134) 2024-04-25T09:53:56.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272d4de640 1 -- 172.21.15.139:0/1850427624 learned_addr learned my addr 172.21.15.139:0/1850427624 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:56.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272d4de640 1 -- 172.21.15.139:0/1850427624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27280afc80 msgr2=0x7f27280b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272d4de640 1 --2- 172.21.15.139:0/1850427624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27280afc80 0x7f27280b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272d4de640 1 -- 172.21.15.139:0/1850427624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27280af8a0 msgr2=0x7f27280b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272d4de640 1 --2- 172.21.15.139:0/1850427624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27280af8a0 0x7f27280b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272d4de640 1 -- 172.21.15.139:0/1850427624 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27281f38c0 con 0x7f272806b700 2024-04-25T09:53:56.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f272d4de640 1 --2- 172.21.15.139:0/1850427624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f272806b700 0x7f27280b50c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f272400c7f0 tx=0x7f272400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.777+0000 7f271dffb640 1 -- 172.21.15.139:0/1850427624 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724015070 con 0x7f272806b700 2024-04-25T09:53:56.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.778+0000 7f271dffb640 1 -- 172.21.15.139:0/1850427624 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2724004240 con 0x7f272806b700 2024-04-25T09:53:56.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.778+0000 7f272f742640 1 -- 172.21.15.139:0/1850427624 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2728154660 con 0x7f272806b700 2024-04-25T09:53:56.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.778+0000 7f271dffb640 1 -- 172.21.15.139:0/1850427624 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724007620 con 0x7f272806b700 2024-04-25T09:53:56.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.778+0000 7f272f742640 1 -- 172.21.15.139:0/1850427624 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2728154c20 con 0x7f272806b700 2024-04-25T09:53:56.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.779+0000 7f272f742640 1 -- 172.21.15.139:0/1850427624 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2728074520 con 0x7f272806b700 2024-04-25T09:53:56.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.779+0000 7f271dffb640 1 -- 172.21.15.139:0/1850427624 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27240079d0 con 0x7f272806b700 2024-04-25T09:53:56.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.779+0000 7f271dffb640 1 --2- 172.21.15.139:0/1850427624 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f270c06b4d0 0x7f270c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:56.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.780+0000 7f271dffb640 1 -- 172.21.15.139:0/1850427624 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f272408aa10 con 0x7f272806b700 2024-04-25T09:53:56.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.780+0000 7f272ccdd640 1 --2- 172.21.15.139:0/1850427624 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f270c06b4d0 0x7f270c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:56.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.780+0000 7f272ccdd640 1 --2- 172.21.15.139:0/1850427624 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f270c06b4d0 0x7f270c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2718034000 tx=0x7f271802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:56.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.784+0000 7f271dffb640 1 -- 172.21.15.139:0/1850427624 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27281f38c0 con 0x7f272806b700 2024-04-25T09:53:56.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.958+0000 7f272f742640 1 -- 172.21.15.139:0/1850427624 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27280ae8d0 con 0x7f272806b700 2024-04-25T09:53:56.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.958+0000 7f271dffb640 1 -- 172.21.15.139:0/1850427624 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2724059490 con 0x7f272806b700 2024-04-25T09:53:56.960 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:53:56.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.960+0000 7f272f742640 1 -- 172.21.15.139:0/1850427624 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f270c06b4d0 msgr2=0x7f270c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.960+0000 7f272f742640 1 --2- 172.21.15.139:0/1850427624 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f270c06b4d0 0x7f270c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2718034000 tx=0x7f271802c000 comp rx=0 tx=0).stop 2024-04-25T09:53:56.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.960+0000 7f272f742640 1 -- 172.21.15.139:0/1850427624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f272806b700 msgr2=0x7f27280b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:56.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.960+0000 7f272f742640 1 --2- 172.21.15.139:0/1850427624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f272806b700 0x7f27280b50c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f272400c7f0 tx=0x7f272400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.960+0000 7f272f742640 1 -- 172.21.15.139:0/1850427624 shutdown_connections 2024-04-25T09:53:56.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.960+0000 7f272f742640 1 --2- 172.21.15.139:0/1850427624 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f270c06b4d0 0x7f270c06d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.960+0000 7f272f742640 1 --2- 172.21.15.139:0/1850427624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27280afc80 0x7f27280b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.960+0000 7f272f742640 1 --2- 172.21.15.139:0/1850427624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27280af8a0 0x7f27280b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.960+0000 7f272f742640 1 --2- 172.21.15.139:0/1850427624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f272806b700 0x7f27280b50c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:56.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.960+0000 7f272f742640 1 -- 172.21.15.139:0/1850427624 >> 172.21.15.139:0/1850427624 conn(0x7f27280b2230 msgr2=0x7f27280adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:56.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.960+0000 7f272f742640 1 -- 172.21.15.139:0/1850427624 shutdown_connections 2024-04-25T09:53:56.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:56.960+0000 7f272f742640 1 -- 172.21.15.139:0/1850427624 wait complete. 2024-04-25T09:53:56.973 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:53:57.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.070+0000 7f636929e640 1 Processor -- start 2024-04-25T09:53:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.070+0000 7f636929e640 1 -- start start 2024-04-25T09:53:57.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.071+0000 7f636929e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f636415dbf0 0x7f63641617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.071+0000 7f636929e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63640b9370 0x7f63640ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.071+0000 7f636929e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6364161dc0 con 0x7f63640cec80 2024-04-25T09:53:57.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.071+0000 7f636929e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63640d1ee0 con 0x7f636415dbf0 2024-04-25T09:53:57.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.071+0000 7f636929e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63641620a0 con 0x7f63640b9370 2024-04-25T09:53:57.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.071+0000 7f6362ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f636415dbf0 0x7f63641617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.071+0000 7f6362ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f636415dbf0 0x7f63641617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47148/0 (socket says 172.21.15.139:47148) 2024-04-25T09:53:57.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.071+0000 7f6362ffd640 1 -- 172.21.15.139:0/615116486 learned_addr learned my addr 172.21.15.139:0/615116486 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.071+0000 7f6362ffd640 1 -- 172.21.15.139:0/615116486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63640b9370 msgr2=0x7f63640ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.071+0000 7f6362ffd640 1 --2- 172.21.15.139:0/615116486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63640b9370 0x7f63640ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.071+0000 7f6362ffd640 1 -- 172.21.15.139:0/615116486 >> v1:172.21.15.150:6789/0 conn(0x7f63640cec80 legacy=0x7f636406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.071+0000 7f6362ffd640 1 -- 172.21.15.139:0/615116486 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6364162380 con 0x7f636415dbf0 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.072+0000 7f6362ffd640 1 --2- 172.21.15.139:0/615116486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f636415dbf0 0x7f63641617e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6354009e50 tx=0x7f635402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca2175778ce4de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.072+0000 7f6361ffb640 1 -- 172.21.15.139:0/615116486 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354036070 con 0x7f636415dbf0 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.072+0000 7f6361ffb640 1 -- 172.21.15.139:0/615116486 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6354031600 con 0x7f636415dbf0 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.072+0000 7f6361ffb640 1 -- 172.21.15.139:0/615116486 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63540319d0 con 0x7f636415dbf0 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.073+0000 7f636929e640 1 -- 172.21.15.139:0/615116486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f636415dbf0 msgr2=0x7f63641617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.073+0000 7f636929e640 1 --2- 172.21.15.139:0/615116486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f636415dbf0 0x7f63641617e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6354009e50 tx=0x7f635402a6c0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.073+0000 7f636929e640 1 -- 172.21.15.139:0/615116486 shutdown_connections 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.073+0000 7f636929e640 1 --2- 172.21.15.139:0/615116486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63640b9370 0x7f63640ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.073+0000 7f636929e640 1 --2- 172.21.15.139:0/615116486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f636415dbf0 0x7f63641617e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.073+0000 7f636929e640 1 -- 172.21.15.139:0/615116486 >> 172.21.15.139:0/615116486 conn(0x7f63640b5480 msgr2=0x7f63640b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.073+0000 7f636929e640 1 -- 172.21.15.139:0/615116486 shutdown_connections 2024-04-25T09:53:57.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.073+0000 7f636929e640 1 -- 172.21.15.139:0/615116486 wait complete. 2024-04-25T09:53:57.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.074+0000 7f636929e640 1 Processor -- start 2024-04-25T09:53:57.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.074+0000 7f636929e640 1 -- start start 2024-04-25T09:53:57.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f636929e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63640b9370 0x7f63641603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f636929e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63640cec80 0x7f636414e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f636929e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f636415dbf0 0x7f636414ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f636929e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636415dfd0 con 0x7f636415dbf0 2024-04-25T09:53:57.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f636929e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63640b97d0 con 0x7f63640b9370 2024-04-25T09:53:57.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f636929e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63641f4490 con 0x7f63640cec80 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f63637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f636415dbf0 0x7f636414ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f63637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f636415dbf0 0x7f636414ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47156/0 (socket says 172.21.15.139:47156) 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f63637fe640 1 -- 172.21.15.139:0/3942568076 learned_addr learned my addr 172.21.15.139:0/3942568076 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f6362ffd640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63640b9370 0x7f63641603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f63627fc640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63640cec80 0x7f636414e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f63637fe640 1 -- 172.21.15.139:0/3942568076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63640cec80 msgr2=0x7f636414e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f63637fe640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63640cec80 0x7f636414e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f63637fe640 1 -- 172.21.15.139:0/3942568076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63640b9370 msgr2=0x7f63641603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f63637fe640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63640b9370 0x7f63641603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.075+0000 7f63637fe640 1 -- 172.21.15.139:0/3942568076 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6364159e00 con 0x7f636415dbf0 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.076+0000 7f63637fe640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f636415dbf0 0x7f636414ee60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f635c00c390 tx=0x7f635c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.076+0000 7f634b7fe640 1 -- 172.21.15.139:0/3942568076 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c017070 con 0x7f636415dbf0 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.076+0000 7f634b7fe640 1 -- 172.21.15.139:0/3942568076 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c0035f0 con 0x7f636415dbf0 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.076+0000 7f636929e640 1 -- 172.21.15.139:0/3942568076 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636415a0e0 con 0x7f636415dbf0 2024-04-25T09:53:57.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.076+0000 7f634b7fe640 1 -- 172.21.15.139:0/3942568076 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c0039e0 con 0x7f636415dbf0 2024-04-25T09:53:57.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.076+0000 7f636929e640 1 -- 172.21.15.139:0/3942568076 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63641608f0 con 0x7f636415dbf0 2024-04-25T09:53:57.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.077+0000 7f636929e640 1 -- 172.21.15.139:0/3942568076 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6364159bf0 con 0x7f636415dbf0 2024-04-25T09:53:57.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.077+0000 7f634b7fe640 1 -- 172.21.15.139:0/3942568076 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f635c01a070 con 0x7f636415dbf0 2024-04-25T09:53:57.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.078+0000 7f634b7fe640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f635006b4d0 0x7f635006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.078+0000 7f634b7fe640 1 -- 172.21.15.139:0/3942568076 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f635c089940 con 0x7f636415dbf0 2024-04-25T09:53:57.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.078+0000 7f6362ffd640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f635006b4d0 0x7f635006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.078+0000 7f6362ffd640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f635006b4d0 0x7f635006d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6354033000 tx=0x7f6354009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.082+0000 7f634b7fe640 1 -- 172.21.15.139:0/3942568076 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f635c057d70 con 0x7f636415dbf0 2024-04-25T09:53:57.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.255+0000 7f636929e640 1 -- 172.21.15.139:0/3942568076 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6364074520 con 0x7f636415dbf0 2024-04-25T09:53:57.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.255+0000 7f634b7fe640 1 -- 172.21.15.139:0/3942568076 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f635c05e0c0 con 0x7f636415dbf0 2024-04-25T09:53:57.258 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:57.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.257+0000 7f636929e640 1 -- 172.21.15.139:0/3942568076 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f635006b4d0 msgr2=0x7f635006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.257+0000 7f636929e640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f635006b4d0 0x7f635006d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6354033000 tx=0x7f6354009000 comp rx=0 tx=0).stop 2024-04-25T09:53:57.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.257+0000 7f636929e640 1 -- 172.21.15.139:0/3942568076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f636415dbf0 msgr2=0x7f636414ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.257+0000 7f636929e640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f636415dbf0 0x7f636414ee60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f635c00c390 tx=0x7f635c00c850 comp rx=0 tx=0).stop 2024-04-25T09:53:57.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.257+0000 7f636929e640 1 -- 172.21.15.139:0/3942568076 shutdown_connections 2024-04-25T09:53:57.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.257+0000 7f636929e640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f636415dbf0 0x7f636414ee60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.257+0000 7f636929e640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f635006b4d0 0x7f635006d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.257+0000 7f636929e640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63640cec80 0x7f636414e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.257+0000 7f636929e640 1 --2- 172.21.15.139:0/3942568076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63640b9370 0x7f63641603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.257+0000 7f636929e640 1 -- 172.21.15.139:0/3942568076 >> 172.21.15.139:0/3942568076 conn(0x7f63640b5480 msgr2=0x7f63640caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.257+0000 7f636929e640 1 -- 172.21.15.139:0/3942568076 shutdown_connections 2024-04-25T09:53:57.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.257+0000 7f636929e640 1 -- 172.21.15.139:0/3942568076 wait complete. 2024-04-25T09:53:57.271 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:53:57.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.369+0000 7fe020e5d640 1 Processor -- start 2024-04-25T09:53:57.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.369+0000 7fe020e5d640 1 -- start start 2024-04-25T09:53:57.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.369+0000 7fe020e5d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe01c15db40 0x7fe01c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.369+0000 7fe020e5d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe01c0b4bc0 0x7fe01c0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.369+0000 7fe020e5d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01c070f50 con 0x7fe01c0b56b0 2024-04-25T09:53:57.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.369+0000 7fe020e5d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01c071230 con 0x7fe01c15db40 2024-04-25T09:53:57.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.369+0000 7fe020e5d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01c071510 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.370+0000 7fe01a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe01c15db40 0x7fe01c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.370+0000 7fe01a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe01c15db40 0x7fe01c0b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47168/0 (socket says 172.21.15.139:47168) 2024-04-25T09:53:57.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.370+0000 7fe01a59c640 1 -- 172.21.15.139:0/1738082789 learned_addr learned my addr 172.21.15.139:0/1738082789 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:57.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.370+0000 7fe013fff640 1 --2- 172.21.15.139:0/1738082789 >> v2:172.21.15.150:3300/0 conn(0x7fe01c0b4bc0 0x7fe01c0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.370+0000 7fe01a59c640 1 -- 172.21.15.139:0/1738082789 >> v2:172.21.15.150:3300/0 conn(0x7fe01c0b4bc0 msgr2=0x7fe01c0b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:57.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.370+0000 7fe01a59c640 1 --2- 172.21.15.139:0/1738082789 >> v2:172.21.15.150:3300/0 conn(0x7fe01c0b4bc0 0x7fe01c0b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.370+0000 7fe01a59c640 1 -- 172.21.15.139:0/1738082789 >> v1:172.21.15.150:6789/0 conn(0x7fe01c0b56b0 legacy=0x7fe01c1615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:57.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.370+0000 7fe01a59c640 1 -- 172.21.15.139:0/1738082789 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c0717f0 con 0x7fe01c15db40 2024-04-25T09:53:57.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.370+0000 7fe013fff640 1 --2- 172.21.15.139:0/1738082789 >> v2:172.21.15.150:3300/0 conn(0x7fe01c0b4bc0 0x7fe01c0b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:57.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.371+0000 7fe01a59c640 1 --2- 172.21.15.139:0/1738082789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c15db40 0x7fe01c0b4680 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe01c06bb40 tx=0x7fe00c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfff0c824339a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.371+0000 7fe019d9b640 1 -- 172.21.15.139:0/1738082789 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c037070 con 0x7fe01c15db40 2024-04-25T09:53:57.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.371+0000 7fe019d9b640 1 -- 172.21.15.139:0/1738082789 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c032440 con 0x7fe01c15db40 2024-04-25T09:53:57.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.371+0000 7fe019d9b640 1 -- 172.21.15.139:0/1738082789 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c032810 con 0x7fe01c15db40 2024-04-25T09:53:57.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.371+0000 7fe020e5d640 1 -- 172.21.15.139:0/1738082789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c15db40 msgr2=0x7fe01c0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.371+0000 7fe020e5d640 1 --2- 172.21.15.139:0/1738082789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c15db40 0x7fe01c0b4680 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe01c06bb40 tx=0x7fe00c02a230 comp rx=0 tx=0).stop 2024-04-25T09:53:57.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.371+0000 7fe020e5d640 1 -- 172.21.15.139:0/1738082789 shutdown_connections 2024-04-25T09:53:57.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.371+0000 7fe020e5d640 1 --2- 172.21.15.139:0/1738082789 >> v2:172.21.15.150:3300/0 conn(0x7fe01c0b4bc0 0x7fe01c0b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.371+0000 7fe020e5d640 1 --2- 172.21.15.139:0/1738082789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c15db40 0x7fe01c0b4680 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.371+0000 7fe020e5d640 1 -- 172.21.15.139:0/1738082789 >> 172.21.15.139:0/1738082789 conn(0x7fe01c06f460 msgr2=0x7fe01c06f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.371+0000 7fe020e5d640 1 -- 172.21.15.139:0/1738082789 shutdown_connections 2024-04-25T09:53:57.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.371+0000 7fe020e5d640 1 -- 172.21.15.139:0/1738082789 wait complete. 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.373+0000 7fe020e5d640 1 Processor -- start 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.373+0000 7fe020e5d640 1 -- start start 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.373+0000 7fe020e5d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c0b4bc0 0x7fe01c1422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.373+0000 7fe020e5d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01c0b56b0 0x7fe01c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.373+0000 7fe020e5d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01c15db40 0x7fe01c15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.373+0000 7fe020e5d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c06cf60 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.373+0000 7fe020e5d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c070be0 con 0x7fe01c15db40 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.373+0000 7fe020e5d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c06d0e0 con 0x7fe01c0b56b0 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.373+0000 7fe01a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c0b4bc0 0x7fe01c1422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.373+0000 7fe01a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c0b4bc0 0x7fe01c1422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47170/0 (socket says 172.21.15.139:47170) 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.373+0000 7fe01a59c640 1 -- 172.21.15.139:0/537633318 learned_addr learned my addr 172.21.15.139:0/537633318 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe01ad9d640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01c15db40 0x7fe01c15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe013fff640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01c0b56b0 0x7fe01c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe01a59c640 1 -- 172.21.15.139:0/537633318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01c0b56b0 msgr2=0x7fe01c15df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe01a59c640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01c0b56b0 0x7fe01c15df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe01a59c640 1 -- 172.21.15.139:0/537633318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01c15db40 msgr2=0x7fe01c15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe01a59c640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01c15db40 0x7fe01c15af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe01a59c640 1 -- 172.21.15.139:0/537633318 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c142810 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe01ad9d640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01c15db40 0x7fe01c15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe01a59c640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c0b4bc0 0x7fe01c1422d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe00c000c00 tx=0x7fe00c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe012ffd640 1 -- 172.21.15.139:0/537633318 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c041020 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe012ffd640 1 -- 172.21.15.139:0/537633318 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c030970 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe020e5d640 1 -- 172.21.15.139:0/537633318 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c160980 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe012ffd640 1 -- 172.21.15.139:0/537633318 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c037040 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.374+0000 7fe020e5d640 1 -- 172.21.15.139:0/537633318 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c160f40 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.375+0000 7fe020e5d640 1 -- 172.21.15.139:0/537633318 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c074520 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.375+0000 7fe012ffd640 1 -- 172.21.15.139:0/537633318 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe00c044020 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.376+0000 7fe012ffd640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdffc06b4d0 0x7fdffc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.376+0000 7fe012ffd640 1 -- 172.21.15.139:0/537633318 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe00c0aaf70 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.376+0000 7fe013fff640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdffc06b4d0 0x7fdffc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.376+0000 7fe013fff640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdffc06b4d0 0x7fdffc06d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe01c1f6ee0 tx=0x7fe004008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.380+0000 7fe012ffd640 1 -- 172.21.15.139:0/537633318 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe00c04b030 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.550+0000 7fe020e5d640 1 -- 172.21.15.139:0/537633318 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe01c0701f0 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.550+0000 7fe012ffd640 1 -- 172.21.15.139:0/537633318 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe01c142810 con 0x7fe01c0b4bc0 2024-04-25T09:53:57.552 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:53:57.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.552+0000 7fe020e5d640 1 -- 172.21.15.139:0/537633318 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdffc06b4d0 msgr2=0x7fdffc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.552+0000 7fe020e5d640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdffc06b4d0 0x7fdffc06d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe01c1f6ee0 tx=0x7fe004008040 comp rx=0 tx=0).stop 2024-04-25T09:53:57.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.552+0000 7fe020e5d640 1 -- 172.21.15.139:0/537633318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c0b4bc0 msgr2=0x7fe01c1422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.552+0000 7fe020e5d640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c0b4bc0 0x7fe01c1422d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe00c000c00 tx=0x7fe00c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.552+0000 7fe020e5d640 1 -- 172.21.15.139:0/537633318 shutdown_connections 2024-04-25T09:53:57.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.552+0000 7fe020e5d640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdffc06b4d0 0x7fdffc06d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.552+0000 7fe020e5d640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01c15db40 0x7fe01c15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.552+0000 7fe020e5d640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01c0b56b0 0x7fe01c15df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.552+0000 7fe020e5d640 1 --2- 172.21.15.139:0/537633318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01c0b4bc0 0x7fe01c1422d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.552+0000 7fe020e5d640 1 -- 172.21.15.139:0/537633318 >> 172.21.15.139:0/537633318 conn(0x7fe01c06f460 msgr2=0x7fe01c0caff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.552+0000 7fe020e5d640 1 -- 172.21.15.139:0/537633318 shutdown_connections 2024-04-25T09:53:57.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.552+0000 7fe020e5d640 1 -- 172.21.15.139:0/537633318 wait complete. 2024-04-25T09:53:57.565 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:53:57.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.663+0000 7fe1d254c640 1 Processor -- start 2024-04-25T09:53:57.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.663+0000 7fe1d254c640 1 -- start start 2024-04-25T09:53:57.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.664+0000 7fe1d254c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1cc15fca0 0x7fe1cc0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.664+0000 7fe1d254c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe1cc0d0220 0x7fe1cc15b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.664+0000 7fe1d254c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1cc099bf0 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.664+0000 7fe1d254c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc0d06c0 con 0x7fe1cc0d0220 2024-04-25T09:53:57.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.664+0000 7fe1d254c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc099ed0 con 0x7fe1cc15fca0 2024-04-25T09:53:57.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.664+0000 7fe1cb7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1cc15fca0 0x7fe1cc0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.664+0000 7fe1d0ae9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe1cc0d0220 0x7fe1cc15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.664+0000 7fe1cbfff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fe1cc15f8c0 0x7fe1cc09a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:53858/0 (socket says 172.21.15.139:53858) 2024-04-25T09:53:57.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.664+0000 7fe1cb7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1cc15fca0 0x7fe1cc0cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:47738/0 (socket says 172.21.15.139:47738) 2024-04-25T09:53:57.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.664+0000 7fe1d0ae9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe1cc0d0220 0x7fe1cc15b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:49630/0 (socket says 172.21.15.139:49630) 2024-04-25T09:53:57.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.664+0000 7fe1cbfff640 1 -- 172.21.15.139:0/1949400269 learned_addr learned my addr 172.21.15.139:0/1949400269 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:57.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.665+0000 7fe1d0ae9640 1 -- 172.21.15.139:0/1949400269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1cc15fca0 msgr2=0x7fe1cc0cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:57.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.665+0000 7fe1d0ae9640 1 --2- 172.21.15.139:0/1949400269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1cc15fca0 0x7fe1cc0cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.665+0000 7fe1d0ae9640 1 -- 172.21.15.139:0/1949400269 >> v1:172.21.15.150:6789/0 conn(0x7fe1cc15f8c0 legacy=0x7fe1cc09a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.665+0000 7fe1d0ae9640 1 -- 172.21.15.139:0/1949400269 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc09a1b0 con 0x7fe1cc0d0220 2024-04-25T09:53:57.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.665+0000 7fe1cb7fe640 1 --2- 172.21.15.139:0/1949400269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1cc15fca0 0x7fe1cc0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:57.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.666+0000 7fe1d0ae9640 1 --2- 172.21.15.139:0/1949400269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0d0220 0x7fe1cc15b560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009d90 tx=0x7fe1c402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5931a83bb172b76f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.666+0000 7fe1caffd640 1 -- 172.21.15.139:0/1949400269 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4007740 con 0x7fe1cc0d0220 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.666+0000 7fe1caffd640 1 -- 172.21.15.139:0/1949400269 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c40078e0 con 0x7fe1cc0d0220 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.666+0000 7fe1caffd640 1 -- 172.21.15.139:0/1949400269 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4007cb0 con 0x7fe1cc0d0220 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.666+0000 7fe1d254c640 1 -- 172.21.15.139:0/1949400269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0d0220 msgr2=0x7fe1cc15b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.666+0000 7fe1d254c640 1 --2- 172.21.15.139:0/1949400269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0d0220 0x7fe1cc15b560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009d90 tx=0x7fe1c402a660 comp rx=0 tx=0).stop 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.666+0000 7fe1d254c640 1 -- 172.21.15.139:0/1949400269 shutdown_connections 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.666+0000 7fe1d254c640 1 --2- 172.21.15.139:0/1949400269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1cc15fca0 0x7fe1cc0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.666+0000 7fe1d254c640 1 --2- 172.21.15.139:0/1949400269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0d0220 0x7fe1cc15b560 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.666+0000 7fe1d254c640 1 -- 172.21.15.139:0/1949400269 >> 172.21.15.139:0/1949400269 conn(0x7fe1cc0b31f0 msgr2=0x7fe1cc0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.666+0000 7fe1d254c640 1 -- 172.21.15.139:0/1949400269 shutdown_connections 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.666+0000 7fe1d254c640 1 -- 172.21.15.139:0/1949400269 wait complete. 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.667+0000 7fe1d254c640 1 Processor -- start 2024-04-25T09:53:57.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.668+0000 7fe1d254c640 1 -- start start 2024-04-25T09:53:57.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.668+0000 7fe1d254c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0d0220 0x7fe1cc0b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.668+0000 7fe1d254c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1cc15f8c0 0x7fe1cc1e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.668+0000 7fe1d254c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1cc15fca0 0x7fe1cc1e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.668+0000 7fe1d254c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc160450 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.668+0000 7fe1d254c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc1602d0 con 0x7fe1cc0d0220 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.668+0000 7fe1d254c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc15baa0 con 0x7fe1cc15fca0 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.668+0000 7fe1cb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1cc15f8c0 0x7fe1cc1e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1cb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1cc15f8c0 0x7fe1cc1e6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47186/0 (socket says 172.21.15.139:47186) 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1cb7fe640 1 -- 172.21.15.139:0/23084627 learned_addr learned my addr 172.21.15.139:0/23084627 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1cb7fe640 1 -- 172.21.15.139:0/23084627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1cc15fca0 msgr2=0x7fe1cc1e6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1cb7fe640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1cc15fca0 0x7fe1cc1e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1cb7fe640 1 -- 172.21.15.139:0/23084627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0d0220 msgr2=0x7fe1cc0b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1cbfff640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0d0220 0x7fe1cc0b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1cb7fe640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0d0220 0x7fe1cc0b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1cb7fe640 1 -- 172.21.15.139:0/23084627 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc0b5d20 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1cbfff640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0d0220 0x7fe1cc0b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:57.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1cb7fe640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1cc15f8c0 0x7fe1cc1e6260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000e990 tx=0x7fe1c000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1c8ff9640 1 -- 172.21.15.139:0/23084627 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0017070 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1c8ff9640 1 -- 172.21.15.139:0/23084627 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000c9e0 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1c8ff9640 1 -- 172.21.15.139:0/23084627 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000cdb0 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1d254c640 1 -- 172.21.15.139:0/23084627 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc0b6000 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.669+0000 7fe1d254c640 1 -- 172.21.15.139:0/23084627 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc0b65c0 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.670+0000 7fe1d254c640 1 -- 172.21.15.139:0/23084627 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc074520 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.671+0000 7fe1c8ff9640 1 -- 172.21.15.139:0/23084627 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1c001a070 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.671+0000 7fe1c8ff9640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe1b806b500 0x7fe1b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.671+0000 7fe1c8ff9640 1 -- 172.21.15.139:0/23084627 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe1c008a3f0 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.671+0000 7fe1cbfff640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe1b806b500 0x7fe1b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.671+0000 7fe1cbfff640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe1b806b500 0x7fe1b806d9c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0b2620 tx=0x7fe1bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.675+0000 7fe1c8ff9640 1 -- 172.21.15.139:0/23084627 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1c0058f00 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.845+0000 7fe1d254c640 1 -- 172.21.15.139:0/23084627 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe1cc1e76f0 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.846+0000 7fe1c8ff9640 1 -- 172.21.15.139:0/23084627 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe1c0058800 con 0x7fe1cc15f8c0 2024-04-25T09:53:57.848 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:53:57.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.847+0000 7fe1d254c640 1 -- 172.21.15.139:0/23084627 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe1b806b500 msgr2=0x7fe1b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.847+0000 7fe1d254c640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe1b806b500 0x7fe1b806d9c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0b2620 tx=0x7fe1bc009040 comp rx=0 tx=0).stop 2024-04-25T09:53:57.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.847+0000 7fe1d254c640 1 -- 172.21.15.139:0/23084627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1cc15f8c0 msgr2=0x7fe1cc1e6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.847+0000 7fe1d254c640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1cc15f8c0 0x7fe1cc1e6260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000e990 tx=0x7fe1c000ee50 comp rx=0 tx=0).stop 2024-04-25T09:53:57.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.848+0000 7fe1d254c640 1 -- 172.21.15.139:0/23084627 shutdown_connections 2024-04-25T09:53:57.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.848+0000 7fe1d254c640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1cc15f8c0 0x7fe1cc1e6260 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.848+0000 7fe1d254c640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe1b806b500 0x7fe1b806d9c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.848+0000 7fe1d254c640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1cc15fca0 0x7fe1cc1e6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.848+0000 7fe1d254c640 1 --2- 172.21.15.139:0/23084627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1cc0d0220 0x7fe1cc0b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.848+0000 7fe1d254c640 1 -- 172.21.15.139:0/23084627 >> 172.21.15.139:0/23084627 conn(0x7fe1cc0b31f0 msgr2=0x7fe1cc0cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.848+0000 7fe1d254c640 1 -- 172.21.15.139:0/23084627 shutdown_connections 2024-04-25T09:53:57.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.848+0000 7fe1d254c640 1 -- 172.21.15.139:0/23084627 wait complete. 2024-04-25T09:53:57.861 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T09:53:57.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.956+0000 7fb123bb6640 1 Processor -- start 2024-04-25T09:53:57.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.956+0000 7fb123bb6640 1 -- start start 2024-04-25T09:53:57.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb123bb6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb11c15efb0 0x7fb11c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb123bb6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb11c0cf900 0x7fb11c06c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb123bb6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c0cfe70 con 0x7fb11c06cd80 2024-04-25T09:53:57.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb123bb6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11c0d0a80 con 0x7fb11c0cf900 2024-04-25T09:53:57.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb123bb6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11c0d0150 con 0x7fb11c15efb0 2024-04-25T09:53:57.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb121151640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb11c0cf900 0x7fb11c06c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb121151640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb11c0cf900 0x7fb11c06c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47198/0 (socket says 172.21.15.139:47198) 2024-04-25T09:53:57.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb121151640 1 -- 172.21.15.139:0/2153184885 learned_addr learned my addr 172.21.15.139:0/2153184885 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:57.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb122153640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb11c06cd80 0x7fb11c099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54526/0 (socket says 172.21.15.139:54526) 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb121952640 1 --2- 172.21.15.139:0/2153184885 >> v2:172.21.15.150:3300/0 conn(0x7fb11c15efb0 0x7fb11c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb121151640 1 -- 172.21.15.139:0/2153184885 >> v2:172.21.15.150:3300/0 conn(0x7fb11c15efb0 msgr2=0x7fb11c0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb121151640 1 --2- 172.21.15.139:0/2153184885 >> v2:172.21.15.150:3300/0 conn(0x7fb11c15efb0 0x7fb11c0d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb121151640 1 -- 172.21.15.139:0/2153184885 >> v1:172.21.15.139:6789/0 conn(0x7fb11c06cd80 legacy=0x7fb11c099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb120950640 1 -- 172.21.15.139:0/2153184885 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507120776 0 0) 0x7fb11c0cfe70 con 0x7fb11c06cd80 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb121151640 1 -- 172.21.15.139:0/2153184885 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c0d0430 con 0x7fb11c0cf900 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.957+0000 7fb121952640 1 --2- 172.21.15.139:0/2153184885 >> v2:172.21.15.150:3300/0 conn(0x7fb11c15efb0 0x7fb11c0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.958+0000 7fb121151640 1 --2- 172.21.15.139:0/2153184885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c0cf900 0x7fb11c06c840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb11c072c10 tx=0x7fb10c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bdcbcaca2994cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.958+0000 7fb120950640 1 -- 172.21.15.139:0/2153184885 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c037070 con 0x7fb11c0cf900 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.958+0000 7fb120950640 1 -- 172.21.15.139:0/2153184885 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c032440 con 0x7fb11c0cf900 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.958+0000 7fb120950640 1 -- 172.21.15.139:0/2153184885 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c032810 con 0x7fb11c0cf900 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.959+0000 7fb123bb6640 1 -- 172.21.15.139:0/2153184885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c0cf900 msgr2=0x7fb11c06c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.959+0000 7fb123bb6640 1 --2- 172.21.15.139:0/2153184885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c0cf900 0x7fb11c06c840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb11c072c10 tx=0x7fb10c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.959+0000 7fb123bb6640 1 -- 172.21.15.139:0/2153184885 shutdown_connections 2024-04-25T09:53:57.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.959+0000 7fb123bb6640 1 --2- 172.21.15.139:0/2153184885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c0cf900 0x7fb11c06c840 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.959+0000 7fb123bb6640 1 --2- 172.21.15.139:0/2153184885 >> v2:172.21.15.150:3300/0 conn(0x7fb11c15efb0 0x7fb11c0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.959+0000 7fb123bb6640 1 -- 172.21.15.139:0/2153184885 >> 172.21.15.139:0/2153184885 conn(0x7fb11c0d1790 msgr2=0x7fb11c0d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:57.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.959+0000 7fb123bb6640 1 -- 172.21.15.139:0/2153184885 shutdown_connections 2024-04-25T09:53:57.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.959+0000 7fb123bb6640 1 -- 172.21.15.139:0/2153184885 wait complete. 2024-04-25T09:53:57.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.960+0000 7fb123bb6640 1 Processor -- start 2024-04-25T09:53:57.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.960+0000 7fb123bb6640 1 -- start start 2024-04-25T09:53:57.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.960+0000 7fb123bb6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb11c06cd80 0x7fb11c1f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.960+0000 7fb123bb6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb11c0cf900 0x7fb11c1e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.960+0000 7fb123bb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c15efb0 0x7fb11c161560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.960+0000 7fb123bb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c09a2e0 con 0x7fb11c15efb0 2024-04-25T09:53:57.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.960+0000 7fb123bb6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c09a160 con 0x7fb11c0cf900 2024-04-25T09:53:57.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.960+0000 7fb123bb6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0cbde0 con 0x7fb11c06cd80 2024-04-25T09:53:57.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb122153640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c15efb0 0x7fb11c161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb122153640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c15efb0 0x7fb11c161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47208/0 (socket says 172.21.15.139:47208) 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb122153640 1 -- 172.21.15.139:0/568417286 learned_addr learned my addr 172.21.15.139:0/568417286 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb121151640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb11c0cf900 0x7fb11c1e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb122153640 1 -- 172.21.15.139:0/568417286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb11c06cd80 msgr2=0x7fb11c1f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb121952640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb11c06cd80 0x7fb11c1f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb122153640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb11c06cd80 0x7fb11c1f33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb122153640 1 -- 172.21.15.139:0/568417286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb11c0cf900 msgr2=0x7fb11c1e91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb122153640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb11c0cf900 0x7fb11c1e91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb122153640 1 -- 172.21.15.139:0/568417286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c1f3920 con 0x7fb11c15efb0 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb121151640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb11c0cf900 0x7fb11c1e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb122153640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c15efb0 0x7fb11c161560 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c7b0 tx=0x7fb11400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb1127fc640 1 -- 172.21.15.139:0/568417286 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114018070 con 0x7fb11c15efb0 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb1127fc640 1 -- 172.21.15.139:0/568417286 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11400a640 con 0x7fb11c15efb0 2024-04-25T09:53:57.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb1127fc640 1 -- 172.21.15.139:0/568417286 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11400aa70 con 0x7fb11c15efb0 2024-04-25T09:53:57.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb123bb6640 1 -- 172.21.15.139:0/568417286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c0b4680 con 0x7fb11c15efb0 2024-04-25T09:53:57.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.961+0000 7fb123bb6640 1 -- 172.21.15.139:0/568417286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c0b4c00 con 0x7fb11c15efb0 2024-04-25T09:53:57.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.962+0000 7fb123bb6640 1 -- 172.21.15.139:0/568417286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c074520 con 0x7fb11c15efb0 2024-04-25T09:53:57.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.963+0000 7fb1127fc640 1 -- 172.21.15.139:0/568417286 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb11401b070 con 0x7fb11c15efb0 2024-04-25T09:53:57.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.963+0000 7fb1127fc640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb10006b4d0 0x7fb10006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:57.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.963+0000 7fb1127fc640 1 -- 172.21.15.139:0/568417286 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb114089bd0 con 0x7fb11c15efb0 2024-04-25T09:53:57.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.963+0000 7fb121952640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb10006b4d0 0x7fb10006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:57.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.964+0000 7fb121952640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb10006b4d0 0x7fb10006d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb11800b840 tx=0x7fb118008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:57.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:57.967+0000 7fb1127fc640 1 -- 172.21.15.139:0/568417286 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb114058650 con 0x7fb11c15efb0 2024-04-25T09:53:58.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.202+0000 7fb123bb6640 1 -- 172.21.15.139:0/568417286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fb11c09ae50 con 0x7fb11c15efb0 2024-04-25T09:53:58.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.854+0000 7fb1127fc640 1 -- 172.21.15.139:0/568417286 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v66) v1 ==== 145+0+0 (secure 0 0 0) 0x7fb114057f50 con 0x7fb11c15efb0 2024-04-25T09:53:58.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.856+0000 7fb123bb6640 1 -- 172.21.15.139:0/568417286 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb10006b4d0 msgr2=0x7fb10006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.856+0000 7fb123bb6640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb10006b4d0 0x7fb10006d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb11800b840 tx=0x7fb118008040 comp rx=0 tx=0).stop 2024-04-25T09:53:58.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.856+0000 7fb123bb6640 1 -- 172.21.15.139:0/568417286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c15efb0 msgr2=0x7fb11c161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.856+0000 7fb123bb6640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c15efb0 0x7fb11c161560 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c7b0 tx=0x7fb11400cc70 comp rx=0 tx=0).stop 2024-04-25T09:53:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.856+0000 7fb123bb6640 1 -- 172.21.15.139:0/568417286 shutdown_connections 2024-04-25T09:53:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.856+0000 7fb123bb6640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb11c15efb0 0x7fb11c161560 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.856+0000 7fb123bb6640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb10006b4d0 0x7fb10006d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.856+0000 7fb123bb6640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb11c0cf900 0x7fb11c1e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.856+0000 7fb123bb6640 1 --2- 172.21.15.139:0/568417286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb11c06cd80 0x7fb11c1f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.856+0000 7fb123bb6640 1 -- 172.21.15.139:0/568417286 >> 172.21.15.139:0/568417286 conn(0x7fb11c0d1790 msgr2=0x7fb11c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.856+0000 7fb123bb6640 1 -- 172.21.15.139:0/568417286 shutdown_connections 2024-04-25T09:53:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.856+0000 7fb123bb6640 1 -- 172.21.15.139:0/568417286 wait complete. 2024-04-25T09:53:58.870 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:53:58.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.970+0000 7f3f6b15c640 1 Processor -- start 2024-04-25T09:53:58.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.970+0000 7f3f6b15c640 1 -- start start 2024-04-25T09:53:58.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.971+0000 7f3f6b15c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f64145250 0x7f3f64070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.971+0000 7f3f6b15c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3f640719d0 0x7f3f64071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.971+0000 7f3f6b15c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f640b1100 con 0x7f3f640d11a0 2024-04-25T09:53:58.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.971+0000 7f3f6b15c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f640ceed0 con 0x7f3f640719d0 2024-04-25T09:53:58.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.971+0000 7f3f6b15c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640ce2c0 con 0x7f3f64145250 2024-04-25T09:53:58.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.971+0000 7f3f63fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3f640719d0 0x7f3f64071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.971+0000 7f3f696f9640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3f640d11a0 0x7f3f6415f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54540/0 (socket says 172.21.15.139:54540) 2024-04-25T09:53:58.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.971+0000 7f3f63fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3f640719d0 0x7f3f64071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47218/0 (socket says 172.21.15.139:47218) 2024-04-25T09:53:58.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.971+0000 7f3f696f9640 1 -- 172.21.15.139:0/1246209888 learned_addr learned my addr 172.21.15.139:0/1246209888 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:58.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f68ef8640 1 --2- 172.21.15.139:0/1246209888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f64145250 0x7f3f64070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419977753 0 0) 0x7f3f640b1100 con 0x7f3f640d11a0 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f54003520 con 0x7f3f640d11a0 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 322616539 0 0) 0x7f3f54003520 con 0x7f3f640d11a0 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f64145250 msgr2=0x7f3f64070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 --2- 172.21.15.139:0/1246209888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f64145250 0x7f3f64070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 >> v2:172.21.15.139:3300/0 conn(0x7f3f640719d0 msgr2=0x7f3f64071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 --2- 172.21.15.139:0/1246209888 >> v2:172.21.15.139:3300/0 conn(0x7f3f640719d0 0x7f3f64071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640ce5a0 con 0x7f3f640d11a0 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f3f5c0019f0 con 0x7f3f640d11a0 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 >> v1:172.21.15.139:6789/0 conn(0x7f3f640d11a0 legacy=0x7f3f6415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 --2- 172.21.15.139:0/1246209888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f540039b0 0x7f3f54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 --2- 172.21.15.139:0/1246209888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f54004800 0x7f3f54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 --2- 172.21.15.139:0/1246209888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f64145250 0x7f3f540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640ceed0 con 0x7f3f540039b0 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640ce2c0 con 0x7f3f54004800 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5400ba40 con 0x7f3f64145250 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f68ef8640 1 --2- 172.21.15.139:0/1246209888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f540039b0 0x7f3f54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f696f9640 1 --2- 172.21.15.139:0/1246209888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f64145250 0x7f3f540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f63fff640 1 --2- 172.21.15.139:0/1246209888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f54004800 0x7f3f54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f68ef8640 1 -- 172.21.15.139:0/1246209888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f64145250 msgr2=0x7f3f540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f68ef8640 1 --2- 172.21.15.139:0/1246209888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f64145250 0x7f3f540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f68ef8640 1 -- 172.21.15.139:0/1246209888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f54004800 msgr2=0x7f3f54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f68ef8640 1 --2- 172.21.15.139:0/1246209888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f54004800 0x7f3f54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.972+0000 7f3f68ef8640 1 -- 172.21.15.139:0/1246209888 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f54003520 con 0x7f3f540039b0 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.973+0000 7f3f68ef8640 1 --2- 172.21.15.139:0/1246209888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f540039b0 0x7f3f54003dd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009aa0 tx=0x7f3f54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd234b01bbcfe7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.973+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58036070 con 0x7f3f540039b0 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.973+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f58031400 con 0x7f3f540039b0 2024-04-25T09:53:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.973+0000 7f3f637fe640 1 -- 172.21.15.139:0/1246209888 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58009670 con 0x7f3f540039b0 2024-04-25T09:53:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.974+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1246209888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f540039b0 msgr2=0x7f3f54003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.974+0000 7f3f6b15c640 1 --2- 172.21.15.139:0/1246209888 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f540039b0 0x7f3f54003dd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009aa0 tx=0x7f3f54005830 comp rx=0 tx=0).stop 2024-04-25T09:53:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.974+0000 7f3f696f9640 1 -- 172.21.15.139:0/1246209888 reap_dead start 2024-04-25T09:53:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.974+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1246209888 shutdown_connections 2024-04-25T09:53:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.974+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1246209888 >> 172.21.15.139:0/1246209888 conn(0x7f3f6406f460 msgr2=0x7f3f6406f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.974+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1246209888 shutdown_connections 2024-04-25T09:53:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.974+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1246209888 wait complete. 2024-04-25T09:53:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.975+0000 7f3f6b15c640 1 Processor -- start 2024-04-25T09:53:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.975+0000 7f3f6b15c640 1 -- start start 2024-04-25T09:53:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f6b15c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f540039b0 0x7f3f64143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f6b15c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f64156d60 0x7f3f64159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f6b15c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f641598b0 0x7f3f6415b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f6b15c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f64145630 con 0x7f3f641598b0 2024-04-25T09:53:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f6b15c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640ceb60 con 0x7f3f64156d60 2024-04-25T09:53:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f6b15c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640b5690 con 0x7f3f540039b0 2024-04-25T09:53:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f696f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f641598b0 0x7f3f6415b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f696f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f641598b0 0x7f3f6415b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47232/0 (socket says 172.21.15.139:47232) 2024-04-25T09:53:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f696f9640 1 -- 172.21.15.139:0/1311675372 learned_addr learned my addr 172.21.15.139:0/1311675372 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f68ef8640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f540039b0 0x7f3f64143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f63fff640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f64156d60 0x7f3f64159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f696f9640 1 -- 172.21.15.139:0/1311675372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f540039b0 msgr2=0x7f3f64143bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f696f9640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f540039b0 0x7f3f64143bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f696f9640 1 -- 172.21.15.139:0/1311675372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f64156d60 msgr2=0x7f3f64159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f696f9640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f64156d60 0x7f3f64159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f696f9640 1 -- 172.21.15.139:0/1311675372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6415a2d0 con 0x7f3f641598b0 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.976+0000 7f3f696f9640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f641598b0 0x7f3f6415b070 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3f6406efc0 tx=0x7f3f5c007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.977+0000 7f3f617fa640 1 -- 172.21.15.139:0/1311675372 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00eaa0 con 0x7f3f641598b0 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.977+0000 7f3f617fa640 1 -- 172.21.15.139:0/1311675372 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c018070 con 0x7f3f641598b0 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.977+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1311675372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6415a5b0 con 0x7f3f641598b0 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.977+0000 7f3f617fa640 1 -- 172.21.15.139:0/1311675372 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00eea0 con 0x7f3f641598b0 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.977+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1311675372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6415ab70 con 0x7f3f641598b0 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.978+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1311675372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f64144db0 con 0x7f3f641598b0 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.978+0000 7f3f617fa640 1 -- 172.21.15.139:0/1311675372 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f5c01d460 con 0x7f3f641598b0 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.978+0000 7f3f617fa640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f4806b500 0x7f3f4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.979+0000 7f3f617fa640 1 -- 172.21.15.139:0/1311675372 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3f5c08a7c0 con 0x7f3f641598b0 2024-04-25T09:53:58.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.979+0000 7f3f68ef8640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f4806b500 0x7f3f4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.979+0000 7f3f68ef8640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f4806b500 0x7f3f4806d9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f58033000 tx=0x7f3f58008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:58.983+0000 7f3f617fa640 1 -- 172.21.15.139:0/1311675372 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f64144db0 con 0x7f3f641598b0 2024-04-25T09:53:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.152+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1311675372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3f64074520 con 0x7f3f641598b0 2024-04-25T09:53:59.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.155+0000 7f3f617fa640 1 -- 172.21.15.139:0/1311675372 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3f5c059240 con 0x7f3f641598b0 2024-04-25T09:53:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.157+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1311675372 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f4806b500 msgr2=0x7f3f4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.157+0000 7f3f6b15c640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f4806b500 0x7f3f4806d9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f58033000 tx=0x7f3f58008030 comp rx=0 tx=0).stop 2024-04-25T09:53:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.157+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1311675372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f641598b0 msgr2=0x7f3f6415b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.157+0000 7f3f6b15c640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f641598b0 0x7f3f6415b070 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3f6406efc0 tx=0x7f3f5c007b10 comp rx=0 tx=0).stop 2024-04-25T09:53:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.157+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1311675372 shutdown_connections 2024-04-25T09:53:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.157+0000 7f3f6b15c640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f641598b0 0x7f3f6415b070 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.157+0000 7f3f6b15c640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f4806b500 0x7f3f4806d9c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.157+0000 7f3f6b15c640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f64156d60 0x7f3f64159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.158+0000 7f3f6b15c640 1 --2- 172.21.15.139:0/1311675372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f540039b0 0x7f3f64143bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.158+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1311675372 >> 172.21.15.139:0/1311675372 conn(0x7f3f6406f460 msgr2=0x7f3f6415eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.158+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1311675372 shutdown_connections 2024-04-25T09:53:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.158+0000 7f3f6b15c640 1 -- 172.21.15.139:0/1311675372 wait complete. 2024-04-25T09:53:59.171 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:53:59.171 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:53:59.202 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:53:59.242 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:53:59.242 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:53:59.242 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:59.258 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:53:59.258 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:59.313 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:53:59.313 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:53:59.368 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:53:59.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.502+0000 7f8ef3fd5640 1 Processor -- start 2024-04-25T09:53:59.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.502+0000 7f8ef3fd5640 1 -- start start 2024-04-25T09:53:59.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.503+0000 7f8ef3fd5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8eec06c1f0 0x7f8eec15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.503+0000 7f8ef3fd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eec15c9c0 0x7f8eec0ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.503+0000 7f8ef3fd5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eec160280 con 0x7f8eec0ad570 2024-04-25T09:53:59.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.503+0000 7f8ef3fd5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eec0af870 con 0x7f8eec06c1f0 2024-04-25T09:53:59.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.503+0000 7f8ef3fd5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec15c530 con 0x7f8eec15c9c0 2024-04-25T09:53:59.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.503+0000 7f8ef2572640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8eec0ad570 0x7f8eec06ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54548/0 (socket says 172.21.15.139:54548) 2024-04-25T09:53:59.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.503+0000 7f8ef2572640 1 -- 172.21.15.139:0/717850342 learned_addr learned my addr 172.21.15.139:0/717850342 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:59.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.503+0000 7f8ef1570640 1 --2- 172.21.15.139:0/717850342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eec15c9c0 0x7f8eec0ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.503+0000 7f8ef1d71640 1 --2- 172.21.15.139:0/717850342 >> v2:172.21.15.150:3300/0 conn(0x7f8eec06c1f0 0x7f8eec15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.503+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722376438 0 0) 0x7f8ee4001490 con 0x7f8eec0ad570 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.503+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ed8003520 con 0x7f8eec0ad570 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3729686410 0 0) 0x7f8ed8003520 con 0x7f8eec0ad570 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eec15c9c0 msgr2=0x7f8eec0ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 --2- 172.21.15.139:0/717850342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eec15c9c0 0x7f8eec0ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 >> v2:172.21.15.150:3300/0 conn(0x7f8eec06c1f0 msgr2=0x7f8eec15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 --2- 172.21.15.139:0/717850342 >> v2:172.21.15.150:3300/0 conn(0x7f8eec06c1f0 0x7f8eec15fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eec0af590 con 0x7f8eec0ad570 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f8ee4001bc0 con 0x7f8eec0ad570 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 >> v1:172.21.15.139:6789/0 conn(0x7f8eec0ad570 legacy=0x7f8eec06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 --2- 172.21.15.139:0/717850342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed80039b0 0x7f8ed8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 --2- 172.21.15.139:0/717850342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eec15c9c0 0x7f8ed8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 --2- 172.21.15.139:0/717850342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ed8004d40 0x7f8ed80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec0af870 con 0x7f8ed8004d40 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec15c530 con 0x7f8ed80039b0 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed800b9c0 con 0x7f8eec15c9c0 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef2572640 1 --2- 172.21.15.139:0/717850342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ed8004d40 0x7f8ed80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef1d71640 1 --2- 172.21.15.139:0/717850342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed80039b0 0x7f8ed8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef1570640 1 --2- 172.21.15.139:0/717850342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eec15c9c0 0x7f8ed8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef2572640 1 -- 172.21.15.139:0/717850342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eec15c9c0 msgr2=0x7f8ed8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef2572640 1 --2- 172.21.15.139:0/717850342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eec15c9c0 0x7f8ed8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef2572640 1 -- 172.21.15.139:0/717850342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed80039b0 msgr2=0x7f8ed8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef2572640 1 --2- 172.21.15.139:0/717850342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed80039b0 0x7f8ed8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.504+0000 7f8ef2572640 1 -- 172.21.15.139:0/717850342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed8003520 con 0x7f8ed8004d40 2024-04-25T09:53:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.505+0000 7f8ef2572640 1 --2- 172.21.15.139:0/717850342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ed8004d40 0x7f8ed80051c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400aa50 tx=0x7f8ed8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2bf5790d521eb94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.505+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400a670 con 0x7f8ed8004d40 2024-04-25T09:53:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.505+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee402f070 con 0x7f8ed8004d40 2024-04-25T09:53:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.505+0000 7f8ef0d6f640 1 -- 172.21.15.139:0/717850342 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4034470 con 0x7f8ed8004d40 2024-04-25T09:53:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.505+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/717850342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ed8004d40 msgr2=0x7f8ed80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.505+0000 7f8ef3fd5640 1 --2- 172.21.15.139:0/717850342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ed8004d40 0x7f8ed80051c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400aa50 tx=0x7f8ed8005830 comp rx=0 tx=0).stop 2024-04-25T09:53:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.505+0000 7f8ef2572640 1 -- 172.21.15.139:0/717850342 reap_dead start 2024-04-25T09:53:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.505+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/717850342 shutdown_connections 2024-04-25T09:53:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.506+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/717850342 >> 172.21.15.139:0/717850342 conn(0x7f8eec0b23d0 msgr2=0x7f8eec0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.506+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/717850342 shutdown_connections 2024-04-25T09:53:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.506+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/717850342 wait complete. 2024-04-25T09:53:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.507+0000 7f8ef3fd5640 1 Processor -- start 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.507+0000 7f8ef3fd5640 1 -- start start 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.507+0000 7f8ef3fd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ed8004d40 0x7f8eec155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.507+0000 7f8ef3fd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eec0b38c0 0x7f8eec1f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.507+0000 7f8ef3fd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eec1f7390 0x7f8eec1f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.507+0000 7f8ef3fd5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec06f630 con 0x7f8eec0b38c0 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.507+0000 7f8ef3fd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec15c1c0 con 0x7f8eec1f7390 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.507+0000 7f8ef3fd5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec06f7b0 con 0x7f8ed8004d40 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef1570640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eec0b38c0 0x7f8eec1f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef1570640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eec0b38c0 0x7f8eec1f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47238/0 (socket says 172.21.15.139:47238) 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef1570640 1 -- 172.21.15.139:0/2413727698 learned_addr learned my addr 172.21.15.139:0/2413727698 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef1d71640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ed8004d40 0x7f8eec155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef2572640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eec1f7390 0x7f8eec1f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef1570640 1 -- 172.21.15.139:0/2413727698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ed8004d40 msgr2=0x7f8eec155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef1570640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ed8004d40 0x7f8eec155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef1570640 1 -- 172.21.15.139:0/2413727698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eec1f7390 msgr2=0x7f8eec1f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef1570640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eec1f7390 0x7f8eec1f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef1570640 1 -- 172.21.15.139:0/2413727698 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eec0b3da0 con 0x7f8eec0b38c0 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef1d71640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ed8004d40 0x7f8eec155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef1570640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eec0b38c0 0x7f8eec1f6e50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0119a0 tx=0x7f8edc011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ee27fc640 1 -- 172.21.15.139:0/2413727698 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc00faf0 con 0x7f8eec0b38c0 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ee27fc640 1 -- 172.21.15.139:0/2413727698 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8edc00fc90 con 0x7f8eec0b38c0 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ee27fc640 1 -- 172.21.15.139:0/2413727698 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc014440 con 0x7f8eec0b38c0 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/2413727698 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eec0b4080 con 0x7f8eec0b38c0 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.508+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/2413727698 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eec0b4640 con 0x7f8eec0b38c0 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.509+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/2413727698 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eec074520 con 0x7f8eec0b38c0 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.510+0000 7f8ee27fc640 1 -- 172.21.15.139:0/2413727698 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8edc012040 con 0x7f8eec0b38c0 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.510+0000 7f8ee27fc640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ed006b4d0 0x7f8ed006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.510+0000 7f8ef1d71640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ed006b4d0 0x7f8ed006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.510+0000 7f8ee27fc640 1 -- 172.21.15.139:0/2413727698 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8edc08ad10 con 0x7f8eec0b38c0 2024-04-25T09:53:59.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.511+0000 7f8ef1d71640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ed006b4d0 0x7f8ed006d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0b5090 tx=0x7f8ee8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.514+0000 7f8ee27fc640 1 -- 172.21.15.139:0/2413727698 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8edc019030 con 0x7f8eec0b38c0 2024-04-25T09:53:59.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.686+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/2413727698 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8eec0cfaf0 con 0x7f8eec0b38c0 2024-04-25T09:53:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.687+0000 7f8ee27fc640 1 -- 172.21.15.139:0/2413727698 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18951 (secure 0 0 0) 0x7f8edc059790 con 0x7f8eec0b38c0 2024-04-25T09:53:59.689 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:53:59.690 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":26,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:56.839903+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:53:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.689+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/2413727698 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ed006b4d0 msgr2=0x7f8ed006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.689+0000 7f8ef3fd5640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ed006b4d0 0x7f8ed006d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0b5090 tx=0x7f8ee8008040 comp rx=0 tx=0).stop 2024-04-25T09:53:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.689+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/2413727698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eec0b38c0 msgr2=0x7f8eec1f6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.689+0000 7f8ef3fd5640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eec0b38c0 0x7f8eec1f6e50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0119a0 tx=0x7f8edc011e60 comp rx=0 tx=0).stop 2024-04-25T09:53:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.689+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/2413727698 shutdown_connections 2024-04-25T09:53:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.689+0000 7f8ef3fd5640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eec0b38c0 0x7f8eec1f6e50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.689+0000 7f8ef3fd5640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ed006b4d0 0x7f8ed006d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.689+0000 7f8ef3fd5640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eec1f7390 0x7f8eec1f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.689+0000 7f8ef3fd5640 1 --2- 172.21.15.139:0/2413727698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ed8004d40 0x7f8eec155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.689+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/2413727698 >> 172.21.15.139:0/2413727698 conn(0x7f8eec0b23d0 msgr2=0x7f8eec06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.689+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/2413727698 shutdown_connections 2024-04-25T09:53:59.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.689+0000 7f8ef3fd5640 1 -- 172.21.15.139:0/2413727698 wait complete. 2024-04-25T09:53:59.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:53:59.703 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:53:59.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.803+0000 7ff2440b3640 1 Processor -- start 2024-04-25T09:53:59.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.804+0000 7ff2440b3640 1 -- start start 2024-04-25T09:53:59.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.804+0000 7ff2440b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff23c1617e0 0x7ff23c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.804+0000 7ff2440b3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff23c0cec80 0x7ff23c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.804+0000 7ff2440b3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c161dc0 con 0x7ff23c15dbf0 2024-04-25T09:53:59.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.804+0000 7ff2440b3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23c0d1ee0 con 0x7ff23c0cec80 2024-04-25T09:53:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.804+0000 7ff2440b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c1620a0 con 0x7ff23c1617e0 2024-04-25T09:53:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.804+0000 7ff242650640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff23c0cec80 0x7ff23c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.804+0000 7ff241e4f640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff23c15dbf0 0x7ff23c0b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54558/0 (socket says 172.21.15.139:54558) 2024-04-25T09:53:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.804+0000 7ff241e4f640 1 -- 172.21.15.139:0/3436917451 learned_addr learned my addr 172.21.15.139:0/3436917451 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:53:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.804+0000 7ff242650640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff23c0cec80 0x7ff23c0d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47242/0 (socket says 172.21.15.139:47242) 2024-04-25T09:53:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.805+0000 7ff242650640 1 -- 172.21.15.139:0/3436917451 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff23c1617e0 msgr2=0x7ff23c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.805+0000 7ff24164e640 1 --2- 172.21.15.139:0/3436917451 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff23c1617e0 0x7ff23c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.805+0000 7ff240e4d640 1 -- 172.21.15.139:0/3436917451 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072855975 0 0) 0x7ff23c161dc0 con 0x7ff23c15dbf0 2024-04-25T09:53:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.805+0000 7ff242650640 1 --2- 172.21.15.139:0/3436917451 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff23c1617e0 0x7ff23c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.805+0000 7ff242650640 1 -- 172.21.15.139:0/3436917451 >> v1:172.21.15.139:6789/0 conn(0x7ff23c15dbf0 legacy=0x7ff23c0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.805+0000 7ff242650640 1 -- 172.21.15.139:0/3436917451 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c162380 con 0x7ff23c0cec80 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.805+0000 7ff24164e640 1 --2- 172.21.15.139:0/3436917451 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff23c1617e0 0x7ff23c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.805+0000 7ff242650640 1 --2- 172.21.15.139:0/3436917451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c0cec80 0x7ff23c0d2370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff2340097a0 tx=0x7ff23402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6140c868437ce0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.806+0000 7ff240e4d640 1 -- 172.21.15.139:0/3436917451 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234037070 con 0x7ff23c0cec80 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.806+0000 7ff240e4d640 1 -- 172.21.15.139:0/3436917451 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23402ae50 con 0x7ff23c0cec80 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.806+0000 7ff240e4d640 1 -- 172.21.15.139:0/3436917451 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234032660 con 0x7ff23c0cec80 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.806+0000 7ff2440b3640 1 -- 172.21.15.139:0/3436917451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c0cec80 msgr2=0x7ff23c0d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.806+0000 7ff2440b3640 1 --2- 172.21.15.139:0/3436917451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c0cec80 0x7ff23c0d2370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff2340097a0 tx=0x7ff23402a100 comp rx=0 tx=0).stop 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.806+0000 7ff2440b3640 1 -- 172.21.15.139:0/3436917451 shutdown_connections 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.806+0000 7ff2440b3640 1 --2- 172.21.15.139:0/3436917451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c0cec80 0x7ff23c0d2370 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.806+0000 7ff2440b3640 1 --2- 172.21.15.139:0/3436917451 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff23c1617e0 0x7ff23c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.806+0000 7ff2440b3640 1 -- 172.21.15.139:0/3436917451 >> 172.21.15.139:0/3436917451 conn(0x7ff23c0b5480 msgr2=0x7ff23c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.806+0000 7ff2440b3640 1 -- 172.21.15.139:0/3436917451 shutdown_connections 2024-04-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.806+0000 7ff2440b3640 1 -- 172.21.15.139:0/3436917451 wait complete. 2024-04-25T09:53:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.807+0000 7ff2440b3640 1 Processor -- start 2024-04-25T09:53:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.807+0000 7ff2440b3640 1 -- start start 2024-04-25T09:53:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff2440b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff23c0cec80 0x7ff23c0c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff2440b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c15dbf0 0x7ff23c151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff2440b3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff23c1617e0 0x7ff23c1f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff2440b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c15dfd0 con 0x7ff23c15dbf0 2024-04-25T09:53:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff2440b3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0d28b0 con 0x7ff23c1617e0 2024-04-25T09:53:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff2440b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c152380 con 0x7ff23c0cec80 2024-04-25T09:53:59.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff24164e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c15dbf0 0x7ff23c151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff24164e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c15dbf0 0x7ff23c151e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47252/0 (socket says 172.21.15.139:47252) 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff24164e640 1 -- 172.21.15.139:0/1403963313 learned_addr learned my addr 172.21.15.139:0/1403963313 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff24164e640 1 -- 172.21.15.139:0/1403963313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff23c0cec80 msgr2=0x7ff23c0c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff24164e640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff23c0cec80 0x7ff23c0c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff242650640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff23c1617e0 0x7ff23c1f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff24164e640 1 -- 172.21.15.139:0/1403963313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff23c1617e0 msgr2=0x7ff23c1f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff24164e640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff23c1617e0 0x7ff23c1f3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.808+0000 7ff24164e640 1 -- 172.21.15.139:0/1403963313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c14f640 con 0x7ff23c15dbf0 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.809+0000 7ff24164e640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c15dbf0 0x7ff23c151e40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c820 tx=0x7ff22c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.809+0000 7ff2327fc640 1 -- 172.21.15.139:0/1403963313 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c015070 con 0x7ff23c15dbf0 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.809+0000 7ff2327fc640 1 -- 172.21.15.139:0/1403963313 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c007a30 con 0x7ff23c15dbf0 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.809+0000 7ff2440b3640 1 -- 172.21.15.139:0/1403963313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23c14f360 con 0x7ff23c15dbf0 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.809+0000 7ff2327fc640 1 -- 172.21.15.139:0/1403963313 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c007de0 con 0x7ff23c15dbf0 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.809+0000 7ff2440b3640 1 -- 172.21.15.139:0/1403963313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23c160110 con 0x7ff23c15dbf0 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.810+0000 7ff2440b3640 1 -- 172.21.15.139:0/1403963313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23c074520 con 0x7ff23c15dbf0 2024-04-25T09:53:59.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.810+0000 7ff2327fc640 1 -- 172.21.15.139:0/1403963313 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff22c018070 con 0x7ff23c15dbf0 2024-04-25T09:53:59.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.811+0000 7ff2327fc640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff22006b4d0 0x7ff22006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.811+0000 7ff2327fc640 1 -- 172.21.15.139:0/1403963313 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff22c08a400 con 0x7ff23c15dbf0 2024-04-25T09:53:59.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.811+0000 7ff241e4f640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff22006b4d0 0x7ff22006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.811+0000 7ff241e4f640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff22006b4d0 0x7ff22006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff238001560 tx=0x7ff238007ca0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.815+0000 7ff2327fc640 1 -- 172.21.15.139:0/1403963313 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff22c058f10 con 0x7ff23c15dbf0 2024-04-25T09:53:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.989+0000 7ff2440b3640 1 -- 172.21.15.139:0/1403963313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff23c0d1680 con 0x7ff23c15dbf0 2024-04-25T09:53:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.989+0000 7ff2327fc640 1 -- 172.21.15.139:0/1403963313 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff22c058810 con 0x7ff23c15dbf0 2024-04-25T09:53:59.991 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:53:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.991+0000 7ff2440b3640 1 -- 172.21.15.139:0/1403963313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff22006b4d0 msgr2=0x7ff22006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.991+0000 7ff2440b3640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff22006b4d0 0x7ff22006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff238001560 tx=0x7ff238007ca0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.991+0000 7ff2440b3640 1 -- 172.21.15.139:0/1403963313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c15dbf0 msgr2=0x7ff23c151e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.991+0000 7ff2440b3640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c15dbf0 0x7ff23c151e40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c820 tx=0x7ff22c00cce0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.991+0000 7ff2440b3640 1 -- 172.21.15.139:0/1403963313 shutdown_connections 2024-04-25T09:53:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.991+0000 7ff2440b3640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c15dbf0 0x7ff23c151e40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.991+0000 7ff2440b3640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff22006b4d0 0x7ff22006d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.991+0000 7ff2440b3640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff23c1617e0 0x7ff23c1f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.991+0000 7ff2440b3640 1 --2- 172.21.15.139:0/1403963313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff23c0cec80 0x7ff23c0c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.991+0000 7ff2440b3640 1 -- 172.21.15.139:0/1403963313 >> 172.21.15.139:0/1403963313 conn(0x7ff23c0b5480 msgr2=0x7ff23c0b6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.991+0000 7ff2440b3640 1 -- 172.21.15.139:0/1403963313 shutdown_connections 2024-04-25T09:53:59.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:53:59.991+0000 7ff2440b3640 1 -- 172.21.15.139:0/1403963313 wait complete. 2024-04-25T09:54:00.004 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:54:00.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.103+0000 7f579a9b7640 1 Processor -- start 2024-04-25T09:54:00.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.103+0000 7f579a9b7640 1 -- start start 2024-04-25T09:54:00.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f579a9b7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f57940b31a0 0x7f579406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f579a9b7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f579406e870 0x7f579415e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f579a9b7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57940b2a50 con 0x7f5794144090 2024-04-25T09:54:00.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f579a9b7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579406ece0 con 0x7f57940b31a0 2024-04-25T09:54:00.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f579a9b7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57940b3610 con 0x7f579406e870 2024-04-25T09:54:00.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f57937fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f57940b31a0 0x7f579406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f5793fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5794144090 0x7f57940cc9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54560/0 (socket says 172.21.15.139:54560) 2024-04-25T09:54:00.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f5793fff640 1 -- 172.21.15.139:0/2745347044 learned_addr learned my addr 172.21.15.139:0/2745347044 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:00.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f5798f54640 1 --2- 172.21.15.139:0/2745347044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f579406e870 0x7f579415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f57937fe640 1 -- 172.21.15.139:0/2745347044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f579406e870 msgr2=0x7f579415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f57937fe640 1 --2- 172.21.15.139:0/2745347044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f579406e870 0x7f579415e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f57937fe640 1 -- 172.21.15.139:0/2745347044 >> v1:172.21.15.139:6789/0 conn(0x7f5794144090 legacy=0x7f57940cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f57937fe640 1 -- 172.21.15.139:0/2745347044 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57940cd0f0 con 0x7f57940b31a0 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f5792ffd640 1 -- 172.21.15.139:0/2745347044 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521338010 0 0) 0x7f577c001110 con 0x7f5794144090 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.104+0000 7f5798f54640 1 --2- 172.21.15.139:0/2745347044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f579406e870 0x7f579415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.105+0000 7f57937fe640 1 --2- 172.21.15.139:0/2745347044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57940b31a0 0x7f579406e330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f57840097a0 tx=0x7f578402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ac0adca54c5a34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.105+0000 7f5792ffd640 1 -- 172.21.15.139:0/2745347044 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784037070 con 0x7f57940b31a0 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.105+0000 7f5792ffd640 1 -- 172.21.15.139:0/2745347044 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578402ae50 con 0x7f57940b31a0 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.105+0000 7f5792ffd640 1 -- 172.21.15.139:0/2745347044 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784032660 con 0x7f57940b31a0 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.106+0000 7f579a9b7640 1 -- 172.21.15.139:0/2745347044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57940b31a0 msgr2=0x7f579406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.106+0000 7f579a9b7640 1 --2- 172.21.15.139:0/2745347044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57940b31a0 0x7f579406e330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f57840097a0 tx=0x7f578402a100 comp rx=0 tx=0).stop 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.106+0000 7f579a9b7640 1 -- 172.21.15.139:0/2745347044 shutdown_connections 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.106+0000 7f579a9b7640 1 --2- 172.21.15.139:0/2745347044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f579406e870 0x7f579415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.106+0000 7f579a9b7640 1 --2- 172.21.15.139:0/2745347044 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57940b31a0 0x7f579406e330 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.106+0000 7f579a9b7640 1 -- 172.21.15.139:0/2745347044 >> 172.21.15.139:0/2745347044 conn(0x7f5794067000 msgr2=0x7f579406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.106+0000 7f579a9b7640 1 -- 172.21.15.139:0/2745347044 shutdown_connections 2024-04-25T09:54:00.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.106+0000 7f579a9b7640 1 -- 172.21.15.139:0/2745347044 wait complete. 2024-04-25T09:54:00.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.107+0000 7f579a9b7640 1 Processor -- start 2024-04-25T09:54:00.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.107+0000 7f579a9b7640 1 -- start start 2024-04-25T09:54:00.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f579a9b7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f579406e870 0x7f57940ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f579a9b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57940b31a0 0x7f579415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f579a9b7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5794144090 0x7f57940ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f579a9b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579415b650 con 0x7f57940b31a0 2024-04-25T09:54:00.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f579a9b7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579415b370 con 0x7f579406e870 2024-04-25T09:54:00.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f579a9b7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579415b930 con 0x7f5794144090 2024-04-25T09:54:00.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f57937fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57940b31a0 0x7f579415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f57937fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57940b31a0 0x7f579415a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47278/0 (socket says 172.21.15.139:47278) 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f57937fe640 1 -- 172.21.15.139:0/874334060 learned_addr learned my addr 172.21.15.139:0/874334060 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f5798f54640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5794144090 0x7f57940ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f5793fff640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f579406e870 0x7f57940ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f57937fe640 1 -- 172.21.15.139:0/874334060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5794144090 msgr2=0x7f57940ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f57937fe640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5794144090 0x7f57940ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f57937fe640 1 -- 172.21.15.139:0/874334060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f579406e870 msgr2=0x7f57940ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f57937fe640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f579406e870 0x7f57940ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f57937fe640 1 -- 172.21.15.139:0/874334060 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57940ad110 con 0x7f57940b31a0 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f57937fe640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57940b31a0 0x7f579415a020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5784035040 tx=0x7f57840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f5790ff9640 1 -- 172.21.15.139:0/874334060 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784041070 con 0x7f57940b31a0 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f5790ff9640 1 -- 172.21.15.139:0/874334060 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578403c670 con 0x7f57940b31a0 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f579a9b7640 1 -- 172.21.15.139:0/874334060 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57940ad3f0 con 0x7f57940b31a0 2024-04-25T09:54:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f5790ff9640 1 -- 172.21.15.139:0/874334060 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784037040 con 0x7f57940b31a0 2024-04-25T09:54:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.108+0000 7f579a9b7640 1 -- 172.21.15.139:0/874334060 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57940ad9b0 con 0x7f57940b31a0 2024-04-25T09:54:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.110+0000 7f579a9b7640 1 -- 172.21.15.139:0/874334060 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5794074520 con 0x7f57940b31a0 2024-04-25T09:54:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.110+0000 7f5790ff9640 1 -- 172.21.15.139:0/874334060 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f578403a070 con 0x7f57940b31a0 2024-04-25T09:54:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.110+0000 7f5790ff9640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f577406b4d0 0x7f577406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.111+0000 7f5790ff9640 1 -- 172.21.15.139:0/874334060 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f57840aa790 con 0x7f57940b31a0 2024-04-25T09:54:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.111+0000 7f5793fff640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f577406b4d0 0x7f577406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.111+0000 7f5793fff640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f577406b4d0 0x7f577406d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f577c000f60 tx=0x7f577c007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.115+0000 7f5790ff9640 1 -- 172.21.15.139:0/874334060 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5784078bc0 con 0x7f57940b31a0 2024-04-25T09:54:00.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.282+0000 7f579a9b7640 1 -- 172.21.15.139:0/874334060 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f579406ac00 con 0x7f57940b31a0 2024-04-25T09:54:00.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.283+0000 7f5790ff9640 1 -- 172.21.15.139:0/874334060 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f578407cc60 con 0x7f57940b31a0 2024-04-25T09:54:00.285 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:54:00.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.285+0000 7f579a9b7640 1 -- 172.21.15.139:0/874334060 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f577406b4d0 msgr2=0x7f577406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.285+0000 7f579a9b7640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f577406b4d0 0x7f577406d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f577c000f60 tx=0x7f577c007b50 comp rx=0 tx=0).stop 2024-04-25T09:54:00.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.285+0000 7f579a9b7640 1 -- 172.21.15.139:0/874334060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57940b31a0 msgr2=0x7f579415a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.285+0000 7f579a9b7640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57940b31a0 0x7f579415a020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5784035040 tx=0x7f57840046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.285+0000 7f579a9b7640 1 -- 172.21.15.139:0/874334060 shutdown_connections 2024-04-25T09:54:00.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.285+0000 7f579a9b7640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57940b31a0 0x7f579415a020 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.285+0000 7f579a9b7640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f577406b4d0 0x7f577406d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.285+0000 7f579a9b7640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5794144090 0x7f57940ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.285+0000 7f579a9b7640 1 --2- 172.21.15.139:0/874334060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f579406e870 0x7f57940ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.285+0000 7f579a9b7640 1 -- 172.21.15.139:0/874334060 >> 172.21.15.139:0/874334060 conn(0x7f5794067000 msgr2=0x7f579406e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.285+0000 7f579a9b7640 1 -- 172.21.15.139:0/874334060 shutdown_connections 2024-04-25T09:54:00.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.285+0000 7f579a9b7640 1 -- 172.21.15.139:0/874334060 wait complete. 2024-04-25T09:54:00.298 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:54:00.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.396+0000 7ff984ab6640 1 Processor -- start 2024-04-25T09:54:00.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.396+0000 7ff984ab6640 1 -- start start 2024-04-25T09:54:00.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.396+0000 7ff984ab6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff9800cd610 0x7ff9800d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.396+0000 7ff984ab6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff980161fd0 0x7ff9800d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.396+0000 7ff984ab6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff980099ca0 con 0x7ff98009ad40 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.396+0000 7ff984ab6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff980099a50 con 0x7ff9800cd610 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.396+0000 7ff984ab6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff980099f80 con 0x7ff980161fd0 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.397+0000 7ff97e59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff9800cd610 0x7ff9800d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.397+0000 7ff97e59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff9800cd610 0x7ff9800d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47294/0 (socket says 172.21.15.139:47294) 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.397+0000 7ff975d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff98009ad40 0x7ff9801618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54576/0 (socket says 172.21.15.139:54576) 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.397+0000 7ff97e59c640 1 -- 172.21.15.139:0/106321686 learned_addr learned my addr 172.21.15.139:0/106321686 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.397+0000 7ff97ed9d640 1 --2- 172.21.15.139:0/106321686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff980161fd0 0x7ff9800d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.397+0000 7ff97e59c640 1 -- 172.21.15.139:0/106321686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff980161fd0 msgr2=0x7ff9800d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.397+0000 7ff97e59c640 1 --2- 172.21.15.139:0/106321686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff980161fd0 0x7ff9800d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.397+0000 7ff97e59c640 1 -- 172.21.15.139:0/106321686 >> v1:172.21.15.139:6789/0 conn(0x7ff98009ad40 legacy=0x7ff9801618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.397+0000 7ff97e59c640 1 -- 172.21.15.139:0/106321686 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98009a260 con 0x7ff9800cd610 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.398+0000 7ff97e59c640 1 --2- 172.21.15.139:0/106321686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9800cd610 0x7ff9800d0370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff970009870 tx=0x7ff97002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1645e09c32235b3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.398+0000 7ff97dd9b640 1 -- 172.21.15.139:0/106321686 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970037070 con 0x7ff9800cd610 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.398+0000 7ff97dd9b640 1 -- 172.21.15.139:0/106321686 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97002ae50 con 0x7ff9800cd610 2024-04-25T09:54:00.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.398+0000 7ff97dd9b640 1 -- 172.21.15.139:0/106321686 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970032860 con 0x7ff9800cd610 2024-04-25T09:54:00.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.398+0000 7ff984ab6640 1 -- 172.21.15.139:0/106321686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9800cd610 msgr2=0x7ff9800d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.398+0000 7ff984ab6640 1 --2- 172.21.15.139:0/106321686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9800cd610 0x7ff9800d0370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff970009870 tx=0x7ff97002a300 comp rx=0 tx=0).stop 2024-04-25T09:54:00.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.398+0000 7ff984ab6640 1 -- 172.21.15.139:0/106321686 shutdown_connections 2024-04-25T09:54:00.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.398+0000 7ff984ab6640 1 --2- 172.21.15.139:0/106321686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff980161fd0 0x7ff9800d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.398+0000 7ff984ab6640 1 --2- 172.21.15.139:0/106321686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9800cd610 0x7ff9800d0370 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.398+0000 7ff984ab6640 1 -- 172.21.15.139:0/106321686 >> 172.21.15.139:0/106321686 conn(0x7ff9800b31f0 msgr2=0x7ff9800b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.398+0000 7ff984ab6640 1 -- 172.21.15.139:0/106321686 shutdown_connections 2024-04-25T09:54:00.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.398+0000 7ff984ab6640 1 -- 172.21.15.139:0/106321686 wait complete. 2024-04-25T09:54:00.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.400+0000 7ff984ab6640 1 Processor -- start 2024-04-25T09:54:00.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.400+0000 7ff984ab6640 1 -- start start 2024-04-25T09:54:00.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.400+0000 7ff984ab6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff98009ad40 0x7ff98015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.400+0000 7ff984ab6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9800cd610 0x7ff9800c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.400+0000 7ff984ab6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff980161fd0 0x7ff9800af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.400+0000 7ff984ab6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800cd9f0 con 0x7ff980161fd0 2024-04-25T09:54:00.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.400+0000 7ff984ab6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98009a820 con 0x7ff98009ad40 2024-04-25T09:54:00.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.400+0000 7ff984ab6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800cdb70 con 0x7ff9800cd610 2024-04-25T09:54:00.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff97ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff980161fd0 0x7ff9800af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff97ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff980161fd0 0x7ff9800af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47308/0 (socket says 172.21.15.139:47308) 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff97ed9d640 1 -- 172.21.15.139:0/2160156503 learned_addr learned my addr 172.21.15.139:0/2160156503 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff975d9b640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9800cd610 0x7ff9800c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff97e59c640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff98009ad40 0x7ff98015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff97ed9d640 1 -- 172.21.15.139:0/2160156503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9800cd610 msgr2=0x7ff9800c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff97ed9d640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9800cd610 0x7ff9800c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff97ed9d640 1 -- 172.21.15.139:0/2160156503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff98009ad40 msgr2=0x7ff98015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff97ed9d640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff98009ad40 0x7ff98015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff97ed9d640 1 -- 172.21.15.139:0/2160156503 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800c6700 con 0x7ff980161fd0 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff97e59c640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff98009ad40 0x7ff98015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff97ed9d640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff980161fd0 0x7ff9800af520 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff97800ef90 tx=0x7ff97800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff9777fe640 1 -- 172.21.15.139:0/2160156503 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978018020 con 0x7ff980161fd0 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff9777fe640 1 -- 172.21.15.139:0/2160156503 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff978014480 con 0x7ff980161fd0 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff984ab6640 1 -- 172.21.15.139:0/2160156503 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9800c69e0 con 0x7ff980161fd0 2024-04-25T09:54:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff9777fe640 1 -- 172.21.15.139:0/2160156503 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978009e60 con 0x7ff980161fd0 2024-04-25T09:54:00.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.401+0000 7ff984ab6640 1 -- 172.21.15.139:0/2160156503 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9800b5f30 con 0x7ff980161fd0 2024-04-25T09:54:00.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.402+0000 7ff984ab6640 1 -- 172.21.15.139:0/2160156503 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff980067420 con 0x7ff980161fd0 2024-04-25T09:54:00.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.403+0000 7ff9777fe640 1 -- 172.21.15.139:0/2160156503 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff978005d10 con 0x7ff980161fd0 2024-04-25T09:54:00.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.403+0000 7ff9777fe640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff96006b500 0x7ff96006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.403+0000 7ff97e59c640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff96006b500 0x7ff96006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.403+0000 7ff9777fe640 1 -- 172.21.15.139:0/2160156503 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff97808b2c0 con 0x7ff980161fd0 2024-04-25T09:54:00.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.404+0000 7ff97e59c640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff96006b500 0x7ff96006d9c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff970034000 tx=0x7ff97002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.407+0000 7ff9777fe640 1 -- 172.21.15.139:0/2160156503 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff980067420 con 0x7ff980161fd0 2024-04-25T09:54:00.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.575+0000 7ff984ab6640 1 -- 172.21.15.139:0/2160156503 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff980074520 con 0x7ff980161fd0 2024-04-25T09:54:00.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.576+0000 7ff9777fe640 1 -- 172.21.15.139:0/2160156503 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff978022030 con 0x7ff980161fd0 2024-04-25T09:54:00.578 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:54:00.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.578+0000 7ff984ab6640 1 -- 172.21.15.139:0/2160156503 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff96006b500 msgr2=0x7ff96006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.578+0000 7ff984ab6640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff96006b500 0x7ff96006d9c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff970034000 tx=0x7ff97002e000 comp rx=0 tx=0).stop 2024-04-25T09:54:00.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.578+0000 7ff984ab6640 1 -- 172.21.15.139:0/2160156503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff980161fd0 msgr2=0x7ff9800af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.578+0000 7ff984ab6640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff980161fd0 0x7ff9800af520 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff97800ef90 tx=0x7ff97800c490 comp rx=0 tx=0).stop 2024-04-25T09:54:00.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.578+0000 7ff984ab6640 1 -- 172.21.15.139:0/2160156503 shutdown_connections 2024-04-25T09:54:00.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.578+0000 7ff984ab6640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff980161fd0 0x7ff9800af520 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.578+0000 7ff984ab6640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff96006b500 0x7ff96006d9c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.578+0000 7ff984ab6640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9800cd610 0x7ff9800c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.578+0000 7ff984ab6640 1 --2- 172.21.15.139:0/2160156503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff98009ad40 0x7ff98015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.578+0000 7ff984ab6640 1 -- 172.21.15.139:0/2160156503 >> 172.21.15.139:0/2160156503 conn(0x7ff9800b31f0 msgr2=0x7ff9800cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.578+0000 7ff984ab6640 1 -- 172.21.15.139:0/2160156503 shutdown_connections 2024-04-25T09:54:00.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.578+0000 7ff984ab6640 1 -- 172.21.15.139:0/2160156503 wait complete. 2024-04-25T09:54:00.591 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:54:00.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.688+0000 7f363ebcf640 1 Processor -- start 2024-04-25T09:54:00.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.689+0000 7f363ebcf640 1 -- start start 2024-04-25T09:54:00.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.689+0000 7f363ebcf640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f363815da30 0x7f36380704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.689+0000 7f363ebcf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3638072d60 0x7f363806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.689+0000 7f363ebcf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36380731a0 con 0x7f363806b0a0 2024-04-25T09:54:00.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.689+0000 7f363ebcf640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36380cd540 con 0x7f363815da30 2024-04-25T09:54:00.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.689+0000 7f363ebcf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36380b3de0 con 0x7f3638072d60 2024-04-25T09:54:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.689+0000 7f363d16c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f363806b0a0 0x7f36380d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54580/0 (socket says 172.21.15.139:54580) 2024-04-25T09:54:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.690+0000 7f363d16c640 1 -- 172.21.15.139:0/2014690352 learned_addr learned my addr 172.21.15.139:0/2014690352 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.690+0000 7f3637fff640 1 --2- 172.21.15.139:0/2014690352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3638072d60 0x7f363806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.690+0000 7f363c96b640 1 --2- 172.21.15.139:0/2014690352 >> v2:172.21.15.150:3300/0 conn(0x7f363815da30 0x7f36380704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.690+0000 7f36377fe640 1 -- 172.21.15.139:0/2014690352 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 860670150 0 0) 0x7f36380731a0 con 0x7f363806b0a0 2024-04-25T09:54:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.690+0000 7f36377fe640 1 -- 172.21.15.139:0/2014690352 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3620003520 con 0x7f363806b0a0 2024-04-25T09:54:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.690+0000 7f3637fff640 1 -- 172.21.15.139:0/2014690352 >> v2:172.21.15.150:3300/0 conn(0x7f363815da30 msgr2=0x7f36380704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.690+0000 7f36377fe640 1 -- 172.21.15.139:0/2014690352 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1912839938 0 0) 0x7f3620003520 con 0x7f363806b0a0 2024-04-25T09:54:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.690+0000 7f3637fff640 1 --2- 172.21.15.139:0/2014690352 >> v2:172.21.15.150:3300/0 conn(0x7f363815da30 0x7f36380704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.690+0000 7f3637fff640 1 -- 172.21.15.139:0/2014690352 >> v1:172.21.15.139:6789/0 conn(0x7f363806b0a0 legacy=0x7f36380d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.690+0000 7f3637fff640 1 -- 172.21.15.139:0/2014690352 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363815de10 con 0x7f3638072d60 2024-04-25T09:54:00.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.691+0000 7f3637fff640 1 --2- 172.21.15.139:0/2014690352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3638072d60 0x7f363806ab60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f362c0097a0 tx=0x7f362c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06c51554cf1b9e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.691+0000 7f36377fe640 1 -- 172.21.15.139:0/2014690352 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c037070 con 0x7f3638072d60 2024-04-25T09:54:00.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.691+0000 7f36377fe640 1 -- 172.21.15.139:0/2014690352 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c02aba0 con 0x7f3638072d60 2024-04-25T09:54:00.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.691+0000 7f36377fe640 1 -- 172.21.15.139:0/2014690352 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36300019f0 con 0x7f3638072d60 2024-04-25T09:54:00.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.691+0000 7f363ebcf640 1 -- 172.21.15.139:0/2014690352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3638072d60 msgr2=0x7f363806ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.691+0000 7f363ebcf640 1 --2- 172.21.15.139:0/2014690352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3638072d60 0x7f363806ab60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f362c0097a0 tx=0x7f362c02a110 comp rx=0 tx=0).stop 2024-04-25T09:54:00.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.691+0000 7f363ebcf640 1 -- 172.21.15.139:0/2014690352 shutdown_connections 2024-04-25T09:54:00.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.691+0000 7f363ebcf640 1 --2- 172.21.15.139:0/2014690352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3638072d60 0x7f363806ab60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.691+0000 7f363ebcf640 1 --2- 172.21.15.139:0/2014690352 >> v2:172.21.15.150:3300/0 conn(0x7f363815da30 0x7f36380704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.691+0000 7f363ebcf640 1 -- 172.21.15.139:0/2014690352 >> 172.21.15.139:0/2014690352 conn(0x7f36380b23f0 msgr2=0x7f36380b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.692+0000 7f363ebcf640 1 -- 172.21.15.139:0/2014690352 shutdown_connections 2024-04-25T09:54:00.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.692+0000 7f363ebcf640 1 -- 172.21.15.139:0/2014690352 wait complete. 2024-04-25T09:54:00.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.693+0000 7f363ebcf640 1 Processor -- start 2024-04-25T09:54:00.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.693+0000 7f363ebcf640 1 -- start start 2024-04-25T09:54:00.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f363ebcf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363806b0a0 0x7f3638148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f363ebcf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3638072d60 0x7f3638158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f363ebcf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363815da30 0x7f3638159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f363ebcf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36380b42a0 con 0x7f3638072d60 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f363ebcf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36380b3b00 con 0x7f363806b0a0 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f363ebcf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36380b4580 con 0x7f363815da30 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f3637fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3638072d60 0x7f3638158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f3637fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3638072d60 0x7f3638158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47322/0 (socket says 172.21.15.139:47322) 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f3637fff640 1 -- 172.21.15.139:0/4225625898 learned_addr learned my addr 172.21.15.139:0/4225625898 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f363c96b640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363806b0a0 0x7f3638148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f363d16c640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363815da30 0x7f3638159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f3637fff640 1 -- 172.21.15.139:0/4225625898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363815da30 msgr2=0x7f3638159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f3637fff640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363815da30 0x7f3638159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f3637fff640 1 -- 172.21.15.139:0/4225625898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363806b0a0 msgr2=0x7f3638148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f3637fff640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363806b0a0 0x7f3638148290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f3637fff640 1 -- 172.21.15.139:0/4225625898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36380ca8f0 con 0x7f3638072d60 2024-04-25T09:54:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f3637fff640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3638072d60 0x7f3638158af0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f362c0097a0 tx=0x7f362c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f36357fa640 1 -- 172.21.15.139:0/4225625898 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c037050 con 0x7f3638072d60 2024-04-25T09:54:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f36357fa640 1 -- 172.21.15.139:0/4225625898 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c043070 con 0x7f3638072d60 2024-04-25T09:54:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f363ebcf640 1 -- 172.21.15.139:0/4225625898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36380cabd0 con 0x7f3638072d60 2024-04-25T09:54:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f36357fa640 1 -- 172.21.15.139:0/4225625898 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c006e10 con 0x7f3638072d60 2024-04-25T09:54:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.694+0000 7f363ebcf640 1 -- 172.21.15.139:0/4225625898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36380cb190 con 0x7f3638072d60 2024-04-25T09:54:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.695+0000 7f363ebcf640 1 -- 172.21.15.139:0/4225625898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36380b2210 con 0x7f3638072d60 2024-04-25T09:54:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.696+0000 7f36357fa640 1 -- 172.21.15.139:0/4225625898 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f362c03ca10 con 0x7f3638072d60 2024-04-25T09:54:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.696+0000 7f36357fa640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f362406b4d0 0x7f362406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.696+0000 7f36357fa640 1 -- 172.21.15.139:0/4225625898 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f362c0ab080 con 0x7f3638072d60 2024-04-25T09:54:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.696+0000 7f363c96b640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f362406b4d0 0x7f362406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.697+0000 7f363c96b640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f362406b4d0 0x7f362406d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3628009b40 tx=0x7f3628009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.700+0000 7f36357fa640 1 -- 172.21.15.139:0/4225625898 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36380b2210 con 0x7f3638072d60 2024-04-25T09:54:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.870+0000 7f363ebcf640 1 -- 172.21.15.139:0/4225625898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3638074520 con 0x7f3638072d60 2024-04-25T09:54:00.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.870+0000 7f36357fa640 1 -- 172.21.15.139:0/4225625898 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f362c0795d0 con 0x7f3638072d60 2024-04-25T09:54:00.873 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:54:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.872+0000 7f363ebcf640 1 -- 172.21.15.139:0/4225625898 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f362406b4d0 msgr2=0x7f362406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.872+0000 7f363ebcf640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f362406b4d0 0x7f362406d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3628009b40 tx=0x7f3628009190 comp rx=0 tx=0).stop 2024-04-25T09:54:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.872+0000 7f363ebcf640 1 -- 172.21.15.139:0/4225625898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3638072d60 msgr2=0x7f3638158af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.872+0000 7f363ebcf640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3638072d60 0x7f3638158af0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f362c0097a0 tx=0x7f362c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.872+0000 7f363ebcf640 1 -- 172.21.15.139:0/4225625898 shutdown_connections 2024-04-25T09:54:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.872+0000 7f363ebcf640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3638072d60 0x7f3638158af0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.872+0000 7f363ebcf640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f362406b4d0 0x7f362406d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.872+0000 7f363ebcf640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363815da30 0x7f3638159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.872+0000 7f363ebcf640 1 --2- 172.21.15.139:0/4225625898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363806b0a0 0x7f3638148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.872+0000 7f363ebcf640 1 -- 172.21.15.139:0/4225625898 >> 172.21.15.139:0/4225625898 conn(0x7f36380b23f0 msgr2=0x7f36380b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.872+0000 7f363ebcf640 1 -- 172.21.15.139:0/4225625898 shutdown_connections 2024-04-25T09:54:00.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.872+0000 7f363ebcf640 1 -- 172.21.15.139:0/4225625898 wait complete. 2024-04-25T09:54:00.887 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:54:00.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.983+0000 7fd59b9ce640 1 Processor -- start 2024-04-25T09:54:00.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.983+0000 7fd59b9ce640 1 -- start start 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd59b9ce640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd594150ed0 0x7fd5940b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd59b9ce640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd5940cdd20 0x7fd594071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd59b9ce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5940b4740 0x7fd594071f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd59b9ce640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59415a480 con 0x7fd5940cdd20 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd59b9ce640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59415a760 con 0x7fd594150ed0 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd59b9ce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59415aa40 con 0x7fd5940b4740 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd598f69640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd5940cdd20 0x7fd594071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd598f69640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd5940cdd20 0x7fd594071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47334/0 (socket says 172.21.15.139:47334) 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd598f69640 1 -- 172.21.15.139:0/1129124060 learned_addr learned my addr 172.21.15.139:0/1129124060 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd599f6b640 1 --2- 172.21.15.139:0/1129124060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5940b4740 0x7fd594071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd59976a640 1 --2- 172.21.15.139:0/1129124060 >> v2:172.21.15.150:3300/0 conn(0x7fd594150ed0 0x7fd5940b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd598f69640 1 -- 172.21.15.139:0/1129124060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5940b4740 msgr2=0x7fd594071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd598f69640 1 --2- 172.21.15.139:0/1129124060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5940b4740 0x7fd594071f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd598f69640 1 -- 172.21.15.139:0/1129124060 >> v2:172.21.15.150:3300/0 conn(0x7fd594150ed0 msgr2=0x7fd5940b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd598f69640 1 --2- 172.21.15.139:0/1129124060 >> v2:172.21.15.150:3300/0 conn(0x7fd594150ed0 0x7fd5940b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd598f69640 1 -- 172.21.15.139:0/1129124060 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59415ad20 con 0x7fd5940cdd20 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.984+0000 7fd59976a640 1 --2- 172.21.15.139:0/1129124060 >> v2:172.21.15.150:3300/0 conn(0x7fd594150ed0 0x7fd5940b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.985+0000 7fd598f69640 1 --2- 172.21.15.139:0/1129124060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5940cdd20 0x7fd594071a50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd59415a970 tx=0x7fd58402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124814d82b80e2d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.985+0000 7fd58bfff640 1 -- 172.21.15.139:0/1129124060 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584037070 con 0x7fd5940cdd20 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.985+0000 7fd58bfff640 1 -- 172.21.15.139:0/1129124060 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd584032440 con 0x7fd5940cdd20 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.985+0000 7fd58bfff640 1 -- 172.21.15.139:0/1129124060 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584032810 con 0x7fd5940cdd20 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.985+0000 7fd59b9ce640 1 -- 172.21.15.139:0/1129124060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5940cdd20 msgr2=0x7fd594071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.985+0000 7fd59b9ce640 1 --2- 172.21.15.139:0/1129124060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5940cdd20 0x7fd594071a50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd59415a970 tx=0x7fd58402a160 comp rx=0 tx=0).stop 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.986+0000 7fd59b9ce640 1 -- 172.21.15.139:0/1129124060 shutdown_connections 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.986+0000 7fd59b9ce640 1 --2- 172.21.15.139:0/1129124060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5940b4740 0x7fd594071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.986+0000 7fd59b9ce640 1 --2- 172.21.15.139:0/1129124060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5940cdd20 0x7fd594071a50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.986+0000 7fd59b9ce640 1 --2- 172.21.15.139:0/1129124060 >> v2:172.21.15.150:3300/0 conn(0x7fd594150ed0 0x7fd5940b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.986+0000 7fd59b9ce640 1 -- 172.21.15.139:0/1129124060 >> 172.21.15.139:0/1129124060 conn(0x7fd594067000 msgr2=0x7fd59406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.986+0000 7fd59b9ce640 1 -- 172.21.15.139:0/1129124060 shutdown_connections 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.986+0000 7fd59b9ce640 1 -- 172.21.15.139:0/1129124060 wait complete. 2024-04-25T09:54:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.987+0000 7fd59b9ce640 1 Processor -- start 2024-04-25T09:54:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.987+0000 7fd59b9ce640 1 -- start start 2024-04-25T09:54:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.987+0000 7fd59b9ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5940b4740 0x7fd5941e42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.987+0000 7fd59b9ce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5940cdd20 0x7fd5941e62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.987+0000 7fd59b9ce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd594150ed0 0x7fd59414b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59b9ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5941512b0 con 0x7fd5940b4740 2024-04-25T09:54:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59b9ce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940b3960 con 0x7fd594150ed0 2024-04-25T09:54:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59b9ce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940711c0 con 0x7fd5940cdd20 2024-04-25T09:54:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59976a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5940b4740 0x7fd5941e42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59976a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5940b4740 0x7fd5941e42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47346/0 (socket says 172.21.15.139:47346) 2024-04-25T09:54:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59976a640 1 -- 172.21.15.139:0/2955876849 learned_addr learned my addr 172.21.15.139:0/2955876849 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd598f69640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5940cdd20 0x7fd5941e62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd599f6b640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd594150ed0 0x7fd59414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59976a640 1 -- 172.21.15.139:0/2955876849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5940cdd20 msgr2=0x7fd5941e62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59976a640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5940cdd20 0x7fd5941e62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59976a640 1 -- 172.21.15.139:0/2955876849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd594150ed0 msgr2=0x7fd59414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59976a640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd594150ed0 0x7fd59414b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59976a640 1 -- 172.21.15.139:0/2955876849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5941e4930 con 0x7fd5940b4740 2024-04-25T09:54:00.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd598f69640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5940cdd20 0x7fd5941e62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:00.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd599f6b640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd594150ed0 0x7fd59414b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:00.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59976a640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5940b4740 0x7fd5941e42a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c7f0 tx=0x7fd59000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd589ffb640 1 -- 172.21.15.139:0/2955876849 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590015070 con 0x7fd5940b4740 2024-04-25T09:54:00.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd589ffb640 1 -- 172.21.15.139:0/2955876849 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590007820 con 0x7fd5940b4740 2024-04-25T09:54:00.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd589ffb640 1 -- 172.21.15.139:0/2955876849 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590007bd0 con 0x7fd5940b4740 2024-04-25T09:54:00.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59b9ce640 1 -- 172.21.15.139:0/2955876849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5941e4c10 con 0x7fd5940b4740 2024-04-25T09:54:00.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.988+0000 7fd59b9ce640 1 -- 172.21.15.139:0/2955876849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5941e51d0 con 0x7fd5940b4740 2024-04-25T09:54:00.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.989+0000 7fd59b9ce640 1 -- 172.21.15.139:0/2955876849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd594074520 con 0x7fd5940b4740 2024-04-25T09:54:00.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.990+0000 7fd589ffb640 1 -- 172.21.15.139:0/2955876849 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd590018070 con 0x7fd5940b4740 2024-04-25T09:54:00.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.990+0000 7fd589ffb640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd57806b4d0 0x7fd57806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.990+0000 7fd598f69640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd57806b4d0 0x7fd57806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.990+0000 7fd589ffb640 1 -- 172.21.15.139:0/2955876849 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd59008a770 con 0x7fd5940b4740 2024-04-25T09:54:00.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.991+0000 7fd598f69640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd57806b4d0 0x7fd57806d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd584034000 tx=0x7fd58402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:00.995+0000 7fd589ffb640 1 -- 172.21.15.139:0/2955876849 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5900591f0 con 0x7fd5940b4740 2024-04-25T09:54:01.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.175+0000 7fd59b9ce640 1 -- 172.21.15.139:0/2955876849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd59406f150 con 0x7fd5940b4740 2024-04-25T09:54:01.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.176+0000 7fd589ffb640 1 -- 172.21.15.139:0/2955876849 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd590058af0 con 0x7fd5940b4740 2024-04-25T09:54:01.178 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T09:54:01.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.178+0000 7fd59b9ce640 1 -- 172.21.15.139:0/2955876849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd57806b4d0 msgr2=0x7fd57806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.178+0000 7fd59b9ce640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd57806b4d0 0x7fd57806d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd584034000 tx=0x7fd58402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:01.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.178+0000 7fd59b9ce640 1 -- 172.21.15.139:0/2955876849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5940b4740 msgr2=0x7fd5941e42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.178+0000 7fd59b9ce640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5940b4740 0x7fd5941e42a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c7f0 tx=0x7fd59000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.178+0000 7fd59b9ce640 1 -- 172.21.15.139:0/2955876849 shutdown_connections 2024-04-25T09:54:01.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.178+0000 7fd59b9ce640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd57806b4d0 0x7fd57806d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.178+0000 7fd59b9ce640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd594150ed0 0x7fd59414b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.178+0000 7fd59b9ce640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5940cdd20 0x7fd5941e62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.178+0000 7fd59b9ce640 1 --2- 172.21.15.139:0/2955876849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5940b4740 0x7fd5941e42a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.178+0000 7fd59b9ce640 1 -- 172.21.15.139:0/2955876849 >> 172.21.15.139:0/2955876849 conn(0x7fd594067000 msgr2=0x7fd594158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.178+0000 7fd59b9ce640 1 -- 172.21.15.139:0/2955876849 shutdown_connections 2024-04-25T09:54:01.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.178+0000 7fd59b9ce640 1 -- 172.21.15.139:0/2955876849 wait complete. 2024-04-25T09:54:01.191 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:54:01.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:01.191 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T09:54:01.216 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:54:01.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:01.216 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:54:01.271 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:54:01.271 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:01.290 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:54:01.290 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:01.347 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:54:01.347 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:01.405 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:01.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.501+0000 7f7365922640 1 Processor -- start 2024-04-25T09:54:01.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.501+0000 7f7365922640 1 -- start start 2024-04-25T09:54:01.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.502+0000 7f7365922640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f736015a860 0x7f73600cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.502+0000 7f7365922640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73600cd1f0 0x7f736006e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.502+0000 7f7365922640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73600af6d0 con 0x7f736015ac40 2024-04-25T09:54:01.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.502+0000 7f7365922640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73600b02e0 con 0x7f736015a860 2024-04-25T09:54:01.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.502+0000 7f7365922640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73600af9b0 con 0x7f73600cd1f0 2024-04-25T09:54:01.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.502+0000 7f735effd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f736015a860 0x7f73600cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.502+0000 7f735effd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f736015a860 0x7f73600cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47362/0 (socket says 172.21.15.139:47362) 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.502+0000 7f735effd640 1 -- 172.21.15.139:0/129217390 learned_addr learned my addr 172.21.15.139:0/129217390 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.503+0000 7f735e7fc640 1 --2- 172.21.15.139:0/129217390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73600cd1f0 0x7f736006e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.503+0000 7f735effd640 1 -- 172.21.15.139:0/129217390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73600cd1f0 msgr2=0x7f736006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.503+0000 7f735effd640 1 --2- 172.21.15.139:0/129217390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73600cd1f0 0x7f736006e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.503+0000 7f735effd640 1 -- 172.21.15.139:0/129217390 >> v1:172.21.15.139:6789/0 conn(0x7f736015ac40 legacy=0x7f736015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.503+0000 7f735effd640 1 -- 172.21.15.139:0/129217390 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73600afc90 con 0x7f736015a860 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.503+0000 7f735effd640 1 --2- 172.21.15.139:0/129217390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736015a860 0x7f73600cccb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7350009e50 tx=0x7f735002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a30b4f23eae2b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.503+0000 7f735dffb640 1 -- 172.21.15.139:0/129217390 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350036070 con 0x7f736015a860 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.503+0000 7f735dffb640 1 -- 172.21.15.139:0/129217390 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7350031600 con 0x7f736015a860 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.504+0000 7f735dffb640 1 -- 172.21.15.139:0/129217390 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73500319d0 con 0x7f736015a860 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.504+0000 7f7365922640 1 -- 172.21.15.139:0/129217390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736015a860 msgr2=0x7f73600cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.504+0000 7f7365922640 1 --2- 172.21.15.139:0/129217390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736015a860 0x7f73600cccb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7350009e50 tx=0x7f735002a710 comp rx=0 tx=0).stop 2024-04-25T09:54:01.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.504+0000 7f7365922640 1 -- 172.21.15.139:0/129217390 shutdown_connections 2024-04-25T09:54:01.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.504+0000 7f7365922640 1 --2- 172.21.15.139:0/129217390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73600cd1f0 0x7f736006e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.504+0000 7f7365922640 1 --2- 172.21.15.139:0/129217390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736015a860 0x7f73600cccb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.504+0000 7f7365922640 1 -- 172.21.15.139:0/129217390 >> 172.21.15.139:0/129217390 conn(0x7f7360066f20 msgr2=0x7f736006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.504+0000 7f7365922640 1 -- 172.21.15.139:0/129217390 shutdown_connections 2024-04-25T09:54:01.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.504+0000 7f7365922640 1 -- 172.21.15.139:0/129217390 wait complete. 2024-04-25T09:54:01.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.505+0000 7f7365922640 1 Processor -- start 2024-04-25T09:54:01.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.505+0000 7f7365922640 1 -- start start 2024-04-25T09:54:01.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f7365922640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73600cd1f0 0x7f736013bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f7365922640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736015a860 0x7f736013dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f7365922640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f736015ac40 0x7f7360155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f7365922640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73600af420 con 0x7f736015a860 2024-04-25T09:54:01.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f7365922640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73600af2a0 con 0x7f73600cd1f0 2024-04-25T09:54:01.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f7365922640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736006e850 con 0x7f736015ac40 2024-04-25T09:54:01.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f735e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736015a860 0x7f736013dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f735e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736015a860 0x7f736013dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47376/0 (socket says 172.21.15.139:47376) 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f735e7fc640 1 -- 172.21.15.139:0/3921572843 learned_addr learned my addr 172.21.15.139:0/3921572843 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f735f7fe640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f736015ac40 0x7f7360155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f735effd640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73600cd1f0 0x7f736013bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f735e7fc640 1 -- 172.21.15.139:0/3921572843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f736015ac40 msgr2=0x7f7360155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f735e7fc640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f736015ac40 0x7f7360155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.506+0000 7f735e7fc640 1 -- 172.21.15.139:0/3921572843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73600cd1f0 msgr2=0x7f736013bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.507+0000 7f735e7fc640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73600cd1f0 0x7f736013bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.507+0000 7f735e7fc640 1 -- 172.21.15.139:0/3921572843 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7360166450 con 0x7f736015a860 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.507+0000 7f735e7fc640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736015a860 0x7f736013dbe0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f735400d800 tx=0x7f735400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.507+0000 7f7347fff640 1 -- 172.21.15.139:0/3921572843 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735400b6c0 con 0x7f736015a860 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.507+0000 7f7347fff640 1 -- 172.21.15.139:0/3921572843 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7354019070 con 0x7f736015a860 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.507+0000 7f7347fff640 1 -- 172.21.15.139:0/3921572843 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735400ba80 con 0x7f736015a860 2024-04-25T09:54:01.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.507+0000 7f7365922640 1 -- 172.21.15.139:0/3921572843 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7360166730 con 0x7f736015a860 2024-04-25T09:54:01.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.507+0000 7f7365922640 1 -- 172.21.15.139:0/3921572843 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7360166cf0 con 0x7f736015a860 2024-04-25T09:54:01.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.508+0000 7f7365922640 1 -- 172.21.15.139:0/3921572843 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7360074520 con 0x7f736015a860 2024-04-25T09:54:01.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.508+0000 7f7347fff640 1 -- 172.21.15.139:0/3921572843 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f735400bc20 con 0x7f736015a860 2024-04-25T09:54:01.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.509+0000 7f7347fff640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f734c06b4d0 0x7f734c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.509+0000 7f7347fff640 1 -- 172.21.15.139:0/3921572843 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f735408a940 con 0x7f736015a860 2024-04-25T09:54:01.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.509+0000 7f735effd640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f734c06b4d0 0x7f734c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.509+0000 7f735effd640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f734c06b4d0 0x7f734c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7350033000 tx=0x7f7350009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.513+0000 7f7347fff640 1 -- 172.21.15.139:0/3921572843 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7354058d70 con 0x7f736015a860 2024-04-25T09:54:01.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.686+0000 7f7365922640 1 -- 172.21.15.139:0/3921572843 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f736006c1a0 con 0x7f736015a860 2024-04-25T09:54:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.687+0000 7f7347fff640 1 -- 172.21.15.139:0/3921572843 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18951 (secure 0 0 0) 0x7f735405ce10 con 0x7f736015a860 2024-04-25T09:54:01.690 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:01.690 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":26,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:56.839903+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.689+0000 7f7365922640 1 -- 172.21.15.139:0/3921572843 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f734c06b4d0 msgr2=0x7f734c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.689+0000 7f7365922640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f734c06b4d0 0x7f734c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7350033000 tx=0x7f7350009000 comp rx=0 tx=0).stop 2024-04-25T09:54:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.689+0000 7f7365922640 1 -- 172.21.15.139:0/3921572843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736015a860 msgr2=0x7f736013dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.689+0000 7f7365922640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736015a860 0x7f736013dbe0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f735400d800 tx=0x7f735400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.689+0000 7f7365922640 1 -- 172.21.15.139:0/3921572843 shutdown_connections 2024-04-25T09:54:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.689+0000 7f7365922640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f736015a860 0x7f736013dbe0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.689+0000 7f7365922640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f734c06b4d0 0x7f734c06d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.689+0000 7f7365922640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f736015ac40 0x7f7360155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.689+0000 7f7365922640 1 --2- 172.21.15.139:0/3921572843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73600cd1f0 0x7f736013bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.689+0000 7f7365922640 1 -- 172.21.15.139:0/3921572843 >> 172.21.15.139:0/3921572843 conn(0x7f7360066f20 msgr2=0x7f736006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.690+0000 7f7365922640 1 -- 172.21.15.139:0/3921572843 shutdown_connections 2024-04-25T09:54:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.690+0000 7f7365922640 1 -- 172.21.15.139:0/3921572843 wait complete. 2024-04-25T09:54:01.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:54:01.703 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.800+0000 7fe381838640 1 Processor -- start 2024-04-25T09:54:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.801+0000 7fe381838640 1 -- start start 2024-04-25T09:54:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.801+0000 7fe381838640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15dbf0 0x7fe37c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.801+0000 7fe381838640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe37c0cee50 0x7fe37c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.801+0000 7fe381838640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37c161dc0 con 0x7fe37c0b9370 2024-04-25T09:54:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.801+0000 7fe381838640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c0d1ee0 con 0x7fe37c0cee50 2024-04-25T09:54:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.801+0000 7fe381838640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c1620a0 con 0x7fe37c15dbf0 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.801+0000 7fe37b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe37c0cee50 0x7fe37c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.801+0000 7fe37b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe37c0cee50 0x7fe37c0d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47386/0 (socket says 172.21.15.139:47386) 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.801+0000 7fe37b7fe640 1 -- 172.21.15.139:0/1887092922 learned_addr learned my addr 172.21.15.139:0/1887092922 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.802+0000 7fe37b7fe640 1 -- 172.21.15.139:0/1887092922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15dbf0 msgr2=0x7fe37c1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.802+0000 7fe37affd640 1 --2- 172.21.15.139:0/1887092922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15dbf0 0x7fe37c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.802+0000 7fe37b7fe640 1 --2- 172.21.15.139:0/1887092922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15dbf0 0x7fe37c1617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.802+0000 7fe37b7fe640 1 -- 172.21.15.139:0/1887092922 >> v1:172.21.15.139:6789/0 conn(0x7fe37c0b9370 legacy=0x7fe37c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.802+0000 7fe37b7fe640 1 -- 172.21.15.139:0/1887092922 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c162380 con 0x7fe37c0cee50 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.802+0000 7fe37b7fe640 1 --2- 172.21.15.139:0/1887092922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0cee50 0x7fe37c0d2370 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe374009870 tx=0x7fe37402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6769e8e616ccb6ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.803+0000 7fe379ffb640 1 -- 172.21.15.139:0/1887092922 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374037070 con 0x7fe37c0cee50 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.803+0000 7fe379ffb640 1 -- 172.21.15.139:0/1887092922 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374032440 con 0x7fe37c0cee50 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.803+0000 7fe379ffb640 1 -- 172.21.15.139:0/1887092922 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374032810 con 0x7fe37c0cee50 2024-04-25T09:54:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.803+0000 7fe381838640 1 -- 172.21.15.139:0/1887092922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0cee50 msgr2=0x7fe37c0d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.803+0000 7fe381838640 1 --2- 172.21.15.139:0/1887092922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0cee50 0x7fe37c0d2370 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe374009870 tx=0x7fe37402a270 comp rx=0 tx=0).stop 2024-04-25T09:54:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.803+0000 7fe381838640 1 -- 172.21.15.139:0/1887092922 shutdown_connections 2024-04-25T09:54:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.803+0000 7fe381838640 1 --2- 172.21.15.139:0/1887092922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0cee50 0x7fe37c0d2370 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.803+0000 7fe381838640 1 --2- 172.21.15.139:0/1887092922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15dbf0 0x7fe37c1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.803+0000 7fe381838640 1 -- 172.21.15.139:0/1887092922 >> 172.21.15.139:0/1887092922 conn(0x7fe37c0b5480 msgr2=0x7fe37c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.803+0000 7fe381838640 1 -- 172.21.15.139:0/1887092922 shutdown_connections 2024-04-25T09:54:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.803+0000 7fe381838640 1 -- 172.21.15.139:0/1887092922 wait complete. 2024-04-25T09:54:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.804+0000 7fe381838640 1 Processor -- start 2024-04-25T09:54:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.804+0000 7fe381838640 1 -- start start 2024-04-25T09:54:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.805+0000 7fe381838640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0b9370 0x7fe37c0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.805+0000 7fe381838640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0cee50 0x7fe37c15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.805+0000 7fe381838640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15dbf0 0x7fe37c155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.805+0000 7fe381838640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c15dfd0 con 0x7fe37c0cee50 2024-04-25T09:54:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.805+0000 7fe381838640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0b97d0 con 0x7fe37c0b9370 2024-04-25T09:54:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.805+0000 7fe381838640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c15f790 con 0x7fe37c15dbf0 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.805+0000 7fe37a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0cee50 0x7fe37c15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.805+0000 7fe37a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0cee50 0x7fe37c15ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47398/0 (socket says 172.21.15.139:47398) 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.805+0000 7fe37a7fc640 1 -- 172.21.15.139:0/3711847444 learned_addr learned my addr 172.21.15.139:0/3711847444 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe37b7fe640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15dbf0 0x7fe37c155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe37affd640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0b9370 0x7fe37c0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe37a7fc640 1 -- 172.21.15.139:0/3711847444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15dbf0 msgr2=0x7fe37c155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe37a7fc640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15dbf0 0x7fe37c155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe37a7fc640 1 -- 172.21.15.139:0/3711847444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0b9370 msgr2=0x7fe37c0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe37a7fc640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0b9370 0x7fe37c0c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe37a7fc640 1 -- 172.21.15.139:0/3711847444 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c15fa70 con 0x7fe37c0cee50 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe37b7fe640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15dbf0 0x7fe37c155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe37affd640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0b9370 0x7fe37c0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe37a7fc640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0cee50 0x7fe37c15ecf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe37000c3c0 tx=0x7fe37000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe363fff640 1 -- 172.21.15.139:0/3711847444 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3700052d0 con 0x7fe37c0cee50 2024-04-25T09:54:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe363fff640 1 -- 172.21.15.139:0/3711847444 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe370005490 con 0x7fe37c0cee50 2024-04-25T09:54:01.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe381838640 1 -- 172.21.15.139:0/3711847444 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c1539c0 con 0x7fe37c0cee50 2024-04-25T09:54:01.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe363fff640 1 -- 172.21.15.139:0/3711847444 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370007260 con 0x7fe37c0cee50 2024-04-25T09:54:01.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.806+0000 7fe381838640 1 -- 172.21.15.139:0/3711847444 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c0c9940 con 0x7fe37c0cee50 2024-04-25T09:54:01.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.807+0000 7fe381838640 1 -- 172.21.15.139:0/3711847444 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37c1537b0 con 0x7fe37c0cee50 2024-04-25T09:54:01.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.808+0000 7fe363fff640 1 -- 172.21.15.139:0/3711847444 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe370007400 con 0x7fe37c0cee50 2024-04-25T09:54:01.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.808+0000 7fe363fff640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe36806b500 0x7fe36806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.808+0000 7fe363fff640 1 -- 172.21.15.139:0/3711847444 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe3700899c0 con 0x7fe37c0cee50 2024-04-25T09:54:01.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.808+0000 7fe37affd640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe36806b500 0x7fe36806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.808+0000 7fe37affd640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe36806b500 0x7fe36806d9c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00e4a0 tx=0x7fe36c007650 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.812+0000 7fe363fff640 1 -- 172.21.15.139:0/3711847444 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe37008be20 con 0x7fe37c0cee50 2024-04-25T09:54:01.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.983+0000 7fe381838640 1 -- 172.21.15.139:0/3711847444 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe37c074520 con 0x7fe37c0cee50 2024-04-25T09:54:01.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.983+0000 7fe363fff640 1 -- 172.21.15.139:0/3711847444 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3700584d0 con 0x7fe37c0cee50 2024-04-25T09:54:01.986 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:54:01.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.985+0000 7fe381838640 1 -- 172.21.15.139:0/3711847444 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe36806b500 msgr2=0x7fe36806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.985+0000 7fe381838640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe36806b500 0x7fe36806d9c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00e4a0 tx=0x7fe36c007650 comp rx=0 tx=0).stop 2024-04-25T09:54:01.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.985+0000 7fe381838640 1 -- 172.21.15.139:0/3711847444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0cee50 msgr2=0x7fe37c15ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.985+0000 7fe381838640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0cee50 0x7fe37c15ecf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe37000c3c0 tx=0x7fe37000c880 comp rx=0 tx=0).stop 2024-04-25T09:54:01.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.985+0000 7fe381838640 1 -- 172.21.15.139:0/3711847444 shutdown_connections 2024-04-25T09:54:01.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.985+0000 7fe381838640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0cee50 0x7fe37c15ecf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.985+0000 7fe381838640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe36806b500 0x7fe36806d9c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.985+0000 7fe381838640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15dbf0 0x7fe37c155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.985+0000 7fe381838640 1 --2- 172.21.15.139:0/3711847444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0b9370 0x7fe37c0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.985+0000 7fe381838640 1 -- 172.21.15.139:0/3711847444 >> 172.21.15.139:0/3711847444 conn(0x7fe37c0b5480 msgr2=0x7fe37c0b7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.985+0000 7fe381838640 1 -- 172.21.15.139:0/3711847444 shutdown_connections 2024-04-25T09:54:01.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:01.986+0000 7fe381838640 1 -- 172.21.15.139:0/3711847444 wait complete. 2024-04-25T09:54:01.998 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:54:02.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.096+0000 7ff5a1509640 1 Processor -- start 2024-04-25T09:54:02.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.096+0000 7ff5a1509640 1 -- start start 2024-04-25T09:54:02.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff5a1509640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff59c15dbf0 0x7ff59c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff5a1509640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff59c0b9370 0x7ff59c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff5a1509640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59c161dc0 con 0x7ff59c0cec80 2024-04-25T09:54:02.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff5a1509640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59c0d1ee0 con 0x7ff59c0b9370 2024-04-25T09:54:02.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff5a1509640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c1620a0 con 0x7ff59c15dbf0 2024-04-25T09:54:02.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff59a7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff59c0b9370 0x7ff59c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff59a7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff59c0b9370 0x7ff59c0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47402/0 (socket says 172.21.15.139:47402) 2024-04-25T09:54:02.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff59a7fc640 1 -- 172.21.15.139:0/1641665910 learned_addr learned my addr 172.21.15.139:0/1641665910 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:02.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff59affd640 1 --2- 172.21.15.139:0/1641665910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff59c15dbf0 0x7ff59c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff59a7fc640 1 -- 172.21.15.139:0/1641665910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff59c15dbf0 msgr2=0x7ff59c1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:02.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff59a7fc640 1 --2- 172.21.15.139:0/1641665910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff59c15dbf0 0x7ff59c1617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff59a7fc640 1 -- 172.21.15.139:0/1641665910 >> v1:172.21.15.150:6789/0 conn(0x7ff59c0cec80 legacy=0x7ff59c06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff59a7fc640 1 -- 172.21.15.139:0/1641665910 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c162380 con 0x7ff59c0b9370 2024-04-25T09:54:02.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.097+0000 7ff59affd640 1 --2- 172.21.15.139:0/1641665910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff59c15dbf0 0x7ff59c1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:02.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.098+0000 7ff59a7fc640 1 --2- 172.21.15.139:0/1641665910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff59c0b9370 0x7ff59c0ce740 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff590009870 tx=0x7ff59002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f91bcec73e059d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.098+0000 7ff599ffb640 1 -- 172.21.15.139:0/1641665910 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590037070 con 0x7ff59c0b9370 2024-04-25T09:54:02.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.098+0000 7ff599ffb640 1 -- 172.21.15.139:0/1641665910 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff590032440 con 0x7ff59c0b9370 2024-04-25T09:54:02.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.098+0000 7ff599ffb640 1 -- 172.21.15.139:0/1641665910 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590032810 con 0x7ff59c0b9370 2024-04-25T09:54:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.099+0000 7ff5a1509640 1 -- 172.21.15.139:0/1641665910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff59c0b9370 msgr2=0x7ff59c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.099+0000 7ff5a1509640 1 --2- 172.21.15.139:0/1641665910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff59c0b9370 0x7ff59c0ce740 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff590009870 tx=0x7ff59002a270 comp rx=0 tx=0).stop 2024-04-25T09:54:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.099+0000 7ff5a1509640 1 -- 172.21.15.139:0/1641665910 shutdown_connections 2024-04-25T09:54:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.099+0000 7ff5a1509640 1 --2- 172.21.15.139:0/1641665910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff59c0b9370 0x7ff59c0ce740 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.099+0000 7ff5a1509640 1 --2- 172.21.15.139:0/1641665910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff59c15dbf0 0x7ff59c1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.099+0000 7ff5a1509640 1 -- 172.21.15.139:0/1641665910 >> 172.21.15.139:0/1641665910 conn(0x7ff59c0b5480 msgr2=0x7ff59c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.099+0000 7ff5a1509640 1 -- 172.21.15.139:0/1641665910 shutdown_connections 2024-04-25T09:54:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.099+0000 7ff5a1509640 1 -- 172.21.15.139:0/1641665910 wait complete. 2024-04-25T09:54:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.100+0000 7ff5a1509640 1 Processor -- start 2024-04-25T09:54:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.100+0000 7ff5a1509640 1 -- start start 2024-04-25T09:54:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff5a1509640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff59c0b9370 0x7ff59c158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff5a1509640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff59c0cec80 0x7ff59c15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff5a1509640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff59c15dbf0 0x7ff59c1f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff5a1509640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c15dfd0 con 0x7ff59c0cec80 2024-04-25T09:54:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff5a1509640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c0b97d0 con 0x7ff59c15dbf0 2024-04-25T09:54:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff5a1509640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c159960 con 0x7ff59c0b9370 2024-04-25T09:54:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff59a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff59c0cec80 0x7ff59c15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff59a7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff59c0cec80 0x7ff59c15ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47408/0 (socket says 172.21.15.139:47408) 2024-04-25T09:54:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff59a7fc640 1 -- 172.21.15.139:0/196854666 learned_addr learned my addr 172.21.15.139:0/196854666 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff59affd640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff59c0b9370 0x7ff59c158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff59a7fc640 1 -- 172.21.15.139:0/196854666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff59c0b9370 msgr2=0x7ff59c158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff59a7fc640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff59c0b9370 0x7ff59c158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff59a7fc640 1 -- 172.21.15.139:0/196854666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff59c15dbf0 msgr2=0x7ff59c1f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff59a7fc640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff59c15dbf0 0x7ff59c1f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff59a7fc640 1 -- 172.21.15.139:0/196854666 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c159c40 con 0x7ff59c0cec80 2024-04-25T09:54:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff59a7fc640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff59c0cec80 0x7ff59c15ace0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff590009870 tx=0x7ff5900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff5837fe640 1 -- 172.21.15.139:0/196854666 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590041070 con 0x7ff59c0cec80 2024-04-25T09:54:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff5837fe640 1 -- 172.21.15.139:0/196854666 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59003c890 con 0x7ff59c0cec80 2024-04-25T09:54:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff5837fe640 1 -- 172.21.15.139:0/196854666 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590037040 con 0x7ff59c0cec80 2024-04-25T09:54:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.101+0000 7ff5a1509640 1 -- 172.21.15.139:0/196854666 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c159f20 con 0x7ff59c0cec80 2024-04-25T09:54:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.102+0000 7ff5a1509640 1 -- 172.21.15.139:0/196854666 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c15a4e0 con 0x7ff59c0cec80 2024-04-25T09:54:02.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.102+0000 7ff5a1509640 1 -- 172.21.15.139:0/196854666 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59c074520 con 0x7ff59c0cec80 2024-04-25T09:54:02.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.103+0000 7ff5837fe640 1 -- 172.21.15.139:0/196854666 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff590044070 con 0x7ff59c0cec80 2024-04-25T09:54:02.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.103+0000 7ff5837fe640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff58806b4d0 0x7ff58806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.103+0000 7ff59affd640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff58806b4d0 0x7ff58806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.103+0000 7ff5837fe640 1 -- 172.21.15.139:0/196854666 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff5900ab150 con 0x7ff59c0cec80 2024-04-25T09:54:02.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.104+0000 7ff59affd640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff58806b4d0 0x7ff58806d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00d8b0 tx=0x7ff58c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.108+0000 7ff5837fe640 1 -- 172.21.15.139:0/196854666 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff590079cf0 con 0x7ff59c0cec80 2024-04-25T09:54:02.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.276+0000 7ff5a1509640 1 -- 172.21.15.139:0/196854666 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff59c0cf270 con 0x7ff59c0cec80 2024-04-25T09:54:02.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.277+0000 7ff5837fe640 1 -- 172.21.15.139:0/196854666 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5900795f0 con 0x7ff59c0cec80 2024-04-25T09:54:02.279 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:54:02.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.278+0000 7ff5a1509640 1 -- 172.21.15.139:0/196854666 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff58806b4d0 msgr2=0x7ff58806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.279+0000 7ff5a1509640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff58806b4d0 0x7ff58806d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00d8b0 tx=0x7ff58c008040 comp rx=0 tx=0).stop 2024-04-25T09:54:02.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.279+0000 7ff5a1509640 1 -- 172.21.15.139:0/196854666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff59c0cec80 msgr2=0x7ff59c15ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.279+0000 7ff5a1509640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff59c0cec80 0x7ff59c15ace0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff590009870 tx=0x7ff5900046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.279+0000 7ff5a1509640 1 -- 172.21.15.139:0/196854666 shutdown_connections 2024-04-25T09:54:02.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.279+0000 7ff5a1509640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff59c0cec80 0x7ff59c15ace0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.279+0000 7ff5a1509640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff58806b4d0 0x7ff58806d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.279+0000 7ff5a1509640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff59c15dbf0 0x7ff59c1f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.279+0000 7ff5a1509640 1 --2- 172.21.15.139:0/196854666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff59c0b9370 0x7ff59c158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.279+0000 7ff5a1509640 1 -- 172.21.15.139:0/196854666 >> 172.21.15.139:0/196854666 conn(0x7ff59c0b5480 msgr2=0x7ff59c0d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:02.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.279+0000 7ff5a1509640 1 -- 172.21.15.139:0/196854666 shutdown_connections 2024-04-25T09:54:02.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.279+0000 7ff5a1509640 1 -- 172.21.15.139:0/196854666 wait complete. 2024-04-25T09:54:02.292 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:54:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.389+0000 7fc828cdd640 1 Processor -- start 2024-04-25T09:54:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.389+0000 7fc828cdd640 1 -- start start 2024-04-25T09:54:02.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc828cdd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc824145230 0x7fc82415be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc828cdd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc8240718d0 0x7fc82415da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc828cdd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82415c960 con 0x7fc82415c3d0 2024-04-25T09:54:02.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc828cdd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82415cc40 con 0x7fc8240718d0 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc828cdd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82415d230 con 0x7fc824145230 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc822d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc8240718d0 0x7fc82415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc822d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc8240718d0 0x7fc82415da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47418/0 (socket says 172.21.15.139:47418) 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc822d9d640 1 -- 172.21.15.139:0/883756240 learned_addr learned my addr 172.21.15.139:0/883756240 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc82259c640 1 --2- 172.21.15.139:0/883756240 >> v2:172.21.15.150:3300/0 conn(0x7fc824145230 0x7fc82415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc822d9d640 1 -- 172.21.15.139:0/883756240 >> v2:172.21.15.150:3300/0 conn(0x7fc824145230 msgr2=0x7fc82415be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc822d9d640 1 --2- 172.21.15.139:0/883756240 >> v2:172.21.15.150:3300/0 conn(0x7fc824145230 0x7fc82415be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc822d9d640 1 -- 172.21.15.139:0/883756240 >> v1:172.21.15.139:6789/0 conn(0x7fc82415c3d0 legacy=0x7fc8240711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc822d9d640 1 -- 172.21.15.139:0/883756240 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82415d510 con 0x7fc8240718d0 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.390+0000 7fc82259c640 1 --2- 172.21.15.139:0/883756240 >> v2:172.21.15.150:3300/0 conn(0x7fc824145230 0x7fc82415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.391+0000 7fc822d9d640 1 --2- 172.21.15.139:0/883756240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8240718d0 0x7fc82415da30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc8240b4990 tx=0x7fc81c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f8d8a998d171b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.391+0000 7fc821d9b640 1 -- 172.21.15.139:0/883756240 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c036070 con 0x7fc8240718d0 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.391+0000 7fc821d9b640 1 -- 172.21.15.139:0/883756240 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c031600 con 0x7fc8240718d0 2024-04-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.391+0000 7fc821d9b640 1 -- 172.21.15.139:0/883756240 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0319d0 con 0x7fc8240718d0 2024-04-25T09:54:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.391+0000 7fc828cdd640 1 -- 172.21.15.139:0/883756240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8240718d0 msgr2=0x7fc82415da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.391+0000 7fc828cdd640 1 --2- 172.21.15.139:0/883756240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8240718d0 0x7fc82415da30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc8240b4990 tx=0x7fc81c02a730 comp rx=0 tx=0).stop 2024-04-25T09:54:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.392+0000 7fc828cdd640 1 -- 172.21.15.139:0/883756240 shutdown_connections 2024-04-25T09:54:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.392+0000 7fc828cdd640 1 --2- 172.21.15.139:0/883756240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8240718d0 0x7fc82415da30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.392+0000 7fc828cdd640 1 --2- 172.21.15.139:0/883756240 >> v2:172.21.15.150:3300/0 conn(0x7fc824145230 0x7fc82415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.392+0000 7fc828cdd640 1 -- 172.21.15.139:0/883756240 >> 172.21.15.139:0/883756240 conn(0x7fc824067000 msgr2=0x7fc82406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.392+0000 7fc828cdd640 1 -- 172.21.15.139:0/883756240 shutdown_connections 2024-04-25T09:54:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.392+0000 7fc828cdd640 1 -- 172.21.15.139:0/883756240 wait complete. 2024-04-25T09:54:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.393+0000 7fc828cdd640 1 Processor -- start 2024-04-25T09:54:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.393+0000 7fc828cdd640 1 -- start start 2024-04-25T09:54:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc828cdd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8240718d0 0x7fc8241f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc828cdd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc824145230 0x7fc8240b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc828cdd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc82415c3d0 0x7fc8240cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc828cdd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82415e0f0 con 0x7fc82415c3d0 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc828cdd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82415df70 con 0x7fc824145230 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc828cdd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82415e270 con 0x7fc8240718d0 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc822d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc82415c3d0 0x7fc8240cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc822d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc82415c3d0 0x7fc8240cac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47422/0 (socket says 172.21.15.139:47422) 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc822d9d640 1 -- 172.21.15.139:0/1383413110 learned_addr learned my addr 172.21.15.139:0/1383413110 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc81bfff640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc824145230 0x7fc8240b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc822d9d640 1 -- 172.21.15.139:0/1383413110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8240718d0 msgr2=0x7fc8241f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc822d9d640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8240718d0 0x7fc8241f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc822d9d640 1 -- 172.21.15.139:0/1383413110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc824145230 msgr2=0x7fc8240b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc822d9d640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc824145230 0x7fc8240b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc822d9d640 1 -- 172.21.15.139:0/1383413110 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240c97a0 con 0x7fc82415c3d0 2024-04-25T09:54:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc822d9d640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc82415c3d0 0x7fc8240cac20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009f40 tx=0x7fc81c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc81affd640 1 -- 172.21.15.139:0/1383413110 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c036050 con 0x7fc82415c3d0 2024-04-25T09:54:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc81affd640 1 -- 172.21.15.139:0/1383413110 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c042070 con 0x7fc82415c3d0 2024-04-25T09:54:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc828cdd640 1 -- 172.21.15.139:0/1383413110 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8240c9a80 con 0x7fc82415c3d0 2024-04-25T09:54:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc81affd640 1 -- 172.21.15.139:0/1383413110 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c009510 con 0x7fc82415c3d0 2024-04-25T09:54:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.394+0000 7fc828cdd640 1 -- 172.21.15.139:0/1383413110 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8240ca040 con 0x7fc82415c3d0 2024-04-25T09:54:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.395+0000 7fc828cdd640 1 -- 172.21.15.139:0/1383413110 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82406baf0 con 0x7fc82415c3d0 2024-04-25T09:54:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.396+0000 7fc81affd640 1 -- 172.21.15.139:0/1383413110 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc81c007bf0 con 0x7fc82415c3d0 2024-04-25T09:54:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.396+0000 7fc81affd640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc80406b4d0 0x7fc80406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.396+0000 7fc81affd640 1 -- 172.21.15.139:0/1383413110 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc81c0ab4a0 con 0x7fc82415c3d0 2024-04-25T09:54:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.396+0000 7fc82259c640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc80406b4d0 0x7fc80406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.396+0000 7fc82259c640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc80406b4d0 0x7fc80406d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc814006fd0 tx=0x7fc814008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.400+0000 7fc81affd640 1 -- 172.21.15.139:0/1383413110 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc81c079f20 con 0x7fc82415c3d0 2024-04-25T09:54:02.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.568+0000 7fc828cdd640 1 -- 172.21.15.139:0/1383413110 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc82406c1a0 con 0x7fc82415c3d0 2024-04-25T09:54:02.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.569+0000 7fc81affd640 1 -- 172.21.15.139:0/1383413110 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc81c079820 con 0x7fc82415c3d0 2024-04-25T09:54:02.572 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:54:02.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.571+0000 7fc828cdd640 1 -- 172.21.15.139:0/1383413110 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc80406b4d0 msgr2=0x7fc80406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.571+0000 7fc828cdd640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc80406b4d0 0x7fc80406d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc814006fd0 tx=0x7fc814008040 comp rx=0 tx=0).stop 2024-04-25T09:54:02.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.571+0000 7fc828cdd640 1 -- 172.21.15.139:0/1383413110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc82415c3d0 msgr2=0x7fc8240cac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.571+0000 7fc828cdd640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc82415c3d0 0x7fc8240cac20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009f40 tx=0x7fc81c004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:02.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.571+0000 7fc828cdd640 1 -- 172.21.15.139:0/1383413110 shutdown_connections 2024-04-25T09:54:02.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.571+0000 7fc828cdd640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc82415c3d0 0x7fc8240cac20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.571+0000 7fc828cdd640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc80406b4d0 0x7fc80406d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.571+0000 7fc828cdd640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc824145230 0x7fc8240b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.571+0000 7fc828cdd640 1 --2- 172.21.15.139:0/1383413110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8240718d0 0x7fc8241f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.571+0000 7fc828cdd640 1 -- 172.21.15.139:0/1383413110 >> 172.21.15.139:0/1383413110 conn(0x7fc824067000 msgr2=0x7fc824144200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:02.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.571+0000 7fc828cdd640 1 -- 172.21.15.139:0/1383413110 shutdown_connections 2024-04-25T09:54:02.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.571+0000 7fc828cdd640 1 -- 172.21.15.139:0/1383413110 wait complete. 2024-04-25T09:54:02.584 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:54:02.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.681+0000 7fe7df389640 1 Processor -- start 2024-04-25T09:54:02.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.682+0000 7fe7df389640 1 -- start start 2024-04-25T09:54:02.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.682+0000 7fe7df389640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7d815dbf0 0x7fe7d81617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.682+0000 7fe7df389640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe7d80b9370 0x7fe7d80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.682+0000 7fe7df389640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe7d80cec80 0x7fe7d806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.682+0000 7fe7df389640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d80d1ee0 con 0x7fe7d80b9370 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.682+0000 7fe7df389640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d8161dc0 con 0x7fe7d80cec80 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.682+0000 7fe7df389640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d81620a0 con 0x7fe7d815dbf0 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.682+0000 7fe7dc924640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe7d80b9370 0x7fe7d80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.683+0000 7fe7dc924640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe7d80b9370 0x7fe7d80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47430/0 (socket says 172.21.15.139:47430) 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.683+0000 7fe7dc924640 1 -- 172.21.15.139:0/2871074196 learned_addr learned my addr 172.21.15.139:0/2871074196 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.683+0000 7fe7dd125640 1 --2- 172.21.15.139:0/2871074196 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7d815dbf0 0x7fe7d81617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.683+0000 7fe7dd926640 1 --2- 172.21.15.139:0/2871074196 >> v2:172.21.15.150:3300/0 conn(0x7fe7d80cec80 0x7fe7d806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.683+0000 7fe7dc924640 1 -- 172.21.15.139:0/2871074196 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7d815dbf0 msgr2=0x7fe7d81617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.683+0000 7fe7dc924640 1 --2- 172.21.15.139:0/2871074196 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7d815dbf0 0x7fe7d81617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.683+0000 7fe7dc924640 1 -- 172.21.15.139:0/2871074196 >> v2:172.21.15.150:3300/0 conn(0x7fe7d80cec80 msgr2=0x7fe7d806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.683+0000 7fe7dc924640 1 --2- 172.21.15.139:0/2871074196 >> v2:172.21.15.150:3300/0 conn(0x7fe7d80cec80 0x7fe7d806c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.683+0000 7fe7dc924640 1 -- 172.21.15.139:0/2871074196 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d8162380 con 0x7fe7d80b9370 2024-04-25T09:54:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.683+0000 7fe7dc924640 1 --2- 172.21.15.139:0/2871074196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d80b9370 0x7fe7d80ce740 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe7d808c170 tx=0x7fe7c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a8059efb7454ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.684+0000 7fe7cffff640 1 -- 172.21.15.139:0/2871074196 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8037070 con 0x7fe7d80b9370 2024-04-25T09:54:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.684+0000 7fe7cffff640 1 -- 172.21.15.139:0/2871074196 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c8032440 con 0x7fe7d80b9370 2024-04-25T09:54:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.684+0000 7fe7cffff640 1 -- 172.21.15.139:0/2871074196 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8032810 con 0x7fe7d80b9370 2024-04-25T09:54:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.684+0000 7fe7df389640 1 -- 172.21.15.139:0/2871074196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d80b9370 msgr2=0x7fe7d80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.684+0000 7fe7df389640 1 --2- 172.21.15.139:0/2871074196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d80b9370 0x7fe7d80ce740 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe7d808c170 tx=0x7fe7c802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.684+0000 7fe7df389640 1 -- 172.21.15.139:0/2871074196 shutdown_connections 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.684+0000 7fe7df389640 1 --2- 172.21.15.139:0/2871074196 >> v2:172.21.15.150:3300/0 conn(0x7fe7d80cec80 0x7fe7d806c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.684+0000 7fe7df389640 1 --2- 172.21.15.139:0/2871074196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d80b9370 0x7fe7d80ce740 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.684+0000 7fe7df389640 1 --2- 172.21.15.139:0/2871074196 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7d815dbf0 0x7fe7d81617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.684+0000 7fe7df389640 1 -- 172.21.15.139:0/2871074196 >> 172.21.15.139:0/2871074196 conn(0x7fe7d80b5480 msgr2=0x7fe7d80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.684+0000 7fe7df389640 1 -- 172.21.15.139:0/2871074196 shutdown_connections 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.684+0000 7fe7df389640 1 -- 172.21.15.139:0/2871074196 wait complete. 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.685+0000 7fe7df389640 1 Processor -- start 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.685+0000 7fe7df389640 1 -- start start 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7df389640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7d80b9370 0x7fe7d814bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7df389640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d80cec80 0x7fe7d815e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7df389640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7d815dbf0 0x7fe7d815b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7df389640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d815dfd0 con 0x7fe7d80cec80 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7df389640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d80b97d0 con 0x7fe7d80b9370 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7df389640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d814c2d0 con 0x7fe7d815dbf0 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7dc924640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d80cec80 0x7fe7d815e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7dc924640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d80cec80 0x7fe7d815e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47434/0 (socket says 172.21.15.139:47434) 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7dc924640 1 -- 172.21.15.139:0/3393767514 learned_addr learned my addr 172.21.15.139:0/3393767514 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7dd926640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7d815dbf0 0x7fe7d815b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7dd125640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7d80b9370 0x7fe7d814bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7dc924640 1 -- 172.21.15.139:0/3393767514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7d815dbf0 msgr2=0x7fe7d815b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7dc924640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7d815dbf0 0x7fe7d815b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7dc924640 1 -- 172.21.15.139:0/3393767514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7d80b9370 msgr2=0x7fe7d814bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7dc924640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7d80b9370 0x7fe7d814bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7dc924640 1 -- 172.21.15.139:0/3393767514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d815ec50 con 0x7fe7d80cec80 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.686+0000 7fe7dd125640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7d80b9370 0x7fe7d814bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.687+0000 7fe7dc924640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d80cec80 0x7fe7d815e2e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000c00 tx=0x7fe7c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.687+0000 7fe7cdffb640 1 -- 172.21.15.139:0/3393767514 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8041070 con 0x7fe7d80cec80 2024-04-25T09:54:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.687+0000 7fe7cdffb640 1 -- 172.21.15.139:0/3393767514 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c803c870 con 0x7fe7d80cec80 2024-04-25T09:54:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.687+0000 7fe7df389640 1 -- 172.21.15.139:0/3393767514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d815ef30 con 0x7fe7d80cec80 2024-04-25T09:54:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.687+0000 7fe7cdffb640 1 -- 172.21.15.139:0/3393767514 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8037040 con 0x7fe7d80cec80 2024-04-25T09:54:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.687+0000 7fe7df389640 1 -- 172.21.15.139:0/3393767514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d815f4f0 con 0x7fe7d80cec80 2024-04-25T09:54:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.688+0000 7fe7df389640 1 -- 172.21.15.139:0/3393767514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d8074520 con 0x7fe7d80cec80 2024-04-25T09:54:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.688+0000 7fe7cdffb640 1 -- 172.21.15.139:0/3393767514 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7c8044070 con 0x7fe7d80cec80 2024-04-25T09:54:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.689+0000 7fe7cdffb640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe7bc06b4d0 0x7fe7bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.689+0000 7fe7cdffb640 1 -- 172.21.15.139:0/3393767514 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe7c80ab390 con 0x7fe7d80cec80 2024-04-25T09:54:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.689+0000 7fe7dd125640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe7bc06b4d0 0x7fe7bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.689+0000 7fe7dd125640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe7bc06b4d0 0x7fe7bc06d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4006fd0 tx=0x7fe7d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.693+0000 7fe7cdffb640 1 -- 172.21.15.139:0/3393767514 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7c8079f30 con 0x7fe7d80cec80 2024-04-25T09:54:02.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.865+0000 7fe7df389640 1 -- 172.21.15.139:0/3393767514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe7d80d1680 con 0x7fe7d80cec80 2024-04-25T09:54:02.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.865+0000 7fe7cdffb640 1 -- 172.21.15.139:0/3393767514 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe7c8079830 con 0x7fe7d80cec80 2024-04-25T09:54:02.868 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:54:02.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.867+0000 7fe7df389640 1 -- 172.21.15.139:0/3393767514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe7bc06b4d0 msgr2=0x7fe7bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.868+0000 7fe7df389640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe7bc06b4d0 0x7fe7bc06d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4006fd0 tx=0x7fe7d4008040 comp rx=0 tx=0).stop 2024-04-25T09:54:02.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.868+0000 7fe7df389640 1 -- 172.21.15.139:0/3393767514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d80cec80 msgr2=0x7fe7d815e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.868+0000 7fe7df389640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d80cec80 0x7fe7d815e2e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000c00 tx=0x7fe7c80046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.868+0000 7fe7df389640 1 -- 172.21.15.139:0/3393767514 shutdown_connections 2024-04-25T09:54:02.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.868+0000 7fe7df389640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d80cec80 0x7fe7d815e2e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.868+0000 7fe7df389640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe7bc06b4d0 0x7fe7bc06d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.868+0000 7fe7df389640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7d815dbf0 0x7fe7d815b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.868+0000 7fe7df389640 1 --2- 172.21.15.139:0/3393767514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7d80b9370 0x7fe7d814bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.868+0000 7fe7df389640 1 -- 172.21.15.139:0/3393767514 >> 172.21.15.139:0/3393767514 conn(0x7fe7d80b5480 msgr2=0x7fe7d80d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:02.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.868+0000 7fe7df389640 1 -- 172.21.15.139:0/3393767514 shutdown_connections 2024-04-25T09:54:02.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.868+0000 7fe7df389640 1 -- 172.21.15.139:0/3393767514 wait complete. 2024-04-25T09:54:02.881 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:54:02.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.976+0000 7f1233e12640 1 Processor -- start 2024-04-25T09:54:02.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.976+0000 7f1233e12640 1 -- start start 2024-04-25T09:54:02.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f1233e12640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f122c15b9b0 0x7f122c15fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f1233e12640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f122c0cea10 0x7f122c0cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f1233e12640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122c0b2a60 con 0x7f122c0cc640 2024-04-25T09:54:02.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f1233e12640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122c160300 con 0x7f122c15b9b0 2024-04-25T09:54:02.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f1233e12640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122c06fb90 con 0x7f122c0cea10 2024-04-25T09:54:02.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f1231bae640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f122c15b9b0 0x7f122c15fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f1231bae640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f122c15b9b0 0x7f122c15fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47446/0 (socket says 172.21.15.139:47446) 2024-04-25T09:54:02.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f1231bae640 1 -- 172.21.15.139:0/1277497909 learned_addr learned my addr 172.21.15.139:0/1277497909 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:02.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f12313ad640 1 --2- 172.21.15.139:0/1277497909 >> v2:172.21.15.150:3300/0 conn(0x7f122c0cea10 0x7f122c0cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f1231bae640 1 -- 172.21.15.139:0/1277497909 >> v2:172.21.15.150:3300/0 conn(0x7f122c0cea10 msgr2=0x7f122c0cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:02.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f1231bae640 1 --2- 172.21.15.139:0/1277497909 >> v2:172.21.15.150:3300/0 conn(0x7f122c0cea10 0x7f122c0cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f1231bae640 1 -- 172.21.15.139:0/1277497909 >> v1:172.21.15.150:6789/0 conn(0x7f122c0cc640 legacy=0x7f122c0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:02.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.977+0000 7f1231bae640 1 -- 172.21.15.139:0/1277497909 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c0b2d00 con 0x7f122c15b9b0 2024-04-25T09:54:02.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.978+0000 7f1231bae640 1 --2- 172.21.15.139:0/1277497909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c15b9b0 0x7f122c15fdc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f122c072dc0 tx=0x7f122802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655c27b0f4f4bc61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.978+0000 7f1230bac640 1 -- 172.21.15.139:0/1277497909 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228037070 con 0x7f122c15b9b0 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.978+0000 7f1230bac640 1 -- 172.21.15.139:0/1277497909 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1228032440 con 0x7f122c15b9b0 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.978+0000 7f1230bac640 1 -- 172.21.15.139:0/1277497909 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228032810 con 0x7f122c15b9b0 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.978+0000 7f1233e12640 1 -- 172.21.15.139:0/1277497909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c15b9b0 msgr2=0x7f122c15fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.978+0000 7f1233e12640 1 --2- 172.21.15.139:0/1277497909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c15b9b0 0x7f122c15fdc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f122c072dc0 tx=0x7f122802a200 comp rx=0 tx=0).stop 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.978+0000 7f1233e12640 1 -- 172.21.15.139:0/1277497909 shutdown_connections 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.979+0000 7f1233e12640 1 --2- 172.21.15.139:0/1277497909 >> v2:172.21.15.150:3300/0 conn(0x7f122c0cea10 0x7f122c0cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.979+0000 7f1233e12640 1 --2- 172.21.15.139:0/1277497909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c15b9b0 0x7f122c15fdc0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.979+0000 7f1233e12640 1 -- 172.21.15.139:0/1277497909 >> 172.21.15.139:0/1277497909 conn(0x7f122c0cf9b0 msgr2=0x7f122c0cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.979+0000 7f1233e12640 1 -- 172.21.15.139:0/1277497909 shutdown_connections 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.979+0000 7f1233e12640 1 -- 172.21.15.139:0/1277497909 wait complete. 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.980+0000 7f1233e12640 1 Processor -- start 2024-04-25T09:54:02.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.980+0000 7f1233e12640 1 -- start start 2024-04-25T09:54:02.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.980+0000 7f1233e12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c0cc640 0x7f122c14c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.980+0000 7f1233e12640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f122c0cea10 0x7f122c1e77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.980+0000 7f1233e12640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f122c15b9b0 0x7f122c1f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.980+0000 7f1233e12640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c0cf3b0 con 0x7f122c0cc640 2024-04-25T09:54:02.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.980+0000 7f1233e12640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c06fe70 con 0x7f122c15b9b0 2024-04-25T09:54:02.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.980+0000 7f1233e12640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122c06f840 con 0x7f122c0cea10 2024-04-25T09:54:02.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.980+0000 7f1231bae640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c0cc640 0x7f122c14c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.980+0000 7f1231bae640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c0cc640 0x7f122c14c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47462/0 (socket says 172.21.15.139:47462) 2024-04-25T09:54:02.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.980+0000 7f1231bae640 1 -- 172.21.15.139:0/1861031400 learned_addr learned my addr 172.21.15.139:0/1861031400 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f12313ad640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f122c0cea10 0x7f122c1e77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f1231bae640 1 -- 172.21.15.139:0/1861031400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f122c0cea10 msgr2=0x7f122c1e77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f12323af640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f122c15b9b0 0x7f122c1f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f1231bae640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f122c0cea10 0x7f122c1e77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f1231bae640 1 -- 172.21.15.139:0/1861031400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f122c15b9b0 msgr2=0x7f122c1f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f1231bae640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f122c15b9b0 0x7f122c1f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f1231bae640 1 -- 172.21.15.139:0/1861031400 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c1e6720 con 0x7f122c0cc640 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f12313ad640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f122c0cea10 0x7f122c1e77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f12323af640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f122c15b9b0 0x7f122c1f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f1231bae640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c0cc640 0x7f122c14c300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1228000c00 tx=0x7f12280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f12227fc640 1 -- 172.21.15.139:0/1861031400 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228041020 con 0x7f122c0cc640 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f12227fc640 1 -- 172.21.15.139:0/1861031400 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12280306c0 con 0x7f122c0cc640 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f1233e12640 1 -- 172.21.15.139:0/1861031400 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c1e6a00 con 0x7f122c0cc640 2024-04-25T09:54:02.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f12227fc640 1 -- 172.21.15.139:0/1861031400 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228037040 con 0x7f122c0cc640 2024-04-25T09:54:02.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.981+0000 7f1233e12640 1 -- 172.21.15.139:0/1861031400 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c1e72a0 con 0x7f122c0cc640 2024-04-25T09:54:02.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.982+0000 7f1233e12640 1 -- 172.21.15.139:0/1861031400 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122c1e6bd0 con 0x7f122c0cc640 2024-04-25T09:54:02.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.983+0000 7f12227fc640 1 -- 172.21.15.139:0/1861031400 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1228044020 con 0x7f122c0cc640 2024-04-25T09:54:02.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.983+0000 7f12227fc640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f121006b4d0 0x7f121006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.983+0000 7f12227fc640 1 -- 172.21.15.139:0/1861031400 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f12280aae30 con 0x7f122c0cc640 2024-04-25T09:54:02.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.983+0000 7f12313ad640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f121006b4d0 0x7f121006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.984+0000 7f12313ad640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f121006b4d0 0x7f121006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f122c14dbe0 tx=0x7f121c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:02.988+0000 7f12227fc640 1 -- 172.21.15.139:0/1861031400 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f122804b030 con 0x7f122c0cc640 2024-04-25T09:54:03.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.169+0000 7f1233e12640 1 -- 172.21.15.139:0/1861031400 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f122c074520 con 0x7f122c0cc640 2024-04-25T09:54:03.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.169+0000 7f12227fc640 1 -- 172.21.15.139:0/1861031400 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f12280798b0 con 0x7f122c0cc640 2024-04-25T09:54:03.172 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T09:54:03.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.171+0000 7f1233e12640 1 -- 172.21.15.139:0/1861031400 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f121006b4d0 msgr2=0x7f121006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.171+0000 7f1233e12640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f121006b4d0 0x7f121006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f122c14dbe0 tx=0x7f121c008040 comp rx=0 tx=0).stop 2024-04-25T09:54:03.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.171+0000 7f1233e12640 1 -- 172.21.15.139:0/1861031400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c0cc640 msgr2=0x7f122c14c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.171+0000 7f1233e12640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c0cc640 0x7f122c14c300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1228000c00 tx=0x7f12280046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.171+0000 7f1233e12640 1 -- 172.21.15.139:0/1861031400 shutdown_connections 2024-04-25T09:54:03.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.171+0000 7f1233e12640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f121006b4d0 0x7f121006d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.171+0000 7f1233e12640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f122c15b9b0 0x7f122c1f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.171+0000 7f1233e12640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f122c0cea10 0x7f122c1e77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.171+0000 7f1233e12640 1 --2- 172.21.15.139:0/1861031400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f122c0cc640 0x7f122c14c300 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.171+0000 7f1233e12640 1 -- 172.21.15.139:0/1861031400 >> 172.21.15.139:0/1861031400 conn(0x7f122c0cf9b0 msgr2=0x7f122c0b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:03.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.171+0000 7f1233e12640 1 -- 172.21.15.139:0/1861031400 shutdown_connections 2024-04-25T09:54:03.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.171+0000 7f1233e12640 1 -- 172.21.15.139:0/1861031400 wait complete. 2024-04-25T09:54:03.184 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T09:54:03.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:03.184 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T09:54:03.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T09:54:03.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:03.216 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:54:03.285 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:54:03.285 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:03.302 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:54:03.302 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:03.357 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:54:03.357 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:03.413 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:03.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.548+0000 7fca007e2640 1 Processor -- start 2024-04-25T09:54:03.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.548+0000 7fca007e2640 1 -- start start 2024-04-25T09:54:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.549+0000 7fca007e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9f815efb0 0x7fc9f80d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.549+0000 7fca007e2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc9f80cf900 0x7fc9f806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.549+0000 7fca007e2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f80cfe70 con 0x7fc9f806cd80 2024-04-25T09:54:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.549+0000 7fca007e2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f80d0a80 con 0x7fc9f80cf900 2024-04-25T09:54:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.549+0000 7fca007e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80d0150 con 0x7fc9f815efb0 2024-04-25T09:54:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.549+0000 7fc9fe57e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9f815efb0 0x7fc9f80d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.549+0000 7fc9fdd7d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc9f80cf900 0x7fc9f806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.549+0000 7fc9fed7f640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc9f806cd80 0x7fc9f8099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:53878/0 (socket says 172.21.15.139:53878) 2024-04-25T09:54:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.549+0000 7fc9fdd7d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc9f80cf900 0x7fc9f806c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:49868/0 (socket says 172.21.15.139:49868) 2024-04-25T09:54:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.549+0000 7fc9fed7f640 1 -- 172.21.15.139:0/158648357 learned_addr learned my addr 172.21.15.139:0/158648357 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.550+0000 7fc9fe57e640 1 -- 172.21.15.139:0/158648357 >> v2:172.21.15.150:3300/0 conn(0x7fc9f80cf900 msgr2=0x7fc9f806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.550+0000 7fc9fe57e640 1 --2- 172.21.15.139:0/158648357 >> v2:172.21.15.150:3300/0 conn(0x7fc9f80cf900 0x7fc9f806c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.550+0000 7fc9fe57e640 1 -- 172.21.15.139:0/158648357 >> v1:172.21.15.150:6789/0 conn(0x7fc9f806cd80 legacy=0x7fc9f8099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.550+0000 7fc9fe57e640 1 -- 172.21.15.139:0/158648357 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80d0430 con 0x7fc9f815efb0 2024-04-25T09:54:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.550+0000 7fc9fdd7d640 1 --2- 172.21.15.139:0/158648357 >> v2:172.21.15.150:3300/0 conn(0x7fc9f80cf900 0x7fc9f806c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.550+0000 7fc9fe57e640 1 --2- 172.21.15.139:0/158648357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9f815efb0 0x7fc9f80d0f10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40097a0 tx=0x7fc9f402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=394437f6f65c5070 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:03.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.550+0000 7fc9fd57c640 1 -- 172.21.15.139:0/158648357 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4037070 con 0x7fc9f815efb0 2024-04-25T09:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.551+0000 7fc9fd57c640 1 -- 172.21.15.139:0/158648357 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f402ae50 con 0x7fc9f815efb0 2024-04-25T09:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.551+0000 7fc9fd57c640 1 -- 172.21.15.139:0/158648357 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4032660 con 0x7fc9f815efb0 2024-04-25T09:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.551+0000 7fca007e2640 1 -- 172.21.15.139:0/158648357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9f815efb0 msgr2=0x7fc9f80d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.551+0000 7fca007e2640 1 --2- 172.21.15.139:0/158648357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9f815efb0 0x7fc9f80d0f10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40097a0 tx=0x7fc9f402a100 comp rx=0 tx=0).stop 2024-04-25T09:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.551+0000 7fca007e2640 1 -- 172.21.15.139:0/158648357 shutdown_connections 2024-04-25T09:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.551+0000 7fca007e2640 1 --2- 172.21.15.139:0/158648357 >> v2:172.21.15.150:3300/0 conn(0x7fc9f80cf900 0x7fc9f806c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.551+0000 7fca007e2640 1 --2- 172.21.15.139:0/158648357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9f815efb0 0x7fc9f80d0f10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.551+0000 7fca007e2640 1 -- 172.21.15.139:0/158648357 >> 172.21.15.139:0/158648357 conn(0x7fc9f80d1790 msgr2=0x7fc9f80d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:03.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.551+0000 7fca007e2640 1 -- 172.21.15.139:0/158648357 shutdown_connections 2024-04-25T09:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.551+0000 7fca007e2640 1 -- 172.21.15.139:0/158648357 wait complete. 2024-04-25T09:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.552+0000 7fca007e2640 1 Processor -- start 2024-04-25T09:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.552+0000 7fca007e2640 1 -- start start 2024-04-25T09:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fca007e2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9f806cd80 0x7fc9f809f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fca007e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9f80cf900 0x7fc9f814c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fca007e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9f815efb0 0x7fc9f814a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fca007e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f809a2e0 con 0x7fc9f815efb0 2024-04-25T09:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fca007e2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f809a160 con 0x7fc9f806cd80 2024-04-25T09:54:03.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fca007e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80cbde0 con 0x7fc9f80cf900 2024-04-25T09:54:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fc9fed7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9f815efb0 0x7fc9f814a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fc9fed7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9f815efb0 0x7fc9f814a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47470/0 (socket says 172.21.15.139:47470) 2024-04-25T09:54:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fc9fed7f640 1 -- 172.21.15.139:0/704853842 learned_addr learned my addr 172.21.15.139:0/704853842 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fc9fe57e640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9f806cd80 0x7fc9f809f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fc9fdd7d640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9f80cf900 0x7fc9f814c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fc9fed7f640 1 -- 172.21.15.139:0/704853842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9f80cf900 msgr2=0x7fc9f814c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fc9fed7f640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9f80cf900 0x7fc9f814c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fc9fed7f640 1 -- 172.21.15.139:0/704853842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9f806cd80 msgr2=0x7fc9f809f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fc9fed7f640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9f806cd80 0x7fc9f809f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.553+0000 7fc9fed7f640 1 -- 172.21.15.139:0/704853842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f814c5e0 con 0x7fc9f815efb0 2024-04-25T09:54:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.554+0000 7fc9fed7f640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9f815efb0 0x7fc9f814a7b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000cb10 tx=0x7fc9f000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.554+0000 7fc9eeffd640 1 -- 172.21.15.139:0/704853842 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0017070 con 0x7fc9f815efb0 2024-04-25T09:54:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.554+0000 7fc9eeffd640 1 -- 172.21.15.139:0/704853842 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f000ac70 con 0x7fc9f815efb0 2024-04-25T09:54:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.554+0000 7fca007e2640 1 -- 172.21.15.139:0/704853842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f814b000 con 0x7fc9f815efb0 2024-04-25T09:54:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.554+0000 7fc9eeffd640 1 -- 172.21.15.139:0/704853842 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f000ae60 con 0x7fc9f815efb0 2024-04-25T09:54:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.554+0000 7fca007e2640 1 -- 172.21.15.139:0/704853842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f814b5c0 con 0x7fc9f815efb0 2024-04-25T09:54:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.555+0000 7fca007e2640 1 -- 172.21.15.139:0/704853842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f814a4d0 con 0x7fc9f815efb0 2024-04-25T09:54:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.555+0000 7fc9eeffd640 1 -- 172.21.15.139:0/704853842 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9f001a070 con 0x7fc9f815efb0 2024-04-25T09:54:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.556+0000 7fc9eeffd640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9dc06b4d0 0x7fc9dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.556+0000 7fc9fe57e640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9dc06b4d0 0x7fc9dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.556+0000 7fc9eeffd640 1 -- 172.21.15.139:0/704853842 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc9f008a160 con 0x7fc9f815efb0 2024-04-25T09:54:03.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.556+0000 7fc9fe57e640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9dc06b4d0 0x7fc9dc06d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4034000 tx=0x7fc9f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:03.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.560+0000 7fc9eeffd640 1 -- 172.21.15.139:0/704853842 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc9f814a4d0 con 0x7fc9f815efb0 2024-04-25T09:54:03.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.732+0000 7fca007e2640 1 -- 172.21.15.139:0/704853842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9f8074520 con 0x7fc9f815efb0 2024-04-25T09:54:03.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.733+0000 7fc9eeffd640 1 -- 172.21.15.139:0/704853842 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18951 (secure 0 0 0) 0x7fc9f0058620 con 0x7fc9f815efb0 2024-04-25T09:54:03.736 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:03.736 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":26,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:56.839903+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:03.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.736+0000 7fca007e2640 1 -- 172.21.15.139:0/704853842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9dc06b4d0 msgr2=0x7fc9dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.736+0000 7fca007e2640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9dc06b4d0 0x7fc9dc06d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4034000 tx=0x7fc9f402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:03.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.736+0000 7fca007e2640 1 -- 172.21.15.139:0/704853842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9f815efb0 msgr2=0x7fc9f814a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.736+0000 7fca007e2640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9f815efb0 0x7fc9f814a7b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000cb10 tx=0x7fc9f000a060 comp rx=0 tx=0).stop 2024-04-25T09:54:03.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.736+0000 7fca007e2640 1 -- 172.21.15.139:0/704853842 shutdown_connections 2024-04-25T09:54:03.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.736+0000 7fca007e2640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9f815efb0 0x7fc9f814a7b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.736+0000 7fca007e2640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9dc06b4d0 0x7fc9dc06d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.736+0000 7fca007e2640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9f80cf900 0x7fc9f814c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.736+0000 7fca007e2640 1 --2- 172.21.15.139:0/704853842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9f806cd80 0x7fc9f809f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.736+0000 7fca007e2640 1 -- 172.21.15.139:0/704853842 >> 172.21.15.139:0/704853842 conn(0x7fc9f80d1790 msgr2=0x7fc9f809a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:03.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.736+0000 7fca007e2640 1 -- 172.21.15.139:0/704853842 shutdown_connections 2024-04-25T09:54:03.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.736+0000 7fca007e2640 1 -- 172.21.15.139:0/704853842 wait complete. 2024-04-25T09:54:03.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:54:03.749 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:03.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.847+0000 7efcdc390640 1 Processor -- start 2024-04-25T09:54:03.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.847+0000 7efcdc390640 1 -- start start 2024-04-25T09:54:03.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.848+0000 7efcdc390640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efcd40b55f0 0x7efcd406c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.848+0000 7efcdc390640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd4070f50 con 0x7efcd415db40 2024-04-25T09:54:03.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.848+0000 7efcdc390640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd4071230 con 0x7efcd40b5170 2024-04-25T09:54:03.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.848+0000 7efcdc390640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd4071510 con 0x7efcd40b55f0 2024-04-25T09:54:03.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.848+0000 7efcda92d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efcd40b55f0 0x7efcd406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.848+0000 7efcda92d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efcd40b55f0 0x7efcd406c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47484/0 (socket says 172.21.15.139:47484) 2024-04-25T09:54:03.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.848+0000 7efcda92d640 1 -- 172.21.15.139:0/1248114563 learned_addr learned my addr 172.21.15.139:0/1248114563 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:03.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.848+0000 7efcda92d640 1 -- 172.21.15.139:0/1248114563 >> v1:172.21.15.150:6789/0 conn(0x7efcd40b5170 legacy=0x7efcd406bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.848+0000 7efcda92d640 1 -- 172.21.15.139:0/1248114563 >> v1:172.21.15.139:6789/0 conn(0x7efcd415db40 legacy=0x7efcd40b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.848+0000 7efcda92d640 1 -- 172.21.15.139:0/1248114563 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd40717f0 con 0x7efcd40b55f0 2024-04-25T09:54:03.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.849+0000 7efcda92d640 1 --2- 172.21.15.139:0/1248114563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd40b55f0 0x7efcd406c660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efccc009e50 tx=0x7efccc02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72f78023e25c7e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:03.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.849+0000 7efcd912a640 1 -- 172.21.15.139:0/1248114563 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc036070 con 0x7efcd40b55f0 2024-04-25T09:54:03.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.849+0000 7efcd912a640 1 -- 172.21.15.139:0/1248114563 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc031600 con 0x7efcd40b55f0 2024-04-25T09:54:03.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.849+0000 7efcd912a640 1 -- 172.21.15.139:0/1248114563 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc0319f0 con 0x7efcd40b55f0 2024-04-25T09:54:03.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.849+0000 7efcdc390640 1 -- 172.21.15.139:0/1248114563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd40b55f0 msgr2=0x7efcd406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.849+0000 7efcdc390640 1 --2- 172.21.15.139:0/1248114563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd40b55f0 0x7efcd406c660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efccc009e50 tx=0x7efccc02a7c0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.850+0000 7efcdc390640 1 -- 172.21.15.139:0/1248114563 shutdown_connections 2024-04-25T09:54:03.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.850+0000 7efcdc390640 1 --2- 172.21.15.139:0/1248114563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd40b55f0 0x7efcd406c660 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.850+0000 7efcdc390640 1 -- 172.21.15.139:0/1248114563 >> 172.21.15.139:0/1248114563 conn(0x7efcd406f460 msgr2=0x7efcd406f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:03.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.850+0000 7efcdc390640 1 -- 172.21.15.139:0/1248114563 shutdown_connections 2024-04-25T09:54:03.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.850+0000 7efcdc390640 1 -- 172.21.15.139:0/1248114563 wait complete. 2024-04-25T09:54:03.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.851+0000 7efcdc390640 1 Processor -- start 2024-04-25T09:54:03.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.851+0000 7efcdc390640 1 -- start start 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.851+0000 7efcdc390640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcd40b5170 0x7efcd40cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.851+0000 7efcdc390640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efcd40b55f0 0x7efcd414f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.851+0000 7efcdc390640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd415db40 0x7efcd41e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.851+0000 7efcdc390640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40b4d90 con 0x7efcd415db40 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.851+0000 7efcdc390640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40b5a80 con 0x7efcd40b5170 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.851+0000 7efcdc390640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4070be0 con 0x7efcd40b55f0 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcda92d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd415db40 0x7efcd41e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcda92d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd415db40 0x7efcd41e7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47488/0 (socket says 172.21.15.139:47488) 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcda92d640 1 -- 172.21.15.139:0/1527892092 learned_addr learned my addr 172.21.15.139:0/1527892092 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcda12c640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcd40b5170 0x7efcd40cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcda92d640 1 -- 172.21.15.139:0/1527892092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efcd40b55f0 msgr2=0x7efcd414f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcd992b640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efcd40b55f0 0x7efcd414f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcda92d640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efcd40b55f0 0x7efcd414f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcda92d640 1 -- 172.21.15.139:0/1527892092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcd40b5170 msgr2=0x7efcd40cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcda92d640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcd40b5170 0x7efcd40cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcda92d640 1 -- 172.21.15.139:0/1527892092 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd41e83d0 con 0x7efcd415db40 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcda12c640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcd40b5170 0x7efcd40cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcd992b640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efcd40b55f0 0x7efcd414f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcda92d640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd415db40 0x7efcd41e7e90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efccc02a800 tx=0x7efccc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efccaffd640 1 -- 172.21.15.139:0/1527892092 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc036050 con 0x7efcd415db40 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efccaffd640 1 -- 172.21.15.139:0/1527892092 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc042070 con 0x7efcd415db40 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efccaffd640 1 -- 172.21.15.139:0/1527892092 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc009510 con 0x7efcd415db40 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcdc390640 1 -- 172.21.15.139:0/1527892092 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd40cc730 con 0x7efcd415db40 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.852+0000 7efcdc390640 1 -- 172.21.15.139:0/1527892092 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd40cccf0 con 0x7efcd415db40 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.853+0000 7efcdc390640 1 -- 172.21.15.139:0/1527892092 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd4074520 con 0x7efcd415db40 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.854+0000 7efccaffd640 1 -- 172.21.15.139:0/1527892092 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efccc004450 con 0x7efcd415db40 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.854+0000 7efccaffd640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efcb806b500 0x7efcb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:03.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.854+0000 7efccaffd640 1 -- 172.21.15.139:0/1527892092 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7efccc0ab540 con 0x7efcd415db40 2024-04-25T09:54:03.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.854+0000 7efcda12c640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efcb806b500 0x7efcb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:03.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.854+0000 7efcda12c640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efcb806b500 0x7efcb806d9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efcd0000f60 tx=0x7efcd0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:03.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:03.858+0000 7efccaffd640 1 -- 172.21.15.139:0/1527892092 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efccc079fc0 con 0x7efcd415db40 2024-04-25T09:54:04.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.030+0000 7efcdc390640 1 -- 172.21.15.139:0/1527892092 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcd40701f0 con 0x7efcd415db40 2024-04-25T09:54:04.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.031+0000 7efccaffd640 1 -- 172.21.15.139:0/1527892092 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7efccc0798c0 con 0x7efcd415db40 2024-04-25T09:54:04.033 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:54:04.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.033+0000 7efcdc390640 1 -- 172.21.15.139:0/1527892092 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efcb806b500 msgr2=0x7efcb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.033+0000 7efcdc390640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efcb806b500 0x7efcb806d9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efcd0000f60 tx=0x7efcd0009040 comp rx=0 tx=0).stop 2024-04-25T09:54:04.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.033+0000 7efcdc390640 1 -- 172.21.15.139:0/1527892092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd415db40 msgr2=0x7efcd41e7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.033+0000 7efcdc390640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd415db40 0x7efcd41e7e90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efccc02a800 tx=0x7efccc004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:04.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.033+0000 7efcdc390640 1 -- 172.21.15.139:0/1527892092 shutdown_connections 2024-04-25T09:54:04.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.033+0000 7efcdc390640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd415db40 0x7efcd41e7e90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.033+0000 7efcdc390640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efcb806b500 0x7efcb806d9c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.033+0000 7efcdc390640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efcd40b55f0 0x7efcd414f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.033+0000 7efcdc390640 1 --2- 172.21.15.139:0/1527892092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcd40b5170 0x7efcd40cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.033+0000 7efcdc390640 1 -- 172.21.15.139:0/1527892092 >> 172.21.15.139:0/1527892092 conn(0x7efcd406f460 msgr2=0x7efcd41533f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:04.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.033+0000 7efcdc390640 1 -- 172.21.15.139:0/1527892092 shutdown_connections 2024-04-25T09:54:04.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.033+0000 7efcdc390640 1 -- 172.21.15.139:0/1527892092 wait complete. 2024-04-25T09:54:04.046 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:54:04.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.143+0000 7fd4e646f640 1 Processor -- start 2024-04-25T09:54:04.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.143+0000 7fd4e646f640 1 -- start start 2024-04-25T09:54:04.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.144+0000 7fd4e646f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd4e006ebf0 0x7fd4e006f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.144+0000 7fd4e646f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e00cf980 con 0x7fd4e00b2db0 2024-04-25T09:54:04.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.144+0000 7fd4e646f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e006ab60 con 0x7fd4e015dc90 2024-04-25T09:54:04.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.144+0000 7fd4e646f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e006ae40 con 0x7fd4e006ebf0 2024-04-25T09:54:04.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.144+0000 7fd4df7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd4e006ebf0 0x7fd4e006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.144+0000 7fd4df7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd4e006ebf0 0x7fd4e006f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47504/0 (socket says 172.21.15.139:47504) 2024-04-25T09:54:04.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.144+0000 7fd4df7fe640 1 -- 172.21.15.139:0/1763988819 learned_addr learned my addr 172.21.15.139:0/1763988819 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:04.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.144+0000 7fd4df7fe640 1 -- 172.21.15.139:0/1763988819 >> v1:172.21.15.150:6789/0 conn(0x7fd4e015dc90 legacy=0x7fd4e00ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:04.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.144+0000 7fd4df7fe640 1 -- 172.21.15.139:0/1763988819 >> v1:172.21.15.139:6789/0 conn(0x7fd4e00b2db0 legacy=0x7fd4e00b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.144+0000 7fd4df7fe640 1 -- 172.21.15.139:0/1763988819 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e006b120 con 0x7fd4e006ebf0 2024-04-25T09:54:04.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.145+0000 7fd4df7fe640 1 --2- 172.21.15.139:0/1763988819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e006ebf0 0x7fd4e006f080 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009870 tx=0x7fd4d402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da91bb54c1001642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:04.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.145+0000 7fd4deffd640 1 -- 172.21.15.139:0/1763988819 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4037070 con 0x7fd4e006ebf0 2024-04-25T09:54:04.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.145+0000 7fd4deffd640 1 -- 172.21.15.139:0/1763988819 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d4032440 con 0x7fd4e006ebf0 2024-04-25T09:54:04.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.145+0000 7fd4deffd640 1 -- 172.21.15.139:0/1763988819 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4032810 con 0x7fd4e006ebf0 2024-04-25T09:54:04.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.146+0000 7fd4e646f640 1 -- 172.21.15.139:0/1763988819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e006ebf0 msgr2=0x7fd4e006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.146+0000 7fd4e646f640 1 --2- 172.21.15.139:0/1763988819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e006ebf0 0x7fd4e006f080 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009870 tx=0x7fd4d402a2a0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.146+0000 7fd4e646f640 1 -- 172.21.15.139:0/1763988819 shutdown_connections 2024-04-25T09:54:04.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.146+0000 7fd4e646f640 1 --2- 172.21.15.139:0/1763988819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e006ebf0 0x7fd4e006f080 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.146+0000 7fd4e646f640 1 -- 172.21.15.139:0/1763988819 >> 172.21.15.139:0/1763988819 conn(0x7fd4e00cfd80 msgr2=0x7fd4e00d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:04.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.146+0000 7fd4e646f640 1 -- 172.21.15.139:0/1763988819 shutdown_connections 2024-04-25T09:54:04.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.146+0000 7fd4e646f640 1 -- 172.21.15.139:0/1763988819 wait complete. 2024-04-25T09:54:04.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.147+0000 7fd4e646f640 1 Processor -- start 2024-04-25T09:54:04.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.147+0000 7fd4e646f640 1 -- start start 2024-04-25T09:54:04.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4e646f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e006ebf0 0x7fd4e0160860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4e646f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e00b2db0 0x7fd4e014bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4e646f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4e015dc90 0x7fd4e015b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4e646f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0072620 con 0x7fd4e006ebf0 2024-04-25T09:54:04.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4e646f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e006f5c0 con 0x7fd4e00b2db0 2024-04-25T09:54:04.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4e646f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e006b920 con 0x7fd4e015dc90 2024-04-25T09:54:04.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4dffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e006ebf0 0x7fd4e0160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4dffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e006ebf0 0x7fd4e0160860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47508/0 (socket says 172.21.15.139:47508) 2024-04-25T09:54:04.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4dffff640 1 -- 172.21.15.139:0/3473066489 learned_addr learned my addr 172.21.15.139:0/3473066489 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:04.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4df7fe640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e00b2db0 0x7fd4e014bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4e4a0c640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4e015dc90 0x7fd4e015b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4dffff640 1 -- 172.21.15.139:0/3473066489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4e015dc90 msgr2=0x7fd4e015b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4dffff640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4e015dc90 0x7fd4e015b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4dffff640 1 -- 172.21.15.139:0/3473066489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e00b2db0 msgr2=0x7fd4e014bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4dffff640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e00b2db0 0x7fd4e014bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4dffff640 1 -- 172.21.15.139:0/3473066489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0156680 con 0x7fd4e006ebf0 2024-04-25T09:54:04.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4dffff640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e006ebf0 0x7fd4e0160860 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000b420 tx=0x7fd4d000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:04.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.148+0000 7fd4dcff9640 1 -- 172.21.15.139:0/3473066489 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0011830 con 0x7fd4e006ebf0 2024-04-25T09:54:04.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.149+0000 7fd4dcff9640 1 -- 172.21.15.139:0/3473066489 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d0016070 con 0x7fd4e006ebf0 2024-04-25T09:54:04.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.149+0000 7fd4dcff9640 1 -- 172.21.15.139:0/3473066489 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0011c10 con 0x7fd4e006ebf0 2024-04-25T09:54:04.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.149+0000 7fd4e646f640 1 -- 172.21.15.139:0/3473066489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e0156960 con 0x7fd4e006ebf0 2024-04-25T09:54:04.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.149+0000 7fd4e646f640 1 -- 172.21.15.139:0/3473066489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e0156f20 con 0x7fd4e006ebf0 2024-04-25T09:54:04.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.149+0000 7fd4e646f640 1 -- 172.21.15.139:0/3473066489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e0074520 con 0x7fd4e006ebf0 2024-04-25T09:54:04.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.150+0000 7fd4dcff9640 1 -- 172.21.15.139:0/3473066489 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4d001b460 con 0x7fd4e006ebf0 2024-04-25T09:54:04.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.150+0000 7fd4dcff9640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.151+0000 7fd4df7fe640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.151+0000 7fd4dcff9640 1 -- 172.21.15.139:0/3473066489 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd4d008a090 con 0x7fd4e006ebf0 2024-04-25T09:54:04.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.151+0000 7fd4df7fe640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4034000 tx=0x7fd4d402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:04.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.155+0000 7fd4dcff9640 1 -- 172.21.15.139:0/3473066489 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4d0020020 con 0x7fd4e006ebf0 2024-04-25T09:54:04.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.322+0000 7fd4e646f640 1 -- 172.21.15.139:0/3473066489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4e0067770 con 0x7fd4e006ebf0 2024-04-25T09:54:04.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.322+0000 7fd4dcff9640 1 -- 172.21.15.139:0/3473066489 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd4d0058ba0 con 0x7fd4e006ebf0 2024-04-25T09:54:04.324 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:54:04.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.324+0000 7fd4e646f640 1 -- 172.21.15.139:0/3473066489 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd4cc06b4d0 msgr2=0x7fd4cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.324+0000 7fd4e646f640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4034000 tx=0x7fd4d402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:04.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.324+0000 7fd4e646f640 1 -- 172.21.15.139:0/3473066489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e006ebf0 msgr2=0x7fd4e0160860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.324+0000 7fd4e646f640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e006ebf0 0x7fd4e0160860 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000b420 tx=0x7fd4d000b8e0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.324+0000 7fd4e646f640 1 -- 172.21.15.139:0/3473066489 shutdown_connections 2024-04-25T09:54:04.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.324+0000 7fd4e646f640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.324+0000 7fd4e646f640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4e015dc90 0x7fd4e015b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.324+0000 7fd4e646f640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e00b2db0 0x7fd4e014bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.324+0000 7fd4e646f640 1 --2- 172.21.15.139:0/3473066489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e006ebf0 0x7fd4e0160860 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.324+0000 7fd4e646f640 1 -- 172.21.15.139:0/3473066489 >> 172.21.15.139:0/3473066489 conn(0x7fd4e00cfd80 msgr2=0x7fd4e00aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:04.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.324+0000 7fd4e646f640 1 -- 172.21.15.139:0/3473066489 shutdown_connections 2024-04-25T09:54:04.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.324+0000 7fd4e646f640 1 -- 172.21.15.139:0/3473066489 wait complete. 2024-04-25T09:54:04.337 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:54:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.434+0000 7fa54e19e640 1 Processor -- start 2024-04-25T09:54:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.434+0000 7fa54e19e640 1 -- start start 2024-04-25T09:54:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.435+0000 7fa54e19e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa5480b4960 0x7fa5480668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.435+0000 7fa54e19e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa5481616d0 0x7fa548161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.435+0000 7fa54e19e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa548162030 con 0x7fa54815d9e0 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.435+0000 7fa54e19e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5480b34f0 con 0x7fa5480b4960 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.435+0000 7fa54e19e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5480b4ed0 con 0x7fa5481616d0 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.435+0000 7fa546ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa5480b4960 0x7fa5480668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.435+0000 7fa546ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa5480b4960 0x7fa5480668f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47522/0 (socket says 172.21.15.139:47522) 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.435+0000 7fa546ffd640 1 -- 172.21.15.139:0/513199102 learned_addr learned my addr 172.21.15.139:0/513199102 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.436+0000 7fa547fff640 1 --2- 172.21.15.139:0/513199102 >> v2:172.21.15.150:3300/0 conn(0x7fa5481616d0 0x7fa548161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.436+0000 7fa546ffd640 1 -- 172.21.15.139:0/513199102 >> v2:172.21.15.150:3300/0 conn(0x7fa5481616d0 msgr2=0x7fa548161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.436+0000 7fa546ffd640 1 --2- 172.21.15.139:0/513199102 >> v2:172.21.15.150:3300/0 conn(0x7fa5481616d0 0x7fa548161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.436+0000 7fa546ffd640 1 -- 172.21.15.139:0/513199102 >> v1:172.21.15.139:6789/0 conn(0x7fa54815d9e0 legacy=0x7fa54806bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.436+0000 7fa546ffd640 1 -- 172.21.15.139:0/513199102 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5480b2f30 con 0x7fa5480b4960 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.436+0000 7fa546ffd640 1 --2- 172.21.15.139:0/513199102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5480b4960 0x7fa5480668f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa5480b6040 tx=0x7fa53c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1c54ce9d98f000 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.436+0000 7fa5467fc640 1 -- 172.21.15.139:0/513199102 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c037070 con 0x7fa5480b4960 2024-04-25T09:54:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.437+0000 7fa5467fc640 1 -- 172.21.15.139:0/513199102 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53c032440 con 0x7fa5480b4960 2024-04-25T09:54:04.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.437+0000 7fa5467fc640 1 -- 172.21.15.139:0/513199102 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c032810 con 0x7fa5480b4960 2024-04-25T09:54:04.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.437+0000 7fa54e19e640 1 -- 172.21.15.139:0/513199102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5480b4960 msgr2=0x7fa5480668f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.437+0000 7fa54e19e640 1 --2- 172.21.15.139:0/513199102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5480b4960 0x7fa5480668f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa5480b6040 tx=0x7fa53c02a270 comp rx=0 tx=0).stop 2024-04-25T09:54:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.437+0000 7fa54e19e640 1 -- 172.21.15.139:0/513199102 shutdown_connections 2024-04-25T09:54:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.437+0000 7fa54e19e640 1 --2- 172.21.15.139:0/513199102 >> v2:172.21.15.150:3300/0 conn(0x7fa5481616d0 0x7fa548161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.437+0000 7fa54e19e640 1 --2- 172.21.15.139:0/513199102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5480b4960 0x7fa5480668f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.437+0000 7fa54e19e640 1 -- 172.21.15.139:0/513199102 >> 172.21.15.139:0/513199102 conn(0x7fa5480cc820 msgr2=0x7fa5480ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.437+0000 7fa54e19e640 1 -- 172.21.15.139:0/513199102 shutdown_connections 2024-04-25T09:54:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.437+0000 7fa54e19e640 1 -- 172.21.15.139:0/513199102 wait complete. 2024-04-25T09:54:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.438+0000 7fa54e19e640 1 Processor -- start 2024-04-25T09:54:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.438+0000 7fa54e19e640 1 -- start start 2024-04-25T09:54:04.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.439+0000 7fa54e19e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5480b4960 0x7fa5481526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.439+0000 7fa54e19e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa54815d9e0 0x7fa548152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.439+0000 7fa54e19e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5481616d0 0x7fa548153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.439+0000 7fa54e19e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54815ddc0 con 0x7fa5480b4960 2024-04-25T09:54:04.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.439+0000 7fa54e19e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa548162430 con 0x7fa5481616d0 2024-04-25T09:54:04.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.439+0000 7fa54e19e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54815df40 con 0x7fa54815d9e0 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.439+0000 7fa5477fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5480b4960 0x7fa5481526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.439+0000 7fa5477fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5480b4960 0x7fa5481526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47528/0 (socket says 172.21.15.139:47528) 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.439+0000 7fa5477fe640 1 -- 172.21.15.139:0/2707673014 learned_addr learned my addr 172.21.15.139:0/2707673014 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa547fff640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5481616d0 0x7fa548153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa546ffd640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa54815d9e0 0x7fa548152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa5477fe640 1 -- 172.21.15.139:0/2707673014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa54815d9e0 msgr2=0x7fa548152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa5477fe640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa54815d9e0 0x7fa548152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa5477fe640 1 -- 172.21.15.139:0/2707673014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5481616d0 msgr2=0x7fa548153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa5477fe640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5481616d0 0x7fa548153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa5477fe640 1 -- 172.21.15.139:0/2707673014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa548153730 con 0x7fa5480b4960 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa5477fe640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5480b4960 0x7fa5481526d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa53800c400 tx=0x7fa53800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa52ffff640 1 -- 172.21.15.139:0/2707673014 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5380052d0 con 0x7fa5480b4960 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa52ffff640 1 -- 172.21.15.139:0/2707673014 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa538005490 con 0x7fa5480b4960 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa54e19e640 1 -- 172.21.15.139:0/2707673014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5481544d0 con 0x7fa5480b4960 2024-04-25T09:54:04.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa52ffff640 1 -- 172.21.15.139:0/2707673014 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538007260 con 0x7fa5480b4960 2024-04-25T09:54:04.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.440+0000 7fa54e19e640 1 -- 172.21.15.139:0/2707673014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa548154a90 con 0x7fa5480b4960 2024-04-25T09:54:04.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.441+0000 7fa54e19e640 1 -- 172.21.15.139:0/2707673014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa548074520 con 0x7fa5480b4960 2024-04-25T09:54:04.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.441+0000 7fa52ffff640 1 -- 172.21.15.139:0/2707673014 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa538007400 con 0x7fa5480b4960 2024-04-25T09:54:04.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.442+0000 7fa52ffff640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa53406b4d0 0x7fa53406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.442+0000 7fa52ffff640 1 -- 172.21.15.139:0/2707673014 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa538089b50 con 0x7fa5480b4960 2024-04-25T09:54:04.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.442+0000 7fa546ffd640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa53406b4d0 0x7fa53406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.442+0000 7fa546ffd640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa53406b4d0 0x7fa53406d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa53c034000 tx=0x7fa53c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:04.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.446+0000 7fa52ffff640 1 -- 172.21.15.139:0/2707673014 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5380585d0 con 0x7fa5480b4960 2024-04-25T09:54:04.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.619+0000 7fa54e19e640 1 -- 172.21.15.139:0/2707673014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5480b0810 con 0x7fa5480b4960 2024-04-25T09:54:04.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.619+0000 7fa52ffff640 1 -- 172.21.15.139:0/2707673014 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa538057ed0 con 0x7fa5480b4960 2024-04-25T09:54:04.621 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:54:04.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.621+0000 7fa54e19e640 1 -- 172.21.15.139:0/2707673014 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa53406b4d0 msgr2=0x7fa53406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.621+0000 7fa54e19e640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa53406b4d0 0x7fa53406d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa53c034000 tx=0x7fa53c02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:04.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.621+0000 7fa54e19e640 1 -- 172.21.15.139:0/2707673014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5480b4960 msgr2=0x7fa5481526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.621+0000 7fa54e19e640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5480b4960 0x7fa5481526d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa53800c400 tx=0x7fa53800c8c0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.621+0000 7fa54e19e640 1 -- 172.21.15.139:0/2707673014 shutdown_connections 2024-04-25T09:54:04.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.621+0000 7fa54e19e640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa53406b4d0 0x7fa53406d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.621+0000 7fa54e19e640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5481616d0 0x7fa548153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.621+0000 7fa54e19e640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa54815d9e0 0x7fa548152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.621+0000 7fa54e19e640 1 --2- 172.21.15.139:0/2707673014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5480b4960 0x7fa5481526d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.621+0000 7fa54e19e640 1 -- 172.21.15.139:0/2707673014 >> 172.21.15.139:0/2707673014 conn(0x7fa5480cc820 msgr2=0x7fa5480b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:04.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.622+0000 7fa54e19e640 1 -- 172.21.15.139:0/2707673014 shutdown_connections 2024-04-25T09:54:04.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.622+0000 7fa54e19e640 1 -- 172.21.15.139:0/2707673014 wait complete. 2024-04-25T09:54:04.634 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:54:04.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.732+0000 7fc53c1c4640 1 Processor -- start 2024-04-25T09:54:04.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.732+0000 7fc53c1c4640 1 -- start start 2024-04-25T09:54:04.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.733+0000 7fc53c1c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc534071a50 0x7fc5340cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.733+0000 7fc53c1c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc5340ce040 0x7fc5340ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.733+0000 7fc53c1c4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc53415c490 con 0x7fc53415bf90 2024-04-25T09:54:04.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.733+0000 7fc53c1c4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53415c770 con 0x7fc5340ce040 2024-04-25T09:54:04.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.733+0000 7fc53c1c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53415ca50 con 0x7fc534071a50 2024-04-25T09:54:04.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.733+0000 7fc53a761640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc5340ce040 0x7fc5340ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.733+0000 7fc53a761640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc5340ce040 0x7fc5340ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47542/0 (socket says 172.21.15.139:47542) 2024-04-25T09:54:04.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.733+0000 7fc53a761640 1 -- 172.21.15.139:0/3981406565 learned_addr learned my addr 172.21.15.139:0/3981406565 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:04.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.733+0000 7fc539f60640 1 --2- 172.21.15.139:0/3981406565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc534071a50 0x7fc5340cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.734+0000 7fc53a761640 1 -- 172.21.15.139:0/3981406565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc534071a50 msgr2=0x7fc5340cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:04.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.734+0000 7fc53a761640 1 --2- 172.21.15.139:0/3981406565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc534071a50 0x7fc5340cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.734+0000 7fc53a761640 1 -- 172.21.15.139:0/3981406565 >> v1:172.21.15.150:6789/0 conn(0x7fc53415bf90 legacy=0x7fc534071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:04.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.734+0000 7fc53a761640 1 -- 172.21.15.139:0/3981406565 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53415cd30 con 0x7fc5340ce040 2024-04-25T09:54:04.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.734+0000 7fc539f60640 1 --2- 172.21.15.139:0/3981406565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc534071a50 0x7fc5340cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:04.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.734+0000 7fc53a761640 1 --2- 172.21.15.139:0/3981406565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5340ce040 0x7fc5340ce4f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009e50 tx=0x7fc52c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36db7b3d2b82d3d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:04.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.734+0000 7fc538f5e640 1 -- 172.21.15.139:0/3981406565 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c036070 con 0x7fc5340ce040 2024-04-25T09:54:04.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.734+0000 7fc538f5e640 1 -- 172.21.15.139:0/3981406565 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c031600 con 0x7fc5340ce040 2024-04-25T09:54:04.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.734+0000 7fc538f5e640 1 -- 172.21.15.139:0/3981406565 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0319d0 con 0x7fc5340ce040 2024-04-25T09:54:04.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.735+0000 7fc53c1c4640 1 -- 172.21.15.139:0/3981406565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5340ce040 msgr2=0x7fc5340ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.735+0000 7fc53c1c4640 1 --2- 172.21.15.139:0/3981406565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5340ce040 0x7fc5340ce4f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009e50 tx=0x7fc52c02a630 comp rx=0 tx=0).stop 2024-04-25T09:54:04.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.735+0000 7fc53c1c4640 1 -- 172.21.15.139:0/3981406565 shutdown_connections 2024-04-25T09:54:04.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.735+0000 7fc53c1c4640 1 --2- 172.21.15.139:0/3981406565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5340ce040 0x7fc5340ce4f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.735+0000 7fc53c1c4640 1 --2- 172.21.15.139:0/3981406565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc534071a50 0x7fc5340cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.735+0000 7fc53c1c4640 1 -- 172.21.15.139:0/3981406565 >> 172.21.15.139:0/3981406565 conn(0x7fc5340cfd80 msgr2=0x7fc5340d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:04.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.735+0000 7fc53c1c4640 1 -- 172.21.15.139:0/3981406565 shutdown_connections 2024-04-25T09:54:04.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.735+0000 7fc53c1c4640 1 -- 172.21.15.139:0/3981406565 wait complete. 2024-04-25T09:54:04.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.736+0000 7fc53c1c4640 1 Processor -- start 2024-04-25T09:54:04.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.736+0000 7fc53c1c4640 1 -- start start 2024-04-25T09:54:04.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53c1c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc534071a50 0x7fc53409d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53c1c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5340ce040 0x7fc53409c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53c1c4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc53415bf90 0x7fc534153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53c1c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc534156280 con 0x7fc5340ce040 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53c1c4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340728b0 con 0x7fc53415bf90 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53c1c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc534156400 con 0x7fc534071a50 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53975f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5340ce040 0x7fc53409c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53975f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5340ce040 0x7fc53409c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47556/0 (socket says 172.21.15.139:47556) 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53975f640 1 -- 172.21.15.139:0/529484313 learned_addr learned my addr 172.21.15.139:0/529484313 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53a761640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc53415bf90 0x7fc534153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc539f60640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc534071a50 0x7fc53409d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53975f640 1 -- 172.21.15.139:0/529484313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc534071a50 msgr2=0x7fc53409d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53975f640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc534071a50 0x7fc53409d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53975f640 1 -- 172.21.15.139:0/529484313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc53415bf90 msgr2=0x7fc534153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53975f640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc53415bf90 0x7fc534153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53975f640 1 -- 172.21.15.139:0/529484313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53409caf0 con 0x7fc5340ce040 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc539f60640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc534071a50 0x7fc53409d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53a761640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc53415bf90 0x7fc534153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.737+0000 7fc53975f640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5340ce040 0x7fc53409c5b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc52400c430 tx=0x7fc52400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.738+0000 7fc52a7fc640 1 -- 172.21.15.139:0/529484313 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524011790 con 0x7fc5340ce040 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.738+0000 7fc52a7fc640 1 -- 172.21.15.139:0/529484313 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc524017070 con 0x7fc5340ce040 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.738+0000 7fc53c1c4640 1 -- 172.21.15.139:0/529484313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53409cd70 con 0x7fc5340ce040 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.738+0000 7fc52a7fc640 1 -- 172.21.15.139:0/529484313 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524011bb0 con 0x7fc5340ce040 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.738+0000 7fc53c1c4640 1 -- 172.21.15.139:0/529484313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53415aa50 con 0x7fc5340ce040 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.739+0000 7fc53c1c4640 1 -- 172.21.15.139:0/529484313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc534074520 con 0x7fc5340ce040 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.739+0000 7fc52a7fc640 1 -- 172.21.15.139:0/529484313 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc52401c460 con 0x7fc5340ce040 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.739+0000 7fc52a7fc640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc51806b500 0x7fc51806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.740+0000 7fc52a7fc640 1 -- 172.21.15.139:0/529484313 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc52408a490 con 0x7fc5340ce040 2024-04-25T09:54:04.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.740+0000 7fc539f60640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc51806b500 0x7fc51806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:04.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.740+0000 7fc539f60640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc51806b500 0x7fc51806d9c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc53000a990 tx=0x7fc530008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:04.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.744+0000 7fc52a7fc640 1 -- 172.21.15.139:0/529484313 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc53409caf0 con 0x7fc5340ce040 2024-04-25T09:54:04.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.915+0000 7fc53c1c4640 1 -- 172.21.15.139:0/529484313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc5340aeda0 con 0x7fc5340ce040 2024-04-25T09:54:04.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.915+0000 7fc52a7fc640 1 -- 172.21.15.139:0/529484313 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc524058f10 con 0x7fc5340ce040 2024-04-25T09:54:04.917 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:54:04.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.917+0000 7fc53c1c4640 1 -- 172.21.15.139:0/529484313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc51806b500 msgr2=0x7fc51806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.917+0000 7fc53c1c4640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc51806b500 0x7fc51806d9c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc53000a990 tx=0x7fc530008040 comp rx=0 tx=0).stop 2024-04-25T09:54:04.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.917+0000 7fc53c1c4640 1 -- 172.21.15.139:0/529484313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5340ce040 msgr2=0x7fc53409c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:04.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.917+0000 7fc53c1c4640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5340ce040 0x7fc53409c5b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc52400c430 tx=0x7fc52400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.917+0000 7fc53c1c4640 1 -- 172.21.15.139:0/529484313 shutdown_connections 2024-04-25T09:54:04.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.917+0000 7fc53c1c4640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5340ce040 0x7fc53409c5b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.917+0000 7fc53c1c4640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc51806b500 0x7fc51806d9c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.917+0000 7fc53c1c4640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc53415bf90 0x7fc534153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.917+0000 7fc53c1c4640 1 --2- 172.21.15.139:0/529484313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc534071a50 0x7fc53409d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:04.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.917+0000 7fc53c1c4640 1 -- 172.21.15.139:0/529484313 >> 172.21.15.139:0/529484313 conn(0x7fc5340cfd80 msgr2=0x7fc53415e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:04.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.917+0000 7fc53c1c4640 1 -- 172.21.15.139:0/529484313 shutdown_connections 2024-04-25T09:54:04.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:04.918+0000 7fc53c1c4640 1 -- 172.21.15.139:0/529484313 wait complete. 2024-04-25T09:54:04.931 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:54:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.027+0000 7fc4b855c640 1 Processor -- start 2024-04-25T09:54:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.027+0000 7fc4b855c640 1 -- start start 2024-04-25T09:54:05.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.028+0000 7fc4b855c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc4b0145250 0x7fc4b0070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.028+0000 7fc4b855c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4b00b5690 0x7fc4b0071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.028+0000 7fc4b855c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b00b1100 con 0x7fc4b00719d0 2024-04-25T09:54:05.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.028+0000 7fc4b855c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b00ceed0 con 0x7fc4b00b5690 2024-04-25T09:54:05.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.028+0000 7fc4b855c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b00ce2c0 con 0x7fc4b0145250 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.028+0000 7fc4b6af9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4b00b5690 0x7fc4b0071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.028+0000 7fc4b6af9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4b00b5690 0x7fc4b0071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47564/0 (socket says 172.21.15.139:47564) 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.028+0000 7fc4b6af9640 1 -- 172.21.15.139:0/230754442 learned_addr learned my addr 172.21.15.139:0/230754442 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.028+0000 7fc4b62f8640 1 --2- 172.21.15.139:0/230754442 >> v2:172.21.15.150:3300/0 conn(0x7fc4b0145250 0x7fc4b0070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.029+0000 7fc4b6af9640 1 -- 172.21.15.139:0/230754442 >> v2:172.21.15.150:3300/0 conn(0x7fc4b0145250 msgr2=0x7fc4b0070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.029+0000 7fc4b6af9640 1 --2- 172.21.15.139:0/230754442 >> v2:172.21.15.150:3300/0 conn(0x7fc4b0145250 0x7fc4b0070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.029+0000 7fc4b6af9640 1 -- 172.21.15.139:0/230754442 >> v1:172.21.15.139:6789/0 conn(0x7fc4b00719d0 legacy=0x7fc4b015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.029+0000 7fc4b6af9640 1 -- 172.21.15.139:0/230754442 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b00ce5a0 con 0x7fc4b00b5690 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.029+0000 7fc4b6af9640 1 --2- 172.21.15.139:0/230754442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b00b5690 0x7fc4b0071120 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc4b006efc0 tx=0x7fc4a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8437752618fca226 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.029+0000 7fc4b52f6640 1 -- 172.21.15.139:0/230754442 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8037070 con 0x7fc4b00b5690 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.030+0000 7fc4b52f6640 1 -- 172.21.15.139:0/230754442 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a8032440 con 0x7fc4b00b5690 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.030+0000 7fc4b52f6640 1 -- 172.21.15.139:0/230754442 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8032810 con 0x7fc4b00b5690 2024-04-25T09:54:05.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.030+0000 7fc4b855c640 1 -- 172.21.15.139:0/230754442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b00b5690 msgr2=0x7fc4b0071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.030+0000 7fc4b855c640 1 --2- 172.21.15.139:0/230754442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b00b5690 0x7fc4b0071120 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc4b006efc0 tx=0x7fc4a802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.030+0000 7fc4b855c640 1 -- 172.21.15.139:0/230754442 shutdown_connections 2024-04-25T09:54:05.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.030+0000 7fc4b855c640 1 --2- 172.21.15.139:0/230754442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b00b5690 0x7fc4b0071120 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.030+0000 7fc4b855c640 1 --2- 172.21.15.139:0/230754442 >> v2:172.21.15.150:3300/0 conn(0x7fc4b0145250 0x7fc4b0070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.030+0000 7fc4b855c640 1 -- 172.21.15.139:0/230754442 >> 172.21.15.139:0/230754442 conn(0x7fc4b006f460 msgr2=0x7fc4b006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:05.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.030+0000 7fc4b855c640 1 -- 172.21.15.139:0/230754442 shutdown_connections 2024-04-25T09:54:05.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.030+0000 7fc4b855c640 1 -- 172.21.15.139:0/230754442 wait complete. 2024-04-25T09:54:05.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.031+0000 7fc4b855c640 1 Processor -- start 2024-04-25T09:54:05.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.031+0000 7fc4b855c640 1 -- start start 2024-04-25T09:54:05.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b855c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b00719d0 0x7fc4b01e5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b855c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4b00b5690 0x7fc4b00b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b855c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4b0145250 0x7fc4b01dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b855c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b0145630 con 0x7fc4b00719d0 2024-04-25T09:54:05.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b855c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00ceb60 con 0x7fc4b0145250 2024-04-25T09:54:05.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b855c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00d11a0 con 0x7fc4b00b5690 2024-04-25T09:54:05.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b62f8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b00719d0 0x7fc4b01e5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b62f8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b00719d0 0x7fc4b01e5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47568/0 (socket says 172.21.15.139:47568) 2024-04-25T09:54:05.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b62f8640 1 -- 172.21.15.139:0/270980898 learned_addr learned my addr 172.21.15.139:0/270980898 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b6af9640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4b0145250 0x7fc4b01dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b5af7640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4b00b5690 0x7fc4b00b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b62f8640 1 -- 172.21.15.139:0/270980898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4b00b5690 msgr2=0x7fc4b00b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b62f8640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4b00b5690 0x7fc4b00b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b62f8640 1 -- 172.21.15.139:0/270980898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4b0145250 msgr2=0x7fc4b01dffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b62f8640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4b0145250 0x7fc4b01dffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b62f8640 1 -- 172.21.15.139:0/270980898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b01e6440 con 0x7fc4b00719d0 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.032+0000 7fc4b62f8640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b00719d0 0x7fc4b01e5f00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00efc0 tx=0x7fc4ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.033+0000 7fc4a6ffd640 1 -- 172.21.15.139:0/270980898 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac018020 con 0x7fc4b00719d0 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.033+0000 7fc4a6ffd640 1 -- 172.21.15.139:0/270980898 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac014480 con 0x7fc4b00719d0 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.033+0000 7fc4a6ffd640 1 -- 172.21.15.139:0/270980898 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac014850 con 0x7fc4b00719d0 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.033+0000 7fc4b855c640 1 -- 172.21.15.139:0/270980898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b0142bc0 con 0x7fc4b00719d0 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.033+0000 7fc4b855c640 1 -- 172.21.15.139:0/270980898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b0143180 con 0x7fc4b00719d0 2024-04-25T09:54:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.034+0000 7fc4b855c640 1 -- 172.21.15.139:0/270980898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b0074520 con 0x7fc4b00719d0 2024-04-25T09:54:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.034+0000 7fc4a6ffd640 1 -- 172.21.15.139:0/270980898 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4ac005d10 con 0x7fc4b00719d0 2024-04-25T09:54:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.035+0000 7fc4a6ffd640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc49406b4d0 0x7fc49406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.035+0000 7fc4b5af7640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc49406b4d0 0x7fc49406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.035+0000 7fc4a6ffd640 1 -- 172.21.15.139:0/270980898 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc4ac08b070 con 0x7fc4b00719d0 2024-04-25T09:54:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.035+0000 7fc4b5af7640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc49406b4d0 0x7fc49406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0144110 tx=0x7fc4a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.039+0000 7fc4a6ffd640 1 -- 172.21.15.139:0/270980898 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4ac022070 con 0x7fc4b00719d0 2024-04-25T09:54:05.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.223+0000 7fc4b855c640 1 -- 172.21.15.139:0/270980898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc4b00701f0 con 0x7fc4b00719d0 2024-04-25T09:54:05.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.223+0000 7fc4a6ffd640 1 -- 172.21.15.139:0/270980898 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc4b01e6440 con 0x7fc4b00719d0 2024-04-25T09:54:05.225 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T09:54:05.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.225+0000 7fc4b855c640 1 -- 172.21.15.139:0/270980898 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc49406b4d0 msgr2=0x7fc49406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.225+0000 7fc4b855c640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc49406b4d0 0x7fc49406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0144110 tx=0x7fc4a0009040 comp rx=0 tx=0).stop 2024-04-25T09:54:05.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.225+0000 7fc4b855c640 1 -- 172.21.15.139:0/270980898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b00719d0 msgr2=0x7fc4b01e5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.225+0000 7fc4b855c640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b00719d0 0x7fc4b01e5f00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00efc0 tx=0x7fc4ac00c490 comp rx=0 tx=0).stop 2024-04-25T09:54:05.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.225+0000 7fc4b855c640 1 -- 172.21.15.139:0/270980898 shutdown_connections 2024-04-25T09:54:05.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.225+0000 7fc4b855c640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc49406b4d0 0x7fc49406d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.225+0000 7fc4b855c640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4b0145250 0x7fc4b01dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.225+0000 7fc4b855c640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4b00b5690 0x7fc4b00b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.225+0000 7fc4b855c640 1 --2- 172.21.15.139:0/270980898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b00719d0 0x7fc4b01e5f00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.225+0000 7fc4b855c640 1 -- 172.21.15.139:0/270980898 >> 172.21.15.139:0/270980898 conn(0x7fc4b006f460 msgr2=0x7fc4b00b6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:05.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.225+0000 7fc4b855c640 1 -- 172.21.15.139:0/270980898 shutdown_connections 2024-04-25T09:54:05.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.225+0000 7fc4b855c640 1 -- 172.21.15.139:0/270980898 wait complete. 2024-04-25T09:54:05.238 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T09:54:05.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:05.239 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T09:54:05.263 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:54:05.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:05.263 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:54:05.319 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:54:05.319 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:05.338 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:54:05.338 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:05.398 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:54:05.398 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:05.456 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:05.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.552+0000 7fc16a910640 1 Processor -- start 2024-04-25T09:54:05.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.552+0000 7fc16a910640 1 -- start start 2024-04-25T09:54:05.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc16a910640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc1640cf900 0x7fc1640d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc16a910640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc164099a50 0x7fc164099e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc16a910640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1640cfe70 con 0x7fc16415efb0 2024-04-25T09:54:05.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc16a910640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1640d0a80 con 0x7fc1640cf900 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc16a910640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16409a3c0 con 0x7fc164099a50 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc1637fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc1640cf900 0x7fc1640d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc1637fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc1640cf900 0x7fc1640d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47580/0 (socket says 172.21.15.139:47580) 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc1637fe640 1 -- 172.21.15.139:0/3175587580 learned_addr learned my addr 172.21.15.139:0/3175587580 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc168ead640 1 --2- 172.21.15.139:0/3175587580 >> v2:172.21.15.150:3300/0 conn(0x7fc164099a50 0x7fc164099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc1637fe640 1 -- 172.21.15.139:0/3175587580 >> v2:172.21.15.150:3300/0 conn(0x7fc164099a50 msgr2=0x7fc164099e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc1637fe640 1 --2- 172.21.15.139:0/3175587580 >> v2:172.21.15.150:3300/0 conn(0x7fc164099a50 0x7fc164099e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc1637fe640 1 -- 172.21.15.139:0/3175587580 >> v1:172.21.15.150:6789/0 conn(0x7fc16415efb0 legacy=0x7fc16406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.553+0000 7fc1637fe640 1 -- 172.21.15.139:0/3175587580 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1640d0150 con 0x7fc1640cf900 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.554+0000 7fc168ead640 1 --2- 172.21.15.139:0/3175587580 >> v2:172.21.15.150:3300/0 conn(0x7fc164099a50 0x7fc164099e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.554+0000 7fc1637fe640 1 --2- 172.21.15.139:0/3175587580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1640cf900 0x7fc1640d0f10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc164072a90 tx=0x7fc15802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d971616c5fa41e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.554+0000 7fc162ffd640 1 -- 172.21.15.139:0/3175587580 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158037070 con 0x7fc1640cf900 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.554+0000 7fc162ffd640 1 -- 172.21.15.139:0/3175587580 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc158032440 con 0x7fc1640cf900 2024-04-25T09:54:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.554+0000 7fc162ffd640 1 -- 172.21.15.139:0/3175587580 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158032810 con 0x7fc1640cf900 2024-04-25T09:54:05.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.555+0000 7fc16a910640 1 -- 172.21.15.139:0/3175587580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1640cf900 msgr2=0x7fc1640d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.555+0000 7fc16a910640 1 --2- 172.21.15.139:0/3175587580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1640cf900 0x7fc1640d0f10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc164072a90 tx=0x7fc15802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.555+0000 7fc16a910640 1 -- 172.21.15.139:0/3175587580 shutdown_connections 2024-04-25T09:54:05.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.555+0000 7fc16a910640 1 --2- 172.21.15.139:0/3175587580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1640cf900 0x7fc1640d0f10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.555+0000 7fc16a910640 1 --2- 172.21.15.139:0/3175587580 >> v2:172.21.15.150:3300/0 conn(0x7fc164099a50 0x7fc164099e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.555+0000 7fc16a910640 1 -- 172.21.15.139:0/3175587580 >> 172.21.15.139:0/3175587580 conn(0x7fc1640d1790 msgr2=0x7fc1640d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:05.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.555+0000 7fc16a910640 1 -- 172.21.15.139:0/3175587580 shutdown_connections 2024-04-25T09:54:05.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.555+0000 7fc16a910640 1 -- 172.21.15.139:0/3175587580 wait complete. 2024-04-25T09:54:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.556+0000 7fc16a910640 1 Processor -- start 2024-04-25T09:54:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.556+0000 7fc16a910640 1 -- start start 2024-04-25T09:54:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc16a910640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc164099a50 0x7fc1640b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc16a910640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1640cf900 0x7fc1641e76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc16a910640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc16415efb0 0x7fc1641e9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc16a910640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16406cf50 con 0x7fc1640cf900 2024-04-25T09:54:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc16a910640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1640d0710 con 0x7fc16415efb0 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc16a910640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1640cbde0 con 0x7fc164099a50 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc1637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1640cf900 0x7fc1641e76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc1637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1640cf900 0x7fc1641e76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47590/0 (socket says 172.21.15.139:47590) 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc1637fe640 1 -- 172.21.15.139:0/23222732 learned_addr learned my addr 172.21.15.139:0/23222732 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc168ead640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc16415efb0 0x7fc1641e9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc163fff640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc164099a50 0x7fc1640b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc1637fe640 1 -- 172.21.15.139:0/23222732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc164099a50 msgr2=0x7fc1640b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc1637fe640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc164099a50 0x7fc1640b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc1637fe640 1 -- 172.21.15.139:0/23222732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc16415efb0 msgr2=0x7fc1641e9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc1637fe640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc16415efb0 0x7fc1641e9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc1637fe640 1 -- 172.21.15.139:0/23222732 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1641e7e30 con 0x7fc1640cf900 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.557+0000 7fc163fff640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc164099a50 0x7fc1640b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.558+0000 7fc1637fe640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1640cf900 0x7fc1641e76e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc15802a1e0 tx=0x7fc1580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.558+0000 7fc160ff9640 1 -- 172.21.15.139:0/23222732 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158037040 con 0x7fc1640cf900 2024-04-25T09:54:05.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.558+0000 7fc160ff9640 1 -- 172.21.15.139:0/23222732 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc158042070 con 0x7fc1640cf900 2024-04-25T09:54:05.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.558+0000 7fc16a910640 1 -- 172.21.15.139:0/23222732 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1641e8110 con 0x7fc1640cf900 2024-04-25T09:54:05.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.558+0000 7fc160ff9640 1 -- 172.21.15.139:0/23222732 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158030bb0 con 0x7fc1640cf900 2024-04-25T09:54:05.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.558+0000 7fc16a910640 1 -- 172.21.15.139:0/23222732 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1641e86d0 con 0x7fc1640cf900 2024-04-25T09:54:05.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.559+0000 7fc16a910640 1 -- 172.21.15.139:0/23222732 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc164074520 con 0x7fc1640cf900 2024-04-25T09:54:05.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.559+0000 7fc160ff9640 1 -- 172.21.15.139:0/23222732 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc158047460 con 0x7fc1640cf900 2024-04-25T09:54:05.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.559+0000 7fc160ff9640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc15006b500 0x7fc15006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.560+0000 7fc160ff9640 1 -- 172.21.15.139:0/23222732 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc1580ab180 con 0x7fc1640cf900 2024-04-25T09:54:05.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.560+0000 7fc163fff640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc15006b500 0x7fc15006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.560+0000 7fc163fff640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc15006b500 0x7fc15006d9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc154006fd0 tx=0x7fc154008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:05.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.564+0000 7fc160ff9640 1 -- 172.21.15.139:0/23222732 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc158079c00 con 0x7fc1640cf900 2024-04-25T09:54:05.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.734+0000 7fc16a910640 1 -- 172.21.15.139:0/23222732 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc164070360 con 0x7fc1640cf900 2024-04-25T09:54:05.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.735+0000 7fc160ff9640 1 -- 172.21.15.139:0/23222732 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18951 (secure 0 0 0) 0x7fc158079500 con 0x7fc1640cf900 2024-04-25T09:54:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:05.738 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":26,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:56.839903+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:05.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.737+0000 7fc16a910640 1 -- 172.21.15.139:0/23222732 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc15006b500 msgr2=0x7fc15006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.737+0000 7fc16a910640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc15006b500 0x7fc15006d9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc154006fd0 tx=0x7fc154008040 comp rx=0 tx=0).stop 2024-04-25T09:54:05.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.737+0000 7fc16a910640 1 -- 172.21.15.139:0/23222732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1640cf900 msgr2=0x7fc1641e76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.737+0000 7fc16a910640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1640cf900 0x7fc1641e76e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc15802a1e0 tx=0x7fc1580046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.737+0000 7fc16a910640 1 -- 172.21.15.139:0/23222732 shutdown_connections 2024-04-25T09:54:05.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.737+0000 7fc16a910640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1640cf900 0x7fc1641e76e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.737+0000 7fc16a910640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc15006b500 0x7fc15006d9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.737+0000 7fc16a910640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc16415efb0 0x7fc1641e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.737+0000 7fc16a910640 1 --2- 172.21.15.139:0/23222732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc164099a50 0x7fc1640b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.737+0000 7fc16a910640 1 -- 172.21.15.139:0/23222732 >> 172.21.15.139:0/23222732 conn(0x7fc1640d1790 msgr2=0x7fc1640737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:05.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.737+0000 7fc16a910640 1 -- 172.21.15.139:0/23222732 shutdown_connections 2024-04-25T09:54:05.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.737+0000 7fc16a910640 1 -- 172.21.15.139:0/23222732 wait complete. 2024-04-25T09:54:05.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:54:05.751 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:05.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.848+0000 7f9c440fc640 1 Processor -- start 2024-04-25T09:54:05.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.848+0000 7f9c440fc640 1 -- start start 2024-04-25T09:54:05.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.849+0000 7f9c440fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3c15b880 0x7f9c3c06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.849+0000 7f9c440fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c3c072db0 0x7f9c3c0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.849+0000 7f9c440fc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c15bc60 con 0x7f9c3c15f3a0 2024-04-25T09:54:05.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.849+0000 7f9c440fc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c070cd0 con 0x7f9c3c072db0 2024-04-25T09:54:05.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.849+0000 7f9c440fc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0cffa0 con 0x7f9c3c15b880 2024-04-25T09:54:05.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.849+0000 7f9c42699640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c3c072db0 0x7f9c3c0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.850+0000 7f9c42699640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c3c072db0 0x7f9c3c0cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47596/0 (socket says 172.21.15.139:47596) 2024-04-25T09:54:05.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.850+0000 7f9c42699640 1 -- 172.21.15.139:0/4076059345 learned_addr learned my addr 172.21.15.139:0/4076059345 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.850+0000 7f9c41e98640 1 --2- 172.21.15.139:0/4076059345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3c15b880 0x7f9c3c06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.850+0000 7f9c42699640 1 -- 172.21.15.139:0/4076059345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3c15b880 msgr2=0x7f9c3c06fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.850+0000 7f9c42699640 1 --2- 172.21.15.139:0/4076059345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3c15b880 0x7f9c3c06fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.850+0000 7f9c42699640 1 -- 172.21.15.139:0/4076059345 >> v1:172.21.15.139:6789/0 conn(0x7f9c3c15f3a0 legacy=0x7f9c3c0726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.850+0000 7f9c42699640 1 -- 172.21.15.139:0/4076059345 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0cf6b0 con 0x7f9c3c072db0 2024-04-25T09:54:05.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.850+0000 7f9c42699640 1 --2- 172.21.15.139:0/4076059345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3c072db0 0x7f9c3c0cd470 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009870 tx=0x7f9c3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c1654e98bfcea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:05.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.851+0000 7f9c40e96640 1 -- 172.21.15.139:0/4076059345 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34037070 con 0x7f9c3c072db0 2024-04-25T09:54:05.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.851+0000 7f9c40e96640 1 -- 172.21.15.139:0/4076059345 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c34032440 con 0x7f9c3c072db0 2024-04-25T09:54:05.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.851+0000 7f9c40e96640 1 -- 172.21.15.139:0/4076059345 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34032810 con 0x7f9c3c072db0 2024-04-25T09:54:05.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.851+0000 7f9c440fc640 1 -- 172.21.15.139:0/4076059345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3c072db0 msgr2=0x7f9c3c0cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.851+0000 7f9c440fc640 1 --2- 172.21.15.139:0/4076059345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3c072db0 0x7f9c3c0cd470 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009870 tx=0x7f9c3402a270 comp rx=0 tx=0).stop 2024-04-25T09:54:05.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.851+0000 7f9c440fc640 1 -- 172.21.15.139:0/4076059345 shutdown_connections 2024-04-25T09:54:05.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.851+0000 7f9c440fc640 1 --2- 172.21.15.139:0/4076059345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3c072db0 0x7f9c3c0cd470 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.851+0000 7f9c440fc640 1 --2- 172.21.15.139:0/4076059345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3c15b880 0x7f9c3c06fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.851+0000 7f9c440fc640 1 -- 172.21.15.139:0/4076059345 >> 172.21.15.139:0/4076059345 conn(0x7f9c3c0b23f0 msgr2=0x7f9c3c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:05.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.851+0000 7f9c440fc640 1 -- 172.21.15.139:0/4076059345 shutdown_connections 2024-04-25T09:54:05.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.851+0000 7f9c440fc640 1 -- 172.21.15.139:0/4076059345 wait complete. 2024-04-25T09:54:05.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.852+0000 7f9c440fc640 1 Processor -- start 2024-04-25T09:54:05.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.852+0000 7f9c440fc640 1 -- start start 2024-04-25T09:54:05.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c440fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3c072db0 0x7f9c3c1f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c440fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3c15b880 0x7f9c3c1e73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c440fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c3c15f3a0 0x7f9c3c1e7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c440fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0cfc30 con 0x7f9c3c15b880 2024-04-25T09:54:05.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c440fc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0cf950 con 0x7f9c3c15f3a0 2024-04-25T09:54:05.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c440fc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c070610 con 0x7f9c3c072db0 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c41697640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3c15b880 0x7f9c3c1e73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c41697640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3c15b880 0x7f9c3c1e73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47602/0 (socket says 172.21.15.139:47602) 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c41697640 1 -- 172.21.15.139:0/2661656025 learned_addr learned my addr 172.21.15.139:0/2661656025 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c41e98640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3c072db0 0x7f9c3c1f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c42699640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c3c15f3a0 0x7f9c3c1e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c41697640 1 -- 172.21.15.139:0/2661656025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3c072db0 msgr2=0x7f9c3c1f18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c41697640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3c072db0 0x7f9c3c1f18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c41697640 1 -- 172.21.15.139:0/2661656025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c3c15f3a0 msgr2=0x7f9c3c1e7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c41697640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c3c15f3a0 0x7f9c3c1e7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c41697640 1 -- 172.21.15.139:0/2661656025 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0c5800 con 0x7f9c3c15b880 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.853+0000 7f9c41e98640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3c072db0 0x7f9c3c1f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.854+0000 7f9c41697640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3c15b880 0x7f9c3c1e73d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00c400 tx=0x7f9c2c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.854+0000 7f9c327fc640 1 -- 172.21.15.139:0/2661656025 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c017070 con 0x7f9c3c15b880 2024-04-25T09:54:05.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.854+0000 7f9c327fc640 1 -- 172.21.15.139:0/2661656025 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c0052d0 con 0x7f9c3c15b880 2024-04-25T09:54:05.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.854+0000 7f9c440fc640 1 -- 172.21.15.139:0/2661656025 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c0c5ae0 con 0x7f9c3c15b880 2024-04-25T09:54:05.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.854+0000 7f9c327fc640 1 -- 172.21.15.139:0/2661656025 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c003830 con 0x7f9c3c15b880 2024-04-25T09:54:05.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.854+0000 7f9c440fc640 1 -- 172.21.15.139:0/2661656025 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c159660 con 0x7f9c3c15b880 2024-04-25T09:54:05.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.855+0000 7f9c440fc640 1 -- 172.21.15.139:0/2661656025 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c074520 con 0x7f9c3c15b880 2024-04-25T09:54:05.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.855+0000 7f9c327fc640 1 -- 172.21.15.139:0/2661656025 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c2c01a070 con 0x7f9c3c15b880 2024-04-25T09:54:05.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.856+0000 7f9c327fc640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c2006b4d0 0x7f9c2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:05.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.856+0000 7f9c41e98640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c2006b4d0 0x7f9c2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:05.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.856+0000 7f9c327fc640 1 -- 172.21.15.139:0/2661656025 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9c2c089960 con 0x7f9c3c15b880 2024-04-25T09:54:05.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.856+0000 7f9c41e98640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c2006b4d0 0x7f9c2006d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c1e7fe0 tx=0x7f9c38007650 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:05.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:05.860+0000 7f9c327fc640 1 -- 172.21.15.139:0/2661656025 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c2c0583e0 con 0x7f9c3c15b880 2024-04-25T09:54:06.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.032+0000 7f9c440fc640 1 -- 172.21.15.139:0/2661656025 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c3c072480 con 0x7f9c3c15b880 2024-04-25T09:54:06.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.033+0000 7f9c327fc640 1 -- 172.21.15.139:0/2661656025 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c2c057ce0 con 0x7f9c3c15b880 2024-04-25T09:54:06.035 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:54:06.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.035+0000 7f9c440fc640 1 -- 172.21.15.139:0/2661656025 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c2006b4d0 msgr2=0x7f9c2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.035+0000 7f9c440fc640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c2006b4d0 0x7f9c2006d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c1e7fe0 tx=0x7f9c38007650 comp rx=0 tx=0).stop 2024-04-25T09:54:06.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.035+0000 7f9c440fc640 1 -- 172.21.15.139:0/2661656025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3c15b880 msgr2=0x7f9c3c1e73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.035+0000 7f9c440fc640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3c15b880 0x7f9c3c1e73d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00c400 tx=0x7f9c2c00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.035+0000 7f9c440fc640 1 -- 172.21.15.139:0/2661656025 shutdown_connections 2024-04-25T09:54:06.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.035+0000 7f9c440fc640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3c15b880 0x7f9c3c1e73d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.035+0000 7f9c440fc640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c2006b4d0 0x7f9c2006d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.035+0000 7f9c440fc640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c3c15f3a0 0x7f9c3c1e7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.035+0000 7f9c440fc640 1 --2- 172.21.15.139:0/2661656025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3c072db0 0x7f9c3c1f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.035+0000 7f9c440fc640 1 -- 172.21.15.139:0/2661656025 >> 172.21.15.139:0/2661656025 conn(0x7f9c3c0b23f0 msgr2=0x7f9c3c15de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:06.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.035+0000 7f9c440fc640 1 -- 172.21.15.139:0/2661656025 shutdown_connections 2024-04-25T09:54:06.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.035+0000 7f9c440fc640 1 -- 172.21.15.139:0/2661656025 wait complete. 2024-04-25T09:54:06.048 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:54:06.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.145+0000 7f66a3c9a640 1 Processor -- start 2024-04-25T09:54:06.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.145+0000 7f66a3c9a640 1 -- start start 2024-04-25T09:54:06.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.146+0000 7f66a3c9a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c0719d0 0x7f669c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.146+0000 7f66a3c9a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669c0b1100 con 0x7f669c145250 2024-04-25T09:54:06.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.146+0000 7f66a3c9a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669c0ce2c0 con 0x7f669c0712f0 2024-04-25T09:54:06.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.146+0000 7f66a3c9a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0ceed0 con 0x7f669c0719d0 2024-04-25T09:54:06.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.146+0000 7f66a1a36640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f669c145250 0x7f669c070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54668/0 (socket says 172.21.15.139:54668) 2024-04-25T09:54:06.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.146+0000 7f66a1a36640 1 -- 172.21.15.139:0/2562827231 learned_addr learned my addr 172.21.15.139:0/2562827231 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:06.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547208285 0 0) 0x7f669c0b1100 con 0x7f669c145250 2024-04-25T09:54:06.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6688003520 con 0x7f669c145250 2024-04-25T09:54:06.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a2237640 1 --2- 172.21.15.139:0/2562827231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c0719d0 0x7f669c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2456313309 0 0) 0x7f6688003520 con 0x7f669c145250 2024-04-25T09:54:06.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c0719d0 msgr2=0x7f669c0b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:06.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 --2- 172.21.15.139:0/2562827231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c0719d0 0x7f669c0b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 >> v1:172.21.15.150:6789/0 conn(0x7f669c0712f0 legacy=0x7f669c0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c0ce5a0 con 0x7f669c145250 2024-04-25T09:54:06.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f66980019f0 con 0x7f669c145250 2024-04-25T09:54:06.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 >> v1:172.21.15.139:6789/0 conn(0x7f669c145250 legacy=0x7f669c070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 --2- 172.21.15.139:0/2562827231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66880039e0 0x7f6688003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 --2- 172.21.15.139:0/2562827231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c0719d0 0x7f6688004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 --2- 172.21.15.139:0/2562827231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6688004d40 0x7f66880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0ceed0 con 0x7f6688004d40 2024-04-25T09:54:06.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668800baf0 con 0x7f66880039e0 2024-04-25T09:54:06.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668800bcf0 con 0x7f669c0719d0 2024-04-25T09:54:06.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a2237640 1 --2- 172.21.15.139:0/2562827231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6688004d40 0x7f66880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a1a36640 1 --2- 172.21.15.139:0/2562827231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66880039e0 0x7f6688003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a1235640 1 --2- 172.21.15.139:0/2562827231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c0719d0 0x7f6688004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a2237640 1 -- 172.21.15.139:0/2562827231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c0719d0 msgr2=0x7f6688004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:06.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a2237640 1 --2- 172.21.15.139:0/2562827231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c0719d0 0x7f6688004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a2237640 1 -- 172.21.15.139:0/2562827231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66880039e0 msgr2=0x7f6688003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:06.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a2237640 1 --2- 172.21.15.139:0/2562827231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66880039e0 0x7f6688003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.147+0000 7f66a2237640 1 -- 172.21.15.139:0/2562827231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6688003520 con 0x7f6688004d40 2024-04-25T09:54:06.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.148+0000 7f66a2237640 1 --2- 172.21.15.139:0/2562827231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6688004d40 0x7f66880051c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6694009ac0 tx=0x7f6688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c35d744b4b8e789 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:06.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.148+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694036070 con 0x7f6688004d40 2024-04-25T09:54:06.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.148+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6694031400 con 0x7f6688004d40 2024-04-25T09:54:06.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.148+0000 7f66a0a34640 1 -- 172.21.15.139:0/2562827231 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694009690 con 0x7f6688004d40 2024-04-25T09:54:06.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.149+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2562827231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6688004d40 msgr2=0x7f66880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.149+0000 7f66a3c9a640 1 --2- 172.21.15.139:0/2562827231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6688004d40 0x7f66880051c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6694009ac0 tx=0x7f6688005830 comp rx=0 tx=0).stop 2024-04-25T09:54:06.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.149+0000 7f66a2237640 1 -- 172.21.15.139:0/2562827231 reap_dead start 2024-04-25T09:54:06.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.149+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2562827231 shutdown_connections 2024-04-25T09:54:06.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.149+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2562827231 >> 172.21.15.139:0/2562827231 conn(0x7f669c06f460 msgr2=0x7f669c06f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:06.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.149+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2562827231 shutdown_connections 2024-04-25T09:54:06.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.149+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2562827231 wait complete. 2024-04-25T09:54:06.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.150+0000 7f66a3c9a640 1 Processor -- start 2024-04-25T09:54:06.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.150+0000 7f66a3c9a640 1 -- start start 2024-04-25T09:54:06.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a3c9a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c1f34b0 0x7f669c1e56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a3c9a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f669c1f48e0 0x7f669c1e5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a3c9a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f669c1f3890 0x7f669c1f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a3c9a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c145630 con 0x7f669c1f48e0 2024-04-25T09:54:06.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a3c9a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0ceb60 con 0x7f669c1f3890 2024-04-25T09:54:06.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a3c9a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0d11a0 con 0x7f669c1f34b0 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a1235640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f669c1f48e0 0x7f669c1e5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a1235640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f669c1f48e0 0x7f669c1e5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47616/0 (socket says 172.21.15.139:47616) 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a1235640 1 -- 172.21.15.139:0/2263247258 learned_addr learned my addr 172.21.15.139:0/2263247258 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a1a36640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c1f34b0 0x7f669c1e56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a2237640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f669c1f3890 0x7f669c1f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a1235640 1 -- 172.21.15.139:0/2263247258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c1f34b0 msgr2=0x7f669c1e56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a1235640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c1f34b0 0x7f669c1e56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a1235640 1 -- 172.21.15.139:0/2263247258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f669c1f3890 msgr2=0x7f669c1f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a1235640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f669c1f3890 0x7f669c1f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a1235640 1 -- 172.21.15.139:0/2263247258 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c1f7ea0 con 0x7f669c1f48e0 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a2237640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f669c1f3890 0x7f669c1f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a1235640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f669c1f48e0 0x7f669c1e5be0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f668c00f820 tx=0x7f668c00fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:06.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66927fc640 1 -- 172.21.15.139:0/2263247258 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c0050d0 con 0x7f669c1f48e0 2024-04-25T09:54:06.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66927fc640 1 -- 172.21.15.139:0/2263247258 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c018070 con 0x7f669c1f48e0 2024-04-25T09:54:06.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2263247258 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669c1f3e00 con 0x7f669c1f48e0 2024-04-25T09:54:06.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66927fc640 1 -- 172.21.15.139:0/2263247258 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c00d730 con 0x7f669c1f48e0 2024-04-25T09:54:06.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.151+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2263247258 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669c1f43c0 con 0x7f669c1f48e0 2024-04-25T09:54:06.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.152+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2263247258 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669c074520 con 0x7f669c1f48e0 2024-04-25T09:54:06.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.153+0000 7f66927fc640 1 -- 172.21.15.139:0/2263247258 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f668c00d8d0 con 0x7f669c1f48e0 2024-04-25T09:54:06.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.153+0000 7f66927fc640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f668006b4d0 0x7f668006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.153+0000 7f66927fc640 1 -- 172.21.15.139:0/2263247258 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f668c08a410 con 0x7f669c1f48e0 2024-04-25T09:54:06.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.153+0000 7f66a1a36640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f668006b4d0 0x7f668006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.154+0000 7f66a1a36640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f668006b4d0 0x7f668006d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6698004bc0 tx=0x7f6698006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:06.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.157+0000 7f66927fc640 1 -- 172.21.15.139:0/2263247258 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f668c058e90 con 0x7f669c1f48e0 2024-04-25T09:54:06.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.329+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2263247258 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f669c0701f0 con 0x7f669c1f48e0 2024-04-25T09:54:06.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.330+0000 7f66927fc640 1 -- 172.21.15.139:0/2263247258 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f668c058790 con 0x7f669c1f48e0 2024-04-25T09:54:06.332 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:54:06.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.332+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2263247258 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f668006b4d0 msgr2=0x7f668006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.332+0000 7f66a3c9a640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f668006b4d0 0x7f668006d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6698004bc0 tx=0x7f6698006040 comp rx=0 tx=0).stop 2024-04-25T09:54:06.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.332+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2263247258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f669c1f48e0 msgr2=0x7f669c1e5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.332+0000 7f66a3c9a640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f669c1f48e0 0x7f669c1e5be0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f668c00f820 tx=0x7f668c00fc20 comp rx=0 tx=0).stop 2024-04-25T09:54:06.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.332+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2263247258 shutdown_connections 2024-04-25T09:54:06.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.332+0000 7f66a3c9a640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f669c1f48e0 0x7f669c1e5be0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.332+0000 7f66a3c9a640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f668006b4d0 0x7f668006d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.332+0000 7f66a3c9a640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f669c1f3890 0x7f669c1f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.332+0000 7f66a3c9a640 1 --2- 172.21.15.139:0/2263247258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669c1f34b0 0x7f669c1e56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.332+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2263247258 >> 172.21.15.139:0/2263247258 conn(0x7f669c06f460 msgr2=0x7f669c0b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:06.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.332+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2263247258 shutdown_connections 2024-04-25T09:54:06.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.332+0000 7f66a3c9a640 1 -- 172.21.15.139:0/2263247258 wait complete. 2024-04-25T09:54:06.345 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:54:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.443+0000 7f9cb0530640 1 Processor -- start 2024-04-25T09:54:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.443+0000 7f9cb0530640 1 -- start start 2024-04-25T09:54:06.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9cb0530640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9ca8145230 0x7f9ca815be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9cb0530640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ca80718d0 0x7f9ca815da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9cb0530640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca815c960 con 0x7f9ca815c3d0 2024-04-25T09:54:06.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9cb0530640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca815cc40 con 0x7f9ca8145230 2024-04-25T09:54:06.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9cb0530640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815d230 con 0x7f9ca80718d0 2024-04-25T09:54:06.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9caeacd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ca80718d0 0x7f9ca815da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9cae2cc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9ca8145230 0x7f9ca815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9caeacd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ca80718d0 0x7f9ca815da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:51170/0 (socket says 172.21.15.139:51170) 2024-04-25T09:54:06.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9cadacb640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9ca815c3d0 0x7f9ca80711c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:57958/0 (socket says 172.21.15.139:57958) 2024-04-25T09:54:06.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9caeacd640 1 -- 172.21.15.139:0/2522778971 learned_addr learned my addr 172.21.15.139:0/2522778971 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:06.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9cae2cc640 1 -- 172.21.15.139:0/2522778971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ca80718d0 msgr2=0x7f9ca815da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:06.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9cae2cc640 1 --2- 172.21.15.139:0/2522778971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ca80718d0 0x7f9ca815da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9cae2cc640 1 -- 172.21.15.139:0/2522778971 >> v1:172.21.15.150:6789/0 conn(0x7f9ca815c3d0 legacy=0x7f9ca80711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.444+0000 7f9cae2cc640 1 -- 172.21.15.139:0/2522778971 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca815d510 con 0x7f9ca8145230 2024-04-25T09:54:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.445+0000 7f9cae2cc640 1 --2- 172.21.15.139:0/2522778971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ca8145230 0x7f9ca815be90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40097a0 tx=0x7f9ca402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4a0974a63daa819 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.445+0000 7f9cad2ca640 1 -- 172.21.15.139:0/2522778971 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4037070 con 0x7f9ca8145230 2024-04-25T09:54:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.445+0000 7f9cad2ca640 1 -- 172.21.15.139:0/2522778971 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca4032440 con 0x7f9ca8145230 2024-04-25T09:54:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.445+0000 7f9cad2ca640 1 -- 172.21.15.139:0/2522778971 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4032810 con 0x7f9ca8145230 2024-04-25T09:54:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.446+0000 7f9cb0530640 1 -- 172.21.15.139:0/2522778971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ca8145230 msgr2=0x7f9ca815be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.446+0000 7f9cb0530640 1 --2- 172.21.15.139:0/2522778971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ca8145230 0x7f9ca815be90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40097a0 tx=0x7f9ca402a180 comp rx=0 tx=0).stop 2024-04-25T09:54:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.446+0000 7f9cb0530640 1 -- 172.21.15.139:0/2522778971 shutdown_connections 2024-04-25T09:54:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.446+0000 7f9cb0530640 1 --2- 172.21.15.139:0/2522778971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ca80718d0 0x7f9ca815da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.446+0000 7f9cb0530640 1 --2- 172.21.15.139:0/2522778971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ca8145230 0x7f9ca815be90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.446+0000 7f9cb0530640 1 -- 172.21.15.139:0/2522778971 >> 172.21.15.139:0/2522778971 conn(0x7f9ca8067000 msgr2=0x7f9ca806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.446+0000 7f9cb0530640 1 -- 172.21.15.139:0/2522778971 shutdown_connections 2024-04-25T09:54:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.446+0000 7f9cb0530640 1 -- 172.21.15.139:0/2522778971 wait complete. 2024-04-25T09:54:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.447+0000 7f9cb0530640 1 Processor -- start 2024-04-25T09:54:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.447+0000 7f9cb0530640 1 -- start start 2024-04-25T09:54:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cb0530640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ca80718d0 0x7f9ca80c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cb0530640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ca8145230 0x7f9ca80b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cb0530640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ca815c3d0 0x7f9ca813e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cb0530640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815e0f0 con 0x7f9ca8145230 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cb0530640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815df70 con 0x7f9ca80718d0 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cb0530640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815e270 con 0x7f9ca815c3d0 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cadacb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ca8145230 0x7f9ca80b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cadacb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ca8145230 0x7f9ca80b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49386/0 (socket says 172.21.15.139:49386) 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cadacb640 1 -- 172.21.15.139:0/1975839865 learned_addr learned my addr 172.21.15.139:0/1975839865 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cae2cc640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ca80718d0 0x7f9ca80c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9caeacd640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ca815c3d0 0x7f9ca813e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cadacb640 1 -- 172.21.15.139:0/1975839865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ca815c3d0 msgr2=0x7f9ca813e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cadacb640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ca815c3d0 0x7f9ca813e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cadacb640 1 -- 172.21.15.139:0/1975839865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ca80718d0 msgr2=0x7f9ca80c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cadacb640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ca80718d0 0x7f9ca80c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cadacb640 1 -- 172.21.15.139:0/1975839865 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca80b38c0 con 0x7f9ca8145230 2024-04-25T09:54:06.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.448+0000 7f9cadacb640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ca8145230 0x7f9ca80b3380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c3c0 tx=0x7f9c9800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:06.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.449+0000 7f9c9effd640 1 -- 172.21.15.139:0/1975839865 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c980074e0 con 0x7f9ca8145230 2024-04-25T09:54:06.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.449+0000 7f9c9effd640 1 -- 172.21.15.139:0/1975839865 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c98007680 con 0x7f9ca8145230 2024-04-25T09:54:06.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.449+0000 7f9cb0530640 1 -- 172.21.15.139:0/1975839865 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca80b3b60 con 0x7f9ca8145230 2024-04-25T09:54:06.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.449+0000 7f9c9effd640 1 -- 172.21.15.139:0/1975839865 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98004a00 con 0x7f9ca8145230 2024-04-25T09:54:06.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.449+0000 7f9cb0530640 1 -- 172.21.15.139:0/1975839865 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca80b4120 con 0x7f9ca8145230 2024-04-25T09:54:06.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.450+0000 7f9cb0530640 1 -- 172.21.15.139:0/1975839865 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca806baf0 con 0x7f9ca8145230 2024-04-25T09:54:06.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.450+0000 7f9c9effd640 1 -- 172.21.15.139:0/1975839865 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c9801a020 con 0x7f9ca8145230 2024-04-25T09:54:06.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.450+0000 7f9c9effd640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c8c06b4d0 0x7f9c8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.451+0000 7f9c9effd640 1 -- 172.21.15.139:0/1975839865 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9c98089b90 con 0x7f9ca8145230 2024-04-25T09:54:06.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.451+0000 7f9cae2cc640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c8c06b4d0 0x7f9c8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.451+0000 7f9cae2cc640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c8c06b4d0 0x7f9c8c06d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4034000 tx=0x7f9ca402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:06.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.455+0000 7f9c9effd640 1 -- 172.21.15.139:0/1975839865 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c98058050 con 0x7f9ca8145230 2024-04-25T09:54:06.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.625+0000 7f9cb0530640 1 -- 172.21.15.139:0/1975839865 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ca815cfb0 con 0x7f9ca8145230 2024-04-25T09:54:06.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.625+0000 7f9c9effd640 1 -- 172.21.15.139:0/1975839865 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ca815cfb0 con 0x7f9ca8145230 2024-04-25T09:54:06.628 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:54:06.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.627+0000 7f9cb0530640 1 -- 172.21.15.139:0/1975839865 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c8c06b4d0 msgr2=0x7f9c8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.627+0000 7f9cb0530640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c8c06b4d0 0x7f9c8c06d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4034000 tx=0x7f9ca402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:06.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.627+0000 7f9cb0530640 1 -- 172.21.15.139:0/1975839865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ca8145230 msgr2=0x7f9ca80b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.627+0000 7f9cb0530640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ca8145230 0x7f9ca80b3380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c3c0 tx=0x7f9c9800c880 comp rx=0 tx=0).stop 2024-04-25T09:54:06.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.627+0000 7f9cb0530640 1 -- 172.21.15.139:0/1975839865 shutdown_connections 2024-04-25T09:54:06.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.627+0000 7f9cb0530640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ca8145230 0x7f9ca80b3380 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.627+0000 7f9cb0530640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c8c06b4d0 0x7f9c8c06d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.627+0000 7f9cb0530640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ca815c3d0 0x7f9ca813e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.627+0000 7f9cb0530640 1 --2- 172.21.15.139:0/1975839865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ca80718d0 0x7f9ca80c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.627+0000 7f9cb0530640 1 -- 172.21.15.139:0/1975839865 >> 172.21.15.139:0/1975839865 conn(0x7f9ca8067000 msgr2=0x7f9ca8144050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:06.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.627+0000 7f9cb0530640 1 -- 172.21.15.139:0/1975839865 shutdown_connections 2024-04-25T09:54:06.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.627+0000 7f9cb0530640 1 -- 172.21.15.139:0/1975839865 wait complete. 2024-04-25T09:54:06.640 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:54:06.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.737+0000 7f2b38ecb640 1 Processor -- start 2024-04-25T09:54:06.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.737+0000 7f2b38ecb640 1 -- start start 2024-04-25T09:54:06.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.738+0000 7f2b38ecb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b34070970 0x7f2b340b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.738+0000 7f2b38ecb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b340b4600 con 0x7f2b3415a870 2024-04-25T09:54:06.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.738+0000 7f2b38ecb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b340d0d40 con 0x7f2b340b4190 2024-04-25T09:54:06.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.738+0000 7f2b38ecb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340d1020 con 0x7f2b34070970 2024-04-25T09:54:06.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.738+0000 7f2b3259c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2b3415a870 0x7f2b34070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59000/0 (socket says 172.21.15.139:59000) 2024-04-25T09:54:06.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.738+0000 7f2b3259c640 1 -- 172.21.15.139:0/2449593818 learned_addr learned my addr 172.21.15.139:0/2449593818 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:06.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b31d9b640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b34070970 0x7f2b340b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847848836 0 0) 0x7f2b340b4600 con 0x7f2b3415a870 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b1c003520 con 0x7f2b3415a870 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282817477 0 0) 0x7f2b340d0d40 con 0x7f2b340b4190 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b340b4600 con 0x7f2b340b4190 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 557732897 0 0) 0x7f2b1c003520 con 0x7f2b3415a870 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b34070970 msgr2=0x7f2b340b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b34070970 0x7f2b340b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 >> v1:172.21.15.150:6789/0 conn(0x7f2b340b4190 legacy=0x7f2b340ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340b4b40 con 0x7f2b3415a870 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f2b240019f0 con 0x7f2b3415a870 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 >> v1:172.21.15.139:6789/0 conn(0x7f2b3415a870 legacy=0x7f2b34070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b1c003b70 0x7f2b1c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b1c004800 0x7f2b1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b34070970 0x7f2b1c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340d1020 con 0x7f2b1c003b70 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c00dd90 con 0x7f2b1c004800 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c00df90 con 0x7f2b34070970 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3259c640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b1c003b70 0x7f2b1c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b31d9b640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b1c004800 0x7f2b1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b32d9d640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b34070970 0x7f2b1c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3259c640 1 -- 172.21.15.139:0/2449593818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b34070970 msgr2=0x7f2b1c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3259c640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b34070970 0x7f2b1c005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3259c640 1 -- 172.21.15.139:0/2449593818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b1c004800 msgr2=0x7f2b1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3259c640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b1c004800 0x7f2b1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b3259c640 1 -- 172.21.15.139:0/2449593818 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c003520 con 0x7f2b1c003b70 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.739+0000 7f2b31d9b640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b1c004800 0x7f2b1c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.740+0000 7f2b3259c640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b1c003b70 0x7f2b1c003f70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400a800 tx=0x7f2b1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77832935aa024e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.740+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2400a450 con 0x7f2b1c003b70 2024-04-25T09:54:06.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.740+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2402f070 con 0x7f2b1c003b70 2024-04-25T09:54:06.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.740+0000 7f2b3159a640 1 -- 172.21.15.139:0/2449593818 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2402bc70 con 0x7f2b1c003b70 2024-04-25T09:54:06.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.741+0000 7f2b38ecb640 1 -- 172.21.15.139:0/2449593818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b1c003b70 msgr2=0x7f2b1c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.741+0000 7f2b38ecb640 1 --2- 172.21.15.139:0/2449593818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b1c003b70 0x7f2b1c003f70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400a800 tx=0x7f2b1c005830 comp rx=0 tx=0).stop 2024-04-25T09:54:06.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.741+0000 7f2b32d9d640 1 -- 172.21.15.139:0/2449593818 reap_dead start 2024-04-25T09:54:06.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.741+0000 7f2b38ecb640 1 -- 172.21.15.139:0/2449593818 shutdown_connections 2024-04-25T09:54:06.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.741+0000 7f2b38ecb640 1 -- 172.21.15.139:0/2449593818 >> 172.21.15.139:0/2449593818 conn(0x7f2b340b23f0 msgr2=0x7f2b340b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:06.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.741+0000 7f2b38ecb640 1 -- 172.21.15.139:0/2449593818 shutdown_connections 2024-04-25T09:54:06.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.741+0000 7f2b38ecb640 1 -- 172.21.15.139:0/2449593818 wait complete. 2024-04-25T09:54:06.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.742+0000 7f2b38ecb640 1 Processor -- start 2024-04-25T09:54:06.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.742+0000 7f2b38ecb640 1 -- start start 2024-04-25T09:54:06.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b38ecb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b1c003b70 0x7f2b3414f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b38ecb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b340ce870 0x7f2b340cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b38ecb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b3415cfa0 0x7f2b3415d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b38ecb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340b5620 con 0x7f2b1c003b70 2024-04-25T09:54:06.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b38ecb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340b5190 con 0x7f2b340ce870 2024-04-25T09:54:06.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b38ecb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340b57a0 con 0x7f2b3415cfa0 2024-04-25T09:54:06.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b3259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b1c003b70 0x7f2b3414f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b3259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b1c003b70 0x7f2b3414f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49414/0 (socket says 172.21.15.139:49414) 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b3259c640 1 -- 172.21.15.139:0/3881822221 learned_addr learned my addr 172.21.15.139:0/3881822221 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b31d9b640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b340ce870 0x7f2b340cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b32d9d640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b3415cfa0 0x7f2b3415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b3259c640 1 -- 172.21.15.139:0/3881822221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b3415cfa0 msgr2=0x7f2b3415d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b3259c640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b3415cfa0 0x7f2b3415d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b3259c640 1 -- 172.21.15.139:0/3881822221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b340ce870 msgr2=0x7f2b340cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b3259c640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b340ce870 0x7f2b340cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b3259c640 1 -- 172.21.15.139:0/3881822221 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b34150f40 con 0x7f2b1c003b70 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b32d9d640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b3415cfa0 0x7f2b3415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b3259c640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b1c003b70 0x7f2b3414f950 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400a800 tx=0x7f2b340b4d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.743+0000 7f2b22ffd640 1 -- 172.21.15.139:0/3881822221 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2402bc00 con 0x7f2b1c003b70 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.744+0000 7f2b22ffd640 1 -- 172.21.15.139:0/3881822221 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2402bda0 con 0x7f2b1c003b70 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.744+0000 7f2b38ecb640 1 -- 172.21.15.139:0/3881822221 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b34151220 con 0x7f2b1c003b70 2024-04-25T09:54:06.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.744+0000 7f2b22ffd640 1 -- 172.21.15.139:0/3881822221 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b240346b0 con 0x7f2b1c003b70 2024-04-25T09:54:06.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.744+0000 7f2b38ecb640 1 -- 172.21.15.139:0/3881822221 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3415d960 con 0x7f2b1c003b70 2024-04-25T09:54:06.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.745+0000 7f2b38ecb640 1 -- 172.21.15.139:0/3881822221 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b34074520 con 0x7f2b1c003b70 2024-04-25T09:54:06.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.745+0000 7f2b22ffd640 1 -- 172.21.15.139:0/3881822221 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b24034850 con 0x7f2b1c003b70 2024-04-25T09:54:06.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.745+0000 7f2b22ffd640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b1406b4d0 0x7f2b1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:06.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.745+0000 7f2b22ffd640 1 -- 172.21.15.139:0/3881822221 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2b240aaba0 con 0x7f2b1c003b70 2024-04-25T09:54:06.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.745+0000 7f2b31d9b640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b1406b4d0 0x7f2b1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:06.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.746+0000 7f2b31d9b640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b1406b4d0 0x7f2b1406d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2b34073370 tx=0x7f2b28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:06.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.750+0000 7f2b22ffd640 1 -- 172.21.15.139:0/3881822221 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b24079590 con 0x7f2b1c003b70 2024-04-25T09:54:06.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.920+0000 7f2b38ecb640 1 -- 172.21.15.139:0/3881822221 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2b34074520 con 0x7f2b1c003b70 2024-04-25T09:54:06.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.921+0000 7f2b22ffd640 1 -- 172.21.15.139:0/3881822221 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2b24078e90 con 0x7f2b1c003b70 2024-04-25T09:54:06.923 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:54:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.923+0000 7f2b38ecb640 1 -- 172.21.15.139:0/3881822221 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b1406b4d0 msgr2=0x7f2b1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.923+0000 7f2b38ecb640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b1406b4d0 0x7f2b1406d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2b34073370 tx=0x7f2b28008040 comp rx=0 tx=0).stop 2024-04-25T09:54:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.923+0000 7f2b38ecb640 1 -- 172.21.15.139:0/3881822221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b1c003b70 msgr2=0x7f2b3414f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.923+0000 7f2b38ecb640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b1c003b70 0x7f2b3414f950 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400a800 tx=0x7f2b340b4d10 comp rx=0 tx=0).stop 2024-04-25T09:54:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.923+0000 7f2b38ecb640 1 -- 172.21.15.139:0/3881822221 shutdown_connections 2024-04-25T09:54:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.923+0000 7f2b38ecb640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b1406b4d0 0x7f2b1406d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.923+0000 7f2b38ecb640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b3415cfa0 0x7f2b3415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.923+0000 7f2b38ecb640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b340ce870 0x7f2b340cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.923+0000 7f2b38ecb640 1 --2- 172.21.15.139:0/3881822221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b1c003b70 0x7f2b3414f950 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.923+0000 7f2b38ecb640 1 -- 172.21.15.139:0/3881822221 >> 172.21.15.139:0/3881822221 conn(0x7f2b340b23f0 msgr2=0x7f2b340b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.923+0000 7f2b38ecb640 1 -- 172.21.15.139:0/3881822221 shutdown_connections 2024-04-25T09:54:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:06.923+0000 7f2b38ecb640 1 -- 172.21.15.139:0/3881822221 wait complete. 2024-04-25T09:54:06.936 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:54:07.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.031+0000 7f9d0ec7c640 1 Processor -- start 2024-04-25T09:54:07.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.032+0000 7f9d0ec7c640 1 -- start start 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.032+0000 7f9d0ec7c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d08145250 0x7f9d08070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.032+0000 7f9d0ec7c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9d080719d0 0x7f9d08071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.032+0000 7f9d0ec7c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d080b1100 con 0x7f9d080d11a0 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.032+0000 7f9d0ec7c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d080ceed0 con 0x7f9d080719d0 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.032+0000 7f9d0ec7c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080ce2c0 con 0x7f9d08145250 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.032+0000 7f9d07fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9d080719d0 0x7f9d08071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.033+0000 7f9d07fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9d080719d0 0x7f9d08071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49430/0 (socket says 172.21.15.139:49430) 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.033+0000 7f9d07fff640 1 -- 172.21.15.139:0/3436639631 learned_addr learned my addr 172.21.15.139:0/3436639631 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.033+0000 7f9d0ca18640 1 --2- 172.21.15.139:0/3436639631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d08145250 0x7f9d08070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.033+0000 7f9d07fff640 1 -- 172.21.15.139:0/3436639631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d08145250 msgr2=0x7f9d08070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.033+0000 7f9d07fff640 1 --2- 172.21.15.139:0/3436639631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d08145250 0x7f9d08070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.033+0000 7f9d07fff640 1 -- 172.21.15.139:0/3436639631 >> v1:172.21.15.139:6789/0 conn(0x7f9d080d11a0 legacy=0x7f9d0815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.033+0000 7f9d07fff640 1 -- 172.21.15.139:0/3436639631 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d080ce5a0 con 0x7f9d080719d0 2024-04-25T09:54:07.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.033+0000 7f9d0ca18640 1 --2- 172.21.15.139:0/3436639631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d08145250 0x7f9d08070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:07.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.033+0000 7f9d07fff640 1 --2- 172.21.15.139:0/3436639631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d080719d0 0x7f9d08071120 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009870 tx=0x7f9cf402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461309717f571dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.034+0000 7f9d077fe640 1 -- 172.21.15.139:0/3436639631 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4037070 con 0x7f9d080719d0 2024-04-25T09:54:07.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.034+0000 7f9d077fe640 1 -- 172.21.15.139:0/3436639631 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf4032440 con 0x7f9d080719d0 2024-04-25T09:54:07.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.034+0000 7f9d077fe640 1 -- 172.21.15.139:0/3436639631 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4032810 con 0x7f9d080719d0 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.034+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/3436639631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d080719d0 msgr2=0x7f9d08071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.034+0000 7f9d0ec7c640 1 --2- 172.21.15.139:0/3436639631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d080719d0 0x7f9d08071120 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009870 tx=0x7f9cf402a130 comp rx=0 tx=0).stop 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.034+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/3436639631 shutdown_connections 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.034+0000 7f9d0ec7c640 1 --2- 172.21.15.139:0/3436639631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d080719d0 0x7f9d08071120 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.034+0000 7f9d0ec7c640 1 --2- 172.21.15.139:0/3436639631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d08145250 0x7f9d08070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.034+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/3436639631 >> 172.21.15.139:0/3436639631 conn(0x7f9d0806f460 msgr2=0x7f9d0806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.034+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/3436639631 shutdown_connections 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.034+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/3436639631 wait complete. 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.035+0000 7f9d0ec7c640 1 Processor -- start 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.035+0000 7f9d0ec7c640 1 -- start start 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.036+0000 7f9d0ec7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d080719d0 0x7f9d081f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.036+0000 7f9d0ec7c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d080d11a0 0x7f9d081f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.036+0000 7f9d0ec7c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d08145250 0x7f9d081f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.036+0000 7f9d0ec7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08145630 con 0x7f9d080719d0 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.036+0000 7f9d0ec7c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080ceb60 con 0x7f9d08145250 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.036+0000 7f9d0ec7c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080b5690 con 0x7f9d080d11a0 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.036+0000 7f9d0ca18640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d080719d0 0x7f9d081f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.036+0000 7f9d0ca18640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d080719d0 0x7f9d081f18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49442/0 (socket says 172.21.15.139:49442) 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.036+0000 7f9d0ca18640 1 -- 172.21.15.139:0/2301030007 learned_addr learned my addr 172.21.15.139:0/2301030007 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d07fff640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d080d11a0 0x7f9d081f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d0d219640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d08145250 0x7f9d081f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d0ca18640 1 -- 172.21.15.139:0/2301030007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d080d11a0 msgr2=0x7f9d081f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d0ca18640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d080d11a0 0x7f9d081f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d0ca18640 1 -- 172.21.15.139:0/2301030007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d08145250 msgr2=0x7f9d081f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d0ca18640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d08145250 0x7f9d081f53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d0ca18640 1 -- 172.21.15.139:0/2301030007 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0813ce30 con 0x7f9d080719d0 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d0d219640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d08145250 0x7f9d081f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d0ca18640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d080719d0 0x7f9d081f18f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00c7f0 tx=0x7f9cfc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d057fa640 1 -- 172.21.15.139:0/2301030007 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc015070 con 0x7f9d080719d0 2024-04-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d057fa640 1 -- 172.21.15.139:0/2301030007 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc007820 con 0x7f9d080719d0 2024-04-25T09:54:07.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d057fa640 1 -- 172.21.15.139:0/2301030007 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc007bd0 con 0x7f9d080719d0 2024-04-25T09:54:07.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/2301030007 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0813d110 con 0x7f9d080719d0 2024-04-25T09:54:07.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.037+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/2301030007 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0813d6d0 con 0x7f9d080719d0 2024-04-25T09:54:07.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.038+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/2301030007 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d08074520 con 0x7f9d080719d0 2024-04-25T09:54:07.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.039+0000 7f9d057fa640 1 -- 172.21.15.139:0/2301030007 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9cfc018070 con 0x7f9d080719d0 2024-04-25T09:54:07.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.039+0000 7f9d057fa640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9cec06b4d0 0x7f9cec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.039+0000 7f9d057fa640 1 -- 172.21.15.139:0/2301030007 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9cfc08a730 con 0x7f9d080719d0 2024-04-25T09:54:07.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.039+0000 7f9d07fff640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9cec06b4d0 0x7f9cec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.039+0000 7f9d07fff640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9cec06b4d0 0x7f9cec06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4034000 tx=0x7f9cf402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.043+0000 7f9d057fa640 1 -- 172.21.15.139:0/2301030007 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9cfc0591b0 con 0x7f9d080719d0 2024-04-25T09:54:07.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.224+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/2301030007 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9d080701f0 con 0x7f9d080719d0 2024-04-25T09:54:07.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.225+0000 7f9d057fa640 1 -- 172.21.15.139:0/2301030007 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9cfc058ab0 con 0x7f9d080719d0 2024-04-25T09:54:07.227 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T09:54:07.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.226+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/2301030007 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9cec06b4d0 msgr2=0x7f9cec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.226+0000 7f9d0ec7c640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9cec06b4d0 0x7f9cec06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4034000 tx=0x7f9cf402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:07.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.227+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/2301030007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d080719d0 msgr2=0x7f9d081f18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.227+0000 7f9d0ec7c640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d080719d0 0x7f9d081f18f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00c7f0 tx=0x7f9cfc00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.227+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/2301030007 shutdown_connections 2024-04-25T09:54:07.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.227+0000 7f9d0ec7c640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9cec06b4d0 0x7f9cec06d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.227+0000 7f9d0ec7c640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d08145250 0x7f9d081f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.227+0000 7f9d0ec7c640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d080d11a0 0x7f9d081f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.227+0000 7f9d0ec7c640 1 --2- 172.21.15.139:0/2301030007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d080719d0 0x7f9d081f18f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.227+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/2301030007 >> 172.21.15.139:0/2301030007 conn(0x7f9d0806f460 msgr2=0x7f9d080b6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:07.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.227+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/2301030007 shutdown_connections 2024-04-25T09:54:07.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.227+0000 7f9d0ec7c640 1 -- 172.21.15.139:0/2301030007 wait complete. 2024-04-25T09:54:07.240 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T09:54:07.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:07.240 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T09:54:07.272 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T09:54:07.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:07.272 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:54:07.330 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:54:07.330 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:07.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.425+0000 7fb302a46640 1 Processor -- start 2024-04-25T09:54:07.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.425+0000 7fb302a46640 1 -- start start 2024-04-25T09:54:07.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb302a46640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb2fc158260 0x7fb2fc15df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb302a46640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb2fc06da20 0x7fb2fc0b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb302a46640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2fc0b3e10 con 0x7fb2fc0b3600 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb302a46640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2fc0b3a50 con 0x7fb2fc158260 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb302a46640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2fc1615c0 con 0x7fb2fc06da20 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb2fbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb2fc158260 0x7fb2fc15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb2fbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb2fc158260 0x7fb2fc15df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49456/0 (socket says 172.21.15.139:49456) 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb2fbfff640 1 -- 172.21.15.139:0/2420201778 learned_addr learned my addr 172.21.15.139:0/2420201778 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb2fb7fe640 1 --2- 172.21.15.139:0/2420201778 >> v2:172.21.15.150:3300/0 conn(0x7fb2fc06da20 0x7fb2fc0b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb2fbfff640 1 -- 172.21.15.139:0/2420201778 >> v2:172.21.15.150:3300/0 conn(0x7fb2fc06da20 msgr2=0x7fb2fc0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb2fbfff640 1 --2- 172.21.15.139:0/2420201778 >> v2:172.21.15.150:3300/0 conn(0x7fb2fc06da20 0x7fb2fc0b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb2fbfff640 1 -- 172.21.15.139:0/2420201778 >> v1:172.21.15.150:6789/0 conn(0x7fb2fc0b3600 legacy=0x7fb2fc06def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.426+0000 7fb2fbfff640 1 -- 172.21.15.139:0/2420201778 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc1618a0 con 0x7fb2fc158260 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.427+0000 7fb2fbfff640 1 --2- 172.21.15.139:0/2420201778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2fc158260 0x7fb2fc15df80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc072d10 tx=0x7fb2ec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a820e95170eadfb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.427+0000 7fb2faffd640 1 -- 172.21.15.139:0/2420201778 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec037070 con 0x7fb2fc158260 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.427+0000 7fb2faffd640 1 -- 172.21.15.139:0/2420201778 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec032440 con 0x7fb2fc158260 2024-04-25T09:54:07.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.427+0000 7fb2faffd640 1 -- 172.21.15.139:0/2420201778 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec032810 con 0x7fb2fc158260 2024-04-25T09:54:07.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.428+0000 7fb302a46640 1 -- 172.21.15.139:0/2420201778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2fc158260 msgr2=0x7fb2fc15df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.428+0000 7fb302a46640 1 --2- 172.21.15.139:0/2420201778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2fc158260 0x7fb2fc15df80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc072d10 tx=0x7fb2ec02a230 comp rx=0 tx=0).stop 2024-04-25T09:54:07.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.428+0000 7fb302a46640 1 -- 172.21.15.139:0/2420201778 shutdown_connections 2024-04-25T09:54:07.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.428+0000 7fb302a46640 1 --2- 172.21.15.139:0/2420201778 >> v2:172.21.15.150:3300/0 conn(0x7fb2fc06da20 0x7fb2fc0b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.428+0000 7fb302a46640 1 --2- 172.21.15.139:0/2420201778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2fc158260 0x7fb2fc15df80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.428+0000 7fb302a46640 1 -- 172.21.15.139:0/2420201778 >> 172.21.15.139:0/2420201778 conn(0x7fb2fc0b23d0 msgr2=0x7fb2fc0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:07.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.428+0000 7fb302a46640 1 -- 172.21.15.139:0/2420201778 shutdown_connections 2024-04-25T09:54:07.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.428+0000 7fb302a46640 1 -- 172.21.15.139:0/2420201778 wait complete. 2024-04-25T09:54:07.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.429+0000 7fb302a46640 1 Processor -- start 2024-04-25T09:54:07.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.429+0000 7fb302a46640 1 -- start start 2024-04-25T09:54:07.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb302a46640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2fc06da20 0x7fb2fc15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb302a46640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2fc0b3600 0x7fb2fc15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb302a46640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2fc158260 0x7fb2fc1604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb302a46640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc162140 con 0x7fb2fc158260 2024-04-25T09:54:07.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb302a46640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc161e60 con 0x7fb2fc06da20 2024-04-25T09:54:07.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb302a46640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc162420 con 0x7fb2fc0b3600 2024-04-25T09:54:07.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb300fe3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2fc158260 0x7fb2fc1604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb300fe3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2fc158260 0x7fb2fc1604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49464/0 (socket says 172.21.15.139:49464) 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb300fe3640 1 -- 172.21.15.139:0/3718307032 learned_addr learned my addr 172.21.15.139:0/3718307032 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb2fb7fe640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2fc0b3600 0x7fb2fc15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb2fbfff640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2fc06da20 0x7fb2fc15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb300fe3640 1 -- 172.21.15.139:0/3718307032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2fc0b3600 msgr2=0x7fb2fc15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb300fe3640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2fc0b3600 0x7fb2fc15ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb300fe3640 1 -- 172.21.15.139:0/3718307032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2fc06da20 msgr2=0x7fb2fc15b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb300fe3640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2fc06da20 0x7fb2fc15b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.430+0000 7fb300fe3640 1 -- 172.21.15.139:0/3718307032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc158ab0 con 0x7fb2fc158260 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.431+0000 7fb300fe3640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2fc158260 0x7fb2fc1604d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400c780 tx=0x7fb2f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.431+0000 7fb2f8ff9640 1 -- 172.21.15.139:0/3718307032 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4004b60 con 0x7fb2fc158260 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.431+0000 7fb2f8ff9640 1 -- 172.21.15.139:0/3718307032 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f4004d00 con 0x7fb2fc158260 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.431+0000 7fb302a46640 1 -- 172.21.15.139:0/3718307032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2fc0cdf10 con 0x7fb2fc158260 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.431+0000 7fb2f8ff9640 1 -- 172.21.15.139:0/3718307032 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f400aa90 con 0x7fb2fc158260 2024-04-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.431+0000 7fb302a46640 1 -- 172.21.15.139:0/3718307032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2fc158d50 con 0x7fb2fc158260 2024-04-25T09:54:07.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.432+0000 7fb302a46640 1 -- 172.21.15.139:0/3718307032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2fc074520 con 0x7fb2fc158260 2024-04-25T09:54:07.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.432+0000 7fb2f8ff9640 1 -- 172.21.15.139:0/3718307032 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2f400ac30 con 0x7fb2fc158260 2024-04-25T09:54:07.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.433+0000 7fb2f8ff9640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb2e806b4d0 0x7fb2e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.433+0000 7fb2f8ff9640 1 -- 172.21.15.139:0/3718307032 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb2f408a730 con 0x7fb2fc158260 2024-04-25T09:54:07.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.433+0000 7fb2fbfff640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb2e806b4d0 0x7fb2e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.433+0000 7fb2fbfff640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb2e806b4d0 0x7fb2e806d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec034000 tx=0x7fb2ec02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.437+0000 7fb2f8ff9640 1 -- 172.21.15.139:0/3718307032 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2f4020070 con 0x7fb2fc158260 2024-04-25T09:54:07.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.641+0000 7fb302a46640 1 -- 172.21.15.139:0/3718307032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2fc0b31b0 con 0x7fb2fc158260 2024-04-25T09:54:07.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.641+0000 7fb2f8ff9640 1 -- 172.21.15.139:0/3718307032 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fb2f4058b60 con 0x7fb2fc158260 2024-04-25T09:54:07.644 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:07.644 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/892226922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":892226922},{"type":"v1","addr":"172.21.15.150:6834","nonce":892226922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/582019435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":582019435},{"type":"v1","addr":"172.21.15.139:6837","nonce":582019435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/2751714076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2751714076},{"type":"v1","addr":"172.21.15.139:6834","nonce":2751714076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:15.042067+0000","modified":"2024-04-25T09:53:30.094430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2160557452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2160557452},{"type":"v1","addr":"172.21.15.150:6835","nonce":2160557452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:54:07.644 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 9 2024-04-25T09:54:07.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.643+0000 7fb302a46640 1 -- 172.21.15.139:0/3718307032 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb2e806b4d0 msgr2=0x7fb2e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.643+0000 7fb302a46640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb2e806b4d0 0x7fb2e806d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec034000 tx=0x7fb2ec02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:07.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.643+0000 7fb302a46640 1 -- 172.21.15.139:0/3718307032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2fc158260 msgr2=0x7fb2fc1604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.643+0000 7fb302a46640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2fc158260 0x7fb2fc1604d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400c780 tx=0x7fb2f400cc40 comp rx=0 tx=0).stop 2024-04-25T09:54:07.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.643+0000 7fb302a46640 1 -- 172.21.15.139:0/3718307032 shutdown_connections 2024-04-25T09:54:07.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.643+0000 7fb302a46640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2fc158260 0x7fb2fc1604d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.643+0000 7fb302a46640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb2e806b4d0 0x7fb2e806d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.643+0000 7fb302a46640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2fc0b3600 0x7fb2fc15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.643+0000 7fb302a46640 1 --2- 172.21.15.139:0/3718307032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2fc06da20 0x7fb2fc15b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.643+0000 7fb302a46640 1 -- 172.21.15.139:0/3718307032 >> 172.21.15.139:0/3718307032 conn(0x7fb2fc0b23d0 msgr2=0x7fb2fc06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:07.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.644+0000 7fb302a46640 1 -- 172.21.15.139:0/3718307032 shutdown_connections 2024-04-25T09:54:07.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.644+0000 7fb302a46640 1 -- 172.21.15.139:0/3718307032 wait complete. 2024-04-25T09:54:07.657 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:07.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.751+0000 7f2a4acd1640 1 Processor -- start 2024-04-25T09:54:07.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.751+0000 7f2a4acd1640 1 -- start start 2024-04-25T09:54:07.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.751+0000 7f2a4acd1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a4406ab60 0x7f2a4406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.751+0000 7f2a4acd1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2a440b36f0 0x7f2a440b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.751+0000 7f2a4acd1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a440d0400 con 0x7f2a44160e90 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.751+0000 7f2a4acd1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a440d14a0 con 0x7f2a440b36f0 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.751+0000 7f2a4acd1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440d06e0 con 0x7f2a4406ab60 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.752+0000 7f2a4926e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2a440b36f0 0x7f2a440b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.752+0000 7f2a48a6d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2a44160e90 0x7f2a440afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59024/0 (socket says 172.21.15.139:59024) 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.752+0000 7f2a4926e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2a440b36f0 0x7f2a440b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49478/0 (socket says 172.21.15.139:49478) 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.752+0000 7f2a48a6d640 1 -- 172.21.15.139:0/3989595978 learned_addr learned my addr 172.21.15.139:0/3989595978 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.752+0000 7f2a43fff640 1 --2- 172.21.15.139:0/3989595978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a4406ab60 0x7f2a4406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.752+0000 7f2a4926e640 1 -- 172.21.15.139:0/3989595978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a4406ab60 msgr2=0x7f2a4406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.752+0000 7f2a4926e640 1 --2- 172.21.15.139:0/3989595978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a4406ab60 0x7f2a4406aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.752+0000 7f2a4926e640 1 -- 172.21.15.139:0/3989595978 >> v1:172.21.15.139:6789/0 conn(0x7f2a44160e90 legacy=0x7f2a440afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.752+0000 7f2a4926e640 1 -- 172.21.15.139:0/3989595978 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a440d09c0 con 0x7f2a440b36f0 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.752+0000 7f2a437fe640 1 -- 172.21.15.139:0/3989595978 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776481641 0 0) 0x7f2a440d0400 con 0x7f2a44160e90 2024-04-25T09:54:07.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.753+0000 7f2a4926e640 1 --2- 172.21.15.139:0/3989595978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a440b36f0 0x7f2a440b3b40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009d90 tx=0x7f2a3c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd95bfadae87503 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.753+0000 7f2a437fe640 1 -- 172.21.15.139:0/3989595978 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c031600 con 0x7f2a440b36f0 2024-04-25T09:54:07.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.753+0000 7f2a437fe640 1 -- 172.21.15.139:0/3989595978 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3c0317a0 con 0x7f2a440b36f0 2024-04-25T09:54:07.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.753+0000 7f2a437fe640 1 -- 172.21.15.139:0/3989595978 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c031b70 con 0x7f2a440b36f0 2024-04-25T09:54:07.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.753+0000 7f2a4acd1640 1 -- 172.21.15.139:0/3989595978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a440b36f0 msgr2=0x7f2a440b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.753+0000 7f2a4acd1640 1 --2- 172.21.15.139:0/3989595978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a440b36f0 0x7f2a440b3b40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009d90 tx=0x7f2a3c02a5d0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.753+0000 7f2a4acd1640 1 -- 172.21.15.139:0/3989595978 shutdown_connections 2024-04-25T09:54:07.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.753+0000 7f2a4acd1640 1 --2- 172.21.15.139:0/3989595978 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a440b36f0 0x7f2a440b3b40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.753+0000 7f2a4acd1640 1 --2- 172.21.15.139:0/3989595978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a4406ab60 0x7f2a4406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.753+0000 7f2a4acd1640 1 -- 172.21.15.139:0/3989595978 >> 172.21.15.139:0/3989595978 conn(0x7f2a440b23f0 msgr2=0x7f2a440b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:07.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.753+0000 7f2a4acd1640 1 -- 172.21.15.139:0/3989595978 shutdown_connections 2024-04-25T09:54:07.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.753+0000 7f2a4acd1640 1 -- 172.21.15.139:0/3989595978 wait complete. 2024-04-25T09:54:07.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.755+0000 7f2a4acd1640 1 Processor -- start 2024-04-25T09:54:07.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.755+0000 7f2a4acd1640 1 -- start start 2024-04-25T09:54:07.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4acd1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4406ab60 0x7f2a4414d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4acd1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a440b36f0 0x7f2a44159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4acd1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a44160e90 0x7f2a4415f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4acd1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a44161270 con 0x7f2a44160e90 2024-04-25T09:54:07.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4926e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a44160e90 0x7f2a4415f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4acd1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440d0f80 con 0x7f2a4406ab60 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4926e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a44160e90 0x7f2a4415f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49488/0 (socket says 172.21.15.139:49488) 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4926e640 1 -- 172.21.15.139:0/254211936 learned_addr learned my addr 172.21.15.139:0/254211936 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4acd1640 1 -- 172.21.15.139:0/254211936 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4415a680 con 0x7f2a440b36f0 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a48a6d640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4406ab60 0x7f2a4414d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a43fff640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a440b36f0 0x7f2a44159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4926e640 1 -- 172.21.15.139:0/254211936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a440b36f0 msgr2=0x7f2a44159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4926e640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a440b36f0 0x7f2a44159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4926e640 1 -- 172.21.15.139:0/254211936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4406ab60 msgr2=0x7f2a4414d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4926e640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4406ab60 0x7f2a4414d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4926e640 1 -- 172.21.15.139:0/254211936 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4415a0c0 con 0x7f2a44160e90 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a48a6d640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4406ab60 0x7f2a4414d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:07.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.756+0000 7f2a4926e640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a44160e90 0x7f2a4415f4d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009eb0 tx=0x7f2a3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.757+0000 7f2a417fa640 1 -- 172.21.15.139:0/254211936 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c040070 con 0x7f2a44160e90 2024-04-25T09:54:07.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.757+0000 7f2a417fa640 1 -- 172.21.15.139:0/254211936 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3c030be0 con 0x7f2a44160e90 2024-04-25T09:54:07.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.757+0000 7f2a417fa640 1 -- 172.21.15.139:0/254211936 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c009450 con 0x7f2a44160e90 2024-04-25T09:54:07.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.757+0000 7f2a4acd1640 1 -- 172.21.15.139:0/254211936 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4415a3a0 con 0x7f2a44160e90 2024-04-25T09:54:07.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.757+0000 7f2a4acd1640 1 -- 172.21.15.139:0/254211936 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a440c9df0 con 0x7f2a44160e90 2024-04-25T09:54:07.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.758+0000 7f2a4acd1640 1 -- 172.21.15.139:0/254211936 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a44072b30 con 0x7f2a44160e90 2024-04-25T09:54:07.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.758+0000 7f2a417fa640 1 -- 172.21.15.139:0/254211936 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a3c039040 con 0x7f2a44160e90 2024-04-25T09:54:07.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.759+0000 7f2a417fa640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2a2806b4d0 0x7f2a2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:07.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.759+0000 7f2a417fa640 1 -- 172.21.15.139:0/254211936 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2a3c0ab210 con 0x7f2a44160e90 2024-04-25T09:54:07.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.759+0000 7f2a48a6d640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2a2806b4d0 0x7f2a2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:07.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.759+0000 7f2a48a6d640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2a2806b4d0 0x7f2a2806d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2a38007fd0 tx=0x7f2a38007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:07.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.763+0000 7f2a417fa640 1 -- 172.21.15.139:0/254211936 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a44072b30 con 0x7f2a44160e90 2024-04-25T09:54:07.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.935+0000 7f2a4acd1640 1 -- 172.21.15.139:0/254211936 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a440c9ff0 con 0x7f2a44160e90 2024-04-25T09:54:07.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.936+0000 7f2a417fa640 1 -- 172.21.15.139:0/254211936 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18951 (secure 0 0 0) 0x7f2a440c9ff0 con 0x7f2a44160e90 2024-04-25T09:54:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:07.939 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":26,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:56.839903+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:07.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.938+0000 7f2a4acd1640 1 -- 172.21.15.139:0/254211936 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2a2806b4d0 msgr2=0x7f2a2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.938+0000 7f2a4acd1640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2a2806b4d0 0x7f2a2806d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2a38007fd0 tx=0x7f2a38007de0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.938+0000 7f2a4acd1640 1 -- 172.21.15.139:0/254211936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a44160e90 msgr2=0x7f2a4415f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:07.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.938+0000 7f2a4acd1640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a44160e90 0x7f2a4415f4d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009eb0 tx=0x7f2a3c004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:07.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.939+0000 7f2a4acd1640 1 -- 172.21.15.139:0/254211936 shutdown_connections 2024-04-25T09:54:07.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.939+0000 7f2a4acd1640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a44160e90 0x7f2a4415f4d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.939+0000 7f2a4acd1640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2a2806b4d0 0x7f2a2806d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.939+0000 7f2a4acd1640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a440b36f0 0x7f2a44159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.939+0000 7f2a4acd1640 1 --2- 172.21.15.139:0/254211936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a4406ab60 0x7f2a4414d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:07.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.939+0000 7f2a4acd1640 1 -- 172.21.15.139:0/254211936 >> 172.21.15.139:0/254211936 conn(0x7f2a440b23f0 msgr2=0x7f2a44070390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:07.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.939+0000 7f2a4acd1640 1 -- 172.21.15.139:0/254211936 shutdown_connections 2024-04-25T09:54:07.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:07.939+0000 7f2a4acd1640 1 -- 172.21.15.139:0/254211936 wait complete. 2024-04-25T09:54:07.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:53:12.914187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:53:14.014283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:53:17.745984+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:54:07.953 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:08.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.051+0000 7f6b7f007640 1 Processor -- start 2024-04-25T09:54:08.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.051+0000 7f6b7f007640 1 -- start start 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b7f007640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6b78161fd0 0x7f6b7815d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b7f007640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b78099ca0 con 0x7f6b780cd610 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b7f007640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b78099f80 con 0x7f6b780d0a80 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b7f007640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b78099a50 con 0x7f6b78161fd0 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b7cda3640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f6b780cd610 0x7f6b780d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59034/0 (socket says 172.21.15.139:59034) 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b7cda3640 1 -- 172.21.15.139:0/4264068411 learned_addr learned my addr 172.21.15.139:0/4264068411 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b7d5a4640 1 --2- 172.21.15.139:0/4264068411 >> v2:172.21.15.150:3300/0 conn(0x7f6b78161fd0 0x7f6b7815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244120618 0 0) 0x7f6b78099ca0 con 0x7f6b780cd610 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b68003520 con 0x7f6b780cd610 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2566927499 0 0) 0x7f6b68003520 con 0x7f6b780cd610 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 >> v2:172.21.15.150:3300/0 conn(0x7f6b78161fd0 msgr2=0x7f6b7815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 --2- 172.21.15.139:0/4264068411 >> v2:172.21.15.150:3300/0 conn(0x7f6b78161fd0 0x7f6b7815d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 >> v1:172.21.15.150:6789/0 conn(0x7f6b780d0a80 legacy=0x7f6b781618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7809a260 con 0x7f6b780cd610 2024-04-25T09:54:08.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f6b6c001670 con 0x7f6b780cd610 2024-04-25T09:54:08.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 >> v1:172.21.15.139:6789/0 conn(0x7f6b780cd610 legacy=0x7f6b780d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 --2- 172.21.15.139:0/4264068411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b680039b0 0x7f6b68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 --2- 172.21.15.139:0/4264068411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b68004800 0x7f6b68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 --2- 172.21.15.139:0/4264068411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 0x7f6b6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78099a50 con 0x7f6b680051c0 2024-04-25T09:54:08.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6800c070 con 0x7f6b680039b0 2024-04-25T09:54:08.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6800c270 con 0x7f6b68004800 2024-04-25T09:54:08.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.052+0000 7f6b7d5a4640 1 --2- 172.21.15.139:0/4264068411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 0x7f6b6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.053+0000 7f6b7cda3640 1 --2- 172.21.15.139:0/4264068411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b680039b0 0x7f6b68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.053+0000 7f6b77fff640 1 --2- 172.21.15.139:0/4264068411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b68004800 0x7f6b68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.053+0000 7f6b7d5a4640 1 -- 172.21.15.139:0/4264068411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b68004800 msgr2=0x7f6b68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.053+0000 7f6b7d5a4640 1 --2- 172.21.15.139:0/4264068411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b68004800 0x7f6b68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.053+0000 7f6b7d5a4640 1 -- 172.21.15.139:0/4264068411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b680039b0 msgr2=0x7f6b68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.053+0000 7f6b7d5a4640 1 --2- 172.21.15.139:0/4264068411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b680039b0 0x7f6b68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.053+0000 7f6b7d5a4640 1 -- 172.21.15.139:0/4264068411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b68003520 con 0x7f6b680051c0 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.053+0000 7f6b7cda3640 1 --2- 172.21.15.139:0/4264068411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b680039b0 0x7f6b68003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.053+0000 7f6b7d5a4640 1 -- 172.21.15.139:0/4264068411 reap_dead start 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.053+0000 7f6b7d5a4640 1 --2- 172.21.15.139:0/4264068411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 0x7f6b6800b9c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6b780c8780 tx=0x7f6b68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e64e4550a023275 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.054+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70009c10 con 0x7f6b680051c0 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.054+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b70031410 con 0x7f6b680051c0 2024-04-25T09:54:08.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.054+0000 7f6b777fe640 1 -- 172.21.15.139:0/4264068411 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b700317c0 con 0x7f6b680051c0 2024-04-25T09:54:08.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.054+0000 7f6b7f007640 1 -- 172.21.15.139:0/4264068411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 msgr2=0x7f6b6800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.054+0000 7f6b7f007640 1 --2- 172.21.15.139:0/4264068411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 0x7f6b6800b9c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6b780c8780 tx=0x7f6b68005830 comp rx=0 tx=0).stop 2024-04-25T09:54:08.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.054+0000 7f6b7f007640 1 -- 172.21.15.139:0/4264068411 shutdown_connections 2024-04-25T09:54:08.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.054+0000 7f6b7f007640 1 --2- 172.21.15.139:0/4264068411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 0x7f6b6800b9c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.054+0000 7f6b7f007640 1 -- 172.21.15.139:0/4264068411 >> 172.21.15.139:0/4264068411 conn(0x7f6b780b31f0 msgr2=0x7f6b780b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:08.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.054+0000 7f6b7f007640 1 -- 172.21.15.139:0/4264068411 shutdown_connections 2024-04-25T09:54:08.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.054+0000 7f6b7f007640 1 -- 172.21.15.139:0/4264068411 wait complete. 2024-04-25T09:54:08.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.055+0000 7f6b7f007640 1 Processor -- start 2024-04-25T09:54:08.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.055+0000 7f6b7f007640 1 -- start start 2024-04-25T09:54:08.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b7f007640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 0x7f6b780c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b7f007640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b780c5eb0 0x7f6b78070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b7f007640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b78158600 0x7f6b78070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b7f007640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780cd9f0 con 0x7f6b680051c0 2024-04-25T09:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b7f007640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7809a820 con 0x7f6b78158600 2024-04-25T09:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b7f007640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780cdb70 con 0x7f6b780c5eb0 2024-04-25T09:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b7cda3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 0x7f6b780c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b7cda3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 0x7f6b780c6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49500/0 (socket says 172.21.15.139:49500) 2024-04-25T09:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b7cda3640 1 -- 172.21.15.139:0/1144387763 learned_addr learned my addr 172.21.15.139:0/1144387763 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b77fff640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b780c5eb0 0x7f6b78070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b7d5a4640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b78158600 0x7f6b78070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b7cda3640 1 -- 172.21.15.139:0/1144387763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b780c5eb0 msgr2=0x7f6b78070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.056+0000 7f6b7cda3640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b780c5eb0 0x7f6b78070580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.057+0000 7f6b7cda3640 1 -- 172.21.15.139:0/1144387763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b78158600 msgr2=0x7f6b78070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.057+0000 7f6b7cda3640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b78158600 0x7f6b78070ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.057+0000 7f6b7cda3640 1 -- 172.21.15.139:0/1144387763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780c6d60 con 0x7f6b680051c0 2024-04-25T09:54:08.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.057+0000 7f6b7cda3640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 0x7f6b780c6510 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00e7b0 tx=0x7f6b6c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.057+0000 7f6b757fa640 1 -- 172.21.15.139:0/1144387763 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00c870 con 0x7f6b680051c0 2024-04-25T09:54:08.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.057+0000 7f6b757fa640 1 -- 172.21.15.139:0/1144387763 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c018070 con 0x7f6b680051c0 2024-04-25T09:54:08.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.057+0000 7f6b7f007640 1 -- 172.21.15.139:0/1144387763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b780c7040 con 0x7f6b680051c0 2024-04-25T09:54:08.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.057+0000 7f6b757fa640 1 -- 172.21.15.139:0/1144387763 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00cc50 con 0x7f6b680051c0 2024-04-25T09:54:08.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.057+0000 7f6b7f007640 1 -- 172.21.15.139:0/1144387763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b780c56b0 con 0x7f6b680051c0 2024-04-25T09:54:08.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.058+0000 7f6b7f007640 1 -- 172.21.15.139:0/1144387763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b78074520 con 0x7f6b680051c0 2024-04-25T09:54:08.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.058+0000 7f6b757fa640 1 -- 172.21.15.139:0/1144387763 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b6c01d460 con 0x7f6b680051c0 2024-04-25T09:54:08.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.059+0000 7f6b757fa640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6b5c06b4d0 0x7f6b5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.059+0000 7f6b757fa640 1 -- 172.21.15.139:0/1144387763 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6b6c08a550 con 0x7f6b680051c0 2024-04-25T09:54:08.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.059+0000 7f6b77fff640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6b5c06b4d0 0x7f6b5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.059+0000 7f6b77fff640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6b5c06b4d0 0x7f6b5c06d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b781598b0 tx=0x7f6b64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.063+0000 7f6b757fa640 1 -- 172.21.15.139:0/1144387763 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b6c058f40 con 0x7f6b680051c0 2024-04-25T09:54:08.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.235+0000 7f6b7f007640 1 -- 172.21.15.139:0/1144387763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b78158d70 con 0x7f6b680051c0 2024-04-25T09:54:08.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.236+0000 7f6b757fa640 1 -- 172.21.15.139:0/1144387763 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b6c058840 con 0x7f6b680051c0 2024-04-25T09:54:08.238 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:54:08.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.238+0000 7f6b7f007640 1 -- 172.21.15.139:0/1144387763 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6b5c06b4d0 msgr2=0x7f6b5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.238+0000 7f6b7f007640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6b5c06b4d0 0x7f6b5c06d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b781598b0 tx=0x7f6b64009040 comp rx=0 tx=0).stop 2024-04-25T09:54:08.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.238+0000 7f6b7f007640 1 -- 172.21.15.139:0/1144387763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 msgr2=0x7f6b780c6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.238+0000 7f6b7f007640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 0x7f6b780c6510 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00e7b0 tx=0x7f6b6c00ec70 comp rx=0 tx=0).stop 2024-04-25T09:54:08.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.238+0000 7f6b7f007640 1 -- 172.21.15.139:0/1144387763 shutdown_connections 2024-04-25T09:54:08.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.238+0000 7f6b7f007640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6b5c06b4d0 0x7f6b5c06d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.238+0000 7f6b7f007640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b78158600 0x7f6b78070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.238+0000 7f6b7f007640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b780c5eb0 0x7f6b78070580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.238+0000 7f6b7f007640 1 --2- 172.21.15.139:0/1144387763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b680051c0 0x7f6b780c6510 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.238+0000 7f6b7f007640 1 -- 172.21.15.139:0/1144387763 >> 172.21.15.139:0/1144387763 conn(0x7f6b780b31f0 msgr2=0x7f6b780cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:08.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.238+0000 7f6b7f007640 1 -- 172.21.15.139:0/1144387763 shutdown_connections 2024-04-25T09:54:08.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.238+0000 7f6b7f007640 1 -- 172.21.15.139:0/1144387763 wait complete. 2024-04-25T09:54:08.251 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:54:08.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.348+0000 7f10b7a38640 1 Processor -- start 2024-04-25T09:54:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.349+0000 7f10b7a38640 1 -- start start 2024-04-25T09:54:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.349+0000 7f10b7a38640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10b006e040 0x7f10b0161db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.349+0000 7f10b7a38640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f10b015f5c0 0x7f10b015fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.349+0000 7f10b7a38640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b006e5c0 con 0x7f10b015b840 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.349+0000 7f10b7a38640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b006e860 con 0x7f10b015f5c0 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.349+0000 7f10b7a38640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b006eb40 con 0x7f10b006e040 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.349+0000 7f10b5fd5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f10b015f5c0 0x7f10b015fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.350+0000 7f10b57d4640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f10b015b840 0x7f10b01616a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59044/0 (socket says 172.21.15.139:59044) 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.350+0000 7f10b57d4640 1 -- 172.21.15.139:0/1894247837 learned_addr learned my addr 172.21.15.139:0/1894247837 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.350+0000 7f10b4fd3640 1 --2- 172.21.15.139:0/1894247837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10b006e040 0x7f10b0161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.350+0000 7f10b5fd5640 1 -- 172.21.15.139:0/1894247837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10b006e040 msgr2=0x7f10b0161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.350+0000 7f10b5fd5640 1 --2- 172.21.15.139:0/1894247837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10b006e040 0x7f10b0161db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.350+0000 7f10b5fd5640 1 -- 172.21.15.139:0/1894247837 >> v1:172.21.15.139:6789/0 conn(0x7f10b015b840 legacy=0x7f10b01616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.350+0000 7f10b5fd5640 1 -- 172.21.15.139:0/1894247837 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b006ee20 con 0x7f10b015f5c0 2024-04-25T09:54:08.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.350+0000 7f10a7fff640 1 -- 172.21.15.139:0/1894247837 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620057723 0 0) 0x7f10b006e5c0 con 0x7f10b015b840 2024-04-25T09:54:08.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.350+0000 7f10b4fd3640 1 --2- 172.21.15.139:0/1894247837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10b006e040 0x7f10b0161db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:08.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.350+0000 7f10b5fd5640 1 --2- 172.21.15.139:0/1894247837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10b015f5c0 0x7f10b015fa40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f10a80097a0 tx=0x7f10a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fca66c3e318b4fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.351+0000 7f10a7fff640 1 -- 172.21.15.139:0/1894247837 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8037070 con 0x7f10b015f5c0 2024-04-25T09:54:08.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.351+0000 7f10a7fff640 1 -- 172.21.15.139:0/1894247837 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a8032440 con 0x7f10b015f5c0 2024-04-25T09:54:08.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.351+0000 7f10a7fff640 1 -- 172.21.15.139:0/1894247837 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8032810 con 0x7f10b015f5c0 2024-04-25T09:54:08.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.351+0000 7f10b7a38640 1 -- 172.21.15.139:0/1894247837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10b015f5c0 msgr2=0x7f10b015fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.351+0000 7f10b7a38640 1 --2- 172.21.15.139:0/1894247837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10b015f5c0 0x7f10b015fa40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f10a80097a0 tx=0x7f10a802a180 comp rx=0 tx=0).stop 2024-04-25T09:54:08.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.351+0000 7f10b7a38640 1 -- 172.21.15.139:0/1894247837 shutdown_connections 2024-04-25T09:54:08.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.351+0000 7f10b7a38640 1 --2- 172.21.15.139:0/1894247837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10b015f5c0 0x7f10b015fa40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.351+0000 7f10b7a38640 1 --2- 172.21.15.139:0/1894247837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10b006e040 0x7f10b0161db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.351+0000 7f10b7a38640 1 -- 172.21.15.139:0/1894247837 >> 172.21.15.139:0/1894247837 conn(0x7f10b00cf9b0 msgr2=0x7f10b00cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:08.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.351+0000 7f10b7a38640 1 -- 172.21.15.139:0/1894247837 shutdown_connections 2024-04-25T09:54:08.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.351+0000 7f10b7a38640 1 -- 172.21.15.139:0/1894247837 wait complete. 2024-04-25T09:54:08.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.352+0000 7f10b7a38640 1 Processor -- start 2024-04-25T09:54:08.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.352+0000 7f10b7a38640 1 -- start start 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b7a38640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10b006e040 0x7f10b014d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b7a38640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10b015b840 0x7f10b015c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b7a38640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10b015f5c0 0x7f10b015c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b7a38640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0162470 con 0x7f10b006e040 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b7a38640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b01622f0 con 0x7f10b015b840 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b7a38640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b015ff80 con 0x7f10b015f5c0 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b57d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10b006e040 0x7f10b014d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b57d4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10b006e040 0x7f10b014d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49520/0 (socket says 172.21.15.139:49520) 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b57d4640 1 -- 172.21.15.139:0/57837209 learned_addr learned my addr 172.21.15.139:0/57837209 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b57d4640 1 -- 172.21.15.139:0/57837209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10b015f5c0 msgr2=0x7f10b015c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b57d4640 1 --2- 172.21.15.139:0/57837209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10b015f5c0 0x7f10b015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b57d4640 1 -- 172.21.15.139:0/57837209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10b015b840 msgr2=0x7f10b015c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b57d4640 1 --2- 172.21.15.139:0/57837209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10b015b840 0x7f10b015c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.353+0000 7f10b57d4640 1 -- 172.21.15.139:0/57837209 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b00cc560 con 0x7f10b006e040 2024-04-25T09:54:08.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.354+0000 7f10b57d4640 1 --2- 172.21.15.139:0/57837209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10b006e040 0x7f10b014d040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00c890 tx=0x7f10ac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.354+0000 7f10a5ffb640 1 -- 172.21.15.139:0/57837209 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac018070 con 0x7f10b006e040 2024-04-25T09:54:08.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.354+0000 7f10a5ffb640 1 -- 172.21.15.139:0/57837209 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ac00a640 con 0x7f10b006e040 2024-04-25T09:54:08.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.354+0000 7f10b7a38640 1 -- 172.21.15.139:0/57837209 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b00cc840 con 0x7f10b006e040 2024-04-25T09:54:08.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.354+0000 7f10a5ffb640 1 -- 172.21.15.139:0/57837209 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00aa30 con 0x7f10b006e040 2024-04-25T09:54:08.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.354+0000 7f10b7a38640 1 -- 172.21.15.139:0/57837209 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b00cce00 con 0x7f10b006e040 2024-04-25T09:54:08.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.355+0000 7f10b7a38640 1 -- 172.21.15.139:0/57837209 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b0074520 con 0x7f10b006e040 2024-04-25T09:54:08.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.355+0000 7f10a5ffb640 1 -- 172.21.15.139:0/57837209 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f10ac01b070 con 0x7f10b006e040 2024-04-25T09:54:08.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.356+0000 7f10a5ffb640 1 --2- 172.21.15.139:0/57837209 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f109406b500 0x7f109406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.356+0000 7f10a5ffb640 1 -- 172.21.15.139:0/57837209 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f10ac089ca0 con 0x7f10b006e040 2024-04-25T09:54:08.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.356+0000 7f10b4fd3640 1 --2- 172.21.15.139:0/57837209 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f109406b500 0x7f109406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.356+0000 7f10b4fd3640 1 --2- 172.21.15.139:0/57837209 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f109406b500 0x7f109406d9c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10a0006f00 tx=0x7f10a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.360+0000 7f10a5ffb640 1 -- 172.21.15.139:0/57837209 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10ac058720 con 0x7f10b006e040 2024-04-25T09:54:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.529+0000 7f10b7a38640 1 -- 172.21.15.139:0/57837209 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10b0068350 con 0x7f10b006e040 2024-04-25T09:54:08.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.530+0000 7f10a5ffb640 1 -- 172.21.15.139:0/57837209 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f10ac058020 con 0x7f10b006e040 2024-04-25T09:54:08.532 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:54:08.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.531+0000 7f10b7a38640 1 -- 172.21.15.139:0/57837209 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f109406b500 msgr2=0x7f109406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.531+0000 7f10b7a38640 1 --2- 172.21.15.139:0/57837209 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f109406b500 0x7f109406d9c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10a0006f00 tx=0x7f10a0008040 comp rx=0 tx=0).stop 2024-04-25T09:54:08.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.531+0000 7f10b7a38640 1 -- 172.21.15.139:0/57837209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10b006e040 msgr2=0x7f10b014d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.531+0000 7f10b7a38640 1 --2- 172.21.15.139:0/57837209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10b006e040 0x7f10b014d040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00c890 tx=0x7f10ac00cd50 comp rx=0 tx=0).stop 2024-04-25T09:54:08.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.532+0000 7f10b7a38640 1 -- 172.21.15.139:0/57837209 shutdown_connections 2024-04-25T09:54:08.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.532+0000 7f10b7a38640 1 --2- 172.21.15.139:0/57837209 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f109406b500 0x7f109406d9c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.532+0000 7f10b7a38640 1 --2- 172.21.15.139:0/57837209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10b015f5c0 0x7f10b015c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.532+0000 7f10b7a38640 1 --2- 172.21.15.139:0/57837209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10b015b840 0x7f10b015c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.532+0000 7f10b7a38640 1 --2- 172.21.15.139:0/57837209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10b006e040 0x7f10b014d040 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.532+0000 7f10b7a38640 1 -- 172.21.15.139:0/57837209 >> 172.21.15.139:0/57837209 conn(0x7f10b00cf9b0 msgr2=0x7f10b0161290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:08.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.532+0000 7f10b7a38640 1 -- 172.21.15.139:0/57837209 shutdown_connections 2024-04-25T09:54:08.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.532+0000 7f10b7a38640 1 -- 172.21.15.139:0/57837209 wait complete. 2024-04-25T09:54:08.545 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:54:08.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.642+0000 7fad3acfc640 1 Processor -- start 2024-04-25T09:54:08.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.642+0000 7fad3acfc640 1 -- start start 2024-04-25T09:54:08.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad3acfc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3415dbf0 0x7fad341617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad3acfc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fad340cee50 0x7fad340d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad3acfc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad34161dc0 con 0x7fad340b9370 2024-04-25T09:54:08.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad3acfc640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad340d1ee0 con 0x7fad340cee50 2024-04-25T09:54:08.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad3acfc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad341620a0 con 0x7fad3415dbf0 2024-04-25T09:54:08.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad33fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fad340b9370 0x7fad340ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59058/0 (socket says 172.21.15.139:59058) 2024-04-25T09:54:08.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad33fff640 1 -- 172.21.15.139:0/2420613658 learned_addr learned my addr 172.21.15.139:0/2420613658 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:08.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad38a98640 1 --2- 172.21.15.139:0/2420613658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3415dbf0 0x7fad341617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad39299640 1 --2- 172.21.15.139:0/2420613658 >> v2:172.21.15.150:3300/0 conn(0x7fad340cee50 0x7fad340d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842564058 0 0) 0x7fad34161dc0 con 0x7fad340b9370 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad24003520 con 0x7fad340b9370 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3778408077 0 0) 0x7fad24003520 con 0x7fad340b9370 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3415dbf0 msgr2=0x7fad341617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 --2- 172.21.15.139:0/2420613658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3415dbf0 0x7fad341617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 >> v2:172.21.15.150:3300/0 conn(0x7fad340cee50 msgr2=0x7fad340d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 --2- 172.21.15.139:0/2420613658 >> v2:172.21.15.150:3300/0 conn(0x7fad340cee50 0x7fad340d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad34162380 con 0x7fad340b9370 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fad20001670 con 0x7fad340b9370 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 >> v1:172.21.15.139:6789/0 conn(0x7fad340b9370 legacy=0x7fad340ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 --2- 172.21.15.139:0/2420613658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3415dbf0 0x7fad240039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 --2- 172.21.15.139:0/2420613658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad24003f30 0x7fad24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 --2- 172.21.15.139:0/2420613658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad24004d40 0x7fad240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad340d1ee0 con 0x7fad24004d40 2024-04-25T09:54:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad341620a0 con 0x7fad24003f30 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2400b9c0 con 0x7fad3415dbf0 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.643+0000 7fad39299640 1 --2- 172.21.15.139:0/2420613658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad24004d40 0x7fad240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.644+0000 7fad38a98640 1 --2- 172.21.15.139:0/2420613658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3415dbf0 0x7fad240039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.644+0000 7fad33fff640 1 --2- 172.21.15.139:0/2420613658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad24003f30 0x7fad24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.644+0000 7fad39299640 1 -- 172.21.15.139:0/2420613658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3415dbf0 msgr2=0x7fad240039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.644+0000 7fad39299640 1 --2- 172.21.15.139:0/2420613658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3415dbf0 0x7fad240039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.644+0000 7fad39299640 1 -- 172.21.15.139:0/2420613658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad24003f30 msgr2=0x7fad24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.644+0000 7fad39299640 1 --2- 172.21.15.139:0/2420613658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad24003f30 0x7fad24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.644+0000 7fad39299640 1 -- 172.21.15.139:0/2420613658 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad24003520 con 0x7fad24004d40 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.644+0000 7fad39299640 1 --2- 172.21.15.139:0/2420613658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad24004d40 0x7fad240051c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009f90 tx=0x7fad24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a3246812cd7ab9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.644+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c009c10 con 0x7fad24004d40 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.645+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c036070 con 0x7fad24004d40 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.645+0000 7fad337fe640 1 -- 172.21.15.139:0/2420613658 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c031670 con 0x7fad24004d40 2024-04-25T09:54:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.645+0000 7fad3acfc640 1 -- 172.21.15.139:0/2420613658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad24004d40 msgr2=0x7fad240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.645+0000 7fad3acfc640 1 --2- 172.21.15.139:0/2420613658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad24004d40 0x7fad240051c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009f90 tx=0x7fad24005830 comp rx=0 tx=0).stop 2024-04-25T09:54:08.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.645+0000 7fad39299640 1 -- 172.21.15.139:0/2420613658 reap_dead start 2024-04-25T09:54:08.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.645+0000 7fad3acfc640 1 -- 172.21.15.139:0/2420613658 shutdown_connections 2024-04-25T09:54:08.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.645+0000 7fad3acfc640 1 -- 172.21.15.139:0/2420613658 >> 172.21.15.139:0/2420613658 conn(0x7fad340b5480 msgr2=0x7fad340b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:08.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.645+0000 7fad3acfc640 1 -- 172.21.15.139:0/2420613658 shutdown_connections 2024-04-25T09:54:08.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.645+0000 7fad3acfc640 1 -- 172.21.15.139:0/2420613658 wait complete. 2024-04-25T09:54:08.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.646+0000 7fad3acfc640 1 Processor -- start 2024-04-25T09:54:08.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.646+0000 7fad3acfc640 1 -- start start 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad3acfc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad24004d40 0x7fad3415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad3acfc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad3414dd30 0x7fad341f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad3acfc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad341f4130 0x7fad340d3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad3acfc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3415dfd0 con 0x7fad3414dd30 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad3acfc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad340b97d0 con 0x7fad341f4130 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad3acfc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3414e440 con 0x7fad24004d40 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad33fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad3414dd30 0x7fad341f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad33fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad3414dd30 0x7fad341f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49542/0 (socket says 172.21.15.139:49542) 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad33fff640 1 -- 172.21.15.139:0/969754923 learned_addr learned my addr 172.21.15.139:0/969754923 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad38a98640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad24004d40 0x7fad3415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad33fff640 1 -- 172.21.15.139:0/969754923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad24004d40 msgr2=0x7fad3415cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad33fff640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad24004d40 0x7fad3415cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad33fff640 1 -- 172.21.15.139:0/969754923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad341f4130 msgr2=0x7fad340d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad39299640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad341f4130 0x7fad340d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.647+0000 7fad33fff640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad341f4130 0x7fad340d3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.648+0000 7fad33fff640 1 -- 172.21.15.139:0/969754923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad34150800 con 0x7fad3414dd30 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.648+0000 7fad33fff640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad3414dd30 0x7fad341f3bf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fad2000e870 tx=0x7fad2000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.648+0000 7fad317fa640 1 -- 172.21.15.139:0/969754923 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2000c8d0 con 0x7fad3414dd30 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.648+0000 7fad317fa640 1 -- 172.21.15.139:0/969754923 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad20018070 con 0x7fad3414dd30 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.648+0000 7fad3acfc640 1 -- 172.21.15.139:0/969754923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad34150ae0 con 0x7fad3414dd30 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.648+0000 7fad317fa640 1 -- 172.21.15.139:0/969754923 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2000ccb0 con 0x7fad3414dd30 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.648+0000 7fad3acfc640 1 -- 172.21.15.139:0/969754923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3414d7e0 con 0x7fad3414dd30 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.649+0000 7fad3acfc640 1 -- 172.21.15.139:0/969754923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad34074520 con 0x7fad3414dd30 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.649+0000 7fad317fa640 1 -- 172.21.15.139:0/969754923 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad2001d460 con 0x7fad3414dd30 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.649+0000 7fad317fa640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad1806b4d0 0x7fad1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.650+0000 7fad317fa640 1 -- 172.21.15.139:0/969754923 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fad2008a4b0 con 0x7fad3414dd30 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.650+0000 7fad38a98640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad1806b4d0 0x7fad1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.650+0000 7fad38a98640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad1806b4d0 0x7fad1806d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fad280099e0 tx=0x7fad28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.654+0000 7fad317fa640 1 -- 172.21.15.139:0/969754923 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad20058fc0 con 0x7fad3414dd30 2024-04-25T09:54:08.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.824+0000 7fad3acfc640 1 -- 172.21.15.139:0/969754923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fad340d1680 con 0x7fad3414dd30 2024-04-25T09:54:08.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.824+0000 7fad317fa640 1 -- 172.21.15.139:0/969754923 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fad200588c0 con 0x7fad3414dd30 2024-04-25T09:54:08.827 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:54:08.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.826+0000 7fad3acfc640 1 -- 172.21.15.139:0/969754923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad1806b4d0 msgr2=0x7fad1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.826+0000 7fad3acfc640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad1806b4d0 0x7fad1806d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fad280099e0 tx=0x7fad28008040 comp rx=0 tx=0).stop 2024-04-25T09:54:08.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.826+0000 7fad3acfc640 1 -- 172.21.15.139:0/969754923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad3414dd30 msgr2=0x7fad341f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.826+0000 7fad3acfc640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad3414dd30 0x7fad341f3bf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fad2000e870 tx=0x7fad2000ed30 comp rx=0 tx=0).stop 2024-04-25T09:54:08.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.826+0000 7fad3acfc640 1 -- 172.21.15.139:0/969754923 shutdown_connections 2024-04-25T09:54:08.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.826+0000 7fad3acfc640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad3414dd30 0x7fad341f3bf0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.826+0000 7fad3acfc640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad1806b4d0 0x7fad1806d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.826+0000 7fad3acfc640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad341f4130 0x7fad340d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.826+0000 7fad3acfc640 1 --2- 172.21.15.139:0/969754923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad24004d40 0x7fad3415cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.826+0000 7fad3acfc640 1 -- 172.21.15.139:0/969754923 >> 172.21.15.139:0/969754923 conn(0x7fad340b5480 msgr2=0x7fad34158930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:08.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.827+0000 7fad3acfc640 1 -- 172.21.15.139:0/969754923 shutdown_connections 2024-04-25T09:54:08.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.827+0000 7fad3acfc640 1 -- 172.21.15.139:0/969754923 wait complete. 2024-04-25T09:54:08.840 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:54:08.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.938+0000 7fd0a735c640 1 Processor -- start 2024-04-25T09:54:08.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.938+0000 7fd0a735c640 1 -- start start 2024-04-25T09:54:08.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.939+0000 7fd0a735c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd0a01617e0 0x7fd0a00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.939+0000 7fd0a735c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd0a00cec80 0x7fd0a00d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.939+0000 7fd0a735c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a0161dc0 con 0x7fd0a015dbf0 2024-04-25T09:54:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.939+0000 7fd0a735c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a00cf0d0 con 0x7fd0a00cec80 2024-04-25T09:54:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.939+0000 7fd0a735c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a00d1ee0 con 0x7fd0a01617e0 2024-04-25T09:54:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.939+0000 7fd0a50f8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd0a015dbf0 0x7fd0a00b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59062/0 (socket says 172.21.15.139:59062) 2024-04-25T09:54:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.939+0000 7fd0a58f9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd0a00cec80 0x7fd0a00d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.939+0000 7fd0a50f8640 1 -- 172.21.15.139:0/187809482 learned_addr learned my addr 172.21.15.139:0/187809482 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.940+0000 7fd0a58f9640 1 -- 172.21.15.139:0/187809482 >> v2:172.21.15.150:3300/0 conn(0x7fd0a01617e0 msgr2=0x7fd0a00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.940+0000 7fd0a58f9640 1 --2- 172.21.15.139:0/187809482 >> v2:172.21.15.150:3300/0 conn(0x7fd0a01617e0 0x7fd0a00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.940+0000 7fd0a58f9640 1 -- 172.21.15.139:0/187809482 >> v1:172.21.15.139:6789/0 conn(0x7fd0a015dbf0 legacy=0x7fd0a00b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.940+0000 7fd09ffff640 1 -- 172.21.15.139:0/187809482 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079798604 0 0) 0x7fd0a0161dc0 con 0x7fd0a015dbf0 2024-04-25T09:54:08.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.940+0000 7fd0a58f9640 1 -- 172.21.15.139:0/187809482 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a01620a0 con 0x7fd0a00cec80 2024-04-25T09:54:08.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.940+0000 7fd0a58f9640 1 --2- 172.21.15.139:0/187809482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0a00cec80 0x7fd0a00d2370 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd0a008c420 tx=0x7fd09002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38fed724e1fa419 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.940+0000 7fd09ffff640 1 -- 172.21.15.139:0/187809482 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090037070 con 0x7fd0a00cec80 2024-04-25T09:54:08.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.940+0000 7fd09ffff640 1 -- 172.21.15.139:0/187809482 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd090032440 con 0x7fd0a00cec80 2024-04-25T09:54:08.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.940+0000 7fd09ffff640 1 -- 172.21.15.139:0/187809482 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09002ae60 con 0x7fd0a00cec80 2024-04-25T09:54:08.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.941+0000 7fd0a735c640 1 -- 172.21.15.139:0/187809482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0a00cec80 msgr2=0x7fd0a00d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.941+0000 7fd0a735c640 1 --2- 172.21.15.139:0/187809482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0a00cec80 0x7fd0a00d2370 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd0a008c420 tx=0x7fd09002a110 comp rx=0 tx=0).stop 2024-04-25T09:54:08.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.941+0000 7fd0a735c640 1 -- 172.21.15.139:0/187809482 shutdown_connections 2024-04-25T09:54:08.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.941+0000 7fd0a735c640 1 --2- 172.21.15.139:0/187809482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0a00cec80 0x7fd0a00d2370 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.941+0000 7fd0a735c640 1 --2- 172.21.15.139:0/187809482 >> v2:172.21.15.150:3300/0 conn(0x7fd0a01617e0 0x7fd0a00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.941+0000 7fd0a735c640 1 -- 172.21.15.139:0/187809482 >> 172.21.15.139:0/187809482 conn(0x7fd0a00b5480 msgr2=0x7fd0a00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:08.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.941+0000 7fd0a735c640 1 -- 172.21.15.139:0/187809482 shutdown_connections 2024-04-25T09:54:08.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.941+0000 7fd0a735c640 1 -- 172.21.15.139:0/187809482 wait complete. 2024-04-25T09:54:08.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.942+0000 7fd0a735c640 1 Processor -- start 2024-04-25T09:54:08.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.942+0000 7fd0a735c640 1 -- start start 2024-04-25T09:54:08.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a735c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a00cec80 0x7fd0a00caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a735c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0a015dbf0 0x7fd0a014d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a735c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0a01617e0 0x7fd0a014dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a735c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a015dfd0 con 0x7fd0a01617e0 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a735c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a0162660 con 0x7fd0a00cec80 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a735c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a01f48b0 con 0x7fd0a015dbf0 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a58f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0a01617e0 0x7fd0a014dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a58f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0a01617e0 0x7fd0a014dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49558/0 (socket says 172.21.15.139:49558) 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a58f9640 1 -- 172.21.15.139:0/474839432 learned_addr learned my addr 172.21.15.139:0/474839432 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a58f9640 1 -- 172.21.15.139:0/474839432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0a015dbf0 msgr2=0x7fd0a014d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a58f9640 1 --2- 172.21.15.139:0/474839432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0a015dbf0 0x7fd0a014d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a58f9640 1 -- 172.21.15.139:0/474839432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a00cec80 msgr2=0x7fd0a00caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a58f9640 1 --2- 172.21.15.139:0/474839432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a00cec80 0x7fd0a00caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a58f9640 1 -- 172.21.15.139:0/474839432 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a01f42f0 con 0x7fd0a01617e0 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.943+0000 7fd0a58f9640 1 --2- 172.21.15.139:0/474839432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0a01617e0 0x7fd0a014dc60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd090009890 tx=0x7fd0900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.944+0000 7fd09dffb640 1 -- 172.21.15.139:0/474839432 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090041070 con 0x7fd0a01617e0 2024-04-25T09:54:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.944+0000 7fd09dffb640 1 -- 172.21.15.139:0/474839432 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09003c870 con 0x7fd0a01617e0 2024-04-25T09:54:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.944+0000 7fd0a735c640 1 -- 172.21.15.139:0/474839432 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a01f45d0 con 0x7fd0a01617e0 2024-04-25T09:54:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.944+0000 7fd09dffb640 1 -- 172.21.15.139:0/474839432 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090037040 con 0x7fd0a01617e0 2024-04-25T09:54:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.944+0000 7fd0a735c640 1 -- 172.21.15.139:0/474839432 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a00cc600 con 0x7fd0a01617e0 2024-04-25T09:54:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.945+0000 7fd0a735c640 1 -- 172.21.15.139:0/474839432 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a0074520 con 0x7fd0a01617e0 2024-04-25T09:54:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.945+0000 7fd09dffb640 1 -- 172.21.15.139:0/474839432 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd090044070 con 0x7fd0a01617e0 2024-04-25T09:54:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.946+0000 7fd09dffb640 1 --2- 172.21.15.139:0/474839432 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd08c06b4d0 0x7fd08c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.946+0000 7fd09dffb640 1 -- 172.21.15.139:0/474839432 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd0900ab110 con 0x7fd0a01617e0 2024-04-25T09:54:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.946+0000 7fd0a50f8640 1 --2- 172.21.15.139:0/474839432 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd08c06b4d0 0x7fd08c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:08.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.946+0000 7fd0a50f8640 1 --2- 172.21.15.139:0/474839432 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd08c06b4d0 0x7fd08c06d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd098001560 tx=0x7fd098007ca0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:08.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:08.950+0000 7fd09dffb640 1 -- 172.21.15.139:0/474839432 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd090079d40 con 0x7fd0a01617e0 2024-04-25T09:54:09.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.119+0000 7fd0a735c640 1 -- 172.21.15.139:0/474839432 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd0a00d1680 con 0x7fd0a01617e0 2024-04-25T09:54:09.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.120+0000 7fd09dffb640 1 -- 172.21.15.139:0/474839432 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd090079640 con 0x7fd0a01617e0 2024-04-25T09:54:09.122 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:54:09.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.122+0000 7fd0a735c640 1 -- 172.21.15.139:0/474839432 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd08c06b4d0 msgr2=0x7fd08c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.122+0000 7fd0a735c640 1 --2- 172.21.15.139:0/474839432 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd08c06b4d0 0x7fd08c06d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd098001560 tx=0x7fd098007ca0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.122+0000 7fd0a735c640 1 -- 172.21.15.139:0/474839432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0a01617e0 msgr2=0x7fd0a014dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.122+0000 7fd0a735c640 1 --2- 172.21.15.139:0/474839432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0a01617e0 0x7fd0a014dc60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd090009890 tx=0x7fd0900046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.122+0000 7fd0a735c640 1 -- 172.21.15.139:0/474839432 shutdown_connections 2024-04-25T09:54:09.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.122+0000 7fd0a735c640 1 --2- 172.21.15.139:0/474839432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0a01617e0 0x7fd0a014dc60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.122+0000 7fd0a735c640 1 --2- 172.21.15.139:0/474839432 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd08c06b4d0 0x7fd08c06d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.122+0000 7fd0a735c640 1 --2- 172.21.15.139:0/474839432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0a015dbf0 0x7fd0a014d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.122+0000 7fd0a735c640 1 --2- 172.21.15.139:0/474839432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a00cec80 0x7fd0a00caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.122+0000 7fd0a735c640 1 -- 172.21.15.139:0/474839432 >> 172.21.15.139:0/474839432 conn(0x7fd0a00b5480 msgr2=0x7fd0a00b7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.122+0000 7fd0a735c640 1 -- 172.21.15.139:0/474839432 shutdown_connections 2024-04-25T09:54:09.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.122+0000 7fd0a735c640 1 -- 172.21.15.139:0/474839432 wait complete. 2024-04-25T09:54:09.135 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:09.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.230+0000 7f377715c640 1 Processor -- start 2024-04-25T09:54:09.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.230+0000 7f377715c640 1 -- start start 2024-04-25T09:54:09.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f377715c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377015ac40 0x7f377006e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f377715c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f377006e850 0x7f37700b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f377715c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37700af6d0 con 0x7f377015a860 2024-04-25T09:54:09.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f377715c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37700b02e0 con 0x7f377006e850 2024-04-25T09:54:09.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f377715c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37700af9b0 con 0x7f377015ac40 2024-04-25T09:54:09.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f3774ef8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f377015a860 0x7f37700cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59070/0 (socket says 172.21.15.139:59070) 2024-04-25T09:54:09.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f3774ef8640 1 -- 172.21.15.139:0/3040949286 learned_addr learned my addr 172.21.15.139:0/3040949286 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:09.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376ffff640 1 --2- 172.21.15.139:0/3040949286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377015ac40 0x7f377006e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f37756f9640 1 --2- 172.21.15.139:0/3040949286 >> v2:172.21.15.150:3300/0 conn(0x7f377006e850 0x7f37700b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784694814 0 0) 0x7f37700af6d0 con 0x7f377015a860 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3760003520 con 0x7f377015a860 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4025836656 0 0) 0x7f3760003520 con 0x7f377015a860 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377015ac40 msgr2=0x7f377006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 --2- 172.21.15.139:0/3040949286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377015ac40 0x7f377006e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 >> v2:172.21.15.150:3300/0 conn(0x7f377006e850 msgr2=0x7f37700b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 --2- 172.21.15.139:0/3040949286 >> v2:172.21.15.150:3300/0 conn(0x7f377006e850 0x7f37700b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37700afc90 con 0x7f377015a860 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f3764001670 con 0x7f377015a860 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 >> v1:172.21.15.139:6789/0 conn(0x7f377015a860 legacy=0x7f37700cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 --2- 172.21.15.139:0/3040949286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37600039b0 0x7f3760003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 --2- 172.21.15.139:0/3040949286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3760004800 0x7f3760004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 --2- 172.21.15.139:0/3040949286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377015ac40 0x7f37600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37700b02e0 con 0x7f37600039b0 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37700af9b0 con 0x7f3760004800 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.231+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376000b9c0 con 0x7f377015ac40 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.232+0000 7f3774ef8640 1 --2- 172.21.15.139:0/3040949286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37600039b0 0x7f3760003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.232+0000 7f376ffff640 1 --2- 172.21.15.139:0/3040949286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3760004800 0x7f3760004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.232+0000 7f37756f9640 1 --2- 172.21.15.139:0/3040949286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377015ac40 0x7f37600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.232+0000 7f3774ef8640 1 -- 172.21.15.139:0/3040949286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377015ac40 msgr2=0x7f37600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.232+0000 7f3774ef8640 1 --2- 172.21.15.139:0/3040949286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377015ac40 0x7f37600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.232+0000 7f3774ef8640 1 -- 172.21.15.139:0/3040949286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3760004800 msgr2=0x7f3760004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.232+0000 7f3774ef8640 1 --2- 172.21.15.139:0/3040949286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3760004800 0x7f3760004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.232+0000 7f3774ef8640 1 -- 172.21.15.139:0/3040949286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3760003520 con 0x7f37600039b0 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.233+0000 7f3774ef8640 1 --2- 172.21.15.139:0/3040949286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37600039b0 0x7f3760003dd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3764001050 tx=0x7f3760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad9e575a3694b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.233+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376402e070 con 0x7f37600039b0 2024-04-25T09:54:09.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.233+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3764005af0 con 0x7f37600039b0 2024-04-25T09:54:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.233+0000 7f376f7fe640 1 -- 172.21.15.139:0/3040949286 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764007d70 con 0x7f37600039b0 2024-04-25T09:54:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.233+0000 7f377715c640 1 -- 172.21.15.139:0/3040949286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37600039b0 msgr2=0x7f3760003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.233+0000 7f377715c640 1 --2- 172.21.15.139:0/3040949286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37600039b0 0x7f3760003dd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3764001050 tx=0x7f3760005830 comp rx=0 tx=0).stop 2024-04-25T09:54:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.233+0000 7f37756f9640 1 -- 172.21.15.139:0/3040949286 reap_dead start 2024-04-25T09:54:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.233+0000 7f377715c640 1 -- 172.21.15.139:0/3040949286 shutdown_connections 2024-04-25T09:54:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.233+0000 7f377715c640 1 -- 172.21.15.139:0/3040949286 >> 172.21.15.139:0/3040949286 conn(0x7f3770066f20 msgr2=0x7f377006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.233+0000 7f377715c640 1 -- 172.21.15.139:0/3040949286 shutdown_connections 2024-04-25T09:54:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.233+0000 7f377715c640 1 -- 172.21.15.139:0/3040949286 wait complete. 2024-04-25T09:54:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.234+0000 7f377715c640 1 Processor -- start 2024-04-25T09:54:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.234+0000 7f377715c640 1 -- start start 2024-04-25T09:54:09.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.235+0000 7f377715c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37600039b0 0x7f377013b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.235+0000 7f377715c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f377013bb10 0x7f377006d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.235+0000 7f377715c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377006d740 0x7f37701e4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.235+0000 7f377715c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37700af2a0 con 0x7f37600039b0 2024-04-25T09:54:09.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.235+0000 7f377715c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37700cd3c0 con 0x7f377013bb10 2024-04-25T09:54:09.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.235+0000 7f377715c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37700af420 con 0x7f377006d740 2024-04-25T09:54:09.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.235+0000 7f3774ef8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37600039b0 0x7f377013b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.235+0000 7f3774ef8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37600039b0 0x7f377013b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49582/0 (socket says 172.21.15.139:49582) 2024-04-25T09:54:09.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.235+0000 7f3774ef8640 1 -- 172.21.15.139:0/3683208697 learned_addr learned my addr 172.21.15.139:0/3683208697 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:09.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f37756f9640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377006d740 0x7f37701e4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f376ffff640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f377013bb10 0x7f377006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f3774ef8640 1 -- 172.21.15.139:0/3683208697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377006d740 msgr2=0x7f37701e4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f3774ef8640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377006d740 0x7f37701e4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f3774ef8640 1 -- 172.21.15.139:0/3683208697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f377013bb10 msgr2=0x7f377006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f3774ef8640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f377013bb10 0x7f377006d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f3774ef8640 1 -- 172.21.15.139:0/3683208697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377013bfe0 con 0x7f37600039b0 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f376ffff640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f377013bb10 0x7f377006d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f3774ef8640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37600039b0 0x7f377013b5d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3764034040 tx=0x7f3764000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f376d7fa640 1 -- 172.21.15.139:0/3683208697 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764042070 con 0x7f37600039b0 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f376d7fa640 1 -- 172.21.15.139:0/3683208697 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3764007480 con 0x7f37600039b0 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f376d7fa640 1 -- 172.21.15.139:0/3683208697 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764007d50 con 0x7f37600039b0 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f377715c640 1 -- 172.21.15.139:0/3683208697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37701f2550 con 0x7f37600039b0 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.236+0000 7f377715c640 1 -- 172.21.15.139:0/3683208697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37701f2b10 con 0x7f37600039b0 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.237+0000 7f377715c640 1 -- 172.21.15.139:0/3683208697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377006db50 con 0x7f37600039b0 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.238+0000 7f376d7fa640 1 -- 172.21.15.139:0/3683208697 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3764045070 con 0x7f37600039b0 2024-04-25T09:54:09.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.238+0000 7f376d7fa640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f375406b4d0 0x7f375406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.238+0000 7f376d7fa640 1 -- 172.21.15.139:0/3683208697 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f37640ab350 con 0x7f37600039b0 2024-04-25T09:54:09.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.238+0000 7f376ffff640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f375406b4d0 0x7f375406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.238+0000 7f376ffff640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f375406b4d0 0x7f375406d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f377013bf10 tx=0x7f375c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.242+0000 7f376d7fa640 1 -- 172.21.15.139:0/3683208697 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f377006db50 con 0x7f37600039b0 2024-04-25T09:54:09.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.448+0000 7f377715c640 1 -- 172.21.15.139:0/3683208697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3770074520 con 0x7f37600039b0 2024-04-25T09:54:09.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.449+0000 7f376d7fa640 1 -- 172.21.15.139:0/3683208697 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f3764005ac0 con 0x7f37600039b0 2024-04-25T09:54:09.452 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:09.452 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/892226922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":892226922},{"type":"v1","addr":"172.21.15.150:6834","nonce":892226922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/582019435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":582019435},{"type":"v1","addr":"172.21.15.139:6837","nonce":582019435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/2751714076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2751714076},{"type":"v1","addr":"172.21.15.139:6834","nonce":2751714076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:15.042067+0000","modified":"2024-04-25T09:53:30.094430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2160557452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2160557452},{"type":"v1","addr":"172.21.15.150:6835","nonce":2160557452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:54:09.452 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 9 2024-04-25T09:54:09.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.450+0000 7f377715c640 1 -- 172.21.15.139:0/3683208697 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f375406b4d0 msgr2=0x7f375406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.450+0000 7f377715c640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f375406b4d0 0x7f375406d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f377013bf10 tx=0x7f375c008040 comp rx=0 tx=0).stop 2024-04-25T09:54:09.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.450+0000 7f377715c640 1 -- 172.21.15.139:0/3683208697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37600039b0 msgr2=0x7f377013b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.450+0000 7f377715c640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37600039b0 0x7f377013b5d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3764034040 tx=0x7f3764000fe0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.451+0000 7f377715c640 1 -- 172.21.15.139:0/3683208697 shutdown_connections 2024-04-25T09:54:09.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.451+0000 7f377715c640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f375406b4d0 0x7f375406d990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.451+0000 7f377715c640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f377006d740 0x7f37701e4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.451+0000 7f377715c640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f377013bb10 0x7f377006d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.451+0000 7f377715c640 1 --2- 172.21.15.139:0/3683208697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37600039b0 0x7f377013b5d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.451+0000 7f377715c640 1 -- 172.21.15.139:0/3683208697 >> 172.21.15.139:0/3683208697 conn(0x7f3770066f20 msgr2=0x7f3770156db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.451+0000 7f377715c640 1 -- 172.21.15.139:0/3683208697 shutdown_connections 2024-04-25T09:54:09.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.451+0000 7f377715c640 1 -- 172.21.15.139:0/3683208697 wait complete. 2024-04-25T09:54:09.464 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:09.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.559+0000 7f93a1c76640 1 Processor -- start 2024-04-25T09:54:09.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.559+0000 7f93a1c76640 1 -- start start 2024-04-25T09:54:09.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.559+0000 7f93a1c76640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f939c15aa00 0x7f939c0b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.559+0000 7f93a1c76640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f939c072280 0x7f939c072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.559+0000 7f93a1c76640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939c15e500 con 0x7f939c0b5110 2024-04-25T09:54:09.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.559+0000 7f93a1c76640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939c0b5c20 con 0x7f939c072280 2024-04-25T09:54:09.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.559+0000 7f93a1c76640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c0b45d0 con 0x7f939c15aa00 2024-04-25T09:54:09.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.560+0000 7f939bfff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f939c072280 0x7f939c072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.560+0000 7f939b7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f939c15aa00 0x7f939c0b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.560+0000 7f939b7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f939c15aa00 0x7f939c0b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:51328/0 (socket says 172.21.15.139:51328) 2024-04-25T09:54:09.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.560+0000 7f9393fff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f939c0b5110 0x7f939c15ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:57994/0 (socket says 172.21.15.139:57994) 2024-04-25T09:54:09.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.560+0000 7f939bfff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f939c072280 0x7f939c072710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:50334/0 (socket says 172.21.15.139:50334) 2024-04-25T09:54:09.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.560+0000 7f939b7fe640 1 -- 172.21.15.139:0/2987520206 learned_addr learned my addr 172.21.15.139:0/2987520206 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:09.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.560+0000 7f939b7fe640 1 -- 172.21.15.139:0/2987520206 >> v2:172.21.15.150:3300/0 conn(0x7f939c072280 msgr2=0x7f939c072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.560+0000 7f939b7fe640 1 --2- 172.21.15.139:0/2987520206 >> v2:172.21.15.150:3300/0 conn(0x7f939c072280 0x7f939c072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.560+0000 7f939b7fe640 1 -- 172.21.15.139:0/2987520206 >> v1:172.21.15.150:6789/0 conn(0x7f939c0b5110 legacy=0x7f939c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.560+0000 7f939b7fe640 1 -- 172.21.15.139:0/2987520206 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c0b5660 con 0x7f939c15aa00 2024-04-25T09:54:09.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.560+0000 7f939bfff640 1 --2- 172.21.15.139:0/2987520206 >> v2:172.21.15.150:3300/0 conn(0x7f939c072280 0x7f939c072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:09.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.561+0000 7f939b7fe640 1 --2- 172.21.15.139:0/2987520206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f939c15aa00 0x7f939c0b4090 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f938c009870 tx=0x7f938c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32ebba815894e577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.561+0000 7f939affd640 1 -- 172.21.15.139:0/2987520206 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c037070 con 0x7f939c15aa00 2024-04-25T09:54:09.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.561+0000 7f939affd640 1 -- 172.21.15.139:0/2987520206 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c032440 con 0x7f939c15aa00 2024-04-25T09:54:09.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.561+0000 7f939affd640 1 -- 172.21.15.139:0/2987520206 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c032810 con 0x7f939c15aa00 2024-04-25T09:54:09.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.561+0000 7f93a1c76640 1 -- 172.21.15.139:0/2987520206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f939c15aa00 msgr2=0x7f939c0b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.561+0000 7f93a1c76640 1 --2- 172.21.15.139:0/2987520206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f939c15aa00 0x7f939c0b4090 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f938c009870 tx=0x7f938c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.561+0000 7f93a1c76640 1 -- 172.21.15.139:0/2987520206 shutdown_connections 2024-04-25T09:54:09.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.561+0000 7f93a1c76640 1 --2- 172.21.15.139:0/2987520206 >> v2:172.21.15.150:3300/0 conn(0x7f939c072280 0x7f939c072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.561+0000 7f93a1c76640 1 --2- 172.21.15.139:0/2987520206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f939c15aa00 0x7f939c0b4090 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.561+0000 7f93a1c76640 1 -- 172.21.15.139:0/2987520206 >> 172.21.15.139:0/2987520206 conn(0x7f939c06e280 msgr2=0x7f939c06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.562+0000 7f93a1c76640 1 -- 172.21.15.139:0/2987520206 shutdown_connections 2024-04-25T09:54:09.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.562+0000 7f93a1c76640 1 -- 172.21.15.139:0/2987520206 wait complete. 2024-04-25T09:54:09.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.563+0000 7f93a1c76640 1 Processor -- start 2024-04-25T09:54:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.563+0000 7f93a1c76640 1 -- start start 2024-04-25T09:54:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.563+0000 7f93a1c76640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f939c072280 0x7f939c15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.563+0000 7f93a1c76640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f939c0b5110 0x7f939c151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.563+0000 7f93a1c76640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f939c152270 0x7f939c158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.563+0000 7f93a1c76640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c0ce610 con 0x7f939c152270 2024-04-25T09:54:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.563+0000 7f93a1c76640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c15e7a0 con 0x7f939c072280 2024-04-25T09:54:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.563+0000 7f93a1c76640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c0ce790 con 0x7f939c0b5110 2024-04-25T09:54:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f939bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f939c152270 0x7f939c158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f939bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f939c152270 0x7f939c158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49592/0 (socket says 172.21.15.139:49592) 2024-04-25T09:54:09.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f939bfff640 1 -- 172.21.15.139:0/3284715375 learned_addr learned my addr 172.21.15.139:0/3284715375 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f939b7fe640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f939c072280 0x7f939c15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f9393fff640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f939c0b5110 0x7f939c151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f939bfff640 1 -- 172.21.15.139:0/3284715375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f939c0b5110 msgr2=0x7f939c151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f939bfff640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f939c0b5110 0x7f939c151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f939bfff640 1 -- 172.21.15.139:0/3284715375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f939c072280 msgr2=0x7f939c15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f939bfff640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f939c072280 0x7f939c15ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f939bfff640 1 -- 172.21.15.139:0/3284715375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c152a40 con 0x7f939c152270 2024-04-25T09:54:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f939bfff640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f939c152270 0x7f939c158b30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f939400e160 tx=0x7f939400e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f9398ff9640 1 -- 172.21.15.139:0/3284715375 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939400b0d0 con 0x7f939c152270 2024-04-25T09:54:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f9398ff9640 1 -- 172.21.15.139:0/3284715375 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939400b270 con 0x7f939c152270 2024-04-25T09:54:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f93a1c76640 1 -- 172.21.15.139:0/3284715375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c158050 con 0x7f939c152270 2024-04-25T09:54:09.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f9398ff9640 1 -- 172.21.15.139:0/3284715375 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394003b90 con 0x7f939c152270 2024-04-25T09:54:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.564+0000 7f93a1c76640 1 -- 172.21.15.139:0/3284715375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c152700 con 0x7f939c152270 2024-04-25T09:54:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.565+0000 7f93a1c76640 1 -- 172.21.15.139:0/3284715375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939c074520 con 0x7f939c152270 2024-04-25T09:54:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.566+0000 7f9398ff9640 1 -- 172.21.15.139:0/3284715375 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f939401c020 con 0x7f939c152270 2024-04-25T09:54:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.566+0000 7f9398ff9640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f937c06b4d0 0x7f937c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.566+0000 7f9398ff9640 1 -- 172.21.15.139:0/3284715375 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f939408c5d0 con 0x7f939c152270 2024-04-25T09:54:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.566+0000 7f939b7fe640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f937c06b4d0 0x7f937c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.567+0000 7f939b7fe640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f937c06b4d0 0x7f937c06d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f938c034000 tx=0x7f938c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.570+0000 7f9398ff9640 1 -- 172.21.15.139:0/3284715375 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9394012040 con 0x7f939c152270 2024-04-25T09:54:09.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.741+0000 7f93a1c76640 1 -- 172.21.15.139:0/3284715375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f939c06f180 con 0x7f939c152270 2024-04-25T09:54:09.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.742+0000 7f9398ff9640 1 -- 172.21.15.139:0/3284715375 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18951 (secure 0 0 0) 0x7f939405aa90 con 0x7f939c152270 2024-04-25T09:54:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:09.747 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":26,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:56.839903+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:09.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.744+0000 7f93a1c76640 1 -- 172.21.15.139:0/3284715375 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f937c06b4d0 msgr2=0x7f937c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.744+0000 7f93a1c76640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f937c06b4d0 0x7f937c06d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f938c034000 tx=0x7f938c02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:09.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.744+0000 7f93a1c76640 1 -- 172.21.15.139:0/3284715375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f939c152270 msgr2=0x7f939c158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.744+0000 7f93a1c76640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f939c152270 0x7f939c158b30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f939400e160 tx=0x7f939400e620 comp rx=0 tx=0).stop 2024-04-25T09:54:09.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.744+0000 7f93a1c76640 1 -- 172.21.15.139:0/3284715375 shutdown_connections 2024-04-25T09:54:09.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.744+0000 7f93a1c76640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f939c152270 0x7f939c158b30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.744+0000 7f93a1c76640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f937c06b4d0 0x7f937c06d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.744+0000 7f93a1c76640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f939c0b5110 0x7f939c151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.744+0000 7f93a1c76640 1 --2- 172.21.15.139:0/3284715375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f939c072280 0x7f939c15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.744+0000 7f93a1c76640 1 -- 172.21.15.139:0/3284715375 >> 172.21.15.139:0/3284715375 conn(0x7f939c06e280 msgr2=0x7f939c06fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.744+0000 7f93a1c76640 1 -- 172.21.15.139:0/3284715375 shutdown_connections 2024-04-25T09:54:09.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.744+0000 7f93a1c76640 1 -- 172.21.15.139:0/3284715375 wait complete. 2024-04-25T09:54:09.758 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:54:09.758 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:09.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.851+0000 7fdc7461f640 1 Processor -- start 2024-04-25T09:54:09.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.852+0000 7fdc7461f640 1 -- start start 2024-04-25T09:54:09.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.852+0000 7fdc7461f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc6c0b0750 0x7fdc6c0b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.852+0000 7fdc7461f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdc6c0d0730 0x7fdc6c0cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc7461f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdc6c0d0b90 0x7fdc6c0cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc7461f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6c0b1070 con 0x7fdc6c0d0730 2024-04-25T09:54:09.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc7461f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6c0cc8c0 con 0x7fdc6c0d0b90 2024-04-25T09:54:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc7461f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0ccb70 con 0x7fdc6c0b0750 2024-04-25T09:54:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc71bba640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdc6c0d0730 0x7fdc6c0cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc71bba640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdc6c0d0730 0x7fdc6c0cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49606/0 (socket says 172.21.15.139:49606) 2024-04-25T09:54:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc71bba640 1 -- 172.21.15.139:0/3037763877 learned_addr learned my addr 172.21.15.139:0/3037763877 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc723bb640 1 --2- 172.21.15.139:0/3037763877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc6c0b0750 0x7fdc6c0b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc71bba640 1 -- 172.21.15.139:0/3037763877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc6c0b0750 msgr2=0x7fdc6c0b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc71bba640 1 --2- 172.21.15.139:0/3037763877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc6c0b0750 0x7fdc6c0b0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc72bbc640 1 --2- 172.21.15.139:0/3037763877 >> v2:172.21.15.150:3300/0 conn(0x7fdc6c0d0b90 0x7fdc6c0cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc71bba640 1 -- 172.21.15.139:0/3037763877 >> v2:172.21.15.150:3300/0 conn(0x7fdc6c0d0b90 msgr2=0x7fdc6c0cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc71bba640 1 --2- 172.21.15.139:0/3037763877 >> v2:172.21.15.150:3300/0 conn(0x7fdc6c0d0b90 0x7fdc6c0cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.853+0000 7fdc71bba640 1 -- 172.21.15.139:0/3037763877 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c0cf970 con 0x7fdc6c0d0730 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc71bba640 1 --2- 172.21.15.139:0/3037763877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c0d0730 0x7fdc6c0cbe40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0cf220 tx=0x7fdc5c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa80dd3c5a9360f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc713b9640 1 -- 172.21.15.139:0/3037763877 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c037070 con 0x7fdc6c0d0730 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc713b9640 1 -- 172.21.15.139:0/3037763877 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5c032440 con 0x7fdc6c0d0730 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc713b9640 1 -- 172.21.15.139:0/3037763877 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c032810 con 0x7fdc6c0d0730 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc7461f640 1 -- 172.21.15.139:0/3037763877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c0d0730 msgr2=0x7fdc6c0cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc7461f640 1 --2- 172.21.15.139:0/3037763877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c0d0730 0x7fdc6c0cbe40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0cf220 tx=0x7fdc5c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc7461f640 1 -- 172.21.15.139:0/3037763877 shutdown_connections 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc7461f640 1 --2- 172.21.15.139:0/3037763877 >> v2:172.21.15.150:3300/0 conn(0x7fdc6c0d0b90 0x7fdc6c0cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc7461f640 1 --2- 172.21.15.139:0/3037763877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c0d0730 0x7fdc6c0cbe40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc7461f640 1 --2- 172.21.15.139:0/3037763877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc6c0b0750 0x7fdc6c0b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc7461f640 1 -- 172.21.15.139:0/3037763877 >> 172.21.15.139:0/3037763877 conn(0x7fdc6c0b3380 msgr2=0x7fdc6c0b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc7461f640 1 -- 172.21.15.139:0/3037763877 shutdown_connections 2024-04-25T09:54:09.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.854+0000 7fdc7461f640 1 -- 172.21.15.139:0/3037763877 wait complete. 2024-04-25T09:54:09.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.856+0000 7fdc7461f640 1 Processor -- start 2024-04-25T09:54:09.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.856+0000 7fdc7461f640 1 -- start start 2024-04-25T09:54:09.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.856+0000 7fdc7461f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc6c0b0750 0x7fdc6c1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.856+0000 7fdc7461f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc6c0d0730 0x7fdc6c151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.856+0000 7fdc7461f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c0c7ee0 0x7fdc6c1f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.856+0000 7fdc7461f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0d0210 con 0x7fdc6c0c7ee0 2024-04-25T09:54:09.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.856+0000 7fdc7461f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0cff30 con 0x7fdc6c0b0750 2024-04-25T09:54:09.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.856+0000 7fdc7461f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c15bcc0 con 0x7fdc6c0d0730 2024-04-25T09:54:09.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.856+0000 7fdc72bbc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c0c7ee0 0x7fdc6c1f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.856+0000 7fdc72bbc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c0c7ee0 0x7fdc6c1f17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49620/0 (socket says 172.21.15.139:49620) 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.856+0000 7fdc72bbc640 1 -- 172.21.15.139:0/3171758333 learned_addr learned my addr 172.21.15.139:0/3171758333 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc71bba640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc6c0d0730 0x7fdc6c151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc723bb640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc6c0b0750 0x7fdc6c1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc72bbc640 1 -- 172.21.15.139:0/3171758333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc6c0d0730 msgr2=0x7fdc6c151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc72bbc640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc6c0d0730 0x7fdc6c151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc72bbc640 1 -- 172.21.15.139:0/3171758333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc6c0b0750 msgr2=0x7fdc6c1549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc72bbc640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc6c0b0750 0x7fdc6c1549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc72bbc640 1 -- 172.21.15.139:0/3171758333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c1521d0 con 0x7fdc6c0c7ee0 2024-04-25T09:54:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc71bba640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc6c0d0730 0x7fdc6c151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc72bbc640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c0c7ee0 0x7fdc6c1f17a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800c550 tx=0x7fdc6800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc62ffd640 1 -- 172.21.15.139:0/3171758333 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68015070 con 0x7fdc6c0c7ee0 2024-04-25T09:54:09.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc62ffd640 1 -- 172.21.15.139:0/3171758333 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc68004220 con 0x7fdc6c0c7ee0 2024-04-25T09:54:09.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc62ffd640 1 -- 172.21.15.139:0/3171758333 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68003610 con 0x7fdc6c0c7ee0 2024-04-25T09:54:09.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc7461f640 1 -- 172.21.15.139:0/3171758333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c0c8370 con 0x7fdc6c0c7ee0 2024-04-25T09:54:09.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.857+0000 7fdc7461f640 1 -- 172.21.15.139:0/3171758333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c0ca340 con 0x7fdc6c0c7ee0 2024-04-25T09:54:09.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.858+0000 7fdc7461f640 1 -- 172.21.15.139:0/3171758333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c074520 con 0x7fdc6c0c7ee0 2024-04-25T09:54:09.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.858+0000 7fdc62ffd640 1 -- 172.21.15.139:0/3171758333 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc680037b0 con 0x7fdc6c0c7ee0 2024-04-25T09:54:09.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.859+0000 7fdc62ffd640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdc5006b4d0 0x7fdc5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.859+0000 7fdc62ffd640 1 -- 172.21.15.139:0/3171758333 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fdc6808a2e0 con 0x7fdc6c0c7ee0 2024-04-25T09:54:09.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.859+0000 7fdc723bb640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdc5006b4d0 0x7fdc5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.859+0000 7fdc723bb640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdc5006b4d0 0x7fdc5006d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdc64007fd0 tx=0x7fdc64007560 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:09.863+0000 7fdc62ffd640 1 -- 172.21.15.139:0/3171758333 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc68058df0 con 0x7fdc6c0c7ee0 2024-04-25T09:54:10.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.072+0000 7fdc7461f640 1 -- 172.21.15.139:0/3171758333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc6c0b3000 con 0x7fdc6c0c7ee0 2024-04-25T09:54:10.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.073+0000 7fdc62ffd640 1 -- 172.21.15.139:0/3171758333 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fdc680586f0 con 0x7fdc6c0c7ee0 2024-04-25T09:54:10.075 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:10.075 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/892226922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":892226922},{"type":"v1","addr":"172.21.15.150:6834","nonce":892226922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/582019435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":582019435},{"type":"v1","addr":"172.21.15.139:6837","nonce":582019435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/2751714076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2751714076},{"type":"v1","addr":"172.21.15.139:6834","nonce":2751714076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:15.042067+0000","modified":"2024-04-25T09:53:30.094430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2160557452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2160557452},{"type":"v1","addr":"172.21.15.150:6835","nonce":2160557452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:54:10.075 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 9 2024-04-25T09:54:10.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.074+0000 7fdc7461f640 1 -- 172.21.15.139:0/3171758333 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdc5006b4d0 msgr2=0x7fdc5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.074+0000 7fdc7461f640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdc5006b4d0 0x7fdc5006d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdc64007fd0 tx=0x7fdc64007560 comp rx=0 tx=0).stop 2024-04-25T09:54:10.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.074+0000 7fdc7461f640 1 -- 172.21.15.139:0/3171758333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c0c7ee0 msgr2=0x7fdc6c1f17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.074+0000 7fdc7461f640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c0c7ee0 0x7fdc6c1f17a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800c550 tx=0x7fdc6800ca10 comp rx=0 tx=0).stop 2024-04-25T09:54:10.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.074+0000 7fdc7461f640 1 -- 172.21.15.139:0/3171758333 shutdown_connections 2024-04-25T09:54:10.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.075+0000 7fdc7461f640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc6c0c7ee0 0x7fdc6c1f17a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.075+0000 7fdc7461f640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdc5006b4d0 0x7fdc5006d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.075+0000 7fdc7461f640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc6c0d0730 0x7fdc6c151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.075+0000 7fdc7461f640 1 --2- 172.21.15.139:0/3171758333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc6c0b0750 0x7fdc6c1549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.075+0000 7fdc7461f640 1 -- 172.21.15.139:0/3171758333 >> 172.21.15.139:0/3171758333 conn(0x7fdc6c0b3380 msgr2=0x7fdc6c0b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:10.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.075+0000 7fdc7461f640 1 -- 172.21.15.139:0/3171758333 shutdown_connections 2024-04-25T09:54:10.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.075+0000 7fdc7461f640 1 -- 172.21.15.139:0/3171758333 wait complete. 2024-04-25T09:54:10.088 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:10.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.182+0000 7fe17d658640 1 Processor -- start 2024-04-25T09:54:10.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.183+0000 7fe17d658640 1 -- start start 2024-04-25T09:54:10.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.183+0000 7fe17d658640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe17815dc60 0x7fe1780b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.183+0000 7fe17d658640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1780d1080 0x7fe1780ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.183+0000 7fe17d658640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1780cf250 con 0x7fe178161850 2024-04-25T09:54:10.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.183+0000 7fe17d658640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1780ce780 con 0x7fe17815dc60 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.183+0000 7fe17d658640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1780d3730 con 0x7fe1780d1080 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.183+0000 7fe1767fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe178161850 0x7fe1780d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59080/0 (socket says 172.21.15.139:59080) 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.183+0000 7fe1767fc640 1 -- 172.21.15.139:0/2643784358 learned_addr learned my addr 172.21.15.139:0/2643784358 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.184+0000 7fe1777fe640 1 --2- 172.21.15.139:0/2643784358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1780d1080 0x7fe1780ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.184+0000 7fe176ffd640 1 --2- 172.21.15.139:0/2643784358 >> v2:172.21.15.150:3300/0 conn(0x7fe17815dc60 0x7fe1780b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.184+0000 7fe175ffb640 1 -- 172.21.15.139:0/2643784358 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238804957 0 0) 0x7fe16c001110 con 0x7fe178161850 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.184+0000 7fe175ffb640 1 -- 172.21.15.139:0/2643784358 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe160003520 con 0x7fe178161850 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.184+0000 7fe175ffb640 1 -- 172.21.15.139:0/2643784358 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4015680248 0 0) 0x7fe160003520 con 0x7fe178161850 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.184+0000 7fe176ffd640 1 -- 172.21.15.139:0/2643784358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1780d1080 msgr2=0x7fe1780ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.184+0000 7fe176ffd640 1 --2- 172.21.15.139:0/2643784358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1780d1080 0x7fe1780ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.184+0000 7fe176ffd640 1 -- 172.21.15.139:0/2643784358 >> v1:172.21.15.139:6789/0 conn(0x7fe178161850 legacy=0x7fe1780d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.184+0000 7fe176ffd640 1 -- 172.21.15.139:0/2643784358 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1780ce9b0 con 0x7fe17815dc60 2024-04-25T09:54:10.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.184+0000 7fe1777fe640 1 --2- 172.21.15.139:0/2643784358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1780d1080 0x7fe1780ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:10.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.185+0000 7fe176ffd640 1 --2- 172.21.15.139:0/2643784358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe17815dc60 0x7fe1780b6c10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe168009d90 tx=0x7fe16802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bfb90f891e91a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.185+0000 7fe175ffb640 1 -- 172.21.15.139:0/2643784358 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c001840 con 0x7fe17815dc60 2024-04-25T09:54:10.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.185+0000 7fe175ffb640 1 -- 172.21.15.139:0/2643784358 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe168007540 con 0x7fe17815dc60 2024-04-25T09:54:10.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.185+0000 7fe175ffb640 1 -- 172.21.15.139:0/2643784358 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1680078f0 con 0x7fe17815dc60 2024-04-25T09:54:10.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.186+0000 7fe17d658640 1 -- 172.21.15.139:0/2643784358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe17815dc60 msgr2=0x7fe1780b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.186+0000 7fe17d658640 1 --2- 172.21.15.139:0/2643784358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe17815dc60 0x7fe1780b6c10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe168009d90 tx=0x7fe16802a5f0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.186+0000 7fe17d658640 1 -- 172.21.15.139:0/2643784358 shutdown_connections 2024-04-25T09:54:10.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.186+0000 7fe17d658640 1 --2- 172.21.15.139:0/2643784358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1780d1080 0x7fe1780ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.186+0000 7fe17d658640 1 --2- 172.21.15.139:0/2643784358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe17815dc60 0x7fe1780b6c10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.186+0000 7fe17d658640 1 -- 172.21.15.139:0/2643784358 >> 172.21.15.139:0/2643784358 conn(0x7fe1780b5910 msgr2=0x7fe1780b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:10.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.186+0000 7fe17d658640 1 -- 172.21.15.139:0/2643784358 shutdown_connections 2024-04-25T09:54:10.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.186+0000 7fe17d658640 1 -- 172.21.15.139:0/2643784358 wait complete. 2024-04-25T09:54:10.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.187+0000 7fe17d658640 1 Processor -- start 2024-04-25T09:54:10.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.187+0000 7fe17d658640 1 -- start start 2024-04-25T09:54:10.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe17d658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1780d1080 0x7fe178148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe17d658640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe17815dc60 0x7fe1780cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe17d658640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe178161850 0x7fe1780b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe17d658640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1780d1fc0 con 0x7fe1780d1080 2024-04-25T09:54:10.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe17d658640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1780cef70 con 0x7fe178161850 2024-04-25T09:54:10.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe17d658640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17815e040 con 0x7fe17815dc60 2024-04-25T09:54:10.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe176ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1780d1080 0x7fe178148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe176ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1780d1080 0x7fe178148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49624/0 (socket says 172.21.15.139:49624) 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe176ffd640 1 -- 172.21.15.139:0/3427247534 learned_addr learned my addr 172.21.15.139:0/3427247534 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe1777fe640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe178161850 0x7fe1780b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe1767fc640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe17815dc60 0x7fe1780cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe176ffd640 1 -- 172.21.15.139:0/3427247534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe17815dc60 msgr2=0x7fe1780cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe176ffd640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe17815dc60 0x7fe1780cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe176ffd640 1 -- 172.21.15.139:0/3427247534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe178161850 msgr2=0x7fe1780b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe176ffd640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe178161850 0x7fe1780b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe176ffd640 1 -- 172.21.15.139:0/3427247534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17815e680 con 0x7fe1780d1080 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe1777fe640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe178161850 0x7fe1780b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe176ffd640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1780d1080 0x7fe178148db0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe168009eb0 tx=0x7fe168004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe15f7fe640 1 -- 172.21.15.139:0/3427247534 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168040070 con 0x7fe1780d1080 2024-04-25T09:54:10.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe15f7fe640 1 -- 172.21.15.139:0/3427247534 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe168031c40 con 0x7fe1780d1080 2024-04-25T09:54:10.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe17d658640 1 -- 172.21.15.139:0/3427247534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1781492f0 con 0x7fe1780d1080 2024-04-25T09:54:10.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe15f7fe640 1 -- 172.21.15.139:0/3427247534 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168009450 con 0x7fe1780d1080 2024-04-25T09:54:10.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.188+0000 7fe17d658640 1 -- 172.21.15.139:0/3427247534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1780cc110 con 0x7fe1780d1080 2024-04-25T09:54:10.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.189+0000 7fe17d658640 1 -- 172.21.15.139:0/3427247534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe178074520 con 0x7fe1780d1080 2024-04-25T09:54:10.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.190+0000 7fe15f7fe640 1 -- 172.21.15.139:0/3427247534 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe168039040 con 0x7fe1780d1080 2024-04-25T09:54:10.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.190+0000 7fe15f7fe640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe16406b4d0 0x7fe16406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.190+0000 7fe15f7fe640 1 -- 172.21.15.139:0/3427247534 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe1680aba20 con 0x7fe1780d1080 2024-04-25T09:54:10.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.190+0000 7fe1767fc640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe16406b4d0 0x7fe16406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.191+0000 7fe1767fc640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe16406b4d0 0x7fe16406d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe1780b7820 tx=0x7fe16c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.194+0000 7fe15f7fe640 1 -- 172.21.15.139:0/3427247534 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe17815e680 con 0x7fe1780d1080 2024-04-25T09:54:10.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.399+0000 7fe17d658640 1 -- 172.21.15.139:0/3427247534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe17815d280 con 0x7fe1780d1080 2024-04-25T09:54:10.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.399+0000 7fe15f7fe640 1 -- 172.21.15.139:0/3427247534 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fe16807a4a0 con 0x7fe1780d1080 2024-04-25T09:54:10.401 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:10.402 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/892226922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":892226922},{"type":"v1","addr":"172.21.15.150:6834","nonce":892226922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/582019435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":582019435},{"type":"v1","addr":"172.21.15.139:6837","nonce":582019435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/2751714076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2751714076},{"type":"v1","addr":"172.21.15.139:6834","nonce":2751714076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:53:15.042067+0000","modified":"2024-04-25T09:53:30.094430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2160557452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2160557452},{"type":"v1","addr":"172.21.15.150:6835","nonce":2160557452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:54:10.402 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 9 2024-04-25T09:54:10.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.401+0000 7fe17d658640 1 -- 172.21.15.139:0/3427247534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe16406b4d0 msgr2=0x7fe16406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.401+0000 7fe17d658640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe16406b4d0 0x7fe16406d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe1780b7820 tx=0x7fe16c009040 comp rx=0 tx=0).stop 2024-04-25T09:54:10.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.401+0000 7fe17d658640 1 -- 172.21.15.139:0/3427247534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1780d1080 msgr2=0x7fe178148db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.401+0000 7fe17d658640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1780d1080 0x7fe178148db0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe168009eb0 tx=0x7fe168004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:10.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.401+0000 7fe17d658640 1 -- 172.21.15.139:0/3427247534 shutdown_connections 2024-04-25T09:54:10.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.401+0000 7fe17d658640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe16406b4d0 0x7fe16406d990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.401+0000 7fe17d658640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe178161850 0x7fe1780b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.401+0000 7fe17d658640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe17815dc60 0x7fe1780cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.401+0000 7fe17d658640 1 --2- 172.21.15.139:0/3427247534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1780d1080 0x7fe178148db0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.401+0000 7fe17d658640 1 -- 172.21.15.139:0/3427247534 >> 172.21.15.139:0/3427247534 conn(0x7fe1780b5910 msgr2=0x7fe178158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:10.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.401+0000 7fe17d658640 1 -- 172.21.15.139:0/3427247534 shutdown_connections 2024-04-25T09:54:10.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.401+0000 7fe17d658640 1 -- 172.21.15.139:0/3427247534 wait complete. 2024-04-25T09:54:10.415 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:10.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.509+0000 7fb3ead53640 1 Processor -- start 2024-04-25T09:54:10.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.509+0000 7fb3ead53640 1 -- start start 2024-04-25T09:54:10.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3ead53640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb3e415b950 0x7fb3e4072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3ead53640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb3e40d0970 0x7fb3e406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3ead53640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e4070a40 con 0x7fb3e406ab60 2024-04-25T09:54:10.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3ead53640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e406b500 con 0x7fb3e415b950 2024-04-25T09:54:10.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3ead53640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e406fc80 con 0x7fb3e40d0970 2024-04-25T09:54:10.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3e8aef640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb3e415b950 0x7fb3e4072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3e8aef640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb3e415b950 0x7fb3e4072a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49636/0 (socket says 172.21.15.139:49636) 2024-04-25T09:54:10.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3e8aef640 1 -- 172.21.15.139:0/3641397077 learned_addr learned my addr 172.21.15.139:0/3641397077 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:10.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3e92f0640 1 --2- 172.21.15.139:0/3641397077 >> v2:172.21.15.150:3300/0 conn(0x7fb3e40d0970 0x7fb3e406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3e8aef640 1 -- 172.21.15.139:0/3641397077 >> v2:172.21.15.150:3300/0 conn(0x7fb3e40d0970 msgr2=0x7fb3e406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:10.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3e8aef640 1 --2- 172.21.15.139:0/3641397077 >> v2:172.21.15.150:3300/0 conn(0x7fb3e40d0970 0x7fb3e406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3e8aef640 1 -- 172.21.15.139:0/3641397077 >> v1:172.21.15.150:6789/0 conn(0x7fb3e406ab60 legacy=0x7fb3e415f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.510+0000 7fb3e8aef640 1 -- 172.21.15.139:0/3641397077 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e406ff60 con 0x7fb3e415b950 2024-04-25T09:54:10.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.511+0000 7fb3e8aef640 1 --2- 172.21.15.139:0/3641397077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e415b950 0x7fb3e4072a60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb3e406bab0 tx=0x7fb3d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f3ff05b609d9f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.511+0000 7fb3e37fe640 1 -- 172.21.15.139:0/3641397077 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4037070 con 0x7fb3e415b950 2024-04-25T09:54:10.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.511+0000 7fb3e37fe640 1 -- 172.21.15.139:0/3641397077 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d4032440 con 0x7fb3e415b950 2024-04-25T09:54:10.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.511+0000 7fb3e37fe640 1 -- 172.21.15.139:0/3641397077 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4032810 con 0x7fb3e415b950 2024-04-25T09:54:10.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.512+0000 7fb3ead53640 1 -- 172.21.15.139:0/3641397077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e415b950 msgr2=0x7fb3e4072a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.512+0000 7fb3ead53640 1 --2- 172.21.15.139:0/3641397077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e415b950 0x7fb3e4072a60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb3e406bab0 tx=0x7fb3d402a270 comp rx=0 tx=0).stop 2024-04-25T09:54:10.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.512+0000 7fb3ead53640 1 -- 172.21.15.139:0/3641397077 shutdown_connections 2024-04-25T09:54:10.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.512+0000 7fb3ead53640 1 --2- 172.21.15.139:0/3641397077 >> v2:172.21.15.150:3300/0 conn(0x7fb3e40d0970 0x7fb3e406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.512+0000 7fb3ead53640 1 --2- 172.21.15.139:0/3641397077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e415b950 0x7fb3e4072a60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.512+0000 7fb3ead53640 1 -- 172.21.15.139:0/3641397077 >> 172.21.15.139:0/3641397077 conn(0x7fb3e40b23f0 msgr2=0x7fb3e40b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:10.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.512+0000 7fb3ead53640 1 -- 172.21.15.139:0/3641397077 shutdown_connections 2024-04-25T09:54:10.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.512+0000 7fb3ead53640 1 -- 172.21.15.139:0/3641397077 wait complete. 2024-04-25T09:54:10.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.513+0000 7fb3ead53640 1 Processor -- start 2024-04-25T09:54:10.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.513+0000 7fb3ead53640 1 -- start start 2024-04-25T09:54:10.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3ead53640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e406ab60 0x7fb3e41f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3ead53640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3e40d0970 0x7fb3e414cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3ead53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e415b950 0x7fb3e414f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3ead53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e40cdce0 con 0x7fb3e415b950 2024-04-25T09:54:10.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3ead53640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e40cda00 con 0x7fb3e406ab60 2024-04-25T09:54:10.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3ead53640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e40cdfc0 con 0x7fb3e40d0970 2024-04-25T09:54:10.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3e92f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e415b950 0x7fb3e414f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3e92f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e415b950 0x7fb3e414f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49646/0 (socket says 172.21.15.139:49646) 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3e92f0640 1 -- 172.21.15.139:0/2801751879 learned_addr learned my addr 172.21.15.139:0/2801751879 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3e8aef640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e406ab60 0x7fb3e41f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3e3fff640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3e40d0970 0x7fb3e414cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3e92f0640 1 -- 172.21.15.139:0/2801751879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3e40d0970 msgr2=0x7fb3e414cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3e92f0640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3e40d0970 0x7fb3e414cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3e92f0640 1 -- 172.21.15.139:0/2801751879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e406ab60 msgr2=0x7fb3e41f2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3e92f0640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e406ab60 0x7fb3e41f2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3e92f0640 1 -- 172.21.15.139:0/2801751879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e414fcc0 con 0x7fb3e415b950 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3e3fff640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3e40d0970 0x7fb3e414cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.514+0000 7fb3e92f0640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e415b950 0x7fb3e414f320 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc007e30 tx=0x7fb3dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.515+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2801751879 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc014620 con 0x7fb3e415b950 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.515+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2801751879 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc019070 con 0x7fb3e415b950 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.515+0000 7fb3ead53640 1 -- 172.21.15.139:0/2801751879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e41f1690 con 0x7fb3e415b950 2024-04-25T09:54:10.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.515+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2801751879 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc014a00 con 0x7fb3e415b950 2024-04-25T09:54:10.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.515+0000 7fb3ead53640 1 -- 172.21.15.139:0/2801751879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e41f1c50 con 0x7fb3e415b950 2024-04-25T09:54:10.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.516+0000 7fb3ead53640 1 -- 172.21.15.139:0/2801751879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e4074520 con 0x7fb3e415b950 2024-04-25T09:54:10.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.516+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2801751879 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3dc014ba0 con 0x7fb3e415b950 2024-04-25T09:54:10.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.516+0000 7fb3e17fa640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb3d006b4d0 0x7fb3d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.516+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2801751879 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb3dc08aef0 con 0x7fb3e415b950 2024-04-25T09:54:10.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.516+0000 7fb3e8aef640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb3d006b4d0 0x7fb3d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.517+0000 7fb3e8aef640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb3d006b4d0 0x7fb3d006d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4034000 tx=0x7fb3d402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.521+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2801751879 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3dc059970 con 0x7fb3e415b950 2024-04-25T09:54:10.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.688+0000 7fb3ead53640 1 -- 172.21.15.139:0/2801751879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3e40d05a0 con 0x7fb3e415b950 2024-04-25T09:54:10.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.690+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2801751879 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18951 (secure 0 0 0) 0x7fb3dc059270 con 0x7fb3e415b950 2024-04-25T09:54:10.692 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:10.692 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":26,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:53:56.839903+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:53:12.914187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:53:14.014283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:53:17.745984+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:10.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.692+0000 7fb3ead53640 1 -- 172.21.15.139:0/2801751879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb3d006b4d0 msgr2=0x7fb3d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.692+0000 7fb3ead53640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb3d006b4d0 0x7fb3d006d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4034000 tx=0x7fb3d402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:10.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.692+0000 7fb3ead53640 1 -- 172.21.15.139:0/2801751879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e415b950 msgr2=0x7fb3e414f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.692+0000 7fb3ead53640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e415b950 0x7fb3e414f320 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc007e30 tx=0x7fb3dc00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.692+0000 7fb3ead53640 1 -- 172.21.15.139:0/2801751879 shutdown_connections 2024-04-25T09:54:10.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.692+0000 7fb3ead53640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e415b950 0x7fb3e414f320 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.692+0000 7fb3ead53640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb3d006b4d0 0x7fb3d006d990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.692+0000 7fb3ead53640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3e40d0970 0x7fb3e414cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.692+0000 7fb3ead53640 1 --2- 172.21.15.139:0/2801751879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3e406ab60 0x7fb3e41f2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.692+0000 7fb3ead53640 1 -- 172.21.15.139:0/2801751879 >> 172.21.15.139:0/2801751879 conn(0x7fb3e40b23f0 msgr2=0x7fb3e4157dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:10.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.692+0000 7fb3ead53640 1 -- 172.21.15.139:0/2801751879 shutdown_connections 2024-04-25T09:54:10.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.692+0000 7fb3ead53640 1 -- 172.21.15.139:0/2801751879 wait complete. 2024-04-25T09:54:10.705 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:54:10.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.800+0000 7fc01b1d5640 1 Processor -- start 2024-04-25T09:54:10.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.800+0000 7fc01b1d5640 1 -- start start 2024-04-25T09:54:10.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.801+0000 7fc01b1d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc01415dbf0 0x7fc0141617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.801+0000 7fc01b1d5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc0140b9370 0x7fc0140ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.801+0000 7fc01b1d5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc0140cec80 0x7fc01406c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.801+0000 7fc01b1d5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0140d1ee0 con 0x7fc0140b9370 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.801+0000 7fc01b1d5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc014161dc0 con 0x7fc0140cec80 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.801+0000 7fc01b1d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0141620a0 con 0x7fc01415dbf0 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.801+0000 7fc013fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc0140b9370 0x7fc0140ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.801+0000 7fc013fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc0140b9370 0x7fc0140ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49648/0 (socket says 172.21.15.139:49648) 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.801+0000 7fc013fff640 1 -- 172.21.15.139:0/704754410 learned_addr learned my addr 172.21.15.139:0/704754410 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.802+0000 7fc013fff640 1 -- 172.21.15.139:0/704754410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc01415dbf0 msgr2=0x7fc0141617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.802+0000 7fc013fff640 1 --2- 172.21.15.139:0/704754410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc01415dbf0 0x7fc0141617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.802+0000 7fc013fff640 1 -- 172.21.15.139:0/704754410 >> v2:172.21.15.150:3300/0 conn(0x7fc0140cec80 msgr2=0x7fc01406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.802+0000 7fc019772640 1 --2- 172.21.15.139:0/704754410 >> v2:172.21.15.150:3300/0 conn(0x7fc0140cec80 0x7fc01406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.802+0000 7fc013fff640 1 --2- 172.21.15.139:0/704754410 >> v2:172.21.15.150:3300/0 conn(0x7fc0140cec80 0x7fc01406c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.802+0000 7fc013fff640 1 -- 172.21.15.139:0/704754410 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc014162380 con 0x7fc0140b9370 2024-04-25T09:54:10.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.802+0000 7fc013fff640 1 --2- 172.21.15.139:0/704754410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0140b9370 0x7fc0140ce740 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc01408c170 tx=0x7fc00402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9563a45e2bec4399 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.803+0000 7fc0137fe640 1 -- 172.21.15.139:0/704754410 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004036070 con 0x7fc0140b9370 2024-04-25T09:54:10.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.803+0000 7fc0137fe640 1 -- 172.21.15.139:0/704754410 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc004031600 con 0x7fc0140b9370 2024-04-25T09:54:10.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.803+0000 7fc0137fe640 1 -- 172.21.15.139:0/704754410 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0040319d0 con 0x7fc0140b9370 2024-04-25T09:54:10.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.803+0000 7fc01b1d5640 1 -- 172.21.15.139:0/704754410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0140b9370 msgr2=0x7fc0140ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.803+0000 7fc01b1d5640 1 --2- 172.21.15.139:0/704754410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0140b9370 0x7fc0140ce740 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc01408c170 tx=0x7fc00402a6a0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.803+0000 7fc01b1d5640 1 -- 172.21.15.139:0/704754410 shutdown_connections 2024-04-25T09:54:10.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.803+0000 7fc01b1d5640 1 --2- 172.21.15.139:0/704754410 >> v2:172.21.15.150:3300/0 conn(0x7fc0140cec80 0x7fc01406c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.803+0000 7fc01b1d5640 1 --2- 172.21.15.139:0/704754410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0140b9370 0x7fc0140ce740 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.803+0000 7fc01b1d5640 1 --2- 172.21.15.139:0/704754410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc01415dbf0 0x7fc0141617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.803+0000 7fc01b1d5640 1 -- 172.21.15.139:0/704754410 >> 172.21.15.139:0/704754410 conn(0x7fc0140b5480 msgr2=0x7fc0140b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:10.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.803+0000 7fc01b1d5640 1 -- 172.21.15.139:0/704754410 shutdown_connections 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.803+0000 7fc01b1d5640 1 -- 172.21.15.139:0/704754410 wait complete. 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.804+0000 7fc01b1d5640 1 Processor -- start 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.804+0000 7fc01b1d5640 1 -- start start 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.805+0000 7fc01b1d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0140b9370 0x7fc0140c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.805+0000 7fc01b1d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0140cec80 0x7fc014145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.805+0000 7fc01b1d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc01415dbf0 0x7fc0141442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.805+0000 7fc01b1d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01415dfd0 con 0x7fc01415dbf0 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.805+0000 7fc01b1d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140b97d0 con 0x7fc0140b9370 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.805+0000 7fc01b1d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140ca360 con 0x7fc0140cec80 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.805+0000 7fc019772640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc01415dbf0 0x7fc0141442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.805+0000 7fc019772640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc01415dbf0 0x7fc0141442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49662/0 (socket says 172.21.15.139:49662) 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.805+0000 7fc019772640 1 -- 172.21.15.139:0/2702380403 learned_addr learned my addr 172.21.15.139:0/2702380403 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.805+0000 7fc013fff640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0140cec80 0x7fc014145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc018f71640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0140b9370 0x7fc0140c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc019772640 1 -- 172.21.15.139:0/2702380403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0140cec80 msgr2=0x7fc014145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc019772640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0140cec80 0x7fc014145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc019772640 1 -- 172.21.15.139:0/2702380403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0140b9370 msgr2=0x7fc0140c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc019772640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0140b9370 0x7fc0140c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc019772640 1 -- 172.21.15.139:0/2702380403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc014145c90 con 0x7fc01415dbf0 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc018f71640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0140b9370 0x7fc0140c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc019772640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc01415dbf0 0x7fc0141442d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00c550 tx=0x7fbffc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc0117fa640 1 -- 172.21.15.139:0/2702380403 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc015070 con 0x7fc01415dbf0 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc0117fa640 1 -- 172.21.15.139:0/2702380403 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc004220 con 0x7fc01415dbf0 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc01b1d5640 1 -- 172.21.15.139:0/2702380403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc014143230 con 0x7fc01415dbf0 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc0117fa640 1 -- 172.21.15.139:0/2702380403 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc003610 con 0x7fc01415dbf0 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.806+0000 7fc01b1d5640 1 -- 172.21.15.139:0/2702380403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0141437f0 con 0x7fc01415dbf0 2024-04-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.807+0000 7fc01b1d5640 1 -- 172.21.15.139:0/2702380403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc014074520 con 0x7fc01415dbf0 2024-04-25T09:54:10.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.807+0000 7fc0117fa640 1 -- 172.21.15.139:0/2702380403 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbffc0037b0 con 0x7fc01415dbf0 2024-04-25T09:54:10.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.808+0000 7fc0117fa640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbff406b4d0 0x7fbff406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:10.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.808+0000 7fc0117fa640 1 -- 172.21.15.139:0/2702380403 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbffc08a2e0 con 0x7fc01415dbf0 2024-04-25T09:54:10.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.808+0000 7fc018f71640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbff406b4d0 0x7fbff406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:10.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.808+0000 7fc018f71640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbff406b4d0 0x7fbff406d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009930 tx=0x7fc00c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:10.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:10.812+0000 7fc0117fa640 1 -- 172.21.15.139:0/2702380403 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc014145c90 con 0x7fc01415dbf0 2024-04-25T09:54:11.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.018+0000 7fc01b1d5640 1 -- 172.21.15.139:0/2702380403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc0140d1680 con 0x7fc01415dbf0 2024-04-25T09:54:11.020 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T09:54:11.018+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:54:11.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.020+0000 7fc0117fa640 1 -- 172.21.15.139:0/2702380403 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbffc058720 con 0x7fc01415dbf0 2024-04-25T09:54:11.022 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:54:11.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.022+0000 7fc01b1d5640 1 -- 172.21.15.139:0/2702380403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbff406b4d0 msgr2=0x7fbff406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.022+0000 7fc01b1d5640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbff406b4d0 0x7fbff406d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009930 tx=0x7fc00c008040 comp rx=0 tx=0).stop 2024-04-25T09:54:11.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.022+0000 7fc01b1d5640 1 -- 172.21.15.139:0/2702380403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc01415dbf0 msgr2=0x7fc0141442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.022+0000 7fc01b1d5640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc01415dbf0 0x7fc0141442d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00c550 tx=0x7fbffc00ca10 comp rx=0 tx=0).stop 2024-04-25T09:54:11.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.022+0000 7fc01b1d5640 1 -- 172.21.15.139:0/2702380403 shutdown_connections 2024-04-25T09:54:11.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.022+0000 7fc01b1d5640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc01415dbf0 0x7fc0141442d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.022+0000 7fc01b1d5640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbff406b4d0 0x7fbff406d990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.022+0000 7fc01b1d5640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0140cec80 0x7fc014145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.022+0000 7fc01b1d5640 1 --2- 172.21.15.139:0/2702380403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0140b9370 0x7fc0140c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.022+0000 7fc01b1d5640 1 -- 172.21.15.139:0/2702380403 >> 172.21.15.139:0/2702380403 conn(0x7fc0140b5480 msgr2=0x7fc0140d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:11.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.022+0000 7fc01b1d5640 1 -- 172.21.15.139:0/2702380403 shutdown_connections 2024-04-25T09:54:11.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.022+0000 7fc01b1d5640 1 -- 172.21.15.139:0/2702380403 wait complete. 2024-04-25T09:54:11.028 INFO:tasks.ceph.mds.d.smithi150.stderr: -4839> 2024-04-25T09:53:19.538+0000 7f9641822640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:54:11.036 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:54:11.059 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-04-25T09:54:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.132+0000 7f1088a38640 1 Processor -- start 2024-04-25T09:54:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.132+0000 7f1088a38640 1 -- start start 2024-04-25T09:54:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f1088a38640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10840cdd20 0x7f1084071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f1088a38640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108415a480 con 0x7f1084150ed0 2024-04-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f1088a38640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108415a760 con 0x7f1084071f90 2024-04-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f1088a38640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108415aa40 con 0x7f10840cdd20 2024-04-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f108259c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1084150ed0 0x7f10840b4200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59086/0 (socket says 172.21.15.139:59086) 2024-04-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f108259c640 1 -- 172.21.15.139:0/1691543967 learned_addr learned my addr 172.21.15.139:0/1691543967 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f1081d9b640 1 --2- 172.21.15.139:0/1691543967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10840cdd20 0x7f1084071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f108159a640 1 -- 172.21.15.139:0/1691543967 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609221055 0 0) 0x7f108415a480 con 0x7f1084150ed0 2024-04-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f108159a640 1 -- 172.21.15.139:0/1691543967 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f106c003520 con 0x7f1084150ed0 2024-04-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f108159a640 1 -- 172.21.15.139:0/1691543967 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629708362 0 0) 0x7f108415a760 con 0x7f1084071f90 2024-04-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f108159a640 1 -- 172.21.15.139:0/1691543967 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f108415a480 con 0x7f1084071f90 2024-04-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f1081d9b640 1 -- 172.21.15.139:0/1691543967 >> v1:172.21.15.150:6789/0 conn(0x7f1084071f90 legacy=0x7f108415da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f1081d9b640 1 -- 172.21.15.139:0/1691543967 >> v1:172.21.15.139:6789/0 conn(0x7f1084150ed0 legacy=0x7f10840b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.133+0000 7f1081d9b640 1 -- 172.21.15.139:0/1691543967 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108415ad20 con 0x7f10840cdd20 2024-04-25T09:54:11.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.134+0000 7f1081d9b640 1 --2- 172.21.15.139:0/1691543967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10840cdd20 0x7f1084071a50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f107800a970 tx=0x7f107802a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ba536e4e8db67fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:11.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.134+0000 7f108159a640 1 -- 172.21.15.139:0/1691543967 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107800b040 con 0x7f10840cdd20 2024-04-25T09:54:11.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.134+0000 7f108159a640 1 -- 172.21.15.139:0/1691543967 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107802f710 con 0x7f10840cdd20 2024-04-25T09:54:11.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.134+0000 7f108159a640 1 -- 172.21.15.139:0/1691543967 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107802fae0 con 0x7f10840cdd20 2024-04-25T09:54:11.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.135+0000 7f1088a38640 1 -- 172.21.15.139:0/1691543967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10840cdd20 msgr2=0x7f1084071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.135+0000 7f1088a38640 1 --2- 172.21.15.139:0/1691543967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10840cdd20 0x7f1084071a50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f107800a970 tx=0x7f107802a430 comp rx=0 tx=0).stop 2024-04-25T09:54:11.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.135+0000 7f1088a38640 1 -- 172.21.15.139:0/1691543967 shutdown_connections 2024-04-25T09:54:11.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.135+0000 7f1088a38640 1 --2- 172.21.15.139:0/1691543967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10840cdd20 0x7f1084071a50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.135+0000 7f1088a38640 1 -- 172.21.15.139:0/1691543967 >> 172.21.15.139:0/1691543967 conn(0x7f1084067000 msgr2=0x7f108406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:11.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.135+0000 7f1088a38640 1 -- 172.21.15.139:0/1691543967 shutdown_connections 2024-04-25T09:54:11.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.135+0000 7f1088a38640 1 -- 172.21.15.139:0/1691543967 wait complete. 2024-04-25T09:54:11.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.136+0000 7f1088a38640 1 Processor -- start 2024-04-25T09:54:11.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.136+0000 7f1088a38640 1 -- start start 2024-04-25T09:54:11.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1088a38640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1084071f90 0x7f1084156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1088a38640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10840cdd20 0x7f108415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1088a38640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1084150ed0 0x7f108415c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1088a38640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10841512b0 con 0x7f1084150ed0 2024-04-25T09:54:11.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1088a38640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10840b3960 con 0x7f1084071f90 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1088a38640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10840b4910 con 0x7f10840cdd20 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1082d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1084150ed0 0x7f108415c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1081d9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10840cdd20 0x7f108415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1082d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1084150ed0 0x7f108415c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49676/0 (socket says 172.21.15.139:49676) 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1082d9d640 1 -- 172.21.15.139:0/4038216534 learned_addr learned my addr 172.21.15.139:0/4038216534 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f108259c640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1084071f90 0x7f1084156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1081d9b640 1 -- 172.21.15.139:0/4038216534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1084071f90 msgr2=0x7f1084156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1081d9b640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1084071f90 0x7f1084156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1081d9b640 1 -- 172.21.15.139:0/4038216534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1084150ed0 msgr2=0x7f108415c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1081d9b640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1084150ed0 0x7f108415c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1081d9b640 1 -- 172.21.15.139:0/4038216534 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10841573e0 con 0x7f10840cdd20 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1082d9d640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1084150ed0 0x7f108415c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.137+0000 7f1081d9b640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10840cdd20 0x7f108415b5b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1078031000 tx=0x7f10780046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.138+0000 7f1072ffd640 1 -- 172.21.15.139:0/4038216534 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107803f070 con 0x7f10840cdd20 2024-04-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.138+0000 7f1072ffd640 1 -- 172.21.15.139:0/4038216534 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107803b620 con 0x7f10840cdd20 2024-04-25T09:54:11.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.138+0000 7f1072ffd640 1 -- 172.21.15.139:0/4038216534 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107803ba30 con 0x7f10840cdd20 2024-04-25T09:54:11.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.138+0000 7f1088a38640 1 -- 172.21.15.139:0/4038216534 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10841576c0 con 0x7f10840cdd20 2024-04-25T09:54:11.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.138+0000 7f1088a38640 1 -- 172.21.15.139:0/4038216534 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108414ce90 con 0x7f10840cdd20 2024-04-25T09:54:11.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.139+0000 7f1088a38640 1 -- 172.21.15.139:0/4038216534 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1084074520 con 0x7f10840cdd20 2024-04-25T09:54:11.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.139+0000 7f1072ffd640 1 -- 172.21.15.139:0/4038216534 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f107803bbd0 con 0x7f10840cdd20 2024-04-25T09:54:11.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.139+0000 7f1072ffd640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f106406b4d0 0x7f106406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:11.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.140+0000 7f1072ffd640 1 -- 172.21.15.139:0/4038216534 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f10780ab520 con 0x7f10840cdd20 2024-04-25T09:54:11.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.140+0000 7f108259c640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f106406b4d0 0x7f106406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:11.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.140+0000 7f108259c640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f106406b4d0 0x7f106406d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10740011e0 tx=0x7f1074007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:11.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.144+0000 7f1072ffd640 1 -- 172.21.15.139:0/4038216534 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1078079980 con 0x7f10840cdd20 2024-04-25T09:54:11.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:11.350+0000 7f1088a38640 1 -- 172.21.15.139:0/4038216534 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f108414ea60 con 0x7f10840cdd20 2024-04-25T09:54:12.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.024+0000 7f1072ffd640 1 -- 172.21.15.139:0/4038216534 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f107807da20 con 0x7f10840cdd20 2024-04-25T09:54:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.026+0000 7f1088a38640 1 -- 172.21.15.139:0/4038216534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f106406b4d0 msgr2=0x7f106406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.026+0000 7f1088a38640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f106406b4d0 0x7f106406d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10740011e0 tx=0x7f1074007d30 comp rx=0 tx=0).stop 2024-04-25T09:54:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.026+0000 7f1088a38640 1 -- 172.21.15.139:0/4038216534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10840cdd20 msgr2=0x7f108415b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.026+0000 7f1088a38640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10840cdd20 0x7f108415b5b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1078031000 tx=0x7f10780046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.026+0000 7f1088a38640 1 -- 172.21.15.139:0/4038216534 shutdown_connections 2024-04-25T09:54:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.026+0000 7f1088a38640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1084150ed0 0x7f108415c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.026+0000 7f1088a38640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f106406b4d0 0x7f106406d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.026+0000 7f1088a38640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10840cdd20 0x7f108415b5b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.026+0000 7f1088a38640 1 --2- 172.21.15.139:0/4038216534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1084071f90 0x7f1084156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.026+0000 7f1088a38640 1 -- 172.21.15.139:0/4038216534 >> 172.21.15.139:0/4038216534 conn(0x7f1084067000 msgr2=0x7f10840b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.026+0000 7f1088a38640 1 -- 172.21.15.139:0/4038216534 shutdown_connections 2024-04-25T09:54:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.026+0000 7f1088a38640 1 -- 172.21.15.139:0/4038216534 wait complete. 2024-04-25T09:54:12.039 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:54:12.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.138+0000 7fb26075c640 1 Processor -- start 2024-04-25T09:54:12.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.138+0000 7fb26075c640 1 -- start start 2024-04-25T09:54:12.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb26075c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb258145230 0x7fb25815be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb26075c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb25815c3d0 0x7fb2580711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb26075c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb258071700 0x7fb25806d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb26075c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb258071b30 con 0x7fb25815c3d0 2024-04-25T09:54:12.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb26075c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25815c960 con 0x7fb258145230 2024-04-25T09:54:12.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb26075c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25815cc40 con 0x7fb258071700 2024-04-25T09:54:12.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb25dcf7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb25815c3d0 0x7fb2580711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb25dcf7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb25815c3d0 0x7fb2580711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49688/0 (socket says 172.21.15.139:49688) 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb25dcf7640 1 -- 172.21.15.139:0/2933923976 learned_addr learned my addr 172.21.15.139:0/2933923976 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb25ecf9640 1 --2- 172.21.15.139:0/2933923976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb258071700 0x7fb25806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb25e4f8640 1 --2- 172.21.15.139:0/2933923976 >> v2:172.21.15.150:3300/0 conn(0x7fb258145230 0x7fb25815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb25dcf7640 1 -- 172.21.15.139:0/2933923976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb258071700 msgr2=0x7fb25806d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb25dcf7640 1 --2- 172.21.15.139:0/2933923976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb258071700 0x7fb25806d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb25dcf7640 1 -- 172.21.15.139:0/2933923976 >> v2:172.21.15.150:3300/0 conn(0x7fb258145230 msgr2=0x7fb25815be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb25dcf7640 1 --2- 172.21.15.139:0/2933923976 >> v2:172.21.15.150:3300/0 conn(0x7fb258145230 0x7fb25815be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.139+0000 7fb25dcf7640 1 -- 172.21.15.139:0/2933923976 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25815d230 con 0x7fb25815c3d0 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.140+0000 7fb25dcf7640 1 --2- 172.21.15.139:0/2933923976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb25815c3d0 0x7fb2580711c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb2580b4990 tx=0x7fb24802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a613176bca0237 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.140+0000 7fb25d4f6640 1 -- 172.21.15.139:0/2933923976 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248037070 con 0x7fb25815c3d0 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.140+0000 7fb25d4f6640 1 -- 172.21.15.139:0/2933923976 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb248032440 con 0x7fb25815c3d0 2024-04-25T09:54:12.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.140+0000 7fb25d4f6640 1 -- 172.21.15.139:0/2933923976 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248032810 con 0x7fb25815c3d0 2024-04-25T09:54:12.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.141+0000 7fb26075c640 1 -- 172.21.15.139:0/2933923976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb25815c3d0 msgr2=0x7fb2580711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:12.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.141+0000 7fb26075c640 1 --2- 172.21.15.139:0/2933923976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb25815c3d0 0x7fb2580711c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb2580b4990 tx=0x7fb24802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.141+0000 7fb26075c640 1 -- 172.21.15.139:0/2933923976 shutdown_connections 2024-04-25T09:54:12.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.141+0000 7fb26075c640 1 --2- 172.21.15.139:0/2933923976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb258071700 0x7fb25806d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.141+0000 7fb26075c640 1 --2- 172.21.15.139:0/2933923976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb25815c3d0 0x7fb2580711c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.141+0000 7fb26075c640 1 --2- 172.21.15.139:0/2933923976 >> v2:172.21.15.150:3300/0 conn(0x7fb258145230 0x7fb25815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.141+0000 7fb26075c640 1 -- 172.21.15.139:0/2933923976 >> 172.21.15.139:0/2933923976 conn(0x7fb258067000 msgr2=0x7fb25806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:12.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.141+0000 7fb26075c640 1 -- 172.21.15.139:0/2933923976 shutdown_connections 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.141+0000 7fb26075c640 1 -- 172.21.15.139:0/2933923976 wait complete. 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.142+0000 7fb26075c640 1 Processor -- start 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.142+0000 7fb26075c640 1 -- start start 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.142+0000 7fb26075c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb258071700 0x7fb25812e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.142+0000 7fb26075c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb258145230 0x7fb25812c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.142+0000 7fb26075c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb25815c3d0 0x7fb25815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.142+0000 7fb26075c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25806d760 con 0x7fb258145230 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.142+0000 7fb26075c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25806e4b0 con 0x7fb258071700 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.142+0000 7fb26075c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25806d8e0 con 0x7fb25815c3d0 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb25dcf7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb258145230 0x7fb25812c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb25dcf7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb258145230 0x7fb25812c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49698/0 (socket says 172.21.15.139:49698) 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb25dcf7640 1 -- 172.21.15.139:0/1342520264 learned_addr learned my addr 172.21.15.139:0/1342520264 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb25dcf7640 1 -- 172.21.15.139:0/1342520264 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb25815c3d0 msgr2=0x7fb25815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb25dcf7640 1 --2- 172.21.15.139:0/1342520264 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb25815c3d0 0x7fb25815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb25dcf7640 1 -- 172.21.15.139:0/1342520264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb258071700 msgr2=0x7fb25812e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:12.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb25dcf7640 1 --2- 172.21.15.139:0/1342520264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb258071700 0x7fb25812e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:12.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb25dcf7640 1 -- 172.21.15.139:0/1342520264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25812e570 con 0x7fb258145230 2024-04-25T09:54:12.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb25dcf7640 1 --2- 172.21.15.139:0/1342520264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb258145230 0x7fb25812c740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb24802a220 tx=0x7fb2480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:12.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb24effd640 1 -- 172.21.15.139:0/1342520264 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248041020 con 0x7fb258145230 2024-04-25T09:54:12.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb24effd640 1 -- 172.21.15.139:0/1342520264 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2480306c0 con 0x7fb258145230 2024-04-25T09:54:12.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb26075c640 1 -- 172.21.15.139:0/1342520264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25812cf90 con 0x7fb258145230 2024-04-25T09:54:12.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb24effd640 1 -- 172.21.15.139:0/1342520264 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248037040 con 0x7fb258145230 2024-04-25T09:54:12.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.143+0000 7fb26075c640 1 -- 172.21.15.139:0/1342520264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25812d550 con 0x7fb258145230 2024-04-25T09:54:12.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.144+0000 7fb26075c640 1 -- 172.21.15.139:0/1342520264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb258074520 con 0x7fb258145230 2024-04-25T09:54:12.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.144+0000 7fb24effd640 1 -- 172.21.15.139:0/1342520264 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb248044020 con 0x7fb258145230 2024-04-25T09:54:12.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.145+0000 7fb24effd640 1 --2- 172.21.15.139:0/1342520264 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb23c06b4d0 0x7fb23c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:12.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.145+0000 7fb24effd640 1 -- 172.21.15.139:0/1342520264 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6821+0+0 (secure 0 0 0) 0x7fb2480aade0 con 0x7fb258145230 2024-04-25T09:54:12.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.145+0000 7fb25e4f8640 1 --2- 172.21.15.139:0/1342520264 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb23c06b4d0 0x7fb23c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:12.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.145+0000 7fb25e4f8640 1 --2- 172.21.15.139:0/1342520264 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb23c06b4d0 0x7fb23c06d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb2540098f0 tx=0x7fb254008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:12.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.149+0000 7fb24effd640 1 -- 172.21.15.139:0/1342520264 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb24804b030 con 0x7fb258145230 2024-04-25T09:54:12.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:12.319+0000 7fb26075c640 1 -- 172.21.15.139:0/1342520264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb25806ac90 con 0x7fb258145230 2024-04-25T09:54:13.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.022+0000 7fb24effd640 1 -- 172.21.15.139:0/1342520264 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb248079890 con 0x7fb258145230 2024-04-25T09:54:13.025 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:54:13.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.025+0000 7fb26075c640 1 -- 172.21.15.139:0/1342520264 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb23c06b4d0 msgr2=0x7fb23c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.025+0000 7fb26075c640 1 --2- 172.21.15.139:0/1342520264 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb23c06b4d0 0x7fb23c06d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb2540098f0 tx=0x7fb254008040 comp rx=0 tx=0).stop 2024-04-25T09:54:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.025+0000 7fb26075c640 1 -- 172.21.15.139:0/1342520264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb258145230 msgr2=0x7fb25812c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.025+0000 7fb26075c640 1 --2- 172.21.15.139:0/1342520264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb258145230 0x7fb25812c740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb24802a220 tx=0x7fb2480046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.025+0000 7fb26075c640 1 -- 172.21.15.139:0/1342520264 shutdown_connections 2024-04-25T09:54:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.025+0000 7fb26075c640 1 --2- 172.21.15.139:0/1342520264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb258145230 0x7fb25812c740 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.025+0000 7fb26075c640 1 --2- 172.21.15.139:0/1342520264 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb23c06b4d0 0x7fb23c06d990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.025+0000 7fb26075c640 1 --2- 172.21.15.139:0/1342520264 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb25815c3d0 0x7fb25815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.025+0000 7fb26075c640 1 --2- 172.21.15.139:0/1342520264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb258071700 0x7fb25812e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.025+0000 7fb26075c640 1 -- 172.21.15.139:0/1342520264 >> 172.21.15.139:0/1342520264 conn(0x7fb258067000 msgr2=0x7fb258144460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.025+0000 7fb26075c640 1 -- 172.21.15.139:0/1342520264 shutdown_connections 2024-04-25T09:54:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.025+0000 7fb26075c640 1 -- 172.21.15.139:0/1342520264 wait complete. 2024-04-25T09:54:13.040 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:54:13.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.140+0000 7fd95117e640 1 Processor -- start 2024-04-25T09:54:13.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.140+0000 7fd95117e640 1 -- start start 2024-04-25T09:54:13.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.140+0000 7fd95117e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd94c0b5620 0x7fd94c0b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.140+0000 7fd95117e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94c0b5a20 con 0x7fd94c15a870 2024-04-25T09:54:13.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.140+0000 7fd95117e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94c0d0d40 con 0x7fd94c070970 2024-04-25T09:54:13.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.140+0000 7fd95117e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd94c0d1020 con 0x7fd94c0b5620 2024-04-25T09:54:13.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.140+0000 7fd94b59e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd94c0b5620 0x7fd94c0b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.141+0000 7fd94ad9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd94c15a870 0x7fd94c070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59096/0 (socket says 172.21.15.139:59096) 2024-04-25T09:54:13.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.141+0000 7fd94ad9d640 1 -- 172.21.15.139:0/2930195540 learned_addr learned my addr 172.21.15.139:0/2930195540 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:13.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.141+0000 7fd94b59e640 1 -- 172.21.15.139:0/2930195540 >> v1:172.21.15.150:6789/0 conn(0x7fd94c070970 legacy=0x7fd94c0ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:13.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.141+0000 7fd94b59e640 1 -- 172.21.15.139:0/2930195540 >> v1:172.21.15.139:6789/0 conn(0x7fd94c15a870 legacy=0x7fd94c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.141+0000 7fd94a59c640 1 -- 172.21.15.139:0/2930195540 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694444754 0 0) 0x7fd94c0b5a20 con 0x7fd94c15a870 2024-04-25T09:54:13.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.141+0000 7fd94b59e640 1 -- 172.21.15.139:0/2930195540 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c0b4b40 con 0x7fd94c0b5620 2024-04-25T09:54:13.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.141+0000 7fd94b59e640 1 --2- 172.21.15.139:0/2930195540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd94c0b5620 0x7fd94c0b4190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd944009d90 tx=0x7fd94402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5234d11fca492e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:13.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.142+0000 7fd94a59c640 1 -- 172.21.15.139:0/2930195540 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944031600 con 0x7fd94c0b5620 2024-04-25T09:54:13.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.142+0000 7fd94a59c640 1 -- 172.21.15.139:0/2930195540 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9440317a0 con 0x7fd94c0b5620 2024-04-25T09:54:13.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.142+0000 7fd94a59c640 1 -- 172.21.15.139:0/2930195540 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944031b70 con 0x7fd94c0b5620 2024-04-25T09:54:13.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.142+0000 7fd95117e640 1 -- 172.21.15.139:0/2930195540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd94c0b5620 msgr2=0x7fd94c0b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.142+0000 7fd95117e640 1 --2- 172.21.15.139:0/2930195540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd94c0b5620 0x7fd94c0b4190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd944009d90 tx=0x7fd94402a610 comp rx=0 tx=0).stop 2024-04-25T09:54:13.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.142+0000 7fd95117e640 1 -- 172.21.15.139:0/2930195540 shutdown_connections 2024-04-25T09:54:13.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.142+0000 7fd95117e640 1 --2- 172.21.15.139:0/2930195540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd94c0b5620 0x7fd94c0b4190 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.142+0000 7fd95117e640 1 -- 172.21.15.139:0/2930195540 >> 172.21.15.139:0/2930195540 conn(0x7fd94c0b23f0 msgr2=0x7fd94c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:13.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.142+0000 7fd95117e640 1 -- 172.21.15.139:0/2930195540 shutdown_connections 2024-04-25T09:54:13.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.142+0000 7fd95117e640 1 -- 172.21.15.139:0/2930195540 wait complete. 2024-04-25T09:54:13.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.144+0000 7fd95117e640 1 Processor -- start 2024-04-25T09:54:13.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.144+0000 7fd95117e640 1 -- start start 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.144+0000 7fd95117e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd94c070970 0x7fd94c14d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.144+0000 7fd95117e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd94c0b5620 0x7fd94c159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.144+0000 7fd95117e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd94c15a870 0x7fd94c159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.144+0000 7fd95117e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c0d05f0 con 0x7fd94c15a870 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.144+0000 7fd95117e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c0b5190 con 0x7fd94c070970 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.144+0000 7fd95117e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c0d0770 con 0x7fd94c0b5620 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.144+0000 7fd94b59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd94c15a870 0x7fd94c159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.144+0000 7fd94b59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd94c15a870 0x7fd94c159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49718/0 (socket says 172.21.15.139:49718) 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.144+0000 7fd94b59e640 1 -- 172.21.15.139:0/3355480665 learned_addr learned my addr 172.21.15.139:0/3355480665 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd94b59e640 1 -- 172.21.15.139:0/3355480665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd94c0b5620 msgr2=0x7fd94c159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd94ad9d640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd94c070970 0x7fd94c14d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd943fff640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd94c0b5620 0x7fd94c159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd94b59e640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd94c0b5620 0x7fd94c159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd94b59e640 1 -- 172.21.15.139:0/3355480665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd94c070970 msgr2=0x7fd94c14d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:13.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd94b59e640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd94c070970 0x7fd94c14d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:13.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd94b59e640 1 -- 172.21.15.139:0/3355480665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c151940 con 0x7fd94c15a870 2024-04-25T09:54:13.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd94ad9d640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd94c070970 0x7fd94c14d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:13.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd943fff640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd94c0b5620 0x7fd94c159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:13.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd94b59e640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd94c15a870 0x7fd94c159960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd944009eb0 tx=0x7fd944004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:13.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd9437fe640 1 -- 172.21.15.139:0/3355480665 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944040070 con 0x7fd94c15a870 2024-04-25T09:54:13.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd9437fe640 1 -- 172.21.15.139:0/3355480665 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd944030b60 con 0x7fd94c15a870 2024-04-25T09:54:13.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd9437fe640 1 -- 172.21.15.139:0/3355480665 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944045460 con 0x7fd94c15a870 2024-04-25T09:54:13.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd95117e640 1 -- 172.21.15.139:0/3355480665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94c151d80 con 0x7fd94c15a870 2024-04-25T09:54:13.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.145+0000 7fd95117e640 1 -- 172.21.15.139:0/3355480665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94c152340 con 0x7fd94c15a870 2024-04-25T09:54:13.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.146+0000 7fd95117e640 1 -- 172.21.15.139:0/3355480665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94c074520 con 0x7fd94c15a870 2024-04-25T09:54:13.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.146+0000 7fd9437fe640 1 -- 172.21.15.139:0/3355480665 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd944039040 con 0x7fd94c15a870 2024-04-25T09:54:13.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.147+0000 7fd9437fe640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd92c06b4d0 0x7fd92c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:13.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.147+0000 7fd9437fe640 1 -- 172.21.15.139:0/3355480665 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6383+0+0 (secure 0 0 0) 0x7fd9440abd70 con 0x7fd94c15a870 2024-04-25T09:54:13.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.147+0000 7fd94ad9d640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd92c06b4d0 0x7fd92c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:13.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.147+0000 7fd94ad9d640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd92c06b4d0 0x7fd92c06d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd93c000f60 tx=0x7fd93c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:13.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.151+0000 7fd9437fe640 1 -- 172.21.15.139:0/3355480665 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd94407aaa0 con 0x7fd94c15a870 2024-04-25T09:54:13.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:13.328+0000 7fd95117e640 1 -- 172.21.15.139:0/3355480665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd94c0cf460 con 0x7fd94c15a870 2024-04-25T09:54:14.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.023+0000 7fd9437fe640 1 -- 172.21.15.139:0/3355480665 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd94407a3a0 con 0x7fd94c15a870 2024-04-25T09:54:14.027 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' removed 2024-04-25T09:54:14.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.026+0000 7fd95117e640 1 -- 172.21.15.139:0/3355480665 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd92c06b4d0 msgr2=0x7fd92c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:14.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.026+0000 7fd95117e640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd92c06b4d0 0x7fd92c06d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd93c000f60 tx=0x7fd93c009040 comp rx=0 tx=0).stop 2024-04-25T09:54:14.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.026+0000 7fd95117e640 1 -- 172.21.15.139:0/3355480665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd94c15a870 msgr2=0x7fd94c159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:14.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.026+0000 7fd95117e640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd94c15a870 0x7fd94c159960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd944009eb0 tx=0x7fd944004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:14.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.026+0000 7fd95117e640 1 -- 172.21.15.139:0/3355480665 shutdown_connections 2024-04-25T09:54:14.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.026+0000 7fd95117e640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd94c15a870 0x7fd94c159960 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.026+0000 7fd95117e640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd92c06b4d0 0x7fd92c06d990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.026+0000 7fd95117e640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd94c0b5620 0x7fd94c159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.026+0000 7fd95117e640 1 --2- 172.21.15.139:0/3355480665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd94c070970 0x7fd94c14d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.026+0000 7fd95117e640 1 -- 172.21.15.139:0/3355480665 >> 172.21.15.139:0/3355480665 conn(0x7fd94c0b23f0 msgr2=0x7fd94c0ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:14.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.026+0000 7fd95117e640 1 -- 172.21.15.139:0/3355480665 shutdown_connections 2024-04-25T09:54:14.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.027+0000 7fd95117e640 1 -- 172.21.15.139:0/3355480665 wait complete. 2024-04-25T09:54:14.043 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T09:54:14.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.142+0000 7fed4d3d5640 1 Processor -- start 2024-04-25T09:54:14.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.142+0000 7fed4d3d5640 1 -- start start 2024-04-25T09:54:14.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed4d3d5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fed4815dbf0 0x7fed481617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed4d3d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed480b9370 0x7fed480ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed4d3d5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fed480cec80 0x7fed4806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed4d3d5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed480d1ee0 con 0x7fed4815dbf0 2024-04-25T09:54:14.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed4d3d5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed48161dc0 con 0x7fed480cec80 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed4d3d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed481620a0 con 0x7fed480b9370 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed46ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fed4815dbf0 0x7fed481617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed46ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fed4815dbf0 0x7fed481617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49720/0 (socket says 172.21.15.139:49720) 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed46ffd640 1 -- 172.21.15.139:0/1933547715 learned_addr learned my addr 172.21.15.139:0/1933547715 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed467fc640 1 --2- 172.21.15.139:0/1933547715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed480b9370 0x7fed480ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed46ffd640 1 -- 172.21.15.139:0/1933547715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed480b9370 msgr2=0x7fed480ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed46ffd640 1 --2- 172.21.15.139:0/1933547715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed480b9370 0x7fed480ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed46ffd640 1 -- 172.21.15.139:0/1933547715 >> v2:172.21.15.150:3300/0 conn(0x7fed480cec80 msgr2=0x7fed4806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed477fe640 1 --2- 172.21.15.139:0/1933547715 >> v2:172.21.15.150:3300/0 conn(0x7fed480cec80 0x7fed4806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed46ffd640 1 --2- 172.21.15.139:0/1933547715 >> v2:172.21.15.150:3300/0 conn(0x7fed480cec80 0x7fed4806c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.143+0000 7fed46ffd640 1 -- 172.21.15.139:0/1933547715 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed48162380 con 0x7fed4815dbf0 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.144+0000 7fed46ffd640 1 --2- 172.21.15.139:0/1933547715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed4815dbf0 0x7fed481617e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fed4808c170 tx=0x7fed3802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ec580b5f6112ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.144+0000 7fed45ffb640 1 -- 172.21.15.139:0/1933547715 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38036070 con 0x7fed4815dbf0 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.144+0000 7fed45ffb640 1 -- 172.21.15.139:0/1933547715 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed38031600 con 0x7fed4815dbf0 2024-04-25T09:54:14.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.144+0000 7fed45ffb640 1 -- 172.21.15.139:0/1933547715 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed380319d0 con 0x7fed4815dbf0 2024-04-25T09:54:14.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.145+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1933547715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed4815dbf0 msgr2=0x7fed481617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:14.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.145+0000 7fed4d3d5640 1 --2- 172.21.15.139:0/1933547715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed4815dbf0 0x7fed481617e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fed4808c170 tx=0x7fed3802a6a0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.145+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1933547715 shutdown_connections 2024-04-25T09:54:14.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.145+0000 7fed4d3d5640 1 --2- 172.21.15.139:0/1933547715 >> v2:172.21.15.150:3300/0 conn(0x7fed480cec80 0x7fed4806c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.145+0000 7fed4d3d5640 1 --2- 172.21.15.139:0/1933547715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed480b9370 0x7fed480ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.145+0000 7fed4d3d5640 1 --2- 172.21.15.139:0/1933547715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed4815dbf0 0x7fed481617e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.145+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1933547715 >> 172.21.15.139:0/1933547715 conn(0x7fed480b5480 msgr2=0x7fed480b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.145+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1933547715 shutdown_connections 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.145+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1933547715 wait complete. 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.146+0000 7fed4d3d5640 1 Processor -- start 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.146+0000 7fed4d3d5640 1 -- start start 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed4d3d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed480b9370 0x7fed48158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed4d3d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed480cec80 0x7fed4815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed4d3d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed4815dbf0 0x7fed480b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed4d3d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4815dfd0 con 0x7fed480b9370 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed4d3d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed480b97d0 con 0x7fed480cec80 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed4d3d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed480b4c00 con 0x7fed4815dbf0 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed46ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed480b9370 0x7fed48158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed46ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed480b9370 0x7fed48158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49734/0 (socket says 172.21.15.139:49734) 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed46ffd640 1 -- 172.21.15.139:0/1274683533 learned_addr learned my addr 172.21.15.139:0/1274683533 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed467fc640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed480cec80 0x7fed4815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed477fe640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed4815dbf0 0x7fed480b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed46ffd640 1 -- 172.21.15.139:0/1274683533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed4815dbf0 msgr2=0x7fed480b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed46ffd640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed4815dbf0 0x7fed480b6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed46ffd640 1 -- 172.21.15.139:0/1274683533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed480cec80 msgr2=0x7fed4815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed46ffd640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed480cec80 0x7fed4815a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed46ffd640 1 -- 172.21.15.139:0/1274683533 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed480b4ee0 con 0x7fed480b9370 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed46ffd640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed480b9370 0x7fed48158e40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fed38034040 tx=0x7fed38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed2f7fe640 1 -- 172.21.15.139:0/1274683533 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38041020 con 0x7fed480b9370 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed2f7fe640 1 -- 172.21.15.139:0/1274683533 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed38007800 con 0x7fed480b9370 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1274683533 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed480aa3f0 con 0x7fed480b9370 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.147+0000 7fed2f7fe640 1 -- 172.21.15.139:0/1274683533 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38036040 con 0x7fed480b9370 2024-04-25T09:54:14.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.148+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1274683533 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed480b51c0 con 0x7fed480b9370 2024-04-25T09:54:14.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.148+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1274683533 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed48074520 con 0x7fed480b9370 2024-04-25T09:54:14.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.149+0000 7fed2f7fe640 1 -- 172.21.15.139:0/1274683533 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed38044020 con 0x7fed480b9370 2024-04-25T09:54:14.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.149+0000 7fed2f7fe640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed3406b500 0x7fed3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:14.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.149+0000 7fed2f7fe640 1 -- 172.21.15.139:0/1274683533 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fed380aae30 con 0x7fed480b9370 2024-04-25T09:54:14.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.149+0000 7fed467fc640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed3406b500 0x7fed3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:14.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.150+0000 7fed467fc640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed3406b500 0x7fed3406d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fed481f8150 tx=0x7fed3c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:14.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.153+0000 7fed2f7fe640 1 -- 172.21.15.139:0/1274683533 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed3804b030 con 0x7fed480b9370 2024-04-25T09:54:14.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:14.322+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1274683533 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fed480d1680 con 0x7fed480b9370 2024-04-25T09:54:15.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.032+0000 7fed2f7fe640 1 -- 172.21.15.139:0/1274683533 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7fed38079660 con 0x7fed480b9370 2024-04-25T09:54:15.036 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data_ec' removed 2024-04-25T09:54:15.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.036+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1274683533 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed3406b500 msgr2=0x7fed3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.036+0000 7fed4d3d5640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed3406b500 0x7fed3406d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fed481f8150 tx=0x7fed3c008040 comp rx=0 tx=0).stop 2024-04-25T09:54:15.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.036+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1274683533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed480b9370 msgr2=0x7fed48158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.036+0000 7fed4d3d5640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed480b9370 0x7fed48158e40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fed38034040 tx=0x7fed38004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:15.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.036+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1274683533 shutdown_connections 2024-04-25T09:54:15.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.036+0000 7fed4d3d5640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed3406b500 0x7fed3406d9c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.036+0000 7fed4d3d5640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed4815dbf0 0x7fed480b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.036+0000 7fed4d3d5640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed480cec80 0x7fed4815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.036+0000 7fed4d3d5640 1 --2- 172.21.15.139:0/1274683533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed480b9370 0x7fed48158e40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.036+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1274683533 >> 172.21.15.139:0/1274683533 conn(0x7fed480b5480 msgr2=0x7fed480c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:15.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.036+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1274683533 shutdown_connections 2024-04-25T09:54:15.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.036+0000 7fed4d3d5640 1 -- 172.21.15.139:0/1274683533 wait complete. 2024-04-25T09:54:15.052 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:54:15.052 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:54:15.052 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:54:15.052 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:15.052 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:54:15.052 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:54:15.053 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:54:15.053 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:15.053 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:54:15.053 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:54:15.053 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:54:15.053 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:15.053 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:54:15.053 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:54:15.054 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:54:15.054 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:15.063 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:54:15.063 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:54:15.065 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:54:15.065 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:54:15.065 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:54:15.066 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:54:15.066 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:54:15.066 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:54:15.106 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:54:15.107 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:15.107 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:54:15.107 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:15.108 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:54:15.108 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:15.108 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:54:15.109 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:15.109 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:54:15.182 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T09:54:15.182 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-04-25T09:54:15.189 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-04-25T09:54:15.189 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-04-25T09:54:15.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.215+0000 7f0369c1c640 1 Processor -- start 2024-04-25T09:54:15.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.215+0000 7f0369c1c640 1 -- start start 2024-04-25T09:54:15.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.216+0000 7f0369c1c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0364071ea0 0x7f0364072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.216+0000 7f0369c1c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f036406e2f0 0x7f03640b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.216+0000 7f0369c1c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03640727c0 con 0x7f03640b2fe0 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.216+0000 7f0369c1c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03640b3760 con 0x7f0364071ea0 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.216+0000 7f03637fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0364071ea0 0x7f0364072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.217+0000 7f0369c1c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036406d810 con 0x7f036406e2f0 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.217+0000 7f0363fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f03640b2fe0 0x7f03640cd2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59126/0 (socket says 172.21.15.139:59126) 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.217+0000 7f0363fff640 1 -- 172.21.15.139:0/2484413315 learned_addr learned my addr 172.21.15.139:0/2484413315 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.217+0000 7f03627fc640 1 -- 172.21.15.139:0/2484413315 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105470092 0 0) 0x7f03640727c0 con 0x7f03640b2fe0 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.217+0000 7f03627fc640 1 -- 172.21.15.139:0/2484413315 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f034c003520 con 0x7f03640b2fe0 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.217+0000 7f0362ffd640 1 --2- 172.21.15.139:0/2484413315 >> v2:172.21.15.150:3300/0 conn(0x7f036406e2f0 0x7f03640b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.217+0000 7f03637fe640 1 -- 172.21.15.139:0/2484413315 >> v2:172.21.15.150:3300/0 conn(0x7f036406e2f0 msgr2=0x7f03640b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.217+0000 7f03637fe640 1 --2- 172.21.15.139:0/2484413315 >> v2:172.21.15.150:3300/0 conn(0x7f036406e2f0 0x7f03640b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.217+0000 7f03637fe640 1 -- 172.21.15.139:0/2484413315 >> v1:172.21.15.139:6789/0 conn(0x7f03640b2fe0 legacy=0x7f03640cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.217+0000 7f03637fe640 1 -- 172.21.15.139:0/2484413315 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036406daf0 con 0x7f0364071ea0 2024-04-25T09:54:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.218+0000 7f03637fe640 1 --2- 172.21.15.139:0/2484413315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0364071ea0 0x7f0364072280 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0364143e60 tx=0x7f035402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339e3908e2efe3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.218+0000 7f03627fc640 1 -- 172.21.15.139:0/2484413315 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0354037070 con 0x7f0364071ea0 2024-04-25T09:54:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.218+0000 7f03627fc640 1 -- 172.21.15.139:0/2484413315 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035402ae50 con 0x7f0364071ea0 2024-04-25T09:54:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.218+0000 7f03627fc640 1 -- 172.21.15.139:0/2484413315 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0354032660 con 0x7f0364071ea0 2024-04-25T09:54:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.219+0000 7f0369c1c640 1 -- 172.21.15.139:0/2484413315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0364071ea0 msgr2=0x7f0364072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.219+0000 7f0369c1c640 1 --2- 172.21.15.139:0/2484413315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0364071ea0 0x7f0364072280 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0364143e60 tx=0x7f035402a140 comp rx=0 tx=0).stop 2024-04-25T09:54:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.219+0000 7f0369c1c640 1 -- 172.21.15.139:0/2484413315 shutdown_connections 2024-04-25T09:54:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.219+0000 7f0369c1c640 1 --2- 172.21.15.139:0/2484413315 >> v2:172.21.15.150:3300/0 conn(0x7f036406e2f0 0x7f03640b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.219+0000 7f0369c1c640 1 --2- 172.21.15.139:0/2484413315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0364071ea0 0x7f0364072280 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.219+0000 7f0369c1c640 1 -- 172.21.15.139:0/2484413315 >> 172.21.15.139:0/2484413315 conn(0x7f03640b17a0 msgr2=0x7f03640b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.219+0000 7f0369c1c640 1 -- 172.21.15.139:0/2484413315 shutdown_connections 2024-04-25T09:54:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.219+0000 7f0369c1c640 1 -- 172.21.15.139:0/2484413315 wait complete. 2024-04-25T09:54:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.220+0000 7f0369c1c640 1 Processor -- start 2024-04-25T09:54:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.220+0000 7f0369c1c640 1 -- start start 2024-04-25T09:54:15.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0369c1c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f036406e2f0 0x7f036413a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0369c1c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0364071ea0 0x7f03640cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0369c1c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03640b2fe0 0x7f03640cff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0369c1c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03641408a0 con 0x7f03640b2fe0 2024-04-25T09:54:15.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0369c1c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03640b3470 con 0x7f0364071ea0 2024-04-25T09:54:15.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0369c1c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0364140a20 con 0x7f036406e2f0 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0363fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03640b2fe0 0x7f03640cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0363fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03640b2fe0 0x7f03640cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49768/0 (socket says 172.21.15.139:49768) 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0363fff640 1 -- 172.21.15.139:0/3844322915 learned_addr learned my addr 172.21.15.139:0/3844322915 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0362ffd640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0364071ea0 0x7f03640cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f03637fe640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f036406e2f0 0x7f036413a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0363fff640 1 -- 172.21.15.139:0/3844322915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f036406e2f0 msgr2=0x7f036413a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0363fff640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f036406e2f0 0x7f036413a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0363fff640 1 -- 172.21.15.139:0/3844322915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0364071ea0 msgr2=0x7f03640cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0363fff640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0364071ea0 0x7f03640cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0363fff640 1 -- 172.21.15.139:0/3844322915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03640c4d40 con 0x7f03640b2fe0 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f03637fe640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f036406e2f0 0x7f036413a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.221+0000 7f0362ffd640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0364071ea0 0x7f03640cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.222+0000 7f0363fff640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03640b2fe0 0x7f03640cff50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f035c00caa0 tx=0x7f035c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.222+0000 7f034bfff640 1 -- 172.21.15.139:0/3844322915 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c017070 con 0x7f03640b2fe0 2024-04-25T09:54:15.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.222+0000 7f034bfff640 1 -- 172.21.15.139:0/3844322915 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c00ac00 con 0x7f03640b2fe0 2024-04-25T09:54:15.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.222+0000 7f0369c1c640 1 -- 172.21.15.139:0/3844322915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036413aa80 con 0x7f03640b2fe0 2024-04-25T09:54:15.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.222+0000 7f034bfff640 1 -- 172.21.15.139:0/3844322915 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c012400 con 0x7f03640b2fe0 2024-04-25T09:54:15.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.222+0000 7f0369c1c640 1 -- 172.21.15.139:0/3844322915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036413e5b0 con 0x7f03640b2fe0 2024-04-25T09:54:15.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.223+0000 7f0349ffb640 1 -- 172.21.15.139:0/3844322915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c005290 con 0x7f03640b2fe0 2024-04-25T09:54:15.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.223+0000 7f034bfff640 1 -- 172.21.15.139:0/3844322915 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f035c01a070 con 0x7f03640b2fe0 2024-04-25T09:54:15.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.224+0000 7f034bfff640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f035006b500 0x7f035006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:15.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.224+0000 7f034bfff640 1 -- 172.21.15.139:0/3844322915 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f035c089ed0 con 0x7f03640b2fe0 2024-04-25T09:54:15.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.224+0000 7f03637fe640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f035006b500 0x7f035006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:15.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.224+0000 7f03637fe640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f035006b500 0x7f035006d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0354034000 tx=0x7f035402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:15.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.228+0000 7f034bfff640 1 -- 172.21.15.139:0/3844322915 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03640c4d40 con 0x7f03640b2fe0 2024-04-25T09:54:15.314 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:54:15.316 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:54:15.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:15.420+0000 7f0349ffb640 1 -- 172.21.15.139:0/3844322915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f033c005850 con 0x7f03640b2fe0 2024-04-25T09:54:16.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.043+0000 7f034bfff640 1 -- 172.21.15.139:0/3844322915 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7f035c058e30 con 0x7f03640b2fe0 2024-04-25T09:54:16.046 INFO:teuthology.orchestra.run.smithi139.stderr: removed all blocklist entries 2024-04-25T09:54:16.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.045+0000 7f0349ffb640 1 -- 172.21.15.139:0/3844322915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f035006b500 msgr2=0x7f035006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.045+0000 7f0349ffb640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f035006b500 0x7f035006d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0354034000 tx=0x7f035402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:16.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.045+0000 7f0349ffb640 1 -- 172.21.15.139:0/3844322915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03640b2fe0 msgr2=0x7f03640cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.045+0000 7f0349ffb640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03640b2fe0 0x7f03640cff50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f035c00caa0 tx=0x7f035c00cf60 comp rx=0 tx=0).stop 2024-04-25T09:54:16.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.045+0000 7f0349ffb640 1 -- 172.21.15.139:0/3844322915 shutdown_connections 2024-04-25T09:54:16.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.045+0000 7f0349ffb640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03640b2fe0 0x7f03640cff50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.045+0000 7f0349ffb640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f035006b500 0x7f035006d9c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.045+0000 7f0349ffb640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0364071ea0 0x7f03640cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.045+0000 7f0349ffb640 1 --2- 172.21.15.139:0/3844322915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f036406e2f0 0x7f036413a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.045+0000 7f0349ffb640 1 -- 172.21.15.139:0/3844322915 >> 172.21.15.139:0/3844322915 conn(0x7f03640b17a0 msgr2=0x7f036415c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.045+0000 7f0349ffb640 1 -- 172.21.15.139:0/3844322915 shutdown_connections 2024-04-25T09:54:16.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.045+0000 7f0349ffb640 1 -- 172.21.15.139:0/3844322915 wait complete. 2024-04-25T09:54:16.060 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:54:16.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.155+0000 7fbe89888640 1 Processor -- start 2024-04-25T09:54:16.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.156+0000 7fbe89888640 1 -- start start 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.156+0000 7fbe89888640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe8415a860 0x7fbe840cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.156+0000 7fbe89888640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbe840cd1f0 0x7fbe8406e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.156+0000 7fbe89888640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbe8415ac40 0x7fbe8406e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.156+0000 7fbe89888640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840b02e0 con 0x7fbe840cd1f0 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.156+0000 7fbe89888640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840af6d0 con 0x7fbe8415ac40 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.156+0000 7fbe89888640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840af9b0 con 0x7fbe8415a860 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.156+0000 7fbe827fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbe840cd1f0 0x7fbe8406e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.156+0000 7fbe827fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbe840cd1f0 0x7fbe8406e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49776/0 (socket says 172.21.15.139:49776) 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.157+0000 7fbe827fc640 1 -- 172.21.15.139:0/2464104350 learned_addr learned my addr 172.21.15.139:0/2464104350 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.157+0000 7fbe82ffd640 1 --2- 172.21.15.139:0/2464104350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe8415a860 0x7fbe840cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.157+0000 7fbe837fe640 1 --2- 172.21.15.139:0/2464104350 >> v2:172.21.15.150:3300/0 conn(0x7fbe8415ac40 0x7fbe8406e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.157+0000 7fbe827fc640 1 -- 172.21.15.139:0/2464104350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe8415a860 msgr2=0x7fbe840cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.157+0000 7fbe827fc640 1 --2- 172.21.15.139:0/2464104350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe8415a860 0x7fbe840cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.157+0000 7fbe827fc640 1 -- 172.21.15.139:0/2464104350 >> v2:172.21.15.150:3300/0 conn(0x7fbe8415ac40 msgr2=0x7fbe8406e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:16.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.157+0000 7fbe827fc640 1 --2- 172.21.15.139:0/2464104350 >> v2:172.21.15.150:3300/0 conn(0x7fbe8415ac40 0x7fbe8406e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.157+0000 7fbe827fc640 1 -- 172.21.15.139:0/2464104350 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840afc90 con 0x7fbe840cd1f0 2024-04-25T09:54:16.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.157+0000 7fbe827fc640 1 --2- 172.21.15.139:0/2464104350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe840cd1f0 0x7fbe8406e310 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbe840b49e0 tx=0x7fbe7402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13de442e7b5075d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.158+0000 7fbe81ffb640 1 -- 172.21.15.139:0/2464104350 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74032790 con 0x7fbe840cd1f0 2024-04-25T09:54:16.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.158+0000 7fbe81ffb640 1 -- 172.21.15.139:0/2464104350 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe74038070 con 0x7fbe840cd1f0 2024-04-25T09:54:16.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.158+0000 7fbe81ffb640 1 -- 172.21.15.139:0/2464104350 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74032b50 con 0x7fbe840cd1f0 2024-04-25T09:54:16.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.158+0000 7fbe89888640 1 -- 172.21.15.139:0/2464104350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe840cd1f0 msgr2=0x7fbe8406e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.158+0000 7fbe89888640 1 --2- 172.21.15.139:0/2464104350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe840cd1f0 0x7fbe8406e310 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbe840b49e0 tx=0x7fbe7402c890 comp rx=0 tx=0).stop 2024-04-25T09:54:16.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.158+0000 7fbe89888640 1 -- 172.21.15.139:0/2464104350 shutdown_connections 2024-04-25T09:54:16.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.158+0000 7fbe89888640 1 --2- 172.21.15.139:0/2464104350 >> v2:172.21.15.150:3300/0 conn(0x7fbe8415ac40 0x7fbe8406e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.158+0000 7fbe89888640 1 --2- 172.21.15.139:0/2464104350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe840cd1f0 0x7fbe8406e310 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.158+0000 7fbe89888640 1 --2- 172.21.15.139:0/2464104350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe8415a860 0x7fbe840cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.158+0000 7fbe89888640 1 -- 172.21.15.139:0/2464104350 >> 172.21.15.139:0/2464104350 conn(0x7fbe84066f20 msgr2=0x7fbe8406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.158+0000 7fbe89888640 1 -- 172.21.15.139:0/2464104350 shutdown_connections 2024-04-25T09:54:16.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.158+0000 7fbe89888640 1 -- 172.21.15.139:0/2464104350 wait complete. 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.159+0000 7fbe89888640 1 Processor -- start 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.160+0000 7fbe89888640 1 -- start start 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.160+0000 7fbe89888640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe840cd1f0 0x7fbe840c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.160+0000 7fbe89888640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe8415a860 0x7fbe8414a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.160+0000 7fbe89888640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe8415ac40 0x7fbe840b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.160+0000 7fbe89888640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840af420 con 0x7fbe8415a860 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.160+0000 7fbe89888640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840af2a0 con 0x7fbe840cd1f0 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.160+0000 7fbe89888640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8415e240 con 0x7fbe8415ac40 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.160+0000 7fbe827fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe8415a860 0x7fbe8414a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.160+0000 7fbe827fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe8415a860 0x7fbe8414a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49778/0 (socket says 172.21.15.139:49778) 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.160+0000 7fbe827fc640 1 -- 172.21.15.139:0/3801692437 learned_addr learned my addr 172.21.15.139:0/3801692437 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.160+0000 7fbe837fe640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe8415ac40 0x7fbe840b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.160+0000 7fbe82ffd640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe840cd1f0 0x7fbe840c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.161+0000 7fbe827fc640 1 -- 172.21.15.139:0/3801692437 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe8415ac40 msgr2=0x7fbe840b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.161+0000 7fbe827fc640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe8415ac40 0x7fbe840b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.161+0000 7fbe827fc640 1 -- 172.21.15.139:0/3801692437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe840cd1f0 msgr2=0x7fbe840c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.161+0000 7fbe827fc640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe840cd1f0 0x7fbe840c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.161+0000 7fbe827fc640 1 -- 172.21.15.139:0/3801692437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe841f6f20 con 0x7fbe8415a860 2024-04-25T09:54:16.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.161+0000 7fbe827fc640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe8415a860 0x7fbe8414a0a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400b2c0 tx=0x7fbe74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.161+0000 7fbe6bfff640 1 -- 172.21.15.139:0/3801692437 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74043070 con 0x7fbe8415a860 2024-04-25T09:54:16.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.161+0000 7fbe6bfff640 1 -- 172.21.15.139:0/3801692437 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7403c620 con 0x7fbe8415a860 2024-04-25T09:54:16.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.161+0000 7fbe89888640 1 -- 172.21.15.139:0/3801692437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe841f7200 con 0x7fbe8415a860 2024-04-25T09:54:16.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.161+0000 7fbe6bfff640 1 -- 172.21.15.139:0/3801692437 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7403c9f0 con 0x7fbe8415a860 2024-04-25T09:54:16.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.161+0000 7fbe89888640 1 -- 172.21.15.139:0/3801692437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe841f77c0 con 0x7fbe8415a860 2024-04-25T09:54:16.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.162+0000 7fbe89888640 1 -- 172.21.15.139:0/3801692437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe84074520 con 0x7fbe8415a860 2024-04-25T09:54:16.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.163+0000 7fbe6bfff640 1 -- 172.21.15.139:0/3801692437 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe74046070 con 0x7fbe8415a860 2024-04-25T09:54:16.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.163+0000 7fbe6bfff640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbe7006b4d0 0x7fbe7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.163+0000 7fbe82ffd640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbe7006b4d0 0x7fbe7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.163+0000 7fbe6bfff640 1 -- 172.21.15.139:0/3801692437 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbe740aa430 con 0x7fbe8415a860 2024-04-25T09:54:16.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.164+0000 7fbe82ffd640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbe7006b4d0 0x7fbe7006d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009ac0 tx=0x7fbe78009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.167+0000 7fbe6bfff640 1 -- 172.21.15.139:0/3801692437 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe74078f70 con 0x7fbe8415a860 2024-04-25T09:54:16.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.411+0000 7fbe89888640 1 -- 172.21.15.139:0/3801692437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbe840cf220 con 0x7fbe8415a860 2024-04-25T09:54:16.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.411+0000 7fbe6bfff640 1 -- 172.21.15.139:0/3801692437 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fbe7407d010 con 0x7fbe8415a860 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_dump": [ 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.a", 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypm0xdaHhAA6nupxCTQhucepW2iEWNUoQ==", 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.b", 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmB9GDKRAA/lmU4OiKO5dFgayYx09GQg==", 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.c", 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmvYgaJRAA/YIE6ralp2WuTK993D7H0Q==", 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.d", 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmg49WMBAAlPY9ZSeOoV296xSdl/m1gg==", 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.0", 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDVJypmG7TpDhAAo9CKuZBw8BsQgB2F+BR5gw==", 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.1", 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDYJypmgR62DBAAotfnvS47WDunP9Ic03gA6Q==", 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:16.418 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.2", 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDbJypmzyqGChAA6wrfIdqpiLm3RO36oorihQ==", 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.3", 2024-04-25T09:54:16.419 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDeJypmt72cCBAAgcLrFC3MUU7wlFySAfLWig==", 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.4", 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDkJypmxJLpDBAAvROZ9AvYdWMqOALEKIAFCw==", 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.5", 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDnJypmTPXEChAA20BghY39PGw6zFyu9yWbBQ==", 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.6", 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDqJypmaBoYCRAAy9eHgUI/WTkE2XdT1RKLsw==", 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:16.421 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.7", 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDtJypmViJlCBAANJSDWVHVbWR03Y1SMDH++Q==", 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.422 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.0", 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg==", 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.1", 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypmOncYARAA6gYjZbRDkhqFX0mpSDbH5Q==", 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:54:16.423 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.2", 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmQs2wORAArzyV70nGDX4486LlFjM70Q==", 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.424 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.3", 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypm90x9BhAA41rZPs0O5mwmuHy1bkPCvw==", 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.admin", 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmOWQcAxAA/ZVXEHYJQZf9UtgZ5g25Ag==", 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:54:16.425 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.bootstrap-mds", 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDyJypm2E2JNhAAg9PRQMS3lYI6WEjHaOkvSg==", 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDyJypm21aJNhAALK2IOU3/X7FQzJ2GFxW3Xg==", 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.426 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.bootstrap-osd", 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDyJypmFF6JNhAA6XLZrJhvswX48J/fk25+zA==", 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDyJypm2mSJNhAAA0x85is4j6xCyYb0vjrN+A==", 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T09:54:16.427 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDyJypmj2uJNhAADVMTvhm107Jz4AXqLNDJBQ==", 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDyJypmb3KJNhAA+vcMBfuFf5ekbH8iQERuiw==", 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.x", 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmt+RVGxAAi1djVoqnoYCDMzJ9A0dmNA==", 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.y", 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmuJ4bFxAAw6a53AgThU/ut+Mk5j0gqQ==", 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T09:54:16.429 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:16.430 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.430 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:16.430 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-04-25T09:54:16.430 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:54:16.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.413+0000 7fbe89888640 1 -- 172.21.15.139:0/3801692437 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbe7006b4d0 msgr2=0x7fbe7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.413+0000 7fbe89888640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbe7006b4d0 0x7fbe7006d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009ac0 tx=0x7fbe78009190 comp rx=0 tx=0).stop 2024-04-25T09:54:16.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.413+0000 7fbe89888640 1 -- 172.21.15.139:0/3801692437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe8415a860 msgr2=0x7fbe8414a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.413+0000 7fbe89888640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe8415a860 0x7fbe8414a0a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400b2c0 tx=0x7fbe74004730 comp rx=0 tx=0).stop 2024-04-25T09:54:16.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.413+0000 7fbe89888640 1 -- 172.21.15.139:0/3801692437 shutdown_connections 2024-04-25T09:54:16.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.413+0000 7fbe89888640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe8415a860 0x7fbe8414a0a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.413+0000 7fbe89888640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbe7006b4d0 0x7fbe7006d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.414+0000 7fbe89888640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe8415ac40 0x7fbe840b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.414+0000 7fbe89888640 1 --2- 172.21.15.139:0/3801692437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe840cd1f0 0x7fbe840c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.414+0000 7fbe89888640 1 -- 172.21.15.139:0/3801692437 >> 172.21.15.139:0/3801692437 conn(0x7fbe84066f20 msgr2=0x7fbe840b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.414+0000 7fbe89888640 1 -- 172.21.15.139:0/3801692437 shutdown_connections 2024-04-25T09:54:16.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.414+0000 7fbe89888640 1 -- 172.21.15.139:0/3801692437 wait complete. 2024-04-25T09:54:16.432 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T09:54:16.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.527+0000 7fde1f007640 1 Processor -- start 2024-04-25T09:54:16.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.527+0000 7fde1f007640 1 -- start start 2024-04-25T09:54:16.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1f007640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fde180cd610 0x7fde180d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1f007640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1809ad40 0x7fde180d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1f007640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde18099ca0 con 0x7fde181618c0 2024-04-25T09:54:16.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1f007640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde18099a50 con 0x7fde180cd610 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1f007640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde18099f80 con 0x7fde1809ad40 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1cda3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fde180cd610 0x7fde180d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1d5a4640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fde181618c0 0x7fde1806be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54886/0 (socket says 172.21.15.139:54886) 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1cda3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fde180cd610 0x7fde180d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41940/0 (socket says 172.21.15.139:41940) 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1d5a4640 1 -- 172.21.15.139:0/1561658412 learned_addr learned my addr 172.21.15.139:0/1561658412 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde17fff640 1 --2- 172.21.15.139:0/1561658412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1809ad40 0x7fde180d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1cda3640 1 -- 172.21.15.139:0/1561658412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1809ad40 msgr2=0x7fde180d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1cda3640 1 --2- 172.21.15.139:0/1561658412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1809ad40 0x7fde180d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1cda3640 1 -- 172.21.15.139:0/1561658412 >> v1:172.21.15.139:6789/0 conn(0x7fde181618c0 legacy=0x7fde1806be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde1cda3640 1 -- 172.21.15.139:0/1561658412 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1809a260 con 0x7fde180cd610 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.528+0000 7fde177fe640 1 -- 172.21.15.139:0/1561658412 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818956833 0 0) 0x7fde18099ca0 con 0x7fde181618c0 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.529+0000 7fde1cda3640 1 --2- 172.21.15.139:0/1561658412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde180cd610 0x7fde180d0370 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fde080097a0 tx=0x7fde0802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8719934323f952f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.529+0000 7fde177fe640 1 -- 172.21.15.139:0/1561658412 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08037070 con 0x7fde180cd610 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.529+0000 7fde177fe640 1 -- 172.21.15.139:0/1561658412 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde08032440 con 0x7fde180cd610 2024-04-25T09:54:16.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.529+0000 7fde177fe640 1 -- 172.21.15.139:0/1561658412 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08032810 con 0x7fde180cd610 2024-04-25T09:54:16.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.530+0000 7fde1f007640 1 -- 172.21.15.139:0/1561658412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde180cd610 msgr2=0x7fde180d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.530+0000 7fde1f007640 1 --2- 172.21.15.139:0/1561658412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde180cd610 0x7fde180d0370 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fde080097a0 tx=0x7fde0802a210 comp rx=0 tx=0).stop 2024-04-25T09:54:16.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.530+0000 7fde1f007640 1 -- 172.21.15.139:0/1561658412 shutdown_connections 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.530+0000 7fde1f007640 1 --2- 172.21.15.139:0/1561658412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1809ad40 0x7fde180d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.530+0000 7fde1f007640 1 --2- 172.21.15.139:0/1561658412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde180cd610 0x7fde180d0370 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.530+0000 7fde1f007640 1 -- 172.21.15.139:0/1561658412 >> 172.21.15.139:0/1561658412 conn(0x7fde180b31f0 msgr2=0x7fde180b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.530+0000 7fde1f007640 1 -- 172.21.15.139:0/1561658412 shutdown_connections 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.530+0000 7fde1f007640 1 -- 172.21.15.139:0/1561658412 wait complete. 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.531+0000 7fde1f007640 1 Processor -- start 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.531+0000 7fde1f007640 1 -- start start 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde1f007640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1809ad40 0x7fde1809ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde1f007640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde180cd610 0x7fde1806bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde1f007640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde181618c0 0x7fde1806c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde1f007640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde180cd9f0 con 0x7fde180cd610 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde1f007640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1809a820 con 0x7fde181618c0 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde1f007640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde180cdb70 con 0x7fde1809ad40 2024-04-25T09:54:16.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde17fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde180cd610 0x7fde1806bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde17fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde180cd610 0x7fde1806bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41944/0 (socket says 172.21.15.139:41944) 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde17fff640 1 -- 172.21.15.139:0/2772461471 learned_addr learned my addr 172.21.15.139:0/2772461471 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde1cda3640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1809ad40 0x7fde1809ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde1d5a4640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde181618c0 0x7fde1806c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde17fff640 1 -- 172.21.15.139:0/2772461471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1809ad40 msgr2=0x7fde1809ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde17fff640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1809ad40 0x7fde1809ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde17fff640 1 -- 172.21.15.139:0/2772461471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde181618c0 msgr2=0x7fde1806c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde17fff640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde181618c0 0x7fde1806c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde17fff640 1 -- 172.21.15.139:0/2772461471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde181f8230 con 0x7fde180cd610 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde17fff640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde180cd610 0x7fde1806bc70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00e9b0 tx=0x7fde0c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.532+0000 7fde157fa640 1 -- 172.21.15.139:0/2772461471 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c017070 con 0x7fde180cd610 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.533+0000 7fde157fa640 1 -- 172.21.15.139:0/2772461471 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c00c9a0 con 0x7fde180cd610 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.533+0000 7fde1f007640 1 -- 172.21.15.139:0/2772461471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde18160260 con 0x7fde180cd610 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.533+0000 7fde157fa640 1 -- 172.21.15.139:0/2772461471 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00cd90 con 0x7fde180cd610 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.533+0000 7fde1f007640 1 -- 172.21.15.139:0/2772461471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde18160820 con 0x7fde180cd610 2024-04-25T09:54:16.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.533+0000 7fde1f007640 1 -- 172.21.15.139:0/2772461471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde18099ca0 con 0x7fde180cd610 2024-04-25T09:54:16.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.534+0000 7fde157fa640 1 -- 172.21.15.139:0/2772461471 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde0c01a070 con 0x7fde180cd610 2024-04-25T09:54:16.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.534+0000 7fde157fa640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fde0406b500 0x7fde0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.534+0000 7fde157fa640 1 -- 172.21.15.139:0/2772461471 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fde0c089f50 con 0x7fde180cd610 2024-04-25T09:54:16.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.534+0000 7fde1cda3640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fde0406b500 0x7fde0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.535+0000 7fde1cda3640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fde0406b500 0x7fde0406d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fde08034000 tx=0x7fde0802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.538+0000 7fde157fa640 1 -- 172.21.15.139:0/2772461471 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde18099ca0 con 0x7fde180cd610 2024-04-25T09:54:16.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.777+0000 7fde1f007640 1 -- 172.21.15.139:0/2772461471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fde1814f400 con 0x7fde180cd610 2024-04-25T09:54:16.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.778+0000 7fde157fa640 1 -- 172.21.15.139:0/2772461471 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7fde1814f400 con 0x7fde180cd610 2024-04-25T09:54:16.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.781+0000 7fde1f007640 1 -- 172.21.15.139:0/2772461471 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fde0406b500 msgr2=0x7fde0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.781+0000 7fde1f007640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fde0406b500 0x7fde0406d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fde08034000 tx=0x7fde0802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:16.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.781+0000 7fde1f007640 1 -- 172.21.15.139:0/2772461471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde180cd610 msgr2=0x7fde1806bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.781+0000 7fde1f007640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde180cd610 0x7fde1806bc70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00e9b0 tx=0x7fde0c00ee70 comp rx=0 tx=0).stop 2024-04-25T09:54:16.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.781+0000 7fde1f007640 1 -- 172.21.15.139:0/2772461471 shutdown_connections 2024-04-25T09:54:16.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.781+0000 7fde1f007640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde180cd610 0x7fde1806bc70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.781+0000 7fde1f007640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fde0406b500 0x7fde0406d9c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.781+0000 7fde1f007640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde181618c0 0x7fde1806c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.781+0000 7fde1f007640 1 --2- 172.21.15.139:0/2772461471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1809ad40 0x7fde1809ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.781+0000 7fde1f007640 1 -- 172.21.15.139:0/2772461471 >> 172.21.15.139:0/2772461471 conn(0x7fde180b31f0 msgr2=0x7fde18070060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.781+0000 7fde1f007640 1 -- 172.21.15.139:0/2772461471 shutdown_connections 2024-04-25T09:54:16.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.781+0000 7fde1f007640 1 -- 172.21.15.139:0/2772461471 wait complete. 2024-04-25T09:54:16.795 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T09:54:16.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.889+0000 7f9c7f70a640 1 Processor -- start 2024-04-25T09:54:16.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.889+0000 7f9c7f70a640 1 -- start start 2024-04-25T09:54:16.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.890+0000 7f9c7f70a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9c780d0a80 0x7f9c781618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.890+0000 7f9c7f70a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c7809ad40 0x7f9c78161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.890+0000 7f9c7f70a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c78099ca0 con 0x7f9c780cd610 2024-04-25T09:54:16.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.890+0000 7f9c7f70a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c78099a50 con 0x7f9c780d0a80 2024-04-25T09:54:16.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.890+0000 7f9c7f70a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c78099f80 con 0x7f9c7809ad40 2024-04-25T09:54:16.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.890+0000 7f9c7cca5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9c780d0a80 0x7f9c781618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.890+0000 7f9c7dca7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c7809ad40 0x7f9c78161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.890+0000 7f9c7d4a6640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9c780cd610 0x7f9c780d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:42666/0 (socket says 172.21.15.139:42666) 2024-04-25T09:54:16.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.890+0000 7f9c7d4a6640 1 -- 172.21.15.139:0/643332540 learned_addr learned my addr 172.21.15.139:0/643332540 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:16.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.891+0000 7f9c7cca5640 1 -- 172.21.15.139:0/643332540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c7809ad40 msgr2=0x7f9c78161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:16.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.891+0000 7f9c7cca5640 1 --2- 172.21.15.139:0/643332540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c7809ad40 0x7f9c78161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.891+0000 7f9c7cca5640 1 -- 172.21.15.139:0/643332540 >> v1:172.21.15.150:6789/0 conn(0x7f9c780cd610 legacy=0x7f9c780d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.891+0000 7f9c7cca5640 1 -- 172.21.15.139:0/643332540 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7809a260 con 0x7f9c780d0a80 2024-04-25T09:54:16.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.891+0000 7f9c6ffff640 1 -- 172.21.15.139:0/643332540 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131069902 0 0) 0x7f9c78099ca0 con 0x7f9c780cd610 2024-04-25T09:54:16.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.891+0000 7f9c7dca7640 1 --2- 172.21.15.139:0/643332540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c7809ad40 0x7f9c78161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:16.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.891+0000 7f9c7cca5640 1 --2- 172.21.15.139:0/643332540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780d0a80 0x7f9c781618c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9c680097a0 tx=0x7f9c6802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f354afc02c6dd93b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.892+0000 7f9c6ffff640 1 -- 172.21.15.139:0/643332540 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c68037070 con 0x7f9c780d0a80 2024-04-25T09:54:16.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.892+0000 7f9c6ffff640 1 -- 172.21.15.139:0/643332540 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c68032440 con 0x7f9c780d0a80 2024-04-25T09:54:16.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.892+0000 7f9c6ffff640 1 -- 172.21.15.139:0/643332540 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c68032810 con 0x7f9c780d0a80 2024-04-25T09:54:16.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.892+0000 7f9c7f70a640 1 -- 172.21.15.139:0/643332540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780d0a80 msgr2=0x7f9c781618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.892+0000 7f9c7f70a640 1 --2- 172.21.15.139:0/643332540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780d0a80 0x7f9c781618c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9c680097a0 tx=0x7f9c6802a210 comp rx=0 tx=0).stop 2024-04-25T09:54:16.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.892+0000 7f9c7f70a640 1 -- 172.21.15.139:0/643332540 shutdown_connections 2024-04-25T09:54:16.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.892+0000 7f9c7f70a640 1 --2- 172.21.15.139:0/643332540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c7809ad40 0x7f9c78161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.892+0000 7f9c7f70a640 1 --2- 172.21.15.139:0/643332540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780d0a80 0x7f9c781618c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.892+0000 7f9c7f70a640 1 -- 172.21.15.139:0/643332540 >> 172.21.15.139:0/643332540 conn(0x7f9c780b31f0 msgr2=0x7f9c780b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.892+0000 7f9c7f70a640 1 -- 172.21.15.139:0/643332540 shutdown_connections 2024-04-25T09:54:16.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.892+0000 7f9c7f70a640 1 -- 172.21.15.139:0/643332540 wait complete. 2024-04-25T09:54:16.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.893+0000 7f9c7f70a640 1 Processor -- start 2024-04-25T09:54:16.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.894+0000 7f9c7f70a640 1 -- start start 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.894+0000 7f9c7f70a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7809ad40 0x7f9c780c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.894+0000 7f9c7f70a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c780cd610 0x7f9c781e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.894+0000 7f9c7f70a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780d0a80 0x7f9c7815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.894+0000 7f9c7f70a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780cd9f0 con 0x7f9c7809ad40 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.894+0000 7f9c7f70a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7809a820 con 0x7f9c780d0a80 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.894+0000 7f9c7f70a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780cdb70 con 0x7f9c780cd610 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.894+0000 7f9c7d4a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7809ad40 0x7f9c780c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.894+0000 7f9c7d4a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7809ad40 0x7f9c780c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41954/0 (socket says 172.21.15.139:41954) 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.894+0000 7f9c7d4a6640 1 -- 172.21.15.139:0/1882377714 learned_addr learned my addr 172.21.15.139:0/1882377714 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c7cca5640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c780cd610 0x7f9c781e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c7dca7640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780d0a80 0x7f9c7815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c7d4a6640 1 -- 172.21.15.139:0/1882377714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c780cd610 msgr2=0x7f9c781e84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c7d4a6640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c780cd610 0x7f9c781e84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c7d4a6640 1 -- 172.21.15.139:0/1882377714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780d0a80 msgr2=0x7f9c7815eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c7d4a6640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780d0a80 0x7f9c7815eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c7d4a6640 1 -- 172.21.15.139:0/1882377714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c780c6dd0 con 0x7f9c7809ad40 2024-04-25T09:54:16.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c7cca5640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c780cd610 0x7f9c781e84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:16.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c7dca7640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780d0a80 0x7f9c7815eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:16.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c7d4a6640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7809ad40 0x7f9c780c5c40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400c4d0 tx=0x7f9c7400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c6dffb640 1 -- 172.21.15.139:0/1882377714 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c74018070 con 0x7f9c7809ad40 2024-04-25T09:54:16.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c6dffb640 1 -- 172.21.15.139:0/1882377714 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c74007500 con 0x7f9c7809ad40 2024-04-25T09:54:16.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c6dffb640 1 -- 172.21.15.139:0/1882377714 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c74004a20 con 0x7f9c7809ad40 2024-04-25T09:54:16.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c7f70a640 1 -- 172.21.15.139:0/1882377714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c780c70b0 con 0x7f9c7809ad40 2024-04-25T09:54:16.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.895+0000 7f9c7f70a640 1 -- 172.21.15.139:0/1882377714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c780c7670 con 0x7f9c7809ad40 2024-04-25T09:54:16.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.896+0000 7f9c7f70a640 1 -- 172.21.15.139:0/1882377714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c78074520 con 0x7f9c7809ad40 2024-04-25T09:54:16.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.897+0000 7f9c6dffb640 1 -- 172.21.15.139:0/1882377714 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c7401b070 con 0x7f9c7809ad40 2024-04-25T09:54:16.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.897+0000 7f9c6dffb640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c5c06b4d0 0x7f9c5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.897+0000 7f9c7cca5640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c5c06b4d0 0x7f9c5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.897+0000 7f9c6dffb640 1 -- 172.21.15.139:0/1882377714 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9c740892f0 con 0x7f9c7809ad40 2024-04-25T09:54:16.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.898+0000 7f9c7cca5640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c5c06b4d0 0x7f9c5c06d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c68034000 tx=0x7f9c6802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:16.902+0000 7f9c6dffb640 1 -- 172.21.15.139:0/1882377714 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c740583c0 con 0x7f9c7809ad40 2024-04-25T09:54:17.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.144+0000 7f9c7f70a640 1 -- 172.21.15.139:0/1882377714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f9c780b3f70 con 0x7f9c7809ad40 2024-04-25T09:54:17.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.145+0000 7f9c6dffb640 1 -- 172.21.15.139:0/1882377714 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9c7405e0c0 con 0x7f9c7809ad40 2024-04-25T09:54:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.147+0000 7f9c7f70a640 1 -- 172.21.15.139:0/1882377714 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c5c06b4d0 msgr2=0x7f9c5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.147+0000 7f9c7f70a640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c5c06b4d0 0x7f9c5c06d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c68034000 tx=0x7f9c6802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.147+0000 7f9c7f70a640 1 -- 172.21.15.139:0/1882377714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7809ad40 msgr2=0x7f9c780c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.147+0000 7f9c7f70a640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7809ad40 0x7f9c780c5c40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400c4d0 tx=0x7f9c7400c990 comp rx=0 tx=0).stop 2024-04-25T09:54:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.148+0000 7f9c7f70a640 1 -- 172.21.15.139:0/1882377714 shutdown_connections 2024-04-25T09:54:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.148+0000 7f9c7f70a640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c5c06b4d0 0x7f9c5c06d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.148+0000 7f9c7f70a640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780d0a80 0x7f9c7815eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.148+0000 7f9c7f70a640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c780cd610 0x7f9c781e84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.148+0000 7f9c7f70a640 1 --2- 172.21.15.139:0/1882377714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7809ad40 0x7f9c780c5c40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.148+0000 7f9c7f70a640 1 -- 172.21.15.139:0/1882377714 >> 172.21.15.139:0/1882377714 conn(0x7f9c780b31f0 msgr2=0x7f9c78070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.148+0000 7f9c7f70a640 1 -- 172.21.15.139:0/1882377714 shutdown_connections 2024-04-25T09:54:17.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.148+0000 7f9c7f70a640 1 -- 172.21.15.139:0/1882377714 wait complete. 2024-04-25T09:54:17.161 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T09:54:17.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.256+0000 7fe382ebd640 1 Processor -- start 2024-04-25T09:54:17.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.256+0000 7fe382ebd640 1 -- start start 2024-04-25T09:54:17.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.256+0000 7fe382ebd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c0af8a0 0x7fe37c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.256+0000 7fe382ebd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe37c0afc80 0x7fe37c0cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.256+0000 7fe382ebd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37c0c6cc0 con 0x7fe37c06b700 2024-04-25T09:54:17.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.256+0000 7fe382ebd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c0c6fa0 con 0x7fe37c0afc80 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.256+0000 7fe382ebd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0c7280 con 0x7fe37c0af8a0 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe38145a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe37c06b700 0x7fe37c06ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54894/0 (socket says 172.21.15.139:54894) 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe38145a640 1 -- 172.21.15.139:0/816475060 learned_addr learned my addr 172.21.15.139:0/816475060 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128763620 0 0) 0x7fe37c0c6cc0 con 0x7fe37c06b700 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe36c003520 con 0x7fe37c06b700 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe380c59640 1 --2- 172.21.15.139:0/816475060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c0af8a0 0x7fe37c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37bfff640 1 --2- 172.21.15.139:0/816475060 >> v2:172.21.15.150:3300/0 conn(0x7fe37c0afc80 0x7fe37c0cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2342800512 0 0) 0x7fe36c003520 con 0x7fe37c06b700 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c0af8a0 msgr2=0x7fe37c0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 --2- 172.21.15.139:0/816475060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c0af8a0 0x7fe37c0b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 >> v2:172.21.15.150:3300/0 conn(0x7fe37c0afc80 msgr2=0x7fe37c0cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 --2- 172.21.15.139:0/816475060 >> v2:172.21.15.150:3300/0 conn(0x7fe37c0afc80 0x7fe37c0cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0cd090 con 0x7fe37c06b700 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fe3740019f0 con 0x7fe37c06b700 2024-04-25T09:54:17.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 >> v1:172.21.15.139:6789/0 conn(0x7fe37c06b700 legacy=0x7fe37c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 --2- 172.21.15.139:0/816475060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c0af8a0 0x7fe36c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 --2- 172.21.15.139:0/816475060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c003f30 0x7fe36c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 --2- 172.21.15.139:0/816475060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe36c004d40 0x7fe36c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0c6fa0 con 0x7fe36c003f30 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0c7280 con 0x7fe36c004d40 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c00b9c0 con 0x7fe37c0af8a0 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe37bfff640 1 --2- 172.21.15.139:0/816475060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c003f30 0x7fe36c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe380c59640 1 --2- 172.21.15.139:0/816475060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c0af8a0 0x7fe36c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.257+0000 7fe38145a640 1 --2- 172.21.15.139:0/816475060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe36c004d40 0x7fe36c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.258+0000 7fe37bfff640 1 -- 172.21.15.139:0/816475060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c0af8a0 msgr2=0x7fe36c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.258+0000 7fe37bfff640 1 --2- 172.21.15.139:0/816475060 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c0af8a0 0x7fe36c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.258+0000 7fe37bfff640 1 -- 172.21.15.139:0/816475060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe36c004d40 msgr2=0x7fe36c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.258+0000 7fe37bfff640 1 --2- 172.21.15.139:0/816475060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe36c004d40 0x7fe36c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.258+0000 7fe37bfff640 1 -- 172.21.15.139:0/816475060 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c003520 con 0x7fe36c003f30 2024-04-25T09:54:17.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.258+0000 7fe37bfff640 1 --2- 172.21.15.139:0/816475060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c003f30 0x7fe36c004800 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe368009ac0 tx=0x7fe36c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59a8b5293fa44d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.258+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368036070 con 0x7fe36c003f30 2024-04-25T09:54:17.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.258+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe368031400 con 0x7fe36c003f30 2024-04-25T09:54:17.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.258+0000 7fe37b7fe640 1 -- 172.21.15.139:0/816475060 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368009690 con 0x7fe36c003f30 2024-04-25T09:54:17.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.259+0000 7fe382ebd640 1 -- 172.21.15.139:0/816475060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c003f30 msgr2=0x7fe36c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.259+0000 7fe382ebd640 1 --2- 172.21.15.139:0/816475060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c003f30 0x7fe36c004800 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe368009ac0 tx=0x7fe36c005830 comp rx=0 tx=0).stop 2024-04-25T09:54:17.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.259+0000 7fe38145a640 1 -- 172.21.15.139:0/816475060 reap_dead start 2024-04-25T09:54:17.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.259+0000 7fe382ebd640 1 -- 172.21.15.139:0/816475060 shutdown_connections 2024-04-25T09:54:17.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.259+0000 7fe382ebd640 1 -- 172.21.15.139:0/816475060 >> 172.21.15.139:0/816475060 conn(0x7fe37c0b2230 msgr2=0x7fe37c0b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.259+0000 7fe382ebd640 1 -- 172.21.15.139:0/816475060 shutdown_connections 2024-04-25T09:54:17.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.259+0000 7fe382ebd640 1 -- 172.21.15.139:0/816475060 wait complete. 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.260+0000 7fe382ebd640 1 Processor -- start 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.260+0000 7fe382ebd640 1 -- start start 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe382ebd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c003f30 0x7fe37c09dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe382ebd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15a800 0x7fe37c09b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe382ebd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0adca0 0x7fe37c0ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe382ebd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c06b270 con 0x7fe36c003f30 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe382ebd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0cddd0 con 0x7fe37c0adca0 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe382ebd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c06b3f0 con 0x7fe37c15a800 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe380c59640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c003f30 0x7fe37c09dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe380c59640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c003f30 0x7fe37c09dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41980/0 (socket says 172.21.15.139:41980) 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe380c59640 1 -- 172.21.15.139:0/442727431 learned_addr learned my addr 172.21.15.139:0/442727431 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe37bfff640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15a800 0x7fe37c09b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe38145a640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0adca0 0x7fe37c0ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe380c59640 1 -- 172.21.15.139:0/442727431 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15a800 msgr2=0x7fe37c09b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe380c59640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15a800 0x7fe37c09b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe380c59640 1 -- 172.21.15.139:0/442727431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0adca0 msgr2=0x7fe37c0ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe380c59640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0adca0 0x7fe37c0ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe380c59640 1 -- 172.21.15.139:0/442727431 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c15ac80 con 0x7fe36c003f30 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe37bfff640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15a800 0x7fe37c09b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe38145a640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0adca0 0x7fe37c0ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.261+0000 7fe380c59640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c003f30 0x7fe37c09dba0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe3700119d0 tx=0x7fe370011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.262+0000 7fe3797fa640 1 -- 172.21.15.139:0/442727431 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37000fb50 con 0x7fe36c003f30 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.262+0000 7fe3797fa640 1 -- 172.21.15.139:0/442727431 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37000fcf0 con 0x7fe36c003f30 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.262+0000 7fe3797fa640 1 -- 172.21.15.139:0/442727431 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37000fea0 con 0x7fe36c003f30 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.262+0000 7fe382ebd640 1 -- 172.21.15.139:0/442727431 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c09ba60 con 0x7fe36c003f30 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.262+0000 7fe382ebd640 1 -- 172.21.15.139:0/442727431 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c09bd00 con 0x7fe36c003f30 2024-04-25T09:54:17.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.263+0000 7fe382ebd640 1 -- 172.21.15.139:0/442727431 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37c074520 con 0x7fe36c003f30 2024-04-25T09:54:17.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.263+0000 7fe3797fa640 1 -- 172.21.15.139:0/442727431 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe370012040 con 0x7fe36c003f30 2024-04-25T09:54:17.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.263+0000 7fe3797fa640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe36006b4d0 0x7fe36006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.264+0000 7fe3797fa640 1 -- 172.21.15.139:0/442727431 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe37008a5e0 con 0x7fe36c003f30 2024-04-25T09:54:17.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.264+0000 7fe37bfff640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe36006b4d0 0x7fe36006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.264+0000 7fe37bfff640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe36006b4d0 0x7fe36006d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe368033000 tx=0x7fe368008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.268+0000 7fe3797fa640 1 -- 172.21.15.139:0/442727431 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe370019030 con 0x7fe36c003f30 2024-04-25T09:54:17.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.505+0000 7fe382ebd640 1 -- 172.21.15.139:0/442727431 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fe37c15bfc0 con 0x7fe36c003f30 2024-04-25T09:54:17.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.507+0000 7fe3797fa640 1 -- 172.21.15.139:0/442727431 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe370059620 con 0x7fe36c003f30 2024-04-25T09:54:17.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.509+0000 7fe382ebd640 1 -- 172.21.15.139:0/442727431 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe36006b4d0 msgr2=0x7fe36006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.509+0000 7fe382ebd640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe36006b4d0 0x7fe36006d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe368033000 tx=0x7fe368008030 comp rx=0 tx=0).stop 2024-04-25T09:54:17.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.509+0000 7fe382ebd640 1 -- 172.21.15.139:0/442727431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c003f30 msgr2=0x7fe37c09dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.509+0000 7fe382ebd640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c003f30 0x7fe37c09dba0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe3700119d0 tx=0x7fe370011e90 comp rx=0 tx=0).stop 2024-04-25T09:54:17.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.509+0000 7fe382ebd640 1 -- 172.21.15.139:0/442727431 shutdown_connections 2024-04-25T09:54:17.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.509+0000 7fe382ebd640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe36006b4d0 0x7fe36006d990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.509+0000 7fe382ebd640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0adca0 0x7fe37c0ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.509+0000 7fe382ebd640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe37c15a800 0x7fe37c09b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.509+0000 7fe382ebd640 1 --2- 172.21.15.139:0/442727431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe36c003f30 0x7fe37c09dba0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.509+0000 7fe382ebd640 1 -- 172.21.15.139:0/442727431 >> 172.21.15.139:0/442727431 conn(0x7fe37c0b2230 msgr2=0x7fe37c0ad510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.509+0000 7fe382ebd640 1 -- 172.21.15.139:0/442727431 shutdown_connections 2024-04-25T09:54:17.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.509+0000 7fe382ebd640 1 -- 172.21.15.139:0/442727431 wait complete. 2024-04-25T09:54:17.523 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T09:54:17.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.617+0000 7f8db495c640 1 Processor -- start 2024-04-25T09:54:17.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.617+0000 7f8db495c640 1 -- start start 2024-04-25T09:54:17.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.618+0000 7f8db495c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8db015db30 0x7f8db00b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.618+0000 7f8db495c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8db00b3c30 0x7f8db006ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.618+0000 7f8db495c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db006fc70 con 0x7f8db006b0a0 2024-04-25T09:54:17.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.618+0000 7f8db495c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db015df10 con 0x7f8db00b3c30 2024-04-25T09:54:17.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.618+0000 7f8db495c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db006ff50 con 0x7f8db015db30 2024-04-25T09:54:17.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.618+0000 7f8dadd9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8db00b3c30 0x7f8db006ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.618+0000 7f8daed9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8db006b0a0 0x7f8db00c9a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54896/0 (socket says 172.21.15.139:54896) 2024-04-25T09:54:17.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.619+0000 7f8dadd9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8db00b3c30 0x7f8db006ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41988/0 (socket says 172.21.15.139:41988) 2024-04-25T09:54:17.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.619+0000 7f8daed9d640 1 -- 172.21.15.139:0/1545122695 learned_addr learned my addr 172.21.15.139:0/1545122695 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:17.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.619+0000 7f8dae59c640 1 --2- 172.21.15.139:0/1545122695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8db015db30 0x7f8db00b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.619+0000 7f8dadd9b640 1 -- 172.21.15.139:0/1545122695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8db015db30 msgr2=0x7f8db00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.619+0000 7f8dadd9b640 1 --2- 172.21.15.139:0/1545122695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8db015db30 0x7f8db00b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.619+0000 7f8dadd9b640 1 -- 172.21.15.139:0/1545122695 >> v1:172.21.15.139:6789/0 conn(0x7f8db006b0a0 legacy=0x7f8db00c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.619+0000 7f8dad59a640 1 -- 172.21.15.139:0/1545122695 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58442454 0 0) 0x7f8db006fc70 con 0x7f8db006b0a0 2024-04-25T09:54:17.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.619+0000 7f8dadd9b640 1 -- 172.21.15.139:0/1545122695 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db0070230 con 0x7f8db00b3c30 2024-04-25T09:54:17.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.619+0000 7f8dae59c640 1 --2- 172.21.15.139:0/1545122695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8db015db30 0x7f8db00b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:17.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.619+0000 7f8dadd9b640 1 --2- 172.21.15.139:0/1545122695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8db00b3c30 0x7f8db006ab60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8da40097a0 tx=0x7f8da402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be516744ab62f95b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.620+0000 7f8dad59a640 1 -- 172.21.15.139:0/1545122695 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4037070 con 0x7f8db00b3c30 2024-04-25T09:54:17.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.620+0000 7f8dad59a640 1 -- 172.21.15.139:0/1545122695 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da4032440 con 0x7f8db00b3c30 2024-04-25T09:54:17.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.620+0000 7f8dad59a640 1 -- 172.21.15.139:0/1545122695 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4032810 con 0x7f8db00b3c30 2024-04-25T09:54:17.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.620+0000 7f8db495c640 1 -- 172.21.15.139:0/1545122695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8db00b3c30 msgr2=0x7f8db006ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.620+0000 7f8db495c640 1 --2- 172.21.15.139:0/1545122695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8db00b3c30 0x7f8db006ab60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8da40097a0 tx=0x7f8da402a180 comp rx=0 tx=0).stop 2024-04-25T09:54:17.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.620+0000 7f8db495c640 1 -- 172.21.15.139:0/1545122695 shutdown_connections 2024-04-25T09:54:17.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.620+0000 7f8db495c640 1 --2- 172.21.15.139:0/1545122695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8db00b3c30 0x7f8db006ab60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.620+0000 7f8db495c640 1 --2- 172.21.15.139:0/1545122695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8db015db30 0x7f8db00b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.620+0000 7f8db495c640 1 -- 172.21.15.139:0/1545122695 >> 172.21.15.139:0/1545122695 conn(0x7f8db00b23f0 msgr2=0x7f8db00b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.620+0000 7f8db495c640 1 -- 172.21.15.139:0/1545122695 shutdown_connections 2024-04-25T09:54:17.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.620+0000 7f8db495c640 1 -- 172.21.15.139:0/1545122695 wait complete. 2024-04-25T09:54:17.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.622+0000 7f8db495c640 1 Processor -- start 2024-04-25T09:54:17.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.622+0000 7f8db495c640 1 -- start start 2024-04-25T09:54:17.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.622+0000 7f8db495c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8db006b0a0 0x7f8db01609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.622+0000 7f8db495c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8db00b3c30 0x7f8db00d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.622+0000 7f8db495c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db015db30 0x7f8db015c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.622+0000 7f8db495c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00cc0f0 con 0x7f8db00b3c30 2024-04-25T09:54:17.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.622+0000 7f8db495c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00cbf70 con 0x7f8db015db30 2024-04-25T09:54:17.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.622+0000 7f8db495c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00cc270 con 0x7f8db006b0a0 2024-04-25T09:54:17.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.622+0000 7f8dadd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8db00b3c30 0x7f8db00d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.622+0000 7f8dadd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8db00b3c30 0x7f8db00d0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41998/0 (socket says 172.21.15.139:41998) 2024-04-25T09:54:17.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.622+0000 7f8dadd9b640 1 -- 172.21.15.139:0/2738304534 learned_addr learned my addr 172.21.15.139:0/2738304534 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:17.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.623+0000 7f8dadd9b640 1 -- 172.21.15.139:0/2738304534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8db006b0a0 msgr2=0x7f8db01609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.623+0000 7f8dadd9b640 1 --2- 172.21.15.139:0/2738304534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8db006b0a0 0x7f8db01609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.623+0000 7f8dadd9b640 1 -- 172.21.15.139:0/2738304534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db015db30 msgr2=0x7f8db015c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.623+0000 7f8dadd9b640 1 --2- 172.21.15.139:0/2738304534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db015db30 0x7f8db015c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.623+0000 7f8dadd9b640 1 -- 172.21.15.139:0/2738304534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db01f38f0 con 0x7f8db00b3c30 2024-04-25T09:54:17.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.623+0000 7f8dadd9b640 1 --2- 172.21.15.139:0/2738304534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8db00b3c30 0x7f8db00d0520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8da40097a0 tx=0x7f8da40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.623+0000 7f8d9effd640 1 -- 172.21.15.139:0/2738304534 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4037050 con 0x7f8db00b3c30 2024-04-25T09:54:17.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.623+0000 7f8d9effd640 1 -- 172.21.15.139:0/2738304534 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da4038040 con 0x7f8db00b3c30 2024-04-25T09:54:17.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.623+0000 7f8db495c640 1 -- 172.21.15.139:0/2738304534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db01f3bd0 con 0x7f8db00b3c30 2024-04-25T09:54:17.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.623+0000 7f8d9effd640 1 -- 172.21.15.139:0/2738304534 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4004490 con 0x7f8db00b3c30 2024-04-25T09:54:17.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.623+0000 7f8db495c640 1 -- 172.21.15.139:0/2738304534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db01f4190 con 0x7f8db00b3c30 2024-04-25T09:54:17.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.624+0000 7f8db495c640 1 -- 172.21.15.139:0/2738304534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db0074520 con 0x7f8db00b3c30 2024-04-25T09:54:17.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.625+0000 7f8d9effd640 1 -- 172.21.15.139:0/2738304534 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8da4030980 con 0x7f8db00b3c30 2024-04-25T09:54:17.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.625+0000 7f8d9effd640 1 --2- 172.21.15.139:0/2738304534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8d9006b4d0 0x7f8d9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.625+0000 7f8d9effd640 1 -- 172.21.15.139:0/2738304534 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8da40aaba0 con 0x7f8db00b3c30 2024-04-25T09:54:17.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.625+0000 7f8dae59c640 1 --2- 172.21.15.139:0/2738304534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8d9006b4d0 0x7f8d9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.625+0000 7f8dae59c640 1 --2- 172.21.15.139:0/2738304534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8d9006b4d0 0x7f8d9006d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8da0008e00 tx=0x7f8da0008330 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.629+0000 7f8d9effd640 1 -- 172.21.15.139:0/2738304534 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8da4079d90 con 0x7f8db00b3c30 2024-04-25T09:54:17.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.865+0000 7f8db495c640 1 -- 172.21.15.139:0/2738304534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f8db00c9ba0 con 0x7f8db00b3c30 2024-04-25T09:54:17.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.867+0000 7f8d9effd640 1 -- 172.21.15.139:0/2738304534 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8da4079690 con 0x7f8db00b3c30 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.869+0000 7f8db495c640 1 -- 172.21.15.139:0/2738304534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8d9006b4d0 msgr2=0x7f8d9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.869+0000 7f8db495c640 1 --2- 172.21.15.139:0/2738304534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8d9006b4d0 0x7f8d9006d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8da0008e00 tx=0x7f8da0008330 comp rx=0 tx=0).stop 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.869+0000 7f8db495c640 1 -- 172.21.15.139:0/2738304534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8db00b3c30 msgr2=0x7f8db00d0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.869+0000 7f8db495c640 1 --2- 172.21.15.139:0/2738304534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8db00b3c30 0x7f8db00d0520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8da40097a0 tx=0x7f8da40046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.869+0000 7f8db495c640 1 -- 172.21.15.139:0/2738304534 shutdown_connections 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.869+0000 7f8db495c640 1 --2- 172.21.15.139:0/2738304534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8db00b3c30 0x7f8db00d0520 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.869+0000 7f8db495c640 1 --2- 172.21.15.139:0/2738304534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8d9006b4d0 0x7f8d9006d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.869+0000 7f8db495c640 1 --2- 172.21.15.139:0/2738304534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db015db30 0x7f8db015c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.869+0000 7f8db495c640 1 --2- 172.21.15.139:0/2738304534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8db006b0a0 0x7f8db01609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.869+0000 7f8db495c640 1 -- 172.21.15.139:0/2738304534 >> 172.21.15.139:0/2738304534 conn(0x7f8db00b23f0 msgr2=0x7f8db00ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.869+0000 7f8db495c640 1 -- 172.21.15.139:0/2738304534 shutdown_connections 2024-04-25T09:54:17.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.869+0000 7f8db495c640 1 -- 172.21.15.139:0/2738304534 wait complete. 2024-04-25T09:54:17.883 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T09:54:17.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.978+0000 7facf5f4c640 1 Processor -- start 2024-04-25T09:54:17.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.978+0000 7facf5f4c640 1 -- start start 2024-04-25T09:54:17.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.979+0000 7facf5f4c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7facf00afc00 0x7facf006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.979+0000 7facf5f4c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7facf00d0080 0x7facf006e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.979+0000 7facf5f4c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf00ccef0 con 0x7facf015cb20 2024-04-25T09:54:17.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.979+0000 7facf5f4c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf00703c0 con 0x7facf00d0080 2024-04-25T09:54:17.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.979+0000 7facf5f4c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf00cd820 con 0x7facf00afc00 2024-04-25T09:54:17.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.979+0000 7facef7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7facf00afc00 0x7facf006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.979+0000 7faceffff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7facf00d0080 0x7facf006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.979+0000 7faceeffd640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7facf015cb20 0x7facf00cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:42668/0 (socket says 172.21.15.139:42668) 2024-04-25T09:54:17.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.979+0000 7faceeffd640 1 -- 172.21.15.139:0/912257879 learned_addr learned my addr 172.21.15.139:0/912257879 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:17.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.980+0000 7facef7fe640 1 -- 172.21.15.139:0/912257879 >> v2:172.21.15.150:3300/0 conn(0x7facf00d0080 msgr2=0x7facf006e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.980+0000 7facef7fe640 1 --2- 172.21.15.139:0/912257879 >> v2:172.21.15.150:3300/0 conn(0x7facf00d0080 0x7facf006e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.980+0000 7facef7fe640 1 -- 172.21.15.139:0/912257879 >> v1:172.21.15.150:6789/0 conn(0x7facf015cb20 legacy=0x7facf00cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.980+0000 7facef7fe640 1 -- 172.21.15.139:0/912257879 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf00cd1d0 con 0x7facf00afc00 2024-04-25T09:54:17.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.980+0000 7facef7fe640 1 --2- 172.21.15.139:0/912257879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7facf00afc00 0x7facf006fb20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7face00097a0 tx=0x7face002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a3d9aa71bf3ce15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.981+0000 7facee7fc640 1 -- 172.21.15.139:0/912257879 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0037070 con 0x7facf00afc00 2024-04-25T09:54:17.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.981+0000 7facee7fc640 1 -- 172.21.15.139:0/912257879 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face002ae50 con 0x7facf00afc00 2024-04-25T09:54:17.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.981+0000 7facee7fc640 1 -- 172.21.15.139:0/912257879 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0032660 con 0x7facf00afc00 2024-04-25T09:54:17.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.981+0000 7facf5f4c640 1 -- 172.21.15.139:0/912257879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7facf00afc00 msgr2=0x7facf006fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.981+0000 7facf5f4c640 1 --2- 172.21.15.139:0/912257879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7facf00afc00 0x7facf006fb20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7face00097a0 tx=0x7face002a100 comp rx=0 tx=0).stop 2024-04-25T09:54:17.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.981+0000 7facf5f4c640 1 -- 172.21.15.139:0/912257879 shutdown_connections 2024-04-25T09:54:17.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.981+0000 7facf5f4c640 1 --2- 172.21.15.139:0/912257879 >> v2:172.21.15.150:3300/0 conn(0x7facf00d0080 0x7facf006e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.981+0000 7facf5f4c640 1 --2- 172.21.15.139:0/912257879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7facf00afc00 0x7facf006fb20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.981+0000 7facf5f4c640 1 -- 172.21.15.139:0/912257879 >> 172.21.15.139:0/912257879 conn(0x7facf00b2230 msgr2=0x7facf00b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.981+0000 7facf5f4c640 1 -- 172.21.15.139:0/912257879 shutdown_connections 2024-04-25T09:54:17.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.981+0000 7facf5f4c640 1 -- 172.21.15.139:0/912257879 wait complete. 2024-04-25T09:54:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.982+0000 7facf5f4c640 1 Processor -- start 2024-04-25T09:54:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.982+0000 7facf5f4c640 1 -- start start 2024-04-25T09:54:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.983+0000 7facf5f4c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7facf00afc00 0x7facf00ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.983+0000 7facf5f4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf00d0080 0x7facf00b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.983+0000 7facf5f4c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf015cb20 0x7facf00b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.983+0000 7facf5f4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf0070060 con 0x7facf00d0080 2024-04-25T09:54:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.983+0000 7facf5f4c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf00affe0 con 0x7facf015cb20 2024-04-25T09:54:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.983+0000 7facf5f4c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf00cdcb0 con 0x7facf00afc00 2024-04-25T09:54:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.983+0000 7faceeffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf00d0080 0x7facf00b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.983+0000 7faceeffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf00d0080 0x7facf00b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42002/0 (socket says 172.21.15.139:42002) 2024-04-25T09:54:17.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.983+0000 7faceeffd640 1 -- 172.21.15.139:0/1024100195 learned_addr learned my addr 172.21.15.139:0/1024100195 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7faceffff640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf015cb20 0x7facf00b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7facef7fe640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7facf00afc00 0x7facf00ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7faceeffd640 1 -- 172.21.15.139:0/1024100195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7facf00afc00 msgr2=0x7facf00ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7faceeffd640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7facf00afc00 0x7facf00ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7faceeffd640 1 -- 172.21.15.139:0/1024100195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf015cb20 msgr2=0x7facf00b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7faceeffd640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf015cb20 0x7facf00b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7faceeffd640 1 -- 172.21.15.139:0/1024100195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf00b3320 con 0x7facf00d0080 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7facef7fe640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7facf00afc00 0x7facf00ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7faceeffd640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf00d0080 0x7facf00b51a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7face400c400 tx=0x7face400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7facd7fff640 1 -- 172.21.15.139:0/1024100195 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4017070 con 0x7facf00d0080 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7facd7fff640 1 -- 172.21.15.139:0/1024100195 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face4005ce0 con 0x7facf00d0080 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7facd7fff640 1 -- 172.21.15.139:0/1024100195 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40072f0 con 0x7facf00d0080 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7facf5f4c640 1 -- 172.21.15.139:0/1024100195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf00b35c0 con 0x7facf00d0080 2024-04-25T09:54:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.984+0000 7facf5f4c640 1 -- 172.21.15.139:0/1024100195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf00b3b80 con 0x7facf00d0080 2024-04-25T09:54:17.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.985+0000 7facf5f4c640 1 -- 172.21.15.139:0/1024100195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf0074520 con 0x7facf00d0080 2024-04-25T09:54:17.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.985+0000 7facd7fff640 1 -- 172.21.15.139:0/1024100195 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7face401a070 con 0x7facf00d0080 2024-04-25T09:54:17.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.986+0000 7facd7fff640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7facdc06b4d0 0x7facdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.986+0000 7facd7fff640 1 -- 172.21.15.139:0/1024100195 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7face40894c0 con 0x7facf00d0080 2024-04-25T09:54:17.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.986+0000 7facef7fe640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7facdc06b4d0 0x7facdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.986+0000 7facef7fe640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7facdc06b4d0 0x7facdc06d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7face0034000 tx=0x7face002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:17.990+0000 7facd7fff640 1 -- 172.21.15.139:0/1024100195 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7face4058470 con 0x7facf00d0080 2024-04-25T09:54:18.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.228+0000 7facf5f4c640 1 -- 172.21.15.139:0/1024100195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7facf00b42a0 con 0x7facf00d0080 2024-04-25T09:54:18.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.230+0000 7facd7fff640 1 -- 172.21.15.139:0/1024100195 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7face4057d70 con 0x7facf00d0080 2024-04-25T09:54:18.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.232+0000 7facf5f4c640 1 -- 172.21.15.139:0/1024100195 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7facdc06b4d0 msgr2=0x7facdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.232+0000 7facf5f4c640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7facdc06b4d0 0x7facdc06d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7face0034000 tx=0x7face002e000 comp rx=0 tx=0).stop 2024-04-25T09:54:18.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.232+0000 7facf5f4c640 1 -- 172.21.15.139:0/1024100195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf00d0080 msgr2=0x7facf00b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.232+0000 7facf5f4c640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf00d0080 0x7facf00b51a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7face400c400 tx=0x7face400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.232+0000 7facf5f4c640 1 -- 172.21.15.139:0/1024100195 shutdown_connections 2024-04-25T09:54:18.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.232+0000 7facf5f4c640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf00d0080 0x7facf00b51a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.232+0000 7facf5f4c640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7facdc06b4d0 0x7facdc06d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.232+0000 7facf5f4c640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf015cb20 0x7facf00b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.232+0000 7facf5f4c640 1 --2- 172.21.15.139:0/1024100195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7facf00afc00 0x7facf00ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.232+0000 7facf5f4c640 1 -- 172.21.15.139:0/1024100195 >> 172.21.15.139:0/1024100195 conn(0x7facf00b2230 msgr2=0x7facf006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:18.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.233+0000 7facf5f4c640 1 -- 172.21.15.139:0/1024100195 shutdown_connections 2024-04-25T09:54:18.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.233+0000 7facf5f4c640 1 -- 172.21.15.139:0/1024100195 wait complete. 2024-04-25T09:54:18.246 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T09:54:18.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.340+0000 7f651362a640 1 Processor -- start 2024-04-25T09:54:18.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.340+0000 7f651362a640 1 -- start start 2024-04-25T09:54:18.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.341+0000 7f651362a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f650c15dbf0 0x7f650c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.341+0000 7f651362a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f650c0cee50 0x7f650c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.341+0000 7f651362a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650c161dc0 con 0x7f650c0b9370 2024-04-25T09:54:18.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.341+0000 7f651362a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f650c0d1ee0 con 0x7f650c15dbf0 2024-04-25T09:54:18.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.341+0000 7f651362a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650c1620a0 con 0x7f650c0cee50 2024-04-25T09:54:18.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.341+0000 7f65113c6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f650c15dbf0 0x7f650c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.341+0000 7f65113c6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f650c15dbf0 0x7f650c1617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42010/0 (socket says 172.21.15.139:42010) 2024-04-25T09:54:18.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.341+0000 7f65113c6640 1 -- 172.21.15.139:0/2436739930 learned_addr learned my addr 172.21.15.139:0/2436739930 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:18.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.341+0000 7f6511bc7640 1 --2- 172.21.15.139:0/2436739930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f650c0cee50 0x7f650c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.342+0000 7f65113c6640 1 -- 172.21.15.139:0/2436739930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f650c0cee50 msgr2=0x7f650c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:18.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.342+0000 7f65113c6640 1 --2- 172.21.15.139:0/2436739930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f650c0cee50 0x7f650c0d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.342+0000 7f65113c6640 1 -- 172.21.15.139:0/2436739930 >> v1:172.21.15.150:6789/0 conn(0x7f650c0b9370 legacy=0x7f650c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.342+0000 7f65113c6640 1 -- 172.21.15.139:0/2436739930 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c162380 con 0x7f650c15dbf0 2024-04-25T09:54:18.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.342+0000 7f6511bc7640 1 --2- 172.21.15.139:0/2436739930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f650c0cee50 0x7f650c0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:18.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.342+0000 7f65113c6640 1 --2- 172.21.15.139:0/2436739930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f650c15dbf0 0x7f650c1617e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009e50 tx=0x7f64fc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f6982341232d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:18.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.342+0000 7f650bfff640 1 -- 172.21.15.139:0/2436739930 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc036070 con 0x7f650c15dbf0 2024-04-25T09:54:18.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.342+0000 7f650bfff640 1 -- 172.21.15.139:0/2436739930 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc031600 con 0x7f650c15dbf0 2024-04-25T09:54:18.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.342+0000 7f650bfff640 1 -- 172.21.15.139:0/2436739930 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc0319d0 con 0x7f650c15dbf0 2024-04-25T09:54:18.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.343+0000 7f651362a640 1 -- 172.21.15.139:0/2436739930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f650c15dbf0 msgr2=0x7f650c1617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.343+0000 7f651362a640 1 --2- 172.21.15.139:0/2436739930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f650c15dbf0 0x7f650c1617e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009e50 tx=0x7f64fc02a710 comp rx=0 tx=0).stop 2024-04-25T09:54:18.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.343+0000 7f651362a640 1 -- 172.21.15.139:0/2436739930 shutdown_connections 2024-04-25T09:54:18.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.343+0000 7f651362a640 1 --2- 172.21.15.139:0/2436739930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f650c0cee50 0x7f650c0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.343+0000 7f651362a640 1 --2- 172.21.15.139:0/2436739930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f650c15dbf0 0x7f650c1617e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.343+0000 7f651362a640 1 -- 172.21.15.139:0/2436739930 >> 172.21.15.139:0/2436739930 conn(0x7f650c0b5480 msgr2=0x7f650c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:18.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.343+0000 7f651362a640 1 -- 172.21.15.139:0/2436739930 shutdown_connections 2024-04-25T09:54:18.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.343+0000 7f651362a640 1 -- 172.21.15.139:0/2436739930 wait complete. 2024-04-25T09:54:18.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.344+0000 7f651362a640 1 Processor -- start 2024-04-25T09:54:18.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.344+0000 7f651362a640 1 -- start start 2024-04-25T09:54:18.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f651362a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f650c0b9370 0x7f650c1f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f651362a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f650c0cee50 0x7f650c15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f651362a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f650c15dbf0 0x7f650c155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f651362a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c15dfd0 con 0x7f650c15dbf0 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f651362a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c0b97d0 con 0x7f650c0b9370 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f651362a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650c154ed0 con 0x7f650c0cee50 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f6511bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f650c15dbf0 0x7f650c155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f6511bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f650c15dbf0 0x7f650c155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42018/0 (socket says 172.21.15.139:42018) 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f6511bc7640 1 -- 172.21.15.139:0/3579755735 learned_addr learned my addr 172.21.15.139:0/3579755735 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f65113c6640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f650c0b9370 0x7f650c1f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f6510bc5640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f650c0cee50 0x7f650c15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f6511bc7640 1 -- 172.21.15.139:0/3579755735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f650c0cee50 msgr2=0x7f650c15ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f6511bc7640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f650c0cee50 0x7f650c15ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f6511bc7640 1 -- 172.21.15.139:0/3579755735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f650c0b9370 msgr2=0x7f650c1f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f6511bc7640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f650c0b9370 0x7f650c1f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.345+0000 7f6511bc7640 1 -- 172.21.15.139:0/3579755735 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c153e30 con 0x7f650c15dbf0 2024-04-25T09:54:18.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.346+0000 7f6511bc7640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f650c15dbf0 0x7f650c155a30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f650400e5e0 tx=0x7f650400eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:18.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.346+0000 7f6509ffb640 1 -- 172.21.15.139:0/3579755735 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650400f040 con 0x7f650c15dbf0 2024-04-25T09:54:18.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.346+0000 7f6509ffb640 1 -- 172.21.15.139:0/3579755735 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6504004230 con 0x7f650c15dbf0 2024-04-25T09:54:18.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.346+0000 7f651362a640 1 -- 172.21.15.139:0/3579755735 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650c154110 con 0x7f650c15dbf0 2024-04-25T09:54:18.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.346+0000 7f6509ffb640 1 -- 172.21.15.139:0/3579755735 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504003610 con 0x7f650c15dbf0 2024-04-25T09:54:18.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.346+0000 7f651362a640 1 -- 172.21.15.139:0/3579755735 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650c1546d0 con 0x7f650c15dbf0 2024-04-25T09:54:18.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.347+0000 7f651362a640 1 -- 172.21.15.139:0/3579755735 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650c074520 con 0x7f650c15dbf0 2024-04-25T09:54:18.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.347+0000 7f6509ffb640 1 -- 172.21.15.139:0/3579755735 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65040037b0 con 0x7f650c15dbf0 2024-04-25T09:54:18.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.348+0000 7f6509ffb640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f64f806b4d0 0x7f64f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.348+0000 7f65113c6640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f64f806b4d0 0x7f64f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.348+0000 7f6509ffb640 1 -- 172.21.15.139:0/3579755735 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6504089c50 con 0x7f650c15dbf0 2024-04-25T09:54:18.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.348+0000 7f65113c6640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f64f806b4d0 0x7f64f806d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f64fc033000 tx=0x7f64fc009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:18.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.352+0000 7f6509ffb640 1 -- 172.21.15.139:0/3579755735 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f650408f020 con 0x7f650c15dbf0 2024-04-25T09:54:18.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.592+0000 7f651362a640 1 -- 172.21.15.139:0/3579755735 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f650c0d1680 con 0x7f650c15dbf0 2024-04-25T09:54:18.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.594+0000 7f6509ffb640 1 -- 172.21.15.139:0/3579755735 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6504017020 con 0x7f650c15dbf0 2024-04-25T09:54:18.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.597+0000 7f651362a640 1 -- 172.21.15.139:0/3579755735 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f64f806b4d0 msgr2=0x7f64f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.597+0000 7f651362a640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f64f806b4d0 0x7f64f806d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f64fc033000 tx=0x7f64fc009000 comp rx=0 tx=0).stop 2024-04-25T09:54:18.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.597+0000 7f651362a640 1 -- 172.21.15.139:0/3579755735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f650c15dbf0 msgr2=0x7f650c155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.597+0000 7f651362a640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f650c15dbf0 0x7f650c155a30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f650400e5e0 tx=0x7f650400eaa0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.597+0000 7f651362a640 1 -- 172.21.15.139:0/3579755735 shutdown_connections 2024-04-25T09:54:18.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.597+0000 7f651362a640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f650c15dbf0 0x7f650c155a30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.597+0000 7f651362a640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f64f806b4d0 0x7f64f806d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.597+0000 7f651362a640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f650c0cee50 0x7f650c15ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.597+0000 7f651362a640 1 --2- 172.21.15.139:0/3579755735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f650c0b9370 0x7f650c1f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.597+0000 7f651362a640 1 -- 172.21.15.139:0/3579755735 >> 172.21.15.139:0/3579755735 conn(0x7f650c0b5480 msgr2=0x7f650c06c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:18.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.597+0000 7f651362a640 1 -- 172.21.15.139:0/3579755735 shutdown_connections 2024-04-25T09:54:18.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.597+0000 7f651362a640 1 -- 172.21.15.139:0/3579755735 wait complete. 2024-04-25T09:54:18.611 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:18.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.704+0000 7fe353389640 1 Processor -- start 2024-04-25T09:54:18.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.704+0000 7fe353389640 1 -- start start 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.705+0000 7fe353389640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe34c155e30 0x7fe34c15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.705+0000 7fe353389640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c15bdc0 0x7fe34c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.705+0000 7fe353389640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34c15c880 con 0x7fe34c0b1740 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.705+0000 7fe353389640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34c06d6f0 con 0x7fe34c155e30 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.705+0000 7fe353389640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c15cb60 con 0x7fe34c15bdc0 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.705+0000 7fe350924640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c15bdc0 0x7fe34c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.705+0000 7fe351125640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe34c155e30 0x7fe34c15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.705+0000 7fe350924640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c15bdc0 0x7fe34c15c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:55702/0 (socket says 172.21.15.139:55702) 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.705+0000 7fe351926640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fe34c0b1740 0x7fe34c06db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:42682/0 (socket says 172.21.15.139:42682) 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.705+0000 7fe350924640 1 -- 172.21.15.139:0/3648804065 learned_addr learned my addr 172.21.15.139:0/3648804065 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.706+0000 7fe350924640 1 -- 172.21.15.139:0/3648804065 >> v2:172.21.15.150:3300/0 conn(0x7fe34c155e30 msgr2=0x7fe34c15cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.706+0000 7fe350924640 1 --2- 172.21.15.139:0/3648804065 >> v2:172.21.15.150:3300/0 conn(0x7fe34c155e30 0x7fe34c15cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.706+0000 7fe350924640 1 -- 172.21.15.139:0/3648804065 >> v1:172.21.15.150:6789/0 conn(0x7fe34c0b1740 legacy=0x7fe34c06db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.706+0000 7fe350924640 1 -- 172.21.15.139:0/3648804065 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c06d380 con 0x7fe34c15bdc0 2024-04-25T09:54:18.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.706+0000 7fe350924640 1 --2- 172.21.15.139:0/3648804065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c15bdc0 0x7fe34c15c240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009870 tx=0x7fe33c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87de9a322f6ac23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:18.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.707+0000 7fe343fff640 1 -- 172.21.15.139:0/3648804065 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c037070 con 0x7fe34c15bdc0 2024-04-25T09:54:18.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.707+0000 7fe343fff640 1 -- 172.21.15.139:0/3648804065 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33c032440 con 0x7fe34c15bdc0 2024-04-25T09:54:18.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.707+0000 7fe343fff640 1 -- 172.21.15.139:0/3648804065 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c032810 con 0x7fe34c15bdc0 2024-04-25T09:54:18.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.707+0000 7fe353389640 1 -- 172.21.15.139:0/3648804065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c15bdc0 msgr2=0x7fe34c15c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.707+0000 7fe353389640 1 --2- 172.21.15.139:0/3648804065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c15bdc0 0x7fe34c15c240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009870 tx=0x7fe33c02a270 comp rx=0 tx=0).stop 2024-04-25T09:54:18.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.707+0000 7fe353389640 1 -- 172.21.15.139:0/3648804065 shutdown_connections 2024-04-25T09:54:18.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.707+0000 7fe353389640 1 --2- 172.21.15.139:0/3648804065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c15bdc0 0x7fe34c15c240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.707+0000 7fe353389640 1 --2- 172.21.15.139:0/3648804065 >> v2:172.21.15.150:3300/0 conn(0x7fe34c155e30 0x7fe34c15cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.707+0000 7fe353389640 1 -- 172.21.15.139:0/3648804065 >> 172.21.15.139:0/3648804065 conn(0x7fe34c066f20 msgr2=0x7fe34c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:18.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.707+0000 7fe353389640 1 -- 172.21.15.139:0/3648804065 shutdown_connections 2024-04-25T09:54:18.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.707+0000 7fe353389640 1 -- 172.21.15.139:0/3648804065 wait complete. 2024-04-25T09:54:18.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.708+0000 7fe353389640 1 Processor -- start 2024-04-25T09:54:18.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.709+0000 7fe353389640 1 -- start start 2024-04-25T09:54:18.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.709+0000 7fe353389640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe34c0b1740 0x7fe34c1f1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.709+0000 7fe353389640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe34c155e30 0x7fe34c143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.709+0000 7fe353389640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c15bdc0 0x7fe34c1544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.709+0000 7fe353389640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c0af260 con 0x7fe34c0b1740 2024-04-25T09:54:18.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.709+0000 7fe353389640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c0af0e0 con 0x7fe34c155e30 2024-04-25T09:54:18.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.709+0000 7fe353389640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c15d4c0 con 0x7fe34c15bdc0 2024-04-25T09:54:18.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.709+0000 7fe351125640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe34c0b1740 0x7fe34c1f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.709+0000 7fe351125640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe34c0b1740 0x7fe34c1f1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42020/0 (socket says 172.21.15.139:42020) 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.709+0000 7fe351125640 1 -- 172.21.15.139:0/2396654855 learned_addr learned my addr 172.21.15.139:0/2396654855 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.709+0000 7fe351926640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c15bdc0 0x7fe34c1544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.709+0000 7fe350924640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe34c155e30 0x7fe34c143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.710+0000 7fe351125640 1 -- 172.21.15.139:0/2396654855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c15bdc0 msgr2=0x7fe34c1544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.710+0000 7fe351125640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c15bdc0 0x7fe34c1544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.710+0000 7fe351125640 1 -- 172.21.15.139:0/2396654855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe34c155e30 msgr2=0x7fe34c143fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.710+0000 7fe351125640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe34c155e30 0x7fe34c143fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.710+0000 7fe351125640 1 -- 172.21.15.139:0/2396654855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c1565b0 con 0x7fe34c0b1740 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.710+0000 7fe351125640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe34c0b1740 0x7fe34c1f1590 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe348009db0 tx=0x7fe34800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.710+0000 7fe341ffb640 1 -- 172.21.15.139:0/2396654855 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348018020 con 0x7fe34c0b1740 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.710+0000 7fe341ffb640 1 -- 172.21.15.139:0/2396654855 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe348014480 con 0x7fe34c0b1740 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.710+0000 7fe353389640 1 -- 172.21.15.139:0/2396654855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34c156890 con 0x7fe34c0b1740 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.710+0000 7fe341ffb640 1 -- 172.21.15.139:0/2396654855 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348014850 con 0x7fe34c0b1740 2024-04-25T09:54:18.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.710+0000 7fe353389640 1 -- 172.21.15.139:0/2396654855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34c159800 con 0x7fe34c0b1740 2024-04-25T09:54:18.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.711+0000 7fe353389640 1 -- 172.21.15.139:0/2396654855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34c074520 con 0x7fe34c0b1740 2024-04-25T09:54:18.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.712+0000 7fe341ffb640 1 -- 172.21.15.139:0/2396654855 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe34801b020 con 0x7fe34c0b1740 2024-04-25T09:54:18.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.712+0000 7fe341ffb640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe33006b4d0 0x7fe33006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:18.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.712+0000 7fe341ffb640 1 -- 172.21.15.139:0/2396654855 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe34808aa20 con 0x7fe34c0b1740 2024-04-25T09:54:18.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.712+0000 7fe350924640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe33006b4d0 0x7fe33006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:18.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.712+0000 7fe350924640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe33006b4d0 0x7fe33006d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe33c034000 tx=0x7fe33c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:18.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.716+0000 7fe341ffb640 1 -- 172.21.15.139:0/2396654855 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe348022030 con 0x7fe34c0b1740 2024-04-25T09:54:18.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.885+0000 7fe353389640 1 -- 172.21.15.139:0/2396654855 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe34c15e390 con 0x7fe34c0b1740 2024-04-25T09:54:18.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.886+0000 7fe341ffb640 1 -- 172.21.15.139:0/2396654855 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13212 (secure 0 0 0) 0x7fe348010040 con 0x7fe34c0b1740 2024-04-25T09:54:18.889 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:18.889 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":34,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:16.057083+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:18.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.888+0000 7fe353389640 1 -- 172.21.15.139:0/2396654855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe33006b4d0 msgr2=0x7fe33006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.888+0000 7fe353389640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe33006b4d0 0x7fe33006d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe33c034000 tx=0x7fe33c02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:18.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.888+0000 7fe353389640 1 -- 172.21.15.139:0/2396654855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe34c0b1740 msgr2=0x7fe34c1f1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:18.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.888+0000 7fe353389640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe34c0b1740 0x7fe34c1f1590 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe348009db0 tx=0x7fe34800c490 comp rx=0 tx=0).stop 2024-04-25T09:54:18.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.888+0000 7fe353389640 1 -- 172.21.15.139:0/2396654855 shutdown_connections 2024-04-25T09:54:18.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.888+0000 7fe353389640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe33006b4d0 0x7fe33006d990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.888+0000 7fe353389640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c15bdc0 0x7fe34c1544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.888+0000 7fe353389640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe34c155e30 0x7fe34c143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.888+0000 7fe353389640 1 --2- 172.21.15.139:0/2396654855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe34c0b1740 0x7fe34c1f1590 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:18.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.888+0000 7fe353389640 1 -- 172.21.15.139:0/2396654855 >> 172.21.15.139:0/2396654855 conn(0x7fe34c066f20 msgr2=0x7fe34c15d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:18.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.888+0000 7fe353389640 1 -- 172.21.15.139:0/2396654855 shutdown_connections 2024-04-25T09:54:18.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:18.888+0000 7fe353389640 1 -- 172.21.15.139:0/2396654855 wait complete. 2024-04-25T09:54:18.902 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:54:18.902 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:19.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.000+0000 7f142ace1640 1 Processor -- start 2024-04-25T09:54:19.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.000+0000 7f142ace1640 1 -- start start 2024-04-25T09:54:19.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f142ace1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14240d0a80 0x7f14241618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f142ace1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f142409ad40 0x7f1424161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f142ace1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1424099ca0 con 0x7f14240cd610 2024-04-25T09:54:19.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f142ace1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1424099a50 con 0x7f142409ad40 2024-04-25T09:54:19.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f142ace1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1424099f80 con 0x7f14240d0a80 2024-04-25T09:54:19.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f1428a7d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f14240cd610 0x7f14240d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54898/0 (socket says 172.21.15.139:54898) 2024-04-25T09:54:19.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f1428a7d640 1 -- 172.21.15.139:0/186978438 learned_addr learned my addr 172.21.15.139:0/186978438 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:19.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834675230 0 0) 0x7f1424099ca0 con 0x7f14240cd610 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1414003520 con 0x7f14240cd610 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f142927e640 1 --2- 172.21.15.139:0/186978438 >> v2:172.21.15.150:3300/0 conn(0x7f142409ad40 0x7f1424161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f1423fff640 1 --2- 172.21.15.139:0/186978438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14240d0a80 0x7f14241618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1669468433 0 0) 0x7f1414003520 con 0x7f14240cd610 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14240d0a80 msgr2=0x7f14241618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 --2- 172.21.15.139:0/186978438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14240d0a80 0x7f14241618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 >> v2:172.21.15.150:3300/0 conn(0x7f142409ad40 msgr2=0x7f1424161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 --2- 172.21.15.139:0/186978438 >> v2:172.21.15.150:3300/0 conn(0x7f142409ad40 0x7f1424161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142409a260 con 0x7f14240cd610 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f1418001670 con 0x7f14240cd610 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 >> v1:172.21.15.139:6789/0 conn(0x7f14240cd610 legacy=0x7f14240d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 --2- 172.21.15.139:0/186978438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14140039b0 0x7f1414003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 --2- 172.21.15.139:0/186978438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14240d0a80 0x7f1414004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.001+0000 7f14237fe640 1 --2- 172.21.15.139:0/186978438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414004d40 0x7f14140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.002+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1424099a50 con 0x7f1414004d40 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.002+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1424099f80 con 0x7f14140039b0 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.002+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141400b9c0 con 0x7f14240d0a80 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.002+0000 7f142927e640 1 --2- 172.21.15.139:0/186978438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414004d40 0x7f14140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.002+0000 7f1428a7d640 1 --2- 172.21.15.139:0/186978438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14140039b0 0x7f1414003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.002+0000 7f1423fff640 1 --2- 172.21.15.139:0/186978438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14240d0a80 0x7f1414004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.002+0000 7f142927e640 1 -- 172.21.15.139:0/186978438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14240d0a80 msgr2=0x7f1414004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.002+0000 7f142927e640 1 --2- 172.21.15.139:0/186978438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14240d0a80 0x7f1414004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.002+0000 7f142927e640 1 -- 172.21.15.139:0/186978438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14140039b0 msgr2=0x7f1414003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.002+0000 7f142927e640 1 --2- 172.21.15.139:0/186978438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14140039b0 0x7f1414003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.002+0000 7f142927e640 1 -- 172.21.15.139:0/186978438 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1414003520 con 0x7f1414004d40 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.003+0000 7f142927e640 1 --2- 172.21.15.139:0/186978438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414004d40 0x7f14140051c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f141c009fc0 tx=0x7f1414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe1b58a77283224 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.003+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c009c10 con 0x7f1414004d40 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.003+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141c036070 con 0x7f1414004d40 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.003+0000 7f14237fe640 1 -- 172.21.15.139:0/186978438 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c02d070 con 0x7f1414004d40 2024-04-25T09:54:19.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.003+0000 7f142ace1640 1 -- 172.21.15.139:0/186978438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414004d40 msgr2=0x7f14140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.003+0000 7f142ace1640 1 --2- 172.21.15.139:0/186978438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414004d40 0x7f14140051c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f141c009fc0 tx=0x7f1414005830 comp rx=0 tx=0).stop 2024-04-25T09:54:19.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.003+0000 7f142927e640 1 -- 172.21.15.139:0/186978438 reap_dead start 2024-04-25T09:54:19.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.003+0000 7f142ace1640 1 -- 172.21.15.139:0/186978438 shutdown_connections 2024-04-25T09:54:19.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.003+0000 7f142ace1640 1 -- 172.21.15.139:0/186978438 >> 172.21.15.139:0/186978438 conn(0x7f14240b31f0 msgr2=0x7f14240b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:19.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.003+0000 7f142ace1640 1 -- 172.21.15.139:0/186978438 shutdown_connections 2024-04-25T09:54:19.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.003+0000 7f142ace1640 1 -- 172.21.15.139:0/186978438 wait complete. 2024-04-25T09:54:19.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.004+0000 7f142ace1640 1 Processor -- start 2024-04-25T09:54:19.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.004+0000 7f142ace1640 1 -- start start 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.005+0000 7f142ace1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414004d40 0x7f142415af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.005+0000 7f142ace1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1424159c30 0x7f142415d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.005+0000 7f142ace1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f142414f710 0x7f142414fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.005+0000 7f142ace1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14240cd9f0 con 0x7f1414004d40 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.005+0000 7f142ace1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142409a820 con 0x7f1424159c30 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.005+0000 7f142ace1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14240cdb70 con 0x7f142414f710 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.005+0000 7f1428a7d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414004d40 0x7f142415af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.005+0000 7f1428a7d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414004d40 0x7f142415af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42042/0 (socket says 172.21.15.139:42042) 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.005+0000 7f1428a7d640 1 -- 172.21.15.139:0/189120615 learned_addr learned my addr 172.21.15.139:0/189120615 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f142927e640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f142414f710 0x7f142414fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f1423fff640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1424159c30 0x7f142415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f1428a7d640 1 -- 172.21.15.139:0/189120615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f142414f710 msgr2=0x7f142414fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f1428a7d640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f142414f710 0x7f142414fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f1428a7d640 1 -- 172.21.15.139:0/189120615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1424159c30 msgr2=0x7f142415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f1428a7d640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1424159c30 0x7f142415d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f1428a7d640 1 -- 172.21.15.139:0/189120615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14241500d0 con 0x7f1414004d40 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f1423fff640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1424159c30 0x7f142415d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f142927e640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f142414f710 0x7f142414fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f1428a7d640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414004d40 0x7f142415af80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f14180017e0 tx=0x7f1418007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f14217fa640 1 -- 172.21.15.139:0/189120615 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141800f9c0 con 0x7f1414004d40 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f14217fa640 1 -- 172.21.15.139:0/189120615 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1418018070 con 0x7f1414004d40 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f142ace1640 1 -- 172.21.15.139:0/189120615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14240b5860 con 0x7f1414004d40 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f14217fa640 1 -- 172.21.15.139:0/189120615 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141800fde0 con 0x7f1414004d40 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.006+0000 7f142ace1640 1 -- 172.21.15.139:0/189120615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14240b5b00 con 0x7f1414004d40 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.007+0000 7f142ace1640 1 -- 172.21.15.139:0/189120615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1424074520 con 0x7f1414004d40 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.007+0000 7f14217fa640 1 -- 172.21.15.139:0/189120615 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f141801d460 con 0x7f1414004d40 2024-04-25T09:54:19.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.008+0000 7f14217fa640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f140806b500 0x7f140806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.008+0000 7f14217fa640 1 -- 172.21.15.139:0/189120615 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f141808a170 con 0x7f1414004d40 2024-04-25T09:54:19.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.008+0000 7f1423fff640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f140806b500 0x7f140806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.008+0000 7f1423fff640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f140806b500 0x7f140806d9c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14240b04c0 tx=0x7f1410008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.012+0000 7f14217fa640 1 -- 172.21.15.139:0/189120615 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14180591b0 con 0x7f1414004d40 2024-04-25T09:54:19.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.183+0000 7f142ace1640 1 -- 172.21.15.139:0/189120615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14240b3f70 con 0x7f1414004d40 2024-04-25T09:54:19.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.183+0000 7f14217fa640 1 -- 172.21.15.139:0/189120615 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1418058ab0 con 0x7f1414004d40 2024-04-25T09:54:19.186 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:54:19.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.185+0000 7f142ace1640 1 -- 172.21.15.139:0/189120615 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f140806b500 msgr2=0x7f140806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.185+0000 7f142ace1640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f140806b500 0x7f140806d9c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14240b04c0 tx=0x7f1410008040 comp rx=0 tx=0).stop 2024-04-25T09:54:19.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.185+0000 7f142ace1640 1 -- 172.21.15.139:0/189120615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414004d40 msgr2=0x7f142415af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.185+0000 7f142ace1640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414004d40 0x7f142415af80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f14180017e0 tx=0x7f1418007720 comp rx=0 tx=0).stop 2024-04-25T09:54:19.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.185+0000 7f142ace1640 1 -- 172.21.15.139:0/189120615 shutdown_connections 2024-04-25T09:54:19.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.185+0000 7f142ace1640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f140806b500 0x7f140806d9c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.185+0000 7f142ace1640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f142414f710 0x7f142414fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.185+0000 7f142ace1640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1424159c30 0x7f142415d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.185+0000 7f142ace1640 1 --2- 172.21.15.139:0/189120615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1414004d40 0x7f142415af80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.185+0000 7f142ace1640 1 -- 172.21.15.139:0/189120615 >> 172.21.15.139:0/189120615 conn(0x7f14240b31f0 msgr2=0x7f14241614d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:19.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.185+0000 7f142ace1640 1 -- 172.21.15.139:0/189120615 shutdown_connections 2024-04-25T09:54:19.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.185+0000 7f142ace1640 1 -- 172.21.15.139:0/189120615 wait complete. 2024-04-25T09:54:19.199 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:54:19.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.293+0000 7fb189ae1640 1 Processor -- start 2024-04-25T09:54:19.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.294+0000 7fb189ae1640 1 -- start start 2024-04-25T09:54:19.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.294+0000 7fb189ae1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1841611e0 0x7fb1840b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.294+0000 7fb189ae1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb1840cea00 0x7fb18406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.294+0000 7fb189ae1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1840d4740 con 0x7fb1840cee60 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.294+0000 7fb189ae1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1840d1fa0 con 0x7fb1840cea00 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.294+0000 7fb189ae1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1840d2560 con 0x7fb1841611e0 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.294+0000 7fb183fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb1840cee60 0x7fb1840d29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54910/0 (socket says 172.21.15.139:54910) 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.294+0000 7fb183fff640 1 -- 172.21.15.139:0/2449043851 learned_addr learned my addr 172.21.15.139:0/2449043851 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb17bfff640 1 --2- 172.21.15.139:0/2449043851 >> v2:172.21.15.150:3300/0 conn(0x7fb1840cea00 0x7fb18406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb1837fe640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1841611e0 0x7fb1840b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314881770 0 0) 0x7fb1840d4740 con 0x7fb1840cee60 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb170003520 con 0x7fb1840cee60 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2570449020 0 0) 0x7fb170003520 con 0x7fb1840cee60 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1841611e0 msgr2=0x7fb1840b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:19.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1841611e0 0x7fb1840b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 >> v2:172.21.15.150:3300/0 conn(0x7fb1840cea00 msgr2=0x7fb18406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 --2- 172.21.15.139:0/2449043851 >> v2:172.21.15.150:3300/0 conn(0x7fb1840cea00 0x7fb18406c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1840b2770 con 0x7fb1840cee60 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb17bfff640 1 --2- 172.21.15.139:0/2449043851 >> v2:172.21.15.150:3300/0 conn(0x7fb1840cea00 0x7fb18406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fb17c0019f0 con 0x7fb1840cee60 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb1837fe640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1841611e0 0x7fb1840b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 >> v1:172.21.15.139:6789/0 conn(0x7fb1840cee60 legacy=0x7fb1840d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1700039b0 0x7fb170003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1841611e0 0x7fb170004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb170004d40 0x7fb1700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1840d1fa0 con 0x7fb170004d40 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1840d2560 con 0x7fb1700039b0 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17000b9c0 con 0x7fb1841611e0 2024-04-25T09:54:19.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb183fff640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb170004d40 0x7fb1700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb1837fe640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1700039b0 0x7fb170003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.295+0000 7fb17bfff640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1841611e0 0x7fb170004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.296+0000 7fb183fff640 1 -- 172.21.15.139:0/2449043851 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1841611e0 msgr2=0x7fb170004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.296+0000 7fb183fff640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1841611e0 0x7fb170004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.296+0000 7fb183fff640 1 -- 172.21.15.139:0/2449043851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1700039b0 msgr2=0x7fb170003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.296+0000 7fb183fff640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1700039b0 0x7fb170003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.296+0000 7fb183fff640 1 -- 172.21.15.139:0/2449043851 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb170003520 con 0x7fb170004d40 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.296+0000 7fb183fff640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb170004d40 0x7fb1700051c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00a850 tx=0x7fb170005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22b9709aee638c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.296+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c00a4a0 con 0x7fb170004d40 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.296+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c02f070 con 0x7fb170004d40 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.296+0000 7fb182ffd640 1 -- 172.21.15.139:0/2449043851 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c02bce0 con 0x7fb170004d40 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.297+0000 7fb189ae1640 1 -- 172.21.15.139:0/2449043851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb170004d40 msgr2=0x7fb1700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.297+0000 7fb189ae1640 1 --2- 172.21.15.139:0/2449043851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb170004d40 0x7fb1700051c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00a850 tx=0x7fb170005830 comp rx=0 tx=0).stop 2024-04-25T09:54:19.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.297+0000 7fb183fff640 1 -- 172.21.15.139:0/2449043851 reap_dead start 2024-04-25T09:54:19.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.297+0000 7fb189ae1640 1 -- 172.21.15.139:0/2449043851 shutdown_connections 2024-04-25T09:54:19.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.297+0000 7fb189ae1640 1 -- 172.21.15.139:0/2449043851 >> 172.21.15.139:0/2449043851 conn(0x7fb1840b5b00 msgr2=0x7fb1840d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:19.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.297+0000 7fb189ae1640 1 -- 172.21.15.139:0/2449043851 shutdown_connections 2024-04-25T09:54:19.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.297+0000 7fb189ae1640 1 -- 172.21.15.139:0/2449043851 wait complete. 2024-04-25T09:54:19.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.298+0000 7fb189ae1640 1 Processor -- start 2024-04-25T09:54:19.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.298+0000 7fb189ae1640 1 -- start start 2024-04-25T09:54:19.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb189ae1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb170004d40 0x7fb18415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb189ae1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1841f3170 0x7fb1840c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb189ae1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb184159c00 0x7fb1840ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb189ae1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1840d21f0 con 0x7fb1841f3170 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb189ae1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1840b2d30 con 0x7fb170004d40 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb189ae1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1841615c0 con 0x7fb184159c00 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb17bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1841f3170 0x7fb1840c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb17bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1841f3170 0x7fb1840c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42074/0 (socket says 172.21.15.139:42074) 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb17bfff640 1 -- 172.21.15.139:0/3802205953 learned_addr learned my addr 172.21.15.139:0/3802205953 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb183fff640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb184159c00 0x7fb1840ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb1837fe640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb170004d40 0x7fb18415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb17bfff640 1 -- 172.21.15.139:0/3802205953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb184159c00 msgr2=0x7fb1840ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb17bfff640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb184159c00 0x7fb1840ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb17bfff640 1 -- 172.21.15.139:0/3802205953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb170004d40 msgr2=0x7fb18415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb17bfff640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb170004d40 0x7fb18415a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb17bfff640 1 -- 172.21.15.139:0/3802205953 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1841f35f0 con 0x7fb1841f3170 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb1837fe640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb170004d40 0x7fb18415a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.299+0000 7fb17bfff640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1841f3170 0x7fb1840c8090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb16c011cd0 tx=0x7fb16c00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.300+0000 7fb180ff9640 1 -- 172.21.15.139:0/3802205953 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00fd20 con 0x7fb1841f3170 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.300+0000 7fb180ff9640 1 -- 172.21.15.139:0/3802205953 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16c004020 con 0x7fb1841f3170 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.300+0000 7fb189ae1640 1 -- 172.21.15.139:0/3802205953 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1840c8ea0 con 0x7fb1841f3170 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.300+0000 7fb180ff9640 1 -- 172.21.15.139:0/3802205953 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c004410 con 0x7fb1841f3170 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.300+0000 7fb189ae1640 1 -- 172.21.15.139:0/3802205953 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1840c8bc0 con 0x7fb1841f3170 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.301+0000 7fb189ae1640 1 -- 172.21.15.139:0/3802205953 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1840cfbd0 con 0x7fb1841f3170 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.301+0000 7fb180ff9640 1 -- 172.21.15.139:0/3802205953 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb16c012040 con 0x7fb1841f3170 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.301+0000 7fb180ff9640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb16406b4d0 0x7fb16406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.301+0000 7fb180ff9640 1 -- 172.21.15.139:0/3802205953 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb16c08a5e0 con 0x7fb1841f3170 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.301+0000 7fb1837fe640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb16406b4d0 0x7fb16406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.302+0000 7fb1837fe640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb16406b4d0 0x7fb16406d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb174009a30 tx=0x7fb174008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.305+0000 7fb180ff9640 1 -- 172.21.15.139:0/3802205953 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1840cfbd0 con 0x7fb1841f3170 2024-04-25T09:54:19.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.476+0000 7fb189ae1640 1 -- 172.21.15.139:0/3802205953 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb184074520 con 0x7fb1841f3170 2024-04-25T09:54:19.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.476+0000 7fb180ff9640 1 -- 172.21.15.139:0/3802205953 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb1841f35f0 con 0x7fb1841f3170 2024-04-25T09:54:19.480 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:19.480 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-04-25T09:54:19.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:19.480 INFO:teuthology.orchestra.run.smithi139.stdout: "poolnum": 1, 2024-04-25T09:54:19.480 INFO:teuthology.orchestra.run.smithi139.stdout: "poolname": ".mgr" 2024-04-25T09:54:19.480 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:19.480 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-04-25T09:54:19.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.478+0000 7fb189ae1640 1 -- 172.21.15.139:0/3802205953 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb16406b4d0 msgr2=0x7fb16406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.478+0000 7fb189ae1640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb16406b4d0 0x7fb16406d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb174009a30 tx=0x7fb174008040 comp rx=0 tx=0).stop 2024-04-25T09:54:19.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.478+0000 7fb189ae1640 1 -- 172.21.15.139:0/3802205953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1841f3170 msgr2=0x7fb1840c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.478+0000 7fb189ae1640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1841f3170 0x7fb1840c8090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb16c011cd0 tx=0x7fb16c00f610 comp rx=0 tx=0).stop 2024-04-25T09:54:19.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.478+0000 7fb189ae1640 1 -- 172.21.15.139:0/3802205953 shutdown_connections 2024-04-25T09:54:19.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.478+0000 7fb189ae1640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1841f3170 0x7fb1840c8090 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.478+0000 7fb189ae1640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb16406b4d0 0x7fb16406d990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.478+0000 7fb189ae1640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb184159c00 0x7fb1840ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.478+0000 7fb189ae1640 1 --2- 172.21.15.139:0/3802205953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb170004d40 0x7fb18415a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.478+0000 7fb189ae1640 1 -- 172.21.15.139:0/3802205953 >> 172.21.15.139:0/3802205953 conn(0x7fb1840b5b00 msgr2=0x7fb1840d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:19.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.478+0000 7fb189ae1640 1 -- 172.21.15.139:0/3802205953 shutdown_connections 2024-04-25T09:54:19.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.478+0000 7fb189ae1640 1 -- 172.21.15.139:0/3802205953 wait complete. 2024-04-25T09:54:19.492 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:54:19.493 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:54:19.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.589+0000 7f0d5bfc4640 1 Processor -- start 2024-04-25T09:54:19.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.590+0000 7f0d5bfc4640 1 -- start start 2024-04-25T09:54:19.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.590+0000 7f0d5bfc4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0d54072db0 0x7f0d540cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.590+0000 7f0d5bfc4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d540cdd30 con 0x7f0d54159530 2024-04-25T09:54:19.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.590+0000 7f0d5bfc4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d540ce010 con 0x7f0d54070930 2024-04-25T09:54:19.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.590+0000 7f0d5bfc4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d540cdaf0 con 0x7f0d54072db0 2024-04-25T09:54:19.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.591+0000 7f0d5a561640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0d54072db0 0x7f0d540cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.591+0000 7f0d5a561640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0d54072db0 0x7f0d540cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42086/0 (socket says 172.21.15.139:42086) 2024-04-25T09:54:19.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.591+0000 7f0d5a561640 1 -- 172.21.15.139:0/1518530921 learned_addr learned my addr 172.21.15.139:0/1518530921 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:19.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.591+0000 7f0d5a561640 1 -- 172.21.15.139:0/1518530921 >> v1:172.21.15.150:6789/0 conn(0x7f0d54070930 legacy=0x7f0d540726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:19.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.591+0000 7f0d5a561640 1 -- 172.21.15.139:0/1518530921 >> v1:172.21.15.139:6789/0 conn(0x7f0d54159530 legacy=0x7f0d540cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.591+0000 7f0d5a561640 1 -- 172.21.15.139:0/1518530921 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5406fce0 con 0x7f0d54072db0 2024-04-25T09:54:19.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.591+0000 7f0d58d5e640 1 -- 172.21.15.139:0/1518530921 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185411049 0 0) 0x7f0d540cdd30 con 0x7f0d54159530 2024-04-25T09:54:19.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.592+0000 7f0d5a561640 1 --2- 172.21.15.139:0/1518530921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d54072db0 0x7f0d540cf6b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009e50 tx=0x7f0d4c02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709601154dec0c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.592+0000 7f0d58d5e640 1 -- 172.21.15.139:0/1518530921 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c036070 con 0x7f0d54072db0 2024-04-25T09:54:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.592+0000 7f0d58d5e640 1 -- 172.21.15.139:0/1518530921 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4c031600 con 0x7f0d54072db0 2024-04-25T09:54:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.592+0000 7f0d58d5e640 1 -- 172.21.15.139:0/1518530921 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0319d0 con 0x7f0d54072db0 2024-04-25T09:54:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.592+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/1518530921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d54072db0 msgr2=0x7f0d540cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.592+0000 7f0d5bfc4640 1 --2- 172.21.15.139:0/1518530921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d54072db0 0x7f0d540cf6b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009e50 tx=0x7f0d4c02a760 comp rx=0 tx=0).stop 2024-04-25T09:54:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.592+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/1518530921 shutdown_connections 2024-04-25T09:54:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.592+0000 7f0d5bfc4640 1 --2- 172.21.15.139:0/1518530921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d54072db0 0x7f0d540cf6b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.592+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/1518530921 >> 172.21.15.139:0/1518530921 conn(0x7f0d540b23f0 msgr2=0x7f0d540b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.592+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/1518530921 shutdown_connections 2024-04-25T09:54:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.592+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/1518530921 wait complete. 2024-04-25T09:54:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.593+0000 7f0d5bfc4640 1 Processor -- start 2024-04-25T09:54:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.593+0000 7f0d5bfc4640 1 -- start start 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.594+0000 7f0d5bfc4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d54070930 0x7f0d540b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.594+0000 7f0d5bfc4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d54072db0 0x7f0d54152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.594+0000 7f0d5bfc4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d54159530 0x7f0d540a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.594+0000 7f0d5bfc4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d540704e0 con 0x7f0d54072db0 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.594+0000 7f0d5bfc4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5406fa90 con 0x7f0d54159530 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.594+0000 7f0d5bfc4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5415bda0 con 0x7f0d54070930 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.594+0000 7f0d5955f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d54072db0 0x7f0d54152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.594+0000 7f0d5955f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d54072db0 0x7f0d54152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42090/0 (socket says 172.21.15.139:42090) 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.594+0000 7f0d5955f640 1 -- 172.21.15.139:0/626407279 learned_addr learned my addr 172.21.15.139:0/626407279 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.594+0000 7f0d59d60640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d54070930 0x7f0d540b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.594+0000 7f0d5a561640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d54159530 0x7f0d540a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.595+0000 7f0d5955f640 1 -- 172.21.15.139:0/626407279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d54070930 msgr2=0x7f0d540b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.595+0000 7f0d5955f640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d54070930 0x7f0d540b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.595+0000 7f0d5955f640 1 -- 172.21.15.139:0/626407279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d54159530 msgr2=0x7f0d540a8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.595+0000 7f0d5955f640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d54159530 0x7f0d540a8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.595+0000 7f0d5955f640 1 -- 172.21.15.139:0/626407279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d540b0920 con 0x7f0d54072db0 2024-04-25T09:54:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.595+0000 7f0d59d60640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d54070930 0x7f0d540b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.595+0000 7f0d5955f640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d54072db0 0x7f0d54152280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400c3f0 tx=0x7f0d4400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.595+0000 7f0d4a7fc640 1 -- 172.21.15.139:0/626407279 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44007020 con 0x7f0d54072db0 2024-04-25T09:54:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.595+0000 7f0d4a7fc640 1 -- 172.21.15.139:0/626407279 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d440071c0 con 0x7f0d54072db0 2024-04-25T09:54:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.595+0000 7f0d4a7fc640 1 -- 172.21.15.139:0/626407279 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44007590 con 0x7f0d54072db0 2024-04-25T09:54:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.595+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/626407279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d541527c0 con 0x7f0d54072db0 2024-04-25T09:54:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.595+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/626407279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d540b0ef0 con 0x7f0d54072db0 2024-04-25T09:54:19.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.596+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/626407279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d54074520 con 0x7f0d54072db0 2024-04-25T09:54:19.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.598+0000 7f0d4a7fc640 1 -- 172.21.15.139:0/626407279 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d44019020 con 0x7f0d54072db0 2024-04-25T09:54:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.598+0000 7f0d4a7fc640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0d3806b4d0 0x7f0d3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.598+0000 7f0d4a7fc640 1 -- 172.21.15.139:0/626407279 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0d4408a220 con 0x7f0d54072db0 2024-04-25T09:54:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.598+0000 7f0d59d60640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0d3806b4d0 0x7f0d3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.599+0000 7f0d59d60640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0d3806b4d0 0x7f0d3806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0d50000f60 tx=0x7f0d50009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.601+0000 7f0d4a7fc640 1 -- 172.21.15.139:0/626407279 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d44059340 con 0x7f0d54072db0 2024-04-25T09:54:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:19.775+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/626407279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0d54072480 con 0x7f0d54072db0 2024-04-25T09:54:20.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.597+0000 7f0d4a7fc640 1 -- 172.21.15.139:0/626407279 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0d44058c40 con 0x7f0d54072db0 2024-04-25T09:54:20.601 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' created 2024-04-25T09:54:20.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.600+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/626407279 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0d3806b4d0 msgr2=0x7f0d3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.600+0000 7f0d5bfc4640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0d3806b4d0 0x7f0d3806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0d50000f60 tx=0x7f0d50009040 comp rx=0 tx=0).stop 2024-04-25T09:54:20.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.600+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/626407279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d54072db0 msgr2=0x7f0d54152280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.600+0000 7f0d5bfc4640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d54072db0 0x7f0d54152280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400c3f0 tx=0x7f0d4400c8b0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.601+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/626407279 shutdown_connections 2024-04-25T09:54:20.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.601+0000 7f0d5bfc4640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d54072db0 0x7f0d54152280 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.601+0000 7f0d5bfc4640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0d3806b4d0 0x7f0d3806d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.601+0000 7f0d5bfc4640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d54159530 0x7f0d540a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.601+0000 7f0d5bfc4640 1 --2- 172.21.15.139:0/626407279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d54070930 0x7f0d540b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.601+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/626407279 >> 172.21.15.139:0/626407279 conn(0x7f0d540b23f0 msgr2=0x7f0d54156190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:20.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.601+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/626407279 shutdown_connections 2024-04-25T09:54:20.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.601+0000 7f0d5bfc4640 1 -- 172.21.15.139:0/626407279 wait complete. 2024-04-25T09:54:20.620 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:54:20.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.727+0000 7f2ecde76640 1 Processor -- start 2024-04-25T09:54:20.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.727+0000 7f2ecde76640 1 -- start start 2024-04-25T09:54:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ecde76640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ec815dbf0 0x7f2ec81617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ecde76640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2ec80b9370 0x7f2ec80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ecde76640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec8161dc0 con 0x7f2ec80cec80 2024-04-25T09:54:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ecde76640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec80d1ee0 con 0x7f2ec80b9370 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ecde76640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec81620a0 con 0x7f2ec815dbf0 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ec6ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2ec80b9370 0x7f2ec80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ec6ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2ec80b9370 0x7f2ec80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42094/0 (socket says 172.21.15.139:42094) 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ec7fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2ec80cec80 0x7f2ec806c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54934/0 (socket says 172.21.15.139:54934) 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ec6ffd640 1 -- 172.21.15.139:0/1820874968 learned_addr learned my addr 172.21.15.139:0/1820874968 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ec6ffd640 1 -- 172.21.15.139:0/1820874968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ec815dbf0 msgr2=0x7f2ec81617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ec6ffd640 1 --2- 172.21.15.139:0/1820874968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ec815dbf0 0x7f2ec81617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ec6ffd640 1 -- 172.21.15.139:0/1820874968 >> v1:172.21.15.139:6789/0 conn(0x7f2ec80cec80 legacy=0x7f2ec806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ec67fc640 1 -- 172.21.15.139:0/1820874968 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674967986 0 0) 0x7f2ec8161dc0 con 0x7f2ec80cec80 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.728+0000 7f2ec6ffd640 1 -- 172.21.15.139:0/1820874968 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec8162380 con 0x7f2ec80b9370 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.729+0000 7f2ec6ffd640 1 --2- 172.21.15.139:0/1820874968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec80b9370 0x7f2ec80ce740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009870 tx=0x7f2ebc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbb0f7b450211c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.729+0000 7f2ec67fc640 1 -- 172.21.15.139:0/1820874968 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc037070 con 0x7f2ec80b9370 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.729+0000 7f2ec67fc640 1 -- 172.21.15.139:0/1820874968 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc032440 con 0x7f2ec80b9370 2024-04-25T09:54:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.729+0000 7f2ec67fc640 1 -- 172.21.15.139:0/1820874968 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc032810 con 0x7f2ec80b9370 2024-04-25T09:54:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.730+0000 7f2ecde76640 1 -- 172.21.15.139:0/1820874968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec80b9370 msgr2=0x7f2ec80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.730+0000 7f2ecde76640 1 --2- 172.21.15.139:0/1820874968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec80b9370 0x7f2ec80ce740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009870 tx=0x7f2ebc02a270 comp rx=0 tx=0).stop 2024-04-25T09:54:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.730+0000 7f2ecde76640 1 -- 172.21.15.139:0/1820874968 shutdown_connections 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.730+0000 7f2ecde76640 1 --2- 172.21.15.139:0/1820874968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec80b9370 0x7f2ec80ce740 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.730+0000 7f2ecde76640 1 --2- 172.21.15.139:0/1820874968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ec815dbf0 0x7f2ec81617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.730+0000 7f2ecde76640 1 -- 172.21.15.139:0/1820874968 >> 172.21.15.139:0/1820874968 conn(0x7f2ec80b5480 msgr2=0x7f2ec80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.730+0000 7f2ecde76640 1 -- 172.21.15.139:0/1820874968 shutdown_connections 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.730+0000 7f2ecde76640 1 -- 172.21.15.139:0/1820874968 wait complete. 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.731+0000 7f2ecde76640 1 Processor -- start 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.731+0000 7f2ecde76640 1 -- start start 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ecde76640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec80b9370 0x7f2ec815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ecde76640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ec80cec80 0x7f2ec8158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ecde76640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec815dbf0 0x7f2ec8158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ecde76640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec815dfd0 con 0x7f2ec815dbf0 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ecde76640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec80b97d0 con 0x7f2ec80b9370 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ecde76640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec815b480 con 0x7f2ec80cec80 2024-04-25T09:54:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ec7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec815dbf0 0x7f2ec8158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ec7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec815dbf0 0x7f2ec8158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42096/0 (socket says 172.21.15.139:42096) 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ec7fff640 1 -- 172.21.15.139:0/1300719792 learned_addr learned my addr 172.21.15.139:0/1300719792 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ec6ffd640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ec80cec80 0x7f2ec8158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ec77fe640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec80b9370 0x7f2ec815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ec7fff640 1 -- 172.21.15.139:0/1300719792 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ec80cec80 msgr2=0x7f2ec8158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ec7fff640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ec80cec80 0x7f2ec8158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ec7fff640 1 -- 172.21.15.139:0/1300719792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec80b9370 msgr2=0x7f2ec815c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ec7fff640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec80b9370 0x7f2ec815c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ec7fff640 1 -- 172.21.15.139:0/1300719792 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec815a870 con 0x7f2ec815dbf0 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ec77fe640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec80b9370 0x7f2ec815c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ec7fff640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec815dbf0 0x7f2ec8158ab0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000c7a0 tx=0x7f2ec000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2eaffff640 1 -- 172.21.15.139:0/1300719792 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0018070 con 0x7f2ec815dbf0 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2eaffff640 1 -- 172.21.15.139:0/1300719792 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec000a640 con 0x7f2ec815dbf0 2024-04-25T09:54:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ecde76640 1 -- 172.21.15.139:0/1300719792 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec815ab50 con 0x7f2ec815dbf0 2024-04-25T09:54:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2eaffff640 1 -- 172.21.15.139:0/1300719792 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec000aab0 con 0x7f2ec815dbf0 2024-04-25T09:54:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.732+0000 7f2ecde76640 1 -- 172.21.15.139:0/1300719792 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec815b110 con 0x7f2ec815dbf0 2024-04-25T09:54:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.733+0000 7f2ecde76640 1 -- 172.21.15.139:0/1300719792 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec8074520 con 0x7f2ec815dbf0 2024-04-25T09:54:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.734+0000 7f2eaffff640 1 -- 172.21.15.139:0/1300719792 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ec001b070 con 0x7f2ec815dbf0 2024-04-25T09:54:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.734+0000 7f2eaffff640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2eb406b500 0x7f2eb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.734+0000 7f2eaffff640 1 -- 172.21.15.139:0/1300719792 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f2ec0089720 con 0x7f2ec815dbf0 2024-04-25T09:54:20.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.734+0000 7f2ec77fe640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2eb406b500 0x7f2eb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.735+0000 7f2ec77fe640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2eb406b500 0x7f2eb406d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8006fd0 tx=0x7f2eb8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.740+0000 7f2eaffff640 1 -- 172.21.15.139:0/1300719792 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ec0058680 con 0x7f2ec815dbf0 2024-04-25T09:54:20.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:20.911+0000 7f2ecde76640 1 -- 172.21.15.139:0/1300719792 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2ec80d1680 con 0x7f2ec815dbf0 2024-04-25T09:54:21.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.607+0000 7f2eaffff640 1 -- 172.21.15.139:0/1300719792 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2ec0057f80 con 0x7f2ec815dbf0 2024-04-25T09:54:21.666 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' created 2024-04-25T09:54:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.610+0000 7f2ecde76640 1 -- 172.21.15.139:0/1300719792 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2eb406b500 msgr2=0x7f2eb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.610+0000 7f2ecde76640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2eb406b500 0x7f2eb406d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8006fd0 tx=0x7f2eb8008040 comp rx=0 tx=0).stop 2024-04-25T09:54:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.610+0000 7f2ecde76640 1 -- 172.21.15.139:0/1300719792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec815dbf0 msgr2=0x7f2ec8158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.610+0000 7f2ecde76640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec815dbf0 0x7f2ec8158ab0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000c7a0 tx=0x7f2ec000cc60 comp rx=0 tx=0).stop 2024-04-25T09:54:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.610+0000 7f2ecde76640 1 -- 172.21.15.139:0/1300719792 shutdown_connections 2024-04-25T09:54:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.610+0000 7f2ecde76640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec815dbf0 0x7f2ec8158ab0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.610+0000 7f2ecde76640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2eb406b500 0x7f2eb406d9c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.610+0000 7f2ecde76640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ec80cec80 0x7f2ec8158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.610+0000 7f2ecde76640 1 --2- 172.21.15.139:0/1300719792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ec80b9370 0x7f2ec815c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.610+0000 7f2ecde76640 1 -- 172.21.15.139:0/1300719792 >> 172.21.15.139:0/1300719792 conn(0x7f2ec80b5480 msgr2=0x7f2ec80d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.610+0000 7f2ecde76640 1 -- 172.21.15.139:0/1300719792 shutdown_connections 2024-04-25T09:54:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.611+0000 7f2ecde76640 1 -- 172.21.15.139:0/1300719792 wait complete. 2024-04-25T09:54:21.669 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:54:21.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.770+0000 7f8ffb1d5640 1 Processor -- start 2024-04-25T09:54:21.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.770+0000 7f8ffb1d5640 1 -- start start 2024-04-25T09:54:21.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ffb1d5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8ff40cd610 0x7f8ff40d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:21.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ffb1d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ff409ad40 0x7f8ff40d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:21.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ffb1d5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8ff41618c0 0x7f8ff4161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:21.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ffb1d5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff4099a50 con 0x7f8ff41618c0 2024-04-25T09:54:21.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ffb1d5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff4099ca0 con 0x7f8ff40cd610 2024-04-25T09:54:21.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ffb1d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4099f80 con 0x7f8ff409ad40 2024-04-25T09:54:21.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ff9772640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8ff41618c0 0x7f8ff4161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:21.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ff9772640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8ff41618c0 0x7f8ff4161d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42104/0 (socket says 172.21.15.139:42104) 2024-04-25T09:54:21.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ff9772640 1 -- 172.21.15.139:0/1866344935 learned_addr learned my addr 172.21.15.139:0/1866344935 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:21.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ff3fff640 1 --2- 172.21.15.139:0/1866344935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ff409ad40 0x7f8ff40d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:21.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ff8f71640 1 --2- 172.21.15.139:0/1866344935 >> v2:172.21.15.150:3300/0 conn(0x7f8ff40cd610 0x7f8ff40d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:21.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ff9772640 1 -- 172.21.15.139:0/1866344935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ff409ad40 msgr2=0x7f8ff40d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ff9772640 1 --2- 172.21.15.139:0/1866344935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ff409ad40 0x7f8ff40d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ff9772640 1 -- 172.21.15.139:0/1866344935 >> v2:172.21.15.150:3300/0 conn(0x7f8ff40cd610 msgr2=0x7f8ff40d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ff9772640 1 --2- 172.21.15.139:0/1866344935 >> v2:172.21.15.150:3300/0 conn(0x7f8ff40cd610 0x7f8ff40d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ff9772640 1 -- 172.21.15.139:0/1866344935 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff409a260 con 0x7f8ff41618c0 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ff3fff640 1 --2- 172.21.15.139:0/1866344935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ff409ad40 0x7f8ff40d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.771+0000 7f8ff8f71640 1 --2- 172.21.15.139:0/1866344935 >> v2:172.21.15.150:3300/0 conn(0x7f8ff40cd610 0x7f8ff40d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.772+0000 7f8ff9772640 1 --2- 172.21.15.139:0/1866344935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ff41618c0 0x7f8ff4161d50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8ff406f210 tx=0x7f8fe402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c618e822022652d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.772+0000 7f8ff37fe640 1 -- 172.21.15.139:0/1866344935 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4037070 con 0x7f8ff41618c0 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.772+0000 7f8ff37fe640 1 -- 172.21.15.139:0/1866344935 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4032440 con 0x7f8ff41618c0 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.772+0000 7f8ff37fe640 1 -- 172.21.15.139:0/1866344935 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4032810 con 0x7f8ff41618c0 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.772+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/1866344935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ff41618c0 msgr2=0x7f8ff4161d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.772+0000 7f8ffb1d5640 1 --2- 172.21.15.139:0/1866344935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ff41618c0 0x7f8ff4161d50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8ff406f210 tx=0x7f8fe402a270 comp rx=0 tx=0).stop 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.772+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/1866344935 shutdown_connections 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.772+0000 7f8ffb1d5640 1 --2- 172.21.15.139:0/1866344935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ff41618c0 0x7f8ff4161d50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.772+0000 7f8ffb1d5640 1 --2- 172.21.15.139:0/1866344935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ff409ad40 0x7f8ff40d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.772+0000 7f8ffb1d5640 1 --2- 172.21.15.139:0/1866344935 >> v2:172.21.15.150:3300/0 conn(0x7f8ff40cd610 0x7f8ff40d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.772+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/1866344935 >> 172.21.15.139:0/1866344935 conn(0x7f8ff40b31f0 msgr2=0x7f8ff40b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:21.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.773+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/1866344935 shutdown_connections 2024-04-25T09:54:21.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.773+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/1866344935 wait complete. 2024-04-25T09:54:21.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.774+0000 7f8ffb1d5640 1 Processor -- start 2024-04-25T09:54:21.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.774+0000 7f8ffb1d5640 1 -- start start 2024-04-25T09:54:21.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.774+0000 7f8ffb1d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ff409ad40 0x7f8ff40c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.774+0000 7f8ffb1d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ff40cd610 0x7f8ff40c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.774+0000 7f8ffb1d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ff41618c0 0x7f8ff4155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.774+0000 7f8ffb1d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40cd9f0 con 0x7f8ff41618c0 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.774+0000 7f8ffb1d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff409a820 con 0x7f8ff409ad40 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.774+0000 7f8ffb1d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40cdb70 con 0x7f8ff40cd610 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff9772640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ff41618c0 0x7f8ff4155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff9772640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ff41618c0 0x7f8ff4155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42108/0 (socket says 172.21.15.139:42108) 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff9772640 1 -- 172.21.15.139:0/3015306307 learned_addr learned my addr 172.21.15.139:0/3015306307 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff8f71640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ff409ad40 0x7f8ff40c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff3fff640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ff40cd610 0x7f8ff40c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff9772640 1 -- 172.21.15.139:0/3015306307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ff40cd610 msgr2=0x7f8ff40c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff9772640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ff40cd610 0x7f8ff40c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff9772640 1 -- 172.21.15.139:0/3015306307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ff409ad40 msgr2=0x7f8ff40c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:21.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff9772640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ff409ad40 0x7f8ff40c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff9772640 1 -- 172.21.15.139:0/3015306307 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff40c58c0 con 0x7f8ff41618c0 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff8f71640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ff409ad40 0x7f8ff40c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff3fff640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ff40cd610 0x7f8ff40c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff9772640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ff41618c0 0x7f8ff4155d50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8fe402a2b0 tx=0x7f8fe40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff17fa640 1 -- 172.21.15.139:0/3015306307 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4041020 con 0x7f8ff41618c0 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff17fa640 1 -- 172.21.15.139:0/3015306307 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe40306c0 con 0x7f8ff41618c0 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ff17fa640 1 -- 172.21.15.139:0/3015306307 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4037040 con 0x7f8ff41618c0 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/3015306307 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff40c5ba0 con 0x7f8ff41618c0 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.775+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/3015306307 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff40c6160 con 0x7f8ff41618c0 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.776+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/3015306307 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff4074520 con 0x7f8ff41618c0 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.777+0000 7f8ff17fa640 1 -- 172.21.15.139:0/3015306307 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fe4044020 con 0x7f8ff41618c0 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.777+0000 7f8ff17fa640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8fd406b4d0 0x7f8fd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.777+0000 7f8ff17fa640 1 -- 172.21.15.139:0/3015306307 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f8fe40aaae0 con 0x7f8ff41618c0 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.777+0000 7f8ff8f71640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8fd406b4d0 0x7f8fd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:21.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.777+0000 7f8ff8f71640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8fd406b4d0 0x7f8fd406d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009540 tx=0x7f8fec00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:21.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.781+0000 7f8ff17fa640 1 -- 172.21.15.139:0/3015306307 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8fe404b030 con 0x7f8ff41618c0 2024-04-25T09:54:21.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:21.987+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/3015306307 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8ff4071200 con 0x7f8ff41618c0 2024-04-25T09:54:22.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.608+0000 7f8ff17fa640 1 -- 172.21.15.139:0/3015306307 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8fe4079830 con 0x7f8ff41618c0 2024-04-25T09:54:22.611 INFO:teuthology.orchestra.run.smithi139.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T09:54:22.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.612+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/3015306307 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8fd406b4d0 msgr2=0x7f8fd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.612+0000 7f8ffb1d5640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8fd406b4d0 0x7f8fd406d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009540 tx=0x7f8fec00b040 comp rx=0 tx=0).stop 2024-04-25T09:54:22.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.612+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/3015306307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ff41618c0 msgr2=0x7f8ff4155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.612+0000 7f8ffb1d5640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ff41618c0 0x7f8ff4155d50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8fe402a2b0 tx=0x7f8fe40046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.616+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/3015306307 shutdown_connections 2024-04-25T09:54:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.616+0000 7f8ffb1d5640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ff41618c0 0x7f8ff4155d50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.616+0000 7f8ffb1d5640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8fd406b4d0 0x7f8fd406d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.616+0000 7f8ffb1d5640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ff40cd610 0x7f8ff40c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.616+0000 7f8ffb1d5640 1 --2- 172.21.15.139:0/3015306307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ff409ad40 0x7f8ff40c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.616+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/3015306307 >> 172.21.15.139:0/3015306307 conn(0x7f8ff40b31f0 msgr2=0x7f8ff40b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:22.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.619+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/3015306307 shutdown_connections 2024-04-25T09:54:22.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.619+0000 7f8ffb1d5640 1 -- 172.21.15.139:0/3015306307 wait complete. 2024-04-25T09:54:22.644 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:22.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.741+0000 7f50db05d640 1 Processor -- start 2024-04-25T09:54:22.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.741+0000 7f50db05d640 1 -- start start 2024-04-25T09:54:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.742+0000 7f50db05d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50d406c1f0 0x7f50d415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.742+0000 7f50db05d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d4160280 con 0x7f50d40ad740 2024-04-25T09:54:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.742+0000 7f50db05d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d40af590 con 0x7f50d415c9c0 2024-04-25T09:54:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.742+0000 7f50db05d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d40af870 con 0x7f50d406c1f0 2024-04-25T09:54:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.742+0000 7f50d8df9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50d406c1f0 0x7f50d415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.742+0000 7f50d8df9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50d406c1f0 0x7f50d415fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42120/0 (socket says 172.21.15.139:42120) 2024-04-25T09:54:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.742+0000 7f50d8df9640 1 -- 172.21.15.139:0/4294921198 learned_addr learned my addr 172.21.15.139:0/4294921198 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.742+0000 7f50d95fa640 1 --1- 172.21.15.139:0/4294921198 >> v1:172.21.15.139:6789/0 conn(0x7f50d40ad740 0x7f50d406ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54946/0 (socket says 172.21.15.139:54946) 2024-04-25T09:54:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.742+0000 7f50d8df9640 1 -- 172.21.15.139:0/4294921198 >> v1:172.21.15.150:6789/0 conn(0x7f50d415c9c0 legacy=0x7f50d40ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.742+0000 7f50d8df9640 1 -- 172.21.15.139:0/4294921198 >> v1:172.21.15.139:6789/0 conn(0x7f50d40ad740 legacy=0x7f50d406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.742+0000 7f50d37fe640 1 -- 172.21.15.139:0/4294921198 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844837442 0 0) 0x7f50cc001490 con 0x7f50d40ad740 2024-04-25T09:54:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.742+0000 7f50d8df9640 1 -- 172.21.15.139:0/4294921198 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d415bee0 con 0x7f50d406c1f0 2024-04-25T09:54:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.743+0000 7f50d8df9640 1 --2- 172.21.15.139:0/4294921198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d406c1f0 0x7f50d415fd40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009870 tx=0x7f50c802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f26602f0465f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.743+0000 7f50d37fe640 1 -- 172.21.15.139:0/4294921198 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8037070 con 0x7f50d406c1f0 2024-04-25T09:54:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.743+0000 7f50d37fe640 1 -- 172.21.15.139:0/4294921198 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8032440 con 0x7f50d406c1f0 2024-04-25T09:54:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.743+0000 7f50d37fe640 1 -- 172.21.15.139:0/4294921198 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8032810 con 0x7f50d406c1f0 2024-04-25T09:54:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.743+0000 7f50db05d640 1 -- 172.21.15.139:0/4294921198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d406c1f0 msgr2=0x7f50d415fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.743+0000 7f50db05d640 1 --2- 172.21.15.139:0/4294921198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d406c1f0 0x7f50d415fd40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009870 tx=0x7f50c802a210 comp rx=0 tx=0).stop 2024-04-25T09:54:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.743+0000 7f50db05d640 1 -- 172.21.15.139:0/4294921198 shutdown_connections 2024-04-25T09:54:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.743+0000 7f50db05d640 1 --2- 172.21.15.139:0/4294921198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d406c1f0 0x7f50d415fd40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.743+0000 7f50db05d640 1 -- 172.21.15.139:0/4294921198 >> 172.21.15.139:0/4294921198 conn(0x7f50d40b23d0 msgr2=0x7f50d40cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.744+0000 7f50db05d640 1 -- 172.21.15.139:0/4294921198 shutdown_connections 2024-04-25T09:54:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.744+0000 7f50db05d640 1 -- 172.21.15.139:0/4294921198 wait complete. 2024-04-25T09:54:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.745+0000 7f50db05d640 1 Processor -- start 2024-04-25T09:54:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.745+0000 7f50db05d640 1 -- start start 2024-04-25T09:54:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.745+0000 7f50db05d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d406c1f0 0x7f50d4140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.745+0000 7f50db05d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50d40ad740 0x7f50d41f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.745+0000 7f50db05d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50d415c9c0 0x7f50d40cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.745+0000 7f50db05d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d406f630 con 0x7f50d406c1f0 2024-04-25T09:54:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.745+0000 7f50db05d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d415c530 con 0x7f50d415c9c0 2024-04-25T09:54:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.745+0000 7f50db05d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d406f7b0 con 0x7f50d40ad740 2024-04-25T09:54:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d8df9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d406c1f0 0x7f50d4140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d8df9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d406c1f0 0x7f50d4140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42130/0 (socket says 172.21.15.139:42130) 2024-04-25T09:54:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d8df9640 1 -- 172.21.15.139:0/2227259343 learned_addr learned my addr 172.21.15.139:0/2227259343 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d95fa640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50d415c9c0 0x7f50d40cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d3fff640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50d40ad740 0x7f50d41f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d8df9640 1 -- 172.21.15.139:0/2227259343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50d40ad740 msgr2=0x7f50d41f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d8df9640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50d40ad740 0x7f50d41f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d8df9640 1 -- 172.21.15.139:0/2227259343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50d415c9c0 msgr2=0x7f50d40cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d8df9640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50d415c9c0 0x7f50d40cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d8df9640 1 -- 172.21.15.139:0/2227259343 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d40ce5a0 con 0x7f50d406c1f0 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d3fff640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50d40ad740 0x7f50d41f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d8df9640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d406c1f0 0x7f50d4140cb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009870 tx=0x7f50c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d17fa640 1 -- 172.21.15.139:0/2227259343 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8037040 con 0x7f50d406c1f0 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d17fa640 1 -- 172.21.15.139:0/2227259343 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8004250 con 0x7f50d406c1f0 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50db05d640 1 -- 172.21.15.139:0/2227259343 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d40cf5d0 con 0x7f50d406c1f0 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50d17fa640 1 -- 172.21.15.139:0/2227259343 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c803c670 con 0x7f50d406c1f0 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.746+0000 7f50db05d640 1 -- 172.21.15.139:0/2227259343 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d40ceaf0 con 0x7f50d406c1f0 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.747+0000 7f50db05d640 1 -- 172.21.15.139:0/2227259343 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d415d2f0 con 0x7f50d406c1f0 2024-04-25T09:54:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.748+0000 7f50d17fa640 1 -- 172.21.15.139:0/2227259343 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50c803a070 con 0x7f50d406c1f0 2024-04-25T09:54:22.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.748+0000 7f50d17fa640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50b806b500 0x7f50b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:22.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.748+0000 7f50d17fa640 1 -- 172.21.15.139:0/2227259343 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f50c80aac80 con 0x7f50d406c1f0 2024-04-25T09:54:22.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.748+0000 7f50d3fff640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50b806b500 0x7f50b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:22.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.749+0000 7f50d3fff640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50b806b500 0x7f50b806d9c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f50d40b49c0 tx=0x7f50c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.752+0000 7f50d17fa640 1 -- 172.21.15.139:0/2227259343 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50d415d2f0 con 0x7f50d406c1f0 2024-04-25T09:54:22.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.924+0000 7f50db05d640 1 -- 172.21.15.139:0/2227259343 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50d4074520 con 0x7f50d406c1f0 2024-04-25T09:54:22.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.925+0000 7f50d17fa640 1 -- 172.21.15.139:0/2227259343 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17003 (secure 0 0 0) 0x7f50c8041020 con 0x7f50d406c1f0 2024-04-25T09:54:22.927 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:22.927 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":37,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:21.988665+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:19.795203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:20.931184+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:22.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.927+0000 7f50db05d640 1 -- 172.21.15.139:0/2227259343 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50b806b500 msgr2=0x7f50b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.927+0000 7f50db05d640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50b806b500 0x7f50b806d9c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f50d40b49c0 tx=0x7f50c0008040 comp rx=0 tx=0).stop 2024-04-25T09:54:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.927+0000 7f50db05d640 1 -- 172.21.15.139:0/2227259343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d406c1f0 msgr2=0x7f50d4140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.927+0000 7f50db05d640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d406c1f0 0x7f50d4140cb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009870 tx=0x7f50c80046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.927+0000 7f50db05d640 1 -- 172.21.15.139:0/2227259343 shutdown_connections 2024-04-25T09:54:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.927+0000 7f50db05d640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50b806b500 0x7f50b806d9c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.927+0000 7f50db05d640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50d415c9c0 0x7f50d40cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.927+0000 7f50db05d640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50d40ad740 0x7f50d41f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.927+0000 7f50db05d640 1 --2- 172.21.15.139:0/2227259343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d406c1f0 0x7f50d4140cb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.927+0000 7f50db05d640 1 -- 172.21.15.139:0/2227259343 >> 172.21.15.139:0/2227259343 conn(0x7f50d40b23d0 msgr2=0x7f50d40afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.927+0000 7f50db05d640 1 -- 172.21.15.139:0/2227259343 shutdown_connections 2024-04-25T09:54:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:22.927+0000 7f50db05d640 1 -- 172.21.15.139:0/2227259343 wait complete. 2024-04-25T09:54:22.940 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:23.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.035+0000 7f8212c76640 1 Processor -- start 2024-04-25T09:54:23.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.036+0000 7f8212c76640 1 -- start start 2024-04-25T09:54:23.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.036+0000 7f8212c76640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f820c15dc60 0x7f820c0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.036+0000 7f8212c76640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820c0cf250 con 0x7f820c0d1080 2024-04-25T09:54:23.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.036+0000 7f8212c76640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820c0ce9b0 con 0x7f820c161850 2024-04-25T09:54:23.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.036+0000 7f8212c76640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0d3730 con 0x7f820c15dc60 2024-04-25T09:54:23.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.036+0000 7f8211213640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f820c0d1080 0x7f820c0d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54954/0 (socket says 172.21.15.139:54954) 2024-04-25T09:54:23.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.037+0000 7f8211213640 1 -- 172.21.15.139:0/895062499 learned_addr learned my addr 172.21.15.139:0/895062499 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:23.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.037+0000 7f8210a12640 1 --2- 172.21.15.139:0/895062499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f820c15dc60 0x7f820c0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.037+0000 7f820b7fe640 1 -- 172.21.15.139:0/895062499 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334294554 0 0) 0x7f8204001490 con 0x7f820c0d1080 2024-04-25T09:54:23.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.037+0000 7f820b7fe640 1 -- 172.21.15.139:0/895062499 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f4003520 con 0x7f820c0d1080 2024-04-25T09:54:23.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.037+0000 7f820b7fe640 1 -- 172.21.15.139:0/895062499 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166479132 0 0) 0x7f820c0ce9b0 con 0x7f820c161850 2024-04-25T09:54:23.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.037+0000 7f820b7fe640 1 -- 172.21.15.139:0/895062499 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8204001490 con 0x7f820c161850 2024-04-25T09:54:23.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.037+0000 7f8210a12640 1 -- 172.21.15.139:0/895062499 >> v1:172.21.15.150:6789/0 conn(0x7f820c161850 legacy=0x7f820c0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.037+0000 7f820b7fe640 1 -- 172.21.15.139:0/895062499 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 101230876 0 0) 0x7f81f4003520 con 0x7f820c0d1080 2024-04-25T09:54:23.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.037+0000 7f8210a12640 1 -- 172.21.15.139:0/895062499 >> v1:172.21.15.139:6789/0 conn(0x7f820c0d1080 legacy=0x7f820c0d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.037+0000 7f8210a12640 1 -- 172.21.15.139:0/895062499 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c0cec90 con 0x7f820c15dc60 2024-04-25T09:54:23.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.038+0000 7f8210a12640 1 --2- 172.21.15.139:0/895062499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f820c15dc60 0x7f820c0b6c10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0097d0 tx=0x7f81fc02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e96daa7a97f3b87c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:23.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.038+0000 7f820b7fe640 1 -- 172.21.15.139:0/895062499 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc037070 con 0x7f820c15dc60 2024-04-25T09:54:23.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.038+0000 7f820b7fe640 1 -- 172.21.15.139:0/895062499 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc02ac30 con 0x7f820c15dc60 2024-04-25T09:54:23.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.038+0000 7f820b7fe640 1 -- 172.21.15.139:0/895062499 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204001bc0 con 0x7f820c15dc60 2024-04-25T09:54:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.038+0000 7f8212c76640 1 -- 172.21.15.139:0/895062499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f820c15dc60 msgr2=0x7f820c0b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.038+0000 7f8212c76640 1 --2- 172.21.15.139:0/895062499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f820c15dc60 0x7f820c0b6c10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0097d0 tx=0x7f81fc02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.039+0000 7f8212c76640 1 -- 172.21.15.139:0/895062499 shutdown_connections 2024-04-25T09:54:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.039+0000 7f8212c76640 1 --2- 172.21.15.139:0/895062499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f820c15dc60 0x7f820c0b6c10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.039+0000 7f8212c76640 1 -- 172.21.15.139:0/895062499 >> 172.21.15.139:0/895062499 conn(0x7f820c0b5910 msgr2=0x7f820c0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.039+0000 7f8212c76640 1 -- 172.21.15.139:0/895062499 shutdown_connections 2024-04-25T09:54:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.039+0000 7f8212c76640 1 -- 172.21.15.139:0/895062499 wait complete. 2024-04-25T09:54:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.040+0000 7f8212c76640 1 Processor -- start 2024-04-25T09:54:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.040+0000 7f8212c76640 1 -- start start 2024-04-25T09:54:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8212c76640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f820c0d1080 0x7f820c154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8212c76640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f820c15dc60 0x7f820c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8212c76640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f820c161850 0x7f820c06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8212c76640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c15e040 con 0x7f820c0d1080 2024-04-25T09:54:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8212c76640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0d1fc0 con 0x7f820c161850 2024-04-25T09:54:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8212c76640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c1f4a90 con 0x7f820c15dc60 2024-04-25T09:54:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8210a12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f820c0d1080 0x7f820c154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8210a12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f820c0d1080 0x7f820c154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42146/0 (socket says 172.21.15.139:42146) 2024-04-25T09:54:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8210a12640 1 -- 172.21.15.139:0/779777288 learned_addr learned my addr 172.21.15.139:0/779777288 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f820bfff640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f820c15dc60 0x7f820c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8211213640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f820c161850 0x7f820c06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8210a12640 1 -- 172.21.15.139:0/779777288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f820c15dc60 msgr2=0x7f820c06c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8210a12640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f820c15dc60 0x7f820c06c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8210a12640 1 -- 172.21.15.139:0/779777288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f820c161850 msgr2=0x7f820c06c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8210a12640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f820c161850 0x7f820c06c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8210a12640 1 -- 172.21.15.139:0/779777288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c1f3a60 con 0x7f820c0d1080 2024-04-25T09:54:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f820bfff640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f820c15dc60 0x7f820c06c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8210a12640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f820c0d1080 0x7f820c154f00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f81fc035040 tx=0x7f81fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f82097fa640 1 -- 172.21.15.139:0/779777288 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc037050 con 0x7f820c0d1080 2024-04-25T09:54:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f82097fa640 1 -- 172.21.15.139:0/779777288 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc043070 con 0x7f820c0d1080 2024-04-25T09:54:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8212c76640 1 -- 172.21.15.139:0/779777288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c1f4f70 con 0x7f820c0d1080 2024-04-25T09:54:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f82097fa640 1 -- 172.21.15.139:0/779777288 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc006e10 con 0x7f820c0d1080 2024-04-25T09:54:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.041+0000 7f8212c76640 1 -- 172.21.15.139:0/779777288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c1f3fb0 con 0x7f820c0d1080 2024-04-25T09:54:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.042+0000 7f8212c76640 1 -- 172.21.15.139:0/779777288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820c074520 con 0x7f820c0d1080 2024-04-25T09:54:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.043+0000 7f82097fa640 1 -- 172.21.15.139:0/779777288 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81fc03ca10 con 0x7f820c0d1080 2024-04-25T09:54:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.043+0000 7f82097fa640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f81f806b4d0 0x7f81f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.043+0000 7f82097fa640 1 -- 172.21.15.139:0/779777288 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81fc0aae10 con 0x7f820c0d1080 2024-04-25T09:54:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.043+0000 7f820bfff640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f81f806b4d0 0x7f81f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.044+0000 7f820bfff640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f81f806b4d0 0x7f81f806d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f820c1f4e40 tx=0x7f8200009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.047+0000 7f82097fa640 1 -- 172.21.15.139:0/779777288 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81fc030e20 con 0x7f820c0d1080 2024-04-25T09:54:23.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.220+0000 7f8212c76640 1 -- 172.21.15.139:0/779777288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f820c0d16f0 con 0x7f820c0d1080 2024-04-25T09:54:23.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.221+0000 7f82097fa640 1 -- 172.21.15.139:0/779777288 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17003 (secure 0 0 0) 0x7f81fc079b00 con 0x7f820c0d1080 2024-04-25T09:54:23.224 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:23.224 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":37,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:21.988665+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:19.795203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:20.931184+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:23.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.224+0000 7f8212c76640 1 -- 172.21.15.139:0/779777288 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f81f806b4d0 msgr2=0x7f81f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.224+0000 7f8212c76640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f81f806b4d0 0x7f81f806d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f820c1f4e40 tx=0x7f8200009040 comp rx=0 tx=0).stop 2024-04-25T09:54:23.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.224+0000 7f8212c76640 1 -- 172.21.15.139:0/779777288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f820c0d1080 msgr2=0x7f820c154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.224+0000 7f8212c76640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f820c0d1080 0x7f820c154f00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f81fc035040 tx=0x7f81fc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.224+0000 7f8212c76640 1 -- 172.21.15.139:0/779777288 shutdown_connections 2024-04-25T09:54:23.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.224+0000 7f8212c76640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f81f806b4d0 0x7f81f806d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.224+0000 7f8212c76640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f820c161850 0x7f820c06c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.224+0000 7f8212c76640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f820c15dc60 0x7f820c06c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.224+0000 7f8212c76640 1 --2- 172.21.15.139:0/779777288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f820c0d1080 0x7f820c154f00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.224+0000 7f8212c76640 1 -- 172.21.15.139:0/779777288 >> 172.21.15.139:0/779777288 conn(0x7f820c0b5910 msgr2=0x7f820c0b8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:23.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.224+0000 7f8212c76640 1 -- 172.21.15.139:0/779777288 shutdown_connections 2024-04-25T09:54:23.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.224+0000 7f8212c76640 1 -- 172.21.15.139:0/779777288 wait complete. 2024-04-25T09:54:23.238 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:54:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.336+0000 7f8a22f4c640 1 Processor -- start 2024-04-25T09:54:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.336+0000 7f8a22f4c640 1 -- start start 2024-04-25T09:54:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.336+0000 7f8a22f4c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a1c0cee50 0x7f8a1c168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.336+0000 7f8a22f4c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1c161dc0 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.336+0000 7f8a22f4c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1c1620a0 con 0x7f8a1c1617e0 2024-04-25T09:54:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.336+0000 7f8a22f4c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c0d1ee0 con 0x7f8a1c0cee50 2024-04-25T09:54:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.336+0000 7f8a20ce8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8a1c15dbf0 0x7f8a1c0b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54968/0 (socket says 172.21.15.139:54968) 2024-04-25T09:54:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.336+0000 7f8a20ce8640 1 -- 172.21.15.139:0/1436279358 learned_addr learned my addr 172.21.15.139:0/1436279358 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.337+0000 7f8a214e9640 1 --2- 172.21.15.139:0/1436279358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a1c0cee50 0x7f8a1c168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.337+0000 7f8a1b7fe640 1 -- 172.21.15.139:0/1436279358 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718703139 0 0) 0x7f8a1c161dc0 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.337+0000 7f8a1b7fe640 1 -- 172.21.15.139:0/1436279358 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a0c003520 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.337+0000 7f8a1b7fe640 1 -- 172.21.15.139:0/1436279358 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101127589 0 0) 0x7f8a1c1620a0 con 0x7f8a1c1617e0 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.337+0000 7f8a1b7fe640 1 -- 172.21.15.139:0/1436279358 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a1c161dc0 con 0x7f8a1c1617e0 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.337+0000 7f8a1b7fe640 1 -- 172.21.15.139:0/1436279358 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 785737292 0 0) 0x7f8a0c003520 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.337+0000 7f8a214e9640 1 -- 172.21.15.139:0/1436279358 >> v1:172.21.15.150:6789/0 conn(0x7f8a1c1617e0 legacy=0x7f8a1c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.337+0000 7f8a214e9640 1 -- 172.21.15.139:0/1436279358 >> v1:172.21.15.139:6789/0 conn(0x7f8a1c15dbf0 legacy=0x7f8a1c0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.337+0000 7f8a214e9640 1 -- 172.21.15.139:0/1436279358 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c162380 con 0x7f8a1c0cee50 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.338+0000 7f8a214e9640 1 --2- 172.21.15.139:0/1436279358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a1c0cee50 0x7f8a1c168b10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0d2060 tx=0x7f8a1402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7a6133271d823d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.338+0000 7f8a1b7fe640 1 -- 172.21.15.139:0/1436279358 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10001670 con 0x7f8a1c0cee50 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.338+0000 7f8a1b7fe640 1 -- 172.21.15.139:0/1436279358 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a14007540 con 0x7f8a1c0cee50 2024-04-25T09:54:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.338+0000 7f8a1b7fe640 1 -- 172.21.15.139:0/1436279358 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14007910 con 0x7f8a1c0cee50 2024-04-25T09:54:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.338+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1436279358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a1c0cee50 msgr2=0x7f8a1c168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.338+0000 7f8a22f4c640 1 --2- 172.21.15.139:0/1436279358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a1c0cee50 0x7f8a1c168b10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0d2060 tx=0x7f8a1402a5d0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.338+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1436279358 shutdown_connections 2024-04-25T09:54:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.338+0000 7f8a22f4c640 1 --2- 172.21.15.139:0/1436279358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a1c0cee50 0x7f8a1c168b10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.338+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1436279358 >> 172.21.15.139:0/1436279358 conn(0x7f8a1c0b5480 msgr2=0x7f8a1c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.339+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1436279358 shutdown_connections 2024-04-25T09:54:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.339+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1436279358 wait complete. 2024-04-25T09:54:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.340+0000 7f8a22f4c640 1 Processor -- start 2024-04-25T09:54:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.340+0000 7f8a22f4c640 1 -- start start 2024-04-25T09:54:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a22f4c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a1c0cee50 0x7f8a1c15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a22f4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a1c15dbf0 0x7f8a1c14bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a22f4c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a1c1617e0 0x7f8a1c1607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a22f4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c15dfd0 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a22f4c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c0d3380 con 0x7f8a1c0cee50 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a22f4c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c0cbd60 con 0x7f8a1c1617e0 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a1bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a1c15dbf0 0x7f8a1c14bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a1bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a1c15dbf0 0x7f8a1c14bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42152/0 (socket says 172.21.15.139:42152) 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a1bfff640 1 -- 172.21.15.139:0/1090089013 learned_addr learned my addr 172.21.15.139:0/1090089013 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a214e9640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a1c1617e0 0x7f8a1c1607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a20ce8640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a1c0cee50 0x7f8a1c15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a1bfff640 1 -- 172.21.15.139:0/1090089013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a1c1617e0 msgr2=0x7f8a1c1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a1bfff640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a1c1617e0 0x7f8a1c1607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a1bfff640 1 -- 172.21.15.139:0/1090089013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a1c0cee50 msgr2=0x7f8a1c15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a1bfff640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a1c0cee50 0x7f8a1c15e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a1bfff640 1 -- 172.21.15.139:0/1090089013 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c150a90 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a20ce8640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a1c0cee50 0x7f8a1c15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a1bfff640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a1c15dbf0 0x7f8a1c14bc60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800c430 tx=0x7f8a0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a197fa640 1 -- 172.21.15.139:0/1090089013 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08017070 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a197fa640 1 -- 172.21.15.139:0/1090089013 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a080052f0 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1090089013 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1c0cc000 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a197fa640 1 -- 172.21.15.139:0/1090089013 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08003830 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.341+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1090089013 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1c0cc5c0 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.342+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1090089013 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1c074520 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.343+0000 7f8a197fa640 1 -- 172.21.15.139:0/1090089013 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a0801a070 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.343+0000 7f8a197fa640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8a0006b500 0x7f8a0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.343+0000 7f8a197fa640 1 -- 172.21.15.139:0/1090089013 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8a080897b0 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.343+0000 7f8a20ce8640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8a0006b500 0x7f8a0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.344+0000 7f8a20ce8640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8a0006b500 0x7f8a0006d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a100017e0 tx=0x7f8a10009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:23.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.348+0000 7f8a197fa640 1 -- 172.21.15.139:0/1090089013 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a080584a0 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.554+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1090089013 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8a1c0d1680 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.623+0000 7f8a197fa640 1 -- 172.21.15.139:0/1090089013 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8a08057da0 con 0x7f8a1c15dbf0 2024-04-25T09:54:23.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.626+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1090089013 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8a0006b500 msgr2=0x7f8a0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.626+0000 7f8a22f4c640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8a0006b500 0x7f8a0006d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a100017e0 tx=0x7f8a10009040 comp rx=0 tx=0).stop 2024-04-25T09:54:23.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.626+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1090089013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a1c15dbf0 msgr2=0x7f8a1c14bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.626+0000 7f8a22f4c640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a1c15dbf0 0x7f8a1c14bc60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800c430 tx=0x7f8a0800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.626+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1090089013 shutdown_connections 2024-04-25T09:54:23.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.626+0000 7f8a22f4c640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a1c15dbf0 0x7f8a1c14bc60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.626+0000 7f8a22f4c640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8a0006b500 0x7f8a0006d9c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.626+0000 7f8a22f4c640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a1c1617e0 0x7f8a1c1607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.626+0000 7f8a22f4c640 1 --2- 172.21.15.139:0/1090089013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a1c0cee50 0x7f8a1c15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.626+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1090089013 >> 172.21.15.139:0/1090089013 conn(0x7f8a1c0b5480 msgr2=0x7f8a1c158380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:23.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.626+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1090089013 shutdown_connections 2024-04-25T09:54:23.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.626+0000 7f8a22f4c640 1 -- 172.21.15.139:0/1090089013 wait complete. 2024-04-25T09:54:23.639 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:54:23.639 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:54:23.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.738+0000 7f867befc640 1 Processor -- start 2024-04-25T09:54:23.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.738+0000 7f867befc640 1 -- start start 2024-04-25T09:54:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f867befc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f867415dbf0 0x7f86741617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f867befc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f86740cee50 0x7f86740d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f867befc640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8674161dc0 con 0x7f86740b9370 2024-04-25T09:54:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f867befc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740d1ee0 con 0x7f86740cee50 2024-04-25T09:54:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f867befc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86741620a0 con 0x7f867415dbf0 2024-04-25T09:54:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f867a499640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f86740cee50 0x7f86740d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f867a499640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f86740cee50 0x7f86740d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42158/0 (socket says 172.21.15.139:42158) 2024-04-25T09:54:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f867a499640 1 -- 172.21.15.139:0/3089406370 learned_addr learned my addr 172.21.15.139:0/3089406370 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f867a499640 1 -- 172.21.15.139:0/3089406370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f867415dbf0 msgr2=0x7f86741617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f8679c98640 1 --2- 172.21.15.139:0/3089406370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f867415dbf0 0x7f86741617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f867a499640 1 --2- 172.21.15.139:0/3089406370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f867415dbf0 0x7f86741617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f867a499640 1 -- 172.21.15.139:0/3089406370 >> v1:172.21.15.150:6789/0 conn(0x7f86740b9370 legacy=0x7f86740ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f867a499640 1 -- 172.21.15.139:0/3089406370 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8674162380 con 0x7f86740cee50 2024-04-25T09:54:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.739+0000 7f8679c98640 1 --2- 172.21.15.139:0/3089406370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f867415dbf0 0x7f86741617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.740+0000 7f867a499640 1 --2- 172.21.15.139:0/3089406370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86740cee50 0x7f86740d2370 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f866c009e50 tx=0x7f866c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd300e1ee75ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.740+0000 7f8678c96640 1 -- 172.21.15.139:0/3089406370 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c036070 con 0x7f86740cee50 2024-04-25T09:54:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.740+0000 7f8678c96640 1 -- 172.21.15.139:0/3089406370 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c031600 con 0x7f86740cee50 2024-04-25T09:54:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.740+0000 7f8678c96640 1 -- 172.21.15.139:0/3089406370 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c0319d0 con 0x7f86740cee50 2024-04-25T09:54:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.740+0000 7f867befc640 1 -- 172.21.15.139:0/3089406370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86740cee50 msgr2=0x7f86740d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.741+0000 7f867befc640 1 --2- 172.21.15.139:0/3089406370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86740cee50 0x7f86740d2370 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f866c009e50 tx=0x7f866c02a6d0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.741+0000 7f867befc640 1 -- 172.21.15.139:0/3089406370 shutdown_connections 2024-04-25T09:54:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.741+0000 7f867befc640 1 --2- 172.21.15.139:0/3089406370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86740cee50 0x7f86740d2370 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.741+0000 7f867befc640 1 --2- 172.21.15.139:0/3089406370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f867415dbf0 0x7f86741617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.741+0000 7f867befc640 1 -- 172.21.15.139:0/3089406370 >> 172.21.15.139:0/3089406370 conn(0x7f86740b5480 msgr2=0x7f86740b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.741+0000 7f867befc640 1 -- 172.21.15.139:0/3089406370 shutdown_connections 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.741+0000 7f867befc640 1 -- 172.21.15.139:0/3089406370 wait complete. 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.742+0000 7f867befc640 1 Processor -- start 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.742+0000 7f867befc640 1 -- start start 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.742+0000 7f867befc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86740b9370 0x7f8674153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.742+0000 7f867befc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86740cee50 0x7f867415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867befc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867415dbf0 0x7f8674155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867befc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867415dfd0 con 0x7f867415dbf0 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867befc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86740b97d0 con 0x7f86740b9370 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867befc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867415e820 con 0x7f86740cee50 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867a499640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867415dbf0 0x7f8674155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867a499640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867415dbf0 0x7f8674155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42166/0 (socket says 172.21.15.139:42166) 2024-04-25T09:54:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867a499640 1 -- 172.21.15.139:0/705281021 learned_addr learned my addr 172.21.15.139:0/705281021 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f8679497640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86740cee50 0x7f867415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f8679c98640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86740b9370 0x7f8674153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867a499640 1 -- 172.21.15.139:0/705281021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86740cee50 msgr2=0x7f867415e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867a499640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86740cee50 0x7f867415e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867a499640 1 -- 172.21.15.139:0/705281021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86740b9370 msgr2=0x7f8674153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867a499640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86740b9370 0x7f8674153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867a499640 1 -- 172.21.15.139:0/705281021 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867415ea50 con 0x7f867415dbf0 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f8679497640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86740cee50 0x7f867415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867a499640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867415dbf0 0x7f8674155620 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f866c034040 tx=0x7f866c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f866a7fc640 1 -- 172.21.15.139:0/705281021 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c036050 con 0x7f867415dbf0 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f866a7fc640 1 -- 172.21.15.139:0/705281021 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c042070 con 0x7f867415dbf0 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867befc640 1 -- 172.21.15.139:0/705281021 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867415ed30 con 0x7f867415dbf0 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f866a7fc640 1 -- 172.21.15.139:0/705281021 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c009510 con 0x7f867415dbf0 2024-04-25T09:54:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.743+0000 7f867befc640 1 -- 172.21.15.139:0/705281021 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867415f2f0 con 0x7f867415dbf0 2024-04-25T09:54:23.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.744+0000 7f867befc640 1 -- 172.21.15.139:0/705281021 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8674074520 con 0x7f867415dbf0 2024-04-25T09:54:23.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.745+0000 7f866a7fc640 1 -- 172.21.15.139:0/705281021 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f866c007bd0 con 0x7f867415dbf0 2024-04-25T09:54:23.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.745+0000 7f866a7fc640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f865806b500 0x7f865806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.745+0000 7f866a7fc640 1 -- 172.21.15.139:0/705281021 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f866c0ab290 con 0x7f867415dbf0 2024-04-25T09:54:23.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.745+0000 7f8679c98640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f865806b500 0x7f865806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.746+0000 7f8679c98640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f865806b500 0x7f865806d9c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f867000b840 tx=0x7f8670008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:23.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.749+0000 7f866a7fc640 1 -- 172.21.15.139:0/705281021 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f866c045360 con 0x7f867415dbf0 2024-04-25T09:54:23.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:23.954+0000 7f867befc640 1 -- 172.21.15.139:0/705281021 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f86740d1680 con 0x7f867415dbf0 2024-04-25T09:54:24.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.628+0000 7f866a7fc640 1 -- 172.21.15.139:0/705281021 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f866c07a010 con 0x7f867415dbf0 2024-04-25T09:54:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.631+0000 7f867befc640 1 -- 172.21.15.139:0/705281021 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f865806b500 msgr2=0x7f865806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.631+0000 7f867befc640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f865806b500 0x7f865806d9c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f867000b840 tx=0x7f8670008040 comp rx=0 tx=0).stop 2024-04-25T09:54:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.631+0000 7f867befc640 1 -- 172.21.15.139:0/705281021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867415dbf0 msgr2=0x7f8674155620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.631+0000 7f867befc640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867415dbf0 0x7f8674155620 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f866c034040 tx=0x7f866c004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.631+0000 7f867befc640 1 -- 172.21.15.139:0/705281021 shutdown_connections 2024-04-25T09:54:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.631+0000 7f867befc640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f867415dbf0 0x7f8674155620 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.631+0000 7f867befc640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f865806b500 0x7f865806d9c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.631+0000 7f867befc640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86740cee50 0x7f867415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.631+0000 7f867befc640 1 --2- 172.21.15.139:0/705281021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86740b9370 0x7f8674153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.631+0000 7f867befc640 1 -- 172.21.15.139:0/705281021 >> 172.21.15.139:0/705281021 conn(0x7f86740b5480 msgr2=0x7f86740b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:24.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.631+0000 7f867befc640 1 -- 172.21.15.139:0/705281021 shutdown_connections 2024-04-25T09:54:24.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.631+0000 7f867befc640 1 -- 172.21.15.139:0/705281021 wait complete. 2024-04-25T09:54:24.645 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:54:24.645 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.741+0000 7f55301b0640 1 Processor -- start 2024-04-25T09:54:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.741+0000 7f55301b0640 1 -- start start 2024-04-25T09:54:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.742+0000 7f55301b0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f55280d0730 0x7f55280cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.742+0000 7f55301b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55280d0b90 0x7f55280cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.742+0000 7f55301b0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55280cc8c0 con 0x7f55280b0750 2024-04-25T09:54:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.742+0000 7f55301b0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55280ccb70 con 0x7f55280d0730 2024-04-25T09:54:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.742+0000 7f55301b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55280cf970 con 0x7f55280d0b90 2024-04-25T09:54:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.742+0000 7f552d74b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f55280d0730 0x7f55280cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.742+0000 7f552d74b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f55280d0730 0x7f55280cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42168/0 (socket says 172.21.15.139:42168) 2024-04-25T09:54:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.742+0000 7f552d74b640 1 -- 172.21.15.139:0/3820411840 learned_addr learned my addr 172.21.15.139:0/3820411840 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.743+0000 7f552e74d640 1 --2- 172.21.15.139:0/3820411840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55280d0b90 0x7f55280cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.743+0000 7f552d74b640 1 -- 172.21.15.139:0/3820411840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55280d0b90 msgr2=0x7f55280cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.743+0000 7f552d74b640 1 --2- 172.21.15.139:0/3820411840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55280d0b90 0x7f55280cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.743+0000 7f552d74b640 1 -- 172.21.15.139:0/3820411840 >> v1:172.21.15.150:6789/0 conn(0x7f55280b0750 legacy=0x7f55280b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:24.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.743+0000 7f552d74b640 1 -- 172.21.15.139:0/3820411840 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55280cfc50 con 0x7f55280d0730 2024-04-25T09:54:24.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.743+0000 7f552d74b640 1 --2- 172.21.15.139:0/3820411840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55280d0730 0x7f55280cbe40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5518009870 tx=0x7f551802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0dd2ba58d47b512 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:24.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.744+0000 7f552cf4a640 1 -- 172.21.15.139:0/3820411840 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518037070 con 0x7f55280d0730 2024-04-25T09:54:24.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.744+0000 7f552cf4a640 1 -- 172.21.15.139:0/3820411840 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5518032440 con 0x7f55280d0730 2024-04-25T09:54:24.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.744+0000 7f552cf4a640 1 -- 172.21.15.139:0/3820411840 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518032810 con 0x7f55280d0730 2024-04-25T09:54:24.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.744+0000 7f55301b0640 1 -- 172.21.15.139:0/3820411840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55280d0730 msgr2=0x7f55280cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.744+0000 7f55301b0640 1 --2- 172.21.15.139:0/3820411840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55280d0730 0x7f55280cbe40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5518009870 tx=0x7f551802a210 comp rx=0 tx=0).stop 2024-04-25T09:54:24.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.744+0000 7f55301b0640 1 -- 172.21.15.139:0/3820411840 shutdown_connections 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.744+0000 7f55301b0640 1 --2- 172.21.15.139:0/3820411840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55280d0b90 0x7f55280cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.744+0000 7f55301b0640 1 --2- 172.21.15.139:0/3820411840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55280d0730 0x7f55280cbe40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.744+0000 7f55301b0640 1 -- 172.21.15.139:0/3820411840 >> 172.21.15.139:0/3820411840 conn(0x7f55280b3380 msgr2=0x7f55280b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.744+0000 7f55301b0640 1 -- 172.21.15.139:0/3820411840 shutdown_connections 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.744+0000 7f55301b0640 1 -- 172.21.15.139:0/3820411840 wait complete. 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.745+0000 7f55301b0640 1 Processor -- start 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.745+0000 7f55301b0640 1 -- start start 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f55301b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55280b0750 0x7f5528150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f55301b0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55280d0730 0x7f552806bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f55301b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f552806d0c0 0x7f552815c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f55301b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552815bcc0 con 0x7f55280b0750 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f55301b0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55280d0210 con 0x7f55280d0730 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f55301b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552815be40 con 0x7f552806d0c0 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f552df4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55280b0750 0x7f5528150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f552df4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55280b0750 0x7f5528150e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42176/0 (socket says 172.21.15.139:42176) 2024-04-25T09:54:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f552df4c640 1 -- 172.21.15.139:0/555121178 learned_addr learned my addr 172.21.15.139:0/555121178 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f552df4c640 1 -- 172.21.15.139:0/555121178 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f552806d0c0 msgr2=0x7f552815c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f552df4c640 1 --2- 172.21.15.139:0/555121178 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f552806d0c0 0x7f552815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f552df4c640 1 -- 172.21.15.139:0/555121178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55280d0730 msgr2=0x7f552806bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f552df4c640 1 --2- 172.21.15.139:0/555121178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55280d0730 0x7f552806bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.746+0000 7f552df4c640 1 -- 172.21.15.139:0/555121178 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552806c570 con 0x7f55280b0750 2024-04-25T09:54:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.747+0000 7f552df4c640 1 --2- 172.21.15.139:0/555121178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55280b0750 0x7f5528150e10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f552400ba50 tx=0x7f552400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.747+0000 7f551e7fc640 1 -- 172.21.15.139:0/555121178 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524019070 con 0x7f55280b0750 2024-04-25T09:54:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.747+0000 7f551e7fc640 1 -- 172.21.15.139:0/555121178 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552400cbc0 con 0x7f55280b0750 2024-04-25T09:54:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.747+0000 7f551e7fc640 1 -- 172.21.15.139:0/555121178 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524014440 con 0x7f55280b0750 2024-04-25T09:54:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.747+0000 7f55301b0640 1 -- 172.21.15.139:0/555121178 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552806c8b0 con 0x7f55280b0750 2024-04-25T09:54:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.747+0000 7f55301b0640 1 -- 172.21.15.139:0/555121178 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552815c2f0 con 0x7f55280b0750 2024-04-25T09:54:24.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.748+0000 7f55301b0640 1 -- 172.21.15.139:0/555121178 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5528074520 con 0x7f55280b0750 2024-04-25T09:54:24.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.748+0000 7f551e7fc640 1 -- 172.21.15.139:0/555121178 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f552401c070 con 0x7f55280b0750 2024-04-25T09:54:24.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.748+0000 7f551e7fc640 1 --2- 172.21.15.139:0/555121178 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f550c06b500 0x7f550c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:24.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.749+0000 7f551e7fc640 1 -- 172.21.15.139:0/555121178 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5524089f50 con 0x7f55280b0750 2024-04-25T09:54:24.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.749+0000 7f552d74b640 1 --2- 172.21.15.139:0/555121178 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f550c06b500 0x7f550c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:24.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.749+0000 7f552d74b640 1 --2- 172.21.15.139:0/555121178 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f550c06b500 0x7f550c06d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5518034000 tx=0x7f551802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:24.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.753+0000 7f551e7fc640 1 -- 172.21.15.139:0/555121178 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f552806c8b0 con 0x7f55280b0750 2024-04-25T09:54:24.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.955+0000 7f55301b0640 1 -- 172.21.15.139:0/555121178 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5528151350 con 0x7f55280b0750 2024-04-25T09:54:24.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.955+0000 7f551e7fc640 1 -- 172.21.15.139:0/555121178 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f5524058680 con 0x7f55280b0750 2024-04-25T09:54:24.958 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:24.958 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/511562594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":511562594},{"type":"v1","addr":"172.21.15.150:6834","nonce":511562594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5041,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/225357317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":225357317},{"type":"v1","addr":"172.21.15.150:6835","nonce":225357317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2499407982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2499407982},{"type":"v1","addr":"172.21.15.139:6836","nonce":2499407982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:21.988975+0000","modified":"2024-04-25T09:54:23.955620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/2069784668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2069784668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2069784668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:54:24.958 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 16 2024-04-25T09:54:24.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.957+0000 7f55301b0640 1 -- 172.21.15.139:0/555121178 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f550c06b500 msgr2=0x7f550c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.957+0000 7f55301b0640 1 --2- 172.21.15.139:0/555121178 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f550c06b500 0x7f550c06d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5518034000 tx=0x7f551802c000 comp rx=0 tx=0).stop 2024-04-25T09:54:24.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.957+0000 7f55301b0640 1 -- 172.21.15.139:0/555121178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55280b0750 msgr2=0x7f5528150e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.957+0000 7f55301b0640 1 --2- 172.21.15.139:0/555121178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55280b0750 0x7f5528150e10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f552400ba50 tx=0x7f552400bf10 comp rx=0 tx=0).stop 2024-04-25T09:54:24.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.957+0000 7f55301b0640 1 -- 172.21.15.139:0/555121178 shutdown_connections 2024-04-25T09:54:24.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.957+0000 7f55301b0640 1 --2- 172.21.15.139:0/555121178 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f550c06b500 0x7f550c06d9c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.957+0000 7f55301b0640 1 --2- 172.21.15.139:0/555121178 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f552806d0c0 0x7f552815c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.957+0000 7f55301b0640 1 --2- 172.21.15.139:0/555121178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55280d0730 0x7f552806bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.957+0000 7f55301b0640 1 --2- 172.21.15.139:0/555121178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55280b0750 0x7f5528150e10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.957+0000 7f55301b0640 1 -- 172.21.15.139:0/555121178 >> 172.21.15.139:0/555121178 conn(0x7f55280b3380 msgr2=0x7f55280b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:24.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.957+0000 7f55301b0640 1 -- 172.21.15.139:0/555121178 shutdown_connections 2024-04-25T09:54:24.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:24.958+0000 7f55301b0640 1 -- 172.21.15.139:0/555121178 wait complete. 2024-04-25T09:54:24.971 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:25.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.067+0000 7f1396215640 1 Processor -- start 2024-04-25T09:54:25.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.067+0000 7f1396215640 1 -- start start 2024-04-25T09:54:25.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f1396215640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f139015aa00 0x7f13900b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f1396215640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1390072280 0x7f1390072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f1396215640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139015e500 con 0x7f13900b5110 2024-04-25T09:54:25.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f1396215640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13900b5c20 con 0x7f1390072280 2024-04-25T09:54:25.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f1396215640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13900b45d0 con 0x7f139015aa00 2024-04-25T09:54:25.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f1387fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f13900b5110 0x7f139015ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54984/0 (socket says 172.21.15.139:54984) 2024-04-25T09:54:25.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f1387fff640 1 -- 172.21.15.139:0/3823386492 learned_addr learned my addr 172.21.15.139:0/3823386492 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:25.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138f7fe640 1 --2- 172.21.15.139:0/3823386492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f139015aa00 0x7f13900b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138ffff640 1 --2- 172.21.15.139:0/3823386492 >> v2:172.21.15.150:3300/0 conn(0x7f1390072280 0x7f1390072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409868575 0 0) 0x7f139015e500 con 0x7f13900b5110 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f137c003520 con 0x7f13900b5110 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4187684617 0 0) 0x7f137c003520 con 0x7f13900b5110 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f139015aa00 msgr2=0x7f13900b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 --2- 172.21.15.139:0/3823386492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f139015aa00 0x7f13900b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 >> v2:172.21.15.150:3300/0 conn(0x7f1390072280 msgr2=0x7f1390072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 --2- 172.21.15.139:0/3823386492 >> v2:172.21.15.150:3300/0 conn(0x7f1390072280 0x7f1390072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13900b5660 con 0x7f13900b5110 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f1378001670 con 0x7f13900b5110 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 >> v1:172.21.15.139:6789/0 conn(0x7f13900b5110 legacy=0x7f139015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 --2- 172.21.15.139:0/3823386492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c0039b0 0x7f137c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 --2- 172.21.15.139:0/3823386492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f137c004800 0x7f137c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 --2- 172.21.15.139:0/3823386492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137c005180 0x7f137c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13900b5c20 con 0x7f137c0039b0 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13900b45d0 con 0x7f137c005180 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c004630 con 0x7f137c004800 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.068+0000 7f138f7fe640 1 --2- 172.21.15.139:0/3823386492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c0039b0 0x7f137c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.069+0000 7f1387fff640 1 --2- 172.21.15.139:0/3823386492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f137c004800 0x7f137c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.069+0000 7f138ffff640 1 --2- 172.21.15.139:0/3823386492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137c005180 0x7f137c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.069+0000 7f138f7fe640 1 -- 172.21.15.139:0/3823386492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f137c004800 msgr2=0x7f137c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.069+0000 7f138f7fe640 1 --2- 172.21.15.139:0/3823386492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f137c004800 0x7f137c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.069+0000 7f138f7fe640 1 -- 172.21.15.139:0/3823386492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137c005180 msgr2=0x7f137c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.069+0000 7f138f7fe640 1 --2- 172.21.15.139:0/3823386492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137c005180 0x7f137c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.069+0000 7f138f7fe640 1 -- 172.21.15.139:0/3823386492 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c003520 con 0x7f137c0039b0 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.069+0000 7f138f7fe640 1 --2- 172.21.15.139:0/3823386492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c0039b0 0x7f137c003dd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1380009ac0 tx=0x7f137c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccca663d44d6272f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.070+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380036070 con 0x7f137c0039b0 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.070+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1380031400 con 0x7f137c0039b0 2024-04-25T09:54:25.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.070+0000 7f138effd640 1 -- 172.21.15.139:0/3823386492 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380009690 con 0x7f137c0039b0 2024-04-25T09:54:25.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.070+0000 7f1396215640 1 -- 172.21.15.139:0/3823386492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c0039b0 msgr2=0x7f137c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.070+0000 7f1396215640 1 --2- 172.21.15.139:0/3823386492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c0039b0 0x7f137c003dd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1380009ac0 tx=0x7f137c005830 comp rx=0 tx=0).stop 2024-04-25T09:54:25.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.070+0000 7f138ffff640 1 -- 172.21.15.139:0/3823386492 reap_dead start 2024-04-25T09:54:25.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.070+0000 7f1396215640 1 -- 172.21.15.139:0/3823386492 shutdown_connections 2024-04-25T09:54:25.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.070+0000 7f1396215640 1 -- 172.21.15.139:0/3823386492 >> 172.21.15.139:0/3823386492 conn(0x7f139006e280 msgr2=0x7f139006e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:25.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.070+0000 7f1396215640 1 -- 172.21.15.139:0/3823386492 shutdown_connections 2024-04-25T09:54:25.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.070+0000 7f1396215640 1 -- 172.21.15.139:0/3823386492 wait complete. 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.071+0000 7f1396215640 1 Processor -- start 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.071+0000 7f1396215640 1 -- start start 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.072+0000 7f1396215640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137c0039b0 0x7f1390153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.072+0000 7f1396215640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1390158a30 0x7f1390158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.072+0000 7f1396215640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f13901575d0 0x7f1390157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.072+0000 7f1396215640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13900b4090 con 0x7f1390158a30 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.072+0000 7f1396215640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139015e7a0 con 0x7f137c0039b0 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.072+0000 7f1396215640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13900b4210 con 0x7f13901575d0 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.072+0000 7f1387fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1390158a30 0x7f1390158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.072+0000 7f1387fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1390158a30 0x7f1390158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42196/0 (socket says 172.21.15.139:42196) 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.072+0000 7f1387fff640 1 -- 172.21.15.139:0/1829955188 learned_addr learned my addr 172.21.15.139:0/1829955188 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.072+0000 7f138ffff640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f13901575d0 0x7f1390157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.072+0000 7f138f7fe640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137c0039b0 0x7f1390153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.073+0000 7f1387fff640 1 -- 172.21.15.139:0/1829955188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f13901575d0 msgr2=0x7f1390157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.073+0000 7f1387fff640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f13901575d0 0x7f1390157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.073+0000 7f1387fff640 1 -- 172.21.15.139:0/1829955188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137c0039b0 msgr2=0x7f1390153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.073+0000 7f1387fff640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137c0039b0 0x7f1390153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.073+0000 7f1387fff640 1 -- 172.21.15.139:0/1829955188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13901f2330 con 0x7f1390158a30 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.073+0000 7f1387fff640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1390158a30 0x7f1390158e90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1378007310 tx=0x7f13780077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.073+0000 7f138cff9640 1 -- 172.21.15.139:0/1829955188 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137800f8e0 con 0x7f1390158a30 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.073+0000 7f138cff9640 1 -- 172.21.15.139:0/1829955188 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378018070 con 0x7f1390158a30 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.073+0000 7f1396215640 1 -- 172.21.15.139:0/1829955188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13900a84b0 con 0x7f1390158a30 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.073+0000 7f138cff9640 1 -- 172.21.15.139:0/1829955188 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137800fd20 con 0x7f1390158a30 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.073+0000 7f1396215640 1 -- 172.21.15.139:0/1829955188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13901f2880 con 0x7f1390158a30 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.074+0000 7f1396215640 1 -- 172.21.15.139:0/1829955188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1390074520 con 0x7f1390158a30 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.074+0000 7f138cff9640 1 -- 172.21.15.139:0/1829955188 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f137801d460 con 0x7f1390158a30 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.075+0000 7f138cff9640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f137006b500 0x7f137006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.075+0000 7f138cff9640 1 -- 172.21.15.139:0/1829955188 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f137808a180 con 0x7f1390158a30 2024-04-25T09:54:25.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.075+0000 7f138f7fe640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f137006b500 0x7f137006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.075+0000 7f138f7fe640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f137006b500 0x7f137006d9c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1380033000 tx=0x7f1380008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:25.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.079+0000 7f138cff9640 1 -- 172.21.15.139:0/1829955188 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f13900a84b0 con 0x7f1390158a30 2024-04-25T09:54:25.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.251+0000 7f1396215640 1 -- 172.21.15.139:0/1829955188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f139006f180 con 0x7f1390158a30 2024-04-25T09:54:25.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.252+0000 7f138cff9640 1 -- 172.21.15.139:0/1829955188 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f1378058850 con 0x7f1390158a30 2024-04-25T09:54:25.254 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:25.254 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":38,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:23.620264+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:19.795203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:20.931184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.254+0000 7f1396215640 1 -- 172.21.15.139:0/1829955188 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f137006b500 msgr2=0x7f137006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.254+0000 7f1396215640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f137006b500 0x7f137006d9c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1380033000 tx=0x7f1380008030 comp rx=0 tx=0).stop 2024-04-25T09:54:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.254+0000 7f1396215640 1 -- 172.21.15.139:0/1829955188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1390158a30 msgr2=0x7f1390158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.254+0000 7f1396215640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1390158a30 0x7f1390158e90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1378007310 tx=0x7f13780077d0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.254+0000 7f1396215640 1 -- 172.21.15.139:0/1829955188 shutdown_connections 2024-04-25T09:54:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.254+0000 7f1396215640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1390158a30 0x7f1390158e90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.254+0000 7f1396215640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f137006b500 0x7f137006d9c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.254+0000 7f1396215640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f13901575d0 0x7f1390157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.254+0000 7f1396215640 1 --2- 172.21.15.139:0/1829955188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137c0039b0 0x7f1390153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.254+0000 7f1396215640 1 -- 172.21.15.139:0/1829955188 >> 172.21.15.139:0/1829955188 conn(0x7f139006e280 msgr2=0x7f13900ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.254+0000 7f1396215640 1 -- 172.21.15.139:0/1829955188 shutdown_connections 2024-04-25T09:54:25.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.254+0000 7f1396215640 1 -- 172.21.15.139:0/1829955188 wait complete. 2024-04-25T09:54:25.268 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:54:25.268 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:54:25.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.363+0000 7ff7a1cf5640 1 Processor -- start 2024-04-25T09:54:25.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.363+0000 7ff7a1cf5640 1 -- start start 2024-04-25T09:54:25.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff7a1cf5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff79c160e90 0x7ff79c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff7a1cf5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff79c06ab60 0x7ff79c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff7a1cf5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff79c06fa70 0x7ff79c06ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff7a1cf5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79c0d14a0 con 0x7ff79c06ab60 2024-04-25T09:54:25.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff7a1cf5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79c0d0400 con 0x7ff79c160e90 2024-04-25T09:54:25.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff7a1cf5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c0d06e0 con 0x7ff79c06fa70 2024-04-25T09:54:25.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff79affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff79c06ab60 0x7ff79c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff79affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff79c06ab60 0x7ff79c06aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42202/0 (socket says 172.21.15.139:42202) 2024-04-25T09:54:25.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff79affd640 1 -- 172.21.15.139:0/1726047400 learned_addr learned my addr 172.21.15.139:0/1726047400 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:25.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff79bfff640 1 --2- 172.21.15.139:0/1726047400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff79c06fa70 0x7ff79c06ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff79b7fe640 1 --2- 172.21.15.139:0/1726047400 >> v2:172.21.15.150:3300/0 conn(0x7ff79c160e90 0x7ff79c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff79affd640 1 -- 172.21.15.139:0/1726047400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff79c06fa70 msgr2=0x7ff79c06ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff79affd640 1 --2- 172.21.15.139:0/1726047400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff79c06fa70 0x7ff79c06ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff79affd640 1 -- 172.21.15.139:0/1726047400 >> v2:172.21.15.150:3300/0 conn(0x7ff79c160e90 msgr2=0x7ff79c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff79affd640 1 --2- 172.21.15.139:0/1726047400 >> v2:172.21.15.150:3300/0 conn(0x7ff79c160e90 0x7ff79c0afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff79affd640 1 -- 172.21.15.139:0/1726047400 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c0d09c0 con 0x7ff79c06ab60 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff79b7fe640 1 --2- 172.21.15.139:0/1726047400 >> v2:172.21.15.150:3300/0 conn(0x7ff79c160e90 0x7ff79c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.364+0000 7ff79bfff640 1 --2- 172.21.15.139:0/1726047400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff79c06fa70 0x7ff79c06ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.365+0000 7ff79affd640 1 --2- 172.21.15.139:0/1726047400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff79c06ab60 0x7ff79c06aff0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009870 tx=0x7ff78c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7dabb609358c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.365+0000 7ff79a7fc640 1 -- 172.21.15.139:0/1726047400 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c037070 con 0x7ff79c06ab60 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.365+0000 7ff79a7fc640 1 -- 172.21.15.139:0/1726047400 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c032440 con 0x7ff79c06ab60 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.365+0000 7ff79a7fc640 1 -- 172.21.15.139:0/1726047400 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c032810 con 0x7ff79c06ab60 2024-04-25T09:54:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.365+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/1726047400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff79c06ab60 msgr2=0x7ff79c06aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.365+0000 7ff7a1cf5640 1 --2- 172.21.15.139:0/1726047400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff79c06ab60 0x7ff79c06aff0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009870 tx=0x7ff78c02a210 comp rx=0 tx=0).stop 2024-04-25T09:54:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.365+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/1726047400 shutdown_connections 2024-04-25T09:54:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.365+0000 7ff7a1cf5640 1 --2- 172.21.15.139:0/1726047400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff79c06fa70 0x7ff79c06ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.365+0000 7ff7a1cf5640 1 --2- 172.21.15.139:0/1726047400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff79c06ab60 0x7ff79c06aff0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.365+0000 7ff7a1cf5640 1 --2- 172.21.15.139:0/1726047400 >> v2:172.21.15.150:3300/0 conn(0x7ff79c160e90 0x7ff79c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.365+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/1726047400 >> 172.21.15.139:0/1726047400 conn(0x7ff79c0b23f0 msgr2=0x7ff79c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.365+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/1726047400 shutdown_connections 2024-04-25T09:54:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.366+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/1726047400 wait complete. 2024-04-25T09:54:25.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.367+0000 7ff7a1cf5640 1 Processor -- start 2024-04-25T09:54:25.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.367+0000 7ff7a1cf5640 1 -- start start 2024-04-25T09:54:25.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.367+0000 7ff7a1cf5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff79c06ab60 0x7ff79c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff7a1cf5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff79c06fa70 0x7ff79c146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff7a1cf5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c160e90 0x7ff79c0cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff7a1cf5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c0b0000 con 0x7ff79c06fa70 2024-04-25T09:54:25.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff7a1cf5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c0d0f80 con 0x7ff79c160e90 2024-04-25T09:54:25.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff7a1cf5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c161270 con 0x7ff79c06ab60 2024-04-25T09:54:25.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff79c06fa70 0x7ff79c146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff79c06fa70 0x7ff79c146a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42206/0 (socket says 172.21.15.139:42206) 2024-04-25T09:54:25.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79affd640 1 -- 172.21.15.139:0/3294419077 learned_addr learned my addr 172.21.15.139:0/3294419077 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79bfff640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c160e90 0x7ff79c0cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79b7fe640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff79c06ab60 0x7ff79c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79affd640 1 -- 172.21.15.139:0/3294419077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff79c06ab60 msgr2=0x7ff79c15e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79affd640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff79c06ab60 0x7ff79c15e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79affd640 1 -- 172.21.15.139:0/3294419077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c160e90 msgr2=0x7ff79c0cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79affd640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c160e90 0x7ff79c0cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79affd640 1 -- 172.21.15.139:0/3294419077 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c1472a0 con 0x7ff79c06fa70 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79b7fe640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff79c06ab60 0x7ff79c15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79bfff640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c160e90 0x7ff79c0cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff79affd640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff79c06fa70 0x7ff79c146a50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff78c000c00 tx=0x7ff78c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff793fff640 1 -- 172.21.15.139:0/3294419077 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c037040 con 0x7ff79c06fa70 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff793fff640 1 -- 172.21.15.139:0/3294419077 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c042070 con 0x7ff79c06fa70 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff793fff640 1 -- 172.21.15.139:0/3294419077 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c030bd0 con 0x7ff79c06fa70 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/3294419077 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79c146f90 con 0x7ff79c06fa70 2024-04-25T09:54:25.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.368+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/3294419077 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79c0cc050 con 0x7ff79c06fa70 2024-04-25T09:54:25.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.369+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/3294419077 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79c074520 con 0x7ff79c06fa70 2024-04-25T09:54:25.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.370+0000 7ff793fff640 1 -- 172.21.15.139:0/3294419077 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff78c047460 con 0x7ff79c06fa70 2024-04-25T09:54:25.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.370+0000 7ff793fff640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff77c06b500 0x7ff77c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.370+0000 7ff793fff640 1 -- 172.21.15.139:0/3294419077 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff78c0aaf50 con 0x7ff79c06fa70 2024-04-25T09:54:25.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.370+0000 7ff79b7fe640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff77c06b500 0x7ff77c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.371+0000 7ff79b7fe640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff77c06b500 0x7ff77c06d9c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff784006fd0 tx=0x7ff784008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:25.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.374+0000 7ff793fff640 1 -- 172.21.15.139:0/3294419077 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff78c045360 con 0x7ff79c06fa70 2024-04-25T09:54:25.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.543+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/3294419077 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff79c0728b0 con 0x7ff77c06b500 2024-04-25T09:54:25.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.549+0000 7ff793fff640 1 -- 172.21.15.139:0/3294419077 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251906 (secure 0 0 0) 0x7ff79c0728b0 con 0x7ff77c06b500 2024-04-25T09:54:25.554 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:25.556 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:54:25.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.553+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/3294419077 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff77c06b500 msgr2=0x7ff77c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.553+0000 7ff7a1cf5640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff77c06b500 0x7ff77c06d9c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff784006fd0 tx=0x7ff784008040 comp rx=0 tx=0).stop 2024-04-25T09:54:25.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.553+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/3294419077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff79c06fa70 msgr2=0x7ff79c146a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.553+0000 7ff7a1cf5640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff79c06fa70 0x7ff79c146a50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff78c000c00 tx=0x7ff78c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.553+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/3294419077 shutdown_connections 2024-04-25T09:54:25.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.553+0000 7ff7a1cf5640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff79c06fa70 0x7ff79c146a50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.553+0000 7ff7a1cf5640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff77c06b500 0x7ff77c06d9c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.553+0000 7ff7a1cf5640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff79c160e90 0x7ff79c0cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.553+0000 7ff7a1cf5640 1 --2- 172.21.15.139:0/3294419077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff79c06ab60 0x7ff79c15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.553+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/3294419077 >> 172.21.15.139:0/3294419077 conn(0x7ff79c0b23f0 msgr2=0x7ff79c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:25.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.553+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/3294419077 shutdown_connections 2024-04-25T09:54:25.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.553+0000 7ff7a1cf5640 1 -- 172.21.15.139:0/3294419077 wait complete. 2024-04-25T09:54:25.566 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-04-25T09:54:24.981265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220600,"kb_used_data":4424,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754120,"statfs":{"total":773094113280,"available":772868218880,"internally_reserved":0,"allocated":4530176,"data_stored":2604582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.381024"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637887+0000","last_change":"2024-04-25T09:54:22.629232+0000","last_active":"2024-04-25T09:54:22.637887+0000","last_peered":"2024-04-25T09:54:22.637887+0000","last_clean":"2024-04-25T09:54:22.637887+0000","last_became_active":"2024-04-25T09:54:22.628954+0000","last_became_peered":"2024-04-25T09:54:22.628954+0000","last_unstale":"2024-04-25T09:54:22.637887+0000","last_undegraded":"2024-04-25T09:54:22.637887+0000","last_fullsized":"2024-04-25T09:54:22.637887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:05.809924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640899+0000","last_change":"2024-04-25T09:54:22.631588+0000","last_active":"2024-04-25T09:54:22.640899+0000","last_peered":"2024-04-25T09:54:22.640899+0000","last_clean":"2024-04-25T09:54:22.640899+0000","last_became_active":"2024-04-25T09:54:22.630768+0000","last_became_peered":"2024-04-25T09:54:22.630768+0000","last_unstale":"2024-04-25T09:54:22.640899+0000","last_undegraded":"2024-04-25T09:54:22.640899+0000","last_fullsized":"2024-04-25T09:54:22.640899+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:48.853059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641256+0000","last_change":"2024-04-25T09:54:22.635708+0000","last_active":"2024-04-25T09:54:22.641256+0000","last_peered":"2024-04-25T09:54:22.641256+0000","last_clean":"2024-04-25T09:54:22.641256+0000","last_became_active":"2024-04-25T09:54:22.635418+0000","last_became_peered":"2024-04-25T09:54:22.635418+0000","last_unstale":"2024-04-25T09:54:22.641256+0000","last_undegraded":"2024-04-25T09:54:22.641256+0000","last_fullsized":"2024-04-25T09:54:22.641256+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:43.969110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.629794+0000","last_change":"2024-04-25T09:54:22.620743+0000","last_active":"2024-04-25T09:54:22.629794+0000","last_peered":"2024-04-25T09:54:22.629794+0000","last_clean":"2024-04-25T09:54:22.629794+0000","last_became_active":"2024-04-25T09:54:22.620402+0000","last_became_peered":"2024-04-25T09:54:22.620402+0000","last_unstale":"2024-04-25T09:54:22.629794+0000","last_undegraded":"2024-04-25T09:54:22.629794+0000","last_fullsized":"2024-04-25T09:54:22.629794+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:05.510825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.624739+0000","last_change":"2024-04-25T09:54:22.619369+0000","last_active":"2024-04-25T09:54:22.624739+0000","last_peered":"2024-04-25T09:54:22.624739+0000","last_clean":"2024-04-25T09:54:22.624739+0000","last_became_active":"2024-04-25T09:54:22.619106+0000","last_became_peered":"2024-04-25T09:54:22.619106+0000","last_unstale":"2024-04-25T09:54:22.624739+0000","last_undegraded":"2024-04-25T09:54:22.624739+0000","last_fullsized":"2024-04-25T09:54:22.624739+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:47.831985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641067+0000","last_change":"2024-04-25T09:54:22.633793+0000","last_active":"2024-04-25T09:54:22.641067+0000","last_peered":"2024-04-25T09:54:22.641067+0000","last_clean":"2024-04-25T09:54:22.641067+0000","last_became_active":"2024-04-25T09:54:22.633520+0000","last_became_peered":"2024-04-25T09:54:22.633520+0000","last_unstale":"2024-04-25T09:54:22.641067+0000","last_undegraded":"2024-04-25T09:54:22.641067+0000","last_fullsized":"2024-04-25T09:54:22.641067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:55.935357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640090+0000","last_change":"2024-04-25T09:54:22.635734+0000","last_active":"2024-04-25T09:54:22.640090+0000","last_peered":"2024-04-25T09:54:22.640090+0000","last_clean":"2024-04-25T09:54:22.640090+0000","last_became_active":"2024-04-25T09:54:22.635018+0000","last_became_peered":"2024-04-25T09:54:22.635018+0000","last_unstale":"2024-04-25T09:54:22.640090+0000","last_undegraded":"2024-04-25T09:54:22.640090+0000","last_fullsized":"2024-04-25T09:54:22.640090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:05.809924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.636823+0000","last_change":"2024-04-25T09:54:22.628209+0000","last_active":"2024-04-25T09:54:22.636823+0000","last_peered":"2024-04-25T09:54:22.636823+0000","last_clean":"2024-04-25T09:54:22.636823+0000","last_became_active":"2024-04-25T09:54:22.627872+0000","last_became_peered":"2024-04-25T09:54:22.627872+0000","last_unstale":"2024-04-25T09:54:22.636823+0000","last_undegraded":"2024-04-25T09:54:22.636823+0000","last_fullsized":"2024-04-25T09:54:22.636823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:17.442893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.644781+0000","last_change":"2024-04-25T09:54:22.642228+0000","last_active":"2024-04-25T09:54:22.644781+0000","last_peered":"2024-04-25T09:54:22.644781+0000","last_clean":"2024-04-25T09:54:22.644781+0000","last_became_active":"2024-04-25T09:54:22.641976+0000","last_became_peered":"2024-04-25T09:54:22.641976+0000","last_unstale":"2024-04-25T09:54:22.644781+0000","last_undegraded":"2024-04-25T09:54:22.644781+0000","last_fullsized":"2024-04-25T09:54:22.644781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:15.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640118+0000","last_change":"2024-04-25T09:54:22.635786+0000","last_active":"2024-04-25T09:54:22.640118+0000","last_peered":"2024-04-25T09:54:22.640118+0000","last_clean":"2024-04-25T09:54:22.640118+0000","last_became_active":"2024-04-25T09:54:22.635459+0000","last_became_peered":"2024-04-25T09:54:22.635459+0000","last_unstale":"2024-04-25T09:54:22.640118+0000","last_undegraded":"2024-04-25T09:54:22.640118+0000","last_fullsized":"2024-04-25T09:54:22.640118+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:35.775552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.639859+0000","last_change":"2024-04-25T09:54:22.633086+0000","last_active":"2024-04-25T09:54:22.639859+0000","last_peered":"2024-04-25T09:54:22.639859+0000","last_clean":"2024-04-25T09:54:22.639859+0000","last_became_active":"2024-04-25T09:54:22.632818+0000","last_became_peered":"2024-04-25T09:54:22.632818+0000","last_unstale":"2024-04-25T09:54:22.639859+0000","last_undegraded":"2024-04-25T09:54:22.639859+0000","last_fullsized":"2024-04-25T09:54:22.639859+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:00.273606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640691+0000","last_change":"2024-04-25T09:54:22.631312+0000","last_active":"2024-04-25T09:54:22.640691+0000","last_peered":"2024-04-25T09:54:22.640691+0000","last_clean":"2024-04-25T09:54:22.640691+0000","last_became_active":"2024-04-25T09:54:22.630966+0000","last_became_peered":"2024-04-25T09:54:22.630966+0000","last_unstale":"2024-04-25T09:54:22.640691+0000","last_undegraded":"2024-04-25T09:54:22.640691+0000","last_fullsized":"2024-04-25T09:54:22.640691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:42.167481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645185+0000","last_change":"2024-04-25T09:54:22.643156+0000","last_active":"2024-04-25T09:54:22.645185+0000","last_peered":"2024-04-25T09:54:22.645185+0000","last_clean":"2024-04-25T09:54:22.645185+0000","last_became_active":"2024-04-25T09:54:22.642950+0000","last_became_peered":"2024-04-25T09:54:22.642950+0000","last_unstale":"2024-04-25T09:54:22.645185+0000","last_undegraded":"2024-04-25T09:54:22.645185+0000","last_fullsized":"2024-04-25T09:54:22.645185+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:32.763822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.636166+0000","last_change":"2024-04-25T09:54:22.629262+0000","last_active":"2024-04-25T09:54:22.636166+0000","last_peered":"2024-04-25T09:54:22.636166+0000","last_clean":"2024-04-25T09:54:22.636166+0000","last_became_active":"2024-04-25T09:54:22.628606+0000","last_became_peered":"2024-04-25T09:54:22.628606+0000","last_unstale":"2024-04-25T09:54:22.636166+0000","last_undegraded":"2024-04-25T09:54:22.636166+0000","last_fullsized":"2024-04-25T09:54:22.636166+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:44.646972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.639776+0000","last_change":"2024-04-25T09:54:22.631071+0000","last_active":"2024-04-25T09:54:22.639776+0000","last_peered":"2024-04-25T09:54:22.639776+0000","last_clean":"2024-04-25T09:54:22.639776+0000","last_became_active":"2024-04-25T09:54:22.630398+0000","last_became_peered":"2024-04-25T09:54:22.630398+0000","last_unstale":"2024-04-25T09:54:22.639776+0000","last_undegraded":"2024-04-25T09:54:22.639776+0000","last_fullsized":"2024-04-25T09:54:22.639776+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:28.420626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645111+0000","last_change":"2024-04-25T09:54:22.643249+0000","last_active":"2024-04-25T09:54:22.645111+0000","last_peered":"2024-04-25T09:54:22.645111+0000","last_clean":"2024-04-25T09:54:22.645111+0000","last_became_active":"2024-04-25T09:54:22.642950+0000","last_became_peered":"2024-04-25T09:54:22.642950+0000","last_unstale":"2024-04-25T09:54:22.645111+0000","last_undegraded":"2024-04-25T09:54:22.645111+0000","last_fullsized":"2024-04-25T09:54:22.645111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:28.420626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.644958+0000","last_change":"2024-04-25T09:54:22.643337+0000","last_active":"2024-04-25T09:54:22.644958+0000","last_peered":"2024-04-25T09:54:22.644958+0000","last_clean":"2024-04-25T09:54:22.644958+0000","last_became_active":"2024-04-25T09:54:22.643074+0000","last_became_peered":"2024-04-25T09:54:22.643074+0000","last_unstale":"2024-04-25T09:54:22.644958+0000","last_undegraded":"2024-04-25T09:54:22.644958+0000","last_fullsized":"2024-04-25T09:54:22.644958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:00.273606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637262+0000","last_change":"2024-04-25T09:54:22.632925+0000","last_active":"2024-04-25T09:54:22.637262+0000","last_peered":"2024-04-25T09:54:22.637262+0000","last_clean":"2024-04-25T09:54:22.637262+0000","last_became_active":"2024-04-25T09:54:22.632629+0000","last_became_peered":"2024-04-25T09:54:22.632629+0000","last_unstale":"2024-04-25T09:54:22.637262+0000","last_undegraded":"2024-04-25T09:54:22.637262+0000","last_fullsized":"2024-04-25T09:54:22.637262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:02.017763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.629691+0000","last_change":"2024-04-25T09:54:22.624361+0000","last_active":"2024-04-25T09:54:22.629691+0000","last_peered":"2024-04-25T09:54:22.629691+0000","last_clean":"2024-04-25T09:54:22.629691+0000","last_became_active":"2024-04-25T09:54:22.624099+0000","last_became_peered":"2024-04-25T09:54:22.624099+0000","last_unstale":"2024-04-25T09:54:22.629691+0000","last_undegraded":"2024-04-25T09:54:22.629691+0000","last_fullsized":"2024-04-25T09:54:22.629691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:17.096153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.628632+0000","last_change":"2024-04-25T09:54:22.626708+0000","last_active":"2024-04-25T09:54:22.628632+0000","last_peered":"2024-04-25T09:54:22.628632+0000","last_clean":"2024-04-25T09:54:22.628632+0000","last_became_active":"2024-04-25T09:54:22.626430+0000","last_became_peered":"2024-04-25T09:54:22.626430+0000","last_unstale":"2024-04-25T09:54:22.628632+0000","last_undegraded":"2024-04-25T09:54:22.628632+0000","last_fullsized":"2024-04-25T09:54:22.628632+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:57.631687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645042+0000","last_change":"2024-04-25T09:54:22.642125+0000","last_active":"2024-04-25T09:54:22.645042+0000","last_peered":"2024-04-25T09:54:22.645042+0000","last_clean":"2024-04-25T09:54:22.645042+0000","last_became_active":"2024-04-25T09:54:22.641836+0000","last_became_peered":"2024-04-25T09:54:22.641836+0000","last_unstale":"2024-04-25T09:54:22.645042+0000","last_undegraded":"2024-04-25T09:54:22.645042+0000","last_fullsized":"2024-04-25T09:54:22.645042+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:37.216863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.630337+0000","last_change":"2024-04-25T09:54:22.625201+0000","last_active":"2024-04-25T09:54:22.630337+0000","last_peered":"2024-04-25T09:54:22.630337+0000","last_clean":"2024-04-25T09:54:22.630337+0000","last_became_active":"2024-04-25T09:54:22.624920+0000","last_became_peered":"2024-04-25T09:54:22.624920+0000","last_unstale":"2024-04-25T09:54:22.630337+0000","last_undegraded":"2024-04-25T09:54:22.630337+0000","last_fullsized":"2024-04-25T09:54:22.630337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:55.935357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637636+0000","last_change":"2024-04-25T09:54:22.631584+0000","last_active":"2024-04-25T09:54:22.637636+0000","last_peered":"2024-04-25T09:54:22.637636+0000","last_clean":"2024-04-25T09:54:22.637636+0000","last_became_active":"2024-04-25T09:54:22.630401+0000","last_became_peered":"2024-04-25T09:54:22.630401+0000","last_unstale":"2024-04-25T09:54:22.637636+0000","last_undegraded":"2024-04-25T09:54:22.637636+0000","last_fullsized":"2024-04-25T09:54:22.637636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:44.049423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637609+0000","last_change":"2024-04-25T09:54:22.627838+0000","last_active":"2024-04-25T09:54:22.637609+0000","last_peered":"2024-04-25T09:54:22.637609+0000","last_clean":"2024-04-25T09:54:22.637609+0000","last_became_active":"2024-04-25T09:54:22.627536+0000","last_became_peered":"2024-04-25T09:54:22.627536+0000","last_unstale":"2024-04-25T09:54:22.637609+0000","last_undegraded":"2024-04-25T09:54:22.637609+0000","last_fullsized":"2024-04-25T09:54:22.637609+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:46.210520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.629881+0000","last_change":"2024-04-25T09:54:22.628389+0000","last_active":"2024-04-25T09:54:22.629881+0000","last_peered":"2024-04-25T09:54:22.629881+0000","last_clean":"2024-04-25T09:54:22.629881+0000","last_became_active":"2024-04-25T09:54:22.628106+0000","last_became_peered":"2024-04-25T09:54:22.628106+0000","last_unstale":"2024-04-25T09:54:22.629881+0000","last_undegraded":"2024-04-25T09:54:22.629881+0000","last_fullsized":"2024-04-25T09:54:22.629881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:17.450735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641000+0000","last_change":"2024-04-25T09:54:22.636714+0000","last_active":"2024-04-25T09:54:22.641000+0000","last_peered":"2024-04-25T09:54:22.641000+0000","last_clean":"2024-04-25T09:54:22.641000+0000","last_became_active":"2024-04-25T09:54:22.636450+0000","last_became_peered":"2024-04-25T09:54:22.636450+0000","last_unstale":"2024-04-25T09:54:22.641000+0000","last_undegraded":"2024-04-25T09:54:22.641000+0000","last_fullsized":"2024-04-25T09:54:22.641000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:43.969110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637171+0000","last_change":"2024-04-25T09:54:22.634667+0000","last_active":"2024-04-25T09:54:22.637171+0000","last_peered":"2024-04-25T09:54:22.637171+0000","last_clean":"2024-04-25T09:54:22.637171+0000","last_became_active":"2024-04-25T09:54:22.632588+0000","last_became_peered":"2024-04-25T09:54:22.632588+0000","last_unstale":"2024-04-25T09:54:22.637171+0000","last_undegraded":"2024-04-25T09:54:22.637171+0000","last_fullsized":"2024-04-25T09:54:22.637171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:42.167481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.639742+0000","last_change":"2024-04-25T09:54:22.634443+0000","last_active":"2024-04-25T09:54:22.639742+0000","last_peered":"2024-04-25T09:54:22.639742+0000","last_clean":"2024-04-25T09:54:22.639742+0000","last_became_active":"2024-04-25T09:54:22.634186+0000","last_became_peered":"2024-04-25T09:54:22.634186+0000","last_unstale":"2024-04-25T09:54:22.639742+0000","last_undegraded":"2024-04-25T09:54:22.639742+0000","last_fullsized":"2024-04-25T09:54:22.639742+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:17.442893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641389+0000","last_change":"2024-04-25T09:54:22.638963+0000","last_active":"2024-04-25T09:54:22.641389+0000","last_peered":"2024-04-25T09:54:22.641389+0000","last_clean":"2024-04-25T09:54:22.641389+0000","last_became_active":"2024-04-25T09:54:22.638673+0000","last_became_peered":"2024-04-25T09:54:22.638673+0000","last_unstale":"2024-04-25T09:54:22.641389+0000","last_undegraded":"2024-04-25T09:54:22.641389+0000","last_fullsized":"2024-04-25T09:54:22.641389+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:55.935357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.628683+0000","last_change":"2024-04-25T09:54:22.625924+0000","last_active":"2024-04-25T09:54:22.628683+0000","last_peered":"2024-04-25T09:54:22.628683+0000","last_clean":"2024-04-25T09:54:22.628683+0000","last_became_active":"2024-04-25T09:54:22.625579+0000","last_became_peered":"2024-04-25T09:54:22.625579+0000","last_unstale":"2024-04-25T09:54:22.628683+0000","last_undegraded":"2024-04-25T09:54:22.628683+0000","last_fullsized":"2024-04-25T09:54:22.628683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:23.664566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.643815+0000","last_change":"2024-04-25T09:54:22.639605+0000","last_active":"2024-04-25T09:54:22.643815+0000","last_peered":"2024-04-25T09:54:22.643815+0000","last_clean":"2024-04-25T09:54:22.643815+0000","last_became_active":"2024-04-25T09:54:22.638280+0000","last_became_peered":"2024-04-25T09:54:22.638280+0000","last_unstale":"2024-04-25T09:54:22.643815+0000","last_undegraded":"2024-04-25T09:54:22.643815+0000","last_fullsized":"2024-04-25T09:54:22.643815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:15.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.644542+0000","last_change":"2024-04-25T09:54:22.637447+0000","last_active":"2024-04-25T09:54:22.644542+0000","last_peered":"2024-04-25T09:54:22.644542+0000","last_clean":"2024-04-25T09:54:22.644542+0000","last_became_active":"2024-04-25T09:54:22.637181+0000","last_became_peered":"2024-04-25T09:54:22.637181+0000","last_unstale":"2024-04-25T09:54:22.644542+0000","last_undegraded":"2024-04-25T09:54:22.644542+0000","last_fullsized":"2024-04-25T09:54:22.644542+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:38.117240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645029+0000","last_change":"2024-04-25T09:54:22.643465+0000","last_active":"2024-04-25T09:54:22.645029+0000","last_peered":"2024-04-25T09:54:22.645029+0000","last_clean":"2024-04-25T09:54:22.645029+0000","last_became_active":"2024-04-25T09:54:22.643181+0000","last_became_peered":"2024-04-25T09:54:22.643181+0000","last_unstale":"2024-04-25T09:54:22.645029+0000","last_undegraded":"2024-04-25T09:54:22.645029+0000","last_fullsized":"2024-04-25T09:54:22.645029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:17.442893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.638197+0000","last_change":"2024-04-25T09:54:21.621031+0000","last_active":"2024-04-25T09:54:22.638197+0000","last_peered":"2024-04-25T09:54:22.638197+0000","last_clean":"2024-04-25T09:54:22.638197+0000","last_became_active":"2024-04-25T09:54:21.620741+0000","last_became_peered":"2024-04-25T09:54:21.620741+0000","last_unstale":"2024-04-25T09:54:22.638197+0000","last_undegraded":"2024-04-25T09:54:22.638197+0000","last_fullsized":"2024-04-25T09:54:22.638197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:30.927191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640968+0000","last_change":"2024-04-25T09:54:22.636146+0000","last_active":"2024-04-25T09:54:22.640968+0000","last_peered":"2024-04-25T09:54:22.640968+0000","last_clean":"2024-04-25T09:54:22.640968+0000","last_became_active":"2024-04-25T09:54:22.635843+0000","last_became_peered":"2024-04-25T09:54:22.635843+0000","last_unstale":"2024-04-25T09:54:22.640968+0000","last_undegraded":"2024-04-25T09:54:22.640968+0000","last_fullsized":"2024-04-25T09:54:22.640968+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:44.049423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.634679+0000","last_change":"2024-04-25T09:54:21.626269+0000","last_active":"2024-04-25T09:54:22.634679+0000","last_peered":"2024-04-25T09:54:22.634679+0000","last_clean":"2024-04-25T09:54:22.634679+0000","last_became_active":"2024-04-25T09:54:21.625976+0000","last_became_peered":"2024-04-25T09:54:21.625976+0000","last_unstale":"2024-04-25T09:54:22.634679+0000","last_undegraded":"2024-04-25T09:54:22.634679+0000","last_fullsized":"2024-04-25T09:54:22.634679+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:16.447999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637486+0000","last_change":"2024-04-25T09:54:22.632816+0000","last_active":"2024-04-25T09:54:22.637486+0000","last_peered":"2024-04-25T09:54:22.637486+0000","last_clean":"2024-04-25T09:54:22.637486+0000","last_became_active":"2024-04-25T09:54:22.631163+0000","last_became_peered":"2024-04-25T09:54:22.631163+0000","last_unstale":"2024-04-25T09:54:22.637486+0000","last_undegraded":"2024-04-25T09:54:22.637486+0000","last_fullsized":"2024-04-25T09:54:22.637486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:43.969110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637186+0000","last_change":"2024-04-25T09:54:21.627063+0000","last_active":"2024-04-25T09:54:22.637186+0000","last_peered":"2024-04-25T09:54:22.637186+0000","last_clean":"2024-04-25T09:54:22.637186+0000","last_became_active":"2024-04-25T09:54:21.626644+0000","last_became_peered":"2024-04-25T09:54:21.626644+0000","last_unstale":"2024-04-25T09:54:22.637186+0000","last_undegraded":"2024-04-25T09:54:22.637186+0000","last_fullsized":"2024-04-25T09:54:22.637186+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:01.015027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637866+0000","last_change":"2024-04-25T09:54:22.626719+0000","last_active":"2024-04-25T09:54:22.637866+0000","last_peered":"2024-04-25T09:54:22.637866+0000","last_clean":"2024-04-25T09:54:22.637866+0000","last_became_active":"2024-04-25T09:54:22.626470+0000","last_became_peered":"2024-04-25T09:54:22.626470+0000","last_unstale":"2024-04-25T09:54:22.637866+0000","last_undegraded":"2024-04-25T09:54:22.637866+0000","last_fullsized":"2024-04-25T09:54:22.637866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:31.082648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641512+0000","last_change":"2024-04-25T09:54:21.622728+0000","last_active":"2024-04-25T09:54:22.641512+0000","last_peered":"2024-04-25T09:54:22.641512+0000","last_clean":"2024-04-25T09:54:22.641512+0000","last_became_active":"2024-04-25T09:54:21.622435+0000","last_became_peered":"2024-04-25T09:54:21.622435+0000","last_unstale":"2024-04-25T09:54:22.641512+0000","last_undegraded":"2024-04-25T09:54:22.641512+0000","last_fullsized":"2024-04-25T09:54:22.641512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:22.661830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.644307+0000","last_change":"2024-04-25T09:54:22.638528+0000","last_active":"2024-04-25T09:54:22.644307+0000","last_peered":"2024-04-25T09:54:22.644307+0000","last_clean":"2024-04-25T09:54:22.644307+0000","last_became_active":"2024-04-25T09:54:22.638221+0000","last_became_peered":"2024-04-25T09:54:22.638221+0000","last_unstale":"2024-04-25T09:54:22.644307+0000","last_undegraded":"2024-04-25T09:54:22.644307+0000","last_fullsized":"2024-04-25T09:54:22.644307+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:38.117240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641749+0000","last_change":"2024-04-25T09:54:21.619668+0000","last_active":"2024-04-25T09:54:22.641749+0000","last_peered":"2024-04-25T09:54:22.641749+0000","last_clean":"2024-04-25T09:54:22.641749+0000","last_became_active":"2024-04-25T09:54:21.619404+0000","last_became_peered":"2024-04-25T09:54:21.619404+0000","last_unstale":"2024-04-25T09:54:22.641749+0000","last_undegraded":"2024-04-25T09:54:22.641749+0000","last_fullsized":"2024-04-25T09:54:22.641749+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:42.966374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.644995+0000","last_change":"2024-04-25T09:54:22.642120+0000","last_active":"2024-04-25T09:54:22.644995+0000","last_peered":"2024-04-25T09:54:22.644995+0000","last_clean":"2024-04-25T09:54:22.644995+0000","last_became_active":"2024-04-25T09:54:22.641778+0000","last_became_peered":"2024-04-25T09:54:22.641778+0000","last_unstale":"2024-04-25T09:54:22.644995+0000","last_undegraded":"2024-04-25T09:54:22.644995+0000","last_fullsized":"2024-04-25T09:54:22.644995+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:32.041828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640059+0000","last_change":"2024-04-25T09:54:21.630270+0000","last_active":"2024-04-25T09:54:22.640059+0000","last_peered":"2024-04-25T09:54:22.640059+0000","last_clean":"2024-04-25T09:54:22.640059+0000","last_became_active":"2024-04-25T09:54:21.629920+0000","last_became_peered":"2024-04-25T09:54:21.629920+0000","last_unstale":"2024-04-25T09:54:22.640059+0000","last_undegraded":"2024-04-25T09:54:22.640059+0000","last_fullsized":"2024-04-25T09:54:22.640059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:54.932621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.638796+0000","last_change":"2024-04-25T09:54:22.634359+0000","last_active":"2024-04-25T09:54:22.638796+0000","last_peered":"2024-04-25T09:54:22.638796+0000","last_clean":"2024-04-25T09:54:22.638796+0000","last_became_active":"2024-04-25T09:54:22.631933+0000","last_became_peered":"2024-04-25T09:54:22.631933+0000","last_unstale":"2024-04-25T09:54:22.638796+0000","last_undegraded":"2024-04-25T09:54:22.638796+0000","last_fullsized":"2024-04-25T09:54:22.638796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:24.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.617921+0000","last_change":"2024-04-25T09:54:21.627673+0000","last_active":"2024-04-25T09:54:22.617921+0000","last_peered":"2024-04-25T09:54:22.617921+0000","last_clean":"2024-04-25T09:54:22.617921+0000","last_became_active":"2024-04-25T09:54:21.627431+0000","last_became_peered":"2024-04-25T09:54:21.627431+0000","last_unstale":"2024-04-25T09:54:22.617921+0000","last_undegraded":"2024-04-25T09:54:22.617921+0000","last_fullsized":"2024-04-25T09:54:22.617921+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:22.661830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640953+0000","last_change":"2024-04-25T09:54:22.633487+0000","last_active":"2024-04-25T09:54:22.640953+0000","last_peered":"2024-04-25T09:54:22.640953+0000","last_clean":"2024-04-25T09:54:22.640953+0000","last_became_active":"2024-04-25T09:54:22.633164+0000","last_became_peered":"2024-04-25T09:54:22.633164+0000","last_unstale":"2024-04-25T09:54:22.640953+0000","last_undegraded":"2024-04-25T09:54:22.640953+0000","last_fullsized":"2024-04-25T09:54:22.640953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:32.763822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.615432+0000","last_change":"2024-04-25T09:54:21.625160+0000","last_active":"2024-04-25T09:54:22.615432+0000","last_peered":"2024-04-25T09:54:22.615432+0000","last_clean":"2024-04-25T09:54:22.615432+0000","last_became_active":"2024-04-25T09:54:21.624854+0000","last_became_peered":"2024-04-25T09:54:21.624854+0000","last_unstale":"2024-04-25T09:54:22.615432+0000","last_undegraded":"2024-04-25T09:54:22.615432+0000","last_fullsized":"2024-04-25T09:54:22.615432+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:56.628951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.636998+0000","last_change":"2024-04-25T09:54:22.625054+0000","last_active":"2024-04-25T09:54:22.636998+0000","last_peered":"2024-04-25T09:54:22.636998+0000","last_clean":"2024-04-25T09:54:22.636998+0000","last_became_active":"2024-04-25T09:54:22.624736+0000","last_became_peered":"2024-04-25T09:54:22.624736+0000","last_unstale":"2024-04-25T09:54:22.636998+0000","last_undegraded":"2024-04-25T09:54:22.636998+0000","last_fullsized":"2024-04-25T09:54:22.636998+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:02.344151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.613004+0000","last_change":"2024-04-25T09:54:21.622690+0000","last_active":"2024-04-25T09:54:22.613004+0000","last_peered":"2024-04-25T09:54:22.613004+0000","last_clean":"2024-04-25T09:54:22.613004+0000","last_became_active":"2024-04-25T09:54:21.622427+0000","last_became_peered":"2024-04-25T09:54:21.622427+0000","last_unstale":"2024-04-25T09:54:22.613004+0000","last_undegraded":"2024-04-25T09:54:22.613004+0000","last_fullsized":"2024-04-25T09:54:22.613004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:23.985952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.639978+0000","last_change":"2024-04-25T09:54:22.636609+0000","last_active":"2024-04-25T09:54:22.639978+0000","last_peered":"2024-04-25T09:54:22.639978+0000","last_clean":"2024-04-25T09:54:22.639978+0000","last_became_active":"2024-04-25T09:54:22.635042+0000","last_became_peered":"2024-04-25T09:54:22.635042+0000","last_unstale":"2024-04-25T09:54:22.639978+0000","last_undegraded":"2024-04-25T09:54:22.639978+0000","last_fullsized":"2024-04-25T09:54:22.639978+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:24.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.621536+0000","last_change":"2024-04-25T09:54:21.623071+0000","last_active":"2024-04-25T09:54:22.621536+0000","last_peered":"2024-04-25T09:54:22.621536+0000","last_clean":"2024-04-25T09:54:22.621536+0000","last_became_active":"2024-04-25T09:54:21.622746+0000","last_became_peered":"2024-04-25T09:54:21.622746+0000","last_unstale":"2024-04-25T09:54:22.621536+0000","last_undegraded":"2024-04-25T09:54:22.621536+0000","last_fullsized":"2024-04-25T09:54:22.621536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:37.114504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640927+0000","last_change":"2024-04-25T09:54:22.635160+0000","last_active":"2024-04-25T09:54:22.640927+0000","last_peered":"2024-04-25T09:54:22.640927+0000","last_clean":"2024-04-25T09:54:22.640927+0000","last_became_active":"2024-04-25T09:54:22.634866+0000","last_became_peered":"2024-04-25T09:54:22.634866+0000","last_unstale":"2024-04-25T09:54:22.640927+0000","last_undegraded":"2024-04-25T09:54:22.640927+0000","last_fullsized":"2024-04-25T09:54:22.640927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:44.646972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.642917+0000","last_change":"2024-04-25T09:54:21.628294+0000","last_active":"2024-04-25T09:54:22.642917+0000","last_peered":"2024-04-25T09:54:22.642917+0000","last_clean":"2024-04-25T09:54:22.642917+0000","last_became_active":"2024-04-25T09:54:21.628022+0000","last_became_peered":"2024-04-25T09:54:21.628022+0000","last_unstale":"2024-04-25T09:54:22.642917+0000","last_undegraded":"2024-04-25T09:54:22.642917+0000","last_fullsized":"2024-04-25T09:54:22.642917+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:23.985952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.628258+0000","last_change":"2024-04-25T09:54:22.621387+0000","last_active":"2024-04-25T09:54:22.628258+0000","last_peered":"2024-04-25T09:54:22.628258+0000","last_clean":"2024-04-25T09:54:22.628258+0000","last_became_active":"2024-04-25T09:54:22.621117+0000","last_became_peered":"2024-04-25T09:54:22.621117+0000","last_unstale":"2024-04-25T09:54:22.628258+0000","last_undegraded":"2024-04-25T09:54:22.628258+0000","last_fullsized":"2024-04-25T09:54:22.628258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:42.167481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645238+0000","last_change":"2024-04-25T09:54:21.626625+0000","last_active":"2024-04-25T09:54:22.645238+0000","last_peered":"2024-04-25T09:54:22.645238+0000","last_clean":"2024-04-25T09:54:22.645238+0000","last_became_active":"2024-04-25T09:54:21.626367+0000","last_became_peered":"2024-04-25T09:54:21.626367+0000","last_unstale":"2024-04-25T09:54:22.645238+0000","last_undegraded":"2024-04-25T09:54:22.645238+0000","last_fullsized":"2024-04-25T09:54:22.645238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:14.611990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641016+0000","last_change":"2024-04-25T09:54:22.636106+0000","last_active":"2024-04-25T09:54:22.641016+0000","last_peered":"2024-04-25T09:54:22.641016+0000","last_clean":"2024-04-25T09:54:22.641016+0000","last_became_active":"2024-04-25T09:54:22.635804+0000","last_became_peered":"2024-04-25T09:54:22.635804+0000","last_unstale":"2024-04-25T09:54:22.641016+0000","last_undegraded":"2024-04-25T09:54:22.641016+0000","last_fullsized":"2024-04-25T09:54:22.641016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:24.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.644564+0000","last_change":"2024-04-25T09:54:21.620904+0000","last_active":"2024-04-25T09:54:22.644564+0000","last_peered":"2024-04-25T09:54:22.644564+0000","last_clean":"2024-04-25T09:54:22.644564+0000","last_became_active":"2024-04-25T09:54:21.620628+0000","last_became_peered":"2024-04-25T09:54:21.620628+0000","last_unstale":"2024-04-25T09:54:22.644564+0000","last_undegraded":"2024-04-25T09:54:22.644564+0000","last_fullsized":"2024-04-25T09:54:22.644564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:59.343752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.636783+0000","last_change":"2024-04-25T09:54:22.629277+0000","last_active":"2024-04-25T09:54:22.636783+0000","last_peered":"2024-04-25T09:54:22.636783+0000","last_clean":"2024-04-25T09:54:22.636783+0000","last_became_active":"2024-04-25T09:54:22.629002+0000","last_became_peered":"2024-04-25T09:54:22.629002+0000","last_unstale":"2024-04-25T09:54:22.636783+0000","last_undegraded":"2024-04-25T09:54:22.636783+0000","last_fullsized":"2024-04-25T09:54:22.636783+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:35.775552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.618448+0000","last_change":"2024-04-25T09:54:21.628544+0000","last_active":"2024-04-25T09:54:22.618448+0000","last_peered":"2024-04-25T09:54:22.618448+0000","last_clean":"2024-04-25T09:54:22.618448+0000","last_became_active":"2024-04-25T09:54:21.628273+0000","last_became_peered":"2024-04-25T09:54:21.628273+0000","last_unstale":"2024-04-25T09:54:22.618448+0000","last_undegraded":"2024-04-25T09:54:22.618448+0000","last_fullsized":"2024-04-25T09:54:22.618448+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:56.628951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.643887+0000","last_change":"2024-04-25T09:54:22.639839+0000","last_active":"2024-04-25T09:54:22.643887+0000","last_peered":"2024-04-25T09:54:22.643887+0000","last_clean":"2024-04-25T09:54:22.643887+0000","last_became_active":"2024-04-25T09:54:22.638248+0000","last_became_peered":"2024-04-25T09:54:22.638248+0000","last_unstale":"2024-04-25T09:54:22.643887+0000","last_undegraded":"2024-04-25T09:54:22.643887+0000","last_fullsized":"2024-04-25T09:54:22.643887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:00.346488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.619958+0000","last_change":"2024-04-25T09:54:21.621771+0000","last_active":"2024-04-25T09:54:22.619958+0000","last_peered":"2024-04-25T09:54:22.619958+0000","last_clean":"2024-04-25T09:54:22.619958+0000","last_became_active":"2024-04-25T09:54:21.621433+0000","last_became_peered":"2024-04-25T09:54:21.621433+0000","last_unstale":"2024-04-25T09:54:22.619958+0000","last_undegraded":"2024-04-25T09:54:22.619958+0000","last_fullsized":"2024-04-25T09:54:22.619958+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:19.872298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637881+0000","last_change":"2024-04-25T09:54:22.628174+0000","last_active":"2024-04-25T09:54:22.637881+0000","last_peered":"2024-04-25T09:54:22.637881+0000","last_clean":"2024-04-25T09:54:22.637881+0000","last_became_active":"2024-04-25T09:54:22.627900+0000","last_became_peered":"2024-04-25T09:54:22.627900+0000","last_unstale":"2024-04-25T09:54:22.637881+0000","last_undegraded":"2024-04-25T09:54:22.637881+0000","last_fullsized":"2024-04-25T09:54:22.637881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:00.273606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645556+0000","last_change":"2024-04-25T09:54:21.629589+0000","last_active":"2024-04-25T09:54:22.645556+0000","last_peered":"2024-04-25T09:54:22.645556+0000","last_clean":"2024-04-25T09:54:22.645556+0000","last_became_active":"2024-04-25T09:54:21.629290+0000","last_became_peered":"2024-04-25T09:54:21.629290+0000","last_unstale":"2024-04-25T09:54:22.645556+0000","last_undegraded":"2024-04-25T09:54:22.645556+0000","last_fullsized":"2024-04-25T09:54:22.645556+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:01.015027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.636854+0000","last_change":"2024-04-25T09:54:22.627570+0000","last_active":"2024-04-25T09:54:22.636854+0000","last_peered":"2024-04-25T09:54:22.636854+0000","last_clean":"2024-04-25T09:54:22.636854+0000","last_became_active":"2024-04-25T09:54:22.627313+0000","last_became_peered":"2024-04-25T09:54:22.627313+0000","last_unstale":"2024-04-25T09:54:22.636854+0000","last_undegraded":"2024-04-25T09:54:22.636854+0000","last_fullsized":"2024-04-25T09:54:22.636854+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:28.420626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641104+0000","last_change":"2024-04-25T09:54:21.628865+0000","last_active":"2024-04-25T09:54:22.641104+0000","last_peered":"2024-04-25T09:54:22.641104+0000","last_clean":"2024-04-25T09:54:22.641104+0000","last_became_active":"2024-04-25T09:54:21.628530+0000","last_became_peered":"2024-04-25T09:54:21.628530+0000","last_unstale":"2024-04-25T09:54:22.641104+0000","last_undegraded":"2024-04-25T09:54:22.641104+0000","last_fullsized":"2024-04-25T09:54:22.641104+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:14.611990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645288+0000","last_change":"2024-04-25T09:54:22.643230+0000","last_active":"2024-04-25T09:54:22.645288+0000","last_peered":"2024-04-25T09:54:22.645288+0000","last_clean":"2024-04-25T09:54:22.645288+0000","last_became_active":"2024-04-25T09:54:22.642966+0000","last_became_peered":"2024-04-25T09:54:22.642966+0000","last_unstale":"2024-04-25T09:54:22.645288+0000","last_undegraded":"2024-04-25T09:54:22.645288+0000","last_fullsized":"2024-04-25T09:54:22.645288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:27.537919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641441+0000","last_change":"2024-04-25T09:54:21.632060+0000","last_active":"2024-04-25T09:54:22.641441+0000","last_peered":"2024-04-25T09:54:22.641441+0000","last_clean":"2024-04-25T09:54:22.641441+0000","last_became_active":"2024-04-25T09:54:21.631637+0000","last_became_peered":"2024-04-25T09:54:21.631637+0000","last_unstale":"2024-04-25T09:54:22.641441+0000","last_undegraded":"2024-04-25T09:54:22.641441+0000","last_fullsized":"2024-04-25T09:54:22.641441+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:16.447999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.636230+0000","last_change":"2024-04-25T09:54:22.630550+0000","last_active":"2024-04-25T09:54:22.636230+0000","last_peered":"2024-04-25T09:54:22.636230+0000","last_clean":"2024-04-25T09:54:22.636230+0000","last_became_active":"2024-04-25T09:54:22.630269+0000","last_became_peered":"2024-04-25T09:54:22.630269+0000","last_unstale":"2024-04-25T09:54:22.636230+0000","last_undegraded":"2024-04-25T09:54:22.636230+0000","last_fullsized":"2024-04-25T09:54:22.636230+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:44.049423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640868+0000","last_change":"2024-04-25T09:54:21.625832+0000","last_active":"2024-04-25T09:54:22.640868+0000","last_peered":"2024-04-25T09:54:22.640868+0000","last_clean":"2024-04-25T09:54:22.640868+0000","last_became_active":"2024-04-25T09:54:21.625556+0000","last_became_peered":"2024-04-25T09:54:21.625556+0000","last_unstale":"2024-04-25T09:54:22.640868+0000","last_undegraded":"2024-04-25T09:54:22.640868+0000","last_fullsized":"2024-04-25T09:54:22.640868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:52.007492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.629650+0000","last_change":"2024-04-25T09:54:22.619869+0000","last_active":"2024-04-25T09:54:22.629650+0000","last_peered":"2024-04-25T09:54:22.629650+0000","last_clean":"2024-04-25T09:54:22.629650+0000","last_became_active":"2024-04-25T09:54:22.619573+0000","last_became_peered":"2024-04-25T09:54:22.619573+0000","last_unstale":"2024-04-25T09:54:22.629650+0000","last_undegraded":"2024-04-25T09:54:22.629650+0000","last_fullsized":"2024-04-25T09:54:22.629650+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:23.041393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.618986+0000","last_change":"2024-04-25T09:54:21.628678+0000","last_active":"2024-04-25T09:54:22.618986+0000","last_peered":"2024-04-25T09:54:22.618986+0000","last_clean":"2024-04-25T09:54:22.618986+0000","last_became_active":"2024-04-25T09:54:21.628420+0000","last_became_peered":"2024-04-25T09:54:21.628420+0000","last_unstale":"2024-04-25T09:54:22.618986+0000","last_undegraded":"2024-04-25T09:54:22.618986+0000","last_fullsized":"2024-04-25T09:54:22.618986+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:16.447999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.643131+0000","last_change":"2024-04-25T09:54:22.639329+0000","last_active":"2024-04-25T09:54:22.643131+0000","last_peered":"2024-04-25T09:54:22.643131+0000","last_clean":"2024-04-25T09:54:22.643131+0000","last_became_active":"2024-04-25T09:54:22.639067+0000","last_became_peered":"2024-04-25T09:54:22.639067+0000","last_unstale":"2024-04-25T09:54:22.643131+0000","last_undegraded":"2024-04-25T09:54:22.643131+0000","last_fullsized":"2024-04-25T09:54:22.643131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:38.117240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641388+0000","last_change":"2024-04-25T09:54:21.619666+0000","last_active":"2024-04-25T09:54:22.641388+0000","last_peered":"2024-04-25T09:54:22.641388+0000","last_clean":"2024-04-25T09:54:22.641388+0000","last_became_active":"2024-04-25T09:54:21.619399+0000","last_became_peered":"2024-04-25T09:54:21.619399+0000","last_unstale":"2024-04-25T09:54:22.641388+0000","last_undegraded":"2024-04-25T09:54:22.641388+0000","last_fullsized":"2024-04-25T09:54:22.641388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:06.957591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640967+0000","last_change":"2024-04-25T09:54:22.635242+0000","last_active":"2024-04-25T09:54:22.640967+0000","last_peered":"2024-04-25T09:54:22.640967+0000","last_clean":"2024-04-25T09:54:22.640967+0000","last_became_active":"2024-04-25T09:54:22.634885+0000","last_became_peered":"2024-04-25T09:54:22.634885+0000","last_unstale":"2024-04-25T09:54:22.640967+0000","last_undegraded":"2024-04-25T09:54:22.640967+0000","last_fullsized":"2024-04-25T09:54:22.640967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:44.049423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.612299+0000","last_change":"2024-04-25T09:54:21.625380+0000","last_active":"2024-04-25T09:54:22.612299+0000","last_peered":"2024-04-25T09:54:22.612299+0000","last_clean":"2024-04-25T09:54:22.612299+0000","last_became_active":"2024-04-25T09:54:21.625128+0000","last_became_peered":"2024-04-25T09:54:21.625128+0000","last_unstale":"2024-04-25T09:54:22.612299+0000","last_undegraded":"2024-04-25T09:54:22.612299+0000","last_fullsized":"2024-04-25T09:54:22.612299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:20.524444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.639251+0000","last_change":"2024-04-25T09:54:22.630647+0000","last_active":"2024-04-25T09:54:22.639251+0000","last_peered":"2024-04-25T09:54:22.639251+0000","last_clean":"2024-04-25T09:54:22.639251+0000","last_became_active":"2024-04-25T09:54:22.630329+0000","last_became_peered":"2024-04-25T09:54:22.630329+0000","last_unstale":"2024-04-25T09:54:22.639251+0000","last_undegraded":"2024-04-25T09:54:22.639251+0000","last_fullsized":"2024-04-25T09:54:22.639251+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:31.082648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.624865+0000","last_change":"2024-04-25T09:54:21.630293+0000","last_active":"2024-04-25T09:54:22.624865+0000","last_peered":"2024-04-25T09:54:22.624865+0000","last_clean":"2024-04-25T09:54:22.624865+0000","last_became_active":"2024-04-25T09:54:21.630005+0000","last_became_peered":"2024-04-25T09:54:21.630005+0000","last_unstale":"2024-04-25T09:54:22.624865+0000","last_undegraded":"2024-04-25T09:54:22.624865+0000","last_fullsized":"2024-04-25T09:54:22.624865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:41.164745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640930+0000","last_change":"2024-04-25T09:54:22.632669+0000","last_active":"2024-04-25T09:54:22.640930+0000","last_peered":"2024-04-25T09:54:22.640930+0000","last_clean":"2024-04-25T09:54:22.640930+0000","last_became_active":"2024-04-25T09:54:22.632254+0000","last_became_peered":"2024-04-25T09:54:22.632254+0000","last_unstale":"2024-04-25T09:54:22.640930+0000","last_undegraded":"2024-04-25T09:54:22.640930+0000","last_fullsized":"2024-04-25T09:54:22.640930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:37.216863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.612533+0000","last_change":"2024-04-25T09:54:21.619855+0000","last_active":"2024-04-25T09:54:22.612533+0000","last_peered":"2024-04-25T09:54:22.612533+0000","last_clean":"2024-04-25T09:54:22.612533+0000","last_became_active":"2024-04-25T09:54:21.619608+0000","last_became_peered":"2024-04-25T09:54:21.619608+0000","last_unstale":"2024-04-25T09:54:22.612533+0000","last_undegraded":"2024-04-25T09:54:22.612533+0000","last_fullsized":"2024-04-25T09:54:22.612533+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:57.986604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640950+0000","last_change":"2024-04-25T09:54:22.636809+0000","last_active":"2024-04-25T09:54:22.640950+0000","last_peered":"2024-04-25T09:54:22.640950+0000","last_clean":"2024-04-25T09:54:22.640950+0000","last_became_active":"2024-04-25T09:54:22.635858+0000","last_became_peered":"2024-04-25T09:54:22.635858+0000","last_unstale":"2024-04-25T09:54:22.640950+0000","last_undegraded":"2024-04-25T09:54:22.640950+0000","last_fullsized":"2024-04-25T09:54:22.640950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:24.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.642788+0000","last_change":"2024-04-25T09:54:21.624009+0000","last_active":"2024-04-25T09:54:22.642788+0000","last_peered":"2024-04-25T09:54:22.642788+0000","last_clean":"2024-04-25T09:54:22.642788+0000","last_became_active":"2024-04-25T09:54:21.623706+0000","last_became_peered":"2024-04-25T09:54:21.623706+0000","last_unstale":"2024-04-25T09:54:22.642788+0000","last_undegraded":"2024-04-25T09:54:22.642788+0000","last_fullsized":"2024-04-25T09:54:22.642788+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:59.343752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.643988+0000","last_change":"2024-04-25T09:54:22.640133+0000","last_active":"2024-04-25T09:54:22.643988+0000","last_peered":"2024-04-25T09:54:22.643988+0000","last_clean":"2024-04-25T09:54:22.643988+0000","last_became_active":"2024-04-25T09:54:22.639770+0000","last_became_peered":"2024-04-25T09:54:22.639770+0000","last_unstale":"2024-04-25T09:54:22.643988+0000","last_undegraded":"2024-04-25T09:54:22.643988+0000","last_fullsized":"2024-04-25T09:54:22.643988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:31.082648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.612651+0000","last_change":"2024-04-25T09:54:21.629092+0000","last_active":"2024-04-25T09:54:22.612651+0000","last_peered":"2024-04-25T09:54:22.612651+0000","last_clean":"2024-04-25T09:54:22.612651+0000","last_became_active":"2024-04-25T09:54:21.628754+0000","last_became_peered":"2024-04-25T09:54:21.628754+0000","last_unstale":"2024-04-25T09:54:22.612651+0000","last_undegraded":"2024-04-25T09:54:22.612651+0000","last_fullsized":"2024-04-25T09:54:22.612651+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:56.099271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640925+0000","last_change":"2024-04-25T09:54:22.631681+0000","last_active":"2024-04-25T09:54:22.640925+0000","last_peered":"2024-04-25T09:54:22.640925+0000","last_clean":"2024-04-25T09:54:22.640925+0000","last_became_active":"2024-04-25T09:54:22.631434+0000","last_became_peered":"2024-04-25T09:54:22.631434+0000","last_unstale":"2024-04-25T09:54:22.640925+0000","last_undegraded":"2024-04-25T09:54:22.640925+0000","last_fullsized":"2024-04-25T09:54:22.640925+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:32.041828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.618902+0000","last_change":"2024-04-25T09:54:21.630777+0000","last_active":"2024-04-25T09:54:22.618902+0000","last_peered":"2024-04-25T09:54:22.618902+0000","last_clean":"2024-04-25T09:54:22.618902+0000","last_became_active":"2024-04-25T09:54:21.630475+0000","last_became_peered":"2024-04-25T09:54:21.630475+0000","last_unstale":"2024-04-25T09:54:22.618902+0000","last_undegraded":"2024-04-25T09:54:22.618902+0000","last_fullsized":"2024-04-25T09:54:22.618902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:56.628951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640987+0000","last_change":"2024-04-25T09:54:22.633510+0000","last_active":"2024-04-25T09:54:22.640987+0000","last_peered":"2024-04-25T09:54:22.640987+0000","last_clean":"2024-04-25T09:54:22.640987+0000","last_became_active":"2024-04-25T09:54:22.633100+0000","last_became_peered":"2024-04-25T09:54:22.633100+0000","last_unstale":"2024-04-25T09:54:22.640987+0000","last_undegraded":"2024-04-25T09:54:22.640987+0000","last_fullsized":"2024-04-25T09:54:22.640987+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:55.935357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.620435+0000","last_change":"2024-04-25T09:54:21.623769+0000","last_active":"2024-04-25T09:54:22.620435+0000","last_peered":"2024-04-25T09:54:22.620435+0000","last_clean":"2024-04-25T09:54:22.620435+0000","last_became_active":"2024-04-25T09:54:21.623500+0000","last_became_peered":"2024-04-25T09:54:21.623500+0000","last_unstale":"2024-04-25T09:54:22.620435+0000","last_undegraded":"2024-04-25T09:54:22.620435+0000","last_fullsized":"2024-04-25T09:54:22.620435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:01.015027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641098+0000","last_change":"2024-04-25T09:54:22.631121+0000","last_active":"2024-04-25T09:54:22.641098+0000","last_peered":"2024-04-25T09:54:22.641098+0000","last_clean":"2024-04-25T09:54:22.641098+0000","last_became_active":"2024-04-25T09:54:22.630838+0000","last_became_peered":"2024-04-25T09:54:22.630838+0000","last_unstale":"2024-04-25T09:54:22.641098+0000","last_undegraded":"2024-04-25T09:54:22.641098+0000","last_fullsized":"2024-04-25T09:54:22.641098+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:42.167481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.618668+0000","last_change":"2024-04-25T09:54:21.623588+0000","last_active":"2024-04-25T09:54:22.618668+0000","last_peered":"2024-04-25T09:54:22.618668+0000","last_clean":"2024-04-25T09:54:22.618668+0000","last_became_active":"2024-04-25T09:54:21.623245+0000","last_became_peered":"2024-04-25T09:54:21.623245+0000","last_unstale":"2024-04-25T09:54:22.618668+0000","last_undegraded":"2024-04-25T09:54:22.618668+0000","last_fullsized":"2024-04-25T09:54:22.618668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:56.285717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.628318+0000","last_change":"2024-04-25T09:54:22.620418+0000","last_active":"2024-04-25T09:54:22.628318+0000","last_peered":"2024-04-25T09:54:22.628318+0000","last_clean":"2024-04-25T09:54:22.628318+0000","last_became_active":"2024-04-25T09:54:22.620152+0000","last_became_peered":"2024-04-25T09:54:22.620152+0000","last_unstale":"2024-04-25T09:54:22.628318+0000","last_undegraded":"2024-04-25T09:54:22.628318+0000","last_fullsized":"2024-04-25T09:54:22.628318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:02.017763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.615140+0000","last_change":"2024-04-25T09:54:21.622933+0000","last_active":"2024-04-25T09:54:22.615140+0000","last_peered":"2024-04-25T09:54:22.615140+0000","last_clean":"2024-04-25T09:54:22.615140+0000","last_became_active":"2024-04-25T09:54:21.622621+0000","last_became_peered":"2024-04-25T09:54:21.622621+0000","last_unstale":"2024-04-25T09:54:22.615140+0000","last_undegraded":"2024-04-25T09:54:22.615140+0000","last_fullsized":"2024-04-25T09:54:22.615140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:30.927191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.638043+0000","last_change":"2024-04-25T09:54:22.634393+0000","last_active":"2024-04-25T09:54:22.638043+0000","last_peered":"2024-04-25T09:54:22.638043+0000","last_clean":"2024-04-25T09:54:22.638043+0000","last_became_active":"2024-04-25T09:54:22.634059+0000","last_became_peered":"2024-04-25T09:54:22.634059+0000","last_unstale":"2024-04-25T09:54:22.638043+0000","last_undegraded":"2024-04-25T09:54:22.638043+0000","last_fullsized":"2024-04-25T09:54:22.638043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:43.969110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.613368+0000","last_change":"2024-04-25T09:54:21.623616+0000","last_active":"2024-04-25T09:54:22.613368+0000","last_peered":"2024-04-25T09:54:22.613368+0000","last_clean":"2024-04-25T09:54:22.613368+0000","last_became_active":"2024-04-25T09:54:21.623356+0000","last_became_peered":"2024-04-25T09:54:21.623356+0000","last_unstale":"2024-04-25T09:54:22.613368+0000","last_undegraded":"2024-04-25T09:54:22.613368+0000","last_fullsized":"2024-04-25T09:54:22.613368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:37.114504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640166+0000","last_change":"2024-04-25T09:54:22.635983+0000","last_active":"2024-04-25T09:54:22.640166+0000","last_peered":"2024-04-25T09:54:22.640166+0000","last_clean":"2024-04-25T09:54:22.640166+0000","last_became_active":"2024-04-25T09:54:22.635313+0000","last_became_peered":"2024-04-25T09:54:22.635313+0000","last_unstale":"2024-04-25T09:54:22.640166+0000","last_undegraded":"2024-04-25T09:54:22.640166+0000","last_fullsized":"2024-04-25T09:54:22.640166+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:48.853059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.626038+0000","last_change":"2024-04-25T09:54:21.630922+0000","last_active":"2024-04-25T09:54:22.626038+0000","last_peered":"2024-04-25T09:54:22.626038+0000","last_clean":"2024-04-25T09:54:22.626038+0000","last_became_active":"2024-04-25T09:54:21.630617+0000","last_became_peered":"2024-04-25T09:54:21.630617+0000","last_unstale":"2024-04-25T09:54:22.626038+0000","last_undegraded":"2024-04-25T09:54:22.626038+0000","last_fullsized":"2024-04-25T09:54:22.626038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:43.644236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":200,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.625072+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:54:22.625072+0000","last_peered":"2024-04-25T09:54:22.625072+0000","last_clean":"2024-04-25T09:54:22.625072+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:54:22.625072+0000","last_undegraded":"2024-04-25T09:54:22.625072+0000","last_fullsized":"2024-04-25T09:54:22.625072+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":430080,"data_stored":183406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":430080,"data_stored":183426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:54:25.571 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:54:25.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.702+0000 7f5ce9529640 1 Processor -- start 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.702+0000 7f5ce9529640 1 -- start start 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.703+0000 7f5ce9529640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5ce415dc60 0x7f5ce40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.703+0000 7f5ce9529640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce4161850 0x7f5ce40ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.703+0000 7f5ce9529640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce40cf250 con 0x7f5ce40d0970 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.703+0000 7f5ce9529640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce40ce780 con 0x7f5ce415dc60 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.703+0000 7f5ce9529640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40d3730 con 0x7f5ce4161850 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.703+0000 7f5ce2ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5ce415dc60 0x7f5ce40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.703+0000 7f5ce2ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5ce415dc60 0x7f5ce40b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42222/0 (socket says 172.21.15.139:42222) 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.703+0000 7f5ce2ffd640 1 -- 172.21.15.139:0/4099404144 learned_addr learned my addr 172.21.15.139:0/4099404144 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.703+0000 7f5ce27fc640 1 --2- 172.21.15.139:0/4099404144 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce4161850 0x7f5ce40ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.704+0000 7f5ce2ffd640 1 -- 172.21.15.139:0/4099404144 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce4161850 msgr2=0x7f5ce40ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.704+0000 7f5ce2ffd640 1 --2- 172.21.15.139:0/4099404144 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce4161850 0x7f5ce40ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.704+0000 7f5ce2ffd640 1 -- 172.21.15.139:0/4099404144 >> v1:172.21.15.150:6789/0 conn(0x7f5ce40d0970 legacy=0x7f5ce40d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:25.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.704+0000 7f5ce2ffd640 1 -- 172.21.15.139:0/4099404144 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce40ce9b0 con 0x7f5ce415dc60 2024-04-25T09:54:25.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.704+0000 7f5ce2ffd640 1 --2- 172.21.15.139:0/4099404144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ce415dc60 0x7f5ce40b6c10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009e80 tx=0x7f5cd402a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ced0f7a4b739f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:25.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.704+0000 7f5ce1ffb640 1 -- 172.21.15.139:0/4099404144 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4036070 con 0x7f5ce415dc60 2024-04-25T09:54:25.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.704+0000 7f5ce1ffb640 1 -- 172.21.15.139:0/4099404144 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd4031600 con 0x7f5ce415dc60 2024-04-25T09:54:25.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.704+0000 7f5ce1ffb640 1 -- 172.21.15.139:0/4099404144 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd40319d0 con 0x7f5ce415dc60 2024-04-25T09:54:25.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.705+0000 7f5ce9529640 1 -- 172.21.15.139:0/4099404144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ce415dc60 msgr2=0x7f5ce40b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.705+0000 7f5ce9529640 1 --2- 172.21.15.139:0/4099404144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ce415dc60 0x7f5ce40b6c10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009e80 tx=0x7f5cd402a750 comp rx=0 tx=0).stop 2024-04-25T09:54:25.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.705+0000 7f5ce9529640 1 -- 172.21.15.139:0/4099404144 shutdown_connections 2024-04-25T09:54:25.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.705+0000 7f5ce9529640 1 --2- 172.21.15.139:0/4099404144 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce4161850 0x7f5ce40ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.705+0000 7f5ce9529640 1 --2- 172.21.15.139:0/4099404144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ce415dc60 0x7f5ce40b6c10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.705+0000 7f5ce9529640 1 -- 172.21.15.139:0/4099404144 >> 172.21.15.139:0/4099404144 conn(0x7f5ce40b5910 msgr2=0x7f5ce40b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:25.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.705+0000 7f5ce9529640 1 -- 172.21.15.139:0/4099404144 shutdown_connections 2024-04-25T09:54:25.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.705+0000 7f5ce9529640 1 -- 172.21.15.139:0/4099404144 wait complete. 2024-04-25T09:54:25.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.706+0000 7f5ce9529640 1 Processor -- start 2024-04-25T09:54:25.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.706+0000 7f5ce9529640 1 -- start start 2024-04-25T09:54:25.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce9529640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce40d0970 0x7f5ce4144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce9529640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ce415dc60 0x7f5ce415bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce9529640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ce4161850 0x7f5ce40c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce9529640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40d1fc0 con 0x7f5ce415dc60 2024-04-25T09:54:25.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce9529640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40cef70 con 0x7f5ce4161850 2024-04-25T09:54:25.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce9529640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce415e040 con 0x7f5ce40d0970 2024-04-25T09:54:25.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ce415dc60 0x7f5ce415bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ce415dc60 0x7f5ce415bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42226/0 (socket says 172.21.15.139:42226) 2024-04-25T09:54:25.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce27fc640 1 -- 172.21.15.139:0/3700363099 learned_addr learned my addr 172.21.15.139:0/3700363099 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:25.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce2ffd640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce40d0970 0x7f5ce4144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce37fe640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ce4161850 0x7f5ce40c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce27fc640 1 -- 172.21.15.139:0/3700363099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce40d0970 msgr2=0x7f5ce4144740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce27fc640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce40d0970 0x7f5ce4144740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce27fc640 1 -- 172.21.15.139:0/3700363099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ce4161850 msgr2=0x7f5ce40c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce27fc640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ce4161850 0x7f5ce40c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce27fc640 1 -- 172.21.15.139:0/3700363099 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce415c5c0 con 0x7f5ce415dc60 2024-04-25T09:54:25.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.707+0000 7f5ce2ffd640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce40d0970 0x7f5ce4144740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:25.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.708+0000 7f5ce27fc640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ce415dc60 0x7f5ce415bd20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800d5f0 tx=0x7f5cd800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:25.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.708+0000 7f5ccb7fe640 1 -- 172.21.15.139:0/3700363099 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80125f0 con 0x7f5ce415dc60 2024-04-25T09:54:25.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.708+0000 7f5ccb7fe640 1 -- 172.21.15.139:0/3700363099 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd8019070 con 0x7f5ce415dc60 2024-04-25T09:54:25.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.708+0000 7f5ce9529640 1 -- 172.21.15.139:0/3700363099 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce415c8a0 con 0x7f5ce415dc60 2024-04-25T09:54:25.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.708+0000 7f5ccb7fe640 1 -- 172.21.15.139:0/3700363099 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80129d0 con 0x7f5ce415dc60 2024-04-25T09:54:25.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.708+0000 7f5ce9529640 1 -- 172.21.15.139:0/3700363099 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce415ce60 con 0x7f5ce415dc60 2024-04-25T09:54:25.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.709+0000 7f5ce9529640 1 -- 172.21.15.139:0/3700363099 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce4074520 con 0x7f5ce415dc60 2024-04-25T09:54:25.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.709+0000 7f5ccb7fe640 1 -- 172.21.15.139:0/3700363099 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5cd8012b70 con 0x7f5ce415dc60 2024-04-25T09:54:25.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.709+0000 7f5ccb7fe640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5cd006b4d0 0x7f5cd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.710+0000 7f5ccb7fe640 1 -- 172.21.15.139:0/3700363099 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5cd808a6c0 con 0x7f5ce415dc60 2024-04-25T09:54:25.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.710+0000 7f5ce2ffd640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5cd006b4d0 0x7f5cd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.710+0000 7f5ce2ffd640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5cd006b4d0 0x7f5cd006d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4033000 tx=0x7f5cd4009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:25.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.714+0000 7f5ccb7fe640 1 -- 172.21.15.139:0/3700363099 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5cd8059440 con 0x7f5ce415dc60 2024-04-25T09:54:25.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.898+0000 7f5ce9529640 1 -- 172.21.15.139:0/3700363099 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ce40d16f0 con 0x7f5cd006b4d0 2024-04-25T09:54:25.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.905+0000 7f5ccb7fe640 1 -- 172.21.15.139:0/3700363099 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251906 (secure 0 0 0) 0x7f5ce40d16f0 con 0x7f5cd006b4d0 2024-04-25T09:54:25.909 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:25.914 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:54:25.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.908+0000 7f5ce9529640 1 -- 172.21.15.139:0/3700363099 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5cd006b4d0 msgr2=0x7f5cd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.908+0000 7f5ce9529640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5cd006b4d0 0x7f5cd006d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4033000 tx=0x7f5cd4009000 comp rx=0 tx=0).stop 2024-04-25T09:54:25.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.908+0000 7f5ce9529640 1 -- 172.21.15.139:0/3700363099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ce415dc60 msgr2=0x7f5ce415bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.908+0000 7f5ce9529640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ce415dc60 0x7f5ce415bd20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800d5f0 tx=0x7f5cd800dab0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.908+0000 7f5ce9529640 1 -- 172.21.15.139:0/3700363099 shutdown_connections 2024-04-25T09:54:25.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.908+0000 7f5ce9529640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ce415dc60 0x7f5ce415bd20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.908+0000 7f5ce9529640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5cd006b4d0 0x7f5cd006d990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.908+0000 7f5ce9529640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ce4161850 0x7f5ce40c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.908+0000 7f5ce9529640 1 --2- 172.21.15.139:0/3700363099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce40d0970 0x7f5ce4144740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.908+0000 7f5ce9529640 1 -- 172.21.15.139:0/3700363099 >> 172.21.15.139:0/3700363099 conn(0x7f5ce40b5910 msgr2=0x7f5ce40b8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:25.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.908+0000 7f5ce9529640 1 -- 172.21.15.139:0/3700363099 shutdown_connections 2024-04-25T09:54:25.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:25.908+0000 7f5ce9529640 1 -- 172.21.15.139:0/3700363099 wait complete. 2024-04-25T09:54:25.922 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-04-25T09:54:24.981265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220600,"kb_used_data":4424,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754120,"statfs":{"total":773094113280,"available":772868218880,"internally_reserved":0,"allocated":4530176,"data_stored":2604582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.381024"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637887+0000","last_change":"2024-04-25T09:54:22.629232+0000","last_active":"2024-04-25T09:54:22.637887+0000","last_peered":"2024-04-25T09:54:22.637887+0000","last_clean":"2024-04-25T09:54:22.637887+0000","last_became_active":"2024-04-25T09:54:22.628954+0000","last_became_peered":"2024-04-25T09:54:22.628954+0000","last_unstale":"2024-04-25T09:54:22.637887+0000","last_undegraded":"2024-04-25T09:54:22.637887+0000","last_fullsized":"2024-04-25T09:54:22.637887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:05.809924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640899+0000","last_change":"2024-04-25T09:54:22.631588+0000","last_active":"2024-04-25T09:54:22.640899+0000","last_peered":"2024-04-25T09:54:22.640899+0000","last_clean":"2024-04-25T09:54:22.640899+0000","last_became_active":"2024-04-25T09:54:22.630768+0000","last_became_peered":"2024-04-25T09:54:22.630768+0000","last_unstale":"2024-04-25T09:54:22.640899+0000","last_undegraded":"2024-04-25T09:54:22.640899+0000","last_fullsized":"2024-04-25T09:54:22.640899+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:48.853059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641256+0000","last_change":"2024-04-25T09:54:22.635708+0000","last_active":"2024-04-25T09:54:22.641256+0000","last_peered":"2024-04-25T09:54:22.641256+0000","last_clean":"2024-04-25T09:54:22.641256+0000","last_became_active":"2024-04-25T09:54:22.635418+0000","last_became_peered":"2024-04-25T09:54:22.635418+0000","last_unstale":"2024-04-25T09:54:22.641256+0000","last_undegraded":"2024-04-25T09:54:22.641256+0000","last_fullsized":"2024-04-25T09:54:22.641256+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:43.969110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.629794+0000","last_change":"2024-04-25T09:54:22.620743+0000","last_active":"2024-04-25T09:54:22.629794+0000","last_peered":"2024-04-25T09:54:22.629794+0000","last_clean":"2024-04-25T09:54:22.629794+0000","last_became_active":"2024-04-25T09:54:22.620402+0000","last_became_peered":"2024-04-25T09:54:22.620402+0000","last_unstale":"2024-04-25T09:54:22.629794+0000","last_undegraded":"2024-04-25T09:54:22.629794+0000","last_fullsized":"2024-04-25T09:54:22.629794+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:05.510825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.624739+0000","last_change":"2024-04-25T09:54:22.619369+0000","last_active":"2024-04-25T09:54:22.624739+0000","last_peered":"2024-04-25T09:54:22.624739+0000","last_clean":"2024-04-25T09:54:22.624739+0000","last_became_active":"2024-04-25T09:54:22.619106+0000","last_became_peered":"2024-04-25T09:54:22.619106+0000","last_unstale":"2024-04-25T09:54:22.624739+0000","last_undegraded":"2024-04-25T09:54:22.624739+0000","last_fullsized":"2024-04-25T09:54:22.624739+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:47.831985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641067+0000","last_change":"2024-04-25T09:54:22.633793+0000","last_active":"2024-04-25T09:54:22.641067+0000","last_peered":"2024-04-25T09:54:22.641067+0000","last_clean":"2024-04-25T09:54:22.641067+0000","last_became_active":"2024-04-25T09:54:22.633520+0000","last_became_peered":"2024-04-25T09:54:22.633520+0000","last_unstale":"2024-04-25T09:54:22.641067+0000","last_undegraded":"2024-04-25T09:54:22.641067+0000","last_fullsized":"2024-04-25T09:54:22.641067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:55.935357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640090+0000","last_change":"2024-04-25T09:54:22.635734+0000","last_active":"2024-04-25T09:54:22.640090+0000","last_peered":"2024-04-25T09:54:22.640090+0000","last_clean":"2024-04-25T09:54:22.640090+0000","last_became_active":"2024-04-25T09:54:22.635018+0000","last_became_peered":"2024-04-25T09:54:22.635018+0000","last_unstale":"2024-04-25T09:54:22.640090+0000","last_undegraded":"2024-04-25T09:54:22.640090+0000","last_fullsized":"2024-04-25T09:54:22.640090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:05.809924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.636823+0000","last_change":"2024-04-25T09:54:22.628209+0000","last_active":"2024-04-25T09:54:22.636823+0000","last_peered":"2024-04-25T09:54:22.636823+0000","last_clean":"2024-04-25T09:54:22.636823+0000","last_became_active":"2024-04-25T09:54:22.627872+0000","last_became_peered":"2024-04-25T09:54:22.627872+0000","last_unstale":"2024-04-25T09:54:22.636823+0000","last_undegraded":"2024-04-25T09:54:22.636823+0000","last_fullsized":"2024-04-25T09:54:22.636823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:17.442893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.644781+0000","last_change":"2024-04-25T09:54:22.642228+0000","last_active":"2024-04-25T09:54:22.644781+0000","last_peered":"2024-04-25T09:54:22.644781+0000","last_clean":"2024-04-25T09:54:22.644781+0000","last_became_active":"2024-04-25T09:54:22.641976+0000","last_became_peered":"2024-04-25T09:54:22.641976+0000","last_unstale":"2024-04-25T09:54:22.644781+0000","last_undegraded":"2024-04-25T09:54:22.644781+0000","last_fullsized":"2024-04-25T09:54:22.644781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:15.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640118+0000","last_change":"2024-04-25T09:54:22.635786+0000","last_active":"2024-04-25T09:54:22.640118+0000","last_peered":"2024-04-25T09:54:22.640118+0000","last_clean":"2024-04-25T09:54:22.640118+0000","last_became_active":"2024-04-25T09:54:22.635459+0000","last_became_peered":"2024-04-25T09:54:22.635459+0000","last_unstale":"2024-04-25T09:54:22.640118+0000","last_undegraded":"2024-04-25T09:54:22.640118+0000","last_fullsized":"2024-04-25T09:54:22.640118+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:35.775552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.639859+0000","last_change":"2024-04-25T09:54:22.633086+0000","last_active":"2024-04-25T09:54:22.639859+0000","last_peered":"2024-04-25T09:54:22.639859+0000","last_clean":"2024-04-25T09:54:22.639859+0000","last_became_active":"2024-04-25T09:54:22.632818+0000","last_became_peered":"2024-04-25T09:54:22.632818+0000","last_unstale":"2024-04-25T09:54:22.639859+0000","last_undegraded":"2024-04-25T09:54:22.639859+0000","last_fullsized":"2024-04-25T09:54:22.639859+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:00.273606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640691+0000","last_change":"2024-04-25T09:54:22.631312+0000","last_active":"2024-04-25T09:54:22.640691+0000","last_peered":"2024-04-25T09:54:22.640691+0000","last_clean":"2024-04-25T09:54:22.640691+0000","last_became_active":"2024-04-25T09:54:22.630966+0000","last_became_peered":"2024-04-25T09:54:22.630966+0000","last_unstale":"2024-04-25T09:54:22.640691+0000","last_undegraded":"2024-04-25T09:54:22.640691+0000","last_fullsized":"2024-04-25T09:54:22.640691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:42.167481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645185+0000","last_change":"2024-04-25T09:54:22.643156+0000","last_active":"2024-04-25T09:54:22.645185+0000","last_peered":"2024-04-25T09:54:22.645185+0000","last_clean":"2024-04-25T09:54:22.645185+0000","last_became_active":"2024-04-25T09:54:22.642950+0000","last_became_peered":"2024-04-25T09:54:22.642950+0000","last_unstale":"2024-04-25T09:54:22.645185+0000","last_undegraded":"2024-04-25T09:54:22.645185+0000","last_fullsized":"2024-04-25T09:54:22.645185+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:32.763822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.636166+0000","last_change":"2024-04-25T09:54:22.629262+0000","last_active":"2024-04-25T09:54:22.636166+0000","last_peered":"2024-04-25T09:54:22.636166+0000","last_clean":"2024-04-25T09:54:22.636166+0000","last_became_active":"2024-04-25T09:54:22.628606+0000","last_became_peered":"2024-04-25T09:54:22.628606+0000","last_unstale":"2024-04-25T09:54:22.636166+0000","last_undegraded":"2024-04-25T09:54:22.636166+0000","last_fullsized":"2024-04-25T09:54:22.636166+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:44.646972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.639776+0000","last_change":"2024-04-25T09:54:22.631071+0000","last_active":"2024-04-25T09:54:22.639776+0000","last_peered":"2024-04-25T09:54:22.639776+0000","last_clean":"2024-04-25T09:54:22.639776+0000","last_became_active":"2024-04-25T09:54:22.630398+0000","last_became_peered":"2024-04-25T09:54:22.630398+0000","last_unstale":"2024-04-25T09:54:22.639776+0000","last_undegraded":"2024-04-25T09:54:22.639776+0000","last_fullsized":"2024-04-25T09:54:22.639776+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:28.420626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645111+0000","last_change":"2024-04-25T09:54:22.643249+0000","last_active":"2024-04-25T09:54:22.645111+0000","last_peered":"2024-04-25T09:54:22.645111+0000","last_clean":"2024-04-25T09:54:22.645111+0000","last_became_active":"2024-04-25T09:54:22.642950+0000","last_became_peered":"2024-04-25T09:54:22.642950+0000","last_unstale":"2024-04-25T09:54:22.645111+0000","last_undegraded":"2024-04-25T09:54:22.645111+0000","last_fullsized":"2024-04-25T09:54:22.645111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:28.420626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.644958+0000","last_change":"2024-04-25T09:54:22.643337+0000","last_active":"2024-04-25T09:54:22.644958+0000","last_peered":"2024-04-25T09:54:22.644958+0000","last_clean":"2024-04-25T09:54:22.644958+0000","last_became_active":"2024-04-25T09:54:22.643074+0000","last_became_peered":"2024-04-25T09:54:22.643074+0000","last_unstale":"2024-04-25T09:54:22.644958+0000","last_undegraded":"2024-04-25T09:54:22.644958+0000","last_fullsized":"2024-04-25T09:54:22.644958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:00.273606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637262+0000","last_change":"2024-04-25T09:54:22.632925+0000","last_active":"2024-04-25T09:54:22.637262+0000","last_peered":"2024-04-25T09:54:22.637262+0000","last_clean":"2024-04-25T09:54:22.637262+0000","last_became_active":"2024-04-25T09:54:22.632629+0000","last_became_peered":"2024-04-25T09:54:22.632629+0000","last_unstale":"2024-04-25T09:54:22.637262+0000","last_undegraded":"2024-04-25T09:54:22.637262+0000","last_fullsized":"2024-04-25T09:54:22.637262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:02.017763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.629691+0000","last_change":"2024-04-25T09:54:22.624361+0000","last_active":"2024-04-25T09:54:22.629691+0000","last_peered":"2024-04-25T09:54:22.629691+0000","last_clean":"2024-04-25T09:54:22.629691+0000","last_became_active":"2024-04-25T09:54:22.624099+0000","last_became_peered":"2024-04-25T09:54:22.624099+0000","last_unstale":"2024-04-25T09:54:22.629691+0000","last_undegraded":"2024-04-25T09:54:22.629691+0000","last_fullsized":"2024-04-25T09:54:22.629691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:17.096153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.628632+0000","last_change":"2024-04-25T09:54:22.626708+0000","last_active":"2024-04-25T09:54:22.628632+0000","last_peered":"2024-04-25T09:54:22.628632+0000","last_clean":"2024-04-25T09:54:22.628632+0000","last_became_active":"2024-04-25T09:54:22.626430+0000","last_became_peered":"2024-04-25T09:54:22.626430+0000","last_unstale":"2024-04-25T09:54:22.628632+0000","last_undegraded":"2024-04-25T09:54:22.628632+0000","last_fullsized":"2024-04-25T09:54:22.628632+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:57.631687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645042+0000","last_change":"2024-04-25T09:54:22.642125+0000","last_active":"2024-04-25T09:54:22.645042+0000","last_peered":"2024-04-25T09:54:22.645042+0000","last_clean":"2024-04-25T09:54:22.645042+0000","last_became_active":"2024-04-25T09:54:22.641836+0000","last_became_peered":"2024-04-25T09:54:22.641836+0000","last_unstale":"2024-04-25T09:54:22.645042+0000","last_undegraded":"2024-04-25T09:54:22.645042+0000","last_fullsized":"2024-04-25T09:54:22.645042+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:37.216863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.630337+0000","last_change":"2024-04-25T09:54:22.625201+0000","last_active":"2024-04-25T09:54:22.630337+0000","last_peered":"2024-04-25T09:54:22.630337+0000","last_clean":"2024-04-25T09:54:22.630337+0000","last_became_active":"2024-04-25T09:54:22.624920+0000","last_became_peered":"2024-04-25T09:54:22.624920+0000","last_unstale":"2024-04-25T09:54:22.630337+0000","last_undegraded":"2024-04-25T09:54:22.630337+0000","last_fullsized":"2024-04-25T09:54:22.630337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:55.935357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637636+0000","last_change":"2024-04-25T09:54:22.631584+0000","last_active":"2024-04-25T09:54:22.637636+0000","last_peered":"2024-04-25T09:54:22.637636+0000","last_clean":"2024-04-25T09:54:22.637636+0000","last_became_active":"2024-04-25T09:54:22.630401+0000","last_became_peered":"2024-04-25T09:54:22.630401+0000","last_unstale":"2024-04-25T09:54:22.637636+0000","last_undegraded":"2024-04-25T09:54:22.637636+0000","last_fullsized":"2024-04-25T09:54:22.637636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:44.049423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637609+0000","last_change":"2024-04-25T09:54:22.627838+0000","last_active":"2024-04-25T09:54:22.637609+0000","last_peered":"2024-04-25T09:54:22.637609+0000","last_clean":"2024-04-25T09:54:22.637609+0000","last_became_active":"2024-04-25T09:54:22.627536+0000","last_became_peered":"2024-04-25T09:54:22.627536+0000","last_unstale":"2024-04-25T09:54:22.637609+0000","last_undegraded":"2024-04-25T09:54:22.637609+0000","last_fullsized":"2024-04-25T09:54:22.637609+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:46.210520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.629881+0000","last_change":"2024-04-25T09:54:22.628389+0000","last_active":"2024-04-25T09:54:22.629881+0000","last_peered":"2024-04-25T09:54:22.629881+0000","last_clean":"2024-04-25T09:54:22.629881+0000","last_became_active":"2024-04-25T09:54:22.628106+0000","last_became_peered":"2024-04-25T09:54:22.628106+0000","last_unstale":"2024-04-25T09:54:22.629881+0000","last_undegraded":"2024-04-25T09:54:22.629881+0000","last_fullsized":"2024-04-25T09:54:22.629881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:17.450735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641000+0000","last_change":"2024-04-25T09:54:22.636714+0000","last_active":"2024-04-25T09:54:22.641000+0000","last_peered":"2024-04-25T09:54:22.641000+0000","last_clean":"2024-04-25T09:54:22.641000+0000","last_became_active":"2024-04-25T09:54:22.636450+0000","last_became_peered":"2024-04-25T09:54:22.636450+0000","last_unstale":"2024-04-25T09:54:22.641000+0000","last_undegraded":"2024-04-25T09:54:22.641000+0000","last_fullsized":"2024-04-25T09:54:22.641000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:43.969110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637171+0000","last_change":"2024-04-25T09:54:22.634667+0000","last_active":"2024-04-25T09:54:22.637171+0000","last_peered":"2024-04-25T09:54:22.637171+0000","last_clean":"2024-04-25T09:54:22.637171+0000","last_became_active":"2024-04-25T09:54:22.632588+0000","last_became_peered":"2024-04-25T09:54:22.632588+0000","last_unstale":"2024-04-25T09:54:22.637171+0000","last_undegraded":"2024-04-25T09:54:22.637171+0000","last_fullsized":"2024-04-25T09:54:22.637171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:42.167481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.639742+0000","last_change":"2024-04-25T09:54:22.634443+0000","last_active":"2024-04-25T09:54:22.639742+0000","last_peered":"2024-04-25T09:54:22.639742+0000","last_clean":"2024-04-25T09:54:22.639742+0000","last_became_active":"2024-04-25T09:54:22.634186+0000","last_became_peered":"2024-04-25T09:54:22.634186+0000","last_unstale":"2024-04-25T09:54:22.639742+0000","last_undegraded":"2024-04-25T09:54:22.639742+0000","last_fullsized":"2024-04-25T09:54:22.639742+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:17.442893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641389+0000","last_change":"2024-04-25T09:54:22.638963+0000","last_active":"2024-04-25T09:54:22.641389+0000","last_peered":"2024-04-25T09:54:22.641389+0000","last_clean":"2024-04-25T09:54:22.641389+0000","last_became_active":"2024-04-25T09:54:22.638673+0000","last_became_peered":"2024-04-25T09:54:22.638673+0000","last_unstale":"2024-04-25T09:54:22.641389+0000","last_undegraded":"2024-04-25T09:54:22.641389+0000","last_fullsized":"2024-04-25T09:54:22.641389+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:55.935357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.628683+0000","last_change":"2024-04-25T09:54:22.625924+0000","last_active":"2024-04-25T09:54:22.628683+0000","last_peered":"2024-04-25T09:54:22.628683+0000","last_clean":"2024-04-25T09:54:22.628683+0000","last_became_active":"2024-04-25T09:54:22.625579+0000","last_became_peered":"2024-04-25T09:54:22.625579+0000","last_unstale":"2024-04-25T09:54:22.628683+0000","last_undegraded":"2024-04-25T09:54:22.628683+0000","last_fullsized":"2024-04-25T09:54:22.628683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:23.664566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.643815+0000","last_change":"2024-04-25T09:54:22.639605+0000","last_active":"2024-04-25T09:54:22.643815+0000","last_peered":"2024-04-25T09:54:22.643815+0000","last_clean":"2024-04-25T09:54:22.643815+0000","last_became_active":"2024-04-25T09:54:22.638280+0000","last_became_peered":"2024-04-25T09:54:22.638280+0000","last_unstale":"2024-04-25T09:54:22.643815+0000","last_undegraded":"2024-04-25T09:54:22.643815+0000","last_fullsized":"2024-04-25T09:54:22.643815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:15.614725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.644542+0000","last_change":"2024-04-25T09:54:22.637447+0000","last_active":"2024-04-25T09:54:22.644542+0000","last_peered":"2024-04-25T09:54:22.644542+0000","last_clean":"2024-04-25T09:54:22.644542+0000","last_became_active":"2024-04-25T09:54:22.637181+0000","last_became_peered":"2024-04-25T09:54:22.637181+0000","last_unstale":"2024-04-25T09:54:22.644542+0000","last_undegraded":"2024-04-25T09:54:22.644542+0000","last_fullsized":"2024-04-25T09:54:22.644542+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:38.117240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645029+0000","last_change":"2024-04-25T09:54:22.643465+0000","last_active":"2024-04-25T09:54:22.645029+0000","last_peered":"2024-04-25T09:54:22.645029+0000","last_clean":"2024-04-25T09:54:22.645029+0000","last_became_active":"2024-04-25T09:54:22.643181+0000","last_became_peered":"2024-04-25T09:54:22.643181+0000","last_unstale":"2024-04-25T09:54:22.645029+0000","last_undegraded":"2024-04-25T09:54:22.645029+0000","last_fullsized":"2024-04-25T09:54:22.645029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:17.442893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.638197+0000","last_change":"2024-04-25T09:54:21.621031+0000","last_active":"2024-04-25T09:54:22.638197+0000","last_peered":"2024-04-25T09:54:22.638197+0000","last_clean":"2024-04-25T09:54:22.638197+0000","last_became_active":"2024-04-25T09:54:21.620741+0000","last_became_peered":"2024-04-25T09:54:21.620741+0000","last_unstale":"2024-04-25T09:54:22.638197+0000","last_undegraded":"2024-04-25T09:54:22.638197+0000","last_fullsized":"2024-04-25T09:54:22.638197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:30.927191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640968+0000","last_change":"2024-04-25T09:54:22.636146+0000","last_active":"2024-04-25T09:54:22.640968+0000","last_peered":"2024-04-25T09:54:22.640968+0000","last_clean":"2024-04-25T09:54:22.640968+0000","last_became_active":"2024-04-25T09:54:22.635843+0000","last_became_peered":"2024-04-25T09:54:22.635843+0000","last_unstale":"2024-04-25T09:54:22.640968+0000","last_undegraded":"2024-04-25T09:54:22.640968+0000","last_fullsized":"2024-04-25T09:54:22.640968+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:44.049423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.634679+0000","last_change":"2024-04-25T09:54:21.626269+0000","last_active":"2024-04-25T09:54:22.634679+0000","last_peered":"2024-04-25T09:54:22.634679+0000","last_clean":"2024-04-25T09:54:22.634679+0000","last_became_active":"2024-04-25T09:54:21.625976+0000","last_became_peered":"2024-04-25T09:54:21.625976+0000","last_unstale":"2024-04-25T09:54:22.634679+0000","last_undegraded":"2024-04-25T09:54:22.634679+0000","last_fullsized":"2024-04-25T09:54:22.634679+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:16.447999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637486+0000","last_change":"2024-04-25T09:54:22.632816+0000","last_active":"2024-04-25T09:54:22.637486+0000","last_peered":"2024-04-25T09:54:22.637486+0000","last_clean":"2024-04-25T09:54:22.637486+0000","last_became_active":"2024-04-25T09:54:22.631163+0000","last_became_peered":"2024-04-25T09:54:22.631163+0000","last_unstale":"2024-04-25T09:54:22.637486+0000","last_undegraded":"2024-04-25T09:54:22.637486+0000","last_fullsized":"2024-04-25T09:54:22.637486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:43.969110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637186+0000","last_change":"2024-04-25T09:54:21.627063+0000","last_active":"2024-04-25T09:54:22.637186+0000","last_peered":"2024-04-25T09:54:22.637186+0000","last_clean":"2024-04-25T09:54:22.637186+0000","last_became_active":"2024-04-25T09:54:21.626644+0000","last_became_peered":"2024-04-25T09:54:21.626644+0000","last_unstale":"2024-04-25T09:54:22.637186+0000","last_undegraded":"2024-04-25T09:54:22.637186+0000","last_fullsized":"2024-04-25T09:54:22.637186+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:01.015027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637866+0000","last_change":"2024-04-25T09:54:22.626719+0000","last_active":"2024-04-25T09:54:22.637866+0000","last_peered":"2024-04-25T09:54:22.637866+0000","last_clean":"2024-04-25T09:54:22.637866+0000","last_became_active":"2024-04-25T09:54:22.626470+0000","last_became_peered":"2024-04-25T09:54:22.626470+0000","last_unstale":"2024-04-25T09:54:22.637866+0000","last_undegraded":"2024-04-25T09:54:22.637866+0000","last_fullsized":"2024-04-25T09:54:22.637866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:31.082648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641512+0000","last_change":"2024-04-25T09:54:21.622728+0000","last_active":"2024-04-25T09:54:22.641512+0000","last_peered":"2024-04-25T09:54:22.641512+0000","last_clean":"2024-04-25T09:54:22.641512+0000","last_became_active":"2024-04-25T09:54:21.622435+0000","last_became_peered":"2024-04-25T09:54:21.622435+0000","last_unstale":"2024-04-25T09:54:22.641512+0000","last_undegraded":"2024-04-25T09:54:22.641512+0000","last_fullsized":"2024-04-25T09:54:22.641512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:22.661830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.644307+0000","last_change":"2024-04-25T09:54:22.638528+0000","last_active":"2024-04-25T09:54:22.644307+0000","last_peered":"2024-04-25T09:54:22.644307+0000","last_clean":"2024-04-25T09:54:22.644307+0000","last_became_active":"2024-04-25T09:54:22.638221+0000","last_became_peered":"2024-04-25T09:54:22.638221+0000","last_unstale":"2024-04-25T09:54:22.644307+0000","last_undegraded":"2024-04-25T09:54:22.644307+0000","last_fullsized":"2024-04-25T09:54:22.644307+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:38.117240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641749+0000","last_change":"2024-04-25T09:54:21.619668+0000","last_active":"2024-04-25T09:54:22.641749+0000","last_peered":"2024-04-25T09:54:22.641749+0000","last_clean":"2024-04-25T09:54:22.641749+0000","last_became_active":"2024-04-25T09:54:21.619404+0000","last_became_peered":"2024-04-25T09:54:21.619404+0000","last_unstale":"2024-04-25T09:54:22.641749+0000","last_undegraded":"2024-04-25T09:54:22.641749+0000","last_fullsized":"2024-04-25T09:54:22.641749+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:42.966374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.644995+0000","last_change":"2024-04-25T09:54:22.642120+0000","last_active":"2024-04-25T09:54:22.644995+0000","last_peered":"2024-04-25T09:54:22.644995+0000","last_clean":"2024-04-25T09:54:22.644995+0000","last_became_active":"2024-04-25T09:54:22.641778+0000","last_became_peered":"2024-04-25T09:54:22.641778+0000","last_unstale":"2024-04-25T09:54:22.644995+0000","last_undegraded":"2024-04-25T09:54:22.644995+0000","last_fullsized":"2024-04-25T09:54:22.644995+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:32.041828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640059+0000","last_change":"2024-04-25T09:54:21.630270+0000","last_active":"2024-04-25T09:54:22.640059+0000","last_peered":"2024-04-25T09:54:22.640059+0000","last_clean":"2024-04-25T09:54:22.640059+0000","last_became_active":"2024-04-25T09:54:21.629920+0000","last_became_peered":"2024-04-25T09:54:21.629920+0000","last_unstale":"2024-04-25T09:54:22.640059+0000","last_undegraded":"2024-04-25T09:54:22.640059+0000","last_fullsized":"2024-04-25T09:54:22.640059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:54.932621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.638796+0000","last_change":"2024-04-25T09:54:22.634359+0000","last_active":"2024-04-25T09:54:22.638796+0000","last_peered":"2024-04-25T09:54:22.638796+0000","last_clean":"2024-04-25T09:54:22.638796+0000","last_became_active":"2024-04-25T09:54:22.631933+0000","last_became_peered":"2024-04-25T09:54:22.631933+0000","last_unstale":"2024-04-25T09:54:22.638796+0000","last_undegraded":"2024-04-25T09:54:22.638796+0000","last_fullsized":"2024-04-25T09:54:22.638796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:24.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.617921+0000","last_change":"2024-04-25T09:54:21.627673+0000","last_active":"2024-04-25T09:54:22.617921+0000","last_peered":"2024-04-25T09:54:22.617921+0000","last_clean":"2024-04-25T09:54:22.617921+0000","last_became_active":"2024-04-25T09:54:21.627431+0000","last_became_peered":"2024-04-25T09:54:21.627431+0000","last_unstale":"2024-04-25T09:54:22.617921+0000","last_undegraded":"2024-04-25T09:54:22.617921+0000","last_fullsized":"2024-04-25T09:54:22.617921+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:22.661830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640953+0000","last_change":"2024-04-25T09:54:22.633487+0000","last_active":"2024-04-25T09:54:22.640953+0000","last_peered":"2024-04-25T09:54:22.640953+0000","last_clean":"2024-04-25T09:54:22.640953+0000","last_became_active":"2024-04-25T09:54:22.633164+0000","last_became_peered":"2024-04-25T09:54:22.633164+0000","last_unstale":"2024-04-25T09:54:22.640953+0000","last_undegraded":"2024-04-25T09:54:22.640953+0000","last_fullsized":"2024-04-25T09:54:22.640953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:32.763822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.615432+0000","last_change":"2024-04-25T09:54:21.625160+0000","last_active":"2024-04-25T09:54:22.615432+0000","last_peered":"2024-04-25T09:54:22.615432+0000","last_clean":"2024-04-25T09:54:22.615432+0000","last_became_active":"2024-04-25T09:54:21.624854+0000","last_became_peered":"2024-04-25T09:54:21.624854+0000","last_unstale":"2024-04-25T09:54:22.615432+0000","last_undegraded":"2024-04-25T09:54:22.615432+0000","last_fullsized":"2024-04-25T09:54:22.615432+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:56.628951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.636998+0000","last_change":"2024-04-25T09:54:22.625054+0000","last_active":"2024-04-25T09:54:22.636998+0000","last_peered":"2024-04-25T09:54:22.636998+0000","last_clean":"2024-04-25T09:54:22.636998+0000","last_became_active":"2024-04-25T09:54:22.624736+0000","last_became_peered":"2024-04-25T09:54:22.624736+0000","last_unstale":"2024-04-25T09:54:22.636998+0000","last_undegraded":"2024-04-25T09:54:22.636998+0000","last_fullsized":"2024-04-25T09:54:22.636998+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:02.344151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.613004+0000","last_change":"2024-04-25T09:54:21.622690+0000","last_active":"2024-04-25T09:54:22.613004+0000","last_peered":"2024-04-25T09:54:22.613004+0000","last_clean":"2024-04-25T09:54:22.613004+0000","last_became_active":"2024-04-25T09:54:21.622427+0000","last_became_peered":"2024-04-25T09:54:21.622427+0000","last_unstale":"2024-04-25T09:54:22.613004+0000","last_undegraded":"2024-04-25T09:54:22.613004+0000","last_fullsized":"2024-04-25T09:54:22.613004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:23.985952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.639978+0000","last_change":"2024-04-25T09:54:22.636609+0000","last_active":"2024-04-25T09:54:22.639978+0000","last_peered":"2024-04-25T09:54:22.639978+0000","last_clean":"2024-04-25T09:54:22.639978+0000","last_became_active":"2024-04-25T09:54:22.635042+0000","last_became_peered":"2024-04-25T09:54:22.635042+0000","last_unstale":"2024-04-25T09:54:22.639978+0000","last_undegraded":"2024-04-25T09:54:22.639978+0000","last_fullsized":"2024-04-25T09:54:22.639978+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:24.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.621536+0000","last_change":"2024-04-25T09:54:21.623071+0000","last_active":"2024-04-25T09:54:22.621536+0000","last_peered":"2024-04-25T09:54:22.621536+0000","last_clean":"2024-04-25T09:54:22.621536+0000","last_became_active":"2024-04-25T09:54:21.622746+0000","last_became_peered":"2024-04-25T09:54:21.622746+0000","last_unstale":"2024-04-25T09:54:22.621536+0000","last_undegraded":"2024-04-25T09:54:22.621536+0000","last_fullsized":"2024-04-25T09:54:22.621536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:37.114504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640927+0000","last_change":"2024-04-25T09:54:22.635160+0000","last_active":"2024-04-25T09:54:22.640927+0000","last_peered":"2024-04-25T09:54:22.640927+0000","last_clean":"2024-04-25T09:54:22.640927+0000","last_became_active":"2024-04-25T09:54:22.634866+0000","last_became_peered":"2024-04-25T09:54:22.634866+0000","last_unstale":"2024-04-25T09:54:22.640927+0000","last_undegraded":"2024-04-25T09:54:22.640927+0000","last_fullsized":"2024-04-25T09:54:22.640927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:44.646972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.642917+0000","last_change":"2024-04-25T09:54:21.628294+0000","last_active":"2024-04-25T09:54:22.642917+0000","last_peered":"2024-04-25T09:54:22.642917+0000","last_clean":"2024-04-25T09:54:22.642917+0000","last_became_active":"2024-04-25T09:54:21.628022+0000","last_became_peered":"2024-04-25T09:54:21.628022+0000","last_unstale":"2024-04-25T09:54:22.642917+0000","last_undegraded":"2024-04-25T09:54:22.642917+0000","last_fullsized":"2024-04-25T09:54:22.642917+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:23.985952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.628258+0000","last_change":"2024-04-25T09:54:22.621387+0000","last_active":"2024-04-25T09:54:22.628258+0000","last_peered":"2024-04-25T09:54:22.628258+0000","last_clean":"2024-04-25T09:54:22.628258+0000","last_became_active":"2024-04-25T09:54:22.621117+0000","last_became_peered":"2024-04-25T09:54:22.621117+0000","last_unstale":"2024-04-25T09:54:22.628258+0000","last_undegraded":"2024-04-25T09:54:22.628258+0000","last_fullsized":"2024-04-25T09:54:22.628258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:42.167481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645238+0000","last_change":"2024-04-25T09:54:21.626625+0000","last_active":"2024-04-25T09:54:22.645238+0000","last_peered":"2024-04-25T09:54:22.645238+0000","last_clean":"2024-04-25T09:54:22.645238+0000","last_became_active":"2024-04-25T09:54:21.626367+0000","last_became_peered":"2024-04-25T09:54:21.626367+0000","last_unstale":"2024-04-25T09:54:22.645238+0000","last_undegraded":"2024-04-25T09:54:22.645238+0000","last_fullsized":"2024-04-25T09:54:22.645238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:14.611990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641016+0000","last_change":"2024-04-25T09:54:22.636106+0000","last_active":"2024-04-25T09:54:22.641016+0000","last_peered":"2024-04-25T09:54:22.641016+0000","last_clean":"2024-04-25T09:54:22.641016+0000","last_became_active":"2024-04-25T09:54:22.635804+0000","last_became_peered":"2024-04-25T09:54:22.635804+0000","last_unstale":"2024-04-25T09:54:22.641016+0000","last_undegraded":"2024-04-25T09:54:22.641016+0000","last_fullsized":"2024-04-25T09:54:22.641016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:24.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.644564+0000","last_change":"2024-04-25T09:54:21.620904+0000","last_active":"2024-04-25T09:54:22.644564+0000","last_peered":"2024-04-25T09:54:22.644564+0000","last_clean":"2024-04-25T09:54:22.644564+0000","last_became_active":"2024-04-25T09:54:21.620628+0000","last_became_peered":"2024-04-25T09:54:21.620628+0000","last_unstale":"2024-04-25T09:54:22.644564+0000","last_undegraded":"2024-04-25T09:54:22.644564+0000","last_fullsized":"2024-04-25T09:54:22.644564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:59.343752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.636783+0000","last_change":"2024-04-25T09:54:22.629277+0000","last_active":"2024-04-25T09:54:22.636783+0000","last_peered":"2024-04-25T09:54:22.636783+0000","last_clean":"2024-04-25T09:54:22.636783+0000","last_became_active":"2024-04-25T09:54:22.629002+0000","last_became_peered":"2024-04-25T09:54:22.629002+0000","last_unstale":"2024-04-25T09:54:22.636783+0000","last_undegraded":"2024-04-25T09:54:22.636783+0000","last_fullsized":"2024-04-25T09:54:22.636783+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:35.775552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.618448+0000","last_change":"2024-04-25T09:54:21.628544+0000","last_active":"2024-04-25T09:54:22.618448+0000","last_peered":"2024-04-25T09:54:22.618448+0000","last_clean":"2024-04-25T09:54:22.618448+0000","last_became_active":"2024-04-25T09:54:21.628273+0000","last_became_peered":"2024-04-25T09:54:21.628273+0000","last_unstale":"2024-04-25T09:54:22.618448+0000","last_undegraded":"2024-04-25T09:54:22.618448+0000","last_fullsized":"2024-04-25T09:54:22.618448+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:56.628951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.643887+0000","last_change":"2024-04-25T09:54:22.639839+0000","last_active":"2024-04-25T09:54:22.643887+0000","last_peered":"2024-04-25T09:54:22.643887+0000","last_clean":"2024-04-25T09:54:22.643887+0000","last_became_active":"2024-04-25T09:54:22.638248+0000","last_became_peered":"2024-04-25T09:54:22.638248+0000","last_unstale":"2024-04-25T09:54:22.643887+0000","last_undegraded":"2024-04-25T09:54:22.643887+0000","last_fullsized":"2024-04-25T09:54:22.643887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:00.346488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.619958+0000","last_change":"2024-04-25T09:54:21.621771+0000","last_active":"2024-04-25T09:54:22.619958+0000","last_peered":"2024-04-25T09:54:22.619958+0000","last_clean":"2024-04-25T09:54:22.619958+0000","last_became_active":"2024-04-25T09:54:21.621433+0000","last_became_peered":"2024-04-25T09:54:21.621433+0000","last_unstale":"2024-04-25T09:54:22.619958+0000","last_undegraded":"2024-04-25T09:54:22.619958+0000","last_fullsized":"2024-04-25T09:54:22.619958+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:19.872298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.637881+0000","last_change":"2024-04-25T09:54:22.628174+0000","last_active":"2024-04-25T09:54:22.637881+0000","last_peered":"2024-04-25T09:54:22.637881+0000","last_clean":"2024-04-25T09:54:22.637881+0000","last_became_active":"2024-04-25T09:54:22.627900+0000","last_became_peered":"2024-04-25T09:54:22.627900+0000","last_unstale":"2024-04-25T09:54:22.637881+0000","last_undegraded":"2024-04-25T09:54:22.637881+0000","last_fullsized":"2024-04-25T09:54:22.637881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:00.273606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645556+0000","last_change":"2024-04-25T09:54:21.629589+0000","last_active":"2024-04-25T09:54:22.645556+0000","last_peered":"2024-04-25T09:54:22.645556+0000","last_clean":"2024-04-25T09:54:22.645556+0000","last_became_active":"2024-04-25T09:54:21.629290+0000","last_became_peered":"2024-04-25T09:54:21.629290+0000","last_unstale":"2024-04-25T09:54:22.645556+0000","last_undegraded":"2024-04-25T09:54:22.645556+0000","last_fullsized":"2024-04-25T09:54:22.645556+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:01.015027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.636854+0000","last_change":"2024-04-25T09:54:22.627570+0000","last_active":"2024-04-25T09:54:22.636854+0000","last_peered":"2024-04-25T09:54:22.636854+0000","last_clean":"2024-04-25T09:54:22.636854+0000","last_became_active":"2024-04-25T09:54:22.627313+0000","last_became_peered":"2024-04-25T09:54:22.627313+0000","last_unstale":"2024-04-25T09:54:22.636854+0000","last_undegraded":"2024-04-25T09:54:22.636854+0000","last_fullsized":"2024-04-25T09:54:22.636854+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:28.420626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641104+0000","last_change":"2024-04-25T09:54:21.628865+0000","last_active":"2024-04-25T09:54:22.641104+0000","last_peered":"2024-04-25T09:54:22.641104+0000","last_clean":"2024-04-25T09:54:22.641104+0000","last_became_active":"2024-04-25T09:54:21.628530+0000","last_became_peered":"2024-04-25T09:54:21.628530+0000","last_unstale":"2024-04-25T09:54:22.641104+0000","last_undegraded":"2024-04-25T09:54:22.641104+0000","last_fullsized":"2024-04-25T09:54:22.641104+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:14.611990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.645288+0000","last_change":"2024-04-25T09:54:22.643230+0000","last_active":"2024-04-25T09:54:22.645288+0000","last_peered":"2024-04-25T09:54:22.645288+0000","last_clean":"2024-04-25T09:54:22.645288+0000","last_became_active":"2024-04-25T09:54:22.642966+0000","last_became_peered":"2024-04-25T09:54:22.642966+0000","last_unstale":"2024-04-25T09:54:22.645288+0000","last_undegraded":"2024-04-25T09:54:22.645288+0000","last_fullsized":"2024-04-25T09:54:22.645288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:27.537919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641441+0000","last_change":"2024-04-25T09:54:21.632060+0000","last_active":"2024-04-25T09:54:22.641441+0000","last_peered":"2024-04-25T09:54:22.641441+0000","last_clean":"2024-04-25T09:54:22.641441+0000","last_became_active":"2024-04-25T09:54:21.631637+0000","last_became_peered":"2024-04-25T09:54:21.631637+0000","last_unstale":"2024-04-25T09:54:22.641441+0000","last_undegraded":"2024-04-25T09:54:22.641441+0000","last_fullsized":"2024-04-25T09:54:22.641441+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:16.447999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.636230+0000","last_change":"2024-04-25T09:54:22.630550+0000","last_active":"2024-04-25T09:54:22.636230+0000","last_peered":"2024-04-25T09:54:22.636230+0000","last_clean":"2024-04-25T09:54:22.636230+0000","last_became_active":"2024-04-25T09:54:22.630269+0000","last_became_peered":"2024-04-25T09:54:22.630269+0000","last_unstale":"2024-04-25T09:54:22.636230+0000","last_undegraded":"2024-04-25T09:54:22.636230+0000","last_fullsized":"2024-04-25T09:54:22.636230+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:44.049423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640868+0000","last_change":"2024-04-25T09:54:21.625832+0000","last_active":"2024-04-25T09:54:22.640868+0000","last_peered":"2024-04-25T09:54:22.640868+0000","last_clean":"2024-04-25T09:54:22.640868+0000","last_became_active":"2024-04-25T09:54:21.625556+0000","last_became_peered":"2024-04-25T09:54:21.625556+0000","last_unstale":"2024-04-25T09:54:22.640868+0000","last_undegraded":"2024-04-25T09:54:22.640868+0000","last_fullsized":"2024-04-25T09:54:22.640868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:52.007492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.629650+0000","last_change":"2024-04-25T09:54:22.619869+0000","last_active":"2024-04-25T09:54:22.629650+0000","last_peered":"2024-04-25T09:54:22.629650+0000","last_clean":"2024-04-25T09:54:22.629650+0000","last_became_active":"2024-04-25T09:54:22.619573+0000","last_became_peered":"2024-04-25T09:54:22.619573+0000","last_unstale":"2024-04-25T09:54:22.629650+0000","last_undegraded":"2024-04-25T09:54:22.629650+0000","last_fullsized":"2024-04-25T09:54:22.629650+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:23.041393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.618986+0000","last_change":"2024-04-25T09:54:21.628678+0000","last_active":"2024-04-25T09:54:22.618986+0000","last_peered":"2024-04-25T09:54:22.618986+0000","last_clean":"2024-04-25T09:54:22.618986+0000","last_became_active":"2024-04-25T09:54:21.628420+0000","last_became_peered":"2024-04-25T09:54:21.628420+0000","last_unstale":"2024-04-25T09:54:22.618986+0000","last_undegraded":"2024-04-25T09:54:22.618986+0000","last_fullsized":"2024-04-25T09:54:22.618986+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:16.447999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.643131+0000","last_change":"2024-04-25T09:54:22.639329+0000","last_active":"2024-04-25T09:54:22.643131+0000","last_peered":"2024-04-25T09:54:22.643131+0000","last_clean":"2024-04-25T09:54:22.643131+0000","last_became_active":"2024-04-25T09:54:22.639067+0000","last_became_peered":"2024-04-25T09:54:22.639067+0000","last_unstale":"2024-04-25T09:54:22.643131+0000","last_undegraded":"2024-04-25T09:54:22.643131+0000","last_fullsized":"2024-04-25T09:54:22.643131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:38.117240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641388+0000","last_change":"2024-04-25T09:54:21.619666+0000","last_active":"2024-04-25T09:54:22.641388+0000","last_peered":"2024-04-25T09:54:22.641388+0000","last_clean":"2024-04-25T09:54:22.641388+0000","last_became_active":"2024-04-25T09:54:21.619399+0000","last_became_peered":"2024-04-25T09:54:21.619399+0000","last_unstale":"2024-04-25T09:54:22.641388+0000","last_undegraded":"2024-04-25T09:54:22.641388+0000","last_fullsized":"2024-04-25T09:54:22.641388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:06.957591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640967+0000","last_change":"2024-04-25T09:54:22.635242+0000","last_active":"2024-04-25T09:54:22.640967+0000","last_peered":"2024-04-25T09:54:22.640967+0000","last_clean":"2024-04-25T09:54:22.640967+0000","last_became_active":"2024-04-25T09:54:22.634885+0000","last_became_peered":"2024-04-25T09:54:22.634885+0000","last_unstale":"2024-04-25T09:54:22.640967+0000","last_undegraded":"2024-04-25T09:54:22.640967+0000","last_fullsized":"2024-04-25T09:54:22.640967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:44.049423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.612299+0000","last_change":"2024-04-25T09:54:21.625380+0000","last_active":"2024-04-25T09:54:22.612299+0000","last_peered":"2024-04-25T09:54:22.612299+0000","last_clean":"2024-04-25T09:54:22.612299+0000","last_became_active":"2024-04-25T09:54:21.625128+0000","last_became_peered":"2024-04-25T09:54:21.625128+0000","last_unstale":"2024-04-25T09:54:22.612299+0000","last_undegraded":"2024-04-25T09:54:22.612299+0000","last_fullsized":"2024-04-25T09:54:22.612299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:20.524444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.639251+0000","last_change":"2024-04-25T09:54:22.630647+0000","last_active":"2024-04-25T09:54:22.639251+0000","last_peered":"2024-04-25T09:54:22.639251+0000","last_clean":"2024-04-25T09:54:22.639251+0000","last_became_active":"2024-04-25T09:54:22.630329+0000","last_became_peered":"2024-04-25T09:54:22.630329+0000","last_unstale":"2024-04-25T09:54:22.639251+0000","last_undegraded":"2024-04-25T09:54:22.639251+0000","last_fullsized":"2024-04-25T09:54:22.639251+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:31.082648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.624865+0000","last_change":"2024-04-25T09:54:21.630293+0000","last_active":"2024-04-25T09:54:22.624865+0000","last_peered":"2024-04-25T09:54:22.624865+0000","last_clean":"2024-04-25T09:54:22.624865+0000","last_became_active":"2024-04-25T09:54:21.630005+0000","last_became_peered":"2024-04-25T09:54:21.630005+0000","last_unstale":"2024-04-25T09:54:22.624865+0000","last_undegraded":"2024-04-25T09:54:22.624865+0000","last_fullsized":"2024-04-25T09:54:22.624865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:41.164745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640930+0000","last_change":"2024-04-25T09:54:22.632669+0000","last_active":"2024-04-25T09:54:22.640930+0000","last_peered":"2024-04-25T09:54:22.640930+0000","last_clean":"2024-04-25T09:54:22.640930+0000","last_became_active":"2024-04-25T09:54:22.632254+0000","last_became_peered":"2024-04-25T09:54:22.632254+0000","last_unstale":"2024-04-25T09:54:22.640930+0000","last_undegraded":"2024-04-25T09:54:22.640930+0000","last_fullsized":"2024-04-25T09:54:22.640930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:37.216863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.612533+0000","last_change":"2024-04-25T09:54:21.619855+0000","last_active":"2024-04-25T09:54:22.612533+0000","last_peered":"2024-04-25T09:54:22.612533+0000","last_clean":"2024-04-25T09:54:22.612533+0000","last_became_active":"2024-04-25T09:54:21.619608+0000","last_became_peered":"2024-04-25T09:54:21.619608+0000","last_unstale":"2024-04-25T09:54:22.612533+0000","last_undegraded":"2024-04-25T09:54:22.612533+0000","last_fullsized":"2024-04-25T09:54:22.612533+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:57.986604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640950+0000","last_change":"2024-04-25T09:54:22.636809+0000","last_active":"2024-04-25T09:54:22.640950+0000","last_peered":"2024-04-25T09:54:22.640950+0000","last_clean":"2024-04-25T09:54:22.640950+0000","last_became_active":"2024-04-25T09:54:22.635858+0000","last_became_peered":"2024-04-25T09:54:22.635858+0000","last_unstale":"2024-04-25T09:54:22.640950+0000","last_undegraded":"2024-04-25T09:54:22.640950+0000","last_fullsized":"2024-04-25T09:54:22.640950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:24.988688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.642788+0000","last_change":"2024-04-25T09:54:21.624009+0000","last_active":"2024-04-25T09:54:22.642788+0000","last_peered":"2024-04-25T09:54:22.642788+0000","last_clean":"2024-04-25T09:54:22.642788+0000","last_became_active":"2024-04-25T09:54:21.623706+0000","last_became_peered":"2024-04-25T09:54:21.623706+0000","last_unstale":"2024-04-25T09:54:22.642788+0000","last_undegraded":"2024-04-25T09:54:22.642788+0000","last_fullsized":"2024-04-25T09:54:22.642788+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:59.343752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.643988+0000","last_change":"2024-04-25T09:54:22.640133+0000","last_active":"2024-04-25T09:54:22.643988+0000","last_peered":"2024-04-25T09:54:22.643988+0000","last_clean":"2024-04-25T09:54:22.643988+0000","last_became_active":"2024-04-25T09:54:22.639770+0000","last_became_peered":"2024-04-25T09:54:22.639770+0000","last_unstale":"2024-04-25T09:54:22.643988+0000","last_undegraded":"2024-04-25T09:54:22.643988+0000","last_fullsized":"2024-04-25T09:54:22.643988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:31.082648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.612651+0000","last_change":"2024-04-25T09:54:21.629092+0000","last_active":"2024-04-25T09:54:22.612651+0000","last_peered":"2024-04-25T09:54:22.612651+0000","last_clean":"2024-04-25T09:54:22.612651+0000","last_became_active":"2024-04-25T09:54:21.628754+0000","last_became_peered":"2024-04-25T09:54:21.628754+0000","last_unstale":"2024-04-25T09:54:22.612651+0000","last_undegraded":"2024-04-25T09:54:22.612651+0000","last_fullsized":"2024-04-25T09:54:22.612651+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:56.099271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640925+0000","last_change":"2024-04-25T09:54:22.631681+0000","last_active":"2024-04-25T09:54:22.640925+0000","last_peered":"2024-04-25T09:54:22.640925+0000","last_clean":"2024-04-25T09:54:22.640925+0000","last_became_active":"2024-04-25T09:54:22.631434+0000","last_became_peered":"2024-04-25T09:54:22.631434+0000","last_unstale":"2024-04-25T09:54:22.640925+0000","last_undegraded":"2024-04-25T09:54:22.640925+0000","last_fullsized":"2024-04-25T09:54:22.640925+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:32.041828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.618902+0000","last_change":"2024-04-25T09:54:21.630777+0000","last_active":"2024-04-25T09:54:22.618902+0000","last_peered":"2024-04-25T09:54:22.618902+0000","last_clean":"2024-04-25T09:54:22.618902+0000","last_became_active":"2024-04-25T09:54:21.630475+0000","last_became_peered":"2024-04-25T09:54:21.630475+0000","last_unstale":"2024-04-25T09:54:22.618902+0000","last_undegraded":"2024-04-25T09:54:22.618902+0000","last_fullsized":"2024-04-25T09:54:22.618902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:56.628951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640987+0000","last_change":"2024-04-25T09:54:22.633510+0000","last_active":"2024-04-25T09:54:22.640987+0000","last_peered":"2024-04-25T09:54:22.640987+0000","last_clean":"2024-04-25T09:54:22.640987+0000","last_became_active":"2024-04-25T09:54:22.633100+0000","last_became_peered":"2024-04-25T09:54:22.633100+0000","last_unstale":"2024-04-25T09:54:22.640987+0000","last_undegraded":"2024-04-25T09:54:22.640987+0000","last_fullsized":"2024-04-25T09:54:22.640987+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:55.935357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.620435+0000","last_change":"2024-04-25T09:54:21.623769+0000","last_active":"2024-04-25T09:54:22.620435+0000","last_peered":"2024-04-25T09:54:22.620435+0000","last_clean":"2024-04-25T09:54:22.620435+0000","last_became_active":"2024-04-25T09:54:21.623500+0000","last_became_peered":"2024-04-25T09:54:21.623500+0000","last_unstale":"2024-04-25T09:54:22.620435+0000","last_undegraded":"2024-04-25T09:54:22.620435+0000","last_fullsized":"2024-04-25T09:54:22.620435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:01.015027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.641098+0000","last_change":"2024-04-25T09:54:22.631121+0000","last_active":"2024-04-25T09:54:22.641098+0000","last_peered":"2024-04-25T09:54:22.641098+0000","last_clean":"2024-04-25T09:54:22.641098+0000","last_became_active":"2024-04-25T09:54:22.630838+0000","last_became_peered":"2024-04-25T09:54:22.630838+0000","last_unstale":"2024-04-25T09:54:22.641098+0000","last_undegraded":"2024-04-25T09:54:22.641098+0000","last_fullsized":"2024-04-25T09:54:22.641098+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:42.167481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.618668+0000","last_change":"2024-04-25T09:54:21.623588+0000","last_active":"2024-04-25T09:54:22.618668+0000","last_peered":"2024-04-25T09:54:22.618668+0000","last_clean":"2024-04-25T09:54:22.618668+0000","last_became_active":"2024-04-25T09:54:21.623245+0000","last_became_peered":"2024-04-25T09:54:21.623245+0000","last_unstale":"2024-04-25T09:54:22.618668+0000","last_undegraded":"2024-04-25T09:54:22.618668+0000","last_fullsized":"2024-04-25T09:54:22.618668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:56.285717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.628318+0000","last_change":"2024-04-25T09:54:22.620418+0000","last_active":"2024-04-25T09:54:22.628318+0000","last_peered":"2024-04-25T09:54:22.628318+0000","last_clean":"2024-04-25T09:54:22.628318+0000","last_became_active":"2024-04-25T09:54:22.620152+0000","last_became_peered":"2024-04-25T09:54:22.620152+0000","last_unstale":"2024-04-25T09:54:22.628318+0000","last_undegraded":"2024-04-25T09:54:22.628318+0000","last_fullsized":"2024-04-25T09:54:22.628318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:02.017763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.615140+0000","last_change":"2024-04-25T09:54:21.622933+0000","last_active":"2024-04-25T09:54:22.615140+0000","last_peered":"2024-04-25T09:54:22.615140+0000","last_clean":"2024-04-25T09:54:22.615140+0000","last_became_active":"2024-04-25T09:54:21.622621+0000","last_became_peered":"2024-04-25T09:54:21.622621+0000","last_unstale":"2024-04-25T09:54:22.615140+0000","last_undegraded":"2024-04-25T09:54:22.615140+0000","last_fullsized":"2024-04-25T09:54:22.615140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:30.927191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.638043+0000","last_change":"2024-04-25T09:54:22.634393+0000","last_active":"2024-04-25T09:54:22.638043+0000","last_peered":"2024-04-25T09:54:22.638043+0000","last_clean":"2024-04-25T09:54:22.638043+0000","last_became_active":"2024-04-25T09:54:22.634059+0000","last_became_peered":"2024-04-25T09:54:22.634059+0000","last_unstale":"2024-04-25T09:54:22.638043+0000","last_undegraded":"2024-04-25T09:54:22.638043+0000","last_fullsized":"2024-04-25T09:54:22.638043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:43.969110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.613368+0000","last_change":"2024-04-25T09:54:21.623616+0000","last_active":"2024-04-25T09:54:22.613368+0000","last_peered":"2024-04-25T09:54:22.613368+0000","last_clean":"2024-04-25T09:54:22.613368+0000","last_became_active":"2024-04-25T09:54:21.623356+0000","last_became_peered":"2024-04-25T09:54:21.623356+0000","last_unstale":"2024-04-25T09:54:22.613368+0000","last_undegraded":"2024-04-25T09:54:22.613368+0000","last_fullsized":"2024-04-25T09:54:22.613368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:37.114504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.640166+0000","last_change":"2024-04-25T09:54:22.635983+0000","last_active":"2024-04-25T09:54:22.640166+0000","last_peered":"2024-04-25T09:54:22.640166+0000","last_clean":"2024-04-25T09:54:22.640166+0000","last_became_active":"2024-04-25T09:54:22.635313+0000","last_became_peered":"2024-04-25T09:54:22.635313+0000","last_unstale":"2024-04-25T09:54:22.640166+0000","last_undegraded":"2024-04-25T09:54:22.640166+0000","last_fullsized":"2024-04-25T09:54:22.640166+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.600062+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.600062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:48.853059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.626038+0000","last_change":"2024-04-25T09:54:21.630922+0000","last_active":"2024-04-25T09:54:22.626038+0000","last_peered":"2024-04-25T09:54:22.626038+0000","last_clean":"2024-04-25T09:54:22.626038+0000","last_became_active":"2024-04-25T09:54:21.630617+0000","last_became_peered":"2024-04-25T09:54:21.630617+0000","last_unstale":"2024-04-25T09:54:22.626038+0000","last_undegraded":"2024-04-25T09:54:22.626038+0000","last_fullsized":"2024-04-25T09:54:22.626038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:20.597326+0000","last_clean_scrub_stamp":"2024-04-25T09:54:20.597326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:43.644236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":200,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T09:54:22.625072+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:54:22.625072+0000","last_peered":"2024-04-25T09:54:22.625072+0000","last_clean":"2024-04-25T09:54:22.625072+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:54:22.625072+0000","last_undegraded":"2024-04-25T09:54:22.625072+0000","last_fullsized":"2024-04-25T09:54:22.625072+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":430080,"data_stored":183406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":430080,"data_stored":183426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:54:25.925 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:54:25.926 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:54:26.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.068+0000 7f083bd3b640 1 Processor -- start 2024-04-25T09:54:26.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.068+0000 7f083bd3b640 1 -- start start 2024-04-25T09:54:26.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f083bd3b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f083415dbf0 0x7f08341617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:26.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f083bd3b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0834161dc0 con 0x7f08340cee50 2024-04-25T09:54:26.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f083bd3b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08341620a0 con 0x7f08340b9370 2024-04-25T09:54:26.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f083bd3b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08340d1ee0 con 0x7f083415dbf0 2024-04-25T09:54:26.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f083a2d8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f08340cee50 0x7f08340d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54988/0 (socket says 172.21.15.139:54988) 2024-04-25T09:54:26.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f083a2d8640 1 -- 172.21.15.139:0/1014721296 learned_addr learned my addr 172.21.15.139:0/1014721296 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:26.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f0839ad7640 1 --2- 172.21.15.139:0/1014721296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f083415dbf0 0x7f08341617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:26.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f0838ad5640 1 -- 172.21.15.139:0/1014721296 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197556399 0 0) 0x7f0834161dc0 con 0x7f08340cee50 2024-04-25T09:54:26.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f0838ad5640 1 -- 172.21.15.139:0/1014721296 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0820003520 con 0x7f08340cee50 2024-04-25T09:54:26.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f0838ad5640 1 -- 172.21.15.139:0/1014721296 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1083566152 0 0) 0x7f0820003520 con 0x7f08340cee50 2024-04-25T09:54:26.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f0839ad7640 1 -- 172.21.15.139:0/1014721296 >> v1:172.21.15.150:6789/0 conn(0x7f08340b9370 legacy=0x7f08340ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f0839ad7640 1 -- 172.21.15.139:0/1014721296 >> v1:172.21.15.139:6789/0 conn(0x7f08340cee50 legacy=0x7f08340d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.069+0000 7f0839ad7640 1 -- 172.21.15.139:0/1014721296 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0834162380 con 0x7f083415dbf0 2024-04-25T09:54:26.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.070+0000 7f0839ad7640 1 --2- 172.21.15.139:0/1014721296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f083415dbf0 0x7f08341617e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08300097d0 tx=0x7f083002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bf802ccff21e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:26.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.070+0000 7f0838ad5640 1 -- 172.21.15.139:0/1014721296 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830037070 con 0x7f083415dbf0 2024-04-25T09:54:26.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.070+0000 7f0838ad5640 1 -- 172.21.15.139:0/1014721296 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083002ac30 con 0x7f083415dbf0 2024-04-25T09:54:26.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.070+0000 7f0838ad5640 1 -- 172.21.15.139:0/1014721296 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c0019f0 con 0x7f083415dbf0 2024-04-25T09:54:26.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.071+0000 7f083bd3b640 1 -- 172.21.15.139:0/1014721296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f083415dbf0 msgr2=0x7f08341617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.071+0000 7f083bd3b640 1 --2- 172.21.15.139:0/1014721296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f083415dbf0 0x7f08341617e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08300097d0 tx=0x7f083002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.071+0000 7f083bd3b640 1 -- 172.21.15.139:0/1014721296 shutdown_connections 2024-04-25T09:54:26.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.071+0000 7f083bd3b640 1 --2- 172.21.15.139:0/1014721296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f083415dbf0 0x7f08341617e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.071+0000 7f083bd3b640 1 -- 172.21.15.139:0/1014721296 >> 172.21.15.139:0/1014721296 conn(0x7f08340b5480 msgr2=0x7f08340b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:26.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.071+0000 7f083bd3b640 1 -- 172.21.15.139:0/1014721296 shutdown_connections 2024-04-25T09:54:26.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.071+0000 7f083bd3b640 1 -- 172.21.15.139:0/1014721296 wait complete. 2024-04-25T09:54:26.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.072+0000 7f083bd3b640 1 Processor -- start 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.072+0000 7f083bd3b640 1 -- start start 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f083bd3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08340b9370 0x7f083415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f083bd3b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08340cee50 0x7f08340c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f083bd3b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f083415dbf0 0x7f08340cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f083bd3b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083415dfd0 con 0x7f08340b9370 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f083bd3b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08340b97d0 con 0x7f083415dbf0 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f083bd3b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08341542f0 con 0x7f08340cee50 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f0839ad7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08340b9370 0x7f083415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f0839ad7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08340b9370 0x7f083415e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42234/0 (socket says 172.21.15.139:42234) 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f0839ad7640 1 -- 172.21.15.139:0/4186261366 learned_addr learned my addr 172.21.15.139:0/4186261366 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f08392d6640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08340cee50 0x7f08340c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f083a2d8640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f083415dbf0 0x7f08340cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f0839ad7640 1 -- 172.21.15.139:0/4186261366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08340cee50 msgr2=0x7f08340c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f0839ad7640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08340cee50 0x7f08340c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f0839ad7640 1 -- 172.21.15.139:0/4186261366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f083415dbf0 msgr2=0x7f08340cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f0839ad7640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f083415dbf0 0x7f08340cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f0839ad7640 1 -- 172.21.15.139:0/4186261366 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08341545d0 con 0x7f08340b9370 2024-04-25T09:54:26.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f08392d6640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08340cee50 0x7f08340c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:26.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f0839ad7640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08340b9370 0x7f083415e2e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f08300097d0 tx=0x7f08300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:26.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f082a7fc640 1 -- 172.21.15.139:0/4186261366 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830042020 con 0x7f08340b9370 2024-04-25T09:54:26.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f082a7fc640 1 -- 172.21.15.139:0/4186261366 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083003d670 con 0x7f08340b9370 2024-04-25T09:54:26.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f083bd3b640 1 -- 172.21.15.139:0/4186261366 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08341548b0 con 0x7f08340b9370 2024-04-25T09:54:26.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f082a7fc640 1 -- 172.21.15.139:0/4186261366 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830037040 con 0x7f08340b9370 2024-04-25T09:54:26.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.073+0000 7f083bd3b640 1 -- 172.21.15.139:0/4186261366 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0834151eb0 con 0x7f08340b9370 2024-04-25T09:54:26.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.074+0000 7f083bd3b640 1 -- 172.21.15.139:0/4186261366 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0834074520 con 0x7f08340b9370 2024-04-25T09:54:26.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.075+0000 7f082a7fc640 1 -- 172.21.15.139:0/4186261366 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0830045020 con 0x7f08340b9370 2024-04-25T09:54:26.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.075+0000 7f082a7fc640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f081806b4d0 0x7f081806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:26.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.075+0000 7f082a7fc640 1 -- 172.21.15.139:0/4186261366 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f08300aad10 con 0x7f08340b9370 2024-04-25T09:54:26.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.075+0000 7f08392d6640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f081806b4d0 0x7f081806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:26.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.076+0000 7f08392d6640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f081806b4d0 0x7f081806d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0824007fd0 tx=0x7f0824009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:26.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.079+0000 7f082a7fc640 1 -- 172.21.15.139:0/4186261366 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f083004c030 con 0x7f08340b9370 2024-04-25T09:54:26.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.313+0000 7f083bd3b640 1 -- 172.21.15.139:0/4186261366 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f08340d1680 con 0x7f08340b9370 2024-04-25T09:54:26.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.315+0000 7f082a7fc640 1 -- 172.21.15.139:0/4186261366 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0830079a00 con 0x7f08340b9370 2024-04-25T09:54:26.317 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2024-04-25T09:54:26.317 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg== 2024-04-25T09:54:26.317 INFO:teuthology.orchestra.run.smithi139.stdout: caps mds = "allow" 2024-04-25T09:54:26.317 INFO:teuthology.orchestra.run.smithi139.stdout: caps mon = "allow r" 2024-04-25T09:54:26.317 INFO:teuthology.orchestra.run.smithi139.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:54:26.318 INFO:teuthology.orchestra.run.smithi139.stderr:updated caps for client.0 2024-04-25T09:54:26.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.317+0000 7f083bd3b640 1 -- 172.21.15.139:0/4186261366 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f081806b4d0 msgr2=0x7f081806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.317+0000 7f083bd3b640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f081806b4d0 0x7f081806d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0824007fd0 tx=0x7f0824009040 comp rx=0 tx=0).stop 2024-04-25T09:54:26.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.317+0000 7f083bd3b640 1 -- 172.21.15.139:0/4186261366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08340b9370 msgr2=0x7f083415e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.317+0000 7f083bd3b640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08340b9370 0x7f083415e2e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f08300097d0 tx=0x7f08300046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.317+0000 7f083bd3b640 1 -- 172.21.15.139:0/4186261366 shutdown_connections 2024-04-25T09:54:26.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.317+0000 7f083bd3b640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f081806b4d0 0x7f081806d990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.317+0000 7f083bd3b640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f083415dbf0 0x7f08340cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.317+0000 7f083bd3b640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08340cee50 0x7f08340c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.317+0000 7f083bd3b640 1 --2- 172.21.15.139:0/4186261366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08340b9370 0x7f083415e2e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.317+0000 7f083bd3b640 1 -- 172.21.15.139:0/4186261366 >> 172.21.15.139:0/4186261366 conn(0x7f08340b5480 msgr2=0x7f08340b77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:26.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.317+0000 7f083bd3b640 1 -- 172.21.15.139:0/4186261366 shutdown_connections 2024-04-25T09:54:26.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.317+0000 7f083bd3b640 1 -- 172.21.15.139:0/4186261366 wait complete. 2024-04-25T09:54:26.330 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:26.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.425+0000 7fb540f70640 1 Processor -- start 2024-04-25T09:54:26.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.425+0000 7fb540f70640 1 -- start start 2024-04-25T09:54:26.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb540f70640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c155e30 0x7fb53c15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:26.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb540f70640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb53c0b1e50 0x7fb53c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb540f70640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb53c15c880 con 0x7fb53c15bdc0 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb540f70640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53c06d6f0 con 0x7fb53c0b1e50 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb540f70640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c15cb60 con 0x7fb53c155e30 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb53a59c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c155e30 0x7fb53c15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb53ad9d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb53c0b1e50 0x7fb53c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb53a59c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c155e30 0x7fb53c15cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:39564/0 (socket says 172.21.15.139:39564) 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb53a59c640 1 -- 172.21.15.139:0/2210259276 learned_addr learned my addr 172.21.15.139:0/2210259276 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb539d9b640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fb53c15bdc0 0x7fb53c0b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:55554/0 (socket says 172.21.15.139:55554) 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb53a59c640 1 -- 172.21.15.139:0/2210259276 >> v2:172.21.15.150:3300/0 conn(0x7fb53c0b1e50 msgr2=0x7fb53c15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb53a59c640 1 --2- 172.21.15.139:0/2210259276 >> v2:172.21.15.150:3300/0 conn(0x7fb53c0b1e50 0x7fb53c15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb53a59c640 1 -- 172.21.15.139:0/2210259276 >> v1:172.21.15.150:6789/0 conn(0x7fb53c15bdc0 legacy=0x7fb53c0b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb53a59c640 1 -- 172.21.15.139:0/2210259276 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c06d380 con 0x7fb53c155e30 2024-04-25T09:54:26.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.426+0000 7fb53ad9d640 1 --2- 172.21.15.139:0/2210259276 >> v2:172.21.15.150:3300/0 conn(0x7fb53c0b1e50 0x7fb53c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:26.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.427+0000 7fb53a59c640 1 --2- 172.21.15.139:0/2210259276 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c155e30 0x7fb53c15cf80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009870 tx=0x7fb52c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95199ba4340e8f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:26.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.427+0000 7fb53959a640 1 -- 172.21.15.139:0/2210259276 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c037070 con 0x7fb53c155e30 2024-04-25T09:54:26.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.427+0000 7fb53959a640 1 -- 172.21.15.139:0/2210259276 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52c032440 con 0x7fb53c155e30 2024-04-25T09:54:26.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.427+0000 7fb53959a640 1 -- 172.21.15.139:0/2210259276 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c032810 con 0x7fb53c155e30 2024-04-25T09:54:26.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.428+0000 7fb540f70640 1 -- 172.21.15.139:0/2210259276 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c155e30 msgr2=0x7fb53c15cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.428+0000 7fb540f70640 1 --2- 172.21.15.139:0/2210259276 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c155e30 0x7fb53c15cf80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009870 tx=0x7fb52c02a270 comp rx=0 tx=0).stop 2024-04-25T09:54:26.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.428+0000 7fb540f70640 1 -- 172.21.15.139:0/2210259276 shutdown_connections 2024-04-25T09:54:26.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.428+0000 7fb540f70640 1 --2- 172.21.15.139:0/2210259276 >> v2:172.21.15.150:3300/0 conn(0x7fb53c0b1e50 0x7fb53c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.428+0000 7fb540f70640 1 --2- 172.21.15.139:0/2210259276 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c155e30 0x7fb53c15cf80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.428+0000 7fb540f70640 1 -- 172.21.15.139:0/2210259276 >> 172.21.15.139:0/2210259276 conn(0x7fb53c066f20 msgr2=0x7fb53c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:26.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.428+0000 7fb540f70640 1 -- 172.21.15.139:0/2210259276 shutdown_connections 2024-04-25T09:54:26.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.428+0000 7fb540f70640 1 -- 172.21.15.139:0/2210259276 wait complete. 2024-04-25T09:54:26.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.429+0000 7fb540f70640 1 Processor -- start 2024-04-25T09:54:26.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.429+0000 7fb540f70640 1 -- start start 2024-04-25T09:54:26.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb540f70640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c0b1e50 0x7fb53c1f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:26.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb540f70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53c155e30 0x7fb53c15a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:26.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb540f70640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb53c15bdc0 0x7fb53c15add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:26.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb540f70640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0af260 con 0x7fb53c155e30 2024-04-25T09:54:26.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb540f70640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0af0e0 con 0x7fb53c15bdc0 2024-04-25T09:54:26.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb540f70640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c15d4c0 con 0x7fb53c0b1e50 2024-04-25T09:54:26.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb539d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53c155e30 0x7fb53c15a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:26.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb539d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53c155e30 0x7fb53c15a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41042/0 (socket says 172.21.15.139:41042) 2024-04-25T09:54:26.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb539d9b640 1 -- 172.21.15.139:0/187483090 learned_addr learned my addr 172.21.15.139:0/187483090 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb53a59c640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c0b1e50 0x7fb53c1f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb53ad9d640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb53c15bdc0 0x7fb53c15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb53a59c640 1 -- 172.21.15.139:0/187483090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb53c15bdc0 msgr2=0x7fb53c15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb53a59c640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb53c15bdc0 0x7fb53c15add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb53a59c640 1 -- 172.21.15.139:0/187483090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53c155e30 msgr2=0x7fb53c15a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb53a59c640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53c155e30 0x7fb53c15a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb53a59c640 1 -- 172.21.15.139:0/187483090 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c0af5d0 con 0x7fb53c0b1e50 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb53ad9d640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb53c15bdc0 0x7fb53c15add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.430+0000 7fb539d9b640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53c155e30 0x7fb53c15a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.431+0000 7fb53a59c640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c0b1e50 0x7fb53c1f3570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb52c031000 tx=0x7fb52c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.431+0000 7fb52affd640 1 -- 172.21.15.139:0/187483090 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c041020 con 0x7fb53c0b1e50 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.431+0000 7fb52affd640 1 -- 172.21.15.139:0/187483090 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52c030640 con 0x7fb53c0b1e50 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.431+0000 7fb52affd640 1 -- 172.21.15.139:0/187483090 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c037040 con 0x7fb53c0b1e50 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.431+0000 7fb540f70640 1 -- 172.21.15.139:0/187483090 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53c1f7fb0 con 0x7fb53c0b1e50 2024-04-25T09:54:26.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.431+0000 7fb540f70640 1 -- 172.21.15.139:0/187483090 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53c0afb50 con 0x7fb53c0b1e50 2024-04-25T09:54:26.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.432+0000 7fb540f70640 1 -- 172.21.15.139:0/187483090 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53c074520 con 0x7fb53c0b1e50 2024-04-25T09:54:26.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.432+0000 7fb52affd640 1 -- 172.21.15.139:0/187483090 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb52c044020 con 0x7fb53c0b1e50 2024-04-25T09:54:26.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.433+0000 7fb52affd640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb51c06b4d0 0x7fb51c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:26.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.433+0000 7fb52affd640 1 -- 172.21.15.139:0/187483090 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb52c0aa980 con 0x7fb53c0b1e50 2024-04-25T09:54:26.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.433+0000 7fb539d9b640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb51c06b4d0 0x7fb51c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:26.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.433+0000 7fb539d9b640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb51c06b4d0 0x7fb51c06d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb53c1f70c0 tx=0x7fb530007f10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:26.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.437+0000 7fb52affd640 1 -- 172.21.15.139:0/187483090 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb52c04b030 con 0x7fb53c0b1e50 2024-04-25T09:54:26.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.665+0000 7fb540f70640 1 -- 172.21.15.139:0/187483090 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb53c06b830 con 0x7fb53c0b1e50 2024-04-25T09:54:26.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.666+0000 7fb52affd640 1 -- 172.21.15.139:0/187483090 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fb52c079590 con 0x7fb53c0b1e50 2024-04-25T09:54:26.668 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:26.668 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/511562594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":511562594},{"type":"v1","addr":"172.21.15.150:6834","nonce":511562594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5041,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/225357317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":225357317},{"type":"v1","addr":"172.21.15.150:6835","nonce":225357317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2499407982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2499407982},{"type":"v1","addr":"172.21.15.139:6836","nonce":2499407982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:21.988975+0000","modified":"2024-04-25T09:54:23.955620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/2069784668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2069784668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2069784668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:54:26.668 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 16 2024-04-25T09:54:26.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.668+0000 7fb540f70640 1 -- 172.21.15.139:0/187483090 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb51c06b4d0 msgr2=0x7fb51c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.668+0000 7fb540f70640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb51c06b4d0 0x7fb51c06d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb53c1f70c0 tx=0x7fb530007f10 comp rx=0 tx=0).stop 2024-04-25T09:54:26.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.668+0000 7fb540f70640 1 -- 172.21.15.139:0/187483090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c0b1e50 msgr2=0x7fb53c1f3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.668+0000 7fb540f70640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c0b1e50 0x7fb53c1f3570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb52c031000 tx=0x7fb52c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.668+0000 7fb540f70640 1 -- 172.21.15.139:0/187483090 shutdown_connections 2024-04-25T09:54:26.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.668+0000 7fb540f70640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb53c155e30 0x7fb53c15a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.668+0000 7fb540f70640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb51c06b4d0 0x7fb51c06d990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.668+0000 7fb540f70640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb53c15bdc0 0x7fb53c15add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.668+0000 7fb540f70640 1 --2- 172.21.15.139:0/187483090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb53c0b1e50 0x7fb53c1f3570 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.668+0000 7fb540f70640 1 -- 172.21.15.139:0/187483090 >> 172.21.15.139:0/187483090 conn(0x7fb53c066f20 msgr2=0x7fb53c154f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:26.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.668+0000 7fb540f70640 1 -- 172.21.15.139:0/187483090 shutdown_connections 2024-04-25T09:54:26.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:26.668+0000 7fb540f70640 1 -- 172.21.15.139:0/187483090 wait complete. 2024-04-25T09:54:27.047 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T09:54:27.045+0000 7effc2d26640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:54:27.085 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:54:21.988975+0000', 'modified': '2024-04-25T09:54:23.955620+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5418}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5418': {'gid': 5418, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6837/2069784668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6835', 'nonce': 2069784668}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 2069784668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:54:27.085 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:54:27.085 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:54:27.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.181+0000 7f529109e640 1 Processor -- start 2024-04-25T09:54:27.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.181+0000 7f529109e640 1 -- start start 2024-04-25T09:54:27.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f529109e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f528c15dbf0 0x7f528c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f529109e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f528c0b9370 0x7f528c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f529109e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528c161dc0 con 0x7f528c0cec80 2024-04-25T09:54:27.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f529109e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528c0d1ee0 con 0x7f528c15dbf0 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f529109e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c1620a0 con 0x7f528c0b9370 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f528ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f528c15dbf0 0x7f528c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f528ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f528c15dbf0 0x7f528c1617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41054/0 (socket says 172.21.15.139:41054) 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f528ad9d640 1 -- 172.21.15.139:0/584760696 learned_addr learned my addr 172.21.15.139:0/584760696 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f528ad9d640 1 -- 172.21.15.139:0/584760696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f528c0b9370 msgr2=0x7f528c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f5283fff640 1 --2- 172.21.15.139:0/584760696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f528c0b9370 0x7f528c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f528ad9d640 1 --2- 172.21.15.139:0/584760696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f528c0b9370 0x7f528c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f528ad9d640 1 -- 172.21.15.139:0/584760696 >> v1:172.21.15.150:6789/0 conn(0x7f528c0cec80 legacy=0x7f528c06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f528ad9d640 1 -- 172.21.15.139:0/584760696 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c162380 con 0x7f528c15dbf0 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.182+0000 7f5283fff640 1 --2- 172.21.15.139:0/584760696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f528c0b9370 0x7f528c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.183+0000 7f528ad9d640 1 --2- 172.21.15.139:0/584760696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f528c15dbf0 0x7f528c1617e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f527c009870 tx=0x7f527c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb58a3d203769e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.183+0000 7f528a59c640 1 -- 172.21.15.139:0/584760696 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c037070 con 0x7f528c15dbf0 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.183+0000 7f528a59c640 1 -- 172.21.15.139:0/584760696 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527c032440 con 0x7f528c15dbf0 2024-04-25T09:54:27.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.183+0000 7f528a59c640 1 -- 172.21.15.139:0/584760696 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c032810 con 0x7f528c15dbf0 2024-04-25T09:54:27.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.184+0000 7f529109e640 1 -- 172.21.15.139:0/584760696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f528c15dbf0 msgr2=0x7f528c1617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.184+0000 7f529109e640 1 --2- 172.21.15.139:0/584760696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f528c15dbf0 0x7f528c1617e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f527c009870 tx=0x7f527c02a210 comp rx=0 tx=0).stop 2024-04-25T09:54:27.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.184+0000 7f529109e640 1 -- 172.21.15.139:0/584760696 shutdown_connections 2024-04-25T09:54:27.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.184+0000 7f529109e640 1 --2- 172.21.15.139:0/584760696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f528c0b9370 0x7f528c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.184+0000 7f529109e640 1 --2- 172.21.15.139:0/584760696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f528c15dbf0 0x7f528c1617e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.184+0000 7f529109e640 1 -- 172.21.15.139:0/584760696 >> 172.21.15.139:0/584760696 conn(0x7f528c0b5480 msgr2=0x7f528c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:27.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.184+0000 7f529109e640 1 -- 172.21.15.139:0/584760696 shutdown_connections 2024-04-25T09:54:27.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.184+0000 7f529109e640 1 -- 172.21.15.139:0/584760696 wait complete. 2024-04-25T09:54:27.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.185+0000 7f529109e640 1 Processor -- start 2024-04-25T09:54:27.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.185+0000 7f529109e640 1 -- start start 2024-04-25T09:54:27.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f529109e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f528c0b9370 0x7f528c160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f529109e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f528c0cec80 0x7f528c153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f529109e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f528c15dbf0 0x7f528c1f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f529109e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c15dfd0 con 0x7f528c0cec80 2024-04-25T09:54:27.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f529109e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0b97d0 con 0x7f528c15dbf0 2024-04-25T09:54:27.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f529109e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c156e00 con 0x7f528c0b9370 2024-04-25T09:54:27.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f5283fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f528c0cec80 0x7f528c153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f5283fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f528c0cec80 0x7f528c153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41070/0 (socket says 172.21.15.139:41070) 2024-04-25T09:54:27.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f5283fff640 1 -- 172.21.15.139:0/612468121 learned_addr learned my addr 172.21.15.139:0/612468121 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:27.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f528ad9d640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f528c0b9370 0x7f528c160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f528b59e640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f528c15dbf0 0x7f528c1f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f5283fff640 1 -- 172.21.15.139:0/612468121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f528c0b9370 msgr2=0x7f528c160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f5283fff640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f528c0b9370 0x7f528c160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f5283fff640 1 -- 172.21.15.139:0/612468121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f528c15dbf0 msgr2=0x7f528c1f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f5283fff640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f528c15dbf0 0x7f528c1f3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f5283fff640 1 -- 172.21.15.139:0/612468121 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c1612f0 con 0x7f528c0cec80 2024-04-25T09:54:27.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f5283fff640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f528c0cec80 0x7f528c153a70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f527400c820 tx=0x7f527400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f52837fe640 1 -- 172.21.15.139:0/612468121 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274015070 con 0x7f528c0cec80 2024-04-25T09:54:27.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f52837fe640 1 -- 172.21.15.139:0/612468121 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5274007a30 con 0x7f528c0cec80 2024-04-25T09:54:27.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f52837fe640 1 -- 172.21.15.139:0/612468121 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274007de0 con 0x7f528c0cec80 2024-04-25T09:54:27.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.186+0000 7f529109e640 1 -- 172.21.15.139:0/612468121 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c156560 con 0x7f528c0cec80 2024-04-25T09:54:27.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.187+0000 7f529109e640 1 -- 172.21.15.139:0/612468121 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c156b20 con 0x7f528c0cec80 2024-04-25T09:54:27.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.188+0000 7f52837fe640 1 -- 172.21.15.139:0/612468121 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5274018070 con 0x7f528c0cec80 2024-04-25T09:54:27.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.188+0000 7f52837fe640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f526c06b4d0 0x7f526c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.189+0000 7f52837fe640 1 -- 172.21.15.139:0/612468121 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f527408a1a0 con 0x7f528c0cec80 2024-04-25T09:54:27.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.189+0000 7f528ad9d640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f526c06b4d0 0x7f526c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.189+0000 7f528ad9d640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f526c06b4d0 0x7f526c06d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f527c034000 tx=0x7f527c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.189+0000 7f5293d9f740 1 Processor -- start 2024-04-25T09:54:27.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.190+0000 7f5293d9f740 1 -- start start 2024-04-25T09:54:27.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.190+0000 7f5293d9f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e5045ec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.190+0000 7f5293d9f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e50460a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.190+0000 7f5293d9f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045c1f10 0x55e504608190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.190+0000 7f5293d9f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e504524a10 con 0x55e5045c1f10 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.190+0000 7f5293d9f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5044cdb30 con 0x55e504609ad0 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.190+0000 7f5293d9f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e504520c50 con 0x55e5045ec230 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.190+0000 7f528b59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045c1f10 0x55e504608190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.190+0000 7f528b59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045c1f10 0x55e504608190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41082/0 (socket says 172.21.15.139:41082) 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.190+0000 7f528b59e640 1 -- 172.21.15.139:0/2395838198 learned_addr learned my addr 172.21.15.139:0/2395838198 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.191+0000 7f528ad9d640 1 --2- 172.21.15.139:0/2395838198 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e5045ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.191+0000 7f5283fff640 1 --2- 172.21.15.139:0/2395838198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e50460a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.191+0000 7f528b59e640 1 -- 172.21.15.139:0/2395838198 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 msgr2=0x55e5045ec610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.191+0000 7f528b59e640 1 --2- 172.21.15.139:0/2395838198 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e5045ec610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.191+0000 7f528b59e640 1 -- 172.21.15.139:0/2395838198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 msgr2=0x55e50460a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.191+0000 7f528b59e640 1 --2- 172.21.15.139:0/2395838198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e50460a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.191+0000 7f528b59e640 1 -- 172.21.15.139:0/2395838198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5044e2e10 con 0x55e5045c1f10 2024-04-25T09:54:27.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.191+0000 7f528b59e640 1 --2- 172.21.15.139:0/2395838198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045c1f10 0x55e504608190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5284005680 tx=0x7f528400f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a75fbb805ea47fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.191+0000 7f5280ff9640 1 -- 172.21.15.139:0/2395838198 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284018020 con 0x55e5045c1f10 2024-04-25T09:54:27.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.191+0000 7f5280ff9640 1 -- 172.21.15.139:0/2395838198 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284014470 con 0x55e5045c1f10 2024-04-25T09:54:27.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.191+0000 7f5280ff9640 1 -- 172.21.15.139:0/2395838198 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284014820 con 0x55e5045c1f10 2024-04-25T09:54:27.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.192+0000 7f5293d9f740 1 -- 172.21.15.139:0/2395838198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045c1f10 msgr2=0x55e504608190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.192+0000 7f5293d9f740 1 --2- 172.21.15.139:0/2395838198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045c1f10 0x55e504608190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5284005680 tx=0x7f528400f5a0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.192+0000 7f5293d9f740 1 -- 172.21.15.139:0/2395838198 shutdown_connections 2024-04-25T09:54:27.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.192+0000 7f5293d9f740 1 --2- 172.21.15.139:0/2395838198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045c1f10 0x55e504608190 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.192+0000 7f5293d9f740 1 --2- 172.21.15.139:0/2395838198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e50460a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.192+0000 7f5293d9f740 1 --2- 172.21.15.139:0/2395838198 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e5045ec610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.192+0000 7f5293d9f740 1 -- 172.21.15.139:0/2395838198 >> 172.21.15.139:0/2395838198 conn(0x55e5045be840 msgr2=0x55e504607040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:27.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.192+0000 7f5293d9f740 1 -- 172.21.15.139:0/2395838198 shutdown_connections 2024-04-25T09:54:27.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.192+0000 7f5293d9f740 1 -- 172.21.15.139:0/2395838198 wait complete. 2024-04-25T09:54:27.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.192+0000 7f5293d9f740 1 -- start start 2024-04-25T09:54:27.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.192+0000 7f5293d9f740 1 Processor -- start 2024-04-25T09:54:27.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5293d9f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5045ec230 0x55e50461b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5293d9f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e504609ad0 0x55e5045ee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5293d9f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ee5c0 0x55e504605ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5293d9f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5045c2cb0 con 0x55e504609ad0 2024-04-25T09:54:27.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5293d9f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5044e10c0 con 0x55e5045ec230 2024-04-25T09:54:27.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5293d9f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5045bc770 con 0x55e5045ee5c0 2024-04-25T09:54:27.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5293d9f740 10 client.0 _ll_register_callbacks cb 0x55e5044d4ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:54:27.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5283fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e504609ad0 0x55e5045ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5283fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e504609ad0 0x55e5045ee080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41090/0 (socket says 172.21.15.139:41090) 2024-04-25T09:54:27.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5283fff640 1 -- 172.21.15.139:0/4184336041 learned_addr learned my addr 172.21.15.139:0/4184336041 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:27.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f528ad9d640 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5045ec230 0x55e50461b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f528b59e640 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ee5c0 0x55e504605ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5283fff640 1 -- 172.21.15.139:0/4184336041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ee5c0 msgr2=0x55e504605ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5283fff640 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ee5c0 0x55e504605ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5283fff640 1 -- 172.21.15.139:0/4184336041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5045ec230 msgr2=0x55e50461b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.193+0000 7f5283fff640 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5045ec230 0x55e50461b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5283fff640 1 -- 172.21.15.139:0/4184336041 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5042392d0 con 0x55e504609ad0 2024-04-25T09:54:27.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f528ad9d640 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5045ec230 0x55e50461b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:27.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5283fff640 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e504609ad0 0x55e5045ee080 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5274000c00 tx=0x7f5274058290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f937d3ef9b9ae526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5269ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:54:27.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5269ffb640 1 -- 172.21.15.139:0/4184336041 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527401f070 con 0x55e504609ad0 2024-04-25T09:54:27.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5269ffb640 1 -- 172.21.15.139:0/4184336041 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5274057c80 con 0x55e504609ad0 2024-04-25T09:54:27.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5293d9f740 1 -- 172.21.15.139:0/4184336041 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5284014470 con 0x55e504609ad0 2024-04-25T09:54:27.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5269ffb640 1 -- 172.21.15.139:0/4184336041 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f527400ce10 con 0x55e504609ad0 2024-04-25T09:54:27.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5293d9f740 10 client.5589 fetch_fsmap learned FSMap version 16 2024-04-25T09:54:27.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5293d9f740 1 -- 172.21.15.139:0/4184336041 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55e5043036b0 con 0x55e504609ad0 2024-04-25T09:54:27.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5269ffb640 1 -- 172.21.15.139:0/4184336041 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284014470 con 0x55e504609ad0 2024-04-25T09:54:27.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5269ffb640 1 -- 172.21.15.139:0/4184336041 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5274054580 con 0x55e504609ad0 2024-04-25T09:54:27.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5293d9f740 10 client.5589 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T09:54:27.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5293d9f740 10 client.5589 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:54:27.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5293d9f740 20 client.5589 populate_metadata read hostname 'smithi139' 2024-04-25T09:54:27.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5293d9f740 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.139:6835/2069784668,v1:172.21.15.139:6837/2069784668] conn(0x55e504619cc0 0x55e504617c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5293d9f740 4 client.5589 mds_command: new command op to 5418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:54:27.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f5293d9f740 1 -- 172.21.15.139:0/4184336041 --> [v2:172.21.15.139:6835/2069784668,v1:172.21.15.139:6837/2069784668] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e504583920 con 0x55e504619cc0 2024-04-25T09:54:27.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.194+0000 7f528ad9d640 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.139:6835/2069784668,v1:172.21.15.139:6837/2069784668] conn(0x55e504619cc0 0x55e504617c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5269ffb640 10 client.5589 ms_handle_connect on v2:172.21.15.139:6835/2069784668 2024-04-25T09:54:27.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f528ad9d640 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.139:6835/2069784668,v1:172.21.15.139:6837/2069784668] conn(0x55e504619cc0 0x55e504617c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40cda764781bd7f9 server_cookie=4a8ab12404bdf377 in_seq=0 out_seq=0 2024-04-25T09:54:27.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5269ffb640 1 -- 172.21.15.139:0/4184336041 <== mds.0 v2:172.21.15.139:6835/2069784668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e504583920 con 0x55e504619cc0 2024-04-25T09:54:27.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5269ffb640 10 client.5589 handle_command_reply: tid=0 2024-04-25T09:54:27.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5293d9f740 1 client.5589 shutdown 2024-04-25T09:54:27.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5293d9f740 1 -- 172.21.15.139:0/4184336041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e504609ad0 msgr2=0x55e5045ee080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5293d9f740 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e504609ad0 0x55e5045ee080 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5274000c00 tx=0x7f5274058290 comp rx=0 tx=0).stop 2024-04-25T09:54:27.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5293d9f740 1 -- 172.21.15.139:0/4184336041 shutdown_connections 2024-04-25T09:54:27.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5293d9f740 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.139:6835/2069784668,v1:172.21.15.139:6837/2069784668] conn(0x55e504619cc0 0x55e504617c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5293d9f740 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e504609ad0 0x55e5045ee080 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5269ffb640 0 client.5589 ms_handle_reset on v2:172.21.15.139:6835/2069784668 2024-04-25T09:54:27.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5293d9f740 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ee5c0 0x55e504605ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5293d9f740 1 --2- 172.21.15.139:0/4184336041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e5045ec230 0x55e50461b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5293d9f740 1 -- 172.21.15.139:0/4184336041 >> 172.21.15.139:0/4184336041 conn(0x55e5045c1f10 msgr2=0x55e5045eb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:27.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5293d9f740 1 -- 172.21.15.139:0/4184336041 shutdown_connections 2024-04-25T09:54:27.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5293d9f740 1 -- 172.21.15.139:0/4184336041 wait complete. 2024-04-25T09:54:27.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.195+0000 7f5293d9f740 20 client.5589 trim_cache size 0 max 16384 2024-04-25T09:54:27.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.211+0000 7f5293d9f740 1 Processor -- start 2024-04-25T09:54:27.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.211+0000 7f5293d9f740 1 -- start start 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5293d9f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e5046019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5293d9f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 0x55e504601fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5293d9f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e5045ed0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5293d9f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c1ff620 con 0x55e5045ee5c0 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5293d9f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c1e8db0 con 0x55e504609ad0 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5293d9f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c1e93b0 con 0x55e5045ec230 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5283fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 0x55e504601fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5283fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 0x55e504601fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41094/0 (socket says 172.21.15.139:41094) 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5283fff640 1 -- 172.21.15.139:0/3024888409 learned_addr learned my addr 172.21.15.139:0/3024888409 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f528ad9d640 1 --2- 172.21.15.139:0/3024888409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e5046019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f528b59e640 1 --2- 172.21.15.139:0/3024888409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e5045ed0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5283fff640 1 -- 172.21.15.139:0/3024888409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 msgr2=0x55e5046019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5283fff640 1 --2- 172.21.15.139:0/3024888409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e5046019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5283fff640 1 -- 172.21.15.139:0/3024888409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 msgr2=0x55e5045ed0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5283fff640 1 --2- 172.21.15.139:0/3024888409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e5045ed0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5283fff640 1 -- 172.21.15.139:0/3024888409 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e504606e40 con 0x55e5045ee5c0 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f528ad9d640 1 --2- 172.21.15.139:0/3024888409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e5046019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f528b59e640 1 --2- 172.21.15.139:0/3024888409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e5045ed0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f5283fff640 1 --2- 172.21.15.139:0/3024888409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 0x55e504601fa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5274054c40 tx=0x7f527405a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f35c044a711c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f526bfff640 1 -- 172.21.15.139:0/3024888409 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274070170 con 0x55e5045ee5c0 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f526bfff640 1 -- 172.21.15.139:0/3024888409 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5274057c80 con 0x55e5045ee5c0 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.212+0000 7f526bfff640 1 -- 172.21.15.139:0/3024888409 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274070310 con 0x55e5045ee5c0 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.213+0000 7f5293d9f740 1 -- 172.21.15.139:0/3024888409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 msgr2=0x55e504601fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.213+0000 7f5293d9f740 1 --2- 172.21.15.139:0/3024888409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 0x55e504601fa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5274054c40 tx=0x7f527405a000 comp rx=0 tx=0).stop 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.213+0000 7f5293d9f740 1 -- 172.21.15.139:0/3024888409 shutdown_connections 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.213+0000 7f5293d9f740 1 --2- 172.21.15.139:0/3024888409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 0x55e504601fa0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.213+0000 7f5293d9f740 1 --2- 172.21.15.139:0/3024888409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e5045ed0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.213+0000 7f5293d9f740 1 --2- 172.21.15.139:0/3024888409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e5046019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.213+0000 7f5293d9f740 1 -- 172.21.15.139:0/3024888409 >> 172.21.15.139:0/3024888409 conn(0x55e5045c1f10 msgr2=0x55e504605c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.213+0000 7f5293d9f740 1 -- 172.21.15.139:0/3024888409 shutdown_connections 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.213+0000 7f5293d9f740 1 -- 172.21.15.139:0/3024888409 wait complete. 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.213+0000 7f5293d9f740 1 -- start start 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.214+0000 7f5293d9f740 1 Processor -- start 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.214+0000 7f5293d9f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e504604e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.214+0000 7f5293d9f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 0x55e50460a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.214+0000 7f5293d9f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e5045bb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.214+0000 7f5293d9f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e504619b40 con 0x55e5045ee5c0 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.214+0000 7f5293d9f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e504619150 con 0x55e504609ad0 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.214+0000 7f5293d9f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e50461a140 con 0x55e5045ec230 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.214+0000 7f5293d9f740 10 client.0 _ll_register_callbacks cb 0x55e5045bcc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.214+0000 7f5283fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 0x55e50460a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.214+0000 7f5283fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 0x55e50460a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41104/0 (socket says 172.21.15.139:41104) 2024-04-25T09:54:27.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.214+0000 7f5283fff640 1 -- 172.21.15.139:0/1711074799 learned_addr learned my addr 172.21.15.139:0/1711074799 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.214+0000 7f528ad9d640 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e504604e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f528b59e640 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e5045bb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5283fff640 1 -- 172.21.15.139:0/1711074799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 msgr2=0x55e504604e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5283fff640 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e504604e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5283fff640 1 -- 172.21.15.139:0/1711074799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 msgr2=0x55e5045bb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5283fff640 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e5045bb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5283fff640 1 -- 172.21.15.139:0/1711074799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5046053a0 con 0x55e5045ee5c0 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f528b59e640 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e5045bb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5283fff640 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 0x55e50460a8e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f52740551b0 tx=0x7f5274088400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcbd50c5ef460c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5269ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5269ffb640 1 -- 172.21.15.139:0/1711074799 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274074020 con 0x55e5045ee5c0 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5269ffb640 1 -- 172.21.15.139:0/1711074799 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5274071d50 con 0x55e5045ee5c0 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5293d9f740 1 -- 172.21.15.139:0/1711074799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5274057c80 con 0x55e5045ee5c0 2024-04-25T09:54:27.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5269ffb640 1 -- 172.21.15.139:0/1711074799 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5274077300 con 0x55e5045ee5c0 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5293d9f740 10 client.5595 fetch_fsmap learned FSMap version 16 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5293d9f740 1 -- 172.21.15.139:0/1711074799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55e5046053a0 con 0x55e5045ee5c0 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5269ffb640 1 -- 172.21.15.139:0/1711074799 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5274057c80 con 0x55e5045ee5c0 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5269ffb640 1 -- 172.21.15.139:0/1711074799 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5274059ab0 con 0x55e5045ee5c0 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5293d9f740 10 client.5595 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5293d9f740 10 client.5595 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5293d9f740 20 client.5595 populate_metadata read hostname 'smithi139' 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5293d9f740 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.139:6835/2069784668,v1:172.21.15.139:6837/2069784668] conn(0x55e504605c70 0x55e504657b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5293d9f740 4 client.5595 mds_command: new command op to 5418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.215+0000 7f5293d9f740 1 -- 172.21.15.139:0/1711074799 --> [v2:172.21.15.139:6835/2069784668,v1:172.21.15.139:6837/2069784668] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e504531f50 con 0x55e504605c70 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f528ad9d640 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.139:6835/2069784668,v1:172.21.15.139:6837/2069784668] conn(0x55e504605c70 0x55e504657b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5269ffb640 10 client.5595 ms_handle_connect on v2:172.21.15.139:6835/2069784668 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f528ad9d640 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.139:6835/2069784668,v1:172.21.15.139:6837/2069784668] conn(0x55e504605c70 0x55e504657b90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3cc18aec9a429bd server_cookie=7912ad018def8b5e in_seq=0 out_seq=0 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5269ffb640 1 -- 172.21.15.139:0/1711074799 <== mds.0 v2:172.21.15.139:6835/2069784668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e504531f50 con 0x55e504605c70 2024-04-25T09:54:27.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5269ffb640 10 client.5595 handle_command_reply: tid=0 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5293d9f740 1 client.5595 shutdown 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5293d9f740 1 -- 172.21.15.139:0/1711074799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 msgr2=0x55e50460a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5293d9f740 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 0x55e50460a8e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f52740551b0 tx=0x7f5274088400 comp rx=0 tx=0).stop 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5293d9f740 1 -- 172.21.15.139:0/1711074799 shutdown_connections 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5293d9f740 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.139:6835/2069784668,v1:172.21.15.139:6837/2069784668] conn(0x55e504605c70 0x55e504657b90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5293d9f740 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e5045ee5c0 0x55e50460a8e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5293d9f740 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e504609ad0 0x55e5045bb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5293d9f740 1 --2- 172.21.15.139:0/1711074799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e5045ec230 0x55e504604e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5269ffb640 0 client.5595 ms_handle_reset on v2:172.21.15.139:6835/2069784668 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5293d9f740 1 -- 172.21.15.139:0/1711074799 >> 172.21.15.139:0/1711074799 conn(0x55e5045c1f10 msgr2=0x55e504618340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5293d9f740 1 -- 172.21.15.139:0/1711074799 shutdown_connections 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5293d9f740 1 -- 172.21.15.139:0/1711074799 wait complete. 2024-04-25T09:54:27.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.216+0000 7f5293d9f740 20 client.5595 trim_cache size 0 max 16384 2024-04-25T09:54:27.228 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:54:27.228 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T09:54:27.228 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T09:54:27.228 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 5418, 2024-04-25T09:54:27.228 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T09:54:27.228 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T09:54:27.228 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T09:54:27.228 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 4.6068734320000004, 2024-04-25T09:54:27.228 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 16, 2024-04-25T09:54:27.228 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 38, 2024-04-25T09:54:27.228 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 38, 2024-04-25T09:54:27.228 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 12.034194639000001 2024-04-25T09:54:27.229 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:54:27.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.217+0000 7f529109e640 1 -- 172.21.15.139:0/612468121 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f526c06b4d0 msgr2=0x7f526c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.217+0000 7f529109e640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f526c06b4d0 0x7f526c06d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f527c034000 tx=0x7f527c02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:27.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.217+0000 7f529109e640 1 -- 172.21.15.139:0/612468121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f528c0cec80 msgr2=0x7f528c153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.217+0000 7f529109e640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f528c0cec80 0x7f528c153a70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f527400c820 tx=0x7f527400cce0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.217+0000 7f529109e640 1 -- 172.21.15.139:0/612468121 shutdown_connections 2024-04-25T09:54:27.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.217+0000 7f529109e640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f528c0cec80 0x7f528c153a70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.217+0000 7f529109e640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f526c06b4d0 0x7f526c06d990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.217+0000 7f529109e640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f528c15dbf0 0x7f528c1f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.217+0000 7f529109e640 1 --2- 172.21.15.139:0/612468121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f528c0b9370 0x7f528c160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.217+0000 7f529109e640 1 -- 172.21.15.139:0/612468121 >> 172.21.15.139:0/612468121 conn(0x7f528c0b5480 msgr2=0x7f528c0b7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:27.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.217+0000 7f529109e640 1 -- 172.21.15.139:0/612468121 shutdown_connections 2024-04-25T09:54:27.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.217+0000 7f529109e640 1 -- 172.21.15.139:0/612468121 wait complete. 2024-04-25T09:54:27.231 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:54:27.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.326+0000 7fa9d94a1640 1 Processor -- start 2024-04-25T09:54:27.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.326+0000 7fa9d94a1640 1 -- start start 2024-04-25T09:54:27.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.327+0000 7fa9d94a1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa9d40d1080 0x7fa9d40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.327+0000 7fa9d94a1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d40cf250 con 0x7fa9d415dc60 2024-04-25T09:54:27.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.327+0000 7fa9d94a1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d40ce9b0 con 0x7fa9d4161850 2024-04-25T09:54:27.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.327+0000 7fa9d94a1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d40d3730 con 0x7fa9d40d1080 2024-04-25T09:54:27.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.327+0000 7fa9cbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa9d40d1080 0x7fa9d40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.327+0000 7fa9cbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa9d40d1080 0x7fa9d40b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41108/0 (socket says 172.21.15.139:41108) 2024-04-25T09:54:27.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.327+0000 7fa9d2ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa9d415dc60 0x7fa9d40d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57908/0 (socket says 172.21.15.139:57908) 2024-04-25T09:54:27.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.327+0000 7fa9cbfff640 1 -- 172.21.15.139:0/4251166436 learned_addr learned my addr 172.21.15.139:0/4251166436 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:27.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.327+0000 7fa9cbfff640 1 -- 172.21.15.139:0/4251166436 >> v1:172.21.15.150:6789/0 conn(0x7fa9d4161850 legacy=0x7fa9d40d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.327+0000 7fa9cbfff640 1 -- 172.21.15.139:0/4251166436 >> v1:172.21.15.139:6789/0 conn(0x7fa9d415dc60 legacy=0x7fa9d40d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.327+0000 7fa9cbfff640 1 -- 172.21.15.139:0/4251166436 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40cec90 con 0x7fa9d40d1080 2024-04-25T09:54:27.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.327+0000 7fa9d27fc640 1 -- 172.21.15.139:0/4251166436 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152556283 0 0) 0x7fa9c4001110 con 0x7fa9d415dc60 2024-04-25T09:54:27.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.328+0000 7fa9cbfff640 1 --2- 172.21.15.139:0/4251166436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d40d1080 0x7fa9d40b6c10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009870 tx=0x7fa9bc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86441d29aa9effc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.328+0000 7fa9d27fc640 1 -- 172.21.15.139:0/4251166436 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc037070 con 0x7fa9d40d1080 2024-04-25T09:54:27.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.328+0000 7fa9d27fc640 1 -- 172.21.15.139:0/4251166436 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9bc032440 con 0x7fa9d40d1080 2024-04-25T09:54:27.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.328+0000 7fa9d27fc640 1 -- 172.21.15.139:0/4251166436 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc032810 con 0x7fa9d40d1080 2024-04-25T09:54:27.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.328+0000 7fa9d94a1640 1 -- 172.21.15.139:0/4251166436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d40d1080 msgr2=0x7fa9d40b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.328+0000 7fa9d94a1640 1 --2- 172.21.15.139:0/4251166436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d40d1080 0x7fa9d40b6c10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009870 tx=0x7fa9bc02a2a0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.328+0000 7fa9d94a1640 1 -- 172.21.15.139:0/4251166436 shutdown_connections 2024-04-25T09:54:27.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.328+0000 7fa9d94a1640 1 --2- 172.21.15.139:0/4251166436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d40d1080 0x7fa9d40b6c10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.329+0000 7fa9d94a1640 1 -- 172.21.15.139:0/4251166436 >> 172.21.15.139:0/4251166436 conn(0x7fa9d40b5910 msgr2=0x7fa9d40b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:27.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.329+0000 7fa9d94a1640 1 -- 172.21.15.139:0/4251166436 shutdown_connections 2024-04-25T09:54:27.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.329+0000 7fa9d94a1640 1 -- 172.21.15.139:0/4251166436 wait complete. 2024-04-25T09:54:27.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.330+0000 7fa9d94a1640 1 Processor -- start 2024-04-25T09:54:27.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.330+0000 7fa9d94a1640 1 -- start start 2024-04-25T09:54:27.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.330+0000 7fa9d94a1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9d40d1080 0x7fa9d4151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.330+0000 7fa9d94a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d415dc60 0x7fa9d40d3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.330+0000 7fa9d94a1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9d4161850 0x7fa9d40d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.330+0000 7fa9d94a1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d415e040 con 0x7fa9d415dc60 2024-04-25T09:54:27.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.330+0000 7fa9d94a1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40d1fc0 con 0x7fa9d4161850 2024-04-25T09:54:27.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.330+0000 7fa9d94a1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c7e20 con 0x7fa9d40d1080 2024-04-25T09:54:27.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.330+0000 7fa9cbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d415dc60 0x7fa9d40d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.330+0000 7fa9cbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d415dc60 0x7fa9d40d3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41120/0 (socket says 172.21.15.139:41120) 2024-04-25T09:54:27.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.330+0000 7fa9cbfff640 1 -- 172.21.15.139:0/524869216 learned_addr learned my addr 172.21.15.139:0/524869216 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:27.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9d2ffd640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9d40d1080 0x7fa9d4151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9d37fe640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9d4161850 0x7fa9d40d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9cbfff640 1 -- 172.21.15.139:0/524869216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9d40d1080 msgr2=0x7fa9d4151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9cbfff640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9d40d1080 0x7fa9d4151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9cbfff640 1 -- 172.21.15.139:0/524869216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9d4161850 msgr2=0x7fa9d40d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9cbfff640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9d4161850 0x7fa9d40d3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9cbfff640 1 -- 172.21.15.139:0/524869216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40c8100 con 0x7fa9d415dc60 2024-04-25T09:54:27.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9cbfff640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d415dc60 0x7fa9d40d3970 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009870 tx=0x7fa9bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9cb7fe640 1 -- 172.21.15.139:0/524869216 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc041070 con 0x7fa9d415dc60 2024-04-25T09:54:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9cb7fe640 1 -- 172.21.15.139:0/524869216 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9bc03c870 con 0x7fa9d415dc60 2024-04-25T09:54:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9d94a1640 1 -- 172.21.15.139:0/524869216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d415f7d0 con 0x7fa9d415dc60 2024-04-25T09:54:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9cb7fe640 1 -- 172.21.15.139:0/524869216 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc037040 con 0x7fa9d415dc60 2024-04-25T09:54:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.331+0000 7fa9d94a1640 1 -- 172.21.15.139:0/524869216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d415fd90 con 0x7fa9d415dc60 2024-04-25T09:54:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.332+0000 7fa9d94a1640 1 -- 172.21.15.139:0/524869216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d4074520 con 0x7fa9d415dc60 2024-04-25T09:54:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.333+0000 7fa9cb7fe640 1 -- 172.21.15.139:0/524869216 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9bc044070 con 0x7fa9d415dc60 2024-04-25T09:54:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.333+0000 7fa9cb7fe640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa9b406b4d0 0x7fa9b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.333+0000 7fa9cb7fe640 1 -- 172.21.15.139:0/524869216 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa9bc0ab000 con 0x7fa9d415dc60 2024-04-25T09:54:27.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.333+0000 7fa9d2ffd640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa9b406b4d0 0x7fa9b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.333+0000 7fa9d2ffd640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa9b406b4d0 0x7fa9b406d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4000f60 tx=0x7fa9c4007970 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.337+0000 7fa9cb7fe640 1 -- 172.21.15.139:0/524869216 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9bc079ea0 con 0x7fa9d415dc60 2024-04-25T09:54:27.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.544+0000 7fa9d94a1640 1 -- 172.21.15.139:0/524869216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa9d40d16f0 con 0x7fa9d415dc60 2024-04-25T09:54:27.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.544+0000 7fa9cb7fe640 1 -- 172.21.15.139:0/524869216 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa9bc0797a0 con 0x7fa9d415dc60 2024-04-25T09:54:27.547 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:54:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.546+0000 7fa9d94a1640 1 -- 172.21.15.139:0/524869216 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa9b406b4d0 msgr2=0x7fa9b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.546+0000 7fa9d94a1640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa9b406b4d0 0x7fa9b406d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4000f60 tx=0x7fa9c4007970 comp rx=0 tx=0).stop 2024-04-25T09:54:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.546+0000 7fa9d94a1640 1 -- 172.21.15.139:0/524869216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d415dc60 msgr2=0x7fa9d40d3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.546+0000 7fa9d94a1640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d415dc60 0x7fa9d40d3970 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009870 tx=0x7fa9bc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.546+0000 7fa9d94a1640 1 -- 172.21.15.139:0/524869216 shutdown_connections 2024-04-25T09:54:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.546+0000 7fa9d94a1640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d415dc60 0x7fa9d40d3970 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.546+0000 7fa9d94a1640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa9b406b4d0 0x7fa9b406d990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.546+0000 7fa9d94a1640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9d4161850 0x7fa9d40d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.546+0000 7fa9d94a1640 1 --2- 172.21.15.139:0/524869216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9d40d1080 0x7fa9d4151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.546+0000 7fa9d94a1640 1 -- 172.21.15.139:0/524869216 >> 172.21.15.139:0/524869216 conn(0x7fa9d40b5910 msgr2=0x7fa9d406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.546+0000 7fa9d94a1640 1 -- 172.21.15.139:0/524869216 shutdown_connections 2024-04-25T09:54:27.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:27.546+0000 7fa9d94a1640 1 -- 172.21.15.139:0/524869216 wait complete. 2024-04-25T09:54:27.560 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:54:27.560 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:54:27.560 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T09:54:27.560 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:54:27.560 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:27.560 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:54:27.560 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:54:27.560 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:54:27.560 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:54:27.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:27.560 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T09:54:27.576 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:54:27.576 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:54:27.576 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:54:27.576 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:27.576 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:54:27.576 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:27.576 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:27.576 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:27.576 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T09:54:27.577 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:27.577 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:27.577 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T09:54:27.577 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:27.577 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:27.577 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:54:27.577 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:27.577 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:54:27.577 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42631sec preferred_lft 42631sec 2024-04-25T09:54:27.577 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T09:54:27.577 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:27.577 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:54:27.578 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:27.578 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:54:27.578 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:27.578 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T09:54:27.578 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:27.578 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:54:27.578 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:54:27.578 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T09:54:27.578 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:27.578 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:54:27.578 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:54:27.579 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T09:54:27.579 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:27.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:27.579 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:54:27.639 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:54:27.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:27.640 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T09:54:27.697 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T09:54:27.697 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:54:27.697 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:54:27.698 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:54:27.698 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:54:27.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:27.698 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:54:27.698 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:54:27.699 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:54:27.699 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:54:27.699 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:54:27.699 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:54:27.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:27.782 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:54:27.837 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:54:27.837 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:54:27.837 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:54:27.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:27.838 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T09:54:27.902 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:54:27.902 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:27.956 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:54:27.956 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:54:27.956 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:28.011 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:54:28.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:28.011 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:54:28.092 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:54:28.093 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:54:28.100 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:54:28.100 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:54:28.100 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T09:54:28.101 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T09:54:28.101 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T09:54:28.101 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:54:28.101 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:54:28.101 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:54:28.101 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:54:28.101 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:54:28.102 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:54:28.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:28.103 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:28.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:28.168 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:54:28.228 INFO:teuthology.orchestra.run.smithi139.stdout:client5607 2024-04-25T09:54:28.229 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:54:28.229 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:28.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:28.284 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-25T09:54:28.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:28.344 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-25T09:54:28.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:28.401 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-25T09:54:28.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:28.457 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-25T09:54:28.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:28.516 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-25T09:54:28.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:28.573 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T09:54:28.632 INFO:teuthology.orchestra.run.smithi139.stdout:./ 2024-04-25T09:54:28.632 INFO:teuthology.orchestra.run.smithi139.stdout:./rfile 2024-04-25T09:54:28.632 INFO:teuthology.orchestra.run.smithi139.stdout:./parentdir 2024-04-25T09:54:28.633 INFO:teuthology.orchestra.run.smithi139.stdout:./parentdir/pfile 2024-04-25T09:54:28.633 INFO:teuthology.orchestra.run.smithi139.stdout:./parentdir/childdir 2024-04-25T09:54:28.633 INFO:teuthology.orchestra.run.smithi139.stdout:./parentdir/childdir/cfile 2024-04-25T09:54:28.633 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T09:54:28.633 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T09:54:28.633 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T09:54:28.633 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:54:28.633 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T09:54:28.633 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T09:54:28.738 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:54:28.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:54:28.747 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T09:54:28.747 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T09:54:28.747 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T09:54:28.747 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:54:28.747 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-25T09:54:28.747 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T09:54:28.852 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627778 2024-04-25T09:54:28.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:54:28.861 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T09:54:28.861 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T09:54:28.861 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T09:54:28.861 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:54:28.861 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-25T09:54:28.861 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T09:54:28.966 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627776 2024-04-25T09:54:28.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:54:28.979 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T09:54:28.979 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T09:54:28.979 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T09:54:28.979 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:54:28.979 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-25T09:54:28.980 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T09:54:29.080 INFO:teuthology.orchestra.run.smithi139.stdout:1099511628279 2024-04-25T09:54:29.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:54:29.089 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T09:54:29.089 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T09:54:29.089 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T09:54:29.089 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:54:29.089 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-25T09:54:29.089 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T09:54:29.193 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627777 2024-04-25T09:54:29.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:54:29.203 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T09:54:29.203 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T09:54:29.203 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T09:54:29.203 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:54:29.203 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-25T09:54:29.203 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T09:54:29.307 INFO:teuthology.orchestra.run.smithi139.stdout:1099511628280 2024-04-25T09:54:29.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:54:29.316 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:54:29.316 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:29.372 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:54:29.372 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:29.427 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:29.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.562+0000 7f29c36f5640 1 Processor -- start 2024-04-25T09:54:29.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.562+0000 7f29c36f5640 1 -- start start 2024-04-25T09:54:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c36f5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f29bc15a860 0x7f29bc0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c36f5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f29bc0cd1f0 0x7f29bc06e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c36f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29bc15ac40 0x7f29bc06e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c36f5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc0b02e0 con 0x7f29bc15a860 2024-04-25T09:54:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c36f5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc0af6d0 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c36f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0af9b0 con 0x7f29bc15ac40 2024-04-25T09:54:29.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c1491640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f29bc15a860 0x7f29bc0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c1491640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f29bc15a860 0x7f29bc0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41136/0 (socket says 172.21.15.139:41136) 2024-04-25T09:54:29.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c1491640 1 -- 172.21.15.139:0/1706618263 learned_addr learned my addr 172.21.15.139:0/1706618263 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:29.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c1c92640 1 --2- 172.21.15.139:0/1706618263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29bc15ac40 0x7f29bc06e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c0c90640 1 --2- 172.21.15.139:0/1706618263 >> v2:172.21.15.150:3300/0 conn(0x7f29bc0cd1f0 0x7f29bc06e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c1491640 1 -- 172.21.15.139:0/1706618263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29bc15ac40 msgr2=0x7f29bc06e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:29.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c1491640 1 --2- 172.21.15.139:0/1706618263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29bc15ac40 0x7f29bc06e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c1491640 1 -- 172.21.15.139:0/1706618263 >> v2:172.21.15.150:3300/0 conn(0x7f29bc0cd1f0 msgr2=0x7f29bc06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:29.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c1491640 1 --2- 172.21.15.139:0/1706618263 >> v2:172.21.15.150:3300/0 conn(0x7f29bc0cd1f0 0x7f29bc06e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.563+0000 7f29c1491640 1 -- 172.21.15.139:0/1706618263 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc0afc90 con 0x7f29bc15a860 2024-04-25T09:54:29.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.564+0000 7f29c1491640 1 --2- 172.21.15.139:0/1706618263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29bc15a860 0x7f29bc0cccb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0b2610 tx=0x7f29b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe3871b9debe48c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.564+0000 7f29b3fff640 1 -- 172.21.15.139:0/1706618263 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8037070 con 0x7f29bc15a860 2024-04-25T09:54:29.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.564+0000 7f29b3fff640 1 -- 172.21.15.139:0/1706618263 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b8032440 con 0x7f29bc15a860 2024-04-25T09:54:29.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.564+0000 7f29b3fff640 1 -- 172.21.15.139:0/1706618263 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8032810 con 0x7f29bc15a860 2024-04-25T09:54:29.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.564+0000 7f29c36f5640 1 -- 172.21.15.139:0/1706618263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29bc15a860 msgr2=0x7f29bc0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.564+0000 7f29c36f5640 1 --2- 172.21.15.139:0/1706618263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29bc15a860 0x7f29bc0cccb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0b2610 tx=0x7f29b802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.564+0000 7f29c36f5640 1 -- 172.21.15.139:0/1706618263 shutdown_connections 2024-04-25T09:54:29.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.564+0000 7f29c36f5640 1 --2- 172.21.15.139:0/1706618263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29bc15ac40 0x7f29bc06e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.564+0000 7f29c36f5640 1 --2- 172.21.15.139:0/1706618263 >> v2:172.21.15.150:3300/0 conn(0x7f29bc0cd1f0 0x7f29bc06e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.564+0000 7f29c36f5640 1 --2- 172.21.15.139:0/1706618263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29bc15a860 0x7f29bc0cccb0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.565+0000 7f29c36f5640 1 -- 172.21.15.139:0/1706618263 >> 172.21.15.139:0/1706618263 conn(0x7f29bc066f20 msgr2=0x7f29bc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:29.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.565+0000 7f29c36f5640 1 -- 172.21.15.139:0/1706618263 shutdown_connections 2024-04-25T09:54:29.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.565+0000 7f29c36f5640 1 -- 172.21.15.139:0/1706618263 wait complete. 2024-04-25T09:54:29.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.566+0000 7f29c36f5640 1 Processor -- start 2024-04-25T09:54:29.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.566+0000 7f29c36f5640 1 -- start start 2024-04-25T09:54:29.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.566+0000 7f29c36f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29bc0cd1f0 0x7f29bc153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.566+0000 7f29c36f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29bc15a860 0x7f29bc1514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.566+0000 7f29c36f5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29bc15ac40 0x7f29bc158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.566+0000 7f29c36f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0af420 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.566+0000 7f29c36f5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0af2a0 con 0x7f29bc15ac40 2024-04-25T09:54:29.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.566+0000 7f29c36f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc15e240 con 0x7f29bc15a860 2024-04-25T09:54:29.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.566+0000 7f29c1491640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29bc0cd1f0 0x7f29bc153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.566+0000 7f29c1491640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29bc0cd1f0 0x7f29bc153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41146/0 (socket says 172.21.15.139:41146) 2024-04-25T09:54:29.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.566+0000 7f29c1491640 1 -- 172.21.15.139:0/2076256588 learned_addr learned my addr 172.21.15.139:0/2076256588 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:29.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29c1c92640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29bc15ac40 0x7f29bc158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29c0c90640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29bc15a860 0x7f29bc1514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29c1491640 1 -- 172.21.15.139:0/2076256588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29bc15a860 msgr2=0x7f29bc1514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29c1491640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29bc15a860 0x7f29bc1514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29c1491640 1 -- 172.21.15.139:0/2076256588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29bc15ac40 msgr2=0x7f29bc158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29c1491640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29bc15ac40 0x7f29bc158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29c1491640 1 -- 172.21.15.139:0/2076256588 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc151de0 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29c1491640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29bc0cd1f0 0x7f29bc153890 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f29b802a220 tx=0x7f29b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29b1ffb640 1 -- 172.21.15.139:0/2076256588 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8041020 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29b1ffb640 1 -- 172.21.15.139:0/2076256588 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b80306c0 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29c36f5640 1 -- 172.21.15.139:0/2076256588 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc1520c0 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29b1ffb640 1 -- 172.21.15.139:0/2076256588 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8037040 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.567+0000 7f29c36f5640 1 -- 172.21.15.139:0/2076256588 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc152680 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.568+0000 7f29c36f5640 1 -- 172.21.15.139:0/2076256588 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29bc074520 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.569+0000 7f29b1ffb640 1 -- 172.21.15.139:0/2076256588 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29b8044020 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.569+0000 7f29b1ffb640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f29a006b500 0x7f29a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.569+0000 7f29b1ffb640 1 -- 172.21.15.139:0/2076256588 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29b80aaba0 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.569+0000 7f29c0c90640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f29a006b500 0x7f29a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.569+0000 7f29c0c90640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f29a006b500 0x7f29a006d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f29bc15c5a0 tx=0x7f29ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.573+0000 7f29b1ffb640 1 -- 172.21.15.139:0/2076256588 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29b804b030 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.747+0000 7f29c36f5640 1 -- 172.21.15.139:0/2076256588 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29bc06ec20 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.748+0000 7f29b1ffb640 1 -- 172.21.15.139:0/2076256588 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f29b8079890 con 0x7f29bc0cd1f0 2024-04-25T09:54:29.751 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:29.751 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":38,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:23.620264+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:19.795203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:20.931184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:29.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.750+0000 7f29c36f5640 1 -- 172.21.15.139:0/2076256588 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f29a006b500 msgr2=0x7f29a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.750+0000 7f29c36f5640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f29a006b500 0x7f29a006d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f29bc15c5a0 tx=0x7f29ac008040 comp rx=0 tx=0).stop 2024-04-25T09:54:29.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.750+0000 7f29c36f5640 1 -- 172.21.15.139:0/2076256588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29bc0cd1f0 msgr2=0x7f29bc153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.750+0000 7f29c36f5640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29bc0cd1f0 0x7f29bc153890 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f29b802a220 tx=0x7f29b80046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.750+0000 7f29c36f5640 1 -- 172.21.15.139:0/2076256588 shutdown_connections 2024-04-25T09:54:29.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.750+0000 7f29c36f5640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f29a006b500 0x7f29a006d9c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.750+0000 7f29c36f5640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29bc15ac40 0x7f29bc158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.750+0000 7f29c36f5640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29bc15a860 0x7f29bc1514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.750+0000 7f29c36f5640 1 --2- 172.21.15.139:0/2076256588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29bc0cd1f0 0x7f29bc153890 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.750+0000 7f29c36f5640 1 -- 172.21.15.139:0/2076256588 >> 172.21.15.139:0/2076256588 conn(0x7f29bc066f20 msgr2=0x7f29bc15d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:29.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.750+0000 7f29c36f5640 1 -- 172.21.15.139:0/2076256588 shutdown_connections 2024-04-25T09:54:29.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.750+0000 7f29c36f5640 1 -- 172.21.15.139:0/2076256588 wait complete. 2024-04-25T09:54:29.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:19.795203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:20.931184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:54:29.764 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:29.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.861+0000 7fdc00961640 1 Processor -- start 2024-04-25T09:54:29.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.861+0000 7fdc00961640 1 -- start start 2024-04-25T09:54:29.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.862+0000 7fdc00961640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbfc15dbf0 0x7fdbfc1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.862+0000 7fdc00961640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdbfc0cee50 0x7fdbfc0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.862+0000 7fdc00961640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbfc161dc0 con 0x7fdbfc0b9370 2024-04-25T09:54:29.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.862+0000 7fdc00961640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbfc0d1ee0 con 0x7fdbfc0cee50 2024-04-25T09:54:29.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.862+0000 7fdc00961640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1620a0 con 0x7fdbfc15dbf0 2024-04-25T09:54:29.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.862+0000 7fdbf9d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdbfc0b9370 0x7fdbfc0ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57922/0 (socket says 172.21.15.139:57922) 2024-04-25T09:54:29.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.862+0000 7fdbf9d9b640 1 -- 172.21.15.139:0/2908492643 learned_addr learned my addr 172.21.15.139:0/2908492643 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:29.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374000125 0 0) 0x7fdbfc161dc0 con 0x7fdbfc0b9370 2024-04-25T09:54:29.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbfad9d640 1 --2- 172.21.15.139:0/2908492643 >> v2:172.21.15.150:3300/0 conn(0x7fdbfc0cee50 0x7fdbfc0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbe4003520 con 0x7fdbfc0b9370 2024-04-25T09:54:29.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbfa59c640 1 --2- 172.21.15.139:0/2908492643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbfc15dbf0 0x7fdbfc1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 150478747 0 0) 0x7fdbe4003520 con 0x7fdbfc0b9370 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbfc15dbf0 msgr2=0x7fdbfc1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 --2- 172.21.15.139:0/2908492643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbfc15dbf0 0x7fdbfc1617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 >> v2:172.21.15.150:3300/0 conn(0x7fdbfc0cee50 msgr2=0x7fdbfc0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 --2- 172.21.15.139:0/2908492643 >> v2:172.21.15.150:3300/0 conn(0x7fdbfc0cee50 0x7fdbfc0d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc162380 con 0x7fdbfc0b9370 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fdbf0001670 con 0x7fdbfc0b9370 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 >> v1:172.21.15.139:6789/0 conn(0x7fdbfc0b9370 legacy=0x7fdbfc0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 --2- 172.21.15.139:0/2908492643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbfc15dbf0 0x7fdbe40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 --2- 172.21.15.139:0/2908492643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbe4003f30 0x7fdbe4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 --2- 172.21.15.139:0/2908492643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbe4004d40 0x7fdbe40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc0d1ee0 con 0x7fdbe4004d40 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1620a0 con 0x7fdbe4003f30 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe400b9c0 con 0x7fdbfc15dbf0 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbfad9d640 1 --2- 172.21.15.139:0/2908492643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbe4004d40 0x7fdbe40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbfa59c640 1 --2- 172.21.15.139:0/2908492643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbfc15dbf0 0x7fdbe40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbf9d9b640 1 --2- 172.21.15.139:0/2908492643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbe4003f30 0x7fdbe4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbfad9d640 1 -- 172.21.15.139:0/2908492643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbfc15dbf0 msgr2=0x7fdbe40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbfad9d640 1 --2- 172.21.15.139:0/2908492643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbfc15dbf0 0x7fdbe40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbfad9d640 1 -- 172.21.15.139:0/2908492643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbe4003f30 msgr2=0x7fdbe4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbfad9d640 1 --2- 172.21.15.139:0/2908492643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbe4003f30 0x7fdbe4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.863+0000 7fdbfad9d640 1 -- 172.21.15.139:0/2908492643 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4003520 con 0x7fdbe4004d40 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.864+0000 7fdbfad9d640 1 --2- 172.21.15.139:0/2908492643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbe4004d40 0x7fdbe40051c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009a90 tx=0x7fdbe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3407b4fd7bd8c272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.864+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4036070 con 0x7fdbe4004d40 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.864+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf4031400 con 0x7fdbe4004d40 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.864+0000 7fdbf959a640 1 -- 172.21.15.139:0/2908492643 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4009690 con 0x7fdbe4004d40 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.865+0000 7fdc00961640 1 -- 172.21.15.139:0/2908492643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbe4004d40 msgr2=0x7fdbe40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.865+0000 7fdc00961640 1 --2- 172.21.15.139:0/2908492643 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbe4004d40 0x7fdbe40051c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009a90 tx=0x7fdbe4005830 comp rx=0 tx=0).stop 2024-04-25T09:54:29.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.865+0000 7fdbfad9d640 1 -- 172.21.15.139:0/2908492643 reap_dead start 2024-04-25T09:54:29.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.865+0000 7fdc00961640 1 -- 172.21.15.139:0/2908492643 shutdown_connections 2024-04-25T09:54:29.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.865+0000 7fdc00961640 1 -- 172.21.15.139:0/2908492643 >> 172.21.15.139:0/2908492643 conn(0x7fdbfc0b5480 msgr2=0x7fdbfc0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:29.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.865+0000 7fdc00961640 1 -- 172.21.15.139:0/2908492643 shutdown_connections 2024-04-25T09:54:29.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.865+0000 7fdc00961640 1 -- 172.21.15.139:0/2908492643 wait complete. 2024-04-25T09:54:29.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.866+0000 7fdc00961640 1 Processor -- start 2024-04-25T09:54:29.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.866+0000 7fdc00961640 1 -- start start 2024-04-25T09:54:29.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdc00961640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbe4004d40 0x7fdbfc15cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdc00961640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbfc14dd30 0x7fdbfc1f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdc00961640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbfc1f4130 0x7fdbfc0d3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdc00961640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc15dfd0 con 0x7fdbfc1f4130 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdc00961640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc0b97d0 con 0x7fdbfc14dd30 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdc00961640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc14e440 con 0x7fdbe4004d40 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdbfad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbfc1f4130 0x7fdbfc0d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdbfad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbfc1f4130 0x7fdbfc0d3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41174/0 (socket says 172.21.15.139:41174) 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdbfad9d640 1 -- 172.21.15.139:0/2324150303 learned_addr learned my addr 172.21.15.139:0/2324150303 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdbfa59c640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbe4004d40 0x7fdbfc15cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdbf9d9b640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbfc14dd30 0x7fdbfc1f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdbfad9d640 1 -- 172.21.15.139:0/2324150303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbe4004d40 msgr2=0x7fdbfc15cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdbfad9d640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbe4004d40 0x7fdbfc15cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdbfad9d640 1 -- 172.21.15.139:0/2324150303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbfc14dd30 msgr2=0x7fdbfc1f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdbfad9d640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbfc14dd30 0x7fdbfc1f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdbfad9d640 1 -- 172.21.15.139:0/2324150303 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc150800 con 0x7fdbfc1f4130 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.867+0000 7fdbfad9d640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbfc1f4130 0x7fdbfc0d3900 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdbf402ab90 tx=0x7fdbf4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.868+0000 7fdbeaffd640 1 -- 172.21.15.139:0/2324150303 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4036040 con 0x7fdbfc1f4130 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.868+0000 7fdbeaffd640 1 -- 172.21.15.139:0/2324150303 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf4041070 con 0x7fdbfc1f4130 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.868+0000 7fdc00961640 1 -- 172.21.15.139:0/2324150303 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc150ae0 con 0x7fdbfc1f4130 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.868+0000 7fdbeaffd640 1 -- 172.21.15.139:0/2324150303 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4009690 con 0x7fdbfc1f4130 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.868+0000 7fdc00961640 1 -- 172.21.15.139:0/2324150303 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc14d7e0 con 0x7fdbfc1f4130 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.869+0000 7fdc00961640 1 -- 172.21.15.139:0/2324150303 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbfc074520 con 0x7fdbfc1f4130 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.869+0000 7fdbeaffd640 1 -- 172.21.15.139:0/2324150303 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdbf4039070 con 0x7fdbfc1f4130 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.869+0000 7fdbeaffd640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdbdc06b4d0 0x7fdbdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.870+0000 7fdbeaffd640 1 -- 172.21.15.139:0/2324150303 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdbf40a9b60 con 0x7fdbfc1f4130 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.870+0000 7fdbfa59c640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdbdc06b4d0 0x7fdbdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.870+0000 7fdbfa59c640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdbdc06b4d0 0x7fdbdc06d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdbec007520 tx=0x7fdbec00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:29.874+0000 7fdbeaffd640 1 -- 172.21.15.139:0/2324150303 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdbf404a020 con 0x7fdbfc1f4130 2024-04-25T09:54:30.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.042+0000 7fdc00961640 1 -- 172.21.15.139:0/2324150303 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdbfc0d1680 con 0x7fdbfc1f4130 2024-04-25T09:54:30.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.042+0000 7fdbeaffd640 1 -- 172.21.15.139:0/2324150303 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdbf4078850 con 0x7fdbfc1f4130 2024-04-25T09:54:30.045 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:54:30.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.044+0000 7fdc00961640 1 -- 172.21.15.139:0/2324150303 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdbdc06b4d0 msgr2=0x7fdbdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.044+0000 7fdc00961640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdbdc06b4d0 0x7fdbdc06d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdbec007520 tx=0x7fdbec00b040 comp rx=0 tx=0).stop 2024-04-25T09:54:30.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.044+0000 7fdc00961640 1 -- 172.21.15.139:0/2324150303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbfc1f4130 msgr2=0x7fdbfc0d3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.044+0000 7fdc00961640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbfc1f4130 0x7fdbfc0d3900 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdbf402ab90 tx=0x7fdbf4004730 comp rx=0 tx=0).stop 2024-04-25T09:54:30.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.044+0000 7fdc00961640 1 -- 172.21.15.139:0/2324150303 shutdown_connections 2024-04-25T09:54:30.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.044+0000 7fdc00961640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbfc1f4130 0x7fdbfc0d3900 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.044+0000 7fdc00961640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdbdc06b4d0 0x7fdbdc06d990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.044+0000 7fdc00961640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbfc14dd30 0x7fdbfc1f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.044+0000 7fdc00961640 1 --2- 172.21.15.139:0/2324150303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbe4004d40 0x7fdbfc15cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.044+0000 7fdc00961640 1 -- 172.21.15.139:0/2324150303 >> 172.21.15.139:0/2324150303 conn(0x7fdbfc0b5480 msgr2=0x7fdbfc158930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:30.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.044+0000 7fdc00961640 1 -- 172.21.15.139:0/2324150303 shutdown_connections 2024-04-25T09:54:30.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.044+0000 7fdc00961640 1 -- 172.21.15.139:0/2324150303 wait complete. 2024-04-25T09:54:30.058 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:54:30.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.155+0000 7fe19118b640 1 Processor -- start 2024-04-25T09:54:30.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.155+0000 7fe19118b640 1 -- start start 2024-04-25T09:54:30.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.155+0000 7fe19118b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe18c0cef00 0x7fe18c168b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.155+0000 7fe19118b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c0d1e60 con 0x7fe18c0b93b0 2024-04-25T09:54:30.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.155+0000 7fe19118b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c162050 con 0x7fe18c15dba0 2024-04-25T09:54:30.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.155+0000 7fe19118b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0d21b0 con 0x7fe18c0cef00 2024-04-25T09:54:30.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.156+0000 7fe183fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe18c0b93b0 0x7fe18c0ce7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57934/0 (socket says 172.21.15.139:57934) 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.156+0000 7fe183fff640 1 -- 172.21.15.139:0/3087862988 learned_addr learned my addr 172.21.15.139:0/3087862988 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.156+0000 7fe18b59e640 1 --2- 172.21.15.139:0/3087862988 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe18c0cef00 0x7fe18c168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.156+0000 7fe18a59c640 1 -- 172.21.15.139:0/3087862988 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534460347 0 0) 0x7fe18c0d1e60 con 0x7fe18c0b93b0 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.156+0000 7fe18a59c640 1 -- 172.21.15.139:0/3087862988 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe178003520 con 0x7fe18c0b93b0 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.156+0000 7fe18a59c640 1 -- 172.21.15.139:0/3087862988 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069434510 0 0) 0x7fe18c162050 con 0x7fe18c15dba0 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.156+0000 7fe18a59c640 1 -- 172.21.15.139:0/3087862988 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe18c0d1e60 con 0x7fe18c15dba0 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.156+0000 7fe18b59e640 1 -- 172.21.15.139:0/3087862988 >> v1:172.21.15.150:6789/0 conn(0x7fe18c15dba0 legacy=0x7fe18c161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.156+0000 7fe18b59e640 1 -- 172.21.15.139:0/3087862988 >> v1:172.21.15.139:6789/0 conn(0x7fe18c0b93b0 legacy=0x7fe18c0ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.156+0000 7fe18a59c640 1 -- 172.21.15.139:0/3087862988 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 743342683 0 0) 0x7fe178003520 con 0x7fe18c0b93b0 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.156+0000 7fe18b59e640 1 -- 172.21.15.139:0/3087862988 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c162330 con 0x7fe18c0cef00 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.157+0000 7fe18b59e640 1 --2- 172.21.15.139:0/3087862988 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe18c0cef00 0x7fe18c168b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0d2330 tx=0x7fe18402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f016e7651f0b5a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.157+0000 7fe18a59c640 1 -- 172.21.15.139:0/3087862988 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174001670 con 0x7fe18c0cef00 2024-04-25T09:54:30.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.157+0000 7fe18a59c640 1 -- 172.21.15.139:0/3087862988 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe184007540 con 0x7fe18c0cef00 2024-04-25T09:54:30.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.157+0000 7fe18a59c640 1 -- 172.21.15.139:0/3087862988 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840078f0 con 0x7fe18c0cef00 2024-04-25T09:54:30.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.158+0000 7fe19118b640 1 -- 172.21.15.139:0/3087862988 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe18c0cef00 msgr2=0x7fe18c168b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.158+0000 7fe19118b640 1 --2- 172.21.15.139:0/3087862988 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe18c0cef00 0x7fe18c168b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0d2330 tx=0x7fe18402a660 comp rx=0 tx=0).stop 2024-04-25T09:54:30.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.158+0000 7fe19118b640 1 -- 172.21.15.139:0/3087862988 shutdown_connections 2024-04-25T09:54:30.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.158+0000 7fe19118b640 1 --2- 172.21.15.139:0/3087862988 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe18c0cef00 0x7fe18c168b50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.158+0000 7fe19118b640 1 -- 172.21.15.139:0/3087862988 >> 172.21.15.139:0/3087862988 conn(0x7fe18c0d04a0 msgr2=0x7fe18c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:30.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.158+0000 7fe19118b640 1 -- 172.21.15.139:0/3087862988 shutdown_connections 2024-04-25T09:54:30.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.158+0000 7fe19118b640 1 -- 172.21.15.139:0/3087862988 wait complete. 2024-04-25T09:54:30.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.159+0000 7fe19118b640 1 Processor -- start 2024-04-25T09:54:30.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.159+0000 7fe19118b640 1 -- start start 2024-04-25T09:54:30.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe19118b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c0b93b0 0x7fe18c152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe19118b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe18c0cef00 0x7fe18c0aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe19118b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe18c15dba0 0x7fe18c1f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe19118b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0d3650 con 0x7fe18c0b93b0 2024-04-25T09:54:30.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe19118b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0b6bc0 con 0x7fe18c0cef00 2024-04-25T09:54:30.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe19118b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15df80 con 0x7fe18c15dba0 2024-04-25T09:54:30.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe18ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c0b93b0 0x7fe18c152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe18ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c0b93b0 0x7fe18c152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41176/0 (socket says 172.21.15.139:41176) 2024-04-25T09:54:30.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe18ad9d640 1 -- 172.21.15.139:0/3548976069 learned_addr learned my addr 172.21.15.139:0/3548976069 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:30.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe183fff640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe18c0cef00 0x7fe18c0aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe18b59e640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe18c15dba0 0x7fe18c1f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe18ad9d640 1 -- 172.21.15.139:0/3548976069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe18c15dba0 msgr2=0x7fe18c1f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe18ad9d640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe18c15dba0 0x7fe18c1f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe18ad9d640 1 -- 172.21.15.139:0/3548976069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe18c0cef00 msgr2=0x7fe18c0aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe18ad9d640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe18c0cef00 0x7fe18c0aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe18ad9d640 1 -- 172.21.15.139:0/3548976069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c0ab120 con 0x7fe18c0b93b0 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe18ad9d640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c0b93b0 0x7fe18c152f20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00c4c0 tx=0x7fe17c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe1837fe640 1 -- 172.21.15.139:0/3548976069 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c0052d0 con 0x7fe18c0b93b0 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe1837fe640 1 -- 172.21.15.139:0/3548976069 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c005470 con 0x7fe18c0b93b0 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe19118b640 1 -- 172.21.15.139:0/3548976069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c0aae40 con 0x7fe18c0b93b0 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe1837fe640 1 -- 172.21.15.139:0/3548976069 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c012870 con 0x7fe18c0b93b0 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.160+0000 7fe19118b640 1 -- 172.21.15.139:0/3548976069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c15f720 con 0x7fe18c0b93b0 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.161+0000 7fe19118b640 1 -- 172.21.15.139:0/3548976069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18c0b6ee0 con 0x7fe18c0b93b0 2024-04-25T09:54:30.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.162+0000 7fe1837fe640 1 -- 172.21.15.139:0/3548976069 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe17c012a10 con 0x7fe18c0b93b0 2024-04-25T09:54:30.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.162+0000 7fe1837fe640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe16c06b4d0 0x7fe16c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.162+0000 7fe1837fe640 1 -- 172.21.15.139:0/3548976069 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe17c08a0a0 con 0x7fe18c0b93b0 2024-04-25T09:54:30.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.162+0000 7fe183fff640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe16c06b4d0 0x7fe16c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.162+0000 7fe183fff640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe16c06b4d0 0x7fe16c06d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0d0ed0 tx=0x7fe174009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.166+0000 7fe1837fe640 1 -- 172.21.15.139:0/3548976069 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe17c058d00 con 0x7fe18c0b93b0 2024-04-25T09:54:30.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.335+0000 7fe19118b640 1 -- 172.21.15.139:0/3548976069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe18c0c72f0 con 0x7fe18c0b93b0 2024-04-25T09:54:30.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.336+0000 7fe1837fe640 1 -- 172.21.15.139:0/3548976069 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe17c058600 con 0x7fe18c0b93b0 2024-04-25T09:54:30.338 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:54:30.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.337+0000 7fe19118b640 1 -- 172.21.15.139:0/3548976069 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe16c06b4d0 msgr2=0x7fe16c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.338+0000 7fe19118b640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe16c06b4d0 0x7fe16c06d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0d0ed0 tx=0x7fe174009040 comp rx=0 tx=0).stop 2024-04-25T09:54:30.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.338+0000 7fe19118b640 1 -- 172.21.15.139:0/3548976069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c0b93b0 msgr2=0x7fe18c152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.338+0000 7fe19118b640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c0b93b0 0x7fe18c152f20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00c4c0 tx=0x7fe17c00c980 comp rx=0 tx=0).stop 2024-04-25T09:54:30.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.338+0000 7fe19118b640 1 -- 172.21.15.139:0/3548976069 shutdown_connections 2024-04-25T09:54:30.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.338+0000 7fe19118b640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe16c06b4d0 0x7fe16c06d990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.338+0000 7fe19118b640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe18c15dba0 0x7fe18c1f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.338+0000 7fe19118b640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe18c0cef00 0x7fe18c0aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.338+0000 7fe19118b640 1 --2- 172.21.15.139:0/3548976069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe18c0b93b0 0x7fe18c152f20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.338+0000 7fe19118b640 1 -- 172.21.15.139:0/3548976069 >> 172.21.15.139:0/3548976069 conn(0x7fe18c0d04a0 msgr2=0x7fe18c0b8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:30.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.338+0000 7fe19118b640 1 -- 172.21.15.139:0/3548976069 shutdown_connections 2024-04-25T09:54:30.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.338+0000 7fe19118b640 1 -- 172.21.15.139:0/3548976069 wait complete. 2024-04-25T09:54:30.351 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:54:30.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.449+0000 7fc672853640 1 Processor -- start 2024-04-25T09:54:30.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.449+0000 7fc672853640 1 -- start start 2024-04-25T09:54:30.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.449+0000 7fc672853640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc66c1617e0 0x7fc66c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.449+0000 7fc672853640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc66c0cec80 0x7fc66c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.449+0000 7fc672853640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66c161dc0 con 0x7fc66c15dbf0 2024-04-25T09:54:30.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.449+0000 7fc672853640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c0d1ee0 con 0x7fc66c0cec80 2024-04-25T09:54:30.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.449+0000 7fc672853640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c1620a0 con 0x7fc66c1617e0 2024-04-25T09:54:30.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.450+0000 7fc670df0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc66c0cec80 0x7fc66c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.450+0000 7fc670df0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc66c0cec80 0x7fc66c0d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41182/0 (socket says 172.21.15.139:41182) 2024-04-25T09:54:30.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.450+0000 7fc670df0640 1 -- 172.21.15.139:0/400745318 learned_addr learned my addr 172.21.15.139:0/400745318 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:30.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.450+0000 7fc66b7fe640 1 --2- 172.21.15.139:0/400745318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc66c1617e0 0x7fc66c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.450+0000 7fc670df0640 1 -- 172.21.15.139:0/400745318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc66c1617e0 msgr2=0x7fc66c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:30.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.450+0000 7fc670df0640 1 --2- 172.21.15.139:0/400745318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc66c1617e0 0x7fc66c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.450+0000 7fc670df0640 1 -- 172.21.15.139:0/400745318 >> v1:172.21.15.150:6789/0 conn(0x7fc66c15dbf0 legacy=0x7fc66c0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.450+0000 7fc670df0640 1 -- 172.21.15.139:0/400745318 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c162380 con 0x7fc66c0cec80 2024-04-25T09:54:30.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.451+0000 7fc670df0640 1 --2- 172.21.15.139:0/400745318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c0cec80 0x7fc66c0d2370 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc664009e50 tx=0x7fc66402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6e6e2f664fee0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.451+0000 7fc66affd640 1 -- 172.21.15.139:0/400745318 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664036070 con 0x7fc66c0cec80 2024-04-25T09:54:30.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.451+0000 7fc66affd640 1 -- 172.21.15.139:0/400745318 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc664031600 con 0x7fc66c0cec80 2024-04-25T09:54:30.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.451+0000 7fc66affd640 1 -- 172.21.15.139:0/400745318 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6640319d0 con 0x7fc66c0cec80 2024-04-25T09:54:30.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.451+0000 7fc672853640 1 -- 172.21.15.139:0/400745318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c0cec80 msgr2=0x7fc66c0d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.451+0000 7fc672853640 1 --2- 172.21.15.139:0/400745318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c0cec80 0x7fc66c0d2370 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc664009e50 tx=0x7fc66402a630 comp rx=0 tx=0).stop 2024-04-25T09:54:30.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.451+0000 7fc672853640 1 -- 172.21.15.139:0/400745318 shutdown_connections 2024-04-25T09:54:30.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.451+0000 7fc672853640 1 --2- 172.21.15.139:0/400745318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c0cec80 0x7fc66c0d2370 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.451+0000 7fc672853640 1 --2- 172.21.15.139:0/400745318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc66c1617e0 0x7fc66c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.451+0000 7fc672853640 1 -- 172.21.15.139:0/400745318 >> 172.21.15.139:0/400745318 conn(0x7fc66c0b5480 msgr2=0x7fc66c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:30.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.451+0000 7fc672853640 1 -- 172.21.15.139:0/400745318 shutdown_connections 2024-04-25T09:54:30.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.451+0000 7fc672853640 1 -- 172.21.15.139:0/400745318 wait complete. 2024-04-25T09:54:30.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.452+0000 7fc672853640 1 Processor -- start 2024-04-25T09:54:30.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.453+0000 7fc672853640 1 -- start start 2024-04-25T09:54:30.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.453+0000 7fc672853640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c0cec80 0x7fc66c15a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.453+0000 7fc672853640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66c15dbf0 0x7fc66c14af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.453+0000 7fc672853640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc66c1617e0 0x7fc66c160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.453+0000 7fc672853640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c15dfd0 con 0x7fc66c0cec80 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.453+0000 7fc672853640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c0d28b0 con 0x7fc66c15dbf0 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.453+0000 7fc672853640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c14b450 con 0x7fc66c1617e0 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.453+0000 7fc66bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c0cec80 0x7fc66c15a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.453+0000 7fc66bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c0cec80 0x7fc66c15a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41196/0 (socket says 172.21.15.139:41196) 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.453+0000 7fc66bfff640 1 -- 172.21.15.139:0/671915561 learned_addr learned my addr 172.21.15.139:0/671915561 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc670df0640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc66c1617e0 0x7fc66c160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc66b7fe640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66c15dbf0 0x7fc66c14af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc66bfff640 1 -- 172.21.15.139:0/671915561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc66c1617e0 msgr2=0x7fc66c160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc66bfff640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc66c1617e0 0x7fc66c160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc66bfff640 1 -- 172.21.15.139:0/671915561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66c15dbf0 msgr2=0x7fc66c14af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc66bfff640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66c15dbf0 0x7fc66c14af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc66bfff640 1 -- 172.21.15.139:0/671915561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c15fad0 con 0x7fc66c0cec80 2024-04-25T09:54:30.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc66b7fe640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66c15dbf0 0x7fc66c14af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:30.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc66bfff640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c0cec80 0x7fc66c15a650 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00c430 tx=0x7fc65c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc668ff9640 1 -- 172.21.15.139:0/671915561 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c011790 con 0x7fc66c0cec80 2024-04-25T09:54:30.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc668ff9640 1 -- 172.21.15.139:0/671915561 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c017070 con 0x7fc66c0cec80 2024-04-25T09:54:30.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc672853640 1 -- 172.21.15.139:0/671915561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c15fdb0 con 0x7fc66c0cec80 2024-04-25T09:54:30.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc668ff9640 1 -- 172.21.15.139:0/671915561 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c011bb0 con 0x7fc66c0cec80 2024-04-25T09:54:30.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.454+0000 7fc672853640 1 -- 172.21.15.139:0/671915561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c160370 con 0x7fc66c0cec80 2024-04-25T09:54:30.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.455+0000 7fc672853640 1 -- 172.21.15.139:0/671915561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c074520 con 0x7fc66c0cec80 2024-04-25T09:54:30.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.456+0000 7fc668ff9640 1 -- 172.21.15.139:0/671915561 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc65c01c460 con 0x7fc66c0cec80 2024-04-25T09:54:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.456+0000 7fc668ff9640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc65806b4d0 0x7fc65806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.456+0000 7fc668ff9640 1 -- 172.21.15.139:0/671915561 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc65c08a0c0 con 0x7fc66c0cec80 2024-04-25T09:54:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.456+0000 7fc66b7fe640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc65806b4d0 0x7fc65806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.456+0000 7fc66b7fe640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc65806b4d0 0x7fc65806d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc660006fd0 tx=0x7fc660008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.460+0000 7fc668ff9640 1 -- 172.21.15.139:0/671915561 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc65c058db0 con 0x7fc66c0cec80 2024-04-25T09:54:30.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.631+0000 7fc672853640 1 -- 172.21.15.139:0/671915561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc66c0d1680 con 0x7fc66c0cec80 2024-04-25T09:54:30.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.632+0000 7fc668ff9640 1 -- 172.21.15.139:0/671915561 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc65c0586b0 con 0x7fc66c0cec80 2024-04-25T09:54:30.634 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:54:30.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.634+0000 7fc672853640 1 -- 172.21.15.139:0/671915561 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc65806b4d0 msgr2=0x7fc65806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.634+0000 7fc672853640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc65806b4d0 0x7fc65806d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc660006fd0 tx=0x7fc660008040 comp rx=0 tx=0).stop 2024-04-25T09:54:30.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.634+0000 7fc672853640 1 -- 172.21.15.139:0/671915561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c0cec80 msgr2=0x7fc66c15a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.634+0000 7fc672853640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c0cec80 0x7fc66c15a650 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00c430 tx=0x7fc65c00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.634+0000 7fc672853640 1 -- 172.21.15.139:0/671915561 shutdown_connections 2024-04-25T09:54:30.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.634+0000 7fc672853640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc65806b4d0 0x7fc65806d990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.634+0000 7fc672853640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc66c1617e0 0x7fc66c160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.634+0000 7fc672853640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66c15dbf0 0x7fc66c14af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.634+0000 7fc672853640 1 --2- 172.21.15.139:0/671915561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc66c0cec80 0x7fc66c15a650 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.634+0000 7fc672853640 1 -- 172.21.15.139:0/671915561 >> 172.21.15.139:0/671915561 conn(0x7fc66c0b5480 msgr2=0x7fc66c0ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:30.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.634+0000 7fc672853640 1 -- 172.21.15.139:0/671915561 shutdown_connections 2024-04-25T09:54:30.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.634+0000 7fc672853640 1 -- 172.21.15.139:0/671915561 wait complete. 2024-04-25T09:54:30.648 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:54:30.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.744+0000 7f688e038640 1 Processor -- start 2024-04-25T09:54:30.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.744+0000 7f688e038640 1 -- start start 2024-04-25T09:54:30.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f688e038640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f68880d0a80 0x7f68881618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f688e038640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f688809ad40 0x7f6888161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f688e038640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6888099ca0 con 0x7f68880cd610 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f688e038640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6888099a50 con 0x7f688809ad40 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f688e038640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6888099f80 con 0x7f68880d0a80 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f6887fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f688809ad40 0x7f6888161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f6887fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f688809ad40 0x7f6888161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41212/0 (socket says 172.21.15.139:41212) 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f6887fff640 1 -- 172.21.15.139:0/4283878011 learned_addr learned my addr 172.21.15.139:0/4283878011 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f6886ffd640 1 --2- 172.21.15.139:0/4283878011 >> v2:172.21.15.150:3300/0 conn(0x7f68880d0a80 0x7f68881618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f6887fff640 1 -- 172.21.15.139:0/4283878011 >> v2:172.21.15.150:3300/0 conn(0x7f68880d0a80 msgr2=0x7f68881618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f6887fff640 1 --2- 172.21.15.139:0/4283878011 >> v2:172.21.15.150:3300/0 conn(0x7f68880d0a80 0x7f68881618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f6887fff640 1 -- 172.21.15.139:0/4283878011 >> v1:172.21.15.150:6789/0 conn(0x7f68880cd610 legacy=0x7f68880d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.745+0000 7f6887fff640 1 -- 172.21.15.139:0/4283878011 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688809a260 con 0x7f688809ad40 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.746+0000 7f6887fff640 1 --2- 172.21.15.139:0/4283878011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f688809ad40 0x7f6888161e00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f68880725c0 tx=0x7f688002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df56b92defd786e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.746+0000 7f68867fc640 1 -- 172.21.15.139:0/4283878011 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880036070 con 0x7f688809ad40 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.746+0000 7f68867fc640 1 -- 172.21.15.139:0/4283878011 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6880031600 con 0x7f688809ad40 2024-04-25T09:54:30.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.746+0000 7f68867fc640 1 -- 172.21.15.139:0/4283878011 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68800319f0 con 0x7f688809ad40 2024-04-25T09:54:30.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.747+0000 7f688e038640 1 -- 172.21.15.139:0/4283878011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f688809ad40 msgr2=0x7f6888161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.747+0000 7f688e038640 1 --2- 172.21.15.139:0/4283878011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f688809ad40 0x7f6888161e00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f68880725c0 tx=0x7f688002a7c0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.747+0000 7f688e038640 1 -- 172.21.15.139:0/4283878011 shutdown_connections 2024-04-25T09:54:30.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.747+0000 7f688e038640 1 --2- 172.21.15.139:0/4283878011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f688809ad40 0x7f6888161e00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.747+0000 7f688e038640 1 --2- 172.21.15.139:0/4283878011 >> v2:172.21.15.150:3300/0 conn(0x7f68880d0a80 0x7f68881618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.747+0000 7f688e038640 1 -- 172.21.15.139:0/4283878011 >> 172.21.15.139:0/4283878011 conn(0x7f68880b31f0 msgr2=0x7f68880b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:30.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.747+0000 7f688e038640 1 -- 172.21.15.139:0/4283878011 shutdown_connections 2024-04-25T09:54:30.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.747+0000 7f688e038640 1 -- 172.21.15.139:0/4283878011 wait complete. 2024-04-25T09:54:30.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.748+0000 7f688e038640 1 Processor -- start 2024-04-25T09:54:30.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.748+0000 7f688e038640 1 -- start start 2024-04-25T09:54:30.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f688e038640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f688809ad40 0x7f68880b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f688e038640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68880cd610 0x7f68880706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f688e038640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68880d0a80 0x7f688814d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f688e038640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68880cd9f0 con 0x7f68880d0a80 2024-04-25T09:54:30.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f688e038640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688809a820 con 0x7f68880cd610 2024-04-25T09:54:30.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f688e038640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68880cdb70 con 0x7f688809ad40 2024-04-25T09:54:30.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f6887fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68880d0a80 0x7f688814d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f6887fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68880d0a80 0x7f688814d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41222/0 (socket says 172.21.15.139:41222) 2024-04-25T09:54:30.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f6887fff640 1 -- 172.21.15.139:0/3079952302 learned_addr learned my addr 172.21.15.139:0/3079952302 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:30.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f6886ffd640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68880cd610 0x7f68880706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f68877fe640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f688809ad40 0x7f68880b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f6887fff640 1 -- 172.21.15.139:0/3079952302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f688809ad40 msgr2=0x7f68880b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f6887fff640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f688809ad40 0x7f68880b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f6887fff640 1 -- 172.21.15.139:0/3079952302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68880cd610 msgr2=0x7f68880706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f6887fff640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68880cd610 0x7f68880706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f6887fff640 1 -- 172.21.15.139:0/3079952302 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68880b62d0 con 0x7f68880d0a80 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.749+0000 7f6887fff640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68880d0a80 0x7f688814d630 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6880009f40 tx=0x7f6880004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.750+0000 7f686ffff640 1 -- 172.21.15.139:0/3079952302 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880036050 con 0x7f68880d0a80 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.750+0000 7f686ffff640 1 -- 172.21.15.139:0/3079952302 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6880037040 con 0x7f68880d0a80 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.750+0000 7f686ffff640 1 -- 172.21.15.139:0/3079952302 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880009510 con 0x7f68880d0a80 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.750+0000 7f688e038640 1 -- 172.21.15.139:0/3079952302 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68880b65b0 con 0x7f68880d0a80 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.750+0000 7f688e038640 1 -- 172.21.15.139:0/3079952302 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68880b6b70 con 0x7f68880d0a80 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.751+0000 7f688e038640 1 -- 172.21.15.139:0/3079952302 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6888074520 con 0x7f68880d0a80 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.751+0000 7f686ffff640 1 -- 172.21.15.139:0/3079952302 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6880044020 con 0x7f68880d0a80 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.751+0000 7f686ffff640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f687406b4d0 0x7f687406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.752+0000 7f686ffff640 1 -- 172.21.15.139:0/3079952302 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68800aa5d0 con 0x7f68880d0a80 2024-04-25T09:54:30.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.752+0000 7f68877fe640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f687406b4d0 0x7f687406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.752+0000 7f68877fe640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f687406b4d0 0x7f687406d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68880d1220 tx=0x7f6878009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.756+0000 7f686ffff640 1 -- 172.21.15.139:0/3079952302 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f688004b020 con 0x7f68880d0a80 2024-04-25T09:54:30.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.939+0000 7f688e038640 1 -- 172.21.15.139:0/3079952302 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f68880b3f70 con 0x7f68880d0a80 2024-04-25T09:54:30.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.939+0000 7f686ffff640 1 -- 172.21.15.139:0/3079952302 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6880078d00 con 0x7f68880d0a80 2024-04-25T09:54:30.941 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T09:54:30.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.941+0000 7f688e038640 1 -- 172.21.15.139:0/3079952302 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f687406b4d0 msgr2=0x7f687406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.941+0000 7f688e038640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f687406b4d0 0x7f687406d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68880d1220 tx=0x7f6878009040 comp rx=0 tx=0).stop 2024-04-25T09:54:30.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.941+0000 7f688e038640 1 -- 172.21.15.139:0/3079952302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68880d0a80 msgr2=0x7f688814d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.941+0000 7f688e038640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68880d0a80 0x7f688814d630 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6880009f40 tx=0x7f6880004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:30.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.941+0000 7f688e038640 1 -- 172.21.15.139:0/3079952302 shutdown_connections 2024-04-25T09:54:30.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.941+0000 7f688e038640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68880d0a80 0x7f688814d630 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.941+0000 7f688e038640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f687406b4d0 0x7f687406d990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.941+0000 7f688e038640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68880cd610 0x7f68880706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.941+0000 7f688e038640 1 --2- 172.21.15.139:0/3079952302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f688809ad40 0x7f68880b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.941+0000 7f688e038640 1 -- 172.21.15.139:0/3079952302 >> 172.21.15.139:0/3079952302 conn(0x7f68880b31f0 msgr2=0x7f68880ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:30.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.941+0000 7f688e038640 1 -- 172.21.15.139:0/3079952302 shutdown_connections 2024-04-25T09:54:30.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:30.941+0000 7f688e038640 1 -- 172.21.15.139:0/3079952302 wait complete. 2024-04-25T09:54:30.954 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:54:30.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:30.955 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:30.998 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:54:30.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:30.999 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:54:31.053 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:31.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.187+0000 7f23dfee1640 1 Processor -- start 2024-04-25T09:54:31.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.188+0000 7f23dfee1640 1 -- start start 2024-04-25T09:54:31.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.188+0000 7f23dfee1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f23d815b5e0 0x7f23d815df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.188+0000 7f23dfee1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f23d80b2db0 0x7f23d80b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.188+0000 7f23dfee1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d80b3b80 con 0x7f23d80b3770 2024-04-25T09:54:31.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.188+0000 7f23dfee1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d806da20 con 0x7f23d815b5e0 2024-04-25T09:54:31.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.188+0000 7f23dfee1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d806f860 con 0x7f23d80b2db0 2024-04-25T09:54:31.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.188+0000 7f23ddc7d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f23d815b5e0 0x7f23d815df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.189+0000 7f23ddc7d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f23d815b5e0 0x7f23d815df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41238/0 (socket says 172.21.15.139:41238) 2024-04-25T09:54:31.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.189+0000 7f23ddc7d640 1 -- 172.21.15.139:0/2469471454 learned_addr learned my addr 172.21.15.139:0/2469471454 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:31.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.189+0000 7f23ddc7d640 1 -- 172.21.15.139:0/2469471454 >> v2:172.21.15.150:3300/0 conn(0x7f23d80b2db0 msgr2=0x7f23d80b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.189+0000 7f23ddc7d640 1 --2- 172.21.15.139:0/2469471454 >> v2:172.21.15.150:3300/0 conn(0x7f23d80b2db0 0x7f23d80b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.189+0000 7f23ddc7d640 1 -- 172.21.15.139:0/2469471454 >> v1:172.21.15.150:6789/0 conn(0x7f23d80b3770 legacy=0x7f23d8161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.189+0000 7f23ddc7d640 1 -- 172.21.15.139:0/2469471454 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d806dc50 con 0x7f23d815b5e0 2024-04-25T09:54:31.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.189+0000 7f23ddc7d640 1 --2- 172.21.15.139:0/2469471454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d815b5e0 0x7f23d815df20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f23d80cc6a0 tx=0x7f23d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0543babf3060a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.190+0000 7f23dcc7b640 1 -- 172.21.15.139:0/2469471454 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4037070 con 0x7f23d815b5e0 2024-04-25T09:54:31.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.190+0000 7f23dcc7b640 1 -- 172.21.15.139:0/2469471454 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d4032440 con 0x7f23d815b5e0 2024-04-25T09:54:31.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.190+0000 7f23dcc7b640 1 -- 172.21.15.139:0/2469471454 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4032810 con 0x7f23d815b5e0 2024-04-25T09:54:31.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.190+0000 7f23dfee1640 1 -- 172.21.15.139:0/2469471454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d815b5e0 msgr2=0x7f23d815df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.190+0000 7f23dfee1640 1 --2- 172.21.15.139:0/2469471454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d815b5e0 0x7f23d815df20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f23d80cc6a0 tx=0x7f23d402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.190+0000 7f23dfee1640 1 -- 172.21.15.139:0/2469471454 shutdown_connections 2024-04-25T09:54:31.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.190+0000 7f23dfee1640 1 --2- 172.21.15.139:0/2469471454 >> v2:172.21.15.150:3300/0 conn(0x7f23d80b2db0 0x7f23d80b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.190+0000 7f23dfee1640 1 --2- 172.21.15.139:0/2469471454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d815b5e0 0x7f23d815df20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.190+0000 7f23dfee1640 1 -- 172.21.15.139:0/2469471454 >> 172.21.15.139:0/2469471454 conn(0x7f23d80b23d0 msgr2=0x7f23d80cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:31.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.190+0000 7f23dfee1640 1 -- 172.21.15.139:0/2469471454 shutdown_connections 2024-04-25T09:54:31.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.190+0000 7f23dfee1640 1 -- 172.21.15.139:0/2469471454 wait complete. 2024-04-25T09:54:31.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.191+0000 7f23dfee1640 1 Processor -- start 2024-04-25T09:54:31.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.191+0000 7f23dfee1640 1 -- start start 2024-04-25T09:54:31.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23dfee1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23d80b2db0 0x7f23d80cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23dfee1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23d80b3770 0x7f23d80cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23dfee1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d815b5e0 0x7f23d814d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23dfee1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d806e4f0 con 0x7f23d815b5e0 2024-04-25T09:54:31.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23dfee1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d806e210 con 0x7f23d80b2db0 2024-04-25T09:54:31.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23dfee1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d806e7d0 con 0x7f23d80b3770 2024-04-25T09:54:31.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23de47e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d815b5e0 0x7f23d814d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23de47e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d815b5e0 0x7f23d814d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41248/0 (socket says 172.21.15.139:41248) 2024-04-25T09:54:31.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23de47e640 1 -- 172.21.15.139:0/777109749 learned_addr learned my addr 172.21.15.139:0/777109749 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:31.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23ddc7d640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23d80b2db0 0x7f23d80cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23dd47c640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23d80b3770 0x7f23d80cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23de47e640 1 -- 172.21.15.139:0/777109749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23d80b3770 msgr2=0x7f23d80cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23de47e640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23d80b3770 0x7f23d80cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23de47e640 1 -- 172.21.15.139:0/777109749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23d80b2db0 msgr2=0x7f23d80cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.192+0000 7f23de47e640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23d80b2db0 0x7f23d80cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.193+0000 7f23de47e640 1 -- 172.21.15.139:0/777109749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d80cf5c0 con 0x7f23d815b5e0 2024-04-25T09:54:31.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.193+0000 7f23dd47c640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23d80b3770 0x7f23d80cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:31.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.193+0000 7f23de47e640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d815b5e0 0x7f23d814d3e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f23d000c780 tx=0x7f23d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.193+0000 7f23ce7fc640 1 -- 172.21.15.139:0/777109749 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0017070 con 0x7f23d815b5e0 2024-04-25T09:54:31.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.193+0000 7f23ce7fc640 1 -- 172.21.15.139:0/777109749 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d0003da0 con 0x7f23d815b5e0 2024-04-25T09:54:31.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.193+0000 7f23dfee1640 1 -- 172.21.15.139:0/777109749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d814e4f0 con 0x7f23d815b5e0 2024-04-25T09:54:31.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.193+0000 7f23ce7fc640 1 -- 172.21.15.139:0/777109749 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d00052b0 con 0x7f23d815b5e0 2024-04-25T09:54:31.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.193+0000 7f23dfee1640 1 -- 172.21.15.139:0/777109749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d814eab0 con 0x7f23d815b5e0 2024-04-25T09:54:31.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.194+0000 7f23dfee1640 1 -- 172.21.15.139:0/777109749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d80cea80 con 0x7f23d815b5e0 2024-04-25T09:54:31.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.194+0000 7f23ce7fc640 1 -- 172.21.15.139:0/777109749 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23d001a070 con 0x7f23d815b5e0 2024-04-25T09:54:31.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.194+0000 7f23ce7fc640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f23bc06b4d0 0x7f23bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.195+0000 7f23ce7fc640 1 -- 172.21.15.139:0/777109749 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f23d00899c0 con 0x7f23d815b5e0 2024-04-25T09:54:31.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.195+0000 7f23ddc7d640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f23bc06b4d0 0x7f23bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.195+0000 7f23ddc7d640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f23bc06b4d0 0x7f23bc06d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f23d4034000 tx=0x7f23d402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.199+0000 7f23ce7fc640 1 -- 172.21.15.139:0/777109749 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f23d80cea80 con 0x7f23d815b5e0 2024-04-25T09:54:31.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.404+0000 7f23dfee1640 1 -- 172.21.15.139:0/777109749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23d8074520 con 0x7f23d815b5e0 2024-04-25T09:54:31.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.405+0000 7f23ce7fc640 1 -- 172.21.15.139:0/777109749 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f23d008be20 con 0x7f23d815b5e0 2024-04-25T09:54:31.407 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:31.407 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/511562594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":511562594},{"type":"v1","addr":"172.21.15.150:6834","nonce":511562594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5041,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/225357317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":225357317},{"type":"v1","addr":"172.21.15.150:6835","nonce":225357317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2499407982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2499407982},{"type":"v1","addr":"172.21.15.139:6836","nonce":2499407982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:21.988975+0000","modified":"2024-04-25T09:54:23.955620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/2069784668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2069784668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2069784668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:54:31.407 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 16 2024-04-25T09:54:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.406+0000 7f23dfee1640 1 -- 172.21.15.139:0/777109749 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f23bc06b4d0 msgr2=0x7f23bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.406+0000 7f23dfee1640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f23bc06b4d0 0x7f23bc06d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f23d4034000 tx=0x7f23d402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.406+0000 7f23dfee1640 1 -- 172.21.15.139:0/777109749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d815b5e0 msgr2=0x7f23d814d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.406+0000 7f23dfee1640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d815b5e0 0x7f23d814d3e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f23d000c780 tx=0x7f23d000cc40 comp rx=0 tx=0).stop 2024-04-25T09:54:31.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.406+0000 7f23dfee1640 1 -- 172.21.15.139:0/777109749 shutdown_connections 2024-04-25T09:54:31.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.406+0000 7f23dfee1640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d815b5e0 0x7f23d814d3e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.406+0000 7f23dfee1640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f23bc06b4d0 0x7f23bc06d990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.406+0000 7f23dfee1640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f23d80b3770 0x7f23d80cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.406+0000 7f23dfee1640 1 --2- 172.21.15.139:0/777109749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23d80b2db0 0x7f23d80cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.407+0000 7f23dfee1640 1 -- 172.21.15.139:0/777109749 >> 172.21.15.139:0/777109749 conn(0x7f23d80b23d0 msgr2=0x7f23d815df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:31.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.407+0000 7f23dfee1640 1 -- 172.21.15.139:0/777109749 shutdown_connections 2024-04-25T09:54:31.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.407+0000 7f23dfee1640 1 -- 172.21.15.139:0/777109749 wait complete. 2024-04-25T09:54:31.420 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T09:54:31.544 INFO:teuthology.orchestra.run.smithi139.stdout:{"message":"","return_code":0} 2024-04-25T09:54:31.544 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:54:31.544 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:31.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.639+0000 7f78474ee640 1 Processor -- start 2024-04-25T09:54:31.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.639+0000 7f78474ee640 1 -- start start 2024-04-25T09:54:31.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.640+0000 7f78474ee640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7840160e90 0x7f78400afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.640+0000 7f78474ee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784006aff0 0x7f7840070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.640+0000 7f78474ee640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784006b480 con 0x7f784006ab60 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.640+0000 7f78474ee640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78400d14a0 con 0x7f7840160e90 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.640+0000 7f78474ee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78400d0400 con 0x7f784006aff0 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.640+0000 7f784528a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7840160e90 0x7f78400afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.640+0000 7f784528a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7840160e90 0x7f78400afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41264/0 (socket says 172.21.15.139:41264) 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.640+0000 7f784528a640 1 -- 172.21.15.139:0/1541383658 learned_addr learned my addr 172.21.15.139:0/1541383658 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.641+0000 7f7845a8b640 1 --2- 172.21.15.139:0/1541383658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784006aff0 0x7f7840070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.641+0000 7f784528a640 1 -- 172.21.15.139:0/1541383658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784006aff0 msgr2=0x7f7840070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.641+0000 7f784528a640 1 --2- 172.21.15.139:0/1541383658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784006aff0 0x7f7840070180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.641+0000 7f784528a640 1 -- 172.21.15.139:0/1541383658 >> v1:172.21.15.150:6789/0 conn(0x7f784006ab60 legacy=0x7f784006fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.641+0000 7f784528a640 1 -- 172.21.15.139:0/1541383658 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400d06e0 con 0x7f7840160e90 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.641+0000 7f7845a8b640 1 --2- 172.21.15.139:0/1541383658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784006aff0 0x7f7840070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:31.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.641+0000 7f784528a640 1 --2- 172.21.15.139:0/1541383658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7840160e90 0x7f78400afac0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f78380098a0 tx=0x7f783802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d33b651c7744fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.641+0000 7f783ffff640 1 -- 172.21.15.139:0/1541383658 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838037070 con 0x7f7840160e90 2024-04-25T09:54:31.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.641+0000 7f783ffff640 1 -- 172.21.15.139:0/1541383658 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7838032440 con 0x7f7840160e90 2024-04-25T09:54:31.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.641+0000 7f783ffff640 1 -- 172.21.15.139:0/1541383658 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838032810 con 0x7f7840160e90 2024-04-25T09:54:31.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.642+0000 7f78474ee640 1 -- 172.21.15.139:0/1541383658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7840160e90 msgr2=0x7f78400afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.642+0000 7f78474ee640 1 --2- 172.21.15.139:0/1541383658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7840160e90 0x7f78400afac0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f78380098a0 tx=0x7f783802a250 comp rx=0 tx=0).stop 2024-04-25T09:54:31.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.642+0000 7f78474ee640 1 -- 172.21.15.139:0/1541383658 shutdown_connections 2024-04-25T09:54:31.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.642+0000 7f78474ee640 1 --2- 172.21.15.139:0/1541383658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784006aff0 0x7f7840070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.642+0000 7f78474ee640 1 --2- 172.21.15.139:0/1541383658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7840160e90 0x7f78400afac0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.642+0000 7f78474ee640 1 -- 172.21.15.139:0/1541383658 >> 172.21.15.139:0/1541383658 conn(0x7f78400b23f0 msgr2=0x7f78400b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:31.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.642+0000 7f78474ee640 1 -- 172.21.15.139:0/1541383658 shutdown_connections 2024-04-25T09:54:31.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.642+0000 7f78474ee640 1 -- 172.21.15.139:0/1541383658 wait complete. 2024-04-25T09:54:31.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.643+0000 7f78474ee640 1 Processor -- start 2024-04-25T09:54:31.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.643+0000 7f78474ee640 1 -- start start 2024-04-25T09:54:31.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f78474ee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784006ab60 0x7f7840149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f78474ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784006aff0 0x7f78400ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f78474ee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7840160e90 0x7f78400ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f78474ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78400d0f80 con 0x7f784006aff0 2024-04-25T09:54:31.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f78474ee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78400d0ca0 con 0x7f7840160e90 2024-04-25T09:54:31.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f78474ee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78400b0000 con 0x7f784006ab60 2024-04-25T09:54:31.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f7844a89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784006aff0 0x7f78400ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f7844a89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784006aff0 0x7f78400ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41272/0 (socket says 172.21.15.139:41272) 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f7844a89640 1 -- 172.21.15.139:0/2041772474 learned_addr learned my addr 172.21.15.139:0/2041772474 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f784528a640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784006ab60 0x7f7840149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f7845a8b640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7840160e90 0x7f78400ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f7844a89640 1 -- 172.21.15.139:0/2041772474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784006ab60 msgr2=0x7f7840149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f7844a89640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784006ab60 0x7f7840149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f7844a89640 1 -- 172.21.15.139:0/2041772474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7840160e90 msgr2=0x7f78400ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f7844a89640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7840160e90 0x7f78400ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.644+0000 7f7844a89640 1 -- 172.21.15.139:0/2041772474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400caf00 con 0x7f784006aff0 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.645+0000 7f7844a89640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784006aff0 0x7f78400ca190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f783400c430 tx=0x7f783400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.645+0000 7f783dffb640 1 -- 172.21.15.139:0/2041772474 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7834011790 con 0x7f784006aff0 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.645+0000 7f783dffb640 1 -- 172.21.15.139:0/2041772474 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7834017070 con 0x7f784006aff0 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.645+0000 7f78474ee640 1 -- 172.21.15.139:0/2041772474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78400cb240 con 0x7f784006aff0 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.645+0000 7f783dffb640 1 -- 172.21.15.139:0/2041772474 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7834011bb0 con 0x7f784006aff0 2024-04-25T09:54:31.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.646+0000 7f78474ee640 1 -- 172.21.15.139:0/2041772474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78400cba80 con 0x7f784006aff0 2024-04-25T09:54:31.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.647+0000 7f78474ee640 1 -- 172.21.15.139:0/2041772474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7840067510 con 0x7f784006aff0 2024-04-25T09:54:31.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.647+0000 7f783dffb640 1 -- 172.21.15.139:0/2041772474 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f783401c460 con 0x7f784006aff0 2024-04-25T09:54:31.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.647+0000 7f783dffb640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f782c06b4d0 0x7f782c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.648+0000 7f783dffb640 1 -- 172.21.15.139:0/2041772474 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f783408a0a0 con 0x7f784006aff0 2024-04-25T09:54:31.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.648+0000 7f784528a640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f782c06b4d0 0x7f782c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.648+0000 7f784528a640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f782c06b4d0 0x7f782c06d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7838034000 tx=0x7f783802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.653+0000 7f783dffb640 1 -- 172.21.15.139:0/2041772474 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7840067510 con 0x7f784006aff0 2024-04-25T09:54:31.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.858+0000 7f78474ee640 1 -- 172.21.15.139:0/2041772474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78400cac70 con 0x7f784006aff0 2024-04-25T09:54:31.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.859+0000 7f783dffb640 1 -- 172.21.15.139:0/2041772474 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f78400cb240 con 0x7f784006aff0 2024-04-25T09:54:31.861 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:31.861 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/511562594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":511562594},{"type":"v1","addr":"172.21.15.150:6834","nonce":511562594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5041,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/225357317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":225357317},{"type":"v1","addr":"172.21.15.150:6835","nonce":225357317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2499407982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2499407982},{"type":"v1","addr":"172.21.15.139:6836","nonce":2499407982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:21.988975+0000","modified":"2024-04-25T09:54:23.955620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/2069784668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2069784668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2069784668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:54:31.861 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 16 2024-04-25T09:54:31.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.861+0000 7f78474ee640 1 -- 172.21.15.139:0/2041772474 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f782c06b4d0 msgr2=0x7f782c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.861+0000 7f78474ee640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f782c06b4d0 0x7f782c06d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7838034000 tx=0x7f783802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:31.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.861+0000 7f78474ee640 1 -- 172.21.15.139:0/2041772474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784006aff0 msgr2=0x7f78400ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.861+0000 7f78474ee640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784006aff0 0x7f78400ca190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f783400c430 tx=0x7f783400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.861+0000 7f78474ee640 1 -- 172.21.15.139:0/2041772474 shutdown_connections 2024-04-25T09:54:31.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.861+0000 7f78474ee640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f784006aff0 0x7f78400ca190 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.861+0000 7f78474ee640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f782c06b4d0 0x7f782c06d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.861+0000 7f78474ee640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7840160e90 0x7f78400ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.861+0000 7f78474ee640 1 --2- 172.21.15.139:0/2041772474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f784006ab60 0x7f7840149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.861+0000 7f78474ee640 1 -- 172.21.15.139:0/2041772474 >> 172.21.15.139:0/2041772474 conn(0x7f78400b23f0 msgr2=0x7f78400b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:31.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.861+0000 7f78474ee640 1 -- 172.21.15.139:0/2041772474 shutdown_connections 2024-04-25T09:54:31.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:31.861+0000 7f78474ee640 1 -- 172.21.15.139:0/2041772474 wait complete. 2024-04-25T09:54:31.874 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2024-04-25T09:54:31.988 INFO:teuthology.orchestra.run.smithi139.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-25T09:54:31.988 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-25T09:54:41.990 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-25T09:54:42.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.037+0000 7f6887578f40 1 Processor -- start 2024-04-25T09:54:42.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.037+0000 7f6887578f40 1 -- start start 2024-04-25T09:54:42.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.037+0000 7f6887578f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616ababecc0 0x5616abb8f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.037+0000 7f6887578f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616abb97630 0x5616abb916b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.037+0000 7f6887578f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 0x5616abb960c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.037+0000 7f6887578f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616aba65df0 con 0x5616abb992b0 2024-04-25T09:54:42.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.037+0000 7f6887578f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616aba69d50 con 0x5616abb97630 2024-04-25T09:54:42.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.037+0000 7f6887578f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616abaf05f0 con 0x5616ababecc0 2024-04-25T09:54:42.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f688587a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 0x5616abb960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f688587a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 0x5616abb960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58298/0 (socket says 172.21.15.139:58298) 2024-04-25T09:54:42.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f688587a640 1 -- 172.21.15.139:0/2911725058 learned_addr learned my addr 172.21.15.139:0/2911725058 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:42.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f6885079640 1 --2- 172.21.15.139:0/2911725058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616ababecc0 0x5616abb8f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f687ffff640 1 --2- 172.21.15.139:0/2911725058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616abb97630 0x5616abb916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f688587a640 1 -- 172.21.15.139:0/2911725058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616ababecc0 msgr2=0x5616abb8f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f688587a640 1 --2- 172.21.15.139:0/2911725058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616ababecc0 0x5616abb8f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f688587a640 1 -- 172.21.15.139:0/2911725058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616abb97630 msgr2=0x5616abb916b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f688587a640 1 --2- 172.21.15.139:0/2911725058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616abb97630 0x5616abb916b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f688587a640 1 -- 172.21.15.139:0/2911725058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616abb8ff90 con 0x5616abb992b0 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f6885079640 1 --2- 172.21.15.139:0/2911725058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616ababecc0 0x5616abb8f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f688587a640 1 --2- 172.21.15.139:0/2911725058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 0x5616abb960c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6880001520 tx=0x7f68800019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3322847971f93e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f687f7fe640 1 -- 172.21.15.139:0/2911725058 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880016070 con 0x5616abb992b0 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f687f7fe640 1 -- 172.21.15.139:0/2911725058 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688000f730 con 0x5616abb992b0 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.038+0000 7f687f7fe640 1 -- 172.21.15.139:0/2911725058 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688000fb20 con 0x5616abb992b0 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.039+0000 7f6887578f40 1 -- 172.21.15.139:0/2911725058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 msgr2=0x5616abb960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.039+0000 7f6887578f40 1 --2- 172.21.15.139:0/2911725058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 0x5616abb960c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6880001520 tx=0x7f68800019e0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.039+0000 7f6887578f40 1 -- 172.21.15.139:0/2911725058 shutdown_connections 2024-04-25T09:54:42.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.039+0000 7f6887578f40 1 --2- 172.21.15.139:0/2911725058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 0x5616abb960c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.039+0000 7f6887578f40 1 --2- 172.21.15.139:0/2911725058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616abb97630 0x5616abb916b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.039+0000 7f6887578f40 1 --2- 172.21.15.139:0/2911725058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616ababecc0 0x5616abb8f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.039+0000 7f6887578f40 1 -- 172.21.15.139:0/2911725058 >> 172.21.15.139:0/2911725058 conn(0x5616abb0c780 msgr2=0x5616abb932c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:42.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.039+0000 7f6887578f40 1 -- 172.21.15.139:0/2911725058 shutdown_connections 2024-04-25T09:54:42.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.039+0000 7f6887578f40 1 -- 172.21.15.139:0/2911725058 wait complete. 2024-04-25T09:54:42.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.040+0000 7f6887578f40 1 Processor -- start 2024-04-25T09:54:42.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.040+0000 7f6887578f40 1 -- start start 2024-04-25T09:54:42.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.040+0000 7f6887578f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616ababecc0 0x5616abb95100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.040+0000 7f6887578f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616abb97630 0x5616ababd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.040+0000 7f6887578f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 0x5616abc1e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.040+0000 7f6887578f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616abb90bd0 con 0x5616abb992b0 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.040+0000 7f6887578f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616abb91220 con 0x5616ababecc0 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.040+0000 7f6887578f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616abb90eb0 con 0x5616abb97630 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.040+0000 7f688587a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 0x5616abc1e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.040+0000 7f688587a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 0x5616abc1e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58302/0 (socket says 172.21.15.139:58302) 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.040+0000 7f688587a640 1 -- 172.21.15.139:0/2752608607 learned_addr learned my addr 172.21.15.139:0/2752608607 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f687ffff640 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616abb97630 0x5616ababd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f6885079640 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616ababecc0 0x5616abb95100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f688587a640 1 -- 172.21.15.139:0/2752608607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616abb97630 msgr2=0x5616ababd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f688587a640 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5616abb97630 0x5616ababd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f688587a640 1 -- 172.21.15.139:0/2752608607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616ababecc0 msgr2=0x5616abb95100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f688587a640 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5616ababecc0 0x5616abb95100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f688587a640 1 -- 172.21.15.139:0/2752608607 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616abb96a80 con 0x5616abb992b0 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f688587a640 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 0x5616abc1e5b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5616abb0bfe0 tx=0x7f6880000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f687e7fc640 1 -- 172.21.15.139:0/2752608607 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880016040 con 0x5616abb992b0 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f687e7fc640 1 -- 172.21.15.139:0/2752608607 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6880011660 con 0x5616abb992b0 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f6887578f40 1 -- 172.21.15.139:0/2752608607 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5616abb959e0 con 0x5616abb992b0 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f687e7fc640 1 -- 172.21.15.139:0/2752608607 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880011a50 con 0x5616abb992b0 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.041+0000 7f6887578f40 1 -- 172.21.15.139:0/2752608607 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616abb95fa0 con 0x5616abb992b0 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.043+0000 7f687e7fc640 1 -- 172.21.15.139:0/2752608607 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6880011bf0 con 0x5616abb992b0 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.043+0000 7f687e7fc640 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6868069eb0 0x7f686806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.043+0000 7f687e7fc640 1 -- 172.21.15.139:0/2752608607 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f688008a780 con 0x5616abb992b0 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.043+0000 7f6885079640 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6868069eb0 0x7f686806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.043+0000 7f6887578f40 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x5616ababd700 0x5616abc21f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.043+0000 7f6887578f40 1 -- 172.21.15.139:0/2752608607 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) v9 -- 0x5616abb8c0d0 con 0x5616ababd700 2024-04-25T09:54:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.044+0000 7f687ffff640 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x5616ababd700 0x5616abc21f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.044+0000 7f6885079640 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6868069eb0 0x7f686806c370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5616abb01860 tx=0x7f68780017b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.044+0000 7f687ffff640 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x5616ababd700 0x5616abc21f50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.044+0000 7f687ffff640 1 -- 172.21.15.139:0/2752608607 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f687005e7e0 con 0x5616ababd700 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.044+0000 7f6887578f40 1 -- 172.21.15.139:0/2752608607 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x5616ababd700 msgr2=0x5616abc21f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.044+0000 7f6887578f40 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x5616ababd700 0x5616abc21f50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.044+0000 7f6887578f40 1 -- 172.21.15.139:0/2752608607 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6868069eb0 msgr2=0x7f686806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.044+0000 7f6887578f40 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6868069eb0 0x7f686806c370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5616abb01860 tx=0x7f68780017b0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.044+0000 7f6887578f40 1 -- 172.21.15.139:0/2752608607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 msgr2=0x5616abc1e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.044+0000 7f6887578f40 1 --2- 172.21.15.139:0/2752608607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5616abb992b0 0x5616abc1e5b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5616abb0bfe0 tx=0x7f6880000f40 comp rx=0 tx=0).stop 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.044+0000 7f688587a640 1 -- 172.21.15.139:0/2752608607 reap_dead start 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.045+0000 7f6887578f40 1 -- 172.21.15.139:0/2752608607 shutdown_connections 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.045+0000 7f6887578f40 1 -- 172.21.15.139:0/2752608607 >> 172.21.15.139:0/2752608607 conn(0x5616abb0c780 msgr2=0x5616abb916b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.045+0000 7f6887578f40 1 -- 172.21.15.139:0/2752608607 shutdown_connections 2024-04-25T09:54:42.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.045+0000 7f6887578f40 1 -- 172.21.15.139:0/2752608607 wait complete. 2024-04-25T09:54:42.052 INFO:teuthology.orchestra.run.smithi139.stdout:mytag 2024-04-25T09:54:42.052 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-25T09:54:42.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.095+0000 7fb0b773cf40 1 Processor -- start 2024-04-25T09:54:42.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.095+0000 7fb0b773cf40 1 -- start start 2024-04-25T09:54:42.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b773cf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d326db580 0x562d327ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b773cf40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d32720430 0x562d327a6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b773cf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562d327aa530 0x562d327966c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b773cf40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d32720b40 con 0x562d32720430 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b773cf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3267fdf0 con 0x562d326db580 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b773cf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d32720de0 con 0x562d327aa530 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b4a3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d32720430 0x562d327a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b4a3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d32720430 0x562d327a6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58312/0 (socket says 172.21.15.139:58312) 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b4a3c640 1 -- 172.21.15.139:0/1009794680 learned_addr learned my addr 172.21.15.139:0/1009794680 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b5a3e640 1 --2- 172.21.15.139:0/1009794680 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562d327aa530 0x562d327966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b523d640 1 --2- 172.21.15.139:0/1009794680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d326db580 0x562d327ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b4a3c640 1 -- 172.21.15.139:0/1009794680 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562d327aa530 msgr2=0x562d327966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b4a3c640 1 --2- 172.21.15.139:0/1009794680 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562d327aa530 0x562d327966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b4a3c640 1 -- 172.21.15.139:0/1009794680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d326db580 msgr2=0x562d327ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b4a3c640 1 --2- 172.21.15.139:0/1009794680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d326db580 0x562d327ae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b4a3c640 1 -- 172.21.15.139:0/1009794680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d32605c50 con 0x562d32720430 2024-04-25T09:54:42.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.096+0000 7fb0b523d640 1 --2- 172.21.15.139:0/1009794680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d326db580 0x562d327ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:42.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0b4a3c640 1 --2- 172.21.15.139:0/1009794680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d32720430 0x562d327a6450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0098e0 tx=0x7fb0ac005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b6982c64a72592 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0a77fe640 1 -- 172.21.15.139:0/1009794680 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac008950 con 0x562d32720430 2024-04-25T09:54:42.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0a77fe640 1 -- 172.21.15.139:0/1009794680 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac018070 con 0x562d32720430 2024-04-25T09:54:42.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0a77fe640 1 -- 172.21.15.139:0/1009794680 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac008d30 con 0x562d32720430 2024-04-25T09:54:42.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0b773cf40 1 -- 172.21.15.139:0/1009794680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d32720430 msgr2=0x562d327a6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0b773cf40 1 --2- 172.21.15.139:0/1009794680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d32720430 0x562d327a6450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0098e0 tx=0x7fb0ac005a00 comp rx=0 tx=0).stop 2024-04-25T09:54:42.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0b773cf40 1 -- 172.21.15.139:0/1009794680 shutdown_connections 2024-04-25T09:54:42.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0b773cf40 1 --2- 172.21.15.139:0/1009794680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d32720430 0x562d327a6450 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0b773cf40 1 --2- 172.21.15.139:0/1009794680 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562d327aa530 0x562d327966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0b773cf40 1 --2- 172.21.15.139:0/1009794680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d326db580 0x562d327ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0b773cf40 1 -- 172.21.15.139:0/1009794680 >> 172.21.15.139:0/1009794680 conn(0x562d326d8be0 msgr2=0x562d327af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0b773cf40 1 -- 172.21.15.139:0/1009794680 shutdown_connections 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.097+0000 7fb0b773cf40 1 -- 172.21.15.139:0/1009794680 wait complete. 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.098+0000 7fb0b773cf40 1 Processor -- start 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.098+0000 7fb0b773cf40 1 -- start start 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b773cf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d326db580 0x562d32793d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b773cf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562d32720430 0x562d3278fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b773cf40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d327aa530 0x562d32796c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b773cf40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d32799330 con 0x562d327aa530 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b773cf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d32799090 con 0x562d326db580 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b773cf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d327995d0 con 0x562d32720430 2024-04-25T09:54:42.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b5a3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d327aa530 0x562d32796c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b5a3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d327aa530 0x562d32796c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58316/0 (socket says 172.21.15.139:58316) 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b5a3e640 1 -- 172.21.15.139:0/3046857829 learned_addr learned my addr 172.21.15.139:0/3046857829 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b4a3c640 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562d32720430 0x562d3278fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b523d640 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d326db580 0x562d32793d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b5a3e640 1 -- 172.21.15.139:0/3046857829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562d32720430 msgr2=0x562d3278fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b5a3e640 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562d32720430 0x562d3278fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b5a3e640 1 -- 172.21.15.139:0/3046857829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d326db580 msgr2=0x562d32793d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b5a3e640 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562d326db580 0x562d32793d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b5a3e640 1 -- 172.21.15.139:0/3046857829 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d326da7c0 con 0x562d327aa530 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b5a3e640 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d327aa530 0x562d32796c90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0057040 tx=0x7fb0b0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0a67fc640 1 -- 172.21.15.139:0/3046857829 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0063030 con 0x562d327aa530 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0a67fc640 1 -- 172.21.15.139:0/3046857829 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b00548b0 con 0x562d327aa530 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0a67fc640 1 -- 172.21.15.139:0/3046857829 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0059040 con 0x562d327aa530 2024-04-25T09:54:42.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b773cf40 1 -- 172.21.15.139:0/3046857829 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562d326daaa0 con 0x562d327aa530 2024-04-25T09:54:42.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.099+0000 7fb0b773cf40 1 -- 172.21.15.139:0/3046857829 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d327afe90 con 0x562d327aa530 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.101+0000 7fb0a67fc640 1 -- 172.21.15.139:0/3046857829 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0b0066030 con 0x562d327aa530 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.101+0000 7fb0a67fc640 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb098069ee0 0x7fb09806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.101+0000 7fb0a67fc640 1 -- 172.21.15.139:0/3046857829 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb0b00cc490 con 0x562d327aa530 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.101+0000 7fb0b523d640 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb098069ee0 0x7fb09806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.101+0000 7fb0b773cf40 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x562d327903b0 0x562d327971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.101+0000 7fb0b773cf40 1 -- 172.21.15.139:0/3046857829 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) v9 -- 0x562d32790810 con 0x562d327903b0 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.102+0000 7fb0b523d640 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb098069ee0 0x7fb09806c3a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8000f80 tx=0x7fb0a8006350 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.102+0000 7fb0b4a3c640 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x562d327903b0 0x562d327971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.102+0000 7fb0b4a3c640 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x562d327903b0 0x562d327971d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.103+0000 7fb0b4a3c640 1 -- 172.21.15.139:0/3046857829 <== osd.5 v2:172.21.15.150:6824/2184499879 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fb0ac0089d0 con 0x562d327903b0 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.103+0000 7fb0b773cf40 1 -- 172.21.15.139:0/3046857829 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x562d327903b0 msgr2=0x562d327971d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.103+0000 7fb0b773cf40 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x562d327903b0 0x562d327971d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.103+0000 7fb0b773cf40 1 -- 172.21.15.139:0/3046857829 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb098069ee0 msgr2=0x7fb09806c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.103+0000 7fb0b773cf40 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb098069ee0 0x7fb09806c3a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8000f80 tx=0x7fb0a8006350 comp rx=0 tx=0).stop 2024-04-25T09:54:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.103+0000 7fb0b773cf40 1 -- 172.21.15.139:0/3046857829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d327aa530 msgr2=0x562d32796c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.103+0000 7fb0b773cf40 1 --2- 172.21.15.139:0/3046857829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562d327aa530 0x562d32796c90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0057040 tx=0x7fb0b0002fb0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.103+0000 7fb0b5a3e640 1 -- 172.21.15.139:0/3046857829 reap_dead start 2024-04-25T09:54:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.103+0000 7fb0b773cf40 1 -- 172.21.15.139:0/3046857829 shutdown_connections 2024-04-25T09:54:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.103+0000 7fb0b773cf40 1 -- 172.21.15.139:0/3046857829 >> 172.21.15.139:0/3046857829 conn(0x562d326d8be0 msgr2=0x562d327a63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.103+0000 7fb0b773cf40 1 -- 172.21.15.139:0/3046857829 shutdown_connections 2024-04-25T09:54:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.103+0000 7fb0b773cf40 1 -- 172.21.15.139:0/3046857829 wait complete. 2024-04-25T09:54:42.111 INFO:teuthology.orchestra.run.smithi139.stdout:mytag 2024-04-25T09:54:42.111 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-25T09:54:42.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.152+0000 7f2704c1bf40 1 Processor -- start 2024-04-25T09:54:42.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.152+0000 7f2704c1bf40 1 -- start start 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.152+0000 7f2704c1bf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26ec0022a0 0x55cdf9932080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.152+0000 7f2704c1bf40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 0x55cdf99332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.152+0000 7f2704c1bf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cdf9929790 0x55cdf9929c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.152+0000 7f2704c1bf40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf98a7b40 con 0x55cdf9932f10 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.152+0000 7f2704c1bf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf9806df0 con 0x55cdf9929790 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.152+0000 7f2704c1bf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdf98a7de0 con 0x7f26ec0022a0 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2701f1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 0x55cdf99332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2701f1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 0x55cdf99332f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58330/0 (socket says 172.21.15.139:58330) 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2701f1d640 1 -- 172.21.15.139:0/4051438723 learned_addr learned my addr 172.21.15.139:0/4051438723 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f270271e640 1 --2- 172.21.15.139:0/4051438723 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26ec0022a0 0x55cdf9932080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2701f1d640 1 -- 172.21.15.139:0/4051438723 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26ec0022a0 msgr2=0x55cdf9932080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2702f1f640 1 --2- 172.21.15.139:0/4051438723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cdf9929790 0x55cdf9929c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2701f1d640 1 --2- 172.21.15.139:0/4051438723 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26ec0022a0 0x55cdf9932080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2701f1d640 1 -- 172.21.15.139:0/4051438723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cdf9929790 msgr2=0x55cdf9929c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2701f1d640 1 --2- 172.21.15.139:0/4051438723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cdf9929790 0x55cdf9929c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2701f1d640 1 -- 172.21.15.139:0/4051438723 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdf978cc50 con 0x55cdf9932f10 2024-04-25T09:54:42.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2701f1d640 1 --2- 172.21.15.139:0/4051438723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 0x55cdf99332f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f26f8005380 tx=0x7f26f8005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4441b1b75ade3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2700f1b640 1 -- 172.21.15.139:0/4051438723 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8007600 con 0x55cdf9932f10 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2700f1b640 1 -- 172.21.15.139:0/4051438723 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f80077a0 con 0x55cdf9932f10 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.153+0000 7f2700f1b640 1 -- 172.21.15.139:0/4051438723 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8007b70 con 0x55cdf9932f10 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.154+0000 7f2704c1bf40 1 -- 172.21.15.139:0/4051438723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 msgr2=0x55cdf99332f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.154+0000 7f2704c1bf40 1 --2- 172.21.15.139:0/4051438723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 0x55cdf99332f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f26f8005380 tx=0x7f26f8005a00 comp rx=0 tx=0).stop 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.154+0000 7f2704c1bf40 1 -- 172.21.15.139:0/4051438723 shutdown_connections 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.154+0000 7f2704c1bf40 1 --2- 172.21.15.139:0/4051438723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 0x55cdf99332f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.154+0000 7f2704c1bf40 1 --2- 172.21.15.139:0/4051438723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cdf9929790 0x55cdf9929c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.154+0000 7f2704c1bf40 1 --2- 172.21.15.139:0/4051438723 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26ec0022a0 0x55cdf9932080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.154+0000 7f2704c1bf40 1 -- 172.21.15.139:0/4051438723 >> 172.21.15.139:0/4051438723 conn(0x55cdf985fbe0 msgr2=0x55cdf9861db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.154+0000 7f2704c1bf40 1 -- 172.21.15.139:0/4051438723 shutdown_connections 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.154+0000 7f2704c1bf40 1 -- 172.21.15.139:0/4051438723 wait complete. 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.154+0000 7f2704c1bf40 1 Processor -- start 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.154+0000 7f2704c1bf40 1 -- start start 2024-04-25T09:54:42.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2704c1bf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cdf9929790 0x55cdf99c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2704c1bf40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 0x55cdf99252f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2704c1bf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26ec0022a0 0x55cdf9924370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2704c1bf40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf9932890 con 0x55cdf9932f10 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2704c1bf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf98a6470 con 0x55cdf9929790 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2704c1bf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdf988f1c0 con 0x7f26ec0022a0 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2701f1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 0x55cdf99252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2701f1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 0x55cdf99252f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58334/0 (socket says 172.21.15.139:58334) 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2701f1d640 1 -- 172.21.15.139:0/2961450836 learned_addr learned my addr 172.21.15.139:0/2961450836 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2702f1f640 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26ec0022a0 0x55cdf9924370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2701f1d640 1 -- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26ec0022a0 msgr2=0x55cdf9924370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2701f1d640 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26ec0022a0 0x55cdf9924370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2701f1d640 1 -- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cdf9929790 msgr2=0x55cdf99c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f270271e640 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cdf9929790 0x55cdf99c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2701f1d640 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cdf9929790 0x55cdf99c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2701f1d640 1 -- 172.21.15.139:0/2961450836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdf992ff20 con 0x55cdf9932f10 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2702f1f640 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26ec0022a0 0x55cdf9924370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f270271e640 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cdf9929790 0x55cdf99c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.155+0000 7f2701f1d640 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 0x55cdf99252f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f26f800c010 tx=0x7f26f8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.156+0000 7f26f37fe640 1 -- 172.21.15.139:0/2961450836 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8007600 con 0x55cdf9932f10 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.156+0000 7f26f37fe640 1 -- 172.21.15.139:0/2961450836 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f801e630 con 0x55cdf9932f10 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.156+0000 7f26f37fe640 1 -- 172.21.15.139:0/2961450836 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f801ea00 con 0x55cdf9932f10 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.156+0000 7f2704c1bf40 1 -- 172.21.15.139:0/2961450836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cdf988b690 con 0x55cdf9932f10 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.156+0000 7f2704c1bf40 1 -- 172.21.15.139:0/2961450836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdf992ef10 con 0x55cdf9932f10 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.157+0000 7f26f37fe640 1 -- 172.21.15.139:0/2961450836 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26f801b040 con 0x55cdf9932f10 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.157+0000 7f26f37fe640 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26e4069eb0 0x7f26e406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.157+0000 7f26f37fe640 1 -- 172.21.15.139:0/2961450836 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f26f808aa50 con 0x55cdf9932f10 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.157+0000 7f270271e640 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26e4069eb0 0x7f26e406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.158+0000 7f2704c1bf40 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55cdf9924ed0 0x55cdf99c1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.158+0000 7f2704c1bf40 1 -- 172.21.15.139:0/2961450836 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) v9 -- 0x55cdf98a53d0 con 0x55cdf9924ed0 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.158+0000 7f270271e640 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26e4069eb0 0x7f26e406c370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f26f4056000 tx=0x7f26f4050000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.158+0000 7f2702f1f640 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55cdf9924ed0 0x55cdf99c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.158+0000 7f2702f1f640 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55cdf9924ed0 0x55cdf99c1840 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.158+0000 7f2702f1f640 1 -- 172.21.15.139:0/2961450836 <== osd.5 v2:172.21.15.150:6824/2184499879 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f26fc00f5f0 con 0x55cdf9924ed0 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.159+0000 7f2704c1bf40 1 -- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55cdf9924ed0 msgr2=0x55cdf99c1840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.159+0000 7f2704c1bf40 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55cdf9924ed0 0x55cdf99c1840 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.159+0000 7f2704c1bf40 1 -- 172.21.15.139:0/2961450836 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26e4069eb0 msgr2=0x7f26e406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.159+0000 7f2704c1bf40 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26e4069eb0 0x7f26e406c370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f26f4056000 tx=0x7f26f4050000 comp rx=0 tx=0).stop 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.159+0000 7f2704c1bf40 1 -- 172.21.15.139:0/2961450836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 msgr2=0x55cdf99252f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.159+0000 7f2704c1bf40 1 --2- 172.21.15.139:0/2961450836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cdf9932f10 0x55cdf99252f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f26f800c010 tx=0x7f26f8002fb0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.159+0000 7f2702f1f640 1 -- 172.21.15.139:0/2961450836 reap_dead start 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.159+0000 7f2704c1bf40 1 -- 172.21.15.139:0/2961450836 shutdown_connections 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.159+0000 7f2704c1bf40 1 -- 172.21.15.139:0/2961450836 >> 172.21.15.139:0/2961450836 conn(0x55cdf985fbe0 msgr2=0x55cdf9861db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.159+0000 7f2704c1bf40 1 -- 172.21.15.139:0/2961450836 shutdown_connections 2024-04-25T09:54:42.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.159+0000 7f2704c1bf40 1 -- 172.21.15.139:0/2961450836 wait complete. 2024-04-25T09:54:42.166 INFO:teuthology.orchestra.run.smithi139.stdout:mytag 2024-04-25T09:54:42.167 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-25T09:54:42.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.247+0000 7f038eb64f40 1 Processor -- start 2024-04-25T09:54:42.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.248+0000 7f038eb64f40 1 -- start start 2024-04-25T09:54:42.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.248+0000 7f038eb64f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56482e7fa580 0x56482e8bea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.248+0000 7f038eb64f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 0x56482e8c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.248+0000 7f038eb64f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56482e8ccb10 0x56482e8ca060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.248+0000 7f038eb64f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482e79edf0 con 0x56482e8c9530 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.248+0000 7f038eb64f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482e7a2d50 con 0x56482e7fa580 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.248+0000 7f038eb64f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56482e8ccf20 con 0x56482e8ccb10 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.248+0000 7f038be64640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 0x56482e8c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.248+0000 7f038be64640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 0x56482e8c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58348/0 (socket says 172.21.15.139:58348) 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.248+0000 7f038be64640 1 -- 172.21.15.139:0/1556256041 learned_addr learned my addr 172.21.15.139:0/1556256041 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.248+0000 7f038ce66640 1 --2- 172.21.15.139:0/1556256041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56482e8ccb10 0x56482e8ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.248+0000 7f038be64640 1 -- 172.21.15.139:0/1556256041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56482e8ccb10 msgr2=0x56482e8ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038be64640 1 --2- 172.21.15.139:0/1556256041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56482e8ccb10 0x56482e8ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038be64640 1 -- 172.21.15.139:0/1556256041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56482e7fa580 msgr2=0x56482e8bea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038c665640 1 --2- 172.21.15.139:0/1556256041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56482e7fa580 0x56482e8bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038be64640 1 --2- 172.21.15.139:0/1556256041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56482e7fa580 0x56482e8bea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038be64640 1 -- 172.21.15.139:0/1556256041 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56482e8c8440 con 0x56482e8c9530 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038ce66640 1 --2- 172.21.15.139:0/1556256041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56482e8ccb10 0x56482e8ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038c665640 1 --2- 172.21.15.139:0/1556256041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56482e7fa580 0x56482e8bea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038be64640 1 --2- 172.21.15.139:0/1556256041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 0x56482e8c9b20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0380001dd0 tx=0x7f038000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4778904bdab1a83b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038ae62640 1 -- 172.21.15.139:0/1556256041 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038000bde0 con 0x56482e8c9530 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038ae62640 1 -- 172.21.15.139:0/1556256041 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038000f400 con 0x56482e8c9530 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038ae62640 1 -- 172.21.15.139:0/1556256041 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038000f830 con 0x56482e8c9530 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038eb64f40 1 -- 172.21.15.139:0/1556256041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 msgr2=0x56482e8c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038eb64f40 1 --2- 172.21.15.139:0/1556256041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 0x56482e8c9b20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0380001dd0 tx=0x7f038000b5b0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038eb64f40 1 -- 172.21.15.139:0/1556256041 shutdown_connections 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038eb64f40 1 --2- 172.21.15.139:0/1556256041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 0x56482e8c9b20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038eb64f40 1 --2- 172.21.15.139:0/1556256041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56482e8ccb10 0x56482e8ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038eb64f40 1 --2- 172.21.15.139:0/1556256041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56482e7fa580 0x56482e8bea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038eb64f40 1 -- 172.21.15.139:0/1556256041 >> 172.21.15.139:0/1556256041 conn(0x56482e7f7be0 msgr2=0x56482e7f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:42.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038eb64f40 1 -- 172.21.15.139:0/1556256041 shutdown_connections 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.249+0000 7f038eb64f40 1 -- 172.21.15.139:0/1556256041 wait complete. 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.250+0000 7f038eb64f40 1 Processor -- start 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.250+0000 7f038eb64f40 1 -- start start 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038eb64f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56482e7fa580 0x56482e8bde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038eb64f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 0x56482e8bd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038eb64f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56482e8ccb10 0x56482e83dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038eb64f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482e7f6280 con 0x56482e8c9530 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038eb64f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482e821bd0 con 0x56482e8ccb10 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038eb64f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56482e821e70 con 0x56482e7fa580 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038be64640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 0x56482e8bd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038be64640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 0x56482e8bd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58350/0 (socket says 172.21.15.139:58350) 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038be64640 1 -- 172.21.15.139:0/1073194721 learned_addr learned my addr 172.21.15.139:0/1073194721 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038c665640 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56482e7fa580 0x56482e8bde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038ce66640 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56482e8ccb10 0x56482e83dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038be64640 1 -- 172.21.15.139:0/1073194721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56482e7fa580 msgr2=0x56482e8bde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038be64640 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56482e7fa580 0x56482e8bde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038be64640 1 -- 172.21.15.139:0/1073194721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56482e8ccb10 msgr2=0x56482e83dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038be64640 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56482e8ccb10 0x56482e83dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038be64640 1 -- 172.21.15.139:0/1073194721 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56482e8bc770 con 0x56482e8c9530 2024-04-25T09:54:42.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038be64640 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 0x56482e8bd810 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0380000c00 tx=0x7f0380000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f0389e60640 1 -- 172.21.15.139:0/1073194721 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380020070 con 0x56482e8c9530 2024-04-25T09:54:42.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f0389e60640 1 -- 172.21.15.139:0/1073194721 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038001c440 con 0x56482e8c9530 2024-04-25T09:54:42.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038eb64f40 1 -- 172.21.15.139:0/1073194721 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56482e8bca50 con 0x56482e8c9530 2024-04-25T09:54:42.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f0389e60640 1 -- 172.21.15.139:0/1073194721 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038001c850 con 0x56482e8c9530 2024-04-25T09:54:42.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.251+0000 7f038eb64f40 1 -- 172.21.15.139:0/1073194721 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56482e8bd010 con 0x56482e8c9530 2024-04-25T09:54:42.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.253+0000 7f0389e60640 1 -- 172.21.15.139:0/1073194721 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0380023070 con 0x56482e8c9530 2024-04-25T09:54:42.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.253+0000 7f0389e60640 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0378069eb0 0x7f037806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.253+0000 7f0389e60640 1 -- 172.21.15.139:0/1073194721 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f038008aa40 con 0x56482e8c9530 2024-04-25T09:54:42.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.253+0000 7f038c665640 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0378069eb0 0x7f037806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.253+0000 7f038eb64f40 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x56482e8bc2e0 0x56482e83b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.253+0000 7f038eb64f40 1 -- 172.21.15.139:0/1073194721 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) v9 -- 0x56482e8c76a0 con 0x56482e8bc2e0 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.253+0000 7f038c665640 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0378069eb0 0x7f037806c370 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f037c0012e0 tx=0x7f037c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.254+0000 7f038ce66640 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x56482e8bc2e0 0x56482e83b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.254+0000 7f038ce66640 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x56482e8bc2e0 0x56482e83b570 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.254+0000 7f038ce66640 1 -- 172.21.15.139:0/1073194721 <== osd.5 v2:172.21.15.150:6824/2184499879 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f038400a140 con 0x56482e8bc2e0 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.255+0000 7f038eb64f40 1 -- 172.21.15.139:0/1073194721 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x56482e8bc2e0 msgr2=0x56482e83b570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.255+0000 7f038eb64f40 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x56482e8bc2e0 0x56482e83b570 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.255+0000 7f038eb64f40 1 -- 172.21.15.139:0/1073194721 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0378069eb0 msgr2=0x7f037806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.255+0000 7f038eb64f40 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0378069eb0 0x7f037806c370 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f037c0012e0 tx=0x7f037c008040 comp rx=0 tx=0).stop 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.255+0000 7f038eb64f40 1 -- 172.21.15.139:0/1073194721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 msgr2=0x56482e8bd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.255+0000 7f038eb64f40 1 --2- 172.21.15.139:0/1073194721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56482e8c9530 0x56482e8bd810 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0380000c00 tx=0x7f0380000ed0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.255+0000 7f038ce66640 1 -- 172.21.15.139:0/1073194721 reap_dead start 2024-04-25T09:54:42.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.255+0000 7f038eb64f40 1 -- 172.21.15.139:0/1073194721 shutdown_connections 2024-04-25T09:54:42.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.255+0000 7f038eb64f40 1 -- 172.21.15.139:0/1073194721 >> 172.21.15.139:0/1073194721 conn(0x56482e7f7be0 msgr2=0x56482e822300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:42.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.255+0000 7f038eb64f40 1 -- 172.21.15.139:0/1073194721 shutdown_connections 2024-04-25T09:54:42.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.255+0000 7f038eb64f40 1 -- 172.21.15.139:0/1073194721 wait complete. 2024-04-25T09:54:42.262 INFO:teuthology.orchestra.run.smithi139.stdout:mytag 2024-04-25T09:54:42.263 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T09:54:42.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.343+0000 7f6581d62f40 1 Processor -- start 2024-04-25T09:54:42.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.343+0000 7f6581d62f40 1 -- start start 2024-04-25T09:54:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f6581d62f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5c3ae70 0x55c6c5d06d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f6581d62f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c6c5d0f350 0x55c6c5d0b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f6581d62f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c6c5d11220 0x55c6c5d0ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f6581d62f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c5c81a80 con 0x55c6c5c3ae70 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f6581d62f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c5bdfdf0 con 0x55c6c5d0f350 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f6581d62f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6c5c81c00 con 0x55c6c5d11220 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f657f863640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5c3ae70 0x55c6c5d06d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f657f863640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5c3ae70 0x55c6c5d06d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58356/0 (socket says 172.21.15.139:58356) 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f657f863640 1 -- 172.21.15.139:0/986076713 learned_addr learned my addr 172.21.15.139:0/986076713 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f6580064640 1 --2- 172.21.15.139:0/986076713 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c6c5d11220 0x55c6c5d0ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f657f062640 1 --2- 172.21.15.139:0/986076713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c6c5d0f350 0x55c6c5d0b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f657f863640 1 -- 172.21.15.139:0/986076713 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c6c5d11220 msgr2=0x55c6c5d0ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f657f863640 1 --2- 172.21.15.139:0/986076713 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c6c5d11220 0x55c6c5d0ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f657f863640 1 -- 172.21.15.139:0/986076713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c6c5d0f350 msgr2=0x55c6c5d0b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f657f863640 1 --2- 172.21.15.139:0/986076713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c6c5d0f350 0x55c6c5d0b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f657f863640 1 -- 172.21.15.139:0/986076713 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c5b65c50 con 0x55c6c5c3ae70 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.344+0000 7f657f863640 1 --2- 172.21.15.139:0/986076713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5c3ae70 0x55c6c5d06d00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f657000a4f0 tx=0x7f657004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b39b705675715ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.345+0000 7f657e060640 1 -- 172.21.15.139:0/986076713 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570059040 con 0x55c6c5c3ae70 2024-04-25T09:54:42.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.345+0000 7f657e060640 1 -- 172.21.15.139:0/986076713 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6570065070 con 0x55c6c5c3ae70 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.345+0000 7f657e060640 1 -- 172.21.15.139:0/986076713 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570054ad0 con 0x55c6c5c3ae70 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.345+0000 7f6581d62f40 1 -- 172.21.15.139:0/986076713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5c3ae70 msgr2=0x55c6c5d06d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.345+0000 7f6581d62f40 1 --2- 172.21.15.139:0/986076713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5c3ae70 0x55c6c5d06d00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f657000a4f0 tx=0x7f657004e000 comp rx=0 tx=0).stop 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.345+0000 7f6581d62f40 1 -- 172.21.15.139:0/986076713 shutdown_connections 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.345+0000 7f6581d62f40 1 --2- 172.21.15.139:0/986076713 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c6c5d11220 0x55c6c5d0ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.345+0000 7f6581d62f40 1 --2- 172.21.15.139:0/986076713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c6c5d0f350 0x55c6c5d0b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.345+0000 7f6581d62f40 1 --2- 172.21.15.139:0/986076713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5c3ae70 0x55c6c5d06d00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.345+0000 7f6581d62f40 1 -- 172.21.15.139:0/986076713 >> 172.21.15.139:0/986076713 conn(0x55c6c5c38be0 msgr2=0x55c6c5c39750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.345+0000 7f6581d62f40 1 -- 172.21.15.139:0/986076713 shutdown_connections 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.345+0000 7f6581d62f40 1 -- 172.21.15.139:0/986076713 wait complete. 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.346+0000 7f6581d62f40 1 Processor -- start 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.346+0000 7f6581d62f40 1 -- start start 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.346+0000 7f6581d62f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c6c5c3ae70 0x55c6c5d04b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.346+0000 7f6581d62f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5d0f350 0x55c6c5c7b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.346+0000 7f6581d62f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c6c5d11220 0x55c6c5c7bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.346+0000 7f6581d62f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c5d07960 con 0x55c6c5d0f350 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.346+0000 7f6581d62f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c5c3a1c0 con 0x55c6c5d11220 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.346+0000 7f6581d62f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6c5d075b0 con 0x55c6c5c3ae70 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.346+0000 7f657f062640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5d0f350 0x55c6c5c7b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.346+0000 7f657f062640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5d0f350 0x55c6c5c7b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58358/0 (socket says 172.21.15.139:58358) 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.346+0000 7f657f062640 1 -- 172.21.15.139:0/861232947 learned_addr learned my addr 172.21.15.139:0/861232947 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.346+0000 7f657f863640 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c6c5c3ae70 0x55c6c5d04b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f6580064640 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c6c5d11220 0x55c6c5c7bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f657f062640 1 -- 172.21.15.139:0/861232947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c6c5c3ae70 msgr2=0x55c6c5d04b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f657f062640 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c6c5c3ae70 0x55c6c5d04b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f657f062640 1 -- 172.21.15.139:0/861232947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c6c5d11220 msgr2=0x55c6c5c7bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f657f062640 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c6c5d11220 0x55c6c5c7bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f657f062640 1 -- 172.21.15.139:0/861232947 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c5da4110 con 0x55c6c5d0f350 2024-04-25T09:54:42.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f6580064640 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c6c5d11220 0x55c6c5c7bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:42.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f657f062640 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5d0f350 0x55c6c5c7b9d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6574008400 tx=0x7f65740084f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f657d05e640 1 -- 172.21.15.139:0/861232947 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65740145f0 con 0x55c6c5d0f350 2024-04-25T09:54:42.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f6581d62f40 1 -- 172.21.15.139:0/861232947 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c6c5c7c750 con 0x55c6c5d0f350 2024-04-25T09:54:42.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f657d05e640 1 -- 172.21.15.139:0/861232947 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6574019070 con 0x55c6c5d0f350 2024-04-25T09:54:42.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f657d05e640 1 -- 172.21.15.139:0/861232947 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65740149b0 con 0x55c6c5d0f350 2024-04-25T09:54:42.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.347+0000 7f6581d62f40 1 -- 172.21.15.139:0/861232947 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6c5d05070 con 0x55c6c5d0f350 2024-04-25T09:54:42.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.348+0000 7f657d05e640 1 -- 172.21.15.139:0/861232947 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6574014b50 con 0x55c6c5d0f350 2024-04-25T09:54:42.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:54:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.349+0000 7f657d05e640 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f656c069eb0 0x7f656c06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.349+0000 7f657d05e640 1 -- 172.21.15.139:0/861232947 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6574088de0 con 0x55c6c5d0f350 2024-04-25T09:54:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.349+0000 7f657f863640 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f656c069eb0 0x7f656c06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.349+0000 7f6581d62f40 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55c6c5c80a60 0x55c6c5da4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.349+0000 7f6581d62f40 1 -- 172.21.15.139:0/861232947 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) v9 -- 0x55c6c5c7ed20 con 0x55c6c5c80a60 2024-04-25T09:54:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.349+0000 7f657f863640 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f656c069eb0 0x7f656c06c370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6570061000 tx=0x7f6570056040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.349+0000 7f6580064640 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55c6c5c80a60 0x55c6c5da4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.349+0000 7f6580064640 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55c6c5c80a60 0x55c6c5da4fb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.350+0000 7f6580064640 1 -- 172.21.15.139:0/861232947 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f65780047c0 con 0x55c6c5c80a60 2024-04-25T09:54:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-25T09:54:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.350+0000 7f6581d62f40 1 -- 172.21.15.139:0/861232947 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55c6c5c80a60 msgr2=0x55c6c5da4fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.350+0000 7f6581d62f40 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55c6c5c80a60 0x55c6c5da4fb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.350+0000 7f6581d62f40 1 -- 172.21.15.139:0/861232947 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f656c069eb0 msgr2=0x7f656c06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.350+0000 7f6581d62f40 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f656c069eb0 0x7f656c06c370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6570061000 tx=0x7f6570056040 comp rx=0 tx=0).stop 2024-04-25T09:54:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.350+0000 7f6581d62f40 1 -- 172.21.15.139:0/861232947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5d0f350 msgr2=0x55c6c5c7b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.350+0000 7f6581d62f40 1 --2- 172.21.15.139:0/861232947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c6c5d0f350 0x55c6c5c7b9d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6574008400 tx=0x7f65740084f0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.350+0000 7f6580064640 1 -- 172.21.15.139:0/861232947 reap_dead start 2024-04-25T09:54:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.350+0000 7f6581d62f40 1 -- 172.21.15.139:0/861232947 shutdown_connections 2024-04-25T09:54:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.350+0000 7f6581d62f40 1 -- 172.21.15.139:0/861232947 >> 172.21.15.139:0/861232947 conn(0x55c6c5c38be0 msgr2=0x55c6c5c36410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.350+0000 7f6581d62f40 1 -- 172.21.15.139:0/861232947 shutdown_connections 2024-04-25T09:54:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.350+0000 7f6581d62f40 1 -- 172.21.15.139:0/861232947 wait complete. 2024-04-25T09:54:42.358 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:54:42.359 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:54:42.438 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:54:42.501 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:54:42.502 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:54:42.502 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:42.517 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:54:42.518 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:54:42.518 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:42.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:54:42.535 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:54:42.535 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:42.573 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:54:42.573 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:54:42.573 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:42.592 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:54:42.592 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:42.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.709+0000 7f6129b96640 1 Processor -- start 2024-04-25T09:54:42.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.709+0000 7f6129b96640 1 -- start start 2024-04-25T09:54:42.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f6129b96640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f612415bdc0 0x7f612415c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f6129b96640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f61240b1740 0x7f61240b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f6129b96640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612415c880 con 0x7f6124155e30 2024-04-25T09:54:42.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f6129b96640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612415d690 con 0x7f61240b1740 2024-04-25T09:54:42.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f6129b96640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612406d6f0 con 0x7f612415bdc0 2024-04-25T09:54:42.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61237fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f6124155e30 0x7f612415cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54596/0 (socket says 172.21.15.139:54596) 2024-04-25T09:54:42.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61237fe640 1 -- 172.21.15.139:0/2185794151 learned_addr learned my addr 172.21.15.139:0/2185794151 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:42.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f6123fff640 1 --2- 172.21.15.139:0/2185794151 >> v2:172.21.15.150:3300/0 conn(0x7f61240b1740 0x7f61240b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f6122ffd640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f612415bdc0 0x7f612415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208716446 0 0) 0x7f612415c880 con 0x7f6124155e30 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f610c003520 con 0x7f6124155e30 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 635287193 0 0) 0x7f610c003520 con 0x7f6124155e30 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f612415bdc0 msgr2=0x7f612415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f612415bdc0 0x7f612415c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 >> v2:172.21.15.150:3300/0 conn(0x7f61240b1740 msgr2=0x7f61240b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 --2- 172.21.15.139:0/2185794151 >> v2:172.21.15.150:3300/0 conn(0x7f61240b1740 0x7f61240b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612415cb60 con 0x7f6124155e30 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f6114001670 con 0x7f6124155e30 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 >> v1:172.21.15.139:6789/0 conn(0x7f6124155e30 legacy=0x7f612415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f610c0039b0 0x7f610c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f610c004800 0x7f610c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f612415bdc0 0x7f610c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612415d690 con 0x7f610c0039b0 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612406d6f0 con 0x7f610c004800 2024-04-25T09:54:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f610c00b9c0 con 0x7f612415bdc0 2024-04-25T09:54:42.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.710+0000 7f61237fe640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f610c0039b0 0x7f610c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.711+0000 7f6122ffd640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f610c004800 0x7f610c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.711+0000 7f6123fff640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f612415bdc0 0x7f610c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.711+0000 7f61237fe640 1 -- 172.21.15.139:0/2185794151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f612415bdc0 msgr2=0x7f610c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.711+0000 7f61237fe640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f612415bdc0 0x7f610c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.711+0000 7f61237fe640 1 -- 172.21.15.139:0/2185794151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f610c004800 msgr2=0x7f610c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:42.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.711+0000 7f61237fe640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f610c004800 0x7f610c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.711+0000 7f61237fe640 1 -- 172.21.15.139:0/2185794151 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610c003520 con 0x7f610c0039b0 2024-04-25T09:54:42.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.711+0000 7f6123fff640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f612415bdc0 0x7f610c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:42.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.711+0000 7f61237fe640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f610c0039b0 0x7f610c003dd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f611400c240 tx=0x7f610c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c869a53cfe1b58b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.711+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611402e070 con 0x7f610c0039b0 2024-04-25T09:54:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.711+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6114005af0 con 0x7f610c0039b0 2024-04-25T09:54:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.711+0000 7f61227fc640 1 -- 172.21.15.139:0/2185794151 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114007d30 con 0x7f610c0039b0 2024-04-25T09:54:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.712+0000 7f6129b96640 1 -- 172.21.15.139:0/2185794151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f610c0039b0 msgr2=0x7f610c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.712+0000 7f6129b96640 1 --2- 172.21.15.139:0/2185794151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f610c0039b0 0x7f610c003dd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f611400c240 tx=0x7f610c005830 comp rx=0 tx=0).stop 2024-04-25T09:54:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.712+0000 7f6123fff640 1 -- 172.21.15.139:0/2185794151 reap_dead start 2024-04-25T09:54:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.712+0000 7f6129b96640 1 -- 172.21.15.139:0/2185794151 shutdown_connections 2024-04-25T09:54:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.712+0000 7f6129b96640 1 -- 172.21.15.139:0/2185794151 >> 172.21.15.139:0/2185794151 conn(0x7f6124066f20 msgr2=0x7f612406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.712+0000 7f6129b96640 1 -- 172.21.15.139:0/2185794151 shutdown_connections 2024-04-25T09:54:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.712+0000 7f6129b96640 1 -- 172.21.15.139:0/2185794151 wait complete. 2024-04-25T09:54:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.713+0000 7f6129b96640 1 Processor -- start 2024-04-25T09:54:42.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.713+0000 7f6129b96640 1 -- start start 2024-04-25T09:54:42.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6129b96640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f610c0039b0 0x7f61241f0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6129b96640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61240b2590 0x7f6124156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6129b96640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61240b2a10 0x7f6124157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6129b96640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61240af0e0 con 0x7f61240b2590 2024-04-25T09:54:42.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6129b96640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6124156210 con 0x7f610c0039b0 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6129b96640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61240af260 con 0x7f61240b2a10 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6122ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61240b2590 0x7f6124156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6122ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61240b2590 0x7f6124156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58378/0 (socket says 172.21.15.139:58378) 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6122ffd640 1 -- 172.21.15.139:0/396730811 learned_addr learned my addr 172.21.15.139:0/396730811 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6123fff640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61240b2a10 0x7f6124157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6122ffd640 1 -- 172.21.15.139:0/396730811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61240b2a10 msgr2=0x7f6124157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f61237fe640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f610c0039b0 0x7f61241f0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6122ffd640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61240b2a10 0x7f6124157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6122ffd640 1 -- 172.21.15.139:0/396730811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f610c0039b0 msgr2=0x7f61241f0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6122ffd640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f610c0039b0 0x7f61241f0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6122ffd640 1 -- 172.21.15.139:0/396730811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61241f0fa0 con 0x7f61240b2590 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f6123fff640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61240b2a10 0x7f6124157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.714+0000 7f61237fe640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f610c0039b0 0x7f61241f0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:42.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.715+0000 7f6122ffd640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61240b2590 0x7f6124156e10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6118011540 tx=0x7f6118011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.715+0000 7f610bfff640 1 -- 172.21.15.139:0/396730811 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118015930 con 0x7f61240b2590 2024-04-25T09:54:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.715+0000 7f610bfff640 1 -- 172.21.15.139:0/396730811 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6118015ad0 con 0x7f61240b2590 2024-04-25T09:54:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.715+0000 7f610bfff640 1 -- 172.21.15.139:0/396730811 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611800f460 con 0x7f61240b2590 2024-04-25T09:54:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.715+0000 7f6129b96640 1 -- 172.21.15.139:0/396730811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61240b2fe0 con 0x7f61240b2590 2024-04-25T09:54:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.715+0000 7f6129b96640 1 -- 172.21.15.139:0/396730811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61240b35a0 con 0x7f61240b2590 2024-04-25T09:54:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.716+0000 7f6129b96640 1 -- 172.21.15.139:0/396730811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6124074520 con 0x7f61240b2590 2024-04-25T09:54:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.716+0000 7f610bfff640 1 -- 172.21.15.139:0/396730811 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f611800f600 con 0x7f61240b2590 2024-04-25T09:54:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.716+0000 7f610bfff640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f611006b4d0 0x7f611006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.717+0000 7f610bfff640 1 -- 172.21.15.139:0/396730811 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f611808ac00 con 0x7f61240b2590 2024-04-25T09:54:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.717+0000 7f61237fe640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f611006b4d0 0x7f611006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:42.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.717+0000 7f61237fe640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f611006b4d0 0x7f611006d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6114033000 tx=0x7f6114009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:42.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.721+0000 7f610bfff640 1 -- 172.21.15.139:0/396730811 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6118091080 con 0x7f61240b2590 2024-04-25T09:54:42.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.925+0000 7f6129b96640 1 -- 172.21.15.139:0/396730811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f612406b830 con 0x7f61240b2590 2024-04-25T09:54:42.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.925+0000 7f610bfff640 1 -- 172.21.15.139:0/396730811 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f6118019030 con 0x7f61240b2590 2024-04-25T09:54:42.928 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:42.928 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/511562594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":511562594},{"type":"v1","addr":"172.21.15.150:6834","nonce":511562594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5041,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/225357317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":225357317},{"type":"v1","addr":"172.21.15.150:6835","nonce":225357317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2499407982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2499407982},{"type":"v1","addr":"172.21.15.139:6836","nonce":2499407982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:21.988975+0000","modified":"2024-04-25T09:54:23.955620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/2069784668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2069784668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2069784668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:54:42.928 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 16 2024-04-25T09:54:42.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.927+0000 7f6129b96640 1 -- 172.21.15.139:0/396730811 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f611006b4d0 msgr2=0x7f611006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.927+0000 7f6129b96640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f611006b4d0 0x7f611006d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6114033000 tx=0x7f6114009030 comp rx=0 tx=0).stop 2024-04-25T09:54:42.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.927+0000 7f6129b96640 1 -- 172.21.15.139:0/396730811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61240b2590 msgr2=0x7f6124156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:42.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.927+0000 7f6129b96640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61240b2590 0x7f6124156e10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6118011540 tx=0x7f6118011a00 comp rx=0 tx=0).stop 2024-04-25T09:54:42.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.927+0000 7f6129b96640 1 -- 172.21.15.139:0/396730811 shutdown_connections 2024-04-25T09:54:42.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.927+0000 7f6129b96640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61240b2590 0x7f6124156e10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.927+0000 7f6129b96640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f611006b4d0 0x7f611006d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.927+0000 7f6129b96640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61240b2a10 0x7f6124157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.927+0000 7f6129b96640 1 --2- 172.21.15.139:0/396730811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f610c0039b0 0x7f61241f0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:42.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.927+0000 7f6129b96640 1 -- 172.21.15.139:0/396730811 >> 172.21.15.139:0/396730811 conn(0x7f6124066f20 msgr2=0x7f6124159b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:42.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.927+0000 7f6129b96640 1 -- 172.21.15.139:0/396730811 shutdown_connections 2024-04-25T09:54:42.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:42.927+0000 7f6129b96640 1 -- 172.21.15.139:0/396730811 wait complete. 2024-04-25T09:54:42.941 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:43.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.036+0000 7fe9d6961640 1 Processor -- start 2024-04-25T09:54:43.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.036+0000 7fe9d6961640 1 -- start start 2024-04-25T09:54:43.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.037+0000 7fe9d6961640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d006c8a0 0x7fe9d00d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.037+0000 7fe9d6961640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe9d00d12c0 0x7fe9d006ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.037+0000 7fe9d6961640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d00b10e0 con 0x7fe9d00b0d00 2024-04-25T09:54:43.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.037+0000 7fe9d6961640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d00d0350 con 0x7fe9d00d12c0 2024-04-25T09:54:43.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.037+0000 7fe9d6961640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d015f070 con 0x7fe9d006c8a0 2024-04-25T09:54:43.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.037+0000 7fe9d4efe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe9d00d12c0 0x7fe9d006ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.037+0000 7fe9cffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe9d00b0d00 0x7fe9d015e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54606/0 (socket says 172.21.15.139:54606) 2024-04-25T09:54:43.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.037+0000 7fe9cffff640 1 -- 172.21.15.139:0/1589821278 learned_addr learned my addr 172.21.15.139:0/1589821278 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:43.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.038+0000 7fe9cf7fe640 1 --2- 172.21.15.139:0/1589821278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d006c8a0 0x7fe9d00d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.038+0000 7fe9d4efe640 1 -- 172.21.15.139:0/1589821278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d006c8a0 msgr2=0x7fe9d00d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.038+0000 7fe9d4efe640 1 --2- 172.21.15.139:0/1589821278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d006c8a0 0x7fe9d00d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.038+0000 7fe9d4efe640 1 -- 172.21.15.139:0/1589821278 >> v1:172.21.15.139:6789/0 conn(0x7fe9d00b0d00 legacy=0x7fe9d015e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.038+0000 7fe9d4efe640 1 -- 172.21.15.139:0/1589821278 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d015e7e0 con 0x7fe9d00d12c0 2024-04-25T09:54:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.038+0000 7fe9ceffd640 1 -- 172.21.15.139:0/1589821278 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281962328 0 0) 0x7fe9d00b10e0 con 0x7fe9d00b0d00 2024-04-25T09:54:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.038+0000 7fe9cf7fe640 1 --2- 172.21.15.139:0/1589821278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d006c8a0 0x7fe9d00d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.038+0000 7fe9d4efe640 1 --2- 172.21.15.139:0/1589821278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9d00d12c0 0x7fe9d006ce00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009d90 tx=0x7fe9c802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a25050e606bc5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.038+0000 7fe9ceffd640 1 -- 172.21.15.139:0/1589821278 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8031600 con 0x7fe9d00d12c0 2024-04-25T09:54:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.039+0000 7fe9ceffd640 1 -- 172.21.15.139:0/1589821278 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c80317a0 con 0x7fe9d00d12c0 2024-04-25T09:54:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.039+0000 7fe9ceffd640 1 -- 172.21.15.139:0/1589821278 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8031b70 con 0x7fe9d00d12c0 2024-04-25T09:54:43.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.039+0000 7fe9d6961640 1 -- 172.21.15.139:0/1589821278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9d00d12c0 msgr2=0x7fe9d006ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.039+0000 7fe9d6961640 1 --2- 172.21.15.139:0/1589821278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9d00d12c0 0x7fe9d006ce00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009d90 tx=0x7fe9c802a5d0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.039+0000 7fe9d6961640 1 -- 172.21.15.139:0/1589821278 shutdown_connections 2024-04-25T09:54:43.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.039+0000 7fe9d6961640 1 --2- 172.21.15.139:0/1589821278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9d00d12c0 0x7fe9d006ce00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.039+0000 7fe9d6961640 1 --2- 172.21.15.139:0/1589821278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d006c8a0 0x7fe9d00d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.039+0000 7fe9d6961640 1 -- 172.21.15.139:0/1589821278 >> 172.21.15.139:0/1589821278 conn(0x7fe9d00b3380 msgr2=0x7fe9d00b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:43.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.039+0000 7fe9d6961640 1 -- 172.21.15.139:0/1589821278 shutdown_connections 2024-04-25T09:54:43.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.039+0000 7fe9d6961640 1 -- 172.21.15.139:0/1589821278 wait complete. 2024-04-25T09:54:43.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.040+0000 7fe9d6961640 1 Processor -- start 2024-04-25T09:54:43.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.040+0000 7fe9d6961640 1 -- start start 2024-04-25T09:54:43.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9d6961640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9d006c8a0 0x7fe9d0146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9d6961640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d00b0d00 0x7fe9d0158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9d6961640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9d00d12c0 0x7fe9d01587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9d6961640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d00d0580 con 0x7fe9d006c8a0 2024-04-25T09:54:43.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9d6961640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d015ed90 con 0x7fe9d00d12c0 2024-04-25T09:54:43.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9d6961640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d006d340 con 0x7fe9d00b0d00 2024-04-25T09:54:43.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9d006c8a0 0x7fe9d0146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9d006c8a0 0x7fe9d0146010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58390/0 (socket says 172.21.15.139:58390) 2024-04-25T09:54:43.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9cffff640 1 -- 172.21.15.139:0/3712770577 learned_addr learned my addr 172.21.15.139:0/3712770577 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:43.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9cf7fe640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d00b0d00 0x7fe9d0158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9cffff640 1 -- 172.21.15.139:0/3712770577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d00b0d00 msgr2=0x7fe9d0158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9cffff640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d00b0d00 0x7fe9d0158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9d4efe640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9d00d12c0 0x7fe9d01587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9cffff640 1 -- 172.21.15.139:0/3712770577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9d00d12c0 msgr2=0x7fe9d01587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.041+0000 7fe9cffff640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9d00d12c0 0x7fe9d01587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.042+0000 7fe9cffff640 1 -- 172.21.15.139:0/3712770577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d0146550 con 0x7fe9d006c8a0 2024-04-25T09:54:43.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.042+0000 7fe9cffff640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9d006c8a0 0x7fe9d0146010 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c4d0 tx=0x7fe9c000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.042+0000 7fe9ccff9640 1 -- 172.21.15.139:0/3712770577 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0018070 con 0x7fe9d006c8a0 2024-04-25T09:54:43.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.042+0000 7fe9ccff9640 1 -- 172.21.15.139:0/3712770577 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c0007500 con 0x7fe9d006c8a0 2024-04-25T09:54:43.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.042+0000 7fe9d6961640 1 -- 172.21.15.139:0/3712770577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d015b110 con 0x7fe9d006c8a0 2024-04-25T09:54:43.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.042+0000 7fe9ccff9640 1 -- 172.21.15.139:0/3712770577 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0004a20 con 0x7fe9d006c8a0 2024-04-25T09:54:43.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.042+0000 7fe9d6961640 1 -- 172.21.15.139:0/3712770577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d00ccfa0 con 0x7fe9d006c8a0 2024-04-25T09:54:43.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.043+0000 7fe9d6961640 1 -- 172.21.15.139:0/3712770577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d006de00 con 0x7fe9d006c8a0 2024-04-25T09:54:43.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.043+0000 7fe9ccff9640 1 -- 172.21.15.139:0/3712770577 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9c001b070 con 0x7fe9d006c8a0 2024-04-25T09:54:43.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.044+0000 7fe9ccff9640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe9bc06b4d0 0x7fe9bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.044+0000 7fe9ccff9640 1 -- 172.21.15.139:0/3712770577 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9c00896b0 con 0x7fe9d006c8a0 2024-04-25T09:54:43.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.044+0000 7fe9cf7fe640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe9bc06b4d0 0x7fe9bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.044+0000 7fe9cf7fe640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe9bc06b4d0 0x7fe9bc06d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe9d01f6420 tx=0x7fe9c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.048+0000 7fe9ccff9640 1 -- 172.21.15.139:0/3712770577 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9c00583a0 con 0x7fe9d006c8a0 2024-04-25T09:54:43.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.218+0000 7fe9d6961640 1 -- 172.21.15.139:0/3712770577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9d00ccd50 con 0x7fe9d006c8a0 2024-04-25T09:54:43.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.219+0000 7fe9ccff9640 1 -- 172.21.15.139:0/3712770577 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16990 (secure 0 0 0) 0x7fe9c0057ca0 con 0x7fe9d006c8a0 2024-04-25T09:54:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:43.222 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":38,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:23.620264+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:19.795203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:20.931184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:43.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.221+0000 7fe9d6961640 1 -- 172.21.15.139:0/3712770577 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe9bc06b4d0 msgr2=0x7fe9bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.221+0000 7fe9d6961640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe9bc06b4d0 0x7fe9bc06d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe9d01f6420 tx=0x7fe9c4008040 comp rx=0 tx=0).stop 2024-04-25T09:54:43.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.221+0000 7fe9d6961640 1 -- 172.21.15.139:0/3712770577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9d006c8a0 msgr2=0x7fe9d0146010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.221+0000 7fe9d6961640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9d006c8a0 0x7fe9d0146010 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c4d0 tx=0x7fe9c000c990 comp rx=0 tx=0).stop 2024-04-25T09:54:43.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.221+0000 7fe9d6961640 1 -- 172.21.15.139:0/3712770577 shutdown_connections 2024-04-25T09:54:43.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.221+0000 7fe9d6961640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe9bc06b4d0 0x7fe9bc06d990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.221+0000 7fe9d6961640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9d00d12c0 0x7fe9d01587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.221+0000 7fe9d6961640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d00b0d00 0x7fe9d0158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.221+0000 7fe9d6961640 1 --2- 172.21.15.139:0/3712770577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9d006c8a0 0x7fe9d0146010 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.221+0000 7fe9d6961640 1 -- 172.21.15.139:0/3712770577 >> 172.21.15.139:0/3712770577 conn(0x7fe9d00b3380 msgr2=0x7fe9d015f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:43.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.221+0000 7fe9d6961640 1 -- 172.21.15.139:0/3712770577 shutdown_connections 2024-04-25T09:54:43.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.221+0000 7fe9d6961640 1 -- 172.21.15.139:0/3712770577 wait complete. 2024-04-25T09:54:43.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:19.795203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:20.931184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:54:43.235 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:43.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.332+0000 7fcdf8891640 1 Processor -- start 2024-04-25T09:54:43.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.332+0000 7fcdf8891640 1 -- start start 2024-04-25T09:54:43.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.333+0000 7fcdf8891640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcdf40afc80 0x7fcdf40b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.333+0000 7fcdf8891640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf406b700 0x7fcdf406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.333+0000 7fcdf8891640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf40c6cc0 con 0x7fcdf40af8a0 2024-04-25T09:54:43.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.333+0000 7fcdf8891640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf40c6fa0 con 0x7fcdf40afc80 2024-04-25T09:54:43.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.333+0000 7fcdf8891640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40c7280 con 0x7fcdf406b700 2024-04-25T09:54:43.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.334+0000 7fcdf2d9d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf406b700 0x7fcdf406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.333+0000 7fcdf1d9b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcdf40afc80 0x7fcdf40b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.334+0000 7fcdf1d9b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcdf40afc80 0x7fcdf40b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:35440/0 (socket says 172.21.15.139:35440) 2024-04-25T09:54:43.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.334+0000 7fcdf1d9b640 1 -- 172.21.15.139:0/994665232 learned_addr learned my addr 172.21.15.139:0/994665232 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:43.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.334+0000 7fcdf2d9d640 1 -- 172.21.15.139:0/994665232 >> v2:172.21.15.150:3300/0 conn(0x7fcdf40afc80 msgr2=0x7fcdf40b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:43.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.334+0000 7fcdf2d9d640 1 --2- 172.21.15.139:0/994665232 >> v2:172.21.15.150:3300/0 conn(0x7fcdf40afc80 0x7fcdf40b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.334+0000 7fcdf2d9d640 1 -- 172.21.15.139:0/994665232 >> v1:172.21.15.150:6789/0 conn(0x7fcdf40af8a0 legacy=0x7fcdf40cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.334+0000 7fcdf2d9d640 1 -- 172.21.15.139:0/994665232 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf40cd090 con 0x7fcdf406b700 2024-04-25T09:54:43.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.334+0000 7fcdf1d9b640 1 --2- 172.21.15.139:0/994665232 >> v2:172.21.15.150:3300/0 conn(0x7fcdf40afc80 0x7fcdf40b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:43.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.335+0000 7fcdf2d9d640 1 --2- 172.21.15.139:0/994665232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf406b700 0x7fcdf406ab60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0097a0 tx=0x7fcdec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1697b1b127c78aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.335+0000 7fcdf159a640 1 -- 172.21.15.139:0/994665232 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec037070 con 0x7fcdf406b700 2024-04-25T09:54:43.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.335+0000 7fcdf159a640 1 -- 172.21.15.139:0/994665232 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec032440 con 0x7fcdf406b700 2024-04-25T09:54:43.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.335+0000 7fcdf159a640 1 -- 172.21.15.139:0/994665232 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec032810 con 0x7fcdf406b700 2024-04-25T09:54:43.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.336+0000 7fcdf8891640 1 -- 172.21.15.139:0/994665232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf406b700 msgr2=0x7fcdf406ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.336+0000 7fcdf8891640 1 --2- 172.21.15.139:0/994665232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf406b700 0x7fcdf406ab60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0097a0 tx=0x7fcdec02a180 comp rx=0 tx=0).stop 2024-04-25T09:54:43.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.336+0000 7fcdf8891640 1 -- 172.21.15.139:0/994665232 shutdown_connections 2024-04-25T09:54:43.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.336+0000 7fcdf8891640 1 --2- 172.21.15.139:0/994665232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf406b700 0x7fcdf406ab60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.336+0000 7fcdf8891640 1 --2- 172.21.15.139:0/994665232 >> v2:172.21.15.150:3300/0 conn(0x7fcdf40afc80 0x7fcdf40b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.336+0000 7fcdf8891640 1 -- 172.21.15.139:0/994665232 >> 172.21.15.139:0/994665232 conn(0x7fcdf40b2230 msgr2=0x7fcdf40b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:43.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.336+0000 7fcdf8891640 1 -- 172.21.15.139:0/994665232 shutdown_connections 2024-04-25T09:54:43.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.336+0000 7fcdf8891640 1 -- 172.21.15.139:0/994665232 wait complete. 2024-04-25T09:54:43.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.337+0000 7fcdf8891640 1 Processor -- start 2024-04-25T09:54:43.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.337+0000 7fcdf8891640 1 -- start start 2024-04-25T09:54:43.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf8891640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdf406b700 0x7fcdf414b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf8891640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf40af8a0 0x7fcdf4155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf8891640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf40afc80 0x7fcdf40b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf8891640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf406b220 con 0x7fcdf406b700 2024-04-25T09:54:43.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf8891640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf406b0a0 con 0x7fcdf40af8a0 2024-04-25T09:54:43.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf8891640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf406b3a0 con 0x7fcdf40afc80 2024-04-25T09:54:43.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdf406b700 0x7fcdf414b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdf406b700 0x7fcdf414b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58392/0 (socket says 172.21.15.139:58392) 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf259c640 1 -- 172.21.15.139:0/2023219017 learned_addr learned my addr 172.21.15.139:0/2023219017 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf2d9d640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf40afc80 0x7fcdf40b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf1d9b640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf40af8a0 0x7fcdf4155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf259c640 1 -- 172.21.15.139:0/2023219017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf40afc80 msgr2=0x7fcdf40b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf259c640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf40afc80 0x7fcdf40b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf259c640 1 -- 172.21.15.139:0/2023219017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf40af8a0 msgr2=0x7fcdf4155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf259c640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf40af8a0 0x7fcdf4155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.338+0000 7fcdf259c640 1 -- 172.21.15.139:0/2023219017 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf4156390 con 0x7fcdf406b700 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.339+0000 7fcdf259c640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdf406b700 0x7fcdf414b400 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c790 tx=0x7fcde400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.339+0000 7fcde2ffd640 1 -- 172.21.15.139:0/2023219017 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4016020 con 0x7fcdf406b700 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.339+0000 7fcde2ffd640 1 -- 172.21.15.139:0/2023219017 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde4006d50 con 0x7fcdf406b700 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.339+0000 7fcdf8891640 1 -- 172.21.15.139:0/2023219017 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf40b4e10 con 0x7fcdf406b700 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.339+0000 7fcde2ffd640 1 -- 172.21.15.139:0/2023219017 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4003bb0 con 0x7fcdf406b700 2024-04-25T09:54:43.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.339+0000 7fcdf8891640 1 -- 172.21.15.139:0/2023219017 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf40b53d0 con 0x7fcdf406b700 2024-04-25T09:54:43.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.340+0000 7fcdf8891640 1 -- 172.21.15.139:0/2023219017 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf4074520 con 0x7fcdf406b700 2024-04-25T09:54:43.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.340+0000 7fcde2ffd640 1 -- 172.21.15.139:0/2023219017 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcde400a850 con 0x7fcdf406b700 2024-04-25T09:54:43.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.340+0000 7fcde2ffd640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcdd406b4d0 0x7fcdd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.340+0000 7fcde2ffd640 1 -- 172.21.15.139:0/2023219017 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcde4089ee0 con 0x7fcdf406b700 2024-04-25T09:54:43.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.341+0000 7fcdf1d9b640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcdd406b4d0 0x7fcdd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.341+0000 7fcdf1d9b640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcdd406b4d0 0x7fcdd406d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40ae4b0 tx=0x7fcde8009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.345+0000 7fcde2ffd640 1 -- 172.21.15.139:0/2023219017 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcde4020020 con 0x7fcdf406b700 2024-04-25T09:54:43.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.514+0000 7fcdf8891640 1 -- 172.21.15.139:0/2023219017 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcdf40c56f0 con 0x7fcdf406b700 2024-04-25T09:54:43.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.514+0000 7fcde2ffd640 1 -- 172.21.15.139:0/2023219017 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcde4058610 con 0x7fcdf406b700 2024-04-25T09:54:43.517 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:54:43.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.516+0000 7fcdf8891640 1 -- 172.21.15.139:0/2023219017 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcdd406b4d0 msgr2=0x7fcdd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.516+0000 7fcdf8891640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcdd406b4d0 0x7fcdd406d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40ae4b0 tx=0x7fcde8009240 comp rx=0 tx=0).stop 2024-04-25T09:54:43.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.516+0000 7fcdf8891640 1 -- 172.21.15.139:0/2023219017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdf406b700 msgr2=0x7fcdf414b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.516+0000 7fcdf8891640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdf406b700 0x7fcdf414b400 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c790 tx=0x7fcde400cc50 comp rx=0 tx=0).stop 2024-04-25T09:54:43.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.516+0000 7fcdf8891640 1 -- 172.21.15.139:0/2023219017 shutdown_connections 2024-04-25T09:54:43.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.516+0000 7fcdf8891640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcdd406b4d0 0x7fcdd406d990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.516+0000 7fcdf8891640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf40afc80 0x7fcdf40b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.516+0000 7fcdf8891640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf40af8a0 0x7fcdf4155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.516+0000 7fcdf8891640 1 --2- 172.21.15.139:0/2023219017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdf406b700 0x7fcdf414b400 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.516+0000 7fcdf8891640 1 -- 172.21.15.139:0/2023219017 >> 172.21.15.139:0/2023219017 conn(0x7fcdf40b2230 msgr2=0x7fcdf40b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:43.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.516+0000 7fcdf8891640 1 -- 172.21.15.139:0/2023219017 shutdown_connections 2024-04-25T09:54:43.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.516+0000 7fcdf8891640 1 -- 172.21.15.139:0/2023219017 wait complete. 2024-04-25T09:54:43.530 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:54:43.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.627+0000 7f6323734640 1 Processor -- start 2024-04-25T09:54:43.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.627+0000 7f6323734640 1 -- start start 2024-04-25T09:54:43.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f6323734640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f631c15dbf0 0x7f631c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f6323734640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f631c0b9370 0x7f631c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f6323734640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f631c0cec80 0x7f631c06c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f6323734640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631c0d1ee0 con 0x7f631c15dbf0 2024-04-25T09:54:43.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f6323734640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631c161dc0 con 0x7f631c0cec80 2024-04-25T09:54:43.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f6323734640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c1620a0 con 0x7f631c0b9370 2024-04-25T09:54:43.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f63214d0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f631c15dbf0 0x7f631c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f63214d0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f631c15dbf0 0x7f631c1617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58404/0 (socket says 172.21.15.139:58404) 2024-04-25T09:54:43.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f63214d0640 1 -- 172.21.15.139:0/1354529570 learned_addr learned my addr 172.21.15.139:0/1354529570 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:43.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f6320ccf640 1 --2- 172.21.15.139:0/1354529570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f631c0b9370 0x7f631c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f6321cd1640 1 --2- 172.21.15.139:0/1354529570 >> v2:172.21.15.150:3300/0 conn(0x7f631c0cec80 0x7f631c06c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f63214d0640 1 -- 172.21.15.139:0/1354529570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f631c0b9370 msgr2=0x7f631c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:43.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f63214d0640 1 --2- 172.21.15.139:0/1354529570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f631c0b9370 0x7f631c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f63214d0640 1 -- 172.21.15.139:0/1354529570 >> v2:172.21.15.150:3300/0 conn(0x7f631c0cec80 msgr2=0x7f631c06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:43.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f63214d0640 1 --2- 172.21.15.139:0/1354529570 >> v2:172.21.15.150:3300/0 conn(0x7f631c0cec80 0x7f631c06c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.628+0000 7f63214d0640 1 -- 172.21.15.139:0/1354529570 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c162380 con 0x7f631c15dbf0 2024-04-25T09:54:43.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.629+0000 7f63214d0640 1 --2- 172.21.15.139:0/1354529570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f631c15dbf0 0x7f631c1617e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f631c08c170 tx=0x7f631802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85009a774139df5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.629+0000 7f6313fff640 1 -- 172.21.15.139:0/1354529570 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318037070 con 0x7f631c15dbf0 2024-04-25T09:54:43.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.629+0000 7f6313fff640 1 -- 172.21.15.139:0/1354529570 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6318032440 con 0x7f631c15dbf0 2024-04-25T09:54:43.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.629+0000 7f6313fff640 1 -- 172.21.15.139:0/1354529570 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318032810 con 0x7f631c15dbf0 2024-04-25T09:54:43.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.629+0000 7f6323734640 1 -- 172.21.15.139:0/1354529570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f631c15dbf0 msgr2=0x7f631c1617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.629+0000 7f6323734640 1 --2- 172.21.15.139:0/1354529570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f631c15dbf0 0x7f631c1617e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f631c08c170 tx=0x7f631802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.629+0000 7f6323734640 1 -- 172.21.15.139:0/1354529570 shutdown_connections 2024-04-25T09:54:43.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.629+0000 7f6323734640 1 --2- 172.21.15.139:0/1354529570 >> v2:172.21.15.150:3300/0 conn(0x7f631c0cec80 0x7f631c06c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.629+0000 7f6323734640 1 --2- 172.21.15.139:0/1354529570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f631c0b9370 0x7f631c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.629+0000 7f6323734640 1 --2- 172.21.15.139:0/1354529570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f631c15dbf0 0x7f631c1617e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.629+0000 7f6323734640 1 -- 172.21.15.139:0/1354529570 >> 172.21.15.139:0/1354529570 conn(0x7f631c0b5480 msgr2=0x7f631c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:43.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.630+0000 7f6323734640 1 -- 172.21.15.139:0/1354529570 shutdown_connections 2024-04-25T09:54:43.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.630+0000 7f6323734640 1 -- 172.21.15.139:0/1354529570 wait complete. 2024-04-25T09:54:43.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.631+0000 7f6323734640 1 Processor -- start 2024-04-25T09:54:43.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.631+0000 7f6323734640 1 -- start start 2024-04-25T09:54:43.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.631+0000 7f6323734640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f631c0b9370 0x7f631c15f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.631+0000 7f6323734640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f631c0cec80 0x7f631c0ab450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.631+0000 7f6323734640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f631c15dbf0 0x7f631c1f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.631+0000 7f6323734640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c15dfd0 con 0x7f631c0cec80 2024-04-25T09:54:43.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.631+0000 7f6323734640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c0b97d0 con 0x7f631c0b9370 2024-04-25T09:54:43.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.631+0000 7f6323734640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c160270 con 0x7f631c15dbf0 2024-04-25T09:54:43.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.631+0000 7f6320ccf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f631c0cec80 0x7f631c0ab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.631+0000 7f6320ccf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f631c0cec80 0x7f631c0ab450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58416/0 (socket says 172.21.15.139:58416) 2024-04-25T09:54:43.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.631+0000 7f6320ccf640 1 -- 172.21.15.139:0/777607948 learned_addr learned my addr 172.21.15.139:0/777607948 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:43.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f6321cd1640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f631c15dbf0 0x7f631c1f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f63214d0640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f631c0b9370 0x7f631c15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f6320ccf640 1 -- 172.21.15.139:0/777607948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f631c15dbf0 msgr2=0x7f631c1f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f6320ccf640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f631c15dbf0 0x7f631c1f37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f6320ccf640 1 -- 172.21.15.139:0/777607948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f631c0b9370 msgr2=0x7f631c15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f6320ccf640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f631c0b9370 0x7f631c15f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f6320ccf640 1 -- 172.21.15.139:0/777607948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c15f940 con 0x7f631c0cec80 2024-04-25T09:54:43.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f6320ccf640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f631c0cec80 0x7f631c0ab450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f630c00e870 tx=0x7f630c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f6311ffb640 1 -- 172.21.15.139:0/777607948 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c017070 con 0x7f631c0cec80 2024-04-25T09:54:43.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f6311ffb640 1 -- 172.21.15.139:0/777607948 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c00ee60 con 0x7f631c0cec80 2024-04-25T09:54:43.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f6323734640 1 -- 172.21.15.139:0/777607948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631c15fc20 con 0x7f631c0cec80 2024-04-25T09:54:43.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f6311ffb640 1 -- 172.21.15.139:0/777607948 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c00cb20 con 0x7f631c0cec80 2024-04-25T09:54:43.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.632+0000 7f6323734640 1 -- 172.21.15.139:0/777607948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631c1f3ce0 con 0x7f631c0cec80 2024-04-25T09:54:43.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.633+0000 7f6323734640 1 -- 172.21.15.139:0/777607948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631c074520 con 0x7f631c0cec80 2024-04-25T09:54:43.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.634+0000 7f6311ffb640 1 -- 172.21.15.139:0/777607948 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f630c01a070 con 0x7f631c0cec80 2024-04-25T09:54:43.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.634+0000 7f6311ffb640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f630006b4d0 0x7f630006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.634+0000 7f63214d0640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f630006b4d0 0x7f630006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.634+0000 7f6311ffb640 1 -- 172.21.15.139:0/777607948 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f630c08a3d0 con 0x7f631c0cec80 2024-04-25T09:54:43.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.635+0000 7f63214d0640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f630006b4d0 0x7f630006d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6318034000 tx=0x7f631802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.639+0000 7f6311ffb640 1 -- 172.21.15.139:0/777607948 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f630c058b00 con 0x7f631c0cec80 2024-04-25T09:54:43.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.807+0000 7f6323734640 1 -- 172.21.15.139:0/777607948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f631c0d1680 con 0x7f631c0cec80 2024-04-25T09:54:43.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.808+0000 7f6311ffb640 1 -- 172.21.15.139:0/777607948 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f630c05cba0 con 0x7f631c0cec80 2024-04-25T09:54:43.810 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:54:43.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.809+0000 7f6323734640 1 -- 172.21.15.139:0/777607948 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f630006b4d0 msgr2=0x7f630006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.809+0000 7f6323734640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f630006b4d0 0x7f630006d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6318034000 tx=0x7f631802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:43.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.809+0000 7f6323734640 1 -- 172.21.15.139:0/777607948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f631c0cec80 msgr2=0x7f631c0ab450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.809+0000 7f6323734640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f631c0cec80 0x7f631c0ab450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f630c00e870 tx=0x7f630c00ed30 comp rx=0 tx=0).stop 2024-04-25T09:54:43.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.809+0000 7f6323734640 1 -- 172.21.15.139:0/777607948 shutdown_connections 2024-04-25T09:54:43.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.809+0000 7f6323734640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f631c0cec80 0x7f631c0ab450 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.809+0000 7f6323734640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f630006b4d0 0x7f630006d990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.809+0000 7f6323734640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f631c15dbf0 0x7f631c1f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.809+0000 7f6323734640 1 --2- 172.21.15.139:0/777607948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f631c0b9370 0x7f631c15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.809+0000 7f6323734640 1 -- 172.21.15.139:0/777607948 >> 172.21.15.139:0/777607948 conn(0x7f631c0b5480 msgr2=0x7f631c0b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:43.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.810+0000 7f6323734640 1 -- 172.21.15.139:0/777607948 shutdown_connections 2024-04-25T09:54:43.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.810+0000 7f6323734640 1 -- 172.21.15.139:0/777607948 wait complete. 2024-04-25T09:54:43.823 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:54:43.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.920+0000 7ffa76d89640 1 Processor -- start 2024-04-25T09:54:43.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.920+0000 7ffa76d89640 1 -- start start 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.921+0000 7ffa76d89640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa7015dbf0 0x7ffa701617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.921+0000 7ffa76d89640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffa700cee50 0x7ffa700d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.921+0000 7ffa76d89640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa70161dc0 con 0x7ffa700b9370 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.921+0000 7ffa76d89640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa700d1ee0 con 0x7ffa700cee50 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.921+0000 7ffa76d89640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa701620a0 con 0x7ffa7015dbf0 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.921+0000 7ffa75326640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffa700cee50 0x7ffa700d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.921+0000 7ffa6ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ffa700b9370 0x7ffa700ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54610/0 (socket says 172.21.15.139:54610) 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.921+0000 7ffa6ffff640 1 -- 172.21.15.139:0/1880623074 learned_addr learned my addr 172.21.15.139:0/1880623074 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.921+0000 7ffa74b25640 1 --2- 172.21.15.139:0/1880623074 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa7015dbf0 0x7ffa701617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.922+0000 7ffa75326640 1 -- 172.21.15.139:0/1880623074 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa7015dbf0 msgr2=0x7ffa701617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.922+0000 7ffa75326640 1 --2- 172.21.15.139:0/1880623074 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa7015dbf0 0x7ffa701617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.922+0000 7ffa75326640 1 -- 172.21.15.139:0/1880623074 >> v1:172.21.15.139:6789/0 conn(0x7ffa700b9370 legacy=0x7ffa700ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.922+0000 7ffa75326640 1 -- 172.21.15.139:0/1880623074 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa70162380 con 0x7ffa700cee50 2024-04-25T09:54:43.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.922+0000 7ffa6f7fe640 1 -- 172.21.15.139:0/1880623074 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975095789 0 0) 0x7ffa70161dc0 con 0x7ffa700b9370 2024-04-25T09:54:43.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.922+0000 7ffa74b25640 1 --2- 172.21.15.139:0/1880623074 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa7015dbf0 0x7ffa701617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:43.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.922+0000 7ffa75326640 1 --2- 172.21.15.139:0/1880623074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa700cee50 0x7ffa700d2370 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009d90 tx=0x7ffa6802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4f83b2df641d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.922+0000 7ffa6f7fe640 1 -- 172.21.15.139:0/1880623074 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68007740 con 0x7ffa700cee50 2024-04-25T09:54:43.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.922+0000 7ffa6f7fe640 1 -- 172.21.15.139:0/1880623074 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa680078e0 con 0x7ffa700cee50 2024-04-25T09:54:43.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.922+0000 7ffa6f7fe640 1 -- 172.21.15.139:0/1880623074 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68007cb0 con 0x7ffa700cee50 2024-04-25T09:54:43.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.923+0000 7ffa76d89640 1 -- 172.21.15.139:0/1880623074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa700cee50 msgr2=0x7ffa700d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.923+0000 7ffa76d89640 1 --2- 172.21.15.139:0/1880623074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa700cee50 0x7ffa700d2370 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009d90 tx=0x7ffa6802a660 comp rx=0 tx=0).stop 2024-04-25T09:54:43.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.923+0000 7ffa76d89640 1 -- 172.21.15.139:0/1880623074 shutdown_connections 2024-04-25T09:54:43.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.923+0000 7ffa76d89640 1 --2- 172.21.15.139:0/1880623074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa700cee50 0x7ffa700d2370 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.923+0000 7ffa76d89640 1 --2- 172.21.15.139:0/1880623074 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa7015dbf0 0x7ffa701617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.923+0000 7ffa76d89640 1 -- 172.21.15.139:0/1880623074 >> 172.21.15.139:0/1880623074 conn(0x7ffa700b5480 msgr2=0x7ffa700b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:43.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.923+0000 7ffa76d89640 1 -- 172.21.15.139:0/1880623074 shutdown_connections 2024-04-25T09:54:43.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.923+0000 7ffa76d89640 1 -- 172.21.15.139:0/1880623074 wait complete. 2024-04-25T09:54:43.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.924+0000 7ffa76d89640 1 Processor -- start 2024-04-25T09:54:43.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.924+0000 7ffa76d89640 1 -- start start 2024-04-25T09:54:43.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa76d89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa700b9370 0x7ffa701f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa76d89640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa700cee50 0x7ffa7015cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa76d89640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa7015dbf0 0x7ffa700d3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa76d89640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7015dfd0 con 0x7ffa700b9370 2024-04-25T09:54:43.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa76d89640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700b97d0 con 0x7ffa700cee50 2024-04-25T09:54:43.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa76d89640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7015f550 con 0x7ffa7015dbf0 2024-04-25T09:54:43.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa74b25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa700b9370 0x7ffa701f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa74b25640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa700b9370 0x7ffa701f7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58442/0 (socket says 172.21.15.139:58442) 2024-04-25T09:54:43.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa74b25640 1 -- 172.21.15.139:0/2627557662 learned_addr learned my addr 172.21.15.139:0/2627557662 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:43.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa6ffff640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa700cee50 0x7ffa7015cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa75326640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa7015dbf0 0x7ffa700d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa74b25640 1 -- 172.21.15.139:0/2627557662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa7015dbf0 msgr2=0x7ffa700d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa74b25640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa7015dbf0 0x7ffa700d3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa74b25640 1 -- 172.21.15.139:0/2627557662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa700cee50 msgr2=0x7ffa7015cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa74b25640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa700cee50 0x7ffa7015cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa74b25640 1 -- 172.21.15.139:0/2627557662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa701f7e90 con 0x7ffa700b9370 2024-04-25T09:54:43.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa6ffff640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa700cee50 0x7ffa7015cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:43.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.925+0000 7ffa75326640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa7015dbf0 0x7ffa700d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:43.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.926+0000 7ffa74b25640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa700b9370 0x7ffa701f7950 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000e7e0 tx=0x7ffa6000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.926+0000 7ffa6d7fa640 1 -- 172.21.15.139:0/2627557662 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000f040 con 0x7ffa700b9370 2024-04-25T09:54:43.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.926+0000 7ffa6d7fa640 1 -- 172.21.15.139:0/2627557662 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60004030 con 0x7ffa700b9370 2024-04-25T09:54:43.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.926+0000 7ffa76d89640 1 -- 172.21.15.139:0/2627557662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa70155930 con 0x7ffa700b9370 2024-04-25T09:54:43.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.926+0000 7ffa6d7fa640 1 -- 172.21.15.139:0/2627557662 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60004400 con 0x7ffa700b9370 2024-04-25T09:54:43.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.926+0000 7ffa76d89640 1 -- 172.21.15.139:0/2627557662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7015ec20 con 0x7ffa700b9370 2024-04-25T09:54:43.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.927+0000 7ffa76d89640 1 -- 172.21.15.139:0/2627557662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa700cf630 con 0x7ffa700b9370 2024-04-25T09:54:43.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.927+0000 7ffa6d7fa640 1 -- 172.21.15.139:0/2627557662 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa6000c6d0 con 0x7ffa700b9370 2024-04-25T09:54:43.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.928+0000 7ffa6d7fa640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.928+0000 7ffa6d7fa640 1 -- 172.21.15.139:0/2627557662 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffa6008a150 con 0x7ffa700b9370 2024-04-25T09:54:43.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.928+0000 7ffa6ffff640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.928+0000 7ffa6ffff640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffa700d4680 tx=0x7ffa64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:43.932+0000 7ffa6d7fa640 1 -- 172.21.15.139:0/2627557662 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa700cf630 con 0x7ffa700b9370 2024-04-25T09:54:44.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.102+0000 7ffa76d89640 1 -- 172.21.15.139:0/2627557662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffa70074520 con 0x7ffa700b9370 2024-04-25T09:54:44.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.102+0000 7ffa6d7fa640 1 -- 172.21.15.139:0/2627557662 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffa701f7e90 con 0x7ffa700b9370 2024-04-25T09:54:44.105 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:54:44.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.105+0000 7ffa76d89640 1 -- 172.21.15.139:0/2627557662 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffa5c06b4d0 msgr2=0x7ffa5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.105+0000 7ffa76d89640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffa700d4680 tx=0x7ffa64009040 comp rx=0 tx=0).stop 2024-04-25T09:54:44.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.105+0000 7ffa76d89640 1 -- 172.21.15.139:0/2627557662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa700b9370 msgr2=0x7ffa701f7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.105+0000 7ffa76d89640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa700b9370 0x7ffa701f7950 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000e7e0 tx=0x7ffa6000eca0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.105+0000 7ffa76d89640 1 -- 172.21.15.139:0/2627557662 shutdown_connections 2024-04-25T09:54:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.105+0000 7ffa76d89640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.105+0000 7ffa76d89640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa7015dbf0 0x7ffa700d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.105+0000 7ffa76d89640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa700cee50 0x7ffa7015cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.105+0000 7ffa76d89640 1 --2- 172.21.15.139:0/2627557662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa700b9370 0x7ffa701f7950 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.105+0000 7ffa76d89640 1 -- 172.21.15.139:0/2627557662 >> 172.21.15.139:0/2627557662 conn(0x7ffa700b5480 msgr2=0x7ffa700d2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.105+0000 7ffa76d89640 1 -- 172.21.15.139:0/2627557662 shutdown_connections 2024-04-25T09:54:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.105+0000 7ffa76d89640 1 -- 172.21.15.139:0/2627557662 wait complete. 2024-04-25T09:54:44.118 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.212+0000 7ff8547d5640 1 Processor -- start 2024-04-25T09:54:44.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.212+0000 7ff8547d5640 1 -- start start 2024-04-25T09:54:44.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff8547d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff84c160e90 0x7ff84c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff8547d5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff84c06afc0 0x7ff84c070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff8547d5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff84c0d0400 con 0x7ff84c06ab60 2024-04-25T09:54:44.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff8547d5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff84c0d14a0 con 0x7ff84c06afc0 2024-04-25T09:54:44.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff8547d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0d06e0 con 0x7ff84c160e90 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff852d72640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff84c06afc0 0x7ff84c070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff852d72640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff84c06afc0 0x7ff84c070180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58454/0 (socket says 172.21.15.139:58454) 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff852d72640 1 -- 172.21.15.139:0/1272685850 learned_addr learned my addr 172.21.15.139:0/1272685850 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff852571640 1 --2- 172.21.15.139:0/1272685850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff84c160e90 0x7ff84c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff852d72640 1 -- 172.21.15.139:0/1272685850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff84c160e90 msgr2=0x7ff84c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff852d72640 1 --2- 172.21.15.139:0/1272685850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff84c160e90 0x7ff84c0afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff852d72640 1 -- 172.21.15.139:0/1272685850 >> v1:172.21.15.150:6789/0 conn(0x7ff84c06ab60 legacy=0x7ff84c06fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.213+0000 7ff852d72640 1 -- 172.21.15.139:0/1272685850 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84c0d09c0 con 0x7ff84c06afc0 2024-04-25T09:54:44.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.214+0000 7ff852d72640 1 --2- 172.21.15.139:0/1272685850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c06afc0 0x7ff84c070180 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff844009870 tx=0x7ff84402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63bdcd80f53caad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.214+0000 7ff85156f640 1 -- 172.21.15.139:0/1272685850 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844037070 con 0x7ff84c06afc0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.214+0000 7ff85156f640 1 -- 172.21.15.139:0/1272685850 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff844032440 con 0x7ff84c06afc0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.214+0000 7ff85156f640 1 -- 172.21.15.139:0/1272685850 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844032810 con 0x7ff84c06afc0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.215+0000 7ff8547d5640 1 -- 172.21.15.139:0/1272685850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c06afc0 msgr2=0x7ff84c070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.215+0000 7ff8547d5640 1 --2- 172.21.15.139:0/1272685850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c06afc0 0x7ff84c070180 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff844009870 tx=0x7ff84402a270 comp rx=0 tx=0).stop 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.215+0000 7ff8547d5640 1 -- 172.21.15.139:0/1272685850 shutdown_connections 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.215+0000 7ff8547d5640 1 --2- 172.21.15.139:0/1272685850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c06afc0 0x7ff84c070180 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.215+0000 7ff8547d5640 1 --2- 172.21.15.139:0/1272685850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff84c160e90 0x7ff84c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.215+0000 7ff8547d5640 1 -- 172.21.15.139:0/1272685850 >> 172.21.15.139:0/1272685850 conn(0x7ff84c0b23f0 msgr2=0x7ff84c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.215+0000 7ff8547d5640 1 -- 172.21.15.139:0/1272685850 shutdown_connections 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.215+0000 7ff8547d5640 1 -- 172.21.15.139:0/1272685850 wait complete. 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.216+0000 7ff8547d5640 1 Processor -- start 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.216+0000 7ff8547d5640 1 -- start start 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff8547d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff84c06ab60 0x7ff84c14c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff8547d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c06afc0 0x7ff84c158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff8547d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff84c160e90 0x7ff84c15a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff8547d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0b0000 con 0x7ff84c06afc0 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff8547d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0d0f80 con 0x7ff84c06ab60 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff8547d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c06b410 con 0x7ff84c160e90 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff851d70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c06afc0 0x7ff84c158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff851d70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c06afc0 0x7ff84c158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58470/0 (socket says 172.21.15.139:58470) 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff851d70640 1 -- 172.21.15.139:0/432621239 learned_addr learned my addr 172.21.15.139:0/432621239 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff852571640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff84c06ab60 0x7ff84c14c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff852d72640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff84c160e90 0x7ff84c15a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff851d70640 1 -- 172.21.15.139:0/432621239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff84c160e90 msgr2=0x7ff84c15a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff851d70640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff84c160e90 0x7ff84c15a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff851d70640 1 -- 172.21.15.139:0/432621239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff84c06ab60 msgr2=0x7ff84c14c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff851d70640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff84c06ab60 0x7ff84c14c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff851d70640 1 -- 172.21.15.139:0/432621239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84c1588d0 con 0x7ff84c06afc0 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff851d70640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c06afc0 0x7ff84c158390 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00c450 tx=0x7ff83c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff842ffd640 1 -- 172.21.15.139:0/432621239 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c0052d0 con 0x7ff84c06afc0 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff842ffd640 1 -- 172.21.15.139:0/432621239 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83c005470 con 0x7ff84c06afc0 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff8547d5640 1 -- 172.21.15.139:0/432621239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84c158ca0 con 0x7ff84c06afc0 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff842ffd640 1 -- 172.21.15.139:0/432621239 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c012870 con 0x7ff84c06afc0 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.217+0000 7ff8547d5640 1 -- 172.21.15.139:0/432621239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84c159260 con 0x7ff84c06afc0 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.218+0000 7ff8547d5640 1 -- 172.21.15.139:0/432621239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84c074520 con 0x7ff84c06afc0 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.219+0000 7ff842ffd640 1 -- 172.21.15.139:0/432621239 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff83c012a10 con 0x7ff84c06afc0 2024-04-25T09:54:44.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.219+0000 7ff842ffd640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff83006b4d0 0x7ff83006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.219+0000 7ff842ffd640 1 -- 172.21.15.139:0/432621239 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff83c089e40 con 0x7ff84c06afc0 2024-04-25T09:54:44.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.219+0000 7ff852571640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff83006b4d0 0x7ff83006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.220+0000 7ff852571640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff83006b4d0 0x7ff83006d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff84c157930 tx=0x7ff84800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:44.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.223+0000 7ff842ffd640 1 -- 172.21.15.139:0/432621239 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff84c1588d0 con 0x7ff84c06afc0 2024-04-25T09:54:44.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.427+0000 7ff8547d5640 1 -- 172.21.15.139:0/432621239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff84c0b43d0 con 0x7ff84c06afc0 2024-04-25T09:54:44.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.428+0000 7ff842ffd640 1 -- 172.21.15.139:0/432621239 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4944 (secure 0 0 0) 0x7ff83c0584e0 con 0x7ff84c06afc0 2024-04-25T09:54:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:44.431 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/511562594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":511562594},{"type":"v1","addr":"172.21.15.150:6834","nonce":511562594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5041,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/225357317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":225357317},{"type":"v1","addr":"172.21.15.150:6835","nonce":225357317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2499407982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2499407982},{"type":"v1","addr":"172.21.15.139:6836","nonce":2499407982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:21.988975+0000","modified":"2024-04-25T09:54:23.955620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/2069784668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2069784668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2069784668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:54:44.431 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 16 2024-04-25T09:54:44.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.430+0000 7ff8547d5640 1 -- 172.21.15.139:0/432621239 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff83006b4d0 msgr2=0x7ff83006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.430+0000 7ff8547d5640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff83006b4d0 0x7ff83006d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff84c157930 tx=0x7ff84800b040 comp rx=0 tx=0).stop 2024-04-25T09:54:44.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.430+0000 7ff8547d5640 1 -- 172.21.15.139:0/432621239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c06afc0 msgr2=0x7ff84c158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.430+0000 7ff8547d5640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c06afc0 0x7ff84c158390 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00c450 tx=0x7ff83c00c910 comp rx=0 tx=0).stop 2024-04-25T09:54:44.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.430+0000 7ff8547d5640 1 -- 172.21.15.139:0/432621239 shutdown_connections 2024-04-25T09:54:44.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.430+0000 7ff8547d5640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff84c06afc0 0x7ff84c158390 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.430+0000 7ff8547d5640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff83006b4d0 0x7ff83006d990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.430+0000 7ff8547d5640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff84c160e90 0x7ff84c15a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.430+0000 7ff8547d5640 1 --2- 172.21.15.139:0/432621239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff84c06ab60 0x7ff84c14c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.430+0000 7ff8547d5640 1 -- 172.21.15.139:0/432621239 >> 172.21.15.139:0/432621239 conn(0x7ff84c0b23f0 msgr2=0x7ff84c0cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:44.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.430+0000 7ff8547d5640 1 -- 172.21.15.139:0/432621239 shutdown_connections 2024-04-25T09:54:44.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.430+0000 7ff8547d5640 1 -- 172.21.15.139:0/432621239 wait complete. 2024-04-25T09:54:44.444 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:44.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.538+0000 7fa2921f4640 1 Processor -- start 2024-04-25T09:54:44.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.538+0000 7fa2921f4640 1 -- start start 2024-04-25T09:54:44.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.538+0000 7fa2921f4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa28c158240 0x7fa28c0d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.538+0000 7fa2921f4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa28c0d1860 0x7fa28c0cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.538+0000 7fa2921f4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa28c0cbf80 con 0x7fa28c0cfba0 2024-04-25T09:54:44.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.538+0000 7fa2921f4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28c158620 con 0x7fa28c0d1860 2024-04-25T09:54:44.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.538+0000 7fa2921f4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0cce10 con 0x7fa28c158240 2024-04-25T09:54:44.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.539+0000 7fa28b7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa28c158240 0x7fa28c0d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.539+0000 7fa28affd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa28c0d1860 0x7fa28c0cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.539+0000 7fa28bfff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fa28c0cfba0 0x7fa28c1616d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:42620/0 (socket says 172.21.15.139:42620) 2024-04-25T09:54:44.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.539+0000 7fa28bfff640 1 -- 172.21.15.139:0/391501541 learned_addr learned my addr 172.21.15.139:0/391501541 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:44.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.539+0000 7fa28b7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa28c158240 0x7fa28c0d1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:34548/0 (socket says 172.21.15.139:34548) 2024-04-25T09:54:44.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.539+0000 7fa28affd640 1 -- 172.21.15.139:0/391501541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa28c158240 msgr2=0x7fa28c0d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:44.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.539+0000 7fa28affd640 1 --2- 172.21.15.139:0/391501541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa28c158240 0x7fa28c0d1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.539+0000 7fa28affd640 1 -- 172.21.15.139:0/391501541 >> v1:172.21.15.150:6789/0 conn(0x7fa28c0cfba0 legacy=0x7fa28c1616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.539+0000 7fa28affd640 1 -- 172.21.15.139:0/391501541 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c0ceb50 con 0x7fa28c0d1860 2024-04-25T09:54:44.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.539+0000 7fa28b7fe640 1 --2- 172.21.15.139:0/391501541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa28c158240 0x7fa28c0d1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:44.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.540+0000 7fa28affd640 1 --2- 172.21.15.139:0/391501541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa28c0d1860 0x7fa28c0cf660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa2800097a0 tx=0x7fa28002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2335b3d4a8ca2da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:44.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.540+0000 7fa28a7fc640 1 -- 172.21.15.139:0/391501541 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280037070 con 0x7fa28c0d1860 2024-04-25T09:54:44.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.540+0000 7fa28a7fc640 1 -- 172.21.15.139:0/391501541 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa280032440 con 0x7fa28c0d1860 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.540+0000 7fa28a7fc640 1 -- 172.21.15.139:0/391501541 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280032810 con 0x7fa28c0d1860 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.541+0000 7fa2921f4640 1 -- 172.21.15.139:0/391501541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa28c0d1860 msgr2=0x7fa28c0cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.541+0000 7fa2921f4640 1 --2- 172.21.15.139:0/391501541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa28c0d1860 0x7fa28c0cf660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa2800097a0 tx=0x7fa28002a180 comp rx=0 tx=0).stop 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.541+0000 7fa2921f4640 1 -- 172.21.15.139:0/391501541 shutdown_connections 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.541+0000 7fa2921f4640 1 --2- 172.21.15.139:0/391501541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa28c0d1860 0x7fa28c0cf660 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.541+0000 7fa2921f4640 1 --2- 172.21.15.139:0/391501541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa28c158240 0x7fa28c0d1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.541+0000 7fa2921f4640 1 -- 172.21.15.139:0/391501541 >> 172.21.15.139:0/391501541 conn(0x7fa28c0b3380 msgr2=0x7fa28c0b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.541+0000 7fa2921f4640 1 -- 172.21.15.139:0/391501541 shutdown_connections 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.541+0000 7fa2921f4640 1 -- 172.21.15.139:0/391501541 wait complete. 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.542+0000 7fa2921f4640 1 Processor -- start 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.542+0000 7fa2921f4640 1 -- start start 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa2921f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa28c0cfba0 0x7fa28c1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa2921f4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa28c0d1860 0x7fa28c0b6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa2921f4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa28c158240 0x7fa28c153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa2921f4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0cc4e0 con 0x7fa28c0cfba0 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa2921f4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0cc200 con 0x7fa28c0d1860 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa2921f4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0cc7c0 con 0x7fa28c158240 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa28b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa28c0cfba0 0x7fa28c1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa28b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa28c0cfba0 0x7fa28c1564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58484/0 (socket says 172.21.15.139:58484) 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa28b7fe640 1 -- 172.21.15.139:0/3138455457 learned_addr learned my addr 172.21.15.139:0/3138455457 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa28bfff640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa28c158240 0x7fa28c153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa28affd640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa28c0d1860 0x7fa28c0b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa28b7fe640 1 -- 172.21.15.139:0/3138455457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa28c158240 msgr2=0x7fa28c153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa28b7fe640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa28c158240 0x7fa28c153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa28b7fe640 1 -- 172.21.15.139:0/3138455457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa28c0d1860 msgr2=0x7fa28c0b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa28b7fe640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa28c0d1860 0x7fa28c0b6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa28b7fe640 1 -- 172.21.15.139:0/3138455457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c0b6ac0 con 0x7fa28c0cfba0 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa28b7fe640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa28c0cfba0 0x7fa28c1564b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00e960 tx=0x7fa27c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa273fff640 1 -- 172.21.15.139:0/3138455457 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c017070 con 0x7fa28c0cfba0 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa273fff640 1 -- 172.21.15.139:0/3138455457 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c00ca00 con 0x7fa28c0cfba0 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa273fff640 1 -- 172.21.15.139:0/3138455457 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00cdd0 con 0x7fa28c0cfba0 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa2921f4640 1 -- 172.21.15.139:0/3138455457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c153f50 con 0x7fa28c0cfba0 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.543+0000 7fa2921f4640 1 -- 172.21.15.139:0/3138455457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c1559d0 con 0x7fa28c0cfba0 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.544+0000 7fa2921f4640 1 -- 172.21.15.139:0/3138455457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c074520 con 0x7fa28c0cfba0 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.545+0000 7fa273fff640 1 -- 172.21.15.139:0/3138455457 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa27c01a070 con 0x7fa28c0cfba0 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.545+0000 7fa273fff640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa27806b4d0 0x7fa27806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.545+0000 7fa273fff640 1 -- 172.21.15.139:0/3138455457 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa27c08a150 con 0x7fa28c0cfba0 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.545+0000 7fa28affd640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa27806b4d0 0x7fa27806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.546+0000 7fa28affd640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa27806b4d0 0x7fa27806d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa280034000 tx=0x7fa28002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:44.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.549+0000 7fa273fff640 1 -- 172.21.15.139:0/3138455457 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa27c058880 con 0x7fa28c0cfba0 2024-04-25T09:54:44.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.723+0000 7fa2921f4640 1 -- 172.21.15.139:0/3138455457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa28c0b7460 con 0x7fa28c0cfba0 2024-04-25T09:54:44.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.724+0000 7fa273fff640 1 -- 172.21.15.139:0/3138455457 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16990 (secure 0 0 0) 0x7fa27c05c920 con 0x7fa28c0cfba0 2024-04-25T09:54:44.727 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:44.727 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":38,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:23.620264+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:19.795203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:20.931184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:44.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.726+0000 7fa2921f4640 1 -- 172.21.15.139:0/3138455457 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa27806b4d0 msgr2=0x7fa27806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.726+0000 7fa2921f4640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa27806b4d0 0x7fa27806d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa280034000 tx=0x7fa28002e000 comp rx=0 tx=0).stop 2024-04-25T09:54:44.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.726+0000 7fa2921f4640 1 -- 172.21.15.139:0/3138455457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa28c0cfba0 msgr2=0x7fa28c1564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.726+0000 7fa2921f4640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa28c0cfba0 0x7fa28c1564b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00e960 tx=0x7fa27c00ee20 comp rx=0 tx=0).stop 2024-04-25T09:54:44.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.726+0000 7fa2921f4640 1 -- 172.21.15.139:0/3138455457 shutdown_connections 2024-04-25T09:54:44.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.726+0000 7fa2921f4640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa27806b4d0 0x7fa27806d990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.726+0000 7fa2921f4640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa28c158240 0x7fa28c153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.726+0000 7fa2921f4640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa28c0d1860 0x7fa28c0b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.726+0000 7fa2921f4640 1 --2- 172.21.15.139:0/3138455457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa28c0cfba0 0x7fa28c1564b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.726+0000 7fa2921f4640 1 -- 172.21.15.139:0/3138455457 >> 172.21.15.139:0/3138455457 conn(0x7fa28c0b3380 msgr2=0x7fa28c15f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:44.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.726+0000 7fa2921f4640 1 -- 172.21.15.139:0/3138455457 shutdown_connections 2024-04-25T09:54:44.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.726+0000 7fa2921f4640 1 -- 172.21.15.139:0/3138455457 wait complete. 2024-04-25T09:54:44.740 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:54:44.740 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:44.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.834+0000 7fc3a3041640 1 Processor -- start 2024-04-25T09:54:44.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.834+0000 7fc3a3041640 1 -- start start 2024-04-25T09:54:44.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc3a3041640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc39c15b5e0 0x7fc39c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc3a3041640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc39c0b2db0 0x7fc39c0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc3a3041640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc39c0b3b80 con 0x7fc39c0b3770 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc3a3041640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc39c06da20 con 0x7fc39c0b2db0 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc3a3041640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc39c06f860 con 0x7fc39c15b5e0 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc39bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc39c0b2db0 0x7fc39c0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc39bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc39c0b2db0 0x7fc39c0b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58498/0 (socket says 172.21.15.139:58498) 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc39bfff640 1 -- 172.21.15.139:0/3162342035 learned_addr learned my addr 172.21.15.139:0/3162342035 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc3a0ddd640 1 --2- 172.21.15.139:0/3162342035 >> v2:172.21.15.150:3300/0 conn(0x7fc39c15b5e0 0x7fc39c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc39bfff640 1 -- 172.21.15.139:0/3162342035 >> v2:172.21.15.150:3300/0 conn(0x7fc39c15b5e0 msgr2=0x7fc39c15df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc39bfff640 1 --2- 172.21.15.139:0/3162342035 >> v2:172.21.15.150:3300/0 conn(0x7fc39c15b5e0 0x7fc39c15df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc39bfff640 1 -- 172.21.15.139:0/3162342035 >> v1:172.21.15.150:6789/0 conn(0x7fc39c0b3770 legacy=0x7fc39c161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.835+0000 7fc39bfff640 1 -- 172.21.15.139:0/3162342035 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c06dc50 con 0x7fc39c0b2db0 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.836+0000 7fc39bfff640 1 --2- 172.21.15.139:0/3162342035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c0b2db0 0x7fc39c0b3230 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc39c066e00 tx=0x7fc39002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150c60c56b1960d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:44.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.836+0000 7fc39b7fe640 1 -- 172.21.15.139:0/3162342035 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc390037070 con 0x7fc39c0b2db0 2024-04-25T09:54:44.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.836+0000 7fc39b7fe640 1 -- 172.21.15.139:0/3162342035 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc390032440 con 0x7fc39c0b2db0 2024-04-25T09:54:44.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.836+0000 7fc39b7fe640 1 -- 172.21.15.139:0/3162342035 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc390032810 con 0x7fc39c0b2db0 2024-04-25T09:54:44.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.837+0000 7fc3a3041640 1 -- 172.21.15.139:0/3162342035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c0b2db0 msgr2=0x7fc39c0b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.837+0000 7fc3a3041640 1 --2- 172.21.15.139:0/3162342035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c0b2db0 0x7fc39c0b3230 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc39c066e00 tx=0x7fc39002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.837+0000 7fc3a3041640 1 -- 172.21.15.139:0/3162342035 shutdown_connections 2024-04-25T09:54:44.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.837+0000 7fc3a3041640 1 --2- 172.21.15.139:0/3162342035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c0b2db0 0x7fc39c0b3230 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.837+0000 7fc3a3041640 1 --2- 172.21.15.139:0/3162342035 >> v2:172.21.15.150:3300/0 conn(0x7fc39c15b5e0 0x7fc39c15df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.837+0000 7fc3a3041640 1 -- 172.21.15.139:0/3162342035 >> 172.21.15.139:0/3162342035 conn(0x7fc39c0b23d0 msgr2=0x7fc39c0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:44.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.837+0000 7fc3a3041640 1 -- 172.21.15.139:0/3162342035 shutdown_connections 2024-04-25T09:54:44.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.837+0000 7fc3a3041640 1 -- 172.21.15.139:0/3162342035 wait complete. 2024-04-25T09:54:44.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.838+0000 7fc3a3041640 1 Processor -- start 2024-04-25T09:54:44.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.838+0000 7fc3a3041640 1 -- start start 2024-04-25T09:54:44.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a3041640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c0b2db0 0x7fc39c0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a3041640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc39c0b3770 0x7fc39c1455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a3041640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc39c15b5e0 0x7fc39c147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a3041640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c06e4f0 con 0x7fc39c0b2db0 2024-04-25T09:54:44.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a3041640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c06e210 con 0x7fc39c15b5e0 2024-04-25T09:54:44.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a3041640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c06e7d0 con 0x7fc39c0b3770 2024-04-25T09:54:44.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a0ddd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c0b2db0 0x7fc39c0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a0ddd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c0b2db0 0x7fc39c0ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58504/0 (socket says 172.21.15.139:58504) 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a0ddd640 1 -- 172.21.15.139:0/1741559112 learned_addr learned my addr 172.21.15.139:0/1741559112 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc39bfff640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc39c0b3770 0x7fc39c1455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a15de640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc39c15b5e0 0x7fc39c147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a0ddd640 1 -- 172.21.15.139:0/1741559112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc39c0b3770 msgr2=0x7fc39c1455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a0ddd640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc39c0b3770 0x7fc39c1455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a0ddd640 1 -- 172.21.15.139:0/1741559112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc39c15b5e0 msgr2=0x7fc39c147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a0ddd640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc39c15b5e0 0x7fc39c147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.839+0000 7fc3a0ddd640 1 -- 172.21.15.139:0/1741559112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c1e76a0 con 0x7fc39c0b2db0 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.840+0000 7fc3a0ddd640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c0b2db0 0x7fc39c0ccb00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc38c00d600 tx=0x7fc38c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.840+0000 7fc3997fa640 1 -- 172.21.15.139:0/1741559112 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0125f0 con 0x7fc39c0b2db0 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.840+0000 7fc3997fa640 1 -- 172.21.15.139:0/1741559112 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c019070 con 0x7fc39c0b2db0 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.840+0000 7fc3a3041640 1 -- 172.21.15.139:0/1741559112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39c1e7980 con 0x7fc39c0b2db0 2024-04-25T09:54:44.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.840+0000 7fc3997fa640 1 -- 172.21.15.139:0/1741559112 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0129f0 con 0x7fc39c0b2db0 2024-04-25T09:54:44.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.840+0000 7fc3a3041640 1 -- 172.21.15.139:0/1741559112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39c146270 con 0x7fc39c0b2db0 2024-04-25T09:54:44.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.841+0000 7fc3a3041640 1 -- 172.21.15.139:0/1741559112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc39c074520 con 0x7fc39c0b2db0 2024-04-25T09:54:44.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.841+0000 7fc3997fa640 1 -- 172.21.15.139:0/1741559112 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc38c012b90 con 0x7fc39c0b2db0 2024-04-25T09:54:44.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.841+0000 7fc3997fa640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc38806b4d0 0x7fc38806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.841+0000 7fc3997fa640 1 -- 172.21.15.139:0/1741559112 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc38c08a780 con 0x7fc39c0b2db0 2024-04-25T09:54:44.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.842+0000 7fc39bfff640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc38806b4d0 0x7fc38806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.842+0000 7fc39bfff640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc38806b4d0 0x7fc38806d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc390034000 tx=0x7fc39002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:44.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:44.846+0000 7fc3997fa640 1 -- 172.21.15.139:0/1741559112 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc38c058d90 con 0x7fc39c0b2db0 2024-04-25T09:54:45.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.051+0000 7fc3a3041640 1 -- 172.21.15.139:0/1741559112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc39c0d0aa0 con 0x7fc39c0b2db0 2024-04-25T09:54:45.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.051+0000 7fc3997fa640 1 -- 172.21.15.139:0/1741559112 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fc38c05ce30 con 0x7fc39c0b2db0 2024-04-25T09:54:45.054 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:45.054 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/511562594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":511562594},{"type":"v1","addr":"172.21.15.150:6834","nonce":511562594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5041,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/225357317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":225357317},{"type":"v1","addr":"172.21.15.150:6835","nonce":225357317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2499407982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2499407982},{"type":"v1","addr":"172.21.15.139:6836","nonce":2499407982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:21.988975+0000","modified":"2024-04-25T09:54:23.955620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/2069784668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2069784668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2069784668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:54:45.054 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 16 2024-04-25T09:54:45.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.053+0000 7fc3a3041640 1 -- 172.21.15.139:0/1741559112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc38806b4d0 msgr2=0x7fc38806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.053+0000 7fc3a3041640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc38806b4d0 0x7fc38806d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc390034000 tx=0x7fc39002e000 comp rx=0 tx=0).stop 2024-04-25T09:54:45.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.053+0000 7fc3a3041640 1 -- 172.21.15.139:0/1741559112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c0b2db0 msgr2=0x7fc39c0ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.053+0000 7fc3a3041640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c0b2db0 0x7fc39c0ccb00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc38c00d600 tx=0x7fc38c00dac0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.053+0000 7fc3a3041640 1 -- 172.21.15.139:0/1741559112 shutdown_connections 2024-04-25T09:54:45.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.053+0000 7fc3a3041640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc38806b4d0 0x7fc38806d990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.053+0000 7fc3a3041640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc39c15b5e0 0x7fc39c147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.053+0000 7fc3a3041640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc39c0b3770 0x7fc39c1455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.053+0000 7fc3a3041640 1 --2- 172.21.15.139:0/1741559112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc39c0b2db0 0x7fc39c0ccb00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.053+0000 7fc3a3041640 1 -- 172.21.15.139:0/1741559112 >> 172.21.15.139:0/1741559112 conn(0x7fc39c0b23d0 msgr2=0x7fc39c160570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:45.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.053+0000 7fc3a3041640 1 -- 172.21.15.139:0/1741559112 shutdown_connections 2024-04-25T09:54:45.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.053+0000 7fc3a3041640 1 -- 172.21.15.139:0/1741559112 wait complete. 2024-04-25T09:54:45.067 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:45.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.162+0000 7f65eb0dd640 1 Processor -- start 2024-04-25T09:54:45.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.162+0000 7f65eb0dd640 1 -- start start 2024-04-25T09:54:45.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.162+0000 7f65eb0dd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f65e415dc60 0x7f65e40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.162+0000 7f65eb0dd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e40cf250 con 0x7f65e40d1080 2024-04-25T09:54:45.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.162+0000 7f65eb0dd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e40ce9b0 con 0x7f65e4161850 2024-04-25T09:54:45.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.162+0000 7f65eb0dd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e40d3730 con 0x7f65e415dc60 2024-04-25T09:54:45.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.162+0000 7f65e8e79640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f65e415dc60 0x7f65e40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.163+0000 7f65e967a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f65e40d1080 0x7f65e40d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54616/0 (socket says 172.21.15.139:54616) 2024-04-25T09:54:45.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.163+0000 7f65e967a640 1 -- 172.21.15.139:0/4097991633 learned_addr learned my addr 172.21.15.139:0/4097991633 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:45.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.163+0000 7f65e37fe640 1 -- 172.21.15.139:0/4097991633 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864674051 0 0) 0x7f65cc001110 con 0x7f65e40d1080 2024-04-25T09:54:45.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.163+0000 7f65e37fe640 1 -- 172.21.15.139:0/4097991633 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65d0003520 con 0x7f65e40d1080 2024-04-25T09:54:45.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.163+0000 7f65e8e79640 1 -- 172.21.15.139:0/4097991633 >> v1:172.21.15.150:6789/0 conn(0x7f65e4161850 legacy=0x7f65e40d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.163+0000 7f65e8e79640 1 -- 172.21.15.139:0/4097991633 >> v1:172.21.15.139:6789/0 conn(0x7f65e40d1080 legacy=0x7f65e40d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.163+0000 7f65e8e79640 1 -- 172.21.15.139:0/4097991633 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40cec90 con 0x7f65e415dc60 2024-04-25T09:54:45.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.164+0000 7f65e8e79640 1 --2- 172.21.15.139:0/4097991633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65e415dc60 0x7f65e40b6c10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009d90 tx=0x7f65dc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e854cf248fcd80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.164+0000 7f65e37fe640 1 -- 172.21.15.139:0/4097991633 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc007740 con 0x7f65e415dc60 2024-04-25T09:54:45.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.164+0000 7f65e37fe640 1 -- 172.21.15.139:0/4097991633 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc0078e0 con 0x7f65e415dc60 2024-04-25T09:54:45.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.164+0000 7f65e37fe640 1 -- 172.21.15.139:0/4097991633 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc007cb0 con 0x7f65e415dc60 2024-04-25T09:54:45.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.164+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4097991633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65e415dc60 msgr2=0x7f65e40b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.164+0000 7f65eb0dd640 1 --2- 172.21.15.139:0/4097991633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65e415dc60 0x7f65e40b6c10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009d90 tx=0x7f65dc02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.164+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4097991633 shutdown_connections 2024-04-25T09:54:45.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.164+0000 7f65eb0dd640 1 --2- 172.21.15.139:0/4097991633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65e415dc60 0x7f65e40b6c10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.164+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4097991633 >> 172.21.15.139:0/4097991633 conn(0x7f65e40b5910 msgr2=0x7f65e40b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:45.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.164+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4097991633 shutdown_connections 2024-04-25T09:54:45.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.164+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4097991633 wait complete. 2024-04-25T09:54:45.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.165+0000 7f65eb0dd640 1 Processor -- start 2024-04-25T09:54:45.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.165+0000 7f65eb0dd640 1 -- start start 2024-04-25T09:54:45.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.166+0000 7f65eb0dd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e40d1080 0x7f65e4150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.166+0000 7f65eb0dd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65e415dc60 0x7f65e40b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.166+0000 7f65eb0dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65e4161850 0x7f65e40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.166+0000 7f65eb0dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e415e040 con 0x7f65e4161850 2024-04-25T09:54:45.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.166+0000 7f65eb0dd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40d1fc0 con 0x7f65e40d1080 2024-04-25T09:54:45.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.166+0000 7f65eb0dd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e415c660 con 0x7f65e415dc60 2024-04-25T09:54:45.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.166+0000 7f65e967a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65e4161850 0x7f65e40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.166+0000 7f65e967a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65e4161850 0x7f65e40b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58528/0 (socket says 172.21.15.139:58528) 2024-04-25T09:54:45.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.166+0000 7f65e967a640 1 -- 172.21.15.139:0/4067026555 learned_addr learned my addr 172.21.15.139:0/4067026555 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:45.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.166+0000 7f65e3fff640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65e415dc60 0x7f65e40b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.166+0000 7f65e8e79640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e40d1080 0x7f65e4150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.167+0000 7f65e967a640 1 -- 172.21.15.139:0/4067026555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65e415dc60 msgr2=0x7f65e40b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.167+0000 7f65e967a640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65e415dc60 0x7f65e40b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.167+0000 7f65e967a640 1 -- 172.21.15.139:0/4067026555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e40d1080 msgr2=0x7f65e4150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.167+0000 7f65e967a640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e40d1080 0x7f65e4150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.167+0000 7f65e967a640 1 -- 172.21.15.139:0/4067026555 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40c9bc0 con 0x7f65e4161850 2024-04-25T09:54:45.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.167+0000 7f65e967a640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65e4161850 0x7f65e40b4bb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c760 tx=0x7f65cc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.167+0000 7f65e17fa640 1 -- 172.21.15.139:0/4067026555 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc017070 con 0x7f65e4161850 2024-04-25T09:54:45.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.167+0000 7f65e17fa640 1 -- 172.21.15.139:0/4067026555 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc005ab0 con 0x7f65e4161850 2024-04-25T09:54:45.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.167+0000 7f65e17fa640 1 -- 172.21.15.139:0/4067026555 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc00ce10 con 0x7f65e4161850 2024-04-25T09:54:45.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.167+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4067026555 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e40c8640 con 0x7f65e4161850 2024-04-25T09:54:45.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.167+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4067026555 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e40cbf80 con 0x7f65e4161850 2024-04-25T09:54:45.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.168+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4067026555 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e4074520 con 0x7f65e4161850 2024-04-25T09:54:45.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.169+0000 7f65e17fa640 1 -- 172.21.15.139:0/4067026555 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65cc01a070 con 0x7f65e4161850 2024-04-25T09:54:45.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.169+0000 7f65e17fa640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f65c406b4d0 0x7f65c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.169+0000 7f65e17fa640 1 -- 172.21.15.139:0/4067026555 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65cc089490 con 0x7f65e4161850 2024-04-25T09:54:45.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.169+0000 7f65e8e79640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f65c406b4d0 0x7f65c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.169+0000 7f65e8e79640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f65c406b4d0 0x7f65c406d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f65dc034000 tx=0x7f65dc02f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.173+0000 7f65e17fa640 1 -- 172.21.15.139:0/4067026555 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65e40cf250 con 0x7f65e4161850 2024-04-25T09:54:45.365 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:54:45.368 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:54:45.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.383+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4067026555 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65e40d16f0 con 0x7f65e4161850 2024-04-25T09:54:45.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.384+0000 7f65e17fa640 1 -- 172.21.15.139:0/4067026555 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f65cc057b30 con 0x7f65e4161850 2024-04-25T09:54:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:45.387 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/511562594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":511562594},{"type":"v1","addr":"172.21.15.150:6834","nonce":511562594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5041,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/225357317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":225357317},{"type":"v1","addr":"172.21.15.150:6835","nonce":225357317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2499407982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2499407982},{"type":"v1","addr":"172.21.15.139:6836","nonce":2499407982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:21.988975+0000","modified":"2024-04-25T09:54:23.955620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/2069784668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2069784668},{"type":"v1","addr":"172.21.15.139:6837","nonce":2069784668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:54:45.387 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 16 2024-04-25T09:54:45.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.386+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4067026555 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f65c406b4d0 msgr2=0x7f65c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.386+0000 7f65eb0dd640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f65c406b4d0 0x7f65c406d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f65dc034000 tx=0x7f65dc02f000 comp rx=0 tx=0).stop 2024-04-25T09:54:45.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.386+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4067026555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65e4161850 msgr2=0x7f65e40b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.386+0000 7f65eb0dd640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65e4161850 0x7f65e40b4bb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c760 tx=0x7f65cc00cc20 comp rx=0 tx=0).stop 2024-04-25T09:54:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.386+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4067026555 shutdown_connections 2024-04-25T09:54:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.386+0000 7f65eb0dd640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65e4161850 0x7f65e40b4bb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.386+0000 7f65eb0dd640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f65c406b4d0 0x7f65c406d990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.386+0000 7f65eb0dd640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65e415dc60 0x7f65e40b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.386+0000 7f65eb0dd640 1 --2- 172.21.15.139:0/4067026555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65e40d1080 0x7f65e4150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.386+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4067026555 >> 172.21.15.139:0/4067026555 conn(0x7f65e40b5910 msgr2=0x7f65e40b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.386+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4067026555 shutdown_connections 2024-04-25T09:54:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.386+0000 7f65eb0dd640 1 -- 172.21.15.139:0/4067026555 wait complete. 2024-04-25T09:54:45.399 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.555+0000 7feb76b9e640 1 Processor -- start 2024-04-25T09:54:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.555+0000 7feb76b9e640 1 -- start start 2024-04-25T09:54:45.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb76b9e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feb70158260 0x7feb7015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb76b9e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feb7006da20 0x7feb700b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb76b9e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb700b3e10 con 0x7feb700b3600 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb76b9e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb700b3a50 con 0x7feb7006da20 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb76b9e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb701615c0 con 0x7feb70158260 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb6ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feb7006da20 0x7feb700b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb6ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feb7006da20 0x7feb700b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58538/0 (socket says 172.21.15.139:58538) 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb6ffff640 1 -- 172.21.15.139:0/263813248 learned_addr learned my addr 172.21.15.139:0/263813248 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb6ffff640 1 -- 172.21.15.139:0/263813248 >> v2:172.21.15.150:3300/0 conn(0x7feb70158260 msgr2=0x7feb7015df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb7493a640 1 --2- 172.21.15.139:0/263813248 >> v2:172.21.15.150:3300/0 conn(0x7feb70158260 0x7feb7015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb6ffff640 1 --2- 172.21.15.139:0/263813248 >> v2:172.21.15.150:3300/0 conn(0x7feb70158260 0x7feb7015df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb6ffff640 1 -- 172.21.15.139:0/263813248 >> v1:172.21.15.150:6789/0 conn(0x7feb700b3600 legacy=0x7feb7006def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.556+0000 7feb6ffff640 1 -- 172.21.15.139:0/263813248 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb701618a0 con 0x7feb7006da20 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.557+0000 7feb6ffff640 1 --2- 172.21.15.139:0/263813248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb7006da20 0x7feb700b30c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7feb7009e0b0 tx=0x7feb5c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23eacdd40933a36c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.557+0000 7feb6f7fe640 1 -- 172.21.15.139:0/263813248 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c037070 con 0x7feb7006da20 2024-04-25T09:54:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.557+0000 7feb6f7fe640 1 -- 172.21.15.139:0/263813248 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c032440 con 0x7feb7006da20 2024-04-25T09:54:45.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.557+0000 7feb6f7fe640 1 -- 172.21.15.139:0/263813248 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c032810 con 0x7feb7006da20 2024-04-25T09:54:45.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.558+0000 7feb76b9e640 1 -- 172.21.15.139:0/263813248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb7006da20 msgr2=0x7feb700b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.558+0000 7feb76b9e640 1 --2- 172.21.15.139:0/263813248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb7006da20 0x7feb700b30c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7feb7009e0b0 tx=0x7feb5c02a230 comp rx=0 tx=0).stop 2024-04-25T09:54:45.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.558+0000 7feb76b9e640 1 -- 172.21.15.139:0/263813248 shutdown_connections 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.558+0000 7feb76b9e640 1 --2- 172.21.15.139:0/263813248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb7006da20 0x7feb700b30c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.558+0000 7feb76b9e640 1 --2- 172.21.15.139:0/263813248 >> v2:172.21.15.150:3300/0 conn(0x7feb70158260 0x7feb7015df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.558+0000 7feb76b9e640 1 -- 172.21.15.139:0/263813248 >> 172.21.15.139:0/263813248 conn(0x7feb700b23d0 msgr2=0x7feb700cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.558+0000 7feb76b9e640 1 -- 172.21.15.139:0/263813248 shutdown_connections 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.558+0000 7feb76b9e640 1 -- 172.21.15.139:0/263813248 wait complete. 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.559+0000 7feb76b9e640 1 Processor -- start 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.559+0000 7feb76b9e640 1 -- start start 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb76b9e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb7006da20 0x7feb700ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb76b9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb700b3600 0x7feb700ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb76b9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb70158260 0x7feb70142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb76b9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb70162140 con 0x7feb70158260 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb76b9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb70161e60 con 0x7feb700b3600 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb76b9e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb70162420 con 0x7feb7006da20 2024-04-25T09:54:45.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb7513b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb70158260 0x7feb70142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb7513b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb70158260 0x7feb70142630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58548/0 (socket says 172.21.15.139:58548) 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb7513b640 1 -- 172.21.15.139:0/1451449195 learned_addr learned my addr 172.21.15.139:0/1451449195 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb6ffff640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb700b3600 0x7feb700ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb7493a640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb7006da20 0x7feb700ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb7513b640 1 -- 172.21.15.139:0/1451449195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb7006da20 msgr2=0x7feb700ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb7513b640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb7006da20 0x7feb700ace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb7513b640 1 -- 172.21.15.139:0/1451449195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb700b3600 msgr2=0x7feb700ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb7513b640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb700b3600 0x7feb700ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb7513b640 1 -- 172.21.15.139:0/1451449195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb700cc6b0 con 0x7feb70158260 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb7513b640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb70158260 0x7feb70142630 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feb6800c780 tx=0x7feb6800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb6d7fa640 1 -- 172.21.15.139:0/1451449195 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68004b60 con 0x7feb70158260 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb6d7fa640 1 -- 172.21.15.139:0/1451449195 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb68004d00 con 0x7feb70158260 2024-04-25T09:54:45.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.560+0000 7feb76b9e640 1 -- 172.21.15.139:0/1451449195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb700ccc50 con 0x7feb70158260 2024-04-25T09:54:45.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.561+0000 7feb6d7fa640 1 -- 172.21.15.139:0/1451449195 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6800aa90 con 0x7feb70158260 2024-04-25T09:54:45.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.561+0000 7feb76b9e640 1 -- 172.21.15.139:0/1451449195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb700ad840 con 0x7feb70158260 2024-04-25T09:54:45.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.562+0000 7feb76b9e640 1 -- 172.21.15.139:0/1451449195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb70074520 con 0x7feb70158260 2024-04-25T09:54:45.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.562+0000 7feb6d7fa640 1 -- 172.21.15.139:0/1451449195 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb6800ac30 con 0x7feb70158260 2024-04-25T09:54:45.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.562+0000 7feb6d7fa640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb5406b4d0 0x7feb5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.562+0000 7feb6d7fa640 1 -- 172.21.15.139:0/1451449195 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feb6808a480 con 0x7feb70158260 2024-04-25T09:54:45.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.562+0000 7feb7493a640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb5406b4d0 0x7feb5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.563+0000 7feb7493a640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb5406b4d0 0x7feb5406d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feb6400aa20 tx=0x7feb64008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.567+0000 7feb6d7fa640 1 -- 172.21.15.139:0/1451449195 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb68020070 con 0x7feb70158260 2024-04-25T09:54:45.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.741+0000 7feb76b9e640 1 -- 172.21.15.139:0/1451449195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb7006c9c0 con 0x7feb70158260 2024-04-25T09:54:45.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.742+0000 7feb6d7fa640 1 -- 172.21.15.139:0/1451449195 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16990 (secure 0 0 0) 0x7feb68058b20 con 0x7feb70158260 2024-04-25T09:54:45.745 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:45.745 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":38,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:23.620264+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:19.795203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:20.931184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:45.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.744+0000 7feb76b9e640 1 -- 172.21.15.139:0/1451449195 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb5406b4d0 msgr2=0x7feb5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.744+0000 7feb76b9e640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb5406b4d0 0x7feb5406d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feb6400aa20 tx=0x7feb64008040 comp rx=0 tx=0).stop 2024-04-25T09:54:45.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.744+0000 7feb76b9e640 1 -- 172.21.15.139:0/1451449195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb70158260 msgr2=0x7feb70142630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.744+0000 7feb76b9e640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb70158260 0x7feb70142630 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feb6800c780 tx=0x7feb6800cc40 comp rx=0 tx=0).stop 2024-04-25T09:54:45.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.744+0000 7feb76b9e640 1 -- 172.21.15.139:0/1451449195 shutdown_connections 2024-04-25T09:54:45.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.744+0000 7feb76b9e640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb70158260 0x7feb70142630 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.744+0000 7feb76b9e640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb5406b4d0 0x7feb5406d990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.744+0000 7feb76b9e640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb700b3600 0x7feb700ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.744+0000 7feb76b9e640 1 --2- 172.21.15.139:0/1451449195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb7006da20 0x7feb700ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.744+0000 7feb76b9e640 1 -- 172.21.15.139:0/1451449195 >> 172.21.15.139:0/1451449195 conn(0x7feb700b23d0 msgr2=0x7feb7015e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:45.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.745+0000 7feb76b9e640 1 -- 172.21.15.139:0/1451449195 shutdown_connections 2024-04-25T09:54:45.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.745+0000 7feb76b9e640 1 -- 172.21.15.139:0/1451449195 wait complete. 2024-04-25T09:54:45.758 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:54:45.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.853+0000 7f1c61534640 1 Processor -- start 2024-04-25T09:54:45.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.853+0000 7f1c61534640 1 -- start start 2024-04-25T09:54:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.853+0000 7f1c61534640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1c5c1617e0 0x7f1c5c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.854+0000 7f1c61534640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c161dc0 con 0x7f1c5c0cec80 2024-04-25T09:54:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.854+0000 7f1c61534640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c1620a0 con 0x7f1c5c15dbf0 2024-04-25T09:54:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.854+0000 7f1c61534640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c0cf0d0 con 0x7f1c5c1617e0 2024-04-25T09:54:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.854+0000 7f1c5a7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1c5c1617e0 0x7f1c5c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.854+0000 7f1c5a7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1c5c1617e0 0x7f1c5c0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58554/0 (socket says 172.21.15.139:58554) 2024-04-25T09:54:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.854+0000 7f1c5b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1c5c0cec80 0x7f1c5c0d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54618/0 (socket says 172.21.15.139:54618) 2024-04-25T09:54:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.854+0000 7f1c5a7fc640 1 -- 172.21.15.139:0/2936809670 learned_addr learned my addr 172.21.15.139:0/2936809670 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.854+0000 7f1c5a7fc640 1 -- 172.21.15.139:0/2936809670 >> v1:172.21.15.150:6789/0 conn(0x7f1c5c15dbf0 legacy=0x7f1c5c0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.854+0000 7f1c5a7fc640 1 -- 172.21.15.139:0/2936809670 >> v1:172.21.15.139:6789/0 conn(0x7f1c5c0cec80 legacy=0x7f1c5c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.854+0000 7f1c5a7fc640 1 -- 172.21.15.139:0/2936809670 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c162380 con 0x7f1c5c1617e0 2024-04-25T09:54:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.855+0000 7f1c5a7fc640 1 --2- 172.21.15.139:0/2936809670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c1617e0 0x7f1c5c0ce740 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009870 tx=0x7f1c5002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8541798f06ca96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.855+0000 7f1c59ffb640 1 -- 172.21.15.139:0/2936809670 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50037070 con 0x7f1c5c1617e0 2024-04-25T09:54:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.855+0000 7f1c59ffb640 1 -- 172.21.15.139:0/2936809670 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c50032440 con 0x7f1c5c1617e0 2024-04-25T09:54:45.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.855+0000 7f1c59ffb640 1 -- 172.21.15.139:0/2936809670 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50032810 con 0x7f1c5c1617e0 2024-04-25T09:54:45.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.855+0000 7f1c61534640 1 -- 172.21.15.139:0/2936809670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c1617e0 msgr2=0x7f1c5c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.855+0000 7f1c61534640 1 --2- 172.21.15.139:0/2936809670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c1617e0 0x7f1c5c0ce740 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009870 tx=0x7f1c5002a210 comp rx=0 tx=0).stop 2024-04-25T09:54:45.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.855+0000 7f1c61534640 1 -- 172.21.15.139:0/2936809670 shutdown_connections 2024-04-25T09:54:45.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.855+0000 7f1c61534640 1 --2- 172.21.15.139:0/2936809670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c1617e0 0x7f1c5c0ce740 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.855+0000 7f1c61534640 1 -- 172.21.15.139:0/2936809670 >> 172.21.15.139:0/2936809670 conn(0x7f1c5c0b5480 msgr2=0x7f1c5c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:45.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.855+0000 7f1c61534640 1 -- 172.21.15.139:0/2936809670 shutdown_connections 2024-04-25T09:54:45.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.855+0000 7f1c61534640 1 -- 172.21.15.139:0/2936809670 wait complete. 2024-04-25T09:54:45.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.857+0000 7f1c61534640 1 Processor -- start 2024-04-25T09:54:45.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.857+0000 7f1c61534640 1 -- start start 2024-04-25T09:54:45.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.857+0000 7f1c61534640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c5c0cec80 0x7f1c5c155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.857+0000 7f1c61534640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c5c15dbf0 0x7f1c5c15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.857+0000 7f1c61534640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c1617e0 0x7f1c5c1519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.857+0000 7f1c61534640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c15dfd0 con 0x7f1c5c1617e0 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.857+0000 7f1c61534640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0d1ee0 con 0x7f1c5c0cec80 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.857+0000 7f1c61534640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c15f3f0 con 0x7f1c5c15dbf0 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.857+0000 7f1c5b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c1617e0 0x7f1c5c1519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.857+0000 7f1c5b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c1617e0 0x7f1c5c1519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58562/0 (socket says 172.21.15.139:58562) 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.857+0000 7f1c5b7fe640 1 -- 172.21.15.139:0/1270793590 learned_addr learned my addr 172.21.15.139:0/1270793590 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c5affd640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c5c0cec80 0x7f1c5c155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c5a7fc640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c5c15dbf0 0x7f1c5c15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c5b7fe640 1 -- 172.21.15.139:0/1270793590 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c5c15dbf0 msgr2=0x7f1c5c15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c5b7fe640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c5c15dbf0 0x7f1c5c15ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c5b7fe640 1 -- 172.21.15.139:0/1270793590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c5c0cec80 msgr2=0x7f1c5c155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c5b7fe640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c5c0cec80 0x7f1c5c155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c5b7fe640 1 -- 172.21.15.139:0/1270793590 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c15f6d0 con 0x7f1c5c1617e0 2024-04-25T09:54:45.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c5b7fe640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c1617e0 0x7f1c5c1519a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400c780 tx=0x7f1c5400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c437fe640 1 -- 172.21.15.139:0/1270793590 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54017070 con 0x7f1c5c1617e0 2024-04-25T09:54:45.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c437fe640 1 -- 172.21.15.139:0/1270793590 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5400a640 con 0x7f1c5c1617e0 2024-04-25T09:54:45.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c61534640 1 -- 172.21.15.139:0/1270793590 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c15f9b0 con 0x7f1c5c1617e0 2024-04-25T09:54:45.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c437fe640 1 -- 172.21.15.139:0/1270793590 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400aa30 con 0x7f1c5c1617e0 2024-04-25T09:54:45.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.858+0000 7f1c61534640 1 -- 172.21.15.139:0/1270793590 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c15ff70 con 0x7f1c5c1617e0 2024-04-25T09:54:45.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.859+0000 7f1c61534640 1 -- 172.21.15.139:0/1270793590 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c074520 con 0x7f1c5c1617e0 2024-04-25T09:54:45.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.859+0000 7f1c437fe640 1 -- 172.21.15.139:0/1270793590 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c5401a070 con 0x7f1c5c1617e0 2024-04-25T09:54:45.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.860+0000 7f1c437fe640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1c4806b4d0 0x7f1c4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.860+0000 7f1c5affd640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1c4806b4d0 0x7f1c4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.860+0000 7f1c437fe640 1 -- 172.21.15.139:0/1270793590 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c540898c0 con 0x7f1c5c1617e0 2024-04-25T09:54:45.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.860+0000 7f1c5affd640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1c4806b4d0 0x7f1c4806d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c004ea0 tx=0x7f1c4c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:45.864+0000 7f1c437fe640 1 -- 172.21.15.139:0/1270793590 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c54057f60 con 0x7f1c5c1617e0 2024-04-25T09:54:46.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.071+0000 7f1c61534640 1 -- 172.21.15.139:0/1270793590 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1c5c0b6710 con 0x7f1c5c1617e0 2024-04-25T09:54:46.074 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T09:54:46.072+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:54:46.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.073+0000 7f1c437fe640 1 -- 172.21.15.139:0/1270793590 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1c5405c000 con 0x7f1c5c1617e0 2024-04-25T09:54:46.075 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:54:46.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.076+0000 7f1c61534640 1 -- 172.21.15.139:0/1270793590 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1c4806b4d0 msgr2=0x7f1c4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.076+0000 7f1c61534640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1c4806b4d0 0x7f1c4806d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c004ea0 tx=0x7f1c4c009040 comp rx=0 tx=0).stop 2024-04-25T09:54:46.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.076+0000 7f1c61534640 1 -- 172.21.15.139:0/1270793590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c1617e0 msgr2=0x7f1c5c1519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.076+0000 7f1c61534640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c1617e0 0x7f1c5c1519a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400c780 tx=0x7f1c5400cc40 comp rx=0 tx=0).stop 2024-04-25T09:54:46.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.076+0000 7f1c61534640 1 -- 172.21.15.139:0/1270793590 shutdown_connections 2024-04-25T09:54:46.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.076+0000 7f1c61534640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c5c1617e0 0x7f1c5c1519a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.076+0000 7f1c61534640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1c4806b4d0 0x7f1c4806d990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.076+0000 7f1c61534640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c5c15dbf0 0x7f1c5c15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.076+0000 7f1c61534640 1 --2- 172.21.15.139:0/1270793590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c5c0cec80 0x7f1c5c155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.076+0000 7f1c61534640 1 -- 172.21.15.139:0/1270793590 >> 172.21.15.139:0/1270793590 conn(0x7f1c5c0b5480 msgr2=0x7f1c5c0b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:46.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.076+0000 7f1c61534640 1 -- 172.21.15.139:0/1270793590 shutdown_connections 2024-04-25T09:54:46.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.076+0000 7f1c61534640 1 -- 172.21.15.139:0/1270793590 wait complete. 2024-04-25T09:54:46.081 INFO:tasks.ceph.mds.a.smithi139.stderr: -6361> 2024-04-25T09:54:27.045+0000 7effc2d26640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:54:46.090 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:54:46.112 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T09:54:46.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.188+0000 7f901cc53640 1 Processor -- start 2024-04-25T09:54:46.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.188+0000 7f901cc53640 1 -- start start 2024-04-25T09:54:46.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.189+0000 7f901cc53640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f901815b560 0x7f901815b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.189+0000 7f901cc53640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90180d03f0 con 0x7f901815fca0 2024-04-25T09:54:46.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.189+0000 7f901cc53640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9018099bf0 con 0x7f901815f8c0 2024-04-25T09:54:46.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.189+0000 7f901cc53640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90180d06c0 con 0x7f901815b560 2024-04-25T09:54:46.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.189+0000 7f9016d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f901815b560 0x7f901815b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.189+0000 7f9015d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f901815fca0 0x7f90180cfce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54636/0 (socket says 172.21.15.139:54636) 2024-04-25T09:54:46.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.189+0000 7f9015d9b640 1 -- 172.21.15.139:0/1807948261 learned_addr learned my addr 172.21.15.139:0/1807948261 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:46.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.189+0000 7f9016d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f901815b560 0x7f901815b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58598/0 (socket says 172.21.15.139:58598) 2024-04-25T09:54:46.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.189+0000 7f9016d9d640 1 -- 172.21.15.139:0/1807948261 >> v1:172.21.15.150:6789/0 conn(0x7f901815f8c0 legacy=0x7f901809a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.189+0000 7f9016d9d640 1 -- 172.21.15.139:0/1807948261 >> v1:172.21.15.139:6789/0 conn(0x7f901815fca0 legacy=0x7f90180cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.189+0000 7f9016d9d640 1 -- 172.21.15.139:0/1807948261 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9018099ed0 con 0x7f901815b560 2024-04-25T09:54:46.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.189+0000 7f901559a640 1 -- 172.21.15.139:0/1807948261 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753483880 0 0) 0x7f90180d03f0 con 0x7f901815fca0 2024-04-25T09:54:46.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.190+0000 7f9016d9d640 1 --2- 172.21.15.139:0/1807948261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f901815b560 0x7f901815b990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9010009d90 tx=0x7f901002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a55b228b6fb7bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:46.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.190+0000 7f901559a640 1 -- 172.21.15.139:0/1807948261 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010007740 con 0x7f901815b560 2024-04-25T09:54:46.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.190+0000 7f901559a640 1 -- 172.21.15.139:0/1807948261 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90100078e0 con 0x7f901815b560 2024-04-25T09:54:46.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.190+0000 7f901559a640 1 -- 172.21.15.139:0/1807948261 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010007cb0 con 0x7f901815b560 2024-04-25T09:54:46.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.191+0000 7f901cc53640 1 -- 172.21.15.139:0/1807948261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f901815b560 msgr2=0x7f901815b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.191+0000 7f901cc53640 1 --2- 172.21.15.139:0/1807948261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f901815b560 0x7f901815b990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9010009d90 tx=0x7f901002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.191+0000 7f901cc53640 1 -- 172.21.15.139:0/1807948261 shutdown_connections 2024-04-25T09:54:46.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.191+0000 7f901cc53640 1 --2- 172.21.15.139:0/1807948261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f901815b560 0x7f901815b990 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.191+0000 7f901cc53640 1 -- 172.21.15.139:0/1807948261 >> 172.21.15.139:0/1807948261 conn(0x7f90180b31f0 msgr2=0x7f90180b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:46.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.191+0000 7f901cc53640 1 -- 172.21.15.139:0/1807948261 shutdown_connections 2024-04-25T09:54:46.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.191+0000 7f901cc53640 1 -- 172.21.15.139:0/1807948261 wait complete. 2024-04-25T09:54:46.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.192+0000 7f901cc53640 1 Processor -- start 2024-04-25T09:54:46.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.192+0000 7f901cc53640 1 -- start start 2024-04-25T09:54:46.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f901cc53640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f901815b560 0x7f90181f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f901cc53640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f901815f8c0 0x7f90180c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f901cc53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f901815fca0 0x7f901815aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f901cc53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901815bed0 con 0x7f901815fca0 2024-04-25T09:54:46.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f901cc53640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901809a490 con 0x7f901815b560 2024-04-25T09:54:46.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f901cc53640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90181602d0 con 0x7f901815f8c0 2024-04-25T09:54:46.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9016d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f901815fca0 0x7f901815aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9016d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f901815fca0 0x7f901815aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58614/0 (socket says 172.21.15.139:58614) 2024-04-25T09:54:46.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9016d9d640 1 -- 172.21.15.139:0/2046973859 learned_addr learned my addr 172.21.15.139:0/2046973859 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:46.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9015d9b640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f901815f8c0 0x7f90180c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f901659c640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f901815b560 0x7f90181f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9016d9d640 1 -- 172.21.15.139:0/2046973859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f901815f8c0 msgr2=0x7f90180c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9016d9d640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f901815f8c0 0x7f90180c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9016d9d640 1 -- 172.21.15.139:0/2046973859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f901815b560 msgr2=0x7f90181f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9016d9d640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f901815b560 0x7f90181f4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9016d9d640 1 -- 172.21.15.139:0/2046973859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90180caf80 con 0x7f901815fca0 2024-04-25T09:54:46.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9016d9d640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f901815fca0 0x7f901815aa90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9010009eb0 tx=0x7f9010004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:46.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9006ffd640 1 -- 172.21.15.139:0/2046973859 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010040070 con 0x7f901815fca0 2024-04-25T09:54:46.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9006ffd640 1 -- 172.21.15.139:0/2046973859 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9010031c40 con 0x7f901815fca0 2024-04-25T09:54:46.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f901cc53640 1 -- 172.21.15.139:0/2046973859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90180c9ee0 con 0x7f901815fca0 2024-04-25T09:54:46.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f9006ffd640 1 -- 172.21.15.139:0/2046973859 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010003990 con 0x7f901815fca0 2024-04-25T09:54:46.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.193+0000 7f901cc53640 1 -- 172.21.15.139:0/2046973859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90180ca4a0 con 0x7f901815fca0 2024-04-25T09:54:46.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.194+0000 7f901cc53640 1 -- 172.21.15.139:0/2046973859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9018074520 con 0x7f901815fca0 2024-04-25T09:54:46.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.195+0000 7f9006ffd640 1 -- 172.21.15.139:0/2046973859 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9010039040 con 0x7f901815fca0 2024-04-25T09:54:46.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.195+0000 7f9006ffd640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ff806b4d0 0x7f8ff806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.195+0000 7f9006ffd640 1 -- 172.21.15.139:0/2046973859 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f90100ab740 con 0x7f901815fca0 2024-04-25T09:54:46.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.195+0000 7f901659c640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ff806b4d0 0x7f8ff806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.196+0000 7f901659c640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ff806b4d0 0x7f8ff806d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9008000f60 tx=0x7f9008009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:46.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.199+0000 7f9006ffd640 1 -- 172.21.15.139:0/2046973859 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f901007a3e0 con 0x7f901815fca0 2024-04-25T09:54:46.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:46.406+0000 7f901cc53640 1 -- 172.21.15.139:0/2046973859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9018067dc0 con 0x7f901815fca0 2024-04-25T09:54:47.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.075+0000 7f9006ffd640 1 -- 172.21.15.139:0/2046973859 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9010079ce0 con 0x7f901815fca0 2024-04-25T09:54:47.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.077+0000 7f901cc53640 1 -- 172.21.15.139:0/2046973859 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ff806b4d0 msgr2=0x7f8ff806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.077+0000 7f901cc53640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ff806b4d0 0x7f8ff806d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9008000f60 tx=0x7f9008009040 comp rx=0 tx=0).stop 2024-04-25T09:54:47.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.077+0000 7f901cc53640 1 -- 172.21.15.139:0/2046973859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f901815fca0 msgr2=0x7f901815aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.077+0000 7f901cc53640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f901815fca0 0x7f901815aa90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9010009eb0 tx=0x7f9010004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:47.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.077+0000 7f901cc53640 1 -- 172.21.15.139:0/2046973859 shutdown_connections 2024-04-25T09:54:47.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.077+0000 7f901cc53640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f901815fca0 0x7f901815aa90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.077+0000 7f901cc53640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ff806b4d0 0x7f8ff806d990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.077+0000 7f901cc53640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f901815f8c0 0x7f90180c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.077+0000 7f901cc53640 1 --2- 172.21.15.139:0/2046973859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f901815b560 0x7f90181f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.077+0000 7f901cc53640 1 -- 172.21.15.139:0/2046973859 >> 172.21.15.139:0/2046973859 conn(0x7f90180b31f0 msgr2=0x7f90180cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.078+0000 7f901cc53640 1 -- 172.21.15.139:0/2046973859 shutdown_connections 2024-04-25T09:54:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.078+0000 7f901cc53640 1 -- 172.21.15.139:0/2046973859 wait complete. 2024-04-25T09:54:47.094 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:54:47.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.240+0000 7f966e153640 1 Processor -- start 2024-04-25T09:54:47.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.240+0000 7f966e153640 1 -- start start 2024-04-25T09:54:47.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f966e153640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f96680cd610 0x7f96680d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f966e153640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f966809ad40 0x7f96680d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f966e153640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9668099ca0 con 0x7f96681618c0 2024-04-25T09:54:47.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f966e153640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9668099a50 con 0x7f96680cd610 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f966e153640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9668099f80 con 0x7f966809ad40 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f96677fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f96680cd610 0x7f96680d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f96677fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f96680cd610 0x7f96680d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34030/0 (socket says 172.21.15.139:34030) 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f96677fe640 1 -- 172.21.15.139:0/2268026099 learned_addr learned my addr 172.21.15.139:0/2268026099 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f9666ffd640 1 --2- 172.21.15.139:0/2268026099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f966809ad40 0x7f96680d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f96677fe640 1 -- 172.21.15.139:0/2268026099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f966809ad40 msgr2=0x7f96680d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f96677fe640 1 --2- 172.21.15.139:0/2268026099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f966809ad40 0x7f96680d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f96677fe640 1 -- 172.21.15.139:0/2268026099 >> v1:172.21.15.150:6789/0 conn(0x7f96681618c0 legacy=0x7f966806be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.241+0000 7f96677fe640 1 -- 172.21.15.139:0/2268026099 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966809a260 con 0x7f96680cd610 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.242+0000 7f96677fe640 1 --2- 172.21.15.139:0/2268026099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96680cd610 0x7f96680d0370 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9658009870 tx=0x7f965802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18f560a0c78afcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.242+0000 7f96667fc640 1 -- 172.21.15.139:0/2268026099 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658037070 con 0x7f96680cd610 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.242+0000 7f96667fc640 1 -- 172.21.15.139:0/2268026099 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9658032440 con 0x7f96680cd610 2024-04-25T09:54:47.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.242+0000 7f96667fc640 1 -- 172.21.15.139:0/2268026099 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658032810 con 0x7f96680cd610 2024-04-25T09:54:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.243+0000 7f966e153640 1 -- 172.21.15.139:0/2268026099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96680cd610 msgr2=0x7f96680d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.243+0000 7f966e153640 1 --2- 172.21.15.139:0/2268026099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96680cd610 0x7f96680d0370 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9658009870 tx=0x7f965802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.243+0000 7f966e153640 1 -- 172.21.15.139:0/2268026099 shutdown_connections 2024-04-25T09:54:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.243+0000 7f966e153640 1 --2- 172.21.15.139:0/2268026099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f966809ad40 0x7f96680d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.243+0000 7f966e153640 1 --2- 172.21.15.139:0/2268026099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96680cd610 0x7f96680d0370 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.243+0000 7f966e153640 1 -- 172.21.15.139:0/2268026099 >> 172.21.15.139:0/2268026099 conn(0x7f96680b31f0 msgr2=0x7f96680b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.243+0000 7f966e153640 1 -- 172.21.15.139:0/2268026099 shutdown_connections 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.243+0000 7f966e153640 1 -- 172.21.15.139:0/2268026099 wait complete. 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.244+0000 7f966e153640 1 Processor -- start 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.244+0000 7f966e153640 1 -- start start 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f966e153640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966809ad40 0x7f9668156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f966e153640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96680cd610 0x7f96681582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f966e153640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96681618c0 0x7f96681f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f966e153640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96680cd9f0 con 0x7f96680cd610 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f966e153640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966809a820 con 0x7f966809ad40 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f966e153640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96680cdb70 con 0x7f96681618c0 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f9666ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96680cd610 0x7f96681582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f9666ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96680cd610 0x7f96681582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34034/0 (socket says 172.21.15.139:34034) 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f9666ffd640 1 -- 172.21.15.139:0/4182352177 learned_addr learned my addr 172.21.15.139:0/4182352177 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f9667fff640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96681618c0 0x7f96681f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f96677fe640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966809ad40 0x7f9668156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f9666ffd640 1 -- 172.21.15.139:0/4182352177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96681618c0 msgr2=0x7f96681f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f9666ffd640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96681618c0 0x7f96681f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f9666ffd640 1 -- 172.21.15.139:0/4182352177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966809ad40 msgr2=0x7f9668156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f9666ffd640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966809ad40 0x7f9668156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f9666ffd640 1 -- 172.21.15.139:0/4182352177 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9668157240 con 0x7f96680cd610 2024-04-25T09:54:47.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f9666ffd640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96680cd610 0x7f96681582e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f965c00d800 tx=0x7f965c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f964ffff640 1 -- 172.21.15.139:0/4182352177 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c00b6c0 con 0x7f96680cd610 2024-04-25T09:54:47.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f964ffff640 1 -- 172.21.15.139:0/4182352177 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c019070 con 0x7f96680cd610 2024-04-25T09:54:47.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f966e153640 1 -- 172.21.15.139:0/4182352177 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9668157520 con 0x7f96680cd610 2024-04-25T09:54:47.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f964ffff640 1 -- 172.21.15.139:0/4182352177 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c00ba80 con 0x7f96680cd610 2024-04-25T09:54:47.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.245+0000 7f966e153640 1 -- 172.21.15.139:0/4182352177 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9668157ae0 con 0x7f96680cd610 2024-04-25T09:54:47.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.246+0000 7f966e153640 1 -- 172.21.15.139:0/4182352177 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9668074520 con 0x7f96680cd610 2024-04-25T09:54:47.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.247+0000 7f964ffff640 1 -- 172.21.15.139:0/4182352177 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f965c00bc20 con 0x7f96680cd610 2024-04-25T09:54:47.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.247+0000 7f964ffff640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.247+0000 7f964ffff640 1 -- 172.21.15.139:0/4182352177 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f965c08a6b0 con 0x7f96680cd610 2024-04-25T09:54:47.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.247+0000 7f96677fe640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.248+0000 7f96677fe640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f965406b4d0 0x7f965406d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9658034000 tx=0x7f965802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.251+0000 7f964ffff640 1 -- 172.21.15.139:0/4182352177 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f965c058d90 con 0x7f96680cd610 2024-04-25T09:54:47.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:47.423+0000 7f966e153640 1 -- 172.21.15.139:0/4182352177 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f96680b2b50 con 0x7f96680cd610 2024-04-25T09:54:48.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.076+0000 7f964ffff640 1 -- 172.21.15.139:0/4182352177 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) v1 ==== 181+0+0 (secure 0 0 0) 0x7f965c088e20 con 0x7f96680cd610 2024-04-25T09:54:48.079 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:54:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.078+0000 7f966e153640 1 -- 172.21.15.139:0/4182352177 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f965406b4d0 msgr2=0x7f965406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.078+0000 7f966e153640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f965406b4d0 0x7f965406d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9658034000 tx=0x7f965802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.079+0000 7f966e153640 1 -- 172.21.15.139:0/4182352177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96680cd610 msgr2=0x7f96681582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.079+0000 7f966e153640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96680cd610 0x7f96681582e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f965c00d800 tx=0x7f965c00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.079+0000 7f966e153640 1 -- 172.21.15.139:0/4182352177 shutdown_connections 2024-04-25T09:54:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.079+0000 7f966e153640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96680cd610 0x7f96681582e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.079+0000 7f966e153640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.079+0000 7f966e153640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96681618c0 0x7f96681f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.079+0000 7f966e153640 1 --2- 172.21.15.139:0/4182352177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f966809ad40 0x7f9668156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.079+0000 7f966e153640 1 -- 172.21.15.139:0/4182352177 >> 172.21.15.139:0/4182352177 conn(0x7f96680b31f0 msgr2=0x7f96680b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.079+0000 7f966e153640 1 -- 172.21.15.139:0/4182352177 shutdown_connections 2024-04-25T09:54:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.079+0000 7f966e153640 1 -- 172.21.15.139:0/4182352177 wait complete. 2024-04-25T09:54:48.095 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:54:48.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.197+0000 7fdf8e0f7640 1 Processor -- start 2024-04-25T09:54:48.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.197+0000 7fdf8e0f7640 1 -- start start 2024-04-25T09:54:48.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.197+0000 7fdf8e0f7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdf8815dc90 0x7fdf880ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:48.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.198+0000 7fdf8e0f7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf880cf980 con 0x7fdf8806f300 2024-04-25T09:54:48.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.198+0000 7fdf8e0f7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf8806ab60 con 0x7fdf880cf3e0 2024-04-25T09:54:48.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.198+0000 7fdf8e0f7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf8806ae40 con 0x7fdf8815dc90 2024-04-25T09:54:48.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.198+0000 7fdf877fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdf8815dc90 0x7fdf880ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:48.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.198+0000 7fdf87fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdf8806f300 0x7fdf880b2db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60298/0 (socket says 172.21.15.139:60298) 2024-04-25T09:54:48.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.198+0000 7fdf877fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdf8815dc90 0x7fdf880ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34048/0 (socket says 172.21.15.139:34048) 2024-04-25T09:54:48.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.198+0000 7fdf87fff640 1 -- 172.21.15.139:0/2660531804 learned_addr learned my addr 172.21.15.139:0/2660531804 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:48.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.198+0000 7fdf877fe640 1 -- 172.21.15.139:0/2660531804 >> v1:172.21.15.150:6789/0 conn(0x7fdf880cf3e0 legacy=0x7fdf8806ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:48.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.198+0000 7fdf877fe640 1 -- 172.21.15.139:0/2660531804 >> v1:172.21.15.139:6789/0 conn(0x7fdf8806f300 legacy=0x7fdf880b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.198+0000 7fdf877fe640 1 -- 172.21.15.139:0/2660531804 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8806b120 con 0x7fdf8815dc90 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.199+0000 7fdf877fe640 1 --2- 172.21.15.139:0/2660531804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf8815dc90 0x7fdf880ceea0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdf780097a0 tx=0x7fdf7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c40e3269f0065f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.199+0000 7fdf867fc640 1 -- 172.21.15.139:0/2660531804 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78037070 con 0x7fdf8815dc90 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.199+0000 7fdf867fc640 1 -- 172.21.15.139:0/2660531804 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf78032440 con 0x7fdf8815dc90 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.199+0000 7fdf867fc640 1 -- 172.21.15.139:0/2660531804 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78032810 con 0x7fdf8815dc90 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.199+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/2660531804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf8815dc90 msgr2=0x7fdf880ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.199+0000 7fdf8e0f7640 1 --2- 172.21.15.139:0/2660531804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf8815dc90 0x7fdf880ceea0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdf780097a0 tx=0x7fdf7802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.199+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/2660531804 shutdown_connections 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.199+0000 7fdf8e0f7640 1 --2- 172.21.15.139:0/2660531804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf8815dc90 0x7fdf880ceea0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.199+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/2660531804 >> 172.21.15.139:0/2660531804 conn(0x7fdf880cfd80 msgr2=0x7fdf880d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.199+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/2660531804 shutdown_connections 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.199+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/2660531804 wait complete. 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.200+0000 7fdf8e0f7640 1 Processor -- start 2024-04-25T09:54:48.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.201+0000 7fdf8e0f7640 1 -- start start 2024-04-25T09:54:48.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.201+0000 7fdf8e0f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf8806f300 0x7fdf8814a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:48.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.201+0000 7fdf8e0f7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf880cf3e0 0x7fdf88156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:48.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.201+0000 7fdf8e0f7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8815dc90 0x7fdf881605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:48.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.201+0000 7fdf8e0f7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8806b920 con 0x7fdf8806f300 2024-04-25T09:54:48.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.201+0000 7fdf8e0f7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf88072620 con 0x7fdf880cf3e0 2024-04-25T09:54:48.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.201+0000 7fdf8e0f7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf880b34c0 con 0x7fdf8815dc90 2024-04-25T09:54:48.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.201+0000 7fdf877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf8806f300 0x7fdf8814a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:48.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.201+0000 7fdf877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf8806f300 0x7fdf8814a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34060/0 (socket says 172.21.15.139:34060) 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.201+0000 7fdf877fe640 1 -- 172.21.15.139:0/1417112537 learned_addr learned my addr 172.21.15.139:0/1417112537 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.202+0000 7fdf877fe640 1 -- 172.21.15.139:0/1417112537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8815dc90 msgr2=0x7fdf881605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.202+0000 7fdf877fe640 1 --2- 172.21.15.139:0/1417112537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8815dc90 0x7fdf881605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.202+0000 7fdf877fe640 1 -- 172.21.15.139:0/1417112537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf880cf3e0 msgr2=0x7fdf88156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.202+0000 7fdf877fe640 1 --2- 172.21.15.139:0/1417112537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf880cf3e0 0x7fdf88156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.202+0000 7fdf877fe640 1 -- 172.21.15.139:0/1417112537 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf88156cf0 con 0x7fdf8806f300 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.202+0000 7fdf877fe640 1 --2- 172.21.15.139:0/1417112537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf8806f300 0x7fdf8814a610 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdf780098c0 tx=0x7fdf780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.202+0000 7fdf6ffff640 1 -- 172.21.15.139:0/1417112537 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78037040 con 0x7fdf8806f300 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.202+0000 7fdf6ffff640 1 -- 172.21.15.139:0/1417112537 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf78042070 con 0x7fdf8806f300 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.202+0000 7fdf6ffff640 1 -- 172.21.15.139:0/1417112537 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78030a70 con 0x7fdf8806f300 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.202+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/1417112537 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf88156fd0 con 0x7fdf8806f300 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.202+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/1417112537 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf88157590 con 0x7fdf8806f300 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.203+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/1417112537 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf88074520 con 0x7fdf8806f300 2024-04-25T09:54:48.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.203+0000 7fdf6ffff640 1 -- 172.21.15.139:0/1417112537 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf78030c10 con 0x7fdf8806f300 2024-04-25T09:54:48.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.204+0000 7fdf6ffff640 1 --2- 172.21.15.139:0/1417112537 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf7406b4d0 0x7fdf7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:48.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.204+0000 7fdf6ffff640 1 -- 172.21.15.139:0/1417112537 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fdf780aacc0 con 0x7fdf8806f300 2024-04-25T09:54:48.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.204+0000 7fdf86ffd640 1 --2- 172.21.15.139:0/1417112537 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf7406b4d0 0x7fdf7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:48.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.204+0000 7fdf86ffd640 1 --2- 172.21.15.139:0/1417112537 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf7406b4d0 0x7fdf7406d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdf880ce7b0 tx=0x7fdf7c0079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:48.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.208+0000 7fdf6ffff640 1 -- 172.21.15.139:0/1417112537 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf78079b10 con 0x7fdf8806f300 2024-04-25T09:54:48.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:48.388+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/1417112537 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdf8815e170 con 0x7fdf8806f300 2024-04-25T09:54:49.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.078+0000 7fdf6ffff640 1 -- 172.21.15.139:0/1417112537 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdf78079410 con 0x7fdf8806f300 2024-04-25T09:54:49.081 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' removed 2024-04-25T09:54:49.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.082+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/1417112537 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf7406b4d0 msgr2=0x7fdf7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.082+0000 7fdf8e0f7640 1 --2- 172.21.15.139:0/1417112537 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf7406b4d0 0x7fdf7406d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdf880ce7b0 tx=0x7fdf7c0079b0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.082+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/1417112537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf8806f300 msgr2=0x7fdf8814a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.082+0000 7fdf8e0f7640 1 --2- 172.21.15.139:0/1417112537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf8806f300 0x7fdf8814a610 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdf780098c0 tx=0x7fdf780046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.083+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/1417112537 shutdown_connections 2024-04-25T09:54:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.083+0000 7fdf8e0f7640 1 --2- 172.21.15.139:0/1417112537 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf7406b4d0 0x7fdf7406d990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.083+0000 7fdf8e0f7640 1 --2- 172.21.15.139:0/1417112537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8815dc90 0x7fdf881605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.083+0000 7fdf8e0f7640 1 --2- 172.21.15.139:0/1417112537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf880cf3e0 0x7fdf88156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.083+0000 7fdf8e0f7640 1 --2- 172.21.15.139:0/1417112537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf8806f300 0x7fdf8814a610 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.083+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/1417112537 >> 172.21.15.139:0/1417112537 conn(0x7fdf880cfd80 msgr2=0x7fdf880cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.083+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/1417112537 shutdown_connections 2024-04-25T09:54:49.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.083+0000 7fdf8e0f7640 1 -- 172.21.15.139:0/1417112537 wait complete. 2024-04-25T09:54:49.101 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:54:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.205+0000 7f045bae1640 1 Processor -- start 2024-04-25T09:54:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.205+0000 7f045bae1640 1 -- start start 2024-04-25T09:54:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.206+0000 7f045bae1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04540d0a80 0x7f04541618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.206+0000 7f045bae1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f045409ad40 0x7f0454161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.206+0000 7f045bae1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0454099ca0 con 0x7f04540cd610 2024-04-25T09:54:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.206+0000 7f045bae1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0454099a50 con 0x7f045409ad40 2024-04-25T09:54:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.206+0000 7f045bae1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0454099f80 con 0x7f04540d0a80 2024-04-25T09:54:49.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.206+0000 7f045907c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04540d0a80 0x7f04541618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.206+0000 7f045a07e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f045409ad40 0x7f0454161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.206+0000 7f045907c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04540d0a80 0x7f04541618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:49872/0 (socket says 172.21.15.139:49872) 2024-04-25T09:54:49.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.206+0000 7f045907c640 1 -- 172.21.15.139:0/836413922 learned_addr learned my addr 172.21.15.139:0/836413922 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:49.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.207+0000 7f045907c640 1 -- 172.21.15.139:0/836413922 >> v2:172.21.15.150:3300/0 conn(0x7f045409ad40 msgr2=0x7f0454161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.207+0000 7f045907c640 1 --2- 172.21.15.139:0/836413922 >> v2:172.21.15.150:3300/0 conn(0x7f045409ad40 0x7f0454161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.207+0000 7f045907c640 1 -- 172.21.15.139:0/836413922 >> v1:172.21.15.150:6789/0 conn(0x7f04540cd610 legacy=0x7f04540d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.207+0000 7f045907c640 1 -- 172.21.15.139:0/836413922 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045409a260 con 0x7f04540d0a80 2024-04-25T09:54:49.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.207+0000 7f045907c640 1 --2- 172.21.15.139:0/836413922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04540d0a80 0x7f04541618c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0444009870 tx=0x7f044402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7052fd736fb11959 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.208+0000 7f045887b640 1 -- 172.21.15.139:0/836413922 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444037070 con 0x7f04540d0a80 2024-04-25T09:54:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.208+0000 7f045887b640 1 -- 172.21.15.139:0/836413922 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044402ae50 con 0x7f04540d0a80 2024-04-25T09:54:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.208+0000 7f045887b640 1 -- 172.21.15.139:0/836413922 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444032860 con 0x7f04540d0a80 2024-04-25T09:54:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.208+0000 7f045bae1640 1 -- 172.21.15.139:0/836413922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04540d0a80 msgr2=0x7f04541618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.208+0000 7f045bae1640 1 --2- 172.21.15.139:0/836413922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04540d0a80 0x7f04541618c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0444009870 tx=0x7f044402a300 comp rx=0 tx=0).stop 2024-04-25T09:54:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.208+0000 7f045bae1640 1 -- 172.21.15.139:0/836413922 shutdown_connections 2024-04-25T09:54:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.208+0000 7f045bae1640 1 --2- 172.21.15.139:0/836413922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04540d0a80 0x7f04541618c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.208+0000 7f045bae1640 1 --2- 172.21.15.139:0/836413922 >> v2:172.21.15.150:3300/0 conn(0x7f045409ad40 0x7f0454161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.208+0000 7f045bae1640 1 -- 172.21.15.139:0/836413922 >> 172.21.15.139:0/836413922 conn(0x7f04540b31f0 msgr2=0x7f04540b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.208+0000 7f045bae1640 1 -- 172.21.15.139:0/836413922 shutdown_connections 2024-04-25T09:54:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.208+0000 7f045bae1640 1 -- 172.21.15.139:0/836413922 wait complete. 2024-04-25T09:54:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.209+0000 7f045bae1640 1 Processor -- start 2024-04-25T09:54:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.209+0000 7f045bae1640 1 -- start start 2024-04-25T09:54:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.210+0000 7f045bae1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f045409ad40 0x7f045415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.210+0000 7f045bae1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04540cd610 0x7f04540c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.210+0000 7f045bae1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04540d0a80 0x7f04540c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.210+0000 7f045bae1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04540cd9f0 con 0x7f04540d0a80 2024-04-25T09:54:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.210+0000 7f045bae1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045409a820 con 0x7f04540cd610 2024-04-25T09:54:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.210+0000 7f045bae1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04540cdb70 con 0x7f045409ad40 2024-04-25T09:54:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.210+0000 7f045a07e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04540d0a80 0x7f04540c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.210+0000 7f045a07e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04540d0a80 0x7f04540c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34062/0 (socket says 172.21.15.139:34062) 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.210+0000 7f045a07e640 1 -- 172.21.15.139:0/425958112 learned_addr learned my addr 172.21.15.139:0/425958112 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.210+0000 7f045987d640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f045409ad40 0x7f045415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.210+0000 7f045907c640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04540cd610 0x7f04540c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.211+0000 7f045a07e640 1 -- 172.21.15.139:0/425958112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f045409ad40 msgr2=0x7f045415a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.211+0000 7f045a07e640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f045409ad40 0x7f045415a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.211+0000 7f045a07e640 1 -- 172.21.15.139:0/425958112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04540cd610 msgr2=0x7f04540c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.211+0000 7f045a07e640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04540cd610 0x7f04540c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.211+0000 7f045a07e640 1 -- 172.21.15.139:0/425958112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04540b6660 con 0x7f04540d0a80 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.211+0000 7f045a07e640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04540d0a80 0x7f04540c76a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f044c00efc0 tx=0x7f044c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.211+0000 7f044a7fc640 1 -- 172.21.15.139:0/425958112 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c018020 con 0x7f04540d0a80 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.211+0000 7f044a7fc640 1 -- 172.21.15.139:0/425958112 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c014480 con 0x7f04540d0a80 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.211+0000 7f045bae1640 1 -- 172.21.15.139:0/425958112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04540b6940 con 0x7f04540d0a80 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.211+0000 7f044a7fc640 1 -- 172.21.15.139:0/425958112 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c014870 con 0x7f04540d0a80 2024-04-25T09:54:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.211+0000 7f045bae1640 1 -- 172.21.15.139:0/425958112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04540b6f00 con 0x7f04540d0a80 2024-04-25T09:54:49.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.212+0000 7f045bae1640 1 -- 172.21.15.139:0/425958112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0454074520 con 0x7f04540d0a80 2024-04-25T09:54:49.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.212+0000 7f044a7fc640 1 -- 172.21.15.139:0/425958112 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f044c005d10 con 0x7f04540d0a80 2024-04-25T09:54:49.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.213+0000 7f044a7fc640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f043806b4d0 0x7f043806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.213+0000 7f044a7fc640 1 -- 172.21.15.139:0/425958112 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f044c08abd0 con 0x7f04540d0a80 2024-04-25T09:54:49.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.213+0000 7f045987d640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f043806b4d0 0x7f043806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.213+0000 7f045987d640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f043806b4d0 0x7f043806d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0450001780 tx=0x7f0450009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.217+0000 7f044a7fc640 1 -- 172.21.15.139:0/425958112 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f044c022070 con 0x7f04540d0a80 2024-04-25T09:54:49.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.386+0000 7f045bae1640 1 -- 172.21.15.139:0/425958112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0454161220 con 0x7f04540d0a80 2024-04-25T09:54:49.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.389+0000 7f044a7fc640 1 -- 172.21.15.139:0/425958112 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f044c010040 con 0x7f04540d0a80 2024-04-25T09:54:49.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.391+0000 7f045bae1640 1 -- 172.21.15.139:0/425958112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f043806b4d0 msgr2=0x7f043806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.391+0000 7f045bae1640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f043806b4d0 0x7f043806d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0450001780 tx=0x7f0450009040 comp rx=0 tx=0).stop 2024-04-25T09:54:49.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.391+0000 7f045bae1640 1 -- 172.21.15.139:0/425958112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04540d0a80 msgr2=0x7f04540c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.391+0000 7f045bae1640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04540d0a80 0x7f04540c76a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f044c00efc0 tx=0x7f044c00c490 comp rx=0 tx=0).stop 2024-04-25T09:54:49.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.391+0000 7f045bae1640 1 -- 172.21.15.139:0/425958112 shutdown_connections 2024-04-25T09:54:49.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.391+0000 7f045bae1640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f04540d0a80 0x7f04540c76a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.391+0000 7f045bae1640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f043806b4d0 0x7f043806d990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.391+0000 7f045bae1640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04540cd610 0x7f04540c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.391+0000 7f045bae1640 1 --2- 172.21.15.139:0/425958112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f045409ad40 0x7f045415a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.391+0000 7f045bae1640 1 -- 172.21.15.139:0/425958112 >> 172.21.15.139:0/425958112 conn(0x7f04540b31f0 msgr2=0x7f04540b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.391+0000 7f045bae1640 1 -- 172.21.15.139:0/425958112 shutdown_connections 2024-04-25T09:54:49.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.391+0000 7f045bae1640 1 -- 172.21.15.139:0/425958112 wait complete. 2024-04-25T09:54:49.404 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T09:54:49.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.500+0000 7ff84a95c640 1 Processor -- start 2024-04-25T09:54:49.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.500+0000 7ff84a95c640 1 -- start start 2024-04-25T09:54:49.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff84a95c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8441611e0 0x7ff8440b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff84a95c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff84406c780 0x7ff8440d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff84a95c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8440d4740 con 0x7ff8440cea00 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff84a95c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8440d1fa0 con 0x7ff84406c780 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff84a95c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440d2560 con 0x7ff8441611e0 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff8437fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff8440cea00 0x7ff84406c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60300/0 (socket says 172.21.15.139:60300) 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff8437fe640 1 -- 172.21.15.139:0/1504885806 learned_addr learned my addr 172.21.15.139:0/1504885806 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff843fff640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8441611e0 0x7ff8440b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371525652 0 0) 0x7ff8440d4740 con 0x7ff8440cea00 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff848ef9640 1 --2- 172.21.15.139:0/1504885806 >> v2:172.21.15.150:3300/0 conn(0x7ff84406c780 0x7ff8440d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff82c003520 con 0x7ff8440cea00 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2987560125 0 0) 0x7ff82c003520 con 0x7ff8440cea00 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8441611e0 msgr2=0x7ff8440b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff842ffd640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8441611e0 0x7ff8440b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 >> v2:172.21.15.150:3300/0 conn(0x7ff84406c780 msgr2=0x7ff8440d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff842ffd640 1 --2- 172.21.15.139:0/1504885806 >> v2:172.21.15.150:3300/0 conn(0x7ff84406c780 0x7ff8440d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8440b2770 con 0x7ff8440cea00 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.501+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7ff838001670 con 0x7ff8440cea00 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 >> v1:172.21.15.139:6789/0 conn(0x7ff8440cea00 legacy=0x7ff84406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff842ffd640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8441611e0 0x7ff82c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff842ffd640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff82c003f80 0x7ff82c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff842ffd640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff82c004d40 0x7ff82c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440d1fa0 con 0x7ff82c003f80 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440d2560 con 0x7ff82c004d40 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c00b9c0 con 0x7ff8441611e0 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff8437fe640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff82c003f80 0x7ff82c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff843fff640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8441611e0 0x7ff82c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff848ef9640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff82c004d40 0x7ff82c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff8437fe640 1 -- 172.21.15.139:0/1504885806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8441611e0 msgr2=0x7ff82c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff8437fe640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8441611e0 0x7ff82c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff8437fe640 1 -- 172.21.15.139:0/1504885806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff82c004d40 msgr2=0x7ff82c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff8437fe640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff82c004d40 0x7ff82c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff8437fe640 1 -- 172.21.15.139:0/1504885806 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c003520 con 0x7ff82c003f80 2024-04-25T09:54:49.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.502+0000 7ff848ef9640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff82c004d40 0x7ff82c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:49.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.503+0000 7ff8437fe640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff82c003f80 0x7ff82c004800 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff838007fd0 tx=0x7ff82c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3e91511a7312a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.503+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838007ca0 con 0x7ff82c003f80 2024-04-25T09:54:49.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.503+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff838005ad0 con 0x7ff82c003f80 2024-04-25T09:54:49.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.503+0000 7ff842ffd640 1 -- 172.21.15.139:0/1504885806 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838007070 con 0x7ff82c003f80 2024-04-25T09:54:49.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.503+0000 7ff84a95c640 1 -- 172.21.15.139:0/1504885806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff82c003f80 msgr2=0x7ff82c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.503+0000 7ff84a95c640 1 --2- 172.21.15.139:0/1504885806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff82c003f80 0x7ff82c004800 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff838007fd0 tx=0x7ff82c005830 comp rx=0 tx=0).stop 2024-04-25T09:54:49.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.503+0000 7ff848ef9640 1 -- 172.21.15.139:0/1504885806 reap_dead start 2024-04-25T09:54:49.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.503+0000 7ff84a95c640 1 -- 172.21.15.139:0/1504885806 shutdown_connections 2024-04-25T09:54:49.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.503+0000 7ff84a95c640 1 -- 172.21.15.139:0/1504885806 >> 172.21.15.139:0/1504885806 conn(0x7ff8440b5b00 msgr2=0x7ff8440d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.503+0000 7ff84a95c640 1 -- 172.21.15.139:0/1504885806 shutdown_connections 2024-04-25T09:54:49.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.503+0000 7ff84a95c640 1 -- 172.21.15.139:0/1504885806 wait complete. 2024-04-25T09:54:49.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.504+0000 7ff84a95c640 1 Processor -- start 2024-04-25T09:54:49.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.505+0000 7ff84a95c640 1 -- start start 2024-04-25T09:54:49.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.505+0000 7ff84a95c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff82c003f80 0x7ff8441f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.505+0000 7ff84a95c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8441f33f0 0x7ff8440c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.505+0000 7ff84a95c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8440c9880 0x7ff8440c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.505+0000 7ff84a95c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440d21f0 con 0x7ff8441f33f0 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.505+0000 7ff84a95c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440b2d30 con 0x7ff82c003f80 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.505+0000 7ff84a95c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8441615c0 con 0x7ff8440c9880 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.505+0000 7ff8437fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8441f33f0 0x7ff8440c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.505+0000 7ff8437fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8441f33f0 0x7ff8440c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34076/0 (socket says 172.21.15.139:34076) 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.505+0000 7ff8437fe640 1 -- 172.21.15.139:0/3861501220 learned_addr learned my addr 172.21.15.139:0/3861501220 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff843fff640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff82c003f80 0x7ff8441f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff848ef9640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8440c9880 0x7ff8440c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff8437fe640 1 -- 172.21.15.139:0/3861501220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8440c9880 msgr2=0x7ff8440c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff8437fe640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8440c9880 0x7ff8440c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff8437fe640 1 -- 172.21.15.139:0/3861501220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff82c003f80 msgr2=0x7ff8441f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff8437fe640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff82c003f80 0x7ff8441f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff8437fe640 1 -- 172.21.15.139:0/3861501220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8440c8340 con 0x7ff8441f33f0 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff8437fe640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8441f33f0 0x7ff8440c9340 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff83800c250 tx=0x7ff838000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff840ff9640 1 -- 172.21.15.139:0/3861501220 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8380015b0 con 0x7ff8441f33f0 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff840ff9640 1 -- 172.21.15.139:0/3861501220 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff838042070 con 0x7ff8441f33f0 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff84a95c640 1 -- 172.21.15.139:0/3861501220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8440c8620 con 0x7ff8441f33f0 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff840ff9640 1 -- 172.21.15.139:0/3861501220 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8380072b0 con 0x7ff8441f33f0 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.506+0000 7ff84a95c640 1 -- 172.21.15.139:0/3861501220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8440c8be0 con 0x7ff8441f33f0 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.507+0000 7ff84a95c640 1 -- 172.21.15.139:0/3861501220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff844074520 con 0x7ff8441f33f0 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.507+0000 7ff840ff9640 1 -- 172.21.15.139:0/3861501220 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff838007450 con 0x7ff8441f33f0 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.508+0000 7ff840ff9640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff83006b500 0x7ff83006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.508+0000 7ff840ff9640 1 -- 172.21.15.139:0/3861501220 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff8380a9f80 con 0x7ff8441f33f0 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.508+0000 7ff843fff640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff83006b500 0x7ff83006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.508+0000 7ff843fff640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff83006b500 0x7ff83006d9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff834009a20 tx=0x7ff834008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.512+0000 7ff840ff9640 1 -- 172.21.15.139:0/3861501220 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff838079000 con 0x7ff8441f33f0 2024-04-25T09:54:49.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:49.754+0000 7ff84a95c640 1 -- 172.21.15.139:0/3861501220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7ff84406ceb0 con 0x7ff8441f33f0 2024-04-25T09:54:50.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.080+0000 7ff840ff9640 1 -- 172.21.15.139:0/3861501220 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v111) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff838005aa0 con 0x7ff8441f33f0 2024-04-25T09:54:50.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.083+0000 7ff84a95c640 1 -- 172.21.15.139:0/3861501220 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff83006b500 msgr2=0x7ff83006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.083+0000 7ff84a95c640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff83006b500 0x7ff83006d9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff834009a20 tx=0x7ff834008040 comp rx=0 tx=0).stop 2024-04-25T09:54:50.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.083+0000 7ff84a95c640 1 -- 172.21.15.139:0/3861501220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8441f33f0 msgr2=0x7ff8440c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.083+0000 7ff84a95c640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8441f33f0 0x7ff8440c9340 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff83800c250 tx=0x7ff838000fe0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.083+0000 7ff84a95c640 1 -- 172.21.15.139:0/3861501220 shutdown_connections 2024-04-25T09:54:50.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.083+0000 7ff84a95c640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8441f33f0 0x7ff8440c9340 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.083+0000 7ff84a95c640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff83006b500 0x7ff83006d9c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.083+0000 7ff84a95c640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8440c9880 0x7ff8440c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.083+0000 7ff84a95c640 1 --2- 172.21.15.139:0/3861501220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff82c003f80 0x7ff8441f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.083+0000 7ff84a95c640 1 -- 172.21.15.139:0/3861501220 >> 172.21.15.139:0/3861501220 conn(0x7ff8440b5b00 msgr2=0x7ff8440b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.083+0000 7ff84a95c640 1 -- 172.21.15.139:0/3861501220 shutdown_connections 2024-04-25T09:54:50.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.083+0000 7ff84a95c640 1 -- 172.21.15.139:0/3861501220 wait complete. 2024-04-25T09:54:50.097 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T09:54:50.097 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-25T09:54:50.097 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T09:54:50.098 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:50.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.192+0000 7fa3957c8640 1 Processor -- start 2024-04-25T09:54:50.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.192+0000 7fa3957c8640 1 -- start start 2024-04-25T09:54:50.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.192+0000 7fa3957c8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3900d1080 0x7fa3900b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.192+0000 7fa3957c8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3900cf250 con 0x7fa390161850 2024-04-25T09:54:50.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.192+0000 7fa3957c8640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3900ce9b0 con 0x7fa39015dc60 2024-04-25T09:54:50.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.192+0000 7fa3957c8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900d3730 con 0x7fa3900d1080 2024-04-25T09:54:50.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38f7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa390161850 0x7fa3900d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60312/0 (socket says 172.21.15.139:60312) 2024-04-25T09:54:50.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38f7fe640 1 -- 172.21.15.139:0/1889852675 learned_addr learned my addr 172.21.15.139:0/1889852675 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:50.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38e7fc640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3900d1080 0x7fa3900b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530693928 0 0) 0x7fa388001490 con 0x7fa390161850 2024-04-25T09:54:50.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa37c003520 con 0x7fa390161850 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 346909593 0 0) 0x7fa37c003520 con 0x7fa390161850 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3900d1080 msgr2=0x7fa3900b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3900d1080 0x7fa3900b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 >> v1:172.21.15.150:6789/0 conn(0x7fa39015dc60 legacy=0x7fa3900d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3900cec90 con 0x7fa390161850 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fa388001bc0 con 0x7fa390161850 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 >> v1:172.21.15.139:6789/0 conn(0x7fa390161850 legacy=0x7fa3900d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3900d1080 0x7fa37c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa37c003f20 0x7fa37c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa37c004d40 0x7fa37c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900d3730 con 0x7fa37c003f20 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c00ba20 con 0x7fa37c004d40 2024-04-25T09:54:50.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c00bc20 con 0x7fa3900d1080 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38e7fc640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa37c003f20 0x7fa37c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38effd640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3900d1080 0x7fa37c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38f7fe640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa37c004d40 0x7fa37c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38e7fc640 1 -- 172.21.15.139:0/1889852675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3900d1080 msgr2=0x7fa37c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38e7fc640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3900d1080 0x7fa37c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38e7fc640 1 -- 172.21.15.139:0/1889852675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa37c004d40 msgr2=0x7fa37c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38e7fc640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa37c004d40 0x7fa37c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38e7fc640 1 -- 172.21.15.139:0/1889852675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c003520 con 0x7fa37c003f20 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.193+0000 7fa38f7fe640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa37c004d40 0x7fa37c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.194+0000 7fa38e7fc640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa37c003f20 0x7fa37c004800 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa378009ac0 tx=0x7fa37c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86051c2c5ae6fdbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.194+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378036070 con 0x7fa37c003f20 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.194+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa378031400 con 0x7fa37c003f20 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.194+0000 7fa38dffb640 1 -- 172.21.15.139:0/1889852675 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378009690 con 0x7fa37c003f20 2024-04-25T09:54:50.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.195+0000 7fa3957c8640 1 -- 172.21.15.139:0/1889852675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa37c003f20 msgr2=0x7fa37c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.195+0000 7fa3957c8640 1 --2- 172.21.15.139:0/1889852675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa37c003f20 0x7fa37c004800 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa378009ac0 tx=0x7fa37c005830 comp rx=0 tx=0).stop 2024-04-25T09:54:50.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.195+0000 7fa38f7fe640 1 -- 172.21.15.139:0/1889852675 reap_dead start 2024-04-25T09:54:50.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.195+0000 7fa3957c8640 1 -- 172.21.15.139:0/1889852675 shutdown_connections 2024-04-25T09:54:50.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.195+0000 7fa3957c8640 1 -- 172.21.15.139:0/1889852675 >> 172.21.15.139:0/1889852675 conn(0x7fa3900b5910 msgr2=0x7fa3900b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.195+0000 7fa3957c8640 1 -- 172.21.15.139:0/1889852675 shutdown_connections 2024-04-25T09:54:50.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.195+0000 7fa3957c8640 1 -- 172.21.15.139:0/1889852675 wait complete. 2024-04-25T09:54:50.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.196+0000 7fa3957c8640 1 Processor -- start 2024-04-25T09:54:50.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.196+0000 7fa3957c8640 1 -- start start 2024-04-25T09:54:50.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa3957c8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa37c003f20 0x7fa390151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa3957c8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3900cd4a0 0x7fa3900cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa3957c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3900d3970 0x7fa3900d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa3957c8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39015e040 con 0x7fa3900d3970 2024-04-25T09:54:50.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa3957c8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900d1fc0 con 0x7fa3900cd4a0 2024-04-25T09:54:50.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa3957c8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3901f3b60 con 0x7fa37c003f20 2024-04-25T09:54:50.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa38f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3900d3970 0x7fa3900d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa38f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3900d3970 0x7fa3900d3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34092/0 (socket says 172.21.15.139:34092) 2024-04-25T09:54:50.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa38f7fe640 1 -- 172.21.15.139:0/3391187334 learned_addr learned my addr 172.21.15.139:0/3391187334 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:50.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa38effd640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa37c003f20 0x7fa390151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa38e7fc640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3900cd4a0 0x7fa3900cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa38f7fe640 1 -- 172.21.15.139:0/3391187334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa37c003f20 msgr2=0x7fa390151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa38f7fe640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa37c003f20 0x7fa390151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa38f7fe640 1 -- 172.21.15.139:0/3391187334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3900cd4a0 msgr2=0x7fa3900cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa38f7fe640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3900cd4a0 0x7fa3900cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.197+0000 7fa38f7fe640 1 -- 172.21.15.139:0/3391187334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3901f3e40 con 0x7fa3900d3970 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.198+0000 7fa38f7fe640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3900d3970 0x7fa3900d3e00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa39006d250 tx=0x7fa38800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.198+0000 7fa3877fe640 1 -- 172.21.15.139:0/3391187334 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388017020 con 0x7fa3900d3970 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.198+0000 7fa3877fe640 1 -- 172.21.15.139:0/3391187334 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38800eb90 con 0x7fa3900d3970 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.198+0000 7fa3877fe640 1 -- 172.21.15.139:0/3391187334 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388013420 con 0x7fa3900d3970 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.198+0000 7fa3957c8640 1 -- 172.21.15.139:0/3391187334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3901f4120 con 0x7fa3900d3970 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.198+0000 7fa3957c8640 1 -- 172.21.15.139:0/3391187334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa390152360 con 0x7fa3900d3970 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.199+0000 7fa3957c8640 1 -- 172.21.15.139:0/3391187334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa390074520 con 0x7fa3900d3970 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.199+0000 7fa3877fe640 1 -- 172.21.15.139:0/3391187334 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa38801a020 con 0x7fa3900d3970 2024-04-25T09:54:50.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.199+0000 7fa3877fe640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa37006b4d0 0x7fa37006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.200+0000 7fa3877fe640 1 -- 172.21.15.139:0/3391187334 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa38808a360 con 0x7fa3900d3970 2024-04-25T09:54:50.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.200+0000 7fa38effd640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa37006b4d0 0x7fa37006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.200+0000 7fa38effd640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa37006b4d0 0x7fa37006d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa380000c00 tx=0x7fa380009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.204+0000 7fa3877fe640 1 -- 172.21.15.139:0/3391187334 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa388021020 con 0x7fa3900d3970 2024-04-25T09:54:50.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.376+0000 7fa3957c8640 1 -- 172.21.15.139:0/3391187334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3900d16f0 con 0x7fa3900d3970 2024-04-25T09:54:50.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.376+0000 7fa3877fe640 1 -- 172.21.15.139:0/3391187334 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+13343 (secure 0 0 0) 0x7fa388088e20 con 0x7fa3900d3970 2024-04-25T09:54:50.379 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:50.379 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":41,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:49.077299+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6837/2069784668":"2024-04-26T09:54:46.072588+0000","172.21.15.139:6835/2069784668":"2024-04-26T09:54:46.072588+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:50.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.379+0000 7fa3957c8640 1 -- 172.21.15.139:0/3391187334 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa37006b4d0 msgr2=0x7fa37006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.379+0000 7fa3957c8640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa37006b4d0 0x7fa37006d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa380000c00 tx=0x7fa380009040 comp rx=0 tx=0).stop 2024-04-25T09:54:50.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.379+0000 7fa3957c8640 1 -- 172.21.15.139:0/3391187334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3900d3970 msgr2=0x7fa3900d3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.379+0000 7fa3957c8640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3900d3970 0x7fa3900d3e00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa39006d250 tx=0x7fa38800e530 comp rx=0 tx=0).stop 2024-04-25T09:54:50.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.379+0000 7fa3957c8640 1 -- 172.21.15.139:0/3391187334 shutdown_connections 2024-04-25T09:54:50.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.379+0000 7fa3957c8640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa3900d3970 0x7fa3900d3e00 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.379+0000 7fa3957c8640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa37006b4d0 0x7fa37006d990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.379+0000 7fa3957c8640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3900cd4a0 0x7fa3900cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.379+0000 7fa3957c8640 1 --2- 172.21.15.139:0/3391187334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa37c003f20 0x7fa390151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.379+0000 7fa3957c8640 1 -- 172.21.15.139:0/3391187334 >> 172.21.15.139:0/3391187334 conn(0x7fa3900b5910 msgr2=0x7fa390153c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.379+0000 7fa3957c8640 1 -- 172.21.15.139:0/3391187334 shutdown_connections 2024-04-25T09:54:50.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.379+0000 7fa3957c8640 1 -- 172.21.15.139:0/3391187334 wait complete. 2024-04-25T09:54:50.392 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:54:50.393 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:50.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.490+0000 7f1d77763640 1 Processor -- start 2024-04-25T09:54:50.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.490+0000 7f1d77763640 1 -- start start 2024-04-25T09:54:50.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d77763640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d70144090 0x7f1d700b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d77763640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1d700cc9e0 0x7f1d7006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d77763640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d700b2a50 con 0x7f1d700cce20 2024-04-25T09:54:50.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d77763640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7015b090 con 0x7f1d700cc9e0 2024-04-25T09:54:50.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d77763640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7015b370 con 0x7f1d70144090 2024-04-25T09:54:50.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d74cfe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1d700cc9e0 0x7f1d7006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d74cfe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1d700cc9e0 0x7f1d7006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34104/0 (socket says 172.21.15.139:34104) 2024-04-25T09:54:50.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d74cfe640 1 -- 172.21.15.139:0/2593157512 learned_addr learned my addr 172.21.15.139:0/2593157512 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:50.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d754ff640 1 --2- 172.21.15.139:0/2593157512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d70144090 0x7f1d700b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d74cfe640 1 -- 172.21.15.139:0/2593157512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d70144090 msgr2=0x7f1d700b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:50.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d74cfe640 1 --2- 172.21.15.139:0/2593157512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d70144090 0x7f1d700b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d74cfe640 1 -- 172.21.15.139:0/2593157512 >> v1:172.21.15.150:6789/0 conn(0x7f1d700cce20 legacy=0x7f1d7015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.491+0000 7f1d74cfe640 1 -- 172.21.15.139:0/2593157512 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7015b650 con 0x7f1d700cc9e0 2024-04-25T09:54:50.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.492+0000 7f1d74cfe640 1 --2- 172.21.15.139:0/2593157512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700cc9e0 0x7f1d7006e330 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009870 tx=0x7f1d6002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3d0e76de10e73e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.492+0000 7f1d67fff640 1 -- 172.21.15.139:0/2593157512 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60037070 con 0x7f1d700cc9e0 2024-04-25T09:54:50.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.492+0000 7f1d67fff640 1 -- 172.21.15.139:0/2593157512 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d60032440 con 0x7f1d700cc9e0 2024-04-25T09:54:50.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.492+0000 7f1d67fff640 1 -- 172.21.15.139:0/2593157512 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60032810 con 0x7f1d700cc9e0 2024-04-25T09:54:50.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.493+0000 7f1d77763640 1 -- 172.21.15.139:0/2593157512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700cc9e0 msgr2=0x7f1d7006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.493+0000 7f1d77763640 1 --2- 172.21.15.139:0/2593157512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700cc9e0 0x7f1d7006e330 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009870 tx=0x7f1d6002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.493+0000 7f1d77763640 1 -- 172.21.15.139:0/2593157512 shutdown_connections 2024-04-25T09:54:50.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.493+0000 7f1d77763640 1 --2- 172.21.15.139:0/2593157512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700cc9e0 0x7f1d7006e330 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.493+0000 7f1d77763640 1 --2- 172.21.15.139:0/2593157512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d70144090 0x7f1d700b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.493+0000 7f1d77763640 1 -- 172.21.15.139:0/2593157512 >> 172.21.15.139:0/2593157512 conn(0x7f1d70067000 msgr2=0x7f1d7006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.493+0000 7f1d77763640 1 -- 172.21.15.139:0/2593157512 shutdown_connections 2024-04-25T09:54:50.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.493+0000 7f1d77763640 1 -- 172.21.15.139:0/2593157512 wait complete. 2024-04-25T09:54:50.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.494+0000 7f1d77763640 1 Processor -- start 2024-04-25T09:54:50.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.494+0000 7f1d77763640 1 -- start start 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d77763640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700cc9e0 0x7f1d7006e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d77763640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d700cce20 0x7f1d70159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d77763640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d70144090 0x7f1d7015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d77763640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d70144470 con 0x7f1d700cc9e0 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d77763640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7015bc10 con 0x7f1d700cce20 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d77763640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7015ad80 con 0x7f1d70144090 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d754ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700cc9e0 0x7f1d7006e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d754ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700cc9e0 0x7f1d7006e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34116/0 (socket says 172.21.15.139:34116) 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d754ff640 1 -- 172.21.15.139:0/146014915 learned_addr learned my addr 172.21.15.139:0/146014915 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d74cfe640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d700cce20 0x7f1d70159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d75d00640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d70144090 0x7f1d7015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d754ff640 1 -- 172.21.15.139:0/146014915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d70144090 msgr2=0x7f1d7015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d754ff640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d70144090 0x7f1d7015a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d754ff640 1 -- 172.21.15.139:0/146014915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d700cce20 msgr2=0x7f1d70159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d754ff640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d700cce20 0x7f1d70159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d754ff640 1 -- 172.21.15.139:0/146014915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7006eb70 con 0x7f1d700cc9e0 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d754ff640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700cc9e0 0x7f1d7006e630 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00efc0 tx=0x7f1d6c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d65ffb640 1 -- 172.21.15.139:0/146014915 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c018020 con 0x7f1d700cc9e0 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d65ffb640 1 -- 172.21.15.139:0/146014915 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c014480 con 0x7f1d700cc9e0 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d77763640 1 -- 172.21.15.139:0/146014915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7006d590 con 0x7f1d700cc9e0 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.495+0000 7f1d65ffb640 1 -- 172.21.15.139:0/146014915 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c014850 con 0x7f1d700cc9e0 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.496+0000 7f1d77763640 1 -- 172.21.15.139:0/146014915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7006db50 con 0x7f1d700cc9e0 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.497+0000 7f1d77763640 1 -- 172.21.15.139:0/146014915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d70074520 con 0x7f1d700cc9e0 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.497+0000 7f1d65ffb640 1 -- 172.21.15.139:0/146014915 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d6c005d10 con 0x7f1d700cc9e0 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.497+0000 7f1d65ffb640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d5406b4d0 0x7f1d5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.497+0000 7f1d65ffb640 1 -- 172.21.15.139:0/146014915 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1d6c08aa70 con 0x7f1d700cc9e0 2024-04-25T09:54:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.498+0000 7f1d74cfe640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d5406b4d0 0x7f1d5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.498+0000 7f1d74cfe640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d5406b4d0 0x7f1d5406d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1d60034000 tx=0x7f1d6002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.502+0000 7f1d65ffb640 1 -- 172.21.15.139:0/146014915 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d6c022070 con 0x7f1d700cc9e0 2024-04-25T09:54:50.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.669+0000 7f1d77763640 1 -- 172.21.15.139:0/146014915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d70143890 con 0x7f1d700cc9e0 2024-04-25T09:54:50.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.669+0000 7f1d65ffb640 1 -- 172.21.15.139:0/146014915 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d6c010040 con 0x7f1d700cc9e0 2024-04-25T09:54:50.672 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:54:50.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.671+0000 7f1d77763640 1 -- 172.21.15.139:0/146014915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d5406b4d0 msgr2=0x7f1d5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.671+0000 7f1d77763640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d5406b4d0 0x7f1d5406d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1d60034000 tx=0x7f1d6002e000 comp rx=0 tx=0).stop 2024-04-25T09:54:50.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.671+0000 7f1d77763640 1 -- 172.21.15.139:0/146014915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700cc9e0 msgr2=0x7f1d7006e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.671+0000 7f1d77763640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700cc9e0 0x7f1d7006e630 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00efc0 tx=0x7f1d6c00c490 comp rx=0 tx=0).stop 2024-04-25T09:54:50.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.671+0000 7f1d77763640 1 -- 172.21.15.139:0/146014915 shutdown_connections 2024-04-25T09:54:50.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.671+0000 7f1d77763640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d5406b4d0 0x7f1d5406d990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.671+0000 7f1d77763640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d70144090 0x7f1d7015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.671+0000 7f1d77763640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d700cce20 0x7f1d70159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.671+0000 7f1d77763640 1 --2- 172.21.15.139:0/146014915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d700cc9e0 0x7f1d7006e630 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.671+0000 7f1d77763640 1 -- 172.21.15.139:0/146014915 >> 172.21.15.139:0/146014915 conn(0x7f1d70067000 msgr2=0x7f1d70140ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.671+0000 7f1d77763640 1 -- 172.21.15.139:0/146014915 shutdown_connections 2024-04-25T09:54:50.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.672+0000 7f1d77763640 1 -- 172.21.15.139:0/146014915 wait complete. 2024-04-25T09:54:50.685 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T09:54:50.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.780+0000 7f6b92bd5640 1 Processor -- start 2024-04-25T09:54:50.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.780+0000 7f6b92bd5640 1 -- start start 2024-04-25T09:54:50.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.780+0000 7f6b92bd5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6b8c147190 0x7f6b8c0b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.780+0000 7f6b92bd5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8c147570 con 0x7f6b8c15ec30 2024-04-25T09:54:50.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.780+0000 7f6b92bd5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8c0b1930 con 0x7f6b8c0af350 2024-04-25T09:54:50.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.780+0000 7f6b92bd5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b8c0b24b0 con 0x7f6b8c147190 2024-04-25T09:54:50.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.781+0000 7f6b90971640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6b8c147190 0x7f6b8c0b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.781+0000 7f6b8bfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f6b8c15ec30 0x7f6b8c0aec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60318/0 (socket says 172.21.15.139:60318) 2024-04-25T09:54:50.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.781+0000 7f6b90971640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6b8c147190 0x7f6b8c0b0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34124/0 (socket says 172.21.15.139:34124) 2024-04-25T09:54:50.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.781+0000 7f6b8bfff640 1 -- 172.21.15.139:0/474170968 learned_addr learned my addr 172.21.15.139:0/474170968 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:50.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.781+0000 7f6b90971640 1 -- 172.21.15.139:0/474170968 >> v1:172.21.15.150:6789/0 conn(0x7f6b8c0af350 legacy=0x7f6b8c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:50.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.781+0000 7f6b90971640 1 -- 172.21.15.139:0/474170968 >> v1:172.21.15.139:6789/0 conn(0x7f6b8c15ec30 legacy=0x7f6b8c0aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.781+0000 7f6b90971640 1 -- 172.21.15.139:0/474170968 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c0b1c10 con 0x7f6b8c147190 2024-04-25T09:54:50.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.781+0000 7f6b8b7fe640 1 -- 172.21.15.139:0/474170968 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613078640 0 0) 0x7f6b8c147570 con 0x7f6b8c15ec30 2024-04-25T09:54:50.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.782+0000 7f6b90971640 1 --2- 172.21.15.139:0/474170968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b8c147190 0x7f6b8c0b0fe0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6b800097a0 tx=0x7f6b8002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88df20e1be553d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.782+0000 7f6b8b7fe640 1 -- 172.21.15.139:0/474170968 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80037070 con 0x7f6b8c147190 2024-04-25T09:54:50.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.782+0000 7f6b8b7fe640 1 -- 172.21.15.139:0/474170968 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b80032440 con 0x7f6b8c147190 2024-04-25T09:54:50.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.782+0000 7f6b8b7fe640 1 -- 172.21.15.139:0/474170968 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80032810 con 0x7f6b8c147190 2024-04-25T09:54:50.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.782+0000 7f6b92bd5640 1 -- 172.21.15.139:0/474170968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b8c147190 msgr2=0x7f6b8c0b0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.782+0000 7f6b92bd5640 1 --2- 172.21.15.139:0/474170968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b8c147190 0x7f6b8c0b0fe0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6b800097a0 tx=0x7f6b8002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.782+0000 7f6b92bd5640 1 -- 172.21.15.139:0/474170968 shutdown_connections 2024-04-25T09:54:50.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.782+0000 7f6b92bd5640 1 --2- 172.21.15.139:0/474170968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b8c147190 0x7f6b8c0b0fe0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.782+0000 7f6b92bd5640 1 -- 172.21.15.139:0/474170968 >> 172.21.15.139:0/474170968 conn(0x7f6b8c066f20 msgr2=0x7f6b8c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.782+0000 7f6b92bd5640 1 -- 172.21.15.139:0/474170968 shutdown_connections 2024-04-25T09:54:50.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.782+0000 7f6b92bd5640 1 -- 172.21.15.139:0/474170968 wait complete. 2024-04-25T09:54:50.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.783+0000 7f6b92bd5640 1 Processor -- start 2024-04-25T09:54:50.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.783+0000 7f6b92bd5640 1 -- start start 2024-04-25T09:54:50.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.784+0000 7f6b92bd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b8c0af350 0x7f6b8c0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.784+0000 7f6b92bd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b8c147190 0x7f6b8c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.784+0000 7f6b92bd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b8c15ec30 0x7f6b8c0b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.784+0000 7f6b92bd5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c0b1520 con 0x7f6b8c0af350 2024-04-25T09:54:50.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.784+0000 7f6b92bd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c0b21d0 con 0x7f6b8c15ec30 2024-04-25T09:54:50.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.784+0000 7f6b92bd5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c0b16a0 con 0x7f6b8c147190 2024-04-25T09:54:50.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.784+0000 7f6b90971640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b8c0af350 0x7f6b8c0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.784+0000 7f6b90971640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b8c0af350 0x7f6b8c0b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34130/0 (socket says 172.21.15.139:34130) 2024-04-25T09:54:50.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.784+0000 7f6b90971640 1 -- 172.21.15.139:0/490273970 learned_addr learned my addr 172.21.15.139:0/490273970 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:50.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.785+0000 7f6b90971640 1 -- 172.21.15.139:0/490273970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b8c147190 msgr2=0x7f6b8c0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.785+0000 7f6b90971640 1 --2- 172.21.15.139:0/490273970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b8c147190 0x7f6b8c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.785+0000 7f6b90971640 1 -- 172.21.15.139:0/490273970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b8c15ec30 msgr2=0x7f6b8c0b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.785+0000 7f6b90971640 1 --2- 172.21.15.139:0/490273970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b8c15ec30 0x7f6b8c0b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.785+0000 7f6b90971640 1 -- 172.21.15.139:0/490273970 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c0cc5f0 con 0x7f6b8c0af350 2024-04-25T09:54:50.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.785+0000 7f6b90971640 1 --2- 172.21.15.139:0/490273970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b8c0af350 0x7f6b8c0b48b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6b800097a0 tx=0x7f6b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.785+0000 7f6b897fa640 1 -- 172.21.15.139:0/490273970 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80041070 con 0x7f6b8c0af350 2024-04-25T09:54:50.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.785+0000 7f6b897fa640 1 -- 172.21.15.139:0/490273970 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8003c870 con 0x7f6b8c0af350 2024-04-25T09:54:50.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.785+0000 7f6b92bd5640 1 -- 172.21.15.139:0/490273970 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8c0cc8d0 con 0x7f6b8c0af350 2024-04-25T09:54:50.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.785+0000 7f6b897fa640 1 -- 172.21.15.139:0/490273970 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80037040 con 0x7f6b8c0af350 2024-04-25T09:54:50.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.785+0000 7f6b92bd5640 1 -- 172.21.15.139:0/490273970 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8c0cce90 con 0x7f6b8c0af350 2024-04-25T09:54:50.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.786+0000 7f6b92bd5640 1 -- 172.21.15.139:0/490273970 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8c0b0fe0 con 0x7f6b8c0af350 2024-04-25T09:54:50.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.787+0000 7f6b897fa640 1 -- 172.21.15.139:0/490273970 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b80044070 con 0x7f6b8c0af350 2024-04-25T09:54:50.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.787+0000 7f6b897fa640 1 --2- 172.21.15.139:0/490273970 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6b7006b4d0 0x7f6b7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.787+0000 7f6b897fa640 1 -- 172.21.15.139:0/490273970 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6b800aac50 con 0x7f6b8c0af350 2024-04-25T09:54:50.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.787+0000 7f6b8bfff640 1 --2- 172.21.15.139:0/490273970 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6b7006b4d0 0x7f6b7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.787+0000 7f6b8bfff640 1 --2- 172.21.15.139:0/490273970 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6b7006b4d0 0x7f6b7006d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b78005880 tx=0x7f6b78007a00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:50.791+0000 7f6b897fa640 1 -- 172.21.15.139:0/490273970 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b8c0b0fe0 con 0x7f6b8c0af350 2024-04-25T09:54:51.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.026+0000 7f6b92bd5640 1 -- 172.21.15.139:0/490273970 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f6b8c074520 con 0x7f6b8c0af350 2024-04-25T09:54:51.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.086+0000 7f6b897fa640 1 -- 172.21.15.139:0/490273970 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v112) v1 ==== 152+0+0 (secure 0 0 0) 0x7f6b800b0020 con 0x7f6b8c0af350 2024-04-25T09:54:51.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.088+0000 7f6b92bd5640 1 -- 172.21.15.139:0/490273970 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6b7006b4d0 msgr2=0x7f6b7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.088+0000 7f6b92bd5640 1 --2- 172.21.15.139:0/490273970 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6b7006b4d0 0x7f6b7006d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b78005880 tx=0x7f6b78007a00 comp rx=0 tx=0).stop 2024-04-25T09:54:51.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.088+0000 7f6b92bd5640 1 -- 172.21.15.139:0/490273970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b8c0af350 msgr2=0x7f6b8c0b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.088+0000 7f6b92bd5640 1 --2- 172.21.15.139:0/490273970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b8c0af350 0x7f6b8c0b48b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6b800097a0 tx=0x7f6b800046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.088+0000 7f6b92bd5640 1 -- 172.21.15.139:0/490273970 shutdown_connections 2024-04-25T09:54:51.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.088+0000 7f6b92bd5640 1 --2- 172.21.15.139:0/490273970 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6b7006b4d0 0x7f6b7006d990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.088+0000 7f6b92bd5640 1 --2- 172.21.15.139:0/490273970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b8c15ec30 0x7f6b8c0b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.088+0000 7f6b92bd5640 1 --2- 172.21.15.139:0/490273970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b8c147190 0x7f6b8c0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.088+0000 7f6b92bd5640 1 --2- 172.21.15.139:0/490273970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b8c0af350 0x7f6b8c0b48b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.088+0000 7f6b92bd5640 1 -- 172.21.15.139:0/490273970 >> 172.21.15.139:0/490273970 conn(0x7f6b8c066f20 msgr2=0x7f6b8c06d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:51.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.088+0000 7f6b92bd5640 1 -- 172.21.15.139:0/490273970 shutdown_connections 2024-04-25T09:54:51.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.088+0000 7f6b92bd5640 1 -- 172.21.15.139:0/490273970 wait complete. 2024-04-25T09:54:51.102 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:54:51.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.200+0000 7f3f8c9c8640 1 Processor -- start 2024-04-25T09:54:51.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.200+0000 7f3f8c9c8640 1 -- start start 2024-04-25T09:54:51.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.200+0000 7f3f8c9c8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f8815da30 0x7f3f880704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.200+0000 7f3f8c9c8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3f8806b270 0x7f3f88167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.200+0000 7f3f8c9c8640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f880731a0 con 0x7f3f88072d60 2024-04-25T09:54:51.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.200+0000 7f3f8c9c8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f880cd540 con 0x7f3f8806b270 2024-04-25T09:54:51.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.200+0000 7f3f8c9c8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880b3de0 con 0x7f3f8815da30 2024-04-25T09:54:51.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.201+0000 7f3f86d9d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3f8806b270 0x7f3f88167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.201+0000 7f3f8659c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f8815da30 0x7f3f880704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.201+0000 7f3f7ffff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f3f88072d60 0x7f3f8806ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:45676/0 (socket says 172.21.15.139:45676) 2024-04-25T09:54:51.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.201+0000 7f3f7ffff640 1 -- 172.21.15.139:0/2501813077 learned_addr learned my addr 172.21.15.139:0/2501813077 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:51.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.201+0000 7f3f86d9d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3f8806b270 0x7f3f88167550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:56762/0 (socket says 172.21.15.139:56762) 2024-04-25T09:54:51.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.201+0000 7f3f86d9d640 1 -- 172.21.15.139:0/2501813077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f8815da30 msgr2=0x7f3f880704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:51.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.201+0000 7f3f86d9d640 1 --2- 172.21.15.139:0/2501813077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f8815da30 0x7f3f880704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.201+0000 7f3f86d9d640 1 -- 172.21.15.139:0/2501813077 >> v1:172.21.15.150:6789/0 conn(0x7f3f88072d60 legacy=0x7f3f8806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.201+0000 7f3f86d9d640 1 -- 172.21.15.139:0/2501813077 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8815de10 con 0x7f3f8806b270 2024-04-25T09:54:51.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.201+0000 7f3f8659c640 1 --2- 172.21.15.139:0/2501813077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f8815da30 0x7f3f880704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:51.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.202+0000 7f3f86d9d640 1 --2- 172.21.15.139:0/2501813077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f8806b270 0x7f3f88167550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3f880cd6c0 tx=0x7f3f8002a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63f5cf4c2e237ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.202+0000 7f3f85d9b640 1 -- 172.21.15.139:0/2501813077 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8002ae60 con 0x7f3f8806b270 2024-04-25T09:54:51.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.202+0000 7f3f85d9b640 1 -- 172.21.15.139:0/2501813077 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f80007740 con 0x7f3f8806b270 2024-04-25T09:54:51.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.202+0000 7f3f85d9b640 1 -- 172.21.15.139:0/2501813077 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80007b10 con 0x7f3f8806b270 2024-04-25T09:54:51.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.202+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/2501813077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f8806b270 msgr2=0x7f3f88167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.202+0000 7f3f8c9c8640 1 --2- 172.21.15.139:0/2501813077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f8806b270 0x7f3f88167550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3f880cd6c0 tx=0x7f3f8002a640 comp rx=0 tx=0).stop 2024-04-25T09:54:51.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.203+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/2501813077 shutdown_connections 2024-04-25T09:54:51.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.203+0000 7f3f8c9c8640 1 --2- 172.21.15.139:0/2501813077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f8806b270 0x7f3f88167550 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.203+0000 7f3f8c9c8640 1 --2- 172.21.15.139:0/2501813077 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f8815da30 0x7f3f880704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.203+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/2501813077 >> 172.21.15.139:0/2501813077 conn(0x7f3f880b23f0 msgr2=0x7f3f880b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:51.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.203+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/2501813077 shutdown_connections 2024-04-25T09:54:51.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.203+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/2501813077 wait complete. 2024-04-25T09:54:51.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.204+0000 7f3f8c9c8640 1 Processor -- start 2024-04-25T09:54:51.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.204+0000 7f3f8c9c8640 1 -- start start 2024-04-25T09:54:51.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.204+0000 7f3f8c9c8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f8806b270 0x7f3f88160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f8c9c8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f88072d60 0x7f3f880c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f8c9c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f8815da30 0x7f3f88158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f8c9c8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880b3b00 con 0x7f3f8815da30 2024-04-25T09:54:51.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f8c9c8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880b3820 con 0x7f3f88072d60 2024-04-25T09:54:51.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f8c9c8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880b42a0 con 0x7f3f8806b270 2024-04-25T09:54:51.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f86d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f8815da30 0x7f3f88158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f86d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f8815da30 0x7f3f88158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34136/0 (socket says 172.21.15.139:34136) 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f86d9d640 1 -- 172.21.15.139:0/1732319079 learned_addr learned my addr 172.21.15.139:0/1732319079 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f8659c640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f8806b270 0x7f3f88160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f7ffff640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f88072d60 0x7f3f880c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f86d9d640 1 -- 172.21.15.139:0/1732319079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f8806b270 msgr2=0x7f3f88160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f86d9d640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f8806b270 0x7f3f88160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f86d9d640 1 -- 172.21.15.139:0/1732319079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f88072d60 msgr2=0x7f3f880c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f86d9d640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f88072d60 0x7f3f880c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f86d9d640 1 -- 172.21.15.139:0/1732319079 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f88160ec0 con 0x7f3f8815da30 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f86d9d640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f8815da30 0x7f3f88158af0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3f8002ac30 tx=0x7f3f80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f7effd640 1 -- 172.21.15.139:0/1732319079 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8003f070 con 0x7f3f8815da30 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f7effd640 1 -- 172.21.15.139:0/1732319079 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8002ae60 con 0x7f3f8815da30 2024-04-25T09:54:51.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/1732319079 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f880ca370 con 0x7f3f8815da30 2024-04-25T09:54:51.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f7effd640 1 -- 172.21.15.139:0/1732319079 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80031c60 con 0x7f3f8815da30 2024-04-25T09:54:51.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.205+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/1732319079 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f880ca930 con 0x7f3f8815da30 2024-04-25T09:54:51.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.206+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/1732319079 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f880caff0 con 0x7f3f8815da30 2024-04-25T09:54:51.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.207+0000 7f3f7effd640 1 -- 172.21.15.139:0/1732319079 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f8003b410 con 0x7f3f8815da30 2024-04-25T09:54:51.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.207+0000 7f3f7effd640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f6806b4d0 0x7f3f6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.207+0000 7f3f7effd640 1 -- 172.21.15.139:0/1732319079 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3f800ab0c0 con 0x7f3f8815da30 2024-04-25T09:54:51.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.207+0000 7f3f8659c640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f6806b4d0 0x7f3f6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.208+0000 7f3f8659c640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f6806b4d0 0x7f3f6806d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009560 tx=0x7f3f78009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.211+0000 7f3f7effd640 1 -- 172.21.15.139:0/1732319079 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f880caff0 con 0x7f3f8815da30 2024-04-25T09:54:51.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.379+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/1732319079 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3f88074520 con 0x7f3f8815da30 2024-04-25T09:54:51.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.382+0000 7f3f7effd640 1 -- 172.21.15.139:0/1732319079 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3f80079af0 con 0x7f3f8815da30 2024-04-25T09:54:51.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.384+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/1732319079 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f6806b4d0 msgr2=0x7f3f6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.384+0000 7f3f8c9c8640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f6806b4d0 0x7f3f6806d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009560 tx=0x7f3f78009190 comp rx=0 tx=0).stop 2024-04-25T09:54:51.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.384+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/1732319079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f8815da30 msgr2=0x7f3f88158af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.384+0000 7f3f8c9c8640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f8815da30 0x7f3f88158af0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3f8002ac30 tx=0x7f3f80004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:51.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.384+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/1732319079 shutdown_connections 2024-04-25T09:54:51.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.384+0000 7f3f8c9c8640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f8815da30 0x7f3f88158af0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.384+0000 7f3f8c9c8640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3f6806b4d0 0x7f3f6806d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.384+0000 7f3f8c9c8640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f88072d60 0x7f3f880c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.384+0000 7f3f8c9c8640 1 --2- 172.21.15.139:0/1732319079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f8806b270 0x7f3f88160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.384+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/1732319079 >> 172.21.15.139:0/1732319079 conn(0x7f3f880b23f0 msgr2=0x7f3f880b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:51.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.384+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/1732319079 shutdown_connections 2024-04-25T09:54:51.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.384+0000 7f3f8c9c8640 1 -- 172.21.15.139:0/1732319079 wait complete. 2024-04-25T09:54:51.397 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:54:51.398 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:54:51.435 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:54:51.481 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:54:51.481 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:54:51.481 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:51.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:54:51.497 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:54:51.497 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:51.514 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:54:51.514 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:54:51.514 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:51.552 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:54:51.552 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:54:51.552 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:51.571 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:54:51.571 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:54:51.572 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:51.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.685+0000 7f5ffd4ac640 1 Processor -- start 2024-04-25T09:54:51.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.685+0000 7f5ffd4ac640 1 -- start start 2024-04-25T09:54:51.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.686+0000 7f5ffd4ac640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5ff815efb0 0x7f5ff80d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.686+0000 7f5ffd4ac640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5ff80cf900 0x7f5ff806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.686+0000 7f5ffd4ac640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff80cfe70 con 0x7f5ff806cd80 2024-04-25T09:54:51.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.686+0000 7f5ffd4ac640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff80d0a80 con 0x7f5ff80cf900 2024-04-25T09:54:51.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.686+0000 7f5ffd4ac640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff80d0150 con 0x7f5ff815efb0 2024-04-25T09:54:51.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.686+0000 7f5ff67fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5ff80cf900 0x7f5ff806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.686+0000 7f5ff67fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5ff80cf900 0x7f5ff806c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34148/0 (socket says 172.21.15.139:34148) 2024-04-25T09:54:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.686+0000 7f5ff67fc640 1 -- 172.21.15.139:0/1394543582 learned_addr learned my addr 172.21.15.139:0/1394543582 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.687+0000 7f5ff67fc640 1 -- 172.21.15.139:0/1394543582 >> v2:172.21.15.150:3300/0 conn(0x7f5ff815efb0 msgr2=0x7f5ff80d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.687+0000 7f5ff6ffd640 1 --2- 172.21.15.139:0/1394543582 >> v2:172.21.15.150:3300/0 conn(0x7f5ff815efb0 0x7f5ff80d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.687+0000 7f5ff67fc640 1 --2- 172.21.15.139:0/1394543582 >> v2:172.21.15.150:3300/0 conn(0x7f5ff815efb0 0x7f5ff80d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.687+0000 7f5ff67fc640 1 -- 172.21.15.139:0/1394543582 >> v1:172.21.15.150:6789/0 conn(0x7f5ff806cd80 legacy=0x7f5ff8099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.687+0000 7f5ff67fc640 1 -- 172.21.15.139:0/1394543582 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80d0430 con 0x7f5ff80cf900 2024-04-25T09:54:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.687+0000 7f5ff67fc640 1 --2- 172.21.15.139:0/1394543582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff80cf900 0x7f5ff806c840 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8072c10 tx=0x7f5fec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4e3608e846002e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.687+0000 7f5ff5ffb640 1 -- 172.21.15.139:0/1394543582 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec037070 con 0x7f5ff80cf900 2024-04-25T09:54:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.687+0000 7f5ff5ffb640 1 -- 172.21.15.139:0/1394543582 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec032440 con 0x7f5ff80cf900 2024-04-25T09:54:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.687+0000 7f5ff5ffb640 1 -- 172.21.15.139:0/1394543582 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec032810 con 0x7f5ff80cf900 2024-04-25T09:54:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.688+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/1394543582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff80cf900 msgr2=0x7f5ff806c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.688+0000 7f5ffd4ac640 1 --2- 172.21.15.139:0/1394543582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff80cf900 0x7f5ff806c840 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8072c10 tx=0x7f5fec02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.688+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/1394543582 shutdown_connections 2024-04-25T09:54:51.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.688+0000 7f5ffd4ac640 1 --2- 172.21.15.139:0/1394543582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff80cf900 0x7f5ff806c840 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.688+0000 7f5ffd4ac640 1 --2- 172.21.15.139:0/1394543582 >> v2:172.21.15.150:3300/0 conn(0x7f5ff815efb0 0x7f5ff80d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.688+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/1394543582 >> 172.21.15.139:0/1394543582 conn(0x7f5ff80d1790 msgr2=0x7f5ff80d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:51.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.688+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/1394543582 shutdown_connections 2024-04-25T09:54:51.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.688+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/1394543582 wait complete. 2024-04-25T09:54:51.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.689+0000 7f5ffd4ac640 1 Processor -- start 2024-04-25T09:54:51.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.689+0000 7f5ffd4ac640 1 -- start start 2024-04-25T09:54:51.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ffd4ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff806cd80 0x7f5ff80b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ffd4ac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff80cf900 0x7f5ff80b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ffd4ac640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ff815efb0 0x7f5ff8153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ffd4ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff809a2e0 con 0x7f5ff806cd80 2024-04-25T09:54:51.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ffd4ac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff809a160 con 0x7f5ff80cf900 2024-04-25T09:54:51.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ffd4ac640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80cbde0 con 0x7f5ff815efb0 2024-04-25T09:54:51.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ff6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff806cd80 0x7f5ff80b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ff6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff806cd80 0x7f5ff80b7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34160/0 (socket says 172.21.15.139:34160) 2024-04-25T09:54:51.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ff6ffd640 1 -- 172.21.15.139:0/3525099386 learned_addr learned my addr 172.21.15.139:0/3525099386 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:51.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ff77fe640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ff815efb0 0x7f5ff8153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ff67fc640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff80cf900 0x7f5ff80b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ff6ffd640 1 -- 172.21.15.139:0/3525099386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ff815efb0 msgr2=0x7f5ff8153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ff6ffd640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ff815efb0 0x7f5ff8153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ff6ffd640 1 -- 172.21.15.139:0/3525099386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff80cf900 msgr2=0x7f5ff80b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ff6ffd640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff80cf900 0x7f5ff80b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.690+0000 7f5ff6ffd640 1 -- 172.21.15.139:0/3525099386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff8154170 con 0x7f5ff806cd80 2024-04-25T09:54:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.691+0000 7f5ff6ffd640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff806cd80 0x7f5ff80b7120 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800ccd0 tx=0x7f5fe800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.691+0000 7f5fdf7fe640 1 -- 172.21.15.139:0/3525099386 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8003990 con 0x7f5ff806cd80 2024-04-25T09:54:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.691+0000 7f5fdf7fe640 1 -- 172.21.15.139:0/3525099386 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8003b30 con 0x7f5ff806cd80 2024-04-25T09:54:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.691+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/3525099386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff81f4420 con 0x7f5ff806cd80 2024-04-25T09:54:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.691+0000 7f5fdf7fe640 1 -- 172.21.15.139:0/3525099386 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8004020 con 0x7f5ff806cd80 2024-04-25T09:54:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.691+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/3525099386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff81f49e0 con 0x7f5ff806cd80 2024-04-25T09:54:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.692+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/3525099386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff8074520 con 0x7f5ff806cd80 2024-04-25T09:54:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.692+0000 7f5fdf7fe640 1 -- 172.21.15.139:0/3525099386 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fe80041c0 con 0x7f5ff806cd80 2024-04-25T09:54:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.693+0000 7f5fdf7fe640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5fe406b4d0 0x7f5fe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.693+0000 7f5fdf7fe640 1 -- 172.21.15.139:0/3525099386 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5fe808a510 con 0x7f5ff806cd80 2024-04-25T09:54:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.693+0000 7f5ff67fc640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5fe406b4d0 0x7f5fe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.693+0000 7f5ff67fc640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5fe406b4d0 0x7f5fe406d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5fec034000 tx=0x7f5fec02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.697+0000 7f5fdf7fe640 1 -- 172.21.15.139:0/3525099386 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5fe8014030 con 0x7f5ff806cd80 2024-04-25T09:54:51.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.901+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/3525099386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ff80b2600 con 0x7f5ff806cd80 2024-04-25T09:54:51.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.902+0000 7f5fdf7fe640 1 -- 172.21.15.139:0/3525099386 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f5fe80594f0 con 0x7f5ff806cd80 2024-04-25T09:54:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:51.905 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/511562594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":511562594},{"type":"v1","addr":"172.21.15.150:6834","nonce":511562594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5041,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/225357317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":225357317},{"type":"v1","addr":"172.21.15.150:6835","nonce":225357317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5415,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2499407982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2499407982},{"type":"v1","addr":"172.21.15.139:6836","nonce":2499407982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5775,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/534990916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":534990916},{"type":"v1","addr":"172.21.15.139:6837","nonce":534990916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[]} 2024-04-25T09:54:51.905 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 18 2024-04-25T09:54:51.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.904+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/3525099386 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5fe406b4d0 msgr2=0x7f5fe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.904+0000 7f5ffd4ac640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5fe406b4d0 0x7f5fe406d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5fec034000 tx=0x7f5fec02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:51.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.904+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/3525099386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff806cd80 msgr2=0x7f5ff80b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.904+0000 7f5ffd4ac640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff806cd80 0x7f5ff80b7120 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800ccd0 tx=0x7f5fe800a5c0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.904+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/3525099386 shutdown_connections 2024-04-25T09:54:51.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.904+0000 7f5ffd4ac640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5fe406b4d0 0x7f5fe406d990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.904+0000 7f5ffd4ac640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ff815efb0 0x7f5ff8153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.904+0000 7f5ffd4ac640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ff80cf900 0x7f5ff80b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.904+0000 7f5ffd4ac640 1 --2- 172.21.15.139:0/3525099386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ff806cd80 0x7f5ff80b7120 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.904+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/3525099386 >> 172.21.15.139:0/3525099386 conn(0x7f5ff80d1790 msgr2=0x7f5ff815dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:51.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.904+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/3525099386 shutdown_connections 2024-04-25T09:54:51.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:51.904+0000 7f5ffd4ac640 1 -- 172.21.15.139:0/3525099386 wait complete. 2024-04-25T09:54:51.917 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:54:51.918 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:54:51.918 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:54:51.918 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:51.918 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:54:51.918 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:54:51.918 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:54:51.919 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:51.919 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:54:51.919 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:54:51.919 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:54:51.919 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:51.919 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:54:51.919 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:54:51.920 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:54:51.920 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:51.928 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:54:51.928 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:54:51.929 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:54:51.929 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:54:51.931 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:54:51.931 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:54:51.932 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:54:51.932 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:54:51.971 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:54:51.971 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:51.971 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:54:51.972 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:51.973 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:54:51.974 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:51.974 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:54:51.975 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:51.975 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:54:52.043 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T09:54:52.044 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-04-25T09:54:52.051 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-04-25T09:54:52.052 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-04-25T09:54:52.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.075+0000 7f83a87d5640 1 Processor -- start 2024-04-25T09:54:52.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.075+0000 7f83a87d5640 1 -- start start 2024-04-25T09:54:52.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.076+0000 7f83a87d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a015da30 0x7f83a00704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.076+0000 7f83a87d5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a00731a0 con 0x7f83a006b270 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.076+0000 7f83a87d5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a015de10 con 0x7f83a0072d60 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.076+0000 7f83a87d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00cd540 con 0x7f83a015da30 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.076+0000 7f83a6d72640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f83a006b270 0x7f83a0167550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60334/0 (socket says 172.21.15.139:60334) 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.076+0000 7f83a6d72640 1 -- 172.21.15.139:0/3169274897 learned_addr learned my addr 172.21.15.139:0/3169274897 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.077+0000 7f83a6571640 1 --2- 172.21.15.139:0/3169274897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a015da30 0x7f83a00704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.077+0000 7f83a556f640 1 -- 172.21.15.139:0/3169274897 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424152614 0 0) 0x7f83a00731a0 con 0x7f83a006b270 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.077+0000 7f83a556f640 1 -- 172.21.15.139:0/3169274897 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f838c003520 con 0x7f83a006b270 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.077+0000 7f83a556f640 1 -- 172.21.15.139:0/3169274897 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709286507 0 0) 0x7f83a015de10 con 0x7f83a0072d60 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.077+0000 7f83a556f640 1 -- 172.21.15.139:0/3169274897 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83a00731a0 con 0x7f83a0072d60 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.077+0000 7f83a6571640 1 -- 172.21.15.139:0/3169274897 >> v1:172.21.15.150:6789/0 conn(0x7f83a0072d60 legacy=0x7f83a006ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.077+0000 7f83a6571640 1 -- 172.21.15.139:0/3169274897 >> v1:172.21.15.139:6789/0 conn(0x7f83a006b270 legacy=0x7f83a0167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.077+0000 7f83a6571640 1 -- 172.21.15.139:0/3169274897 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a00d1470 con 0x7f83a015da30 2024-04-25T09:54:52.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.078+0000 7f83a6571640 1 --2- 172.21.15.139:0/3169274897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a015da30 0x7f83a00704a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f839c0097d0 tx=0x7f839c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=450d23a62f0b8d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.078+0000 7f83a556f640 1 -- 172.21.15.139:0/3169274897 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c037070 con 0x7f83a015da30 2024-04-25T09:54:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.078+0000 7f83a556f640 1 -- 172.21.15.139:0/3169274897 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c032440 con 0x7f83a015da30 2024-04-25T09:54:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.078+0000 7f83a556f640 1 -- 172.21.15.139:0/3169274897 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c032810 con 0x7f83a015da30 2024-04-25T09:54:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.078+0000 7f83a87d5640 1 -- 172.21.15.139:0/3169274897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a015da30 msgr2=0x7f83a00704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.078+0000 7f83a87d5640 1 --2- 172.21.15.139:0/3169274897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a015da30 0x7f83a00704a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f839c0097d0 tx=0x7f839c02a210 comp rx=0 tx=0).stop 2024-04-25T09:54:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.079+0000 7f83a87d5640 1 -- 172.21.15.139:0/3169274897 shutdown_connections 2024-04-25T09:54:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.079+0000 7f83a87d5640 1 --2- 172.21.15.139:0/3169274897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a015da30 0x7f83a00704a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.079+0000 7f83a87d5640 1 -- 172.21.15.139:0/3169274897 >> 172.21.15.139:0/3169274897 conn(0x7f83a00b23f0 msgr2=0x7f83a00b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.079+0000 7f83a87d5640 1 -- 172.21.15.139:0/3169274897 shutdown_connections 2024-04-25T09:54:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.079+0000 7f83a87d5640 1 -- 172.21.15.139:0/3169274897 wait complete. 2024-04-25T09:54:52.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.080+0000 7f83a87d5640 1 Processor -- start 2024-04-25T09:54:52.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.080+0000 7f83a87d5640 1 -- start start 2024-04-25T09:54:52.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.080+0000 7f83a87d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a006b270 0x7f83a014c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.080+0000 7f83a87d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0072d60 0x7f83a0160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.080+0000 7f83a87d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a015da30 0x7f83a006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.080+0000 7f83a87d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00b3b00 con 0x7f83a015da30 2024-04-25T09:54:52.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.080+0000 7f83a87d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00b3de0 con 0x7f83a0072d60 2024-04-25T09:54:52.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.080+0000 7f83a87d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00b42a0 con 0x7f83a006b270 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a6d72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a015da30 0x7f83a006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a6d72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a015da30 0x7f83a006fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34184/0 (socket says 172.21.15.139:34184) 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a6d72640 1 -- 172.21.15.139:0/2970138446 learned_addr learned my addr 172.21.15.139:0/2970138446 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a6571640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a006b270 0x7f83a014c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a5d70640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0072d60 0x7f83a0160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a6d72640 1 -- 172.21.15.139:0/2970138446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a006b270 msgr2=0x7f83a014c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a6d72640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a006b270 0x7f83a014c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a6d72640 1 -- 172.21.15.139:0/2970138446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0072d60 msgr2=0x7f83a0160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a6d72640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0072d60 0x7f83a0160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a6d72640 1 -- 172.21.15.139:0/2970138446 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a00afb20 con 0x7f83a015da30 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a6d72640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a015da30 0x7f83a006fa90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f839800caf0 tx=0x7f839800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f8396ffd640 1 -- 172.21.15.139:0/2970138446 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398017070 con 0x7f83a015da30 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f8396ffd640 1 -- 172.21.15.139:0/2970138446 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839800aca0 con 0x7f83a015da30 2024-04-25T09:54:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f8396ffd640 1 -- 172.21.15.139:0/2970138446 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398012620 con 0x7f83a015da30 2024-04-25T09:54:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a87d5640 1 -- 172.21.15.139:0/2970138446 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a015a6d0 con 0x7f83a015da30 2024-04-25T09:54:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.081+0000 7f83a87d5640 1 -- 172.21.15.139:0/2970138446 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a015f930 con 0x7f83a015da30 2024-04-25T09:54:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.082+0000 7f8394ff9640 1 -- 172.21.15.139:0/2970138446 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837c005290 con 0x7f83a015da30 2024-04-25T09:54:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.083+0000 7f8396ffd640 1 -- 172.21.15.139:0/2970138446 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f839801a070 con 0x7f83a015da30 2024-04-25T09:54:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.083+0000 7f8396ffd640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f838406b500 0x7f838406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.083+0000 7f83a6571640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f838406b500 0x7f838406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.083+0000 7f8396ffd640 1 -- 172.21.15.139:0/2970138446 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8398089d20 con 0x7f83a015da30 2024-04-25T09:54:52.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.084+0000 7f83a6571640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f838406b500 0x7f838406d9c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f839c034000 tx=0x7f839c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.087+0000 7f8396ffd640 1 -- 172.21.15.139:0/2970138446 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8398058da0 con 0x7f83a015da30 2024-04-25T09:54:52.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:52.269+0000 7f8394ff9640 1 -- 172.21.15.139:0/2970138446 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f837c005850 con 0x7f83a015da30 2024-04-25T09:54:53.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.101+0000 7f8396ffd640 1 -- 172.21.15.139:0/2970138446 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) v1 ==== 97+0+0 (secure 0 0 0) 0x7f83980586a0 con 0x7f83a015da30 2024-04-25T09:54:53.103 INFO:teuthology.orchestra.run.smithi139.stderr: removed all blocklist entries 2024-04-25T09:54:53.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.103+0000 7f8394ff9640 1 -- 172.21.15.139:0/2970138446 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f838406b500 msgr2=0x7f838406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.103+0000 7f8394ff9640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f838406b500 0x7f838406d9c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f839c034000 tx=0x7f839c02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:53.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.103+0000 7f8394ff9640 1 -- 172.21.15.139:0/2970138446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a015da30 msgr2=0x7f83a006fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.103+0000 7f8394ff9640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a015da30 0x7f83a006fa90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f839800caf0 tx=0x7f839800a060 comp rx=0 tx=0).stop 2024-04-25T09:54:53.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.103+0000 7f8394ff9640 1 -- 172.21.15.139:0/2970138446 shutdown_connections 2024-04-25T09:54:53.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.103+0000 7f8394ff9640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83a015da30 0x7f83a006fa90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.103+0000 7f8394ff9640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f838406b500 0x7f838406d9c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.103+0000 7f8394ff9640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0072d60 0x7f83a0160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.103+0000 7f8394ff9640 1 --2- 172.21.15.139:0/2970138446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a006b270 0x7f83a014c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.103+0000 7f8394ff9640 1 -- 172.21.15.139:0/2970138446 >> 172.21.15.139:0/2970138446 conn(0x7f83a00b23f0 msgr2=0x7f83a00cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:53.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.104+0000 7f8394ff9640 1 -- 172.21.15.139:0/2970138446 shutdown_connections 2024-04-25T09:54:53.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.104+0000 7f8394ff9640 1 -- 172.21.15.139:0/2970138446 wait complete. 2024-04-25T09:54:53.119 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:54:53.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.218+0000 7fc0371d5640 1 Processor -- start 2024-04-25T09:54:53.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.218+0000 7fc0371d5640 1 -- start start 2024-04-25T09:54:53.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.218+0000 7fc0371d5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc03015dc60 0x7fc0300b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.218+0000 7fc0371d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc030161850 0x7fc0300ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.218+0000 7fc0371d5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0300cf250 con 0x7fc0300d0970 2024-04-25T09:54:53.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.218+0000 7fc0371d5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0300ce780 con 0x7fc03015dc60 2024-04-25T09:54:53.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.218+0000 7fc0371d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0300d3730 con 0x7fc030161850 2024-04-25T09:54:53.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.219+0000 7fc034f71640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc03015dc60 0x7fc0300b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.219+0000 7fc034f71640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc03015dc60 0x7fc0300b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34188/0 (socket says 172.21.15.139:34188) 2024-04-25T09:54:53.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.219+0000 7fc034f71640 1 -- 172.21.15.139:0/538974702 learned_addr learned my addr 172.21.15.139:0/538974702 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.219+0000 7fc035772640 1 --1- 172.21.15.139:0/538974702 >> v1:172.21.15.139:6789/0 conn(0x7fc0300d0970 0x7fc0300d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60338/0 (socket says 172.21.15.139:60338) 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.219+0000 7fc02ffff640 1 --2- 172.21.15.139:0/538974702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc030161850 0x7fc0300ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.219+0000 7fc034f71640 1 -- 172.21.15.139:0/538974702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc030161850 msgr2=0x7fc0300ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.219+0000 7fc034f71640 1 --2- 172.21.15.139:0/538974702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc030161850 0x7fc0300ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.219+0000 7fc034f71640 1 -- 172.21.15.139:0/538974702 >> v1:172.21.15.139:6789/0 conn(0x7fc0300d0970 legacy=0x7fc0300d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.219+0000 7fc034f71640 1 -- 172.21.15.139:0/538974702 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0300ce9b0 con 0x7fc03015dc60 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.219+0000 7fc02f7fe640 1 -- 172.21.15.139:0/538974702 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483266532 0 0) 0x7fc028001490 con 0x7fc0300d0970 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.220+0000 7fc034f71640 1 --2- 172.21.15.139:0/538974702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03015dc60 0x7fc0300b6c10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc0240098a0 tx=0x7fc02402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe411c085057110 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.220+0000 7fc02f7fe640 1 -- 172.21.15.139:0/538974702 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024037070 con 0x7fc03015dc60 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.220+0000 7fc02f7fe640 1 -- 172.21.15.139:0/538974702 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc024032440 con 0x7fc03015dc60 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.220+0000 7fc02f7fe640 1 -- 172.21.15.139:0/538974702 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024032810 con 0x7fc03015dc60 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.220+0000 7fc0371d5640 1 -- 172.21.15.139:0/538974702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03015dc60 msgr2=0x7fc0300b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.220+0000 7fc0371d5640 1 --2- 172.21.15.139:0/538974702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03015dc60 0x7fc0300b6c10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc0240098a0 tx=0x7fc02402a290 comp rx=0 tx=0).stop 2024-04-25T09:54:53.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.220+0000 7fc0371d5640 1 -- 172.21.15.139:0/538974702 shutdown_connections 2024-04-25T09:54:53.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.220+0000 7fc0371d5640 1 --2- 172.21.15.139:0/538974702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc030161850 0x7fc0300ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.220+0000 7fc0371d5640 1 --2- 172.21.15.139:0/538974702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03015dc60 0x7fc0300b6c10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.220+0000 7fc0371d5640 1 -- 172.21.15.139:0/538974702 >> 172.21.15.139:0/538974702 conn(0x7fc0300b5910 msgr2=0x7fc0300b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:53.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.220+0000 7fc0371d5640 1 -- 172.21.15.139:0/538974702 shutdown_connections 2024-04-25T09:54:53.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.220+0000 7fc0371d5640 1 -- 172.21.15.139:0/538974702 wait complete. 2024-04-25T09:54:53.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.221+0000 7fc0371d5640 1 Processor -- start 2024-04-25T09:54:53.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.222+0000 7fc0371d5640 1 -- start start 2024-04-25T09:54:53.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.222+0000 7fc0371d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0300d0970 0x7fc030154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.222+0000 7fc0371d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03015dc60 0x7fc0300c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.222+0000 7fc0371d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc030161850 0x7fc0300c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.222+0000 7fc0371d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0300d1fc0 con 0x7fc03015dc60 2024-04-25T09:54:53.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.222+0000 7fc0371d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0300cef70 con 0x7fc0300d0970 2024-04-25T09:54:53.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.222+0000 7fc0371d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03015e040 con 0x7fc030161850 2024-04-25T09:54:53.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc02ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03015dc60 0x7fc0300c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc02ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03015dc60 0x7fc0300c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34204/0 (socket says 172.21.15.139:34204) 2024-04-25T09:54:53.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc02ffff640 1 -- 172.21.15.139:0/1725779715 learned_addr learned my addr 172.21.15.139:0/1725779715 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc034f71640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0300d0970 0x7fc030154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc035772640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc030161850 0x7fc0300c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc02ffff640 1 -- 172.21.15.139:0/1725779715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc030161850 msgr2=0x7fc0300c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc02ffff640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc030161850 0x7fc0300c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc02ffff640 1 -- 172.21.15.139:0/1725779715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0300d0970 msgr2=0x7fc030154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc02ffff640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0300d0970 0x7fc030154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc02ffff640 1 -- 172.21.15.139:0/1725779715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0301553b0 con 0x7fc03015dc60 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc02ffff640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03015dc60 0x7fc0300c8690 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00d620 tx=0x7fc01c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc02d7fa640 1 -- 172.21.15.139:0/1725779715 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00b620 con 0x7fc03015dc60 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc02d7fa640 1 -- 172.21.15.139:0/1725779715 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c019070 con 0x7fc03015dc60 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc0371d5640 1 -- 172.21.15.139:0/1725779715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0301504e0 con 0x7fc03015dc60 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc02d7fa640 1 -- 172.21.15.139:0/1725779715 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00ba00 con 0x7fc03015dc60 2024-04-25T09:54:53.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.223+0000 7fc0371d5640 1 -- 172.21.15.139:0/1725779715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc030150aa0 con 0x7fc03015dc60 2024-04-25T09:54:53.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.224+0000 7fc0371d5640 1 -- 172.21.15.139:0/1725779715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc030074520 con 0x7fc03015dc60 2024-04-25T09:54:53.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.224+0000 7fc02d7fa640 1 -- 172.21.15.139:0/1725779715 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc01c00bba0 con 0x7fc03015dc60 2024-04-25T09:54:53.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.225+0000 7fc02d7fa640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc01406b4d0 0x7fc01406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.225+0000 7fc02d7fa640 1 -- 172.21.15.139:0/1725779715 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc01c08a380 con 0x7fc03015dc60 2024-04-25T09:54:53.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.225+0000 7fc034f71640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc01406b4d0 0x7fc01406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.225+0000 7fc034f71640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc01406b4d0 0x7fc01406d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc024034000 tx=0x7fc02402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.229+0000 7fc02d7fa640 1 -- 172.21.15.139:0/1725779715 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc01c059450 con 0x7fc03015dc60 2024-04-25T09:54:53.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.463+0000 7fc0371d5640 1 -- 172.21.15.139:0/1725779715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc0300d16f0 con 0x7fc03015dc60 2024-04-25T09:54:53.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.463+0000 7fc02d7fa640 1 -- 172.21.15.139:0/1725779715 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc01c058d50 con 0x7fc03015dc60 2024-04-25T09:54:53.466 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:53.466 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:54:53.466 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_dump": [ 2024-04-25T09:54:53.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.466 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.a", 2024-04-25T09:54:53.466 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypm0xdaHhAA6nupxCTQhucepW2iEWNUoQ==", 2024-04-25T09:54:53.466 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.466 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.b", 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmB9GDKRAA/lmU4OiKO5dFgayYx09GQg==", 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.c", 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmvYgaJRAA/YIE6ralp2WuTK993D7H0Q==", 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.d", 2024-04-25T09:54:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmg49WMBAAlPY9ZSeOoV296xSdl/m1gg==", 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.0", 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDVJypmG7TpDhAAo9CKuZBw8BsQgB2F+BR5gw==", 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.1", 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDYJypmgR62DBAAotfnvS47WDunP9Ic03gA6Q==", 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.2", 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDbJypmzyqGChAA6wrfIdqpiLm3RO36oorihQ==", 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.3", 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDeJypmt72cCBAAgcLrFC3MUU7wlFySAfLWig==", 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.4", 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDkJypmxJLpDBAAvROZ9AvYdWMqOALEKIAFCw==", 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.5", 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDnJypmTPXEChAA20BghY39PGw6zFyu9yWbBQ==", 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.6", 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDqJypmaBoYCRAAy9eHgUI/WTkE2XdT1RKLsw==", 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.7", 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDtJypmViJlCBAANJSDWVHVbWR03Y1SMDH++Q==", 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.0", 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg==", 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow r", 2024-04-25T09:54:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.1", 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypmOncYARAA6gYjZbRDkhqFX0mpSDbH5Q==", 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.2", 2024-04-25T09:54:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmQs2wORAArzyV70nGDX4486LlFjM70Q==", 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.3", 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypm90x9BhAA41rZPs0O5mwmuHy1bkPCvw==", 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:54:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.admin", 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmOWQcAxAA/ZVXEHYJQZf9UtgZ5g25Ag==", 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.x", 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmt+RVGxAAi1djVoqnoYCDMzJ9A0dmNA==", 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.y", 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmuJ4bFxAAw6a53AgThU/ut+Mk5j0gqQ==", 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T09:54:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:54:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-04-25T09:54:53.479 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:54:53.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.465+0000 7fc0371d5640 1 -- 172.21.15.139:0/1725779715 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc01406b4d0 msgr2=0x7fc01406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.465+0000 7fc0371d5640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc01406b4d0 0x7fc01406d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc024034000 tx=0x7fc02402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:53.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.465+0000 7fc0371d5640 1 -- 172.21.15.139:0/1725779715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03015dc60 msgr2=0x7fc0300c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.465+0000 7fc0371d5640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03015dc60 0x7fc0300c8690 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00d620 tx=0x7fc01c00dae0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.465+0000 7fc0371d5640 1 -- 172.21.15.139:0/1725779715 shutdown_connections 2024-04-25T09:54:53.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.465+0000 7fc0371d5640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc03015dc60 0x7fc0300c8690 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.465+0000 7fc0371d5640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc01406b4d0 0x7fc01406d990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.465+0000 7fc0371d5640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc030161850 0x7fc0300c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.465+0000 7fc0371d5640 1 --2- 172.21.15.139:0/1725779715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0300d0970 0x7fc030154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.465+0000 7fc0371d5640 1 -- 172.21.15.139:0/1725779715 >> 172.21.15.139:0/1725779715 conn(0x7fc0300b5910 msgr2=0x7fc03015f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:53.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.465+0000 7fc0371d5640 1 -- 172.21.15.139:0/1725779715 shutdown_connections 2024-04-25T09:54:53.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.466+0000 7fc0371d5640 1 -- 172.21.15.139:0/1725779715 wait complete. 2024-04-25T09:54:53.481 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:53.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.576+0000 7f57efff9640 1 Processor -- start 2024-04-25T09:54:53.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.576+0000 7f57efff9640 1 -- start start 2024-04-25T09:54:53.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57efff9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f57e815efb0 0x7f57e80d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57efff9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f57e80cf900 0x7f57e806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57efff9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e80cfe70 con 0x7f57e806cd80 2024-04-25T09:54:53.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57efff9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e80d0a80 con 0x7f57e80cf900 2024-04-25T09:54:53.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57efff9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e80d0150 con 0x7f57e815efb0 2024-04-25T09:54:53.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57ed594640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f57e80cf900 0x7f57e806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57ee596640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f57e806cd80 0x7f57e8099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60340/0 (socket says 172.21.15.139:60340) 2024-04-25T09:54:53.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57ed594640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f57e80cf900 0x7f57e806c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34216/0 (socket says 172.21.15.139:34216) 2024-04-25T09:54:53.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57ee596640 1 -- 172.21.15.139:0/898985145 learned_addr learned my addr 172.21.15.139:0/898985145 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:53.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57ed594640 1 -- 172.21.15.139:0/898985145 >> v2:172.21.15.150:3300/0 conn(0x7f57e815efb0 msgr2=0x7f57e80d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:53.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57ed594640 1 --2- 172.21.15.139:0/898985145 >> v2:172.21.15.150:3300/0 conn(0x7f57e815efb0 0x7f57e80d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57ed594640 1 -- 172.21.15.139:0/898985145 >> v1:172.21.15.139:6789/0 conn(0x7f57e806cd80 legacy=0x7f57e8099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57ed594640 1 -- 172.21.15.139:0/898985145 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e80d0430 con 0x7f57e80cf900 2024-04-25T09:54:53.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.577+0000 7f57ecd93640 1 -- 172.21.15.139:0/898985145 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304769701 0 0) 0x7f57e80cfe70 con 0x7f57e806cd80 2024-04-25T09:54:53.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.578+0000 7f57ed594640 1 --2- 172.21.15.139:0/898985145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57e80cf900 0x7f57e806c840 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f57e8072c10 tx=0x7f57d802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa60537baacb07f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.578+0000 7f57ecd93640 1 -- 172.21.15.139:0/898985145 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8037070 con 0x7f57e80cf900 2024-04-25T09:54:53.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.578+0000 7f57ecd93640 1 -- 172.21.15.139:0/898985145 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d8032440 con 0x7f57e80cf900 2024-04-25T09:54:53.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.578+0000 7f57ecd93640 1 -- 172.21.15.139:0/898985145 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d802ae60 con 0x7f57e80cf900 2024-04-25T09:54:53.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.578+0000 7f57efff9640 1 -- 172.21.15.139:0/898985145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57e80cf900 msgr2=0x7f57e806c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.578+0000 7f57efff9640 1 --2- 172.21.15.139:0/898985145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57e80cf900 0x7f57e806c840 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f57e8072c10 tx=0x7f57d802a110 comp rx=0 tx=0).stop 2024-04-25T09:54:53.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.579+0000 7f57efff9640 1 -- 172.21.15.139:0/898985145 shutdown_connections 2024-04-25T09:54:53.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.579+0000 7f57efff9640 1 --2- 172.21.15.139:0/898985145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57e80cf900 0x7f57e806c840 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.579+0000 7f57efff9640 1 --2- 172.21.15.139:0/898985145 >> v2:172.21.15.150:3300/0 conn(0x7f57e815efb0 0x7f57e80d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.579+0000 7f57efff9640 1 -- 172.21.15.139:0/898985145 >> 172.21.15.139:0/898985145 conn(0x7f57e80d1790 msgr2=0x7f57e80d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:53.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.579+0000 7f57efff9640 1 -- 172.21.15.139:0/898985145 shutdown_connections 2024-04-25T09:54:53.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.579+0000 7f57efff9640 1 -- 172.21.15.139:0/898985145 wait complete. 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.580+0000 7f57efff9640 1 Processor -- start 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.580+0000 7f57efff9640 1 -- start start 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.580+0000 7f57efff9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e806cd80 0x7f57e8161e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.580+0000 7f57efff9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57e80cf900 0x7f57e815fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.580+0000 7f57efff9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57e815efb0 0x7f57e80c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.580+0000 7f57efff9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e809a2e0 con 0x7f57e80cf900 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.580+0000 7f57efff9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e809a160 con 0x7f57e806cd80 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.580+0000 7f57efff9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80cbde0 con 0x7f57e815efb0 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57ed594640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57e80cf900 0x7f57e815fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57ed594640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57e80cf900 0x7f57e815fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34228/0 (socket says 172.21.15.139:34228) 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57ed594640 1 -- 172.21.15.139:0/817479053 learned_addr learned my addr 172.21.15.139:0/817479053 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57edd95640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e806cd80 0x7f57e8161e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57ed594640 1 -- 172.21.15.139:0/817479053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57e815efb0 msgr2=0x7f57e80c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57ed594640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57e815efb0 0x7f57e80c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57ed594640 1 -- 172.21.15.139:0/817479053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e806cd80 msgr2=0x7f57e8161e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57ed594640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e806cd80 0x7f57e8161e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57ed594640 1 -- 172.21.15.139:0/817479053 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e81602e0 con 0x7f57e80cf900 2024-04-25T09:54:53.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57edd95640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e806cd80 0x7f57e8161e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:53.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57ed594640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57e80cf900 0x7f57e815fda0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009890 tx=0x7f57d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57de7fc640 1 -- 172.21.15.139:0/817479053 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8041070 con 0x7f57e80cf900 2024-04-25T09:54:53.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57de7fc640 1 -- 172.21.15.139:0/817479053 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d803c890 con 0x7f57e80cf900 2024-04-25T09:54:53.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57efff9640 1 -- 172.21.15.139:0/817479053 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e8160580 con 0x7f57e80cf900 2024-04-25T09:54:53.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57de7fc640 1 -- 172.21.15.139:0/817479053 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8037040 con 0x7f57e80cf900 2024-04-25T09:54:53.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.581+0000 7f57efff9640 1 -- 172.21.15.139:0/817479053 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e8160b40 con 0x7f57e80cf900 2024-04-25T09:54:53.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.582+0000 7f57efff9640 1 -- 172.21.15.139:0/817479053 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e80b4650 con 0x7f57e80cf900 2024-04-25T09:54:53.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.583+0000 7f57de7fc640 1 -- 172.21.15.139:0/817479053 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57d8044070 con 0x7f57e80cf900 2024-04-25T09:54:53.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.583+0000 7f57de7fc640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f57cc06b500 0x7f57cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.583+0000 7f57de7fc640 1 -- 172.21.15.139:0/817479053 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f57d80aac40 con 0x7f57e80cf900 2024-04-25T09:54:53.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.583+0000 7f57edd95640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f57cc06b500 0x7f57cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.583+0000 7f57edd95640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f57cc06b500 0x7f57cc06d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f57e80c7b90 tx=0x7f57e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.587+0000 7f57de7fc640 1 -- 172.21.15.139:0/817479053 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57d8079e30 con 0x7f57e80cf900 2024-04-25T09:54:53.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.759+0000 7f57efff9640 1 -- 172.21.15.139:0/817479053 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57e8074520 con 0x7f57e80cf900 2024-04-25T09:54:53.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.760+0000 7f57de7fc640 1 -- 172.21.15.139:0/817479053 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+13212 (secure 0 0 0) 0x7f57d8079730 con 0x7f57e80cf900 2024-04-25T09:54:53.763 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:53.763 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":46,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:53.100497+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:53.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.762+0000 7f57efff9640 1 -- 172.21.15.139:0/817479053 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f57cc06b500 msgr2=0x7f57cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.762+0000 7f57efff9640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f57cc06b500 0x7f57cc06d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f57e80c7b90 tx=0x7f57e4008040 comp rx=0 tx=0).stop 2024-04-25T09:54:53.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.762+0000 7f57efff9640 1 -- 172.21.15.139:0/817479053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57e80cf900 msgr2=0x7f57e815fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.762+0000 7f57efff9640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57e80cf900 0x7f57e815fda0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009890 tx=0x7f57d80046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.762+0000 7f57efff9640 1 -- 172.21.15.139:0/817479053 shutdown_connections 2024-04-25T09:54:53.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.762+0000 7f57efff9640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57e80cf900 0x7f57e815fda0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.762+0000 7f57efff9640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f57cc06b500 0x7f57cc06d9c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.762+0000 7f57efff9640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57e815efb0 0x7f57e80c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.762+0000 7f57efff9640 1 --2- 172.21.15.139:0/817479053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e806cd80 0x7f57e8161e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.762+0000 7f57efff9640 1 -- 172.21.15.139:0/817479053 >> 172.21.15.139:0/817479053 conn(0x7f57e80d1790 msgr2=0x7f57e806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:53.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.762+0000 7f57efff9640 1 -- 172.21.15.139:0/817479053 shutdown_connections 2024-04-25T09:54:53.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.762+0000 7f57efff9640 1 -- 172.21.15.139:0/817479053 wait complete. 2024-04-25T09:54:53.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:54:53.776 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:53.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.873+0000 7f00a0488640 1 Processor -- start 2024-04-25T09:54:53.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.873+0000 7f00a0488640 1 -- start start 2024-04-25T09:54:53.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f00a0488640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00980cd610 0x7f00980d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f00a0488640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f009809ad40 0x7f00980d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f00a0488640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f00981618c0 0x7f0098161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f00a0488640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0098099a50 con 0x7f009809ad40 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f00a0488640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0098099ca0 con 0x7f00981618c0 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f00a0488640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0098099f80 con 0x7f00980cd610 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f009da23640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f009809ad40 0x7f00980d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f009da23640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f009809ad40 0x7f00980d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34238/0 (socket says 172.21.15.139:34238) 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f009da23640 1 -- 172.21.15.139:0/3673421642 learned_addr learned my addr 172.21.15.139:0/3673421642 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f009ea25640 1 --2- 172.21.15.139:0/3673421642 >> v2:172.21.15.150:3300/0 conn(0x7f00981618c0 0x7f0098161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f009da23640 1 -- 172.21.15.139:0/3673421642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00980cd610 msgr2=0x7f00980d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f009da23640 1 --2- 172.21.15.139:0/3673421642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00980cd610 0x7f00980d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f009da23640 1 -- 172.21.15.139:0/3673421642 >> v2:172.21.15.150:3300/0 conn(0x7f00981618c0 msgr2=0x7f0098161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:53.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f009da23640 1 --2- 172.21.15.139:0/3673421642 >> v2:172.21.15.150:3300/0 conn(0x7f00981618c0 0x7f0098161d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f009da23640 1 -- 172.21.15.139:0/3673421642 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009809a260 con 0x7f009809ad40 2024-04-25T09:54:53.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.874+0000 7f009ea25640 1 --2- 172.21.15.139:0/3673421642 >> v2:172.21.15.150:3300/0 conn(0x7f00981618c0 0x7f0098161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:53.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.875+0000 7f009da23640 1 --2- 172.21.15.139:0/3673421642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f009809ad40 0x7f00980d08b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f00980c8760 tx=0x7f008802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75b75f48dcfa391 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.875+0000 7f009d222640 1 -- 172.21.15.139:0/3673421642 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088037070 con 0x7f009809ad40 2024-04-25T09:54:53.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.875+0000 7f009d222640 1 -- 172.21.15.139:0/3673421642 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0088032440 con 0x7f009809ad40 2024-04-25T09:54:53.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.875+0000 7f009d222640 1 -- 172.21.15.139:0/3673421642 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088032810 con 0x7f009809ad40 2024-04-25T09:54:53.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.875+0000 7f00a0488640 1 -- 172.21.15.139:0/3673421642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f009809ad40 msgr2=0x7f00980d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.875+0000 7f00a0488640 1 --2- 172.21.15.139:0/3673421642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f009809ad40 0x7f00980d08b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f00980c8760 tx=0x7f008802a270 comp rx=0 tx=0).stop 2024-04-25T09:54:53.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.875+0000 7f00a0488640 1 -- 172.21.15.139:0/3673421642 shutdown_connections 2024-04-25T09:54:53.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.875+0000 7f00a0488640 1 --2- 172.21.15.139:0/3673421642 >> v2:172.21.15.150:3300/0 conn(0x7f00981618c0 0x7f0098161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.875+0000 7f00a0488640 1 --2- 172.21.15.139:0/3673421642 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f009809ad40 0x7f00980d08b0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.875+0000 7f00a0488640 1 --2- 172.21.15.139:0/3673421642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00980cd610 0x7f00980d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.875+0000 7f00a0488640 1 -- 172.21.15.139:0/3673421642 >> 172.21.15.139:0/3673421642 conn(0x7f00980b31f0 msgr2=0x7f00980b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:53.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.876+0000 7f00a0488640 1 -- 172.21.15.139:0/3673421642 shutdown_connections 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.876+0000 7f00a0488640 1 -- 172.21.15.139:0/3673421642 wait complete. 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.877+0000 7f00a0488640 1 Processor -- start 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.877+0000 7f00a0488640 1 -- start start 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.877+0000 7f00a0488640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f009809ad40 0x7f009815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.877+0000 7f00a0488640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00980cd610 0x7f00980b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.877+0000 7f00a0488640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00981618c0 0x7f00981f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.877+0000 7f00a0488640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00980cd9f0 con 0x7f00980cd610 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.877+0000 7f00a0488640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009809a820 con 0x7f00981618c0 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.877+0000 7f00a0488640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00980cdb70 con 0x7f009809ad40 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.877+0000 7f009da23640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00980cd610 0x7f00980b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.877+0000 7f009da23640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00980cd610 0x7f00980b6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34254/0 (socket says 172.21.15.139:34254) 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.877+0000 7f009da23640 1 -- 172.21.15.139:0/632120701 learned_addr learned my addr 172.21.15.139:0/632120701 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.878+0000 7f009da23640 1 -- 172.21.15.139:0/632120701 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f009809ad40 msgr2=0x7f009815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.878+0000 7f009da23640 1 --2- 172.21.15.139:0/632120701 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f009809ad40 0x7f009815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.878+0000 7f009da23640 1 -- 172.21.15.139:0/632120701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00981618c0 msgr2=0x7f00981f7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.878+0000 7f009da23640 1 --2- 172.21.15.139:0/632120701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00981618c0 0x7f00981f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.878+0000 7f009da23640 1 -- 172.21.15.139:0/632120701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009815b350 con 0x7f00980cd610 2024-04-25T09:54:53.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.878+0000 7f009da23640 1 --2- 172.21.15.139:0/632120701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00980cd610 0x7f00980b6aa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0088000c00 tx=0x7f00880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.878+0000 7f008effd640 1 -- 172.21.15.139:0/632120701 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088041020 con 0x7f00980cd610 2024-04-25T09:54:53.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.878+0000 7f008effd640 1 -- 172.21.15.139:0/632120701 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0088030970 con 0x7f00980cd610 2024-04-25T09:54:53.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.878+0000 7f008effd640 1 -- 172.21.15.139:0/632120701 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088037040 con 0x7f00980cd610 2024-04-25T09:54:53.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.878+0000 7f00a0488640 1 -- 172.21.15.139:0/632120701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009815b630 con 0x7f00980cd610 2024-04-25T09:54:53.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.878+0000 7f00a0488640 1 -- 172.21.15.139:0/632120701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009815bbf0 con 0x7f00980cd610 2024-04-25T09:54:53.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.879+0000 7f00a0488640 1 -- 172.21.15.139:0/632120701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0098074520 con 0x7f00980cd610 2024-04-25T09:54:53.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.879+0000 7f008effd640 1 -- 172.21.15.139:0/632120701 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0088044020 con 0x7f00980cd610 2024-04-25T09:54:53.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.880+0000 7f008effd640 1 --2- 172.21.15.139:0/632120701 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f007c06b4d0 0x7f007c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.880+0000 7f008effd640 1 -- 172.21.15.139:0/632120701 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f00880aa9b0 con 0x7f00980cd610 2024-04-25T09:54:53.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.880+0000 7f009e224640 1 --2- 172.21.15.139:0/632120701 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f007c06b4d0 0x7f007c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.880+0000 7f009e224640 1 --2- 172.21.15.139:0/632120701 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f007c06b4d0 0x7f007c06d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0094009710 tx=0x7f0094006c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:53.884+0000 7f008effd640 1 -- 172.21.15.139:0/632120701 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f008804b030 con 0x7f00980cd610 2024-04-25T09:54:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.053+0000 7f00a0488640 1 -- 172.21.15.139:0/632120701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00980ae680 con 0x7f00980cd610 2024-04-25T09:54:54.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.053+0000 7f008effd640 1 -- 172.21.15.139:0/632120701 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0088079a80 con 0x7f00980cd610 2024-04-25T09:54:54.056 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:54:54.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.055+0000 7f00a0488640 1 -- 172.21.15.139:0/632120701 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f007c06b4d0 msgr2=0x7f007c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.055+0000 7f00a0488640 1 --2- 172.21.15.139:0/632120701 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f007c06b4d0 0x7f007c06d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0094009710 tx=0x7f0094006c90 comp rx=0 tx=0).stop 2024-04-25T09:54:54.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.055+0000 7f00a0488640 1 -- 172.21.15.139:0/632120701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00980cd610 msgr2=0x7f00980b6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.055+0000 7f00a0488640 1 --2- 172.21.15.139:0/632120701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00980cd610 0x7f00980b6aa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0088000c00 tx=0x7f00880046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.055+0000 7f00a0488640 1 -- 172.21.15.139:0/632120701 shutdown_connections 2024-04-25T09:54:54.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.055+0000 7f00a0488640 1 --2- 172.21.15.139:0/632120701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00980cd610 0x7f00980b6aa0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.055+0000 7f00a0488640 1 --2- 172.21.15.139:0/632120701 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f007c06b4d0 0x7f007c06d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.055+0000 7f00a0488640 1 --2- 172.21.15.139:0/632120701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00981618c0 0x7f00981f7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.055+0000 7f00a0488640 1 --2- 172.21.15.139:0/632120701 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f009809ad40 0x7f009815ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.055+0000 7f00a0488640 1 -- 172.21.15.139:0/632120701 >> 172.21.15.139:0/632120701 conn(0x7f00980b31f0 msgr2=0x7f009815fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:54.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.055+0000 7f00a0488640 1 -- 172.21.15.139:0/632120701 shutdown_connections 2024-04-25T09:54:54.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.055+0000 7f00a0488640 1 -- 172.21.15.139:0/632120701 wait complete. 2024-04-25T09:54:54.069 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:54:54.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.163+0000 7fa7a4e7c640 1 Processor -- start 2024-04-25T09:54:54.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.163+0000 7fa7a4e7c640 1 -- start start 2024-04-25T09:54:54.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.164+0000 7fa7a4e7c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa7a015a860 0x7fa7a00cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.164+0000 7fa7a4e7c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7a006ea20 0x7fa7a00b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.164+0000 7fa7a4e7c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a00af6d0 con 0x7fa7a00cd1f0 2024-04-25T09:54:54.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.164+0000 7fa7a4e7c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a00b02e0 con 0x7fa7a015a860 2024-04-25T09:54:54.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.164+0000 7fa7a4e7c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a00af9b0 con 0x7fa7a006ea20 2024-04-25T09:54:54.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.165+0000 7fa79e59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa7a015a860 0x7fa7a00cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.165+0000 7fa79dd9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa7a00cd1f0 0x7fa7a006e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60344/0 (socket says 172.21.15.139:60344) 2024-04-25T09:54:54.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.165+0000 7fa79dd9b640 1 -- 172.21.15.139:0/4149418380 learned_addr learned my addr 172.21.15.139:0/4149418380 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:54.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.165+0000 7fa79ed9d640 1 --2- 172.21.15.139:0/4149418380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7a006ea20 0x7fa7a00b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.165+0000 7fa79e59c640 1 -- 172.21.15.139:0/4149418380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7a006ea20 msgr2=0x7fa7a00b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:54.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.165+0000 7fa79e59c640 1 --2- 172.21.15.139:0/4149418380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7a006ea20 0x7fa7a00b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.165+0000 7fa79e59c640 1 -- 172.21.15.139:0/4149418380 >> v1:172.21.15.139:6789/0 conn(0x7fa7a00cd1f0 legacy=0x7fa7a006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.165+0000 7fa79d59a640 1 -- 172.21.15.139:0/4149418380 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682663608 0 0) 0x7fa7a00af6d0 con 0x7fa7a00cd1f0 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.165+0000 7fa79e59c640 1 -- 172.21.15.139:0/4149418380 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a00afc90 con 0x7fa7a015a860 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.166+0000 7fa79e59c640 1 --2- 172.21.15.139:0/4149418380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a015a860 0x7fa7a00cccb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa7940097a0 tx=0x7fa79402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4eb4c52d415a1d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.166+0000 7fa79d59a640 1 -- 172.21.15.139:0/4149418380 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794037070 con 0x7fa7a015a860 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.166+0000 7fa79d59a640 1 -- 172.21.15.139:0/4149418380 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa794032440 con 0x7fa7a015a860 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.166+0000 7fa79d59a640 1 -- 172.21.15.139:0/4149418380 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794032810 con 0x7fa7a015a860 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.166+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/4149418380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a015a860 msgr2=0x7fa7a00cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.166+0000 7fa7a4e7c640 1 --2- 172.21.15.139:0/4149418380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a015a860 0x7fa7a00cccb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa7940097a0 tx=0x7fa79402a180 comp rx=0 tx=0).stop 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.166+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/4149418380 shutdown_connections 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.166+0000 7fa7a4e7c640 1 --2- 172.21.15.139:0/4149418380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7a006ea20 0x7fa7a00b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.166+0000 7fa7a4e7c640 1 --2- 172.21.15.139:0/4149418380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a015a860 0x7fa7a00cccb0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.166+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/4149418380 >> 172.21.15.139:0/4149418380 conn(0x7fa7a0066f20 msgr2=0x7fa7a006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:54.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.166+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/4149418380 shutdown_connections 2024-04-25T09:54:54.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.166+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/4149418380 wait complete. 2024-04-25T09:54:54.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.168+0000 7fa7a4e7c640 1 Processor -- start 2024-04-25T09:54:54.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.168+0000 7fa7a4e7c640 1 -- start start 2024-04-25T09:54:54.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.168+0000 7fa7a4e7c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7a006ea20 0x7fa7a00b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.168+0000 7fa7a4e7c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7a00cd1f0 0x7fa7a00b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.168+0000 7fa7a4e7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a015a860 0x7fa7a00b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.168+0000 7fa7a4e7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a00af420 con 0x7fa7a015a860 2024-04-25T09:54:54.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.168+0000 7fa7a4e7c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a00af2a0 con 0x7fa7a00cd1f0 2024-04-25T09:54:54.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.168+0000 7fa7a4e7c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a015ac40 con 0x7fa7a006ea20 2024-04-25T09:54:54.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa79ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a015a860 0x7fa7a00b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa79ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a015a860 0x7fa7a00b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34272/0 (socket says 172.21.15.139:34272) 2024-04-25T09:54:54.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa79ed9d640 1 -- 172.21.15.139:0/1449716513 learned_addr learned my addr 172.21.15.139:0/1449716513 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:54.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa79dd9b640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7a00cd1f0 0x7fa7a00b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa79e59c640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7a006ea20 0x7fa7a00b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa79ed9d640 1 -- 172.21.15.139:0/1449716513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7a006ea20 msgr2=0x7fa7a00b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa79ed9d640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7a006ea20 0x7fa7a00b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa79ed9d640 1 -- 172.21.15.139:0/1449716513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7a00cd1f0 msgr2=0x7fa7a00b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa79ed9d640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7a00cd1f0 0x7fa7a00b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa79ed9d640 1 -- 172.21.15.139:0/1449716513 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a00b3930 con 0x7fa7a015a860 2024-04-25T09:54:54.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa79ed9d640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a015a860 0x7fa7a00b4090 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa79800cd00 tx=0x7fa79800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa78effd640 1 -- 172.21.15.139:0/1449716513 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798003b90 con 0x7fa7a015a860 2024-04-25T09:54:54.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa78effd640 1 -- 172.21.15.139:0/1449716513 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa798003d50 con 0x7fa7a015a860 2024-04-25T09:54:54.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/1449716513 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a01f3d30 con 0x7fa7a015a860 2024-04-25T09:54:54.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa78effd640 1 -- 172.21.15.139:0/1449716513 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798004260 con 0x7fa7a015a860 2024-04-25T09:54:54.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.169+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/1449716513 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a01f42f0 con 0x7fa7a015a860 2024-04-25T09:54:54.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.170+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/1449716513 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a0074520 con 0x7fa7a015a860 2024-04-25T09:54:54.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.171+0000 7fa78effd640 1 -- 172.21.15.139:0/1449716513 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa798004400 con 0x7fa7a015a860 2024-04-25T09:54:54.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.171+0000 7fa78effd640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa78006b4d0 0x7fa78006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.171+0000 7fa78effd640 1 -- 172.21.15.139:0/1449716513 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa79808a940 con 0x7fa7a015a860 2024-04-25T09:54:54.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.171+0000 7fa79e59c640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa78006b4d0 0x7fa78006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.172+0000 7fa79e59c640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa78006b4d0 0x7fa78006d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa794034000 tx=0x7fa79402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.175+0000 7fa78effd640 1 -- 172.21.15.139:0/1449716513 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa798059330 con 0x7fa7a015a860 2024-04-25T09:54:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.347+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/1449716513 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa7a0159da0 con 0x7fa7a015a860 2024-04-25T09:54:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.347+0000 7fa78effd640 1 -- 172.21.15.139:0/1449716513 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v46) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa79805d3d0 con 0x7fa7a015a860 2024-04-25T09:54:54.350 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:54.350 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-04-25T09:54:54.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:54:54.350 INFO:teuthology.orchestra.run.smithi139.stdout: "poolnum": 1, 2024-04-25T09:54:54.350 INFO:teuthology.orchestra.run.smithi139.stdout: "poolname": ".mgr" 2024-04-25T09:54:54.350 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:54:54.350 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-04-25T09:54:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.349+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/1449716513 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa78006b4d0 msgr2=0x7fa78006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.349+0000 7fa7a4e7c640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa78006b4d0 0x7fa78006d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa794034000 tx=0x7fa79402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.349+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/1449716513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a015a860 msgr2=0x7fa7a00b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.349+0000 7fa7a4e7c640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a015a860 0x7fa7a00b4090 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa79800cd00 tx=0x7fa79800a660 comp rx=0 tx=0).stop 2024-04-25T09:54:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.349+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/1449716513 shutdown_connections 2024-04-25T09:54:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.349+0000 7fa7a4e7c640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a015a860 0x7fa7a00b4090 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.349+0000 7fa7a4e7c640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa78006b4d0 0x7fa78006d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.349+0000 7fa7a4e7c640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7a00cd1f0 0x7fa7a00b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.349+0000 7fa7a4e7c640 1 --2- 172.21.15.139:0/1449716513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7a006ea20 0x7fa7a00b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.349+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/1449716513 >> 172.21.15.139:0/1449716513 conn(0x7fa7a0066f20 msgr2=0x7fa7a0157fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.349+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/1449716513 shutdown_connections 2024-04-25T09:54:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.349+0000 7fa7a4e7c640 1 -- 172.21.15.139:0/1449716513 wait complete. 2024-04-25T09:54:54.362 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:54:54.363 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:54:54.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.461+0000 7f22ebd34640 1 Processor -- start 2024-04-25T09:54:54.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.461+0000 7f22ebd34640 1 -- start start 2024-04-25T09:54:54.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ebd34640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f22e4144090 0x7f22e40b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ebd34640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22e40cc9e0 0x7f22e406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ebd34640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22e40cce20 0x7f22e406e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ebd34640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e40b2a50 con 0x7f22e40cce20 2024-04-25T09:54:54.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ebd34640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e415b090 con 0x7f22e4144090 2024-04-25T09:54:54.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ebd34640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e415b370 con 0x7f22e40cc9e0 2024-04-25T09:54:54.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ea2d1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22e40cce20 0x7f22e406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ea2d1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22e40cce20 0x7f22e406e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34282/0 (socket says 172.21.15.139:34282) 2024-04-25T09:54:54.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ea2d1640 1 -- 172.21.15.139:0/2361992946 learned_addr learned my addr 172.21.15.139:0/2361992946 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:54.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ea2d1640 1 -- 172.21.15.139:0/2361992946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22e40cc9e0 msgr2=0x7f22e406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:54.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ea2d1640 1 --2- 172.21.15.139:0/2361992946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22e40cc9e0 0x7f22e406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ea2d1640 1 -- 172.21.15.139:0/2361992946 >> v2:172.21.15.150:3300/0 conn(0x7f22e4144090 msgr2=0x7f22e40b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:54.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22e9ad0640 1 --2- 172.21.15.139:0/2361992946 >> v2:172.21.15.150:3300/0 conn(0x7f22e4144090 0x7f22e40b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ea2d1640 1 --2- 172.21.15.139:0/2361992946 >> v2:172.21.15.150:3300/0 conn(0x7f22e4144090 0x7f22e40b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.462+0000 7f22ea2d1640 1 -- 172.21.15.139:0/2361992946 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e415b650 con 0x7f22e40cce20 2024-04-25T09:54:54.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.463+0000 7f22e9ad0640 1 --2- 172.21.15.139:0/2361992946 >> v2:172.21.15.150:3300/0 conn(0x7f22e4144090 0x7f22e40b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:54.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.463+0000 7f22ea2d1640 1 --2- 172.21.15.139:0/2361992946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40cce20 0x7f22e406e870 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f22e40b4ce0 tx=0x7f22dc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69caaef8cc217217 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.463+0000 7f22e8ace640 1 -- 172.21.15.139:0/2361992946 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc037070 con 0x7f22e40cce20 2024-04-25T09:54:54.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.463+0000 7f22e8ace640 1 -- 172.21.15.139:0/2361992946 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc032440 con 0x7f22e40cce20 2024-04-25T09:54:54.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.463+0000 7f22e8ace640 1 -- 172.21.15.139:0/2361992946 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc032810 con 0x7f22e40cce20 2024-04-25T09:54:54.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.464+0000 7f22ebd34640 1 -- 172.21.15.139:0/2361992946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40cce20 msgr2=0x7f22e406e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.464+0000 7f22ebd34640 1 --2- 172.21.15.139:0/2361992946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40cce20 0x7f22e406e870 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f22e40b4ce0 tx=0x7f22dc02a140 comp rx=0 tx=0).stop 2024-04-25T09:54:54.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.464+0000 7f22ebd34640 1 -- 172.21.15.139:0/2361992946 shutdown_connections 2024-04-25T09:54:54.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.464+0000 7f22ebd34640 1 --2- 172.21.15.139:0/2361992946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40cce20 0x7f22e406e870 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.464+0000 7f22ebd34640 1 --2- 172.21.15.139:0/2361992946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22e40cc9e0 0x7f22e406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.464+0000 7f22ebd34640 1 --2- 172.21.15.139:0/2361992946 >> v2:172.21.15.150:3300/0 conn(0x7f22e4144090 0x7f22e40b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.464+0000 7f22ebd34640 1 -- 172.21.15.139:0/2361992946 >> 172.21.15.139:0/2361992946 conn(0x7f22e4067000 msgr2=0x7f22e406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:54.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.464+0000 7f22ebd34640 1 -- 172.21.15.139:0/2361992946 shutdown_connections 2024-04-25T09:54:54.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.464+0000 7f22ebd34640 1 -- 172.21.15.139:0/2361992946 wait complete. 2024-04-25T09:54:54.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.465+0000 7f22ebd34640 1 Processor -- start 2024-04-25T09:54:54.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.465+0000 7f22ebd34640 1 -- start start 2024-04-25T09:54:54.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22ebd34640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40cc9e0 0x7f22e4158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22ebd34640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22e40cce20 0x7f22e4158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22ebd34640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e4144090 0x7f22e41638f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22ebd34640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4144470 con 0x7f22e40cc9e0 2024-04-25T09:54:54.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22ebd34640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e415bc10 con 0x7f22e4144090 2024-04-25T09:54:54.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22ebd34640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e415ad80 con 0x7f22e40cce20 2024-04-25T09:54:54.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22e9ad0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40cc9e0 0x7f22e4158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22e9ad0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40cc9e0 0x7f22e4158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34290/0 (socket says 172.21.15.139:34290) 2024-04-25T09:54:54.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22e9ad0640 1 -- 172.21.15.139:0/872417832 learned_addr learned my addr 172.21.15.139:0/872417832 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:54.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22ea2d1640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e4144090 0x7f22e41638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22e9ad0640 1 -- 172.21.15.139:0/872417832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22e40cce20 msgr2=0x7f22e4158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22e92cf640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22e40cce20 0x7f22e4158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22e9ad0640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22e40cce20 0x7f22e4158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22e9ad0640 1 -- 172.21.15.139:0/872417832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e4144090 msgr2=0x7f22e41638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22e9ad0640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e4144090 0x7f22e41638f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22e9ad0640 1 -- 172.21.15.139:0/872417832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e4166b60 con 0x7f22e40cc9e0 2024-04-25T09:54:54.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22e9ad0640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40cc9e0 0x7f22e4158fa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f22e000ce00 tx=0x7f22e000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22da7fc640 1 -- 172.21.15.139:0/872417832 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e000f890 con 0x7f22e40cc9e0 2024-04-25T09:54:54.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22da7fc640 1 -- 172.21.15.139:0/872417832 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e000fa30 con 0x7f22e40cc9e0 2024-04-25T09:54:54.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22da7fc640 1 -- 172.21.15.139:0/872417832 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e000fe40 con 0x7f22e40cc9e0 2024-04-25T09:54:54.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.466+0000 7f22ebd34640 1 -- 172.21.15.139:0/872417832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e4159680 con 0x7f22e40cc9e0 2024-04-25T09:54:54.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.467+0000 7f22ebd34640 1 -- 172.21.15.139:0/872417832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e4159c40 con 0x7f22e40cc9e0 2024-04-25T09:54:54.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.468+0000 7f22ebd34640 1 -- 172.21.15.139:0/872417832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e4074520 con 0x7f22e40cc9e0 2024-04-25T09:54:54.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.468+0000 7f22da7fc640 1 -- 172.21.15.139:0/872417832 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22e0018400 con 0x7f22e40cc9e0 2024-04-25T09:54:54.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.468+0000 7f22da7fc640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f22c806b500 0x7f22c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.469+0000 7f22da7fc640 1 -- 172.21.15.139:0/872417832 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f22e008ae00 con 0x7f22e40cc9e0 2024-04-25T09:54:54.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.469+0000 7f22e92cf640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f22c806b500 0x7f22c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.469+0000 7f22e92cf640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f22c806b500 0x7f22c806d9c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f22e40aead0 tx=0x7f22d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.473+0000 7f22da7fc640 1 -- 172.21.15.139:0/872417832 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22e0014030 con 0x7f22e40cc9e0 2024-04-25T09:54:54.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:54.642+0000 7f22ebd34640 1 -- 172.21.15.139:0/872417832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f22e406d410 con 0x7f22e40cc9e0 2024-04-25T09:54:55.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.106+0000 7f22da7fc640 1 -- 172.21.15.139:0/872417832 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f22e000ae20 con 0x7f22e40cc9e0 2024-04-25T09:54:55.109 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' created 2024-04-25T09:54:55.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.109+0000 7f22ebd34640 1 -- 172.21.15.139:0/872417832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f22c806b500 msgr2=0x7f22c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.109+0000 7f22ebd34640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f22c806b500 0x7f22c806d9c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f22e40aead0 tx=0x7f22d4008040 comp rx=0 tx=0).stop 2024-04-25T09:54:55.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.109+0000 7f22ebd34640 1 -- 172.21.15.139:0/872417832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40cc9e0 msgr2=0x7f22e4158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.109+0000 7f22ebd34640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40cc9e0 0x7f22e4158fa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f22e000ce00 tx=0x7f22e000a650 comp rx=0 tx=0).stop 2024-04-25T09:54:55.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.109+0000 7f22ebd34640 1 -- 172.21.15.139:0/872417832 shutdown_connections 2024-04-25T09:54:55.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.109+0000 7f22ebd34640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f22c806b500 0x7f22c806d9c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.109+0000 7f22ebd34640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e4144090 0x7f22e41638f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.109+0000 7f22ebd34640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22e40cce20 0x7f22e4158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.109+0000 7f22ebd34640 1 --2- 172.21.15.139:0/872417832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22e40cc9e0 0x7f22e4158fa0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.109+0000 7f22ebd34640 1 -- 172.21.15.139:0/872417832 >> 172.21.15.139:0/872417832 conn(0x7f22e4067000 msgr2=0x7f22e40b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:55.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.109+0000 7f22ebd34640 1 -- 172.21.15.139:0/872417832 shutdown_connections 2024-04-25T09:54:55.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.109+0000 7f22ebd34640 1 -- 172.21.15.139:0/872417832 wait complete. 2024-04-25T09:54:55.126 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:54:55.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.228+0000 7f7feeca3640 1 Processor -- start 2024-04-25T09:54:55.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.229+0000 7f7feeca3640 1 -- start start 2024-04-25T09:54:55.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.229+0000 7f7feeca3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fe80712f0 0x7f7fe80b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.229+0000 7f7feeca3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7fe80719d0 0x7f7fe80b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.229+0000 7f7feeca3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe80b1100 con 0x7f7fe8145250 2024-04-25T09:54:55.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.229+0000 7f7feeca3640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe80ceed0 con 0x7f7fe80719d0 2024-04-25T09:54:55.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.229+0000 7f7feeca3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe80ce2c0 con 0x7f7fe80712f0 2024-04-25T09:54:55.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.230+0000 7f7fdffff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fe80712f0 0x7f7fe80b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.230+0000 7f7fed240640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7fe80719d0 0x7f7fe80b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.230+0000 7f7feca3f640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f7fe8145250 0x7f7fe8070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:45708/0 (socket says 172.21.15.139:45708) 2024-04-25T09:54:55.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.230+0000 7f7feca3f640 1 -- 172.21.15.139:0/782662557 learned_addr learned my addr 172.21.15.139:0/782662557 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:55.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.230+0000 7f7fdffff640 1 -- 172.21.15.139:0/782662557 >> v2:172.21.15.150:3300/0 conn(0x7f7fe80719d0 msgr2=0x7f7fe80b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.230+0000 7f7fdffff640 1 --2- 172.21.15.139:0/782662557 >> v2:172.21.15.150:3300/0 conn(0x7f7fe80719d0 0x7f7fe80b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.230+0000 7f7fdffff640 1 -- 172.21.15.139:0/782662557 >> v1:172.21.15.150:6789/0 conn(0x7f7fe8145250 legacy=0x7f7fe8070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.230+0000 7f7fdffff640 1 -- 172.21.15.139:0/782662557 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe80ce5a0 con 0x7f7fe80712f0 2024-04-25T09:54:55.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.230+0000 7f7fed240640 1 --2- 172.21.15.139:0/782662557 >> v2:172.21.15.150:3300/0 conn(0x7f7fe80719d0 0x7f7fe80b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:55.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.231+0000 7f7fdffff640 1 --2- 172.21.15.139:0/782662557 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fe80712f0 0x7f7fe80b4680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80097a0 tx=0x7f7fd802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e96b1a51c771119f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.231+0000 7f7fdf7fe640 1 -- 172.21.15.139:0/782662557 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8037070 con 0x7f7fe80712f0 2024-04-25T09:54:55.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.231+0000 7f7fdf7fe640 1 -- 172.21.15.139:0/782662557 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd802ae50 con 0x7f7fe80712f0 2024-04-25T09:54:55.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.231+0000 7f7fdf7fe640 1 -- 172.21.15.139:0/782662557 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8032660 con 0x7f7fe80712f0 2024-04-25T09:54:55.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.232+0000 7f7feeca3640 1 -- 172.21.15.139:0/782662557 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fe80712f0 msgr2=0x7f7fe80b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.232+0000 7f7feeca3640 1 --2- 172.21.15.139:0/782662557 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fe80712f0 0x7f7fe80b4680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80097a0 tx=0x7f7fd802a100 comp rx=0 tx=0).stop 2024-04-25T09:54:55.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.232+0000 7f7feeca3640 1 -- 172.21.15.139:0/782662557 shutdown_connections 2024-04-25T09:54:55.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.232+0000 7f7feeca3640 1 --2- 172.21.15.139:0/782662557 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fe80712f0 0x7f7fe80b4680 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.232+0000 7f7feeca3640 1 --2- 172.21.15.139:0/782662557 >> v2:172.21.15.150:3300/0 conn(0x7f7fe80719d0 0x7f7fe80b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.232+0000 7f7feeca3640 1 -- 172.21.15.139:0/782662557 >> 172.21.15.139:0/782662557 conn(0x7f7fe806f460 msgr2=0x7f7fe806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:55.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.232+0000 7f7feeca3640 1 -- 172.21.15.139:0/782662557 shutdown_connections 2024-04-25T09:54:55.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.232+0000 7f7feeca3640 1 -- 172.21.15.139:0/782662557 wait complete. 2024-04-25T09:54:55.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.233+0000 7f7feeca3640 1 Processor -- start 2024-04-25T09:54:55.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.233+0000 7f7feeca3640 1 -- start start 2024-04-25T09:54:55.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feeca3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fe80712f0 0x7f7fe813b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feeca3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fe80719d0 0x7f7fe8141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feeca3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fe8145250 0x7f7fe8141660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feeca3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8145630 con 0x7f7fe80712f0 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feeca3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe80ceb60 con 0x7f7fe80719d0 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feeca3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe80d11a0 con 0x7f7fe8145250 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feca3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fe80712f0 0x7f7fe813b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feca3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fe80712f0 0x7f7fe813b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34300/0 (socket says 172.21.15.139:34300) 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feca3f640 1 -- 172.21.15.139:0/3857671427 learned_addr learned my addr 172.21.15.139:0/3857671427 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7fed240640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fe8145250 0x7f7fe8141660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7fdffff640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fe80719d0 0x7f7fe8141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feca3f640 1 -- 172.21.15.139:0/3857671427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fe8145250 msgr2=0x7f7fe8141660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feca3f640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fe8145250 0x7f7fe8141660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feca3f640 1 -- 172.21.15.139:0/3857671427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fe80719d0 msgr2=0x7f7fe8141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feca3f640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fe80719d0 0x7f7fe8141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feca3f640 1 -- 172.21.15.139:0/3857671427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe8140b30 con 0x7f7fe80712f0 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7fdffff640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fe80719d0 0x7f7fe8141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feca3f640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fe80712f0 0x7f7fe813b060 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400c3c0 tx=0x7f7fe400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7fdd7fa640 1 -- 172.21.15.139:0/3857671427 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4017070 con 0x7f7fe80712f0 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7fdd7fa640 1 -- 172.21.15.139:0/3857671427 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe4005ce0 con 0x7f7fe80712f0 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7feeca3640 1 -- 172.21.15.139:0/3857671427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe813e7f0 con 0x7f7fe80712f0 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.234+0000 7f7fdd7fa640 1 -- 172.21.15.139:0/3857671427 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe40072f0 con 0x7f7fe80712f0 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.235+0000 7f7feeca3640 1 -- 172.21.15.139:0/3857671427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe8140050 con 0x7f7fe80712f0 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.235+0000 7f7feeca3640 1 -- 172.21.15.139:0/3857671427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe8074520 con 0x7f7fe80712f0 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.236+0000 7f7fdd7fa640 1 -- 172.21.15.139:0/3857671427 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fe401a070 con 0x7f7fe80712f0 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.236+0000 7f7fdd7fa640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7fd406b4d0 0x7f7fd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.236+0000 7f7fdd7fa640 1 -- 172.21.15.139:0/3857671427 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f7fe40894b0 con 0x7f7fe80712f0 2024-04-25T09:54:55.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.236+0000 7f7fdffff640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7fd406b4d0 0x7f7fd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.237+0000 7f7fdffff640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7fd406b4d0 0x7f7fd406d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8034000 tx=0x7f7fd802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.241+0000 7f7fdd7fa640 1 -- 172.21.15.139:0/3857671427 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7fe4058380 con 0x7f7fe80712f0 2024-04-25T09:54:55.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:55.413+0000 7f7feeca3640 1 -- 172.21.15.139:0/3857671427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7fe81e6870 con 0x7f7fe80712f0 2024-04-25T09:54:56.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.112+0000 7f7fdd7fa640 1 -- 172.21.15.139:0/3857671427 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v48) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7fe4057c80 con 0x7f7fe80712f0 2024-04-25T09:54:56.115 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' created 2024-04-25T09:54:56.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.115+0000 7f7feeca3640 1 -- 172.21.15.139:0/3857671427 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7fd406b4d0 msgr2=0x7f7fd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:56.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.115+0000 7f7feeca3640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7fd406b4d0 0x7f7fd406d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8034000 tx=0x7f7fd802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:56.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.115+0000 7f7feeca3640 1 -- 172.21.15.139:0/3857671427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fe80712f0 msgr2=0x7f7fe813b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:56.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.115+0000 7f7feeca3640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fe80712f0 0x7f7fe813b060 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400c3c0 tx=0x7f7fe400c7c0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.116+0000 7f7feeca3640 1 -- 172.21.15.139:0/3857671427 shutdown_connections 2024-04-25T09:54:56.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.116+0000 7f7feeca3640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7fd406b4d0 0x7f7fd406d990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.116+0000 7f7feeca3640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fe8145250 0x7f7fe8141660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.116+0000 7f7feeca3640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fe80719d0 0x7f7fe8141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.116+0000 7f7feeca3640 1 --2- 172.21.15.139:0/3857671427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fe80712f0 0x7f7fe813b060 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.116+0000 7f7feeca3640 1 -- 172.21.15.139:0/3857671427 >> 172.21.15.139:0/3857671427 conn(0x7f7fe806f460 msgr2=0x7f7fe8070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:56.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.122+0000 7f7feeca3640 1 -- 172.21.15.139:0/3857671427 shutdown_connections 2024-04-25T09:54:56.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.123+0000 7f7feeca3640 1 -- 172.21.15.139:0/3857671427 wait complete. 2024-04-25T09:54:56.146 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:54:56.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.246+0000 7f9c3a64d640 1 Processor -- start 2024-04-25T09:54:56.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.246+0000 7f9c3a64d640 1 -- start start 2024-04-25T09:54:56.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.247+0000 7f9c3a64d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c3406b270 0x7f9c340704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:56.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.247+0000 7f9c3a64d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9c34072d60 0x7f9c34167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:56.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.247+0000 7f9c3a64d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c340731d0 con 0x7f9c3415da30 2024-04-25T09:54:56.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.247+0000 7f9c3a64d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c340cd540 con 0x7f9c3406b270 2024-04-25T09:54:56.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.247+0000 7f9c3a64d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c340b3de0 con 0x7f9c34072d60 2024-04-25T09:54:56.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.247+0000 7f9c337fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c3406b270 0x7f9c340704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:56.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.247+0000 7f9c337fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c3406b270 0x7f9c340704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34310/0 (socket says 172.21.15.139:34310) 2024-04-25T09:54:56.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.247+0000 7f9c337fe640 1 -- 172.21.15.139:0/3039516171 learned_addr learned my addr 172.21.15.139:0/3039516171 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:56.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.248+0000 7f9c38bea640 1 --2- 172.21.15.139:0/3039516171 >> v2:172.21.15.150:3300/0 conn(0x7f9c34072d60 0x7f9c34167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:56.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.248+0000 7f9c337fe640 1 -- 172.21.15.139:0/3039516171 >> v2:172.21.15.150:3300/0 conn(0x7f9c34072d60 msgr2=0x7f9c34167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:56.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.248+0000 7f9c337fe640 1 --2- 172.21.15.139:0/3039516171 >> v2:172.21.15.150:3300/0 conn(0x7f9c34072d60 0x7f9c34167550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.248+0000 7f9c337fe640 1 -- 172.21.15.139:0/3039516171 >> v1:172.21.15.150:6789/0 conn(0x7f9c3415da30 legacy=0x7f9c3406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:56.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.248+0000 7f9c337fe640 1 -- 172.21.15.139:0/3039516171 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3415de10 con 0x7f9c3406b270 2024-04-25T09:54:56.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.248+0000 7f9c337fe640 1 --2- 172.21.15.139:0/3039516171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3406b270 0x7f9c340704a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9c340b3490 tx=0x7f9c2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58eefadfffa094f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:56.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.249+0000 7f9c32ffd640 1 -- 172.21.15.139:0/3039516171 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28037070 con 0x7f9c3406b270 2024-04-25T09:54:56.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.249+0000 7f9c32ffd640 1 -- 172.21.15.139:0/3039516171 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c28032440 con 0x7f9c3406b270 2024-04-25T09:54:56.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.249+0000 7f9c32ffd640 1 -- 172.21.15.139:0/3039516171 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28032810 con 0x7f9c3406b270 2024-04-25T09:54:56.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.249+0000 7f9c3a64d640 1 -- 172.21.15.139:0/3039516171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3406b270 msgr2=0x7f9c340704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:56.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.249+0000 7f9c3a64d640 1 --2- 172.21.15.139:0/3039516171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3406b270 0x7f9c340704a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9c340b3490 tx=0x7f9c2802a270 comp rx=0 tx=0).stop 2024-04-25T09:54:56.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.249+0000 7f9c3a64d640 1 -- 172.21.15.139:0/3039516171 shutdown_connections 2024-04-25T09:54:56.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.249+0000 7f9c3a64d640 1 --2- 172.21.15.139:0/3039516171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3406b270 0x7f9c340704a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.249+0000 7f9c3a64d640 1 --2- 172.21.15.139:0/3039516171 >> v2:172.21.15.150:3300/0 conn(0x7f9c34072d60 0x7f9c34167550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.249+0000 7f9c3a64d640 1 -- 172.21.15.139:0/3039516171 >> 172.21.15.139:0/3039516171 conn(0x7f9c340b23f0 msgr2=0x7f9c340b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:56.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.249+0000 7f9c3a64d640 1 -- 172.21.15.139:0/3039516171 shutdown_connections 2024-04-25T09:54:56.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.249+0000 7f9c3a64d640 1 -- 172.21.15.139:0/3039516171 wait complete. 2024-04-25T09:54:56.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.250+0000 7f9c3a64d640 1 Processor -- start 2024-04-25T09:54:56.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.251+0000 7f9c3a64d640 1 -- start start 2024-04-25T09:54:56.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.251+0000 7f9c3a64d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3406b270 0x7f9c34155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.251+0000 7f9c3a64d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34072d60 0x7f9c341525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.251+0000 7f9c3a64d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3415da30 0x7f9c3415e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.251+0000 7f9c3a64d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340b3b00 con 0x7f9c3406b270 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.251+0000 7f9c3a64d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340b3820 con 0x7f9c34072d60 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.251+0000 7f9c3a64d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340b42a0 con 0x7f9c3415da30 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.251+0000 7f9c33fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3406b270 0x7f9c34155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.251+0000 7f9c33fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3406b270 0x7f9c34155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:34326/0 (socket says 172.21.15.139:34326) 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.251+0000 7f9c33fff640 1 -- 172.21.15.139:0/320867330 learned_addr learned my addr 172.21.15.139:0/320867330 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.251+0000 7f9c38bea640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3415da30 0x7f9c3415e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.252+0000 7f9c337fe640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34072d60 0x7f9c341525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.252+0000 7f9c33fff640 1 -- 172.21.15.139:0/320867330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3415da30 msgr2=0x7f9c3415e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.252+0000 7f9c33fff640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3415da30 0x7f9c3415e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.252+0000 7f9c33fff640 1 -- 172.21.15.139:0/320867330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34072d60 msgr2=0x7f9c341525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:56.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.252+0000 7f9c33fff640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34072d60 0x7f9c341525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.252+0000 7f9c33fff640 1 -- 172.21.15.139:0/320867330 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c340ca630 con 0x7f9c3406b270 2024-04-25T09:54:56.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.252+0000 7f9c33fff640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3406b270 0x7f9c34155d00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400c400 tx=0x7f9c2400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:56.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.252+0000 7f9c30ff9640 1 -- 172.21.15.139:0/320867330 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240052d0 con 0x7f9c3406b270 2024-04-25T09:54:56.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.252+0000 7f9c30ff9640 1 -- 172.21.15.139:0/320867330 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24005490 con 0x7f9c3406b270 2024-04-25T09:54:56.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.252+0000 7f9c3a64d640 1 -- 172.21.15.139:0/320867330 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c340ca8d0 con 0x7f9c3406b270 2024-04-25T09:54:56.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.252+0000 7f9c30ff9640 1 -- 172.21.15.139:0/320867330 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24007260 con 0x7f9c3406b270 2024-04-25T09:54:56.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.252+0000 7f9c3a64d640 1 -- 172.21.15.139:0/320867330 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c340cae90 con 0x7f9c3406b270 2024-04-25T09:54:56.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.253+0000 7f9c3a64d640 1 -- 172.21.15.139:0/320867330 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c34074520 con 0x7f9c3406b270 2024-04-25T09:54:56.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.254+0000 7f9c30ff9640 1 -- 172.21.15.139:0/320867330 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c24007400 con 0x7f9c3406b270 2024-04-25T09:54:56.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.254+0000 7f9c30ff9640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c2006b4d0 0x7f9c2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:56.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.254+0000 7f9c30ff9640 1 -- 172.21.15.139:0/320867330 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f9c24089840 con 0x7f9c3406b270 2024-04-25T09:54:56.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.254+0000 7f9c337fe640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c2006b4d0 0x7f9c2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:56.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.254+0000 7f9c337fe640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c2006b4d0 0x7f9c2006d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9c28034000 tx=0x7f9c2802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:56.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.259+0000 7f9c30ff9640 1 -- 172.21.15.139:0/320867330 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c24058590 con 0x7f9c3406b270 2024-04-25T09:54:56.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:56.464+0000 7f9c3a64d640 1 -- 172.21.15.139:0/320867330 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9c3415ba80 con 0x7f9c3406b270 2024-04-25T09:54:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.113+0000 7f9c30ff9640 1 -- 172.21.15.139:0/320867330 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9c24057e90 con 0x7f9c3406b270 2024-04-25T09:54:57.116 INFO:teuthology.orchestra.run.smithi139.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T09:54:57.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.116+0000 7f9c3a64d640 1 -- 172.21.15.139:0/320867330 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c2006b4d0 msgr2=0x7f9c2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.116+0000 7f9c3a64d640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c2006b4d0 0x7f9c2006d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9c28034000 tx=0x7f9c2802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:57.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.116+0000 7f9c3a64d640 1 -- 172.21.15.139:0/320867330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3406b270 msgr2=0x7f9c34155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.116+0000 7f9c3a64d640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3406b270 0x7f9c34155d00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400c400 tx=0x7f9c2400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.116+0000 7f9c3a64d640 1 -- 172.21.15.139:0/320867330 shutdown_connections 2024-04-25T09:54:57.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.116+0000 7f9c3a64d640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9c2006b4d0 0x7f9c2006d990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.116+0000 7f9c3a64d640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c3415da30 0x7f9c3415e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.116+0000 7f9c3a64d640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34072d60 0x7f9c341525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.116+0000 7f9c3a64d640 1 --2- 172.21.15.139:0/320867330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c3406b270 0x7f9c34155d00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.116+0000 7f9c3a64d640 1 -- 172.21.15.139:0/320867330 >> 172.21.15.139:0/320867330 conn(0x7f9c340b23f0 msgr2=0x7f9c34157d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:57.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.116+0000 7f9c3a64d640 1 -- 172.21.15.139:0/320867330 shutdown_connections 2024-04-25T09:54:57.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.116+0000 7f9c3a64d640 1 -- 172.21.15.139:0/320867330 wait complete. 2024-04-25T09:54:57.144 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:57.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.243+0000 7fae946bd640 1 Processor -- start 2024-04-25T09:54:57.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.243+0000 7fae946bd640 1 -- start start 2024-04-25T09:54:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.244+0000 7fae946bd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fae8c15dc60 0x7fae8c0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.244+0000 7fae946bd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae8c161850 0x7fae8c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.244+0000 7fae946bd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae8c0d0970 0x7fae8c0d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.244+0000 7fae946bd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8c0ce780 con 0x7fae8c161850 2024-04-25T09:54:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.244+0000 7fae946bd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8c0d3730 con 0x7fae8c15dc60 2024-04-25T09:54:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.244+0000 7fae946bd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c0cf250 con 0x7fae8c0d0970 2024-04-25T09:54:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.245+0000 7fae91c58640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae8c161850 0x7fae8c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.245+0000 7fae91c58640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae8c161850 0x7fae8c0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47094/0 (socket says 172.21.15.139:47094) 2024-04-25T09:54:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.245+0000 7fae91c58640 1 -- 172.21.15.139:0/2604962348 learned_addr learned my addr 172.21.15.139:0/2604962348 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.245+0000 7fae92459640 1 --2- 172.21.15.139:0/2604962348 >> v2:172.21.15.150:3300/0 conn(0x7fae8c15dc60 0x7fae8c0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.245+0000 7fae92c5a640 1 --2- 172.21.15.139:0/2604962348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae8c0d0970 0x7fae8c0d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.245+0000 7fae91c58640 1 -- 172.21.15.139:0/2604962348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae8c0d0970 msgr2=0x7fae8c0d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.245+0000 7fae91c58640 1 --2- 172.21.15.139:0/2604962348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae8c0d0970 0x7fae8c0d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.245+0000 7fae91c58640 1 -- 172.21.15.139:0/2604962348 >> v2:172.21.15.150:3300/0 conn(0x7fae8c15dc60 msgr2=0x7fae8c0b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.245+0000 7fae91c58640 1 --2- 172.21.15.139:0/2604962348 >> v2:172.21.15.150:3300/0 conn(0x7fae8c15dc60 0x7fae8c0b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.245+0000 7fae91c58640 1 -- 172.21.15.139:0/2604962348 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c0ce9b0 con 0x7fae8c161850 2024-04-25T09:54:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.245+0000 7fae92459640 1 --2- 172.21.15.139:0/2604962348 >> v2:172.21.15.150:3300/0 conn(0x7fae8c15dc60 0x7fae8c0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae91c58640 1 --2- 172.21.15.139:0/2604962348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c161850 0x7fae8c0ce240 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fae8c08c680 tx=0x7fae7c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f092606571a796be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae91457640 1 -- 172.21.15.139:0/2604962348 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c037070 con 0x7fae8c161850 2024-04-25T09:54:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae91457640 1 -- 172.21.15.139:0/2604962348 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c032440 con 0x7fae8c161850 2024-04-25T09:54:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae91457640 1 -- 172.21.15.139:0/2604962348 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c032810 con 0x7fae8c161850 2024-04-25T09:54:57.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae946bd640 1 -- 172.21.15.139:0/2604962348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c161850 msgr2=0x7fae8c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae946bd640 1 --2- 172.21.15.139:0/2604962348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c161850 0x7fae8c0ce240 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fae8c08c680 tx=0x7fae7c02a210 comp rx=0 tx=0).stop 2024-04-25T09:54:57.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae946bd640 1 -- 172.21.15.139:0/2604962348 shutdown_connections 2024-04-25T09:54:57.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae946bd640 1 --2- 172.21.15.139:0/2604962348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae8c0d0970 0x7fae8c0d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae946bd640 1 --2- 172.21.15.139:0/2604962348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c161850 0x7fae8c0ce240 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae946bd640 1 --2- 172.21.15.139:0/2604962348 >> v2:172.21.15.150:3300/0 conn(0x7fae8c15dc60 0x7fae8c0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae946bd640 1 -- 172.21.15.139:0/2604962348 >> 172.21.15.139:0/2604962348 conn(0x7fae8c0b5910 msgr2=0x7fae8c0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:57.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae946bd640 1 -- 172.21.15.139:0/2604962348 shutdown_connections 2024-04-25T09:54:57.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.246+0000 7fae946bd640 1 -- 172.21.15.139:0/2604962348 wait complete. 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.247+0000 7fae946bd640 1 Processor -- start 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.248+0000 7fae946bd640 1 -- start start 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.248+0000 7fae946bd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae8c0d0970 0x7fae8c0c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.248+0000 7fae946bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c15dc60 0x7fae8c15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.248+0000 7fae946bd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae8c161850 0x7fae8c1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.248+0000 7fae946bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c0d1fc0 con 0x7fae8c15dc60 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.248+0000 7fae946bd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c0cef70 con 0x7fae8c0d0970 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.248+0000 7fae946bd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c15e040 con 0x7fae8c161850 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.248+0000 7fae91c58640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c15dc60 0x7fae8c15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.248+0000 7fae91c58640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c15dc60 0x7fae8c15e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47110/0 (socket says 172.21.15.139:47110) 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.248+0000 7fae91c58640 1 -- 172.21.15.139:0/3721241997 learned_addr learned my addr 172.21.15.139:0/3721241997 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae91c58640 1 -- 172.21.15.139:0/3721241997 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae8c161850 msgr2=0x7fae8c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae92459640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae8c0d0970 0x7fae8c0c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae92c5a640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae8c161850 0x7fae8c1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae91c58640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae8c161850 0x7fae8c1563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae91c58640 1 -- 172.21.15.139:0/3721241997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae8c0d0970 msgr2=0x7fae8c0c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae91c58640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae8c0d0970 0x7fae8c0c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae91c58640 1 -- 172.21.15.139:0/3721241997 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c160ec0 con 0x7fae8c15dc60 2024-04-25T09:54:57.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae92459640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae8c0d0970 0x7fae8c0c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:57.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae91c58640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c15dc60 0x7fae8c15e320 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0090b0 tx=0x7fae7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:57.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae82ffd640 1 -- 172.21.15.139:0/3721241997 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c041020 con 0x7fae8c15dc60 2024-04-25T09:54:57.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae82ffd640 1 -- 172.21.15.139:0/3721241997 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c0306e0 con 0x7fae8c15dc60 2024-04-25T09:54:57.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae82ffd640 1 -- 172.21.15.139:0/3721241997 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c037040 con 0x7fae8c15dc60 2024-04-25T09:54:57.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae946bd640 1 -- 172.21.15.139:0/3721241997 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8c1558d0 con 0x7fae8c15dc60 2024-04-25T09:54:57.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.249+0000 7fae946bd640 1 -- 172.21.15.139:0/3721241997 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8c155e90 con 0x7fae8c15dc60 2024-04-25T09:54:57.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.250+0000 7fae946bd640 1 -- 172.21.15.139:0/3721241997 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8c074520 con 0x7fae8c15dc60 2024-04-25T09:54:57.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.250+0000 7fae82ffd640 1 -- 172.21.15.139:0/3721241997 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae7c044020 con 0x7fae8c15dc60 2024-04-25T09:54:57.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.251+0000 7fae82ffd640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae7006b4d0 0x7fae7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.251+0000 7fae82ffd640 1 -- 172.21.15.139:0/3721241997 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fae7c0aab20 con 0x7fae8c15dc60 2024-04-25T09:54:57.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.251+0000 7fae92459640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae7006b4d0 0x7fae7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.251+0000 7fae92459640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae7006b4d0 0x7fae7006d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae84009540 tx=0x7fae8400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:57.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.255+0000 7fae82ffd640 1 -- 172.21.15.139:0/3721241997 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae7c04b030 con 0x7fae8c15dc60 2024-04-25T09:54:57.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.425+0000 7fae946bd640 1 -- 172.21.15.139:0/3721241997 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae8c0d16f0 con 0x7fae8c15dc60 2024-04-25T09:54:57.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.426+0000 7fae82ffd640 1 -- 172.21.15.139:0/3721241997 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+17003 (secure 0 0 0) 0x7fae7c0798a0 con 0x7fae8c15dc60 2024-04-25T09:54:57.428 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:57.429 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":49,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:56.465907+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:54.661529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:55.431933+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:57.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.428+0000 7fae946bd640 1 -- 172.21.15.139:0/3721241997 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae7006b4d0 msgr2=0x7fae7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.428+0000 7fae946bd640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae7006b4d0 0x7fae7006d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae84009540 tx=0x7fae8400b040 comp rx=0 tx=0).stop 2024-04-25T09:54:57.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.428+0000 7fae946bd640 1 -- 172.21.15.139:0/3721241997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c15dc60 msgr2=0x7fae8c15e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.428+0000 7fae946bd640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c15dc60 0x7fae8c15e320 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0090b0 tx=0x7fae7c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.428+0000 7fae946bd640 1 -- 172.21.15.139:0/3721241997 shutdown_connections 2024-04-25T09:54:57.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.428+0000 7fae946bd640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c15dc60 0x7fae8c15e320 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.428+0000 7fae946bd640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fae7006b4d0 0x7fae7006d990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.428+0000 7fae946bd640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae8c161850 0x7fae8c1563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.428+0000 7fae946bd640 1 --2- 172.21.15.139:0/3721241997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae8c0d0970 0x7fae8c0c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.428+0000 7fae946bd640 1 -- 172.21.15.139:0/3721241997 >> 172.21.15.139:0/3721241997 conn(0x7fae8c0b5910 msgr2=0x7fae8c15c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:57.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.428+0000 7fae946bd640 1 -- 172.21.15.139:0/3721241997 shutdown_connections 2024-04-25T09:54:57.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.428+0000 7fae946bd640 1 -- 172.21.15.139:0/3721241997 wait complete. 2024-04-25T09:54:57.441 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:57.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.536+0000 7f448281f640 1 Processor -- start 2024-04-25T09:54:57.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.536+0000 7f448281f640 1 -- start start 2024-04-25T09:54:57.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.537+0000 7f448281f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c160e90 0x7f447c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.537+0000 7f448281f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f447c06ab60 0x7f447c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.537+0000 7f448281f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c0d0400 con 0x7f447c06fa70 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.537+0000 7f448281f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447c0d14a0 con 0x7f447c06ab60 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.537+0000 7f448281f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c06b500 con 0x7f447c160e90 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.537+0000 7f4480dbc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f447c06fa70 0x7f447c06fec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35948/0 (socket says 172.21.15.139:35948) 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.537+0000 7f4480dbc640 1 -- 172.21.15.139:0/3850149711 learned_addr learned my addr 172.21.15.139:0/3850149711 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447b7fe640 1 --2- 172.21.15.139:0/3850149711 >> v2:172.21.15.150:3300/0 conn(0x7f447c06ab60 0x7f447c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447bfff640 1 --2- 172.21.15.139:0/3850149711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c160e90 0x7f447c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331826509 0 0) 0x7f447c0d0400 con 0x7f447c06fa70 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4464003520 con 0x7f447c06fa70 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2400573337 0 0) 0x7f4464003520 con 0x7f447c06fa70 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c160e90 msgr2=0x7f447c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 --2- 172.21.15.139:0/3850149711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c160e90 0x7f447c0afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 >> v2:172.21.15.150:3300/0 conn(0x7f447c06ab60 msgr2=0x7f447c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 --2- 172.21.15.139:0/3850149711 >> v2:172.21.15.150:3300/0 conn(0x7f447c06ab60 0x7f447c06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c0d06e0 con 0x7f447c06fa70 2024-04-25T09:54:57.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f44740019f0 con 0x7f447c06fa70 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 >> v1:172.21.15.139:6789/0 conn(0x7f447c06fa70 legacy=0x7f447c06fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 --2- 172.21.15.139:0/3850149711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44640039b0 0x7f4464003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 --2- 172.21.15.139:0/3850149711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c160e90 0x7f4464004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 --2- 172.21.15.139:0/3850149711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4464004d40 0x7f44640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0d14a0 con 0x7f4464004d40 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446400b9c0 con 0x7f44640039b0 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446400bb40 con 0x7f447c160e90 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f4480dbc640 1 --2- 172.21.15.139:0/3850149711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4464004d40 0x7f44640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447bfff640 1 --2- 172.21.15.139:0/3850149711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44640039b0 0x7f4464003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f447b7fe640 1 --2- 172.21.15.139:0/3850149711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c160e90 0x7f4464004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f4480dbc640 1 -- 172.21.15.139:0/3850149711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c160e90 msgr2=0x7f4464004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f4480dbc640 1 --2- 172.21.15.139:0/3850149711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c160e90 0x7f4464004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f4480dbc640 1 -- 172.21.15.139:0/3850149711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44640039b0 msgr2=0x7f4464003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f4480dbc640 1 --2- 172.21.15.139:0/3850149711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44640039b0 0x7f4464003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.538+0000 7f4480dbc640 1 -- 172.21.15.139:0/3850149711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464003520 con 0x7f4464004d40 2024-04-25T09:54:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.539+0000 7f4480dbc640 1 --2- 172.21.15.139:0/3850149711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4464004d40 0x7f44640051c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f447400a870 tx=0x7f4464005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c9d9197e718c72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.539+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447400a4c0 con 0x7f4464004d40 2024-04-25T09:54:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.539+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447402f070 con 0x7f4464004d40 2024-04-25T09:54:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.539+0000 7f447affd640 1 -- 172.21.15.139:0/3850149711 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447402bcc0 con 0x7f4464004d40 2024-04-25T09:54:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.540+0000 7f448281f640 1 -- 172.21.15.139:0/3850149711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4464004d40 msgr2=0x7f44640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.540+0000 7f448281f640 1 --2- 172.21.15.139:0/3850149711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4464004d40 0x7f44640051c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f447400a870 tx=0x7f4464005830 comp rx=0 tx=0).stop 2024-04-25T09:54:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.540+0000 7f4480dbc640 1 -- 172.21.15.139:0/3850149711 reap_dead start 2024-04-25T09:54:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.540+0000 7f448281f640 1 -- 172.21.15.139:0/3850149711 shutdown_connections 2024-04-25T09:54:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.540+0000 7f448281f640 1 -- 172.21.15.139:0/3850149711 >> 172.21.15.139:0/3850149711 conn(0x7f447c0b23f0 msgr2=0x7f447c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.540+0000 7f448281f640 1 -- 172.21.15.139:0/3850149711 shutdown_connections 2024-04-25T09:54:57.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.540+0000 7f448281f640 1 -- 172.21.15.139:0/3850149711 wait complete. 2024-04-25T09:54:57.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.541+0000 7f448281f640 1 Processor -- start 2024-04-25T09:54:57.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.541+0000 7f448281f640 1 -- start start 2024-04-25T09:54:57.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f448281f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c14c120 0x7f447c15f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f448281f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f447c14c500 0x7f447c1576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f448281f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f447c159bf0 0x7f447c157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f448281f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0d0f80 con 0x7f447c159bf0 2024-04-25T09:54:57.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f448281f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0d0ca0 con 0x7f447c14c500 2024-04-25T09:54:57.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f448281f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c161270 con 0x7f447c14c120 2024-04-25T09:54:57.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f4480dbc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f447c159bf0 0x7f447c157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f4480dbc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f447c159bf0 0x7f447c157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47122/0 (socket says 172.21.15.139:47122) 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f4480dbc640 1 -- 172.21.15.139:0/645951396 learned_addr learned my addr 172.21.15.139:0/645951396 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f447b7fe640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f447c14c500 0x7f447c1576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f447bfff640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c14c120 0x7f447c15f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f4480dbc640 1 -- 172.21.15.139:0/645951396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c14c120 msgr2=0x7f447c15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f4480dbc640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c14c120 0x7f447c15f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f4480dbc640 1 -- 172.21.15.139:0/645951396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f447c14c500 msgr2=0x7f447c1576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f4480dbc640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f447c14c500 0x7f447c1576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f4480dbc640 1 -- 172.21.15.139:0/645951396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c15f920 con 0x7f447c159bf0 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f4480dbc640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f447c159bf0 0x7f447c157c10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f447400a0b0 tx=0x7f447c0d05d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f4478ff9640 1 -- 172.21.15.139:0/645951396 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474002030 con 0x7f447c159bf0 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f4478ff9640 1 -- 172.21.15.139:0/645951396 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4474034470 con 0x7f447c159bf0 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f448281f640 1 -- 172.21.15.139:0/645951396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c15fc00 con 0x7f447c159bf0 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f4478ff9640 1 -- 172.21.15.139:0/645951396 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474034860 con 0x7f447c159bf0 2024-04-25T09:54:57.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.542+0000 7f448281f640 1 -- 172.21.15.139:0/645951396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c1571b0 con 0x7f447c159bf0 2024-04-25T09:54:57.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.543+0000 7f448281f640 1 -- 172.21.15.139:0/645951396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c1573b0 con 0x7f447c159bf0 2024-04-25T09:54:57.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.544+0000 7f4478ff9640 1 -- 172.21.15.139:0/645951396 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4474034a00 con 0x7f447c159bf0 2024-04-25T09:54:57.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.544+0000 7f4478ff9640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f446806b500 0x7f446806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.545+0000 7f447bfff640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f446806b500 0x7f446806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.545+0000 7f4478ff9640 1 -- 172.21.15.139:0/645951396 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f44740aa000 con 0x7f447c159bf0 2024-04-25T09:54:57.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.545+0000 7f447bfff640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f446806b500 0x7f446806d9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f447c1622b0 tx=0x7f446c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:57.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.549+0000 7f4478ff9640 1 -- 172.21.15.139:0/645951396 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f447c1573b0 con 0x7f447c159bf0 2024-04-25T09:54:57.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.719+0000 7f448281f640 1 -- 172.21.15.139:0/645951396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f447c0d0140 con 0x7f447c159bf0 2024-04-25T09:54:57.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.721+0000 7f4478ff9640 1 -- 172.21.15.139:0/645951396 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+17003 (secure 0 0 0) 0x7f4474078e40 con 0x7f447c159bf0 2024-04-25T09:54:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:57.723 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":49,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:56.465907+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:54.661529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:55.431933+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:57.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.723+0000 7f448281f640 1 -- 172.21.15.139:0/645951396 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f446806b500 msgr2=0x7f446806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.723+0000 7f448281f640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f446806b500 0x7f446806d9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f447c1622b0 tx=0x7f446c009190 comp rx=0 tx=0).stop 2024-04-25T09:54:57.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.723+0000 7f448281f640 1 -- 172.21.15.139:0/645951396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f447c159bf0 msgr2=0x7f447c157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.723+0000 7f448281f640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f447c159bf0 0x7f447c157c10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f447400a0b0 tx=0x7f447c0d05d0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.723+0000 7f448281f640 1 -- 172.21.15.139:0/645951396 shutdown_connections 2024-04-25T09:54:57.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.723+0000 7f448281f640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f447c159bf0 0x7f447c157c10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.723+0000 7f448281f640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f446806b500 0x7f446806d9c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.723+0000 7f448281f640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f447c14c500 0x7f447c1576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.723+0000 7f448281f640 1 --2- 172.21.15.139:0/645951396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f447c14c120 0x7f447c15f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.723+0000 7f448281f640 1 -- 172.21.15.139:0/645951396 >> 172.21.15.139:0/645951396 conn(0x7f447c0b23f0 msgr2=0x7f447c06fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:57.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.723+0000 7f448281f640 1 -- 172.21.15.139:0/645951396 shutdown_connections 2024-04-25T09:54:57.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.723+0000 7f448281f640 1 -- 172.21.15.139:0/645951396 wait complete. 2024-04-25T09:54:57.736 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:54:57.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.834+0000 7f70cd777640 1 Processor -- start 2024-04-25T09:54:57.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.834+0000 7f70cd777640 1 -- start start 2024-04-25T09:54:57.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.834+0000 7f70cd777640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f70c815dbf0 0x7f70c81617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70cd777640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c80b9370 0x7f70c80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70cd777640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70c80cec80 0x7f70c806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70cd777640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c80d1ee0 con 0x7f70c80cec80 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70cd777640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c8161dc0 con 0x7f70c815dbf0 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70cd777640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c81620a0 con 0x7f70c80b9370 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70c77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70c80cec80 0x7f70c806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70c77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70c80cec80 0x7f70c806c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47124/0 (socket says 172.21.15.139:47124) 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70c77fe640 1 -- 172.21.15.139:0/17392232 learned_addr learned my addr 172.21.15.139:0/17392232 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70c6ffd640 1 --2- 172.21.15.139:0/17392232 >> v2:172.21.15.150:3300/0 conn(0x7f70c815dbf0 0x7f70c81617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70bffff640 1 --2- 172.21.15.139:0/17392232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c80b9370 0x7f70c80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70c77fe640 1 -- 172.21.15.139:0/17392232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c80b9370 msgr2=0x7f70c80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70c77fe640 1 --2- 172.21.15.139:0/17392232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c80b9370 0x7f70c80ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70c77fe640 1 -- 172.21.15.139:0/17392232 >> v2:172.21.15.150:3300/0 conn(0x7f70c815dbf0 msgr2=0x7f70c81617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70c77fe640 1 --2- 172.21.15.139:0/17392232 >> v2:172.21.15.150:3300/0 conn(0x7f70c815dbf0 0x7f70c81617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70c77fe640 1 -- 172.21.15.139:0/17392232 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c8162380 con 0x7f70c80cec80 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.835+0000 7f70bffff640 1 --2- 172.21.15.139:0/17392232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c80b9370 0x7f70c80ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:57.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70c77fe640 1 --2- 172.21.15.139:0/17392232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c80cec80 0x7f70c806c210 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f70c806d330 tx=0x7f70c002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2a1d5bcf3561df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:57.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70c67fc640 1 -- 172.21.15.139:0/17392232 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0036070 con 0x7f70c80cec80 2024-04-25T09:54:57.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70c67fc640 1 -- 172.21.15.139:0/17392232 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c0031600 con 0x7f70c80cec80 2024-04-25T09:54:57.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70c67fc640 1 -- 172.21.15.139:0/17392232 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00319d0 con 0x7f70c80cec80 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70cd777640 1 -- 172.21.15.139:0/17392232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c80cec80 msgr2=0x7f70c806c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70cd777640 1 --2- 172.21.15.139:0/17392232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c80cec80 0x7f70c806c210 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f70c806d330 tx=0x7f70c002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70cd777640 1 -- 172.21.15.139:0/17392232 shutdown_connections 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70cd777640 1 --2- 172.21.15.139:0/17392232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c80cec80 0x7f70c806c210 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70cd777640 1 --2- 172.21.15.139:0/17392232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c80b9370 0x7f70c80ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70cd777640 1 --2- 172.21.15.139:0/17392232 >> v2:172.21.15.150:3300/0 conn(0x7f70c815dbf0 0x7f70c81617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70cd777640 1 -- 172.21.15.139:0/17392232 >> 172.21.15.139:0/17392232 conn(0x7f70c80b5480 msgr2=0x7f70c80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70cd777640 1 -- 172.21.15.139:0/17392232 shutdown_connections 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.836+0000 7f70cd777640 1 -- 172.21.15.139:0/17392232 wait complete. 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.838+0000 7f70cd777640 1 Processor -- start 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.838+0000 7f70cd777640 1 -- start start 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.838+0000 7f70cd777640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c80b9370 0x7f70c8160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.838+0000 7f70cd777640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c80cec80 0x7f70c80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.838+0000 7f70cd777640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c815dbf0 0x7f70c815b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.838+0000 7f70cd777640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c815dfd0 con 0x7f70c80b9370 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.838+0000 7f70cd777640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c80b97d0 con 0x7f70c815dbf0 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.838+0000 7f70cd777640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c815fbe0 con 0x7f70c80cec80 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.838+0000 7f70c6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c80b9370 0x7f70c8160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.838+0000 7f70c6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c80b9370 0x7f70c8160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47138/0 (socket says 172.21.15.139:47138) 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.838+0000 7f70c6ffd640 1 -- 172.21.15.139:0/3334698786 learned_addr learned my addr 172.21.15.139:0/3334698786 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70c77fe640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c815dbf0 0x7f70c815b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70bffff640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c80cec80 0x7f70c80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70c6ffd640 1 -- 172.21.15.139:0/3334698786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c80cec80 msgr2=0x7f70c80c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70c6ffd640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c80cec80 0x7f70c80c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70c6ffd640 1 -- 172.21.15.139:0/3334698786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c815dbf0 msgr2=0x7f70c815b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70c6ffd640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c815dbf0 0x7f70c815b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70c6ffd640 1 -- 172.21.15.139:0/3334698786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c8161280 con 0x7f70c80b9370 2024-04-25T09:54:57.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70bffff640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c80cec80 0x7f70c80c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:57.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70c77fe640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c815dbf0 0x7f70c815b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:57.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70c6ffd640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c80b9370 0x7f70c8160bf0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f70b800e9b0 tx=0x7f70b800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:57.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70bf7fe640 1 -- 172.21.15.139:0/3334698786 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b8017070 con 0x7f70c80b9370 2024-04-25T09:54:57.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70bf7fe640 1 -- 172.21.15.139:0/3334698786 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b800c9a0 con 0x7f70c80b9370 2024-04-25T09:54:57.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70cd777640 1 -- 172.21.15.139:0/3334698786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c815ee20 con 0x7f70c80b9370 2024-04-25T09:54:57.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70bf7fe640 1 -- 172.21.15.139:0/3334698786 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b800cd90 con 0x7f70c80b9370 2024-04-25T09:54:57.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.839+0000 7f70cd777640 1 -- 172.21.15.139:0/3334698786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c815f6c0 con 0x7f70c80b9370 2024-04-25T09:54:57.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.840+0000 7f70cd777640 1 -- 172.21.15.139:0/3334698786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c8074520 con 0x7f70c80b9370 2024-04-25T09:54:57.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.840+0000 7f70bf7fe640 1 -- 172.21.15.139:0/3334698786 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70b801a070 con 0x7f70c80b9370 2024-04-25T09:54:57.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.841+0000 7f70bf7fe640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f70a806b4d0 0x7f70a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:57.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.841+0000 7f70bf7fe640 1 -- 172.21.15.139:0/3334698786 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f70b808a310 con 0x7f70c80b9370 2024-04-25T09:54:57.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.841+0000 7f70bffff640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f70a806b4d0 0x7f70a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:57.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.841+0000 7f70bffff640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f70a806b4d0 0x7f70a806d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f70b0006fd0 tx=0x7f70b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:57.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:57.845+0000 7f70bf7fe640 1 -- 172.21.15.139:0/3334698786 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70b8059090 con 0x7f70c80b9370 2024-04-25T09:54:58.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.051+0000 7f70cd777640 1 -- 172.21.15.139:0/3334698786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f70c80d1680 con 0x7f70c80b9370 2024-04-25T09:54:58.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.133+0000 7f70bf7fe640 1 -- 172.21.15.139:0/3334698786 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v21) v1 ==== 118+0+0 (secure 0 0 0) 0x7f70b8058990 con 0x7f70c80b9370 2024-04-25T09:54:58.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.136+0000 7f70cd777640 1 -- 172.21.15.139:0/3334698786 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f70a806b4d0 msgr2=0x7f70a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:58.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.136+0000 7f70cd777640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f70a806b4d0 0x7f70a806d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f70b0006fd0 tx=0x7f70b0008040 comp rx=0 tx=0).stop 2024-04-25T09:54:58.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.136+0000 7f70cd777640 1 -- 172.21.15.139:0/3334698786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c80b9370 msgr2=0x7f70c8160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:58.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.136+0000 7f70cd777640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c80b9370 0x7f70c8160bf0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f70b800e9b0 tx=0x7f70b800ee70 comp rx=0 tx=0).stop 2024-04-25T09:54:58.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.136+0000 7f70cd777640 1 -- 172.21.15.139:0/3334698786 shutdown_connections 2024-04-25T09:54:58.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.136+0000 7f70cd777640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f70a806b4d0 0x7f70a806d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.136+0000 7f70cd777640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c815dbf0 0x7f70c815b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.136+0000 7f70cd777640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c80cec80 0x7f70c80c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.136+0000 7f70cd777640 1 --2- 172.21.15.139:0/3334698786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70c80b9370 0x7f70c8160bf0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.136+0000 7f70cd777640 1 -- 172.21.15.139:0/3334698786 >> 172.21.15.139:0/3334698786 conn(0x7f70c80b5480 msgr2=0x7f70c80b6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:58.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.136+0000 7f70cd777640 1 -- 172.21.15.139:0/3334698786 shutdown_connections 2024-04-25T09:54:58.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.136+0000 7f70cd777640 1 -- 172.21.15.139:0/3334698786 wait complete. 2024-04-25T09:54:58.150 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:54:58.151 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:54:58.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.247+0000 7f591005a640 1 Processor -- start 2024-04-25T09:54:58.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.247+0000 7f591005a640 1 -- start start 2024-04-25T09:54:58.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.248+0000 7f591005a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f59080afc00 0x7f590806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:58.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.248+0000 7f591005a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f590815cb20 0x7f59080cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:58.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.248+0000 7f591005a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59080d02c0 con 0x7f59080cfeb0 2024-04-25T09:54:58.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.248+0000 7f591005a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59080703c0 con 0x7f59080afc00 2024-04-25T09:54:58.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.248+0000 7f591005a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59080cd820 con 0x7f590815cb20 2024-04-25T09:54:58.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.248+0000 7f590ddf6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f59080afc00 0x7f590806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:58.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.248+0000 7f590e5f7640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f59080cfeb0 0x7f590806ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35950/0 (socket says 172.21.15.139:35950) 2024-04-25T09:54:58.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.248+0000 7f590e5f7640 1 -- 172.21.15.139:0/1292722815 learned_addr learned my addr 172.21.15.139:0/1292722815 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:58.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.249+0000 7f590d5f5640 1 --2- 172.21.15.139:0/1292722815 >> v2:172.21.15.150:3300/0 conn(0x7f590815cb20 0x7f59080cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:58.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.249+0000 7f590ddf6640 1 -- 172.21.15.139:0/1292722815 >> v2:172.21.15.150:3300/0 conn(0x7f590815cb20 msgr2=0x7f59080cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.249+0000 7f590ddf6640 1 --2- 172.21.15.139:0/1292722815 >> v2:172.21.15.150:3300/0 conn(0x7f590815cb20 0x7f59080cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.249+0000 7f590cdf4640 1 -- 172.21.15.139:0/1292722815 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351758 0 0) 0x7f59080d02c0 con 0x7f59080cfeb0 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.249+0000 7f590ddf6640 1 -- 172.21.15.139:0/1292722815 >> v1:172.21.15.139:6789/0 conn(0x7f59080cfeb0 legacy=0x7f590806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.249+0000 7f590ddf6640 1 -- 172.21.15.139:0/1292722815 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59080ccef0 con 0x7f59080afc00 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.249+0000 7f590ddf6640 1 --2- 172.21.15.139:0/1292722815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080afc00 0x7f590806fb20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f590815f3d0 tx=0x7f590402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36af8682209222f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.249+0000 7f590cdf4640 1 -- 172.21.15.139:0/1292722815 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904037070 con 0x7f59080afc00 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.249+0000 7f590cdf4640 1 -- 172.21.15.139:0/1292722815 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590402add0 con 0x7f59080afc00 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.249+0000 7f590cdf4640 1 -- 172.21.15.139:0/1292722815 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904032660 con 0x7f59080afc00 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.250+0000 7f591005a640 1 -- 172.21.15.139:0/1292722815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080afc00 msgr2=0x7f590806fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.250+0000 7f591005a640 1 --2- 172.21.15.139:0/1292722815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080afc00 0x7f590806fb20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f590815f3d0 tx=0x7f590402a080 comp rx=0 tx=0).stop 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.250+0000 7f591005a640 1 -- 172.21.15.139:0/1292722815 shutdown_connections 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.250+0000 7f591005a640 1 --2- 172.21.15.139:0/1292722815 >> v2:172.21.15.150:3300/0 conn(0x7f590815cb20 0x7f59080cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.250+0000 7f591005a640 1 --2- 172.21.15.139:0/1292722815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080afc00 0x7f590806fb20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.250+0000 7f591005a640 1 -- 172.21.15.139:0/1292722815 >> 172.21.15.139:0/1292722815 conn(0x7f59080b2230 msgr2=0x7f59080b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:58.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.250+0000 7f591005a640 1 -- 172.21.15.139:0/1292722815 shutdown_connections 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.250+0000 7f591005a640 1 -- 172.21.15.139:0/1292722815 wait complete. 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.251+0000 7f591005a640 1 Processor -- start 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.251+0000 7f591005a640 1 -- start start 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f591005a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59080afc00 0x7f590814c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f591005a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080cfeb0 0x7f59080ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f591005a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f590815cb20 0x7f59080aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f591005a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59080cd4b0 con 0x7f59080cfeb0 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f591005a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59080affe0 con 0x7f590815cb20 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f591005a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5908070060 con 0x7f59080afc00 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f590d5f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080cfeb0 0x7f59080ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f590d5f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080cfeb0 0x7f59080ae560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47154/0 (socket says 172.21.15.139:47154) 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f590d5f5640 1 -- 172.21.15.139:0/1820439244 learned_addr learned my addr 172.21.15.139:0/1820439244 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f590e5f7640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f590815cb20 0x7f59080aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f590ddf6640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59080afc00 0x7f590814c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:58.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f590d5f5640 1 -- 172.21.15.139:0/1820439244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59080afc00 msgr2=0x7f590814c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f590d5f5640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59080afc00 0x7f590814c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f590d5f5640 1 -- 172.21.15.139:0/1820439244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f590815cb20 msgr2=0x7f59080aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f590d5f5640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f590815cb20 0x7f59080aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f590d5f5640 1 -- 172.21.15.139:0/1820439244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59080b41d0 con 0x7f59080cfeb0 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.252+0000 7f590ddf6640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59080afc00 0x7f590814c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.253+0000 7f590d5f5640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080cfeb0 0x7f59080ae560 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f58f800c820 tx=0x7f58f800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.253+0000 7f58fe7fc640 1 -- 172.21.15.139:0/1820439244 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8015070 con 0x7f59080cfeb0 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.253+0000 7f58fe7fc640 1 -- 172.21.15.139:0/1820439244 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f8004020 con 0x7f59080cfeb0 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.253+0000 7f58fe7fc640 1 -- 172.21.15.139:0/1820439244 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f80043f0 con 0x7f59080cfeb0 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.253+0000 7f591005a640 1 -- 172.21.15.139:0/1820439244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59080c55d0 con 0x7f59080cfeb0 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.253+0000 7f591005a640 1 -- 172.21.15.139:0/1820439244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59080c5b90 con 0x7f59080cfeb0 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.254+0000 7f591005a640 1 -- 172.21.15.139:0/1820439244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5908074520 con 0x7f59080cfeb0 2024-04-25T09:54:58.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.254+0000 7f58fe7fc640 1 -- 172.21.15.139:0/1820439244 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58f8007830 con 0x7f59080cfeb0 2024-04-25T09:54:58.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.255+0000 7f58fe7fc640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f58ec06b4d0 0x7f58ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:58.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.255+0000 7f590ddf6640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f58ec06b4d0 0x7f58ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:58.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.255+0000 7f58fe7fc640 1 -- 172.21.15.139:0/1820439244 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58f808a0e0 con 0x7f59080cfeb0 2024-04-25T09:54:58.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.255+0000 7f590ddf6640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f58ec06b4d0 0x7f58ec06d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5904034000 tx=0x7f590402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:58.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.259+0000 7f58fe7fc640 1 -- 172.21.15.139:0/1820439244 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58f8058e60 con 0x7f59080cfeb0 2024-04-25T09:54:58.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:58.464+0000 7f591005a640 1 -- 172.21.15.139:0/1820439244 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f59080d0ae0 con 0x7f59080cfeb0 2024-04-25T09:54:59.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.138+0000 7f58fe7fc640 1 -- 172.21.15.139:0/1820439244 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f58f8058760 con 0x7f59080cfeb0 2024-04-25T09:54:59.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.140+0000 7f591005a640 1 -- 172.21.15.139:0/1820439244 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f58ec06b4d0 msgr2=0x7f58ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.140+0000 7f591005a640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f58ec06b4d0 0x7f58ec06d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5904034000 tx=0x7f590402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:59.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.140+0000 7f591005a640 1 -- 172.21.15.139:0/1820439244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080cfeb0 msgr2=0x7f59080ae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.140+0000 7f591005a640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080cfeb0 0x7f59080ae560 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f58f800c820 tx=0x7f58f800cce0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.141+0000 7f591005a640 1 -- 172.21.15.139:0/1820439244 shutdown_connections 2024-04-25T09:54:59.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.141+0000 7f591005a640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080cfeb0 0x7f59080ae560 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.141+0000 7f591005a640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f58ec06b4d0 0x7f58ec06d990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.141+0000 7f591005a640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f590815cb20 0x7f59080aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.141+0000 7f591005a640 1 --2- 172.21.15.139:0/1820439244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59080afc00 0x7f590814c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.141+0000 7f591005a640 1 -- 172.21.15.139:0/1820439244 >> 172.21.15.139:0/1820439244 conn(0x7f59080b2230 msgr2=0x7f59080b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:59.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.141+0000 7f591005a640 1 -- 172.21.15.139:0/1820439244 shutdown_connections 2024-04-25T09:54:59.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.141+0000 7f591005a640 1 -- 172.21.15.139:0/1820439244 wait complete. 2024-04-25T09:54:59.157 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:54:59.157 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:59.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.255+0000 7ffab04dd640 1 Processor -- start 2024-04-25T09:54:59.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.255+0000 7ffab04dd640 1 -- start start 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffab04dd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffaa815cde0 0x7ffaa815d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffab04dd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffaa80cc2a0 0x7ffaa809a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffab04dd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa815d1c0 con 0x7ffaa815db50 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffab04dd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa80cc720 con 0x7ffaa815cde0 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffab04dd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa80cd460 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffaae279640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffaa815cde0 0x7ffaa815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffaae279640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffaa815cde0 0x7ffaa815d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47156/0 (socket says 172.21.15.139:47156) 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffaaea7a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ffaa815db50 0x7ffaa80cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35966/0 (socket says 172.21.15.139:35966) 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffaae279640 1 -- 172.21.15.139:0/4085099814 learned_addr learned my addr 172.21.15.139:0/4085099814 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffaada78640 1 --2- 172.21.15.139:0/4085099814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffaa80cc2a0 0x7ffaa809a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffaae279640 1 -- 172.21.15.139:0/4085099814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffaa80cc2a0 msgr2=0x7ffaa809a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffaae279640 1 --2- 172.21.15.139:0/4085099814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffaa80cc2a0 0x7ffaa809a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffaae279640 1 -- 172.21.15.139:0/4085099814 >> v1:172.21.15.139:6789/0 conn(0x7ffaa815db50 legacy=0x7ffaa80cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffaad277640 1 -- 172.21.15.139:0/4085099814 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019511459 0 0) 0x7ffaa815d1c0 con 0x7ffaa815db50 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.256+0000 7ffaae279640 1 -- 172.21.15.139:0/4085099814 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8099bf0 con 0x7ffaa815cde0 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.257+0000 7ffaae279640 1 --2- 172.21.15.139:0/4085099814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa815cde0 0x7ffaa815d610 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009870 tx=0x7ffaa402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b4afe9b442090e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.257+0000 7ffaad277640 1 -- 172.21.15.139:0/4085099814 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4037070 con 0x7ffaa815cde0 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.257+0000 7ffaad277640 1 -- 172.21.15.139:0/4085099814 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa4032440 con 0x7ffaa815cde0 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.257+0000 7ffaad277640 1 -- 172.21.15.139:0/4085099814 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4032810 con 0x7ffaa815cde0 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.257+0000 7ffab04dd640 1 -- 172.21.15.139:0/4085099814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa815cde0 msgr2=0x7ffaa815d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.257+0000 7ffab04dd640 1 --2- 172.21.15.139:0/4085099814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa815cde0 0x7ffaa815d610 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009870 tx=0x7ffaa402a270 comp rx=0 tx=0).stop 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.258+0000 7ffab04dd640 1 -- 172.21.15.139:0/4085099814 shutdown_connections 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.258+0000 7ffab04dd640 1 --2- 172.21.15.139:0/4085099814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffaa80cc2a0 0x7ffaa809a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.258+0000 7ffab04dd640 1 --2- 172.21.15.139:0/4085099814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa815cde0 0x7ffaa815d610 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.258+0000 7ffab04dd640 1 -- 172.21.15.139:0/4085099814 >> 172.21.15.139:0/4085099814 conn(0x7ffaa80d1790 msgr2=0x7ffaa80d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.258+0000 7ffab04dd640 1 -- 172.21.15.139:0/4085099814 shutdown_connections 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.258+0000 7ffab04dd640 1 -- 172.21.15.139:0/4085099814 wait complete. 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.259+0000 7ffab04dd640 1 Processor -- start 2024-04-25T09:54:59.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.259+0000 7ffab04dd640 1 -- start start 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.259+0000 7ffab04dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa80cc2a0 0x7ffaa8157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.259+0000 7ffab04dd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffaa815cde0 0x7ffaa815a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.259+0000 7ffab04dd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffaa815db50 0x7ffaa809a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.259+0000 7ffab04dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa80cf4e0 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.259+0000 7ffab04dd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa80cf360 con 0x7ffaa815db50 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.259+0000 7ffab04dd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa815a2c0 con 0x7ffaa815cde0 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.259+0000 7ffaae279640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa80cc2a0 0x7ffaa8157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffaae279640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa80cc2a0 0x7ffaa8157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47160/0 (socket says 172.21.15.139:47160) 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffaae279640 1 -- 172.21.15.139:0/4057722631 learned_addr learned my addr 172.21.15.139:0/4057722631 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffaae279640 1 -- 172.21.15.139:0/4057722631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffaa815cde0 msgr2=0x7ffaa815a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffaae279640 1 --2- 172.21.15.139:0/4057722631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffaa815cde0 0x7ffaa815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffaae279640 1 -- 172.21.15.139:0/4057722631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffaa815db50 msgr2=0x7ffaa809a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffaae279640 1 --2- 172.21.15.139:0/4057722631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffaa815db50 0x7ffaa809a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffaae279640 1 -- 172.21.15.139:0/4057722631 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8147650 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffaae279640 1 --2- 172.21.15.139:0/4057722631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa80cc2a0 0x7ffaa8157f20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009870 tx=0x7ffaa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffa9effd640 1 -- 172.21.15.139:0/4057722631 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4041020 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffa9effd640 1 -- 172.21.15.139:0/4057722631 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa40306c0 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffab04dd640 1 -- 172.21.15.139:0/4057722631 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa8159c70 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffa9effd640 1 -- 172.21.15.139:0/4057722631 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4037040 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.260+0000 7ffab04dd640 1 -- 172.21.15.139:0/4057722631 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa815ac90 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.261+0000 7ffab04dd640 1 -- 172.21.15.139:0/4057722631 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa8154200 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.262+0000 7ffa9effd640 1 -- 172.21.15.139:0/4057722631 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffaa4044020 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.262+0000 7ffa9effd640 1 --2- 172.21.15.139:0/4057722631 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffa8c06b500 0x7ffa8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.262+0000 7ffaada78640 1 --2- 172.21.15.139:0/4057722631 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffa8c06b500 0x7ffa8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.262+0000 7ffa9effd640 1 -- 172.21.15.139:0/4057722631 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffaa40aaa60 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.262+0000 7ffaada78640 1 --2- 172.21.15.139:0/4057722631 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffa8c06b500 0x7ffa8c06d9c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffa98006fd0 tx=0x7ffa98008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:59.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.266+0000 7ffa9effd640 1 -- 172.21.15.139:0/4057722631 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffaa8154200 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.470+0000 7ffab04dd640 1 -- 172.21.15.139:0/4057722631 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaa8074520 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.470+0000 7ffa9effd640 1 -- 172.21.15.139:0/4057722631 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4947 (secure 0 0 0) 0x7ffaa404b030 con 0x7ffaa80cc2a0 2024-04-25T09:54:59.474 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:59.474 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1742557158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1742557158},{"type":"v1","addr":"172.21.15.150:6834","nonce":1742557158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3496803834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3496803834},{"type":"v1","addr":"172.21.15.150:6835","nonce":3496803834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2014537692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2014537692},{"type":"v1","addr":"172.21.15.139:6835","nonce":2014537692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:56.466203+0000","modified":"2024-04-25T09:54:58.466766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5850":{"gid":5850,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/338275708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":338275708},{"type":"v1","addr":"172.21.15.139:6837","nonce":338275708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:54:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 22 2024-04-25T09:54:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.473+0000 7ffab04dd640 1 -- 172.21.15.139:0/4057722631 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffa8c06b500 msgr2=0x7ffa8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.473+0000 7ffab04dd640 1 --2- 172.21.15.139:0/4057722631 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffa8c06b500 0x7ffa8c06d9c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffa98006fd0 tx=0x7ffa98008040 comp rx=0 tx=0).stop 2024-04-25T09:54:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.473+0000 7ffab04dd640 1 -- 172.21.15.139:0/4057722631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa80cc2a0 msgr2=0x7ffaa8157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.473+0000 7ffab04dd640 1 --2- 172.21.15.139:0/4057722631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa80cc2a0 0x7ffaa8157f20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009870 tx=0x7ffaa40046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.473+0000 7ffab04dd640 1 -- 172.21.15.139:0/4057722631 shutdown_connections 2024-04-25T09:54:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.473+0000 7ffab04dd640 1 --2- 172.21.15.139:0/4057722631 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffa8c06b500 0x7ffa8c06d9c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.473+0000 7ffab04dd640 1 --2- 172.21.15.139:0/4057722631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffaa815db50 0x7ffaa809a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.473+0000 7ffab04dd640 1 --2- 172.21.15.139:0/4057722631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffaa815cde0 0x7ffaa815a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.473+0000 7ffab04dd640 1 --2- 172.21.15.139:0/4057722631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaa80cc2a0 0x7ffaa8157f20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.473+0000 7ffab04dd640 1 -- 172.21.15.139:0/4057722631 >> 172.21.15.139:0/4057722631 conn(0x7ffaa80d1790 msgr2=0x7ffaa80c78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.473+0000 7ffab04dd640 1 -- 172.21.15.139:0/4057722631 shutdown_connections 2024-04-25T09:54:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.473+0000 7ffab04dd640 1 -- 172.21.15.139:0/4057722631 wait complete. 2024-04-25T09:54:59.487 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:59.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.580+0000 7f8eb536a640 1 Processor -- start 2024-04-25T09:54:59.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.580+0000 7f8eb536a640 1 -- start start 2024-04-25T09:54:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eb536a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb015e2b0 0x7f8eb006ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eb536a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8eb00ae600 0x7f8eb00b4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eb536a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb015e740 con 0x7f8eb015a4c0 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eb536a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb015ea20 con 0x7f8eb00ae600 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eb536a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb015ed00 con 0x7f8eb015e2b0 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eaf7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8eb00ae600 0x7f8eb00b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eaf7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8eb00ae600 0x7f8eb00b4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47162/0 (socket says 172.21.15.139:47162) 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eaf7fe640 1 -- 172.21.15.139:0/342507017 learned_addr learned my addr 172.21.15.139:0/342507017 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eae7fc640 1 --2- 172.21.15.139:0/342507017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb015e2b0 0x7f8eb006ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eaf7fe640 1 -- 172.21.15.139:0/342507017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb015e2b0 msgr2=0x7f8eb006ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eaf7fe640 1 --2- 172.21.15.139:0/342507017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb015e2b0 0x7f8eb006ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eaf7fe640 1 -- 172.21.15.139:0/342507017 >> v1:172.21.15.150:6789/0 conn(0x7f8eb015a4c0 legacy=0x7f8eb006bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.581+0000 7f8eaf7fe640 1 -- 172.21.15.139:0/342507017 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb015efe0 con 0x7f8eb00ae600 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.582+0000 7f8eaf7fe640 1 --2- 172.21.15.139:0/342507017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00ae600 0x7f8eb00b4310 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009e50 tx=0x7f8ea802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fdb3478c16d83c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.582+0000 7f8eadffb640 1 -- 172.21.15.139:0/342507017 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8036070 con 0x7f8eb00ae600 2024-04-25T09:54:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.582+0000 7f8eadffb640 1 -- 172.21.15.139:0/342507017 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea8031600 con 0x7f8eb00ae600 2024-04-25T09:54:59.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.582+0000 7f8eadffb640 1 -- 172.21.15.139:0/342507017 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80319d0 con 0x7f8eb00ae600 2024-04-25T09:54:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.583+0000 7f8eb536a640 1 -- 172.21.15.139:0/342507017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00ae600 msgr2=0x7f8eb00b4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.583+0000 7f8eb536a640 1 --2- 172.21.15.139:0/342507017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00ae600 0x7f8eb00b4310 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009e50 tx=0x7f8ea802a7a0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.583+0000 7f8eb536a640 1 -- 172.21.15.139:0/342507017 shutdown_connections 2024-04-25T09:54:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.583+0000 7f8eb536a640 1 --2- 172.21.15.139:0/342507017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00ae600 0x7f8eb00b4310 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.583+0000 7f8eb536a640 1 --2- 172.21.15.139:0/342507017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb015e2b0 0x7f8eb006ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.583+0000 7f8eb536a640 1 -- 172.21.15.139:0/342507017 >> 172.21.15.139:0/342507017 conn(0x7f8eb0071a00 msgr2=0x7f8eb0071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.583+0000 7f8eb536a640 1 -- 172.21.15.139:0/342507017 shutdown_connections 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.583+0000 7f8eb536a640 1 -- 172.21.15.139:0/342507017 wait complete. 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.584+0000 7f8eb536a640 1 Processor -- start 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.584+0000 7f8eb536a640 1 -- start start 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eb536a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00ae600 0x7f8eb015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eb536a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb015a4c0 0x7f8eb014d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eb536a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb00ce820 0x7f8eb015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eb536a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb015a8a0 con 0x7f8eb00ae600 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eb536a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb006c490 con 0x7f8eb00ce820 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eb536a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb015aa20 con 0x7f8eb015a4c0 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eaeffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00ae600 0x7f8eb015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eaeffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00ae600 0x7f8eb015ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47172/0 (socket says 172.21.15.139:47172) 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eaeffd640 1 -- 172.21.15.139:0/377610108 learned_addr learned my addr 172.21.15.139:0/377610108 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eae7fc640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb015a4c0 0x7f8eb014d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eaf7fe640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb00ce820 0x7f8eb015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eaeffd640 1 -- 172.21.15.139:0/377610108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb015a4c0 msgr2=0x7f8eb014d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eaeffd640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb015a4c0 0x7f8eb014d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eaeffd640 1 -- 172.21.15.139:0/377610108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb00ce820 msgr2=0x7f8eb015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eaeffd640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb00ce820 0x7f8eb015c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eaeffd640 1 -- 172.21.15.139:0/377610108 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb00cecd0 con 0x7f8eb00ae600 2024-04-25T09:54:59.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eaeffd640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00ae600 0x7f8eb015ad00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000c400 tx=0x7f8ea000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:59.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8e977fe640 1 -- 172.21.15.139:0/377610108 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00052d0 con 0x7f8eb00ae600 2024-04-25T09:54:59.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8e977fe640 1 -- 172.21.15.139:0/377610108 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea0005490 con 0x7f8eb00ae600 2024-04-25T09:54:59.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eb536a640 1 -- 172.21.15.139:0/377610108 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb015b400 con 0x7f8eb00ae600 2024-04-25T09:54:59.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8e977fe640 1 -- 172.21.15.139:0/377610108 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0007260 con 0x7f8eb00ae600 2024-04-25T09:54:59.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.585+0000 7f8eb536a640 1 -- 172.21.15.139:0/377610108 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb015b9c0 con 0x7f8eb00ae600 2024-04-25T09:54:59.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.586+0000 7f8eb536a640 1 -- 172.21.15.139:0/377610108 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb0074520 con 0x7f8eb00ae600 2024-04-25T09:54:59.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.587+0000 7f8e977fe640 1 -- 172.21.15.139:0/377610108 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ea0007400 con 0x7f8eb00ae600 2024-04-25T09:54:59.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.587+0000 7f8e977fe640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.587+0000 7f8e977fe640 1 -- 172.21.15.139:0/377610108 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ea0089830 con 0x7f8eb00ae600 2024-04-25T09:54:59.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.587+0000 7f8eae7fc640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.588+0000 7f8eae7fc640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8eb01598b0 tx=0x7f8ea4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:59.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.592+0000 7f8e977fe640 1 -- 172.21.15.139:0/377610108 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ea0057ed0 con 0x7f8eb00ae600 2024-04-25T09:54:59.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.762+0000 7f8eb536a640 1 -- 172.21.15.139:0/377610108 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8eb015dfe0 con 0x7f8eb00ae600 2024-04-25T09:54:59.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.764+0000 7f8e977fe640 1 -- 172.21.15.139:0/377610108 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f8ea005bf70 con 0x7f8eb00ae600 2024-04-25T09:54:59.767 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:54:59.768 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":50,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:58.130580+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:54.661529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:55.431933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.766+0000 7f8eb536a640 1 -- 172.21.15.139:0/377610108 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8e9c06b4d0 msgr2=0x7f8e9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.766+0000 7f8eb536a640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8eb01598b0 tx=0x7f8ea4009190 comp rx=0 tx=0).stop 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.766+0000 7f8eb536a640 1 -- 172.21.15.139:0/377610108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00ae600 msgr2=0x7f8eb015ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.766+0000 7f8eb536a640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00ae600 0x7f8eb015ad00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000c400 tx=0x7f8ea000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.766+0000 7f8eb536a640 1 -- 172.21.15.139:0/377610108 shutdown_connections 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.766+0000 7f8eb536a640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.766+0000 7f8eb536a640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb00ce820 0x7f8eb015c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.766+0000 7f8eb536a640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8eb015a4c0 0x7f8eb014d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.766+0000 7f8eb536a640 1 --2- 172.21.15.139:0/377610108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00ae600 0x7f8eb015ad00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.766+0000 7f8eb536a640 1 -- 172.21.15.139:0/377610108 >> 172.21.15.139:0/377610108 conn(0x7f8eb0071a00 msgr2=0x7f8eb0073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.766+0000 7f8eb536a640 1 -- 172.21.15.139:0/377610108 shutdown_connections 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.766+0000 7f8eb536a640 1 -- 172.21.15.139:0/377610108 wait complete. 2024-04-25T09:54:59.780 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:54:59.781 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:54:59.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.874+0000 7f98bb163640 1 Processor -- start 2024-04-25T09:54:59.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.874+0000 7f98bb163640 1 -- start start 2024-04-25T09:54:59.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.875+0000 7f98bb163640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f98b415bdc0 0x7f98b415c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.875+0000 7f98bb163640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98b40b1740 0x7f98b40b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.875+0000 7f98bb163640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b415c880 con 0x7f98b4155e30 2024-04-25T09:54:59.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.875+0000 7f98bb163640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b415d690 con 0x7f98b415bdc0 2024-04-25T09:54:59.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.875+0000 7f98bb163640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b406d6f0 con 0x7f98b40b1740 2024-04-25T09:54:59.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.875+0000 7f98b8eff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f98b4155e30 0x7f98b415cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35980/0 (socket says 172.21.15.139:35980) 2024-04-25T09:54:59.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.875+0000 7f98b8eff640 1 -- 172.21.15.139:0/3888984117 learned_addr learned my addr 172.21.15.139:0/3888984117 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:54:59.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.875+0000 7f98b9700640 1 --2- 172.21.15.139:0/3888984117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98b40b1740 0x7f98b40b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.875+0000 7f98b3fff640 1 --2- 172.21.15.139:0/3888984117 >> v2:172.21.15.150:3300/0 conn(0x7f98b415bdc0 0x7f98b415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.875+0000 7f98b37fe640 1 -- 172.21.15.139:0/3888984117 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673061929 0 0) 0x7f98b415c880 con 0x7f98b4155e30 2024-04-25T09:54:59.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.875+0000 7f98b37fe640 1 -- 172.21.15.139:0/3888984117 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f989c003520 con 0x7f98b4155e30 2024-04-25T09:54:59.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.876+0000 7f98b9700640 1 -- 172.21.15.139:0/3888984117 >> v2:172.21.15.150:3300/0 conn(0x7f98b415bdc0 msgr2=0x7f98b415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:59.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.876+0000 7f98b9700640 1 --2- 172.21.15.139:0/3888984117 >> v2:172.21.15.150:3300/0 conn(0x7f98b415bdc0 0x7f98b415c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.876+0000 7f98b37fe640 1 -- 172.21.15.139:0/3888984117 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 994345917 0 0) 0x7f989c003520 con 0x7f98b4155e30 2024-04-25T09:54:59.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.876+0000 7f98b9700640 1 -- 172.21.15.139:0/3888984117 >> v1:172.21.15.139:6789/0 conn(0x7f98b4155e30 legacy=0x7f98b415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.876+0000 7f98b9700640 1 -- 172.21.15.139:0/3888984117 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b415cb60 con 0x7f98b40b1740 2024-04-25T09:54:59.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.876+0000 7f98b9700640 1 --2- 172.21.15.139:0/3888984117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98b40b1740 0x7f98b40b1bd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009d90 tx=0x7f98ac02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b929400021af8cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:59.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.877+0000 7f98b37fe640 1 -- 172.21.15.139:0/3888984117 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4001670 con 0x7f98b40b1740 2024-04-25T09:54:59.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.877+0000 7f98b37fe640 1 -- 172.21.15.139:0/3888984117 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac007540 con 0x7f98b40b1740 2024-04-25T09:54:59.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.877+0000 7f98b37fe640 1 -- 172.21.15.139:0/3888984117 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0078f0 con 0x7f98b40b1740 2024-04-25T09:54:59.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.877+0000 7f98bb163640 1 -- 172.21.15.139:0/3888984117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98b40b1740 msgr2=0x7f98b40b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.877+0000 7f98bb163640 1 --2- 172.21.15.139:0/3888984117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98b40b1740 0x7f98b40b1bd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009d90 tx=0x7f98ac02a5f0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.877+0000 7f98bb163640 1 -- 172.21.15.139:0/3888984117 shutdown_connections 2024-04-25T09:54:59.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.877+0000 7f98bb163640 1 --2- 172.21.15.139:0/3888984117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98b40b1740 0x7f98b40b1bd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.877+0000 7f98bb163640 1 --2- 172.21.15.139:0/3888984117 >> v2:172.21.15.150:3300/0 conn(0x7f98b415bdc0 0x7f98b415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.877+0000 7f98bb163640 1 -- 172.21.15.139:0/3888984117 >> 172.21.15.139:0/3888984117 conn(0x7f98b4066f20 msgr2=0x7f98b406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:59.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.877+0000 7f98bb163640 1 -- 172.21.15.139:0/3888984117 shutdown_connections 2024-04-25T09:54:59.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.877+0000 7f98bb163640 1 -- 172.21.15.139:0/3888984117 wait complete. 2024-04-25T09:54:59.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.878+0000 7f98bb163640 1 Processor -- start 2024-04-25T09:54:59.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.878+0000 7f98bb163640 1 -- start start 2024-04-25T09:54:59.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.879+0000 7f98bb163640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98b40b1740 0x7f98b41542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.879+0000 7f98bb163640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98b4155e30 0x7f98b40c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.879+0000 7f98bb163640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98b415bdc0 0x7f98b40b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.879+0000 7f98bb163640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b40af0e0 con 0x7f98b4155e30 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.879+0000 7f98bb163640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b4156210 con 0x7f98b40b1740 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.879+0000 7f98bb163640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b40af260 con 0x7f98b415bdc0 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.879+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98b4155e30 0x7f98b40c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.879+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98b4155e30 0x7f98b40c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47176/0 (socket says 172.21.15.139:47176) 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.879+0000 7f98b3fff640 1 -- 172.21.15.139:0/1569824362 learned_addr learned my addr 172.21.15.139:0/1569824362 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98b8eff640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98b40b1740 0x7f98b41542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98b9700640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98b415bdc0 0x7f98b40b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98b3fff640 1 -- 172.21.15.139:0/1569824362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98b415bdc0 msgr2=0x7f98b40b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98b3fff640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98b415bdc0 0x7f98b40b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98b3fff640 1 -- 172.21.15.139:0/1569824362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98b40b1740 msgr2=0x7f98b41542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98b3fff640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98b40b1740 0x7f98b41542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98b3fff640 1 -- 172.21.15.139:0/1569824362 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b41547f0 con 0x7f98b4155e30 2024-04-25T09:54:59.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98b3fff640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98b4155e30 0x7f98b40c4f60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f98a800e870 tx=0x7f98a800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:59.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98b17fa640 1 -- 172.21.15.139:0/1569824362 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8017070 con 0x7f98b4155e30 2024-04-25T09:54:59.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98b17fa640 1 -- 172.21.15.139:0/1569824362 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a800c940 con 0x7f98b4155e30 2024-04-25T09:54:59.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98bb163640 1 -- 172.21.15.139:0/1569824362 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b40b0470 con 0x7f98b4155e30 2024-04-25T09:54:59.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98b17fa640 1 -- 172.21.15.139:0/1569824362 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a800cd30 con 0x7f98b4155e30 2024-04-25T09:54:59.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.880+0000 7f98bb163640 1 -- 172.21.15.139:0/1569824362 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b40b0a30 con 0x7f98b4155e30 2024-04-25T09:54:59.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.881+0000 7f98bb163640 1 -- 172.21.15.139:0/1569824362 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b4074520 con 0x7f98b4155e30 2024-04-25T09:54:59.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.882+0000 7f98b17fa640 1 -- 172.21.15.139:0/1569824362 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98a801a070 con 0x7f98b4155e30 2024-04-25T09:54:59.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.882+0000 7f98b17fa640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f98a006b4d0 0x7f98a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.882+0000 7f98b8eff640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f98a006b4d0 0x7f98a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.882+0000 7f98b17fa640 1 -- 172.21.15.139:0/1569824362 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f98a808a1e0 con 0x7f98b4155e30 2024-04-25T09:54:59.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.883+0000 7f98b8eff640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f98a006b4d0 0x7f98a006d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f98a4005880 tx=0x7f98a40057f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:59.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:54:59.886+0000 7f98b17fa640 1 -- 172.21.15.139:0/1569824362 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98a80587f0 con 0x7f98b4155e30 2024-04-25T09:55:00.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.055+0000 7f98bb163640 1 -- 172.21.15.139:0/1569824362 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98b406b830 con 0x7f98a006b4d0 2024-04-25T09:55:00.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.061+0000 7f98b17fa640 1 -- 172.21.15.139:0/1569824362 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251889 (secure 0 0 0) 0x7f98b406b830 con 0x7f98a006b4d0 2024-04-25T09:55:00.067 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:00.069 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:55:00.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.064+0000 7f98bb163640 1 -- 172.21.15.139:0/1569824362 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f98a006b4d0 msgr2=0x7f98a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.064+0000 7f98bb163640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f98a006b4d0 0x7f98a006d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f98a4005880 tx=0x7f98a40057f0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.064+0000 7f98bb163640 1 -- 172.21.15.139:0/1569824362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98b4155e30 msgr2=0x7f98b40c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.064+0000 7f98bb163640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98b4155e30 0x7f98b40c4f60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f98a800e870 tx=0x7f98a800ed30 comp rx=0 tx=0).stop 2024-04-25T09:55:00.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.064+0000 7f98bb163640 1 -- 172.21.15.139:0/1569824362 shutdown_connections 2024-04-25T09:55:00.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.064+0000 7f98bb163640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98b4155e30 0x7f98b40c4f60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.064+0000 7f98bb163640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f98a006b4d0 0x7f98a006d990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.064+0000 7f98bb163640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98b415bdc0 0x7f98b40b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.064+0000 7f98bb163640 1 --2- 172.21.15.139:0/1569824362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98b40b1740 0x7f98b41542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.064+0000 7f98bb163640 1 -- 172.21.15.139:0/1569824362 >> 172.21.15.139:0/1569824362 conn(0x7f98b4066f20 msgr2=0x7f98b4073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:00.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.064+0000 7f98bb163640 1 -- 172.21.15.139:0/1569824362 shutdown_connections 2024-04-25T09:55:00.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.064+0000 7f98bb163640 1 -- 172.21.15.139:0/1569824362 wait complete. 2024-04-25T09:55:00.079 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-04-25T09:54:58.987994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221976,"kb_used_data":5768,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752744,"statfs":{"total":773094113280,"available":772866809856,"internally_reserved":0,"allocated":5906432,"data_stored":3237996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000580"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.138921+0000","last_change":"2024-04-25T09:54:57.134155+0000","last_active":"2024-04-25T09:54:57.138921+0000","last_peered":"2024-04-25T09:54:57.138921+0000","last_clean":"2024-04-25T09:54:57.138921+0000","last_became_active":"2024-04-25T09:54:57.133892+0000","last_became_peered":"2024-04-25T09:54:57.133892+0000","last_unstale":"2024-04-25T09:54:57.138921+0000","last_undegraded":"2024-04-25T09:54:57.138921+0000","last_fullsized":"2024-04-25T09:54:57.138921+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:28.927694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145633+0000","last_change":"2024-04-25T09:54:57.140060+0000","last_active":"2024-04-25T09:54:57.145633+0000","last_peered":"2024-04-25T09:54:57.145633+0000","last_clean":"2024-04-25T09:54:57.145633+0000","last_became_active":"2024-04-25T09:54:57.139420+0000","last_became_peered":"2024-04-25T09:54:57.139420+0000","last_unstale":"2024-04-25T09:54:57.145633+0000","last_undegraded":"2024-04-25T09:54:57.145633+0000","last_fullsized":"2024-04-25T09:54:57.145633+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:21.118874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133967+0000","last_change":"2024-04-25T09:54:57.130491+0000","last_active":"2024-04-25T09:54:57.133967+0000","last_peered":"2024-04-25T09:54:57.133967+0000","last_clean":"2024-04-25T09:54:57.133967+0000","last_became_active":"2024-04-25T09:54:57.130232+0000","last_became_peered":"2024-04-25T09:54:57.130232+0000","last_unstale":"2024-04-25T09:54:57.133967+0000","last_undegraded":"2024-04-25T09:54:57.133967+0000","last_fullsized":"2024-04-25T09:54:57.133967+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:48.083442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.151445+0000","last_change":"2024-04-25T09:54:57.148574+0000","last_active":"2024-04-25T09:54:57.151445+0000","last_peered":"2024-04-25T09:54:57.151445+0000","last_clean":"2024-04-25T09:54:57.151445+0000","last_became_active":"2024-04-25T09:54:57.148284+0000","last_became_peered":"2024-04-25T09:54:57.148284+0000","last_unstale":"2024-04-25T09:54:57.151445+0000","last_undegraded":"2024-04-25T09:54:57.151445+0000","last_fullsized":"2024-04-25T09:54:57.151445+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:27.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.136327+0000","last_change":"2024-04-25T09:54:57.129129+0000","last_active":"2024-04-25T09:54:57.136327+0000","last_peered":"2024-04-25T09:54:57.136327+0000","last_clean":"2024-04-25T09:54:57.136327+0000","last_became_active":"2024-04-25T09:54:57.128839+0000","last_became_peered":"2024-04-25T09:54:57.128839+0000","last_unstale":"2024-04-25T09:54:57.136327+0000","last_undegraded":"2024-04-25T09:54:57.136327+0000","last_fullsized":"2024-04-25T09:54:57.136327+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:28.927694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.149512+0000","last_change":"2024-04-25T09:54:57.142040+0000","last_active":"2024-04-25T09:54:57.149512+0000","last_peered":"2024-04-25T09:54:57.149512+0000","last_clean":"2024-04-25T09:54:57.149512+0000","last_became_active":"2024-04-25T09:54:57.141781+0000","last_became_peered":"2024-04-25T09:54:57.141781+0000","last_unstale":"2024-04-25T09:54:57.149512+0000","last_undegraded":"2024-04-25T09:54:57.149512+0000","last_fullsized":"2024-04-25T09:54:57.149512+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:40.623235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.146237+0000","last_change":"2024-04-25T09:54:57.142017+0000","last_active":"2024-04-25T09:54:57.146237+0000","last_peered":"2024-04-25T09:54:57.146237+0000","last_clean":"2024-04-25T09:54:57.146237+0000","last_became_active":"2024-04-25T09:54:57.141514+0000","last_became_peered":"2024-04-25T09:54:57.141514+0000","last_unstale":"2024-04-25T09:54:57.146237+0000","last_undegraded":"2024-04-25T09:54:57.146237+0000","last_fullsized":"2024-04-25T09:54:57.146237+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:26.700661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153317+0000","last_change":"2024-04-25T09:54:57.147430+0000","last_active":"2024-04-25T09:54:57.153317+0000","last_peered":"2024-04-25T09:54:57.153317+0000","last_clean":"2024-04-25T09:54:57.153317+0000","last_became_active":"2024-04-25T09:54:57.147193+0000","last_became_peered":"2024-04-25T09:54:57.147193+0000","last_unstale":"2024-04-25T09:54:57.153317+0000","last_undegraded":"2024-04-25T09:54:57.153317+0000","last_fullsized":"2024-04-25T09:54:57.153317+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:41.646452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153272+0000","last_change":"2024-04-25T09:54:57.146684+0000","last_active":"2024-04-25T09:54:57.153272+0000","last_peered":"2024-04-25T09:54:57.153272+0000","last_clean":"2024-04-25T09:54:57.153272+0000","last_became_active":"2024-04-25T09:54:57.146438+0000","last_became_peered":"2024-04-25T09:54:57.146438+0000","last_unstale":"2024-04-25T09:54:57.153272+0000","last_undegraded":"2024-04-25T09:54:57.153272+0000","last_fullsized":"2024-04-25T09:54:57.153272+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:18.716099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.150365+0000","last_change":"2024-04-25T09:54:57.146471+0000","last_active":"2024-04-25T09:54:57.150365+0000","last_peered":"2024-04-25T09:54:57.150365+0000","last_clean":"2024-04-25T09:54:57.150365+0000","last_became_active":"2024-04-25T09:54:57.145678+0000","last_became_peered":"2024-04-25T09:54:57.145678+0000","last_unstale":"2024-04-25T09:54:57.150365+0000","last_undegraded":"2024-04-25T09:54:57.150365+0000","last_fullsized":"2024-04-25T09:54:57.150365+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:16.487981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.143129+0000","last_change":"2024-04-25T09:54:57.137284+0000","last_active":"2024-04-25T09:54:57.143129+0000","last_peered":"2024-04-25T09:54:57.143129+0000","last_clean":"2024-04-25T09:54:57.143129+0000","last_became_active":"2024-04-25T09:54:57.137025+0000","last_became_peered":"2024-04-25T09:54:57.137025+0000","last_unstale":"2024-04-25T09:54:57.143129+0000","last_undegraded":"2024-04-25T09:54:57.143129+0000","last_fullsized":"2024-04-25T09:54:57.143129+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:51.950481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153339+0000","last_change":"2024-04-25T09:54:57.146961+0000","last_active":"2024-04-25T09:54:57.153339+0000","last_peered":"2024-04-25T09:54:57.153339+0000","last_clean":"2024-04-25T09:54:57.153339+0000","last_became_active":"2024-04-25T09:54:57.146588+0000","last_became_peered":"2024-04-25T09:54:57.146588+0000","last_unstale":"2024-04-25T09:54:57.153339+0000","last_undegraded":"2024-04-25T09:54:57.153339+0000","last_fullsized":"2024-04-25T09:54:57.153339+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:20.214884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.149552+0000","last_change":"2024-04-25T09:54:57.140222+0000","last_active":"2024-04-25T09:54:57.149552+0000","last_peered":"2024-04-25T09:54:57.149552+0000","last_clean":"2024-04-25T09:54:57.149552+0000","last_became_active":"2024-04-25T09:54:57.139948+0000","last_became_peered":"2024-04-25T09:54:57.139948+0000","last_unstale":"2024-04-25T09:54:57.149552+0000","last_undegraded":"2024-04-25T09:54:57.149552+0000","last_fullsized":"2024-04-25T09:54:57.149552+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:03.262970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133925+0000","last_change":"2024-04-25T09:54:57.129803+0000","last_active":"2024-04-25T09:54:57.133925+0000","last_peered":"2024-04-25T09:54:57.133925+0000","last_clean":"2024-04-25T09:54:57.133925+0000","last_became_active":"2024-04-25T09:54:57.129053+0000","last_became_peered":"2024-04-25T09:54:57.129053+0000","last_unstale":"2024-04-25T09:54:57.133925+0000","last_undegraded":"2024-04-25T09:54:57.133925+0000","last_fullsized":"2024-04-25T09:54:57.133925+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:10.677088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137346+0000","last_change":"2024-04-25T09:54:57.132487+0000","last_active":"2024-04-25T09:54:57.137346+0000","last_peered":"2024-04-25T09:54:57.137346+0000","last_clean":"2024-04-25T09:54:57.137346+0000","last_became_active":"2024-04-25T09:54:57.132214+0000","last_became_peered":"2024-04-25T09:54:57.132214+0000","last_unstale":"2024-04-25T09:54:57.137346+0000","last_undegraded":"2024-04-25T09:54:57.137346+0000","last_fullsized":"2024-04-25T09:54:57.137346+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:02.465811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.149733+0000","last_change":"2024-04-25T09:54:57.141399+0000","last_active":"2024-04-25T09:54:57.149733+0000","last_peered":"2024-04-25T09:54:57.149733+0000","last_clean":"2024-04-25T09:54:57.149733+0000","last_became_active":"2024-04-25T09:54:57.141133+0000","last_became_peered":"2024-04-25T09:54:57.141133+0000","last_unstale":"2024-04-25T09:54:57.149733+0000","last_undegraded":"2024-04-25T09:54:57.149733+0000","last_fullsized":"2024-04-25T09:54:57.149733+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:13.099355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.141529+0000","last_change":"2024-04-25T09:54:57.135389+0000","last_active":"2024-04-25T09:54:57.141529+0000","last_peered":"2024-04-25T09:54:57.141529+0000","last_clean":"2024-04-25T09:54:57.141529+0000","last_became_active":"2024-04-25T09:54:57.135132+0000","last_became_peered":"2024-04-25T09:54:57.135132+0000","last_unstale":"2024-04-25T09:54:57.141529+0000","last_undegraded":"2024-04-25T09:54:57.141529+0000","last_fullsized":"2024-04-25T09:54:57.141529+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:40.882549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145654+0000","last_change":"2024-04-25T09:54:57.140271+0000","last_active":"2024-04-25T09:54:57.145654+0000","last_peered":"2024-04-25T09:54:57.145654+0000","last_clean":"2024-04-25T09:54:57.145654+0000","last_became_active":"2024-04-25T09:54:57.139742+0000","last_became_peered":"2024-04-25T09:54:57.139742+0000","last_unstale":"2024-04-25T09:54:57.145654+0000","last_undegraded":"2024-04-25T09:54:57.145654+0000","last_fullsized":"2024-04-25T09:54:57.145654+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:03.700244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133946+0000","last_change":"2024-04-25T09:54:57.128688+0000","last_active":"2024-04-25T09:54:57.133946+0000","last_peered":"2024-04-25T09:54:57.133946+0000","last_clean":"2024-04-25T09:54:57.133946+0000","last_became_active":"2024-04-25T09:54:57.127892+0000","last_became_peered":"2024-04-25T09:54:57.127892+0000","last_unstale":"2024-04-25T09:54:57.133946+0000","last_undegraded":"2024-04-25T09:54:57.133946+0000","last_fullsized":"2024-04-25T09:54:57.133946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:26.467246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.140345+0000","last_change":"2024-04-25T09:54:57.134475+0000","last_active":"2024-04-25T09:54:57.140345+0000","last_peered":"2024-04-25T09:54:57.140345+0000","last_clean":"2024-04-25T09:54:57.140345+0000","last_became_active":"2024-04-25T09:54:57.134200+0000","last_became_peered":"2024-04-25T09:54:57.134200+0000","last_unstale":"2024-04-25T09:54:57.140345+0000","last_undegraded":"2024-04-25T09:54:57.140345+0000","last_fullsized":"2024-04-25T09:54:57.140345+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:18.173854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.143262+0000","last_change":"2024-04-25T09:54:57.139077+0000","last_active":"2024-04-25T09:54:57.143262+0000","last_peered":"2024-04-25T09:54:57.143262+0000","last_clean":"2024-04-25T09:54:57.143262+0000","last_became_active":"2024-04-25T09:54:57.138699+0000","last_became_peered":"2024-04-25T09:54:57.138699+0000","last_unstale":"2024-04-25T09:54:57.143262+0000","last_undegraded":"2024-04-25T09:54:57.143262+0000","last_fullsized":"2024-04-25T09:54:57.143262+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:13.099355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153315+0000","last_change":"2024-04-25T09:54:57.147320+0000","last_active":"2024-04-25T09:54:57.153315+0000","last_peered":"2024-04-25T09:54:57.153315+0000","last_clean":"2024-04-25T09:54:57.153315+0000","last_became_active":"2024-04-25T09:54:57.147083+0000","last_became_peered":"2024-04-25T09:54:57.147083+0000","last_unstale":"2024-04-25T09:54:57.153315+0000","last_undegraded":"2024-04-25T09:54:57.153315+0000","last_fullsized":"2024-04-25T09:54:57.153315+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:03.670032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.143022+0000","last_change":"2024-04-25T09:54:57.136509+0000","last_active":"2024-04-25T09:54:57.143022+0000","last_peered":"2024-04-25T09:54:57.143022+0000","last_clean":"2024-04-25T09:54:57.143022+0000","last_became_active":"2024-04-25T09:54:57.136259+0000","last_became_peered":"2024-04-25T09:54:57.136259+0000","last_unstale":"2024-04-25T09:54:57.143022+0000","last_undegraded":"2024-04-25T09:54:57.143022+0000","last_fullsized":"2024-04-25T09:54:57.143022+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:34.781194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.134011+0000","last_change":"2024-04-25T09:54:57.130661+0000","last_active":"2024-04-25T09:54:57.134011+0000","last_peered":"2024-04-25T09:54:57.134011+0000","last_clean":"2024-04-25T09:54:57.134011+0000","last_became_active":"2024-04-25T09:54:57.129190+0000","last_became_peered":"2024-04-25T09:54:57.129190+0000","last_unstale":"2024-04-25T09:54:57.134011+0000","last_undegraded":"2024-04-25T09:54:57.134011+0000","last_fullsized":"2024-04-25T09:54:57.134011+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:29.718809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.143897+0000","last_change":"2024-04-25T09:54:57.140635+0000","last_active":"2024-04-25T09:54:57.143897+0000","last_peered":"2024-04-25T09:54:57.143897+0000","last_clean":"2024-04-25T09:54:57.143897+0000","last_became_active":"2024-04-25T09:54:57.140340+0000","last_became_peered":"2024-04-25T09:54:57.140340+0000","last_unstale":"2024-04-25T09:54:57.143897+0000","last_undegraded":"2024-04-25T09:54:57.143897+0000","last_fullsized":"2024-04-25T09:54:57.143897+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:00.061134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145999+0000","last_change":"2024-04-25T09:54:57.145356+0000","last_active":"2024-04-25T09:54:57.145999+0000","last_peered":"2024-04-25T09:54:57.145999+0000","last_clean":"2024-04-25T09:54:57.145999+0000","last_became_active":"2024-04-25T09:54:57.145032+0000","last_became_peered":"2024-04-25T09:54:57.145032+0000","last_unstale":"2024-04-25T09:54:57.145999+0000","last_undegraded":"2024-04-25T09:54:57.145999+0000","last_fullsized":"2024-04-25T09:54:57.145999+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:16.656767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.139193+0000","last_change":"2024-04-25T09:54:57.136196+0000","last_active":"2024-04-25T09:54:57.139193+0000","last_peered":"2024-04-25T09:54:57.139193+0000","last_clean":"2024-04-25T09:54:57.139193+0000","last_became_active":"2024-04-25T09:54:57.135927+0000","last_became_peered":"2024-04-25T09:54:57.135927+0000","last_unstale":"2024-04-25T09:54:57.139193+0000","last_undegraded":"2024-04-25T09:54:57.139193+0000","last_fullsized":"2024-04-25T09:54:57.139193+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:18.173854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145985+0000","last_change":"2024-04-25T09:54:57.141050+0000","last_active":"2024-04-25T09:54:57.145985+0000","last_peered":"2024-04-25T09:54:57.145985+0000","last_clean":"2024-04-25T09:54:57.145985+0000","last_became_active":"2024-04-25T09:54:57.140765+0000","last_became_peered":"2024-04-25T09:54:57.140765+0000","last_unstale":"2024-04-25T09:54:57.145985+0000","last_undegraded":"2024-04-25T09:54:57.145985+0000","last_fullsized":"2024-04-25T09:54:57.145985+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:25.632038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133857+0000","last_change":"2024-04-25T09:54:57.128890+0000","last_active":"2024-04-25T09:54:57.133857+0000","last_peered":"2024-04-25T09:54:57.133857+0000","last_clean":"2024-04-25T09:54:57.133857+0000","last_became_active":"2024-04-25T09:54:57.128600+0000","last_became_peered":"2024-04-25T09:54:57.128600+0000","last_unstale":"2024-04-25T09:54:57.133857+0000","last_undegraded":"2024-04-25T09:54:57.133857+0000","last_fullsized":"2024-04-25T09:54:57.133857+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:33.907952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137399+0000","last_change":"2024-04-25T09:54:57.132629+0000","last_active":"2024-04-25T09:54:57.137399+0000","last_peered":"2024-04-25T09:54:57.137399+0000","last_clean":"2024-04-25T09:54:57.137399+0000","last_became_active":"2024-04-25T09:54:57.132356+0000","last_became_peered":"2024-04-25T09:54:57.132356+0000","last_unstale":"2024-04-25T09:54:57.137399+0000","last_undegraded":"2024-04-25T09:54:57.137399+0000","last_fullsized":"2024-04-25T09:54:57.137399+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:10.677088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.146218+0000","last_change":"2024-04-25T09:54:57.141802+0000","last_active":"2024-04-25T09:54:57.146218+0000","last_peered":"2024-04-25T09:54:57.146218+0000","last_clean":"2024-04-25T09:54:57.146218+0000","last_became_active":"2024-04-25T09:54:57.141156+0000","last_became_peered":"2024-04-25T09:54:57.141156+0000","last_unstale":"2024-04-25T09:54:57.146218+0000","last_undegraded":"2024-04-25T09:54:57.146218+0000","last_fullsized":"2024-04-25T09:54:57.146218+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.248939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.140273+0000","last_change":"2024-04-25T09:54:57.137957+0000","last_active":"2024-04-25T09:54:57.140273+0000","last_peered":"2024-04-25T09:54:57.140273+0000","last_clean":"2024-04-25T09:54:57.140273+0000","last_became_active":"2024-04-25T09:54:57.136307+0000","last_became_peered":"2024-04-25T09:54:57.136307+0000","last_unstale":"2024-04-25T09:54:57.140273+0000","last_undegraded":"2024-04-25T09:54:57.140273+0000","last_fullsized":"2024-04-25T09:54:57.140273+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:00.061134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137467+0000","last_change":"2024-04-25T09:54:57.134362+0000","last_active":"2024-04-25T09:54:57.137467+0000","last_peered":"2024-04-25T09:54:57.137467+0000","last_clean":"2024-04-25T09:54:57.137467+0000","last_became_active":"2024-04-25T09:54:57.133972+0000","last_became_peered":"2024-04-25T09:54:57.133972+0000","last_unstale":"2024-04-25T09:54:57.137467+0000","last_undegraded":"2024-04-25T09:54:57.137467+0000","last_fullsized":"2024-04-25T09:54:57.137467+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:29.718809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.140868+0000","last_change":"2024-04-25T09:54:56.131484+0000","last_active":"2024-04-25T09:54:57.140868+0000","last_peered":"2024-04-25T09:54:57.140868+0000","last_clean":"2024-04-25T09:54:57.140868+0000","last_became_active":"2024-04-25T09:54:56.131213+0000","last_became_peered":"2024-04-25T09:54:56.131213+0000","last_unstale":"2024-04-25T09:54:57.140868+0000","last_undegraded":"2024-04-25T09:54:57.140868+0000","last_fullsized":"2024-04-25T09:54:57.140868+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:06.565946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137994+0000","last_change":"2024-04-25T09:54:57.135457+0000","last_active":"2024-04-25T09:54:57.137994+0000","last_peered":"2024-04-25T09:54:57.137994+0000","last_clean":"2024-04-25T09:54:57.137994+0000","last_became_active":"2024-04-25T09:54:57.135175+0000","last_became_peered":"2024-04-25T09:54:57.135175+0000","last_unstale":"2024-04-25T09:54:57.137994+0000","last_undegraded":"2024-04-25T09:54:57.137994+0000","last_fullsized":"2024-04-25T09:54:57.137994+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:03.700244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.149172+0000","last_change":"2024-04-25T09:54:56.125575+0000","last_active":"2024-04-25T09:54:57.149172+0000","last_peered":"2024-04-25T09:54:57.149172+0000","last_clean":"2024-04-25T09:54:57.149172+0000","last_became_active":"2024-04-25T09:54:56.125286+0000","last_became_peered":"2024-04-25T09:54:56.125286+0000","last_unstale":"2024-04-25T09:54:57.149172+0000","last_undegraded":"2024-04-25T09:54:57.149172+0000","last_fullsized":"2024-04-25T09:54:57.149172+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:32.072748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.139854+0000","last_change":"2024-04-25T09:54:57.134629+0000","last_active":"2024-04-25T09:54:57.139854+0000","last_peered":"2024-04-25T09:54:57.139854+0000","last_clean":"2024-04-25T09:54:57.139854+0000","last_became_active":"2024-04-25T09:54:57.134372+0000","last_became_peered":"2024-04-25T09:54:57.134372+0000","last_unstale":"2024-04-25T09:54:57.139854+0000","last_undegraded":"2024-04-25T09:54:57.139854+0000","last_fullsized":"2024-04-25T09:54:57.139854+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:03.262970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.148076+0000","last_change":"2024-04-25T09:54:56.131847+0000","last_active":"2024-04-25T09:54:57.148076+0000","last_peered":"2024-04-25T09:54:57.148076+0000","last_clean":"2024-04-25T09:54:57.148076+0000","last_became_active":"2024-04-25T09:54:56.131586+0000","last_became_peered":"2024-04-25T09:54:56.131586+0000","last_unstale":"2024-04-25T09:54:57.148076+0000","last_undegraded":"2024-04-25T09:54:57.148076+0000","last_fullsized":"2024-04-25T09:54:57.148076+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.602141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153255+0000","last_change":"2024-04-25T09:54:57.146456+0000","last_active":"2024-04-25T09:54:57.153255+0000","last_peered":"2024-04-25T09:54:57.153255+0000","last_clean":"2024-04-25T09:54:57.153255+0000","last_became_active":"2024-04-25T09:54:57.146210+0000","last_became_peered":"2024-04-25T09:54:57.146210+0000","last_unstale":"2024-04-25T09:54:57.153255+0000","last_undegraded":"2024-04-25T09:54:57.153255+0000","last_fullsized":"2024-04-25T09:54:57.153255+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:28.119357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.148108+0000","last_change":"2024-04-25T09:54:56.127140+0000","last_active":"2024-04-25T09:54:57.148108+0000","last_peered":"2024-04-25T09:54:57.148108+0000","last_clean":"2024-04-25T09:54:57.148108+0000","last_became_active":"2024-04-25T09:54:56.126860+0000","last_became_peered":"2024-04-25T09:54:56.126860+0000","last_unstale":"2024-04-25T09:54:57.148108+0000","last_undegraded":"2024-04-25T09:54:57.148108+0000","last_fullsized":"2024-04-25T09:54:57.148108+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:09.672795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.136908+0000","last_change":"2024-04-25T09:54:57.131179+0000","last_active":"2024-04-25T09:54:57.136908+0000","last_peered":"2024-04-25T09:54:57.136908+0000","last_clean":"2024-04-25T09:54:57.136908+0000","last_became_active":"2024-04-25T09:54:57.130888+0000","last_became_peered":"2024-04-25T09:54:57.130888+0000","last_unstale":"2024-04-25T09:54:57.136908+0000","last_undegraded":"2024-04-25T09:54:57.136908+0000","last_fullsized":"2024-04-25T09:54:57.136908+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:50.052465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.144526+0000","last_change":"2024-04-25T09:54:56.132128+0000","last_active":"2024-04-25T09:54:57.144526+0000","last_peered":"2024-04-25T09:54:57.144526+0000","last_clean":"2024-04-25T09:54:57.144526+0000","last_became_active":"2024-04-25T09:54:56.131030+0000","last_became_peered":"2024-04-25T09:54:56.131030+0000","last_unstale":"2024-04-25T09:54:57.144526+0000","last_undegraded":"2024-04-25T09:54:57.144526+0000","last_fullsized":"2024-04-25T09:54:57.144526+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:15.483688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137463+0000","last_change":"2024-04-25T09:54:57.134231+0000","last_active":"2024-04-25T09:54:57.137463+0000","last_peered":"2024-04-25T09:54:57.137463+0000","last_clean":"2024-04-25T09:54:57.137463+0000","last_became_active":"2024-04-25T09:54:57.133947+0000","last_became_peered":"2024-04-25T09:54:57.133947+0000","last_unstale":"2024-04-25T09:54:57.137463+0000","last_undegraded":"2024-04-25T09:54:57.137463+0000","last_fullsized":"2024-04-25T09:54:57.137463+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:48.083442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.148703+0000","last_change":"2024-04-25T09:54:56.132053+0000","last_active":"2024-04-25T09:54:57.148703+0000","last_peered":"2024-04-25T09:54:57.148703+0000","last_clean":"2024-04-25T09:54:57.148703+0000","last_became_active":"2024-04-25T09:54:56.131759+0000","last_became_peered":"2024-04-25T09:54:56.131759+0000","last_unstale":"2024-04-25T09:54:57.148703+0000","last_undegraded":"2024-04-25T09:54:57.148703+0000","last_fullsized":"2024-04-25T09:54:57.148703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.602141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153214+0000","last_change":"2024-04-25T09:54:57.145880+0000","last_active":"2024-04-25T09:54:57.153214+0000","last_peered":"2024-04-25T09:54:57.153214+0000","last_clean":"2024-04-25T09:54:57.153214+0000","last_became_active":"2024-04-25T09:54:57.145620+0000","last_became_peered":"2024-04-25T09:54:57.145620+0000","last_unstale":"2024-04-25T09:54:57.153214+0000","last_undegraded":"2024-04-25T09:54:57.153214+0000","last_fullsized":"2024-04-25T09:54:57.153214+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:26.700661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.122621+0000","last_change":"2024-04-25T09:54:56.134125+0000","last_active":"2024-04-25T09:54:57.122621+0000","last_peered":"2024-04-25T09:54:57.122621+0000","last_clean":"2024-04-25T09:54:57.122621+0000","last_became_active":"2024-04-25T09:54:56.133818+0000","last_became_peered":"2024-04-25T09:54:56.133818+0000","last_unstale":"2024-04-25T09:54:57.122621+0000","last_undegraded":"2024-04-25T09:54:57.122621+0000","last_fullsized":"2024-04-25T09:54:57.122621+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:32.072748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145740+0000","last_change":"2024-04-25T09:54:57.142226+0000","last_active":"2024-04-25T09:54:57.145740+0000","last_peered":"2024-04-25T09:54:57.145740+0000","last_clean":"2024-04-25T09:54:57.145740+0000","last_became_active":"2024-04-25T09:54:57.141978+0000","last_became_peered":"2024-04-25T09:54:57.141978+0000","last_unstale":"2024-04-25T09:54:57.145740+0000","last_undegraded":"2024-04-25T09:54:57.145740+0000","last_fullsized":"2024-04-25T09:54:57.145740+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:40.882549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.119069+0000","last_change":"2024-04-25T09:54:56.130721+0000","last_active":"2024-04-25T09:54:57.119069+0000","last_peered":"2024-04-25T09:54:57.119069+0000","last_clean":"2024-04-25T09:54:57.119069+0000","last_became_active":"2024-04-25T09:54:56.130461+0000","last_became_peered":"2024-04-25T09:54:56.130461+0000","last_unstale":"2024-04-25T09:54:57.119069+0000","last_undegraded":"2024-04-25T09:54:57.119069+0000","last_fullsized":"2024-04-25T09:54:57.119069+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:39.618943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137323+0000","last_change":"2024-04-25T09:54:57.135635+0000","last_active":"2024-04-25T09:54:57.137323+0000","last_peered":"2024-04-25T09:54:57.137323+0000","last_clean":"2024-04-25T09:54:57.137323+0000","last_became_active":"2024-04-25T09:54:57.135342+0000","last_became_peered":"2024-04-25T09:54:57.135342+0000","last_unstale":"2024-04-25T09:54:57.137323+0000","last_undegraded":"2024-04-25T09:54:57.137323+0000","last_fullsized":"2024-04-25T09:54:57.137323+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:03.262970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.125976+0000","last_change":"2024-04-25T09:54:56.132520+0000","last_active":"2024-04-25T09:54:57.125976+0000","last_peered":"2024-04-25T09:54:57.125976+0000","last_clean":"2024-04-25T09:54:57.125976+0000","last_became_active":"2024-04-25T09:54:56.132178+0000","last_became_peered":"2024-04-25T09:54:56.132178+0000","last_unstale":"2024-04-25T09:54:57.125976+0000","last_undegraded":"2024-04-25T09:54:57.125976+0000","last_fullsized":"2024-04-25T09:54:57.125976+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:25.462953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133170+0000","last_change":"2024-04-25T09:54:57.128730+0000","last_active":"2024-04-25T09:54:57.133170+0000","last_peered":"2024-04-25T09:54:57.133170+0000","last_clean":"2024-04-25T09:54:57.133170+0000","last_became_active":"2024-04-25T09:54:57.128446+0000","last_became_peered":"2024-04-25T09:54:57.128446+0000","last_unstale":"2024-04-25T09:54:57.133170+0000","last_undegraded":"2024-04-25T09:54:57.133170+0000","last_fullsized":"2024-04-25T09:54:57.133170+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:48.083442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.118709+0000","last_change":"2024-04-25T09:54:56.135178+0000","last_active":"2024-04-25T09:54:57.118709+0000","last_peered":"2024-04-25T09:54:57.118709+0000","last_clean":"2024-04-25T09:54:57.118709+0000","last_became_active":"2024-04-25T09:54:56.134909+0000","last_became_peered":"2024-04-25T09:54:56.134909+0000","last_unstale":"2024-04-25T09:54:57.118709+0000","last_undegraded":"2024-04-25T09:54:57.118709+0000","last_fullsized":"2024-04-25T09:54:57.118709+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:32.072748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133944+0000","last_change":"2024-04-25T09:54:57.130040+0000","last_active":"2024-04-25T09:54:57.133944+0000","last_peered":"2024-04-25T09:54:57.133944+0000","last_clean":"2024-04-25T09:54:57.133944+0000","last_became_active":"2024-04-25T09:54:57.129264+0000","last_became_peered":"2024-04-25T09:54:57.129264+0000","last_unstale":"2024-04-25T09:54:57.133944+0000","last_undegraded":"2024-04-25T09:54:57.133944+0000","last_fullsized":"2024-04-25T09:54:57.133944+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:21.118874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.150977+0000","last_change":"2024-04-25T09:54:56.135230+0000","last_active":"2024-04-25T09:54:57.150977+0000","last_peered":"2024-04-25T09:54:57.150977+0000","last_clean":"2024-04-25T09:54:57.150977+0000","last_became_active":"2024-04-25T09:54:56.134942+0000","last_became_peered":"2024-04-25T09:54:56.134942+0000","last_unstale":"2024-04-25T09:54:57.150977+0000","last_undegraded":"2024-04-25T09:54:57.150977+0000","last_fullsized":"2024-04-25T09:54:57.150977+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:05.618687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.140739+0000","last_change":"2024-04-25T09:54:57.138897+0000","last_active":"2024-04-25T09:54:57.140739+0000","last_peered":"2024-04-25T09:54:57.140739+0000","last_clean":"2024-04-25T09:54:57.140739+0000","last_became_active":"2024-04-25T09:54:57.138622+0000","last_became_peered":"2024-04-25T09:54:57.138622+0000","last_unstale":"2024-04-25T09:54:57.140739+0000","last_undegraded":"2024-04-25T09:54:57.140739+0000","last_fullsized":"2024-04-25T09:54:57.140739+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:13.099355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153423+0000","last_change":"2024-04-25T09:54:56.129682+0000","last_active":"2024-04-25T09:54:57.153423+0000","last_peered":"2024-04-25T09:54:57.153423+0000","last_clean":"2024-04-25T09:54:57.153423+0000","last_became_active":"2024-04-25T09:54:56.129372+0000","last_became_peered":"2024-04-25T09:54:56.129372+0000","last_unstale":"2024-04-25T09:54:57.153423+0000","last_undegraded":"2024-04-25T09:54:57.153423+0000","last_fullsized":"2024-04-25T09:54:57.153423+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:39.878256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.138035+0000","last_change":"2024-04-25T09:54:57.135635+0000","last_active":"2024-04-25T09:54:57.138035+0000","last_peered":"2024-04-25T09:54:57.138035+0000","last_clean":"2024-04-25T09:54:57.138035+0000","last_became_active":"2024-04-25T09:54:57.134845+0000","last_became_peered":"2024-04-25T09:54:57.134845+0000","last_unstale":"2024-04-25T09:54:57.138035+0000","last_undegraded":"2024-04-25T09:54:57.138035+0000","last_fullsized":"2024-04-25T09:54:57.138035+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:40.882549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.146096+0000","last_change":"2024-04-25T09:54:56.133637+0000","last_active":"2024-04-25T09:54:57.146096+0000","last_peered":"2024-04-25T09:54:57.146096+0000","last_clean":"2024-04-25T09:54:57.146096+0000","last_became_active":"2024-04-25T09:54:56.133317+0000","last_became_peered":"2024-04-25T09:54:56.133317+0000","last_unstale":"2024-04-25T09:54:57.146096+0000","last_undegraded":"2024-04-25T09:54:57.146096+0000","last_fullsized":"2024-04-25T09:54:57.146096+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:56.136101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.138400+0000","last_change":"2024-04-25T09:54:57.133529+0000","last_active":"2024-04-25T09:54:57.138400+0000","last_peered":"2024-04-25T09:54:57.138400+0000","last_clean":"2024-04-25T09:54:57.138400+0000","last_became_active":"2024-04-25T09:54:57.133244+0000","last_became_peered":"2024-04-25T09:54:57.133244+0000","last_unstale":"2024-04-25T09:54:57.138400+0000","last_undegraded":"2024-04-25T09:54:57.138400+0000","last_fullsized":"2024-04-25T09:54:57.138400+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:03.700244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.124530+0000","last_change":"2024-04-25T09:54:56.135711+0000","last_active":"2024-04-25T09:54:57.124530+0000","last_peered":"2024-04-25T09:54:57.124530+0000","last_clean":"2024-04-25T09:54:57.124530+0000","last_became_active":"2024-04-25T09:54:56.135426+0000","last_became_peered":"2024-04-25T09:54:56.135426+0000","last_unstale":"2024-04-25T09:54:57.124530+0000","last_undegraded":"2024-04-25T09:54:57.124530+0000","last_fullsized":"2024-04-25T09:54:57.124530+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.602141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153408+0000","last_change":"2024-04-25T09:54:57.150520+0000","last_active":"2024-04-25T09:54:57.153408+0000","last_peered":"2024-04-25T09:54:57.153408+0000","last_clean":"2024-04-25T09:54:57.153408+0000","last_became_active":"2024-04-25T09:54:57.150259+0000","last_became_peered":"2024-04-25T09:54:57.150259+0000","last_unstale":"2024-04-25T09:54:57.153408+0000","last_undegraded":"2024-04-25T09:54:57.153408+0000","last_fullsized":"2024-04-25T09:54:57.153408+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:36.490077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.125114+0000","last_change":"2024-04-25T09:54:56.132949+0000","last_active":"2024-04-25T09:54:57.125114+0000","last_peered":"2024-04-25T09:54:57.125114+0000","last_clean":"2024-04-25T09:54:57.125114+0000","last_became_active":"2024-04-25T09:54:56.132215+0000","last_became_peered":"2024-04-25T09:54:56.132215+0000","last_unstale":"2024-04-25T09:54:57.125114+0000","last_undegraded":"2024-04-25T09:54:57.125114+0000","last_fullsized":"2024-04-25T09:54:57.125114+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:54.230115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.141550+0000","last_change":"2024-04-25T09:54:57.136471+0000","last_active":"2024-04-25T09:54:57.141550+0000","last_peered":"2024-04-25T09:54:57.141550+0000","last_clean":"2024-04-25T09:54:57.141550+0000","last_became_active":"2024-04-25T09:54:57.136223+0000","last_became_peered":"2024-04-25T09:54:57.136223+0000","last_unstale":"2024-04-25T09:54:57.141550+0000","last_undegraded":"2024-04-25T09:54:57.141550+0000","last_fullsized":"2024-04-25T09:54:57.141550+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:27.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145641+0000","last_change":"2024-04-25T09:54:56.127601+0000","last_active":"2024-04-25T09:54:57.145641+0000","last_peered":"2024-04-25T09:54:57.145641+0000","last_clean":"2024-04-25T09:54:57.145641+0000","last_became_active":"2024-04-25T09:54:56.127337+0000","last_became_peered":"2024-04-25T09:54:56.127337+0000","last_unstale":"2024-04-25T09:54:57.145641+0000","last_undegraded":"2024-04-25T09:54:57.145641+0000","last_fullsized":"2024-04-25T09:54:57.145641+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:47.079149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145697+0000","last_change":"2024-04-25T09:54:57.139553+0000","last_active":"2024-04-25T09:54:57.145697+0000","last_peered":"2024-04-25T09:54:57.145697+0000","last_clean":"2024-04-25T09:54:57.145697+0000","last_became_active":"2024-04-25T09:54:57.139302+0000","last_became_peered":"2024-04-25T09:54:57.139302+0000","last_unstale":"2024-04-25T09:54:57.145697+0000","last_undegraded":"2024-04-25T09:54:57.145697+0000","last_fullsized":"2024-04-25T09:54:57.145697+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:00.061134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.148146+0000","last_change":"2024-04-25T09:54:56.131623+0000","last_active":"2024-04-25T09:54:57.148146+0000","last_peered":"2024-04-25T09:54:57.148146+0000","last_clean":"2024-04-25T09:54:57.148146+0000","last_became_active":"2024-04-25T09:54:56.131360+0000","last_became_peered":"2024-04-25T09:54:56.131360+0000","last_unstale":"2024-04-25T09:54:57.148146+0000","last_undegraded":"2024-04-25T09:54:57.148146+0000","last_fullsized":"2024-04-25T09:54:57.148146+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:54.230115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.138885+0000","last_change":"2024-04-25T09:54:57.134955+0000","last_active":"2024-04-25T09:54:57.138885+0000","last_peered":"2024-04-25T09:54:57.138885+0000","last_clean":"2024-04-25T09:54:57.138885+0000","last_became_active":"2024-04-25T09:54:57.134700+0000","last_became_peered":"2024-04-25T09:54:57.134700+0000","last_unstale":"2024-04-25T09:54:57.138885+0000","last_undegraded":"2024-04-25T09:54:57.138885+0000","last_fullsized":"2024-04-25T09:54:57.138885+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:50.052465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.143265+0000","last_change":"2024-04-25T09:54:56.129987+0000","last_active":"2024-04-25T09:54:57.143265+0000","last_peered":"2024-04-25T09:54:57.143265+0000","last_clean":"2024-04-25T09:54:57.143265+0000","last_became_active":"2024-04-25T09:54:56.129678+0000","last_became_peered":"2024-04-25T09:54:56.129678+0000","last_unstale":"2024-04-25T09:54:57.143265+0000","last_undegraded":"2024-04-25T09:54:57.143265+0000","last_fullsized":"2024-04-25T09:54:57.143265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:04.585943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137211+0000","last_change":"2024-04-25T09:54:57.131707+0000","last_active":"2024-04-25T09:54:57.137211+0000","last_peered":"2024-04-25T09:54:57.137211+0000","last_clean":"2024-04-25T09:54:57.137211+0000","last_became_active":"2024-04-25T09:54:57.131432+0000","last_became_peered":"2024-04-25T09:54:57.131432+0000","last_unstale":"2024-04-25T09:54:57.137211+0000","last_undegraded":"2024-04-25T09:54:57.137211+0000","last_fullsized":"2024-04-25T09:54:57.137211+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:18.173854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137576+0000","last_change":"2024-04-25T09:54:56.135033+0000","last_active":"2024-04-25T09:54:57.137576+0000","last_peered":"2024-04-25T09:54:57.137576+0000","last_clean":"2024-04-25T09:54:57.137576+0000","last_became_active":"2024-04-25T09:54:56.134722+0000","last_became_peered":"2024-04-25T09:54:56.134722+0000","last_unstale":"2024-04-25T09:54:57.137576+0000","last_undegraded":"2024-04-25T09:54:57.137576+0000","last_fullsized":"2024-04-25T09:54:57.137576+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:27.923401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.150367+0000","last_change":"2024-04-25T09:54:57.147320+0000","last_active":"2024-04-25T09:54:57.150367+0000","last_peered":"2024-04-25T09:54:57.150367+0000","last_clean":"2024-04-25T09:54:57.150367+0000","last_became_active":"2024-04-25T09:54:57.147078+0000","last_became_peered":"2024-04-25T09:54:57.147078+0000","last_unstale":"2024-04-25T09:54:57.150367+0000","last_undegraded":"2024-04-25T09:54:57.150367+0000","last_fullsized":"2024-04-25T09:54:57.150367+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:57.140394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.123784+0000","last_change":"2024-04-25T09:54:56.133612+0000","last_active":"2024-04-25T09:54:57.123784+0000","last_peered":"2024-04-25T09:54:57.123784+0000","last_clean":"2024-04-25T09:54:57.123784+0000","last_became_active":"2024-04-25T09:54:56.133198+0000","last_became_peered":"2024-04-25T09:54:56.133198+0000","last_unstale":"2024-04-25T09:54:57.123784+0000","last_undegraded":"2024-04-25T09:54:57.123784+0000","last_fullsized":"2024-04-25T09:54:57.123784+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:08.630451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137847+0000","last_change":"2024-04-25T09:54:57.133165+0000","last_active":"2024-04-25T09:54:57.137847+0000","last_peered":"2024-04-25T09:54:57.137847+0000","last_clean":"2024-04-25T09:54:57.137847+0000","last_became_active":"2024-04-25T09:54:57.132827+0000","last_became_peered":"2024-04-25T09:54:57.132827+0000","last_unstale":"2024-04-25T09:54:57.137847+0000","last_undegraded":"2024-04-25T09:54:57.137847+0000","last_fullsized":"2024-04-25T09:54:57.137847+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:29.718809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.151298+0000","last_change":"2024-04-25T09:54:56.129114+0000","last_active":"2024-04-25T09:54:57.151298+0000","last_peered":"2024-04-25T09:54:57.151298+0000","last_clean":"2024-04-25T09:54:57.151298+0000","last_became_active":"2024-04-25T09:54:56.128803+0000","last_became_peered":"2024-04-25T09:54:56.128803+0000","last_unstale":"2024-04-25T09:54:57.151298+0000","last_undegraded":"2024-04-25T09:54:57.151298+0000","last_fullsized":"2024-04-25T09:54:57.151298+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:08.630451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.140431+0000","last_change":"2024-04-25T09:54:57.133448+0000","last_active":"2024-04-25T09:54:57.140431+0000","last_peered":"2024-04-25T09:54:57.140431+0000","last_clean":"2024-04-25T09:54:57.140431+0000","last_became_active":"2024-04-25T09:54:57.133076+0000","last_became_peered":"2024-04-25T09:54:57.133076+0000","last_unstale":"2024-04-25T09:54:57.140431+0000","last_undegraded":"2024-04-25T09:54:57.140431+0000","last_fullsized":"2024-04-25T09:54:57.140431+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:50.052465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.132810+0000","last_change":"2024-04-25T09:54:56.127989+0000","last_active":"2024-04-25T09:54:57.132810+0000","last_peered":"2024-04-25T09:54:57.132810+0000","last_clean":"2024-04-25T09:54:57.132810+0000","last_became_active":"2024-04-25T09:54:56.126968+0000","last_became_peered":"2024-04-25T09:54:56.126968+0000","last_unstale":"2024-04-25T09:54:57.132810+0000","last_undegraded":"2024-04-25T09:54:57.132810+0000","last_fullsized":"2024-04-25T09:54:57.132810+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:28.714516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133892+0000","last_change":"2024-04-25T09:54:57.129752+0000","last_active":"2024-04-25T09:54:57.133892+0000","last_peered":"2024-04-25T09:54:57.133892+0000","last_clean":"2024-04-25T09:54:57.133892+0000","last_became_active":"2024-04-25T09:54:57.129349+0000","last_became_peered":"2024-04-25T09:54:57.129349+0000","last_unstale":"2024-04-25T09:54:57.133892+0000","last_undegraded":"2024-04-25T09:54:57.133892+0000","last_fullsized":"2024-04-25T09:54:57.133892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:02.465811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.119361+0000","last_change":"2024-04-25T09:54:56.129836+0000","last_active":"2024-04-25T09:54:57.119361+0000","last_peered":"2024-04-25T09:54:57.119361+0000","last_clean":"2024-04-25T09:54:57.119361+0000","last_became_active":"2024-04-25T09:54:56.129540+0000","last_became_peered":"2024-04-25T09:54:56.129540+0000","last_unstale":"2024-04-25T09:54:57.119361+0000","last_undegraded":"2024-04-25T09:54:57.119361+0000","last_fullsized":"2024-04-25T09:54:57.119361+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:24.627746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.150356+0000","last_change":"2024-04-25T09:54:57.148723+0000","last_active":"2024-04-25T09:54:57.150356+0000","last_peered":"2024-04-25T09:54:57.150356+0000","last_clean":"2024-04-25T09:54:57.150356+0000","last_became_active":"2024-04-25T09:54:57.148474+0000","last_became_peered":"2024-04-25T09:54:57.148474+0000","last_unstale":"2024-04-25T09:54:57.150356+0000","last_undegraded":"2024-04-25T09:54:57.150356+0000","last_fullsized":"2024-04-25T09:54:57.150356+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:19.646682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.123756+0000","last_change":"2024-04-25T09:54:56.132506+0000","last_active":"2024-04-25T09:54:57.123756+0000","last_peered":"2024-04-25T09:54:57.123756+0000","last_clean":"2024-04-25T09:54:57.123756+0000","last_became_active":"2024-04-25T09:54:56.132219+0000","last_became_peered":"2024-04-25T09:54:56.132219+0000","last_unstale":"2024-04-25T09:54:57.123756+0000","last_undegraded":"2024-04-25T09:54:57.123756+0000","last_fullsized":"2024-04-25T09:54:57.123756+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:01.923921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.139729+0000","last_change":"2024-04-25T09:54:57.137712+0000","last_active":"2024-04-25T09:54:57.139729+0000","last_peered":"2024-04-25T09:54:57.139729+0000","last_clean":"2024-04-25T09:54:57.139729+0000","last_became_active":"2024-04-25T09:54:57.135931+0000","last_became_peered":"2024-04-25T09:54:57.135931+0000","last_unstale":"2024-04-25T09:54:57.139729+0000","last_undegraded":"2024-04-25T09:54:57.139729+0000","last_fullsized":"2024-04-25T09:54:57.139729+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:27.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.140441+0000","last_change":"2024-04-25T09:54:56.135595+0000","last_active":"2024-04-25T09:54:57.140441+0000","last_peered":"2024-04-25T09:54:57.140441+0000","last_clean":"2024-04-25T09:54:57.140441+0000","last_became_active":"2024-04-25T09:54:56.135263+0000","last_became_peered":"2024-04-25T09:54:56.135263+0000","last_unstale":"2024-04-25T09:54:57.140441+0000","last_undegraded":"2024-04-25T09:54:57.140441+0000","last_fullsized":"2024-04-25T09:54:57.140441+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:20.570537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.150300+0000","last_change":"2024-04-25T09:54:57.145292+0000","last_active":"2024-04-25T09:54:57.150300+0000","last_peered":"2024-04-25T09:54:57.150300+0000","last_clean":"2024-04-25T09:54:57.150300+0000","last_became_active":"2024-04-25T09:54:57.144946+0000","last_became_peered":"2024-04-25T09:54:57.144946+0000","last_unstale":"2024-04-25T09:54:57.150300+0000","last_undegraded":"2024-04-25T09:54:57.150300+0000","last_fullsized":"2024-04-25T09:54:57.150300+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:25.632038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.126214+0000","last_change":"2024-04-25T09:54:56.139944+0000","last_active":"2024-04-25T09:54:57.126214+0000","last_peered":"2024-04-25T09:54:57.126214+0000","last_clean":"2024-04-25T09:54:57.126214+0000","last_became_active":"2024-04-25T09:54:56.139710+0000","last_became_peered":"2024-04-25T09:54:56.139710+0000","last_unstale":"2024-04-25T09:54:57.126214+0000","last_undegraded":"2024-04-25T09:54:57.126214+0000","last_fullsized":"2024-04-25T09:54:57.126214+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:32.903659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.146584+0000","last_change":"2024-04-25T09:54:57.137871+0000","last_active":"2024-04-25T09:54:57.146584+0000","last_peered":"2024-04-25T09:54:57.146584+0000","last_clean":"2024-04-25T09:54:57.146584+0000","last_became_active":"2024-04-25T09:54:57.137608+0000","last_became_peered":"2024-04-25T09:54:57.137608+0000","last_unstale":"2024-04-25T09:54:57.146584+0000","last_undegraded":"2024-04-25T09:54:57.146584+0000","last_fullsized":"2024-04-25T09:54:57.146584+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:27.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.120268+0000","last_change":"2024-04-25T09:54:56.132960+0000","last_active":"2024-04-25T09:54:57.120268+0000","last_peered":"2024-04-25T09:54:57.120268+0000","last_clean":"2024-04-25T09:54:57.120268+0000","last_became_active":"2024-04-25T09:54:56.132686+0000","last_became_peered":"2024-04-25T09:54:56.132686+0000","last_unstale":"2024-04-25T09:54:57.120268+0000","last_undegraded":"2024-04-25T09:54:57.120268+0000","last_fullsized":"2024-04-25T09:54:57.120268+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:18.642389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.152839+0000","last_change":"2024-04-25T09:54:57.149052+0000","last_active":"2024-04-25T09:54:57.152839+0000","last_peered":"2024-04-25T09:54:57.152839+0000","last_clean":"2024-04-25T09:54:57.152839+0000","last_became_active":"2024-04-25T09:54:57.148769+0000","last_became_peered":"2024-04-25T09:54:57.148769+0000","last_unstale":"2024-04-25T09:54:57.152839+0000","last_undegraded":"2024-04-25T09:54:57.152839+0000","last_fullsized":"2024-04-25T09:54:57.152839+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:28.119357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.132904+0000","last_change":"2024-04-25T09:54:56.127762+0000","last_active":"2024-04-25T09:54:57.132904+0000","last_peered":"2024-04-25T09:54:57.132904+0000","last_clean":"2024-04-25T09:54:57.132904+0000","last_became_active":"2024-04-25T09:54:56.127131+0000","last_became_peered":"2024-04-25T09:54:56.127131+0000","last_unstale":"2024-04-25T09:54:57.132904+0000","last_undegraded":"2024-04-25T09:54:57.132904+0000","last_fullsized":"2024-04-25T09:54:57.132904+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:20.114582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.138022+0000","last_change":"2024-04-25T09:54:57.134598+0000","last_active":"2024-04-25T09:54:57.138022+0000","last_peered":"2024-04-25T09:54:57.138022+0000","last_clean":"2024-04-25T09:54:57.138022+0000","last_became_active":"2024-04-25T09:54:57.134367+0000","last_became_peered":"2024-04-25T09:54:57.134367+0000","last_unstale":"2024-04-25T09:54:57.138022+0000","last_undegraded":"2024-04-25T09:54:57.138022+0000","last_fullsized":"2024-04-25T09:54:57.138022+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:21.118874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.132984+0000","last_change":"2024-04-25T09:54:56.127052+0000","last_active":"2024-04-25T09:54:57.132984+0000","last_peered":"2024-04-25T09:54:57.132984+0000","last_clean":"2024-04-25T09:54:57.132984+0000","last_became_active":"2024-04-25T09:54:56.126781+0000","last_became_peered":"2024-04-25T09:54:56.126781+0000","last_unstale":"2024-04-25T09:54:57.132984+0000","last_undegraded":"2024-04-25T09:54:57.132984+0000","last_fullsized":"2024-04-25T09:54:57.132984+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:02.695951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.146164+0000","last_change":"2024-04-25T09:54:57.141310+0000","last_active":"2024-04-25T09:54:57.146164+0000","last_peered":"2024-04-25T09:54:57.146164+0000","last_clean":"2024-04-25T09:54:57.146164+0000","last_became_active":"2024-04-25T09:54:57.141047+0000","last_became_peered":"2024-04-25T09:54:57.141047+0000","last_unstale":"2024-04-25T09:54:57.146164+0000","last_undegraded":"2024-04-25T09:54:57.146164+0000","last_fullsized":"2024-04-25T09:54:57.146164+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:16.656767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.118754+0000","last_change":"2024-04-25T09:54:56.127914+0000","last_active":"2024-04-25T09:54:57.118754+0000","last_peered":"2024-04-25T09:54:57.118754+0000","last_clean":"2024-04-25T09:54:57.118754+0000","last_became_active":"2024-04-25T09:54:56.127633+0000","last_became_peered":"2024-04-25T09:54:56.127633+0000","last_unstale":"2024-04-25T09:54:57.118754+0000","last_undegraded":"2024-04-25T09:54:57.118754+0000","last_fullsized":"2024-04-25T09:54:57.118754+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:08.630451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153235+0000","last_change":"2024-04-25T09:54:57.145867+0000","last_active":"2024-04-25T09:54:57.153235+0000","last_peered":"2024-04-25T09:54:57.153235+0000","last_clean":"2024-04-25T09:54:57.153235+0000","last_became_active":"2024-04-25T09:54:57.145360+0000","last_became_peered":"2024-04-25T09:54:57.145360+0000","last_unstale":"2024-04-25T09:54:57.153235+0000","last_undegraded":"2024-04-25T09:54:57.153235+0000","last_fullsized":"2024-04-25T09:54:57.153235+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.248939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.122548+0000","last_change":"2024-04-25T09:54:56.133958+0000","last_active":"2024-04-25T09:54:57.122548+0000","last_peered":"2024-04-25T09:54:57.122548+0000","last_clean":"2024-04-25T09:54:57.122548+0000","last_became_active":"2024-04-25T09:54:56.133695+0000","last_became_peered":"2024-04-25T09:54:56.133695+0000","last_unstale":"2024-04-25T09:54:57.122548+0000","last_undegraded":"2024-04-25T09:54:57.122548+0000","last_fullsized":"2024-04-25T09:54:57.122548+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:01.461519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.143478+0000","last_change":"2024-04-25T09:54:57.139528+0000","last_active":"2024-04-25T09:54:57.143478+0000","last_peered":"2024-04-25T09:54:57.143478+0000","last_clean":"2024-04-25T09:54:57.143478+0000","last_became_active":"2024-04-25T09:54:57.139238+0000","last_became_peered":"2024-04-25T09:54:57.139238+0000","last_unstale":"2024-04-25T09:54:57.143478+0000","last_undegraded":"2024-04-25T09:54:57.143478+0000","last_fullsized":"2024-04-25T09:54:57.143478+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:40.623235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.120963+0000","last_change":"2024-04-25T09:54:56.133355+0000","last_active":"2024-04-25T09:54:57.120963+0000","last_peered":"2024-04-25T09:54:57.120963+0000","last_clean":"2024-04-25T09:54:57.120963+0000","last_became_active":"2024-04-25T09:54:56.133055+0000","last_became_peered":"2024-04-25T09:54:56.133055+0000","last_unstale":"2024-04-25T09:54:57.120963+0000","last_undegraded":"2024-04-25T09:54:57.120963+0000","last_fullsized":"2024-04-25T09:54:57.120963+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:05.618687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.129154+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:54:57.129154+0000","last_peered":"2024-04-25T09:54:57.129154+0000","last_clean":"2024-04-25T09:54:57.129154+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:54:57.129154+0000","last_undegraded":"2024-04-25T09:54:57.129154+0000","last_fullsized":"2024-04-25T09:54:57.129154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1183744,"data_stored":847047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":257207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":581632,"data_stored":256555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":585728,"data_stored":257137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":598016,"data_stored":258189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":258167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":256715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1183744,"data_stored":846979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:55:00.083 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:55:00.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.213+0000 7f3b2d022640 1 Processor -- start 2024-04-25T09:55:00.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.213+0000 7f3b2d022640 1 -- start start 2024-04-25T09:55:00.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.214+0000 7f3b2d022640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3b2806aff0 0x7f3b280b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.214+0000 7f3b2d022640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2806b480 con 0x7f3b28160e90 2024-04-25T09:55:00.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.214+0000 7f3b2d022640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b280d0400 con 0x7f3b2806ab60 2024-04-25T09:55:00.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.214+0000 7f3b2d022640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b280d14a0 con 0x7f3b2806aff0 2024-04-25T09:55:00.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.214+0000 7f3b26d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3b28160e90 0x7f3b280afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35996/0 (socket says 172.21.15.139:35996) 2024-04-25T09:55:00.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.214+0000 7f3b26d9d640 1 -- 172.21.15.139:0/3215587545 learned_addr learned my addr 172.21.15.139:0/3215587545 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:00.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.214+0000 7f3b2759e640 1 --2- 172.21.15.139:0/3215587545 >> v2:172.21.15.150:3300/0 conn(0x7f3b2806aff0 0x7f3b280b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.214+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14961468 0 0) 0x7f3b2806b480 con 0x7f3b28160e90 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.214+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b10003520 con 0x7f3b28160e90 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.214+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547578978 0 0) 0x7f3b280d0400 con 0x7f3b2806ab60 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.214+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b2806b480 con 0x7f3b2806ab60 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2833128044 0 0) 0x7f3b10003520 con 0x7f3b28160e90 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 >> v2:172.21.15.150:3300/0 conn(0x7f3b2806aff0 msgr2=0x7f3b280b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 --2- 172.21.15.139:0/3215587545 >> v2:172.21.15.150:3300/0 conn(0x7f3b2806aff0 0x7f3b280b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 >> v1:172.21.15.150:6789/0 conn(0x7f3b2806ab60 legacy=0x7f3b280b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b280d06e0 con 0x7f3b28160e90 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f3b180019f0 con 0x7f3b28160e90 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 >> v1:172.21.15.139:6789/0 conn(0x7f3b28160e90 legacy=0x7f3b280afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 --2- 172.21.15.139:0/3215587545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b10003b40 0x7f3b10003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 --2- 172.21.15.139:0/3215587545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b10004800 0x7f3b10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 --2- 172.21.15.139:0/3215587545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b100051c0 0x7f3b1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280d14a0 con 0x7f3b10003b40 2024-04-25T09:55:00.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1000c110 con 0x7f3b100051c0 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1000c310 con 0x7f3b10004800 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b26d9d640 1 --2- 172.21.15.139:0/3215587545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b10003b40 0x7f3b10003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b2659c640 1 --2- 172.21.15.139:0/3215587545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b10004800 0x7f3b10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b2759e640 1 --2- 172.21.15.139:0/3215587545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b100051c0 0x7f3b1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b26d9d640 1 -- 172.21.15.139:0/3215587545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b10004800 msgr2=0x7f3b10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b26d9d640 1 --2- 172.21.15.139:0/3215587545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b10004800 0x7f3b10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b26d9d640 1 -- 172.21.15.139:0/3215587545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b100051c0 msgr2=0x7f3b1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b26d9d640 1 --2- 172.21.15.139:0/3215587545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b100051c0 0x7f3b1000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b2759e640 1 -- 172.21.15.139:0/3215587545 reap_dead start 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.215+0000 7f3b26d9d640 1 -- 172.21.15.139:0/3215587545 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b10003520 con 0x7f3b10003b40 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.216+0000 7f3b26d9d640 1 --2- 172.21.15.139:0/3215587545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b10003b40 0x7f3b10003f40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800a810 tx=0x7f3b10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d7b50539ff41f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.216+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800a430 con 0x7f3b10003b40 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.216+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1802f070 con 0x7f3b10003b40 2024-04-25T09:55:00.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.216+0000 7f3b25d9b640 1 -- 172.21.15.139:0/3215587545 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1802bcb0 con 0x7f3b10003b40 2024-04-25T09:55:00.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.216+0000 7f3b2d022640 1 -- 172.21.15.139:0/3215587545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b10003b40 msgr2=0x7f3b10003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.216+0000 7f3b2d022640 1 --2- 172.21.15.139:0/3215587545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b10003b40 0x7f3b10003f40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800a810 tx=0x7f3b10005830 comp rx=0 tx=0).stop 2024-04-25T09:55:00.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.216+0000 7f3b2d022640 1 -- 172.21.15.139:0/3215587545 shutdown_connections 2024-04-25T09:55:00.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.216+0000 7f3b2d022640 1 --2- 172.21.15.139:0/3215587545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b10003b40 0x7f3b10003f40 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.216+0000 7f3b2d022640 1 -- 172.21.15.139:0/3215587545 >> 172.21.15.139:0/3215587545 conn(0x7f3b280b23f0 msgr2=0x7f3b280b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:00.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.216+0000 7f3b2d022640 1 -- 172.21.15.139:0/3215587545 shutdown_connections 2024-04-25T09:55:00.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.216+0000 7f3b2d022640 1 -- 172.21.15.139:0/3215587545 wait complete. 2024-04-25T09:55:00.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.218+0000 7f3b2d022640 1 Processor -- start 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.218+0000 7f3b2d022640 1 -- start start 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.218+0000 7f3b2d022640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b10003b40 0x7f3b28154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.218+0000 7f3b2d022640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b280af000 0x7f3b28154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.218+0000 7f3b2d022640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b28155010 0x7f3b281e9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2d022640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280d0f80 con 0x7f3b28155010 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2d022640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280d0ca0 con 0x7f3b280af000 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2d022640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b28161270 con 0x7f3b10003b40 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b28155010 0x7f3b281e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b28155010 0x7f3b281e9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47202/0 (socket says 172.21.15.139:47202) 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2759e640 1 -- 172.21.15.139:0/3595751706 learned_addr learned my addr 172.21.15.139:0/3595751706 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2659c640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b280af000 0x7f3b28154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b26d9d640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b10003b40 0x7f3b28154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2759e640 1 -- 172.21.15.139:0/3595751706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b10003b40 msgr2=0x7f3b28154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2759e640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b10003b40 0x7f3b28154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2759e640 1 -- 172.21.15.139:0/3595751706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b280af000 msgr2=0x7f3b28154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2759e640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b280af000 0x7f3b28154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2759e640 1 -- 172.21.15.139:0/3595751706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b280a8700 con 0x7f3b28155010 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2759e640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b28155010 0x7f3b281e9470 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3b280b5cf0 tx=0x7f3b20011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b0f7fe640 1 -- 172.21.15.139:0/3595751706 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20019020 con 0x7f3b28155010 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b0f7fe640 1 -- 172.21.15.139:0/3595751706 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b20003610 con 0x7f3b28155010 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b0f7fe640 1 -- 172.21.15.139:0/3595751706 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20003a00 con 0x7f3b28155010 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2d022640 1 -- 172.21.15.139:0/3595751706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b280a89e0 con 0x7f3b28155010 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.219+0000 7f3b2d022640 1 -- 172.21.15.139:0/3595751706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b281e9b90 con 0x7f3b28155010 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.220+0000 7f3b2d022640 1 -- 172.21.15.139:0/3595751706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b28074520 con 0x7f3b28155010 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.221+0000 7f3b0f7fe640 1 -- 172.21.15.139:0/3595751706 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b2001c020 con 0x7f3b28155010 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.221+0000 7f3b0f7fe640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b1406b4d0 0x7f3b1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.221+0000 7f3b0f7fe640 1 -- 172.21.15.139:0/3595751706 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3b2008a9b0 con 0x7f3b28155010 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.221+0000 7f3b26d9d640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b1406b4d0 0x7f3b1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.222+0000 7f3b26d9d640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b1406b4d0 0x7f3b1406d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3b18033000 tx=0x7f3b18038040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:00.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.226+0000 7f3b0f7fe640 1 -- 172.21.15.139:0/3595751706 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b20023020 con 0x7f3b28155010 2024-04-25T09:55:00.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.391+0000 7f3b2d022640 1 -- 172.21.15.139:0/3595751706 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b28154850 con 0x7f3b1406b4d0 2024-04-25T09:55:00.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.397+0000 7f3b0f7fe640 1 -- 172.21.15.139:0/3595751706 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251889 (secure 0 0 0) 0x7f3b1802f040 con 0x7f3b1406b4d0 2024-04-25T09:55:00.403 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:00.404 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:55:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.400+0000 7f3b2d022640 1 -- 172.21.15.139:0/3595751706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b1406b4d0 msgr2=0x7f3b1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.400+0000 7f3b2d022640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b1406b4d0 0x7f3b1406d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3b18033000 tx=0x7f3b18038040 comp rx=0 tx=0).stop 2024-04-25T09:55:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.400+0000 7f3b2d022640 1 -- 172.21.15.139:0/3595751706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b28155010 msgr2=0x7f3b281e9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.400+0000 7f3b2d022640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b28155010 0x7f3b281e9470 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3b280b5cf0 tx=0x7f3b20011920 comp rx=0 tx=0).stop 2024-04-25T09:55:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.400+0000 7f3b2d022640 1 -- 172.21.15.139:0/3595751706 shutdown_connections 2024-04-25T09:55:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.400+0000 7f3b2d022640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b28155010 0x7f3b281e9470 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.400+0000 7f3b2d022640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b1406b4d0 0x7f3b1406d990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.400+0000 7f3b2d022640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b280af000 0x7f3b28154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.400+0000 7f3b2d022640 1 --2- 172.21.15.139:0/3595751706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b10003b40 0x7f3b28154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.400+0000 7f3b2d022640 1 -- 172.21.15.139:0/3595751706 >> 172.21.15.139:0/3595751706 conn(0x7f3b280b23f0 msgr2=0x7f3b281614e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:00.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.400+0000 7f3b2d022640 1 -- 172.21.15.139:0/3595751706 shutdown_connections 2024-04-25T09:55:00.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.400+0000 7f3b2d022640 1 -- 172.21.15.139:0/3595751706 wait complete. 2024-04-25T09:55:00.414 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-04-25T09:54:58.987994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221976,"kb_used_data":5768,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752744,"statfs":{"total":773094113280,"available":772866809856,"internally_reserved":0,"allocated":5906432,"data_stored":3237996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000580"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.138921+0000","last_change":"2024-04-25T09:54:57.134155+0000","last_active":"2024-04-25T09:54:57.138921+0000","last_peered":"2024-04-25T09:54:57.138921+0000","last_clean":"2024-04-25T09:54:57.138921+0000","last_became_active":"2024-04-25T09:54:57.133892+0000","last_became_peered":"2024-04-25T09:54:57.133892+0000","last_unstale":"2024-04-25T09:54:57.138921+0000","last_undegraded":"2024-04-25T09:54:57.138921+0000","last_fullsized":"2024-04-25T09:54:57.138921+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:28.927694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145633+0000","last_change":"2024-04-25T09:54:57.140060+0000","last_active":"2024-04-25T09:54:57.145633+0000","last_peered":"2024-04-25T09:54:57.145633+0000","last_clean":"2024-04-25T09:54:57.145633+0000","last_became_active":"2024-04-25T09:54:57.139420+0000","last_became_peered":"2024-04-25T09:54:57.139420+0000","last_unstale":"2024-04-25T09:54:57.145633+0000","last_undegraded":"2024-04-25T09:54:57.145633+0000","last_fullsized":"2024-04-25T09:54:57.145633+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:21.118874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133967+0000","last_change":"2024-04-25T09:54:57.130491+0000","last_active":"2024-04-25T09:54:57.133967+0000","last_peered":"2024-04-25T09:54:57.133967+0000","last_clean":"2024-04-25T09:54:57.133967+0000","last_became_active":"2024-04-25T09:54:57.130232+0000","last_became_peered":"2024-04-25T09:54:57.130232+0000","last_unstale":"2024-04-25T09:54:57.133967+0000","last_undegraded":"2024-04-25T09:54:57.133967+0000","last_fullsized":"2024-04-25T09:54:57.133967+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:48.083442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.151445+0000","last_change":"2024-04-25T09:54:57.148574+0000","last_active":"2024-04-25T09:54:57.151445+0000","last_peered":"2024-04-25T09:54:57.151445+0000","last_clean":"2024-04-25T09:54:57.151445+0000","last_became_active":"2024-04-25T09:54:57.148284+0000","last_became_peered":"2024-04-25T09:54:57.148284+0000","last_unstale":"2024-04-25T09:54:57.151445+0000","last_undegraded":"2024-04-25T09:54:57.151445+0000","last_fullsized":"2024-04-25T09:54:57.151445+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:27.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.136327+0000","last_change":"2024-04-25T09:54:57.129129+0000","last_active":"2024-04-25T09:54:57.136327+0000","last_peered":"2024-04-25T09:54:57.136327+0000","last_clean":"2024-04-25T09:54:57.136327+0000","last_became_active":"2024-04-25T09:54:57.128839+0000","last_became_peered":"2024-04-25T09:54:57.128839+0000","last_unstale":"2024-04-25T09:54:57.136327+0000","last_undegraded":"2024-04-25T09:54:57.136327+0000","last_fullsized":"2024-04-25T09:54:57.136327+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:28.927694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.149512+0000","last_change":"2024-04-25T09:54:57.142040+0000","last_active":"2024-04-25T09:54:57.149512+0000","last_peered":"2024-04-25T09:54:57.149512+0000","last_clean":"2024-04-25T09:54:57.149512+0000","last_became_active":"2024-04-25T09:54:57.141781+0000","last_became_peered":"2024-04-25T09:54:57.141781+0000","last_unstale":"2024-04-25T09:54:57.149512+0000","last_undegraded":"2024-04-25T09:54:57.149512+0000","last_fullsized":"2024-04-25T09:54:57.149512+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:40.623235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.146237+0000","last_change":"2024-04-25T09:54:57.142017+0000","last_active":"2024-04-25T09:54:57.146237+0000","last_peered":"2024-04-25T09:54:57.146237+0000","last_clean":"2024-04-25T09:54:57.146237+0000","last_became_active":"2024-04-25T09:54:57.141514+0000","last_became_peered":"2024-04-25T09:54:57.141514+0000","last_unstale":"2024-04-25T09:54:57.146237+0000","last_undegraded":"2024-04-25T09:54:57.146237+0000","last_fullsized":"2024-04-25T09:54:57.146237+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:26.700661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153317+0000","last_change":"2024-04-25T09:54:57.147430+0000","last_active":"2024-04-25T09:54:57.153317+0000","last_peered":"2024-04-25T09:54:57.153317+0000","last_clean":"2024-04-25T09:54:57.153317+0000","last_became_active":"2024-04-25T09:54:57.147193+0000","last_became_peered":"2024-04-25T09:54:57.147193+0000","last_unstale":"2024-04-25T09:54:57.153317+0000","last_undegraded":"2024-04-25T09:54:57.153317+0000","last_fullsized":"2024-04-25T09:54:57.153317+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:41.646452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153272+0000","last_change":"2024-04-25T09:54:57.146684+0000","last_active":"2024-04-25T09:54:57.153272+0000","last_peered":"2024-04-25T09:54:57.153272+0000","last_clean":"2024-04-25T09:54:57.153272+0000","last_became_active":"2024-04-25T09:54:57.146438+0000","last_became_peered":"2024-04-25T09:54:57.146438+0000","last_unstale":"2024-04-25T09:54:57.153272+0000","last_undegraded":"2024-04-25T09:54:57.153272+0000","last_fullsized":"2024-04-25T09:54:57.153272+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:18.716099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.150365+0000","last_change":"2024-04-25T09:54:57.146471+0000","last_active":"2024-04-25T09:54:57.150365+0000","last_peered":"2024-04-25T09:54:57.150365+0000","last_clean":"2024-04-25T09:54:57.150365+0000","last_became_active":"2024-04-25T09:54:57.145678+0000","last_became_peered":"2024-04-25T09:54:57.145678+0000","last_unstale":"2024-04-25T09:54:57.150365+0000","last_undegraded":"2024-04-25T09:54:57.150365+0000","last_fullsized":"2024-04-25T09:54:57.150365+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:16.487981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.143129+0000","last_change":"2024-04-25T09:54:57.137284+0000","last_active":"2024-04-25T09:54:57.143129+0000","last_peered":"2024-04-25T09:54:57.143129+0000","last_clean":"2024-04-25T09:54:57.143129+0000","last_became_active":"2024-04-25T09:54:57.137025+0000","last_became_peered":"2024-04-25T09:54:57.137025+0000","last_unstale":"2024-04-25T09:54:57.143129+0000","last_undegraded":"2024-04-25T09:54:57.143129+0000","last_fullsized":"2024-04-25T09:54:57.143129+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:51.950481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153339+0000","last_change":"2024-04-25T09:54:57.146961+0000","last_active":"2024-04-25T09:54:57.153339+0000","last_peered":"2024-04-25T09:54:57.153339+0000","last_clean":"2024-04-25T09:54:57.153339+0000","last_became_active":"2024-04-25T09:54:57.146588+0000","last_became_peered":"2024-04-25T09:54:57.146588+0000","last_unstale":"2024-04-25T09:54:57.153339+0000","last_undegraded":"2024-04-25T09:54:57.153339+0000","last_fullsized":"2024-04-25T09:54:57.153339+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:20.214884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.149552+0000","last_change":"2024-04-25T09:54:57.140222+0000","last_active":"2024-04-25T09:54:57.149552+0000","last_peered":"2024-04-25T09:54:57.149552+0000","last_clean":"2024-04-25T09:54:57.149552+0000","last_became_active":"2024-04-25T09:54:57.139948+0000","last_became_peered":"2024-04-25T09:54:57.139948+0000","last_unstale":"2024-04-25T09:54:57.149552+0000","last_undegraded":"2024-04-25T09:54:57.149552+0000","last_fullsized":"2024-04-25T09:54:57.149552+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:03.262970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133925+0000","last_change":"2024-04-25T09:54:57.129803+0000","last_active":"2024-04-25T09:54:57.133925+0000","last_peered":"2024-04-25T09:54:57.133925+0000","last_clean":"2024-04-25T09:54:57.133925+0000","last_became_active":"2024-04-25T09:54:57.129053+0000","last_became_peered":"2024-04-25T09:54:57.129053+0000","last_unstale":"2024-04-25T09:54:57.133925+0000","last_undegraded":"2024-04-25T09:54:57.133925+0000","last_fullsized":"2024-04-25T09:54:57.133925+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:10.677088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137346+0000","last_change":"2024-04-25T09:54:57.132487+0000","last_active":"2024-04-25T09:54:57.137346+0000","last_peered":"2024-04-25T09:54:57.137346+0000","last_clean":"2024-04-25T09:54:57.137346+0000","last_became_active":"2024-04-25T09:54:57.132214+0000","last_became_peered":"2024-04-25T09:54:57.132214+0000","last_unstale":"2024-04-25T09:54:57.137346+0000","last_undegraded":"2024-04-25T09:54:57.137346+0000","last_fullsized":"2024-04-25T09:54:57.137346+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:02.465811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.149733+0000","last_change":"2024-04-25T09:54:57.141399+0000","last_active":"2024-04-25T09:54:57.149733+0000","last_peered":"2024-04-25T09:54:57.149733+0000","last_clean":"2024-04-25T09:54:57.149733+0000","last_became_active":"2024-04-25T09:54:57.141133+0000","last_became_peered":"2024-04-25T09:54:57.141133+0000","last_unstale":"2024-04-25T09:54:57.149733+0000","last_undegraded":"2024-04-25T09:54:57.149733+0000","last_fullsized":"2024-04-25T09:54:57.149733+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:13.099355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.141529+0000","last_change":"2024-04-25T09:54:57.135389+0000","last_active":"2024-04-25T09:54:57.141529+0000","last_peered":"2024-04-25T09:54:57.141529+0000","last_clean":"2024-04-25T09:54:57.141529+0000","last_became_active":"2024-04-25T09:54:57.135132+0000","last_became_peered":"2024-04-25T09:54:57.135132+0000","last_unstale":"2024-04-25T09:54:57.141529+0000","last_undegraded":"2024-04-25T09:54:57.141529+0000","last_fullsized":"2024-04-25T09:54:57.141529+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:40.882549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145654+0000","last_change":"2024-04-25T09:54:57.140271+0000","last_active":"2024-04-25T09:54:57.145654+0000","last_peered":"2024-04-25T09:54:57.145654+0000","last_clean":"2024-04-25T09:54:57.145654+0000","last_became_active":"2024-04-25T09:54:57.139742+0000","last_became_peered":"2024-04-25T09:54:57.139742+0000","last_unstale":"2024-04-25T09:54:57.145654+0000","last_undegraded":"2024-04-25T09:54:57.145654+0000","last_fullsized":"2024-04-25T09:54:57.145654+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:03.700244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133946+0000","last_change":"2024-04-25T09:54:57.128688+0000","last_active":"2024-04-25T09:54:57.133946+0000","last_peered":"2024-04-25T09:54:57.133946+0000","last_clean":"2024-04-25T09:54:57.133946+0000","last_became_active":"2024-04-25T09:54:57.127892+0000","last_became_peered":"2024-04-25T09:54:57.127892+0000","last_unstale":"2024-04-25T09:54:57.133946+0000","last_undegraded":"2024-04-25T09:54:57.133946+0000","last_fullsized":"2024-04-25T09:54:57.133946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:26.467246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.140345+0000","last_change":"2024-04-25T09:54:57.134475+0000","last_active":"2024-04-25T09:54:57.140345+0000","last_peered":"2024-04-25T09:54:57.140345+0000","last_clean":"2024-04-25T09:54:57.140345+0000","last_became_active":"2024-04-25T09:54:57.134200+0000","last_became_peered":"2024-04-25T09:54:57.134200+0000","last_unstale":"2024-04-25T09:54:57.140345+0000","last_undegraded":"2024-04-25T09:54:57.140345+0000","last_fullsized":"2024-04-25T09:54:57.140345+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:18.173854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.143262+0000","last_change":"2024-04-25T09:54:57.139077+0000","last_active":"2024-04-25T09:54:57.143262+0000","last_peered":"2024-04-25T09:54:57.143262+0000","last_clean":"2024-04-25T09:54:57.143262+0000","last_became_active":"2024-04-25T09:54:57.138699+0000","last_became_peered":"2024-04-25T09:54:57.138699+0000","last_unstale":"2024-04-25T09:54:57.143262+0000","last_undegraded":"2024-04-25T09:54:57.143262+0000","last_fullsized":"2024-04-25T09:54:57.143262+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:13.099355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153315+0000","last_change":"2024-04-25T09:54:57.147320+0000","last_active":"2024-04-25T09:54:57.153315+0000","last_peered":"2024-04-25T09:54:57.153315+0000","last_clean":"2024-04-25T09:54:57.153315+0000","last_became_active":"2024-04-25T09:54:57.147083+0000","last_became_peered":"2024-04-25T09:54:57.147083+0000","last_unstale":"2024-04-25T09:54:57.153315+0000","last_undegraded":"2024-04-25T09:54:57.153315+0000","last_fullsized":"2024-04-25T09:54:57.153315+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:03.670032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.143022+0000","last_change":"2024-04-25T09:54:57.136509+0000","last_active":"2024-04-25T09:54:57.143022+0000","last_peered":"2024-04-25T09:54:57.143022+0000","last_clean":"2024-04-25T09:54:57.143022+0000","last_became_active":"2024-04-25T09:54:57.136259+0000","last_became_peered":"2024-04-25T09:54:57.136259+0000","last_unstale":"2024-04-25T09:54:57.143022+0000","last_undegraded":"2024-04-25T09:54:57.143022+0000","last_fullsized":"2024-04-25T09:54:57.143022+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:34.781194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.134011+0000","last_change":"2024-04-25T09:54:57.130661+0000","last_active":"2024-04-25T09:54:57.134011+0000","last_peered":"2024-04-25T09:54:57.134011+0000","last_clean":"2024-04-25T09:54:57.134011+0000","last_became_active":"2024-04-25T09:54:57.129190+0000","last_became_peered":"2024-04-25T09:54:57.129190+0000","last_unstale":"2024-04-25T09:54:57.134011+0000","last_undegraded":"2024-04-25T09:54:57.134011+0000","last_fullsized":"2024-04-25T09:54:57.134011+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:29.718809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.143897+0000","last_change":"2024-04-25T09:54:57.140635+0000","last_active":"2024-04-25T09:54:57.143897+0000","last_peered":"2024-04-25T09:54:57.143897+0000","last_clean":"2024-04-25T09:54:57.143897+0000","last_became_active":"2024-04-25T09:54:57.140340+0000","last_became_peered":"2024-04-25T09:54:57.140340+0000","last_unstale":"2024-04-25T09:54:57.143897+0000","last_undegraded":"2024-04-25T09:54:57.143897+0000","last_fullsized":"2024-04-25T09:54:57.143897+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:00.061134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145999+0000","last_change":"2024-04-25T09:54:57.145356+0000","last_active":"2024-04-25T09:54:57.145999+0000","last_peered":"2024-04-25T09:54:57.145999+0000","last_clean":"2024-04-25T09:54:57.145999+0000","last_became_active":"2024-04-25T09:54:57.145032+0000","last_became_peered":"2024-04-25T09:54:57.145032+0000","last_unstale":"2024-04-25T09:54:57.145999+0000","last_undegraded":"2024-04-25T09:54:57.145999+0000","last_fullsized":"2024-04-25T09:54:57.145999+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:16.656767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.139193+0000","last_change":"2024-04-25T09:54:57.136196+0000","last_active":"2024-04-25T09:54:57.139193+0000","last_peered":"2024-04-25T09:54:57.139193+0000","last_clean":"2024-04-25T09:54:57.139193+0000","last_became_active":"2024-04-25T09:54:57.135927+0000","last_became_peered":"2024-04-25T09:54:57.135927+0000","last_unstale":"2024-04-25T09:54:57.139193+0000","last_undegraded":"2024-04-25T09:54:57.139193+0000","last_fullsized":"2024-04-25T09:54:57.139193+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:18.173854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145985+0000","last_change":"2024-04-25T09:54:57.141050+0000","last_active":"2024-04-25T09:54:57.145985+0000","last_peered":"2024-04-25T09:54:57.145985+0000","last_clean":"2024-04-25T09:54:57.145985+0000","last_became_active":"2024-04-25T09:54:57.140765+0000","last_became_peered":"2024-04-25T09:54:57.140765+0000","last_unstale":"2024-04-25T09:54:57.145985+0000","last_undegraded":"2024-04-25T09:54:57.145985+0000","last_fullsized":"2024-04-25T09:54:57.145985+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:25.632038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133857+0000","last_change":"2024-04-25T09:54:57.128890+0000","last_active":"2024-04-25T09:54:57.133857+0000","last_peered":"2024-04-25T09:54:57.133857+0000","last_clean":"2024-04-25T09:54:57.133857+0000","last_became_active":"2024-04-25T09:54:57.128600+0000","last_became_peered":"2024-04-25T09:54:57.128600+0000","last_unstale":"2024-04-25T09:54:57.133857+0000","last_undegraded":"2024-04-25T09:54:57.133857+0000","last_fullsized":"2024-04-25T09:54:57.133857+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:33.907952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137399+0000","last_change":"2024-04-25T09:54:57.132629+0000","last_active":"2024-04-25T09:54:57.137399+0000","last_peered":"2024-04-25T09:54:57.137399+0000","last_clean":"2024-04-25T09:54:57.137399+0000","last_became_active":"2024-04-25T09:54:57.132356+0000","last_became_peered":"2024-04-25T09:54:57.132356+0000","last_unstale":"2024-04-25T09:54:57.137399+0000","last_undegraded":"2024-04-25T09:54:57.137399+0000","last_fullsized":"2024-04-25T09:54:57.137399+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:10.677088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.146218+0000","last_change":"2024-04-25T09:54:57.141802+0000","last_active":"2024-04-25T09:54:57.146218+0000","last_peered":"2024-04-25T09:54:57.146218+0000","last_clean":"2024-04-25T09:54:57.146218+0000","last_became_active":"2024-04-25T09:54:57.141156+0000","last_became_peered":"2024-04-25T09:54:57.141156+0000","last_unstale":"2024-04-25T09:54:57.146218+0000","last_undegraded":"2024-04-25T09:54:57.146218+0000","last_fullsized":"2024-04-25T09:54:57.146218+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.248939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.140273+0000","last_change":"2024-04-25T09:54:57.137957+0000","last_active":"2024-04-25T09:54:57.140273+0000","last_peered":"2024-04-25T09:54:57.140273+0000","last_clean":"2024-04-25T09:54:57.140273+0000","last_became_active":"2024-04-25T09:54:57.136307+0000","last_became_peered":"2024-04-25T09:54:57.136307+0000","last_unstale":"2024-04-25T09:54:57.140273+0000","last_undegraded":"2024-04-25T09:54:57.140273+0000","last_fullsized":"2024-04-25T09:54:57.140273+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:00.061134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137467+0000","last_change":"2024-04-25T09:54:57.134362+0000","last_active":"2024-04-25T09:54:57.137467+0000","last_peered":"2024-04-25T09:54:57.137467+0000","last_clean":"2024-04-25T09:54:57.137467+0000","last_became_active":"2024-04-25T09:54:57.133972+0000","last_became_peered":"2024-04-25T09:54:57.133972+0000","last_unstale":"2024-04-25T09:54:57.137467+0000","last_undegraded":"2024-04-25T09:54:57.137467+0000","last_fullsized":"2024-04-25T09:54:57.137467+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:29.718809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.140868+0000","last_change":"2024-04-25T09:54:56.131484+0000","last_active":"2024-04-25T09:54:57.140868+0000","last_peered":"2024-04-25T09:54:57.140868+0000","last_clean":"2024-04-25T09:54:57.140868+0000","last_became_active":"2024-04-25T09:54:56.131213+0000","last_became_peered":"2024-04-25T09:54:56.131213+0000","last_unstale":"2024-04-25T09:54:57.140868+0000","last_undegraded":"2024-04-25T09:54:57.140868+0000","last_fullsized":"2024-04-25T09:54:57.140868+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:06.565946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137994+0000","last_change":"2024-04-25T09:54:57.135457+0000","last_active":"2024-04-25T09:54:57.137994+0000","last_peered":"2024-04-25T09:54:57.137994+0000","last_clean":"2024-04-25T09:54:57.137994+0000","last_became_active":"2024-04-25T09:54:57.135175+0000","last_became_peered":"2024-04-25T09:54:57.135175+0000","last_unstale":"2024-04-25T09:54:57.137994+0000","last_undegraded":"2024-04-25T09:54:57.137994+0000","last_fullsized":"2024-04-25T09:54:57.137994+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:03.700244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.149172+0000","last_change":"2024-04-25T09:54:56.125575+0000","last_active":"2024-04-25T09:54:57.149172+0000","last_peered":"2024-04-25T09:54:57.149172+0000","last_clean":"2024-04-25T09:54:57.149172+0000","last_became_active":"2024-04-25T09:54:56.125286+0000","last_became_peered":"2024-04-25T09:54:56.125286+0000","last_unstale":"2024-04-25T09:54:57.149172+0000","last_undegraded":"2024-04-25T09:54:57.149172+0000","last_fullsized":"2024-04-25T09:54:57.149172+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:32.072748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.139854+0000","last_change":"2024-04-25T09:54:57.134629+0000","last_active":"2024-04-25T09:54:57.139854+0000","last_peered":"2024-04-25T09:54:57.139854+0000","last_clean":"2024-04-25T09:54:57.139854+0000","last_became_active":"2024-04-25T09:54:57.134372+0000","last_became_peered":"2024-04-25T09:54:57.134372+0000","last_unstale":"2024-04-25T09:54:57.139854+0000","last_undegraded":"2024-04-25T09:54:57.139854+0000","last_fullsized":"2024-04-25T09:54:57.139854+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:03.262970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.148076+0000","last_change":"2024-04-25T09:54:56.131847+0000","last_active":"2024-04-25T09:54:57.148076+0000","last_peered":"2024-04-25T09:54:57.148076+0000","last_clean":"2024-04-25T09:54:57.148076+0000","last_became_active":"2024-04-25T09:54:56.131586+0000","last_became_peered":"2024-04-25T09:54:56.131586+0000","last_unstale":"2024-04-25T09:54:57.148076+0000","last_undegraded":"2024-04-25T09:54:57.148076+0000","last_fullsized":"2024-04-25T09:54:57.148076+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.602141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153255+0000","last_change":"2024-04-25T09:54:57.146456+0000","last_active":"2024-04-25T09:54:57.153255+0000","last_peered":"2024-04-25T09:54:57.153255+0000","last_clean":"2024-04-25T09:54:57.153255+0000","last_became_active":"2024-04-25T09:54:57.146210+0000","last_became_peered":"2024-04-25T09:54:57.146210+0000","last_unstale":"2024-04-25T09:54:57.153255+0000","last_undegraded":"2024-04-25T09:54:57.153255+0000","last_fullsized":"2024-04-25T09:54:57.153255+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:28.119357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.148108+0000","last_change":"2024-04-25T09:54:56.127140+0000","last_active":"2024-04-25T09:54:57.148108+0000","last_peered":"2024-04-25T09:54:57.148108+0000","last_clean":"2024-04-25T09:54:57.148108+0000","last_became_active":"2024-04-25T09:54:56.126860+0000","last_became_peered":"2024-04-25T09:54:56.126860+0000","last_unstale":"2024-04-25T09:54:57.148108+0000","last_undegraded":"2024-04-25T09:54:57.148108+0000","last_fullsized":"2024-04-25T09:54:57.148108+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:09.672795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.136908+0000","last_change":"2024-04-25T09:54:57.131179+0000","last_active":"2024-04-25T09:54:57.136908+0000","last_peered":"2024-04-25T09:54:57.136908+0000","last_clean":"2024-04-25T09:54:57.136908+0000","last_became_active":"2024-04-25T09:54:57.130888+0000","last_became_peered":"2024-04-25T09:54:57.130888+0000","last_unstale":"2024-04-25T09:54:57.136908+0000","last_undegraded":"2024-04-25T09:54:57.136908+0000","last_fullsized":"2024-04-25T09:54:57.136908+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:50.052465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.144526+0000","last_change":"2024-04-25T09:54:56.132128+0000","last_active":"2024-04-25T09:54:57.144526+0000","last_peered":"2024-04-25T09:54:57.144526+0000","last_clean":"2024-04-25T09:54:57.144526+0000","last_became_active":"2024-04-25T09:54:56.131030+0000","last_became_peered":"2024-04-25T09:54:56.131030+0000","last_unstale":"2024-04-25T09:54:57.144526+0000","last_undegraded":"2024-04-25T09:54:57.144526+0000","last_fullsized":"2024-04-25T09:54:57.144526+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:15.483688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137463+0000","last_change":"2024-04-25T09:54:57.134231+0000","last_active":"2024-04-25T09:54:57.137463+0000","last_peered":"2024-04-25T09:54:57.137463+0000","last_clean":"2024-04-25T09:54:57.137463+0000","last_became_active":"2024-04-25T09:54:57.133947+0000","last_became_peered":"2024-04-25T09:54:57.133947+0000","last_unstale":"2024-04-25T09:54:57.137463+0000","last_undegraded":"2024-04-25T09:54:57.137463+0000","last_fullsized":"2024-04-25T09:54:57.137463+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:48.083442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.148703+0000","last_change":"2024-04-25T09:54:56.132053+0000","last_active":"2024-04-25T09:54:57.148703+0000","last_peered":"2024-04-25T09:54:57.148703+0000","last_clean":"2024-04-25T09:54:57.148703+0000","last_became_active":"2024-04-25T09:54:56.131759+0000","last_became_peered":"2024-04-25T09:54:56.131759+0000","last_unstale":"2024-04-25T09:54:57.148703+0000","last_undegraded":"2024-04-25T09:54:57.148703+0000","last_fullsized":"2024-04-25T09:54:57.148703+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.602141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153214+0000","last_change":"2024-04-25T09:54:57.145880+0000","last_active":"2024-04-25T09:54:57.153214+0000","last_peered":"2024-04-25T09:54:57.153214+0000","last_clean":"2024-04-25T09:54:57.153214+0000","last_became_active":"2024-04-25T09:54:57.145620+0000","last_became_peered":"2024-04-25T09:54:57.145620+0000","last_unstale":"2024-04-25T09:54:57.153214+0000","last_undegraded":"2024-04-25T09:54:57.153214+0000","last_fullsized":"2024-04-25T09:54:57.153214+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:26.700661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.122621+0000","last_change":"2024-04-25T09:54:56.134125+0000","last_active":"2024-04-25T09:54:57.122621+0000","last_peered":"2024-04-25T09:54:57.122621+0000","last_clean":"2024-04-25T09:54:57.122621+0000","last_became_active":"2024-04-25T09:54:56.133818+0000","last_became_peered":"2024-04-25T09:54:56.133818+0000","last_unstale":"2024-04-25T09:54:57.122621+0000","last_undegraded":"2024-04-25T09:54:57.122621+0000","last_fullsized":"2024-04-25T09:54:57.122621+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:32.072748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145740+0000","last_change":"2024-04-25T09:54:57.142226+0000","last_active":"2024-04-25T09:54:57.145740+0000","last_peered":"2024-04-25T09:54:57.145740+0000","last_clean":"2024-04-25T09:54:57.145740+0000","last_became_active":"2024-04-25T09:54:57.141978+0000","last_became_peered":"2024-04-25T09:54:57.141978+0000","last_unstale":"2024-04-25T09:54:57.145740+0000","last_undegraded":"2024-04-25T09:54:57.145740+0000","last_fullsized":"2024-04-25T09:54:57.145740+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:40.882549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.119069+0000","last_change":"2024-04-25T09:54:56.130721+0000","last_active":"2024-04-25T09:54:57.119069+0000","last_peered":"2024-04-25T09:54:57.119069+0000","last_clean":"2024-04-25T09:54:57.119069+0000","last_became_active":"2024-04-25T09:54:56.130461+0000","last_became_peered":"2024-04-25T09:54:56.130461+0000","last_unstale":"2024-04-25T09:54:57.119069+0000","last_undegraded":"2024-04-25T09:54:57.119069+0000","last_fullsized":"2024-04-25T09:54:57.119069+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:39.618943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137323+0000","last_change":"2024-04-25T09:54:57.135635+0000","last_active":"2024-04-25T09:54:57.137323+0000","last_peered":"2024-04-25T09:54:57.137323+0000","last_clean":"2024-04-25T09:54:57.137323+0000","last_became_active":"2024-04-25T09:54:57.135342+0000","last_became_peered":"2024-04-25T09:54:57.135342+0000","last_unstale":"2024-04-25T09:54:57.137323+0000","last_undegraded":"2024-04-25T09:54:57.137323+0000","last_fullsized":"2024-04-25T09:54:57.137323+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:03.262970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.125976+0000","last_change":"2024-04-25T09:54:56.132520+0000","last_active":"2024-04-25T09:54:57.125976+0000","last_peered":"2024-04-25T09:54:57.125976+0000","last_clean":"2024-04-25T09:54:57.125976+0000","last_became_active":"2024-04-25T09:54:56.132178+0000","last_became_peered":"2024-04-25T09:54:56.132178+0000","last_unstale":"2024-04-25T09:54:57.125976+0000","last_undegraded":"2024-04-25T09:54:57.125976+0000","last_fullsized":"2024-04-25T09:54:57.125976+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:25.462953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133170+0000","last_change":"2024-04-25T09:54:57.128730+0000","last_active":"2024-04-25T09:54:57.133170+0000","last_peered":"2024-04-25T09:54:57.133170+0000","last_clean":"2024-04-25T09:54:57.133170+0000","last_became_active":"2024-04-25T09:54:57.128446+0000","last_became_peered":"2024-04-25T09:54:57.128446+0000","last_unstale":"2024-04-25T09:54:57.133170+0000","last_undegraded":"2024-04-25T09:54:57.133170+0000","last_fullsized":"2024-04-25T09:54:57.133170+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:48.083442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.118709+0000","last_change":"2024-04-25T09:54:56.135178+0000","last_active":"2024-04-25T09:54:57.118709+0000","last_peered":"2024-04-25T09:54:57.118709+0000","last_clean":"2024-04-25T09:54:57.118709+0000","last_became_active":"2024-04-25T09:54:56.134909+0000","last_became_peered":"2024-04-25T09:54:56.134909+0000","last_unstale":"2024-04-25T09:54:57.118709+0000","last_undegraded":"2024-04-25T09:54:57.118709+0000","last_fullsized":"2024-04-25T09:54:57.118709+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:32.072748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133944+0000","last_change":"2024-04-25T09:54:57.130040+0000","last_active":"2024-04-25T09:54:57.133944+0000","last_peered":"2024-04-25T09:54:57.133944+0000","last_clean":"2024-04-25T09:54:57.133944+0000","last_became_active":"2024-04-25T09:54:57.129264+0000","last_became_peered":"2024-04-25T09:54:57.129264+0000","last_unstale":"2024-04-25T09:54:57.133944+0000","last_undegraded":"2024-04-25T09:54:57.133944+0000","last_fullsized":"2024-04-25T09:54:57.133944+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:21.118874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.150977+0000","last_change":"2024-04-25T09:54:56.135230+0000","last_active":"2024-04-25T09:54:57.150977+0000","last_peered":"2024-04-25T09:54:57.150977+0000","last_clean":"2024-04-25T09:54:57.150977+0000","last_became_active":"2024-04-25T09:54:56.134942+0000","last_became_peered":"2024-04-25T09:54:56.134942+0000","last_unstale":"2024-04-25T09:54:57.150977+0000","last_undegraded":"2024-04-25T09:54:57.150977+0000","last_fullsized":"2024-04-25T09:54:57.150977+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:05.618687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.140739+0000","last_change":"2024-04-25T09:54:57.138897+0000","last_active":"2024-04-25T09:54:57.140739+0000","last_peered":"2024-04-25T09:54:57.140739+0000","last_clean":"2024-04-25T09:54:57.140739+0000","last_became_active":"2024-04-25T09:54:57.138622+0000","last_became_peered":"2024-04-25T09:54:57.138622+0000","last_unstale":"2024-04-25T09:54:57.140739+0000","last_undegraded":"2024-04-25T09:54:57.140739+0000","last_fullsized":"2024-04-25T09:54:57.140739+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:13.099355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153423+0000","last_change":"2024-04-25T09:54:56.129682+0000","last_active":"2024-04-25T09:54:57.153423+0000","last_peered":"2024-04-25T09:54:57.153423+0000","last_clean":"2024-04-25T09:54:57.153423+0000","last_became_active":"2024-04-25T09:54:56.129372+0000","last_became_peered":"2024-04-25T09:54:56.129372+0000","last_unstale":"2024-04-25T09:54:57.153423+0000","last_undegraded":"2024-04-25T09:54:57.153423+0000","last_fullsized":"2024-04-25T09:54:57.153423+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:39.878256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.138035+0000","last_change":"2024-04-25T09:54:57.135635+0000","last_active":"2024-04-25T09:54:57.138035+0000","last_peered":"2024-04-25T09:54:57.138035+0000","last_clean":"2024-04-25T09:54:57.138035+0000","last_became_active":"2024-04-25T09:54:57.134845+0000","last_became_peered":"2024-04-25T09:54:57.134845+0000","last_unstale":"2024-04-25T09:54:57.138035+0000","last_undegraded":"2024-04-25T09:54:57.138035+0000","last_fullsized":"2024-04-25T09:54:57.138035+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:40.882549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.146096+0000","last_change":"2024-04-25T09:54:56.133637+0000","last_active":"2024-04-25T09:54:57.146096+0000","last_peered":"2024-04-25T09:54:57.146096+0000","last_clean":"2024-04-25T09:54:57.146096+0000","last_became_active":"2024-04-25T09:54:56.133317+0000","last_became_peered":"2024-04-25T09:54:56.133317+0000","last_unstale":"2024-04-25T09:54:57.146096+0000","last_undegraded":"2024-04-25T09:54:57.146096+0000","last_fullsized":"2024-04-25T09:54:57.146096+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:56.136101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.138400+0000","last_change":"2024-04-25T09:54:57.133529+0000","last_active":"2024-04-25T09:54:57.138400+0000","last_peered":"2024-04-25T09:54:57.138400+0000","last_clean":"2024-04-25T09:54:57.138400+0000","last_became_active":"2024-04-25T09:54:57.133244+0000","last_became_peered":"2024-04-25T09:54:57.133244+0000","last_unstale":"2024-04-25T09:54:57.138400+0000","last_undegraded":"2024-04-25T09:54:57.138400+0000","last_fullsized":"2024-04-25T09:54:57.138400+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:03.700244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.124530+0000","last_change":"2024-04-25T09:54:56.135711+0000","last_active":"2024-04-25T09:54:57.124530+0000","last_peered":"2024-04-25T09:54:57.124530+0000","last_clean":"2024-04-25T09:54:57.124530+0000","last_became_active":"2024-04-25T09:54:56.135426+0000","last_became_peered":"2024-04-25T09:54:56.135426+0000","last_unstale":"2024-04-25T09:54:57.124530+0000","last_undegraded":"2024-04-25T09:54:57.124530+0000","last_fullsized":"2024-04-25T09:54:57.124530+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:56.602141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153408+0000","last_change":"2024-04-25T09:54:57.150520+0000","last_active":"2024-04-25T09:54:57.153408+0000","last_peered":"2024-04-25T09:54:57.153408+0000","last_clean":"2024-04-25T09:54:57.153408+0000","last_became_active":"2024-04-25T09:54:57.150259+0000","last_became_peered":"2024-04-25T09:54:57.150259+0000","last_unstale":"2024-04-25T09:54:57.153408+0000","last_undegraded":"2024-04-25T09:54:57.153408+0000","last_fullsized":"2024-04-25T09:54:57.153408+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:36.490077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.125114+0000","last_change":"2024-04-25T09:54:56.132949+0000","last_active":"2024-04-25T09:54:57.125114+0000","last_peered":"2024-04-25T09:54:57.125114+0000","last_clean":"2024-04-25T09:54:57.125114+0000","last_became_active":"2024-04-25T09:54:56.132215+0000","last_became_peered":"2024-04-25T09:54:56.132215+0000","last_unstale":"2024-04-25T09:54:57.125114+0000","last_undegraded":"2024-04-25T09:54:57.125114+0000","last_fullsized":"2024-04-25T09:54:57.125114+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:54.230115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.141550+0000","last_change":"2024-04-25T09:54:57.136471+0000","last_active":"2024-04-25T09:54:57.141550+0000","last_peered":"2024-04-25T09:54:57.141550+0000","last_clean":"2024-04-25T09:54:57.141550+0000","last_became_active":"2024-04-25T09:54:57.136223+0000","last_became_peered":"2024-04-25T09:54:57.136223+0000","last_unstale":"2024-04-25T09:54:57.141550+0000","last_undegraded":"2024-04-25T09:54:57.141550+0000","last_fullsized":"2024-04-25T09:54:57.141550+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:27.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145641+0000","last_change":"2024-04-25T09:54:56.127601+0000","last_active":"2024-04-25T09:54:57.145641+0000","last_peered":"2024-04-25T09:54:57.145641+0000","last_clean":"2024-04-25T09:54:57.145641+0000","last_became_active":"2024-04-25T09:54:56.127337+0000","last_became_peered":"2024-04-25T09:54:56.127337+0000","last_unstale":"2024-04-25T09:54:57.145641+0000","last_undegraded":"2024-04-25T09:54:57.145641+0000","last_fullsized":"2024-04-25T09:54:57.145641+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:47.079149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.145697+0000","last_change":"2024-04-25T09:54:57.139553+0000","last_active":"2024-04-25T09:54:57.145697+0000","last_peered":"2024-04-25T09:54:57.145697+0000","last_clean":"2024-04-25T09:54:57.145697+0000","last_became_active":"2024-04-25T09:54:57.139302+0000","last_became_peered":"2024-04-25T09:54:57.139302+0000","last_unstale":"2024-04-25T09:54:57.145697+0000","last_undegraded":"2024-04-25T09:54:57.145697+0000","last_fullsized":"2024-04-25T09:54:57.145697+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:00.061134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.148146+0000","last_change":"2024-04-25T09:54:56.131623+0000","last_active":"2024-04-25T09:54:57.148146+0000","last_peered":"2024-04-25T09:54:57.148146+0000","last_clean":"2024-04-25T09:54:57.148146+0000","last_became_active":"2024-04-25T09:54:56.131360+0000","last_became_peered":"2024-04-25T09:54:56.131360+0000","last_unstale":"2024-04-25T09:54:57.148146+0000","last_undegraded":"2024-04-25T09:54:57.148146+0000","last_fullsized":"2024-04-25T09:54:57.148146+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:54.230115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.138885+0000","last_change":"2024-04-25T09:54:57.134955+0000","last_active":"2024-04-25T09:54:57.138885+0000","last_peered":"2024-04-25T09:54:57.138885+0000","last_clean":"2024-04-25T09:54:57.138885+0000","last_became_active":"2024-04-25T09:54:57.134700+0000","last_became_peered":"2024-04-25T09:54:57.134700+0000","last_unstale":"2024-04-25T09:54:57.138885+0000","last_undegraded":"2024-04-25T09:54:57.138885+0000","last_fullsized":"2024-04-25T09:54:57.138885+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:50.052465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.143265+0000","last_change":"2024-04-25T09:54:56.129987+0000","last_active":"2024-04-25T09:54:57.143265+0000","last_peered":"2024-04-25T09:54:57.143265+0000","last_clean":"2024-04-25T09:54:57.143265+0000","last_became_active":"2024-04-25T09:54:56.129678+0000","last_became_peered":"2024-04-25T09:54:56.129678+0000","last_unstale":"2024-04-25T09:54:57.143265+0000","last_undegraded":"2024-04-25T09:54:57.143265+0000","last_fullsized":"2024-04-25T09:54:57.143265+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:04.585943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137211+0000","last_change":"2024-04-25T09:54:57.131707+0000","last_active":"2024-04-25T09:54:57.137211+0000","last_peered":"2024-04-25T09:54:57.137211+0000","last_clean":"2024-04-25T09:54:57.137211+0000","last_became_active":"2024-04-25T09:54:57.131432+0000","last_became_peered":"2024-04-25T09:54:57.131432+0000","last_unstale":"2024-04-25T09:54:57.137211+0000","last_undegraded":"2024-04-25T09:54:57.137211+0000","last_fullsized":"2024-04-25T09:54:57.137211+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:18.173854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137576+0000","last_change":"2024-04-25T09:54:56.135033+0000","last_active":"2024-04-25T09:54:57.137576+0000","last_peered":"2024-04-25T09:54:57.137576+0000","last_clean":"2024-04-25T09:54:57.137576+0000","last_became_active":"2024-04-25T09:54:56.134722+0000","last_became_peered":"2024-04-25T09:54:56.134722+0000","last_unstale":"2024-04-25T09:54:57.137576+0000","last_undegraded":"2024-04-25T09:54:57.137576+0000","last_fullsized":"2024-04-25T09:54:57.137576+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:27.923401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.150367+0000","last_change":"2024-04-25T09:54:57.147320+0000","last_active":"2024-04-25T09:54:57.150367+0000","last_peered":"2024-04-25T09:54:57.150367+0000","last_clean":"2024-04-25T09:54:57.150367+0000","last_became_active":"2024-04-25T09:54:57.147078+0000","last_became_peered":"2024-04-25T09:54:57.147078+0000","last_unstale":"2024-04-25T09:54:57.150367+0000","last_undegraded":"2024-04-25T09:54:57.150367+0000","last_fullsized":"2024-04-25T09:54:57.150367+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:57.140394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.123784+0000","last_change":"2024-04-25T09:54:56.133612+0000","last_active":"2024-04-25T09:54:57.123784+0000","last_peered":"2024-04-25T09:54:57.123784+0000","last_clean":"2024-04-25T09:54:57.123784+0000","last_became_active":"2024-04-25T09:54:56.133198+0000","last_became_peered":"2024-04-25T09:54:56.133198+0000","last_unstale":"2024-04-25T09:54:57.123784+0000","last_undegraded":"2024-04-25T09:54:57.123784+0000","last_fullsized":"2024-04-25T09:54:57.123784+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:08.630451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.137847+0000","last_change":"2024-04-25T09:54:57.133165+0000","last_active":"2024-04-25T09:54:57.137847+0000","last_peered":"2024-04-25T09:54:57.137847+0000","last_clean":"2024-04-25T09:54:57.137847+0000","last_became_active":"2024-04-25T09:54:57.132827+0000","last_became_peered":"2024-04-25T09:54:57.132827+0000","last_unstale":"2024-04-25T09:54:57.137847+0000","last_undegraded":"2024-04-25T09:54:57.137847+0000","last_fullsized":"2024-04-25T09:54:57.137847+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:29.718809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.151298+0000","last_change":"2024-04-25T09:54:56.129114+0000","last_active":"2024-04-25T09:54:57.151298+0000","last_peered":"2024-04-25T09:54:57.151298+0000","last_clean":"2024-04-25T09:54:57.151298+0000","last_became_active":"2024-04-25T09:54:56.128803+0000","last_became_peered":"2024-04-25T09:54:56.128803+0000","last_unstale":"2024-04-25T09:54:57.151298+0000","last_undegraded":"2024-04-25T09:54:57.151298+0000","last_fullsized":"2024-04-25T09:54:57.151298+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:08.630451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.140431+0000","last_change":"2024-04-25T09:54:57.133448+0000","last_active":"2024-04-25T09:54:57.140431+0000","last_peered":"2024-04-25T09:54:57.140431+0000","last_clean":"2024-04-25T09:54:57.140431+0000","last_became_active":"2024-04-25T09:54:57.133076+0000","last_became_peered":"2024-04-25T09:54:57.133076+0000","last_unstale":"2024-04-25T09:54:57.140431+0000","last_undegraded":"2024-04-25T09:54:57.140431+0000","last_fullsized":"2024-04-25T09:54:57.140431+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:50.052465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.132810+0000","last_change":"2024-04-25T09:54:56.127989+0000","last_active":"2024-04-25T09:54:57.132810+0000","last_peered":"2024-04-25T09:54:57.132810+0000","last_clean":"2024-04-25T09:54:57.132810+0000","last_became_active":"2024-04-25T09:54:56.126968+0000","last_became_peered":"2024-04-25T09:54:56.126968+0000","last_unstale":"2024-04-25T09:54:57.132810+0000","last_undegraded":"2024-04-25T09:54:57.132810+0000","last_fullsized":"2024-04-25T09:54:57.132810+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:28.714516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.133892+0000","last_change":"2024-04-25T09:54:57.129752+0000","last_active":"2024-04-25T09:54:57.133892+0000","last_peered":"2024-04-25T09:54:57.133892+0000","last_clean":"2024-04-25T09:54:57.133892+0000","last_became_active":"2024-04-25T09:54:57.129349+0000","last_became_peered":"2024-04-25T09:54:57.129349+0000","last_unstale":"2024-04-25T09:54:57.133892+0000","last_undegraded":"2024-04-25T09:54:57.133892+0000","last_fullsized":"2024-04-25T09:54:57.133892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:02.465811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.119361+0000","last_change":"2024-04-25T09:54:56.129836+0000","last_active":"2024-04-25T09:54:57.119361+0000","last_peered":"2024-04-25T09:54:57.119361+0000","last_clean":"2024-04-25T09:54:57.119361+0000","last_became_active":"2024-04-25T09:54:56.129540+0000","last_became_peered":"2024-04-25T09:54:56.129540+0000","last_unstale":"2024-04-25T09:54:57.119361+0000","last_undegraded":"2024-04-25T09:54:57.119361+0000","last_fullsized":"2024-04-25T09:54:57.119361+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:24.627746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.150356+0000","last_change":"2024-04-25T09:54:57.148723+0000","last_active":"2024-04-25T09:54:57.150356+0000","last_peered":"2024-04-25T09:54:57.150356+0000","last_clean":"2024-04-25T09:54:57.150356+0000","last_became_active":"2024-04-25T09:54:57.148474+0000","last_became_peered":"2024-04-25T09:54:57.148474+0000","last_unstale":"2024-04-25T09:54:57.150356+0000","last_undegraded":"2024-04-25T09:54:57.150356+0000","last_fullsized":"2024-04-25T09:54:57.150356+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:19.646682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.123756+0000","last_change":"2024-04-25T09:54:56.132506+0000","last_active":"2024-04-25T09:54:57.123756+0000","last_peered":"2024-04-25T09:54:57.123756+0000","last_clean":"2024-04-25T09:54:57.123756+0000","last_became_active":"2024-04-25T09:54:56.132219+0000","last_became_peered":"2024-04-25T09:54:56.132219+0000","last_unstale":"2024-04-25T09:54:57.123756+0000","last_undegraded":"2024-04-25T09:54:57.123756+0000","last_fullsized":"2024-04-25T09:54:57.123756+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:01.923921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.139729+0000","last_change":"2024-04-25T09:54:57.137712+0000","last_active":"2024-04-25T09:54:57.139729+0000","last_peered":"2024-04-25T09:54:57.139729+0000","last_clean":"2024-04-25T09:54:57.139729+0000","last_became_active":"2024-04-25T09:54:57.135931+0000","last_became_peered":"2024-04-25T09:54:57.135931+0000","last_unstale":"2024-04-25T09:54:57.139729+0000","last_undegraded":"2024-04-25T09:54:57.139729+0000","last_fullsized":"2024-04-25T09:54:57.139729+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:27.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.140441+0000","last_change":"2024-04-25T09:54:56.135595+0000","last_active":"2024-04-25T09:54:57.140441+0000","last_peered":"2024-04-25T09:54:57.140441+0000","last_clean":"2024-04-25T09:54:57.140441+0000","last_became_active":"2024-04-25T09:54:56.135263+0000","last_became_peered":"2024-04-25T09:54:56.135263+0000","last_unstale":"2024-04-25T09:54:57.140441+0000","last_undegraded":"2024-04-25T09:54:57.140441+0000","last_fullsized":"2024-04-25T09:54:57.140441+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:20.570537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.150300+0000","last_change":"2024-04-25T09:54:57.145292+0000","last_active":"2024-04-25T09:54:57.150300+0000","last_peered":"2024-04-25T09:54:57.150300+0000","last_clean":"2024-04-25T09:54:57.150300+0000","last_became_active":"2024-04-25T09:54:57.144946+0000","last_became_peered":"2024-04-25T09:54:57.144946+0000","last_unstale":"2024-04-25T09:54:57.150300+0000","last_undegraded":"2024-04-25T09:54:57.150300+0000","last_fullsized":"2024-04-25T09:54:57.150300+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:25.632038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.126214+0000","last_change":"2024-04-25T09:54:56.139944+0000","last_active":"2024-04-25T09:54:57.126214+0000","last_peered":"2024-04-25T09:54:57.126214+0000","last_clean":"2024-04-25T09:54:57.126214+0000","last_became_active":"2024-04-25T09:54:56.139710+0000","last_became_peered":"2024-04-25T09:54:56.139710+0000","last_unstale":"2024-04-25T09:54:57.126214+0000","last_undegraded":"2024-04-25T09:54:57.126214+0000","last_fullsized":"2024-04-25T09:54:57.126214+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:32.903659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.146584+0000","last_change":"2024-04-25T09:54:57.137871+0000","last_active":"2024-04-25T09:54:57.146584+0000","last_peered":"2024-04-25T09:54:57.146584+0000","last_clean":"2024-04-25T09:54:57.146584+0000","last_became_active":"2024-04-25T09:54:57.137608+0000","last_became_peered":"2024-04-25T09:54:57.137608+0000","last_unstale":"2024-04-25T09:54:57.146584+0000","last_undegraded":"2024-04-25T09:54:57.146584+0000","last_fullsized":"2024-04-25T09:54:57.146584+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:27.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.120268+0000","last_change":"2024-04-25T09:54:56.132960+0000","last_active":"2024-04-25T09:54:57.120268+0000","last_peered":"2024-04-25T09:54:57.120268+0000","last_clean":"2024-04-25T09:54:57.120268+0000","last_became_active":"2024-04-25T09:54:56.132686+0000","last_became_peered":"2024-04-25T09:54:56.132686+0000","last_unstale":"2024-04-25T09:54:57.120268+0000","last_undegraded":"2024-04-25T09:54:57.120268+0000","last_fullsized":"2024-04-25T09:54:57.120268+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:18.642389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.152839+0000","last_change":"2024-04-25T09:54:57.149052+0000","last_active":"2024-04-25T09:54:57.152839+0000","last_peered":"2024-04-25T09:54:57.152839+0000","last_clean":"2024-04-25T09:54:57.152839+0000","last_became_active":"2024-04-25T09:54:57.148769+0000","last_became_peered":"2024-04-25T09:54:57.148769+0000","last_unstale":"2024-04-25T09:54:57.152839+0000","last_undegraded":"2024-04-25T09:54:57.152839+0000","last_fullsized":"2024-04-25T09:54:57.152839+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:28.119357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.132904+0000","last_change":"2024-04-25T09:54:56.127762+0000","last_active":"2024-04-25T09:54:57.132904+0000","last_peered":"2024-04-25T09:54:57.132904+0000","last_clean":"2024-04-25T09:54:57.132904+0000","last_became_active":"2024-04-25T09:54:56.127131+0000","last_became_peered":"2024-04-25T09:54:56.127131+0000","last_unstale":"2024-04-25T09:54:57.132904+0000","last_undegraded":"2024-04-25T09:54:57.132904+0000","last_fullsized":"2024-04-25T09:54:57.132904+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:20.114582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.138022+0000","last_change":"2024-04-25T09:54:57.134598+0000","last_active":"2024-04-25T09:54:57.138022+0000","last_peered":"2024-04-25T09:54:57.138022+0000","last_clean":"2024-04-25T09:54:57.138022+0000","last_became_active":"2024-04-25T09:54:57.134367+0000","last_became_peered":"2024-04-25T09:54:57.134367+0000","last_unstale":"2024-04-25T09:54:57.138022+0000","last_undegraded":"2024-04-25T09:54:57.138022+0000","last_fullsized":"2024-04-25T09:54:57.138022+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:21.118874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.132984+0000","last_change":"2024-04-25T09:54:56.127052+0000","last_active":"2024-04-25T09:54:57.132984+0000","last_peered":"2024-04-25T09:54:57.132984+0000","last_clean":"2024-04-25T09:54:57.132984+0000","last_became_active":"2024-04-25T09:54:56.126781+0000","last_became_peered":"2024-04-25T09:54:56.126781+0000","last_unstale":"2024-04-25T09:54:57.132984+0000","last_undegraded":"2024-04-25T09:54:57.132984+0000","last_fullsized":"2024-04-25T09:54:57.132984+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:02.695951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.146164+0000","last_change":"2024-04-25T09:54:57.141310+0000","last_active":"2024-04-25T09:54:57.146164+0000","last_peered":"2024-04-25T09:54:57.146164+0000","last_clean":"2024-04-25T09:54:57.146164+0000","last_became_active":"2024-04-25T09:54:57.141047+0000","last_became_peered":"2024-04-25T09:54:57.141047+0000","last_unstale":"2024-04-25T09:54:57.146164+0000","last_undegraded":"2024-04-25T09:54:57.146164+0000","last_fullsized":"2024-04-25T09:54:57.146164+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:16.656767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.118754+0000","last_change":"2024-04-25T09:54:56.127914+0000","last_active":"2024-04-25T09:54:57.118754+0000","last_peered":"2024-04-25T09:54:57.118754+0000","last_clean":"2024-04-25T09:54:57.118754+0000","last_became_active":"2024-04-25T09:54:56.127633+0000","last_became_peered":"2024-04-25T09:54:56.127633+0000","last_unstale":"2024-04-25T09:54:57.118754+0000","last_undegraded":"2024-04-25T09:54:57.118754+0000","last_fullsized":"2024-04-25T09:54:57.118754+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:08.630451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.153235+0000","last_change":"2024-04-25T09:54:57.145867+0000","last_active":"2024-04-25T09:54:57.153235+0000","last_peered":"2024-04-25T09:54:57.153235+0000","last_clean":"2024-04-25T09:54:57.153235+0000","last_became_active":"2024-04-25T09:54:57.145360+0000","last_became_peered":"2024-04-25T09:54:57.145360+0000","last_unstale":"2024-04-25T09:54:57.153235+0000","last_undegraded":"2024-04-25T09:54:57.153235+0000","last_fullsized":"2024-04-25T09:54:57.153235+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.248939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.122548+0000","last_change":"2024-04-25T09:54:56.133958+0000","last_active":"2024-04-25T09:54:57.122548+0000","last_peered":"2024-04-25T09:54:57.122548+0000","last_clean":"2024-04-25T09:54:57.122548+0000","last_became_active":"2024-04-25T09:54:56.133695+0000","last_became_peered":"2024-04-25T09:54:56.133695+0000","last_unstale":"2024-04-25T09:54:57.122548+0000","last_undegraded":"2024-04-25T09:54:57.122548+0000","last_fullsized":"2024-04-25T09:54:57.122548+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:01.461519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.143478+0000","last_change":"2024-04-25T09:54:57.139528+0000","last_active":"2024-04-25T09:54:57.143478+0000","last_peered":"2024-04-25T09:54:57.143478+0000","last_clean":"2024-04-25T09:54:57.143478+0000","last_became_active":"2024-04-25T09:54:57.139238+0000","last_became_peered":"2024-04-25T09:54:57.139238+0000","last_unstale":"2024-04-25T09:54:57.143478+0000","last_undegraded":"2024-04-25T09:54:57.143478+0000","last_fullsized":"2024-04-25T09:54:57.143478+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:56.107649+0000","last_clean_scrub_stamp":"2024-04-25T09:54:56.107649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:40.623235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.120963+0000","last_change":"2024-04-25T09:54:56.133355+0000","last_active":"2024-04-25T09:54:57.120963+0000","last_peered":"2024-04-25T09:54:57.120963+0000","last_clean":"2024-04-25T09:54:57.120963+0000","last_became_active":"2024-04-25T09:54:56.133055+0000","last_became_peered":"2024-04-25T09:54:56.133055+0000","last_unstale":"2024-04-25T09:54:57.120963+0000","last_undegraded":"2024-04-25T09:54:57.120963+0000","last_fullsized":"2024-04-25T09:54:57.120963+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:55.103357+0000","last_clean_scrub_stamp":"2024-04-25T09:54:55.103357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:05.618687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T09:54:57.129154+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:54:57.129154+0000","last_peered":"2024-04-25T09:54:57.129154+0000","last_clean":"2024-04-25T09:54:57.129154+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:54:57.129154+0000","last_undegraded":"2024-04-25T09:54:57.129154+0000","last_fullsized":"2024-04-25T09:54:57.129154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1183744,"data_stored":847047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":257207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":581632,"data_stored":256555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":585728,"data_stored":257137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":598016,"data_stored":258189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":258167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":256715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1183744,"data_stored":846979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:55:00.418 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:55:00.418 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:55:00.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.555+0000 7fe0f7edd640 1 Processor -- start 2024-04-25T09:55:00.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.556+0000 7fe0f7edd640 1 -- start start 2024-04-25T09:55:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.556+0000 7fe0f7edd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe0f01617e0 0x7fe0f00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.556+0000 7fe0f7edd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe0f00cec80 0x7fe0f00d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.556+0000 7fe0f7edd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f0161dc0 con 0x7fe0f015dbf0 2024-04-25T09:55:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.556+0000 7fe0f7edd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f00cf0d0 con 0x7fe0f00cec80 2024-04-25T09:55:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.556+0000 7fe0f7edd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f00d1ee0 con 0x7fe0f01617e0 2024-04-25T09:55:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.556+0000 7fe0f647a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe0f00cec80 0x7fe0f00d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.556+0000 7fe0f647a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe0f00cec80 0x7fe0f00d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47206/0 (socket says 172.21.15.139:47206) 2024-04-25T09:55:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.556+0000 7fe0f647a640 1 -- 172.21.15.139:0/2980435339 learned_addr learned my addr 172.21.15.139:0/2980435339 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.557+0000 7fe0f5478640 1 --2- 172.21.15.139:0/2980435339 >> v2:172.21.15.150:3300/0 conn(0x7fe0f01617e0 0x7fe0f00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.557+0000 7fe0f647a640 1 -- 172.21.15.139:0/2980435339 >> v2:172.21.15.150:3300/0 conn(0x7fe0f01617e0 msgr2=0x7fe0f00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.557+0000 7fe0f647a640 1 --2- 172.21.15.139:0/2980435339 >> v2:172.21.15.150:3300/0 conn(0x7fe0f01617e0 0x7fe0f00ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.557+0000 7fe0f647a640 1 -- 172.21.15.139:0/2980435339 >> v1:172.21.15.139:6789/0 conn(0x7fe0f015dbf0 legacy=0x7fe0f00b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.557+0000 7fe0f4c77640 1 -- 172.21.15.139:0/2980435339 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797282367 0 0) 0x7fe0f0161dc0 con 0x7fe0f015dbf0 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.557+0000 7fe0f647a640 1 -- 172.21.15.139:0/2980435339 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f01620a0 con 0x7fe0f00cec80 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.557+0000 7fe0f5478640 1 --2- 172.21.15.139:0/2980435339 >> v2:172.21.15.150:3300/0 conn(0x7fe0f01617e0 0x7fe0f00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.557+0000 7fe0f647a640 1 --2- 172.21.15.139:0/2980435339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f00cec80 0x7fe0f00d2370 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe0f008c420 tx=0x7fe0e802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26510dfc8d96eb17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.558+0000 7fe0f4c77640 1 -- 172.21.15.139:0/2980435339 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8037070 con 0x7fe0f00cec80 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.558+0000 7fe0f4c77640 1 -- 172.21.15.139:0/2980435339 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e8032440 con 0x7fe0f00cec80 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.558+0000 7fe0f4c77640 1 -- 172.21.15.139:0/2980435339 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8032810 con 0x7fe0f00cec80 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.558+0000 7fe0f7edd640 1 -- 172.21.15.139:0/2980435339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f00cec80 msgr2=0x7fe0f00d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.558+0000 7fe0f7edd640 1 --2- 172.21.15.139:0/2980435339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f00cec80 0x7fe0f00d2370 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe0f008c420 tx=0x7fe0e802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.558+0000 7fe0f7edd640 1 -- 172.21.15.139:0/2980435339 shutdown_connections 2024-04-25T09:55:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.558+0000 7fe0f7edd640 1 --2- 172.21.15.139:0/2980435339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f00cec80 0x7fe0f00d2370 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.558+0000 7fe0f7edd640 1 --2- 172.21.15.139:0/2980435339 >> v2:172.21.15.150:3300/0 conn(0x7fe0f01617e0 0x7fe0f00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.558+0000 7fe0f7edd640 1 -- 172.21.15.139:0/2980435339 >> 172.21.15.139:0/2980435339 conn(0x7fe0f00b5480 msgr2=0x7fe0f00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:00.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.558+0000 7fe0f7edd640 1 -- 172.21.15.139:0/2980435339 shutdown_connections 2024-04-25T09:55:00.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.558+0000 7fe0f7edd640 1 -- 172.21.15.139:0/2980435339 wait complete. 2024-04-25T09:55:00.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.559+0000 7fe0f7edd640 1 Processor -- start 2024-04-25T09:55:00.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.559+0000 7fe0f7edd640 1 -- start start 2024-04-25T09:55:00.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.560+0000 7fe0f7edd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0f00cec80 0x7fe0f00cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.560+0000 7fe0f7edd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f015dbf0 0x7fe0f0150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.560+0000 7fe0f7edd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0f01617e0 0x7fe0f015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.560+0000 7fe0f7edd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f015dfd0 con 0x7fe0f015dbf0 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.560+0000 7fe0f7edd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f0162660 con 0x7fe0f01617e0 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.560+0000 7fe0f7edd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f00cd6a0 con 0x7fe0f00cec80 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.560+0000 7fe0f5478640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f015dbf0 0x7fe0f0150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.560+0000 7fe0f5478640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f015dbf0 0x7fe0f0150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47222/0 (socket says 172.21.15.139:47222) 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.560+0000 7fe0f5478640 1 -- 172.21.15.139:0/1109976169 learned_addr learned my addr 172.21.15.139:0/1109976169 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.560+0000 7fe0f647a640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0f01617e0 0x7fe0f015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.560+0000 7fe0f5c79640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0f00cec80 0x7fe0f00cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0f5478640 1 -- 172.21.15.139:0/1109976169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0f00cec80 msgr2=0x7fe0f00cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0f5478640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0f00cec80 0x7fe0f00cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0f5478640 1 -- 172.21.15.139:0/1109976169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0f01617e0 msgr2=0x7fe0f015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0f5478640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0f01617e0 0x7fe0f015ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0f5478640 1 -- 172.21.15.139:0/1109976169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f015e480 con 0x7fe0f015dbf0 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0f647a640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0f01617e0 0x7fe0f015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0f5c79640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0f00cec80 0x7fe0f00cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0f5478640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f015dbf0 0x7fe0f0150b90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000c7f0 tx=0x7fe0e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0e67fc640 1 -- 172.21.15.139:0/1109976169 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0015070 con 0x7fe0f015dbf0 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0e67fc640 1 -- 172.21.15.139:0/1109976169 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e0007820 con 0x7fe0f015dbf0 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0f7edd640 1 -- 172.21.15.139:0/1109976169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f00cd0e0 con 0x7fe0f015dbf0 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0e67fc640 1 -- 172.21.15.139:0/1109976169 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0007bd0 con 0x7fe0f015dbf0 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.561+0000 7fe0f7edd640 1 -- 172.21.15.139:0/1109976169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f015e720 con 0x7fe0f015dbf0 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.562+0000 7fe0f7edd640 1 -- 172.21.15.139:0/1109976169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f0074520 con 0x7fe0f015dbf0 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.562+0000 7fe0e67fc640 1 -- 172.21.15.139:0/1109976169 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0e0018070 con 0x7fe0f015dbf0 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.563+0000 7fe0e67fc640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe0d406b4d0 0x7fe0d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.563+0000 7fe0f5c79640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe0d406b4d0 0x7fe0d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.563+0000 7fe0e67fc640 1 -- 172.21.15.139:0/1109976169 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0e008a550 con 0x7fe0f015dbf0 2024-04-25T09:55:00.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.563+0000 7fe0f5c79640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe0d406b4d0 0x7fe0d406d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0012e0 tx=0x7fe0ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:00.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.567+0000 7fe0e67fc640 1 -- 172.21.15.139:0/1109976169 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0e00591b0 con 0x7fe0f015dbf0 2024-04-25T09:55:00.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.803+0000 7fe0f7edd640 1 -- 172.21.15.139:0/1109976169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe0f015f1c0 con 0x7fe0f015dbf0 2024-04-25T09:55:00.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.805+0000 7fe0e67fc640 1 -- 172.21.15.139:0/1109976169 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe0e0058ab0 con 0x7fe0f015dbf0 2024-04-25T09:55:00.809 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2024-04-25T09:55:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg== 2024-04-25T09:55:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: caps mds = "allow" 2024-04-25T09:55:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: caps mon = "allow r" 2024-04-25T09:55:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:55:00.809 INFO:teuthology.orchestra.run.smithi139.stderr:updated caps for client.0 2024-04-25T09:55:00.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.808+0000 7fe0f7edd640 1 -- 172.21.15.139:0/1109976169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe0d406b4d0 msgr2=0x7fe0d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.808+0000 7fe0f7edd640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe0d406b4d0 0x7fe0d406d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0012e0 tx=0x7fe0ec009040 comp rx=0 tx=0).stop 2024-04-25T09:55:00.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.808+0000 7fe0f7edd640 1 -- 172.21.15.139:0/1109976169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f015dbf0 msgr2=0x7fe0f0150b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.808+0000 7fe0f7edd640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f015dbf0 0x7fe0f0150b90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000c7f0 tx=0x7fe0e000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.808+0000 7fe0f7edd640 1 -- 172.21.15.139:0/1109976169 shutdown_connections 2024-04-25T09:55:00.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.808+0000 7fe0f7edd640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0f015dbf0 0x7fe0f0150b90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.808+0000 7fe0f7edd640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe0d406b4d0 0x7fe0d406d990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.808+0000 7fe0f7edd640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0f01617e0 0x7fe0f015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.808+0000 7fe0f7edd640 1 --2- 172.21.15.139:0/1109976169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0f00cec80 0x7fe0f00cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.808+0000 7fe0f7edd640 1 -- 172.21.15.139:0/1109976169 >> 172.21.15.139:0/1109976169 conn(0x7fe0f00b5480 msgr2=0x7fe0f00d3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:00.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.808+0000 7fe0f7edd640 1 -- 172.21.15.139:0/1109976169 shutdown_connections 2024-04-25T09:55:00.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.808+0000 7fe0f7edd640 1 -- 172.21.15.139:0/1109976169 wait complete. 2024-04-25T09:55:00.822 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:00.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.915+0000 7f0000c38640 1 Processor -- start 2024-04-25T09:55:00.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.915+0000 7f0000c38640 1 -- start start 2024-04-25T09:55:00.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.916+0000 7f0000c38640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efffc15cde0 0x7efffc15d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.916+0000 7f0000c38640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efffc15d1c0 con 0x7efffc0cc2a0 2024-04-25T09:55:00.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.916+0000 7f0000c38640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efffc0cd460 con 0x7efffc15db50 2024-04-25T09:55:00.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.916+0000 7f0000c38640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0cc720 con 0x7efffc15cde0 2024-04-25T09:55:00.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.916+0000 7efff9d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7efffc0cc2a0 0x7efffc09a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:36016/0 (socket says 172.21.15.139:36016) 2024-04-25T09:55:00.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.916+0000 7efff9d9b640 1 -- 172.21.15.139:0/1212628275 learned_addr learned my addr 172.21.15.139:0/1212628275 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:00.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.916+0000 7efffa59c640 1 --2- 172.21.15.139:0/1212628275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efffc15cde0 0x7efffc15d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.917+0000 7efff959a640 1 -- 172.21.15.139:0/1212628275 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045487667 0 0) 0x7efffc15d1c0 con 0x7efffc0cc2a0 2024-04-25T09:55:00.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.917+0000 7efff959a640 1 -- 172.21.15.139:0/1212628275 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effe4003520 con 0x7efffc0cc2a0 2024-04-25T09:55:00.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.917+0000 7efff959a640 1 -- 172.21.15.139:0/1212628275 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485760883 0 0) 0x7efffc0cd460 con 0x7efffc15db50 2024-04-25T09:55:00.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.917+0000 7efffa59c640 1 -- 172.21.15.139:0/1212628275 >> v1:172.21.15.150:6789/0 conn(0x7efffc15db50 legacy=0x7efffc0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.917+0000 7efffa59c640 1 -- 172.21.15.139:0/1212628275 >> v1:172.21.15.139:6789/0 conn(0x7efffc0cc2a0 legacy=0x7efffc09a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.917+0000 7efffa59c640 1 -- 172.21.15.139:0/1212628275 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc099bf0 con 0x7efffc15cde0 2024-04-25T09:55:00.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.918+0000 7efffa59c640 1 --2- 172.21.15.139:0/1212628275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efffc15cde0 0x7efffc15d610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7effec009dc0 tx=0x7effec02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d32081a429bfce75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:00.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.918+0000 7efff959a640 1 -- 172.21.15.139:0/1212628275 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec007740 con 0x7efffc15cde0 2024-04-25T09:55:00.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.918+0000 7efff959a640 1 -- 172.21.15.139:0/1212628275 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec0078e0 con 0x7efffc15cde0 2024-04-25T09:55:00.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.918+0000 7efff959a640 1 -- 172.21.15.139:0/1212628275 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec007c90 con 0x7efffc15cde0 2024-04-25T09:55:00.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.918+0000 7f0000c38640 1 -- 172.21.15.139:0/1212628275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efffc15cde0 msgr2=0x7efffc15d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.918+0000 7f0000c38640 1 --2- 172.21.15.139:0/1212628275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efffc15cde0 0x7efffc15d610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7effec009dc0 tx=0x7effec02a6d0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.918+0000 7f0000c38640 1 -- 172.21.15.139:0/1212628275 shutdown_connections 2024-04-25T09:55:00.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.918+0000 7f0000c38640 1 --2- 172.21.15.139:0/1212628275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efffc15cde0 0x7efffc15d610 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.918+0000 7f0000c38640 1 -- 172.21.15.139:0/1212628275 >> 172.21.15.139:0/1212628275 conn(0x7efffc0d1790 msgr2=0x7efffc0d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:00.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.918+0000 7f0000c38640 1 -- 172.21.15.139:0/1212628275 shutdown_connections 2024-04-25T09:55:00.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.918+0000 7f0000c38640 1 -- 172.21.15.139:0/1212628275 wait complete. 2024-04-25T09:55:00.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.920+0000 7f0000c38640 1 Processor -- start 2024-04-25T09:55:00.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.920+0000 7f0000c38640 1 -- start start 2024-04-25T09:55:00.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.920+0000 7f0000c38640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efffc0cc2a0 0x7efffc0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.920+0000 7f0000c38640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efffc15cde0 0x7efffc0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.920+0000 7f0000c38640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efffc15db50 0x7efffc159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.920+0000 7f0000c38640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0cf4e0 con 0x7efffc15cde0 2024-04-25T09:55:00.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.920+0000 7f0000c38640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0cf360 con 0x7efffc0cc2a0 2024-04-25T09:55:00.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.920+0000 7f0000c38640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0b74c0 con 0x7efffc15db50 2024-04-25T09:55:00.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7efff9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efffc15cde0 0x7efffc0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7efff9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efffc15cde0 0x7efffc0c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47230/0 (socket says 172.21.15.139:47230) 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7efff9d9b640 1 -- 172.21.15.139:0/1360934387 learned_addr learned my addr 172.21.15.139:0/1360934387 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7efffa59c640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efffc0cc2a0 0x7efffc0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7efffad9d640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efffc15db50 0x7efffc159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7efff9d9b640 1 -- 172.21.15.139:0/1360934387 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efffc15db50 msgr2=0x7efffc159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7efff9d9b640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efffc15db50 0x7efffc159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7efff9d9b640 1 -- 172.21.15.139:0/1360934387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efffc0cc2a0 msgr2=0x7efffc0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7efff9d9b640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efffc0cc2a0 0x7efffc0b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7efff9d9b640 1 -- 172.21.15.139:0/1360934387 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc0b6420 con 0x7efffc15cde0 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7efff9d9b640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efffc15cde0 0x7efffc0c9df0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efff000c460 tx=0x7efff000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7effeaffd640 1 -- 172.21.15.139:0/1360934387 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0017070 con 0x7efffc15cde0 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7effeaffd640 1 -- 172.21.15.139:0/1360934387 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff0005ce0 con 0x7efffc15cde0 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7f0000c38640 1 -- 172.21.15.139:0/1360934387 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efffc0b6700 con 0x7efffc15cde0 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7effeaffd640 1 -- 172.21.15.139:0/1360934387 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0007290 con 0x7efffc15cde0 2024-04-25T09:55:00.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.921+0000 7f0000c38640 1 -- 172.21.15.139:0/1360934387 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efffc0b6fa0 con 0x7efffc15cde0 2024-04-25T09:55:00.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.922+0000 7f0000c38640 1 -- 172.21.15.139:0/1360934387 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efffc0b6af0 con 0x7efffc15cde0 2024-04-25T09:55:00.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.923+0000 7effeaffd640 1 -- 172.21.15.139:0/1360934387 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efff001a070 con 0x7efffc15cde0 2024-04-25T09:55:00.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.923+0000 7effeaffd640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7effdc06b4d0 0x7effdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:00.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.923+0000 7effeaffd640 1 -- 172.21.15.139:0/1360934387 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efff0089710 con 0x7efffc15cde0 2024-04-25T09:55:00.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.923+0000 7efffa59c640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7effdc06b4d0 0x7effdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:00.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.924+0000 7efffa59c640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7effdc06b4d0 0x7effdc06d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7effec034000 tx=0x7effec02f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:00.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:00.927+0000 7effeaffd640 1 -- 172.21.15.139:0/1360934387 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efff0058400 con 0x7efffc15cde0 2024-04-25T09:55:01.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.134+0000 7f0000c38640 1 -- 172.21.15.139:0/1360934387 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efffc074520 con 0x7efffc15cde0 2024-04-25T09:55:01.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.134+0000 7effeaffd640 1 -- 172.21.15.139:0/1360934387 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4947 (secure 0 0 0) 0x7efff0057d00 con 0x7efffc15cde0 2024-04-25T09:55:01.138 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:01.138 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1742557158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1742557158},{"type":"v1","addr":"172.21.15.150:6834","nonce":1742557158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3496803834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3496803834},{"type":"v1","addr":"172.21.15.150:6835","nonce":3496803834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2014537692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2014537692},{"type":"v1","addr":"172.21.15.139:6835","nonce":2014537692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:56.466203+0000","modified":"2024-04-25T09:54:58.466766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5850":{"gid":5850,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/338275708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":338275708},{"type":"v1","addr":"172.21.15.139:6837","nonce":338275708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:55:01.138 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 22 2024-04-25T09:55:01.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.136+0000 7f0000c38640 1 -- 172.21.15.139:0/1360934387 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7effdc06b4d0 msgr2=0x7effdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.136+0000 7f0000c38640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7effdc06b4d0 0x7effdc06d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7effec034000 tx=0x7effec02f000 comp rx=0 tx=0).stop 2024-04-25T09:55:01.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.136+0000 7f0000c38640 1 -- 172.21.15.139:0/1360934387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efffc15cde0 msgr2=0x7efffc0c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.136+0000 7f0000c38640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efffc15cde0 0x7efffc0c9df0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efff000c460 tx=0x7efff000c920 comp rx=0 tx=0).stop 2024-04-25T09:55:01.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.136+0000 7f0000c38640 1 -- 172.21.15.139:0/1360934387 shutdown_connections 2024-04-25T09:55:01.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.136+0000 7f0000c38640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efffc15cde0 0x7efffc0c9df0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.136+0000 7f0000c38640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7effdc06b4d0 0x7effdc06d990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.136+0000 7f0000c38640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efffc15db50 0x7efffc159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.136+0000 7f0000c38640 1 --2- 172.21.15.139:0/1360934387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efffc0cc2a0 0x7efffc0b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.136+0000 7f0000c38640 1 -- 172.21.15.139:0/1360934387 >> 172.21.15.139:0/1360934387 conn(0x7efffc0d1790 msgr2=0x7efffc0c5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:01.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.136+0000 7f0000c38640 1 -- 172.21.15.139:0/1360934387 shutdown_connections 2024-04-25T09:55:01.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.136+0000 7f0000c38640 1 -- 172.21.15.139:0/1360934387 wait complete. 2024-04-25T09:55:01.554 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:54:56.466203+0000', 'modified': '2024-04-25T09:54:58.466766+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5850}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5850': {'gid': 5850, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6837/338275708', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 338275708}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 338275708}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:55:01.554 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:55:01.554 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:55:01.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.649+0000 7fad08bc1640 1 Processor -- start 2024-04-25T09:55:01.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.650+0000 7fad08bc1640 1 -- start start 2024-04-25T09:55:01.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.650+0000 7fad08bc1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad04144090 0x7fad040b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.650+0000 7fad08bc1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fad0406ea40 0x7fad0415e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.650+0000 7fad08bc1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad040b2a50 con 0x7fad040cc9e0 2024-04-25T09:55:01.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.650+0000 7fad08bc1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0415b090 con 0x7fad0406ea40 2024-04-25T09:55:01.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.650+0000 7fad08bc1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0415b370 con 0x7fad04144090 2024-04-25T09:55:01.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7facfbfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fad040cc9e0 0x7fad0406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:36022/0 (socket says 172.21.15.139:36022) 2024-04-25T09:55:01.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7facfbfff640 1 -- 172.21.15.139:0/1807399733 learned_addr learned my addr 172.21.15.139:0/1807399733 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:01.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad02d9d640 1 --2- 172.21.15.139:0/1807399733 >> v2:172.21.15.150:3300/0 conn(0x7fad0406ea40 0x7fad0415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad0259c640 1 --2- 172.21.15.139:0/1807399733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad04144090 0x7fad040b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581566554 0 0) 0x7facec001110 con 0x7fad040cc9e0 2024-04-25T09:55:01.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf0003520 con 0x7fad040cc9e0 2024-04-25T09:55:01.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 77269706 0 0) 0x7facf0003520 con 0x7fad040cc9e0 2024-04-25T09:55:01.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad04144090 msgr2=0x7fad040b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 --2- 172.21.15.139:0/1807399733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad04144090 0x7fad040b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 >> v2:172.21.15.150:3300/0 conn(0x7fad0406ea40 msgr2=0x7fad0415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 --2- 172.21.15.139:0/1807399733 >> v2:172.21.15.150:3300/0 conn(0x7fad0406ea40 0x7fad0415e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0415b650 con 0x7fad040cc9e0 2024-04-25T09:55:01.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7facec001840 con 0x7fad040cc9e0 2024-04-25T09:55:01.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 >> v1:172.21.15.139:6789/0 conn(0x7fad040cc9e0 legacy=0x7fad0406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 --2- 172.21.15.139:0/1807399733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad04144090 0x7facf00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 --2- 172.21.15.139:0/1807399733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0003f30 0x7facf0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 --2- 172.21.15.139:0/1807399733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf0004d40 0x7facf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0415b090 con 0x7facf0004d40 2024-04-25T09:55:01.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0415b370 con 0x7facf0003f30 2024-04-25T09:55:01.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf000ba10 con 0x7fad04144090 2024-04-25T09:55:01.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad02d9d640 1 --2- 172.21.15.139:0/1807399733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf0004d40 0x7facf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7fad0259c640 1 --2- 172.21.15.139:0/1807399733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad04144090 0x7facf00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.651+0000 7facfbfff640 1 --2- 172.21.15.139:0/1807399733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0003f30 0x7facf0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.652+0000 7fad02d9d640 1 -- 172.21.15.139:0/1807399733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad04144090 msgr2=0x7facf00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.652+0000 7fad02d9d640 1 --2- 172.21.15.139:0/1807399733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad04144090 0x7facf00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.652+0000 7fad02d9d640 1 -- 172.21.15.139:0/1807399733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0003f30 msgr2=0x7facf0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.652+0000 7fad02d9d640 1 --2- 172.21.15.139:0/1807399733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0003f30 0x7facf0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.652+0000 7fad02d9d640 1 -- 172.21.15.139:0/1807399733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf0003520 con 0x7facf0004d40 2024-04-25T09:55:01.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.652+0000 7fad02d9d640 1 --2- 172.21.15.139:0/1807399733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf0004d40 0x7facf00051c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7facfc010320 tx=0x7facf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d7db82bdbcf504 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.652+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc034070 con 0x7facf0004d40 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.652+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc00b2d0 con 0x7facf0004d40 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.652+0000 7fad01d9b640 1 -- 172.21.15.139:0/1807399733 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc00be20 con 0x7facf0004d40 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.653+0000 7fad08bc1640 1 -- 172.21.15.139:0/1807399733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf0004d40 msgr2=0x7facf00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.653+0000 7fad08bc1640 1 --2- 172.21.15.139:0/1807399733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf0004d40 0x7facf00051c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7facfc010320 tx=0x7facf0005830 comp rx=0 tx=0).stop 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.653+0000 7fad02d9d640 1 -- 172.21.15.139:0/1807399733 reap_dead start 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.653+0000 7fad08bc1640 1 -- 172.21.15.139:0/1807399733 shutdown_connections 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.653+0000 7fad08bc1640 1 -- 172.21.15.139:0/1807399733 >> 172.21.15.139:0/1807399733 conn(0x7fad04067000 msgr2=0x7fad0406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.653+0000 7fad08bc1640 1 -- 172.21.15.139:0/1807399733 shutdown_connections 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.653+0000 7fad08bc1640 1 -- 172.21.15.139:0/1807399733 wait complete. 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.654+0000 7fad08bc1640 1 Processor -- start 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.654+0000 7fad08bc1640 1 -- start start 2024-04-25T09:55:01.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad08bc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041591b0 0x7fad0415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad08bc1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad04159590 0x7fad0415c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad08bc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad04138e10 0x7fad041392c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad08bc1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad04144470 con 0x7fad041591b0 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad08bc1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0415bc10 con 0x7fad04138e10 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad08bc1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0415ad80 con 0x7fad04159590 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad0259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041591b0 0x7fad0415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad0259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041591b0 0x7fad0415d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47252/0 (socket says 172.21.15.139:47252) 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad0259c640 1 -- 172.21.15.139:0/2790536403 learned_addr learned my addr 172.21.15.139:0/2790536403 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad0259c640 1 -- 172.21.15.139:0/2790536403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad04159590 msgr2=0x7fad0415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad0259c640 1 --2- 172.21.15.139:0/2790536403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad04159590 0x7fad0415c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad0259c640 1 -- 172.21.15.139:0/2790536403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad04138e10 msgr2=0x7fad041392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad0259c640 1 --2- 172.21.15.139:0/2790536403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad04138e10 0x7fad041392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad0259c640 1 -- 172.21.15.139:0/2790536403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0415cdc0 con 0x7fad041591b0 2024-04-25T09:55:01.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.655+0000 7fad0259c640 1 --2- 172.21.15.139:0/2790536403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041591b0 0x7fad0415d1f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7facf40119a0 tx=0x7facf4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.656+0000 7facfaffd640 1 -- 172.21.15.139:0/2790536403 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf400fa70 con 0x7fad041591b0 2024-04-25T09:55:01.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.656+0000 7facfaffd640 1 -- 172.21.15.139:0/2790536403 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf400fc10 con 0x7fad041591b0 2024-04-25T09:55:01.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.656+0000 7fad08bc1640 1 -- 172.21.15.139:0/2790536403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0415d7d0 con 0x7fad041591b0 2024-04-25T09:55:01.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.656+0000 7facfaffd640 1 -- 172.21.15.139:0/2790536403 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4014440 con 0x7fad041591b0 2024-04-25T09:55:01.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.656+0000 7fad08bc1640 1 -- 172.21.15.139:0/2790536403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0413ea00 con 0x7fad041591b0 2024-04-25T09:55:01.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.657+0000 7facfaffd640 1 -- 172.21.15.139:0/2790536403 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7facf4012040 con 0x7fad041591b0 2024-04-25T09:55:01.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.657+0000 7facfaffd640 1 --2- 172.21.15.139:0/2790536403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7face406b4d0 0x7face406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.658+0000 7facfaffd640 1 -- 172.21.15.139:0/2790536403 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7facf408a9b0 con 0x7fad041591b0 2024-04-25T09:55:01.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.658+0000 7facfbfff640 1 --2- 172.21.15.139:0/2790536403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7face406b4d0 0x7face406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.658+0000 7facfbfff640 1 --2- 172.21.15.139:0/2790536403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7face406b4d0 0x7face406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7facec0012b0 tx=0x7facec000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.658+0000 7fad0b99f740 1 Processor -- start 2024-04-25T09:55:01.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.659+0000 7fad0b99f740 1 -- start start 2024-04-25T09:55:01.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.659+0000 7fad0b99f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 0x55fb5303c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.659+0000 7fad0b99f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb53059ad0 0x55fb5305a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.659+0000 7fad0b99f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb53011f10 0x55fb53058190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.659+0000 7fad0b99f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb52f74a10 con 0x55fb5303c230 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.659+0000 7fad0b99f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb52f1db30 con 0x55fb53059ad0 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.659+0000 7fad0b99f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb52f70c50 con 0x55fb53011f10 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.659+0000 7fad0259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 0x55fb5303c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.659+0000 7fad0259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 0x55fb5303c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47258/0 (socket says 172.21.15.139:47258) 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.659+0000 7fad0259c640 1 -- 172.21.15.139:0/1865662723 learned_addr learned my addr 172.21.15.139:0/1865662723 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.660+0000 7fad0259c640 1 -- 172.21.15.139:0/1865662723 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb53011f10 msgr2=0x55fb53058190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.660+0000 7fad0259c640 1 --2- 172.21.15.139:0/1865662723 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb53011f10 0x55fb53058190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.660+0000 7facfbfff640 1 --2- 172.21.15.139:0/1865662723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb53059ad0 0x55fb5305a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.660+0000 7fad0259c640 1 -- 172.21.15.139:0/1865662723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb53059ad0 msgr2=0x55fb5305a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.660+0000 7fad0259c640 1 --2- 172.21.15.139:0/1865662723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb53059ad0 0x55fb5305a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.660+0000 7fad0259c640 1 -- 172.21.15.139:0/1865662723 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb52f32e10 con 0x55fb5303c230 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.660+0000 7fad0259c640 1 --2- 172.21.15.139:0/1865662723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 0x55fb5303c610 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7facf4061210 tx=0x7facf40591a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8323d2c13d5486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.660+0000 7face3fff640 1 -- 172.21.15.139:0/1865662723 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf408a050 con 0x55fb5303c230 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.660+0000 7face3fff640 1 -- 172.21.15.139:0/1865662723 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf40573e0 con 0x55fb5303c230 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.660+0000 7face3fff640 1 -- 172.21.15.139:0/1865662723 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf40612b0 con 0x55fb5303c230 2024-04-25T09:55:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.661+0000 7fad0b99f740 1 -- 172.21.15.139:0/1865662723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 msgr2=0x55fb5303c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.661+0000 7fad0b99f740 1 --2- 172.21.15.139:0/1865662723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 0x55fb5303c610 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7facf4061210 tx=0x7facf40591a0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.661+0000 7fad0b99f740 1 -- 172.21.15.139:0/1865662723 shutdown_connections 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.661+0000 7fad0b99f740 1 --2- 172.21.15.139:0/1865662723 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb53011f10 0x55fb53058190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.661+0000 7fad0b99f740 1 --2- 172.21.15.139:0/1865662723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb53059ad0 0x55fb5305a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.661+0000 7fad0b99f740 1 --2- 172.21.15.139:0/1865662723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 0x55fb5303c610 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.661+0000 7fad0b99f740 1 -- 172.21.15.139:0/1865662723 >> 172.21.15.139:0/1865662723 conn(0x55fb5300e840 msgr2=0x55fb53057040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.661+0000 7fad0b99f740 1 -- 172.21.15.139:0/1865662723 shutdown_connections 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.661+0000 7fad0b99f740 1 -- 172.21.15.139:0/1865662723 wait complete. 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.661+0000 7fad0b99f740 1 -- start start 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.661+0000 7fad0b99f740 1 Processor -- start 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.662+0000 7fad0b99f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb5303c230 0x55fb53055fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.662+0000 7fad0b99f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb53059ad0 0x55fb5303f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.662+0000 7fad0b99f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb5303f760 0x55fb530675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.662+0000 7fad0b99f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb53012cb0 con 0x55fb53059ad0 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.662+0000 7fad0b99f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb52f310c0 con 0x55fb5303c230 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.662+0000 7fad0b99f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb5300c770 con 0x55fb5303f760 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.662+0000 7facfbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb53059ad0 0x55fb5303f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.662+0000 7fad0b99f740 10 client.0 _ll_register_callbacks cb 0x55fb52f24ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.662+0000 7facfbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb53059ad0 0x55fb5303f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47274/0 (socket says 172.21.15.139:47274) 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.662+0000 7facfbfff640 1 -- 172.21.15.139:0/3501344906 learned_addr learned my addr 172.21.15.139:0/3501344906 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.662+0000 7fad0259c640 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb5303c230 0x55fb53055fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.662+0000 7fad02d9d640 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb5303f760 0x55fb530675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7facfbfff640 1 -- 172.21.15.139:0/3501344906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb5303f760 msgr2=0x55fb530675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7facfbfff640 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb5303f760 0x55fb530675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7facfbfff640 1 -- 172.21.15.139:0/3501344906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb5303c230 msgr2=0x55fb53055fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7facfbfff640 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb5303c230 0x55fb53055fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7facfbfff640 1 -- 172.21.15.139:0/3501344906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb52c892d0 con 0x55fb53059ad0 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7facfbfff640 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb53059ad0 0x55fb5303f220 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7facec0136a0 tx=0x7facec007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ced5767908217c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7face17fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7face17fa640 1 -- 172.21.15.139:0/3501344906 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec026060 con 0x55fb53059ad0 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7face17fa640 1 -- 172.21.15.139:0/3501344906 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7facec021c30 con 0x55fb53059ad0 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7fad0b99f740 1 -- 172.21.15.139:0/3501344906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facf40573e0 con 0x55fb53059ad0 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7face17fa640 1 -- 172.21.15.139:0/3501344906 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7facec019070 con 0x55fb53059ad0 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7fad0b99f740 10 client.6000 fetch_fsmap learned FSMap version 22 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7fad0b99f740 1 -- 172.21.15.139:0/3501344906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55fb52d536b0 con 0x55fb53059ad0 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7face17fa640 1 -- 172.21.15.139:0/3501344906 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf40573e0 con 0x55fb53059ad0 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7face17fa640 1 -- 172.21.15.139:0/3501344906 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7facec01f070 con 0x55fb53059ad0 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7fad0b99f740 10 client.6000 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7fad0b99f740 10 client.6000 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7fad0b99f740 20 client.6000 populate_metadata read hostname 'smithi139' 2024-04-25T09:55:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7fad0b99f740 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55fb5304e7e0 0x55fb5304ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7fad0b99f740 4 client.6000 mds_command: new command op to 5850 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7fad0b99f740 1 -- 172.21.15.139:0/3501344906 --> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb52fd3920 con 0x55fb5304e7e0 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.663+0000 7fad0259c640 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55fb5304e7e0 0x55fb5304ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.664+0000 7face17fa640 10 client.6000 ms_handle_connect on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.664+0000 7fad0259c640 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55fb5304e7e0 0x55fb5304ebc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe0461ac2b43580f server_cookie=9e48b28974626626 in_seq=0 out_seq=0 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.664+0000 7face17fa640 1 -- 172.21.15.139:0/3501344906 <== mds.0 v2:172.21.15.139:6836/338275708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb52fd3920 con 0x55fb5304e7e0 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.664+0000 7face17fa640 10 client.6000 handle_command_reply: tid=0 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.664+0000 7fad0b99f740 1 client.6000 shutdown 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.664+0000 7fad0b99f740 1 -- 172.21.15.139:0/3501344906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb53059ad0 msgr2=0x55fb5303f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.664+0000 7fad0b99f740 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb53059ad0 0x55fb5303f220 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7facec0136a0 tx=0x7facec007ae0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.665+0000 7fad0b99f740 1 -- 172.21.15.139:0/3501344906 shutdown_connections 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.665+0000 7fad0b99f740 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55fb5304e7e0 0x55fb5304ebc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.665+0000 7fad0b99f740 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb53059ad0 0x55fb5303f220 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.665+0000 7fad0b99f740 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb5303f760 0x55fb530675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.665+0000 7fad0b99f740 1 --2- 172.21.15.139:0/3501344906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb5303c230 0x55fb53055fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.665+0000 7face17fa640 0 client.6000 ms_handle_reset on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:01.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.665+0000 7fad0b99f740 1 -- 172.21.15.139:0/3501344906 >> 172.21.15.139:0/3501344906 conn(0x55fb53011f10 msgr2=0x55fb5305a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:01.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.665+0000 7fad0b99f740 1 -- 172.21.15.139:0/3501344906 shutdown_connections 2024-04-25T09:55:01.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.665+0000 7fad0b99f740 1 -- 172.21.15.139:0/3501344906 wait complete. 2024-04-25T09:55:01.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.665+0000 7fad0b99f740 20 client.6000 trim_cache size 0 max 16384 2024-04-25T09:55:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.680+0000 7fad0b99f740 1 Processor -- start 2024-04-25T09:55:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.680+0000 7fad0b99f740 1 -- start start 2024-04-25T09:55:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7fad0b99f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb5303c230 0x55fb53055560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7fad0b99f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303f760 0x55fb53039fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7fad0b99f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb53059ad0 0x55fb530124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7fad0b99f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad041fcee0 con 0x55fb5303f760 2024-04-25T09:55:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7fad0b99f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad041fc0c0 con 0x55fb53059ad0 2024-04-25T09:55:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7fad0b99f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad041400b0 con 0x55fb5303c230 2024-04-25T09:55:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7facfbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303f760 0x55fb53039fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7facfbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303f760 0x55fb53039fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47276/0 (socket says 172.21.15.139:47276) 2024-04-25T09:55:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7facfbfff640 1 -- 172.21.15.139:0/554083229 learned_addr learned my addr 172.21.15.139:0/554083229 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7facfbfff640 1 -- 172.21.15.139:0/554083229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb5303c230 msgr2=0x55fb53055560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7facfbfff640 1 --2- 172.21.15.139:0/554083229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb5303c230 0x55fb53055560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7facfbfff640 1 -- 172.21.15.139:0/554083229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb53059ad0 msgr2=0x55fb530124e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7facfbfff640 1 --2- 172.21.15.139:0/554083229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb53059ad0 0x55fb530124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.681+0000 7facfbfff640 1 -- 172.21.15.139:0/554083229 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb53008550 con 0x55fb5303f760 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7facfbfff640 1 --2- 172.21.15.139:0/554083229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303f760 0x55fb53039fd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7facec007f00 tx=0x7facec01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf157bd7ba9825a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7face37fe640 1 -- 172.21.15.139:0/554083229 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec012830 con 0x55fb5303f760 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7face37fe640 1 -- 172.21.15.139:0/554083229 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec012aa0 con 0x55fb5303f760 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7face37fe640 1 -- 172.21.15.139:0/554083229 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec012c70 con 0x55fb5303f760 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7fad0b99f740 1 -- 172.21.15.139:0/554083229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303f760 msgr2=0x55fb53039fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7fad0b99f740 1 --2- 172.21.15.139:0/554083229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303f760 0x55fb53039fd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7facec007f00 tx=0x7facec01f000 comp rx=0 tx=0).stop 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7fad0b99f740 1 -- 172.21.15.139:0/554083229 shutdown_connections 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7fad0b99f740 1 --2- 172.21.15.139:0/554083229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303f760 0x55fb53039fd0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7fad0b99f740 1 --2- 172.21.15.139:0/554083229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb53059ad0 0x55fb530124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7fad0b99f740 1 --2- 172.21.15.139:0/554083229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb5303c230 0x55fb53055560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7fad0b99f740 1 -- 172.21.15.139:0/554083229 >> 172.21.15.139:0/554083229 conn(0x55fb53011f10 msgr2=0x55fb53055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7fad0b99f740 1 -- 172.21.15.139:0/554083229 shutdown_connections 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.682+0000 7fad0b99f740 1 -- 172.21.15.139:0/554083229 wait complete. 2024-04-25T09:55:01.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.683+0000 7fad0b99f740 1 -- start start 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.683+0000 7fad0b99f740 1 Processor -- start 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0b99f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 0x55fb530b9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0b99f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb5303f760 0x55fb53011260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0b99f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb53059ad0 0x55fb53056780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0b99f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5304f4a0 con 0x55fb5303c230 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0b99f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad041fbac0 con 0x55fb5303f760 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0b99f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb5306b3f0 con 0x55fb53059ad0 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0b99f740 10 client.0 _ll_register_callbacks cb 0x55fb5300cc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 0x55fb530b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 0x55fb530b9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47286/0 (socket says 172.21.15.139:47286) 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0259c640 1 -- 172.21.15.139:0/1035456759 learned_addr learned my addr 172.21.15.139:0/1035456759 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad02d9d640 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb53059ad0 0x55fb53056780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7facfbfff640 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb5303f760 0x55fb53011260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0259c640 1 -- 172.21.15.139:0/1035456759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb53059ad0 msgr2=0x55fb53056780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0259c640 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb53059ad0 0x55fb53056780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0259c640 1 -- 172.21.15.139:0/1035456759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb5303f760 msgr2=0x55fb53011260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0259c640 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb5303f760 0x55fb53011260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0259c640 1 -- 172.21.15.139:0/1035456759 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb530c67c0 con 0x55fb5303c230 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7facfbfff640 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb5303f760 0x55fb53011260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0259c640 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 0x55fb530b9250 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7facf405d0f0 tx=0x7facf4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86876cb22fef19d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7face17fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7face17fa640 1 -- 172.21.15.139:0/1035456759 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf408a040 con 0x55fb5303c230 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7face17fa640 1 -- 172.21.15.139:0/1035456759 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7facf40824c0 con 0x55fb5303c230 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0b99f740 1 -- 172.21.15.139:0/1035456759 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facec012aa0 con 0x55fb5303c230 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7face17fa640 1 -- 172.21.15.139:0/1035456759 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7facf4057200 con 0x55fb5303c230 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0b99f740 10 client.6006 fetch_fsmap learned FSMap version 22 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.684+0000 7fad0b99f740 1 -- 172.21.15.139:0/1035456759 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55fb53055db0 con 0x55fb5303c230 2024-04-25T09:55:01.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7face17fa640 1 -- 172.21.15.139:0/1035456759 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec012aa0 con 0x55fb5303c230 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7face17fa640 1 -- 172.21.15.139:0/1035456759 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7facec0016c0 con 0x55fb5303c230 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7fad0b99f740 10 client.6006 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7fad0b99f740 10 client.6006 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7fad0b99f740 20 client.6006 populate_metadata read hostname 'smithi139' 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7fad0b99f740 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55fb5300bb60 0x55fb5300bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7fad0b99f740 4 client.6006 mds_command: new command op to 5850 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7fad0b99f740 1 -- 172.21.15.139:0/1035456759 --> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fb52f81f50 con 0x55fb5300bb60 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7facfbfff640 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55fb5300bb60 0x55fb5300bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7face17fa640 10 client.6006 ms_handle_connect on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7facfbfff640 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55fb5300bb60 0x55fb5300bf40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83141d1daeb8d506 server_cookie=bdfabd1739407f5b in_seq=0 out_seq=0 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7face17fa640 1 -- 172.21.15.139:0/1035456759 <== mds.0 v2:172.21.15.139:6836/338275708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7facec03a160 con 0x55fb5300bb60 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7face17fa640 10 client.6006 handle_command_reply: tid=0 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7fad0b99f740 1 client.6006 shutdown 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7fad0b99f740 1 -- 172.21.15.139:0/1035456759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 msgr2=0x55fb530b9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.685+0000 7fad0b99f740 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 0x55fb530b9250 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7facf405d0f0 tx=0x7facf4000bc0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad0b99f740 1 -- 172.21.15.139:0/1035456759 shutdown_connections 2024-04-25T09:55:01.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad0b99f740 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55fb5300bb60 0x55fb5300bf40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad0b99f740 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fb53059ad0 0x55fb53056780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad0b99f740 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb5303f760 0x55fb53011260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad0b99f740 1 --2- 172.21.15.139:0/1035456759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fb5303c230 0x55fb530b9250 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7face17fa640 0 client.6006 ms_handle_reset on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:01.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad0b99f740 1 -- 172.21.15.139:0/1035456759 >> 172.21.15.139:0/1035456759 conn(0x55fb53011f10 msgr2=0x55fb530579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:01.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad0b99f740 1 -- 172.21.15.139:0/1035456759 shutdown_connections 2024-04-25T09:55:01.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad0b99f740 1 -- 172.21.15.139:0/1035456759 wait complete. 2024-04-25T09:55:01.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad0b99f740 20 client.6006 trim_cache size 0 max 16384 2024-04-25T09:55:01.697 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:55:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T09:55:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T09:55:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 5850, 2024-04-25T09:55:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T09:55:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T09:55:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T09:55:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 4.5726690100000003, 2024-04-25T09:55:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 22, 2024-04-25T09:55:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 50, 2024-04-25T09:55:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 50, 2024-04-25T09:55:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 9.6433882549999996 2024-04-25T09:55:01.698 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:55:01.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad08bc1640 1 -- 172.21.15.139:0/2790536403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7face406b4d0 msgr2=0x7face406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad08bc1640 1 --2- 172.21.15.139:0/2790536403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7face406b4d0 0x7face406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7facec0012b0 tx=0x7facec000bc0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad08bc1640 1 -- 172.21.15.139:0/2790536403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041591b0 msgr2=0x7fad0415d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad08bc1640 1 --2- 172.21.15.139:0/2790536403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041591b0 0x7fad0415d1f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7facf40119a0 tx=0x7facf4011e60 comp rx=0 tx=0).stop 2024-04-25T09:55:01.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad08bc1640 1 -- 172.21.15.139:0/2790536403 shutdown_connections 2024-04-25T09:55:01.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad08bc1640 1 --2- 172.21.15.139:0/2790536403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7face406b4d0 0x7face406d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad08bc1640 1 --2- 172.21.15.139:0/2790536403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad04138e10 0x7fad041392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad08bc1640 1 --2- 172.21.15.139:0/2790536403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad04159590 0x7fad0415c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad08bc1640 1 --2- 172.21.15.139:0/2790536403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041591b0 0x7fad0415d1f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad08bc1640 1 -- 172.21.15.139:0/2790536403 >> 172.21.15.139:0/2790536403 conn(0x7fad04067000 msgr2=0x7fad040b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:01.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad08bc1640 1 -- 172.21.15.139:0/2790536403 shutdown_connections 2024-04-25T09:55:01.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.686+0000 7fad08bc1640 1 -- 172.21.15.139:0/2790536403 wait complete. 2024-04-25T09:55:01.701 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:55:01.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.794+0000 7f59ff57e640 1 Processor -- start 2024-04-25T09:55:01.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.794+0000 7f59ff57e640 1 -- start start 2024-04-25T09:55:01.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59ff57e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f59f8158260 0x7f59f815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59ff57e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f59f806da20 0x7f59f80b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59ff57e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59f80b3600 0x7f59f806def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59ff57e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f80b3e10 con 0x7f59f8158260 2024-04-25T09:55:01.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59ff57e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f81615c0 con 0x7f59f806da20 2024-04-25T09:55:01.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59ff57e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f81618a0 con 0x7f59f80b3600 2024-04-25T09:55:01.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59fd31a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f59f8158260 0x7f59f815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59fd31a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f59f8158260 0x7f59f815df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47300/0 (socket says 172.21.15.139:47300) 2024-04-25T09:55:01.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59fd31a640 1 -- 172.21.15.139:0/3570685313 learned_addr learned my addr 172.21.15.139:0/3570685313 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:01.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59fdb1b640 1 --2- 172.21.15.139:0/3570685313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59f80b3600 0x7f59f806def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59fcb19640 1 --2- 172.21.15.139:0/3570685313 >> v2:172.21.15.150:3300/0 conn(0x7f59f806da20 0x7f59f80b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59fd31a640 1 -- 172.21.15.139:0/3570685313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59f80b3600 msgr2=0x7f59f806def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59fd31a640 1 --2- 172.21.15.139:0/3570685313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59f80b3600 0x7f59f806def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59fd31a640 1 -- 172.21.15.139:0/3570685313 >> v2:172.21.15.150:3300/0 conn(0x7f59f806da20 msgr2=0x7f59f80b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59fd31a640 1 --2- 172.21.15.139:0/3570685313 >> v2:172.21.15.150:3300/0 conn(0x7f59f806da20 0x7f59f80b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.795+0000 7f59fd31a640 1 -- 172.21.15.139:0/3570685313 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f8161b80 con 0x7f59f8158260 2024-04-25T09:55:01.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.796+0000 7f59fd31a640 1 --2- 172.21.15.139:0/3570685313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f8158260 0x7f59f815df80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f59f8072d10 tx=0x7f59f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909d3ed9c4a018e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.796+0000 7f59effff640 1 -- 172.21.15.139:0/3570685313 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4037070 con 0x7f59f8158260 2024-04-25T09:55:01.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.796+0000 7f59effff640 1 -- 172.21.15.139:0/3570685313 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f4032440 con 0x7f59f8158260 2024-04-25T09:55:01.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.796+0000 7f59effff640 1 -- 172.21.15.139:0/3570685313 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4032810 con 0x7f59f8158260 2024-04-25T09:55:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.797+0000 7f59ff57e640 1 -- 172.21.15.139:0/3570685313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f8158260 msgr2=0x7f59f815df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.797+0000 7f59ff57e640 1 --2- 172.21.15.139:0/3570685313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f8158260 0x7f59f815df80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f59f8072d10 tx=0x7f59f402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.797+0000 7f59ff57e640 1 -- 172.21.15.139:0/3570685313 shutdown_connections 2024-04-25T09:55:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.797+0000 7f59ff57e640 1 --2- 172.21.15.139:0/3570685313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59f80b3600 0x7f59f806def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.797+0000 7f59ff57e640 1 --2- 172.21.15.139:0/3570685313 >> v2:172.21.15.150:3300/0 conn(0x7f59f806da20 0x7f59f80b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.797+0000 7f59ff57e640 1 --2- 172.21.15.139:0/3570685313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f8158260 0x7f59f815df80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.797+0000 7f59ff57e640 1 -- 172.21.15.139:0/3570685313 >> 172.21.15.139:0/3570685313 conn(0x7f59f80b23d0 msgr2=0x7f59f80cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.797+0000 7f59ff57e640 1 -- 172.21.15.139:0/3570685313 shutdown_connections 2024-04-25T09:55:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.797+0000 7f59ff57e640 1 -- 172.21.15.139:0/3570685313 wait complete. 2024-04-25T09:55:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.798+0000 7f59ff57e640 1 Processor -- start 2024-04-25T09:55:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.798+0000 7f59ff57e640 1 -- start start 2024-04-25T09:55:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59ff57e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59f806da20 0x7f59f81e7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59ff57e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f80b3600 0x7f59f81f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59ff57e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8158260 0x7f59f80b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59ff57e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f8162420 con 0x7f59f80b3600 2024-04-25T09:55:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59ff57e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f8162140 con 0x7f59f8158260 2024-04-25T09:55:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59ff57e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f806e430 con 0x7f59f806da20 2024-04-25T09:55:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59fcb19640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f80b3600 0x7f59f81f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59fcb19640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f80b3600 0x7f59f81f35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47304/0 (socket says 172.21.15.139:47304) 2024-04-25T09:55:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59fcb19640 1 -- 172.21.15.139:0/2802535933 learned_addr learned my addr 172.21.15.139:0/2802535933 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:01.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59fd31a640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59f806da20 0x7f59f81e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59fdb1b640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8158260 0x7f59f80b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59fcb19640 1 -- 172.21.15.139:0/2802535933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59f806da20 msgr2=0x7f59f81e7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59fcb19640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59f806da20 0x7f59f81e7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59fcb19640 1 -- 172.21.15.139:0/2802535933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8158260 msgr2=0x7f59f80b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59fcb19640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8158260 0x7f59f80b0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59fcb19640 1 -- 172.21.15.139:0/2802535933 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f81e7b90 con 0x7f59f80b3600 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59fcb19640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f80b3600 0x7f59f81f35a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f59e800d830 tx=0x7f59e800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.799+0000 7f59edffb640 1 -- 172.21.15.139:0/2802535933 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e800b6c0 con 0x7f59f80b3600 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.800+0000 7f59edffb640 1 -- 172.21.15.139:0/2802535933 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e8019070 con 0x7f59f80b3600 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.800+0000 7f59edffb640 1 -- 172.21.15.139:0/2802535933 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e800ba80 con 0x7f59f80b3600 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.800+0000 7f59ff57e640 1 -- 172.21.15.139:0/2802535933 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f81e7e70 con 0x7f59f80b3600 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.800+0000 7f59ff57e640 1 -- 172.21.15.139:0/2802535933 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f81e8430 con 0x7f59f80b3600 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.801+0000 7f59ff57e640 1 -- 172.21.15.139:0/2802535933 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f8074520 con 0x7f59f80b3600 2024-04-25T09:55:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.801+0000 7f59edffb640 1 -- 172.21.15.139:0/2802535933 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59e801e460 con 0x7f59f80b3600 2024-04-25T09:55:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.801+0000 7f59edffb640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f59dc06b4d0 0x7f59dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.801+0000 7f59edffb640 1 -- 172.21.15.139:0/2802535933 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f59e808aa90 con 0x7f59f80b3600 2024-04-25T09:55:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.801+0000 7f59fd31a640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f59dc06b4d0 0x7f59dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.802+0000 7f59fd31a640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f59dc06b4d0 0x7f59dc06d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f59f4034000 tx=0x7f59f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:01.806+0000 7f59edffb640 1 -- 172.21.15.139:0/2802535933 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59e8059780 con 0x7f59f80b3600 2024-04-25T09:55:02.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.011+0000 7f59ff57e640 1 -- 172.21.15.139:0/2802535933 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f59f8157dd0 con 0x7f59f80b3600 2024-04-25T09:55:02.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.011+0000 7f59edffb640 1 -- 172.21.15.139:0/2802535933 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7f59e8059080 con 0x7f59f80b3600 2024-04-25T09:55:02.015 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:55:02.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.013+0000 7f59ff57e640 1 -- 172.21.15.139:0/2802535933 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f59dc06b4d0 msgr2=0x7f59dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:02.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.013+0000 7f59ff57e640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f59dc06b4d0 0x7f59dc06d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f59f4034000 tx=0x7f59f402e000 comp rx=0 tx=0).stop 2024-04-25T09:55:02.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.013+0000 7f59ff57e640 1 -- 172.21.15.139:0/2802535933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f80b3600 msgr2=0x7f59f81f35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:02.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.013+0000 7f59ff57e640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f80b3600 0x7f59f81f35a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f59e800d830 tx=0x7f59e800dcf0 comp rx=0 tx=0).stop 2024-04-25T09:55:02.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.013+0000 7f59ff57e640 1 -- 172.21.15.139:0/2802535933 shutdown_connections 2024-04-25T09:55:02.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.013+0000 7f59ff57e640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f80b3600 0x7f59f81f35a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:02.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.013+0000 7f59ff57e640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f59dc06b4d0 0x7f59dc06d990 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:02.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.013+0000 7f59ff57e640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8158260 0x7f59f80b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:02.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.013+0000 7f59ff57e640 1 --2- 172.21.15.139:0/2802535933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59f806da20 0x7f59f81e7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:02.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.013+0000 7f59ff57e640 1 -- 172.21.15.139:0/2802535933 >> 172.21.15.139:0/2802535933 conn(0x7f59f80b23d0 msgr2=0x7f59f80aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:02.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.014+0000 7f59ff57e640 1 -- 172.21.15.139:0/2802535933 shutdown_connections 2024-04-25T09:55:02.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02.014+0000 7f59ff57e640 1 -- 172.21.15.139:0/2802535933 wait complete. 2024-04-25T09:55:02.027 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:55:02.028 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:55:02.028 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T09:55:02.028 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:55:02.028 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:02.028 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:55:02.028 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:55:02.028 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:55:02.028 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:55:02.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.028 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T09:55:02.044 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:55:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:55:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:55:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:55:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.044 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42596sec preferred_lft 42596sec 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:55:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:55:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T09:55:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.046 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:55:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:55:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T09:55:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.046 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:55:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:55:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T09:55:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.047 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:55:02.106 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:55:02.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.108 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T09:55:02.164 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T09:55:02.164 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:55:02.164 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:55:02.164 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:55:02.164 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:55:02.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.164 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:55:02.165 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:55:02.165 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:55:02.165 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:55:02.165 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:55:02.165 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:55:02.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.247 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:55:02.302 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:55:02.302 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:55:02.302 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:55:02.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.302 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T09:55:02.366 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:55:02.366 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:02.390 INFO:tasks.ceph.mds.c.smithi139.stderr:2024-04-25T09:55:02.387+0000 7efe2051c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:55:02.404 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:55:02.404 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:55:02.404 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:02.458 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:55:02.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.459 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:55:02.541 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:55:02.541 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:55:02.550 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:55:02.550 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:55:02.550 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T09:55:02.550 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T09:55:02.550 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T09:55:02.550 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:55:02.550 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:55:02.550 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:55:02.550 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:55:02.550 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:55:02.551 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:55:02.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.551 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:02.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.622 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:55:02.683 INFO:teuthology.orchestra.run.smithi139.stdout:client5231 2024-04-25T09:55:02.683 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:55:02.683 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:02.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.739 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-25T09:55:02.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.797 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-25T09:55:02.853 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T09:55:02.854 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T09:55:02.854 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T09:55:02.854 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:55:02.854 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-25T09:55:02.854 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T09:55:02.957 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627777 2024-04-25T09:55:02.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:55:02.967 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:03.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.101+0000 7fdd719f5640 1 Processor -- start 2024-04-25T09:55:03.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.101+0000 7fdd719f5640 1 -- start start 2024-04-25T09:55:03.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.102+0000 7fdd719f5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdd6c0d1080 0x7fdd6c0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.102+0000 7fdd719f5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6c0cf250 con 0x7fdd6c161850 2024-04-25T09:55:03.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.102+0000 7fdd719f5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6c0ce9b0 con 0x7fdd6c15dc60 2024-04-25T09:55:03.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.102+0000 7fdd719f5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c0d3730 con 0x7fdd6c0d1080 2024-04-25T09:55:03.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.102+0000 7fdd6a7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdd6c0d1080 0x7fdd6c0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.102+0000 7fdd6b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdd6c161850 0x7fdd6c0d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:36034/0 (socket says 172.21.15.139:36034) 2024-04-25T09:55:03.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.102+0000 7fdd6b7fe640 1 -- 172.21.15.139:0/996684952 learned_addr learned my addr 172.21.15.139:0/996684952 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:03.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.102+0000 7fdd6a7fc640 1 -- 172.21.15.139:0/996684952 >> v1:172.21.15.150:6789/0 conn(0x7fdd6c15dc60 legacy=0x7fdd6c0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.102+0000 7fdd69ffb640 1 -- 172.21.15.139:0/996684952 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664898037 0 0) 0x7fdd64001110 con 0x7fdd6c161850 2024-04-25T09:55:03.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.102+0000 7fdd6a7fc640 1 -- 172.21.15.139:0/996684952 >> v1:172.21.15.139:6789/0 conn(0x7fdd6c161850 legacy=0x7fdd6c0d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.102+0000 7fdd6a7fc640 1 -- 172.21.15.139:0/996684952 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0cec90 con 0x7fdd6c0d1080 2024-04-25T09:55:03.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.103+0000 7fdd6a7fc640 1 --2- 172.21.15.139:0/996684952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd6c0d1080 0x7fdd6c0b6c10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdd600097a0 tx=0x7fdd6002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305d558bbdc1d1bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.103+0000 7fdd69ffb640 1 -- 172.21.15.139:0/996684952 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60037070 con 0x7fdd6c0d1080 2024-04-25T09:55:03.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.103+0000 7fdd69ffb640 1 -- 172.21.15.139:0/996684952 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd60032440 con 0x7fdd6c0d1080 2024-04-25T09:55:03.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.103+0000 7fdd69ffb640 1 -- 172.21.15.139:0/996684952 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60032810 con 0x7fdd6c0d1080 2024-04-25T09:55:03.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.104+0000 7fdd719f5640 1 -- 172.21.15.139:0/996684952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd6c0d1080 msgr2=0x7fdd6c0b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.104+0000 7fdd719f5640 1 --2- 172.21.15.139:0/996684952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd6c0d1080 0x7fdd6c0b6c10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdd600097a0 tx=0x7fdd6002a230 comp rx=0 tx=0).stop 2024-04-25T09:55:03.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.104+0000 7fdd719f5640 1 -- 172.21.15.139:0/996684952 shutdown_connections 2024-04-25T09:55:03.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.104+0000 7fdd719f5640 1 --2- 172.21.15.139:0/996684952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd6c0d1080 0x7fdd6c0b6c10 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.104+0000 7fdd719f5640 1 -- 172.21.15.139:0/996684952 >> 172.21.15.139:0/996684952 conn(0x7fdd6c0b5910 msgr2=0x7fdd6c0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.104+0000 7fdd719f5640 1 -- 172.21.15.139:0/996684952 shutdown_connections 2024-04-25T09:55:03.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.104+0000 7fdd719f5640 1 -- 172.21.15.139:0/996684952 wait complete. 2024-04-25T09:55:03.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.105+0000 7fdd719f5640 1 Processor -- start 2024-04-25T09:55:03.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.105+0000 7fdd719f5640 1 -- start start 2024-04-25T09:55:03.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd719f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd6c0d1080 0x7fdd6c14daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd719f5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c15dc60 0x7fdd6c160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd719f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd6c161850 0x7fdd6c153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd719f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c15e040 con 0x7fdd6c0d1080 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd719f5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0d1fc0 con 0x7fdd6c15dc60 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd719f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c14e030 con 0x7fdd6c161850 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd6affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd6c0d1080 0x7fdd6c14daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd6affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd6c0d1080 0x7fdd6c14daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47334/0 (socket says 172.21.15.139:47334) 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd6affd640 1 -- 172.21.15.139:0/256942368 learned_addr learned my addr 172.21.15.139:0/256942368 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd6affd640 1 -- 172.21.15.139:0/256942368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd6c161850 msgr2=0x7fdd6c153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd6affd640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd6c161850 0x7fdd6c153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd6affd640 1 -- 172.21.15.139:0/256942368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c15dc60 msgr2=0x7fdd6c160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd6a7fc640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c15dc60 0x7fdd6c160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd6affd640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c15dc60 0x7fdd6c160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd6affd640 1 -- 172.21.15.139:0/256942368 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c15fef0 con 0x7fdd6c0d1080 2024-04-25T09:55:03.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd6a7fc640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c15dc60 0x7fdd6c160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:03.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd6affd640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd6c0d1080 0x7fdd6c14daf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00c870 tx=0x7fdd5c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd53fff640 1 -- 172.21.15.139:0/256942368 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c003b70 con 0x7fdd6c0d1080 2024-04-25T09:55:03.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd53fff640 1 -- 172.21.15.139:0/256942368 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c003d30 con 0x7fdd6c0d1080 2024-04-25T09:55:03.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.106+0000 7fdd719f5640 1 -- 172.21.15.139:0/256942368 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c1601d0 con 0x7fdd6c0d1080 2024-04-25T09:55:03.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.107+0000 7fdd53fff640 1 -- 172.21.15.139:0/256942368 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c00aa90 con 0x7fdd6c0d1080 2024-04-25T09:55:03.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.107+0000 7fdd719f5640 1 -- 172.21.15.139:0/256942368 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c157e50 con 0x7fdd6c0d1080 2024-04-25T09:55:03.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.107+0000 7fdd719f5640 1 -- 172.21.15.139:0/256942368 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c074520 con 0x7fdd6c0d1080 2024-04-25T09:55:03.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.108+0000 7fdd53fff640 1 -- 172.21.15.139:0/256942368 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd5c00ac30 con 0x7fdd6c0d1080 2024-04-25T09:55:03.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.108+0000 7fdd53fff640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdd5806b500 0x7fdd5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.108+0000 7fdd53fff640 1 -- 172.21.15.139:0/256942368 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd5c08a280 con 0x7fdd6c0d1080 2024-04-25T09:55:03.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.108+0000 7fdd6a7fc640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdd5806b500 0x7fdd5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.109+0000 7fdd6a7fc640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdd5806b500 0x7fdd5806d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdd60034000 tx=0x7fdd6002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.112+0000 7fdd53fff640 1 -- 172.21.15.139:0/256942368 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd5c059000 con 0x7fdd6c0d1080 2024-04-25T09:55:03.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.319+0000 7fdd719f5640 1 -- 172.21.15.139:0/256942368 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd6c0d16f0 con 0x7fdd6c0d1080 2024-04-25T09:55:03.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.320+0000 7fdd53fff640 1 -- 172.21.15.139:0/256942368 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fdd5c058900 con 0x7fdd6c0d1080 2024-04-25T09:55:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:03.324 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1742557158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1742557158},{"type":"v1","addr":"172.21.15.150:6834","nonce":1742557158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3496803834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3496803834},{"type":"v1","addr":"172.21.15.150:6835","nonce":3496803834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2014537692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2014537692},{"type":"v1","addr":"172.21.15.139:6835","nonce":2014537692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:56.466203+0000","modified":"2024-04-25T09:54:58.466766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5850":{"gid":5850,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/338275708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":338275708},{"type":"v1","addr":"172.21.15.139:6837","nonce":338275708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:55:03.324 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 22 2024-04-25T09:55:03.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.322+0000 7fdd719f5640 1 -- 172.21.15.139:0/256942368 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdd5806b500 msgr2=0x7fdd5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.322+0000 7fdd719f5640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdd5806b500 0x7fdd5806d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdd60034000 tx=0x7fdd6002e000 comp rx=0 tx=0).stop 2024-04-25T09:55:03.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.322+0000 7fdd719f5640 1 -- 172.21.15.139:0/256942368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd6c0d1080 msgr2=0x7fdd6c14daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.322+0000 7fdd719f5640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd6c0d1080 0x7fdd6c14daf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00c870 tx=0x7fdd5c00cd30 comp rx=0 tx=0).stop 2024-04-25T09:55:03.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.322+0000 7fdd719f5640 1 -- 172.21.15.139:0/256942368 shutdown_connections 2024-04-25T09:55:03.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.322+0000 7fdd719f5640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdd5806b500 0x7fdd5806d9c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.322+0000 7fdd719f5640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd6c161850 0x7fdd6c153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.322+0000 7fdd719f5640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c15dc60 0x7fdd6c160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.322+0000 7fdd719f5640 1 --2- 172.21.15.139:0/256942368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd6c0d1080 0x7fdd6c14daf0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.322+0000 7fdd719f5640 1 -- 172.21.15.139:0/256942368 >> 172.21.15.139:0/256942368 conn(0x7fdd6c0b5910 msgr2=0x7fdd6c0b8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.322+0000 7fdd719f5640 1 -- 172.21.15.139:0/256942368 shutdown_connections 2024-04-25T09:55:03.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.322+0000 7fdd719f5640 1 -- 172.21.15.139:0/256942368 wait complete. 2024-04-25T09:55:03.337 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T09:55:03.455 INFO:teuthology.orchestra.run.smithi139.stdout:{"message":"","return_code":0} 2024-04-25T09:55:03.456 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:55:03.456 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T09:55:03.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4044d1bf40 1 Processor -- start 2024-04-25T09:55:03.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4044d1bf40 1 -- start start 2024-04-25T09:55:03.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4044d1bf40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40300022d0 0x564d41908520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4044d1bf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564d419034e0 0x564d418fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4044d1bf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d419038c0 0x564d418fbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4044d1bf40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d417d9df0 con 0x7f40300022d0 2024-04-25T09:55:03.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4044d1bf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d417ddd50 con 0x564d419038c0 2024-04-25T09:55:03.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4044d1bf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d4187ab40 con 0x564d419034e0 2024-04-25T09:55:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4041d41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40300022d0 0x564d41908520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4041d41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40300022d0 0x564d41908520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47340/0 (socket says 172.21.15.139:47340) 2024-04-25T09:55:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4041d41640 1 -- 172.21.15.139:0/3751617569 learned_addr learned my addr 172.21.15.139:0/3751617569 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4042542640 1 --2- 172.21.15.139:0/3751617569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d419038c0 0x564d418fbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.495+0000 7f4041540640 1 --2- 172.21.15.139:0/3751617569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564d419034e0 0x564d418fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4041d41640 1 -- 172.21.15.139:0/3751617569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564d419034e0 msgr2=0x564d418fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4041d41640 1 --2- 172.21.15.139:0/3751617569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564d419034e0 0x564d418fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4041d41640 1 -- 172.21.15.139:0/3751617569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d419038c0 msgr2=0x564d418fbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4041d41640 1 --2- 172.21.15.139:0/3751617569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d419038c0 0x564d418fbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4041d41640 1 -- 172.21.15.139:0/3751617569 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d41906aa0 con 0x7f40300022d0 2024-04-25T09:55:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4041d41640 1 --2- 172.21.15.139:0/3751617569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40300022d0 0x564d41908520 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4034026080 tx=0x7f4034006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7e1a1203cb808d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f403bfff640 1 -- 172.21.15.139:0/3751617569 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40340019b0 con 0x7f40300022d0 2024-04-25T09:55:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f403bfff640 1 -- 172.21.15.139:0/3751617569 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4034006070 con 0x7f40300022d0 2024-04-25T09:55:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f403bfff640 1 -- 172.21.15.139:0/3751617569 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403404d910 con 0x7f40300022d0 2024-04-25T09:55:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4044d1bf40 1 -- 172.21.15.139:0/3751617569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40300022d0 msgr2=0x564d41908520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4044d1bf40 1 --2- 172.21.15.139:0/3751617569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40300022d0 0x564d41908520 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4034026080 tx=0x7f4034006000 comp rx=0 tx=0).stop 2024-04-25T09:55:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4044d1bf40 1 -- 172.21.15.139:0/3751617569 shutdown_connections 2024-04-25T09:55:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4044d1bf40 1 --2- 172.21.15.139:0/3751617569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d419038c0 0x564d418fbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4044d1bf40 1 --2- 172.21.15.139:0/3751617569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564d419034e0 0x564d418fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4044d1bf40 1 --2- 172.21.15.139:0/3751617569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40300022d0 0x564d41908520 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4044d1bf40 1 -- 172.21.15.139:0/3751617569 >> 172.21.15.139:0/3751617569 conn(0x564d41832be0 msgr2=0x564d418fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4044d1bf40 1 -- 172.21.15.139:0/3751617569 shutdown_connections 2024-04-25T09:55:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.496+0000 7f4044d1bf40 1 -- 172.21.15.139:0/3751617569 wait complete. 2024-04-25T09:55:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.497+0000 7f4044d1bf40 1 Processor -- start 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.497+0000 7f4044d1bf40 1 -- start start 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4044d1bf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d419034e0 0x564d418f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4044d1bf40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d419038c0 0x564d418f2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4044d1bf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40300022d0 0x564d418fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4044d1bf40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d419076b0 con 0x564d419038c0 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4044d1bf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d41915430 con 0x564d419034e0 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4044d1bf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d41914820 con 0x7f40300022d0 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4041540640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d419038c0 0x564d418f2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4041540640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d419038c0 0x564d418f2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47342/0 (socket says 172.21.15.139:47342) 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4041540640 1 -- 172.21.15.139:0/4100681088 learned_addr learned my addr 172.21.15.139:0/4100681088 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4041d41640 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d419034e0 0x564d418f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4042542640 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40300022d0 0x564d418fcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4041540640 1 -- 172.21.15.139:0/4100681088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40300022d0 msgr2=0x564d418fcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4041540640 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40300022d0 0x564d418fcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4041540640 1 -- 172.21.15.139:0/4100681088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d419034e0 msgr2=0x564d418f0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4041540640 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d419034e0 0x564d418f0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4041540640 1 -- 172.21.15.139:0/4100681088 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d41914b00 con 0x564d419038c0 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4041540640 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d419038c0 0x564d418f2d00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f402c009ea0 tx=0x7f402c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f403affd640 1 -- 172.21.15.139:0/4100681088 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c01a070 con 0x564d419038c0 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f403affd640 1 -- 172.21.15.139:0/4100681088 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c016610 con 0x564d419038c0 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4044d1bf40 1 -- 172.21.15.139:0/4100681088 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564d41914de0 con 0x564d419038c0 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f403affd640 1 -- 172.21.15.139:0/4100681088 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c016a00 con 0x564d419038c0 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.498+0000 7f4044d1bf40 1 -- 172.21.15.139:0/4100681088 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d418f1540 con 0x564d419038c0 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.500+0000 7f403affd640 1 -- 172.21.15.139:0/4100681088 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f402c01d070 con 0x564d419038c0 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.500+0000 7f403affd640 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4024069eb0 0x7f402406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.500+0000 7f403affd640 1 -- 172.21.15.139:0/4100681088 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f402c0895c0 con 0x564d419038c0 2024-04-25T09:55:03.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.500+0000 7f4041d41640 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4024069eb0 0x7f402406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.500+0000 7f4044d1bf40 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x564d419a5110 0x564d419a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.500+0000 7f4044d1bf40 1 -- 172.21.15.139:0/4100681088 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50) v9 -- 0x564d419158c0 con 0x564d419a5110 2024-04-25T09:55:03.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.500+0000 7f4042542640 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x564d419a5110 0x564d419a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4041d41640 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4024069eb0 0x7f402406c370 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f40340260b0 tx=0x7f4034050000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4042542640 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x564d419a5110 0x564d419a5520 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4042542640 1 -- 172.21.15.139:0/4100681088 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f403c00e040 con 0x564d419a5110 2024-04-25T09:55:03.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4044d1bf40 1 -- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x564d419a5110 msgr2=0x564d419a5520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4044d1bf40 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x564d419a5110 0x564d419a5520 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4044d1bf40 1 -- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4024069eb0 msgr2=0x7f402406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4044d1bf40 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4024069eb0 0x7f402406c370 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f40340260b0 tx=0x7f4034050000 comp rx=0 tx=0).stop 2024-04-25T09:55:03.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4044d1bf40 1 -- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d419038c0 msgr2=0x564d418f2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4044d1bf40 1 --2- 172.21.15.139:0/4100681088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d419038c0 0x564d418f2d00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f402c009ea0 tx=0x7f402c009f90 comp rx=0 tx=0).stop 2024-04-25T09:55:03.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4042542640 1 -- 172.21.15.139:0/4100681088 reap_dead start 2024-04-25T09:55:03.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4044d1bf40 1 -- 172.21.15.139:0/4100681088 shutdown_connections 2024-04-25T09:55:03.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4044d1bf40 1 -- 172.21.15.139:0/4100681088 >> 172.21.15.139:0/4100681088 conn(0x564d41832be0 msgr2=0x564d418fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4044d1bf40 1 -- 172.21.15.139:0/4100681088 shutdown_connections 2024-04-25T09:55:03.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.501+0000 7f4044d1bf40 1 -- 172.21.15.139:0/4100681088 wait complete. 2024-04-25T09:55:03.512 INFO:teuthology.orchestra.run.smithi139.stdout:Yalphaparent_a 2024-04-25T09:55:03.512 DEBUG:teuthology.orchestra.run.smithi139:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T09:55:03.634 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:55:03.634 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 1099511627777, 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "ancestors": [ 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "dirino": 1099511627776, 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "dname": "alpha", 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 2 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "dirino": 1, 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "dname": "parent_a", 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 4 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-04-25T09:55:03.635 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 8, 2024-04-25T09:55:03.636 INFO:teuthology.orchestra.run.smithi139.stdout: "old_pools": [] 2024-04-25T09:55:03.636 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:55:03.636 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:03.651 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-25T09:55:03.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.731+0000 7f49ce461f40 1 Processor -- start 2024-04-25T09:55:03.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.731+0000 7f49ce461f40 1 -- start start 2024-04-25T09:55:03.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.731+0000 7f49ce461f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ed14f7580 0x562ed15c2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.731+0000 7f49ce461f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed153c430 0x562ed1538570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.731+0000 7f49ce461f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562ed15c6530 0x562ed14f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.731+0000 7f49ce461f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed149bdf0 con 0x562ed153c430 2024-04-25T09:55:03.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.731+0000 7f49ce461f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed149fd50 con 0x562ed14f7580 2024-04-25T09:55:03.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.731+0000 7f49ce461f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ed153cb40 con 0x562ed15c6530 2024-04-25T09:55:03.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49cb761640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed153c430 0x562ed1538570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49cb761640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed153c430 0x562ed1538570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47356/0 (socket says 172.21.15.139:47356) 2024-04-25T09:55:03.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49cb761640 1 -- 172.21.15.139:0/956233779 learned_addr learned my addr 172.21.15.139:0/956233779 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:03.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49cbf62640 1 --2- 172.21.15.139:0/956233779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ed14f7580 0x562ed15c2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49cc763640 1 --2- 172.21.15.139:0/956233779 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562ed15c6530 0x562ed14f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49cb761640 1 -- 172.21.15.139:0/956233779 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562ed15c6530 msgr2=0x562ed14f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49cb761640 1 --2- 172.21.15.139:0/956233779 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562ed15c6530 0x562ed14f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49cb761640 1 -- 172.21.15.139:0/956233779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ed14f7580 msgr2=0x562ed15c2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49cb761640 1 --2- 172.21.15.139:0/956233779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ed14f7580 0x562ed15c2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49cb761640 1 -- 172.21.15.139:0/956233779 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ed14f7390 con 0x562ed153c430 2024-04-25T09:55:03.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49cb761640 1 --2- 172.21.15.139:0/956233779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed153c430 0x562ed1538570 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f49c0057040 tx=0x7f49c004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdebdd57188eb9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49ca75f640 1 -- 172.21.15.139:0/956233779 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0059040 con 0x562ed153c430 2024-04-25T09:55:03.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49ca75f640 1 -- 172.21.15.139:0/956233779 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c0064070 con 0x562ed153c430 2024-04-25T09:55:03.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.732+0000 7f49ca75f640 1 -- 172.21.15.139:0/956233779 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c000a300 con 0x562ed153c430 2024-04-25T09:55:03.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.733+0000 7f49ce461f40 1 -- 172.21.15.139:0/956233779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed153c430 msgr2=0x562ed1538570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.733+0000 7f49ce461f40 1 --2- 172.21.15.139:0/956233779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed153c430 0x562ed1538570 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f49c0057040 tx=0x7f49c004e000 comp rx=0 tx=0).stop 2024-04-25T09:55:03.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.733+0000 7f49ce461f40 1 -- 172.21.15.139:0/956233779 shutdown_connections 2024-04-25T09:55:03.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.733+0000 7f49ce461f40 1 --2- 172.21.15.139:0/956233779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed153c430 0x562ed1538570 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.733+0000 7f49ce461f40 1 --2- 172.21.15.139:0/956233779 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562ed15c6530 0x562ed14f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.733+0000 7f49ce461f40 1 --2- 172.21.15.139:0/956233779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ed14f7580 0x562ed15c2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.733+0000 7f49ce461f40 1 -- 172.21.15.139:0/956233779 >> 172.21.15.139:0/956233779 conn(0x562ed14f4be0 msgr2=0x562ed15ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.733+0000 7f49ce461f40 1 -- 172.21.15.139:0/956233779 shutdown_connections 2024-04-25T09:55:03.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.733+0000 7f49ce461f40 1 -- 172.21.15.139:0/956233779 wait complete. 2024-04-25T09:55:03.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.733+0000 7f49ce461f40 1 Processor -- start 2024-04-25T09:55:03.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.733+0000 7f49ce461f40 1 -- start start 2024-04-25T09:55:03.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49ce461f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed14f7580 0x562ed15cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49ce461f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562ed153c430 0x562ed1516be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49ce461f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ed15c6530 0x562ed15c0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49ce461f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed14f5780 con 0x562ed14f7580 2024-04-25T09:55:03.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49ce461f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed15c8110 con 0x562ed15c6530 2024-04-25T09:55:03.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49ce461f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ed15c8a40 con 0x562ed153c430 2024-04-25T09:55:03.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49cbf62640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed14f7580 0x562ed15cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49cbf62640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed14f7580 0x562ed15cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47372/0 (socket says 172.21.15.139:47372) 2024-04-25T09:55:03.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49cbf62640 1 -- 172.21.15.139:0/2354919161 learned_addr learned my addr 172.21.15.139:0/2354919161 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:03.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49cb761640 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562ed153c430 0x562ed1516be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49cc763640 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ed15c6530 0x562ed15c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49cbf62640 1 -- 172.21.15.139:0/2354919161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562ed153c430 msgr2=0x562ed1516be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49cbf62640 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562ed153c430 0x562ed1516be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49cbf62640 1 -- 172.21.15.139:0/2354919161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ed15c6530 msgr2=0x562ed15c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49cbf62640 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562ed15c6530 0x562ed15c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49cbf62640 1 -- 172.21.15.139:0/2354919161 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ed15bf180 con 0x562ed14f7580 2024-04-25T09:55:03.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49cb761640 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562ed153c430 0x562ed1516be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:03.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.734+0000 7f49cbf62640 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed14f7580 0x562ed15cb5d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00e500 tx=0x7f49bc00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.735+0000 7f49c975d640 1 -- 172.21.15.139:0/2354919161 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc016020 con 0x562ed14f7580 2024-04-25T09:55:03.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.735+0000 7f49c975d640 1 -- 172.21.15.139:0/2354919161 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc0126c0 con 0x562ed14f7580 2024-04-25T09:55:03.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.735+0000 7f49c975d640 1 -- 172.21.15.139:0/2354919161 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc012a70 con 0x562ed14f7580 2024-04-25T09:55:03.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.735+0000 7f49ce461f40 1 -- 172.21.15.139:0/2354919161 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562ed15bf3f0 con 0x562ed14f7580 2024-04-25T09:55:03.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.735+0000 7f49ce461f40 1 -- 172.21.15.139:0/2354919161 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ed15ca610 con 0x562ed14f7580 2024-04-25T09:55:03.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.736+0000 7f49c975d640 1 -- 172.21.15.139:0/2354919161 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49bc019020 con 0x562ed14f7580 2024-04-25T09:55:03.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.736+0000 7f49c975d640 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f49b8069eb0 0x7f49b806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.736+0000 7f49cb761640 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f49b8069eb0 0x7f49b806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.736+0000 7f49c975d640 1 -- 172.21.15.139:0/2354919161 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f49bc089110 con 0x562ed14f7580 2024-04-25T09:55:03.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.737+0000 7f49ce461f40 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x562ed1517120 0x562ed166f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.737+0000 7f49ce461f40 1 -- 172.21.15.139:0/2354919161 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50) v9 -- 0x562ed1520870 con 0x562ed1517120 2024-04-25T09:55:03.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.737+0000 7f49cc763640 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x562ed1517120 0x562ed166f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.737+0000 7f49cb761640 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f49b8069eb0 0x7f49b806c370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49c0060000 tx=0x7f49c0051000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.737+0000 7f49cc763640 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x562ed1517120 0x562ed166f340 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.738+0000 7f49cc763640 1 -- 172.21.15.139:0/2354919161 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v50'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f49c40104b0 con 0x562ed1517120 2024-04-25T09:55:03.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.738+0000 7f49ce461f40 1 -- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x562ed1517120 msgr2=0x562ed166f340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.738+0000 7f49ce461f40 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x562ed1517120 0x562ed166f340 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.738+0000 7f49ce461f40 1 -- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f49b8069eb0 msgr2=0x7f49b806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.738+0000 7f49ce461f40 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f49b8069eb0 0x7f49b806c370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49c0060000 tx=0x7f49c0051000 comp rx=0 tx=0).stop 2024-04-25T09:55:03.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.738+0000 7f49ce461f40 1 -- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed14f7580 msgr2=0x562ed15cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.738+0000 7f49ce461f40 1 --2- 172.21.15.139:0/2354919161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ed14f7580 0x562ed15cb5d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00e500 tx=0x7f49bc00e9c0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.738+0000 7f49cc763640 1 -- 172.21.15.139:0/2354919161 reap_dead start 2024-04-25T09:55:03.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.738+0000 7f49ce461f40 1 -- 172.21.15.139:0/2354919161 shutdown_connections 2024-04-25T09:55:03.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.738+0000 7f49ce461f40 1 -- 172.21.15.139:0/2354919161 >> 172.21.15.139:0/2354919161 conn(0x562ed14f4be0 msgr2=0x562ed15234f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.738+0000 7f49ce461f40 1 -- 172.21.15.139:0/2354919161 shutdown_connections 2024-04-25T09:55:03.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.738+0000 7f49ce461f40 1 -- 172.21.15.139:0/2354919161 wait complete. 2024-04-25T09:55:03.748 DEBUG:teuthology.orchestra.run.smithi139:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T09:55:03.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:03.790 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-25T09:55:03.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.888+0000 7fb9202c4640 1 Processor -- start 2024-04-25T09:55:03.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.888+0000 7fb9202c4640 1 -- start start 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb9202c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb91815dbf0 0x7fb9181617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb9202c4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb9180cee50 0x7fb9180d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb9202c4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb918161dc0 con 0x7fb9180b9370 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb9202c4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9180d1ee0 con 0x7fb91815dbf0 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb9202c4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9181620a0 con 0x7fb9180cee50 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb91e060640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb91815dbf0 0x7fb9181617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb91d85f640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb9180b9370 0x7fb9180ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:36068/0 (socket says 172.21.15.139:36068) 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb91d85f640 1 -- 172.21.15.139:0/1176908367 learned_addr learned my addr 172.21.15.139:0/1176908367 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb91e060640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb91815dbf0 0x7fb9181617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47388/0 (socket says 172.21.15.139:47388) 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb91e861640 1 --2- 172.21.15.139:0/1176908367 >> v2:172.21.15.150:3300/0 conn(0x7fb9180cee50 0x7fb9180d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb91e060640 1 -- 172.21.15.139:0/1176908367 >> v2:172.21.15.150:3300/0 conn(0x7fb9180cee50 msgr2=0x7fb9180d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb91e060640 1 --2- 172.21.15.139:0/1176908367 >> v2:172.21.15.150:3300/0 conn(0x7fb9180cee50 0x7fb9180d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb91e060640 1 -- 172.21.15.139:0/1176908367 >> v1:172.21.15.139:6789/0 conn(0x7fb9180b9370 legacy=0x7fb9180ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.889+0000 7fb91e060640 1 -- 172.21.15.139:0/1176908367 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918162380 con 0x7fb91815dbf0 2024-04-25T09:55:03.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.890+0000 7fb91e060640 1 --2- 172.21.15.139:0/1176908367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb91815dbf0 0x7fb9181617e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb91808ce00 tx=0x7fb91402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729898ebb65eb603 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.890+0000 7fb91d05e640 1 -- 172.21.15.139:0/1176908367 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb914037070 con 0x7fb91815dbf0 2024-04-25T09:55:03.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.890+0000 7fb91d05e640 1 -- 172.21.15.139:0/1176908367 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb914032440 con 0x7fb91815dbf0 2024-04-25T09:55:03.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.890+0000 7fb91d05e640 1 -- 172.21.15.139:0/1176908367 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb914032810 con 0x7fb91815dbf0 2024-04-25T09:55:03.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.891+0000 7fb9202c4640 1 -- 172.21.15.139:0/1176908367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb91815dbf0 msgr2=0x7fb9181617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.891+0000 7fb9202c4640 1 --2- 172.21.15.139:0/1176908367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb91815dbf0 0x7fb9181617e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb91808ce00 tx=0x7fb91402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.891+0000 7fb9202c4640 1 -- 172.21.15.139:0/1176908367 shutdown_connections 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.891+0000 7fb9202c4640 1 --2- 172.21.15.139:0/1176908367 >> v2:172.21.15.150:3300/0 conn(0x7fb9180cee50 0x7fb9180d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.891+0000 7fb9202c4640 1 --2- 172.21.15.139:0/1176908367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb91815dbf0 0x7fb9181617e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.891+0000 7fb9202c4640 1 -- 172.21.15.139:0/1176908367 >> 172.21.15.139:0/1176908367 conn(0x7fb9180b5480 msgr2=0x7fb9180b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.891+0000 7fb9202c4640 1 -- 172.21.15.139:0/1176908367 shutdown_connections 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.891+0000 7fb9202c4640 1 -- 172.21.15.139:0/1176908367 wait complete. 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.892+0000 7fb9202c4640 1 Processor -- start 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.892+0000 7fb9202c4640 1 -- start start 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb9202c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9180b9370 0x7fb918158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb9202c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180cee50 0x7fb91815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb9202c4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb91815dbf0 0x7fb9181f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb9202c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91815dfd0 con 0x7fb9180cee50 2024-04-25T09:55:03.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb9202c4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9180b97d0 con 0x7fb91815dbf0 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb9202c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91815a640 con 0x7fb9180b9370 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb91d85f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180cee50 0x7fb91815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb91d85f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180cee50 0x7fb91815cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47394/0 (socket says 172.21.15.139:47394) 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb91d85f640 1 -- 172.21.15.139:0/3332525598 learned_addr learned my addr 172.21.15.139:0/3332525598 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb91e861640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb91815dbf0 0x7fb9181f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb91d85f640 1 -- 172.21.15.139:0/3332525598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9180b9370 msgr2=0x7fb918158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb91d85f640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9180b9370 0x7fb918158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb91d85f640 1 -- 172.21.15.139:0/3332525598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb91815dbf0 msgr2=0x7fb9181f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb91d85f640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb91815dbf0 0x7fb9181f3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb91d85f640 1 -- 172.21.15.139:0/3332525598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918159a70 con 0x7fb9180cee50 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb91e861640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb91815dbf0 0x7fb9181f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.893+0000 7fb91d85f640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180cee50 0x7fb91815cb40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb90800c3c0 tx=0x7fb90800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.894+0000 7fb90effd640 1 -- 172.21.15.139:0/3332525598 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb908017070 con 0x7fb9180cee50 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.894+0000 7fb90effd640 1 -- 172.21.15.139:0/3332525598 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb908005ce0 con 0x7fb9180cee50 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.894+0000 7fb9202c4640 1 -- 172.21.15.139:0/3332525598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb918159d10 con 0x7fb9180cee50 2024-04-25T09:55:03.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.894+0000 7fb90effd640 1 -- 172.21.15.139:0/3332525598 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9080072f0 con 0x7fb9180cee50 2024-04-25T09:55:03.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.894+0000 7fb9202c4640 1 -- 172.21.15.139:0/3332525598 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb918159480 con 0x7fb9180cee50 2024-04-25T09:55:03.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.895+0000 7fb90effd640 1 -- 172.21.15.139:0/3332525598 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb90801a070 con 0x7fb9180cee50 2024-04-25T09:55:03.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.895+0000 7fb90effd640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb8fc06b4d0 0x7fb8fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.896+0000 7fb90effd640 1 -- 172.21.15.139:0/3332525598 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb908089770 con 0x7fb9180cee50 2024-04-25T09:55:03.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.896+0000 7fb91e060640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb8fc06b4d0 0x7fb8fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.896+0000 7fb91e060640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb8fc06b4d0 0x7fb8fc06d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb914034000 tx=0x7fb91402c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.897+0000 7fb922fca740 1 Processor -- start 2024-04-25T09:55:03.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.897+0000 7fb922fca740 1 -- start start 2024-04-25T09:55:03.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb922fca740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac04bb0 0x560b8ac04f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb922fca740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8abff830 0x560b8ac037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb922fca740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 0x560b8ac04130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb922fca740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ab68ac0 con 0x560b8ac03cf0 2024-04-25T09:55:03.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb922fca740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ab11d80 con 0x560b8ac04bb0 2024-04-25T09:55:03.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb922fca740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8ab64b60 con 0x560b8abff830 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb91e861640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 0x560b8ac04130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb91e861640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 0x560b8ac04130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47410/0 (socket says 172.21.15.139:47410) 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb91e861640 1 -- 172.21.15.139:0/4002591610 learned_addr learned my addr 172.21.15.139:0/4002591610 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb91e060640 1 --2- 172.21.15.139:0/4002591610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac04bb0 0x560b8ac04f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb91d85f640 1 --2- 172.21.15.139:0/4002591610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8abff830 0x560b8ac037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb91e861640 1 -- 172.21.15.139:0/4002591610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8abff830 msgr2=0x560b8ac037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb91e861640 1 --2- 172.21.15.139:0/4002591610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8abff830 0x560b8ac037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb91e861640 1 -- 172.21.15.139:0/4002591610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac04bb0 msgr2=0x560b8ac04f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb91e861640 1 --2- 172.21.15.139:0/4002591610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac04bb0 0x560b8ac04f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb91e861640 1 -- 172.21.15.139:0/4002591610 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b8a9457e0 con 0x560b8ac03cf0 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb91e861640 1 --2- 172.21.15.139:0/4002591610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 0x560b8ac04130 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb910007360 tx=0x7fb910007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f947cd38ccc94f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb8fbfff640 1 -- 172.21.15.139:0/4002591610 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910003b90 con 0x560b8ac03cf0 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb8fbfff640 1 -- 172.21.15.139:0/4002591610 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb910003d30 con 0x560b8ac03cf0 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.898+0000 7fb8fbfff640 1 -- 172.21.15.139:0/4002591610 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910004250 con 0x560b8ac03cf0 2024-04-25T09:55:03.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.899+0000 7fb922fca740 1 -- 172.21.15.139:0/4002591610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 msgr2=0x560b8ac04130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.899+0000 7fb922fca740 1 --2- 172.21.15.139:0/4002591610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 0x560b8ac04130 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb910007360 tx=0x7fb910007820 comp rx=0 tx=0).stop 2024-04-25T09:55:03.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.899+0000 7fb922fca740 1 -- 172.21.15.139:0/4002591610 shutdown_connections 2024-04-25T09:55:03.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.899+0000 7fb922fca740 1 --2- 172.21.15.139:0/4002591610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 0x560b8ac04130 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.899+0000 7fb922fca740 1 --2- 172.21.15.139:0/4002591610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8abff830 0x560b8ac037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.899+0000 7fb922fca740 1 --2- 172.21.15.139:0/4002591610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac04bb0 0x560b8ac04f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.899+0000 7fb922fca740 1 -- 172.21.15.139:0/4002591610 >> 172.21.15.139:0/4002591610 conn(0x560b8ac01cb0 msgr2=0x560b8ac4a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.899+0000 7fb922fca740 1 -- 172.21.15.139:0/4002591610 shutdown_connections 2024-04-25T09:55:03.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.899+0000 7fb922fca740 1 -- 172.21.15.139:0/4002591610 wait complete. 2024-04-25T09:55:03.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.900+0000 7fb922fca740 1 -- start start 2024-04-25T09:55:03.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.900+0000 7fb922fca740 1 Processor -- start 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb922fca740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac03cf0 0x560b8ac29370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb922fca740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac04bb0 0x560b8ac4df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb922fca740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac298b0 0x560b8ac4e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb922fca740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ac30740 con 0x560b8ac04bb0 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb922fca740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ab07530 con 0x560b8ac03cf0 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb922fca740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8ac2fbc0 con 0x560b8ac298b0 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb922fca740 10 client.0 _ll_register_callbacks cb 0x560b8ab5f130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb91d85f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac04bb0 0x560b8ac4df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb91d85f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac04bb0 0x560b8ac4df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47412/0 (socket says 172.21.15.139:47412) 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb91d85f640 1 -- 172.21.15.139:0/3865362173 learned_addr learned my addr 172.21.15.139:0/3865362173 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb91e861640 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac298b0 0x560b8ac4e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb91e060640 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac03cf0 0x560b8ac29370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb91d85f640 1 -- 172.21.15.139:0/3865362173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac298b0 msgr2=0x560b8ac4e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb91d85f640 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac298b0 0x560b8ac4e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb91d85f640 1 -- 172.21.15.139:0/3865362173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac03cf0 msgr2=0x560b8ac29370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb91d85f640 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac03cf0 0x560b8ac29370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb91d85f640 1 -- 172.21.15.139:0/3865362173 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b8a9a5f70 con 0x560b8ac04bb0 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb91d85f640 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac04bb0 0x560b8ac4df80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb908054170 tx=0x7fb908057e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6aa717c8585610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb8f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb8f97fa640 1 -- 172.21.15.139:0/3865362173 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb908086060 con 0x560b8ac04bb0 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb8f97fa640 1 -- 172.21.15.139:0/3865362173 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb908007490 con 0x560b8ac04bb0 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.901+0000 7fb922fca740 1 -- 172.21.15.139:0/3865362173 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb910003d30 con 0x560b8ac04bb0 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb8f97fa640 1 -- 172.21.15.139:0/3865362173 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb90806f070 con 0x560b8ac04bb0 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb922fca740 10 client.6054 fetch_fsmap learned FSMap version 22 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb922fca740 1 -- 172.21.15.139:0/3865362173 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x560b8a83d170 con 0x560b8ac04bb0 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb8f97fa640 1 -- 172.21.15.139:0/3865362173 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb910003d30 con 0x560b8ac04bb0 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb8f97fa640 1 -- 172.21.15.139:0/3865362173 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb9080571b0 con 0x560b8ac04bb0 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb922fca740 10 client.6054 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb922fca740 10 client.6054 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb922fca740 20 client.6054 populate_metadata read hostname 'smithi139' 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb922fca740 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x560b8ac4f310 0x560b8ac4f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb922fca740 4 client.6054 mds_command: new command op to 5850 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb922fca740 1 -- 172.21.15.139:0/3865362173 --> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b8ab76000 con 0x560b8ac4f310 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb91e060640 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x560b8ac4f310 0x560b8ac4f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb8f97fa640 10 client.6054 ms_handle_connect on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.902+0000 7fb91e060640 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x560b8ac4f310 0x560b8ac4f6f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=193f8eadd262a3d9 server_cookie=719c1dd1ba64d17c in_seq=0 out_seq=0 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb8f97fa640 1 -- 172.21.15.139:0/3865362173 <== mds.0 v2:172.21.15.139:6836/338275708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560b8ab76000 con 0x560b8ac4f310 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb8f97fa640 10 client.6054 handle_command_reply: tid=0 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb922fca740 1 client.6054 shutdown 2024-04-25T09:55:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb922fca740 1 -- 172.21.15.139:0/3865362173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac04bb0 msgr2=0x560b8ac4df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb922fca740 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac04bb0 0x560b8ac4df80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb908054170 tx=0x7fb908057e30 comp rx=0 tx=0).stop 2024-04-25T09:55:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb922fca740 1 -- 172.21.15.139:0/3865362173 shutdown_connections 2024-04-25T09:55:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb922fca740 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x560b8ac4f310 0x560b8ac4f6f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb922fca740 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac04bb0 0x560b8ac4df80 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb922fca740 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac298b0 0x560b8ac4e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb8f97fa640 0 client.6054 ms_handle_reset on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb922fca740 1 --2- 172.21.15.139:0/3865362173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac03cf0 0x560b8ac29370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb922fca740 1 -- 172.21.15.139:0/3865362173 >> 172.21.15.139:0/3865362173 conn(0x560b8abff830 msgr2=0x560b8ac056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.903+0000 7fb922fca740 1 -- 172.21.15.139:0/3865362173 shutdown_connections 2024-04-25T09:55:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.904+0000 7fb922fca740 1 -- 172.21.15.139:0/3865362173 wait complete. 2024-04-25T09:55:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.904+0000 7fb922fca740 20 client.6054 trim_cache size 0 max 16384 2024-04-25T09:55:03.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.909+0000 7f649a491640 1 Processor -- start 2024-04-25T09:55:03.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.909+0000 7f649a491640 1 -- start start 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.910+0000 7f649a491640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f649415f1a0 0x7f649406fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.910+0000 7f649a491640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64940731a0 con 0x7f64940b2f80 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.910+0000 7f649a491640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6494070580 con 0x7f649406ab60 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.910+0000 7f649a491640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6494070860 con 0x7f649415f1a0 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.910+0000 7f64937fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f64940b2f80 0x7f6494072a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:36084/0 (socket says 172.21.15.139:36084) 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.910+0000 7f64937fe640 1 -- 172.21.15.139:0/1542362661 learned_addr learned my addr 172.21.15.139:0/1542362661 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.910+0000 7f6493fff640 1 --2- 172.21.15.139:0/1542362661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f649415f1a0 0x7f649406fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.910+0000 7f6492ffd640 1 -- 172.21.15.139:0/1542362661 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695011880 0 0) 0x7f64940731a0 con 0x7f64940b2f80 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.911+0000 7f6492ffd640 1 -- 172.21.15.139:0/1542362661 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f647c003520 con 0x7f64940b2f80 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.911+0000 7f6492ffd640 1 -- 172.21.15.139:0/1542362661 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807455004 0 0) 0x7f6494070580 con 0x7f649406ab60 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.911+0000 7f6492ffd640 1 -- 172.21.15.139:0/1542362661 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64940731a0 con 0x7f649406ab60 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.911+0000 7f6493fff640 1 -- 172.21.15.139:0/1542362661 >> v1:172.21.15.150:6789/0 conn(0x7f649406ab60 legacy=0x7f6494142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.911+0000 7f6493fff640 1 -- 172.21.15.139:0/1542362661 >> v1:172.21.15.139:6789/0 conn(0x7f64940b2f80 legacy=0x7f6494072a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.911+0000 7f6493fff640 1 -- 172.21.15.139:0/1542362661 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6494070b40 con 0x7f649415f1a0 2024-04-25T09:55:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.912+0000 7f6493fff640 1 --2- 172.21.15.139:0/1542362661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f649415f1a0 0x7f649406fc80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f64840097d0 tx=0x7f648402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc222e137f78ae04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.912+0000 7f6492ffd640 1 -- 172.21.15.139:0/1542362661 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484037070 con 0x7f649415f1a0 2024-04-25T09:55:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.912+0000 7f6492ffd640 1 -- 172.21.15.139:0/1542362661 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6484032440 con 0x7f649415f1a0 2024-04-25T09:55:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.912+0000 7f6492ffd640 1 -- 172.21.15.139:0/1542362661 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64840327f0 con 0x7f649415f1a0 2024-04-25T09:55:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.912+0000 7f649a491640 1 -- 172.21.15.139:0/1542362661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f649415f1a0 msgr2=0x7f649406fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.913+0000 7f649a491640 1 --2- 172.21.15.139:0/1542362661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f649415f1a0 0x7f649406fc80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f64840097d0 tx=0x7f648402a210 comp rx=0 tx=0).stop 2024-04-25T09:55:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.913+0000 7f649a491640 1 -- 172.21.15.139:0/1542362661 shutdown_connections 2024-04-25T09:55:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.913+0000 7f649a491640 1 --2- 172.21.15.139:0/1542362661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f649415f1a0 0x7f649406fc80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.913+0000 7f649a491640 1 -- 172.21.15.139:0/1542362661 >> 172.21.15.139:0/1542362661 conn(0x7f64940b23f0 msgr2=0x7f64940b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.913+0000 7f649a491640 1 -- 172.21.15.139:0/1542362661 shutdown_connections 2024-04-25T09:55:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.913+0000 7f649a491640 1 -- 172.21.15.139:0/1542362661 wait complete. 2024-04-25T09:55:03.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.914+0000 7f649a491640 1 Processor -- start 2024-04-25T09:55:03.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.914+0000 7f649a491640 1 -- start start 2024-04-25T09:55:03.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f649a491640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649406ab60 0x7f6494141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f649a491640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64940b2f80 0x7f6494142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f649a491640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f649415f1a0 0x7f64940b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f649a491640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6494072570 con 0x7f649415f1a0 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f649a491640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6494072290 con 0x7f649406ab60 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f649a491640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64940701c0 con 0x7f64940b2f80 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6498a2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f649415f1a0 0x7f64940b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6498a2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f649415f1a0 0x7f64940b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47424/0 (socket says 172.21.15.139:47424) 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6498a2e640 1 -- 172.21.15.139:0/1349623033 learned_addr learned my addr 172.21.15.139:0/1349623033 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6493fff640 1 --2- 172.21.15.139:0/1349623033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649406ab60 0x7f6494141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f64937fe640 1 --2- 172.21.15.139:0/1349623033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64940b2f80 0x7f6494142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6498a2e640 1 -- 172.21.15.139:0/1349623033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64940b2f80 msgr2=0x7f6494142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6498a2e640 1 --2- 172.21.15.139:0/1349623033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64940b2f80 0x7f6494142270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6498a2e640 1 -- 172.21.15.139:0/1349623033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649406ab60 msgr2=0x7f6494141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6498a2e640 1 --2- 172.21.15.139:0/1349623033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f649406ab60 0x7f6494141d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6498a2e640 1 -- 172.21.15.139:0/1349623033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64940ac9f0 con 0x7f649415f1a0 2024-04-25T09:55:03.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6498a2e640 1 --2- 172.21.15.139:0/1349623033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f649415f1a0 0x7f64940b45f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c710 tx=0x7f648c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6490ff9640 1 -- 172.21.15.139:0/1349623033 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c003b70 con 0x7f649415f1a0 2024-04-25T09:55:03.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6490ff9640 1 -- 172.21.15.139:0/1349623033 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c003d10 con 0x7f649415f1a0 2024-04-25T09:55:03.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f6490ff9640 1 -- 172.21.15.139:0/1349623033 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c00aa50 con 0x7f649415f1a0 2024-04-25T09:55:03.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.915+0000 7f649a491640 1 -- 172.21.15.139:0/1349623033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64940acc90 con 0x7f649415f1a0 2024-04-25T09:55:03.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.916+0000 7f649a491640 1 -- 172.21.15.139:0/1349623033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64940ad250 con 0x7f649415f1a0 2024-04-25T09:55:03.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.916+0000 7f649a491640 1 -- 172.21.15.139:0/1349623033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f6494074520 con 0x7f649415f1a0 2024-04-25T09:55:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.917+0000 7f6490ff9640 1 -- 172.21.15.139:0/1349623033 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f648c00abf0 con 0x7f649415f1a0 2024-04-25T09:55:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.917+0000 7f6490ff9640 1 --2- 172.21.15.139:0/1349623033 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f648006b500 0x7f648006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.917+0000 7f6490ff9640 1 -- 172.21.15.139:0/1349623033 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f648c08aa60 con 0x7f649415f1a0 2024-04-25T09:55:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.917+0000 7f6493fff640 1 --2- 172.21.15.139:0/1349623033 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f648006b500 0x7f648006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.918+0000 7f6490ff9640 1 -- 172.21.15.139:0/1349623033 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+501 (secure 0 0 0) 0x7f648c08b2d0 con 0x7f649415f1a0 2024-04-25T09:55:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.918+0000 7f649a491640 1 -- 172.21.15.139:0/1349623033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f649406df40 con 0x7f649415f1a0 2024-04-25T09:55:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.918+0000 7f6493fff640 1 --2- 172.21.15.139:0/1349623033 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f648006b500 0x7f648006d9c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6484034000 tx=0x7f648402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.920+0000 7fb922fca740 1 Processor -- start 2024-04-25T09:55:03.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.920+0000 7fb922fca740 1 -- start start 2024-04-25T09:55:03.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb922fca740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 0x560b8ac05180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb922fca740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac04bb0 0x560b8ac26330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb922fca740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac298b0 0x560b8ac26870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb922fca740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb918200470 con 0x560b8ac03cf0 2024-04-25T09:55:03.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb922fca740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9181ff630 con 0x560b8ac04bb0 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb922fca740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9181fea30 con 0x560b8ac298b0 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb91e060640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 0x560b8ac05180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb91e060640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 0x560b8ac05180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47432/0 (socket says 172.21.15.139:47432) 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb91e060640 1 -- 172.21.15.139:0/1281031627 learned_addr learned my addr 172.21.15.139:0/1281031627 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb91e861640 1 --2- 172.21.15.139:0/1281031627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac298b0 0x560b8ac26870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb91d85f640 1 --2- 172.21.15.139:0/1281031627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac04bb0 0x560b8ac26330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb91e060640 1 -- 172.21.15.139:0/1281031627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac298b0 msgr2=0x560b8ac26870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb91e060640 1 --2- 172.21.15.139:0/1281031627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac298b0 0x560b8ac26870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb91e060640 1 -- 172.21.15.139:0/1281031627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac04bb0 msgr2=0x560b8ac26330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb91e060640 1 --2- 172.21.15.139:0/1281031627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac04bb0 0x560b8ac26330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb91e060640 1 -- 172.21.15.139:0/1281031627 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b8ac2ab50 con 0x560b8ac03cf0 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.921+0000 7fb91e060640 1 --2- 172.21.15.139:0/1281031627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 0x560b8ac05180 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb914009860 tx=0x7fb914008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5015275ffcc55e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.922+0000 7fb8fb7fe640 1 -- 172.21.15.139:0/1281031627 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb914037040 con 0x560b8ac03cf0 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.922+0000 7fb8fb7fe640 1 -- 172.21.15.139:0/1281031627 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb914050b00 con 0x560b8ac03cf0 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.922+0000 7fb8fb7fe640 1 -- 172.21.15.139:0/1281031627 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb914050310 con 0x560b8ac03cf0 2024-04-25T09:55:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.922+0000 7fb922fca740 1 -- 172.21.15.139:0/1281031627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 msgr2=0x560b8ac05180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.922+0000 7fb922fca740 1 --2- 172.21.15.139:0/1281031627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 0x560b8ac05180 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb914009860 tx=0x7fb914008030 comp rx=0 tx=0).stop 2024-04-25T09:55:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.922+0000 7fb922fca740 1 -- 172.21.15.139:0/1281031627 shutdown_connections 2024-04-25T09:55:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.922+0000 7fb922fca740 1 --2- 172.21.15.139:0/1281031627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac298b0 0x560b8ac26870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.922+0000 7fb922fca740 1 --2- 172.21.15.139:0/1281031627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac04bb0 0x560b8ac26330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.922+0000 7fb922fca740 1 --2- 172.21.15.139:0/1281031627 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac03cf0 0x560b8ac05180 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.922+0000 7fb922fca740 1 -- 172.21.15.139:0/1281031627 >> 172.21.15.139:0/1281031627 conn(0x560b8abff830 msgr2=0x560b8ac6d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.922+0000 7fb922fca740 1 -- 172.21.15.139:0/1281031627 shutdown_connections 2024-04-25T09:55:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.922+0000 7fb922fca740 1 -- 172.21.15.139:0/1281031627 wait complete. 2024-04-25T09:55:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.923+0000 7fb922fca740 1 -- start start 2024-04-25T09:55:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.923+0000 7fb922fca740 1 Processor -- start 2024-04-25T09:55:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb922fca740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac03cf0 0x560b8acb2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb922fca740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac04bb0 0x560b8acb5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb922fca740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac298b0 0x560b8acb5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb922fca740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ac2f820 con 0x560b8ac298b0 2024-04-25T09:55:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb922fca740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9181ff030 con 0x560b8ac03cf0 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb922fca740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8ac4fe00 con 0x560b8ac04bb0 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb922fca740 10 client.0 _ll_register_callbacks cb 0x7fb910009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91e861640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac298b0 0x560b8acb5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91e861640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac298b0 0x560b8acb5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47444/0 (socket says 172.21.15.139:47444) 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91e861640 1 -- 172.21.15.139:0/4186957649 learned_addr learned my addr 172.21.15.139:0/4186957649 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91d85f640 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac04bb0 0x560b8acb5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91e060640 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac03cf0 0x560b8acb2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91e861640 1 -- 172.21.15.139:0/4186957649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac04bb0 msgr2=0x560b8acb5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91e861640 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac04bb0 0x560b8acb5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91e861640 1 -- 172.21.15.139:0/4186957649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac03cf0 msgr2=0x560b8acb2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91e861640 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac03cf0 0x560b8acb2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91e861640 1 -- 172.21.15.139:0/4186957649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b8ac019a0 con 0x560b8ac298b0 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91e060640 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac03cf0 0x560b8acb2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91d85f640 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac04bb0 0x560b8acb5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb91e861640 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac298b0 0x560b8acb5aa0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb910007d20 tx=0x7fb910004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b2c49eb84ca45e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.924+0000 7fb8f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb8f97fa640 1 -- 172.21.15.139:0/4186957649 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910004240 con 0x560b8ac298b0 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb8f97fa640 1 -- 172.21.15.139:0/4186957649 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb9100043e0 con 0x560b8ac298b0 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb922fca740 1 -- 172.21.15.139:0/4186957649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb914050b00 con 0x560b8ac298b0 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb8f97fa640 1 -- 172.21.15.139:0/4186957649 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb910033050 con 0x560b8ac298b0 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb922fca740 10 client.6066 fetch_fsmap learned FSMap version 22 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb922fca740 1 -- 172.21.15.139:0/4186957649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x560b8ac24f20 con 0x560b8ac298b0 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb8f97fa640 1 -- 172.21.15.139:0/4186957649 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb914050b00 con 0x560b8ac298b0 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb8f97fa640 1 -- 172.21.15.139:0/4186957649 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb910036020 con 0x560b8ac298b0 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb922fca740 10 client.6066 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb922fca740 10 client.6066 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb922fca740 20 client.6066 populate_metadata read hostname 'smithi139' 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb922fca740 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x560b8ac2d1f0 0x560b8ac59360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb922fca740 4 client.6066 mds_command: new command op to 5850 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb922fca740 1 -- 172.21.15.139:0/4186957649 --> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x560b8ac508c0 con 0x560b8ac2d1f0 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb91e060640 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x560b8ac2d1f0 0x560b8ac59360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.925+0000 7fb8f97fa640 10 client.6066 ms_handle_connect on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb91e060640 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x560b8ac2d1f0 0x560b8ac59360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f229ca6562aa3c60 server_cookie=4a1fac1b223ae36f in_seq=0 out_seq=0 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb8f97fa640 1 -- 172.21.15.139:0/4186957649 <== mds.0 v2:172.21.15.139:6836/338275708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560b8ac508c0 con 0x560b8ac2d1f0 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb8f97fa640 10 client.6066 handle_command_reply: tid=0 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb922fca740 1 client.6066 shutdown 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb922fca740 1 -- 172.21.15.139:0/4186957649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac298b0 msgr2=0x560b8acb5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb922fca740 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac298b0 0x560b8acb5aa0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb910007d20 tx=0x7fb910004ab0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb922fca740 1 -- 172.21.15.139:0/4186957649 shutdown_connections 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb922fca740 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x560b8ac2d1f0 0x560b8ac59360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb8f97fa640 0 client.6066 ms_handle_reset on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb922fca740 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b8ac298b0 0x560b8acb5aa0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb922fca740 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b8ac04bb0 0x560b8acb5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb922fca740 1 --2- 172.21.15.139:0/4186957649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b8ac03cf0 0x560b8acb2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.926+0000 7fb922fca740 1 -- 172.21.15.139:0/4186957649 >> 172.21.15.139:0/4186957649 conn(0x560b8abff830 msgr2=0x560b8ac29c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb922fca740 1 -- 172.21.15.139:0/4186957649 shutdown_connections 2024-04-25T09:55:03.936 INFO:tasks.ceph.mds.c.smithi139.stderr:2024-04-25T09:55:03.927+0000 7efe1c514640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5231=0-4194304@1} caps={5231=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5231 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x563dd530d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T09:55:03.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb922fca740 1 -- 172.21.15.139:0/4186957649 wait complete. 2024-04-25T09:55:03.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb922fca740 20 client.6066 trim_cache size 0 max 16384 2024-04-25T09:55:03.937 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:55:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "return_code": 0, 2024-04-25T09:55:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_tag": "6041bcc3-3169-4103-88f5-5b90a0e6c62b", 2024-04-25T09:55:03.937 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": "asynchronous" 2024-04-25T09:55:03.937 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:55:03.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb9202c4640 1 -- 172.21.15.139:0/3332525598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb8fc06b4d0 msgr2=0x7fb8fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb9202c4640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb8fc06b4d0 0x7fb8fc06d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb914034000 tx=0x7fb91402c000 comp rx=0 tx=0).stop 2024-04-25T09:55:03.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb9202c4640 1 -- 172.21.15.139:0/3332525598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180cee50 msgr2=0x7fb91815cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb9202c4640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180cee50 0x7fb91815cb40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb90800c3c0 tx=0x7fb90800c880 comp rx=0 tx=0).stop 2024-04-25T09:55:03.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb9202c4640 1 -- 172.21.15.139:0/3332525598 shutdown_connections 2024-04-25T09:55:03.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb9202c4640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180cee50 0x7fb91815cb40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb9202c4640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb8fc06b4d0 0x7fb8fc06d990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb9202c4640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb91815dbf0 0x7fb9181f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb9202c4640 1 --2- 172.21.15.139:0/3332525598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9180b9370 0x7fb918158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb9202c4640 1 -- 172.21.15.139:0/3332525598 >> 172.21.15.139:0/3332525598 conn(0x7fb9180b5480 msgr2=0x7fb9180d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.927+0000 7fb9202c4640 1 -- 172.21.15.139:0/3332525598 shutdown_connections 2024-04-25T09:55:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:03.928+0000 7fb9202c4640 1 -- 172.21.15.139:0/3332525598 wait complete. 2024-04-25T09:55:03.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:03.942 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:55:04.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.036+0000 7ff22c78b640 1 Processor -- start 2024-04-25T09:55:04.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.036+0000 7ff22c78b640 1 -- start start 2024-04-25T09:55:04.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.037+0000 7ff22c78b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff22415dbf0 0x7ff2241617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.037+0000 7ff22c78b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2240cee50 0x7ff2240d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.037+0000 7ff22c78b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff224161dc0 con 0x7ff2240b9370 2024-04-25T09:55:04.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.037+0000 7ff22c78b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2240d1ee0 con 0x7ff22415dbf0 2024-04-25T09:55:04.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.037+0000 7ff22c78b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2241620a0 con 0x7ff2240cee50 2024-04-25T09:55:04.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.038+0000 7ff22ad28640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2240cee50 0x7ff2240d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.038+0000 7ff22a527640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff22415dbf0 0x7ff2241617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.038+0000 7ff22ad28640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2240cee50 0x7ff2240d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:55424/0 (socket says 172.21.15.139:55424) 2024-04-25T09:55:04.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.038+0000 7ff22ad28640 1 -- 172.21.15.139:0/252710235 learned_addr learned my addr 172.21.15.139:0/252710235 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:04.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.038+0000 7ff229d26640 1 --1- 172.21.15.139:0/252710235 >> v1:172.21.15.150:6789/0 conn(0x7ff2240b9370 0x7ff2240ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:37218/0 (socket says 172.21.15.139:37218) 2024-04-25T09:55:04.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.038+0000 7ff22ad28640 1 -- 172.21.15.139:0/252710235 >> v2:172.21.15.150:3300/0 conn(0x7ff22415dbf0 msgr2=0x7ff2241617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.038+0000 7ff22ad28640 1 --2- 172.21.15.139:0/252710235 >> v2:172.21.15.150:3300/0 conn(0x7ff22415dbf0 0x7ff2241617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.038+0000 7ff22ad28640 1 -- 172.21.15.139:0/252710235 >> v1:172.21.15.150:6789/0 conn(0x7ff2240b9370 legacy=0x7ff2240ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.038+0000 7ff22ad28640 1 -- 172.21.15.139:0/252710235 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224162380 con 0x7ff2240cee50 2024-04-25T09:55:04.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.038+0000 7ff229525640 1 -- 172.21.15.139:0/252710235 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391328326 0 0) 0x7ff224161dc0 con 0x7ff2240b9370 2024-04-25T09:55:04.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.039+0000 7ff22ad28640 1 --2- 172.21.15.139:0/252710235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2240cee50 0x7ff2240d2370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009870 tx=0x7ff21c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c21e69f1ad048bea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.039+0000 7ff229525640 1 -- 172.21.15.139:0/252710235 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c037070 con 0x7ff2240cee50 2024-04-25T09:55:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.039+0000 7ff229525640 1 -- 172.21.15.139:0/252710235 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c032440 con 0x7ff2240cee50 2024-04-25T09:55:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.039+0000 7ff229525640 1 -- 172.21.15.139:0/252710235 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c032810 con 0x7ff2240cee50 2024-04-25T09:55:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.039+0000 7ff22c78b640 1 -- 172.21.15.139:0/252710235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2240cee50 msgr2=0x7ff2240d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.039+0000 7ff22c78b640 1 --2- 172.21.15.139:0/252710235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2240cee50 0x7ff2240d2370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009870 tx=0x7ff21c02a270 comp rx=0 tx=0).stop 2024-04-25T09:55:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.039+0000 7ff22c78b640 1 -- 172.21.15.139:0/252710235 shutdown_connections 2024-04-25T09:55:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.039+0000 7ff22c78b640 1 --2- 172.21.15.139:0/252710235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2240cee50 0x7ff2240d2370 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.039+0000 7ff22c78b640 1 --2- 172.21.15.139:0/252710235 >> v2:172.21.15.150:3300/0 conn(0x7ff22415dbf0 0x7ff2241617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.039+0000 7ff22c78b640 1 -- 172.21.15.139:0/252710235 >> 172.21.15.139:0/252710235 conn(0x7ff2240b5480 msgr2=0x7ff2240b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.040+0000 7ff22c78b640 1 -- 172.21.15.139:0/252710235 shutdown_connections 2024-04-25T09:55:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.040+0000 7ff22c78b640 1 -- 172.21.15.139:0/252710235 wait complete. 2024-04-25T09:55:04.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.041+0000 7ff22c78b640 1 Processor -- start 2024-04-25T09:55:04.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.041+0000 7ff22c78b640 1 -- start start 2024-04-25T09:55:04.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.041+0000 7ff22c78b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2240b9370 0x7ff22415f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.041+0000 7ff22c78b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2240cee50 0x7ff2240cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.041+0000 7ff22c78b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff22415dbf0 0x7ff2240cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.041+0000 7ff22c78b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22415dfd0 con 0x7ff22415dbf0 2024-04-25T09:55:04.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.041+0000 7ff22c78b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2240b97d0 con 0x7ff2240b9370 2024-04-25T09:55:04.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.041+0000 7ff22c78b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22415fff0 con 0x7ff2240cee50 2024-04-25T09:55:04.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.041+0000 7ff22ad28640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff22415dbf0 0x7ff2240cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff22ad28640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff22415dbf0 0x7ff2240cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47448/0 (socket says 172.21.15.139:47448) 2024-04-25T09:55:04.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff22ad28640 1 -- 172.21.15.139:0/2159349856 learned_addr learned my addr 172.21.15.139:0/2159349856 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:04.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff229d26640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2240cee50 0x7ff2240cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff22a527640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2240b9370 0x7ff22415f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff22ad28640 1 -- 172.21.15.139:0/2159349856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2240cee50 msgr2=0x7ff2240cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff22ad28640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2240cee50 0x7ff2240cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff22ad28640 1 -- 172.21.15.139:0/2159349856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2240b9370 msgr2=0x7ff22415f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff22ad28640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2240b9370 0x7ff22415f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff22ad28640 1 -- 172.21.15.139:0/2159349856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2241602d0 con 0x7ff22415dbf0 2024-04-25T09:55:04.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff22ad28640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff22415dbf0 0x7ff2240cc540 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009870 tx=0x7ff21c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff21affd640 1 -- 172.21.15.139:0/2159349856 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c037040 con 0x7ff22415dbf0 2024-04-25T09:55:04.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff21affd640 1 -- 172.21.15.139:0/2159349856 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c004030 con 0x7ff22415dbf0 2024-04-25T09:55:04.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff22c78b640 1 -- 172.21.15.139:0/2159349856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2241605b0 con 0x7ff22415dbf0 2024-04-25T09:55:04.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff21affd640 1 -- 172.21.15.139:0/2159349856 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c004420 con 0x7ff22415dbf0 2024-04-25T09:55:04.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.042+0000 7ff22c78b640 1 -- 172.21.15.139:0/2159349856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff224160b70 con 0x7ff22415dbf0 2024-04-25T09:55:04.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.044+0000 7ff21affd640 1 -- 172.21.15.139:0/2159349856 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff21c03a070 con 0x7ff22415dbf0 2024-04-25T09:55:04.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.044+0000 7ff21affd640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff20806b4d0 0x7ff20806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.044+0000 7ff21affd640 1 -- 172.21.15.139:0/2159349856 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff21c0aa7f0 con 0x7ff22415dbf0 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.044+0000 7ff22a527640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff20806b4d0 0x7ff20806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.044+0000 7ff22a527640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff20806b4d0 0x7ff20806d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff2200113e0 tx=0x7ff22000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.045+0000 7ff22f485740 1 Processor -- start 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.045+0000 7ff22f485740 1 -- start start 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff22f485740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542b10 0x564fdc545320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff22f485740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc545860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff22f485740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc515770 0x564fdc518f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff22f485740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fdc47eda0 con 0x564fdc542ef0 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff22f485740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fdc427b80 con 0x564fdc515770 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff22f485740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fdc47ae40 con 0x564fdc542b10 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff229d26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc545860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff229d26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc545860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47452/0 (socket says 172.21.15.139:47452) 2024-04-25T09:55:04.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff229d26640 1 -- 172.21.15.139:0/2235140961 learned_addr learned my addr 172.21.15.139:0/2235140961 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff22ad28640 1 --2- 172.21.15.139:0/2235140961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc515770 0x564fdc518f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff229d26640 1 -- 172.21.15.139:0/2235140961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542b10 msgr2=0x564fdc545320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff22a527640 1 --2- 172.21.15.139:0/2235140961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542b10 0x564fdc545320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff229d26640 1 --2- 172.21.15.139:0/2235140961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542b10 0x564fdc545320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff229d26640 1 -- 172.21.15.139:0/2235140961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc515770 msgr2=0x564fdc518f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff229d26640 1 --2- 172.21.15.139:0/2235140961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc515770 0x564fdc518f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff229d26640 1 -- 172.21.15.139:0/2235140961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fdc431f10 con 0x564fdc542ef0 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.046+0000 7ff22ad28640 1 --2- 172.21.15.139:0/2235140961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc515770 0x564fdc518f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff229d26640 1 --2- 172.21.15.139:0/2235140961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc545860 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff214000f60 tx=0x7ff214004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b5c507aa6125f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff207fff640 1 -- 172.21.15.139:0/2235140961 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21401a070 con 0x564fdc542ef0 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff207fff640 1 -- 172.21.15.139:0/2235140961 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff214001760 con 0x564fdc542ef0 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff207fff640 1 -- 172.21.15.139:0/2235140961 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214001b30 con 0x564fdc542ef0 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff22f485740 1 -- 172.21.15.139:0/2235140961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 msgr2=0x564fdc545860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff22f485740 1 --2- 172.21.15.139:0/2235140961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc545860 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff214000f60 tx=0x7ff214004bb0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff22f485740 1 -- 172.21.15.139:0/2235140961 shutdown_connections 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff22f485740 1 --2- 172.21.15.139:0/2235140961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc545860 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff22f485740 1 --2- 172.21.15.139:0/2235140961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc515770 0x564fdc518f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff22f485740 1 --2- 172.21.15.139:0/2235140961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542b10 0x564fdc545320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff22f485740 1 -- 172.21.15.139:0/2235140961 >> 172.21.15.139:0/2235140961 conn(0x564fdc5179e0 msgr2=0x564fdc560320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff22f485740 1 -- 172.21.15.139:0/2235140961 shutdown_connections 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.047+0000 7ff22f485740 1 -- 172.21.15.139:0/2235140961 wait complete. 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.048+0000 7ff22f485740 1 -- start start 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.048+0000 7ff22f485740 1 Processor -- start 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff22f485740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc542b10 0x564fdc52cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff22f485740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc52f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff22f485740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc52c5e0 0x564fdc52e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff22f485740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fdc51ab90 con 0x564fdc542ef0 2024-04-25T09:55:04.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff22f485740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fdc41d4b0 con 0x564fdc542b10 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff22f485740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fdc51ae70 con 0x564fdc52c5e0 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff22f485740 10 client.0 _ll_register_callbacks cb 0x564fdc45d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff229d26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc52f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff229d26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc52f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47464/0 (socket says 172.21.15.139:47464) 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff229d26640 1 -- 172.21.15.139:0/3170782269 learned_addr learned my addr 172.21.15.139:0/3170782269 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff22a527640 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc542b10 0x564fdc52cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff22ad28640 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc52c5e0 0x564fdc52e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff229d26640 1 -- 172.21.15.139:0/3170782269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc52c5e0 msgr2=0x564fdc52e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff229d26640 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc52c5e0 0x564fdc52e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff229d26640 1 -- 172.21.15.139:0/3170782269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc542b10 msgr2=0x564fdc52cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff229d26640 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc542b10 0x564fdc52cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff229d26640 1 -- 172.21.15.139:0/3170782269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564fdc25d7d0 con 0x564fdc542ef0 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff22a527640 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc542b10 0x564fdc52cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff229d26640 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc52f7e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x564fdc51d070 tx=0x7ff214000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2763f05186ddd7c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.049+0000 7ff2057fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff2057fa640 1 -- 172.21.15.139:0/3170782269 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214026060 con 0x564fdc542ef0 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff2057fa640 1 -- 172.21.15.139:0/3170782269 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff214014a00 con 0x564fdc542ef0 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff22f485740 1 -- 172.21.15.139:0/3170782269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff214001760 con 0x564fdc542ef0 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff2057fa640 1 -- 172.21.15.139:0/3170782269 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff21401a030 con 0x564fdc542ef0 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff22f485740 10 client.6075 fetch_fsmap learned FSMap version 22 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff22f485740 1 -- 172.21.15.139:0/3170782269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x564fdc1933f0 con 0x564fdc542ef0 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff2057fa640 1 -- 172.21.15.139:0/3170782269 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff214001760 con 0x564fdc542ef0 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff2057fa640 1 -- 172.21.15.139:0/3170782269 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff214014c00 con 0x564fdc542ef0 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff22f485740 10 client.6075 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff22f485740 10 client.6075 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff22f485740 20 client.6075 populate_metadata read hostname 'smithi139' 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff22f485740 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x564fdc55f700 0x564fdc563770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff22f485740 4 client.6075 mds_command: new command op to 5850 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff22f485740 1 -- 172.21.15.139:0/3170782269 --> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fdc48c2e0 con 0x564fdc55f700 2024-04-25T09:55:04.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff22a527640 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x564fdc55f700 0x564fdc563770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff2057fa640 10 client.6075 ms_handle_connect on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.050+0000 7ff22a527640 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x564fdc55f700 0x564fdc563770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb0c269a2947b9c0 server_cookie=fd397f039ea1990a in_seq=0 out_seq=0 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff2057fa640 1 -- 172.21.15.139:0/3170782269 <== mds.0 v2:172.21.15.139:6836/338275708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564fdc48c2e0 con 0x564fdc55f700 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff2057fa640 10 client.6075 handle_command_reply: tid=0 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff22f485740 1 client.6075 shutdown 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff22f485740 1 -- 172.21.15.139:0/3170782269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 msgr2=0x564fdc52f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff22f485740 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc52f7e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x564fdc51d070 tx=0x7ff214000ed0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff22f485740 1 -- 172.21.15.139:0/3170782269 shutdown_connections 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff22f485740 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x564fdc55f700 0x564fdc563770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff22f485740 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc52f7e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff22f485740 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc52c5e0 0x564fdc52e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff2057fa640 0 client.6075 ms_handle_reset on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff22f485740 1 --2- 172.21.15.139:0/3170782269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc542b10 0x564fdc52cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff22f485740 1 -- 172.21.15.139:0/3170782269 >> 172.21.15.139:0/3170782269 conn(0x564fdc515770 msgr2=0x564fdc55eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff22f485740 1 -- 172.21.15.139:0/3170782269 shutdown_connections 2024-04-25T09:55:04.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff22f485740 1 -- 172.21.15.139:0/3170782269 wait complete. 2024-04-25T09:55:04.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.051+0000 7ff22f485740 20 client.6075 trim_cache size 0 max 16384 2024-04-25T09:55:04.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.066+0000 7ff22f485740 1 Processor -- start 2024-04-25T09:55:04.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.066+0000 7ff22f485740 1 -- start start 2024-04-25T09:55:04.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22f485740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc52c5e0 0x564fdc568670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22f485740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542b10 0x564fdc55e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22f485740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc52ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22f485740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2242002c0 con 0x564fdc542ef0 2024-04-25T09:55:04.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22f485740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2241ff440 con 0x564fdc52c5e0 2024-04-25T09:55:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22f485740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2241f44f0 con 0x564fdc542b10 2024-04-25T09:55:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22ad28640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc52ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22ad28640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc52ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47480/0 (socket says 172.21.15.139:47480) 2024-04-25T09:55:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22ad28640 1 -- 172.21.15.139:0/3873348960 learned_addr learned my addr 172.21.15.139:0/3873348960 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff229d26640 1 --2- 172.21.15.139:0/3873348960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542b10 0x564fdc55e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22a527640 1 --2- 172.21.15.139:0/3873348960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc52c5e0 0x564fdc568670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22ad28640 1 -- 172.21.15.139:0/3873348960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542b10 msgr2=0x564fdc55e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22ad28640 1 --2- 172.21.15.139:0/3873348960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542b10 0x564fdc55e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22ad28640 1 -- 172.21.15.139:0/3873348960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc52c5e0 msgr2=0x564fdc568670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22ad28640 1 --2- 172.21.15.139:0/3873348960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc52c5e0 0x564fdc568670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22ad28640 1 -- 172.21.15.139:0/3873348960 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fdc1530f0 con 0x564fdc542ef0 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff229d26640 1 --2- 172.21.15.139:0/3873348960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542b10 0x564fdc55e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.067+0000 7ff22a527640 1 --2- 172.21.15.139:0/3873348960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc52c5e0 0x564fdc568670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff22ad28640 1 --2- 172.21.15.139:0/3873348960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc52ea20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff21c080fc0 tx=0x7ff21c07f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d112e68d1b8dd79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff2077fe640 1 -- 172.21.15.139:0/3873348960 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c04a040 con 0x564fdc542ef0 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff2077fe640 1 -- 172.21.15.139:0/3873348960 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c07c070 con 0x564fdc542ef0 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff2077fe640 1 -- 172.21.15.139:0/3873348960 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c07cf30 con 0x564fdc542ef0 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff22f485740 1 -- 172.21.15.139:0/3873348960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 msgr2=0x564fdc52ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff22f485740 1 --2- 172.21.15.139:0/3873348960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc52ea20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff21c080fc0 tx=0x7ff21c07f040 comp rx=0 tx=0).stop 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff22f485740 1 -- 172.21.15.139:0/3873348960 shutdown_connections 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff22f485740 1 --2- 172.21.15.139:0/3873348960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc542ef0 0x564fdc52ea20 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff22f485740 1 --2- 172.21.15.139:0/3873348960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542b10 0x564fdc55e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff22f485740 1 --2- 172.21.15.139:0/3873348960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc52c5e0 0x564fdc568670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff22f485740 1 -- 172.21.15.139:0/3873348960 >> 172.21.15.139:0/3873348960 conn(0x564fdc515770 msgr2=0x564fdc510e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff22f485740 1 -- 172.21.15.139:0/3873348960 shutdown_connections 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.068+0000 7ff22f485740 1 -- 172.21.15.139:0/3873348960 wait complete. 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.069+0000 7ff22f485740 1 -- start start 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.069+0000 7ff22f485740 1 Processor -- start 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22f485740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc52c5e0 0x564fdc555cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22f485740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc542b10 0x564fdc541c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22f485740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542ef0 0x564fdc514600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22f485740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2241fee40 con 0x564fdc52c5e0 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22f485740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fdc5567a0 con 0x564fdc542b10 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22f485740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fdc55e540 con 0x564fdc542ef0 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22f485740 10 client.0 _ll_register_callbacks cb 0x7ff2140070c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22a527640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc52c5e0 0x564fdc555cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22a527640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc52c5e0 0x564fdc555cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47496/0 (socket says 172.21.15.139:47496) 2024-04-25T09:55:04.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22a527640 1 -- 172.21.15.139:0/1299999882 learned_addr learned my addr 172.21.15.139:0/1299999882 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:04.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22ad28640 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542ef0 0x564fdc514600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff229d26640 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc542b10 0x564fdc541c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22a527640 1 -- 172.21.15.139:0/1299999882 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542ef0 msgr2=0x564fdc514600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22a527640 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542ef0 0x564fdc514600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22a527640 1 -- 172.21.15.139:0/1299999882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc542b10 msgr2=0x564fdc541c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22a527640 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc542b10 0x564fdc541c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22a527640 1 -- 172.21.15.139:0/1299999882 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564fdc55f030 con 0x564fdc52c5e0 2024-04-25T09:55:04.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff22a527640 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc52c5e0 0x564fdc555cb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x564fdc52d6e0 tx=0x7ff220004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7963e609359b2d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff2057fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:55:04.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.070+0000 7ff2057fa640 1 -- 172.21.15.139:0/1299999882 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22001e040 con 0x564fdc52c5e0 2024-04-25T09:55:04.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff2057fa640 1 -- 172.21.15.139:0/1299999882 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff220029430 con 0x564fdc52c5e0 2024-04-25T09:55:04.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff22f485740 1 -- 172.21.15.139:0/1299999882 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff21c07c070 con 0x564fdc52c5e0 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff2057fa640 1 -- 172.21.15.139:0/1299999882 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff220039050 con 0x564fdc52c5e0 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff22f485740 10 client.6081 fetch_fsmap learned FSMap version 22 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff22f485740 1 -- 172.21.15.139:0/1299999882 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x564fdc556540 con 0x564fdc52c5e0 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff2057fa640 1 -- 172.21.15.139:0/1299999882 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c07c070 con 0x564fdc52c5e0 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff2057fa640 1 -- 172.21.15.139:0/1299999882 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff220029630 con 0x564fdc52c5e0 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff22f485740 10 client.6081 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff22f485740 10 client.6081 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff22f485740 20 client.6081 populate_metadata read hostname 'smithi139' 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff22f485740 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x564fdc5c9b10 0x564fdc52ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff22f485740 4 client.6081 mds_command: new command op to 5850 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff22f485740 1 -- 172.21.15.139:0/1299999882 --> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564fdc486630 con 0x564fdc5c9b10 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff229d26640 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x564fdc5c9b10 0x564fdc52ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff2057fa640 10 client.6081 ms_handle_connect on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff229d26640 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x564fdc5c9b10 0x564fdc52ea20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21eb1e3e1e711e03 server_cookie=66380e793b9dcb12 in_seq=0 out_seq=0 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.071+0000 7ff2057fa640 1 -- 172.21.15.139:0/1299999882 <== mds.0 v2:172.21.15.139:6836/338275708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x564fdc486630 con 0x564fdc5c9b10 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff2057fa640 10 client.6081 handle_command_reply: tid=0 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22f485740 1 client.6081 shutdown 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22f485740 1 -- 172.21.15.139:0/1299999882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc52c5e0 msgr2=0x564fdc555cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22f485740 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc52c5e0 0x564fdc555cb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x564fdc52d6e0 tx=0x7ff220004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22f485740 1 -- 172.21.15.139:0/1299999882 shutdown_connections 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22f485740 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x564fdc5c9b10 0x564fdc52ea20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22f485740 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fdc542ef0 0x564fdc514600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22f485740 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fdc542b10 0x564fdc541c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22f485740 1 --2- 172.21.15.139:0/1299999882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564fdc52c5e0 0x564fdc555cb0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff2057fa640 0 client.6081 ms_handle_reset on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22f485740 1 -- 172.21.15.139:0/1299999882 >> 172.21.15.139:0/1299999882 conn(0x564fdc515770 msgr2=0x564fdc51c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22f485740 1 -- 172.21.15.139:0/1299999882 shutdown_connections 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22f485740 1 -- 172.21.15.139:0/1299999882 wait complete. 2024-04-25T09:55:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22f485740 20 client.6081 trim_cache size 0 max 16384 2024-04-25T09:55:04.084 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:55:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:55:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": { 2024-04-25T09:55:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "6041bcc3-3169-4103-88f5-5b90a0e6c62b": { 2024-04-25T09:55:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/", 2024-04-25T09:55:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "tag": "6041bcc3-3169-4103-88f5-5b90a0e6c62b", 2024-04-25T09:55:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: "options": "recursive,repair" 2024-04-25T09:55:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:04.084 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:04.085 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:55:04.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22c78b640 1 -- 172.21.15.139:0/2159349856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff20806b4d0 msgr2=0x7ff20806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22c78b640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff20806b4d0 0x7ff20806d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff2200113e0 tx=0x7ff22000b040 comp rx=0 tx=0).stop 2024-04-25T09:55:04.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22c78b640 1 -- 172.21.15.139:0/2159349856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff22415dbf0 msgr2=0x7ff2240cc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22c78b640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff22415dbf0 0x7ff2240cc540 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009870 tx=0x7ff21c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22c78b640 1 -- 172.21.15.139:0/2159349856 shutdown_connections 2024-04-25T09:55:04.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22c78b640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff22415dbf0 0x7ff2240cc540 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22c78b640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff20806b4d0 0x7ff20806d990 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22c78b640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2240cee50 0x7ff2240cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22c78b640 1 --2- 172.21.15.139:0/2159349856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2240b9370 0x7ff22415f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22c78b640 1 -- 172.21.15.139:0/2159349856 >> 172.21.15.139:0/2159349856 conn(0x7ff2240b5480 msgr2=0x7ff224158150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.072+0000 7ff22c78b640 1 -- 172.21.15.139:0/2159349856 shutdown_connections 2024-04-25T09:55:04.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.073+0000 7ff22c78b640 1 -- 172.21.15.139:0/2159349856 wait complete. 2024-04-25T09:55:04.087 INFO:tasks.cephfs.filesystem:scrub status for tag:6041bcc3-3169-4103-88f5-5b90a0e6c62b - {'path': '/', 'tag': '6041bcc3-3169-4103-88f5-5b90a0e6c62b', 'options': 'recursive,repair'} 2024-04-25T09:55:04.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:04.815+0000 7f6490ff9640 1 -- 172.21.15.139:0/1349623033 <== mon.0 v2:172.21.15.139:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-25T09:55:03.928047+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f648c088f10 con 0x7f649415f1a0 2024-04-25T09:55:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: cluster: 2024-04-25T09:55:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: id: 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T09:55:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: health: HEALTH_OK 2024-04-25T09:55:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: services: 2024-04-25T09:55:04.819 INFO:teuthology.orchestra.run.smithi139.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: mgr: y(active, since 2m), standbys: x 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: data: 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: volumes: 1/1 healthy 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: pools: 3 pools, 97 pgs 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: objects: 23 objects, 579 KiB 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: usage: 217 MiB used, 720 GiB / 720 GiB avail 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: pgs: 97 active+clean 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: io: 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: client: 2.0 KiB/s wr, 0 op/s rd, 6 op/s wr 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:04.820 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:04.821 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T09:55:03.928047+0000 mds.c [INF] scrub queued for path: / 2024-04-25T09:55:04.821 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T09:55:03.928051+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T09:55:04.821 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T09:55:03.928055+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-25T09:55:04.821 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T09:55:03.928893+0000 mds.c [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-25T09:55:04.821 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T09:55:03.928901+0000 mds.c [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-25T09:55:04.821 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T09:55:03.929499+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T09:55:08.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:08.822+0000 7f6490ff9640 1 -- 172.21.15.139:0/1349623033 <== mon.0 v2:172.21.15.139:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-25T09:55:07.046514+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f648c08b000 con 0x7f649415f1a0 2024-04-25T09:55:08.824 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T09:55:07.046514+0000 mds.c [INF] scrub summary: idle 2024-04-25T09:55:15.422 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:55:15.425 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:55:34.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:34.089 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:55:34.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.196+0000 7f4e25363640 1 Processor -- start 2024-04-25T09:55:34.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.196+0000 7f4e25363640 1 -- start start 2024-04-25T09:55:34.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.197+0000 7f4e25363640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e200afc00 0x7f4e2006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.197+0000 7f4e25363640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4e2015cb20 0x7f4e200cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.197+0000 7f4e25363640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e200ccef0 con 0x7f4e200cfeb0 2024-04-25T09:55:34.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.197+0000 7f4e25363640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e200d0320 con 0x7f4e2015cb20 2024-04-25T09:55:34.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.197+0000 7f4e25363640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200703c0 con 0x7f4e200afc00 2024-04-25T09:55:34.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.197+0000 7f4e167fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4e2015cb20 0x7f4e200cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.197+0000 7f4e1f7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4e200cfeb0 0x7f4e2006ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48808/0 (socket says 172.21.15.139:48808) 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.198+0000 7f4e1f7fe640 1 -- 172.21.15.139:0/1845203447 learned_addr learned my addr 172.21.15.139:0/1845203447 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.198+0000 7f4e1effd640 1 --2- 172.21.15.139:0/1845203447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e200afc00 0x7f4e2006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.198+0000 7f4e167fc640 1 -- 172.21.15.139:0/1845203447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e200afc00 msgr2=0x7f4e2006fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.198+0000 7f4e1e7fc640 1 -- 172.21.15.139:0/1845203447 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566896489 0 0) 0x7f4e200ccef0 con 0x7f4e200cfeb0 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.198+0000 7f4e167fc640 1 --2- 172.21.15.139:0/1845203447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e200afc00 0x7f4e2006fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.198+0000 7f4e167fc640 1 -- 172.21.15.139:0/1845203447 >> v1:172.21.15.139:6789/0 conn(0x7f4e200cfeb0 legacy=0x7f4e2006ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.198+0000 7f4e167fc640 1 -- 172.21.15.139:0/1845203447 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e200cd1d0 con 0x7f4e2015cb20 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.198+0000 7f4e1effd640 1 --2- 172.21.15.139:0/1845203447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e200afc00 0x7f4e2006fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.199+0000 7f4e167fc640 1 --2- 172.21.15.139:0/1845203447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e2015cb20 0x7f4e200cf970 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4e080097a0 tx=0x7f4e0802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b07aa5bbc66bd29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.199+0000 7f4e1e7fc640 1 -- 172.21.15.139:0/1845203447 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08037070 con 0x7f4e2015cb20 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.199+0000 7f4e1e7fc640 1 -- 172.21.15.139:0/1845203447 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0802ae50 con 0x7f4e2015cb20 2024-04-25T09:55:34.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.199+0000 7f4e1e7fc640 1 -- 172.21.15.139:0/1845203447 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08032660 con 0x7f4e2015cb20 2024-04-25T09:55:34.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.199+0000 7f4e25363640 1 -- 172.21.15.139:0/1845203447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e2015cb20 msgr2=0x7f4e200cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.199+0000 7f4e25363640 1 --2- 172.21.15.139:0/1845203447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e2015cb20 0x7f4e200cf970 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4e080097a0 tx=0x7f4e0802a100 comp rx=0 tx=0).stop 2024-04-25T09:55:34.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.199+0000 7f4e25363640 1 -- 172.21.15.139:0/1845203447 shutdown_connections 2024-04-25T09:55:34.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.199+0000 7f4e25363640 1 --2- 172.21.15.139:0/1845203447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e2015cb20 0x7f4e200cf970 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.199+0000 7f4e25363640 1 --2- 172.21.15.139:0/1845203447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e200afc00 0x7f4e2006fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.199+0000 7f4e25363640 1 -- 172.21.15.139:0/1845203447 >> 172.21.15.139:0/1845203447 conn(0x7f4e200b2230 msgr2=0x7f4e200b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.199+0000 7f4e25363640 1 -- 172.21.15.139:0/1845203447 shutdown_connections 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.199+0000 7f4e25363640 1 -- 172.21.15.139:0/1845203447 wait complete. 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.200+0000 7f4e25363640 1 Processor -- start 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.201+0000 7f4e25363640 1 -- start start 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.201+0000 7f4e25363640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e200afc00 0x7f4e20152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.201+0000 7f4e25363640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e200cfeb0 0x7f4e200c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.201+0000 7f4e25363640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2015cb20 0x7f4e200c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.201+0000 7f4e25363640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200affe0 con 0x7f4e200cfeb0 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.201+0000 7f4e25363640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200cd820 con 0x7f4e2015cb20 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.201+0000 7f4e25363640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e20070060 con 0x7f4e200afc00 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e167fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e200cfeb0 0x7f4e200c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e167fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e200cfeb0 0x7f4e200c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46520/0 (socket says 172.21.15.139:46520) 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e167fc640 1 -- 172.21.15.139:0/1089518121 learned_addr learned my addr 172.21.15.139:0/1089518121 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e1f7fe640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2015cb20 0x7f4e200c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e1effd640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e200afc00 0x7f4e20152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e167fc640 1 -- 172.21.15.139:0/1089518121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e200afc00 msgr2=0x7f4e20152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e167fc640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e200afc00 0x7f4e20152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e167fc640 1 -- 172.21.15.139:0/1089518121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2015cb20 msgr2=0x7f4e200c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e167fc640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2015cb20 0x7f4e200c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e167fc640 1 -- 172.21.15.139:0/1089518121 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2015b2b0 con 0x7f4e200cfeb0 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e1f7fe640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2015cb20 0x7f4e200c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e167fc640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e200cfeb0 0x7f4e200c5ac0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4e080097a0 tx=0x7f4e080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e17fff640 1 -- 172.21.15.139:0/1089518121 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08037040 con 0x7f4e200cfeb0 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e17fff640 1 -- 172.21.15.139:0/1089518121 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e08004250 con 0x7f4e200cfeb0 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e25363640 1 -- 172.21.15.139:0/1089518121 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2015d260 con 0x7f4e200cfeb0 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e17fff640 1 -- 172.21.15.139:0/1089518121 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08003610 con 0x7f4e200cfeb0 2024-04-25T09:55:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.202+0000 7f4e25363640 1 -- 172.21.15.139:0/1089518121 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e200b3a80 con 0x7f4e200cfeb0 2024-04-25T09:55:34.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.204+0000 7f4e17fff640 1 -- 172.21.15.139:0/1089518121 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e080037b0 con 0x7f4e200cfeb0 2024-04-25T09:55:34.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.204+0000 7f4e17fff640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4e0006b500 0x7f4e0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.204+0000 7f4e17fff640 1 -- 172.21.15.139:0/1089518121 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4e080ab240 con 0x7f4e200cfeb0 2024-04-25T09:55:34.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.204+0000 7f4e1effd640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4e0006b500 0x7f4e0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.205+0000 7f4e1effd640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4e0006b500 0x7f4e0006d9c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000a950 tx=0x7f4e10008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.205+0000 7f4e2806f740 1 Processor -- start 2024-04-25T09:55:34.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.205+0000 7f4e2806f740 1 -- start start 2024-04-25T09:55:34.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e2806f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 0x55ded56e8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e2806f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56e5ef0 0x55ded56e8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e2806f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56b8770 0x55ded56bbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e2806f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded5621da0 con 0x55ded56e5b10 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e2806f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded55cab80 con 0x55ded56e5ef0 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e2806f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ded561de40 con 0x55ded56b8770 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e1effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 0x55ded56e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e1effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 0x55ded56e8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46524/0 (socket says 172.21.15.139:46524) 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e1effd640 1 -- 172.21.15.139:0/2010030786 learned_addr learned my addr 172.21.15.139:0/2010030786 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e1effd640 1 -- 172.21.15.139:0/2010030786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56b8770 msgr2=0x55ded56bbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e1effd640 1 --2- 172.21.15.139:0/2010030786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56b8770 0x55ded56bbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e167fc640 1 --2- 172.21.15.139:0/2010030786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56e5ef0 0x55ded56e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e1effd640 1 -- 172.21.15.139:0/2010030786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56e5ef0 msgr2=0x55ded56e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e1effd640 1 --2- 172.21.15.139:0/2010030786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56e5ef0 0x55ded56e8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.206+0000 7f4e1effd640 1 -- 172.21.15.139:0/2010030786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ded55d4f10 con 0x55ded56e5b10 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.207+0000 7f4e1effd640 1 --2- 172.21.15.139:0/2010030786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 0x55ded56e8320 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000a160 tx=0x7f4e10011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0235addc1bb4429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.207+0000 7f4e14ff9640 1 -- 172.21.15.139:0/2010030786 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1001a050 con 0x55ded56e5b10 2024-04-25T09:55:34.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.207+0000 7f4e14ff9640 1 -- 172.21.15.139:0/2010030786 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e10024070 con 0x55ded56e5b10 2024-04-25T09:55:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.207+0000 7f4e14ff9640 1 -- 172.21.15.139:0/2010030786 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1001f440 con 0x55ded56e5b10 2024-04-25T09:55:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.207+0000 7f4e2806f740 1 -- 172.21.15.139:0/2010030786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 msgr2=0x55ded56e8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.207+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2010030786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 0x55ded56e8320 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000a160 tx=0x7f4e10011000 comp rx=0 tx=0).stop 2024-04-25T09:55:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.208+0000 7f4e2806f740 1 -- 172.21.15.139:0/2010030786 shutdown_connections 2024-04-25T09:55:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.208+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2010030786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56b8770 0x55ded56bbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.208+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2010030786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56e5ef0 0x55ded56e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.208+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2010030786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 0x55ded56e8320 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.208+0000 7f4e2806f740 1 -- 172.21.15.139:0/2010030786 >> 172.21.15.139:0/2010030786 conn(0x55ded56ba9e0 msgr2=0x55ded5703320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.208+0000 7f4e2806f740 1 -- 172.21.15.139:0/2010030786 shutdown_connections 2024-04-25T09:55:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.208+0000 7f4e2806f740 1 -- 172.21.15.139:0/2010030786 wait complete. 2024-04-25T09:55:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.208+0000 7f4e2806f740 1 -- start start 2024-04-25T09:55:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.208+0000 7f4e2806f740 1 Processor -- start 2024-04-25T09:55:34.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e2806f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5b10 0x55ded56dc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e2806f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5ef0 0x55ded56ebb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e2806f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56ec090 0x55ded56e9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e2806f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded56bdb90 con 0x55ded56e5ef0 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e2806f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded55c04b0 con 0x55ded56ec090 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e2806f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ded56bde70 con 0x55ded56e5b10 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e2806f740 10 client.0 _ll_register_callbacks cb 0x55ded5600180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e167fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5ef0 0x55ded56ebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e167fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5ef0 0x55ded56ebb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46526/0 (socket says 172.21.15.139:46526) 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e167fc640 1 -- 172.21.15.139:0/2504514063 learned_addr learned my addr 172.21.15.139:0/2504514063 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e1effd640 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5b10 0x55ded56dc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e1f7fe640 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56ec090 0x55ded56e9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e167fc640 1 -- 172.21.15.139:0/2504514063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5b10 msgr2=0x55ded56dc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e167fc640 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5b10 0x55ded56dc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e167fc640 1 -- 172.21.15.139:0/2504514063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56ec090 msgr2=0x55ded56e9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e167fc640 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56ec090 0x55ded56e9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e167fc640 1 -- 172.21.15.139:0/2504514063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ded54007d0 con 0x55ded56e5ef0 2024-04-25T09:55:34.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e1effd640 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5b10 0x55ded56dc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:34.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.209+0000 7f4e1f7fe640 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56ec090 0x55ded56e9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:34.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4e167fc640 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5ef0 0x55ded56ebb50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4e08075db0 tx=0x7f4e0807a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80df2a0b730817e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4dfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:55:34.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4dfdffb640 1 -- 172.21.15.139:0/2504514063 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08091020 con 0x55ded56e5ef0 2024-04-25T09:55:34.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4dfdffb640 1 -- 172.21.15.139:0/2504514063 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4e08079db0 con 0x55ded56e5ef0 2024-04-25T09:55:34.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4e2806f740 1 -- 172.21.15.139:0/2504514063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e10024070 con 0x55ded56e5ef0 2024-04-25T09:55:34.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4dfdffb640 1 -- 172.21.15.139:0/2504514063 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e0807daa0 con 0x55ded56e5ef0 2024-04-25T09:55:34.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4e2806f740 10 client.6096 fetch_fsmap learned FSMap version 22 2024-04-25T09:55:34.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4e2806f740 1 -- 172.21.15.139:0/2504514063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55ded53363f0 con 0x55ded56e5ef0 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4dfdffb640 1 -- 172.21.15.139:0/2504514063 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e10024070 con 0x55ded56e5ef0 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4dfdffb640 1 -- 172.21.15.139:0/2504514063 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e0804f060 con 0x55ded56e5ef0 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4e2806f740 10 client.6096 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4e2806f740 10 client.6096 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4e2806f740 20 client.6096 populate_metadata read hostname 'smithi139' 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55ded5704920 0x55ded5704d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4e2806f740 4 client.6096 mds_command: new command op to 5850 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4e2806f740 1 -- 172.21.15.139:0/2504514063 --> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ded562f2e0 con 0x55ded5704920 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.210+0000 7f4e1effd640 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55ded5704920 0x55ded5704d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4dfdffb640 10 client.6096 ms_handle_connect on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e1effd640 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55ded5704920 0x55ded5704d00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98430c7952c62f59 server_cookie=46c479b77f1564bb in_seq=0 out_seq=0 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4dfdffb640 1 -- 172.21.15.139:0/2504514063 <== mds.0 v2:172.21.15.139:6836/338275708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ded562f2e0 con 0x55ded5704920 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4dfdffb640 10 client.6096 handle_command_reply: tid=0 2024-04-25T09:55:34.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e2806f740 1 client.6096 shutdown 2024-04-25T09:55:34.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e2806f740 1 -- 172.21.15.139:0/2504514063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5ef0 msgr2=0x55ded56ebb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5ef0 0x55ded56ebb50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4e08075db0 tx=0x7f4e0807a050 comp rx=0 tx=0).stop 2024-04-25T09:55:34.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e2806f740 1 -- 172.21.15.139:0/2504514063 shutdown_connections 2024-04-25T09:55:34.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55ded5704920 0x55ded5704d00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5ef0 0x55ded56ebb50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56ec090 0x55ded56e9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4dfdffb640 0 client.6096 ms_handle_reset on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:34.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2504514063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5b10 0x55ded56dc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e2806f740 1 -- 172.21.15.139:0/2504514063 >> 172.21.15.139:0/2504514063 conn(0x55ded56b8770 msgr2=0x55ded56b9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:34.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e2806f740 1 -- 172.21.15.139:0/2504514063 shutdown_connections 2024-04-25T09:55:34.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e2806f740 1 -- 172.21.15.139:0/2504514063 wait complete. 2024-04-25T09:55:34.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.211+0000 7f4e2806f740 20 client.6096 trim_cache size 0 max 16384 2024-04-25T09:55:34.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.226+0000 7f4e2806f740 1 Processor -- start 2024-04-25T09:55:34.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.226+0000 7f4e2806f740 1 -- start start 2024-04-25T09:55:34.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e2806f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 0x55ded56dc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e2806f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5ef0 0x55ded56e9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e2806f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56ec090 0x55ded56e9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e2806f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e202002a0 con 0x55ded56e5b10 2024-04-25T09:55:34.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e2806f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e201ff4a0 con 0x55ded56ec090 2024-04-25T09:55:34.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e2806f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2015e7d0 con 0x55ded56e5ef0 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e1effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 0x55ded56dc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e1effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 0x55ded56dc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46530/0 (socket says 172.21.15.139:46530) 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e1effd640 1 -- 172.21.15.139:0/4207035418 learned_addr learned my addr 172.21.15.139:0/4207035418 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e167fc640 1 --2- 172.21.15.139:0/4207035418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5ef0 0x55ded56e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e1f7fe640 1 --2- 172.21.15.139:0/4207035418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56ec090 0x55ded56e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e1effd640 1 -- 172.21.15.139:0/4207035418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5ef0 msgr2=0x55ded56e9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e1effd640 1 --2- 172.21.15.139:0/4207035418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5ef0 0x55ded56e9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e1effd640 1 -- 172.21.15.139:0/4207035418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56ec090 msgr2=0x55ded56e9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e1effd640 1 --2- 172.21.15.139:0/4207035418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56ec090 0x55ded56e9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e1effd640 1 -- 172.21.15.139:0/4207035418 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ded5705070 con 0x55ded56e5b10 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4e1effd640 1 --2- 172.21.15.139:0/4207035418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 0x55ded56dc950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4e10021040 tx=0x7f4e10013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54905c125112a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.227+0000 7f4dfffff640 1 -- 172.21.15.139:0/4207035418 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10037070 con 0x55ded56e5b10 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.228+0000 7f4dfffff640 1 -- 172.21.15.139:0/4207035418 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e10024040 con 0x55ded56e5b10 2024-04-25T09:55:34.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.228+0000 7f4dfffff640 1 -- 172.21.15.139:0/4207035418 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1001a040 con 0x55ded56e5b10 2024-04-25T09:55:34.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.228+0000 7f4e2806f740 1 -- 172.21.15.139:0/4207035418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 msgr2=0x55ded56dc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.228+0000 7f4e2806f740 1 --2- 172.21.15.139:0/4207035418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 0x55ded56dc950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4e10021040 tx=0x7f4e10013000 comp rx=0 tx=0).stop 2024-04-25T09:55:34.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.228+0000 7f4e2806f740 1 -- 172.21.15.139:0/4207035418 shutdown_connections 2024-04-25T09:55:34.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.228+0000 7f4e2806f740 1 --2- 172.21.15.139:0/4207035418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56ec090 0x55ded56e9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.228+0000 7f4e2806f740 1 --2- 172.21.15.139:0/4207035418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5ef0 0x55ded56e9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.228+0000 7f4e2806f740 1 --2- 172.21.15.139:0/4207035418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56e5b10 0x55ded56dc950 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.228+0000 7f4e2806f740 1 -- 172.21.15.139:0/4207035418 >> 172.21.15.139:0/4207035418 conn(0x55ded56b8770 msgr2=0x55ded56b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:34.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.228+0000 7f4e2806f740 1 -- 172.21.15.139:0/4207035418 shutdown_connections 2024-04-25T09:55:34.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.228+0000 7f4e2806f740 1 -- 172.21.15.139:0/4207035418 wait complete. 2024-04-25T09:55:34.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.229+0000 7f4e2806f740 1 -- start start 2024-04-25T09:55:34.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.229+0000 7f4e2806f740 1 Processor -- start 2024-04-25T09:55:34.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.229+0000 7f4e2806f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56e5b10 0x55ded5703320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.229+0000 7f4e2806f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5ef0 0x55ded5703860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.229+0000 7f4e2806f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56ec090 0x55ded56fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.229+0000 7f4e2806f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded56e9f70 con 0x55ded56ec090 2024-04-25T09:55:34.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.229+0000 7f4e2806f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e201fef00 con 0x55ded56e5b10 2024-04-25T09:55:34.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.229+0000 7f4e2806f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ded56ebc50 con 0x55ded56e5ef0 2024-04-25T09:55:34.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.229+0000 7f4e2806f740 10 client.0 _ll_register_callbacks cb 0x7f4e1000a1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:34.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.229+0000 7f4e1f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56ec090 0x55ded56fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e1f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56ec090 0x55ded56fc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46544/0 (socket says 172.21.15.139:46544) 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e1f7fe640 1 -- 172.21.15.139:0/2411773259 learned_addr learned my addr 172.21.15.139:0/2411773259 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e1effd640 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56e5b10 0x55ded5703320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e167fc640 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5ef0 0x55ded5703860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e1f7fe640 1 -- 172.21.15.139:0/2411773259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5ef0 msgr2=0x55ded5703860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e1f7fe640 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5ef0 0x55ded5703860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e1f7fe640 1 -- 172.21.15.139:0/2411773259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56e5b10 msgr2=0x55ded5703320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e1f7fe640 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56e5b10 0x55ded5703320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e1f7fe640 1 -- 172.21.15.139:0/2411773259 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ded52f60f0 con 0x55ded56ec090 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e1f7fe640 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56ec090 0x55ded56fc3a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4e18000f80 tx=0x7f4e180074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be68b1fc9d5abaa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4dfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4dfdffb640 1 -- 172.21.15.139:0/2411773259 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18010040 con 0x55ded56ec090 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4dfdffb640 1 -- 172.21.15.139:0/2411773259 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4e18015610 con 0x55ded56ec090 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e2806f740 1 -- 172.21.15.139:0/2411773259 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e10024040 con 0x55ded56ec090 2024-04-25T09:55:34.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4dfdffb640 1 -- 172.21.15.139:0/2411773259 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e18022050 con 0x55ded56ec090 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e2806f740 10 client.6102 fetch_fsmap learned FSMap version 22 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4e2806f740 1 -- 172.21.15.139:0/2411773259 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55ded56bd8e0 con 0x55ded56ec090 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.230+0000 7f4dfdffb640 1 -- 172.21.15.139:0/2411773259 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e10024040 con 0x55ded56ec090 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4dfdffb640 1 -- 172.21.15.139:0/2411773259 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e18025020 con 0x55ded56ec090 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 10 client.6102 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 10 client.6102 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 20 client.6102 populate_metadata read hostname 'smithi139' 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55ded570b5b0 0x55ded56fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 4 client.6102 mds_command: new command op to 5850 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 -- 172.21.15.139:0/2411773259 --> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ded5629630 con 0x55ded570b5b0 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e1effd640 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55ded570b5b0 0x55ded56fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e1effd640 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55ded570b5b0 0x55ded56fc8e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4df8bdfcd3213c1 server_cookie=76737fe757db450 in_seq=0 out_seq=0 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4dfdffb640 10 client.6102 ms_handle_connect on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:34.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4dfdffb640 1 -- 172.21.15.139:0/2411773259 <== mds.0 v2:172.21.15.139:6836/338275708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ded5629630 con 0x55ded570b5b0 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4dfdffb640 10 client.6102 handle_command_reply: tid=0 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 client.6102 shutdown 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 -- 172.21.15.139:0/2411773259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56ec090 msgr2=0x55ded56fc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56ec090 0x55ded56fc3a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4e18000f80 tx=0x7f4e180074e0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 -- 172.21.15.139:0/2411773259 shutdown_connections 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.139:6836/338275708,v1:172.21.15.139:6837/338275708] conn(0x55ded570b5b0 0x55ded56fc8e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ded56ec090 0x55ded56fc3a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ded56e5ef0 0x55ded5703860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4dfdffb640 0 client.6102 ms_handle_reset on v2:172.21.15.139:6836/338275708 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 --2- 172.21.15.139:0/2411773259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ded56e5b10 0x55ded5703320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 -- 172.21.15.139:0/2411773259 >> 172.21.15.139:0/2411773259 conn(0x55ded56b8770 msgr2=0x55ded56e82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 -- 172.21.15.139:0/2411773259 shutdown_connections 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 1 -- 172.21.15.139:0/2411773259 wait complete. 2024-04-25T09:55:34.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.231+0000 7f4e2806f740 20 client.6102 trim_cache size 0 max 16384 2024-04-25T09:55:34.241 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:55:34.241 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "no active scrubs running", 2024-04-25T09:55:34.241 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": {} 2024-04-25T09:55:34.241 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:55:34.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.232+0000 7f4e25363640 1 -- 172.21.15.139:0/1089518121 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4e0006b500 msgr2=0x7f4e0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.232+0000 7f4e25363640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4e0006b500 0x7f4e0006d9c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000a950 tx=0x7f4e10008040 comp rx=0 tx=0).stop 2024-04-25T09:55:34.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.232+0000 7f4e25363640 1 -- 172.21.15.139:0/1089518121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e200cfeb0 msgr2=0x7f4e200c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.232+0000 7f4e25363640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e200cfeb0 0x7f4e200c5ac0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4e080097a0 tx=0x7f4e080046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.232+0000 7f4e25363640 1 -- 172.21.15.139:0/1089518121 shutdown_connections 2024-04-25T09:55:34.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.232+0000 7f4e25363640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e200cfeb0 0x7f4e200c5ac0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.232+0000 7f4e25363640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4e0006b500 0x7f4e0006d9c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.232+0000 7f4e25363640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e2015cb20 0x7f4e200c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.232+0000 7f4e25363640 1 --2- 172.21.15.139:0/1089518121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e200afc00 0x7f4e20152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.232+0000 7f4e25363640 1 -- 172.21.15.139:0/1089518121 >> 172.21.15.139:0/1089518121 conn(0x7f4e200b2230 msgr2=0x7f4e200cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:34.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.232+0000 7f4e25363640 1 -- 172.21.15.139:0/1089518121 shutdown_connections 2024-04-25T09:55:34.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.232+0000 7f4e25363640 1 -- 172.21.15.139:0/1089518121 wait complete. 2024-04-25T09:55:34.245 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:55:34.347 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:34.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.441+0000 7fd24f3c8640 1 Processor -- start 2024-04-25T09:55:34.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.441+0000 7fd24f3c8640 1 -- start start 2024-04-25T09:55:34.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd24f3c8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd24806c780 0x7fd2480d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd24f3c8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2480cf110 con 0x7fd2480b6af0 2024-04-25T09:55:34.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd24f3c8640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2480b6f50 con 0x7fd2481611e0 2024-04-25T09:55:34.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd24f3c8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480d1fa0 con 0x7fd24806c780 2024-04-25T09:55:34.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd24c963640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd2480b6af0 0x7fd24806c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48820/0 (socket says 172.21.15.139:48820) 2024-04-25T09:55:34.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd24c963640 1 -- 172.21.15.139:0/1228032668 learned_addr learned my addr 172.21.15.139:0/1228032668 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:34.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd24d965640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd24806c780 0x7fd2480d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259264733 0 0) 0x7fd2480cf110 con 0x7fd2480b6af0 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd234003520 con 0x7fd2480b6af0 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175375098 0 0) 0x7fd2480b6f50 con 0x7fd2481611e0 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2480cf110 con 0x7fd2481611e0 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 160366590 0 0) 0x7fd234003520 con 0x7fd2480b6af0 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd24806c780 msgr2=0x7fd2480d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd24806c780 0x7fd2480d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 >> v1:172.21.15.150:6789/0 conn(0x7fd2481611e0 legacy=0x7fd2480cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2480d4740 con 0x7fd2480b6af0 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fd238001670 con 0x7fd2480b6af0 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 >> v1:172.21.15.139:6789/0 conn(0x7fd2480b6af0 legacy=0x7fd24806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd234003b70 0x7fd234003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd234004800 0x7fd234004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2340051c0 0x7fd23400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480d1fa0 con 0x7fd234003b70 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23400bfe0 con 0x7fd2340051c0 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23400c1e0 con 0x7fd234004800 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd24d965640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd24806c780 0x7fd2480d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.442+0000 7fd24d164640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd234003b70 0x7fd234003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.443+0000 7fd24c963640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd234004800 0x7fd234004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.443+0000 7fd24d965640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2340051c0 0x7fd23400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.443+0000 7fd24d164640 1 -- 172.21.15.139:0/1228032668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd234004800 msgr2=0x7fd234004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.443+0000 7fd24d164640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd234004800 0x7fd234004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.443+0000 7fd24d164640 1 -- 172.21.15.139:0/1228032668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2340051c0 msgr2=0x7fd23400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.443+0000 7fd24d164640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2340051c0 0x7fd23400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.443+0000 7fd24d164640 1 -- 172.21.15.139:0/1228032668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd234003520 con 0x7fd234003b70 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.443+0000 7fd24d164640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd234003b70 0x7fd234003f70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd2440018c0 tx=0x7fd234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8221b2eadeb163 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.443+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2440014e0 con 0x7fd234003b70 2024-04-25T09:55:34.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.444+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2440052f0 con 0x7fd234003b70 2024-04-25T09:55:34.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.444+0000 7fd23ffff640 1 -- 172.21.15.139:0/1228032668 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24402ac90 con 0x7fd234003b70 2024-04-25T09:55:34.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.444+0000 7fd24f3c8640 1 -- 172.21.15.139:0/1228032668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd234003b70 msgr2=0x7fd234003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.444+0000 7fd24f3c8640 1 --2- 172.21.15.139:0/1228032668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd234003b70 0x7fd234003f70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd2440018c0 tx=0x7fd234005830 comp rx=0 tx=0).stop 2024-04-25T09:55:34.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.444+0000 7fd24d965640 1 -- 172.21.15.139:0/1228032668 reap_dead start 2024-04-25T09:55:34.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.444+0000 7fd24f3c8640 1 -- 172.21.15.139:0/1228032668 shutdown_connections 2024-04-25T09:55:34.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.444+0000 7fd24f3c8640 1 -- 172.21.15.139:0/1228032668 >> 172.21.15.139:0/1228032668 conn(0x7fd2480b5b00 msgr2=0x7fd2480d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:34.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.444+0000 7fd24f3c8640 1 -- 172.21.15.139:0/1228032668 shutdown_connections 2024-04-25T09:55:34.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.444+0000 7fd24f3c8640 1 -- 172.21.15.139:0/1228032668 wait complete. 2024-04-25T09:55:34.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.445+0000 7fd24f3c8640 1 Processor -- start 2024-04-25T09:55:34.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.445+0000 7fd24f3c8640 1 -- start start 2024-04-25T09:55:34.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24f3c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd234003b70 0x7fd24815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24f3c8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2481600d0 0x7fd248160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24f3c8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2481f3a30 0x7fd2481f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24f3c8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480b2a50 con 0x7fd234003b70 2024-04-25T09:55:34.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24f3c8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480d2560 con 0x7fd2481f3a30 2024-04-25T09:55:34.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24f3c8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480b2d30 con 0x7fd2481600d0 2024-04-25T09:55:34.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24d164640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd234003b70 0x7fd24815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24d164640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd234003b70 0x7fd24815a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46558/0 (socket says 172.21.15.139:46558) 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24d164640 1 -- 172.21.15.139:0/261597810 learned_addr learned my addr 172.21.15.139:0/261597810 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24d965640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2481f3a30 0x7fd2481f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24d164640 1 -- 172.21.15.139:0/261597810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2481600d0 msgr2=0x7fd248160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24c963640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2481600d0 0x7fd248160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24d164640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2481600d0 0x7fd248160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24d164640 1 -- 172.21.15.139:0/261597810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2481f3a30 msgr2=0x7fd2481f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24d164640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2481f3a30 0x7fd2481f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24d164640 1 -- 172.21.15.139:0/261597810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2481f4790 con 0x7fd234003b70 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.446+0000 7fd24d965640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2481f3a30 0x7fd2481f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.447+0000 7fd24d164640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd234003b70 0x7fd24815a9f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd244000c00 tx=0x7fd2440012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.447+0000 7fd23dffb640 1 -- 172.21.15.139:0/261597810 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244040020 con 0x7fd234003b70 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.447+0000 7fd23dffb640 1 -- 172.21.15.139:0/261597810 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2440054a0 con 0x7fd234003b70 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.447+0000 7fd24f3c8640 1 -- 172.21.15.139:0/261597810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2481f4d70 con 0x7fd234003b70 2024-04-25T09:55:34.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.447+0000 7fd23dffb640 1 -- 172.21.15.139:0/261597810 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244007450 con 0x7fd234003b70 2024-04-25T09:55:34.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.447+0000 7fd24f3c8640 1 -- 172.21.15.139:0/261597810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24815af30 con 0x7fd234003b70 2024-04-25T09:55:34.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.448+0000 7fd24f3c8640 1 -- 172.21.15.139:0/261597810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd248074df0 con 0x7fd234003b70 2024-04-25T09:55:34.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.448+0000 7fd23dffb640 1 -- 172.21.15.139:0/261597810 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd244043020 con 0x7fd234003b70 2024-04-25T09:55:34.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.449+0000 7fd23dffb640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd22c06b500 0x7fd22c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.449+0000 7fd23dffb640 1 -- 172.21.15.139:0/261597810 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2440aa2b0 con 0x7fd234003b70 2024-04-25T09:55:34.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.449+0000 7fd24c963640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd22c06b500 0x7fd22c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.449+0000 7fd24c963640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd22c06b500 0x7fd22c06d9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd2480ca560 tx=0x7fd238000f10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.453+0000 7fd23dffb640 1 -- 172.21.15.139:0/261597810 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd248074df0 con 0x7fd234003b70 2024-04-25T09:55:34.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.656+0000 7fd24f3c8640 1 -- 172.21.15.139:0/261597810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd248074520 con 0x7fd234003b70 2024-04-25T09:55:34.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.657+0000 7fd23dffb640 1 -- 172.21.15.139:0/261597810 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fd24404a030 con 0x7fd234003b70 2024-04-25T09:55:34.660 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:34.660 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1742557158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1742557158},{"type":"v1","addr":"172.21.15.150:6834","nonce":1742557158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3496803834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3496803834},{"type":"v1","addr":"172.21.15.150:6835","nonce":3496803834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2014537692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2014537692},{"type":"v1","addr":"172.21.15.139:6835","nonce":2014537692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:56.466203+0000","modified":"2024-04-25T09:54:58.466766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5850":{"gid":5850,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/338275708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":338275708},{"type":"v1","addr":"172.21.15.139:6837","nonce":338275708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:55:34.660 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 22 2024-04-25T09:55:34.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.659+0000 7fd24f3c8640 1 -- 172.21.15.139:0/261597810 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd22c06b500 msgr2=0x7fd22c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.659+0000 7fd24f3c8640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd22c06b500 0x7fd22c06d9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd2480ca560 tx=0x7fd238000f10 comp rx=0 tx=0).stop 2024-04-25T09:55:34.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.659+0000 7fd24f3c8640 1 -- 172.21.15.139:0/261597810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd234003b70 msgr2=0x7fd24815a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.659+0000 7fd24f3c8640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd234003b70 0x7fd24815a9f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd244000c00 tx=0x7fd2440012d0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.659+0000 7fd24f3c8640 1 -- 172.21.15.139:0/261597810 shutdown_connections 2024-04-25T09:55:34.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.659+0000 7fd24f3c8640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd22c06b500 0x7fd22c06d9c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.659+0000 7fd24f3c8640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2481f3a30 0x7fd2481f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.659+0000 7fd24f3c8640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2481600d0 0x7fd248160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.659+0000 7fd24f3c8640 1 --2- 172.21.15.139:0/261597810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd234003b70 0x7fd24815a9f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.659+0000 7fd24f3c8640 1 -- 172.21.15.139:0/261597810 >> 172.21.15.139:0/261597810 conn(0x7fd2480b5b00 msgr2=0x7fd24806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:34.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.660+0000 7fd24f3c8640 1 -- 172.21.15.139:0/261597810 shutdown_connections 2024-04-25T09:55:34.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.660+0000 7fd24f3c8640 1 -- 172.21.15.139:0/261597810 wait complete. 2024-04-25T09:55:34.673 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T09:55:34.792 INFO:teuthology.orchestra.run.smithi139.stdout:{"message":"","return_code":0} 2024-04-25T09:55:34.793 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:55:34.793 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T09:55:34.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.832+0000 7f2a814f4f40 1 Processor -- start 2024-04-25T09:55:34.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.832+0000 7f2a814f4f40 1 -- start start 2024-04-25T09:55:34.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a814f4f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5618bc4476b0 0x5618bc5afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a814f4f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc44be70 0x5618bc447a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a814f4f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618bc503e50 0x5618bc447fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a814f4f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618bc507580 con 0x5618bc44be70 2024-04-25T09:55:34.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a814f4f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618bc3f0df0 con 0x5618bc503e50 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a814f4f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618bc5216c0 con 0x5618bc4476b0 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7e7f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc44be70 0x5618bc447a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7e7f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc44be70 0x5618bc447a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46562/0 (socket says 172.21.15.139:46562) 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7e7f4640 1 -- 172.21.15.139:0/354252613 learned_addr learned my addr 172.21.15.139:0/354252613 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7eff5640 1 --2- 172.21.15.139:0/354252613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5618bc4476b0 0x5618bc5afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7f7f6640 1 --2- 172.21.15.139:0/354252613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618bc503e50 0x5618bc447fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7e7f4640 1 -- 172.21.15.139:0/354252613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5618bc4476b0 msgr2=0x5618bc5afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7e7f4640 1 --2- 172.21.15.139:0/354252613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5618bc4476b0 0x5618bc5afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7e7f4640 1 -- 172.21.15.139:0/354252613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618bc503e50 msgr2=0x5618bc447fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7e7f4640 1 --2- 172.21.15.139:0/354252613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618bc503e50 0x5618bc447fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7e7f4640 1 -- 172.21.15.139:0/354252613 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618bc376c50 con 0x5618bc44be70 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7e7f4640 1 --2- 172.21.15.139:0/354252613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc44be70 0x5618bc447a90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2a740098e0 tx=0x7f2a74005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fe7d6823bdd1bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7d7f2640 1 -- 172.21.15.139:0/354252613 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74008950 con 0x5618bc44be70 2024-04-25T09:55:34.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7d7f2640 1 -- 172.21.15.139:0/354252613 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a74018070 con 0x5618bc44be70 2024-04-25T09:55:34.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.833+0000 7f2a7d7f2640 1 -- 172.21.15.139:0/354252613 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74008d30 con 0x5618bc44be70 2024-04-25T09:55:34.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.834+0000 7f2a814f4f40 1 -- 172.21.15.139:0/354252613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc44be70 msgr2=0x5618bc447a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.834+0000 7f2a814f4f40 1 --2- 172.21.15.139:0/354252613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc44be70 0x5618bc447a90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2a740098e0 tx=0x7f2a74005a00 comp rx=0 tx=0).stop 2024-04-25T09:55:34.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.834+0000 7f2a814f4f40 1 -- 172.21.15.139:0/354252613 shutdown_connections 2024-04-25T09:55:34.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.834+0000 7f2a814f4f40 1 --2- 172.21.15.139:0/354252613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc44be70 0x5618bc447a90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.834+0000 7f2a814f4f40 1 --2- 172.21.15.139:0/354252613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618bc503e50 0x5618bc447fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.834+0000 7f2a814f4f40 1 --2- 172.21.15.139:0/354252613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5618bc4476b0 0x5618bc5afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.834+0000 7f2a814f4f40 1 -- 172.21.15.139:0/354252613 >> 172.21.15.139:0/354252613 conn(0x5618bc449be0 msgr2=0x5618bc5207f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:34.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.834+0000 7f2a814f4f40 1 -- 172.21.15.139:0/354252613 shutdown_connections 2024-04-25T09:55:34.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.834+0000 7f2a814f4f40 1 -- 172.21.15.139:0/354252613 wait complete. 2024-04-25T09:55:34.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.834+0000 7f2a814f4f40 1 Processor -- start 2024-04-25T09:55:34.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.834+0000 7f2a814f4f40 1 -- start start 2024-04-25T09:55:34.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a814f4f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618bc4476b0 0x5618bc51dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a814f4f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5618bc44be70 0x5618bc48ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a814f4f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc503e50 0x5618bc5a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a814f4f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618bc51ffe0 con 0x5618bc503e50 2024-04-25T09:55:34.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a814f4f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618bc44c330 con 0x5618bc4476b0 2024-04-25T09:55:34.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a814f4f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618bc479030 con 0x5618bc44be70 2024-04-25T09:55:34.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a7f7f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc503e50 0x5618bc5a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a7f7f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc503e50 0x5618bc5a9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46578/0 (socket says 172.21.15.139:46578) 2024-04-25T09:55:34.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a7f7f6640 1 -- 172.21.15.139:0/966834912 learned_addr learned my addr 172.21.15.139:0/966834912 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:34.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a7e7f4640 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5618bc44be70 0x5618bc48ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a7eff5640 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618bc4476b0 0x5618bc51dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a7f7f6640 1 -- 172.21.15.139:0/966834912 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5618bc44be70 msgr2=0x5618bc48ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a7f7f6640 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5618bc44be70 0x5618bc48ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a7f7f6640 1 -- 172.21.15.139:0/966834912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618bc4476b0 msgr2=0x5618bc51dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a7f7f6640 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618bc4476b0 0x5618bc51dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.835+0000 7f2a7f7f6640 1 -- 172.21.15.139:0/966834912 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618bc5b0840 con 0x5618bc503e50 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.836+0000 7f2a7f7f6640 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc503e50 0x5618bc5a9a70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800ef60 tx=0x7f2a7800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.836+0000 7f2a67fff640 1 -- 172.21.15.139:0/966834912 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800ccc0 con 0x5618bc503e50 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.836+0000 7f2a67fff640 1 -- 172.21.15.139:0/966834912 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a78016070 con 0x5618bc503e50 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.836+0000 7f2a67fff640 1 -- 172.21.15.139:0/966834912 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78011690 con 0x5618bc503e50 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.836+0000 7f2a814f4f40 1 -- 172.21.15.139:0/966834912 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5618bc48e120 con 0x5618bc503e50 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.836+0000 7f2a814f4f40 1 -- 172.21.15.139:0/966834912 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618bc5068b0 con 0x5618bc503e50 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.837+0000 7f2a67fff640 1 -- 172.21.15.139:0/966834912 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a78011830 con 0x5618bc503e50 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.837+0000 7f2a67fff640 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2a6c069eb0 0x7f2a6c06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.838+0000 7f2a67fff640 1 -- 172.21.15.139:0/966834912 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2a78088e70 con 0x5618bc503e50 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.838+0000 7f2a7eff5640 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2a6c069eb0 0x7f2a6c06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.838+0000 7f2a814f4f40 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5618bc5a9fb0 0x5618bc522a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.838+0000 7f2a814f4f40 1 -- 172.21.15.139:0/966834912 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50) v9 -- 0x5618bc522f50 con 0x5618bc5a9fb0 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.838+0000 7f2a7e7f4640 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5618bc5a9fb0 0x5618bc522a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.838+0000 7f2a7eff5640 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2a6c069eb0 0x7f2a6c06c370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a70056000 tx=0x7f2a70007030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.838+0000 7f2a7e7f4640 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5618bc5a9fb0 0x5618bc522a10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.838+0000 7f2a7e7f4640 1 -- 172.21.15.139:0/966834912 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f2a74008a00 con 0x5618bc5a9fb0 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.839+0000 7f2a814f4f40 1 -- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5618bc5a9fb0 msgr2=0x5618bc522a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.839+0000 7f2a814f4f40 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5618bc5a9fb0 0x5618bc522a10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.839+0000 7f2a814f4f40 1 -- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2a6c069eb0 msgr2=0x7f2a6c06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.839+0000 7f2a814f4f40 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2a6c069eb0 0x7f2a6c06c370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a70056000 tx=0x7f2a70007030 comp rx=0 tx=0).stop 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.839+0000 7f2a814f4f40 1 -- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc503e50 msgr2=0x5618bc5a9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.839+0000 7f2a814f4f40 1 --2- 172.21.15.139:0/966834912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5618bc503e50 0x5618bc5a9a70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800ef60 tx=0x7f2a7800ef90 comp rx=0 tx=0).stop 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.839+0000 7f2a7f7f6640 1 -- 172.21.15.139:0/966834912 reap_dead start 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.839+0000 7f2a814f4f40 1 -- 172.21.15.139:0/966834912 shutdown_connections 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.839+0000 7f2a814f4f40 1 -- 172.21.15.139:0/966834912 >> 172.21.15.139:0/966834912 conn(0x5618bc449be0 msgr2=0x5618bc5207f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.839+0000 7f2a814f4f40 1 -- 172.21.15.139:0/966834912 shutdown_connections 2024-04-25T09:55:34.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:34.839+0000 7f2a814f4f40 1 -- 172.21.15.139:0/966834912 wait complete. 2024-04-25T09:55:34.847 INFO:teuthology.orchestra.run.smithi139.stdout:Yalphaparent_a 2024-04-25T09:55:34.848 DEBUG:teuthology.orchestra.run.smithi139:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T09:55:34.933 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:55:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 1099511627777, 2024-04-25T09:55:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: "ancestors": [ 2024-04-25T09:55:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: "dirino": 1099511627776, 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: "dname": "alpha", 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 4 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: "dirino": 1, 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: "dname": "parent_a", 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 7 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 8, 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: "old_pools": [] 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:55:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:34.936 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:55:34.936 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:55:35.010 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:55:35.049 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:55:35.049 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:55:35.050 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:35.065 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:55:35.066 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:35.121 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:35.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.255+0000 7f389899e640 1 Processor -- start 2024-04-25T09:55:35.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.255+0000 7f389899e640 1 -- start start 2024-04-25T09:55:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.255+0000 7f389899e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f389415b940 0x7f38940d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.255+0000 7f389899e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f389415bd20 0x7f389415c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.255+0000 7f389899e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38940d1250 con 0x7f389415f370 2024-04-25T09:55:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.255+0000 7f389899e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38940b2750 con 0x7f389415bd20 2024-04-25T09:55:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.255+0000 7f389899e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389409a840 con 0x7f389415b940 2024-04-25T09:55:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3893fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f389415f370 0x7f389415f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48836/0 (socket says 172.21.15.139:48836) 2024-04-25T09:55:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3893fff640 1 -- 172.21.15.139:0/1688201856 learned_addr learned my addr 172.21.15.139:0/1688201856 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:35.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f38937fe640 1 --2- 172.21.15.139:0/1688201856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f389415b940 0x7f38940d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f388bfff640 1 --2- 172.21.15.139:0/1688201856 >> v2:172.21.15.150:3300/0 conn(0x7f389415bd20 0x7f389415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168059432 0 0) 0x7f388c001490 con 0x7f389415f370 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3880003520 con 0x7f389415f370 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2988974469 0 0) 0x7f3880003520 con 0x7f389415f370 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f389415b940 msgr2=0x7f38940d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 --2- 172.21.15.139:0/1688201856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f389415b940 0x7f38940d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 >> v2:172.21.15.150:3300/0 conn(0x7f389415bd20 msgr2=0x7f389415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 --2- 172.21.15.139:0/1688201856 >> v2:172.21.15.150:3300/0 conn(0x7f389415bd20 0x7f389415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38940b29a0 con 0x7f389415f370 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f388c001bc0 con 0x7f389415f370 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f388bfff640 1 --2- 172.21.15.139:0/1688201856 >> v2:172.21.15.150:3300/0 conn(0x7f389415bd20 0x7f389415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 >> v1:172.21.15.139:6789/0 conn(0x7f389415f370 legacy=0x7f389415f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 --2- 172.21.15.139:0/1688201856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38800039b0 0x7f3880003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 --2- 172.21.15.139:0/1688201856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3880004800 0x7f3880004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 --2- 172.21.15.139:0/1688201856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f389415b940 0x7f38800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38940b2750 con 0x7f3880004800 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389409a840 con 0x7f38800039b0 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388000b9c0 con 0x7f389415b940 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f388bfff640 1 --2- 172.21.15.139:0/1688201856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3880004800 0x7f3880004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f38937fe640 1 --2- 172.21.15.139:0/1688201856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38800039b0 0x7f3880003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.256+0000 7f3893fff640 1 --2- 172.21.15.139:0/1688201856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f389415b940 0x7f38800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.257+0000 7f388bfff640 1 -- 172.21.15.139:0/1688201856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f389415b940 msgr2=0x7f38800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.257+0000 7f388bfff640 1 --2- 172.21.15.139:0/1688201856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f389415b940 0x7f38800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.257+0000 7f388bfff640 1 -- 172.21.15.139:0/1688201856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38800039b0 msgr2=0x7f3880003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.257+0000 7f388bfff640 1 --2- 172.21.15.139:0/1688201856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38800039b0 0x7f3880003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.257+0000 7f388bfff640 1 -- 172.21.15.139:0/1688201856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3880003520 con 0x7f3880004800 2024-04-25T09:55:35.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.257+0000 7f388bfff640 1 --2- 172.21.15.139:0/1688201856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3880004800 0x7f3880004c80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f387c009c90 tx=0x7f3880005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf49988e948df53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.257+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c036070 con 0x7f3880004800 2024-04-25T09:55:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.257+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c031400 con 0x7f3880004800 2024-04-25T09:55:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.257+0000 7f3892ffd640 1 -- 172.21.15.139:0/1688201856 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c009860 con 0x7f3880004800 2024-04-25T09:55:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.258+0000 7f389899e640 1 -- 172.21.15.139:0/1688201856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3880004800 msgr2=0x7f3880004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.258+0000 7f389899e640 1 --2- 172.21.15.139:0/1688201856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3880004800 0x7f3880004c80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f387c009c90 tx=0x7f3880005830 comp rx=0 tx=0).stop 2024-04-25T09:55:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.258+0000 7f3893fff640 1 -- 172.21.15.139:0/1688201856 reap_dead start 2024-04-25T09:55:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.258+0000 7f389899e640 1 -- 172.21.15.139:0/1688201856 shutdown_connections 2024-04-25T09:55:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.258+0000 7f389899e640 1 -- 172.21.15.139:0/1688201856 >> 172.21.15.139:0/1688201856 conn(0x7f38940b30d0 msgr2=0x7f38940b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.258+0000 7f389899e640 1 -- 172.21.15.139:0/1688201856 shutdown_connections 2024-04-25T09:55:35.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.258+0000 7f389899e640 1 -- 172.21.15.139:0/1688201856 wait complete. 2024-04-25T09:55:35.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.259+0000 7f389899e640 1 Processor -- start 2024-04-25T09:55:35.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.259+0000 7f389899e640 1 -- start start 2024-04-25T09:55:35.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.260+0000 7f389899e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3880004800 0x7f38940b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.260+0000 7f389899e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38941f49c0 0x7f3894157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.260+0000 7f389899e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3894159a30 0x7f3894155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.260+0000 7f389899e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389415cb00 con 0x7f3894159a30 2024-04-25T09:55:35.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.260+0000 7f389899e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389415c6a0 con 0x7f3880004800 2024-04-25T09:55:35.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.260+0000 7f389899e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389415cc80 con 0x7f38941f49c0 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.260+0000 7f3893fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3894159a30 0x7f3894155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.260+0000 7f3893fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3894159a30 0x7f3894155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46592/0 (socket says 172.21.15.139:46592) 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.260+0000 7f3893fff640 1 -- 172.21.15.139:0/3816249959 learned_addr learned my addr 172.21.15.139:0/3816249959 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.260+0000 7f38937fe640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3880004800 0x7f38940b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.260+0000 7f388bfff640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38941f49c0 0x7f3894157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.261+0000 7f3893fff640 1 -- 172.21.15.139:0/3816249959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38941f49c0 msgr2=0x7f3894157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.261+0000 7f3893fff640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38941f49c0 0x7f3894157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.261+0000 7f3893fff640 1 -- 172.21.15.139:0/3816249959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3880004800 msgr2=0x7f38940b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.261+0000 7f3893fff640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3880004800 0x7f38940b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.261+0000 7f3893fff640 1 -- 172.21.15.139:0/3816249959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38941f1350 con 0x7f3894159a30 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.261+0000 7f3893fff640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3894159a30 0x7f3894155c90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f38940b2700 tx=0x7f388c00aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.261+0000 7f3890ff9640 1 -- 172.21.15.139:0/3816249959 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c007860 con 0x7f3894159a30 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.261+0000 7f3890ff9640 1 -- 172.21.15.139:0/3816249959 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c007a00 con 0x7f3894159a30 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.261+0000 7f389899e640 1 -- 172.21.15.139:0/3816249959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38940b6230 con 0x7f3894159a30 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.261+0000 7f3890ff9640 1 -- 172.21.15.139:0/3816249959 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c0135f0 con 0x7f3894159a30 2024-04-25T09:55:35.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.261+0000 7f389899e640 1 -- 172.21.15.139:0/3816249959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38941582d0 con 0x7f3894159a30 2024-04-25T09:55:35.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.262+0000 7f389899e640 1 -- 172.21.15.139:0/3816249959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3894073d80 con 0x7f3894159a30 2024-04-25T09:55:35.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.263+0000 7f3890ff9640 1 -- 172.21.15.139:0/3816249959 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f388c013790 con 0x7f3894159a30 2024-04-25T09:55:35.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.263+0000 7f3890ff9640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f387406b4d0 0x7f387406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.263+0000 7f3890ff9640 1 -- 172.21.15.139:0/3816249959 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f388c08acb0 con 0x7f3894159a30 2024-04-25T09:55:35.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.263+0000 7f38937fe640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f387406b4d0 0x7f387406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.263+0000 7f38937fe640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f387406b4d0 0x7f387406d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3884009a60 tx=0x7f3884009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.267+0000 7f3890ff9640 1 -- 172.21.15.139:0/3816249959 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3894073d80 con 0x7f3894159a30 2024-04-25T09:55:35.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.437+0000 7f389899e640 1 -- 172.21.15.139:0/3816249959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f389404ef90 con 0x7f3894159a30 2024-04-25T09:55:35.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.438+0000 7f3890ff9640 1 -- 172.21.15.139:0/3816249959 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f38940d1250 con 0x7f3894159a30 2024-04-25T09:55:35.441 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:35.441 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":50,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:58.130580+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:54.661529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:55.431933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.440+0000 7f389899e640 1 -- 172.21.15.139:0/3816249959 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f387406b4d0 msgr2=0x7f387406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.440+0000 7f389899e640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f387406b4d0 0x7f387406d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3884009a60 tx=0x7f3884009190 comp rx=0 tx=0).stop 2024-04-25T09:55:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.440+0000 7f389899e640 1 -- 172.21.15.139:0/3816249959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3894159a30 msgr2=0x7f3894155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.440+0000 7f389899e640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3894159a30 0x7f3894155c90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f38940b2700 tx=0x7f388c00aee0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.440+0000 7f389899e640 1 -- 172.21.15.139:0/3816249959 shutdown_connections 2024-04-25T09:55:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.440+0000 7f389899e640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3894159a30 0x7f3894155c90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.440+0000 7f389899e640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f387406b4d0 0x7f387406d990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.440+0000 7f389899e640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38941f49c0 0x7f3894157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.440+0000 7f389899e640 1 --2- 172.21.15.139:0/3816249959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3880004800 0x7f38940b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.440+0000 7f389899e640 1 -- 172.21.15.139:0/3816249959 >> 172.21.15.139:0/3816249959 conn(0x7f38940b30d0 msgr2=0x7f38940b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.440+0000 7f389899e640 1 -- 172.21.15.139:0/3816249959 shutdown_connections 2024-04-25T09:55:35.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.440+0000 7f389899e640 1 -- 172.21.15.139:0/3816249959 wait complete. 2024-04-25T09:55:35.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:54.661529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:55.431933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:55:35.454 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:35.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.551+0000 7f16ea763640 1 Processor -- start 2024-04-25T09:55:35.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.551+0000 7f16ea763640 1 -- start start 2024-04-25T09:55:35.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16ea763640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f16e40cd610 0x7f16e40d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16ea763640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f16e409ad40 0x7f16e40d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16ea763640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16e41618c0 0x7f16e4161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16ea763640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e4099a50 con 0x7f16e409ad40 2024-04-25T09:55:35.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16ea763640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e4099ca0 con 0x7f16e40cd610 2024-04-25T09:55:35.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16ea763640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e4099f80 con 0x7f16e41618c0 2024-04-25T09:55:35.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16e37fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f16e409ad40 0x7f16e40d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16e37fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f16e409ad40 0x7f16e40d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46600/0 (socket says 172.21.15.139:46600) 2024-04-25T09:55:35.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16e37fe640 1 -- 172.21.15.139:0/3041803793 learned_addr learned my addr 172.21.15.139:0/3041803793 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:35.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16e8d00640 1 --2- 172.21.15.139:0/3041803793 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16e41618c0 0x7f16e4161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16e3fff640 1 --2- 172.21.15.139:0/3041803793 >> v2:172.21.15.150:3300/0 conn(0x7f16e40cd610 0x7f16e40d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16e37fe640 1 -- 172.21.15.139:0/3041803793 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16e41618c0 msgr2=0x7f16e4161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16e37fe640 1 --2- 172.21.15.139:0/3041803793 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16e41618c0 0x7f16e4161d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16e37fe640 1 -- 172.21.15.139:0/3041803793 >> v2:172.21.15.150:3300/0 conn(0x7f16e40cd610 msgr2=0x7f16e40d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16e37fe640 1 --2- 172.21.15.139:0/3041803793 >> v2:172.21.15.150:3300/0 conn(0x7f16e40cd610 0x7f16e40d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16e37fe640 1 -- 172.21.15.139:0/3041803793 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e409a260 con 0x7f16e409ad40 2024-04-25T09:55:35.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.552+0000 7f16e3fff640 1 --2- 172.21.15.139:0/3041803793 >> v2:172.21.15.150:3300/0 conn(0x7f16e40cd610 0x7f16e40d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:35.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.553+0000 7f16e37fe640 1 --2- 172.21.15.139:0/3041803793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e409ad40 0x7f16e40d08b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f16e406f210 tx=0x7f16d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2080415707fe95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.553+0000 7f16e2ffd640 1 -- 172.21.15.139:0/3041803793 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8037070 con 0x7f16e409ad40 2024-04-25T09:55:35.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.553+0000 7f16e2ffd640 1 -- 172.21.15.139:0/3041803793 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d8032440 con 0x7f16e409ad40 2024-04-25T09:55:35.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.553+0000 7f16e2ffd640 1 -- 172.21.15.139:0/3041803793 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8032810 con 0x7f16e409ad40 2024-04-25T09:55:35.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.554+0000 7f16ea763640 1 -- 172.21.15.139:0/3041803793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e409ad40 msgr2=0x7f16e40d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.554+0000 7f16ea763640 1 --2- 172.21.15.139:0/3041803793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e409ad40 0x7f16e40d08b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f16e406f210 tx=0x7f16d802a270 comp rx=0 tx=0).stop 2024-04-25T09:55:35.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.554+0000 7f16ea763640 1 -- 172.21.15.139:0/3041803793 shutdown_connections 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.554+0000 7f16ea763640 1 --2- 172.21.15.139:0/3041803793 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16e41618c0 0x7f16e4161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.554+0000 7f16ea763640 1 --2- 172.21.15.139:0/3041803793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e409ad40 0x7f16e40d08b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.554+0000 7f16ea763640 1 --2- 172.21.15.139:0/3041803793 >> v2:172.21.15.150:3300/0 conn(0x7f16e40cd610 0x7f16e40d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.554+0000 7f16ea763640 1 -- 172.21.15.139:0/3041803793 >> 172.21.15.139:0/3041803793 conn(0x7f16e40b31f0 msgr2=0x7f16e40b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.554+0000 7f16ea763640 1 -- 172.21.15.139:0/3041803793 shutdown_connections 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.554+0000 7f16ea763640 1 -- 172.21.15.139:0/3041803793 wait complete. 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.555+0000 7f16ea763640 1 Processor -- start 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.555+0000 7f16ea763640 1 -- start start 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16ea763640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e409ad40 0x7f16e40c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16ea763640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16e40cd610 0x7f16e41567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16ea763640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16e41618c0 0x7f16e40c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16ea763640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40cd9f0 con 0x7f16e409ad40 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16ea763640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e409a820 con 0x7f16e40cd610 2024-04-25T09:55:35.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16ea763640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40cdb70 con 0x7f16e41618c0 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e409ad40 0x7f16e40c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e409ad40 0x7f16e40c5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46616/0 (socket says 172.21.15.139:46616) 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e3fff640 1 -- 172.21.15.139:0/2188278500 learned_addr learned my addr 172.21.15.139:0/2188278500 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e37fe640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16e40cd610 0x7f16e41567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e8d00640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16e41618c0 0x7f16e40c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e3fff640 1 -- 172.21.15.139:0/2188278500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16e41618c0 msgr2=0x7f16e40c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e3fff640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16e41618c0 0x7f16e40c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e3fff640 1 -- 172.21.15.139:0/2188278500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16e40cd610 msgr2=0x7f16e41567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e3fff640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16e40cd610 0x7f16e41567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e3fff640 1 -- 172.21.15.139:0/2188278500 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e40c5b10 con 0x7f16e409ad40 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e8d00640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16e41618c0 0x7f16e40c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e3fff640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e409ad40 0x7f16e40c5210 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f16d400c7f0 tx=0x7f16d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e0ff9640 1 -- 172.21.15.139:0/2188278500 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4015070 con 0x7f16e409ad40 2024-04-25T09:55:35.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16e0ff9640 1 -- 172.21.15.139:0/2188278500 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d40079d0 con 0x7f16e409ad40 2024-04-25T09:55:35.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.556+0000 7f16ea763640 1 -- 172.21.15.139:0/2188278500 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e40c5df0 con 0x7f16e409ad40 2024-04-25T09:55:35.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.557+0000 7f16e0ff9640 1 -- 172.21.15.139:0/2188278500 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4007da0 con 0x7f16e409ad40 2024-04-25T09:55:35.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.557+0000 7f16ea763640 1 -- 172.21.15.139:0/2188278500 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e40c63b0 con 0x7f16e409ad40 2024-04-25T09:55:35.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.557+0000 7f16ea763640 1 -- 172.21.15.139:0/2188278500 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e4074520 con 0x7f16e409ad40 2024-04-25T09:55:35.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.558+0000 7f16e0ff9640 1 -- 172.21.15.139:0/2188278500 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16d4018070 con 0x7f16e409ad40 2024-04-25T09:55:35.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.558+0000 7f16e0ff9640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f16d006b4d0 0x7f16d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.558+0000 7f16e0ff9640 1 -- 172.21.15.139:0/2188278500 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16d408a120 con 0x7f16e409ad40 2024-04-25T09:55:35.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.558+0000 7f16e37fe640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f16d006b4d0 0x7f16d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.559+0000 7f16e37fe640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f16d006b4d0 0x7f16d006d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f16d8034000 tx=0x7f16d802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.563+0000 7f16e0ff9640 1 -- 172.21.15.139:0/2188278500 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16d4058850 con 0x7f16e409ad40 2024-04-25T09:55:35.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.733+0000 7f16ea763640 1 -- 172.21.15.139:0/2188278500 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16e40b3f70 con 0x7f16e409ad40 2024-04-25T09:55:35.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.733+0000 7f16e0ff9640 1 -- 172.21.15.139:0/2188278500 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16d405c8f0 con 0x7f16e409ad40 2024-04-25T09:55:35.736 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:55:35.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.735+0000 7f16ea763640 1 -- 172.21.15.139:0/2188278500 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f16d006b4d0 msgr2=0x7f16d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.735+0000 7f16ea763640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f16d006b4d0 0x7f16d006d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f16d8034000 tx=0x7f16d802e000 comp rx=0 tx=0).stop 2024-04-25T09:55:35.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.735+0000 7f16ea763640 1 -- 172.21.15.139:0/2188278500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e409ad40 msgr2=0x7f16e40c5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.735+0000 7f16ea763640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e409ad40 0x7f16e40c5210 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f16d400c7f0 tx=0x7f16d400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.735+0000 7f16ea763640 1 -- 172.21.15.139:0/2188278500 shutdown_connections 2024-04-25T09:55:35.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.735+0000 7f16ea763640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f16d006b4d0 0x7f16d006d990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.735+0000 7f16ea763640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16e41618c0 0x7f16e40c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.735+0000 7f16ea763640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16e40cd610 0x7f16e41567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.735+0000 7f16ea763640 1 --2- 172.21.15.139:0/2188278500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16e409ad40 0x7f16e40c5210 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.735+0000 7f16ea763640 1 -- 172.21.15.139:0/2188278500 >> 172.21.15.139:0/2188278500 conn(0x7f16e40b31f0 msgr2=0x7f16e415fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.735+0000 7f16ea763640 1 -- 172.21.15.139:0/2188278500 shutdown_connections 2024-04-25T09:55:35.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.735+0000 7f16ea763640 1 -- 172.21.15.139:0/2188278500 wait complete. 2024-04-25T09:55:35.748 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.845+0000 7f2d2c1c8640 1 Processor -- start 2024-04-25T09:55:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.845+0000 7f2d2c1c8640 1 -- start start 2024-04-25T09:55:35.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.846+0000 7f2d2c1c8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2d2415dbf0 0x7f2d241617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.846+0000 7f2d2c1c8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d240b9370 0x7f2d240ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.846+0000 7f2d2c1c8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2d240cec80 0x7f2d2406c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.846+0000 7f2d2c1c8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d240d1ee0 con 0x7f2d2415dbf0 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.846+0000 7f2d2c1c8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d24161dc0 con 0x7f2d240cec80 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.846+0000 7f2d2c1c8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241620a0 con 0x7f2d240b9370 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.846+0000 7f2d29f64640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2d2415dbf0 0x7f2d241617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.846+0000 7f2d29f64640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2d2415dbf0 0x7f2d241617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46620/0 (socket says 172.21.15.139:46620) 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.846+0000 7f2d29f64640 1 -- 172.21.15.139:0/2245565082 learned_addr learned my addr 172.21.15.139:0/2245565082 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.847+0000 7f2d2a765640 1 --2- 172.21.15.139:0/2245565082 >> v2:172.21.15.150:3300/0 conn(0x7f2d240cec80 0x7f2d2406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.847+0000 7f2d29763640 1 --2- 172.21.15.139:0/2245565082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d240b9370 0x7f2d240ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.847+0000 7f2d29f64640 1 -- 172.21.15.139:0/2245565082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d240b9370 msgr2=0x7f2d240ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.847+0000 7f2d29f64640 1 --2- 172.21.15.139:0/2245565082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d240b9370 0x7f2d240ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.847+0000 7f2d29f64640 1 -- 172.21.15.139:0/2245565082 >> v2:172.21.15.150:3300/0 conn(0x7f2d240cec80 msgr2=0x7f2d2406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.847+0000 7f2d29f64640 1 --2- 172.21.15.139:0/2245565082 >> v2:172.21.15.150:3300/0 conn(0x7f2d240cec80 0x7f2d2406c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.847+0000 7f2d29f64640 1 -- 172.21.15.139:0/2245565082 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d24162380 con 0x7f2d2415dbf0 2024-04-25T09:55:35.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.847+0000 7f2d2a765640 1 --2- 172.21.15.139:0/2245565082 >> v2:172.21.15.150:3300/0 conn(0x7f2d240cec80 0x7f2d2406c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:35.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.847+0000 7f2d29f64640 1 --2- 172.21.15.139:0/2245565082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d2415dbf0 0x7f2d241617e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2d2408c170 tx=0x7f2d2002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c712579bee2b281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.848+0000 7f2d28f62640 1 -- 172.21.15.139:0/2245565082 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20037070 con 0x7f2d2415dbf0 2024-04-25T09:55:35.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.848+0000 7f2d28f62640 1 -- 172.21.15.139:0/2245565082 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d20032440 con 0x7f2d2415dbf0 2024-04-25T09:55:35.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.848+0000 7f2d28f62640 1 -- 172.21.15.139:0/2245565082 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20032810 con 0x7f2d2415dbf0 2024-04-25T09:55:35.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.848+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/2245565082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d2415dbf0 msgr2=0x7f2d241617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.848+0000 7f2d2c1c8640 1 --2- 172.21.15.139:0/2245565082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d2415dbf0 0x7f2d241617e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2d2408c170 tx=0x7f2d2002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.848+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/2245565082 shutdown_connections 2024-04-25T09:55:35.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.848+0000 7f2d2c1c8640 1 --2- 172.21.15.139:0/2245565082 >> v2:172.21.15.150:3300/0 conn(0x7f2d240cec80 0x7f2d2406c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.848+0000 7f2d2c1c8640 1 --2- 172.21.15.139:0/2245565082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d240b9370 0x7f2d240ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.848+0000 7f2d2c1c8640 1 --2- 172.21.15.139:0/2245565082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d2415dbf0 0x7f2d241617e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.848+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/2245565082 >> 172.21.15.139:0/2245565082 conn(0x7f2d240b5480 msgr2=0x7f2d240b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.848+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/2245565082 shutdown_connections 2024-04-25T09:55:35.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.848+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/2245565082 wait complete. 2024-04-25T09:55:35.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.849+0000 7f2d2c1c8640 1 Processor -- start 2024-04-25T09:55:35.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.849+0000 7f2d2c1c8640 1 -- start start 2024-04-25T09:55:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d2c1c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d240b9370 0x7f2d2414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d2c1c8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d240cec80 0x7f2d240aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d2c1c8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2415dbf0 0x7f2d240d2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d2c1c8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2415dfd0 con 0x7f2d240b9370 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d2c1c8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240b97d0 con 0x7f2d2415dbf0 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d2c1c8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240aada0 con 0x7f2d240cec80 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d29f64640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d240b9370 0x7f2d2414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d29f64640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d240b9370 0x7f2d2414ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46632/0 (socket says 172.21.15.139:46632) 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d29f64640 1 -- 172.21.15.139:0/3733342317 learned_addr learned my addr 172.21.15.139:0/3733342317 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d29763640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d240cec80 0x7f2d240aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d2a765640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2415dbf0 0x7f2d240d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d29f64640 1 -- 172.21.15.139:0/3733342317 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d240cec80 msgr2=0x7f2d240aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.850+0000 7f2d29f64640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d240cec80 0x7f2d240aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.851+0000 7f2d29f64640 1 -- 172.21.15.139:0/3733342317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2415dbf0 msgr2=0x7f2d240d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.851+0000 7f2d29f64640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2415dbf0 0x7f2d240d2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.851+0000 7f2d29f64640 1 -- 172.21.15.139:0/3733342317 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d240ab080 con 0x7f2d240b9370 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.851+0000 7f2d29763640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d240cec80 0x7f2d240aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.851+0000 7f2d29f64640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d240b9370 0x7f2d2414ec00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2d20000c00 tx=0x7f2d200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.851+0000 7f2d1a7fc640 1 -- 172.21.15.139:0/3733342317 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20041070 con 0x7f2d240b9370 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.851+0000 7f2d1a7fc640 1 -- 172.21.15.139:0/3733342317 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2003c870 con 0x7f2d240b9370 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.851+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/3733342317 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d240ab360 con 0x7f2d240b9370 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.851+0000 7f2d1a7fc640 1 -- 172.21.15.139:0/3733342317 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20037040 con 0x7f2d240b9370 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.851+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/3733342317 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d240ab920 con 0x7f2d240b9370 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.852+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/3733342317 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2414f200 con 0x7f2d240b9370 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.852+0000 7f2d1a7fc640 1 -- 172.21.15.139:0/3733342317 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d20044070 con 0x7f2d240b9370 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.853+0000 7f2d1a7fc640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d0806b4d0 0x7f2d0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.853+0000 7f2d1a7fc640 1 -- 172.21.15.139:0/3733342317 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d200aae60 con 0x7f2d240b9370 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.853+0000 7f2d29763640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d0806b4d0 0x7f2d0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.853+0000 7f2d29763640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d0806b4d0 0x7f2d0806d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2d24159400 tx=0x7f2d14008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:35.857+0000 7f2d1a7fc640 1 -- 172.21.15.139:0/3733342317 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d20079be0 con 0x7f2d240b9370 2024-04-25T09:55:36.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.026+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/3733342317 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d24074520 con 0x7f2d240b9370 2024-04-25T09:55:36.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.027+0000 7f2d1a7fc640 1 -- 172.21.15.139:0/3733342317 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d200794e0 con 0x7f2d240b9370 2024-04-25T09:55:36.029 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:55:36.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.028+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/3733342317 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d0806b4d0 msgr2=0x7f2d0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.028+0000 7f2d2c1c8640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d0806b4d0 0x7f2d0806d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2d24159400 tx=0x7f2d14008040 comp rx=0 tx=0).stop 2024-04-25T09:55:36.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.028+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/3733342317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d240b9370 msgr2=0x7f2d2414ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.028+0000 7f2d2c1c8640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d240b9370 0x7f2d2414ec00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2d20000c00 tx=0x7f2d200046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.028+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/3733342317 shutdown_connections 2024-04-25T09:55:36.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.028+0000 7f2d2c1c8640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d0806b4d0 0x7f2d0806d990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.028+0000 7f2d2c1c8640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2415dbf0 0x7f2d240d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.028+0000 7f2d2c1c8640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d240cec80 0x7f2d240aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.028+0000 7f2d2c1c8640 1 --2- 172.21.15.139:0/3733342317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d240b9370 0x7f2d2414ec00 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.028+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/3733342317 >> 172.21.15.139:0/3733342317 conn(0x7f2d240b5480 msgr2=0x7f2d240d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.029+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/3733342317 shutdown_connections 2024-04-25T09:55:36.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.029+0000 7f2d2c1c8640 1 -- 172.21.15.139:0/3733342317 wait complete. 2024-04-25T09:55:36.042 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:36.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.140+0000 7fb7bd09e640 1 Processor -- start 2024-04-25T09:55:36.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.140+0000 7fb7bd09e640 1 -- start start 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.141+0000 7fb7bd09e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb7b80cee50 0x7fb7b8168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.141+0000 7fb7bd09e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b8161dc0 con 0x7fb7b815dbf0 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.141+0000 7fb7bd09e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b81620a0 con 0x7fb7b81617e0 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.141+0000 7fb7bd09e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b80d1ee0 con 0x7fb7b80cee50 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.141+0000 7fb7b6d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb7b815dbf0 0x7fb7b80b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48844/0 (socket says 172.21.15.139:48844) 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.141+0000 7fb7b6d9d640 1 -- 172.21.15.139:0/701382746 learned_addr learned my addr 172.21.15.139:0/701382746 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.141+0000 7fb7b759e640 1 --2- 172.21.15.139:0/701382746 >> v2:172.21.15.150:3300/0 conn(0x7fb7b80cee50 0x7fb7b8168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.141+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353388180 0 0) 0x7fb7b8161dc0 con 0x7fb7b815dbf0 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.141+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7a4003520 con 0x7fb7b815dbf0 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654908535 0 0) 0x7fb7b81620a0 con 0x7fb7b81617e0 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7b8161dc0 con 0x7fb7b81617e0 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2574453313 0 0) 0x7fb7a4003520 con 0x7fb7b815dbf0 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 >> v2:172.21.15.150:3300/0 conn(0x7fb7b80cee50 msgr2=0x7fb7b8168b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:36.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 --2- 172.21.15.139:0/701382746 >> v2:172.21.15.150:3300/0 conn(0x7fb7b80cee50 0x7fb7b8168b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 >> v1:172.21.15.150:6789/0 conn(0x7fb7b81617e0 legacy=0x7fb7b80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b8162380 con 0x7fb7b815dbf0 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fb7a8001670 con 0x7fb7b815dbf0 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 >> v1:172.21.15.139:6789/0 conn(0x7fb7b815dbf0 legacy=0x7fb7b80b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 --2- 172.21.15.139:0/701382746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a4003b40 0x7fb7a4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 --2- 172.21.15.139:0/701382746 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a4004800 0x7fb7a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 --2- 172.21.15.139:0/701382746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7a40051c0 0x7fb7a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b80d1ee0 con 0x7fb7a40051c0 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a400c0f0 con 0x7fb7a4003b40 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a400c2f0 con 0x7fb7a4004800 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b759e640 1 --2- 172.21.15.139:0/701382746 >> v2:172.21.15.150:3300/0 conn(0x7fb7b80cee50 0x7fb7b8168b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b759e640 1 --2- 172.21.15.139:0/701382746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7a40051c0 0x7fb7a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7affff640 1 --2- 172.21.15.139:0/701382746 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a4004800 0x7fb7a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b6d9d640 1 --2- 172.21.15.139:0/701382746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a4003b40 0x7fb7a4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b759e640 1 -- 172.21.15.139:0/701382746 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a4004800 msgr2=0x7fb7a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b759e640 1 --2- 172.21.15.139:0/701382746 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7a4004800 0x7fb7a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b759e640 1 -- 172.21.15.139:0/701382746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a4003b40 msgr2=0x7fb7a4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b759e640 1 --2- 172.21.15.139:0/701382746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a4003b40 0x7fb7a4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.142+0000 7fb7b759e640 1 -- 172.21.15.139:0/701382746 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a4003520 con 0x7fb7a40051c0 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.143+0000 7fb7b759e640 1 -- 172.21.15.139:0/701382746 reap_dead start 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.143+0000 7fb7b759e640 1 --2- 172.21.15.139:0/701382746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7a40051c0 0x7fb7a400b9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e320 tx=0x7fb7a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523623d2f7fcaeeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.143+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0009dc0 con 0x7fb7a40051c0 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.143+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b0003bb0 con 0x7fb7a40051c0 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.143+0000 7fb7b659c640 1 -- 172.21.15.139:0/701382746 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0004020 con 0x7fb7a40051c0 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.144+0000 7fb7bd09e640 1 -- 172.21.15.139:0/701382746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7a40051c0 msgr2=0x7fb7a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.144+0000 7fb7bd09e640 1 --2- 172.21.15.139:0/701382746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7a40051c0 0x7fb7a400b9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e320 tx=0x7fb7a4005830 comp rx=0 tx=0).stop 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.144+0000 7fb7bd09e640 1 -- 172.21.15.139:0/701382746 shutdown_connections 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.144+0000 7fb7bd09e640 1 --2- 172.21.15.139:0/701382746 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7a40051c0 0x7fb7a400b9c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.144+0000 7fb7bd09e640 1 -- 172.21.15.139:0/701382746 >> 172.21.15.139:0/701382746 conn(0x7fb7b80b5480 msgr2=0x7fb7b80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.144+0000 7fb7bd09e640 1 -- 172.21.15.139:0/701382746 shutdown_connections 2024-04-25T09:55:36.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.144+0000 7fb7bd09e640 1 -- 172.21.15.139:0/701382746 wait complete. 2024-04-25T09:55:36.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.145+0000 7fb7bd09e640 1 Processor -- start 2024-04-25T09:55:36.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.145+0000 7fb7bd09e640 1 -- start start 2024-04-25T09:55:36.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7bd09e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a40051c0 0x7fb7b8159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7bd09e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b81f4170 0x7fb7b80cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7bd09e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7b8151290 0x7fb7b8151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7bd09e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b815dfd0 con 0x7fb7b8151290 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7bd09e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b80b9370 con 0x7fb7a40051c0 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7bd09e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b815a120 con 0x7fb7b81f4170 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7b759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7b8151290 0x7fb7b8151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7b759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7b8151290 0x7fb7b8151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46658/0 (socket says 172.21.15.139:46658) 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7b759e640 1 -- 172.21.15.139:0/2137477543 learned_addr learned my addr 172.21.15.139:0/2137477543 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7b6d9d640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a40051c0 0x7fb7b8159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7affff640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b81f4170 0x7fb7b80cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7b759e640 1 -- 172.21.15.139:0/2137477543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b81f4170 msgr2=0x7fb7b80cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7b759e640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b81f4170 0x7fb7b80cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7b759e640 1 -- 172.21.15.139:0/2137477543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a40051c0 msgr2=0x7fb7b8159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7b759e640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a40051c0 0x7fb7b8159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7b759e640 1 -- 172.21.15.139:0/2137477543 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b815a3c0 con 0x7fb7b8151290 2024-04-25T09:55:36.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7b759e640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7b8151290 0x7fb7b8151740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e440 tx=0x7fb7b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7af7fe640 1 -- 172.21.15.139:0/2137477543 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0046070 con 0x7fb7b8151290 2024-04-25T09:55:36.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7af7fe640 1 -- 172.21.15.139:0/2137477543 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b002e410 con 0x7fb7b8151290 2024-04-25T09:55:36.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7bd09e640 1 -- 172.21.15.139:0/2137477543 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b815a6a0 con 0x7fb7b8151290 2024-04-25T09:55:36.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.146+0000 7fb7af7fe640 1 -- 172.21.15.139:0/2137477543 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b002e800 con 0x7fb7b8151290 2024-04-25T09:55:36.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.147+0000 7fb7bd09e640 1 -- 172.21.15.139:0/2137477543 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b815ac60 con 0x7fb7b8151290 2024-04-25T09:55:36.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.147+0000 7fb7bd09e640 1 -- 172.21.15.139:0/2137477543 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b8074520 con 0x7fb7b8151290 2024-04-25T09:55:36.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.148+0000 7fb7af7fe640 1 -- 172.21.15.139:0/2137477543 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7b0005d10 con 0x7fb7b8151290 2024-04-25T09:55:36.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.148+0000 7fb7af7fe640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb79806b500 0x7fb79806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.148+0000 7fb7af7fe640 1 -- 172.21.15.139:0/2137477543 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb7b00aad80 con 0x7fb7b8151290 2024-04-25T09:55:36.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.148+0000 7fb7b6d9d640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb79806b500 0x7fb79806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.149+0000 7fb7b6d9d640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb79806b500 0x7fb79806d9c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8001200 tx=0x7fb7a8001e20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.152+0000 7fb7af7fe640 1 -- 172.21.15.139:0/2137477543 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7b0032030 con 0x7fb7b8151290 2024-04-25T09:55:36.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.321+0000 7fb7bd09e640 1 -- 172.21.15.139:0/2137477543 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7b80ca2d0 con 0x7fb7b8151290 2024-04-25T09:55:36.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.322+0000 7fb7af7fe640 1 -- 172.21.15.139:0/2137477543 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7b00795d0 con 0x7fb7b8151290 2024-04-25T09:55:36.325 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:55:36.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.324+0000 7fb7bd09e640 1 -- 172.21.15.139:0/2137477543 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb79806b500 msgr2=0x7fb79806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.324+0000 7fb7bd09e640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb79806b500 0x7fb79806d9c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8001200 tx=0x7fb7a8001e20 comp rx=0 tx=0).stop 2024-04-25T09:55:36.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.324+0000 7fb7bd09e640 1 -- 172.21.15.139:0/2137477543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7b8151290 msgr2=0x7fb7b8151740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.324+0000 7fb7bd09e640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7b8151290 0x7fb7b8151740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e440 tx=0x7fb7b0009750 comp rx=0 tx=0).stop 2024-04-25T09:55:36.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.324+0000 7fb7bd09e640 1 -- 172.21.15.139:0/2137477543 shutdown_connections 2024-04-25T09:55:36.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.324+0000 7fb7bd09e640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7b8151290 0x7fb7b8151740 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.324+0000 7fb7bd09e640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb79806b500 0x7fb79806d9c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.324+0000 7fb7bd09e640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b81f4170 0x7fb7b80cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.324+0000 7fb7bd09e640 1 --2- 172.21.15.139:0/2137477543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7a40051c0 0x7fb7b8159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.324+0000 7fb7bd09e640 1 -- 172.21.15.139:0/2137477543 >> 172.21.15.139:0/2137477543 conn(0x7fb7b80b5480 msgr2=0x7fb7b81588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.324+0000 7fb7bd09e640 1 -- 172.21.15.139:0/2137477543 shutdown_connections 2024-04-25T09:55:36.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.324+0000 7fb7bd09e640 1 -- 172.21.15.139:0/2137477543 wait complete. 2024-04-25T09:55:36.337 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:55:36.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.433+0000 7f2921607640 1 Processor -- start 2024-04-25T09:55:36.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.433+0000 7f2921607640 1 -- start start 2024-04-25T09:55:36.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f2921607640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f291c15dbf0 0x7f291c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f2921607640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f291c0b9370 0x7f291c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f2921607640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291c161dc0 con 0x7f291c0cec80 2024-04-25T09:55:36.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f2921607640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291c0d1ee0 con 0x7f291c0b9370 2024-04-25T09:55:36.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f2921607640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c1620a0 con 0x7f291c15dbf0 2024-04-25T09:55:36.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f291a7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f291c0b9370 0x7f291c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f291b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f291c0cec80 0x7f291c06c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37644/0 (socket says 172.21.15.139:37644) 2024-04-25T09:55:36.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f291a7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f291c0b9370 0x7f291c0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54778/0 (socket says 172.21.15.139:54778) 2024-04-25T09:55:36.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f291b7fe640 1 -- 172.21.15.139:0/750596675 learned_addr learned my addr 172.21.15.139:0/750596675 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:36.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f291affd640 1 --2- 172.21.15.139:0/750596675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f291c15dbf0 0x7f291c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f291a7fc640 1 -- 172.21.15.139:0/750596675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f291c15dbf0 msgr2=0x7f291c1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:36.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f291a7fc640 1 --2- 172.21.15.139:0/750596675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f291c15dbf0 0x7f291c1617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f291a7fc640 1 -- 172.21.15.139:0/750596675 >> v1:172.21.15.139:6789/0 conn(0x7f291c0cec80 legacy=0x7f291c06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.434+0000 7f291a7fc640 1 -- 172.21.15.139:0/750596675 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c162380 con 0x7f291c0b9370 2024-04-25T09:55:36.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.435+0000 7f291a7fc640 1 --2- 172.21.15.139:0/750596675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f291c0b9370 0x7f291c0ce740 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f29100097a0 tx=0x7f291002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafb72a01859bed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.435+0000 7f2919ffb640 1 -- 172.21.15.139:0/750596675 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910037070 con 0x7f291c0b9370 2024-04-25T09:55:36.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.435+0000 7f2919ffb640 1 -- 172.21.15.139:0/750596675 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291002ad10 con 0x7f291c0b9370 2024-04-25T09:55:36.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.435+0000 7f2919ffb640 1 -- 172.21.15.139:0/750596675 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910032660 con 0x7f291c0b9370 2024-04-25T09:55:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.436+0000 7f2921607640 1 -- 172.21.15.139:0/750596675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f291c0b9370 msgr2=0x7f291c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.436+0000 7f2921607640 1 --2- 172.21.15.139:0/750596675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f291c0b9370 0x7f291c0ce740 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f29100097a0 tx=0x7f291002a080 comp rx=0 tx=0).stop 2024-04-25T09:55:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.436+0000 7f2921607640 1 -- 172.21.15.139:0/750596675 shutdown_connections 2024-04-25T09:55:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.436+0000 7f2921607640 1 --2- 172.21.15.139:0/750596675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f291c0b9370 0x7f291c0ce740 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.436+0000 7f2921607640 1 --2- 172.21.15.139:0/750596675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f291c15dbf0 0x7f291c1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.436+0000 7f2921607640 1 -- 172.21.15.139:0/750596675 >> 172.21.15.139:0/750596675 conn(0x7f291c0b5480 msgr2=0x7f291c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.436+0000 7f2921607640 1 -- 172.21.15.139:0/750596675 shutdown_connections 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.436+0000 7f2921607640 1 -- 172.21.15.139:0/750596675 wait complete. 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.437+0000 7f2921607640 1 Processor -- start 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.437+0000 7f2921607640 1 -- start start 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f2921607640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f291c0b9370 0x7f291c152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f2921607640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f291c0cec80 0x7f291c15a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f2921607640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f291c15dbf0 0x7f291c153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f2921607640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c15dfd0 con 0x7f291c0b9370 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f2921607640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c0b97d0 con 0x7f291c0cec80 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f2921607640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c154600 con 0x7f291c15dbf0 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f291affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f291c0b9370 0x7f291c152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f291affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f291c0b9370 0x7f291c152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54794/0 (socket says 172.21.15.139:54794) 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f291affd640 1 -- 172.21.15.139:0/3485016896 learned_addr learned my addr 172.21.15.139:0/3485016896 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f291a7fc640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f291c0cec80 0x7f291c15a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f291b7fe640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f291c15dbf0 0x7f291c153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f291affd640 1 -- 172.21.15.139:0/3485016896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f291c15dbf0 msgr2=0x7f291c153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f291affd640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f291c15dbf0 0x7f291c153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f291affd640 1 -- 172.21.15.139:0/3485016896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f291c0cec80 msgr2=0x7f291c15a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f291affd640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f291c0cec80 0x7f291c15a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f291affd640 1 -- 172.21.15.139:0/3485016896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c1548e0 con 0x7f291c0b9370 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f291b7fe640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f291c15dbf0 0x7f291c153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.438+0000 7f291affd640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f291c0b9370 0x7f291c152e50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f290c007ef0 tx=0x7f290c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.439+0000 7f29037fe640 1 -- 172.21.15.139:0/3485016896 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c014620 con 0x7f291c0b9370 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.439+0000 7f29037fe640 1 -- 172.21.15.139:0/3485016896 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c019070 con 0x7f291c0b9370 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.439+0000 7f2921607640 1 -- 172.21.15.139:0/3485016896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291c15f3d0 con 0x7f291c0b9370 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.439+0000 7f29037fe640 1 -- 172.21.15.139:0/3485016896 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c014a00 con 0x7f291c0b9370 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.439+0000 7f2921607640 1 -- 172.21.15.139:0/3485016896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291c1610d0 con 0x7f291c0b9370 2024-04-25T09:55:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.440+0000 7f2921607640 1 -- 172.21.15.139:0/3485016896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291c074520 con 0x7f291c0b9370 2024-04-25T09:55:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.440+0000 7f29037fe640 1 -- 172.21.15.139:0/3485016896 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f290c014ba0 con 0x7f291c0b9370 2024-04-25T09:55:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.440+0000 7f29037fe640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f290806b4d0 0x7f290806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.441+0000 7f291a7fc640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f290806b4d0 0x7f290806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.441+0000 7f29037fe640 1 -- 172.21.15.139:0/3485016896 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f290c08ac60 con 0x7f291c0b9370 2024-04-25T09:55:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.441+0000 7f291a7fc640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f290806b4d0 0x7f290806d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2910034000 tx=0x7f291002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.445+0000 7f29037fe640 1 -- 172.21.15.139:0/3485016896 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f290c0599e0 con 0x7f291c0b9370 2024-04-25T09:55:36.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.627+0000 7f2921607640 1 -- 172.21.15.139:0/3485016896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f291c0d1680 con 0x7f291c0b9370 2024-04-25T09:55:36.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.627+0000 7f29037fe640 1 -- 172.21.15.139:0/3485016896 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) v1 ==== 80+0+0 (secure 0 0 0) 0x7f290c0592e0 con 0x7f291c0b9370 2024-04-25T09:55:36.630 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.629+0000 7f2921607640 1 -- 172.21.15.139:0/3485016896 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f290806b4d0 msgr2=0x7f290806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.629+0000 7f2921607640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f290806b4d0 0x7f290806d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2910034000 tx=0x7f291002e000 comp rx=0 tx=0).stop 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.629+0000 7f2921607640 1 -- 172.21.15.139:0/3485016896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f291c0b9370 msgr2=0x7f291c152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.629+0000 7f2921607640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f291c0b9370 0x7f291c152e50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f290c007ef0 tx=0x7f290c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.629+0000 7f2921607640 1 -- 172.21.15.139:0/3485016896 shutdown_connections 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.629+0000 7f2921607640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f290806b4d0 0x7f290806d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.629+0000 7f2921607640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f291c15dbf0 0x7f291c153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.629+0000 7f2921607640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f291c0cec80 0x7f291c15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.629+0000 7f2921607640 1 --2- 172.21.15.139:0/3485016896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f291c0b9370 0x7f291c152e50 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.629+0000 7f2921607640 1 -- 172.21.15.139:0/3485016896 >> 172.21.15.139:0/3485016896 conn(0x7f291c0b5480 msgr2=0x7f291c0d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.629+0000 7f2921607640 1 -- 172.21.15.139:0/3485016896 shutdown_connections 2024-04-25T09:55:36.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.629+0000 7f2921607640 1 -- 172.21.15.139:0/3485016896 wait complete. 2024-04-25T09:55:36.642 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:55:36.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:36.643 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:36.683 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:55:36.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:36.683 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:55:36.739 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:55:36.739 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:36.756 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:55:36.756 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:55:36.757 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:36.794 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:55:36.794 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:55:36.794 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:36.813 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:55:36.814 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:36.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.928+0000 7f2f2c709640 1 Processor -- start 2024-04-25T09:55:36.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.928+0000 7f2f2c709640 1 -- start start 2024-04-25T09:55:36.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.929+0000 7f2f2c709640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2f24155e30 0x7f2f2415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.929+0000 7f2f2c709640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f2415bdc0 0x7f2f2415c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.929+0000 7f2f2c709640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2f240b1740 0x7f2f240b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.929+0000 7f2f2c709640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2406d6f0 con 0x7f2f24155e30 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.929+0000 7f2f2c709640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2415c880 con 0x7f2f240b1740 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.929+0000 7f2f2c709640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2415cb60 con 0x7f2f2415bdc0 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.929+0000 7f2f2a4a5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2f24155e30 0x7f2f2415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.929+0000 7f2f2a4a5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2f24155e30 0x7f2f2415cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54798/0 (socket says 172.21.15.139:54798) 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.929+0000 7f2f2a4a5640 1 -- 172.21.15.139:0/1505376577 learned_addr learned my addr 172.21.15.139:0/1505376577 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.929+0000 7f2f2aca6640 1 --2- 172.21.15.139:0/1505376577 >> v2:172.21.15.150:3300/0 conn(0x7f2f240b1740 0x7f2f240b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.929+0000 7f2f29ca4640 1 --2- 172.21.15.139:0/1505376577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f2415bdc0 0x7f2f2415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.930+0000 7f2f2a4a5640 1 -- 172.21.15.139:0/1505376577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f2415bdc0 msgr2=0x7f2f2415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.930+0000 7f2f2a4a5640 1 --2- 172.21.15.139:0/1505376577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f2415bdc0 0x7f2f2415c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.930+0000 7f2f2a4a5640 1 -- 172.21.15.139:0/1505376577 >> v2:172.21.15.150:3300/0 conn(0x7f2f240b1740 msgr2=0x7f2f240b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:36.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.930+0000 7f2f2a4a5640 1 --2- 172.21.15.139:0/1505376577 >> v2:172.21.15.150:3300/0 conn(0x7f2f240b1740 0x7f2f240b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.930+0000 7f2f2a4a5640 1 -- 172.21.15.139:0/1505376577 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2406d380 con 0x7f2f24155e30 2024-04-25T09:55:36.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.930+0000 7f2f2aca6640 1 --2- 172.21.15.139:0/1505376577 >> v2:172.21.15.150:3300/0 conn(0x7f2f240b1740 0x7f2f240b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:36.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.930+0000 7f2f2a4a5640 1 --2- 172.21.15.139:0/1505376577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f24155e30 0x7f2f2415cf80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2f2415c780 tx=0x7f2f2002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8d195392c012c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.930+0000 7f2f294a3640 1 -- 172.21.15.139:0/1505376577 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20036070 con 0x7f2f24155e30 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.930+0000 7f2f294a3640 1 -- 172.21.15.139:0/1505376577 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f20031600 con 0x7f2f24155e30 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.930+0000 7f2f294a3640 1 -- 172.21.15.139:0/1505376577 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f200319d0 con 0x7f2f24155e30 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.931+0000 7f2f2c709640 1 -- 172.21.15.139:0/1505376577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f24155e30 msgr2=0x7f2f2415cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.931+0000 7f2f2c709640 1 --2- 172.21.15.139:0/1505376577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f24155e30 0x7f2f2415cf80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2f2415c780 tx=0x7f2f2002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.931+0000 7f2f2c709640 1 -- 172.21.15.139:0/1505376577 shutdown_connections 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.931+0000 7f2f2c709640 1 --2- 172.21.15.139:0/1505376577 >> v2:172.21.15.150:3300/0 conn(0x7f2f240b1740 0x7f2f240b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.931+0000 7f2f2c709640 1 --2- 172.21.15.139:0/1505376577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f2415bdc0 0x7f2f2415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.931+0000 7f2f2c709640 1 --2- 172.21.15.139:0/1505376577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f24155e30 0x7f2f2415cf80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.931+0000 7f2f2c709640 1 -- 172.21.15.139:0/1505376577 >> 172.21.15.139:0/1505376577 conn(0x7f2f24066f20 msgr2=0x7f2f2406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.931+0000 7f2f2c709640 1 -- 172.21.15.139:0/1505376577 shutdown_connections 2024-04-25T09:55:36.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.931+0000 7f2f2c709640 1 -- 172.21.15.139:0/1505376577 wait complete. 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.932+0000 7f2f2c709640 1 Processor -- start 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.932+0000 7f2f2c709640 1 -- start start 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f2c709640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f240b1740 0x7f2f241e5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f2c709640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f24155e30 0x7f2f2415b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f2c709640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f2415bdc0 0x7f2f24158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f2c709640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240af260 con 0x7f2f24155e30 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f2c709640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240af0e0 con 0x7f2f2415bdc0 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f2c709640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2415d4c0 con 0x7f2f240b1740 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f29ca4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f24155e30 0x7f2f2415b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f29ca4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f24155e30 0x7f2f2415b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54806/0 (socket says 172.21.15.139:54806) 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f29ca4640 1 -- 172.21.15.139:0/1955212427 learned_addr learned my addr 172.21.15.139:0/1955212427 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f2aca6640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f2415bdc0 0x7f2f24158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f2a4a5640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f240b1740 0x7f2f241e5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f29ca4640 1 -- 172.21.15.139:0/1955212427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f240b1740 msgr2=0x7f2f241e5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f29ca4640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f240b1740 0x7f2f241e5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f29ca4640 1 -- 172.21.15.139:0/1955212427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f2415bdc0 msgr2=0x7f2f24158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f29ca4640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f2415bdc0 0x7f2f24158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f29ca4640 1 -- 172.21.15.139:0/1955212427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f241e6b30 con 0x7f2f24155e30 2024-04-25T09:55:36.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.933+0000 7f2f2aca6640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f2415bdc0 0x7f2f24158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:36.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.934+0000 7f2f29ca4640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f24155e30 0x7f2f2415b2e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400d5f0 tx=0x7f2f1400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.934+0000 7f2f1affd640 1 -- 172.21.15.139:0/1955212427 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140125f0 con 0x7f2f24155e30 2024-04-25T09:55:36.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.934+0000 7f2f1affd640 1 -- 172.21.15.139:0/1955212427 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f14019070 con 0x7f2f24155e30 2024-04-25T09:55:36.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.934+0000 7f2f2c709640 1 -- 172.21.15.139:0/1955212427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f241e6290 con 0x7f2f24155e30 2024-04-25T09:55:36.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.934+0000 7f2f1affd640 1 -- 172.21.15.139:0/1955212427 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140129d0 con 0x7f2f24155e30 2024-04-25T09:55:36.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.934+0000 7f2f2c709640 1 -- 172.21.15.139:0/1955212427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f241e6850 con 0x7f2f24155e30 2024-04-25T09:55:36.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.935+0000 7f2f2c709640 1 -- 172.21.15.139:0/1955212427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f24074520 con 0x7f2f24155e30 2024-04-25T09:55:36.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.935+0000 7f2f1affd640 1 -- 172.21.15.139:0/1955212427 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f14012b70 con 0x7f2f24155e30 2024-04-25T09:55:36.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.936+0000 7f2f1affd640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2f0806b4d0 0x7f2f0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.936+0000 7f2f1affd640 1 -- 172.21.15.139:0/1955212427 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f1408a7a0 con 0x7f2f24155e30 2024-04-25T09:55:36.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.936+0000 7f2f2a4a5640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2f0806b4d0 0x7f2f0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.936+0000 7f2f2a4a5640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2f0806b4d0 0x7f2f0806d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2f20033000 tx=0x7f2f20009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:36.940+0000 7f2f1affd640 1 -- 172.21.15.139:0/1955212427 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f14059490 con 0x7f2f24155e30 2024-04-25T09:55:37.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.150+0000 7f2f2c709640 1 -- 172.21.15.139:0/1955212427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f2406b830 con 0x7f2f24155e30 2024-04-25T09:55:37.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.150+0000 7f2f1affd640 1 -- 172.21.15.139:0/1955212427 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f2f14058d90 con 0x7f2f24155e30 2024-04-25T09:55:37.153 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:37.153 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1742557158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1742557158},{"type":"v1","addr":"172.21.15.150:6834","nonce":1742557158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3496803834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3496803834},{"type":"v1","addr":"172.21.15.150:6835","nonce":3496803834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2014537692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2014537692},{"type":"v1","addr":"172.21.15.139:6835","nonce":2014537692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:56.466203+0000","modified":"2024-04-25T09:54:58.466766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5850":{"gid":5850,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/338275708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":338275708},{"type":"v1","addr":"172.21.15.139:6837","nonce":338275708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:55:37.154 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 22 2024-04-25T09:55:37.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.153+0000 7f2f2c709640 1 -- 172.21.15.139:0/1955212427 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2f0806b4d0 msgr2=0x7f2f0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.153+0000 7f2f2c709640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2f0806b4d0 0x7f2f0806d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2f20033000 tx=0x7f2f20009000 comp rx=0 tx=0).stop 2024-04-25T09:55:37.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.153+0000 7f2f2c709640 1 -- 172.21.15.139:0/1955212427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f24155e30 msgr2=0x7f2f2415b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.153+0000 7f2f2c709640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f24155e30 0x7f2f2415b2e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400d5f0 tx=0x7f2f1400dab0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.153+0000 7f2f2c709640 1 -- 172.21.15.139:0/1955212427 shutdown_connections 2024-04-25T09:55:37.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.153+0000 7f2f2c709640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f24155e30 0x7f2f2415b2e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.153+0000 7f2f2c709640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2f0806b4d0 0x7f2f0806d990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.153+0000 7f2f2c709640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f2415bdc0 0x7f2f24158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.153+0000 7f2f2c709640 1 --2- 172.21.15.139:0/1955212427 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f240b1740 0x7f2f241e5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.153+0000 7f2f2c709640 1 -- 172.21.15.139:0/1955212427 >> 172.21.15.139:0/1955212427 conn(0x7f2f24066f20 msgr2=0x7f2f2415ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.153+0000 7f2f2c709640 1 -- 172.21.15.139:0/1955212427 shutdown_connections 2024-04-25T09:55:37.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.153+0000 7f2f2c709640 1 -- 172.21.15.139:0/1955212427 wait complete. 2024-04-25T09:55:37.166 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:37.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.260+0000 7f1d16415640 1 Processor -- start 2024-04-25T09:55:37.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.260+0000 7f1d16415640 1 -- start start 2024-04-25T09:55:37.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d16415640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1d1006ab60 0x7f1d1006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d16415640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1d100cf960 0x7f1d100cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d16415640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d10070a40 con 0x7f1d1015b950 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d16415640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d10073170 con 0x7f1d100cf960 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d16415640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1006b500 con 0x7f1d1006ab60 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d149b2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1d100cf960 0x7f1d100cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d149b2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1d100cf960 0x7f1d100cfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54820/0 (socket says 172.21.15.139:54820) 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d149b2640 1 -- 172.21.15.139:0/3353680243 learned_addr learned my addr 172.21.15.139:0/3353680243 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d0ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1d1015b950 0x7f1d10072a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37646/0 (socket says 172.21.15.139:37646) 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d0f7fe640 1 --2- 172.21.15.139:0/3353680243 >> v2:172.21.15.150:3300/0 conn(0x7f1d1006ab60 0x7f1d1006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d149b2640 1 -- 172.21.15.139:0/3353680243 >> v2:172.21.15.150:3300/0 conn(0x7f1d1006ab60 msgr2=0x7f1d1006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d149b2640 1 --2- 172.21.15.139:0/3353680243 >> v2:172.21.15.150:3300/0 conn(0x7f1d1006ab60 0x7f1d1006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d149b2640 1 -- 172.21.15.139:0/3353680243 >> v1:172.21.15.139:6789/0 conn(0x7f1d1015b950 legacy=0x7f1d10072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d149b2640 1 -- 172.21.15.139:0/3353680243 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1006fc80 con 0x7f1d100cf960 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.261+0000 7f1d0effd640 1 -- 172.21.15.139:0/3353680243 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414388085 0 0) 0x7f1d10070a40 con 0x7f1d1015b950 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.262+0000 7f1d149b2640 1 --2- 172.21.15.139:0/3353680243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d100cf960 0x7f1d100cfdf0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1d100d1680 tx=0x7f1d0802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a408c9afeb5a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.262+0000 7f1d0effd640 1 -- 172.21.15.139:0/3353680243 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08036070 con 0x7f1d100cf960 2024-04-25T09:55:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.262+0000 7f1d0effd640 1 -- 172.21.15.139:0/3353680243 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d08031600 con 0x7f1d100cf960 2024-04-25T09:55:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.262+0000 7f1d0effd640 1 -- 172.21.15.139:0/3353680243 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d080319d0 con 0x7f1d100cf960 2024-04-25T09:55:37.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.262+0000 7f1d16415640 1 -- 172.21.15.139:0/3353680243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d100cf960 msgr2=0x7f1d100cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.262+0000 7f1d16415640 1 --2- 172.21.15.139:0/3353680243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d100cf960 0x7f1d100cfdf0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1d100d1680 tx=0x7f1d0802a730 comp rx=0 tx=0).stop 2024-04-25T09:55:37.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.262+0000 7f1d16415640 1 -- 172.21.15.139:0/3353680243 shutdown_connections 2024-04-25T09:55:37.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.262+0000 7f1d16415640 1 --2- 172.21.15.139:0/3353680243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d100cf960 0x7f1d100cfdf0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.262+0000 7f1d16415640 1 --2- 172.21.15.139:0/3353680243 >> v2:172.21.15.150:3300/0 conn(0x7f1d1006ab60 0x7f1d1006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.262+0000 7f1d16415640 1 -- 172.21.15.139:0/3353680243 >> 172.21.15.139:0/3353680243 conn(0x7f1d100b23f0 msgr2=0x7f1d100b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.263+0000 7f1d16415640 1 -- 172.21.15.139:0/3353680243 shutdown_connections 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.263+0000 7f1d16415640 1 -- 172.21.15.139:0/3353680243 wait complete. 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.264+0000 7f1d16415640 1 Processor -- start 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.264+0000 7f1d16415640 1 -- start start 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.264+0000 7f1d16415640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d1006ab60 0x7f1d100aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.264+0000 7f1d16415640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d100cf960 0x7f1d1015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.264+0000 7f1d16415640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d1015b950 0x7f1d100b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.264+0000 7f1d16415640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100cda00 con 0x7f1d100cf960 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.264+0000 7f1d16415640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d10070240 con 0x7f1d1006ab60 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.264+0000 7f1d16415640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100cdce0 con 0x7f1d1015b950 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d100cf960 0x7f1d1015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d100cf960 0x7f1d1015e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54834/0 (socket says 172.21.15.139:54834) 2024-04-25T09:55:37.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0f7fe640 1 -- 172.21.15.139:0/505461404 learned_addr learned my addr 172.21.15.139:0/505461404 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0ffff640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d1006ab60 0x7f1d100aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d149b2640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d1015b950 0x7f1d100b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0f7fe640 1 -- 172.21.15.139:0/505461404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d1015b950 msgr2=0x7f1d100b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0f7fe640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d1015b950 0x7f1d100b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0f7fe640 1 -- 172.21.15.139:0/505461404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d1006ab60 msgr2=0x7f1d100aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0f7fe640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d1006ab60 0x7f1d100aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0f7fe640 1 -- 172.21.15.139:0/505461404 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d100af220 con 0x7f1d100cf960 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d149b2640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d1015b950 0x7f1d100b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0f7fe640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d100cf960 0x7f1d1015e8c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000e9b0 tx=0x7f1d0000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0cff9640 1 -- 172.21.15.139:0/505461404 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00017070 con 0x7f1d100cf960 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0cff9640 1 -- 172.21.15.139:0/505461404 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0000c9a0 con 0x7f1d100cf960 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d16415640 1 -- 172.21.15.139:0/505461404 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1015f260 con 0x7f1d100cf960 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d0cff9640 1 -- 172.21.15.139:0/505461404 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000cd90 con 0x7f1d100cf960 2024-04-25T09:55:37.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.265+0000 7f1d16415640 1 -- 172.21.15.139:0/505461404 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1015f820 con 0x7f1d100cf960 2024-04-25T09:55:37.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.266+0000 7f1d16415640 1 -- 172.21.15.139:0/505461404 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d10074520 con 0x7f1d100cf960 2024-04-25T09:55:37.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.267+0000 7f1d0cff9640 1 -- 172.21.15.139:0/505461404 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d0001a070 con 0x7f1d100cf960 2024-04-25T09:55:37.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.267+0000 7f1d0cff9640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1cfc06b4d0 0x7f1cfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.267+0000 7f1d0ffff640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1cfc06b4d0 0x7f1cfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.267+0000 7f1d0cff9640 1 -- 172.21.15.139:0/505461404 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1d0008a260 con 0x7f1d100cf960 2024-04-25T09:55:37.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.267+0000 7f1d0ffff640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1cfc06b4d0 0x7f1cfc06d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1d04007fd0 tx=0x7f1d04007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.271+0000 7f1d0cff9640 1 -- 172.21.15.139:0/505461404 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d00058990 con 0x7f1d100cf960 2024-04-25T09:55:37.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.445+0000 7f1d16415640 1 -- 172.21.15.139:0/505461404 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d1006d2b0 con 0x7f1d100cf960 2024-04-25T09:55:37.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.446+0000 7f1d0cff9640 1 -- 172.21.15.139:0/505461404 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f1d0005ca30 con 0x7f1d100cf960 2024-04-25T09:55:37.449 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:37.449 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":50,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:58.130580+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:54.661529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:55.431933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:37.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.448+0000 7f1d16415640 1 -- 172.21.15.139:0/505461404 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1cfc06b4d0 msgr2=0x7f1cfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.448+0000 7f1d16415640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1cfc06b4d0 0x7f1cfc06d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1d04007fd0 tx=0x7f1d04007de0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.448+0000 7f1d16415640 1 -- 172.21.15.139:0/505461404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d100cf960 msgr2=0x7f1d1015e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.448+0000 7f1d16415640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d100cf960 0x7f1d1015e8c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000e9b0 tx=0x7f1d0000ee70 comp rx=0 tx=0).stop 2024-04-25T09:55:37.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.448+0000 7f1d16415640 1 -- 172.21.15.139:0/505461404 shutdown_connections 2024-04-25T09:55:37.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.448+0000 7f1d16415640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d100cf960 0x7f1d1015e8c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.448+0000 7f1d16415640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1cfc06b4d0 0x7f1cfc06d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.448+0000 7f1d16415640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d1015b950 0x7f1d100b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.448+0000 7f1d16415640 1 --2- 172.21.15.139:0/505461404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d1006ab60 0x7f1d100aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.448+0000 7f1d16415640 1 -- 172.21.15.139:0/505461404 >> 172.21.15.139:0/505461404 conn(0x7f1d100b23f0 msgr2=0x7f1d100d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.448+0000 7f1d16415640 1 -- 172.21.15.139:0/505461404 shutdown_connections 2024-04-25T09:55:37.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.448+0000 7f1d16415640 1 -- 172.21.15.139:0/505461404 wait complete. 2024-04-25T09:55:37.462 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:54.661529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:55.431933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:55:37.462 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:37.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.559+0000 7f79487f9640 1 Processor -- start 2024-04-25T09:55:37.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.559+0000 7f79487f9640 1 -- start start 2024-04-25T09:55:37.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.560+0000 7f79487f9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f79400d0080 0x7f79401664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.560+0000 7f79487f9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79400ccef0 con 0x7f794015cb20 2024-04-25T09:55:37.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.560+0000 7f79487f9640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79400cd1d0 con 0x7f79400afc00 2024-04-25T09:55:37.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.560+0000 7f79487f9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79400703c0 con 0x7f79400d0080 2024-04-25T09:55:37.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.560+0000 7f7945d94640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f794015cb20 0x7f79400cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37656/0 (socket says 172.21.15.139:37656) 2024-04-25T09:55:37.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.560+0000 7f7945d94640 1 -- 172.21.15.139:0/1676150201 learned_addr learned my addr 172.21.15.139:0/1676150201 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:37.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.560+0000 7f7946d96640 1 --2- 172.21.15.139:0/1676150201 >> v2:172.21.15.150:3300/0 conn(0x7f79400d0080 0x7f79401664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.560+0000 7f7945593640 1 -- 172.21.15.139:0/1676150201 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396776047 0 0) 0x7f79400ccef0 con 0x7f794015cb20 2024-04-25T09:55:37.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.560+0000 7f7945593640 1 -- 172.21.15.139:0/1676150201 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f792c003520 con 0x7f794015cb20 2024-04-25T09:55:37.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.561+0000 7f7946d96640 1 -- 172.21.15.139:0/1676150201 >> v1:172.21.15.150:6789/0 conn(0x7f79400afc00 legacy=0x7f794006fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.561+0000 7f7946d96640 1 -- 172.21.15.139:0/1676150201 >> v1:172.21.15.139:6789/0 conn(0x7f794015cb20 legacy=0x7f79400cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.561+0000 7f7946d96640 1 -- 172.21.15.139:0/1676150201 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400cd4b0 con 0x7f79400d0080 2024-04-25T09:55:37.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.561+0000 7f7946d96640 1 --2- 172.21.15.139:0/1676150201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79400d0080 0x7f79401664d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7938009d90 tx=0x7f793802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=630d2b98df1f516d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.561+0000 7f7945593640 1 -- 172.21.15.139:0/1676150201 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938031600 con 0x7f79400d0080 2024-04-25T09:55:37.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.562+0000 7f7945593640 1 -- 172.21.15.139:0/1676150201 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79380317a0 con 0x7f79400d0080 2024-04-25T09:55:37.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.562+0000 7f7945593640 1 -- 172.21.15.139:0/1676150201 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938031b50 con 0x7f79400d0080 2024-04-25T09:55:37.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.562+0000 7f79487f9640 1 -- 172.21.15.139:0/1676150201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79400d0080 msgr2=0x7f79401664d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.562+0000 7f79487f9640 1 --2- 172.21.15.139:0/1676150201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79400d0080 0x7f79401664d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7938009d90 tx=0x7f793802a5d0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.562+0000 7f79487f9640 1 -- 172.21.15.139:0/1676150201 shutdown_connections 2024-04-25T09:55:37.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.562+0000 7f79487f9640 1 --2- 172.21.15.139:0/1676150201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79400d0080 0x7f79401664d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.562+0000 7f79487f9640 1 -- 172.21.15.139:0/1676150201 >> 172.21.15.139:0/1676150201 conn(0x7f79400b2230 msgr2=0x7f79400b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.562+0000 7f79487f9640 1 -- 172.21.15.139:0/1676150201 shutdown_connections 2024-04-25T09:55:37.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.562+0000 7f79487f9640 1 -- 172.21.15.139:0/1676150201 wait complete. 2024-04-25T09:55:37.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.563+0000 7f79487f9640 1 Processor -- start 2024-04-25T09:55:37.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.563+0000 7f79487f9640 1 -- start start 2024-04-25T09:55:37.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.564+0000 7f79487f9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79400afc00 0x7f7940156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.564+0000 7f79487f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79400d0080 0x7f79400c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.564+0000 7f79487f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f794015cb20 0x7f7940143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.564+0000 7f79487f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400affe0 con 0x7f794015cb20 2024-04-25T09:55:37.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.564+0000 7f79487f9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400cd820 con 0x7f79400afc00 2024-04-25T09:55:37.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.564+0000 7f79487f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7940166a10 con 0x7f79400d0080 2024-04-25T09:55:37.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.564+0000 7f7946d96640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f794015cb20 0x7f7940143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.564+0000 7f7946d96640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f794015cb20 0x7f7940143b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54842/0 (socket says 172.21.15.139:54842) 2024-04-25T09:55:37.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.564+0000 7f7946d96640 1 -- 172.21.15.139:0/874151994 learned_addr learned my addr 172.21.15.139:0/874151994 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:37.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.564+0000 7f7946595640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79400afc00 0x7f7940156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.564+0000 7f7945d94640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79400d0080 0x7f79400c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.565+0000 7f7946d96640 1 -- 172.21.15.139:0/874151994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79400d0080 msgr2=0x7f79400c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.565+0000 7f7946d96640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79400d0080 0x7f79400c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.565+0000 7f7946d96640 1 -- 172.21.15.139:0/874151994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79400afc00 msgr2=0x7f7940156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.565+0000 7f7946d96640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79400afc00 0x7f7940156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.565+0000 7f7946d96640 1 -- 172.21.15.139:0/874151994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400c5c30 con 0x7f794015cb20 2024-04-25T09:55:37.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.565+0000 7f7946d96640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f794015cb20 0x7f7940143b20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f793802abc0 tx=0x7f7938004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.565+0000 7f7936ffd640 1 -- 172.21.15.139:0/874151994 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938009450 con 0x7f794015cb20 2024-04-25T09:55:37.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.565+0000 7f7936ffd640 1 -- 172.21.15.139:0/874151994 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7938031b50 con 0x7f794015cb20 2024-04-25T09:55:37.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.565+0000 7f79487f9640 1 -- 172.21.15.139:0/874151994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79400c5f10 con 0x7f794015cb20 2024-04-25T09:55:37.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.565+0000 7f7936ffd640 1 -- 172.21.15.139:0/874151994 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938033070 con 0x7f794015cb20 2024-04-25T09:55:37.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.565+0000 7f79487f9640 1 -- 172.21.15.139:0/874151994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79400c64d0 con 0x7f794015cb20 2024-04-25T09:55:37.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.566+0000 7f79487f9640 1 -- 172.21.15.139:0/874151994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7940074520 con 0x7f794015cb20 2024-04-25T09:55:37.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.566+0000 7f7936ffd640 1 -- 172.21.15.139:0/874151994 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7938033210 con 0x7f794015cb20 2024-04-25T09:55:37.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.567+0000 7f7936ffd640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f792406b4d0 0x7f792406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.567+0000 7f7946595640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f792406b4d0 0x7f792406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.567+0000 7f7936ffd640 1 -- 172.21.15.139:0/874151994 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79380ab110 con 0x7f794015cb20 2024-04-25T09:55:37.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.567+0000 7f7946595640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f792406b4d0 0x7f792406d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f793c0016e0 tx=0x7f793c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.571+0000 7f7936ffd640 1 -- 172.21.15.139:0/874151994 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f793803f030 con 0x7f794015cb20 2024-04-25T09:55:37.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.739+0000 7f79487f9640 1 -- 172.21.15.139:0/874151994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f794015eb00 con 0x7f794015cb20 2024-04-25T09:55:37.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.739+0000 7f7936ffd640 1 -- 172.21.15.139:0/874151994 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79380797f0 con 0x7f794015cb20 2024-04-25T09:55:37.742 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:55:37.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.741+0000 7f79487f9640 1 -- 172.21.15.139:0/874151994 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f792406b4d0 msgr2=0x7f792406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.741+0000 7f79487f9640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f792406b4d0 0x7f792406d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f793c0016e0 tx=0x7f793c009040 comp rx=0 tx=0).stop 2024-04-25T09:55:37.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.741+0000 7f79487f9640 1 -- 172.21.15.139:0/874151994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f794015cb20 msgr2=0x7f7940143b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.741+0000 7f79487f9640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f794015cb20 0x7f7940143b20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f793802abc0 tx=0x7f7938004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:37.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.742+0000 7f79487f9640 1 -- 172.21.15.139:0/874151994 shutdown_connections 2024-04-25T09:55:37.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.742+0000 7f79487f9640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f794015cb20 0x7f7940143b20 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.742+0000 7f79487f9640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f792406b4d0 0x7f792406d990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.742+0000 7f79487f9640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79400d0080 0x7f79400c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.742+0000 7f79487f9640 1 --2- 172.21.15.139:0/874151994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79400afc00 0x7f7940156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.742+0000 7f79487f9640 1 -- 172.21.15.139:0/874151994 >> 172.21.15.139:0/874151994 conn(0x7f79400b2230 msgr2=0x7f79400b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.742+0000 7f79487f9640 1 -- 172.21.15.139:0/874151994 shutdown_connections 2024-04-25T09:55:37.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.742+0000 7f79487f9640 1 -- 172.21.15.139:0/874151994 wait complete. 2024-04-25T09:55:37.755 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:37.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.852+0000 7fefbbdce640 1 Processor -- start 2024-04-25T09:55:37.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.852+0000 7fefbbdce640 1 -- start start 2024-04-25T09:55:37.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.853+0000 7fefbbdce640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fefb40afc80 0x7fefb40b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.853+0000 7fefbbdce640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb40c6cc0 con 0x7fefb406b700 2024-04-25T09:55:37.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.853+0000 7fefbbdce640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb40c6fa0 con 0x7fefb40af8a0 2024-04-25T09:55:37.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.853+0000 7fefbbdce640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb40c7280 con 0x7fefb40afc80 2024-04-25T09:55:37.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.853+0000 7fefb9369640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fefb40afc80 0x7fefb40b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.853+0000 7fefba36b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fefb406b700 0x7fefb406ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37662/0 (socket says 172.21.15.139:37662) 2024-04-25T09:55:37.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.853+0000 7fefba36b640 1 -- 172.21.15.139:0/1327361009 learned_addr learned my addr 172.21.15.139:0/1327361009 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:37.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.854+0000 7fefb9369640 1 -- 172.21.15.139:0/1327361009 >> v1:172.21.15.150:6789/0 conn(0x7fefb40af8a0 legacy=0x7fefb40cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.854+0000 7fefb9369640 1 -- 172.21.15.139:0/1327361009 >> v1:172.21.15.139:6789/0 conn(0x7fefb406b700 legacy=0x7fefb406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.854+0000 7fefb9369640 1 -- 172.21.15.139:0/1327361009 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb40cd090 con 0x7fefb40afc80 2024-04-25T09:55:37.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.854+0000 7fefb8b68640 1 -- 172.21.15.139:0/1327361009 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718985298 0 0) 0x7fefb40c6cc0 con 0x7fefb406b700 2024-04-25T09:55:37.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.854+0000 7fefb9369640 1 --2- 172.21.15.139:0/1327361009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefb40afc80 0x7fefb40b2d50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fefa40097a0 tx=0x7fefa402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8208fdbfb074a8c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.854+0000 7fefb8b68640 1 -- 172.21.15.139:0/1327361009 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4037070 con 0x7fefb40afc80 2024-04-25T09:55:37.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.854+0000 7fefb8b68640 1 -- 172.21.15.139:0/1327361009 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa4032440 con 0x7fefb40afc80 2024-04-25T09:55:37.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.854+0000 7fefb8b68640 1 -- 172.21.15.139:0/1327361009 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4032810 con 0x7fefb40afc80 2024-04-25T09:55:37.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.855+0000 7fefbbdce640 1 -- 172.21.15.139:0/1327361009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefb40afc80 msgr2=0x7fefb40b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.855+0000 7fefbbdce640 1 --2- 172.21.15.139:0/1327361009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefb40afc80 0x7fefb40b2d50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fefa40097a0 tx=0x7fefa402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.855+0000 7fefbbdce640 1 -- 172.21.15.139:0/1327361009 shutdown_connections 2024-04-25T09:55:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.855+0000 7fefbbdce640 1 --2- 172.21.15.139:0/1327361009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefb40afc80 0x7fefb40b2d50 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.855+0000 7fefbbdce640 1 -- 172.21.15.139:0/1327361009 >> 172.21.15.139:0/1327361009 conn(0x7fefb40b2230 msgr2=0x7fefb40b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.855+0000 7fefbbdce640 1 -- 172.21.15.139:0/1327361009 shutdown_connections 2024-04-25T09:55:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.855+0000 7fefbbdce640 1 -- 172.21.15.139:0/1327361009 wait complete. 2024-04-25T09:55:37.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.856+0000 7fefbbdce640 1 Processor -- start 2024-04-25T09:55:37.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.856+0000 7fefbbdce640 1 -- start start 2024-04-25T09:55:37.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefbbdce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefb406b700 0x7fefb40b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefbbdce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefb40af8a0 0x7fefb414f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefbbdce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefb40afc80 0x7fefb409d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefbbdce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb406b3f0 con 0x7fefb40afc80 2024-04-25T09:55:37.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefbbdce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb406b270 con 0x7fefb40af8a0 2024-04-25T09:55:37.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefbbdce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40c54a0 con 0x7fefb406b700 2024-04-25T09:55:37.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefba36b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefb40afc80 0x7fefb409d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefba36b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefb40afc80 0x7fefb409d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54860/0 (socket says 172.21.15.139:54860) 2024-04-25T09:55:37.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefba36b640 1 -- 172.21.15.139:0/2609746206 learned_addr learned my addr 172.21.15.139:0/2609746206 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:37.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefb9369640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefb40af8a0 0x7fefb414f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefb9b6a640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefb406b700 0x7fefb40b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefba36b640 1 -- 172.21.15.139:0/2609746206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefb406b700 msgr2=0x7fefb40b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefba36b640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefb406b700 0x7fefb40b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefba36b640 1 -- 172.21.15.139:0/2609746206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefb40af8a0 msgr2=0x7fefb414f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefba36b640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefb40af8a0 0x7fefb414f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefba36b640 1 -- 172.21.15.139:0/2609746206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb40b4db0 con 0x7fefb40afc80 2024-04-25T09:55:37.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefb9369640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefb40af8a0 0x7fefb414f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:37.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefb9b6a640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefb406b700 0x7fefb40b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:37.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.857+0000 7fefba36b640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefb40afc80 0x7fefb409d010 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fefac00c7a0 tx=0x7fefac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.858+0000 7fefaa7fc640 1 -- 172.21.15.139:0/2609746206 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac018070 con 0x7fefb40afc80 2024-04-25T09:55:37.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.858+0000 7fefaa7fc640 1 -- 172.21.15.139:0/2609746206 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac00a640 con 0x7fefb40afc80 2024-04-25T09:55:37.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.858+0000 7fefbbdce640 1 -- 172.21.15.139:0/2609746206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb40b5090 con 0x7fefb40afc80 2024-04-25T09:55:37.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.858+0000 7fefaa7fc640 1 -- 172.21.15.139:0/2609746206 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac00aab0 con 0x7fefb40afc80 2024-04-25T09:55:37.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.858+0000 7fefbbdce640 1 -- 172.21.15.139:0/2609746206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb409a8e0 con 0x7fefb40afc80 2024-04-25T09:55:37.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.859+0000 7fefbbdce640 1 -- 172.21.15.139:0/2609746206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb40c69b0 con 0x7fefb40afc80 2024-04-25T09:55:37.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.859+0000 7fefaa7fc640 1 -- 172.21.15.139:0/2609746206 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefac01b070 con 0x7fefb40afc80 2024-04-25T09:55:37.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.859+0000 7fefaa7fc640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef9806b4d0 0x7fef9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.859+0000 7fefaa7fc640 1 -- 172.21.15.139:0/2609746206 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fefac0899d0 con 0x7fefb40afc80 2024-04-25T09:55:37.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.859+0000 7fefb9b6a640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef9806b4d0 0x7fef9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.860+0000 7fefb9b6a640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef9806b4d0 0x7fef9806d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fefb0000f60 tx=0x7fefb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:37.864+0000 7fefaa7fc640 1 -- 172.21.15.139:0/2609746206 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fefb40c69b0 con 0x7fefb40afc80 2024-04-25T09:55:38.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.034+0000 7fefbbdce640 1 -- 172.21.15.139:0/2609746206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fefb4074520 con 0x7fefb40afc80 2024-04-25T09:55:38.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.034+0000 7fefaa7fc640 1 -- 172.21.15.139:0/2609746206 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7fefac0586c0 con 0x7fefb40afc80 2024-04-25T09:55:38.037 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:55:38.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.036+0000 7fefbbdce640 1 -- 172.21.15.139:0/2609746206 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef9806b4d0 msgr2=0x7fef9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.036+0000 7fefbbdce640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef9806b4d0 0x7fef9806d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fefb0000f60 tx=0x7fefb0009040 comp rx=0 tx=0).stop 2024-04-25T09:55:38.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.036+0000 7fefbbdce640 1 -- 172.21.15.139:0/2609746206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefb40afc80 msgr2=0x7fefb409d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.036+0000 7fefbbdce640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefb40afc80 0x7fefb409d010 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fefac00c7a0 tx=0x7fefac00cc60 comp rx=0 tx=0).stop 2024-04-25T09:55:38.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.036+0000 7fefbbdce640 1 -- 172.21.15.139:0/2609746206 shutdown_connections 2024-04-25T09:55:38.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.036+0000 7fefbbdce640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefb40afc80 0x7fefb409d010 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.036+0000 7fefbbdce640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef9806b4d0 0x7fef9806d990 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.036+0000 7fefbbdce640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefb40af8a0 0x7fefb414f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.036+0000 7fefbbdce640 1 --2- 172.21.15.139:0/2609746206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefb406b700 0x7fefb40b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.036+0000 7fefbbdce640 1 -- 172.21.15.139:0/2609746206 >> 172.21.15.139:0/2609746206 conn(0x7fefb40b2230 msgr2=0x7fefb40aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.036+0000 7fefbbdce640 1 -- 172.21.15.139:0/2609746206 shutdown_connections 2024-04-25T09:55:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.036+0000 7fefbbdce640 1 -- 172.21.15.139:0/2609746206 wait complete. 2024-04-25T09:55:38.050 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:38.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.147+0000 7f724adee640 1 Processor -- start 2024-04-25T09:55:38.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.147+0000 7f724adee640 1 -- start start 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.148+0000 7f724adee640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f72440d0a80 0x7f72441618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.148+0000 7f724adee640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7244099ca0 con 0x7f72440cd610 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.148+0000 7f724adee640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7244099f80 con 0x7f724409ad40 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.148+0000 7f724adee640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7244099a50 con 0x7f72440d0a80 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.148+0000 7f7248b8a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f72440cd610 0x7f72440d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37678/0 (socket says 172.21.15.139:37678) 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.148+0000 7f7248b8a640 1 -- 172.21.15.139:0/1775265839 learned_addr learned my addr 172.21.15.139:0/1775265839 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.148+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61143331 0 0) 0x7f7244099ca0 con 0x7f72440cd610 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.148+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f722c003520 con 0x7f72440cd610 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.148+0000 7f7243fff640 1 --2- 172.21.15.139:0/1775265839 >> v2:172.21.15.150:3300/0 conn(0x7f72440d0a80 0x7f72441618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 218014332 0 0) 0x7f722c003520 con 0x7f72440cd610 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 >> v2:172.21.15.150:3300/0 conn(0x7f72440d0a80 msgr2=0x7f72441618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 --2- 172.21.15.139:0/1775265839 >> v2:172.21.15.150:3300/0 conn(0x7f72440d0a80 0x7f72441618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 >> v1:172.21.15.150:6789/0 conn(0x7f724409ad40 legacy=0x7f72440cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724409a260 con 0x7f72440cd610 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f7243fff640 1 --2- 172.21.15.139:0/1775265839 >> v2:172.21.15.150:3300/0 conn(0x7f72440d0a80 0x7f72441618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f7234001670 con 0x7f72440cd610 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 >> v1:172.21.15.139:6789/0 conn(0x7f72440cd610 legacy=0x7f72440d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 --2- 172.21.15.139:0/1775265839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c0039b0 0x7f722c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 --2- 172.21.15.139:0/1775265839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c004800 0x7f722c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 --2- 172.21.15.139:0/1775265839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0051c0 0x7f722c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7244099a50 con 0x7f722c0051c0 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c00c0f0 con 0x7f722c004800 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c00c2f0 con 0x7f722c0039b0 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f724938b640 1 --2- 172.21.15.139:0/1775265839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0051c0 0x7f722c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f7243fff640 1 --2- 172.21.15.139:0/1775265839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c004800 0x7f722c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f7248b8a640 1 --2- 172.21.15.139:0/1775265839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c0039b0 0x7f722c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f724938b640 1 -- 172.21.15.139:0/1775265839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c0039b0 msgr2=0x7f722c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f724938b640 1 --2- 172.21.15.139:0/1775265839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c0039b0 0x7f722c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f724938b640 1 -- 172.21.15.139:0/1775265839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c004800 msgr2=0x7f722c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f724938b640 1 --2- 172.21.15.139:0/1775265839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c004800 0x7f722c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.149+0000 7f724938b640 1 -- 172.21.15.139:0/1775265839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c003520 con 0x7f722c0051c0 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.150+0000 7f724938b640 1 -- 172.21.15.139:0/1775265839 reap_dead start 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.150+0000 7f724938b640 1 --2- 172.21.15.139:0/1775265839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0051c0 0x7f722c00b9c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f723c001e50 tx=0x7f722c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96210a884e279ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.150+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c001a70 con 0x7f722c0051c0 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.150+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c003970 con 0x7f722c0051c0 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.150+0000 7f72437fe640 1 -- 172.21.15.139:0/1775265839 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c003d60 con 0x7f722c0051c0 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.150+0000 7f724adee640 1 -- 172.21.15.139:0/1775265839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0051c0 msgr2=0x7f722c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.150+0000 7f724adee640 1 --2- 172.21.15.139:0/1775265839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0051c0 0x7f722c00b9c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f723c001e50 tx=0x7f722c005830 comp rx=0 tx=0).stop 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.150+0000 7f724adee640 1 -- 172.21.15.139:0/1775265839 shutdown_connections 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.150+0000 7f724adee640 1 --2- 172.21.15.139:0/1775265839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0051c0 0x7f722c00b9c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.151+0000 7f724adee640 1 -- 172.21.15.139:0/1775265839 >> 172.21.15.139:0/1775265839 conn(0x7f72440b31f0 msgr2=0x7f72440b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.151+0000 7f724adee640 1 -- 172.21.15.139:0/1775265839 shutdown_connections 2024-04-25T09:55:38.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.151+0000 7f724adee640 1 -- 172.21.15.139:0/1775265839 wait complete. 2024-04-25T09:55:38.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.152+0000 7f724adee640 1 Processor -- start 2024-04-25T09:55:38.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.152+0000 7f724adee640 1 -- start start 2024-04-25T09:55:38.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724adee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c0051c0 0x7f724415dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724adee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7244157a10 0x7f72440b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724adee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72440cdcf0 0x7f72440ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724adee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440cd9f0 con 0x7f72440cdcf0 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724adee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724409a820 con 0x7f722c0051c0 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724adee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72440cdb70 con 0x7f7244157a10 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724938b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72440cdcf0 0x7f72440ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724938b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72440cdcf0 0x7f72440ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54874/0 (socket says 172.21.15.139:54874) 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724938b640 1 -- 172.21.15.139:0/1837788952 learned_addr learned my addr 172.21.15.139:0/1837788952 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f7248b8a640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c0051c0 0x7f724415dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f7243fff640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7244157a10 0x7f72440b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724938b640 1 -- 172.21.15.139:0/1837788952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7244157a10 msgr2=0x7f72440b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724938b640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7244157a10 0x7f72440b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724938b640 1 -- 172.21.15.139:0/1837788952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c0051c0 msgr2=0x7f724415dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724938b640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c0051c0 0x7f724415dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724938b640 1 -- 172.21.15.139:0/1837788952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7244157ef0 con 0x7f72440cdcf0 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f7248b8a640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c0051c0 0x7f724415dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f7243fff640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7244157a10 0x7f72440b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724938b640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72440cdcf0 0x7f72440ce180 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f723c001f70 tx=0x7f723c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f72417fa640 1 -- 172.21.15.139:0/1837788952 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c001a70 con 0x7f72440cdcf0 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f72417fa640 1 -- 172.21.15.139:0/1837788952 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c02ac70 con 0x7f72440cdcf0 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724adee640 1 -- 172.21.15.139:0/1837788952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72441581d0 con 0x7f72440cdcf0 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f72417fa640 1 -- 172.21.15.139:0/1837788952 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c006d70 con 0x7f72440cdcf0 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.153+0000 7f724adee640 1 -- 172.21.15.139:0/1837788952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7244158790 con 0x7f72440cdcf0 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.154+0000 7f724adee640 1 -- 172.21.15.139:0/1837788952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7244074520 con 0x7f72440cdcf0 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.155+0000 7f72417fa640 1 -- 172.21.15.139:0/1837788952 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f723c007030 con 0x7f72440cdcf0 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.155+0000 7f72417fa640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f723006b4d0 0x7f723006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.155+0000 7f7248b8a640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f723006b4d0 0x7f723006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.155+0000 7f72417fa640 1 -- 172.21.15.139:0/1837788952 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f723c0a9f70 con 0x7f72440cdcf0 2024-04-25T09:55:38.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.156+0000 7f7248b8a640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f723006b4d0 0x7f723006d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f72440b6430 tx=0x7f7234009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.159+0000 7f72417fa640 1 -- 172.21.15.139:0/1837788952 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f723c043060 con 0x7f72440cdcf0 2024-04-25T09:55:38.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.329+0000 7f724adee640 1 -- 172.21.15.139:0/1837788952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72440b3f70 con 0x7f72440cdcf0 2024-04-25T09:55:38.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.329+0000 7f72417fa640 1 -- 172.21.15.139:0/1837788952 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f723c078cf0 con 0x7f72440cdcf0 2024-04-25T09:55:38.332 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:55:38.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.331+0000 7f724adee640 1 -- 172.21.15.139:0/1837788952 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f723006b4d0 msgr2=0x7f723006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.331+0000 7f724adee640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f723006b4d0 0x7f723006d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f72440b6430 tx=0x7f7234009040 comp rx=0 tx=0).stop 2024-04-25T09:55:38.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.331+0000 7f724adee640 1 -- 172.21.15.139:0/1837788952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72440cdcf0 msgr2=0x7f72440ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.331+0000 7f724adee640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72440cdcf0 0x7f72440ce180 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f723c001f70 tx=0x7f723c0014a0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.331+0000 7f724adee640 1 -- 172.21.15.139:0/1837788952 shutdown_connections 2024-04-25T09:55:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.331+0000 7f724adee640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72440cdcf0 0x7f72440ce180 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.331+0000 7f724adee640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f723006b4d0 0x7f723006d990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.331+0000 7f724adee640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7244157a10 0x7f72440b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.331+0000 7f724adee640 1 --2- 172.21.15.139:0/1837788952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c0051c0 0x7f724415dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.331+0000 7f724adee640 1 -- 172.21.15.139:0/1837788952 >> 172.21.15.139:0/1837788952 conn(0x7f72440b31f0 msgr2=0x7f72440b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.331+0000 7f724adee640 1 -- 172.21.15.139:0/1837788952 shutdown_connections 2024-04-25T09:55:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.331+0000 7f724adee640 1 -- 172.21.15.139:0/1837788952 wait complete. 2024-04-25T09:55:38.345 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:38.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.439+0000 7f80134b3640 1 Processor -- start 2024-04-25T09:55:38.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.439+0000 7f80134b3640 1 -- start start 2024-04-25T09:55:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f80134b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f800c15b980 0x7f800c06fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f80134b3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f800c06eda0 0x7f800c06ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f80134b3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800c0cd030 con 0x7f800c06b0a0 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f80134b3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800c070580 con 0x7f800c06eda0 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f80134b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c070860 con 0x7f800c15b980 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f8010a4e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f800c06eda0 0x7f800c06ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f8010a4e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f800c06eda0 0x7f800c06ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54886/0 (socket says 172.21.15.139:54886) 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f8010a4e640 1 -- 172.21.15.139:0/3290386615 learned_addr learned my addr 172.21.15.139:0/3290386615 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f8010a4e640 1 -- 172.21.15.139:0/3290386615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f800c15b980 msgr2=0x7f800c06fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f801124f640 1 --2- 172.21.15.139:0/3290386615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f800c15b980 0x7f800c06fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f8010a4e640 1 --2- 172.21.15.139:0/3290386615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f800c15b980 0x7f800c06fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f8010a4e640 1 -- 172.21.15.139:0/3290386615 >> v1:172.21.15.150:6789/0 conn(0x7f800c06b0a0 legacy=0x7f800c15bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.440+0000 7f8010a4e640 1 -- 172.21.15.139:0/3290386615 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c070b40 con 0x7f800c06eda0 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.441+0000 7f8010a4e640 1 --2- 172.21.15.139:0/3290386615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c06eda0 0x7f800c06ab60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009870 tx=0x7f7ffc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a8ca7c446ba610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.441+0000 7f8003fff640 1 -- 172.21.15.139:0/3290386615 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc037070 con 0x7f800c06eda0 2024-04-25T09:55:38.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.441+0000 7f8003fff640 1 -- 172.21.15.139:0/3290386615 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc032440 con 0x7f800c06eda0 2024-04-25T09:55:38.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.441+0000 7f8003fff640 1 -- 172.21.15.139:0/3290386615 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc032810 con 0x7f800c06eda0 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.441+0000 7f80134b3640 1 -- 172.21.15.139:0/3290386615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c06eda0 msgr2=0x7f800c06ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.441+0000 7f80134b3640 1 --2- 172.21.15.139:0/3290386615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c06eda0 0x7f800c06ab60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009870 tx=0x7f7ffc02a180 comp rx=0 tx=0).stop 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.442+0000 7f80134b3640 1 -- 172.21.15.139:0/3290386615 shutdown_connections 2024-04-25T09:55:38.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.442+0000 7f80134b3640 1 --2- 172.21.15.139:0/3290386615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c06eda0 0x7f800c06ab60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.442+0000 7f80134b3640 1 --2- 172.21.15.139:0/3290386615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f800c15b980 0x7f800c06fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.442+0000 7f80134b3640 1 -- 172.21.15.139:0/3290386615 >> 172.21.15.139:0/3290386615 conn(0x7f800c06e280 msgr2=0x7f800c06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.442+0000 7f80134b3640 1 -- 172.21.15.139:0/3290386615 shutdown_connections 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.442+0000 7f80134b3640 1 -- 172.21.15.139:0/3290386615 wait complete. 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.443+0000 7f80134b3640 1 Processor -- start 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.443+0000 7f80134b3640 1 -- start start 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.443+0000 7f80134b3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f800c06b0a0 0x7f800c145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.443+0000 7f80134b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f800c06eda0 0x7f800c15e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.443+0000 7f80134b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c15b980 0x7f800c072790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.443+0000 7f80134b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c15c470 con 0x7f800c15b980 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.443+0000 7f80134b3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c070270 con 0x7f800c06b0a0 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.443+0000 7f80134b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c15c5f0 con 0x7f800c06eda0 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8011a50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c15b980 0x7f800c072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8011a50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c15b980 0x7f800c072790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54900/0 (socket says 172.21.15.139:54900) 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8011a50640 1 -- 172.21.15.139:0/753505767 learned_addr learned my addr 172.21.15.139:0/753505767 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f801124f640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f800c06b0a0 0x7f800c145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8010a4e640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f800c06eda0 0x7f800c15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8011a50640 1 -- 172.21.15.139:0/753505767 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f800c06eda0 msgr2=0x7f800c15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8011a50640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f800c06eda0 0x7f800c15e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8011a50640 1 -- 172.21.15.139:0/753505767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f800c06b0a0 msgr2=0x7f800c145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8011a50640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f800c06b0a0 0x7f800c145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8011a50640 1 -- 172.21.15.139:0/753505767 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c145e00 con 0x7f800c15b980 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8011a50640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c15b980 0x7f800c072790 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f800400c7b0 tx=0x7f800400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8001ffb640 1 -- 172.21.15.139:0/753505767 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004004b60 con 0x7f800c15b980 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8001ffb640 1 -- 172.21.15.139:0/753505767 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8004004d00 con 0x7f800c15b980 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f8001ffb640 1 -- 172.21.15.139:0/753505767 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800400aa90 con 0x7f800c15b980 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f80134b3640 1 -- 172.21.15.139:0/753505767 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c1460e0 con 0x7f800c15b980 2024-04-25T09:55:38.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.444+0000 7f80134b3640 1 -- 172.21.15.139:0/753505767 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c1466a0 con 0x7f800c15b980 2024-04-25T09:55:38.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.445+0000 7f80134b3640 1 -- 172.21.15.139:0/753505767 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c074520 con 0x7f800c15b980 2024-04-25T09:55:38.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.446+0000 7f8001ffb640 1 -- 172.21.15.139:0/753505767 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f800400ac30 con 0x7f800c15b980 2024-04-25T09:55:38.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.446+0000 7f8001ffb640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ff006b4d0 0x7f7ff006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.446+0000 7f801124f640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ff006b4d0 0x7f7ff006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.446+0000 7f8001ffb640 1 -- 172.21.15.139:0/753505767 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f800408a5d0 con 0x7f800c15b980 2024-04-25T09:55:38.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.447+0000 7f801124f640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ff006b4d0 0x7f7ff006d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8008006fd0 tx=0x7f8008008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.450+0000 7f8001ffb640 1 -- 172.21.15.139:0/753505767 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8004020070 con 0x7f800c15b980 2024-04-25T09:55:38.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.655+0000 7f80134b3640 1 -- 172.21.15.139:0/753505767 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f800c0cf840 con 0x7f800c15b980 2024-04-25T09:55:38.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.656+0000 7f8001ffb640 1 -- 172.21.15.139:0/753505767 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f8004059350 con 0x7f800c15b980 2024-04-25T09:55:38.659 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:38.659 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1742557158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1742557158},{"type":"v1","addr":"172.21.15.150:6834","nonce":1742557158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3496803834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3496803834},{"type":"v1","addr":"172.21.15.150:6835","nonce":3496803834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2014537692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2014537692},{"type":"v1","addr":"172.21.15.139:6835","nonce":2014537692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:56.466203+0000","modified":"2024-04-25T09:54:58.466766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5850":{"gid":5850,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/338275708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":338275708},{"type":"v1","addr":"172.21.15.139:6837","nonce":338275708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:55:38.659 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 22 2024-04-25T09:55:38.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.658+0000 7f80134b3640 1 -- 172.21.15.139:0/753505767 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ff006b4d0 msgr2=0x7f7ff006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.658+0000 7f80134b3640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ff006b4d0 0x7f7ff006d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8008006fd0 tx=0x7f8008008040 comp rx=0 tx=0).stop 2024-04-25T09:55:38.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.658+0000 7f80134b3640 1 -- 172.21.15.139:0/753505767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c15b980 msgr2=0x7f800c072790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.658+0000 7f80134b3640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c15b980 0x7f800c072790 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f800400c7b0 tx=0x7f800400cc70 comp rx=0 tx=0).stop 2024-04-25T09:55:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.658+0000 7f80134b3640 1 -- 172.21.15.139:0/753505767 shutdown_connections 2024-04-25T09:55:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.658+0000 7f80134b3640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c15b980 0x7f800c072790 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.658+0000 7f80134b3640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ff006b4d0 0x7f7ff006d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.658+0000 7f80134b3640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f800c06eda0 0x7f800c15e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.658+0000 7f80134b3640 1 --2- 172.21.15.139:0/753505767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f800c06b0a0 0x7f800c145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.658+0000 7f80134b3640 1 -- 172.21.15.139:0/753505767 >> 172.21.15.139:0/753505767 conn(0x7f800c06e280 msgr2=0x7f800c15a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.658+0000 7f80134b3640 1 -- 172.21.15.139:0/753505767 shutdown_connections 2024-04-25T09:55:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.658+0000 7f80134b3640 1 -- 172.21.15.139:0/753505767 wait complete. 2024-04-25T09:55:38.672 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:38.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.765+0000 7fc167141640 1 Processor -- start 2024-04-25T09:55:38.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.765+0000 7fc167141640 1 -- start start 2024-04-25T09:55:38.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.766+0000 7fc167141640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1600d0730 0x7fc1600cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.766+0000 7fc167141640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc1600d0b90 0x7fc1600cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.766+0000 7fc167141640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1600cc8c0 con 0x7fc1600b0750 2024-04-25T09:55:38.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.766+0000 7fc167141640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1600ccb70 con 0x7fc1600d0b90 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.766+0000 7fc167141640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1600cf970 con 0x7fc1600d0730 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.766+0000 7fc1656de640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc1600d0b90 0x7fc1600cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.766+0000 7fc164edd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc1600b0750 0x7fc1600b0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37682/0 (socket says 172.21.15.139:37682) 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.766+0000 7fc164edd640 1 -- 172.21.15.139:0/1308559564 learned_addr learned my addr 172.21.15.139:0/1308559564 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.767+0000 7fc15ffff640 1 --2- 172.21.15.139:0/1308559564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1600d0730 0x7fc1600cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.767+0000 7fc1656de640 1 -- 172.21.15.139:0/1308559564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1600d0730 msgr2=0x7fc1600cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.767+0000 7fc1656de640 1 --2- 172.21.15.139:0/1308559564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1600d0730 0x7fc1600cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.767+0000 7fc1656de640 1 -- 172.21.15.139:0/1308559564 >> v1:172.21.15.139:6789/0 conn(0x7fc1600b0750 legacy=0x7fc1600b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.767+0000 7fc15f7fe640 1 -- 172.21.15.139:0/1308559564 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827510475 0 0) 0x7fc1600cc8c0 con 0x7fc1600b0750 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.767+0000 7fc1656de640 1 -- 172.21.15.139:0/1308559564 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1600cfc50 con 0x7fc1600d0b90 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.767+0000 7fc1656de640 1 --2- 172.21.15.139:0/1308559564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1600d0b90 0x7fc1600cc380 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc158009d90 tx=0x7fc15802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221690f3e6b32726 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.767+0000 7fc15f7fe640 1 -- 172.21.15.139:0/1308559564 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158007740 con 0x7fc1600d0b90 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.767+0000 7fc15f7fe640 1 -- 172.21.15.139:0/1308559564 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1580078e0 con 0x7fc1600d0b90 2024-04-25T09:55:38.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.767+0000 7fc15f7fe640 1 -- 172.21.15.139:0/1308559564 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158007cb0 con 0x7fc1600d0b90 2024-04-25T09:55:38.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.768+0000 7fc167141640 1 -- 172.21.15.139:0/1308559564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1600d0b90 msgr2=0x7fc1600cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.768+0000 7fc167141640 1 --2- 172.21.15.139:0/1308559564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1600d0b90 0x7fc1600cc380 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc158009d90 tx=0x7fc15802a660 comp rx=0 tx=0).stop 2024-04-25T09:55:38.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.768+0000 7fc167141640 1 -- 172.21.15.139:0/1308559564 shutdown_connections 2024-04-25T09:55:38.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.768+0000 7fc167141640 1 --2- 172.21.15.139:0/1308559564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1600d0b90 0x7fc1600cc380 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.768+0000 7fc167141640 1 --2- 172.21.15.139:0/1308559564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1600d0730 0x7fc1600cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.768+0000 7fc167141640 1 -- 172.21.15.139:0/1308559564 >> 172.21.15.139:0/1308559564 conn(0x7fc1600b3380 msgr2=0x7fc1600b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.768+0000 7fc167141640 1 -- 172.21.15.139:0/1308559564 shutdown_connections 2024-04-25T09:55:38.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.768+0000 7fc167141640 1 -- 172.21.15.139:0/1308559564 wait complete. 2024-04-25T09:55:38.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.769+0000 7fc167141640 1 Processor -- start 2024-04-25T09:55:38.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.769+0000 7fc167141640 1 -- start start 2024-04-25T09:55:38.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc167141640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1600b0750 0x7fc160158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc167141640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1600d0730 0x7fc1600c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc167141640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1601f58f0 0x7fc1600c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc167141640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16015bcc0 con 0x7fc1600b0750 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc167141640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1600d0210 con 0x7fc1600d0730 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc167141640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16015be40 con 0x7fc1601f58f0 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc164edd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1600b0750 0x7fc160158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc164edd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1600b0750 0x7fc160158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54922/0 (socket says 172.21.15.139:54922) 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc164edd640 1 -- 172.21.15.139:0/2602764959 learned_addr learned my addr 172.21.15.139:0/2602764959 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc164edd640 1 -- 172.21.15.139:0/2602764959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1601f58f0 msgr2=0x7fc1600c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc164edd640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1601f58f0 0x7fc1600c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc15ffff640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1600d0730 0x7fc1600c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc164edd640 1 -- 172.21.15.139:0/2602764959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1600d0730 msgr2=0x7fc1600c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc164edd640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1600d0730 0x7fc1600c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.770+0000 7fc164edd640 1 -- 172.21.15.139:0/2602764959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1600c9230 con 0x7fc1600b0750 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.771+0000 7fc164edd640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1600b0750 0x7fc160158b40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc15400c430 tx=0x7fc15400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.771+0000 7fc15d7fa640 1 -- 172.21.15.139:0/2602764959 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1540074e0 con 0x7fc1600b0750 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.771+0000 7fc15d7fa640 1 -- 172.21.15.139:0/2602764959 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1540047e0 con 0x7fc1600b0750 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.771+0000 7fc167141640 1 -- 172.21.15.139:0/2602764959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1600c9510 con 0x7fc1600b0750 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.771+0000 7fc15d7fa640 1 -- 172.21.15.139:0/2602764959 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154004bd0 con 0x7fc1600b0750 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.771+0000 7fc167141640 1 -- 172.21.15.139:0/2602764959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1600c97f0 con 0x7fc1600b0750 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.772+0000 7fc167141640 1 -- 172.21.15.139:0/2602764959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc160074520 con 0x7fc1600b0750 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.772+0000 7fc15d7fa640 1 -- 172.21.15.139:0/2602764959 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc154011070 con 0x7fc1600b0750 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.772+0000 7fc15d7fa640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc14406b4d0 0x7fc14406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.773+0000 7fc15d7fa640 1 -- 172.21.15.139:0/2602764959 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc154089f20 con 0x7fc1600b0750 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.773+0000 7fc15ffff640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc14406b4d0 0x7fc14406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.773+0000 7fc15ffff640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc14406b4d0 0x7fc14406d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc14c006fd0 tx=0x7fc14c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.777+0000 7fc15d7fa640 1 -- 172.21.15.139:0/2602764959 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc154058ca0 con 0x7fc1600b0750 2024-04-25T09:55:38.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.948+0000 7fc167141640 1 -- 172.21.15.139:0/2602764959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc16015b330 con 0x7fc1600b0750 2024-04-25T09:55:38.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.949+0000 7fc15d7fa640 1 -- 172.21.15.139:0/2602764959 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16990 (secure 0 0 0) 0x7fc1540585a0 con 0x7fc1600b0750 2024-04-25T09:55:38.952 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:38.952 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":50,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:58.130580+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:54.661529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:55.431933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:38.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.951+0000 7fc167141640 1 -- 172.21.15.139:0/2602764959 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc14406b4d0 msgr2=0x7fc14406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.951+0000 7fc167141640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc14406b4d0 0x7fc14406d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc14c006fd0 tx=0x7fc14c008040 comp rx=0 tx=0).stop 2024-04-25T09:55:38.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.951+0000 7fc167141640 1 -- 172.21.15.139:0/2602764959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1600b0750 msgr2=0x7fc160158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.951+0000 7fc167141640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1600b0750 0x7fc160158b40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc15400c430 tx=0x7fc15400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.951+0000 7fc167141640 1 -- 172.21.15.139:0/2602764959 shutdown_connections 2024-04-25T09:55:38.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.951+0000 7fc167141640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc14406b4d0 0x7fc14406d990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.951+0000 7fc167141640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1601f58f0 0x7fc1600c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.951+0000 7fc167141640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1600d0730 0x7fc1600c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.951+0000 7fc167141640 1 --2- 172.21.15.139:0/2602764959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1600b0750 0x7fc160158b40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.951+0000 7fc167141640 1 -- 172.21.15.139:0/2602764959 >> 172.21.15.139:0/2602764959 conn(0x7fc1600b3380 msgr2=0x7fc1600cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.951+0000 7fc167141640 1 -- 172.21.15.139:0/2602764959 shutdown_connections 2024-04-25T09:55:38.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:38.951+0000 7fc167141640 1 -- 172.21.15.139:0/2602764959 wait complete. 2024-04-25T09:55:38.965 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:55:38.965 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:39.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.059+0000 7fc351f4c640 1 Processor -- start 2024-04-25T09:55:39.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.059+0000 7fc351f4c640 1 -- start start 2024-04-25T09:55:39.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc351f4c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc34c15ae40 0x7fc34c0b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc351f4c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc34c0cefe0 0x7fc34c06d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc351f4c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34c0ce220 con 0x7fc34c0b0210 2024-04-25T09:55:39.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc351f4c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc34c0b1070 con 0x7fc34c0cefe0 2024-04-25T09:55:39.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc351f4c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc34c0cdff0 con 0x7fc34c15ae40 2024-04-25T09:55:39.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc34bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc34c0cefe0 0x7fc34c06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc34bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc34c0cefe0 0x7fc34c06d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54936/0 (socket says 172.21.15.139:54936) 2024-04-25T09:55:39.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc34bfff640 1 -- 172.21.15.139:0/1355427848 learned_addr learned my addr 172.21.15.139:0/1355427848 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:39.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc34affd640 1 --2- 172.21.15.139:0/1355427848 >> v2:172.21.15.150:3300/0 conn(0x7fc34c15ae40 0x7fc34c0b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc34bfff640 1 -- 172.21.15.139:0/1355427848 >> v2:172.21.15.150:3300/0 conn(0x7fc34c15ae40 msgr2=0x7fc34c0b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc34bfff640 1 --2- 172.21.15.139:0/1355427848 >> v2:172.21.15.150:3300/0 conn(0x7fc34c15ae40 0x7fc34c0b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc34bfff640 1 -- 172.21.15.139:0/1355427848 >> v1:172.21.15.150:6789/0 conn(0x7fc34c0b0210 legacy=0x7fc34c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc34bfff640 1 -- 172.21.15.139:0/1355427848 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c0ce500 con 0x7fc34c0cefe0 2024-04-25T09:55:39.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.060+0000 7fc34affd640 1 --2- 172.21.15.139:0/1355427848 >> v2:172.21.15.150:3300/0 conn(0x7fc34c15ae40 0x7fc34c0b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:39.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.061+0000 7fc34bfff640 1 --2- 172.21.15.139:0/1355427848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc34c0cefe0 0x7fc34c06d200 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc34c06f330 tx=0x7fc34402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d496f986b8a6e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.061+0000 7fc34a7fc640 1 -- 172.21.15.139:0/1355427848 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344038070 con 0x7fc34c0cefe0 2024-04-25T09:55:39.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.061+0000 7fc34a7fc640 1 -- 172.21.15.139:0/1355427848 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc344033630 con 0x7fc34c0cefe0 2024-04-25T09:55:39.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.061+0000 7fc34a7fc640 1 -- 172.21.15.139:0/1355427848 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34402f050 con 0x7fc34c0cefe0 2024-04-25T09:55:39.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.062+0000 7fc351f4c640 1 -- 172.21.15.139:0/1355427848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc34c0cefe0 msgr2=0x7fc34c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.062+0000 7fc351f4c640 1 --2- 172.21.15.139:0/1355427848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc34c0cefe0 0x7fc34c06d200 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc34c06f330 tx=0x7fc34402b840 comp rx=0 tx=0).stop 2024-04-25T09:55:39.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.062+0000 7fc351f4c640 1 -- 172.21.15.139:0/1355427848 shutdown_connections 2024-04-25T09:55:39.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.062+0000 7fc351f4c640 1 --2- 172.21.15.139:0/1355427848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc34c0cefe0 0x7fc34c06d200 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.062+0000 7fc351f4c640 1 --2- 172.21.15.139:0/1355427848 >> v2:172.21.15.150:3300/0 conn(0x7fc34c15ae40 0x7fc34c0b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.062+0000 7fc351f4c640 1 -- 172.21.15.139:0/1355427848 >> 172.21.15.139:0/1355427848 conn(0x7fc34c066f20 msgr2=0x7fc34c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.062+0000 7fc351f4c640 1 -- 172.21.15.139:0/1355427848 shutdown_connections 2024-04-25T09:55:39.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.062+0000 7fc351f4c640 1 -- 172.21.15.139:0/1355427848 wait complete. 2024-04-25T09:55:39.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.063+0000 7fc351f4c640 1 Processor -- start 2024-04-25T09:55:39.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.063+0000 7fc351f4c640 1 -- start start 2024-04-25T09:55:39.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc351f4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc34c0b0210 0x7fc34c147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc351f4c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc34c0cefe0 0x7fc34c0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc351f4c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc34c15ae40 0x7fc34c158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc351f4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c15b3e0 con 0x7fc34c0b0210 2024-04-25T09:55:39.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc351f4c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0b1ea0 con 0x7fc34c0cefe0 2024-04-25T09:55:39.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc351f4c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0ceb50 con 0x7fc34c15ae40 2024-04-25T09:55:39.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc34b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc34c0b0210 0x7fc34c147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc34b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc34c0b0210 0x7fc34c147100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54938/0 (socket says 172.21.15.139:54938) 2024-04-25T09:55:39.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc34b7fe640 1 -- 172.21.15.139:0/2767998801 learned_addr learned my addr 172.21.15.139:0/2767998801 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:39.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc34affd640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc34c0cefe0 0x7fc34c0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc34bfff640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc34c15ae40 0x7fc34c158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc34b7fe640 1 -- 172.21.15.139:0/2767998801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc34c15ae40 msgr2=0x7fc34c158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc34b7fe640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc34c15ae40 0x7fc34c158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc34b7fe640 1 -- 172.21.15.139:0/2767998801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc34c0cefe0 msgr2=0x7fc34c0ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc34b7fe640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc34c0cefe0 0x7fc34c0ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc34b7fe640 1 -- 172.21.15.139:0/2767998801 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c147640 con 0x7fc34c0b0210 2024-04-25T09:55:39.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.064+0000 7fc34b7fe640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc34c0b0210 0x7fc34c147100 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00c400 tx=0x7fc33c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.065+0000 7fc333fff640 1 -- 172.21.15.139:0/2767998801 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c0052d0 con 0x7fc34c0b0210 2024-04-25T09:55:39.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.065+0000 7fc333fff640 1 -- 172.21.15.139:0/2767998801 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33c005490 con 0x7fc34c0b0210 2024-04-25T09:55:39.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.065+0000 7fc333fff640 1 -- 172.21.15.139:0/2767998801 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c007260 con 0x7fc34c0b0210 2024-04-25T09:55:39.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.065+0000 7fc351f4c640 1 -- 172.21.15.139:0/2767998801 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34c0a9e90 con 0x7fc34c0b0210 2024-04-25T09:55:39.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.065+0000 7fc351f4c640 1 -- 172.21.15.139:0/2767998801 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34c0aa450 con 0x7fc34c0b0210 2024-04-25T09:55:39.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.066+0000 7fc351f4c640 1 -- 172.21.15.139:0/2767998801 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34c074520 con 0x7fc34c0b0210 2024-04-25T09:55:39.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.066+0000 7fc333fff640 1 -- 172.21.15.139:0/2767998801 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc33c007400 con 0x7fc34c0b0210 2024-04-25T09:55:39.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.066+0000 7fc333fff640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc33806b4d0 0x7fc33806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.067+0000 7fc333fff640 1 -- 172.21.15.139:0/2767998801 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc33c089910 con 0x7fc34c0b0210 2024-04-25T09:55:39.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.067+0000 7fc34affd640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc33806b4d0 0x7fc33806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.067+0000 7fc34affd640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc33806b4d0 0x7fc33806d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc34c1f54b0 tx=0x7fc340008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.071+0000 7fc333fff640 1 -- 172.21.15.139:0/2767998801 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc34c147640 con 0x7fc34c0b0210 2024-04-25T09:55:39.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.275+0000 7fc351f4c640 1 -- 172.21.15.139:0/2767998801 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc34c0b2040 con 0x7fc34c0b0210 2024-04-25T09:55:39.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.276+0000 7fc333fff640 1 -- 172.21.15.139:0/2767998801 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fc33c058580 con 0x7fc34c0b0210 2024-04-25T09:55:39.279 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:39.279 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1742557158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1742557158},{"type":"v1","addr":"172.21.15.150:6834","nonce":1742557158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3496803834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3496803834},{"type":"v1","addr":"172.21.15.150:6835","nonce":3496803834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2014537692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2014537692},{"type":"v1","addr":"172.21.15.139:6835","nonce":2014537692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:56.466203+0000","modified":"2024-04-25T09:54:58.466766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5850":{"gid":5850,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/338275708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":338275708},{"type":"v1","addr":"172.21.15.139:6837","nonce":338275708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:55:39.279 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 22 2024-04-25T09:55:39.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.278+0000 7fc351f4c640 1 -- 172.21.15.139:0/2767998801 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc33806b4d0 msgr2=0x7fc33806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.278+0000 7fc351f4c640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc33806b4d0 0x7fc33806d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc34c1f54b0 tx=0x7fc340008040 comp rx=0 tx=0).stop 2024-04-25T09:55:39.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.278+0000 7fc351f4c640 1 -- 172.21.15.139:0/2767998801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc34c0b0210 msgr2=0x7fc34c147100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.278+0000 7fc351f4c640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc34c0b0210 0x7fc34c147100 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00c400 tx=0x7fc33c00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.278+0000 7fc351f4c640 1 -- 172.21.15.139:0/2767998801 shutdown_connections 2024-04-25T09:55:39.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.278+0000 7fc351f4c640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc33806b4d0 0x7fc33806d990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.278+0000 7fc351f4c640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc34c15ae40 0x7fc34c158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.278+0000 7fc351f4c640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc34c0cefe0 0x7fc34c0ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.278+0000 7fc351f4c640 1 --2- 172.21.15.139:0/2767998801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc34c0b0210 0x7fc34c147100 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.278+0000 7fc351f4c640 1 -- 172.21.15.139:0/2767998801 >> 172.21.15.139:0/2767998801 conn(0x7fc34c066f20 msgr2=0x7fc34c154150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.278+0000 7fc351f4c640 1 -- 172.21.15.139:0/2767998801 shutdown_connections 2024-04-25T09:55:39.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.278+0000 7fc351f4c640 1 -- 172.21.15.139:0/2767998801 wait complete. 2024-04-25T09:55:39.291 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:39.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.384+0000 7fc400ccb640 1 Processor -- start 2024-04-25T09:55:39.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.384+0000 7fc400ccb640 1 -- start start 2024-04-25T09:55:39.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.385+0000 7fc400ccb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc3fc15dc60 0x7fc3fc0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.385+0000 7fc400ccb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc3fc161850 0x7fc3fc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.385+0000 7fc400ccb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3fc0cf250 con 0x7fc3fc0d0970 2024-04-25T09:55:39.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.385+0000 7fc400ccb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3fc0ce780 con 0x7fc3fc15dc60 2024-04-25T09:55:39.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.385+0000 7fc400ccb640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3fc0d3730 con 0x7fc3fc161850 2024-04-25T09:55:39.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.385+0000 7fc3fa59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc3fc15dc60 0x7fc3fc0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.385+0000 7fc3fa59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc3fc15dc60 0x7fc3fc0b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54946/0 (socket says 172.21.15.139:54946) 2024-04-25T09:55:39.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.385+0000 7fc3fa59c640 1 -- 172.21.15.139:0/794865522 learned_addr learned my addr 172.21.15.139:0/794865522 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:39.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.386+0000 7fc3f9d9b640 1 --2- 172.21.15.139:0/794865522 >> v2:172.21.15.150:3300/0 conn(0x7fc3fc161850 0x7fc3fc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.386+0000 7fc3fa59c640 1 -- 172.21.15.139:0/794865522 >> v2:172.21.15.150:3300/0 conn(0x7fc3fc161850 msgr2=0x7fc3fc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.386+0000 7fc3fa59c640 1 --2- 172.21.15.139:0/794865522 >> v2:172.21.15.150:3300/0 conn(0x7fc3fc161850 0x7fc3fc0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.386+0000 7fc3fa59c640 1 -- 172.21.15.139:0/794865522 >> v1:172.21.15.150:6789/0 conn(0x7fc3fc0d0970 legacy=0x7fc3fc0d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.386+0000 7fc3fa59c640 1 -- 172.21.15.139:0/794865522 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc0ce9b0 con 0x7fc3fc15dc60 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.386+0000 7fc3f9d9b640 1 --2- 172.21.15.139:0/794865522 >> v2:172.21.15.150:3300/0 conn(0x7fc3fc161850 0x7fc3fc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.386+0000 7fc3fa59c640 1 --2- 172.21.15.139:0/794865522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3fc15dc60 0x7fc3fc0b6c10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc068750 tx=0x7fc3ec02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce8331cad4909e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.386+0000 7fc3f959a640 1 -- 172.21.15.139:0/794865522 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec036070 con 0x7fc3fc15dc60 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.387+0000 7fc3f959a640 1 -- 172.21.15.139:0/794865522 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec031600 con 0x7fc3fc15dc60 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.387+0000 7fc3f959a640 1 -- 172.21.15.139:0/794865522 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0319d0 con 0x7fc3fc15dc60 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.387+0000 7fc400ccb640 1 -- 172.21.15.139:0/794865522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3fc15dc60 msgr2=0x7fc3fc0b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.387+0000 7fc400ccb640 1 --2- 172.21.15.139:0/794865522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3fc15dc60 0x7fc3fc0b6c10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc068750 tx=0x7fc3ec02a730 comp rx=0 tx=0).stop 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.387+0000 7fc400ccb640 1 -- 172.21.15.139:0/794865522 shutdown_connections 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.387+0000 7fc400ccb640 1 --2- 172.21.15.139:0/794865522 >> v2:172.21.15.150:3300/0 conn(0x7fc3fc161850 0x7fc3fc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.387+0000 7fc400ccb640 1 --2- 172.21.15.139:0/794865522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3fc15dc60 0x7fc3fc0b6c10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.387+0000 7fc400ccb640 1 -- 172.21.15.139:0/794865522 >> 172.21.15.139:0/794865522 conn(0x7fc3fc0b5910 msgr2=0x7fc3fc0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.387+0000 7fc400ccb640 1 -- 172.21.15.139:0/794865522 shutdown_connections 2024-04-25T09:55:39.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.387+0000 7fc400ccb640 1 -- 172.21.15.139:0/794865522 wait complete. 2024-04-25T09:55:39.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.388+0000 7fc400ccb640 1 Processor -- start 2024-04-25T09:55:39.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.389+0000 7fc400ccb640 1 -- start start 2024-04-25T09:55:39.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.389+0000 7fc400ccb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3fc0d0970 0x7fc3fc15a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.389+0000 7fc400ccb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3fc15dc60 0x7fc3fc0c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.389+0000 7fc400ccb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3fc161850 0x7fc3fc0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.389+0000 7fc400ccb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0d1fc0 con 0x7fc3fc0d0970 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.389+0000 7fc400ccb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0cef70 con 0x7fc3fc161850 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.389+0000 7fc400ccb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc15e040 con 0x7fc3fc15dc60 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.389+0000 7fc3fa59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3fc0d0970 0x7fc3fc15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.389+0000 7fc3fa59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3fc0d0970 0x7fc3fc15a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54950/0 (socket says 172.21.15.139:54950) 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.389+0000 7fc3fa59c640 1 -- 172.21.15.139:0/2355634840 learned_addr learned my addr 172.21.15.139:0/2355634840 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.390+0000 7fc3fa59c640 1 -- 172.21.15.139:0/2355634840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3fc15dc60 msgr2=0x7fc3fc0c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.390+0000 7fc3fa59c640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3fc15dc60 0x7fc3fc0c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.390+0000 7fc3fa59c640 1 -- 172.21.15.139:0/2355634840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3fc161850 msgr2=0x7fc3fc0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.390+0000 7fc3fad9d640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3fc161850 0x7fc3fc0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.390+0000 7fc3fa59c640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3fc161850 0x7fc3fc0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.390+0000 7fc3fa59c640 1 -- 172.21.15.139:0/2355634840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc15aaf0 con 0x7fc3fc0d0970 2024-04-25T09:55:39.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.390+0000 7fc3fa59c640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3fc0d0970 0x7fc3fc15a030 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec034040 tx=0x7fc3ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.390+0000 7fc3eaffd640 1 -- 172.21.15.139:0/2355634840 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec036050 con 0x7fc3fc0d0970 2024-04-25T09:55:39.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.390+0000 7fc3eaffd640 1 -- 172.21.15.139:0/2355634840 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec042070 con 0x7fc3fc0d0970 2024-04-25T09:55:39.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.390+0000 7fc3eaffd640 1 -- 172.21.15.139:0/2355634840 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec009510 con 0x7fc3fc0d0970 2024-04-25T09:55:39.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.390+0000 7fc400ccb640 1 -- 172.21.15.139:0/2355634840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3fc15ae30 con 0x7fc3fc0d0970 2024-04-25T09:55:39.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.390+0000 7fc400ccb640 1 -- 172.21.15.139:0/2355634840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3fc15b670 con 0x7fc3fc0d0970 2024-04-25T09:55:39.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.391+0000 7fc400ccb640 1 -- 172.21.15.139:0/2355634840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3fc0b9120 con 0x7fc3fc0d0970 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.391+0000 7fc3eaffd640 1 -- 172.21.15.139:0/2355634840 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3ec007c50 con 0x7fc3fc0d0970 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.392+0000 7fc3eaffd640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc3dc06b500 0x7fc3dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.392+0000 7fc3eaffd640 1 -- 172.21.15.139:0/2355634840 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc3ec0ab1f0 con 0x7fc3fc0d0970 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.392+0000 7fc3f9d9b640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc3dc06b500 0x7fc3dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.392+0000 7fc3f9d9b640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc3dc06b500 0x7fc3dc06d9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0006fd0 tx=0x7fc3f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.396+0000 7fc3eaffd640 1 -- 172.21.15.139:0/2355634840 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3fc0b9120 con 0x7fc3fc0d0970 2024-04-25T09:55:39.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.600+0000 7fc400ccb640 1 -- 172.21.15.139:0/2355634840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3fc074520 con 0x7fc3fc0d0970 2024-04-25T09:55:39.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.601+0000 7fc3eaffd640 1 -- 172.21.15.139:0/2355634840 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fc3fc15ae30 con 0x7fc3fc0d0970 2024-04-25T09:55:39.604 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:39.604 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1742557158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1742557158},{"type":"v1","addr":"172.21.15.150:6834","nonce":1742557158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3496803834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3496803834},{"type":"v1","addr":"172.21.15.150:6835","nonce":3496803834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2014537692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2014537692},{"type":"v1","addr":"172.21.15.139:6835","nonce":2014537692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:56.466203+0000","modified":"2024-04-25T09:54:58.466766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5850":{"gid":5850,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/338275708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":338275708},{"type":"v1","addr":"172.21.15.139:6837","nonce":338275708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:55:39.604 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 22 2024-04-25T09:55:39.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.603+0000 7fc400ccb640 1 -- 172.21.15.139:0/2355634840 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc3dc06b500 msgr2=0x7fc3dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.603+0000 7fc400ccb640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc3dc06b500 0x7fc3dc06d9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0006fd0 tx=0x7fc3f0008040 comp rx=0 tx=0).stop 2024-04-25T09:55:39.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.603+0000 7fc400ccb640 1 -- 172.21.15.139:0/2355634840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3fc0d0970 msgr2=0x7fc3fc15a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.603+0000 7fc400ccb640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3fc0d0970 0x7fc3fc15a030 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec034040 tx=0x7fc3ec004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:39.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.603+0000 7fc400ccb640 1 -- 172.21.15.139:0/2355634840 shutdown_connections 2024-04-25T09:55:39.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.603+0000 7fc400ccb640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc3dc06b500 0x7fc3dc06d9c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.603+0000 7fc400ccb640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3fc161850 0x7fc3fc0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.603+0000 7fc400ccb640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3fc15dc60 0x7fc3fc0c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.603+0000 7fc400ccb640 1 --2- 172.21.15.139:0/2355634840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3fc0d0970 0x7fc3fc15a030 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.603+0000 7fc400ccb640 1 -- 172.21.15.139:0/2355634840 >> 172.21.15.139:0/2355634840 conn(0x7fc3fc0b5910 msgr2=0x7fc3fc15e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.603+0000 7fc400ccb640 1 -- 172.21.15.139:0/2355634840 shutdown_connections 2024-04-25T09:55:39.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.603+0000 7fc400ccb640 1 -- 172.21.15.139:0/2355634840 wait complete. 2024-04-25T09:55:39.617 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.711+0000 7f5f10749640 1 Processor -- start 2024-04-25T09:55:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.711+0000 7f5f10749640 1 -- start start 2024-04-25T09:55:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f10749640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5f080cf900 0x7f5f080d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f10749640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5f08099a50 0x7f5f08099e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f10749640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f080cfe70 con 0x7f5f0815efb0 2024-04-25T09:55:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f10749640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f080d0a80 con 0x7f5f08099a50 2024-04-25T09:55:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f10749640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0809a3c0 con 0x7f5f080cf900 2024-04-25T09:55:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f0ece6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5f08099a50 0x7f5f08099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f0ece6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5f08099a50 0x7f5f08099e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54966/0 (socket says 172.21.15.139:54966) 2024-04-25T09:55:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f0e4e5640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5f0815efb0 0x7f5f0806c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37694/0 (socket says 172.21.15.139:37694) 2024-04-25T09:55:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f0ece6640 1 -- 172.21.15.139:0/2455975337 learned_addr learned my addr 172.21.15.139:0/2455975337 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f0dce4640 1 --2- 172.21.15.139:0/2455975337 >> v2:172.21.15.150:3300/0 conn(0x7f5f080cf900 0x7f5f080d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f0ece6640 1 -- 172.21.15.139:0/2455975337 >> v2:172.21.15.150:3300/0 conn(0x7f5f080cf900 msgr2=0x7f5f080d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f0ece6640 1 --2- 172.21.15.139:0/2455975337 >> v2:172.21.15.150:3300/0 conn(0x7f5f080cf900 0x7f5f080d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f0ece6640 1 -- 172.21.15.139:0/2455975337 >> v1:172.21.15.139:6789/0 conn(0x7f5f0815efb0 legacy=0x7f5f0806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f0ece6640 1 -- 172.21.15.139:0/2455975337 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f080d0150 con 0x7f5f08099a50 2024-04-25T09:55:39.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.712+0000 7f5f0d4e3640 1 -- 172.21.15.139:0/2455975337 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299709177 0 0) 0x7f5f080cfe70 con 0x7f5f0815efb0 2024-04-25T09:55:39.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.713+0000 7f5f0ece6640 1 --2- 172.21.15.139:0/2455975337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f08099a50 0x7f5f08099e80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5f0809aad0 tx=0x7f5f0002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e780464c0f10f591 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.713+0000 7f5f0d4e3640 1 -- 172.21.15.139:0/2455975337 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00036070 con 0x7f5f08099a50 2024-04-25T09:55:39.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.713+0000 7f5f0d4e3640 1 -- 172.21.15.139:0/2455975337 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f00031600 con 0x7f5f08099a50 2024-04-25T09:55:39.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.713+0000 7f5f0d4e3640 1 -- 172.21.15.139:0/2455975337 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f000319d0 con 0x7f5f08099a50 2024-04-25T09:55:39.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.713+0000 7f5f10749640 1 -- 172.21.15.139:0/2455975337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f08099a50 msgr2=0x7f5f08099e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.713+0000 7f5f10749640 1 --2- 172.21.15.139:0/2455975337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f08099a50 0x7f5f08099e80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5f0809aad0 tx=0x7f5f0002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.713+0000 7f5f10749640 1 -- 172.21.15.139:0/2455975337 shutdown_connections 2024-04-25T09:55:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.713+0000 7f5f10749640 1 --2- 172.21.15.139:0/2455975337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f08099a50 0x7f5f08099e80 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.713+0000 7f5f10749640 1 --2- 172.21.15.139:0/2455975337 >> v2:172.21.15.150:3300/0 conn(0x7f5f080cf900 0x7f5f080d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.713+0000 7f5f10749640 1 -- 172.21.15.139:0/2455975337 >> 172.21.15.139:0/2455975337 conn(0x7f5f080d1790 msgr2=0x7f5f080d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.714+0000 7f5f10749640 1 -- 172.21.15.139:0/2455975337 shutdown_connections 2024-04-25T09:55:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.714+0000 7f5f10749640 1 -- 172.21.15.139:0/2455975337 wait complete. 2024-04-25T09:55:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.715+0000 7f5f10749640 1 Processor -- start 2024-04-25T09:55:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.715+0000 7f5f10749640 1 -- start start 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.715+0000 7f5f10749640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f08099a50 0x7f5f080b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.715+0000 7f5f10749640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f080cf900 0x7f5f081f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.715+0000 7f5f10749640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f0815efb0 0x7f5f081f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.715+0000 7f5f10749640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0806cf50 con 0x7f5f08099a50 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.715+0000 7f5f10749640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f080d0710 con 0x7f5f080cf900 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.715+0000 7f5f10749640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f080cbde0 con 0x7f5f0815efb0 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.715+0000 7f5f0e4e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f08099a50 0x7f5f080b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.715+0000 7f5f0e4e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f08099a50 0x7f5f080b67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54980/0 (socket says 172.21.15.139:54980) 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.715+0000 7f5f0e4e5640 1 -- 172.21.15.139:0/1036934981 learned_addr learned my addr 172.21.15.139:0/1036934981 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5f0ece6640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f0815efb0 0x7f5f081f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5f0dce4640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f080cf900 0x7f5f081f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5f0e4e5640 1 -- 172.21.15.139:0/1036934981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f0815efb0 msgr2=0x7f5f081f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5f0e4e5640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f0815efb0 0x7f5f081f7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5f0e4e5640 1 -- 172.21.15.139:0/1036934981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f080cf900 msgr2=0x7f5f081f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5f0e4e5640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f080cf900 0x7f5f081f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5f0e4e5640 1 -- 172.21.15.139:0/1036934981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f080c5880 con 0x7f5f08099a50 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5f0ece6640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f0815efb0 0x7f5f081f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5f0dce4640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f080cf900 0x7f5f081f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5f0e4e5640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f08099a50 0x7f5f080b67c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00c430 tx=0x7f5efc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5efaffd640 1 -- 172.21.15.139:0/1036934981 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0074e0 con 0x7f5f08099a50 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5efaffd640 1 -- 172.21.15.139:0/1036934981 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5efc0047e0 con 0x7f5f08099a50 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5f10749640 1 -- 172.21.15.139:0/1036934981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f080c5b60 con 0x7f5f08099a50 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5efaffd640 1 -- 172.21.15.139:0/1036934981 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc004bd0 con 0x7f5f08099a50 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.716+0000 7f5f10749640 1 -- 172.21.15.139:0/1036934981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f080c6120 con 0x7f5f08099a50 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.717+0000 7f5f10749640 1 -- 172.21.15.139:0/1036934981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f08074520 con 0x7f5f08099a50 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.717+0000 7f5efaffd640 1 -- 172.21.15.139:0/1036934981 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5efc011070 con 0x7f5f08099a50 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.718+0000 7f5efaffd640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5eec06b4d0 0x7f5eec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.718+0000 7f5efaffd640 1 -- 172.21.15.139:0/1036934981 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5efc08a050 con 0x7f5f08099a50 2024-04-25T09:55:39.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.718+0000 7f5f0dce4640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5eec06b4d0 0x7f5eec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.718+0000 7f5f0dce4640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5eec06b4d0 0x7f5eec06d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5f080b5d80 tx=0x7f5f04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.722+0000 7f5efaffd640 1 -- 172.21.15.139:0/1036934981 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5efc058d40 con 0x7f5f08099a50 2024-04-25T09:55:39.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.894+0000 7f5f10749640 1 -- 172.21.15.139:0/1036934981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f0806cc20 con 0x7f5f08099a50 2024-04-25T09:55:39.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.895+0000 7f5efaffd640 1 -- 172.21.15.139:0/1036934981 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f5efc058640 con 0x7f5f08099a50 2024-04-25T09:55:39.898 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:39.898 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":50,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:54:58.130580+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:54.661529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:55.431933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:39.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.897+0000 7f5f10749640 1 -- 172.21.15.139:0/1036934981 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5eec06b4d0 msgr2=0x7f5eec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.897+0000 7f5f10749640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5eec06b4d0 0x7f5eec06d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5f080b5d80 tx=0x7f5f04008040 comp rx=0 tx=0).stop 2024-04-25T09:55:39.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.897+0000 7f5f10749640 1 -- 172.21.15.139:0/1036934981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f08099a50 msgr2=0x7f5f080b67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.897+0000 7f5f10749640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f08099a50 0x7f5f080b67c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00c430 tx=0x7f5efc00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.897+0000 7f5f10749640 1 -- 172.21.15.139:0/1036934981 shutdown_connections 2024-04-25T09:55:39.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.897+0000 7f5f10749640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5eec06b4d0 0x7f5eec06d990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.897+0000 7f5f10749640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f0815efb0 0x7f5f081f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.897+0000 7f5f10749640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f080cf900 0x7f5f081f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.897+0000 7f5f10749640 1 --2- 172.21.15.139:0/1036934981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f08099a50 0x7f5f080b67c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.897+0000 7f5f10749640 1 -- 172.21.15.139:0/1036934981 >> 172.21.15.139:0/1036934981 conn(0x7f5f080d1790 msgr2=0x7f5f080ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.897+0000 7f5f10749640 1 -- 172.21.15.139:0/1036934981 shutdown_connections 2024-04-25T09:55:39.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:39.897+0000 7f5f10749640 1 -- 172.21.15.139:0/1036934981 wait complete. 2024-04-25T09:55:39.911 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:55:40.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.006+0000 7febab728640 1 Processor -- start 2024-04-25T09:55:40.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.006+0000 7febab728640 1 -- start start 2024-04-25T09:55:40.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.006+0000 7febab728640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feba40af8a0 0x7feba40b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7febab728640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feba406b700 0x7feba406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7febab728640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba40c6cc0 con 0x7feba40afc80 2024-04-25T09:55:40.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7febab728640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba40c6fa0 con 0x7feba406b700 2024-04-25T09:55:40.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7febab728640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba40c7280 con 0x7feba40af8a0 2024-04-25T09:55:40.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7feba9cc5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feba406b700 0x7feba406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7feba8cc3640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7feba40afc80 0x7feba40cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37704/0 (socket says 172.21.15.139:37704) 2024-04-25T09:55:40.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7feba8cc3640 1 -- 172.21.15.139:0/2715235835 learned_addr learned my addr 172.21.15.139:0/2715235835 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:40.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7feba9cc5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feba406b700 0x7feba406ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54984/0 (socket says 172.21.15.139:54984) 2024-04-25T09:55:40.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7feba94c4640 1 --2- 172.21.15.139:0/2715235835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feba40af8a0 0x7feba40b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7feba9cc5640 1 -- 172.21.15.139:0/2715235835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feba40af8a0 msgr2=0x7feba40b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:40.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7feba9cc5640 1 --2- 172.21.15.139:0/2715235835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feba40af8a0 0x7feba40b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7feba9cc5640 1 -- 172.21.15.139:0/2715235835 >> v1:172.21.15.139:6789/0 conn(0x7feba40afc80 legacy=0x7feba40cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.007+0000 7feba9cc5640 1 -- 172.21.15.139:0/2715235835 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba40cd090 con 0x7feba406b700 2024-04-25T09:55:40.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.008+0000 7feba9cc5640 1 --2- 172.21.15.139:0/2715235835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba406b700 0x7feba406ab60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009d90 tx=0x7feb9c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06c092627eccf52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.008+0000 7feb9bfff640 1 -- 172.21.15.139:0/2715235835 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c007740 con 0x7feba406b700 2024-04-25T09:55:40.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.008+0000 7feb9bfff640 1 -- 172.21.15.139:0/2715235835 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c0078e0 con 0x7feba406b700 2024-04-25T09:55:40.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.008+0000 7feb9bfff640 1 -- 172.21.15.139:0/2715235835 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c007cb0 con 0x7feba406b700 2024-04-25T09:55:40.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.008+0000 7febab728640 1 -- 172.21.15.139:0/2715235835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba406b700 msgr2=0x7feba406ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.008+0000 7febab728640 1 --2- 172.21.15.139:0/2715235835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba406b700 0x7feba406ab60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009d90 tx=0x7feb9c02a660 comp rx=0 tx=0).stop 2024-04-25T09:55:40.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.008+0000 7febab728640 1 -- 172.21.15.139:0/2715235835 shutdown_connections 2024-04-25T09:55:40.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.008+0000 7febab728640 1 --2- 172.21.15.139:0/2715235835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba406b700 0x7feba406ab60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.008+0000 7febab728640 1 --2- 172.21.15.139:0/2715235835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feba40af8a0 0x7feba40b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.008+0000 7febab728640 1 -- 172.21.15.139:0/2715235835 >> 172.21.15.139:0/2715235835 conn(0x7feba40b2230 msgr2=0x7feba40b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:40.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.008+0000 7febab728640 1 -- 172.21.15.139:0/2715235835 shutdown_connections 2024-04-25T09:55:40.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.009+0000 7febab728640 1 -- 172.21.15.139:0/2715235835 wait complete. 2024-04-25T09:55:40.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.010+0000 7febab728640 1 Processor -- start 2024-04-25T09:55:40.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.010+0000 7febab728640 1 -- start start 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.010+0000 7febab728640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feba406b700 0x7feba4160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.010+0000 7febab728640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feba40af8a0 0x7feba414d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.010+0000 7febab728640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba40afc80 0x7feba414d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.010+0000 7febab728640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba406b220 con 0x7feba40afc80 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.010+0000 7febab728640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba406b0a0 con 0x7feba406b700 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.010+0000 7febab728640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba406b3a0 con 0x7feba40af8a0 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.010+0000 7feba9cc5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba40afc80 0x7feba414d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.010+0000 7feba9cc5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba40afc80 0x7feba414d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54994/0 (socket says 172.21.15.139:54994) 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.010+0000 7feba9cc5640 1 -- 172.21.15.139:0/3498286370 learned_addr learned my addr 172.21.15.139:0/3498286370 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7feba8cc3640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feba40af8a0 0x7feba414d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7feba94c4640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feba406b700 0x7feba4160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7feba9cc5640 1 -- 172.21.15.139:0/3498286370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feba40af8a0 msgr2=0x7feba414d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7feba9cc5640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feba40af8a0 0x7feba414d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7feba9cc5640 1 -- 172.21.15.139:0/3498286370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feba406b700 msgr2=0x7feba4160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7feba9cc5640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feba406b700 0x7feba4160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7feba9cc5640 1 -- 172.21.15.139:0/3498286370 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba40b5930 con 0x7feba40afc80 2024-04-25T09:55:40.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7feba9cc5640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba40afc80 0x7feba414d5c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009eb0 tx=0x7feb9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7feb99ffb640 1 -- 172.21.15.139:0/3498286370 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c03f070 con 0x7feba40afc80 2024-04-25T09:55:40.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7feb99ffb640 1 -- 172.21.15.139:0/3498286370 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c031bc0 con 0x7feba40afc80 2024-04-25T09:55:40.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7feb99ffb640 1 -- 172.21.15.139:0/3498286370 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c044460 con 0x7feba40afc80 2024-04-25T09:55:40.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7febab728640 1 -- 172.21.15.139:0/3498286370 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba40b5c10 con 0x7feba40afc80 2024-04-25T09:55:40.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.011+0000 7febab728640 1 -- 172.21.15.139:0/3498286370 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba4161230 con 0x7feba40afc80 2024-04-25T09:55:40.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.012+0000 7febab728640 1 -- 172.21.15.139:0/3498286370 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba415e240 con 0x7feba40afc80 2024-04-25T09:55:40.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.013+0000 7feb99ffb640 1 -- 172.21.15.139:0/3498286370 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb9c007790 con 0x7feba40afc80 2024-04-25T09:55:40.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.013+0000 7feb99ffb640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb8806b4d0 0x7feb8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.013+0000 7feb99ffb640 1 -- 172.21.15.139:0/3498286370 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feb9c0abba0 con 0x7feba40afc80 2024-04-25T09:55:40.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.013+0000 7feba94c4640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb8806b4d0 0x7feb8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.014+0000 7feba94c4640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb8806b4d0 0x7feb8806d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feba0006fd0 tx=0x7feba0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.017+0000 7feb99ffb640 1 -- 172.21.15.139:0/3498286370 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb9c07a890 con 0x7feba40afc80 2024-04-25T09:55:40.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.223+0000 7febab728640 1 -- 172.21.15.139:0/3498286370 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feba4074520 con 0x7feba40afc80 2024-04-25T09:55:40.226 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T09:55:40.224+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:55:40.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.225+0000 7feb99ffb640 1 -- 172.21.15.139:0/3498286370 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7feb9c07a190 con 0x7feba40afc80 2024-04-25T09:55:40.228 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:55:40.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.228+0000 7febab728640 1 -- 172.21.15.139:0/3498286370 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb8806b4d0 msgr2=0x7feb8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.228+0000 7febab728640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb8806b4d0 0x7feb8806d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feba0006fd0 tx=0x7feba0008040 comp rx=0 tx=0).stop 2024-04-25T09:55:40.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.228+0000 7febab728640 1 -- 172.21.15.139:0/3498286370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba40afc80 msgr2=0x7feba414d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.228+0000 7febab728640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba40afc80 0x7feba414d5c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009eb0 tx=0x7feb9c004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:40.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.228+0000 7febab728640 1 -- 172.21.15.139:0/3498286370 shutdown_connections 2024-04-25T09:55:40.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.228+0000 7febab728640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feba40afc80 0x7feba414d5c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.228+0000 7febab728640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb8806b4d0 0x7feb8806d990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.228+0000 7febab728640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feba40af8a0 0x7feba414d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.228+0000 7febab728640 1 --2- 172.21.15.139:0/3498286370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feba406b700 0x7feba4160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.228+0000 7febab728640 1 -- 172.21.15.139:0/3498286370 >> 172.21.15.139:0/3498286370 conn(0x7feba40b2230 msgr2=0x7feba415cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:40.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.228+0000 7febab728640 1 -- 172.21.15.139:0/3498286370 shutdown_connections 2024-04-25T09:55:40.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.228+0000 7febab728640 1 -- 172.21.15.139:0/3498286370 wait complete. 2024-04-25T09:55:40.234 INFO:tasks.ceph.mds.c.smithi139.stderr: -5675> 2024-04-25T09:55:02.387+0000 7efe2051c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:55:40.236 INFO:tasks.ceph.mds.c.smithi139.stderr: -3175> 2024-04-25T09:55:03.927+0000 7efe1c514640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5231=0-4194304@1} caps={5231=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5231 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x563dd530d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T09:55:40.242 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:55:40.263 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-04-25T09:55:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.338+0000 7efe7a15c640 1 Processor -- start 2024-04-25T09:55:40.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.338+0000 7efe7a15c640 1 -- start start 2024-04-25T09:55:40.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.339+0000 7efe7a15c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efe7415a870 0x7efe740b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.339+0000 7efe7a15c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe740b4190 0x7efe740d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.339+0000 7efe7a15c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe740b45f0 con 0x7efe74070260 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.339+0000 7efe7a15c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe740d1020 con 0x7efe7415a870 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.339+0000 7efe7a15c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe740b5190 con 0x7efe740b4190 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.340+0000 7efe737fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efe7415a870 0x7efe740b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.340+0000 7efe72ffd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe740b4190 0x7efe740d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.340+0000 7efe73fff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7efe74070260 0x7efe740ce610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:49552/0 (socket says 172.21.15.139:49552) 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.340+0000 7efe737fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efe7415a870 0x7efe740b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:40888/0 (socket says 172.21.15.139:40888) 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.340+0000 7efe73fff640 1 -- 172.21.15.139:0/3625743663 learned_addr learned my addr 172.21.15.139:0/3625743663 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.340+0000 7efe72ffd640 1 -- 172.21.15.139:0/3625743663 >> v2:172.21.15.150:3300/0 conn(0x7efe7415a870 msgr2=0x7efe740b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.340+0000 7efe72ffd640 1 --2- 172.21.15.139:0/3625743663 >> v2:172.21.15.150:3300/0 conn(0x7efe7415a870 0x7efe740b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.340+0000 7efe72ffd640 1 -- 172.21.15.139:0/3625743663 >> v1:172.21.15.150:6789/0 conn(0x7efe74070260 legacy=0x7efe740ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.340+0000 7efe72ffd640 1 -- 172.21.15.139:0/3625743663 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe740d0d40 con 0x7efe740b4190 2024-04-25T09:55:40.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.340+0000 7efe737fe640 1 --2- 172.21.15.139:0/3625743663 >> v2:172.21.15.150:3300/0 conn(0x7efe7415a870 0x7efe740b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.341+0000 7efe72ffd640 1 --2- 172.21.15.139:0/3625743663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe740b4190 0x7efe740d05f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efe680097a0 tx=0x7efe6802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc80c2dc00f249da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.341+0000 7efe727fc640 1 -- 172.21.15.139:0/3625743663 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68037070 con 0x7efe740b4190 2024-04-25T09:55:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.341+0000 7efe727fc640 1 -- 172.21.15.139:0/3625743663 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6802adb0 con 0x7efe740b4190 2024-04-25T09:55:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.341+0000 7efe727fc640 1 -- 172.21.15.139:0/3625743663 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68032660 con 0x7efe740b4190 2024-04-25T09:55:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.341+0000 7efe7a15c640 1 -- 172.21.15.139:0/3625743663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe740b4190 msgr2=0x7efe740d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.341+0000 7efe7a15c640 1 --2- 172.21.15.139:0/3625743663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe740b4190 0x7efe740d05f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efe680097a0 tx=0x7efe6802a060 comp rx=0 tx=0).stop 2024-04-25T09:55:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.341+0000 7efe7a15c640 1 -- 172.21.15.139:0/3625743663 shutdown_connections 2024-04-25T09:55:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.341+0000 7efe7a15c640 1 --2- 172.21.15.139:0/3625743663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe740b4190 0x7efe740d05f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.341+0000 7efe7a15c640 1 --2- 172.21.15.139:0/3625743663 >> v2:172.21.15.150:3300/0 conn(0x7efe7415a870 0x7efe740b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.341+0000 7efe7a15c640 1 -- 172.21.15.139:0/3625743663 >> 172.21.15.139:0/3625743663 conn(0x7efe740b23f0 msgr2=0x7efe740b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.341+0000 7efe7a15c640 1 -- 172.21.15.139:0/3625743663 shutdown_connections 2024-04-25T09:55:40.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.341+0000 7efe7a15c640 1 -- 172.21.15.139:0/3625743663 wait complete. 2024-04-25T09:55:40.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.343+0000 7efe7a15c640 1 Processor -- start 2024-04-25T09:55:40.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.343+0000 7efe7a15c640 1 -- start start 2024-04-25T09:55:40.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.343+0000 7efe7a15c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe74070260 0x7efe7414fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.343+0000 7efe7a15c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe740b4190 0x7efe7414db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.343+0000 7efe7a15c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7415a870 0x7efe7414e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.343+0000 7efe7a15c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe740706f0 con 0x7efe740b4190 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.343+0000 7efe7a15c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe740b4e20 con 0x7efe74070260 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.343+0000 7efe7a15c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe74070870 con 0x7efe7415a870 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe72ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe740b4190 0x7efe7414db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe72ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe740b4190 0x7efe7414db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55010/0 (socket says 172.21.15.139:55010) 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe72ffd640 1 -- 172.21.15.139:0/2885434504 learned_addr learned my addr 172.21.15.139:0/2885434504 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe73fff640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7415a870 0x7efe7414e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe737fe640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe74070260 0x7efe7414fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe72ffd640 1 -- 172.21.15.139:0/2885434504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7415a870 msgr2=0x7efe7414e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe72ffd640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7415a870 0x7efe7414e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe72ffd640 1 -- 172.21.15.139:0/2885434504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe74070260 msgr2=0x7efe7414fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe72ffd640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe74070260 0x7efe7414fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe72ffd640 1 -- 172.21.15.139:0/2885434504 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe74150600 con 0x7efe740b4190 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe72ffd640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe740b4190 0x7efe7414db90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7efe680098c0 tx=0x7efe680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe5bfff640 1 -- 172.21.15.139:0/2885434504 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68041030 con 0x7efe740b4190 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe5bfff640 1 -- 172.21.15.139:0/2885434504 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6803c670 con 0x7efe740b4190 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe7a15c640 1 -- 172.21.15.139:0/2885434504 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7414e610 con 0x7efe740b4190 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe5bfff640 1 -- 172.21.15.139:0/2885434504 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68037040 con 0x7efe740b4190 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.344+0000 7efe7a15c640 1 -- 172.21.15.139:0/2885434504 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7415c050 con 0x7efe740b4190 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.345+0000 7efe7a15c640 1 -- 172.21.15.139:0/2885434504 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe74074520 con 0x7efe740b4190 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.346+0000 7efe5bfff640 1 -- 172.21.15.139:0/2885434504 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe68044070 con 0x7efe740b4190 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.346+0000 7efe5bfff640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe6006b4d0 0x7efe6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.346+0000 7efe5bfff640 1 -- 172.21.15.139:0/2885434504 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6187+0+0 (secure 0 0 0) 0x7efe680aabd0 con 0x7efe740b4190 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.346+0000 7efe737fe640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe6006b4d0 0x7efe6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.347+0000 7efe737fe640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe6006b4d0 0x7efe6006d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efe740b0530 tx=0x7efe64008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.350+0000 7efe5bfff640 1 -- 172.21.15.139:0/2885434504 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe68044e40 con 0x7efe740b4190 2024-04-25T09:55:40.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:40.555+0000 7efe7a15c640 1 -- 172.21.15.139:0/2885434504 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efe740b2100 con 0x7efe740b4190 2024-04-25T09:55:41.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.226+0000 7efe5bfff640 1 -- 172.21.15.139:0/2885434504 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7efe68079990 con 0x7efe740b4190 2024-04-25T09:55:41.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.230+0000 7efe7a15c640 1 -- 172.21.15.139:0/2885434504 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe6006b4d0 msgr2=0x7efe6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.230+0000 7efe7a15c640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe6006b4d0 0x7efe6006d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efe740b0530 tx=0x7efe64008040 comp rx=0 tx=0).stop 2024-04-25T09:55:41.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.230+0000 7efe7a15c640 1 -- 172.21.15.139:0/2885434504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe740b4190 msgr2=0x7efe7414db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.230+0000 7efe7a15c640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe740b4190 0x7efe7414db90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7efe680098c0 tx=0x7efe680046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.230+0000 7efe7a15c640 1 -- 172.21.15.139:0/2885434504 shutdown_connections 2024-04-25T09:55:41.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.230+0000 7efe7a15c640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe740b4190 0x7efe7414db90 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.230+0000 7efe7a15c640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe6006b4d0 0x7efe6006d990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.230+0000 7efe7a15c640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7415a870 0x7efe7414e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.230+0000 7efe7a15c640 1 --2- 172.21.15.139:0/2885434504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe74070260 0x7efe7414fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.230+0000 7efe7a15c640 1 -- 172.21.15.139:0/2885434504 >> 172.21.15.139:0/2885434504 conn(0x7efe740b23f0 msgr2=0x7efe740cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:41.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.230+0000 7efe7a15c640 1 -- 172.21.15.139:0/2885434504 shutdown_connections 2024-04-25T09:55:41.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.230+0000 7efe7a15c640 1 -- 172.21.15.139:0/2885434504 wait complete. 2024-04-25T09:55:41.245 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:55:41.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.387+0000 7fa2e5ff9640 1 Processor -- start 2024-04-25T09:55:41.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.387+0000 7fa2e5ff9640 1 -- start start 2024-04-25T09:55:41.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.388+0000 7fa2e5ff9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa2e00d0a80 0x7fa2e01618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.388+0000 7fa2e5ff9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e009ad40 0x7fa2e0161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.388+0000 7fa2e5ff9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e0099ca0 con 0x7fa2e00cd610 2024-04-25T09:55:41.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.388+0000 7fa2e5ff9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e0099a50 con 0x7fa2e00d0a80 2024-04-25T09:55:41.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.388+0000 7fa2e5ff9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0099f80 con 0x7fa2e009ad40 2024-04-25T09:55:41.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.388+0000 7fa2df7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa2e00cd610 0x7fa2e00d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37732/0 (socket says 172.21.15.139:37732) 2024-04-25T09:55:41.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.388+0000 7fa2df7fe640 1 -- 172.21.15.139:0/1436031648 learned_addr learned my addr 172.21.15.139:0/1436031648 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:41.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173715675 0 0) 0x7fa2e0099ca0 con 0x7fa2e00cd610 2024-04-25T09:55:41.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c8003520 con 0x7fa2e00cd610 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2dffff640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e009ad40 0x7fa2e0161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2deffd640 1 --2- 172.21.15.139:0/1436031648 >> v2:172.21.15.150:3300/0 conn(0x7fa2e00d0a80 0x7fa2e01618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2839251729 0 0) 0x7fa2c8003520 con 0x7fa2e00cd610 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e009ad40 msgr2=0x7fa2e0161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e009ad40 0x7fa2e0161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 >> v2:172.21.15.150:3300/0 conn(0x7fa2e00d0a80 msgr2=0x7fa2e01618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 --2- 172.21.15.139:0/1436031648 >> v2:172.21.15.150:3300/0 conn(0x7fa2e00d0a80 0x7fa2e01618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e009a260 con 0x7fa2e00cd610 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fa2d0001670 con 0x7fa2e00cd610 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 >> v1:172.21.15.139:6789/0 conn(0x7fa2e00cd610 legacy=0x7fa2e00d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c80039b0 0x7fa2c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e009ad40 0x7fa2c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2c8004d40 0x7fa2c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0099a50 con 0x7fa2c8004d40 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0099f80 con 0x7fa2c80039b0 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c800b9c0 con 0x7fa2e009ad40 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2dffff640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2c8004d40 0x7fa2c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2df7fe640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c80039b0 0x7fa2c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2deffd640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e009ad40 0x7fa2c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2dffff640 1 -- 172.21.15.139:0/1436031648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e009ad40 msgr2=0x7fa2c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2dffff640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e009ad40 0x7fa2c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2dffff640 1 -- 172.21.15.139:0/1436031648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c80039b0 msgr2=0x7fa2c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2dffff640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c80039b0 0x7fa2c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2dffff640 1 -- 172.21.15.139:0/1436031648 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c8003520 con 0x7fa2c8004d40 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.389+0000 7fa2df7fe640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2c80039b0 0x7fa2c8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:41.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.390+0000 7fa2dffff640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2c8004d40 0x7fa2c80051c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009fc0 tx=0x7fa2c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2a6f25f9255713 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:41.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.390+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8009c10 con 0x7fa2c8004d40 2024-04-25T09:55:41.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.390+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d8036070 con 0x7fa2c8004d40 2024-04-25T09:55:41.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.390+0000 7fa2de7fc640 1 -- 172.21.15.139:0/1436031648 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8031880 con 0x7fa2c8004d40 2024-04-25T09:55:41.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.391+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1436031648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2c8004d40 msgr2=0x7fa2c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.391+0000 7fa2e5ff9640 1 --2- 172.21.15.139:0/1436031648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2c8004d40 0x7fa2c80051c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009fc0 tx=0x7fa2c8005830 comp rx=0 tx=0).stop 2024-04-25T09:55:41.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.391+0000 7fa2dffff640 1 -- 172.21.15.139:0/1436031648 reap_dead start 2024-04-25T09:55:41.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.391+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1436031648 shutdown_connections 2024-04-25T09:55:41.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.391+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1436031648 >> 172.21.15.139:0/1436031648 conn(0x7fa2e00b31f0 msgr2=0x7fa2e00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:41.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.391+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1436031648 shutdown_connections 2024-04-25T09:55:41.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.391+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1436031648 wait complete. 2024-04-25T09:55:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.392+0000 7fa2e5ff9640 1 Processor -- start 2024-04-25T09:55:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.392+0000 7fa2e5ff9640 1 -- start start 2024-04-25T09:55:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2e5ff9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e01d36a0 0x7fa2e00c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2e5ff9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e00b6d90 0x7fa2e00b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2e5ff9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2e00ae8b0 0x7fa2e00aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2e5ff9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00cd9f0 con 0x7fa2e00b6d90 2024-04-25T09:55:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2e5ff9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e009a820 con 0x7fa2e00ae8b0 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2e5ff9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00cdb70 con 0x7fa2e01d36a0 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2deffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e00b6d90 0x7fa2e00b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2deffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e00b6d90 0x7fa2e00b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55030/0 (socket says 172.21.15.139:55030) 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2deffd640 1 -- 172.21.15.139:0/1723331101 learned_addr learned my addr 172.21.15.139:0/1723331101 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2df7fe640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e01d36a0 0x7fa2e00c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2dffff640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2e00ae8b0 0x7fa2e00aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2deffd640 1 -- 172.21.15.139:0/1723331101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e01d36a0 msgr2=0x7fa2e00c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2deffd640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e01d36a0 0x7fa2e00c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2deffd640 1 -- 172.21.15.139:0/1723331101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2e00ae8b0 msgr2=0x7fa2e00aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2deffd640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2e00ae8b0 0x7fa2e00aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2deffd640 1 -- 172.21.15.139:0/1723331101 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00b5cf0 con 0x7fa2e00b6d90 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2df7fe640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e01d36a0 0x7fa2e00c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2deffd640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e00b6d90 0x7fa2e00b55f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4011aa0 tx=0x7fa2d4011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.393+0000 7fa2c7fff640 1 -- 172.21.15.139:0/1723331101 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400fb80 con 0x7fa2e00b6d90 2024-04-25T09:55:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.394+0000 7fa2c7fff640 1 -- 172.21.15.139:0/1723331101 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d400fd20 con 0x7fa2e00b6d90 2024-04-25T09:55:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.394+0000 7fa2c7fff640 1 -- 172.21.15.139:0/1723331101 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4014660 con 0x7fa2e00b6d90 2024-04-25T09:55:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.394+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1723331101 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e00b5fd0 con 0x7fa2e00b6d90 2024-04-25T09:55:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.394+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1723331101 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e00b6590 con 0x7fa2e00b6d90 2024-04-25T09:55:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.394+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1723331101 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e0066e10 con 0x7fa2e00b6d90 2024-04-25T09:55:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.395+0000 7fa2c7fff640 1 -- 172.21.15.139:0/1723331101 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa2d4012040 con 0x7fa2e00b6d90 2024-04-25T09:55:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.395+0000 7fa2c7fff640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa2cc06b500 0x7fa2cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.395+0000 7fa2c7fff640 1 -- 172.21.15.139:0/1723331101 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa2d408aaf0 con 0x7fa2e00b6d90 2024-04-25T09:55:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.395+0000 7fa2df7fe640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa2cc06b500 0x7fa2cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.396+0000 7fa2df7fe640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa2cc06b500 0x7fa2cc06d9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0000fb0 tx=0x7fa2d0001e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:41.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.399+0000 7fa2c7fff640 1 -- 172.21.15.139:0/1723331101 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2e0066e10 con 0x7fa2e00b6d90 2024-04-25T09:55:41.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:41.569+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1723331101 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa2e0074520 con 0x7fa2e00b6d90 2024-04-25T09:55:42.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.228+0000 7fa2c7fff640 1 -- 172.21.15.139:0/1723331101 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa2d4019030 con 0x7fa2e00b6d90 2024-04-25T09:55:42.231 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:55:42.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.231+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1723331101 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa2cc06b500 msgr2=0x7fa2cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.231+0000 7fa2e5ff9640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa2cc06b500 0x7fa2cc06d9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0000fb0 tx=0x7fa2d0001e30 comp rx=0 tx=0).stop 2024-04-25T09:55:42.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.231+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1723331101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e00b6d90 msgr2=0x7fa2e00b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.231+0000 7fa2e5ff9640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e00b6d90 0x7fa2e00b55f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4011aa0 tx=0x7fa2d4011f60 comp rx=0 tx=0).stop 2024-04-25T09:55:42.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.232+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1723331101 shutdown_connections 2024-04-25T09:55:42.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.232+0000 7fa2e5ff9640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e00b6d90 0x7fa2e00b55f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.232+0000 7fa2e5ff9640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa2cc06b500 0x7fa2cc06d9c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.232+0000 7fa2e5ff9640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2e00ae8b0 0x7fa2e00aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.232+0000 7fa2e5ff9640 1 --2- 172.21.15.139:0/1723331101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e01d36a0 0x7fa2e00c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.232+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1723331101 >> 172.21.15.139:0/1723331101 conn(0x7fa2e00b31f0 msgr2=0x7fa2e0070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:42.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.232+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1723331101 shutdown_connections 2024-04-25T09:55:42.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.232+0000 7fa2e5ff9640 1 -- 172.21.15.139:0/1723331101 wait complete. 2024-04-25T09:55:42.247 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:55:42.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.353+0000 7fe5fc64a640 1 Processor -- start 2024-04-25T09:55:42.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.353+0000 7fe5fc64a640 1 -- start start 2024-04-25T09:55:42.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.354+0000 7fe5fc64a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe5f40d0a80 0x7fe5f41618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:42.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.354+0000 7fe5fc64a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5f409ad40 0x7fe5f4161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.354+0000 7fe5fc64a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f4099ca0 con 0x7fe5f40cd610 2024-04-25T09:55:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.354+0000 7fe5fc64a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f4099a50 con 0x7fe5f40d0a80 2024-04-25T09:55:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.354+0000 7fe5fc64a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f4099f80 con 0x7fe5f409ad40 2024-04-25T09:55:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.354+0000 7fe5fa3e6640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe5f40cd610 0x7fe5f40d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37734/0 (socket says 172.21.15.139:37734) 2024-04-25T09:55:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.354+0000 7fe5fa3e6640 1 -- 172.21.15.139:0/650257872 learned_addr learned my addr 172.21.15.139:0/650257872 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:42.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.354+0000 7fe5f9be5640 1 --2- 172.21.15.139:0/650257872 >> v2:172.21.15.150:3300/0 conn(0x7fe5f40d0a80 0x7fe5f41618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.355+0000 7fe5f93e4640 1 -- 172.21.15.139:0/650257872 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403832233 0 0) 0x7fe5f4099ca0 con 0x7fe5f40cd610 2024-04-25T09:55:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.355+0000 7fe5f93e4640 1 -- 172.21.15.139:0/650257872 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e0003520 con 0x7fe5f40cd610 2024-04-25T09:55:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.355+0000 7fe5fabe7640 1 --2- 172.21.15.139:0/650257872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5f409ad40 0x7fe5f4161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.355+0000 7fe5f9be5640 1 -- 172.21.15.139:0/650257872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5f409ad40 msgr2=0x7fe5f4161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.355+0000 7fe5f9be5640 1 --2- 172.21.15.139:0/650257872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5f409ad40 0x7fe5f4161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.355+0000 7fe5f9be5640 1 -- 172.21.15.139:0/650257872 >> v1:172.21.15.139:6789/0 conn(0x7fe5f40cd610 legacy=0x7fe5f40d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.355+0000 7fe5f9be5640 1 -- 172.21.15.139:0/650257872 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f409a260 con 0x7fe5f40d0a80 2024-04-25T09:55:42.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.356+0000 7fe5f9be5640 1 --2- 172.21.15.139:0/650257872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f40d0a80 0x7fe5f41618c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40097a0 tx=0x7fe5e402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee48f1ccb4d5512b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:42.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.356+0000 7fe5f93e4640 1 -- 172.21.15.139:0/650257872 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4037070 con 0x7fe5f40d0a80 2024-04-25T09:55:42.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.356+0000 7fe5f93e4640 1 -- 172.21.15.139:0/650257872 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e4032440 con 0x7fe5f40d0a80 2024-04-25T09:55:42.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.356+0000 7fe5f93e4640 1 -- 172.21.15.139:0/650257872 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40327f0 con 0x7fe5f40d0a80 2024-04-25T09:55:42.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.357+0000 7fe5fc64a640 1 -- 172.21.15.139:0/650257872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f40d0a80 msgr2=0x7fe5f41618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.357+0000 7fe5fc64a640 1 --2- 172.21.15.139:0/650257872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f40d0a80 0x7fe5f41618c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40097a0 tx=0x7fe5e402a1c0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.358+0000 7fe5fc64a640 1 -- 172.21.15.139:0/650257872 shutdown_connections 2024-04-25T09:55:42.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.358+0000 7fe5fc64a640 1 --2- 172.21.15.139:0/650257872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5f409ad40 0x7fe5f4161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.358+0000 7fe5fc64a640 1 --2- 172.21.15.139:0/650257872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f40d0a80 0x7fe5f41618c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.358+0000 7fe5fc64a640 1 -- 172.21.15.139:0/650257872 >> 172.21.15.139:0/650257872 conn(0x7fe5f40b31f0 msgr2=0x7fe5f40b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:42.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.358+0000 7fe5fc64a640 1 -- 172.21.15.139:0/650257872 shutdown_connections 2024-04-25T09:55:42.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.358+0000 7fe5fc64a640 1 -- 172.21.15.139:0/650257872 wait complete. 2024-04-25T09:55:42.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.359+0000 7fe5fc64a640 1 Processor -- start 2024-04-25T09:55:42.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.359+0000 7fe5fc64a640 1 -- start start 2024-04-25T09:55:42.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.359+0000 7fe5fc64a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f409ad40 0x7fe5f414fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:42.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.359+0000 7fe5fc64a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5f40cd610 0x7fe5f40b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:42.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.359+0000 7fe5fc64a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5f40d0a80 0x7fe5f4070080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:42.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.359+0000 7fe5fc64a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40cd9f0 con 0x7fe5f40cd610 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.359+0000 7fe5fc64a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f409a820 con 0x7fe5f409ad40 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.359+0000 7fe5fc64a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40cdb70 con 0x7fe5f40d0a80 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5f9be5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5f40cd610 0x7fe5f40b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5f9be5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5f40cd610 0x7fe5f40b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55038/0 (socket says 172.21.15.139:55038) 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5f9be5640 1 -- 172.21.15.139:0/224175958 learned_addr learned my addr 172.21.15.139:0/224175958 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5fabe7640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5f40d0a80 0x7fe5f4070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5fa3e6640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f409ad40 0x7fe5f414fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5f9be5640 1 -- 172.21.15.139:0/224175958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5f40d0a80 msgr2=0x7fe5f4070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5f9be5640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5f40d0a80 0x7fe5f4070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5f9be5640 1 -- 172.21.15.139:0/224175958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f409ad40 msgr2=0x7fe5f414fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5f9be5640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f409ad40 0x7fe5f414fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5f9be5640 1 -- 172.21.15.139:0/224175958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f40b6440 con 0x7fe5f40cd610 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5fabe7640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5f40d0a80 0x7fe5f4070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5f9be5640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5f40cd610 0x7fe5f40b5ba0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe5e402a200 tx=0x7fe5e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:42.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5eaffd640 1 -- 172.21.15.139:0/224175958 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4037040 con 0x7fe5f40cd610 2024-04-25T09:55:42.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5eaffd640 1 -- 172.21.15.139:0/224175958 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e4042070 con 0x7fe5f40cd610 2024-04-25T09:55:42.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5fc64a640 1 -- 172.21.15.139:0/224175958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f40b6720 con 0x7fe5f40cd610 2024-04-25T09:55:42.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5eaffd640 1 -- 172.21.15.139:0/224175958 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40308c0 con 0x7fe5f40cd610 2024-04-25T09:55:42.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.360+0000 7fe5fc64a640 1 -- 172.21.15.139:0/224175958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f40c56b0 con 0x7fe5f40cd610 2024-04-25T09:55:42.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.362+0000 7fe5fc64a640 1 -- 172.21.15.139:0/224175958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f4074520 con 0x7fe5f40cd610 2024-04-25T09:55:42.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.362+0000 7fe5eaffd640 1 -- 172.21.15.139:0/224175958 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5e4030a60 con 0x7fe5f40cd610 2024-04-25T09:55:42.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.362+0000 7fe5eaffd640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe5d806b500 0x7fe5d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:42.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.362+0000 7fe5fa3e6640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe5d806b500 0x7fe5d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:42.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.362+0000 7fe5eaffd640 1 -- 172.21.15.139:0/224175958 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fe5e40aaaf0 con 0x7fe5f40cd610 2024-04-25T09:55:42.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.363+0000 7fe5fa3e6640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe5d806b500 0x7fe5d806d9c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0007ac0 tx=0x7fe5f0007a10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:42.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.367+0000 7fe5eaffd640 1 -- 172.21.15.139:0/224175958 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5e4079940 con 0x7fe5f40cd610 2024-04-25T09:55:42.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:42.537+0000 7fe5fc64a640 1 -- 172.21.15.139:0/224175958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe5f4071030 con 0x7fe5f40cd610 2024-04-25T09:55:43.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.229+0000 7fe5eaffd640 1 -- 172.21.15.139:0/224175958 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe5e4079240 con 0x7fe5f40cd610 2024-04-25T09:55:43.233 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' removed 2024-04-25T09:55:43.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.232+0000 7fe5fc64a640 1 -- 172.21.15.139:0/224175958 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe5d806b500 msgr2=0x7fe5d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.232+0000 7fe5fc64a640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe5d806b500 0x7fe5d806d9c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0007ac0 tx=0x7fe5f0007a10 comp rx=0 tx=0).stop 2024-04-25T09:55:43.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.232+0000 7fe5fc64a640 1 -- 172.21.15.139:0/224175958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5f40cd610 msgr2=0x7fe5f40b5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.232+0000 7fe5fc64a640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5f40cd610 0x7fe5f40b5ba0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe5e402a200 tx=0x7fe5e40046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.233+0000 7fe5fc64a640 1 -- 172.21.15.139:0/224175958 shutdown_connections 2024-04-25T09:55:43.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.233+0000 7fe5fc64a640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5f40cd610 0x7fe5f40b5ba0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.233+0000 7fe5fc64a640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe5d806b500 0x7fe5d806d9c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.233+0000 7fe5fc64a640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5f40d0a80 0x7fe5f4070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.233+0000 7fe5fc64a640 1 --2- 172.21.15.139:0/224175958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f409ad40 0x7fe5f414fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.233+0000 7fe5fc64a640 1 -- 172.21.15.139:0/224175958 >> 172.21.15.139:0/224175958 conn(0x7fe5f40b31f0 msgr2=0x7fe5f40afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:43.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.233+0000 7fe5fc64a640 1 -- 172.21.15.139:0/224175958 shutdown_connections 2024-04-25T09:55:43.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.233+0000 7fe5fc64a640 1 -- 172.21.15.139:0/224175958 wait complete. 2024-04-25T09:55:43.250 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:55:43.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.347+0000 7f4884310640 1 Processor -- start 2024-04-25T09:55:43.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.347+0000 7f4884310640 1 -- start start 2024-04-25T09:55:43.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.347+0000 7f4884310640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f487c0cd610 0x7f487c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f4884310640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f487c09ad40 0x7f487c0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f4884310640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c099ca0 con 0x7f487c1618c0 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f4884310640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487c099a50 con 0x7f487c09ad40 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f4884310640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487c099f80 con 0x7f487c0cd610 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f48818ab640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f487c09ad40 0x7f487c0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f48818ab640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f487c09ad40 0x7f487c0d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55048/0 (socket says 172.21.15.139:55048) 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f48818ab640 1 -- 172.21.15.139:0/2905299215 learned_addr learned my addr 172.21.15.139:0/2905299215 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f48820ac640 1 --2- 172.21.15.139:0/2905299215 >> v2:172.21.15.150:3300/0 conn(0x7f487c0cd610 0x7f487c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f48818ab640 1 -- 172.21.15.139:0/2905299215 >> v2:172.21.15.150:3300/0 conn(0x7f487c0cd610 msgr2=0x7f487c0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f48818ab640 1 --2- 172.21.15.139:0/2905299215 >> v2:172.21.15.150:3300/0 conn(0x7f487c0cd610 0x7f487c0d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f48818ab640 1 -- 172.21.15.139:0/2905299215 >> v1:172.21.15.150:6789/0 conn(0x7f487c1618c0 legacy=0x7f487c06be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f48818ab640 1 -- 172.21.15.139:0/2905299215 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c09a260 con 0x7f487c09ad40 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.348+0000 7f48820ac640 1 --2- 172.21.15.139:0/2905299215 >> v2:172.21.15.150:3300/0 conn(0x7f487c0cd610 0x7f487c0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.349+0000 7f48818ab640 1 --2- 172.21.15.139:0/2905299215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f487c09ad40 0x7f487c0d08b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f487c06f210 tx=0x7f486c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a180e17822a4478a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.349+0000 7f48810aa640 1 -- 172.21.15.139:0/2905299215 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c037070 con 0x7f487c09ad40 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.349+0000 7f48810aa640 1 -- 172.21.15.139:0/2905299215 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c032440 con 0x7f487c09ad40 2024-04-25T09:55:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.349+0000 7f48810aa640 1 -- 172.21.15.139:0/2905299215 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c032810 con 0x7f487c09ad40 2024-04-25T09:55:43.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.349+0000 7f4884310640 1 -- 172.21.15.139:0/2905299215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f487c09ad40 msgr2=0x7f487c0d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.349+0000 7f4884310640 1 --2- 172.21.15.139:0/2905299215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f487c09ad40 0x7f487c0d08b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f487c06f210 tx=0x7f486c02a2c0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.349+0000 7f4884310640 1 -- 172.21.15.139:0/2905299215 shutdown_connections 2024-04-25T09:55:43.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.349+0000 7f4884310640 1 --2- 172.21.15.139:0/2905299215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f487c09ad40 0x7f487c0d08b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.349+0000 7f4884310640 1 --2- 172.21.15.139:0/2905299215 >> v2:172.21.15.150:3300/0 conn(0x7f487c0cd610 0x7f487c0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.349+0000 7f4884310640 1 -- 172.21.15.139:0/2905299215 >> 172.21.15.139:0/2905299215 conn(0x7f487c0b31f0 msgr2=0x7f487c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.349+0000 7f4884310640 1 -- 172.21.15.139:0/2905299215 shutdown_connections 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.349+0000 7f4884310640 1 -- 172.21.15.139:0/2905299215 wait complete. 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.351+0000 7f4884310640 1 Processor -- start 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.351+0000 7f4884310640 1 -- start start 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.351+0000 7f4884310640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f487c09ad40 0x7f487c0aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.351+0000 7f4884310640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f487c0cd610 0x7f487c15c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.351+0000 7f4884310640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f487c1618c0 0x7f487c0b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.351+0000 7f4884310640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0cd9f0 con 0x7f487c09ad40 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.351+0000 7f4884310640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c09a820 con 0x7f487c0cd610 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.351+0000 7f4884310640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0cdb70 con 0x7f487c1618c0 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.351+0000 7f48820ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f487c09ad40 0x7f487c0aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.351+0000 7f48820ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f487c09ad40 0x7f487c0aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55050/0 (socket says 172.21.15.139:55050) 2024-04-25T09:55:43.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.351+0000 7f48820ac640 1 -- 172.21.15.139:0/335514576 learned_addr learned my addr 172.21.15.139:0/335514576 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f48818ab640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f487c0cd610 0x7f487c15c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f48828ad640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f487c1618c0 0x7f487c0b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f48820ac640 1 -- 172.21.15.139:0/335514576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f487c1618c0 msgr2=0x7f487c0b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f48820ac640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f487c1618c0 0x7f487c0b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f48820ac640 1 -- 172.21.15.139:0/335514576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f487c0cd610 msgr2=0x7f487c15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f48820ac640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f487c0cd610 0x7f487c15c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f48820ac640 1 -- 172.21.15.139:0/335514576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c0b4870 con 0x7f487c09ad40 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f48818ab640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f487c0cd610 0x7f487c15c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f48828ad640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f487c1618c0 0x7f487c0b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f48820ac640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f487c09ad40 0x7f487c0aee40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f487800c7f0 tx=0x7f487800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f4872ffd640 1 -- 172.21.15.139:0/335514576 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878015070 con 0x7f487c09ad40 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f4872ffd640 1 -- 172.21.15.139:0/335514576 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4878007a30 con 0x7f487c09ad40 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f4884310640 1 -- 172.21.15.139:0/335514576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c0af540 con 0x7f487c09ad40 2024-04-25T09:55:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f4872ffd640 1 -- 172.21.15.139:0/335514576 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878007e00 con 0x7f487c09ad40 2024-04-25T09:55:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.352+0000 7f4884310640 1 -- 172.21.15.139:0/335514576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c0afb00 con 0x7f487c09ad40 2024-04-25T09:55:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.353+0000 7f4884310640 1 -- 172.21.15.139:0/335514576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487c0afd40 con 0x7f487c09ad40 2024-04-25T09:55:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.354+0000 7f4872ffd640 1 -- 172.21.15.139:0/335514576 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4878018070 con 0x7f487c09ad40 2024-04-25T09:55:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.354+0000 7f4872ffd640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f486006b4d0 0x7f486006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.354+0000 7f4872ffd640 1 -- 172.21.15.139:0/335514576 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4878089f70 con 0x7f487c09ad40 2024-04-25T09:55:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.354+0000 7f48818ab640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f486006b4d0 0x7f486006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.354+0000 7f48818ab640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f486006b4d0 0x7f486006d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f486c034000 tx=0x7f486c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:43.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.358+0000 7f4872ffd640 1 -- 172.21.15.139:0/335514576 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f487c0afd40 con 0x7f487c09ad40 2024-04-25T09:55:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.538+0000 7f4884310640 1 -- 172.21.15.139:0/335514576 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f487c074520 con 0x7f487c09ad40 2024-04-25T09:55:43.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.541+0000 7f4872ffd640 1 -- 172.21.15.139:0/335514576 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4878058ff0 con 0x7f487c09ad40 2024-04-25T09:55:43.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.543+0000 7f4884310640 1 -- 172.21.15.139:0/335514576 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f486006b4d0 msgr2=0x7f486006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.543+0000 7f4884310640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f486006b4d0 0x7f486006d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f486c034000 tx=0x7f486c02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:43.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.543+0000 7f4884310640 1 -- 172.21.15.139:0/335514576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f487c09ad40 msgr2=0x7f487c0aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.543+0000 7f4884310640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f487c09ad40 0x7f487c0aee40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f487800c7f0 tx=0x7f487800cbf0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.543+0000 7f4884310640 1 -- 172.21.15.139:0/335514576 shutdown_connections 2024-04-25T09:55:43.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.543+0000 7f4884310640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f486006b4d0 0x7f486006d990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.543+0000 7f4884310640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f487c1618c0 0x7f487c0b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.543+0000 7f4884310640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f487c0cd610 0x7f487c15c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.543+0000 7f4884310640 1 --2- 172.21.15.139:0/335514576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f487c09ad40 0x7f487c0aee40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.543+0000 7f4884310640 1 -- 172.21.15.139:0/335514576 >> 172.21.15.139:0/335514576 conn(0x7f487c0b31f0 msgr2=0x7f487c156f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:43.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.543+0000 7f4884310640 1 -- 172.21.15.139:0/335514576 shutdown_connections 2024-04-25T09:55:43.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.543+0000 7f4884310640 1 -- 172.21.15.139:0/335514576 wait complete. 2024-04-25T09:55:43.558 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T09:55:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.651+0000 7f22de6b3640 1 Processor -- start 2024-04-25T09:55:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.651+0000 7f22de6b3640 1 -- start start 2024-04-25T09:55:43.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22de6b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22d80b37d0 0x7f22d815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22de6b3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d80b3e10 con 0x7f22d806da20 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22de6b3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d81615c0 con 0x7f22d8158260 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22de6b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d81618a0 con 0x7f22d80b37d0 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22dcc50640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f22d806da20 0x7f22d806ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37748/0 (socket says 172.21.15.139:37748) 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22dcc50640 1 -- 172.21.15.139:0/1874578943 learned_addr learned my addr 172.21.15.139:0/1874578943 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3324345482 0 0) 0x7f22d0001490 con 0x7f22d806da20 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22c0003520 con 0x7f22d806da20 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22d77fe640 1 --2- 172.21.15.139:0/1874578943 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22d80b37d0 0x7f22d815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2537604113 0 0) 0x7f22c0003520 con 0x7f22d806da20 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22d80b37d0 msgr2=0x7f22d815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22d6ffd640 1 --2- 172.21.15.139:0/1874578943 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22d80b37d0 0x7f22d815df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 >> v1:172.21.15.150:6789/0 conn(0x7f22d8158260 legacy=0x7f22d80b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d8161b80 con 0x7f22d806da20 2024-04-25T09:55:43.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f22d0001bc0 con 0x7f22d806da20 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 >> v1:172.21.15.139:6789/0 conn(0x7f22d806da20 legacy=0x7f22d806ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.652+0000 7f22d6ffd640 1 --2- 172.21.15.139:0/1874578943 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22d80b37d0 0x7f22c00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22d6ffd640 1 --2- 172.21.15.139:0/1874578943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22c0004010 0x7f22c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22d6ffd640 1 --2- 172.21.15.139:0/1874578943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0004d40 0x7f22c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d81618a0 con 0x7f22c0004d40 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c000bb80 con 0x7f22c0004010 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c000bd80 con 0x7f22d80b37d0 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22dcc50640 1 --2- 172.21.15.139:0/1874578943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0004d40 0x7f22c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22d77fe640 1 --2- 172.21.15.139:0/1874578943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22c0004010 0x7f22c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22d7fff640 1 --2- 172.21.15.139:0/1874578943 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22d80b37d0 0x7f22c00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22dcc50640 1 -- 172.21.15.139:0/1874578943 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22d80b37d0 msgr2=0x7f22c00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22dcc50640 1 --2- 172.21.15.139:0/1874578943 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22d80b37d0 0x7f22c00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22dcc50640 1 -- 172.21.15.139:0/1874578943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22c0004010 msgr2=0x7f22c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22dcc50640 1 --2- 172.21.15.139:0/1874578943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22c0004010 0x7f22c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.653+0000 7f22dcc50640 1 -- 172.21.15.139:0/1874578943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c0003520 con 0x7f22c0004d40 2024-04-25T09:55:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.654+0000 7f22dcc50640 1 --2- 172.21.15.139:0/1874578943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0004d40 0x7f22c00051c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f22d000aa20 tx=0x7f22c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8fef367305d24b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.654+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000a670 con 0x7f22c0004d40 2024-04-25T09:55:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.654+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d002f070 con 0x7f22c0004d40 2024-04-25T09:55:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.654+0000 7f22d6ffd640 1 -- 172.21.15.139:0/1874578943 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0034470 con 0x7f22c0004d40 2024-04-25T09:55:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.654+0000 7f22de6b3640 1 -- 172.21.15.139:0/1874578943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0004d40 msgr2=0x7f22c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.654+0000 7f22de6b3640 1 --2- 172.21.15.139:0/1874578943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c0004d40 0x7f22c00051c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f22d000aa20 tx=0x7f22c0005830 comp rx=0 tx=0).stop 2024-04-25T09:55:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.654+0000 7f22dcc50640 1 -- 172.21.15.139:0/1874578943 reap_dead start 2024-04-25T09:55:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.654+0000 7f22de6b3640 1 -- 172.21.15.139:0/1874578943 shutdown_connections 2024-04-25T09:55:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.654+0000 7f22de6b3640 1 -- 172.21.15.139:0/1874578943 >> 172.21.15.139:0/1874578943 conn(0x7f22d80b23d0 msgr2=0x7f22d80cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.654+0000 7f22de6b3640 1 -- 172.21.15.139:0/1874578943 shutdown_connections 2024-04-25T09:55:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.654+0000 7f22de6b3640 1 -- 172.21.15.139:0/1874578943 wait complete. 2024-04-25T09:55:43.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.655+0000 7f22de6b3640 1 Processor -- start 2024-04-25T09:55:43.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.656+0000 7f22de6b3640 1 -- start start 2024-04-25T09:55:43.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.656+0000 7f22de6b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22c0004d40 0x7f22d80ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.656+0000 7f22de6b3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22d81e7420 0x7f22d8144710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.656+0000 7f22de6b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d8156710 0x7f22d8156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.656+0000 7f22de6b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8162420 con 0x7f22d8156710 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.656+0000 7f22de6b3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8162140 con 0x7f22d81e7420 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.656+0000 7f22de6b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d815df80 con 0x7f22c0004d40 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.656+0000 7f22dcc50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d8156710 0x7f22d8156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.656+0000 7f22dcc50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d8156710 0x7f22d8156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55064/0 (socket says 172.21.15.139:55064) 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.656+0000 7f22dcc50640 1 -- 172.21.15.139:0/1919595439 learned_addr learned my addr 172.21.15.139:0/1919595439 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22d77fe640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22d81e7420 0x7f22d8144710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22dcc50640 1 -- 172.21.15.139:0/1919595439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22c0004d40 msgr2=0x7f22d80ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22d7fff640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22c0004d40 0x7f22d80ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22dcc50640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22c0004d40 0x7f22d80ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22dcc50640 1 -- 172.21.15.139:0/1919595439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22d81e7420 msgr2=0x7f22d8144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:43.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22dcc50640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22d81e7420 0x7f22d8144710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:43.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22dcc50640 1 -- 172.21.15.139:0/1919595439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d81e7c50 con 0x7f22d8156710 2024-04-25T09:55:43.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22d77fe640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22d81e7420 0x7f22d8144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:43.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22dcc50640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d8156710 0x7f22d8156b90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f22d000a0b0 tx=0x7f22d002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:43.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22d4ff9640 1 -- 172.21.15.139:0/1919595439 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000a670 con 0x7f22d8156710 2024-04-25T09:55:43.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22d4ff9640 1 -- 172.21.15.139:0/1919595439 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d0001490 con 0x7f22d8156710 2024-04-25T09:55:43.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22de6b3640 1 -- 172.21.15.139:0/1919595439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d81e7f30 con 0x7f22d8156710 2024-04-25T09:55:43.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22d4ff9640 1 -- 172.21.15.139:0/1919595439 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0034670 con 0x7f22d8156710 2024-04-25T09:55:43.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.657+0000 7f22de6b3640 1 -- 172.21.15.139:0/1919595439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d81e84f0 con 0x7f22d8156710 2024-04-25T09:55:43.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.658+0000 7f22de6b3640 1 -- 172.21.15.139:0/1919595439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d8074520 con 0x7f22d8156710 2024-04-25T09:55:43.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.658+0000 7f22d4ff9640 1 -- 172.21.15.139:0/1919595439 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22d0034810 con 0x7f22d8156710 2024-04-25T09:55:43.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.659+0000 7f22d4ff9640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f22c406b500 0x7f22c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:43.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.659+0000 7f22d4ff9640 1 -- 172.21.15.139:0/1919595439 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f22d00aa260 con 0x7f22d8156710 2024-04-25T09:55:43.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.659+0000 7f22d7fff640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f22c406b500 0x7f22c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:43.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.659+0000 7f22d7fff640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f22c406b500 0x7f22c406d9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f22c8005590 tx=0x7f22c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:43.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.663+0000 7f22d4ff9640 1 -- 172.21.15.139:0/1919595439 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22d80b3e10 con 0x7f22d8156710 2024-04-25T09:55:43.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:43.898+0000 7f22de6b3640 1 -- 172.21.15.139:0/1919595439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f22d80d0aa0 con 0x7f22d8156710 2024-04-25T09:55:44.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.230+0000 7f22d4ff9640 1 -- 172.21.15.139:0/1919595439 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v149) v1 ==== 149+0+0 (secure 0 0 0) 0x7f22d00791c0 con 0x7f22d8156710 2024-04-25T09:55:44.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.233+0000 7f22de6b3640 1 -- 172.21.15.139:0/1919595439 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f22c406b500 msgr2=0x7f22c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.233+0000 7f22de6b3640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f22c406b500 0x7f22c406d9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f22c8005590 tx=0x7f22c8009040 comp rx=0 tx=0).stop 2024-04-25T09:55:44.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.233+0000 7f22de6b3640 1 -- 172.21.15.139:0/1919595439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d8156710 msgr2=0x7f22d8156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.233+0000 7f22de6b3640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d8156710 0x7f22d8156b90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f22d000a0b0 tx=0x7f22d002d070 comp rx=0 tx=0).stop 2024-04-25T09:55:44.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.233+0000 7f22de6b3640 1 -- 172.21.15.139:0/1919595439 shutdown_connections 2024-04-25T09:55:44.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.233+0000 7f22de6b3640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22d8156710 0x7f22d8156b90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.233+0000 7f22de6b3640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f22c406b500 0x7f22c406d9c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.233+0000 7f22de6b3640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22d81e7420 0x7f22d8144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.233+0000 7f22de6b3640 1 --2- 172.21.15.139:0/1919595439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22c0004d40 0x7f22d80ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.233+0000 7f22de6b3640 1 -- 172.21.15.139:0/1919595439 >> 172.21.15.139:0/1919595439 conn(0x7f22d80b23d0 msgr2=0x7f22d806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:44.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.233+0000 7f22de6b3640 1 -- 172.21.15.139:0/1919595439 shutdown_connections 2024-04-25T09:55:44.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.233+0000 7f22de6b3640 1 -- 172.21.15.139:0/1919595439 wait complete. 2024-04-25T09:55:44.247 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-25T09:55:44.248 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T09:55:44.248 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T09:55:44.248 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:44.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.341+0000 7fe59b46b640 1 Processor -- start 2024-04-25T09:55:44.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.342+0000 7fe59b46b640 1 -- start start 2024-04-25T09:55:44.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.342+0000 7fe59b46b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe59415ae40 0x7fe5940b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.342+0000 7fe59b46b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5940ce220 con 0x7fe5940b0210 2024-04-25T09:55:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.342+0000 7fe59b46b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5940ce500 con 0x7fe5940cefe0 2024-04-25T09:55:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.342+0000 7fe59b46b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5940b1070 con 0x7fe59415ae40 2024-04-25T09:55:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.343+0000 7fe599207640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe5940b0210 0x7fe5940b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37758/0 (socket says 172.21.15.139:37758) 2024-04-25T09:55:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.343+0000 7fe598a06640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe59415ae40 0x7fe5940b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.343+0000 7fe599207640 1 -- 172.21.15.139:0/3902730928 learned_addr learned my addr 172.21.15.139:0/3902730928 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.343+0000 7fe598a06640 1 -- 172.21.15.139:0/3902730928 >> v1:172.21.15.150:6789/0 conn(0x7fe5940cefe0 legacy=0x7fe59406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.343+0000 7fe598a06640 1 -- 172.21.15.139:0/3902730928 >> v1:172.21.15.139:6789/0 conn(0x7fe5940b0210 legacy=0x7fe5940b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.343+0000 7fe598a06640 1 -- 172.21.15.139:0/3902730928 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5940ce7e0 con 0x7fe59415ae40 2024-04-25T09:55:44.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.343+0000 7fe598a06640 1 --2- 172.21.15.139:0/3902730928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe59415ae40 0x7fe5940b1960 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe5840097a0 tx=0x7fe58402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848cff3cf7605224 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.344+0000 7fe58bfff640 1 -- 172.21.15.139:0/3902730928 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584037070 con 0x7fe59415ae40 2024-04-25T09:55:44.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.344+0000 7fe58bfff640 1 -- 172.21.15.139:0/3902730928 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe584032440 con 0x7fe59415ae40 2024-04-25T09:55:44.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.344+0000 7fe58bfff640 1 -- 172.21.15.139:0/3902730928 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584032810 con 0x7fe59415ae40 2024-04-25T09:55:44.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.344+0000 7fe59b46b640 1 -- 172.21.15.139:0/3902730928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe59415ae40 msgr2=0x7fe5940b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.344+0000 7fe59b46b640 1 --2- 172.21.15.139:0/3902730928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe59415ae40 0x7fe5940b1960 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe5840097a0 tx=0x7fe58402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.344+0000 7fe59b46b640 1 -- 172.21.15.139:0/3902730928 shutdown_connections 2024-04-25T09:55:44.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.344+0000 7fe59b46b640 1 --2- 172.21.15.139:0/3902730928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe59415ae40 0x7fe5940b1960 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.344+0000 7fe59b46b640 1 -- 172.21.15.139:0/3902730928 >> 172.21.15.139:0/3902730928 conn(0x7fe594066f20 msgr2=0x7fe59406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:44.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.344+0000 7fe59b46b640 1 -- 172.21.15.139:0/3902730928 shutdown_connections 2024-04-25T09:55:44.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.344+0000 7fe59b46b640 1 -- 172.21.15.139:0/3902730928 wait complete. 2024-04-25T09:55:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.345+0000 7fe59b46b640 1 Processor -- start 2024-04-25T09:55:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.346+0000 7fe59b46b640 1 -- start start 2024-04-25T09:55:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.346+0000 7fe59b46b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5940b0210 0x7fe594146880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.346+0000 7fe59b46b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5940cefe0 0x7fe594144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.346+0000 7fe59b46b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe59415ae40 0x7fe59414acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.346+0000 7fe59b46b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5940b1ea0 con 0x7fe5940cefe0 2024-04-25T09:55:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.346+0000 7fe59b46b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59415b3e0 con 0x7fe59415ae40 2024-04-25T09:55:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.346+0000 7fe59b46b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5940ceb50 con 0x7fe5940b0210 2024-04-25T09:55:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.346+0000 7fe598a06640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5940cefe0 0x7fe594144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.346+0000 7fe598a06640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5940cefe0 0x7fe594144340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55078/0 (socket says 172.21.15.139:55078) 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.346+0000 7fe598a06640 1 -- 172.21.15.139:0/4204517856 learned_addr learned my addr 172.21.15.139:0/4204517856 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe599207640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5940b0210 0x7fe594146880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe599a08640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe59415ae40 0x7fe59414acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe598a06640 1 -- 172.21.15.139:0/4204517856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5940b0210 msgr2=0x7fe594146880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe598a06640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5940b0210 0x7fe594146880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe598a06640 1 -- 172.21.15.139:0/4204517856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe59415ae40 msgr2=0x7fe59414acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe598a06640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe59415ae40 0x7fe59414acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe598a06640 1 -- 172.21.15.139:0/4204517856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe594140180 con 0x7fe5940cefe0 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe599207640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5940b0210 0x7fe594146880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe598a06640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5940cefe0 0x7fe594144340 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe584000c00 tx=0x7fe5840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe589ffb640 1 -- 172.21.15.139:0/4204517856 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584037040 con 0x7fe5940cefe0 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe589ffb640 1 -- 172.21.15.139:0/4204517856 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe584042070 con 0x7fe5940cefe0 2024-04-25T09:55:44.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe59b46b640 1 -- 172.21.15.139:0/4204517856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe594144d30 con 0x7fe5940cefe0 2024-04-25T09:55:44.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe589ffb640 1 -- 172.21.15.139:0/4204517856 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584030a70 con 0x7fe5940cefe0 2024-04-25T09:55:44.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.347+0000 7fe59b46b640 1 -- 172.21.15.139:0/4204517856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5941452f0 con 0x7fe5940cefe0 2024-04-25T09:55:44.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.348+0000 7fe59b46b640 1 -- 172.21.15.139:0/4204517856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe594074520 con 0x7fe5940cefe0 2024-04-25T09:55:44.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.349+0000 7fe589ffb640 1 -- 172.21.15.139:0/4204517856 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe584030c10 con 0x7fe5940cefe0 2024-04-25T09:55:44.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.349+0000 7fe589ffb640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe57806b4d0 0x7fe57806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.349+0000 7fe589ffb640 1 -- 172.21.15.139:0/4204517856 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe5840aaae0 con 0x7fe5940cefe0 2024-04-25T09:55:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.349+0000 7fe599207640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe57806b4d0 0x7fe57806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.349+0000 7fe599207640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe57806b4d0 0x7fe57806d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe58c000f60 tx=0x7fe58c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.353+0000 7fe589ffb640 1 -- 172.21.15.139:0/4204517856 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe584045360 con 0x7fe5940cefe0 2024-04-25T09:55:44.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.523+0000 7fe59b46b640 1 -- 172.21.15.139:0/4204517856 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5940b2040 con 0x7fe5940cefe0 2024-04-25T09:55:44.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.523+0000 7fe589ffb640 1 -- 172.21.15.139:0/4204517856 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+13341 (secure 0 0 0) 0x7fe584079b60 con 0x7fe5940cefe0 2024-04-25T09:55:44.527 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:44.527 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":53,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:55:43.229111+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6837/338275708":"2024-04-26T09:55:40.224691+0000","172.21.15.139:6836/338275708":"2024-04-26T09:55:40.224691+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:44.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.525+0000 7fe59b46b640 1 -- 172.21.15.139:0/4204517856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe57806b4d0 msgr2=0x7fe57806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.525+0000 7fe59b46b640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe57806b4d0 0x7fe57806d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe58c000f60 tx=0x7fe58c009040 comp rx=0 tx=0).stop 2024-04-25T09:55:44.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.525+0000 7fe59b46b640 1 -- 172.21.15.139:0/4204517856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5940cefe0 msgr2=0x7fe594144340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.525+0000 7fe59b46b640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5940cefe0 0x7fe594144340 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe584000c00 tx=0x7fe5840046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.525+0000 7fe59b46b640 1 -- 172.21.15.139:0/4204517856 shutdown_connections 2024-04-25T09:55:44.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.525+0000 7fe59b46b640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5940cefe0 0x7fe594144340 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.525+0000 7fe59b46b640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe57806b4d0 0x7fe57806d990 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.525+0000 7fe59b46b640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe59415ae40 0x7fe59414acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.525+0000 7fe59b46b640 1 --2- 172.21.15.139:0/4204517856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5940b0210 0x7fe594146880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.525+0000 7fe59b46b640 1 -- 172.21.15.139:0/4204517856 >> 172.21.15.139:0/4204517856 conn(0x7fe594066f20 msgr2=0x7fe59415b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:44.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.526+0000 7fe59b46b640 1 -- 172.21.15.139:0/4204517856 shutdown_connections 2024-04-25T09:55:44.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.526+0000 7fe59b46b640 1 -- 172.21.15.139:0/4204517856 wait complete. 2024-04-25T09:55:44.540 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:55:44.540 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:44.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.637+0000 7f7fad349640 1 Processor -- start 2024-04-25T09:55:44.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.637+0000 7f7fad349640 1 -- start start 2024-04-25T09:55:44.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.637+0000 7f7fad349640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7fa815dbf0 0x7f7fa81617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.637+0000 7f7fad349640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7fa80cee50 0x7f7fa80d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.637+0000 7f7fad349640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa8161dc0 con 0x7f7fa80b9370 2024-04-25T09:55:44.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.637+0000 7f7fad349640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa80d1ee0 con 0x7f7fa80cee50 2024-04-25T09:55:44.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.637+0000 7f7fad349640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa81620a0 con 0x7f7fa815dbf0 2024-04-25T09:55:44.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.638+0000 7f7fa77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7fa80cee50 0x7f7fa80d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.638+0000 7f7fa77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7fa80cee50 0x7f7fa80d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55086/0 (socket says 172.21.15.139:55086) 2024-04-25T09:55:44.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.638+0000 7f7fa77fe640 1 -- 172.21.15.139:0/3255308760 learned_addr learned my addr 172.21.15.139:0/3255308760 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:44.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.638+0000 7f7fa77fe640 1 -- 172.21.15.139:0/3255308760 >> v2:172.21.15.150:3300/0 conn(0x7f7fa815dbf0 msgr2=0x7f7fa81617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:44.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.638+0000 7f7fa6ffd640 1 --2- 172.21.15.139:0/3255308760 >> v2:172.21.15.150:3300/0 conn(0x7f7fa815dbf0 0x7f7fa81617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.638+0000 7f7fa77fe640 1 --2- 172.21.15.139:0/3255308760 >> v2:172.21.15.150:3300/0 conn(0x7f7fa815dbf0 0x7f7fa81617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.638+0000 7f7fa77fe640 1 -- 172.21.15.139:0/3255308760 >> v1:172.21.15.150:6789/0 conn(0x7f7fa80b9370 legacy=0x7f7fa80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:44.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.638+0000 7f7fa77fe640 1 -- 172.21.15.139:0/3255308760 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8162380 con 0x7f7fa80cee50 2024-04-25T09:55:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.639+0000 7f7fa77fe640 1 --2- 172.21.15.139:0/3255308760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa80cee50 0x7f7fa80d2370 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7fa806d330 tx=0x7f7fa002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b174f3b227752c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.639+0000 7f7fa5ffb640 1 -- 172.21.15.139:0/3255308760 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0037070 con 0x7f7fa80cee50 2024-04-25T09:55:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.639+0000 7f7fa5ffb640 1 -- 172.21.15.139:0/3255308760 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa0032440 con 0x7f7fa80cee50 2024-04-25T09:55:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.639+0000 7f7fa5ffb640 1 -- 172.21.15.139:0/3255308760 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0032810 con 0x7f7fa80cee50 2024-04-25T09:55:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.639+0000 7f7fad349640 1 -- 172.21.15.139:0/3255308760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa80cee50 msgr2=0x7f7fa80d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.639+0000 7f7fad349640 1 --2- 172.21.15.139:0/3255308760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa80cee50 0x7f7fa80d2370 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7fa806d330 tx=0x7f7fa002a230 comp rx=0 tx=0).stop 2024-04-25T09:55:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.639+0000 7f7fad349640 1 -- 172.21.15.139:0/3255308760 shutdown_connections 2024-04-25T09:55:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.639+0000 7f7fad349640 1 --2- 172.21.15.139:0/3255308760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa80cee50 0x7f7fa80d2370 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.639+0000 7f7fad349640 1 --2- 172.21.15.139:0/3255308760 >> v2:172.21.15.150:3300/0 conn(0x7f7fa815dbf0 0x7f7fa81617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.639+0000 7f7fad349640 1 -- 172.21.15.139:0/3255308760 >> 172.21.15.139:0/3255308760 conn(0x7f7fa80b5480 msgr2=0x7f7fa80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.639+0000 7f7fad349640 1 -- 172.21.15.139:0/3255308760 shutdown_connections 2024-04-25T09:55:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.639+0000 7f7fad349640 1 -- 172.21.15.139:0/3255308760 wait complete. 2024-04-25T09:55:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.641+0000 7f7fad349640 1 Processor -- start 2024-04-25T09:55:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.641+0000 7f7fad349640 1 -- start start 2024-04-25T09:55:44.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.641+0000 7f7fad349640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa80b9370 0x7f7fa80c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.641+0000 7f7fad349640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fa80cee50 0x7f7fa81559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.641+0000 7f7fad349640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa815dbf0 0x7f7fa8155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.641+0000 7f7fad349640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa815dfd0 con 0x7f7fa815dbf0 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.641+0000 7f7fad349640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80b97d0 con 0x7f7fa80b9370 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.641+0000 7f7fad349640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa8155170 con 0x7f7fa80cee50 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.641+0000 7f7fa77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa815dbf0 0x7f7fa8155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.641+0000 7f7fa77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa815dbf0 0x7f7fa8155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55094/0 (socket says 172.21.15.139:55094) 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.641+0000 7f7fa77fe640 1 -- 172.21.15.139:0/2480079125 learned_addr learned my addr 172.21.15.139:0/2480079125 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.642+0000 7f7fa6ffd640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa80b9370 0x7f7fa80c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.642+0000 7f7fa77fe640 1 -- 172.21.15.139:0/2480079125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fa80cee50 msgr2=0x7f7fa81559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.642+0000 7f7fa77fe640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fa80cee50 0x7f7fa81559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.642+0000 7f7fa77fe640 1 -- 172.21.15.139:0/2480079125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa80b9370 msgr2=0x7f7fa80c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.642+0000 7f7fa77fe640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa80b9370 0x7f7fa80c94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.642+0000 7f7fa77fe640 1 -- 172.21.15.139:0/2480079125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8154560 con 0x7f7fa815dbf0 2024-04-25T09:55:44.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.642+0000 7f7fa77fe640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa815dbf0 0x7f7fa8155f30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00090b0 tx=0x7f7fa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.642+0000 7f7f8f7fe640 1 -- 172.21.15.139:0/2480079125 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0041020 con 0x7f7fa815dbf0 2024-04-25T09:55:44.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.642+0000 7f7f8f7fe640 1 -- 172.21.15.139:0/2480079125 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa0030990 con 0x7f7fa815dbf0 2024-04-25T09:55:44.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.642+0000 7f7fad349640 1 -- 172.21.15.139:0/2480079125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa8154840 con 0x7f7fa815dbf0 2024-04-25T09:55:44.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.642+0000 7f7f8f7fe640 1 -- 172.21.15.139:0/2480079125 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0037040 con 0x7f7fa815dbf0 2024-04-25T09:55:44.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.642+0000 7f7fad349640 1 -- 172.21.15.139:0/2480079125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa8154e00 con 0x7f7fa815dbf0 2024-04-25T09:55:44.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.643+0000 7f7fad349640 1 -- 172.21.15.139:0/2480079125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa8074520 con 0x7f7fa815dbf0 2024-04-25T09:55:44.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.643+0000 7f7f8f7fe640 1 -- 172.21.15.139:0/2480079125 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fa0044020 con 0x7f7fa815dbf0 2024-04-25T09:55:44.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.644+0000 7f7f8f7fe640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f9406b4d0 0x7f7f9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.644+0000 7f7f8f7fe640 1 -- 172.21.15.139:0/2480079125 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7fa00aaa50 con 0x7f7fa815dbf0 2024-04-25T09:55:44.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.644+0000 7f7fa6ffd640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f9406b4d0 0x7f7f9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.644+0000 7f7fa6ffd640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f9406b4d0 0x7f7f9406d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7f98008ef0 tx=0x7f7f98008440 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.648+0000 7f7f8f7fe640 1 -- 172.21.15.139:0/2480079125 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7fa004b030 con 0x7f7fa815dbf0 2024-04-25T09:55:44.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.817+0000 7f7fad349640 1 -- 172.21.15.139:0/2480079125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fa80d1680 con 0x7f7fa815dbf0 2024-04-25T09:55:44.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.817+0000 7f7f8f7fe640 1 -- 172.21.15.139:0/2480079125 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fa0079a40 con 0x7f7fa815dbf0 2024-04-25T09:55:44.821 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:55:44.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.819+0000 7f7fad349640 1 -- 172.21.15.139:0/2480079125 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f9406b4d0 msgr2=0x7f7f9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.819+0000 7f7fad349640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f9406b4d0 0x7f7f9406d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7f98008ef0 tx=0x7f7f98008440 comp rx=0 tx=0).stop 2024-04-25T09:55:44.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.819+0000 7f7fad349640 1 -- 172.21.15.139:0/2480079125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa815dbf0 msgr2=0x7f7fa8155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.819+0000 7f7fad349640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa815dbf0 0x7f7fa8155f30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00090b0 tx=0x7f7fa00046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.819+0000 7f7fad349640 1 -- 172.21.15.139:0/2480079125 shutdown_connections 2024-04-25T09:55:44.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.819+0000 7f7fad349640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa815dbf0 0x7f7fa8155f30 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.819+0000 7f7fad349640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f9406b4d0 0x7f7f9406d990 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.819+0000 7f7fad349640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fa80cee50 0x7f7fa81559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.819+0000 7f7fad349640 1 --2- 172.21.15.139:0/2480079125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa80b9370 0x7f7fa80c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.819+0000 7f7fad349640 1 -- 172.21.15.139:0/2480079125 >> 172.21.15.139:0/2480079125 conn(0x7f7fa80b5480 msgr2=0x7f7fa80d3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:44.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.819+0000 7f7fad349640 1 -- 172.21.15.139:0/2480079125 shutdown_connections 2024-04-25T09:55:44.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.819+0000 7f7fad349640 1 -- 172.21.15.139:0/2480079125 wait complete. 2024-04-25T09:55:44.834 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T09:55:44.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.927+0000 7ff0fd607640 1 Processor -- start 2024-04-25T09:55:44.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.927+0000 7ff0fd607640 1 -- start start 2024-04-25T09:55:44.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.928+0000 7ff0fd607640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff0f815dc60 0x7ff0f80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.928+0000 7ff0fd607640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f80cf250 con 0x7ff0f80d1080 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.928+0000 7ff0fd607640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f80ce9b0 con 0x7ff0f8161850 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.928+0000 7ff0fd607640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f80d3730 con 0x7ff0f815dc60 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.928+0000 7ff0f77fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff0f80d1080 0x7ff0f80d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37768/0 (socket says 172.21.15.139:37768) 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.928+0000 7ff0f77fe640 1 -- 172.21.15.139:0/2564085579 learned_addr learned my addr 172.21.15.139:0/2564085579 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.928+0000 7ff0f6ffd640 1 --2- 172.21.15.139:0/2564085579 >> v2:172.21.15.150:3300/0 conn(0x7ff0f815dc60 0x7ff0f80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311162327 0 0) 0x7ff0f0001110 con 0x7ff0f80d1080 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0e0003520 con 0x7ff0f80d1080 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515099618 0 0) 0x7ff0f80ce9b0 con 0x7ff0f8161850 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0f0001110 con 0x7ff0f8161850 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 335174927 0 0) 0x7ff0e0003520 con 0x7ff0f80d1080 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 >> v2:172.21.15.150:3300/0 conn(0x7ff0f815dc60 msgr2=0x7ff0f80b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:44.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 --2- 172.21.15.139:0/2564085579 >> v2:172.21.15.150:3300/0 conn(0x7ff0f815dc60 0x7ff0f80b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 >> v1:172.21.15.150:6789/0 conn(0x7ff0f8161850 legacy=0x7ff0f80d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f80cec90 con 0x7ff0f80d1080 2024-04-25T09:55:44.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7ff0f0001840 con 0x7ff0f80d1080 2024-04-25T09:55:44.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 >> v1:172.21.15.139:6789/0 conn(0x7ff0f80d1080 legacy=0x7ff0f80d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 --2- 172.21.15.139:0/2564085579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0e0003b40 0x7ff0e0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 --2- 172.21.15.139:0/2564085579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0e0004800 0x7ff0e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 --2- 172.21.15.139:0/2564085579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0e00051c0 0x7ff0e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f80d3730 con 0x7ff0e0003b40 2024-04-25T09:55:44.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e000c040 con 0x7ff0e0004800 2024-04-25T09:55:44.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e000c240 con 0x7ff0e00051c0 2024-04-25T09:55:44.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f6ffd640 1 --2- 172.21.15.139:0/2564085579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0e0003b40 0x7ff0e0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f67fc640 1 --2- 172.21.15.139:0/2564085579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0e0004800 0x7ff0e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f77fe640 1 --2- 172.21.15.139:0/2564085579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0e00051c0 0x7ff0e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f6ffd640 1 -- 172.21.15.139:0/2564085579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0e00051c0 msgr2=0x7ff0e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f6ffd640 1 --2- 172.21.15.139:0/2564085579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0e00051c0 0x7ff0e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f6ffd640 1 -- 172.21.15.139:0/2564085579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0e0004800 msgr2=0x7ff0e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f6ffd640 1 --2- 172.21.15.139:0/2564085579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0e0004800 0x7ff0e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f6ffd640 1 -- 172.21.15.139:0/2564085579 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e0003520 con 0x7ff0e0003b40 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.929+0000 7ff0f77fe640 1 -- 172.21.15.139:0/2564085579 reap_dead start 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.930+0000 7ff0f6ffd640 1 --2- 172.21.15.139:0/2564085579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0e0003b40 0x7ff0e0003f40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009fc0 tx=0x7ff0e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6dcb8688e2e3497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.930+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8009c10 con 0x7ff0e0003b40 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.930+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e8036070 con 0x7ff0e0003b40 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.930+0000 7ff0f5ffb640 1 -- 172.21.15.139:0/2564085579 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8031880 con 0x7ff0e0003b40 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.931+0000 7ff0fd607640 1 -- 172.21.15.139:0/2564085579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0e0003b40 msgr2=0x7ff0e0003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.931+0000 7ff0fd607640 1 --2- 172.21.15.139:0/2564085579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0e0003b40 0x7ff0e0003f40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009fc0 tx=0x7ff0e0005830 comp rx=0 tx=0).stop 2024-04-25T09:55:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.931+0000 7ff0fd607640 1 -- 172.21.15.139:0/2564085579 shutdown_connections 2024-04-25T09:55:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.931+0000 7ff0fd607640 1 --2- 172.21.15.139:0/2564085579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0e0003b40 0x7ff0e0003f40 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.931+0000 7ff0fd607640 1 -- 172.21.15.139:0/2564085579 >> 172.21.15.139:0/2564085579 conn(0x7ff0f80b5910 msgr2=0x7ff0f80b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.931+0000 7ff0fd607640 1 -- 172.21.15.139:0/2564085579 shutdown_connections 2024-04-25T09:55:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.931+0000 7ff0fd607640 1 -- 172.21.15.139:0/2564085579 wait complete. 2024-04-25T09:55:44.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.932+0000 7ff0fd607640 1 Processor -- start 2024-04-25T09:55:44.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.932+0000 7ff0fd607640 1 -- start start 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0fd607640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0e0003b40 0x7ff0f815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0fd607640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f80c9de0 0x7ff0f8156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0fd607640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0f80d3970 0x7ff0f80d3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0fd607640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f815e040 con 0x7ff0f80c9de0 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0fd607640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f80d1fc0 con 0x7ff0f80d3970 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0fd607640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f80c8260 con 0x7ff0e0003b40 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0f67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f80c9de0 0x7ff0f8156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0f67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f80c9de0 0x7ff0f8156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55112/0 (socket says 172.21.15.139:55112) 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0f67fc640 1 -- 172.21.15.139:0/3269117343 learned_addr learned my addr 172.21.15.139:0/3269117343 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0f6ffd640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0e0003b40 0x7ff0f815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0f77fe640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0f80d3970 0x7ff0f80d3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0f67fc640 1 -- 172.21.15.139:0/3269117343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0e0003b40 msgr2=0x7ff0f815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0f67fc640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0e0003b40 0x7ff0f815aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0f67fc640 1 -- 172.21.15.139:0/3269117343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0f80d3970 msgr2=0x7ff0f80d3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0f67fc640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0f80d3970 0x7ff0f80d3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0f67fc640 1 -- 172.21.15.139:0/3269117343 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f80c7650 con 0x7ff0f80c9de0 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0f77fe640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0f80d3970 0x7ff0f80d3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0f67fc640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f80c9de0 0x7ff0f8156230 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00f7a0 tx=0x7ff0ec00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0df7fe640 1 -- 172.21.15.139:0/3269117343 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec00fe50 con 0x7ff0f80c9de0 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0df7fe640 1 -- 172.21.15.139:0/3269117343 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec018070 con 0x7ff0f80c9de0 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0fd607640 1 -- 172.21.15.139:0/3269117343 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f80c7930 con 0x7ff0f80c9de0 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.933+0000 7ff0df7fe640 1 -- 172.21.15.139:0/3269117343 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec00d730 con 0x7ff0f80c9de0 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.934+0000 7ff0fd607640 1 -- 172.21.15.139:0/3269117343 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f80c7ef0 con 0x7ff0f80c9de0 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.935+0000 7ff0fd607640 1 -- 172.21.15.139:0/3269117343 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f8074520 con 0x7ff0f80c9de0 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.935+0000 7ff0df7fe640 1 -- 172.21.15.139:0/3269117343 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0ec00d8d0 con 0x7ff0f80c9de0 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.935+0000 7ff0df7fe640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff0e406b500 0x7ff0e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.936+0000 7ff0f6ffd640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff0e406b500 0x7ff0e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.936+0000 7ff0df7fe640 1 -- 172.21.15.139:0/3269117343 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff0ec089ec0 con 0x7ff0f80c9de0 2024-04-25T09:55:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.936+0000 7ff0f6ffd640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff0e406b500 0x7ff0e406d9c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8035000 tx=0x7ff0e802f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:44.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:44.940+0000 7ff0df7fe640 1 -- 172.21.15.139:0/3269117343 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0ec058f40 con 0x7ff0f80c9de0 2024-04-25T09:55:45.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.182+0000 7ff0fd607640 1 -- 172.21.15.139:0/3269117343 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7ff0f8161160 con 0x7ff0f80c9de0 2024-04-25T09:55:45.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.238+0000 7ff0df7fe640 1 -- 172.21.15.139:0/3269117343 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v150) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff0ec058840 con 0x7ff0f80c9de0 2024-04-25T09:55:45.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.240+0000 7ff0fd607640 1 -- 172.21.15.139:0/3269117343 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff0e406b500 msgr2=0x7ff0e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.240+0000 7ff0fd607640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff0e406b500 0x7ff0e406d9c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8035000 tx=0x7ff0e802f000 comp rx=0 tx=0).stop 2024-04-25T09:55:45.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.240+0000 7ff0fd607640 1 -- 172.21.15.139:0/3269117343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f80c9de0 msgr2=0x7ff0f8156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.240+0000 7ff0fd607640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f80c9de0 0x7ff0f8156230 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00f7a0 tx=0x7ff0ec00fc60 comp rx=0 tx=0).stop 2024-04-25T09:55:45.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.240+0000 7ff0fd607640 1 -- 172.21.15.139:0/3269117343 shutdown_connections 2024-04-25T09:55:45.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.240+0000 7ff0fd607640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f80c9de0 0x7ff0f8156230 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.240+0000 7ff0fd607640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff0e406b500 0x7ff0e406d9c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.240+0000 7ff0fd607640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0f80d3970 0x7ff0f80d3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.240+0000 7ff0fd607640 1 --2- 172.21.15.139:0/3269117343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0e0003b40 0x7ff0f815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.240+0000 7ff0fd607640 1 -- 172.21.15.139:0/3269117343 >> 172.21.15.139:0/3269117343 conn(0x7ff0f80b5910 msgr2=0x7ff0f806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:45.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.240+0000 7ff0fd607640 1 -- 172.21.15.139:0/3269117343 shutdown_connections 2024-04-25T09:55:45.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.240+0000 7ff0fd607640 1 -- 172.21.15.139:0/3269117343 wait complete. 2024-04-25T09:55:45.255 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:55:45.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.351+0000 7ff4077d5640 1 Processor -- start 2024-04-25T09:55:45.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.351+0000 7ff4077d5640 1 -- start start 2024-04-25T09:55:45.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff4077d5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff40015dbf0 0x7ff4001617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff4077d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4000b9370 0x7ff4000ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff4077d5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff400161dc0 con 0x7ff4000cec80 2024-04-25T09:55:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff4077d5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4000d1ee0 con 0x7ff40015dbf0 2024-04-25T09:55:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff4077d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4001620a0 con 0x7ff4000b9370 2024-04-25T09:55:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff405571640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff40015dbf0 0x7ff4001617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff405571640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff40015dbf0 0x7ff4001617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55120/0 (socket says 172.21.15.139:55120) 2024-04-25T09:55:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff405d72640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff4000cec80 0x7ff40006c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37784/0 (socket says 172.21.15.139:37784) 2024-04-25T09:55:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff405571640 1 -- 172.21.15.139:0/1406315753 learned_addr learned my addr 172.21.15.139:0/1406315753 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff404d70640 1 --2- 172.21.15.139:0/1406315753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4000b9370 0x7ff4000ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff405571640 1 -- 172.21.15.139:0/1406315753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4000b9370 msgr2=0x7ff4000ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff405571640 1 --2- 172.21.15.139:0/1406315753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4000b9370 0x7ff4000ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff405571640 1 -- 172.21.15.139:0/1406315753 >> v1:172.21.15.139:6789/0 conn(0x7ff4000cec80 legacy=0x7ff40006c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff405571640 1 -- 172.21.15.139:0/1406315753 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff400162380 con 0x7ff40015dbf0 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.352+0000 7ff3f7fff640 1 -- 172.21.15.139:0/1406315753 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744541632 0 0) 0x7ff400161dc0 con 0x7ff4000cec80 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.353+0000 7ff405571640 1 --2- 172.21.15.139:0/1406315753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff40015dbf0 0x7ff4001617e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009870 tx=0x7ff3fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6359842987721f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.353+0000 7ff3f7fff640 1 -- 172.21.15.139:0/1406315753 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc037070 con 0x7ff40015dbf0 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.353+0000 7ff3f7fff640 1 -- 172.21.15.139:0/1406315753 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3fc032440 con 0x7ff40015dbf0 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.353+0000 7ff3f7fff640 1 -- 172.21.15.139:0/1406315753 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc032810 con 0x7ff40015dbf0 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.354+0000 7ff4077d5640 1 -- 172.21.15.139:0/1406315753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff40015dbf0 msgr2=0x7ff4001617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.354+0000 7ff4077d5640 1 --2- 172.21.15.139:0/1406315753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff40015dbf0 0x7ff4001617e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009870 tx=0x7ff3fc02a270 comp rx=0 tx=0).stop 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.354+0000 7ff4077d5640 1 -- 172.21.15.139:0/1406315753 shutdown_connections 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.354+0000 7ff4077d5640 1 --2- 172.21.15.139:0/1406315753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4000b9370 0x7ff4000ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.354+0000 7ff4077d5640 1 --2- 172.21.15.139:0/1406315753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff40015dbf0 0x7ff4001617e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.354+0000 7ff4077d5640 1 -- 172.21.15.139:0/1406315753 >> 172.21.15.139:0/1406315753 conn(0x7ff4000b5480 msgr2=0x7ff4000b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.354+0000 7ff4077d5640 1 -- 172.21.15.139:0/1406315753 shutdown_connections 2024-04-25T09:55:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.354+0000 7ff4077d5640 1 -- 172.21.15.139:0/1406315753 wait complete. 2024-04-25T09:55:45.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.355+0000 7ff4077d5640 1 Processor -- start 2024-04-25T09:55:45.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.355+0000 7ff4077d5640 1 -- start start 2024-04-25T09:55:45.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff4077d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4000b9370 0x7ff40015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff4077d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4000cec80 0x7ff40015a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff4077d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff40015dbf0 0x7ff40015e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff4077d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40015dfd0 con 0x7ff4000b9370 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff4077d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4000b97d0 con 0x7ff4000cec80 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff4077d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff400159f20 con 0x7ff40015dbf0 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff405571640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4000b9370 0x7ff40015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff405571640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4000b9370 0x7ff40015b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55136/0 (socket says 172.21.15.139:55136) 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff405571640 1 -- 172.21.15.139:0/685983452 learned_addr learned my addr 172.21.15.139:0/685983452 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff405d72640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff40015dbf0 0x7ff40015e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff404d70640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4000cec80 0x7ff40015a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff405571640 1 -- 172.21.15.139:0/685983452 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff40015dbf0 msgr2=0x7ff40015e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff405571640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff40015dbf0 0x7ff40015e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff405571640 1 -- 172.21.15.139:0/685983452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4000cec80 msgr2=0x7ff40015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff405571640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4000cec80 0x7ff40015a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff405571640 1 -- 172.21.15.139:0/685983452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff400160db0 con 0x7ff4000b9370 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff405d72640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff40015dbf0 0x7ff40015e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff405571640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4000b9370 0x7ff40015b000 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009870 tx=0x7ff3fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff3f5ffb640 1 -- 172.21.15.139:0/685983452 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc041070 con 0x7ff4000b9370 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff3f5ffb640 1 -- 172.21.15.139:0/685983452 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3fc03c870 con 0x7ff4000b9370 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff4077d5640 1 -- 172.21.15.139:0/685983452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff400161090 con 0x7ff4000b9370 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.356+0000 7ff3f5ffb640 1 -- 172.21.15.139:0/685983452 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc037040 con 0x7ff4000b9370 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.357+0000 7ff4077d5640 1 -- 172.21.15.139:0/685983452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff400159bb0 con 0x7ff4000b9370 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.357+0000 7ff4077d5640 1 -- 172.21.15.139:0/685983452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff400074520 con 0x7ff4000b9370 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.358+0000 7ff3f5ffb640 1 -- 172.21.15.139:0/685983452 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3fc044070 con 0x7ff4000b9370 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.358+0000 7ff3f5ffb640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3e406b500 0x7ff3e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.358+0000 7ff3f5ffb640 1 -- 172.21.15.139:0/685983452 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff3fc0aae70 con 0x7ff4000b9370 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.358+0000 7ff404d70640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3e406b500 0x7ff3e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.359+0000 7ff404d70640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3e406b500 0x7ff3e406d9c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0006fd0 tx=0x7ff3f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.362+0000 7ff3f5ffb640 1 -- 172.21.15.139:0/685983452 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3fc079f80 con 0x7ff4000b9370 2024-04-25T09:55:45.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.531+0000 7ff4077d5640 1 -- 172.21.15.139:0/685983452 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff4000d1680 con 0x7ff4000b9370 2024-04-25T09:55:45.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.535+0000 7ff3f5ffb640 1 -- 172.21.15.139:0/685983452 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff3fc079880 con 0x7ff4000b9370 2024-04-25T09:55:45.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.537+0000 7ff4077d5640 1 -- 172.21.15.139:0/685983452 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3e406b500 msgr2=0x7ff3e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.537+0000 7ff4077d5640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3e406b500 0x7ff3e406d9c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0006fd0 tx=0x7ff3f0008040 comp rx=0 tx=0).stop 2024-04-25T09:55:45.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.537+0000 7ff4077d5640 1 -- 172.21.15.139:0/685983452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4000b9370 msgr2=0x7ff40015b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.537+0000 7ff4077d5640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4000b9370 0x7ff40015b000 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009870 tx=0x7ff3fc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.537+0000 7ff4077d5640 1 -- 172.21.15.139:0/685983452 shutdown_connections 2024-04-25T09:55:45.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.537+0000 7ff4077d5640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3e406b500 0x7ff3e406d9c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.537+0000 7ff4077d5640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff40015dbf0 0x7ff40015e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.537+0000 7ff4077d5640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4000cec80 0x7ff40015a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.537+0000 7ff4077d5640 1 --2- 172.21.15.139:0/685983452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4000b9370 0x7ff40015b000 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.537+0000 7ff4077d5640 1 -- 172.21.15.139:0/685983452 >> 172.21.15.139:0/685983452 conn(0x7ff4000b5480 msgr2=0x7ff4000b7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:45.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.537+0000 7ff4077d5640 1 -- 172.21.15.139:0/685983452 shutdown_connections 2024-04-25T09:55:45.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.537+0000 7ff4077d5640 1 -- 172.21.15.139:0/685983452 wait complete. 2024-04-25T09:55:45.552 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:55:45.552 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:55:45.579 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:55:45.596 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:55:45.634 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:55:45.635 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:55:45.635 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:45.638 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:55:45.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:55:45.653 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:55:45.653 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:45.694 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:55:45.694 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:55:45.694 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:45.709 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:55:45.709 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:55:45.709 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:45.751 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:55:45.751 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:55:45.751 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:45.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.844+0000 7ff2aa5e2640 1 Processor -- start 2024-04-25T09:55:45.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.844+0000 7ff2aa5e2640 1 -- start start 2024-04-25T09:55:45.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.845+0000 7ff2aa5e2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff2a406ab60 0x7ff2a406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.845+0000 7ff2aa5e2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a4070a40 con 0x7ff2a415b950 2024-04-25T09:55:45.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.845+0000 7ff2aa5e2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a406fc80 con 0x7ff2a40cf960 2024-04-25T09:55:45.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.845+0000 7ff2aa5e2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a4073170 con 0x7ff2a406ab60 2024-04-25T09:55:45.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.845+0000 7ff2a37fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff2a406ab60 0x7ff2a406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.845+0000 7ff2a3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff2a415b950 0x7ff2a4072a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37794/0 (socket says 172.21.15.139:37794) 2024-04-25T09:55:45.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.845+0000 7ff2a3fff640 1 -- 172.21.15.139:0/3402148113 learned_addr learned my addr 172.21.15.139:0/3402148113 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:45.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.845+0000 7ff2a37fe640 1 --2- 172.21.15.139:0/3402148113 >> v2:172.21.15.139:3300/0 conn(0x7ff2a406ab60 0x7ff2a406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55146/0 (socket says 172.21.15.139:55146) 2024-04-25T09:55:45.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.846+0000 7ff2a37fe640 1 -- 172.21.15.139:0/3402148113 >> v1:172.21.15.150:6789/0 conn(0x7ff2a40cf960 legacy=0x7ff2a40cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.846+0000 7ff2a37fe640 1 -- 172.21.15.139:0/3402148113 >> v1:172.21.15.139:6789/0 conn(0x7ff2a415b950 legacy=0x7ff2a4072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.846+0000 7ff2a37fe640 1 -- 172.21.15.139:0/3402148113 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a406ff60 con 0x7ff2a406ab60 2024-04-25T09:55:45.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.846+0000 7ff2a2ffd640 1 -- 172.21.15.139:0/3402148113 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332994536 0 0) 0x7ff2a4070a40 con 0x7ff2a415b950 2024-04-25T09:55:45.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.846+0000 7ff2a37fe640 1 --2- 172.21.15.139:0/3402148113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2a406ab60 0x7ff2a406afc0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff2980097a0 tx=0x7ff29802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dab0eacfbf721de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.846+0000 7ff2a2ffd640 1 -- 172.21.15.139:0/3402148113 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298037070 con 0x7ff2a406ab60 2024-04-25T09:55:45.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.846+0000 7ff2a2ffd640 1 -- 172.21.15.139:0/3402148113 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff298032440 con 0x7ff2a406ab60 2024-04-25T09:55:45.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.846+0000 7ff2a2ffd640 1 -- 172.21.15.139:0/3402148113 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298032810 con 0x7ff2a406ab60 2024-04-25T09:55:45.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.847+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3402148113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2a406ab60 msgr2=0x7ff2a406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.847+0000 7ff2aa5e2640 1 --2- 172.21.15.139:0/3402148113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2a406ab60 0x7ff2a406afc0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff2980097a0 tx=0x7ff29802a230 comp rx=0 tx=0).stop 2024-04-25T09:55:45.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.847+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3402148113 shutdown_connections 2024-04-25T09:55:45.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.847+0000 7ff2aa5e2640 1 --2- 172.21.15.139:0/3402148113 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2a406ab60 0x7ff2a406afc0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.847+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3402148113 >> 172.21.15.139:0/3402148113 conn(0x7ff2a40b23f0 msgr2=0x7ff2a40b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:45.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.847+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3402148113 shutdown_connections 2024-04-25T09:55:45.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.847+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3402148113 wait complete. 2024-04-25T09:55:45.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.848+0000 7ff2aa5e2640 1 Processor -- start 2024-04-25T09:55:45.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.848+0000 7ff2aa5e2640 1 -- start start 2024-04-25T09:55:45.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2aa5e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2a406ab60 0x7ff2a40a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2aa5e2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2a40cf960 0x7ff2a415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2aa5e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2a415b950 0x7ff2a414a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2aa5e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40cda00 con 0x7ff2a406ab60 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2aa5e2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40cdce0 con 0x7ff2a40cf960 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2aa5e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40cdfc0 con 0x7ff2a415b950 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2a406ab60 0x7ff2a40a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2a406ab60 0x7ff2a40a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55154/0 (socket says 172.21.15.139:55154) 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2a3fff640 1 -- 172.21.15.139:0/3221472011 learned_addr learned my addr 172.21.15.139:0/3221472011 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2a37fe640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2a40cf960 0x7ff2a415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2a3fff640 1 -- 172.21.15.139:0/3221472011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2a415b950 msgr2=0x7ff2a414a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2a8b7f640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2a415b950 0x7ff2a414a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2a3fff640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2a415b950 0x7ff2a414a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2a3fff640 1 -- 172.21.15.139:0/3221472011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2a40cf960 msgr2=0x7ff2a415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:45.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2a3fff640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2a40cf960 0x7ff2a415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2a3fff640 1 -- 172.21.15.139:0/3221472011 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a415d5d0 con 0x7ff2a406ab60 2024-04-25T09:55:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2a37fe640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2a40cf960 0x7ff2a415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.849+0000 7ff2a8b7f640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2a415b950 0x7ff2a414a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.850+0000 7ff2a3fff640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2a406ab60 0x7ff2a40a95e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff29400c3f0 tx=0x7ff29400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.850+0000 7ff2a0ff9640 1 -- 172.21.15.139:0/3221472011 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2940074e0 con 0x7ff2a406ab60 2024-04-25T09:55:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.850+0000 7ff2a0ff9640 1 -- 172.21.15.139:0/3221472011 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2940047e0 con 0x7ff2a406ab60 2024-04-25T09:55:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.850+0000 7ff2a0ff9640 1 -- 172.21.15.139:0/3221472011 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294004c10 con 0x7ff2a406ab60 2024-04-25T09:55:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.850+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3221472011 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a415d8b0 con 0x7ff2a406ab60 2024-04-25T09:55:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.850+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3221472011 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a415e150 con 0x7ff2a406ab60 2024-04-25T09:55:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.851+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3221472011 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a415da80 con 0x7ff2a406ab60 2024-04-25T09:55:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.851+0000 7ff2a0ff9640 1 -- 172.21.15.139:0/3221472011 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff294011070 con 0x7ff2a406ab60 2024-04-25T09:55:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.852+0000 7ff2a0ff9640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff29006b500 0x7ff29006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.852+0000 7ff2a0ff9640 1 -- 172.21.15.139:0/3221472011 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff294089c10 con 0x7ff2a406ab60 2024-04-25T09:55:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.852+0000 7ff2a37fe640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff29006b500 0x7ff29006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:45.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.852+0000 7ff2a37fe640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff29006b500 0x7ff29006d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff298034000 tx=0x7ff29802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:45.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:45.856+0000 7ff2a0ff9640 1 -- 172.21.15.139:0/3221472011 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff29401a2b0 con 0x7ff2a406ab60 2024-04-25T09:55:46.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.061+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3221472011 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2a4074520 con 0x7ff2a406ab60 2024-04-25T09:55:46.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.062+0000 7ff2a0ff9640 1 -- 172.21.15.139:0/3221472011 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+3711 (secure 0 0 0) 0x7ff294058c90 con 0x7ff2a406ab60 2024-04-25T09:55:46.066 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:46.066 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5153,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1742557158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1742557158},{"type":"v1","addr":"172.21.15.150:6834","nonce":1742557158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3496803834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3496803834},{"type":"v1","addr":"172.21.15.150:6835","nonce":3496803834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2014537692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2014537692},{"type":"v1","addr":"172.21.15.139:6835","nonce":2014537692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":6237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2352173876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2352173876},{"type":"v1","addr":"172.21.15.139:6837","nonce":2352173876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[]} 2024-04-25T09:55:46.066 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 24 2024-04-25T09:55:46.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.064+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3221472011 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff29006b500 msgr2=0x7ff29006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:46.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.064+0000 7ff2aa5e2640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff29006b500 0x7ff29006d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff298034000 tx=0x7ff29802e000 comp rx=0 tx=0).stop 2024-04-25T09:55:46.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.064+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3221472011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2a406ab60 msgr2=0x7ff2a40a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:46.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.064+0000 7ff2aa5e2640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2a406ab60 0x7ff2a40a95e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff29400c3f0 tx=0x7ff29400c8b0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.064+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3221472011 shutdown_connections 2024-04-25T09:55:46.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.064+0000 7ff2aa5e2640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff29006b500 0x7ff29006d9c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.064+0000 7ff2aa5e2640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2a415b950 0x7ff2a414a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.064+0000 7ff2aa5e2640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2a40cf960 0x7ff2a415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.064+0000 7ff2aa5e2640 1 --2- 172.21.15.139:0/3221472011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2a406ab60 0x7ff2a40a95e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.064+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3221472011 >> 172.21.15.139:0/3221472011 conn(0x7ff2a40b23f0 msgr2=0x7ff2a415ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:46.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.064+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3221472011 shutdown_connections 2024-04-25T09:55:46.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.064+0000 7ff2aa5e2640 1 -- 172.21.15.139:0/3221472011 wait complete. 2024-04-25T09:55:46.079 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:55:46.079 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:55:46.079 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:55:46.079 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:55:46.080 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:55:46.080 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:55:46.080 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:55:46.080 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:55:46.080 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:55:46.080 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:55:46.081 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:55:46.081 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:55:46.081 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:55:46.081 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:55:46.081 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:55:46.081 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:55:46.089 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:55:46.089 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:55:46.090 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:55:46.090 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:55:46.093 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:55:46.093 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:55:46.094 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:55:46.094 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:55:46.132 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:55:46.132 DEBUG:teuthology.parallel:result is None 2024-04-25T09:55:46.132 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:55:46.133 DEBUG:teuthology.parallel:result is None 2024-04-25T09:55:46.136 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:55:46.136 DEBUG:teuthology.parallel:result is None 2024-04-25T09:55:46.136 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:55:46.137 DEBUG:teuthology.parallel:result is None 2024-04-25T09:55:46.137 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:55:46.205 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-04-25T09:55:46.206 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T09:55:46.217 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-04-25T09:55:46.217 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-04-25T09:55:46.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.237+0000 7f1b984dd640 1 Processor -- start 2024-04-25T09:55:46.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.237+0000 7f1b984dd640 1 -- start start 2024-04-25T09:55:46.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.238+0000 7f1b984dd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1b90070970 0x7f1b900b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:46.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.238+0000 7f1b984dd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b900d0d40 con 0x7f1b900b4190 2024-04-25T09:55:46.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.238+0000 7f1b984dd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b900b4b40 con 0x7f1b9015a870 2024-04-25T09:55:46.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.238+0000 7f1b984dd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b900d1020 con 0x7f1b90070970 2024-04-25T09:55:46.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.238+0000 7f1b95a78640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1b90070970 0x7f1b900b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:46.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.238+0000 7f1b96a7a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1b900b4190 0x7f1b900ce610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37800/0 (socket says 172.21.15.139:37800) 2024-04-25T09:55:46.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.238+0000 7f1b96a7a640 1 -- 172.21.15.139:0/1172425976 learned_addr learned my addr 172.21.15.139:0/1172425976 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:46.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.238+0000 7f1b95277640 1 -- 172.21.15.139:0/1172425976 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294122431 0 0) 0x7f1b900d0d40 con 0x7f1b900b4190 2024-04-25T09:55:46.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.238+0000 7f1b95a78640 1 -- 172.21.15.139:0/1172425976 >> v1:172.21.15.150:6789/0 conn(0x7f1b9015a870 legacy=0x7f1b90070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:46.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.238+0000 7f1b95a78640 1 -- 172.21.15.139:0/1172425976 >> v1:172.21.15.139:6789/0 conn(0x7f1b900b4190 legacy=0x7f1b900ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:46.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.238+0000 7f1b95a78640 1 -- 172.21.15.139:0/1172425976 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900b4e20 con 0x7f1b90070970 2024-04-25T09:55:46.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.239+0000 7f1b95a78640 1 --2- 172.21.15.139:0/1172425976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b90070970 0x7f1b900b5620 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0097a0 tx=0x7f1b8c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f21cb610b34c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:46.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.243+0000 7f1b95277640 1 -- 172.21.15.139:0/1172425976 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c037070 con 0x7f1b90070970 2024-04-25T09:55:46.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.243+0000 7f1b95277640 1 -- 172.21.15.139:0/1172425976 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c032440 con 0x7f1b90070970 2024-04-25T09:55:46.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.243+0000 7f1b95277640 1 -- 172.21.15.139:0/1172425976 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c032810 con 0x7f1b90070970 2024-04-25T09:55:46.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.243+0000 7f1b984dd640 1 -- 172.21.15.139:0/1172425976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b90070970 msgr2=0x7f1b900b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:46.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.243+0000 7f1b984dd640 1 --2- 172.21.15.139:0/1172425976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b90070970 0x7f1b900b5620 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0097a0 tx=0x7f1b8c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.244+0000 7f1b984dd640 1 -- 172.21.15.139:0/1172425976 shutdown_connections 2024-04-25T09:55:46.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.244+0000 7f1b984dd640 1 --2- 172.21.15.139:0/1172425976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b90070970 0x7f1b900b5620 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.244+0000 7f1b984dd640 1 -- 172.21.15.139:0/1172425976 >> 172.21.15.139:0/1172425976 conn(0x7f1b900b23f0 msgr2=0x7f1b900b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:46.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.244+0000 7f1b984dd640 1 -- 172.21.15.139:0/1172425976 shutdown_connections 2024-04-25T09:55:46.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.244+0000 7f1b984dd640 1 -- 172.21.15.139:0/1172425976 wait complete. 2024-04-25T09:55:46.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.245+0000 7f1b984dd640 1 Processor -- start 2024-04-25T09:55:46.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.245+0000 7f1b984dd640 1 -- start start 2024-04-25T09:55:46.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.246+0000 7f1b984dd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b90070970 0x7f1b9014d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:46.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.246+0000 7f1b984dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b900b4190 0x7f1b90150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:46.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.246+0000 7f1b984dd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b9015a870 0x7f1b901507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:46.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.246+0000 7f1b984dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900d05f0 con 0x7f1b900b4190 2024-04-25T09:55:46.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.246+0000 7f1b984dd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900b5190 con 0x7f1b90070970 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.246+0000 7f1b984dd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900d0770 con 0x7f1b9015a870 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.246+0000 7f1b95a78640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b900b4190 0x7f1b90150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.246+0000 7f1b95a78640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b900b4190 0x7f1b90150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55188/0 (socket says 172.21.15.139:55188) 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.246+0000 7f1b95a78640 1 -- 172.21.15.139:0/3521765870 learned_addr learned my addr 172.21.15.139:0/3521765870 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.246+0000 7f1b96279640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b90070970 0x7f1b9014d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.246+0000 7f1b96a7a640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b9015a870 0x7f1b901507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.247+0000 7f1b95a78640 1 -- 172.21.15.139:0/3521765870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b9015a870 msgr2=0x7f1b901507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.247+0000 7f1b95a78640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b9015a870 0x7f1b901507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.247+0000 7f1b95a78640 1 -- 172.21.15.139:0/3521765870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b90070970 msgr2=0x7f1b9014d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.247+0000 7f1b95a78640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b90070970 0x7f1b9014d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.247+0000 7f1b95a78640 1 -- 172.21.15.139:0/3521765870 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900af010 con 0x7f1b900b4190 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.247+0000 7f1b95a78640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b900b4190 0x7f1b90150280 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0097a0 tx=0x7f1b8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.247+0000 7f1b86ffd640 1 -- 172.21.15.139:0/3521765870 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c037050 con 0x7f1b900b4190 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.247+0000 7f1b86ffd640 1 -- 172.21.15.139:0/3521765870 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c038040 con 0x7f1b900b4190 2024-04-25T09:55:46.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.247+0000 7f1b86ffd640 1 -- 172.21.15.139:0/3521765870 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c004490 con 0x7f1b900b4190 2024-04-25T09:55:46.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.247+0000 7f1b984dd640 1 -- 172.21.15.139:0/3521765870 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b900af2d0 con 0x7f1b900b4190 2024-04-25T09:55:46.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.247+0000 7f1b984dd640 1 -- 172.21.15.139:0/3521765870 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9014d820 con 0x7f1b900b4190 2024-04-25T09:55:46.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.248+0000 7f1b984dd640 1 -- 172.21.15.139:0/3521765870 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b90074520 con 0x7f1b900b4190 2024-04-25T09:55:46.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.249+0000 7f1b86ffd640 1 -- 172.21.15.139:0/3521765870 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b8c030900 con 0x7f1b900b4190 2024-04-25T09:55:46.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.249+0000 7f1b86ffd640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b7406b4d0 0x7f1b7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:46.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.250+0000 7f1b86ffd640 1 -- 172.21.15.139:0/3521765870 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1b8c0ab450 con 0x7f1b900b4190 2024-04-25T09:55:46.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.250+0000 7f1b96279640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b7406b4d0 0x7f1b7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:46.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.250+0000 7f1b96279640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b7406b4d0 0x7f1b7406d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000f60 tx=0x7f1b80009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:46.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.253+0000 7f1b86ffd640 1 -- 172.21.15.139:0/3521765870 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b8c079ff0 con 0x7f1b900b4190 2024-04-25T09:55:46.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:46.436+0000 7f1b984dd640 1 -- 172.21.15.139:0/3521765870 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1b900cf640 con 0x7f1b900b4190 2024-04-25T09:55:47.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.263+0000 7f1b86ffd640 1 -- 172.21.15.139:0/3521765870 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1b8c07e090 con 0x7f1b900b4190 2024-04-25T09:55:47.267 INFO:teuthology.orchestra.run.smithi139.stderr: removed all blocklist entries 2024-04-25T09:55:47.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.265+0000 7f1b984dd640 1 -- 172.21.15.139:0/3521765870 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b7406b4d0 msgr2=0x7f1b7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.265+0000 7f1b984dd640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b7406b4d0 0x7f1b7406d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000f60 tx=0x7f1b80009040 comp rx=0 tx=0).stop 2024-04-25T09:55:47.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.265+0000 7f1b984dd640 1 -- 172.21.15.139:0/3521765870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b900b4190 msgr2=0x7f1b90150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.265+0000 7f1b984dd640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b900b4190 0x7f1b90150280 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0097a0 tx=0x7f1b8c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.265+0000 7f1b984dd640 1 -- 172.21.15.139:0/3521765870 shutdown_connections 2024-04-25T09:55:47.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.265+0000 7f1b984dd640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b900b4190 0x7f1b90150280 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.265+0000 7f1b984dd640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b7406b4d0 0x7f1b7406d990 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.265+0000 7f1b984dd640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b9015a870 0x7f1b901507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.265+0000 7f1b984dd640 1 --2- 172.21.15.139:0/3521765870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b90070970 0x7f1b9014d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.265+0000 7f1b984dd640 1 -- 172.21.15.139:0/3521765870 >> 172.21.15.139:0/3521765870 conn(0x7f1b900b23f0 msgr2=0x7f1b900ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:47.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.265+0000 7f1b984dd640 1 -- 172.21.15.139:0/3521765870 shutdown_connections 2024-04-25T09:55:47.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.265+0000 7f1b984dd640 1 -- 172.21.15.139:0/3521765870 wait complete. 2024-04-25T09:55:47.282 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:55:47.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.378+0000 7f0b9f6ee640 1 Processor -- start 2024-04-25T09:55:47.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.379+0000 7f0b9f6ee640 1 -- start start 2024-04-25T09:55:47.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.379+0000 7f0b9f6ee640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0b980b5110 0x7f0b980b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.379+0000 7f0b9f6ee640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0b98072280 0x7f0b98072730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.379+0000 7f0b9f6ee640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9815e500 con 0x7f0b9815aa00 2024-04-25T09:55:47.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.379+0000 7f0b9f6ee640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b980b5c20 con 0x7f0b98072280 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.379+0000 7f0b9f6ee640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b980b45d0 con 0x7f0b980b5110 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.380+0000 7f0b9dc8b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0b98072280 0x7f0b98072730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.380+0000 7f0b9dc8b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0b98072280 0x7f0b98072730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58126/0 (socket says 172.21.15.139:58126) 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.380+0000 7f0b9dc8b640 1 -- 172.21.15.139:0/4071500735 learned_addr learned my addr 172.21.15.139:0/4071500735 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.380+0000 7f0b9cc89640 1 --2- 172.21.15.139:0/4071500735 >> v2:172.21.15.150:3300/0 conn(0x7f0b980b5110 0x7f0b980b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.380+0000 7f0b9dc8b640 1 -- 172.21.15.139:0/4071500735 >> v2:172.21.15.150:3300/0 conn(0x7f0b980b5110 msgr2=0x7f0b980b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.380+0000 7f0b9dc8b640 1 --2- 172.21.15.139:0/4071500735 >> v2:172.21.15.150:3300/0 conn(0x7f0b980b5110 0x7f0b980b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.380+0000 7f0b9dc8b640 1 -- 172.21.15.139:0/4071500735 >> v1:172.21.15.150:6789/0 conn(0x7f0b9815aa00 legacy=0x7f0b9815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.380+0000 7f0b9dc8b640 1 -- 172.21.15.139:0/4071500735 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b980b5660 con 0x7f0b98072280 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.381+0000 7f0b9dc8b640 1 --2- 172.21.15.139:0/4071500735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b98072280 0x7f0b98072730 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0b88009870 tx=0x7f0b8802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3121cabffe859132 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.381+0000 7f0b97fff640 1 -- 172.21.15.139:0/4071500735 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88037070 con 0x7f0b98072280 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.381+0000 7f0b97fff640 1 -- 172.21.15.139:0/4071500735 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b88032440 con 0x7f0b98072280 2024-04-25T09:55:47.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.381+0000 7f0b97fff640 1 -- 172.21.15.139:0/4071500735 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88032810 con 0x7f0b98072280 2024-04-25T09:55:47.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.381+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/4071500735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b98072280 msgr2=0x7f0b98072730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.381+0000 7f0b9f6ee640 1 --2- 172.21.15.139:0/4071500735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b98072280 0x7f0b98072730 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0b88009870 tx=0x7f0b8802a210 comp rx=0 tx=0).stop 2024-04-25T09:55:47.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.381+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/4071500735 shutdown_connections 2024-04-25T09:55:47.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.381+0000 7f0b9f6ee640 1 --2- 172.21.15.139:0/4071500735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b98072280 0x7f0b98072730 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.381+0000 7f0b9f6ee640 1 --2- 172.21.15.139:0/4071500735 >> v2:172.21.15.150:3300/0 conn(0x7f0b980b5110 0x7f0b980b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.381+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/4071500735 >> 172.21.15.139:0/4071500735 conn(0x7f0b9806e280 msgr2=0x7f0b9806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:47.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.381+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/4071500735 shutdown_connections 2024-04-25T09:55:47.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.381+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/4071500735 wait complete. 2024-04-25T09:55:47.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.383+0000 7f0b9f6ee640 1 Processor -- start 2024-04-25T09:55:47.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.383+0000 7f0b9f6ee640 1 -- start start 2024-04-25T09:55:47.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.383+0000 7f0b9f6ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b98072280 0x7f0b98152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.383+0000 7f0b9f6ee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b980b5110 0x7f0b98157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.383+0000 7f0b9f6ee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b98156900 0x7f0b980c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.383+0000 7f0b9f6ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980ce610 con 0x7f0b98072280 2024-04-25T09:55:47.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.383+0000 7f0b9f6ee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9815e7a0 con 0x7f0b980b5110 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.383+0000 7f0b9f6ee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980ce790 con 0x7f0b98156900 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.383+0000 7f0b9d48a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b98072280 0x7f0b98152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.383+0000 7f0b9d48a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b98072280 0x7f0b98152cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58136/0 (socket says 172.21.15.139:58136) 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.383+0000 7f0b9d48a640 1 -- 172.21.15.139:0/1379000239 learned_addr learned my addr 172.21.15.139:0/1379000239 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.384+0000 7f0b9d48a640 1 -- 172.21.15.139:0/1379000239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b98156900 msgr2=0x7f0b980c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.384+0000 7f0b9d48a640 1 --2- 172.21.15.139:0/1379000239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b98156900 0x7f0b980c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.384+0000 7f0b9d48a640 1 -- 172.21.15.139:0/1379000239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b980b5110 msgr2=0x7f0b98157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.384+0000 7f0b9d48a640 1 --2- 172.21.15.139:0/1379000239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b980b5110 0x7f0b98157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.384+0000 7f0b9d48a640 1 -- 172.21.15.139:0/1379000239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b98156e00 con 0x7f0b98072280 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.384+0000 7f0b9d48a640 1 --2- 172.21.15.139:0/1379000239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b98072280 0x7f0b98152cd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000c730 tx=0x7f0b9000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.384+0000 7f0b95ffb640 1 -- 172.21.15.139:0/1379000239 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90016020 con 0x7f0b98072280 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.384+0000 7f0b95ffb640 1 -- 172.21.15.139:0/1379000239 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b90006d50 con 0x7f0b98072280 2024-04-25T09:55:47.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.384+0000 7f0b95ffb640 1 -- 172.21.15.139:0/1379000239 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90003bb0 con 0x7f0b98072280 2024-04-25T09:55:47.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.384+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/1379000239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b981570e0 con 0x7f0b98072280 2024-04-25T09:55:47.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.384+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/1379000239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b981576a0 con 0x7f0b98072280 2024-04-25T09:55:47.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.385+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/1379000239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b98074520 con 0x7f0b98072280 2024-04-25T09:55:47.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.386+0000 7f0b95ffb640 1 -- 172.21.15.139:0/1379000239 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b9000a850 con 0x7f0b98072280 2024-04-25T09:55:47.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.386+0000 7f0b95ffb640 1 --2- 172.21.15.139:0/1379000239 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0b8406b4d0 0x7f0b8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.386+0000 7f0b95ffb640 1 -- 172.21.15.139:0/1379000239 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0b90089c60 con 0x7f0b98072280 2024-04-25T09:55:47.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.386+0000 7f0b9cc89640 1 --2- 172.21.15.139:0/1379000239 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0b8406b4d0 0x7f0b8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.386+0000 7f0b9cc89640 1 --2- 172.21.15.139:0/1379000239 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0b8406b4d0 0x7f0b8406d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c006a90 tx=0x7f0b8c006a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.390+0000 7f0b95ffb640 1 -- 172.21.15.139:0/1379000239 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b90020020 con 0x7f0b98072280 2024-04-25T09:55:47.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.625+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/1379000239 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0b9806f180 con 0x7f0b98072280 2024-04-25T09:55:47.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.625+0000 7f0b95ffb640 1 -- 172.21.15.139:0/1379000239 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0b90019310 con 0x7f0b98072280 2024-04-25T09:55:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:47.630 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:55:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_dump": [ 2024-04-25T09:55:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.a", 2024-04-25T09:55:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypm0xdaHhAA6nupxCTQhucepW2iEWNUoQ==", 2024-04-25T09:55:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:55:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:55:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:55:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.b", 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmB9GDKRAA/lmU4OiKO5dFgayYx09GQg==", 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.c", 2024-04-25T09:55:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmvYgaJRAA/YIE6ralp2WuTK993D7H0Q==", 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.d", 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmg49WMBAAlPY9ZSeOoV296xSdl/m1gg==", 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:55:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.0", 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDVJypmG7TpDhAAo9CKuZBw8BsQgB2F+BR5gw==", 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.1", 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDYJypmgR62DBAAotfnvS47WDunP9Ic03gA6Q==", 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.2", 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDbJypmzyqGChAA6wrfIdqpiLm3RO36oorihQ==", 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.3", 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDeJypmt72cCBAAgcLrFC3MUU7wlFySAfLWig==", 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.4", 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDkJypmxJLpDBAAvROZ9AvYdWMqOALEKIAFCw==", 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.5", 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDnJypmTPXEChAA20BghY39PGw6zFyu9yWbBQ==", 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:55:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.6", 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDqJypmaBoYCRAAy9eHgUI/WTkE2XdT1RKLsw==", 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.7", 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDtJypmViJlCBAANJSDWVHVbWR03Y1SMDH++Q==", 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.0", 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg==", 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow r", 2024-04-25T09:55:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.1", 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypmOncYARAA6gYjZbRDkhqFX0mpSDbH5Q==", 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.2", 2024-04-25T09:55:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmQs2wORAArzyV70nGDX4486LlFjM70Q==", 2024-04-25T09:55:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:55:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:55:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:55:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:55:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.3", 2024-04-25T09:55:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypm90x9BhAA41rZPs0O5mwmuHy1bkPCvw==", 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.admin", 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmOWQcAxAA/ZVXEHYJQZf9UtgZ5g25Ag==", 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:55:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.x", 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmt+RVGxAAi1djVoqnoYCDMzJ9A0dmNA==", 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:55:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.y", 2024-04-25T09:55:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmuJ4bFxAAw6a53AgThU/ut+Mk5j0gqQ==", 2024-04-25T09:55:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:55:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:55:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T09:55:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:55:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-04-25T09:55:47.643 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:55:47.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.627+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/1379000239 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0b8406b4d0 msgr2=0x7f0b8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.628+0000 7f0b9f6ee640 1 --2- 172.21.15.139:0/1379000239 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0b8406b4d0 0x7f0b8406d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c006a90 tx=0x7f0b8c006a20 comp rx=0 tx=0).stop 2024-04-25T09:55:47.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.628+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/1379000239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b98072280 msgr2=0x7f0b98152cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.628+0000 7f0b9f6ee640 1 --2- 172.21.15.139:0/1379000239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b98072280 0x7f0b98152cd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000c730 tx=0x7f0b9000cbf0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.628+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/1379000239 shutdown_connections 2024-04-25T09:55:47.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.628+0000 7f0b9f6ee640 1 --2- 172.21.15.139:0/1379000239 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0b8406b4d0 0x7f0b8406d990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.628+0000 7f0b9f6ee640 1 --2- 172.21.15.139:0/1379000239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b98156900 0x7f0b980c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.628+0000 7f0b9f6ee640 1 --2- 172.21.15.139:0/1379000239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b980b5110 0x7f0b98157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.628+0000 7f0b9f6ee640 1 --2- 172.21.15.139:0/1379000239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b98072280 0x7f0b98152cd0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.628+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/1379000239 >> 172.21.15.139:0/1379000239 conn(0x7f0b9806e280 msgr2=0x7f0b980b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:47.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.628+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/1379000239 shutdown_connections 2024-04-25T09:55:47.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.628+0000 7f0b9f6ee640 1 -- 172.21.15.139:0/1379000239 wait complete. 2024-04-25T09:55:47.646 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:47.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.739+0000 7f85fc5d5640 1 Processor -- start 2024-04-25T09:55:47.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.739+0000 7f85fc5d5640 1 -- start start 2024-04-25T09:55:47.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.740+0000 7f85fc5d5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f85f415dc60 0x7f85f40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.740+0000 7f85fc5d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f40d1080 0x7f85f40ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.740+0000 7f85fc5d5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f40cf250 con 0x7f85f4161850 2024-04-25T09:55:47.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.740+0000 7f85fc5d5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f40ce780 con 0x7f85f415dc60 2024-04-25T09:55:47.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.740+0000 7f85fc5d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f40d3730 con 0x7f85f40d1080 2024-04-25T09:55:47.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f9b70640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f85f4161850 0x7f85f40d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56250/0 (socket says 172.21.15.139:56250) 2024-04-25T09:55:47.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f9b70640 1 -- 172.21.15.139:0/714953475 learned_addr learned my addr 172.21.15.139:0/714953475 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:47.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85fab72640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f40d1080 0x7f85f40ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85fa371640 1 --2- 172.21.15.139:0/714953475 >> v2:172.21.15.150:3300/0 conn(0x7f85f415dc60 0x7f85f40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281414257 0 0) 0x7f85f0001110 con 0x7f85f4161850 2024-04-25T09:55:47.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85e0003520 con 0x7f85f4161850 2024-04-25T09:55:47.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1587066399 0 0) 0x7f85e0003520 con 0x7f85f4161850 2024-04-25T09:55:47.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f40d1080 msgr2=0x7f85f40ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:47.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f40d1080 0x7f85f40ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 >> v2:172.21.15.150:3300/0 conn(0x7f85f415dc60 msgr2=0x7f85f40b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:47.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 --2- 172.21.15.139:0/714953475 >> v2:172.21.15.150:3300/0 conn(0x7f85f415dc60 0x7f85f40b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f40ce9b0 con 0x7f85f4161850 2024-04-25T09:55:47.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f85f0001840 con 0x7f85f4161850 2024-04-25T09:55:47.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 >> v1:172.21.15.139:6789/0 conn(0x7f85f4161850 legacy=0x7f85f40d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f40d1080 0x7f85e00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85e0003f30 0x7f85e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85e0004d40 0x7f85e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f40d3730 con 0x7f85e0004d40 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e000b9c0 con 0x7f85e0003f30 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e000bb40 con 0x7f85f40d1080 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85fa371640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f40d1080 0x7f85e00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85fab72640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85e0004d40 0x7f85e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.741+0000 7f85f9b70640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85e0003f30 0x7f85e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.742+0000 7f85fab72640 1 -- 172.21.15.139:0/714953475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f40d1080 msgr2=0x7f85e00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.742+0000 7f85fab72640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f40d1080 0x7f85e00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.742+0000 7f85fab72640 1 -- 172.21.15.139:0/714953475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85e0003f30 msgr2=0x7f85e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.742+0000 7f85fab72640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85e0003f30 0x7f85e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.742+0000 7f85fab72640 1 -- 172.21.15.139:0/714953475 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0003520 con 0x7f85e0004d40 2024-04-25T09:55:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.742+0000 7f85fa371640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f40d1080 0x7f85e00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.742+0000 7f85fab72640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85e0004d40 0x7f85e00051c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009fc0 tx=0x7f85e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b61861c31de798 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.742+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec009c10 con 0x7f85e0004d40 2024-04-25T09:55:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.742+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec036070 con 0x7f85e0004d40 2024-04-25T09:55:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.742+0000 7f85f936f640 1 -- 172.21.15.139:0/714953475 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec031670 con 0x7f85e0004d40 2024-04-25T09:55:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.743+0000 7f85fc5d5640 1 -- 172.21.15.139:0/714953475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85e0004d40 msgr2=0x7f85e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.743+0000 7f85fc5d5640 1 --2- 172.21.15.139:0/714953475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85e0004d40 0x7f85e00051c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009fc0 tx=0x7f85e0005830 comp rx=0 tx=0).stop 2024-04-25T09:55:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.743+0000 7f85fab72640 1 -- 172.21.15.139:0/714953475 reap_dead start 2024-04-25T09:55:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.743+0000 7f85fc5d5640 1 -- 172.21.15.139:0/714953475 shutdown_connections 2024-04-25T09:55:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.743+0000 7f85fc5d5640 1 -- 172.21.15.139:0/714953475 >> 172.21.15.139:0/714953475 conn(0x7f85f40b5910 msgr2=0x7f85f40b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.743+0000 7f85fc5d5640 1 -- 172.21.15.139:0/714953475 shutdown_connections 2024-04-25T09:55:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.743+0000 7f85fc5d5640 1 -- 172.21.15.139:0/714953475 wait complete. 2024-04-25T09:55:47.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.744+0000 7f85fc5d5640 1 Processor -- start 2024-04-25T09:55:47.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.744+0000 7f85fc5d5640 1 -- start start 2024-04-25T09:55:47.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fc5d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85e0004d40 0x7f85f40c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fc5d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f415be00 0x7f85f415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fc5d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f4159f90 0x7f85f415a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fc5d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f40d1fc0 con 0x7f85f4159f90 2024-04-25T09:55:47.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fc5d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f40cef70 con 0x7f85e0004d40 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fc5d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f415e040 con 0x7f85f415be00 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fab72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f4159f90 0x7f85f415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fab72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f4159f90 0x7f85f415a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58156/0 (socket says 172.21.15.139:58156) 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fab72640 1 -- 172.21.15.139:0/2214884000 learned_addr learned my addr 172.21.15.139:0/2214884000 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85f9b70640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f415be00 0x7f85f415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fa371640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85e0004d40 0x7f85f40c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fab72640 1 -- 172.21.15.139:0/2214884000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f415be00 msgr2=0x7f85f415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fab72640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f415be00 0x7f85f415c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fab72640 1 -- 172.21.15.139:0/2214884000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85e0004d40 msgr2=0x7f85f40c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.745+0000 7f85fab72640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85e0004d40 0x7f85f40c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.746+0000 7f85fab72640 1 -- 172.21.15.139:0/2214884000 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f415c870 con 0x7f85f4159f90 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.746+0000 7f85fab72640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f4159f90 0x7f85f415a410 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00e230 tx=0x7f85ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.746+0000 7f85eaffd640 1 -- 172.21.15.139:0/2214884000 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4009690 con 0x7f85f4159f90 2024-04-25T09:55:47.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.746+0000 7f85eaffd640 1 -- 172.21.15.139:0/2214884000 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec031dd0 con 0x7f85f4159f90 2024-04-25T09:55:47.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.746+0000 7f85fc5d5640 1 -- 172.21.15.139:0/2214884000 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f415cb50 con 0x7f85f4159f90 2024-04-25T09:55:47.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.746+0000 7f85eaffd640 1 -- 172.21.15.139:0/2214884000 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec009c10 con 0x7f85f4159f90 2024-04-25T09:55:47.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.746+0000 7f85fc5d5640 1 -- 172.21.15.139:0/2214884000 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f415d110 con 0x7f85f4159f90 2024-04-25T09:55:47.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.747+0000 7f85fc5d5640 1 -- 172.21.15.139:0/2214884000 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f4074e80 con 0x7f85f4159f90 2024-04-25T09:55:47.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.747+0000 7f85eaffd640 1 -- 172.21.15.139:0/2214884000 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85ec005d10 con 0x7f85f4159f90 2024-04-25T09:55:47.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.748+0000 7f85eaffd640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f85d806b500 0x7f85d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:47.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.748+0000 7f85eaffd640 1 -- 172.21.15.139:0/2214884000 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f85ec0ab0e0 con 0x7f85f4159f90 2024-04-25T09:55:47.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.748+0000 7f85fa371640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f85d806b500 0x7f85d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:47.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.748+0000 7f85fa371640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f85d806b500 0x7f85d806d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009c00 tx=0x7f85e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:47.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.752+0000 7f85eaffd640 1 -- 172.21.15.139:0/2214884000 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85f4074e80 con 0x7f85f4159f90 2024-04-25T09:55:47.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.921+0000 7f85fc5d5640 1 -- 172.21.15.139:0/2214884000 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85f4074520 con 0x7f85f4159f90 2024-04-25T09:55:47.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.922+0000 7f85eaffd640 1 -- 172.21.15.139:0/2214884000 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+13212 (secure 0 0 0) 0x7f85ec049060 con 0x7f85f4159f90 2024-04-25T09:55:47.926 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:47.926 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":58,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:55:47.261570+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:47.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.924+0000 7f85fc5d5640 1 -- 172.21.15.139:0/2214884000 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f85d806b500 msgr2=0x7f85d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.924+0000 7f85fc5d5640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f85d806b500 0x7f85d806d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009c00 tx=0x7f85e4008040 comp rx=0 tx=0).stop 2024-04-25T09:55:47.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.924+0000 7f85fc5d5640 1 -- 172.21.15.139:0/2214884000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f4159f90 msgr2=0x7f85f415a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:47.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.924+0000 7f85fc5d5640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f4159f90 0x7f85f415a410 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00e230 tx=0x7f85ec009750 comp rx=0 tx=0).stop 2024-04-25T09:55:47.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.924+0000 7f85fc5d5640 1 -- 172.21.15.139:0/2214884000 shutdown_connections 2024-04-25T09:55:47.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.924+0000 7f85fc5d5640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85f4159f90 0x7f85f415a410 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.924+0000 7f85fc5d5640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f85d806b500 0x7f85d806d9c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.924+0000 7f85fc5d5640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85f415be00 0x7f85f415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.924+0000 7f85fc5d5640 1 --2- 172.21.15.139:0/2214884000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85e0004d40 0x7f85f40c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:47.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.924+0000 7f85fc5d5640 1 -- 172.21.15.139:0/2214884000 >> 172.21.15.139:0/2214884000 conn(0x7f85f40b5910 msgr2=0x7f85f41562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:47.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.924+0000 7f85fc5d5640 1 -- 172.21.15.139:0/2214884000 shutdown_connections 2024-04-25T09:55:47.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:47.924+0000 7f85fc5d5640 1 -- 172.21.15.139:0/2214884000 wait complete. 2024-04-25T09:55:47.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:55:47.939 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:48.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.036+0000 7fcc8bd49640 1 Processor -- start 2024-04-25T09:55:48.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.036+0000 7fcc8bd49640 1 -- start start 2024-04-25T09:55:48.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc8bd49640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc840afc80 0x7fcc840b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc8bd49640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcc8406b700 0x7fcc8406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc8bd49640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc840c6cc0 con 0x7fcc840af8a0 2024-04-25T09:55:48.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc8bd49640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc840c6fa0 con 0x7fcc8406b700 2024-04-25T09:55:48.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc8bd49640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc840c7280 con 0x7fcc840afc80 2024-04-25T09:55:48.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc892e4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc840afc80 0x7fcc840b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc8a2e6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcc8406b700 0x7fcc8406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc89ae5640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fcc840af8a0 0x7fcc840cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:41764/0 (socket says 172.21.15.139:41764) 2024-04-25T09:55:48.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc89ae5640 1 -- 172.21.15.139:0/3621351866 learned_addr learned my addr 172.21.15.139:0/3621351866 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:48.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc892e4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc840afc80 0x7fcc840b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:49520/0 (socket says 172.21.15.139:49520) 2024-04-25T09:55:48.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc8a2e6640 1 -- 172.21.15.139:0/3621351866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc840afc80 msgr2=0x7fcc840b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:48.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc8a2e6640 1 --2- 172.21.15.139:0/3621351866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc840afc80 0x7fcc840b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.037+0000 7fcc8a2e6640 1 -- 172.21.15.139:0/3621351866 >> v1:172.21.15.150:6789/0 conn(0x7fcc840af8a0 legacy=0x7fcc840cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.038+0000 7fcc8a2e6640 1 -- 172.21.15.139:0/3621351866 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc840cd090 con 0x7fcc8406b700 2024-04-25T09:55:48.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.038+0000 7fcc88ae3640 1 -- 172.21.15.139:0/3621351866 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602525265 0 0) 0x7fcc840c6cc0 con 0x7fcc840af8a0 2024-04-25T09:55:48.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.038+0000 7fcc8a2e6640 1 --2- 172.21.15.139:0/3621351866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc8406b700 0x7fcc8406ab60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009d90 tx=0x7fcc7c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2213f55990525ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:48.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.038+0000 7fcc88ae3640 1 -- 172.21.15.139:0/3621351866 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c007740 con 0x7fcc8406b700 2024-04-25T09:55:48.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.038+0000 7fcc88ae3640 1 -- 172.21.15.139:0/3621351866 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c0078e0 con 0x7fcc8406b700 2024-04-25T09:55:48.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.038+0000 7fcc88ae3640 1 -- 172.21.15.139:0/3621351866 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c007cb0 con 0x7fcc8406b700 2024-04-25T09:55:48.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.039+0000 7fcc8bd49640 1 -- 172.21.15.139:0/3621351866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc8406b700 msgr2=0x7fcc8406ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.039+0000 7fcc8bd49640 1 --2- 172.21.15.139:0/3621351866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc8406b700 0x7fcc8406ab60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009d90 tx=0x7fcc7c02a660 comp rx=0 tx=0).stop 2024-04-25T09:55:48.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.039+0000 7fcc8bd49640 1 -- 172.21.15.139:0/3621351866 shutdown_connections 2024-04-25T09:55:48.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.039+0000 7fcc8bd49640 1 --2- 172.21.15.139:0/3621351866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc840afc80 0x7fcc840b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.039+0000 7fcc8bd49640 1 --2- 172.21.15.139:0/3621351866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc8406b700 0x7fcc8406ab60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.039+0000 7fcc8bd49640 1 -- 172.21.15.139:0/3621351866 >> 172.21.15.139:0/3621351866 conn(0x7fcc840b2230 msgr2=0x7fcc840b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:48.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.039+0000 7fcc8bd49640 1 -- 172.21.15.139:0/3621351866 shutdown_connections 2024-04-25T09:55:48.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.039+0000 7fcc8bd49640 1 -- 172.21.15.139:0/3621351866 wait complete. 2024-04-25T09:55:48.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.040+0000 7fcc8bd49640 1 Processor -- start 2024-04-25T09:55:48.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.040+0000 7fcc8bd49640 1 -- start start 2024-04-25T09:55:48.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc8bd49640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8406b700 0x7fcc84159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc8bd49640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc840af8a0 0x7fcc841551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc8bd49640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc840afc80 0x7fcc84155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc8bd49640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8406b220 con 0x7fcc8406b700 2024-04-25T09:55:48.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc8bd49640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8406b0a0 con 0x7fcc840af8a0 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc8bd49640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8406b3a0 con 0x7fcc840afc80 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc89ae5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8406b700 0x7fcc84159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc89ae5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8406b700 0x7fcc84159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58162/0 (socket says 172.21.15.139:58162) 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc89ae5640 1 -- 172.21.15.139:0/2483425169 learned_addr learned my addr 172.21.15.139:0/2483425169 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc892e4640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc840af8a0 0x7fcc841551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc89ae5640 1 -- 172.21.15.139:0/2483425169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc840afc80 msgr2=0x7fcc84155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc8a2e6640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc840afc80 0x7fcc84155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc89ae5640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc840afc80 0x7fcc84155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc89ae5640 1 -- 172.21.15.139:0/2483425169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc840af8a0 msgr2=0x7fcc841551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc89ae5640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc840af8a0 0x7fcc841551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.041+0000 7fcc89ae5640 1 -- 172.21.15.139:0/2483425169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc841545b0 con 0x7fcc8406b700 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.042+0000 7fcc89ae5640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8406b700 0x7fcc84159550 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000c4d0 tx=0x7fcc8000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.042+0000 7fcc7a7fc640 1 -- 172.21.15.139:0/2483425169 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80018070 con 0x7fcc8406b700 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.042+0000 7fcc7a7fc640 1 -- 172.21.15.139:0/2483425169 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc80007500 con 0x7fcc8406b700 2024-04-25T09:55:48.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.042+0000 7fcc8bd49640 1 -- 172.21.15.139:0/2483425169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc84153510 con 0x7fcc8406b700 2024-04-25T09:55:48.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.042+0000 7fcc7a7fc640 1 -- 172.21.15.139:0/2483425169 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80004a20 con 0x7fcc8406b700 2024-04-25T09:55:48.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.042+0000 7fcc8bd49640 1 -- 172.21.15.139:0/2483425169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc84153ad0 con 0x7fcc8406b700 2024-04-25T09:55:48.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.043+0000 7fcc8bd49640 1 -- 172.21.15.139:0/2483425169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8406df40 con 0x7fcc8406b700 2024-04-25T09:55:48.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.043+0000 7fcc7a7fc640 1 -- 172.21.15.139:0/2483425169 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc8001b070 con 0x7fcc8406b700 2024-04-25T09:55:48.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.044+0000 7fcc7a7fc640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcc6806b4d0 0x7fcc6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.044+0000 7fcc7a7fc640 1 -- 172.21.15.139:0/2483425169 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcc800892f0 con 0x7fcc8406b700 2024-04-25T09:55:48.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.044+0000 7fcc892e4640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcc6806b4d0 0x7fcc6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.044+0000 7fcc892e4640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcc6806b4d0 0x7fcc6806d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcc841f6820 tx=0x7fcc74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:48.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.048+0000 7fcc7a7fc640 1 -- 172.21.15.139:0/2483425169 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc8406df40 con 0x7fcc8406b700 2024-04-25T09:55:48.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.216+0000 7fcc8bd49640 1 -- 172.21.15.139:0/2483425169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc8406ac00 con 0x7fcc8406b700 2024-04-25T09:55:48.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.217+0000 7fcc7a7fc640 1 -- 172.21.15.139:0/2483425169 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc800583c0 con 0x7fcc8406b700 2024-04-25T09:55:48.221 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:55:48.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.219+0000 7fcc8bd49640 1 -- 172.21.15.139:0/2483425169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcc6806b4d0 msgr2=0x7fcc6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.219+0000 7fcc8bd49640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcc6806b4d0 0x7fcc6806d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcc841f6820 tx=0x7fcc74008040 comp rx=0 tx=0).stop 2024-04-25T09:55:48.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.219+0000 7fcc8bd49640 1 -- 172.21.15.139:0/2483425169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8406b700 msgr2=0x7fcc84159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.219+0000 7fcc8bd49640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8406b700 0x7fcc84159550 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000c4d0 tx=0x7fcc8000c990 comp rx=0 tx=0).stop 2024-04-25T09:55:48.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.219+0000 7fcc8bd49640 1 -- 172.21.15.139:0/2483425169 shutdown_connections 2024-04-25T09:55:48.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.219+0000 7fcc8bd49640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcc6806b4d0 0x7fcc6806d990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.219+0000 7fcc8bd49640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc840afc80 0x7fcc84155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.219+0000 7fcc8bd49640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc840af8a0 0x7fcc841551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.219+0000 7fcc8bd49640 1 --2- 172.21.15.139:0/2483425169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8406b700 0x7fcc84159550 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.219+0000 7fcc8bd49640 1 -- 172.21.15.139:0/2483425169 >> 172.21.15.139:0/2483425169 conn(0x7fcc840b2230 msgr2=0x7fcc8415d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:48.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.219+0000 7fcc8bd49640 1 -- 172.21.15.139:0/2483425169 shutdown_connections 2024-04-25T09:55:48.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.219+0000 7fcc8bd49640 1 -- 172.21.15.139:0/2483425169 wait complete. 2024-04-25T09:55:48.233 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:55:48.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.326+0000 7fc63099e640 1 Processor -- start 2024-04-25T09:55:48.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.326+0000 7fc63099e640 1 -- start start 2024-04-25T09:55:48.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.327+0000 7fc63099e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc62c15a860 0x7fc62c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.327+0000 7fc63099e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc62c0cd1f0 0x7fc62c06e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.327+0000 7fc63099e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc62c15ac40 0x7fc62c06e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.327+0000 7fc63099e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62c0b02e0 con 0x7fc62c15a860 2024-04-25T09:55:48.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.327+0000 7fc63099e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62c0af6d0 con 0x7fc62c15ac40 2024-04-25T09:55:48.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.327+0000 7fc63099e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0af9b0 con 0x7fc62c0cd1f0 2024-04-25T09:55:48.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.327+0000 7fc62a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc62c15a860 0x7fc62c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.327+0000 7fc62a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc62c15a860 0x7fc62c0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58178/0 (socket says 172.21.15.139:58178) 2024-04-25T09:55:48.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.327+0000 7fc62a59c640 1 -- 172.21.15.139:0/1013770914 learned_addr learned my addr 172.21.15.139:0/1013770914 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:48.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.328+0000 7fc62ad9d640 1 --2- 172.21.15.139:0/1013770914 >> v2:172.21.15.150:3300/0 conn(0x7fc62c15ac40 0x7fc62c06e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.328+0000 7fc629d9b640 1 --2- 172.21.15.139:0/1013770914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc62c0cd1f0 0x7fc62c06e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.328+0000 7fc62a59c640 1 -- 172.21.15.139:0/1013770914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc62c0cd1f0 msgr2=0x7fc62c06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:48.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.328+0000 7fc62a59c640 1 --2- 172.21.15.139:0/1013770914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc62c0cd1f0 0x7fc62c06e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.328+0000 7fc62a59c640 1 -- 172.21.15.139:0/1013770914 >> v2:172.21.15.150:3300/0 conn(0x7fc62c15ac40 msgr2=0x7fc62c06e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:48.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.328+0000 7fc62a59c640 1 --2- 172.21.15.139:0/1013770914 >> v2:172.21.15.150:3300/0 conn(0x7fc62c15ac40 0x7fc62c06e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.328+0000 7fc62a59c640 1 -- 172.21.15.139:0/1013770914 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c0afc90 con 0x7fc62c15a860 2024-04-25T09:55:48.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.328+0000 7fc62a59c640 1 --2- 172.21.15.139:0/1013770914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc62c15a860 0x7fc62c0cccb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0b49e0 tx=0x7fc61c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0f7a301c029132 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:48.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.328+0000 7fc62959a640 1 -- 172.21.15.139:0/1013770914 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c036070 con 0x7fc62c15a860 2024-04-25T09:55:48.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.328+0000 7fc62959a640 1 -- 172.21.15.139:0/1013770914 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c031600 con 0x7fc62c15a860 2024-04-25T09:55:48.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.328+0000 7fc62959a640 1 -- 172.21.15.139:0/1013770914 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0319d0 con 0x7fc62c15a860 2024-04-25T09:55:48.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.329+0000 7fc63099e640 1 -- 172.21.15.139:0/1013770914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc62c15a860 msgr2=0x7fc62c0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.329+0000 7fc63099e640 1 --2- 172.21.15.139:0/1013770914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc62c15a860 0x7fc62c0cccb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0b49e0 tx=0x7fc61c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.329+0000 7fc63099e640 1 -- 172.21.15.139:0/1013770914 shutdown_connections 2024-04-25T09:55:48.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.329+0000 7fc63099e640 1 --2- 172.21.15.139:0/1013770914 >> v2:172.21.15.150:3300/0 conn(0x7fc62c15ac40 0x7fc62c06e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.329+0000 7fc63099e640 1 --2- 172.21.15.139:0/1013770914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc62c0cd1f0 0x7fc62c06e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.329+0000 7fc63099e640 1 --2- 172.21.15.139:0/1013770914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc62c15a860 0x7fc62c0cccb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.329+0000 7fc63099e640 1 -- 172.21.15.139:0/1013770914 >> 172.21.15.139:0/1013770914 conn(0x7fc62c066f20 msgr2=0x7fc62c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:48.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.329+0000 7fc63099e640 1 -- 172.21.15.139:0/1013770914 shutdown_connections 2024-04-25T09:55:48.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.329+0000 7fc63099e640 1 -- 172.21.15.139:0/1013770914 wait complete. 2024-04-25T09:55:48.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.330+0000 7fc63099e640 1 Processor -- start 2024-04-25T09:55:48.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.330+0000 7fc63099e640 1 -- start start 2024-04-25T09:55:48.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.331+0000 7fc63099e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc62c0cd1f0 0x7fc62c13e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.331+0000 7fc63099e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc62c15a860 0x7fc62c14f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.331+0000 7fc63099e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc62c15ac40 0x7fc62c0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.331+0000 7fc63099e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0af420 con 0x7fc62c15a860 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.331+0000 7fc63099e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0af2a0 con 0x7fc62c15ac40 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.331+0000 7fc63099e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c15e240 con 0x7fc62c0cd1f0 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.331+0000 7fc629d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc62c15a860 0x7fc62c14f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.331+0000 7fc629d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc62c15a860 0x7fc62c14f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58182/0 (socket says 172.21.15.139:58182) 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.331+0000 7fc629d9b640 1 -- 172.21.15.139:0/1274988277 learned_addr learned my addr 172.21.15.139:0/1274988277 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.331+0000 7fc62a59c640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc62c0cd1f0 0x7fc62c13e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.332+0000 7fc62ad9d640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc62c15ac40 0x7fc62c0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.332+0000 7fc629d9b640 1 -- 172.21.15.139:0/1274988277 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc62c0cd1f0 msgr2=0x7fc62c13e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.332+0000 7fc629d9b640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc62c0cd1f0 0x7fc62c13e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.332+0000 7fc629d9b640 1 -- 172.21.15.139:0/1274988277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc62c15ac40 msgr2=0x7fc62c0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.332+0000 7fc629d9b640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc62c15ac40 0x7fc62c0b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.332+0000 7fc629d9b640 1 -- 172.21.15.139:0/1274988277 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c150270 con 0x7fc62c15a860 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.332+0000 7fc629d9b640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc62c15a860 0x7fc62c14f8b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc62000c7f0 tx=0x7fc62000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:48.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.332+0000 7fc61affd640 1 -- 172.21.15.139:0/1274988277 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620015070 con 0x7fc62c15a860 2024-04-25T09:55:48.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.332+0000 7fc61affd640 1 -- 172.21.15.139:0/1274988277 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc620004220 con 0x7fc62c15a860 2024-04-25T09:55:48.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.332+0000 7fc63099e640 1 -- 172.21.15.139:0/1274988277 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62c150550 con 0x7fc62c15a860 2024-04-25T09:55:48.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.332+0000 7fc61affd640 1 -- 172.21.15.139:0/1274988277 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620007620 con 0x7fc62c15a860 2024-04-25T09:55:48.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.332+0000 7fc63099e640 1 -- 172.21.15.139:0/1274988277 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c150b10 con 0x7fc62c15a860 2024-04-25T09:55:48.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.333+0000 7fc63099e640 1 -- 172.21.15.139:0/1274988277 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c074520 con 0x7fc62c15a860 2024-04-25T09:55:48.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.333+0000 7fc61affd640 1 -- 172.21.15.139:0/1274988277 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6200079d0 con 0x7fc62c15a860 2024-04-25T09:55:48.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.334+0000 7fc61affd640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc60c06b4d0 0x7fc60c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.334+0000 7fc62a59c640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc60c06b4d0 0x7fc60c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.334+0000 7fc61affd640 1 -- 172.21.15.139:0/1274988277 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc62008a180 con 0x7fc62c15a860 2024-04-25T09:55:48.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.334+0000 7fc62a59c640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc60c06b4d0 0x7fc60c06d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc61c033000 tx=0x7fc61c009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:48.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.338+0000 7fc61affd640 1 -- 172.21.15.139:0/1274988277 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc620059130 con 0x7fc62c15a860 2024-04-25T09:55:48.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.508+0000 7fc63099e640 1 -- 172.21.15.139:0/1274988277 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc62c06e120 con 0x7fc62c15a860 2024-04-25T09:55:48.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.508+0000 7fc61affd640 1 -- 172.21.15.139:0/1274988277 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc620058a30 con 0x7fc62c15a860 2024-04-25T09:55:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:48.512 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-04-25T09:55:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:55:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: "poolnum": 1, 2024-04-25T09:55:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: "poolname": ".mgr" 2024-04-25T09:55:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:55:48.513 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-04-25T09:55:48.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.510+0000 7fc63099e640 1 -- 172.21.15.139:0/1274988277 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc60c06b4d0 msgr2=0x7fc60c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.510+0000 7fc63099e640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc60c06b4d0 0x7fc60c06d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc61c033000 tx=0x7fc61c009000 comp rx=0 tx=0).stop 2024-04-25T09:55:48.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.510+0000 7fc63099e640 1 -- 172.21.15.139:0/1274988277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc62c15a860 msgr2=0x7fc62c14f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.510+0000 7fc63099e640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc62c15a860 0x7fc62c14f8b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc62000c7f0 tx=0x7fc62000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.510+0000 7fc63099e640 1 -- 172.21.15.139:0/1274988277 shutdown_connections 2024-04-25T09:55:48.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.510+0000 7fc63099e640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc62c15a860 0x7fc62c14f8b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.510+0000 7fc63099e640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc60c06b4d0 0x7fc60c06d990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.510+0000 7fc63099e640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc62c15ac40 0x7fc62c0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.510+0000 7fc63099e640 1 --2- 172.21.15.139:0/1274988277 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc62c0cd1f0 0x7fc62c13e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.510+0000 7fc63099e640 1 -- 172.21.15.139:0/1274988277 >> 172.21.15.139:0/1274988277 conn(0x7fc62c066f20 msgr2=0x7fc62c15c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:48.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.510+0000 7fc63099e640 1 -- 172.21.15.139:0/1274988277 shutdown_connections 2024-04-25T09:55:48.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.510+0000 7fc63099e640 1 -- 172.21.15.139:0/1274988277 wait complete. 2024-04-25T09:55:48.525 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:55:48.525 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:55:48.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.622+0000 7fcc30ba5640 1 Processor -- start 2024-04-25T09:55:48.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.622+0000 7fcc30ba5640 1 -- start start 2024-04-25T09:55:48.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc30ba5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcc2c15dbf0 0x7fcc2c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc30ba5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c0b9370 0x7fcc2c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc30ba5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2c161dc0 con 0x7fcc2c0cec80 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc30ba5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2c0d1ee0 con 0x7fcc2c15dbf0 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc30ba5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1620a0 con 0x7fcc2c0b9370 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2ad9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fcc2c0cec80 0x7fcc2c06c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56262/0 (socket says 172.21.15.139:56262) 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2ad9d640 1 -- 172.21.15.139:0/3562513647 learned_addr learned my addr 172.21.15.139:0/3562513647 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1896197768 0 0) 0x7fcc2c161dc0 con 0x7fcc2c0cec80 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc29d9b640 1 --2- 172.21.15.139:0/3562513647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c0b9370 0x7fcc2c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc14003520 con 0x7fcc2c0cec80 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2a59c640 1 --2- 172.21.15.139:0/3562513647 >> v2:172.21.15.150:3300/0 conn(0x7fcc2c15dbf0 0x7fcc2c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 507580065 0 0) 0x7fcc14003520 con 0x7fcc2c0cec80 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c0b9370 msgr2=0x7fcc2c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 --2- 172.21.15.139:0/3562513647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c0b9370 0x7fcc2c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 >> v2:172.21.15.150:3300/0 conn(0x7fcc2c15dbf0 msgr2=0x7fcc2c1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:48.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 --2- 172.21.15.139:0/3562513647 >> v2:172.21.15.150:3300/0 conn(0x7fcc2c15dbf0 0x7fcc2c1617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c162380 con 0x7fcc2c0cec80 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fcc240019f0 con 0x7fcc2c0cec80 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2a59c640 1 --2- 172.21.15.139:0/3562513647 >> v2:172.21.15.150:3300/0 conn(0x7fcc2c15dbf0 0x7fcc2c1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 >> v1:172.21.15.139:6789/0 conn(0x7fcc2c0cec80 legacy=0x7fcc2c06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 --2- 172.21.15.139:0/3562513647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc140039b0 0x7fcc14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 --2- 172.21.15.139:0/3562513647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c0b9370 0x7fcc14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 --2- 172.21.15.139:0/3562513647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc14004d40 0x7fcc140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0d1ee0 con 0x7fcc140039b0 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1620a0 con 0x7fcc14004d40 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.623+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1400b9c0 con 0x7fcc2c0b9370 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.624+0000 7fcc2a59c640 1 --2- 172.21.15.139:0/3562513647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc140039b0 0x7fcc14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.624+0000 7fcc29d9b640 1 --2- 172.21.15.139:0/3562513647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c0b9370 0x7fcc14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.624+0000 7fcc2ad9d640 1 --2- 172.21.15.139:0/3562513647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc14004d40 0x7fcc140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.624+0000 7fcc2a59c640 1 -- 172.21.15.139:0/3562513647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c0b9370 msgr2=0x7fcc14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:48.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.624+0000 7fcc2a59c640 1 --2- 172.21.15.139:0/3562513647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c0b9370 0x7fcc14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.624+0000 7fcc2a59c640 1 -- 172.21.15.139:0/3562513647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc14004d40 msgr2=0x7fcc140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.624+0000 7fcc2a59c640 1 --2- 172.21.15.139:0/3562513647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc14004d40 0x7fcc140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.624+0000 7fcc2a59c640 1 -- 172.21.15.139:0/3562513647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc14003520 con 0x7fcc140039b0 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.624+0000 7fcc2a59c640 1 --2- 172.21.15.139:0/3562513647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc140039b0 0x7fcc14003dd0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0098d0 tx=0x7fcc14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03724f4a7dd94b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.625+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00f040 con 0x7fcc140039b0 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.625+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c02ad50 con 0x7fcc140039b0 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.625+0000 7fcc2959a640 1 -- 172.21.15.139:0/3562513647 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c004250 con 0x7fcc140039b0 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.625+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3562513647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc140039b0 msgr2=0x7fcc14003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.625+0000 7fcc30ba5640 1 --2- 172.21.15.139:0/3562513647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc140039b0 0x7fcc14003dd0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0098d0 tx=0x7fcc14005830 comp rx=0 tx=0).stop 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.625+0000 7fcc2ad9d640 1 -- 172.21.15.139:0/3562513647 reap_dead start 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.625+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3562513647 shutdown_connections 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.625+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3562513647 >> 172.21.15.139:0/3562513647 conn(0x7fcc2c0b5480 msgr2=0x7fcc2c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:48.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.625+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3562513647 shutdown_connections 2024-04-25T09:55:48.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.625+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3562513647 wait complete. 2024-04-25T09:55:48.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.627+0000 7fcc30ba5640 1 Processor -- start 2024-04-25T09:55:48.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.627+0000 7fcc30ba5640 1 -- start start 2024-04-25T09:55:48.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.627+0000 7fcc30ba5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc140039b0 0x7fcc2c1554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.627+0000 7fcc30ba5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c15fb30 0x7fcc2c15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.627+0000 7fcc30ba5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c147e70 0x7fcc2c155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.627+0000 7fcc30ba5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c15dfd0 con 0x7fcc2c147e70 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.627+0000 7fcc30ba5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0b97d0 con 0x7fcc140039b0 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.627+0000 7fcc30ba5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1541c0 con 0x7fcc2c15fb30 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.627+0000 7fcc2ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c147e70 0x7fcc2c155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.627+0000 7fcc2ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c147e70 0x7fcc2c155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58200/0 (socket says 172.21.15.139:58200) 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.627+0000 7fcc2ad9d640 1 -- 172.21.15.139:0/3814791269 learned_addr learned my addr 172.21.15.139:0/3814791269 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc2a59c640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc140039b0 0x7fcc2c1554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc29d9b640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c15fb30 0x7fcc2c15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc2ad9d640 1 -- 172.21.15.139:0/3814791269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c15fb30 msgr2=0x7fcc2c15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc2ad9d640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c15fb30 0x7fcc2c15e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc2ad9d640 1 -- 172.21.15.139:0/3814791269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc140039b0 msgr2=0x7fcc2c1554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc2ad9d640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc140039b0 0x7fcc2c1554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc2ad9d640 1 -- 172.21.15.139:0/3814791269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c154410 con 0x7fcc2c147e70 2024-04-25T09:55:48.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc2ad9d640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c147e70 0x7fcc2c155db0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcc24001330 tx=0x7fcc2400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:48.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc1affd640 1 -- 172.21.15.139:0/3814791269 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400ed40 con 0x7fcc2c147e70 2024-04-25T09:55:48.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc1affd640 1 -- 172.21.15.139:0/3814791269 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc24018070 con 0x7fcc2c147e70 2024-04-25T09:55:48.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3814791269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2c1546f0 con 0x7fcc2c147e70 2024-04-25T09:55:48.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc1affd640 1 -- 172.21.15.139:0/3814791269 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24013660 con 0x7fcc2c147e70 2024-04-25T09:55:48.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.628+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3814791269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c154cb0 con 0x7fcc2c147e70 2024-04-25T09:55:48.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.629+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3814791269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2c074520 con 0x7fcc2c147e70 2024-04-25T09:55:48.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.629+0000 7fcc1affd640 1 -- 172.21.15.139:0/3814791269 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc24013800 con 0x7fcc2c147e70 2024-04-25T09:55:48.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.630+0000 7fcc1affd640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcc0c06b500 0x7fcc0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:48.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.630+0000 7fcc1affd640 1 -- 172.21.15.139:0/3814791269 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcc2408a340 con 0x7fcc2c147e70 2024-04-25T09:55:48.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.630+0000 7fcc2a59c640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcc0c06b500 0x7fcc0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:48.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.630+0000 7fcc2a59c640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcc0c06b500 0x7fcc0c06d9c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c033000 tx=0x7fcc1c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:48.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.634+0000 7fcc1affd640 1 -- 172.21.15.139:0/3814791269 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc240592f0 con 0x7fcc2c147e70 2024-04-25T09:55:48.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:48.803+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3814791269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcc2c0d1680 con 0x7fcc2c147e70 2024-04-25T09:55:49.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.271+0000 7fcc1affd640 1 -- 172.21.15.139:0/3814791269 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcc24058bf0 con 0x7fcc2c147e70 2024-04-25T09:55:49.275 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' created 2024-04-25T09:55:49.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.273+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3814791269 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcc0c06b500 msgr2=0x7fcc0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.273+0000 7fcc30ba5640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcc0c06b500 0x7fcc0c06d9c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c033000 tx=0x7fcc1c02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:49.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.273+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3814791269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c147e70 msgr2=0x7fcc2c155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.273+0000 7fcc30ba5640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c147e70 0x7fcc2c155db0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcc24001330 tx=0x7fcc2400e530 comp rx=0 tx=0).stop 2024-04-25T09:55:49.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.274+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3814791269 shutdown_connections 2024-04-25T09:55:49.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.274+0000 7fcc30ba5640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c147e70 0x7fcc2c155db0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.274+0000 7fcc30ba5640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcc0c06b500 0x7fcc0c06d9c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.274+0000 7fcc30ba5640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc2c15fb30 0x7fcc2c15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.274+0000 7fcc30ba5640 1 --2- 172.21.15.139:0/3814791269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc140039b0 0x7fcc2c1554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.274+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3814791269 >> 172.21.15.139:0/3814791269 conn(0x7fcc2c0b5480 msgr2=0x7fcc2c0d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:49.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.274+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3814791269 shutdown_connections 2024-04-25T09:55:49.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.274+0000 7fcc30ba5640 1 -- 172.21.15.139:0/3814791269 wait complete. 2024-04-25T09:55:49.292 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:55:49.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.393+0000 7f7f2bd70640 1 Processor -- start 2024-04-25T09:55:49.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.393+0000 7f7f2bd70640 1 -- start start 2024-04-25T09:55:49.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.394+0000 7f7f2bd70640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7f240cd610 0x7f7f240d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.394+0000 7f7f2bd70640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7f24161fd0 0x7f7f240d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.394+0000 7f7f2bd70640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f24099ca0 con 0x7f7f2409ad40 2024-04-25T09:55:49.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.394+0000 7f7f2bd70640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f24099a50 con 0x7f7f24161fd0 2024-04-25T09:55:49.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.394+0000 7f7f2bd70640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f24099f80 con 0x7f7f240cd610 2024-04-25T09:55:49.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.394+0000 7f7f2a30d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7f24161fd0 0x7f7f240d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.394+0000 7f7f2a30d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7f24161fd0 0x7f7f240d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58204/0 (socket says 172.21.15.139:58204) 2024-04-25T09:55:49.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.394+0000 7f7f2a30d640 1 -- 172.21.15.139:0/759377670 learned_addr learned my addr 172.21.15.139:0/759377670 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:49.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.395+0000 7f7f2a30d640 1 -- 172.21.15.139:0/759377670 >> v2:172.21.15.150:3300/0 conn(0x7f7f240cd610 msgr2=0x7f7f240d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:49.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.395+0000 7f7f2a30d640 1 --2- 172.21.15.139:0/759377670 >> v2:172.21.15.150:3300/0 conn(0x7f7f240cd610 0x7f7f240d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.395+0000 7f7f2a30d640 1 -- 172.21.15.139:0/759377670 >> v1:172.21.15.139:6789/0 conn(0x7f7f2409ad40 legacy=0x7f7f241618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.395+0000 7f7f2a30d640 1 -- 172.21.15.139:0/759377670 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2409a260 con 0x7f7f24161fd0 2024-04-25T09:55:49.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.395+0000 7f7f28b0a640 1 -- 172.21.15.139:0/759377670 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126796285 0 0) 0x7f7f24099ca0 con 0x7f7f2409ad40 2024-04-25T09:55:49.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.395+0000 7f7f2a30d640 1 --2- 172.21.15.139:0/759377670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f24161fd0 0x7f7f240d08b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7f2406f210 tx=0x7f7f1c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4dee365b6e1212 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:49.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.395+0000 7f7f28b0a640 1 -- 172.21.15.139:0/759377670 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c036070 con 0x7f7f24161fd0 2024-04-25T09:55:49.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.395+0000 7f7f28b0a640 1 -- 172.21.15.139:0/759377670 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c031600 con 0x7f7f24161fd0 2024-04-25T09:55:49.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.395+0000 7f7f28b0a640 1 -- 172.21.15.139:0/759377670 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0319f0 con 0x7f7f24161fd0 2024-04-25T09:55:49.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.396+0000 7f7f2bd70640 1 -- 172.21.15.139:0/759377670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f24161fd0 msgr2=0x7f7f240d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.396+0000 7f7f2bd70640 1 --2- 172.21.15.139:0/759377670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f24161fd0 0x7f7f240d08b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7f2406f210 tx=0x7f7f1c02a7c0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.396+0000 7f7f2bd70640 1 -- 172.21.15.139:0/759377670 shutdown_connections 2024-04-25T09:55:49.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.396+0000 7f7f2bd70640 1 --2- 172.21.15.139:0/759377670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f24161fd0 0x7f7f240d08b0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.396+0000 7f7f2bd70640 1 --2- 172.21.15.139:0/759377670 >> v2:172.21.15.150:3300/0 conn(0x7f7f240cd610 0x7f7f240d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.396+0000 7f7f2bd70640 1 -- 172.21.15.139:0/759377670 >> 172.21.15.139:0/759377670 conn(0x7f7f240b31f0 msgr2=0x7f7f240b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:49.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.396+0000 7f7f2bd70640 1 -- 172.21.15.139:0/759377670 shutdown_connections 2024-04-25T09:55:49.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.396+0000 7f7f2bd70640 1 -- 172.21.15.139:0/759377670 wait complete. 2024-04-25T09:55:49.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.397+0000 7f7f2bd70640 1 Processor -- start 2024-04-25T09:55:49.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.397+0000 7f7f2bd70640 1 -- start start 2024-04-25T09:55:49.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2bd70640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f2409ad40 0x7f7f2414bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2bd70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f240cd610 0x7f7f2415ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2bd70640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f24161fd0 0x7f7f2415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2bd70640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240cd9f0 con 0x7f7f240cd610 2024-04-25T09:55:49.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2bd70640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2409a820 con 0x7f7f24161fd0 2024-04-25T09:55:49.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2bd70640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240cdb70 con 0x7f7f2409ad40 2024-04-25T09:55:49.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2930b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f240cd610 0x7f7f2415ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2930b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f240cd610 0x7f7f2415ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58214/0 (socket says 172.21.15.139:58214) 2024-04-25T09:55:49.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2930b640 1 -- 172.21.15.139:0/397702010 learned_addr learned my addr 172.21.15.139:0/397702010 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:49.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2a30d640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f24161fd0 0x7f7f2415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2930b640 1 -- 172.21.15.139:0/397702010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f2409ad40 msgr2=0x7f7f2414bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f29b0c640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f2409ad40 0x7f7f2414bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2930b640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f2409ad40 0x7f7f2414bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2930b640 1 -- 172.21.15.139:0/397702010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f24161fd0 msgr2=0x7f7f2415f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:49.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2930b640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f24161fd0 0x7f7f2415f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:49.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2930b640 1 -- 172.21.15.139:0/397702010 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2414c080 con 0x7f7f240cd610 2024-04-25T09:55:49.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f2930b640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f240cd610 0x7f7f2415ec60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400c4e0 tx=0x7f7f1400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:49.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f1a7fc640 1 -- 172.21.15.139:0/397702010 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14017070 con 0x7f7f240cd610 2024-04-25T09:55:49.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.398+0000 7f7f1a7fc640 1 -- 172.21.15.139:0/397702010 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f140035f0 con 0x7f7f240cd610 2024-04-25T09:55:49.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.399+0000 7f7f2bd70640 1 -- 172.21.15.139:0/397702010 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2415c520 con 0x7f7f240cd610 2024-04-25T09:55:49.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.399+0000 7f7f1a7fc640 1 -- 172.21.15.139:0/397702010 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f140039e0 con 0x7f7f240cd610 2024-04-25T09:55:49.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.399+0000 7f7f2bd70640 1 -- 172.21.15.139:0/397702010 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2415cae0 con 0x7f7f240cd610 2024-04-25T09:55:49.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.400+0000 7f7f2bd70640 1 -- 172.21.15.139:0/397702010 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f24074520 con 0x7f7f240cd610 2024-04-25T09:55:49.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.400+0000 7f7f1a7fc640 1 -- 172.21.15.139:0/397702010 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f1401a070 con 0x7f7f240cd610 2024-04-25T09:55:49.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.401+0000 7f7f1a7fc640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f0806b4d0 0x7f7f0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:49.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.401+0000 7f7f1a7fc640 1 -- 172.21.15.139:0/397702010 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f7f14089430 con 0x7f7f240cd610 2024-04-25T09:55:49.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.401+0000 7f7f29b0c640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f0806b4d0 0x7f7f0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:49.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.401+0000 7f7f29b0c640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f0806b4d0 0x7f7f0806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000a810 tx=0x7f7f20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:49.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.405+0000 7f7f1a7fc640 1 -- 172.21.15.139:0/397702010 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f2414c080 con 0x7f7f240cd610 2024-04-25T09:55:49.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:49.577+0000 7f7f2bd70640 1 -- 172.21.15.139:0/397702010 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7f240b3f70 con 0x7f7f240cd610 2024-04-25T09:55:50.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.280+0000 7f7f1a7fc640 1 -- 172.21.15.139:0/397702010 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7f14057cb0 con 0x7f7f240cd610 2024-04-25T09:55:50.285 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' created 2024-04-25T09:55:50.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.284+0000 7f7f2bd70640 1 -- 172.21.15.139:0/397702010 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f0806b4d0 msgr2=0x7f7f0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.284+0000 7f7f2bd70640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f0806b4d0 0x7f7f0806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000a810 tx=0x7f7f20008040 comp rx=0 tx=0).stop 2024-04-25T09:55:50.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.284+0000 7f7f2bd70640 1 -- 172.21.15.139:0/397702010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f240cd610 msgr2=0x7f7f2415ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.284+0000 7f7f2bd70640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f240cd610 0x7f7f2415ec60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400c4e0 tx=0x7f7f1400c9a0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.284+0000 7f7f2bd70640 1 -- 172.21.15.139:0/397702010 shutdown_connections 2024-04-25T09:55:50.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.284+0000 7f7f2bd70640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f240cd610 0x7f7f2415ec60 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.284+0000 7f7f2bd70640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7f0806b4d0 0x7f7f0806d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.284+0000 7f7f2bd70640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f24161fd0 0x7f7f2415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.284+0000 7f7f2bd70640 1 --2- 172.21.15.139:0/397702010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f2409ad40 0x7f7f2414bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.284+0000 7f7f2bd70640 1 -- 172.21.15.139:0/397702010 >> 172.21.15.139:0/397702010 conn(0x7f7f240b31f0 msgr2=0x7f7f2406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:50.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.284+0000 7f7f2bd70640 1 -- 172.21.15.139:0/397702010 shutdown_connections 2024-04-25T09:55:50.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.284+0000 7f7f2bd70640 1 -- 172.21.15.139:0/397702010 wait complete. 2024-04-25T09:55:50.307 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:55:50.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.405+0000 7f97c0ebc640 1 Processor -- start 2024-04-25T09:55:50.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.405+0000 7f97c0ebc640 1 -- start start 2024-04-25T09:55:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.405+0000 7f97c0ebc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97bc1617e0 0x7f97bc0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.405+0000 7f97c0ebc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97bc161dc0 con 0x7f97bc15dbf0 2024-04-25T09:55:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.405+0000 7f97c0ebc640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97bc1620a0 con 0x7f97bc0cec80 2024-04-25T09:55:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.405+0000 7f97c0ebc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc0d1ee0 con 0x7f97bc1617e0 2024-04-25T09:55:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97ba59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f97bc15dbf0 0x7f97bc0b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56280/0 (socket says 172.21.15.139:56280) 2024-04-25T09:55:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97ba59c640 1 -- 172.21.15.139:0/1781043257 learned_addr learned my addr 172.21.15.139:0/1781043257 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240261572 0 0) 0x7f97bc161dc0 con 0x7f97bc15dbf0 2024-04-25T09:55:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a8003520 con 0x7f97bc15dbf0 2024-04-25T09:55:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b9d9b640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97bc1617e0 0x7f97bc0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1788876705 0 0) 0x7f97a8003520 con 0x7f97bc15dbf0 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97bc1617e0 msgr2=0x7f97bc0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97bc1617e0 0x7f97bc0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 >> v1:172.21.15.150:6789/0 conn(0x7f97bc0cec80 legacy=0x7f97bc0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc162380 con 0x7f97bc15dbf0 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f97b40019f0 con 0x7f97bc15dbf0 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 >> v1:172.21.15.139:6789/0 conn(0x7f97bc15dbf0 legacy=0x7f97bc0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97a80039e0 0x7f97a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97bc1617e0 0x7f97a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8004d40 0x7f97a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc0d1ee0 con 0x7f97a80039e0 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a800baf0 con 0x7f97a8004d40 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a800bcf0 con 0x7f97bc1617e0 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97ba59c640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97a80039e0 0x7f97a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97b9d9b640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97bc1617e0 0x7f97a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.406+0000 7f97bad9d640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8004d40 0x7f97a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.407+0000 7f97ba59c640 1 -- 172.21.15.139:0/1781043257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97bc1617e0 msgr2=0x7f97a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.407+0000 7f97ba59c640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97bc1617e0 0x7f97a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.407+0000 7f97ba59c640 1 -- 172.21.15.139:0/1781043257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8004d40 msgr2=0x7f97a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.407+0000 7f97ba59c640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8004d40 0x7f97a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.407+0000 7f97ba59c640 1 -- 172.21.15.139:0/1781043257 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8003520 con 0x7f97a80039e0 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.407+0000 7f97b9d9b640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97bc1617e0 0x7f97a8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.407+0000 7f97ba59c640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97a80039e0 0x7f97a8003e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f97b400a860 tx=0x7f97a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acacedbce7a72df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.407+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b400a480 con 0x7f97a80039e0 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.407+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b402f070 con 0x7f97a80039e0 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.407+0000 7f97b959a640 1 -- 172.21.15.139:0/1781043257 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b402bc40 con 0x7f97a80039e0 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.408+0000 7f97c0ebc640 1 -- 172.21.15.139:0/1781043257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97a80039e0 msgr2=0x7f97a8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.408+0000 7f97c0ebc640 1 --2- 172.21.15.139:0/1781043257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97a80039e0 0x7f97a8003e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f97b400a860 tx=0x7f97a8005830 comp rx=0 tx=0).stop 2024-04-25T09:55:50.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.408+0000 7f97bad9d640 1 -- 172.21.15.139:0/1781043257 reap_dead start 2024-04-25T09:55:50.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.408+0000 7f97c0ebc640 1 -- 172.21.15.139:0/1781043257 shutdown_connections 2024-04-25T09:55:50.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.408+0000 7f97c0ebc640 1 -- 172.21.15.139:0/1781043257 >> 172.21.15.139:0/1781043257 conn(0x7f97bc0b5480 msgr2=0x7f97bc0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:50.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.408+0000 7f97c0ebc640 1 -- 172.21.15.139:0/1781043257 shutdown_connections 2024-04-25T09:55:50.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.408+0000 7f97c0ebc640 1 -- 172.21.15.139:0/1781043257 wait complete. 2024-04-25T09:55:50.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.409+0000 7f97c0ebc640 1 Processor -- start 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.409+0000 7f97c0ebc640 1 -- start start 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97c0ebc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97a80039e0 0x7f97bc15c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97c0ebc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97bc15e2e0 0x7f97bc0c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97c0ebc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97bc1532d0 0x7f97bc1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97c0ebc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc15dfd0 con 0x7f97bc1532d0 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97c0ebc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc0ce740 con 0x7f97bc15e2e0 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97c0ebc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc153b90 con 0x7f97a80039e0 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97bad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97bc1532d0 0x7f97bc1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97bad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97bc1532d0 0x7f97bc1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58226/0 (socket says 172.21.15.139:58226) 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97bad9d640 1 -- 172.21.15.139:0/3253747904 learned_addr learned my addr 172.21.15.139:0/3253747904 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97b9d9b640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97bc15e2e0 0x7f97bc0c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97ba59c640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97a80039e0 0x7f97bc15c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97bad9d640 1 -- 172.21.15.139:0/3253747904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97a80039e0 msgr2=0x7f97bc15c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97bad9d640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97a80039e0 0x7f97bc15c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97bad9d640 1 -- 172.21.15.139:0/3253747904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97bc15e2e0 msgr2=0x7f97bc0c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97bad9d640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97bc15e2e0 0x7f97bc0c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97bad9d640 1 -- 172.21.15.139:0/3253747904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc153e00 con 0x7f97bc1532d0 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.410+0000 7f97ba59c640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97a80039e0 0x7f97bc15c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.411+0000 7f97bad9d640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97bc1532d0 0x7f97bc1582c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f97bc08c420 tx=0x7f97ac00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.411+0000 7f97b2ffd640 1 -- 172.21.15.139:0/3253747904 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac019070 con 0x7f97bc1532d0 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.411+0000 7f97b2ffd640 1 -- 172.21.15.139:0/3253747904 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac007e00 con 0x7f97bc1532d0 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.411+0000 7f97b2ffd640 1 -- 172.21.15.139:0/3253747904 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac0054a0 con 0x7f97bc1532d0 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.411+0000 7f97c0ebc640 1 -- 172.21.15.139:0/3253747904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97bc1540e0 con 0x7f97bc1532d0 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.411+0000 7f97c0ebc640 1 -- 172.21.15.139:0/3253747904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97bc151e00 con 0x7f97bc1532d0 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.412+0000 7f97c0ebc640 1 -- 172.21.15.139:0/3253747904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97bc074520 con 0x7f97bc1532d0 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.412+0000 7f97b2ffd640 1 -- 172.21.15.139:0/3253747904 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97ac01c070 con 0x7f97bc1532d0 2024-04-25T09:55:50.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.413+0000 7f97b2ffd640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f979c06b670 0x7f979c06db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:50.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.413+0000 7f97b2ffd640 1 -- 172.21.15.139:0/3253747904 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f97ac0893c0 con 0x7f97bc1532d0 2024-04-25T09:55:50.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.413+0000 7f97ba59c640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f979c06b670 0x7f979c06db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:50.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.413+0000 7f97ba59c640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f979c06b670 0x7f979c06db30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f97b4033000 tx=0x7f97b4009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:50.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.418+0000 7f97b2ffd640 1 -- 172.21.15.139:0/3253747904 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97ac058110 con 0x7f97bc1532d0 2024-04-25T09:55:50.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:50.623+0000 7f97c0ebc640 1 -- 172.21.15.139:0/3253747904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f97bc0d1680 con 0x7f97bc1532d0 2024-04-25T09:55:51.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.298+0000 7f97b2ffd640 1 -- 172.21.15.139:0/3253747904 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v26) v1 ==== 173+0+0 (secure 0 0 0) 0x7f97ac057a10 con 0x7f97bc1532d0 2024-04-25T09:55:51.302 INFO:teuthology.orchestra.run.smithi139.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T09:55:51.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.307+0000 7f97c0ebc640 1 -- 172.21.15.139:0/3253747904 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f979c06b670 msgr2=0x7f979c06db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.307+0000 7f97c0ebc640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f979c06b670 0x7f979c06db30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f97b4033000 tx=0x7f97b4009030 comp rx=0 tx=0).stop 2024-04-25T09:55:51.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.307+0000 7f97c0ebc640 1 -- 172.21.15.139:0/3253747904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97bc1532d0 msgr2=0x7f97bc1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.307+0000 7f97c0ebc640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97bc1532d0 0x7f97bc1582c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f97bc08c420 tx=0x7f97ac00e820 comp rx=0 tx=0).stop 2024-04-25T09:55:51.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.307+0000 7f97c0ebc640 1 -- 172.21.15.139:0/3253747904 shutdown_connections 2024-04-25T09:55:51.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.307+0000 7f97c0ebc640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97bc1532d0 0x7f97bc1582c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.307+0000 7f97c0ebc640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f979c06b670 0x7f979c06db30 secure :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f97b4033000 tx=0x7f97b4009030 comp rx=0 tx=0).stop 2024-04-25T09:55:51.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.307+0000 7f97c0ebc640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97bc15e2e0 0x7f97bc0c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.307+0000 7f97c0ebc640 1 --2- 172.21.15.139:0/3253747904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97a80039e0 0x7f97bc15c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.307+0000 7f97c0ebc640 1 -- 172.21.15.139:0/3253747904 >> 172.21.15.139:0/3253747904 conn(0x7f97bc0b5480 msgr2=0x7f97bc0d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:51.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.308+0000 7f97c0ebc640 1 -- 172.21.15.139:0/3253747904 shutdown_connections 2024-04-25T09:55:51.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.308+0000 7f97c0ebc640 1 -- 172.21.15.139:0/3253747904 wait complete. 2024-04-25T09:55:51.332 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:51.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.428+0000 7f6598b77640 1 Processor -- start 2024-04-25T09:55:51.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.429+0000 7f6598b77640 1 -- start start 2024-04-25T09:55:51.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f6598b77640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f659415ac40 0x7f659406e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f6598b77640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65940af6d0 con 0x7f659415a860 2024-04-25T09:55:51.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f6598b77640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65940af9b0 con 0x7f659406e850 2024-04-25T09:55:51.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f6598b77640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65940b02e0 con 0x7f659415ac40 2024-04-25T09:55:51.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659259c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f659415a860 0x7f65940cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56290/0 (socket says 172.21.15.139:56290) 2024-04-25T09:55:51.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659259c640 1 -- 172.21.15.139:0/4257285584 learned_addr learned my addr 172.21.15.139:0/4257285584 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:51.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f6591d9b640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f659415ac40 0x7f659406e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289570661 0 0) 0x7f65940af6d0 con 0x7f659415a860 2024-04-25T09:55:51.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f657c003520 con 0x7f659415a860 2024-04-25T09:55:51.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 86779569 0 0) 0x7f657c003520 con 0x7f659415a860 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f659415ac40 msgr2=0x7f659406e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f659415ac40 0x7f659406e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 >> v1:172.21.15.150:6789/0 conn(0x7f659406e850 legacy=0x7f65940b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65940afc90 con 0x7f659415a860 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f65840019f0 con 0x7f659415a860 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 >> v1:172.21.15.139:6789/0 conn(0x7f659415a860 legacy=0x7f65940cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f657c0039e0 0x7f657c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f657c004340 0x7f657c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f657c005020 0x7f657c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65940b02e0 con 0x7f657c005020 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c00bf30 con 0x7f657c004340 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c00c130 con 0x7f657c0039e0 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f6591d9b640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f659415ac40 0x7f659406e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:51.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f6592d9d640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f657c005020 0x7f657c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.430+0000 7f659259c640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f657c0039e0 0x7f657c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.431+0000 7f6591d9b640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f657c004340 0x7f657c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.431+0000 7f6592d9d640 1 -- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f657c0039e0 msgr2=0x7f657c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.431+0000 7f6592d9d640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f657c0039e0 0x7f657c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.431+0000 7f6592d9d640 1 -- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f657c004340 msgr2=0x7f657c004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.431+0000 7f6592d9d640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f657c004340 0x7f657c004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.431+0000 7f6592d9d640 1 -- 172.21.15.139:0/4257285584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c003520 con 0x7f657c005020 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.431+0000 7f6591d9b640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f657c004340 0x7f657c004820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.431+0000 7f6592d9d640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f657c005020 0x7f657c00b9c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f658c001910 tx=0x7f657c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588a0af1bb82615f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.432+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c037070 con 0x7f657c005020 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.432+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c02ac40 con 0x7f657c005020 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.432+0000 7f659159a640 1 -- 172.21.15.139:0/4257285584 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c0014e0 con 0x7f657c005020 2024-04-25T09:55:51.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.432+0000 7f6598b77640 1 -- 172.21.15.139:0/4257285584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f657c005020 msgr2=0x7f657c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.432+0000 7f6598b77640 1 --2- 172.21.15.139:0/4257285584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f657c005020 0x7f657c00b9c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f658c001910 tx=0x7f657c005830 comp rx=0 tx=0).stop 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.432+0000 7f6592d9d640 1 -- 172.21.15.139:0/4257285584 reap_dead start 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.432+0000 7f6598b77640 1 -- 172.21.15.139:0/4257285584 shutdown_connections 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.432+0000 7f6598b77640 1 -- 172.21.15.139:0/4257285584 >> 172.21.15.139:0/4257285584 conn(0x7f6594066f20 msgr2=0x7f659406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.432+0000 7f6598b77640 1 -- 172.21.15.139:0/4257285584 shutdown_connections 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.432+0000 7f6598b77640 1 -- 172.21.15.139:0/4257285584 wait complete. 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.433+0000 7f6598b77640 1 Processor -- start 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.434+0000 7f6598b77640 1 -- start start 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.434+0000 7f6598b77640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f657c005020 0x7f659415b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.434+0000 7f6598b77640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f659413cb30 0x7f659415b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.434+0000 7f6598b77640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f659415e2f0 0x7f659415e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.434+0000 7f6598b77640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65940af2a0 con 0x7f659415e2f0 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.434+0000 7f6598b77640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65940cd3c0 con 0x7f659413cb30 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.434+0000 7f6598b77640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65940af420 con 0x7f657c005020 2024-04-25T09:55:51.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.434+0000 7f6592d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f659415e2f0 0x7f659415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.434+0000 7f6592d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f659415e2f0 0x7f659415e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58242/0 (socket says 172.21.15.139:58242) 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.434+0000 7f6592d9d640 1 -- 172.21.15.139:0/2282628348 learned_addr learned my addr 172.21.15.139:0/2282628348 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f659259c640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f657c005020 0x7f659415b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f6592d9d640 1 -- 172.21.15.139:0/2282628348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f657c005020 msgr2=0x7f659415b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f6592d9d640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f657c005020 0x7f659415b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f6591d9b640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f659413cb30 0x7f659415b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f6592d9d640 1 -- 172.21.15.139:0/2282628348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f659413cb30 msgr2=0x7f659415b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f6592d9d640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f659413cb30 0x7f659415b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f6592d9d640 1 -- 172.21.15.139:0/2282628348 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659414fe80 con 0x7f659415e2f0 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f6592d9d640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f659415e2f0 0x7f659415e770 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f658c035040 tx=0x7f658c0012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f6582ffd640 1 -- 172.21.15.139:0/2282628348 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c037040 con 0x7f659415e2f0 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f6582ffd640 1 -- 172.21.15.139:0/2282628348 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c041070 con 0x7f659415e2f0 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f6598b77640 1 -- 172.21.15.139:0/2282628348 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659415d530 con 0x7f659415e2f0 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f6582ffd640 1 -- 172.21.15.139:0/2282628348 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c0014e0 con 0x7f659415e2f0 2024-04-25T09:55:51.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.435+0000 7f6598b77640 1 -- 172.21.15.139:0/2282628348 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659415daf0 con 0x7f659415e2f0 2024-04-25T09:55:51.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.436+0000 7f6598b77640 1 -- 172.21.15.139:0/2282628348 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6594074520 con 0x7f659415e2f0 2024-04-25T09:55:51.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.436+0000 7f6582ffd640 1 -- 172.21.15.139:0/2282628348 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f658c03a070 con 0x7f659415e2f0 2024-04-25T09:55:51.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.437+0000 7f6582ffd640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f657406b4d0 0x7f657406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.437+0000 7f6582ffd640 1 -- 172.21.15.139:0/2282628348 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f658c0a9730 con 0x7f659415e2f0 2024-04-25T09:55:51.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.437+0000 7f659259c640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f657406b4d0 0x7f657406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.437+0000 7f659259c640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f657406b4d0 0x7f657406d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6584004c20 tx=0x7f6584006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:51.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.441+0000 7f6582ffd640 1 -- 172.21.15.139:0/2282628348 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f658c04b020 con 0x7f659415e2f0 2024-04-25T09:55:51.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.613+0000 7f6598b77640 1 -- 172.21.15.139:0/2282628348 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65940b1ef0 con 0x7f659415e2f0 2024-04-25T09:55:51.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.614+0000 7f6582ffd640 1 -- 172.21.15.139:0/2282628348 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16945 (secure 0 0 0) 0x7f658c078420 con 0x7f659415e2f0 2024-04-25T09:55:51.617 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:51.617 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":61,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:55:50.625984+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:48.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:49.597505+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:51.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.616+0000 7f6598b77640 1 -- 172.21.15.139:0/2282628348 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f657406b4d0 msgr2=0x7f657406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.616+0000 7f6598b77640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f657406b4d0 0x7f657406d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6584004c20 tx=0x7f6584006040 comp rx=0 tx=0).stop 2024-04-25T09:55:51.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.616+0000 7f6598b77640 1 -- 172.21.15.139:0/2282628348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f659415e2f0 msgr2=0x7f659415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.616+0000 7f6598b77640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f659415e2f0 0x7f659415e770 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f658c035040 tx=0x7f658c0012b0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.616+0000 7f6598b77640 1 -- 172.21.15.139:0/2282628348 shutdown_connections 2024-04-25T09:55:51.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.616+0000 7f6598b77640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f659415e2f0 0x7f659415e770 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.616+0000 7f6598b77640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f657406b4d0 0x7f657406d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.616+0000 7f6598b77640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f659413cb30 0x7f659415b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.616+0000 7f6598b77640 1 --2- 172.21.15.139:0/2282628348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f657c005020 0x7f659415b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.616+0000 7f6598b77640 1 -- 172.21.15.139:0/2282628348 >> 172.21.15.139:0/2282628348 conn(0x7f6594066f20 msgr2=0x7f659406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:51.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.616+0000 7f6598b77640 1 -- 172.21.15.139:0/2282628348 shutdown_connections 2024-04-25T09:55:51.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.616+0000 7f6598b77640 1 -- 172.21.15.139:0/2282628348 wait complete. 2024-04-25T09:55:51.630 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:51.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.724+0000 7fcdc1a88640 1 Processor -- start 2024-04-25T09:55:51.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.724+0000 7fcdc1a88640 1 -- start start 2024-04-25T09:55:51.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdc1a88640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcdbc158260 0x7fcdbc15df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdc1a88640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcdbc0b37d0 0x7fcdbc06ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdc1a88640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdbc0b3e10 con 0x7fcdbc06da20 2024-04-25T09:55:51.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdc1a88640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdbc1615c0 con 0x7fcdbc158260 2024-04-25T09:55:51.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdc1a88640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdbc1618a0 con 0x7fcdbc0b37d0 2024-04-25T09:55:51.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdbb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcdbc158260 0x7fcdbc15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdbb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcdbc158260 0x7fcdbc15df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58252/0 (socket says 172.21.15.139:58252) 2024-04-25T09:55:51.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdbb7fe640 1 -- 172.21.15.139:0/2167103651 learned_addr learned my addr 172.21.15.139:0/2167103651 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:51.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdbaffd640 1 --1- 172.21.15.139:0/2167103651 >> v1:172.21.15.139:6789/0 conn(0x7fcdbc06da20 0x7fcdbc0b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56298/0 (socket says 172.21.15.139:56298) 2024-04-25T09:55:51.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdbbfff640 1 --2- 172.21.15.139:0/2167103651 >> v2:172.21.15.150:3300/0 conn(0x7fcdbc0b37d0 0x7fcdbc06ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdbb7fe640 1 -- 172.21.15.139:0/2167103651 >> v2:172.21.15.150:3300/0 conn(0x7fcdbc0b37d0 msgr2=0x7fcdbc06ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:51.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdbb7fe640 1 --2- 172.21.15.139:0/2167103651 >> v2:172.21.15.150:3300/0 conn(0x7fcdbc0b37d0 0x7fcdbc06ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdbb7fe640 1 -- 172.21.15.139:0/2167103651 >> v1:172.21.15.139:6789/0 conn(0x7fcdbc06da20 legacy=0x7fcdbc0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdbb7fe640 1 -- 172.21.15.139:0/2167103651 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdbc161b80 con 0x7fcdbc158260 2024-04-25T09:55:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.725+0000 7fcdba7fc640 1 -- 172.21.15.139:0/2167103651 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636592498 0 0) 0x7fcdb0001110 con 0x7fcdbc06da20 2024-04-25T09:55:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.726+0000 7fcdbb7fe640 1 --2- 172.21.15.139:0/2167103651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc158260 0x7fcdbc15df80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc161500 tx=0x7fcdac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33c991851df9638 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.726+0000 7fcdba7fc640 1 -- 172.21.15.139:0/2167103651 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac037070 con 0x7fcdbc158260 2024-04-25T09:55:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.726+0000 7fcdba7fc640 1 -- 172.21.15.139:0/2167103651 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdac032440 con 0x7fcdbc158260 2024-04-25T09:55:51.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.726+0000 7fcdba7fc640 1 -- 172.21.15.139:0/2167103651 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac032810 con 0x7fcdbc158260 2024-04-25T09:55:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.727+0000 7fcdc1a88640 1 -- 172.21.15.139:0/2167103651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc158260 msgr2=0x7fcdbc15df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.727+0000 7fcdc1a88640 1 --2- 172.21.15.139:0/2167103651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc158260 0x7fcdbc15df80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc161500 tx=0x7fcdac02a270 comp rx=0 tx=0).stop 2024-04-25T09:55:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.727+0000 7fcdc1a88640 1 -- 172.21.15.139:0/2167103651 shutdown_connections 2024-04-25T09:55:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.727+0000 7fcdc1a88640 1 --2- 172.21.15.139:0/2167103651 >> v2:172.21.15.150:3300/0 conn(0x7fcdbc0b37d0 0x7fcdbc06ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.727+0000 7fcdc1a88640 1 --2- 172.21.15.139:0/2167103651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc158260 0x7fcdbc15df80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.727+0000 7fcdc1a88640 1 -- 172.21.15.139:0/2167103651 >> 172.21.15.139:0/2167103651 conn(0x7fcdbc0b23d0 msgr2=0x7fcdbc0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:51.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.727+0000 7fcdc1a88640 1 -- 172.21.15.139:0/2167103651 shutdown_connections 2024-04-25T09:55:51.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.727+0000 7fcdc1a88640 1 -- 172.21.15.139:0/2167103651 wait complete. 2024-04-25T09:55:51.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.728+0000 7fcdc1a88640 1 Processor -- start 2024-04-25T09:55:51.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.728+0000 7fcdc1a88640 1 -- start start 2024-04-25T09:55:51.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdc1a88640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdbc06da20 0x7fcdbc148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdc1a88640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdbc0b37d0 0x7fcdbc144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdc1a88640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc158260 0x7fcdbc158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdc1a88640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc162420 con 0x7fcdbc158260 2024-04-25T09:55:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdc1a88640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc162140 con 0x7fcdbc0b37d0 2024-04-25T09:55:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdc1a88640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc06b2e0 con 0x7fcdbc06da20 2024-04-25T09:55:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdbbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc158260 0x7fcdbc158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdbbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc158260 0x7fcdbc158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58256/0 (socket says 172.21.15.139:58256) 2024-04-25T09:55:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdbbfff640 1 -- 172.21.15.139:0/283960262 learned_addr learned my addr 172.21.15.139:0/283960262 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdbb7fe640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdbc06da20 0x7fcdbc148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdbaffd640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdbc0b37d0 0x7fcdbc144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdbbfff640 1 -- 172.21.15.139:0/283960262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdbc06da20 msgr2=0x7fcdbc148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdbbfff640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdbc06da20 0x7fcdbc148200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdbbfff640 1 -- 172.21.15.139:0/283960262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdbc0b37d0 msgr2=0x7fcdbc144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdbbfff640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdbc0b37d0 0x7fcdbc144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdbbfff640 1 -- 172.21.15.139:0/283960262 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdbc148740 con 0x7fcdbc158260 2024-04-25T09:55:51.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdbaffd640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdbc0b37d0 0x7fcdbc144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:51.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcdbbfff640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc158260 0x7fcdbc158690 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4007e60 tx=0x7fcdb400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:51.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.729+0000 7fcda3fff640 1 -- 172.21.15.139:0/283960262 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4014620 con 0x7fcdbc158260 2024-04-25T09:55:51.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.730+0000 7fcda3fff640 1 -- 172.21.15.139:0/283960262 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb4019070 con 0x7fcdbc158260 2024-04-25T09:55:51.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.730+0000 7fcdc1a88640 1 -- 172.21.15.139:0/283960262 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdbc145bb0 con 0x7fcdbc158260 2024-04-25T09:55:51.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.730+0000 7fcda3fff640 1 -- 172.21.15.139:0/283960262 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4014a00 con 0x7fcdbc158260 2024-04-25T09:55:51.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.730+0000 7fcdc1a88640 1 -- 172.21.15.139:0/283960262 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc146170 con 0x7fcdbc158260 2024-04-25T09:55:51.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.731+0000 7fcdc1a88640 1 -- 172.21.15.139:0/283960262 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdbc074520 con 0x7fcdbc158260 2024-04-25T09:55:51.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.731+0000 7fcda3fff640 1 -- 172.21.15.139:0/283960262 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdb4014ba0 con 0x7fcdbc158260 2024-04-25T09:55:51.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.732+0000 7fcda3fff640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcda806b4d0 0x7fcda806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:51.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.732+0000 7fcda3fff640 1 -- 172.21.15.139:0/283960262 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcdb408ab70 con 0x7fcdbc158260 2024-04-25T09:55:51.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.732+0000 7fcdbb7fe640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcda806b4d0 0x7fcda806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:51.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.732+0000 7fcdbb7fe640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcda806b4d0 0x7fcda806d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcdac034000 tx=0x7fcdac02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:51.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.736+0000 7fcda3fff640 1 -- 172.21.15.139:0/283960262 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdbc148740 con 0x7fcdbc158260 2024-04-25T09:55:51.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.909+0000 7fcdc1a88640 1 -- 172.21.15.139:0/283960262 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdbc0d0aa0 con 0x7fcdbc158260 2024-04-25T09:55:51.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.910+0000 7fcda3fff640 1 -- 172.21.15.139:0/283960262 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16945 (secure 0 0 0) 0x7fcdb40598f0 con 0x7fcdbc158260 2024-04-25T09:55:51.913 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:51.913 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":61,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:55:50.625984+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:48.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:49.597505+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:51.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.912+0000 7fcdc1a88640 1 -- 172.21.15.139:0/283960262 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcda806b4d0 msgr2=0x7fcda806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.912+0000 7fcdc1a88640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcda806b4d0 0x7fcda806d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcdac034000 tx=0x7fcdac02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:51.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.912+0000 7fcdc1a88640 1 -- 172.21.15.139:0/283960262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc158260 msgr2=0x7fcdbc158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:51.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.912+0000 7fcdc1a88640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc158260 0x7fcdbc158690 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4007e60 tx=0x7fcdb400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.912+0000 7fcdc1a88640 1 -- 172.21.15.139:0/283960262 shutdown_connections 2024-04-25T09:55:51.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.912+0000 7fcdc1a88640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc158260 0x7fcdbc158690 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.912+0000 7fcdc1a88640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcda806b4d0 0x7fcda806d990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.912+0000 7fcdc1a88640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdbc0b37d0 0x7fcdbc144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.912+0000 7fcdc1a88640 1 --2- 172.21.15.139:0/283960262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdbc06da20 0x7fcdbc148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:51.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.912+0000 7fcdc1a88640 1 -- 172.21.15.139:0/283960262 >> 172.21.15.139:0/283960262 conn(0x7fcdbc0b23d0 msgr2=0x7fcdbc15d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:51.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.912+0000 7fcdc1a88640 1 -- 172.21.15.139:0/283960262 shutdown_connections 2024-04-25T09:55:51.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:51.912+0000 7fcdc1a88640 1 -- 172.21.15.139:0/283960262 wait complete. 2024-04-25T09:55:51.926 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:55:52.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.023+0000 7f294b7f9640 1 Processor -- start 2024-04-25T09:55:52.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.023+0000 7f294b7f9640 1 -- start start 2024-04-25T09:55:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.024+0000 7f294b7f9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f294415da30 0x7f29440704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.024+0000 7f294b7f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2944072d60 0x7f294406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.024+0000 7f294b7f9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29440731a0 con 0x7f294406b0a0 2024-04-25T09:55:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.024+0000 7f294b7f9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29440cd540 con 0x7f294415da30 2024-04-25T09:55:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.024+0000 7f294b7f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29440b3de0 con 0x7f2944072d60 2024-04-25T09:55:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.024+0000 7f2949d96640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f294406b0a0 0x7f29440d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56310/0 (socket says 172.21.15.139:56310) 2024-04-25T09:55:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.024+0000 7f2949d96640 1 -- 172.21.15.139:0/1018356835 learned_addr learned my addr 172.21.15.139:0/1018356835 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.024+0000 7f2948d94640 1 --2- 172.21.15.139:0/1018356835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2944072d60 0x7f294406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.025+0000 7f2949595640 1 --2- 172.21.15.139:0/1018356835 >> v2:172.21.15.150:3300/0 conn(0x7f294415da30 0x7f29440704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.025+0000 7f293bfff640 1 -- 172.21.15.139:0/1018356835 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394871447 0 0) 0x7f29440731a0 con 0x7f294406b0a0 2024-04-25T09:55:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.025+0000 7f293bfff640 1 -- 172.21.15.139:0/1018356835 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2930003520 con 0x7f294406b0a0 2024-04-25T09:55:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.025+0000 7f293bfff640 1 -- 172.21.15.139:0/1018356835 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3585018214 0 0) 0x7f2930003520 con 0x7f294406b0a0 2024-04-25T09:55:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.025+0000 7f2948d94640 1 -- 172.21.15.139:0/1018356835 >> v2:172.21.15.150:3300/0 conn(0x7f294415da30 msgr2=0x7f29440704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.025+0000 7f2948d94640 1 --2- 172.21.15.139:0/1018356835 >> v2:172.21.15.150:3300/0 conn(0x7f294415da30 0x7f29440704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.025+0000 7f2948d94640 1 -- 172.21.15.139:0/1018356835 >> v1:172.21.15.139:6789/0 conn(0x7f294406b0a0 legacy=0x7f29440d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.025+0000 7f2948d94640 1 -- 172.21.15.139:0/1018356835 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294415de10 con 0x7f2944072d60 2024-04-25T09:55:52.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.025+0000 7f2949595640 1 --2- 172.21.15.139:0/1018356835 >> v2:172.21.15.150:3300/0 conn(0x7f294415da30 0x7f29440704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:52.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.026+0000 7f2948d94640 1 --2- 172.21.15.139:0/1018356835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2944072d60 0x7f294406ab60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f29340097a0 tx=0x7f293402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67b02f0320fdd278 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:52.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.026+0000 7f293bfff640 1 -- 172.21.15.139:0/1018356835 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934037070 con 0x7f2944072d60 2024-04-25T09:55:52.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.026+0000 7f293bfff640 1 -- 172.21.15.139:0/1018356835 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293402aba0 con 0x7f2944072d60 2024-04-25T09:55:52.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.026+0000 7f293bfff640 1 -- 172.21.15.139:0/1018356835 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c001670 con 0x7f2944072d60 2024-04-25T09:55:52.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.026+0000 7f294b7f9640 1 -- 172.21.15.139:0/1018356835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2944072d60 msgr2=0x7f294406ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.026+0000 7f294b7f9640 1 --2- 172.21.15.139:0/1018356835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2944072d60 0x7f294406ab60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f29340097a0 tx=0x7f293402a110 comp rx=0 tx=0).stop 2024-04-25T09:55:52.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.026+0000 7f294b7f9640 1 -- 172.21.15.139:0/1018356835 shutdown_connections 2024-04-25T09:55:52.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.026+0000 7f294b7f9640 1 --2- 172.21.15.139:0/1018356835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2944072d60 0x7f294406ab60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.026+0000 7f294b7f9640 1 --2- 172.21.15.139:0/1018356835 >> v2:172.21.15.150:3300/0 conn(0x7f294415da30 0x7f29440704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.026+0000 7f294b7f9640 1 -- 172.21.15.139:0/1018356835 >> 172.21.15.139:0/1018356835 conn(0x7f29440b23f0 msgr2=0x7f29440b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:52.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.026+0000 7f294b7f9640 1 -- 172.21.15.139:0/1018356835 shutdown_connections 2024-04-25T09:55:52.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.026+0000 7f294b7f9640 1 -- 172.21.15.139:0/1018356835 wait complete. 2024-04-25T09:55:52.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.028+0000 7f294b7f9640 1 Processor -- start 2024-04-25T09:55:52.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.028+0000 7f294b7f9640 1 -- start start 2024-04-25T09:55:52.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.028+0000 7f294b7f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f294406b0a0 0x7f2944148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.028+0000 7f294b7f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2944072d60 0x7f2944158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.028+0000 7f294b7f9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f294415da30 0x7f2944159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.028+0000 7f294b7f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29440b42a0 con 0x7f294406b0a0 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.028+0000 7f294b7f9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29440b3b00 con 0x7f294415da30 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.028+0000 7f294b7f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29440b4580 con 0x7f2944072d60 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.028+0000 7f2949595640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f294406b0a0 0x7f2944148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.028+0000 7f2949595640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f294406b0a0 0x7f2944148290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58258/0 (socket says 172.21.15.139:58258) 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.028+0000 7f2949595640 1 -- 172.21.15.139:0/1281661414 learned_addr learned my addr 172.21.15.139:0/1281661414 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f2949595640 1 -- 172.21.15.139:0/1281661414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2944072d60 msgr2=0x7f2944158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f2948d94640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2944072d60 0x7f2944158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f2949d96640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f294415da30 0x7f2944159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f2949595640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2944072d60 0x7f2944158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f2949595640 1 -- 172.21.15.139:0/1281661414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f294415da30 msgr2=0x7f2944159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f2949595640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f294415da30 0x7f2944159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f2949595640 1 -- 172.21.15.139:0/1281661414 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29440ca8f0 con 0x7f294406b0a0 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f2949595640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f294406b0a0 0x7f2944148290 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f294000efc0 tx=0x7f294000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:52.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f2939ffb640 1 -- 172.21.15.139:0/1281661414 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940018020 con 0x7f294406b0a0 2024-04-25T09:55:52.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f2939ffb640 1 -- 172.21.15.139:0/1281661414 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940014680 con 0x7f294406b0a0 2024-04-25T09:55:52.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f294b7f9640 1 -- 172.21.15.139:0/1281661414 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29440cabd0 con 0x7f294406b0a0 2024-04-25T09:55:52.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f2939ffb640 1 -- 172.21.15.139:0/1281661414 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940014a50 con 0x7f294406b0a0 2024-04-25T09:55:52.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.029+0000 7f294b7f9640 1 -- 172.21.15.139:0/1281661414 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29440cb190 con 0x7f294406b0a0 2024-04-25T09:55:52.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.030+0000 7f294b7f9640 1 -- 172.21.15.139:0/1281661414 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29440b2210 con 0x7f294406b0a0 2024-04-25T09:55:52.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.031+0000 7f2939ffb640 1 -- 172.21.15.139:0/1281661414 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2940005d10 con 0x7f294406b0a0 2024-04-25T09:55:52.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.031+0000 7f2939ffb640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f292806b4d0 0x7f292806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.031+0000 7f2939ffb640 1 -- 172.21.15.139:0/1281661414 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f294008b280 con 0x7f294406b0a0 2024-04-25T09:55:52.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.031+0000 7f2948d94640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f292806b4d0 0x7f292806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.032+0000 7f2948d94640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f292806b4d0 0x7f292806d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2934034000 tx=0x7f293402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:52.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.035+0000 7f2939ffb640 1 -- 172.21.15.139:0/1281661414 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29440b2210 con 0x7f294406b0a0 2024-04-25T09:55:52.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.242+0000 7f294b7f9640 1 -- 172.21.15.139:0/1281661414 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2944074520 con 0x7f294406b0a0 2024-04-25T09:55:52.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.296+0000 7f2939ffb640 1 -- 172.21.15.139:0/1281661414 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v27) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2940022030 con 0x7f294406b0a0 2024-04-25T09:55:52.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.298+0000 7f294b7f9640 1 -- 172.21.15.139:0/1281661414 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f292806b4d0 msgr2=0x7f292806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.298+0000 7f294b7f9640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f292806b4d0 0x7f292806d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2934034000 tx=0x7f293402e000 comp rx=0 tx=0).stop 2024-04-25T09:55:52.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.298+0000 7f294b7f9640 1 -- 172.21.15.139:0/1281661414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f294406b0a0 msgr2=0x7f2944148290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.298+0000 7f294b7f9640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f294406b0a0 0x7f2944148290 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f294000efc0 tx=0x7f294000c490 comp rx=0 tx=0).stop 2024-04-25T09:55:52.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.298+0000 7f294b7f9640 1 -- 172.21.15.139:0/1281661414 shutdown_connections 2024-04-25T09:55:52.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.298+0000 7f294b7f9640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f292806b4d0 0x7f292806d990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.298+0000 7f294b7f9640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f294415da30 0x7f2944159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.298+0000 7f294b7f9640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2944072d60 0x7f2944158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.298+0000 7f294b7f9640 1 --2- 172.21.15.139:0/1281661414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f294406b0a0 0x7f2944148290 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.298+0000 7f294b7f9640 1 -- 172.21.15.139:0/1281661414 >> 172.21.15.139:0/1281661414 conn(0x7f29440b23f0 msgr2=0x7f29440b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:52.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.298+0000 7f294b7f9640 1 -- 172.21.15.139:0/1281661414 shutdown_connections 2024-04-25T09:55:52.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.298+0000 7f294b7f9640 1 -- 172.21.15.139:0/1281661414 wait complete. 2024-04-25T09:55:52.314 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:55:52.314 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:55:52.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.411+0000 7f6eccefc640 1 Processor -- start 2024-04-25T09:55:52.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.411+0000 7f6eccefc640 1 -- start start 2024-04-25T09:55:52.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.412+0000 7f6eccefc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6ec80b31a0 0x7f6ec806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.412+0000 7f6eccefc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ec806e870 0x7f6ec815e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.412+0000 7f6eccefc640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec80b2a50 con 0x7f6ec8144090 2024-04-25T09:55:52.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.412+0000 7f6eccefc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec806ece0 con 0x7f6ec80b31a0 2024-04-25T09:55:52.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.412+0000 7f6eccefc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80b3610 con 0x7f6ec806e870 2024-04-25T09:55:52.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.412+0000 7f6ec5d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6ec80b31a0 0x7f6ec806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.412+0000 7f6ec5d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6ec80b31a0 0x7f6ec806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58262/0 (socket says 172.21.15.139:58262) 2024-04-25T09:55:52.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.412+0000 7f6ec5d9b640 1 -- 172.21.15.139:0/1660896710 learned_addr learned my addr 172.21.15.139:0/1660896710 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:52.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.412+0000 7f6ec5d9b640 1 -- 172.21.15.139:0/1660896710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ec806e870 msgr2=0x7f6ec815e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:52.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.412+0000 7f6ec5d9b640 1 --2- 172.21.15.139:0/1660896710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ec806e870 0x7f6ec815e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.412+0000 7f6ec5d9b640 1 -- 172.21.15.139:0/1660896710 >> v1:172.21.15.150:6789/0 conn(0x7f6ec8144090 legacy=0x7f6ec80cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:52.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.412+0000 7f6ec5d9b640 1 -- 172.21.15.139:0/1660896710 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec80cd0f0 con 0x7f6ec80b31a0 2024-04-25T09:55:52.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.413+0000 7f6ec5d9b640 1 --2- 172.21.15.139:0/1660896710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec80b31a0 0x7f6ec806e330 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009870 tx=0x7f6ebc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416f344f51ead58d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:52.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.413+0000 7f6ec559a640 1 -- 172.21.15.139:0/1660896710 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc037070 con 0x7f6ec80b31a0 2024-04-25T09:55:52.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.413+0000 7f6ec559a640 1 -- 172.21.15.139:0/1660896710 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc032440 con 0x7f6ec80b31a0 2024-04-25T09:55:52.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.413+0000 7f6ec559a640 1 -- 172.21.15.139:0/1660896710 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc032810 con 0x7f6ec80b31a0 2024-04-25T09:55:52.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.414+0000 7f6eccefc640 1 -- 172.21.15.139:0/1660896710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec80b31a0 msgr2=0x7f6ec806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.414+0000 7f6eccefc640 1 --2- 172.21.15.139:0/1660896710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec80b31a0 0x7f6ec806e330 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009870 tx=0x7f6ebc02a180 comp rx=0 tx=0).stop 2024-04-25T09:55:52.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.414+0000 7f6eccefc640 1 -- 172.21.15.139:0/1660896710 shutdown_connections 2024-04-25T09:55:52.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.414+0000 7f6eccefc640 1 --2- 172.21.15.139:0/1660896710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ec806e870 0x7f6ec815e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.414+0000 7f6eccefc640 1 --2- 172.21.15.139:0/1660896710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec80b31a0 0x7f6ec806e330 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.414+0000 7f6eccefc640 1 -- 172.21.15.139:0/1660896710 >> 172.21.15.139:0/1660896710 conn(0x7f6ec8067000 msgr2=0x7f6ec806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:52.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.414+0000 7f6eccefc640 1 -- 172.21.15.139:0/1660896710 shutdown_connections 2024-04-25T09:55:52.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.414+0000 7f6eccefc640 1 -- 172.21.15.139:0/1660896710 wait complete. 2024-04-25T09:55:52.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.415+0000 7f6eccefc640 1 Processor -- start 2024-04-25T09:55:52.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.415+0000 7f6eccefc640 1 -- start start 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6eccefc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ec806e870 0x7f6ec81f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6eccefc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec80b31a0 0x7f6ec81f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6eccefc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ec8144090 0x7f6ec806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6eccefc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec815b650 con 0x7f6ec80b31a0 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6eccefc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec815b370 con 0x7f6ec806e870 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6eccefc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec815b930 con 0x7f6ec8144090 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6ec5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec80b31a0 0x7f6ec81f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6ec5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec80b31a0 0x7f6ec81f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58270/0 (socket says 172.21.15.139:58270) 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6ec5d9b640 1 -- 172.21.15.139:0/292417360 learned_addr learned my addr 172.21.15.139:0/292417360 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6ec659c640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ec806e870 0x7f6ec81f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6ec6d9d640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ec8144090 0x7f6ec806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6ec5d9b640 1 -- 172.21.15.139:0/292417360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ec8144090 msgr2=0x7f6ec806d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6ec5d9b640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ec8144090 0x7f6ec806d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6ec5d9b640 1 -- 172.21.15.139:0/292417360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ec806e870 msgr2=0x7f6ec81f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:52.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6ec5d9b640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ec806e870 0x7f6ec81f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:52.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6ec5d9b640 1 -- 172.21.15.139:0/292417360 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec81f23a0 con 0x7f6ec80b31a0 2024-04-25T09:55:52.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6ec659c640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ec806e870 0x7f6ec81f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:52.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6ec5d9b640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec80b31a0 0x7f6ec81f1440 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009870 tx=0x7f6ebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:52.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6eb6ffd640 1 -- 172.21.15.139:0/292417360 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc041070 con 0x7f6ec80b31a0 2024-04-25T09:55:52.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6eb6ffd640 1 -- 172.21.15.139:0/292417360 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc03c870 con 0x7f6ec80b31a0 2024-04-25T09:55:52.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6eb6ffd640 1 -- 172.21.15.139:0/292417360 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc037040 con 0x7f6ec80b31a0 2024-04-25T09:55:52.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6eccefc640 1 -- 172.21.15.139:0/292417360 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec81f1de0 con 0x7f6ec80b31a0 2024-04-25T09:55:52.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.416+0000 7f6eccefc640 1 -- 172.21.15.139:0/292417360 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec81f4ae0 con 0x7f6ec80b31a0 2024-04-25T09:55:52.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.417+0000 7f6eccefc640 1 -- 172.21.15.139:0/292417360 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec8074520 con 0x7f6ec80b31a0 2024-04-25T09:55:52.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.418+0000 7f6eb6ffd640 1 -- 172.21.15.139:0/292417360 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ebc044070 con 0x7f6ec80b31a0 2024-04-25T09:55:52.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.418+0000 7f6eb6ffd640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6ea806b500 0x7f6ea806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:52.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.418+0000 7f6eb6ffd640 1 -- 172.21.15.139:0/292417360 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ebc0aae20 con 0x7f6ec80b31a0 2024-04-25T09:55:52.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.418+0000 7f6ec659c640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6ea806b500 0x7f6ea806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:52.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.419+0000 7f6ec659c640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6ea806b500 0x7f6ea806d9c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80cf080 tx=0x7f6eb8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:52.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.423+0000 7f6eb6ffd640 1 -- 172.21.15.139:0/292417360 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ec80b2a50 con 0x7f6ec80b31a0 2024-04-25T09:55:52.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:52.633+0000 7f6eccefc640 1 -- 172.21.15.139:0/292417360 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6ec806e000 con 0x7f6ec80b31a0 2024-04-25T09:55:53.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.305+0000 7f6eb6ffd640 1 -- 172.21.15.139:0/292417360 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ebc079ba0 con 0x7f6ec80b31a0 2024-04-25T09:55:53.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.307+0000 7f6eccefc640 1 -- 172.21.15.139:0/292417360 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6ea806b500 msgr2=0x7f6ea806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.307+0000 7f6eccefc640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6ea806b500 0x7f6ea806d9c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80cf080 tx=0x7f6eb8009040 comp rx=0 tx=0).stop 2024-04-25T09:55:53.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.307+0000 7f6eccefc640 1 -- 172.21.15.139:0/292417360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec80b31a0 msgr2=0x7f6ec81f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.307+0000 7f6eccefc640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec80b31a0 0x7f6ec81f1440 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009870 tx=0x7f6ebc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.307+0000 7f6eccefc640 1 -- 172.21.15.139:0/292417360 shutdown_connections 2024-04-25T09:55:53.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.307+0000 7f6eccefc640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ec80b31a0 0x7f6ec81f1440 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.307+0000 7f6eccefc640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6ea806b500 0x7f6ea806d9c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.307+0000 7f6eccefc640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ec8144090 0x7f6ec806d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.307+0000 7f6eccefc640 1 --2- 172.21.15.139:0/292417360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ec806e870 0x7f6ec81f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.307+0000 7f6eccefc640 1 -- 172.21.15.139:0/292417360 >> 172.21.15.139:0/292417360 conn(0x7f6ec8067000 msgr2=0x7f6ec80af260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:53.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.307+0000 7f6eccefc640 1 -- 172.21.15.139:0/292417360 shutdown_connections 2024-04-25T09:55:53.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.307+0000 7f6eccefc640 1 -- 172.21.15.139:0/292417360 wait complete. 2024-04-25T09:55:53.322 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:55:53.322 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:53.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.417+0000 7f8e316ee640 1 Processor -- start 2024-04-25T09:55:53.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.417+0000 7f8e316ee640 1 -- start start 2024-04-25T09:55:53.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e316ee640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8e2c0b6af0 0x7f8e2c06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e316ee640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8e2c06c5b0 0x7f8e2c0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e316ee640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2c0cf110 con 0x7f8e2c1611e0 2024-04-25T09:55:53.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e316ee640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2c0d1fa0 con 0x7f8e2c06c5b0 2024-04-25T09:55:53.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e316ee640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2c0d2560 con 0x7f8e2c0b6af0 2024-04-25T09:55:53.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e2b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8e2c06c5b0 0x7f8e2c0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e2b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8e2c06c5b0 0x7f8e2c0d29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58274/0 (socket says 172.21.15.139:58274) 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e2b7fe640 1 -- 172.21.15.139:0/4061652028 learned_addr learned my addr 172.21.15.139:0/4061652028 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e2a7fc640 1 --2- 172.21.15.139:0/4061652028 >> v2:172.21.15.150:3300/0 conn(0x7f8e2c0b6af0 0x7f8e2c06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e2b7fe640 1 -- 172.21.15.139:0/4061652028 >> v2:172.21.15.150:3300/0 conn(0x7f8e2c0b6af0 msgr2=0x7f8e2c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e2b7fe640 1 --2- 172.21.15.139:0/4061652028 >> v2:172.21.15.150:3300/0 conn(0x7f8e2c0b6af0 0x7f8e2c06c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e2b7fe640 1 -- 172.21.15.139:0/4061652028 >> v1:172.21.15.150:6789/0 conn(0x7f8e2c1611e0 legacy=0x7f8e2c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e2b7fe640 1 -- 172.21.15.139:0/4061652028 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c0b6f50 con 0x7f8e2c06c5b0 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.418+0000 7f8e2a7fc640 1 --2- 172.21.15.139:0/4061652028 >> v2:172.21.15.150:3300/0 conn(0x7f8e2c0b6af0 0x7f8e2c06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.419+0000 7f8e2b7fe640 1 --2- 172.21.15.139:0/4061652028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e2c06c5b0 0x7f8e2c0d29f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0b3410 tx=0x7f8e2402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a7e008d82bc0c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.419+0000 7f8e29ffb640 1 -- 172.21.15.139:0/4061652028 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24037070 con 0x7f8e2c06c5b0 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.419+0000 7f8e29ffb640 1 -- 172.21.15.139:0/4061652028 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e24032440 con 0x7f8e2c06c5b0 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.419+0000 7f8e29ffb640 1 -- 172.21.15.139:0/4061652028 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24032810 con 0x7f8e2c06c5b0 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.420+0000 7f8e316ee640 1 -- 172.21.15.139:0/4061652028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e2c06c5b0 msgr2=0x7f8e2c0d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.420+0000 7f8e316ee640 1 --2- 172.21.15.139:0/4061652028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e2c06c5b0 0x7f8e2c0d29f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0b3410 tx=0x7f8e2402a230 comp rx=0 tx=0).stop 2024-04-25T09:55:53.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.420+0000 7f8e316ee640 1 -- 172.21.15.139:0/4061652028 shutdown_connections 2024-04-25T09:55:53.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.420+0000 7f8e316ee640 1 --2- 172.21.15.139:0/4061652028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e2c06c5b0 0x7f8e2c0d29f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.420+0000 7f8e316ee640 1 --2- 172.21.15.139:0/4061652028 >> v2:172.21.15.150:3300/0 conn(0x7f8e2c0b6af0 0x7f8e2c06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.420+0000 7f8e316ee640 1 -- 172.21.15.139:0/4061652028 >> 172.21.15.139:0/4061652028 conn(0x7f8e2c0b5b00 msgr2=0x7f8e2c0d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:53.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.420+0000 7f8e316ee640 1 -- 172.21.15.139:0/4061652028 shutdown_connections 2024-04-25T09:55:53.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.420+0000 7f8e316ee640 1 -- 172.21.15.139:0/4061652028 wait complete. 2024-04-25T09:55:53.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.421+0000 7f8e316ee640 1 Processor -- start 2024-04-25T09:55:53.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.421+0000 7f8e316ee640 1 -- start start 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e316ee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e2c06c5b0 0x7f8e2c1f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e316ee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e2c0b6af0 0x7f8e2c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e316ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e2c1611e0 0x7f8e2c0cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e316ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c0b2a50 con 0x7f8e2c1611e0 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e316ee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c0b2770 con 0x7f8e2c06c5b0 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e316ee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c0b2d30 con 0x7f8e2c0b6af0 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e2b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e2c1611e0 0x7f8e2c0cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e2b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e2c1611e0 0x7f8e2c0cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58286/0 (socket says 172.21.15.139:58286) 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e2b7fe640 1 -- 172.21.15.139:0/194328304 learned_addr learned my addr 172.21.15.139:0/194328304 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e2affd640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e2c06c5b0 0x7f8e2c1f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e2a7fc640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e2c0b6af0 0x7f8e2c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e2b7fe640 1 -- 172.21.15.139:0/194328304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e2c0b6af0 msgr2=0x7f8e2c155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e2b7fe640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e2c0b6af0 0x7f8e2c155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e2b7fe640 1 -- 172.21.15.139:0/194328304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e2c06c5b0 msgr2=0x7f8e2c1f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e2b7fe640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e2c06c5b0 0x7f8e2c1f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e2b7fe640 1 -- 172.21.15.139:0/194328304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c1581d0 con 0x7f8e2c1611e0 2024-04-25T09:55:53.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e2a7fc640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e2c0b6af0 0x7f8e2c155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:53.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.422+0000 7f8e2b7fe640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e2c1611e0 0x7f8e2c0cbdc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8e240090b0 tx=0x7f8e240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.423+0000 7f8e137fe640 1 -- 172.21.15.139:0/194328304 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24041070 con 0x7f8e2c1611e0 2024-04-25T09:55:53.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.423+0000 7f8e137fe640 1 -- 172.21.15.139:0/194328304 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2403c950 con 0x7f8e2c1611e0 2024-04-25T09:55:53.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.423+0000 7f8e316ee640 1 -- 172.21.15.139:0/194328304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2c1584b0 con 0x7f8e2c1611e0 2024-04-25T09:55:53.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.423+0000 7f8e137fe640 1 -- 172.21.15.139:0/194328304 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24037040 con 0x7f8e2c1611e0 2024-04-25T09:55:53.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.423+0000 7f8e316ee640 1 -- 172.21.15.139:0/194328304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2c15d1b0 con 0x7f8e2c1611e0 2024-04-25T09:55:53.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.424+0000 7f8e316ee640 1 -- 172.21.15.139:0/194328304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e2c074520 con 0x7f8e2c1611e0 2024-04-25T09:55:53.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.424+0000 7f8e137fe640 1 -- 172.21.15.139:0/194328304 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e24044070 con 0x7f8e2c1611e0 2024-04-25T09:55:53.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.424+0000 7f8e137fe640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8e1806b500 0x7f8e1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.425+0000 7f8e2affd640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8e1806b500 0x7f8e1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.425+0000 7f8e137fe640 1 -- 172.21.15.139:0/194328304 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8e240aadb0 con 0x7f8e2c1611e0 2024-04-25T09:55:53.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.425+0000 7f8e2affd640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8e1806b500 0x7f8e1806d9c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009fd0 tx=0x7f8e1c00d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.429+0000 7f8e137fe640 1 -- 172.21.15.139:0/194328304 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e240794e0 con 0x7f8e2c1611e0 2024-04-25T09:55:53.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.636+0000 7f8e316ee640 1 -- 172.21.15.139:0/194328304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e2c06ceb0 con 0x7f8e2c1611e0 2024-04-25T09:55:53.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.637+0000 7f8e137fe640 1 -- 172.21.15.139:0/194328304 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f8e2407d580 con 0x7f8e2c1611e0 2024-04-25T09:55:53.640 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:53.640 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/829268166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":829268166},{"type":"v1","addr":"172.21.15.150:6835","nonce":829268166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3075039902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3075039902},{"type":"v1","addr":"172.21.15.150:6834","nonce":3075039902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/627726069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":627726069},{"type":"v1","addr":"172.21.15.139:6836","nonce":627726069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:50.626292+0000","modified":"2024-04-25T09:55:52.634859+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/3541444834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3541444834},{"type":"v1","addr":"172.21.15.139:6837","nonce":3541444834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:55:53.640 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 28 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.639+0000 7f8e316ee640 1 -- 172.21.15.139:0/194328304 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8e1806b500 msgr2=0x7f8e1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.639+0000 7f8e316ee640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8e1806b500 0x7f8e1806d9c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009fd0 tx=0x7f8e1c00d040 comp rx=0 tx=0).stop 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.639+0000 7f8e316ee640 1 -- 172.21.15.139:0/194328304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e2c1611e0 msgr2=0x7f8e2c0cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.639+0000 7f8e316ee640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e2c1611e0 0x7f8e2c0cbdc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8e240090b0 tx=0x7f8e240046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.639+0000 7f8e316ee640 1 -- 172.21.15.139:0/194328304 shutdown_connections 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.639+0000 7f8e316ee640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e2c1611e0 0x7f8e2c0cbdc0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.639+0000 7f8e316ee640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8e1806b500 0x7f8e1806d9c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.639+0000 7f8e316ee640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e2c0b6af0 0x7f8e2c155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.639+0000 7f8e316ee640 1 --2- 172.21.15.139:0/194328304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e2c06c5b0 0x7f8e2c1f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.639+0000 7f8e316ee640 1 -- 172.21.15.139:0/194328304 >> 172.21.15.139:0/194328304 conn(0x7f8e2c0b5b00 msgr2=0x7f8e2c1615c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:53.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.639+0000 7f8e316ee640 1 -- 172.21.15.139:0/194328304 shutdown_connections 2024-04-25T09:55:53.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.639+0000 7f8e316ee640 1 -- 172.21.15.139:0/194328304 wait complete. 2024-04-25T09:55:53.653 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:53.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.747+0000 7fc125615640 1 Processor -- start 2024-04-25T09:55:53.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.747+0000 7fc125615640 1 -- start start 2024-04-25T09:55:53.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.748+0000 7fc125615640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc12015a860 0x7fc1200cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.748+0000 7fc125615640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12006ea20 0x7fc1200b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.748+0000 7fc125615640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1200af6d0 con 0x7fc1200cd1f0 2024-04-25T09:55:53.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.748+0000 7fc125615640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1200b02e0 con 0x7fc12015a860 2024-04-25T09:55:53.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.748+0000 7fc125615640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1200af9b0 con 0x7fc12006ea20 2024-04-25T09:55:53.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.748+0000 7fc11effd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc12015a860 0x7fc1200cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.748+0000 7fc11effd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc12015a860 0x7fc1200cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58292/0 (socket says 172.21.15.139:58292) 2024-04-25T09:55:53.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.748+0000 7fc11effd640 1 -- 172.21.15.139:0/2526074644 learned_addr learned my addr 172.21.15.139:0/2526074644 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:53.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.748+0000 7fc11effd640 1 -- 172.21.15.139:0/2526074644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12006ea20 msgr2=0x7fc1200b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:53.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.748+0000 7fc11f7fe640 1 --2- 172.21.15.139:0/2526074644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12006ea20 0x7fc1200b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.748+0000 7fc11effd640 1 --2- 172.21.15.139:0/2526074644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12006ea20 0x7fc1200b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.748+0000 7fc11effd640 1 -- 172.21.15.139:0/2526074644 >> v1:172.21.15.150:6789/0 conn(0x7fc1200cd1f0 legacy=0x7fc12006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:53.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.749+0000 7fc11effd640 1 -- 172.21.15.139:0/2526074644 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1200afc90 con 0x7fc12015a860 2024-04-25T09:55:53.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.749+0000 7fc11effd640 1 --2- 172.21.15.139:0/2526074644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc12015a860 0x7fc1200cccb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc110009870 tx=0x7fc11002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132bc812f209e3cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.749+0000 7fc11dffb640 1 -- 172.21.15.139:0/2526074644 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110037070 con 0x7fc12015a860 2024-04-25T09:55:53.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.749+0000 7fc11dffb640 1 -- 172.21.15.139:0/2526074644 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc110032440 con 0x7fc12015a860 2024-04-25T09:55:53.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.749+0000 7fc11dffb640 1 -- 172.21.15.139:0/2526074644 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110032810 con 0x7fc12015a860 2024-04-25T09:55:53.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.750+0000 7fc125615640 1 -- 172.21.15.139:0/2526074644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc12015a860 msgr2=0x7fc1200cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.750+0000 7fc125615640 1 --2- 172.21.15.139:0/2526074644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc12015a860 0x7fc1200cccb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc110009870 tx=0x7fc11002a210 comp rx=0 tx=0).stop 2024-04-25T09:55:53.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.750+0000 7fc125615640 1 -- 172.21.15.139:0/2526074644 shutdown_connections 2024-04-25T09:55:53.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.750+0000 7fc125615640 1 --2- 172.21.15.139:0/2526074644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12006ea20 0x7fc1200b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.750+0000 7fc125615640 1 --2- 172.21.15.139:0/2526074644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc12015a860 0x7fc1200cccb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.750+0000 7fc125615640 1 -- 172.21.15.139:0/2526074644 >> 172.21.15.139:0/2526074644 conn(0x7fc120066f20 msgr2=0x7fc12006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:53.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.750+0000 7fc125615640 1 -- 172.21.15.139:0/2526074644 shutdown_connections 2024-04-25T09:55:53.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.750+0000 7fc125615640 1 -- 172.21.15.139:0/2526074644 wait complete. 2024-04-25T09:55:53.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.751+0000 7fc125615640 1 Processor -- start 2024-04-25T09:55:53.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.751+0000 7fc125615640 1 -- start start 2024-04-25T09:55:53.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc125615640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12006ea20 0x7fc120142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc125615640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1200cd1f0 0x7fc1200addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc125615640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1201527a0 0x7fc120152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc125615640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1200af420 con 0x7fc1201527a0 2024-04-25T09:55:53.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc125615640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1200af2a0 con 0x7fc12006ea20 2024-04-25T09:55:53.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc125615640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12015ac40 con 0x7fc1200cd1f0 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc11f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1201527a0 0x7fc120152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc11f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1201527a0 0x7fc120152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58300/0 (socket says 172.21.15.139:58300) 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc11f7fe640 1 -- 172.21.15.139:0/4250056118 learned_addr learned my addr 172.21.15.139:0/4250056118 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc11effd640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12006ea20 0x7fc120142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc11e7fc640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1200cd1f0 0x7fc1200addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc11f7fe640 1 -- 172.21.15.139:0/4250056118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1200cd1f0 msgr2=0x7fc1200addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc11f7fe640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1200cd1f0 0x7fc1200addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc11f7fe640 1 -- 172.21.15.139:0/4250056118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12006ea20 msgr2=0x7fc120142d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc11f7fe640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12006ea20 0x7fc120142d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc11f7fe640 1 -- 172.21.15.139:0/4250056118 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc120146510 con 0x7fc1201527a0 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc11e7fc640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1200cd1f0 0x7fc1200addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.752+0000 7fc11effd640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12006ea20 0x7fc120142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.753+0000 7fc11f7fe640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1201527a0 0x7fc120152c50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc11800cd30 tx=0x7fc11800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.753+0000 7fc1077fe640 1 -- 172.21.15.139:0/4250056118 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc118003b90 con 0x7fc1201527a0 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.753+0000 7fc1077fe640 1 -- 172.21.15.139:0/4250056118 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc118003d50 con 0x7fc1201527a0 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.753+0000 7fc1077fe640 1 -- 172.21.15.139:0/4250056118 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc118004260 con 0x7fc1201527a0 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.753+0000 7fc125615640 1 -- 172.21.15.139:0/4250056118 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc120143240 con 0x7fc1201527a0 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.753+0000 7fc125615640 1 -- 172.21.15.139:0/4250056118 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1200ad2f0 con 0x7fc1201527a0 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.754+0000 7fc125615640 1 -- 172.21.15.139:0/4250056118 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc120074520 con 0x7fc1201527a0 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.754+0000 7fc1077fe640 1 -- 172.21.15.139:0/4250056118 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc118004400 con 0x7fc1201527a0 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.754+0000 7fc1077fe640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc10c06b4d0 0x7fc10c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.755+0000 7fc1077fe640 1 -- 172.21.15.139:0/4250056118 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc11808ae10 con 0x7fc1201527a0 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.755+0000 7fc11effd640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc10c06b4d0 0x7fc10c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:53.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.755+0000 7fc11effd640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc10c06b4d0 0x7fc10c06d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc110034000 tx=0x7fc11002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:53.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.759+0000 7fc1077fe640 1 -- 172.21.15.139:0/4250056118 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc118059bc0 con 0x7fc1201527a0 2024-04-25T09:55:53.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.930+0000 7fc125615640 1 -- 172.21.15.139:0/4250056118 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1201585c0 con 0x7fc1201527a0 2024-04-25T09:55:53.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.931+0000 7fc1077fe640 1 -- 172.21.15.139:0/4250056118 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16932 (secure 0 0 0) 0x7fc1180594c0 con 0x7fc1201527a0 2024-04-25T09:55:53.934 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:53.934 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":62,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:55:52.296181+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:48.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:49.597505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:53.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.933+0000 7fc125615640 1 -- 172.21.15.139:0/4250056118 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc10c06b4d0 msgr2=0x7fc10c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.933+0000 7fc125615640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc10c06b4d0 0x7fc10c06d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc110034000 tx=0x7fc11002e000 comp rx=0 tx=0).stop 2024-04-25T09:55:53.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.933+0000 7fc125615640 1 -- 172.21.15.139:0/4250056118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1201527a0 msgr2=0x7fc120152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:53.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.933+0000 7fc125615640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1201527a0 0x7fc120152c50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc11800cd30 tx=0x7fc11800a660 comp rx=0 tx=0).stop 2024-04-25T09:55:53.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.933+0000 7fc125615640 1 -- 172.21.15.139:0/4250056118 shutdown_connections 2024-04-25T09:55:53.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.933+0000 7fc125615640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1201527a0 0x7fc120152c50 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.933+0000 7fc125615640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc10c06b4d0 0x7fc10c06d990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.933+0000 7fc125615640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1200cd1f0 0x7fc1200addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.933+0000 7fc125615640 1 --2- 172.21.15.139:0/4250056118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12006ea20 0x7fc120142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:53.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.933+0000 7fc125615640 1 -- 172.21.15.139:0/4250056118 >> 172.21.15.139:0/4250056118 conn(0x7fc120066f20 msgr2=0x7fc12006e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:53.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.933+0000 7fc125615640 1 -- 172.21.15.139:0/4250056118 shutdown_connections 2024-04-25T09:55:53.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:53.933+0000 7fc125615640 1 -- 172.21.15.139:0/4250056118 wait complete. 2024-04-25T09:55:53.947 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:55:53.948 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:55:54.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.042+0000 7fd7be3be640 1 Processor -- start 2024-04-25T09:55:54.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.042+0000 7fd7be3be640 1 -- start start 2024-04-25T09:55:54.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.043+0000 7fd7be3be640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd7b8160e90 0x7fd7b80afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.043+0000 7fd7be3be640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7b806ab60 0x7fd7b806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.043+0000 7fd7be3be640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b80d0400 con 0x7fd7b806fa70 2024-04-25T09:55:54.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.043+0000 7fd7be3be640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b80d14a0 con 0x7fd7b8160e90 2024-04-25T09:55:54.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.043+0000 7fd7be3be640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80d06e0 con 0x7fd7b806ab60 2024-04-25T09:55:54.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.043+0000 7fd7b77fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7b806ab60 0x7fd7b806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.043+0000 7fd7b7fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd7b8160e90 0x7fd7b80afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.044+0000 7fd7b77fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7b806ab60 0x7fd7b806aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:49704/0 (socket says 172.21.15.139:49704) 2024-04-25T09:55:54.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.044+0000 7fd7b7fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd7b8160e90 0x7fd7b80afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:44100/0 (socket says 172.21.15.139:44100) 2024-04-25T09:55:54.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.044+0000 7fd7b77fe640 1 -- 172.21.15.139:0/3862081564 learned_addr learned my addr 172.21.15.139:0/3862081564 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:54.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.044+0000 7fd7bc95b640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fd7b806fa70 0x7fd7b806fec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:41834/0 (socket says 172.21.15.139:41834) 2024-04-25T09:55:54.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.044+0000 7fd7b6ffd640 1 -- 172.21.15.139:0/3862081564 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209700165 0 0) 0x7fd7b80d0400 con 0x7fd7b806fa70 2024-04-25T09:55:54.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.044+0000 7fd7b6ffd640 1 -- 172.21.15.139:0/3862081564 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a0003520 con 0x7fd7b806fa70 2024-04-25T09:55:54.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.044+0000 7fd7b77fe640 1 -- 172.21.15.139:0/3862081564 >> v2:172.21.15.150:3300/0 conn(0x7fd7b8160e90 msgr2=0x7fd7b80afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:54.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.044+0000 7fd7b77fe640 1 --2- 172.21.15.139:0/3862081564 >> v2:172.21.15.150:3300/0 conn(0x7fd7b8160e90 0x7fd7b80afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.044+0000 7fd7b77fe640 1 -- 172.21.15.139:0/3862081564 >> v1:172.21.15.150:6789/0 conn(0x7fd7b806fa70 legacy=0x7fd7b806fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.044+0000 7fd7b77fe640 1 -- 172.21.15.139:0/3862081564 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80d09c0 con 0x7fd7b806ab60 2024-04-25T09:55:54.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.044+0000 7fd7b7fff640 1 --2- 172.21.15.139:0/3862081564 >> v2:172.21.15.150:3300/0 conn(0x7fd7b8160e90 0x7fd7b80afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:54.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.045+0000 7fd7b77fe640 1 --2- 172.21.15.139:0/3862081564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7b806ab60 0x7fd7b806aff0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009870 tx=0x7fd7ac02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6257f7db716ca64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.045+0000 7fd7b6ffd640 1 -- 172.21.15.139:0/3862081564 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac037070 con 0x7fd7b806ab60 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.045+0000 7fd7b6ffd640 1 -- 172.21.15.139:0/3862081564 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac032440 con 0x7fd7b806ab60 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.045+0000 7fd7b6ffd640 1 -- 172.21.15.139:0/3862081564 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac032810 con 0x7fd7b806ab60 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.045+0000 7fd7be3be640 1 -- 172.21.15.139:0/3862081564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7b806ab60 msgr2=0x7fd7b806aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.045+0000 7fd7be3be640 1 --2- 172.21.15.139:0/3862081564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7b806ab60 0x7fd7b806aff0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009870 tx=0x7fd7ac02a270 comp rx=0 tx=0).stop 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.045+0000 7fd7be3be640 1 -- 172.21.15.139:0/3862081564 shutdown_connections 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.045+0000 7fd7be3be640 1 --2- 172.21.15.139:0/3862081564 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7b806ab60 0x7fd7b806aff0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.045+0000 7fd7be3be640 1 --2- 172.21.15.139:0/3862081564 >> v2:172.21.15.150:3300/0 conn(0x7fd7b8160e90 0x7fd7b80afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.045+0000 7fd7be3be640 1 -- 172.21.15.139:0/3862081564 >> 172.21.15.139:0/3862081564 conn(0x7fd7b80b23f0 msgr2=0x7fd7b80b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.045+0000 7fd7be3be640 1 -- 172.21.15.139:0/3862081564 shutdown_connections 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.045+0000 7fd7be3be640 1 -- 172.21.15.139:0/3862081564 wait complete. 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.046+0000 7fd7be3be640 1 Processor -- start 2024-04-25T09:55:54.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.047+0000 7fd7be3be640 1 -- start start 2024-04-25T09:55:54.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.047+0000 7fd7be3be640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7b806ab60 0x7fd7b8157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.047+0000 7fd7be3be640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7b806fa70 0x7fd7b80c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.047+0000 7fd7be3be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7b8160e90 0x7fd7b81f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.047+0000 7fd7be3be640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80b0000 con 0x7fd7b8160e90 2024-04-25T09:55:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.047+0000 7fd7be3be640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80d0f80 con 0x7fd7b806ab60 2024-04-25T09:55:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.047+0000 7fd7be3be640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b8161270 con 0x7fd7b806fa70 2024-04-25T09:55:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.047+0000 7fd7bc95b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7b8160e90 0x7fd7b81f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.047+0000 7fd7bc95b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7b8160e90 0x7fd7b81f6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58308/0 (socket says 172.21.15.139:58308) 2024-04-25T09:55:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7bc95b640 1 -- 172.21.15.139:0/3049496975 learned_addr learned my addr 172.21.15.139:0/3049496975 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7b7fff640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7b806ab60 0x7fd7b8157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7b77fe640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7b806fa70 0x7fd7b80c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7bc95b640 1 -- 172.21.15.139:0/3049496975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7b806fa70 msgr2=0x7fd7b80c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7bc95b640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7b806fa70 0x7fd7b80c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7bc95b640 1 -- 172.21.15.139:0/3049496975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7b806ab60 msgr2=0x7fd7b8157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7bc95b640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7b806ab60 0x7fd7b8157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7bc95b640 1 -- 172.21.15.139:0/3049496975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b8158de0 con 0x7fd7b8160e90 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7b7fff640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7b806ab60 0x7fd7b8157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7bc95b640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7b8160e90 0x7fd7b81f6fc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000c7c0 tx=0x7fd7b000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7b4ff9640 1 -- 172.21.15.139:0/3049496975 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0017070 con 0x7fd7b8160e90 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7b4ff9640 1 -- 172.21.15.139:0/3049496975 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b000a640 con 0x7fd7b8160e90 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7be3be640 1 -- 172.21.15.139:0/3049496975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b81f6ac0 con 0x7fd7b8160e90 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7b4ff9640 1 -- 172.21.15.139:0/3049496975 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b000aa10 con 0x7fd7b8160e90 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.048+0000 7fd7be3be640 1 -- 172.21.15.139:0/3049496975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b80ccbb0 con 0x7fd7b8160e90 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.049+0000 7fd7be3be640 1 -- 172.21.15.139:0/3049496975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b8074520 con 0x7fd7b8160e90 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.049+0000 7fd7b4ff9640 1 -- 172.21.15.139:0/3049496975 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7b001a070 con 0x7fd7b8160e90 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.050+0000 7fd7b4ff9640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7a406b4d0 0x7fd7a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.050+0000 7fd7b4ff9640 1 -- 172.21.15.139:0/3049496975 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7b00899e0 con 0x7fd7b8160e90 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.050+0000 7fd7b7fff640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7a406b4d0 0x7fd7a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.050+0000 7fd7b7fff640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7a406b4d0 0x7fd7a406d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80ced50 tx=0x7fd7a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:54.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.054+0000 7fd7b4ff9640 1 -- 172.21.15.139:0/3049496975 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7b8158de0 con 0x7fd7b8160e90 2024-04-25T09:55:54.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.222+0000 7fd7be3be640 1 -- 172.21.15.139:0/3049496975 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7b81577a0 con 0x7fd7a406b4d0 2024-04-25T09:55:54.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.228+0000 7fd7b4ff9640 1 -- 172.21.15.139:0/3049496975 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250954 (secure 0 0 0) 0x7fd7a8011980 con 0x7fd7a406b4d0 2024-04-25T09:55:54.233 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:54.235 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:55:54.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.230+0000 7fd7be3be640 1 -- 172.21.15.139:0/3049496975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7a406b4d0 msgr2=0x7fd7a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.230+0000 7fd7be3be640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7a406b4d0 0x7fd7a406d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80ced50 tx=0x7fd7a8008040 comp rx=0 tx=0).stop 2024-04-25T09:55:54.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.230+0000 7fd7be3be640 1 -- 172.21.15.139:0/3049496975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7b8160e90 msgr2=0x7fd7b81f6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.230+0000 7fd7be3be640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7b8160e90 0x7fd7b81f6fc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000c7c0 tx=0x7fd7b000cc80 comp rx=0 tx=0).stop 2024-04-25T09:55:54.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.231+0000 7fd7be3be640 1 -- 172.21.15.139:0/3049496975 shutdown_connections 2024-04-25T09:55:54.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.231+0000 7fd7be3be640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7b8160e90 0x7fd7b81f6fc0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.231+0000 7fd7be3be640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7a406b4d0 0x7fd7a406d990 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.231+0000 7fd7be3be640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7b806fa70 0x7fd7b80c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.231+0000 7fd7be3be640 1 --2- 172.21.15.139:0/3049496975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7b806ab60 0x7fd7b8157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.231+0000 7fd7be3be640 1 -- 172.21.15.139:0/3049496975 >> 172.21.15.139:0/3049496975 conn(0x7fd7b80b23f0 msgr2=0x7fd7b8161550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:54.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.231+0000 7fd7be3be640 1 -- 172.21.15.139:0/3049496975 shutdown_connections 2024-04-25T09:55:54.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.231+0000 7fd7be3be640 1 -- 172.21.15.139:0/3049496975 wait complete. 2024-04-25T09:55:54.244 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-04-25T09:55:53.000810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":223164,"kb_used_data":6972,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751556,"statfs":{"total":773094113280,"available":772865593344,"internally_reserved":0,"allocated":7139328,"data_stored":3817033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000728"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302671+0000","last_change":"2024-04-25T09:55:51.303411+0000","last_active":"2024-04-25T09:55:52.302671+0000","last_peered":"2024-04-25T09:55:52.302671+0000","last_clean":"2024-04-25T09:55:52.302671+0000","last_became_active":"2024-04-25T09:55:51.303129+0000","last_became_peered":"2024-04-25T09:55:51.303129+0000","last_unstale":"2024-04-25T09:55:52.302671+0000","last_undegraded":"2024-04-25T09:55:52.302671+0000","last_fullsized":"2024-04-25T09:55:52.302671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:35.354743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302461+0000","last_change":"2024-04-25T09:55:51.301262+0000","last_active":"2024-04-25T09:55:52.302461+0000","last_peered":"2024-04-25T09:55:52.302461+0000","last_clean":"2024-04-25T09:55:52.302461+0000","last_became_active":"2024-04-25T09:55:51.300989+0000","last_became_peered":"2024-04-25T09:55:51.300989+0000","last_unstale":"2024-04-25T09:55:52.302461+0000","last_undegraded":"2024-04-25T09:55:52.302461+0000","last_fullsized":"2024-04-25T09:55:52.302461+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:49.815961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322973+0000","last_change":"2024-04-25T09:55:51.321176+0000","last_active":"2024-04-25T09:55:51.322973+0000","last_peered":"2024-04-25T09:55:51.322973+0000","last_clean":"2024-04-25T09:55:51.322973+0000","last_became_active":"2024-04-25T09:55:51.319170+0000","last_became_peered":"2024-04-25T09:55:51.319170+0000","last_unstale":"2024-04-25T09:55:51.322973+0000","last_undegraded":"2024-04-25T09:55:51.322973+0000","last_fullsized":"2024-04-25T09:55:51.322973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305476+0000","last_change":"2024-04-25T09:55:51.303613+0000","last_active":"2024-04-25T09:55:51.305476+0000","last_peered":"2024-04-25T09:55:51.305476+0000","last_clean":"2024-04-25T09:55:51.305476+0000","last_became_active":"2024-04-25T09:55:51.303266+0000","last_became_peered":"2024-04-25T09:55:51.303266+0000","last_unstale":"2024-04-25T09:55:51.305476+0000","last_undegraded":"2024-04-25T09:55:51.305476+0000","last_fullsized":"2024-04-25T09:55:51.305476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:37.811562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321590+0000","last_change":"2024-04-25T09:55:51.317347+0000","last_active":"2024-04-25T09:55:51.321590+0000","last_peered":"2024-04-25T09:55:51.321590+0000","last_clean":"2024-04-25T09:55:51.321590+0000","last_became_active":"2024-04-25T09:55:51.316810+0000","last_became_peered":"2024-04-25T09:55:51.316810+0000","last_unstale":"2024-04-25T09:55:51.321590+0000","last_undegraded":"2024-04-25T09:55:51.321590+0000","last_fullsized":"2024-04-25T09:55:51.321590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:32.510934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.320186+0000","last_change":"2024-04-25T09:55:51.319541+0000","last_active":"2024-04-25T09:55:51.320186+0000","last_peered":"2024-04-25T09:55:51.320186+0000","last_clean":"2024-04-25T09:55:51.320186+0000","last_became_active":"2024-04-25T09:55:51.319218+0000","last_became_peered":"2024-04-25T09:55:51.319218+0000","last_unstale":"2024-04-25T09:55:51.320186+0000","last_undegraded":"2024-04-25T09:55:51.320186+0000","last_fullsized":"2024-04-25T09:55:51.320186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303121+0000","last_change":"2024-04-25T09:55:51.320026+0000","last_active":"2024-04-25T09:55:52.303121+0000","last_peered":"2024-04-25T09:55:52.303121+0000","last_clean":"2024-04-25T09:55:52.303121+0000","last_became_active":"2024-04-25T09:55:51.319656+0000","last_became_peered":"2024-04-25T09:55:51.319656+0000","last_unstale":"2024-04-25T09:55:52.303121+0000","last_undegraded":"2024-04-25T09:55:52.303121+0000","last_fullsized":"2024-04-25T09:55:52.303121+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.317650+0000","last_change":"2024-04-25T09:55:51.314164+0000","last_active":"2024-04-25T09:55:51.317650+0000","last_peered":"2024-04-25T09:55:51.317650+0000","last_clean":"2024-04-25T09:55:51.317650+0000","last_became_active":"2024-04-25T09:55:51.313888+0000","last_became_peered":"2024-04-25T09:55:51.313888+0000","last_unstale":"2024-04-25T09:55:51.317650+0000","last_undegraded":"2024-04-25T09:55:51.317650+0000","last_fullsized":"2024-04-25T09:55:51.317650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:12.483744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321807+0000","last_change":"2024-04-25T09:55:51.316804+0000","last_active":"2024-04-25T09:55:51.321807+0000","last_peered":"2024-04-25T09:55:51.321807+0000","last_clean":"2024-04-25T09:55:51.321807+0000","last_became_active":"2024-04-25T09:55:51.314608+0000","last_became_peered":"2024-04-25T09:55:51.314608+0000","last_unstale":"2024-04-25T09:55:51.321807+0000","last_undegraded":"2024-04-25T09:55:51.321807+0000","last_fullsized":"2024-04-25T09:55:51.321807+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:10.716067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.311232+0000","last_change":"2024-04-25T09:55:51.306747+0000","last_active":"2024-04-25T09:55:51.311232+0000","last_peered":"2024-04-25T09:55:51.311232+0000","last_clean":"2024-04-25T09:55:51.311232+0000","last_became_active":"2024-04-25T09:55:51.306393+0000","last_became_peered":"2024-04-25T09:55:51.306393+0000","last_unstale":"2024-04-25T09:55:51.311232+0000","last_undegraded":"2024-04-25T09:55:51.311232+0000","last_fullsized":"2024-04-25T09:55:51.311232+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.962859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303278+0000","last_change":"2024-04-25T09:55:51.311594+0000","last_active":"2024-04-25T09:55:52.303278+0000","last_peered":"2024-04-25T09:55:52.303278+0000","last_clean":"2024-04-25T09:55:52.303278+0000","last_became_active":"2024-04-25T09:55:51.311271+0000","last_became_peered":"2024-04-25T09:55:51.311271+0000","last_unstale":"2024-04-25T09:55:52.303278+0000","last_undegraded":"2024-04-25T09:55:52.303278+0000","last_fullsized":"2024-04-25T09:55:52.303278+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:44.151672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303327+0000","last_change":"2024-04-25T09:55:51.305962+0000","last_active":"2024-04-25T09:55:52.303327+0000","last_peered":"2024-04-25T09:55:52.303327+0000","last_clean":"2024-04-25T09:55:52.303327+0000","last_became_active":"2024-04-25T09:55:51.305232+0000","last_became_peered":"2024-04-25T09:55:51.305232+0000","last_unstale":"2024-04-25T09:55:52.303327+0000","last_undegraded":"2024-04-25T09:55:52.303327+0000","last_fullsized":"2024-04-25T09:55:52.303327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:26.442633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322877+0000","last_change":"2024-04-25T09:55:51.321119+0000","last_active":"2024-04-25T09:55:51.322877+0000","last_peered":"2024-04-25T09:55:51.322877+0000","last_clean":"2024-04-25T09:55:51.322877+0000","last_became_active":"2024-04-25T09:55:51.320822+0000","last_became_peered":"2024-04-25T09:55:51.320822+0000","last_unstale":"2024-04-25T09:55:51.322877+0000","last_undegraded":"2024-04-25T09:55:51.322877+0000","last_fullsized":"2024-04-25T09:55:51.322877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.318297+0000","last_change":"2024-04-25T09:55:51.313921+0000","last_active":"2024-04-25T09:55:51.318297+0000","last_peered":"2024-04-25T09:55:51.318297+0000","last_clean":"2024-04-25T09:55:51.318297+0000","last_became_active":"2024-04-25T09:55:51.313653+0000","last_became_peered":"2024-04-25T09:55:51.313653+0000","last_unstale":"2024-04-25T09:55:51.318297+0000","last_undegraded":"2024-04-25T09:55:51.318297+0000","last_fullsized":"2024-04-25T09:55:51.318297+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:11.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302647+0000","last_change":"2024-04-25T09:55:51.306904+0000","last_active":"2024-04-25T09:55:52.302647+0000","last_peered":"2024-04-25T09:55:52.302647+0000","last_clean":"2024-04-25T09:55:52.302647+0000","last_became_active":"2024-04-25T09:55:51.306564+0000","last_became_peered":"2024-04-25T09:55:51.306564+0000","last_unstale":"2024-04-25T09:55:52.302647+0000","last_undegraded":"2024-04-25T09:55:52.302647+0000","last_fullsized":"2024-04-25T09:55:52.302647+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:25.733756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305038+0000","last_change":"2024-04-25T09:55:51.297807+0000","last_active":"2024-04-25T09:55:51.305038+0000","last_peered":"2024-04-25T09:55:51.305038+0000","last_clean":"2024-04-25T09:55:51.305038+0000","last_became_active":"2024-04-25T09:55:51.297473+0000","last_became_peered":"2024-04-25T09:55:51.297473+0000","last_unstale":"2024-04-25T09:55:51.305038+0000","last_undegraded":"2024-04-25T09:55:51.305038+0000","last_fullsized":"2024-04-25T09:55:51.305038+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:14.381048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316239+0000","last_change":"2024-04-25T09:55:51.308066+0000","last_active":"2024-04-25T09:55:51.316239+0000","last_peered":"2024-04-25T09:55:51.316239+0000","last_clean":"2024-04-25T09:55:51.316239+0000","last_became_active":"2024-04-25T09:55:51.307793+0000","last_became_peered":"2024-04-25T09:55:51.307793+0000","last_unstale":"2024-04-25T09:55:51.316239+0000","last_undegraded":"2024-04-25T09:55:51.316239+0000","last_fullsized":"2024-04-25T09:55:51.316239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.962859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.319697+0000","last_change":"2024-04-25T09:55:51.318475+0000","last_active":"2024-04-25T09:55:51.319697+0000","last_peered":"2024-04-25T09:55:51.319697+0000","last_clean":"2024-04-25T09:55:51.319697+0000","last_became_active":"2024-04-25T09:55:51.318203+0000","last_became_peered":"2024-04-25T09:55:51.318203+0000","last_unstale":"2024-04-25T09:55:51.319697+0000","last_undegraded":"2024-04-25T09:55:51.319697+0000","last_fullsized":"2024-04-25T09:55:51.319697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:23.692837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303459+0000","last_change":"2024-04-25T09:55:51.306284+0000","last_active":"2024-04-25T09:55:52.303459+0000","last_peered":"2024-04-25T09:55:52.303459+0000","last_clean":"2024-04-25T09:55:52.303459+0000","last_became_active":"2024-04-25T09:55:51.305652+0000","last_became_peered":"2024-04-25T09:55:51.305652+0000","last_unstale":"2024-04-25T09:55:52.303459+0000","last_undegraded":"2024-04-25T09:55:52.303459+0000","last_fullsized":"2024-04-25T09:55:52.303459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316425+0000","last_change":"2024-04-25T09:55:51.315278+0000","last_active":"2024-04-25T09:55:51.316425+0000","last_peered":"2024-04-25T09:55:51.316425+0000","last_clean":"2024-04-25T09:55:51.316425+0000","last_became_active":"2024-04-25T09:55:51.314991+0000","last_became_peered":"2024-04-25T09:55:51.314991+0000","last_unstale":"2024-04-25T09:55:51.316425+0000","last_undegraded":"2024-04-25T09:55:51.316425+0000","last_fullsized":"2024-04-25T09:55:51.316425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:27.808147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316412+0000","last_change":"2024-04-25T09:55:51.315286+0000","last_active":"2024-04-25T09:55:51.316412+0000","last_peered":"2024-04-25T09:55:51.316412+0000","last_clean":"2024-04-25T09:55:51.316412+0000","last_became_active":"2024-04-25T09:55:51.315029+0000","last_became_peered":"2024-04-25T09:55:51.315029+0000","last_unstale":"2024-04-25T09:55:51.316412+0000","last_undegraded":"2024-04-25T09:55:51.316412+0000","last_fullsized":"2024-04-25T09:55:51.316412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:15.037916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323697+0000","last_change":"2024-04-25T09:55:51.322865+0000","last_active":"2024-04-25T09:55:51.323697+0000","last_peered":"2024-04-25T09:55:51.323697+0000","last_clean":"2024-04-25T09:55:51.323697+0000","last_became_active":"2024-04-25T09:55:51.321449+0000","last_became_peered":"2024-04-25T09:55:51.321449+0000","last_unstale":"2024-04-25T09:55:51.323697+0000","last_undegraded":"2024-04-25T09:55:51.323697+0000","last_fullsized":"2024-04-25T09:55:51.323697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:44.151672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303636+0000","last_change":"2024-04-25T09:55:51.309760+0000","last_active":"2024-04-25T09:55:52.303636+0000","last_peered":"2024-04-25T09:55:52.303636+0000","last_clean":"2024-04-25T09:55:52.303636+0000","last_became_active":"2024-04-25T09:55:51.309495+0000","last_became_peered":"2024-04-25T09:55:51.309495+0000","last_unstale":"2024-04-25T09:55:52.303636+0000","last_undegraded":"2024-04-25T09:55:52.303636+0000","last_fullsized":"2024-04-25T09:55:52.303636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:27.808147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303476+0000","last_change":"2024-04-25T09:55:51.308968+0000","last_active":"2024-04-25T09:55:52.303476+0000","last_peered":"2024-04-25T09:55:52.303476+0000","last_clean":"2024-04-25T09:55:52.303476+0000","last_became_active":"2024-04-25T09:55:51.307967+0000","last_became_peered":"2024-04-25T09:55:51.307967+0000","last_unstale":"2024-04-25T09:55:52.303476+0000","last_undegraded":"2024-04-25T09:55:52.303476+0000","last_fullsized":"2024-04-25T09:55:52.303476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:37.934348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304536+0000","last_change":"2024-04-25T09:55:51.298586+0000","last_active":"2024-04-25T09:55:51.304536+0000","last_peered":"2024-04-25T09:55:51.304536+0000","last_clean":"2024-04-25T09:55:51.304536+0000","last_became_active":"2024-04-25T09:55:51.298272+0000","last_became_peered":"2024-04-25T09:55:51.298272+0000","last_unstale":"2024-04-25T09:55:51.304536+0000","last_undegraded":"2024-04-25T09:55:51.304536+0000","last_fullsized":"2024-04-25T09:55:51.304536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:30.656241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302366+0000","last_change":"2024-04-25T09:55:51.318634+0000","last_active":"2024-04-25T09:55:52.302366+0000","last_peered":"2024-04-25T09:55:52.302366+0000","last_clean":"2024-04-25T09:55:52.302366+0000","last_became_active":"2024-04-25T09:55:51.318347+0000","last_became_peered":"2024-04-25T09:55:51.318347+0000","last_unstale":"2024-04-25T09:55:52.302366+0000","last_undegraded":"2024-04-25T09:55:52.302366+0000","last_fullsized":"2024-04-25T09:55:52.302366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:18.171353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.314075+0000","last_change":"2024-04-25T09:55:51.312167+0000","last_active":"2024-04-25T09:55:51.314075+0000","last_peered":"2024-04-25T09:55:51.314075+0000","last_clean":"2024-04-25T09:55:51.314075+0000","last_became_active":"2024-04-25T09:55:51.311785+0000","last_became_peered":"2024-04-25T09:55:51.311785+0000","last_unstale":"2024-04-25T09:55:51.314075+0000","last_undegraded":"2024-04-25T09:55:51.314075+0000","last_fullsized":"2024-04-25T09:55:51.314075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:53.653931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303192+0000","last_change":"2024-04-25T09:55:51.320446+0000","last_active":"2024-04-25T09:55:52.303192+0000","last_peered":"2024-04-25T09:55:52.303192+0000","last_clean":"2024-04-25T09:55:52.303192+0000","last_became_active":"2024-04-25T09:55:51.320112+0000","last_became_peered":"2024-04-25T09:55:51.320112+0000","last_unstale":"2024-04-25T09:55:52.303192+0000","last_undegraded":"2024-04-25T09:55:52.303192+0000","last_fullsized":"2024-04-25T09:55:52.303192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304965+0000","last_change":"2024-04-25T09:55:51.301513+0000","last_active":"2024-04-25T09:55:51.304965+0000","last_peered":"2024-04-25T09:55:51.304965+0000","last_clean":"2024-04-25T09:55:51.304965+0000","last_became_active":"2024-04-25T09:55:51.295610+0000","last_became_peered":"2024-04-25T09:55:51.295610+0000","last_unstale":"2024-04-25T09:55:51.304965+0000","last_undegraded":"2024-04-25T09:55:51.304965+0000","last_fullsized":"2024-04-25T09:55:51.304965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.757412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.303022+0000","last_change":"2024-04-25T09:55:51.300604+0000","last_active":"2024-04-25T09:55:51.303022+0000","last_peered":"2024-04-25T09:55:51.303022+0000","last_clean":"2024-04-25T09:55:51.303022+0000","last_became_active":"2024-04-25T09:55:51.300286+0000","last_became_peered":"2024-04-25T09:55:51.300286+0000","last_unstale":"2024-04-25T09:55:51.303022+0000","last_undegraded":"2024-04-25T09:55:51.303022+0000","last_fullsized":"2024-04-25T09:55:51.303022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316388+0000","last_change":"2024-04-25T09:55:51.310013+0000","last_active":"2024-04-25T09:55:51.316388+0000","last_peered":"2024-04-25T09:55:51.316388+0000","last_clean":"2024-04-25T09:55:51.316388+0000","last_became_active":"2024-04-25T09:55:51.309619+0000","last_became_peered":"2024-04-25T09:55:51.309619+0000","last_unstale":"2024-04-25T09:55:51.316388+0000","last_undegraded":"2024-04-25T09:55:51.316388+0000","last_fullsized":"2024-04-25T09:55:51.316388+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:30.874963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302699+0000","last_change":"2024-04-25T09:55:50.305070+0000","last_active":"2024-04-25T09:55:52.302699+0000","last_peered":"2024-04-25T09:55:52.302699+0000","last_clean":"2024-04-25T09:55:52.302699+0000","last_became_active":"2024-04-25T09:55:50.304749+0000","last_became_peered":"2024-04-25T09:55:50.304749+0000","last_unstale":"2024-04-25T09:55:52.302699+0000","last_undegraded":"2024-04-25T09:55:52.302699+0000","last_fullsized":"2024-04-25T09:55:52.302699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:43.049220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.309334+0000","last_change":"2024-04-25T09:55:51.305186+0000","last_active":"2024-04-25T09:55:51.309334+0000","last_peered":"2024-04-25T09:55:51.309334+0000","last_clean":"2024-04-25T09:55:51.309334+0000","last_became_active":"2024-04-25T09:55:51.304843+0000","last_became_peered":"2024-04-25T09:55:51.304843+0000","last_unstale":"2024-04-25T09:55:51.309334+0000","last_undegraded":"2024-04-25T09:55:51.309334+0000","last_fullsized":"2024-04-25T09:55:51.309334+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:46.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323347+0000","last_change":"2024-04-25T09:55:50.305062+0000","last_active":"2024-04-25T09:55:51.323347+0000","last_peered":"2024-04-25T09:55:51.323347+0000","last_clean":"2024-04-25T09:55:51.323347+0000","last_became_active":"2024-04-25T09:55:50.304760+0000","last_became_peered":"2024-04-25T09:55:50.304760+0000","last_unstale":"2024-04-25T09:55:51.323347+0000","last_undegraded":"2024-04-25T09:55:51.323347+0000","last_fullsized":"2024-04-25T09:55:51.323347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:10.306355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.308626+0000","last_change":"2024-04-25T09:55:51.307444+0000","last_active":"2024-04-25T09:55:51.308626+0000","last_peered":"2024-04-25T09:55:51.308626+0000","last_clean":"2024-04-25T09:55:51.308626+0000","last_became_active":"2024-04-25T09:55:51.307171+0000","last_became_peered":"2024-04-25T09:55:51.307171+0000","last_unstale":"2024-04-25T09:55:51.308626+0000","last_undegraded":"2024-04-25T09:55:51.308626+0000","last_fullsized":"2024-04-25T09:55:51.308626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:26.442633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.317020+0000","last_change":"2024-04-25T09:55:50.298308+0000","last_active":"2024-04-25T09:55:51.317020+0000","last_peered":"2024-04-25T09:55:51.317020+0000","last_clean":"2024-04-25T09:55:51.317020+0000","last_became_active":"2024-04-25T09:55:50.298018+0000","last_became_peered":"2024-04-25T09:55:50.298018+0000","last_unstale":"2024-04-25T09:55:51.317020+0000","last_undegraded":"2024-04-25T09:55:51.317020+0000","last_fullsized":"2024-04-25T09:55:51.317020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:52.646641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322753+0000","last_change":"2024-04-25T09:55:51.317521+0000","last_active":"2024-04-25T09:55:51.322753+0000","last_peered":"2024-04-25T09:55:51.322753+0000","last_clean":"2024-04-25T09:55:51.322753+0000","last_became_active":"2024-04-25T09:55:51.317265+0000","last_became_peered":"2024-04-25T09:55:51.317265+0000","last_unstale":"2024-04-25T09:55:51.322753+0000","last_undegraded":"2024-04-25T09:55:51.322753+0000","last_fullsized":"2024-04-25T09:55:51.322753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316812+0000","last_change":"2024-04-25T09:55:50.301119+0000","last_active":"2024-04-25T09:55:51.316812+0000","last_peered":"2024-04-25T09:55:51.316812+0000","last_clean":"2024-04-25T09:55:51.316812+0000","last_became_active":"2024-04-25T09:55:50.300321+0000","last_became_peered":"2024-04-25T09:55:50.300321+0000","last_unstale":"2024-04-25T09:55:51.316812+0000","last_undegraded":"2024-04-25T09:55:51.316812+0000","last_fullsized":"2024-04-25T09:55:51.316812+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:09.708778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.311448+0000","last_change":"2024-04-25T09:55:50.297259+0000","last_active":"2024-04-25T09:55:51.311448+0000","last_peered":"2024-04-25T09:55:51.311448+0000","last_clean":"2024-04-25T09:55:51.311448+0000","last_became_active":"2024-04-25T09:55:50.296982+0000","last_became_peered":"2024-04-25T09:55:50.296982+0000","last_unstale":"2024-04-25T09:55:51.311448+0000","last_undegraded":"2024-04-25T09:55:51.311448+0000","last_fullsized":"2024-04-25T09:55:51.311448+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:27.161990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302048+0000","last_change":"2024-04-25T09:55:51.309288+0000","last_active":"2024-04-25T09:55:52.302048+0000","last_peered":"2024-04-25T09:55:52.302048+0000","last_clean":"2024-04-25T09:55:52.302048+0000","last_became_active":"2024-04-25T09:55:51.309018+0000","last_became_peered":"2024-04-25T09:55:51.309018+0000","last_unstale":"2024-04-25T09:55:52.302048+0000","last_undegraded":"2024-04-25T09:55:52.302048+0000","last_fullsized":"2024-04-25T09:55:52.302048+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:47.578362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.310309+0000","last_change":"2024-04-25T09:55:50.298200+0000","last_active":"2024-04-25T09:55:51.310309+0000","last_peered":"2024-04-25T09:55:51.310309+0000","last_clean":"2024-04-25T09:55:51.310309+0000","last_became_active":"2024-04-25T09:55:50.297950+0000","last_became_peered":"2024-04-25T09:55:50.297950+0000","last_unstale":"2024-04-25T09:55:51.310309+0000","last_undegraded":"2024-04-25T09:55:51.310309+0000","last_fullsized":"2024-04-25T09:55:51.310309+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:50.309907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316534+0000","last_change":"2024-04-25T09:55:51.309251+0000","last_active":"2024-04-25T09:55:51.316534+0000","last_peered":"2024-04-25T09:55:51.316534+0000","last_clean":"2024-04-25T09:55:51.316534+0000","last_became_active":"2024-04-25T09:55:51.308805+0000","last_became_peered":"2024-04-25T09:55:51.308805+0000","last_unstale":"2024-04-25T09:55:51.316534+0000","last_undegraded":"2024-04-25T09:55:51.316534+0000","last_fullsized":"2024-04-25T09:55:51.316534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:21.995603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291503+0000","last_change":"2024-04-25T09:55:50.299777+0000","last_active":"2024-04-25T09:55:51.291503+0000","last_peered":"2024-04-25T09:55:51.291503+0000","last_clean":"2024-04-25T09:55:51.291503+0000","last_became_active":"2024-04-25T09:55:50.299522+0000","last_became_peered":"2024-04-25T09:55:50.299522+0000","last_unstale":"2024-04-25T09:55:51.291503+0000","last_undegraded":"2024-04-25T09:55:51.291503+0000","last_fullsized":"2024-04-25T09:55:51.291503+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:09.708778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.298022+0000","last_change":"2024-04-25T09:55:51.296452+0000","last_active":"2024-04-25T09:55:51.298022+0000","last_peered":"2024-04-25T09:55:51.298022+0000","last_clean":"2024-04-25T09:55:51.298022+0000","last_became_active":"2024-04-25T09:55:51.296189+0000","last_became_peered":"2024-04-25T09:55:51.296189+0000","last_unstale":"2024-04-25T09:55:51.298022+0000","last_undegraded":"2024-04-25T09:55:51.298022+0000","last_fullsized":"2024-04-25T09:55:51.298022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.296447+0000","last_change":"2024-04-25T09:55:50.305088+0000","last_active":"2024-04-25T09:55:51.296447+0000","last_peered":"2024-04-25T09:55:51.296447+0000","last_clean":"2024-04-25T09:55:51.296447+0000","last_became_active":"2024-04-25T09:55:50.304789+0000","last_became_peered":"2024-04-25T09:55:50.304789+0000","last_unstale":"2024-04-25T09:55:51.296447+0000","last_undegraded":"2024-04-25T09:55:51.296447+0000","last_fullsized":"2024-04-25T09:55:51.296447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:28.750122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302469+0000","last_change":"2024-04-25T09:55:51.295832+0000","last_active":"2024-04-25T09:55:52.302469+0000","last_peered":"2024-04-25T09:55:52.302469+0000","last_clean":"2024-04-25T09:55:52.302469+0000","last_became_active":"2024-04-25T09:55:51.295549+0000","last_became_peered":"2024-04-25T09:55:51.295549+0000","last_unstale":"2024-04-25T09:55:52.302469+0000","last_undegraded":"2024-04-25T09:55:52.302469+0000","last_fullsized":"2024-04-25T09:55:52.302469+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:31.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.308605+0000","last_change":"2024-04-25T09:55:50.307313+0000","last_active":"2024-04-25T09:55:51.308605+0000","last_peered":"2024-04-25T09:55:51.308605+0000","last_clean":"2024-04-25T09:55:51.308605+0000","last_became_active":"2024-04-25T09:55:50.306998+0000","last_became_peered":"2024-04-25T09:55:50.306998+0000","last_unstale":"2024-04-25T09:55:51.308605+0000","last_undegraded":"2024-04-25T09:55:51.308605+0000","last_fullsized":"2024-04-25T09:55:51.308605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:13.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305345+0000","last_change":"2024-04-25T09:55:51.302618+0000","last_active":"2024-04-25T09:55:51.305345+0000","last_peered":"2024-04-25T09:55:51.305345+0000","last_clean":"2024-04-25T09:55:51.305345+0000","last_became_active":"2024-04-25T09:55:51.301598+0000","last_became_peered":"2024-04-25T09:55:51.301598+0000","last_unstale":"2024-04-25T09:55:51.305345+0000","last_undegraded":"2024-04-25T09:55:51.305345+0000","last_fullsized":"2024-04-25T09:55:51.305345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:37.934348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.292155+0000","last_change":"2024-04-25T09:55:50.300463+0000","last_active":"2024-04-25T09:55:51.292155+0000","last_peered":"2024-04-25T09:55:51.292155+0000","last_clean":"2024-04-25T09:55:51.292155+0000","last_became_active":"2024-04-25T09:55:50.299576+0000","last_became_peered":"2024-04-25T09:55:50.299576+0000","last_unstale":"2024-04-25T09:55:51.292155+0000","last_undegraded":"2024-04-25T09:55:51.292155+0000","last_fullsized":"2024-04-25T09:55:51.292155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:56.828907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307569+0000","last_change":"2024-04-25T09:55:51.306375+0000","last_active":"2024-04-25T09:55:51.307569+0000","last_peered":"2024-04-25T09:55:51.307569+0000","last_clean":"2024-04-25T09:55:51.307569+0000","last_became_active":"2024-04-25T09:55:51.305748+0000","last_became_peered":"2024-04-25T09:55:51.305748+0000","last_unstale":"2024-04-25T09:55:51.307569+0000","last_undegraded":"2024-04-25T09:55:51.307569+0000","last_fullsized":"2024-04-25T09:55:51.307569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:37.811562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322267+0000","last_change":"2024-04-25T09:55:50.307971+0000","last_active":"2024-04-25T09:55:51.322267+0000","last_peered":"2024-04-25T09:55:51.322267+0000","last_clean":"2024-04-25T09:55:51.322267+0000","last_became_active":"2024-04-25T09:55:50.307650+0000","last_became_peered":"2024-04-25T09:55:50.307650+0000","last_unstale":"2024-04-25T09:55:51.322267+0000","last_undegraded":"2024-04-25T09:55:51.322267+0000","last_fullsized":"2024-04-25T09:55:51.322267+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:30.506756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305389+0000","last_change":"2024-04-25T09:55:51.302214+0000","last_active":"2024-04-25T09:55:51.305389+0000","last_peered":"2024-04-25T09:55:51.305389+0000","last_clean":"2024-04-25T09:55:51.305389+0000","last_became_active":"2024-04-25T09:55:51.301887+0000","last_became_peered":"2024-04-25T09:55:51.301887+0000","last_unstale":"2024-04-25T09:55:51.305389+0000","last_undegraded":"2024-04-25T09:55:51.305389+0000","last_fullsized":"2024-04-25T09:55:51.305389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.447273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302496+0000","last_change":"2024-04-25T09:55:50.299295+0000","last_active":"2024-04-25T09:55:52.302496+0000","last_peered":"2024-04-25T09:55:52.302496+0000","last_clean":"2024-04-25T09:55:52.302496+0000","last_became_active":"2024-04-25T09:55:50.296672+0000","last_became_peered":"2024-04-25T09:55:50.296672+0000","last_unstale":"2024-04-25T09:55:52.302496+0000","last_undegraded":"2024-04-25T09:55:52.302496+0000","last_fullsized":"2024-04-25T09:55:52.302496+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:05.248918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315783+0000","last_change":"2024-04-25T09:55:51.312207+0000","last_active":"2024-04-25T09:55:51.315783+0000","last_peered":"2024-04-25T09:55:51.315783+0000","last_clean":"2024-04-25T09:55:51.315783+0000","last_became_active":"2024-04-25T09:55:51.311902+0000","last_became_peered":"2024-04-25T09:55:51.311902+0000","last_unstale":"2024-04-25T09:55:51.315783+0000","last_undegraded":"2024-04-25T09:55:51.315783+0000","last_fullsized":"2024-04-25T09:55:51.315783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.335930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322307+0000","last_change":"2024-04-25T09:55:50.307406+0000","last_active":"2024-04-25T09:55:51.322307+0000","last_peered":"2024-04-25T09:55:51.322307+0000","last_clean":"2024-04-25T09:55:51.322307+0000","last_became_active":"2024-04-25T09:55:50.306864+0000","last_became_peered":"2024-04-25T09:55:50.306864+0000","last_unstale":"2024-04-25T09:55:51.322307+0000","last_undegraded":"2024-04-25T09:55:51.322307+0000","last_fullsized":"2024-04-25T09:55:51.322307+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:54.613991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303023+0000","last_change":"2024-04-25T09:55:51.306953+0000","last_active":"2024-04-25T09:55:52.303023+0000","last_peered":"2024-04-25T09:55:52.303023+0000","last_clean":"2024-04-25T09:55:52.303023+0000","last_became_active":"2024-04-25T09:55:51.306664+0000","last_became_peered":"2024-04-25T09:55:51.306664+0000","last_unstale":"2024-04-25T09:55:52.303023+0000","last_undegraded":"2024-04-25T09:55:52.303023+0000","last_fullsized":"2024-04-25T09:55:52.303023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303002+0000","last_change":"2024-04-25T09:55:50.303832+0000","last_active":"2024-04-25T09:55:52.303002+0000","last_peered":"2024-04-25T09:55:52.303002+0000","last_clean":"2024-04-25T09:55:52.303002+0000","last_became_active":"2024-04-25T09:55:50.303513+0000","last_became_peered":"2024-04-25T09:55:50.303513+0000","last_unstale":"2024-04-25T09:55:52.303002+0000","last_undegraded":"2024-04-25T09:55:52.303002+0000","last_fullsized":"2024-04-25T09:55:52.303002+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:16.796943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315474+0000","last_change":"2024-04-25T09:55:51.306576+0000","last_active":"2024-04-25T09:55:51.315474+0000","last_peered":"2024-04-25T09:55:51.315474+0000","last_clean":"2024-04-25T09:55:51.315474+0000","last_became_active":"2024-04-25T09:55:51.305692+0000","last_became_peered":"2024-04-25T09:55:51.305692+0000","last_unstale":"2024-04-25T09:55:51.315474+0000","last_undegraded":"2024-04-25T09:55:51.315474+0000","last_fullsized":"2024-04-25T09:55:51.315474+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:46.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304282+0000","last_change":"2024-04-25T09:55:50.307077+0000","last_active":"2024-04-25T09:55:51.304282+0000","last_peered":"2024-04-25T09:55:51.304282+0000","last_clean":"2024-04-25T09:55:51.304282+0000","last_became_active":"2024-04-25T09:55:50.306743+0000","last_became_peered":"2024-04-25T09:55:50.306743+0000","last_unstale":"2024-04-25T09:55:51.304282+0000","last_undegraded":"2024-04-25T09:55:51.304282+0000","last_fullsized":"2024-04-25T09:55:51.304282+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:34.347454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302240+0000","last_change":"2024-04-25T09:55:51.315727+0000","last_active":"2024-04-25T09:55:52.302240+0000","last_peered":"2024-04-25T09:55:52.302240+0000","last_clean":"2024-04-25T09:55:52.302240+0000","last_became_active":"2024-04-25T09:55:51.315426+0000","last_became_peered":"2024-04-25T09:55:51.315426+0000","last_unstale":"2024-04-25T09:55:52.302240+0000","last_undegraded":"2024-04-25T09:55:52.302240+0000","last_fullsized":"2024-04-25T09:55:52.302240+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:08.905576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"61'8","reported_seq":20,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.324184+0000","last_change":"2024-04-25T09:55:50.302489+0000","last_active":"2024-04-25T09:55:51.324184+0000","last_peered":"2024-04-25T09:55:51.324184+0000","last_clean":"2024-04-25T09:55:51.324184+0000","last_became_active":"2024-04-25T09:55:50.302222+0000","last_became_peered":"2024-04-25T09:55:50.302222+0000","last_unstale":"2024-04-25T09:55:51.324184+0000","last_undegraded":"2024-04-25T09:55:51.324184+0000","last_fullsized":"2024-04-25T09:55:51.324184+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:34.805327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.312652+0000","last_change":"2024-04-25T09:55:51.310660+0000","last_active":"2024-04-25T09:55:51.312652+0000","last_peered":"2024-04-25T09:55:51.312652+0000","last_clean":"2024-04-25T09:55:51.312652+0000","last_became_active":"2024-04-25T09:55:51.310407+0000","last_became_peered":"2024-04-25T09:55:51.310407+0000","last_unstale":"2024-04-25T09:55:51.312652+0000","last_undegraded":"2024-04-25T09:55:51.312652+0000","last_fullsized":"2024-04-25T09:55:51.312652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:41.794719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303710+0000","last_change":"2024-04-25T09:55:50.303136+0000","last_active":"2024-04-25T09:55:52.303710+0000","last_peered":"2024-04-25T09:55:52.303710+0000","last_clean":"2024-04-25T09:55:52.303710+0000","last_became_active":"2024-04-25T09:55:50.302805+0000","last_became_peered":"2024-04-25T09:55:50.302805+0000","last_unstale":"2024-04-25T09:55:52.303710+0000","last_undegraded":"2024-04-25T09:55:52.303710+0000","last_fullsized":"2024-04-25T09:55:52.303710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.898286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321648+0000","last_change":"2024-04-25T09:55:51.318231+0000","last_active":"2024-04-25T09:55:51.321648+0000","last_peered":"2024-04-25T09:55:51.321648+0000","last_clean":"2024-04-25T09:55:51.321648+0000","last_became_active":"2024-04-25T09:55:51.317964+0000","last_became_peered":"2024-04-25T09:55:51.317964+0000","last_unstale":"2024-04-25T09:55:51.321648+0000","last_undegraded":"2024-04-25T09:55:51.321648+0000","last_fullsized":"2024-04-25T09:55:51.321648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:01.234580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321558+0000","last_change":"2024-04-25T09:55:50.307310+0000","last_active":"2024-04-25T09:55:51.321558+0000","last_peered":"2024-04-25T09:55:51.321558+0000","last_clean":"2024-04-25T09:55:51.321558+0000","last_became_active":"2024-04-25T09:55:50.306877+0000","last_became_peered":"2024-04-25T09:55:50.306877+0000","last_unstale":"2024-04-25T09:55:51.321558+0000","last_undegraded":"2024-04-25T09:55:51.321558+0000","last_fullsized":"2024-04-25T09:55:51.321558+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.808671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321032+0000","last_change":"2024-04-25T09:55:51.319026+0000","last_active":"2024-04-25T09:55:51.321032+0000","last_peered":"2024-04-25T09:55:51.321032+0000","last_clean":"2024-04-25T09:55:51.321032+0000","last_became_active":"2024-04-25T09:55:51.318720+0000","last_became_peered":"2024-04-25T09:55:51.318720+0000","last_unstale":"2024-04-25T09:55:51.321032+0000","last_undegraded":"2024-04-25T09:55:51.321032+0000","last_fullsized":"2024-04-25T09:55:51.321032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:21.995603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323351+0000","last_change":"2024-04-25T09:55:50.309565+0000","last_active":"2024-04-25T09:55:51.323351+0000","last_peered":"2024-04-25T09:55:51.323351+0000","last_clean":"2024-04-25T09:55:51.323351+0000","last_became_active":"2024-04-25T09:55:50.309335+0000","last_became_peered":"2024-04-25T09:55:50.309335+0000","last_unstale":"2024-04-25T09:55:51.323351+0000","last_undegraded":"2024-04-25T09:55:51.323351+0000","last_fullsized":"2024-04-25T09:55:51.323351+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:24.726466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.306954+0000","last_change":"2024-04-25T09:55:51.304138+0000","last_active":"2024-04-25T09:55:51.306954+0000","last_peered":"2024-04-25T09:55:51.306954+0000","last_clean":"2024-04-25T09:55:51.306954+0000","last_became_active":"2024-04-25T09:55:51.303842+0000","last_became_peered":"2024-04-25T09:55:51.303842+0000","last_unstale":"2024-04-25T09:55:51.306954+0000","last_undegraded":"2024-04-25T09:55:51.306954+0000","last_fullsized":"2024-04-25T09:55:51.306954+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.447273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291578+0000","last_change":"2024-04-25T09:55:50.295568+0000","last_active":"2024-04-25T09:55:51.291578+0000","last_peered":"2024-04-25T09:55:51.291578+0000","last_clean":"2024-04-25T09:55:51.291578+0000","last_became_active":"2024-04-25T09:55:50.295151+0000","last_became_peered":"2024-04-25T09:55:50.295151+0000","last_unstale":"2024-04-25T09:55:51.291578+0000","last_undegraded":"2024-04-25T09:55:51.291578+0000","last_fullsized":"2024-04-25T09:55:51.291578+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:43.198244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.299612+0000","last_change":"2024-04-25T09:55:51.297690+0000","last_active":"2024-04-25T09:55:51.299612+0000","last_peered":"2024-04-25T09:55:51.299612+0000","last_clean":"2024-04-25T09:55:51.299612+0000","last_became_active":"2024-04-25T09:55:51.297427+0000","last_became_peered":"2024-04-25T09:55:51.297427+0000","last_unstale":"2024-04-25T09:55:51.299612+0000","last_undegraded":"2024-04-25T09:55:51.299612+0000","last_fullsized":"2024-04-25T09:55:51.299612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.318135+0000","last_change":"2024-04-25T09:55:50.299667+0000","last_active":"2024-04-25T09:55:51.318135+0000","last_peered":"2024-04-25T09:55:51.318135+0000","last_clean":"2024-04-25T09:55:51.318135+0000","last_became_active":"2024-04-25T09:55:50.298743+0000","last_became_peered":"2024-04-25T09:55:50.298743+0000","last_unstale":"2024-04-25T09:55:51.318135+0000","last_undegraded":"2024-04-25T09:55:51.318135+0000","last_fullsized":"2024-04-25T09:55:51.318135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:13.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303024+0000","last_change":"2024-04-25T09:55:51.307575+0000","last_active":"2024-04-25T09:55:52.303024+0000","last_peered":"2024-04-25T09:55:52.303024+0000","last_clean":"2024-04-25T09:55:52.303024+0000","last_became_active":"2024-04-25T09:55:51.307279+0000","last_became_peered":"2024-04-25T09:55:51.307279+0000","last_unstale":"2024-04-25T09:55:52.303024+0000","last_undegraded":"2024-04-25T09:55:52.303024+0000","last_fullsized":"2024-04-25T09:55:52.303024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:10.716067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303126+0000","last_change":"2024-04-25T09:55:50.300890+0000","last_active":"2024-04-25T09:55:52.303126+0000","last_peered":"2024-04-25T09:55:52.303126+0000","last_clean":"2024-04-25T09:55:52.303126+0000","last_became_active":"2024-04-25T09:55:50.300508+0000","last_became_peered":"2024-04-25T09:55:50.300508+0000","last_unstale":"2024-04-25T09:55:52.303126+0000","last_undegraded":"2024-04-25T09:55:52.303126+0000","last_fullsized":"2024-04-25T09:55:52.303126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.808671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316507+0000","last_change":"2024-04-25T09:55:51.309120+0000","last_active":"2024-04-25T09:55:51.316507+0000","last_peered":"2024-04-25T09:55:51.316507+0000","last_clean":"2024-04-25T09:55:51.316507+0000","last_became_active":"2024-04-25T09:55:51.308443+0000","last_became_peered":"2024-04-25T09:55:51.308443+0000","last_unstale":"2024-04-25T09:55:51.316507+0000","last_undegraded":"2024-04-25T09:55:51.316507+0000","last_fullsized":"2024-04-25T09:55:51.316507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.335930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.294489+0000","last_change":"2024-04-25T09:55:50.305023+0000","last_active":"2024-04-25T09:55:51.294489+0000","last_peered":"2024-04-25T09:55:51.294489+0000","last_clean":"2024-04-25T09:55:51.294489+0000","last_became_active":"2024-04-25T09:55:50.304744+0000","last_became_peered":"2024-04-25T09:55:50.304744+0000","last_unstale":"2024-04-25T09:55:51.294489+0000","last_undegraded":"2024-04-25T09:55:51.294489+0000","last_fullsized":"2024-04-25T09:55:51.294489+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.787430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307022+0000","last_change":"2024-04-25T09:55:51.304569+0000","last_active":"2024-04-25T09:55:51.307022+0000","last_peered":"2024-04-25T09:55:51.307022+0000","last_clean":"2024-04-25T09:55:51.307022+0000","last_became_active":"2024-04-25T09:55:51.304198+0000","last_became_peered":"2024-04-25T09:55:51.304198+0000","last_unstale":"2024-04-25T09:55:51.307022+0000","last_undegraded":"2024-04-25T09:55:51.307022+0000","last_fullsized":"2024-04-25T09:55:51.307022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303857+0000","last_change":"2024-04-25T09:55:50.305335+0000","last_active":"2024-04-25T09:55:52.303857+0000","last_peered":"2024-04-25T09:55:52.303857+0000","last_clean":"2024-04-25T09:55:52.303857+0000","last_became_active":"2024-04-25T09:55:50.304742+0000","last_became_peered":"2024-04-25T09:55:50.304742+0000","last_unstale":"2024-04-25T09:55:52.303857+0000","last_undegraded":"2024-04-25T09:55:52.303857+0000","last_fullsized":"2024-04-25T09:55:52.303857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:17.164064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"62'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305088+0000","last_change":"2024-04-25T09:55:50.299644+0000","last_active":"2024-04-25T09:55:52.305088+0000","last_peered":"2024-04-25T09:55:52.305088+0000","last_clean":"2024-04-25T09:55:52.305088+0000","last_became_active":"2024-04-25T09:55:50.299368+0000","last_became_peered":"2024-04-25T09:55:50.299368+0000","last_unstale":"2024-04-25T09:55:52.305088+0000","last_undegraded":"2024-04-25T09:55:52.305088+0000","last_fullsized":"2024-04-25T09:55:52.305088+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:46.571072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303546+0000","last_change":"2024-04-25T09:55:51.308785+0000","last_active":"2024-04-25T09:55:52.303546+0000","last_peered":"2024-04-25T09:55:52.303546+0000","last_clean":"2024-04-25T09:55:52.303546+0000","last_became_active":"2024-04-25T09:55:51.308476+0000","last_became_peered":"2024-04-25T09:55:51.308476+0000","last_unstale":"2024-04-25T09:55:52.303546+0000","last_undegraded":"2024-04-25T09:55:52.303546+0000","last_fullsized":"2024-04-25T09:55:52.303546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:15.037916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.295008+0000","last_change":"2024-04-25T09:55:50.298279+0000","last_active":"2024-04-25T09:55:51.295008+0000","last_peered":"2024-04-25T09:55:51.295008+0000","last_clean":"2024-04-25T09:55:51.295008+0000","last_became_active":"2024-04-25T09:55:50.298007+0000","last_became_peered":"2024-04-25T09:55:50.298007+0000","last_unstale":"2024-04-25T09:55:51.295008+0000","last_undegraded":"2024-04-25T09:55:51.295008+0000","last_fullsized":"2024-04-25T09:55:51.295008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:11.476455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321671+0000","last_change":"2024-04-25T09:55:51.318440+0000","last_active":"2024-04-25T09:55:51.321671+0000","last_peered":"2024-04-25T09:55:51.321671+0000","last_clean":"2024-04-25T09:55:51.321671+0000","last_became_active":"2024-04-25T09:55:51.318162+0000","last_became_peered":"2024-04-25T09:55:51.318162+0000","last_unstale":"2024-04-25T09:55:51.321671+0000","last_undegraded":"2024-04-25T09:55:51.321671+0000","last_fullsized":"2024-04-25T09:55:51.321671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:44.056510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302555+0000","last_change":"2024-04-25T09:55:50.296784+0000","last_active":"2024-04-25T09:55:52.302555+0000","last_peered":"2024-04-25T09:55:52.302555+0000","last_clean":"2024-04-25T09:55:52.302555+0000","last_became_active":"2024-04-25T09:55:50.296190+0000","last_became_peered":"2024-04-25T09:55:50.296190+0000","last_unstale":"2024-04-25T09:55:52.302555+0000","last_undegraded":"2024-04-25T09:55:52.302555+0000","last_fullsized":"2024-04-25T09:55:52.302555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:31.503645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316493+0000","last_change":"2024-04-25T09:55:51.310409+0000","last_active":"2024-04-25T09:55:51.316493+0000","last_peered":"2024-04-25T09:55:51.316493+0000","last_clean":"2024-04-25T09:55:51.316493+0000","last_became_active":"2024-04-25T09:55:51.310147+0000","last_became_peered":"2024-04-25T09:55:51.310147+0000","last_unstale":"2024-04-25T09:55:51.316493+0000","last_undegraded":"2024-04-25T09:55:51.316493+0000","last_fullsized":"2024-04-25T09:55:51.316493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:06.256207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.306556+0000","last_change":"2024-04-25T09:55:50.307476+0000","last_active":"2024-04-25T09:55:51.306556+0000","last_peered":"2024-04-25T09:55:51.306556+0000","last_clean":"2024-04-25T09:55:51.306556+0000","last_became_active":"2024-04-25T09:55:50.307206+0000","last_became_peered":"2024-04-25T09:55:50.307206+0000","last_unstale":"2024-04-25T09:55:51.306556+0000","last_undegraded":"2024-04-25T09:55:51.306556+0000","last_fullsized":"2024-04-25T09:55:51.306556+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:05.108894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315150+0000","last_change":"2024-04-25T09:55:51.313741+0000","last_active":"2024-04-25T09:55:51.315150+0000","last_peered":"2024-04-25T09:55:51.315150+0000","last_clean":"2024-04-25T09:55:51.315150+0000","last_became_active":"2024-04-25T09:55:51.313475+0000","last_became_peered":"2024-04-25T09:55:51.313475+0000","last_unstale":"2024-04-25T09:55:51.315150+0000","last_undegraded":"2024-04-25T09:55:51.315150+0000","last_fullsized":"2024-04-25T09:55:51.315150+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:07.129449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291688+0000","last_change":"2024-04-25T09:55:50.300744+0000","last_active":"2024-04-25T09:55:51.291688+0000","last_peered":"2024-04-25T09:55:51.291688+0000","last_clean":"2024-04-25T09:55:51.291688+0000","last_became_active":"2024-04-25T09:55:50.300415+0000","last_became_peered":"2024-04-25T09:55:50.300415+0000","last_unstale":"2024-04-25T09:55:51.291688+0000","last_undegraded":"2024-04-25T09:55:51.291688+0000","last_fullsized":"2024-04-25T09:55:51.291688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.144383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302118+0000","last_change":"2024-04-25T09:55:51.307744+0000","last_active":"2024-04-25T09:55:52.302118+0000","last_peered":"2024-04-25T09:55:52.302118+0000","last_clean":"2024-04-25T09:55:52.302118+0000","last_became_active":"2024-04-25T09:55:51.307427+0000","last_became_peered":"2024-04-25T09:55:51.307427+0000","last_unstale":"2024-04-25T09:55:52.302118+0000","last_undegraded":"2024-04-25T09:55:52.302118+0000","last_fullsized":"2024-04-25T09:55:52.302118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:17.804233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291814+0000","last_change":"2024-04-25T09:55:50.301696+0000","last_active":"2024-04-25T09:55:51.291814+0000","last_peered":"2024-04-25T09:55:51.291814+0000","last_clean":"2024-04-25T09:55:51.291814+0000","last_became_active":"2024-04-25T09:55:50.299321+0000","last_became_peered":"2024-04-25T09:55:50.299321+0000","last_unstale":"2024-04-25T09:55:51.291814+0000","last_undegraded":"2024-04-25T09:55:51.291814+0000","last_fullsized":"2024-04-25T09:55:51.291814+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:25.435344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.301997+0000","last_change":"2024-04-25T09:55:50.301123+0000","last_active":"2024-04-25T09:55:52.301997+0000","last_peered":"2024-04-25T09:55:52.301997+0000","last_clean":"2024-04-25T09:55:52.301997+0000","last_became_active":"2024-04-25T09:55:50.300861+0000","last_became_peered":"2024-04-25T09:55:50.300861+0000","last_unstale":"2024-04-25T09:55:52.301997+0000","last_undegraded":"2024-04-25T09:55:52.301997+0000","last_fullsized":"2024-04-25T09:55:52.301997+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:00.227291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316474+0000","last_change":"2024-04-25T09:55:51.309200+0000","last_active":"2024-04-25T09:55:51.316474+0000","last_peered":"2024-04-25T09:55:51.316474+0000","last_clean":"2024-04-25T09:55:51.316474+0000","last_became_active":"2024-04-25T09:55:51.308915+0000","last_became_peered":"2024-04-25T09:55:51.308915+0000","last_unstale":"2024-04-25T09:55:51.316474+0000","last_undegraded":"2024-04-25T09:55:51.316474+0000","last_fullsized":"2024-04-25T09:55:51.316474+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:07.129449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291165+0000","last_change":"2024-04-25T09:55:50.298052+0000","last_active":"2024-04-25T09:55:51.291165+0000","last_peered":"2024-04-25T09:55:51.291165+0000","last_clean":"2024-04-25T09:55:51.291165+0000","last_became_active":"2024-04-25T09:55:50.297769+0000","last_became_peered":"2024-04-25T09:55:50.297769+0000","last_unstale":"2024-04-25T09:55:51.291165+0000","last_undegraded":"2024-04-25T09:55:51.291165+0000","last_fullsized":"2024-04-25T09:55:51.291165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:11.476455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":244,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307824+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:55:51.307824+0000","last_peered":"2024-04-25T09:55:51.307824+0000","last_clean":"2024-04-25T09:55:51.307824+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:55:51.307824+0000","last_undegraded":"2024-04-25T09:55:51.307824+0000","last_fullsized":"2024-04-25T09:55:51.307824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":1347584,"data_stored":920910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":741376,"data_stored":329800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":753664,"data_stored":336480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":749568,"data_stored":331378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":737280,"data_stored":329766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":700416,"data_stored":309731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":778240,"data_stored":338834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1331200,"data_stored":920134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:55:54.249 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:55:54.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.378+0000 7f100bd02640 1 Processor -- start 2024-04-25T09:55:54.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.378+0000 7f100bd02640 1 -- start start 2024-04-25T09:55:54.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.379+0000 7f100bd02640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f100415ac40 0x7f100406e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.379+0000 7f100bd02640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f100406e850 0x7f10040b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.379+0000 7f100bd02640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10040af6d0 con 0x7f100415a860 2024-04-25T09:55:54.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.379+0000 7f100bd02640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10040b02e0 con 0x7f100406e850 2024-04-25T09:55:54.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.379+0000 7f100bd02640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10040af9b0 con 0x7f100415ac40 2024-04-25T09:55:54.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.379+0000 7f1009a9e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f100415a860 0x7f10040cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56324/0 (socket says 172.21.15.139:56324) 2024-04-25T09:55:54.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.379+0000 7f1009a9e640 1 -- 172.21.15.139:0/566561093 learned_addr learned my addr 172.21.15.139:0/566561093 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:54.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.379+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159904148 0 0) 0x7f10040af6d0 con 0x7f100415a860 2024-04-25T09:55:54.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.379+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff0003520 con 0x7f100415a860 2024-04-25T09:55:54.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f100929d640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f100415ac40 0x7f100406e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f100a29f640 1 --2- 172.21.15.139:0/566561093 >> v2:172.21.15.150:3300/0 conn(0x7f100406e850 0x7f10040b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3780191662 0 0) 0x7f0ff0003520 con 0x7f100415a860 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f100415ac40 msgr2=0x7f100406e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f100415ac40 0x7f100406e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 >> v2:172.21.15.150:3300/0 conn(0x7f100406e850 msgr2=0x7f10040b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 --2- 172.21.15.139:0/566561093 >> v2:172.21.15.150:3300/0 conn(0x7f100406e850 0x7f10040b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10040afc90 con 0x7f100415a860 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f10000019f0 con 0x7f100415a860 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 >> v1:172.21.15.139:6789/0 conn(0x7f100415a860 legacy=0x7f10040cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff00039b0 0x7f0ff0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ff0004800 0x7f0ff0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff00051c0 0x7f0ff000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040b02e0 con 0x7f0ff00039b0 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040af9b0 con 0x7f0ff0004800 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff000bf30 con 0x7f0ff00051c0 2024-04-25T09:55:54.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f100929d640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f100415ac40 0x7f100406e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1009a9e640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff00039b0 0x7f0ff0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f100a29f640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff00051c0 0x7f0ff000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f100929d640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ff0004800 0x7f0ff0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1009a9e640 1 -- 172.21.15.139:0/566561093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff00051c0 msgr2=0x7f0ff000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1009a9e640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff00051c0 0x7f0ff000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1009a9e640 1 -- 172.21.15.139:0/566561093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ff0004800 msgr2=0x7f0ff0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1009a9e640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ff0004800 0x7f0ff0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.380+0000 7f1009a9e640 1 -- 172.21.15.139:0/566561093 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0003520 con 0x7f0ff00039b0 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.381+0000 7f1009a9e640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff00039b0 0x7f0ff0003dd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f100000a8a0 tx=0x7f0ff0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7993a89433f47d61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.381+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100000a4c0 con 0x7f0ff00039b0 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.381+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100002f070 con 0x7f0ff00039b0 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.381+0000 7f1008a9c640 1 -- 172.21.15.139:0/566561093 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100002bd10 con 0x7f0ff00039b0 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.382+0000 7f100bd02640 1 -- 172.21.15.139:0/566561093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff00039b0 msgr2=0x7f0ff0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.382+0000 7f100bd02640 1 --2- 172.21.15.139:0/566561093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff00039b0 0x7f0ff0003dd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f100000a8a0 tx=0x7f0ff0005830 comp rx=0 tx=0).stop 2024-04-25T09:55:54.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.382+0000 7f100a29f640 1 -- 172.21.15.139:0/566561093 reap_dead start 2024-04-25T09:55:54.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.382+0000 7f100bd02640 1 -- 172.21.15.139:0/566561093 shutdown_connections 2024-04-25T09:55:54.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.382+0000 7f100bd02640 1 -- 172.21.15.139:0/566561093 >> 172.21.15.139:0/566561093 conn(0x7f1004066f20 msgr2=0x7f100406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:54.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.382+0000 7f100bd02640 1 -- 172.21.15.139:0/566561093 shutdown_connections 2024-04-25T09:55:54.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.382+0000 7f100bd02640 1 -- 172.21.15.139:0/566561093 wait complete. 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.383+0000 7f100bd02640 1 Processor -- start 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.383+0000 7f100bd02640 1 -- start start 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.383+0000 7f100bd02640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff00039b0 0x7f1004135ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.383+0000 7f100bd02640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1004155ea0 0x7f1004154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.383+0000 7f100bd02640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1004154550 0x7f1004150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.383+0000 7f100bd02640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040af2a0 con 0x7f0ff00039b0 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.383+0000 7f100bd02640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040cd3c0 con 0x7f1004154550 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.383+0000 7f100bd02640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10040af420 con 0x7f1004155ea0 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f1009a9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff00039b0 0x7f1004135ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f1009a9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff00039b0 0x7f1004135ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58326/0 (socket says 172.21.15.139:58326) 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f1009a9e640 1 -- 172.21.15.139:0/3572085574 learned_addr learned my addr 172.21.15.139:0/3572085574 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f100929d640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1004155ea0 0x7f1004154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f100a29f640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1004154550 0x7f1004150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f1009a9e640 1 -- 172.21.15.139:0/3572085574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1004155ea0 msgr2=0x7f1004154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f1009a9e640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1004155ea0 0x7f1004154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f1009a9e640 1 -- 172.21.15.139:0/3572085574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1004154550 msgr2=0x7f1004150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f1009a9e640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1004154550 0x7f1004150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f1009a9e640 1 -- 172.21.15.139:0/3572085574 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1004136400 con 0x7f0ff00039b0 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f1009a9e640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff00039b0 0x7f1004135ec0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f100000a9c0 tx=0x7f1000001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f0ffa7fc640 1 -- 172.21.15.139:0/3572085574 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000042030 con 0x7f0ff00039b0 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f0ffa7fc640 1 -- 172.21.15.139:0/3572085574 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1000004e30 con 0x7f0ff00039b0 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f100bd02640 1 -- 172.21.15.139:0/3572085574 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1004156360 con 0x7f0ff00039b0 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f0ffa7fc640 1 -- 172.21.15.139:0/3572085574 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100002bab0 con 0x7f0ff00039b0 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.384+0000 7f100bd02640 1 -- 172.21.15.139:0/3572085574 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1004156920 con 0x7f0ff00039b0 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.385+0000 7f100bd02640 1 -- 172.21.15.139:0/3572085574 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1004154960 con 0x7f0ff00039b0 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.386+0000 7f0ffa7fc640 1 -- 172.21.15.139:0/3572085574 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1000045070 con 0x7f0ff00039b0 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.386+0000 7f0ffa7fc640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fe806b500 0x7f0fe806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.386+0000 7f0ffa7fc640 1 -- 172.21.15.139:0/3572085574 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f10000ad010 con 0x7f0ff00039b0 2024-04-25T09:55:54.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.386+0000 7f100929d640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fe806b500 0x7f0fe806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.386+0000 7f100929d640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fe806b500 0x7f0fe806d9c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f10041f3de0 tx=0x7f0ff4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:54.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.390+0000 7f0ffa7fc640 1 -- 172.21.15.139:0/3572085574 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1004154960 con 0x7f0ff00039b0 2024-04-25T09:55:54.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.555+0000 7f100bd02640 1 -- 172.21.15.139:0/3572085574 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10040cd150 con 0x7f0fe806b500 2024-04-25T09:55:54.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.561+0000 7f0ffa7fc640 1 -- 172.21.15.139:0/3572085574 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250954 (secure 0 0 0) 0x7f10040cd150 con 0x7f0fe806b500 2024-04-25T09:55:54.567 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:54.568 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:55:54.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.564+0000 7f100bd02640 1 -- 172.21.15.139:0/3572085574 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fe806b500 msgr2=0x7f0fe806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.564+0000 7f100bd02640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fe806b500 0x7f0fe806d9c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f10041f3de0 tx=0x7f0ff4008040 comp rx=0 tx=0).stop 2024-04-25T09:55:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.564+0000 7f100bd02640 1 -- 172.21.15.139:0/3572085574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff00039b0 msgr2=0x7f1004135ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.564+0000 7f100bd02640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff00039b0 0x7f1004135ec0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f100000a9c0 tx=0x7f1000001dc0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.564+0000 7f100bd02640 1 -- 172.21.15.139:0/3572085574 shutdown_connections 2024-04-25T09:55:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.564+0000 7f100bd02640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fe806b500 0x7f0fe806d9c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.564+0000 7f100bd02640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1004154550 0x7f1004150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.564+0000 7f100bd02640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1004155ea0 0x7f1004154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.564+0000 7f100bd02640 1 --2- 172.21.15.139:0/3572085574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ff00039b0 0x7f1004135ec0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.564+0000 7f100bd02640 1 -- 172.21.15.139:0/3572085574 >> 172.21.15.139:0/3572085574 conn(0x7f1004066f20 msgr2=0x7f1004141c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.564+0000 7f100bd02640 1 -- 172.21.15.139:0/3572085574 shutdown_connections 2024-04-25T09:55:54.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.564+0000 7f100bd02640 1 -- 172.21.15.139:0/3572085574 wait complete. 2024-04-25T09:55:54.578 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-04-25T09:55:53.000810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":223164,"kb_used_data":6972,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751556,"statfs":{"total":773094113280,"available":772865593344,"internally_reserved":0,"allocated":7139328,"data_stored":3817033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000728"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302671+0000","last_change":"2024-04-25T09:55:51.303411+0000","last_active":"2024-04-25T09:55:52.302671+0000","last_peered":"2024-04-25T09:55:52.302671+0000","last_clean":"2024-04-25T09:55:52.302671+0000","last_became_active":"2024-04-25T09:55:51.303129+0000","last_became_peered":"2024-04-25T09:55:51.303129+0000","last_unstale":"2024-04-25T09:55:52.302671+0000","last_undegraded":"2024-04-25T09:55:52.302671+0000","last_fullsized":"2024-04-25T09:55:52.302671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:35.354743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302461+0000","last_change":"2024-04-25T09:55:51.301262+0000","last_active":"2024-04-25T09:55:52.302461+0000","last_peered":"2024-04-25T09:55:52.302461+0000","last_clean":"2024-04-25T09:55:52.302461+0000","last_became_active":"2024-04-25T09:55:51.300989+0000","last_became_peered":"2024-04-25T09:55:51.300989+0000","last_unstale":"2024-04-25T09:55:52.302461+0000","last_undegraded":"2024-04-25T09:55:52.302461+0000","last_fullsized":"2024-04-25T09:55:52.302461+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:49.815961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322973+0000","last_change":"2024-04-25T09:55:51.321176+0000","last_active":"2024-04-25T09:55:51.322973+0000","last_peered":"2024-04-25T09:55:51.322973+0000","last_clean":"2024-04-25T09:55:51.322973+0000","last_became_active":"2024-04-25T09:55:51.319170+0000","last_became_peered":"2024-04-25T09:55:51.319170+0000","last_unstale":"2024-04-25T09:55:51.322973+0000","last_undegraded":"2024-04-25T09:55:51.322973+0000","last_fullsized":"2024-04-25T09:55:51.322973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305476+0000","last_change":"2024-04-25T09:55:51.303613+0000","last_active":"2024-04-25T09:55:51.305476+0000","last_peered":"2024-04-25T09:55:51.305476+0000","last_clean":"2024-04-25T09:55:51.305476+0000","last_became_active":"2024-04-25T09:55:51.303266+0000","last_became_peered":"2024-04-25T09:55:51.303266+0000","last_unstale":"2024-04-25T09:55:51.305476+0000","last_undegraded":"2024-04-25T09:55:51.305476+0000","last_fullsized":"2024-04-25T09:55:51.305476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:37.811562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321590+0000","last_change":"2024-04-25T09:55:51.317347+0000","last_active":"2024-04-25T09:55:51.321590+0000","last_peered":"2024-04-25T09:55:51.321590+0000","last_clean":"2024-04-25T09:55:51.321590+0000","last_became_active":"2024-04-25T09:55:51.316810+0000","last_became_peered":"2024-04-25T09:55:51.316810+0000","last_unstale":"2024-04-25T09:55:51.321590+0000","last_undegraded":"2024-04-25T09:55:51.321590+0000","last_fullsized":"2024-04-25T09:55:51.321590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:32.510934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.320186+0000","last_change":"2024-04-25T09:55:51.319541+0000","last_active":"2024-04-25T09:55:51.320186+0000","last_peered":"2024-04-25T09:55:51.320186+0000","last_clean":"2024-04-25T09:55:51.320186+0000","last_became_active":"2024-04-25T09:55:51.319218+0000","last_became_peered":"2024-04-25T09:55:51.319218+0000","last_unstale":"2024-04-25T09:55:51.320186+0000","last_undegraded":"2024-04-25T09:55:51.320186+0000","last_fullsized":"2024-04-25T09:55:51.320186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303121+0000","last_change":"2024-04-25T09:55:51.320026+0000","last_active":"2024-04-25T09:55:52.303121+0000","last_peered":"2024-04-25T09:55:52.303121+0000","last_clean":"2024-04-25T09:55:52.303121+0000","last_became_active":"2024-04-25T09:55:51.319656+0000","last_became_peered":"2024-04-25T09:55:51.319656+0000","last_unstale":"2024-04-25T09:55:52.303121+0000","last_undegraded":"2024-04-25T09:55:52.303121+0000","last_fullsized":"2024-04-25T09:55:52.303121+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.317650+0000","last_change":"2024-04-25T09:55:51.314164+0000","last_active":"2024-04-25T09:55:51.317650+0000","last_peered":"2024-04-25T09:55:51.317650+0000","last_clean":"2024-04-25T09:55:51.317650+0000","last_became_active":"2024-04-25T09:55:51.313888+0000","last_became_peered":"2024-04-25T09:55:51.313888+0000","last_unstale":"2024-04-25T09:55:51.317650+0000","last_undegraded":"2024-04-25T09:55:51.317650+0000","last_fullsized":"2024-04-25T09:55:51.317650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:12.483744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321807+0000","last_change":"2024-04-25T09:55:51.316804+0000","last_active":"2024-04-25T09:55:51.321807+0000","last_peered":"2024-04-25T09:55:51.321807+0000","last_clean":"2024-04-25T09:55:51.321807+0000","last_became_active":"2024-04-25T09:55:51.314608+0000","last_became_peered":"2024-04-25T09:55:51.314608+0000","last_unstale":"2024-04-25T09:55:51.321807+0000","last_undegraded":"2024-04-25T09:55:51.321807+0000","last_fullsized":"2024-04-25T09:55:51.321807+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:10.716067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.311232+0000","last_change":"2024-04-25T09:55:51.306747+0000","last_active":"2024-04-25T09:55:51.311232+0000","last_peered":"2024-04-25T09:55:51.311232+0000","last_clean":"2024-04-25T09:55:51.311232+0000","last_became_active":"2024-04-25T09:55:51.306393+0000","last_became_peered":"2024-04-25T09:55:51.306393+0000","last_unstale":"2024-04-25T09:55:51.311232+0000","last_undegraded":"2024-04-25T09:55:51.311232+0000","last_fullsized":"2024-04-25T09:55:51.311232+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.962859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303278+0000","last_change":"2024-04-25T09:55:51.311594+0000","last_active":"2024-04-25T09:55:52.303278+0000","last_peered":"2024-04-25T09:55:52.303278+0000","last_clean":"2024-04-25T09:55:52.303278+0000","last_became_active":"2024-04-25T09:55:51.311271+0000","last_became_peered":"2024-04-25T09:55:51.311271+0000","last_unstale":"2024-04-25T09:55:52.303278+0000","last_undegraded":"2024-04-25T09:55:52.303278+0000","last_fullsized":"2024-04-25T09:55:52.303278+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:44.151672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303327+0000","last_change":"2024-04-25T09:55:51.305962+0000","last_active":"2024-04-25T09:55:52.303327+0000","last_peered":"2024-04-25T09:55:52.303327+0000","last_clean":"2024-04-25T09:55:52.303327+0000","last_became_active":"2024-04-25T09:55:51.305232+0000","last_became_peered":"2024-04-25T09:55:51.305232+0000","last_unstale":"2024-04-25T09:55:52.303327+0000","last_undegraded":"2024-04-25T09:55:52.303327+0000","last_fullsized":"2024-04-25T09:55:52.303327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:26.442633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322877+0000","last_change":"2024-04-25T09:55:51.321119+0000","last_active":"2024-04-25T09:55:51.322877+0000","last_peered":"2024-04-25T09:55:51.322877+0000","last_clean":"2024-04-25T09:55:51.322877+0000","last_became_active":"2024-04-25T09:55:51.320822+0000","last_became_peered":"2024-04-25T09:55:51.320822+0000","last_unstale":"2024-04-25T09:55:51.322877+0000","last_undegraded":"2024-04-25T09:55:51.322877+0000","last_fullsized":"2024-04-25T09:55:51.322877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.318297+0000","last_change":"2024-04-25T09:55:51.313921+0000","last_active":"2024-04-25T09:55:51.318297+0000","last_peered":"2024-04-25T09:55:51.318297+0000","last_clean":"2024-04-25T09:55:51.318297+0000","last_became_active":"2024-04-25T09:55:51.313653+0000","last_became_peered":"2024-04-25T09:55:51.313653+0000","last_unstale":"2024-04-25T09:55:51.318297+0000","last_undegraded":"2024-04-25T09:55:51.318297+0000","last_fullsized":"2024-04-25T09:55:51.318297+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:11.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302647+0000","last_change":"2024-04-25T09:55:51.306904+0000","last_active":"2024-04-25T09:55:52.302647+0000","last_peered":"2024-04-25T09:55:52.302647+0000","last_clean":"2024-04-25T09:55:52.302647+0000","last_became_active":"2024-04-25T09:55:51.306564+0000","last_became_peered":"2024-04-25T09:55:51.306564+0000","last_unstale":"2024-04-25T09:55:52.302647+0000","last_undegraded":"2024-04-25T09:55:52.302647+0000","last_fullsized":"2024-04-25T09:55:52.302647+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:25.733756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305038+0000","last_change":"2024-04-25T09:55:51.297807+0000","last_active":"2024-04-25T09:55:51.305038+0000","last_peered":"2024-04-25T09:55:51.305038+0000","last_clean":"2024-04-25T09:55:51.305038+0000","last_became_active":"2024-04-25T09:55:51.297473+0000","last_became_peered":"2024-04-25T09:55:51.297473+0000","last_unstale":"2024-04-25T09:55:51.305038+0000","last_undegraded":"2024-04-25T09:55:51.305038+0000","last_fullsized":"2024-04-25T09:55:51.305038+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:14.381048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316239+0000","last_change":"2024-04-25T09:55:51.308066+0000","last_active":"2024-04-25T09:55:51.316239+0000","last_peered":"2024-04-25T09:55:51.316239+0000","last_clean":"2024-04-25T09:55:51.316239+0000","last_became_active":"2024-04-25T09:55:51.307793+0000","last_became_peered":"2024-04-25T09:55:51.307793+0000","last_unstale":"2024-04-25T09:55:51.316239+0000","last_undegraded":"2024-04-25T09:55:51.316239+0000","last_fullsized":"2024-04-25T09:55:51.316239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.962859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.319697+0000","last_change":"2024-04-25T09:55:51.318475+0000","last_active":"2024-04-25T09:55:51.319697+0000","last_peered":"2024-04-25T09:55:51.319697+0000","last_clean":"2024-04-25T09:55:51.319697+0000","last_became_active":"2024-04-25T09:55:51.318203+0000","last_became_peered":"2024-04-25T09:55:51.318203+0000","last_unstale":"2024-04-25T09:55:51.319697+0000","last_undegraded":"2024-04-25T09:55:51.319697+0000","last_fullsized":"2024-04-25T09:55:51.319697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:23.692837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303459+0000","last_change":"2024-04-25T09:55:51.306284+0000","last_active":"2024-04-25T09:55:52.303459+0000","last_peered":"2024-04-25T09:55:52.303459+0000","last_clean":"2024-04-25T09:55:52.303459+0000","last_became_active":"2024-04-25T09:55:51.305652+0000","last_became_peered":"2024-04-25T09:55:51.305652+0000","last_unstale":"2024-04-25T09:55:52.303459+0000","last_undegraded":"2024-04-25T09:55:52.303459+0000","last_fullsized":"2024-04-25T09:55:52.303459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316425+0000","last_change":"2024-04-25T09:55:51.315278+0000","last_active":"2024-04-25T09:55:51.316425+0000","last_peered":"2024-04-25T09:55:51.316425+0000","last_clean":"2024-04-25T09:55:51.316425+0000","last_became_active":"2024-04-25T09:55:51.314991+0000","last_became_peered":"2024-04-25T09:55:51.314991+0000","last_unstale":"2024-04-25T09:55:51.316425+0000","last_undegraded":"2024-04-25T09:55:51.316425+0000","last_fullsized":"2024-04-25T09:55:51.316425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:27.808147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316412+0000","last_change":"2024-04-25T09:55:51.315286+0000","last_active":"2024-04-25T09:55:51.316412+0000","last_peered":"2024-04-25T09:55:51.316412+0000","last_clean":"2024-04-25T09:55:51.316412+0000","last_became_active":"2024-04-25T09:55:51.315029+0000","last_became_peered":"2024-04-25T09:55:51.315029+0000","last_unstale":"2024-04-25T09:55:51.316412+0000","last_undegraded":"2024-04-25T09:55:51.316412+0000","last_fullsized":"2024-04-25T09:55:51.316412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:15.037916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323697+0000","last_change":"2024-04-25T09:55:51.322865+0000","last_active":"2024-04-25T09:55:51.323697+0000","last_peered":"2024-04-25T09:55:51.323697+0000","last_clean":"2024-04-25T09:55:51.323697+0000","last_became_active":"2024-04-25T09:55:51.321449+0000","last_became_peered":"2024-04-25T09:55:51.321449+0000","last_unstale":"2024-04-25T09:55:51.323697+0000","last_undegraded":"2024-04-25T09:55:51.323697+0000","last_fullsized":"2024-04-25T09:55:51.323697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:44.151672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303636+0000","last_change":"2024-04-25T09:55:51.309760+0000","last_active":"2024-04-25T09:55:52.303636+0000","last_peered":"2024-04-25T09:55:52.303636+0000","last_clean":"2024-04-25T09:55:52.303636+0000","last_became_active":"2024-04-25T09:55:51.309495+0000","last_became_peered":"2024-04-25T09:55:51.309495+0000","last_unstale":"2024-04-25T09:55:52.303636+0000","last_undegraded":"2024-04-25T09:55:52.303636+0000","last_fullsized":"2024-04-25T09:55:52.303636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:27.808147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303476+0000","last_change":"2024-04-25T09:55:51.308968+0000","last_active":"2024-04-25T09:55:52.303476+0000","last_peered":"2024-04-25T09:55:52.303476+0000","last_clean":"2024-04-25T09:55:52.303476+0000","last_became_active":"2024-04-25T09:55:51.307967+0000","last_became_peered":"2024-04-25T09:55:51.307967+0000","last_unstale":"2024-04-25T09:55:52.303476+0000","last_undegraded":"2024-04-25T09:55:52.303476+0000","last_fullsized":"2024-04-25T09:55:52.303476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:37.934348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304536+0000","last_change":"2024-04-25T09:55:51.298586+0000","last_active":"2024-04-25T09:55:51.304536+0000","last_peered":"2024-04-25T09:55:51.304536+0000","last_clean":"2024-04-25T09:55:51.304536+0000","last_became_active":"2024-04-25T09:55:51.298272+0000","last_became_peered":"2024-04-25T09:55:51.298272+0000","last_unstale":"2024-04-25T09:55:51.304536+0000","last_undegraded":"2024-04-25T09:55:51.304536+0000","last_fullsized":"2024-04-25T09:55:51.304536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:30.656241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302366+0000","last_change":"2024-04-25T09:55:51.318634+0000","last_active":"2024-04-25T09:55:52.302366+0000","last_peered":"2024-04-25T09:55:52.302366+0000","last_clean":"2024-04-25T09:55:52.302366+0000","last_became_active":"2024-04-25T09:55:51.318347+0000","last_became_peered":"2024-04-25T09:55:51.318347+0000","last_unstale":"2024-04-25T09:55:52.302366+0000","last_undegraded":"2024-04-25T09:55:52.302366+0000","last_fullsized":"2024-04-25T09:55:52.302366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:18.171353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.314075+0000","last_change":"2024-04-25T09:55:51.312167+0000","last_active":"2024-04-25T09:55:51.314075+0000","last_peered":"2024-04-25T09:55:51.314075+0000","last_clean":"2024-04-25T09:55:51.314075+0000","last_became_active":"2024-04-25T09:55:51.311785+0000","last_became_peered":"2024-04-25T09:55:51.311785+0000","last_unstale":"2024-04-25T09:55:51.314075+0000","last_undegraded":"2024-04-25T09:55:51.314075+0000","last_fullsized":"2024-04-25T09:55:51.314075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:53.653931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303192+0000","last_change":"2024-04-25T09:55:51.320446+0000","last_active":"2024-04-25T09:55:52.303192+0000","last_peered":"2024-04-25T09:55:52.303192+0000","last_clean":"2024-04-25T09:55:52.303192+0000","last_became_active":"2024-04-25T09:55:51.320112+0000","last_became_peered":"2024-04-25T09:55:51.320112+0000","last_unstale":"2024-04-25T09:55:52.303192+0000","last_undegraded":"2024-04-25T09:55:52.303192+0000","last_fullsized":"2024-04-25T09:55:52.303192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304965+0000","last_change":"2024-04-25T09:55:51.301513+0000","last_active":"2024-04-25T09:55:51.304965+0000","last_peered":"2024-04-25T09:55:51.304965+0000","last_clean":"2024-04-25T09:55:51.304965+0000","last_became_active":"2024-04-25T09:55:51.295610+0000","last_became_peered":"2024-04-25T09:55:51.295610+0000","last_unstale":"2024-04-25T09:55:51.304965+0000","last_undegraded":"2024-04-25T09:55:51.304965+0000","last_fullsized":"2024-04-25T09:55:51.304965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.757412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.303022+0000","last_change":"2024-04-25T09:55:51.300604+0000","last_active":"2024-04-25T09:55:51.303022+0000","last_peered":"2024-04-25T09:55:51.303022+0000","last_clean":"2024-04-25T09:55:51.303022+0000","last_became_active":"2024-04-25T09:55:51.300286+0000","last_became_peered":"2024-04-25T09:55:51.300286+0000","last_unstale":"2024-04-25T09:55:51.303022+0000","last_undegraded":"2024-04-25T09:55:51.303022+0000","last_fullsized":"2024-04-25T09:55:51.303022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316388+0000","last_change":"2024-04-25T09:55:51.310013+0000","last_active":"2024-04-25T09:55:51.316388+0000","last_peered":"2024-04-25T09:55:51.316388+0000","last_clean":"2024-04-25T09:55:51.316388+0000","last_became_active":"2024-04-25T09:55:51.309619+0000","last_became_peered":"2024-04-25T09:55:51.309619+0000","last_unstale":"2024-04-25T09:55:51.316388+0000","last_undegraded":"2024-04-25T09:55:51.316388+0000","last_fullsized":"2024-04-25T09:55:51.316388+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:30.874963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302699+0000","last_change":"2024-04-25T09:55:50.305070+0000","last_active":"2024-04-25T09:55:52.302699+0000","last_peered":"2024-04-25T09:55:52.302699+0000","last_clean":"2024-04-25T09:55:52.302699+0000","last_became_active":"2024-04-25T09:55:50.304749+0000","last_became_peered":"2024-04-25T09:55:50.304749+0000","last_unstale":"2024-04-25T09:55:52.302699+0000","last_undegraded":"2024-04-25T09:55:52.302699+0000","last_fullsized":"2024-04-25T09:55:52.302699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:43.049220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.309334+0000","last_change":"2024-04-25T09:55:51.305186+0000","last_active":"2024-04-25T09:55:51.309334+0000","last_peered":"2024-04-25T09:55:51.309334+0000","last_clean":"2024-04-25T09:55:51.309334+0000","last_became_active":"2024-04-25T09:55:51.304843+0000","last_became_peered":"2024-04-25T09:55:51.304843+0000","last_unstale":"2024-04-25T09:55:51.309334+0000","last_undegraded":"2024-04-25T09:55:51.309334+0000","last_fullsized":"2024-04-25T09:55:51.309334+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:46.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323347+0000","last_change":"2024-04-25T09:55:50.305062+0000","last_active":"2024-04-25T09:55:51.323347+0000","last_peered":"2024-04-25T09:55:51.323347+0000","last_clean":"2024-04-25T09:55:51.323347+0000","last_became_active":"2024-04-25T09:55:50.304760+0000","last_became_peered":"2024-04-25T09:55:50.304760+0000","last_unstale":"2024-04-25T09:55:51.323347+0000","last_undegraded":"2024-04-25T09:55:51.323347+0000","last_fullsized":"2024-04-25T09:55:51.323347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:10.306355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.308626+0000","last_change":"2024-04-25T09:55:51.307444+0000","last_active":"2024-04-25T09:55:51.308626+0000","last_peered":"2024-04-25T09:55:51.308626+0000","last_clean":"2024-04-25T09:55:51.308626+0000","last_became_active":"2024-04-25T09:55:51.307171+0000","last_became_peered":"2024-04-25T09:55:51.307171+0000","last_unstale":"2024-04-25T09:55:51.308626+0000","last_undegraded":"2024-04-25T09:55:51.308626+0000","last_fullsized":"2024-04-25T09:55:51.308626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:26.442633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.317020+0000","last_change":"2024-04-25T09:55:50.298308+0000","last_active":"2024-04-25T09:55:51.317020+0000","last_peered":"2024-04-25T09:55:51.317020+0000","last_clean":"2024-04-25T09:55:51.317020+0000","last_became_active":"2024-04-25T09:55:50.298018+0000","last_became_peered":"2024-04-25T09:55:50.298018+0000","last_unstale":"2024-04-25T09:55:51.317020+0000","last_undegraded":"2024-04-25T09:55:51.317020+0000","last_fullsized":"2024-04-25T09:55:51.317020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:52.646641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322753+0000","last_change":"2024-04-25T09:55:51.317521+0000","last_active":"2024-04-25T09:55:51.322753+0000","last_peered":"2024-04-25T09:55:51.322753+0000","last_clean":"2024-04-25T09:55:51.322753+0000","last_became_active":"2024-04-25T09:55:51.317265+0000","last_became_peered":"2024-04-25T09:55:51.317265+0000","last_unstale":"2024-04-25T09:55:51.322753+0000","last_undegraded":"2024-04-25T09:55:51.322753+0000","last_fullsized":"2024-04-25T09:55:51.322753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316812+0000","last_change":"2024-04-25T09:55:50.301119+0000","last_active":"2024-04-25T09:55:51.316812+0000","last_peered":"2024-04-25T09:55:51.316812+0000","last_clean":"2024-04-25T09:55:51.316812+0000","last_became_active":"2024-04-25T09:55:50.300321+0000","last_became_peered":"2024-04-25T09:55:50.300321+0000","last_unstale":"2024-04-25T09:55:51.316812+0000","last_undegraded":"2024-04-25T09:55:51.316812+0000","last_fullsized":"2024-04-25T09:55:51.316812+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:09.708778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.311448+0000","last_change":"2024-04-25T09:55:50.297259+0000","last_active":"2024-04-25T09:55:51.311448+0000","last_peered":"2024-04-25T09:55:51.311448+0000","last_clean":"2024-04-25T09:55:51.311448+0000","last_became_active":"2024-04-25T09:55:50.296982+0000","last_became_peered":"2024-04-25T09:55:50.296982+0000","last_unstale":"2024-04-25T09:55:51.311448+0000","last_undegraded":"2024-04-25T09:55:51.311448+0000","last_fullsized":"2024-04-25T09:55:51.311448+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:27.161990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302048+0000","last_change":"2024-04-25T09:55:51.309288+0000","last_active":"2024-04-25T09:55:52.302048+0000","last_peered":"2024-04-25T09:55:52.302048+0000","last_clean":"2024-04-25T09:55:52.302048+0000","last_became_active":"2024-04-25T09:55:51.309018+0000","last_became_peered":"2024-04-25T09:55:51.309018+0000","last_unstale":"2024-04-25T09:55:52.302048+0000","last_undegraded":"2024-04-25T09:55:52.302048+0000","last_fullsized":"2024-04-25T09:55:52.302048+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:47.578362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.310309+0000","last_change":"2024-04-25T09:55:50.298200+0000","last_active":"2024-04-25T09:55:51.310309+0000","last_peered":"2024-04-25T09:55:51.310309+0000","last_clean":"2024-04-25T09:55:51.310309+0000","last_became_active":"2024-04-25T09:55:50.297950+0000","last_became_peered":"2024-04-25T09:55:50.297950+0000","last_unstale":"2024-04-25T09:55:51.310309+0000","last_undegraded":"2024-04-25T09:55:51.310309+0000","last_fullsized":"2024-04-25T09:55:51.310309+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:50.309907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316534+0000","last_change":"2024-04-25T09:55:51.309251+0000","last_active":"2024-04-25T09:55:51.316534+0000","last_peered":"2024-04-25T09:55:51.316534+0000","last_clean":"2024-04-25T09:55:51.316534+0000","last_became_active":"2024-04-25T09:55:51.308805+0000","last_became_peered":"2024-04-25T09:55:51.308805+0000","last_unstale":"2024-04-25T09:55:51.316534+0000","last_undegraded":"2024-04-25T09:55:51.316534+0000","last_fullsized":"2024-04-25T09:55:51.316534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:21.995603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291503+0000","last_change":"2024-04-25T09:55:50.299777+0000","last_active":"2024-04-25T09:55:51.291503+0000","last_peered":"2024-04-25T09:55:51.291503+0000","last_clean":"2024-04-25T09:55:51.291503+0000","last_became_active":"2024-04-25T09:55:50.299522+0000","last_became_peered":"2024-04-25T09:55:50.299522+0000","last_unstale":"2024-04-25T09:55:51.291503+0000","last_undegraded":"2024-04-25T09:55:51.291503+0000","last_fullsized":"2024-04-25T09:55:51.291503+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:09.708778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.298022+0000","last_change":"2024-04-25T09:55:51.296452+0000","last_active":"2024-04-25T09:55:51.298022+0000","last_peered":"2024-04-25T09:55:51.298022+0000","last_clean":"2024-04-25T09:55:51.298022+0000","last_became_active":"2024-04-25T09:55:51.296189+0000","last_became_peered":"2024-04-25T09:55:51.296189+0000","last_unstale":"2024-04-25T09:55:51.298022+0000","last_undegraded":"2024-04-25T09:55:51.298022+0000","last_fullsized":"2024-04-25T09:55:51.298022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.296447+0000","last_change":"2024-04-25T09:55:50.305088+0000","last_active":"2024-04-25T09:55:51.296447+0000","last_peered":"2024-04-25T09:55:51.296447+0000","last_clean":"2024-04-25T09:55:51.296447+0000","last_became_active":"2024-04-25T09:55:50.304789+0000","last_became_peered":"2024-04-25T09:55:50.304789+0000","last_unstale":"2024-04-25T09:55:51.296447+0000","last_undegraded":"2024-04-25T09:55:51.296447+0000","last_fullsized":"2024-04-25T09:55:51.296447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:28.750122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302469+0000","last_change":"2024-04-25T09:55:51.295832+0000","last_active":"2024-04-25T09:55:52.302469+0000","last_peered":"2024-04-25T09:55:52.302469+0000","last_clean":"2024-04-25T09:55:52.302469+0000","last_became_active":"2024-04-25T09:55:51.295549+0000","last_became_peered":"2024-04-25T09:55:51.295549+0000","last_unstale":"2024-04-25T09:55:52.302469+0000","last_undegraded":"2024-04-25T09:55:52.302469+0000","last_fullsized":"2024-04-25T09:55:52.302469+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:31.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.308605+0000","last_change":"2024-04-25T09:55:50.307313+0000","last_active":"2024-04-25T09:55:51.308605+0000","last_peered":"2024-04-25T09:55:51.308605+0000","last_clean":"2024-04-25T09:55:51.308605+0000","last_became_active":"2024-04-25T09:55:50.306998+0000","last_became_peered":"2024-04-25T09:55:50.306998+0000","last_unstale":"2024-04-25T09:55:51.308605+0000","last_undegraded":"2024-04-25T09:55:51.308605+0000","last_fullsized":"2024-04-25T09:55:51.308605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:13.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305345+0000","last_change":"2024-04-25T09:55:51.302618+0000","last_active":"2024-04-25T09:55:51.305345+0000","last_peered":"2024-04-25T09:55:51.305345+0000","last_clean":"2024-04-25T09:55:51.305345+0000","last_became_active":"2024-04-25T09:55:51.301598+0000","last_became_peered":"2024-04-25T09:55:51.301598+0000","last_unstale":"2024-04-25T09:55:51.305345+0000","last_undegraded":"2024-04-25T09:55:51.305345+0000","last_fullsized":"2024-04-25T09:55:51.305345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:37.934348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.292155+0000","last_change":"2024-04-25T09:55:50.300463+0000","last_active":"2024-04-25T09:55:51.292155+0000","last_peered":"2024-04-25T09:55:51.292155+0000","last_clean":"2024-04-25T09:55:51.292155+0000","last_became_active":"2024-04-25T09:55:50.299576+0000","last_became_peered":"2024-04-25T09:55:50.299576+0000","last_unstale":"2024-04-25T09:55:51.292155+0000","last_undegraded":"2024-04-25T09:55:51.292155+0000","last_fullsized":"2024-04-25T09:55:51.292155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:56.828907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307569+0000","last_change":"2024-04-25T09:55:51.306375+0000","last_active":"2024-04-25T09:55:51.307569+0000","last_peered":"2024-04-25T09:55:51.307569+0000","last_clean":"2024-04-25T09:55:51.307569+0000","last_became_active":"2024-04-25T09:55:51.305748+0000","last_became_peered":"2024-04-25T09:55:51.305748+0000","last_unstale":"2024-04-25T09:55:51.307569+0000","last_undegraded":"2024-04-25T09:55:51.307569+0000","last_fullsized":"2024-04-25T09:55:51.307569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:37.811562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322267+0000","last_change":"2024-04-25T09:55:50.307971+0000","last_active":"2024-04-25T09:55:51.322267+0000","last_peered":"2024-04-25T09:55:51.322267+0000","last_clean":"2024-04-25T09:55:51.322267+0000","last_became_active":"2024-04-25T09:55:50.307650+0000","last_became_peered":"2024-04-25T09:55:50.307650+0000","last_unstale":"2024-04-25T09:55:51.322267+0000","last_undegraded":"2024-04-25T09:55:51.322267+0000","last_fullsized":"2024-04-25T09:55:51.322267+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:30.506756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305389+0000","last_change":"2024-04-25T09:55:51.302214+0000","last_active":"2024-04-25T09:55:51.305389+0000","last_peered":"2024-04-25T09:55:51.305389+0000","last_clean":"2024-04-25T09:55:51.305389+0000","last_became_active":"2024-04-25T09:55:51.301887+0000","last_became_peered":"2024-04-25T09:55:51.301887+0000","last_unstale":"2024-04-25T09:55:51.305389+0000","last_undegraded":"2024-04-25T09:55:51.305389+0000","last_fullsized":"2024-04-25T09:55:51.305389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.447273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302496+0000","last_change":"2024-04-25T09:55:50.299295+0000","last_active":"2024-04-25T09:55:52.302496+0000","last_peered":"2024-04-25T09:55:52.302496+0000","last_clean":"2024-04-25T09:55:52.302496+0000","last_became_active":"2024-04-25T09:55:50.296672+0000","last_became_peered":"2024-04-25T09:55:50.296672+0000","last_unstale":"2024-04-25T09:55:52.302496+0000","last_undegraded":"2024-04-25T09:55:52.302496+0000","last_fullsized":"2024-04-25T09:55:52.302496+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:05.248918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315783+0000","last_change":"2024-04-25T09:55:51.312207+0000","last_active":"2024-04-25T09:55:51.315783+0000","last_peered":"2024-04-25T09:55:51.315783+0000","last_clean":"2024-04-25T09:55:51.315783+0000","last_became_active":"2024-04-25T09:55:51.311902+0000","last_became_peered":"2024-04-25T09:55:51.311902+0000","last_unstale":"2024-04-25T09:55:51.315783+0000","last_undegraded":"2024-04-25T09:55:51.315783+0000","last_fullsized":"2024-04-25T09:55:51.315783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.335930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322307+0000","last_change":"2024-04-25T09:55:50.307406+0000","last_active":"2024-04-25T09:55:51.322307+0000","last_peered":"2024-04-25T09:55:51.322307+0000","last_clean":"2024-04-25T09:55:51.322307+0000","last_became_active":"2024-04-25T09:55:50.306864+0000","last_became_peered":"2024-04-25T09:55:50.306864+0000","last_unstale":"2024-04-25T09:55:51.322307+0000","last_undegraded":"2024-04-25T09:55:51.322307+0000","last_fullsized":"2024-04-25T09:55:51.322307+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:54.613991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303023+0000","last_change":"2024-04-25T09:55:51.306953+0000","last_active":"2024-04-25T09:55:52.303023+0000","last_peered":"2024-04-25T09:55:52.303023+0000","last_clean":"2024-04-25T09:55:52.303023+0000","last_became_active":"2024-04-25T09:55:51.306664+0000","last_became_peered":"2024-04-25T09:55:51.306664+0000","last_unstale":"2024-04-25T09:55:52.303023+0000","last_undegraded":"2024-04-25T09:55:52.303023+0000","last_fullsized":"2024-04-25T09:55:52.303023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303002+0000","last_change":"2024-04-25T09:55:50.303832+0000","last_active":"2024-04-25T09:55:52.303002+0000","last_peered":"2024-04-25T09:55:52.303002+0000","last_clean":"2024-04-25T09:55:52.303002+0000","last_became_active":"2024-04-25T09:55:50.303513+0000","last_became_peered":"2024-04-25T09:55:50.303513+0000","last_unstale":"2024-04-25T09:55:52.303002+0000","last_undegraded":"2024-04-25T09:55:52.303002+0000","last_fullsized":"2024-04-25T09:55:52.303002+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:16.796943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315474+0000","last_change":"2024-04-25T09:55:51.306576+0000","last_active":"2024-04-25T09:55:51.315474+0000","last_peered":"2024-04-25T09:55:51.315474+0000","last_clean":"2024-04-25T09:55:51.315474+0000","last_became_active":"2024-04-25T09:55:51.305692+0000","last_became_peered":"2024-04-25T09:55:51.305692+0000","last_unstale":"2024-04-25T09:55:51.315474+0000","last_undegraded":"2024-04-25T09:55:51.315474+0000","last_fullsized":"2024-04-25T09:55:51.315474+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:46.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304282+0000","last_change":"2024-04-25T09:55:50.307077+0000","last_active":"2024-04-25T09:55:51.304282+0000","last_peered":"2024-04-25T09:55:51.304282+0000","last_clean":"2024-04-25T09:55:51.304282+0000","last_became_active":"2024-04-25T09:55:50.306743+0000","last_became_peered":"2024-04-25T09:55:50.306743+0000","last_unstale":"2024-04-25T09:55:51.304282+0000","last_undegraded":"2024-04-25T09:55:51.304282+0000","last_fullsized":"2024-04-25T09:55:51.304282+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:34.347454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302240+0000","last_change":"2024-04-25T09:55:51.315727+0000","last_active":"2024-04-25T09:55:52.302240+0000","last_peered":"2024-04-25T09:55:52.302240+0000","last_clean":"2024-04-25T09:55:52.302240+0000","last_became_active":"2024-04-25T09:55:51.315426+0000","last_became_peered":"2024-04-25T09:55:51.315426+0000","last_unstale":"2024-04-25T09:55:52.302240+0000","last_undegraded":"2024-04-25T09:55:52.302240+0000","last_fullsized":"2024-04-25T09:55:52.302240+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:08.905576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"61'8","reported_seq":20,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.324184+0000","last_change":"2024-04-25T09:55:50.302489+0000","last_active":"2024-04-25T09:55:51.324184+0000","last_peered":"2024-04-25T09:55:51.324184+0000","last_clean":"2024-04-25T09:55:51.324184+0000","last_became_active":"2024-04-25T09:55:50.302222+0000","last_became_peered":"2024-04-25T09:55:50.302222+0000","last_unstale":"2024-04-25T09:55:51.324184+0000","last_undegraded":"2024-04-25T09:55:51.324184+0000","last_fullsized":"2024-04-25T09:55:51.324184+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:34.805327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.312652+0000","last_change":"2024-04-25T09:55:51.310660+0000","last_active":"2024-04-25T09:55:51.312652+0000","last_peered":"2024-04-25T09:55:51.312652+0000","last_clean":"2024-04-25T09:55:51.312652+0000","last_became_active":"2024-04-25T09:55:51.310407+0000","last_became_peered":"2024-04-25T09:55:51.310407+0000","last_unstale":"2024-04-25T09:55:51.312652+0000","last_undegraded":"2024-04-25T09:55:51.312652+0000","last_fullsized":"2024-04-25T09:55:51.312652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:41.794719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303710+0000","last_change":"2024-04-25T09:55:50.303136+0000","last_active":"2024-04-25T09:55:52.303710+0000","last_peered":"2024-04-25T09:55:52.303710+0000","last_clean":"2024-04-25T09:55:52.303710+0000","last_became_active":"2024-04-25T09:55:50.302805+0000","last_became_peered":"2024-04-25T09:55:50.302805+0000","last_unstale":"2024-04-25T09:55:52.303710+0000","last_undegraded":"2024-04-25T09:55:52.303710+0000","last_fullsized":"2024-04-25T09:55:52.303710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.898286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321648+0000","last_change":"2024-04-25T09:55:51.318231+0000","last_active":"2024-04-25T09:55:51.321648+0000","last_peered":"2024-04-25T09:55:51.321648+0000","last_clean":"2024-04-25T09:55:51.321648+0000","last_became_active":"2024-04-25T09:55:51.317964+0000","last_became_peered":"2024-04-25T09:55:51.317964+0000","last_unstale":"2024-04-25T09:55:51.321648+0000","last_undegraded":"2024-04-25T09:55:51.321648+0000","last_fullsized":"2024-04-25T09:55:51.321648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:01.234580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321558+0000","last_change":"2024-04-25T09:55:50.307310+0000","last_active":"2024-04-25T09:55:51.321558+0000","last_peered":"2024-04-25T09:55:51.321558+0000","last_clean":"2024-04-25T09:55:51.321558+0000","last_became_active":"2024-04-25T09:55:50.306877+0000","last_became_peered":"2024-04-25T09:55:50.306877+0000","last_unstale":"2024-04-25T09:55:51.321558+0000","last_undegraded":"2024-04-25T09:55:51.321558+0000","last_fullsized":"2024-04-25T09:55:51.321558+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.808671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321032+0000","last_change":"2024-04-25T09:55:51.319026+0000","last_active":"2024-04-25T09:55:51.321032+0000","last_peered":"2024-04-25T09:55:51.321032+0000","last_clean":"2024-04-25T09:55:51.321032+0000","last_became_active":"2024-04-25T09:55:51.318720+0000","last_became_peered":"2024-04-25T09:55:51.318720+0000","last_unstale":"2024-04-25T09:55:51.321032+0000","last_undegraded":"2024-04-25T09:55:51.321032+0000","last_fullsized":"2024-04-25T09:55:51.321032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:21.995603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323351+0000","last_change":"2024-04-25T09:55:50.309565+0000","last_active":"2024-04-25T09:55:51.323351+0000","last_peered":"2024-04-25T09:55:51.323351+0000","last_clean":"2024-04-25T09:55:51.323351+0000","last_became_active":"2024-04-25T09:55:50.309335+0000","last_became_peered":"2024-04-25T09:55:50.309335+0000","last_unstale":"2024-04-25T09:55:51.323351+0000","last_undegraded":"2024-04-25T09:55:51.323351+0000","last_fullsized":"2024-04-25T09:55:51.323351+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:24.726466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.306954+0000","last_change":"2024-04-25T09:55:51.304138+0000","last_active":"2024-04-25T09:55:51.306954+0000","last_peered":"2024-04-25T09:55:51.306954+0000","last_clean":"2024-04-25T09:55:51.306954+0000","last_became_active":"2024-04-25T09:55:51.303842+0000","last_became_peered":"2024-04-25T09:55:51.303842+0000","last_unstale":"2024-04-25T09:55:51.306954+0000","last_undegraded":"2024-04-25T09:55:51.306954+0000","last_fullsized":"2024-04-25T09:55:51.306954+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.447273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291578+0000","last_change":"2024-04-25T09:55:50.295568+0000","last_active":"2024-04-25T09:55:51.291578+0000","last_peered":"2024-04-25T09:55:51.291578+0000","last_clean":"2024-04-25T09:55:51.291578+0000","last_became_active":"2024-04-25T09:55:50.295151+0000","last_became_peered":"2024-04-25T09:55:50.295151+0000","last_unstale":"2024-04-25T09:55:51.291578+0000","last_undegraded":"2024-04-25T09:55:51.291578+0000","last_fullsized":"2024-04-25T09:55:51.291578+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:43.198244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.299612+0000","last_change":"2024-04-25T09:55:51.297690+0000","last_active":"2024-04-25T09:55:51.299612+0000","last_peered":"2024-04-25T09:55:51.299612+0000","last_clean":"2024-04-25T09:55:51.299612+0000","last_became_active":"2024-04-25T09:55:51.297427+0000","last_became_peered":"2024-04-25T09:55:51.297427+0000","last_unstale":"2024-04-25T09:55:51.299612+0000","last_undegraded":"2024-04-25T09:55:51.299612+0000","last_fullsized":"2024-04-25T09:55:51.299612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.318135+0000","last_change":"2024-04-25T09:55:50.299667+0000","last_active":"2024-04-25T09:55:51.318135+0000","last_peered":"2024-04-25T09:55:51.318135+0000","last_clean":"2024-04-25T09:55:51.318135+0000","last_became_active":"2024-04-25T09:55:50.298743+0000","last_became_peered":"2024-04-25T09:55:50.298743+0000","last_unstale":"2024-04-25T09:55:51.318135+0000","last_undegraded":"2024-04-25T09:55:51.318135+0000","last_fullsized":"2024-04-25T09:55:51.318135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:13.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303024+0000","last_change":"2024-04-25T09:55:51.307575+0000","last_active":"2024-04-25T09:55:52.303024+0000","last_peered":"2024-04-25T09:55:52.303024+0000","last_clean":"2024-04-25T09:55:52.303024+0000","last_became_active":"2024-04-25T09:55:51.307279+0000","last_became_peered":"2024-04-25T09:55:51.307279+0000","last_unstale":"2024-04-25T09:55:52.303024+0000","last_undegraded":"2024-04-25T09:55:52.303024+0000","last_fullsized":"2024-04-25T09:55:52.303024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:10.716067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303126+0000","last_change":"2024-04-25T09:55:50.300890+0000","last_active":"2024-04-25T09:55:52.303126+0000","last_peered":"2024-04-25T09:55:52.303126+0000","last_clean":"2024-04-25T09:55:52.303126+0000","last_became_active":"2024-04-25T09:55:50.300508+0000","last_became_peered":"2024-04-25T09:55:50.300508+0000","last_unstale":"2024-04-25T09:55:52.303126+0000","last_undegraded":"2024-04-25T09:55:52.303126+0000","last_fullsized":"2024-04-25T09:55:52.303126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.808671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316507+0000","last_change":"2024-04-25T09:55:51.309120+0000","last_active":"2024-04-25T09:55:51.316507+0000","last_peered":"2024-04-25T09:55:51.316507+0000","last_clean":"2024-04-25T09:55:51.316507+0000","last_became_active":"2024-04-25T09:55:51.308443+0000","last_became_peered":"2024-04-25T09:55:51.308443+0000","last_unstale":"2024-04-25T09:55:51.316507+0000","last_undegraded":"2024-04-25T09:55:51.316507+0000","last_fullsized":"2024-04-25T09:55:51.316507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.335930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.294489+0000","last_change":"2024-04-25T09:55:50.305023+0000","last_active":"2024-04-25T09:55:51.294489+0000","last_peered":"2024-04-25T09:55:51.294489+0000","last_clean":"2024-04-25T09:55:51.294489+0000","last_became_active":"2024-04-25T09:55:50.304744+0000","last_became_peered":"2024-04-25T09:55:50.304744+0000","last_unstale":"2024-04-25T09:55:51.294489+0000","last_undegraded":"2024-04-25T09:55:51.294489+0000","last_fullsized":"2024-04-25T09:55:51.294489+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.787430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307022+0000","last_change":"2024-04-25T09:55:51.304569+0000","last_active":"2024-04-25T09:55:51.307022+0000","last_peered":"2024-04-25T09:55:51.307022+0000","last_clean":"2024-04-25T09:55:51.307022+0000","last_became_active":"2024-04-25T09:55:51.304198+0000","last_became_peered":"2024-04-25T09:55:51.304198+0000","last_unstale":"2024-04-25T09:55:51.307022+0000","last_undegraded":"2024-04-25T09:55:51.307022+0000","last_fullsized":"2024-04-25T09:55:51.307022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303857+0000","last_change":"2024-04-25T09:55:50.305335+0000","last_active":"2024-04-25T09:55:52.303857+0000","last_peered":"2024-04-25T09:55:52.303857+0000","last_clean":"2024-04-25T09:55:52.303857+0000","last_became_active":"2024-04-25T09:55:50.304742+0000","last_became_peered":"2024-04-25T09:55:50.304742+0000","last_unstale":"2024-04-25T09:55:52.303857+0000","last_undegraded":"2024-04-25T09:55:52.303857+0000","last_fullsized":"2024-04-25T09:55:52.303857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:17.164064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"62'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305088+0000","last_change":"2024-04-25T09:55:50.299644+0000","last_active":"2024-04-25T09:55:52.305088+0000","last_peered":"2024-04-25T09:55:52.305088+0000","last_clean":"2024-04-25T09:55:52.305088+0000","last_became_active":"2024-04-25T09:55:50.299368+0000","last_became_peered":"2024-04-25T09:55:50.299368+0000","last_unstale":"2024-04-25T09:55:52.305088+0000","last_undegraded":"2024-04-25T09:55:52.305088+0000","last_fullsized":"2024-04-25T09:55:52.305088+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:46.571072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303546+0000","last_change":"2024-04-25T09:55:51.308785+0000","last_active":"2024-04-25T09:55:52.303546+0000","last_peered":"2024-04-25T09:55:52.303546+0000","last_clean":"2024-04-25T09:55:52.303546+0000","last_became_active":"2024-04-25T09:55:51.308476+0000","last_became_peered":"2024-04-25T09:55:51.308476+0000","last_unstale":"2024-04-25T09:55:52.303546+0000","last_undegraded":"2024-04-25T09:55:52.303546+0000","last_fullsized":"2024-04-25T09:55:52.303546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:15.037916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.295008+0000","last_change":"2024-04-25T09:55:50.298279+0000","last_active":"2024-04-25T09:55:51.295008+0000","last_peered":"2024-04-25T09:55:51.295008+0000","last_clean":"2024-04-25T09:55:51.295008+0000","last_became_active":"2024-04-25T09:55:50.298007+0000","last_became_peered":"2024-04-25T09:55:50.298007+0000","last_unstale":"2024-04-25T09:55:51.295008+0000","last_undegraded":"2024-04-25T09:55:51.295008+0000","last_fullsized":"2024-04-25T09:55:51.295008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:11.476455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321671+0000","last_change":"2024-04-25T09:55:51.318440+0000","last_active":"2024-04-25T09:55:51.321671+0000","last_peered":"2024-04-25T09:55:51.321671+0000","last_clean":"2024-04-25T09:55:51.321671+0000","last_became_active":"2024-04-25T09:55:51.318162+0000","last_became_peered":"2024-04-25T09:55:51.318162+0000","last_unstale":"2024-04-25T09:55:51.321671+0000","last_undegraded":"2024-04-25T09:55:51.321671+0000","last_fullsized":"2024-04-25T09:55:51.321671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:44.056510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302555+0000","last_change":"2024-04-25T09:55:50.296784+0000","last_active":"2024-04-25T09:55:52.302555+0000","last_peered":"2024-04-25T09:55:52.302555+0000","last_clean":"2024-04-25T09:55:52.302555+0000","last_became_active":"2024-04-25T09:55:50.296190+0000","last_became_peered":"2024-04-25T09:55:50.296190+0000","last_unstale":"2024-04-25T09:55:52.302555+0000","last_undegraded":"2024-04-25T09:55:52.302555+0000","last_fullsized":"2024-04-25T09:55:52.302555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:31.503645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316493+0000","last_change":"2024-04-25T09:55:51.310409+0000","last_active":"2024-04-25T09:55:51.316493+0000","last_peered":"2024-04-25T09:55:51.316493+0000","last_clean":"2024-04-25T09:55:51.316493+0000","last_became_active":"2024-04-25T09:55:51.310147+0000","last_became_peered":"2024-04-25T09:55:51.310147+0000","last_unstale":"2024-04-25T09:55:51.316493+0000","last_undegraded":"2024-04-25T09:55:51.316493+0000","last_fullsized":"2024-04-25T09:55:51.316493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:06.256207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.306556+0000","last_change":"2024-04-25T09:55:50.307476+0000","last_active":"2024-04-25T09:55:51.306556+0000","last_peered":"2024-04-25T09:55:51.306556+0000","last_clean":"2024-04-25T09:55:51.306556+0000","last_became_active":"2024-04-25T09:55:50.307206+0000","last_became_peered":"2024-04-25T09:55:50.307206+0000","last_unstale":"2024-04-25T09:55:51.306556+0000","last_undegraded":"2024-04-25T09:55:51.306556+0000","last_fullsized":"2024-04-25T09:55:51.306556+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:05.108894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315150+0000","last_change":"2024-04-25T09:55:51.313741+0000","last_active":"2024-04-25T09:55:51.315150+0000","last_peered":"2024-04-25T09:55:51.315150+0000","last_clean":"2024-04-25T09:55:51.315150+0000","last_became_active":"2024-04-25T09:55:51.313475+0000","last_became_peered":"2024-04-25T09:55:51.313475+0000","last_unstale":"2024-04-25T09:55:51.315150+0000","last_undegraded":"2024-04-25T09:55:51.315150+0000","last_fullsized":"2024-04-25T09:55:51.315150+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:07.129449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291688+0000","last_change":"2024-04-25T09:55:50.300744+0000","last_active":"2024-04-25T09:55:51.291688+0000","last_peered":"2024-04-25T09:55:51.291688+0000","last_clean":"2024-04-25T09:55:51.291688+0000","last_became_active":"2024-04-25T09:55:50.300415+0000","last_became_peered":"2024-04-25T09:55:50.300415+0000","last_unstale":"2024-04-25T09:55:51.291688+0000","last_undegraded":"2024-04-25T09:55:51.291688+0000","last_fullsized":"2024-04-25T09:55:51.291688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.144383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302118+0000","last_change":"2024-04-25T09:55:51.307744+0000","last_active":"2024-04-25T09:55:52.302118+0000","last_peered":"2024-04-25T09:55:52.302118+0000","last_clean":"2024-04-25T09:55:52.302118+0000","last_became_active":"2024-04-25T09:55:51.307427+0000","last_became_peered":"2024-04-25T09:55:51.307427+0000","last_unstale":"2024-04-25T09:55:52.302118+0000","last_undegraded":"2024-04-25T09:55:52.302118+0000","last_fullsized":"2024-04-25T09:55:52.302118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:17.804233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291814+0000","last_change":"2024-04-25T09:55:50.301696+0000","last_active":"2024-04-25T09:55:51.291814+0000","last_peered":"2024-04-25T09:55:51.291814+0000","last_clean":"2024-04-25T09:55:51.291814+0000","last_became_active":"2024-04-25T09:55:50.299321+0000","last_became_peered":"2024-04-25T09:55:50.299321+0000","last_unstale":"2024-04-25T09:55:51.291814+0000","last_undegraded":"2024-04-25T09:55:51.291814+0000","last_fullsized":"2024-04-25T09:55:51.291814+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:25.435344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.301997+0000","last_change":"2024-04-25T09:55:50.301123+0000","last_active":"2024-04-25T09:55:52.301997+0000","last_peered":"2024-04-25T09:55:52.301997+0000","last_clean":"2024-04-25T09:55:52.301997+0000","last_became_active":"2024-04-25T09:55:50.300861+0000","last_became_peered":"2024-04-25T09:55:50.300861+0000","last_unstale":"2024-04-25T09:55:52.301997+0000","last_undegraded":"2024-04-25T09:55:52.301997+0000","last_fullsized":"2024-04-25T09:55:52.301997+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:00.227291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316474+0000","last_change":"2024-04-25T09:55:51.309200+0000","last_active":"2024-04-25T09:55:51.316474+0000","last_peered":"2024-04-25T09:55:51.316474+0000","last_clean":"2024-04-25T09:55:51.316474+0000","last_became_active":"2024-04-25T09:55:51.308915+0000","last_became_peered":"2024-04-25T09:55:51.308915+0000","last_unstale":"2024-04-25T09:55:51.316474+0000","last_undegraded":"2024-04-25T09:55:51.316474+0000","last_fullsized":"2024-04-25T09:55:51.316474+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:07.129449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291165+0000","last_change":"2024-04-25T09:55:50.298052+0000","last_active":"2024-04-25T09:55:51.291165+0000","last_peered":"2024-04-25T09:55:51.291165+0000","last_clean":"2024-04-25T09:55:51.291165+0000","last_became_active":"2024-04-25T09:55:50.297769+0000","last_became_peered":"2024-04-25T09:55:50.297769+0000","last_unstale":"2024-04-25T09:55:51.291165+0000","last_undegraded":"2024-04-25T09:55:51.291165+0000","last_fullsized":"2024-04-25T09:55:51.291165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:11.476455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":244,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307824+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:55:51.307824+0000","last_peered":"2024-04-25T09:55:51.307824+0000","last_clean":"2024-04-25T09:55:51.307824+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:55:51.307824+0000","last_undegraded":"2024-04-25T09:55:51.307824+0000","last_fullsized":"2024-04-25T09:55:51.307824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":1347584,"data_stored":920910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":741376,"data_stored":329800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":753664,"data_stored":336480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":749568,"data_stored":331378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":737280,"data_stored":329766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":700416,"data_stored":309731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":778240,"data_stored":338834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1331200,"data_stored":920134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:55:54.582 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:55:54.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.713+0000 7fad26ec4640 1 Processor -- start 2024-04-25T09:55:54.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.713+0000 7fad26ec4640 1 -- start start 2024-04-25T09:55:54.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.714+0000 7fad26ec4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad2015a860 0x7fad200cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.714+0000 7fad26ec4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad200af6d0 con 0x7fad2006ea20 2024-04-25T09:55:54.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.714+0000 7fad26ec4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad200af9b0 con 0x7fad200cd1f0 2024-04-25T09:55:54.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.714+0000 7fad26ec4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad200b02e0 con 0x7fad2015a860 2024-04-25T09:55:54.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.714+0000 7fad24c60640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad2015a860 0x7fad200cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.714+0000 7fad24c60640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad2015a860 0x7fad200cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58340/0 (socket says 172.21.15.139:58340) 2024-04-25T09:55:54.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.714+0000 7fad25461640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fad2006ea20 0x7fad200b0770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56328/0 (socket says 172.21.15.139:56328) 2024-04-25T09:55:54.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.714+0000 7fad24c60640 1 -- 172.21.15.139:0/2876824493 learned_addr learned my addr 172.21.15.139:0/2876824493 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:54.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.715+0000 7fad24c60640 1 -- 172.21.15.139:0/2876824493 >> v1:172.21.15.150:6789/0 conn(0x7fad200cd1f0 legacy=0x7fad2006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.715+0000 7fad24c60640 1 -- 172.21.15.139:0/2876824493 >> v1:172.21.15.139:6789/0 conn(0x7fad2006ea20 legacy=0x7fad200b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.715+0000 7fad24c60640 1 -- 172.21.15.139:0/2876824493 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad200afc90 con 0x7fad2015a860 2024-04-25T09:55:54.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.715+0000 7fad24c60640 1 --2- 172.21.15.139:0/2876824493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2015a860 0x7fad200cccb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fad10009870 tx=0x7fad1002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb86c0fa09b7bbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:54.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.715+0000 7fad1f7fe640 1 -- 172.21.15.139:0/2876824493 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10037070 con 0x7fad2015a860 2024-04-25T09:55:54.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.715+0000 7fad1f7fe640 1 -- 172.21.15.139:0/2876824493 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1002ae50 con 0x7fad2015a860 2024-04-25T09:55:54.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.715+0000 7fad1f7fe640 1 -- 172.21.15.139:0/2876824493 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10032860 con 0x7fad2015a860 2024-04-25T09:55:54.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.716+0000 7fad26ec4640 1 -- 172.21.15.139:0/2876824493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2015a860 msgr2=0x7fad200cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.716+0000 7fad26ec4640 1 --2- 172.21.15.139:0/2876824493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2015a860 0x7fad200cccb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fad10009870 tx=0x7fad1002a300 comp rx=0 tx=0).stop 2024-04-25T09:55:54.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.716+0000 7fad26ec4640 1 -- 172.21.15.139:0/2876824493 shutdown_connections 2024-04-25T09:55:54.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.716+0000 7fad26ec4640 1 --2- 172.21.15.139:0/2876824493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2015a860 0x7fad200cccb0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.716+0000 7fad26ec4640 1 -- 172.21.15.139:0/2876824493 >> 172.21.15.139:0/2876824493 conn(0x7fad20066f20 msgr2=0x7fad2006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:54.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.716+0000 7fad26ec4640 1 -- 172.21.15.139:0/2876824493 shutdown_connections 2024-04-25T09:55:54.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.716+0000 7fad26ec4640 1 -- 172.21.15.139:0/2876824493 wait complete. 2024-04-25T09:55:54.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.717+0000 7fad26ec4640 1 Processor -- start 2024-04-25T09:55:54.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.717+0000 7fad26ec4640 1 -- start start 2024-04-25T09:55:54.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad26ec4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad2006ea20 0x7fad2014aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad26ec4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad200cd1f0 0x7fad201e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad26ec4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2015a860 0x7fad2014e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad26ec4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200af420 con 0x7fad200cd1f0 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad26ec4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200af2a0 con 0x7fad2006ea20 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad26ec4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2015ac40 con 0x7fad2015a860 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad1ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad200cd1f0 0x7fad201e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad1ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad200cd1f0 0x7fad201e45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58352/0 (socket says 172.21.15.139:58352) 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad1ffff640 1 -- 172.21.15.139:0/855119781 learned_addr learned my addr 172.21.15.139:0/855119781 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad25461640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2015a860 0x7fad2014e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad24c60640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad2006ea20 0x7fad2014aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad1ffff640 1 -- 172.21.15.139:0/855119781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2015a860 msgr2=0x7fad2014e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad1ffff640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2015a860 0x7fad2014e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad1ffff640 1 -- 172.21.15.139:0/855119781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad2006ea20 msgr2=0x7fad2014aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad1ffff640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad2006ea20 0x7fad2014aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.718+0000 7fad1ffff640 1 -- 172.21.15.139:0/855119781 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad201dfcf0 con 0x7fad200cd1f0 2024-04-25T09:55:54.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.719+0000 7fad1ffff640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad200cd1f0 0x7fad201e45e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fad1400c400 tx=0x7fad1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:54.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.719+0000 7fad1d7fa640 1 -- 172.21.15.139:0/855119781 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14017070 con 0x7fad200cd1f0 2024-04-25T09:55:54.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.719+0000 7fad1d7fa640 1 -- 172.21.15.139:0/855119781 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad140052d0 con 0x7fad200cd1f0 2024-04-25T09:55:54.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.719+0000 7fad26ec4640 1 -- 172.21.15.139:0/855119781 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2014eb80 con 0x7fad200cd1f0 2024-04-25T09:55:54.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.719+0000 7fad1d7fa640 1 -- 172.21.15.139:0/855119781 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14003830 con 0x7fad200cd1f0 2024-04-25T09:55:54.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.719+0000 7fad26ec4640 1 -- 172.21.15.139:0/855119781 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2006d9a0 con 0x7fad200cd1f0 2024-04-25T09:55:54.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.720+0000 7fad26ec4640 1 -- 172.21.15.139:0/855119781 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad20074520 con 0x7fad200cd1f0 2024-04-25T09:55:54.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.720+0000 7fad1d7fa640 1 -- 172.21.15.139:0/855119781 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad1401a070 con 0x7fad200cd1f0 2024-04-25T09:55:54.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.721+0000 7fad1d7fa640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad0c06b500 0x7fad0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:54.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.721+0000 7fad24c60640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad0c06b500 0x7fad0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:54.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.721+0000 7fad1d7fa640 1 -- 172.21.15.139:0/855119781 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fad140897d0 con 0x7fad200cd1f0 2024-04-25T09:55:54.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.721+0000 7fad24c60640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad0c06b500 0x7fad0c06d9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fad10034000 tx=0x7fad1002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:54.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.725+0000 7fad1d7fa640 1 -- 172.21.15.139:0/855119781 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad14058440 con 0x7fad200cd1f0 2024-04-25T09:55:54.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.895+0000 7fad26ec4640 1 -- 172.21.15.139:0/855119781 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad20159280 con 0x7fad0c06b500 2024-04-25T09:55:54.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.901+0000 7fad1d7fa640 1 -- 172.21.15.139:0/855119781 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250954 (secure 0 0 0) 0x7fad20159280 con 0x7fad0c06b500 2024-04-25T09:55:54.907 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:54.908 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:55:54.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.904+0000 7fad26ec4640 1 -- 172.21.15.139:0/855119781 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad0c06b500 msgr2=0x7fad0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.904+0000 7fad26ec4640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad0c06b500 0x7fad0c06d9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fad10034000 tx=0x7fad1002e000 comp rx=0 tx=0).stop 2024-04-25T09:55:54.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.904+0000 7fad26ec4640 1 -- 172.21.15.139:0/855119781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad200cd1f0 msgr2=0x7fad201e45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:54.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.904+0000 7fad26ec4640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad200cd1f0 0x7fad201e45e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fad1400c400 tx=0x7fad1400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.904+0000 7fad26ec4640 1 -- 172.21.15.139:0/855119781 shutdown_connections 2024-04-25T09:55:54.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.904+0000 7fad26ec4640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad200cd1f0 0x7fad201e45e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.904+0000 7fad26ec4640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad0c06b500 0x7fad0c06d9c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.904+0000 7fad26ec4640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2015a860 0x7fad2014e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.904+0000 7fad26ec4640 1 --2- 172.21.15.139:0/855119781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad2006ea20 0x7fad2014aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:54.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.904+0000 7fad26ec4640 1 -- 172.21.15.139:0/855119781 >> 172.21.15.139:0/855119781 conn(0x7fad20066f20 msgr2=0x7fad20157280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:54.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.904+0000 7fad26ec4640 1 -- 172.21.15.139:0/855119781 shutdown_connections 2024-04-25T09:55:54.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:54.904+0000 7fad26ec4640 1 -- 172.21.15.139:0/855119781 wait complete. 2024-04-25T09:55:54.918 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-04-25T09:55:53.000810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":223164,"kb_used_data":6972,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751556,"statfs":{"total":773094113280,"available":772865593344,"internally_reserved":0,"allocated":7139328,"data_stored":3817033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000728"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302671+0000","last_change":"2024-04-25T09:55:51.303411+0000","last_active":"2024-04-25T09:55:52.302671+0000","last_peered":"2024-04-25T09:55:52.302671+0000","last_clean":"2024-04-25T09:55:52.302671+0000","last_became_active":"2024-04-25T09:55:51.303129+0000","last_became_peered":"2024-04-25T09:55:51.303129+0000","last_unstale":"2024-04-25T09:55:52.302671+0000","last_undegraded":"2024-04-25T09:55:52.302671+0000","last_fullsized":"2024-04-25T09:55:52.302671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:35.354743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302461+0000","last_change":"2024-04-25T09:55:51.301262+0000","last_active":"2024-04-25T09:55:52.302461+0000","last_peered":"2024-04-25T09:55:52.302461+0000","last_clean":"2024-04-25T09:55:52.302461+0000","last_became_active":"2024-04-25T09:55:51.300989+0000","last_became_peered":"2024-04-25T09:55:51.300989+0000","last_unstale":"2024-04-25T09:55:52.302461+0000","last_undegraded":"2024-04-25T09:55:52.302461+0000","last_fullsized":"2024-04-25T09:55:52.302461+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:49.815961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322973+0000","last_change":"2024-04-25T09:55:51.321176+0000","last_active":"2024-04-25T09:55:51.322973+0000","last_peered":"2024-04-25T09:55:51.322973+0000","last_clean":"2024-04-25T09:55:51.322973+0000","last_became_active":"2024-04-25T09:55:51.319170+0000","last_became_peered":"2024-04-25T09:55:51.319170+0000","last_unstale":"2024-04-25T09:55:51.322973+0000","last_undegraded":"2024-04-25T09:55:51.322973+0000","last_fullsized":"2024-04-25T09:55:51.322973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305476+0000","last_change":"2024-04-25T09:55:51.303613+0000","last_active":"2024-04-25T09:55:51.305476+0000","last_peered":"2024-04-25T09:55:51.305476+0000","last_clean":"2024-04-25T09:55:51.305476+0000","last_became_active":"2024-04-25T09:55:51.303266+0000","last_became_peered":"2024-04-25T09:55:51.303266+0000","last_unstale":"2024-04-25T09:55:51.305476+0000","last_undegraded":"2024-04-25T09:55:51.305476+0000","last_fullsized":"2024-04-25T09:55:51.305476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:37.811562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321590+0000","last_change":"2024-04-25T09:55:51.317347+0000","last_active":"2024-04-25T09:55:51.321590+0000","last_peered":"2024-04-25T09:55:51.321590+0000","last_clean":"2024-04-25T09:55:51.321590+0000","last_became_active":"2024-04-25T09:55:51.316810+0000","last_became_peered":"2024-04-25T09:55:51.316810+0000","last_unstale":"2024-04-25T09:55:51.321590+0000","last_undegraded":"2024-04-25T09:55:51.321590+0000","last_fullsized":"2024-04-25T09:55:51.321590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:32.510934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.320186+0000","last_change":"2024-04-25T09:55:51.319541+0000","last_active":"2024-04-25T09:55:51.320186+0000","last_peered":"2024-04-25T09:55:51.320186+0000","last_clean":"2024-04-25T09:55:51.320186+0000","last_became_active":"2024-04-25T09:55:51.319218+0000","last_became_peered":"2024-04-25T09:55:51.319218+0000","last_unstale":"2024-04-25T09:55:51.320186+0000","last_undegraded":"2024-04-25T09:55:51.320186+0000","last_fullsized":"2024-04-25T09:55:51.320186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303121+0000","last_change":"2024-04-25T09:55:51.320026+0000","last_active":"2024-04-25T09:55:52.303121+0000","last_peered":"2024-04-25T09:55:52.303121+0000","last_clean":"2024-04-25T09:55:52.303121+0000","last_became_active":"2024-04-25T09:55:51.319656+0000","last_became_peered":"2024-04-25T09:55:51.319656+0000","last_unstale":"2024-04-25T09:55:52.303121+0000","last_undegraded":"2024-04-25T09:55:52.303121+0000","last_fullsized":"2024-04-25T09:55:52.303121+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.317650+0000","last_change":"2024-04-25T09:55:51.314164+0000","last_active":"2024-04-25T09:55:51.317650+0000","last_peered":"2024-04-25T09:55:51.317650+0000","last_clean":"2024-04-25T09:55:51.317650+0000","last_became_active":"2024-04-25T09:55:51.313888+0000","last_became_peered":"2024-04-25T09:55:51.313888+0000","last_unstale":"2024-04-25T09:55:51.317650+0000","last_undegraded":"2024-04-25T09:55:51.317650+0000","last_fullsized":"2024-04-25T09:55:51.317650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:12.483744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321807+0000","last_change":"2024-04-25T09:55:51.316804+0000","last_active":"2024-04-25T09:55:51.321807+0000","last_peered":"2024-04-25T09:55:51.321807+0000","last_clean":"2024-04-25T09:55:51.321807+0000","last_became_active":"2024-04-25T09:55:51.314608+0000","last_became_peered":"2024-04-25T09:55:51.314608+0000","last_unstale":"2024-04-25T09:55:51.321807+0000","last_undegraded":"2024-04-25T09:55:51.321807+0000","last_fullsized":"2024-04-25T09:55:51.321807+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:10.716067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.311232+0000","last_change":"2024-04-25T09:55:51.306747+0000","last_active":"2024-04-25T09:55:51.311232+0000","last_peered":"2024-04-25T09:55:51.311232+0000","last_clean":"2024-04-25T09:55:51.311232+0000","last_became_active":"2024-04-25T09:55:51.306393+0000","last_became_peered":"2024-04-25T09:55:51.306393+0000","last_unstale":"2024-04-25T09:55:51.311232+0000","last_undegraded":"2024-04-25T09:55:51.311232+0000","last_fullsized":"2024-04-25T09:55:51.311232+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.962859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303278+0000","last_change":"2024-04-25T09:55:51.311594+0000","last_active":"2024-04-25T09:55:52.303278+0000","last_peered":"2024-04-25T09:55:52.303278+0000","last_clean":"2024-04-25T09:55:52.303278+0000","last_became_active":"2024-04-25T09:55:51.311271+0000","last_became_peered":"2024-04-25T09:55:51.311271+0000","last_unstale":"2024-04-25T09:55:52.303278+0000","last_undegraded":"2024-04-25T09:55:52.303278+0000","last_fullsized":"2024-04-25T09:55:52.303278+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:44.151672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303327+0000","last_change":"2024-04-25T09:55:51.305962+0000","last_active":"2024-04-25T09:55:52.303327+0000","last_peered":"2024-04-25T09:55:52.303327+0000","last_clean":"2024-04-25T09:55:52.303327+0000","last_became_active":"2024-04-25T09:55:51.305232+0000","last_became_peered":"2024-04-25T09:55:51.305232+0000","last_unstale":"2024-04-25T09:55:52.303327+0000","last_undegraded":"2024-04-25T09:55:52.303327+0000","last_fullsized":"2024-04-25T09:55:52.303327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:26.442633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322877+0000","last_change":"2024-04-25T09:55:51.321119+0000","last_active":"2024-04-25T09:55:51.322877+0000","last_peered":"2024-04-25T09:55:51.322877+0000","last_clean":"2024-04-25T09:55:51.322877+0000","last_became_active":"2024-04-25T09:55:51.320822+0000","last_became_peered":"2024-04-25T09:55:51.320822+0000","last_unstale":"2024-04-25T09:55:51.322877+0000","last_undegraded":"2024-04-25T09:55:51.322877+0000","last_fullsized":"2024-04-25T09:55:51.322877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.318297+0000","last_change":"2024-04-25T09:55:51.313921+0000","last_active":"2024-04-25T09:55:51.318297+0000","last_peered":"2024-04-25T09:55:51.318297+0000","last_clean":"2024-04-25T09:55:51.318297+0000","last_became_active":"2024-04-25T09:55:51.313653+0000","last_became_peered":"2024-04-25T09:55:51.313653+0000","last_unstale":"2024-04-25T09:55:51.318297+0000","last_undegraded":"2024-04-25T09:55:51.318297+0000","last_fullsized":"2024-04-25T09:55:51.318297+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:11.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302647+0000","last_change":"2024-04-25T09:55:51.306904+0000","last_active":"2024-04-25T09:55:52.302647+0000","last_peered":"2024-04-25T09:55:52.302647+0000","last_clean":"2024-04-25T09:55:52.302647+0000","last_became_active":"2024-04-25T09:55:51.306564+0000","last_became_peered":"2024-04-25T09:55:51.306564+0000","last_unstale":"2024-04-25T09:55:52.302647+0000","last_undegraded":"2024-04-25T09:55:52.302647+0000","last_fullsized":"2024-04-25T09:55:52.302647+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:25.733756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305038+0000","last_change":"2024-04-25T09:55:51.297807+0000","last_active":"2024-04-25T09:55:51.305038+0000","last_peered":"2024-04-25T09:55:51.305038+0000","last_clean":"2024-04-25T09:55:51.305038+0000","last_became_active":"2024-04-25T09:55:51.297473+0000","last_became_peered":"2024-04-25T09:55:51.297473+0000","last_unstale":"2024-04-25T09:55:51.305038+0000","last_undegraded":"2024-04-25T09:55:51.305038+0000","last_fullsized":"2024-04-25T09:55:51.305038+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:14.381048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316239+0000","last_change":"2024-04-25T09:55:51.308066+0000","last_active":"2024-04-25T09:55:51.316239+0000","last_peered":"2024-04-25T09:55:51.316239+0000","last_clean":"2024-04-25T09:55:51.316239+0000","last_became_active":"2024-04-25T09:55:51.307793+0000","last_became_peered":"2024-04-25T09:55:51.307793+0000","last_unstale":"2024-04-25T09:55:51.316239+0000","last_undegraded":"2024-04-25T09:55:51.316239+0000","last_fullsized":"2024-04-25T09:55:51.316239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.962859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.319697+0000","last_change":"2024-04-25T09:55:51.318475+0000","last_active":"2024-04-25T09:55:51.319697+0000","last_peered":"2024-04-25T09:55:51.319697+0000","last_clean":"2024-04-25T09:55:51.319697+0000","last_became_active":"2024-04-25T09:55:51.318203+0000","last_became_peered":"2024-04-25T09:55:51.318203+0000","last_unstale":"2024-04-25T09:55:51.319697+0000","last_undegraded":"2024-04-25T09:55:51.319697+0000","last_fullsized":"2024-04-25T09:55:51.319697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:23.692837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303459+0000","last_change":"2024-04-25T09:55:51.306284+0000","last_active":"2024-04-25T09:55:52.303459+0000","last_peered":"2024-04-25T09:55:52.303459+0000","last_clean":"2024-04-25T09:55:52.303459+0000","last_became_active":"2024-04-25T09:55:51.305652+0000","last_became_peered":"2024-04-25T09:55:51.305652+0000","last_unstale":"2024-04-25T09:55:52.303459+0000","last_undegraded":"2024-04-25T09:55:52.303459+0000","last_fullsized":"2024-04-25T09:55:52.303459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316425+0000","last_change":"2024-04-25T09:55:51.315278+0000","last_active":"2024-04-25T09:55:51.316425+0000","last_peered":"2024-04-25T09:55:51.316425+0000","last_clean":"2024-04-25T09:55:51.316425+0000","last_became_active":"2024-04-25T09:55:51.314991+0000","last_became_peered":"2024-04-25T09:55:51.314991+0000","last_unstale":"2024-04-25T09:55:51.316425+0000","last_undegraded":"2024-04-25T09:55:51.316425+0000","last_fullsized":"2024-04-25T09:55:51.316425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:27.808147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316412+0000","last_change":"2024-04-25T09:55:51.315286+0000","last_active":"2024-04-25T09:55:51.316412+0000","last_peered":"2024-04-25T09:55:51.316412+0000","last_clean":"2024-04-25T09:55:51.316412+0000","last_became_active":"2024-04-25T09:55:51.315029+0000","last_became_peered":"2024-04-25T09:55:51.315029+0000","last_unstale":"2024-04-25T09:55:51.316412+0000","last_undegraded":"2024-04-25T09:55:51.316412+0000","last_fullsized":"2024-04-25T09:55:51.316412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:15.037916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323697+0000","last_change":"2024-04-25T09:55:51.322865+0000","last_active":"2024-04-25T09:55:51.323697+0000","last_peered":"2024-04-25T09:55:51.323697+0000","last_clean":"2024-04-25T09:55:51.323697+0000","last_became_active":"2024-04-25T09:55:51.321449+0000","last_became_peered":"2024-04-25T09:55:51.321449+0000","last_unstale":"2024-04-25T09:55:51.323697+0000","last_undegraded":"2024-04-25T09:55:51.323697+0000","last_fullsized":"2024-04-25T09:55:51.323697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:44.151672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303636+0000","last_change":"2024-04-25T09:55:51.309760+0000","last_active":"2024-04-25T09:55:52.303636+0000","last_peered":"2024-04-25T09:55:52.303636+0000","last_clean":"2024-04-25T09:55:52.303636+0000","last_became_active":"2024-04-25T09:55:51.309495+0000","last_became_peered":"2024-04-25T09:55:51.309495+0000","last_unstale":"2024-04-25T09:55:52.303636+0000","last_undegraded":"2024-04-25T09:55:52.303636+0000","last_fullsized":"2024-04-25T09:55:52.303636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:27.808147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303476+0000","last_change":"2024-04-25T09:55:51.308968+0000","last_active":"2024-04-25T09:55:52.303476+0000","last_peered":"2024-04-25T09:55:52.303476+0000","last_clean":"2024-04-25T09:55:52.303476+0000","last_became_active":"2024-04-25T09:55:51.307967+0000","last_became_peered":"2024-04-25T09:55:51.307967+0000","last_unstale":"2024-04-25T09:55:52.303476+0000","last_undegraded":"2024-04-25T09:55:52.303476+0000","last_fullsized":"2024-04-25T09:55:52.303476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:37.934348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304536+0000","last_change":"2024-04-25T09:55:51.298586+0000","last_active":"2024-04-25T09:55:51.304536+0000","last_peered":"2024-04-25T09:55:51.304536+0000","last_clean":"2024-04-25T09:55:51.304536+0000","last_became_active":"2024-04-25T09:55:51.298272+0000","last_became_peered":"2024-04-25T09:55:51.298272+0000","last_unstale":"2024-04-25T09:55:51.304536+0000","last_undegraded":"2024-04-25T09:55:51.304536+0000","last_fullsized":"2024-04-25T09:55:51.304536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:30.656241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302366+0000","last_change":"2024-04-25T09:55:51.318634+0000","last_active":"2024-04-25T09:55:52.302366+0000","last_peered":"2024-04-25T09:55:52.302366+0000","last_clean":"2024-04-25T09:55:52.302366+0000","last_became_active":"2024-04-25T09:55:51.318347+0000","last_became_peered":"2024-04-25T09:55:51.318347+0000","last_unstale":"2024-04-25T09:55:52.302366+0000","last_undegraded":"2024-04-25T09:55:52.302366+0000","last_fullsized":"2024-04-25T09:55:52.302366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:18.171353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.314075+0000","last_change":"2024-04-25T09:55:51.312167+0000","last_active":"2024-04-25T09:55:51.314075+0000","last_peered":"2024-04-25T09:55:51.314075+0000","last_clean":"2024-04-25T09:55:51.314075+0000","last_became_active":"2024-04-25T09:55:51.311785+0000","last_became_peered":"2024-04-25T09:55:51.311785+0000","last_unstale":"2024-04-25T09:55:51.314075+0000","last_undegraded":"2024-04-25T09:55:51.314075+0000","last_fullsized":"2024-04-25T09:55:51.314075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:53.653931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303192+0000","last_change":"2024-04-25T09:55:51.320446+0000","last_active":"2024-04-25T09:55:52.303192+0000","last_peered":"2024-04-25T09:55:52.303192+0000","last_clean":"2024-04-25T09:55:52.303192+0000","last_became_active":"2024-04-25T09:55:51.320112+0000","last_became_peered":"2024-04-25T09:55:51.320112+0000","last_unstale":"2024-04-25T09:55:52.303192+0000","last_undegraded":"2024-04-25T09:55:52.303192+0000","last_fullsized":"2024-04-25T09:55:52.303192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304965+0000","last_change":"2024-04-25T09:55:51.301513+0000","last_active":"2024-04-25T09:55:51.304965+0000","last_peered":"2024-04-25T09:55:51.304965+0000","last_clean":"2024-04-25T09:55:51.304965+0000","last_became_active":"2024-04-25T09:55:51.295610+0000","last_became_peered":"2024-04-25T09:55:51.295610+0000","last_unstale":"2024-04-25T09:55:51.304965+0000","last_undegraded":"2024-04-25T09:55:51.304965+0000","last_fullsized":"2024-04-25T09:55:51.304965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.757412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.303022+0000","last_change":"2024-04-25T09:55:51.300604+0000","last_active":"2024-04-25T09:55:51.303022+0000","last_peered":"2024-04-25T09:55:51.303022+0000","last_clean":"2024-04-25T09:55:51.303022+0000","last_became_active":"2024-04-25T09:55:51.300286+0000","last_became_peered":"2024-04-25T09:55:51.300286+0000","last_unstale":"2024-04-25T09:55:51.303022+0000","last_undegraded":"2024-04-25T09:55:51.303022+0000","last_fullsized":"2024-04-25T09:55:51.303022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316388+0000","last_change":"2024-04-25T09:55:51.310013+0000","last_active":"2024-04-25T09:55:51.316388+0000","last_peered":"2024-04-25T09:55:51.316388+0000","last_clean":"2024-04-25T09:55:51.316388+0000","last_became_active":"2024-04-25T09:55:51.309619+0000","last_became_peered":"2024-04-25T09:55:51.309619+0000","last_unstale":"2024-04-25T09:55:51.316388+0000","last_undegraded":"2024-04-25T09:55:51.316388+0000","last_fullsized":"2024-04-25T09:55:51.316388+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:30.874963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302699+0000","last_change":"2024-04-25T09:55:50.305070+0000","last_active":"2024-04-25T09:55:52.302699+0000","last_peered":"2024-04-25T09:55:52.302699+0000","last_clean":"2024-04-25T09:55:52.302699+0000","last_became_active":"2024-04-25T09:55:50.304749+0000","last_became_peered":"2024-04-25T09:55:50.304749+0000","last_unstale":"2024-04-25T09:55:52.302699+0000","last_undegraded":"2024-04-25T09:55:52.302699+0000","last_fullsized":"2024-04-25T09:55:52.302699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:43.049220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.309334+0000","last_change":"2024-04-25T09:55:51.305186+0000","last_active":"2024-04-25T09:55:51.309334+0000","last_peered":"2024-04-25T09:55:51.309334+0000","last_clean":"2024-04-25T09:55:51.309334+0000","last_became_active":"2024-04-25T09:55:51.304843+0000","last_became_peered":"2024-04-25T09:55:51.304843+0000","last_unstale":"2024-04-25T09:55:51.309334+0000","last_undegraded":"2024-04-25T09:55:51.309334+0000","last_fullsized":"2024-04-25T09:55:51.309334+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:46.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323347+0000","last_change":"2024-04-25T09:55:50.305062+0000","last_active":"2024-04-25T09:55:51.323347+0000","last_peered":"2024-04-25T09:55:51.323347+0000","last_clean":"2024-04-25T09:55:51.323347+0000","last_became_active":"2024-04-25T09:55:50.304760+0000","last_became_peered":"2024-04-25T09:55:50.304760+0000","last_unstale":"2024-04-25T09:55:51.323347+0000","last_undegraded":"2024-04-25T09:55:51.323347+0000","last_fullsized":"2024-04-25T09:55:51.323347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:10.306355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.308626+0000","last_change":"2024-04-25T09:55:51.307444+0000","last_active":"2024-04-25T09:55:51.308626+0000","last_peered":"2024-04-25T09:55:51.308626+0000","last_clean":"2024-04-25T09:55:51.308626+0000","last_became_active":"2024-04-25T09:55:51.307171+0000","last_became_peered":"2024-04-25T09:55:51.307171+0000","last_unstale":"2024-04-25T09:55:51.308626+0000","last_undegraded":"2024-04-25T09:55:51.308626+0000","last_fullsized":"2024-04-25T09:55:51.308626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:26.442633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.317020+0000","last_change":"2024-04-25T09:55:50.298308+0000","last_active":"2024-04-25T09:55:51.317020+0000","last_peered":"2024-04-25T09:55:51.317020+0000","last_clean":"2024-04-25T09:55:51.317020+0000","last_became_active":"2024-04-25T09:55:50.298018+0000","last_became_peered":"2024-04-25T09:55:50.298018+0000","last_unstale":"2024-04-25T09:55:51.317020+0000","last_undegraded":"2024-04-25T09:55:51.317020+0000","last_fullsized":"2024-04-25T09:55:51.317020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:52.646641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322753+0000","last_change":"2024-04-25T09:55:51.317521+0000","last_active":"2024-04-25T09:55:51.322753+0000","last_peered":"2024-04-25T09:55:51.322753+0000","last_clean":"2024-04-25T09:55:51.322753+0000","last_became_active":"2024-04-25T09:55:51.317265+0000","last_became_peered":"2024-04-25T09:55:51.317265+0000","last_unstale":"2024-04-25T09:55:51.322753+0000","last_undegraded":"2024-04-25T09:55:51.322753+0000","last_fullsized":"2024-04-25T09:55:51.322753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316812+0000","last_change":"2024-04-25T09:55:50.301119+0000","last_active":"2024-04-25T09:55:51.316812+0000","last_peered":"2024-04-25T09:55:51.316812+0000","last_clean":"2024-04-25T09:55:51.316812+0000","last_became_active":"2024-04-25T09:55:50.300321+0000","last_became_peered":"2024-04-25T09:55:50.300321+0000","last_unstale":"2024-04-25T09:55:51.316812+0000","last_undegraded":"2024-04-25T09:55:51.316812+0000","last_fullsized":"2024-04-25T09:55:51.316812+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:09.708778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.311448+0000","last_change":"2024-04-25T09:55:50.297259+0000","last_active":"2024-04-25T09:55:51.311448+0000","last_peered":"2024-04-25T09:55:51.311448+0000","last_clean":"2024-04-25T09:55:51.311448+0000","last_became_active":"2024-04-25T09:55:50.296982+0000","last_became_peered":"2024-04-25T09:55:50.296982+0000","last_unstale":"2024-04-25T09:55:51.311448+0000","last_undegraded":"2024-04-25T09:55:51.311448+0000","last_fullsized":"2024-04-25T09:55:51.311448+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:27.161990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302048+0000","last_change":"2024-04-25T09:55:51.309288+0000","last_active":"2024-04-25T09:55:52.302048+0000","last_peered":"2024-04-25T09:55:52.302048+0000","last_clean":"2024-04-25T09:55:52.302048+0000","last_became_active":"2024-04-25T09:55:51.309018+0000","last_became_peered":"2024-04-25T09:55:51.309018+0000","last_unstale":"2024-04-25T09:55:52.302048+0000","last_undegraded":"2024-04-25T09:55:52.302048+0000","last_fullsized":"2024-04-25T09:55:52.302048+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:47.578362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.310309+0000","last_change":"2024-04-25T09:55:50.298200+0000","last_active":"2024-04-25T09:55:51.310309+0000","last_peered":"2024-04-25T09:55:51.310309+0000","last_clean":"2024-04-25T09:55:51.310309+0000","last_became_active":"2024-04-25T09:55:50.297950+0000","last_became_peered":"2024-04-25T09:55:50.297950+0000","last_unstale":"2024-04-25T09:55:51.310309+0000","last_undegraded":"2024-04-25T09:55:51.310309+0000","last_fullsized":"2024-04-25T09:55:51.310309+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:50.309907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316534+0000","last_change":"2024-04-25T09:55:51.309251+0000","last_active":"2024-04-25T09:55:51.316534+0000","last_peered":"2024-04-25T09:55:51.316534+0000","last_clean":"2024-04-25T09:55:51.316534+0000","last_became_active":"2024-04-25T09:55:51.308805+0000","last_became_peered":"2024-04-25T09:55:51.308805+0000","last_unstale":"2024-04-25T09:55:51.316534+0000","last_undegraded":"2024-04-25T09:55:51.316534+0000","last_fullsized":"2024-04-25T09:55:51.316534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:21.995603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291503+0000","last_change":"2024-04-25T09:55:50.299777+0000","last_active":"2024-04-25T09:55:51.291503+0000","last_peered":"2024-04-25T09:55:51.291503+0000","last_clean":"2024-04-25T09:55:51.291503+0000","last_became_active":"2024-04-25T09:55:50.299522+0000","last_became_peered":"2024-04-25T09:55:50.299522+0000","last_unstale":"2024-04-25T09:55:51.291503+0000","last_undegraded":"2024-04-25T09:55:51.291503+0000","last_fullsized":"2024-04-25T09:55:51.291503+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:09.708778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.298022+0000","last_change":"2024-04-25T09:55:51.296452+0000","last_active":"2024-04-25T09:55:51.298022+0000","last_peered":"2024-04-25T09:55:51.298022+0000","last_clean":"2024-04-25T09:55:51.298022+0000","last_became_active":"2024-04-25T09:55:51.296189+0000","last_became_peered":"2024-04-25T09:55:51.296189+0000","last_unstale":"2024-04-25T09:55:51.298022+0000","last_undegraded":"2024-04-25T09:55:51.298022+0000","last_fullsized":"2024-04-25T09:55:51.298022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.296447+0000","last_change":"2024-04-25T09:55:50.305088+0000","last_active":"2024-04-25T09:55:51.296447+0000","last_peered":"2024-04-25T09:55:51.296447+0000","last_clean":"2024-04-25T09:55:51.296447+0000","last_became_active":"2024-04-25T09:55:50.304789+0000","last_became_peered":"2024-04-25T09:55:50.304789+0000","last_unstale":"2024-04-25T09:55:51.296447+0000","last_undegraded":"2024-04-25T09:55:51.296447+0000","last_fullsized":"2024-04-25T09:55:51.296447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:28.750122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302469+0000","last_change":"2024-04-25T09:55:51.295832+0000","last_active":"2024-04-25T09:55:52.302469+0000","last_peered":"2024-04-25T09:55:52.302469+0000","last_clean":"2024-04-25T09:55:52.302469+0000","last_became_active":"2024-04-25T09:55:51.295549+0000","last_became_peered":"2024-04-25T09:55:51.295549+0000","last_unstale":"2024-04-25T09:55:52.302469+0000","last_undegraded":"2024-04-25T09:55:52.302469+0000","last_fullsized":"2024-04-25T09:55:52.302469+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:31.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.308605+0000","last_change":"2024-04-25T09:55:50.307313+0000","last_active":"2024-04-25T09:55:51.308605+0000","last_peered":"2024-04-25T09:55:51.308605+0000","last_clean":"2024-04-25T09:55:51.308605+0000","last_became_active":"2024-04-25T09:55:50.306998+0000","last_became_peered":"2024-04-25T09:55:50.306998+0000","last_unstale":"2024-04-25T09:55:51.308605+0000","last_undegraded":"2024-04-25T09:55:51.308605+0000","last_fullsized":"2024-04-25T09:55:51.308605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:13.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305345+0000","last_change":"2024-04-25T09:55:51.302618+0000","last_active":"2024-04-25T09:55:51.305345+0000","last_peered":"2024-04-25T09:55:51.305345+0000","last_clean":"2024-04-25T09:55:51.305345+0000","last_became_active":"2024-04-25T09:55:51.301598+0000","last_became_peered":"2024-04-25T09:55:51.301598+0000","last_unstale":"2024-04-25T09:55:51.305345+0000","last_undegraded":"2024-04-25T09:55:51.305345+0000","last_fullsized":"2024-04-25T09:55:51.305345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:37.934348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.292155+0000","last_change":"2024-04-25T09:55:50.300463+0000","last_active":"2024-04-25T09:55:51.292155+0000","last_peered":"2024-04-25T09:55:51.292155+0000","last_clean":"2024-04-25T09:55:51.292155+0000","last_became_active":"2024-04-25T09:55:50.299576+0000","last_became_peered":"2024-04-25T09:55:50.299576+0000","last_unstale":"2024-04-25T09:55:51.292155+0000","last_undegraded":"2024-04-25T09:55:51.292155+0000","last_fullsized":"2024-04-25T09:55:51.292155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:56.828907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307569+0000","last_change":"2024-04-25T09:55:51.306375+0000","last_active":"2024-04-25T09:55:51.307569+0000","last_peered":"2024-04-25T09:55:51.307569+0000","last_clean":"2024-04-25T09:55:51.307569+0000","last_became_active":"2024-04-25T09:55:51.305748+0000","last_became_peered":"2024-04-25T09:55:51.305748+0000","last_unstale":"2024-04-25T09:55:51.307569+0000","last_undegraded":"2024-04-25T09:55:51.307569+0000","last_fullsized":"2024-04-25T09:55:51.307569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:37.811562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322267+0000","last_change":"2024-04-25T09:55:50.307971+0000","last_active":"2024-04-25T09:55:51.322267+0000","last_peered":"2024-04-25T09:55:51.322267+0000","last_clean":"2024-04-25T09:55:51.322267+0000","last_became_active":"2024-04-25T09:55:50.307650+0000","last_became_peered":"2024-04-25T09:55:50.307650+0000","last_unstale":"2024-04-25T09:55:51.322267+0000","last_undegraded":"2024-04-25T09:55:51.322267+0000","last_fullsized":"2024-04-25T09:55:51.322267+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:30.506756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305389+0000","last_change":"2024-04-25T09:55:51.302214+0000","last_active":"2024-04-25T09:55:51.305389+0000","last_peered":"2024-04-25T09:55:51.305389+0000","last_clean":"2024-04-25T09:55:51.305389+0000","last_became_active":"2024-04-25T09:55:51.301887+0000","last_became_peered":"2024-04-25T09:55:51.301887+0000","last_unstale":"2024-04-25T09:55:51.305389+0000","last_undegraded":"2024-04-25T09:55:51.305389+0000","last_fullsized":"2024-04-25T09:55:51.305389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.447273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302496+0000","last_change":"2024-04-25T09:55:50.299295+0000","last_active":"2024-04-25T09:55:52.302496+0000","last_peered":"2024-04-25T09:55:52.302496+0000","last_clean":"2024-04-25T09:55:52.302496+0000","last_became_active":"2024-04-25T09:55:50.296672+0000","last_became_peered":"2024-04-25T09:55:50.296672+0000","last_unstale":"2024-04-25T09:55:52.302496+0000","last_undegraded":"2024-04-25T09:55:52.302496+0000","last_fullsized":"2024-04-25T09:55:52.302496+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:05.248918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315783+0000","last_change":"2024-04-25T09:55:51.312207+0000","last_active":"2024-04-25T09:55:51.315783+0000","last_peered":"2024-04-25T09:55:51.315783+0000","last_clean":"2024-04-25T09:55:51.315783+0000","last_became_active":"2024-04-25T09:55:51.311902+0000","last_became_peered":"2024-04-25T09:55:51.311902+0000","last_unstale":"2024-04-25T09:55:51.315783+0000","last_undegraded":"2024-04-25T09:55:51.315783+0000","last_fullsized":"2024-04-25T09:55:51.315783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.335930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322307+0000","last_change":"2024-04-25T09:55:50.307406+0000","last_active":"2024-04-25T09:55:51.322307+0000","last_peered":"2024-04-25T09:55:51.322307+0000","last_clean":"2024-04-25T09:55:51.322307+0000","last_became_active":"2024-04-25T09:55:50.306864+0000","last_became_peered":"2024-04-25T09:55:50.306864+0000","last_unstale":"2024-04-25T09:55:51.322307+0000","last_undegraded":"2024-04-25T09:55:51.322307+0000","last_fullsized":"2024-04-25T09:55:51.322307+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:54.613991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303023+0000","last_change":"2024-04-25T09:55:51.306953+0000","last_active":"2024-04-25T09:55:52.303023+0000","last_peered":"2024-04-25T09:55:52.303023+0000","last_clean":"2024-04-25T09:55:52.303023+0000","last_became_active":"2024-04-25T09:55:51.306664+0000","last_became_peered":"2024-04-25T09:55:51.306664+0000","last_unstale":"2024-04-25T09:55:52.303023+0000","last_undegraded":"2024-04-25T09:55:52.303023+0000","last_fullsized":"2024-04-25T09:55:52.303023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303002+0000","last_change":"2024-04-25T09:55:50.303832+0000","last_active":"2024-04-25T09:55:52.303002+0000","last_peered":"2024-04-25T09:55:52.303002+0000","last_clean":"2024-04-25T09:55:52.303002+0000","last_became_active":"2024-04-25T09:55:50.303513+0000","last_became_peered":"2024-04-25T09:55:50.303513+0000","last_unstale":"2024-04-25T09:55:52.303002+0000","last_undegraded":"2024-04-25T09:55:52.303002+0000","last_fullsized":"2024-04-25T09:55:52.303002+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:16.796943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315474+0000","last_change":"2024-04-25T09:55:51.306576+0000","last_active":"2024-04-25T09:55:51.315474+0000","last_peered":"2024-04-25T09:55:51.315474+0000","last_clean":"2024-04-25T09:55:51.315474+0000","last_became_active":"2024-04-25T09:55:51.305692+0000","last_became_peered":"2024-04-25T09:55:51.305692+0000","last_unstale":"2024-04-25T09:55:51.315474+0000","last_undegraded":"2024-04-25T09:55:51.315474+0000","last_fullsized":"2024-04-25T09:55:51.315474+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:46.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304282+0000","last_change":"2024-04-25T09:55:50.307077+0000","last_active":"2024-04-25T09:55:51.304282+0000","last_peered":"2024-04-25T09:55:51.304282+0000","last_clean":"2024-04-25T09:55:51.304282+0000","last_became_active":"2024-04-25T09:55:50.306743+0000","last_became_peered":"2024-04-25T09:55:50.306743+0000","last_unstale":"2024-04-25T09:55:51.304282+0000","last_undegraded":"2024-04-25T09:55:51.304282+0000","last_fullsized":"2024-04-25T09:55:51.304282+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:34.347454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302240+0000","last_change":"2024-04-25T09:55:51.315727+0000","last_active":"2024-04-25T09:55:52.302240+0000","last_peered":"2024-04-25T09:55:52.302240+0000","last_clean":"2024-04-25T09:55:52.302240+0000","last_became_active":"2024-04-25T09:55:51.315426+0000","last_became_peered":"2024-04-25T09:55:51.315426+0000","last_unstale":"2024-04-25T09:55:52.302240+0000","last_undegraded":"2024-04-25T09:55:52.302240+0000","last_fullsized":"2024-04-25T09:55:52.302240+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:08.905576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"61'8","reported_seq":20,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.324184+0000","last_change":"2024-04-25T09:55:50.302489+0000","last_active":"2024-04-25T09:55:51.324184+0000","last_peered":"2024-04-25T09:55:51.324184+0000","last_clean":"2024-04-25T09:55:51.324184+0000","last_became_active":"2024-04-25T09:55:50.302222+0000","last_became_peered":"2024-04-25T09:55:50.302222+0000","last_unstale":"2024-04-25T09:55:51.324184+0000","last_undegraded":"2024-04-25T09:55:51.324184+0000","last_fullsized":"2024-04-25T09:55:51.324184+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:34.805327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.312652+0000","last_change":"2024-04-25T09:55:51.310660+0000","last_active":"2024-04-25T09:55:51.312652+0000","last_peered":"2024-04-25T09:55:51.312652+0000","last_clean":"2024-04-25T09:55:51.312652+0000","last_became_active":"2024-04-25T09:55:51.310407+0000","last_became_peered":"2024-04-25T09:55:51.310407+0000","last_unstale":"2024-04-25T09:55:51.312652+0000","last_undegraded":"2024-04-25T09:55:51.312652+0000","last_fullsized":"2024-04-25T09:55:51.312652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:41.794719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303710+0000","last_change":"2024-04-25T09:55:50.303136+0000","last_active":"2024-04-25T09:55:52.303710+0000","last_peered":"2024-04-25T09:55:52.303710+0000","last_clean":"2024-04-25T09:55:52.303710+0000","last_became_active":"2024-04-25T09:55:50.302805+0000","last_became_peered":"2024-04-25T09:55:50.302805+0000","last_unstale":"2024-04-25T09:55:52.303710+0000","last_undegraded":"2024-04-25T09:55:52.303710+0000","last_fullsized":"2024-04-25T09:55:52.303710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.898286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321648+0000","last_change":"2024-04-25T09:55:51.318231+0000","last_active":"2024-04-25T09:55:51.321648+0000","last_peered":"2024-04-25T09:55:51.321648+0000","last_clean":"2024-04-25T09:55:51.321648+0000","last_became_active":"2024-04-25T09:55:51.317964+0000","last_became_peered":"2024-04-25T09:55:51.317964+0000","last_unstale":"2024-04-25T09:55:51.321648+0000","last_undegraded":"2024-04-25T09:55:51.321648+0000","last_fullsized":"2024-04-25T09:55:51.321648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:01.234580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321558+0000","last_change":"2024-04-25T09:55:50.307310+0000","last_active":"2024-04-25T09:55:51.321558+0000","last_peered":"2024-04-25T09:55:51.321558+0000","last_clean":"2024-04-25T09:55:51.321558+0000","last_became_active":"2024-04-25T09:55:50.306877+0000","last_became_peered":"2024-04-25T09:55:50.306877+0000","last_unstale":"2024-04-25T09:55:51.321558+0000","last_undegraded":"2024-04-25T09:55:51.321558+0000","last_fullsized":"2024-04-25T09:55:51.321558+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.808671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321032+0000","last_change":"2024-04-25T09:55:51.319026+0000","last_active":"2024-04-25T09:55:51.321032+0000","last_peered":"2024-04-25T09:55:51.321032+0000","last_clean":"2024-04-25T09:55:51.321032+0000","last_became_active":"2024-04-25T09:55:51.318720+0000","last_became_peered":"2024-04-25T09:55:51.318720+0000","last_unstale":"2024-04-25T09:55:51.321032+0000","last_undegraded":"2024-04-25T09:55:51.321032+0000","last_fullsized":"2024-04-25T09:55:51.321032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:21.995603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323351+0000","last_change":"2024-04-25T09:55:50.309565+0000","last_active":"2024-04-25T09:55:51.323351+0000","last_peered":"2024-04-25T09:55:51.323351+0000","last_clean":"2024-04-25T09:55:51.323351+0000","last_became_active":"2024-04-25T09:55:50.309335+0000","last_became_peered":"2024-04-25T09:55:50.309335+0000","last_unstale":"2024-04-25T09:55:51.323351+0000","last_undegraded":"2024-04-25T09:55:51.323351+0000","last_fullsized":"2024-04-25T09:55:51.323351+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:24.726466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.306954+0000","last_change":"2024-04-25T09:55:51.304138+0000","last_active":"2024-04-25T09:55:51.306954+0000","last_peered":"2024-04-25T09:55:51.306954+0000","last_clean":"2024-04-25T09:55:51.306954+0000","last_became_active":"2024-04-25T09:55:51.303842+0000","last_became_peered":"2024-04-25T09:55:51.303842+0000","last_unstale":"2024-04-25T09:55:51.306954+0000","last_undegraded":"2024-04-25T09:55:51.306954+0000","last_fullsized":"2024-04-25T09:55:51.306954+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.447273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291578+0000","last_change":"2024-04-25T09:55:50.295568+0000","last_active":"2024-04-25T09:55:51.291578+0000","last_peered":"2024-04-25T09:55:51.291578+0000","last_clean":"2024-04-25T09:55:51.291578+0000","last_became_active":"2024-04-25T09:55:50.295151+0000","last_became_peered":"2024-04-25T09:55:50.295151+0000","last_unstale":"2024-04-25T09:55:51.291578+0000","last_undegraded":"2024-04-25T09:55:51.291578+0000","last_fullsized":"2024-04-25T09:55:51.291578+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:43.198244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.299612+0000","last_change":"2024-04-25T09:55:51.297690+0000","last_active":"2024-04-25T09:55:51.299612+0000","last_peered":"2024-04-25T09:55:51.299612+0000","last_clean":"2024-04-25T09:55:51.299612+0000","last_became_active":"2024-04-25T09:55:51.297427+0000","last_became_peered":"2024-04-25T09:55:51.297427+0000","last_unstale":"2024-04-25T09:55:51.299612+0000","last_undegraded":"2024-04-25T09:55:51.299612+0000","last_fullsized":"2024-04-25T09:55:51.299612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.318135+0000","last_change":"2024-04-25T09:55:50.299667+0000","last_active":"2024-04-25T09:55:51.318135+0000","last_peered":"2024-04-25T09:55:51.318135+0000","last_clean":"2024-04-25T09:55:51.318135+0000","last_became_active":"2024-04-25T09:55:50.298743+0000","last_became_peered":"2024-04-25T09:55:50.298743+0000","last_unstale":"2024-04-25T09:55:51.318135+0000","last_undegraded":"2024-04-25T09:55:51.318135+0000","last_fullsized":"2024-04-25T09:55:51.318135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:13.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303024+0000","last_change":"2024-04-25T09:55:51.307575+0000","last_active":"2024-04-25T09:55:52.303024+0000","last_peered":"2024-04-25T09:55:52.303024+0000","last_clean":"2024-04-25T09:55:52.303024+0000","last_became_active":"2024-04-25T09:55:51.307279+0000","last_became_peered":"2024-04-25T09:55:51.307279+0000","last_unstale":"2024-04-25T09:55:52.303024+0000","last_undegraded":"2024-04-25T09:55:52.303024+0000","last_fullsized":"2024-04-25T09:55:52.303024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:10.716067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303126+0000","last_change":"2024-04-25T09:55:50.300890+0000","last_active":"2024-04-25T09:55:52.303126+0000","last_peered":"2024-04-25T09:55:52.303126+0000","last_clean":"2024-04-25T09:55:52.303126+0000","last_became_active":"2024-04-25T09:55:50.300508+0000","last_became_peered":"2024-04-25T09:55:50.300508+0000","last_unstale":"2024-04-25T09:55:52.303126+0000","last_undegraded":"2024-04-25T09:55:52.303126+0000","last_fullsized":"2024-04-25T09:55:52.303126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.808671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316507+0000","last_change":"2024-04-25T09:55:51.309120+0000","last_active":"2024-04-25T09:55:51.316507+0000","last_peered":"2024-04-25T09:55:51.316507+0000","last_clean":"2024-04-25T09:55:51.316507+0000","last_became_active":"2024-04-25T09:55:51.308443+0000","last_became_peered":"2024-04-25T09:55:51.308443+0000","last_unstale":"2024-04-25T09:55:51.316507+0000","last_undegraded":"2024-04-25T09:55:51.316507+0000","last_fullsized":"2024-04-25T09:55:51.316507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.335930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.294489+0000","last_change":"2024-04-25T09:55:50.305023+0000","last_active":"2024-04-25T09:55:51.294489+0000","last_peered":"2024-04-25T09:55:51.294489+0000","last_clean":"2024-04-25T09:55:51.294489+0000","last_became_active":"2024-04-25T09:55:50.304744+0000","last_became_peered":"2024-04-25T09:55:50.304744+0000","last_unstale":"2024-04-25T09:55:51.294489+0000","last_undegraded":"2024-04-25T09:55:51.294489+0000","last_fullsized":"2024-04-25T09:55:51.294489+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.787430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307022+0000","last_change":"2024-04-25T09:55:51.304569+0000","last_active":"2024-04-25T09:55:51.307022+0000","last_peered":"2024-04-25T09:55:51.307022+0000","last_clean":"2024-04-25T09:55:51.307022+0000","last_became_active":"2024-04-25T09:55:51.304198+0000","last_became_peered":"2024-04-25T09:55:51.304198+0000","last_unstale":"2024-04-25T09:55:51.307022+0000","last_undegraded":"2024-04-25T09:55:51.307022+0000","last_fullsized":"2024-04-25T09:55:51.307022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303857+0000","last_change":"2024-04-25T09:55:50.305335+0000","last_active":"2024-04-25T09:55:52.303857+0000","last_peered":"2024-04-25T09:55:52.303857+0000","last_clean":"2024-04-25T09:55:52.303857+0000","last_became_active":"2024-04-25T09:55:50.304742+0000","last_became_peered":"2024-04-25T09:55:50.304742+0000","last_unstale":"2024-04-25T09:55:52.303857+0000","last_undegraded":"2024-04-25T09:55:52.303857+0000","last_fullsized":"2024-04-25T09:55:52.303857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:17.164064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"62'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305088+0000","last_change":"2024-04-25T09:55:50.299644+0000","last_active":"2024-04-25T09:55:52.305088+0000","last_peered":"2024-04-25T09:55:52.305088+0000","last_clean":"2024-04-25T09:55:52.305088+0000","last_became_active":"2024-04-25T09:55:50.299368+0000","last_became_peered":"2024-04-25T09:55:50.299368+0000","last_unstale":"2024-04-25T09:55:52.305088+0000","last_undegraded":"2024-04-25T09:55:52.305088+0000","last_fullsized":"2024-04-25T09:55:52.305088+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:46.571072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303546+0000","last_change":"2024-04-25T09:55:51.308785+0000","last_active":"2024-04-25T09:55:52.303546+0000","last_peered":"2024-04-25T09:55:52.303546+0000","last_clean":"2024-04-25T09:55:52.303546+0000","last_became_active":"2024-04-25T09:55:51.308476+0000","last_became_peered":"2024-04-25T09:55:51.308476+0000","last_unstale":"2024-04-25T09:55:52.303546+0000","last_undegraded":"2024-04-25T09:55:52.303546+0000","last_fullsized":"2024-04-25T09:55:52.303546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:15.037916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.295008+0000","last_change":"2024-04-25T09:55:50.298279+0000","last_active":"2024-04-25T09:55:51.295008+0000","last_peered":"2024-04-25T09:55:51.295008+0000","last_clean":"2024-04-25T09:55:51.295008+0000","last_became_active":"2024-04-25T09:55:50.298007+0000","last_became_peered":"2024-04-25T09:55:50.298007+0000","last_unstale":"2024-04-25T09:55:51.295008+0000","last_undegraded":"2024-04-25T09:55:51.295008+0000","last_fullsized":"2024-04-25T09:55:51.295008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:11.476455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321671+0000","last_change":"2024-04-25T09:55:51.318440+0000","last_active":"2024-04-25T09:55:51.321671+0000","last_peered":"2024-04-25T09:55:51.321671+0000","last_clean":"2024-04-25T09:55:51.321671+0000","last_became_active":"2024-04-25T09:55:51.318162+0000","last_became_peered":"2024-04-25T09:55:51.318162+0000","last_unstale":"2024-04-25T09:55:51.321671+0000","last_undegraded":"2024-04-25T09:55:51.321671+0000","last_fullsized":"2024-04-25T09:55:51.321671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:44.056510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302555+0000","last_change":"2024-04-25T09:55:50.296784+0000","last_active":"2024-04-25T09:55:52.302555+0000","last_peered":"2024-04-25T09:55:52.302555+0000","last_clean":"2024-04-25T09:55:52.302555+0000","last_became_active":"2024-04-25T09:55:50.296190+0000","last_became_peered":"2024-04-25T09:55:50.296190+0000","last_unstale":"2024-04-25T09:55:52.302555+0000","last_undegraded":"2024-04-25T09:55:52.302555+0000","last_fullsized":"2024-04-25T09:55:52.302555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:31.503645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316493+0000","last_change":"2024-04-25T09:55:51.310409+0000","last_active":"2024-04-25T09:55:51.316493+0000","last_peered":"2024-04-25T09:55:51.316493+0000","last_clean":"2024-04-25T09:55:51.316493+0000","last_became_active":"2024-04-25T09:55:51.310147+0000","last_became_peered":"2024-04-25T09:55:51.310147+0000","last_unstale":"2024-04-25T09:55:51.316493+0000","last_undegraded":"2024-04-25T09:55:51.316493+0000","last_fullsized":"2024-04-25T09:55:51.316493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:06.256207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.306556+0000","last_change":"2024-04-25T09:55:50.307476+0000","last_active":"2024-04-25T09:55:51.306556+0000","last_peered":"2024-04-25T09:55:51.306556+0000","last_clean":"2024-04-25T09:55:51.306556+0000","last_became_active":"2024-04-25T09:55:50.307206+0000","last_became_peered":"2024-04-25T09:55:50.307206+0000","last_unstale":"2024-04-25T09:55:51.306556+0000","last_undegraded":"2024-04-25T09:55:51.306556+0000","last_fullsized":"2024-04-25T09:55:51.306556+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:05.108894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315150+0000","last_change":"2024-04-25T09:55:51.313741+0000","last_active":"2024-04-25T09:55:51.315150+0000","last_peered":"2024-04-25T09:55:51.315150+0000","last_clean":"2024-04-25T09:55:51.315150+0000","last_became_active":"2024-04-25T09:55:51.313475+0000","last_became_peered":"2024-04-25T09:55:51.313475+0000","last_unstale":"2024-04-25T09:55:51.315150+0000","last_undegraded":"2024-04-25T09:55:51.315150+0000","last_fullsized":"2024-04-25T09:55:51.315150+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:07.129449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291688+0000","last_change":"2024-04-25T09:55:50.300744+0000","last_active":"2024-04-25T09:55:51.291688+0000","last_peered":"2024-04-25T09:55:51.291688+0000","last_clean":"2024-04-25T09:55:51.291688+0000","last_became_active":"2024-04-25T09:55:50.300415+0000","last_became_peered":"2024-04-25T09:55:50.300415+0000","last_unstale":"2024-04-25T09:55:51.291688+0000","last_undegraded":"2024-04-25T09:55:51.291688+0000","last_fullsized":"2024-04-25T09:55:51.291688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.144383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302118+0000","last_change":"2024-04-25T09:55:51.307744+0000","last_active":"2024-04-25T09:55:52.302118+0000","last_peered":"2024-04-25T09:55:52.302118+0000","last_clean":"2024-04-25T09:55:52.302118+0000","last_became_active":"2024-04-25T09:55:51.307427+0000","last_became_peered":"2024-04-25T09:55:51.307427+0000","last_unstale":"2024-04-25T09:55:52.302118+0000","last_undegraded":"2024-04-25T09:55:52.302118+0000","last_fullsized":"2024-04-25T09:55:52.302118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:17.804233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291814+0000","last_change":"2024-04-25T09:55:50.301696+0000","last_active":"2024-04-25T09:55:51.291814+0000","last_peered":"2024-04-25T09:55:51.291814+0000","last_clean":"2024-04-25T09:55:51.291814+0000","last_became_active":"2024-04-25T09:55:50.299321+0000","last_became_peered":"2024-04-25T09:55:50.299321+0000","last_unstale":"2024-04-25T09:55:51.291814+0000","last_undegraded":"2024-04-25T09:55:51.291814+0000","last_fullsized":"2024-04-25T09:55:51.291814+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:25.435344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:55:50.273814+0000","last_change":"2024-04-25T09:55:50.273814+0000","last_active":"2024-04-25T09:55:50.273814+0000","last_peered":"2024-04-25T09:55:50.273814+0000","last_clean":"2024-04-25T09:55:50.273814+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:55:50.273814+0000","last_undegraded":"2024-04-25T09:55:50.273814+0000","last_fullsized":"2024-04-25T09:55:50.273814+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.301997+0000","last_change":"2024-04-25T09:55:50.301123+0000","last_active":"2024-04-25T09:55:52.301997+0000","last_peered":"2024-04-25T09:55:52.301997+0000","last_clean":"2024-04-25T09:55:52.301997+0000","last_became_active":"2024-04-25T09:55:50.300861+0000","last_became_peered":"2024-04-25T09:55:50.300861+0000","last_unstale":"2024-04-25T09:55:52.301997+0000","last_undegraded":"2024-04-25T09:55:52.301997+0000","last_fullsized":"2024-04-25T09:55:52.301997+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:00.227291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316474+0000","last_change":"2024-04-25T09:55:51.309200+0000","last_active":"2024-04-25T09:55:51.316474+0000","last_peered":"2024-04-25T09:55:51.316474+0000","last_clean":"2024-04-25T09:55:51.316474+0000","last_became_active":"2024-04-25T09:55:51.308915+0000","last_became_peered":"2024-04-25T09:55:51.308915+0000","last_unstale":"2024-04-25T09:55:51.316474+0000","last_undegraded":"2024-04-25T09:55:51.316474+0000","last_fullsized":"2024-04-25T09:55:51.316474+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:07.129449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291165+0000","last_change":"2024-04-25T09:55:50.298052+0000","last_active":"2024-04-25T09:55:51.291165+0000","last_peered":"2024-04-25T09:55:51.291165+0000","last_clean":"2024-04-25T09:55:51.291165+0000","last_became_active":"2024-04-25T09:55:50.297769+0000","last_became_peered":"2024-04-25T09:55:50.297769+0000","last_unstale":"2024-04-25T09:55:51.291165+0000","last_undegraded":"2024-04-25T09:55:51.291165+0000","last_fullsized":"2024-04-25T09:55:51.291165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:11.476455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":244,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307824+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:55:51.307824+0000","last_peered":"2024-04-25T09:55:51.307824+0000","last_clean":"2024-04-25T09:55:51.307824+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:55:51.307824+0000","last_undegraded":"2024-04-25T09:55:51.307824+0000","last_fullsized":"2024-04-25T09:55:51.307824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":1347584,"data_stored":920910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":741376,"data_stored":329800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":753664,"data_stored":336480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":749568,"data_stored":331378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":737280,"data_stored":329766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":700416,"data_stored":309731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":778240,"data_stored":338834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1331200,"data_stored":920134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:55:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2024-04-25T09:55:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:55:50.273814+0000', 'last_change': '2024-04-25T09:55:50.273814+0000', 'last_active': '2024-04-25T09:55:50.273814+0000', 'last_peered': '2024-04-25T09:55:50.273814+0000', 'last_clean': '2024-04-25T09:55:50.273814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:55:50.273814+0000', 'last_undegraded': '2024-04-25T09:55:50.273814+0000', 'last_fullsized': '2024-04-25T09:55:50.273814+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'last_clean_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:55:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2024-04-25T09:55:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:55:50.273814+0000', 'last_change': '2024-04-25T09:55:50.273814+0000', 'last_active': '2024-04-25T09:55:50.273814+0000', 'last_peered': '2024-04-25T09:55:50.273814+0000', 'last_clean': '2024-04-25T09:55:50.273814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:55:50.273814+0000', 'last_undegraded': '2024-04-25T09:55:50.273814+0000', 'last_fullsized': '2024-04-25T09:55:50.273814+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'last_clean_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:55:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2024-04-25T09:55:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:55:50.273814+0000', 'last_change': '2024-04-25T09:55:50.273814+0000', 'last_active': '2024-04-25T09:55:50.273814+0000', 'last_peered': '2024-04-25T09:55:50.273814+0000', 'last_clean': '2024-04-25T09:55:50.273814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:55:50.273814+0000', 'last_undegraded': '2024-04-25T09:55:50.273814+0000', 'last_fullsized': '2024-04-25T09:55:50.273814+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'last_clean_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:55:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2024-04-25T09:55:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:55:50.273814+0000', 'last_change': '2024-04-25T09:55:50.273814+0000', 'last_active': '2024-04-25T09:55:50.273814+0000', 'last_peered': '2024-04-25T09:55:50.273814+0000', 'last_clean': '2024-04-25T09:55:50.273814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:55:50.273814+0000', 'last_undegraded': '2024-04-25T09:55:50.273814+0000', 'last_fullsized': '2024-04-25T09:55:50.273814+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'last_clean_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:55:54.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2024-04-25T09:55:54.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:55:50.273814+0000', 'last_change': '2024-04-25T09:55:50.273814+0000', 'last_active': '2024-04-25T09:55:50.273814+0000', 'last_peered': '2024-04-25T09:55:50.273814+0000', 'last_clean': '2024-04-25T09:55:50.273814+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:55:50.273814+0000', 'last_undegraded': '2024-04-25T09:55:50.273814+0000', 'last_fullsized': '2024-04-25T09:55:50.273814+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'last_clean_scrub_stamp': '2024-04-25T09:55:50.273814+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:55:54.923 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:55:55.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.053+0000 7ff44d3c4640 1 Processor -- start 2024-04-25T09:55:55.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.053+0000 7ff44d3c4640 1 -- start start 2024-04-25T09:55:55.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.054+0000 7ff44d3c4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff44815efb0 0x7ff4480d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.054+0000 7ff44d3c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4480cf900 0x7ff44806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.054+0000 7ff44d3c4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4480cfe70 con 0x7ff44806cd80 2024-04-25T09:55:55.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.054+0000 7ff44d3c4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4480d0a80 con 0x7ff44815efb0 2024-04-25T09:55:55.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.054+0000 7ff44d3c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480d0150 con 0x7ff4480cf900 2024-04-25T09:55:55.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.054+0000 7ff4477fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff44806cd80 0x7ff448099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56334/0 (socket says 172.21.15.139:56334) 2024-04-25T09:55:55.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.054+0000 7ff4477fe640 1 -- 172.21.15.139:0/3963647197 learned_addr learned my addr 172.21.15.139:0/3963647197 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:55.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.054+0000 7ff4467fc640 1 --2- 172.21.15.139:0/3963647197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4480cf900 0x7ff44806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.054+0000 7ff446ffd640 1 --2- 172.21.15.139:0/3963647197 >> v2:172.21.15.150:3300/0 conn(0x7ff44815efb0 0x7ff4480d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.054+0000 7ff445ffb640 1 -- 172.21.15.139:0/3963647197 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474601905 0 0) 0x7ff4480cfe70 con 0x7ff44806cd80 2024-04-25T09:55:55.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.054+0000 7ff445ffb640 1 -- 172.21.15.139:0/3963647197 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff430003520 con 0x7ff44806cd80 2024-04-25T09:55:55.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.055+0000 7ff445ffb640 1 -- 172.21.15.139:0/3963647197 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2519515369 0 0) 0x7ff430003520 con 0x7ff44806cd80 2024-04-25T09:55:55.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.055+0000 7ff4467fc640 1 -- 172.21.15.139:0/3963647197 >> v2:172.21.15.150:3300/0 conn(0x7ff44815efb0 msgr2=0x7ff4480d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:55.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.055+0000 7ff4467fc640 1 --2- 172.21.15.139:0/3963647197 >> v2:172.21.15.150:3300/0 conn(0x7ff44815efb0 0x7ff4480d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.055+0000 7ff4467fc640 1 -- 172.21.15.139:0/3963647197 >> v1:172.21.15.139:6789/0 conn(0x7ff44806cd80 legacy=0x7ff448099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.055+0000 7ff4467fc640 1 -- 172.21.15.139:0/3963647197 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4480d0430 con 0x7ff4480cf900 2024-04-25T09:55:55.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.056+0000 7ff4467fc640 1 --2- 172.21.15.139:0/3963647197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4480cf900 0x7ff44806c840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0097a0 tx=0x7ff43c02a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c625e6b92c08b837 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.056+0000 7ff445ffb640 1 -- 172.21.15.139:0/3963647197 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c037070 con 0x7ff4480cf900 2024-04-25T09:55:55.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.056+0000 7ff445ffb640 1 -- 172.21.15.139:0/3963647197 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c02ab10 con 0x7ff4480cf900 2024-04-25T09:55:55.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.056+0000 7ff445ffb640 1 -- 172.21.15.139:0/3963647197 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4400019f0 con 0x7ff4480cf900 2024-04-25T09:55:55.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.056+0000 7ff44d3c4640 1 -- 172.21.15.139:0/3963647197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4480cf900 msgr2=0x7ff44806c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.056+0000 7ff44d3c4640 1 --2- 172.21.15.139:0/3963647197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4480cf900 0x7ff44806c840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0097a0 tx=0x7ff43c02a080 comp rx=0 tx=0).stop 2024-04-25T09:55:55.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.056+0000 7ff44d3c4640 1 -- 172.21.15.139:0/3963647197 shutdown_connections 2024-04-25T09:55:55.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.056+0000 7ff44d3c4640 1 --2- 172.21.15.139:0/3963647197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4480cf900 0x7ff44806c840 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.056+0000 7ff44d3c4640 1 --2- 172.21.15.139:0/3963647197 >> v2:172.21.15.150:3300/0 conn(0x7ff44815efb0 0x7ff4480d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.056+0000 7ff44d3c4640 1 -- 172.21.15.139:0/3963647197 >> 172.21.15.139:0/3963647197 conn(0x7ff4480d1790 msgr2=0x7ff4480d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.056+0000 7ff44d3c4640 1 -- 172.21.15.139:0/3963647197 shutdown_connections 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.056+0000 7ff44d3c4640 1 -- 172.21.15.139:0/3963647197 wait complete. 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.058+0000 7ff44d3c4640 1 Processor -- start 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.058+0000 7ff44d3c4640 1 -- start start 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.058+0000 7ff44d3c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff44806cd80 0x7ff44815ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.058+0000 7ff44d3c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4480cf900 0x7ff4480c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.058+0000 7ff44d3c4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff44815efb0 0x7ff4480c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.058+0000 7ff44d3c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44809a2e0 con 0x7ff44806cd80 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.058+0000 7ff44d3c4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44809a160 con 0x7ff44815efb0 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.058+0000 7ff44d3c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480cbde0 con 0x7ff4480cf900 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.058+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff44806cd80 0x7ff44815ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.058+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff44806cd80 0x7ff44815ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58356/0 (socket says 172.21.15.139:58356) 2024-04-25T09:55:55.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.058+0000 7ff446ffd640 1 -- 172.21.15.139:0/1058295850 learned_addr learned my addr 172.21.15.139:0/1058295850 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:55.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.059+0000 7ff446ffd640 1 -- 172.21.15.139:0/1058295850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4480cf900 msgr2=0x7ff4480c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.059+0000 7ff446ffd640 1 --2- 172.21.15.139:0/1058295850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4480cf900 0x7ff4480c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.059+0000 7ff446ffd640 1 -- 172.21.15.139:0/1058295850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff44815efb0 msgr2=0x7ff4480c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.059+0000 7ff446ffd640 1 --2- 172.21.15.139:0/1058295850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff44815efb0 0x7ff4480c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.059+0000 7ff446ffd640 1 -- 172.21.15.139:0/1058295850 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff448160740 con 0x7ff44806cd80 2024-04-25T09:55:55.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.059+0000 7ff446ffd640 1 --2- 172.21.15.139:0/1058295850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff44806cd80 0x7ff44815ffe0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff43800e870 tx=0x7ff43800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.059+0000 7ff42f7fe640 1 -- 172.21.15.139:0/1058295850 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438017070 con 0x7ff44806cd80 2024-04-25T09:55:55.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.059+0000 7ff42f7fe640 1 -- 172.21.15.139:0/1058295850 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43800c940 con 0x7ff44806cd80 2024-04-25T09:55:55.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.059+0000 7ff42f7fe640 1 -- 172.21.15.139:0/1058295850 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800cd30 con 0x7ff44806cd80 2024-04-25T09:55:55.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.059+0000 7ff44d3c4640 1 -- 172.21.15.139:0/1058295850 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff448160a20 con 0x7ff44806cd80 2024-04-25T09:55:55.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.059+0000 7ff44d3c4640 1 -- 172.21.15.139:0/1058295850 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff448160fe0 con 0x7ff44806cd80 2024-04-25T09:55:55.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.060+0000 7ff44d3c4640 1 -- 172.21.15.139:0/1058295850 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff448074520 con 0x7ff44806cd80 2024-04-25T09:55:55.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.060+0000 7ff42f7fe640 1 -- 172.21.15.139:0/1058295850 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff43801a070 con 0x7ff44806cd80 2024-04-25T09:55:55.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.061+0000 7ff42f7fe640 1 --2- 172.21.15.139:0/1058295850 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff43406b4d0 0x7ff43406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.061+0000 7ff42f7fe640 1 -- 172.21.15.139:0/1058295850 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff43808a5e0 con 0x7ff44806cd80 2024-04-25T09:55:55.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.061+0000 7ff4467fc640 1 --2- 172.21.15.139:0/1058295850 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff43406b4d0 0x7ff43406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.061+0000 7ff4467fc640 1 --2- 172.21.15.139:0/1058295850 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff43406b4d0 0x7ff43406d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff43c034000 tx=0x7ff43c02c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.065+0000 7ff42f7fe640 1 -- 172.21.15.139:0/1058295850 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4380592e0 con 0x7ff44806cd80 2024-04-25T09:55:55.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.300+0000 7ff44d3c4640 1 -- 172.21.15.139:0/1058295850 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff4480b4880 con 0x7ff44806cd80 2024-04-25T09:55:55.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.300+0000 7ff42f7fe640 1 -- 172.21.15.139:0/1058295850 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1509 (secure 0 0 0) 0x7ff438058be0 con 0x7ff44806cd80 2024-04-25T09:55:55.303 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:55.303 INFO:teuthology.orchestra.run.smithi139.stdout:{"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":184,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038782,"num_in_osds":8,"osd_in_since":1714038773,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":92},{"state_name":"unknown","count":5}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":228519936,"bytes_avail":772865593344,"bytes_total":773094113280,"unknown_pgs_ratio":0.051546391099691391,"write_bytes_sec":3583,"read_op_per_sec":0,"write_op_per_sec":11},"fsmap":{"epoch":28,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":6312}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-04-25T09:55:40.998670+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:55:55.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.302+0000 7ff44d3c4640 1 -- 172.21.15.139:0/1058295850 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff43406b4d0 msgr2=0x7ff43406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.302+0000 7ff44d3c4640 1 --2- 172.21.15.139:0/1058295850 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff43406b4d0 0x7ff43406d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff43c034000 tx=0x7ff43c02c000 comp rx=0 tx=0).stop 2024-04-25T09:55:55.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.302+0000 7ff44d3c4640 1 -- 172.21.15.139:0/1058295850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff44806cd80 msgr2=0x7ff44815ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.302+0000 7ff44d3c4640 1 --2- 172.21.15.139:0/1058295850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff44806cd80 0x7ff44815ffe0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff43800e870 tx=0x7ff43800ed30 comp rx=0 tx=0).stop 2024-04-25T09:55:55.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.302+0000 7ff44d3c4640 1 -- 172.21.15.139:0/1058295850 shutdown_connections 2024-04-25T09:55:55.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.302+0000 7ff44d3c4640 1 --2- 172.21.15.139:0/1058295850 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff43406b4d0 0x7ff43406d990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.302+0000 7ff44d3c4640 1 --2- 172.21.15.139:0/1058295850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff44815efb0 0x7ff4480c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.302+0000 7ff44d3c4640 1 --2- 172.21.15.139:0/1058295850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4480cf900 0x7ff4480c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.302+0000 7ff44d3c4640 1 --2- 172.21.15.139:0/1058295850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff44806cd80 0x7ff44815ffe0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.302+0000 7ff44d3c4640 1 -- 172.21.15.139:0/1058295850 >> 172.21.15.139:0/1058295850 conn(0x7ff4480d1790 msgr2=0x7ff44806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:55.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.302+0000 7ff44d3c4640 1 -- 172.21.15.139:0/1058295850 shutdown_connections 2024-04-25T09:55:55.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.302+0000 7ff44d3c4640 1 -- 172.21.15.139:0/1058295850 wait complete. 2024-04-25T09:55:55.316 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:55:55.317 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:55:55.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.410+0000 7f0a0be91640 1 Processor -- start 2024-04-25T09:55:55.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.410+0000 7f0a0be91640 1 -- start start 2024-04-25T09:55:55.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a0be91640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a0406ab60 0x7f0a0406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a0be91640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0a040cf960 0x7f0a040cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a0be91640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a04070a40 con 0x7f0a0415b950 2024-04-25T09:55:55.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a0be91640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a04073170 con 0x7f0a040cf960 2024-04-25T09:55:55.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a0be91640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0406b500 con 0x7f0a0406ab60 2024-04-25T09:55:55.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a09c2d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0a0415b950 0x7f0a04072a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56344/0 (socket says 172.21.15.139:56344) 2024-04-25T09:55:55.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a09c2d640 1 -- 172.21.15.139:0/1413297508 learned_addr learned my addr 172.21.15.139:0/1413297508 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:55.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a0a42e640 1 --2- 172.21.15.139:0/1413297508 >> v2:172.21.15.139:3300/0 conn(0x7f0a040cf960 0x7f0a040cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a0942c640 1 --2- 172.21.15.139:0/1413297508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a0406ab60 0x7f0a0406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a0a42e640 1 -- 172.21.15.139:0/1413297508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a0406ab60 msgr2=0x7f0a0406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:55.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a0a42e640 1 --2- 172.21.15.139:0/1413297508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a0406ab60 0x7f0a0406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a0a42e640 1 -- 172.21.15.139:0/1413297508 >> v1:172.21.15.139:6789/0 conn(0x7f0a0415b950 legacy=0x7f0a04072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a08c2b640 1 -- 172.21.15.139:0/1413297508 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428010939 0 0) 0x7f0a04070a40 con 0x7f0a0415b950 2024-04-25T09:55:55.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.411+0000 7f0a0a42e640 1 -- 172.21.15.139:0/1413297508 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0406fc80 con 0x7f0a040cf960 2024-04-25T09:55:55.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.412+0000 7f0a0a42e640 1 --2- 172.21.15.139:0/1413297508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a040cf960 0x7f0a040cfdf0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009d90 tx=0x7f09fc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da6273b581b8c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.412+0000 7f0a08c2b640 1 -- 172.21.15.139:0/1413297508 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc007740 con 0x7f0a040cf960 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.412+0000 7f0a08c2b640 1 -- 172.21.15.139:0/1413297508 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc0078e0 con 0x7f0a040cf960 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.412+0000 7f0a08c2b640 1 -- 172.21.15.139:0/1413297508 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc007cb0 con 0x7f0a040cf960 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.412+0000 7f0a0be91640 1 -- 172.21.15.139:0/1413297508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a040cf960 msgr2=0x7f0a040cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.412+0000 7f0a0be91640 1 --2- 172.21.15.139:0/1413297508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a040cf960 0x7f0a040cfdf0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009d90 tx=0x7f09fc02a660 comp rx=0 tx=0).stop 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.413+0000 7f0a0be91640 1 -- 172.21.15.139:0/1413297508 shutdown_connections 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.413+0000 7f0a0be91640 1 --2- 172.21.15.139:0/1413297508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a040cf960 0x7f0a040cfdf0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.413+0000 7f0a0be91640 1 --2- 172.21.15.139:0/1413297508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a0406ab60 0x7f0a0406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.413+0000 7f0a0be91640 1 -- 172.21.15.139:0/1413297508 >> 172.21.15.139:0/1413297508 conn(0x7f0a040b23f0 msgr2=0x7f0a040b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.413+0000 7f0a0be91640 1 -- 172.21.15.139:0/1413297508 shutdown_connections 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.413+0000 7f0a0be91640 1 -- 172.21.15.139:0/1413297508 wait complete. 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.414+0000 7f0a0be91640 1 Processor -- start 2024-04-25T09:55:55.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.414+0000 7f0a0be91640 1 -- start start 2024-04-25T09:55:55.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.414+0000 7f0a0be91640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a0406ab60 0x7f0a041f1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.414+0000 7f0a0be91640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a040cf960 0x7f0a040ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.414+0000 7f0a0be91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a0415b950 0x7f0a040a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.414+0000 7f0a0be91640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040cda00 con 0x7f0a0415b950 2024-04-25T09:55:55.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.414+0000 7f0a0be91640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a04070240 con 0x7f0a0406ab60 2024-04-25T09:55:55.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.414+0000 7f0a0be91640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040cdce0 con 0x7f0a040cf960 2024-04-25T09:55:55.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f0a0a42e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a0415b950 0x7f0a040a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f0a0a42e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a0415b950 0x7f0a040a93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58380/0 (socket says 172.21.15.139:58380) 2024-04-25T09:55:55.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f0a0a42e640 1 -- 172.21.15.139:0/764816895 learned_addr learned my addr 172.21.15.139:0/764816895 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:55.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f0a0942c640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a040cf960 0x7f0a040ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f0a0a42e640 1 -- 172.21.15.139:0/764816895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a040cf960 msgr2=0x7f0a040ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f0a0a42e640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a040cf960 0x7f0a040ad0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f0a0a42e640 1 -- 172.21.15.139:0/764816895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a0406ab60 msgr2=0x7f0a041f1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f0a0a42e640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a0406ab60 0x7f0a041f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f0a0a42e640 1 -- 172.21.15.139:0/764816895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040a8790 con 0x7f0a0415b950 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f0a0a42e640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a0415b950 0x7f0a040a93a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f09fc02ac50 tx=0x7f09fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f09fa7fc640 1 -- 172.21.15.139:0/764816895 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc040070 con 0x7f0a0415b950 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f09fa7fc640 1 -- 172.21.15.139:0/764816895 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc031c90 con 0x7f0a0415b950 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f0a0be91640 1 -- 172.21.15.139:0/764816895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a040a9f40 con 0x7f0a0415b950 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f09fa7fc640 1 -- 172.21.15.139:0/764816895 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc045680 con 0x7f0a0415b950 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.415+0000 7f0a0be91640 1 -- 172.21.15.139:0/764816895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a040aa980 con 0x7f0a0415b950 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.416+0000 7f0a0be91640 1 -- 172.21.15.139:0/764816895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a04074520 con 0x7f0a0415b950 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.417+0000 7f09fa7fc640 1 -- 172.21.15.139:0/764816895 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09fc039040 con 0x7f0a0415b950 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.417+0000 7f09fa7fc640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f09e806b4d0 0x7f09e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.417+0000 7f09fa7fc640 1 -- 172.21.15.139:0/764816895 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09fc0abc80 con 0x7f0a0415b950 2024-04-25T09:55:55.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.417+0000 7f0a09c2d640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f09e806b4d0 0x7f09e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:55.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.417+0000 7f0a09c2d640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f09e806b4d0 0x7f09e806d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0a00007fd0 tx=0x7f0a00007960 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:55.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.421+0000 7f09fa7fc640 1 -- 172.21.15.139:0/764816895 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09fc043360 con 0x7f0a0415b950 2024-04-25T09:55:55.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.587+0000 7f0a0be91640 1 -- 172.21.15.139:0/764816895 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a0406ece0 con 0x7f09e806b4d0 2024-04-25T09:55:55.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.593+0000 7f09fa7fc640 1 -- 172.21.15.139:0/764816895 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251973 (secure 0 0 0) 0x7f0a0406ece0 con 0x7f09e806b4d0 2024-04-25T09:55:55.598 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:55.600 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:55:55.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.596+0000 7f0a0be91640 1 -- 172.21.15.139:0/764816895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f09e806b4d0 msgr2=0x7f09e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.596+0000 7f0a0be91640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f09e806b4d0 0x7f09e806d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0a00007fd0 tx=0x7f0a00007960 comp rx=0 tx=0).stop 2024-04-25T09:55:55.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.596+0000 7f0a0be91640 1 -- 172.21.15.139:0/764816895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a0415b950 msgr2=0x7f0a040a93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:55.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.596+0000 7f0a0be91640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a0415b950 0x7f0a040a93a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f09fc02ac50 tx=0x7f09fc004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:55.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.596+0000 7f0a0be91640 1 -- 172.21.15.139:0/764816895 shutdown_connections 2024-04-25T09:55:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.596+0000 7f0a0be91640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a0415b950 0x7f0a040a93a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.596+0000 7f0a0be91640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f09e806b4d0 0x7f09e806d990 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.596+0000 7f0a0be91640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a040cf960 0x7f0a040ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.596+0000 7f0a0be91640 1 --2- 172.21.15.139:0/764816895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a0406ab60 0x7f0a041f1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.596+0000 7f0a0be91640 1 -- 172.21.15.139:0/764816895 >> 172.21.15.139:0/764816895 conn(0x7f0a040b23f0 msgr2=0x7f0a040d0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.596+0000 7f0a0be91640 1 -- 172.21.15.139:0/764816895 shutdown_connections 2024-04-25T09:55:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:55.596+0000 7f0a0be91640 1 -- 172.21.15.139:0/764816895 wait complete. 2024-04-25T09:55:55.610 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-04-25T09:55:55.001140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223204,"kb_used_data":7020,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751516,"statfs":{"total":773094113280,"available":772865552384,"internally_reserved":0,"allocated":7188480,"data_stored":3843760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.725017"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302671+0000","last_change":"2024-04-25T09:55:51.303411+0000","last_active":"2024-04-25T09:55:52.302671+0000","last_peered":"2024-04-25T09:55:52.302671+0000","last_clean":"2024-04-25T09:55:52.302671+0000","last_became_active":"2024-04-25T09:55:51.303129+0000","last_became_peered":"2024-04-25T09:55:51.303129+0000","last_unstale":"2024-04-25T09:55:52.302671+0000","last_undegraded":"2024-04-25T09:55:52.302671+0000","last_fullsized":"2024-04-25T09:55:52.302671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:35.354743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302461+0000","last_change":"2024-04-25T09:55:51.301262+0000","last_active":"2024-04-25T09:55:52.302461+0000","last_peered":"2024-04-25T09:55:52.302461+0000","last_clean":"2024-04-25T09:55:52.302461+0000","last_became_active":"2024-04-25T09:55:51.300989+0000","last_became_peered":"2024-04-25T09:55:51.300989+0000","last_unstale":"2024-04-25T09:55:52.302461+0000","last_undegraded":"2024-04-25T09:55:52.302461+0000","last_fullsized":"2024-04-25T09:55:52.302461+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:49.815961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322973+0000","last_change":"2024-04-25T09:55:51.321176+0000","last_active":"2024-04-25T09:55:51.322973+0000","last_peered":"2024-04-25T09:55:51.322973+0000","last_clean":"2024-04-25T09:55:51.322973+0000","last_became_active":"2024-04-25T09:55:51.319170+0000","last_became_peered":"2024-04-25T09:55:51.319170+0000","last_unstale":"2024-04-25T09:55:51.322973+0000","last_undegraded":"2024-04-25T09:55:51.322973+0000","last_fullsized":"2024-04-25T09:55:51.322973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305476+0000","last_change":"2024-04-25T09:55:51.303613+0000","last_active":"2024-04-25T09:55:51.305476+0000","last_peered":"2024-04-25T09:55:51.305476+0000","last_clean":"2024-04-25T09:55:51.305476+0000","last_became_active":"2024-04-25T09:55:51.303266+0000","last_became_peered":"2024-04-25T09:55:51.303266+0000","last_unstale":"2024-04-25T09:55:51.305476+0000","last_undegraded":"2024-04-25T09:55:51.305476+0000","last_fullsized":"2024-04-25T09:55:51.305476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:37.811562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321590+0000","last_change":"2024-04-25T09:55:51.317347+0000","last_active":"2024-04-25T09:55:51.321590+0000","last_peered":"2024-04-25T09:55:51.321590+0000","last_clean":"2024-04-25T09:55:51.321590+0000","last_became_active":"2024-04-25T09:55:51.316810+0000","last_became_peered":"2024-04-25T09:55:51.316810+0000","last_unstale":"2024-04-25T09:55:51.321590+0000","last_undegraded":"2024-04-25T09:55:51.321590+0000","last_fullsized":"2024-04-25T09:55:51.321590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:32.510934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.320186+0000","last_change":"2024-04-25T09:55:51.319541+0000","last_active":"2024-04-25T09:55:51.320186+0000","last_peered":"2024-04-25T09:55:51.320186+0000","last_clean":"2024-04-25T09:55:51.320186+0000","last_became_active":"2024-04-25T09:55:51.319218+0000","last_became_peered":"2024-04-25T09:55:51.319218+0000","last_unstale":"2024-04-25T09:55:51.320186+0000","last_undegraded":"2024-04-25T09:55:51.320186+0000","last_fullsized":"2024-04-25T09:55:51.320186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305610+0000","last_change":"2024-04-25T09:55:51.313960+0000","last_active":"2024-04-25T09:55:52.305610+0000","last_peered":"2024-04-25T09:55:52.305610+0000","last_clean":"2024-04-25T09:55:52.305610+0000","last_became_active":"2024-04-25T09:55:51.313619+0000","last_became_peered":"2024-04-25T09:55:51.313619+0000","last_unstale":"2024-04-25T09:55:52.305610+0000","last_undegraded":"2024-04-25T09:55:52.305610+0000","last_fullsized":"2024-04-25T09:55:52.305610+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:20.633410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303121+0000","last_change":"2024-04-25T09:55:51.320026+0000","last_active":"2024-04-25T09:55:52.303121+0000","last_peered":"2024-04-25T09:55:52.303121+0000","last_clean":"2024-04-25T09:55:52.303121+0000","last_became_active":"2024-04-25T09:55:51.319656+0000","last_became_peered":"2024-04-25T09:55:51.319656+0000","last_unstale":"2024-04-25T09:55:52.303121+0000","last_undegraded":"2024-04-25T09:55:52.303121+0000","last_fullsized":"2024-04-25T09:55:52.303121+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.317650+0000","last_change":"2024-04-25T09:55:51.314164+0000","last_active":"2024-04-25T09:55:51.317650+0000","last_peered":"2024-04-25T09:55:51.317650+0000","last_clean":"2024-04-25T09:55:51.317650+0000","last_became_active":"2024-04-25T09:55:51.313888+0000","last_became_peered":"2024-04-25T09:55:51.313888+0000","last_unstale":"2024-04-25T09:55:51.317650+0000","last_undegraded":"2024-04-25T09:55:51.317650+0000","last_fullsized":"2024-04-25T09:55:51.317650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:12.483744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321807+0000","last_change":"2024-04-25T09:55:51.316804+0000","last_active":"2024-04-25T09:55:51.321807+0000","last_peered":"2024-04-25T09:55:51.321807+0000","last_clean":"2024-04-25T09:55:51.321807+0000","last_became_active":"2024-04-25T09:55:51.314608+0000","last_became_peered":"2024-04-25T09:55:51.314608+0000","last_unstale":"2024-04-25T09:55:51.321807+0000","last_undegraded":"2024-04-25T09:55:51.321807+0000","last_fullsized":"2024-04-25T09:55:51.321807+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:10.716067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.311232+0000","last_change":"2024-04-25T09:55:51.306747+0000","last_active":"2024-04-25T09:55:51.311232+0000","last_peered":"2024-04-25T09:55:51.311232+0000","last_clean":"2024-04-25T09:55:51.311232+0000","last_became_active":"2024-04-25T09:55:51.306393+0000","last_became_peered":"2024-04-25T09:55:51.306393+0000","last_unstale":"2024-04-25T09:55:51.311232+0000","last_undegraded":"2024-04-25T09:55:51.311232+0000","last_fullsized":"2024-04-25T09:55:51.311232+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.962859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303278+0000","last_change":"2024-04-25T09:55:51.311594+0000","last_active":"2024-04-25T09:55:52.303278+0000","last_peered":"2024-04-25T09:55:52.303278+0000","last_clean":"2024-04-25T09:55:52.303278+0000","last_became_active":"2024-04-25T09:55:51.311271+0000","last_became_peered":"2024-04-25T09:55:51.311271+0000","last_unstale":"2024-04-25T09:55:52.303278+0000","last_undegraded":"2024-04-25T09:55:52.303278+0000","last_fullsized":"2024-04-25T09:55:52.303278+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:44.151672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303327+0000","last_change":"2024-04-25T09:55:51.305962+0000","last_active":"2024-04-25T09:55:52.303327+0000","last_peered":"2024-04-25T09:55:52.303327+0000","last_clean":"2024-04-25T09:55:52.303327+0000","last_became_active":"2024-04-25T09:55:51.305232+0000","last_became_peered":"2024-04-25T09:55:51.305232+0000","last_unstale":"2024-04-25T09:55:52.303327+0000","last_undegraded":"2024-04-25T09:55:52.303327+0000","last_fullsized":"2024-04-25T09:55:52.303327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:26.442633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322877+0000","last_change":"2024-04-25T09:55:51.321119+0000","last_active":"2024-04-25T09:55:51.322877+0000","last_peered":"2024-04-25T09:55:51.322877+0000","last_clean":"2024-04-25T09:55:51.322877+0000","last_became_active":"2024-04-25T09:55:51.320822+0000","last_became_peered":"2024-04-25T09:55:51.320822+0000","last_unstale":"2024-04-25T09:55:51.322877+0000","last_undegraded":"2024-04-25T09:55:51.322877+0000","last_fullsized":"2024-04-25T09:55:51.322877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.318297+0000","last_change":"2024-04-25T09:55:51.313921+0000","last_active":"2024-04-25T09:55:51.318297+0000","last_peered":"2024-04-25T09:55:51.318297+0000","last_clean":"2024-04-25T09:55:51.318297+0000","last_became_active":"2024-04-25T09:55:51.313653+0000","last_became_peered":"2024-04-25T09:55:51.313653+0000","last_unstale":"2024-04-25T09:55:51.318297+0000","last_undegraded":"2024-04-25T09:55:51.318297+0000","last_fullsized":"2024-04-25T09:55:51.318297+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:11.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305541+0000","last_change":"2024-04-25T09:55:51.305285+0000","last_active":"2024-04-25T09:55:52.305541+0000","last_peered":"2024-04-25T09:55:52.305541+0000","last_clean":"2024-04-25T09:55:52.305541+0000","last_became_active":"2024-04-25T09:55:51.305018+0000","last_became_peered":"2024-04-25T09:55:51.305018+0000","last_unstale":"2024-04-25T09:55:52.305541+0000","last_undegraded":"2024-04-25T09:55:52.305541+0000","last_fullsized":"2024-04-25T09:55:52.305541+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:49.400572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302647+0000","last_change":"2024-04-25T09:55:51.306904+0000","last_active":"2024-04-25T09:55:52.302647+0000","last_peered":"2024-04-25T09:55:52.302647+0000","last_clean":"2024-04-25T09:55:52.302647+0000","last_became_active":"2024-04-25T09:55:51.306564+0000","last_became_peered":"2024-04-25T09:55:51.306564+0000","last_unstale":"2024-04-25T09:55:52.302647+0000","last_undegraded":"2024-04-25T09:55:52.302647+0000","last_fullsized":"2024-04-25T09:55:52.302647+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:25.733756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305038+0000","last_change":"2024-04-25T09:55:51.297807+0000","last_active":"2024-04-25T09:55:51.305038+0000","last_peered":"2024-04-25T09:55:51.305038+0000","last_clean":"2024-04-25T09:55:51.305038+0000","last_became_active":"2024-04-25T09:55:51.297473+0000","last_became_peered":"2024-04-25T09:55:51.297473+0000","last_unstale":"2024-04-25T09:55:51.305038+0000","last_undegraded":"2024-04-25T09:55:51.305038+0000","last_fullsized":"2024-04-25T09:55:51.305038+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:14.381048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316239+0000","last_change":"2024-04-25T09:55:51.308066+0000","last_active":"2024-04-25T09:55:51.316239+0000","last_peered":"2024-04-25T09:55:51.316239+0000","last_clean":"2024-04-25T09:55:51.316239+0000","last_became_active":"2024-04-25T09:55:51.307793+0000","last_became_peered":"2024-04-25T09:55:51.307793+0000","last_unstale":"2024-04-25T09:55:51.316239+0000","last_undegraded":"2024-04-25T09:55:51.316239+0000","last_fullsized":"2024-04-25T09:55:51.316239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.962859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.319697+0000","last_change":"2024-04-25T09:55:51.318475+0000","last_active":"2024-04-25T09:55:51.319697+0000","last_peered":"2024-04-25T09:55:51.319697+0000","last_clean":"2024-04-25T09:55:51.319697+0000","last_became_active":"2024-04-25T09:55:51.318203+0000","last_became_peered":"2024-04-25T09:55:51.318203+0000","last_unstale":"2024-04-25T09:55:51.319697+0000","last_undegraded":"2024-04-25T09:55:51.319697+0000","last_fullsized":"2024-04-25T09:55:51.319697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:23.692837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303459+0000","last_change":"2024-04-25T09:55:51.306284+0000","last_active":"2024-04-25T09:55:52.303459+0000","last_peered":"2024-04-25T09:55:52.303459+0000","last_clean":"2024-04-25T09:55:52.303459+0000","last_became_active":"2024-04-25T09:55:51.305652+0000","last_became_peered":"2024-04-25T09:55:51.305652+0000","last_unstale":"2024-04-25T09:55:52.303459+0000","last_undegraded":"2024-04-25T09:55:52.303459+0000","last_fullsized":"2024-04-25T09:55:52.303459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316425+0000","last_change":"2024-04-25T09:55:51.315278+0000","last_active":"2024-04-25T09:55:51.316425+0000","last_peered":"2024-04-25T09:55:51.316425+0000","last_clean":"2024-04-25T09:55:51.316425+0000","last_became_active":"2024-04-25T09:55:51.314991+0000","last_became_peered":"2024-04-25T09:55:51.314991+0000","last_unstale":"2024-04-25T09:55:51.316425+0000","last_undegraded":"2024-04-25T09:55:51.316425+0000","last_fullsized":"2024-04-25T09:55:51.316425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:27.808147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316412+0000","last_change":"2024-04-25T09:55:51.315286+0000","last_active":"2024-04-25T09:55:51.316412+0000","last_peered":"2024-04-25T09:55:51.316412+0000","last_clean":"2024-04-25T09:55:51.316412+0000","last_became_active":"2024-04-25T09:55:51.315029+0000","last_became_peered":"2024-04-25T09:55:51.315029+0000","last_unstale":"2024-04-25T09:55:51.316412+0000","last_undegraded":"2024-04-25T09:55:51.316412+0000","last_fullsized":"2024-04-25T09:55:51.316412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:15.037916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323697+0000","last_change":"2024-04-25T09:55:51.322865+0000","last_active":"2024-04-25T09:55:51.323697+0000","last_peered":"2024-04-25T09:55:51.323697+0000","last_clean":"2024-04-25T09:55:51.323697+0000","last_became_active":"2024-04-25T09:55:51.321449+0000","last_became_peered":"2024-04-25T09:55:51.321449+0000","last_unstale":"2024-04-25T09:55:51.323697+0000","last_undegraded":"2024-04-25T09:55:51.323697+0000","last_fullsized":"2024-04-25T09:55:51.323697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:44.151672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303636+0000","last_change":"2024-04-25T09:55:51.309760+0000","last_active":"2024-04-25T09:55:52.303636+0000","last_peered":"2024-04-25T09:55:52.303636+0000","last_clean":"2024-04-25T09:55:52.303636+0000","last_became_active":"2024-04-25T09:55:51.309495+0000","last_became_peered":"2024-04-25T09:55:51.309495+0000","last_unstale":"2024-04-25T09:55:52.303636+0000","last_undegraded":"2024-04-25T09:55:52.303636+0000","last_fullsized":"2024-04-25T09:55:52.303636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:27.808147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303476+0000","last_change":"2024-04-25T09:55:51.308968+0000","last_active":"2024-04-25T09:55:52.303476+0000","last_peered":"2024-04-25T09:55:52.303476+0000","last_clean":"2024-04-25T09:55:52.303476+0000","last_became_active":"2024-04-25T09:55:51.307967+0000","last_became_peered":"2024-04-25T09:55:51.307967+0000","last_unstale":"2024-04-25T09:55:52.303476+0000","last_undegraded":"2024-04-25T09:55:52.303476+0000","last_fullsized":"2024-04-25T09:55:52.303476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:37.934348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304536+0000","last_change":"2024-04-25T09:55:51.298586+0000","last_active":"2024-04-25T09:55:51.304536+0000","last_peered":"2024-04-25T09:55:51.304536+0000","last_clean":"2024-04-25T09:55:51.304536+0000","last_became_active":"2024-04-25T09:55:51.298272+0000","last_became_peered":"2024-04-25T09:55:51.298272+0000","last_unstale":"2024-04-25T09:55:51.304536+0000","last_undegraded":"2024-04-25T09:55:51.304536+0000","last_fullsized":"2024-04-25T09:55:51.304536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:30.656241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302366+0000","last_change":"2024-04-25T09:55:51.318634+0000","last_active":"2024-04-25T09:55:52.302366+0000","last_peered":"2024-04-25T09:55:52.302366+0000","last_clean":"2024-04-25T09:55:52.302366+0000","last_became_active":"2024-04-25T09:55:51.318347+0000","last_became_peered":"2024-04-25T09:55:51.318347+0000","last_unstale":"2024-04-25T09:55:52.302366+0000","last_undegraded":"2024-04-25T09:55:52.302366+0000","last_fullsized":"2024-04-25T09:55:52.302366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:18.171353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.314075+0000","last_change":"2024-04-25T09:55:51.312167+0000","last_active":"2024-04-25T09:55:51.314075+0000","last_peered":"2024-04-25T09:55:51.314075+0000","last_clean":"2024-04-25T09:55:51.314075+0000","last_became_active":"2024-04-25T09:55:51.311785+0000","last_became_peered":"2024-04-25T09:55:51.311785+0000","last_unstale":"2024-04-25T09:55:51.314075+0000","last_undegraded":"2024-04-25T09:55:51.314075+0000","last_fullsized":"2024-04-25T09:55:51.314075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:53.653931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303192+0000","last_change":"2024-04-25T09:55:51.320446+0000","last_active":"2024-04-25T09:55:52.303192+0000","last_peered":"2024-04-25T09:55:52.303192+0000","last_clean":"2024-04-25T09:55:52.303192+0000","last_became_active":"2024-04-25T09:55:51.320112+0000","last_became_peered":"2024-04-25T09:55:51.320112+0000","last_unstale":"2024-04-25T09:55:52.303192+0000","last_undegraded":"2024-04-25T09:55:52.303192+0000","last_fullsized":"2024-04-25T09:55:52.303192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304965+0000","last_change":"2024-04-25T09:55:51.301513+0000","last_active":"2024-04-25T09:55:51.304965+0000","last_peered":"2024-04-25T09:55:51.304965+0000","last_clean":"2024-04-25T09:55:51.304965+0000","last_became_active":"2024-04-25T09:55:51.295610+0000","last_became_peered":"2024-04-25T09:55:51.295610+0000","last_unstale":"2024-04-25T09:55:51.304965+0000","last_undegraded":"2024-04-25T09:55:51.304965+0000","last_fullsized":"2024-04-25T09:55:51.304965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.757412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.303022+0000","last_change":"2024-04-25T09:55:51.300604+0000","last_active":"2024-04-25T09:55:51.303022+0000","last_peered":"2024-04-25T09:55:51.303022+0000","last_clean":"2024-04-25T09:55:51.303022+0000","last_became_active":"2024-04-25T09:55:51.300286+0000","last_became_peered":"2024-04-25T09:55:51.300286+0000","last_unstale":"2024-04-25T09:55:51.303022+0000","last_undegraded":"2024-04-25T09:55:51.303022+0000","last_fullsized":"2024-04-25T09:55:51.303022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316388+0000","last_change":"2024-04-25T09:55:51.310013+0000","last_active":"2024-04-25T09:55:51.316388+0000","last_peered":"2024-04-25T09:55:51.316388+0000","last_clean":"2024-04-25T09:55:51.316388+0000","last_became_active":"2024-04-25T09:55:51.309619+0000","last_became_peered":"2024-04-25T09:55:51.309619+0000","last_unstale":"2024-04-25T09:55:51.316388+0000","last_undegraded":"2024-04-25T09:55:51.316388+0000","last_fullsized":"2024-04-25T09:55:51.316388+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:30.874963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302699+0000","last_change":"2024-04-25T09:55:50.305070+0000","last_active":"2024-04-25T09:55:52.302699+0000","last_peered":"2024-04-25T09:55:52.302699+0000","last_clean":"2024-04-25T09:55:52.302699+0000","last_became_active":"2024-04-25T09:55:50.304749+0000","last_became_peered":"2024-04-25T09:55:50.304749+0000","last_unstale":"2024-04-25T09:55:52.302699+0000","last_undegraded":"2024-04-25T09:55:52.302699+0000","last_fullsized":"2024-04-25T09:55:52.302699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:43.049220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.309334+0000","last_change":"2024-04-25T09:55:51.305186+0000","last_active":"2024-04-25T09:55:51.309334+0000","last_peered":"2024-04-25T09:55:51.309334+0000","last_clean":"2024-04-25T09:55:51.309334+0000","last_became_active":"2024-04-25T09:55:51.304843+0000","last_became_peered":"2024-04-25T09:55:51.304843+0000","last_unstale":"2024-04-25T09:55:51.309334+0000","last_undegraded":"2024-04-25T09:55:51.309334+0000","last_fullsized":"2024-04-25T09:55:51.309334+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:46.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323347+0000","last_change":"2024-04-25T09:55:50.305062+0000","last_active":"2024-04-25T09:55:51.323347+0000","last_peered":"2024-04-25T09:55:51.323347+0000","last_clean":"2024-04-25T09:55:51.323347+0000","last_became_active":"2024-04-25T09:55:50.304760+0000","last_became_peered":"2024-04-25T09:55:50.304760+0000","last_unstale":"2024-04-25T09:55:51.323347+0000","last_undegraded":"2024-04-25T09:55:51.323347+0000","last_fullsized":"2024-04-25T09:55:51.323347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:10.306355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.308626+0000","last_change":"2024-04-25T09:55:51.307444+0000","last_active":"2024-04-25T09:55:51.308626+0000","last_peered":"2024-04-25T09:55:51.308626+0000","last_clean":"2024-04-25T09:55:51.308626+0000","last_became_active":"2024-04-25T09:55:51.307171+0000","last_became_peered":"2024-04-25T09:55:51.307171+0000","last_unstale":"2024-04-25T09:55:51.308626+0000","last_undegraded":"2024-04-25T09:55:51.308626+0000","last_fullsized":"2024-04-25T09:55:51.308626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:26.442633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.317020+0000","last_change":"2024-04-25T09:55:50.298308+0000","last_active":"2024-04-25T09:55:51.317020+0000","last_peered":"2024-04-25T09:55:51.317020+0000","last_clean":"2024-04-25T09:55:51.317020+0000","last_became_active":"2024-04-25T09:55:50.298018+0000","last_became_peered":"2024-04-25T09:55:50.298018+0000","last_unstale":"2024-04-25T09:55:51.317020+0000","last_undegraded":"2024-04-25T09:55:51.317020+0000","last_fullsized":"2024-04-25T09:55:51.317020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:52.646641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322753+0000","last_change":"2024-04-25T09:55:51.317521+0000","last_active":"2024-04-25T09:55:51.322753+0000","last_peered":"2024-04-25T09:55:51.322753+0000","last_clean":"2024-04-25T09:55:51.322753+0000","last_became_active":"2024-04-25T09:55:51.317265+0000","last_became_peered":"2024-04-25T09:55:51.317265+0000","last_unstale":"2024-04-25T09:55:51.322753+0000","last_undegraded":"2024-04-25T09:55:51.322753+0000","last_fullsized":"2024-04-25T09:55:51.322753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316812+0000","last_change":"2024-04-25T09:55:50.301119+0000","last_active":"2024-04-25T09:55:51.316812+0000","last_peered":"2024-04-25T09:55:51.316812+0000","last_clean":"2024-04-25T09:55:51.316812+0000","last_became_active":"2024-04-25T09:55:50.300321+0000","last_became_peered":"2024-04-25T09:55:50.300321+0000","last_unstale":"2024-04-25T09:55:51.316812+0000","last_undegraded":"2024-04-25T09:55:51.316812+0000","last_fullsized":"2024-04-25T09:55:51.316812+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:09.708778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305589+0000","last_change":"2024-04-25T09:55:51.297568+0000","last_active":"2024-04-25T09:55:52.305589+0000","last_peered":"2024-04-25T09:55:52.305589+0000","last_clean":"2024-04-25T09:55:52.305589+0000","last_became_active":"2024-04-25T09:55:51.297305+0000","last_became_peered":"2024-04-25T09:55:51.297305+0000","last_unstale":"2024-04-25T09:55:52.305589+0000","last_undegraded":"2024-04-25T09:55:52.305589+0000","last_fullsized":"2024-04-25T09:55:52.305589+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:00.789145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.311448+0000","last_change":"2024-04-25T09:55:50.297259+0000","last_active":"2024-04-25T09:55:51.311448+0000","last_peered":"2024-04-25T09:55:51.311448+0000","last_clean":"2024-04-25T09:55:51.311448+0000","last_became_active":"2024-04-25T09:55:50.296982+0000","last_became_peered":"2024-04-25T09:55:50.296982+0000","last_unstale":"2024-04-25T09:55:51.311448+0000","last_undegraded":"2024-04-25T09:55:51.311448+0000","last_fullsized":"2024-04-25T09:55:51.311448+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:27.161990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302048+0000","last_change":"2024-04-25T09:55:51.309288+0000","last_active":"2024-04-25T09:55:52.302048+0000","last_peered":"2024-04-25T09:55:52.302048+0000","last_clean":"2024-04-25T09:55:52.302048+0000","last_became_active":"2024-04-25T09:55:51.309018+0000","last_became_peered":"2024-04-25T09:55:51.309018+0000","last_unstale":"2024-04-25T09:55:52.302048+0000","last_undegraded":"2024-04-25T09:55:52.302048+0000","last_fullsized":"2024-04-25T09:55:52.302048+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:47.578362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.310309+0000","last_change":"2024-04-25T09:55:50.298200+0000","last_active":"2024-04-25T09:55:51.310309+0000","last_peered":"2024-04-25T09:55:51.310309+0000","last_clean":"2024-04-25T09:55:51.310309+0000","last_became_active":"2024-04-25T09:55:50.297950+0000","last_became_peered":"2024-04-25T09:55:50.297950+0000","last_unstale":"2024-04-25T09:55:51.310309+0000","last_undegraded":"2024-04-25T09:55:51.310309+0000","last_fullsized":"2024-04-25T09:55:51.310309+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:50.309907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316534+0000","last_change":"2024-04-25T09:55:51.309251+0000","last_active":"2024-04-25T09:55:51.316534+0000","last_peered":"2024-04-25T09:55:51.316534+0000","last_clean":"2024-04-25T09:55:51.316534+0000","last_became_active":"2024-04-25T09:55:51.308805+0000","last_became_peered":"2024-04-25T09:55:51.308805+0000","last_unstale":"2024-04-25T09:55:51.316534+0000","last_undegraded":"2024-04-25T09:55:51.316534+0000","last_fullsized":"2024-04-25T09:55:51.316534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:21.995603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291503+0000","last_change":"2024-04-25T09:55:50.299777+0000","last_active":"2024-04-25T09:55:51.291503+0000","last_peered":"2024-04-25T09:55:51.291503+0000","last_clean":"2024-04-25T09:55:51.291503+0000","last_became_active":"2024-04-25T09:55:50.299522+0000","last_became_peered":"2024-04-25T09:55:50.299522+0000","last_unstale":"2024-04-25T09:55:51.291503+0000","last_undegraded":"2024-04-25T09:55:51.291503+0000","last_fullsized":"2024-04-25T09:55:51.291503+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:09.708778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.298022+0000","last_change":"2024-04-25T09:55:51.296452+0000","last_active":"2024-04-25T09:55:51.298022+0000","last_peered":"2024-04-25T09:55:51.298022+0000","last_clean":"2024-04-25T09:55:51.298022+0000","last_became_active":"2024-04-25T09:55:51.296189+0000","last_became_peered":"2024-04-25T09:55:51.296189+0000","last_unstale":"2024-04-25T09:55:51.298022+0000","last_undegraded":"2024-04-25T09:55:51.298022+0000","last_fullsized":"2024-04-25T09:55:51.298022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.296447+0000","last_change":"2024-04-25T09:55:50.305088+0000","last_active":"2024-04-25T09:55:51.296447+0000","last_peered":"2024-04-25T09:55:51.296447+0000","last_clean":"2024-04-25T09:55:51.296447+0000","last_became_active":"2024-04-25T09:55:50.304789+0000","last_became_peered":"2024-04-25T09:55:50.304789+0000","last_unstale":"2024-04-25T09:55:51.296447+0000","last_undegraded":"2024-04-25T09:55:51.296447+0000","last_fullsized":"2024-04-25T09:55:51.296447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:28.750122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302469+0000","last_change":"2024-04-25T09:55:51.295832+0000","last_active":"2024-04-25T09:55:52.302469+0000","last_peered":"2024-04-25T09:55:52.302469+0000","last_clean":"2024-04-25T09:55:52.302469+0000","last_became_active":"2024-04-25T09:55:51.295549+0000","last_became_peered":"2024-04-25T09:55:51.295549+0000","last_unstale":"2024-04-25T09:55:52.302469+0000","last_undegraded":"2024-04-25T09:55:52.302469+0000","last_fullsized":"2024-04-25T09:55:52.302469+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:31.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.308605+0000","last_change":"2024-04-25T09:55:50.307313+0000","last_active":"2024-04-25T09:55:51.308605+0000","last_peered":"2024-04-25T09:55:51.308605+0000","last_clean":"2024-04-25T09:55:51.308605+0000","last_became_active":"2024-04-25T09:55:50.306998+0000","last_became_peered":"2024-04-25T09:55:50.306998+0000","last_unstale":"2024-04-25T09:55:51.308605+0000","last_undegraded":"2024-04-25T09:55:51.308605+0000","last_fullsized":"2024-04-25T09:55:51.308605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:13.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305345+0000","last_change":"2024-04-25T09:55:51.302618+0000","last_active":"2024-04-25T09:55:51.305345+0000","last_peered":"2024-04-25T09:55:51.305345+0000","last_clean":"2024-04-25T09:55:51.305345+0000","last_became_active":"2024-04-25T09:55:51.301598+0000","last_became_peered":"2024-04-25T09:55:51.301598+0000","last_unstale":"2024-04-25T09:55:51.305345+0000","last_undegraded":"2024-04-25T09:55:51.305345+0000","last_fullsized":"2024-04-25T09:55:51.305345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:37.934348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.292155+0000","last_change":"2024-04-25T09:55:50.300463+0000","last_active":"2024-04-25T09:55:51.292155+0000","last_peered":"2024-04-25T09:55:51.292155+0000","last_clean":"2024-04-25T09:55:51.292155+0000","last_became_active":"2024-04-25T09:55:50.299576+0000","last_became_peered":"2024-04-25T09:55:50.299576+0000","last_unstale":"2024-04-25T09:55:51.292155+0000","last_undegraded":"2024-04-25T09:55:51.292155+0000","last_fullsized":"2024-04-25T09:55:51.292155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:56.828907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307569+0000","last_change":"2024-04-25T09:55:51.306375+0000","last_active":"2024-04-25T09:55:51.307569+0000","last_peered":"2024-04-25T09:55:51.307569+0000","last_clean":"2024-04-25T09:55:51.307569+0000","last_became_active":"2024-04-25T09:55:51.305748+0000","last_became_peered":"2024-04-25T09:55:51.305748+0000","last_unstale":"2024-04-25T09:55:51.307569+0000","last_undegraded":"2024-04-25T09:55:51.307569+0000","last_fullsized":"2024-04-25T09:55:51.307569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:37.811562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322267+0000","last_change":"2024-04-25T09:55:50.307971+0000","last_active":"2024-04-25T09:55:51.322267+0000","last_peered":"2024-04-25T09:55:51.322267+0000","last_clean":"2024-04-25T09:55:51.322267+0000","last_became_active":"2024-04-25T09:55:50.307650+0000","last_became_peered":"2024-04-25T09:55:50.307650+0000","last_unstale":"2024-04-25T09:55:51.322267+0000","last_undegraded":"2024-04-25T09:55:51.322267+0000","last_fullsized":"2024-04-25T09:55:51.322267+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:30.506756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305389+0000","last_change":"2024-04-25T09:55:51.302214+0000","last_active":"2024-04-25T09:55:51.305389+0000","last_peered":"2024-04-25T09:55:51.305389+0000","last_clean":"2024-04-25T09:55:51.305389+0000","last_became_active":"2024-04-25T09:55:51.301887+0000","last_became_peered":"2024-04-25T09:55:51.301887+0000","last_unstale":"2024-04-25T09:55:51.305389+0000","last_undegraded":"2024-04-25T09:55:51.305389+0000","last_fullsized":"2024-04-25T09:55:51.305389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.447273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302496+0000","last_change":"2024-04-25T09:55:50.299295+0000","last_active":"2024-04-25T09:55:52.302496+0000","last_peered":"2024-04-25T09:55:52.302496+0000","last_clean":"2024-04-25T09:55:52.302496+0000","last_became_active":"2024-04-25T09:55:50.296672+0000","last_became_peered":"2024-04-25T09:55:50.296672+0000","last_unstale":"2024-04-25T09:55:52.302496+0000","last_undegraded":"2024-04-25T09:55:52.302496+0000","last_fullsized":"2024-04-25T09:55:52.302496+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:05.248918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315783+0000","last_change":"2024-04-25T09:55:51.312207+0000","last_active":"2024-04-25T09:55:51.315783+0000","last_peered":"2024-04-25T09:55:51.315783+0000","last_clean":"2024-04-25T09:55:51.315783+0000","last_became_active":"2024-04-25T09:55:51.311902+0000","last_became_peered":"2024-04-25T09:55:51.311902+0000","last_unstale":"2024-04-25T09:55:51.315783+0000","last_undegraded":"2024-04-25T09:55:51.315783+0000","last_fullsized":"2024-04-25T09:55:51.315783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.335930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322307+0000","last_change":"2024-04-25T09:55:50.307406+0000","last_active":"2024-04-25T09:55:51.322307+0000","last_peered":"2024-04-25T09:55:51.322307+0000","last_clean":"2024-04-25T09:55:51.322307+0000","last_became_active":"2024-04-25T09:55:50.306864+0000","last_became_peered":"2024-04-25T09:55:50.306864+0000","last_unstale":"2024-04-25T09:55:51.322307+0000","last_undegraded":"2024-04-25T09:55:51.322307+0000","last_fullsized":"2024-04-25T09:55:51.322307+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:54.613991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303023+0000","last_change":"2024-04-25T09:55:51.306953+0000","last_active":"2024-04-25T09:55:52.303023+0000","last_peered":"2024-04-25T09:55:52.303023+0000","last_clean":"2024-04-25T09:55:52.303023+0000","last_became_active":"2024-04-25T09:55:51.306664+0000","last_became_peered":"2024-04-25T09:55:51.306664+0000","last_unstale":"2024-04-25T09:55:52.303023+0000","last_undegraded":"2024-04-25T09:55:52.303023+0000","last_fullsized":"2024-04-25T09:55:52.303023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303002+0000","last_change":"2024-04-25T09:55:50.303832+0000","last_active":"2024-04-25T09:55:52.303002+0000","last_peered":"2024-04-25T09:55:52.303002+0000","last_clean":"2024-04-25T09:55:52.303002+0000","last_became_active":"2024-04-25T09:55:50.303513+0000","last_became_peered":"2024-04-25T09:55:50.303513+0000","last_unstale":"2024-04-25T09:55:52.303002+0000","last_undegraded":"2024-04-25T09:55:52.303002+0000","last_fullsized":"2024-04-25T09:55:52.303002+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:16.796943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315474+0000","last_change":"2024-04-25T09:55:51.306576+0000","last_active":"2024-04-25T09:55:51.315474+0000","last_peered":"2024-04-25T09:55:51.315474+0000","last_clean":"2024-04-25T09:55:51.315474+0000","last_became_active":"2024-04-25T09:55:51.305692+0000","last_became_peered":"2024-04-25T09:55:51.305692+0000","last_unstale":"2024-04-25T09:55:51.315474+0000","last_undegraded":"2024-04-25T09:55:51.315474+0000","last_fullsized":"2024-04-25T09:55:51.315474+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:46.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304282+0000","last_change":"2024-04-25T09:55:50.307077+0000","last_active":"2024-04-25T09:55:51.304282+0000","last_peered":"2024-04-25T09:55:51.304282+0000","last_clean":"2024-04-25T09:55:51.304282+0000","last_became_active":"2024-04-25T09:55:50.306743+0000","last_became_peered":"2024-04-25T09:55:50.306743+0000","last_unstale":"2024-04-25T09:55:51.304282+0000","last_undegraded":"2024-04-25T09:55:51.304282+0000","last_fullsized":"2024-04-25T09:55:51.304282+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:34.347454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302240+0000","last_change":"2024-04-25T09:55:51.315727+0000","last_active":"2024-04-25T09:55:52.302240+0000","last_peered":"2024-04-25T09:55:52.302240+0000","last_clean":"2024-04-25T09:55:52.302240+0000","last_became_active":"2024-04-25T09:55:51.315426+0000","last_became_peered":"2024-04-25T09:55:51.315426+0000","last_unstale":"2024-04-25T09:55:52.302240+0000","last_undegraded":"2024-04-25T09:55:52.302240+0000","last_fullsized":"2024-04-25T09:55:52.302240+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:08.905576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"61'8","reported_seq":20,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.324184+0000","last_change":"2024-04-25T09:55:50.302489+0000","last_active":"2024-04-25T09:55:51.324184+0000","last_peered":"2024-04-25T09:55:51.324184+0000","last_clean":"2024-04-25T09:55:51.324184+0000","last_became_active":"2024-04-25T09:55:50.302222+0000","last_became_peered":"2024-04-25T09:55:50.302222+0000","last_unstale":"2024-04-25T09:55:51.324184+0000","last_undegraded":"2024-04-25T09:55:51.324184+0000","last_fullsized":"2024-04-25T09:55:51.324184+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:34.805327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.312652+0000","last_change":"2024-04-25T09:55:51.310660+0000","last_active":"2024-04-25T09:55:51.312652+0000","last_peered":"2024-04-25T09:55:51.312652+0000","last_clean":"2024-04-25T09:55:51.312652+0000","last_became_active":"2024-04-25T09:55:51.310407+0000","last_became_peered":"2024-04-25T09:55:51.310407+0000","last_unstale":"2024-04-25T09:55:51.312652+0000","last_undegraded":"2024-04-25T09:55:51.312652+0000","last_fullsized":"2024-04-25T09:55:51.312652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:41.794719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303710+0000","last_change":"2024-04-25T09:55:50.303136+0000","last_active":"2024-04-25T09:55:52.303710+0000","last_peered":"2024-04-25T09:55:52.303710+0000","last_clean":"2024-04-25T09:55:52.303710+0000","last_became_active":"2024-04-25T09:55:50.302805+0000","last_became_peered":"2024-04-25T09:55:50.302805+0000","last_unstale":"2024-04-25T09:55:52.303710+0000","last_undegraded":"2024-04-25T09:55:52.303710+0000","last_fullsized":"2024-04-25T09:55:52.303710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.898286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321648+0000","last_change":"2024-04-25T09:55:51.318231+0000","last_active":"2024-04-25T09:55:51.321648+0000","last_peered":"2024-04-25T09:55:51.321648+0000","last_clean":"2024-04-25T09:55:51.321648+0000","last_became_active":"2024-04-25T09:55:51.317964+0000","last_became_peered":"2024-04-25T09:55:51.317964+0000","last_unstale":"2024-04-25T09:55:51.321648+0000","last_undegraded":"2024-04-25T09:55:51.321648+0000","last_fullsized":"2024-04-25T09:55:51.321648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:01.234580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321558+0000","last_change":"2024-04-25T09:55:50.307310+0000","last_active":"2024-04-25T09:55:51.321558+0000","last_peered":"2024-04-25T09:55:51.321558+0000","last_clean":"2024-04-25T09:55:51.321558+0000","last_became_active":"2024-04-25T09:55:50.306877+0000","last_became_peered":"2024-04-25T09:55:50.306877+0000","last_unstale":"2024-04-25T09:55:51.321558+0000","last_undegraded":"2024-04-25T09:55:51.321558+0000","last_fullsized":"2024-04-25T09:55:51.321558+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.808671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321032+0000","last_change":"2024-04-25T09:55:51.319026+0000","last_active":"2024-04-25T09:55:51.321032+0000","last_peered":"2024-04-25T09:55:51.321032+0000","last_clean":"2024-04-25T09:55:51.321032+0000","last_became_active":"2024-04-25T09:55:51.318720+0000","last_became_peered":"2024-04-25T09:55:51.318720+0000","last_unstale":"2024-04-25T09:55:51.321032+0000","last_undegraded":"2024-04-25T09:55:51.321032+0000","last_fullsized":"2024-04-25T09:55:51.321032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:21.995603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323351+0000","last_change":"2024-04-25T09:55:50.309565+0000","last_active":"2024-04-25T09:55:51.323351+0000","last_peered":"2024-04-25T09:55:51.323351+0000","last_clean":"2024-04-25T09:55:51.323351+0000","last_became_active":"2024-04-25T09:55:50.309335+0000","last_became_peered":"2024-04-25T09:55:50.309335+0000","last_unstale":"2024-04-25T09:55:51.323351+0000","last_undegraded":"2024-04-25T09:55:51.323351+0000","last_fullsized":"2024-04-25T09:55:51.323351+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:24.726466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.306954+0000","last_change":"2024-04-25T09:55:51.304138+0000","last_active":"2024-04-25T09:55:51.306954+0000","last_peered":"2024-04-25T09:55:51.306954+0000","last_clean":"2024-04-25T09:55:51.306954+0000","last_became_active":"2024-04-25T09:55:51.303842+0000","last_became_peered":"2024-04-25T09:55:51.303842+0000","last_unstale":"2024-04-25T09:55:51.306954+0000","last_undegraded":"2024-04-25T09:55:51.306954+0000","last_fullsized":"2024-04-25T09:55:51.306954+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.447273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291578+0000","last_change":"2024-04-25T09:55:50.295568+0000","last_active":"2024-04-25T09:55:51.291578+0000","last_peered":"2024-04-25T09:55:51.291578+0000","last_clean":"2024-04-25T09:55:51.291578+0000","last_became_active":"2024-04-25T09:55:50.295151+0000","last_became_peered":"2024-04-25T09:55:50.295151+0000","last_unstale":"2024-04-25T09:55:51.291578+0000","last_undegraded":"2024-04-25T09:55:51.291578+0000","last_fullsized":"2024-04-25T09:55:51.291578+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:43.198244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.299612+0000","last_change":"2024-04-25T09:55:51.297690+0000","last_active":"2024-04-25T09:55:51.299612+0000","last_peered":"2024-04-25T09:55:51.299612+0000","last_clean":"2024-04-25T09:55:51.299612+0000","last_became_active":"2024-04-25T09:55:51.297427+0000","last_became_peered":"2024-04-25T09:55:51.297427+0000","last_unstale":"2024-04-25T09:55:51.299612+0000","last_undegraded":"2024-04-25T09:55:51.299612+0000","last_fullsized":"2024-04-25T09:55:51.299612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.318135+0000","last_change":"2024-04-25T09:55:50.299667+0000","last_active":"2024-04-25T09:55:51.318135+0000","last_peered":"2024-04-25T09:55:51.318135+0000","last_clean":"2024-04-25T09:55:51.318135+0000","last_became_active":"2024-04-25T09:55:50.298743+0000","last_became_peered":"2024-04-25T09:55:50.298743+0000","last_unstale":"2024-04-25T09:55:51.318135+0000","last_undegraded":"2024-04-25T09:55:51.318135+0000","last_fullsized":"2024-04-25T09:55:51.318135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:13.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303024+0000","last_change":"2024-04-25T09:55:51.307575+0000","last_active":"2024-04-25T09:55:52.303024+0000","last_peered":"2024-04-25T09:55:52.303024+0000","last_clean":"2024-04-25T09:55:52.303024+0000","last_became_active":"2024-04-25T09:55:51.307279+0000","last_became_peered":"2024-04-25T09:55:51.307279+0000","last_unstale":"2024-04-25T09:55:52.303024+0000","last_undegraded":"2024-04-25T09:55:52.303024+0000","last_fullsized":"2024-04-25T09:55:52.303024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:10.716067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303126+0000","last_change":"2024-04-25T09:55:50.300890+0000","last_active":"2024-04-25T09:55:52.303126+0000","last_peered":"2024-04-25T09:55:52.303126+0000","last_clean":"2024-04-25T09:55:52.303126+0000","last_became_active":"2024-04-25T09:55:50.300508+0000","last_became_peered":"2024-04-25T09:55:50.300508+0000","last_unstale":"2024-04-25T09:55:52.303126+0000","last_undegraded":"2024-04-25T09:55:52.303126+0000","last_fullsized":"2024-04-25T09:55:52.303126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.808671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316507+0000","last_change":"2024-04-25T09:55:51.309120+0000","last_active":"2024-04-25T09:55:51.316507+0000","last_peered":"2024-04-25T09:55:51.316507+0000","last_clean":"2024-04-25T09:55:51.316507+0000","last_became_active":"2024-04-25T09:55:51.308443+0000","last_became_peered":"2024-04-25T09:55:51.308443+0000","last_unstale":"2024-04-25T09:55:51.316507+0000","last_undegraded":"2024-04-25T09:55:51.316507+0000","last_fullsized":"2024-04-25T09:55:51.316507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.335930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.294489+0000","last_change":"2024-04-25T09:55:50.305023+0000","last_active":"2024-04-25T09:55:51.294489+0000","last_peered":"2024-04-25T09:55:51.294489+0000","last_clean":"2024-04-25T09:55:51.294489+0000","last_became_active":"2024-04-25T09:55:50.304744+0000","last_became_peered":"2024-04-25T09:55:50.304744+0000","last_unstale":"2024-04-25T09:55:51.294489+0000","last_undegraded":"2024-04-25T09:55:51.294489+0000","last_fullsized":"2024-04-25T09:55:51.294489+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.787430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307022+0000","last_change":"2024-04-25T09:55:51.304569+0000","last_active":"2024-04-25T09:55:51.307022+0000","last_peered":"2024-04-25T09:55:51.307022+0000","last_clean":"2024-04-25T09:55:51.307022+0000","last_became_active":"2024-04-25T09:55:51.304198+0000","last_became_peered":"2024-04-25T09:55:51.304198+0000","last_unstale":"2024-04-25T09:55:51.307022+0000","last_undegraded":"2024-04-25T09:55:51.307022+0000","last_fullsized":"2024-04-25T09:55:51.307022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303857+0000","last_change":"2024-04-25T09:55:50.305335+0000","last_active":"2024-04-25T09:55:52.303857+0000","last_peered":"2024-04-25T09:55:52.303857+0000","last_clean":"2024-04-25T09:55:52.303857+0000","last_became_active":"2024-04-25T09:55:50.304742+0000","last_became_peered":"2024-04-25T09:55:50.304742+0000","last_unstale":"2024-04-25T09:55:52.303857+0000","last_undegraded":"2024-04-25T09:55:52.303857+0000","last_fullsized":"2024-04-25T09:55:52.303857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:17.164064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305756+0000","last_change":"2024-04-25T09:55:51.304994+0000","last_active":"2024-04-25T09:55:52.305756+0000","last_peered":"2024-04-25T09:55:52.305756+0000","last_clean":"2024-04-25T09:55:52.305756+0000","last_became_active":"2024-04-25T09:55:51.304728+0000","last_became_peered":"2024-04-25T09:55:51.304728+0000","last_unstale":"2024-04-25T09:55:52.305756+0000","last_undegraded":"2024-04-25T09:55:52.305756+0000","last_fullsized":"2024-04-25T09:55:52.305756+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:56.631976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"62'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305088+0000","last_change":"2024-04-25T09:55:50.299644+0000","last_active":"2024-04-25T09:55:52.305088+0000","last_peered":"2024-04-25T09:55:52.305088+0000","last_clean":"2024-04-25T09:55:52.305088+0000","last_became_active":"2024-04-25T09:55:50.299368+0000","last_became_peered":"2024-04-25T09:55:50.299368+0000","last_unstale":"2024-04-25T09:55:52.305088+0000","last_undegraded":"2024-04-25T09:55:52.305088+0000","last_fullsized":"2024-04-25T09:55:52.305088+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:46.571072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303546+0000","last_change":"2024-04-25T09:55:51.308785+0000","last_active":"2024-04-25T09:55:52.303546+0000","last_peered":"2024-04-25T09:55:52.303546+0000","last_clean":"2024-04-25T09:55:52.303546+0000","last_became_active":"2024-04-25T09:55:51.308476+0000","last_became_peered":"2024-04-25T09:55:51.308476+0000","last_unstale":"2024-04-25T09:55:52.303546+0000","last_undegraded":"2024-04-25T09:55:52.303546+0000","last_fullsized":"2024-04-25T09:55:52.303546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:15.037916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.295008+0000","last_change":"2024-04-25T09:55:50.298279+0000","last_active":"2024-04-25T09:55:51.295008+0000","last_peered":"2024-04-25T09:55:51.295008+0000","last_clean":"2024-04-25T09:55:51.295008+0000","last_became_active":"2024-04-25T09:55:50.298007+0000","last_became_peered":"2024-04-25T09:55:50.298007+0000","last_unstale":"2024-04-25T09:55:51.295008+0000","last_undegraded":"2024-04-25T09:55:51.295008+0000","last_fullsized":"2024-04-25T09:55:51.295008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:11.476455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321671+0000","last_change":"2024-04-25T09:55:51.318440+0000","last_active":"2024-04-25T09:55:51.321671+0000","last_peered":"2024-04-25T09:55:51.321671+0000","last_clean":"2024-04-25T09:55:51.321671+0000","last_became_active":"2024-04-25T09:55:51.318162+0000","last_became_peered":"2024-04-25T09:55:51.318162+0000","last_unstale":"2024-04-25T09:55:51.321671+0000","last_undegraded":"2024-04-25T09:55:51.321671+0000","last_fullsized":"2024-04-25T09:55:51.321671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:44.056510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302555+0000","last_change":"2024-04-25T09:55:50.296784+0000","last_active":"2024-04-25T09:55:52.302555+0000","last_peered":"2024-04-25T09:55:52.302555+0000","last_clean":"2024-04-25T09:55:52.302555+0000","last_became_active":"2024-04-25T09:55:50.296190+0000","last_became_peered":"2024-04-25T09:55:50.296190+0000","last_unstale":"2024-04-25T09:55:52.302555+0000","last_undegraded":"2024-04-25T09:55:52.302555+0000","last_fullsized":"2024-04-25T09:55:52.302555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:31.503645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316493+0000","last_change":"2024-04-25T09:55:51.310409+0000","last_active":"2024-04-25T09:55:51.316493+0000","last_peered":"2024-04-25T09:55:51.316493+0000","last_clean":"2024-04-25T09:55:51.316493+0000","last_became_active":"2024-04-25T09:55:51.310147+0000","last_became_peered":"2024-04-25T09:55:51.310147+0000","last_unstale":"2024-04-25T09:55:51.316493+0000","last_undegraded":"2024-04-25T09:55:51.316493+0000","last_fullsized":"2024-04-25T09:55:51.316493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:06.256207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.306556+0000","last_change":"2024-04-25T09:55:50.307476+0000","last_active":"2024-04-25T09:55:51.306556+0000","last_peered":"2024-04-25T09:55:51.306556+0000","last_clean":"2024-04-25T09:55:51.306556+0000","last_became_active":"2024-04-25T09:55:50.307206+0000","last_became_peered":"2024-04-25T09:55:50.307206+0000","last_unstale":"2024-04-25T09:55:51.306556+0000","last_undegraded":"2024-04-25T09:55:51.306556+0000","last_fullsized":"2024-04-25T09:55:51.306556+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:05.108894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315150+0000","last_change":"2024-04-25T09:55:51.313741+0000","last_active":"2024-04-25T09:55:51.315150+0000","last_peered":"2024-04-25T09:55:51.315150+0000","last_clean":"2024-04-25T09:55:51.315150+0000","last_became_active":"2024-04-25T09:55:51.313475+0000","last_became_peered":"2024-04-25T09:55:51.313475+0000","last_unstale":"2024-04-25T09:55:51.315150+0000","last_undegraded":"2024-04-25T09:55:51.315150+0000","last_fullsized":"2024-04-25T09:55:51.315150+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:07.129449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291688+0000","last_change":"2024-04-25T09:55:50.300744+0000","last_active":"2024-04-25T09:55:51.291688+0000","last_peered":"2024-04-25T09:55:51.291688+0000","last_clean":"2024-04-25T09:55:51.291688+0000","last_became_active":"2024-04-25T09:55:50.300415+0000","last_became_peered":"2024-04-25T09:55:50.300415+0000","last_unstale":"2024-04-25T09:55:51.291688+0000","last_undegraded":"2024-04-25T09:55:51.291688+0000","last_fullsized":"2024-04-25T09:55:51.291688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.144383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302118+0000","last_change":"2024-04-25T09:55:51.307744+0000","last_active":"2024-04-25T09:55:52.302118+0000","last_peered":"2024-04-25T09:55:52.302118+0000","last_clean":"2024-04-25T09:55:52.302118+0000","last_became_active":"2024-04-25T09:55:51.307427+0000","last_became_peered":"2024-04-25T09:55:51.307427+0000","last_unstale":"2024-04-25T09:55:52.302118+0000","last_undegraded":"2024-04-25T09:55:52.302118+0000","last_fullsized":"2024-04-25T09:55:52.302118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:17.804233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291814+0000","last_change":"2024-04-25T09:55:50.301696+0000","last_active":"2024-04-25T09:55:51.291814+0000","last_peered":"2024-04-25T09:55:51.291814+0000","last_clean":"2024-04-25T09:55:51.291814+0000","last_became_active":"2024-04-25T09:55:50.299321+0000","last_became_peered":"2024-04-25T09:55:50.299321+0000","last_unstale":"2024-04-25T09:55:51.291814+0000","last_undegraded":"2024-04-25T09:55:51.291814+0000","last_fullsized":"2024-04-25T09:55:51.291814+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:25.435344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305700+0000","last_change":"2024-04-25T09:55:51.305887+0000","last_active":"2024-04-25T09:55:52.305700+0000","last_peered":"2024-04-25T09:55:52.305700+0000","last_clean":"2024-04-25T09:55:52.305700+0000","last_became_active":"2024-04-25T09:55:51.305626+0000","last_became_peered":"2024-04-25T09:55:51.305626+0000","last_unstale":"2024-04-25T09:55:52.305700+0000","last_undegraded":"2024-04-25T09:55:52.305700+0000","last_fullsized":"2024-04-25T09:55:52.305700+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:28.074116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.301997+0000","last_change":"2024-04-25T09:55:50.301123+0000","last_active":"2024-04-25T09:55:52.301997+0000","last_peered":"2024-04-25T09:55:52.301997+0000","last_clean":"2024-04-25T09:55:52.301997+0000","last_became_active":"2024-04-25T09:55:50.300861+0000","last_became_peered":"2024-04-25T09:55:50.300861+0000","last_unstale":"2024-04-25T09:55:52.301997+0000","last_undegraded":"2024-04-25T09:55:52.301997+0000","last_fullsized":"2024-04-25T09:55:52.301997+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:00.227291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316474+0000","last_change":"2024-04-25T09:55:51.309200+0000","last_active":"2024-04-25T09:55:51.316474+0000","last_peered":"2024-04-25T09:55:51.316474+0000","last_clean":"2024-04-25T09:55:51.316474+0000","last_became_active":"2024-04-25T09:55:51.308915+0000","last_became_peered":"2024-04-25T09:55:51.308915+0000","last_unstale":"2024-04-25T09:55:51.316474+0000","last_undegraded":"2024-04-25T09:55:51.316474+0000","last_fullsized":"2024-04-25T09:55:51.316474+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:07.129449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291165+0000","last_change":"2024-04-25T09:55:50.298052+0000","last_active":"2024-04-25T09:55:51.291165+0000","last_peered":"2024-04-25T09:55:51.291165+0000","last_clean":"2024-04-25T09:55:51.291165+0000","last_became_active":"2024-04-25T09:55:50.297769+0000","last_became_peered":"2024-04-25T09:55:50.297769+0000","last_unstale":"2024-04-25T09:55:51.291165+0000","last_undegraded":"2024-04-25T09:55:51.291165+0000","last_fullsized":"2024-04-25T09:55:51.291165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:11.476455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":244,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307824+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:55:51.307824+0000","last_peered":"2024-04-25T09:55:51.307824+0000","last_clean":"2024-04-25T09:55:51.307824+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:55:51.307824+0000","last_undegraded":"2024-04-25T09:55:51.307824+0000","last_fullsized":"2024-04-25T09:55:51.307824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":1347584,"data_stored":920910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":741376,"data_stored":329800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":753664,"data_stored":336480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":749568,"data_stored":331378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":737280,"data_stored":329766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":336458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":778240,"data_stored":338834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1331200,"data_stored":920134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:55:56.544 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T09:55:56.541+0000 7f41d3e3b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:55:58.614 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:55:58.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.720+0000 7f058b789640 1 Processor -- start 2024-04-25T09:55:58.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.721+0000 7f058b789640 1 -- start start 2024-04-25T09:55:58.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.721+0000 7f058b789640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f058406be40 0x7f05840acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:58.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.721+0000 7f058b789640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f058406c220 0x7f058415d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.721+0000 7f058b789640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05840ad2d0 con 0x7f0584072520 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.721+0000 7f058b789640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05840b2de0 con 0x7f058406be40 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.722+0000 7f058b789640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05840b30c0 con 0x7f058406c220 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.722+0000 7f0589525640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f058406be40 0x7f05840acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.722+0000 7f0589d26640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0584072520 0x7f0584161760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:44196/0 (socket says 172.21.15.139:44196) 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.722+0000 7f0589d26640 1 -- 172.21.15.139:0/509037034 learned_addr learned my addr 172.21.15.139:0/509037034 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.722+0000 7f0588d24640 1 --2- 172.21.15.139:0/509037034 >> v2:172.21.15.150:3300/0 conn(0x7f058406c220 0x7f058415d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.722+0000 7f0589525640 1 -- 172.21.15.139:0/509037034 >> v2:172.21.15.150:3300/0 conn(0x7f058406c220 msgr2=0x7f058415d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.722+0000 7f0589525640 1 --2- 172.21.15.139:0/509037034 >> v2:172.21.15.150:3300/0 conn(0x7f058406c220 0x7f058415d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.722+0000 7f0589525640 1 -- 172.21.15.139:0/509037034 >> v1:172.21.15.139:6789/0 conn(0x7f0584072520 legacy=0x7f0584161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.722+0000 7f0589525640 1 -- 172.21.15.139:0/509037034 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05840b33a0 con 0x7f058406be40 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.722+0000 7f057bfff640 1 -- 172.21.15.139:0/509037034 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759894797 0 0) 0x7f05840ad2d0 con 0x7f0584072520 2024-04-25T09:55:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.723+0000 7f0589525640 1 --2- 172.21.15.139:0/509037034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f058406be40 0x7f05840acd90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f05841616a0 tx=0x7f057c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfbfb24885aa468 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.723+0000 7f057bfff640 1 -- 172.21.15.139:0/509037034 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c037070 con 0x7f058406be40 2024-04-25T09:55:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.723+0000 7f057bfff640 1 -- 172.21.15.139:0/509037034 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c032440 con 0x7f058406be40 2024-04-25T09:55:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.723+0000 7f057bfff640 1 -- 172.21.15.139:0/509037034 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c032810 con 0x7f058406be40 2024-04-25T09:55:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.723+0000 7f058b789640 1 -- 172.21.15.139:0/509037034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f058406be40 msgr2=0x7f05840acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.723+0000 7f058b789640 1 --2- 172.21.15.139:0/509037034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f058406be40 0x7f05840acd90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f05841616a0 tx=0x7f057c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.723+0000 7f058b789640 1 -- 172.21.15.139:0/509037034 shutdown_connections 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.723+0000 7f058b789640 1 --2- 172.21.15.139:0/509037034 >> v2:172.21.15.150:3300/0 conn(0x7f058406c220 0x7f058415d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.723+0000 7f058b789640 1 --2- 172.21.15.139:0/509037034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f058406be40 0x7f05840acd90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.723+0000 7f058b789640 1 -- 172.21.15.139:0/509037034 >> 172.21.15.139:0/509037034 conn(0x7f0584071a00 msgr2=0x7f0584071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.724+0000 7f058b789640 1 -- 172.21.15.139:0/509037034 shutdown_connections 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.724+0000 7f058b789640 1 -- 172.21.15.139:0/509037034 wait complete. 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.725+0000 7f058b789640 1 Processor -- start 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.725+0000 7f058b789640 1 -- start start 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.725+0000 7f058b789640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f058406be40 0x7f05841f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.725+0000 7f058b789640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f058406c220 0x7f058415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.725+0000 7f058b789640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0584072520 0x7f05841e8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.725+0000 7f058b789640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058406d9d0 con 0x7f058406c220 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.725+0000 7f058b789640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05840b3960 con 0x7f058406be40 2024-04-25T09:55:58.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.725+0000 7f058b789640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05840b4e90 con 0x7f0584072520 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0588d24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f058406c220 0x7f058415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0588d24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f058406c220 0x7f058415c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40148/0 (socket says 172.21.15.139:40148) 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0588d24640 1 -- 172.21.15.139:0/3807736632 learned_addr learned my addr 172.21.15.139:0/3807736632 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0589d26640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0584072520 0x7f05841e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0589525640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f058406be40 0x7f05841f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0588d24640 1 -- 172.21.15.139:0/3807736632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0584072520 msgr2=0x7f05841e8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0588d24640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0584072520 0x7f05841e8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0588d24640 1 -- 172.21.15.139:0/3807736632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f058406be40 msgr2=0x7f05841f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0588d24640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f058406be40 0x7f05841f4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0588d24640 1 -- 172.21.15.139:0/3807736632 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05841e71a0 con 0x7f058406c220 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0588d24640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f058406c220 0x7f058415c160 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f057400d800 tx=0x7f057400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0579ffb640 1 -- 172.21.15.139:0/3807736632 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400b6c0 con 0x7f058406c220 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0579ffb640 1 -- 172.21.15.139:0/3807736632 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0574019070 con 0x7f058406c220 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f058b789640 1 -- 172.21.15.139:0/3807736632 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05841e7480 con 0x7f058406c220 2024-04-25T09:55:58.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f0579ffb640 1 -- 172.21.15.139:0/3807736632 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400ba80 con 0x7f058406c220 2024-04-25T09:55:58.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.726+0000 7f058b789640 1 -- 172.21.15.139:0/3807736632 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05841e7a40 con 0x7f058406c220 2024-04-25T09:55:58.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.728+0000 7f058b789640 1 -- 172.21.15.139:0/3807736632 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0584074520 con 0x7f058406c220 2024-04-25T09:55:58.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.728+0000 7f0579ffb640 1 -- 172.21.15.139:0/3807736632 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f057400bc20 con 0x7f058406c220 2024-04-25T09:55:58.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.728+0000 7f0579ffb640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f056806b4d0 0x7f056806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:58.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.728+0000 7f0579ffb640 1 -- 172.21.15.139:0/3807736632 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f057408a870 con 0x7f058406c220 2024-04-25T09:55:58.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.728+0000 7f0589525640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f056806b4d0 0x7f056806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:58.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.729+0000 7f0589525640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f056806b4d0 0x7f056806d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f057c034000 tx=0x7f057c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:58.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.732+0000 7f0579ffb640 1 -- 172.21.15.139:0/3807736632 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0574059560 con 0x7f058406c220 2024-04-25T09:55:58.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.899+0000 7f058b789640 1 -- 172.21.15.139:0/3807736632 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f058415d0b0 con 0x7f056806b4d0 2024-04-25T09:55:58.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.905+0000 7f0579ffb640 1 -- 172.21.15.139:0/3807736632 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251973 (secure 0 0 0) 0x7f058415d0b0 con 0x7f056806b4d0 2024-04-25T09:55:58.911 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:58.912 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:55:58.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.908+0000 7f058b789640 1 -- 172.21.15.139:0/3807736632 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f056806b4d0 msgr2=0x7f056806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:58.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.908+0000 7f058b789640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f056806b4d0 0x7f056806d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f057c034000 tx=0x7f057c02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:58.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.908+0000 7f058b789640 1 -- 172.21.15.139:0/3807736632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f058406c220 msgr2=0x7f058415c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:58.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.908+0000 7f058b789640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f058406c220 0x7f058415c160 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f057400d800 tx=0x7f057400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.908+0000 7f058b789640 1 -- 172.21.15.139:0/3807736632 shutdown_connections 2024-04-25T09:55:58.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.908+0000 7f058b789640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f058406c220 0x7f058415c160 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.908+0000 7f058b789640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f056806b4d0 0x7f056806d990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.908+0000 7f058b789640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0584072520 0x7f05841e8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.908+0000 7f058b789640 1 --2- 172.21.15.139:0/3807736632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f058406be40 0x7f05841f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:58.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.908+0000 7f058b789640 1 -- 172.21.15.139:0/3807736632 >> 172.21.15.139:0/3807736632 conn(0x7f0584071a00 msgr2=0x7f058406f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:58.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.908+0000 7f058b789640 1 -- 172.21.15.139:0/3807736632 shutdown_connections 2024-04-25T09:55:58.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:58.909+0000 7f058b789640 1 -- 172.21.15.139:0/3807736632 wait complete. 2024-04-25T09:55:58.922 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-04-25T09:55:57.324781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223204,"kb_used_data":7020,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751516,"statfs":{"total":773094113280,"available":772865552384,"internally_reserved":0,"allocated":7188480,"data_stored":3843760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.038637"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302671+0000","last_change":"2024-04-25T09:55:51.303411+0000","last_active":"2024-04-25T09:55:52.302671+0000","last_peered":"2024-04-25T09:55:52.302671+0000","last_clean":"2024-04-25T09:55:52.302671+0000","last_became_active":"2024-04-25T09:55:51.303129+0000","last_became_peered":"2024-04-25T09:55:51.303129+0000","last_unstale":"2024-04-25T09:55:52.302671+0000","last_undegraded":"2024-04-25T09:55:52.302671+0000","last_fullsized":"2024-04-25T09:55:52.302671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:35.354743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302461+0000","last_change":"2024-04-25T09:55:51.301262+0000","last_active":"2024-04-25T09:55:52.302461+0000","last_peered":"2024-04-25T09:55:52.302461+0000","last_clean":"2024-04-25T09:55:52.302461+0000","last_became_active":"2024-04-25T09:55:51.300989+0000","last_became_peered":"2024-04-25T09:55:51.300989+0000","last_unstale":"2024-04-25T09:55:52.302461+0000","last_undegraded":"2024-04-25T09:55:52.302461+0000","last_fullsized":"2024-04-25T09:55:52.302461+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:49.815961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322973+0000","last_change":"2024-04-25T09:55:51.321176+0000","last_active":"2024-04-25T09:55:51.322973+0000","last_peered":"2024-04-25T09:55:51.322973+0000","last_clean":"2024-04-25T09:55:51.322973+0000","last_became_active":"2024-04-25T09:55:51.319170+0000","last_became_peered":"2024-04-25T09:55:51.319170+0000","last_unstale":"2024-04-25T09:55:51.322973+0000","last_undegraded":"2024-04-25T09:55:51.322973+0000","last_fullsized":"2024-04-25T09:55:51.322973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305476+0000","last_change":"2024-04-25T09:55:51.303613+0000","last_active":"2024-04-25T09:55:51.305476+0000","last_peered":"2024-04-25T09:55:51.305476+0000","last_clean":"2024-04-25T09:55:51.305476+0000","last_became_active":"2024-04-25T09:55:51.303266+0000","last_became_peered":"2024-04-25T09:55:51.303266+0000","last_unstale":"2024-04-25T09:55:51.305476+0000","last_undegraded":"2024-04-25T09:55:51.305476+0000","last_fullsized":"2024-04-25T09:55:51.305476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:37.811562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321590+0000","last_change":"2024-04-25T09:55:51.317347+0000","last_active":"2024-04-25T09:55:51.321590+0000","last_peered":"2024-04-25T09:55:51.321590+0000","last_clean":"2024-04-25T09:55:51.321590+0000","last_became_active":"2024-04-25T09:55:51.316810+0000","last_became_peered":"2024-04-25T09:55:51.316810+0000","last_unstale":"2024-04-25T09:55:51.321590+0000","last_undegraded":"2024-04-25T09:55:51.321590+0000","last_fullsized":"2024-04-25T09:55:51.321590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:32.510934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.320186+0000","last_change":"2024-04-25T09:55:51.319541+0000","last_active":"2024-04-25T09:55:51.320186+0000","last_peered":"2024-04-25T09:55:51.320186+0000","last_clean":"2024-04-25T09:55:51.320186+0000","last_became_active":"2024-04-25T09:55:51.319218+0000","last_became_peered":"2024-04-25T09:55:51.319218+0000","last_unstale":"2024-04-25T09:55:51.320186+0000","last_undegraded":"2024-04-25T09:55:51.320186+0000","last_fullsized":"2024-04-25T09:55:51.320186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305610+0000","last_change":"2024-04-25T09:55:51.313960+0000","last_active":"2024-04-25T09:55:52.305610+0000","last_peered":"2024-04-25T09:55:52.305610+0000","last_clean":"2024-04-25T09:55:52.305610+0000","last_became_active":"2024-04-25T09:55:51.313619+0000","last_became_peered":"2024-04-25T09:55:51.313619+0000","last_unstale":"2024-04-25T09:55:52.305610+0000","last_undegraded":"2024-04-25T09:55:52.305610+0000","last_fullsized":"2024-04-25T09:55:52.305610+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:20.633410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303121+0000","last_change":"2024-04-25T09:55:51.320026+0000","last_active":"2024-04-25T09:55:52.303121+0000","last_peered":"2024-04-25T09:55:52.303121+0000","last_clean":"2024-04-25T09:55:52.303121+0000","last_became_active":"2024-04-25T09:55:51.319656+0000","last_became_peered":"2024-04-25T09:55:51.319656+0000","last_unstale":"2024-04-25T09:55:52.303121+0000","last_undegraded":"2024-04-25T09:55:52.303121+0000","last_fullsized":"2024-04-25T09:55:52.303121+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.317650+0000","last_change":"2024-04-25T09:55:51.314164+0000","last_active":"2024-04-25T09:55:51.317650+0000","last_peered":"2024-04-25T09:55:51.317650+0000","last_clean":"2024-04-25T09:55:51.317650+0000","last_became_active":"2024-04-25T09:55:51.313888+0000","last_became_peered":"2024-04-25T09:55:51.313888+0000","last_unstale":"2024-04-25T09:55:51.317650+0000","last_undegraded":"2024-04-25T09:55:51.317650+0000","last_fullsized":"2024-04-25T09:55:51.317650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:12.483744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321807+0000","last_change":"2024-04-25T09:55:51.316804+0000","last_active":"2024-04-25T09:55:51.321807+0000","last_peered":"2024-04-25T09:55:51.321807+0000","last_clean":"2024-04-25T09:55:51.321807+0000","last_became_active":"2024-04-25T09:55:51.314608+0000","last_became_peered":"2024-04-25T09:55:51.314608+0000","last_unstale":"2024-04-25T09:55:51.321807+0000","last_undegraded":"2024-04-25T09:55:51.321807+0000","last_fullsized":"2024-04-25T09:55:51.321807+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:10.716067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.311232+0000","last_change":"2024-04-25T09:55:51.306747+0000","last_active":"2024-04-25T09:55:51.311232+0000","last_peered":"2024-04-25T09:55:51.311232+0000","last_clean":"2024-04-25T09:55:51.311232+0000","last_became_active":"2024-04-25T09:55:51.306393+0000","last_became_peered":"2024-04-25T09:55:51.306393+0000","last_unstale":"2024-04-25T09:55:51.311232+0000","last_undegraded":"2024-04-25T09:55:51.311232+0000","last_fullsized":"2024-04-25T09:55:51.311232+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.962859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303278+0000","last_change":"2024-04-25T09:55:51.311594+0000","last_active":"2024-04-25T09:55:52.303278+0000","last_peered":"2024-04-25T09:55:52.303278+0000","last_clean":"2024-04-25T09:55:52.303278+0000","last_became_active":"2024-04-25T09:55:51.311271+0000","last_became_peered":"2024-04-25T09:55:51.311271+0000","last_unstale":"2024-04-25T09:55:52.303278+0000","last_undegraded":"2024-04-25T09:55:52.303278+0000","last_fullsized":"2024-04-25T09:55:52.303278+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:44.151672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303327+0000","last_change":"2024-04-25T09:55:51.305962+0000","last_active":"2024-04-25T09:55:52.303327+0000","last_peered":"2024-04-25T09:55:52.303327+0000","last_clean":"2024-04-25T09:55:52.303327+0000","last_became_active":"2024-04-25T09:55:51.305232+0000","last_became_peered":"2024-04-25T09:55:51.305232+0000","last_unstale":"2024-04-25T09:55:52.303327+0000","last_undegraded":"2024-04-25T09:55:52.303327+0000","last_fullsized":"2024-04-25T09:55:52.303327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:26.442633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322877+0000","last_change":"2024-04-25T09:55:51.321119+0000","last_active":"2024-04-25T09:55:51.322877+0000","last_peered":"2024-04-25T09:55:51.322877+0000","last_clean":"2024-04-25T09:55:51.322877+0000","last_became_active":"2024-04-25T09:55:51.320822+0000","last_became_peered":"2024-04-25T09:55:51.320822+0000","last_unstale":"2024-04-25T09:55:51.322877+0000","last_undegraded":"2024-04-25T09:55:51.322877+0000","last_fullsized":"2024-04-25T09:55:51.322877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.318297+0000","last_change":"2024-04-25T09:55:51.313921+0000","last_active":"2024-04-25T09:55:51.318297+0000","last_peered":"2024-04-25T09:55:51.318297+0000","last_clean":"2024-04-25T09:55:51.318297+0000","last_became_active":"2024-04-25T09:55:51.313653+0000","last_became_peered":"2024-04-25T09:55:51.313653+0000","last_unstale":"2024-04-25T09:55:51.318297+0000","last_undegraded":"2024-04-25T09:55:51.318297+0000","last_fullsized":"2024-04-25T09:55:51.318297+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:11.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305541+0000","last_change":"2024-04-25T09:55:51.305285+0000","last_active":"2024-04-25T09:55:52.305541+0000","last_peered":"2024-04-25T09:55:52.305541+0000","last_clean":"2024-04-25T09:55:52.305541+0000","last_became_active":"2024-04-25T09:55:51.305018+0000","last_became_peered":"2024-04-25T09:55:51.305018+0000","last_unstale":"2024-04-25T09:55:52.305541+0000","last_undegraded":"2024-04-25T09:55:52.305541+0000","last_fullsized":"2024-04-25T09:55:52.305541+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:49.400572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302647+0000","last_change":"2024-04-25T09:55:51.306904+0000","last_active":"2024-04-25T09:55:52.302647+0000","last_peered":"2024-04-25T09:55:52.302647+0000","last_clean":"2024-04-25T09:55:52.302647+0000","last_became_active":"2024-04-25T09:55:51.306564+0000","last_became_peered":"2024-04-25T09:55:51.306564+0000","last_unstale":"2024-04-25T09:55:52.302647+0000","last_undegraded":"2024-04-25T09:55:52.302647+0000","last_fullsized":"2024-04-25T09:55:52.302647+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:25.733756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305038+0000","last_change":"2024-04-25T09:55:51.297807+0000","last_active":"2024-04-25T09:55:51.305038+0000","last_peered":"2024-04-25T09:55:51.305038+0000","last_clean":"2024-04-25T09:55:51.305038+0000","last_became_active":"2024-04-25T09:55:51.297473+0000","last_became_peered":"2024-04-25T09:55:51.297473+0000","last_unstale":"2024-04-25T09:55:51.305038+0000","last_undegraded":"2024-04-25T09:55:51.305038+0000","last_fullsized":"2024-04-25T09:55:51.305038+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:14.381048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316239+0000","last_change":"2024-04-25T09:55:51.308066+0000","last_active":"2024-04-25T09:55:51.316239+0000","last_peered":"2024-04-25T09:55:51.316239+0000","last_clean":"2024-04-25T09:55:51.316239+0000","last_became_active":"2024-04-25T09:55:51.307793+0000","last_became_peered":"2024-04-25T09:55:51.307793+0000","last_unstale":"2024-04-25T09:55:51.316239+0000","last_undegraded":"2024-04-25T09:55:51.316239+0000","last_fullsized":"2024-04-25T09:55:51.316239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.962859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.319697+0000","last_change":"2024-04-25T09:55:51.318475+0000","last_active":"2024-04-25T09:55:51.319697+0000","last_peered":"2024-04-25T09:55:51.319697+0000","last_clean":"2024-04-25T09:55:51.319697+0000","last_became_active":"2024-04-25T09:55:51.318203+0000","last_became_peered":"2024-04-25T09:55:51.318203+0000","last_unstale":"2024-04-25T09:55:51.319697+0000","last_undegraded":"2024-04-25T09:55:51.319697+0000","last_fullsized":"2024-04-25T09:55:51.319697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:23.692837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303459+0000","last_change":"2024-04-25T09:55:51.306284+0000","last_active":"2024-04-25T09:55:52.303459+0000","last_peered":"2024-04-25T09:55:52.303459+0000","last_clean":"2024-04-25T09:55:52.303459+0000","last_became_active":"2024-04-25T09:55:51.305652+0000","last_became_peered":"2024-04-25T09:55:51.305652+0000","last_unstale":"2024-04-25T09:55:52.303459+0000","last_undegraded":"2024-04-25T09:55:52.303459+0000","last_fullsized":"2024-04-25T09:55:52.303459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316425+0000","last_change":"2024-04-25T09:55:51.315278+0000","last_active":"2024-04-25T09:55:51.316425+0000","last_peered":"2024-04-25T09:55:51.316425+0000","last_clean":"2024-04-25T09:55:51.316425+0000","last_became_active":"2024-04-25T09:55:51.314991+0000","last_became_peered":"2024-04-25T09:55:51.314991+0000","last_unstale":"2024-04-25T09:55:51.316425+0000","last_undegraded":"2024-04-25T09:55:51.316425+0000","last_fullsized":"2024-04-25T09:55:51.316425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:27.808147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316412+0000","last_change":"2024-04-25T09:55:51.315286+0000","last_active":"2024-04-25T09:55:51.316412+0000","last_peered":"2024-04-25T09:55:51.316412+0000","last_clean":"2024-04-25T09:55:51.316412+0000","last_became_active":"2024-04-25T09:55:51.315029+0000","last_became_peered":"2024-04-25T09:55:51.315029+0000","last_unstale":"2024-04-25T09:55:51.316412+0000","last_undegraded":"2024-04-25T09:55:51.316412+0000","last_fullsized":"2024-04-25T09:55:51.316412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:15.037916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323697+0000","last_change":"2024-04-25T09:55:51.322865+0000","last_active":"2024-04-25T09:55:51.323697+0000","last_peered":"2024-04-25T09:55:51.323697+0000","last_clean":"2024-04-25T09:55:51.323697+0000","last_became_active":"2024-04-25T09:55:51.321449+0000","last_became_peered":"2024-04-25T09:55:51.321449+0000","last_unstale":"2024-04-25T09:55:51.323697+0000","last_undegraded":"2024-04-25T09:55:51.323697+0000","last_fullsized":"2024-04-25T09:55:51.323697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:44.151672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303636+0000","last_change":"2024-04-25T09:55:51.309760+0000","last_active":"2024-04-25T09:55:52.303636+0000","last_peered":"2024-04-25T09:55:52.303636+0000","last_clean":"2024-04-25T09:55:52.303636+0000","last_became_active":"2024-04-25T09:55:51.309495+0000","last_became_peered":"2024-04-25T09:55:51.309495+0000","last_unstale":"2024-04-25T09:55:52.303636+0000","last_undegraded":"2024-04-25T09:55:52.303636+0000","last_fullsized":"2024-04-25T09:55:52.303636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:27.808147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303476+0000","last_change":"2024-04-25T09:55:51.308968+0000","last_active":"2024-04-25T09:55:52.303476+0000","last_peered":"2024-04-25T09:55:52.303476+0000","last_clean":"2024-04-25T09:55:52.303476+0000","last_became_active":"2024-04-25T09:55:51.307967+0000","last_became_peered":"2024-04-25T09:55:51.307967+0000","last_unstale":"2024-04-25T09:55:52.303476+0000","last_undegraded":"2024-04-25T09:55:52.303476+0000","last_fullsized":"2024-04-25T09:55:52.303476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:37.934348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304536+0000","last_change":"2024-04-25T09:55:51.298586+0000","last_active":"2024-04-25T09:55:51.304536+0000","last_peered":"2024-04-25T09:55:51.304536+0000","last_clean":"2024-04-25T09:55:51.304536+0000","last_became_active":"2024-04-25T09:55:51.298272+0000","last_became_peered":"2024-04-25T09:55:51.298272+0000","last_unstale":"2024-04-25T09:55:51.304536+0000","last_undegraded":"2024-04-25T09:55:51.304536+0000","last_fullsized":"2024-04-25T09:55:51.304536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:30.656241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302366+0000","last_change":"2024-04-25T09:55:51.318634+0000","last_active":"2024-04-25T09:55:52.302366+0000","last_peered":"2024-04-25T09:55:52.302366+0000","last_clean":"2024-04-25T09:55:52.302366+0000","last_became_active":"2024-04-25T09:55:51.318347+0000","last_became_peered":"2024-04-25T09:55:51.318347+0000","last_unstale":"2024-04-25T09:55:52.302366+0000","last_undegraded":"2024-04-25T09:55:52.302366+0000","last_fullsized":"2024-04-25T09:55:52.302366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:18.171353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.314075+0000","last_change":"2024-04-25T09:55:51.312167+0000","last_active":"2024-04-25T09:55:51.314075+0000","last_peered":"2024-04-25T09:55:51.314075+0000","last_clean":"2024-04-25T09:55:51.314075+0000","last_became_active":"2024-04-25T09:55:51.311785+0000","last_became_peered":"2024-04-25T09:55:51.311785+0000","last_unstale":"2024-04-25T09:55:51.314075+0000","last_undegraded":"2024-04-25T09:55:51.314075+0000","last_fullsized":"2024-04-25T09:55:51.314075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:53.653931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303192+0000","last_change":"2024-04-25T09:55:51.320446+0000","last_active":"2024-04-25T09:55:52.303192+0000","last_peered":"2024-04-25T09:55:52.303192+0000","last_clean":"2024-04-25T09:55:52.303192+0000","last_became_active":"2024-04-25T09:55:51.320112+0000","last_became_peered":"2024-04-25T09:55:51.320112+0000","last_unstale":"2024-04-25T09:55:52.303192+0000","last_undegraded":"2024-04-25T09:55:52.303192+0000","last_fullsized":"2024-04-25T09:55:52.303192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304965+0000","last_change":"2024-04-25T09:55:51.301513+0000","last_active":"2024-04-25T09:55:51.304965+0000","last_peered":"2024-04-25T09:55:51.304965+0000","last_clean":"2024-04-25T09:55:51.304965+0000","last_became_active":"2024-04-25T09:55:51.295610+0000","last_became_peered":"2024-04-25T09:55:51.295610+0000","last_unstale":"2024-04-25T09:55:51.304965+0000","last_undegraded":"2024-04-25T09:55:51.304965+0000","last_fullsized":"2024-04-25T09:55:51.304965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.757412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.303022+0000","last_change":"2024-04-25T09:55:51.300604+0000","last_active":"2024-04-25T09:55:51.303022+0000","last_peered":"2024-04-25T09:55:51.303022+0000","last_clean":"2024-04-25T09:55:51.303022+0000","last_became_active":"2024-04-25T09:55:51.300286+0000","last_became_peered":"2024-04-25T09:55:51.300286+0000","last_unstale":"2024-04-25T09:55:51.303022+0000","last_undegraded":"2024-04-25T09:55:51.303022+0000","last_fullsized":"2024-04-25T09:55:51.303022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:44.205533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316388+0000","last_change":"2024-04-25T09:55:51.310013+0000","last_active":"2024-04-25T09:55:51.316388+0000","last_peered":"2024-04-25T09:55:51.316388+0000","last_clean":"2024-04-25T09:55:51.316388+0000","last_became_active":"2024-04-25T09:55:51.309619+0000","last_became_peered":"2024-04-25T09:55:51.309619+0000","last_unstale":"2024-04-25T09:55:51.316388+0000","last_undegraded":"2024-04-25T09:55:51.316388+0000","last_fullsized":"2024-04-25T09:55:51.316388+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:30.874963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302699+0000","last_change":"2024-04-25T09:55:50.305070+0000","last_active":"2024-04-25T09:55:52.302699+0000","last_peered":"2024-04-25T09:55:52.302699+0000","last_clean":"2024-04-25T09:55:52.302699+0000","last_became_active":"2024-04-25T09:55:50.304749+0000","last_became_peered":"2024-04-25T09:55:50.304749+0000","last_unstale":"2024-04-25T09:55:52.302699+0000","last_undegraded":"2024-04-25T09:55:52.302699+0000","last_fullsized":"2024-04-25T09:55:52.302699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:43.049220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.309334+0000","last_change":"2024-04-25T09:55:51.305186+0000","last_active":"2024-04-25T09:55:51.309334+0000","last_peered":"2024-04-25T09:55:51.309334+0000","last_clean":"2024-04-25T09:55:51.309334+0000","last_became_active":"2024-04-25T09:55:51.304843+0000","last_became_peered":"2024-04-25T09:55:51.304843+0000","last_unstale":"2024-04-25T09:55:51.309334+0000","last_undegraded":"2024-04-25T09:55:51.309334+0000","last_fullsized":"2024-04-25T09:55:51.309334+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:46.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323347+0000","last_change":"2024-04-25T09:55:50.305062+0000","last_active":"2024-04-25T09:55:51.323347+0000","last_peered":"2024-04-25T09:55:51.323347+0000","last_clean":"2024-04-25T09:55:51.323347+0000","last_became_active":"2024-04-25T09:55:50.304760+0000","last_became_peered":"2024-04-25T09:55:50.304760+0000","last_unstale":"2024-04-25T09:55:51.323347+0000","last_undegraded":"2024-04-25T09:55:51.323347+0000","last_fullsized":"2024-04-25T09:55:51.323347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:10.306355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.308626+0000","last_change":"2024-04-25T09:55:51.307444+0000","last_active":"2024-04-25T09:55:51.308626+0000","last_peered":"2024-04-25T09:55:51.308626+0000","last_clean":"2024-04-25T09:55:51.308626+0000","last_became_active":"2024-04-25T09:55:51.307171+0000","last_became_peered":"2024-04-25T09:55:51.307171+0000","last_unstale":"2024-04-25T09:55:51.308626+0000","last_undegraded":"2024-04-25T09:55:51.308626+0000","last_fullsized":"2024-04-25T09:55:51.308626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:26.442633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.317020+0000","last_change":"2024-04-25T09:55:50.298308+0000","last_active":"2024-04-25T09:55:51.317020+0000","last_peered":"2024-04-25T09:55:51.317020+0000","last_clean":"2024-04-25T09:55:51.317020+0000","last_became_active":"2024-04-25T09:55:50.298018+0000","last_became_peered":"2024-04-25T09:55:50.298018+0000","last_unstale":"2024-04-25T09:55:51.317020+0000","last_undegraded":"2024-04-25T09:55:51.317020+0000","last_fullsized":"2024-04-25T09:55:51.317020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:52.646641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322753+0000","last_change":"2024-04-25T09:55:51.317521+0000","last_active":"2024-04-25T09:55:51.322753+0000","last_peered":"2024-04-25T09:55:51.322753+0000","last_clean":"2024-04-25T09:55:51.322753+0000","last_became_active":"2024-04-25T09:55:51.317265+0000","last_became_peered":"2024-04-25T09:55:51.317265+0000","last_unstale":"2024-04-25T09:55:51.322753+0000","last_undegraded":"2024-04-25T09:55:51.322753+0000","last_fullsized":"2024-04-25T09:55:51.322753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316812+0000","last_change":"2024-04-25T09:55:50.301119+0000","last_active":"2024-04-25T09:55:51.316812+0000","last_peered":"2024-04-25T09:55:51.316812+0000","last_clean":"2024-04-25T09:55:51.316812+0000","last_became_active":"2024-04-25T09:55:50.300321+0000","last_became_peered":"2024-04-25T09:55:50.300321+0000","last_unstale":"2024-04-25T09:55:51.316812+0000","last_undegraded":"2024-04-25T09:55:51.316812+0000","last_fullsized":"2024-04-25T09:55:51.316812+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:09.708778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305589+0000","last_change":"2024-04-25T09:55:51.297568+0000","last_active":"2024-04-25T09:55:52.305589+0000","last_peered":"2024-04-25T09:55:52.305589+0000","last_clean":"2024-04-25T09:55:52.305589+0000","last_became_active":"2024-04-25T09:55:51.297305+0000","last_became_peered":"2024-04-25T09:55:51.297305+0000","last_unstale":"2024-04-25T09:55:52.305589+0000","last_undegraded":"2024-04-25T09:55:52.305589+0000","last_fullsized":"2024-04-25T09:55:52.305589+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:00.789145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.311448+0000","last_change":"2024-04-25T09:55:50.297259+0000","last_active":"2024-04-25T09:55:51.311448+0000","last_peered":"2024-04-25T09:55:51.311448+0000","last_clean":"2024-04-25T09:55:51.311448+0000","last_became_active":"2024-04-25T09:55:50.296982+0000","last_became_peered":"2024-04-25T09:55:50.296982+0000","last_unstale":"2024-04-25T09:55:51.311448+0000","last_undegraded":"2024-04-25T09:55:51.311448+0000","last_fullsized":"2024-04-25T09:55:51.311448+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:27.161990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302048+0000","last_change":"2024-04-25T09:55:51.309288+0000","last_active":"2024-04-25T09:55:52.302048+0000","last_peered":"2024-04-25T09:55:52.302048+0000","last_clean":"2024-04-25T09:55:52.302048+0000","last_became_active":"2024-04-25T09:55:51.309018+0000","last_became_peered":"2024-04-25T09:55:51.309018+0000","last_unstale":"2024-04-25T09:55:52.302048+0000","last_undegraded":"2024-04-25T09:55:52.302048+0000","last_fullsized":"2024-04-25T09:55:52.302048+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:47.578362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.310309+0000","last_change":"2024-04-25T09:55:50.298200+0000","last_active":"2024-04-25T09:55:51.310309+0000","last_peered":"2024-04-25T09:55:51.310309+0000","last_clean":"2024-04-25T09:55:51.310309+0000","last_became_active":"2024-04-25T09:55:50.297950+0000","last_became_peered":"2024-04-25T09:55:50.297950+0000","last_unstale":"2024-04-25T09:55:51.310309+0000","last_undegraded":"2024-04-25T09:55:51.310309+0000","last_fullsized":"2024-04-25T09:55:51.310309+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:50.309907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316534+0000","last_change":"2024-04-25T09:55:51.309251+0000","last_active":"2024-04-25T09:55:51.316534+0000","last_peered":"2024-04-25T09:55:51.316534+0000","last_clean":"2024-04-25T09:55:51.316534+0000","last_became_active":"2024-04-25T09:55:51.308805+0000","last_became_peered":"2024-04-25T09:55:51.308805+0000","last_unstale":"2024-04-25T09:55:51.316534+0000","last_undegraded":"2024-04-25T09:55:51.316534+0000","last_fullsized":"2024-04-25T09:55:51.316534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:21.995603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291503+0000","last_change":"2024-04-25T09:55:50.299777+0000","last_active":"2024-04-25T09:55:51.291503+0000","last_peered":"2024-04-25T09:55:51.291503+0000","last_clean":"2024-04-25T09:55:51.291503+0000","last_became_active":"2024-04-25T09:55:50.299522+0000","last_became_peered":"2024-04-25T09:55:50.299522+0000","last_unstale":"2024-04-25T09:55:51.291503+0000","last_undegraded":"2024-04-25T09:55:51.291503+0000","last_fullsized":"2024-04-25T09:55:51.291503+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:09.708778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.298022+0000","last_change":"2024-04-25T09:55:51.296452+0000","last_active":"2024-04-25T09:55:51.298022+0000","last_peered":"2024-04-25T09:55:51.298022+0000","last_clean":"2024-04-25T09:55:51.298022+0000","last_became_active":"2024-04-25T09:55:51.296189+0000","last_became_peered":"2024-04-25T09:55:51.296189+0000","last_unstale":"2024-04-25T09:55:51.298022+0000","last_undegraded":"2024-04-25T09:55:51.298022+0000","last_fullsized":"2024-04-25T09:55:51.298022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:02.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.296447+0000","last_change":"2024-04-25T09:55:50.305088+0000","last_active":"2024-04-25T09:55:51.296447+0000","last_peered":"2024-04-25T09:55:51.296447+0000","last_clean":"2024-04-25T09:55:51.296447+0000","last_became_active":"2024-04-25T09:55:50.304789+0000","last_became_peered":"2024-04-25T09:55:50.304789+0000","last_unstale":"2024-04-25T09:55:51.296447+0000","last_undegraded":"2024-04-25T09:55:51.296447+0000","last_fullsized":"2024-04-25T09:55:51.296447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:28.750122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302469+0000","last_change":"2024-04-25T09:55:51.295832+0000","last_active":"2024-04-25T09:55:52.302469+0000","last_peered":"2024-04-25T09:55:52.302469+0000","last_clean":"2024-04-25T09:55:52.302469+0000","last_became_active":"2024-04-25T09:55:51.295549+0000","last_became_peered":"2024-04-25T09:55:51.295549+0000","last_unstale":"2024-04-25T09:55:52.302469+0000","last_undegraded":"2024-04-25T09:55:52.302469+0000","last_fullsized":"2024-04-25T09:55:52.302469+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:31.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.308605+0000","last_change":"2024-04-25T09:55:50.307313+0000","last_active":"2024-04-25T09:55:51.308605+0000","last_peered":"2024-04-25T09:55:51.308605+0000","last_clean":"2024-04-25T09:55:51.308605+0000","last_became_active":"2024-04-25T09:55:50.306998+0000","last_became_peered":"2024-04-25T09:55:50.306998+0000","last_unstale":"2024-04-25T09:55:51.308605+0000","last_undegraded":"2024-04-25T09:55:51.308605+0000","last_fullsized":"2024-04-25T09:55:51.308605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:13.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305345+0000","last_change":"2024-04-25T09:55:51.302618+0000","last_active":"2024-04-25T09:55:51.305345+0000","last_peered":"2024-04-25T09:55:51.305345+0000","last_clean":"2024-04-25T09:55:51.305345+0000","last_became_active":"2024-04-25T09:55:51.301598+0000","last_became_peered":"2024-04-25T09:55:51.301598+0000","last_unstale":"2024-04-25T09:55:51.305345+0000","last_undegraded":"2024-04-25T09:55:51.305345+0000","last_fullsized":"2024-04-25T09:55:51.305345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:37.934348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.292155+0000","last_change":"2024-04-25T09:55:50.300463+0000","last_active":"2024-04-25T09:55:51.292155+0000","last_peered":"2024-04-25T09:55:51.292155+0000","last_clean":"2024-04-25T09:55:51.292155+0000","last_became_active":"2024-04-25T09:55:50.299576+0000","last_became_peered":"2024-04-25T09:55:50.299576+0000","last_unstale":"2024-04-25T09:55:51.292155+0000","last_undegraded":"2024-04-25T09:55:51.292155+0000","last_fullsized":"2024-04-25T09:55:51.292155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:56.828907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307569+0000","last_change":"2024-04-25T09:55:51.306375+0000","last_active":"2024-04-25T09:55:51.307569+0000","last_peered":"2024-04-25T09:55:51.307569+0000","last_clean":"2024-04-25T09:55:51.307569+0000","last_became_active":"2024-04-25T09:55:51.305748+0000","last_became_peered":"2024-04-25T09:55:51.305748+0000","last_unstale":"2024-04-25T09:55:51.307569+0000","last_undegraded":"2024-04-25T09:55:51.307569+0000","last_fullsized":"2024-04-25T09:55:51.307569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:37.811562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322267+0000","last_change":"2024-04-25T09:55:50.307971+0000","last_active":"2024-04-25T09:55:51.322267+0000","last_peered":"2024-04-25T09:55:51.322267+0000","last_clean":"2024-04-25T09:55:51.322267+0000","last_became_active":"2024-04-25T09:55:50.307650+0000","last_became_peered":"2024-04-25T09:55:50.307650+0000","last_unstale":"2024-04-25T09:55:51.322267+0000","last_undegraded":"2024-04-25T09:55:51.322267+0000","last_fullsized":"2024-04-25T09:55:51.322267+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:30.506756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.305389+0000","last_change":"2024-04-25T09:55:51.302214+0000","last_active":"2024-04-25T09:55:51.305389+0000","last_peered":"2024-04-25T09:55:51.305389+0000","last_clean":"2024-04-25T09:55:51.305389+0000","last_became_active":"2024-04-25T09:55:51.301887+0000","last_became_peered":"2024-04-25T09:55:51.301887+0000","last_unstale":"2024-04-25T09:55:51.305389+0000","last_undegraded":"2024-04-25T09:55:51.305389+0000","last_fullsized":"2024-04-25T09:55:51.305389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.447273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302496+0000","last_change":"2024-04-25T09:55:50.299295+0000","last_active":"2024-04-25T09:55:52.302496+0000","last_peered":"2024-04-25T09:55:52.302496+0000","last_clean":"2024-04-25T09:55:52.302496+0000","last_became_active":"2024-04-25T09:55:50.296672+0000","last_became_peered":"2024-04-25T09:55:50.296672+0000","last_unstale":"2024-04-25T09:55:52.302496+0000","last_undegraded":"2024-04-25T09:55:52.302496+0000","last_fullsized":"2024-04-25T09:55:52.302496+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:05.248918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315783+0000","last_change":"2024-04-25T09:55:51.312207+0000","last_active":"2024-04-25T09:55:51.315783+0000","last_peered":"2024-04-25T09:55:51.315783+0000","last_clean":"2024-04-25T09:55:51.315783+0000","last_became_active":"2024-04-25T09:55:51.311902+0000","last_became_peered":"2024-04-25T09:55:51.311902+0000","last_unstale":"2024-04-25T09:55:51.315783+0000","last_undegraded":"2024-04-25T09:55:51.315783+0000","last_fullsized":"2024-04-25T09:55:51.315783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.335930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.322307+0000","last_change":"2024-04-25T09:55:50.307406+0000","last_active":"2024-04-25T09:55:51.322307+0000","last_peered":"2024-04-25T09:55:51.322307+0000","last_clean":"2024-04-25T09:55:51.322307+0000","last_became_active":"2024-04-25T09:55:50.306864+0000","last_became_peered":"2024-04-25T09:55:50.306864+0000","last_unstale":"2024-04-25T09:55:51.322307+0000","last_undegraded":"2024-04-25T09:55:51.322307+0000","last_fullsized":"2024-04-25T09:55:51.322307+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:54.613991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303023+0000","last_change":"2024-04-25T09:55:51.306953+0000","last_active":"2024-04-25T09:55:52.303023+0000","last_peered":"2024-04-25T09:55:52.303023+0000","last_clean":"2024-04-25T09:55:52.303023+0000","last_became_active":"2024-04-25T09:55:51.306664+0000","last_became_peered":"2024-04-25T09:55:51.306664+0000","last_unstale":"2024-04-25T09:55:52.303023+0000","last_undegraded":"2024-04-25T09:55:52.303023+0000","last_fullsized":"2024-04-25T09:55:52.303023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303002+0000","last_change":"2024-04-25T09:55:50.303832+0000","last_active":"2024-04-25T09:55:52.303002+0000","last_peered":"2024-04-25T09:55:52.303002+0000","last_clean":"2024-04-25T09:55:52.303002+0000","last_became_active":"2024-04-25T09:55:50.303513+0000","last_became_peered":"2024-04-25T09:55:50.303513+0000","last_unstale":"2024-04-25T09:55:52.303002+0000","last_undegraded":"2024-04-25T09:55:52.303002+0000","last_fullsized":"2024-04-25T09:55:52.303002+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:16.796943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315474+0000","last_change":"2024-04-25T09:55:51.306576+0000","last_active":"2024-04-25T09:55:51.315474+0000","last_peered":"2024-04-25T09:55:51.315474+0000","last_clean":"2024-04-25T09:55:51.315474+0000","last_became_active":"2024-04-25T09:55:51.305692+0000","last_became_peered":"2024-04-25T09:55:51.305692+0000","last_unstale":"2024-04-25T09:55:51.315474+0000","last_undegraded":"2024-04-25T09:55:51.315474+0000","last_fullsized":"2024-04-25T09:55:51.315474+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:46.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.304282+0000","last_change":"2024-04-25T09:55:50.307077+0000","last_active":"2024-04-25T09:55:51.304282+0000","last_peered":"2024-04-25T09:55:51.304282+0000","last_clean":"2024-04-25T09:55:51.304282+0000","last_became_active":"2024-04-25T09:55:50.306743+0000","last_became_peered":"2024-04-25T09:55:50.306743+0000","last_unstale":"2024-04-25T09:55:51.304282+0000","last_undegraded":"2024-04-25T09:55:51.304282+0000","last_fullsized":"2024-04-25T09:55:51.304282+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:34.347454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302240+0000","last_change":"2024-04-25T09:55:51.315727+0000","last_active":"2024-04-25T09:55:52.302240+0000","last_peered":"2024-04-25T09:55:52.302240+0000","last_clean":"2024-04-25T09:55:52.302240+0000","last_became_active":"2024-04-25T09:55:51.315426+0000","last_became_peered":"2024-04-25T09:55:51.315426+0000","last_unstale":"2024-04-25T09:55:52.302240+0000","last_undegraded":"2024-04-25T09:55:52.302240+0000","last_fullsized":"2024-04-25T09:55:52.302240+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:08.905576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"61'8","reported_seq":20,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.324184+0000","last_change":"2024-04-25T09:55:50.302489+0000","last_active":"2024-04-25T09:55:51.324184+0000","last_peered":"2024-04-25T09:55:51.324184+0000","last_clean":"2024-04-25T09:55:51.324184+0000","last_became_active":"2024-04-25T09:55:50.302222+0000","last_became_peered":"2024-04-25T09:55:50.302222+0000","last_unstale":"2024-04-25T09:55:51.324184+0000","last_undegraded":"2024-04-25T09:55:51.324184+0000","last_fullsized":"2024-04-25T09:55:51.324184+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:34.805327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.312652+0000","last_change":"2024-04-25T09:55:51.310660+0000","last_active":"2024-04-25T09:55:51.312652+0000","last_peered":"2024-04-25T09:55:51.312652+0000","last_clean":"2024-04-25T09:55:51.312652+0000","last_became_active":"2024-04-25T09:55:51.310407+0000","last_became_peered":"2024-04-25T09:55:51.310407+0000","last_unstale":"2024-04-25T09:55:51.312652+0000","last_undegraded":"2024-04-25T09:55:51.312652+0000","last_fullsized":"2024-04-25T09:55:51.312652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:41.794719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303710+0000","last_change":"2024-04-25T09:55:50.303136+0000","last_active":"2024-04-25T09:55:52.303710+0000","last_peered":"2024-04-25T09:55:52.303710+0000","last_clean":"2024-04-25T09:55:52.303710+0000","last_became_active":"2024-04-25T09:55:50.302805+0000","last_became_peered":"2024-04-25T09:55:50.302805+0000","last_unstale":"2024-04-25T09:55:52.303710+0000","last_undegraded":"2024-04-25T09:55:52.303710+0000","last_fullsized":"2024-04-25T09:55:52.303710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.898286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321648+0000","last_change":"2024-04-25T09:55:51.318231+0000","last_active":"2024-04-25T09:55:51.321648+0000","last_peered":"2024-04-25T09:55:51.321648+0000","last_clean":"2024-04-25T09:55:51.321648+0000","last_became_active":"2024-04-25T09:55:51.317964+0000","last_became_peered":"2024-04-25T09:55:51.317964+0000","last_unstale":"2024-04-25T09:55:51.321648+0000","last_undegraded":"2024-04-25T09:55:51.321648+0000","last_fullsized":"2024-04-25T09:55:51.321648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:01.234580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321558+0000","last_change":"2024-04-25T09:55:50.307310+0000","last_active":"2024-04-25T09:55:51.321558+0000","last_peered":"2024-04-25T09:55:51.321558+0000","last_clean":"2024-04-25T09:55:51.321558+0000","last_became_active":"2024-04-25T09:55:50.306877+0000","last_became_peered":"2024-04-25T09:55:50.306877+0000","last_unstale":"2024-04-25T09:55:51.321558+0000","last_undegraded":"2024-04-25T09:55:51.321558+0000","last_fullsized":"2024-04-25T09:55:51.321558+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.808671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321032+0000","last_change":"2024-04-25T09:55:51.319026+0000","last_active":"2024-04-25T09:55:51.321032+0000","last_peered":"2024-04-25T09:55:51.321032+0000","last_clean":"2024-04-25T09:55:51.321032+0000","last_became_active":"2024-04-25T09:55:51.318720+0000","last_became_peered":"2024-04-25T09:55:51.318720+0000","last_unstale":"2024-04-25T09:55:51.321032+0000","last_undegraded":"2024-04-25T09:55:51.321032+0000","last_fullsized":"2024-04-25T09:55:51.321032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:21.995603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.323351+0000","last_change":"2024-04-25T09:55:50.309565+0000","last_active":"2024-04-25T09:55:51.323351+0000","last_peered":"2024-04-25T09:55:51.323351+0000","last_clean":"2024-04-25T09:55:51.323351+0000","last_became_active":"2024-04-25T09:55:50.309335+0000","last_became_peered":"2024-04-25T09:55:50.309335+0000","last_unstale":"2024-04-25T09:55:51.323351+0000","last_undegraded":"2024-04-25T09:55:51.323351+0000","last_fullsized":"2024-04-25T09:55:51.323351+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:24.726466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.306954+0000","last_change":"2024-04-25T09:55:51.304138+0000","last_active":"2024-04-25T09:55:51.306954+0000","last_peered":"2024-04-25T09:55:51.306954+0000","last_clean":"2024-04-25T09:55:51.306954+0000","last_became_active":"2024-04-25T09:55:51.303842+0000","last_became_peered":"2024-04-25T09:55:51.303842+0000","last_unstale":"2024-04-25T09:55:51.306954+0000","last_undegraded":"2024-04-25T09:55:51.306954+0000","last_fullsized":"2024-04-25T09:55:51.306954+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:36.447273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291578+0000","last_change":"2024-04-25T09:55:50.295568+0000","last_active":"2024-04-25T09:55:51.291578+0000","last_peered":"2024-04-25T09:55:51.291578+0000","last_clean":"2024-04-25T09:55:51.291578+0000","last_became_active":"2024-04-25T09:55:50.295151+0000","last_became_peered":"2024-04-25T09:55:50.295151+0000","last_unstale":"2024-04-25T09:55:51.291578+0000","last_undegraded":"2024-04-25T09:55:51.291578+0000","last_fullsized":"2024-04-25T09:55:51.291578+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:43.198244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.299612+0000","last_change":"2024-04-25T09:55:51.297690+0000","last_active":"2024-04-25T09:55:51.299612+0000","last_peered":"2024-04-25T09:55:51.299612+0000","last_clean":"2024-04-25T09:55:51.299612+0000","last_became_active":"2024-04-25T09:55:51.297427+0000","last_became_peered":"2024-04-25T09:55:51.297427+0000","last_unstale":"2024-04-25T09:55:51.299612+0000","last_undegraded":"2024-04-25T09:55:51.299612+0000","last_fullsized":"2024-04-25T09:55:51.299612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.169279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.318135+0000","last_change":"2024-04-25T09:55:50.299667+0000","last_active":"2024-04-25T09:55:51.318135+0000","last_peered":"2024-04-25T09:55:51.318135+0000","last_clean":"2024-04-25T09:55:51.318135+0000","last_became_active":"2024-04-25T09:55:50.298743+0000","last_became_peered":"2024-04-25T09:55:50.298743+0000","last_unstale":"2024-04-25T09:55:51.318135+0000","last_undegraded":"2024-04-25T09:55:51.318135+0000","last_fullsized":"2024-04-25T09:55:51.318135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:13.831051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303024+0000","last_change":"2024-04-25T09:55:51.307575+0000","last_active":"2024-04-25T09:55:52.303024+0000","last_peered":"2024-04-25T09:55:52.303024+0000","last_clean":"2024-04-25T09:55:52.303024+0000","last_became_active":"2024-04-25T09:55:51.307279+0000","last_became_peered":"2024-04-25T09:55:51.307279+0000","last_unstale":"2024-04-25T09:55:52.303024+0000","last_undegraded":"2024-04-25T09:55:52.303024+0000","last_fullsized":"2024-04-25T09:55:52.303024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:10.716067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303126+0000","last_change":"2024-04-25T09:55:50.300890+0000","last_active":"2024-04-25T09:55:52.303126+0000","last_peered":"2024-04-25T09:55:52.303126+0000","last_clean":"2024-04-25T09:55:52.303126+0000","last_became_active":"2024-04-25T09:55:50.300508+0000","last_became_peered":"2024-04-25T09:55:50.300508+0000","last_unstale":"2024-04-25T09:55:52.303126+0000","last_undegraded":"2024-04-25T09:55:52.303126+0000","last_fullsized":"2024-04-25T09:55:52.303126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.808671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316507+0000","last_change":"2024-04-25T09:55:51.309120+0000","last_active":"2024-04-25T09:55:51.316507+0000","last_peered":"2024-04-25T09:55:51.316507+0000","last_clean":"2024-04-25T09:55:51.316507+0000","last_became_active":"2024-04-25T09:55:51.308443+0000","last_became_peered":"2024-04-25T09:55:51.308443+0000","last_unstale":"2024-04-25T09:55:51.316507+0000","last_undegraded":"2024-04-25T09:55:51.316507+0000","last_fullsized":"2024-04-25T09:55:51.316507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.335930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.294489+0000","last_change":"2024-04-25T09:55:50.305023+0000","last_active":"2024-04-25T09:55:51.294489+0000","last_peered":"2024-04-25T09:55:51.294489+0000","last_clean":"2024-04-25T09:55:51.294489+0000","last_became_active":"2024-04-25T09:55:50.304744+0000","last_became_peered":"2024-04-25T09:55:50.304744+0000","last_unstale":"2024-04-25T09:55:51.294489+0000","last_undegraded":"2024-04-25T09:55:51.294489+0000","last_fullsized":"2024-04-25T09:55:51.294489+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.787430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307022+0000","last_change":"2024-04-25T09:55:51.304569+0000","last_active":"2024-04-25T09:55:51.307022+0000","last_peered":"2024-04-25T09:55:51.307022+0000","last_clean":"2024-04-25T09:55:51.307022+0000","last_became_active":"2024-04-25T09:55:51.304198+0000","last_became_peered":"2024-04-25T09:55:51.304198+0000","last_unstale":"2024-04-25T09:55:51.307022+0000","last_undegraded":"2024-04-25T09:55:51.307022+0000","last_fullsized":"2024-04-25T09:55:51.307022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:51.317196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303857+0000","last_change":"2024-04-25T09:55:50.305335+0000","last_active":"2024-04-25T09:55:52.303857+0000","last_peered":"2024-04-25T09:55:52.303857+0000","last_clean":"2024-04-25T09:55:52.303857+0000","last_became_active":"2024-04-25T09:55:50.304742+0000","last_became_peered":"2024-04-25T09:55:50.304742+0000","last_unstale":"2024-04-25T09:55:52.303857+0000","last_undegraded":"2024-04-25T09:55:52.303857+0000","last_fullsized":"2024-04-25T09:55:52.303857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:17.164064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305756+0000","last_change":"2024-04-25T09:55:51.304994+0000","last_active":"2024-04-25T09:55:52.305756+0000","last_peered":"2024-04-25T09:55:52.305756+0000","last_clean":"2024-04-25T09:55:52.305756+0000","last_became_active":"2024-04-25T09:55:51.304728+0000","last_became_peered":"2024-04-25T09:55:51.304728+0000","last_unstale":"2024-04-25T09:55:52.305756+0000","last_undegraded":"2024-04-25T09:55:52.305756+0000","last_fullsized":"2024-04-25T09:55:52.305756+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:56.631976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"62'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305088+0000","last_change":"2024-04-25T09:55:50.299644+0000","last_active":"2024-04-25T09:55:52.305088+0000","last_peered":"2024-04-25T09:55:52.305088+0000","last_clean":"2024-04-25T09:55:52.305088+0000","last_became_active":"2024-04-25T09:55:50.299368+0000","last_became_peered":"2024-04-25T09:55:50.299368+0000","last_unstale":"2024-04-25T09:55:52.305088+0000","last_undegraded":"2024-04-25T09:55:52.305088+0000","last_fullsized":"2024-04-25T09:55:52.305088+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:46.571072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.303546+0000","last_change":"2024-04-25T09:55:51.308785+0000","last_active":"2024-04-25T09:55:52.303546+0000","last_peered":"2024-04-25T09:55:52.303546+0000","last_clean":"2024-04-25T09:55:52.303546+0000","last_became_active":"2024-04-25T09:55:51.308476+0000","last_became_peered":"2024-04-25T09:55:51.308476+0000","last_unstale":"2024-04-25T09:55:52.303546+0000","last_undegraded":"2024-04-25T09:55:52.303546+0000","last_fullsized":"2024-04-25T09:55:52.303546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:15.037916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.295008+0000","last_change":"2024-04-25T09:55:50.298279+0000","last_active":"2024-04-25T09:55:51.295008+0000","last_peered":"2024-04-25T09:55:51.295008+0000","last_clean":"2024-04-25T09:55:51.295008+0000","last_became_active":"2024-04-25T09:55:50.298007+0000","last_became_peered":"2024-04-25T09:55:50.298007+0000","last_unstale":"2024-04-25T09:55:51.295008+0000","last_undegraded":"2024-04-25T09:55:51.295008+0000","last_fullsized":"2024-04-25T09:55:51.295008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:11.476455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.321671+0000","last_change":"2024-04-25T09:55:51.318440+0000","last_active":"2024-04-25T09:55:51.321671+0000","last_peered":"2024-04-25T09:55:51.321671+0000","last_clean":"2024-04-25T09:55:51.321671+0000","last_became_active":"2024-04-25T09:55:51.318162+0000","last_became_peered":"2024-04-25T09:55:51.318162+0000","last_unstale":"2024-04-25T09:55:51.321671+0000","last_undegraded":"2024-04-25T09:55:51.321671+0000","last_fullsized":"2024-04-25T09:55:51.321671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:44.056510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302555+0000","last_change":"2024-04-25T09:55:50.296784+0000","last_active":"2024-04-25T09:55:52.302555+0000","last_peered":"2024-04-25T09:55:52.302555+0000","last_clean":"2024-04-25T09:55:52.302555+0000","last_became_active":"2024-04-25T09:55:50.296190+0000","last_became_peered":"2024-04-25T09:55:50.296190+0000","last_unstale":"2024-04-25T09:55:52.302555+0000","last_undegraded":"2024-04-25T09:55:52.302555+0000","last_fullsized":"2024-04-25T09:55:52.302555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:31.503645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316493+0000","last_change":"2024-04-25T09:55:51.310409+0000","last_active":"2024-04-25T09:55:51.316493+0000","last_peered":"2024-04-25T09:55:51.316493+0000","last_clean":"2024-04-25T09:55:51.316493+0000","last_became_active":"2024-04-25T09:55:51.310147+0000","last_became_peered":"2024-04-25T09:55:51.310147+0000","last_unstale":"2024-04-25T09:55:51.316493+0000","last_undegraded":"2024-04-25T09:55:51.316493+0000","last_fullsized":"2024-04-25T09:55:51.316493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:06.256207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.306556+0000","last_change":"2024-04-25T09:55:50.307476+0000","last_active":"2024-04-25T09:55:51.306556+0000","last_peered":"2024-04-25T09:55:51.306556+0000","last_clean":"2024-04-25T09:55:51.306556+0000","last_became_active":"2024-04-25T09:55:50.307206+0000","last_became_peered":"2024-04-25T09:55:50.307206+0000","last_unstale":"2024-04-25T09:55:51.306556+0000","last_undegraded":"2024-04-25T09:55:51.306556+0000","last_fullsized":"2024-04-25T09:55:51.306556+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:05.108894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.315150+0000","last_change":"2024-04-25T09:55:51.313741+0000","last_active":"2024-04-25T09:55:51.315150+0000","last_peered":"2024-04-25T09:55:51.315150+0000","last_clean":"2024-04-25T09:55:51.315150+0000","last_became_active":"2024-04-25T09:55:51.313475+0000","last_became_peered":"2024-04-25T09:55:51.313475+0000","last_unstale":"2024-04-25T09:55:51.315150+0000","last_undegraded":"2024-04-25T09:55:51.315150+0000","last_fullsized":"2024-04-25T09:55:51.315150+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:07.129449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291688+0000","last_change":"2024-04-25T09:55:50.300744+0000","last_active":"2024-04-25T09:55:51.291688+0000","last_peered":"2024-04-25T09:55:51.291688+0000","last_clean":"2024-04-25T09:55:51.291688+0000","last_became_active":"2024-04-25T09:55:50.300415+0000","last_became_peered":"2024-04-25T09:55:50.300415+0000","last_unstale":"2024-04-25T09:55:51.291688+0000","last_undegraded":"2024-04-25T09:55:51.291688+0000","last_fullsized":"2024-04-25T09:55:51.291688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.144383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.302118+0000","last_change":"2024-04-25T09:55:51.307744+0000","last_active":"2024-04-25T09:55:52.302118+0000","last_peered":"2024-04-25T09:55:52.302118+0000","last_clean":"2024-04-25T09:55:52.302118+0000","last_became_active":"2024-04-25T09:55:51.307427+0000","last_became_peered":"2024-04-25T09:55:51.307427+0000","last_unstale":"2024-04-25T09:55:52.302118+0000","last_undegraded":"2024-04-25T09:55:52.302118+0000","last_fullsized":"2024-04-25T09:55:52.302118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:17.804233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291814+0000","last_change":"2024-04-25T09:55:50.301696+0000","last_active":"2024-04-25T09:55:51.291814+0000","last_peered":"2024-04-25T09:55:51.291814+0000","last_clean":"2024-04-25T09:55:51.291814+0000","last_became_active":"2024-04-25T09:55:50.299321+0000","last_became_peered":"2024-04-25T09:55:50.299321+0000","last_unstale":"2024-04-25T09:55:51.291814+0000","last_undegraded":"2024-04-25T09:55:51.291814+0000","last_fullsized":"2024-04-25T09:55:51.291814+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:25.435344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.305700+0000","last_change":"2024-04-25T09:55:51.305887+0000","last_active":"2024-04-25T09:55:52.305700+0000","last_peered":"2024-04-25T09:55:52.305700+0000","last_clean":"2024-04-25T09:55:52.305700+0000","last_became_active":"2024-04-25T09:55:51.305626+0000","last_became_peered":"2024-04-25T09:55:51.305626+0000","last_unstale":"2024-04-25T09:55:52.305700+0000","last_undegraded":"2024-04-25T09:55:52.305700+0000","last_fullsized":"2024-04-25T09:55:52.305700+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:28.074116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:55:52.301997+0000","last_change":"2024-04-25T09:55:50.301123+0000","last_active":"2024-04-25T09:55:52.301997+0000","last_peered":"2024-04-25T09:55:52.301997+0000","last_clean":"2024-04-25T09:55:52.301997+0000","last_became_active":"2024-04-25T09:55:50.300861+0000","last_became_peered":"2024-04-25T09:55:50.300861+0000","last_unstale":"2024-04-25T09:55:52.301997+0000","last_undegraded":"2024-04-25T09:55:52.301997+0000","last_fullsized":"2024-04-25T09:55:52.301997+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:00.227291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.316474+0000","last_change":"2024-04-25T09:55:51.309200+0000","last_active":"2024-04-25T09:55:51.316474+0000","last_peered":"2024-04-25T09:55:51.316474+0000","last_clean":"2024-04-25T09:55:51.316474+0000","last_became_active":"2024-04-25T09:55:51.308915+0000","last_became_peered":"2024-04-25T09:55:51.308915+0000","last_unstale":"2024-04-25T09:55:51.316474+0000","last_undegraded":"2024-04-25T09:55:51.316474+0000","last_fullsized":"2024-04-25T09:55:51.316474+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:50.273814+0000","last_clean_scrub_stamp":"2024-04-25T09:55:50.273814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:07.129449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.291165+0000","last_change":"2024-04-25T09:55:50.298052+0000","last_active":"2024-04-25T09:55:51.291165+0000","last_peered":"2024-04-25T09:55:51.291165+0000","last_clean":"2024-04-25T09:55:51.291165+0000","last_became_active":"2024-04-25T09:55:50.297769+0000","last_became_peered":"2024-04-25T09:55:50.297769+0000","last_unstale":"2024-04-25T09:55:51.291165+0000","last_undegraded":"2024-04-25T09:55:51.291165+0000","last_fullsized":"2024-04-25T09:55:51.291165+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:49.266525+0000","last_clean_scrub_stamp":"2024-04-25T09:55:49.266525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:11.476455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":244,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:55:51.307824+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:55:51.307824+0000","last_peered":"2024-04-25T09:55:51.307824+0000","last_clean":"2024-04-25T09:55:51.307824+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:55:51.307824+0000","last_undegraded":"2024-04-25T09:55:51.307824+0000","last_fullsized":"2024-04-25T09:55:51.307824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":1347584,"data_stored":920910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":741376,"data_stored":329800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":753664,"data_stored":336480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":749568,"data_stored":331378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":737280,"data_stored":329766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":336458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":778240,"data_stored":338834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1331200,"data_stored":920134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:55:58.926 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:55:58.927 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:55:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.064+0000 7fb4e19f5640 1 Processor -- start 2024-04-25T09:55:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.064+0000 7fb4e19f5640 1 -- start start 2024-04-25T09:55:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.065+0000 7fb4e19f5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb4dc15a870 0x7fb4dc0b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.065+0000 7fb4e19f5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb4dc0b4190 0x7fb4dc0d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.065+0000 7fb4e19f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4dc070260 0x7fb4dc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.065+0000 7fb4e19f5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4dc070c10 con 0x7fb4dc15a870 2024-04-25T09:55:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.065+0000 7fb4e19f5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4dc0d1020 con 0x7fb4dc0b4190 2024-04-25T09:55:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.065+0000 7fb4e19f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0b5190 con 0x7fb4dc070260 2024-04-25T09:55:59.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.065+0000 7fb4daffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb4dc15a870 0x7fb4dc0b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.065+0000 7fb4daffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb4dc15a870 0x7fb4dc0b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40162/0 (socket says 172.21.15.139:40162) 2024-04-25T09:55:59.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.065+0000 7fb4daffd640 1 -- 172.21.15.139:0/3727514864 learned_addr learned my addr 172.21.15.139:0/3727514864 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:59.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.066+0000 7fb4db7fe640 1 --2- 172.21.15.139:0/3727514864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4dc070260 0x7fb4dc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.066+0000 7fb4d3fff640 1 --2- 172.21.15.139:0/3727514864 >> v2:172.21.15.150:3300/0 conn(0x7fb4dc0b4190 0x7fb4dc0d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.066+0000 7fb4daffd640 1 -- 172.21.15.139:0/3727514864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4dc070260 msgr2=0x7fb4dc0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:59.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.066+0000 7fb4daffd640 1 --2- 172.21.15.139:0/3727514864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4dc070260 0x7fb4dc0706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.066+0000 7fb4daffd640 1 -- 172.21.15.139:0/3727514864 >> v2:172.21.15.150:3300/0 conn(0x7fb4dc0b4190 msgr2=0x7fb4dc0d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:59.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.066+0000 7fb4daffd640 1 --2- 172.21.15.139:0/3727514864 >> v2:172.21.15.150:3300/0 conn(0x7fb4dc0b4190 0x7fb4dc0d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.066+0000 7fb4daffd640 1 -- 172.21.15.139:0/3727514864 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc0b45f0 con 0x7fb4dc15a870 2024-04-25T09:55:59.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.066+0000 7fb4daffd640 1 --2- 172.21.15.139:0/3727514864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4dc15a870 0x7fb4dc0b5620 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0afed0 tx=0x7fb4cc02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43f63d24a6afc8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:59.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.066+0000 7fb4da7fc640 1 -- 172.21.15.139:0/3727514864 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc037070 con 0x7fb4dc15a870 2024-04-25T09:55:59.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.066+0000 7fb4da7fc640 1 -- 172.21.15.139:0/3727514864 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc02ae00 con 0x7fb4dc15a870 2024-04-25T09:55:59.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.066+0000 7fb4da7fc640 1 -- 172.21.15.139:0/3727514864 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc032660 con 0x7fb4dc15a870 2024-04-25T09:55:59.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.067+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3727514864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4dc15a870 msgr2=0x7fb4dc0b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.067+0000 7fb4e19f5640 1 --2- 172.21.15.139:0/3727514864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4dc15a870 0x7fb4dc0b5620 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0afed0 tx=0x7fb4cc02a0b0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.067+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3727514864 shutdown_connections 2024-04-25T09:55:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.067+0000 7fb4e19f5640 1 --2- 172.21.15.139:0/3727514864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4dc070260 0x7fb4dc0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.067+0000 7fb4e19f5640 1 --2- 172.21.15.139:0/3727514864 >> v2:172.21.15.150:3300/0 conn(0x7fb4dc0b4190 0x7fb4dc0d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.067+0000 7fb4e19f5640 1 --2- 172.21.15.139:0/3727514864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4dc15a870 0x7fb4dc0b5620 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.067+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3727514864 >> 172.21.15.139:0/3727514864 conn(0x7fb4dc0b23f0 msgr2=0x7fb4dc0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.067+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3727514864 shutdown_connections 2024-04-25T09:55:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.067+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3727514864 wait complete. 2024-04-25T09:55:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.068+0000 7fb4e19f5640 1 Processor -- start 2024-04-25T09:55:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.068+0000 7fb4e19f5640 1 -- start start 2024-04-25T09:55:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4e19f5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4dc070260 0x7fb4dc0a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4e19f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4dc0b4190 0x7fb4dc0a9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4e19f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4dc15a870 0x7fb4dc1e2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4e19f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0b4e20 con 0x7fb4dc0b4190 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4e19f5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0b4b40 con 0x7fb4dc070260 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4e19f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0ce610 con 0x7fb4dc15a870 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4d3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4dc0b4190 0x7fb4dc0a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4d3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4dc0b4190 0x7fb4dc0a9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40166/0 (socket says 172.21.15.139:40166) 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4d3fff640 1 -- 172.21.15.139:0/3195105464 learned_addr learned my addr 172.21.15.139:0/3195105464 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4daffd640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4dc070260 0x7fb4dc0a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4db7fe640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4dc15a870 0x7fb4dc1e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4d3fff640 1 -- 172.21.15.139:0/3195105464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4dc15a870 msgr2=0x7fb4dc1e2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4d3fff640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4dc15a870 0x7fb4dc1e2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4d3fff640 1 -- 172.21.15.139:0/3195105464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4dc070260 msgr2=0x7fb4dc0a8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4d3fff640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4dc070260 0x7fb4dc0a8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4d3fff640 1 -- 172.21.15.139:0/3195105464 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc0a9a30 con 0x7fb4dc0b4190 2024-04-25T09:55:59.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.069+0000 7fb4d3fff640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4dc0b4190 0x7fb4dc0a9400 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400fa70 tx=0x7fb4c400ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:59.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.070+0000 7fb4d8ff9640 1 -- 172.21.15.139:0/3195105464 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4015070 con 0x7fb4dc0b4190 2024-04-25T09:55:59.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.070+0000 7fb4d8ff9640 1 -- 172.21.15.139:0/3195105464 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c4007c90 con 0x7fb4dc0b4190 2024-04-25T09:55:59.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.070+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3195105464 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4dc0a9d10 con 0x7fb4dc0b4190 2024-04-25T09:55:59.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.070+0000 7fb4d8ff9640 1 -- 172.21.15.139:0/3195105464 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c401a680 con 0x7fb4dc0b4190 2024-04-25T09:55:59.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.070+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3195105464 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc0aa2d0 con 0x7fb4dc0b4190 2024-04-25T09:55:59.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.071+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3195105464 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4dc074520 con 0x7fb4dc0b4190 2024-04-25T09:55:59.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.071+0000 7fb4d8ff9640 1 -- 172.21.15.139:0/3195105464 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4c401a820 con 0x7fb4dc0b4190 2024-04-25T09:55:59.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.071+0000 7fb4d8ff9640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb4bc06b500 0x7fb4bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.071+0000 7fb4d8ff9640 1 -- 172.21.15.139:0/3195105464 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fb4c408cc40 con 0x7fb4dc0b4190 2024-04-25T09:55:59.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.071+0000 7fb4daffd640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb4bc06b500 0x7fb4bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.072+0000 7fb4daffd640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb4bc06b500 0x7fb4bc06d9c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc034000 tx=0x7fb4cc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:59.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.076+0000 7fb4d8ff9640 1 -- 172.21.15.139:0/3195105464 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4c405b930 con 0x7fb4dc0b4190 2024-04-25T09:55:59.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.318+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3195105464 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb4dc0cf760 con 0x7fb4dc0b4190 2024-04-25T09:55:59.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.320+0000 7fb4d8ff9640 1 -- 172.21.15.139:0/3195105464 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb4c405b230 con 0x7fb4dc0b4190 2024-04-25T09:55:59.323 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2024-04-25T09:55:59.323 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg== 2024-04-25T09:55:59.323 INFO:teuthology.orchestra.run.smithi139.stdout: caps mds = "allow" 2024-04-25T09:55:59.323 INFO:teuthology.orchestra.run.smithi139.stdout: caps mon = "allow r" 2024-04-25T09:55:59.323 INFO:teuthology.orchestra.run.smithi139.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:55:59.323 INFO:teuthology.orchestra.run.smithi139.stderr:updated caps for client.0 2024-04-25T09:55:59.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.322+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3195105464 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb4bc06b500 msgr2=0x7fb4bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:59.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.322+0000 7fb4e19f5640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb4bc06b500 0x7fb4bc06d9c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc034000 tx=0x7fb4cc02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:59.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.322+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3195105464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4dc0b4190 msgr2=0x7fb4dc0a9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:59.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.322+0000 7fb4e19f5640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4dc0b4190 0x7fb4dc0a9400 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400fa70 tx=0x7fb4c400ff30 comp rx=0 tx=0).stop 2024-04-25T09:55:59.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.322+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3195105464 shutdown_connections 2024-04-25T09:55:59.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.322+0000 7fb4e19f5640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4dc0b4190 0x7fb4dc0a9400 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.322+0000 7fb4e19f5640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb4bc06b500 0x7fb4bc06d9c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.322+0000 7fb4e19f5640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4dc15a870 0x7fb4dc1e2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.322+0000 7fb4e19f5640 1 --2- 172.21.15.139:0/3195105464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4dc070260 0x7fb4dc0a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.322+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3195105464 >> 172.21.15.139:0/3195105464 conn(0x7fb4dc0b23f0 msgr2=0x7fb4dc0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:59.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.323+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3195105464 shutdown_connections 2024-04-25T09:55:59.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.323+0000 7fb4e19f5640 1 -- 172.21.15.139:0/3195105464 wait complete. 2024-04-25T09:55:59.337 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:59.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.431+0000 7f37ae11d640 1 Processor -- start 2024-04-25T09:55:59.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.431+0000 7f37ae11d640 1 -- start start 2024-04-25T09:55:59.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.432+0000 7f37ae11d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f37a815aa00 0x7f37a80b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.432+0000 7f37ae11d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37a80b5110 0x7f37a815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.432+0000 7f37ae11d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f37a8072280 0x7f37a8072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.432+0000 7f37ae11d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a80b5c20 con 0x7f37a8072280 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.432+0000 7f37ae11d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a80b45d0 con 0x7f37a815aa00 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.432+0000 7f37ae11d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a815e500 con 0x7f37a80b5110 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.432+0000 7f37a7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f37a8072280 0x7f37a8072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.432+0000 7f37a7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f37a8072280 0x7f37a8072710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40168/0 (socket says 172.21.15.139:40168) 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.432+0000 7f37a7fff640 1 -- 172.21.15.139:0/561847414 learned_addr learned my addr 172.21.15.139:0/561847414 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.433+0000 7f379effd640 1 --2- 172.21.15.139:0/561847414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37a80b5110 0x7f37a815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.433+0000 7f37a77fe640 1 --2- 172.21.15.139:0/561847414 >> v2:172.21.15.150:3300/0 conn(0x7f37a815aa00 0x7f37a80b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.433+0000 7f37a7fff640 1 -- 172.21.15.139:0/561847414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37a80b5110 msgr2=0x7f37a815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.433+0000 7f37a7fff640 1 --2- 172.21.15.139:0/561847414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37a80b5110 0x7f37a815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.433+0000 7f37a7fff640 1 -- 172.21.15.139:0/561847414 >> v2:172.21.15.150:3300/0 conn(0x7f37a815aa00 msgr2=0x7f37a80b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.433+0000 7f37a7fff640 1 --2- 172.21.15.139:0/561847414 >> v2:172.21.15.150:3300/0 conn(0x7f37a815aa00 0x7f37a80b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.433+0000 7f37a7fff640 1 -- 172.21.15.139:0/561847414 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a80b5660 con 0x7f37a8072280 2024-04-25T09:55:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.433+0000 7f37a7fff640 1 --2- 172.21.15.139:0/561847414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37a8072280 0x7f37a8072710 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f37a804f610 tx=0x7f37a002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5024aef07020fbd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:59.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.434+0000 7f37a6ffd640 1 -- 172.21.15.139:0/561847414 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0036070 con 0x7f37a8072280 2024-04-25T09:55:59.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.434+0000 7f37a6ffd640 1 -- 172.21.15.139:0/561847414 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a0031600 con 0x7f37a8072280 2024-04-25T09:55:59.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.434+0000 7f37a6ffd640 1 -- 172.21.15.139:0/561847414 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a00319d0 con 0x7f37a8072280 2024-04-25T09:55:59.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.434+0000 7f37ae11d640 1 -- 172.21.15.139:0/561847414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37a8072280 msgr2=0x7f37a8072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.434+0000 7f37ae11d640 1 --2- 172.21.15.139:0/561847414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37a8072280 0x7f37a8072710 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f37a804f610 tx=0x7f37a002a6c0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.434+0000 7f37ae11d640 1 -- 172.21.15.139:0/561847414 shutdown_connections 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.434+0000 7f37ae11d640 1 --2- 172.21.15.139:0/561847414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37a8072280 0x7f37a8072710 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.434+0000 7f37ae11d640 1 --2- 172.21.15.139:0/561847414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37a80b5110 0x7f37a815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.434+0000 7f37ae11d640 1 --2- 172.21.15.139:0/561847414 >> v2:172.21.15.150:3300/0 conn(0x7f37a815aa00 0x7f37a80b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.434+0000 7f37ae11d640 1 -- 172.21.15.139:0/561847414 >> 172.21.15.139:0/561847414 conn(0x7f37a806e280 msgr2=0x7f37a806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.434+0000 7f37ae11d640 1 -- 172.21.15.139:0/561847414 shutdown_connections 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.434+0000 7f37ae11d640 1 -- 172.21.15.139:0/561847414 wait complete. 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.435+0000 7f37ae11d640 1 Processor -- start 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.436+0000 7f37ae11d640 1 -- start start 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.436+0000 7f37ae11d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37a8072280 0x7f37a8149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.436+0000 7f37ae11d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37a80b5110 0x7f37a80cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.436+0000 7f37ae11d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37a80cb1b0 0x7f37a80c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.436+0000 7f37ae11d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a815f200 con 0x7f37a8072280 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.436+0000 7f37ae11d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a815e7a0 con 0x7f37a80b5110 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.436+0000 7f37ae11d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a80ce610 con 0x7f37a80cb1b0 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.436+0000 7f37a77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37a8072280 0x7f37a8149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.436+0000 7f37a77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37a8072280 0x7f37a8149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40182/0 (socket says 172.21.15.139:40182) 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.436+0000 7f37a77fe640 1 -- 172.21.15.139:0/2079317536 learned_addr learned my addr 172.21.15.139:0/2079317536 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.436+0000 7f37a7fff640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37a80cb1b0 0x7f37a80c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.436+0000 7f379effd640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37a80b5110 0x7f37a80cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.437+0000 7f37a77fe640 1 -- 172.21.15.139:0/2079317536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37a80cb1b0 msgr2=0x7f37a80c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.437+0000 7f37a77fe640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37a80cb1b0 0x7f37a80c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.437+0000 7f37a77fe640 1 -- 172.21.15.139:0/2079317536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37a80b5110 msgr2=0x7f37a80cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.437+0000 7f37a77fe640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37a80b5110 0x7f37a80cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.437+0000 7f37a77fe640 1 -- 172.21.15.139:0/2079317536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a80c6dc0 con 0x7f37a8072280 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.437+0000 7f379effd640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37a80b5110 0x7f37a80cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.437+0000 7f37a77fe640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37a8072280 0x7f37a8149f60 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f379800d620 tx=0x7f379800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:59.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.437+0000 7f37a4ff9640 1 -- 172.21.15.139:0/2079317536 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379800b620 con 0x7f37a8072280 2024-04-25T09:55:59.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.437+0000 7f37a4ff9640 1 -- 172.21.15.139:0/2079317536 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3798019070 con 0x7f37a8072280 2024-04-25T09:55:59.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.437+0000 7f37a4ff9640 1 -- 172.21.15.139:0/2079317536 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379800ba00 con 0x7f37a8072280 2024-04-25T09:55:59.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.437+0000 7f37ae11d640 1 -- 172.21.15.139:0/2079317536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a80c85a0 con 0x7f37a8072280 2024-04-25T09:55:59.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.437+0000 7f37ae11d640 1 -- 172.21.15.139:0/2079317536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a80c8b60 con 0x7f37a8072280 2024-04-25T09:55:59.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.438+0000 7f37ae11d640 1 -- 172.21.15.139:0/2079317536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a80c83c0 con 0x7f37a8072280 2024-04-25T09:55:59.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.438+0000 7f37a4ff9640 1 -- 172.21.15.139:0/2079317536 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f379800bba0 con 0x7f37a8072280 2024-04-25T09:55:59.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.439+0000 7f37a4ff9640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f378806b4d0 0x7f378806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:59.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.439+0000 7f37a4ff9640 1 -- 172.21.15.139:0/2079317536 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f379808a700 con 0x7f37a8072280 2024-04-25T09:55:59.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.439+0000 7f379effd640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f378806b4d0 0x7f378806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:59.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.439+0000 7f379effd640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f378806b4d0 0x7f378806d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3790006fd0 tx=0x7f3790008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.443+0000 7f37a4ff9640 1 -- 172.21.15.139:0/2079317536 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37a80c83c0 con 0x7f37a8072280 2024-04-25T09:55:59.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.650+0000 7f37ae11d640 1 -- 172.21.15.139:0/2079317536 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37a80cb5c0 con 0x7f37a8072280 2024-04-25T09:55:59.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.651+0000 7f37a4ff9640 1 -- 172.21.15.139:0/2079317536 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f37a80cb5c0 con 0x7f37a8072280 2024-04-25T09:55:59.654 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:55:59.654 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/829268166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":829268166},{"type":"v1","addr":"172.21.15.150:6835","nonce":829268166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3075039902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3075039902},{"type":"v1","addr":"172.21.15.150:6834","nonce":3075039902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/627726069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":627726069},{"type":"v1","addr":"172.21.15.139:6836","nonce":627726069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:50.626292+0000","modified":"2024-04-25T09:55:52.634859+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/3541444834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3541444834},{"type":"v1","addr":"172.21.15.139:6837","nonce":3541444834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:55:59.654 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 28 2024-04-25T09:55:59.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.653+0000 7f37ae11d640 1 -- 172.21.15.139:0/2079317536 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f378806b4d0 msgr2=0x7f378806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:59.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.653+0000 7f37ae11d640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f378806b4d0 0x7f378806d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3790006fd0 tx=0x7f3790008040 comp rx=0 tx=0).stop 2024-04-25T09:55:59.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.653+0000 7f37ae11d640 1 -- 172.21.15.139:0/2079317536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37a8072280 msgr2=0x7f37a8149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:59.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.653+0000 7f37ae11d640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37a8072280 0x7f37a8149f60 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f379800d620 tx=0x7f379800dae0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.653+0000 7f37ae11d640 1 -- 172.21.15.139:0/2079317536 shutdown_connections 2024-04-25T09:55:59.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.653+0000 7f37ae11d640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f378806b4d0 0x7f378806d990 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.653+0000 7f37ae11d640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37a80cb1b0 0x7f37a80c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.653+0000 7f37ae11d640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37a80b5110 0x7f37a80cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.653+0000 7f37ae11d640 1 --2- 172.21.15.139:0/2079317536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37a8072280 0x7f37a8149f60 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:59.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.653+0000 7f37ae11d640 1 -- 172.21.15.139:0/2079317536 >> 172.21.15.139:0/2079317536 conn(0x7f37a806e280 msgr2=0x7f37a80b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:59.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.653+0000 7f37ae11d640 1 -- 172.21.15.139:0/2079317536 shutdown_connections 2024-04-25T09:55:59.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:55:59.653+0000 7f37ae11d640 1 -- 172.21.15.139:0/2079317536 wait complete. 2024-04-25T09:56:00.070 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:55:50.626292+0000', 'modified': '2024-04-25T09:55:52.634859+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6312}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6312': {'gid': 6312, 'name': 'a', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6837/3541444834', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6835', 'nonce': 3541444834}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 3541444834}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:56:00.070 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:56:00.071 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:56:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.170+0000 7f740d491640 1 Processor -- start 2024-04-25T09:56:00.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.170+0000 7f740d491640 1 -- start start 2024-04-25T09:56:00.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.170+0000 7f740d491640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f740815dbf0 0x7f74081617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.170+0000 7f740d491640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f74080b9370 0x7f74080ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.170+0000 7f740d491640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74080cec80 0x7f740806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.170+0000 7f740d491640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74080d1ee0 con 0x7f74080b9370 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.170+0000 7f740d491640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7408161dc0 con 0x7f740815dbf0 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.170+0000 7f740d491640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74081620a0 con 0x7f74080cec80 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.171+0000 7f74067fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f74080b9370 0x7f74080ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.171+0000 7f74067fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f74080b9370 0x7f74080ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40192/0 (socket says 172.21.15.139:40192) 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.171+0000 7f74067fc640 1 -- 172.21.15.139:0/3298331971 learned_addr learned my addr 172.21.15.139:0/3298331971 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.171+0000 7f74077fe640 1 --2- 172.21.15.139:0/3298331971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74080cec80 0x7f740806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.171+0000 7f7406ffd640 1 --2- 172.21.15.139:0/3298331971 >> v2:172.21.15.150:3300/0 conn(0x7f740815dbf0 0x7f74081617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.171+0000 7f74067fc640 1 -- 172.21.15.139:0/3298331971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74080cec80 msgr2=0x7f740806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.171+0000 7f74067fc640 1 --2- 172.21.15.139:0/3298331971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74080cec80 0x7f740806c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.171+0000 7f74067fc640 1 -- 172.21.15.139:0/3298331971 >> v2:172.21.15.150:3300/0 conn(0x7f740815dbf0 msgr2=0x7f74081617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.171+0000 7f74067fc640 1 --2- 172.21.15.139:0/3298331971 >> v2:172.21.15.150:3300/0 conn(0x7f740815dbf0 0x7f74081617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.171+0000 7f74067fc640 1 -- 172.21.15.139:0/3298331971 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408162380 con 0x7f74080b9370 2024-04-25T09:56:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.171+0000 7f7406ffd640 1 --2- 172.21.15.139:0/3298331971 >> v2:172.21.15.150:3300/0 conn(0x7f740815dbf0 0x7f74081617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.171+0000 7f74077fe640 1 --2- 172.21.15.139:0/3298331971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74080cec80 0x7f740806c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f74067fc640 1 --2- 172.21.15.139:0/3298331971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74080b9370 0x7f74080ce740 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f740806d330 tx=0x7f73fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21f21d2ab51e65e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f7405ffb640 1 -- 172.21.15.139:0/3298331971 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc037070 con 0x7f74080b9370 2024-04-25T09:56:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f7405ffb640 1 -- 172.21.15.139:0/3298331971 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc032440 con 0x7f74080b9370 2024-04-25T09:56:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f7405ffb640 1 -- 172.21.15.139:0/3298331971 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc032810 con 0x7f74080b9370 2024-04-25T09:56:00.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f740d491640 1 -- 172.21.15.139:0/3298331971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74080b9370 msgr2=0x7f74080ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f740d491640 1 --2- 172.21.15.139:0/3298331971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74080b9370 0x7f74080ce740 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f740806d330 tx=0x7f73fc02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f740d491640 1 -- 172.21.15.139:0/3298331971 shutdown_connections 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f740d491640 1 --2- 172.21.15.139:0/3298331971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74080cec80 0x7f740806c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f740d491640 1 --2- 172.21.15.139:0/3298331971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74080b9370 0x7f74080ce740 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f740d491640 1 --2- 172.21.15.139:0/3298331971 >> v2:172.21.15.150:3300/0 conn(0x7f740815dbf0 0x7f74081617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f740d491640 1 -- 172.21.15.139:0/3298331971 >> 172.21.15.139:0/3298331971 conn(0x7f74080b5480 msgr2=0x7f74080b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f740d491640 1 -- 172.21.15.139:0/3298331971 shutdown_connections 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.172+0000 7f740d491640 1 -- 172.21.15.139:0/3298331971 wait complete. 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.174+0000 7f740d491640 1 Processor -- start 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.174+0000 7f740d491640 1 -- start start 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.174+0000 7f740d491640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74080b9370 0x7f74080b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.174+0000 7f740d491640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74080cec80 0x7f740814ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.174+0000 7f740d491640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f740815dbf0 0x7f740814b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.174+0000 7f740d491640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740815dfd0 con 0x7f74080cec80 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.174+0000 7f740d491640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74080b97d0 con 0x7f74080b9370 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.174+0000 7f740d491640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740815b410 con 0x7f740815dbf0 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.174+0000 7f74067fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74080cec80 0x7f740814ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.174+0000 7f74067fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74080cec80 0x7f740814ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40208/0 (socket says 172.21.15.139:40208) 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.174+0000 7f74067fc640 1 -- 172.21.15.139:0/161403755 learned_addr learned my addr 172.21.15.139:0/161403755 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.175+0000 7f74067fc640 1 -- 172.21.15.139:0/161403755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f740815dbf0 msgr2=0x7f740814b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.175+0000 7f74067fc640 1 --2- 172.21.15.139:0/161403755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f740815dbf0 0x7f740814b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.175+0000 7f74067fc640 1 -- 172.21.15.139:0/161403755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74080b9370 msgr2=0x7f74080b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.175+0000 7f74067fc640 1 --2- 172.21.15.139:0/161403755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74080b9370 0x7f74080b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.175+0000 7f74067fc640 1 -- 172.21.15.139:0/161403755 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408148e50 con 0x7f74080cec80 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.175+0000 7f74067fc640 1 --2- 172.21.15.139:0/161403755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74080cec80 0x7f740814ae00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f73fc02a220 tx=0x7f73fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.175+0000 7f73ef7fe640 1 -- 172.21.15.139:0/161403755 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc041020 con 0x7f74080cec80 2024-04-25T09:56:00.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.175+0000 7f73ef7fe640 1 -- 172.21.15.139:0/161403755 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc0306c0 con 0x7f74080cec80 2024-04-25T09:56:00.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.175+0000 7f740d491640 1 -- 172.21.15.139:0/161403755 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7408149130 con 0x7f74080cec80 2024-04-25T09:56:00.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.175+0000 7f73ef7fe640 1 -- 172.21.15.139:0/161403755 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc037040 con 0x7f74080cec80 2024-04-25T09:56:00.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.175+0000 7f740d491640 1 -- 172.21.15.139:0/161403755 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74080b51f0 con 0x7f74080cec80 2024-04-25T09:56:00.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.176+0000 7f73ef7fe640 1 -- 172.21.15.139:0/161403755 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73fc044020 con 0x7f74080cec80 2024-04-25T09:56:00.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.177+0000 7f73ef7fe640 1 --2- 172.21.15.139:0/161403755 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f73f406b500 0x7f73f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.177+0000 7f73ef7fe640 1 -- 172.21.15.139:0/161403755 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f73fc0aabd0 con 0x7f74080cec80 2024-04-25T09:56:00.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.177+0000 7f7406ffd640 1 --2- 172.21.15.139:0/161403755 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f73f406b500 0x7f73f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.177+0000 7f7406ffd640 1 --2- 172.21.15.139:0/161403755 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f73f406b500 0x7f73f406d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009eb0 tx=0x7f73f80094a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:00.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.178+0000 7f741018a740 1 Processor -- start 2024-04-25T09:56:00.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.178+0000 7f741018a740 1 -- start start 2024-04-25T09:56:00.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f741018a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa888230 0x55f9aa888610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f741018a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8a6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f741018a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa85df10 0x55f9aa8a4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f741018a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9aa7c0a10 con 0x55f9aa85df10 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f741018a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9aa769b30 con 0x55f9aa8a5ad0 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f741018a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9aa7bcc50 con 0x55f9aa888230 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f74077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa85df10 0x55f9aa8a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f74077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa85df10 0x55f9aa8a4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40212/0 (socket says 172.21.15.139:40212) 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f74077fe640 1 -- 172.21.15.139:0/2119406272 learned_addr learned my addr 172.21.15.139:0/2119406272 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f7406ffd640 1 --2- 172.21.15.139:0/2119406272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa888230 0x55f9aa888610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f74067fc640 1 --2- 172.21.15.139:0/2119406272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f74077fe640 1 -- 172.21.15.139:0/2119406272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa888230 msgr2=0x55f9aa888610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f74077fe640 1 --2- 172.21.15.139:0/2119406272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa888230 0x55f9aa888610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f74077fe640 1 -- 172.21.15.139:0/2119406272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa8a5ad0 msgr2=0x55f9aa8a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f74077fe640 1 --2- 172.21.15.139:0/2119406272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8a6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f74077fe640 1 -- 172.21.15.139:0/2119406272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9aa77ee10 con 0x55f9aa85df10 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f74077fe640 1 --2- 172.21.15.139:0/2119406272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa85df10 0x55f9aa8a4190 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f740000b6c0 tx=0x7f740000bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ba074725833ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f73ecff9640 1 -- 172.21.15.139:0/2119406272 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400019070 con 0x55f9aa85df10 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.179+0000 7f73ecff9640 1 -- 172.21.15.139:0/2119406272 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7400004030 con 0x55f9aa85df10 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.180+0000 7f73ecff9640 1 -- 172.21.15.139:0/2119406272 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400004400 con 0x55f9aa85df10 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.180+0000 7f741018a740 1 -- 172.21.15.139:0/2119406272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa85df10 msgr2=0x55f9aa8a4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.180+0000 7f741018a740 1 --2- 172.21.15.139:0/2119406272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa85df10 0x55f9aa8a4190 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f740000b6c0 tx=0x7f740000bb80 comp rx=0 tx=0).stop 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.180+0000 7f741018a740 1 -- 172.21.15.139:0/2119406272 shutdown_connections 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.180+0000 7f741018a740 1 --2- 172.21.15.139:0/2119406272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa85df10 0x55f9aa8a4190 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.180+0000 7f741018a740 1 --2- 172.21.15.139:0/2119406272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.180+0000 7f741018a740 1 --2- 172.21.15.139:0/2119406272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa888230 0x55f9aa888610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.180+0000 7f741018a740 1 -- 172.21.15.139:0/2119406272 >> 172.21.15.139:0/2119406272 conn(0x55f9aa85a840 msgr2=0x55f9aa8a3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.180+0000 7f741018a740 1 -- 172.21.15.139:0/2119406272 shutdown_connections 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.180+0000 7f741018a740 1 -- 172.21.15.139:0/2119406272 wait complete. 2024-04-25T09:56:00.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.181+0000 7f741018a740 1 -- start start 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.181+0000 7f741018a740 1 Processor -- start 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f741018a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa888230 0x55f9aa88bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f741018a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f741018a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa884d60 0x55f9aa8579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f741018a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9aa85ecb0 con 0x55f9aa884d60 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f741018a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9aa77d0c0 con 0x55f9aa8a5ad0 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f741018a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9aa858770 con 0x55f9aa888230 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f741018a740 10 client.0 _ll_register_callbacks cb 0x55f9aa770ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f74077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa884d60 0x55f9aa8579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f74077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa884d60 0x55f9aa8579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40228/0 (socket says 172.21.15.139:40228) 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f74077fe640 1 -- 172.21.15.139:0/1562287861 learned_addr learned my addr 172.21.15.139:0/1562287861 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f74067fc640 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f7406ffd640 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa888230 0x55f9aa88bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f74077fe640 1 -- 172.21.15.139:0/1562287861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa888230 msgr2=0x55f9aa88bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f74077fe640 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa888230 0x55f9aa88bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f74077fe640 1 -- 172.21.15.139:0/1562287861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa8a5ad0 msgr2=0x55f9aa8574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f74077fe640 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f74077fe640 1 -- 172.21.15.139:0/1562287861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9aa4d52d0 con 0x55f9aa884d60 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f7406ffd640 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa888230 0x55f9aa88bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f74077fe640 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa884d60 0x55f9aa8579e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7400017040 tx=0x7f7400006df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb88923fb86d895 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f73e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1562287861 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400023060 con 0x55f9aa884d60 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1562287861 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f740000db50 con 0x55f9aa884d60 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f741018a740 1 -- 172.21.15.139:0/1562287861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7400004030 con 0x55f9aa884d60 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1562287861 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7400029050 con 0x55f9aa884d60 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f741018a740 10 client.6486 fetch_fsmap learned FSMap version 28 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.182+0000 7f741018a740 1 -- 172.21.15.139:0/1562287861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55f9aa59f6b0 con 0x55f9aa884d60 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.183+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1562287861 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7400004030 con 0x55f9aa884d60 2024-04-25T09:56:00.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.183+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1562287861 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f740002c020 con 0x55f9aa884d60 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.183+0000 7f741018a740 10 client.6486 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.183+0000 7f741018a740 10 client.6486 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.183+0000 7f741018a740 20 client.6486 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.183+0000 7f741018a740 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55f9aa8b6f70 0x55f9aa88a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.183+0000 7f741018a740 4 client.6486 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.183+0000 7f741018a740 1 -- 172.21.15.139:0/1562287861 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9aa81f920 con 0x55f9aa8b6f70 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.183+0000 7f7406ffd640 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55f9aa8b6f70 0x55f9aa88a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.183+0000 7f73e5ffb640 10 client.6486 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.183+0000 7f7406ffd640 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55f9aa8b6f70 0x55f9aa88a5f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c126eed4c17b1d8a server_cookie=249cec2ee238b9cb in_seq=0 out_seq=0 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1562287861 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f9aa81f920 con 0x55f9aa8b6f70 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f73e5ffb640 10 client.6486 handle_command_reply: tid=0 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f741018a740 1 client.6486 shutdown 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f741018a740 1 -- 172.21.15.139:0/1562287861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa884d60 msgr2=0x55f9aa8579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f741018a740 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa884d60 0x55f9aa8579e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7400017040 tx=0x7f7400006df0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f741018a740 1 -- 172.21.15.139:0/1562287861 shutdown_connections 2024-04-25T09:56:00.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f741018a740 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55f9aa8b6f70 0x55f9aa88a5f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f741018a740 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa884d60 0x55f9aa8579e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f73e5ffb640 0 client.6486 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:00.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f741018a740 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f741018a740 1 --2- 172.21.15.139:0/1562287861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa888230 0x55f9aa88bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f741018a740 1 -- 172.21.15.139:0/1562287861 >> 172.21.15.139:0/1562287861 conn(0x55f9aa85df10 msgr2=0x55f9aa8a28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:00.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f741018a740 1 -- 172.21.15.139:0/1562287861 shutdown_connections 2024-04-25T09:56:00.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f741018a740 1 -- 172.21.15.139:0/1562287861 wait complete. 2024-04-25T09:56:00.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.184+0000 7f741018a740 20 client.6486 trim_cache size 0 max 16384 2024-04-25T09:56:00.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.199+0000 7f741018a740 1 Processor -- start 2024-04-25T09:56:00.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.199+0000 7f741018a740 1 -- start start 2024-04-25T09:56:00.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.200+0000 7f741018a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa884d60 0x55f9aa85e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.200+0000 7f741018a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa888230 0x55f9aa85e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.200+0000 7f741018a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8a6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.200+0000 7f741018a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74081ff920 con 0x55f9aa8a5ad0 2024-04-25T09:56:00.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.200+0000 7f741018a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74080abe00 con 0x55f9aa888230 2024-04-25T09:56:00.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.200+0000 7f741018a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74081ff320 con 0x55f9aa884d60 2024-04-25T09:56:00.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.200+0000 7f74077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.200+0000 7f74077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8a6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40242/0 (socket says 172.21.15.139:40242) 2024-04-25T09:56:00.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.200+0000 7f74077fe640 1 -- 172.21.15.139:0/2186495634 learned_addr learned my addr 172.21.15.139:0/2186495634 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:00.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.200+0000 7f7406ffd640 1 --2- 172.21.15.139:0/2186495634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa884d60 0x55f9aa85e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f74067fc640 1 --2- 172.21.15.139:0/2186495634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa888230 0x55f9aa85e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f74077fe640 1 -- 172.21.15.139:0/2186495634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa884d60 msgr2=0x55f9aa85e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f74077fe640 1 --2- 172.21.15.139:0/2186495634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa884d60 0x55f9aa85e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f74077fe640 1 -- 172.21.15.139:0/2186495634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa888230 msgr2=0x55f9aa85e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f74077fe640 1 --2- 172.21.15.139:0/2186495634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa888230 0x55f9aa85e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f74077fe640 1 -- 172.21.15.139:0/2186495634 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9aa8b3410 con 0x55f9aa8a5ad0 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f7406ffd640 1 --2- 172.21.15.139:0/2186495634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa884d60 0x55f9aa85e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f74067fc640 1 --2- 172.21.15.139:0/2186495634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa888230 0x55f9aa85e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f74077fe640 1 --2- 172.21.15.139:0/2186495634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8a6410 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f740000b2a0 tx=0x7f740001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0616b435647358 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f73e7fff640 1 -- 172.21.15.139:0/2186495634 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400019050 con 0x55f9aa8a5ad0 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f73e7fff640 1 -- 172.21.15.139:0/2186495634 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740001a040 con 0x55f9aa8a5ad0 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f73e7fff640 1 -- 172.21.15.139:0/2186495634 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74000036b0 con 0x55f9aa8a5ad0 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f741018a740 1 -- 172.21.15.139:0/2186495634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa8a5ad0 msgr2=0x55f9aa8a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f741018a740 1 --2- 172.21.15.139:0/2186495634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8a6410 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f740000b2a0 tx=0x7f740001b000 comp rx=0 tx=0).stop 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f741018a740 1 -- 172.21.15.139:0/2186495634 shutdown_connections 2024-04-25T09:56:00.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f741018a740 1 --2- 172.21.15.139:0/2186495634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa8a5ad0 0x55f9aa8a6410 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f741018a740 1 --2- 172.21.15.139:0/2186495634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa888230 0x55f9aa85e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f741018a740 1 --2- 172.21.15.139:0/2186495634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa884d60 0x55f9aa85e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f741018a740 1 -- 172.21.15.139:0/2186495634 >> 172.21.15.139:0/2186495634 conn(0x55f9aa85df10 msgr2=0x55f9aa8574a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f741018a740 1 -- 172.21.15.139:0/2186495634 shutdown_connections 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.201+0000 7f741018a740 1 -- 172.21.15.139:0/2186495634 wait complete. 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.202+0000 7f741018a740 1 -- start start 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.202+0000 7f741018a740 1 Processor -- start 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f741018a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa884d60 0x55f9aa8c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f741018a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa888230 0x55f9aa8a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f741018a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa8a5ad0 0x55f9aa908360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f741018a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9aa8b3fc0 con 0x55f9aa888230 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f741018a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9aa8b3c40 con 0x55f9aa884d60 2024-04-25T09:56:00.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f741018a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9aa8b73f0 con 0x55f9aa8a5ad0 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f741018a740 10 client.0 _ll_register_callbacks cb 0x55f9aa858c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f74067fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa888230 0x55f9aa8a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f74067fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa888230 0x55f9aa8a64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40248/0 (socket says 172.21.15.139:40248) 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f74067fc640 1 -- 172.21.15.139:0/1444233480 learned_addr learned my addr 172.21.15.139:0/1444233480 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f7406ffd640 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa884d60 0x55f9aa8c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f74077fe640 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa8a5ad0 0x55f9aa908360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f74067fc640 1 -- 172.21.15.139:0/1444233480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa8a5ad0 msgr2=0x55f9aa908360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f74067fc640 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa8a5ad0 0x55f9aa908360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f74067fc640 1 -- 172.21.15.139:0/1444233480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa884d60 msgr2=0x55f9aa8c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f74067fc640 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa884d60 0x55f9aa8c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f74067fc640 1 -- 172.21.15.139:0/1444233480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9aa9088a0 con 0x55f9aa888230 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f74077fe640 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa8a5ad0 0x55f9aa908360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f74067fc640 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa888230 0x55f9aa8a64a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0a8e50 tx=0x7f73fc0798f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9ce7043993295d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:00.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f73e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1444233480 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc0a7060 con 0x55f9aa888230 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1444233480 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f73fc0815b0 con 0x55f9aa888230 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.203+0000 7f741018a740 1 -- 172.21.15.139:0/1444233480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f740001a040 con 0x55f9aa888230 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1444233480 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73fc095050 con 0x55f9aa888230 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 10 client.6492 fetch_fsmap learned FSMap version 28 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 1 -- 172.21.15.139:0/1444233480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55f9aa8bd520 con 0x55f9aa888230 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1444233480 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740001a040 con 0x55f9aa888230 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1444233480 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f73fc03a030 con 0x55f9aa888230 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 10 client.6492 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 10 client.6492 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 20 client.6492 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55f9aa85e5c0 0x55f9aa89ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 4 client.6492 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 1 -- 172.21.15.139:0/1444233480 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f9aa7cdf50 con 0x55f9aa85e5c0 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f7406ffd640 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55f9aa85e5c0 0x55f9aa89ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f73e5ffb640 10 client.6492 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f7406ffd640 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55f9aa85e5c0 0x55f9aa89ffa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b33ad204e89110b server_cookie=f952d92a771757aa in_seq=0 out_seq=0 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1444233480 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55f9aa7cdf50 con 0x55f9aa85e5c0 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f73e5ffb640 10 client.6492 handle_command_reply: tid=0 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 1 client.6492 shutdown 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 1 -- 172.21.15.139:0/1444233480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa888230 msgr2=0x55f9aa8a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa888230 0x55f9aa8a64a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0a8e50 tx=0x7f73fc0798f0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 1 -- 172.21.15.139:0/1444233480 shutdown_connections 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55f9aa85e5c0 0x55f9aa89ffa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f9aa888230 0x55f9aa8a64a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f73e5ffb640 0 client.6492 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:00.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f9aa8a5ad0 0x55f9aa908360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.204+0000 7f741018a740 1 --2- 172.21.15.139:0/1444233480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f9aa884d60 0x55f9aa8c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f741018a740 1 -- 172.21.15.139:0/1444233480 >> 172.21.15.139:0/1444233480 conn(0x55f9aa85df10 msgr2=0x55f9aa8574a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:00.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f741018a740 1 -- 172.21.15.139:0/1444233480 shutdown_connections 2024-04-25T09:56:00.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f741018a740 1 -- 172.21.15.139:0/1444233480 wait complete. 2024-04-25T09:56:00.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f741018a740 20 client.6492 trim_cache size 0 max 16384 2024-04-25T09:56:00.216 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:56:00.216 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T09:56:00.216 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T09:56:00.216 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 6312, 2024-04-25T09:56:00.216 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T09:56:00.216 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T09:56:00.217 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T09:56:00.217 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 8.9079474009999995, 2024-04-25T09:56:00.217 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 28, 2024-04-25T09:56:00.217 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 64, 2024-04-25T09:56:00.217 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 62, 2024-04-25T09:56:00.217 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 13.999738544 2024-04-25T09:56:00.217 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:56:00.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f740d491640 1 -- 172.21.15.139:0/161403755 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f73f406b500 msgr2=0x7f73f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f740d491640 1 --2- 172.21.15.139:0/161403755 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f73f406b500 0x7f73f406d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009eb0 tx=0x7f73f80094a0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f740d491640 1 -- 172.21.15.139:0/161403755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74080cec80 msgr2=0x7f740814ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f740d491640 1 --2- 172.21.15.139:0/161403755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74080cec80 0x7f740814ae00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f73fc02a220 tx=0x7f73fc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f740d491640 1 -- 172.21.15.139:0/161403755 shutdown_connections 2024-04-25T09:56:00.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f740d491640 1 --2- 172.21.15.139:0/161403755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74080cec80 0x7f740814ae00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f740d491640 1 --2- 172.21.15.139:0/161403755 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f73f406b500 0x7f73f406d9c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f740d491640 1 --2- 172.21.15.139:0/161403755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f740815dbf0 0x7f740814b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f740d491640 1 --2- 172.21.15.139:0/161403755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74080b9370 0x7f74080b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f740d491640 1 -- 172.21.15.139:0/161403755 >> 172.21.15.139:0/161403755 conn(0x7f74080b5480 msgr2=0x7f74080caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:00.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f740d491640 1 -- 172.21.15.139:0/161403755 shutdown_connections 2024-04-25T09:56:00.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.205+0000 7f740d491640 1 -- 172.21.15.139:0/161403755 wait complete. 2024-04-25T09:56:00.220 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:56:00.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.313+0000 7fd613522640 1 Processor -- start 2024-04-25T09:56:00.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.313+0000 7fd613522640 1 -- start start 2024-04-25T09:56:00.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.314+0000 7fd613522640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd60c15daa0 0x7fd60c0af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.314+0000 7fd613522640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd60c0afc70 0x7fd60c06ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.314+0000 7fd613522640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd60c06b530 con 0x7fd60c06b0a0 2024-04-25T09:56:00.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.314+0000 7fd613522640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd60c1625d0 con 0x7fd60c15daa0 2024-04-25T09:56:00.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.314+0000 7fd613522640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c161810 con 0x7fd60c0afc70 2024-04-25T09:56:00.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.315+0000 7fd6112be640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd60c15daa0 0x7fd60c0af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.315+0000 7fd610abd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd60c0afc70 0x7fd60c06ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.315+0000 7fd6112be640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd60c15daa0 0x7fd60c0af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:55698/0 (socket says 172.21.15.139:55698) 2024-04-25T09:56:00.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.315+0000 7fd6112be640 1 -- 172.21.15.139:0/1504037944 learned_addr learned my addr 172.21.15.139:0/1504037944 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:00.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.315+0000 7fd611abf640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fd60c06b0a0 0x7fd60c06fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:56962/0 (socket says 172.21.15.139:56962) 2024-04-25T09:56:00.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.315+0000 7fd6112be640 1 -- 172.21.15.139:0/1504037944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd60c0afc70 msgr2=0x7fd60c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:00.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.315+0000 7fd6112be640 1 --2- 172.21.15.139:0/1504037944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd60c0afc70 0x7fd60c06ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.315+0000 7fd6112be640 1 -- 172.21.15.139:0/1504037944 >> v1:172.21.15.150:6789/0 conn(0x7fd60c06b0a0 legacy=0x7fd60c06fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.315+0000 7fd6112be640 1 -- 172.21.15.139:0/1504037944 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c161af0 con 0x7fd60c15daa0 2024-04-25T09:56:00.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.316+0000 7fd6112be640 1 --2- 172.21.15.139:0/1504037944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd60c15daa0 0x7fd60c0af730 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd608009e50 tx=0x7fd60802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61b36b7c8ddf73f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:00.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.316+0000 7fd603fff640 1 -- 172.21.15.139:0/1504037944 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608036070 con 0x7fd60c15daa0 2024-04-25T09:56:00.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.316+0000 7fd603fff640 1 -- 172.21.15.139:0/1504037944 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd608031600 con 0x7fd60c15daa0 2024-04-25T09:56:00.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.316+0000 7fd603fff640 1 -- 172.21.15.139:0/1504037944 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6080319d0 con 0x7fd60c15daa0 2024-04-25T09:56:00.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.316+0000 7fd613522640 1 -- 172.21.15.139:0/1504037944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd60c15daa0 msgr2=0x7fd60c0af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.316+0000 7fd613522640 1 --2- 172.21.15.139:0/1504037944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd60c15daa0 0x7fd60c0af730 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd608009e50 tx=0x7fd60802a710 comp rx=0 tx=0).stop 2024-04-25T09:56:00.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.316+0000 7fd613522640 1 -- 172.21.15.139:0/1504037944 shutdown_connections 2024-04-25T09:56:00.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.316+0000 7fd613522640 1 --2- 172.21.15.139:0/1504037944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd60c0afc70 0x7fd60c06ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.316+0000 7fd613522640 1 --2- 172.21.15.139:0/1504037944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd60c15daa0 0x7fd60c0af730 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.316+0000 7fd613522640 1 -- 172.21.15.139:0/1504037944 >> 172.21.15.139:0/1504037944 conn(0x7fd60c06e280 msgr2=0x7fd60c06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:00.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.317+0000 7fd613522640 1 -- 172.21.15.139:0/1504037944 shutdown_connections 2024-04-25T09:56:00.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.317+0000 7fd613522640 1 -- 172.21.15.139:0/1504037944 wait complete. 2024-04-25T09:56:00.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.318+0000 7fd613522640 1 Processor -- start 2024-04-25T09:56:00.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.318+0000 7fd613522640 1 -- start start 2024-04-25T09:56:00.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.318+0000 7fd613522640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd60c06b0a0 0x7fd60c15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.318+0000 7fd613522640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd60c0afc70 0x7fd60c1521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.318+0000 7fd613522640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd60c15daa0 0x7fd60c153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.318+0000 7fd613522640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c15de80 con 0x7fd60c0afc70 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.318+0000 7fd613522640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c1620b0 con 0x7fd60c06b0a0 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.318+0000 7fd613522640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c0b4690 con 0x7fd60c15daa0 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd610abd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd60c0afc70 0x7fd60c1521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd610abd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd60c0afc70 0x7fd60c1521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40254/0 (socket says 172.21.15.139:40254) 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd6112be640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd60c06b0a0 0x7fd60c15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd610abd640 1 -- 172.21.15.139:0/1628790550 learned_addr learned my addr 172.21.15.139:0/1628790550 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd611abf640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd60c15daa0 0x7fd60c153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd610abd640 1 -- 172.21.15.139:0/1628790550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd60c15daa0 msgr2=0x7fd60c153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd610abd640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd60c15daa0 0x7fd60c153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd610abd640 1 -- 172.21.15.139:0/1628790550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd60c06b0a0 msgr2=0x7fd60c15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd610abd640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd60c06b0a0 0x7fd60c15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd610abd640 1 -- 172.21.15.139:0/1628790550 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c160f30 con 0x7fd60c0afc70 2024-04-25T09:56:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd6112be640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd60c06b0a0 0x7fd60c15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd610abd640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd60c0afc70 0x7fd60c1521a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00e9b0 tx=0x7fd5fc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd601ffb640 1 -- 172.21.15.139:0/1628790550 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc017070 con 0x7fd60c0afc70 2024-04-25T09:56:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd601ffb640 1 -- 172.21.15.139:0/1628790550 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc00c9a0 con 0x7fd60c0afc70 2024-04-25T09:56:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd613522640 1 -- 172.21.15.139:0/1628790550 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60c15f590 con 0x7fd60c0afc70 2024-04-25T09:56:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd601ffb640 1 -- 172.21.15.139:0/1628790550 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc00cd90 con 0x7fd60c0afc70 2024-04-25T09:56:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.319+0000 7fd613522640 1 -- 172.21.15.139:0/1628790550 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60c1e7740 con 0x7fd60c0afc70 2024-04-25T09:56:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.320+0000 7fd613522640 1 -- 172.21.15.139:0/1628790550 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd60c074520 con 0x7fd60c0afc70 2024-04-25T09:56:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.321+0000 7fd601ffb640 1 -- 172.21.15.139:0/1628790550 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5fc01a070 con 0x7fd60c0afc70 2024-04-25T09:56:00.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.321+0000 7fd601ffb640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd5f006b4d0 0x7fd5f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:00.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.321+0000 7fd601ffb640 1 -- 172.21.15.139:0/1628790550 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fd5fc08a3e0 con 0x7fd60c0afc70 2024-04-25T09:56:00.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.321+0000 7fd6112be640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd5f006b4d0 0x7fd5f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:00.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.322+0000 7fd6112be640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd5f006b4d0 0x7fd5f006d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd608033000 tx=0x7fd608009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.325+0000 7fd601ffb640 1 -- 172.21.15.139:0/1628790550 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5fc058970 con 0x7fd60c0afc70 2024-04-25T09:56:00.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.531+0000 7fd613522640 1 -- 172.21.15.139:0/1628790550 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd60c0afa50 con 0x7fd60c0afc70 2024-04-25T09:56:00.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.531+0000 7fd601ffb640 1 -- 172.21.15.139:0/1628790550 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd5fc05ca10 con 0x7fd60c0afc70 2024-04-25T09:56:00.535 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:56:00.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.533+0000 7fd613522640 1 -- 172.21.15.139:0/1628790550 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd5f006b4d0 msgr2=0x7fd5f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.533+0000 7fd613522640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd5f006b4d0 0x7fd5f006d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd608033000 tx=0x7fd608009000 comp rx=0 tx=0).stop 2024-04-25T09:56:00.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.533+0000 7fd613522640 1 -- 172.21.15.139:0/1628790550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd60c0afc70 msgr2=0x7fd60c1521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:00.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.533+0000 7fd613522640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd60c0afc70 0x7fd60c1521a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00e9b0 tx=0x7fd5fc00ee70 comp rx=0 tx=0).stop 2024-04-25T09:56:00.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.533+0000 7fd613522640 1 -- 172.21.15.139:0/1628790550 shutdown_connections 2024-04-25T09:56:00.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.533+0000 7fd613522640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd60c0afc70 0x7fd60c1521a0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.533+0000 7fd613522640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd5f006b4d0 0x7fd5f006d990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.533+0000 7fd613522640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd60c15daa0 0x7fd60c153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.533+0000 7fd613522640 1 --2- 172.21.15.139:0/1628790550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd60c06b0a0 0x7fd60c15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:00.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.533+0000 7fd613522640 1 -- 172.21.15.139:0/1628790550 >> 172.21.15.139:0/1628790550 conn(0x7fd60c06e280 msgr2=0x7fd60c06fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:00.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.533+0000 7fd613522640 1 -- 172.21.15.139:0/1628790550 shutdown_connections 2024-04-25T09:56:00.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:00.533+0000 7fd613522640 1 -- 172.21.15.139:0/1628790550 wait complete. 2024-04-25T09:56:00.547 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:56:00.547 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:56:00.547 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T09:56:00.547 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:56:00.547 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:00.547 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:56:00.547 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:56:00.547 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:56:00.547 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:56:00.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:00.548 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T09:56:00.563 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:56:00.563 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:56:00.563 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:00.564 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42538sec preferred_lft 42538sec 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:56:00.565 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:56:00.566 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T09:56:00.566 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:56:00.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:00.566 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:56:00.625 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:56:00.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:00.627 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T09:56:00.681 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T09:56:00.682 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:56:00.682 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:56:00.682 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:56:00.682 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:56:00.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:00.682 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:56:00.682 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:56:00.683 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:56:00.683 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:56:00.683 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:56:00.683 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:56:00.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:00.765 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:56:00.820 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:56:00.821 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:56:00.821 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:56:00.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:00.821 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T09:56:00.884 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:56:00.884 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:00.939 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:56:00.940 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:56:00.940 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:00.994 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:56:00.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:00.995 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:56:01.075 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:56:01.075 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:56:01.083 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:56:01.083 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:56:01.083 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T09:56:01.083 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T09:56:01.083 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T09:56:01.083 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:56:01.084 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:56:01.084 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:56:01.084 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:56:01.084 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:56:01.085 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:56:01.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:01.086 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:01.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:01.150 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:56:01.209 INFO:teuthology.orchestra.run.smithi139.stdout:client5474 2024-04-25T09:56:01.210 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:56:01.210 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:01.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:01.265 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-25T09:56:01.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:01.323 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-25T09:56:01.379 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T09:56:01.379 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T09:56:01.379 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T09:56:01.379 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:56:01.379 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-25T09:56:01.379 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T09:56:01.484 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627777 2024-04-25T09:56:01.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:56:01.493 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:01.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.626+0000 7f7644f10640 1 Processor -- start 2024-04-25T09:56:01.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.627+0000 7f7644f10640 1 -- start start 2024-04-25T09:56:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.627+0000 7f7644f10640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7640072990 0x7f76400b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.627+0000 7f7644f10640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764015e500 con 0x7f764015aa00 2024-04-25T09:56:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.627+0000 7f7644f10640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76400b5660 con 0x7f76400b5110 2024-04-25T09:56:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.627+0000 7f7644f10640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76400b5c20 con 0x7f7640072990 2024-04-25T09:56:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.627+0000 7f763ed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7640072990 0x7f76400b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.627+0000 7f763e59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f764015aa00 0x7f764015ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:44206/0 (socket says 172.21.15.139:44206) 2024-04-25T09:56:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.627+0000 7f763ed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7640072990 0x7f76400b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40256/0 (socket says 172.21.15.139:40256) 2024-04-25T09:56:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.628+0000 7f763e59c640 1 -- 172.21.15.139:0/709602621 learned_addr learned my addr 172.21.15.139:0/709602621 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:56:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.628+0000 7f763ed9d640 1 -- 172.21.15.139:0/709602621 >> v1:172.21.15.150:6789/0 conn(0x7f76400b5110 legacy=0x7f7640072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.628+0000 7f763ed9d640 1 -- 172.21.15.139:0/709602621 >> v1:172.21.15.139:6789/0 conn(0x7f764015aa00 legacy=0x7f764015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.628+0000 7f763dd9b640 1 -- 172.21.15.139:0/709602621 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349297836 0 0) 0x7f764015e500 con 0x7f764015aa00 2024-04-25T09:56:01.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.628+0000 7f763ed9d640 1 -- 172.21.15.139:0/709602621 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400b5940 con 0x7f7640072990 2024-04-25T09:56:01.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.628+0000 7f763ed9d640 1 --2- 172.21.15.139:0/709602621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7640072990 0x7f76400b4090 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7638009dc0 tx=0x7f763802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c17f877a34ffd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:01.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.629+0000 7f763dd9b640 1 -- 172.21.15.139:0/709602621 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638007740 con 0x7f7640072990 2024-04-25T09:56:01.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.629+0000 7f763dd9b640 1 -- 172.21.15.139:0/709602621 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76380078e0 con 0x7f7640072990 2024-04-25T09:56:01.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.629+0000 7f763dd9b640 1 -- 172.21.15.139:0/709602621 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638007cb0 con 0x7f7640072990 2024-04-25T09:56:01.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.629+0000 7f7644f10640 1 -- 172.21.15.139:0/709602621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7640072990 msgr2=0x7f76400b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:01.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.629+0000 7f7644f10640 1 --2- 172.21.15.139:0/709602621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7640072990 0x7f76400b4090 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7638009dc0 tx=0x7f763802a640 comp rx=0 tx=0).stop 2024-04-25T09:56:01.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.629+0000 7f7644f10640 1 -- 172.21.15.139:0/709602621 shutdown_connections 2024-04-25T09:56:01.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.629+0000 7f7644f10640 1 --2- 172.21.15.139:0/709602621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7640072990 0x7f76400b4090 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.629+0000 7f7644f10640 1 -- 172.21.15.139:0/709602621 >> 172.21.15.139:0/709602621 conn(0x7f764006e280 msgr2=0x7f764006e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:01.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.629+0000 7f7644f10640 1 -- 172.21.15.139:0/709602621 shutdown_connections 2024-04-25T09:56:01.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.629+0000 7f7644f10640 1 -- 172.21.15.139:0/709602621 wait complete. 2024-04-25T09:56:01.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.630+0000 7f7644f10640 1 Processor -- start 2024-04-25T09:56:01.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.631+0000 7f7644f10640 1 -- start start 2024-04-25T09:56:01.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.631+0000 7f7644f10640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7640072990 0x7f7640151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:01.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.631+0000 7f7644f10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76400b5110 0x7f76401f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:01.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.631+0000 7f7644f10640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7640145580 0x7f764015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:01.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.631+0000 7f7644f10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76400ce610 con 0x7f76400b5110 2024-04-25T09:56:01.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.631+0000 7f7644f10640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76400b45d0 con 0x7f7640072990 2024-04-25T09:56:01.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.631+0000 7f7644f10640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76400ce790 con 0x7f7640145580 2024-04-25T09:56:01.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.631+0000 7f7637fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76400b5110 0x7f76401f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:01.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.631+0000 7f7637fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76400b5110 0x7f76401f5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40262/0 (socket says 172.21.15.139:40262) 2024-04-25T09:56:01.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.631+0000 7f7637fff640 1 -- 172.21.15.139:0/2776950715 learned_addr learned my addr 172.21.15.139:0/2776950715 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:01.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f763e59c640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7640072990 0x7f7640151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:01.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f7637fff640 1 -- 172.21.15.139:0/2776950715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7640145580 msgr2=0x7f764015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:01.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f7637fff640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7640145580 0x7f764015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f7637fff640 1 -- 172.21.15.139:0/2776950715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7640072990 msgr2=0x7f7640151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:01.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f7637fff640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7640072990 0x7f7640151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f7637fff640 1 -- 172.21.15.139:0/2776950715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76401516d0 con 0x7f76400b5110 2024-04-25T09:56:01.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f763e59c640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7640072990 0x7f7640151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:01.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f7637fff640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76400b5110 0x7f76401f5190 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f762800c410 tx=0x7f762800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:01.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f7636ffd640 1 -- 172.21.15.139:0/2776950715 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628017070 con 0x7f76400b5110 2024-04-25T09:56:01.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f7636ffd640 1 -- 172.21.15.139:0/2776950715 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76280035f0 con 0x7f76400b5110 2024-04-25T09:56:01.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f7636ffd640 1 -- 172.21.15.139:0/2776950715 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628003a20 con 0x7f76400b5110 2024-04-25T09:56:01.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f7644f10640 1 -- 172.21.15.139:0/2776950715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76401f5a70 con 0x7f76400b5110 2024-04-25T09:56:01.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.632+0000 7f7644f10640 1 -- 172.21.15.139:0/2776950715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76401f6030 con 0x7f76400b5110 2024-04-25T09:56:01.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.633+0000 7f7644f10640 1 -- 172.21.15.139:0/2776950715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7640074520 con 0x7f76400b5110 2024-04-25T09:56:01.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.633+0000 7f7636ffd640 1 -- 172.21.15.139:0/2776950715 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f762801a070 con 0x7f76400b5110 2024-04-25T09:56:01.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.634+0000 7f7636ffd640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f762006b4d0 0x7f762006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:01.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.634+0000 7f763e59c640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f762006b4d0 0x7f762006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:01.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.634+0000 7f7636ffd640 1 -- 172.21.15.139:0/2776950715 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f7628089900 con 0x7f76400b5110 2024-04-25T09:56:01.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.634+0000 7f763e59c640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f762006b4d0 0x7f762006d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7630000f60 tx=0x7f7630009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:01.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.638+0000 7f7636ffd640 1 -- 172.21.15.139:0/2776950715 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76401516d0 con 0x7f76400b5110 2024-04-25T09:56:01.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.843+0000 7f7644f10640 1 -- 172.21.15.139:0/2776950715 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76400d0350 con 0x7f76400b5110 2024-04-25T09:56:01.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.844+0000 7f7636ffd640 1 -- 172.21.15.139:0/2776950715 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f7628058560 con 0x7f76400b5110 2024-04-25T09:56:01.847 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:56:01.847 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/829268166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":829268166},{"type":"v1","addr":"172.21.15.150:6835","nonce":829268166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3075039902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3075039902},{"type":"v1","addr":"172.21.15.150:6834","nonce":3075039902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/627726069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":627726069},{"type":"v1","addr":"172.21.15.139:6836","nonce":627726069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:50.626292+0000","modified":"2024-04-25T09:55:52.634859+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/3541444834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3541444834},{"type":"v1","addr":"172.21.15.139:6837","nonce":3541444834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:56:01.847 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 28 2024-04-25T09:56:01.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.846+0000 7f7644f10640 1 -- 172.21.15.139:0/2776950715 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f762006b4d0 msgr2=0x7f762006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:01.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.846+0000 7f7644f10640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f762006b4d0 0x7f762006d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7630000f60 tx=0x7f7630009040 comp rx=0 tx=0).stop 2024-04-25T09:56:01.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.846+0000 7f7644f10640 1 -- 172.21.15.139:0/2776950715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76400b5110 msgr2=0x7f76401f5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:01.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.846+0000 7f7644f10640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76400b5110 0x7f76401f5190 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f762800c410 tx=0x7f762800c8d0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.846+0000 7f7644f10640 1 -- 172.21.15.139:0/2776950715 shutdown_connections 2024-04-25T09:56:01.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.846+0000 7f7644f10640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76400b5110 0x7f76401f5190 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.846+0000 7f7644f10640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f762006b4d0 0x7f762006d990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.846+0000 7f7644f10640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7640145580 0x7f764015b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.846+0000 7f7644f10640 1 --2- 172.21.15.139:0/2776950715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7640072990 0x7f7640151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:01.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.846+0000 7f7644f10640 1 -- 172.21.15.139:0/2776950715 >> 172.21.15.139:0/2776950715 conn(0x7f764006e280 msgr2=0x7f764006b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:01.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.846+0000 7f7644f10640 1 -- 172.21.15.139:0/2776950715 shutdown_connections 2024-04-25T09:56:01.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:01.846+0000 7f7644f10640 1 -- 172.21.15.139:0/2776950715 wait complete. 2024-04-25T09:56:01.860 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T09:56:01.978 INFO:teuthology.orchestra.run.smithi139.stdout:{"message":"","return_code":0} 2024-04-25T09:56:01.978 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:56:01.978 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T09:56:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.017+0000 7f419833cf40 1 Processor -- start 2024-04-25T09:56:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.017+0000 7f419833cf40 1 -- start start 2024-04-25T09:56:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f419833cf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc8bbf66b0 0x55dc8bcaa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f419833cf40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 0x55dc8bc3fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f419833cf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc8bcb2e50 0x55dc8bd622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f419833cf40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8bcb6580 con 0x55dc8bbfae70 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f419833cf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8bb9fdf0 con 0x55dc8bcb2e50 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f419833cf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc8bbfb7c0 con 0x55dc8bbf66b0 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f418ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 0x55dc8bc3fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f418ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 0x55dc8bc3fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40280/0 (socket says 172.21.15.139:40280) 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f418ffff640 1 -- 172.21.15.139:0/927647053 learned_addr learned my addr 172.21.15.139:0/927647053 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f4195e3d640 1 --2- 172.21.15.139:0/927647053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc8bbf66b0 0x55dc8bcaa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f419663e640 1 --2- 172.21.15.139:0/927647053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc8bcb2e50 0x55dc8bd622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f418ffff640 1 -- 172.21.15.139:0/927647053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc8bbf66b0 msgr2=0x55dc8bcaa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f418ffff640 1 --2- 172.21.15.139:0/927647053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc8bbf66b0 0x55dc8bcaa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f418ffff640 1 -- 172.21.15.139:0/927647053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc8bcb2e50 msgr2=0x55dc8bd622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f418ffff640 1 --2- 172.21.15.139:0/927647053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc8bcb2e50 0x55dc8bd622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f418ffff640 1 -- 172.21.15.139:0/927647053 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc8bb25c50 con 0x55dc8bbfae70 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f418ffff640 1 --2- 172.21.15.139:0/927647053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 0x55dc8bc3fc40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f418000a610 tx=0x7f4180005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5891eba3c5c46adb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f4194e3b640 1 -- 172.21.15.139:0/927647053 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180008a60 con 0x55dc8bbfae70 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f4194e3b640 1 -- 172.21.15.139:0/927647053 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4180008c00 con 0x55dc8bbfae70 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.018+0000 7f4194e3b640 1 -- 172.21.15.139:0/927647053 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180013440 con 0x55dc8bbfae70 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.019+0000 7f419833cf40 1 -- 172.21.15.139:0/927647053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 msgr2=0x55dc8bc3fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.019+0000 7f419833cf40 1 --2- 172.21.15.139:0/927647053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 0x55dc8bc3fc40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f418000a610 tx=0x7f4180005f90 comp rx=0 tx=0).stop 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.019+0000 7f419833cf40 1 -- 172.21.15.139:0/927647053 shutdown_connections 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.019+0000 7f419833cf40 1 --2- 172.21.15.139:0/927647053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 0x55dc8bc3fc40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.019+0000 7f419833cf40 1 --2- 172.21.15.139:0/927647053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc8bcb2e50 0x55dc8bd622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.019+0000 7f419833cf40 1 --2- 172.21.15.139:0/927647053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc8bbf66b0 0x55dc8bcaa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.019+0000 7f419833cf40 1 -- 172.21.15.139:0/927647053 >> 172.21.15.139:0/927647053 conn(0x55dc8bbf8be0 msgr2=0x55dc8bcb5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.019+0000 7f419833cf40 1 -- 172.21.15.139:0/927647053 shutdown_connections 2024-04-25T09:56:02.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.019+0000 7f419833cf40 1 -- 172.21.15.139:0/927647053 wait complete. 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.019+0000 7f419833cf40 1 Processor -- start 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.019+0000 7f419833cf40 1 -- start start 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f419833cf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc8bbf66b0 0x55dc8bcb4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f419833cf40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 0x55dc8bcb9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f419833cf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc8bcb2e50 0x55dc8bccd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f419833cf40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8bcd14c0 con 0x55dc8bbfae70 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f419833cf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8bc24520 con 0x55dc8bcb2e50 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f419833cf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc8bcd2ea0 con 0x55dc8bbf66b0 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f418ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 0x55dc8bcb9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f418ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 0x55dc8bcb9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40288/0 (socket says 172.21.15.139:40288) 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f418ffff640 1 -- 172.21.15.139:0/3312426689 learned_addr learned my addr 172.21.15.139:0/3312426689 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f4195e3d640 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc8bbf66b0 0x55dc8bcb4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f419663e640 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc8bcb2e50 0x55dc8bccd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f418ffff640 1 -- 172.21.15.139:0/3312426689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc8bbf66b0 msgr2=0x55dc8bcb4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.020+0000 7f418ffff640 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc8bbf66b0 0x55dc8bcb4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.021+0000 7f418ffff640 1 -- 172.21.15.139:0/3312426689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc8bcb2e50 msgr2=0x55dc8bccd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.021+0000 7f418ffff640 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc8bcb2e50 0x55dc8bccd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.021+0000 7f418ffff640 1 -- 172.21.15.139:0/3312426689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc8bccaff0 con 0x55dc8bbfae70 2024-04-25T09:56:02.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.021+0000 7f419663e640 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc8bcb2e50 0x55dc8bccd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:02.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.021+0000 7f418ffff640 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 0x55dc8bcb9260 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f418000a530 tx=0x7f4180002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.021+0000 7f418effd640 1 -- 172.21.15.139:0/3312426689 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180022020 con 0x55dc8bbfae70 2024-04-25T09:56:02.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.021+0000 7f418effd640 1 -- 172.21.15.139:0/3312426689 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41800054d0 con 0x55dc8bbfae70 2024-04-25T09:56:02.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.021+0000 7f419833cf40 1 -- 172.21.15.139:0/3312426689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dc8bccb2d0 con 0x55dc8bbfae70 2024-04-25T09:56:02.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.021+0000 7f418effd640 1 -- 172.21.15.139:0/3312426689 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180018040 con 0x55dc8bbfae70 2024-04-25T09:56:02.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.021+0000 7f419833cf40 1 -- 172.21.15.139:0/3312426689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc8bccb890 con 0x55dc8bbfae70 2024-04-25T09:56:02.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.022+0000 7f418effd640 1 -- 172.21.15.139:0/3312426689 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4180025020 con 0x55dc8bbfae70 2024-04-25T09:56:02.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.022+0000 7f418effd640 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4178069ee0 0x7f417806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.023+0000 7f418effd640 1 -- 172.21.15.139:0/3312426689 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f418008aed0 con 0x55dc8bbfae70 2024-04-25T09:56:02.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.023+0000 7f4195e3d640 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4178069ee0 0x7f417806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.023+0000 7f419833cf40 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55dc8bcb97a0 0x55dc8bcb6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.023+0000 7f419833cf40 1 -- 172.21.15.139:0/3312426689 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v9 -- 0x55dc8bcb7450 con 0x55dc8bcb97a0 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.023+0000 7f4195e3d640 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4178069ee0 0x7f417806c3a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4188057000 tx=0x7f4188052000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.023+0000 7f419663e640 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55dc8bcb97a0 0x55dc8bcb6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.023+0000 7f419663e640 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55dc8bcb97a0 0x55dc8bcb6f10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.024+0000 7f419663e640 1 -- 172.21.15.139:0/3312426689 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f4190011040 con 0x55dc8bcb97a0 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.024+0000 7f419833cf40 1 -- 172.21.15.139:0/3312426689 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55dc8bcb97a0 msgr2=0x55dc8bcb6f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.024+0000 7f419833cf40 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55dc8bcb97a0 0x55dc8bcb6f10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.024+0000 7f419833cf40 1 -- 172.21.15.139:0/3312426689 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4178069ee0 msgr2=0x7f417806c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.024+0000 7f419833cf40 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4178069ee0 0x7f417806c3a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4188057000 tx=0x7f4188052000 comp rx=0 tx=0).stop 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.024+0000 7f419833cf40 1 -- 172.21.15.139:0/3312426689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 msgr2=0x55dc8bcb9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.024+0000 7f419833cf40 1 --2- 172.21.15.139:0/3312426689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc8bbfae70 0x55dc8bcb9260 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f418000a530 tx=0x7f4180002fb0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.024+0000 7f419663e640 1 -- 172.21.15.139:0/3312426689 reap_dead start 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.024+0000 7f419833cf40 1 -- 172.21.15.139:0/3312426689 shutdown_connections 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.024+0000 7f419833cf40 1 -- 172.21.15.139:0/3312426689 >> 172.21.15.139:0/3312426689 conn(0x55dc8bbf8be0 msgr2=0x55dc8bbfa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.024+0000 7f419833cf40 1 -- 172.21.15.139:0/3312426689 shutdown_connections 2024-04-25T09:56:02.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.024+0000 7f419833cf40 1 -- 172.21.15.139:0/3312426689 wait complete. 2024-04-25T09:56:02.032 INFO:teuthology.orchestra.run.smithi139.stdout:Xfiledir_test 2024-04-25T09:56:02.032 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:56:02.033 DEBUG:teuthology.orchestra.run.smithi139:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T09:56:02.117 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 1099511627777, 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: "ancestors": [ 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: "dirino": 1099511627776, 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: "dname": "file", 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 2 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: "dirino": 1, 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: "dname": "dir_test", 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 4 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:56:02.118 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-04-25T09:56:02.119 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 10, 2024-04-25T09:56:02.119 INFO:teuthology.orchestra.run.smithi139.stdout: "old_pools": [] 2024-04-25T09:56:02.119 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:56:02.119 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:56:02.120 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-25T09:56:02.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.200+0000 7f0a1e528f40 1 Processor -- start 2024-04-25T09:56:02.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.200+0000 7f0a1e528f40 1 -- start start 2024-04-25T09:56:02.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1e528f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7773b0 0x559a9b776db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1e528f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a9b7bc430 0x559a9b772930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1e528f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a9b846530 0x559a9b846a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1e528f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9b7777c0 con 0x559a9b7773b0 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1e528f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9b71bdf0 con 0x559a9b7bc430 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1e528f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a9b7bcb40 con 0x559a9b846530 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1c02b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7773b0 0x559a9b776db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1c02b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7773b0 0x559a9b776db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40290/0 (socket says 172.21.15.139:40290) 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1c02b640 1 -- 172.21.15.139:0/3222887058 learned_addr learned my addr 172.21.15.139:0/3222887058 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1c82c640 1 --2- 172.21.15.139:0/3222887058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a9b846530 0x559a9b846a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1b82a640 1 --2- 172.21.15.139:0/3222887058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a9b7bc430 0x559a9b772930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1c02b640 1 -- 172.21.15.139:0/3222887058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a9b846530 msgr2=0x559a9b846a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1c02b640 1 --2- 172.21.15.139:0/3222887058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a9b846530 0x559a9b846a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1c02b640 1 -- 172.21.15.139:0/3222887058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a9b7bc430 msgr2=0x559a9b772930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1c02b640 1 --2- 172.21.15.139:0/3222887058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a9b7bc430 0x559a9b772930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1c02b640 1 -- 172.21.15.139:0/3222887058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a9b6a1c50 con 0x559a9b7773b0 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.201+0000 7f0a1b82a640 1 --2- 172.21.15.139:0/3222887058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a9b7bc430 0x559a9b772930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1c02b640 1 --2- 172.21.15.139:0/3222887058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7773b0 0x559a9b776db0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c055040 tx=0x7f0a0c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7206a2d25b3e858e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1a828640 1 -- 172.21.15.139:0/3222887058 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c056040 con 0x559a9b7773b0 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1a828640 1 -- 172.21.15.139:0/3222887058 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c060030 con 0x559a9b7773b0 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1a828640 1 -- 172.21.15.139:0/3222887058 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c052ae0 con 0x559a9b7773b0 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1e528f40 1 -- 172.21.15.139:0/3222887058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7773b0 msgr2=0x559a9b776db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1e528f40 1 --2- 172.21.15.139:0/3222887058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7773b0 0x559a9b776db0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c055040 tx=0x7f0a0c051000 comp rx=0 tx=0).stop 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1e528f40 1 -- 172.21.15.139:0/3222887058 shutdown_connections 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1e528f40 1 --2- 172.21.15.139:0/3222887058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a9b846530 0x559a9b846a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1e528f40 1 --2- 172.21.15.139:0/3222887058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a9b7bc430 0x559a9b772930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1e528f40 1 --2- 172.21.15.139:0/3222887058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7773b0 0x559a9b776db0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1e528f40 1 -- 172.21.15.139:0/3222887058 >> 172.21.15.139:0/3222887058 conn(0x559a9b774be0 msgr2=0x559a9b847aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1e528f40 1 -- 172.21.15.139:0/3222887058 shutdown_connections 2024-04-25T09:56:02.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.202+0000 7f0a1e528f40 1 -- 172.21.15.139:0/3222887058 wait complete. 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.203+0000 7f0a1e528f40 1 Processor -- start 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.203+0000 7f0a1e528f40 1 -- start start 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.203+0000 7f0a1e528f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a9b7773b0 0x559a9b846c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.203+0000 7f0a1e528f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7bc430 0x559a9b837bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.203+0000 7f0a1e528f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a9b846530 0x559a9b838110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.203+0000 7f0a1e528f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9b840720 con 0x559a9b7bc430 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.203+0000 7f0a1e528f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9b849ae0 con 0x559a9b846530 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.203+0000 7f0a1e528f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a9b849d30 con 0x559a9b7773b0 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1b82a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7bc430 0x559a9b837bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1b82a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7bc430 0x559a9b837bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40296/0 (socket says 172.21.15.139:40296) 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1b82a640 1 -- 172.21.15.139:0/2796538071 learned_addr learned my addr 172.21.15.139:0/2796538071 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1c02b640 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a9b7773b0 0x559a9b846c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1c82c640 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a9b846530 0x559a9b838110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1b82a640 1 -- 172.21.15.139:0/2796538071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a9b7773b0 msgr2=0x559a9b846c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1b82a640 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a9b7773b0 0x559a9b846c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1b82a640 1 -- 172.21.15.139:0/2796538071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a9b846530 msgr2=0x559a9b838110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1b82a640 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a9b846530 0x559a9b838110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1b82a640 1 -- 172.21.15.139:0/2796538071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a9b83d2d0 con 0x559a9b7bc430 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1b82a640 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7bc430 0x559a9b837bd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x559a9b848c50 tx=0x7f0a10009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a19826640 1 -- 172.21.15.139:0/2796538071 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10008ce0 con 0x559a9b7bc430 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a19826640 1 -- 172.21.15.139:0/2796538071 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a10015460 con 0x559a9b7bc430 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a19826640 1 -- 172.21.15.139:0/2796538071 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10015810 con 0x559a9b7bc430 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1e528f40 1 -- 172.21.15.139:0/2796538071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559a9b8db450 con 0x559a9b7bc430 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.204+0000 7f0a1e528f40 1 -- 172.21.15.139:0/2796538071 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a9b83bcf0 con 0x559a9b7bc430 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.206+0000 7f0a19826640 1 -- 172.21.15.139:0/2796538071 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a1001c020 con 0x559a9b7bc430 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.206+0000 7f0a19826640 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0a08069ee0 0x7f0a0806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.206+0000 7f0a19826640 1 -- 172.21.15.139:0/2796538071 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f0a10088cb0 con 0x559a9b7bc430 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.206+0000 7f0a1c02b640 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0a08069ee0 0x7f0a0806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.206+0000 7f0a1e528f40 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x559a9b8454d0 0x559a9b8ed2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.206+0000 7f0a1e528f40 1 -- 172.21.15.139:0/2796538071 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v9 -- 0x559a9b847150 con 0x559a9b8454d0 2024-04-25T09:56:02.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.206+0000 7f0a1c82c640 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x559a9b8454d0 0x559a9b8ed2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.206+0000 7f0a1c02b640 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0a08069ee0 0x7f0a0806c3a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c060000 tx=0x7f0a0c052000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.207+0000 7f0a1c82c640 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x559a9b8454d0 0x559a9b8ed2f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.208+0000 7f0a1c82c640 1 -- 172.21.15.139:0/2796538071 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v64'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0a14010600 con 0x559a9b8454d0 2024-04-25T09:56:02.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.208+0000 7f0a1e528f40 1 -- 172.21.15.139:0/2796538071 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x559a9b8454d0 msgr2=0x559a9b8ed2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.208+0000 7f0a1e528f40 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x559a9b8454d0 0x559a9b8ed2f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.208+0000 7f0a1e528f40 1 -- 172.21.15.139:0/2796538071 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0a08069ee0 msgr2=0x7f0a0806c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.208+0000 7f0a1e528f40 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0a08069ee0 0x7f0a0806c3a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c060000 tx=0x7f0a0c052000 comp rx=0 tx=0).stop 2024-04-25T09:56:02.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.208+0000 7f0a1e528f40 1 -- 172.21.15.139:0/2796538071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7bc430 msgr2=0x559a9b837bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.208+0000 7f0a1e528f40 1 --2- 172.21.15.139:0/2796538071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559a9b7bc430 0x559a9b837bd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x559a9b848c50 tx=0x7f0a10009eb0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.208+0000 7f0a1c82c640 1 -- 172.21.15.139:0/2796538071 reap_dead start 2024-04-25T09:56:02.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.208+0000 7f0a1e528f40 1 -- 172.21.15.139:0/2796538071 shutdown_connections 2024-04-25T09:56:02.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.208+0000 7f0a1e528f40 1 -- 172.21.15.139:0/2796538071 >> 172.21.15.139:0/2796538071 conn(0x559a9b774be0 msgr2=0x559a9b776520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.208+0000 7f0a1e528f40 1 -- 172.21.15.139:0/2796538071 shutdown_connections 2024-04-25T09:56:02.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.208+0000 7f0a1e528f40 1 -- 172.21.15.139:0/2796538071 wait complete. 2024-04-25T09:56:02.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:02.217 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-25T09:56:02.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.355+0000 7f4a2a891640 1 Processor -- start 2024-04-25T09:56:02.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.355+0000 7f4a2a891640 1 -- start start 2024-04-25T09:56:02.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a2a891640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a241617e0 0x7f4a240ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a2a891640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4a240cec80 0x7f4a240d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a2a891640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a24161dc0 con 0x7f4a2415dbf0 2024-04-25T09:56:02.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a2a891640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a240d1ee0 con 0x7f4a240cec80 2024-04-25T09:56:02.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a2a891640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a241620a0 con 0x7f4a241617e0 2024-04-25T09:56:02.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a28e2e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4a240cec80 0x7f4a240d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a28e2e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4a240cec80 0x7f4a240d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40304/0 (socket says 172.21.15.139:40304) 2024-04-25T09:56:02.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a23fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4a2415dbf0 0x7f4a240b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:44220/0 (socket says 172.21.15.139:44220) 2024-04-25T09:56:02.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a28e2e640 1 -- 172.21.15.139:0/4047357703 learned_addr learned my addr 172.21.15.139:0/4047357703 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a237fe640 1 --2- 172.21.15.139:0/4047357703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a241617e0 0x7f4a240ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a28e2e640 1 -- 172.21.15.139:0/4047357703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a241617e0 msgr2=0x7f4a240ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a28e2e640 1 --2- 172.21.15.139:0/4047357703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a241617e0 0x7f4a240ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a28e2e640 1 -- 172.21.15.139:0/4047357703 >> v1:172.21.15.139:6789/0 conn(0x7f4a2415dbf0 legacy=0x7f4a240b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a28e2e640 1 -- 172.21.15.139:0/4047357703 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a24162380 con 0x7f4a240cec80 2024-04-25T09:56:02.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.356+0000 7f4a22ffd640 1 -- 172.21.15.139:0/4047357703 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585685450 0 0) 0x7f4a24161dc0 con 0x7f4a2415dbf0 2024-04-25T09:56:02.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.357+0000 7f4a28e2e640 1 --2- 172.21.15.139:0/4047357703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a240cec80 0x7f4a240d2370 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009e50 tx=0x7f4a1c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2315d80c55ac2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.357+0000 7f4a22ffd640 1 -- 172.21.15.139:0/4047357703 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c036070 con 0x7f4a240cec80 2024-04-25T09:56:02.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.357+0000 7f4a22ffd640 1 -- 172.21.15.139:0/4047357703 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c031600 con 0x7f4a240cec80 2024-04-25T09:56:02.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.357+0000 7f4a22ffd640 1 -- 172.21.15.139:0/4047357703 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c0319d0 con 0x7f4a240cec80 2024-04-25T09:56:02.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.358+0000 7f4a2a891640 1 -- 172.21.15.139:0/4047357703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a240cec80 msgr2=0x7f4a240d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.358+0000 7f4a2a891640 1 --2- 172.21.15.139:0/4047357703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a240cec80 0x7f4a240d2370 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009e50 tx=0x7f4a1c02a680 comp rx=0 tx=0).stop 2024-04-25T09:56:02.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.358+0000 7f4a2a891640 1 -- 172.21.15.139:0/4047357703 shutdown_connections 2024-04-25T09:56:02.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.358+0000 7f4a2a891640 1 --2- 172.21.15.139:0/4047357703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a240cec80 0x7f4a240d2370 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.358+0000 7f4a2a891640 1 --2- 172.21.15.139:0/4047357703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a241617e0 0x7f4a240ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.358+0000 7f4a2a891640 1 -- 172.21.15.139:0/4047357703 >> 172.21.15.139:0/4047357703 conn(0x7f4a240b5480 msgr2=0x7f4a240b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.358+0000 7f4a2a891640 1 -- 172.21.15.139:0/4047357703 shutdown_connections 2024-04-25T09:56:02.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.358+0000 7f4a2a891640 1 -- 172.21.15.139:0/4047357703 wait complete. 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.359+0000 7f4a2a891640 1 Processor -- start 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.359+0000 7f4a2a891640 1 -- start start 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a2a891640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a240cec80 0x7f4a2415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a2a891640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a2415dbf0 0x7f4a2414d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a2a891640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a241617e0 0x7f4a2414b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a2a891640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2415dfd0 con 0x7f4a2415dbf0 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a2a891640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240d28b0 con 0x7f4a240cec80 2024-04-25T09:56:02.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a2a891640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cbfc0 con 0x7f4a241617e0 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a237fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a2415dbf0 0x7f4a2414d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a237fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a2415dbf0 0x7f4a2414d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40310/0 (socket says 172.21.15.139:40310) 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a237fe640 1 -- 172.21.15.139:0/266673342 learned_addr learned my addr 172.21.15.139:0/266673342 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a23fff640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a240cec80 0x7f4a2415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a28e2e640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a241617e0 0x7f4a2414b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a237fe640 1 -- 172.21.15.139:0/266673342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a241617e0 msgr2=0x7f4a2414b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a237fe640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a241617e0 0x7f4a2414b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a237fe640 1 -- 172.21.15.139:0/266673342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a240cec80 msgr2=0x7f4a2415e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a237fe640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a240cec80 0x7f4a2415e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a237fe640 1 -- 172.21.15.139:0/266673342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240cc2a0 con 0x7f4a2415dbf0 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a23fff640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a240cec80 0x7f4a2415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a28e2e640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a241617e0 0x7f4a2414b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a237fe640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a2415dbf0 0x7f4a2414d900 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800d620 tx=0x7f4a1800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a20ff9640 1 -- 172.21.15.139:0/266673342 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1800b620 con 0x7f4a2415dbf0 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a20ff9640 1 -- 172.21.15.139:0/266673342 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a18019070 con 0x7f4a2415dbf0 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a20ff9640 1 -- 172.21.15.139:0/266673342 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1800ba00 con 0x7f4a2415dbf0 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.360+0000 7f4a2a891640 1 -- 172.21.15.139:0/266673342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a240cc580 con 0x7f4a2415dbf0 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.361+0000 7f4a2a891640 1 -- 172.21.15.139:0/266673342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a24150b50 con 0x7f4a2415dbf0 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.362+0000 7f4a20ff9640 1 -- 172.21.15.139:0/266673342 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a1800bba0 con 0x7f4a2415dbf0 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.362+0000 7f4a20ff9640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4a1006b4d0 0x7f4a1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.362+0000 7f4a20ff9640 1 -- 172.21.15.139:0/266673342 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f4a1808a820 con 0x7f4a2415dbf0 2024-04-25T09:56:02.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.362+0000 7f4a23fff640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4a1006b4d0 0x7f4a1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.363+0000 7f4a23fff640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4a1006b4d0 0x7f4a1006d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4a14000f60 tx=0x7f4a14007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.363+0000 7f4a2d58a740 1 Processor -- start 2024-04-25T09:56:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.363+0000 7f4a2d58a740 1 -- start start 2024-04-25T09:56:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a2d58a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0e9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a2d58a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7460 0x55a8fe0e9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a2d58a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0b9ce0 0x55a8fe0bd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a2d58a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8fe022af0 con 0x55a8fe0e7080 2024-04-25T09:56:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a2d58a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8fdfcbde0 con 0x55a8fe0e7460 2024-04-25T09:56:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a2d58a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8fe01eb90 con 0x55a8fe0b9ce0 2024-04-25T09:56:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0e9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40318/0 (socket says 172.21.15.139:40318) 2024-04-25T09:56:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a23fff640 1 -- 172.21.15.139:0/1053508716 learned_addr learned my addr 172.21.15.139:0/1053508716 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a237fe640 1 --2- 172.21.15.139:0/1053508716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7460 0x55a8fe0e9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a28e2e640 1 --2- 172.21.15.139:0/1053508716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0b9ce0 0x55a8fe0bd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a23fff640 1 -- 172.21.15.139:0/1053508716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0b9ce0 msgr2=0x55a8fe0bd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a23fff640 1 --2- 172.21.15.139:0/1053508716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0b9ce0 0x55a8fe0bd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a23fff640 1 -- 172.21.15.139:0/1053508716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7460 msgr2=0x55a8fe0e9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a23fff640 1 --2- 172.21.15.139:0/1053508716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7460 0x55a8fe0e9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a23fff640 1 -- 172.21.15.139:0/1053508716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8fde037b0 con 0x55a8fe0e7080 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a28e2e640 1 --2- 172.21.15.139:0/1053508716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0b9ce0 0x55a8fe0bd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.364+0000 7f4a23fff640 1 --2- 172.21.15.139:0/1053508716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0e9890 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4a140077e0 tx=0x7f4a14012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfefa89bbecf00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.365+0000 7f4a09ffb640 1 -- 172.21.15.139:0/1053508716 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14015110 con 0x55a8fe0e7080 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.365+0000 7f4a09ffb640 1 -- 172.21.15.139:0/1053508716 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a140159c0 con 0x55a8fe0e7080 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.365+0000 7f4a09ffb640 1 -- 172.21.15.139:0/1053508716 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14015db0 con 0x55a8fe0e7080 2024-04-25T09:56:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.365+0000 7f4a2d58a740 1 -- 172.21.15.139:0/1053508716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 msgr2=0x55a8fe0e9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.365+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/1053508716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0e9890 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4a140077e0 tx=0x7f4a14012000 comp rx=0 tx=0).stop 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.365+0000 7f4a2d58a740 1 -- 172.21.15.139:0/1053508716 shutdown_connections 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.365+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/1053508716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0b9ce0 0x55a8fe0bd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.365+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/1053508716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7460 0x55a8fe0e9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.365+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/1053508716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0e9890 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.365+0000 7f4a2d58a740 1 -- 172.21.15.139:0/1053508716 >> 172.21.15.139:0/1053508716 conn(0x55a8fe0bbf50 msgr2=0x55a8fe104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.365+0000 7f4a2d58a740 1 -- 172.21.15.139:0/1053508716 shutdown_connections 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.365+0000 7f4a2d58a740 1 -- 172.21.15.139:0/1053508716 wait complete. 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.366+0000 7f4a2d58a740 1 -- start start 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.366+0000 7f4a2d58a740 1 Processor -- start 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a2d58a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7080 0x55a8fe0d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a2d58a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0e7460 0x55a8fe0d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a2d58a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe106d00 0x55a8fe0eb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a2d58a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8fe0bf100 con 0x55a8fe106d00 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a2d58a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8fdfc13e0 con 0x55a8fe0e7080 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a2d58a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8fe0bf3e0 con 0x55a8fe0e7460 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a2d58a740 10 client.0 _ll_register_callbacks cb 0x55a8fdfded50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a28e2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe106d00 0x55a8fe0eb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a28e2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe106d00 0x55a8fe0eb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40324/0 (socket says 172.21.15.139:40324) 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a28e2e640 1 -- 172.21.15.139:0/757035131 learned_addr learned my addr 172.21.15.139:0/757035131 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a23fff640 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7080 0x55a8fe0d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a237fe640 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0e7460 0x55a8fe0d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a28e2e640 1 -- 172.21.15.139:0/757035131 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0e7460 msgr2=0x55a8fe0d58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a28e2e640 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0e7460 0x55a8fe0d58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a28e2e640 1 -- 172.21.15.139:0/757035131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7080 msgr2=0x55a8fe0d51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a28e2e640 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7080 0x55a8fe0d51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a28e2e640 1 -- 172.21.15.139:0/757035131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8fde63f40 con 0x55a8fe106d00 2024-04-25T09:56:02.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a28e2e640 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe106d00 0x55a8fe0eb140 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c02a6c0 tx=0x7f4a1c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c835d3b36ae76a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a02ffd640 1 -- 172.21.15.139:0/757035131 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c036040 con 0x55a8fe106d00 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a02ffd640 1 -- 172.21.15.139:0/757035131 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f4a1c03c420 con 0x55a8fe106d00 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.367+0000 7f4a2d58a740 1 -- 172.21.15.139:0/757035131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a140159c0 con 0x55a8fe106d00 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a02ffd640 1 -- 172.21.15.139:0/757035131 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a1c04e050 con 0x55a8fe106d00 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a2d58a740 10 client.6534 fetch_fsmap learned FSMap version 28 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a2d58a740 1 -- 172.21.15.139:0/757035131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55a8fdcf70a0 con 0x55a8fe106d00 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a02ffd640 1 -- 172.21.15.139:0/757035131 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a140159c0 con 0x55a8fe106d00 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a02ffd640 1 -- 172.21.15.139:0/757035131 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4a1c051020 con 0x55a8fe106d00 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a2d58a740 10 client.6534 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a2d58a740 10 client.6534 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a2d58a740 20 client.6534 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a8fe0eb680 0x55a8fe0ec080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a2d58a740 4 client.6534 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a2d58a740 1 -- 172.21.15.139:0/757035131 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8fe030030 con 0x55a8fe0eb680 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a23fff640 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a8fe0eb680 0x55a8fe0ec080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a02ffd640 10 client.6534 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.368+0000 7f4a23fff640 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a8fe0eb680 0x55a8fe0ec080 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f14d5212d15d3c25 server_cookie=e67d6399ca3a156e in_seq=0 out_seq=0 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a02ffd640 1 -- 172.21.15.139:0/757035131 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a8fe030030 con 0x55a8fe0eb680 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a02ffd640 10 client.6534 handle_command_reply: tid=0 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a2d58a740 1 client.6534 shutdown 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a2d58a740 1 -- 172.21.15.139:0/757035131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe106d00 msgr2=0x55a8fe0eb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe106d00 0x55a8fe0eb140 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c02a6c0 tx=0x7f4a1c004ab0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a2d58a740 1 -- 172.21.15.139:0/757035131 shutdown_connections 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a8fe0eb680 0x55a8fe0ec080 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe106d00 0x55a8fe0eb140 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0e7460 0x55a8fe0d58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a02ffd640 0 client.6534 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:02.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/757035131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7080 0x55a8fe0d51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a2d58a740 1 -- 172.21.15.139:0/757035131 >> 172.21.15.139:0/757035131 conn(0x55a8fe0b9ce0 msgr2=0x55a8fe0b8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a2d58a740 1 -- 172.21.15.139:0/757035131 shutdown_connections 2024-04-25T09:56:02.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a2d58a740 1 -- 172.21.15.139:0/757035131 wait complete. 2024-04-25T09:56:02.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.369+0000 7f4a2d58a740 20 client.6534 trim_cache size 0 max 16384 2024-04-25T09:56:02.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.384+0000 7f4a2d58a740 1 Processor -- start 2024-04-25T09:56:02.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.384+0000 7f4a2d58a740 1 -- start start 2024-04-25T09:56:02.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a2d58a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0bdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a2d58a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0e7460 0x55a8fe0bdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a2d58a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe106d00 0x55a8fe103d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a2d58a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8fe0d86d0 con 0x55a8fe0e7080 2024-04-25T09:56:02.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a2d58a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a24200560 con 0x55a8fe106d00 2024-04-25T09:56:02.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a2d58a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a241ff6b0 con 0x55a8fe0e7460 2024-04-25T09:56:02.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0bdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0bdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40334/0 (socket says 172.21.15.139:40334) 2024-04-25T09:56:02.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a23fff640 1 -- 172.21.15.139:0/1585131455 learned_addr learned my addr 172.21.15.139:0/1585131455 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a28e2e640 1 --2- 172.21.15.139:0/1585131455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe106d00 0x55a8fe103d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a237fe640 1 --2- 172.21.15.139:0/1585131455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0e7460 0x55a8fe0bdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a23fff640 1 -- 172.21.15.139:0/1585131455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0e7460 msgr2=0x55a8fe0bdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a23fff640 1 --2- 172.21.15.139:0/1585131455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0e7460 0x55a8fe0bdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a23fff640 1 -- 172.21.15.139:0/1585131455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe106d00 msgr2=0x55a8fe103d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a23fff640 1 --2- 172.21.15.139:0/1585131455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe106d00 0x55a8fe103d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a23fff640 1 -- 172.21.15.139:0/1585131455 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8fdfdef00 con 0x55a8fe0e7080 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.385+0000 7f4a23fff640 1 --2- 172.21.15.139:0/1585131455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0bdab0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4a14007e20 tx=0x7f4a1400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64728da7a7c5669 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.386+0000 7f4a097fa640 1 -- 172.21.15.139:0/1585131455 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140159c0 con 0x55a8fe0e7080 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.386+0000 7f4a097fa640 1 -- 172.21.15.139:0/1585131455 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a14015b60 con 0x55a8fe0e7080 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.386+0000 7f4a097fa640 1 -- 172.21.15.139:0/1585131455 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1402e030 con 0x55a8fe0e7080 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.386+0000 7f4a2d58a740 1 -- 172.21.15.139:0/1585131455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 msgr2=0x55a8fe0bdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.386+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/1585131455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0bdab0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4a14007e20 tx=0x7f4a1400a000 comp rx=0 tx=0).stop 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.386+0000 7f4a2d58a740 1 -- 172.21.15.139:0/1585131455 shutdown_connections 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.386+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/1585131455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe106d00 0x55a8fe103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.386+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/1585131455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe0e7460 0x55a8fe0bdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.386+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/1585131455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0bdab0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.386+0000 7f4a2d58a740 1 -- 172.21.15.139:0/1585131455 >> 172.21.15.139:0/1585131455 conn(0x55a8fe0b9ce0 msgr2=0x55a8fe0ba9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.386+0000 7f4a2d58a740 1 -- 172.21.15.139:0/1585131455 shutdown_connections 2024-04-25T09:56:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.386+0000 7f4a2d58a740 1 -- 172.21.15.139:0/1585131455 wait complete. 2024-04-25T09:56:02.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.387+0000 7f4a2d58a740 1 -- start start 2024-04-25T09:56:02.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.387+0000 7f4a2d58a740 1 Processor -- start 2024-04-25T09:56:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a2d58a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a2d58a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7460 0x55a8fe0d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a2d58a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe106d00 0x55a8fe0d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a2d58a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cb6b0 con 0x55a8fe0e7080 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a2d58a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a24150830 con 0x55a8fe0e7460 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a2d58a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8fe105440 con 0x55a8fe106d00 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a2d58a740 10 client.0 _ll_register_callbacks cb 0x7f4a1c034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0e4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40344/0 (socket says 172.21.15.139:40344) 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a23fff640 1 -- 172.21.15.139:0/45422495 learned_addr learned my addr 172.21.15.139:0/45422495 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a237fe640 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7460 0x55a8fe0d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a28e2e640 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe106d00 0x55a8fe0d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a23fff640 1 -- 172.21.15.139:0/45422495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe106d00 msgr2=0x55a8fe0d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a23fff640 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe106d00 0x55a8fe0d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a23fff640 1 -- 172.21.15.139:0/45422495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7460 msgr2=0x55a8fe0d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a23fff640 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7460 0x55a8fe0d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a23fff640 1 -- 172.21.15.139:0/45422495 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8fe0b9520 con 0x55a8fe0e7080 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a23fff640 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0e4c70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x55a8fe0c15e0 tx=0x7f4a14000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7004df4c5fae6cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a02ffd640 1 -- 172.21.15.139:0/45422495 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14015e50 con 0x55a8fe0e7080 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a02ffd640 1 -- 172.21.15.139:0/45422495 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f4a14010040 con 0x55a8fe0e7080 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a2d58a740 1 -- 172.21.15.139:0/45422495 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a14015b60 con 0x55a8fe0e7080 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a02ffd640 1 -- 172.21.15.139:0/45422495 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a1403f050 con 0x55a8fe0e7080 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a2d58a740 10 client.6540 fetch_fsmap learned FSMap version 28 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.388+0000 7f4a2d58a740 1 -- 172.21.15.139:0/45422495 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55a8fe0be310 con 0x55a8fe0e7080 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.389+0000 7f4a02ffd640 1 -- 172.21.15.139:0/45422495 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a14015b60 con 0x55a8fe0e7080 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.389+0000 7f4a02ffd640 1 -- 172.21.15.139:0/45422495 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4a14042020 con 0x55a8fe0e7080 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.389+0000 7f4a2d58a740 10 client.6540 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.389+0000 7f4a2d58a740 10 client.6540 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.389+0000 7f4a2d58a740 20 client.6540 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:02.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.389+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a8fe0c1090 0x55a8fe170940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.389+0000 7f4a2d58a740 4 client.6540 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.389+0000 7f4a2d58a740 1 -- 172.21.15.139:0/45422495 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x55a8fe02a380 con 0x55a8fe0c1090 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.389+0000 7f4a237fe640 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a8fe0c1090 0x55a8fe170940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.389+0000 7f4a237fe640 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a8fe0c1090 0x55a8fe170940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=868336c486f6b7d2 server_cookie=761e631931807347 in_seq=0 out_seq=0 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.389+0000 7f4a02ffd640 10 client.6540 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a02ffd640 1 -- 172.21.15.139:0/45422495 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a8fe02a380 con 0x55a8fe0c1090 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a02ffd640 10 client.6540 handle_command_reply: tid=0 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a2d58a740 1 client.6540 shutdown 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a2d58a740 1 -- 172.21.15.139:0/45422495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 msgr2=0x55a8fe0e4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0e4c70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x55a8fe0c15e0 tx=0x7f4a14000ed0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a2d58a740 1 -- 172.21.15.139:0/45422495 shutdown_connections 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a8fe0c1090 0x55a8fe170940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a8fe106d00 0x55a8fe0d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a02ffd640 0 client.6540 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a8fe0e7460 0x55a8fe0d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a2d58a740 1 --2- 172.21.15.139:0/45422495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8fe0e7080 0x55a8fe0e4c70 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a2d58a740 1 -- 172.21.15.139:0/45422495 >> 172.21.15.139:0/45422495 conn(0x55a8fe0b9ce0 msgr2=0x55a8fe100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a2d58a740 1 -- 172.21.15.139:0/45422495 shutdown_connections 2024-04-25T09:56:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a2d58a740 1 -- 172.21.15.139:0/45422495 wait complete. 2024-04-25T09:56:02.400 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f41cfe33640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5474=0-4194304@1} caps={5474=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5474 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x564683923b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T09:56:02.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.390+0000 7f4a2d58a740 20 client.6540 trim_cache size 0 max 16384 2024-04-25T09:56:02.401 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:56:02.401 INFO:teuthology.orchestra.run.smithi139.stdout: "return_code": 0, 2024-04-25T09:56:02.401 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_tag": "d825108b-8934-464e-b93c-7fd8f4c9ff87", 2024-04-25T09:56:02.401 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": "asynchronous" 2024-04-25T09:56:02.401 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:56:02.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.391+0000 7f4a2a891640 1 -- 172.21.15.139:0/266673342 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4a1006b4d0 msgr2=0x7f4a1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.391+0000 7f4a2a891640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4a1006b4d0 0x7f4a1006d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4a14000f60 tx=0x7f4a14007c10 comp rx=0 tx=0).stop 2024-04-25T09:56:02.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.391+0000 7f4a2a891640 1 -- 172.21.15.139:0/266673342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a2415dbf0 msgr2=0x7f4a2414d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.391+0000 7f4a2a891640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a2415dbf0 0x7f4a2414d900 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800d620 tx=0x7f4a1800dae0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.391+0000 7f4a2a891640 1 -- 172.21.15.139:0/266673342 shutdown_connections 2024-04-25T09:56:02.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.391+0000 7f4a2a891640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a2415dbf0 0x7f4a2414d900 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.391+0000 7f4a2a891640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4a1006b4d0 0x7f4a1006d990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.391+0000 7f4a2a891640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a241617e0 0x7f4a2414b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.391+0000 7f4a2a891640 1 --2- 172.21.15.139:0/266673342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a240cec80 0x7f4a2415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.391+0000 7f4a2a891640 1 -- 172.21.15.139:0/266673342 >> 172.21.15.139:0/266673342 conn(0x7f4a240b5480 msgr2=0x7f4a240d2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.391+0000 7f4a2a891640 1 -- 172.21.15.139:0/266673342 shutdown_connections 2024-04-25T09:56:02.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.391+0000 7f4a2a891640 1 -- 172.21.15.139:0/266673342 wait complete. 2024-04-25T09:56:02.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:02.404 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T09:56:02.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.500+0000 7faa06722640 1 Processor -- start 2024-04-25T09:56:02.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.500+0000 7faa06722640 1 -- start start 2024-04-25T09:56:02.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7faa06722640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0015dbf0 0x7faa001617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7faa06722640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faa000cee50 0x7faa000d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7faa06722640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa00161dc0 con 0x7faa000b9370 2024-04-25T09:56:02.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7faa06722640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa000d1ee0 con 0x7faa000cee50 2024-04-25T09:56:02.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7faa06722640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa001620a0 con 0x7faa0015dbf0 2024-04-25T09:56:02.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7faa04cbf640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faa000cee50 0x7faa000d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0015dbf0 0x7faa001617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0015dbf0 0x7faa001617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:37210/0 (socket says 172.21.15.139:37210) 2024-04-25T09:56:02.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7fa9ff7fe640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7faa000b9370 0x7faa000ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:56986/0 (socket says 172.21.15.139:56986) 2024-04-25T09:56:02.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7faa04cbf640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faa000cee50 0x7faa000d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:55800/0 (socket says 172.21.15.139:55800) 2024-04-25T09:56:02.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7fa9fffff640 1 -- 172.21.15.139:0/466002098 learned_addr learned my addr 172.21.15.139:0/466002098 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7fa9fffff640 1 -- 172.21.15.139:0/466002098 >> v2:172.21.15.150:3300/0 conn(0x7faa000cee50 msgr2=0x7faa000d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7fa9fffff640 1 --2- 172.21.15.139:0/466002098 >> v2:172.21.15.150:3300/0 conn(0x7faa000cee50 0x7faa000d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7fa9fffff640 1 -- 172.21.15.139:0/466002098 >> v1:172.21.15.150:6789/0 conn(0x7faa000b9370 legacy=0x7faa000ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7fa9fffff640 1 -- 172.21.15.139:0/466002098 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00162380 con 0x7faa0015dbf0 2024-04-25T09:56:02.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.501+0000 7faa04cbf640 1 --2- 172.21.15.139:0/466002098 >> v2:172.21.15.150:3300/0 conn(0x7faa000cee50 0x7faa000d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:02.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.502+0000 7fa9fffff640 1 --2- 172.21.15.139:0/466002098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0015dbf0 0x7faa001617e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009870 tx=0x7fa9f002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=646a18dad164ade7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.502+0000 7fa9feffd640 1 -- 172.21.15.139:0/466002098 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0037070 con 0x7faa0015dbf0 2024-04-25T09:56:02.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.502+0000 7fa9feffd640 1 -- 172.21.15.139:0/466002098 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f0032440 con 0x7faa0015dbf0 2024-04-25T09:56:02.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.502+0000 7fa9feffd640 1 -- 172.21.15.139:0/466002098 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0032810 con 0x7faa0015dbf0 2024-04-25T09:56:02.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.503+0000 7faa06722640 1 -- 172.21.15.139:0/466002098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0015dbf0 msgr2=0x7faa001617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.503+0000 7faa06722640 1 --2- 172.21.15.139:0/466002098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0015dbf0 0x7faa001617e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009870 tx=0x7fa9f002a270 comp rx=0 tx=0).stop 2024-04-25T09:56:02.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.503+0000 7faa06722640 1 -- 172.21.15.139:0/466002098 shutdown_connections 2024-04-25T09:56:02.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.503+0000 7faa06722640 1 --2- 172.21.15.139:0/466002098 >> v2:172.21.15.150:3300/0 conn(0x7faa000cee50 0x7faa000d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.503+0000 7faa06722640 1 --2- 172.21.15.139:0/466002098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0015dbf0 0x7faa001617e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.503+0000 7faa06722640 1 -- 172.21.15.139:0/466002098 >> 172.21.15.139:0/466002098 conn(0x7faa000b5480 msgr2=0x7faa000b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.503+0000 7faa06722640 1 -- 172.21.15.139:0/466002098 shutdown_connections 2024-04-25T09:56:02.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.503+0000 7faa06722640 1 -- 172.21.15.139:0/466002098 wait complete. 2024-04-25T09:56:02.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.504+0000 7faa06722640 1 Processor -- start 2024-04-25T09:56:02.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.504+0000 7faa06722640 1 -- start start 2024-04-25T09:56:02.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7faa06722640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa000b9370 0x7faa0015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7faa06722640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa000cee50 0x7faa0015e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7faa06722640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0015dbf0 0x7faa0015e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7faa06722640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0015dfd0 con 0x7faa000cee50 2024-04-25T09:56:02.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7faa06722640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa000b97d0 con 0x7faa000b9370 2024-04-25T09:56:02.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7faa06722640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0014c4e0 con 0x7faa0015dbf0 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7fa9ff7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa000cee50 0x7faa0015e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7fa9ff7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa000cee50 0x7faa0015e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40350/0 (socket says 172.21.15.139:40350) 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7fa9ff7fe640 1 -- 172.21.15.139:0/3392986081 learned_addr learned my addr 172.21.15.139:0/3392986081 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7faa04cbf640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0015dbf0 0x7faa0015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7fa9fffff640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa000b9370 0x7faa0015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7fa9ff7fe640 1 -- 172.21.15.139:0/3392986081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0015dbf0 msgr2=0x7faa0015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7fa9ff7fe640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0015dbf0 0x7faa0015e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7fa9ff7fe640 1 -- 172.21.15.139:0/3392986081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa000b9370 msgr2=0x7faa0015f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7fa9ff7fe640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa000b9370 0x7faa0015f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.505+0000 7fa9ff7fe640 1 -- 172.21.15.139:0/3392986081 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00159ff0 con 0x7faa000cee50 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.506+0000 7fa9ff7fe640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa000cee50 0x7faa0015e2e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400c3c0 tx=0x7fa9f400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.506+0000 7fa9fcff9640 1 -- 172.21.15.139:0/3392986081 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40074e0 con 0x7faa000cee50 2024-04-25T09:56:02.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.506+0000 7fa9fcff9640 1 -- 172.21.15.139:0/3392986081 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f4007680 con 0x7faa000cee50 2024-04-25T09:56:02.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.506+0000 7fa9fcff9640 1 -- 172.21.15.139:0/3392986081 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4004a00 con 0x7faa000cee50 2024-04-25T09:56:02.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.506+0000 7faa06722640 1 -- 172.21.15.139:0/3392986081 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0015a2d0 con 0x7faa000cee50 2024-04-25T09:56:02.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.506+0000 7faa06722640 1 -- 172.21.15.139:0/3392986081 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0015a890 con 0x7faa000cee50 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.507+0000 7fa9fcff9640 1 -- 172.21.15.139:0/3392986081 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9f401a020 con 0x7faa000cee50 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.508+0000 7fa9fcff9640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa9ec06b4d0 0x7fa9ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.508+0000 7fa9fffff640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa9ec06b4d0 0x7fa9ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.508+0000 7fa9fcff9640 1 -- 172.21.15.139:0/3392986081 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fa9f40899d0 con 0x7faa000cee50 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.508+0000 7fa9fffff640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa9ec06b4d0 0x7fa9ec06d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0034000 tx=0x7fa9f002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.509+0000 7faa09428740 1 Processor -- start 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.509+0000 7faa09428740 1 -- start start 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.509+0000 7faa09428740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67b10 0x560422b6a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.509+0000 7faa09428740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b6a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.509+0000 7faa09428740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b3a770 0x560422b3df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.509+0000 7faa09428740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x560422aa3da0 con 0x560422b67ef0 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.509+0000 7faa09428740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560422a4cb80 con 0x560422b3a770 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.509+0000 7faa09428740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560422a9fe40 con 0x560422b67b10 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.509+0000 7fa9ff7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b6a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.509+0000 7fa9ff7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b6a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40356/0 (socket says 172.21.15.139:40356) 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.509+0000 7fa9ff7fe640 1 -- 172.21.15.139:0/879745611 learned_addr learned my addr 172.21.15.139:0/879745611 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7faa04cbf640 1 --2- 172.21.15.139:0/879745611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b3a770 0x560422b3df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7fa9fffff640 1 --2- 172.21.15.139:0/879745611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67b10 0x560422b6a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7fa9ff7fe640 1 -- 172.21.15.139:0/879745611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67b10 msgr2=0x560422b6a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7fa9ff7fe640 1 --2- 172.21.15.139:0/879745611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67b10 0x560422b6a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7fa9ff7fe640 1 -- 172.21.15.139:0/879745611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b3a770 msgr2=0x560422b3df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7fa9ff7fe640 1 --2- 172.21.15.139:0/879745611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b3a770 0x560422b3df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7fa9ff7fe640 1 -- 172.21.15.139:0/879745611 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560422a56f10 con 0x560422b67ef0 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7faa04cbf640 1 --2- 172.21.15.139:0/879745611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b3a770 0x560422b3df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7fa9fffff640 1 --2- 172.21.15.139:0/879745611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67b10 0x560422b6a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7fa9ff7fe640 1 --2- 172.21.15.139:0/879745611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b6a860 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4000f30 tx=0x7fa9f4058100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5c5c3f73eaf084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7fa9e5ffb640 1 -- 172.21.15.139:0/879745611 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40562c0 con 0x560422b67ef0 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7fa9e5ffb640 1 -- 172.21.15.139:0/879745611 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f4054630 con 0x560422b67ef0 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.510+0000 7fa9e5ffb640 1 -- 172.21.15.139:0/879745611 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4058460 con 0x560422b67ef0 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.511+0000 7faa09428740 1 -- 172.21.15.139:0/879745611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 msgr2=0x560422b6a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.511+0000 7faa09428740 1 --2- 172.21.15.139:0/879745611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b6a860 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4000f30 tx=0x7fa9f4058100 comp rx=0 tx=0).stop 2024-04-25T09:56:02.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.511+0000 7faa09428740 1 -- 172.21.15.139:0/879745611 shutdown_connections 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.511+0000 7faa09428740 1 --2- 172.21.15.139:0/879745611 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b6a860 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.511+0000 7faa09428740 1 --2- 172.21.15.139:0/879745611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b3a770 0x560422b3df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.511+0000 7faa09428740 1 --2- 172.21.15.139:0/879745611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67b10 0x560422b6a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.511+0000 7faa09428740 1 -- 172.21.15.139:0/879745611 >> 172.21.15.139:0/879745611 conn(0x560422b3c9e0 msgr2=0x560422b85320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.511+0000 7faa09428740 1 -- 172.21.15.139:0/879745611 shutdown_connections 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.511+0000 7faa09428740 1 -- 172.21.15.139:0/879745611 wait complete. 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.511+0000 7faa09428740 1 -- start start 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.511+0000 7faa09428740 1 Processor -- start 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7faa09428740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67b10 0x560422b5e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7faa09428740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67ef0 0x560422b3b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7faa09428740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b5d6a0 0x560422b3b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7faa09428740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x560422b3fb90 con 0x560422b67b10 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7faa09428740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560422a424b0 con 0x560422b5d6a0 2024-04-25T09:56:02.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7faa09428740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560422b3fe70 con 0x560422b67ef0 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7faa09428740 10 client.0 _ll_register_callbacks cb 0x560422a82180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67b10 0x560422b5e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67b10 0x560422b5e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40358/0 (socket says 172.21.15.139:40358) 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7fa9fffff640 1 -- 172.21.15.139:0/2825421547 learned_addr learned my addr 172.21.15.139:0/2825421547 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7faa04cbf640 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b5d6a0 0x560422b3b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7fa9ff7fe640 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67ef0 0x560422b3b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7fa9fffff640 1 -- 172.21.15.139:0/2825421547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67ef0 msgr2=0x560422b3b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7fa9fffff640 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67ef0 0x560422b3b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7fa9fffff640 1 -- 172.21.15.139:0/2825421547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b5d6a0 msgr2=0x560422b3b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7fa9fffff640 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b5d6a0 0x560422b3b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7fa9fffff640 1 -- 172.21.15.139:0/2825421547 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604228827d0 con 0x560422b67b10 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.512+0000 7fa9ff7fe640 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67ef0 0x560422b3b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7fa9fffff640 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67b10 0x560422b5e520 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa9f003e040 tx=0x7fa9f002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44252a90b7b25f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7fa9deffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7fa9deffd640 1 -- 172.21.15.139:0/2825421547 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0037040 con 0x560422b67b10 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7fa9deffd640 1 -- 172.21.15.139:0/2825421547 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fa9f004a670 con 0x560422b67b10 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7faa09428740 1 -- 172.21.15.139:0/2825421547 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9f4054630 con 0x560422b67b10 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7fa9deffd640 1 -- 172.21.15.139:0/2825421547 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9f004a870 con 0x560422b67b10 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7faa09428740 10 client.6549 fetch_fsmap learned FSMap version 28 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7faa09428740 1 -- 172.21.15.139:0/2825421547 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5604227b83f0 con 0x560422b67b10 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7fa9deffd640 1 -- 172.21.15.139:0/2825421547 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f4054630 con 0x560422b67b10 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7fa9deffd640 1 -- 172.21.15.139:0/2825421547 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa9f004aa10 con 0x560422b67b10 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7faa09428740 10 client.6549 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7faa09428740 10 client.6549 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7faa09428740 20 client.6549 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7faa09428740 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x560422b5fb60 0x560422b86450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7faa09428740 4 client.6549 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7faa09428740 1 -- 172.21.15.139:0/2825421547 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560422ab12e0 con 0x560422b5fb60 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.513+0000 7fa9ff7fe640 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x560422b5fb60 0x560422b86450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7fa9deffd640 10 client.6549 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7fa9ff7fe640 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x560422b5fb60 0x560422b86450 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7850f456364d1f server_cookie=db915dd1db0b066a in_seq=0 out_seq=0 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7fa9deffd640 1 -- 172.21.15.139:0/2825421547 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560422ab12e0 con 0x560422b5fb60 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7fa9deffd640 10 client.6549 handle_command_reply: tid=0 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7faa09428740 1 client.6549 shutdown 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7faa09428740 1 -- 172.21.15.139:0/2825421547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67b10 msgr2=0x560422b5e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7faa09428740 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67b10 0x560422b5e520 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa9f003e040 tx=0x7fa9f002c070 comp rx=0 tx=0).stop 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7faa09428740 1 -- 172.21.15.139:0/2825421547 shutdown_connections 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7faa09428740 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x560422b5fb60 0x560422b86450 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7faa09428740 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b5d6a0 0x560422b3b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7faa09428740 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67ef0 0x560422b3b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7fa9deffd640 0 client.6549 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:02.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7faa09428740 1 --2- 172.21.15.139:0/2825421547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67b10 0x560422b5e520 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7faa09428740 1 -- 172.21.15.139:0/2825421547 >> 172.21.15.139:0/2825421547 conn(0x560422b3a770 msgr2=0x560422b66dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7faa09428740 1 -- 172.21.15.139:0/2825421547 shutdown_connections 2024-04-25T09:56:02.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7faa09428740 1 -- 172.21.15.139:0/2825421547 wait complete. 2024-04-25T09:56:02.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.514+0000 7faa09428740 20 client.6549 trim_cache size 0 max 16384 2024-04-25T09:56:02.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.529+0000 7faa09428740 1 Processor -- start 2024-04-25T09:56:02.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.529+0000 7faa09428740 1 -- start start 2024-04-25T09:56:02.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa09428740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b5d6a0 0x560422b35b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa09428740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b67b10 0x560422b60ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa09428740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b39600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa09428740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa00200700 con 0x560422b67ef0 2024-04-25T09:56:02.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa09428740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa001ff820 con 0x560422b67b10 2024-04-25T09:56:02.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa09428740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa001ff200 con 0x560422b5d6a0 2024-04-25T09:56:02.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa04cbf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b39600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa04cbf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b39600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40360/0 (socket says 172.21.15.139:40360) 2024-04-25T09:56:02.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa04cbf640 1 -- 172.21.15.139:0/1817874059 learned_addr learned my addr 172.21.15.139:0/1817874059 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7fa9fffff640 1 --2- 172.21.15.139:0/1817874059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b5d6a0 0x560422b35b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7fa9ff7fe640 1 --2- 172.21.15.139:0/1817874059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b67b10 0x560422b60ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa04cbf640 1 -- 172.21.15.139:0/1817874059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b5d6a0 msgr2=0x560422b35b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa04cbf640 1 --2- 172.21.15.139:0/1817874059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b5d6a0 0x560422b35b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa04cbf640 1 -- 172.21.15.139:0/1817874059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b67b10 msgr2=0x560422b60ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa04cbf640 1 --2- 172.21.15.139:0/1817874059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b67b10 0x560422b60ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7faa04cbf640 1 -- 172.21.15.139:0/1817874059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604227780f0 con 0x560422b67ef0 2024-04-25T09:56:02.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.530+0000 7fa9fffff640 1 --2- 172.21.15.139:0/1817874059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b5d6a0 0x560422b35b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:02.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7faa04cbf640 1 --2- 172.21.15.139:0/1817874059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b39600 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8003d20 tx=0x7fa9f8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8eb11929df4cce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7fa9e57fa640 1 -- 172.21.15.139:0/1817874059 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80178e0 con 0x560422b67ef0 2024-04-25T09:56:02.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7fa9e57fa640 1 -- 172.21.15.139:0/1817874059 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f8017a80 con 0x560422b67ef0 2024-04-25T09:56:02.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7fa9e57fa640 1 -- 172.21.15.139:0/1817874059 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8017e50 con 0x560422b67ef0 2024-04-25T09:56:02.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7faa09428740 1 -- 172.21.15.139:0/1817874059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 msgr2=0x560422b39600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7faa09428740 1 --2- 172.21.15.139:0/1817874059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b39600 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8003d20 tx=0x7fa9f8009750 comp rx=0 tx=0).stop 2024-04-25T09:56:02.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7faa09428740 1 -- 172.21.15.139:0/1817874059 shutdown_connections 2024-04-25T09:56:02.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7faa09428740 1 --2- 172.21.15.139:0/1817874059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b39600 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7faa09428740 1 --2- 172.21.15.139:0/1817874059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b67b10 0x560422b60ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7faa09428740 1 --2- 172.21.15.139:0/1817874059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b5d6a0 0x560422b35b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7faa09428740 1 -- 172.21.15.139:0/1817874059 >> 172.21.15.139:0/1817874059 conn(0x560422b3a770 msgr2=0x560422b407a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7faa09428740 1 -- 172.21.15.139:0/1817874059 shutdown_connections 2024-04-25T09:56:02.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.531+0000 7faa09428740 1 -- 172.21.15.139:0/1817874059 wait complete. 2024-04-25T09:56:02.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.532+0000 7faa09428740 1 -- start start 2024-04-25T09:56:02.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.532+0000 7faa09428740 1 Processor -- start 2024-04-25T09:56:02.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa09428740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b5d6a0 0x560422b5bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa09428740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67b10 0x560422b86c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa09428740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b39600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa09428740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x560422b853c0 con 0x560422b67ef0 2024-04-25T09:56:02.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa09428740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560422ba8080 con 0x560422b5d6a0 2024-04-25T09:56:02.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa09428740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560422a654b0 con 0x560422b67b10 2024-04-25T09:56:02.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa09428740 10 client.0 _ll_register_callbacks cb 0x7fa9f40542e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:02.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa04cbf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b39600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa04cbf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b39600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40368/0 (socket says 172.21.15.139:40368) 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa04cbf640 1 -- 172.21.15.139:0/1131843957 learned_addr learned my addr 172.21.15.139:0/1131843957 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7fa9fffff640 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b5d6a0 0x560422b5bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7fa9ff7fe640 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67b10 0x560422b86c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa04cbf640 1 -- 172.21.15.139:0/1131843957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67b10 msgr2=0x560422b86c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa04cbf640 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67b10 0x560422b86c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa04cbf640 1 -- 172.21.15.139:0/1131843957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b5d6a0 msgr2=0x560422b5bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa04cbf640 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b5d6a0 0x560422b5bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa04cbf640 1 -- 172.21.15.139:0/1131843957 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560422ba7a60 con 0x560422b67ef0 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7faa04cbf640 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b39600 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8004a10 tx=0x7fa9f8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213113b9b5484577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.533+0000 7fa9deffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7fa9deffd640 1 -- 172.21.15.139:0/1131843957 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f802e060 con 0x560422b67ef0 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7fa9deffd640 1 -- 172.21.15.139:0/1131843957 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fa9f801f810 con 0x560422b67ef0 2024-04-25T09:56:02.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7faa09428740 1 -- 172.21.15.139:0/1131843957 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9f8017a80 con 0x560422b67ef0 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7fa9deffd640 1 -- 172.21.15.139:0/1131843957 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9f8034050 con 0x560422b67ef0 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7faa09428740 10 client.6555 fetch_fsmap learned FSMap version 28 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7faa09428740 1 -- 172.21.15.139:0/1131843957 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x560422b60e00 con 0x560422b67ef0 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7fa9deffd640 1 -- 172.21.15.139:0/1131843957 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f8017a80 con 0x560422b67ef0 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7fa9deffd640 1 -- 172.21.15.139:0/1131843957 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa9f8037020 con 0x560422b67ef0 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7faa09428740 10 client.6555 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7faa09428740 10 client.6555 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7faa09428740 20 client.6555 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7faa09428740 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x560422b3f340 0x560422b39b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7faa09428740 4 client.6555 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7faa09428740 1 -- 172.21.15.139:0/1131843957 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560422aab630 con 0x560422b3f340 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7fa9fffff640 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x560422b3f340 0x560422b39b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7fa9fffff640 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x560422b3f340 0x560422b39b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9ede6913bc48a97 server_cookie=a3de17f0047570a7 in_seq=0 out_seq=0 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7fa9deffd640 10 client.6555 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:02.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7fa9deffd640 1 -- 172.21.15.139:0/1131843957 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x560422aab630 con 0x560422b3f340 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.534+0000 7fa9deffd640 10 client.6555 handle_command_reply: tid=0 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa09428740 1 client.6555 shutdown 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa09428740 1 -- 172.21.15.139:0/1131843957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 msgr2=0x560422b39600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa09428740 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b39600 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8004a10 tx=0x7fa9f8004ab0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa09428740 1 -- 172.21.15.139:0/1131843957 shutdown_connections 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa09428740 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x560422b3f340 0x560422b39b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa09428740 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560422b67ef0 0x560422b39600 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa09428740 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560422b67b10 0x560422b86c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa09428740 1 --2- 172.21.15.139:0/1131843957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560422b5d6a0 0x560422b5bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7fa9deffd640 0 client.6555 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa09428740 1 -- 172.21.15.139:0/1131843957 >> 172.21.15.139:0/1131843957 conn(0x560422b3a770 msgr2=0x560422b6d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa09428740 1 -- 172.21.15.139:0/1131843957 shutdown_connections 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa09428740 1 -- 172.21.15.139:0/1131843957 wait complete. 2024-04-25T09:56:02.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa09428740 20 client.6555 trim_cache size 0 max 16384 2024-04-25T09:56:02.545 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:56:02.545 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:56:02.545 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": { 2024-04-25T09:56:02.545 INFO:teuthology.orchestra.run.smithi139.stdout: "d825108b-8934-464e-b93c-7fd8f4c9ff87": { 2024-04-25T09:56:02.545 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/", 2024-04-25T09:56:02.545 INFO:teuthology.orchestra.run.smithi139.stdout: "tag": "d825108b-8934-464e-b93c-7fd8f4c9ff87", 2024-04-25T09:56:02.545 INFO:teuthology.orchestra.run.smithi139.stdout: "options": "recursive" 2024-04-25T09:56:02.545 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:56:02.545 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:56:02.545 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:56:02.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa06722640 1 -- 172.21.15.139:0/3392986081 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa9ec06b4d0 msgr2=0x7fa9ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa06722640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa9ec06b4d0 0x7fa9ec06d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0034000 tx=0x7fa9f002e000 comp rx=0 tx=0).stop 2024-04-25T09:56:02.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa06722640 1 -- 172.21.15.139:0/3392986081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa000cee50 msgr2=0x7faa0015e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:02.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa06722640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa000cee50 0x7faa0015e2e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400c3c0 tx=0x7fa9f400c7c0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa06722640 1 -- 172.21.15.139:0/3392986081 shutdown_connections 2024-04-25T09:56:02.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa06722640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa000cee50 0x7faa0015e2e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa06722640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa9ec06b4d0 0x7fa9ec06d990 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa06722640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0015dbf0 0x7faa0015e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa06722640 1 --2- 172.21.15.139:0/3392986081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa000b9370 0x7faa0015f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:02.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.535+0000 7faa06722640 1 -- 172.21.15.139:0/3392986081 >> 172.21.15.139:0/3392986081 conn(0x7faa000b5480 msgr2=0x7faa000b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:02.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.536+0000 7faa06722640 1 -- 172.21.15.139:0/3392986081 shutdown_connections 2024-04-25T09:56:02.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:02.536+0000 7faa06722640 1 -- 172.21.15.139:0/3392986081 wait complete. 2024-04-25T09:56:02.549 INFO:tasks.cephfs.filesystem:scrub status for tag:d825108b-8934-464e-b93c-7fd8f4c9ff87 - {'path': '/', 'tag': 'd825108b-8934-464e-b93c-7fd8f4c9ff87', 'options': 'recursive'} 2024-04-25T09:56:07.350 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T09:56:07.347+0000 7f73a902d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T09:56:15.744 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:56:15.747 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:56:32.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:32.550 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T09:56:32.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.658+0000 7fd4b6207640 1 Processor -- start 2024-04-25T09:56:32.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.658+0000 7fd4b6207640 1 -- start start 2024-04-25T09:56:32.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4b6207640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4b0071650 0x7fd4b015e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4b6207640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b015c960 con 0x7fd4b0145230 2024-04-25T09:56:32.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4b6207640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b015cc40 con 0x7fd4b00711c0 2024-04-25T09:56:32.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4b6207640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b015d230 con 0x7fd4b0071650 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4af7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd4b0145230 0x7fd4b015be90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:39594/0 (socket says 172.21.15.139:39594) 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4af7fe640 1 -- 172.21.15.139:0/3277771635 learned_addr learned my addr 172.21.15.139:0/3277771635 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4affff640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4b0071650 0x7fd4b015e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497122909 0 0) 0x7fd4b015c960 con 0x7fd4b0145230 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd49c003520 con 0x7fd4b0145230 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296891330 0 0) 0x7fd4b015cc40 con 0x7fd4b00711c0 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4b015c960 con 0x7fd4b00711c0 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 344253469 0 0) 0x7fd49c003520 con 0x7fd4b0145230 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4b0071650 msgr2=0x7fd4b015e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4b0071650 0x7fd4b015e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 >> v1:172.21.15.150:6789/0 conn(0x7fd4b00711c0 legacy=0x7fd4b015da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b015d510 con 0x7fd4b0145230 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fd4a0001690 con 0x7fd4b0145230 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 >> v1:172.21.15.139:6789/0 conn(0x7fd4b0145230 legacy=0x7fd4b015be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd49c003b70 0x7fd49c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd49c004800 0x7fd49c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd49c0051c0 0x7fd49c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b015d230 con 0x7fd49c004800 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c00bfb0 con 0x7fd49c0051c0 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.659+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c00c1b0 con 0x7fd49c003b70 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.660+0000 7fd4affff640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4b0071650 0x7fd4b015e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.660+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd49c004800 0x7fd49c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.660+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd49c003b70 0x7fd49c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.660+0000 7fd4affff640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd49c0051c0 0x7fd49c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.660+0000 7fd4a7fff640 1 -- 172.21.15.139:0/3277771635 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd49c003b70 msgr2=0x7fd49c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.660+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd49c003b70 0x7fd49c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.660+0000 7fd4a7fff640 1 -- 172.21.15.139:0/3277771635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd49c0051c0 msgr2=0x7fd49c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.660+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd49c0051c0 0x7fd49c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.660+0000 7fd4a7fff640 1 -- 172.21.15.139:0/3277771635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c003520 con 0x7fd49c004800 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.661+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd49c004800 0x7fd49c004c80 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd4980018c0 tx=0x7fd49c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a685f232adaa9219 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.661+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4980014e0 con 0x7fd49c004800 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.661+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4980052f0 con 0x7fd49c004800 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.661+0000 7fd4aeffd640 1 -- 172.21.15.139:0/3277771635 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49802ac90 con 0x7fd49c004800 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.661+0000 7fd4b6207640 1 -- 172.21.15.139:0/3277771635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd49c004800 msgr2=0x7fd49c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.661+0000 7fd4b6207640 1 --2- 172.21.15.139:0/3277771635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd49c004800 0x7fd49c004c80 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd4980018c0 tx=0x7fd49c005830 comp rx=0 tx=0).stop 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.661+0000 7fd4affff640 1 -- 172.21.15.139:0/3277771635 reap_dead start 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.661+0000 7fd4b6207640 1 -- 172.21.15.139:0/3277771635 shutdown_connections 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.661+0000 7fd4b6207640 1 -- 172.21.15.139:0/3277771635 >> 172.21.15.139:0/3277771635 conn(0x7fd4b0067000 msgr2=0x7fd4b006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.661+0000 7fd4b6207640 1 -- 172.21.15.139:0/3277771635 shutdown_connections 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.661+0000 7fd4b6207640 1 -- 172.21.15.139:0/3277771635 wait complete. 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.663+0000 7fd4b6207640 1 Processor -- start 2024-04-25T09:56:32.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.663+0000 7fd4b6207640 1 -- start start 2024-04-25T09:56:32.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.663+0000 7fd4b6207640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd49c004800 0x7fd4b00ab830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.663+0000 7fd4b6207640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4b00c6270 0x7fd4b00ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.663+0000 7fd4b6207640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4b00c66d0 0x7fd4b00adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.663+0000 7fd4b6207640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b015c5a0 con 0x7fd4b00c6270 2024-04-25T09:56:32.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.663+0000 7fd4b6207640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b006e4b0 con 0x7fd49c004800 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.663+0000 7fd4b6207640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b015c720 con 0x7fd4b00c66d0 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.663+0000 7fd4a7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4b00c6270 0x7fd4b00ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.663+0000 7fd4a7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4b00c6270 0x7fd4b00ada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56018/0 (socket says 172.21.15.139:56018) 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.663+0000 7fd4a7fff640 1 -- 172.21.15.139:0/598400583 learned_addr learned my addr 172.21.15.139:0/598400583 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4affff640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4b00c66d0 0x7fd4b00adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd49c004800 0x7fd4b00ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4a7fff640 1 -- 172.21.15.139:0/598400583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4b00c66d0 msgr2=0x7fd4b00adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4b00c66d0 0x7fd4b00adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4a7fff640 1 -- 172.21.15.139:0/598400583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd49c004800 msgr2=0x7fd4b00ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd49c004800 0x7fd4b00ab830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4a7fff640 1 -- 172.21.15.139:0/598400583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b00ae5d0 con 0x7fd4b00c6270 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4b00c6270 0x7fd4b00ada80 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd498008fd0 tx=0x7fd4980012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4acff9640 1 -- 172.21.15.139:0/598400583 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498007a50 con 0x7fd4b00c6270 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4acff9640 1 -- 172.21.15.139:0/598400583 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd498041070 con 0x7fd4b00c6270 2024-04-25T09:56:32.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4b6207640 1 -- 172.21.15.139:0/598400583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b00c4770 con 0x7fd4b00c6270 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4acff9640 1 -- 172.21.15.139:0/598400583 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4980069d0 con 0x7fd4b00c6270 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.664+0000 7fd4b6207640 1 -- 172.21.15.139:0/598400583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b00c4d30 con 0x7fd4b00c6270 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.666+0000 7fd4acff9640 1 -- 172.21.15.139:0/598400583 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4980052f0 con 0x7fd4b00c6270 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.666+0000 7fd4acff9640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd49006b500 0x7fd49006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.666+0000 7fd4acff9640 1 -- 172.21.15.139:0/598400583 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fd4980aa730 con 0x7fd4b00c6270 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.666+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd49006b500 0x7fd49006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.666+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd49006b500 0x7fd49006d9c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0001050 tx=0x7fd4a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.667+0000 7fd4b8fe9740 1 Processor -- start 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.667+0000 7fd4b8fe9740 1 -- start start 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4b8fe9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 0x56130ef998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4b8fe9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130ef69770 0x56130ef6f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4b8fe9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb5470 0x56130efb58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4b8fe9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130eed2da0 con 0x56130ef99510 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4b8fe9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130ee7bb80 con 0x56130efb5470 2024-04-25T09:56:32.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4b8fe9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56130eecee40 con 0x56130ef69770 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4af7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 0x56130ef998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4af7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 0x56130ef998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56030/0 (socket says 172.21.15.139:56030) 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4af7fe640 1 -- 172.21.15.139:0/3253100662 learned_addr learned my addr 172.21.15.139:0/3253100662 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4affff640 1 --2- 172.21.15.139:0/3253100662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb5470 0x56130efb58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/3253100662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130ef69770 0x56130ef6f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4af7fe640 1 -- 172.21.15.139:0/3253100662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130ef69770 msgr2=0x56130ef6f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/3253100662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130ef69770 0x56130ef6f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4af7fe640 1 -- 172.21.15.139:0/3253100662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb5470 msgr2=0x56130efb58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/3253100662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb5470 0x56130efb58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4af7fe640 1 -- 172.21.15.139:0/3253100662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56130ee85f10 con 0x56130ef99510 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.668+0000 7fd4affff640 1 --2- 172.21.15.139:0/3253100662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb5470 0x56130efb58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.669+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/3253100662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 0x56130ef998f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0007250 tx=0x7fd4a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7448a36ac8a0bd74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.669+0000 7fd4a57fa640 1 -- 172.21.15.139:0/3253100662 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0014b80 con 0x56130ef99510 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.669+0000 7fd4a57fa640 1 -- 172.21.15.139:0/3253100662 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a0014d20 con 0x56130ef99510 2024-04-25T09:56:32.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.669+0000 7fd4a57fa640 1 -- 172.21.15.139:0/3253100662 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0022620 con 0x56130ef99510 2024-04-25T09:56:32.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.669+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/3253100662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 msgr2=0x56130ef998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.669+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/3253100662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 0x56130ef998f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0007250 tx=0x7fd4a0013000 comp rx=0 tx=0).stop 2024-04-25T09:56:32.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.669+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/3253100662 shutdown_connections 2024-04-25T09:56:32.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.669+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/3253100662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb5470 0x56130efb58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.669+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/3253100662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130ef69770 0x56130ef6f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.669+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/3253100662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 0x56130ef998f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.669+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/3253100662 >> 172.21.15.139:0/3253100662 conn(0x56130ef6b9e0 msgr2=0x56130efb4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:32.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.669+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/3253100662 shutdown_connections 2024-04-25T09:56:32.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.670+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/3253100662 wait complete. 2024-04-25T09:56:32.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.670+0000 7fd4b8fe9740 1 -- start start 2024-04-25T09:56:32.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.670+0000 7fd4b8fe9740 1 Processor -- start 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4b8fe9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 0x56130efc0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4b8fe9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130efb5470 0x56130ef7da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4b8fe9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb7030 0x56130ef7df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4b8fe9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130ef6fce0 con 0x56130ef99510 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4b8fe9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130ee714b0 con 0x56130efb7030 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4b8fe9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56130ef6f310 con 0x56130efb5470 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4b8fe9740 10 client.0 _ll_register_callbacks cb 0x56130eeb1180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4af7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 0x56130efc0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4af7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 0x56130efc0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56034/0 (socket says 172.21.15.139:56034) 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4af7fe640 1 -- 172.21.15.139:0/1840391879 learned_addr learned my addr 172.21.15.139:0/1840391879 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4af7fe640 1 -- 172.21.15.139:0/1840391879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130efb5470 msgr2=0x56130ef7da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/1840391879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130efb5470 0x56130ef7da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4af7fe640 1 -- 172.21.15.139:0/1840391879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb7030 msgr2=0x56130ef7df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/1840391879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb7030 0x56130ef7df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4af7fe640 1 -- 172.21.15.139:0/1840391879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56130ecb17d0 con 0x56130ef99510 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/1840391879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 0x56130efc0730 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0001eb0 tx=0x7fd4b015d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf20a726f510182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.671+0000 7fd48e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd48e7fc640 1 -- 172.21.15.139:0/1840391879 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0022960 con 0x56130ef99510 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd48e7fc640 1 -- 172.21.15.139:0/1840391879 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fd4a0022b00 con 0x56130ef99510 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/1840391879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4a0014d20 con 0x56130ef99510 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd48e7fc640 1 -- 172.21.15.139:0/1840391879 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4a003a050 con 0x56130ef99510 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd4b8fe9740 10 client.6567 fetch_fsmap learned FSMap version 29 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/1840391879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x56130ebe73f0 con 0x56130ef99510 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd48e7fc640 1 -- 172.21.15.139:0/1840391879 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a0014d20 con 0x56130ef99510 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd48e7fc640 1 -- 172.21.15.139:0/1840391879 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd4a003d020 con 0x56130ef99510 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd4b8fe9740 10 client.6567 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd4b8fe9740 10 client.6567 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd4b8fe9740 20 client.6567 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:32.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/1840391879 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x56130efd40b0 0x56130efd64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd4b8fe9740 4 client.6567 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/1840391879 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56130eee02e0 con 0x56130efd40b0 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/1840391879 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x56130efd40b0 0x56130efd64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/1840391879 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x56130efd40b0 0x56130efd64a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5936b3f0c7e5b8c6 server_cookie=fa97f492b217bead in_seq=0 out_seq=0 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.672+0000 7fd48e7fc640 10 client.6567 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd48e7fc640 1 -- 172.21.15.139:0/1840391879 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56130eee02e0 con 0x56130efd40b0 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd48e7fc640 10 client.6567 handle_command_reply: tid=0 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd4b8fe9740 1 client.6567 shutdown 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/1840391879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 msgr2=0x56130efc0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/1840391879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 0x56130efc0730 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0001eb0 tx=0x7fd4b015d6e0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/1840391879 shutdown_connections 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/1840391879 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x56130efd40b0 0x56130efd64a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/1840391879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb7030 0x56130ef7df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/1840391879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130efb5470 0x56130ef7da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd48e7fc640 0 client.6567 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:32.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/1840391879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130ef99510 0x56130efc0730 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/1840391879 >> 172.21.15.139:0/1840391879 conn(0x56130ef69770 msgr2=0x56130ef96440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:32.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/1840391879 shutdown_connections 2024-04-25T09:56:32.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/1840391879 wait complete. 2024-04-25T09:56:32.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.673+0000 7fd4b8fe9740 20 client.6567 trim_cache size 0 max 16384 2024-04-25T09:56:32.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.688+0000 7fd4b8fe9740 1 Processor -- start 2024-04-25T09:56:32.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.688+0000 7fd4b8fe9740 1 -- start start 2024-04-25T09:56:32.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.689+0000 7fd4b8fe9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130ef99510 0x56130ef706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.689+0000 7fd4b8fe9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 0x56130efda870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.689+0000 7fd4b8fe9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb7030 0x56130ef6f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.689+0000 7fd4b8fe9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b01fca50 con 0x56130efb5470 2024-04-25T09:56:32.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.689+0000 7fd4b8fe9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b01fbc90 con 0x56130efb7030 2024-04-25T09:56:32.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.689+0000 7fd4b8fe9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00b4260 con 0x56130ef99510 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.689+0000 7fd4a7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 0x56130efda870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4a7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 0x56130efda870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56046/0 (socket says 172.21.15.139:56046) 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4a7fff640 1 -- 172.21.15.139:0/2727694217 learned_addr learned my addr 172.21.15.139:0/2727694217 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/2727694217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130ef99510 0x56130ef706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4affff640 1 --2- 172.21.15.139:0/2727694217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb7030 0x56130ef6f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4a7fff640 1 -- 172.21.15.139:0/2727694217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130ef99510 msgr2=0x56130ef706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/2727694217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130ef99510 0x56130ef706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4a7fff640 1 -- 172.21.15.139:0/2727694217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb7030 msgr2=0x56130ef6f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/2727694217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb7030 0x56130ef6f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4a7fff640 1 -- 172.21.15.139:0/2727694217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56130efb92c0 con 0x56130efb5470 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/2727694217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130ef99510 0x56130ef706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4affff640 1 --2- 172.21.15.139:0/2727694217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb7030 0x56130ef6f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/2727694217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 0x56130efda870 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd49802a700 tx=0x7fd4980aa040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6734423f13031551 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4a4ff9640 1 -- 172.21.15.139:0/2727694217 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4980a6020 con 0x56130efb5470 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4a4ff9640 1 -- 172.21.15.139:0/2727694217 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd498078d20 con 0x56130efb5470 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.690+0000 7fd4a4ff9640 1 -- 172.21.15.139:0/2727694217 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4980a8e60 con 0x56130efb5470 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.691+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/2727694217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 msgr2=0x56130efda870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.691+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/2727694217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 0x56130efda870 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd49802a700 tx=0x7fd4980aa040 comp rx=0 tx=0).stop 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.691+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/2727694217 shutdown_connections 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.691+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/2727694217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 0x56130efda870 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.691+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/2727694217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130efb7030 0x56130ef6f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.691+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/2727694217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130ef99510 0x56130ef706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.691+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/2727694217 >> 172.21.15.139:0/2727694217 conn(0x56130ef69770 msgr2=0x56130efc0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.691+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/2727694217 shutdown_connections 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.691+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/2727694217 wait complete. 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.691+0000 7fd4b8fe9740 1 -- start start 2024-04-25T09:56:32.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.691+0000 7fd4b8fe9740 1 Processor -- start 2024-04-25T09:56:32.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4b8fe9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130ef99510 0x56130efda870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4b8fe9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 0x56130efdadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4b8fe9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130efb7030 0x56130f01fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4b8fe9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130ee94520 con 0x56130efb5470 2024-04-25T09:56:32.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4b8fe9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b01fb7f0 con 0x56130ef99510 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4b8fe9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56130ef9a3d0 con 0x56130efb7030 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4b8fe9740 10 client.0 _ll_register_callbacks cb 0x7fd4a00070c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4a7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 0x56130efdadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4a7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 0x56130efdadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56052/0 (socket says 172.21.15.139:56052) 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4a7fff640 1 -- 172.21.15.139:0/851641830 learned_addr learned my addr 172.21.15.139:0/851641830 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4affff640 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130efb7030 0x56130f01fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130ef99510 0x56130efda870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4a7fff640 1 -- 172.21.15.139:0/851641830 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130efb7030 msgr2=0x56130f01fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130efb7030 0x56130f01fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4a7fff640 1 -- 172.21.15.139:0/851641830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130ef99510 msgr2=0x56130efda870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130ef99510 0x56130efda870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4a7fff640 1 -- 172.21.15.139:0/851641830 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56130eba70f0 con 0x56130efb5470 2024-04-25T09:56:32.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.692+0000 7fd4affff640 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130efb7030 0x56130f01fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd4a7fff640 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 0x56130efdadb0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd498078a20 tx=0x7fd49803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f617f80966f5723 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd48e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd48e7fc640 1 -- 172.21.15.139:0/851641830 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498097060 con 0x56130efb5470 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd48e7fc640 1 -- 172.21.15.139:0/851641830 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fd498077020 con 0x56130efb5470 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/851641830 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd498078d20 con 0x56130efb5470 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd48e7fc640 1 -- 172.21.15.139:0/851641830 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd49809e050 con 0x56130efb5470 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd4b8fe9740 10 client.6573 fetch_fsmap learned FSMap version 29 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/851641830 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x56130ef95cf0 con 0x56130efb5470 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd48e7fc640 1 -- 172.21.15.139:0/851641830 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd498078d20 con 0x56130efb5470 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd48e7fc640 1 -- 172.21.15.139:0/851641830 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd49808cce0 con 0x56130efb5470 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd4b8fe9740 10 client.6573 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd4b8fe9740 10 client.6573 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd4b8fe9740 20 client.6573 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x56130efb5a80 0x56130f020240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd4b8fe9740 4 client.6573 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/851641830 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56130efb6ba0 con 0x56130efb5a80 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.693+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x56130efb5a80 0x56130f020240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4af7fe640 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x56130efb5a80 0x56130f020240 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38261d503c0567aa server_cookie=f9b5c84ab40e18f5 in_seq=0 out_seq=0 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd48e7fc640 10 client.6573 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd48e7fc640 1 -- 172.21.15.139:0/851641830 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56130efb6ba0 con 0x56130efb5a80 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd48e7fc640 10 client.6573 handle_command_reply: tid=0 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b8fe9740 1 client.6573 shutdown 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/851641830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 msgr2=0x56130efdadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 0x56130efdadb0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd498078a20 tx=0x7fd49803b000 comp rx=0 tx=0).stop 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/851641830 shutdown_connections 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x56130efb5a80 0x56130f020240 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56130efb5470 0x56130efdadb0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56130efb7030 0x56130f01fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd48e7fc640 0 client.6573 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b8fe9740 1 --2- 172.21.15.139:0/851641830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56130ef99510 0x56130efda870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/851641830 >> 172.21.15.139:0/851641830 conn(0x56130ef69770 msgr2=0x56130ef91e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:32.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/851641830 shutdown_connections 2024-04-25T09:56:32.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b8fe9740 1 -- 172.21.15.139:0/851641830 wait complete. 2024-04-25T09:56:32.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b8fe9740 20 client.6573 trim_cache size 0 max 16384 2024-04-25T09:56:32.705 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:56:32.705 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "no active scrubs running", 2024-04-25T09:56:32.705 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": {} 2024-04-25T09:56:32.705 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:56:32.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b6207640 1 -- 172.21.15.139:0/598400583 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd49006b500 msgr2=0x7fd49006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b6207640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd49006b500 0x7fd49006d9c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0001050 tx=0x7fd4a0009040 comp rx=0 tx=0).stop 2024-04-25T09:56:32.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b6207640 1 -- 172.21.15.139:0/598400583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4b00c6270 msgr2=0x7fd4b00ada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.694+0000 7fd4b6207640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4b00c6270 0x7fd4b00ada80 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd498008fd0 tx=0x7fd4980012d0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.695+0000 7fd4b6207640 1 -- 172.21.15.139:0/598400583 shutdown_connections 2024-04-25T09:56:32.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.695+0000 7fd4b6207640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4b00c6270 0x7fd4b00ada80 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.695+0000 7fd4b6207640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd49006b500 0x7fd49006d9c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.695+0000 7fd4b6207640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4b00c66d0 0x7fd4b00adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.695+0000 7fd4b6207640 1 --2- 172.21.15.139:0/598400583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd49c004800 0x7fd4b00ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.695+0000 7fd4b6207640 1 -- 172.21.15.139:0/598400583 >> 172.21.15.139:0/598400583 conn(0x7fd4b0067000 msgr2=0x7fd4b0144440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:32.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.695+0000 7fd4b6207640 1 -- 172.21.15.139:0/598400583 shutdown_connections 2024-04-25T09:56:32.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.695+0000 7fd4b6207640 1 -- 172.21.15.139:0/598400583 wait complete. 2024-04-25T09:56:32.709 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:56:32.710 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:32.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.802+0000 7fe28371b640 1 Processor -- start 2024-04-25T09:56:32.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.802+0000 7fe28371b640 1 -- start start 2024-04-25T09:56:32.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.803+0000 7fe28371b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe27c15a860 0x7fe27c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.803+0000 7fe28371b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe27c0cd1f0 0x7fe27c06e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.803+0000 7fe28371b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27c0af6d0 con 0x7fe27c15ac40 2024-04-25T09:56:32.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.803+0000 7fe28371b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27c0b02e0 con 0x7fe27c15a860 2024-04-25T09:56:32.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.803+0000 7fe28371b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0af9b0 con 0x7fe27c0cd1f0 2024-04-25T09:56:32.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.803+0000 7fe281cb8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe27c15ac40 0x7fe27c15e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:39600/0 (socket says 172.21.15.139:39600) 2024-04-25T09:56:32.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.803+0000 7fe281cb8640 1 -- 172.21.15.139:0/1653224602 learned_addr learned my addr 172.21.15.139:0/1653224602 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:56:32.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408967013 0 0) 0x7fe27c0af6d0 con 0x7fe27c15ac40 2024-04-25T09:56:32.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe268003520 con 0x7fe27c15ac40 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe2814b7640 1 --2- 172.21.15.139:0/1653224602 >> v2:172.21.15.150:3300/0 conn(0x7fe27c15a860 0x7fe27c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe280cb6640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe27c0cd1f0 0x7fe27c06e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1676490997 0 0) 0x7fe268003520 con 0x7fe27c15ac40 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe27c0cd1f0 msgr2=0x7fe27c06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe27c0cd1f0 0x7fe27c06e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 >> v2:172.21.15.150:3300/0 conn(0x7fe27c15a860 msgr2=0x7fe27c0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 --2- 172.21.15.139:0/1653224602 >> v2:172.21.15.150:3300/0 conn(0x7fe27c15a860 0x7fe27c0cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c0afc90 con 0x7fe27c15ac40 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fe274001670 con 0x7fe27c15ac40 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe280cb6640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe27c0cd1f0 0x7fe27c06e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 >> v1:172.21.15.139:6789/0 conn(0x7fe27c15ac40 legacy=0x7fe27c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2680039b0 0x7fe268003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe27c0cd1f0 0x7fe268004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe268004d40 0x7fe2680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0b02e0 con 0x7fe2680039b0 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0af9b0 con 0x7fe268004d40 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26800dbd0 con 0x7fe27c0cd1f0 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe2814b7640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2680039b0 0x7fe268003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe280cb6640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe27c0cd1f0 0x7fe268004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe281cb8640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe268004d40 0x7fe2680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe2814b7640 1 -- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe27c0cd1f0 msgr2=0x7fe268004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe2814b7640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe27c0cd1f0 0x7fe268004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe2814b7640 1 -- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe268004d40 msgr2=0x7fe2680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe2814b7640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe268004d40 0x7fe2680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe2814b7640 1 -- 172.21.15.139:0/1653224602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe268003520 con 0x7fe2680039b0 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.804+0000 7fe281cb8640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe268004d40 0x7fe2680051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.805+0000 7fe2814b7640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2680039b0 0x7fe268003dd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe278009fc0 tx=0x7fe268005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d33d8e3592379d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.805+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278009c10 con 0x7fe2680039b0 2024-04-25T09:56:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.805+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe278036070 con 0x7fe2680039b0 2024-04-25T09:56:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.805+0000 7fe273fff640 1 -- 172.21.15.139:0/1653224602 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27802d070 con 0x7fe2680039b0 2024-04-25T09:56:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.806+0000 7fe28371b640 1 -- 172.21.15.139:0/1653224602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2680039b0 msgr2=0x7fe268003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.806+0000 7fe28371b640 1 --2- 172.21.15.139:0/1653224602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2680039b0 0x7fe268003dd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe278009fc0 tx=0x7fe268005830 comp rx=0 tx=0).stop 2024-04-25T09:56:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.806+0000 7fe281cb8640 1 -- 172.21.15.139:0/1653224602 reap_dead start 2024-04-25T09:56:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.806+0000 7fe28371b640 1 -- 172.21.15.139:0/1653224602 shutdown_connections 2024-04-25T09:56:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.806+0000 7fe28371b640 1 -- 172.21.15.139:0/1653224602 >> 172.21.15.139:0/1653224602 conn(0x7fe27c066f20 msgr2=0x7fe27c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.806+0000 7fe28371b640 1 -- 172.21.15.139:0/1653224602 shutdown_connections 2024-04-25T09:56:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.806+0000 7fe28371b640 1 -- 172.21.15.139:0/1653224602 wait complete. 2024-04-25T09:56:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.807+0000 7fe28371b640 1 Processor -- start 2024-04-25T09:56:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.807+0000 7fe28371b640 1 -- start start 2024-04-25T09:56:32.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.807+0000 7fe28371b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2680039b0 0x7fe27c140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.807+0000 7fe28371b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe27c0b3bb0 0x7fe27c14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.807+0000 7fe28371b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe27c15cfb0 0x7fe27c150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe28371b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0af420 con 0x7fe27c0b3bb0 2024-04-25T09:56:32.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe28371b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0af2a0 con 0x7fe27c15cfb0 2024-04-25T09:56:32.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe28371b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c15e950 con 0x7fe2680039b0 2024-04-25T09:56:32.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe280cb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe27c0b3bb0 0x7fe27c14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe280cb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe27c0b3bb0 0x7fe27c14c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56070/0 (socket says 172.21.15.139:56070) 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe280cb6640 1 -- 172.21.15.139:0/1692538398 learned_addr learned my addr 172.21.15.139:0/1692538398 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe2814b7640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2680039b0 0x7fe27c140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe281cb8640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe27c15cfb0 0x7fe27c150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe280cb6640 1 -- 172.21.15.139:0/1692538398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2680039b0 msgr2=0x7fe27c140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe280cb6640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2680039b0 0x7fe27c140190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe280cb6640 1 -- 172.21.15.139:0/1692538398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe27c15cfb0 msgr2=0x7fe27c150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe280cb6640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe27c15cfb0 0x7fe27c150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe280cb6640 1 -- 172.21.15.139:0/1692538398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c14cdd0 con 0x7fe27c0b3bb0 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe280cb6640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe27c0b3bb0 0x7fe27c14c890 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe26c010890 tx=0x7fe26c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe271ffb640 1 -- 172.21.15.139:0/1692538398 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c01a070 con 0x7fe27c0b3bb0 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe271ffb640 1 -- 172.21.15.139:0/1692538398 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c00cb40 con 0x7fe27c0b3bb0 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe28371b640 1 -- 172.21.15.139:0/1692538398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c15e180 con 0x7fe27c0b3bb0 2024-04-25T09:56:32.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.808+0000 7fe271ffb640 1 -- 172.21.15.139:0/1692538398 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00f040 con 0x7fe27c0b3bb0 2024-04-25T09:56:32.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.809+0000 7fe28371b640 1 -- 172.21.15.139:0/1692538398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c15dbc0 con 0x7fe27c0b3bb0 2024-04-25T09:56:32.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.810+0000 7fe28371b640 1 -- 172.21.15.139:0/1692538398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27c074520 con 0x7fe27c0b3bb0 2024-04-25T09:56:32.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.810+0000 7fe271ffb640 1 -- 172.21.15.139:0/1692538398 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe26c015470 con 0x7fe27c0b3bb0 2024-04-25T09:56:32.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.810+0000 7fe271ffb640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe26006b4d0 0x7fe26006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:32.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.811+0000 7fe271ffb640 1 -- 172.21.15.139:0/1692538398 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fe26c08aa00 con 0x7fe27c0b3bb0 2024-04-25T09:56:32.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.811+0000 7fe2814b7640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe26006b4d0 0x7fe26006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:32.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.811+0000 7fe2814b7640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe26006b4d0 0x7fe26006d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe278035000 tx=0x7fe27802f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:32.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:32.815+0000 7fe271ffb640 1 -- 172.21.15.139:0/1692538398 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe26c059660 con 0x7fe27c0b3bb0 2024-04-25T09:56:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.021+0000 7fe28371b640 1 -- 172.21.15.139:0/1692538398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe27c071f60 con 0x7fe27c0b3bb0 2024-04-25T09:56:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.021+0000 7fe271ffb640 1 -- 172.21.15.139:0/1692538398 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4945 (secure 0 0 0) 0x7fe26c058f60 con 0x7fe27c0b3bb0 2024-04-25T09:56:33.026 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:56:33.026 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/829268166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":829268166},{"type":"v1","addr":"172.21.15.150:6835","nonce":829268166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3075039902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3075039902},{"type":"v1","addr":"172.21.15.150:6834","nonce":3075039902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/627726069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":627726069},{"type":"v1","addr":"172.21.15.139:6836","nonce":627726069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:50.626292+0000","modified":"2024-04-25T09:56:07.327126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":7,"addr":"172.21.15.139:6837/3541444834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3541444834},{"type":"v1","addr":"172.21.15.139:6837","nonce":3541444834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:56:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 29 2024-04-25T09:56:33.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.023+0000 7fe28371b640 1 -- 172.21.15.139:0/1692538398 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe26006b4d0 msgr2=0x7fe26006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.023+0000 7fe28371b640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe26006b4d0 0x7fe26006d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe278035000 tx=0x7fe27802f000 comp rx=0 tx=0).stop 2024-04-25T09:56:33.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.023+0000 7fe28371b640 1 -- 172.21.15.139:0/1692538398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe27c0b3bb0 msgr2=0x7fe27c14c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.023+0000 7fe28371b640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe27c0b3bb0 0x7fe27c14c890 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe26c010890 tx=0x7fe26c010d50 comp rx=0 tx=0).stop 2024-04-25T09:56:33.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.023+0000 7fe28371b640 1 -- 172.21.15.139:0/1692538398 shutdown_connections 2024-04-25T09:56:33.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.023+0000 7fe28371b640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe27c0b3bb0 0x7fe27c14c890 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.023+0000 7fe28371b640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe26006b4d0 0x7fe26006d990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.023+0000 7fe28371b640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe27c15cfb0 0x7fe27c150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.023+0000 7fe28371b640 1 --2- 172.21.15.139:0/1692538398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2680039b0 0x7fe27c140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.023+0000 7fe28371b640 1 -- 172.21.15.139:0/1692538398 >> 172.21.15.139:0/1692538398 conn(0x7fe27c066f20 msgr2=0x7fe27c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.023+0000 7fe28371b640 1 -- 172.21.15.139:0/1692538398 shutdown_connections 2024-04-25T09:56:33.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.023+0000 7fe28371b640 1 -- 172.21.15.139:0/1692538398 wait complete. 2024-04-25T09:56:33.039 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T09:56:33.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.133+0000 7feb93bf4640 1 Processor -- start 2024-04-25T09:56:33.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.134+0000 7feb93bf4640 1 -- start start 2024-04-25T09:56:33.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.134+0000 7feb93bf4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb8c0d0a80 0x7feb8c1618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.134+0000 7feb93bf4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feb8c09ad40 0x7feb8c161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.134+0000 7feb93bf4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb8c099ca0 con 0x7feb8c0cd610 2024-04-25T09:56:33.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.134+0000 7feb93bf4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8c099a50 con 0x7feb8c09ad40 2024-04-25T09:56:33.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.134+0000 7feb93bf4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c099f80 con 0x7feb8c0d0a80 2024-04-25T09:56:33.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.135+0000 7feb9118f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb8c0d0a80 0x7feb8c1618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.135+0000 7feb92191640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feb8c09ad40 0x7feb8c161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.135+0000 7feb9118f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb8c0d0a80 0x7feb8c1618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:46412/0 (socket says 172.21.15.139:46412) 2024-04-25T09:56:33.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.135+0000 7feb92191640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feb8c09ad40 0x7feb8c161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:43492/0 (socket says 172.21.15.139:43492) 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.135+0000 7feb9118f640 1 -- 172.21.15.139:0/1071672099 learned_addr learned my addr 172.21.15.139:0/1071672099 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.135+0000 7feb91990640 1 --1- 172.21.15.139:0/1071672099 >> v1:172.21.15.150:6789/0 conn(0x7feb8c0cd610 0x7feb8c0d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:44574/0 (socket says 172.21.15.139:44574) 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.135+0000 7feb92191640 1 -- 172.21.15.139:0/1071672099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb8c0d0a80 msgr2=0x7feb8c1618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.135+0000 7feb92191640 1 --2- 172.21.15.139:0/1071672099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb8c0d0a80 0x7feb8c1618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.135+0000 7feb92191640 1 -- 172.21.15.139:0/1071672099 >> v1:172.21.15.150:6789/0 conn(0x7feb8c0cd610 legacy=0x7feb8c0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.135+0000 7feb92191640 1 -- 172.21.15.139:0/1071672099 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8c09a260 con 0x7feb8c09ad40 2024-04-25T09:56:33.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.135+0000 7feb9098e640 1 -- 172.21.15.139:0/1071672099 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800247618 0 0) 0x7feb8c099ca0 con 0x7feb8c0cd610 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.136+0000 7feb92191640 1 --2- 172.21.15.139:0/1071672099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c09ad40 0x7feb8c161e00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feb840097a0 tx=0x7feb8402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c67b052655c64b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.136+0000 7feb9098e640 1 -- 172.21.15.139:0/1071672099 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84037070 con 0x7feb8c09ad40 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.136+0000 7feb9098e640 1 -- 172.21.15.139:0/1071672099 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb84032440 con 0x7feb8c09ad40 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.136+0000 7feb9098e640 1 -- 172.21.15.139:0/1071672099 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84032810 con 0x7feb8c09ad40 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.137+0000 7feb93bf4640 1 -- 172.21.15.139:0/1071672099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c09ad40 msgr2=0x7feb8c161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.137+0000 7feb93bf4640 1 --2- 172.21.15.139:0/1071672099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c09ad40 0x7feb8c161e00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feb840097a0 tx=0x7feb8402a210 comp rx=0 tx=0).stop 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.137+0000 7feb93bf4640 1 -- 172.21.15.139:0/1071672099 shutdown_connections 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.137+0000 7feb93bf4640 1 --2- 172.21.15.139:0/1071672099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb8c0d0a80 0x7feb8c1618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.137+0000 7feb93bf4640 1 --2- 172.21.15.139:0/1071672099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c09ad40 0x7feb8c161e00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.137+0000 7feb93bf4640 1 -- 172.21.15.139:0/1071672099 >> 172.21.15.139:0/1071672099 conn(0x7feb8c0b31f0 msgr2=0x7feb8c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.137+0000 7feb93bf4640 1 -- 172.21.15.139:0/1071672099 shutdown_connections 2024-04-25T09:56:33.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.137+0000 7feb93bf4640 1 -- 172.21.15.139:0/1071672099 wait complete. 2024-04-25T09:56:33.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.138+0000 7feb93bf4640 1 Processor -- start 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.138+0000 7feb93bf4640 1 -- start start 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb93bf4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb8c09ad40 0x7feb8c1e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb93bf4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c0cd610 0x7feb8c1e7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb93bf4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb8c0d0a80 0x7feb8c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb93bf4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c0cd9f0 con 0x7feb8c0d0a80 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb93bf4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c09a820 con 0x7feb8c0cd610 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb93bf4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c0cdb70 con 0x7feb8c09ad40 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb92191640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb8c0d0a80 0x7feb8c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb92191640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb8c0d0a80 0x7feb8c0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56080/0 (socket says 172.21.15.139:56080) 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb92191640 1 -- 172.21.15.139:0/1579455123 learned_addr learned my addr 172.21.15.139:0/1579455123 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb9118f640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c0cd610 0x7feb8c1e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb91990640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb8c09ad40 0x7feb8c1e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb92191640 1 -- 172.21.15.139:0/1579455123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb8c09ad40 msgr2=0x7feb8c1e8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb92191640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb8c09ad40 0x7feb8c1e8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb92191640 1 -- 172.21.15.139:0/1579455123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c0cd610 msgr2=0x7feb8c1e7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb92191640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c0cd610 0x7feb8c1e7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb92191640 1 -- 172.21.15.139:0/1579455123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8c161150 con 0x7feb8c0d0a80 2024-04-25T09:56:33.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb91990640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb8c09ad40 0x7feb8c1e8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:33.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb92191640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb8c0d0a80 0x7feb8c0b4320 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7feb840097a0 tx=0x7feb840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.139+0000 7feb827fc640 1 -- 172.21.15.139:0/1579455123 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84041070 con 0x7feb8c0d0a80 2024-04-25T09:56:33.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.140+0000 7feb827fc640 1 -- 172.21.15.139:0/1579455123 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8403c870 con 0x7feb8c0d0a80 2024-04-25T09:56:33.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.140+0000 7feb93bf4640 1 -- 172.21.15.139:0/1579455123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8c1e7b50 con 0x7feb8c0d0a80 2024-04-25T09:56:33.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.140+0000 7feb827fc640 1 -- 172.21.15.139:0/1579455123 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84037040 con 0x7feb8c0d0a80 2024-04-25T09:56:33.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.140+0000 7feb93bf4640 1 -- 172.21.15.139:0/1579455123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c1e8110 con 0x7feb8c0d0a80 2024-04-25T09:56:33.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.141+0000 7feb827fc640 1 -- 172.21.15.139:0/1579455123 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb84044070 con 0x7feb8c0d0a80 2024-04-25T09:56:33.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.142+0000 7feb827fc640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb7006b4d0 0x7feb7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.142+0000 7feb827fc640 1 -- 172.21.15.139:0/1579455123 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7feb840aafb0 con 0x7feb8c0d0a80 2024-04-25T09:56:33.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.142+0000 7feb91990640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb7006b4d0 0x7feb7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.142+0000 7feb91990640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb7006b4d0 0x7feb7006d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7feb880012e0 tx=0x7feb88009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.143+0000 7feb969cf740 1 Processor -- start 2024-04-25T09:56:33.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.143+0000 7feb969cf740 1 -- start start 2024-04-25T09:56:33.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.143+0000 7feb969cf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e4815b590 0x558e4815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.143+0000 7feb969cf740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e4815b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.143+0000 7feb969cf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e481864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.143+0000 7feb969cf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e480bec60 con 0x558e48186050 2024-04-25T09:56:33.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.143+0000 7feb969cf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e4805dd80 con 0x558e4815b590 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.143+0000 7feb969cf740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e480bad00 con 0x558e48197c60 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb92191640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e481864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb92191640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e481864d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56090/0 (socket says 172.21.15.139:56090) 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb92191640 1 -- 172.21.15.139:0/3523631894 learned_addr learned my addr 172.21.15.139:0/3523631894 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb91990640 1 --2- 172.21.15.139:0/3523631894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e4815b590 0x558e4815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb9118f640 1 --2- 172.21.15.139:0/3523631894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e4815b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb92191640 1 -- 172.21.15.139:0/3523631894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 msgr2=0x558e4815b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb92191640 1 --2- 172.21.15.139:0/3523631894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e4815b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb92191640 1 -- 172.21.15.139:0/3523631894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e4815b590 msgr2=0x558e4815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb92191640 1 --2- 172.21.15.139:0/3523631894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e4815b590 0x558e4815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb92191640 1 -- 172.21.15.139:0/3523631894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e47e9f7c0 con 0x558e48186050 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb9118f640 1 --2- 172.21.15.139:0/3523631894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e4815b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb92191640 1 --2- 172.21.15.139:0/3523631894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e481864d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7feb840a92f0 tx=0x7feb84079020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c0eadc23f3d313 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb6f7fe640 1 -- 172.21.15.139:0/3523631894 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb840796c0 con 0x558e48186050 2024-04-25T09:56:33.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb6f7fe640 1 -- 172.21.15.139:0/3523631894 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8407d810 con 0x558e48186050 2024-04-25T09:56:33.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.144+0000 7feb6f7fe640 1 -- 172.21.15.139:0/3523631894 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84081dd0 con 0x558e48186050 2024-04-25T09:56:33.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.145+0000 7feb969cf740 1 -- 172.21.15.139:0/3523631894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 msgr2=0x558e481864d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.145+0000 7feb969cf740 1 --2- 172.21.15.139:0/3523631894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e481864d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7feb840a92f0 tx=0x7feb84079020 comp rx=0 tx=0).stop 2024-04-25T09:56:33.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.145+0000 7feb969cf740 1 -- 172.21.15.139:0/3523631894 shutdown_connections 2024-04-25T09:56:33.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.145+0000 7feb969cf740 1 --2- 172.21.15.139:0/3523631894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e481864d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.145+0000 7feb969cf740 1 --2- 172.21.15.139:0/3523631894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e4815b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.145+0000 7feb969cf740 1 --2- 172.21.15.139:0/3523631894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e4815b590 0x558e4815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.145+0000 7feb969cf740 1 -- 172.21.15.139:0/3523631894 >> 172.21.15.139:0/3523631894 conn(0x558e481236e0 msgr2=0x558e481a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.145+0000 7feb969cf740 1 -- 172.21.15.139:0/3523631894 shutdown_connections 2024-04-25T09:56:33.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.145+0000 7feb969cf740 1 -- 172.21.15.139:0/3523631894 wait complete. 2024-04-25T09:56:33.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.145+0000 7feb969cf740 1 -- start start 2024-04-25T09:56:33.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.145+0000 7feb969cf740 1 Processor -- start 2024-04-25T09:56:33.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.146+0000 7feb969cf740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e4815b590 0x558e4819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.146+0000 7feb969cf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e48187300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.146+0000 7feb969cf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e48197c60 0x558e481b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.146+0000 7feb969cf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e480bec60 con 0x558e48186050 2024-04-25T09:56:33.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.146+0000 7feb969cf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e4805b840 con 0x558e48197c60 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.146+0000 7feb969cf740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e481b12e0 con 0x558e4815b590 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.146+0000 7feb969cf740 10 client.0 _ll_register_callbacks cb 0x558e48075980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.146+0000 7feb9118f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e48187300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.146+0000 7feb9118f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e48187300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56094/0 (socket says 172.21.15.139:56094) 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.146+0000 7feb9118f640 1 -- 172.21.15.139:0/2734450871 learned_addr learned my addr 172.21.15.139:0/2734450871 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb91990640 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e4815b590 0x558e4819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb92191640 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e48197c60 0x558e481b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb9118f640 1 -- 172.21.15.139:0/2734450871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e4815b590 msgr2=0x558e4819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb9118f640 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e4815b590 0x558e4819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb9118f640 1 -- 172.21.15.139:0/2734450871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e48197c60 msgr2=0x558e481b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb9118f640 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e48197c60 0x558e481b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb9118f640 1 -- 172.21.15.139:0/2734450871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e47efff50 con 0x558e48186050 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb9118f640 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e48187300 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009d30 tx=0x7feb7c0136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0d5cd1705ec9cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:33.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb6cff9640 1 -- 172.21.15.139:0/2734450871 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c01a030 con 0x558e48186050 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb6cff9640 1 -- 172.21.15.139:0/2734450871 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7feb7c016460 con 0x558e48186050 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb969cf740 1 -- 172.21.15.139:0/2734450871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb8407d810 con 0x558e48186050 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb6cff9640 1 -- 172.21.15.139:0/2734450871 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb7c005990 con 0x558e48186050 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb969cf740 10 client.6588 fetch_fsmap learned FSMap version 29 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb969cf740 1 -- 172.21.15.139:0/2734450871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x558e47d930b0 con 0x558e48186050 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb6cff9640 1 -- 172.21.15.139:0/2734450871 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8407d810 con 0x558e48186050 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb6cff9640 1 -- 172.21.15.139:0/2734450871 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feb7c029080 con 0x558e48186050 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb969cf740 10 client.6588 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb969cf740 10 client.6588 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb969cf740 20 client.6588 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.147+0000 7feb969cf740 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x558e481b21e0 0x558e481b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.148+0000 7feb969cf740 4 client.6588 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.148+0000 7feb969cf740 1 -- 172.21.15.139:0/2734450871 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e4807ee60 con 0x558e481b21e0 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.148+0000 7feb91990640 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x558e481b21e0 0x558e481b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.148+0000 7feb91990640 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x558e481b21e0 0x558e481b25c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d19e439e9476f55 server_cookie=abe70b8cebb53b47 in_seq=0 out_seq=0 2024-04-25T09:56:33.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.148+0000 7feb6cff9640 10 client.6588 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.148+0000 7feb6cff9640 1 -- 172.21.15.139:0/2734450871 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e4807ee60 con 0x558e481b21e0 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.148+0000 7feb6cff9640 10 client.6588 handle_command_reply: tid=0 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.148+0000 7feb969cf740 1 client.6588 shutdown 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.148+0000 7feb969cf740 1 -- 172.21.15.139:0/2734450871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 msgr2=0x558e48187300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.148+0000 7feb969cf740 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e48187300 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009d30 tx=0x7feb7c0136b0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.149+0000 7feb969cf740 1 -- 172.21.15.139:0/2734450871 shutdown_connections 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.149+0000 7feb969cf740 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x558e481b21e0 0x558e481b25c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.149+0000 7feb969cf740 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e48187300 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.149+0000 7feb969cf740 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e48197c60 0x558e481b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.149+0000 7feb969cf740 1 --2- 172.21.15.139:0/2734450871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e4815b590 0x558e4819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.149+0000 7feb6cff9640 0 client.6588 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.149+0000 7feb969cf740 1 -- 172.21.15.139:0/2734450871 >> 172.21.15.139:0/2734450871 conn(0x558e481236e0 msgr2=0x558e481a43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.149+0000 7feb969cf740 1 -- 172.21.15.139:0/2734450871 shutdown_connections 2024-04-25T09:56:33.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.149+0000 7feb969cf740 1 -- 172.21.15.139:0/2734450871 wait complete. 2024-04-25T09:56:33.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.149+0000 7feb969cf740 20 client.6588 trim_cache size 0 max 16384 2024-04-25T09:56:33.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.160+0000 7feb969cf740 1 Processor -- start 2024-04-25T09:56:33.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.160+0000 7feb969cf740 1 -- start start 2024-04-25T09:56:33.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb969cf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e4815b590 0x558e4819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb969cf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e48186050 0x558e481969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb969cf740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e48196f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb969cf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1ff610 con 0x558e4815b590 2024-04-25T09:56:33.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb969cf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1fea20 con 0x558e48186050 2024-04-25T09:56:33.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb969cf740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1ff020 con 0x558e48197c60 2024-04-25T09:56:33.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb91990640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e4815b590 0x558e4819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb91990640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e4815b590 0x558e4819e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56108/0 (socket says 172.21.15.139:56108) 2024-04-25T09:56:33.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb91990640 1 -- 172.21.15.139:0/1926794483 learned_addr learned my addr 172.21.15.139:0/1926794483 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb92191640 1 --2- 172.21.15.139:0/1926794483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e48196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb9118f640 1 --2- 172.21.15.139:0/1926794483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e48186050 0x558e481969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb91990640 1 -- 172.21.15.139:0/1926794483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 msgr2=0x558e48196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb91990640 1 --2- 172.21.15.139:0/1926794483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e48196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb91990640 1 -- 172.21.15.139:0/1926794483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e48186050 msgr2=0x558e481969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb91990640 1 --2- 172.21.15.139:0/1926794483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e48186050 0x558e481969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb91990640 1 -- 172.21.15.139:0/1926794483 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e48186d10 con 0x558e4815b590 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.161+0000 7feb92191640 1 --2- 172.21.15.139:0/1926794483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e48196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb91990640 1 --2- 172.21.15.139:0/1926794483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e4815b590 0x558e4819e5c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x558e48184430 tx=0x7feb88010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b653138b371c9d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb6effd640 1 -- 172.21.15.139:0/1926794483 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8802b070 con 0x558e4815b590 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb6effd640 1 -- 172.21.15.139:0/1926794483 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb88026150 con 0x558e4815b590 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb6effd640 1 -- 172.21.15.139:0/1926794483 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88026e60 con 0x558e4815b590 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb969cf740 1 -- 172.21.15.139:0/1926794483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e4815b590 msgr2=0x558e4819e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb969cf740 1 --2- 172.21.15.139:0/1926794483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e4815b590 0x558e4819e5c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x558e48184430 tx=0x7feb88010000 comp rx=0 tx=0).stop 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb969cf740 1 -- 172.21.15.139:0/1926794483 shutdown_connections 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb969cf740 1 --2- 172.21.15.139:0/1926794483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e48196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb969cf740 1 --2- 172.21.15.139:0/1926794483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e48186050 0x558e481969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb969cf740 1 --2- 172.21.15.139:0/1926794483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e4815b590 0x558e4819e5c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb969cf740 1 -- 172.21.15.139:0/1926794483 >> 172.21.15.139:0/1926794483 conn(0x558e481236e0 msgr2=0x558e4819dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb969cf740 1 -- 172.21.15.139:0/1926794483 shutdown_connections 2024-04-25T09:56:33.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.162+0000 7feb969cf740 1 -- 172.21.15.139:0/1926794483 wait complete. 2024-04-25T09:56:33.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.163+0000 7feb969cf740 1 -- start start 2024-04-25T09:56:33.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.163+0000 7feb969cf740 1 Processor -- start 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb969cf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e4815b590 0x558e4819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb969cf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e481846e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb969cf740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e48184c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb969cf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e48187840 con 0x558e48186050 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb969cf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e48188ab0 con 0x558e4815b590 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb969cf740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e481b4720 con 0x558e48197c60 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb969cf740 10 client.0 _ll_register_callbacks cb 0x7feb84009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb9118f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e481846e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb9118f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e481846e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56116/0 (socket says 172.21.15.139:56116) 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb9118f640 1 -- 172.21.15.139:0/709484743 learned_addr learned my addr 172.21.15.139:0/709484743 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb91990640 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e4815b590 0x558e4819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb92191640 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e48184c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb9118f640 1 -- 172.21.15.139:0/709484743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 msgr2=0x558e48184c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb9118f640 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e48184c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb9118f640 1 -- 172.21.15.139:0/709484743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e4815b590 msgr2=0x558e4819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb9118f640 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e4815b590 0x558e4819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb9118f640 1 -- 172.21.15.139:0/709484743 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e4819ac50 con 0x558e48186050 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb9118f640 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e481846e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009d30 tx=0x7feb7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d3a1ef290a6eff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb6cff9640 1 -- 172.21.15.139:0/709484743 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c01a070 con 0x558e48186050 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb6cff9640 1 -- 172.21.15.139:0/709484743 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7feb7c02da70 con 0x558e48186050 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.164+0000 7feb969cf740 1 -- 172.21.15.139:0/709484743 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb88026150 con 0x558e48186050 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb6cff9640 1 -- 172.21.15.139:0/709484743 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb7c038050 con 0x558e48186050 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb969cf740 10 client.6594 fetch_fsmap learned FSMap version 29 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb969cf740 1 -- 172.21.15.139:0/709484743 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x558e481b76f0 con 0x558e48186050 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb6cff9640 1 -- 172.21.15.139:0/709484743 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb88026150 con 0x558e48186050 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb6cff9640 1 -- 172.21.15.139:0/709484743 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feb7c02dc70 con 0x558e48186050 2024-04-25T09:56:33.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb969cf740 10 client.6594 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb969cf740 10 client.6594 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb969cf740 20 client.6594 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb969cf740 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x558e48187300 0x558e48196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb969cf740 4 client.6594 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb969cf740 1 -- 172.21.15.139:0/709484743 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x558e480cc1a0 con 0x558e48187300 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb91990640 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x558e48187300 0x558e48196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb6cff9640 10 client.6594 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb91990640 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x558e48187300 0x558e48196ae0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25fbbd22cc904ca server_cookie=8aef09a87b0c5e43 in_seq=0 out_seq=0 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.165+0000 7feb6cff9640 1 -- 172.21.15.139:0/709484743 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x558e480cc1a0 con 0x558e48187300 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb6cff9640 10 client.6594 handle_command_reply: tid=0 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb969cf740 1 client.6594 shutdown 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb969cf740 1 -- 172.21.15.139:0/709484743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 msgr2=0x558e481846e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb969cf740 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e481846e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009d30 tx=0x7feb7c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb969cf740 1 -- 172.21.15.139:0/709484743 shutdown_connections 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb969cf740 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x558e48187300 0x558e48196ae0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb969cf740 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558e48186050 0x558e481846e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb969cf740 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558e48197c60 0x558e48184c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb6cff9640 0 client.6594 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:33.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb969cf740 1 --2- 172.21.15.139:0/709484743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e4815b590 0x558e4819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb969cf740 1 -- 172.21.15.139:0/709484743 >> 172.21.15.139:0/709484743 conn(0x558e481236e0 msgr2=0x558e48189890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb969cf740 1 -- 172.21.15.139:0/709484743 shutdown_connections 2024-04-25T09:56:33.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb969cf740 1 -- 172.21.15.139:0/709484743 wait complete. 2024-04-25T09:56:33.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb969cf740 20 client.6594 trim_cache size 0 max 16384 2024-04-25T09:56:33.176 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:56:33.176 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-04-25T09:56:33.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:56:33.176 INFO:teuthology.orchestra.run.smithi139.stdout: "damage_type": "backtrace", 2024-04-25T09:56:33.176 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 4087375650, 2024-04-25T09:56:33.177 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 1099511627777, 2024-04-25T09:56:33.177 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/dir_test/file" 2024-04-25T09:56:33.177 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:56:33.177 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-04-25T09:56:33.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb93bf4640 1 -- 172.21.15.139:0/1579455123 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb7006b4d0 msgr2=0x7feb7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb93bf4640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb7006b4d0 0x7feb7006d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7feb880012e0 tx=0x7feb88009040 comp rx=0 tx=0).stop 2024-04-25T09:56:33.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb93bf4640 1 -- 172.21.15.139:0/1579455123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb8c0d0a80 msgr2=0x7feb8c0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb93bf4640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb8c0d0a80 0x7feb8c0b4320 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7feb840097a0 tx=0x7feb840046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb93bf4640 1 -- 172.21.15.139:0/1579455123 shutdown_connections 2024-04-25T09:56:33.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb93bf4640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb8c0d0a80 0x7feb8c0b4320 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb93bf4640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb7006b4d0 0x7feb7006d990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb93bf4640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb8c0cd610 0x7feb8c1e7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb93bf4640 1 --2- 172.21.15.139:0/1579455123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb8c09ad40 0x7feb8c1e8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.166+0000 7feb93bf4640 1 -- 172.21.15.139:0/1579455123 >> 172.21.15.139:0/1579455123 conn(0x7feb8c0b31f0 msgr2=0x7feb8c0b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.167+0000 7feb93bf4640 1 -- 172.21.15.139:0/1579455123 shutdown_connections 2024-04-25T09:56:33.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.167+0000 7feb93bf4640 1 -- 172.21.15.139:0/1579455123 wait complete. 2024-04-25T09:56:33.181 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T09:56:33.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.272+0000 7f41cbc3f640 1 Processor -- start 2024-04-25T09:56:33.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.273+0000 7f41cbc3f640 1 -- start start 2024-04-25T09:56:33.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.273+0000 7f41cbc3f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c415dc60 0x7f41c40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.273+0000 7f41cbc3f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f41c40d1080 0x7f41c40ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.273+0000 7f41cbc3f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c40cf250 con 0x7f41c4161850 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.273+0000 7f41cbc3f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41c40ce780 con 0x7f41c40d1080 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.273+0000 7f41cbc3f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c40d3730 con 0x7f41c415dc60 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.273+0000 7f41ca1dc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f41c40d1080 0x7f41c40ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.273+0000 7f41c91da640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f41c4161850 0x7f41c40d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:39606/0 (socket says 172.21.15.139:39606) 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.273+0000 7f41ca1dc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f41c40d1080 0x7f41c40ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56118/0 (socket says 172.21.15.139:56118) 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.273+0000 7f41c91da640 1 -- 172.21.15.139:0/627971161 learned_addr learned my addr 172.21.15.139:0/627971161 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.274+0000 7f41ca1dc640 1 -- 172.21.15.139:0/627971161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c415dc60 msgr2=0x7f41c40b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.274+0000 7f41c99db640 1 --2- 172.21.15.139:0/627971161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c415dc60 0x7f41c40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.274+0000 7f41c89d9640 1 -- 172.21.15.139:0/627971161 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660279216 0 0) 0x7f41b4001110 con 0x7f41c4161850 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.274+0000 7f41ca1dc640 1 --2- 172.21.15.139:0/627971161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c415dc60 0x7f41c40b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.274+0000 7f41ca1dc640 1 -- 172.21.15.139:0/627971161 >> v1:172.21.15.139:6789/0 conn(0x7f41c4161850 legacy=0x7f41c40d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.274+0000 7f41ca1dc640 1 -- 172.21.15.139:0/627971161 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c40ce9b0 con 0x7f41c40d1080 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.274+0000 7f41c99db640 1 --2- 172.21.15.139:0/627971161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c415dc60 0x7f41c40b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.274+0000 7f41ca1dc640 1 --2- 172.21.15.139:0/627971161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c40d1080 0x7f41c40ce240 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009d90 tx=0x7f41bc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c198867b03111ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.275+0000 7f41c89d9640 1 -- 172.21.15.139:0/627971161 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc007740 con 0x7f41c40d1080 2024-04-25T09:56:33.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.275+0000 7f41c89d9640 1 -- 172.21.15.139:0/627971161 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc0078e0 con 0x7f41c40d1080 2024-04-25T09:56:33.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.275+0000 7f41c89d9640 1 -- 172.21.15.139:0/627971161 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc007cb0 con 0x7f41c40d1080 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.275+0000 7f41cbc3f640 1 -- 172.21.15.139:0/627971161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c40d1080 msgr2=0x7f41c40ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.275+0000 7f41cbc3f640 1 --2- 172.21.15.139:0/627971161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c40d1080 0x7f41c40ce240 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009d90 tx=0x7f41bc02a660 comp rx=0 tx=0).stop 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.275+0000 7f41cbc3f640 1 -- 172.21.15.139:0/627971161 shutdown_connections 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.275+0000 7f41cbc3f640 1 --2- 172.21.15.139:0/627971161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c40d1080 0x7f41c40ce240 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.275+0000 7f41cbc3f640 1 --2- 172.21.15.139:0/627971161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c415dc60 0x7f41c40b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.275+0000 7f41cbc3f640 1 -- 172.21.15.139:0/627971161 >> 172.21.15.139:0/627971161 conn(0x7f41c40b5910 msgr2=0x7f41c40b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.275+0000 7f41cbc3f640 1 -- 172.21.15.139:0/627971161 shutdown_connections 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.275+0000 7f41cbc3f640 1 -- 172.21.15.139:0/627971161 wait complete. 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.276+0000 7f41cbc3f640 1 Processor -- start 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.276+0000 7f41cbc3f640 1 -- start start 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.277+0000 7f41cbc3f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c40d1080 0x7f41c415e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.277+0000 7f41cbc3f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c415dc60 0x7f41c4151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.277+0000 7f41cbc3f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c4161850 0x7f41c4159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.277+0000 7f41cbc3f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c40d1fc0 con 0x7f41c415dc60 2024-04-25T09:56:33.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.277+0000 7f41cbc3f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c40cef70 con 0x7f41c4161850 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.277+0000 7f41cbc3f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c415e040 con 0x7f41c40d1080 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.277+0000 7f41c91da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c415dc60 0x7f41c4151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.277+0000 7f41c91da640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c415dc60 0x7f41c4151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56124/0 (socket says 172.21.15.139:56124) 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.277+0000 7f41c91da640 1 -- 172.21.15.139:0/364595732 learned_addr learned my addr 172.21.15.139:0/364595732 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.277+0000 7f41ca1dc640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c4161850 0x7f41c4159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.278+0000 7f41c99db640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c40d1080 0x7f41c415e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.278+0000 7f41c91da640 1 -- 172.21.15.139:0/364595732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c40d1080 msgr2=0x7f41c415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.278+0000 7f41c91da640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c40d1080 0x7f41c415e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.278+0000 7f41c91da640 1 -- 172.21.15.139:0/364595732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c4161850 msgr2=0x7f41c4159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.278+0000 7f41c91da640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c4161850 0x7f41c4159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.278+0000 7f41c91da640 1 -- 172.21.15.139:0/364595732 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c415aa60 con 0x7f41c415dc60 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.278+0000 7f41c91da640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c415dc60 0x7f41c4151dd0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c420 tx=0x7f41b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.278+0000 7f41ba7fc640 1 -- 172.21.15.139:0/364595732 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b40074e0 con 0x7f41c415dc60 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.278+0000 7f41ba7fc640 1 -- 172.21.15.139:0/364595732 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b40047e0 con 0x7f41c415dc60 2024-04-25T09:56:33.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.278+0000 7f41cbc3f640 1 -- 172.21.15.139:0/364595732 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c415ad40 con 0x7f41c415dc60 2024-04-25T09:56:33.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.278+0000 7f41ba7fc640 1 -- 172.21.15.139:0/364595732 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b4004c10 con 0x7f41c415dc60 2024-04-25T09:56:33.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.278+0000 7f41cbc3f640 1 -- 172.21.15.139:0/364595732 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c415b300 con 0x7f41c415dc60 2024-04-25T09:56:33.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.279+0000 7f41cbc3f640 1 -- 172.21.15.139:0/364595732 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c4074520 con 0x7f41c415dc60 2024-04-25T09:56:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.279+0000 7f41ba7fc640 1 -- 172.21.15.139:0/364595732 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41b4011070 con 0x7f41c415dc60 2024-04-25T09:56:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.280+0000 7f41ba7fc640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f41a806b4d0 0x7f41a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.280+0000 7f41ba7fc640 1 -- 172.21.15.139:0/364595732 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f41b4089fc0 con 0x7f41c415dc60 2024-04-25T09:56:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.280+0000 7f41c99db640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f41a806b4d0 0x7f41a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.280+0000 7f41c99db640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f41a806b4d0 0x7f41a806d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f41c000a990 tx=0x7f41c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.284+0000 7f41ba7fc640 1 -- 172.21.15.139:0/364595732 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41c40cf250 con 0x7f41c415dc60 2024-04-25T09:56:33.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.492+0000 7f41cbc3f640 1 -- 172.21.15.139:0/364595732 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f41c40d16f0 con 0x7f41c415dc60 2024-04-25T09:56:33.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.492+0000 7f41ba7fc640 1 -- 172.21.15.139:0/364595732 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f41b4058c30 con 0x7f41c415dc60 2024-04-25T09:56:33.496 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:56:33.496 INFO:teuthology.orchestra.run.smithi139.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T09:56:33.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.494+0000 7f41cbc3f640 1 -- 172.21.15.139:0/364595732 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f41a806b4d0 msgr2=0x7f41a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.494+0000 7f41cbc3f640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f41a806b4d0 0x7f41a806d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f41c000a990 tx=0x7f41c0008040 comp rx=0 tx=0).stop 2024-04-25T09:56:33.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.494+0000 7f41cbc3f640 1 -- 172.21.15.139:0/364595732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c415dc60 msgr2=0x7f41c4151dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.494+0000 7f41cbc3f640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c415dc60 0x7f41c4151dd0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c420 tx=0x7f41b400c8e0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.494+0000 7f41cbc3f640 1 -- 172.21.15.139:0/364595732 shutdown_connections 2024-04-25T09:56:33.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.494+0000 7f41cbc3f640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41c415dc60 0x7f41c4151dd0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.494+0000 7f41cbc3f640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f41a806b4d0 0x7f41a806d990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.494+0000 7f41cbc3f640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c4161850 0x7f41c4159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.494+0000 7f41cbc3f640 1 --2- 172.21.15.139:0/364595732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c40d1080 0x7f41c415e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.494+0000 7f41cbc3f640 1 -- 172.21.15.139:0/364595732 >> 172.21.15.139:0/364595732 conn(0x7f41c40b5910 msgr2=0x7f41c40b7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.494+0000 7f41cbc3f640 1 -- 172.21.15.139:0/364595732 shutdown_connections 2024-04-25T09:56:33.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.494+0000 7f41cbc3f640 1 -- 172.21.15.139:0/364595732 wait complete. 2024-04-25T09:56:33.508 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T09:56:33.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:33.509 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-25T09:56:33.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.605+0000 7f5fae77e640 1 Processor -- start 2024-04-25T09:56:33.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.605+0000 7f5fae77e640 1 -- start start 2024-04-25T09:56:33.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.606+0000 7f5fae77e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5fa80cd610 0x7f5fa80d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.606+0000 7f5fae77e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fa809ad40 0x7f5fa80d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.606+0000 7f5fae77e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa8099ca0 con 0x7f5fa81618c0 2024-04-25T09:56:33.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.606+0000 7f5fae77e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa8099a50 con 0x7f5fa80cd610 2024-04-25T09:56:33.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.606+0000 7f5fae77e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa8099f80 con 0x7f5fa809ad40 2024-04-25T09:56:33.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.606+0000 7f5fa77fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fa809ad40 0x7f5fa80d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.606+0000 7f5fa7fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5fa80cd610 0x7f5fa80d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.606+0000 7f5fa77fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fa809ad40 0x7f5fa80d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:46472/0 (socket says 172.21.15.139:46472) 2024-04-25T09:56:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.606+0000 7f5fa77fe640 1 -- 172.21.15.139:0/3191878312 learned_addr learned my addr 172.21.15.139:0/3191878312 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.607+0000 7f5fa77fe640 1 -- 172.21.15.139:0/3191878312 >> v2:172.21.15.150:3300/0 conn(0x7f5fa80cd610 msgr2=0x7f5fa80d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.607+0000 7f5fa77fe640 1 --2- 172.21.15.139:0/3191878312 >> v2:172.21.15.150:3300/0 conn(0x7f5fa80cd610 0x7f5fa80d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.607+0000 7f5fa77fe640 1 -- 172.21.15.139:0/3191878312 >> v1:172.21.15.150:6789/0 conn(0x7f5fa81618c0 legacy=0x7f5fa806be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.607+0000 7f5fa77fe640 1 -- 172.21.15.139:0/3191878312 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa809a260 con 0x7f5fa809ad40 2024-04-25T09:56:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.607+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/3191878312 >> v2:172.21.15.150:3300/0 conn(0x7f5fa80cd610 0x7f5fa80d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:33.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.608+0000 7f5fa77fe640 1 --2- 172.21.15.139:0/3191878312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fa809ad40 0x7f5fa80d08b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009870 tx=0x7f5f9c02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c47f6a86006db0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.608+0000 7f5fa6ffd640 1 -- 172.21.15.139:0/3191878312 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c037070 con 0x7f5fa809ad40 2024-04-25T09:56:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.608+0000 7f5fa6ffd640 1 -- 172.21.15.139:0/3191878312 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c02ae50 con 0x7f5fa809ad40 2024-04-25T09:56:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.608+0000 7f5fa6ffd640 1 -- 172.21.15.139:0/3191878312 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c032860 con 0x7f5fa809ad40 2024-04-25T09:56:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.608+0000 7f5fae77e640 1 -- 172.21.15.139:0/3191878312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fa809ad40 msgr2=0x7f5fa80d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.608+0000 7f5fae77e640 1 --2- 172.21.15.139:0/3191878312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fa809ad40 0x7f5fa80d08b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009870 tx=0x7f5f9c02a300 comp rx=0 tx=0).stop 2024-04-25T09:56:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.608+0000 7f5fae77e640 1 -- 172.21.15.139:0/3191878312 shutdown_connections 2024-04-25T09:56:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.608+0000 7f5fae77e640 1 --2- 172.21.15.139:0/3191878312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fa809ad40 0x7f5fa80d08b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.608+0000 7f5fae77e640 1 --2- 172.21.15.139:0/3191878312 >> v2:172.21.15.150:3300/0 conn(0x7f5fa80cd610 0x7f5fa80d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.608+0000 7f5fae77e640 1 -- 172.21.15.139:0/3191878312 >> 172.21.15.139:0/3191878312 conn(0x7f5fa80b31f0 msgr2=0x7f5fa80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.608+0000 7f5fae77e640 1 -- 172.21.15.139:0/3191878312 shutdown_connections 2024-04-25T09:56:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.608+0000 7f5fae77e640 1 -- 172.21.15.139:0/3191878312 wait complete. 2024-04-25T09:56:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.609+0000 7f5fae77e640 1 Processor -- start 2024-04-25T09:56:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.610+0000 7f5fae77e640 1 -- start start 2024-04-25T09:56:33.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.610+0000 7f5fae77e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fa809ad40 0x7f5fa814aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.610+0000 7f5fae77e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fa80cd610 0x7f5fa81574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.610+0000 7f5fae77e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fa81618c0 0x7f5fa8157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.610+0000 7f5fae77e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80cd9f0 con 0x7f5fa81618c0 2024-04-25T09:56:33.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.610+0000 7f5fae77e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa809a820 con 0x7f5fa80cd610 2024-04-25T09:56:33.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.610+0000 7f5fae77e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80cdb70 con 0x7f5fa809ad40 2024-04-25T09:56:33.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.610+0000 7f5facd1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fa81618c0 0x7f5fa8157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.610+0000 7f5facd1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fa81618c0 0x7f5fa8157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56132/0 (socket says 172.21.15.139:56132) 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5facd1b640 1 -- 172.21.15.139:0/1744759308 learned_addr learned my addr 172.21.15.139:0/1744759308 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fa809ad40 0x7f5fa814aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5fa77fe640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fa80cd610 0x7f5fa81574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5facd1b640 1 -- 172.21.15.139:0/1744759308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fa809ad40 msgr2=0x7f5fa814aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5facd1b640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fa809ad40 0x7f5fa814aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5facd1b640 1 -- 172.21.15.139:0/1744759308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fa80cd610 msgr2=0x7f5fa81574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5facd1b640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fa80cd610 0x7f5fa81574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5facd1b640 1 -- 172.21.15.139:0/1744759308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa815f5e0 con 0x7f5fa81618c0 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5facd1b640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fa81618c0 0x7f5fa8157a00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c6b0 tx=0x7f5fa000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5fa4ff9640 1 -- 172.21.15.139:0/1744759308 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0003b70 con 0x7f5fa81618c0 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5fa4ff9640 1 -- 172.21.15.139:0/1744759308 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa0003d30 con 0x7f5fa81618c0 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5fa4ff9640 1 -- 172.21.15.139:0/1744759308 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0005310 con 0x7f5fa81618c0 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5fae77e640 1 -- 172.21.15.139:0/1744759308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa815f8c0 con 0x7f5fa81618c0 2024-04-25T09:56:33.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.611+0000 7f5fae77e640 1 -- 172.21.15.139:0/1744759308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa80c60f0 con 0x7f5fa81618c0 2024-04-25T09:56:33.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.613+0000 7f5fa4ff9640 1 -- 172.21.15.139:0/1744759308 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fa00054b0 con 0x7f5fa81618c0 2024-04-25T09:56:33.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.613+0000 7f5fa4ff9640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5f9406b4d0 0x7f5f9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.613+0000 7f5fa4ff9640 1 -- 172.21.15.139:0/1744759308 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f5fa008a140 con 0x7f5fa81618c0 2024-04-25T09:56:33.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.613+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5f9406b4d0 0x7f5f9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.614+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5f9406b4d0 0x7f5f9406d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009560 tx=0x7f5f98009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.614+0000 7f5fb1122740 1 Processor -- start 2024-04-25T09:56:33.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.614+0000 7f5fb1122740 1 -- start start 2024-04-25T09:56:33.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5fb1122740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2710 0x55919f2c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5fb1122740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2af0 0x55919f2c5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5fb1122740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f295370 0x55919f298b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5fb1122740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919f1fdbc0 con 0x55919f295370 2024-04-25T09:56:33.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5fb1122740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919f1a6e70 con 0x55919f2c2af0 2024-04-25T09:56:33.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5fb1122740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55919f1f9c60 con 0x55919f2c2710 2024-04-25T09:56:33.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5facd1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f295370 0x55919f298b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5facd1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f295370 0x55919f298b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56136/0 (socket says 172.21.15.139:56136) 2024-04-25T09:56:33.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5facd1b640 1 -- 172.21.15.139:0/2312713979 learned_addr learned my addr 172.21.15.139:0/2312713979 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/2312713979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2710 0x55919f2c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5fa77fe640 1 --2- 172.21.15.139:0/2312713979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2af0 0x55919f2c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5facd1b640 1 -- 172.21.15.139:0/2312713979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2710 msgr2=0x55919f2c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5facd1b640 1 --2- 172.21.15.139:0/2312713979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2710 0x55919f2c4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5facd1b640 1 -- 172.21.15.139:0/2312713979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2af0 msgr2=0x55919f2c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5facd1b640 1 --2- 172.21.15.139:0/2312713979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2af0 0x55919f2c5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5facd1b640 1 -- 172.21.15.139:0/2312713979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55919f1c7ad0 con 0x55919f295370 2024-04-25T09:56:33.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/2312713979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2710 0x55919f2c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.615+0000 7f5facd1b640 1 --2- 172.21.15.139:0/2312713979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f295370 0x55919f298b80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00883f0 tx=0x7f5fa0058810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7123c684032bf47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.616+0000 7f5f8dffb640 1 -- 172.21.15.139:0/2312713979 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00057a0 con 0x55919f295370 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.616+0000 7f5f8dffb640 1 -- 172.21.15.139:0/2312713979 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa00567f0 con 0x55919f295370 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.616+0000 7f5f8dffb640 1 -- 172.21.15.139:0/2312713979 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0054dc0 con 0x55919f295370 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.616+0000 7f5fb1122740 1 -- 172.21.15.139:0/2312713979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f295370 msgr2=0x55919f298b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.616+0000 7f5fb1122740 1 --2- 172.21.15.139:0/2312713979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f295370 0x55919f298b80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00883f0 tx=0x7f5fa0058810 comp rx=0 tx=0).stop 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.616+0000 7f5fb1122740 1 -- 172.21.15.139:0/2312713979 shutdown_connections 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.616+0000 7f5fb1122740 1 --2- 172.21.15.139:0/2312713979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f295370 0x55919f298b80 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.616+0000 7f5fb1122740 1 --2- 172.21.15.139:0/2312713979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2af0 0x55919f2c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.616+0000 7f5fb1122740 1 --2- 172.21.15.139:0/2312713979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2710 0x55919f2c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.616+0000 7f5fb1122740 1 -- 172.21.15.139:0/2312713979 >> 172.21.15.139:0/2312713979 conn(0x55919f2975e0 msgr2=0x55919f2dff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.616+0000 7f5fb1122740 1 -- 172.21.15.139:0/2312713979 shutdown_connections 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.616+0000 7f5fb1122740 1 -- 172.21.15.139:0/2312713979 wait complete. 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.617+0000 7f5fb1122740 1 -- start start 2024-04-25T09:56:33.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.617+0000 7f5fb1122740 1 Processor -- start 2024-04-25T09:56:33.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5fb1122740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2710 0x55919f2c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5fb1122740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2af0 0x55919f2c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5fb1122740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 0x55919f2bb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5fb1122740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919f29a790 con 0x55919f2bc3d0 2024-04-25T09:56:33.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5fb1122740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919f19c4e0 con 0x55919f2c2710 2024-04-25T09:56:33.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5fb1122740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55919f29aa70 con 0x55919f2c2af0 2024-04-25T09:56:33.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5fb1122740 10 client.0 _ll_register_callbacks cb 0x55919f1b9de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:33.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5facd1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 0x55919f2bb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5facd1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 0x55919f2bb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56142/0 (socket says 172.21.15.139:56142) 2024-04-25T09:56:33.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5facd1b640 1 -- 172.21.15.139:0/686619503 learned_addr learned my addr 172.21.15.139:0/686619503 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5fa77fe640 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2af0 0x55919f2c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2710 0x55919f2c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5facd1b640 1 -- 172.21.15.139:0/686619503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2af0 msgr2=0x55919f2c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5facd1b640 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2af0 0x55919f2c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5facd1b640 1 -- 172.21.15.139:0/686619503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2710 msgr2=0x55919f2c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5facd1b640 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2710 0x55919f2c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5facd1b640 1 -- 172.21.15.139:0/686619503 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55919efde940 con 0x55919f2bc3d0 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5facd1b640 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 0x55919f2bb330 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c680 tx=0x7f5fa0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d50dc9a3a6db52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5f86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5f86ffd640 1 -- 172.21.15.139:0/686619503 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0057ba0 con 0x55919f2bc3d0 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5f86ffd640 1 -- 172.21.15.139:0/686619503 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f5fa0057d40 con 0x55919f2bc3d0 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5fb1122740 1 -- 172.21.15.139:0/686619503 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fa00567f0 con 0x55919f2bc3d0 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5f86ffd640 1 -- 172.21.15.139:0/686619503 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fa00544a0 con 0x55919f2bc3d0 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5fb1122740 10 client.6612 fetch_fsmap learned FSMap version 29 2024-04-25T09:56:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.618+0000 7f5fb1122740 1 -- 172.21.15.139:0/686619503 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55919f03f0d0 con 0x55919f2bc3d0 2024-04-25T09:56:33.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.619+0000 7f5f86ffd640 1 -- 172.21.15.139:0/686619503 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa00567f0 con 0x55919f2bc3d0 2024-04-25T09:56:33.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.619+0000 7f5f86ffd640 1 -- 172.21.15.139:0/686619503 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5fa0054640 con 0x55919f2bc3d0 2024-04-25T09:56:33.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.619+0000 7f5fb1122740 10 client.6612 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T09:56:33.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.619+0000 7f5fb1122740 10 client.6612 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:56:33.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.619+0000 7f5fb1122740 20 client.6612 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:33.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.619+0000 7f5fb1122740 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55919f2b9290 0x55919f2bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.619+0000 7f5fb1122740 4 client.6612 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:33.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.619+0000 7f5fb1122740 1 -- 172.21.15.139:0/686619503 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55919f20b100 con 0x55919f2b9290 2024-04-25T09:56:33.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.619+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55919f2b9290 0x55919f2bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.619+0000 7f5f86ffd640 10 client.6612 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.619+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55919f2b9290 0x55919f2bb870 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83d38493582fdfc server_cookie=b20d6764d3cb5606 in_seq=0 out_seq=0 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5f86ffd640 1 -- 172.21.15.139:0/686619503 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55919f20b100 con 0x55919f2b9290 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5f86ffd640 10 client.6612 handle_command_reply: tid=0 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5fb1122740 1 client.6612 shutdown 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5fb1122740 1 -- 172.21.15.139:0/686619503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 msgr2=0x55919f2bb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5fb1122740 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 0x55919f2bb330 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c680 tx=0x7f5fa0001250 comp rx=0 tx=0).stop 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5fb1122740 1 -- 172.21.15.139:0/686619503 shutdown_connections 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5fb1122740 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55919f2b9290 0x55919f2bb870 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5fb1122740 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 0x55919f2bb330 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5fb1122740 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2af0 0x55919f2c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5f86ffd640 0 client.6612 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5fb1122740 1 --2- 172.21.15.139:0/686619503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2710 0x55919f2c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5fb1122740 1 -- 172.21.15.139:0/686619503 >> 172.21.15.139:0/686619503 conn(0x55919f295370 msgr2=0x55919f2c1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5fb1122740 1 -- 172.21.15.139:0/686619503 shutdown_connections 2024-04-25T09:56:33.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5fb1122740 1 -- 172.21.15.139:0/686619503 wait complete. 2024-04-25T09:56:33.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.620+0000 7f5fb1122740 20 client.6612 trim_cache size 0 max 16384 2024-04-25T09:56:33.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.635+0000 7f5fb1122740 1 Processor -- start 2024-04-25T09:56:33.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.635+0000 7f5fb1122740 1 -- start start 2024-04-25T09:56:33.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5fb1122740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2bc3d0 0x55919f2c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5fb1122740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2710 0x55919f2c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5fb1122740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2c2af0 0x55919f2c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5fb1122740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa82006d0 con 0x55919f2c2af0 2024-04-25T09:56:33.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5fb1122740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81ff840 con 0x55919f2c2710 2024-04-25T09:56:33.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5fb1122740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80c7af0 con 0x55919f2bc3d0 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5facd1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2c2af0 0x55919f2c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5facd1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2c2af0 0x55919f2c0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56148/0 (socket says 172.21.15.139:56148) 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5facd1b640 1 -- 172.21.15.139:0/3192289051 learned_addr learned my addr 172.21.15.139:0/3192289051 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5fa77fe640 1 --2- 172.21.15.139:0/3192289051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2710 0x55919f2c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/3192289051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2bc3d0 0x55919f2c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5facd1b640 1 -- 172.21.15.139:0/3192289051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2bc3d0 msgr2=0x55919f2c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5facd1b640 1 --2- 172.21.15.139:0/3192289051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2bc3d0 0x55919f2c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5facd1b640 1 -- 172.21.15.139:0/3192289051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2710 msgr2=0x55919f2c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5facd1b640 1 --2- 172.21.15.139:0/3192289051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2710 0x55919f2c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5facd1b640 1 -- 172.21.15.139:0/3192289051 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55919eed2230 con 0x55919f2c2af0 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/3192289051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2bc3d0 0x55919f2c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5facd1b640 1 --2- 172.21.15.139:0/3192289051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2c2af0 0x55919f2c0d40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00703a0 tx=0x7f5fa0085000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb068ff06bdf35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5f8d7fa640 1 -- 172.21.15.139:0/3192289051 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0054dc0 con 0x55919f2c2af0 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5f8d7fa640 1 -- 172.21.15.139:0/3192289051 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa0054640 con 0x55919f2c2af0 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.636+0000 7f5f8d7fa640 1 -- 172.21.15.139:0/3192289051 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00547f0 con 0x55919f2c2af0 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.637+0000 7f5fb1122740 1 -- 172.21.15.139:0/3192289051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2c2af0 msgr2=0x55919f2c0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.637+0000 7f5fb1122740 1 --2- 172.21.15.139:0/3192289051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2c2af0 0x55919f2c0d40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00703a0 tx=0x7f5fa0085000 comp rx=0 tx=0).stop 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.637+0000 7f5fb1122740 1 -- 172.21.15.139:0/3192289051 shutdown_connections 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.637+0000 7f5fb1122740 1 --2- 172.21.15.139:0/3192289051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2c2af0 0x55919f2c0d40 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.637+0000 7f5fb1122740 1 --2- 172.21.15.139:0/3192289051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2710 0x55919f2c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.637+0000 7f5fb1122740 1 --2- 172.21.15.139:0/3192289051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2bc3d0 0x55919f2c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.637+0000 7f5fb1122740 1 -- 172.21.15.139:0/3192289051 >> 172.21.15.139:0/3192289051 conn(0x55919f295370 msgr2=0x55919f2dcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.637+0000 7f5fb1122740 1 -- 172.21.15.139:0/3192289051 shutdown_connections 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.637+0000 7f5fb1122740 1 -- 172.21.15.139:0/3192289051 wait complete. 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.637+0000 7f5fb1122740 1 -- start start 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.638+0000 7f5fb1122740 1 Processor -- start 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.638+0000 7f5fb1122740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 0x55919f34a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.638+0000 7f5fb1122740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2710 0x55919f2f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.638+0000 7f5fb1122740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2af0 0x55919f2c0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.638+0000 7f5fb1122740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919f2f0880 con 0x55919f2bc3d0 2024-04-25T09:56:33.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.638+0000 7f5fb1122740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81ff2d0 con 0x55919f2c2af0 2024-04-25T09:56:33.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.638+0000 7f5fb1122740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55919f2bab30 con 0x55919f2c2710 2024-04-25T09:56:33.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.638+0000 7f5fb1122740 10 client.0 _ll_register_callbacks cb 0x7f5fa00580b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:33.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fa7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 0x55919f34a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fa7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 0x55919f34a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56158/0 (socket says 172.21.15.139:56158) 2024-04-25T09:56:33.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fa7fff640 1 -- 172.21.15.139:0/2978344911 learned_addr learned my addr 172.21.15.139:0/2978344911 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fa77fe640 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2710 0x55919f2f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5facd1b640 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2af0 0x55919f2c0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fa7fff640 1 -- 172.21.15.139:0/2978344911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2710 msgr2=0x55919f2f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2710 0x55919f2f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fa7fff640 1 -- 172.21.15.139:0/2978344911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2af0 msgr2=0x55919f2c0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2af0 0x55919f2c0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fa7fff640 1 -- 172.21.15.139:0/2978344911 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55919f2e2e20 con 0x55919f2bc3d0 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fa77fe640 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2710 0x55919f2f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5facd1b640 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2af0 0x55919f2c0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fa7fff640 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 0x55919f34a3c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x55919f2c6410 tx=0x7f5f98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9772865c61de9321 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5f86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5f86ffd640 1 -- 172.21.15.139:0/2978344911 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9801e040 con 0x55919f2bc3d0 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5f86ffd640 1 -- 172.21.15.139:0/2978344911 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f5f9800f040 con 0x55919f2bc3d0 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fb1122740 1 -- 172.21.15.139:0/2978344911 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fa0054640 con 0x55919f2bc3d0 2024-04-25T09:56:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5f86ffd640 1 -- 172.21.15.139:0/2978344911 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f980044b0 con 0x55919f2bc3d0 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fb1122740 10 client.6618 fetch_fsmap learned FSMap version 29 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fb1122740 1 -- 172.21.15.139:0/2978344911 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55919f2f3100 con 0x55919f2bc3d0 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5f86ffd640 1 -- 172.21.15.139:0/2978344911 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa0054640 con 0x55919f2bc3d0 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5f86ffd640 1 -- 172.21.15.139:0/2978344911 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5f98038050 con 0x55919f2bc3d0 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fb1122740 10 client.6618 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fb1122740 10 client.6618 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fb1122740 20 client.6618 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fb1122740 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55919f294260 0x55919f2c1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fb1122740 4 client.6618 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.639+0000 7f5fb1122740 1 -- 172.21.15.139:0/2978344911 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) v1 -- 0x55919f2baea0 con 0x55919f294260 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.640+0000 7f5fa77fe640 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55919f294260 0x55919f2c1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.640+0000 7f5f86ffd640 10 client.6618 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.640+0000 7f5fa77fe640 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55919f294260 0x55919f2c1250 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bc7a2a08fee7839 server_cookie=16fc12a1a269744 in_seq=0 out_seq=0 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.640+0000 7f5f86ffd640 1 -- 172.21.15.139:0/2978344911 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55919f2baea0 con 0x55919f294260 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.640+0000 7f5f86ffd640 10 client.6618 handle_command_reply: tid=0 2024-04-25T09:56:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.640+0000 7f5fb1122740 1 client.6618 shutdown 2024-04-25T09:56:33.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f5fb1122740 1 -- 172.21.15.139:0/2978344911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 msgr2=0x55919f34a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f5fb1122740 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 0x55919f34a3c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x55919f2c6410 tx=0x7f5f98004730 comp rx=0 tx=0).stop 2024-04-25T09:56:33.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f5fb1122740 1 -- 172.21.15.139:0/2978344911 shutdown_connections 2024-04-25T09:56:33.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f5fb1122740 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55919f294260 0x55919f2c1250 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f5fb1122740 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55919f2c2af0 0x55919f2c0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f5fb1122740 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55919f2c2710 0x55919f2f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f5fb1122740 1 --2- 172.21.15.139:0/2978344911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55919f2bc3d0 0x55919f34a3c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f5fb1122740 1 -- 172.21.15.139:0/2978344911 >> 172.21.15.139:0/2978344911 conn(0x55919f295370 msgr2=0x55919f299e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f5f86ffd640 0 client.6618 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:33.657 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f41cfe33640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5474=0-4194304@1} caps={5474=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5474 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x564683923b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T09:56:33.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f5fb1122740 1 -- 172.21.15.139:0/2978344911 shutdown_connections 2024-04-25T09:56:33.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f5fb1122740 1 -- 172.21.15.139:0/2978344911 wait complete. 2024-04-25T09:56:33.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.641+0000 7f5fb1122740 20 client.6618 trim_cache size 0 max 16384 2024-04-25T09:56:33.657 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:56:33.658 INFO:teuthology.orchestra.run.smithi139.stdout: "return_code": 0, 2024-04-25T09:56:33.658 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_tag": "16befb4f-9a03-497d-bec3-690bcbf08e33", 2024-04-25T09:56:33.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": "asynchronous" 2024-04-25T09:56:33.658 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:56:33.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.642+0000 7f5fae77e640 1 -- 172.21.15.139:0/1744759308 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5f9406b4d0 msgr2=0x7f5f9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.642+0000 7f5fae77e640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5f9406b4d0 0x7f5f9406d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009560 tx=0x7f5f98009190 comp rx=0 tx=0).stop 2024-04-25T09:56:33.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.642+0000 7f5fae77e640 1 -- 172.21.15.139:0/1744759308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fa81618c0 msgr2=0x7f5fa8157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.642+0000 7f5fae77e640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fa81618c0 0x7f5fa8157a00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c6b0 tx=0x7f5fa000cb70 comp rx=0 tx=0).stop 2024-04-25T09:56:33.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.642+0000 7f5fae77e640 1 -- 172.21.15.139:0/1744759308 shutdown_connections 2024-04-25T09:56:33.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.642+0000 7f5fae77e640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fa81618c0 0x7f5fa8157a00 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.642+0000 7f5fae77e640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5f9406b4d0 0x7f5f9406d990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.642+0000 7f5fae77e640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fa80cd610 0x7f5fa81574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.642+0000 7f5fae77e640 1 --2- 172.21.15.139:0/1744759308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fa809ad40 0x7f5fa814aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.642+0000 7f5fae77e640 1 -- 172.21.15.139:0/1744759308 >> 172.21.15.139:0/1744759308 conn(0x7f5fa80b31f0 msgr2=0x7f5fa80b5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.642+0000 7f5fae77e640 1 -- 172.21.15.139:0/1744759308 shutdown_connections 2024-04-25T09:56:33.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.642+0000 7f5fae77e640 1 -- 172.21.15.139:0/1744759308 wait complete. 2024-04-25T09:56:33.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:33.660 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T09:56:33.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.755+0000 7f34b9fe0640 1 Processor -- start 2024-04-25T09:56:33.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.755+0000 7f34b9fe0640 1 -- start start 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.756+0000 7f34b9fe0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34b40cd610 0x7f34b40d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.756+0000 7f34b9fe0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34b409ad40 0x7f34b40d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.756+0000 7f34b9fe0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b4099ca0 con 0x7f34b41618c0 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.756+0000 7f34b9fe0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b4099a50 con 0x7f34b409ad40 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.756+0000 7f34b9fe0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b4099f80 con 0x7f34b40cd610 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.756+0000 7f34b2ffd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34b409ad40 0x7f34b40d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.757+0000 7f34b37fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34b40cd610 0x7f34b40d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.757+0000 7f34b3fff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f34b41618c0 0x7f34b406be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:44594/0 (socket says 172.21.15.139:44594) 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.757+0000 7f34b3fff640 1 -- 172.21.15.139:0/2565378856 learned_addr learned my addr 172.21.15.139:0/2565378856 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.757+0000 7f34b2ffd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34b409ad40 0x7f34b40d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:43602/0 (socket says 172.21.15.139:43602) 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.757+0000 7f34b2ffd640 1 -- 172.21.15.139:0/2565378856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34b40cd610 msgr2=0x7f34b40d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.757+0000 7f34b2ffd640 1 --2- 172.21.15.139:0/2565378856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34b40cd610 0x7f34b40d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.757+0000 7f34b2ffd640 1 -- 172.21.15.139:0/2565378856 >> v1:172.21.15.150:6789/0 conn(0x7f34b41618c0 legacy=0x7f34b406be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.757+0000 7f34b2ffd640 1 -- 172.21.15.139:0/2565378856 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b409a260 con 0x7f34b409ad40 2024-04-25T09:56:33.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.757+0000 7f34b27fc640 1 -- 172.21.15.139:0/2565378856 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582914614 0 0) 0x7f34b4099ca0 con 0x7f34b41618c0 2024-04-25T09:56:33.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.758+0000 7f34b2ffd640 1 --2- 172.21.15.139:0/2565378856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b409ad40 0x7f34b40d08b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f34a80097a0 tx=0x7f34a802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87dd1028652dbc1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.758+0000 7f34b27fc640 1 -- 172.21.15.139:0/2565378856 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8037070 con 0x7f34b409ad40 2024-04-25T09:56:33.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.758+0000 7f34b27fc640 1 -- 172.21.15.139:0/2565378856 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a8032440 con 0x7f34b409ad40 2024-04-25T09:56:33.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.758+0000 7f34b27fc640 1 -- 172.21.15.139:0/2565378856 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8032810 con 0x7f34b409ad40 2024-04-25T09:56:33.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.758+0000 7f34b9fe0640 1 -- 172.21.15.139:0/2565378856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b409ad40 msgr2=0x7f34b40d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.758+0000 7f34b9fe0640 1 --2- 172.21.15.139:0/2565378856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b409ad40 0x7f34b40d08b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f34a80097a0 tx=0x7f34a802a210 comp rx=0 tx=0).stop 2024-04-25T09:56:33.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.758+0000 7f34b9fe0640 1 -- 172.21.15.139:0/2565378856 shutdown_connections 2024-04-25T09:56:33.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.758+0000 7f34b9fe0640 1 --2- 172.21.15.139:0/2565378856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b409ad40 0x7f34b40d08b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.758+0000 7f34b9fe0640 1 --2- 172.21.15.139:0/2565378856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34b40cd610 0x7f34b40d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.758+0000 7f34b9fe0640 1 -- 172.21.15.139:0/2565378856 >> 172.21.15.139:0/2565378856 conn(0x7f34b40b31f0 msgr2=0x7f34b40b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.758+0000 7f34b9fe0640 1 -- 172.21.15.139:0/2565378856 shutdown_connections 2024-04-25T09:56:33.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.758+0000 7f34b9fe0640 1 -- 172.21.15.139:0/2565378856 wait complete. 2024-04-25T09:56:33.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.760+0000 7f34b9fe0640 1 Processor -- start 2024-04-25T09:56:33.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.760+0000 7f34b9fe0640 1 -- start start 2024-04-25T09:56:33.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.760+0000 7f34b9fe0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b409ad40 0x7f34b40c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.760+0000 7f34b9fe0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34b40cd610 0x7f34b41f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.760+0000 7f34b9fe0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34b41618c0 0x7f34b41f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.760+0000 7f34b9fe0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40cd9f0 con 0x7f34b41618c0 2024-04-25T09:56:33.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.760+0000 7f34b9fe0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b409a820 con 0x7f34b409ad40 2024-04-25T09:56:33.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.760+0000 7f34b9fe0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40cdb70 con 0x7f34b40cd610 2024-04-25T09:56:33.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34b41618c0 0x7f34b41f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34b41618c0 0x7f34b41f31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56170/0 (socket says 172.21.15.139:56170) 2024-04-25T09:56:33.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b3fff640 1 -- 172.21.15.139:0/3760176214 learned_addr learned my addr 172.21.15.139:0/3760176214 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b37fe640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b409ad40 0x7f34b40c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b2ffd640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34b40cd610 0x7f34b41f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b3fff640 1 -- 172.21.15.139:0/3760176214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34b40cd610 msgr2=0x7f34b41f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b3fff640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34b40cd610 0x7f34b41f4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b3fff640 1 -- 172.21.15.139:0/3760176214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b409ad40 msgr2=0x7f34b40c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b3fff640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b409ad40 0x7f34b40c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b3fff640 1 -- 172.21.15.139:0/3760176214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b4158ed0 con 0x7f34b41618c0 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b3fff640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34b41618c0 0x7f34b41f31d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00c730 tx=0x7f34ac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f349bfff640 1 -- 172.21.15.139:0/3760176214 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac018070 con 0x7f34b41618c0 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f349bfff640 1 -- 172.21.15.139:0/3760176214 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ac00ce40 con 0x7f34b41618c0 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b9fe0640 1 -- 172.21.15.139:0/3760176214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b41591b0 con 0x7f34b41618c0 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f349bfff640 1 -- 172.21.15.139:0/3760176214 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00a860 con 0x7f34b41618c0 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.761+0000 7f34b9fe0640 1 -- 172.21.15.139:0/3760176214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b4159770 con 0x7f34b41618c0 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.763+0000 7f349bfff640 1 -- 172.21.15.139:0/3760176214 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34ac01b070 con 0x7f34b41618c0 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.763+0000 7f349bfff640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f34a006b4d0 0x7f34a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.763+0000 7f349bfff640 1 -- 172.21.15.139:0/3760176214 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f34ac0898e0 con 0x7f34b41618c0 2024-04-25T09:56:33.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.763+0000 7f34b37fe640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f34a006b4d0 0x7f34a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.764+0000 7f34b37fe640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f34a006b4d0 0x7f34a006d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f34b415a8d0 tx=0x7f34a4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.764+0000 7f34bcdc1740 1 Processor -- start 2024-04-25T09:56:33.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.764+0000 7f34bcdc1740 1 -- start start 2024-04-25T09:56:33.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.765+0000 7f34bcdc1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fb10 0x55a570352320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.765+0000 7f34bcdc1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 0x55a570352860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.765+0000 7f34bcdc1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570322770 0x55a570325f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.765+0000 7f34bcdc1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57028bda0 con 0x55a570322770 2024-04-25T09:56:33.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.765+0000 7f34bcdc1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a570234b80 con 0x55a57034fb10 2024-04-25T09:56:33.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.765+0000 7f34bcdc1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a570287e40 con 0x55a57034fef0 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.765+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570322770 0x55a570325f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.765+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570322770 0x55a570325f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56182/0 (socket says 172.21.15.139:56182) 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.765+0000 7f34b3fff640 1 -- 172.21.15.139:0/2459091865 learned_addr learned my addr 172.21.15.139:0/2459091865 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.765+0000 7f34b3fff640 1 -- 172.21.15.139:0/2459091865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 msgr2=0x55a570352860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.765+0000 7f34b3fff640 1 --2- 172.21.15.139:0/2459091865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 0x55a570352860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.766+0000 7f34b3fff640 1 -- 172.21.15.139:0/2459091865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fb10 msgr2=0x55a570352320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.766+0000 7f34b3fff640 1 --2- 172.21.15.139:0/2459091865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fb10 0x55a570352320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.766+0000 7f34b3fff640 1 -- 172.21.15.139:0/2459091865 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a57023ef10 con 0x55a570322770 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.766+0000 7f34b3fff640 1 --2- 172.21.15.139:0/2459091865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570322770 0x55a570325f80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00c700 tx=0x7f34ac05c020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfc734f99fe1532 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.766+0000 7f34997fa640 1 -- 172.21.15.139:0/2459091865 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac056280 con 0x55a570322770 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.766+0000 7f34997fa640 1 -- 172.21.15.139:0/2459091865 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ac0545f0 con 0x55a570322770 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.766+0000 7f34997fa640 1 -- 172.21.15.139:0/2459091865 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac05e290 con 0x55a570322770 2024-04-25T09:56:33.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.766+0000 7f34bcdc1740 1 -- 172.21.15.139:0/2459091865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570322770 msgr2=0x55a570325f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.766+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/2459091865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570322770 0x55a570325f80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00c700 tx=0x7f34ac05c020 comp rx=0 tx=0).stop 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.767+0000 7f34bcdc1740 1 -- 172.21.15.139:0/2459091865 shutdown_connections 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.767+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/2459091865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570322770 0x55a570325f80 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.767+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/2459091865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 0x55a570352860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.767+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/2459091865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fb10 0x55a570352320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.767+0000 7f34bcdc1740 1 -- 172.21.15.139:0/2459091865 >> 172.21.15.139:0/2459091865 conn(0x55a5703249e0 msgr2=0x55a57036d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.767+0000 7f34bcdc1740 1 -- 172.21.15.139:0/2459091865 shutdown_connections 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.767+0000 7f34bcdc1740 1 -- 172.21.15.139:0/2459091865 wait complete. 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.767+0000 7f34bcdc1740 1 -- start start 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.767+0000 7f34bcdc1740 1 Processor -- start 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34bcdc1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fb10 0x55a5703453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34bcdc1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 0x55a57036fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34bcdc1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570346490 0x55a5703542f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34bcdc1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a570327b90 con 0x55a570346490 2024-04-25T09:56:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34bcdc1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57022a4b0 con 0x55a57034fb10 2024-04-25T09:56:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34bcdc1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a570327e70 con 0x55a57034fef0 2024-04-25T09:56:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34bcdc1740 10 client.0 _ll_register_callbacks cb 0x55a57026a180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570346490 0x55a5703542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570346490 0x55a5703542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56198/0 (socket says 172.21.15.139:56198) 2024-04-25T09:56:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34b3fff640 1 -- 172.21.15.139:0/1393398228 learned_addr learned my addr 172.21.15.139:0/1393398228 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34b37fe640 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fb10 0x55a5703453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34b2ffd640 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 0x55a57036fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34b3fff640 1 -- 172.21.15.139:0/1393398228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 msgr2=0x55a57036fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34b3fff640 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 0x55a57036fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34b3fff640 1 -- 172.21.15.139:0/1393398228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fb10 msgr2=0x55a5703453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34b3fff640 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fb10 0x55a5703453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34b3fff640 1 -- 172.21.15.139:0/1393398228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a57006a7d0 con 0x55a570346490 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34b3fff640 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570346490 0x55a5703542f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0543e0 tx=0x7f34ac014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a5f34df97a8a89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.768+0000 7f34927fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34927fc640 1 -- 172.21.15.139:0/1393398228 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac074060 con 0x55a570346490 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34bcdc1740 1 -- 172.21.15.139:0/1393398228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34ac0545f0 con 0x55a570346490 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34927fc640 1 -- 172.21.15.139:0/1393398228 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f34ac0575d0 con 0x55a570346490 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34927fc640 1 -- 172.21.15.139:0/1393398228 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34ac054220 con 0x55a570346490 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34bcdc1740 10 client.6627 fetch_fsmap learned FSMap version 29 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34bcdc1740 1 -- 172.21.15.139:0/1393398228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55a56ffa03f0 con 0x55a570346490 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34927fc640 1 -- 172.21.15.139:0/1393398228 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ac0545f0 con 0x55a570346490 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34927fc640 1 -- 172.21.15.139:0/1393398228 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f34ac053cd0 con 0x55a570346490 2024-04-25T09:56:33.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34bcdc1740 10 client.6627 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34bcdc1740 10 client.6627 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34bcdc1740 20 client.6627 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a570354830 0x55a5703238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34bcdc1740 4 client.6627 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34bcdc1740 1 -- 172.21.15.139:0/1393398228 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5702992e0 con 0x55a570354830 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34b37fe640 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a570354830 0x55a5703238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34b37fe640 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a570354830 0x55a5703238d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c51164355d5170f3 server_cookie=53faab9a8c4706e6 in_seq=0 out_seq=0 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.769+0000 7f34927fc640 10 client.6627 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34927fc640 1 -- 172.21.15.139:0/1393398228 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5702992e0 con 0x55a570354830 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34927fc640 10 client.6627 handle_command_reply: tid=0 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34bcdc1740 1 client.6627 shutdown 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34bcdc1740 1 -- 172.21.15.139:0/1393398228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570346490 msgr2=0x55a5703542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570346490 0x55a5703542f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0543e0 tx=0x7f34ac014070 comp rx=0 tx=0).stop 2024-04-25T09:56:33.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34bcdc1740 1 -- 172.21.15.139:0/1393398228 shutdown_connections 2024-04-25T09:56:33.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a570354830 0x55a5703238d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a570346490 0x55a5703542f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 0x55a57036fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34927fc640 0 client.6627 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:33.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/1393398228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fb10 0x55a5703453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34bcdc1740 1 -- 172.21.15.139:0/1393398228 >> 172.21.15.139:0/1393398228 conn(0x55a570322770 msgr2=0x55a57034edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34bcdc1740 1 -- 172.21.15.139:0/1393398228 shutdown_connections 2024-04-25T09:56:33.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34bcdc1740 1 -- 172.21.15.139:0/1393398228 wait complete. 2024-04-25T09:56:33.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.770+0000 7f34bcdc1740 20 client.6627 trim_cache size 0 max 16384 2024-04-25T09:56:33.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.785+0000 7f34bcdc1740 1 Processor -- start 2024-04-25T09:56:33.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.785+0000 7f34bcdc1740 1 -- start start 2024-04-25T09:56:33.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34bcdc1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a570346490 0x55a570322e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34bcdc1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 0x55a570327510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34bcdc1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 0x55a5703217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34bcdc1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b4200640 con 0x55a57034fb10 2024-04-25T09:56:33.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34bcdc1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b41ff760 con 0x55a570346490 2024-04-25T09:56:33.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34bcdc1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40c5bc0 con 0x55a57034fef0 2024-04-25T09:56:33.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 0x55a570327510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 0x55a570327510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56206/0 (socket says 172.21.15.139:56206) 2024-04-25T09:56:33.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b2ffd640 1 -- 172.21.15.139:0/983279404 learned_addr learned my addr 172.21.15.139:0/983279404 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b37fe640 1 --2- 172.21.15.139:0/983279404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a570346490 0x55a570322e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b2ffd640 1 -- 172.21.15.139:0/983279404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 msgr2=0x55a5703217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b3fff640 1 --2- 172.21.15.139:0/983279404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 0x55a5703217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b2ffd640 1 --2- 172.21.15.139:0/983279404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 0x55a5703217f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b2ffd640 1 -- 172.21.15.139:0/983279404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a570346490 msgr2=0x55a570322e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b2ffd640 1 --2- 172.21.15.139:0/983279404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a570346490 0x55a570322e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b2ffd640 1 -- 172.21.15.139:0/983279404 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a56ff600f0 con 0x55a57034fb10 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b37fe640 1 --2- 172.21.15.139:0/983279404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a570346490 0x55a570322e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b3fff640 1 --2- 172.21.15.139:0/983279404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 0x55a5703217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.786+0000 7f34b2ffd640 1 --2- 172.21.15.139:0/983279404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 0x55a570327510 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f34a80097a0 tx=0x7f34a802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e012891dc1897301 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.787+0000 7f3498ff9640 1 -- 172.21.15.139:0/983279404 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8037040 con 0x55a57034fb10 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.787+0000 7f3498ff9640 1 -- 172.21.15.139:0/983279404 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a8032820 con 0x55a57034fb10 2024-04-25T09:56:33.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.787+0000 7f3498ff9640 1 -- 172.21.15.139:0/983279404 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8032c10 con 0x55a57034fb10 2024-04-25T09:56:33.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.787+0000 7f34bcdc1740 1 -- 172.21.15.139:0/983279404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 msgr2=0x55a570327510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.787+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/983279404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 0x55a570327510 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f34a80097a0 tx=0x7f34a802c000 comp rx=0 tx=0).stop 2024-04-25T09:56:33.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.787+0000 7f34bcdc1740 1 -- 172.21.15.139:0/983279404 shutdown_connections 2024-04-25T09:56:33.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.787+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/983279404 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 0x55a570327510 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.787+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/983279404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a57034fef0 0x55a5703217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.787+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/983279404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a570346490 0x55a570322e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.787+0000 7f34bcdc1740 1 -- 172.21.15.139:0/983279404 >> 172.21.15.139:0/983279404 conn(0x55a570322770 msgr2=0x55a570369fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.787+0000 7f34bcdc1740 1 -- 172.21.15.139:0/983279404 shutdown_connections 2024-04-25T09:56:33.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.787+0000 7f34bcdc1740 1 -- 172.21.15.139:0/983279404 wait complete. 2024-04-25T09:56:33.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.788+0000 7f34bcdc1740 1 -- start start 2024-04-25T09:56:33.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.788+0000 7f34bcdc1740 1 Processor -- start 2024-04-25T09:56:33.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34bcdc1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a570346490 0x55a5703bffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34bcdc1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 0x55a57037f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34bcdc1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fef0 0x55a570353af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34bcdc1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a570346170 con 0x55a57034fb10 2024-04-25T09:56:33.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34bcdc1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40c61c0 con 0x55a57034fef0 2024-04-25T09:56:33.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34bcdc1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5703565d0 con 0x55a570346490 2024-04-25T09:56:33.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34bcdc1740 10 client.0 _ll_register_callbacks cb 0x7f34a8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:33.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34b2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 0x55a57037f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34b2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 0x55a57037f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56210/0 (socket says 172.21.15.139:56210) 2024-04-25T09:56:33.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34b2ffd640 1 -- 172.21.15.139:0/3752491813 learned_addr learned my addr 172.21.15.139:0/3752491813 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:56:33.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34b3fff640 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fef0 0x55a570353af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34b2ffd640 1 -- 172.21.15.139:0/3752491813 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a570346490 msgr2=0x55a5703bffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34b37fe640 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a570346490 0x55a5703bffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34b2ffd640 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a570346490 0x55a5703bffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34b2ffd640 1 -- 172.21.15.139:0/3752491813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fef0 msgr2=0x55a570353af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:33.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34b2ffd640 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fef0 0x55a570353af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34b2ffd640 1 -- 172.21.15.139:0/3752491813 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a570344480 con 0x55a57034fb10 2024-04-25T09:56:33.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34b3fff640 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fef0 0x55a570353af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:33.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34b2ffd640 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 0x55a57037f930 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f34a8000c00 tx=0x7f34a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1324ff33774974ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:33.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34927fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:56:33.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34927fc640 1 -- 172.21.15.139:0/3752491813 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8037040 con 0x55a57034fb10 2024-04-25T09:56:33.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34927fc640 1 -- 172.21.15.139:0/3752491813 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f34a8052670 con 0x55a57034fb10 2024-04-25T09:56:33.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34bcdc1740 1 -- 172.21.15.139:0/3752491813 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34a8032820 con 0x55a57034fb10 2024-04-25T09:56:33.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34927fc640 1 -- 172.21.15.139:0/3752491813 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34a80528b0 con 0x55a57034fb10 2024-04-25T09:56:33.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34bcdc1740 10 client.6633 fetch_fsmap learned FSMap version 29 2024-04-25T09:56:33.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34bcdc1740 1 -- 172.21.15.139:0/3752491813 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55a57036d820 con 0x55a57034fb10 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.789+0000 7f34927fc640 1 -- 172.21.15.139:0/3752491813 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a8032820 con 0x55a57034fb10 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34927fc640 1 -- 172.21.15.139:0/3752491813 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f34a8052a50 con 0x55a57034fb10 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 10 client.6633 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 10 client.6633 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 20 client.6633 populate_metadata read hostname 'smithi139' 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a57034f190 0x55a570354030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 4 client.6633 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 -- 172.21.15.139:0/3752491813 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a570355c80 con 0x55a57034f190 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34b37fe640 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a57034f190 0x55a570354030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34927fc640 10 client.6633 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34b37fe640 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a57034f190 0x55a570354030 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19223e2bc9b8a3c server_cookie=b71ca021943a3733 in_seq=0 out_seq=0 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34927fc640 1 -- 172.21.15.139:0/3752491813 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55a570355c80 con 0x55a57034f190 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34927fc640 10 client.6633 handle_command_reply: tid=0 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 client.6633 shutdown 2024-04-25T09:56:33.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 -- 172.21.15.139:0/3752491813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 msgr2=0x55a57037f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 0x55a57037f930 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f34a8000c00 tx=0x7f34a8004730 comp rx=0 tx=0).stop 2024-04-25T09:56:33.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 -- 172.21.15.139:0/3752491813 shutdown_connections 2024-04-25T09:56:33.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55a57034f190 0x55a570354030 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a57034fb10 0x55a57037f930 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a57034fef0 0x55a570353af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34927fc640 0 client.6633 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:56:33.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 --2- 172.21.15.139:0/3752491813 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a570346490 0x55a5703bffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 -- 172.21.15.139:0/3752491813 >> 172.21.15.139:0/3752491813 conn(0x55a570322770 msgr2=0x55a570369fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 -- 172.21.15.139:0/3752491813 shutdown_connections 2024-04-25T09:56:33.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 1 -- 172.21.15.139:0/3752491813 wait complete. 2024-04-25T09:56:33.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.790+0000 7f34bcdc1740 20 client.6633 trim_cache size 0 max 16384 2024-04-25T09:56:33.803 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:56:33.803 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:56:33.803 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": { 2024-04-25T09:56:33.803 INFO:teuthology.orchestra.run.smithi139.stdout: "16befb4f-9a03-497d-bec3-690bcbf08e33": { 2024-04-25T09:56:33.803 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/", 2024-04-25T09:56:33.803 INFO:teuthology.orchestra.run.smithi139.stdout: "tag": "16befb4f-9a03-497d-bec3-690bcbf08e33", 2024-04-25T09:56:33.803 INFO:teuthology.orchestra.run.smithi139.stdout: "options": "recursive,repair,force" 2024-04-25T09:56:33.803 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:56:33.803 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:56:33.803 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:56:33.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.791+0000 7f34b9fe0640 1 -- 172.21.15.139:0/3760176214 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f34a006b4d0 msgr2=0x7f34a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.791+0000 7f34b9fe0640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f34a006b4d0 0x7f34a006d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f34b415a8d0 tx=0x7f34a4009190 comp rx=0 tx=0).stop 2024-04-25T09:56:33.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.791+0000 7f34b9fe0640 1 -- 172.21.15.139:0/3760176214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34b41618c0 msgr2=0x7f34b41f31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:33.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.791+0000 7f34b9fe0640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34b41618c0 0x7f34b41f31d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00c730 tx=0x7f34ac00cbf0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.791+0000 7f34b9fe0640 1 -- 172.21.15.139:0/3760176214 shutdown_connections 2024-04-25T09:56:33.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.791+0000 7f34b9fe0640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34b41618c0 0x7f34b41f31d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.791+0000 7f34b9fe0640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f34a006b4d0 0x7f34a006d990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.791+0000 7f34b9fe0640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34b40cd610 0x7f34b41f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.791+0000 7f34b9fe0640 1 --2- 172.21.15.139:0/3760176214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b409ad40 0x7f34b40c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:33.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.791+0000 7f34b9fe0640 1 -- 172.21.15.139:0/3760176214 >> 172.21.15.139:0/3760176214 conn(0x7f34b40b31f0 msgr2=0x7f34b40b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:33.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.791+0000 7f34b9fe0640 1 -- 172.21.15.139:0/3760176214 shutdown_connections 2024-04-25T09:56:33.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:56:33.791+0000 7f34b9fe0640 1 -- 172.21.15.139:0/3760176214 wait complete. 2024-04-25T09:56:33.806 INFO:tasks.cephfs.filesystem:scrub status for tag:16befb4f-9a03-497d-bec3-690bcbf08e33 - {'path': '/', 'tag': '16befb4f-9a03-497d-bec3-690bcbf08e33', 'options': 'recursive,repair,force'} 2024-04-25T09:56:45.867 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:56:45.871 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:57:03.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:03.808 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T09:57:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.917+0000 7f2691022640 1 Processor -- start 2024-04-25T09:57:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.917+0000 7f2691022640 1 -- start start 2024-04-25T09:57:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f2691022640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f268c0afd00 0x7f268c15cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f2691022640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f268c06fb20 0x7f268c0cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f2691022640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268c0ccef0 con 0x7f268c0cfeb0 2024-04-25T09:57:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f2691022640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0703c0 con 0x7f268c0afd00 2024-04-25T09:57:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f2691022640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0cd820 con 0x7f268c06fb20 2024-04-25T09:57:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f268ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f268c0afd00 0x7f268c15cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f268ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f268c0afd00 0x7f268c15cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49602/0 (socket says 172.21.15.139:49602) 2024-04-25T09:57:03.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f268ad9d640 1 -- 172.21.15.139:0/2943033287 learned_addr learned my addr 172.21.15.139:0/2943033287 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:03.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f268a59c640 1 --2- 172.21.15.139:0/2943033287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f268c06fb20 0x7f268c0cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f268ad9d640 1 -- 172.21.15.139:0/2943033287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f268c06fb20 msgr2=0x7f268c0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f268ad9d640 1 --2- 172.21.15.139:0/2943033287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f268c06fb20 0x7f268c0cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f268ad9d640 1 -- 172.21.15.139:0/2943033287 >> v1:172.21.15.150:6789/0 conn(0x7f268c0cfeb0 legacy=0x7f268c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.918+0000 7f268ad9d640 1 -- 172.21.15.139:0/2943033287 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0cd1d0 con 0x7f268c0afd00 2024-04-25T09:57:03.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.919+0000 7f268ad9d640 1 --2- 172.21.15.139:0/2943033287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f268c0afd00 0x7f268c15cad0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f267c009870 tx=0x7f267c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339bf0aab73bcaab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.919+0000 7f2689d9b640 1 -- 172.21.15.139:0/2943033287 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c037070 con 0x7f268c0afd00 2024-04-25T09:57:03.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.919+0000 7f2689d9b640 1 -- 172.21.15.139:0/2943033287 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c032440 con 0x7f268c0afd00 2024-04-25T09:57:03.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.919+0000 7f2689d9b640 1 -- 172.21.15.139:0/2943033287 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c032810 con 0x7f268c0afd00 2024-04-25T09:57:03.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.919+0000 7f2691022640 1 -- 172.21.15.139:0/2943033287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f268c0afd00 msgr2=0x7f268c15cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.919+0000 7f2691022640 1 --2- 172.21.15.139:0/2943033287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f268c0afd00 0x7f268c15cad0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f267c009870 tx=0x7f267c02a270 comp rx=0 tx=0).stop 2024-04-25T09:57:03.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.920+0000 7f2691022640 1 -- 172.21.15.139:0/2943033287 shutdown_connections 2024-04-25T09:57:03.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.920+0000 7f2691022640 1 --2- 172.21.15.139:0/2943033287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f268c06fb20 0x7f268c0cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.920+0000 7f2691022640 1 --2- 172.21.15.139:0/2943033287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f268c0afd00 0x7f268c15cad0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.920+0000 7f2691022640 1 -- 172.21.15.139:0/2943033287 >> 172.21.15.139:0/2943033287 conn(0x7f268c0b2230 msgr2=0x7f268c0b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:03.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.920+0000 7f2691022640 1 -- 172.21.15.139:0/2943033287 shutdown_connections 2024-04-25T09:57:03.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.920+0000 7f2691022640 1 -- 172.21.15.139:0/2943033287 wait complete. 2024-04-25T09:57:03.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.921+0000 7f2691022640 1 Processor -- start 2024-04-25T09:57:03.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.921+0000 7f2691022640 1 -- start start 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f2691022640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f268c06fb20 0x7f268c09ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f2691022640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f268c0afd00 0x7f268c14ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f2691022640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f268c0cfeb0 0x7f268c09d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f2691022640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c070100 con 0x7f268c0cfeb0 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f2691022640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c06ff80 con 0x7f268c0afd00 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f2691022640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0cdcb0 con 0x7f268c06fb20 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f268b59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f268c0cfeb0 0x7f268c09d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f268b59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f268c0cfeb0 0x7f268c09d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49614/0 (socket says 172.21.15.139:49614) 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f268b59e640 1 -- 172.21.15.139:0/2167884998 learned_addr learned my addr 172.21.15.139:0/2167884998 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f268ad9d640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f268c06fb20 0x7f268c09ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f268a59c640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f268c0afd00 0x7f268c14ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f268b59e640 1 -- 172.21.15.139:0/2167884998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f268c06fb20 msgr2=0x7f268c09ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f268b59e640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f268c06fb20 0x7f268c09ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f268b59e640 1 -- 172.21.15.139:0/2167884998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f268c0afd00 msgr2=0x7f268c14ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f268b59e640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f268c0afd00 0x7f268c14ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.922+0000 7f268b59e640 1 -- 172.21.15.139:0/2167884998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c14b550 con 0x7f268c0cfeb0 2024-04-25T09:57:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.923+0000 7f268b59e640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f268c0cfeb0 0x7f268c09d830 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f268400c6b0 tx=0x7f268400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.923+0000 7f26737fe640 1 -- 172.21.15.139:0/2167884998 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684003b70 con 0x7f268c0cfeb0 2024-04-25T09:57:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.923+0000 7f26737fe640 1 -- 172.21.15.139:0/2167884998 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684003d30 con 0x7f268c0cfeb0 2024-04-25T09:57:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.923+0000 7f2691022640 1 -- 172.21.15.139:0/2167884998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c14b830 con 0x7f268c0cfeb0 2024-04-25T09:57:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.923+0000 7f26737fe640 1 -- 172.21.15.139:0/2167884998 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684005310 con 0x7f268c0cfeb0 2024-04-25T09:57:03.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.923+0000 7f2691022640 1 -- 172.21.15.139:0/2167884998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c14bdf0 con 0x7f268c0cfeb0 2024-04-25T09:57:03.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.924+0000 7f26737fe640 1 -- 172.21.15.139:0/2167884998 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26840054b0 con 0x7f268c0cfeb0 2024-04-25T09:57:03.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.925+0000 7f26737fe640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f267806b4d0 0x7f267806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.925+0000 7f26737fe640 1 -- 172.21.15.139:0/2167884998 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f268408a120 con 0x7f268c0cfeb0 2024-04-25T09:57:03.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.925+0000 7f268ad9d640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f267806b4d0 0x7f267806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.925+0000 7f268ad9d640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f267806b4d0 0x7f267806d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f267c034000 tx=0x7f267c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.926+0000 7f2693dfd740 1 Processor -- start 2024-04-25T09:57:03.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.926+0000 7f2693dfd740 1 -- start start 2024-04-25T09:57:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f2693dfd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815af8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f2693dfd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815af8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f2693dfd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815ac8770 0x563815acbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f2693dfd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x563815a31da0 con 0x563815ac8770 2024-04-25T09:57:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f2693dfd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638159dab80 con 0x563815af5b10 2024-04-25T09:57:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f2693dfd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563815a2de40 con 0x563815af5ef0 2024-04-25T09:57:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f268b59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815ac8770 0x563815acbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f268b59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815ac8770 0x563815acbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49630/0 (socket says 172.21.15.139:49630) 2024-04-25T09:57:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f268b59e640 1 -- 172.21.15.139:0/2272051013 learned_addr learned my addr 172.21.15.139:0/2272051013 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:03.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f268ad9d640 1 --2- 172.21.15.139:0/2272051013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815af8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f268b59e640 1 -- 172.21.15.139:0/2272051013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 msgr2=0x563815af8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f268a59c640 1 --2- 172.21.15.139:0/2272051013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815af8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f268b59e640 1 --2- 172.21.15.139:0/2272051013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815af8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f268b59e640 1 -- 172.21.15.139:0/2272051013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 msgr2=0x563815af8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f268b59e640 1 --2- 172.21.15.139:0/2272051013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815af8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f268b59e640 1 -- 172.21.15.139:0/2272051013 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638159e4f10 con 0x563815ac8770 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f268ad9d640 1 --2- 172.21.15.139:0/2272051013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815af8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f268b59e640 1 --2- 172.21.15.139:0/2272051013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815ac8770 0x563815acbf80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f26840586c0 tx=0x7f26840583a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e45b8824dcdaa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f2670ff9640 1 -- 172.21.15.139:0/2272051013 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684070020 con 0x563815ac8770 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f2670ff9640 1 -- 172.21.15.139:0/2272051013 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26840701c0 con 0x563815ac8770 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.927+0000 7f2670ff9640 1 -- 172.21.15.139:0/2272051013 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684056a30 con 0x563815ac8770 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.928+0000 7f2693dfd740 1 -- 172.21.15.139:0/2272051013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815ac8770 msgr2=0x563815acbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.928+0000 7f2693dfd740 1 --2- 172.21.15.139:0/2272051013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815ac8770 0x563815acbf80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f26840586c0 tx=0x7f26840583a0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.928+0000 7f2693dfd740 1 -- 172.21.15.139:0/2272051013 shutdown_connections 2024-04-25T09:57:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.928+0000 7f2693dfd740 1 --2- 172.21.15.139:0/2272051013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815ac8770 0x563815acbf80 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.928+0000 7f2693dfd740 1 --2- 172.21.15.139:0/2272051013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815af8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.928+0000 7f2693dfd740 1 --2- 172.21.15.139:0/2272051013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815af8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.928+0000 7f2693dfd740 1 -- 172.21.15.139:0/2272051013 >> 172.21.15.139:0/2272051013 conn(0x563815aca9e0 msgr2=0x563815b13320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.928+0000 7f2693dfd740 1 -- 172.21.15.139:0/2272051013 shutdown_connections 2024-04-25T09:57:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.928+0000 7f2693dfd740 1 -- 172.21.15.139:0/2272051013 wait complete. 2024-04-25T09:57:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.929+0000 7f2693dfd740 1 -- start start 2024-04-25T09:57:03.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.929+0000 7f2693dfd740 1 Processor -- start 2024-04-25T09:57:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f2693dfd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815aeb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f2693dfd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815b272f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f2693dfd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 0x563815b16ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f2693dfd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x563815acdb90 con 0x563815aec970 2024-04-25T09:57:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f2693dfd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638159d04b0 con 0x563815af5b10 2024-04-25T09:57:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f2693dfd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563815acde70 con 0x563815af5ef0 2024-04-25T09:57:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f2693dfd740 10 client.0 _ll_register_callbacks cb 0x563815a10180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268b59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 0x563815b16ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268b59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 0x563815b16ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49646/0 (socket says 172.21.15.139:49646) 2024-04-25T09:57:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268b59e640 1 -- 172.21.15.139:0/928098140 learned_addr learned my addr 172.21.15.139:0/928098140 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:03.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268a59c640 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815b272f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268ad9d640 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815aeb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268b59e640 1 -- 172.21.15.139:0/928098140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 msgr2=0x563815b272f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268b59e640 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815b272f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268b59e640 1 -- 172.21.15.139:0/928098140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 msgr2=0x563815aeb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268b59e640 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815aeb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268b59e640 1 -- 172.21.15.139:0/928098140 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638158107d0 con 0x563815aec970 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268a59c640 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815b272f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268ad9d640 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815aeb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f268b59e640 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 0x563815b16ad0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f268400c680 tx=0x7f2684001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69df10c5c047fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f2669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f2669ffb640 1 -- 172.21.15.139:0/928098140 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684070020 con 0x563815aec970 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f2669ffb640 1 -- 172.21.15.139:0/928098140 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f2684054da0 con 0x563815aec970 2024-04-25T09:57:03.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.930+0000 7f2693dfd740 1 -- 172.21.15.139:0/928098140 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26840701c0 con 0x563815aec970 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f2669ffb640 1 -- 172.21.15.139:0/928098140 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2684077200 con 0x563815aec970 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f2693dfd740 10 client.6645 fetch_fsmap learned FSMap version 30 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f2693dfd740 1 -- 172.21.15.139:0/928098140 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x5638157463f0 con 0x563815aec970 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f2669ffb640 1 -- 172.21.15.139:0/928098140 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26840701c0 con 0x563815aec970 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f2669ffb640 1 -- 172.21.15.139:0/928098140 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2684057b80 con 0x563815aec970 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f2693dfd740 10 client.6645 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f2693dfd740 10 client.6645 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f2693dfd740 20 client.6645 populate_metadata read hostname 'smithi139' 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f2693dfd740 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x563815aef850 0x563815aefc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f2693dfd740 4 client.6645 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f2693dfd740 1 -- 172.21.15.139:0/928098140 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563815a3f2e0 con 0x563815aef850 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f268ad9d640 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x563815aef850 0x563815aefc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f268ad9d640 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x563815aef850 0x563815aefc60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c1b5e8a214a9205 server_cookie=c807ac123f4850df in_seq=0 out_seq=0 2024-04-25T09:57:03.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.931+0000 7f2669ffb640 10 client.6645 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:57:03.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2669ffb640 1 -- 172.21.15.139:0/928098140 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563815a3f2e0 con 0x563815aef850 2024-04-25T09:57:03.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2669ffb640 10 client.6645 handle_command_reply: tid=0 2024-04-25T09:57:03.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2693dfd740 1 client.6645 shutdown 2024-04-25T09:57:03.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2693dfd740 1 -- 172.21.15.139:0/928098140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 msgr2=0x563815b16ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2693dfd740 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 0x563815b16ad0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f268400c680 tx=0x7f2684001250 comp rx=0 tx=0).stop 2024-04-25T09:57:03.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2693dfd740 1 -- 172.21.15.139:0/928098140 shutdown_connections 2024-04-25T09:57:03.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2693dfd740 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x563815aef850 0x563815aefc60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2693dfd740 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 0x563815b16ad0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2693dfd740 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815b272f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2693dfd740 1 --2- 172.21.15.139:0/928098140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815aeb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2669ffb640 0 client.6645 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:57:03.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2693dfd740 1 -- 172.21.15.139:0/928098140 >> 172.21.15.139:0/928098140 conn(0x563815ac8770 msgr2=0x563815ac9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:03.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2693dfd740 1 -- 172.21.15.139:0/928098140 shutdown_connections 2024-04-25T09:57:03.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2693dfd740 1 -- 172.21.15.139:0/928098140 wait complete. 2024-04-25T09:57:03.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.932+0000 7f2693dfd740 20 client.6645 trim_cache size 0 max 16384 2024-04-25T09:57:03.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.947+0000 7f2693dfd740 1 Processor -- start 2024-04-25T09:57:03.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.947+0000 7f2693dfd740 1 -- start start 2024-04-25T09:57:03.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f2693dfd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 0x563815b161d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f2693dfd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815aefc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f2693dfd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815af0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f2693dfd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c1fe6f0 con 0x563815aec970 2024-04-25T09:57:03.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f2693dfd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c1fd810 con 0x563815af5b10 2024-04-25T09:57:03.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f2693dfd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c1fcce0 con 0x563815af5ef0 2024-04-25T09:57:03.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f268ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 0x563815b161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f268ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 0x563815b161d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49648/0 (socket says 172.21.15.139:49648) 2024-04-25T09:57:03.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f268ad9d640 1 -- 172.21.15.139:0/1223376612 learned_addr learned my addr 172.21.15.139:0/1223376612 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:03.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f268b59e640 1 --2- 172.21.15.139:0/1223376612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815af0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f268ad9d640 1 -- 172.21.15.139:0/1223376612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 msgr2=0x563815af0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f268ad9d640 1 --2- 172.21.15.139:0/1223376612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815af0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f268ad9d640 1 -- 172.21.15.139:0/1223376612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 msgr2=0x563815aefc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f268a59c640 1 --2- 172.21.15.139:0/1223376612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815aefc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f268ad9d640 1 --2- 172.21.15.139:0/1223376612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815aefc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f268ad9d640 1 -- 172.21.15.139:0/1223376612 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638157060f0 con 0x563815aec970 2024-04-25T09:57:03.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.948+0000 7f268a59c640 1 --2- 172.21.15.139:0/1223376612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815aefc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:03.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f268ad9d640 1 --2- 172.21.15.139:0/1223376612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 0x563815b161d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f267c009990 tx=0x7f267c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54db891adedddf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f266bfff640 1 -- 172.21.15.139:0/1223376612 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c037040 con 0x563815aec970 2024-04-25T09:57:03.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f266bfff640 1 -- 172.21.15.139:0/1223376612 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c052070 con 0x563815aec970 2024-04-25T09:57:03.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f266bfff640 1 -- 172.21.15.139:0/1223376612 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c04e420 con 0x563815aec970 2024-04-25T09:57:03.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f2693dfd740 1 -- 172.21.15.139:0/1223376612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 msgr2=0x563815b161d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f2693dfd740 1 --2- 172.21.15.139:0/1223376612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 0x563815b161d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f267c009990 tx=0x7f267c02c000 comp rx=0 tx=0).stop 2024-04-25T09:57:03.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f2693dfd740 1 -- 172.21.15.139:0/1223376612 shutdown_connections 2024-04-25T09:57:03.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f2693dfd740 1 --2- 172.21.15.139:0/1223376612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815af0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f2693dfd740 1 --2- 172.21.15.139:0/1223376612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815af5b10 0x563815aefc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f2693dfd740 1 --2- 172.21.15.139:0/1223376612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815aec970 0x563815b161d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f2693dfd740 1 -- 172.21.15.139:0/1223376612 >> 172.21.15.139:0/1223376612 conn(0x563815ac8770 msgr2=0x563815acdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:03.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f2693dfd740 1 -- 172.21.15.139:0/1223376612 shutdown_connections 2024-04-25T09:57:03.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.949+0000 7f2693dfd740 1 -- 172.21.15.139:0/1223376612 wait complete. 2024-04-25T09:57:03.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.950+0000 7f2693dfd740 1 -- start start 2024-04-25T09:57:03.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.950+0000 7f2693dfd740 1 Processor -- start 2024-04-25T09:57:03.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2693dfd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815aec970 0x563815b6ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2693dfd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815af5b10 0x563815af42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2693dfd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815b89ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2693dfd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x563815ac8140 con 0x563815af5b10 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2693dfd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c1fd210 con 0x563815aec970 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2693dfd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563815b36830 con 0x563815af5ef0 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2693dfd740 10 client.0 _ll_register_callbacks cb 0x7f26840549d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f268a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815af5b10 0x563815af42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f268a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815af5b10 0x563815af42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49654/0 (socket says 172.21.15.139:49654) 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f268a59c640 1 -- 172.21.15.139:0/4217497138 learned_addr learned my addr 172.21.15.139:0/4217497138 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f268ad9d640 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815aec970 0x563815b6ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f268b59e640 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815b89ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f268a59c640 1 -- 172.21.15.139:0/4217497138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 msgr2=0x563815b89ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f268a59c640 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815b89ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f268a59c640 1 -- 172.21.15.139:0/4217497138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815aec970 msgr2=0x563815b6ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f268a59c640 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815aec970 0x563815b6ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f268a59c640 1 -- 172.21.15.139:0/4217497138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563815b095d0 con 0x563815af5b10 2024-04-25T09:57:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f268a59c640 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815af5b10 0x563815af42e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f26800070f0 tx=0x7f26800146c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f12c1e7f4cb454 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:57:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2669ffb640 1 -- 172.21.15.139:0/4217497138 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268001c070 con 0x563815af5b10 2024-04-25T09:57:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2669ffb640 1 -- 172.21.15.139:0/4217497138 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f2680017470 con 0x563815af5b10 2024-04-25T09:57:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2693dfd740 1 -- 172.21.15.139:0/4217497138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f267c052070 con 0x563815af5b10 2024-04-25T09:57:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2669ffb640 1 -- 172.21.15.139:0/4217497138 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2680027050 con 0x563815af5b10 2024-04-25T09:57:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2693dfd740 10 client.6651 fetch_fsmap learned FSMap version 30 2024-04-25T09:57:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.951+0000 7f2693dfd740 1 -- 172.21.15.139:0/4217497138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x563815b22ef0 con 0x563815af5b10 2024-04-25T09:57:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2669ffb640 1 -- 172.21.15.139:0/4217497138 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c052070 con 0x563815af5b10 2024-04-25T09:57:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2669ffb640 1 -- 172.21.15.139:0/4217497138 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f268002a020 con 0x563815af5b10 2024-04-25T09:57:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 10 client.6651 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T09:57:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 10 client.6651 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:57:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 20 client.6651 populate_metadata read hostname 'smithi139' 2024-04-25T09:57:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x563815af4820 0x563815aefc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 4 client.6651 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:57:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 1 -- 172.21.15.139:0/4217497138 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563815af1470 con 0x563815af4820 2024-04-25T09:57:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f268ad9d640 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x563815af4820 0x563815aefc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2669ffb640 10 client.6651 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:57:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f268ad9d640 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x563815af4820 0x563815aefc30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7220a50bb4071 server_cookie=f1b61d9f8fd0f8ad in_seq=0 out_seq=0 2024-04-25T09:57:03.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2669ffb640 1 -- 172.21.15.139:0/4217497138 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563815af1470 con 0x563815af4820 2024-04-25T09:57:03.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2669ffb640 10 client.6651 handle_command_reply: tid=0 2024-04-25T09:57:03.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 1 client.6651 shutdown 2024-04-25T09:57:03.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 1 -- 172.21.15.139:0/4217497138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815af5b10 msgr2=0x563815af42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815af5b10 0x563815af42e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f26800070f0 tx=0x7f26800146c0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 1 -- 172.21.15.139:0/4217497138 shutdown_connections 2024-04-25T09:57:03.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x563815af4820 0x563815aefc30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563815af5b10 0x563815af42e0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2669ffb640 0 client.6651 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:57:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563815af5ef0 0x563815b89ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 1 --2- 172.21.15.139:0/4217497138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563815aec970 0x563815b6ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.952+0000 7f2693dfd740 1 -- 172.21.15.139:0/4217497138 >> 172.21.15.139:0/4217497138 conn(0x563815ac8770 msgr2=0x563815b161d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2693dfd740 1 -- 172.21.15.139:0/4217497138 shutdown_connections 2024-04-25T09:57:03.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2693dfd740 1 -- 172.21.15.139:0/4217497138 wait complete. 2024-04-25T09:57:03.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2693dfd740 20 client.6651 trim_cache size 0 max 16384 2024-04-25T09:57:03.972 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:57:03.972 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "no active scrubs running", 2024-04-25T09:57:03.973 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": {} 2024-04-25T09:57:03.973 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:57:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2691022640 1 -- 172.21.15.139:0/2167884998 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f267806b4d0 msgr2=0x7f267806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2691022640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f267806b4d0 0x7f267806d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f267c034000 tx=0x7f267c02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2691022640 1 -- 172.21.15.139:0/2167884998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f268c0cfeb0 msgr2=0x7f268c09d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2691022640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f268c0cfeb0 0x7f268c09d830 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f268400c6b0 tx=0x7f268400cb70 comp rx=0 tx=0).stop 2024-04-25T09:57:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2691022640 1 -- 172.21.15.139:0/2167884998 shutdown_connections 2024-04-25T09:57:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2691022640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f268c0cfeb0 0x7f268c09d830 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2691022640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f267806b4d0 0x7f267806d990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2691022640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f268c0afd00 0x7f268c14ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2691022640 1 --2- 172.21.15.139:0/2167884998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f268c06fb20 0x7f268c09ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2691022640 1 -- 172.21.15.139:0/2167884998 >> 172.21.15.139:0/2167884998 conn(0x7f268c0b2230 msgr2=0x7f268c15b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:03.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2691022640 1 -- 172.21.15.139:0/2167884998 shutdown_connections 2024-04-25T09:57:03.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:03.953+0000 7f2691022640 1 -- 172.21.15.139:0/2167884998 wait complete. 2024-04-25T09:57:03.977 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:57:03.978 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:04.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.071+0000 7f7f20688640 1 Processor -- start 2024-04-25T09:57:04.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.071+0000 7f7f20688640 1 -- start start 2024-04-25T09:57:04.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.072+0000 7f7f20688640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7f180cfd70 0x7f7f18168830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.072+0000 7f7f20688640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f180cbf80 con 0x7f7f180d1860 2024-04-25T09:57:04.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.072+0000 7f7f20688640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f180ceb50 con 0x7f7f18158240 2024-04-25T09:57:04.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.072+0000 7f7f20688640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f18158620 con 0x7f7f180cfd70 2024-04-25T09:57:04.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.073+0000 7f7f1ec25640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7f180cfd70 0x7f7f18168830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.073+0000 7f7f1ec25640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7f180cfd70 0x7f7f18168830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49664/0 (socket says 172.21.15.139:49664) 2024-04-25T09:57:04.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.073+0000 7f7f1ec25640 1 -- 172.21.15.139:0/3382548886 learned_addr learned my addr 172.21.15.139:0/3382548886 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:04.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.073+0000 7f7f1dc23640 1 --1- 172.21.15.139:0/3382548886 >> v1:172.21.15.139:6789/0 conn(0x7f7f180d1860 0x7f7f180cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37522/0 (socket says 172.21.15.139:37522) 2024-04-25T09:57:04.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.073+0000 7f7f1ec25640 1 -- 172.21.15.139:0/3382548886 >> v1:172.21.15.150:6789/0 conn(0x7f7f18158240 legacy=0x7f7f180d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.073+0000 7f7f1ec25640 1 -- 172.21.15.139:0/3382548886 >> v1:172.21.15.139:6789/0 conn(0x7f7f180d1860 legacy=0x7f7f180cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.073+0000 7f7f1ec25640 1 -- 172.21.15.139:0/3382548886 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f180cd2a0 con 0x7f7f180cfd70 2024-04-25T09:57:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.073+0000 7f7f1ec25640 1 --2- 172.21.15.139:0/3382548886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f180cfd70 0x7f7f18168830 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009e50 tx=0x7f7f1002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cc07dcf5042b25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.074+0000 7f7f1d422640 1 -- 172.21.15.139:0/3382548886 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10036070 con 0x7f7f180cfd70 2024-04-25T09:57:04.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.074+0000 7f7f1d422640 1 -- 172.21.15.139:0/3382548886 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f10031600 con 0x7f7f180cfd70 2024-04-25T09:57:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.074+0000 7f7f1d422640 1 -- 172.21.15.139:0/3382548886 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f100319d0 con 0x7f7f180cfd70 2024-04-25T09:57:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.074+0000 7f7f20688640 1 -- 172.21.15.139:0/3382548886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f180cfd70 msgr2=0x7f7f18168830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.074+0000 7f7f20688640 1 --2- 172.21.15.139:0/3382548886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f180cfd70 0x7f7f18168830 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009e50 tx=0x7f7f1002a680 comp rx=0 tx=0).stop 2024-04-25T09:57:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.074+0000 7f7f20688640 1 -- 172.21.15.139:0/3382548886 shutdown_connections 2024-04-25T09:57:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.074+0000 7f7f20688640 1 --2- 172.21.15.139:0/3382548886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f180cfd70 0x7f7f18168830 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.074+0000 7f7f20688640 1 -- 172.21.15.139:0/3382548886 >> 172.21.15.139:0/3382548886 conn(0x7f7f180b3380 msgr2=0x7f7f180b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.074+0000 7f7f20688640 1 -- 172.21.15.139:0/3382548886 shutdown_connections 2024-04-25T09:57:04.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.074+0000 7f7f20688640 1 -- 172.21.15.139:0/3382548886 wait complete. 2024-04-25T09:57:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.075+0000 7f7f20688640 1 Processor -- start 2024-04-25T09:57:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.076+0000 7f7f20688640 1 -- start start 2024-04-25T09:57:04.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.076+0000 7f7f20688640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f180cfd70 0x7f7f18153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.076+0000 7f7f20688640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f180d1860 0x7f7f180c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.076+0000 7f7f20688640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f18158240 0x7f7f18155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.076+0000 7f7f20688640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f180cc4e0 con 0x7f7f180d1860 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.076+0000 7f7f20688640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f180cce10 con 0x7f7f180cfd70 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.076+0000 7f7f20688640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f180cc7c0 con 0x7f7f18158240 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.076+0000 7f7f1dc23640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f180d1860 0x7f7f180c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.076+0000 7f7f1dc23640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f180d1860 0x7f7f180c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49672/0 (socket says 172.21.15.139:49672) 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.076+0000 7f7f1dc23640 1 -- 172.21.15.139:0/1244820583 learned_addr learned my addr 172.21.15.139:0/1244820583 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f1dc23640 1 -- 172.21.15.139:0/1244820583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f18158240 msgr2=0x7f7f18155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f1dc23640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f18158240 0x7f7f18155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f1dc23640 1 -- 172.21.15.139:0/1244820583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f180cfd70 msgr2=0x7f7f18153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f1e424640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f180cfd70 0x7f7f18153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f1dc23640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f180cfd70 0x7f7f18153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f1dc23640 1 -- 172.21.15.139:0/1244820583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f180c53c0 con 0x7f7f180d1860 2024-04-25T09:57:04.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f1e424640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f180cfd70 0x7f7f18153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:04.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f1dc23640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f180d1860 0x7f7f180c7150 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400c890 tx=0x7f7f1400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f0effd640 1 -- 172.21.15.139:0/1244820583 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14018070 con 0x7f7f180d1860 2024-04-25T09:57:04.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f0effd640 1 -- 172.21.15.139:0/1244820583 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f14007480 con 0x7f7f180d1860 2024-04-25T09:57:04.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f20688640 1 -- 172.21.15.139:0/1244820583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f18153da0 con 0x7f7f180d1860 2024-04-25T09:57:04.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f0effd640 1 -- 172.21.15.139:0/1244820583 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f140049e0 con 0x7f7f180d1860 2024-04-25T09:57:04.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.077+0000 7f7f20688640 1 -- 172.21.15.139:0/1244820583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f180c7e10 con 0x7f7f180d1860 2024-04-25T09:57:04.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.078+0000 7f7f20688640 1 -- 172.21.15.139:0/1244820583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f18074520 con 0x7f7f180d1860 2024-04-25T09:57:04.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.078+0000 7f7f0effd640 1 -- 172.21.15.139:0/1244820583 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f1401b070 con 0x7f7f180d1860 2024-04-25T09:57:04.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.079+0000 7f7f0effd640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7efc06b4d0 0x7f7efc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.079+0000 7f7f0effd640 1 -- 172.21.15.139:0/1244820583 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f7f14089bf0 con 0x7f7f180d1860 2024-04-25T09:57:04.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.079+0000 7f7f1e424640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7efc06b4d0 0x7f7efc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.079+0000 7f7f1e424640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7efc06b4d0 0x7f7efc06d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7f08003eb0 tx=0x7f7f08009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.083+0000 7f7f0effd640 1 -- 172.21.15.139:0/1244820583 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f180cbf80 con 0x7f7f180d1860 2024-04-25T09:57:04.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.287+0000 7f7f20688640 1 -- 172.21.15.139:0/1244820583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f180b7490 con 0x7f7f180d1860 2024-04-25T09:57:04.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.287+0000 7f7f0effd640 1 -- 172.21.15.139:0/1244820583 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f7f14058850 con 0x7f7f180d1860 2024-04-25T09:57:04.291 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:04.291 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/829268166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":829268166},{"type":"v1","addr":"172.21.15.150:6835","nonce":829268166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3075039902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3075039902},{"type":"v1","addr":"172.21.15.150:6834","nonce":3075039902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/627726069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":627726069},{"type":"v1","addr":"172.21.15.139:6836","nonce":627726069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:50.626292+0000","modified":"2024-04-25T09:56:39.327977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.139:6837/3541444834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3541444834},{"type":"v1","addr":"172.21.15.139:6837","nonce":3541444834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:57:04.291 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 30 2024-04-25T09:57:04.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.289+0000 7f7f20688640 1 -- 172.21.15.139:0/1244820583 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7efc06b4d0 msgr2=0x7f7efc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.289+0000 7f7f20688640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7efc06b4d0 0x7f7efc06d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7f08003eb0 tx=0x7f7f08009040 comp rx=0 tx=0).stop 2024-04-25T09:57:04.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.289+0000 7f7f20688640 1 -- 172.21.15.139:0/1244820583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f180d1860 msgr2=0x7f7f180c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.289+0000 7f7f20688640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f180d1860 0x7f7f180c7150 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400c890 tx=0x7f7f1400cd50 comp rx=0 tx=0).stop 2024-04-25T09:57:04.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.289+0000 7f7f20688640 1 -- 172.21.15.139:0/1244820583 shutdown_connections 2024-04-25T09:57:04.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.289+0000 7f7f20688640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f180d1860 0x7f7f180c7150 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.289+0000 7f7f20688640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7efc06b4d0 0x7f7efc06d990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.289+0000 7f7f20688640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f18158240 0x7f7f18155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.289+0000 7f7f20688640 1 --2- 172.21.15.139:0/1244820583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f180cfd70 0x7f7f18153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.289+0000 7f7f20688640 1 -- 172.21.15.139:0/1244820583 >> 172.21.15.139:0/1244820583 conn(0x7f7f180b3380 msgr2=0x7f7f1815e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.289+0000 7f7f20688640 1 -- 172.21.15.139:0/1244820583 shutdown_connections 2024-04-25T09:57:04.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.289+0000 7f7f20688640 1 -- 172.21.15.139:0/1244820583 wait complete. 2024-04-25T09:57:04.303 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T09:57:04.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.399+0000 7fda267b0640 1 Processor -- start 2024-04-25T09:57:04.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.399+0000 7fda267b0640 1 -- start start 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.399+0000 7fda267b0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fda20070970 0x7fda200b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.399+0000 7fda267b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda200b4190 0x7fda200d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.399+0000 7fda267b0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda200b45e0 con 0x7fda2015a870 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.399+0000 7fda267b0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda200d1020 con 0x7fda20070970 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.399+0000 7fda267b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda200b5190 con 0x7fda200b4190 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.400+0000 7fda1f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fda20070970 0x7fda200b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.400+0000 7fda1f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fda20070970 0x7fda200b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49682/0 (socket says 172.21.15.139:49682) 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.400+0000 7fda1f7fe640 1 -- 172.21.15.139:0/2501691897 learned_addr learned my addr 172.21.15.139:0/2501691897 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.400+0000 7fda1ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fda2015a870 0x7fda20070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37528/0 (socket says 172.21.15.139:37528) 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.400+0000 7fda24d4d640 1 --2- 172.21.15.139:0/2501691897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda200b4190 0x7fda200d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.400+0000 7fda1f7fe640 1 -- 172.21.15.139:0/2501691897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda200b4190 msgr2=0x7fda200d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.400+0000 7fda1f7fe640 1 --2- 172.21.15.139:0/2501691897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda200b4190 0x7fda200d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.400+0000 7fda1f7fe640 1 -- 172.21.15.139:0/2501691897 >> v1:172.21.15.139:6789/0 conn(0x7fda2015a870 legacy=0x7fda20070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.400+0000 7fda1effd640 1 -- 172.21.15.139:0/2501691897 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276024237 0 0) 0x7fda200b45e0 con 0x7fda2015a870 2024-04-25T09:57:04.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.400+0000 7fda1f7fe640 1 -- 172.21.15.139:0/2501691897 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda200d0d40 con 0x7fda20070970 2024-04-25T09:57:04.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.400+0000 7fda24d4d640 1 --2- 172.21.15.139:0/2501691897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda200b4190 0x7fda200d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:04.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.401+0000 7fda1f7fe640 1 --2- 172.21.15.139:0/2501691897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda20070970 0x7fda200b5620 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fda14009870 tx=0x7fda1402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3246b300b609b683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.401+0000 7fda1effd640 1 -- 172.21.15.139:0/2501691897 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14037070 con 0x7fda20070970 2024-04-25T09:57:04.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.401+0000 7fda1effd640 1 -- 172.21.15.139:0/2501691897 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda14032440 con 0x7fda20070970 2024-04-25T09:57:04.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.401+0000 7fda1effd640 1 -- 172.21.15.139:0/2501691897 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14032810 con 0x7fda20070970 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.401+0000 7fda267b0640 1 -- 172.21.15.139:0/2501691897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda20070970 msgr2=0x7fda200b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.401+0000 7fda267b0640 1 --2- 172.21.15.139:0/2501691897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda20070970 0x7fda200b5620 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fda14009870 tx=0x7fda1402a130 comp rx=0 tx=0).stop 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.401+0000 7fda267b0640 1 -- 172.21.15.139:0/2501691897 shutdown_connections 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.401+0000 7fda267b0640 1 --2- 172.21.15.139:0/2501691897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda200b4190 0x7fda200d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.401+0000 7fda267b0640 1 --2- 172.21.15.139:0/2501691897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda20070970 0x7fda200b5620 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.401+0000 7fda267b0640 1 -- 172.21.15.139:0/2501691897 >> 172.21.15.139:0/2501691897 conn(0x7fda200b23f0 msgr2=0x7fda200b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.401+0000 7fda267b0640 1 -- 172.21.15.139:0/2501691897 shutdown_connections 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.401+0000 7fda267b0640 1 -- 172.21.15.139:0/2501691897 wait complete. 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.402+0000 7fda267b0640 1 Processor -- start 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.403+0000 7fda267b0640 1 -- start start 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.403+0000 7fda267b0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda20070970 0x7fda20157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.403+0000 7fda267b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda200b4190 0x7fda20139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.403+0000 7fda267b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda2015a870 0x7fda200b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.403+0000 7fda267b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda200ce610 con 0x7fda200b4190 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.403+0000 7fda267b0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda200b4e20 con 0x7fda20070970 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.403+0000 7fda267b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda200ce790 con 0x7fda2015a870 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.403+0000 7fda1f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda200b4190 0x7fda20139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.403+0000 7fda1f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda200b4190 0x7fda20139ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49684/0 (socket says 172.21.15.139:49684) 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.403+0000 7fda1f7fe640 1 -- 172.21.15.139:0/1140977487 learned_addr learned my addr 172.21.15.139:0/1140977487 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda1ffff640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda20070970 0x7fda20157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda24d4d640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda2015a870 0x7fda200b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda1f7fe640 1 -- 172.21.15.139:0/1140977487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda2015a870 msgr2=0x7fda200b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda1f7fe640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda2015a870 0x7fda200b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda1f7fe640 1 -- 172.21.15.139:0/1140977487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda20070970 msgr2=0x7fda20157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda1f7fe640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda20070970 0x7fda20157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda1f7fe640 1 -- 172.21.15.139:0/1140977487 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda201f5cd0 con 0x7fda200b4190 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda1f7fe640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda200b4190 0x7fda20139ef0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fda14000c00 tx=0x7fda140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda1cff9640 1 -- 172.21.15.139:0/1140977487 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14037040 con 0x7fda200b4190 2024-04-25T09:57:04.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda1cff9640 1 -- 172.21.15.139:0/1140977487 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda14042070 con 0x7fda200b4190 2024-04-25T09:57:04.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda267b0640 1 -- 172.21.15.139:0/1140977487 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda20153340 con 0x7fda200b4190 2024-04-25T09:57:04.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda1cff9640 1 -- 172.21.15.139:0/1140977487 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14030bd0 con 0x7fda200b4190 2024-04-25T09:57:04.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.404+0000 7fda267b0640 1 -- 172.21.15.139:0/1140977487 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda20153900 con 0x7fda200b4190 2024-04-25T09:57:04.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.406+0000 7fda1cff9640 1 -- 172.21.15.139:0/1140977487 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda14047460 con 0x7fda200b4190 2024-04-25T09:57:04.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.406+0000 7fda1cff9640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fda0c06b4d0 0x7fda0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.406+0000 7fda1cff9640 1 -- 172.21.15.139:0/1140977487 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fda140aaea0 con 0x7fda200b4190 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.406+0000 7fda1ffff640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fda0c06b4d0 0x7fda0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.406+0000 7fda1ffff640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fda0c06b4d0 0x7fda0c06d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fda10007fd0 tx=0x7fda10007e00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.407+0000 7fda2958d740 1 Processor -- start 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.407+0000 7fda2958d740 1 -- start start 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda2958d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 0x55cad3e60760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda2958d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e3a730 0x55cad3e7f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda2958d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e7f6e0 0x55cad3e375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda2958d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad3d9bc60 con 0x55cad3e60380 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda2958d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad3d3ad80 con 0x55cad3e3a730 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda2958d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cad3d97d00 con 0x55cad3e7f6e0 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda1ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 0x55cad3e60760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda1ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 0x55cad3e60760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49694/0 (socket says 172.21.15.139:49694) 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda1ffff640 1 -- 172.21.15.139:0/1300324879 learned_addr learned my addr 172.21.15.139:0/1300324879 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda1f7fe640 1 --2- 172.21.15.139:0/1300324879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e3a730 0x55cad3e7f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda24d4d640 1 --2- 172.21.15.139:0/1300324879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e7f6e0 0x55cad3e375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda1ffff640 1 -- 172.21.15.139:0/1300324879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e7f6e0 msgr2=0x55cad3e375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda1ffff640 1 --2- 172.21.15.139:0/1300324879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e7f6e0 0x55cad3e375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda1ffff640 1 -- 172.21.15.139:0/1300324879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e3a730 msgr2=0x55cad3e7f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda1ffff640 1 --2- 172.21.15.139:0/1300324879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e3a730 0x55cad3e7f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda1ffff640 1 -- 172.21.15.139:0/1300324879 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cad3b7c7c0 con 0x55cad3e60380 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda24d4d640 1 --2- 172.21.15.139:0/1300324879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e7f6e0 0x55cad3e375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda1ffff640 1 --2- 172.21.15.139:0/1300324879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 0x55cad3e60760 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fda10007b70 tx=0x7fda10013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8111749d41c204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda05ffb640 1 -- 172.21.15.139:0/1300324879 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10026070 con 0x55cad3e60380 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda05ffb640 1 -- 172.21.15.139:0/1300324879 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda10001cf0 con 0x55cad3e60380 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.408+0000 7fda05ffb640 1 -- 172.21.15.139:0/1300324879 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10013d80 con 0x55cad3e60380 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.409+0000 7fda2958d740 1 -- 172.21.15.139:0/1300324879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 msgr2=0x55cad3e60760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.409+0000 7fda2958d740 1 --2- 172.21.15.139:0/1300324879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 0x55cad3e60760 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fda10007b70 tx=0x7fda10013000 comp rx=0 tx=0).stop 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.409+0000 7fda2958d740 1 -- 172.21.15.139:0/1300324879 shutdown_connections 2024-04-25T09:57:04.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.409+0000 7fda2958d740 1 --2- 172.21.15.139:0/1300324879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e7f6e0 0x55cad3e375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.409+0000 7fda2958d740 1 --2- 172.21.15.139:0/1300324879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e3a730 0x55cad3e7f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.409+0000 7fda2958d740 1 --2- 172.21.15.139:0/1300324879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 0x55cad3e60760 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.409+0000 7fda2958d740 1 -- 172.21.15.139:0/1300324879 >> 172.21.15.139:0/1300324879 conn(0x55cad3e006e0 msgr2=0x55cad3e7e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.409+0000 7fda2958d740 1 -- 172.21.15.139:0/1300324879 shutdown_connections 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.409+0000 7fda2958d740 1 -- 172.21.15.139:0/1300324879 wait complete. 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.409+0000 7fda2958d740 1 -- start start 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.410+0000 7fda2958d740 1 Processor -- start 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.410+0000 7fda2958d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e3a730 0x55cad3e62eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.410+0000 7fda2958d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e60380 0x55cad3e38ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.410+0000 7fda2958d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e5a520 0x55cad3e39010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.410+0000 7fda2958d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad3e34b90 con 0x55cad3e5a520 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.410+0000 7fda2958d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad3d38840 con 0x55cad3e3a730 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.410+0000 7fda2958d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cad3e396c0 con 0x55cad3e60380 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.410+0000 7fda2958d740 10 client.0 _ll_register_callbacks cb 0x55cad3d52980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:04.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.410+0000 7fda24d4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e5a520 0x55cad3e39010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda24d4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e5a520 0x55cad3e39010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49702/0 (socket says 172.21.15.139:49702) 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda24d4d640 1 -- 172.21.15.139:0/809571243 learned_addr learned my addr 172.21.15.139:0/809571243 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda1ffff640 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e3a730 0x55cad3e62eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda1f7fe640 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e60380 0x55cad3e38ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda24d4d640 1 -- 172.21.15.139:0/809571243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e60380 msgr2=0x55cad3e38ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda24d4d640 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e60380 0x55cad3e38ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda24d4d640 1 -- 172.21.15.139:0/809571243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e3a730 msgr2=0x55cad3e62eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda24d4d640 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e3a730 0x55cad3e62eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda24d4d640 1 -- 172.21.15.139:0/809571243 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cad3bdcf50 con 0x55cad3e5a520 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda24d4d640 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e5a520 0x55cad3e39010 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fda1800ef50 tx=0x7fda1800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df7d4cd7b8a12e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fd9feffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fd9feffd640 1 -- 172.21.15.139:0/809571243 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1801c060 con 0x55cad3e5a520 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fd9feffd640 1 -- 172.21.15.139:0/809571243 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fda18017410 con 0x55cad3e5a520 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda2958d740 1 -- 172.21.15.139:0/809571243 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda10001cf0 con 0x55cad3e5a520 2024-04-25T09:57:04.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fd9feffd640 1 -- 172.21.15.139:0/809571243 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda18017610 con 0x55cad3e5a520 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda2958d740 10 client.6675 fetch_fsmap learned FSMap version 30 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda2958d740 1 -- 172.21.15.139:0/809571243 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55cad3a700b0 con 0x55cad3e5a520 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fd9feffd640 1 -- 172.21.15.139:0/809571243 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda10001cf0 con 0x55cad3e5a520 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fd9feffd640 1 -- 172.21.15.139:0/809571243 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fda180177b0 con 0x55cad3e5a520 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda2958d740 10 client.6675 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda2958d740 10 client.6675 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda2958d740 20 client.6675 populate_metadata read hostname 'smithi139' 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda2958d740 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55cad3e60d10 0x55cad3e610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda2958d740 4 client.6675 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.411+0000 7fda2958d740 1 -- 172.21.15.139:0/809571243 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cad3d5be60 con 0x55cad3e60d10 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.412+0000 7fda1ffff640 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55cad3e60d10 0x55cad3e610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.412+0000 7fd9feffd640 10 client.6675 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.412+0000 7fda1ffff640 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55cad3e60d10 0x55cad3e610f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4407448fe2308c5b server_cookie=6e06bded4125285 in_seq=0 out_seq=0 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.412+0000 7fd9feffd640 1 -- 172.21.15.139:0/809571243 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cad3d5be60 con 0x55cad3e60d10 2024-04-25T09:57:04.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.412+0000 7fd9feffd640 10 client.6675 handle_command_reply: tid=0 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.412+0000 7fda2958d740 1 client.6675 shutdown 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.413+0000 7fda2958d740 1 -- 172.21.15.139:0/809571243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e5a520 msgr2=0x55cad3e39010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.413+0000 7fda2958d740 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e5a520 0x55cad3e39010 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fda1800ef50 tx=0x7fda1800c7f0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.413+0000 7fda2958d740 1 -- 172.21.15.139:0/809571243 shutdown_connections 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.413+0000 7fda2958d740 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55cad3e60d10 0x55cad3e610f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.413+0000 7fda2958d740 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e5a520 0x55cad3e39010 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.413+0000 7fda2958d740 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e60380 0x55cad3e38ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.413+0000 7fd9feffd640 0 client.6675 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.413+0000 7fda2958d740 1 --2- 172.21.15.139:0/809571243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e3a730 0x55cad3e62eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.413+0000 7fda2958d740 1 -- 172.21.15.139:0/809571243 >> 172.21.15.139:0/809571243 conn(0x55cad3e006e0 msgr2=0x55cad3e80d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.413+0000 7fda2958d740 1 -- 172.21.15.139:0/809571243 shutdown_connections 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.413+0000 7fda2958d740 1 -- 172.21.15.139:0/809571243 wait complete. 2024-04-25T09:57:04.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.413+0000 7fda2958d740 20 client.6675 trim_cache size 0 max 16384 2024-04-25T09:57:04.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.424+0000 7fda2958d740 1 Processor -- start 2024-04-25T09:57:04.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.424+0000 7fda2958d740 1 -- start start 2024-04-25T09:57:04.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda2958d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e3a730 0x55cad3e7e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda2958d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e5a520 0x55cad3e32e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda2958d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 0x55cad3e65f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda2958d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda20166660 con 0x55cad3e60380 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda2958d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2015cb40 con 0x55cad3e5a520 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda2958d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda20154c00 con 0x55cad3e3a730 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda24d4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 0x55cad3e65f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda24d4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 0x55cad3e65f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49710/0 (socket says 172.21.15.139:49710) 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda24d4d640 1 -- 172.21.15.139:0/2948960214 learned_addr learned my addr 172.21.15.139:0/2948960214 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda24d4d640 1 -- 172.21.15.139:0/2948960214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e3a730 msgr2=0x55cad3e7e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda24d4d640 1 --2- 172.21.15.139:0/2948960214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e3a730 0x55cad3e7e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda24d4d640 1 -- 172.21.15.139:0/2948960214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e5a520 msgr2=0x55cad3e32e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda24d4d640 1 --2- 172.21.15.139:0/2948960214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e5a520 0x55cad3e32e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda24d4d640 1 -- 172.21.15.139:0/2948960214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cad3e368f0 con 0x55cad3e60380 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda24d4d640 1 --2- 172.21.15.139:0/2948960214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 0x55cad3e65f00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fda1800cd70 tx=0x7fda18014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84733dcb422e5b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.425+0000 7fda057fa640 1 -- 172.21.15.139:0/2948960214 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18007e70 con 0x55cad3e60380 2024-04-25T09:57:04.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.426+0000 7fda057fa640 1 -- 172.21.15.139:0/2948960214 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1802a070 con 0x55cad3e60380 2024-04-25T09:57:04.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.426+0000 7fda057fa640 1 -- 172.21.15.139:0/2948960214 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda180177b0 con 0x55cad3e60380 2024-04-25T09:57:04.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.426+0000 7fda2958d740 1 -- 172.21.15.139:0/2948960214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 msgr2=0x55cad3e65f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.426+0000 7fda2958d740 1 --2- 172.21.15.139:0/2948960214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 0x55cad3e65f00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fda1800cd70 tx=0x7fda18014000 comp rx=0 tx=0).stop 2024-04-25T09:57:04.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.426+0000 7fda2958d740 1 -- 172.21.15.139:0/2948960214 shutdown_connections 2024-04-25T09:57:04.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.426+0000 7fda2958d740 1 --2- 172.21.15.139:0/2948960214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e60380 0x55cad3e65f00 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.426+0000 7fda2958d740 1 --2- 172.21.15.139:0/2948960214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e5a520 0x55cad3e32e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.426+0000 7fda2958d740 1 --2- 172.21.15.139:0/2948960214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e3a730 0x55cad3e7e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.426+0000 7fda2958d740 1 -- 172.21.15.139:0/2948960214 >> 172.21.15.139:0/2948960214 conn(0x55cad3e006e0 msgr2=0x55cad3e73dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.426+0000 7fda2958d740 1 -- 172.21.15.139:0/2948960214 shutdown_connections 2024-04-25T09:57:04.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.426+0000 7fda2958d740 1 -- 172.21.15.139:0/2948960214 wait complete. 2024-04-25T09:57:04.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.427+0000 7fda2958d740 1 -- start start 2024-04-25T09:57:04.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.427+0000 7fda2958d740 1 Processor -- start 2024-04-25T09:57:04.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda2958d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e3a730 0x55cad3ee3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda2958d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e5a520 0x55cad3e65ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda2958d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e60380 0x55cad3e66420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda2958d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad3e64910 con 0x55cad3e3a730 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda2958d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cad3e64110 con 0x55cad3e60380 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda2958d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cad3e5ad60 con 0x55cad3e5a520 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda2958d740 10 client.0 _ll_register_callbacks cb 0x7fda100072e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda1ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e3a730 0x55cad3ee3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda1ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e3a730 0x55cad3ee3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49716/0 (socket says 172.21.15.139:49716) 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda1ffff640 1 -- 172.21.15.139:0/704687305 learned_addr learned my addr 172.21.15.139:0/704687305 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda1f7fe640 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e5a520 0x55cad3e65ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda1ffff640 1 -- 172.21.15.139:0/704687305 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e5a520 msgr2=0x55cad3e65ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda1ffff640 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e5a520 0x55cad3e65ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda24d4d640 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e60380 0x55cad3e66420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda1ffff640 1 -- 172.21.15.139:0/704687305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e60380 msgr2=0x55cad3e66420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda1ffff640 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e60380 0x55cad3e66420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda1ffff640 1 -- 172.21.15.139:0/704687305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cad3ea5320 con 0x55cad3e3a730 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda1ffff640 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e3a730 0x55cad3ee3f10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x55cad3e8a9c0 tx=0x7fda10001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5fb442b8ebe544 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fd9feffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fd9feffd640 1 -- 172.21.15.139:0/704687305 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10035020 con 0x55cad3e3a730 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fd9feffd640 1 -- 172.21.15.139:0/704687305 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fda100310b0 con 0x55cad3e3a730 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda2958d740 1 -- 172.21.15.139:0/704687305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda1802a070 con 0x55cad3e3a730 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fd9feffd640 1 -- 172.21.15.139:0/704687305 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda1003f050 con 0x55cad3e3a730 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda2958d740 10 client.6681 fetch_fsmap learned FSMap version 30 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.428+0000 7fda2958d740 1 -- 172.21.15.139:0/704687305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55cad3ee5e20 con 0x55cad3e3a730 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fd9feffd640 1 -- 172.21.15.139:0/704687305 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1802a070 con 0x55cad3e3a730 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fd9feffd640 1 -- 172.21.15.139:0/704687305 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fda10042020 con 0x55cad3e3a730 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fda2958d740 10 client.6681 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fda2958d740 10 client.6681 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fda2958d740 20 client.6681 populate_metadata read hostname 'smithi139' 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fda2958d740 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55cad3ee5880 0x55cad3e88be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fda2958d740 4 client.6681 mds_command: new command op to 6312 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fda2958d740 1 -- 172.21.15.139:0/704687305 --> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55cad3da91a0 con 0x55cad3ee5880 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fda1f7fe640 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55cad3ee5880 0x55cad3e88be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fd9feffd640 10 client.6681 ms_handle_connect on v2:172.21.15.139:6835/3541444834 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fda1f7fe640 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55cad3ee5880 0x55cad3e88be0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f1a4ec4d41748e server_cookie=de50730aee2356b5 in_seq=0 out_seq=0 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fd9feffd640 1 -- 172.21.15.139:0/704687305 <== mds.0 v2:172.21.15.139:6835/3541444834 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cad3da91a0 con 0x55cad3ee5880 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fd9feffd640 10 client.6681 handle_command_reply: tid=0 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.429+0000 7fda2958d740 1 client.6681 shutdown 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda2958d740 1 -- 172.21.15.139:0/704687305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e3a730 msgr2=0x55cad3ee3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda2958d740 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e3a730 0x55cad3ee3f10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x55cad3e8a9c0 tx=0x7fda10001080 comp rx=0 tx=0).stop 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda2958d740 1 -- 172.21.15.139:0/704687305 shutdown_connections 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda2958d740 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.139:6835/3541444834,v1:172.21.15.139:6837/3541444834] conn(0x55cad3ee5880 0x55cad3e88be0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda2958d740 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cad3e60380 0x55cad3e66420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda2958d740 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55cad3e5a520 0x55cad3e65ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda2958d740 1 --2- 172.21.15.139:0/704687305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cad3e3a730 0x55cad3ee3f10 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fd9feffd640 0 client.6681 ms_handle_reset on v2:172.21.15.139:6835/3541444834 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda2958d740 1 -- 172.21.15.139:0/704687305 >> 172.21.15.139:0/704687305 conn(0x55cad3e006e0 msgr2=0x55cad3e89e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda2958d740 1 -- 172.21.15.139:0/704687305 shutdown_connections 2024-04-25T09:57:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda2958d740 1 -- 172.21.15.139:0/704687305 wait complete. 2024-04-25T09:57:04.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda2958d740 20 client.6681 trim_cache size 0 max 16384 2024-04-25T09:57:04.439 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:04.439 INFO:teuthology.orchestra.run.smithi139.stdout:[] 2024-04-25T09:57:04.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda267b0640 1 -- 172.21.15.139:0/1140977487 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fda0c06b4d0 msgr2=0x7fda0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda267b0640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fda0c06b4d0 0x7fda0c06d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fda10007fd0 tx=0x7fda10007e00 comp rx=0 tx=0).stop 2024-04-25T09:57:04.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda267b0640 1 -- 172.21.15.139:0/1140977487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda200b4190 msgr2=0x7fda20139ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda267b0640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda200b4190 0x7fda20139ef0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fda14000c00 tx=0x7fda140046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda267b0640 1 -- 172.21.15.139:0/1140977487 shutdown_connections 2024-04-25T09:57:04.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda267b0640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda200b4190 0x7fda20139ef0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda267b0640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fda0c06b4d0 0x7fda0c06d990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda267b0640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda2015a870 0x7fda200b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda267b0640 1 --2- 172.21.15.139:0/1140977487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda20070970 0x7fda20157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.430+0000 7fda267b0640 1 -- 172.21.15.139:0/1140977487 >> 172.21.15.139:0/1140977487 conn(0x7fda200b23f0 msgr2=0x7fda200b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.431+0000 7fda267b0640 1 -- 172.21.15.139:0/1140977487 shutdown_connections 2024-04-25T09:57:04.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.431+0000 7fda267b0640 1 -- 172.21.15.139:0/1140977487 wait complete. 2024-04-25T09:57:04.444 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T09:57:04.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.537+0000 7fcdb214c640 1 Processor -- start 2024-04-25T09:57:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.537+0000 7fcdb214c640 1 -- start start 2024-04-25T09:57:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdb214c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcdac15b950 0x7fcdac072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdb214c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcdac06ab60 0x7fcdac06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdb214c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac070a40 con 0x7fcdac15f580 2024-04-25T09:57:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdb214c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac06b500 con 0x7fcdac06ab60 2024-04-25T09:57:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdb214c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac06fc80 con 0x7fcdac15b950 2024-04-25T09:57:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdaaffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcdac06ab60 0x7fcdac06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdaaffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcdac06ab60 0x7fcdac06afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49730/0 (socket says 172.21.15.139:49730) 2024-04-25T09:57:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdaaffd640 1 -- 172.21.15.139:0/60019321 learned_addr learned my addr 172.21.15.139:0/60019321 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdab7fe640 1 --2- 172.21.15.139:0/60019321 >> v2:172.21.15.150:3300/0 conn(0x7fcdac15b950 0x7fcdac072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdaaffd640 1 -- 172.21.15.139:0/60019321 >> v2:172.21.15.150:3300/0 conn(0x7fcdac15b950 msgr2=0x7fcdac072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdaaffd640 1 --2- 172.21.15.139:0/60019321 >> v2:172.21.15.150:3300/0 conn(0x7fcdac15b950 0x7fcdac072a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdaaffd640 1 -- 172.21.15.139:0/60019321 >> v1:172.21.15.150:6789/0 conn(0x7fcdac15f580 legacy=0x7fcdac15fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.538+0000 7fcdaaffd640 1 -- 172.21.15.139:0/60019321 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac06ff60 con 0x7fcdac06ab60 2024-04-25T09:57:04.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.539+0000 7fcdaaffd640 1 --2- 172.21.15.139:0/60019321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdac06ab60 0x7fcdac06afc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0d16d0 tx=0x7fcda002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35429d787bd6e843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.539+0000 7fcdaa7fc640 1 -- 172.21.15.139:0/60019321 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0037070 con 0x7fcdac06ab60 2024-04-25T09:57:04.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.539+0000 7fcdaa7fc640 1 -- 172.21.15.139:0/60019321 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda0032440 con 0x7fcdac06ab60 2024-04-25T09:57:04.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.539+0000 7fcdaa7fc640 1 -- 172.21.15.139:0/60019321 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0032810 con 0x7fcdac06ab60 2024-04-25T09:57:04.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.540+0000 7fcdb214c640 1 -- 172.21.15.139:0/60019321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdac06ab60 msgr2=0x7fcdac06afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.540+0000 7fcdb214c640 1 --2- 172.21.15.139:0/60019321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdac06ab60 0x7fcdac06afc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0d16d0 tx=0x7fcda002a270 comp rx=0 tx=0).stop 2024-04-25T09:57:04.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.540+0000 7fcdb214c640 1 -- 172.21.15.139:0/60019321 shutdown_connections 2024-04-25T09:57:04.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.540+0000 7fcdb214c640 1 --2- 172.21.15.139:0/60019321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdac06ab60 0x7fcdac06afc0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.540+0000 7fcdb214c640 1 --2- 172.21.15.139:0/60019321 >> v2:172.21.15.150:3300/0 conn(0x7fcdac15b950 0x7fcdac072a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.540+0000 7fcdb214c640 1 -- 172.21.15.139:0/60019321 >> 172.21.15.139:0/60019321 conn(0x7fcdac0b23f0 msgr2=0x7fcdac0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.540+0000 7fcdb214c640 1 -- 172.21.15.139:0/60019321 shutdown_connections 2024-04-25T09:57:04.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.540+0000 7fcdb214c640 1 -- 172.21.15.139:0/60019321 wait complete. 2024-04-25T09:57:04.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.541+0000 7fcdb214c640 1 Processor -- start 2024-04-25T09:57:04.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.541+0000 7fcdb214c640 1 -- start start 2024-04-25T09:57:04.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdb214c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdac06ab60 0x7fcdac1f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdb214c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac15b950 0x7fcdac144f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdb214c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdac15f580 0x7fcdac1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdb214c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0cdce0 con 0x7fcdac15f580 2024-04-25T09:57:04.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdb214c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0cda00 con 0x7fcdac06ab60 2024-04-25T09:57:04.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdb214c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0cdfc0 con 0x7fcdac15b950 2024-04-25T09:57:04.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdabfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdac15f580 0x7fcdac1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdabfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdac15f580 0x7fcdac1590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49746/0 (socket says 172.21.15.139:49746) 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdabfff640 1 -- 172.21.15.139:0/2245449829 learned_addr learned my addr 172.21.15.139:0/2245449829 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdab7fe640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdac06ab60 0x7fcdac1f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdaaffd640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac15b950 0x7fcdac144f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdabfff640 1 -- 172.21.15.139:0/2245449829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac15b950 msgr2=0x7fcdac144f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdabfff640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac15b950 0x7fcdac144f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdabfff640 1 -- 172.21.15.139:0/2245449829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdac06ab60 msgr2=0x7fcdac1f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdabfff640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdac06ab60 0x7fcdac1f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdabfff640 1 -- 172.21.15.139:0/2245449829 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac1f5aa0 con 0x7fcdac15f580 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdaaffd640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac15b950 0x7fcdac144f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdabfff640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdac15f580 0x7fcdac1590e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcda400c6b0 tx=0x7fcda400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcd93fff640 1 -- 172.21.15.139:0/2245449829 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4003b70 con 0x7fcdac15f580 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcd93fff640 1 -- 172.21.15.139:0/2245449829 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda4003d30 con 0x7fcdac15f580 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdb214c640 1 -- 172.21.15.139:0/2245449829 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac15d010 con 0x7fcdac15f580 2024-04-25T09:57:04.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcd93fff640 1 -- 172.21.15.139:0/2245449829 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4005310 con 0x7fcdac15f580 2024-04-25T09:57:04.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.542+0000 7fcdb214c640 1 -- 172.21.15.139:0/2245449829 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac15d2b0 con 0x7fcdac15f580 2024-04-25T09:57:04.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.543+0000 7fcdb214c640 1 -- 172.21.15.139:0/2245449829 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdac074520 con 0x7fcdac15f580 2024-04-25T09:57:04.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.544+0000 7fcd93fff640 1 -- 172.21.15.139:0/2245449829 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcda40054b0 con 0x7fcdac15f580 2024-04-25T09:57:04.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.544+0000 7fcd93fff640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd9806b4d0 0x7fcd9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.544+0000 7fcd93fff640 1 -- 172.21.15.139:0/2245449829 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fcda408a160 con 0x7fcdac15f580 2024-04-25T09:57:04.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.544+0000 7fcdab7fe640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd9806b4d0 0x7fcd9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.545+0000 7fcdab7fe640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd9806b4d0 0x7fcd9806d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c006fd0 tx=0x7fcd9c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.548+0000 7fcd93fff640 1 -- 172.21.15.139:0/2245449829 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdac1f5aa0 con 0x7fcdac15f580 2024-04-25T09:57:04.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.755+0000 7fcdb214c640 1 -- 172.21.15.139:0/2245449829 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcdac0af300 con 0x7fcdac15f580 2024-04-25T09:57:04.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.755+0000 7fcd93fff640 1 -- 172.21.15.139:0/2245449829 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fcda4058e50 con 0x7fcdac15f580 2024-04-25T09:57:04.758 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:04.759 INFO:teuthology.orchestra.run.smithi139.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T09:57:04.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.757+0000 7fcdb214c640 1 -- 172.21.15.139:0/2245449829 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd9806b4d0 msgr2=0x7fcd9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.757+0000 7fcdb214c640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd9806b4d0 0x7fcd9806d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c006fd0 tx=0x7fcd9c008040 comp rx=0 tx=0).stop 2024-04-25T09:57:04.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.757+0000 7fcdb214c640 1 -- 172.21.15.139:0/2245449829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdac15f580 msgr2=0x7fcdac1590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.757+0000 7fcdb214c640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdac15f580 0x7fcdac1590e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcda400c6b0 tx=0x7fcda400cb70 comp rx=0 tx=0).stop 2024-04-25T09:57:04.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.757+0000 7fcdb214c640 1 -- 172.21.15.139:0/2245449829 shutdown_connections 2024-04-25T09:57:04.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.757+0000 7fcdb214c640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdac15f580 0x7fcdac1590e0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.757+0000 7fcdb214c640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd9806b4d0 0x7fcd9806d990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.757+0000 7fcdb214c640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac15b950 0x7fcdac144f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.757+0000 7fcdb214c640 1 --2- 172.21.15.139:0/2245449829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdac06ab60 0x7fcdac1f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.757+0000 7fcdb214c640 1 -- 172.21.15.139:0/2245449829 >> 172.21.15.139:0/2245449829 conn(0x7fcdac0b23f0 msgr2=0x7fcdac155eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.757+0000 7fcdb214c640 1 -- 172.21.15.139:0/2245449829 shutdown_connections 2024-04-25T09:57:04.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:04.757+0000 7fcdb214c640 1 -- 172.21.15.139:0/2245449829 wait complete. 2024-04-25T09:57:04.771 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T09:57:04.771 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:57:04.772 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:04.814 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:04.871 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:57:04.871 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:57:04.871 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:04.886 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:57:04.886 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:04.941 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:05.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.076+0000 7fb91d7e0640 1 Processor -- start 2024-04-25T09:57:05.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.076+0000 7fb91d7e0640 1 -- start start 2024-04-25T09:57:05.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.076+0000 7fb91d7e0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb91815dc60 0x7fb9180b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.076+0000 7fb91d7e0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb9180d1080 0x7fb9180ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.076+0000 7fb91d7e0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9180cf250 con 0x7fb918161850 2024-04-25T09:57:05.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.076+0000 7fb91d7e0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9180ce780 con 0x7fb91815dc60 2024-04-25T09:57:05.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.076+0000 7fb91d7e0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9180d3730 con 0x7fb9180d1080 2024-04-25T09:57:05.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.077+0000 7fb916ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb91815dc60 0x7fb9180b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.077+0000 7fb916ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb91815dc60 0x7fb9180b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49754/0 (socket says 172.21.15.139:49754) 2024-04-25T09:57:05.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.077+0000 7fb9167fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb918161850 0x7fb9180d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37532/0 (socket says 172.21.15.139:37532) 2024-04-25T09:57:05.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.077+0000 7fb916ffd640 1 -- 172.21.15.139:0/3601269757 learned_addr learned my addr 172.21.15.139:0/3601269757 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:05.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.077+0000 7fb916ffd640 1 -- 172.21.15.139:0/3601269757 >> v2:172.21.15.150:3300/0 conn(0x7fb9180d1080 msgr2=0x7fb9180ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.077+0000 7fb916ffd640 1 --2- 172.21.15.139:0/3601269757 >> v2:172.21.15.150:3300/0 conn(0x7fb9180d1080 0x7fb9180ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.077+0000 7fb916ffd640 1 -- 172.21.15.139:0/3601269757 >> v1:172.21.15.139:6789/0 conn(0x7fb918161850 legacy=0x7fb9180d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.077+0000 7fb915ffb640 1 -- 172.21.15.139:0/3601269757 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807837044 0 0) 0x7fb900001110 con 0x7fb918161850 2024-04-25T09:57:05.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.077+0000 7fb916ffd640 1 -- 172.21.15.139:0/3601269757 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9180ce9b0 con 0x7fb91815dc60 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.078+0000 7fb916ffd640 1 --2- 172.21.15.139:0/3601269757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb91815dc60 0x7fb9180b6c10 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb9180d1560 tx=0x7fb91002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cc71bc7f4fc3d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.078+0000 7fb915ffb640 1 -- 172.21.15.139:0/3601269757 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910036070 con 0x7fb91815dc60 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.078+0000 7fb915ffb640 1 -- 172.21.15.139:0/3601269757 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb910031600 con 0x7fb91815dc60 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.078+0000 7fb915ffb640 1 -- 172.21.15.139:0/3601269757 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9100319d0 con 0x7fb91815dc60 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.078+0000 7fb91d7e0640 1 -- 172.21.15.139:0/3601269757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb91815dc60 msgr2=0x7fb9180b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.078+0000 7fb91d7e0640 1 --2- 172.21.15.139:0/3601269757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb91815dc60 0x7fb9180b6c10 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb9180d1560 tx=0x7fb91002a750 comp rx=0 tx=0).stop 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.078+0000 7fb91d7e0640 1 -- 172.21.15.139:0/3601269757 shutdown_connections 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.078+0000 7fb91d7e0640 1 --2- 172.21.15.139:0/3601269757 >> v2:172.21.15.150:3300/0 conn(0x7fb9180d1080 0x7fb9180ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.078+0000 7fb91d7e0640 1 --2- 172.21.15.139:0/3601269757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb91815dc60 0x7fb9180b6c10 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.078+0000 7fb91d7e0640 1 -- 172.21.15.139:0/3601269757 >> 172.21.15.139:0/3601269757 conn(0x7fb9180b5910 msgr2=0x7fb9180b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.078+0000 7fb91d7e0640 1 -- 172.21.15.139:0/3601269757 shutdown_connections 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.078+0000 7fb91d7e0640 1 -- 172.21.15.139:0/3601269757 wait complete. 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.080+0000 7fb91d7e0640 1 Processor -- start 2024-04-25T09:57:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.080+0000 7fb91d7e0640 1 -- start start 2024-04-25T09:57:05.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.080+0000 7fb91d7e0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9180d1080 0x7fb918152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.080+0000 7fb91d7e0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb91815dc60 0x7fb9180b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.080+0000 7fb91d7e0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb918161850 0x7fb9180c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.080+0000 7fb91d7e0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9180d1fc0 con 0x7fb918161850 2024-04-25T09:57:05.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.080+0000 7fb91d7e0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9180cef70 con 0x7fb91815dc60 2024-04-25T09:57:05.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.080+0000 7fb91d7e0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91815e040 con 0x7fb9180d1080 2024-04-25T09:57:05.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.080+0000 7fb9177fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb918161850 0x7fb9180c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.080+0000 7fb9177fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb918161850 0x7fb9180c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49766/0 (socket says 172.21.15.139:49766) 2024-04-25T09:57:05.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.080+0000 7fb9177fe640 1 -- 172.21.15.139:0/4208056189 learned_addr learned my addr 172.21.15.139:0/4208056189 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb9167fc640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb91815dc60 0x7fb9180b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb916ffd640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9180d1080 0x7fb918152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb9177fe640 1 -- 172.21.15.139:0/4208056189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9180d1080 msgr2=0x7fb918152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb9177fe640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9180d1080 0x7fb918152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb9177fe640 1 -- 172.21.15.139:0/4208056189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb91815dc60 msgr2=0x7fb9180b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb9177fe640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb91815dc60 0x7fb9180b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb9177fe640 1 -- 172.21.15.139:0/4208056189 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918152330 con 0x7fb918161850 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb916ffd640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9180d1080 0x7fb918152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb9177fe640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb918161850 0x7fb9180c6190 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b410 tx=0x7fb90800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb90f7fe640 1 -- 172.21.15.139:0/4208056189 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb908015020 con 0x7fb918161850 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb90f7fe640 1 -- 172.21.15.139:0/4208056189 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb908011850 con 0x7fb918161850 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb90f7fe640 1 -- 172.21.15.139:0/4208056189 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb908011c40 con 0x7fb918161850 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb91d7e0640 1 -- 172.21.15.139:0/4208056189 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9180c9b00 con 0x7fb918161850 2024-04-25T09:57:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.081+0000 7fb91d7e0640 1 -- 172.21.15.139:0/4208056189 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9181528d0 con 0x7fb918161850 2024-04-25T09:57:05.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.082+0000 7fb91d7e0640 1 -- 172.21.15.139:0/4208056189 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb918074520 con 0x7fb918161850 2024-04-25T09:57:05.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.083+0000 7fb90f7fe640 1 -- 172.21.15.139:0/4208056189 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb908018020 con 0x7fb918161850 2024-04-25T09:57:05.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.083+0000 7fb90f7fe640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb8f806b4d0 0x7fb8f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.083+0000 7fb90f7fe640 1 -- 172.21.15.139:0/4208056189 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fb90808a2e0 con 0x7fb918161850 2024-04-25T09:57:05.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.083+0000 7fb916ffd640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb8f806b4d0 0x7fb8f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.083+0000 7fb916ffd640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb8f806b4d0 0x7fb8f806d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb910033000 tx=0x7fb910009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.087+0000 7fb90f7fe640 1 -- 172.21.15.139:0/4208056189 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb90801f020 con 0x7fb918161850 2024-04-25T09:57:05.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.258+0000 7fb91d7e0640 1 -- 172.21.15.139:0/4208056189 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb918152aa0 con 0x7fb918161850 2024-04-25T09:57:05.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.260+0000 7fb90f7fe640 1 -- 172.21.15.139:0/4208056189 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17173 (secure 0 0 0) 0x7fb9180c9b00 con 0x7fb918161850 2024-04-25T09:57:05.264 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:05.264 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":64,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:55:57.317641+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:48.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:49.597505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.22","mappings":[{"from":7,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:05.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.261+0000 7fb91d7e0640 1 -- 172.21.15.139:0/4208056189 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb8f806b4d0 msgr2=0x7fb8f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.261+0000 7fb91d7e0640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb8f806b4d0 0x7fb8f806d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb910033000 tx=0x7fb910009000 comp rx=0 tx=0).stop 2024-04-25T09:57:05.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.261+0000 7fb91d7e0640 1 -- 172.21.15.139:0/4208056189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb918161850 msgr2=0x7fb9180c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.261+0000 7fb91d7e0640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb918161850 0x7fb9180c6190 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b410 tx=0x7fb90800b8d0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.262+0000 7fb91d7e0640 1 -- 172.21.15.139:0/4208056189 shutdown_connections 2024-04-25T09:57:05.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.262+0000 7fb91d7e0640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb918161850 0x7fb9180c6190 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.262+0000 7fb91d7e0640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb8f806b4d0 0x7fb8f806d990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.262+0000 7fb91d7e0640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb91815dc60 0x7fb9180b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.262+0000 7fb91d7e0640 1 --2- 172.21.15.139:0/4208056189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9180d1080 0x7fb918152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.262+0000 7fb91d7e0640 1 -- 172.21.15.139:0/4208056189 >> 172.21.15.139:0/4208056189 conn(0x7fb9180b5910 msgr2=0x7fb918153600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:05.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.262+0000 7fb91d7e0640 1 -- 172.21.15.139:0/4208056189 shutdown_connections 2024-04-25T09:57:05.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.262+0000 7fb91d7e0640 1 -- 172.21.15.139:0/4208056189 wait complete. 2024-04-25T09:57:05.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:55:48.823437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:55:49.597505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:05.276 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:05.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.373+0000 7efda293b640 1 Processor -- start 2024-04-25T09:57:05.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.373+0000 7efda293b640 1 -- start start 2024-04-25T09:57:05.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.374+0000 7efda293b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efd9c15dc90 0x7efd9c0ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.374+0000 7efda293b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c06f300 0x7efd9c0b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.374+0000 7efda293b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9c0cf980 con 0x7efd9c0cf3e0 2024-04-25T09:57:05.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.374+0000 7efda293b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9c06ab60 con 0x7efd9c15dc90 2024-04-25T09:57:05.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.374+0000 7efda293b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c06ae40 con 0x7efd9c06f300 2024-04-25T09:57:05.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.374+0000 7efd9b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7efd9c0cf3e0 0x7efd9c06ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37548/0 (socket says 172.21.15.139:37548) 2024-04-25T09:57:05.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.374+0000 7efd9b7fe640 1 -- 172.21.15.139:0/132793421 learned_addr learned my addr 172.21.15.139:0/132793421 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:05.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.374+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656261098 0 0) 0x7efd9c0cf980 con 0x7efd9c0cf3e0 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.374+0000 7efda0ed8640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c06f300 0x7efd9c0b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.374+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd84003520 con 0x7efd9c0cf3e0 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.374+0000 7efd9bfff640 1 --2- 172.21.15.139:0/132793421 >> v2:172.21.15.150:3300/0 conn(0x7efd9c15dc90 0x7efd9c0ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3734725285 0 0) 0x7efd84003520 con 0x7efd9c0cf3e0 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c06f300 msgr2=0x7efd9c0b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c06f300 0x7efd9c0b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 >> v2:172.21.15.150:3300/0 conn(0x7efd9c15dc90 msgr2=0x7efd9c0ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 --2- 172.21.15.139:0/132793421 >> v2:172.21.15.150:3300/0 conn(0x7efd9c15dc90 0x7efd9c0ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c06b120 con 0x7efd9c0cf3e0 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7efd90001690 con 0x7efd9c0cf3e0 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 >> v1:172.21.15.139:6789/0 conn(0x7efd9c0cf3e0 legacy=0x7efd9c06ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c06f300 0x7efd840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd84003f80 0x7efd84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd84004d40 0x7efd840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c06ab60 con 0x7efd84004d40 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c06ae40 con 0x7efd84003f80 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8400b9c0 con 0x7efd9c06f300 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efda0ed8640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd84004d40 0x7efd840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9b7fe640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd84003f80 0x7efd84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9bfff640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c06f300 0x7efd840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efda0ed8640 1 -- 172.21.15.139:0/132793421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c06f300 msgr2=0x7efd840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efda0ed8640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c06f300 0x7efd840039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efda0ed8640 1 -- 172.21.15.139:0/132793421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd84003f80 msgr2=0x7efd84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efda0ed8640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd84003f80 0x7efd84004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efda0ed8640 1 -- 172.21.15.139:0/132793421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd84003520 con 0x7efd84004d40 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.375+0000 7efd9bfff640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c06f300 0x7efd840039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:05.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.376+0000 7efda0ed8640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd84004d40 0x7efd840051c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7efd9400e170 tx=0x7efd84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7db2e05edebbd3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.376+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94034020 con 0x7efd84004d40 2024-04-25T09:57:05.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.376+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd940305f0 con 0x7efd84004d40 2024-04-25T09:57:05.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.376+0000 7efd9affd640 1 -- 172.21.15.139:0/132793421 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94009c10 con 0x7efd84004d40 2024-04-25T09:57:05.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.376+0000 7efda293b640 1 -- 172.21.15.139:0/132793421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd84004d40 msgr2=0x7efd840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.376+0000 7efda293b640 1 --2- 172.21.15.139:0/132793421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd84004d40 0x7efd840051c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7efd9400e170 tx=0x7efd84005830 comp rx=0 tx=0).stop 2024-04-25T09:57:05.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.376+0000 7efda0ed8640 1 -- 172.21.15.139:0/132793421 reap_dead start 2024-04-25T09:57:05.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.376+0000 7efda293b640 1 -- 172.21.15.139:0/132793421 shutdown_connections 2024-04-25T09:57:05.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.377+0000 7efda293b640 1 -- 172.21.15.139:0/132793421 >> 172.21.15.139:0/132793421 conn(0x7efd9c0cfd80 msgr2=0x7efd9c0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:05.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.377+0000 7efda293b640 1 -- 172.21.15.139:0/132793421 shutdown_connections 2024-04-25T09:57:05.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.377+0000 7efda293b640 1 -- 172.21.15.139:0/132793421 wait complete. 2024-04-25T09:57:05.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.378+0000 7efda293b640 1 Processor -- start 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.378+0000 7efda293b640 1 -- start start 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda293b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd84004d40 0x7efd9c160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda293b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c0c5e60 0x7efd9c15f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda293b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd9c1f4300 0x7efd9c0c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda293b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c06b920 con 0x7efd9c1f4300 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda293b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c072620 con 0x7efd84004d40 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda293b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c0b2db0 con 0x7efd9c0c5e60 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda0ed8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd9c1f4300 0x7efd9c0c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda0ed8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd9c1f4300 0x7efd9c0c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49780/0 (socket says 172.21.15.139:49780) 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda0ed8640 1 -- 172.21.15.139:0/1846058103 learned_addr learned my addr 172.21.15.139:0/1846058103 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efd9b7fe640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c0c5e60 0x7efd9c15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda0ed8640 1 -- 172.21.15.139:0/1846058103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c0c5e60 msgr2=0x7efd9c15f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efd9bfff640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd84004d40 0x7efd9c160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda0ed8640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c0c5e60 0x7efd9c15f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda0ed8640 1 -- 172.21.15.139:0/1846058103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd84004d40 msgr2=0x7efd9c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda0ed8640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd84004d40 0x7efd9c160180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda0ed8640 1 -- 172.21.15.139:0/1846058103 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c0c4dc0 con 0x7efd9c1f4300 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda0ed8640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd9c1f4300 0x7efd9c0c6480 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7efd94009f90 tx=0x7efd94009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efd98ff9640 1 -- 172.21.15.139:0/1846058103 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94034040 con 0x7efd9c1f4300 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efd98ff9640 1 -- 172.21.15.139:0/1846058103 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd94003990 con 0x7efd9c1f4300 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda293b640 1 -- 172.21.15.139:0/1846058103 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c0c50a0 con 0x7efd9c1f4300 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efd98ff9640 1 -- 172.21.15.139:0/1846058103 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94009c10 con 0x7efd9c1f4300 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.379+0000 7efda293b640 1 -- 172.21.15.139:0/1846058103 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9c0c5660 con 0x7efd9c1f4300 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.380+0000 7efda293b640 1 -- 172.21.15.139:0/1846058103 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9c074520 con 0x7efd9c1f4300 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.381+0000 7efd98ff9640 1 -- 172.21.15.139:0/1846058103 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd94037020 con 0x7efd9c1f4300 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.381+0000 7efd98ff9640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd8806b4d0 0x7efd8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.381+0000 7efd98ff9640 1 -- 172.21.15.139:0/1846058103 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7efd940ab050 con 0x7efd9c1f4300 2024-04-25T09:57:05.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.381+0000 7efd9bfff640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd8806b4d0 0x7efd8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.381+0000 7efd9bfff640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd8806b4d0 0x7efd8806d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0099b0 tx=0x7efd8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.385+0000 7efd98ff9640 1 -- 172.21.15.139:0/1846058103 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd94049060 con 0x7efd9c1f4300 2024-04-25T09:57:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.554+0000 7efda293b640 1 -- 172.21.15.139:0/1846058103 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd9c0d0290 con 0x7efd9c1f4300 2024-04-25T09:57:05.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.555+0000 7efd98ff9640 1 -- 172.21.15.139:0/1846058103 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd94079cb0 con 0x7efd9c1f4300 2024-04-25T09:57:05.558 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:57:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.557+0000 7efda293b640 1 -- 172.21.15.139:0/1846058103 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd8806b4d0 msgr2=0x7efd8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.557+0000 7efda293b640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd8806b4d0 0x7efd8806d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0099b0 tx=0x7efd8c008040 comp rx=0 tx=0).stop 2024-04-25T09:57:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.557+0000 7efda293b640 1 -- 172.21.15.139:0/1846058103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd9c1f4300 msgr2=0x7efd9c0c6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.557+0000 7efda293b640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd9c1f4300 0x7efd9c0c6480 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7efd94009f90 tx=0x7efd94009750 comp rx=0 tx=0).stop 2024-04-25T09:57:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.557+0000 7efda293b640 1 -- 172.21.15.139:0/1846058103 shutdown_connections 2024-04-25T09:57:05.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.557+0000 7efda293b640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd9c1f4300 0x7efd9c0c6480 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.557+0000 7efda293b640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd8806b4d0 0x7efd8806d990 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.557+0000 7efda293b640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd9c0c5e60 0x7efd9c15f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.557+0000 7efda293b640 1 --2- 172.21.15.139:0/1846058103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd84004d40 0x7efd9c160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.557+0000 7efda293b640 1 -- 172.21.15.139:0/1846058103 >> 172.21.15.139:0/1846058103 conn(0x7efd9c0cfd80 msgr2=0x7efd9c0b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.557+0000 7efda293b640 1 -- 172.21.15.139:0/1846058103 shutdown_connections 2024-04-25T09:57:05.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.557+0000 7efda293b640 1 -- 172.21.15.139:0/1846058103 wait complete. 2024-04-25T09:57:05.571 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:57:05.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.667+0000 7fca3e710640 1 Processor -- start 2024-04-25T09:57:05.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.667+0000 7fca3e710640 1 -- start start 2024-04-25T09:57:05.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.667+0000 7fca3e710640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fca380cd610 0x7fca380d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.667+0000 7fca3e710640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fca3809ad40 0x7fca380d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca3e710640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca381618c0 0x7fca38161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca3e710640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca38099a50 con 0x7fca380cd610 2024-04-25T09:57:05.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca3e710640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca38099ca0 con 0x7fca3809ad40 2024-04-25T09:57:05.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca3e710640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca38099f80 con 0x7fca381618c0 2024-04-25T09:57:05.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca37fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fca380cd610 0x7fca380d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca37fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fca380cd610 0x7fca380d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49790/0 (socket says 172.21.15.139:49790) 2024-04-25T09:57:05.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca37fff640 1 -- 172.21.15.139:0/3044355412 learned_addr learned my addr 172.21.15.139:0/3044355412 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:05.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca3ccad640 1 --2- 172.21.15.139:0/3044355412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca381618c0 0x7fca38161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca377fe640 1 --2- 172.21.15.139:0/3044355412 >> v2:172.21.15.150:3300/0 conn(0x7fca3809ad40 0x7fca380d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca37fff640 1 -- 172.21.15.139:0/3044355412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca381618c0 msgr2=0x7fca38161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca37fff640 1 --2- 172.21.15.139:0/3044355412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca381618c0 0x7fca38161d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca37fff640 1 -- 172.21.15.139:0/3044355412 >> v2:172.21.15.150:3300/0 conn(0x7fca3809ad40 msgr2=0x7fca380d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca37fff640 1 --2- 172.21.15.139:0/3044355412 >> v2:172.21.15.150:3300/0 conn(0x7fca3809ad40 0x7fca380d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca37fff640 1 -- 172.21.15.139:0/3044355412 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3809a260 con 0x7fca380cd610 2024-04-25T09:57:05.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca377fe640 1 --2- 172.21.15.139:0/3044355412 >> v2:172.21.15.150:3300/0 conn(0x7fca3809ad40 0x7fca380d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:05.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.668+0000 7fca3ccad640 1 --2- 172.21.15.139:0/3044355412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca381618c0 0x7fca38161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:05.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca37fff640 1 --2- 172.21.15.139:0/3044355412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca380cd610 0x7fca380d0370 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fca380725c0 tx=0x7fca2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9148d1b0428e93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca36ffd640 1 -- 172.21.15.139:0/3044355412 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28037070 con 0x7fca380cd610 2024-04-25T09:57:05.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca36ffd640 1 -- 172.21.15.139:0/3044355412 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca28032440 con 0x7fca380cd610 2024-04-25T09:57:05.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca36ffd640 1 -- 172.21.15.139:0/3044355412 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28032810 con 0x7fca380cd610 2024-04-25T09:57:05.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca3e710640 1 -- 172.21.15.139:0/3044355412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca380cd610 msgr2=0x7fca380d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca3e710640 1 --2- 172.21.15.139:0/3044355412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca380cd610 0x7fca380d0370 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fca380725c0 tx=0x7fca2802a270 comp rx=0 tx=0).stop 2024-04-25T09:57:05.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca3e710640 1 -- 172.21.15.139:0/3044355412 shutdown_connections 2024-04-25T09:57:05.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca3e710640 1 --2- 172.21.15.139:0/3044355412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca381618c0 0x7fca38161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca3e710640 1 --2- 172.21.15.139:0/3044355412 >> v2:172.21.15.150:3300/0 conn(0x7fca3809ad40 0x7fca380d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca3e710640 1 --2- 172.21.15.139:0/3044355412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca380cd610 0x7fca380d0370 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca3e710640 1 -- 172.21.15.139:0/3044355412 >> 172.21.15.139:0/3044355412 conn(0x7fca380b31f0 msgr2=0x7fca380b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:05.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca3e710640 1 -- 172.21.15.139:0/3044355412 shutdown_connections 2024-04-25T09:57:05.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.669+0000 7fca3e710640 1 -- 172.21.15.139:0/3044355412 wait complete. 2024-04-25T09:57:05.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.671+0000 7fca3e710640 1 Processor -- start 2024-04-25T09:57:05.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.671+0000 7fca3e710640 1 -- start start 2024-04-25T09:57:05.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.671+0000 7fca3e710640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca3809ad40 0x7fca3814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.671+0000 7fca3e710640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca380cd610 0x7fca380aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.671+0000 7fca3e710640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca381618c0 0x7fca380af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.671+0000 7fca3e710640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca380cd9f0 con 0x7fca380cd610 2024-04-25T09:57:05.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.671+0000 7fca3e710640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3809a820 con 0x7fca381618c0 2024-04-25T09:57:05.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.671+0000 7fca3e710640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca380cdb70 con 0x7fca3809ad40 2024-04-25T09:57:05.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.671+0000 7fca377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca380cd610 0x7fca380aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca380cd610 0x7fca380aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49802/0 (socket says 172.21.15.139:49802) 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca377fe640 1 -- 172.21.15.139:0/1441425110 learned_addr learned my addr 172.21.15.139:0/1441425110 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca377fe640 1 -- 172.21.15.139:0/1441425110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca3809ad40 msgr2=0x7fca3814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca377fe640 1 --2- 172.21.15.139:0/1441425110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca3809ad40 0x7fca3814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca377fe640 1 -- 172.21.15.139:0/1441425110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca381618c0 msgr2=0x7fca380af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca377fe640 1 --2- 172.21.15.139:0/1441425110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca381618c0 0x7fca380af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca377fe640 1 -- 172.21.15.139:0/1441425110 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380b4fe0 con 0x7fca380cd610 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca377fe640 1 --2- 172.21.15.139:0/1441425110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca380cd610 0x7fca380aef80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00e810 tx=0x7fca2c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca34ff9640 1 -- 172.21.15.139:0/1441425110 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c00f040 con 0x7fca380cd610 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca34ff9640 1 -- 172.21.15.139:0/1441425110 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c004030 con 0x7fca380cd610 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca3e710640 1 -- 172.21.15.139:0/1441425110 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca380b52c0 con 0x7fca380cd610 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca34ff9640 1 -- 172.21.15.139:0/1441425110 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c004400 con 0x7fca380cd610 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.672+0000 7fca3e710640 1 -- 172.21.15.139:0/1441425110 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca380b5880 con 0x7fca380cd610 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.673+0000 7fca3e710640 1 -- 172.21.15.139:0/1441425110 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca38074520 con 0x7fca380cd610 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.673+0000 7fca34ff9640 1 -- 172.21.15.139:0/1441425110 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca2c00c6d0 con 0x7fca380cd610 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.674+0000 7fca34ff9640 1 --2- 172.21.15.139:0/1441425110 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca2406b500 0x7fca2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.674+0000 7fca37fff640 1 --2- 172.21.15.139:0/1441425110 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca2406b500 0x7fca2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.674+0000 7fca34ff9640 1 -- 172.21.15.139:0/1441425110 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fca2c08a050 con 0x7fca380cd610 2024-04-25T09:57:05.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.674+0000 7fca37fff640 1 --2- 172.21.15.139:0/1441425110 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca2406b500 0x7fca2406d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fca28034000 tx=0x7fca2802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.678+0000 7fca34ff9640 1 -- 172.21.15.139:0/1441425110 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca2c017020 con 0x7fca380cd610 2024-04-25T09:57:05.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.847+0000 7fca3e710640 1 -- 172.21.15.139:0/1441425110 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca380b3f70 con 0x7fca380cd610 2024-04-25T09:57:05.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.847+0000 7fca34ff9640 1 -- 172.21.15.139:0/1441425110 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+11 (secure 0 0 0) 0x7fca2c058d40 con 0x7fca380cd610 2024-04-25T09:57:05.850 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:57:05.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.849+0000 7fca3e710640 1 -- 172.21.15.139:0/1441425110 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca2406b500 msgr2=0x7fca2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.849+0000 7fca3e710640 1 --2- 172.21.15.139:0/1441425110 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca2406b500 0x7fca2406d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fca28034000 tx=0x7fca2802c000 comp rx=0 tx=0).stop 2024-04-25T09:57:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.849+0000 7fca3e710640 1 -- 172.21.15.139:0/1441425110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca380cd610 msgr2=0x7fca380aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.849+0000 7fca3e710640 1 --2- 172.21.15.139:0/1441425110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca380cd610 0x7fca380aef80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00e810 tx=0x7fca2c00ecd0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.849+0000 7fca3e710640 1 -- 172.21.15.139:0/1441425110 shutdown_connections 2024-04-25T09:57:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.849+0000 7fca3e710640 1 --2- 172.21.15.139:0/1441425110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca380cd610 0x7fca380aef80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.849+0000 7fca3e710640 1 --2- 172.21.15.139:0/1441425110 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca2406b500 0x7fca2406d9c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.849+0000 7fca3e710640 1 --2- 172.21.15.139:0/1441425110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca381618c0 0x7fca380af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.849+0000 7fca3e710640 1 --2- 172.21.15.139:0/1441425110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca3809ad40 0x7fca3814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.849+0000 7fca3e710640 1 -- 172.21.15.139:0/1441425110 >> 172.21.15.139:0/1441425110 conn(0x7fca380b31f0 msgr2=0x7fca380b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.849+0000 7fca3e710640 1 -- 172.21.15.139:0/1441425110 shutdown_connections 2024-04-25T09:57:05.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.849+0000 7fca3e710640 1 -- 172.21.15.139:0/1441425110 wait complete. 2024-04-25T09:57:05.863 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:57:05.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.959+0000 7f14d777e640 1 Processor -- start 2024-04-25T09:57:05.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.959+0000 7f14d777e640 1 -- start start 2024-04-25T09:57:05.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.960+0000 7f14d777e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14d015bf90 0x7f14d00cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.960+0000 7f14d777e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d015c490 con 0x7f14d00ce040 2024-04-25T09:57:05.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.960+0000 7f14d777e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d015c770 con 0x7f14d0071a50 2024-04-25T09:57:05.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.960+0000 7f14d777e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d015ca50 con 0x7f14d015bf90 2024-04-25T09:57:05.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.960+0000 7f14d4d19640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14d015bf90 0x7f14d00cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.960+0000 7f14d5d1b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f14d00ce040 0x7f14d00ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37560/0 (socket says 172.21.15.139:37560) 2024-04-25T09:57:05.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.960+0000 7f14d4d19640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14d015bf90 0x7f14d00cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49812/0 (socket says 172.21.15.139:49812) 2024-04-25T09:57:05.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.960+0000 7f14d5d1b640 1 -- 172.21.15.139:0/3338827297 learned_addr learned my addr 172.21.15.139:0/3338827297 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:05.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.960+0000 7f14d4d19640 1 -- 172.21.15.139:0/3338827297 >> v1:172.21.15.150:6789/0 conn(0x7f14d0071a50 legacy=0x7f14d0071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.960+0000 7f14d4d19640 1 -- 172.21.15.139:0/3338827297 >> v1:172.21.15.139:6789/0 conn(0x7f14d00ce040 legacy=0x7f14d00ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.960+0000 7f14d4d19640 1 -- 172.21.15.139:0/3338827297 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d015cd30 con 0x7f14d015bf90 2024-04-25T09:57:05.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.960+0000 7f14c7fff640 1 -- 172.21.15.139:0/3338827297 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540793758 0 0) 0x7f14d015c490 con 0x7f14d00ce040 2024-04-25T09:57:05.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.961+0000 7f14d4d19640 1 --2- 172.21.15.139:0/3338827297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d015bf90 0x7f14d00cc2c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f14c00097a0 tx=0x7f14c002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66170a5d4cf846d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.961+0000 7f14c7fff640 1 -- 172.21.15.139:0/3338827297 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0037070 con 0x7f14d015bf90 2024-04-25T09:57:05.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.961+0000 7f14c7fff640 1 -- 172.21.15.139:0/3338827297 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c002ae50 con 0x7f14d015bf90 2024-04-25T09:57:05.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.961+0000 7f14c7fff640 1 -- 172.21.15.139:0/3338827297 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0032660 con 0x7f14d015bf90 2024-04-25T09:57:05.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.961+0000 7f14d777e640 1 -- 172.21.15.139:0/3338827297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d015bf90 msgr2=0x7f14d00cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.961+0000 7f14d777e640 1 --2- 172.21.15.139:0/3338827297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d015bf90 0x7f14d00cc2c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f14c00097a0 tx=0x7f14c002a140 comp rx=0 tx=0).stop 2024-04-25T09:57:05.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.962+0000 7f14d777e640 1 -- 172.21.15.139:0/3338827297 shutdown_connections 2024-04-25T09:57:05.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.962+0000 7f14d777e640 1 --2- 172.21.15.139:0/3338827297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d015bf90 0x7f14d00cc2c0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.962+0000 7f14d777e640 1 -- 172.21.15.139:0/3338827297 >> 172.21.15.139:0/3338827297 conn(0x7f14d00cfd80 msgr2=0x7f14d00d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:05.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.962+0000 7f14d777e640 1 -- 172.21.15.139:0/3338827297 shutdown_connections 2024-04-25T09:57:05.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.962+0000 7f14d777e640 1 -- 172.21.15.139:0/3338827297 wait complete. 2024-04-25T09:57:05.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.963+0000 7f14d777e640 1 Processor -- start 2024-04-25T09:57:05.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.963+0000 7f14d777e640 1 -- start start 2024-04-25T09:57:05.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.963+0000 7f14d777e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14d0071a50 0x7f14d015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.963+0000 7f14d777e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d00ce040 0x7f14d009b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.963+0000 7f14d777e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14d015bf90 0x7f14d006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.963+0000 7f14d777e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0156280 con 0x7f14d00ce040 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.963+0000 7f14d777e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00728b0 con 0x7f14d0071a50 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.963+0000 7f14d777e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0156400 con 0x7f14d015bf90 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d4d19640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d00ce040 0x7f14d009b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d4d19640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d00ce040 0x7f14d009b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49818/0 (socket says 172.21.15.139:49818) 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d4d19640 1 -- 172.21.15.139:0/4133438702 learned_addr learned my addr 172.21.15.139:0/4133438702 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d5d1b640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14d015bf90 0x7f14d006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d551a640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14d0071a50 0x7f14d015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d4d19640 1 -- 172.21.15.139:0/4133438702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14d015bf90 msgr2=0x7f14d006ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d4d19640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14d015bf90 0x7f14d006ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d4d19640 1 -- 172.21.15.139:0/4133438702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14d0071a50 msgr2=0x7f14d015ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d4d19640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14d0071a50 0x7f14d015ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d4d19640 1 -- 172.21.15.139:0/4133438702 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d009c2f0 con 0x7f14d00ce040 2024-04-25T09:57:05.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d551a640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14d0071a50 0x7f14d015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:05.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d4d19640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d00ce040 0x7f14d009b850 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f14c0035040 tx=0x7f14c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14c5ffb640 1 -- 172.21.15.139:0/4133438702 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0042070 con 0x7f14d00ce040 2024-04-25T09:57:05.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14c5ffb640 1 -- 172.21.15.139:0/4133438702 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c003d650 con 0x7f14d00ce040 2024-04-25T09:57:05.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d777e640 1 -- 172.21.15.139:0/4133438702 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d009c5d0 con 0x7f14d00ce040 2024-04-25T09:57:05.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14c5ffb640 1 -- 172.21.15.139:0/4133438702 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0037040 con 0x7f14d00ce040 2024-04-25T09:57:05.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.964+0000 7f14d777e640 1 -- 172.21.15.139:0/4133438702 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d009cb90 con 0x7f14d00ce040 2024-04-25T09:57:05.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.965+0000 7f14d777e640 1 -- 172.21.15.139:0/4133438702 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d0074520 con 0x7f14d00ce040 2024-04-25T09:57:05.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.966+0000 7f14c5ffb640 1 -- 172.21.15.139:0/4133438702 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14c0045070 con 0x7f14d00ce040 2024-04-25T09:57:05.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.966+0000 7f14c5ffb640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f14b406b4d0 0x7f14b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.966+0000 7f14d551a640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f14b406b4d0 0x7f14b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.966+0000 7f14c5ffb640 1 -- 172.21.15.139:0/4133438702 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f14c00aaa90 con 0x7f14d00ce040 2024-04-25T09:57:05.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.967+0000 7f14d551a640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f14b406b4d0 0x7f14b406d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f14d00b4590 tx=0x7f14cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:05.971+0000 7f14c5ffb640 1 -- 172.21.15.139:0/4133438702 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14c0079780 con 0x7f14d00ce040 2024-04-25T09:57:06.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.140+0000 7f14d777e640 1 -- 172.21.15.139:0/4133438702 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14d009d030 con 0x7f14d00ce040 2024-04-25T09:57:06.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.140+0000 7f14c5ffb640 1 -- 172.21.15.139:0/4133438702 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+11 (secure 0 0 0) 0x7f14c0079080 con 0x7f14d00ce040 2024-04-25T09:57:06.144 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:57:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.142+0000 7f14d777e640 1 -- 172.21.15.139:0/4133438702 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f14b406b4d0 msgr2=0x7f14b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.142+0000 7f14d777e640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f14b406b4d0 0x7f14b406d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f14d00b4590 tx=0x7f14cc009040 comp rx=0 tx=0).stop 2024-04-25T09:57:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.142+0000 7f14d777e640 1 -- 172.21.15.139:0/4133438702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d00ce040 msgr2=0x7f14d009b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.142+0000 7f14d777e640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d00ce040 0x7f14d009b850 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f14c0035040 tx=0x7f14c00046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.142+0000 7f14d777e640 1 -- 172.21.15.139:0/4133438702 shutdown_connections 2024-04-25T09:57:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.142+0000 7f14d777e640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d00ce040 0x7f14d009b850 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.142+0000 7f14d777e640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f14b406b4d0 0x7f14b406d990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.142+0000 7f14d777e640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14d015bf90 0x7f14d006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.142+0000 7f14d777e640 1 --2- 172.21.15.139:0/4133438702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14d0071a50 0x7f14d015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.142+0000 7f14d777e640 1 -- 172.21.15.139:0/4133438702 >> 172.21.15.139:0/4133438702 conn(0x7f14d00cfd80 msgr2=0x7f14d00ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.142+0000 7f14d777e640 1 -- 172.21.15.139:0/4133438702 shutdown_connections 2024-04-25T09:57:06.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.142+0000 7f14d777e640 1 -- 172.21.15.139:0/4133438702 wait complete. 2024-04-25T09:57:06.156 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:57:06.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.252+0000 7f91cedab640 1 Processor -- start 2024-04-25T09:57:06.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.252+0000 7f91cedab640 1 -- start start 2024-04-25T09:57:06.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91cedab640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f91c815f8c0 0x7f91c809a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91cedab640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c815fca0 0x7f91c80cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91cedab640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f91c80d0220 0x7f91c815b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91cedab640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c80d06c0 con 0x7f91c815f8c0 2024-04-25T09:57:06.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91cedab640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c8099bf0 con 0x7f91c80d0220 2024-04-25T09:57:06.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91cedab640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c8099ed0 con 0x7f91c815fca0 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91ccb47640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f91c815f8c0 0x7f91c809a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91ccb47640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f91c815f8c0 0x7f91c809a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49834/0 (socket says 172.21.15.139:49834) 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91ccb47640 1 -- 172.21.15.139:0/3915482847 learned_addr learned my addr 172.21.15.139:0/3915482847 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91ccb47640 1 -- 172.21.15.139:0/3915482847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c815fca0 msgr2=0x7f91c80cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91c7fff640 1 --2- 172.21.15.139:0/3915482847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c815fca0 0x7f91c80cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91cd348640 1 --2- 172.21.15.139:0/3915482847 >> v2:172.21.15.150:3300/0 conn(0x7f91c80d0220 0x7f91c815b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91ccb47640 1 --2- 172.21.15.139:0/3915482847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c815fca0 0x7f91c80cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91ccb47640 1 -- 172.21.15.139:0/3915482847 >> v2:172.21.15.150:3300/0 conn(0x7f91c80d0220 msgr2=0x7f91c815b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91ccb47640 1 --2- 172.21.15.139:0/3915482847 >> v2:172.21.15.150:3300/0 conn(0x7f91c80d0220 0x7f91c815b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91ccb47640 1 -- 172.21.15.139:0/3915482847 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c809a1b0 con 0x7f91c815f8c0 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91c7fff640 1 --2- 172.21.15.139:0/3915482847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c815fca0 0x7f91c80cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.253+0000 7f91cd348640 1 --2- 172.21.15.139:0/3915482847 >> v2:172.21.15.150:3300/0 conn(0x7f91c80d0220 0x7f91c815b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91ccb47640 1 --2- 172.21.15.139:0/3915482847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91c815f8c0 0x7f91c809a9b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f91c8072620 tx=0x7f91bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208d4029b0ad3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:06.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91c77fe640 1 -- 172.21.15.139:0/3915482847 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc037070 con 0x7f91c815f8c0 2024-04-25T09:57:06.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91c77fe640 1 -- 172.21.15.139:0/3915482847 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc032440 con 0x7f91c815f8c0 2024-04-25T09:57:06.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91c77fe640 1 -- 172.21.15.139:0/3915482847 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc032810 con 0x7f91c815f8c0 2024-04-25T09:57:06.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91cedab640 1 -- 172.21.15.139:0/3915482847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91c815f8c0 msgr2=0x7f91c809a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91cedab640 1 --2- 172.21.15.139:0/3915482847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91c815f8c0 0x7f91c809a9b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f91c8072620 tx=0x7f91bc02a270 comp rx=0 tx=0).stop 2024-04-25T09:57:06.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91cedab640 1 -- 172.21.15.139:0/3915482847 shutdown_connections 2024-04-25T09:57:06.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91cedab640 1 --2- 172.21.15.139:0/3915482847 >> v2:172.21.15.150:3300/0 conn(0x7f91c80d0220 0x7f91c815b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91cedab640 1 --2- 172.21.15.139:0/3915482847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c815fca0 0x7f91c80cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91cedab640 1 --2- 172.21.15.139:0/3915482847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91c815f8c0 0x7f91c809a9b0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91cedab640 1 -- 172.21.15.139:0/3915482847 >> 172.21.15.139:0/3915482847 conn(0x7f91c80b31f0 msgr2=0x7f91c80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91cedab640 1 -- 172.21.15.139:0/3915482847 shutdown_connections 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.254+0000 7f91cedab640 1 -- 172.21.15.139:0/3915482847 wait complete. 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.256+0000 7f91cedab640 1 Processor -- start 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.256+0000 7f91cedab640 1 -- start start 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.256+0000 7f91cedab640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91c80d0220 0x7f91c8147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.256+0000 7f91cedab640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c815f8c0 0x7f91c8147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.256+0000 7f91cedab640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c815fca0 0x7f91c81e7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.256+0000 7f91cedab640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c81602d0 con 0x7f91c80d0220 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.256+0000 7f91cedab640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c809aef0 con 0x7f91c815fca0 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.256+0000 7f91cedab640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c8160450 con 0x7f91c815f8c0 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91ccb47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91c80d0220 0x7f91c8147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91ccb47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91c80d0220 0x7f91c8147b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49840/0 (socket says 172.21.15.139:49840) 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91ccb47640 1 -- 172.21.15.139:0/4010047194 learned_addr learned my addr 172.21.15.139:0/4010047194 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91cd348640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c815fca0 0x7f91c81e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91c7fff640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c815f8c0 0x7f91c8147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91ccb47640 1 -- 172.21.15.139:0/4010047194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c815f8c0 msgr2=0x7f91c8147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91ccb47640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c815f8c0 0x7f91c8147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91ccb47640 1 -- 172.21.15.139:0/4010047194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c815fca0 msgr2=0x7f91c81e7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91ccb47640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c815fca0 0x7f91c81e7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91ccb47640 1 -- 172.21.15.139:0/4010047194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8148060 con 0x7f91c80d0220 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91cd348640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c815fca0 0x7f91c81e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91ccb47640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91c80d0220 0x7f91c8147b20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f91bc02a2b0 tx=0x7f91bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91c57fa640 1 -- 172.21.15.139:0/4010047194 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc037040 con 0x7f91c80d0220 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91c57fa640 1 -- 172.21.15.139:0/4010047194 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc042070 con 0x7f91c80d0220 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91cedab640 1 -- 172.21.15.139:0/4010047194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c80b7180 con 0x7f91c80d0220 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91c57fa640 1 -- 172.21.15.139:0/4010047194 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc030bb0 con 0x7f91c80d0220 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.257+0000 7f91cedab640 1 -- 172.21.15.139:0/4010047194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c81e6e60 con 0x7f91c80d0220 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.258+0000 7f91cedab640 1 -- 172.21.15.139:0/4010047194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c815b730 con 0x7f91c80d0220 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.259+0000 7f91c57fa640 1 -- 172.21.15.139:0/4010047194 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91bc047460 con 0x7f91c80d0220 2024-04-25T09:57:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.259+0000 7f91c57fa640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f91ac06b4d0 0x7f91ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.259+0000 7f91c57fa640 1 -- 172.21.15.139:0/4010047194 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f91bc0ab070 con 0x7f91c80d0220 2024-04-25T09:57:06.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.259+0000 7f91c7fff640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f91ac06b4d0 0x7f91ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:06.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.259+0000 7f91c7fff640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f91ac06b4d0 0x7f91ac06d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f91c815df00 tx=0x7f91b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:06.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.263+0000 7f91c57fa640 1 -- 172.21.15.139:0/4010047194 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91c815b730 con 0x7f91c80d0220 2024-04-25T09:57:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.447+0000 7f91cedab640 1 -- 172.21.15.139:0/4010047194 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f91c8074520 con 0x7f91c80d0220 2024-04-25T09:57:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.447+0000 7f91c57fa640 1 -- 172.21.15.139:0/4010047194 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v64) v1 ==== 80+0+0 (secure 0 0 0) 0x7f91bc045360 con 0x7f91c80d0220 2024-04-25T09:57:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T09:57:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.449+0000 7f91cedab640 1 -- 172.21.15.139:0/4010047194 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f91ac06b4d0 msgr2=0x7f91ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.449+0000 7f91cedab640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f91ac06b4d0 0x7f91ac06d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f91c815df00 tx=0x7f91b4008040 comp rx=0 tx=0).stop 2024-04-25T09:57:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.449+0000 7f91cedab640 1 -- 172.21.15.139:0/4010047194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91c80d0220 msgr2=0x7f91c8147b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.449+0000 7f91cedab640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91c80d0220 0x7f91c8147b20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f91bc02a2b0 tx=0x7f91bc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.449+0000 7f91cedab640 1 -- 172.21.15.139:0/4010047194 shutdown_connections 2024-04-25T09:57:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.449+0000 7f91cedab640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f91ac06b4d0 0x7f91ac06d990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.449+0000 7f91cedab640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c815fca0 0x7f91c81e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.449+0000 7f91cedab640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c815f8c0 0x7f91c8147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.449+0000 7f91cedab640 1 --2- 172.21.15.139:0/4010047194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91c80d0220 0x7f91c8147b20 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.449+0000 7f91cedab640 1 -- 172.21.15.139:0/4010047194 >> 172.21.15.139:0/4010047194 conn(0x7f91c80b31f0 msgr2=0x7f91c80c58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.450+0000 7f91cedab640 1 -- 172.21.15.139:0/4010047194 shutdown_connections 2024-04-25T09:57:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.450+0000 7f91cedab640 1 -- 172.21.15.139:0/4010047194 wait complete. 2024-04-25T09:57:06.463 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:57:06.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:06.464 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:06.500 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:57:06.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:06.501 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:57:06.556 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:57:06.556 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:06.573 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:06.573 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:57:06.573 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:06.611 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:57:06.611 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:57:06.611 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:06.629 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:06.629 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:06.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.744+0000 7fca81b4c640 1 Processor -- start 2024-04-25T09:57:06.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.744+0000 7fca81b4c640 1 -- start start 2024-04-25T09:57:06.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca81b4c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fca7c06ab60 0x7fca7c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca81b4c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca7c0b36f0 0x7fca7c0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca81b4c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca7c0d0400 con 0x7fca7c160e90 2024-04-25T09:57:06.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca81b4c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca7c0d14a0 con 0x7fca7c06ab60 2024-04-25T09:57:06.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca81b4c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0d06e0 con 0x7fca7c0b36f0 2024-04-25T09:57:06.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca7affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fca7c06ab60 0x7fca7c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:06.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca7b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fca7c160e90 0x7fca7c0afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41184/0 (socket says 172.21.15.139:41184) 2024-04-25T09:57:06.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca7b7fe640 1 -- 172.21.15.139:0/759989580 learned_addr learned my addr 172.21.15.139:0/759989580 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:06.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca7affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fca7c06ab60 0x7fca7c06aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53470/0 (socket says 172.21.15.139:53470) 2024-04-25T09:57:06.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca7affd640 1 -- 172.21.15.139:0/759989580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca7c0b36f0 msgr2=0x7fca7c0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:06.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca7affd640 1 --2- 172.21.15.139:0/759989580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca7c0b36f0 0x7fca7c0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca7affd640 1 -- 172.21.15.139:0/759989580 >> v1:172.21.15.139:6789/0 conn(0x7fca7c160e90 legacy=0x7fca7c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca7affd640 1 -- 172.21.15.139:0/759989580 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c0d09c0 con 0x7fca7c06ab60 2024-04-25T09:57:06.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.745+0000 7fca7a7fc640 1 -- 172.21.15.139:0/759989580 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227075805 0 0) 0x7fca7c0d0400 con 0x7fca7c160e90 2024-04-25T09:57:06.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.746+0000 7fca7affd640 1 --2- 172.21.15.139:0/759989580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c06ab60 0x7fca7c06aff0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fca700097d0 tx=0x7fca7002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e019043b946963b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:06.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.746+0000 7fca7a7fc640 1 -- 172.21.15.139:0/759989580 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70037070 con 0x7fca7c06ab60 2024-04-25T09:57:06.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.746+0000 7fca7a7fc640 1 -- 172.21.15.139:0/759989580 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7002ae40 con 0x7fca7c06ab60 2024-04-25T09:57:06.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.746+0000 7fca7a7fc640 1 -- 172.21.15.139:0/759989580 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70032660 con 0x7fca7c06ab60 2024-04-25T09:57:06.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.747+0000 7fca81b4c640 1 -- 172.21.15.139:0/759989580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c06ab60 msgr2=0x7fca7c06aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.747+0000 7fca81b4c640 1 --2- 172.21.15.139:0/759989580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c06ab60 0x7fca7c06aff0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fca700097d0 tx=0x7fca7002a130 comp rx=0 tx=0).stop 2024-04-25T09:57:06.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.747+0000 7fca81b4c640 1 -- 172.21.15.139:0/759989580 shutdown_connections 2024-04-25T09:57:06.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.747+0000 7fca81b4c640 1 --2- 172.21.15.139:0/759989580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca7c0b36f0 0x7fca7c0b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.747+0000 7fca81b4c640 1 --2- 172.21.15.139:0/759989580 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c06ab60 0x7fca7c06aff0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.747+0000 7fca81b4c640 1 -- 172.21.15.139:0/759989580 >> 172.21.15.139:0/759989580 conn(0x7fca7c0b23f0 msgr2=0x7fca7c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:06.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.747+0000 7fca81b4c640 1 -- 172.21.15.139:0/759989580 shutdown_connections 2024-04-25T09:57:06.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.747+0000 7fca81b4c640 1 -- 172.21.15.139:0/759989580 wait complete. 2024-04-25T09:57:06.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.748+0000 7fca81b4c640 1 Processor -- start 2024-04-25T09:57:06.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.748+0000 7fca81b4c640 1 -- start start 2024-04-25T09:57:06.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca81b4c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca7c06ab60 0x7fca7c1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca81b4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c0b36f0 0x7fca7c0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca81b4c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca7c160e90 0x7fca7c145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca81b4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c161270 con 0x7fca7c0b36f0 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca81b4c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0d0f80 con 0x7fca7c06ab60 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca81b4c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c1461b0 con 0x7fca7c160e90 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca7affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c0b36f0 0x7fca7c0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca7affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c0b36f0 0x7fca7c0cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53476/0 (socket says 172.21.15.139:53476) 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca7affd640 1 -- 172.21.15.139:0/4138493163 learned_addr learned my addr 172.21.15.139:0/4138493163 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca7bfff640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca7c160e90 0x7fca7c145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca7b7fe640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca7c06ab60 0x7fca7c1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca7affd640 1 -- 172.21.15.139:0/4138493163 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca7c160e90 msgr2=0x7fca7c145c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca7affd640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca7c160e90 0x7fca7c145c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca7affd640 1 -- 172.21.15.139:0/4138493163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca7c06ab60 msgr2=0x7fca7c1533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca7affd640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca7c06ab60 0x7fca7c1533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.749+0000 7fca7affd640 1 -- 172.21.15.139:0/4138493163 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c154db0 con 0x7fca7c0b36f0 2024-04-25T09:57:06.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.750+0000 7fca7affd640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c0b36f0 0x7fca7c0cbe10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fca7002a170 tx=0x7fca700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:06.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.750+0000 7fca63fff640 1 -- 172.21.15.139:0/4138493163 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70042070 con 0x7fca7c0b36f0 2024-04-25T09:57:06.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.750+0000 7fca63fff640 1 -- 172.21.15.139:0/4138493163 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7003e670 con 0x7fca7c0b36f0 2024-04-25T09:57:06.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.750+0000 7fca81b4c640 1 -- 172.21.15.139:0/4138493163 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c155090 con 0x7fca7c0b36f0 2024-04-25T09:57:06.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.750+0000 7fca63fff640 1 -- 172.21.15.139:0/4138493163 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70037040 con 0x7fca7c0b36f0 2024-04-25T09:57:06.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.750+0000 7fca81b4c640 1 -- 172.21.15.139:0/4138493163 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c155650 con 0x7fca7c0b36f0 2024-04-25T09:57:06.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.751+0000 7fca81b4c640 1 -- 172.21.15.139:0/4138493163 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c074520 con 0x7fca7c0b36f0 2024-04-25T09:57:06.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.751+0000 7fca63fff640 1 -- 172.21.15.139:0/4138493163 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca70045070 con 0x7fca7c0b36f0 2024-04-25T09:57:06.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.752+0000 7fca63fff640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca6806b500 0x7fca6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:06.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.752+0000 7fca63fff640 1 -- 172.21.15.139:0/4138493163 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fca700aa8e0 con 0x7fca7c0b36f0 2024-04-25T09:57:06.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.752+0000 7fca7b7fe640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca6806b500 0x7fca6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:06.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.752+0000 7fca7b7fe640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca6806b500 0x7fca6806d9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fca6c007fd0 tx=0x7fca6c007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:06.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.756+0000 7fca63fff640 1 -- 172.21.15.139:0/4138493163 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca70078f80 con 0x7fca7c0b36f0 2024-04-25T09:57:06.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.961+0000 7fca81b4c640 1 -- 172.21.15.139:0/4138493163 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca7c0728b0 con 0x7fca7c0b36f0 2024-04-25T09:57:06.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.962+0000 7fca63fff640 1 -- 172.21.15.139:0/4138493163 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fca7007d020 con 0x7fca7c0b36f0 2024-04-25T09:57:06.966 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:06.966 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/829268166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":829268166},{"type":"v1","addr":"172.21.15.150:6835","nonce":829268166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3075039902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3075039902},{"type":"v1","addr":"172.21.15.150:6834","nonce":3075039902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/627726069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":627726069},{"type":"v1","addr":"172.21.15.139:6836","nonce":627726069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:50.626292+0000","modified":"2024-04-25T09:56:39.327977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.139:6837/3541444834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3541444834},{"type":"v1","addr":"172.21.15.139:6837","nonce":3541444834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:57:06.966 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 30 2024-04-25T09:57:06.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.964+0000 7fca81b4c640 1 -- 172.21.15.139:0/4138493163 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca6806b500 msgr2=0x7fca6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.964+0000 7fca81b4c640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca6806b500 0x7fca6806d9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fca6c007fd0 tx=0x7fca6c007de0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.964+0000 7fca81b4c640 1 -- 172.21.15.139:0/4138493163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c0b36f0 msgr2=0x7fca7c0cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.964+0000 7fca81b4c640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c0b36f0 0x7fca7c0cbe10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fca7002a170 tx=0x7fca700046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.964+0000 7fca81b4c640 1 -- 172.21.15.139:0/4138493163 shutdown_connections 2024-04-25T09:57:06.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.964+0000 7fca81b4c640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fca7c0b36f0 0x7fca7c0cbe10 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.964+0000 7fca81b4c640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fca6806b500 0x7fca6806d9c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.964+0000 7fca81b4c640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fca7c160e90 0x7fca7c145c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.964+0000 7fca81b4c640 1 --2- 172.21.15.139:0/4138493163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca7c06ab60 0x7fca7c1533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.964+0000 7fca81b4c640 1 -- 172.21.15.139:0/4138493163 >> 172.21.15.139:0/4138493163 conn(0x7fca7c0b23f0 msgr2=0x7fca7c06fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:06.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.964+0000 7fca81b4c640 1 -- 172.21.15.139:0/4138493163 shutdown_connections 2024-04-25T09:57:06.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:06.964+0000 7fca81b4c640 1 -- 172.21.15.139:0/4138493163 wait complete. 2024-04-25T09:57:06.978 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.073+0000 7fb52562a640 1 Processor -- start 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.073+0000 7fb52562a640 1 -- start start 2024-04-25T09:57:07.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.074+0000 7fb52562a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5200d1080 0x7fb5200b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.074+0000 7fb52562a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb520161850 0x7fb5200ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.074+0000 7fb52562a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5200cf250 con 0x7fb52015dc60 2024-04-25T09:57:07.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.074+0000 7fb52562a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5200ce780 con 0x7fb520161850 2024-04-25T09:57:07.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.074+0000 7fb52562a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200d3730 con 0x7fb5200d1080 2024-04-25T09:57:07.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.074+0000 7fb51e7fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5200d1080 0x7fb5200b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.074+0000 7fb51f7fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb520161850 0x7fb5200ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.074+0000 7fb51f7fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb520161850 0x7fb5200ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:45582/0 (socket says 172.21.15.139:45582) 2024-04-25T09:57:07.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.074+0000 7fb51f7fe640 1 -- 172.21.15.139:0/3419670054 learned_addr learned my addr 172.21.15.139:0/3419670054 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:07.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.075+0000 7fb51e7fc640 1 -- 172.21.15.139:0/3419670054 >> v2:172.21.15.150:3300/0 conn(0x7fb520161850 msgr2=0x7fb5200ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:07.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.075+0000 7fb51e7fc640 1 --2- 172.21.15.139:0/3419670054 >> v2:172.21.15.150:3300/0 conn(0x7fb520161850 0x7fb5200ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.075+0000 7fb51e7fc640 1 -- 172.21.15.139:0/3419670054 >> v1:172.21.15.150:6789/0 conn(0x7fb52015dc60 legacy=0x7fb5200d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.075+0000 7fb51e7fc640 1 -- 172.21.15.139:0/3419670054 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5200ce9b0 con 0x7fb5200d1080 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.075+0000 7fb51e7fc640 1 --2- 172.21.15.139:0/3419670054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5200d1080 0x7fb5200b6c10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb5140097a0 tx=0x7fb51402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ece5ed7c2300defa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.075+0000 7fb51dffb640 1 -- 172.21.15.139:0/3419670054 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514037070 con 0x7fb5200d1080 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.076+0000 7fb51dffb640 1 -- 172.21.15.139:0/3419670054 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb514032440 con 0x7fb5200d1080 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.076+0000 7fb51dffb640 1 -- 172.21.15.139:0/3419670054 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514032810 con 0x7fb5200d1080 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.076+0000 7fb52562a640 1 -- 172.21.15.139:0/3419670054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5200d1080 msgr2=0x7fb5200b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.076+0000 7fb52562a640 1 --2- 172.21.15.139:0/3419670054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5200d1080 0x7fb5200b6c10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb5140097a0 tx=0x7fb51402a180 comp rx=0 tx=0).stop 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.076+0000 7fb52562a640 1 -- 172.21.15.139:0/3419670054 shutdown_connections 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.076+0000 7fb52562a640 1 --2- 172.21.15.139:0/3419670054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5200d1080 0x7fb5200b6c10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.076+0000 7fb52562a640 1 --2- 172.21.15.139:0/3419670054 >> v2:172.21.15.150:3300/0 conn(0x7fb520161850 0x7fb5200ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.076+0000 7fb52562a640 1 -- 172.21.15.139:0/3419670054 >> 172.21.15.139:0/3419670054 conn(0x7fb5200b5910 msgr2=0x7fb5200b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.076+0000 7fb52562a640 1 -- 172.21.15.139:0/3419670054 shutdown_connections 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.076+0000 7fb52562a640 1 -- 172.21.15.139:0/3419670054 wait complete. 2024-04-25T09:57:07.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.077+0000 7fb52562a640 1 Processor -- start 2024-04-25T09:57:07.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.077+0000 7fb52562a640 1 -- start start 2024-04-25T09:57:07.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.078+0000 7fb52562a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5200d1080 0x7fb520151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.078+0000 7fb52562a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb52015dc60 0x7fb5201f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.078+0000 7fb52562a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb520161850 0x7fb5200c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.078+0000 7fb52562a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200d1fc0 con 0x7fb5200d1080 2024-04-25T09:57:07.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.078+0000 7fb52562a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200cef70 con 0x7fb520161850 2024-04-25T09:57:07.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.078+0000 7fb52562a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52015e040 con 0x7fb52015dc60 2024-04-25T09:57:07.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.078+0000 7fb51effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5200d1080 0x7fb520151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.078+0000 7fb51effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5200d1080 0x7fb520151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53486/0 (socket says 172.21.15.139:53486) 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.078+0000 7fb51effd640 1 -- 172.21.15.139:0/1329929849 learned_addr learned my addr 172.21.15.139:0/1329929849 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.078+0000 7fb51e7fc640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb52015dc60 0x7fb5201f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.078+0000 7fb51f7fe640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb520161850 0x7fb5200c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.079+0000 7fb51effd640 1 -- 172.21.15.139:0/1329929849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb52015dc60 msgr2=0x7fb5201f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.079+0000 7fb51effd640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb52015dc60 0x7fb5201f3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.079+0000 7fb51effd640 1 -- 172.21.15.139:0/1329929849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb520161850 msgr2=0x7fb5200c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.079+0000 7fb51effd640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb520161850 0x7fb5200c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.079+0000 7fb51effd640 1 -- 172.21.15.139:0/1329929849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5201f40f0 con 0x7fb5200d1080 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.079+0000 7fb51effd640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5200d1080 0x7fb520151ff0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c430 tx=0x7fb51000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.079+0000 7fb5077fe640 1 -- 172.21.15.139:0/1329929849 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5100052d0 con 0x7fb5200d1080 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.079+0000 7fb5077fe640 1 -- 172.21.15.139:0/1329929849 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb510005490 con 0x7fb5200d1080 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.079+0000 7fb52562a640 1 -- 172.21.15.139:0/1329929849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5201f43d0 con 0x7fb5200d1080 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.079+0000 7fb5077fe640 1 -- 172.21.15.139:0/1329929849 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510007260 con 0x7fb5200d1080 2024-04-25T09:57:07.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.079+0000 7fb52562a640 1 -- 172.21.15.139:0/1329929849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5201f4a00 con 0x7fb5200d1080 2024-04-25T09:57:07.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.080+0000 7fb52562a640 1 -- 172.21.15.139:0/1329929849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb520074520 con 0x7fb5200d1080 2024-04-25T09:57:07.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.080+0000 7fb5077fe640 1 -- 172.21.15.139:0/1329929849 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb510007400 con 0x7fb5200d1080 2024-04-25T09:57:07.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.081+0000 7fb5077fe640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb50c06b4d0 0x7fb50c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.081+0000 7fb5077fe640 1 -- 172.21.15.139:0/1329929849 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fb5100897d0 con 0x7fb5200d1080 2024-04-25T09:57:07.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.081+0000 7fb51e7fc640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb50c06b4d0 0x7fb50c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.081+0000 7fb51e7fc640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb50c06b4d0 0x7fb50c06d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb5140097a0 tx=0x7fb51402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.085+0000 7fb5077fe640 1 -- 172.21.15.139:0/1329929849 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5200cf250 con 0x7fb5200d1080 2024-04-25T09:57:07.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.257+0000 7fb52562a640 1 -- 172.21.15.139:0/1329929849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5200d16f0 con 0x7fb5200d1080 2024-04-25T09:57:07.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.258+0000 7fb5077fe640 1 -- 172.21.15.139:0/1329929849 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17173 (secure 0 0 0) 0x7fb510057e70 con 0x7fb5200d1080 2024-04-25T09:57:07.262 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:07.262 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":64,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:55:57.317641+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:48.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:49.597505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.22","mappings":[{"from":7,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:07.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.260+0000 7fb52562a640 1 -- 172.21.15.139:0/1329929849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb50c06b4d0 msgr2=0x7fb50c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.260+0000 7fb52562a640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb50c06b4d0 0x7fb50c06d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb5140097a0 tx=0x7fb51402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:07.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.260+0000 7fb52562a640 1 -- 172.21.15.139:0/1329929849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5200d1080 msgr2=0x7fb520151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.260+0000 7fb52562a640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5200d1080 0x7fb520151ff0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c430 tx=0x7fb51000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.260+0000 7fb52562a640 1 -- 172.21.15.139:0/1329929849 shutdown_connections 2024-04-25T09:57:07.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.260+0000 7fb52562a640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb50c06b4d0 0x7fb50c06d990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.260+0000 7fb52562a640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb520161850 0x7fb5200c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.260+0000 7fb52562a640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb52015dc60 0x7fb5201f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.260+0000 7fb52562a640 1 --2- 172.21.15.139:0/1329929849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5200d1080 0x7fb520151ff0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.260+0000 7fb52562a640 1 -- 172.21.15.139:0/1329929849 >> 172.21.15.139:0/1329929849 conn(0x7fb5200b5910 msgr2=0x7fb5200c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.260+0000 7fb52562a640 1 -- 172.21.15.139:0/1329929849 shutdown_connections 2024-04-25T09:57:07.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.260+0000 7fb52562a640 1 -- 172.21.15.139:0/1329929849 wait complete. 2024-04-25T09:57:07.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:55:48.823437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:55:49.597505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:07.275 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:07.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.373+0000 7f2cb886f640 1 Processor -- start 2024-04-25T09:57:07.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.373+0000 7f2cb886f640 1 -- start start 2024-04-25T09:57:07.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.374+0000 7f2cb886f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb40cd610 0x7f2cb40d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.374+0000 7f2cb886f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2cb409ad40 0x7f2cb40d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.374+0000 7f2cb886f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb4099ca0 con 0x7f2cb41618c0 2024-04-25T09:57:07.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.374+0000 7f2cb886f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb4099a50 con 0x7f2cb409ad40 2024-04-25T09:57:07.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.374+0000 7f2cb886f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb4099f80 con 0x7f2cb40cd610 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.374+0000 7f2cb2d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2cb41618c0 0x7f2cb406be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41194/0 (socket says 172.21.15.139:41194) 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.374+0000 7f2cb2d9d640 1 -- 172.21.15.139:0/3670435265 learned_addr learned my addr 172.21.15.139:0/3670435265 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.374+0000 7f2cb259c640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb40cd610 0x7f2cb40d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.374+0000 7f2cb1d9b640 1 --2- 172.21.15.139:0/3670435265 >> v2:172.21.15.150:3300/0 conn(0x7f2cb409ad40 0x7f2cb40d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.374+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393886050 0 0) 0x7f2cb4099ca0 con 0x7f2cb41618c0 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.374+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c9c003520 con 0x7f2cb41618c0 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.374+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4026801469 0 0) 0x7f2c9c003520 con 0x7f2cb41618c0 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb40cd610 msgr2=0x7f2cb40d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb40cd610 0x7f2cb40d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 >> v2:172.21.15.150:3300/0 conn(0x7f2cb409ad40 msgr2=0x7f2cb40d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 --2- 172.21.15.139:0/3670435265 >> v2:172.21.15.150:3300/0 conn(0x7f2cb409ad40 0x7f2cb40d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb409a260 con 0x7f2cb41618c0 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f2cac0019f0 con 0x7f2cb41618c0 2024-04-25T09:57:07.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 >> v1:172.21.15.139:6789/0 conn(0x7f2cb41618c0 legacy=0x7f2cb406be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c9c0039b0 0x7f2c9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c9c0043a0 0x7f2c9c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9c004ef0 0x7f2c9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb4099a50 con 0x7f2c9c004ef0 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb4099f80 con 0x7f2c9c0043a0 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c00bf00 con 0x7f2c9c0039b0 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb259c640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb40cd610 0x7f2cb40d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb1d9b640 1 --2- 172.21.15.139:0/3670435265 >> v2:172.21.15.150:3300/0 conn(0x7f2cb409ad40 0x7f2cb40d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb2d9d640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9c004ef0 0x7f2c9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb1d9b640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c9c0043a0 0x7f2c9c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb259c640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c9c0039b0 0x7f2c9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb2d9d640 1 -- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c9c0039b0 msgr2=0x7f2c9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:07.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb2d9d640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c9c0039b0 0x7f2c9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb2d9d640 1 -- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c9c0043a0 msgr2=0x7f2c9c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb2d9d640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c9c0043a0 0x7f2c9c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb2d9d640 1 -- 172.21.15.139:0/3670435265 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c003520 con 0x7f2c9c004ef0 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.375+0000 7f2cb1d9b640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c9c0043a0 0x7f2c9c004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.376+0000 7f2cb2d9d640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9c004ef0 0x7f2c9c00b9c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00a8d0 tx=0x7f2c9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fbaef5994b9116 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.376+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00a4c0 con 0x7f2c9c004ef0 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.376+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac02f070 con 0x7f2c9c004ef0 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.376+0000 7f2cb159a640 1 -- 172.21.15.139:0/3670435265 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac02bd40 con 0x7f2c9c004ef0 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.376+0000 7f2cb886f640 1 -- 172.21.15.139:0/3670435265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9c004ef0 msgr2=0x7f2c9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.376+0000 7f2cb886f640 1 --2- 172.21.15.139:0/3670435265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9c004ef0 0x7f2c9c00b9c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00a8d0 tx=0x7f2c9c005830 comp rx=0 tx=0).stop 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.376+0000 7f2cb2d9d640 1 -- 172.21.15.139:0/3670435265 reap_dead start 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.376+0000 7f2cb886f640 1 -- 172.21.15.139:0/3670435265 shutdown_connections 2024-04-25T09:57:07.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.376+0000 7f2cb886f640 1 -- 172.21.15.139:0/3670435265 >> 172.21.15.139:0/3670435265 conn(0x7f2cb40b31f0 msgr2=0x7f2cb40b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.376+0000 7f2cb886f640 1 -- 172.21.15.139:0/3670435265 shutdown_connections 2024-04-25T09:57:07.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.376+0000 7f2cb886f640 1 -- 172.21.15.139:0/3670435265 wait complete. 2024-04-25T09:57:07.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.378+0000 7f2cb886f640 1 Processor -- start 2024-04-25T09:57:07.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.378+0000 7f2cb886f640 1 -- start start 2024-04-25T09:57:07.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.378+0000 7f2cb886f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c9c004ef0 0x7f2cb409eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.378+0000 7f2cb886f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb415f600 0x7f2cb41f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.378+0000 7f2cb886f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4070450 0x7f2cb4070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.378+0000 7f2cb886f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40cd9f0 con 0x7f2cb415f600 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.378+0000 7f2cb886f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb409a820 con 0x7f2c9c004ef0 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.378+0000 7f2cb886f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40cdb70 con 0x7f2cb4070450 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb415f600 0x7f2cb41f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb415f600 0x7f2cb41f3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53498/0 (socket says 172.21.15.139:53498) 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb1d9b640 1 -- 172.21.15.139:0/3591145285 learned_addr learned my addr 172.21.15.139:0/3591145285 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb2d9d640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4070450 0x7f2cb4070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb259c640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c9c004ef0 0x7f2cb409eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb1d9b640 1 -- 172.21.15.139:0/3591145285 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4070450 msgr2=0x7f2cb4070900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb1d9b640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4070450 0x7f2cb4070900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb1d9b640 1 -- 172.21.15.139:0/3591145285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c9c004ef0 msgr2=0x7f2cb409eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb1d9b640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c9c004ef0 0x7f2cb409eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb1d9b640 1 -- 172.21.15.139:0/3591145285 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb40c8540 con 0x7f2cb415f600 2024-04-25T09:57:07.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb1d9b640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb415f600 0x7f2cb41f3480 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80118d0 tx=0x7f2ca8011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2ca2ffd640 1 -- 172.21.15.139:0/3591145285 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800fbd0 con 0x7f2cb415f600 2024-04-25T09:57:07.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2ca2ffd640 1 -- 172.21.15.139:0/3591145285 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca800fd70 con 0x7f2cb415f600 2024-04-25T09:57:07.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb886f640 1 -- 172.21.15.139:0/3591145285 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb41f39c0 con 0x7f2cb415f600 2024-04-25T09:57:07.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2ca2ffd640 1 -- 172.21.15.139:0/3591145285 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8015660 con 0x7f2cb415f600 2024-04-25T09:57:07.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.379+0000 7f2cb886f640 1 -- 172.21.15.139:0/3591145285 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb40c7ca0 con 0x7f2cb415f600 2024-04-25T09:57:07.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.380+0000 7f2cb886f640 1 -- 172.21.15.139:0/3591145285 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb4074520 con 0x7f2cb415f600 2024-04-25T09:57:07.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.380+0000 7f2ca2ffd640 1 -- 172.21.15.139:0/3591145285 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ca8015800 con 0x7f2cb415f600 2024-04-25T09:57:07.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.381+0000 7f2ca2ffd640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2c9406b500 0x7f2c9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.381+0000 7f2ca2ffd640 1 -- 172.21.15.139:0/3591145285 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f2ca808b000 con 0x7f2cb415f600 2024-04-25T09:57:07.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.381+0000 7f2cb259c640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2c9406b500 0x7f2c9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.381+0000 7f2cb259c640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2c9406b500 0x7f2c9406d9c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2cb406f210 tx=0x7f2ca4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.385+0000 7f2ca2ffd640 1 -- 172.21.15.139:0/3591145285 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ca8019030 con 0x7f2cb415f600 2024-04-25T09:57:07.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.555+0000 7f2cb886f640 1 -- 172.21.15.139:0/3591145285 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cb40b3f70 con 0x7f2cb415f600 2024-04-25T09:57:07.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.556+0000 7f2ca2ffd640 1 -- 172.21.15.139:0/3591145285 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ca8059c60 con 0x7f2cb415f600 2024-04-25T09:57:07.559 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:57:07.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.558+0000 7f2cb886f640 1 -- 172.21.15.139:0/3591145285 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2c9406b500 msgr2=0x7f2c9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.558+0000 7f2cb886f640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2c9406b500 0x7f2c9406d9c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2cb406f210 tx=0x7f2ca4008040 comp rx=0 tx=0).stop 2024-04-25T09:57:07.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.558+0000 7f2cb886f640 1 -- 172.21.15.139:0/3591145285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb415f600 msgr2=0x7f2cb41f3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.558+0000 7f2cb886f640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb415f600 0x7f2cb41f3480 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80118d0 tx=0x7f2ca8011d90 comp rx=0 tx=0).stop 2024-04-25T09:57:07.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.558+0000 7f2cb886f640 1 -- 172.21.15.139:0/3591145285 shutdown_connections 2024-04-25T09:57:07.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.558+0000 7f2cb886f640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb415f600 0x7f2cb41f3480 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.558+0000 7f2cb886f640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2c9406b500 0x7f2c9406d9c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.558+0000 7f2cb886f640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4070450 0x7f2cb4070900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.558+0000 7f2cb886f640 1 --2- 172.21.15.139:0/3591145285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c9c004ef0 0x7f2cb409eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.558+0000 7f2cb886f640 1 -- 172.21.15.139:0/3591145285 >> 172.21.15.139:0/3591145285 conn(0x7f2cb40b31f0 msgr2=0x7f2cb40b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.558+0000 7f2cb886f640 1 -- 172.21.15.139:0/3591145285 shutdown_connections 2024-04-25T09:57:07.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.558+0000 7f2cb886f640 1 -- 172.21.15.139:0/3591145285 wait complete. 2024-04-25T09:57:07.572 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:57:07.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.669+0000 7f4ff3f34640 1 Processor -- start 2024-04-25T09:57:07.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.669+0000 7f4ff3f34640 1 -- start start 2024-04-25T09:57:07.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff3f34640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4fec144090 0x7f4fec0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff3f34640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4fec0cc9e0 0x7f4fec06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff3f34640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fec0cce50 0x7f4fec06e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff3f34640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fec0b2a50 con 0x7f4fec144090 2024-04-25T09:57:07.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff3f34640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fec15b090 con 0x7f4fec0cc9e0 2024-04-25T09:57:07.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff3f34640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec15b370 con 0x7f4fec0cce50 2024-04-25T09:57:07.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff1cd0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4fec144090 0x7f4fec0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff1cd0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4fec144090 0x7f4fec0b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53504/0 (socket says 172.21.15.139:53504) 2024-04-25T09:57:07.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff1cd0640 1 -- 172.21.15.139:0/1294966810 learned_addr learned my addr 172.21.15.139:0/1294966810 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:07.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff1cd0640 1 -- 172.21.15.139:0/1294966810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fec0cce50 msgr2=0x7f4fec06e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:07.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff1cd0640 1 --2- 172.21.15.139:0/1294966810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fec0cce50 0x7f4fec06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff1cd0640 1 -- 172.21.15.139:0/1294966810 >> v2:172.21.15.150:3300/0 conn(0x7f4fec0cc9e0 msgr2=0x7f4fec06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:07.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff1cd0640 1 --2- 172.21.15.139:0/1294966810 >> v2:172.21.15.150:3300/0 conn(0x7f4fec0cc9e0 0x7f4fec06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.670+0000 7f4ff1cd0640 1 -- 172.21.15.139:0/1294966810 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec15b650 con 0x7f4fec144090 2024-04-25T09:57:07.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.671+0000 7f4ff1cd0640 1 --2- 172.21.15.139:0/1294966810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fec144090 0x7f4fec0b31a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0b2780 tx=0x7f4fe802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a6fbfa733922a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.671+0000 7f4ff0cce640 1 -- 172.21.15.139:0/1294966810 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8037070 con 0x7f4fec144090 2024-04-25T09:57:07.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.671+0000 7f4ff0cce640 1 -- 172.21.15.139:0/1294966810 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe8032440 con 0x7f4fec144090 2024-04-25T09:57:07.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.671+0000 7f4ff0cce640 1 -- 172.21.15.139:0/1294966810 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8032810 con 0x7f4fec144090 2024-04-25T09:57:07.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.671+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1294966810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fec144090 msgr2=0x7f4fec0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.671+0000 7f4ff3f34640 1 --2- 172.21.15.139:0/1294966810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fec144090 0x7f4fec0b31a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0b2780 tx=0x7f4fe802a140 comp rx=0 tx=0).stop 2024-04-25T09:57:07.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.671+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1294966810 shutdown_connections 2024-04-25T09:57:07.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.671+0000 7f4ff3f34640 1 --2- 172.21.15.139:0/1294966810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fec0cce50 0x7f4fec06e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.671+0000 7f4ff3f34640 1 --2- 172.21.15.139:0/1294966810 >> v2:172.21.15.150:3300/0 conn(0x7f4fec0cc9e0 0x7f4fec06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.671+0000 7f4ff3f34640 1 --2- 172.21.15.139:0/1294966810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fec144090 0x7f4fec0b31a0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.671+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1294966810 >> 172.21.15.139:0/1294966810 conn(0x7f4fec067000 msgr2=0x7f4fec06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.671+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1294966810 shutdown_connections 2024-04-25T09:57:07.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.672+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1294966810 wait complete. 2024-04-25T09:57:07.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.673+0000 7f4ff3f34640 1 Processor -- start 2024-04-25T09:57:07.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.673+0000 7f4ff3f34640 1 -- start start 2024-04-25T09:57:07.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.673+0000 7f4ff3f34640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fec0cc9e0 0x7f4fec163670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.673+0000 7f4ff3f34640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fec0cce50 0x7f4fec12dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.673+0000 7f4ff3f34640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fec144090 0x7f4fec0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.673+0000 7f4ff3f34640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec144470 con 0x7f4fec0cce50 2024-04-25T09:57:07.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.673+0000 7f4ff3f34640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec15bc10 con 0x7f4fec0cc9e0 2024-04-25T09:57:07.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.673+0000 7f4ff3f34640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec15ad80 con 0x7f4fec144090 2024-04-25T09:57:07.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.673+0000 7f4ff14cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fec0cce50 0x7f4fec12dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.673+0000 7f4ff14cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fec0cce50 0x7f4fec12dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53514/0 (socket says 172.21.15.139:53514) 2024-04-25T09:57:07.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.673+0000 7f4ff14cf640 1 -- 172.21.15.139:0/1993557921 learned_addr learned my addr 172.21.15.139:0/1993557921 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:07.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4ff1cd0640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fec0cc9e0 0x7f4fec163670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4ff24d1640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fec144090 0x7f4fec0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4ff14cf640 1 -- 172.21.15.139:0/1993557921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fec144090 msgr2=0x7f4fec0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4ff14cf640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fec144090 0x7f4fec0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4ff14cf640 1 -- 172.21.15.139:0/1993557921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fec0cc9e0 msgr2=0x7f4fec163670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4ff14cf640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fec0cc9e0 0x7f4fec163670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4ff14cf640 1 -- 172.21.15.139:0/1993557921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec163bb0 con 0x7f4fec0cce50 2024-04-25T09:57:07.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4ff24d1640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fec144090 0x7f4fec0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:07.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4ff14cf640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fec0cce50 0x7f4fec12dae0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00b410 tx=0x7f4fdc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4fe27fc640 1 -- 172.21.15.139:0/1993557921 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc015020 con 0x7f4fec0cce50 2024-04-25T09:57:07.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4fe27fc640 1 -- 172.21.15.139:0/1993557921 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc011850 con 0x7f4fec0cce50 2024-04-25T09:57:07.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4fe27fc640 1 -- 172.21.15.139:0/1993557921 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc011c40 con 0x7f4fec0cce50 2024-04-25T09:57:07.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1993557921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fec163e80 con 0x7f4fec0cce50 2024-04-25T09:57:07.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.674+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1993557921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec164440 con 0x7f4fec0cce50 2024-04-25T09:57:07.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.675+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1993557921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fec074520 con 0x7f4fec0cce50 2024-04-25T09:57:07.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.676+0000 7f4fe27fc640 1 -- 172.21.15.139:0/1993557921 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fdc018020 con 0x7f4fec0cce50 2024-04-25T09:57:07.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.676+0000 7f4fe27fc640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4fd006b4d0 0x7f4fd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.676+0000 7f4fe27fc640 1 -- 172.21.15.139:0/1993557921 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f4fdc08a350 con 0x7f4fec0cce50 2024-04-25T09:57:07.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.676+0000 7f4ff1cd0640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4fd006b4d0 0x7f4fd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.676+0000 7f4ff1cd0640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4fd006b4d0 0x7f4fd006d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8034000 tx=0x7f4fe802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.680+0000 7f4fe27fc640 1 -- 172.21.15.139:0/1993557921 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fdc01f020 con 0x7f4fec0cce50 2024-04-25T09:57:07.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.850+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1993557921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4fec164a90 con 0x7f4fec0cce50 2024-04-25T09:57:07.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.850+0000 7f4fe27fc640 1 -- 172.21.15.139:0/1993557921 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4fdc0172c0 con 0x7f4fec0cce50 2024-04-25T09:57:07.853 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:57:07.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.852+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1993557921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4fd006b4d0 msgr2=0x7f4fd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.852+0000 7f4ff3f34640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4fd006b4d0 0x7f4fd006d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8034000 tx=0x7f4fe802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:07.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.852+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1993557921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fec0cce50 msgr2=0x7f4fec12dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.852+0000 7f4ff3f34640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fec0cce50 0x7f4fec12dae0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00b410 tx=0x7f4fdc00b8d0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.852+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1993557921 shutdown_connections 2024-04-25T09:57:07.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.852+0000 7f4ff3f34640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4fec0cce50 0x7f4fec12dae0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.852+0000 7f4ff3f34640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4fd006b4d0 0x7f4fd006d990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.852+0000 7f4ff3f34640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fec144090 0x7f4fec0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.852+0000 7f4ff3f34640 1 --2- 172.21.15.139:0/1993557921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fec0cc9e0 0x7f4fec163670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.852+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1993557921 >> 172.21.15.139:0/1993557921 conn(0x7f4fec067000 msgr2=0x7f4fec06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.852+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1993557921 shutdown_connections 2024-04-25T09:57:07.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.852+0000 7f4ff3f34640 1 -- 172.21.15.139:0/1993557921 wait complete. 2024-04-25T09:57:07.866 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:57:07.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.962+0000 7f81dd5d5640 1 Processor -- start 2024-04-25T09:57:07.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.962+0000 7f81dd5d5640 1 -- start start 2024-04-25T09:57:07.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.963+0000 7f81dd5d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81d80712f0 0x7f81d80b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.963+0000 7f81dd5d5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f81d80719d0 0x7f81d80b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.963+0000 7f81dd5d5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d80b1100 con 0x7f81d8145250 2024-04-25T09:57:07.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.963+0000 7f81dd5d5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d80ceed0 con 0x7f81d80719d0 2024-04-25T09:57:07.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.963+0000 7f81dd5d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80ce2c0 con 0x7f81d80712f0 2024-04-25T09:57:07.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.963+0000 7f81d77fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f81d80719d0 0x7f81d80b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.963+0000 7f81d67fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81d80712f0 0x7f81d80b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.963+0000 7f81d6ffd640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f81d8145250 0x7f81d8070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:39418/0 (socket says 172.21.15.139:39418) 2024-04-25T09:57:07.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.963+0000 7f81d6ffd640 1 -- 172.21.15.139:0/1723674225 learned_addr learned my addr 172.21.15.139:0/1723674225 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:07.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.964+0000 7f81d67fc640 1 -- 172.21.15.139:0/1723674225 >> v2:172.21.15.150:3300/0 conn(0x7f81d80719d0 msgr2=0x7f81d80b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:07.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.964+0000 7f81d67fc640 1 --2- 172.21.15.139:0/1723674225 >> v2:172.21.15.150:3300/0 conn(0x7f81d80719d0 0x7f81d80b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.964+0000 7f81d67fc640 1 -- 172.21.15.139:0/1723674225 >> v1:172.21.15.150:6789/0 conn(0x7f81d8145250 legacy=0x7f81d8070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.964+0000 7f81d5ffb640 1 -- 172.21.15.139:0/1723674225 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051047853 0 0) 0x7f81d80b1100 con 0x7f81d8145250 2024-04-25T09:57:07.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.964+0000 7f81d67fc640 1 -- 172.21.15.139:0/1723674225 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d80ce5a0 con 0x7f81d80712f0 2024-04-25T09:57:07.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.964+0000 7f81d77fe640 1 --2- 172.21.15.139:0/1723674225 >> v2:172.21.15.150:3300/0 conn(0x7f81d80719d0 0x7f81d80b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:07.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.964+0000 7f81d67fc640 1 --2- 172.21.15.139:0/1723674225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81d80712f0 0x7f81d80b4680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0097a0 tx=0x7f81cc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bb262e5aa2e61ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.965+0000 7f81d5ffb640 1 -- 172.21.15.139:0/1723674225 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc037070 con 0x7f81d80712f0 2024-04-25T09:57:07.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.965+0000 7f81d5ffb640 1 -- 172.21.15.139:0/1723674225 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc02ae50 con 0x7f81d80712f0 2024-04-25T09:57:07.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.965+0000 7f81d5ffb640 1 -- 172.21.15.139:0/1723674225 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc032660 con 0x7f81d80712f0 2024-04-25T09:57:07.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.965+0000 7f81dd5d5640 1 -- 172.21.15.139:0/1723674225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81d80712f0 msgr2=0x7f81d80b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.965+0000 7f81dd5d5640 1 --2- 172.21.15.139:0/1723674225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81d80712f0 0x7f81d80b4680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0097a0 tx=0x7f81cc02a100 comp rx=0 tx=0).stop 2024-04-25T09:57:07.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.965+0000 7f81dd5d5640 1 -- 172.21.15.139:0/1723674225 shutdown_connections 2024-04-25T09:57:07.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.965+0000 7f81dd5d5640 1 --2- 172.21.15.139:0/1723674225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81d80712f0 0x7f81d80b4680 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.965+0000 7f81dd5d5640 1 --2- 172.21.15.139:0/1723674225 >> v2:172.21.15.150:3300/0 conn(0x7f81d80719d0 0x7f81d80b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.965+0000 7f81dd5d5640 1 -- 172.21.15.139:0/1723674225 >> 172.21.15.139:0/1723674225 conn(0x7f81d806f460 msgr2=0x7f81d806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.965+0000 7f81dd5d5640 1 -- 172.21.15.139:0/1723674225 shutdown_connections 2024-04-25T09:57:07.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.965+0000 7f81dd5d5640 1 -- 172.21.15.139:0/1723674225 wait complete. 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.967+0000 7f81dd5d5640 1 Processor -- start 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.967+0000 7f81dd5d5640 1 -- start start 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.967+0000 7f81dd5d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81d80712f0 0x7f81d813de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.967+0000 7f81dd5d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81d80719d0 0x7f81d80c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.967+0000 7f81dd5d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81d8145250 0x7f81d81e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.967+0000 7f81dd5d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d8145630 con 0x7f81d80712f0 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.967+0000 7f81dd5d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80ceb60 con 0x7f81d8145250 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.967+0000 7f81dd5d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80d11a0 con 0x7f81d80719d0 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.967+0000 7f81d6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81d80712f0 0x7f81d813de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81d6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81d80712f0 0x7f81d813de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53520/0 (socket says 172.21.15.139:53520) 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81d6ffd640 1 -- 172.21.15.139:0/3975340012 learned_addr learned my addr 172.21.15.139:0/3975340012 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81d77fe640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81d8145250 0x7f81d81e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81d67fc640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81d80719d0 0x7f81d80c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81d6ffd640 1 -- 172.21.15.139:0/3975340012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81d80719d0 msgr2=0x7f81d80c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81d6ffd640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81d80719d0 0x7f81d80c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81d6ffd640 1 -- 172.21.15.139:0/3975340012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81d8145250 msgr2=0x7f81d81e51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81d6ffd640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81d8145250 0x7f81d81e51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81d6ffd640 1 -- 172.21.15.139:0/3975340012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d813e370 con 0x7f81d80712f0 2024-04-25T09:57:07.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81d6ffd640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81d80712f0 0x7f81d813de30 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f81c800c7c0 tx=0x7f81c800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81bf7fe640 1 -- 172.21.15.139:0/3975340012 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8018070 con 0x7f81d80712f0 2024-04-25T09:57:07.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81bf7fe640 1 -- 172.21.15.139:0/3975340012 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c800ce10 con 0x7f81d80712f0 2024-04-25T09:57:07.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81dd5d5640 1 -- 172.21.15.139:0/3975340012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d81e4180 con 0x7f81d80712f0 2024-04-25T09:57:07.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81bf7fe640 1 -- 172.21.15.139:0/3975340012 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c800a880 con 0x7f81d80712f0 2024-04-25T09:57:07.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.968+0000 7f81dd5d5640 1 -- 172.21.15.139:0/3975340012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d81e49c0 con 0x7f81d80712f0 2024-04-25T09:57:07.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.969+0000 7f81dd5d5640 1 -- 172.21.15.139:0/3975340012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d80cf750 con 0x7f81d80712f0 2024-04-25T09:57:07.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.970+0000 7f81bf7fe640 1 -- 172.21.15.139:0/3975340012 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81c801b070 con 0x7f81d80712f0 2024-04-25T09:57:07.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.970+0000 7f81bf7fe640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f81c406b4d0 0x7f81c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.970+0000 7f81d67fc640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f81c406b4d0 0x7f81c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.970+0000 7f81bf7fe640 1 -- 172.21.15.139:0/3975340012 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f81c8089930 con 0x7f81d80712f0 2024-04-25T09:57:07.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.970+0000 7f81d67fc640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f81c406b4d0 0x7f81c406d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f81cc034000 tx=0x7f81cc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:07.974+0000 7f81bf7fe640 1 -- 172.21.15.139:0/3975340012 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81d80cf750 con 0x7f81d80712f0 2024-04-25T09:57:08.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.144+0000 7f81dd5d5640 1 -- 172.21.15.139:0/3975340012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81d8074520 con 0x7f81d80712f0 2024-04-25T09:57:08.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.144+0000 7f81bf7fe640 1 -- 172.21.15.139:0/3975340012 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+11 (secure 0 0 0) 0x7f81d81e4180 con 0x7f81d80712f0 2024-04-25T09:57:08.147 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:57:08.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.146+0000 7f81dd5d5640 1 -- 172.21.15.139:0/3975340012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f81c406b4d0 msgr2=0x7f81c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.146+0000 7f81dd5d5640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f81c406b4d0 0x7f81c406d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f81cc034000 tx=0x7f81cc02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:08.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.146+0000 7f81dd5d5640 1 -- 172.21.15.139:0/3975340012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81d80712f0 msgr2=0x7f81d813de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.146+0000 7f81dd5d5640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81d80712f0 0x7f81d813de30 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f81c800c7c0 tx=0x7f81c800cbc0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.146+0000 7f81dd5d5640 1 -- 172.21.15.139:0/3975340012 shutdown_connections 2024-04-25T09:57:08.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.146+0000 7f81dd5d5640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f81c406b4d0 0x7f81c406d990 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.146+0000 7f81dd5d5640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81d8145250 0x7f81d81e51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.146+0000 7f81dd5d5640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81d80719d0 0x7f81d80c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.146+0000 7f81dd5d5640 1 --2- 172.21.15.139:0/3975340012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81d80712f0 0x7f81d813de30 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.146+0000 7f81dd5d5640 1 -- 172.21.15.139:0/3975340012 >> 172.21.15.139:0/3975340012 conn(0x7f81d806f460 msgr2=0x7f81d80b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:08.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.146+0000 7f81dd5d5640 1 -- 172.21.15.139:0/3975340012 shutdown_connections 2024-04-25T09:57:08.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.146+0000 7f81dd5d5640 1 -- 172.21.15.139:0/3975340012 wait complete. 2024-04-25T09:57:08.160 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:08.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.253+0000 7f99aaebd640 1 Processor -- start 2024-04-25T09:57:08.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.253+0000 7f99aaebd640 1 -- start start 2024-04-25T09:57:08.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.254+0000 7f99aaebd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a4160e90 0x7f99a40afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.254+0000 7f99aaebd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f99a406ab60 0x7f99a406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.254+0000 7f99aaebd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a40d0400 con 0x7f99a406fa70 2024-04-25T09:57:08.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.254+0000 7f99aaebd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a40d14a0 con 0x7f99a406ab60 2024-04-25T09:57:08.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.254+0000 7f99aaebd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a406b500 con 0x7f99a4160e90 2024-04-25T09:57:08.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.254+0000 7f99a8c59640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a4160e90 0x7f99a40afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.254+0000 7f99a3fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f99a406ab60 0x7f99a406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.254+0000 7f99a8c59640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a4160e90 0x7f99a40afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:44820/0 (socket says 172.21.15.139:44820) 2024-04-25T09:57:08.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.254+0000 7f99a8c59640 1 -- 172.21.15.139:0/2252346875 learned_addr learned my addr 172.21.15.139:0/2252346875 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:08.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.254+0000 7f99a945a640 1 --1- 172.21.15.139:0/2252346875 >> v1:172.21.15.150:6789/0 conn(0x7f99a406fa70 0x7f99a406fec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:39422/0 (socket says 172.21.15.139:39422) 2024-04-25T09:57:08.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.255+0000 7f99a8c59640 1 -- 172.21.15.139:0/2252346875 >> v2:172.21.15.150:3300/0 conn(0x7f99a406ab60 msgr2=0x7f99a406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:08.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.255+0000 7f99a8c59640 1 --2- 172.21.15.139:0/2252346875 >> v2:172.21.15.150:3300/0 conn(0x7f99a406ab60 0x7f99a406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.255+0000 7f99a8c59640 1 -- 172.21.15.139:0/2252346875 >> v1:172.21.15.150:6789/0 conn(0x7f99a406fa70 legacy=0x7f99a406fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.255+0000 7f99a8c59640 1 -- 172.21.15.139:0/2252346875 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a40d06e0 con 0x7f99a4160e90 2024-04-25T09:57:08.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.255+0000 7f99a8c59640 1 --2- 172.21.15.139:0/2252346875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a4160e90 0x7f99a40afac0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9994009870 tx=0x7f999402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9f0e74fc6043d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.256+0000 7f99a37fe640 1 -- 172.21.15.139:0/2252346875 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994037070 con 0x7f99a4160e90 2024-04-25T09:57:08.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.256+0000 7f99a37fe640 1 -- 172.21.15.139:0/2252346875 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9994032440 con 0x7f99a4160e90 2024-04-25T09:57:08.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.256+0000 7f99a37fe640 1 -- 172.21.15.139:0/2252346875 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994032810 con 0x7f99a4160e90 2024-04-25T09:57:08.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.256+0000 7f99aaebd640 1 -- 172.21.15.139:0/2252346875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a4160e90 msgr2=0x7f99a40afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.256+0000 7f99aaebd640 1 --2- 172.21.15.139:0/2252346875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a4160e90 0x7f99a40afac0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9994009870 tx=0x7f999402a270 comp rx=0 tx=0).stop 2024-04-25T09:57:08.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.256+0000 7f99aaebd640 1 -- 172.21.15.139:0/2252346875 shutdown_connections 2024-04-25T09:57:08.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.256+0000 7f99aaebd640 1 --2- 172.21.15.139:0/2252346875 >> v2:172.21.15.150:3300/0 conn(0x7f99a406ab60 0x7f99a406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.256+0000 7f99aaebd640 1 --2- 172.21.15.139:0/2252346875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a4160e90 0x7f99a40afac0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.256+0000 7f99aaebd640 1 -- 172.21.15.139:0/2252346875 >> 172.21.15.139:0/2252346875 conn(0x7f99a40b23f0 msgr2=0x7f99a40b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:08.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.256+0000 7f99aaebd640 1 -- 172.21.15.139:0/2252346875 shutdown_connections 2024-04-25T09:57:08.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.256+0000 7f99aaebd640 1 -- 172.21.15.139:0/2252346875 wait complete. 2024-04-25T09:57:08.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.257+0000 7f99aaebd640 1 Processor -- start 2024-04-25T09:57:08.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.257+0000 7f99aaebd640 1 -- start start 2024-04-25T09:57:08.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.258+0000 7f99aaebd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a406ab60 0x7f99a414d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.258+0000 7f99aaebd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a406fa70 0x7f99a414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.258+0000 7f99aaebd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a4160e90 0x7f99a41e9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.258+0000 7f99aaebd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40d0f80 con 0x7f99a406fa70 2024-04-25T09:57:08.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.258+0000 7f99aaebd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40d0ca0 con 0x7f99a4160e90 2024-04-25T09:57:08.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.258+0000 7f99aaebd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40b0000 con 0x7f99a406ab60 2024-04-25T09:57:08.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.258+0000 7f99a3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a406fa70 0x7f99a414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.258+0000 7f99a3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a406fa70 0x7f99a414f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53530/0 (socket says 172.21.15.139:53530) 2024-04-25T09:57:08.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.258+0000 7f99a3fff640 1 -- 172.21.15.139:0/1715369956 learned_addr learned my addr 172.21.15.139:0/1715369956 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:08.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.258+0000 7f99a8c59640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a406ab60 0x7f99a414d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.258+0000 7f99a945a640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a4160e90 0x7f99a41e9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.259+0000 7f99a3fff640 1 -- 172.21.15.139:0/1715369956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a406ab60 msgr2=0x7f99a414d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.259+0000 7f99a3fff640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a406ab60 0x7f99a414d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.259+0000 7f99a3fff640 1 -- 172.21.15.139:0/1715369956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a4160e90 msgr2=0x7f99a41e9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.259+0000 7f99a3fff640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a4160e90 0x7f99a41e9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.259+0000 7f99a3fff640 1 -- 172.21.15.139:0/1715369956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a414f8b0 con 0x7f99a406fa70 2024-04-25T09:57:08.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.259+0000 7f99a3fff640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a406fa70 0x7f99a414f370 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f999800e9b0 tx=0x7f999800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.259+0000 7f99a17fa640 1 -- 172.21.15.139:0/1715369956 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998017070 con 0x7f99a406fa70 2024-04-25T09:57:08.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.259+0000 7f99a17fa640 1 -- 172.21.15.139:0/1715369956 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999800c9a0 con 0x7f99a406fa70 2024-04-25T09:57:08.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.259+0000 7f99aaebd640 1 -- 172.21.15.139:0/1715369956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a41532c0 con 0x7f99a406fa70 2024-04-25T09:57:08.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.259+0000 7f99a17fa640 1 -- 172.21.15.139:0/1715369956 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999800cd90 con 0x7f99a406fa70 2024-04-25T09:57:08.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.259+0000 7f99aaebd640 1 -- 172.21.15.139:0/1715369956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a4153880 con 0x7f99a406fa70 2024-04-25T09:57:08.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.260+0000 7f99aaebd640 1 -- 172.21.15.139:0/1715369956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a4074520 con 0x7f99a406fa70 2024-04-25T09:57:08.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.260+0000 7f99a17fa640 1 -- 172.21.15.139:0/1715369956 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f999801a070 con 0x7f99a406fa70 2024-04-25T09:57:08.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.261+0000 7f99a17fa640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f999006b4d0 0x7f999006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.261+0000 7f99a8c59640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f999006b4d0 0x7f999006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.261+0000 7f99a17fa640 1 -- 172.21.15.139:0/1715369956 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f999808a2d0 con 0x7f99a406fa70 2024-04-25T09:57:08.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.261+0000 7f99a8c59640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f999006b4d0 0x7f999006d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9994034000 tx=0x7f999402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.265+0000 7f99a17fa640 1 -- 172.21.15.139:0/1715369956 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99a414f8b0 con 0x7f99a406fa70 2024-04-25T09:57:08.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.471+0000 7f99aaebd640 1 -- 172.21.15.139:0/1715369956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99a40728b0 con 0x7f99a406fa70 2024-04-25T09:57:08.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.471+0000 7f99a17fa640 1 -- 172.21.15.139:0/1715369956 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f9998058970 con 0x7f99a406fa70 2024-04-25T09:57:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:08.475 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/829268166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":829268166},{"type":"v1","addr":"172.21.15.150:6835","nonce":829268166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3075039902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3075039902},{"type":"v1","addr":"172.21.15.150:6834","nonce":3075039902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/627726069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":627726069},{"type":"v1","addr":"172.21.15.139:6836","nonce":627726069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:50.626292+0000","modified":"2024-04-25T09:56:39.327977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.139:6837/3541444834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3541444834},{"type":"v1","addr":"172.21.15.139:6837","nonce":3541444834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:57:08.475 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 30 2024-04-25T09:57:08.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.473+0000 7f99aaebd640 1 -- 172.21.15.139:0/1715369956 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f999006b4d0 msgr2=0x7f999006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.473+0000 7f99aaebd640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f999006b4d0 0x7f999006d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9994034000 tx=0x7f999402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:08.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.473+0000 7f99aaebd640 1 -- 172.21.15.139:0/1715369956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a406fa70 msgr2=0x7f99a414f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.473+0000 7f99aaebd640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a406fa70 0x7f99a414f370 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f999800e9b0 tx=0x7f999800ee70 comp rx=0 tx=0).stop 2024-04-25T09:57:08.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.473+0000 7f99aaebd640 1 -- 172.21.15.139:0/1715369956 shutdown_connections 2024-04-25T09:57:08.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.473+0000 7f99aaebd640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a406fa70 0x7f99a414f370 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.473+0000 7f99aaebd640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f999006b4d0 0x7f999006d990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.473+0000 7f99aaebd640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a4160e90 0x7f99a41e9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.473+0000 7f99aaebd640 1 --2- 172.21.15.139:0/1715369956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99a406ab60 0x7f99a414d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.473+0000 7f99aaebd640 1 -- 172.21.15.139:0/1715369956 >> 172.21.15.139:0/1715369956 conn(0x7f99a40b23f0 msgr2=0x7f99a40b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:08.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.473+0000 7f99aaebd640 1 -- 172.21.15.139:0/1715369956 shutdown_connections 2024-04-25T09:57:08.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.473+0000 7f99aaebd640 1 -- 172.21.15.139:0/1715369956 wait complete. 2024-04-25T09:57:08.488 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:08.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.580+0000 7ff3cc79b640 1 Processor -- start 2024-04-25T09:57:08.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.580+0000 7ff3cc79b640 1 -- start start 2024-04-25T09:57:08.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.581+0000 7ff3cc79b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff3c40b0750 0x7ff3c40b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.581+0000 7ff3cc79b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c40b1070 con 0x7ff3c40d0730 2024-04-25T09:57:08.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.581+0000 7ff3cc79b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c40ccb70 con 0x7ff3c40d0b90 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.581+0000 7ff3cc79b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c40cf970 con 0x7ff3c40b0750 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9d36640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff3c40d0730 0x7ff3c40cbe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41210/0 (socket says 172.21.15.139:41210) 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9d36640 1 -- 172.21.15.139:0/2567115434 learned_addr learned my addr 172.21.15.139:0/2567115434 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3ca537640 1 --2- 172.21.15.139:0/2567115434 >> v2:172.21.15.150:3300/0 conn(0x7ff3c40b0750 0x7ff3c40b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371222727 0 0) 0x7ff3c40b1070 con 0x7ff3c40d0730 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3b0003520 con 0x7ff3c40d0730 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2470835693 0 0) 0x7ff3b0003520 con 0x7ff3c40d0730 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 >> v2:172.21.15.150:3300/0 conn(0x7ff3c40b0750 msgr2=0x7ff3c40b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 --2- 172.21.15.139:0/2567115434 >> v2:172.21.15.150:3300/0 conn(0x7ff3c40b0750 0x7ff3c40b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 >> v1:172.21.15.150:6789/0 conn(0x7ff3c40d0b90 legacy=0x7ff3c40b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c40cfc50 con 0x7ff3c40d0730 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7ff3b4001690 con 0x7ff3c40d0730 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 >> v1:172.21.15.139:6789/0 conn(0x7ff3c40d0730 legacy=0x7ff3c40cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 --2- 172.21.15.139:0/2567115434 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3b00039b0 0x7ff3b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 --2- 172.21.15.139:0/2567115434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0004800 0x7ff3b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 --2- 172.21.15.139:0/2567115434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b00051c0 0x7ff3b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40cf970 con 0x7ff3b00051c0 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b000c060 con 0x7ff3b0004800 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b000c260 con 0x7ff3b00039b0 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3cad38640 1 --2- 172.21.15.139:0/2567115434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b00051c0 0x7ff3b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3c9d36640 1 --2- 172.21.15.139:0/2567115434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0004800 0x7ff3b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.582+0000 7ff3ca537640 1 --2- 172.21.15.139:0/2567115434 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3b00039b0 0x7ff3b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.583+0000 7ff3cad38640 1 -- 172.21.15.139:0/2567115434 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3b00039b0 msgr2=0x7ff3b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.583+0000 7ff3cad38640 1 --2- 172.21.15.139:0/2567115434 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3b00039b0 0x7ff3b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.583+0000 7ff3cad38640 1 -- 172.21.15.139:0/2567115434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0004800 msgr2=0x7ff3b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.583+0000 7ff3cad38640 1 --2- 172.21.15.139:0/2567115434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0004800 0x7ff3b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.583+0000 7ff3cad38640 1 -- 172.21.15.139:0/2567115434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b0003520 con 0x7ff3b00051c0 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.583+0000 7ff3cad38640 1 -- 172.21.15.139:0/2567115434 reap_dead start 2024-04-25T09:57:08.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.583+0000 7ff3cad38640 1 --2- 172.21.15.139:0/2567115434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b00051c0 0x7ff3b000b9c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40cbdf0 tx=0x7ff3b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde242dacba0c7fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.583+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc001a70 con 0x7ff3b00051c0 2024-04-25T09:57:08.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.583+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc02ae10 con 0x7ff3b00051c0 2024-04-25T09:57:08.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.583+0000 7ff3c9535640 1 -- 172.21.15.139:0/2567115434 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc003b90 con 0x7ff3b00051c0 2024-04-25T09:57:08.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.584+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2567115434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b00051c0 msgr2=0x7ff3b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.584+0000 7ff3cc79b640 1 --2- 172.21.15.139:0/2567115434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b00051c0 0x7ff3b000b9c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40cbdf0 tx=0x7ff3b0005830 comp rx=0 tx=0).stop 2024-04-25T09:57:08.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.584+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2567115434 shutdown_connections 2024-04-25T09:57:08.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.584+0000 7ff3cc79b640 1 --2- 172.21.15.139:0/2567115434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b00051c0 0x7ff3b000b9c0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.584+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2567115434 >> 172.21.15.139:0/2567115434 conn(0x7ff3c40b3380 msgr2=0x7ff3c40b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:08.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.584+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2567115434 shutdown_connections 2024-04-25T09:57:08.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.584+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2567115434 wait complete. 2024-04-25T09:57:08.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.585+0000 7ff3cc79b640 1 Processor -- start 2024-04-25T09:57:08.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.585+0000 7ff3cc79b640 1 -- start start 2024-04-25T09:57:08.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cc79b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3b00051c0 0x7ff3c4147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cc79b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c41535c0 0x7ff3c40c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cc79b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3c40c9e50 0x7ff3c4151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cc79b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c415bcc0 con 0x7ff3c40c9e50 2024-04-25T09:57:08.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cc79b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40d0210 con 0x7ff3c41535c0 2024-04-25T09:57:08.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cc79b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c415be40 con 0x7ff3b00051c0 2024-04-25T09:57:08.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cad38640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3c40c9e50 0x7ff3c4151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cad38640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3c40c9e50 0x7ff3c4151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53556/0 (socket says 172.21.15.139:53556) 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cad38640 1 -- 172.21.15.139:0/2774592072 learned_addr learned my addr 172.21.15.139:0/2774592072 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3ca537640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3b00051c0 0x7ff3c4147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3c9d36640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c41535c0 0x7ff3c40c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cad38640 1 -- 172.21.15.139:0/2774592072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3b00051c0 msgr2=0x7ff3c4147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cad38640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3b00051c0 0x7ff3c4147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cad38640 1 -- 172.21.15.139:0/2774592072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c41535c0 msgr2=0x7ff3c40c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cad38640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c41535c0 0x7ff3c40c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cad38640 1 -- 172.21.15.139:0/2774592072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c4151050 con 0x7ff3c40c9e50 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3c9d36640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c41535c0 0x7ff3c40c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.586+0000 7ff3cad38640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3c40c9e50 0x7ff3c4151940 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc001f10 tx=0x7ff3bc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.587+0000 7ff3baffd640 1 -- 172.21.15.139:0/2774592072 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc001a70 con 0x7ff3c40c9e50 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.587+0000 7ff3baffd640 1 -- 172.21.15.139:0/2774592072 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc02ac10 con 0x7ff3c40c9e50 2024-04-25T09:57:08.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.587+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2774592072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c4152520 con 0x7ff3c40c9e50 2024-04-25T09:57:08.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.587+0000 7ff3baffd640 1 -- 172.21.15.139:0/2774592072 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc005770 con 0x7ff3c40c9e50 2024-04-25T09:57:08.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.587+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2774592072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c4152ae0 con 0x7ff3c40c9e50 2024-04-25T09:57:08.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.588+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2774592072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c4074520 con 0x7ff3c40c9e50 2024-04-25T09:57:08.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.588+0000 7ff3baffd640 1 -- 172.21.15.139:0/2774592072 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3bc032440 con 0x7ff3c40c9e50 2024-04-25T09:57:08.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.588+0000 7ff3baffd640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3a806b4d0 0x7ff3a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.589+0000 7ff3baffd640 1 -- 172.21.15.139:0/2774592072 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7ff3bc0a9720 con 0x7ff3c40c9e50 2024-04-25T09:57:08.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.589+0000 7ff3ca537640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3a806b4d0 0x7ff3a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.589+0000 7ff3ca537640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3a806b4d0 0x7ff3a806d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009a80 tx=0x7ff3c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.593+0000 7ff3baffd640 1 -- 172.21.15.139:0/2774592072 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3c4151050 con 0x7ff3c40c9e50 2024-04-25T09:57:08.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.762+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2774592072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3c40b3f40 con 0x7ff3c40c9e50 2024-04-25T09:57:08.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.763+0000 7ff3baffd640 1 -- 172.21.15.139:0/2774592072 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17173 (secure 0 0 0) 0x7ff3bc077d30 con 0x7ff3c40c9e50 2024-04-25T09:57:08.766 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:08.767 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":64,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:55:57.317641+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:48.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:49.597505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.22","mappings":[{"from":7,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:08.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.765+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2774592072 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3a806b4d0 msgr2=0x7ff3a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.765+0000 7ff3cc79b640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3a806b4d0 0x7ff3a806d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009a80 tx=0x7ff3c0008040 comp rx=0 tx=0).stop 2024-04-25T09:57:08.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.765+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2774592072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3c40c9e50 msgr2=0x7ff3c4151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.765+0000 7ff3cc79b640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3c40c9e50 0x7ff3c4151940 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc001f10 tx=0x7ff3bc0014a0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.765+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2774592072 shutdown_connections 2024-04-25T09:57:08.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.765+0000 7ff3cc79b640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3c40c9e50 0x7ff3c4151940 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.765+0000 7ff3cc79b640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff3a806b4d0 0x7ff3a806d990 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.765+0000 7ff3cc79b640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3c41535c0 0x7ff3c40c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.765+0000 7ff3cc79b640 1 --2- 172.21.15.139:0/2774592072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3b00051c0 0x7ff3c4147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.765+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2774592072 >> 172.21.15.139:0/2774592072 conn(0x7ff3c40b3380 msgr2=0x7ff3c40d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:08.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.765+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2774592072 shutdown_connections 2024-04-25T09:57:08.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.765+0000 7ff3cc79b640 1 -- 172.21.15.139:0/2774592072 wait complete. 2024-04-25T09:57:08.779 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:57:08.779 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:08.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.873+0000 7f4b7fa5d640 1 Processor -- start 2024-04-25T09:57:08.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.873+0000 7f4b7fa5d640 1 -- start start 2024-04-25T09:57:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7fa5d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b7815efb0 0x7f4b780d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7fa5d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4b780cf900 0x7f4b7806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7fa5d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4b7806cd80 0x7f4b78099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7fa5d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b780d0a80 con 0x7f4b7806cd80 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7fa5d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b780cfe70 con 0x7f4b780cf900 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7fa5d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b780d0150 con 0x7f4b7815efb0 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7dffa640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4b7806cd80 0x7f4b78099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7dffa640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4b7806cd80 0x7f4b78099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53560/0 (socket says 172.21.15.139:53560) 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7dffa640 1 -- 172.21.15.139:0/1491398934 learned_addr learned my addr 172.21.15.139:0/1491398934 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7d7f9640 1 --2- 172.21.15.139:0/1491398934 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b7815efb0 0x7f4b780d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7cff8640 1 --2- 172.21.15.139:0/1491398934 >> v2:172.21.15.150:3300/0 conn(0x7f4b780cf900 0x7f4b7806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7dffa640 1 -- 172.21.15.139:0/1491398934 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b7815efb0 msgr2=0x7f4b780d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7dffa640 1 --2- 172.21.15.139:0/1491398934 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b7815efb0 0x7f4b780d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.874+0000 7f4b7dffa640 1 -- 172.21.15.139:0/1491398934 >> v2:172.21.15.150:3300/0 conn(0x7f4b780cf900 msgr2=0x7f4b7806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.875+0000 7f4b7dffa640 1 --2- 172.21.15.139:0/1491398934 >> v2:172.21.15.150:3300/0 conn(0x7f4b780cf900 0x7f4b7806c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.875+0000 7f4b7dffa640 1 -- 172.21.15.139:0/1491398934 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b780d0430 con 0x7f4b7806cd80 2024-04-25T09:57:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.875+0000 7f4b7cff8640 1 --2- 172.21.15.139:0/1491398934 >> v2:172.21.15.150:3300/0 conn(0x7f4b780cf900 0x7f4b7806c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.875+0000 7f4b7dffa640 1 --2- 172.21.15.139:0/1491398934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b7806cd80 0x7f4b78099a50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4b78072a60 tx=0x7f4b7002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775a632dc0e2b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.875+0000 7f4b6ffff640 1 -- 172.21.15.139:0/1491398934 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70036070 con 0x7f4b7806cd80 2024-04-25T09:57:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.875+0000 7f4b6ffff640 1 -- 172.21.15.139:0/1491398934 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b70031600 con 0x7f4b7806cd80 2024-04-25T09:57:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.875+0000 7f4b6ffff640 1 -- 172.21.15.139:0/1491398934 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b700319d0 con 0x7f4b7806cd80 2024-04-25T09:57:08.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.876+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/1491398934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b7806cd80 msgr2=0x7f4b78099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.876+0000 7f4b7fa5d640 1 --2- 172.21.15.139:0/1491398934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b7806cd80 0x7f4b78099a50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4b78072a60 tx=0x7f4b7002a610 comp rx=0 tx=0).stop 2024-04-25T09:57:08.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.876+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/1491398934 shutdown_connections 2024-04-25T09:57:08.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.876+0000 7f4b7fa5d640 1 --2- 172.21.15.139:0/1491398934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b7806cd80 0x7f4b78099a50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.876+0000 7f4b7fa5d640 1 --2- 172.21.15.139:0/1491398934 >> v2:172.21.15.150:3300/0 conn(0x7f4b780cf900 0x7f4b7806c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.876+0000 7f4b7fa5d640 1 --2- 172.21.15.139:0/1491398934 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b7815efb0 0x7f4b780d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.876+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/1491398934 >> 172.21.15.139:0/1491398934 conn(0x7f4b780d1790 msgr2=0x7f4b780d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:08.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.876+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/1491398934 shutdown_connections 2024-04-25T09:57:08.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.876+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/1491398934 wait complete. 2024-04-25T09:57:08.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.877+0000 7f4b7fa5d640 1 Processor -- start 2024-04-25T09:57:08.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.877+0000 7f4b7fa5d640 1 -- start start 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7fa5d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b7806cd80 0x7f4b78154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7fa5d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b780cf900 0x7f4b78161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7fa5d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b7815efb0 0x7f4b780b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7fa5d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7809a110 con 0x7f4b780cf900 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7fa5d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b78099f90 con 0x7f4b7806cd80 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7fa5d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7809a290 con 0x7f4b7815efb0 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7cff8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b780cf900 0x7f4b78161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7cff8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b780cf900 0x7f4b78161ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53564/0 (socket says 172.21.15.139:53564) 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7cff8640 1 -- 172.21.15.139:0/3200493403 learned_addr learned my addr 172.21.15.139:0/3200493403 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7d7f9640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b7806cd80 0x7f4b78154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7dffa640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b7815efb0 0x7f4b780b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7cff8640 1 -- 172.21.15.139:0/3200493403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b7815efb0 msgr2=0x7f4b780b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7cff8640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b7815efb0 0x7f4b780b6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7cff8640 1 -- 172.21.15.139:0/3200493403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b7806cd80 msgr2=0x7f4b78154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7cff8640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b7806cd80 0x7f4b78154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.878+0000 7f4b7cff8640 1 -- 172.21.15.139:0/3200493403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7815fd70 con 0x7f4b780cf900 2024-04-25T09:57:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.879+0000 7f4b7cff8640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b780cf900 0x7f4b78161ab0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c7f0 tx=0x7f4b6800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.879+0000 7f4b6dffb640 1 -- 172.21.15.139:0/3200493403 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68015070 con 0x7f4b780cf900 2024-04-25T09:57:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.879+0000 7f4b6dffb640 1 -- 172.21.15.139:0/3200493403 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b68007a30 con 0x7f4b780cf900 2024-04-25T09:57:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.879+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/3200493403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b78154bc0 con 0x7f4b780cf900 2024-04-25T09:57:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.879+0000 7f4b6dffb640 1 -- 172.21.15.139:0/3200493403 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68007e00 con 0x7f4b780cf900 2024-04-25T09:57:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.879+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/3200493403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b781602d0 con 0x7f4b780cf900 2024-04-25T09:57:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.880+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/3200493403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b78074520 con 0x7f4b780cf900 2024-04-25T09:57:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.880+0000 7f4b6dffb640 1 -- 172.21.15.139:0/3200493403 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b68018070 con 0x7f4b780cf900 2024-04-25T09:57:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.881+0000 7f4b6dffb640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.881+0000 7f4b6dffb640 1 -- 172.21.15.139:0/3200493403 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f4b6808a290 con 0x7f4b780cf900 2024-04-25T09:57:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.881+0000 7f4b7d7f9640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.881+0000 7f4b7d7f9640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4b7815bf70 tx=0x7f4b74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:08.885+0000 7f4b6dffb640 1 -- 172.21.15.139:0/3200493403 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b7815fd70 con 0x7f4b780cf900 2024-04-25T09:57:09.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.090+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/3200493403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b7809ae50 con 0x7f4b780cf900 2024-04-25T09:57:09.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.091+0000 7f4b6dffb640 1 -- 172.21.15.139:0/3200493403 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f4b680588a0 con 0x7f4b780cf900 2024-04-25T09:57:09.094 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:09.094 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/829268166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":829268166},{"type":"v1","addr":"172.21.15.150:6835","nonce":829268166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3075039902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3075039902},{"type":"v1","addr":"172.21.15.150:6834","nonce":3075039902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/627726069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":627726069},{"type":"v1","addr":"172.21.15.139:6836","nonce":627726069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:50.626292+0000","modified":"2024-04-25T09:56:39.327977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.139:6837/3541444834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3541444834},{"type":"v1","addr":"172.21.15.139:6837","nonce":3541444834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 30 2024-04-25T09:57:09.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.093+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/3200493403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4b5c06b4d0 msgr2=0x7f4b5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.093+0000 7f4b7fa5d640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4b7815bf70 tx=0x7f4b74008040 comp rx=0 tx=0).stop 2024-04-25T09:57:09.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.093+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/3200493403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b780cf900 msgr2=0x7f4b78161ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.093+0000 7f4b7fa5d640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b780cf900 0x7f4b78161ab0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c7f0 tx=0x7f4b6800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.093+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/3200493403 shutdown_connections 2024-04-25T09:57:09.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.093+0000 7f4b7fa5d640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b780cf900 0x7f4b78161ab0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.093+0000 7f4b7fa5d640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.093+0000 7f4b7fa5d640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b7815efb0 0x7f4b780b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.093+0000 7f4b7fa5d640 1 --2- 172.21.15.139:0/3200493403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b7806cd80 0x7f4b78154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.093+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/3200493403 >> 172.21.15.139:0/3200493403 conn(0x7f4b780d1790 msgr2=0x7f4b78070720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:09.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.093+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/3200493403 shutdown_connections 2024-04-25T09:57:09.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.093+0000 7f4b7fa5d640 1 -- 172.21.15.139:0/3200493403 wait complete. 2024-04-25T09:57:09.107 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:09.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.200+0000 7efe2bd63640 1 Processor -- start 2024-04-25T09:57:09.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.200+0000 7efe2bd63640 1 -- start start 2024-04-25T09:57:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.201+0000 7efe2bd63640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe241611e0 0x7efe240b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.201+0000 7efe2bd63640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efe240cea00 0x7efe2406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.201+0000 7efe2bd63640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efe240cee60 0x7efe2406c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.201+0000 7efe2bd63640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe240d1fa0 con 0x7efe240cee60 2024-04-25T09:57:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.201+0000 7efe2bd63640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe240d2560 con 0x7efe240cea00 2024-04-25T09:57:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.201+0000 7efe2bd63640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe240d4740 con 0x7efe241611e0 2024-04-25T09:57:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.201+0000 7efe2a300640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efe240cee60 0x7efe2406c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.201+0000 7efe2a300640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efe240cee60 0x7efe2406c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53568/0 (socket says 172.21.15.139:53568) 2024-04-25T09:57:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.201+0000 7efe2a300640 1 -- 172.21.15.139:0/1256867969 learned_addr learned my addr 172.21.15.139:0/1256867969 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.201+0000 7efe29aff640 1 --2- 172.21.15.139:0/1256867969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe241611e0 0x7efe240b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.202+0000 7efe2a300640 1 -- 172.21.15.139:0/1256867969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe241611e0 msgr2=0x7efe240b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.202+0000 7efe292fe640 1 --2- 172.21.15.139:0/1256867969 >> v2:172.21.15.150:3300/0 conn(0x7efe240cea00 0x7efe2406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.202+0000 7efe2a300640 1 --2- 172.21.15.139:0/1256867969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe241611e0 0x7efe240b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.202+0000 7efe2a300640 1 -- 172.21.15.139:0/1256867969 >> v2:172.21.15.150:3300/0 conn(0x7efe240cea00 msgr2=0x7efe2406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.202+0000 7efe2a300640 1 --2- 172.21.15.139:0/1256867969 >> v2:172.21.15.150:3300/0 conn(0x7efe240cea00 0x7efe2406c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.202+0000 7efe2a300640 1 -- 172.21.15.139:0/1256867969 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe240b2770 con 0x7efe240cee60 2024-04-25T09:57:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.202+0000 7efe29aff640 1 --2- 172.21.15.139:0/1256867969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe241611e0 0x7efe240b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:09.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.202+0000 7efe2a300640 1 --2- 172.21.15.139:0/1256867969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe240cee60 0x7efe2406c5b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7efe20009e50 tx=0x7efe2002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72976708197e9e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:09.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.202+0000 7efe28afd640 1 -- 172.21.15.139:0/1256867969 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20036070 con 0x7efe240cee60 2024-04-25T09:57:09.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.202+0000 7efe28afd640 1 -- 172.21.15.139:0/1256867969 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe20031600 con 0x7efe240cee60 2024-04-25T09:57:09.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.202+0000 7efe28afd640 1 -- 172.21.15.139:0/1256867969 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe200319d0 con 0x7efe240cee60 2024-04-25T09:57:09.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.203+0000 7efe2bd63640 1 -- 172.21.15.139:0/1256867969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe240cee60 msgr2=0x7efe2406c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.203+0000 7efe2bd63640 1 --2- 172.21.15.139:0/1256867969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe240cee60 0x7efe2406c5b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7efe20009e50 tx=0x7efe2002a6c0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.203+0000 7efe2bd63640 1 -- 172.21.15.139:0/1256867969 shutdown_connections 2024-04-25T09:57:09.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.203+0000 7efe2bd63640 1 --2- 172.21.15.139:0/1256867969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe240cee60 0x7efe2406c5b0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.203+0000 7efe2bd63640 1 --2- 172.21.15.139:0/1256867969 >> v2:172.21.15.150:3300/0 conn(0x7efe240cea00 0x7efe2406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.203+0000 7efe2bd63640 1 --2- 172.21.15.139:0/1256867969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe241611e0 0x7efe240b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.203+0000 7efe2bd63640 1 -- 172.21.15.139:0/1256867969 >> 172.21.15.139:0/1256867969 conn(0x7efe240b5b00 msgr2=0x7efe240d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:09.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.203+0000 7efe2bd63640 1 -- 172.21.15.139:0/1256867969 shutdown_connections 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.203+0000 7efe2bd63640 1 -- 172.21.15.139:0/1256867969 wait complete. 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.204+0000 7efe2bd63640 1 Processor -- start 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.204+0000 7efe2bd63640 1 -- start start 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe2bd63640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe240cea00 0x7efe2415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe2bd63640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe240cee60 0x7efe241f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe2bd63640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe241611e0 0x7efe24160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe2bd63640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe240d21f0 con 0x7efe240cee60 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe2bd63640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe240b2d30 con 0x7efe240cea00 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe2bd63640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe241615c0 con 0x7efe241611e0 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe292fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe240cee60 0x7efe241f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe292fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe240cee60 0x7efe241f39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53576/0 (socket says 172.21.15.139:53576) 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe292fe640 1 -- 172.21.15.139:0/1921644440 learned_addr learned my addr 172.21.15.139:0/1921644440 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe29aff640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe240cea00 0x7efe2415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe2a300640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe241611e0 0x7efe24160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe292fe640 1 -- 172.21.15.139:0/1921644440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe241611e0 msgr2=0x7efe24160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe292fe640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe241611e0 0x7efe24160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe292fe640 1 -- 172.21.15.139:0/1921644440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe240cea00 msgr2=0x7efe2415bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe292fe640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe240cea00 0x7efe2415bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe292fe640 1 -- 172.21.15.139:0/1921644440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2415f7a0 con 0x7efe240cee60 2024-04-25T09:57:09.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.205+0000 7efe292fe640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe240cee60 0x7efe241f39f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7efe1400c7f0 tx=0x7efe1400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:09.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.206+0000 7efe1a7fc640 1 -- 172.21.15.139:0/1921644440 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14015070 con 0x7efe240cee60 2024-04-25T09:57:09.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.206+0000 7efe1a7fc640 1 -- 172.21.15.139:0/1921644440 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe14004220 con 0x7efe240cee60 2024-04-25T09:57:09.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.206+0000 7efe2bd63640 1 -- 172.21.15.139:0/1921644440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe241f43d0 con 0x7efe240cee60 2024-04-25T09:57:09.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.206+0000 7efe1a7fc640 1 -- 172.21.15.139:0/1921644440 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14007620 con 0x7efe240cee60 2024-04-25T09:57:09.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.206+0000 7efe2bd63640 1 -- 172.21.15.139:0/1921644440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2415fd00 con 0x7efe240cee60 2024-04-25T09:57:09.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.207+0000 7efe2bd63640 1 -- 172.21.15.139:0/1921644440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2415ff00 con 0x7efe240cee60 2024-04-25T09:57:09.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.207+0000 7efe1a7fc640 1 -- 172.21.15.139:0/1921644440 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe140079d0 con 0x7efe240cee60 2024-04-25T09:57:09.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.207+0000 7efe1a7fc640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe0806b4d0 0x7efe0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.207+0000 7efe1a7fc640 1 -- 172.21.15.139:0/1921644440 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7efe1408a500 con 0x7efe240cee60 2024-04-25T09:57:09.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.208+0000 7efe29aff640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe0806b4d0 0x7efe0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.208+0000 7efe29aff640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe0806b4d0 0x7efe0806d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00a950 tx=0x7efe1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:09.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.212+0000 7efe1a7fc640 1 -- 172.21.15.139:0/1921644440 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe2415ff00 con 0x7efe240cee60 2024-04-25T09:57:09.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.415+0000 7efe2bd63640 1 -- 172.21.15.139:0/1921644440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe24074520 con 0x7efe240cee60 2024-04-25T09:57:09.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.416+0000 7efe1a7fc640 1 -- 172.21.15.139:0/1921644440 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4946 (secure 0 0 0) 0x7efe2415f7a0 con 0x7efe240cee60 2024-04-25T09:57:09.419 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:09.419 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/829268166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":829268166},{"type":"v1","addr":"172.21.15.150:6835","nonce":829268166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3075039902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3075039902},{"type":"v1","addr":"172.21.15.150:6834","nonce":3075039902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/627726069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":627726069},{"type":"v1","addr":"172.21.15.139:6836","nonce":627726069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:50.626292+0000","modified":"2024-04-25T09:56:39.327977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.139:6837/3541444834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3541444834},{"type":"v1","addr":"172.21.15.139:6837","nonce":3541444834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:57:09.419 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 30 2024-04-25T09:57:09.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.417+0000 7efe2bd63640 1 -- 172.21.15.139:0/1921644440 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe0806b4d0 msgr2=0x7efe0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.417+0000 7efe2bd63640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe0806b4d0 0x7efe0806d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00a950 tx=0x7efe1c008040 comp rx=0 tx=0).stop 2024-04-25T09:57:09.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.417+0000 7efe2bd63640 1 -- 172.21.15.139:0/1921644440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe240cee60 msgr2=0x7efe241f39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.417+0000 7efe2bd63640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe240cee60 0x7efe241f39f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7efe1400c7f0 tx=0x7efe1400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.418+0000 7efe2bd63640 1 -- 172.21.15.139:0/1921644440 shutdown_connections 2024-04-25T09:57:09.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.418+0000 7efe2bd63640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe240cee60 0x7efe241f39f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.418+0000 7efe2bd63640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe0806b4d0 0x7efe0806d990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.418+0000 7efe2bd63640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe241611e0 0x7efe24160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.418+0000 7efe2bd63640 1 --2- 172.21.15.139:0/1921644440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe240cea00 0x7efe2415bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.418+0000 7efe2bd63640 1 -- 172.21.15.139:0/1921644440 >> 172.21.15.139:0/1921644440 conn(0x7efe240b5b00 msgr2=0x7efe2406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:09.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.418+0000 7efe2bd63640 1 -- 172.21.15.139:0/1921644440 shutdown_connections 2024-04-25T09:57:09.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.418+0000 7efe2bd63640 1 -- 172.21.15.139:0/1921644440 wait complete. 2024-04-25T09:57:09.432 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.526+0000 7f9b8bd6a640 1 Processor -- start 2024-04-25T09:57:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.527+0000 7f9b8bd6a640 1 -- start start 2024-04-25T09:57:09.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.527+0000 7f9b8bd6a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8415bdc0 0x7f9b8415c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.527+0000 7f9b8bd6a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9b840b1740 0x7f9b840b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.527+0000 7f9b8bd6a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8415c880 con 0x7f9b84155e30 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.527+0000 7f9b8bd6a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8415d690 con 0x7f9b840b1740 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.527+0000 7f9b8bd6a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8406d6f0 con 0x7f9b8415bdc0 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b89b06640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9b84155e30 0x7f9b8415cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41218/0 (socket says 172.21.15.139:41218) 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b89b06640 1 -- 172.21.15.139:0/1214135818 learned_addr learned my addr 172.21.15.139:0/1214135818 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590380143 0 0) 0x7f9b8415c880 con 0x7f9b84155e30 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b8a307640 1 --2- 172.21.15.139:0/1214135818 >> v2:172.21.15.150:3300/0 conn(0x7f9b840b1740 0x7f9b840b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b89305640 1 --2- 172.21.15.139:0/1214135818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8415bdc0 0x7f9b8415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b70003520 con 0x7f9b84155e30 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 401359565 0 0) 0x7f9b70003520 con 0x7f9b84155e30 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8415bdc0 msgr2=0x7f9b8415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 --2- 172.21.15.139:0/1214135818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8415bdc0 0x7f9b8415c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 >> v2:172.21.15.150:3300/0 conn(0x7f9b840b1740 msgr2=0x7f9b840b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:09.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 --2- 172.21.15.139:0/1214135818 >> v2:172.21.15.150:3300/0 conn(0x7f9b840b1740 0x7f9b840b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8415cb60 con 0x7f9b84155e30 2024-04-25T09:57:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f9b74001670 con 0x7f9b84155e30 2024-04-25T09:57:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 >> v1:172.21.15.139:6789/0 conn(0x7f9b84155e30 legacy=0x7f9b8415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 --2- 172.21.15.139:0/1214135818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8415bdc0 0x7f9b700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 --2- 172.21.15.139:0/1214135818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b70003f30 0x7f9b70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 --2- 172.21.15.139:0/1214135818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b70004d40 0x7f9b700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8415d690 con 0x7f9b70003f30 2024-04-25T09:57:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8406d6f0 con 0x7f9b70004d40 2024-04-25T09:57:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7000b9c0 con 0x7f9b8415bdc0 2024-04-25T09:57:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b89305640 1 --2- 172.21.15.139:0/1214135818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b70003f30 0x7f9b70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b89b06640 1 --2- 172.21.15.139:0/1214135818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8415bdc0 0x7f9b700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.528+0000 7f9b8a307640 1 --2- 172.21.15.139:0/1214135818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b70004d40 0x7f9b700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.529+0000 7f9b89305640 1 -- 172.21.15.139:0/1214135818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8415bdc0 msgr2=0x7f9b700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.529+0000 7f9b89305640 1 --2- 172.21.15.139:0/1214135818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8415bdc0 0x7f9b700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.529+0000 7f9b89305640 1 -- 172.21.15.139:0/1214135818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b70004d40 msgr2=0x7f9b700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.529+0000 7f9b89305640 1 --2- 172.21.15.139:0/1214135818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b70004d40 0x7f9b700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.529+0000 7f9b89305640 1 -- 172.21.15.139:0/1214135818 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b70003520 con 0x7f9b70003f30 2024-04-25T09:57:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.529+0000 7f9b89305640 1 --2- 172.21.15.139:0/1214135818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b70003f30 0x7f9b70004800 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009ac0 tx=0x7f9b70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b502e212cc87d95b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.529+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80036070 con 0x7f9b70003f30 2024-04-25T09:57:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.529+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b80031400 con 0x7f9b70003f30 2024-04-25T09:57:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.529+0000 7f9b88b04640 1 -- 172.21.15.139:0/1214135818 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80009690 con 0x7f9b70003f30 2024-04-25T09:57:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.530+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1214135818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b70003f30 msgr2=0x7f9b70004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.530+0000 7f9b8bd6a640 1 --2- 172.21.15.139:0/1214135818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b70003f30 0x7f9b70004800 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009ac0 tx=0x7f9b70005830 comp rx=0 tx=0).stop 2024-04-25T09:57:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.530+0000 7f9b8a307640 1 -- 172.21.15.139:0/1214135818 reap_dead start 2024-04-25T09:57:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.530+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1214135818 shutdown_connections 2024-04-25T09:57:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.530+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1214135818 >> 172.21.15.139:0/1214135818 conn(0x7f9b84066f20 msgr2=0x7f9b8406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.530+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1214135818 shutdown_connections 2024-04-25T09:57:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.530+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1214135818 wait complete. 2024-04-25T09:57:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.531+0000 7f9b8bd6a640 1 Processor -- start 2024-04-25T09:57:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.531+0000 7f9b8bd6a640 1 -- start start 2024-04-25T09:57:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b8bd6a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b70003f30 0x7f9b84148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b8bd6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b840acb20 0x7f9b840acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b8bd6a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b84158190 0x7f9b84158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b8bd6a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840af0e0 con 0x7f9b840acb20 2024-04-25T09:57:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b8bd6a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b84156210 con 0x7f9b84158190 2024-04-25T09:57:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b8bd6a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840af260 con 0x7f9b70003f30 2024-04-25T09:57:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b89305640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b840acb20 0x7f9b840acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b89305640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b840acb20 0x7f9b840acfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53592/0 (socket says 172.21.15.139:53592) 2024-04-25T09:57:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b89305640 1 -- 172.21.15.139:0/1551716 learned_addr learned my addr 172.21.15.139:0/1551716 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b89305640 1 -- 172.21.15.139:0/1551716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b70003f30 msgr2=0x7f9b84148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b89b06640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b70003f30 0x7f9b84148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b8a307640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b84158190 0x7f9b84158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b89305640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b70003f30 0x7f9b84148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b89305640 1 -- 172.21.15.139:0/1551716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b84158190 msgr2=0x7f9b84158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b89305640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b84158190 0x7f9b84158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b89305640 1 -- 172.21.15.139:0/1551716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b840ad690 con 0x7f9b840acb20 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b8a307640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b84158190 0x7f9b84158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.532+0000 7f9b89b06640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b70003f30 0x7f9b84148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.533+0000 7f9b89305640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b840acb20 0x7f9b840acfa0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9b80000c00 tx=0x7f9b80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.533+0000 7f9b7a7fc640 1 -- 172.21.15.139:0/1551716 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80040020 con 0x7f9b840acb20 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.533+0000 7f9b7a7fc640 1 -- 172.21.15.139:0/1551716 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b80003c60 con 0x7f9b840acb20 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.533+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1551716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b840ad970 con 0x7f9b840acb20 2024-04-25T09:57:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.533+0000 7f9b7a7fc640 1 -- 172.21.15.139:0/1551716 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80036040 con 0x7f9b840acb20 2024-04-25T09:57:09.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.533+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1551716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b840adf30 con 0x7f9b840acb20 2024-04-25T09:57:09.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.534+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1551716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b84074520 con 0x7f9b840acb20 2024-04-25T09:57:09.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.534+0000 7f9b7a7fc640 1 -- 172.21.15.139:0/1551716 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b80043020 con 0x7f9b840acb20 2024-04-25T09:57:09.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.534+0000 7f9b7a7fc640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b6806b500 0x7f9b6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.535+0000 7f9b7a7fc640 1 -- 172.21.15.139:0/1551716 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f9b800aa460 con 0x7f9b840acb20 2024-04-25T09:57:09.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.535+0000 7f9b89b06640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b6806b500 0x7f9b6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.535+0000 7f9b89b06640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b6806b500 0x7f9b6806d9c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9b840b05b0 tx=0x7f9b74009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:09.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.539+0000 7f9b7a7fc640 1 -- 172.21.15.139:0/1551716 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b8004a070 con 0x7f9b840acb20 2024-04-25T09:57:09.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.708+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1551716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b8406b830 con 0x7f9b840acb20 2024-04-25T09:57:09.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.709+0000 7f9b7a7fc640 1 -- 172.21.15.139:0/1551716 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17173 (secure 0 0 0) 0x7f9b800044e0 con 0x7f9b840acb20 2024-04-25T09:57:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:09.712 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":64,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:55:57.317641+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:48.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:49.597505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.22","mappings":[{"from":7,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:09.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.711+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1551716 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b6806b500 msgr2=0x7f9b6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.711+0000 7f9b8bd6a640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b6806b500 0x7f9b6806d9c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9b840b05b0 tx=0x7f9b74009040 comp rx=0 tx=0).stop 2024-04-25T09:57:09.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.711+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1551716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b840acb20 msgr2=0x7f9b840acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.711+0000 7f9b8bd6a640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b840acb20 0x7f9b840acfa0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9b80000c00 tx=0x7f9b80004730 comp rx=0 tx=0).stop 2024-04-25T09:57:09.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.711+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1551716 shutdown_connections 2024-04-25T09:57:09.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.711+0000 7f9b8bd6a640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b840acb20 0x7f9b840acfa0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.711+0000 7f9b8bd6a640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9b6806b500 0x7f9b6806d9c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.711+0000 7f9b8bd6a640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b84158190 0x7f9b84158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.711+0000 7f9b8bd6a640 1 --2- 172.21.15.139:0/1551716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b70003f30 0x7f9b84148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.711+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1551716 >> 172.21.15.139:0/1551716 conn(0x7f9b84066f20 msgr2=0x7f9b840ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:09.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.711+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1551716 shutdown_connections 2024-04-25T09:57:09.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.711+0000 7f9b8bd6a640 1 -- 172.21.15.139:0/1551716 wait complete. 2024-04-25T09:57:09.725 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:57:09.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.820+0000 7ff58a291640 1 Processor -- start 2024-04-25T09:57:09.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.820+0000 7ff58a291640 1 -- start start 2024-04-25T09:57:09.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff58a291640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff5840cee50 0x7ff584168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff58a291640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff584161dc0 con 0x7ff5841617e0 2024-04-25T09:57:09.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff58a291640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5841620a0 con 0x7ff58415dbf0 2024-04-25T09:57:09.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff58a291640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5840d1ee0 con 0x7ff5840cee50 2024-04-25T09:57:09.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff5837fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff5841617e0 0x7ff5840ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41230/0 (socket says 172.21.15.139:41230) 2024-04-25T09:57:09.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff5837fe640 1 -- 172.21.15.139:0/3844760966 learned_addr learned my addr 172.21.15.139:0/3844760966 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:09.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405923721 0 0) 0x7ff584161dc0 con 0x7ff5841617e0 2024-04-25T09:57:09.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff56c003520 con 0x7ff5841617e0 2024-04-25T09:57:09.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff58882e640 1 --2- 172.21.15.139:0/3844760966 >> v2:172.21.15.150:3300/0 conn(0x7ff5840cee50 0x7ff584168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2993360733 0 0) 0x7ff56c003520 con 0x7ff5841617e0 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 >> v2:172.21.15.150:3300/0 conn(0x7ff5840cee50 msgr2=0x7ff584168b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 --2- 172.21.15.139:0/3844760966 >> v2:172.21.15.150:3300/0 conn(0x7ff5840cee50 0x7ff584168b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 >> v1:172.21.15.150:6789/0 conn(0x7ff58415dbf0 legacy=0x7ff5840b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff584162380 con 0x7ff5841617e0 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7ff578001670 con 0x7ff5841617e0 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 >> v1:172.21.15.139:6789/0 conn(0x7ff5841617e0 legacy=0x7ff5840ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 --2- 172.21.15.139:0/3844760966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff56c0039b0 0x7ff56c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 --2- 172.21.15.139:0/3844760966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff56c004800 0x7ff56c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 --2- 172.21.15.139:0/3844760966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff56c0051c0 0x7ff56c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5840d1ee0 con 0x7ff56c0039b0 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c00c160 con 0x7ff56c004800 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c00c360 con 0x7ff56c0051c0 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.821+0000 7ff583fff640 1 --2- 172.21.15.139:0/3844760966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff56c0039b0 0x7ff56c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.822+0000 7ff58882e640 1 --2- 172.21.15.139:0/3844760966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff56c0051c0 0x7ff56c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.822+0000 7ff5837fe640 1 --2- 172.21.15.139:0/3844760966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff56c004800 0x7ff56c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.822+0000 7ff583fff640 1 -- 172.21.15.139:0/3844760966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff56c0051c0 msgr2=0x7ff56c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.822+0000 7ff583fff640 1 --2- 172.21.15.139:0/3844760966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff56c0051c0 0x7ff56c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.822+0000 7ff583fff640 1 -- 172.21.15.139:0/3844760966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff56c004800 msgr2=0x7ff56c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.822+0000 7ff583fff640 1 --2- 172.21.15.139:0/3844760966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff56c004800 0x7ff56c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.822+0000 7ff583fff640 1 -- 172.21.15.139:0/3844760966 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c003520 con 0x7ff56c0039b0 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.822+0000 7ff58882e640 1 -- 172.21.15.139:0/3844760966 reap_dead start 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.822+0000 7ff5837fe640 1 --2- 172.21.15.139:0/3844760966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff56c004800 0x7ff56c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.822+0000 7ff583fff640 1 --2- 172.21.15.139:0/3844760966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff56c0039b0 0x7ff56c003dd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff574001750 tx=0x7ff56c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5822609d1161a562 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.823+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff574001370 con 0x7ff56c0039b0 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.823+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5740052d0 con 0x7ff56c0039b0 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.823+0000 7ff582ffd640 1 -- 172.21.15.139:0/3844760966 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57402abc0 con 0x7ff56c0039b0 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.823+0000 7ff58a291640 1 -- 172.21.15.139:0/3844760966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff56c0039b0 msgr2=0x7ff56c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.823+0000 7ff58a291640 1 --2- 172.21.15.139:0/3844760966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff56c0039b0 0x7ff56c003dd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff574001750 tx=0x7ff56c005830 comp rx=0 tx=0).stop 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.823+0000 7ff58a291640 1 -- 172.21.15.139:0/3844760966 shutdown_connections 2024-04-25T09:57:09.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.823+0000 7ff58a291640 1 --2- 172.21.15.139:0/3844760966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff56c0039b0 0x7ff56c003dd0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.823+0000 7ff58a291640 1 -- 172.21.15.139:0/3844760966 >> 172.21.15.139:0/3844760966 conn(0x7ff5840b5480 msgr2=0x7ff5840b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:09.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.823+0000 7ff58a291640 1 -- 172.21.15.139:0/3844760966 shutdown_connections 2024-04-25T09:57:09.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.823+0000 7ff58a291640 1 -- 172.21.15.139:0/3844760966 wait complete. 2024-04-25T09:57:09.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.824+0000 7ff58a291640 1 Processor -- start 2024-04-25T09:57:09.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.824+0000 7ff58a291640 1 -- start start 2024-04-25T09:57:09.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.825+0000 7ff58a291640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff56c0039b0 0x7ff58414e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.825+0000 7ff58a291640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff584159960 0x7ff58415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.825+0000 7ff58a291640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5840cccb0 0x7ff5840cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.825+0000 7ff58a291640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58415dfd0 con 0x7ff584159960 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.825+0000 7ff58a291640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5840b9370 con 0x7ff5840cccb0 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.825+0000 7ff58a291640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5840c7040 con 0x7ff56c0039b0 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.825+0000 7ff5837fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff584159960 0x7ff58415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.825+0000 7ff5837fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff584159960 0x7ff58415ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53604/0 (socket says 172.21.15.139:53604) 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.825+0000 7ff5837fe640 1 -- 172.21.15.139:0/4077733616 learned_addr learned my addr 172.21.15.139:0/4077733616 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.825+0000 7ff583fff640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff56c0039b0 0x7ff58414e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.825+0000 7ff58882e640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5840cccb0 0x7ff5840cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.826+0000 7ff5837fe640 1 -- 172.21.15.139:0/4077733616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff56c0039b0 msgr2=0x7ff58414e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.826+0000 7ff5837fe640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff56c0039b0 0x7ff58414e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.826+0000 7ff5837fe640 1 -- 172.21.15.139:0/4077733616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5840cccb0 msgr2=0x7ff5840cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.826+0000 7ff5837fe640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5840cccb0 0x7ff5840cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.826+0000 7ff5837fe640 1 -- 172.21.15.139:0/4077733616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5840c7320 con 0x7ff584159960 2024-04-25T09:57:09.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.826+0000 7ff5837fe640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff584159960 0x7ff58415ce30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff57800e780 tx=0x7ff57800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:09.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.826+0000 7ff580ff9640 1 -- 172.21.15.139:0/4077733616 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57800ee30 con 0x7ff584159960 2024-04-25T09:57:09.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.826+0000 7ff580ff9640 1 -- 172.21.15.139:0/4077733616 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff578018070 con 0x7ff584159960 2024-04-25T09:57:09.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.826+0000 7ff58a291640 1 -- 172.21.15.139:0/4077733616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5840c7600 con 0x7ff584159960 2024-04-25T09:57:09.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.826+0000 7ff580ff9640 1 -- 172.21.15.139:0/4077733616 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57800cab0 con 0x7ff584159960 2024-04-25T09:57:09.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.826+0000 7ff58a291640 1 -- 172.21.15.139:0/4077733616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5840c7bc0 con 0x7ff584159960 2024-04-25T09:57:09.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.827+0000 7ff58a291640 1 -- 172.21.15.139:0/4077733616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff584074520 con 0x7ff584159960 2024-04-25T09:57:09.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.827+0000 7ff580ff9640 1 -- 172.21.15.139:0/4077733616 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff57800cc50 con 0x7ff584159960 2024-04-25T09:57:09.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.828+0000 7ff580ff9640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff57006b4d0 0x7ff57006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.828+0000 7ff580ff9640 1 -- 172.21.15.139:0/4077733616 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6246+0+0 (secure 0 0 0) 0x7ff57808a0b0 con 0x7ff584159960 2024-04-25T09:57:09.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.828+0000 7ff583fff640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff57006b4d0 0x7ff57006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.828+0000 7ff583fff640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff57006b4d0 0x7ff57006d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff574032000 tx=0x7ff57402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:09.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:09.832+0000 7ff580ff9640 1 -- 172.21.15.139:0/4077733616 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5780586c0 con 0x7ff584159960 2024-04-25T09:57:10.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.036+0000 7ff58a291640 1 -- 172.21.15.139:0/4077733616 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff5840b7570 con 0x7ff584159960 2024-04-25T09:57:10.040 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T09:57:10.037+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:57:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.038+0000 7ff580ff9640 1 -- 172.21.15.139:0/4077733616 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff57805c760 con 0x7ff584159960 2024-04-25T09:57:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:57:10.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.041+0000 7ff58a291640 1 -- 172.21.15.139:0/4077733616 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff57006b4d0 msgr2=0x7ff57006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.041+0000 7ff58a291640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff57006b4d0 0x7ff57006d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff574032000 tx=0x7ff57402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:10.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.041+0000 7ff58a291640 1 -- 172.21.15.139:0/4077733616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff584159960 msgr2=0x7ff58415ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.041+0000 7ff58a291640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff584159960 0x7ff58415ce30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff57800e780 tx=0x7ff57800ec40 comp rx=0 tx=0).stop 2024-04-25T09:57:10.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.041+0000 7ff58a291640 1 -- 172.21.15.139:0/4077733616 shutdown_connections 2024-04-25T09:57:10.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.041+0000 7ff58a291640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff584159960 0x7ff58415ce30 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.041+0000 7ff58a291640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff57006b4d0 0x7ff57006d990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.041+0000 7ff58a291640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5840cccb0 0x7ff5840cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.041+0000 7ff58a291640 1 --2- 172.21.15.139:0/4077733616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff56c0039b0 0x7ff58414e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.041+0000 7ff58a291640 1 -- 172.21.15.139:0/4077733616 >> 172.21.15.139:0/4077733616 conn(0x7ff5840b5480 msgr2=0x7ff5840d3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:10.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.041+0000 7ff58a291640 1 -- 172.21.15.139:0/4077733616 shutdown_connections 2024-04-25T09:57:10.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.041+0000 7ff58a291640 1 -- 172.21.15.139:0/4077733616 wait complete. 2024-04-25T09:57:10.047 INFO:tasks.ceph.mds.a.smithi139.stderr: -7211> 2024-04-25T09:55:56.541+0000 7f41d3e3b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:57:10.049 INFO:tasks.ceph.mds.a.smithi139.stderr: -4787> 2024-04-25T09:56:02.390+0000 7f41cfe33640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5474=0-4194304@1} caps={5474=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5474 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x564683923b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T09:57:10.051 INFO:tasks.ceph.mds.a.smithi139.stderr: -3058> 2024-04-25T09:56:33.641+0000 7f41cfe33640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5474=0-4194304@1} caps={5474=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5474 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x564683923b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T09:57:10.056 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:57:10.079 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T09:57:10.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.151+0000 7fe4c21c1640 1 Processor -- start 2024-04-25T09:57:10.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.151+0000 7fe4c21c1640 1 -- start start 2024-04-25T09:57:10.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.152+0000 7fe4c21c1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe4bc0cd610 0x7fe4bc0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.152+0000 7fe4c21c1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4bc099ca0 con 0x7fe4bc09ad40 2024-04-25T09:57:10.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.152+0000 7fe4c21c1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4bc099f80 con 0x7fe4bc161fd0 2024-04-25T09:57:10.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.152+0000 7fe4c21c1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4bc099a50 con 0x7fe4bc0cd610 2024-04-25T09:57:10.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.152+0000 7fe4bb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe4bc0cd610 0x7fe4bc0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.152+0000 7fe4bb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe4bc0cd610 0x7fe4bc0d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53620/0 (socket says 172.21.15.139:53620) 2024-04-25T09:57:10.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.152+0000 7fe4baffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe4bc09ad40 0x7fe4bc1618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41252/0 (socket says 172.21.15.139:41252) 2024-04-25T09:57:10.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.152+0000 7fe4bb7fe640 1 -- 172.21.15.139:0/1582662723 learned_addr learned my addr 172.21.15.139:0/1582662723 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:10.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.152+0000 7fe4bb7fe640 1 -- 172.21.15.139:0/1582662723 >> v1:172.21.15.150:6789/0 conn(0x7fe4bc161fd0 legacy=0x7fe4bc0cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.152+0000 7fe4bb7fe640 1 -- 172.21.15.139:0/1582662723 >> v1:172.21.15.139:6789/0 conn(0x7fe4bc09ad40 legacy=0x7fe4bc1618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.152+0000 7fe4bb7fe640 1 -- 172.21.15.139:0/1582662723 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc09a260 con 0x7fe4bc0cd610 2024-04-25T09:57:10.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.152+0000 7fe4ba7fc640 1 -- 172.21.15.139:0/1582662723 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571995664 0 0) 0x7fe4bc099ca0 con 0x7fe4bc09ad40 2024-04-25T09:57:10.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.153+0000 7fe4bb7fe640 1 --2- 172.21.15.139:0/1582662723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4bc0cd610 0x7fe4bc0d0370 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009870 tx=0x7fe4ac02a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f268f8d7cad1ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:10.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.153+0000 7fe4ba7fc640 1 -- 172.21.15.139:0/1582662723 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac037070 con 0x7fe4bc0cd610 2024-04-25T09:57:10.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.153+0000 7fe4ba7fc640 1 -- 172.21.15.139:0/1582662723 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac032640 con 0x7fe4bc0cd610 2024-04-25T09:57:10.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.153+0000 7fe4ba7fc640 1 -- 172.21.15.139:0/1582662723 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac032a10 con 0x7fe4bc0cd610 2024-04-25T09:57:10.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.154+0000 7fe4c21c1640 1 -- 172.21.15.139:0/1582662723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4bc0cd610 msgr2=0x7fe4bc0d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.154+0000 7fe4c21c1640 1 --2- 172.21.15.139:0/1582662723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4bc0cd610 0x7fe4bc0d0370 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009870 tx=0x7fe4ac02a3c0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.154+0000 7fe4c21c1640 1 -- 172.21.15.139:0/1582662723 shutdown_connections 2024-04-25T09:57:10.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.154+0000 7fe4c21c1640 1 --2- 172.21.15.139:0/1582662723 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4bc0cd610 0x7fe4bc0d0370 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.154+0000 7fe4c21c1640 1 -- 172.21.15.139:0/1582662723 >> 172.21.15.139:0/1582662723 conn(0x7fe4bc0b31f0 msgr2=0x7fe4bc0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:10.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.154+0000 7fe4c21c1640 1 -- 172.21.15.139:0/1582662723 shutdown_connections 2024-04-25T09:57:10.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.154+0000 7fe4c21c1640 1 -- 172.21.15.139:0/1582662723 wait complete. 2024-04-25T09:57:10.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.155+0000 7fe4c21c1640 1 Processor -- start 2024-04-25T09:57:10.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.155+0000 7fe4c21c1640 1 -- start start 2024-04-25T09:57:10.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4c21c1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4bc09ad40 0x7fe4bc1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4c21c1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4bc0cd610 0x7fe4bc0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4c21c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4bc161fd0 0x7fe4bc0c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4c21c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc0cd9f0 con 0x7fe4bc161fd0 2024-04-25T09:57:10.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4c21c1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc09a820 con 0x7fe4bc0cd610 2024-04-25T09:57:10.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4c21c1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc0cdb70 con 0x7fe4bc09ad40 2024-04-25T09:57:10.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4bc161fd0 0x7fe4bc0c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4bc161fd0 0x7fe4bc0c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53636/0 (socket says 172.21.15.139:53636) 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4bbfff640 1 -- 172.21.15.139:0/2062565737 learned_addr learned my addr 172.21.15.139:0/2062565737 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4bb7fe640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4bc09ad40 0x7fe4bc1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4bbfff640 1 -- 172.21.15.139:0/2062565737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4bc09ad40 msgr2=0x7fe4bc1597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4bbfff640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4bc09ad40 0x7fe4bc1597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4bbfff640 1 -- 172.21.15.139:0/2062565737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4bc0cd610 msgr2=0x7fe4bc0b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4bbfff640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4bc0cd610 0x7fe4bc0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4bbfff640 1 -- 172.21.15.139:0/2062565737 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc15a090 con 0x7fe4bc161fd0 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4bbfff640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4bc161fd0 0x7fe4bc0c72f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400c680 tx=0x7fe4b400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4a3fff640 1 -- 172.21.15.139:0/2062565737 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4003b70 con 0x7fe4bc161fd0 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4a3fff640 1 -- 172.21.15.139:0/2062565737 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b4003d10 con 0x7fe4bc161fd0 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4a3fff640 1 -- 172.21.15.139:0/2062565737 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4005310 con 0x7fe4bc161fd0 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.156+0000 7fe4c21c1640 1 -- 172.21.15.139:0/2062565737 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4bc1f3cb0 con 0x7fe4bc161fd0 2024-04-25T09:57:10.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.157+0000 7fe4c21c1640 1 -- 172.21.15.139:0/2062565737 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4bc0c60d0 con 0x7fe4bc161fd0 2024-04-25T09:57:10.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.157+0000 7fe4c21c1640 1 -- 172.21.15.139:0/2062565737 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4bc074520 con 0x7fe4bc161fd0 2024-04-25T09:57:10.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.158+0000 7fe4a3fff640 1 -- 172.21.15.139:0/2062565737 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4b40054b0 con 0x7fe4bc161fd0 2024-04-25T09:57:10.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.158+0000 7fe4a3fff640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe4a806b4d0 0x7fe4a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.158+0000 7fe4a3fff640 1 -- 172.21.15.139:0/2062565737 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6332+0+0 (secure 0 0 0) 0x7fe4b408a2b0 con 0x7fe4bc161fd0 2024-04-25T09:57:10.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.158+0000 7fe4bb7fe640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe4a806b4d0 0x7fe4a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.159+0000 7fe4bb7fe640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe4a806b4d0 0x7fe4a806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac034000 tx=0x7fe4ac02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:10.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.162+0000 7fe4a3fff640 1 -- 172.21.15.139:0/2062565737 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe4bc15a090 con 0x7fe4bc161fd0 2024-04-25T09:57:10.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:10.372+0000 7fe4c21c1640 1 -- 172.21.15.139:0/2062565737 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe4bc0b3f70 con 0x7fe4bc161fd0 2024-04-25T09:57:11.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.039+0000 7fe4a3fff640 1 -- 172.21.15.139:0/2062565737 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe4b4058eb0 con 0x7fe4bc161fd0 2024-04-25T09:57:11.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.043+0000 7fe4c21c1640 1 -- 172.21.15.139:0/2062565737 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe4a806b4d0 msgr2=0x7fe4a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.043+0000 7fe4c21c1640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe4a806b4d0 0x7fe4a806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac034000 tx=0x7fe4ac02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.043+0000 7fe4c21c1640 1 -- 172.21.15.139:0/2062565737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4bc161fd0 msgr2=0x7fe4bc0c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.043+0000 7fe4c21c1640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4bc161fd0 0x7fe4bc0c72f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400c680 tx=0x7fe4b400cb40 comp rx=0 tx=0).stop 2024-04-25T09:57:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.043+0000 7fe4c21c1640 1 -- 172.21.15.139:0/2062565737 shutdown_connections 2024-04-25T09:57:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.043+0000 7fe4c21c1640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4bc161fd0 0x7fe4bc0c72f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.043+0000 7fe4c21c1640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe4a806b4d0 0x7fe4a806d990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.043+0000 7fe4c21c1640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4bc0cd610 0x7fe4bc0b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.043+0000 7fe4c21c1640 1 --2- 172.21.15.139:0/2062565737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4bc09ad40 0x7fe4bc1597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.043+0000 7fe4c21c1640 1 -- 172.21.15.139:0/2062565737 >> 172.21.15.139:0/2062565737 conn(0x7fe4bc0b31f0 msgr2=0x7fe4bc0738a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.043+0000 7fe4c21c1640 1 -- 172.21.15.139:0/2062565737 shutdown_connections 2024-04-25T09:57:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.043+0000 7fe4c21c1640 1 -- 172.21.15.139:0/2062565737 wait complete. 2024-04-25T09:57:11.060 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:57:11.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.204+0000 7f7a37989640 1 Processor -- start 2024-04-25T09:57:11.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.204+0000 7f7a37989640 1 -- start start 2024-04-25T09:57:11.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a37989640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7a3015cb20 0x7f7a300cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a37989640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7a300cfeb0 0x7f7a3006e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a37989640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a300d02c0 con 0x7f7a300afc00 2024-04-25T09:57:11.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a37989640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a300703c0 con 0x7f7a3015cb20 2024-04-25T09:57:11.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a37989640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a300cd820 con 0x7f7a300cfeb0 2024-04-25T09:57:11.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a34f24640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7a3015cb20 0x7f7a300cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a34f24640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7a3015cb20 0x7f7a300cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53650/0 (socket says 172.21.15.139:53650) 2024-04-25T09:57:11.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a34f24640 1 -- 172.21.15.139:0/2988382308 learned_addr learned my addr 172.21.15.139:0/2988382308 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:11.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a35725640 1 --1- 172.21.15.139:0/2988382308 >> v1:172.21.15.139:6789/0 conn(0x7f7a300afc00 0x7f7a3006fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41260/0 (socket says 172.21.15.139:41260) 2024-04-25T09:57:11.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a35f26640 1 --2- 172.21.15.139:0/2988382308 >> v2:172.21.15.150:3300/0 conn(0x7f7a300cfeb0 0x7f7a3006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a34f24640 1 -- 172.21.15.139:0/2988382308 >> v2:172.21.15.150:3300/0 conn(0x7f7a300cfeb0 msgr2=0x7f7a3006e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:11.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a34f24640 1 --2- 172.21.15.139:0/2988382308 >> v2:172.21.15.150:3300/0 conn(0x7f7a300cfeb0 0x7f7a3006e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a34f24640 1 -- 172.21.15.139:0/2988382308 >> v1:172.21.15.139:6789/0 conn(0x7f7a300afc00 legacy=0x7f7a3006fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a27fff640 1 -- 172.21.15.139:0/2988382308 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461826912 0 0) 0x7f7a300d02c0 con 0x7f7a300afc00 2024-04-25T09:57:11.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.205+0000 7f7a34f24640 1 -- 172.21.15.139:0/2988382308 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300ccef0 con 0x7f7a3015cb20 2024-04-25T09:57:11.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.206+0000 7f7a34f24640 1 --2- 172.21.15.139:0/2988382308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a3015cb20 0x7f7a300cf970 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7a30159a70 tx=0x7f7a2002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3611bc9c34a55580 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.206+0000 7f7a27fff640 1 -- 172.21.15.139:0/2988382308 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20037070 con 0x7f7a3015cb20 2024-04-25T09:57:11.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.206+0000 7f7a27fff640 1 -- 172.21.15.139:0/2988382308 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a20032440 con 0x7f7a3015cb20 2024-04-25T09:57:11.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.206+0000 7f7a27fff640 1 -- 172.21.15.139:0/2988382308 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20032810 con 0x7f7a3015cb20 2024-04-25T09:57:11.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.206+0000 7f7a37989640 1 -- 172.21.15.139:0/2988382308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a3015cb20 msgr2=0x7f7a300cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.206+0000 7f7a37989640 1 --2- 172.21.15.139:0/2988382308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a3015cb20 0x7f7a300cf970 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7a30159a70 tx=0x7f7a2002a150 comp rx=0 tx=0).stop 2024-04-25T09:57:11.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.206+0000 7f7a37989640 1 -- 172.21.15.139:0/2988382308 shutdown_connections 2024-04-25T09:57:11.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.206+0000 7f7a37989640 1 --2- 172.21.15.139:0/2988382308 >> v2:172.21.15.150:3300/0 conn(0x7f7a300cfeb0 0x7f7a3006e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.206+0000 7f7a37989640 1 --2- 172.21.15.139:0/2988382308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a3015cb20 0x7f7a300cf970 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.206+0000 7f7a37989640 1 -- 172.21.15.139:0/2988382308 >> 172.21.15.139:0/2988382308 conn(0x7f7a300b2230 msgr2=0x7f7a300b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.207+0000 7f7a37989640 1 -- 172.21.15.139:0/2988382308 shutdown_connections 2024-04-25T09:57:11.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.207+0000 7f7a37989640 1 -- 172.21.15.139:0/2988382308 wait complete. 2024-04-25T09:57:11.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.208+0000 7f7a37989640 1 Processor -- start 2024-04-25T09:57:11.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.208+0000 7f7a37989640 1 -- start start 2024-04-25T09:57:11.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.208+0000 7f7a37989640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a300afc00 0x7f7a301585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.208+0000 7f7a37989640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a300cfeb0 0x7f7a3014e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.208+0000 7f7a37989640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a3015cb20 0x7f7a300ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.208+0000 7f7a37989640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300cd4b0 con 0x7f7a300cfeb0 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.208+0000 7f7a37989640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300affe0 con 0x7f7a3015cb20 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.208+0000 7f7a37989640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300cdcb0 con 0x7f7a300afc00 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.208+0000 7f7a34f24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a300cfeb0 0x7f7a3014e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.208+0000 7f7a34f24640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a300cfeb0 0x7f7a3014e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53666/0 (socket says 172.21.15.139:53666) 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.208+0000 7f7a34f24640 1 -- 172.21.15.139:0/3018639048 learned_addr learned my addr 172.21.15.139:0/3018639048 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a35725640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a300afc00 0x7f7a301585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a35f26640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a3015cb20 0x7f7a300ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a34f24640 1 -- 172.21.15.139:0/3018639048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a300afc00 msgr2=0x7f7a301585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a34f24640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a300afc00 0x7f7a301585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a34f24640 1 -- 172.21.15.139:0/3018639048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a3015cb20 msgr2=0x7f7a300ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a34f24640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a3015cb20 0x7f7a300ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a34f24640 1 -- 172.21.15.139:0/3018639048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300b5ba0 con 0x7f7a300cfeb0 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a35f26640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a3015cb20 0x7f7a300ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a34f24640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a300cfeb0 0x7f7a3014e280 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7a2002a190 tx=0x7f7a200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a25ffb640 1 -- 172.21.15.139:0/3018639048 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20037040 con 0x7f7a300cfeb0 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a25ffb640 1 -- 172.21.15.139:0/3018639048 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a20042070 con 0x7f7a300cfeb0 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a37989640 1 -- 172.21.15.139:0/3018639048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a300b5e80 con 0x7f7a300cfeb0 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a25ffb640 1 -- 172.21.15.139:0/3018639048 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20030bb0 con 0x7f7a300cfeb0 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.209+0000 7f7a37989640 1 -- 172.21.15.139:0/3018639048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a300b6440 con 0x7f7a300cfeb0 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.210+0000 7f7a37989640 1 -- 172.21.15.139:0/3018639048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a30074520 con 0x7f7a300cfeb0 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.211+0000 7f7a25ffb640 1 -- 172.21.15.139:0/3018639048 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a20047460 con 0x7f7a300cfeb0 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.211+0000 7f7a25ffb640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7a1406b4d0 0x7f7a1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.211+0000 7f7a25ffb640 1 -- 172.21.15.139:0/3018639048 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6332+0+0 (secure 0 0 0) 0x7f7a200ab010 con 0x7f7a300cfeb0 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.211+0000 7f7a35725640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7a1406b4d0 0x7f7a1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.212+0000 7f7a35725640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7a1406b4d0 0x7f7a1406d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c000f60 tx=0x7f7a2c007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.216+0000 7f7a25ffb640 1 -- 172.21.15.139:0/3018639048 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a20079cb0 con 0x7f7a300cfeb0 2024-04-25T09:57:11.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:11.389+0000 7f7a37989640 1 -- 172.21.15.139:0/3018639048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7a3006edf0 con 0x7f7a300cfeb0 2024-04-25T09:57:12.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.041+0000 7f7a25ffb640 1 -- 172.21.15.139:0/3018639048 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v66) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7a200795b0 con 0x7f7a300cfeb0 2024-04-25T09:57:12.044 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:57:12.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.045+0000 7f7a37989640 1 -- 172.21.15.139:0/3018639048 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7a1406b4d0 msgr2=0x7f7a1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.045+0000 7f7a37989640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7a1406b4d0 0x7f7a1406d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c000f60 tx=0x7f7a2c007c00 comp rx=0 tx=0).stop 2024-04-25T09:57:12.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.045+0000 7f7a37989640 1 -- 172.21.15.139:0/3018639048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a300cfeb0 msgr2=0x7f7a3014e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.045+0000 7f7a37989640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a300cfeb0 0x7f7a3014e280 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7a2002a190 tx=0x7f7a200046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.045+0000 7f7a37989640 1 -- 172.21.15.139:0/3018639048 shutdown_connections 2024-04-25T09:57:12.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.045+0000 7f7a37989640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a300cfeb0 0x7f7a3014e280 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.045+0000 7f7a37989640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7a1406b4d0 0x7f7a1406d990 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.045+0000 7f7a37989640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a3015cb20 0x7f7a300ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.045+0000 7f7a37989640 1 --2- 172.21.15.139:0/3018639048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a300afc00 0x7f7a301585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.045+0000 7f7a37989640 1 -- 172.21.15.139:0/3018639048 >> 172.21.15.139:0/3018639048 conn(0x7f7a300b2230 msgr2=0x7f7a300cb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:12.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.045+0000 7f7a37989640 1 -- 172.21.15.139:0/3018639048 shutdown_connections 2024-04-25T09:57:12.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.045+0000 7f7a37989640 1 -- 172.21.15.139:0/3018639048 wait complete. 2024-04-25T09:57:12.062 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:57:12.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.166+0000 7fd9bcfa5640 1 Processor -- start 2024-04-25T09:57:12.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.166+0000 7fd9bcfa5640 1 -- start start 2024-04-25T09:57:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9bcfa5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd9b81617e0 0x7fd9b80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9bcfa5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b8161dc0 con 0x7fd9b80cec80 2024-04-25T09:57:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9bcfa5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b81620a0 con 0x7fd9b815dbf0 2024-04-25T09:57:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9bcfa5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b80cf0d0 con 0x7fd9b81617e0 2024-04-25T09:57:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b6d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd9b80cec80 0x7fd9b80d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41270/0 (socket says 172.21.15.139:41270) 2024-04-25T09:57:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b6d9d640 1 -- 172.21.15.139:0/1639446456 learned_addr learned my addr 172.21.15.139:0/1639446456 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b5d9b640 1 --2- 172.21.15.139:0/1639446456 >> v2:172.21.15.150:3300/0 conn(0x7fd9b81617e0 0x7fd9b80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59827761 0 0) 0x7fd9b8161dc0 con 0x7fd9b80cec80 2024-04-25T09:57:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9a0003520 con 0x7fd9b80cec80 2024-04-25T09:57:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1221142125 0 0) 0x7fd9a0003520 con 0x7fd9b80cec80 2024-04-25T09:57:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 >> v2:172.21.15.150:3300/0 conn(0x7fd9b81617e0 msgr2=0x7fd9b80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 --2- 172.21.15.139:0/1639446456 >> v2:172.21.15.150:3300/0 conn(0x7fd9b81617e0 0x7fd9b80ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 >> v1:172.21.15.150:6789/0 conn(0x7fd9b815dbf0 legacy=0x7fd9b80b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b8162380 con 0x7fd9b80cec80 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fd9b00019f0 con 0x7fd9b80cec80 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 >> v1:172.21.15.139:6789/0 conn(0x7fd9b80cec80 legacy=0x7fd9b80d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 --2- 172.21.15.139:0/1639446456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9a00039b0 0x7fd9a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 --2- 172.21.15.139:0/1639446456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a0004800 0x7fd9a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 --2- 172.21.15.139:0/1639446456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a00051c0 0x7fd9a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80cf0d0 con 0x7fd9a00039b0 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a000c160 con 0x7fd9a0004800 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.167+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a000c360 con 0x7fd9a00051c0 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.168+0000 7fd9b659c640 1 --2- 172.21.15.139:0/1639446456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9a00039b0 0x7fd9a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.168+0000 7fd9b5d9b640 1 --2- 172.21.15.139:0/1639446456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a0004800 0x7fd9a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.168+0000 7fd9b6d9d640 1 --2- 172.21.15.139:0/1639446456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a00051c0 0x7fd9a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.168+0000 7fd9b659c640 1 -- 172.21.15.139:0/1639446456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a00051c0 msgr2=0x7fd9a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.168+0000 7fd9b659c640 1 --2- 172.21.15.139:0/1639446456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a00051c0 0x7fd9a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.168+0000 7fd9b659c640 1 -- 172.21.15.139:0/1639446456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a0004800 msgr2=0x7fd9a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.168+0000 7fd9b659c640 1 --2- 172.21.15.139:0/1639446456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a0004800 0x7fd9a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.168+0000 7fd9b659c640 1 -- 172.21.15.139:0/1639446456 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a0003520 con 0x7fd9a00039b0 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.168+0000 7fd9b6d9d640 1 --2- 172.21.15.139:0/1639446456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a00051c0 0x7fd9a000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.168+0000 7fd9b6d9d640 1 -- 172.21.15.139:0/1639446456 reap_dead start 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.168+0000 7fd9b659c640 1 --2- 172.21.15.139:0/1639446456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9a00039b0 0x7fd9a0003dd0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8001750 tx=0x7fd9a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334630dd3d4c306c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.169+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8001370 con 0x7fd9a00039b0 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.169+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a80052d0 con 0x7fd9a00039b0 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.169+0000 7fd9b559a640 1 -- 172.21.15.139:0/1639446456 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a802abc0 con 0x7fd9a00039b0 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.169+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1639446456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9a00039b0 msgr2=0x7fd9a0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.169+0000 7fd9bcfa5640 1 --2- 172.21.15.139:0/1639446456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9a00039b0 0x7fd9a0003dd0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8001750 tx=0x7fd9a0005830 comp rx=0 tx=0).stop 2024-04-25T09:57:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.169+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1639446456 shutdown_connections 2024-04-25T09:57:12.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.169+0000 7fd9bcfa5640 1 --2- 172.21.15.139:0/1639446456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9a00039b0 0x7fd9a0003dd0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.169+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1639446456 >> 172.21.15.139:0/1639446456 conn(0x7fd9b80b5480 msgr2=0x7fd9b80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:12.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.169+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1639446456 shutdown_connections 2024-04-25T09:57:12.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.169+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1639446456 wait complete. 2024-04-25T09:57:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.171+0000 7fd9bcfa5640 1 Processor -- start 2024-04-25T09:57:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.171+0000 7fd9bcfa5640 1 -- start start 2024-04-25T09:57:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.171+0000 7fd9bcfa5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a00039b0 0x7fd9b8151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.171+0000 7fd9bcfa5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9b8154bf0 0x7fd9b8151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.171+0000 7fd9bcfa5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9b80cca50 0x7fd9b80cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.171+0000 7fd9bcfa5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b815dfd0 con 0x7fd9b8154bf0 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.171+0000 7fd9bcfa5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80d1ee0 con 0x7fd9b80cca50 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.171+0000 7fd9bcfa5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80c7480 con 0x7fd9a00039b0 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.171+0000 7fd9b5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9b8154bf0 0x7fd9b8151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.171+0000 7fd9b5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9b8154bf0 0x7fd9b8151a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53676/0 (socket says 172.21.15.139:53676) 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.171+0000 7fd9b5d9b640 1 -- 172.21.15.139:0/1114373891 learned_addr learned my addr 172.21.15.139:0/1114373891 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9b659c640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a00039b0 0x7fd9b8151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9b5d9b640 1 -- 172.21.15.139:0/1114373891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a00039b0 msgr2=0x7fd9b8151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9b5d9b640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a00039b0 0x7fd9b8151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9b6d9d640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9b80cca50 0x7fd9b80cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9b5d9b640 1 -- 172.21.15.139:0/1114373891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9b80cca50 msgr2=0x7fd9b80cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9b5d9b640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9b80cca50 0x7fd9b80cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9b5d9b640 1 -- 172.21.15.139:0/1114373891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b8155070 con 0x7fd9b8154bf0 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9b5d9b640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9b8154bf0 0x7fd9b8151a90 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac011aa0 tx=0x7fd9ac011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9a6ffd640 1 -- 172.21.15.139:0/1114373891 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00fb80 con 0x7fd9b8154bf0 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9a6ffd640 1 -- 172.21.15.139:0/1114373891 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac00fd20 con 0x7fd9b8154bf0 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1114373891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b8155320 con 0x7fd9b8154bf0 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9a6ffd640 1 -- 172.21.15.139:0/1114373891 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac014660 con 0x7fd9b8154bf0 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.172+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1114373891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b80c7720 con 0x7fd9b8154bf0 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.173+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1114373891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b8074520 con 0x7fd9b8154bf0 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.173+0000 7fd9a6ffd640 1 -- 172.21.15.139:0/1114373891 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9ac012040 con 0x7fd9b8154bf0 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.174+0000 7fd9a6ffd640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd99806b4d0 0x7fd99806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.174+0000 7fd9a6ffd640 1 -- 172.21.15.139:0/1114373891 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5865+0+0 (secure 0 0 0) 0x7fd9ac08a920 con 0x7fd9b8154bf0 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.174+0000 7fd9b659c640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd99806b4d0 0x7fd99806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.174+0000 7fd9b659c640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd99806b4d0 0x7fd99806d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8032000 tx=0x7fd9a802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.178+0000 7fd9a6ffd640 1 -- 172.21.15.139:0/1114373891 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9ac019030 con 0x7fd9b8154bf0 2024-04-25T09:57:12.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:12.352+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1114373891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd9b80d1680 con 0x7fd9b8154bf0 2024-04-25T09:57:13.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.041+0000 7fd9a6ffd640 1 -- 172.21.15.139:0/1114373891 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v67) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd9ac059700 con 0x7fd9b8154bf0 2024-04-25T09:57:13.046 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' removed 2024-04-25T09:57:13.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.045+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1114373891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd99806b4d0 msgr2=0x7fd99806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.045+0000 7fd9bcfa5640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd99806b4d0 0x7fd99806d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8032000 tx=0x7fd9a802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:13.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.045+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1114373891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9b8154bf0 msgr2=0x7fd9b8151a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.045+0000 7fd9bcfa5640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9b8154bf0 0x7fd9b8151a90 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac011aa0 tx=0x7fd9ac011f60 comp rx=0 tx=0).stop 2024-04-25T09:57:13.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.045+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1114373891 shutdown_connections 2024-04-25T09:57:13.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.045+0000 7fd9bcfa5640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9b8154bf0 0x7fd9b8151a90 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.045+0000 7fd9bcfa5640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd99806b4d0 0x7fd99806d990 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.045+0000 7fd9bcfa5640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9b80cca50 0x7fd9b80cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.045+0000 7fd9bcfa5640 1 --2- 172.21.15.139:0/1114373891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a00039b0 0x7fd9b8151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.045+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1114373891 >> 172.21.15.139:0/1114373891 conn(0x7fd9b80b5480 msgr2=0x7fd9b8158ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.045+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1114373891 shutdown_connections 2024-04-25T09:57:13.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.045+0000 7fd9bcfa5640 1 -- 172.21.15.139:0/1114373891 wait complete. 2024-04-25T09:57:13.063 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:57:13.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.159+0000 7f77dfaa3640 1 Processor -- start 2024-04-25T09:57:13.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.159+0000 7f77dfaa3640 1 -- start start 2024-04-25T09:57:13.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.160+0000 7f77dfaa3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f77d80afc80 0x7f77d80b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.160+0000 7f77dfaa3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f77d806b700 0x7f77d806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.160+0000 7f77dfaa3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d80c6cc0 con 0x7f77d80af8a0 2024-04-25T09:57:13.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.160+0000 7f77dfaa3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d80c6fa0 con 0x7f77d80afc80 2024-04-25T09:57:13.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.160+0000 7f77dfaa3640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d80c7280 con 0x7f77d806b700 2024-04-25T09:57:13.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.160+0000 7f77dd03e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f77d80afc80 0x7f77d80b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.160+0000 7f77dd03e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f77d80afc80 0x7f77d80b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53682/0 (socket says 172.21.15.139:53682) 2024-04-25T09:57:13.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.160+0000 7f77dd03e640 1 -- 172.21.15.139:0/621213699 learned_addr learned my addr 172.21.15.139:0/621213699 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:13.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.160+0000 7f77de040640 1 --2- 172.21.15.139:0/621213699 >> v2:172.21.15.150:3300/0 conn(0x7f77d806b700 0x7f77d806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.161+0000 7f77dd03e640 1 -- 172.21.15.139:0/621213699 >> v2:172.21.15.150:3300/0 conn(0x7f77d806b700 msgr2=0x7f77d806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.161+0000 7f77dd03e640 1 --2- 172.21.15.139:0/621213699 >> v2:172.21.15.150:3300/0 conn(0x7f77d806b700 0x7f77d806ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.161+0000 7f77dd03e640 1 -- 172.21.15.139:0/621213699 >> v1:172.21.15.150:6789/0 conn(0x7f77d80af8a0 legacy=0x7f77d80cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.161+0000 7f77dd03e640 1 -- 172.21.15.139:0/621213699 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80cd090 con 0x7f77d80afc80 2024-04-25T09:57:13.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.161+0000 7f77de040640 1 --2- 172.21.15.139:0/621213699 >> v2:172.21.15.150:3300/0 conn(0x7f77d806b700 0x7f77d806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:13.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.161+0000 7f77dd03e640 1 --2- 172.21.15.139:0/621213699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d80afc80 0x7f77d80b2d50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f77d806f060 tx=0x7f77c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3ba2af637e3969 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.161+0000 7f77dc83d640 1 -- 172.21.15.139:0/621213699 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8037070 con 0x7f77d80afc80 2024-04-25T09:57:13.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.161+0000 7f77dc83d640 1 -- 172.21.15.139:0/621213699 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c8032440 con 0x7f77d80afc80 2024-04-25T09:57:13.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.161+0000 7f77dc83d640 1 -- 172.21.15.139:0/621213699 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8032810 con 0x7f77d80afc80 2024-04-25T09:57:13.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.162+0000 7f77dfaa3640 1 -- 172.21.15.139:0/621213699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d80afc80 msgr2=0x7f77d80b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.162+0000 7f77dfaa3640 1 --2- 172.21.15.139:0/621213699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d80afc80 0x7f77d80b2d50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f77d806f060 tx=0x7f77c802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.162+0000 7f77dfaa3640 1 -- 172.21.15.139:0/621213699 shutdown_connections 2024-04-25T09:57:13.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.162+0000 7f77dfaa3640 1 --2- 172.21.15.139:0/621213699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d80afc80 0x7f77d80b2d50 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.162+0000 7f77dfaa3640 1 --2- 172.21.15.139:0/621213699 >> v2:172.21.15.150:3300/0 conn(0x7f77d806b700 0x7f77d806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.162+0000 7f77dfaa3640 1 -- 172.21.15.139:0/621213699 >> 172.21.15.139:0/621213699 conn(0x7f77d80b2230 msgr2=0x7f77d80b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.162+0000 7f77dfaa3640 1 -- 172.21.15.139:0/621213699 shutdown_connections 2024-04-25T09:57:13.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.162+0000 7f77dfaa3640 1 -- 172.21.15.139:0/621213699 wait complete. 2024-04-25T09:57:13.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.163+0000 7f77dfaa3640 1 Processor -- start 2024-04-25T09:57:13.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.163+0000 7f77dfaa3640 1 -- start start 2024-04-25T09:57:13.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dfaa3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d806b700 0x7f77d80b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dfaa3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d80af8a0 0x7f77d81f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dfaa3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77d80afc80 0x7f77d80b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dfaa3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d806b220 con 0x7f77d80af8a0 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dfaa3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d806b0a0 con 0x7f77d806b700 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dfaa3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d806b3a0 con 0x7f77d80afc80 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dd03e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d80af8a0 0x7f77d81f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dd03e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d80af8a0 0x7f77d81f2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53684/0 (socket says 172.21.15.139:53684) 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dd03e640 1 -- 172.21.15.139:0/3865649895 learned_addr learned my addr 172.21.15.139:0/3865649895 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dd83f640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d806b700 0x7f77d80b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77de040640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77d80afc80 0x7f77d80b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dd03e640 1 -- 172.21.15.139:0/3865649895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77d80afc80 msgr2=0x7f77d80b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dd03e640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77d80afc80 0x7f77d80b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dd03e640 1 -- 172.21.15.139:0/3865649895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d806b700 msgr2=0x7f77d80b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dd03e640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d806b700 0x7f77d80b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dd03e640 1 -- 172.21.15.139:0/3865649895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80b63f0 con 0x7f77d80af8a0 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.164+0000 7f77dd03e640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d80af8a0 0x7f77d81f2d60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f77c8000c00 tx=0x7f77c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.165+0000 7f77ce7fc640 1 -- 172.21.15.139:0/3865649895 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8037040 con 0x7f77d80af8a0 2024-04-25T09:57:13.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.165+0000 7f77ce7fc640 1 -- 172.21.15.139:0/3865649895 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c8042070 con 0x7f77d80af8a0 2024-04-25T09:57:13.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.165+0000 7f77ce7fc640 1 -- 172.21.15.139:0/3865649895 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8030b90 con 0x7f77d80af8a0 2024-04-25T09:57:13.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.165+0000 7f77dfaa3640 1 -- 172.21.15.139:0/3865649895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d80b5e30 con 0x7f77d80af8a0 2024-04-25T09:57:13.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.165+0000 7f77dfaa3640 1 -- 172.21.15.139:0/3865649895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d81f3660 con 0x7f77d80af8a0 2024-04-25T09:57:13.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.166+0000 7f77dfaa3640 1 -- 172.21.15.139:0/3865649895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d8074520 con 0x7f77d80af8a0 2024-04-25T09:57:13.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.166+0000 7f77ce7fc640 1 -- 172.21.15.139:0/3865649895 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f77c8047460 con 0x7f77d80af8a0 2024-04-25T09:57:13.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.166+0000 7f77ce7fc640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f77bc06b4d0 0x7f77bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.166+0000 7f77ce7fc640 1 -- 172.21.15.139:0/3865649895 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f77c80aad60 con 0x7f77d80af8a0 2024-04-25T09:57:13.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.166+0000 7f77dd83f640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f77bc06b4d0 0x7f77bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.167+0000 7f77dd83f640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f77bc06b4d0 0x7f77bc06d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f77d4004ea0 tx=0x7f77d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.170+0000 7f77ce7fc640 1 -- 172.21.15.139:0/3865649895 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f77c8079d50 con 0x7f77d80af8a0 2024-04-25T09:57:13.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.340+0000 7f77dfaa3640 1 -- 172.21.15.139:0/3865649895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f77d806ad20 con 0x7f77d80af8a0 2024-04-25T09:57:13.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.343+0000 7f77ce7fc640 1 -- 172.21.15.139:0/3865649895 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f77c8079650 con 0x7f77d80af8a0 2024-04-25T09:57:13.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.345+0000 7f77dfaa3640 1 -- 172.21.15.139:0/3865649895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f77bc06b4d0 msgr2=0x7f77bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.345+0000 7f77dfaa3640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f77bc06b4d0 0x7f77bc06d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f77d4004ea0 tx=0x7f77d4009040 comp rx=0 tx=0).stop 2024-04-25T09:57:13.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.345+0000 7f77dfaa3640 1 -- 172.21.15.139:0/3865649895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d80af8a0 msgr2=0x7f77d81f2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.345+0000 7f77dfaa3640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d80af8a0 0x7f77d81f2d60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f77c8000c00 tx=0x7f77c80046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.345+0000 7f77dfaa3640 1 -- 172.21.15.139:0/3865649895 shutdown_connections 2024-04-25T09:57:13.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.345+0000 7f77dfaa3640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77d80af8a0 0x7f77d81f2d60 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.345+0000 7f77dfaa3640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f77bc06b4d0 0x7f77bc06d990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.345+0000 7f77dfaa3640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77d80afc80 0x7f77d80b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.345+0000 7f77dfaa3640 1 --2- 172.21.15.139:0/3865649895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d806b700 0x7f77d80b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.345+0000 7f77dfaa3640 1 -- 172.21.15.139:0/3865649895 >> 172.21.15.139:0/3865649895 conn(0x7f77d80b2230 msgr2=0x7f77d80ad510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.345+0000 7f77dfaa3640 1 -- 172.21.15.139:0/3865649895 shutdown_connections 2024-04-25T09:57:13.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.345+0000 7f77dfaa3640 1 -- 172.21.15.139:0/3865649895 wait complete. 2024-04-25T09:57:13.361 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T09:57:13.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.454+0000 7f573669e640 1 Processor -- start 2024-04-25T09:57:13.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.454+0000 7f573669e640 1 -- start start 2024-04-25T09:57:13.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f573669e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300cf900 0x7f57300d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f573669e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57300cfe70 con 0x7f5730099a50 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f573669e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57300d0150 con 0x7f573015efb0 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f573669e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57300d0a80 con 0x7f57300cf900 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f5734c3b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5730099a50 0x7f57300cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41274/0 (socket says 172.21.15.139:41274) 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f5734c3b640 1 -- 172.21.15.139:0/142668785 learned_addr learned my addr 172.21.15.139:0/142668785 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572f7fe640 1 --2- 172.21.15.139:0/142668785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300cf900 0x7f57300d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979294029 0 0) 0x7f57300cfe70 con 0x7f5730099a50 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5718003520 con 0x7f5730099a50 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3751761265 0 0) 0x7f5718003520 con 0x7f5730099a50 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300cf900 msgr2=0x7f57300d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572effd640 1 --2- 172.21.15.139:0/142668785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300cf900 0x7f57300d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 >> v1:172.21.15.150:6789/0 conn(0x7f573015efb0 legacy=0x7f573006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57300d0430 con 0x7f5730099a50 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f57280019f0 con 0x7f5730099a50 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 >> v1:172.21.15.139:6789/0 conn(0x7f5730099a50 legacy=0x7f57300cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572effd640 1 --2- 172.21.15.139:0/142668785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57180039e0 0x7f5718003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572effd640 1 --2- 172.21.15.139:0/142668785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5718004800 0x7f5718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.455+0000 7f572effd640 1 --2- 172.21.15.139:0/142668785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300cf900 0x7f5718005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.456+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57300d0a80 con 0x7f57180039e0 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.456+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571800bb50 con 0x7f5718004800 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.456+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571800bd50 con 0x7f57300cf900 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.456+0000 7f572ffff640 1 --2- 172.21.15.139:0/142668785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57180039e0 0x7f5718003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.456+0000 7f572f7fe640 1 --2- 172.21.15.139:0/142668785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5718004800 0x7f5718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.456+0000 7f5734c3b640 1 --2- 172.21.15.139:0/142668785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300cf900 0x7f5718005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.456+0000 7f572ffff640 1 -- 172.21.15.139:0/142668785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300cf900 msgr2=0x7f5718005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.456+0000 7f572ffff640 1 --2- 172.21.15.139:0/142668785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300cf900 0x7f5718005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.456+0000 7f572ffff640 1 -- 172.21.15.139:0/142668785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5718004800 msgr2=0x7f5718004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.456+0000 7f572ffff640 1 --2- 172.21.15.139:0/142668785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5718004800 0x7f5718004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.456+0000 7f572ffff640 1 -- 172.21.15.139:0/142668785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5718003520 con 0x7f57180039e0 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.457+0000 7f572ffff640 1 --2- 172.21.15.139:0/142668785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57180039e0 0x7f5718003e00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5720001750 tx=0x7f5718005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a5654026a0b1ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.457+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5720001370 con 0x7f57180039e0 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.457+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57200052d0 con 0x7f57180039e0 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.457+0000 7f572effd640 1 -- 172.21.15.139:0/142668785 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572002abe0 con 0x7f57180039e0 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.457+0000 7f573669e640 1 -- 172.21.15.139:0/142668785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57180039e0 msgr2=0x7f5718003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.457+0000 7f573669e640 1 --2- 172.21.15.139:0/142668785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57180039e0 0x7f5718003e00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5720001750 tx=0x7f5718005830 comp rx=0 tx=0).stop 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.457+0000 7f5734c3b640 1 -- 172.21.15.139:0/142668785 reap_dead start 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.457+0000 7f573669e640 1 -- 172.21.15.139:0/142668785 shutdown_connections 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.457+0000 7f573669e640 1 -- 172.21.15.139:0/142668785 >> 172.21.15.139:0/142668785 conn(0x7f57300d1790 msgr2=0x7f57300d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.457+0000 7f573669e640 1 -- 172.21.15.139:0/142668785 shutdown_connections 2024-04-25T09:57:13.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.457+0000 7f573669e640 1 -- 172.21.15.139:0/142668785 wait complete. 2024-04-25T09:57:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.458+0000 7f573669e640 1 Processor -- start 2024-04-25T09:57:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.459+0000 7f573669e640 1 -- start start 2024-04-25T09:57:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.459+0000 7f573669e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57180039e0 0x7f573015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.459+0000 7f573669e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300b5a10 0x7f57301f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.459+0000 7f573669e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5730157b70 0x7f5730157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.459+0000 7f573669e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57300d0f10 con 0x7f5730157b70 2024-04-25T09:57:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.459+0000 7f573669e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573006cf50 con 0x7f57180039e0 2024-04-25T09:57:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.459+0000 7f573669e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57300d1090 con 0x7f57300b5a10 2024-04-25T09:57:13.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.459+0000 7f5734c3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5730157b70 0x7f5730157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.459+0000 7f5734c3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5730157b70 0x7f5730157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53714/0 (socket says 172.21.15.139:53714) 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.459+0000 7f5734c3b640 1 -- 172.21.15.139:0/2030317764 learned_addr learned my addr 172.21.15.139:0/2030317764 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f572f7fe640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300b5a10 0x7f57301f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f572ffff640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57180039e0 0x7f573015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f5734c3b640 1 -- 172.21.15.139:0/2030317764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300b5a10 msgr2=0x7f57301f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f5734c3b640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300b5a10 0x7f57301f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f5734c3b640 1 -- 172.21.15.139:0/2030317764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57180039e0 msgr2=0x7f573015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f5734c3b640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57180039e0 0x7f573015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f5734c3b640 1 -- 172.21.15.139:0/2030317764 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57301f4d00 con 0x7f5730157b70 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f572ffff640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57180039e0 0x7f573015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f5734c3b640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5730157b70 0x7f5730157ff0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5730072a90 tx=0x7f5728007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f572cff9640 1 -- 172.21.15.139:0/2030317764 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572800aab0 con 0x7f5730157b70 2024-04-25T09:57:13.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f572cff9640 1 -- 172.21.15.139:0/2030317764 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5728019070 con 0x7f5730157b70 2024-04-25T09:57:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f573669e640 1 -- 172.21.15.139:0/2030317764 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5730158950 con 0x7f5730157b70 2024-04-25T09:57:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f572cff9640 1 -- 172.21.15.139:0/2030317764 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280133f0 con 0x7f5730157b70 2024-04-25T09:57:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.460+0000 7f573669e640 1 -- 172.21.15.139:0/2030317764 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5730159550 con 0x7f5730157b70 2024-04-25T09:57:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.461+0000 7f573669e640 1 -- 172.21.15.139:0/2030317764 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5730074520 con 0x7f5730157b70 2024-04-25T09:57:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.461+0000 7f572cff9640 1 -- 172.21.15.139:0/2030317764 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5728013590 con 0x7f5730157b70 2024-04-25T09:57:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.462+0000 7f572cff9640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f571c06b4d0 0x7f571c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.462+0000 7f572cff9640 1 -- 172.21.15.139:0/2030317764 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f572808a590 con 0x7f5730157b70 2024-04-25T09:57:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.462+0000 7f572ffff640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f571c06b4d0 0x7f571c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.462+0000 7f572ffff640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f571c06b4d0 0x7f571c06d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5720001750 tx=0x7f572002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.466+0000 7f572cff9640 1 -- 172.21.15.139:0/2030317764 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57301f4d00 con 0x7f5730157b70 2024-04-25T09:57:13.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:13.700+0000 7f573669e640 1 -- 172.21.15.139:0/2030317764 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f573009ae50 con 0x7f5730157b70 2024-04-25T09:57:14.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.042+0000 7f572cff9640 1 -- 172.21.15.139:0/2030317764 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v211) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5728059580 con 0x7f5730157b70 2024-04-25T09:57:14.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.044+0000 7f573669e640 1 -- 172.21.15.139:0/2030317764 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f571c06b4d0 msgr2=0x7f571c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.044+0000 7f573669e640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f571c06b4d0 0x7f571c06d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5720001750 tx=0x7f572002e000 comp rx=0 tx=0).stop 2024-04-25T09:57:14.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.044+0000 7f573669e640 1 -- 172.21.15.139:0/2030317764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5730157b70 msgr2=0x7f5730157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.044+0000 7f573669e640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5730157b70 0x7f5730157ff0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5730072a90 tx=0x7f5728007b10 comp rx=0 tx=0).stop 2024-04-25T09:57:14.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.045+0000 7f573669e640 1 -- 172.21.15.139:0/2030317764 shutdown_connections 2024-04-25T09:57:14.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.045+0000 7f573669e640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5730157b70 0x7f5730157ff0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.045+0000 7f573669e640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f571c06b4d0 0x7f571c06d990 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.045+0000 7f573669e640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57300b5a10 0x7f57301f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.045+0000 7f573669e640 1 --2- 172.21.15.139:0/2030317764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57180039e0 0x7f573015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.045+0000 7f573669e640 1 -- 172.21.15.139:0/2030317764 >> 172.21.15.139:0/2030317764 conn(0x7f57300d1790 msgr2=0x7f57300cae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.045+0000 7f573669e640 1 -- 172.21.15.139:0/2030317764 shutdown_connections 2024-04-25T09:57:14.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.045+0000 7f573669e640 1 -- 172.21.15.139:0/2030317764 wait complete. 2024-04-25T09:57:14.060 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T09:57:14.060 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T09:57:14.060 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T09:57:14.061 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:14.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.154+0000 7ff3319e2640 1 Processor -- start 2024-04-25T09:57:14.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.154+0000 7ff3319e2640 1 -- start start 2024-04-25T09:57:14.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff3319e2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff32c0d33a0 0x7ff32c06c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff3319e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c0d4660 0x7ff32c06c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff3319e2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c0d1f40 con 0x7ff32c161120 2024-04-25T09:57:14.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff3319e2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32c0cf190 con 0x7ff32c0d33a0 2024-04-25T09:57:14.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff3319e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0ce980 con 0x7ff32c0d4660 2024-04-25T09:57:14.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff323fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff32c0d33a0 0x7ff32c06c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff32affd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff32c161120 0x7ff32c0b6ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41276/0 (socket says 172.21.15.139:41276) 2024-04-25T09:57:14.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff32affd640 1 -- 172.21.15.139:0/4273900173 learned_addr learned my addr 172.21.15.139:0/4273900173 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:14.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff32b7fe640 1 --2- 172.21.15.139:0/4273900173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c0d4660 0x7ff32c06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff323fff640 1 -- 172.21.15.139:0/4273900173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c0d4660 msgr2=0x7ff32c06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff323fff640 1 --2- 172.21.15.139:0/4273900173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c0d4660 0x7ff32c06c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff323fff640 1 -- 172.21.15.139:0/4273900173 >> v1:172.21.15.139:6789/0 conn(0x7ff32c161120 legacy=0x7ff32c0b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.155+0000 7ff323fff640 1 -- 172.21.15.139:0/4273900173 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0cebd0 con 0x7ff32c0d33a0 2024-04-25T09:57:14.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.156+0000 7ff323fff640 1 --2- 172.21.15.139:0/4273900173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff32c0d33a0 0x7ff32c06c030 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0ceb00 tx=0x7ff31402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a894f7dc75e8355a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.156+0000 7ff32a7fc640 1 -- 172.21.15.139:0/4273900173 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff314037070 con 0x7ff32c0d33a0 2024-04-25T09:57:14.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.156+0000 7ff32a7fc640 1 -- 172.21.15.139:0/4273900173 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31402ae20 con 0x7ff32c0d33a0 2024-04-25T09:57:14.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.156+0000 7ff32a7fc640 1 -- 172.21.15.139:0/4273900173 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff314032660 con 0x7ff32c0d33a0 2024-04-25T09:57:14.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.156+0000 7ff3319e2640 1 -- 172.21.15.139:0/4273900173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff32c0d33a0 msgr2=0x7ff32c06c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.156+0000 7ff3319e2640 1 --2- 172.21.15.139:0/4273900173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff32c0d33a0 0x7ff32c06c030 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0ceb00 tx=0x7ff31402a110 comp rx=0 tx=0).stop 2024-04-25T09:57:14.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.157+0000 7ff3319e2640 1 -- 172.21.15.139:0/4273900173 shutdown_connections 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.157+0000 7ff3319e2640 1 --2- 172.21.15.139:0/4273900173 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c0d4660 0x7ff32c06c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.157+0000 7ff3319e2640 1 --2- 172.21.15.139:0/4273900173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff32c0d33a0 0x7ff32c06c030 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.157+0000 7ff3319e2640 1 -- 172.21.15.139:0/4273900173 >> 172.21.15.139:0/4273900173 conn(0x7ff32c0d0a60 msgr2=0x7ff32c0d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.157+0000 7ff3319e2640 1 -- 172.21.15.139:0/4273900173 shutdown_connections 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.157+0000 7ff3319e2640 1 -- 172.21.15.139:0/4273900173 wait complete. 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.158+0000 7ff3319e2640 1 Processor -- start 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.158+0000 7ff3319e2640 1 -- start start 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.158+0000 7ff3319e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff32c0d4660 0x7ff32c156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.158+0000 7ff3319e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c161120 0x7ff32c0b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.158+0000 7ff3319e2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff32c149c80 0x7ff32c0b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.158+0000 7ff3319e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0b71f0 con 0x7ff32c0d4660 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.158+0000 7ff3319e2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0d21e0 con 0x7ff32c149c80 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.158+0000 7ff3319e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c161500 con 0x7ff32c161120 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.158+0000 7ff32affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff32c0d4660 0x7ff32c156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff32affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff32c0d4660 0x7ff32c156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53742/0 (socket says 172.21.15.139:53742) 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff32affd640 1 -- 172.21.15.139:0/4160566412 learned_addr learned my addr 172.21.15.139:0/4160566412 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff323fff640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c161120 0x7ff32c0b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff32affd640 1 -- 172.21.15.139:0/4160566412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c161120 msgr2=0x7ff32c0b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff32affd640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c161120 0x7ff32c0b5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff32affd640 1 -- 172.21.15.139:0/4160566412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff32c149c80 msgr2=0x7ff32c0b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff32affd640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff32c149c80 0x7ff32c0b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff32affd640 1 -- 172.21.15.139:0/4160566412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c159280 con 0x7ff32c0d4660 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff32affd640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff32c0d4660 0x7ff32c156b70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00c400 tx=0x7ff31c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff328ff9640 1 -- 172.21.15.139:0/4160566412 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c017070 con 0x7ff32c0d4660 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff328ff9640 1 -- 172.21.15.139:0/4160566412 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c005ce0 con 0x7ff32c0d4660 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff3319e2640 1 -- 172.21.15.139:0/4160566412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32c159560 con 0x7ff32c0d4660 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff328ff9640 1 -- 172.21.15.139:0/4160566412 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c0072f0 con 0x7ff32c0d4660 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.159+0000 7ff3319e2640 1 -- 172.21.15.139:0/4160566412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c159b20 con 0x7ff32c0d4660 2024-04-25T09:57:14.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.160+0000 7ff3319e2640 1 -- 172.21.15.139:0/4160566412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c074520 con 0x7ff32c0d4660 2024-04-25T09:57:14.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.160+0000 7ff328ff9640 1 -- 172.21.15.139:0/4160566412 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff31c01a070 con 0x7ff32c0d4660 2024-04-25T09:57:14.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.161+0000 7ff328ff9640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff30c06b500 0x7ff30c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.161+0000 7ff328ff9640 1 -- 172.21.15.139:0/4160566412 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff31c0894b0 con 0x7ff32c0d4660 2024-04-25T09:57:14.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.161+0000 7ff323fff640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff30c06b500 0x7ff30c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.161+0000 7ff323fff640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff30c06b500 0x7ff30c06d9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff314034000 tx=0x7ff31402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.165+0000 7ff328ff9640 1 -- 172.21.15.139:0/4160566412 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff31c0584a0 con 0x7ff32c0d4660 2024-04-25T09:57:14.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.334+0000 7ff3319e2640 1 -- 172.21.15.139:0/4160566412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff32c06cd80 con 0x7ff32c0d4660 2024-04-25T09:57:14.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.335+0000 7ff328ff9640 1 -- 172.21.15.139:0/4160566412 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+13344 (secure 0 0 0) 0x7ff31c057da0 con 0x7ff32c0d4660 2024-04-25T09:57:14.339 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:14.339 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":67,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:57:13.042038+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6837/3541444834":"2024-04-26T09:57:10.037738+0000","172.21.15.139:6835/3541444834":"2024-04-26T09:57:10.037738+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:14.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.337+0000 7ff3319e2640 1 -- 172.21.15.139:0/4160566412 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff30c06b500 msgr2=0x7ff30c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.337+0000 7ff3319e2640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff30c06b500 0x7ff30c06d9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff314034000 tx=0x7ff31402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:14.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.337+0000 7ff3319e2640 1 -- 172.21.15.139:0/4160566412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff32c0d4660 msgr2=0x7ff32c156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.337+0000 7ff3319e2640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff32c0d4660 0x7ff32c156b70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00c400 tx=0x7ff31c00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.337+0000 7ff3319e2640 1 -- 172.21.15.139:0/4160566412 shutdown_connections 2024-04-25T09:57:14.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.337+0000 7ff3319e2640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff30c06b500 0x7ff30c06d9c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.337+0000 7ff3319e2640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff32c149c80 0x7ff32c0b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.337+0000 7ff3319e2640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c161120 0x7ff32c0b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.337+0000 7ff3319e2640 1 --2- 172.21.15.139:0/4160566412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff32c0d4660 0x7ff32c156b70 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.337+0000 7ff3319e2640 1 -- 172.21.15.139:0/4160566412 >> 172.21.15.139:0/4160566412 conn(0x7ff32c0d0a60 msgr2=0x7ff32c06c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.337+0000 7ff3319e2640 1 -- 172.21.15.139:0/4160566412 shutdown_connections 2024-04-25T09:57:14.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.337+0000 7ff3319e2640 1 -- 172.21.15.139:0/4160566412 wait complete. 2024-04-25T09:57:14.352 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:14.353 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:14.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.448+0000 7f4425a88640 1 Processor -- start 2024-04-25T09:57:14.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.448+0000 7f4425a88640 1 -- start start 2024-04-25T09:57:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.449+0000 7f4425a88640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f442015dc90 0x7f44200ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.449+0000 7f4425a88640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44200cf980 con 0x7f442006f300 2024-04-25T09:57:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.449+0000 7f4425a88640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442006ab60 con 0x7f44200cf3e0 2024-04-25T09:57:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.449+0000 7f4425a88640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f442006ae40 con 0x7f442015dc90 2024-04-25T09:57:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.449+0000 7f441f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f442015dc90 0x7f44200ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.449+0000 7f441ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f442006f300 0x7f44200b2db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41282/0 (socket says 172.21.15.139:41282) 2024-04-25T09:57:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.449+0000 7f441ffff640 1 -- 172.21.15.139:0/426673598 learned_addr learned my addr 172.21.15.139:0/426673598 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.449+0000 7f441f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f442015dc90 0x7f44200ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53746/0 (socket says 172.21.15.139:53746) 2024-04-25T09:57:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.450+0000 7f441f7fe640 1 -- 172.21.15.139:0/426673598 >> v1:172.21.15.150:6789/0 conn(0x7f44200cf3e0 legacy=0x7f442006ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.450+0000 7f441f7fe640 1 -- 172.21.15.139:0/426673598 >> v1:172.21.15.139:6789/0 conn(0x7f442006f300 legacy=0x7f44200b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.450+0000 7f441f7fe640 1 -- 172.21.15.139:0/426673598 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442006b120 con 0x7f442015dc90 2024-04-25T09:57:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.450+0000 7f441effd640 1 -- 172.21.15.139:0/426673598 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104603257 0 0) 0x7f44200cf980 con 0x7f442006f300 2024-04-25T09:57:14.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.450+0000 7f441f7fe640 1 --2- 172.21.15.139:0/426673598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f442015dc90 0x7f44200ceea0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f44100097a0 tx=0x7f441002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6386f953569b3953 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.451+0000 7f441effd640 1 -- 172.21.15.139:0/426673598 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410037070 con 0x7f442015dc90 2024-04-25T09:57:14.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.451+0000 7f441effd640 1 -- 172.21.15.139:0/426673598 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4410032440 con 0x7f442015dc90 2024-04-25T09:57:14.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.451+0000 7f441effd640 1 -- 172.21.15.139:0/426673598 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410032810 con 0x7f442015dc90 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.451+0000 7f4425a88640 1 -- 172.21.15.139:0/426673598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f442015dc90 msgr2=0x7f44200ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.451+0000 7f4425a88640 1 --2- 172.21.15.139:0/426673598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f442015dc90 0x7f44200ceea0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f44100097a0 tx=0x7f441002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.451+0000 7f4425a88640 1 -- 172.21.15.139:0/426673598 shutdown_connections 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.451+0000 7f4425a88640 1 --2- 172.21.15.139:0/426673598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f442015dc90 0x7f44200ceea0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.451+0000 7f4425a88640 1 -- 172.21.15.139:0/426673598 >> 172.21.15.139:0/426673598 conn(0x7f44200cfd80 msgr2=0x7f44200d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.451+0000 7f4425a88640 1 -- 172.21.15.139:0/426673598 shutdown_connections 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.451+0000 7f4425a88640 1 -- 172.21.15.139:0/426673598 wait complete. 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.452+0000 7f4425a88640 1 Processor -- start 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.452+0000 7f4425a88640 1 -- start start 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f4425a88640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442006f300 0x7f4420148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f4425a88640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44200cf3e0 0x7f442015e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f4425a88640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f442015dc90 0x7f442015e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f4425a88640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442006b920 con 0x7f442015dc90 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f4425a88640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4420072620 con 0x7f442006f300 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f4425a88640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44200b34c0 con 0x7f44200cf3e0 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f441ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f442015dc90 0x7f442015e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f441ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f442015dc90 0x7f442015e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53754/0 (socket says 172.21.15.139:53754) 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f441ffff640 1 -- 172.21.15.139:0/3722230756 learned_addr learned my addr 172.21.15.139:0/3722230756 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f441f7fe640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442006f300 0x7f4420148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f4417fff640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44200cf3e0 0x7f442015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f441ffff640 1 -- 172.21.15.139:0/3722230756 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44200cf3e0 msgr2=0x7f442015e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f441ffff640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44200cf3e0 0x7f442015e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f441ffff640 1 -- 172.21.15.139:0/3722230756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442006f300 msgr2=0x7f4420148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f441ffff640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442006f300 0x7f4420148800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.453+0000 7f441ffff640 1 -- 172.21.15.139:0/3722230756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420148d40 con 0x7f442015dc90 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.454+0000 7f441ffff640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f442015dc90 0x7f442015e5b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f441800c800 tx=0x7f441800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.454+0000 7f441cff9640 1 -- 172.21.15.139:0/3722230756 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418018070 con 0x7f442015dc90 2024-04-25T09:57:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.454+0000 7f441cff9640 1 -- 172.21.15.139:0/3722230756 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4418004d80 con 0x7f442015dc90 2024-04-25T09:57:14.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.454+0000 7f4425a88640 1 -- 172.21.15.139:0/3722230756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442015ebe0 con 0x7f442015dc90 2024-04-25T09:57:14.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.454+0000 7f441cff9640 1 -- 172.21.15.139:0/3722230756 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441800a880 con 0x7f442015dc90 2024-04-25T09:57:14.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.454+0000 7f4425a88640 1 -- 172.21.15.139:0/3722230756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442015f1a0 con 0x7f442015dc90 2024-04-25T09:57:14.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.455+0000 7f4425a88640 1 -- 172.21.15.139:0/3722230756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4420074520 con 0x7f442015dc90 2024-04-25T09:57:14.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.455+0000 7f441cff9640 1 -- 172.21.15.139:0/3722230756 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f441801b070 con 0x7f442015dc90 2024-04-25T09:57:14.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.455+0000 7f441cff9640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f440006b4d0 0x7f440006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.456+0000 7f441cff9640 1 -- 172.21.15.139:0/3722230756 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f44180895e0 con 0x7f442015dc90 2024-04-25T09:57:14.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.456+0000 7f441f7fe640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f440006b4d0 0x7f440006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.456+0000 7f441f7fe640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f440006b4d0 0x7f440006d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4410034000 tx=0x7f441002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.460+0000 7f441cff9640 1 -- 172.21.15.139:0/3722230756 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4418058670 con 0x7f442015dc90 2024-04-25T09:57:14.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.629+0000 7f4425a88640 1 -- 172.21.15.139:0/3722230756 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f442006dae0 con 0x7f442015dc90 2024-04-25T09:57:14.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.629+0000 7f441cff9640 1 -- 172.21.15.139:0/3722230756 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4418057f70 con 0x7f442015dc90 2024-04-25T09:57:14.633 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:57:14.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.631+0000 7f4425a88640 1 -- 172.21.15.139:0/3722230756 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f440006b4d0 msgr2=0x7f440006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.631+0000 7f4425a88640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f440006b4d0 0x7f440006d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4410034000 tx=0x7f441002e000 comp rx=0 tx=0).stop 2024-04-25T09:57:14.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.631+0000 7f4425a88640 1 -- 172.21.15.139:0/3722230756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f442015dc90 msgr2=0x7f442015e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.631+0000 7f4425a88640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f442015dc90 0x7f442015e5b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f441800c800 tx=0x7f441800ccc0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.631+0000 7f4425a88640 1 -- 172.21.15.139:0/3722230756 shutdown_connections 2024-04-25T09:57:14.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.631+0000 7f4425a88640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f442015dc90 0x7f442015e5b0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.631+0000 7f4425a88640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f440006b4d0 0x7f440006d990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.631+0000 7f4425a88640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44200cf3e0 0x7f442015e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.631+0000 7f4425a88640 1 --2- 172.21.15.139:0/3722230756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442006f300 0x7f4420148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.631+0000 7f4425a88640 1 -- 172.21.15.139:0/3722230756 >> 172.21.15.139:0/3722230756 conn(0x7f44200cfd80 msgr2=0x7f44200b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.631+0000 7f4425a88640 1 -- 172.21.15.139:0/3722230756 shutdown_connections 2024-04-25T09:57:14.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.631+0000 7f4425a88640 1 -- 172.21.15.139:0/3722230756 wait complete. 2024-04-25T09:57:14.646 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T09:57:14.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.739+0000 7f2b7c3b0640 1 Processor -- start 2024-04-25T09:57:14.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.739+0000 7f2b7c3b0640 1 -- start start 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7c3b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b7415efb0 0x7f2b740d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7c3b0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b740cfe70 con 0x7f2b74099a50 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7c3b0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b740d0150 con 0x7f2b740cf900 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7c3b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b740d0a80 con 0x7f2b7415efb0 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7a94d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2b74099a50 0x7f2b740cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41292/0 (socket says 172.21.15.139:41292) 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7a94d640 1 -- 172.21.15.139:0/1583358243 learned_addr learned my addr 172.21.15.139:0/1583358243 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7a14c640 1 --2- 172.21.15.139:0/1583358243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b7415efb0 0x7f2b740d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599012566 0 0) 0x7f2b740cfe70 con 0x7f2b74099a50 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b60003520 con 0x7f2b74099a50 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1011533715 0 0) 0x7f2b60003520 con 0x7f2b74099a50 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b7415efb0 msgr2=0x7f2b740d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 --2- 172.21.15.139:0/1583358243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b7415efb0 0x7f2b740d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 >> v1:172.21.15.150:6789/0 conn(0x7f2b740cf900 legacy=0x7f2b7406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b740d0430 con 0x7f2b74099a50 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f2b6c0019f0 con 0x7f2b74099a50 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 >> v1:172.21.15.139:6789/0 conn(0x7f2b74099a50 legacy=0x7f2b740cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 --2- 172.21.15.139:0/1583358243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b600039e0 0x7f2b60003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 --2- 172.21.15.139:0/1583358243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b7415efb0 0x7f2b60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 --2- 172.21.15.139:0/1583358243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b60004d40 0x7f2b600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b740d0a80 con 0x7f2b600039e0 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6000ba70 con 0x7f2b60004d40 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6000bc70 con 0x7f2b7415efb0 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.740+0000 7f2b7a14c640 1 --2- 172.21.15.139:0/1583358243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b600039e0 0x7f2b60003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.741+0000 7f2b7994b640 1 --2- 172.21.15.139:0/1583358243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b7415efb0 0x7f2b60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.741+0000 7f2b7a94d640 1 --2- 172.21.15.139:0/1583358243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b60004d40 0x7f2b600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.741+0000 7f2b7a14c640 1 -- 172.21.15.139:0/1583358243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b7415efb0 msgr2=0x7f2b60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.741+0000 7f2b7a14c640 1 --2- 172.21.15.139:0/1583358243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b7415efb0 0x7f2b60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.741+0000 7f2b7a14c640 1 -- 172.21.15.139:0/1583358243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b60004d40 msgr2=0x7f2b600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.741+0000 7f2b7a14c640 1 --2- 172.21.15.139:0/1583358243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b60004d40 0x7f2b600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.741+0000 7f2b7a14c640 1 -- 172.21.15.139:0/1583358243 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b60003520 con 0x7f2b600039e0 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.741+0000 7f2b7a14c640 1 --2- 172.21.15.139:0/1583358243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b600039e0 0x7f2b60003dc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009ac0 tx=0x7f2b60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24a4388ea09129d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.741+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70036070 con 0x7f2b600039e0 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.742+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b70031400 con 0x7f2b600039e0 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.742+0000 7f2b7914a640 1 -- 172.21.15.139:0/1583358243 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70009690 con 0x7f2b600039e0 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.742+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/1583358243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b600039e0 msgr2=0x7f2b60003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.742+0000 7f2b7c3b0640 1 --2- 172.21.15.139:0/1583358243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b600039e0 0x7f2b60003dc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009ac0 tx=0x7f2b60005830 comp rx=0 tx=0).stop 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.742+0000 7f2b7a94d640 1 -- 172.21.15.139:0/1583358243 reap_dead start 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.742+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/1583358243 shutdown_connections 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.742+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/1583358243 >> 172.21.15.139:0/1583358243 conn(0x7f2b740d1790 msgr2=0x7f2b740d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.742+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/1583358243 shutdown_connections 2024-04-25T09:57:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.742+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/1583358243 wait complete. 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.743+0000 7f2b7c3b0640 1 Processor -- start 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.743+0000 7f2b7c3b0640 1 -- start start 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7c3b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b600039e0 0x7f2b74157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7c3b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b74159b80 0x7f2b740b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7c3b0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b740b62f0 0x7f2b740b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7c3b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b740d0f10 con 0x7f2b74159b80 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7c3b0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7406cf50 con 0x7f2b740b62f0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7c3b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b740d1090 con 0x7f2b600039e0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7994b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b74159b80 0x7f2b740b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7994b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b74159b80 0x7f2b740b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53766/0 (socket says 172.21.15.139:53766) 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7994b640 1 -- 172.21.15.139:0/3953444531 learned_addr learned my addr 172.21.15.139:0/3953444531 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7a14c640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b600039e0 0x7f2b74157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7a94d640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b740b62f0 0x7f2b740b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7994b640 1 -- 172.21.15.139:0/3953444531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b600039e0 msgr2=0x7f2b74157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7994b640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b600039e0 0x7f2b74157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7994b640 1 -- 172.21.15.139:0/3953444531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b740b62f0 msgr2=0x7f2b740b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7994b640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b740b62f0 0x7f2b740b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7994b640 1 -- 172.21.15.139:0/3953444531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b740c7ca0 con 0x7f2b74159b80 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.744+0000 7f2b7994b640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b74159b80 0x7f2b740b5490 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400f830 tx=0x7f2b6400fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.745+0000 7f2b6affd640 1 -- 172.21.15.139:0/3953444531 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640052e0 con 0x7f2b74159b80 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.745+0000 7f2b6affd640 1 -- 172.21.15.139:0/3953444531 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b64018070 con 0x7f2b74159b80 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.745+0000 7f2b6affd640 1 -- 172.21.15.139:0/3953444531 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400d8f0 con 0x7f2b74159b80 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.745+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/3953444531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b740c7f40 con 0x7f2b74159b80 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.745+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/3953444531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b740c5740 con 0x7f2b74159b80 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.746+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/3953444531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b740cec10 con 0x7f2b74159b80 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.746+0000 7f2b6affd640 1 -- 172.21.15.139:0/3953444531 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b6400da90 con 0x7f2b74159b80 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.746+0000 7f2b6affd640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b5806b500 0x7f2b5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.746+0000 7f2b6affd640 1 -- 172.21.15.139:0/3953444531 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2b64089fa0 con 0x7f2b74159b80 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.746+0000 7f2b7a14c640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b5806b500 0x7f2b5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.747+0000 7f2b7a14c640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b5806b500 0x7f2b5806d9c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2b70033000 tx=0x7f2b70008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.750+0000 7f2b6affd640 1 -- 172.21.15.139:0/3953444531 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b740cec10 con 0x7f2b74159b80 2024-04-25T09:57:14.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:14.988+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/3953444531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f2b74074520 con 0x7f2b74159b80 2024-04-25T09:57:15.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.047+0000 7f2b6affd640 1 -- 172.21.15.139:0/3953444531 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v212) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2b6401a310 con 0x7f2b74159b80 2024-04-25T09:57:15.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.049+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/3953444531 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b5806b500 msgr2=0x7f2b5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.049+0000 7f2b7c3b0640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b5806b500 0x7f2b5806d9c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2b70033000 tx=0x7f2b70008030 comp rx=0 tx=0).stop 2024-04-25T09:57:15.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.049+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/3953444531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b74159b80 msgr2=0x7f2b740b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.049+0000 7f2b7c3b0640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b74159b80 0x7f2b740b5490 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400f830 tx=0x7f2b6400fcf0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.049+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/3953444531 shutdown_connections 2024-04-25T09:57:15.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.049+0000 7f2b7c3b0640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b74159b80 0x7f2b740b5490 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.049+0000 7f2b7c3b0640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b5806b500 0x7f2b5806d9c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.049+0000 7f2b7c3b0640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b740b62f0 0x7f2b740b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.049+0000 7f2b7c3b0640 1 --2- 172.21.15.139:0/3953444531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b600039e0 0x7f2b74157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.049+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/3953444531 >> 172.21.15.139:0/3953444531 conn(0x7f2b740d1790 msgr2=0x7f2b740cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.049+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/3953444531 shutdown_connections 2024-04-25T09:57:15.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.049+0000 7f2b7c3b0640 1 -- 172.21.15.139:0/3953444531 wait complete. 2024-04-25T09:57:15.064 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:57:15.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.159+0000 7f554cb7e640 1 Processor -- start 2024-04-25T09:57:15.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.159+0000 7f554cb7e640 1 -- start start 2024-04-25T09:57:15.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f554cb7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55480b31a0 0x7f554806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f554cb7e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f554806e870 0x7f554815e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f554cb7e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55480b2a50 con 0x7f5548144090 2024-04-25T09:57:15.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f554cb7e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55480b35e0 con 0x7f554806e870 2024-04-25T09:57:15.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f554cb7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55480cd0f0 con 0x7f55480b31a0 2024-04-25T09:57:15.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f5546d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f554806e870 0x7f554815e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f554659c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5548144090 0x7f55480cc9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41300/0 (socket says 172.21.15.139:41300) 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f554659c640 1 -- 172.21.15.139:0/3552324544 learned_addr learned my addr 172.21.15.139:0/3552324544 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f5545d9b640 1 --2- 172.21.15.139:0/3552324544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55480b31a0 0x7f554806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f5546d9d640 1 -- 172.21.15.139:0/3552324544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55480b31a0 msgr2=0x7f554806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f5546d9d640 1 --2- 172.21.15.139:0/3552324544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55480b31a0 0x7f554806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f5546d9d640 1 -- 172.21.15.139:0/3552324544 >> v1:172.21.15.139:6789/0 conn(0x7f5548144090 legacy=0x7f55480cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.161+0000 7f5546d9d640 1 -- 172.21.15.139:0/3552324544 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554815b090 con 0x7f554806e870 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.160+0000 7f554559a640 1 -- 172.21.15.139:0/3552324544 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381436358 0 0) 0x7f5538001490 con 0x7f5548144090 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.161+0000 7f5545d9b640 1 --2- 172.21.15.139:0/3552324544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55480b31a0 0x7f554806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.161+0000 7f5546d9d640 1 --2- 172.21.15.139:0/3552324544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554806e870 0x7f554815e610 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f554000ba40 tx=0x7f554002c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fa7fe613df79f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.161+0000 7f554559a640 1 -- 172.21.15.139:0/3552324544 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540031710 con 0x7f554806e870 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.161+0000 7f554559a640 1 -- 172.21.15.139:0/3552324544 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55400318b0 con 0x7f554806e870 2024-04-25T09:57:15.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.161+0000 7f554559a640 1 -- 172.21.15.139:0/3552324544 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540031c80 con 0x7f554806e870 2024-04-25T09:57:15.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.162+0000 7f554cb7e640 1 -- 172.21.15.139:0/3552324544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554806e870 msgr2=0x7f554815e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.162+0000 7f554cb7e640 1 --2- 172.21.15.139:0/3552324544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554806e870 0x7f554815e610 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f554000ba40 tx=0x7f554002c7b0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.162+0000 7f554cb7e640 1 -- 172.21.15.139:0/3552324544 shutdown_connections 2024-04-25T09:57:15.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.162+0000 7f554cb7e640 1 --2- 172.21.15.139:0/3552324544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554806e870 0x7f554815e610 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.162+0000 7f554cb7e640 1 --2- 172.21.15.139:0/3552324544 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55480b31a0 0x7f554806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.162+0000 7f554cb7e640 1 -- 172.21.15.139:0/3552324544 >> 172.21.15.139:0/3552324544 conn(0x7f5548067000 msgr2=0x7f554806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.162+0000 7f554cb7e640 1 -- 172.21.15.139:0/3552324544 shutdown_connections 2024-04-25T09:57:15.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.162+0000 7f554cb7e640 1 -- 172.21.15.139:0/3552324544 wait complete. 2024-04-25T09:57:15.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.163+0000 7f554cb7e640 1 Processor -- start 2024-04-25T09:57:15.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.163+0000 7f554cb7e640 1 -- start start 2024-04-25T09:57:15.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554cb7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554806e870 0x7f55481f0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554cb7e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55480b31a0 0x7f55480a8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554cb7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5548144090 0x7f55480a9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554cb7e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554815b930 con 0x7f554806e870 2024-04-25T09:57:15.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554cb7e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554815b650 con 0x7f55480b31a0 2024-04-25T09:57:15.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554cb7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554815bc10 con 0x7f5548144090 2024-04-25T09:57:15.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554806e870 0x7f55481f0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554806e870 0x7f55481f0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53776/0 (socket says 172.21.15.139:53776) 2024-04-25T09:57:15.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554659c640 1 -- 172.21.15.139:0/2885420289 learned_addr learned my addr 172.21.15.139:0/2885420289 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:15.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554659c640 1 -- 172.21.15.139:0/2885420289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5548144090 msgr2=0x7f55480a9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554659c640 1 --2- 172.21.15.139:0/2885420289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5548144090 0x7f55480a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554659c640 1 -- 172.21.15.139:0/2885420289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55480b31a0 msgr2=0x7f55480a8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554659c640 1 --2- 172.21.15.139:0/2885420289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55480b31a0 0x7f55480a8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554659c640 1 -- 172.21.15.139:0/2885420289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554815c080 con 0x7f554806e870 2024-04-25T09:57:15.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.164+0000 7f554659c640 1 --2- 172.21.15.139:0/2885420289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554806e870 0x7f55481f0250 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f553800c7f0 tx=0x7f553800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.165+0000 7f5536ffd640 1 -- 172.21.15.139:0/2885420289 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538018070 con 0x7f554806e870 2024-04-25T09:57:15.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.165+0000 7f5536ffd640 1 -- 172.21.15.139:0/2885420289 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553800a640 con 0x7f554806e870 2024-04-25T09:57:15.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.165+0000 7f554cb7e640 1 -- 172.21.15.139:0/2885420289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554815c360 con 0x7f554806e870 2024-04-25T09:57:15.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.165+0000 7f5536ffd640 1 -- 172.21.15.139:0/2885420289 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553800aa30 con 0x7f554806e870 2024-04-25T09:57:15.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.165+0000 7f554cb7e640 1 -- 172.21.15.139:0/2885420289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55481f0790 con 0x7f554806e870 2024-04-25T09:57:15.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.166+0000 7f554cb7e640 1 -- 172.21.15.139:0/2885420289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5548074520 con 0x7f554806e870 2024-04-25T09:57:15.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.166+0000 7f5536ffd640 1 -- 172.21.15.139:0/2885420289 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f553801b070 con 0x7f554806e870 2024-04-25T09:57:15.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.166+0000 7f5536ffd640 1 --2- 172.21.15.139:0/2885420289 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f552806b500 0x7f552806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.166+0000 7f5545d9b640 1 --2- 172.21.15.139:0/2885420289 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f552806b500 0x7f552806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.166+0000 7f5536ffd640 1 -- 172.21.15.139:0/2885420289 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f55380896c0 con 0x7f554806e870 2024-04-25T09:57:15.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.167+0000 7f5545d9b640 1 --2- 172.21.15.139:0/2885420289 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f552806b500 0x7f552806d9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f553c006fd0 tx=0x7f553c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.170+0000 7f5536ffd640 1 -- 172.21.15.139:0/2885420289 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55480b2a50 con 0x7f554806e870 2024-04-25T09:57:15.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.340+0000 7f554cb7e640 1 -- 172.21.15.139:0/2885420289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f55480cec20 con 0x7f554806e870 2024-04-25T09:57:15.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.345+0000 7f5536ffd640 1 -- 172.21.15.139:0/2885420289 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f55380586b0 con 0x7f554806e870 2024-04-25T09:57:15.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.347+0000 7f554cb7e640 1 -- 172.21.15.139:0/2885420289 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f552806b500 msgr2=0x7f552806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.347+0000 7f554cb7e640 1 --2- 172.21.15.139:0/2885420289 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f552806b500 0x7f552806d9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f553c006fd0 tx=0x7f553c008040 comp rx=0 tx=0).stop 2024-04-25T09:57:15.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.347+0000 7f554cb7e640 1 -- 172.21.15.139:0/2885420289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554806e870 msgr2=0x7f55481f0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.347+0000 7f554cb7e640 1 --2- 172.21.15.139:0/2885420289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554806e870 0x7f55481f0250 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f553800c7f0 tx=0x7f553800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.347+0000 7f554cb7e640 1 -- 172.21.15.139:0/2885420289 shutdown_connections 2024-04-25T09:57:15.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.347+0000 7f554cb7e640 1 --2- 172.21.15.139:0/2885420289 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f552806b500 0x7f552806d9c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.347+0000 7f554cb7e640 1 --2- 172.21.15.139:0/2885420289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5548144090 0x7f55480a9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.347+0000 7f554cb7e640 1 --2- 172.21.15.139:0/2885420289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55480b31a0 0x7f55480a8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.347+0000 7f554cb7e640 1 --2- 172.21.15.139:0/2885420289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554806e870 0x7f55481f0250 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.347+0000 7f554cb7e640 1 -- 172.21.15.139:0/2885420289 >> 172.21.15.139:0/2885420289 conn(0x7f5548067000 msgr2=0x7f554806dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.347+0000 7f554cb7e640 1 -- 172.21.15.139:0/2885420289 shutdown_connections 2024-04-25T09:57:15.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.347+0000 7f554cb7e640 1 -- 172.21.15.139:0/2885420289 wait complete. 2024-04-25T09:57:15.363 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:57:15.363 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:15.394 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:15.446 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:57:15.446 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:57:15.446 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:15.462 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:57:15.462 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:57:15.462 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:15.479 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:15.479 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:57:15.479 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:15.517 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:57:15.517 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:57:15.517 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:15.536 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:15.536 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:57:15.536 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:15.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.649+0000 7fbccffce640 1 Processor -- start 2024-04-25T09:57:15.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.650+0000 7fbccffce640 1 -- start start 2024-04-25T09:57:15.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.650+0000 7fbccffce640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbcc815ae40 0x7fbcc80b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.650+0000 7fbccffce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcc80cefe0 0x7fbcc806d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.650+0000 7fbccffce640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc80ce220 con 0x7fbcc80b0210 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.650+0000 7fbccffce640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc80b1070 con 0x7fbcc815ae40 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.650+0000 7fbccffce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80cdff0 con 0x7fbcc80cefe0 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbccd569640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbcc815ae40 0x7fbcc80b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbcce56b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcc80cefe0 0x7fbcc806d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbccdd6a640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fbcc80b0210 0x7fbcc80b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:39518/0 (socket says 172.21.15.139:39518) 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbcce56b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcc80cefe0 0x7fbcc806d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:45142/0 (socket says 172.21.15.139:45142) 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbccdd6a640 1 -- 172.21.15.139:0/3949882995 learned_addr learned my addr 172.21.15.139:0/3949882995 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbccd569640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbcc815ae40 0x7fbcc80b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:45968/0 (socket says 172.21.15.139:45968) 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbcccd68640 1 -- 172.21.15.139:0/3949882995 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046775152 0 0) 0x7fbcc80ce220 con 0x7fbcc80b0210 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbcccd68640 1 -- 172.21.15.139:0/3949882995 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcb4003520 con 0x7fbcc80b0210 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbccd569640 1 -- 172.21.15.139:0/3949882995 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcc80cefe0 msgr2=0x7fbcc806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbccd569640 1 --2- 172.21.15.139:0/3949882995 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcc80cefe0 0x7fbcc806d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbccd569640 1 -- 172.21.15.139:0/3949882995 >> v1:172.21.15.150:6789/0 conn(0x7fbcc80b0210 legacy=0x7fbcc80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbccd569640 1 -- 172.21.15.139:0/3949882995 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc80ce500 con 0x7fbcc815ae40 2024-04-25T09:57:15.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.651+0000 7fbcce56b640 1 --2- 172.21.15.139:0/3949882995 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcc80cefe0 0x7fbcc806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:15.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.652+0000 7fbccd569640 1 --2- 172.21.15.139:0/3949882995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcc815ae40 0x7fbcc80b1960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800a940 tx=0x7fbcb802a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11cb42a5a7dc483e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.652+0000 7fbcccd68640 1 -- 172.21.15.139:0/3949882995 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb800b040 con 0x7fbcc815ae40 2024-04-25T09:57:15.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.652+0000 7fbcccd68640 1 -- 172.21.15.139:0/3949882995 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb802f710 con 0x7fbcc815ae40 2024-04-25T09:57:15.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.652+0000 7fbcccd68640 1 -- 172.21.15.139:0/3949882995 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb802fae0 con 0x7fbcc815ae40 2024-04-25T09:57:15.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.653+0000 7fbccffce640 1 -- 172.21.15.139:0/3949882995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcc815ae40 msgr2=0x7fbcc80b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.653+0000 7fbccffce640 1 --2- 172.21.15.139:0/3949882995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcc815ae40 0x7fbcc80b1960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800a940 tx=0x7fbcb802a430 comp rx=0 tx=0).stop 2024-04-25T09:57:15.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.653+0000 7fbccffce640 1 -- 172.21.15.139:0/3949882995 shutdown_connections 2024-04-25T09:57:15.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.653+0000 7fbccffce640 1 --2- 172.21.15.139:0/3949882995 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcc80cefe0 0x7fbcc806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.653+0000 7fbccffce640 1 --2- 172.21.15.139:0/3949882995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcc815ae40 0x7fbcc80b1960 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.653+0000 7fbccffce640 1 -- 172.21.15.139:0/3949882995 >> 172.21.15.139:0/3949882995 conn(0x7fbcc8066f20 msgr2=0x7fbcc806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.653+0000 7fbccffce640 1 -- 172.21.15.139:0/3949882995 shutdown_connections 2024-04-25T09:57:15.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.653+0000 7fbccffce640 1 -- 172.21.15.139:0/3949882995 wait complete. 2024-04-25T09:57:15.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.654+0000 7fbccffce640 1 Processor -- start 2024-04-25T09:57:15.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.654+0000 7fbccffce640 1 -- start start 2024-04-25T09:57:15.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccffce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc80b0210 0x7fbcc81ef110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccffce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcc80cefe0 0x7fbcc81f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccffce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcc815ae40 0x7fbcc8159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccffce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc815b3e0 con 0x7fbcc80b0210 2024-04-25T09:57:15.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccffce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80b1ea0 con 0x7fbcc80cefe0 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccffce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80ceb50 con 0x7fbcc815ae40 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccdd6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc80b0210 0x7fbcc81ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccdd6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc80b0210 0x7fbcc81ef110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53790/0 (socket says 172.21.15.139:53790) 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccdd6a640 1 -- 172.21.15.139:0/2049232329 learned_addr learned my addr 172.21.15.139:0/2049232329 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbcce56b640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcc815ae40 0x7fbcc8159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccd569640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcc80cefe0 0x7fbcc81f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccdd6a640 1 -- 172.21.15.139:0/2049232329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcc815ae40 msgr2=0x7fbcc8159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccdd6a640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcc815ae40 0x7fbcc8159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccdd6a640 1 -- 172.21.15.139:0/2049232329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcc80cefe0 msgr2=0x7fbcc81f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccdd6a640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcc80cefe0 0x7fbcc81f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccdd6a640 1 -- 172.21.15.139:0/2049232329 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc81f3c20 con 0x7fbcc80b0210 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccdd6a640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc80b0210 0x7fbcc81ef110 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000cb40 tx=0x7fbcc000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbcbe7fc640 1 -- 172.21.15.139:0/2049232329 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0017070 con 0x7fbcc80b0210 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbcbe7fc640 1 -- 172.21.15.139:0/2049232329 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc000ac30 con 0x7fbcc80b0210 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccffce640 1 -- 172.21.15.139:0/2049232329 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc8142140 con 0x7fbcc80b0210 2024-04-25T09:57:15.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbcbe7fc640 1 -- 172.21.15.139:0/2049232329 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0012620 con 0x7fbcc80b0210 2024-04-25T09:57:15.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.655+0000 7fbccffce640 1 -- 172.21.15.139:0/2049232329 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc8142700 con 0x7fbcc80b0210 2024-04-25T09:57:15.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.656+0000 7fbccffce640 1 -- 172.21.15.139:0/2049232329 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc8074520 con 0x7fbcc80b0210 2024-04-25T09:57:15.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.657+0000 7fbcbe7fc640 1 -- 172.21.15.139:0/2049232329 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcc001a070 con 0x7fbcc80b0210 2024-04-25T09:57:15.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.657+0000 7fbcbe7fc640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbcac06b4d0 0x7fbcac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.657+0000 7fbccd569640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbcac06b4d0 0x7fbcac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.657+0000 7fbcbe7fc640 1 -- 172.21.15.139:0/2049232329 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbcc0089d50 con 0x7fbcc80b0210 2024-04-25T09:57:15.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.658+0000 7fbccd569640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbcac06b4d0 0x7fbcac06d990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8034000 tx=0x7fbcb800c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.662+0000 7fbcbe7fc640 1 -- 172.21.15.139:0/2049232329 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbcc008d030 con 0x7fbcc80b0210 2024-04-25T09:57:15.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.873+0000 7fbccffce640 1 -- 172.21.15.139:0/2049232329 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcc813fbf0 con 0x7fbcc80b0210 2024-04-25T09:57:15.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.874+0000 7fbcbe7fc640 1 -- 172.21.15.139:0/2049232329 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fbcc81f3c20 con 0x7fbcc80b0210 2024-04-25T09:57:15.878 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:15.878 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/829268166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":829268166},{"type":"v1","addr":"172.21.15.150:6835","nonce":829268166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3075039902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3075039902},{"type":"v1","addr":"172.21.15.150:6834","nonce":3075039902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/627726069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":627726069},{"type":"v1","addr":"172.21.15.139:6836","nonce":627726069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6789,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2986107995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2986107995},{"type":"v1","addr":"172.21.15.139:6837","nonce":2986107995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[]} 2024-04-25T09:57:15.879 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 32 2024-04-25T09:57:15.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.876+0000 7fbccffce640 1 -- 172.21.15.139:0/2049232329 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbcac06b4d0 msgr2=0x7fbcac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.876+0000 7fbccffce640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbcac06b4d0 0x7fbcac06d990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8034000 tx=0x7fbcb800c040 comp rx=0 tx=0).stop 2024-04-25T09:57:15.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.876+0000 7fbccffce640 1 -- 172.21.15.139:0/2049232329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc80b0210 msgr2=0x7fbcc81ef110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.876+0000 7fbccffce640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc80b0210 0x7fbcc81ef110 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000cb40 tx=0x7fbcc000cf40 comp rx=0 tx=0).stop 2024-04-25T09:57:15.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.876+0000 7fbccffce640 1 -- 172.21.15.139:0/2049232329 shutdown_connections 2024-04-25T09:57:15.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.876+0000 7fbccffce640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbcac06b4d0 0x7fbcac06d990 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.876+0000 7fbccffce640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcc815ae40 0x7fbcc8159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.876+0000 7fbccffce640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcc80cefe0 0x7fbcc81f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.876+0000 7fbccffce640 1 --2- 172.21.15.139:0/2049232329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc80b0210 0x7fbcc81ef110 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.876+0000 7fbccffce640 1 -- 172.21.15.139:0/2049232329 >> 172.21.15.139:0/2049232329 conn(0x7fbcc8066f20 msgr2=0x7fbcc815bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.876+0000 7fbccffce640 1 -- 172.21.15.139:0/2049232329 shutdown_connections 2024-04-25T09:57:15.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:15.876+0000 7fbccffce640 1 -- 172.21.15.139:0/2049232329 wait complete. 2024-04-25T09:57:15.891 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:57:15.891 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:57:15.891 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:57:15.892 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:15.892 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:57:15.892 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:57:15.892 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:57:15.892 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:15.892 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:57:15.892 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:57:15.893 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:57:15.893 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:15.893 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:57:15.893 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:57:15.893 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:57:15.893 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:15.902 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:57:15.902 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:57:15.902 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:57:15.903 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:57:15.905 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:57:15.905 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:57:15.906 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:57:15.906 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:57:15.945 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:57:15.945 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:15.946 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:57:15.946 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:15.948 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:57:15.948 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:15.949 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:57:15.949 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:15.949 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:57:16.018 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:57:16.018 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T09:57:16.019 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-04-25T09:57:16.021 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:57:16.028 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-04-25T09:57:16.028 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-04-25T09:57:16.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.048+0000 7fd25b637640 1 Processor -- start 2024-04-25T09:57:16.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.049+0000 7fd25b637640 1 -- start start 2024-04-25T09:57:16.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.049+0000 7fd25b637640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd25415a870 0x7fd2540b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.049+0000 7fd25b637640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd2540b4190 0x7fd2540d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.049+0000 7fd25b637640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd254070260 0x7fd2540706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd258bd2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd2540b4190 0x7fd2540d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd2593d3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd25415a870 0x7fd2540b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd258bd2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd2540b4190 0x7fd2540d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53812/0 (socket says 172.21.15.139:53812) 2024-04-25T09:57:16.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd2593d3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd25415a870 0x7fd2540b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:45192/0 (socket says 172.21.15.139:45192) 2024-04-25T09:57:16.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd258bd2640 1 -- 172.21.15.139:0/1734946463 learned_addr learned my addr 172.21.15.139:0/1734946463 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:16.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd25b637640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2540d1020 con 0x7fd2540b4190 2024-04-25T09:57:16.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd259bd4640 1 --2- 172.21.15.139:0/1734946463 >> v2:172.21.15.150:3300/0 conn(0x7fd254070260 0x7fd2540706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd25b637640 1 -- 172.21.15.139:0/1734946463 --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2540b5190 con 0x7fd254070260 2024-04-25T09:57:16.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd25b637640 1 -- 172.21.15.139:0/1734946463 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2540b45f0 con 0x7fd25415a870 2024-04-25T09:57:16.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd258bd2640 1 -- 172.21.15.139:0/1734946463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd25415a870 msgr2=0x7fd2540b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:16.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd258bd2640 1 --2- 172.21.15.139:0/1734946463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd25415a870 0x7fd2540b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd258bd2640 1 -- 172.21.15.139:0/1734946463 >> v2:172.21.15.150:3300/0 conn(0x7fd254070260 msgr2=0x7fd2540706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:16.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd258bd2640 1 --2- 172.21.15.139:0/1734946463 >> v2:172.21.15.150:3300/0 conn(0x7fd254070260 0x7fd2540706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.050+0000 7fd258bd2640 1 -- 172.21.15.139:0/1734946463 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2540d0d40 con 0x7fd2540b4190 2024-04-25T09:57:16.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.051+0000 7fd258bd2640 1 --2- 172.21.15.139:0/1734946463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2540b4190 0x7fd2540d05f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd254070e00 tx=0x7fd244029fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df128bb7f8df007e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:16.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.055+0000 7fd24bfff640 1 -- 172.21.15.139:0/1734946463 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24402aae0 con 0x7fd2540b4190 2024-04-25T09:57:16.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.055+0000 7fd24bfff640 1 -- 172.21.15.139:0/1734946463 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24402ac80 con 0x7fd2540b4190 2024-04-25T09:57:16.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.055+0000 7fd24bfff640 1 -- 172.21.15.139:0/1734946463 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24400b670 con 0x7fd2540b4190 2024-04-25T09:57:16.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.056+0000 7fd25b637640 1 -- 172.21.15.139:0/1734946463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2540b4190 msgr2=0x7fd2540d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:16.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.056+0000 7fd25b637640 1 --2- 172.21.15.139:0/1734946463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2540b4190 0x7fd2540d05f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd254070e00 tx=0x7fd244029fa0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.056+0000 7fd25b637640 1 -- 172.21.15.139:0/1734946463 shutdown_connections 2024-04-25T09:57:16.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.056+0000 7fd25b637640 1 --2- 172.21.15.139:0/1734946463 >> v2:172.21.15.150:3300/0 conn(0x7fd254070260 0x7fd2540706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.056+0000 7fd25b637640 1 --2- 172.21.15.139:0/1734946463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2540b4190 0x7fd2540d05f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.056+0000 7fd25b637640 1 --2- 172.21.15.139:0/1734946463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd25415a870 0x7fd2540b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.056+0000 7fd25b637640 1 -- 172.21.15.139:0/1734946463 >> 172.21.15.139:0/1734946463 conn(0x7fd2540b23f0 msgr2=0x7fd2540b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:16.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.056+0000 7fd25b637640 1 -- 172.21.15.139:0/1734946463 shutdown_connections 2024-04-25T09:57:16.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.056+0000 7fd25b637640 1 -- 172.21.15.139:0/1734946463 wait complete. 2024-04-25T09:57:16.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.057+0000 7fd25b637640 1 Processor -- start 2024-04-25T09:57:16.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.057+0000 7fd25b637640 1 -- start start 2024-04-25T09:57:16.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd25b637640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd254070260 0x7fd25414ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd25b637640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2540b4190 0x7fd25413bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd25b637640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd25415a870 0x7fd25413edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd25b637640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2540ce610 con 0x7fd25415a870 2024-04-25T09:57:16.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd25b637640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2540b4e20 con 0x7fd2540b4190 2024-04-25T09:57:16.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd25b637640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2540ce790 con 0x7fd254070260 2024-04-25T09:57:16.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd259bd4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd25415a870 0x7fd25413edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd259bd4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd25415a870 0x7fd25413edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53818/0 (socket says 172.21.15.139:53818) 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd259bd4640 1 -- 172.21.15.139:0/2048529407 learned_addr learned my addr 172.21.15.139:0/2048529407 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd2593d3640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd254070260 0x7fd25414ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd258bd2640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2540b4190 0x7fd25413bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd259bd4640 1 -- 172.21.15.139:0/2048529407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd254070260 msgr2=0x7fd25414ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd259bd4640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd254070260 0x7fd25414ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd259bd4640 1 -- 172.21.15.139:0/2048529407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2540b4190 msgr2=0x7fd25413bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd259bd4640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2540b4190 0x7fd25413bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd259bd4640 1 -- 172.21.15.139:0/2048529407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25413f2f0 con 0x7fd25415a870 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.058+0000 7fd259bd4640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd25415a870 0x7fd25413edb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0042f0 tx=0x7fd24c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.059+0000 7fd249ffb640 1 -- 172.21.15.139:0/2048529407 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c0176a0 con 0x7fd25415a870 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.059+0000 7fd249ffb640 1 -- 172.21.15.139:0/2048529407 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c022070 con 0x7fd25415a870 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.059+0000 7fd25b637640 1 -- 172.21.15.139:0/2048529407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25414f590 con 0x7fd25415a870 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.059+0000 7fd249ffb640 1 -- 172.21.15.139:0/2048529407 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c017a60 con 0x7fd25415a870 2024-04-25T09:57:16.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.059+0000 7fd25b637640 1 -- 172.21.15.139:0/2048529407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25414fb50 con 0x7fd25415a870 2024-04-25T09:57:16.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.060+0000 7fd25b637640 1 -- 172.21.15.139:0/2048529407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd254074520 con 0x7fd25415a870 2024-04-25T09:57:16.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.060+0000 7fd249ffb640 1 -- 172.21.15.139:0/2048529407 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd24c005d10 con 0x7fd25415a870 2024-04-25T09:57:16.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.061+0000 7fd249ffb640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd23806b4d0 0x7fd23806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:16.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.061+0000 7fd249ffb640 1 -- 172.21.15.139:0/2048529407 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd24c092bd0 con 0x7fd25415a870 2024-04-25T09:57:16.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.061+0000 7fd2593d3640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd23806b4d0 0x7fd23806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:16.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.061+0000 7fd2593d3640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd23806b4d0 0x7fd23806d990 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd254157100 tx=0x7fd250009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:16.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.065+0000 7fd249ffb640 1 -- 172.21.15.139:0/2048529407 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd25413f2f0 con 0x7fd25415a870 2024-04-25T09:57:16.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:16.250+0000 7fd25b637640 1 -- 172.21.15.139:0/2048529407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd2540729e0 con 0x7fd25415a870 2024-04-25T09:57:17.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.077+0000 7fd249ffb640 1 -- 172.21.15.139:0/2048529407 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v72) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd24c061600 con 0x7fd25415a870 2024-04-25T09:57:17.082 INFO:teuthology.orchestra.run.smithi139.stderr: removed all blocklist entries 2024-04-25T09:57:17.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.080+0000 7fd25b637640 1 -- 172.21.15.139:0/2048529407 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd23806b4d0 msgr2=0x7fd23806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.080+0000 7fd25b637640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd23806b4d0 0x7fd23806d990 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd254157100 tx=0x7fd250009190 comp rx=0 tx=0).stop 2024-04-25T09:57:17.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.080+0000 7fd25b637640 1 -- 172.21.15.139:0/2048529407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd25415a870 msgr2=0x7fd25413edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.080+0000 7fd25b637640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd25415a870 0x7fd25413edb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0042f0 tx=0x7fd24c0047b0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.080+0000 7fd25b637640 1 -- 172.21.15.139:0/2048529407 shutdown_connections 2024-04-25T09:57:17.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.080+0000 7fd25b637640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd25415a870 0x7fd25413edb0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.080+0000 7fd25b637640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd23806b4d0 0x7fd23806d990 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.080+0000 7fd25b637640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2540b4190 0x7fd25413bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.080+0000 7fd25b637640 1 --2- 172.21.15.139:0/2048529407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd254070260 0x7fd25414ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.080+0000 7fd25b637640 1 -- 172.21.15.139:0/2048529407 >> 172.21.15.139:0/2048529407 conn(0x7fd2540b23f0 msgr2=0x7fd2540b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:17.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.080+0000 7fd25b637640 1 -- 172.21.15.139:0/2048529407 shutdown_connections 2024-04-25T09:57:17.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.080+0000 7fd25b637640 1 -- 172.21.15.139:0/2048529407 wait complete. 2024-04-25T09:57:17.097 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:57:17.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.234+0000 7f30dab89640 1 Processor -- start 2024-04-25T09:57:17.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.234+0000 7f30dab89640 1 -- start start 2024-04-25T09:57:17.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30dab89640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f30d415bdc0 0x7f30d415c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30dab89640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30d40b1740 0x7f30d40b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30dab89640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d415c880 con 0x7f30d4155e30 2024-04-25T09:57:17.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30dab89640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d415d690 con 0x7f30d415bdc0 2024-04-25T09:57:17.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30dab89640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d406d6f0 con 0x7f30d40b1740 2024-04-25T09:57:17.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30d3fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f30d415bdc0 0x7f30d415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30d3fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f30d415bdc0 0x7f30d415c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49770/0 (socket says 172.21.15.139:49770) 2024-04-25T09:57:17.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30d3fff640 1 -- 172.21.15.139:0/2744716433 learned_addr learned my addr 172.21.15.139:0/2744716433 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:17.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30d9126640 1 --2- 172.21.15.139:0/2744716433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30d40b1740 0x7f30d40b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30d3fff640 1 -- 172.21.15.139:0/2744716433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30d40b1740 msgr2=0x7f30d40b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:17.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30d3fff640 1 --2- 172.21.15.139:0/2744716433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30d40b1740 0x7f30d40b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30d3fff640 1 -- 172.21.15.139:0/2744716433 >> v1:172.21.15.150:6789/0 conn(0x7f30d4155e30 legacy=0x7f30d415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:17.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.235+0000 7f30d3fff640 1 -- 172.21.15.139:0/2744716433 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d415cb60 con 0x7f30d415bdc0 2024-04-25T09:57:17.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.236+0000 7f30d3fff640 1 --2- 172.21.15.139:0/2744716433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d415bdc0 0x7f30d415c240 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009870 tx=0x7f30c002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d65a2d6e1d67b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:17.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.236+0000 7f30d37fe640 1 -- 172.21.15.139:0/2744716433 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0037070 con 0x7f30d415bdc0 2024-04-25T09:57:17.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.236+0000 7f30d37fe640 1 -- 172.21.15.139:0/2744716433 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c0032440 con 0x7f30d415bdc0 2024-04-25T09:57:17.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.236+0000 7f30d37fe640 1 -- 172.21.15.139:0/2744716433 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0032810 con 0x7f30d415bdc0 2024-04-25T09:57:17.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.237+0000 7f30dab89640 1 -- 172.21.15.139:0/2744716433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d415bdc0 msgr2=0x7f30d415c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.237+0000 7f30dab89640 1 --2- 172.21.15.139:0/2744716433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d415bdc0 0x7f30d415c240 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009870 tx=0x7f30c002a210 comp rx=0 tx=0).stop 2024-04-25T09:57:17.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.237+0000 7f30dab89640 1 -- 172.21.15.139:0/2744716433 shutdown_connections 2024-04-25T09:57:17.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.237+0000 7f30dab89640 1 --2- 172.21.15.139:0/2744716433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30d40b1740 0x7f30d40b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.237+0000 7f30dab89640 1 --2- 172.21.15.139:0/2744716433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d415bdc0 0x7f30d415c240 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.237+0000 7f30dab89640 1 -- 172.21.15.139:0/2744716433 >> 172.21.15.139:0/2744716433 conn(0x7f30d4066f20 msgr2=0x7f30d406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:17.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.237+0000 7f30dab89640 1 -- 172.21.15.139:0/2744716433 shutdown_connections 2024-04-25T09:57:17.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.237+0000 7f30dab89640 1 -- 172.21.15.139:0/2744716433 wait complete. 2024-04-25T09:57:17.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.238+0000 7f30dab89640 1 Processor -- start 2024-04-25T09:57:17.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.238+0000 7f30dab89640 1 -- start start 2024-04-25T09:57:17.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30dab89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d40b1740 0x7f30d413d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30dab89640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30d4155e30 0x7f30d41e6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30dab89640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30d415bdc0 0x7f30d4158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30dab89640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40af0e0 con 0x7f30d40b1740 2024-04-25T09:57:17.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30dab89640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4156210 con 0x7f30d4155e30 2024-04-25T09:57:17.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30dab89640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40af260 con 0x7f30d415bdc0 2024-04-25T09:57:17.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d8925640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d40b1740 0x7f30d413d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d8925640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d40b1740 0x7f30d413d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49782/0 (socket says 172.21.15.139:49782) 2024-04-25T09:57:17.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d8925640 1 -- 172.21.15.139:0/3438727290 learned_addr learned my addr 172.21.15.139:0/3438727290 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:17.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d9126640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30d415bdc0 0x7f30d4158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d3fff640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30d4155e30 0x7f30d41e6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d8925640 1 -- 172.21.15.139:0/3438727290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30d415bdc0 msgr2=0x7f30d4158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d8925640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30d415bdc0 0x7f30d4158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d8925640 1 -- 172.21.15.139:0/3438727290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30d4155e30 msgr2=0x7f30d41e6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d8925640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30d4155e30 0x7f30d41e6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d8925640 1 -- 172.21.15.139:0/3438727290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d41e75c0 con 0x7f30d40b1740 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d3fff640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30d4155e30 0x7f30d41e6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d8925640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d40b1740 0x7f30d413d9f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f30c800c430 tx=0x7f30c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d17fa640 1 -- 172.21.15.139:0/3438727290 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8011790 con 0x7f30d40b1740 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d17fa640 1 -- 172.21.15.139:0/3438727290 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c8017070 con 0x7f30d40b1740 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30d17fa640 1 -- 172.21.15.139:0/3438727290 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8011bb0 con 0x7f30d40b1740 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.239+0000 7f30dab89640 1 -- 172.21.15.139:0/3438727290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d41e78a0 con 0x7f30d40b1740 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.240+0000 7f30dab89640 1 -- 172.21.15.139:0/3438727290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d41e7b80 con 0x7f30d40b1740 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.240+0000 7f30dab89640 1 -- 172.21.15.139:0/3438727290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d4074520 con 0x7f30d40b1740 2024-04-25T09:57:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.241+0000 7f30d17fa640 1 -- 172.21.15.139:0/3438727290 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30c801c460 con 0x7f30d40b1740 2024-04-25T09:57:17.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.241+0000 7f30d17fa640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f30b806b4d0 0x7f30b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.241+0000 7f30d17fa640 1 -- 172.21.15.139:0/3438727290 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f30c8089f30 con 0x7f30d40b1740 2024-04-25T09:57:17.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.241+0000 7f30d3fff640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f30b806b4d0 0x7f30b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.242+0000 7f30d3fff640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f30b806b4d0 0x7f30b806d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f30c0034000 tx=0x7f30c002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:17.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.245+0000 7f30d17fa640 1 -- 172.21.15.139:0/3438727290 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30c8058f70 con 0x7f30d40b1740 2024-04-25T09:57:17.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.479+0000 7f30dab89640 1 -- 172.21.15.139:0/3438727290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f30d40cd250 con 0x7f30d40b1740 2024-04-25T09:57:17.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.479+0000 7f30d17fa640 1 -- 172.21.15.139:0/3438727290 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f30c8058870 con 0x7f30d40b1740 2024-04-25T09:57:17.484 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:17.484 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:57:17.484 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_dump": [ 2024-04-25T09:57:17.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.484 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.a", 2024-04-25T09:57:17.484 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypm0xdaHhAA6nupxCTQhucepW2iEWNUoQ==", 2024-04-25T09:57:17.484 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.484 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.b", 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmB9GDKRAA/lmU4OiKO5dFgayYx09GQg==", 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.485 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.c", 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmvYgaJRAA/YIE6ralp2WuTK993D7H0Q==", 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.d", 2024-04-25T09:57:17.486 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmg49WMBAAlPY9ZSeOoV296xSdl/m1gg==", 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.0", 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDVJypmG7TpDhAAo9CKuZBw8BsQgB2F+BR5gw==", 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.1", 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDYJypmgR62DBAAotfnvS47WDunP9Ic03gA6Q==", 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.2", 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDbJypmzyqGChAA6wrfIdqpiLm3RO36oorihQ==", 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:17.488 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.3", 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDeJypmt72cCBAAgcLrFC3MUU7wlFySAfLWig==", 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.489 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.4", 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDkJypmxJLpDBAAvROZ9AvYdWMqOALEKIAFCw==", 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.5", 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDnJypmTPXEChAA20BghY39PGw6zFyu9yWbBQ==", 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.490 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.6", 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDqJypmaBoYCRAAy9eHgUI/WTkE2XdT1RKLsw==", 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.7", 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDtJypmViJlCBAANJSDWVHVbWR03Y1SMDH++Q==", 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:17.491 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.0", 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg==", 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow r", 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.1", 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypmOncYARAA6gYjZbRDkhqFX0mpSDbH5Q==", 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.2", 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmQs2wORAArzyV70nGDX4486LlFjM70Q==", 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:57:17.493 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.3", 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypm90x9BhAA41rZPs0O5mwmuHy1bkPCvw==", 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:57:17.494 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.admin", 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmOWQcAxAA/ZVXEHYJQZf9UtgZ5g25Ag==", 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.x", 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmt+RVGxAAi1djVoqnoYCDMzJ9A0dmNA==", 2024-04-25T09:57:17.495 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.y", 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmuJ4bFxAAw6a53AgThU/ut+Mk5j0gqQ==", 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-04-25T09:57:17.496 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:57:17.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.481+0000 7f30dab89640 1 -- 172.21.15.139:0/3438727290 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f30b806b4d0 msgr2=0x7f30b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.481+0000 7f30dab89640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f30b806b4d0 0x7f30b806d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f30c0034000 tx=0x7f30c002e000 comp rx=0 tx=0).stop 2024-04-25T09:57:17.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.481+0000 7f30dab89640 1 -- 172.21.15.139:0/3438727290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d40b1740 msgr2=0x7f30d413d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.481+0000 7f30dab89640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d40b1740 0x7f30d413d9f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f30c800c430 tx=0x7f30c800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.482+0000 7f30dab89640 1 -- 172.21.15.139:0/3438727290 shutdown_connections 2024-04-25T09:57:17.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.482+0000 7f30dab89640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f30b806b4d0 0x7f30b806d990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.482+0000 7f30dab89640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30d415bdc0 0x7f30d4158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.482+0000 7f30dab89640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30d4155e30 0x7f30d41e6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.482+0000 7f30dab89640 1 --2- 172.21.15.139:0/3438727290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30d40b1740 0x7f30d413d9f0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.482+0000 7f30dab89640 1 -- 172.21.15.139:0/3438727290 >> 172.21.15.139:0/3438727290 conn(0x7f30d4066f20 msgr2=0x7f30d406db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:17.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.482+0000 7f30dab89640 1 -- 172.21.15.139:0/3438727290 shutdown_connections 2024-04-25T09:57:17.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.482+0000 7f30dab89640 1 -- 172.21.15.139:0/3438727290 wait complete. 2024-04-25T09:57:17.499 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:17.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.592+0000 7f28a8790640 1 Processor -- start 2024-04-25T09:57:17.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.592+0000 7f28a8790640 1 -- start start 2024-04-25T09:57:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.592+0000 7f28a8790640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28a0158260 0x7f28a015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.592+0000 7f28a8790640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f28a006da20 0x7f28a00b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.592+0000 7f28a8790640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f28a00b3600 0x7f28a006def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a8790640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a00b3a70 con 0x7f28a006da20 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a8790640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a00b3e10 con 0x7f28a00b3600 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a8790640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a01615c0 con 0x7f28a0158260 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a5d2b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f28a006da20 0x7f28a00b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a5d2b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f28a006da20 0x7f28a00b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49792/0 (socket says 172.21.15.139:49792) 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a5d2b640 1 -- 172.21.15.139:0/3964564340 learned_addr learned my addr 172.21.15.139:0/3964564340 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a652c640 1 --2- 172.21.15.139:0/3964564340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28a0158260 0x7f28a015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a6d2d640 1 --2- 172.21.15.139:0/3964564340 >> v2:172.21.15.150:3300/0 conn(0x7f28a00b3600 0x7f28a006def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a5d2b640 1 -- 172.21.15.139:0/3964564340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28a0158260 msgr2=0x7f28a015df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a5d2b640 1 --2- 172.21.15.139:0/3964564340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28a0158260 0x7f28a015df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a5d2b640 1 -- 172.21.15.139:0/3964564340 >> v2:172.21.15.150:3300/0 conn(0x7f28a00b3600 msgr2=0x7f28a006def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a5d2b640 1 --2- 172.21.15.139:0/3964564340 >> v2:172.21.15.150:3300/0 conn(0x7f28a00b3600 0x7f28a006def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a5d2b640 1 -- 172.21.15.139:0/3964564340 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a01618a0 con 0x7f28a006da20 2024-04-25T09:57:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a6d2d640 1 --2- 172.21.15.139:0/3964564340 >> v2:172.21.15.150:3300/0 conn(0x7f28a00b3600 0x7f28a006def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.593+0000 7f28a652c640 1 --2- 172.21.15.139:0/3964564340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28a0158260 0x7f28a015df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.594+0000 7f28a5d2b640 1 --2- 172.21.15.139:0/3964564340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a006da20 0x7f28a00b30c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f28a0066a90 tx=0x7f289002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6570ad2bfb6e8d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.594+0000 7f28a552a640 1 -- 172.21.15.139:0/3964564340 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890037070 con 0x7f28a006da20 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.594+0000 7f28a552a640 1 -- 172.21.15.139:0/3964564340 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2890032440 con 0x7f28a006da20 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.594+0000 7f28a552a640 1 -- 172.21.15.139:0/3964564340 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890032810 con 0x7f28a006da20 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.594+0000 7f28a8790640 1 -- 172.21.15.139:0/3964564340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a006da20 msgr2=0x7f28a00b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.594+0000 7f28a8790640 1 --2- 172.21.15.139:0/3964564340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a006da20 0x7f28a00b30c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f28a0066a90 tx=0x7f289002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.594+0000 7f28a8790640 1 -- 172.21.15.139:0/3964564340 shutdown_connections 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.594+0000 7f28a8790640 1 --2- 172.21.15.139:0/3964564340 >> v2:172.21.15.150:3300/0 conn(0x7f28a00b3600 0x7f28a006def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.594+0000 7f28a8790640 1 --2- 172.21.15.139:0/3964564340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a006da20 0x7f28a00b30c0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.594+0000 7f28a8790640 1 --2- 172.21.15.139:0/3964564340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28a0158260 0x7f28a015df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.594+0000 7f28a8790640 1 -- 172.21.15.139:0/3964564340 >> 172.21.15.139:0/3964564340 conn(0x7f28a00b23d0 msgr2=0x7f28a00cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.595+0000 7f28a8790640 1 -- 172.21.15.139:0/3964564340 shutdown_connections 2024-04-25T09:57:17.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.595+0000 7f28a8790640 1 -- 172.21.15.139:0/3964564340 wait complete. 2024-04-25T09:57:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.596+0000 7f28a8790640 1 Processor -- start 2024-04-25T09:57:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.596+0000 7f28a8790640 1 -- start start 2024-04-25T09:57:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.596+0000 7f28a8790640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a006da20 0x7f28a0155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.596+0000 7f28a8790640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28a00b3600 0x7f28a015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.596+0000 7f28a8790640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28a0158260 0x7f28a01f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.596+0000 7f28a8790640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0162140 con 0x7f28a006da20 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.596+0000 7f28a8790640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0161e60 con 0x7f28a00b3600 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.596+0000 7f28a8790640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0162420 con 0x7f28a0158260 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.596+0000 7f28a652c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a006da20 0x7f28a0155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.596+0000 7f28a652c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a006da20 0x7f28a0155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49804/0 (socket says 172.21.15.139:49804) 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.596+0000 7f28a652c640 1 -- 172.21.15.139:0/331913838 learned_addr learned my addr 172.21.15.139:0/331913838 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f28a5d2b640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28a00b3600 0x7f28a015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f28a6d2d640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28a0158260 0x7f28a01f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f28a652c640 1 -- 172.21.15.139:0/331913838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28a0158260 msgr2=0x7f28a01f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f28a652c640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28a0158260 0x7f28a01f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f28a652c640 1 -- 172.21.15.139:0/331913838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28a00b3600 msgr2=0x7f28a015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f28a652c640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28a00b3600 0x7f28a015b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f28a652c640 1 -- 172.21.15.139:0/331913838 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a0155e60 con 0x7f28a006da20 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f28a6d2d640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28a0158260 0x7f28a01f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f28a652c640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a006da20 0x7f28a0155920 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f289c00c7f0 tx=0x7f289c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f2896ffd640 1 -- 172.21.15.139:0/331913838 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c015070 con 0x7f28a006da20 2024-04-25T09:57:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f2896ffd640 1 -- 172.21.15.139:0/331913838 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289c007820 con 0x7f28a006da20 2024-04-25T09:57:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f28a8790640 1 -- 172.21.15.139:0/331913838 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a015fb90 con 0x7f28a006da20 2024-04-25T09:57:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f2896ffd640 1 -- 172.21.15.139:0/331913838 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c007bd0 con 0x7f28a006da20 2024-04-25T09:57:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.597+0000 7f28a8790640 1 -- 172.21.15.139:0/331913838 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a0160150 con 0x7f28a006da20 2024-04-25T09:57:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.598+0000 7f28a8790640 1 -- 172.21.15.139:0/331913838 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a0074520 con 0x7f28a006da20 2024-04-25T09:57:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.598+0000 7f2896ffd640 1 -- 172.21.15.139:0/331913838 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f289c018070 con 0x7f28a006da20 2024-04-25T09:57:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.599+0000 7f2896ffd640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f288406b4d0 0x7f288406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.599+0000 7f2896ffd640 1 -- 172.21.15.139:0/331913838 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f289c08a070 con 0x7f28a006da20 2024-04-25T09:57:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.599+0000 7f28a5d2b640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f288406b4d0 0x7f288406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.599+0000 7f28a5d2b640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f288406b4d0 0x7f288406d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2890034000 tx=0x7f289002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:17.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.603+0000 7f2896ffd640 1 -- 172.21.15.139:0/331913838 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f28a0155e60 con 0x7f28a006da20 2024-04-25T09:57:17.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.774+0000 7f28a8790640 1 -- 172.21.15.139:0/331913838 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28a006b030 con 0x7f28a006da20 2024-04-25T09:57:17.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.774+0000 7f2896ffd640 1 -- 172.21.15.139:0/331913838 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+13213 (secure 0 0 0) 0x7f289c059140 con 0x7f28a006da20 2024-04-25T09:57:17.779 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:17.779 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":72,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:57:17.076455+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:17.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.777+0000 7f28a8790640 1 -- 172.21.15.139:0/331913838 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f288406b4d0 msgr2=0x7f288406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.777+0000 7f28a8790640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f288406b4d0 0x7f288406d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2890034000 tx=0x7f289002e000 comp rx=0 tx=0).stop 2024-04-25T09:57:17.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.777+0000 7f28a8790640 1 -- 172.21.15.139:0/331913838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a006da20 msgr2=0x7f28a0155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.777+0000 7f28a8790640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a006da20 0x7f28a0155920 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f289c00c7f0 tx=0x7f289c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.777+0000 7f28a8790640 1 -- 172.21.15.139:0/331913838 shutdown_connections 2024-04-25T09:57:17.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.777+0000 7f28a8790640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f288406b4d0 0x7f288406d990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.777+0000 7f28a8790640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28a0158260 0x7f28a01f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.777+0000 7f28a8790640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28a00b3600 0x7f28a015b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.777+0000 7f28a8790640 1 --2- 172.21.15.139:0/331913838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28a006da20 0x7f28a0155920 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.777+0000 7f28a8790640 1 -- 172.21.15.139:0/331913838 >> 172.21.15.139:0/331913838 conn(0x7f28a00b23d0 msgr2=0x7f28a00afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:17.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.777+0000 7f28a8790640 1 -- 172.21.15.139:0/331913838 shutdown_connections 2024-04-25T09:57:17.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.777+0000 7f28a8790640 1 -- 172.21.15.139:0/331913838 wait complete. 2024-04-25T09:57:17.792 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:17.792 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:17.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.888+0000 7fbf31dee640 1 Processor -- start 2024-04-25T09:57:17.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.888+0000 7fbf31dee640 1 -- start start 2024-04-25T09:57:17.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf31dee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf2c0af8a0 0x7fbf2c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf31dee640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbf2c0afc80 0x7fbf2c0cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf31dee640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbf2c06b700 0x7fbf2c06ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf31dee640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2c0c6cc0 con 0x7fbf2c06b700 2024-04-25T09:57:17.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf31dee640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2c0c6fa0 con 0x7fbf2c0afc80 2024-04-25T09:57:17.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf31dee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c0c7280 con 0x7fbf2c0af8a0 2024-04-25T09:57:17.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf2bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbf2c06b700 0x7fbf2c06ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf2bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbf2c06b700 0x7fbf2c06ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49806/0 (socket says 172.21.15.139:49806) 2024-04-25T09:57:17.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf2bfff640 1 -- 172.21.15.139:0/733064459 learned_addr learned my addr 172.21.15.139:0/733064459 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:17.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf2affd640 1 --2- 172.21.15.139:0/733064459 >> v2:172.21.15.150:3300/0 conn(0x7fbf2c0afc80 0x7fbf2c0cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf2b7fe640 1 --2- 172.21.15.139:0/733064459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf2c0af8a0 0x7fbf2c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf2bfff640 1 -- 172.21.15.139:0/733064459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf2c0af8a0 msgr2=0x7fbf2c0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:17.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf2bfff640 1 --2- 172.21.15.139:0/733064459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf2c0af8a0 0x7fbf2c0b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf2bfff640 1 -- 172.21.15.139:0/733064459 >> v2:172.21.15.150:3300/0 conn(0x7fbf2c0afc80 msgr2=0x7fbf2c0cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:17.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf2bfff640 1 --2- 172.21.15.139:0/733064459 >> v2:172.21.15.150:3300/0 conn(0x7fbf2c0afc80 0x7fbf2c0cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.889+0000 7fbf2bfff640 1 -- 172.21.15.139:0/733064459 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c0cd090 con 0x7fbf2c06b700 2024-04-25T09:57:17.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.890+0000 7fbf2bfff640 1 --2- 172.21.15.139:0/733064459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf2c06b700 0x7fbf2c06ab60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0ce150 tx=0x7fbf2402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2999802892f4732d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:17.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.890+0000 7fbf2a7fc640 1 -- 172.21.15.139:0/733064459 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24036070 con 0x7fbf2c06b700 2024-04-25T09:57:17.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.890+0000 7fbf2a7fc640 1 -- 172.21.15.139:0/733064459 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf24031600 con 0x7fbf2c06b700 2024-04-25T09:57:17.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.890+0000 7fbf2a7fc640 1 -- 172.21.15.139:0/733064459 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf240319d0 con 0x7fbf2c06b700 2024-04-25T09:57:17.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.890+0000 7fbf31dee640 1 -- 172.21.15.139:0/733064459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf2c06b700 msgr2=0x7fbf2c06ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.890+0000 7fbf31dee640 1 --2- 172.21.15.139:0/733064459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf2c06b700 0x7fbf2c06ab60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0ce150 tx=0x7fbf2402a6a0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.890+0000 7fbf31dee640 1 -- 172.21.15.139:0/733064459 shutdown_connections 2024-04-25T09:57:17.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.890+0000 7fbf31dee640 1 --2- 172.21.15.139:0/733064459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf2c06b700 0x7fbf2c06ab60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.890+0000 7fbf31dee640 1 --2- 172.21.15.139:0/733064459 >> v2:172.21.15.150:3300/0 conn(0x7fbf2c0afc80 0x7fbf2c0cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.890+0000 7fbf31dee640 1 --2- 172.21.15.139:0/733064459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf2c0af8a0 0x7fbf2c0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.890+0000 7fbf31dee640 1 -- 172.21.15.139:0/733064459 >> 172.21.15.139:0/733064459 conn(0x7fbf2c0b2230 msgr2=0x7fbf2c0b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.891+0000 7fbf31dee640 1 -- 172.21.15.139:0/733064459 shutdown_connections 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.891+0000 7fbf31dee640 1 -- 172.21.15.139:0/733064459 wait complete. 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.892+0000 7fbf31dee640 1 Processor -- start 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.892+0000 7fbf31dee640 1 -- start start 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.892+0000 7fbf31dee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf2c06b700 0x7fbf2c15f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf31dee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c0af8a0 0x7fbf2c09a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf31dee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf2c0afc80 0x7fbf2c15af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf31dee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c06b0a0 con 0x7fbf2c0afc80 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf31dee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c0cddd0 con 0x7fbf2c0af8a0 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf31dee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c06b220 con 0x7fbf2c06b700 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf2bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf2c0afc80 0x7fbf2c15af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf2bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf2c0afc80 0x7fbf2c15af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49814/0 (socket says 172.21.15.139:49814) 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf2bfff640 1 -- 172.21.15.139:0/855434857 learned_addr learned my addr 172.21.15.139:0/855434857 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf2affd640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c0af8a0 0x7fbf2c09a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf2b7fe640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf2c06b700 0x7fbf2c15f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf2bfff640 1 -- 172.21.15.139:0/855434857 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf2c06b700 msgr2=0x7fbf2c15f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf2bfff640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf2c06b700 0x7fbf2c15f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf2bfff640 1 -- 172.21.15.139:0/855434857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c0af8a0 msgr2=0x7fbf2c09a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf2bfff640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c0af8a0 0x7fbf2c09a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf2bfff640 1 -- 172.21.15.139:0/855434857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c15f6e0 con 0x7fbf2c0afc80 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf2affd640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c0af8a0 0x7fbf2c09a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf2bfff640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf2c0afc80 0x7fbf2c15af00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009f40 tx=0x7fbf24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf13fff640 1 -- 172.21.15.139:0/855434857 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24036050 con 0x7fbf2c0afc80 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf13fff640 1 -- 172.21.15.139:0/855434857 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf24042070 con 0x7fbf2c0afc80 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf13fff640 1 -- 172.21.15.139:0/855434857 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24009510 con 0x7fbf2c0afc80 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf31dee640 1 -- 172.21.15.139:0/855434857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2c15f9c0 con 0x7fbf2c0afc80 2024-04-25T09:57:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.893+0000 7fbf31dee640 1 -- 172.21.15.139:0/855434857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2c15ff80 con 0x7fbf2c0afc80 2024-04-25T09:57:17.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.894+0000 7fbf31dee640 1 -- 172.21.15.139:0/855434857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2c06eb40 con 0x7fbf2c0afc80 2024-04-25T09:57:17.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.895+0000 7fbf13fff640 1 -- 172.21.15.139:0/855434857 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf24007bf0 con 0x7fbf2c0afc80 2024-04-25T09:57:17.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.895+0000 7fbf13fff640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf1806b500 0x7fbf1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:17.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.895+0000 7fbf2b7fe640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf1806b500 0x7fbf1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:17.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.895+0000 7fbf13fff640 1 -- 172.21.15.139:0/855434857 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbf240aacf0 con 0x7fbf2c0afc80 2024-04-25T09:57:17.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.896+0000 7fbf2b7fe640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf1806b500 0x7fbf1806d9c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009990 tx=0x7fbf1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:17.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:17.900+0000 7fbf13fff640 1 -- 172.21.15.139:0/855434857 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf2c06eb40 con 0x7fbf2c0afc80 2024-04-25T09:57:18.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.072+0000 7fbf31dee640 1 -- 172.21.15.139:0/855434857 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf2c15ccb0 con 0x7fbf2c0afc80 2024-04-25T09:57:18.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.072+0000 7fbf13fff640 1 -- 172.21.15.139:0/855434857 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf240af070 con 0x7fbf2c0afc80 2024-04-25T09:57:18.077 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:57:18.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.074+0000 7fbf31dee640 1 -- 172.21.15.139:0/855434857 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf1806b500 msgr2=0x7fbf1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.074+0000 7fbf31dee640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf1806b500 0x7fbf1806d9c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009990 tx=0x7fbf1c008040 comp rx=0 tx=0).stop 2024-04-25T09:57:18.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.074+0000 7fbf31dee640 1 -- 172.21.15.139:0/855434857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf2c0afc80 msgr2=0x7fbf2c15af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.074+0000 7fbf31dee640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf2c0afc80 0x7fbf2c15af00 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009f40 tx=0x7fbf24004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:18.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.074+0000 7fbf31dee640 1 -- 172.21.15.139:0/855434857 shutdown_connections 2024-04-25T09:57:18.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.074+0000 7fbf31dee640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf2c0afc80 0x7fbf2c15af00 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.074+0000 7fbf31dee640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf1806b500 0x7fbf1806d9c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.074+0000 7fbf31dee640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c0af8a0 0x7fbf2c09a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.074+0000 7fbf31dee640 1 --2- 172.21.15.139:0/855434857 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf2c06b700 0x7fbf2c15f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.074+0000 7fbf31dee640 1 -- 172.21.15.139:0/855434857 >> 172.21.15.139:0/855434857 conn(0x7fbf2c0b2230 msgr2=0x7fbf2c0c6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:18.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.075+0000 7fbf31dee640 1 -- 172.21.15.139:0/855434857 shutdown_connections 2024-04-25T09:57:18.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.075+0000 7fbf31dee640 1 -- 172.21.15.139:0/855434857 wait complete. 2024-04-25T09:57:18.090 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:57:18.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.183+0000 7fc47f446640 1 Processor -- start 2024-04-25T09:57:18.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.183+0000 7fc47f446640 1 -- start start 2024-04-25T09:57:18.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.184+0000 7fc47f446640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc4781611e0 0x7fc4780b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.184+0000 7fc47f446640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4780cea00 0x7fc47806c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.184+0000 7fc47f446640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4780d4740 con 0x7fc4780cee60 2024-04-25T09:57:18.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.184+0000 7fc47f446640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4780d1fa0 con 0x7fc4780cea00 2024-04-25T09:57:18.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.184+0000 7fc47f446640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4780d2560 con 0x7fc4781611e0 2024-04-25T09:57:18.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.184+0000 7fc47c9e1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4780cea00 0x7fc47806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.184+0000 7fc47d9e3640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc4780cee60 0x7fc4780d29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57006/0 (socket says 172.21.15.139:57006) 2024-04-25T09:57:18.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.184+0000 7fc47d9e3640 1 -- 172.21.15.139:0/63708748 learned_addr learned my addr 172.21.15.139:0/63708748 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:18.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.184+0000 7fc47d1e2640 1 --2- 172.21.15.139:0/63708748 >> v2:172.21.15.150:3300/0 conn(0x7fc4781611e0 0x7fc4780b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.185+0000 7fc47c9e1640 1 -- 172.21.15.139:0/63708748 >> v2:172.21.15.150:3300/0 conn(0x7fc4781611e0 msgr2=0x7fc4780b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:18.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.185+0000 7fc47c9e1640 1 --2- 172.21.15.139:0/63708748 >> v2:172.21.15.150:3300/0 conn(0x7fc4781611e0 0x7fc4780b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.185+0000 7fc47c9e1640 1 -- 172.21.15.139:0/63708748 >> v1:172.21.15.139:6789/0 conn(0x7fc4780cee60 legacy=0x7fc4780d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.185+0000 7fc46ffff640 1 -- 172.21.15.139:0/63708748 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496577061 0 0) 0x7fc4780d4740 con 0x7fc4780cee60 2024-04-25T09:57:18.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.185+0000 7fc47c9e1640 1 -- 172.21.15.139:0/63708748 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4780b2770 con 0x7fc4780cea00 2024-04-25T09:57:18.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.185+0000 7fc47c9e1640 1 --2- 172.21.15.139:0/63708748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4780cea00 0x7fc47806c070 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc47808c420 tx=0x7fc46802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c517a16b1071c496 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.186+0000 7fc46ffff640 1 -- 172.21.15.139:0/63708748 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468037070 con 0x7fc4780cea00 2024-04-25T09:57:18.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.186+0000 7fc46ffff640 1 -- 172.21.15.139:0/63708748 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc468032440 con 0x7fc4780cea00 2024-04-25T09:57:18.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.186+0000 7fc46ffff640 1 -- 172.21.15.139:0/63708748 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468032810 con 0x7fc4780cea00 2024-04-25T09:57:18.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.186+0000 7fc47f446640 1 -- 172.21.15.139:0/63708748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4780cea00 msgr2=0x7fc47806c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.186+0000 7fc47f446640 1 --2- 172.21.15.139:0/63708748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4780cea00 0x7fc47806c070 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc47808c420 tx=0x7fc46802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.186+0000 7fc47f446640 1 -- 172.21.15.139:0/63708748 shutdown_connections 2024-04-25T09:57:18.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.186+0000 7fc47f446640 1 --2- 172.21.15.139:0/63708748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4780cea00 0x7fc47806c070 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.186+0000 7fc47f446640 1 --2- 172.21.15.139:0/63708748 >> v2:172.21.15.150:3300/0 conn(0x7fc4781611e0 0x7fc4780b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.186+0000 7fc47f446640 1 -- 172.21.15.139:0/63708748 >> 172.21.15.139:0/63708748 conn(0x7fc4780b5b00 msgr2=0x7fc4780d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:18.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.186+0000 7fc47f446640 1 -- 172.21.15.139:0/63708748 shutdown_connections 2024-04-25T09:57:18.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.186+0000 7fc47f446640 1 -- 172.21.15.139:0/63708748 wait complete. 2024-04-25T09:57:18.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.187+0000 7fc47f446640 1 Processor -- start 2024-04-25T09:57:18.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.187+0000 7fc47f446640 1 -- start start 2024-04-25T09:57:18.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47f446640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4780cea00 0x7fc4780cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47f446640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4780cee60 0x7fc4780cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47f446640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4781611e0 0x7fc4780c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47f446640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4780d21f0 con 0x7fc4781611e0 2024-04-25T09:57:18.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47f446640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4780b2d30 con 0x7fc4780cea00 2024-04-25T09:57:18.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47f446640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4781615c0 con 0x7fc4780cee60 2024-04-25T09:57:18.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47d9e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4781611e0 0x7fc4780c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47d9e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4781611e0 0x7fc4780c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49838/0 (socket says 172.21.15.139:49838) 2024-04-25T09:57:18.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47d9e3640 1 -- 172.21.15.139:0/2799409993 learned_addr learned my addr 172.21.15.139:0/2799409993 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:18.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47d9e3640 1 -- 172.21.15.139:0/2799409993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4780cee60 msgr2=0x7fc4780cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47d9e3640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4780cee60 0x7fc4780cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47d1e2640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4780cea00 0x7fc4780cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47d9e3640 1 -- 172.21.15.139:0/2799409993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4780cea00 msgr2=0x7fc4780cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47d9e3640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4780cea00 0x7fc4780cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.188+0000 7fc47d9e3640 1 -- 172.21.15.139:0/2799409993 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4780cb3e0 con 0x7fc4781611e0 2024-04-25T09:57:18.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.189+0000 7fc47d9e3640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4781611e0 0x7fc4780c8df0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc47000c7a0 tx=0x7fc47000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.189+0000 7fc46dffb640 1 -- 172.21.15.139:0/2799409993 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470018070 con 0x7fc4781611e0 2024-04-25T09:57:18.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.189+0000 7fc46dffb640 1 -- 172.21.15.139:0/2799409993 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47000a640 con 0x7fc4781611e0 2024-04-25T09:57:18.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.189+0000 7fc46dffb640 1 -- 172.21.15.139:0/2799409993 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47000aab0 con 0x7fc4781611e0 2024-04-25T09:57:18.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.189+0000 7fc47f446640 1 -- 172.21.15.139:0/2799409993 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4780cb6c0 con 0x7fc4781611e0 2024-04-25T09:57:18.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.189+0000 7fc47f446640 1 -- 172.21.15.139:0/2799409993 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4780cbc80 con 0x7fc4781611e0 2024-04-25T09:57:18.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.190+0000 7fc47f446640 1 -- 172.21.15.139:0/2799409993 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc478074520 con 0x7fc4781611e0 2024-04-25T09:57:18.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.190+0000 7fc46dffb640 1 -- 172.21.15.139:0/2799409993 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc47001b070 con 0x7fc4781611e0 2024-04-25T09:57:18.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.191+0000 7fc46dffb640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc45c06b4d0 0x7fc45c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.191+0000 7fc46dffb640 1 -- 172.21.15.139:0/2799409993 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc470089690 con 0x7fc4781611e0 2024-04-25T09:57:18.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.191+0000 7fc47d1e2640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc45c06b4d0 0x7fc45c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.191+0000 7fc47d1e2640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc45c06b4d0 0x7fc45c06d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc474009a10 tx=0x7fc474008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.195+0000 7fc46dffb640 1 -- 172.21.15.139:0/2799409993 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4700586d0 con 0x7fc4781611e0 2024-04-25T09:57:18.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.367+0000 7fc47f446640 1 -- 172.21.15.139:0/2799409993 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc47806ceb0 con 0x7fc4781611e0 2024-04-25T09:57:18.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.367+0000 7fc46dffb640 1 -- 172.21.15.139:0/2799409993 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc470057fd0 con 0x7fc4781611e0 2024-04-25T09:57:18.372 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:18.372 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-04-25T09:57:18.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:57:18.372 INFO:teuthology.orchestra.run.smithi139.stdout: "poolnum": 1, 2024-04-25T09:57:18.372 INFO:teuthology.orchestra.run.smithi139.stdout: "poolname": ".mgr" 2024-04-25T09:57:18.372 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:18.372 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-04-25T09:57:18.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.369+0000 7fc47f446640 1 -- 172.21.15.139:0/2799409993 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc45c06b4d0 msgr2=0x7fc45c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.369+0000 7fc47f446640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc45c06b4d0 0x7fc45c06d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc474009a10 tx=0x7fc474008040 comp rx=0 tx=0).stop 2024-04-25T09:57:18.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.369+0000 7fc47f446640 1 -- 172.21.15.139:0/2799409993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4781611e0 msgr2=0x7fc4780c8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.369+0000 7fc47f446640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4781611e0 0x7fc4780c8df0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc47000c7a0 tx=0x7fc47000cc60 comp rx=0 tx=0).stop 2024-04-25T09:57:18.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.369+0000 7fc47f446640 1 -- 172.21.15.139:0/2799409993 shutdown_connections 2024-04-25T09:57:18.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.369+0000 7fc47f446640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4781611e0 0x7fc4780c8df0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.369+0000 7fc47f446640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc45c06b4d0 0x7fc45c06d990 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.369+0000 7fc47f446640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4780cee60 0x7fc4780cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.369+0000 7fc47f446640 1 --2- 172.21.15.139:0/2799409993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4780cea00 0x7fc4780cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.369+0000 7fc47f446640 1 -- 172.21.15.139:0/2799409993 >> 172.21.15.139:0/2799409993 conn(0x7fc4780b5b00 msgr2=0x7fc4780d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:18.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.370+0000 7fc47f446640 1 -- 172.21.15.139:0/2799409993 shutdown_connections 2024-04-25T09:57:18.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.370+0000 7fc47f446640 1 -- 172.21.15.139:0/2799409993 wait complete. 2024-04-25T09:57:18.385 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:57:18.385 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:57:18.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.482+0000 7f36afb70640 1 Processor -- start 2024-04-25T09:57:18.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.482+0000 7f36afb70640 1 -- start start 2024-04-25T09:57:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36afb70640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f36a8145230 0x7f36a815be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36afb70640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36a815c3d0 0x7f36a80711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36afb70640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f36a8071700 0x7f36a806d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36afb70640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a8071b50 con 0x7f36a8145230 2024-04-25T09:57:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36afb70640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a815c960 con 0x7f36a8071700 2024-04-25T09:57:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36afb70640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a815cc40 con 0x7f36a815c3d0 2024-04-25T09:57:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36ad90c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f36a8145230 0x7f36a815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36ad90c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f36a8145230 0x7f36a815be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49842/0 (socket says 172.21.15.139:49842) 2024-04-25T09:57:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36ad90c640 1 -- 172.21.15.139:0/641785228 learned_addr learned my addr 172.21.15.139:0/641785228 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36ad90c640 1 -- 172.21.15.139:0/641785228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36a815c3d0 msgr2=0x7f36a80711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36ad90c640 1 --2- 172.21.15.139:0/641785228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36a815c3d0 0x7f36a80711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36ad90c640 1 -- 172.21.15.139:0/641785228 >> v2:172.21.15.150:3300/0 conn(0x7f36a8071700 msgr2=0x7f36a806d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36ad90c640 1 --2- 172.21.15.139:0/641785228 >> v2:172.21.15.150:3300/0 conn(0x7f36a8071700 0x7f36a806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.483+0000 7f36ad90c640 1 -- 172.21.15.139:0/641785228 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a815d230 con 0x7f36a8145230 2024-04-25T09:57:18.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.484+0000 7f36ad90c640 1 --2- 172.21.15.139:0/641785228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a8145230 0x7f36a815be90 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f36a806f320 tx=0x7f36a402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9355c7277f54ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.484+0000 7f36ac90a640 1 -- 172.21.15.139:0/641785228 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4037070 con 0x7f36a8145230 2024-04-25T09:57:18.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.484+0000 7f36ac90a640 1 -- 172.21.15.139:0/641785228 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a4032440 con 0x7f36a8145230 2024-04-25T09:57:18.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.484+0000 7f36ac90a640 1 -- 172.21.15.139:0/641785228 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4032810 con 0x7f36a8145230 2024-04-25T09:57:18.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.484+0000 7f36afb70640 1 -- 172.21.15.139:0/641785228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a8145230 msgr2=0x7f36a815be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.484+0000 7f36afb70640 1 --2- 172.21.15.139:0/641785228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a8145230 0x7f36a815be90 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f36a806f320 tx=0x7f36a402a140 comp rx=0 tx=0).stop 2024-04-25T09:57:18.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.485+0000 7f36afb70640 1 -- 172.21.15.139:0/641785228 shutdown_connections 2024-04-25T09:57:18.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.485+0000 7f36afb70640 1 --2- 172.21.15.139:0/641785228 >> v2:172.21.15.150:3300/0 conn(0x7f36a8071700 0x7f36a806d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.485+0000 7f36afb70640 1 --2- 172.21.15.139:0/641785228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36a815c3d0 0x7f36a80711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.485+0000 7f36afb70640 1 --2- 172.21.15.139:0/641785228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a8145230 0x7f36a815be90 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.485+0000 7f36afb70640 1 -- 172.21.15.139:0/641785228 >> 172.21.15.139:0/641785228 conn(0x7f36a8067000 msgr2=0x7f36a806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:18.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.485+0000 7f36afb70640 1 -- 172.21.15.139:0/641785228 shutdown_connections 2024-04-25T09:57:18.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.485+0000 7f36afb70640 1 -- 172.21.15.139:0/641785228 wait complete. 2024-04-25T09:57:18.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.486+0000 7f36afb70640 1 Processor -- start 2024-04-25T09:57:18.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.486+0000 7f36afb70640 1 -- start start 2024-04-25T09:57:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.486+0000 7f36afb70640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36a8071700 0x7f36a813efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.486+0000 7f36afb70640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36a8145230 0x7f36a80adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.486+0000 7f36afb70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a815c3d0 0x7f36a80adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.486+0000 7f36afb70640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a806d760 con 0x7f36a815c3d0 2024-04-25T09:57:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.486+0000 7f36afb70640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a806e4b0 con 0x7f36a8071700 2024-04-25T09:57:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.486+0000 7f36afb70640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a806d8e0 con 0x7f36a8145230 2024-04-25T09:57:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36ae10d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a815c3d0 0x7f36a80adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36ae10d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a815c3d0 0x7f36a80adff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49844/0 (socket says 172.21.15.139:49844) 2024-04-25T09:57:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36ae10d640 1 -- 172.21.15.139:0/1489742924 learned_addr learned my addr 172.21.15.139:0/1489742924 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:18.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36ad90c640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36a8071700 0x7f36a813efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36ad10b640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36a8145230 0x7f36a80adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36ae10d640 1 -- 172.21.15.139:0/1489742924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36a8145230 msgr2=0x7f36a80adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36ae10d640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36a8145230 0x7f36a80adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36ae10d640 1 -- 172.21.15.139:0/1489742924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36a8071700 msgr2=0x7f36a813efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36ae10d640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36a8071700 0x7f36a813efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36ae10d640 1 -- 172.21.15.139:0/1489742924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a81f4710 con 0x7f36a815c3d0 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36ad90c640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36a8071700 0x7f36a813efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36ae10d640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a815c3d0 0x7f36a80adff0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f36a000b9a0 tx=0x7f36a000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f369e7fc640 1 -- 172.21.15.139:0/1489742924 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0019070 con 0x7f36a815c3d0 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f369e7fc640 1 -- 172.21.15.139:0/1489742924 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a000cbc0 con 0x7f36a815c3d0 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f369e7fc640 1 -- 172.21.15.139:0/1489742924 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0014440 con 0x7f36a815c3d0 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36afb70640 1 -- 172.21.15.139:0/1489742924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a81f3950 con 0x7f36a815c3d0 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.487+0000 7f36afb70640 1 -- 172.21.15.139:0/1489742924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a81f3f10 con 0x7f36a815c3d0 2024-04-25T09:57:18.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.488+0000 7f36afb70640 1 -- 172.21.15.139:0/1489742924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a806baf0 con 0x7f36a815c3d0 2024-04-25T09:57:18.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.489+0000 7f369e7fc640 1 -- 172.21.15.139:0/1489742924 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36a001c070 con 0x7f36a815c3d0 2024-04-25T09:57:18.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.489+0000 7f369e7fc640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f368c06b4d0 0x7f368c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.489+0000 7f36ad90c640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f368c06b4d0 0x7f368c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.489+0000 7f369e7fc640 1 -- 172.21.15.139:0/1489742924 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f36a0089d90 con 0x7f36a815c3d0 2024-04-25T09:57:18.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.490+0000 7f36ad90c640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f368c06b4d0 0x7f368c06d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f36a4034000 tx=0x7f36a402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.494+0000 7f369e7fc640 1 -- 172.21.15.139:0/1489742924 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36a0058dd0 con 0x7f36a815c3d0 2024-04-25T09:57:18.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:18.661+0000 7f36afb70640 1 -- 172.21.15.139:0/1489742924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f36a80c56e0 con 0x7f36a815c3d0 2024-04-25T09:57:19.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.084+0000 7f369e7fc640 1 -- 172.21.15.139:0/1489742924 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) v1 ==== 138+0+0 (secure 0 0 0) 0x7f36a00586d0 con 0x7f36a815c3d0 2024-04-25T09:57:19.088 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' created 2024-04-25T09:57:19.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.086+0000 7f36afb70640 1 -- 172.21.15.139:0/1489742924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f368c06b4d0 msgr2=0x7f368c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.086+0000 7f36afb70640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f368c06b4d0 0x7f368c06d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f36a4034000 tx=0x7f36a402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:19.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.086+0000 7f36afb70640 1 -- 172.21.15.139:0/1489742924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a815c3d0 msgr2=0x7f36a80adff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.086+0000 7f36afb70640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a815c3d0 0x7f36a80adff0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f36a000b9a0 tx=0x7f36a000be60 comp rx=0 tx=0).stop 2024-04-25T09:57:19.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.086+0000 7f36afb70640 1 -- 172.21.15.139:0/1489742924 shutdown_connections 2024-04-25T09:57:19.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.086+0000 7f36afb70640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a815c3d0 0x7f36a80adff0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.086+0000 7f36afb70640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f368c06b4d0 0x7f368c06d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.086+0000 7f36afb70640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36a8145230 0x7f36a80adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.086+0000 7f36afb70640 1 --2- 172.21.15.139:0/1489742924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36a8071700 0x7f36a813efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.086+0000 7f36afb70640 1 -- 172.21.15.139:0/1489742924 >> 172.21.15.139:0/1489742924 conn(0x7f36a8067000 msgr2=0x7f36a815e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.086+0000 7f36afb70640 1 -- 172.21.15.139:0/1489742924 shutdown_connections 2024-04-25T09:57:19.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.086+0000 7f36afb70640 1 -- 172.21.15.139:0/1489742924 wait complete. 2024-04-25T09:57:19.105 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:57:19.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.206+0000 7ff0ab18b640 1 Processor -- start 2024-04-25T09:57:19.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.206+0000 7ff0ab18b640 1 -- start start 2024-04-25T09:57:19.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.206+0000 7ff0ab18b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff0a4161fd0 0x7ff0a415d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.206+0000 7ff0ab18b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a4099ca0 con 0x7ff0a40cd610 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.206+0000 7ff0ab18b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a4099f80 con 0x7ff0a40d0a80 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.206+0000 7ff0ab18b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a4099a50 con 0x7ff0a4161fd0 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.207+0000 7ff0a9728640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff0a4161fd0 0x7ff0a415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.207+0000 7ff0a8f27640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff0a40cd610 0x7ff0a40d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57008/0 (socket says 172.21.15.139:57008) 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.207+0000 7ff0a8f27640 1 -- 172.21.15.139:0/2652088307 learned_addr learned my addr 172.21.15.139:0/2652088307 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.207+0000 7ff0a9728640 1 -- 172.21.15.139:0/2652088307 >> v1:172.21.15.150:6789/0 conn(0x7ff0a40d0a80 legacy=0x7ff0a41618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.207+0000 7ff0a37fe640 1 -- 172.21.15.139:0/2652088307 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662134191 0 0) 0x7ff0a4099ca0 con 0x7ff0a40cd610 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.207+0000 7ff0a9728640 1 -- 172.21.15.139:0/2652088307 >> v1:172.21.15.139:6789/0 conn(0x7ff0a40cd610 legacy=0x7ff0a40d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.207+0000 7ff0a9728640 1 -- 172.21.15.139:0/2652088307 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a409a260 con 0x7ff0a4161fd0 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.208+0000 7ff0a9728640 1 --2- 172.21.15.139:0/2652088307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a4161fd0 0x7ff0a415d370 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009d90 tx=0x7ff09c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b361177a892e942 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.208+0000 7ff0a37fe640 1 -- 172.21.15.139:0/2652088307 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c007740 con 0x7ff0a4161fd0 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.208+0000 7ff0a37fe640 1 -- 172.21.15.139:0/2652088307 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c0078e0 con 0x7ff0a4161fd0 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.208+0000 7ff0a37fe640 1 -- 172.21.15.139:0/2652088307 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c007cb0 con 0x7ff0a4161fd0 2024-04-25T09:57:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.208+0000 7ff0ab18b640 1 -- 172.21.15.139:0/2652088307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a4161fd0 msgr2=0x7ff0a415d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.208+0000 7ff0ab18b640 1 --2- 172.21.15.139:0/2652088307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a4161fd0 0x7ff0a415d370 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009d90 tx=0x7ff09c02a730 comp rx=0 tx=0).stop 2024-04-25T09:57:19.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.208+0000 7ff0ab18b640 1 -- 172.21.15.139:0/2652088307 shutdown_connections 2024-04-25T09:57:19.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.208+0000 7ff0ab18b640 1 --2- 172.21.15.139:0/2652088307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a4161fd0 0x7ff0a415d370 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.208+0000 7ff0ab18b640 1 -- 172.21.15.139:0/2652088307 >> 172.21.15.139:0/2652088307 conn(0x7ff0a40b31f0 msgr2=0x7ff0a40b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.208+0000 7ff0ab18b640 1 -- 172.21.15.139:0/2652088307 shutdown_connections 2024-04-25T09:57:19.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.208+0000 7ff0ab18b640 1 -- 172.21.15.139:0/2652088307 wait complete. 2024-04-25T09:57:19.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.209+0000 7ff0ab18b640 1 Processor -- start 2024-04-25T09:57:19.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.209+0000 7ff0ab18b640 1 -- start start 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.210+0000 7ff0ab18b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0a40cd610 0x7ff0a414db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.210+0000 7ff0ab18b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a40d0a80 0x7ff0a40b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.210+0000 7ff0ab18b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a4161fd0 0x7ff0a41e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.210+0000 7ff0ab18b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a40cd9f0 con 0x7ff0a40d0a80 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.210+0000 7ff0ab18b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a409a820 con 0x7ff0a4161fd0 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.210+0000 7ff0ab18b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a40cdb70 con 0x7ff0a40cd610 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.210+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a40d0a80 0x7ff0a40b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.210+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a40d0a80 0x7ff0a40b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49852/0 (socket says 172.21.15.139:49852) 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.210+0000 7ff0a3fff640 1 -- 172.21.15.139:0/3459016428 learned_addr learned my addr 172.21.15.139:0/3459016428 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0a9728640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a4161fd0 0x7ff0a41e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0a8f27640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0a40cd610 0x7ff0a414db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0a3fff640 1 -- 172.21.15.139:0/3459016428 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0a40cd610 msgr2=0x7ff0a414db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0a3fff640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0a40cd610 0x7ff0a414db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0a3fff640 1 -- 172.21.15.139:0/3459016428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a4161fd0 msgr2=0x7ff0a41e88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0a3fff640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a4161fd0 0x7ff0a41e88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0a3fff640 1 -- 172.21.15.139:0/3459016428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a414e080 con 0x7ff0a40d0a80 2024-04-25T09:57:19.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0a8f27640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0a40cd610 0x7ff0a414db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:19.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0a3fff640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a40d0a80 0x7ff0a40b4190 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7ff09000c430 tx=0x7ff09000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0a17fa640 1 -- 172.21.15.139:0/3459016428 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090011790 con 0x7ff0a40d0a80 2024-04-25T09:57:19.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0a17fa640 1 -- 172.21.15.139:0/3459016428 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff090017070 con 0x7ff0a40d0a80 2024-04-25T09:57:19.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0a17fa640 1 -- 172.21.15.139:0/3459016428 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090011bb0 con 0x7ff0a40d0a80 2024-04-25T09:57:19.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0ab18b640 1 -- 172.21.15.139:0/3459016428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a40b4bc0 con 0x7ff0a40d0a80 2024-04-25T09:57:19.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.211+0000 7ff0ab18b640 1 -- 172.21.15.139:0/3459016428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a40b5180 con 0x7ff0a40d0a80 2024-04-25T09:57:19.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.212+0000 7ff0ab18b640 1 -- 172.21.15.139:0/3459016428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a4074520 con 0x7ff0a40d0a80 2024-04-25T09:57:19.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.213+0000 7ff0a17fa640 1 -- 172.21.15.139:0/3459016428 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff09001c460 con 0x7ff0a40d0a80 2024-04-25T09:57:19.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.213+0000 7ff0a17fa640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff08806b4d0 0x7ff08806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.213+0000 7ff0a17fa640 1 -- 172.21.15.139:0/3459016428 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5619+0+0 (secure 0 0 0) 0x7ff09008a090 con 0x7ff0a40d0a80 2024-04-25T09:57:19.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.213+0000 7ff0a8f27640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff08806b4d0 0x7ff08806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.213+0000 7ff0a8f27640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff08806b4d0 0x7ff08806d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff098000f60 tx=0x7ff098009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.217+0000 7ff0a17fa640 1 -- 172.21.15.139:0/3459016428 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff090058f60 con 0x7ff0a40d0a80 2024-04-25T09:57:19.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:19.389+0000 7ff0ab18b640 1 -- 172.21.15.139:0/3459016428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff0a406e360 con 0x7ff0a40d0a80 2024-04-25T09:57:20.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.089+0000 7ff0a17fa640 1 -- 172.21.15.139:0/3459016428 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v74) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff090058860 con 0x7ff0a40d0a80 2024-04-25T09:57:20.093 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' created 2024-04-25T09:57:20.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.096+0000 7ff0ab18b640 1 -- 172.21.15.139:0/3459016428 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff08806b4d0 msgr2=0x7ff08806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.096+0000 7ff0ab18b640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff08806b4d0 0x7ff08806d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff098000f60 tx=0x7ff098009040 comp rx=0 tx=0).stop 2024-04-25T09:57:20.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.096+0000 7ff0ab18b640 1 -- 172.21.15.139:0/3459016428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a40d0a80 msgr2=0x7ff0a40b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.096+0000 7ff0ab18b640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a40d0a80 0x7ff0a40b4190 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7ff09000c430 tx=0x7ff09000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.097+0000 7ff0ab18b640 1 -- 172.21.15.139:0/3459016428 shutdown_connections 2024-04-25T09:57:20.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.097+0000 7ff0ab18b640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0a40d0a80 0x7ff0a40b4190 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.097+0000 7ff0ab18b640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff08806b4d0 0x7ff08806d990 secure :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff098000f60 tx=0x7ff098009040 comp rx=0 tx=0).stop 2024-04-25T09:57:20.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.097+0000 7ff0ab18b640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0a4161fd0 0x7ff0a41e88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.097+0000 7ff0ab18b640 1 --2- 172.21.15.139:0/3459016428 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0a40cd610 0x7ff0a414db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.098+0000 7ff0ab18b640 1 -- 172.21.15.139:0/3459016428 >> 172.21.15.139:0/3459016428 conn(0x7ff0a40b31f0 msgr2=0x7ff0a40b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:20.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.106+0000 7ff0ab18b640 1 -- 172.21.15.139:0/3459016428 shutdown_connections 2024-04-25T09:57:20.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.108+0000 7ff0ab18b640 1 -- 172.21.15.139:0/3459016428 wait complete. 2024-04-25T09:57:20.126 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:57:20.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.225+0000 7fd5b0e15640 1 Processor -- start 2024-04-25T09:57:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.225+0000 7fd5b0e15640 1 -- start start 2024-04-25T09:57:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.225+0000 7fd5b0e15640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd5ac0cdd20 0x7fd5ac071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.225+0000 7fd5b0e15640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ac15a480 con 0x7fd5ac150ed0 2024-04-25T09:57:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.225+0000 7fd5b0e15640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ac15a760 con 0x7fd5ac071f90 2024-04-25T09:57:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.225+0000 7fd5b0e15640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ac15aa40 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.225+0000 7fd5a9d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd5ac0cdd20 0x7fd5ac071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.226+0000 7fd5aa59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd5ac150ed0 0x7fd5ac0b4200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57018/0 (socket says 172.21.15.139:57018) 2024-04-25T09:57:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.226+0000 7fd5aa59c640 1 -- 172.21.15.139:0/3289582756 learned_addr learned my addr 172.21.15.139:0/3289582756 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:20.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.226+0000 7fd5a9d9b640 1 -- 172.21.15.139:0/3289582756 >> v1:172.21.15.150:6789/0 conn(0x7fd5ac071f90 legacy=0x7fd5ac15da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.226+0000 7fd5a9d9b640 1 -- 172.21.15.139:0/3289582756 >> v1:172.21.15.139:6789/0 conn(0x7fd5ac150ed0 legacy=0x7fd5ac0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.226+0000 7fd5a9d9b640 1 -- 172.21.15.139:0/3289582756 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac15ad20 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.226+0000 7fd5a9d9b640 1 --2- 172.21.15.139:0/3289582756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5ac0cdd20 0x7fd5ac071a50 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00ba70 tx=0x7fd59c02c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe739fbab8329a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.227+0000 7fd5a959a640 1 -- 172.21.15.139:0/3289582756 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c03a070 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.227+0000 7fd5a959a640 1 -- 172.21.15.139:0/3289582756 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c032480 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.227+0000 7fd5a959a640 1 -- 172.21.15.139:0/3289582756 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c032850 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.227+0000 7fd5b0e15640 1 -- 172.21.15.139:0/3289582756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5ac0cdd20 msgr2=0x7fd5ac071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.227+0000 7fd5b0e15640 1 --2- 172.21.15.139:0/3289582756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5ac0cdd20 0x7fd5ac071a50 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00ba70 tx=0x7fd59c02c5b0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.227+0000 7fd5b0e15640 1 -- 172.21.15.139:0/3289582756 shutdown_connections 2024-04-25T09:57:20.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.227+0000 7fd5b0e15640 1 --2- 172.21.15.139:0/3289582756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5ac0cdd20 0x7fd5ac071a50 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.227+0000 7fd5b0e15640 1 -- 172.21.15.139:0/3289582756 >> 172.21.15.139:0/3289582756 conn(0x7fd5ac067000 msgr2=0x7fd5ac06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:20.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.227+0000 7fd5b0e15640 1 -- 172.21.15.139:0/3289582756 shutdown_connections 2024-04-25T09:57:20.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.227+0000 7fd5b0e15640 1 -- 172.21.15.139:0/3289582756 wait complete. 2024-04-25T09:57:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.229+0000 7fd5b0e15640 1 Processor -- start 2024-04-25T09:57:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.229+0000 7fd5b0e15640 1 -- start start 2024-04-25T09:57:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.229+0000 7fd5b0e15640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5ac071f90 0x7fd5ac1ef230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.229+0000 7fd5b0e15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5ac0cdd20 0x7fd5ac0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.229+0000 7fd5b0e15640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5ac150ed0 0x7fd5ac0b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.229+0000 7fd5b0e15640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac1512b0 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.229+0000 7fd5b0e15640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac0b3960 con 0x7fd5ac150ed0 2024-04-25T09:57:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.229+0000 7fd5b0e15640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac0b4910 con 0x7fd5ac071f90 2024-04-25T09:57:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.229+0000 7fd5a9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5ac0cdd20 0x7fd5ac0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.229+0000 7fd5a9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5ac0cdd20 0x7fd5ac0b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49860/0 (socket says 172.21.15.139:49860) 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.229+0000 7fd5a9d9b640 1 -- 172.21.15.139:0/2603145508 learned_addr learned my addr 172.21.15.139:0/2603145508 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd5aad9d640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5ac150ed0 0x7fd5ac0b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd5aa59c640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5ac071f90 0x7fd5ac1ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd5a9d9b640 1 -- 172.21.15.139:0/2603145508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5ac071f90 msgr2=0x7fd5ac1ef230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd5a9d9b640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5ac071f90 0x7fd5ac1ef230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd5a9d9b640 1 -- 172.21.15.139:0/2603145508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5ac150ed0 msgr2=0x7fd5ac0b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd5a9d9b640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5ac150ed0 0x7fd5ac0b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd5a9d9b640 1 -- 172.21.15.139:0/2603145508 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac1ef770 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd5a9d9b640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5ac0cdd20 0x7fd5ac0b12d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00b9c0 tx=0x7fd59c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd59affd640 1 -- 172.21.15.139:0/2603145508 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c03a040 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd59affd640 1 -- 172.21.15.139:0/2603145508 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c004030 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd59affd640 1 -- 172.21.15.139:0/2603145508 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c004400 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd5b0e15640 1 -- 172.21.15.139:0/2603145508 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac1467a0 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.230+0000 7fd5b0e15640 1 -- 172.21.15.139:0/2603145508 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac146d60 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.231+0000 7fd5b0e15640 1 -- 172.21.15.139:0/2603145508 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac1ef940 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.232+0000 7fd59affd640 1 -- 172.21.15.139:0/2603145508 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd59c02cd10 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.232+0000 7fd59affd640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd58c06b4d0 0x7fd58c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.232+0000 7fd59affd640 1 -- 172.21.15.139:0/2603145508 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fd59c0ad660 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.232+0000 7fd5aa59c640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd58c06b4d0 0x7fd58c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.233+0000 7fd5aa59c640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd58c06b4d0 0x7fd58c06d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0000f60 tx=0x7fd5a0007c40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.237+0000 7fd59affd640 1 -- 172.21.15.139:0/2603145508 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd59c07c560 con 0x7fd5ac0cdd20 2024-04-25T09:57:20.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:20.442+0000 7fd5b0e15640 1 -- 172.21.15.139:0/2603145508 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd5ac1ef940 con 0x7fd5ac0cdd20 2024-04-25T09:57:21.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.103+0000 7fd59affd640 1 -- 172.21.15.139:0/2603145508 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v34) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd59c048e40 con 0x7fd5ac0cdd20 2024-04-25T09:57:21.107 INFO:teuthology.orchestra.run.smithi139.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T09:57:21.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.108+0000 7fd5b0e15640 1 -- 172.21.15.139:0/2603145508 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd58c06b4d0 msgr2=0x7fd58c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.108+0000 7fd5b0e15640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd58c06b4d0 0x7fd58c06d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0000f60 tx=0x7fd5a0007c40 comp rx=0 tx=0).stop 2024-04-25T09:57:21.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.108+0000 7fd5b0e15640 1 -- 172.21.15.139:0/2603145508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5ac0cdd20 msgr2=0x7fd5ac0b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.108+0000 7fd5b0e15640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5ac0cdd20 0x7fd5ac0b12d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00b9c0 tx=0x7fd59c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.111+0000 7fd5b0e15640 1 -- 172.21.15.139:0/2603145508 shutdown_connections 2024-04-25T09:57:21.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.111+0000 7fd5b0e15640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5ac0cdd20 0x7fd5ac0b12d0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.111+0000 7fd5b0e15640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd58c06b4d0 0x7fd58c06d990 secure :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0000f60 tx=0x7fd5a0007c40 comp rx=0 tx=0).stop 2024-04-25T09:57:21.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.111+0000 7fd5b0e15640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5ac150ed0 0x7fd5ac0b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.111+0000 7fd5b0e15640 1 --2- 172.21.15.139:0/2603145508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5ac071f90 0x7fd5ac1ef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.111+0000 7fd5b0e15640 1 -- 172.21.15.139:0/2603145508 >> 172.21.15.139:0/2603145508 conn(0x7fd5ac067000 msgr2=0x7fd5ac159820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:21.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.113+0000 7fd5b0e15640 1 -- 172.21.15.139:0/2603145508 shutdown_connections 2024-04-25T09:57:21.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.113+0000 7fd5b0e15640 1 -- 172.21.15.139:0/2603145508 wait complete. 2024-04-25T09:57:21.137 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:21.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.233+0000 7f64584fc640 1 Processor -- start 2024-04-25T09:57:21.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.233+0000 7f64584fc640 1 -- start start 2024-04-25T09:57:21.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.234+0000 7f64584fc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f64500d1080 0x7f64500b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.234+0000 7f64584fc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64500cf250 con 0x7f645015dc60 2024-04-25T09:57:21.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.234+0000 7f64584fc640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64500ce9b0 con 0x7f6450161850 2024-04-25T09:57:21.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.234+0000 7f64584fc640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64500d3730 con 0x7f64500d1080 2024-04-25T09:57:21.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.234+0000 7f6456298640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f645015dc60 0x7f64500d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57032/0 (socket says 172.21.15.139:57032) 2024-04-25T09:57:21.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.234+0000 7f6456298640 1 -- 172.21.15.139:0/875318586 learned_addr learned my addr 172.21.15.139:0/875318586 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:21.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.234+0000 7f6455a97640 1 --2- 172.21.15.139:0/875318586 >> v2:172.21.15.150:3300/0 conn(0x7f64500d1080 0x7f64500b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.235+0000 7f6455296640 1 -- 172.21.15.139:0/875318586 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362814605 0 0) 0x7f644c001490 con 0x7f645015dc60 2024-04-25T09:57:21.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.235+0000 7f6455296640 1 -- 172.21.15.139:0/875318586 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643c003520 con 0x7f645015dc60 2024-04-25T09:57:21.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.235+0000 7f6455296640 1 -- 172.21.15.139:0/875318586 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667750104 0 0) 0x7f64500ce9b0 con 0x7f6450161850 2024-04-25T09:57:21.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.235+0000 7f6455296640 1 -- 172.21.15.139:0/875318586 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f644c001490 con 0x7f6450161850 2024-04-25T09:57:21.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.235+0000 7f6455a97640 1 -- 172.21.15.139:0/875318586 >> v1:172.21.15.150:6789/0 conn(0x7f6450161850 legacy=0x7f64500d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.235+0000 7f6455a97640 1 -- 172.21.15.139:0/875318586 >> v1:172.21.15.139:6789/0 conn(0x7f645015dc60 legacy=0x7f64500d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.235+0000 7f6455296640 1 -- 172.21.15.139:0/875318586 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1501037558 0 0) 0x7f643c003520 con 0x7f645015dc60 2024-04-25T09:57:21.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.235+0000 7f6455a97640 1 -- 172.21.15.139:0/875318586 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64500cec90 con 0x7f64500d1080 2024-04-25T09:57:21.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.236+0000 7f6455a97640 1 --2- 172.21.15.139:0/875318586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64500d1080 0x7f64500b6c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f64400097a0 tx=0x7f644002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9174f428e4c1ef99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.236+0000 7f6455296640 1 -- 172.21.15.139:0/875318586 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6440037070 con 0x7f64500d1080 2024-04-25T09:57:21.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.236+0000 7f6455296640 1 -- 172.21.15.139:0/875318586 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644002ac50 con 0x7f64500d1080 2024-04-25T09:57:21.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.236+0000 7f6455296640 1 -- 172.21.15.139:0/875318586 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c001bc0 con 0x7f64500d1080 2024-04-25T09:57:21.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.236+0000 7f64584fc640 1 -- 172.21.15.139:0/875318586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64500d1080 msgr2=0x7f64500b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.236+0000 7f64584fc640 1 --2- 172.21.15.139:0/875318586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64500d1080 0x7f64500b6c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f64400097a0 tx=0x7f644002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.236+0000 7f64584fc640 1 -- 172.21.15.139:0/875318586 shutdown_connections 2024-04-25T09:57:21.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.236+0000 7f64584fc640 1 --2- 172.21.15.139:0/875318586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64500d1080 0x7f64500b6c10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.236+0000 7f64584fc640 1 -- 172.21.15.139:0/875318586 >> 172.21.15.139:0/875318586 conn(0x7f64500b5910 msgr2=0x7f64500b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:21.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.237+0000 7f64584fc640 1 -- 172.21.15.139:0/875318586 shutdown_connections 2024-04-25T09:57:21.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.237+0000 7f64584fc640 1 -- 172.21.15.139:0/875318586 wait complete. 2024-04-25T09:57:21.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.238+0000 7f64584fc640 1 Processor -- start 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.238+0000 7f64584fc640 1 -- start start 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.238+0000 7f64584fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64500d1080 0x7f64500c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.238+0000 7f64584fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f645015dc60 0x7f64500c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.238+0000 7f64584fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6450161850 0x7f6450149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.238+0000 7f64584fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645015e040 con 0x7f64500d1080 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.238+0000 7f64584fc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64500d1fc0 con 0x7f6450161850 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.238+0000 7f64584fc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6450149ea0 con 0x7f645015dc60 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.238+0000 7f6456298640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64500d1080 0x7f64500c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.238+0000 7f6456298640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64500d1080 0x7f64500c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49874/0 (socket says 172.21.15.139:49874) 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f6456298640 1 -- 172.21.15.139:0/853373498 learned_addr learned my addr 172.21.15.139:0/853373498 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f6455a97640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f645015dc60 0x7f64500c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f6456298640 1 -- 172.21.15.139:0/853373498 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f645015dc60 msgr2=0x7f64500c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f6456a99640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6450161850 0x7f6450149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f6456298640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f645015dc60 0x7f64500c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f6456298640 1 -- 172.21.15.139:0/853373498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6450161850 msgr2=0x7f6450149960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f6456298640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6450161850 0x7f6450149960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f6456298640 1 -- 172.21.15.139:0/853373498 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64501488c0 con 0x7f64500d1080 2024-04-25T09:57:21.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f6456298640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64500d1080 0x7f64500c9ed0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f644c001350 tx=0x7f644c001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f6446ffd640 1 -- 172.21.15.139:0/853373498 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c017070 con 0x7f64500d1080 2024-04-25T09:57:21.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f6446ffd640 1 -- 172.21.15.139:0/853373498 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c012400 con 0x7f64500d1080 2024-04-25T09:57:21.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f64584fc640 1 -- 172.21.15.139:0/853373498 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6450148ba0 con 0x7f64500d1080 2024-04-25T09:57:21.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f6446ffd640 1 -- 172.21.15.139:0/853373498 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c0127f0 con 0x7f64500d1080 2024-04-25T09:57:21.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.239+0000 7f64584fc640 1 -- 172.21.15.139:0/853373498 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6450149160 con 0x7f64500d1080 2024-04-25T09:57:21.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.240+0000 7f64584fc640 1 -- 172.21.15.139:0/853373498 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645006d1d0 con 0x7f64500d1080 2024-04-25T09:57:21.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.240+0000 7f6446ffd640 1 -- 172.21.15.139:0/853373498 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f644c01a070 con 0x7f64500d1080 2024-04-25T09:57:21.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.241+0000 7f6446ffd640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f643406b500 0x7f643406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.241+0000 7f6446ffd640 1 -- 172.21.15.139:0/853373498 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f644c08a510 con 0x7f64500d1080 2024-04-25T09:57:21.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.241+0000 7f6455a97640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f643406b500 0x7f643406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.241+0000 7f6455a97640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f643406b500 0x7f643406d9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6440034000 tx=0x7f644002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.245+0000 7f6446ffd640 1 -- 172.21.15.139:0/853373498 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f645006d1d0 con 0x7f64500d1080 2024-04-25T09:57:21.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.415+0000 7f64584fc640 1 -- 172.21.15.139:0/853373498 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6450074520 con 0x7f64500d1080 2024-04-25T09:57:21.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.416+0000 7f6446ffd640 1 -- 172.21.15.139:0/853373498 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+17006 (secure 0 0 0) 0x7f64500cf250 con 0x7f64500d1080 2024-04-25T09:57:21.420 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:21.420 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":75,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:57:20.444502+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:18.681554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:19.409382+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:21.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.418+0000 7f64584fc640 1 -- 172.21.15.139:0/853373498 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f643406b500 msgr2=0x7f643406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.418+0000 7f64584fc640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f643406b500 0x7f643406d9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6440034000 tx=0x7f644002e000 comp rx=0 tx=0).stop 2024-04-25T09:57:21.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.418+0000 7f64584fc640 1 -- 172.21.15.139:0/853373498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64500d1080 msgr2=0x7f64500c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.418+0000 7f64584fc640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64500d1080 0x7f64500c9ed0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f644c001350 tx=0x7f644c001cf0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.418+0000 7f64584fc640 1 -- 172.21.15.139:0/853373498 shutdown_connections 2024-04-25T09:57:21.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.418+0000 7f64584fc640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f643406b500 0x7f643406d9c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.418+0000 7f64584fc640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6450161850 0x7f6450149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.418+0000 7f64584fc640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f645015dc60 0x7f64500c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.418+0000 7f64584fc640 1 --2- 172.21.15.139:0/853373498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64500d1080 0x7f64500c9ed0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.418+0000 7f64584fc640 1 -- 172.21.15.139:0/853373498 >> 172.21.15.139:0/853373498 conn(0x7f64500b5910 msgr2=0x7f6450073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:21.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.418+0000 7f64584fc640 1 -- 172.21.15.139:0/853373498 shutdown_connections 2024-04-25T09:57:21.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.418+0000 7f64584fc640 1 -- 172.21.15.139:0/853373498 wait complete. 2024-04-25T09:57:21.433 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:21.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.526+0000 7fef4a69e640 1 Processor -- start 2024-04-25T09:57:21.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.526+0000 7fef4a69e640 1 -- start start 2024-04-25T09:57:21.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.526+0000 7fef4a69e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fef440b0750 0x7fef440b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.526+0000 7fef4a69e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef440d0b90 0x7fef440cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.526+0000 7fef4a69e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef440b1070 con 0x7fef440d0730 2024-04-25T09:57:21.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.526+0000 7fef4a69e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef440ccb70 con 0x7fef440b0750 2024-04-25T09:57:21.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.526+0000 7fef4a69e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef440cf970 con 0x7fef440d0b90 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef437fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fef440d0730 0x7fef440cbe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57048/0 (socket says 172.21.15.139:57048) 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef437fe640 1 -- 172.21.15.139:0/1264377182 learned_addr learned my addr 172.21.15.139:0/1264377182 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef48c3b640 1 --2- 172.21.15.139:0/1264377182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef440d0b90 0x7fef440cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef43fff640 1 --2- 172.21.15.139:0/1264377182 >> v2:172.21.15.150:3300/0 conn(0x7fef440b0750 0x7fef440b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115199517 0 0) 0x7fef440b1070 con 0x7fef440d0730 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef30003520 con 0x7fef440d0730 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 767289376 0 0) 0x7fef30003520 con 0x7fef440d0730 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef440d0b90 msgr2=0x7fef440cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 --2- 172.21.15.139:0/1264377182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef440d0b90 0x7fef440cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 >> v2:172.21.15.150:3300/0 conn(0x7fef440b0750 msgr2=0x7fef440b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 --2- 172.21.15.139:0/1264377182 >> v2:172.21.15.150:3300/0 conn(0x7fef440b0750 0x7fef440b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef440cfc50 con 0x7fef440d0730 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fef34001670 con 0x7fef440d0730 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 >> v1:172.21.15.139:6789/0 conn(0x7fef440d0730 legacy=0x7fef440cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 --2- 172.21.15.139:0/1264377182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef300039b0 0x7fef30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 --2- 172.21.15.139:0/1264377182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef30004310 0x7fef30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 --2- 172.21.15.139:0/1264377182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef30004e50 0x7fef300052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef440ccb70 con 0x7fef30004e50 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef440cf970 con 0x7fef30004310 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3000bd40 con 0x7fef300039b0 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef48c3b640 1 --2- 172.21.15.139:0/1264377182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef30004e50 0x7fef300052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef43fff640 1 --2- 172.21.15.139:0/1264377182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef300039b0 0x7fef30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.527+0000 7fef437fe640 1 --2- 172.21.15.139:0/1264377182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef30004310 0x7fef30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.528+0000 7fef48c3b640 1 -- 172.21.15.139:0/1264377182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef300039b0 msgr2=0x7fef30003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.528+0000 7fef48c3b640 1 --2- 172.21.15.139:0/1264377182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef300039b0 0x7fef30003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.528+0000 7fef48c3b640 1 -- 172.21.15.139:0/1264377182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef30004310 msgr2=0x7fef30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.528+0000 7fef48c3b640 1 --2- 172.21.15.139:0/1264377182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef30004310 0x7fef30004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.528+0000 7fef48c3b640 1 -- 172.21.15.139:0/1264377182 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef30003520 con 0x7fef30004e50 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.528+0000 7fef48c3b640 1 --2- 172.21.15.139:0/1264377182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef30004e50 0x7fef300052d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009f90 tx=0x7fef30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8479710c701d612 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.528+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c009c10 con 0x7fef30004e50 2024-04-25T09:57:21.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.528+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c036070 con 0x7fef30004e50 2024-04-25T09:57:21.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.528+0000 7fef42ffd640 1 -- 172.21.15.139:0/1264377182 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c031850 con 0x7fef30004e50 2024-04-25T09:57:21.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.529+0000 7fef4a69e640 1 -- 172.21.15.139:0/1264377182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef30004e50 msgr2=0x7fef300052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.529+0000 7fef4a69e640 1 --2- 172.21.15.139:0/1264377182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef30004e50 0x7fef300052d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009f90 tx=0x7fef30005830 comp rx=0 tx=0).stop 2024-04-25T09:57:21.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.529+0000 7fef48c3b640 1 -- 172.21.15.139:0/1264377182 reap_dead start 2024-04-25T09:57:21.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.529+0000 7fef4a69e640 1 -- 172.21.15.139:0/1264377182 shutdown_connections 2024-04-25T09:57:21.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.529+0000 7fef4a69e640 1 -- 172.21.15.139:0/1264377182 >> 172.21.15.139:0/1264377182 conn(0x7fef440b3380 msgr2=0x7fef440b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:21.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.529+0000 7fef4a69e640 1 -- 172.21.15.139:0/1264377182 shutdown_connections 2024-04-25T09:57:21.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.529+0000 7fef4a69e640 1 -- 172.21.15.139:0/1264377182 wait complete. 2024-04-25T09:57:21.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.530+0000 7fef4a69e640 1 Processor -- start 2024-04-25T09:57:21.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.530+0000 7fef4a69e640 1 -- start start 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef4a69e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef440afed0 0x7fef441580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef4a69e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef44158600 0x7fef4415aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef4a69e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440b02b0 0x7fef4415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef4a69e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4415bcc0 con 0x7fef440b02b0 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef4a69e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef440d0210 con 0x7fef44158600 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef4a69e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4415be40 con 0x7fef440afed0 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef48c3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440b02b0 0x7fef4415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef48c3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440b02b0 0x7fef4415aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49886/0 (socket says 172.21.15.139:49886) 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef48c3b640 1 -- 172.21.15.139:0/3108538921 learned_addr learned my addr 172.21.15.139:0/3108538921 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef43fff640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef440afed0 0x7fef441580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef437fe640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef44158600 0x7fef4415aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef48c3b640 1 -- 172.21.15.139:0/3108538921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef440afed0 msgr2=0x7fef441580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef48c3b640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef440afed0 0x7fef441580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef48c3b640 1 -- 172.21.15.139:0/3108538921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef44158600 msgr2=0x7fef4415aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef48c3b640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef44158600 0x7fef4415aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.531+0000 7fef48c3b640 1 -- 172.21.15.139:0/3108538921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef44158b10 con 0x7fef440b02b0 2024-04-25T09:57:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.532+0000 7fef48c3b640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440b02b0 0x7fef4415aff0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fef3c033040 tx=0x7fef3c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.532+0000 7fef40ff9640 1 -- 172.21.15.139:0/3108538921 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c009c10 con 0x7fef440b02b0 2024-04-25T09:57:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.532+0000 7fef40ff9640 1 -- 172.21.15.139:0/3108538921 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c003990 con 0x7fef440b02b0 2024-04-25T09:57:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.532+0000 7fef4a69e640 1 -- 172.21.15.139:0/3108538921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4414d3e0 con 0x7fef440b02b0 2024-04-25T09:57:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.532+0000 7fef40ff9640 1 -- 172.21.15.139:0/3108538921 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c003d80 con 0x7fef440b02b0 2024-04-25T09:57:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.532+0000 7fef4a69e640 1 -- 172.21.15.139:0/3108538921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4414d9a0 con 0x7fef440b02b0 2024-04-25T09:57:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.533+0000 7fef4a69e640 1 -- 172.21.15.139:0/3108538921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef441513c0 con 0x7fef440b02b0 2024-04-25T09:57:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.533+0000 7fef40ff9640 1 -- 172.21.15.139:0/3108538921 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef3c005d10 con 0x7fef440b02b0 2024-04-25T09:57:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.534+0000 7fef40ff9640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef2406b500 0x7fef2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.534+0000 7fef40ff9640 1 -- 172.21.15.139:0/3108538921 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fef3c0abae0 con 0x7fef440b02b0 2024-04-25T09:57:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.534+0000 7fef43fff640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef2406b500 0x7fef2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.534+0000 7fef43fff640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef2406b500 0x7fef2406d9c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009b80 tx=0x7fef2c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.538+0000 7fef40ff9640 1 -- 172.21.15.139:0/3108538921 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef441513c0 con 0x7fef440b02b0 2024-04-25T09:57:21.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.708+0000 7fef4a69e640 1 -- 172.21.15.139:0/3108538921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef44074520 con 0x7fef440b02b0 2024-04-25T09:57:21.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.710+0000 7fef40ff9640 1 -- 172.21.15.139:0/3108538921 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+17006 (secure 0 0 0) 0x7fef3c04a060 con 0x7fef440b02b0 2024-04-25T09:57:21.713 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:21.714 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":75,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:57:20.444502+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:18.681554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:19.409382+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:21.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.712+0000 7fef4a69e640 1 -- 172.21.15.139:0/3108538921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef2406b500 msgr2=0x7fef2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.712+0000 7fef4a69e640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef2406b500 0x7fef2406d9c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009b80 tx=0x7fef2c008040 comp rx=0 tx=0).stop 2024-04-25T09:57:21.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.712+0000 7fef4a69e640 1 -- 172.21.15.139:0/3108538921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440b02b0 msgr2=0x7fef4415aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.712+0000 7fef4a69e640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440b02b0 0x7fef4415aff0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fef3c033040 tx=0x7fef3c009750 comp rx=0 tx=0).stop 2024-04-25T09:57:21.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.712+0000 7fef4a69e640 1 -- 172.21.15.139:0/3108538921 shutdown_connections 2024-04-25T09:57:21.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.712+0000 7fef4a69e640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef440b02b0 0x7fef4415aff0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.712+0000 7fef4a69e640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef2406b500 0x7fef2406d9c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.712+0000 7fef4a69e640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef44158600 0x7fef4415aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.712+0000 7fef4a69e640 1 --2- 172.21.15.139:0/3108538921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef440afed0 0x7fef441580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.712+0000 7fef4a69e640 1 -- 172.21.15.139:0/3108538921 >> 172.21.15.139:0/3108538921 conn(0x7fef440b3380 msgr2=0x7fef440b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:21.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.712+0000 7fef4a69e640 1 -- 172.21.15.139:0/3108538921 shutdown_connections 2024-04-25T09:57:21.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.712+0000 7fef4a69e640 1 -- 172.21.15.139:0/3108538921 wait complete. 2024-04-25T09:57:21.726 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:57:21.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.823+0000 7f4ccee91640 1 Processor -- start 2024-04-25T09:57:21.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.823+0000 7f4ccee91640 1 -- start start 2024-04-25T09:57:21.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.823+0000 7f4ccee91640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4cc81617e0 0x7f4cc80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.823+0000 7f4ccee91640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4cc80cec80 0x7f4cc80d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.823+0000 7f4ccee91640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc8161dc0 con 0x7f4cc815dbf0 2024-04-25T09:57:21.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.823+0000 7f4ccee91640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc80cf0d0 con 0x7f4cc81617e0 2024-04-25T09:57:21.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.823+0000 7f4ccee91640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc80d1ee0 con 0x7f4cc80cec80 2024-04-25T09:57:21.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.824+0000 7f4cc7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4cc81617e0 0x7f4cc80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.824+0000 7f4cc7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4cc81617e0 0x7f4cc80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49898/0 (socket says 172.21.15.139:49898) 2024-04-25T09:57:21.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.824+0000 7f4cc7fff640 1 -- 172.21.15.139:0/3760532120 learned_addr learned my addr 172.21.15.139:0/3760532120 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:21.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.824+0000 7f4ccd42e640 1 --2- 172.21.15.139:0/3760532120 >> v2:172.21.15.150:3300/0 conn(0x7f4cc80cec80 0x7f4cc80d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.824+0000 7f4cc7fff640 1 -- 172.21.15.139:0/3760532120 >> v2:172.21.15.150:3300/0 conn(0x7f4cc80cec80 msgr2=0x7f4cc80d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:21.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.824+0000 7f4cc7fff640 1 --2- 172.21.15.139:0/3760532120 >> v2:172.21.15.150:3300/0 conn(0x7f4cc80cec80 0x7f4cc80d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.824+0000 7f4cc7fff640 1 -- 172.21.15.139:0/3760532120 >> v1:172.21.15.150:6789/0 conn(0x7f4cc815dbf0 legacy=0x7f4cc80b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:21.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.824+0000 7f4cc7fff640 1 -- 172.21.15.139:0/3760532120 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc81620a0 con 0x7f4cc81617e0 2024-04-25T09:57:21.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.825+0000 7f4cc7fff640 1 --2- 172.21.15.139:0/3760532120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc81617e0 0x7f4cc80ce740 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4cc808ce00 tx=0x7f4cb402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97c6bad4914bab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.825+0000 7f4cc77fe640 1 -- 172.21.15.139:0/3760532120 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4037070 con 0x7f4cc81617e0 2024-04-25T09:57:21.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.825+0000 7f4cc77fe640 1 -- 172.21.15.139:0/3760532120 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb4032440 con 0x7f4cc81617e0 2024-04-25T09:57:21.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.825+0000 7f4cc77fe640 1 -- 172.21.15.139:0/3760532120 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4032810 con 0x7f4cc81617e0 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.825+0000 7f4ccee91640 1 -- 172.21.15.139:0/3760532120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc81617e0 msgr2=0x7f4cc80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.825+0000 7f4ccee91640 1 --2- 172.21.15.139:0/3760532120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc81617e0 0x7f4cc80ce740 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4cc808ce00 tx=0x7f4cb402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.825+0000 7f4ccee91640 1 -- 172.21.15.139:0/3760532120 shutdown_connections 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.825+0000 7f4ccee91640 1 --2- 172.21.15.139:0/3760532120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc81617e0 0x7f4cc80ce740 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.825+0000 7f4ccee91640 1 --2- 172.21.15.139:0/3760532120 >> v2:172.21.15.150:3300/0 conn(0x7f4cc80cec80 0x7f4cc80d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.825+0000 7f4ccee91640 1 -- 172.21.15.139:0/3760532120 >> 172.21.15.139:0/3760532120 conn(0x7f4cc80b5480 msgr2=0x7f4cc80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.825+0000 7f4ccee91640 1 -- 172.21.15.139:0/3760532120 shutdown_connections 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.825+0000 7f4ccee91640 1 -- 172.21.15.139:0/3760532120 wait complete. 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.826+0000 7f4ccee91640 1 Processor -- start 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.826+0000 7f4ccee91640 1 -- start start 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.827+0000 7f4ccee91640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cc80cec80 0x7f4cc81f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.827+0000 7f4ccee91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc815dbf0 0x7f4cc80ab830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.827+0000 7f4ccee91640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4cc81617e0 0x7f4cc814e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.827+0000 7f4ccee91640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc815dfd0 con 0x7f4cc815dbf0 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.827+0000 7f4ccee91640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc8162660 con 0x7f4cc80cec80 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.827+0000 7f4ccee91640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc815a330 con 0x7f4cc81617e0 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.827+0000 7f4cc7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc815dbf0 0x7f4cc80ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.827+0000 7f4cc7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc815dbf0 0x7f4cc80ab830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49904/0 (socket says 172.21.15.139:49904) 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.827+0000 7f4cc7fff640 1 -- 172.21.15.139:0/193504372 learned_addr learned my addr 172.21.15.139:0/193504372 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4cccc2d640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cc80cec80 0x7f4cc81f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4ccd42e640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4cc81617e0 0x7f4cc814e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4cc7fff640 1 -- 172.21.15.139:0/193504372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4cc81617e0 msgr2=0x7f4cc814e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4cc7fff640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4cc81617e0 0x7f4cc814e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4cc7fff640 1 -- 172.21.15.139:0/193504372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cc80cec80 msgr2=0x7f4cc81f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4cc7fff640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cc80cec80 0x7f4cc81f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4cc7fff640 1 -- 172.21.15.139:0/193504372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc8159a40 con 0x7f4cc815dbf0 2024-04-25T09:57:21.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4cccc2d640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cc80cec80 0x7f4cc81f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:21.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4cc7fff640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc815dbf0 0x7f4cc80ab830 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4000c00 tx=0x7f4cb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4cc57fa640 1 -- 172.21.15.139:0/193504372 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4041020 con 0x7f4cc815dbf0 2024-04-25T09:57:21.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4cc57fa640 1 -- 172.21.15.139:0/193504372 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb40306c0 con 0x7f4cc815dbf0 2024-04-25T09:57:21.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4cc57fa640 1 -- 172.21.15.139:0/193504372 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4037040 con 0x7f4cc815dbf0 2024-04-25T09:57:21.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4ccee91640 1 -- 172.21.15.139:0/193504372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc80ab280 con 0x7f4cc815dbf0 2024-04-25T09:57:21.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.828+0000 7f4ccee91640 1 -- 172.21.15.139:0/193504372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc8159fc0 con 0x7f4cc815dbf0 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.829+0000 7f4ccee91640 1 -- 172.21.15.139:0/193504372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc8074520 con 0x7f4cc815dbf0 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.830+0000 7f4cc57fa640 1 -- 172.21.15.139:0/193504372 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cb4044020 con 0x7f4cc815dbf0 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.830+0000 7f4cc57fa640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4cac06b4d0 0x7f4cac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.830+0000 7f4cc57fa640 1 -- 172.21.15.139:0/193504372 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4cb40aab40 con 0x7f4cc815dbf0 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.830+0000 7f4cccc2d640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4cac06b4d0 0x7f4cac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:21.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.830+0000 7f4cccc2d640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4cac06b4d0 0x7f4cac06d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc003eb0 tx=0x7f4cbc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:21.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:21.834+0000 7f4cc57fa640 1 -- 172.21.15.139:0/193504372 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4cb404b030 con 0x7f4cc815dbf0 2024-04-25T09:57:22.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.040+0000 7f4ccee91640 1 -- 172.21.15.139:0/193504372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4cc80b8e60 con 0x7f4cc815dbf0 2024-04-25T09:57:22.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.107+0000 7f4cc57fa640 1 -- 172.21.15.139:0/193504372 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4cb4079830 con 0x7f4cc815dbf0 2024-04-25T09:57:22.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.108+0000 7f4ccee91640 1 -- 172.21.15.139:0/193504372 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4cac06b4d0 msgr2=0x7f4cac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.108+0000 7f4ccee91640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4cac06b4d0 0x7f4cac06d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc003eb0 tx=0x7f4cbc009040 comp rx=0 tx=0).stop 2024-04-25T09:57:22.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.108+0000 7f4ccee91640 1 -- 172.21.15.139:0/193504372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc815dbf0 msgr2=0x7f4cc80ab830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.108+0000 7f4ccee91640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc815dbf0 0x7f4cc80ab830 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4000c00 tx=0x7f4cb40046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.109+0000 7f4ccee91640 1 -- 172.21.15.139:0/193504372 shutdown_connections 2024-04-25T09:57:22.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.109+0000 7f4ccee91640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cc815dbf0 0x7f4cc80ab830 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.109+0000 7f4ccee91640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4cac06b4d0 0x7f4cac06d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.109+0000 7f4ccee91640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4cc81617e0 0x7f4cc814e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.109+0000 7f4ccee91640 1 --2- 172.21.15.139:0/193504372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cc80cec80 0x7f4cc81f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.109+0000 7f4ccee91640 1 -- 172.21.15.139:0/193504372 >> 172.21.15.139:0/193504372 conn(0x7f4cc80b5480 msgr2=0x7f4cc80cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:22.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.109+0000 7f4ccee91640 1 -- 172.21.15.139:0/193504372 shutdown_connections 2024-04-25T09:57:22.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.109+0000 7f4ccee91640 1 -- 172.21.15.139:0/193504372 wait complete. 2024-04-25T09:57:22.123 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:57:22.124 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:57:22.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.220+0000 7f8c2fd53640 1 Processor -- start 2024-04-25T09:57:22.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.220+0000 7f8c2fd53640 1 -- start start 2024-04-25T09:57:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.221+0000 7f8c2fd53640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8c2806ea40 0x7f8c2815e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.221+0000 7f8c2fd53640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c280b2a50 con 0x7f8c28144090 2024-04-25T09:57:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.221+0000 7f8c2fd53640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c280cd0f0 con 0x7f8c280b31a0 2024-04-25T09:57:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.221+0000 7f8c2fd53640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c280b3610 con 0x7f8c2806ea40 2024-04-25T09:57:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.221+0000 7f8c2daef640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8c28144090 0x7f8c280cc9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57052/0 (socket says 172.21.15.139:57052) 2024-04-25T09:57:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.221+0000 7f8c2daef640 1 -- 172.21.15.139:0/995059026 learned_addr learned my addr 172.21.15.139:0/995059026 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.221+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750109096 0 0) 0x7f8c24001490 con 0x7f8c28144090 2024-04-25T09:57:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c14003520 con 0x7f8c28144090 2024-04-25T09:57:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2e2f0640 1 --2- 172.21.15.139:0/995059026 >> v2:172.21.15.150:3300/0 conn(0x7f8c2806ea40 0x7f8c2815e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3527565770 0 0) 0x7f8c14003520 con 0x7f8c28144090 2024-04-25T09:57:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 >> v2:172.21.15.150:3300/0 conn(0x7f8c2806ea40 msgr2=0x7f8c2815e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 --2- 172.21.15.139:0/995059026 >> v2:172.21.15.150:3300/0 conn(0x7f8c2806ea40 0x7f8c2815e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 >> v1:172.21.15.150:6789/0 conn(0x7f8c280b31a0 legacy=0x7f8c2806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2815b090 con 0x7f8c28144090 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f8c24001bc0 con 0x7f8c28144090 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 >> v1:172.21.15.139:6789/0 conn(0x7f8c28144090 legacy=0x7f8c280cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 --2- 172.21.15.139:0/995059026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c140039b0 0x7f8c14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 --2- 172.21.15.139:0/995059026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c14004800 0x7f8c14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 --2- 172.21.15.139:0/995059026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c140051c0 0x7f8c1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c280b3610 con 0x7f8c140051c0 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1400c160 con 0x7f8c140039b0 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1400e570 con 0x7f8c14004800 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2e2f0640 1 --2- 172.21.15.139:0/995059026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c140051c0 0x7f8c1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2daef640 1 --2- 172.21.15.139:0/995059026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c140039b0 0x7f8c14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2d2ee640 1 --2- 172.21.15.139:0/995059026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c14004800 0x7f8c14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2e2f0640 1 -- 172.21.15.139:0/995059026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c14004800 msgr2=0x7f8c14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2e2f0640 1 --2- 172.21.15.139:0/995059026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c14004800 0x7f8c14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2e2f0640 1 -- 172.21.15.139:0/995059026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c140039b0 msgr2=0x7f8c14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2e2f0640 1 --2- 172.21.15.139:0/995059026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c140039b0 0x7f8c14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.222+0000 7f8c2e2f0640 1 -- 172.21.15.139:0/995059026 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c14003520 con 0x7f8c140051c0 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.223+0000 7f8c2e2f0640 1 -- 172.21.15.139:0/995059026 reap_dead start 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.223+0000 7f8c2e2f0640 1 --2- 172.21.15.139:0/995059026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c140051c0 0x7f8c1400b9c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009a90 tx=0x7f8c14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d32e469fb9ab16e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.223+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20036070 con 0x7f8c140051c0 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.223+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c20031400 con 0x7f8c140051c0 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.223+0000 7f8c2caed640 1 -- 172.21.15.139:0/995059026 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20009690 con 0x7f8c140051c0 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.224+0000 7f8c2fd53640 1 -- 172.21.15.139:0/995059026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c140051c0 msgr2=0x7f8c1400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.224+0000 7f8c2fd53640 1 --2- 172.21.15.139:0/995059026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c140051c0 0x7f8c1400b9c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009a90 tx=0x7f8c14005830 comp rx=0 tx=0).stop 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.224+0000 7f8c2fd53640 1 -- 172.21.15.139:0/995059026 shutdown_connections 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.224+0000 7f8c2fd53640 1 --2- 172.21.15.139:0/995059026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c140051c0 0x7f8c1400b9c0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.224+0000 7f8c2fd53640 1 -- 172.21.15.139:0/995059026 >> 172.21.15.139:0/995059026 conn(0x7f8c28067000 msgr2=0x7f8c2806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.224+0000 7f8c2fd53640 1 -- 172.21.15.139:0/995059026 shutdown_connections 2024-04-25T09:57:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.224+0000 7f8c2fd53640 1 -- 172.21.15.139:0/995059026 wait complete. 2024-04-25T09:57:22.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.225+0000 7f8c2fd53640 1 Processor -- start 2024-04-25T09:57:22.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.225+0000 7f8c2fd53640 1 -- start start 2024-04-25T09:57:22.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2fd53640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c140051c0 0x7f8c281429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2fd53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c2813dbf0 0x7f8c2813e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2fd53640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c280ac930 0x7f8c280acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2fd53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2815b930 con 0x7f8c2813dbf0 2024-04-25T09:57:22.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2fd53640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2815b650 con 0x7f8c140051c0 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2fd53640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2815bc10 con 0x7f8c280ac930 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2d2ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c2813dbf0 0x7f8c2813e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2d2ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c2813dbf0 0x7f8c2813e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49920/0 (socket says 172.21.15.139:49920) 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2d2ee640 1 -- 172.21.15.139:0/832692638 learned_addr learned my addr 172.21.15.139:0/832692638 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2daef640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c140051c0 0x7f8c281429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2e2f0640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c280ac930 0x7f8c280acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2d2ee640 1 -- 172.21.15.139:0/832692638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c280ac930 msgr2=0x7f8c280acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2d2ee640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c280ac930 0x7f8c280acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2d2ee640 1 -- 172.21.15.139:0/832692638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c140051c0 msgr2=0x7f8c281429a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2d2ee640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c140051c0 0x7f8c281429a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2d2ee640 1 -- 172.21.15.139:0/832692638 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c280ad660 con 0x7f8c2813dbf0 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2d2ee640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c2813dbf0 0x7f8c2813e050 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800f650 tx=0x7f8c1800fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c1e7fc640 1 -- 172.21.15.139:0/832692638 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c180052e0 con 0x7f8c2813dbf0 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c1e7fc640 1 -- 172.21.15.139:0/832692638 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c18018070 con 0x7f8c2813dbf0 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2fd53640 1 -- 172.21.15.139:0/832692638 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c28131220 con 0x7f8c2813dbf0 2024-04-25T09:57:22.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c1e7fc640 1 -- 172.21.15.139:0/832692638 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18003810 con 0x7f8c2813dbf0 2024-04-25T09:57:22.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.226+0000 7f8c2fd53640 1 -- 172.21.15.139:0/832692638 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2813e590 con 0x7f8c2813dbf0 2024-04-25T09:57:22.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.227+0000 7f8c2fd53640 1 -- 172.21.15.139:0/832692638 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c28131010 con 0x7f8c2813dbf0 2024-04-25T09:57:22.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.228+0000 7f8c1e7fc640 1 -- 172.21.15.139:0/832692638 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c180039b0 con 0x7f8c2813dbf0 2024-04-25T09:57:22.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.228+0000 7f8c1e7fc640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c0c06b4d0 0x7f8c0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:22.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.228+0000 7f8c1e7fc640 1 -- 172.21.15.139:0/832692638 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c1808a000 con 0x7f8c2813dbf0 2024-04-25T09:57:22.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.228+0000 7f8c2daef640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c0c06b4d0 0x7f8c0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:22.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.228+0000 7f8c2daef640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c0c06b4d0 0x7f8c0c06d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8c240013d0 tx=0x7f8c24006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:22.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.232+0000 7f8c1e7fc640 1 -- 172.21.15.139:0/832692638 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c18058cf0 con 0x7f8c2813dbf0 2024-04-25T09:57:22.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:22.441+0000 7f8c2fd53640 1 -- 172.21.15.139:0/832692638 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8c28074520 con 0x7f8c2813dbf0 2024-04-25T09:57:23.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.113+0000 7f8c1e7fc640 1 -- 172.21.15.139:0/832692638 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8c180585f0 con 0x7f8c2813dbf0 2024-04-25T09:57:23.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.116+0000 7f8c2fd53640 1 -- 172.21.15.139:0/832692638 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c0c06b4d0 msgr2=0x7f8c0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.116+0000 7f8c2fd53640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c0c06b4d0 0x7f8c0c06d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8c240013d0 tx=0x7f8c24006040 comp rx=0 tx=0).stop 2024-04-25T09:57:23.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.116+0000 7f8c2fd53640 1 -- 172.21.15.139:0/832692638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c2813dbf0 msgr2=0x7f8c2813e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.116+0000 7f8c2fd53640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c2813dbf0 0x7f8c2813e050 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800f650 tx=0x7f8c1800fa50 comp rx=0 tx=0).stop 2024-04-25T09:57:23.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.116+0000 7f8c2fd53640 1 -- 172.21.15.139:0/832692638 shutdown_connections 2024-04-25T09:57:23.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.116+0000 7f8c2fd53640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c2813dbf0 0x7f8c2813e050 secure :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800f650 tx=0x7f8c1800fa50 comp rx=0 tx=0).stop 2024-04-25T09:57:23.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.116+0000 7f8c2fd53640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c0c06b4d0 0x7f8c0c06d990 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.116+0000 7f8c2fd53640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c280ac930 0x7f8c280acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.116+0000 7f8c2fd53640 1 --2- 172.21.15.139:0/832692638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c140051c0 0x7f8c281429a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.116+0000 7f8c2fd53640 1 -- 172.21.15.139:0/832692638 >> 172.21.15.139:0/832692638 conn(0x7f8c28067000 msgr2=0x7f8c280cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:23.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.116+0000 7f8c2fd53640 1 -- 172.21.15.139:0/832692638 shutdown_connections 2024-04-25T09:57:23.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.116+0000 7f8c2fd53640 1 -- 172.21.15.139:0/832692638 wait complete. 2024-04-25T09:57:23.131 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:57:23.131 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:23.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.226+0000 7f4a73e12640 1 Processor -- start 2024-04-25T09:57:23.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.226+0000 7f4a73e12640 1 -- start start 2024-04-25T09:57:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a73e12640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6c15a860 0x7f4a6c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a73e12640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a6c0af6d0 con 0x7f4a6c06ea20 2024-04-25T09:57:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a73e12640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a6c0af9b0 con 0x7f4a6c0cd1f0 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a73e12640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0b02e0 con 0x7f4a6c15a860 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a723af640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4a6c06ea20 0x7f4a6c0b0770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57056/0 (socket says 172.21.15.139:57056) 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a723af640 1 -- 172.21.15.139:0/3263967408 learned_addr learned my addr 172.21.15.139:0/3263967408 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834055901 0 0) 0x7f4a6c0af6d0 con 0x7f4a6c06ea20 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a58003520 con 0x7f4a6c06ea20 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a71bae640 1 --2- 172.21.15.139:0/3263967408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6c15a860 0x7f4a6c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2121351129 0 0) 0x7f4a58003520 con 0x7f4a6c06ea20 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6c15a860 msgr2=0x7f4a6c0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a70bac640 1 --2- 172.21.15.139:0/3263967408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6c15a860 0x7f4a6c0cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 >> v1:172.21.15.150:6789/0 conn(0x7f4a6c0cd1f0 legacy=0x7f4a6c06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c0afc90 con 0x7f4a6c06ea20 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f4a640019f0 con 0x7f4a6c06ea20 2024-04-25T09:57:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.227+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 >> v1:172.21.15.139:6789/0 conn(0x7f4a6c06ea20 legacy=0x7f4a6c0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a70bac640 1 --2- 172.21.15.139:0/3263967408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a580039e0 0x7f4a58003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a70bac640 1 --2- 172.21.15.139:0/3263967408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6c15a860 0x7f4a58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a70bac640 1 --2- 172.21.15.139:0/3263967408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a58004d40 0x7f4a580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0b02e0 con 0x7f4a58004d40 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5800bb50 con 0x7f4a580039e0 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5800bd20 con 0x7f4a6c15a860 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a723af640 1 --2- 172.21.15.139:0/3263967408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a58004d40 0x7f4a580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a71bae640 1 --2- 172.21.15.139:0/3263967408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a580039e0 0x7f4a58003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a713ad640 1 --2- 172.21.15.139:0/3263967408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6c15a860 0x7f4a58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a723af640 1 -- 172.21.15.139:0/3263967408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6c15a860 msgr2=0x7f4a58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a723af640 1 --2- 172.21.15.139:0/3263967408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a6c15a860 0x7f4a58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a723af640 1 -- 172.21.15.139:0/3263967408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a580039e0 msgr2=0x7f4a58003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a723af640 1 --2- 172.21.15.139:0/3263967408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a580039e0 0x7f4a58003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.228+0000 7f4a723af640 1 -- 172.21.15.139:0/3263967408 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a58003520 con 0x7f4a58004d40 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.229+0000 7f4a723af640 1 --2- 172.21.15.139:0/3263967408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a58004d40 0x7f4a580051c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400a830 tx=0x7f4a58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465aedfc5e120d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.229+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400a480 con 0x7f4a58004d40 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.229+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6402f070 con 0x7f4a58004d40 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.229+0000 7f4a70bac640 1 -- 172.21.15.139:0/3263967408 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6402bc90 con 0x7f4a58004d40 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.229+0000 7f4a73e12640 1 -- 172.21.15.139:0/3263967408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a58004d40 msgr2=0x7f4a580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.229+0000 7f4a73e12640 1 --2- 172.21.15.139:0/3263967408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a58004d40 0x7f4a580051c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400a830 tx=0x7f4a58005830 comp rx=0 tx=0).stop 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.229+0000 7f4a723af640 1 -- 172.21.15.139:0/3263967408 reap_dead start 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.229+0000 7f4a73e12640 1 -- 172.21.15.139:0/3263967408 shutdown_connections 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.229+0000 7f4a73e12640 1 -- 172.21.15.139:0/3263967408 >> 172.21.15.139:0/3263967408 conn(0x7f4a6c066f20 msgr2=0x7f4a6c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.229+0000 7f4a73e12640 1 -- 172.21.15.139:0/3263967408 shutdown_connections 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.229+0000 7f4a73e12640 1 -- 172.21.15.139:0/3263967408 wait complete. 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.230+0000 7f4a73e12640 1 Processor -- start 2024-04-25T09:57:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.230+0000 7f4a73e12640 1 -- start start 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a73e12640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a58004d40 0x7f4a6c13e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a73e12640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a6c14ead0 0x7f4a6c14ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a73e12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a6c0b2770 0x7f4a6c0b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a73e12640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0af420 con 0x7f4a6c0b2770 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a73e12640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0af2a0 con 0x7f4a6c14ead0 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a73e12640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c15ac40 con 0x7f4a58004d40 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a723af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a6c0b2770 0x7f4a6c0b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a723af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a6c0b2770 0x7f4a6c0b2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49948/0 (socket says 172.21.15.139:49948) 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a723af640 1 -- 172.21.15.139:0/3102505771 learned_addr learned my addr 172.21.15.139:0/3102505771 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a71bae640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a58004d40 0x7f4a6c13e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a713ad640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a6c14ead0 0x7f4a6c14ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a723af640 1 -- 172.21.15.139:0/3102505771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a58004d40 msgr2=0x7f4a6c13e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a723af640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a58004d40 0x7f4a6c13e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a723af640 1 -- 172.21.15.139:0/3102505771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a6c14ead0 msgr2=0x7f4a6c14ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.231+0000 7f4a723af640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a6c14ead0 0x7f4a6c14ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.232+0000 7f4a723af640 1 -- 172.21.15.139:0/3102505771 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c14d090 con 0x7f4a6c0b2770 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.232+0000 7f4a71bae640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a58004d40 0x7f4a6c13e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.232+0000 7f4a713ad640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a6c14ead0 0x7f4a6c14ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.232+0000 7f4a723af640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a6c0b2770 0x7f4a6c0b2bf0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4a6402b500 tx=0x7f4a6c0b0250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.232+0000 7f4a627fc640 1 -- 172.21.15.139:0/3102505771 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400a480 con 0x7f4a6c0b2770 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.232+0000 7f4a627fc640 1 -- 172.21.15.139:0/3102505771 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6402cb50 con 0x7f4a6c0b2770 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.232+0000 7f4a73e12640 1 -- 172.21.15.139:0/3102505771 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6c14d370 con 0x7f4a6c0b2770 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.232+0000 7f4a627fc640 1 -- 172.21.15.139:0/3102505771 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64034470 con 0x7f4a6c0b2770 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.232+0000 7f4a73e12640 1 -- 172.21.15.139:0/3102505771 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6c14d930 con 0x7f4a6c0b2770 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.233+0000 7f4a73e12640 1 -- 172.21.15.139:0/3102505771 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6c0b3130 con 0x7f4a6c0b2770 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.233+0000 7f4a627fc640 1 -- 172.21.15.139:0/3102505771 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a64034610 con 0x7f4a6c0b2770 2024-04-25T09:57:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.234+0000 7f4a627fc640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4a5006b4d0 0x7f4a5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.234+0000 7f4a627fc640 1 -- 172.21.15.139:0/3102505771 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a640aa440 con 0x7f4a6c0b2770 2024-04-25T09:57:23.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.234+0000 7f4a71bae640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4a5006b4d0 0x7f4a5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.234+0000 7f4a71bae640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4a5006b4d0 0x7f4a5006d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4a68006fd0 tx=0x7f4a68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.238+0000 7f4a627fc640 1 -- 172.21.15.139:0/3102505771 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a640791c0 con 0x7f4a6c0b2770 2024-04-25T09:57:23.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.443+0000 7f4a73e12640 1 -- 172.21.15.139:0/3102505771 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a6c074520 con 0x7f4a6c0b2770 2024-04-25T09:57:23.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.444+0000 7f4a627fc640 1 -- 172.21.15.139:0/3102505771 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f4a64078ac0 con 0x7f4a6c0b2770 2024-04-25T09:57:23.447 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:23.448 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1384380923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":1384380923},{"type":"v1","addr":"172.21.15.139:6837","nonce":1384380923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:20.444843+0000","modified":"2024-04-25T09:57:22.443106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6834/1932986598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1932986598},{"type":"v1","addr":"172.21.15.150:6834","nonce":1932986598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:57:23.448 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 36 2024-04-25T09:57:23.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.446+0000 7f4a73e12640 1 -- 172.21.15.139:0/3102505771 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4a5006b4d0 msgr2=0x7f4a5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.446+0000 7f4a73e12640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4a5006b4d0 0x7f4a5006d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4a68006fd0 tx=0x7f4a68008040 comp rx=0 tx=0).stop 2024-04-25T09:57:23.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.446+0000 7f4a73e12640 1 -- 172.21.15.139:0/3102505771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a6c0b2770 msgr2=0x7f4a6c0b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.446+0000 7f4a73e12640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a6c0b2770 0x7f4a6c0b2bf0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4a6402b500 tx=0x7f4a6c0b0250 comp rx=0 tx=0).stop 2024-04-25T09:57:23.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.446+0000 7f4a73e12640 1 -- 172.21.15.139:0/3102505771 shutdown_connections 2024-04-25T09:57:23.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.446+0000 7f4a73e12640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a6c0b2770 0x7f4a6c0b2bf0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.446+0000 7f4a73e12640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4a5006b4d0 0x7f4a5006d990 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.446+0000 7f4a73e12640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a6c14ead0 0x7f4a6c14ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.446+0000 7f4a73e12640 1 --2- 172.21.15.139:0/3102505771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a58004d40 0x7f4a6c13e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.446+0000 7f4a73e12640 1 -- 172.21.15.139:0/3102505771 >> 172.21.15.139:0/3102505771 conn(0x7f4a6c066f20 msgr2=0x7f4a6c0b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:23.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.446+0000 7f4a73e12640 1 -- 172.21.15.139:0/3102505771 shutdown_connections 2024-04-25T09:57:23.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.446+0000 7f4a73e12640 1 -- 172.21.15.139:0/3102505771 wait complete. 2024-04-25T09:57:23.460 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:23.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.553+0000 7ffb1ffc1640 1 Processor -- start 2024-04-25T09:57:23.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.553+0000 7ffb1ffc1640 1 -- start start 2024-04-25T09:57:23.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1ffc1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb1815a860 0x7ffb180cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1ffc1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffb180cd1f0 0x7ffb1806e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1ffc1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb180af6d0 con 0x7ffb1815ac40 2024-04-25T09:57:23.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1ffc1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb180b02e0 con 0x7ffb180cd1f0 2024-04-25T09:57:23.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1ffc1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb180af9b0 con 0x7ffb1815a860 2024-04-25T09:57:23.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1d55c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffb180cd1f0 0x7ffb1806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1d55c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffb180cd1f0 0x7ffb1806e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49950/0 (socket says 172.21.15.139:49950) 2024-04-25T09:57:23.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1d55c640 1 -- 172.21.15.139:0/2823837634 learned_addr learned my addr 172.21.15.139:0/2823837634 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:23.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1dd5d640 1 --2- 172.21.15.139:0/2823837634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb1815a860 0x7ffb180cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1d55c640 1 -- 172.21.15.139:0/2823837634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb1815a860 msgr2=0x7ffb180cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:23.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1d55c640 1 --2- 172.21.15.139:0/2823837634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb1815a860 0x7ffb180cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1d55c640 1 -- 172.21.15.139:0/2823837634 >> v1:172.21.15.150:6789/0 conn(0x7ffb1815ac40 legacy=0x7ffb1815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.554+0000 7ffb1d55c640 1 -- 172.21.15.139:0/2823837634 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb180afc90 con 0x7ffb180cd1f0 2024-04-25T09:57:23.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.555+0000 7ffb1d55c640 1 --2- 172.21.15.139:0/2823837634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb180cd1f0 0x7ffb1806e310 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800bb10 tx=0x7ffb0802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137edf820f0c773a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.555+0000 7ffb1cd5b640 1 -- 172.21.15.139:0/2823837634 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08036020 con 0x7ffb180cd1f0 2024-04-25T09:57:23.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.555+0000 7ffb1cd5b640 1 -- 172.21.15.139:0/2823837634 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb08032820 con 0x7ffb180cd1f0 2024-04-25T09:57:23.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.555+0000 7ffb1cd5b640 1 -- 172.21.15.139:0/2823837634 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08032bf0 con 0x7ffb180cd1f0 2024-04-25T09:57:23.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.555+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/2823837634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb180cd1f0 msgr2=0x7ffb1806e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.555+0000 7ffb1ffc1640 1 --2- 172.21.15.139:0/2823837634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb180cd1f0 0x7ffb1806e310 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800bb10 tx=0x7ffb0802c920 comp rx=0 tx=0).stop 2024-04-25T09:57:23.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.556+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/2823837634 shutdown_connections 2024-04-25T09:57:23.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.556+0000 7ffb1ffc1640 1 --2- 172.21.15.139:0/2823837634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb180cd1f0 0x7ffb1806e310 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.556+0000 7ffb1ffc1640 1 --2- 172.21.15.139:0/2823837634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb1815a860 0x7ffb180cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.556+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/2823837634 >> 172.21.15.139:0/2823837634 conn(0x7ffb18066f20 msgr2=0x7ffb1806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:23.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.556+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/2823837634 shutdown_connections 2024-04-25T09:57:23.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.556+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/2823837634 wait complete. 2024-04-25T09:57:23.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.557+0000 7ffb1ffc1640 1 Processor -- start 2024-04-25T09:57:23.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.557+0000 7ffb1ffc1640 1 -- start start 2024-04-25T09:57:23.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.557+0000 7ffb1ffc1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb180cd1f0 0x7ffb1814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.557+0000 7ffb1ffc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb1815a860 0x7ffb180ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.557+0000 7ffb1ffc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb1815ac40 0x7ffb18152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.557+0000 7ffb1ffc1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb180af420 con 0x7ffb1815ac40 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.557+0000 7ffb1ffc1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb180af2a0 con 0x7ffb1815a860 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.557+0000 7ffb1ffc1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1806e850 con 0x7ffb180cd1f0 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1e55e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb1815ac40 0x7ffb18152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1e55e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb1815ac40 0x7ffb18152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49966/0 (socket says 172.21.15.139:49966) 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1e55e640 1 -- 172.21.15.139:0/520998929 learned_addr learned my addr 172.21.15.139:0/520998929 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1d55c640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb1815a860 0x7ffb180ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1dd5d640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb180cd1f0 0x7ffb1814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1e55e640 1 -- 172.21.15.139:0/520998929 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb180cd1f0 msgr2=0x7ffb1814fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1e55e640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb180cd1f0 0x7ffb1814fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1e55e640 1 -- 172.21.15.139:0/520998929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb1815a860 msgr2=0x7ffb180ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1e55e640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb1815a860 0x7ffb180ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1e55e640 1 -- 172.21.15.139:0/520998929 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb181efe10 con 0x7ffb1815ac40 2024-04-25T09:57:23.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1d55c640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb1815a860 0x7ffb180ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:23.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1dd5d640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb180cd1f0 0x7ffb1814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:23.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1e55e640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb1815ac40 0x7ffb18152e40 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000c760 tx=0x7ffb1000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb0e7fc640 1 -- 172.21.15.139:0/520998929 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10003b70 con 0x7ffb1815ac40 2024-04-25T09:57:23.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb0e7fc640 1 -- 172.21.15.139:0/520998929 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb10003d10 con 0x7ffb1815ac40 2024-04-25T09:57:23.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/520998929 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb181efb30 con 0x7ffb1815ac40 2024-04-25T09:57:23.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb0e7fc640 1 -- 172.21.15.139:0/520998929 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000aa30 con 0x7ffb1815ac40 2024-04-25T09:57:23.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.558+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/520998929 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb180ac630 con 0x7ffb1815ac40 2024-04-25T09:57:23.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.559+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/520998929 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb18074520 con 0x7ffb1815ac40 2024-04-25T09:57:23.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.559+0000 7ffb0e7fc640 1 -- 172.21.15.139:0/520998929 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb1000abd0 con 0x7ffb1815ac40 2024-04-25T09:57:23.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.560+0000 7ffb0e7fc640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffafc06b4d0 0x7ffafc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.560+0000 7ffb0e7fc640 1 -- 172.21.15.139:0/520998929 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb1008a0d0 con 0x7ffb1815ac40 2024-04-25T09:57:23.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.560+0000 7ffb1dd5d640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffafc06b4d0 0x7ffafc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.560+0000 7ffb1dd5d640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffafc06b4d0 0x7ffafc06d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb14006fd0 tx=0x7ffb14008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.564+0000 7ffb0e7fc640 1 -- 172.21.15.139:0/520998929 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb10058e50 con 0x7ffb1815ac40 2024-04-25T09:57:23.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.735+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/520998929 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb1806e4d0 con 0x7ffb1815ac40 2024-04-25T09:57:23.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.736+0000 7ffb0e7fc640 1 -- 172.21.15.139:0/520998929 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16993 (secure 0 0 0) 0x7ffb10058750 con 0x7ffb1815ac40 2024-04-25T09:57:23.740 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:23.740 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":76,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:57:22.102239+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:18.681554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:19.409382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.738+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/520998929 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffafc06b4d0 msgr2=0x7ffafc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.738+0000 7ffb1ffc1640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffafc06b4d0 0x7ffafc06d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb14006fd0 tx=0x7ffb14008040 comp rx=0 tx=0).stop 2024-04-25T09:57:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.738+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/520998929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb1815ac40 msgr2=0x7ffb18152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.738+0000 7ffb1ffc1640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb1815ac40 0x7ffb18152e40 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000c760 tx=0x7ffb1000cc20 comp rx=0 tx=0).stop 2024-04-25T09:57:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.738+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/520998929 shutdown_connections 2024-04-25T09:57:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.738+0000 7ffb1ffc1640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb1815ac40 0x7ffb18152e40 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.738+0000 7ffb1ffc1640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffafc06b4d0 0x7ffafc06d990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.738+0000 7ffb1ffc1640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb1815a860 0x7ffb180ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.738+0000 7ffb1ffc1640 1 --2- 172.21.15.139:0/520998929 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffb180cd1f0 0x7ffb1814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.738+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/520998929 >> 172.21.15.139:0/520998929 conn(0x7ffb18066f20 msgr2=0x7ffb18156fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.738+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/520998929 shutdown_connections 2024-04-25T09:57:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.738+0000 7ffb1ffc1640 1 -- 172.21.15.139:0/520998929 wait complete. 2024-04-25T09:57:23.753 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:57:23.753 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:57:23.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.848+0000 7f8a596c4640 1 Processor -- start 2024-04-25T09:57:23.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.848+0000 7f8a596c4640 1 -- start start 2024-04-25T09:57:23.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a596c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a540d1080 0x7f8a540b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a596c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8a54161850 0x7f8a540ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a596c4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a540cf250 con 0x7f8a5415dc60 2024-04-25T09:57:23.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a596c4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a540ce780 con 0x7f8a54161850 2024-04-25T09:57:23.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a596c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a540d3730 con 0x7f8a540d1080 2024-04-25T09:57:23.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a537fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8a54161850 0x7f8a540ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a537fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8a54161850 0x7f8a540ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49972/0 (socket says 172.21.15.139:49972) 2024-04-25T09:57:23.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a537fe640 1 -- 172.21.15.139:0/3094044401 learned_addr learned my addr 172.21.15.139:0/3094044401 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:23.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a4bfff640 1 --2- 172.21.15.139:0/3094044401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a540d1080 0x7f8a540b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a537fe640 1 -- 172.21.15.139:0/3094044401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a540d1080 msgr2=0x7f8a540b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:23.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a537fe640 1 --2- 172.21.15.139:0/3094044401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a540d1080 0x7f8a540b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a537fe640 1 -- 172.21.15.139:0/3094044401 >> v1:172.21.15.150:6789/0 conn(0x7f8a5415dc60 legacy=0x7f8a540d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:23.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.849+0000 7f8a537fe640 1 -- 172.21.15.139:0/3094044401 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a540ce9b0 con 0x7f8a54161850 2024-04-25T09:57:23.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.850+0000 7f8a537fe640 1 --2- 172.21.15.139:0/3094044401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a54161850 0x7f8a540ce240 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009e80 tx=0x7f8a4c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8a5f5df8ce63c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.850+0000 7f8a527fc640 1 -- 172.21.15.139:0/3094044401 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c036070 con 0x7f8a54161850 2024-04-25T09:57:23.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.850+0000 7f8a527fc640 1 -- 172.21.15.139:0/3094044401 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4c031600 con 0x7f8a54161850 2024-04-25T09:57:23.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.850+0000 7f8a527fc640 1 -- 172.21.15.139:0/3094044401 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c0319d0 con 0x7f8a54161850 2024-04-25T09:57:23.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.850+0000 7f8a596c4640 1 -- 172.21.15.139:0/3094044401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a54161850 msgr2=0x7f8a540ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.850+0000 7f8a596c4640 1 --2- 172.21.15.139:0/3094044401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a54161850 0x7f8a540ce240 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009e80 tx=0x7f8a4c02a750 comp rx=0 tx=0).stop 2024-04-25T09:57:23.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.850+0000 7f8a596c4640 1 -- 172.21.15.139:0/3094044401 shutdown_connections 2024-04-25T09:57:23.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.850+0000 7f8a596c4640 1 --2- 172.21.15.139:0/3094044401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a54161850 0x7f8a540ce240 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.850+0000 7f8a596c4640 1 --2- 172.21.15.139:0/3094044401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a540d1080 0x7f8a540b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.850+0000 7f8a596c4640 1 -- 172.21.15.139:0/3094044401 >> 172.21.15.139:0/3094044401 conn(0x7f8a540b5910 msgr2=0x7f8a540b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:23.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.851+0000 7f8a596c4640 1 -- 172.21.15.139:0/3094044401 shutdown_connections 2024-04-25T09:57:23.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.851+0000 7f8a596c4640 1 -- 172.21.15.139:0/3094044401 wait complete. 2024-04-25T09:57:23.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.852+0000 7f8a596c4640 1 Processor -- start 2024-04-25T09:57:23.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.852+0000 7f8a596c4640 1 -- start start 2024-04-25T09:57:23.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.852+0000 7f8a596c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a540d1080 0x7f8a54150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.852+0000 7f8a596c4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a5415dc60 0x7f8a541e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.852+0000 7f8a596c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a54161850 0x7f8a541e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.852+0000 7f8a596c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a540d1fc0 con 0x7f8a540d1080 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.852+0000 7f8a596c4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a540cef70 con 0x7f8a5415dc60 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.852+0000 7f8a596c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5415e040 con 0x7f8a54161850 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.852+0000 7f8a52ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a540d1080 0x7f8a54150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.852+0000 7f8a52ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a540d1080 0x7f8a54150850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49984/0 (socket says 172.21.15.139:49984) 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.852+0000 7f8a52ffd640 1 -- 172.21.15.139:0/1198624570 learned_addr learned my addr 172.21.15.139:0/1198624570 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.853+0000 7f8a52ffd640 1 -- 172.21.15.139:0/1198624570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a54161850 msgr2=0x7f8a541e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.853+0000 7f8a52ffd640 1 --2- 172.21.15.139:0/1198624570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a54161850 0x7f8a541e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.853+0000 7f8a52ffd640 1 -- 172.21.15.139:0/1198624570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a5415dc60 msgr2=0x7f8a541e7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.853+0000 7f8a52ffd640 1 --2- 172.21.15.139:0/1198624570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a5415dc60 0x7f8a541e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.853+0000 7f8a52ffd640 1 -- 172.21.15.139:0/1198624570 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a541e86f0 con 0x7f8a540d1080 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.853+0000 7f8a52ffd640 1 --2- 172.21.15.139:0/1198624570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a540d1080 0x7f8a54150850 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400c400 tx=0x7f8a4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.853+0000 7f8a4b7fe640 1 -- 172.21.15.139:0/1198624570 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a440052d0 con 0x7f8a540d1080 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.853+0000 7f8a4b7fe640 1 -- 172.21.15.139:0/1198624570 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a44005490 con 0x7f8a540d1080 2024-04-25T09:57:23.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.853+0000 7f8a596c4640 1 -- 172.21.15.139:0/1198624570 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a541e89d0 con 0x7f8a540d1080 2024-04-25T09:57:23.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.853+0000 7f8a4b7fe640 1 -- 172.21.15.139:0/1198624570 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a44007260 con 0x7f8a540d1080 2024-04-25T09:57:23.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.853+0000 7f8a596c4640 1 -- 172.21.15.139:0/1198624570 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a54151120 con 0x7f8a540d1080 2024-04-25T09:57:23.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.854+0000 7f8a596c4640 1 -- 172.21.15.139:0/1198624570 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a54074520 con 0x7f8a540d1080 2024-04-25T09:57:23.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.854+0000 7f8a4b7fe640 1 -- 172.21.15.139:0/1198624570 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a44007400 con 0x7f8a540d1080 2024-04-25T09:57:23.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.855+0000 7f8a4b7fe640 1 --2- 172.21.15.139:0/1198624570 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8a3406b4d0 0x7f8a3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:23.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.855+0000 7f8a4bfff640 1 --2- 172.21.15.139:0/1198624570 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8a3406b4d0 0x7f8a3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:23.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.855+0000 7f8a4b7fe640 1 -- 172.21.15.139:0/1198624570 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8a44089830 con 0x7f8a540d1080 2024-04-25T09:57:23.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.855+0000 7f8a4bfff640 1 --2- 172.21.15.139:0/1198624570 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8a3406b4d0 0x7f8a3406d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c006fd0 tx=0x7f8a3c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:23.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:23.859+0000 7f8a4b7fe640 1 -- 172.21.15.139:0/1198624570 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a540cf250 con 0x7f8a540d1080 2024-04-25T09:57:24.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.025+0000 7f8a596c4640 1 -- 172.21.15.139:0/1198624570 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a540d16f0 con 0x7f8a3406b4d0 2024-04-25T09:57:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.031+0000 7f8a4b7fe640 1 -- 172.21.15.139:0/1198624570 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252071 (secure 0 0 0) 0x7f8a540d16f0 con 0x7f8a3406b4d0 2024-04-25T09:57:24.037 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:24.038 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:57:24.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.034+0000 7f8a596c4640 1 -- 172.21.15.139:0/1198624570 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8a3406b4d0 msgr2=0x7f8a3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.034+0000 7f8a596c4640 1 --2- 172.21.15.139:0/1198624570 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8a3406b4d0 0x7f8a3406d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c006fd0 tx=0x7f8a3c008040 comp rx=0 tx=0).stop 2024-04-25T09:57:24.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.034+0000 7f8a596c4640 1 -- 172.21.15.139:0/1198624570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a540d1080 msgr2=0x7f8a54150850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.034+0000 7f8a596c4640 1 --2- 172.21.15.139:0/1198624570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a540d1080 0x7f8a54150850 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400c400 tx=0x7f8a4400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.034+0000 7f8a596c4640 1 -- 172.21.15.139:0/1198624570 shutdown_connections 2024-04-25T09:57:24.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.034+0000 7f8a596c4640 1 --2- 172.21.15.139:0/1198624570 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8a3406b4d0 0x7f8a3406d990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.034+0000 7f8a596c4640 1 --2- 172.21.15.139:0/1198624570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a54161850 0x7f8a541e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.034+0000 7f8a596c4640 1 --2- 172.21.15.139:0/1198624570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a5415dc60 0x7f8a541e7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.034+0000 7f8a596c4640 1 --2- 172.21.15.139:0/1198624570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a540d1080 0x7f8a54150850 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.034+0000 7f8a596c4640 1 -- 172.21.15.139:0/1198624570 >> 172.21.15.139:0/1198624570 conn(0x7f8a540b5910 msgr2=0x7f8a5415c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.034+0000 7f8a596c4640 1 -- 172.21.15.139:0/1198624570 shutdown_connections 2024-04-25T09:57:24.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.034+0000 7f8a596c4640 1 -- 172.21.15.139:0/1198624570 wait complete. 2024-04-25T09:57:24.048 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-04-25T09:57:23.022608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237128,"kb_used_data":8464,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754737592,"statfs":{"total":773094113280,"available":772851294208,"internally_reserved":0,"allocated":8667136,"data_stored":4556908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000779"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618596+0000","last_change":"2024-04-25T09:57:21.120454+0000","last_active":"2024-04-25T09:57:22.618596+0000","last_peered":"2024-04-25T09:57:22.618596+0000","last_clean":"2024-04-25T09:57:22.618596+0000","last_became_active":"2024-04-25T09:57:21.120183+0000","last_became_peered":"2024-04-25T09:57:21.120183+0000","last_unstale":"2024-04-25T09:57:22.618596+0000","last_undegraded":"2024-04-25T09:57:22.618596+0000","last_fullsized":"2024-04-25T09:57:22.618596+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:44.986065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113720+0000","last_change":"2024-04-25T09:57:21.117564+0000","last_active":"2024-04-25T09:57:22.113720+0000","last_peered":"2024-04-25T09:57:22.113720+0000","last_clean":"2024-04-25T09:57:22.113720+0000","last_became_active":"2024-04-25T09:57:21.117237+0000","last_became_peered":"2024-04-25T09:57:21.117237+0000","last_unstale":"2024-04-25T09:57:22.113720+0000","last_undegraded":"2024-04-25T09:57:22.113720+0000","last_fullsized":"2024-04-25T09:57:22.113720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:51.923393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257289+0000","last_change":"2024-04-25T09:57:21.114125+0000","last_active":"2024-04-25T09:57:22.257289+0000","last_peered":"2024-04-25T09:57:22.257289+0000","last_clean":"2024-04-25T09:57:22.257289+0000","last_became_active":"2024-04-25T09:57:21.113376+0000","last_became_peered":"2024-04-25T09:57:21.113376+0000","last_unstale":"2024-04-25T09:57:22.257289+0000","last_undegraded":"2024-04-25T09:57:22.257289+0000","last_fullsized":"2024-04-25T09:57:22.257289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:27.183028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360172+0000","last_change":"2024-04-25T09:57:21.123359+0000","last_active":"2024-04-25T09:57:22.360172+0000","last_peered":"2024-04-25T09:57:22.360172+0000","last_clean":"2024-04-25T09:57:22.360172+0000","last_became_active":"2024-04-25T09:57:21.123051+0000","last_became_peered":"2024-04-25T09:57:21.123051+0000","last_unstale":"2024-04-25T09:57:22.360172+0000","last_undegraded":"2024-04-25T09:57:22.360172+0000","last_fullsized":"2024-04-25T09:57:22.360172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:47.465523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418803+0000","last_change":"2024-04-25T09:57:21.107727+0000","last_active":"2024-04-25T09:57:22.418803+0000","last_peered":"2024-04-25T09:57:22.418803+0000","last_clean":"2024-04-25T09:57:22.418803+0000","last_became_active":"2024-04-25T09:57:21.107444+0000","last_became_peered":"2024-04-25T09:57:21.107444+0000","last_unstale":"2024-04-25T09:57:22.418803+0000","last_undegraded":"2024-04-25T09:57:22.418803+0000","last_fullsized":"2024-04-25T09:57:22.418803+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:21.129618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617759+0000","last_change":"2024-04-25T09:57:21.121964+0000","last_active":"2024-04-25T09:57:22.617759+0000","last_peered":"2024-04-25T09:57:22.617759+0000","last_clean":"2024-04-25T09:57:22.617759+0000","last_became_active":"2024-04-25T09:57:21.121655+0000","last_became_peered":"2024-04-25T09:57:21.121655+0000","last_unstale":"2024-04-25T09:57:22.617759+0000","last_undegraded":"2024-04-25T09:57:22.617759+0000","last_fullsized":"2024-04-25T09:57:22.617759+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:58.172060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137349+0000","last_change":"2024-04-25T09:57:21.107916+0000","last_active":"2024-04-25T09:57:22.137349+0000","last_peered":"2024-04-25T09:57:22.137349+0000","last_clean":"2024-04-25T09:57:22.137349+0000","last_became_active":"2024-04-25T09:57:21.107633+0000","last_became_peered":"2024-04-25T09:57:21.107633+0000","last_unstale":"2024-04-25T09:57:22.137349+0000","last_undegraded":"2024-04-25T09:57:22.137349+0000","last_fullsized":"2024-04-25T09:57:22.137349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:49.691316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418557+0000","last_change":"2024-04-25T09:57:21.108364+0000","last_active":"2024-04-25T09:57:22.418557+0000","last_peered":"2024-04-25T09:57:22.418557+0000","last_clean":"2024-04-25T09:57:22.418557+0000","last_became_active":"2024-04-25T09:57:21.108086+0000","last_became_peered":"2024-04-25T09:57:21.108086+0000","last_unstale":"2024-04-25T09:57:22.418557+0000","last_undegraded":"2024-04-25T09:57:22.418557+0000","last_fullsized":"2024-04-25T09:57:22.418557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:42.296165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.110961+0000","last_change":"2024-04-25T09:57:21.120098+0000","last_active":"2024-04-25T09:57:22.110961+0000","last_peered":"2024-04-25T09:57:22.110961+0000","last_clean":"2024-04-25T09:57:22.110961+0000","last_became_active":"2024-04-25T09:57:21.119798+0000","last_became_peered":"2024-04-25T09:57:21.119798+0000","last_unstale":"2024-04-25T09:57:22.110961+0000","last_undegraded":"2024-04-25T09:57:22.110961+0000","last_fullsized":"2024-04-25T09:57:22.110961+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:14.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360171+0000","last_change":"2024-04-25T09:57:21.110790+0000","last_active":"2024-04-25T09:57:22.360171+0000","last_peered":"2024-04-25T09:57:22.360171+0000","last_clean":"2024-04-25T09:57:22.360171+0000","last_became_active":"2024-04-25T09:57:21.110488+0000","last_became_peered":"2024-04-25T09:57:21.110488+0000","last_unstale":"2024-04-25T09:57:22.360171+0000","last_undegraded":"2024-04-25T09:57:22.360171+0000","last_fullsized":"2024-04-25T09:57:22.360171+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:46.896264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113707+0000","last_change":"2024-04-25T09:57:21.117563+0000","last_active":"2024-04-25T09:57:22.113707+0000","last_peered":"2024-04-25T09:57:22.113707+0000","last_clean":"2024-04-25T09:57:22.113707+0000","last_became_active":"2024-04-25T09:57:21.117228+0000","last_became_peered":"2024-04-25T09:57:21.117228+0000","last_unstale":"2024-04-25T09:57:22.113707+0000","last_undegraded":"2024-04-25T09:57:22.113707+0000","last_fullsized":"2024-04-25T09:57:22.113707+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:15.380983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360213+0000","last_change":"2024-04-25T09:57:21.112520+0000","last_active":"2024-04-25T09:57:22.360213+0000","last_peered":"2024-04-25T09:57:22.360213+0000","last_clean":"2024-04-25T09:57:22.360213+0000","last_became_active":"2024-04-25T09:57:21.112165+0000","last_became_peered":"2024-04-25T09:57:21.112165+0000","last_unstale":"2024-04-25T09:57:22.360213+0000","last_undegraded":"2024-04-25T09:57:22.360213+0000","last_fullsized":"2024-04-25T09:57:22.360213+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:33.042997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418857+0000","last_change":"2024-04-25T09:57:21.111137+0000","last_active":"2024-04-25T09:57:22.418857+0000","last_peered":"2024-04-25T09:57:22.418857+0000","last_clean":"2024-04-25T09:57:22.418857+0000","last_became_active":"2024-04-25T09:57:21.110858+0000","last_became_peered":"2024-04-25T09:57:21.110858+0000","last_unstale":"2024-04-25T09:57:22.418857+0000","last_undegraded":"2024-04-25T09:57:22.418857+0000","last_fullsized":"2024-04-25T09:57:22.418857+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:13.964094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617979+0000","last_change":"2024-04-25T09:57:21.120742+0000","last_active":"2024-04-25T09:57:22.617979+0000","last_peered":"2024-04-25T09:57:22.617979+0000","last_clean":"2024-04-25T09:57:22.617979+0000","last_became_active":"2024-04-25T09:57:21.120446+0000","last_became_peered":"2024-04-25T09:57:21.120446+0000","last_unstale":"2024-04-25T09:57:22.617979+0000","last_undegraded":"2024-04-25T09:57:22.617979+0000","last_fullsized":"2024-04-25T09:57:22.617979+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:44.826600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113362+0000","last_change":"2024-04-25T09:57:21.109684+0000","last_active":"2024-04-25T09:57:22.113362+0000","last_peered":"2024-04-25T09:57:22.113362+0000","last_clean":"2024-04-25T09:57:22.113362+0000","last_became_active":"2024-04-25T09:57:21.109419+0000","last_became_peered":"2024-04-25T09:57:21.109419+0000","last_unstale":"2024-04-25T09:57:22.113362+0000","last_undegraded":"2024-04-25T09:57:22.113362+0000","last_fullsized":"2024-04-25T09:57:22.113362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:53.001563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618434+0000","last_change":"2024-04-25T09:57:21.112431+0000","last_active":"2024-04-25T09:57:22.618434+0000","last_peered":"2024-04-25T09:57:22.618434+0000","last_clean":"2024-04-25T09:57:22.618434+0000","last_became_active":"2024-04-25T09:57:21.112158+0000","last_became_peered":"2024-04-25T09:57:21.112158+0000","last_unstale":"2024-04-25T09:57:22.618434+0000","last_undegraded":"2024-04-25T09:57:22.618434+0000","last_fullsized":"2024-04-25T09:57:22.618434+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:22.651722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418971+0000","last_change":"2024-04-25T09:57:21.113119+0000","last_active":"2024-04-25T09:57:22.418971+0000","last_peered":"2024-04-25T09:57:22.418971+0000","last_clean":"2024-04-25T09:57:22.418971+0000","last_became_active":"2024-04-25T09:57:21.112870+0000","last_became_peered":"2024-04-25T09:57:21.112870+0000","last_unstale":"2024-04-25T09:57:22.418971+0000","last_undegraded":"2024-04-25T09:57:22.418971+0000","last_fullsized":"2024-04-25T09:57:22.418971+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:56.255054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257312+0000","last_change":"2024-04-25T09:57:21.120220+0000","last_active":"2024-04-25T09:57:22.257312+0000","last_peered":"2024-04-25T09:57:22.257312+0000","last_clean":"2024-04-25T09:57:22.257312+0000","last_became_active":"2024-04-25T09:57:21.119961+0000","last_became_peered":"2024-04-25T09:57:21.119961+0000","last_unstale":"2024-04-25T09:57:22.257312+0000","last_undegraded":"2024-04-25T09:57:22.257312+0000","last_fullsized":"2024-04-25T09:57:22.257312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:49.691316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617938+0000","last_change":"2024-04-25T09:57:21.114955+0000","last_active":"2024-04-25T09:57:22.617938+0000","last_peered":"2024-04-25T09:57:22.617938+0000","last_clean":"2024-04-25T09:57:22.617938+0000","last_became_active":"2024-04-25T09:57:21.114678+0000","last_became_peered":"2024-04-25T09:57:21.114678+0000","last_unstale":"2024-04-25T09:57:22.617938+0000","last_undegraded":"2024-04-25T09:57:22.617938+0000","last_fullsized":"2024-04-25T09:57:22.617938+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:14.135232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418983+0000","last_change":"2024-04-25T09:57:21.104652+0000","last_active":"2024-04-25T09:57:22.418983+0000","last_peered":"2024-04-25T09:57:22.418983+0000","last_clean":"2024-04-25T09:57:22.418983+0000","last_became_active":"2024-04-25T09:57:21.104382+0000","last_became_peered":"2024-04-25T09:57:21.104382+0000","last_unstale":"2024-04-25T09:57:22.418983+0000","last_undegraded":"2024-04-25T09:57:22.418983+0000","last_fullsized":"2024-04-25T09:57:22.418983+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:32.090384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.111106+0000","last_change":"2024-04-25T09:57:21.122880+0000","last_active":"2024-04-25T09:57:22.111106+0000","last_peered":"2024-04-25T09:57:22.111106+0000","last_clean":"2024-04-25T09:57:22.111106+0000","last_became_active":"2024-04-25T09:57:21.122555+0000","last_became_peered":"2024-04-25T09:57:21.122555+0000","last_unstale":"2024-04-25T09:57:22.111106+0000","last_undegraded":"2024-04-25T09:57:22.111106+0000","last_fullsized":"2024-04-25T09:57:22.111106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:06.392815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418650+0000","last_change":"2024-04-25T09:57:21.111844+0000","last_active":"2024-04-25T09:57:22.418650+0000","last_peered":"2024-04-25T09:57:22.418650+0000","last_clean":"2024-04-25T09:57:22.418650+0000","last_became_active":"2024-04-25T09:57:21.111542+0000","last_became_peered":"2024-04-25T09:57:21.111542+0000","last_unstale":"2024-04-25T09:57:22.418650+0000","last_undegraded":"2024-04-25T09:57:22.418650+0000","last_fullsized":"2024-04-25T09:57:22.418650+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:57.981700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.110801+0000","last_change":"2024-04-25T09:57:21.115645+0000","last_active":"2024-04-25T09:57:22.110801+0000","last_peered":"2024-04-25T09:57:22.110801+0000","last_clean":"2024-04-25T09:57:22.110801+0000","last_became_active":"2024-04-25T09:57:21.115370+0000","last_became_peered":"2024-04-25T09:57:21.115370+0000","last_unstale":"2024-04-25T09:57:22.110801+0000","last_undegraded":"2024-04-25T09:57:22.110801+0000","last_fullsized":"2024-04-25T09:57:22.110801+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:26.191533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113349+0000","last_change":"2024-04-25T09:57:21.111467+0000","last_active":"2024-04-25T09:57:22.113349+0000","last_peered":"2024-04-25T09:57:22.113349+0000","last_clean":"2024-04-25T09:57:22.113349+0000","last_became_active":"2024-04-25T09:57:21.111189+0000","last_became_peered":"2024-04-25T09:57:21.111189+0000","last_unstale":"2024-04-25T09:57:22.113349+0000","last_undegraded":"2024-04-25T09:57:22.113349+0000","last_fullsized":"2024-04-25T09:57:22.113349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:14.897617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113535+0000","last_change":"2024-04-25T09:57:21.111492+0000","last_active":"2024-04-25T09:57:22.113535+0000","last_peered":"2024-04-25T09:57:22.113535+0000","last_clean":"2024-04-25T09:57:22.113535+0000","last_became_active":"2024-04-25T09:57:21.111218+0000","last_became_peered":"2024-04-25T09:57:21.111218+0000","last_unstale":"2024-04-25T09:57:22.113535+0000","last_undegraded":"2024-04-25T09:57:22.113535+0000","last_fullsized":"2024-04-25T09:57:22.113535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:58.172060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360171+0000","last_change":"2024-04-25T09:57:21.115655+0000","last_active":"2024-04-25T09:57:22.360171+0000","last_peered":"2024-04-25T09:57:22.360171+0000","last_clean":"2024-04-25T09:57:22.360171+0000","last_became_active":"2024-04-25T09:57:21.115405+0000","last_became_peered":"2024-04-25T09:57:21.115405+0000","last_unstale":"2024-04-25T09:57:22.360171+0000","last_undegraded":"2024-04-25T09:57:22.360171+0000","last_fullsized":"2024-04-25T09:57:22.360171+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:43.604808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618411+0000","last_change":"2024-04-25T09:57:21.111294+0000","last_active":"2024-04-25T09:57:22.618411+0000","last_peered":"2024-04-25T09:57:22.618411+0000","last_clean":"2024-04-25T09:57:22.618411+0000","last_became_active":"2024-04-25T09:57:21.111039+0000","last_became_peered":"2024-04-25T09:57:21.111039+0000","last_unstale":"2024-04-25T09:57:22.618411+0000","last_undegraded":"2024-04-25T09:57:22.618411+0000","last_fullsized":"2024-04-25T09:57:22.618411+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:51.923393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418997+0000","last_change":"2024-04-25T09:57:21.105983+0000","last_active":"2024-04-25T09:57:22.418997+0000","last_peered":"2024-04-25T09:57:22.418997+0000","last_clean":"2024-04-25T09:57:22.418997+0000","last_became_active":"2024-04-25T09:57:21.105709+0000","last_became_peered":"2024-04-25T09:57:21.105709+0000","last_unstale":"2024-04-25T09:57:22.418997+0000","last_undegraded":"2024-04-25T09:57:22.418997+0000","last_fullsized":"2024-04-25T09:57:22.418997+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:07.746769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113502+0000","last_change":"2024-04-25T09:57:21.116340+0000","last_active":"2024-04-25T09:57:22.113502+0000","last_peered":"2024-04-25T09:57:22.113502+0000","last_clean":"2024-04-25T09:57:22.113502+0000","last_became_active":"2024-04-25T09:57:21.116047+0000","last_became_peered":"2024-04-25T09:57:21.116047+0000","last_unstale":"2024-04-25T09:57:22.113502+0000","last_undegraded":"2024-04-25T09:57:22.113502+0000","last_fullsized":"2024-04-25T09:57:22.113502+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:27.828672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618209+0000","last_change":"2024-04-25T09:57:21.112039+0000","last_active":"2024-04-25T09:57:22.618209+0000","last_peered":"2024-04-25T09:57:22.618209+0000","last_clean":"2024-04-25T09:57:22.618209+0000","last_became_active":"2024-04-25T09:57:21.111773+0000","last_became_peered":"2024-04-25T09:57:21.111773+0000","last_unstale":"2024-04-25T09:57:22.618209+0000","last_undegraded":"2024-04-25T09:57:22.618209+0000","last_fullsized":"2024-04-25T09:57:22.618209+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:44.826600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257098+0000","last_change":"2024-04-25T09:57:21.112364+0000","last_active":"2024-04-25T09:57:22.257098+0000","last_peered":"2024-04-25T09:57:22.257098+0000","last_clean":"2024-04-25T09:57:22.257098+0000","last_became_active":"2024-04-25T09:57:21.112083+0000","last_became_peered":"2024-04-25T09:57:21.112083+0000","last_unstale":"2024-04-25T09:57:22.257098+0000","last_undegraded":"2024-04-25T09:57:22.257098+0000","last_fullsized":"2024-04-25T09:57:22.257098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:11.795871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618312+0000","last_change":"2024-04-25T09:57:21.121149+0000","last_active":"2024-04-25T09:57:22.618312+0000","last_peered":"2024-04-25T09:57:22.618312+0000","last_clean":"2024-04-25T09:57:22.618312+0000","last_became_active":"2024-04-25T09:57:21.120575+0000","last_became_peered":"2024-04-25T09:57:21.120575+0000","last_unstale":"2024-04-25T09:57:22.618312+0000","last_undegraded":"2024-04-25T09:57:22.618312+0000","last_fullsized":"2024-04-25T09:57:22.618312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:15.380983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257161+0000","last_change":"2024-04-25T09:57:21.119784+0000","last_active":"2024-04-25T09:57:22.257161+0000","last_peered":"2024-04-25T09:57:22.257161+0000","last_clean":"2024-04-25T09:57:22.257161+0000","last_became_active":"2024-04-25T09:57:21.119516+0000","last_became_peered":"2024-04-25T09:57:21.119516+0000","last_unstale":"2024-04-25T09:57:22.257161+0000","last_undegraded":"2024-04-25T09:57:22.257161+0000","last_fullsized":"2024-04-25T09:57:22.257161+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:28.602725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"75'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418901+0000","last_change":"2024-04-25T09:57:20.102637+0000","last_active":"2024-04-25T09:57:22.418901+0000","last_peered":"2024-04-25T09:57:22.418901+0000","last_clean":"2024-04-25T09:57:22.418901+0000","last_became_active":"2024-04-25T09:57:20.102355+0000","last_became_peered":"2024-04-25T09:57:20.102355+0000","last_unstale":"2024-04-25T09:57:22.418901+0000","last_undegraded":"2024-04-25T09:57:22.418901+0000","last_fullsized":"2024-04-25T09:57:22.418901+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:26.062409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113711+0000","last_change":"2024-04-25T09:57:21.110169+0000","last_active":"2024-04-25T09:57:22.113711+0000","last_peered":"2024-04-25T09:57:22.113711+0000","last_clean":"2024-04-25T09:57:22.113711+0000","last_became_active":"2024-04-25T09:57:21.109878+0000","last_became_peered":"2024-04-25T09:57:21.109878+0000","last_unstale":"2024-04-25T09:57:22.113711+0000","last_undegraded":"2024-04-25T09:57:22.113711+0000","last_fullsized":"2024-04-25T09:57:22.113711+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:22.651722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618449+0000","last_change":"2024-04-25T09:57:20.103923+0000","last_active":"2024-04-25T09:57:22.618449+0000","last_peered":"2024-04-25T09:57:22.618449+0000","last_clean":"2024-04-25T09:57:22.618449+0000","last_became_active":"2024-04-25T09:57:20.103647+0000","last_became_peered":"2024-04-25T09:57:20.103647+0000","last_unstale":"2024-04-25T09:57:22.618449+0000","last_undegraded":"2024-04-25T09:57:22.618449+0000","last_fullsized":"2024-04-25T09:57:22.618449+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:34.401685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.111100+0000","last_change":"2024-04-25T09:57:21.118426+0000","last_active":"2024-04-25T09:57:22.111100+0000","last_peered":"2024-04-25T09:57:22.111100+0000","last_clean":"2024-04-25T09:57:22.111100+0000","last_became_active":"2024-04-25T09:57:21.118107+0000","last_became_peered":"2024-04-25T09:57:21.118107+0000","last_unstale":"2024-04-25T09:57:22.111100+0000","last_undegraded":"2024-04-25T09:57:22.111100+0000","last_fullsized":"2024-04-25T09:57:22.111100+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:50.756060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618388+0000","last_change":"2024-04-25T09:57:20.109590+0000","last_active":"2024-04-25T09:57:22.618388+0000","last_peered":"2024-04-25T09:57:22.618388+0000","last_clean":"2024-04-25T09:57:22.618388+0000","last_became_active":"2024-04-25T09:57:20.109336+0000","last_became_peered":"2024-04-25T09:57:20.109336+0000","last_unstale":"2024-04-25T09:57:22.618388+0000","last_undegraded":"2024-04-25T09:57:22.618388+0000","last_fullsized":"2024-04-25T09:57:22.618388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:24.866748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.110973+0000","last_change":"2024-04-25T09:57:21.122813+0000","last_active":"2024-04-25T09:57:22.110973+0000","last_peered":"2024-04-25T09:57:22.110973+0000","last_clean":"2024-04-25T09:57:22.110973+0000","last_became_active":"2024-04-25T09:57:21.122187+0000","last_became_peered":"2024-04-25T09:57:21.122187+0000","last_unstale":"2024-04-25T09:57:22.110973+0000","last_undegraded":"2024-04-25T09:57:22.110973+0000","last_fullsized":"2024-04-25T09:57:22.110973+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:03.360678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137303+0000","last_change":"2024-04-25T09:57:20.112199+0000","last_active":"2024-04-25T09:57:22.137303+0000","last_peered":"2024-04-25T09:57:22.137303+0000","last_clean":"2024-04-25T09:57:22.137303+0000","last_became_active":"2024-04-25T09:57:20.111907+0000","last_became_peered":"2024-04-25T09:57:20.111907+0000","last_unstale":"2024-04-25T09:57:22.137303+0000","last_undegraded":"2024-04-25T09:57:22.137303+0000","last_fullsized":"2024-04-25T09:57:22.137303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:40.105458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113599+0000","last_change":"2024-04-25T09:57:21.112913+0000","last_active":"2024-04-25T09:57:22.113599+0000","last_peered":"2024-04-25T09:57:22.113599+0000","last_clean":"2024-04-25T09:57:22.113599+0000","last_became_active":"2024-04-25T09:57:21.112471+0000","last_became_peered":"2024-04-25T09:57:21.112471+0000","last_unstale":"2024-04-25T09:57:22.113599+0000","last_undegraded":"2024-04-25T09:57:22.113599+0000","last_fullsized":"2024-04-25T09:57:22.113599+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:23.671368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.111085+0000","last_change":"2024-04-25T09:57:20.106997+0000","last_active":"2024-04-25T09:57:22.111085+0000","last_peered":"2024-04-25T09:57:22.111085+0000","last_clean":"2024-04-25T09:57:22.111085+0000","last_became_active":"2024-04-25T09:57:20.106697+0000","last_became_peered":"2024-04-25T09:57:20.106697+0000","last_unstale":"2024-04-25T09:57:22.111085+0000","last_undegraded":"2024-04-25T09:57:22.111085+0000","last_fullsized":"2024-04-25T09:57:22.111085+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:06.051591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113772+0000","last_change":"2024-04-25T09:57:21.117351+0000","last_active":"2024-04-25T09:57:22.113772+0000","last_peered":"2024-04-25T09:57:22.113772+0000","last_clean":"2024-04-25T09:57:22.113772+0000","last_became_active":"2024-04-25T09:57:21.117064+0000","last_became_peered":"2024-04-25T09:57:21.117064+0000","last_unstale":"2024-04-25T09:57:22.113772+0000","last_undegraded":"2024-04-25T09:57:22.113772+0000","last_fullsized":"2024-04-25T09:57:22.113772+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:44.986065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.359922+0000","last_change":"2024-04-25T09:57:20.110505+0000","last_active":"2024-04-25T09:57:22.359922+0000","last_peered":"2024-04-25T09:57:22.359922+0000","last_clean":"2024-04-25T09:57:22.359922+0000","last_became_active":"2024-04-25T09:57:20.110230+0000","last_became_peered":"2024-04-25T09:57:20.110230+0000","last_unstale":"2024-04-25T09:57:22.359922+0000","last_undegraded":"2024-04-25T09:57:22.359922+0000","last_fullsized":"2024-04-25T09:57:22.359922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:39.225413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.111193+0000","last_change":"2024-04-25T09:57:21.118869+0000","last_active":"2024-04-25T09:57:22.111193+0000","last_peered":"2024-04-25T09:57:22.111193+0000","last_clean":"2024-04-25T09:57:22.111193+0000","last_became_active":"2024-04-25T09:57:21.118504+0000","last_became_peered":"2024-04-25T09:57:21.118504+0000","last_unstale":"2024-04-25T09:57:22.111193+0000","last_undegraded":"2024-04-25T09:57:22.111193+0000","last_fullsized":"2024-04-25T09:57:22.111193+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:11.795871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113454+0000","last_change":"2024-04-25T09:57:20.110833+0000","last_active":"2024-04-25T09:57:22.113454+0000","last_peered":"2024-04-25T09:57:22.113454+0000","last_clean":"2024-04-25T09:57:22.113454+0000","last_became_active":"2024-04-25T09:57:20.110567+0000","last_became_peered":"2024-04-25T09:57:20.110567+0000","last_unstale":"2024-04-25T09:57:22.113454+0000","last_undegraded":"2024-04-25T09:57:22.113454+0000","last_fullsized":"2024-04-25T09:57:22.113454+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:30.224542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137668+0000","last_change":"2024-04-25T09:57:21.115420+0000","last_active":"2024-04-25T09:57:22.137668+0000","last_peered":"2024-04-25T09:57:22.137668+0000","last_clean":"2024-04-25T09:57:22.137668+0000","last_became_active":"2024-04-25T09:57:21.114957+0000","last_became_peered":"2024-04-25T09:57:21.114957+0000","last_unstale":"2024-04-25T09:57:22.137668+0000","last_undegraded":"2024-04-25T09:57:22.137668+0000","last_fullsized":"2024-04-25T09:57:22.137668+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:23.467346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418698+0000","last_change":"2024-04-25T09:57:20.101814+0000","last_active":"2024-04-25T09:57:22.418698+0000","last_peered":"2024-04-25T09:57:22.418698+0000","last_clean":"2024-04-25T09:57:22.418698+0000","last_became_active":"2024-04-25T09:57:20.101558+0000","last_became_peered":"2024-04-25T09:57:20.101558+0000","last_unstale":"2024-04-25T09:57:22.418698+0000","last_undegraded":"2024-04-25T09:57:22.418698+0000","last_fullsized":"2024-04-25T09:57:22.418698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:16.385636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113814+0000","last_change":"2024-04-25T09:57:21.110914+0000","last_active":"2024-04-25T09:57:22.113814+0000","last_peered":"2024-04-25T09:57:22.113814+0000","last_clean":"2024-04-25T09:57:22.113814+0000","last_became_active":"2024-04-25T09:57:21.110604+0000","last_became_peered":"2024-04-25T09:57:21.110604+0000","last_unstale":"2024-04-25T09:57:22.113814+0000","last_undegraded":"2024-04-25T09:57:22.113814+0000","last_fullsized":"2024-04-25T09:57:22.113814+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:00.063570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618579+0000","last_change":"2024-04-25T09:57:20.110007+0000","last_active":"2024-04-25T09:57:22.618579+0000","last_peered":"2024-04-25T09:57:22.618579+0000","last_clean":"2024-04-25T09:57:22.618579+0000","last_became_active":"2024-04-25T09:57:20.109695+0000","last_became_peered":"2024-04-25T09:57:20.109695+0000","last_unstale":"2024-04-25T09:57:22.618579+0000","last_undegraded":"2024-04-25T09:57:22.618579+0000","last_fullsized":"2024-04-25T09:57:22.618579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:45.891118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360084+0000","last_change":"2024-04-25T09:57:21.119640+0000","last_active":"2024-04-25T09:57:22.360084+0000","last_peered":"2024-04-25T09:57:22.360084+0000","last_clean":"2024-04-25T09:57:22.360084+0000","last_became_active":"2024-04-25T09:57:21.119306+0000","last_became_peered":"2024-04-25T09:57:21.119306+0000","last_unstale":"2024-04-25T09:57:22.360084+0000","last_undegraded":"2024-04-25T09:57:22.360084+0000","last_fullsized":"2024-04-25T09:57:22.360084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:25.871895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360288+0000","last_change":"2024-04-25T09:57:20.113310+0000","last_active":"2024-04-25T09:57:22.360288+0000","last_peered":"2024-04-25T09:57:22.360288+0000","last_clean":"2024-04-25T09:57:22.360288+0000","last_became_active":"2024-04-25T09:57:20.113015+0000","last_became_peered":"2024-04-25T09:57:20.113015+0000","last_unstale":"2024-04-25T09:57:22.360288+0000","last_undegraded":"2024-04-25T09:57:22.360288+0000","last_fullsized":"2024-04-25T09:57:22.360288+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:16.759726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618050+0000","last_change":"2024-04-25T09:57:21.107081+0000","last_active":"2024-04-25T09:57:22.618050+0000","last_peered":"2024-04-25T09:57:22.618050+0000","last_clean":"2024-04-25T09:57:22.618050+0000","last_became_active":"2024-04-25T09:57:21.105586+0000","last_became_peered":"2024-04-25T09:57:21.105586+0000","last_unstale":"2024-04-25T09:57:22.618050+0000","last_undegraded":"2024-04-25T09:57:22.618050+0000","last_fullsized":"2024-04-25T09:57:22.618050+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:15.380983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"75'3","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418957+0000","last_change":"2024-04-25T09:57:20.109564+0000","last_active":"2024-04-25T09:57:22.418957+0000","last_peered":"2024-04-25T09:57:22.418957+0000","last_clean":"2024-04-25T09:57:22.418957+0000","last_became_active":"2024-04-25T09:57:20.109298+0000","last_became_peered":"2024-04-25T09:57:20.109298+0000","last_unstale":"2024-04-25T09:57:22.418957+0000","last_undegraded":"2024-04-25T09:57:22.418957+0000","last_fullsized":"2024-04-25T09:57:22.418957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:46.611507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257423+0000","last_change":"2024-04-25T09:57:21.114715+0000","last_active":"2024-04-25T09:57:22.257423+0000","last_peered":"2024-04-25T09:57:22.257423+0000","last_clean":"2024-04-25T09:57:22.257423+0000","last_became_active":"2024-04-25T09:57:21.113989+0000","last_became_peered":"2024-04-25T09:57:21.113989+0000","last_unstale":"2024-04-25T09:57:22.257423+0000","last_undegraded":"2024-04-25T09:57:22.257423+0000","last_fullsized":"2024-04-25T09:57:22.257423+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:28.226431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"75'6","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137419+0000","last_change":"2024-04-25T09:57:20.112803+0000","last_active":"2024-04-25T09:57:22.137419+0000","last_peered":"2024-04-25T09:57:22.137419+0000","last_clean":"2024-04-25T09:57:22.137419+0000","last_became_active":"2024-04-25T09:57:20.112522+0000","last_became_peered":"2024-04-25T09:57:20.112522+0000","last_unstale":"2024-04-25T09:57:22.137419+0000","last_undegraded":"2024-04-25T09:57:22.137419+0000","last_fullsized":"2024-04-25T09:57:22.137419+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:08.519297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360288+0000","last_change":"2024-04-25T09:57:21.113149+0000","last_active":"2024-04-25T09:57:22.360288+0000","last_peered":"2024-04-25T09:57:22.360288+0000","last_clean":"2024-04-25T09:57:22.360288+0000","last_became_active":"2024-04-25T09:57:21.112750+0000","last_became_peered":"2024-04-25T09:57:21.112750+0000","last_unstale":"2024-04-25T09:57:22.360288+0000","last_undegraded":"2024-04-25T09:57:22.360288+0000","last_fullsized":"2024-04-25T09:57:22.360288+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:14.897617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.359928+0000","last_change":"2024-04-25T09:57:20.108560+0000","last_active":"2024-04-25T09:57:22.359928+0000","last_peered":"2024-04-25T09:57:22.359928+0000","last_clean":"2024-04-25T09:57:22.359928+0000","last_became_active":"2024-04-25T09:57:20.108279+0000","last_became_peered":"2024-04-25T09:57:20.108279+0000","last_unstale":"2024-04-25T09:57:22.359928+0000","last_undegraded":"2024-04-25T09:57:22.359928+0000","last_fullsized":"2024-04-25T09:57:22.359928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:22.462199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137463+0000","last_change":"2024-04-25T09:57:21.108479+0000","last_active":"2024-04-25T09:57:22.137463+0000","last_peered":"2024-04-25T09:57:22.137463+0000","last_clean":"2024-04-25T09:57:22.137463+0000","last_became_active":"2024-04-25T09:57:21.108155+0000","last_became_peered":"2024-04-25T09:57:21.108155+0000","last_unstale":"2024-04-25T09:57:22.137463+0000","last_undegraded":"2024-04-25T09:57:22.137463+0000","last_fullsized":"2024-04-25T09:57:22.137463+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:43.569154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618200+0000","last_change":"2024-04-25T09:57:20.106999+0000","last_active":"2024-04-25T09:57:22.618200+0000","last_peered":"2024-04-25T09:57:22.618200+0000","last_clean":"2024-04-25T09:57:22.618200+0000","last_became_active":"2024-04-25T09:57:20.106088+0000","last_became_peered":"2024-04-25T09:57:20.106088+0000","last_unstale":"2024-04-25T09:57:22.618200+0000","last_undegraded":"2024-04-25T09:57:22.618200+0000","last_fullsized":"2024-04-25T09:57:22.618200+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:54.488223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137576+0000","last_change":"2024-04-25T09:57:21.120546+0000","last_active":"2024-04-25T09:57:22.137576+0000","last_peered":"2024-04-25T09:57:22.137576+0000","last_clean":"2024-04-25T09:57:22.137576+0000","last_became_active":"2024-04-25T09:57:21.119633+0000","last_became_peered":"2024-04-25T09:57:21.119633+0000","last_unstale":"2024-04-25T09:57:22.137576+0000","last_undegraded":"2024-04-25T09:57:22.137576+0000","last_fullsized":"2024-04-25T09:57:22.137576+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:40.230560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418617+0000","last_change":"2024-04-25T09:57:20.109834+0000","last_active":"2024-04-25T09:57:22.418617+0000","last_peered":"2024-04-25T09:57:22.418617+0000","last_clean":"2024-04-25T09:57:22.418617+0000","last_became_active":"2024-04-25T09:57:20.106429+0000","last_became_peered":"2024-04-25T09:57:20.106429+0000","last_unstale":"2024-04-25T09:57:22.418617+0000","last_undegraded":"2024-04-25T09:57:22.418617+0000","last_fullsized":"2024-04-25T09:57:22.418617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:40.571580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137630+0000","last_change":"2024-04-25T09:57:21.113824+0000","last_active":"2024-04-25T09:57:22.137630+0000","last_peered":"2024-04-25T09:57:22.137630+0000","last_clean":"2024-04-25T09:57:22.137630+0000","last_became_active":"2024-04-25T09:57:21.113539+0000","last_became_peered":"2024-04-25T09:57:21.113539+0000","last_unstale":"2024-04-25T09:57:22.137630+0000","last_undegraded":"2024-04-25T09:57:22.137630+0000","last_fullsized":"2024-04-25T09:57:22.137630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:31.229689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"75'8","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113463+0000","last_change":"2024-04-25T09:57:20.107762+0000","last_active":"2024-04-25T09:57:22.113463+0000","last_peered":"2024-04-25T09:57:22.113463+0000","last_clean":"2024-04-25T09:57:22.113463+0000","last_became_active":"2024-04-25T09:57:20.107415+0000","last_became_peered":"2024-04-25T09:57:20.107415+0000","last_unstale":"2024-04-25T09:57:22.113463+0000","last_undegraded":"2024-04-25T09:57:22.113463+0000","last_fullsized":"2024-04-25T09:57:22.113463+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:39.225413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617743+0000","last_change":"2024-04-25T09:57:21.105218+0000","last_active":"2024-04-25T09:57:22.617743+0000","last_peered":"2024-04-25T09:57:22.617743+0000","last_clean":"2024-04-25T09:57:22.617743+0000","last_became_active":"2024-04-25T09:57:21.104949+0000","last_became_peered":"2024-04-25T09:57:21.104949+0000","last_unstale":"2024-04-25T09:57:22.617743+0000","last_undegraded":"2024-04-25T09:57:22.617743+0000","last_fullsized":"2024-04-25T09:57:22.617743+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:27.828672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618688+0000","last_change":"2024-04-25T09:57:20.110970+0000","last_active":"2024-04-25T09:57:22.618688+0000","last_peered":"2024-04-25T09:57:22.618688+0000","last_clean":"2024-04-25T09:57:22.618688+0000","last_became_active":"2024-04-25T09:57:20.110691+0000","last_became_peered":"2024-04-25T09:57:20.110691+0000","last_unstale":"2024-04-25T09:57:22.618688+0000","last_undegraded":"2024-04-25T09:57:22.618688+0000","last_fullsized":"2024-04-25T09:57:22.618688+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:32.037850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418605+0000","last_change":"2024-04-25T09:57:21.114261+0000","last_active":"2024-04-25T09:57:22.418605+0000","last_peered":"2024-04-25T09:57:22.418605+0000","last_clean":"2024-04-25T09:57:22.418605+0000","last_became_active":"2024-04-25T09:57:21.113925+0000","last_became_peered":"2024-04-25T09:57:21.113925+0000","last_unstale":"2024-04-25T09:57:22.418605+0000","last_undegraded":"2024-04-25T09:57:22.418605+0000","last_fullsized":"2024-04-25T09:57:22.418605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:40.528488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.111219+0000","last_change":"2024-04-25T09:57:20.112382+0000","last_active":"2024-04-25T09:57:22.111219+0000","last_peered":"2024-04-25T09:57:22.111219+0000","last_clean":"2024-04-25T09:57:22.111219+0000","last_became_active":"2024-04-25T09:57:20.112100+0000","last_became_peered":"2024-04-25T09:57:20.112100+0000","last_unstale":"2024-04-25T09:57:22.111219+0000","last_undegraded":"2024-04-25T09:57:22.111219+0000","last_fullsized":"2024-04-25T09:57:22.111219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:52.500111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418687+0000","last_change":"2024-04-25T09:57:21.112450+0000","last_active":"2024-04-25T09:57:22.418687+0000","last_peered":"2024-04-25T09:57:22.418687+0000","last_clean":"2024-04-25T09:57:22.418687+0000","last_became_active":"2024-04-25T09:57:21.112193+0000","last_became_peered":"2024-04-25T09:57:21.112193+0000","last_unstale":"2024-04-25T09:57:22.418687+0000","last_undegraded":"2024-04-25T09:57:22.418687+0000","last_fullsized":"2024-04-25T09:57:22.418687+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:14.017954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.419051+0000","last_change":"2024-04-25T09:57:20.102910+0000","last_active":"2024-04-25T09:57:22.419051+0000","last_peered":"2024-04-25T09:57:22.419051+0000","last_clean":"2024-04-25T09:57:22.419051+0000","last_became_active":"2024-04-25T09:57:20.102635+0000","last_became_peered":"2024-04-25T09:57:20.102635+0000","last_unstale":"2024-04-25T09:57:22.419051+0000","last_undegraded":"2024-04-25T09:57:22.419051+0000","last_fullsized":"2024-04-25T09:57:22.419051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:46.978627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617873+0000","last_change":"2024-04-25T09:57:21.104013+0000","last_active":"2024-04-25T09:57:22.617873+0000","last_peered":"2024-04-25T09:57:22.617873+0000","last_clean":"2024-04-25T09:57:22.617873+0000","last_became_active":"2024-04-25T09:57:21.103636+0000","last_became_peered":"2024-04-25T09:57:21.103636+0000","last_unstale":"2024-04-25T09:57:22.617873+0000","last_undegraded":"2024-04-25T09:57:22.617873+0000","last_fullsized":"2024-04-25T09:57:22.617873+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:23.671368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257184+0000","last_change":"2024-04-25T09:57:20.109793+0000","last_active":"2024-04-25T09:57:22.257184+0000","last_peered":"2024-04-25T09:57:22.257184+0000","last_clean":"2024-04-25T09:57:22.257184+0000","last_became_active":"2024-04-25T09:57:20.109531+0000","last_became_peered":"2024-04-25T09:57:20.109531+0000","last_unstale":"2024-04-25T09:57:22.257184+0000","last_undegraded":"2024-04-25T09:57:22.257184+0000","last_fullsized":"2024-04-25T09:57:22.257184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:40.105458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113613+0000","last_change":"2024-04-25T09:57:21.111171+0000","last_active":"2024-04-25T09:57:22.113613+0000","last_peered":"2024-04-25T09:57:22.113613+0000","last_clean":"2024-04-25T09:57:22.113613+0000","last_became_active":"2024-04-25T09:57:21.110751+0000","last_became_peered":"2024-04-25T09:57:21.110751+0000","last_unstale":"2024-04-25T09:57:22.113613+0000","last_undegraded":"2024-04-25T09:57:22.113613+0000","last_fullsized":"2024-04-25T09:57:22.113613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:14.135232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617931+0000","last_change":"2024-04-25T09:57:20.109957+0000","last_active":"2024-04-25T09:57:22.617931+0000","last_peered":"2024-04-25T09:57:22.617931+0000","last_clean":"2024-04-25T09:57:22.617931+0000","last_became_active":"2024-04-25T09:57:20.109662+0000","last_became_peered":"2024-04-25T09:57:20.109662+0000","last_unstale":"2024-04-25T09:57:22.617931+0000","last_undegraded":"2024-04-25T09:57:22.617931+0000","last_fullsized":"2024-04-25T09:57:22.617931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:54.488223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113807+0000","last_change":"2024-04-25T09:57:21.123074+0000","last_active":"2024-04-25T09:57:22.113807+0000","last_peered":"2024-04-25T09:57:22.113807+0000","last_clean":"2024-04-25T09:57:22.113807+0000","last_became_active":"2024-04-25T09:57:21.122707+0000","last_became_peered":"2024-04-25T09:57:21.122707+0000","last_unstale":"2024-04-25T09:57:22.113807+0000","last_undegraded":"2024-04-25T09:57:22.113807+0000","last_fullsized":"2024-04-25T09:57:22.113807+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:25.099931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617743+0000","last_change":"2024-04-25T09:57:20.107160+0000","last_active":"2024-04-25T09:57:22.617743+0000","last_peered":"2024-04-25T09:57:22.617743+0000","last_clean":"2024-04-25T09:57:22.617743+0000","last_became_active":"2024-04-25T09:57:20.106839+0000","last_became_peered":"2024-04-25T09:57:20.106839+0000","last_unstale":"2024-04-25T09:57:22.617743+0000","last_undegraded":"2024-04-25T09:57:22.617743+0000","last_fullsized":"2024-04-25T09:57:22.617743+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:03.384506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137713+0000","last_change":"2024-04-25T09:57:21.109127+0000","last_active":"2024-04-25T09:57:22.137713+0000","last_peered":"2024-04-25T09:57:22.137713+0000","last_clean":"2024-04-25T09:57:22.137713+0000","last_became_active":"2024-04-25T09:57:21.108859+0000","last_became_peered":"2024-04-25T09:57:21.108859+0000","last_unstale":"2024-04-25T09:57:22.137713+0000","last_undegraded":"2024-04-25T09:57:22.137713+0000","last_fullsized":"2024-04-25T09:57:22.137713+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:17.764873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617669+0000","last_change":"2024-04-25T09:57:20.106767+0000","last_active":"2024-04-25T09:57:22.617669+0000","last_peered":"2024-04-25T09:57:22.617669+0000","last_clean":"2024-04-25T09:57:22.617669+0000","last_became_active":"2024-04-25T09:57:20.106470+0000","last_became_peered":"2024-04-25T09:57:20.106470+0000","last_unstale":"2024-04-25T09:57:22.617669+0000","last_undegraded":"2024-04-25T09:57:22.617669+0000","last_fullsized":"2024-04-25T09:57:22.617669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:34.401685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.110739+0000","last_change":"2024-04-25T09:57:21.114427+0000","last_active":"2024-04-25T09:57:22.110739+0000","last_peered":"2024-04-25T09:57:22.110739+0000","last_clean":"2024-04-25T09:57:22.110739+0000","last_became_active":"2024-04-25T09:57:21.114147+0000","last_became_peered":"2024-04-25T09:57:21.114147+0000","last_unstale":"2024-04-25T09:57:22.110739+0000","last_undegraded":"2024-04-25T09:57:22.110739+0000","last_fullsized":"2024-04-25T09:57:22.110739+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:47.518329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360143+0000","last_change":"2024-04-25T09:57:20.110133+0000","last_active":"2024-04-25T09:57:22.360143+0000","last_peered":"2024-04-25T09:57:22.360143+0000","last_clean":"2024-04-25T09:57:22.360143+0000","last_became_active":"2024-04-25T09:57:20.109848+0000","last_became_peered":"2024-04-25T09:57:20.109848+0000","last_unstale":"2024-04-25T09:57:22.360143+0000","last_undegraded":"2024-04-25T09:57:22.360143+0000","last_fullsized":"2024-04-25T09:57:22.360143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:30.224542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137376+0000","last_change":"2024-04-25T09:57:21.108633+0000","last_active":"2024-04-25T09:57:22.137376+0000","last_peered":"2024-04-25T09:57:22.137376+0000","last_clean":"2024-04-25T09:57:22.137376+0000","last_became_active":"2024-04-25T09:57:21.108213+0000","last_became_peered":"2024-04-25T09:57:21.108213+0000","last_unstale":"2024-04-25T09:57:22.137376+0000","last_undegraded":"2024-04-25T09:57:22.137376+0000","last_fullsized":"2024-04-25T09:57:22.137376+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:28.226431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113785+0000","last_change":"2024-04-25T09:57:20.100845+0000","last_active":"2024-04-25T09:57:22.113785+0000","last_peered":"2024-04-25T09:57:22.113785+0000","last_clean":"2024-04-25T09:57:22.113785+0000","last_became_active":"2024-04-25T09:57:20.100556+0000","last_became_peered":"2024-04-25T09:57:20.100556+0000","last_unstale":"2024-04-25T09:57:22.113785+0000","last_undegraded":"2024-04-25T09:57:22.113785+0000","last_fullsized":"2024-04-25T09:57:22.113785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:22.462199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.111145+0000","last_change":"2024-04-25T09:57:21.115393+0000","last_active":"2024-04-25T09:57:22.111145+0000","last_peered":"2024-04-25T09:57:22.111145+0000","last_clean":"2024-04-25T09:57:22.111145+0000","last_became_active":"2024-04-25T09:57:21.114967+0000","last_became_peered":"2024-04-25T09:57:21.114967+0000","last_unstale":"2024-04-25T09:57:22.111145+0000","last_undegraded":"2024-04-25T09:57:22.111145+0000","last_fullsized":"2024-04-25T09:57:22.111145+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:06.740795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618671+0000","last_change":"2024-04-25T09:57:20.108428+0000","last_active":"2024-04-25T09:57:22.618671+0000","last_peered":"2024-04-25T09:57:22.618671+0000","last_clean":"2024-04-25T09:57:22.618671+0000","last_became_active":"2024-04-25T09:57:20.107988+0000","last_became_peered":"2024-04-25T09:57:20.107988+0000","last_unstale":"2024-04-25T09:57:22.618671+0000","last_undegraded":"2024-04-25T09:57:22.618671+0000","last_fullsized":"2024-04-25T09:57:22.618671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:46.460376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113695+0000","last_change":"2024-04-25T09:57:21.113961+0000","last_active":"2024-04-25T09:57:22.113695+0000","last_peered":"2024-04-25T09:57:22.113695+0000","last_clean":"2024-04-25T09:57:22.113695+0000","last_became_active":"2024-04-25T09:57:21.113691+0000","last_became_peered":"2024-04-25T09:57:21.113691+0000","last_unstale":"2024-04-25T09:57:22.113695+0000","last_undegraded":"2024-04-25T09:57:22.113695+0000","last_fullsized":"2024-04-25T09:57:22.113695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:44.826600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618256+0000","last_change":"2024-04-25T09:57:20.107995+0000","last_active":"2024-04-25T09:57:22.618256+0000","last_peered":"2024-04-25T09:57:22.618256+0000","last_clean":"2024-04-25T09:57:22.618256+0000","last_became_active":"2024-04-25T09:57:20.107706+0000","last_became_peered":"2024-04-25T09:57:20.107706+0000","last_unstale":"2024-04-25T09:57:22.618256+0000","last_undegraded":"2024-04-25T09:57:22.618256+0000","last_fullsized":"2024-04-25T09:57:22.618256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:42.599662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137340+0000","last_change":"2024-04-25T09:57:21.111614+0000","last_active":"2024-04-25T09:57:22.137340+0000","last_peered":"2024-04-25T09:57:22.137340+0000","last_clean":"2024-04-25T09:57:22.137340+0000","last_became_active":"2024-04-25T09:57:21.111356+0000","last_became_peered":"2024-04-25T09:57:21.111356+0000","last_unstale":"2024-04-25T09:57:22.137340+0000","last_undegraded":"2024-04-25T09:57:22.137340+0000","last_fullsized":"2024-04-25T09:57:22.137340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:22.534444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137234+0000","last_change":"2024-04-25T09:57:20.112262+0000","last_active":"2024-04-25T09:57:22.137234+0000","last_peered":"2024-04-25T09:57:22.137234+0000","last_clean":"2024-04-25T09:57:22.137234+0000","last_became_active":"2024-04-25T09:57:20.111983+0000","last_became_peered":"2024-04-25T09:57:20.111983+0000","last_unstale":"2024-04-25T09:57:22.137234+0000","last_undegraded":"2024-04-25T09:57:22.137234+0000","last_fullsized":"2024-04-25T09:57:22.137234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:11.053241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137472+0000","last_change":"2024-04-25T09:57:21.117031+0000","last_active":"2024-04-25T09:57:22.137472+0000","last_peered":"2024-04-25T09:57:22.137472+0000","last_clean":"2024-04-25T09:57:22.137472+0000","last_became_active":"2024-04-25T09:57:21.116726+0000","last_became_peered":"2024-04-25T09:57:21.116726+0000","last_unstale":"2024-04-25T09:57:22.137472+0000","last_undegraded":"2024-04-25T09:57:22.137472+0000","last_fullsized":"2024-04-25T09:57:22.137472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:58.703264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418782+0000","last_change":"2024-04-25T09:57:20.104924+0000","last_active":"2024-04-25T09:57:22.418782+0000","last_peered":"2024-04-25T09:57:22.418782+0000","last_clean":"2024-04-25T09:57:22.418782+0000","last_became_active":"2024-04-25T09:57:20.104649+0000","last_became_peered":"2024-04-25T09:57:20.104649+0000","last_unstale":"2024-04-25T09:57:22.418782+0000","last_undegraded":"2024-04-25T09:57:22.418782+0000","last_fullsized":"2024-04-25T09:57:22.418782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:37.712850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257334+0000","last_change":"2024-04-25T09:57:21.112958+0000","last_active":"2024-04-25T09:57:22.257334+0000","last_peered":"2024-04-25T09:57:22.257334+0000","last_clean":"2024-04-25T09:57:22.257334+0000","last_became_active":"2024-04-25T09:57:21.112628+0000","last_became_peered":"2024-04-25T09:57:21.112628+0000","last_unstale":"2024-04-25T09:57:22.257334+0000","last_undegraded":"2024-04-25T09:57:22.257334+0000","last_fullsized":"2024-04-25T09:57:22.257334+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:22.534444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137489+0000","last_change":"2024-04-25T09:57:20.106732+0000","last_active":"2024-04-25T09:57:22.137489+0000","last_peered":"2024-04-25T09:57:22.137489+0000","last_clean":"2024-04-25T09:57:22.137489+0000","last_became_active":"2024-04-25T09:57:20.106452+0000","last_became_peered":"2024-04-25T09:57:20.106452+0000","last_unstale":"2024-04-25T09:57:22.137489+0000","last_undegraded":"2024-04-25T09:57:22.137489+0000","last_fullsized":"2024-04-25T09:57:22.137489+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:27.279085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257474+0000","last_change":"2024-04-25T09:57:21.109731+0000","last_active":"2024-04-25T09:57:22.257474+0000","last_peered":"2024-04-25T09:57:22.257474+0000","last_clean":"2024-04-25T09:57:22.257474+0000","last_became_active":"2024-04-25T09:57:21.109454+0000","last_became_peered":"2024-04-25T09:57:21.109454+0000","last_unstale":"2024-04-25T09:57:22.257474+0000","last_undegraded":"2024-04-25T09:57:22.257474+0000","last_fullsized":"2024-04-25T09:57:22.257474+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:43.569154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257011+0000","last_change":"2024-04-25T09:57:20.109206+0000","last_active":"2024-04-25T09:57:22.257011+0000","last_peered":"2024-04-25T09:57:22.257011+0000","last_clean":"2024-04-25T09:57:22.257011+0000","last_became_active":"2024-04-25T09:57:20.108272+0000","last_became_peered":"2024-04-25T09:57:20.108272+0000","last_unstale":"2024-04-25T09:57:22.257011+0000","last_undegraded":"2024-04-25T09:57:22.257011+0000","last_fullsized":"2024-04-25T09:57:22.257011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:11.053241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618598+0000","last_change":"2024-04-25T09:57:21.108222+0000","last_active":"2024-04-25T09:57:22.618598+0000","last_peered":"2024-04-25T09:57:22.618598+0000","last_clean":"2024-04-25T09:57:22.618598+0000","last_became_active":"2024-04-25T09:57:21.107928+0000","last_became_peered":"2024-04-25T09:57:21.107928+0000","last_unstale":"2024-04-25T09:57:22.618598+0000","last_undegraded":"2024-04-25T09:57:22.618598+0000","last_fullsized":"2024-04-25T09:57:22.618598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:25.099931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618076+0000","last_change":"2024-04-25T09:57:20.107108+0000","last_active":"2024-04-25T09:57:22.618076+0000","last_peered":"2024-04-25T09:57:22.618076+0000","last_clean":"2024-04-25T09:57:22.618076+0000","last_became_active":"2024-04-25T09:57:20.106402+0000","last_became_peered":"2024-04-25T09:57:20.106402+0000","last_unstale":"2024-04-25T09:57:22.618076+0000","last_undegraded":"2024-04-25T09:57:22.618076+0000","last_fullsized":"2024-04-25T09:57:22.618076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:24.866748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":273,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618399+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:57:22.618399+0000","last_peered":"2024-04-25T09:57:22.618399+0000","last_clean":"2024-04-25T09:57:22.618399+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:57:22.618399+0000","last_undegraded":"2024-04-25T09:57:22.618399+0000","last_fullsized":"2024-04-25T09:57:22.618399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":1013482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":424704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":921600,"data_stored":416434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":950272,"data_stored":424808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":423674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":929792,"data_stored":417142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":423204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1523712,"data_stored":1013460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:57:24.053 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:57:24.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.182+0000 7f973c9e0640 1 Processor -- start 2024-04-25T09:57:24.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.182+0000 7f973c9e0640 1 -- start start 2024-04-25T09:57:24.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.183+0000 7f973c9e0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f973815efb0 0x7f97380d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.183+0000 7f973c9e0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97380cfe70 con 0x7f9738099a50 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.183+0000 7f973c9e0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97380d0150 con 0x7f97380cf900 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.183+0000 7f973c9e0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97380d0a80 con 0x7f973815efb0 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.183+0000 7f973659c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f973815efb0 0x7f97380d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.183+0000 7f9736d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9738099a50 0x7f97380cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57062/0 (socket says 172.21.15.139:57062) 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.183+0000 7f9736d9d640 1 -- 172.21.15.139:0/2852460934 learned_addr learned my addr 172.21.15.139:0/2852460934 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.184+0000 7f973659c640 1 -- 172.21.15.139:0/2852460934 >> v1:172.21.15.150:6789/0 conn(0x7f97380cf900 legacy=0x7f973806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.184+0000 7f973659c640 1 -- 172.21.15.139:0/2852460934 >> v1:172.21.15.139:6789/0 conn(0x7f9738099a50 legacy=0x7f97380cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.184+0000 7f973659c640 1 -- 172.21.15.139:0/2852460934 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380d0430 con 0x7f973815efb0 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.184+0000 7f973559a640 1 -- 172.21.15.139:0/2852460934 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777217788 0 0) 0x7f97380cfe70 con 0x7f9738099a50 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.184+0000 7f973659c640 1 --2- 172.21.15.139:0/2852460934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f973815efb0 0x7f97380d0f10 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f97280097a0 tx=0x7f972802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4090c980c1327fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.184+0000 7f973559a640 1 -- 172.21.15.139:0/2852460934 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728037070 con 0x7f973815efb0 2024-04-25T09:57:24.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.184+0000 7f973559a640 1 -- 172.21.15.139:0/2852460934 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972802ae50 con 0x7f973815efb0 2024-04-25T09:57:24.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.184+0000 7f973559a640 1 -- 172.21.15.139:0/2852460934 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728032660 con 0x7f973815efb0 2024-04-25T09:57:24.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.185+0000 7f973c9e0640 1 -- 172.21.15.139:0/2852460934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f973815efb0 msgr2=0x7f97380d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.185+0000 7f973c9e0640 1 --2- 172.21.15.139:0/2852460934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f973815efb0 0x7f97380d0f10 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f97280097a0 tx=0x7f972802a140 comp rx=0 tx=0).stop 2024-04-25T09:57:24.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.185+0000 7f973c9e0640 1 -- 172.21.15.139:0/2852460934 shutdown_connections 2024-04-25T09:57:24.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.185+0000 7f973c9e0640 1 --2- 172.21.15.139:0/2852460934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f973815efb0 0x7f97380d0f10 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.185+0000 7f973c9e0640 1 -- 172.21.15.139:0/2852460934 >> 172.21.15.139:0/2852460934 conn(0x7f97380d1790 msgr2=0x7f97380d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.185+0000 7f973c9e0640 1 -- 172.21.15.139:0/2852460934 shutdown_connections 2024-04-25T09:57:24.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.185+0000 7f973c9e0640 1 -- 172.21.15.139:0/2852460934 wait complete. 2024-04-25T09:57:24.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.186+0000 7f973c9e0640 1 Processor -- start 2024-04-25T09:57:24.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.186+0000 7f973c9e0640 1 -- start start 2024-04-25T09:57:24.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f973c9e0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9738099a50 0x7f9738160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f973c9e0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97380cf900 0x7f97380c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f973c9e0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f973815efb0 0x7f97380ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f973c9e0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9738099e80 con 0x7f97380cf900 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f973c9e0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973806cf50 con 0x7f973815efb0 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f973c9e0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973809a000 con 0x7f9738099a50 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f9735d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97380cf900 0x7f97380c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f9735d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97380cf900 0x7f97380c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50000/0 (socket says 172.21.15.139:50000) 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f9735d9b640 1 -- 172.21.15.139:0/1610044897 learned_addr learned my addr 172.21.15.139:0/1610044897 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f9736d9d640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f973815efb0 0x7f97380ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f973659c640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9738099a50 0x7f9738160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f9735d9b640 1 -- 172.21.15.139:0/1610044897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9738099a50 msgr2=0x7f9738160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f9735d9b640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9738099a50 0x7f9738160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f9735d9b640 1 -- 172.21.15.139:0/1610044897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f973815efb0 msgr2=0x7f97380ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f9735d9b640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f973815efb0 0x7f97380ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f9735d9b640 1 -- 172.21.15.139:0/1610044897 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9738161160 con 0x7f97380cf900 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.187+0000 7f9736d9d640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f973815efb0 0x7f97380ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.188+0000 7f9735d9b640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97380cf900 0x7f97380c9b90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f972c00c430 tx=0x7f972c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.188+0000 7f9726ffd640 1 -- 172.21.15.139:0/1610044897 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c011790 con 0x7f97380cf900 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.188+0000 7f9726ffd640 1 -- 172.21.15.139:0/1610044897 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c017070 con 0x7f97380cf900 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.188+0000 7f973c9e0640 1 -- 172.21.15.139:0/1610044897 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9738161400 con 0x7f97380cf900 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.188+0000 7f9726ffd640 1 -- 172.21.15.139:0/1610044897 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c011bb0 con 0x7f97380cf900 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.188+0000 7f973c9e0640 1 -- 172.21.15.139:0/1610044897 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97381619c0 con 0x7f97380cf900 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.189+0000 7f973c9e0640 1 -- 172.21.15.139:0/1610044897 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9738074520 con 0x7f97380cf900 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.189+0000 7f9726ffd640 1 -- 172.21.15.139:0/1610044897 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f972c01c460 con 0x7f97380cf900 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.189+0000 7f9726ffd640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f971806b4d0 0x7f971806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.190+0000 7f9726ffd640 1 -- 172.21.15.139:0/1610044897 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f972c08a1e0 con 0x7f97380cf900 2024-04-25T09:57:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.190+0000 7f973659c640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f971806b4d0 0x7f971806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.190+0000 7f973659c640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f971806b4d0 0x7f971806d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9728034000 tx=0x7f972802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.194+0000 7f9726ffd640 1 -- 172.21.15.139:0/1610044897 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f972c058f60 con 0x7f97380cf900 2024-04-25T09:57:24.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.361+0000 7f973c9e0640 1 -- 172.21.15.139:0/1610044897 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97380cb350 con 0x7f971806b4d0 2024-04-25T09:57:24.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.367+0000 7f9726ffd640 1 -- 172.21.15.139:0/1610044897 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252071 (secure 0 0 0) 0x7f97380cb350 con 0x7f971806b4d0 2024-04-25T09:57:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:24.374 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:57:24.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.370+0000 7f973c9e0640 1 -- 172.21.15.139:0/1610044897 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f971806b4d0 msgr2=0x7f971806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.370+0000 7f973c9e0640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f971806b4d0 0x7f971806d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9728034000 tx=0x7f972802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.370+0000 7f973c9e0640 1 -- 172.21.15.139:0/1610044897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97380cf900 msgr2=0x7f97380c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.370+0000 7f973c9e0640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97380cf900 0x7f97380c9b90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f972c00c430 tx=0x7f972c00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.370+0000 7f973c9e0640 1 -- 172.21.15.139:0/1610044897 shutdown_connections 2024-04-25T09:57:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.370+0000 7f973c9e0640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97380cf900 0x7f97380c9b90 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.370+0000 7f973c9e0640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f971806b4d0 0x7f971806d990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.370+0000 7f973c9e0640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f973815efb0 0x7f97380ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.370+0000 7f973c9e0640 1 --2- 172.21.15.139:0/1610044897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9738099a50 0x7f9738160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.370+0000 7f973c9e0640 1 -- 172.21.15.139:0/1610044897 >> 172.21.15.139:0/1610044897 conn(0x7f97380d1790 msgr2=0x7f9738159960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.370+0000 7f973c9e0640 1 -- 172.21.15.139:0/1610044897 shutdown_connections 2024-04-25T09:57:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.370+0000 7f973c9e0640 1 -- 172.21.15.139:0/1610044897 wait complete. 2024-04-25T09:57:24.384 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-04-25T09:57:23.022608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237128,"kb_used_data":8464,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754737592,"statfs":{"total":773094113280,"available":772851294208,"internally_reserved":0,"allocated":8667136,"data_stored":4556908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000779"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618596+0000","last_change":"2024-04-25T09:57:21.120454+0000","last_active":"2024-04-25T09:57:22.618596+0000","last_peered":"2024-04-25T09:57:22.618596+0000","last_clean":"2024-04-25T09:57:22.618596+0000","last_became_active":"2024-04-25T09:57:21.120183+0000","last_became_peered":"2024-04-25T09:57:21.120183+0000","last_unstale":"2024-04-25T09:57:22.618596+0000","last_undegraded":"2024-04-25T09:57:22.618596+0000","last_fullsized":"2024-04-25T09:57:22.618596+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:44.986065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113720+0000","last_change":"2024-04-25T09:57:21.117564+0000","last_active":"2024-04-25T09:57:22.113720+0000","last_peered":"2024-04-25T09:57:22.113720+0000","last_clean":"2024-04-25T09:57:22.113720+0000","last_became_active":"2024-04-25T09:57:21.117237+0000","last_became_peered":"2024-04-25T09:57:21.117237+0000","last_unstale":"2024-04-25T09:57:22.113720+0000","last_undegraded":"2024-04-25T09:57:22.113720+0000","last_fullsized":"2024-04-25T09:57:22.113720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:51.923393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257289+0000","last_change":"2024-04-25T09:57:21.114125+0000","last_active":"2024-04-25T09:57:22.257289+0000","last_peered":"2024-04-25T09:57:22.257289+0000","last_clean":"2024-04-25T09:57:22.257289+0000","last_became_active":"2024-04-25T09:57:21.113376+0000","last_became_peered":"2024-04-25T09:57:21.113376+0000","last_unstale":"2024-04-25T09:57:22.257289+0000","last_undegraded":"2024-04-25T09:57:22.257289+0000","last_fullsized":"2024-04-25T09:57:22.257289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:27.183028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360172+0000","last_change":"2024-04-25T09:57:21.123359+0000","last_active":"2024-04-25T09:57:22.360172+0000","last_peered":"2024-04-25T09:57:22.360172+0000","last_clean":"2024-04-25T09:57:22.360172+0000","last_became_active":"2024-04-25T09:57:21.123051+0000","last_became_peered":"2024-04-25T09:57:21.123051+0000","last_unstale":"2024-04-25T09:57:22.360172+0000","last_undegraded":"2024-04-25T09:57:22.360172+0000","last_fullsized":"2024-04-25T09:57:22.360172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:47.465523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418803+0000","last_change":"2024-04-25T09:57:21.107727+0000","last_active":"2024-04-25T09:57:22.418803+0000","last_peered":"2024-04-25T09:57:22.418803+0000","last_clean":"2024-04-25T09:57:22.418803+0000","last_became_active":"2024-04-25T09:57:21.107444+0000","last_became_peered":"2024-04-25T09:57:21.107444+0000","last_unstale":"2024-04-25T09:57:22.418803+0000","last_undegraded":"2024-04-25T09:57:22.418803+0000","last_fullsized":"2024-04-25T09:57:22.418803+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:21.129618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617759+0000","last_change":"2024-04-25T09:57:21.121964+0000","last_active":"2024-04-25T09:57:22.617759+0000","last_peered":"2024-04-25T09:57:22.617759+0000","last_clean":"2024-04-25T09:57:22.617759+0000","last_became_active":"2024-04-25T09:57:21.121655+0000","last_became_peered":"2024-04-25T09:57:21.121655+0000","last_unstale":"2024-04-25T09:57:22.617759+0000","last_undegraded":"2024-04-25T09:57:22.617759+0000","last_fullsized":"2024-04-25T09:57:22.617759+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:58.172060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137349+0000","last_change":"2024-04-25T09:57:21.107916+0000","last_active":"2024-04-25T09:57:22.137349+0000","last_peered":"2024-04-25T09:57:22.137349+0000","last_clean":"2024-04-25T09:57:22.137349+0000","last_became_active":"2024-04-25T09:57:21.107633+0000","last_became_peered":"2024-04-25T09:57:21.107633+0000","last_unstale":"2024-04-25T09:57:22.137349+0000","last_undegraded":"2024-04-25T09:57:22.137349+0000","last_fullsized":"2024-04-25T09:57:22.137349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:49.691316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418557+0000","last_change":"2024-04-25T09:57:21.108364+0000","last_active":"2024-04-25T09:57:22.418557+0000","last_peered":"2024-04-25T09:57:22.418557+0000","last_clean":"2024-04-25T09:57:22.418557+0000","last_became_active":"2024-04-25T09:57:21.108086+0000","last_became_peered":"2024-04-25T09:57:21.108086+0000","last_unstale":"2024-04-25T09:57:22.418557+0000","last_undegraded":"2024-04-25T09:57:22.418557+0000","last_fullsized":"2024-04-25T09:57:22.418557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:42.296165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.110961+0000","last_change":"2024-04-25T09:57:21.120098+0000","last_active":"2024-04-25T09:57:22.110961+0000","last_peered":"2024-04-25T09:57:22.110961+0000","last_clean":"2024-04-25T09:57:22.110961+0000","last_became_active":"2024-04-25T09:57:21.119798+0000","last_became_peered":"2024-04-25T09:57:21.119798+0000","last_unstale":"2024-04-25T09:57:22.110961+0000","last_undegraded":"2024-04-25T09:57:22.110961+0000","last_fullsized":"2024-04-25T09:57:22.110961+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:14.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360171+0000","last_change":"2024-04-25T09:57:21.110790+0000","last_active":"2024-04-25T09:57:22.360171+0000","last_peered":"2024-04-25T09:57:22.360171+0000","last_clean":"2024-04-25T09:57:22.360171+0000","last_became_active":"2024-04-25T09:57:21.110488+0000","last_became_peered":"2024-04-25T09:57:21.110488+0000","last_unstale":"2024-04-25T09:57:22.360171+0000","last_undegraded":"2024-04-25T09:57:22.360171+0000","last_fullsized":"2024-04-25T09:57:22.360171+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:46.896264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113707+0000","last_change":"2024-04-25T09:57:21.117563+0000","last_active":"2024-04-25T09:57:22.113707+0000","last_peered":"2024-04-25T09:57:22.113707+0000","last_clean":"2024-04-25T09:57:22.113707+0000","last_became_active":"2024-04-25T09:57:21.117228+0000","last_became_peered":"2024-04-25T09:57:21.117228+0000","last_unstale":"2024-04-25T09:57:22.113707+0000","last_undegraded":"2024-04-25T09:57:22.113707+0000","last_fullsized":"2024-04-25T09:57:22.113707+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:15.380983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360213+0000","last_change":"2024-04-25T09:57:21.112520+0000","last_active":"2024-04-25T09:57:22.360213+0000","last_peered":"2024-04-25T09:57:22.360213+0000","last_clean":"2024-04-25T09:57:22.360213+0000","last_became_active":"2024-04-25T09:57:21.112165+0000","last_became_peered":"2024-04-25T09:57:21.112165+0000","last_unstale":"2024-04-25T09:57:22.360213+0000","last_undegraded":"2024-04-25T09:57:22.360213+0000","last_fullsized":"2024-04-25T09:57:22.360213+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:33.042997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418857+0000","last_change":"2024-04-25T09:57:21.111137+0000","last_active":"2024-04-25T09:57:22.418857+0000","last_peered":"2024-04-25T09:57:22.418857+0000","last_clean":"2024-04-25T09:57:22.418857+0000","last_became_active":"2024-04-25T09:57:21.110858+0000","last_became_peered":"2024-04-25T09:57:21.110858+0000","last_unstale":"2024-04-25T09:57:22.418857+0000","last_undegraded":"2024-04-25T09:57:22.418857+0000","last_fullsized":"2024-04-25T09:57:22.418857+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:13.964094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617979+0000","last_change":"2024-04-25T09:57:21.120742+0000","last_active":"2024-04-25T09:57:22.617979+0000","last_peered":"2024-04-25T09:57:22.617979+0000","last_clean":"2024-04-25T09:57:22.617979+0000","last_became_active":"2024-04-25T09:57:21.120446+0000","last_became_peered":"2024-04-25T09:57:21.120446+0000","last_unstale":"2024-04-25T09:57:22.617979+0000","last_undegraded":"2024-04-25T09:57:22.617979+0000","last_fullsized":"2024-04-25T09:57:22.617979+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:44.826600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113362+0000","last_change":"2024-04-25T09:57:21.109684+0000","last_active":"2024-04-25T09:57:22.113362+0000","last_peered":"2024-04-25T09:57:22.113362+0000","last_clean":"2024-04-25T09:57:22.113362+0000","last_became_active":"2024-04-25T09:57:21.109419+0000","last_became_peered":"2024-04-25T09:57:21.109419+0000","last_unstale":"2024-04-25T09:57:22.113362+0000","last_undegraded":"2024-04-25T09:57:22.113362+0000","last_fullsized":"2024-04-25T09:57:22.113362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:53.001563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618434+0000","last_change":"2024-04-25T09:57:21.112431+0000","last_active":"2024-04-25T09:57:22.618434+0000","last_peered":"2024-04-25T09:57:22.618434+0000","last_clean":"2024-04-25T09:57:22.618434+0000","last_became_active":"2024-04-25T09:57:21.112158+0000","last_became_peered":"2024-04-25T09:57:21.112158+0000","last_unstale":"2024-04-25T09:57:22.618434+0000","last_undegraded":"2024-04-25T09:57:22.618434+0000","last_fullsized":"2024-04-25T09:57:22.618434+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:22.651722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418971+0000","last_change":"2024-04-25T09:57:21.113119+0000","last_active":"2024-04-25T09:57:22.418971+0000","last_peered":"2024-04-25T09:57:22.418971+0000","last_clean":"2024-04-25T09:57:22.418971+0000","last_became_active":"2024-04-25T09:57:21.112870+0000","last_became_peered":"2024-04-25T09:57:21.112870+0000","last_unstale":"2024-04-25T09:57:22.418971+0000","last_undegraded":"2024-04-25T09:57:22.418971+0000","last_fullsized":"2024-04-25T09:57:22.418971+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:56.255054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257312+0000","last_change":"2024-04-25T09:57:21.120220+0000","last_active":"2024-04-25T09:57:22.257312+0000","last_peered":"2024-04-25T09:57:22.257312+0000","last_clean":"2024-04-25T09:57:22.257312+0000","last_became_active":"2024-04-25T09:57:21.119961+0000","last_became_peered":"2024-04-25T09:57:21.119961+0000","last_unstale":"2024-04-25T09:57:22.257312+0000","last_undegraded":"2024-04-25T09:57:22.257312+0000","last_fullsized":"2024-04-25T09:57:22.257312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:49.691316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617938+0000","last_change":"2024-04-25T09:57:21.114955+0000","last_active":"2024-04-25T09:57:22.617938+0000","last_peered":"2024-04-25T09:57:22.617938+0000","last_clean":"2024-04-25T09:57:22.617938+0000","last_became_active":"2024-04-25T09:57:21.114678+0000","last_became_peered":"2024-04-25T09:57:21.114678+0000","last_unstale":"2024-04-25T09:57:22.617938+0000","last_undegraded":"2024-04-25T09:57:22.617938+0000","last_fullsized":"2024-04-25T09:57:22.617938+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:14.135232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418983+0000","last_change":"2024-04-25T09:57:21.104652+0000","last_active":"2024-04-25T09:57:22.418983+0000","last_peered":"2024-04-25T09:57:22.418983+0000","last_clean":"2024-04-25T09:57:22.418983+0000","last_became_active":"2024-04-25T09:57:21.104382+0000","last_became_peered":"2024-04-25T09:57:21.104382+0000","last_unstale":"2024-04-25T09:57:22.418983+0000","last_undegraded":"2024-04-25T09:57:22.418983+0000","last_fullsized":"2024-04-25T09:57:22.418983+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:32.090384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.111106+0000","last_change":"2024-04-25T09:57:21.122880+0000","last_active":"2024-04-25T09:57:22.111106+0000","last_peered":"2024-04-25T09:57:22.111106+0000","last_clean":"2024-04-25T09:57:22.111106+0000","last_became_active":"2024-04-25T09:57:21.122555+0000","last_became_peered":"2024-04-25T09:57:21.122555+0000","last_unstale":"2024-04-25T09:57:22.111106+0000","last_undegraded":"2024-04-25T09:57:22.111106+0000","last_fullsized":"2024-04-25T09:57:22.111106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:06.392815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418650+0000","last_change":"2024-04-25T09:57:21.111844+0000","last_active":"2024-04-25T09:57:22.418650+0000","last_peered":"2024-04-25T09:57:22.418650+0000","last_clean":"2024-04-25T09:57:22.418650+0000","last_became_active":"2024-04-25T09:57:21.111542+0000","last_became_peered":"2024-04-25T09:57:21.111542+0000","last_unstale":"2024-04-25T09:57:22.418650+0000","last_undegraded":"2024-04-25T09:57:22.418650+0000","last_fullsized":"2024-04-25T09:57:22.418650+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:57.981700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.110801+0000","last_change":"2024-04-25T09:57:21.115645+0000","last_active":"2024-04-25T09:57:22.110801+0000","last_peered":"2024-04-25T09:57:22.110801+0000","last_clean":"2024-04-25T09:57:22.110801+0000","last_became_active":"2024-04-25T09:57:21.115370+0000","last_became_peered":"2024-04-25T09:57:21.115370+0000","last_unstale":"2024-04-25T09:57:22.110801+0000","last_undegraded":"2024-04-25T09:57:22.110801+0000","last_fullsized":"2024-04-25T09:57:22.110801+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:26.191533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113349+0000","last_change":"2024-04-25T09:57:21.111467+0000","last_active":"2024-04-25T09:57:22.113349+0000","last_peered":"2024-04-25T09:57:22.113349+0000","last_clean":"2024-04-25T09:57:22.113349+0000","last_became_active":"2024-04-25T09:57:21.111189+0000","last_became_peered":"2024-04-25T09:57:21.111189+0000","last_unstale":"2024-04-25T09:57:22.113349+0000","last_undegraded":"2024-04-25T09:57:22.113349+0000","last_fullsized":"2024-04-25T09:57:22.113349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:14.897617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113535+0000","last_change":"2024-04-25T09:57:21.111492+0000","last_active":"2024-04-25T09:57:22.113535+0000","last_peered":"2024-04-25T09:57:22.113535+0000","last_clean":"2024-04-25T09:57:22.113535+0000","last_became_active":"2024-04-25T09:57:21.111218+0000","last_became_peered":"2024-04-25T09:57:21.111218+0000","last_unstale":"2024-04-25T09:57:22.113535+0000","last_undegraded":"2024-04-25T09:57:22.113535+0000","last_fullsized":"2024-04-25T09:57:22.113535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:58.172060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360171+0000","last_change":"2024-04-25T09:57:21.115655+0000","last_active":"2024-04-25T09:57:22.360171+0000","last_peered":"2024-04-25T09:57:22.360171+0000","last_clean":"2024-04-25T09:57:22.360171+0000","last_became_active":"2024-04-25T09:57:21.115405+0000","last_became_peered":"2024-04-25T09:57:21.115405+0000","last_unstale":"2024-04-25T09:57:22.360171+0000","last_undegraded":"2024-04-25T09:57:22.360171+0000","last_fullsized":"2024-04-25T09:57:22.360171+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:43.604808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618411+0000","last_change":"2024-04-25T09:57:21.111294+0000","last_active":"2024-04-25T09:57:22.618411+0000","last_peered":"2024-04-25T09:57:22.618411+0000","last_clean":"2024-04-25T09:57:22.618411+0000","last_became_active":"2024-04-25T09:57:21.111039+0000","last_became_peered":"2024-04-25T09:57:21.111039+0000","last_unstale":"2024-04-25T09:57:22.618411+0000","last_undegraded":"2024-04-25T09:57:22.618411+0000","last_fullsized":"2024-04-25T09:57:22.618411+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:51.923393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418997+0000","last_change":"2024-04-25T09:57:21.105983+0000","last_active":"2024-04-25T09:57:22.418997+0000","last_peered":"2024-04-25T09:57:22.418997+0000","last_clean":"2024-04-25T09:57:22.418997+0000","last_became_active":"2024-04-25T09:57:21.105709+0000","last_became_peered":"2024-04-25T09:57:21.105709+0000","last_unstale":"2024-04-25T09:57:22.418997+0000","last_undegraded":"2024-04-25T09:57:22.418997+0000","last_fullsized":"2024-04-25T09:57:22.418997+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:07.746769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113502+0000","last_change":"2024-04-25T09:57:21.116340+0000","last_active":"2024-04-25T09:57:22.113502+0000","last_peered":"2024-04-25T09:57:22.113502+0000","last_clean":"2024-04-25T09:57:22.113502+0000","last_became_active":"2024-04-25T09:57:21.116047+0000","last_became_peered":"2024-04-25T09:57:21.116047+0000","last_unstale":"2024-04-25T09:57:22.113502+0000","last_undegraded":"2024-04-25T09:57:22.113502+0000","last_fullsized":"2024-04-25T09:57:22.113502+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:27.828672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618209+0000","last_change":"2024-04-25T09:57:21.112039+0000","last_active":"2024-04-25T09:57:22.618209+0000","last_peered":"2024-04-25T09:57:22.618209+0000","last_clean":"2024-04-25T09:57:22.618209+0000","last_became_active":"2024-04-25T09:57:21.111773+0000","last_became_peered":"2024-04-25T09:57:21.111773+0000","last_unstale":"2024-04-25T09:57:22.618209+0000","last_undegraded":"2024-04-25T09:57:22.618209+0000","last_fullsized":"2024-04-25T09:57:22.618209+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:44.826600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257098+0000","last_change":"2024-04-25T09:57:21.112364+0000","last_active":"2024-04-25T09:57:22.257098+0000","last_peered":"2024-04-25T09:57:22.257098+0000","last_clean":"2024-04-25T09:57:22.257098+0000","last_became_active":"2024-04-25T09:57:21.112083+0000","last_became_peered":"2024-04-25T09:57:21.112083+0000","last_unstale":"2024-04-25T09:57:22.257098+0000","last_undegraded":"2024-04-25T09:57:22.257098+0000","last_fullsized":"2024-04-25T09:57:22.257098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:11.795871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618312+0000","last_change":"2024-04-25T09:57:21.121149+0000","last_active":"2024-04-25T09:57:22.618312+0000","last_peered":"2024-04-25T09:57:22.618312+0000","last_clean":"2024-04-25T09:57:22.618312+0000","last_became_active":"2024-04-25T09:57:21.120575+0000","last_became_peered":"2024-04-25T09:57:21.120575+0000","last_unstale":"2024-04-25T09:57:22.618312+0000","last_undegraded":"2024-04-25T09:57:22.618312+0000","last_fullsized":"2024-04-25T09:57:22.618312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:15.380983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257161+0000","last_change":"2024-04-25T09:57:21.119784+0000","last_active":"2024-04-25T09:57:22.257161+0000","last_peered":"2024-04-25T09:57:22.257161+0000","last_clean":"2024-04-25T09:57:22.257161+0000","last_became_active":"2024-04-25T09:57:21.119516+0000","last_became_peered":"2024-04-25T09:57:21.119516+0000","last_unstale":"2024-04-25T09:57:22.257161+0000","last_undegraded":"2024-04-25T09:57:22.257161+0000","last_fullsized":"2024-04-25T09:57:22.257161+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:28.602725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"75'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418901+0000","last_change":"2024-04-25T09:57:20.102637+0000","last_active":"2024-04-25T09:57:22.418901+0000","last_peered":"2024-04-25T09:57:22.418901+0000","last_clean":"2024-04-25T09:57:22.418901+0000","last_became_active":"2024-04-25T09:57:20.102355+0000","last_became_peered":"2024-04-25T09:57:20.102355+0000","last_unstale":"2024-04-25T09:57:22.418901+0000","last_undegraded":"2024-04-25T09:57:22.418901+0000","last_fullsized":"2024-04-25T09:57:22.418901+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:26.062409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113711+0000","last_change":"2024-04-25T09:57:21.110169+0000","last_active":"2024-04-25T09:57:22.113711+0000","last_peered":"2024-04-25T09:57:22.113711+0000","last_clean":"2024-04-25T09:57:22.113711+0000","last_became_active":"2024-04-25T09:57:21.109878+0000","last_became_peered":"2024-04-25T09:57:21.109878+0000","last_unstale":"2024-04-25T09:57:22.113711+0000","last_undegraded":"2024-04-25T09:57:22.113711+0000","last_fullsized":"2024-04-25T09:57:22.113711+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:22.651722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618449+0000","last_change":"2024-04-25T09:57:20.103923+0000","last_active":"2024-04-25T09:57:22.618449+0000","last_peered":"2024-04-25T09:57:22.618449+0000","last_clean":"2024-04-25T09:57:22.618449+0000","last_became_active":"2024-04-25T09:57:20.103647+0000","last_became_peered":"2024-04-25T09:57:20.103647+0000","last_unstale":"2024-04-25T09:57:22.618449+0000","last_undegraded":"2024-04-25T09:57:22.618449+0000","last_fullsized":"2024-04-25T09:57:22.618449+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:34.401685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.111100+0000","last_change":"2024-04-25T09:57:21.118426+0000","last_active":"2024-04-25T09:57:22.111100+0000","last_peered":"2024-04-25T09:57:22.111100+0000","last_clean":"2024-04-25T09:57:22.111100+0000","last_became_active":"2024-04-25T09:57:21.118107+0000","last_became_peered":"2024-04-25T09:57:21.118107+0000","last_unstale":"2024-04-25T09:57:22.111100+0000","last_undegraded":"2024-04-25T09:57:22.111100+0000","last_fullsized":"2024-04-25T09:57:22.111100+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:50.756060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618388+0000","last_change":"2024-04-25T09:57:20.109590+0000","last_active":"2024-04-25T09:57:22.618388+0000","last_peered":"2024-04-25T09:57:22.618388+0000","last_clean":"2024-04-25T09:57:22.618388+0000","last_became_active":"2024-04-25T09:57:20.109336+0000","last_became_peered":"2024-04-25T09:57:20.109336+0000","last_unstale":"2024-04-25T09:57:22.618388+0000","last_undegraded":"2024-04-25T09:57:22.618388+0000","last_fullsized":"2024-04-25T09:57:22.618388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:24.866748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.110973+0000","last_change":"2024-04-25T09:57:21.122813+0000","last_active":"2024-04-25T09:57:22.110973+0000","last_peered":"2024-04-25T09:57:22.110973+0000","last_clean":"2024-04-25T09:57:22.110973+0000","last_became_active":"2024-04-25T09:57:21.122187+0000","last_became_peered":"2024-04-25T09:57:21.122187+0000","last_unstale":"2024-04-25T09:57:22.110973+0000","last_undegraded":"2024-04-25T09:57:22.110973+0000","last_fullsized":"2024-04-25T09:57:22.110973+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:03.360678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137303+0000","last_change":"2024-04-25T09:57:20.112199+0000","last_active":"2024-04-25T09:57:22.137303+0000","last_peered":"2024-04-25T09:57:22.137303+0000","last_clean":"2024-04-25T09:57:22.137303+0000","last_became_active":"2024-04-25T09:57:20.111907+0000","last_became_peered":"2024-04-25T09:57:20.111907+0000","last_unstale":"2024-04-25T09:57:22.137303+0000","last_undegraded":"2024-04-25T09:57:22.137303+0000","last_fullsized":"2024-04-25T09:57:22.137303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:40.105458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113599+0000","last_change":"2024-04-25T09:57:21.112913+0000","last_active":"2024-04-25T09:57:22.113599+0000","last_peered":"2024-04-25T09:57:22.113599+0000","last_clean":"2024-04-25T09:57:22.113599+0000","last_became_active":"2024-04-25T09:57:21.112471+0000","last_became_peered":"2024-04-25T09:57:21.112471+0000","last_unstale":"2024-04-25T09:57:22.113599+0000","last_undegraded":"2024-04-25T09:57:22.113599+0000","last_fullsized":"2024-04-25T09:57:22.113599+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:23.671368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.111085+0000","last_change":"2024-04-25T09:57:20.106997+0000","last_active":"2024-04-25T09:57:22.111085+0000","last_peered":"2024-04-25T09:57:22.111085+0000","last_clean":"2024-04-25T09:57:22.111085+0000","last_became_active":"2024-04-25T09:57:20.106697+0000","last_became_peered":"2024-04-25T09:57:20.106697+0000","last_unstale":"2024-04-25T09:57:22.111085+0000","last_undegraded":"2024-04-25T09:57:22.111085+0000","last_fullsized":"2024-04-25T09:57:22.111085+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:06.051591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113772+0000","last_change":"2024-04-25T09:57:21.117351+0000","last_active":"2024-04-25T09:57:22.113772+0000","last_peered":"2024-04-25T09:57:22.113772+0000","last_clean":"2024-04-25T09:57:22.113772+0000","last_became_active":"2024-04-25T09:57:21.117064+0000","last_became_peered":"2024-04-25T09:57:21.117064+0000","last_unstale":"2024-04-25T09:57:22.113772+0000","last_undegraded":"2024-04-25T09:57:22.113772+0000","last_fullsized":"2024-04-25T09:57:22.113772+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:44.986065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.359922+0000","last_change":"2024-04-25T09:57:20.110505+0000","last_active":"2024-04-25T09:57:22.359922+0000","last_peered":"2024-04-25T09:57:22.359922+0000","last_clean":"2024-04-25T09:57:22.359922+0000","last_became_active":"2024-04-25T09:57:20.110230+0000","last_became_peered":"2024-04-25T09:57:20.110230+0000","last_unstale":"2024-04-25T09:57:22.359922+0000","last_undegraded":"2024-04-25T09:57:22.359922+0000","last_fullsized":"2024-04-25T09:57:22.359922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:39.225413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.111193+0000","last_change":"2024-04-25T09:57:21.118869+0000","last_active":"2024-04-25T09:57:22.111193+0000","last_peered":"2024-04-25T09:57:22.111193+0000","last_clean":"2024-04-25T09:57:22.111193+0000","last_became_active":"2024-04-25T09:57:21.118504+0000","last_became_peered":"2024-04-25T09:57:21.118504+0000","last_unstale":"2024-04-25T09:57:22.111193+0000","last_undegraded":"2024-04-25T09:57:22.111193+0000","last_fullsized":"2024-04-25T09:57:22.111193+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:11.795871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113454+0000","last_change":"2024-04-25T09:57:20.110833+0000","last_active":"2024-04-25T09:57:22.113454+0000","last_peered":"2024-04-25T09:57:22.113454+0000","last_clean":"2024-04-25T09:57:22.113454+0000","last_became_active":"2024-04-25T09:57:20.110567+0000","last_became_peered":"2024-04-25T09:57:20.110567+0000","last_unstale":"2024-04-25T09:57:22.113454+0000","last_undegraded":"2024-04-25T09:57:22.113454+0000","last_fullsized":"2024-04-25T09:57:22.113454+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:30.224542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137668+0000","last_change":"2024-04-25T09:57:21.115420+0000","last_active":"2024-04-25T09:57:22.137668+0000","last_peered":"2024-04-25T09:57:22.137668+0000","last_clean":"2024-04-25T09:57:22.137668+0000","last_became_active":"2024-04-25T09:57:21.114957+0000","last_became_peered":"2024-04-25T09:57:21.114957+0000","last_unstale":"2024-04-25T09:57:22.137668+0000","last_undegraded":"2024-04-25T09:57:22.137668+0000","last_fullsized":"2024-04-25T09:57:22.137668+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:23.467346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418698+0000","last_change":"2024-04-25T09:57:20.101814+0000","last_active":"2024-04-25T09:57:22.418698+0000","last_peered":"2024-04-25T09:57:22.418698+0000","last_clean":"2024-04-25T09:57:22.418698+0000","last_became_active":"2024-04-25T09:57:20.101558+0000","last_became_peered":"2024-04-25T09:57:20.101558+0000","last_unstale":"2024-04-25T09:57:22.418698+0000","last_undegraded":"2024-04-25T09:57:22.418698+0000","last_fullsized":"2024-04-25T09:57:22.418698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:16.385636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113814+0000","last_change":"2024-04-25T09:57:21.110914+0000","last_active":"2024-04-25T09:57:22.113814+0000","last_peered":"2024-04-25T09:57:22.113814+0000","last_clean":"2024-04-25T09:57:22.113814+0000","last_became_active":"2024-04-25T09:57:21.110604+0000","last_became_peered":"2024-04-25T09:57:21.110604+0000","last_unstale":"2024-04-25T09:57:22.113814+0000","last_undegraded":"2024-04-25T09:57:22.113814+0000","last_fullsized":"2024-04-25T09:57:22.113814+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:00.063570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618579+0000","last_change":"2024-04-25T09:57:20.110007+0000","last_active":"2024-04-25T09:57:22.618579+0000","last_peered":"2024-04-25T09:57:22.618579+0000","last_clean":"2024-04-25T09:57:22.618579+0000","last_became_active":"2024-04-25T09:57:20.109695+0000","last_became_peered":"2024-04-25T09:57:20.109695+0000","last_unstale":"2024-04-25T09:57:22.618579+0000","last_undegraded":"2024-04-25T09:57:22.618579+0000","last_fullsized":"2024-04-25T09:57:22.618579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:45.891118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360084+0000","last_change":"2024-04-25T09:57:21.119640+0000","last_active":"2024-04-25T09:57:22.360084+0000","last_peered":"2024-04-25T09:57:22.360084+0000","last_clean":"2024-04-25T09:57:22.360084+0000","last_became_active":"2024-04-25T09:57:21.119306+0000","last_became_peered":"2024-04-25T09:57:21.119306+0000","last_unstale":"2024-04-25T09:57:22.360084+0000","last_undegraded":"2024-04-25T09:57:22.360084+0000","last_fullsized":"2024-04-25T09:57:22.360084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:25.871895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360288+0000","last_change":"2024-04-25T09:57:20.113310+0000","last_active":"2024-04-25T09:57:22.360288+0000","last_peered":"2024-04-25T09:57:22.360288+0000","last_clean":"2024-04-25T09:57:22.360288+0000","last_became_active":"2024-04-25T09:57:20.113015+0000","last_became_peered":"2024-04-25T09:57:20.113015+0000","last_unstale":"2024-04-25T09:57:22.360288+0000","last_undegraded":"2024-04-25T09:57:22.360288+0000","last_fullsized":"2024-04-25T09:57:22.360288+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:16.759726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618050+0000","last_change":"2024-04-25T09:57:21.107081+0000","last_active":"2024-04-25T09:57:22.618050+0000","last_peered":"2024-04-25T09:57:22.618050+0000","last_clean":"2024-04-25T09:57:22.618050+0000","last_became_active":"2024-04-25T09:57:21.105586+0000","last_became_peered":"2024-04-25T09:57:21.105586+0000","last_unstale":"2024-04-25T09:57:22.618050+0000","last_undegraded":"2024-04-25T09:57:22.618050+0000","last_fullsized":"2024-04-25T09:57:22.618050+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:15.380983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"75'3","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418957+0000","last_change":"2024-04-25T09:57:20.109564+0000","last_active":"2024-04-25T09:57:22.418957+0000","last_peered":"2024-04-25T09:57:22.418957+0000","last_clean":"2024-04-25T09:57:22.418957+0000","last_became_active":"2024-04-25T09:57:20.109298+0000","last_became_peered":"2024-04-25T09:57:20.109298+0000","last_unstale":"2024-04-25T09:57:22.418957+0000","last_undegraded":"2024-04-25T09:57:22.418957+0000","last_fullsized":"2024-04-25T09:57:22.418957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:46.611507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257423+0000","last_change":"2024-04-25T09:57:21.114715+0000","last_active":"2024-04-25T09:57:22.257423+0000","last_peered":"2024-04-25T09:57:22.257423+0000","last_clean":"2024-04-25T09:57:22.257423+0000","last_became_active":"2024-04-25T09:57:21.113989+0000","last_became_peered":"2024-04-25T09:57:21.113989+0000","last_unstale":"2024-04-25T09:57:22.257423+0000","last_undegraded":"2024-04-25T09:57:22.257423+0000","last_fullsized":"2024-04-25T09:57:22.257423+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:28.226431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"75'6","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137419+0000","last_change":"2024-04-25T09:57:20.112803+0000","last_active":"2024-04-25T09:57:22.137419+0000","last_peered":"2024-04-25T09:57:22.137419+0000","last_clean":"2024-04-25T09:57:22.137419+0000","last_became_active":"2024-04-25T09:57:20.112522+0000","last_became_peered":"2024-04-25T09:57:20.112522+0000","last_unstale":"2024-04-25T09:57:22.137419+0000","last_undegraded":"2024-04-25T09:57:22.137419+0000","last_fullsized":"2024-04-25T09:57:22.137419+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:08.519297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360288+0000","last_change":"2024-04-25T09:57:21.113149+0000","last_active":"2024-04-25T09:57:22.360288+0000","last_peered":"2024-04-25T09:57:22.360288+0000","last_clean":"2024-04-25T09:57:22.360288+0000","last_became_active":"2024-04-25T09:57:21.112750+0000","last_became_peered":"2024-04-25T09:57:21.112750+0000","last_unstale":"2024-04-25T09:57:22.360288+0000","last_undegraded":"2024-04-25T09:57:22.360288+0000","last_fullsized":"2024-04-25T09:57:22.360288+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:14.897617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.359928+0000","last_change":"2024-04-25T09:57:20.108560+0000","last_active":"2024-04-25T09:57:22.359928+0000","last_peered":"2024-04-25T09:57:22.359928+0000","last_clean":"2024-04-25T09:57:22.359928+0000","last_became_active":"2024-04-25T09:57:20.108279+0000","last_became_peered":"2024-04-25T09:57:20.108279+0000","last_unstale":"2024-04-25T09:57:22.359928+0000","last_undegraded":"2024-04-25T09:57:22.359928+0000","last_fullsized":"2024-04-25T09:57:22.359928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:22.462199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137463+0000","last_change":"2024-04-25T09:57:21.108479+0000","last_active":"2024-04-25T09:57:22.137463+0000","last_peered":"2024-04-25T09:57:22.137463+0000","last_clean":"2024-04-25T09:57:22.137463+0000","last_became_active":"2024-04-25T09:57:21.108155+0000","last_became_peered":"2024-04-25T09:57:21.108155+0000","last_unstale":"2024-04-25T09:57:22.137463+0000","last_undegraded":"2024-04-25T09:57:22.137463+0000","last_fullsized":"2024-04-25T09:57:22.137463+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:43.569154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618200+0000","last_change":"2024-04-25T09:57:20.106999+0000","last_active":"2024-04-25T09:57:22.618200+0000","last_peered":"2024-04-25T09:57:22.618200+0000","last_clean":"2024-04-25T09:57:22.618200+0000","last_became_active":"2024-04-25T09:57:20.106088+0000","last_became_peered":"2024-04-25T09:57:20.106088+0000","last_unstale":"2024-04-25T09:57:22.618200+0000","last_undegraded":"2024-04-25T09:57:22.618200+0000","last_fullsized":"2024-04-25T09:57:22.618200+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:54.488223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137576+0000","last_change":"2024-04-25T09:57:21.120546+0000","last_active":"2024-04-25T09:57:22.137576+0000","last_peered":"2024-04-25T09:57:22.137576+0000","last_clean":"2024-04-25T09:57:22.137576+0000","last_became_active":"2024-04-25T09:57:21.119633+0000","last_became_peered":"2024-04-25T09:57:21.119633+0000","last_unstale":"2024-04-25T09:57:22.137576+0000","last_undegraded":"2024-04-25T09:57:22.137576+0000","last_fullsized":"2024-04-25T09:57:22.137576+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:40.230560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418617+0000","last_change":"2024-04-25T09:57:20.109834+0000","last_active":"2024-04-25T09:57:22.418617+0000","last_peered":"2024-04-25T09:57:22.418617+0000","last_clean":"2024-04-25T09:57:22.418617+0000","last_became_active":"2024-04-25T09:57:20.106429+0000","last_became_peered":"2024-04-25T09:57:20.106429+0000","last_unstale":"2024-04-25T09:57:22.418617+0000","last_undegraded":"2024-04-25T09:57:22.418617+0000","last_fullsized":"2024-04-25T09:57:22.418617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:40.571580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137630+0000","last_change":"2024-04-25T09:57:21.113824+0000","last_active":"2024-04-25T09:57:22.137630+0000","last_peered":"2024-04-25T09:57:22.137630+0000","last_clean":"2024-04-25T09:57:22.137630+0000","last_became_active":"2024-04-25T09:57:21.113539+0000","last_became_peered":"2024-04-25T09:57:21.113539+0000","last_unstale":"2024-04-25T09:57:22.137630+0000","last_undegraded":"2024-04-25T09:57:22.137630+0000","last_fullsized":"2024-04-25T09:57:22.137630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:31.229689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"75'8","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113463+0000","last_change":"2024-04-25T09:57:20.107762+0000","last_active":"2024-04-25T09:57:22.113463+0000","last_peered":"2024-04-25T09:57:22.113463+0000","last_clean":"2024-04-25T09:57:22.113463+0000","last_became_active":"2024-04-25T09:57:20.107415+0000","last_became_peered":"2024-04-25T09:57:20.107415+0000","last_unstale":"2024-04-25T09:57:22.113463+0000","last_undegraded":"2024-04-25T09:57:22.113463+0000","last_fullsized":"2024-04-25T09:57:22.113463+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:39.225413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617743+0000","last_change":"2024-04-25T09:57:21.105218+0000","last_active":"2024-04-25T09:57:22.617743+0000","last_peered":"2024-04-25T09:57:22.617743+0000","last_clean":"2024-04-25T09:57:22.617743+0000","last_became_active":"2024-04-25T09:57:21.104949+0000","last_became_peered":"2024-04-25T09:57:21.104949+0000","last_unstale":"2024-04-25T09:57:22.617743+0000","last_undegraded":"2024-04-25T09:57:22.617743+0000","last_fullsized":"2024-04-25T09:57:22.617743+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:27.828672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618688+0000","last_change":"2024-04-25T09:57:20.110970+0000","last_active":"2024-04-25T09:57:22.618688+0000","last_peered":"2024-04-25T09:57:22.618688+0000","last_clean":"2024-04-25T09:57:22.618688+0000","last_became_active":"2024-04-25T09:57:20.110691+0000","last_became_peered":"2024-04-25T09:57:20.110691+0000","last_unstale":"2024-04-25T09:57:22.618688+0000","last_undegraded":"2024-04-25T09:57:22.618688+0000","last_fullsized":"2024-04-25T09:57:22.618688+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:32.037850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418605+0000","last_change":"2024-04-25T09:57:21.114261+0000","last_active":"2024-04-25T09:57:22.418605+0000","last_peered":"2024-04-25T09:57:22.418605+0000","last_clean":"2024-04-25T09:57:22.418605+0000","last_became_active":"2024-04-25T09:57:21.113925+0000","last_became_peered":"2024-04-25T09:57:21.113925+0000","last_unstale":"2024-04-25T09:57:22.418605+0000","last_undegraded":"2024-04-25T09:57:22.418605+0000","last_fullsized":"2024-04-25T09:57:22.418605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:40.528488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.111219+0000","last_change":"2024-04-25T09:57:20.112382+0000","last_active":"2024-04-25T09:57:22.111219+0000","last_peered":"2024-04-25T09:57:22.111219+0000","last_clean":"2024-04-25T09:57:22.111219+0000","last_became_active":"2024-04-25T09:57:20.112100+0000","last_became_peered":"2024-04-25T09:57:20.112100+0000","last_unstale":"2024-04-25T09:57:22.111219+0000","last_undegraded":"2024-04-25T09:57:22.111219+0000","last_fullsized":"2024-04-25T09:57:22.111219+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:52.500111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418687+0000","last_change":"2024-04-25T09:57:21.112450+0000","last_active":"2024-04-25T09:57:22.418687+0000","last_peered":"2024-04-25T09:57:22.418687+0000","last_clean":"2024-04-25T09:57:22.418687+0000","last_became_active":"2024-04-25T09:57:21.112193+0000","last_became_peered":"2024-04-25T09:57:21.112193+0000","last_unstale":"2024-04-25T09:57:22.418687+0000","last_undegraded":"2024-04-25T09:57:22.418687+0000","last_fullsized":"2024-04-25T09:57:22.418687+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:14.017954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.419051+0000","last_change":"2024-04-25T09:57:20.102910+0000","last_active":"2024-04-25T09:57:22.419051+0000","last_peered":"2024-04-25T09:57:22.419051+0000","last_clean":"2024-04-25T09:57:22.419051+0000","last_became_active":"2024-04-25T09:57:20.102635+0000","last_became_peered":"2024-04-25T09:57:20.102635+0000","last_unstale":"2024-04-25T09:57:22.419051+0000","last_undegraded":"2024-04-25T09:57:22.419051+0000","last_fullsized":"2024-04-25T09:57:22.419051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:46.978627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617873+0000","last_change":"2024-04-25T09:57:21.104013+0000","last_active":"2024-04-25T09:57:22.617873+0000","last_peered":"2024-04-25T09:57:22.617873+0000","last_clean":"2024-04-25T09:57:22.617873+0000","last_became_active":"2024-04-25T09:57:21.103636+0000","last_became_peered":"2024-04-25T09:57:21.103636+0000","last_unstale":"2024-04-25T09:57:22.617873+0000","last_undegraded":"2024-04-25T09:57:22.617873+0000","last_fullsized":"2024-04-25T09:57:22.617873+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:23.671368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257184+0000","last_change":"2024-04-25T09:57:20.109793+0000","last_active":"2024-04-25T09:57:22.257184+0000","last_peered":"2024-04-25T09:57:22.257184+0000","last_clean":"2024-04-25T09:57:22.257184+0000","last_became_active":"2024-04-25T09:57:20.109531+0000","last_became_peered":"2024-04-25T09:57:20.109531+0000","last_unstale":"2024-04-25T09:57:22.257184+0000","last_undegraded":"2024-04-25T09:57:22.257184+0000","last_fullsized":"2024-04-25T09:57:22.257184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:40.105458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113613+0000","last_change":"2024-04-25T09:57:21.111171+0000","last_active":"2024-04-25T09:57:22.113613+0000","last_peered":"2024-04-25T09:57:22.113613+0000","last_clean":"2024-04-25T09:57:22.113613+0000","last_became_active":"2024-04-25T09:57:21.110751+0000","last_became_peered":"2024-04-25T09:57:21.110751+0000","last_unstale":"2024-04-25T09:57:22.113613+0000","last_undegraded":"2024-04-25T09:57:22.113613+0000","last_fullsized":"2024-04-25T09:57:22.113613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:14.135232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617931+0000","last_change":"2024-04-25T09:57:20.109957+0000","last_active":"2024-04-25T09:57:22.617931+0000","last_peered":"2024-04-25T09:57:22.617931+0000","last_clean":"2024-04-25T09:57:22.617931+0000","last_became_active":"2024-04-25T09:57:20.109662+0000","last_became_peered":"2024-04-25T09:57:20.109662+0000","last_unstale":"2024-04-25T09:57:22.617931+0000","last_undegraded":"2024-04-25T09:57:22.617931+0000","last_fullsized":"2024-04-25T09:57:22.617931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:54.488223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113807+0000","last_change":"2024-04-25T09:57:21.123074+0000","last_active":"2024-04-25T09:57:22.113807+0000","last_peered":"2024-04-25T09:57:22.113807+0000","last_clean":"2024-04-25T09:57:22.113807+0000","last_became_active":"2024-04-25T09:57:21.122707+0000","last_became_peered":"2024-04-25T09:57:21.122707+0000","last_unstale":"2024-04-25T09:57:22.113807+0000","last_undegraded":"2024-04-25T09:57:22.113807+0000","last_fullsized":"2024-04-25T09:57:22.113807+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:25.099931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617743+0000","last_change":"2024-04-25T09:57:20.107160+0000","last_active":"2024-04-25T09:57:22.617743+0000","last_peered":"2024-04-25T09:57:22.617743+0000","last_clean":"2024-04-25T09:57:22.617743+0000","last_became_active":"2024-04-25T09:57:20.106839+0000","last_became_peered":"2024-04-25T09:57:20.106839+0000","last_unstale":"2024-04-25T09:57:22.617743+0000","last_undegraded":"2024-04-25T09:57:22.617743+0000","last_fullsized":"2024-04-25T09:57:22.617743+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:03.384506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137713+0000","last_change":"2024-04-25T09:57:21.109127+0000","last_active":"2024-04-25T09:57:22.137713+0000","last_peered":"2024-04-25T09:57:22.137713+0000","last_clean":"2024-04-25T09:57:22.137713+0000","last_became_active":"2024-04-25T09:57:21.108859+0000","last_became_peered":"2024-04-25T09:57:21.108859+0000","last_unstale":"2024-04-25T09:57:22.137713+0000","last_undegraded":"2024-04-25T09:57:22.137713+0000","last_fullsized":"2024-04-25T09:57:22.137713+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:17.764873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.617669+0000","last_change":"2024-04-25T09:57:20.106767+0000","last_active":"2024-04-25T09:57:22.617669+0000","last_peered":"2024-04-25T09:57:22.617669+0000","last_clean":"2024-04-25T09:57:22.617669+0000","last_became_active":"2024-04-25T09:57:20.106470+0000","last_became_peered":"2024-04-25T09:57:20.106470+0000","last_unstale":"2024-04-25T09:57:22.617669+0000","last_undegraded":"2024-04-25T09:57:22.617669+0000","last_fullsized":"2024-04-25T09:57:22.617669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:34.401685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.110739+0000","last_change":"2024-04-25T09:57:21.114427+0000","last_active":"2024-04-25T09:57:22.110739+0000","last_peered":"2024-04-25T09:57:22.110739+0000","last_clean":"2024-04-25T09:57:22.110739+0000","last_became_active":"2024-04-25T09:57:21.114147+0000","last_became_peered":"2024-04-25T09:57:21.114147+0000","last_unstale":"2024-04-25T09:57:22.110739+0000","last_undegraded":"2024-04-25T09:57:22.110739+0000","last_fullsized":"2024-04-25T09:57:22.110739+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:47.518329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.360143+0000","last_change":"2024-04-25T09:57:20.110133+0000","last_active":"2024-04-25T09:57:22.360143+0000","last_peered":"2024-04-25T09:57:22.360143+0000","last_clean":"2024-04-25T09:57:22.360143+0000","last_became_active":"2024-04-25T09:57:20.109848+0000","last_became_peered":"2024-04-25T09:57:20.109848+0000","last_unstale":"2024-04-25T09:57:22.360143+0000","last_undegraded":"2024-04-25T09:57:22.360143+0000","last_fullsized":"2024-04-25T09:57:22.360143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:30.224542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137376+0000","last_change":"2024-04-25T09:57:21.108633+0000","last_active":"2024-04-25T09:57:22.137376+0000","last_peered":"2024-04-25T09:57:22.137376+0000","last_clean":"2024-04-25T09:57:22.137376+0000","last_became_active":"2024-04-25T09:57:21.108213+0000","last_became_peered":"2024-04-25T09:57:21.108213+0000","last_unstale":"2024-04-25T09:57:22.137376+0000","last_undegraded":"2024-04-25T09:57:22.137376+0000","last_fullsized":"2024-04-25T09:57:22.137376+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:28.226431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113785+0000","last_change":"2024-04-25T09:57:20.100845+0000","last_active":"2024-04-25T09:57:22.113785+0000","last_peered":"2024-04-25T09:57:22.113785+0000","last_clean":"2024-04-25T09:57:22.113785+0000","last_became_active":"2024-04-25T09:57:20.100556+0000","last_became_peered":"2024-04-25T09:57:20.100556+0000","last_unstale":"2024-04-25T09:57:22.113785+0000","last_undegraded":"2024-04-25T09:57:22.113785+0000","last_fullsized":"2024-04-25T09:57:22.113785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:22.462199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.111145+0000","last_change":"2024-04-25T09:57:21.115393+0000","last_active":"2024-04-25T09:57:22.111145+0000","last_peered":"2024-04-25T09:57:22.111145+0000","last_clean":"2024-04-25T09:57:22.111145+0000","last_became_active":"2024-04-25T09:57:21.114967+0000","last_became_peered":"2024-04-25T09:57:21.114967+0000","last_unstale":"2024-04-25T09:57:22.111145+0000","last_undegraded":"2024-04-25T09:57:22.111145+0000","last_fullsized":"2024-04-25T09:57:22.111145+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:06.740795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618671+0000","last_change":"2024-04-25T09:57:20.108428+0000","last_active":"2024-04-25T09:57:22.618671+0000","last_peered":"2024-04-25T09:57:22.618671+0000","last_clean":"2024-04-25T09:57:22.618671+0000","last_became_active":"2024-04-25T09:57:20.107988+0000","last_became_peered":"2024-04-25T09:57:20.107988+0000","last_unstale":"2024-04-25T09:57:22.618671+0000","last_undegraded":"2024-04-25T09:57:22.618671+0000","last_fullsized":"2024-04-25T09:57:22.618671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:46.460376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.113695+0000","last_change":"2024-04-25T09:57:21.113961+0000","last_active":"2024-04-25T09:57:22.113695+0000","last_peered":"2024-04-25T09:57:22.113695+0000","last_clean":"2024-04-25T09:57:22.113695+0000","last_became_active":"2024-04-25T09:57:21.113691+0000","last_became_peered":"2024-04-25T09:57:21.113691+0000","last_unstale":"2024-04-25T09:57:22.113695+0000","last_undegraded":"2024-04-25T09:57:22.113695+0000","last_fullsized":"2024-04-25T09:57:22.113695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:44.826600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618256+0000","last_change":"2024-04-25T09:57:20.107995+0000","last_active":"2024-04-25T09:57:22.618256+0000","last_peered":"2024-04-25T09:57:22.618256+0000","last_clean":"2024-04-25T09:57:22.618256+0000","last_became_active":"2024-04-25T09:57:20.107706+0000","last_became_peered":"2024-04-25T09:57:20.107706+0000","last_unstale":"2024-04-25T09:57:22.618256+0000","last_undegraded":"2024-04-25T09:57:22.618256+0000","last_fullsized":"2024-04-25T09:57:22.618256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:42.599662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137340+0000","last_change":"2024-04-25T09:57:21.111614+0000","last_active":"2024-04-25T09:57:22.137340+0000","last_peered":"2024-04-25T09:57:22.137340+0000","last_clean":"2024-04-25T09:57:22.137340+0000","last_became_active":"2024-04-25T09:57:21.111356+0000","last_became_peered":"2024-04-25T09:57:21.111356+0000","last_unstale":"2024-04-25T09:57:22.137340+0000","last_undegraded":"2024-04-25T09:57:22.137340+0000","last_fullsized":"2024-04-25T09:57:22.137340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:22.534444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137234+0000","last_change":"2024-04-25T09:57:20.112262+0000","last_active":"2024-04-25T09:57:22.137234+0000","last_peered":"2024-04-25T09:57:22.137234+0000","last_clean":"2024-04-25T09:57:22.137234+0000","last_became_active":"2024-04-25T09:57:20.111983+0000","last_became_peered":"2024-04-25T09:57:20.111983+0000","last_unstale":"2024-04-25T09:57:22.137234+0000","last_undegraded":"2024-04-25T09:57:22.137234+0000","last_fullsized":"2024-04-25T09:57:22.137234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:11.053241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137472+0000","last_change":"2024-04-25T09:57:21.117031+0000","last_active":"2024-04-25T09:57:22.137472+0000","last_peered":"2024-04-25T09:57:22.137472+0000","last_clean":"2024-04-25T09:57:22.137472+0000","last_became_active":"2024-04-25T09:57:21.116726+0000","last_became_peered":"2024-04-25T09:57:21.116726+0000","last_unstale":"2024-04-25T09:57:22.137472+0000","last_undegraded":"2024-04-25T09:57:22.137472+0000","last_fullsized":"2024-04-25T09:57:22.137472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:58.703264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.418782+0000","last_change":"2024-04-25T09:57:20.104924+0000","last_active":"2024-04-25T09:57:22.418782+0000","last_peered":"2024-04-25T09:57:22.418782+0000","last_clean":"2024-04-25T09:57:22.418782+0000","last_became_active":"2024-04-25T09:57:20.104649+0000","last_became_peered":"2024-04-25T09:57:20.104649+0000","last_unstale":"2024-04-25T09:57:22.418782+0000","last_undegraded":"2024-04-25T09:57:22.418782+0000","last_fullsized":"2024-04-25T09:57:22.418782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:37.712850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257334+0000","last_change":"2024-04-25T09:57:21.112958+0000","last_active":"2024-04-25T09:57:22.257334+0000","last_peered":"2024-04-25T09:57:22.257334+0000","last_clean":"2024-04-25T09:57:22.257334+0000","last_became_active":"2024-04-25T09:57:21.112628+0000","last_became_peered":"2024-04-25T09:57:21.112628+0000","last_unstale":"2024-04-25T09:57:22.257334+0000","last_undegraded":"2024-04-25T09:57:22.257334+0000","last_fullsized":"2024-04-25T09:57:22.257334+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:22.534444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.137489+0000","last_change":"2024-04-25T09:57:20.106732+0000","last_active":"2024-04-25T09:57:22.137489+0000","last_peered":"2024-04-25T09:57:22.137489+0000","last_clean":"2024-04-25T09:57:22.137489+0000","last_became_active":"2024-04-25T09:57:20.106452+0000","last_became_peered":"2024-04-25T09:57:20.106452+0000","last_unstale":"2024-04-25T09:57:22.137489+0000","last_undegraded":"2024-04-25T09:57:22.137489+0000","last_fullsized":"2024-04-25T09:57:22.137489+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:27.279085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257474+0000","last_change":"2024-04-25T09:57:21.109731+0000","last_active":"2024-04-25T09:57:22.257474+0000","last_peered":"2024-04-25T09:57:22.257474+0000","last_clean":"2024-04-25T09:57:22.257474+0000","last_became_active":"2024-04-25T09:57:21.109454+0000","last_became_peered":"2024-04-25T09:57:21.109454+0000","last_unstale":"2024-04-25T09:57:22.257474+0000","last_undegraded":"2024-04-25T09:57:22.257474+0000","last_fullsized":"2024-04-25T09:57:22.257474+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:43.569154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.257011+0000","last_change":"2024-04-25T09:57:20.109206+0000","last_active":"2024-04-25T09:57:22.257011+0000","last_peered":"2024-04-25T09:57:22.257011+0000","last_clean":"2024-04-25T09:57:22.257011+0000","last_became_active":"2024-04-25T09:57:20.108272+0000","last_became_peered":"2024-04-25T09:57:20.108272+0000","last_unstale":"2024-04-25T09:57:22.257011+0000","last_undegraded":"2024-04-25T09:57:22.257011+0000","last_fullsized":"2024-04-25T09:57:22.257011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:11.053241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618598+0000","last_change":"2024-04-25T09:57:21.108222+0000","last_active":"2024-04-25T09:57:22.618598+0000","last_peered":"2024-04-25T09:57:22.618598+0000","last_clean":"2024-04-25T09:57:22.618598+0000","last_became_active":"2024-04-25T09:57:21.107928+0000","last_became_peered":"2024-04-25T09:57:21.107928+0000","last_unstale":"2024-04-25T09:57:22.618598+0000","last_undegraded":"2024-04-25T09:57:22.618598+0000","last_fullsized":"2024-04-25T09:57:22.618598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:20.086235+0000","last_clean_scrub_stamp":"2024-04-25T09:57:20.086235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:25.099931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618076+0000","last_change":"2024-04-25T09:57:20.107108+0000","last_active":"2024-04-25T09:57:22.618076+0000","last_peered":"2024-04-25T09:57:22.618076+0000","last_clean":"2024-04-25T09:57:22.618076+0000","last_became_active":"2024-04-25T09:57:20.106402+0000","last_became_peered":"2024-04-25T09:57:20.106402+0000","last_unstale":"2024-04-25T09:57:22.618076+0000","last_undegraded":"2024-04-25T09:57:22.618076+0000","last_fullsized":"2024-04-25T09:57:22.618076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:19.081088+0000","last_clean_scrub_stamp":"2024-04-25T09:57:19.081088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:24.866748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":273,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T09:57:22.618399+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:57:22.618399+0000","last_peered":"2024-04-25T09:57:22.618399+0000","last_clean":"2024-04-25T09:57:22.618399+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:57:22.618399+0000","last_undegraded":"2024-04-25T09:57:22.618399+0000","last_fullsized":"2024-04-25T09:57:22.618399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":1013482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":424704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":921600,"data_stored":416434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":950272,"data_stored":424808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":423674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":929792,"data_stored":417142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":423204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1523712,"data_stored":1013460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:57:24.388 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:57:24.388 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:57:24.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.524+0000 7f367ee4d640 1 Processor -- start 2024-04-25T09:57:24.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.524+0000 7f367ee4d640 1 -- start start 2024-04-25T09:57:24.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.525+0000 7f367ee4d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f36780af8a0 0x7f36780b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.525+0000 7f367ee4d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36780c6cc0 con 0x7f367806b700 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.525+0000 7f367ee4d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36780c6fa0 con 0x7f36780afc80 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.525+0000 7f367ee4d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36780c7280 con 0x7f36780af8a0 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.525+0000 7f367d3ea640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f367806b700 0x7f367806ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57074/0 (socket says 172.21.15.139:57074) 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.525+0000 7f367d3ea640 1 -- 172.21.15.139:0/1884813383 learned_addr learned my addr 172.21.15.139:0/1884813383 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.525+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839390869 0 0) 0x7f36780c6cc0 con 0x7f367806b700 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f367cbe9640 1 --2- 172.21.15.139:0/1884813383 >> v2:172.21.15.150:3300/0 conn(0x7f36780af8a0 0x7f36780b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3660003520 con 0x7f367806b700 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3516612511 0 0) 0x7f3660003520 con 0x7f367806b700 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 >> v2:172.21.15.150:3300/0 conn(0x7f36780af8a0 msgr2=0x7f36780b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 --2- 172.21.15.139:0/1884813383 >> v2:172.21.15.150:3300/0 conn(0x7f36780af8a0 0x7f36780b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 >> v1:172.21.15.150:6789/0 conn(0x7f36780afc80 legacy=0x7f36780cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36780cd090 con 0x7f367806b700 2024-04-25T09:57:24.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f36700019f0 con 0x7f367806b700 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 >> v1:172.21.15.139:6789/0 conn(0x7f367806b700 legacy=0x7f367806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 --2- 172.21.15.139:0/1884813383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36600039b0 0x7f3660003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 --2- 172.21.15.139:0/1884813383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3660004800 0x7f3660004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 --2- 172.21.15.139:0/1884813383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36600051c0 0x7f366000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36780c7280 con 0x7f3660004800 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366000c160 con 0x7f36600051c0 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366000c360 con 0x7f36600039b0 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f3677fff640 1 --2- 172.21.15.139:0/1884813383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3660004800 0x7f3660004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f367cbe9640 1 --2- 172.21.15.139:0/1884813383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36600039b0 0x7f3660003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f367d3ea640 1 --2- 172.21.15.139:0/1884813383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36600051c0 0x7f366000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f3677fff640 1 -- 172.21.15.139:0/1884813383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36600039b0 msgr2=0x7f3660003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f3677fff640 1 --2- 172.21.15.139:0/1884813383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36600039b0 0x7f3660003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f3677fff640 1 -- 172.21.15.139:0/1884813383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36600051c0 msgr2=0x7f366000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f3677fff640 1 --2- 172.21.15.139:0/1884813383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36600051c0 0x7f366000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f367d3ea640 1 -- 172.21.15.139:0/1884813383 reap_dead start 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.526+0000 7f3677fff640 1 -- 172.21.15.139:0/1884813383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3660003520 con 0x7f3660004800 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.527+0000 7f3677fff640 1 --2- 172.21.15.139:0/1884813383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3660004800 0x7f3660004c80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f366c001750 tx=0x7f3660005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0dfae2616d30f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.527+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c001370 con 0x7f3660004800 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.527+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c0052d0 con 0x7f3660004800 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.527+0000 7f36777fe640 1 -- 172.21.15.139:0/1884813383 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c02abc0 con 0x7f3660004800 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.528+0000 7f367ee4d640 1 -- 172.21.15.139:0/1884813383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3660004800 msgr2=0x7f3660004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.528+0000 7f367ee4d640 1 --2- 172.21.15.139:0/1884813383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3660004800 0x7f3660004c80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f366c001750 tx=0x7f3660005830 comp rx=0 tx=0).stop 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.528+0000 7f367ee4d640 1 -- 172.21.15.139:0/1884813383 shutdown_connections 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.528+0000 7f367ee4d640 1 --2- 172.21.15.139:0/1884813383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3660004800 0x7f3660004c80 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.528+0000 7f367ee4d640 1 -- 172.21.15.139:0/1884813383 >> 172.21.15.139:0/1884813383 conn(0x7f36780b2230 msgr2=0x7f36780b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.528+0000 7f367ee4d640 1 -- 172.21.15.139:0/1884813383 shutdown_connections 2024-04-25T09:57:24.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.528+0000 7f367ee4d640 1 -- 172.21.15.139:0/1884813383 wait complete. 2024-04-25T09:57:24.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.529+0000 7f367ee4d640 1 Processor -- start 2024-04-25T09:57:24.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.529+0000 7f367ee4d640 1 -- start start 2024-04-25T09:57:24.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f367ee4d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3660004800 0x7f3678152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f367ee4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36781538e0 0x7f3678153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f367ee4d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36780b10b0 0x7f36780b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f367ee4d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36780b2ed0 con 0x7f36781538e0 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f367ee4d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36780b2d50 con 0x7f36780b10b0 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f367ee4d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36780b3050 con 0x7f3660004800 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f3677fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36781538e0 0x7f3678153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f3677fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36781538e0 0x7f3678153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50018/0 (socket says 172.21.15.139:50018) 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f3677fff640 1 -- 172.21.15.139:0/1246695052 learned_addr learned my addr 172.21.15.139:0/1246695052 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f367cbe9640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3660004800 0x7f3678152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f367d3ea640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36780b10b0 0x7f36780b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f3677fff640 1 -- 172.21.15.139:0/1246695052 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3660004800 msgr2=0x7f3678152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f3677fff640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3660004800 0x7f3678152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f3677fff640 1 -- 172.21.15.139:0/1246695052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36780b10b0 msgr2=0x7f36780b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f3677fff640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36780b10b0 0x7f36780b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f3677fff640 1 -- 172.21.15.139:0/1246695052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36781531e0 con 0x7f36781538e0 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f367d3ea640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36780b10b0 0x7f36780b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f3677fff640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36781538e0 0x7f3678153d60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f366c005040 tx=0x7f366c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.530+0000 7f36757fa640 1 -- 172.21.15.139:0/1246695052 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c037040 con 0x7f36781538e0 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.531+0000 7f36757fa640 1 -- 172.21.15.139:0/1246695052 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c040070 con 0x7f36781538e0 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.531+0000 7f36757fa640 1 -- 172.21.15.139:0/1246695052 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c007a30 con 0x7f36781538e0 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.531+0000 7f367ee4d640 1 -- 172.21.15.139:0/1246695052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36781f6ee0 con 0x7f36781538e0 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.531+0000 7f367ee4d640 1 -- 172.21.15.139:0/1246695052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36781f74a0 con 0x7f36781538e0 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.531+0000 7f367ee4d640 1 -- 172.21.15.139:0/1246695052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36780b1a70 con 0x7f36781538e0 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.532+0000 7f36757fa640 1 -- 172.21.15.139:0/1246695052 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f366c0052d0 con 0x7f36781538e0 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.532+0000 7f36757fa640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f366406b4d0 0x7f366406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.532+0000 7f36757fa640 1 -- 172.21.15.139:0/1246695052 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f366c0aa060 con 0x7f36781538e0 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.532+0000 7f367cbe9640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f366406b4d0 0x7f366406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.533+0000 7f367cbe9640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f366406b4d0 0x7f366406d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3668009a00 tx=0x7f3668008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.536+0000 7f36757fa640 1 -- 172.21.15.139:0/1246695052 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f366c078cc0 con 0x7f36781538e0 2024-04-25T09:57:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.770+0000 7f367ee4d640 1 -- 172.21.15.139:0/1246695052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3678074520 con 0x7f36781538e0 2024-04-25T09:57:24.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.772+0000 7f36757fa640 1 -- 172.21.15.139:0/1246695052 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f366c0785c0 con 0x7f36781538e0 2024-04-25T09:57:24.775 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2024-04-25T09:57:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg== 2024-04-25T09:57:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: caps mds = "allow" 2024-04-25T09:57:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: caps mon = "allow r" 2024-04-25T09:57:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:57:24.776 INFO:teuthology.orchestra.run.smithi139.stderr:updated caps for client.0 2024-04-25T09:57:24.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.774+0000 7f367ee4d640 1 -- 172.21.15.139:0/1246695052 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f366406b4d0 msgr2=0x7f366406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.774+0000 7f367ee4d640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f366406b4d0 0x7f366406d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3668009a00 tx=0x7f3668008040 comp rx=0 tx=0).stop 2024-04-25T09:57:24.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.774+0000 7f367ee4d640 1 -- 172.21.15.139:0/1246695052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36781538e0 msgr2=0x7f3678153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.774+0000 7f367ee4d640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36781538e0 0x7f3678153d60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f366c005040 tx=0x7f366c001120 comp rx=0 tx=0).stop 2024-04-25T09:57:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.774+0000 7f367ee4d640 1 -- 172.21.15.139:0/1246695052 shutdown_connections 2024-04-25T09:57:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.774+0000 7f367ee4d640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36781538e0 0x7f3678153d60 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.774+0000 7f367ee4d640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f366406b4d0 0x7f366406d990 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.774+0000 7f367ee4d640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36780b10b0 0x7f36780b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.774+0000 7f367ee4d640 1 --2- 172.21.15.139:0/1246695052 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3660004800 0x7f3678152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.774+0000 7f367ee4d640 1 -- 172.21.15.139:0/1246695052 >> 172.21.15.139:0/1246695052 conn(0x7f36780b2230 msgr2=0x7f36780b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.774+0000 7f367ee4d640 1 -- 172.21.15.139:0/1246695052 shutdown_connections 2024-04-25T09:57:24.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.774+0000 7f367ee4d640 1 -- 172.21.15.139:0/1246695052 wait complete. 2024-04-25T09:57:24.788 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:24.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.882+0000 7f4acd66f640 1 Processor -- start 2024-04-25T09:57:24.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.882+0000 7f4acd66f640 1 -- start start 2024-04-25T09:57:24.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.883+0000 7f4acd66f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4ac815daf0 0x7f4ac806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.883+0000 7f4acd66f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815ded0 0x7f4ac80b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.883+0000 7f4acd66f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac806d050 con 0x7f4ac80cea10 2024-04-25T09:57:24.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.883+0000 7f4acd66f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac80ce580 con 0x7f4ac815daf0 2024-04-25T09:57:24.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.883+0000 7f4acd66f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80d3750 con 0x7f4ac815ded0 2024-04-25T09:57:24.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.883+0000 7f4ac67fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815ded0 0x7f4ac80b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.883+0000 7f4ac67fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815ded0 0x7f4ac80b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:44748/0 (socket says 172.21.15.139:44748) 2024-04-25T09:57:24.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.883+0000 7f4ac67fc640 1 -- 172.21.15.139:0/1904635275 learned_addr learned my addr 172.21.15.139:0/1904635275 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:24.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.883+0000 7f4ac6ffd640 1 --2- 172.21.15.139:0/1904635275 >> v2:172.21.15.150:3300/0 conn(0x7f4ac815daf0 0x7f4ac806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.884+0000 7f4ac67fc640 1 -- 172.21.15.139:0/1904635275 >> v2:172.21.15.150:3300/0 conn(0x7f4ac815daf0 msgr2=0x7f4ac806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.884+0000 7f4ac67fc640 1 --2- 172.21.15.139:0/1904635275 >> v2:172.21.15.150:3300/0 conn(0x7f4ac815daf0 0x7f4ac806cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.884+0000 7f4ac67fc640 1 -- 172.21.15.139:0/1904635275 >> v1:172.21.15.150:6789/0 conn(0x7f4ac80cea10 legacy=0x7f4ac81617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.884+0000 7f4ac67fc640 1 -- 172.21.15.139:0/1904635275 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac80ceea0 con 0x7f4ac815ded0 2024-04-25T09:57:24.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.884+0000 7f4ac6ffd640 1 --2- 172.21.15.139:0/1904635275 >> v2:172.21.15.150:3300/0 conn(0x7f4ac815daf0 0x7f4ac806cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:24.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.884+0000 7f4ac5ffb640 1 -- 172.21.15.139:0/1904635275 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772923596 0 0) 0x7f4ac806d050 con 0x7f4ac80cea10 2024-04-25T09:57:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.884+0000 7f4ac67fc640 1 --2- 172.21.15.139:0/1904635275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815ded0 0x7f4ac80b5450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009870 tx=0x7f4abc02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=741e43ad8d631801 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.884+0000 7f4ac5ffb640 1 -- 172.21.15.139:0/1904635275 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc037070 con 0x7f4ac815ded0 2024-04-25T09:57:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.884+0000 7f4ac5ffb640 1 -- 172.21.15.139:0/1904635275 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc032440 con 0x7f4ac815ded0 2024-04-25T09:57:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.884+0000 7f4ac5ffb640 1 -- 172.21.15.139:0/1904635275 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc032810 con 0x7f4ac815ded0 2024-04-25T09:57:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.885+0000 7f4acd66f640 1 -- 172.21.15.139:0/1904635275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815ded0 msgr2=0x7f4ac80b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.885+0000 7f4acd66f640 1 --2- 172.21.15.139:0/1904635275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815ded0 0x7f4ac80b5450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009870 tx=0x7f4abc02a250 comp rx=0 tx=0).stop 2024-04-25T09:57:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.885+0000 7f4acd66f640 1 -- 172.21.15.139:0/1904635275 shutdown_connections 2024-04-25T09:57:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.885+0000 7f4acd66f640 1 --2- 172.21.15.139:0/1904635275 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815ded0 0x7f4ac80b5450 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.885+0000 7f4acd66f640 1 --2- 172.21.15.139:0/1904635275 >> v2:172.21.15.150:3300/0 conn(0x7f4ac815daf0 0x7f4ac806cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.885+0000 7f4acd66f640 1 -- 172.21.15.139:0/1904635275 >> 172.21.15.139:0/1904635275 conn(0x7f4ac80d0450 msgr2=0x7f4ac80d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.885+0000 7f4acd66f640 1 -- 172.21.15.139:0/1904635275 shutdown_connections 2024-04-25T09:57:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.885+0000 7f4acd66f640 1 -- 172.21.15.139:0/1904635275 wait complete. 2024-04-25T09:57:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.886+0000 7f4acd66f640 1 Processor -- start 2024-04-25T09:57:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.886+0000 7f4acd66f640 1 -- start start 2024-04-25T09:57:24.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.887+0000 7f4acd66f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815daf0 0x7f4ac80c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.887+0000 7f4acd66f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac815ded0 0x7f4ac815f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.887+0000 7f4acd66f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ac815f840 0x7f4ac80c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.887+0000 7f4acd66f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80d1f40 con 0x7f4ac815f840 2024-04-25T09:57:24.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.887+0000 7f4acd66f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80cf180 con 0x7f4ac815ded0 2024-04-25T09:57:24.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.887+0000 7f4acd66f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80d2220 con 0x7f4ac815daf0 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.887+0000 7f4ac77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ac815f840 0x7f4ac80c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.887+0000 7f4ac77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ac815f840 0x7f4ac80c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50032/0 (socket says 172.21.15.139:50032) 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.887+0000 7f4ac77fe640 1 -- 172.21.15.139:0/684493601 learned_addr learned my addr 172.21.15.139:0/684493601 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.887+0000 7f4ac6ffd640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815daf0 0x7f4ac80c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.887+0000 7f4ac67fc640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac815ded0 0x7f4ac815f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4ac77fe640 1 -- 172.21.15.139:0/684493601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815daf0 msgr2=0x7f4ac80c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4ac77fe640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815daf0 0x7f4ac80c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4ac77fe640 1 -- 172.21.15.139:0/684493601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac815ded0 msgr2=0x7f4ac815f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4ac77fe640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac815ded0 0x7f4ac815f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4ac77fe640 1 -- 172.21.15.139:0/684493601 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac815fda0 con 0x7f4ac815f840 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4ac6ffd640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815daf0 0x7f4ac80c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4ac67fc640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac815ded0 0x7f4ac815f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4ac77fe640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ac815f840 0x7f4ac80c7330 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000c7b0 tx=0x7f4ac000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4aaf7fe640 1 -- 172.21.15.139:0/684493601 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0003ba0 con 0x7f4ac815f840 2024-04-25T09:57:24.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4aaf7fe640 1 -- 172.21.15.139:0/684493601 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac0003d40 con 0x7f4ac815f840 2024-04-25T09:57:24.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4acd66f640 1 -- 172.21.15.139:0/684493601 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac8160080 con 0x7f4ac815f840 2024-04-25T09:57:24.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4aaf7fe640 1 -- 172.21.15.139:0/684493601 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0005270 con 0x7f4ac815f840 2024-04-25T09:57:24.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.888+0000 7f4acd66f640 1 -- 172.21.15.139:0/684493601 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac8160640 con 0x7f4ac815f840 2024-04-25T09:57:24.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.889+0000 7f4acd66f640 1 -- 172.21.15.139:0/684493601 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac808b600 con 0x7f4ac815f840 2024-04-25T09:57:24.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.889+0000 7f4aaf7fe640 1 -- 172.21.15.139:0/684493601 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ac0005410 con 0x7f4ac815f840 2024-04-25T09:57:24.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.890+0000 7f4aaf7fe640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ab406b4d0 0x7f4ab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.890+0000 7f4aaf7fe640 1 -- 172.21.15.139:0/684493601 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4ac008a300 con 0x7f4ac815f840 2024-04-25T09:57:24.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.890+0000 7f4ac6ffd640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ab406b4d0 0x7f4ab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.890+0000 7f4ac6ffd640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ab406b4d0 0x7f4ab406d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009ba0 tx=0x7f4ab8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:24.894+0000 7f4aaf7fe640 1 -- 172.21.15.139:0/684493601 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ac808b600 con 0x7f4ac815f840 2024-04-25T09:57:25.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.100+0000 7f4acd66f640 1 -- 172.21.15.139:0/684493601 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ac8074520 con 0x7f4ac815f840 2024-04-25T09:57:25.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.101+0000 7f4aaf7fe640 1 -- 172.21.15.139:0/684493601 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f4ac0058f60 con 0x7f4ac815f840 2024-04-25T09:57:25.104 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:25.104 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1384380923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":1384380923},{"type":"v1","addr":"172.21.15.139:6837","nonce":1384380923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:20.444843+0000","modified":"2024-04-25T09:57:22.443106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6834/1932986598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1932986598},{"type":"v1","addr":"172.21.15.150:6834","nonce":1932986598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:57:25.105 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 36 2024-04-25T09:57:25.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.103+0000 7f4acd66f640 1 -- 172.21.15.139:0/684493601 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ab406b4d0 msgr2=0x7f4ab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.103+0000 7f4acd66f640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ab406b4d0 0x7f4ab406d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009ba0 tx=0x7f4ab8008040 comp rx=0 tx=0).stop 2024-04-25T09:57:25.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.103+0000 7f4acd66f640 1 -- 172.21.15.139:0/684493601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ac815f840 msgr2=0x7f4ac80c7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.103+0000 7f4acd66f640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ac815f840 0x7f4ac80c7330 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000c7b0 tx=0x7f4ac000cbb0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.103+0000 7f4acd66f640 1 -- 172.21.15.139:0/684493601 shutdown_connections 2024-04-25T09:57:25.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.103+0000 7f4acd66f640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ac815f840 0x7f4ac80c7330 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.103+0000 7f4acd66f640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ab406b4d0 0x7f4ab406d990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.103+0000 7f4acd66f640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac815ded0 0x7f4ac815f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.103+0000 7f4acd66f640 1 --2- 172.21.15.139:0/684493601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ac815daf0 0x7f4ac80c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.103+0000 7f4acd66f640 1 -- 172.21.15.139:0/684493601 >> 172.21.15.139:0/684493601 conn(0x7f4ac80d0450 msgr2=0x7f4ac80d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.103+0000 7f4acd66f640 1 -- 172.21.15.139:0/684493601 shutdown_connections 2024-04-25T09:57:25.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.103+0000 7f4acd66f640 1 -- 172.21.15.139:0/684493601 wait complete. 2024-04-25T09:57:25.519 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:57:20.444843+0000', 'modified': '2024-04-25T09:57:22.443106+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6885}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6885': {'gid': 6885, 'name': 'd', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6834/1932986598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6832', 'nonce': 1932986598}, {'type': 'v1', 'addr': '172.21.15.150:6834', 'nonce': 1932986598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:57:25.519 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:57:25.519 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T09:57:25.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.615+0000 7f6ac7963640 1 Processor -- start 2024-04-25T09:57:25.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.615+0000 7f6ac7963640 1 -- start start 2024-04-25T09:57:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.616+0000 7f6ac7963640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ac015dbf0 0x7f6ac01617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.616+0000 7f6ac7963640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ac00cee50 0x7f6ac00d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.616+0000 7f6ac7963640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac0161dc0 con 0x7f6ac00b9370 2024-04-25T09:57:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.616+0000 7f6ac7963640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac00d1ee0 con 0x7f6ac00cee50 2024-04-25T09:57:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.616+0000 7f6ac7963640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac01620a0 con 0x7f6ac015dbf0 2024-04-25T09:57:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.616+0000 7f6ac4efe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f6ac00b9370 0x7f6ac00ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57076/0 (socket says 172.21.15.139:57076) 2024-04-25T09:57:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.616+0000 7f6ac4efe640 1 -- 172.21.15.139:0/2236514440 learned_addr learned my addr 172.21.15.139:0/2236514440 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893693847 0 0) 0x7f6ac0161dc0 con 0x7f6ac00b9370 2024-04-25T09:57:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aac003520 con 0x7f6ac00b9370 2024-04-25T09:57:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ac5f00640 1 --2- 172.21.15.139:0/2236514440 >> v2:172.21.15.150:3300/0 conn(0x7f6ac00cee50 0x7f6ac00d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/2236514440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ac015dbf0 0x7f6ac01617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 864204243 0 0) 0x7f6aac003520 con 0x7f6ac00b9370 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ac015dbf0 msgr2=0x7f6ac01617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 --2- 172.21.15.139:0/2236514440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ac015dbf0 0x7f6ac01617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 >> v2:172.21.15.150:3300/0 conn(0x7f6ac00cee50 msgr2=0x7f6ac00d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 --2- 172.21.15.139:0/2236514440 >> v2:172.21.15.150:3300/0 conn(0x7f6ac00cee50 0x7f6ac00d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac0162380 con 0x7f6ac00b9370 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f6ab0001670 con 0x7f6ac00b9370 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 >> v1:172.21.15.139:6789/0 conn(0x7f6ac00b9370 legacy=0x7f6ac00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 --2- 172.21.15.139:0/2236514440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aac0039b0 0x7f6aac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 --2- 172.21.15.139:0/2236514440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ac015dbf0 0x7f6aac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 --2- 172.21.15.139:0/2236514440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aac004d40 0x7f6aac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac00d1ee0 con 0x7f6aac0039b0 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac01620a0 con 0x7f6aac004d40 2024-04-25T09:57:25.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac00b9c0 con 0x7f6ac015dbf0 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/2236514440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aac0039b0 0x7f6aac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ac5f00640 1 --2- 172.21.15.139:0/2236514440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aac004d40 0x7f6aac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ac4efe640 1 --2- 172.21.15.139:0/2236514440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ac015dbf0 0x7f6aac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ac56ff640 1 -- 172.21.15.139:0/2236514440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ac015dbf0 msgr2=0x7f6aac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/2236514440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ac015dbf0 0x7f6aac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ac56ff640 1 -- 172.21.15.139:0/2236514440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aac004d40 msgr2=0x7f6aac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/2236514440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aac004d40 0x7f6aac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.617+0000 7f6ac56ff640 1 -- 172.21.15.139:0/2236514440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac003520 con 0x7f6aac0039b0 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.618+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/2236514440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aac0039b0 0x7f6aac003dd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009fc0 tx=0x7f6aac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df19d77cb6c4d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.618+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc009c10 con 0x7f6aac0039b0 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.618+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc036070 con 0x7f6aac0039b0 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.618+0000 7f6ab7fff640 1 -- 172.21.15.139:0/2236514440 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc031880 con 0x7f6aac0039b0 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.619+0000 7f6ac7963640 1 -- 172.21.15.139:0/2236514440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aac0039b0 msgr2=0x7f6aac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.619+0000 7f6ac7963640 1 --2- 172.21.15.139:0/2236514440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6aac0039b0 0x7f6aac003dd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009fc0 tx=0x7f6aac005830 comp rx=0 tx=0).stop 2024-04-25T09:57:25.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.619+0000 7f6ac5f00640 1 -- 172.21.15.139:0/2236514440 reap_dead start 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.619+0000 7f6ac7963640 1 -- 172.21.15.139:0/2236514440 shutdown_connections 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.619+0000 7f6ac7963640 1 -- 172.21.15.139:0/2236514440 >> 172.21.15.139:0/2236514440 conn(0x7f6ac00b5480 msgr2=0x7f6ac00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.619+0000 7f6ac7963640 1 -- 172.21.15.139:0/2236514440 shutdown_connections 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.619+0000 7f6ac7963640 1 -- 172.21.15.139:0/2236514440 wait complete. 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.620+0000 7f6ac7963640 1 Processor -- start 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.620+0000 7f6ac7963640 1 -- start start 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac7963640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aac0039b0 0x7f6ac00d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac7963640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ac015ecf0 0x7f6ac015f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac7963640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ac0151a60 0x7f6ac0151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac7963640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac015dfd0 con 0x7f6ac015ecf0 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac7963640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac00b97d0 con 0x7f6ac0151a60 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac7963640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac0159b40 con 0x7f6aac0039b0 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac4efe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ac015ecf0 0x7f6ac015f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac4efe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ac015ecf0 0x7f6ac015f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50048/0 (socket says 172.21.15.139:50048) 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac4efe640 1 -- 172.21.15.139:0/3786504047 learned_addr learned my addr 172.21.15.139:0/3786504047 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aac0039b0 0x7f6ac00d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac5f00640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ac0151a60 0x7f6ac0151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac4efe640 1 -- 172.21.15.139:0/3786504047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aac0039b0 msgr2=0x7f6ac00d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac4efe640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aac0039b0 0x7f6ac00d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac4efe640 1 -- 172.21.15.139:0/3786504047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ac0151a60 msgr2=0x7f6ac0151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac4efe640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ac0151a60 0x7f6ac0151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac4efe640 1 -- 172.21.15.139:0/3786504047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac0152420 con 0x7f6ac015ecf0 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac4efe640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ac015ecf0 0x7f6ac015f170 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0007340 tx=0x7f6ab0007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ab5ffb640 1 -- 172.21.15.139:0/3786504047 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000f940 con 0x7f6ac015ecf0 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ab5ffb640 1 -- 172.21.15.139:0/3786504047 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab0018070 con 0x7f6ac015ecf0 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ac7963640 1 -- 172.21.15.139:0/3786504047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac015f7b0 con 0x7f6ac015ecf0 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.621+0000 7f6ab5ffb640 1 -- 172.21.15.139:0/3786504047 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000fd00 con 0x7f6ac015ecf0 2024-04-25T09:57:25.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.622+0000 7f6ac7963640 1 -- 172.21.15.139:0/3786504047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac015fd70 con 0x7f6ac015ecf0 2024-04-25T09:57:25.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.623+0000 7f6ab5ffb640 1 -- 172.21.15.139:0/3786504047 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ab001d460 con 0x7f6ac015ecf0 2024-04-25T09:57:25.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.623+0000 7f6ab5ffb640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6aa406b4d0 0x7f6aa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.624+0000 7f6ab5ffb640 1 -- 172.21.15.139:0/3786504047 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ab008a2f0 con 0x7f6ac015ecf0 2024-04-25T09:57:25.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.624+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6aa406b4d0 0x7f6aa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.624+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6aa406b4d0 0x7f6aa406d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6abc035000 tx=0x7f6abc02f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.624+0000 7f6aca66d740 1 Processor -- start 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.625+0000 7f6aca66d740 1 -- start start 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.625+0000 7f6aca66d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df75e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.625+0000 7f6aca66d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df77c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.625+0000 7f6aca66d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df733f10 0x5593df77a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.625+0000 7f6aca66d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593df696a10 con 0x5593df77bad0 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.625+0000 7f6aca66d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593df63fb30 con 0x5593df75e230 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.625+0000 7f6aca66d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593df692c50 con 0x5593df733f10 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.625+0000 7f6ac4efe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df77c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.625+0000 7f6ac4efe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df77c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50062/0 (socket says 172.21.15.139:50062) 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.625+0000 7f6ac4efe640 1 -- 172.21.15.139:0/1610217508 learned_addr learned my addr 172.21.15.139:0/1610217508 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.625+0000 7f6ac5f00640 1 --2- 172.21.15.139:0/1610217508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df733f10 0x5593df77a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.625+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/1610217508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df75e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6ac4efe640 1 -- 172.21.15.139:0/1610217508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df733f10 msgr2=0x5593df77a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6ac4efe640 1 --2- 172.21.15.139:0/1610217508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df733f10 0x5593df77a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6ac4efe640 1 -- 172.21.15.139:0/1610217508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 msgr2=0x5593df75e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6ac4efe640 1 --2- 172.21.15.139:0/1610217508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df75e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6ac4efe640 1 -- 172.21.15.139:0/1610217508 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593df654e10 con 0x5593df77bad0 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/1610217508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df75e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6ac4efe640 1 --2- 172.21.15.139:0/1610217508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df77c020 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0054dd0 tx=0x7f6ab000ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a6224e8f9782db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6aaaffd640 1 -- 172.21.15.139:0/1610217508 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab005cac0 con 0x5593df77bad0 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6aaaffd640 1 -- 172.21.15.139:0/1610217508 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab0061080 con 0x5593df77bad0 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6aaaffd640 1 -- 172.21.15.139:0/1610217508 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0056c70 con 0x5593df77bad0 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6aca66d740 1 -- 172.21.15.139:0/1610217508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 msgr2=0x5593df77c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6aca66d740 1 --2- 172.21.15.139:0/1610217508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df77c020 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0054dd0 tx=0x7f6ab000ff60 comp rx=0 tx=0).stop 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6aca66d740 1 -- 172.21.15.139:0/1610217508 shutdown_connections 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6aca66d740 1 --2- 172.21.15.139:0/1610217508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df77c020 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6aca66d740 1 --2- 172.21.15.139:0/1610217508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df733f10 0x5593df77a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6aca66d740 1 --2- 172.21.15.139:0/1610217508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df75e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6aca66d740 1 -- 172.21.15.139:0/1610217508 >> 172.21.15.139:0/1610217508 conn(0x5593df730840 msgr2=0x5593df779040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6aca66d740 1 -- 172.21.15.139:0/1610217508 shutdown_connections 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.626+0000 7f6aca66d740 1 -- 172.21.15.139:0/1610217508 wait complete. 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.627+0000 7f6aca66d740 1 -- start start 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.627+0000 7f6aca66d740 1 Processor -- start 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6aca66d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df747c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6aca66d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df789af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6aca66d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df74deb0 0x5593df747260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6aca66d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593df734cb0 con 0x5593df77bad0 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6aca66d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593df6530c0 con 0x5593df75e230 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6aca66d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593df72e770 con 0x5593df74deb0 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6aca66d740 10 client.0 _ll_register_callbacks cb 0x5593df646ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6ac4efe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df789af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6ac4efe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df789af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50072/0 (socket says 172.21.15.139:50072) 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6ac4efe640 1 -- 172.21.15.139:0/812519621 learned_addr learned my addr 172.21.15.139:0/812519621 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:25.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df747c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6ac5f00640 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df74deb0 0x5593df747260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6ac4efe640 1 -- 172.21.15.139:0/812519621 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df74deb0 msgr2=0x5593df747260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6ac4efe640 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df74deb0 0x5593df747260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6ac4efe640 1 -- 172.21.15.139:0/812519621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 msgr2=0x5593df747c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6ac4efe640 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df747c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6ac4efe640 1 -- 172.21.15.139:0/812519621 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593df3ab2d0 con 0x5593df77bad0 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6ac4efe640 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df789af0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0088770 tx=0x7f6ab0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e374335e35e3ae9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.628+0000 7f6a9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6a9ffff640 1 -- 172.21.15.139:0/812519621 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0088370 con 0x5593df77bad0 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6a9ffff640 1 -- 172.21.15.139:0/812519621 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ab0057dc0 con 0x5593df77bad0 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6aca66d740 1 -- 172.21.15.139:0/812519621 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ab0061080 con 0x5593df77bad0 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6a9ffff640 1 -- 172.21.15.139:0/812519621 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ab0057fe0 con 0x5593df77bad0 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6aca66d740 10 client.7017 fetch_fsmap learned FSMap version 36 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6aca66d740 1 -- 172.21.15.139:0/812519621 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x5593df4756b0 con 0x5593df77bad0 2024-04-25T09:57:25.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6a9ffff640 1 -- 172.21.15.139:0/812519621 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab0061080 con 0x5593df77bad0 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6a9ffff640 1 -- 172.21.15.139:0/812519621 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6ab00701c0 con 0x5593df77bad0 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6aca66d740 10 client.7017 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6aca66d740 10 client.7017 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6aca66d740 20 client.7017 populate_metadata read hostname 'smithi139' 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6aca66d740 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.150:6832/1932986598,v1:172.21.15.150:6834/1932986598] conn(0x5593df77f060 0x5593df74d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6aca66d740 4 client.7017 mds_command: new command op to 6885 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6aca66d740 1 -- 172.21.15.139:0/812519621 --> [v2:172.21.15.150:6832/1932986598,v1:172.21.15.150:6834/1932986598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593df6f5920 con 0x5593df77f060 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.629+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.150:6832/1932986598,v1:172.21.15.150:6834/1932986598] conn(0x5593df77f060 0x5593df74d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.630+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.150:6832/1932986598,v1:172.21.15.150:6834/1932986598] conn(0x5593df77f060 0x5593df74d160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bade0f90262a3e24 server_cookie=17cfd43b3bae25c7 in_seq=0 out_seq=0 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.630+0000 7f6a9ffff640 10 client.7017 ms_handle_connect on v2:172.21.15.150:6832/1932986598 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.630+0000 7f6a9ffff640 1 -- 172.21.15.139:0/812519621 <== mds.0 v2:172.21.15.150:6832/1932986598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5593df6f5920 con 0x5593df77f060 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.630+0000 7f6a9ffff640 10 client.7017 handle_command_reply: tid=0 2024-04-25T09:57:25.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.630+0000 7f6aca66d740 1 client.7017 shutdown 2024-04-25T09:57:25.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.630+0000 7f6aca66d740 1 -- 172.21.15.139:0/812519621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 msgr2=0x5593df789af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.630+0000 7f6aca66d740 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df789af0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0088770 tx=0x7f6ab0015040 comp rx=0 tx=0).stop 2024-04-25T09:57:25.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.630+0000 7f6aca66d740 1 -- 172.21.15.139:0/812519621 shutdown_connections 2024-04-25T09:57:25.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.630+0000 7f6aca66d740 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.150:6832/1932986598,v1:172.21.15.150:6834/1932986598] conn(0x5593df77f060 0x5593df74d160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.630+0000 7f6aca66d740 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df789af0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.630+0000 7f6aca66d740 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df74deb0 0x5593df747260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.630+0000 7f6aca66d740 1 --2- 172.21.15.139:0/812519621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df747c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.631+0000 7f6a9ffff640 0 client.7017 ms_handle_reset on v2:172.21.15.150:6832/1932986598 2024-04-25T09:57:25.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.631+0000 7f6aca66d740 1 -- 172.21.15.139:0/812519621 >> 172.21.15.139:0/812519621 conn(0x5593df733f10 msgr2=0x5593df7787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.631+0000 7f6aca66d740 1 -- 172.21.15.139:0/812519621 shutdown_connections 2024-04-25T09:57:25.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.631+0000 7f6aca66d740 1 -- 172.21.15.139:0/812519621 wait complete. 2024-04-25T09:57:25.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.631+0000 7f6aca66d740 20 client.7017 trim_cache size 0 max 16384 2024-04-25T09:57:25.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.646+0000 7f6aca66d740 1 Processor -- start 2024-04-25T09:57:25.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.646+0000 7f6aca66d740 1 -- start start 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6aca66d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df74deb0 0x5593df75f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6aca66d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df77bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6aca66d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df77bad0 0x5593df77c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6aca66d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac01ffc20 con 0x5593df74deb0 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6aca66d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac01ff020 con 0x5593df75e230 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6aca66d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac01ff620 con 0x5593df77bad0 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6ac56ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df74deb0 0x5593df75f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6ac56ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df74deb0 0x5593df75f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50078/0 (socket says 172.21.15.139:50078) 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6ac56ff640 1 -- 172.21.15.139:0/1034785086 learned_addr learned my addr 172.21.15.139:0/1034785086 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6ac4efe640 1 --2- 172.21.15.139:0/1034785086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df77bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6ac5f00640 1 --2- 172.21.15.139:0/1034785086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df77bad0 0x5593df77c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6ac56ff640 1 -- 172.21.15.139:0/1034785086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df77bad0 msgr2=0x5593df77c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/1034785086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df77bad0 0x5593df77c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6ac56ff640 1 -- 172.21.15.139:0/1034785086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 msgr2=0x5593df77bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/1034785086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df77bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6ac56ff640 1 -- 172.21.15.139:0/1034785086 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593df64de40 con 0x5593df74deb0 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6ac5f00640 1 --2- 172.21.15.139:0/1034785086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df77bad0 0x5593df77c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/1034785086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df74deb0 0x5593df75f9b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00e230 tx=0x7f6abc02d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db263634e6cc3b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6aaa7fc640 1 -- 172.21.15.139:0/1034785086 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc031dd0 con 0x5593df74deb0 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6aaa7fc640 1 -- 172.21.15.139:0/1034785086 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc036040 con 0x5593df74deb0 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.647+0000 7f6aaa7fc640 1 -- 172.21.15.139:0/1034785086 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc054330 con 0x5593df74deb0 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.648+0000 7f6aca66d740 1 -- 172.21.15.139:0/1034785086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df74deb0 msgr2=0x5593df75f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.648+0000 7f6aca66d740 1 --2- 172.21.15.139:0/1034785086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df74deb0 0x5593df75f9b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00e230 tx=0x7f6abc02d000 comp rx=0 tx=0).stop 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.648+0000 7f6aca66d740 1 -- 172.21.15.139:0/1034785086 shutdown_connections 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.648+0000 7f6aca66d740 1 --2- 172.21.15.139:0/1034785086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df77bad0 0x5593df77c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.648+0000 7f6aca66d740 1 --2- 172.21.15.139:0/1034785086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df75e230 0x5593df77bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.648+0000 7f6aca66d740 1 --2- 172.21.15.139:0/1034785086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df74deb0 0x5593df75f9b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.648+0000 7f6aca66d740 1 -- 172.21.15.139:0/1034785086 >> 172.21.15.139:0/1034785086 conn(0x5593df733f10 msgr2=0x5593df7342f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.648+0000 7f6aca66d740 1 -- 172.21.15.139:0/1034785086 shutdown_connections 2024-04-25T09:57:25.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.648+0000 7f6aca66d740 1 -- 172.21.15.139:0/1034785086 wait complete. 2024-04-25T09:57:25.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.648+0000 7f6aca66d740 1 -- start start 2024-04-25T09:57:25.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.648+0000 7f6aca66d740 1 Processor -- start 2024-04-25T09:57:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.649+0000 7f6aca66d740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df74deb0 0x5593df76e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.649+0000 7f6aca66d740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df75e230 0x5593df7e2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.649+0000 7f6aca66d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df75cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.649+0000 7f6aca66d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593df734cb0 con 0x5593df77bad0 2024-04-25T09:57:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.649+0000 7f6aca66d740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593df6530c0 con 0x5593df74deb0 2024-04-25T09:57:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.649+0000 7f6aca66d740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593df74a9b0 con 0x5593df75e230 2024-04-25T09:57:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.649+0000 7f6aca66d740 10 client.0 _ll_register_callbacks cb 0x5593df72ec50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.649+0000 7f6ac5f00640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df75cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.649+0000 7f6ac5f00640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df75cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50080/0 (socket says 172.21.15.139:50080) 2024-04-25T09:57:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.649+0000 7f6ac5f00640 1 -- 172.21.15.139:0/3813959380 learned_addr learned my addr 172.21.15.139:0/3813959380 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.649+0000 7f6ac4efe640 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df75e230 0x5593df7e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df74deb0 0x5593df76e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6ac5f00640 1 -- 172.21.15.139:0/3813959380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df75e230 msgr2=0x5593df7e2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6ac5f00640 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df75e230 0x5593df7e2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6ac5f00640 1 -- 172.21.15.139:0/3813959380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df74deb0 msgr2=0x5593df76e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6ac5f00640 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df74deb0 0x5593df76e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6ac5f00640 1 -- 172.21.15.139:0/3813959380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593df72e3e0 con 0x5593df77bad0 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6ac5f00640 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df75cd60 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80186d0 tx=0x7f6ab80187c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d434dc95642bade0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6a9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6a9ffff640 1 -- 172.21.15.139:0/3813959380 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8018110 con 0x5593df77bad0 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6a9ffff640 1 -- 172.21.15.139:0/3813959380 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ab800f040 con 0x5593df77bad0 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6aca66d740 1 -- 172.21.15.139:0/3813959380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6abc036040 con 0x5593df77bad0 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6a9ffff640 1 -- 172.21.15.139:0/3813959380 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ab8028050 con 0x5593df77bad0 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6aca66d740 10 client.7023 fetch_fsmap learned FSMap version 36 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6aca66d740 1 -- 172.21.15.139:0/3813959380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x5593df75d520 con 0x5593df77bad0 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6a9ffff640 1 -- 172.21.15.139:0/3813959380 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc036040 con 0x5593df77bad0 2024-04-25T09:57:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6a9ffff640 1 -- 172.21.15.139:0/3813959380 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6ab802b020 con 0x5593df77bad0 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6aca66d740 10 client.7023 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6aca66d740 10 client.7023 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6aca66d740 20 client.7023 populate_metadata read hostname 'smithi139' 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6aca66d740 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.150:6832/1932986598,v1:172.21.15.150:6834/1932986598] conn(0x5593df745e90 0x5593df746270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6aca66d740 4 client.7023 mds_command: new command op to 6885 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.650+0000 7f6aca66d740 1 -- 172.21.15.139:0/3813959380 --> [v2:172.21.15.150:6832/1932986598,v1:172.21.15.150:6834/1932986598] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5593df75f260 con 0x5593df745e90 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.150:6832/1932986598,v1:172.21.15.150:6834/1932986598] conn(0x5593df745e90 0x5593df746270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6ac56ff640 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.150:6832/1932986598,v1:172.21.15.150:6834/1932986598] conn(0x5593df745e90 0x5593df746270 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d5fd9886a0710ec server_cookie=daa5414b4431f317 in_seq=0 out_seq=0 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6a9ffff640 10 client.7023 ms_handle_connect on v2:172.21.15.150:6832/1932986598 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6a9ffff640 1 -- 172.21.15.139:0/3813959380 <== mds.0 v2:172.21.15.150:6832/1932986598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5593df75f260 con 0x5593df745e90 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6a9ffff640 10 client.7023 handle_command_reply: tid=0 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6aca66d740 1 client.7023 shutdown 2024-04-25T09:57:25.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6aca66d740 1 -- 172.21.15.139:0/3813959380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 msgr2=0x5593df75cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6aca66d740 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df75cd60 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80186d0 tx=0x7f6ab80187c0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6aca66d740 1 -- 172.21.15.139:0/3813959380 shutdown_connections 2024-04-25T09:57:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6aca66d740 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.150:6832/1932986598,v1:172.21.15.150:6834/1932986598] conn(0x5593df745e90 0x5593df746270 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6aca66d740 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5593df77bad0 0x5593df75cd60 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6aca66d740 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5593df75e230 0x5593df7e2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6a9ffff640 0 client.7023 ms_handle_reset on v2:172.21.15.150:6832/1932986598 2024-04-25T09:57:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6aca66d740 1 --2- 172.21.15.139:0/3813959380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593df74deb0 0x5593df76e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6aca66d740 1 -- 172.21.15.139:0/3813959380 >> 172.21.15.139:0/3813959380 conn(0x5593df733f10 msgr2=0x5593df7342f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6aca66d740 1 -- 172.21.15.139:0/3813959380 shutdown_connections 2024-04-25T09:57:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.651+0000 7f6aca66d740 1 -- 172.21.15.139:0/3813959380 wait complete. 2024-04-25T09:57:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6aca66d740 20 client.7023 trim_cache size 0 max 16384 2024-04-25T09:57:25.662 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:57:25.662 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T09:57:25.662 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T09:57:25.662 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 6885, 2024-04-25T09:57:25.662 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T09:57:25.662 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T09:57:25.662 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T09:57:25.662 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 4.5495105950000001, 2024-04-25T09:57:25.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 36, 2024-04-25T09:57:25.662 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 76, 2024-04-25T09:57:25.662 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T09:57:25.662 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 9.6244158780000006 2024-04-25T09:57:25.663 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:57:25.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6ac7963640 1 -- 172.21.15.139:0/3786504047 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6aa406b4d0 msgr2=0x7f6aa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6ac7963640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6aa406b4d0 0x7f6aa406d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6abc035000 tx=0x7f6abc02f000 comp rx=0 tx=0).stop 2024-04-25T09:57:25.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6ac7963640 1 -- 172.21.15.139:0/3786504047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ac015ecf0 msgr2=0x7f6ac015f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6ac7963640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ac015ecf0 0x7f6ac015f170 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0007340 tx=0x7f6ab0007740 comp rx=0 tx=0).stop 2024-04-25T09:57:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6ac7963640 1 -- 172.21.15.139:0/3786504047 shutdown_connections 2024-04-25T09:57:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6ac7963640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ac015ecf0 0x7f6ac015f170 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6ac7963640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6aa406b4d0 0x7f6aa406d990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6ac7963640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ac0151a60 0x7f6ac0151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6ac7963640 1 --2- 172.21.15.139:0/3786504047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aac0039b0 0x7f6ac00d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6ac7963640 1 -- 172.21.15.139:0/3786504047 >> 172.21.15.139:0/3786504047 conn(0x7f6ac00b5480 msgr2=0x7f6ac00d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6ac7963640 1 -- 172.21.15.139:0/3786504047 shutdown_connections 2024-04-25T09:57:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.652+0000 7f6ac7963640 1 -- 172.21.15.139:0/3786504047 wait complete. 2024-04-25T09:57:25.666 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:57:25.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.759+0000 7efd3facb640 1 Processor -- start 2024-04-25T09:57:25.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.759+0000 7efd3facb640 1 -- start start 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3facb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efd38160e90 0x7efd380afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3facb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3806ab60 0x7efd3806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3facb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380d0400 con 0x7efd3806fa70 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3facb640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd380d14a0 con 0x7efd38160e90 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3facb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd380d06e0 con 0x7efd3806ab60 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3d867640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efd38160e90 0x7efd380afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3d066640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3806ab60 0x7efd3806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3e068640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7efd3806fa70 0x7efd3806fec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:33756/0 (socket says 172.21.15.139:33756) 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3e068640 1 -- 172.21.15.139:0/3274930854 learned_addr learned my addr 172.21.15.139:0/3274930854 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3d867640 1 -- 172.21.15.139:0/3274930854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3806ab60 msgr2=0x7efd3806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3d867640 1 --2- 172.21.15.139:0/3274930854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3806ab60 0x7efd3806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3d867640 1 -- 172.21.15.139:0/3274930854 >> v1:172.21.15.150:6789/0 conn(0x7efd3806fa70 legacy=0x7efd3806fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.760+0000 7efd3d867640 1 -- 172.21.15.139:0/3274930854 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380d09c0 con 0x7efd38160e90 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.761+0000 7efd3d867640 1 --2- 172.21.15.139:0/3274930854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd38160e90 0x7efd380afac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efd340097a0 tx=0x7efd3402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb735c17377b72c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.761+0000 7efd3c865640 1 -- 172.21.15.139:0/3274930854 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34037070 con 0x7efd38160e90 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.761+0000 7efd3c865640 1 -- 172.21.15.139:0/3274930854 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34032440 con 0x7efd38160e90 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.761+0000 7efd3c865640 1 -- 172.21.15.139:0/3274930854 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34032810 con 0x7efd38160e90 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.762+0000 7efd3facb640 1 -- 172.21.15.139:0/3274930854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd38160e90 msgr2=0x7efd380afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.762+0000 7efd3facb640 1 --2- 172.21.15.139:0/3274930854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd38160e90 0x7efd380afac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efd340097a0 tx=0x7efd3402a180 comp rx=0 tx=0).stop 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.762+0000 7efd3facb640 1 -- 172.21.15.139:0/3274930854 shutdown_connections 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.762+0000 7efd3facb640 1 --2- 172.21.15.139:0/3274930854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3806ab60 0x7efd3806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.762+0000 7efd3facb640 1 --2- 172.21.15.139:0/3274930854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd38160e90 0x7efd380afac0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.762+0000 7efd3facb640 1 -- 172.21.15.139:0/3274930854 >> 172.21.15.139:0/3274930854 conn(0x7efd380b23f0 msgr2=0x7efd380b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.762+0000 7efd3facb640 1 -- 172.21.15.139:0/3274930854 shutdown_connections 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.762+0000 7efd3facb640 1 -- 172.21.15.139:0/3274930854 wait complete. 2024-04-25T09:57:25.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.763+0000 7efd3facb640 1 Processor -- start 2024-04-25T09:57:25.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.763+0000 7efd3facb640 1 -- start start 2024-04-25T09:57:25.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3facb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd3806ab60 0x7efd38146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3facb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd3806fa70 0x7efd380a9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3facb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd38160e90 0x7efd380ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3facb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380b0000 con 0x7efd3806ab60 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3facb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380d0f80 con 0x7efd3806fa70 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3facb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd38161270 con 0x7efd38160e90 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3d867640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd3806ab60 0x7efd38146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3d867640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd3806ab60 0x7efd38146a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50084/0 (socket says 172.21.15.139:50084) 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3d867640 1 -- 172.21.15.139:0/3886832738 learned_addr learned my addr 172.21.15.139:0/3886832738 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3d066640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd3806fa70 0x7efd380a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3e068640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd38160e90 0x7efd380ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3d867640 1 -- 172.21.15.139:0/3886832738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd38160e90 msgr2=0x7efd380ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3d867640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd38160e90 0x7efd380ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3d867640 1 -- 172.21.15.139:0/3886832738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd3806fa70 msgr2=0x7efd380a9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3d867640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd3806fa70 0x7efd380a9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3d867640 1 -- 172.21.15.139:0/3886832738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380aa0f0 con 0x7efd3806ab60 2024-04-25T09:57:25.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.764+0000 7efd3d066640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd3806fa70 0x7efd380a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:25.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.765+0000 7efd3d867640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd3806ab60 0x7efd38146a50 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7efd34035040 tx=0x7efd340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.765+0000 7efd2e7fc640 1 -- 172.21.15.139:0/3886832738 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34037040 con 0x7efd3806ab60 2024-04-25T09:57:25.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.765+0000 7efd2e7fc640 1 -- 172.21.15.139:0/3886832738 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34030500 con 0x7efd3806ab60 2024-04-25T09:57:25.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.765+0000 7efd2e7fc640 1 -- 172.21.15.139:0/3886832738 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340308d0 con 0x7efd3806ab60 2024-04-25T09:57:25.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.765+0000 7efd3facb640 1 -- 172.21.15.139:0/3886832738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd380aa3d0 con 0x7efd3806ab60 2024-04-25T09:57:25.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.765+0000 7efd3facb640 1 -- 172.21.15.139:0/3886832738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd380aa990 con 0x7efd3806ab60 2024-04-25T09:57:25.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.766+0000 7efd3facb640 1 -- 172.21.15.139:0/3886832738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd38074ee0 con 0x7efd3806ab60 2024-04-25T09:57:25.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.766+0000 7efd2e7fc640 1 -- 172.21.15.139:0/3886832738 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd34044020 con 0x7efd3806ab60 2024-04-25T09:57:25.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.767+0000 7efd2e7fc640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd1c06b4d0 0x7efd1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.767+0000 7efd2e7fc640 1 -- 172.21.15.139:0/3886832738 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd340aa5a0 con 0x7efd3806ab60 2024-04-25T09:57:25.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.767+0000 7efd3d066640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd1c06b4d0 0x7efd1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.767+0000 7efd3d066640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd1c06b4d0 0x7efd1c06d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efd380cdd60 tx=0x7efd28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.771+0000 7efd2e7fc640 1 -- 172.21.15.139:0/3886832738 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd38074ee0 con 0x7efd3806ab60 2024-04-25T09:57:25.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.977+0000 7efd3facb640 1 -- 172.21.15.139:0/3886832738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efd38074520 con 0x7efd3806ab60 2024-04-25T09:57:25.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.977+0000 7efd2e7fc640 1 -- 172.21.15.139:0/3886832738 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) v1 ==== 53+0+73 (secure 0 0 0) 0x7efd34079320 con 0x7efd3806ab60 2024-04-25T09:57:25.981 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:57:25.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.979+0000 7efd3facb640 1 -- 172.21.15.139:0/3886832738 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd1c06b4d0 msgr2=0x7efd1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.979+0000 7efd3facb640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd1c06b4d0 0x7efd1c06d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efd380cdd60 tx=0x7efd28008040 comp rx=0 tx=0).stop 2024-04-25T09:57:25.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.979+0000 7efd3facb640 1 -- 172.21.15.139:0/3886832738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd3806ab60 msgr2=0x7efd38146a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.979+0000 7efd3facb640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd3806ab60 0x7efd38146a50 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7efd34035040 tx=0x7efd340046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.979+0000 7efd3facb640 1 -- 172.21.15.139:0/3886832738 shutdown_connections 2024-04-25T09:57:25.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.979+0000 7efd3facb640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd1c06b4d0 0x7efd1c06d990 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.979+0000 7efd3facb640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd38160e90 0x7efd380ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.979+0000 7efd3facb640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd3806fa70 0x7efd380a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.979+0000 7efd3facb640 1 --2- 172.21.15.139:0/3886832738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd3806ab60 0x7efd38146a50 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.979+0000 7efd3facb640 1 -- 172.21.15.139:0/3886832738 >> 172.21.15.139:0/3886832738 conn(0x7efd380b23f0 msgr2=0x7efd380cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.979+0000 7efd3facb640 1 -- 172.21.15.139:0/3886832738 shutdown_connections 2024-04-25T09:57:25.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:25.980+0000 7efd3facb640 1 -- 172.21.15.139:0/3886832738 wait complete. 2024-04-25T09:57:25.993 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:57:25.994 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:57:25.994 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T09:57:25.994 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:57:25.994 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:25.994 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:57:25.994 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:57:25.994 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:57:25.994 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:57:25.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:25.994 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T09:57:26.009 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:57:26.009 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:57:26.009 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:57:26.009 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:26.010 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42452sec preferred_lft 42452sec 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:57:26.011 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:57:26.012 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T09:57:26.012 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:26.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.012 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:57:26.072 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:57:26.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.074 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T09:57:26.129 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T09:57:26.130 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:57:26.130 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:57:26.130 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:57:26.130 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:57:26.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.130 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:57:26.130 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:57:26.130 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:57:26.130 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:57:26.131 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:57:26.131 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:57:26.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.214 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:57:26.268 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:57:26.269 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:57:26.269 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:57:26.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.269 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:57:26.332 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:26.355 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-04-25T09:57:26.352+0000 7fcab9dd5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:57:26.388 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:57:26.388 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:57:26.388 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:26.445 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:57:26.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.445 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:57:26.524 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:57:26.524 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:57:26.537 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:57:26.538 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:57:26.538 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T09:57:26.538 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T09:57:26.538 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T09:57:26.538 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:57:26.538 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:57:26.538 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:57:26.539 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:57:26.539 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:57:26.540 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:57:26.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.540 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:26.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.602 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:57:26.664 INFO:teuthology.orchestra.run.smithi139.stdout:client5759 2024-04-25T09:57:26.665 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:57:26.665 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:26.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.720 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-25T09:57:26.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.780 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-25T09:57:26.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.837 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-25T09:57:26.892 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T09:57:26.892 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T09:57:26.892 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T09:57:26.892 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:57:26.892 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-25T09:57:26.892 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T09:57:26.997 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627776 2024-04-25T09:57:27.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:57:27.006 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:57:27.006 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:27.061 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:57:27.061 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:27.116 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:27.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.249+0000 7f132e7d5640 1 Processor -- start 2024-04-25T09:57:27.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.249+0000 7f132e7d5640 1 -- start start 2024-04-25T09:57:27.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f132e7d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f132815a860 0x7f13280cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f132e7d5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f13280cd1f0 0x7f132806e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f132e7d5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f132815ac40 0x7f132806e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f132e7d5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13280b02e0 con 0x7f13280cd1f0 2024-04-25T09:57:27.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f132e7d5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13280af6d0 con 0x7f132815ac40 2024-04-25T09:57:27.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f132e7d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13280af9b0 con 0x7f132815a860 2024-04-25T09:57:27.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f13277fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f13280cd1f0 0x7f132806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f13277fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f13280cd1f0 0x7f132806e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35384/0 (socket says 172.21.15.139:35384) 2024-04-25T09:57:27.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f13277fe640 1 -- 172.21.15.139:0/2187757155 learned_addr learned my addr 172.21.15.139:0/2187757155 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:27.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f13277fe640 1 -- 172.21.15.139:0/2187757155 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f132815a860 msgr2=0x7f13280cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f13277fe640 1 --2- 172.21.15.139:0/2187757155 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f132815a860 0x7f13280cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f13277fe640 1 -- 172.21.15.139:0/2187757155 >> v2:172.21.15.150:3300/0 conn(0x7f132815ac40 msgr2=0x7f132806e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f13277fe640 1 --2- 172.21.15.139:0/2187757155 >> v2:172.21.15.150:3300/0 conn(0x7f132815ac40 0x7f132806e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.250+0000 7f13277fe640 1 -- 172.21.15.139:0/2187757155 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13280afc90 con 0x7f13280cd1f0 2024-04-25T09:57:27.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.251+0000 7f13277fe640 1 --2- 172.21.15.139:0/2187757155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13280cd1f0 0x7f132806e310 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f13280b49e0 tx=0x7f131c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602369ba6958689f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.251+0000 7f1326ffd640 1 -- 172.21.15.139:0/2187757155 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c037070 con 0x7f13280cd1f0 2024-04-25T09:57:27.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.251+0000 7f1326ffd640 1 -- 172.21.15.139:0/2187757155 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131c032440 con 0x7f13280cd1f0 2024-04-25T09:57:27.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.251+0000 7f1326ffd640 1 -- 172.21.15.139:0/2187757155 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c032810 con 0x7f13280cd1f0 2024-04-25T09:57:27.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.252+0000 7f132e7d5640 1 -- 172.21.15.139:0/2187757155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13280cd1f0 msgr2=0x7f132806e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.252+0000 7f132e7d5640 1 --2- 172.21.15.139:0/2187757155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13280cd1f0 0x7f132806e310 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f13280b49e0 tx=0x7f131c02a200 comp rx=0 tx=0).stop 2024-04-25T09:57:27.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.252+0000 7f132e7d5640 1 -- 172.21.15.139:0/2187757155 shutdown_connections 2024-04-25T09:57:27.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.252+0000 7f132e7d5640 1 --2- 172.21.15.139:0/2187757155 >> v2:172.21.15.150:3300/0 conn(0x7f132815ac40 0x7f132806e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.252+0000 7f132e7d5640 1 --2- 172.21.15.139:0/2187757155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13280cd1f0 0x7f132806e310 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.252+0000 7f132e7d5640 1 --2- 172.21.15.139:0/2187757155 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f132815a860 0x7f13280cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.252+0000 7f132e7d5640 1 -- 172.21.15.139:0/2187757155 >> 172.21.15.139:0/2187757155 conn(0x7f1328066f20 msgr2=0x7f132806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.252+0000 7f132e7d5640 1 -- 172.21.15.139:0/2187757155 shutdown_connections 2024-04-25T09:57:27.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.252+0000 7f132e7d5640 1 -- 172.21.15.139:0/2187757155 wait complete. 2024-04-25T09:57:27.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.253+0000 7f132e7d5640 1 Processor -- start 2024-04-25T09:57:27.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.253+0000 7f132e7d5640 1 -- start start 2024-04-25T09:57:27.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f132e7d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13280cd1f0 0x7f1328145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f132e7d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f132815a860 0x7f13280acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f132e7d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f132815ac40 0x7f132806d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f132e7d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13280af420 con 0x7f13280cd1f0 2024-04-25T09:57:27.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f132e7d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13280af2a0 con 0x7f132815a860 2024-04-25T09:57:27.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f132e7d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132815e240 con 0x7f132815ac40 2024-04-25T09:57:27.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f1327fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13280cd1f0 0x7f1328145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f1327fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13280cd1f0 0x7f1328145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35398/0 (socket says 172.21.15.139:35398) 2024-04-25T09:57:27.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f1327fff640 1 -- 172.21.15.139:0/928166112 learned_addr learned my addr 172.21.15.139:0/928166112 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f13277fe640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f132815a860 0x7f13280acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f132cd72640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f132815ac40 0x7f132806d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f1327fff640 1 -- 172.21.15.139:0/928166112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f132815ac40 msgr2=0x7f132806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f1327fff640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f132815ac40 0x7f132806d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f1327fff640 1 -- 172.21.15.139:0/928166112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f132815a860 msgr2=0x7f13280acbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f1327fff640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f132815a860 0x7f13280acbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f1327fff640 1 -- 172.21.15.139:0/928166112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13280ad540 con 0x7f13280cd1f0 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.254+0000 7f1327fff640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13280cd1f0 0x7f1328145ff0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f131800b4d0 tx=0x7f131800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.255+0000 7f1324ff9640 1 -- 172.21.15.139:0/928166112 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318014070 con 0x7f13280cd1f0 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.255+0000 7f1324ff9640 1 -- 172.21.15.139:0/928166112 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1318010870 con 0x7f13280cd1f0 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.255+0000 7f132e7d5640 1 -- 172.21.15.139:0/928166112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13280ad820 con 0x7f13280cd1f0 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.255+0000 7f1324ff9640 1 -- 172.21.15.139:0/928166112 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318010c60 con 0x7f13280cd1f0 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.255+0000 7f132e7d5640 1 -- 172.21.15.139:0/928166112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13280adde0 con 0x7f13280cd1f0 2024-04-25T09:57:27.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.256+0000 7f132e7d5640 1 -- 172.21.15.139:0/928166112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1328074520 con 0x7f13280cd1f0 2024-04-25T09:57:27.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.256+0000 7f1324ff9640 1 -- 172.21.15.139:0/928166112 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1318017070 con 0x7f13280cd1f0 2024-04-25T09:57:27.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.256+0000 7f1324ff9640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f131406b4d0 0x7f131406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.257+0000 7f13277fe640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f131406b4d0 0x7f131406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.257+0000 7f1324ff9640 1 -- 172.21.15.139:0/928166112 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f131808a1a0 con 0x7f13280cd1f0 2024-04-25T09:57:27.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.257+0000 7f13277fe640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f131406b4d0 0x7f131406d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f131c034000 tx=0x7f131c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.261+0000 7f1324ff9640 1 -- 172.21.15.139:0/928166112 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1318058e90 con 0x7f13280cd1f0 2024-04-25T09:57:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.430+0000 7f132e7d5640 1 -- 172.21.15.139:0/928166112 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f132815da40 con 0x7f13280cd1f0 2024-04-25T09:57:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.431+0000 7f1324ff9640 1 -- 172.21.15.139:0/928166112 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16993 (secure 0 0 0) 0x7f1318058790 con 0x7f13280cd1f0 2024-04-25T09:57:27.435 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:27.435 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":76,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:57:22.102239+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:18.681554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:19.409382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:27.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.433+0000 7f132e7d5640 1 -- 172.21.15.139:0/928166112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f131406b4d0 msgr2=0x7f131406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.433+0000 7f132e7d5640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f131406b4d0 0x7f131406d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f131c034000 tx=0x7f131c02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:27.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.433+0000 7f132e7d5640 1 -- 172.21.15.139:0/928166112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13280cd1f0 msgr2=0x7f1328145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.433+0000 7f132e7d5640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13280cd1f0 0x7f1328145ff0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f131800b4d0 tx=0x7f131800b990 comp rx=0 tx=0).stop 2024-04-25T09:57:27.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.433+0000 7f132e7d5640 1 -- 172.21.15.139:0/928166112 shutdown_connections 2024-04-25T09:57:27.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.433+0000 7f132e7d5640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f131406b4d0 0x7f131406d990 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.433+0000 7f132e7d5640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f132815ac40 0x7f132806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.433+0000 7f132e7d5640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f132815a860 0x7f13280acbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.433+0000 7f132e7d5640 1 --2- 172.21.15.139:0/928166112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13280cd1f0 0x7f1328145ff0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.433+0000 7f132e7d5640 1 -- 172.21.15.139:0/928166112 >> 172.21.15.139:0/928166112 conn(0x7f1328066f20 msgr2=0x7f13280b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.433+0000 7f132e7d5640 1 -- 172.21.15.139:0/928166112 shutdown_connections 2024-04-25T09:57:27.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.433+0000 7f132e7d5640 1 -- 172.21.15.139:0/928166112 wait complete. 2024-04-25T09:57:27.448 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:57:18.681554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:57:19.409382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:27.448 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:27.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.544+0000 7f654810a640 1 Processor -- start 2024-04-25T09:57:27.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.544+0000 7f654810a640 1 -- start start 2024-04-25T09:57:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.545+0000 7f654810a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6540144090 0x7f65400b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.545+0000 7f654810a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f654006ea40 0x7f654015e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.545+0000 7f654810a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65400b2a50 con 0x7f65400cc9e0 2024-04-25T09:57:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.545+0000 7f654810a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f654015b090 con 0x7f6540144090 2024-04-25T09:57:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.545+0000 7f654810a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654015b370 con 0x7f654006ea40 2024-04-25T09:57:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.545+0000 7f65466a7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f654006ea40 0x7f654015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.545+0000 7f6545ea6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6540144090 0x7f65400b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.545+0000 7f65466a7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f654006ea40 0x7f654015e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:45038/0 (socket says 172.21.15.139:45038) 2024-04-25T09:57:27.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.545+0000 7f65466a7640 1 -- 172.21.15.139:0/3604454193 learned_addr learned my addr 172.21.15.139:0/3604454193 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:27.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.546+0000 7f65466a7640 1 -- 172.21.15.139:0/3604454193 >> v2:172.21.15.150:3300/0 conn(0x7f6540144090 msgr2=0x7f65400b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.546+0000 7f65466a7640 1 --2- 172.21.15.139:0/3604454193 >> v2:172.21.15.150:3300/0 conn(0x7f6540144090 0x7f65400b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.546+0000 7f65466a7640 1 -- 172.21.15.139:0/3604454193 >> v1:172.21.15.150:6789/0 conn(0x7f65400cc9e0 legacy=0x7f654006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.546+0000 7f65466a7640 1 -- 172.21.15.139:0/3604454193 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654015b650 con 0x7f654006ea40 2024-04-25T09:57:27.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.546+0000 7f6545ea6640 1 --2- 172.21.15.139:0/3604454193 >> v2:172.21.15.150:3300/0 conn(0x7f6540144090 0x7f65400b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:27.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.546+0000 7f65466a7640 1 --2- 172.21.15.139:0/3604454193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f654006ea40 0x7f654015e610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6538009e50 tx=0x7f653802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=303f225fc0d7c48a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.547+0000 7f6544ea4640 1 -- 172.21.15.139:0/3604454193 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538036070 con 0x7f654006ea40 2024-04-25T09:57:27.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.547+0000 7f6544ea4640 1 -- 172.21.15.139:0/3604454193 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6538031600 con 0x7f654006ea40 2024-04-25T09:57:27.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.547+0000 7f6544ea4640 1 -- 172.21.15.139:0/3604454193 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65380319d0 con 0x7f654006ea40 2024-04-25T09:57:27.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.547+0000 7f654810a640 1 -- 172.21.15.139:0/3604454193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f654006ea40 msgr2=0x7f654015e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.547+0000 7f654810a640 1 --2- 172.21.15.139:0/3604454193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f654006ea40 0x7f654015e610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6538009e50 tx=0x7f653802a680 comp rx=0 tx=0).stop 2024-04-25T09:57:27.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.547+0000 7f654810a640 1 -- 172.21.15.139:0/3604454193 shutdown_connections 2024-04-25T09:57:27.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.547+0000 7f654810a640 1 --2- 172.21.15.139:0/3604454193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f654006ea40 0x7f654015e610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.547+0000 7f654810a640 1 --2- 172.21.15.139:0/3604454193 >> v2:172.21.15.150:3300/0 conn(0x7f6540144090 0x7f65400b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.547+0000 7f654810a640 1 -- 172.21.15.139:0/3604454193 >> 172.21.15.139:0/3604454193 conn(0x7f6540067000 msgr2=0x7f654006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.547+0000 7f654810a640 1 -- 172.21.15.139:0/3604454193 shutdown_connections 2024-04-25T09:57:27.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.547+0000 7f654810a640 1 -- 172.21.15.139:0/3604454193 wait complete. 2024-04-25T09:57:27.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.548+0000 7f654810a640 1 Processor -- start 2024-04-25T09:57:27.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.548+0000 7f654810a640 1 -- start start 2024-04-25T09:57:27.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.549+0000 7f654810a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f654006ea40 0x7f65400c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.549+0000 7f654810a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65400cc9e0 0x7f65401f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.549+0000 7f654810a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6540144090 0x7f654015d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.549+0000 7f654810a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6540144470 con 0x7f654006ea40 2024-04-25T09:57:27.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.549+0000 7f654810a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654015bc10 con 0x7f6540144090 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.549+0000 7f654810a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654015ad80 con 0x7f65400cc9e0 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.549+0000 7f6545ea6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f654006ea40 0x7f65400c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.549+0000 7f6545ea6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f654006ea40 0x7f65400c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35408/0 (socket says 172.21.15.139:35408) 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.549+0000 7f6545ea6640 1 -- 172.21.15.139:0/3231798923 learned_addr learned my addr 172.21.15.139:0/3231798923 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.549+0000 7f65466a7640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6540144090 0x7f654015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.549+0000 7f6545ea6640 1 -- 172.21.15.139:0/3231798923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65400cc9e0 msgr2=0x7f65401f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.549+0000 7f65456a5640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65400cc9e0 0x7f65401f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.550+0000 7f6545ea6640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65400cc9e0 0x7f65401f33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.550+0000 7f6545ea6640 1 -- 172.21.15.139:0/3231798923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6540144090 msgr2=0x7f654015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.550+0000 7f6545ea6640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6540144090 0x7f654015d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.550+0000 7f6545ea6640 1 -- 172.21.15.139:0/3231798923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65401f3da0 con 0x7f654006ea40 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.550+0000 7f65466a7640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6540144090 0x7f654015d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.550+0000 7f65456a5640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65400cc9e0 0x7f65401f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:27.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.550+0000 7f6545ea6640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f654006ea40 0x7f65400c4560 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f653c00ebe0 tx=0x7f653c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.550+0000 7f65367fc640 1 -- 172.21.15.139:0/3231798923 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c017070 con 0x7f654006ea40 2024-04-25T09:57:27.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.550+0000 7f65367fc640 1 -- 172.21.15.139:0/3231798923 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c00cc30 con 0x7f654006ea40 2024-04-25T09:57:27.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.550+0000 7f654810a640 1 -- 172.21.15.139:0/3231798923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65401f4080 con 0x7f654006ea40 2024-04-25T09:57:27.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.550+0000 7f65367fc640 1 -- 172.21.15.139:0/3231798923 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c013470 con 0x7f654006ea40 2024-04-25T09:57:27.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.550+0000 7f654810a640 1 -- 172.21.15.139:0/3231798923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65400c5970 con 0x7f654006ea40 2024-04-25T09:57:27.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.551+0000 7f654810a640 1 -- 172.21.15.139:0/3231798923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6540074520 con 0x7f654006ea40 2024-04-25T09:57:27.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.551+0000 7f65367fc640 1 -- 172.21.15.139:0/3231798923 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f653c01a070 con 0x7f654006ea40 2024-04-25T09:57:27.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.551+0000 7f65367fc640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f652406b4d0 0x7f652406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.552+0000 7f65367fc640 1 -- 172.21.15.139:0/3231798923 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f653c08a610 con 0x7f654006ea40 2024-04-25T09:57:27.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.552+0000 7f65456a5640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f652406b4d0 0x7f652406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.552+0000 7f65456a5640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f652406b4d0 0x7f652406d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f654015dc00 tx=0x7f6530009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.556+0000 7f65367fc640 1 -- 172.21.15.139:0/3231798923 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f653c059390 con 0x7f654006ea40 2024-04-25T09:57:27.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.726+0000 7f654810a640 1 -- 172.21.15.139:0/3231798923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f654006c1a0 con 0x7f654006ea40 2024-04-25T09:57:27.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.727+0000 7f65367fc640 1 -- 172.21.15.139:0/3231798923 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f653c058c90 con 0x7f654006ea40 2024-04-25T09:57:27.730 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:57:27.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.729+0000 7f654810a640 1 -- 172.21.15.139:0/3231798923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f652406b4d0 msgr2=0x7f652406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.729+0000 7f654810a640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f652406b4d0 0x7f652406d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f654015dc00 tx=0x7f6530009040 comp rx=0 tx=0).stop 2024-04-25T09:57:27.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.729+0000 7f654810a640 1 -- 172.21.15.139:0/3231798923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f654006ea40 msgr2=0x7f65400c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.729+0000 7f654810a640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f654006ea40 0x7f65400c4560 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f653c00ebe0 tx=0x7f653c00c520 comp rx=0 tx=0).stop 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.729+0000 7f654810a640 1 -- 172.21.15.139:0/3231798923 shutdown_connections 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.729+0000 7f654810a640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f652406b4d0 0x7f652406d990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.729+0000 7f654810a640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6540144090 0x7f654015d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.729+0000 7f654810a640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65400cc9e0 0x7f65401f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.729+0000 7f654810a640 1 --2- 172.21.15.139:0/3231798923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f654006ea40 0x7f65400c4560 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.729+0000 7f654810a640 1 -- 172.21.15.139:0/3231798923 >> 172.21.15.139:0/3231798923 conn(0x7f6540067000 msgr2=0x7f65400b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.729+0000 7f654810a640 1 -- 172.21.15.139:0/3231798923 shutdown_connections 2024-04-25T09:57:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.729+0000 7f654810a640 1 -- 172.21.15.139:0/3231798923 wait complete. 2024-04-25T09:57:27.743 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:57:27.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.840+0000 7fabcca6f640 1 Processor -- start 2024-04-25T09:57:27.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.840+0000 7fabcca6f640 1 -- start start 2024-04-25T09:57:27.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.841+0000 7fabcca6f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fabc80b5620 0x7fabc80b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.841+0000 7fabcca6f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc80b5a20 con 0x7fabc815a870 2024-04-25T09:57:27.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.841+0000 7fabcca6f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc80d0d40 con 0x7fabc8070970 2024-04-25T09:57:27.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.841+0000 7fabcca6f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc80d1020 con 0x7fabc80b5620 2024-04-25T09:57:27.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.841+0000 7fabc6d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fabc80b5620 0x7fabc80b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.841+0000 7fabc659c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fabc815a870 0x7fabc8070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:49420/0 (socket says 172.21.15.139:49420) 2024-04-25T09:57:27.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.841+0000 7fabc659c640 1 -- 172.21.15.139:0/1193933863 learned_addr learned my addr 172.21.15.139:0/1193933863 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:27.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.842+0000 7fabc6d9d640 1 -- 172.21.15.139:0/1193933863 >> v1:172.21.15.150:6789/0 conn(0x7fabc8070970 legacy=0x7fabc80ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.842+0000 7fabc6d9d640 1 -- 172.21.15.139:0/1193933863 >> v1:172.21.15.139:6789/0 conn(0x7fabc815a870 legacy=0x7fabc8070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.842+0000 7fabc6d9d640 1 -- 172.21.15.139:0/1193933863 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc80b4b40 con 0x7fabc80b5620 2024-04-25T09:57:27.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.842+0000 7fabc559a640 1 -- 172.21.15.139:0/1193933863 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172762365 0 0) 0x7fabc80b5a20 con 0x7fabc815a870 2024-04-25T09:57:27.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.842+0000 7fabc6d9d640 1 --2- 172.21.15.139:0/1193933863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc80b5620 0x7fabc80b4190 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fabc00097a0 tx=0x7fabc002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2c530e3151d5ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.843+0000 7fabc559a640 1 -- 172.21.15.139:0/1193933863 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0037070 con 0x7fabc80b5620 2024-04-25T09:57:27.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.843+0000 7fabc559a640 1 -- 172.21.15.139:0/1193933863 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc002ae50 con 0x7fabc80b5620 2024-04-25T09:57:27.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.843+0000 7fabc559a640 1 -- 172.21.15.139:0/1193933863 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0032660 con 0x7fabc80b5620 2024-04-25T09:57:27.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.843+0000 7fabcca6f640 1 -- 172.21.15.139:0/1193933863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc80b5620 msgr2=0x7fabc80b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.843+0000 7fabcca6f640 1 --2- 172.21.15.139:0/1193933863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc80b5620 0x7fabc80b4190 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fabc00097a0 tx=0x7fabc002a140 comp rx=0 tx=0).stop 2024-04-25T09:57:27.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.843+0000 7fabcca6f640 1 -- 172.21.15.139:0/1193933863 shutdown_connections 2024-04-25T09:57:27.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.843+0000 7fabcca6f640 1 --2- 172.21.15.139:0/1193933863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc80b5620 0x7fabc80b4190 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.843+0000 7fabcca6f640 1 -- 172.21.15.139:0/1193933863 >> 172.21.15.139:0/1193933863 conn(0x7fabc80b23f0 msgr2=0x7fabc80b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.843+0000 7fabcca6f640 1 -- 172.21.15.139:0/1193933863 shutdown_connections 2024-04-25T09:57:27.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.843+0000 7fabcca6f640 1 -- 172.21.15.139:0/1193933863 wait complete. 2024-04-25T09:57:27.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.845+0000 7fabcca6f640 1 Processor -- start 2024-04-25T09:57:27.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.845+0000 7fabcca6f640 1 -- start start 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.845+0000 7fabcca6f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc8070970 0x7fabc8154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.845+0000 7fabcca6f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc80b5620 0x7fabc81506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.845+0000 7fabcca6f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc815a870 0x7fabc8159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.845+0000 7fabcca6f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80d05f0 con 0x7fabc80b5620 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.845+0000 7fabcca6f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80b5190 con 0x7fabc8070970 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.845+0000 7fabcca6f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80d0770 con 0x7fabc815a870 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.845+0000 7fabc5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc80b5620 0x7fabc81506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.845+0000 7fabc5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc80b5620 0x7fabc81506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35422/0 (socket says 172.21.15.139:35422) 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.845+0000 7fabc5d9b640 1 -- 172.21.15.139:0/1975721714 learned_addr learned my addr 172.21.15.139:0/1975721714 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabc6d9d640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc815a870 0x7fabc8159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabc659c640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc8070970 0x7fabc8154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabc5d9b640 1 -- 172.21.15.139:0/1975721714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc815a870 msgr2=0x7fabc8159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabc5d9b640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc815a870 0x7fabc8159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabc5d9b640 1 -- 172.21.15.139:0/1975721714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc8070970 msgr2=0x7fabc8154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabc5d9b640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc8070970 0x7fabc8154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabc5d9b640 1 -- 172.21.15.139:0/1975721714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc8151470 con 0x7fabc80b5620 2024-04-25T09:57:27.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabc659c640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc8070970 0x7fabc8154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabc5d9b640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc80b5620 0x7fabc81506b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00c430 tx=0x7fabbc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabb6ffd640 1 -- 172.21.15.139:0/1975721714 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc011790 con 0x7fabc80b5620 2024-04-25T09:57:27.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabb6ffd640 1 -- 172.21.15.139:0/1975721714 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc017070 con 0x7fabc80b5620 2024-04-25T09:57:27.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabcca6f640 1 -- 172.21.15.139:0/1975721714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc80aed40 con 0x7fabc80b5620 2024-04-25T09:57:27.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabb6ffd640 1 -- 172.21.15.139:0/1975721714 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc011bb0 con 0x7fabc80b5620 2024-04-25T09:57:27.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.846+0000 7fabcca6f640 1 -- 172.21.15.139:0/1975721714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc80af300 con 0x7fabc80b5620 2024-04-25T09:57:27.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.847+0000 7fabcca6f640 1 -- 172.21.15.139:0/1975721714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc8074520 con 0x7fabc80b5620 2024-04-25T09:57:27.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.847+0000 7fabb6ffd640 1 -- 172.21.15.139:0/1975721714 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabbc01c460 con 0x7fabc80b5620 2024-04-25T09:57:27.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.848+0000 7fabb6ffd640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faba806b4d0 0x7faba806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.848+0000 7fabb6ffd640 1 -- 172.21.15.139:0/1975721714 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fabbc08a250 con 0x7fabc80b5620 2024-04-25T09:57:27.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.848+0000 7fabc659c640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faba806b4d0 0x7faba806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.848+0000 7fabc659c640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faba806b4d0 0x7faba806d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fabb80012e0 tx=0x7fabb8005b90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:27.852+0000 7fabb6ffd640 1 -- 172.21.15.139:0/1975721714 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fabbc058f40 con 0x7fabc80b5620 2024-04-25T09:57:28.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.021+0000 7fabcca6f640 1 -- 172.21.15.139:0/1975721714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fabc80cf1f0 con 0x7fabc80b5620 2024-04-25T09:57:28.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.021+0000 7fabb6ffd640 1 -- 172.21.15.139:0/1975721714 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+11 (secure 0 0 0) 0x7fabbc058840 con 0x7fabc80b5620 2024-04-25T09:57:28.025 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:57:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.023+0000 7fabcca6f640 1 -- 172.21.15.139:0/1975721714 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faba806b4d0 msgr2=0x7faba806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.023+0000 7fabcca6f640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faba806b4d0 0x7faba806d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fabb80012e0 tx=0x7fabb8005b90 comp rx=0 tx=0).stop 2024-04-25T09:57:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.023+0000 7fabcca6f640 1 -- 172.21.15.139:0/1975721714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc80b5620 msgr2=0x7fabc81506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.023+0000 7fabcca6f640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc80b5620 0x7fabc81506b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00c430 tx=0x7fabbc00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.024+0000 7fabcca6f640 1 -- 172.21.15.139:0/1975721714 shutdown_connections 2024-04-25T09:57:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.024+0000 7fabcca6f640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc80b5620 0x7fabc81506b0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.024+0000 7fabcca6f640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faba806b4d0 0x7faba806d990 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.024+0000 7fabcca6f640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc815a870 0x7fabc8159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.024+0000 7fabcca6f640 1 --2- 172.21.15.139:0/1975721714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc8070970 0x7fabc8154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.024+0000 7fabcca6f640 1 -- 172.21.15.139:0/1975721714 >> 172.21.15.139:0/1975721714 conn(0x7fabc80b23f0 msgr2=0x7fabc80b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.024+0000 7fabcca6f640 1 -- 172.21.15.139:0/1975721714 shutdown_connections 2024-04-25T09:57:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.024+0000 7fabcca6f640 1 -- 172.21.15.139:0/1975721714 wait complete. 2024-04-25T09:57:28.038 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:57:28.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.136+0000 7fe788488640 1 Processor -- start 2024-04-25T09:57:28.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.136+0000 7fe788488640 1 -- start start 2024-04-25T09:57:28.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.137+0000 7fe788488640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe780071a50 0x7fe7800cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.137+0000 7fe788488640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe7800ce040 0x7fe7800ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.137+0000 7fe788488640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78015c490 con 0x7fe78015bf90 2024-04-25T09:57:28.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.137+0000 7fe788488640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78015c770 con 0x7fe7800ce040 2024-04-25T09:57:28.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.137+0000 7fe788488640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78015ca50 con 0x7fe780071a50 2024-04-25T09:57:28.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.137+0000 7fe786a25640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe7800ce040 0x7fe7800ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.137+0000 7fe785a23640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe78015bf90 0x7fe780071e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:49426/0 (socket says 172.21.15.139:49426) 2024-04-25T09:57:28.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.137+0000 7fe786a25640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe7800ce040 0x7fe7800ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35436/0 (socket says 172.21.15.139:35436) 2024-04-25T09:57:28.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.137+0000 7fe785a23640 1 -- 172.21.15.139:0/4061418804 learned_addr learned my addr 172.21.15.139:0/4061418804 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:28.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.138+0000 7fe786224640 1 --2- 172.21.15.139:0/4061418804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe780071a50 0x7fe7800cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.138+0000 7fe786a25640 1 -- 172.21.15.139:0/4061418804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe780071a50 msgr2=0x7fe7800cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.138+0000 7fe786a25640 1 --2- 172.21.15.139:0/4061418804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe780071a50 0x7fe7800cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.138+0000 7fe786a25640 1 -- 172.21.15.139:0/4061418804 >> v1:172.21.15.139:6789/0 conn(0x7fe78015bf90 legacy=0x7fe780071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.138+0000 7fe785222640 1 -- 172.21.15.139:0/4061418804 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100439824 0 0) 0x7fe78015c490 con 0x7fe78015bf90 2024-04-25T09:57:28.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.138+0000 7fe786a25640 1 -- 172.21.15.139:0/4061418804 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78015cd30 con 0x7fe7800ce040 2024-04-25T09:57:28.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.138+0000 7fe786a25640 1 --2- 172.21.15.139:0/4061418804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7800ce040 0x7fe7800ce4f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe7780097a0 tx=0x7fe77802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77ce99815a7ab0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.138+0000 7fe785222640 1 -- 172.21.15.139:0/4061418804 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778037070 con 0x7fe7800ce040 2024-04-25T09:57:28.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.138+0000 7fe785222640 1 -- 172.21.15.139:0/4061418804 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77802ae50 con 0x7fe7800ce040 2024-04-25T09:57:28.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.139+0000 7fe785222640 1 -- 172.21.15.139:0/4061418804 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778032660 con 0x7fe7800ce040 2024-04-25T09:57:28.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.139+0000 7fe788488640 1 -- 172.21.15.139:0/4061418804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7800ce040 msgr2=0x7fe7800ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.139+0000 7fe788488640 1 --2- 172.21.15.139:0/4061418804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7800ce040 0x7fe7800ce4f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe7780097a0 tx=0x7fe77802a100 comp rx=0 tx=0).stop 2024-04-25T09:57:28.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.139+0000 7fe788488640 1 -- 172.21.15.139:0/4061418804 shutdown_connections 2024-04-25T09:57:28.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.139+0000 7fe788488640 1 --2- 172.21.15.139:0/4061418804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7800ce040 0x7fe7800ce4f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.139+0000 7fe788488640 1 --2- 172.21.15.139:0/4061418804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe780071a50 0x7fe7800cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.139+0000 7fe788488640 1 -- 172.21.15.139:0/4061418804 >> 172.21.15.139:0/4061418804 conn(0x7fe7800cfd80 msgr2=0x7fe7800d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.139+0000 7fe788488640 1 -- 172.21.15.139:0/4061418804 shutdown_connections 2024-04-25T09:57:28.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.139+0000 7fe788488640 1 -- 172.21.15.139:0/4061418804 wait complete. 2024-04-25T09:57:28.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.140+0000 7fe788488640 1 Processor -- start 2024-04-25T09:57:28.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.140+0000 7fe788488640 1 -- start start 2024-04-25T09:57:28.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe788488640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe780071a50 0x7fe780153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe788488640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7800ce040 0x7fe7801f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe788488640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78015bf90 0x7fe7800ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe788488640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe780156280 con 0x7fe7800ce040 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe788488640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800728b0 con 0x7fe78015bf90 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe788488640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe780156400 con 0x7fe780071a50 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe785a23640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7800ce040 0x7fe7801f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe785a23640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7800ce040 0x7fe7801f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35440/0 (socket says 172.21.15.139:35440) 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe785a23640 1 -- 172.21.15.139:0/1452019470 learned_addr learned my addr 172.21.15.139:0/1452019470 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe785a23640 1 -- 172.21.15.139:0/1452019470 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe780071a50 msgr2=0x7fe780153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe785a23640 1 --2- 172.21.15.139:0/1452019470 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe780071a50 0x7fe780153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe785a23640 1 -- 172.21.15.139:0/1452019470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78015bf90 msgr2=0x7fe7800ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe785a23640 1 --2- 172.21.15.139:0/1452019470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78015bf90 0x7fe7800ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe785a23640 1 -- 172.21.15.139:0/1452019470 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78009be20 con 0x7fe7800ce040 2024-04-25T09:57:28.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.141+0000 7fe785a23640 1 --2- 172.21.15.139:0/1452019470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7800ce040 0x7fe7801f2350 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe77000c4a0 tx=0x7fe77000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.142+0000 7fe776ffd640 1 -- 172.21.15.139:0/1452019470 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770018070 con 0x7fe7800ce040 2024-04-25T09:57:28.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.142+0000 7fe776ffd640 1 -- 172.21.15.139:0/1452019470 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7700074e0 con 0x7fe7800ce040 2024-04-25T09:57:28.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.142+0000 7fe788488640 1 -- 172.21.15.139:0/1452019470 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78009c100 con 0x7fe7800ce040 2024-04-25T09:57:28.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.142+0000 7fe776ffd640 1 -- 172.21.15.139:0/1452019470 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770004a20 con 0x7fe7800ce040 2024-04-25T09:57:28.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.142+0000 7fe788488640 1 -- 172.21.15.139:0/1452019470 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78009c6c0 con 0x7fe7800ce040 2024-04-25T09:57:28.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.143+0000 7fe788488640 1 -- 172.21.15.139:0/1452019470 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe780074520 con 0x7fe7800ce040 2024-04-25T09:57:28.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.143+0000 7fe776ffd640 1 -- 172.21.15.139:0/1452019470 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe77001b070 con 0x7fe7800ce040 2024-04-25T09:57:28.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.143+0000 7fe776ffd640 1 --2- 172.21.15.139:0/1452019470 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe76406b4d0 0x7fe76406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.143+0000 7fe776ffd640 1 -- 172.21.15.139:0/1452019470 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7700896a0 con 0x7fe7800ce040 2024-04-25T09:57:28.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.144+0000 7fe786224640 1 --2- 172.21.15.139:0/1452019470 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe76406b4d0 0x7fe76406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.144+0000 7fe786224640 1 --2- 172.21.15.139:0/1452019470 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe76406b4d0 0x7fe76406d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe77c007dc0 tx=0x7fe77c0072f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.148+0000 7fe776ffd640 1 -- 172.21.15.139:0/1452019470 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe770058420 con 0x7fe7800ce040 2024-04-25T09:57:28.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.316+0000 7fe788488640 1 -- 172.21.15.139:0/1452019470 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7800d04e0 con 0x7fe7800ce040 2024-04-25T09:57:28.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.317+0000 7fe776ffd640 1 -- 172.21.15.139:0/1452019470 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe770057d20 con 0x7fe7800ce040 2024-04-25T09:57:28.320 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:57:28.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.319+0000 7fe788488640 1 -- 172.21.15.139:0/1452019470 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe76406b4d0 msgr2=0x7fe76406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.319+0000 7fe788488640 1 --2- 172.21.15.139:0/1452019470 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe76406b4d0 0x7fe76406d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe77c007dc0 tx=0x7fe77c0072f0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.319+0000 7fe788488640 1 -- 172.21.15.139:0/1452019470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7800ce040 msgr2=0x7fe7801f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.319+0000 7fe788488640 1 --2- 172.21.15.139:0/1452019470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7800ce040 0x7fe7801f2350 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe77000c4a0 tx=0x7fe77000c960 comp rx=0 tx=0).stop 2024-04-25T09:57:28.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.319+0000 7fe788488640 1 -- 172.21.15.139:0/1452019470 shutdown_connections 2024-04-25T09:57:28.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.319+0000 7fe788488640 1 --2- 172.21.15.139:0/1452019470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7800ce040 0x7fe7801f2350 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.319+0000 7fe788488640 1 --2- 172.21.15.139:0/1452019470 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe76406b4d0 0x7fe76406d990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.319+0000 7fe788488640 1 --2- 172.21.15.139:0/1452019470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe78015bf90 0x7fe7800ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.319+0000 7fe788488640 1 --2- 172.21.15.139:0/1452019470 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe780071a50 0x7fe780153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.319+0000 7fe788488640 1 -- 172.21.15.139:0/1452019470 >> 172.21.15.139:0/1452019470 conn(0x7fe7800cfd80 msgr2=0x7fe78015e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.319+0000 7fe788488640 1 -- 172.21.15.139:0/1452019470 shutdown_connections 2024-04-25T09:57:28.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.319+0000 7fe788488640 1 -- 172.21.15.139:0/1452019470 wait complete. 2024-04-25T09:57:28.333 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:57:28.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.429+0000 7f10630f5640 1 Processor -- start 2024-04-25T09:57:28.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.429+0000 7f10630f5640 1 -- start start 2024-04-25T09:57:28.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.429+0000 7f10630f5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f105c0712f0 0x7f105c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.429+0000 7f10630f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f105c0719d0 0x7f105c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.429+0000 7f10630f5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c0b1100 con 0x7f105c145250 2024-04-25T09:57:28.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.429+0000 7f10630f5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105c0ceed0 con 0x7f105c0712f0 2024-04-25T09:57:28.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.429+0000 7f10630f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0ce2c0 con 0x7f105c0719d0 2024-04-25T09:57:28.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.430+0000 7f105bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f105c0712f0 0x7f105c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.430+0000 7f105bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f105c0712f0 0x7f105c0b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35450/0 (socket says 172.21.15.139:35450) 2024-04-25T09:57:28.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.430+0000 7f105bfff640 1 -- 172.21.15.139:0/863935430 learned_addr learned my addr 172.21.15.139:0/863935430 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:28.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.430+0000 7f1061692640 1 --2- 172.21.15.139:0/863935430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f105c0719d0 0x7f105c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.430+0000 7f105bfff640 1 -- 172.21.15.139:0/863935430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f105c0719d0 msgr2=0x7f105c0b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.430+0000 7f105bfff640 1 --2- 172.21.15.139:0/863935430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f105c0719d0 0x7f105c0b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.430+0000 7f105bfff640 1 -- 172.21.15.139:0/863935430 >> v1:172.21.15.150:6789/0 conn(0x7f105c145250 legacy=0x7f105c070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.430+0000 7f105bfff640 1 -- 172.21.15.139:0/863935430 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c0ce5a0 con 0x7f105c0712f0 2024-04-25T09:57:28.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.431+0000 7f105bfff640 1 --2- 172.21.15.139:0/863935430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0712f0 0x7f105c0b4680 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1048009870 tx=0x7f104802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb3c56cea0096d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.431+0000 7f105b7fe640 1 -- 172.21.15.139:0/863935430 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048037070 con 0x7f105c0712f0 2024-04-25T09:57:28.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.431+0000 7f105b7fe640 1 -- 172.21.15.139:0/863935430 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1048032440 con 0x7f105c0712f0 2024-04-25T09:57:28.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.431+0000 7f105b7fe640 1 -- 172.21.15.139:0/863935430 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048032810 con 0x7f105c0712f0 2024-04-25T09:57:28.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.431+0000 7f10630f5640 1 -- 172.21.15.139:0/863935430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0712f0 msgr2=0x7f105c0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.431+0000 7f10630f5640 1 --2- 172.21.15.139:0/863935430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0712f0 0x7f105c0b4680 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1048009870 tx=0x7f104802a1c0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.431+0000 7f10630f5640 1 -- 172.21.15.139:0/863935430 shutdown_connections 2024-04-25T09:57:28.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.431+0000 7f10630f5640 1 --2- 172.21.15.139:0/863935430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f105c0719d0 0x7f105c0b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.431+0000 7f10630f5640 1 --2- 172.21.15.139:0/863935430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0712f0 0x7f105c0b4680 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.431+0000 7f10630f5640 1 -- 172.21.15.139:0/863935430 >> 172.21.15.139:0/863935430 conn(0x7f105c06f460 msgr2=0x7f105c06f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.431+0000 7f10630f5640 1 -- 172.21.15.139:0/863935430 shutdown_connections 2024-04-25T09:57:28.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.431+0000 7f10630f5640 1 -- 172.21.15.139:0/863935430 wait complete. 2024-04-25T09:57:28.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.433+0000 7f10630f5640 1 Processor -- start 2024-04-25T09:57:28.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.433+0000 7f10630f5640 1 -- start start 2024-04-25T09:57:28.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.433+0000 7f10630f5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f105c0712f0 0x7f105c1e5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.433+0000 7f10630f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0719d0 0x7f105c1e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.433+0000 7f10630f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f105c145250 0x7f105c13d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.433+0000 7f10630f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c145630 con 0x7f105c0719d0 2024-04-25T09:57:28.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.433+0000 7f10630f5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0ceb60 con 0x7f105c0712f0 2024-04-25T09:57:28.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.433+0000 7f10630f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0d11a0 con 0x7f105c145250 2024-04-25T09:57:28.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f105bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0719d0 0x7f105c1e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f105bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0719d0 0x7f105c1e40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35460/0 (socket says 172.21.15.139:35460) 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f105bfff640 1 -- 172.21.15.139:0/3594970551 learned_addr learned my addr 172.21.15.139:0/3594970551 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f1060e91640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f105c0712f0 0x7f105c1e5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f1061692640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f105c145250 0x7f105c13d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f105bfff640 1 -- 172.21.15.139:0/3594970551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f105c145250 msgr2=0x7f105c13d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f105bfff640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f105c145250 0x7f105c13d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f105bfff640 1 -- 172.21.15.139:0/3594970551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f105c0712f0 msgr2=0x7f105c1e5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f105bfff640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f105c0712f0 0x7f105c1e5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f105bfff640 1 -- 172.21.15.139:0/3594970551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c1e4610 con 0x7f105c0719d0 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f1060e91640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f105c0712f0 0x7f105c1e5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f1061692640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f105c145250 0x7f105c13d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f105bfff640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0719d0 0x7f105c1e40d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1048000c00 tx=0x7f10480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f10597fa640 1 -- 172.21.15.139:0/3594970551 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048037040 con 0x7f105c0719d0 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f10597fa640 1 -- 172.21.15.139:0/3594970551 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1048042070 con 0x7f105c0719d0 2024-04-25T09:57:28.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f10630f5640 1 -- 172.21.15.139:0/3594970551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c1e48b0 con 0x7f105c0719d0 2024-04-25T09:57:28.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f10597fa640 1 -- 172.21.15.139:0/3594970551 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048030880 con 0x7f105c0719d0 2024-04-25T09:57:28.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.434+0000 7f10630f5640 1 -- 172.21.15.139:0/3594970551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c1e4e70 con 0x7f105c0719d0 2024-04-25T09:57:28.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.435+0000 7f10630f5640 1 -- 172.21.15.139:0/3594970551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c074520 con 0x7f105c0719d0 2024-04-25T09:57:28.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.436+0000 7f10597fa640 1 -- 172.21.15.139:0/3594970551 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f104802ad10 con 0x7f105c0719d0 2024-04-25T09:57:28.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.436+0000 7f10597fa640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f104006b500 0x7f104006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.436+0000 7f10597fa640 1 -- 172.21.15.139:0/3594970551 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f10480aa810 con 0x7f105c0719d0 2024-04-25T09:57:28.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.436+0000 7f1060e91640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f104006b500 0x7f104006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.436+0000 7f1060e91640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f104006b500 0x7f104006d9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f10500015e0 tx=0x7f1050009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.440+0000 7f10597fa640 1 -- 172.21.15.139:0/3594970551 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1048079590 con 0x7f105c0719d0 2024-04-25T09:57:28.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.623+0000 7f10630f5640 1 -- 172.21.15.139:0/3594970551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f105c0701f0 con 0x7f105c0719d0 2024-04-25T09:57:28.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.623+0000 7f10597fa640 1 -- 172.21.15.139:0/3594970551 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v76) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1048078e90 con 0x7f105c0719d0 2024-04-25T09:57:28.626 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T09:57:28.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.625+0000 7f10630f5640 1 -- 172.21.15.139:0/3594970551 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f104006b500 msgr2=0x7f104006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.625+0000 7f10630f5640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f104006b500 0x7f104006d9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f10500015e0 tx=0x7f1050009040 comp rx=0 tx=0).stop 2024-04-25T09:57:28.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.625+0000 7f10630f5640 1 -- 172.21.15.139:0/3594970551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0719d0 msgr2=0x7f105c1e40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.625+0000 7f10630f5640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0719d0 0x7f105c1e40d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1048000c00 tx=0x7f10480046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.625+0000 7f10630f5640 1 -- 172.21.15.139:0/3594970551 shutdown_connections 2024-04-25T09:57:28.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.625+0000 7f10630f5640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f105c0719d0 0x7f105c1e40d0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.625+0000 7f10630f5640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f104006b500 0x7f104006d9c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.625+0000 7f10630f5640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f105c145250 0x7f105c13d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.625+0000 7f10630f5640 1 --2- 172.21.15.139:0/3594970551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f105c0712f0 0x7f105c1e5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.625+0000 7f10630f5640 1 -- 172.21.15.139:0/3594970551 >> 172.21.15.139:0/3594970551 conn(0x7f105c06f460 msgr2=0x7f105c1439e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.625+0000 7f10630f5640 1 -- 172.21.15.139:0/3594970551 shutdown_connections 2024-04-25T09:57:28.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.625+0000 7f10630f5640 1 -- 172.21.15.139:0/3594970551 wait complete. 2024-04-25T09:57:28.639 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:57:28.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:28.639 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:28.690 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:57:28.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:28.690 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:57:28.715 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:28.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.848+0000 7fa4c04b3640 1 Processor -- start 2024-04-25T09:57:28.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.848+0000 7fa4c04b3640 1 -- start start 2024-04-25T09:57:28.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.849+0000 7fa4c04b3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa4b8161850 0x7fa4b80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.849+0000 7fa4c04b3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b80cf250 con 0x7fa4b815dc60 2024-04-25T09:57:28.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.849+0000 7fa4c04b3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b80ce9b0 con 0x7fa4b80d1080 2024-04-25T09:57:28.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.849+0000 7fa4c04b3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4b80d3730 con 0x7fa4b8161850 2024-04-25T09:57:28.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.849+0000 7fa4bea50640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa4b8161850 0x7fa4b80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.849+0000 7fa4be24f640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa4b815dc60 0x7fa4b80d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:49432/0 (socket says 172.21.15.139:49432) 2024-04-25T09:57:28.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.849+0000 7fa4bea50640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa4b8161850 0x7fa4b80b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35476/0 (socket says 172.21.15.139:35476) 2024-04-25T09:57:28.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.849+0000 7fa4be24f640 1 -- 172.21.15.139:0/3245903501 learned_addr learned my addr 172.21.15.139:0/3245903501 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:28.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.850+0000 7fa4bea50640 1 -- 172.21.15.139:0/3245903501 >> v1:172.21.15.150:6789/0 conn(0x7fa4b80d1080 legacy=0x7fa4b80d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.850+0000 7fa4bea50640 1 -- 172.21.15.139:0/3245903501 >> v1:172.21.15.139:6789/0 conn(0x7fa4b815dc60 legacy=0x7fa4b80d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.850+0000 7fa4bea50640 1 -- 172.21.15.139:0/3245903501 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b80cec90 con 0x7fa4b8161850 2024-04-25T09:57:28.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.850+0000 7fa4bd24d640 1 -- 172.21.15.139:0/3245903501 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28969759 0 0) 0x7fa4b4001110 con 0x7fa4b815dc60 2024-04-25T09:57:28.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.850+0000 7fa4bea50640 1 --2- 172.21.15.139:0/3245903501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b8161850 0x7fa4b80b6c10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009d90 tx=0x7fa4b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b213593f605eeea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.851+0000 7fa4bd24d640 1 -- 172.21.15.139:0/3245903501 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0007740 con 0x7fa4b8161850 2024-04-25T09:57:28.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.851+0000 7fa4bd24d640 1 -- 172.21.15.139:0/3245903501 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b00078e0 con 0x7fa4b8161850 2024-04-25T09:57:28.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.851+0000 7fa4bd24d640 1 -- 172.21.15.139:0/3245903501 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0007cb0 con 0x7fa4b8161850 2024-04-25T09:57:28.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.851+0000 7fa4c04b3640 1 -- 172.21.15.139:0/3245903501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b8161850 msgr2=0x7fa4b80b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.851+0000 7fa4c04b3640 1 --2- 172.21.15.139:0/3245903501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b8161850 0x7fa4b80b6c10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009d90 tx=0x7fa4b002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.851+0000 7fa4c04b3640 1 -- 172.21.15.139:0/3245903501 shutdown_connections 2024-04-25T09:57:28.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.851+0000 7fa4c04b3640 1 --2- 172.21.15.139:0/3245903501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b8161850 0x7fa4b80b6c10 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.851+0000 7fa4c04b3640 1 -- 172.21.15.139:0/3245903501 >> 172.21.15.139:0/3245903501 conn(0x7fa4b80b5910 msgr2=0x7fa4b80b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.851+0000 7fa4c04b3640 1 -- 172.21.15.139:0/3245903501 shutdown_connections 2024-04-25T09:57:28.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.851+0000 7fa4c04b3640 1 -- 172.21.15.139:0/3245903501 wait complete. 2024-04-25T09:57:28.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.852+0000 7fa4c04b3640 1 Processor -- start 2024-04-25T09:57:28.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.852+0000 7fa4c04b3640 1 -- start start 2024-04-25T09:57:28.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.853+0000 7fa4c04b3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b80d1080 0x7fa4b8160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.853+0000 7fa4c04b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4b815dc60 0x7fa4b815c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.853+0000 7fa4c04b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b8161850 0x7fa4b815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.853+0000 7fa4c04b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b815e040 con 0x7fa4b8161850 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.853+0000 7fa4c04b3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b80d1fc0 con 0x7fa4b80d1080 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.853+0000 7fa4c04b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b815fd30 con 0x7fa4b815dc60 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.853+0000 7fa4bea50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b8161850 0x7fa4b815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.853+0000 7fa4bea50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b8161850 0x7fa4b815ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35490/0 (socket says 172.21.15.139:35490) 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.853+0000 7fa4bea50640 1 -- 172.21.15.139:0/2953719489 learned_addr learned my addr 172.21.15.139:0/2953719489 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.853+0000 7fa4be24f640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b80d1080 0x7fa4b8160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.853+0000 7fa4bda4e640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4b815dc60 0x7fa4b815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.854+0000 7fa4bea50640 1 -- 172.21.15.139:0/2953719489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4b815dc60 msgr2=0x7fa4b815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.854+0000 7fa4bea50640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4b815dc60 0x7fa4b815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.854+0000 7fa4bea50640 1 -- 172.21.15.139:0/2953719489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b80d1080 msgr2=0x7fa4b8160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.854+0000 7fa4bea50640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b80d1080 0x7fa4b8160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.854+0000 7fa4bea50640 1 -- 172.21.15.139:0/2953719489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b815f440 con 0x7fa4b8161850 2024-04-25T09:57:28.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.854+0000 7fa4be24f640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b80d1080 0x7fa4b8160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:28.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.854+0000 7fa4bea50640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b8161850 0x7fa4b815ceb0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009eb0 tx=0x7fa4b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.854+0000 7fa4aeffd640 1 -- 172.21.15.139:0/2953719489 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0040070 con 0x7fa4b8161850 2024-04-25T09:57:28.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.854+0000 7fa4aeffd640 1 -- 172.21.15.139:0/2953719489 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b0031c40 con 0x7fa4b8161850 2024-04-25T09:57:28.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.854+0000 7fa4c04b3640 1 -- 172.21.15.139:0/2953719489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b815f6e0 con 0x7fa4b8161850 2024-04-25T09:57:28.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.854+0000 7fa4aeffd640 1 -- 172.21.15.139:0/2953719489 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0003990 con 0x7fa4b8161850 2024-04-25T09:57:28.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.854+0000 7fa4c04b3640 1 -- 172.21.15.139:0/2953719489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b8161120 con 0x7fa4b8161850 2024-04-25T09:57:28.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.855+0000 7fa4c04b3640 1 -- 172.21.15.139:0/2953719489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b8074520 con 0x7fa4b8161850 2024-04-25T09:57:28.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.855+0000 7fa4aeffd640 1 -- 172.21.15.139:0/2953719489 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4b0039040 con 0x7fa4b8161850 2024-04-25T09:57:28.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.856+0000 7fa4aeffd640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa49c06b4d0 0x7fa49c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.856+0000 7fa4aeffd640 1 -- 172.21.15.139:0/2953719489 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa4b00ab270 con 0x7fa4b8161850 2024-04-25T09:57:28.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.856+0000 7fa4be24f640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa49c06b4d0 0x7fa49c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.856+0000 7fa4be24f640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa49c06b4d0 0x7fa49c06d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000f60 tx=0x7fa4b4007a30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:28.860+0000 7fa4aeffd640 1 -- 172.21.15.139:0/2953719489 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4b0079ff0 con 0x7fa4b8161850 2024-04-25T09:57:29.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.067+0000 7fa4c04b3640 1 -- 172.21.15.139:0/2953719489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4b80b50a0 con 0x7fa4b8161850 2024-04-25T09:57:29.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.068+0000 7fa4aeffd640 1 -- 172.21.15.139:0/2953719489 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fa4b00798f0 con 0x7fa4b8161850 2024-04-25T09:57:29.071 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:29.072 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":36,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1384380923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":1384380923},{"type":"v1","addr":"172.21.15.139:6837","nonce":1384380923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:20.444843+0000","modified":"2024-04-25T09:57:22.443106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6834/1932986598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1932986598},{"type":"v1","addr":"172.21.15.150:6834","nonce":1932986598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:57:29.072 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 36 2024-04-25T09:57:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.070+0000 7fa4c04b3640 1 -- 172.21.15.139:0/2953719489 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa49c06b4d0 msgr2=0x7fa49c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.070+0000 7fa4c04b3640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa49c06b4d0 0x7fa49c06d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000f60 tx=0x7fa4b4007a30 comp rx=0 tx=0).stop 2024-04-25T09:57:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.070+0000 7fa4c04b3640 1 -- 172.21.15.139:0/2953719489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b8161850 msgr2=0x7fa4b815ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.070+0000 7fa4c04b3640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b8161850 0x7fa4b815ceb0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009eb0 tx=0x7fa4b0004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.070+0000 7fa4c04b3640 1 -- 172.21.15.139:0/2953719489 shutdown_connections 2024-04-25T09:57:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.070+0000 7fa4c04b3640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b8161850 0x7fa4b815ceb0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.070+0000 7fa4c04b3640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa49c06b4d0 0x7fa49c06d990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.070+0000 7fa4c04b3640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4b815dc60 0x7fa4b815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.070+0000 7fa4c04b3640 1 --2- 172.21.15.139:0/2953719489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b80d1080 0x7fa4b8160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.070+0000 7fa4c04b3640 1 -- 172.21.15.139:0/2953719489 >> 172.21.15.139:0/2953719489 conn(0x7fa4b80b5910 msgr2=0x7fa4b80d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.070+0000 7fa4c04b3640 1 -- 172.21.15.139:0/2953719489 shutdown_connections 2024-04-25T09:57:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.070+0000 7fa4c04b3640 1 -- 172.21.15.139:0/2953719489 wait complete. 2024-04-25T09:57:29.084 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T09:57:29.216 INFO:teuthology.orchestra.run.smithi150.stdout:{"message":"","return_code":0} 2024-04-25T09:57:29.216 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:57:29.216 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:57:29.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.311+0000 7f5134db0640 1 Processor -- start 2024-04-25T09:57:29.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.311+0000 7f5134db0640 1 -- start start 2024-04-25T09:57:29.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f5134db0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f51300cbcf0 0x7f51300ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f5134db0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f51300cecd0 0x7f51301643f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f5134db0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51300cf270 con 0x7f51300cd0f0 2024-04-25T09:57:29.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f5134db0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51300cf550 con 0x7f51300cbcf0 2024-04-25T09:57:29.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f5134db0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51300cf830 con 0x7f51300cecd0 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f5127fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f51300cbcf0 0x7f51300ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f512e59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f51300cd0f0 0x7f51300cd4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:49446/0 (socket says 172.21.15.139:49446) 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f512e59c640 1 -- 172.21.15.139:0/2361642709 learned_addr learned my addr 172.21.15.139:0/2361642709 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f512ed9d640 1 --2- 172.21.15.139:0/2361642709 >> v2:172.21.15.150:3300/0 conn(0x7f51300cecd0 0x7f51301643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f512dd9b640 1 -- 172.21.15.139:0/2361642709 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616721128 0 0) 0x7f51300cf270 con 0x7f51300cd0f0 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f512dd9b640 1 -- 172.21.15.139:0/2361642709 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f511c005730 con 0x7f51300cd0f0 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f5127fff640 1 -- 172.21.15.139:0/2361642709 >> v2:172.21.15.150:3300/0 conn(0x7f51300cecd0 msgr2=0x7f51301643f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f5127fff640 1 --2- 172.21.15.139:0/2361642709 >> v2:172.21.15.150:3300/0 conn(0x7f51300cecd0 0x7f51301643f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f5127fff640 1 -- 172.21.15.139:0/2361642709 >> v1:172.21.15.139:6789/0 conn(0x7f51300cd0f0 legacy=0x7f51300cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.312+0000 7f5127fff640 1 -- 172.21.15.139:0/2361642709 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300cfb10 con 0x7f51300cbcf0 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.313+0000 7f5127fff640 1 --2- 172.21.15.139:0/2361642709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51300cbcf0 0x7f51300ce790 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f51300afec0 tx=0x7f511802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5828f7407293f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.313+0000 7f512dd9b640 1 -- 172.21.15.139:0/2361642709 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118037070 con 0x7f51300cbcf0 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.313+0000 7f512dd9b640 1 -- 172.21.15.139:0/2361642709 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5118032440 con 0x7f51300cbcf0 2024-04-25T09:57:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.313+0000 7f512dd9b640 1 -- 172.21.15.139:0/2361642709 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511802ae60 con 0x7f51300cbcf0 2024-04-25T09:57:29.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.313+0000 7f5134db0640 1 -- 172.21.15.139:0/2361642709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51300cbcf0 msgr2=0x7f51300ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.313+0000 7f5134db0640 1 --2- 172.21.15.139:0/2361642709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51300cbcf0 0x7f51300ce790 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f51300afec0 tx=0x7f511802a110 comp rx=0 tx=0).stop 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.313+0000 7f5134db0640 1 -- 172.21.15.139:0/2361642709 shutdown_connections 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.313+0000 7f5134db0640 1 --2- 172.21.15.139:0/2361642709 >> v2:172.21.15.150:3300/0 conn(0x7f51300cecd0 0x7f51301643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.313+0000 7f5134db0640 1 --2- 172.21.15.139:0/2361642709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51300cbcf0 0x7f51300ce790 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.313+0000 7f5134db0640 1 -- 172.21.15.139:0/2361642709 >> 172.21.15.139:0/2361642709 conn(0x7f51300b2230 msgr2=0x7f51300b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.314+0000 7f5134db0640 1 -- 172.21.15.139:0/2361642709 shutdown_connections 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.314+0000 7f5134db0640 1 -- 172.21.15.139:0/2361642709 wait complete. 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.315+0000 7f5134db0640 1 Processor -- start 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.315+0000 7f5134db0640 1 -- start start 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.315+0000 7f5134db0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300cbcf0 0x7f5130155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.315+0000 7f5134db0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51300cd0f0 0x7f51300b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.315+0000 7f5134db0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51300cecd0 0x7f51301f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.315+0000 7f5134db0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51300b2ea0 con 0x7f51300cd0f0 2024-04-25T09:57:29.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.315+0000 7f5134db0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51300cdbe0 con 0x7f51300cecd0 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.315+0000 7f5134db0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51300b3020 con 0x7f51300cbcf0 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5127fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51300cd0f0 0x7f51300b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5127fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51300cd0f0 0x7f51300b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35504/0 (socket says 172.21.15.139:35504) 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5127fff640 1 -- 172.21.15.139:0/1094924956 learned_addr learned my addr 172.21.15.139:0/1094924956 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f512ed9d640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51300cecd0 0x7f51301f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f512e59c640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300cbcf0 0x7f5130155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5127fff640 1 -- 172.21.15.139:0/1094924956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300cbcf0 msgr2=0x7f5130155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5127fff640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300cbcf0 0x7f5130155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5127fff640 1 -- 172.21.15.139:0/1094924956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51300cecd0 msgr2=0x7f51301f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5127fff640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51300cecd0 0x7f51301f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5127fff640 1 -- 172.21.15.139:0/1094924956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130154580 con 0x7f51300cd0f0 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f512e59c640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300cbcf0 0x7f5130155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5127fff640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51300cd0f0 0x7f51300b5890 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5118000c00 tx=0x7f51180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5126ffd640 1 -- 172.21.15.139:0/1094924956 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118041070 con 0x7f51300cd0f0 2024-04-25T09:57:29.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5126ffd640 1 -- 172.21.15.139:0/1094924956 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511803c890 con 0x7f51300cd0f0 2024-04-25T09:57:29.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5134db0640 1 -- 172.21.15.139:0/1094924956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5130154860 con 0x7f51300cd0f0 2024-04-25T09:57:29.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5126ffd640 1 -- 172.21.15.139:0/1094924956 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118037040 con 0x7f51300cd0f0 2024-04-25T09:57:29.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.316+0000 7f5134db0640 1 -- 172.21.15.139:0/1094924956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5130154e20 con 0x7f51300cd0f0 2024-04-25T09:57:29.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.317+0000 7f5134db0640 1 -- 172.21.15.139:0/1094924956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51300cf270 con 0x7f51300cd0f0 2024-04-25T09:57:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.318+0000 7f5126ffd640 1 -- 172.21.15.139:0/1094924956 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5118044070 con 0x7f51300cd0f0 2024-04-25T09:57:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.318+0000 7f5126ffd640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f511006b4d0 0x7f511006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.318+0000 7f5126ffd640 1 -- 172.21.15.139:0/1094924956 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f51180aacd0 con 0x7f51300cd0f0 2024-04-25T09:57:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.318+0000 7f512e59c640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f511006b4d0 0x7f511006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.319+0000 7f512e59c640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f511006b4d0 0x7f511006d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5120007fd0 tx=0x7f5120009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.322+0000 7f5126ffd640 1 -- 172.21.15.139:0/1094924956 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51300cf270 con 0x7f51300cd0f0 2024-04-25T09:57:29.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.526+0000 7f5134db0640 1 -- 172.21.15.139:0/1094924956 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f513006ea70 con 0x7f51300cd0f0 2024-04-25T09:57:29.529 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T09:57:29.526+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:57:29.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.528+0000 7f5126ffd640 1 -- 172.21.15.139:0/1094924956 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7f513006ea70 con 0x7f51300cd0f0 2024-04-25T09:57:29.531 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:57:29.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.530+0000 7f5134db0640 1 -- 172.21.15.139:0/1094924956 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f511006b4d0 msgr2=0x7f511006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.530+0000 7f5134db0640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f511006b4d0 0x7f511006d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5120007fd0 tx=0x7f5120009040 comp rx=0 tx=0).stop 2024-04-25T09:57:29.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.531+0000 7f5134db0640 1 -- 172.21.15.139:0/1094924956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51300cd0f0 msgr2=0x7f51300b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.531+0000 7f5134db0640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51300cd0f0 0x7f51300b5890 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5118000c00 tx=0x7f51180046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.531+0000 7f5134db0640 1 -- 172.21.15.139:0/1094924956 shutdown_connections 2024-04-25T09:57:29.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.531+0000 7f5134db0640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51300cd0f0 0x7f51300b5890 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.531+0000 7f5134db0640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f511006b4d0 0x7f511006d990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.531+0000 7f5134db0640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51300cecd0 0x7f51301f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.531+0000 7f5134db0640 1 --2- 172.21.15.139:0/1094924956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300cbcf0 0x7f5130155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.531+0000 7f5134db0640 1 -- 172.21.15.139:0/1094924956 >> 172.21.15.139:0/1094924956 conn(0x7f51300b2230 msgr2=0x7f513006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:29.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.531+0000 7f5134db0640 1 -- 172.21.15.139:0/1094924956 shutdown_connections 2024-04-25T09:57:29.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.531+0000 7f5134db0640 1 -- 172.21.15.139:0/1094924956 wait complete. 2024-04-25T09:57:29.537 INFO:tasks.ceph.mds.d.smithi150.stderr: -4254> 2024-04-25T09:57:26.352+0000 7fcab9dd5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:57:29.545 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T09:57:29.566 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-04-25T09:57:29.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.585+0000 7fc2c51fef40 1 Processor -- start 2024-04-25T09:57:29.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.585+0000 7fc2c51fef40 1 -- start start 2024-04-25T09:57:29.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c51fef40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54d99e0 0x5609a544e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c51fef40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609a5379bf0 0x5609a5375910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c51fef40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609a54e0430 0x5609a54e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c51fef40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a53bebb0 con 0x5609a54d99e0 2024-04-25T09:57:29.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c51fef40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a531ddf0 con 0x5609a54e0430 2024-04-25T09:57:29.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c51fef40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609a544fed0 con 0x5609a5379bf0 2024-04-25T09:57:29.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c2cff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54d99e0 0x5609a544e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c2cff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54d99e0 0x5609a544e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35532/0 (socket says 172.21.15.139:35532) 2024-04-25T09:57:29.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c2cff640 1 -- 172.21.15.139:0/648523223 learned_addr learned my addr 172.21.15.139:0/648523223 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:29.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c3500640 1 --2- 172.21.15.139:0/648523223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609a54e0430 0x5609a54e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c24fe640 1 --2- 172.21.15.139:0/648523223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609a5379bf0 0x5609a5375910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c2cff640 1 -- 172.21.15.139:0/648523223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609a5379bf0 msgr2=0x5609a5375910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c2cff640 1 --2- 172.21.15.139:0/648523223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609a5379bf0 0x5609a5375910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c2cff640 1 -- 172.21.15.139:0/648523223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609a54e0430 msgr2=0x5609a54e08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c2cff640 1 --2- 172.21.15.139:0/648523223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609a54e0430 0x5609a54e08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c2cff640 1 -- 172.21.15.139:0/648523223 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a52a3c50 con 0x5609a54d99e0 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c24fe640 1 --2- 172.21.15.139:0/648523223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609a5379bf0 0x5609a5375910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c3500640 1 --2- 172.21.15.139:0/648523223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609a54e0430 0x5609a54e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c2cff640 1 --2- 172.21.15.139:0/648523223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54d99e0 0x5609a544e7a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009910 tx=0x7fc2b4005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a032dc96559736df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c14fc640 1 -- 172.21.15.139:0/648523223 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4008950 con 0x5609a54d99e0 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c14fc640 1 -- 172.21.15.139:0/648523223 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b4008af0 con 0x5609a54d99e0 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.586+0000 7fc2c14fc640 1 -- 172.21.15.139:0/648523223 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4015470 con 0x5609a54d99e0 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.587+0000 7fc2c51fef40 1 -- 172.21.15.139:0/648523223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54d99e0 msgr2=0x5609a544e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.587+0000 7fc2c51fef40 1 --2- 172.21.15.139:0/648523223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54d99e0 0x5609a544e7a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009910 tx=0x7fc2b4005a00 comp rx=0 tx=0).stop 2024-04-25T09:57:29.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.587+0000 7fc2c51fef40 1 -- 172.21.15.139:0/648523223 shutdown_connections 2024-04-25T09:57:29.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.587+0000 7fc2c51fef40 1 --2- 172.21.15.139:0/648523223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609a54e0430 0x5609a54e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.587+0000 7fc2c51fef40 1 --2- 172.21.15.139:0/648523223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609a5379bf0 0x5609a5375910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.587+0000 7fc2c51fef40 1 --2- 172.21.15.139:0/648523223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54d99e0 0x5609a544e7a0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.587+0000 7fc2c51fef40 1 -- 172.21.15.139:0/648523223 >> 172.21.15.139:0/648523223 conn(0x5609a53a2170 msgr2=0x5609a53a50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:29.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.587+0000 7fc2c51fef40 1 -- 172.21.15.139:0/648523223 shutdown_connections 2024-04-25T09:57:29.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.587+0000 7fc2c51fef40 1 -- 172.21.15.139:0/648523223 wait complete. 2024-04-25T09:57:29.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.587+0000 7fc2c51fef40 1 Processor -- start 2024-04-25T09:57:29.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.588+0000 7fc2c51fef40 1 -- start start 2024-04-25T09:57:29.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.588+0000 7fc2c51fef40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609a5379bf0 0x5609a54df8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.588+0000 7fc2c51fef40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609a54d99e0 0x5609a54e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.588+0000 7fc2c51fef40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54e0430 0x5609a54e9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.588+0000 7fc2c51fef40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a544db10 con 0x5609a54e0430 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.588+0000 7fc2c51fef40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a53c1990 con 0x5609a54d99e0 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.588+0000 7fc2c51fef40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609a53bfe50 con 0x5609a5379bf0 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.588+0000 7fc2c3500640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54e0430 0x5609a54e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.588+0000 7fc2c3500640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54e0430 0x5609a54e9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35546/0 (socket says 172.21.15.139:35546) 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.588+0000 7fc2c3500640 1 -- 172.21.15.139:0/2764674484 learned_addr learned my addr 172.21.15.139:0/2764674484 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.588+0000 7fc2c2cff640 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609a5379bf0 0x5609a54df8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.588+0000 7fc2c24fe640 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609a54d99e0 0x5609a54e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.589+0000 7fc2c3500640 1 -- 172.21.15.139:0/2764674484 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609a5379bf0 msgr2=0x5609a54df8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.589+0000 7fc2c3500640 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609a5379bf0 0x5609a54df8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.589+0000 7fc2c3500640 1 -- 172.21.15.139:0/2764674484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609a54d99e0 msgr2=0x5609a54e8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.589+0000 7fc2c3500640 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609a54d99e0 0x5609a54e8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.589+0000 7fc2c3500640 1 -- 172.21.15.139:0/2764674484 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a5448aa0 con 0x5609a54e0430 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.589+0000 7fc2c3500640 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54e0430 0x5609a54e9060 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc000f80 tx=0x5609a544e2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.589+0000 7fc2abfff640 1 -- 172.21.15.139:0/2764674484 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0069a0 con 0x5609a54e0430 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.589+0000 7fc2abfff640 1 -- 172.21.15.139:0/2764674484 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc006070 con 0x5609a54e0430 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.589+0000 7fc2c51fef40 1 -- 172.21.15.139:0/2764674484 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5609a53c03f0 con 0x5609a54e0430 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.589+0000 7fc2abfff640 1 -- 172.21.15.139:0/2764674484 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc04d950 con 0x5609a54e0430 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.589+0000 7fc2c51fef40 1 -- 172.21.15.139:0/2764674484 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609a53c0110 con 0x5609a54e0430 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.590+0000 7fc2abfff640 1 -- 172.21.15.139:0/2764674484 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2bc060070 con 0x5609a54e0430 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.591+0000 7fc2abfff640 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc2b0069ee0 0x7fc2b006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.591+0000 7fc2c2cff640 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc2b0069ee0 0x7fc2b006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.591+0000 7fc2abfff640 1 -- 172.21.15.139:0/2764674484 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc2bc0cad20 con 0x5609a54e0430 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.591+0000 7fc2c51fef40 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x5609a53a3350 0x5609a5376c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.591+0000 7fc2c51fef40 1 -- 172.21.15.139:0/2764674484 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77) v9 -- 0x5609a54e95a0 con 0x5609a53a3350 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.591+0000 7fc2c24fe640 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x5609a53a3350 0x5609a5376c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.591+0000 7fc2c2cff640 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc2b0069ee0 0x7fc2b006c3a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4005a40 tx=0x7fc2b4015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.591+0000 7fc2c24fe640 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x5609a53a3350 0x5609a5376c10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.594+0000 7fc2c24fe640 1 -- 172.21.15.139:0/2764674484 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v77'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc2b8009b60 con 0x5609a53a3350 2024-04-25T09:57:29.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.594+0000 7fc2c51fef40 1 -- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x5609a53a3350 msgr2=0x5609a5376c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.595+0000 7fc2c51fef40 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x5609a53a3350 0x5609a5376c10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.595+0000 7fc2c51fef40 1 -- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc2b0069ee0 msgr2=0x7fc2b006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.595+0000 7fc2c51fef40 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc2b0069ee0 0x7fc2b006c3a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4005a40 tx=0x7fc2b4015000 comp rx=0 tx=0).stop 2024-04-25T09:57:29.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.595+0000 7fc2c51fef40 1 -- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54e0430 msgr2=0x5609a54e9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.595+0000 7fc2c51fef40 1 --2- 172.21.15.139:0/2764674484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609a54e0430 0x5609a54e9060 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc000f80 tx=0x5609a544e2e0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.595+0000 7fc2c3500640 1 -- 172.21.15.139:0/2764674484 reap_dead start 2024-04-25T09:57:29.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.595+0000 7fc2c51fef40 1 -- 172.21.15.139:0/2764674484 shutdown_connections 2024-04-25T09:57:29.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.595+0000 7fc2c51fef40 1 -- 172.21.15.139:0/2764674484 >> 172.21.15.139:0/2764674484 conn(0x5609a53a2170 msgr2=0x5609a537a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:29.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.595+0000 7fc2c51fef40 1 -- 172.21.15.139:0/2764674484 shutdown_connections 2024-04-25T09:57:29.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.595+0000 7fc2c51fef40 1 -- 172.21.15.139:0/2764674484 wait complete. 2024-04-25T09:57:29.600 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T09:57:29.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.696+0000 7fedd4f09640 1 Processor -- start 2024-04-25T09:57:29.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.696+0000 7fedd4f09640 1 -- start start 2024-04-25T09:57:29.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedd4f09640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fedd0145230 0x7fedd015be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedd4f09640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fedd00718d0 0x7fedd015da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedd4f09640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd015c960 con 0x7fedd015c3d0 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedd4f09640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd015cc40 con 0x7fedd0145230 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedd4f09640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd015d230 con 0x7fedd00718d0 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedce59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fedd0145230 0x7fedd015be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedce59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fedd0145230 0x7fedd015be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35562/0 (socket says 172.21.15.139:35562) 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedce59c640 1 -- 172.21.15.139:0/1276917733 learned_addr learned my addr 172.21.15.139:0/1276917733 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedced9d640 1 --2- 172.21.15.139:0/1276917733 >> v2:172.21.15.150:3300/0 conn(0x7fedd00718d0 0x7fedd015da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedce59c640 1 -- 172.21.15.139:0/1276917733 >> v2:172.21.15.150:3300/0 conn(0x7fedd00718d0 msgr2=0x7fedd015da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedce59c640 1 --2- 172.21.15.139:0/1276917733 >> v2:172.21.15.150:3300/0 conn(0x7fedd00718d0 0x7fedd015da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedce59c640 1 -- 172.21.15.139:0/1276917733 >> v1:172.21.15.150:6789/0 conn(0x7fedd015c3d0 legacy=0x7fedd00711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:29.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.697+0000 7fedce59c640 1 -- 172.21.15.139:0/1276917733 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd015d510 con 0x7fedd0145230 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.698+0000 7fedce59c640 1 --2- 172.21.15.139:0/1276917733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd0145230 0x7fedd015be90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fedd015f3e0 tx=0x7fedc002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de3fde76ed58dca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.698+0000 7fedcd59a640 1 -- 172.21.15.139:0/1276917733 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0036070 con 0x7fedd0145230 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.698+0000 7fedcd59a640 1 -- 172.21.15.139:0/1276917733 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc0031600 con 0x7fedd0145230 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.698+0000 7fedcd59a640 1 -- 172.21.15.139:0/1276917733 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc00319d0 con 0x7fedd0145230 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.698+0000 7fedd4f09640 1 -- 172.21.15.139:0/1276917733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd0145230 msgr2=0x7fedd015be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.698+0000 7fedd4f09640 1 --2- 172.21.15.139:0/1276917733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd0145230 0x7fedd015be90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fedd015f3e0 tx=0x7fedc002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.699+0000 7fedd4f09640 1 -- 172.21.15.139:0/1276917733 shutdown_connections 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.699+0000 7fedd4f09640 1 --2- 172.21.15.139:0/1276917733 >> v2:172.21.15.150:3300/0 conn(0x7fedd00718d0 0x7fedd015da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.699+0000 7fedd4f09640 1 --2- 172.21.15.139:0/1276917733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd0145230 0x7fedd015be90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.699+0000 7fedd4f09640 1 -- 172.21.15.139:0/1276917733 >> 172.21.15.139:0/1276917733 conn(0x7fedd0067000 msgr2=0x7fedd006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:29.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.699+0000 7fedd4f09640 1 -- 172.21.15.139:0/1276917733 shutdown_connections 2024-04-25T09:57:29.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.699+0000 7fedd4f09640 1 -- 172.21.15.139:0/1276917733 wait complete. 2024-04-25T09:57:29.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.700+0000 7fedd4f09640 1 Processor -- start 2024-04-25T09:57:29.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.700+0000 7fedd4f09640 1 -- start start 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedd4f09640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedd00718d0 0x7fedd01e4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedd4f09640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedd0145230 0x7fedd015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedd4f09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd015c3d0 0x7fedd012d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedd4f09640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd015e0f0 con 0x7fedd015c3d0 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedd4f09640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd015df70 con 0x7fedd0145230 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedd4f09640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd015e270 con 0x7fedd00718d0 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedced9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd015c3d0 0x7fedd012d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedced9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd015c3d0 0x7fedd012d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35566/0 (socket says 172.21.15.139:35566) 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedced9d640 1 -- 172.21.15.139:0/3802719298 learned_addr learned my addr 172.21.15.139:0/3802719298 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedcdd9b640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedd0145230 0x7fedd015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedce59c640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedd00718d0 0x7fedd01e4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedced9d640 1 -- 172.21.15.139:0/3802719298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedd00718d0 msgr2=0x7fedd01e4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedced9d640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedd00718d0 0x7fedd01e4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedced9d640 1 -- 172.21.15.139:0/3802719298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedd0145230 msgr2=0x7fedd015b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:29.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedced9d640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedd0145230 0x7fedd015b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:29.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedced9d640 1 -- 172.21.15.139:0/3802719298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd01e57a0 con 0x7fedd015c3d0 2024-04-25T09:57:29.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedced9d640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd015c3d0 0x7fedd012d630 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fedc800c7f0 tx=0x7fedc800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedbeffd640 1 -- 172.21.15.139:0/3802719298 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8015070 con 0x7fedd015c3d0 2024-04-25T09:57:29.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedbeffd640 1 -- 172.21.15.139:0/3802719298 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc8004220 con 0x7fedd015c3d0 2024-04-25T09:57:29.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.701+0000 7fedd4f09640 1 -- 172.21.15.139:0/3802719298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd01e5a60 con 0x7fedd015c3d0 2024-04-25T09:57:29.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.702+0000 7fedbeffd640 1 -- 172.21.15.139:0/3802719298 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8007620 con 0x7fedd015c3d0 2024-04-25T09:57:29.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.702+0000 7fedd4f09640 1 -- 172.21.15.139:0/3802719298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd012db70 con 0x7fedd015c3d0 2024-04-25T09:57:29.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.703+0000 7fedd4f09640 1 -- 172.21.15.139:0/3802719298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd006baf0 con 0x7fedd015c3d0 2024-04-25T09:57:29.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.703+0000 7fedbeffd640 1 -- 172.21.15.139:0/3802719298 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fedc80079d0 con 0x7fedd015c3d0 2024-04-25T09:57:29.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.703+0000 7fedbeffd640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fedb006b500 0x7fedb006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:29.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.703+0000 7fedbeffd640 1 -- 172.21.15.139:0/3802719298 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fedc808a3c0 con 0x7fedd015c3d0 2024-04-25T09:57:29.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.703+0000 7fedce59c640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fedb006b500 0x7fedb006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:29.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.704+0000 7fedce59c640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fedb006b500 0x7fedb006d9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fedc0033000 tx=0x7fedc0009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:29.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.708+0000 7fedbeffd640 1 -- 172.21.15.139:0/3802719298 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fedc8058970 con 0x7fedd015c3d0 2024-04-25T09:57:29.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:29.915+0000 7fedd4f09640 1 -- 172.21.15.139:0/3802719298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fedd0074520 con 0x7fedd015c3d0 2024-04-25T09:57:30.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.530+0000 7fedbeffd640 1 -- 172.21.15.139:0/3802719298 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v38) v1 ==== 162+0+0 (secure 0 0 0) 0x7fedc805ca10 con 0x7fedd015c3d0 2024-04-25T09:57:30.533 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T09:57:30.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.532+0000 7fedd4f09640 1 -- 172.21.15.139:0/3802719298 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fedb006b500 msgr2=0x7fedb006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.532+0000 7fedd4f09640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fedb006b500 0x7fedb006d9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fedc0033000 tx=0x7fedc0009000 comp rx=0 tx=0).stop 2024-04-25T09:57:30.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.532+0000 7fedd4f09640 1 -- 172.21.15.139:0/3802719298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd015c3d0 msgr2=0x7fedd012d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.532+0000 7fedd4f09640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd015c3d0 0x7fedd012d630 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fedc800c7f0 tx=0x7fedc800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.532+0000 7fedd4f09640 1 -- 172.21.15.139:0/3802719298 shutdown_connections 2024-04-25T09:57:30.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.532+0000 7fedd4f09640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd015c3d0 0x7fedd012d630 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.532+0000 7fedd4f09640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fedb006b500 0x7fedb006d9c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.532+0000 7fedd4f09640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedd0145230 0x7fedd015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.532+0000 7fedd4f09640 1 --2- 172.21.15.139:0/3802719298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedd00718d0 0x7fedd01e4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.532+0000 7fedd4f09640 1 -- 172.21.15.139:0/3802719298 >> 172.21.15.139:0/3802719298 conn(0x7fedd0067000 msgr2=0x7fedd0144340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:30.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.532+0000 7fedd4f09640 1 -- 172.21.15.139:0/3802719298 shutdown_connections 2024-04-25T09:57:30.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.533+0000 7fedd4f09640 1 -- 172.21.15.139:0/3802719298 wait complete. 2024-04-25T09:57:30.550 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:30.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.647+0000 7f961d1f5640 1 Processor -- start 2024-04-25T09:57:30.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.647+0000 7f961d1f5640 1 -- start start 2024-04-25T09:57:30.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f961d1f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f961815efb0 0x7f96180d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f961d1f5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f96180cf900 0x7f961806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f961d1f5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96180cfe70 con 0x7f961806cd80 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f961d1f5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96180d0a80 con 0x7f96180cf900 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f961d1f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96180d0150 con 0x7f961815efb0 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f961659c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f96180cf900 0x7f961806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f961759e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f961806cd80 0x7f9618099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:49466/0 (socket says 172.21.15.139:49466) 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f961759e640 1 -- 172.21.15.139:0/1213239829 learned_addr learned my addr 172.21.15.139:0/1213239829 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f9616d9d640 1 --2- 172.21.15.139:0/1213239829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f961815efb0 0x7f96180d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f961659c640 1 -- 172.21.15.139:0/1213239829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f961815efb0 msgr2=0x7f96180d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f961659c640 1 --2- 172.21.15.139:0/1213239829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f961815efb0 0x7f96180d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f961659c640 1 -- 172.21.15.139:0/1213239829 >> v1:172.21.15.139:6789/0 conn(0x7f961806cd80 legacy=0x7f9618099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f961659c640 1 -- 172.21.15.139:0/1213239829 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96180d0430 con 0x7f96180cf900 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.648+0000 7f9615d9b640 1 -- 172.21.15.139:0/1213239829 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288560019 0 0) 0x7f96180cfe70 con 0x7f961806cd80 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.649+0000 7f961659c640 1 --2- 172.21.15.139:0/1213239829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96180cf900 0x7f961806c840 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f960c0097a0 tx=0x7f960c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9776bf358c2e0cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.649+0000 7f9615d9b640 1 -- 172.21.15.139:0/1213239829 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c037070 con 0x7f96180cf900 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.649+0000 7f9615d9b640 1 -- 172.21.15.139:0/1213239829 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c02ae50 con 0x7f96180cf900 2024-04-25T09:57:30.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.649+0000 7f9615d9b640 1 -- 172.21.15.139:0/1213239829 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c032660 con 0x7f96180cf900 2024-04-25T09:57:30.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.650+0000 7f961d1f5640 1 -- 172.21.15.139:0/1213239829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96180cf900 msgr2=0x7f961806c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.650+0000 7f961d1f5640 1 --2- 172.21.15.139:0/1213239829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96180cf900 0x7f961806c840 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f960c0097a0 tx=0x7f960c02a100 comp rx=0 tx=0).stop 2024-04-25T09:57:30.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.650+0000 7f961d1f5640 1 -- 172.21.15.139:0/1213239829 shutdown_connections 2024-04-25T09:57:30.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.650+0000 7f961d1f5640 1 --2- 172.21.15.139:0/1213239829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96180cf900 0x7f961806c840 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.650+0000 7f961d1f5640 1 --2- 172.21.15.139:0/1213239829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f961815efb0 0x7f96180d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.650+0000 7f961d1f5640 1 -- 172.21.15.139:0/1213239829 >> 172.21.15.139:0/1213239829 conn(0x7f96180d1790 msgr2=0x7f96180d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:30.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.650+0000 7f961d1f5640 1 -- 172.21.15.139:0/1213239829 shutdown_connections 2024-04-25T09:57:30.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.650+0000 7f961d1f5640 1 -- 172.21.15.139:0/1213239829 wait complete. 2024-04-25T09:57:30.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.651+0000 7f961d1f5640 1 Processor -- start 2024-04-25T09:57:30.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.651+0000 7f961d1f5640 1 -- start start 2024-04-25T09:57:30.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961d1f5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f961806cd80 0x7f961815de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961d1f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96180cf900 0x7f96181e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961d1f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f961815efb0 0x7f96181e8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961d1f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961809a2e0 con 0x7f961815efb0 2024-04-25T09:57:30.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961d1f5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961809a160 con 0x7f961806cd80 2024-04-25T09:57:30.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961d1f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96180cbde0 con 0x7f96180cf900 2024-04-25T09:57:30.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f961815efb0 0x7f96181e8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f961815efb0 0x7f96181e8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35574/0 (socket says 172.21.15.139:35574) 2024-04-25T09:57:30.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961759e640 1 -- 172.21.15.139:0/1475472058 learned_addr learned my addr 172.21.15.139:0/1475472058 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:30.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961659c640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96180cf900 0x7f96181e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f9616d9d640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f961806cd80 0x7f961815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961759e640 1 -- 172.21.15.139:0/1475472058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96180cf900 msgr2=0x7f96181e80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961759e640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96180cf900 0x7f96181e80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961759e640 1 -- 172.21.15.139:0/1475472058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f961806cd80 msgr2=0x7f961815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961759e640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f961806cd80 0x7f961815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.652+0000 7f961759e640 1 -- 172.21.15.139:0/1475472058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96181f3bc0 con 0x7f961815efb0 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.653+0000 7f961759e640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f961815efb0 0x7f96181e8600 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f961000c730 tx=0x7f961000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.653+0000 7f95ff7fe640 1 -- 172.21.15.139:0/1475472058 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610018070 con 0x7f961815efb0 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.653+0000 7f95ff7fe640 1 -- 172.21.15.139:0/1475472058 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961000ce40 con 0x7f961815efb0 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.653+0000 7f961d1f5640 1 -- 172.21.15.139:0/1475472058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96181e8be0 con 0x7f961815efb0 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.653+0000 7f95ff7fe640 1 -- 172.21.15.139:0/1475472058 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961000a860 con 0x7f961815efb0 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.653+0000 7f961d1f5640 1 -- 172.21.15.139:0/1475472058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96180b6280 con 0x7f961815efb0 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.654+0000 7f961d1f5640 1 -- 172.21.15.139:0/1475472058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9618074520 con 0x7f961815efb0 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.654+0000 7f95ff7fe640 1 -- 172.21.15.139:0/1475472058 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f961001b070 con 0x7f961815efb0 2024-04-25T09:57:30.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.655+0000 7f95ff7fe640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f960406b4d0 0x7f960406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:30.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.655+0000 7f9616d9d640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f960406b4d0 0x7f960406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:30.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.655+0000 7f95ff7fe640 1 -- 172.21.15.139:0/1475472058 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9610089810 con 0x7f961815efb0 2024-04-25T09:57:30.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.655+0000 7f9616d9d640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f960406b4d0 0x7f960406d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f96080079a0 tx=0x7f9608008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:30.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.659+0000 7f95ff7fe640 1 -- 172.21.15.139:0/1475472058 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9610057ef0 con 0x7f961815efb0 2024-04-25T09:57:30.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.864+0000 7f961d1f5640 1 -- 172.21.15.139:0/1475472058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f961809ae50 con 0x7f961815efb0 2024-04-25T09:57:30.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.865+0000 7f95ff7fe640 1 -- 172.21.15.139:0/1475472058 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f961005bf90 con 0x7f961815efb0 2024-04-25T09:57:30.868 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:30.869 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/982251530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":982251530},{"type":"v1","addr":"172.21.15.150:6834","nonce":982251530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:20.444843+0000","modified":"2024-04-25T09:57:30.533163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6837/1384380923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":1384380923},{"type":"v1","addr":"172.21.15.139:6837","nonce":1384380923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:57:30.869 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 39 2024-04-25T09:57:30.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.867+0000 7f961d1f5640 1 -- 172.21.15.139:0/1475472058 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f960406b4d0 msgr2=0x7f960406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.867+0000 7f961d1f5640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f960406b4d0 0x7f960406d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f96080079a0 tx=0x7f9608008040 comp rx=0 tx=0).stop 2024-04-25T09:57:30.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.867+0000 7f961d1f5640 1 -- 172.21.15.139:0/1475472058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f961815efb0 msgr2=0x7f96181e8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:30.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.867+0000 7f961d1f5640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f961815efb0 0x7f96181e8600 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f961000c730 tx=0x7f961000cbf0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.867+0000 7f961d1f5640 1 -- 172.21.15.139:0/1475472058 shutdown_connections 2024-04-25T09:57:30.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.867+0000 7f961d1f5640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f961815efb0 0x7f96181e8600 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.867+0000 7f961d1f5640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f960406b4d0 0x7f960406d990 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.867+0000 7f961d1f5640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96180cf900 0x7f96181e80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.867+0000 7f961d1f5640 1 --2- 172.21.15.139:0/1475472058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f961806cd80 0x7f961815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:30.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.867+0000 7f961d1f5640 1 -- 172.21.15.139:0/1475472058 >> 172.21.15.139:0/1475472058 conn(0x7f96180d1790 msgr2=0x7f961806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:30.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.867+0000 7f961d1f5640 1 -- 172.21.15.139:0/1475472058 shutdown_connections 2024-04-25T09:57:30.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:30.867+0000 7f961d1f5640 1 -- 172.21.15.139:0/1475472058 wait complete. 2024-04-25T09:57:31.283 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:57:20.444843+0000', 'modified': '2024-04-25T09:57:30.533163+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 77, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6876}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6876': {'gid': 6876, 'name': 'c', 'rank': 0, 'incarnation': 39, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.139:6837/1384380923', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 1384380923}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 1384380923}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:57:31.284 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6876:up:replay 2024-04-25T09:57:32.285 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.390+0000 7f6396e93640 1 Processor -- start 2024-04-25T09:57:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.390+0000 7f6396e93640 1 -- start start 2024-04-25T09:57:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f6396e93640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f639015cde0 0x7f639015d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f6396e93640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f63900cc2a0 0x7f639009a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f6396e93640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639015d1c0 con 0x7f639015db50 2024-04-25T09:57:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f6396e93640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63900cc720 con 0x7f63900cc2a0 2024-04-25T09:57:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f6396e93640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63900cd460 con 0x7f639015cde0 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f638ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f63900cc2a0 0x7f639009a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f638ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f63900cc2a0 0x7f639009a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35586/0 (socket says 172.21.15.139:35586) 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f638ffff640 1 -- 172.21.15.139:0/3117914560 learned_addr learned my addr 172.21.15.139:0/3117914560 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f6394c2f640 1 --2- 172.21.15.139:0/3117914560 >> v2:172.21.15.150:3300/0 conn(0x7f639015cde0 0x7f639015d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f638ffff640 1 -- 172.21.15.139:0/3117914560 >> v2:172.21.15.150:3300/0 conn(0x7f639015cde0 msgr2=0x7f639015d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f638ffff640 1 --2- 172.21.15.139:0/3117914560 >> v2:172.21.15.150:3300/0 conn(0x7f639015cde0 0x7f639015d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f638ffff640 1 -- 172.21.15.139:0/3117914560 >> v1:172.21.15.150:6789/0 conn(0x7f639015db50 legacy=0x7f63900cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f638ffff640 1 -- 172.21.15.139:0/3117914560 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6390099bf0 con 0x7f63900cc2a0 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.391+0000 7f6394c2f640 1 --2- 172.21.15.139:0/3117914560 >> v2:172.21.15.150:3300/0 conn(0x7f639015cde0 0x7f639015d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.392+0000 7f638ffff640 1 --2- 172.21.15.139:0/3117914560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63900cc2a0 0x7f639009a3f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6390072c10 tx=0x7f637c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff635e592c04ea79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.392+0000 7f638f7fe640 1 -- 172.21.15.139:0/3117914560 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c037070 con 0x7f63900cc2a0 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.392+0000 7f638f7fe640 1 -- 172.21.15.139:0/3117914560 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c032440 con 0x7f63900cc2a0 2024-04-25T09:57:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.392+0000 7f638f7fe640 1 -- 172.21.15.139:0/3117914560 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c032810 con 0x7f63900cc2a0 2024-04-25T09:57:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.393+0000 7f6396e93640 1 -- 172.21.15.139:0/3117914560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63900cc2a0 msgr2=0x7f639009a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.393+0000 7f6396e93640 1 --2- 172.21.15.139:0/3117914560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63900cc2a0 0x7f639009a3f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6390072c10 tx=0x7f637c02a270 comp rx=0 tx=0).stop 2024-04-25T09:57:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.393+0000 7f6396e93640 1 -- 172.21.15.139:0/3117914560 shutdown_connections 2024-04-25T09:57:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.393+0000 7f6396e93640 1 --2- 172.21.15.139:0/3117914560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63900cc2a0 0x7f639009a3f0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.393+0000 7f6396e93640 1 --2- 172.21.15.139:0/3117914560 >> v2:172.21.15.150:3300/0 conn(0x7f639015cde0 0x7f639015d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.393+0000 7f6396e93640 1 -- 172.21.15.139:0/3117914560 >> 172.21.15.139:0/3117914560 conn(0x7f63900d1790 msgr2=0x7f63900d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.393+0000 7f6396e93640 1 -- 172.21.15.139:0/3117914560 shutdown_connections 2024-04-25T09:57:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.393+0000 7f6396e93640 1 -- 172.21.15.139:0/3117914560 wait complete. 2024-04-25T09:57:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.394+0000 7f6396e93640 1 Processor -- start 2024-04-25T09:57:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.394+0000 7f6396e93640 1 -- start start 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6396e93640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63900cc2a0 0x7f63900b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6396e93640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f639015cde0 0x7f63900b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6396e93640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f639015db50 0x7f63900cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6396e93640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63900cf4e0 con 0x7f63900cc2a0 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6396e93640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63900cf360 con 0x7f639015db50 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6396e93640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63900b53f0 con 0x7f639015cde0 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6394c2f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63900cc2a0 0x7f63900b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6394c2f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63900cc2a0 0x7f63900b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35596/0 (socket says 172.21.15.139:35596) 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6394c2f640 1 -- 172.21.15.139:0/2950055663 learned_addr learned my addr 172.21.15.139:0/2950055663 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6395430640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f639015db50 0x7f63900cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f638ffff640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f639015cde0 0x7f63900b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6394c2f640 1 -- 172.21.15.139:0/2950055663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f639015cde0 msgr2=0x7f63900b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6394c2f640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f639015cde0 0x7f63900b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6394c2f640 1 -- 172.21.15.139:0/2950055663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f639015db50 msgr2=0x7f63900cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6394c2f640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f639015db50 0x7f63900cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6394c2f640 1 -- 172.21.15.139:0/2950055663 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63900b4e30 con 0x7f63900cc2a0 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.395+0000 7f6395430640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f639015db50 0x7f63900cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.396+0000 7f6394c2f640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63900cc2a0 0x7f63900b45e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f638400d7d0 tx=0x7f638400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.396+0000 7f638d7fa640 1 -- 172.21.15.139:0/2950055663 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6384004030 con 0x7f63900cc2a0 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.396+0000 7f638d7fa640 1 -- 172.21.15.139:0/2950055663 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63840041d0 con 0x7f63900cc2a0 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.396+0000 7f6396e93640 1 -- 172.21.15.139:0/2950055663 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63900b5170 con 0x7f63900cc2a0 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.396+0000 7f638d7fa640 1 -- 172.21.15.139:0/2950055663 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63840096a0 con 0x7f63900cc2a0 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.396+0000 7f6396e93640 1 -- 172.21.15.139:0/2950055663 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63900b4b20 con 0x7f63900cc2a0 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.397+0000 7f6396e93640 1 -- 172.21.15.139:0/2950055663 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6390074520 con 0x7f63900cc2a0 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.397+0000 7f638d7fa640 1 -- 172.21.15.139:0/2950055663 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6384009840 con 0x7f63900cc2a0 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.398+0000 7f638d7fa640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f637406b4d0 0x7f637406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.398+0000 7f638d7fa640 1 -- 172.21.15.139:0/2950055663 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f638408a770 con 0x7f63900cc2a0 2024-04-25T09:57:32.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.398+0000 7f638ffff640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f637406b4d0 0x7f637406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:32.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.398+0000 7f638ffff640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f637406b4d0 0x7f637406d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f637c034000 tx=0x7f637c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:32.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.402+0000 7f638d7fa640 1 -- 172.21.15.139:0/2950055663 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63900b5170 con 0x7f63900cc2a0 2024-04-25T09:57:32.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.611+0000 7f6396e93640 1 -- 172.21.15.139:0/2950055663 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63900b6600 con 0x7f63900cc2a0 2024-04-25T09:57:32.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.611+0000 7f638d7fa640 1 -- 172.21.15.139:0/2950055663 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f6384059380 con 0x7f63900cc2a0 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/982251530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":982251530},{"type":"v1","addr":"172.21.15.150:6834","nonce":982251530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:20.444843+0000","modified":"2024-04-25T09:57:31.538573+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.139:6837/1384380923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":1384380923},{"type":"v1","addr":"172.21.15.139:6837","nonce":1384380923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:57:32.615 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 41 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.613+0000 7f6396e93640 1 -- 172.21.15.139:0/2950055663 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f637406b4d0 msgr2=0x7f637406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.613+0000 7f6396e93640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f637406b4d0 0x7f637406d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f637c034000 tx=0x7f637c02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.613+0000 7f6396e93640 1 -- 172.21.15.139:0/2950055663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63900cc2a0 msgr2=0x7f63900b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.613+0000 7f6396e93640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63900cc2a0 0x7f63900b45e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f638400d7d0 tx=0x7f638400dc90 comp rx=0 tx=0).stop 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.613+0000 7f6396e93640 1 -- 172.21.15.139:0/2950055663 shutdown_connections 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.613+0000 7f6396e93640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f637406b4d0 0x7f637406d990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.613+0000 7f6396e93640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f639015db50 0x7f63900cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.613+0000 7f6396e93640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f639015cde0 0x7f63900b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.613+0000 7f6396e93640 1 --2- 172.21.15.139:0/2950055663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63900cc2a0 0x7f63900b45e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.613+0000 7f6396e93640 1 -- 172.21.15.139:0/2950055663 >> 172.21.15.139:0/2950055663 conn(0x7f63900d1790 msgr2=0x7f63900c76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.613+0000 7f6396e93640 1 -- 172.21.15.139:0/2950055663 shutdown_connections 2024-04-25T09:57:32.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:32.613+0000 7f6396e93640 1 -- 172.21.15.139:0/2950055663 wait complete. 2024-04-25T09:57:33.030 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:57:20.444843+0000', 'modified': '2024-04-25T09:57:31.538573+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 77, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6876}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6876': {'gid': 6876, 'name': 'c', 'rank': 0, 'incarnation': 39, 'state': 'up:rejoin', 'state_seq': 6, 'addr': '172.21.15.139:6837/1384380923', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 1384380923}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 1384380923}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:57:33.030 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6876:up:rejoin 2024-04-25T09:57:34.031 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:34.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.137+0000 7fd53661f640 1 Processor -- start 2024-04-25T09:57:34.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.137+0000 7fd53661f640 1 -- start start 2024-04-25T09:57:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.137+0000 7fd53661f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53015ac40 0x7fd53006e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.137+0000 7fd53661f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd53006e850 0x7fd5300b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.137+0000 7fd53661f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5300af6d0 con 0x7fd53015a860 2024-04-25T09:57:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.137+0000 7fd53661f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5300b02e0 con 0x7fd53006e850 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.137+0000 7fd53661f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300af9b0 con 0x7fd53015ac40 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd53015a860 0x7fd5300cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:49472/0 (socket says 172.21.15.139:49472) 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52ffff640 1 -- 172.21.15.139:0/1963329084 learned_addr learned my addr 172.21.15.139:0/1963329084 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd534bbc640 1 --2- 172.21.15.139:0/1963329084 >> v2:172.21.15.150:3300/0 conn(0x7fd53006e850 0x7fd5300b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368554268 0 0) 0x7fd5300af6d0 con 0x7fd53015a860 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52f7fe640 1 --2- 172.21.15.139:0/1963329084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53015ac40 0x7fd53006e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd51c003520 con 0x7fd53015a860 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4067876789 0 0) 0x7fd51c003520 con 0x7fd53015a860 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53015ac40 msgr2=0x7fd53006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 --2- 172.21.15.139:0/1963329084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53015ac40 0x7fd53006e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 >> v2:172.21.15.150:3300/0 conn(0x7fd53006e850 msgr2=0x7fd5300b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 --2- 172.21.15.139:0/1963329084 >> v2:172.21.15.150:3300/0 conn(0x7fd53006e850 0x7fd5300b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5300afc90 con 0x7fd53015a860 2024-04-25T09:57:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fd518001670 con 0x7fd53015a860 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 >> v1:172.21.15.139:6789/0 conn(0x7fd53015a860 legacy=0x7fd5300cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 --2- 172.21.15.139:0/1963329084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd51c0039b0 0x7fd51c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 --2- 172.21.15.139:0/1963329084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53015ac40 0x7fd51c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 --2- 172.21.15.139:0/1963329084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd51c004d40 0x7fd51c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300b02e0 con 0x7fd51c004d40 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300af9b0 con 0x7fd51c0039b0 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c00b9c0 con 0x7fd53015ac40 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd534bbc640 1 --2- 172.21.15.139:0/1963329084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd51c004d40 0x7fd51c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52ffff640 1 --2- 172.21.15.139:0/1963329084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd51c0039b0 0x7fd51c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.138+0000 7fd52f7fe640 1 --2- 172.21.15.139:0/1963329084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53015ac40 0x7fd51c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.139+0000 7fd534bbc640 1 -- 172.21.15.139:0/1963329084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53015ac40 msgr2=0x7fd51c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.139+0000 7fd534bbc640 1 --2- 172.21.15.139:0/1963329084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53015ac40 0x7fd51c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.139+0000 7fd534bbc640 1 -- 172.21.15.139:0/1963329084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd51c0039b0 msgr2=0x7fd51c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.139+0000 7fd534bbc640 1 --2- 172.21.15.139:0/1963329084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd51c0039b0 0x7fd51c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.139+0000 7fd534bbc640 1 -- 172.21.15.139:0/1963329084 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c003520 con 0x7fd51c004d40 2024-04-25T09:57:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.139+0000 7fd534bbc640 1 --2- 172.21.15.139:0/1963329084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd51c004d40 0x7fd51c0051c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fd528009fc0 tx=0x7fd51c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff17f10cfd7e41d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.139+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528009c10 con 0x7fd51c004d40 2024-04-25T09:57:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.139+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd528036070 con 0x7fd51c004d40 2024-04-25T09:57:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.139+0000 7fd52effd640 1 -- 172.21.15.139:0/1963329084 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52802d070 con 0x7fd51c004d40 2024-04-25T09:57:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.140+0000 7fd53661f640 1 -- 172.21.15.139:0/1963329084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd51c004d40 msgr2=0x7fd51c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.140+0000 7fd53661f640 1 --2- 172.21.15.139:0/1963329084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd51c004d40 0x7fd51c0051c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fd528009fc0 tx=0x7fd51c005830 comp rx=0 tx=0).stop 2024-04-25T09:57:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.140+0000 7fd534bbc640 1 -- 172.21.15.139:0/1963329084 reap_dead start 2024-04-25T09:57:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.140+0000 7fd53661f640 1 -- 172.21.15.139:0/1963329084 shutdown_connections 2024-04-25T09:57:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.140+0000 7fd53661f640 1 -- 172.21.15.139:0/1963329084 >> 172.21.15.139:0/1963329084 conn(0x7fd530066f20 msgr2=0x7fd53006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.140+0000 7fd53661f640 1 -- 172.21.15.139:0/1963329084 shutdown_connections 2024-04-25T09:57:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.140+0000 7fd53661f640 1 -- 172.21.15.139:0/1963329084 wait complete. 2024-04-25T09:57:34.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.141+0000 7fd53661f640 1 Processor -- start 2024-04-25T09:57:34.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.141+0000 7fd53661f640 1 -- start start 2024-04-25T09:57:34.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.142+0000 7fd53661f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd53013b670 0x7fd530150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.142+0000 7fd53661f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53014f480 0x7fd530138730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.142+0000 7fd53661f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd53013ba50 0x7fd530138c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.142+0000 7fd53661f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300af2a0 con 0x7fd53013ba50 2024-04-25T09:57:34.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.142+0000 7fd53661f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300cd3c0 con 0x7fd53013b670 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.142+0000 7fd53661f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300af420 con 0x7fd53014f480 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.142+0000 7fd534bbc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd53013ba50 0x7fd530138c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.142+0000 7fd534bbc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd53013ba50 0x7fd530138c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35606/0 (socket says 172.21.15.139:35606) 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.142+0000 7fd534bbc640 1 -- 172.21.15.139:0/1902611152 learned_addr learned my addr 172.21.15.139:0/1902611152 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.142+0000 7fd52ffff640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd53013b670 0x7fd530150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.142+0000 7fd52f7fe640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53014f480 0x7fd530138730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.143+0000 7fd534bbc640 1 -- 172.21.15.139:0/1902611152 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53014f480 msgr2=0x7fd530138730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.143+0000 7fd534bbc640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53014f480 0x7fd530138730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.143+0000 7fd534bbc640 1 -- 172.21.15.139:0/1902611152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd53013b670 msgr2=0x7fd530150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.143+0000 7fd534bbc640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd53013b670 0x7fd530150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.143+0000 7fd534bbc640 1 -- 172.21.15.139:0/1902611152 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53014f980 con 0x7fd53013ba50 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.143+0000 7fd534bbc640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd53013ba50 0x7fd530138c70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fd528033040 tx=0x7fd528009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.143+0000 7fd52cff9640 1 -- 172.21.15.139:0/1902611152 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528009c10 con 0x7fd53013ba50 2024-04-25T09:57:34.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.143+0000 7fd52cff9640 1 -- 172.21.15.139:0/1902611152 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd528003bb0 con 0x7fd53013ba50 2024-04-25T09:57:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.143+0000 7fd53661f640 1 -- 172.21.15.139:0/1902611152 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53014fc60 con 0x7fd53013ba50 2024-04-25T09:57:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.143+0000 7fd52cff9640 1 -- 172.21.15.139:0/1902611152 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528004020 con 0x7fd53013ba50 2024-04-25T09:57:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.143+0000 7fd53661f640 1 -- 172.21.15.139:0/1902611152 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd530150220 con 0x7fd53013ba50 2024-04-25T09:57:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.144+0000 7fd53661f640 1 -- 172.21.15.139:0/1902611152 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd530074520 con 0x7fd53013ba50 2024-04-25T09:57:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.145+0000 7fd52cff9640 1 -- 172.21.15.139:0/1902611152 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd528005d10 con 0x7fd53013ba50 2024-04-25T09:57:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.145+0000 7fd52cff9640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd51006b410 0x7fd51006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.145+0000 7fd52cff9640 1 -- 172.21.15.139:0/1902611152 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd5280ab530 con 0x7fd53013ba50 2024-04-25T09:57:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.145+0000 7fd52ffff640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd51006b410 0x7fd51006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.146+0000 7fd52ffff640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd51006b410 0x7fd51006d8d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd518004840 tx=0x7fd518001c50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.149+0000 7fd52cff9640 1 -- 172.21.15.139:0/1902611152 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd52807a320 con 0x7fd53013ba50 2024-04-25T09:57:34.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.355+0000 7fd53661f640 1 -- 172.21.15.139:0/1902611152 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd530155e50 con 0x7fd53013ba50 2024-04-25T09:57:34.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.356+0000 7fd52cff9640 1 -- 172.21.15.139:0/1902611152 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fd528079c20 con 0x7fd53013ba50 2024-04-25T09:57:34.359 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:34.359 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/982251530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":982251530},{"type":"v1","addr":"172.21.15.150:6834","nonce":982251530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:20.444843+0000","modified":"2024-04-25T09:57:32.540682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":7,"addr":"172.21.15.139:6837/1384380923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":1384380923},{"type":"v1","addr":"172.21.15.139:6837","nonce":1384380923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:57:34.359 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 42 2024-04-25T09:57:34.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.358+0000 7fd53661f640 1 -- 172.21.15.139:0/1902611152 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd51006b410 msgr2=0x7fd51006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.358+0000 7fd53661f640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd51006b410 0x7fd51006d8d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd518004840 tx=0x7fd518001c50 comp rx=0 tx=0).stop 2024-04-25T09:57:34.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.358+0000 7fd53661f640 1 -- 172.21.15.139:0/1902611152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd53013ba50 msgr2=0x7fd530138c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.358+0000 7fd53661f640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd53013ba50 0x7fd530138c70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fd528033040 tx=0x7fd528009750 comp rx=0 tx=0).stop 2024-04-25T09:57:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.358+0000 7fd53661f640 1 -- 172.21.15.139:0/1902611152 shutdown_connections 2024-04-25T09:57:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.358+0000 7fd53661f640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd53013ba50 0x7fd530138c70 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.358+0000 7fd53661f640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd51006b410 0x7fd51006d8d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.358+0000 7fd53661f640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53014f480 0x7fd530138730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.358+0000 7fd53661f640 1 --2- 172.21.15.139:0/1902611152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd53013b670 0x7fd530150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.358+0000 7fd53661f640 1 -- 172.21.15.139:0/1902611152 >> 172.21.15.139:0/1902611152 conn(0x7fd530066f20 msgr2=0x7fd5300b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.358+0000 7fd53661f640 1 -- 172.21.15.139:0/1902611152 shutdown_connections 2024-04-25T09:57:34.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.358+0000 7fd53661f640 1 -- 172.21.15.139:0/1902611152 wait complete. 2024-04-25T09:57:34.775 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:57:20.444843+0000', 'modified': '2024-04-25T09:57:32.540682+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 77, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6876}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6876': {'gid': 6876, 'name': 'c', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.139:6837/1384380923', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 1384380923}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 1384380923}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:57:34.775 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:57:34.775 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:57:34.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.871+0000 7f66539b7640 1 Processor -- start 2024-04-25T09:57:34.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.871+0000 7f66539b7640 1 -- start start 2024-04-25T09:57:34.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f66539b7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f664c15f8c0 0x7f664c09a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f66539b7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c0d03f0 con 0x7f664c15b560 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f66539b7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c099bf0 con 0x7f664c15fca0 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f66539b7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c0d06c0 con 0x7f664c15f8c0 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6651f54640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f664c15b560 0x7f664c15b990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:49474/0 (socket says 172.21.15.139:49474) 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6651f54640 1 -- 172.21.15.139:0/432667506 learned_addr learned my addr 172.21.15.139:0/432667506 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6651753640 1 --2- 172.21.15.139:0/432667506 >> v2:172.21.15.150:3300/0 conn(0x7f664c15f8c0 0x7f664c09a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032873433 0 0) 0x7f664c0d03f0 con 0x7f664c15b560 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6638003520 con 0x7f664c15b560 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3916846570 0 0) 0x7f6638003520 con 0x7f664c15b560 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 >> v2:172.21.15.150:3300/0 conn(0x7f664c15f8c0 msgr2=0x7f664c09a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 --2- 172.21.15.139:0/432667506 >> v2:172.21.15.150:3300/0 conn(0x7f664c15f8c0 0x7f664c09a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 >> v1:172.21.15.150:6789/0 conn(0x7f664c15fca0 legacy=0x7f664c0cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c099ed0 con 0x7f664c15b560 2024-04-25T09:57:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f66440019f0 con 0x7f664c15b560 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 >> v1:172.21.15.139:6789/0 conn(0x7f664c15b560 legacy=0x7f664c15b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 --2- 172.21.15.139:0/432667506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66380039b0 0x7f6638003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 --2- 172.21.15.139:0/432667506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6638004800 0x7f6638004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 --2- 172.21.15.139:0/432667506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66380051c0 0x7f663800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0d06c0 con 0x7f6638004800 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.872+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663800c060 con 0x7f66380051c0 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.873+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663800c260 con 0x7f66380039b0 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.873+0000 7f6650f52640 1 --2- 172.21.15.139:0/432667506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6638004800 0x7f6638004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.873+0000 7f6651753640 1 --2- 172.21.15.139:0/432667506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66380039b0 0x7f6638003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.873+0000 7f6651f54640 1 --2- 172.21.15.139:0/432667506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66380051c0 0x7f663800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.873+0000 7f6650f52640 1 -- 172.21.15.139:0/432667506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66380039b0 msgr2=0x7f6638003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.873+0000 7f6650f52640 1 --2- 172.21.15.139:0/432667506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66380039b0 0x7f6638003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.873+0000 7f6650f52640 1 -- 172.21.15.139:0/432667506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66380051c0 msgr2=0x7f663800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.873+0000 7f6650f52640 1 --2- 172.21.15.139:0/432667506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66380051c0 0x7f663800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.873+0000 7f6650f52640 1 -- 172.21.15.139:0/432667506 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6638003520 con 0x7f6638004800 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.873+0000 7f6651f54640 1 --2- 172.21.15.139:0/432667506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66380051c0 0x7f663800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.873+0000 7f6651f54640 1 -- 172.21.15.139:0/432667506 reap_dead start 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.873+0000 7f6650f52640 1 --2- 172.21.15.139:0/432667506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6638004800 0x7f6638004c80 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f663c001750 tx=0x7f6638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd40797ed2d27ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.874+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c001370 con 0x7f6638004800 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.874+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c0052d0 con 0x7f6638004800 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.874+0000 7f6643fff640 1 -- 172.21.15.139:0/432667506 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c02abc0 con 0x7f6638004800 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.874+0000 7f66539b7640 1 -- 172.21.15.139:0/432667506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6638004800 msgr2=0x7f6638004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.874+0000 7f66539b7640 1 --2- 172.21.15.139:0/432667506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6638004800 0x7f6638004c80 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f663c001750 tx=0x7f6638005830 comp rx=0 tx=0).stop 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.874+0000 7f66539b7640 1 -- 172.21.15.139:0/432667506 shutdown_connections 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.874+0000 7f66539b7640 1 --2- 172.21.15.139:0/432667506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6638004800 0x7f6638004c80 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.874+0000 7f66539b7640 1 -- 172.21.15.139:0/432667506 >> 172.21.15.139:0/432667506 conn(0x7f664c0b31f0 msgr2=0x7f664c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.874+0000 7f66539b7640 1 -- 172.21.15.139:0/432667506 shutdown_connections 2024-04-25T09:57:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.874+0000 7f66539b7640 1 -- 172.21.15.139:0/432667506 wait complete. 2024-04-25T09:57:34.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.875+0000 7f66539b7640 1 Processor -- start 2024-04-25T09:57:34.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.875+0000 7f66539b7640 1 -- start start 2024-04-25T09:57:34.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f66539b7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6638004800 0x7f664c15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f66539b7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f664c06bc70 0x7f664c0b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f66539b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c0b4a20 0x7f664c0ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f66539b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c1602d0 con 0x7f664c0b4a20 2024-04-25T09:57:34.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f66539b7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c09a490 con 0x7f664c06bc70 2024-04-25T09:57:34.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f66539b7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c160450 con 0x7f6638004800 2024-04-25T09:57:34.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f6651f54640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c0b4a20 0x7f664c0ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f6651f54640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c0b4a20 0x7f664c0ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35624/0 (socket says 172.21.15.139:35624) 2024-04-25T09:57:34.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f6651f54640 1 -- 172.21.15.139:0/3117249231 learned_addr learned my addr 172.21.15.139:0/3117249231 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f6651753640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6638004800 0x7f664c15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f6650f52640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f664c06bc70 0x7f664c0b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f6651f54640 1 -- 172.21.15.139:0/3117249231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6638004800 msgr2=0x7f664c15ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f6651f54640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6638004800 0x7f664c15ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f6651f54640 1 -- 172.21.15.139:0/3117249231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f664c06bc70 msgr2=0x7f664c0b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f6651f54640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f664c06bc70 0x7f664c0b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.876+0000 7f6651f54640 1 -- 172.21.15.139:0/3117249231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c0c92e0 con 0x7f664c0b4a20 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.877+0000 7f6651f54640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c0b4a20 0x7f664c0ca0a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f664c072620 tx=0x7f6644007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.877+0000 7f6641ffb640 1 -- 172.21.15.139:0/3117249231 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664400e9c0 con 0x7f664c0b4a20 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.877+0000 7f6641ffb640 1 -- 172.21.15.139:0/3117249231 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6644018070 con 0x7f664c0b4a20 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.877+0000 7f66539b7640 1 -- 172.21.15.139:0/3117249231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c0c95c0 con 0x7f664c0b4a20 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.877+0000 7f6641ffb640 1 -- 172.21.15.139:0/3117249231 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664400ede0 con 0x7f664c0b4a20 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.877+0000 7f66539b7640 1 -- 172.21.15.139:0/3117249231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c15b040 con 0x7f664c0b4a20 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.878+0000 7f6641ffb640 1 -- 172.21.15.139:0/3117249231 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f664401d460 con 0x7f664c0b4a20 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.879+0000 7f6641ffb640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f663006b500 0x7f663006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.879+0000 7f6641ffb640 1 -- 172.21.15.139:0/3117249231 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f664408a620 con 0x7f664c0b4a20 2024-04-25T09:57:34.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.879+0000 7f6651753640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f663006b500 0x7f663006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.879+0000 7f6651753640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f663006b500 0x7f663006d9c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6648009a90 tx=0x7f6648008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.880+0000 7f6656796740 1 Processor -- start 2024-04-25T09:57:34.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.880+0000 7f6656796740 1 -- start start 2024-04-25T09:57:34.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.880+0000 7f6656796740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 0x555e49d27610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.880+0000 7f6656796740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d44ad0 0x555e49d45020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.880+0000 7f6656796740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49cfcf10 0x555e49d43190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.880+0000 7f6656796740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e49c5fa10 con 0x555e49d27230 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.880+0000 7f6656796740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e49c08b30 con 0x555e49d44ad0 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.880+0000 7f6656796740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e49c5bc50 con 0x555e49cfcf10 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.880+0000 7f6651753640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 0x555e49d27610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.880+0000 7f6651753640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 0x555e49d27610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35640/0 (socket says 172.21.15.139:35640) 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.880+0000 7f6651753640 1 -- 172.21.15.139:0/3015780220 learned_addr learned my addr 172.21.15.139:0/3015780220 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.881+0000 7f6650f52640 1 --2- 172.21.15.139:0/3015780220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d44ad0 0x555e49d45020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.881+0000 7f6651f54640 1 --2- 172.21.15.139:0/3015780220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49cfcf10 0x555e49d43190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.881+0000 7f6651753640 1 -- 172.21.15.139:0/3015780220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49cfcf10 msgr2=0x555e49d43190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.881+0000 7f6651753640 1 --2- 172.21.15.139:0/3015780220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49cfcf10 0x555e49d43190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.881+0000 7f6651753640 1 -- 172.21.15.139:0/3015780220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d44ad0 msgr2=0x555e49d45020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.881+0000 7f6651753640 1 --2- 172.21.15.139:0/3015780220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d44ad0 0x555e49d45020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.881+0000 7f6651753640 1 -- 172.21.15.139:0/3015780220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e49c1de10 con 0x555e49d27230 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.881+0000 7f6651753640 1 --2- 172.21.15.139:0/3015780220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 0x555e49d27610 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6648013de0 tx=0x7f6648016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e76b20ccb74105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.881+0000 7f6636ffd640 1 -- 172.21.15.139:0/3015780220 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648023440 con 0x555e49d27230 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.881+0000 7f6636ffd640 1 -- 172.21.15.139:0/3015780220 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6648028070 con 0x555e49d27230 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.881+0000 7f6636ffd640 1 -- 172.21.15.139:0/3015780220 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480237e0 con 0x555e49d27230 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.882+0000 7f6656796740 1 -- 172.21.15.139:0/3015780220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 msgr2=0x555e49d27610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.882+0000 7f6656796740 1 --2- 172.21.15.139:0/3015780220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 0x555e49d27610 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6648013de0 tx=0x7f6648016000 comp rx=0 tx=0).stop 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.882+0000 7f6656796740 1 -- 172.21.15.139:0/3015780220 shutdown_connections 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.882+0000 7f6656796740 1 --2- 172.21.15.139:0/3015780220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49cfcf10 0x555e49d43190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.882+0000 7f6656796740 1 --2- 172.21.15.139:0/3015780220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d44ad0 0x555e49d45020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.882+0000 7f6656796740 1 --2- 172.21.15.139:0/3015780220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 0x555e49d27610 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.882+0000 7f6656796740 1 -- 172.21.15.139:0/3015780220 >> 172.21.15.139:0/3015780220 conn(0x555e49cf9840 msgr2=0x555e49d42040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.882+0000 7f6656796740 1 -- 172.21.15.139:0/3015780220 shutdown_connections 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.882+0000 7f6656796740 1 -- 172.21.15.139:0/3015780220 wait complete. 2024-04-25T09:57:34.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.882+0000 7f6656796740 1 -- start start 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.882+0000 7f6656796740 1 Processor -- start 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.883+0000 7f6656796740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 0x555e49d433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.883+0000 7f6656796740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d44ad0 0x555e49d28850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.883+0000 7f6656796740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d299f0 0x555e49d51f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.883+0000 7f6656796740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e49cfdcb0 con 0x555e49d27230 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.883+0000 7f6656796740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e49c1c0c0 con 0x555e49d299f0 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.883+0000 7f6656796740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e49cf7770 con 0x555e49d44ad0 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.883+0000 7f6656796740 10 client.0 _ll_register_callbacks cb 0x555e49c0fec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.883+0000 7f6651753640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 0x555e49d433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.883+0000 7f6651753640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 0x555e49d433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35646/0 (socket says 172.21.15.139:35646) 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.883+0000 7f6651753640 1 -- 172.21.15.139:0/266269837 learned_addr learned my addr 172.21.15.139:0/266269837 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.883+0000 7f6651f54640 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d299f0 0x555e49d51f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.883+0000 7f6650f52640 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d44ad0 0x555e49d28850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6651753640 1 -- 172.21.15.139:0/266269837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d44ad0 msgr2=0x555e49d28850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6651753640 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d44ad0 0x555e49d28850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6651753640 1 -- 172.21.15.139:0/266269837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d299f0 msgr2=0x555e49d51f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6651753640 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d299f0 0x555e49d51f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6651753640 1 -- 172.21.15.139:0/266269837 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e499742d0 con 0x555e49d27230 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6651753640 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 0x555e49d433c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6648008010 tx=0x7f6648004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92abc4cdcb5eaf8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f662bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f662bfff640 1 -- 172.21.15.139:0/266269837 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648004020 con 0x555e49d27230 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f662bfff640 1 -- 172.21.15.139:0/266269837 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f66480041c0 con 0x555e49d27230 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6656796740 1 -- 172.21.15.139:0/266269837 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6648028070 con 0x555e49d27230 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f662bfff640 1 -- 172.21.15.139:0/266269837 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f664803f050 con 0x555e49d27230 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6656796740 10 client.7134 fetch_fsmap learned FSMap version 42 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6656796740 1 -- 172.21.15.139:0/266269837 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x555e49a3e6b0 con 0x555e49d27230 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f662bfff640 1 -- 172.21.15.139:0/266269837 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6648028070 con 0x555e49d27230 2024-04-25T09:57:34.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f662bfff640 1 -- 172.21.15.139:0/266269837 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6648042020 con 0x555e49d27230 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6656796740 10 client.7134 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6656796740 10 client.7134 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6656796740 20 client.7134 populate_metadata read hostname 'smithi139' 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6656796740 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x555e49d557a0 0x555e49d55ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6656796740 4 client.7134 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6656796740 1 -- 172.21.15.139:0/266269837 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e49cbe920 con 0x555e49d557a0 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.884+0000 7f6650f52640 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x555e49d557a0 0x555e49d55ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f662bfff640 10 client.7134 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f6650f52640 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x555e49d557a0 0x555e49d55ba0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e26f8a06899f1b server_cookie=d0338dd2907d1c66 in_seq=0 out_seq=0 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f662bfff640 1 -- 172.21.15.139:0/266269837 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e49cbe920 con 0x555e49d557a0 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f662bfff640 10 client.7134 handle_command_reply: tid=0 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f6656796740 1 client.7134 shutdown 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f6656796740 1 -- 172.21.15.139:0/266269837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 msgr2=0x555e49d433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f6656796740 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 0x555e49d433c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6648008010 tx=0x7f6648004730 comp rx=0 tx=0).stop 2024-04-25T09:57:34.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f6656796740 1 -- 172.21.15.139:0/266269837 shutdown_connections 2024-04-25T09:57:34.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f6656796740 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x555e49d557a0 0x555e49d55ba0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f6656796740 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d299f0 0x555e49d51f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f6656796740 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d44ad0 0x555e49d28850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f662bfff640 0 client.7134 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:57:34.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f6656796740 1 --2- 172.21.15.139:0/266269837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d27230 0x555e49d433c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.885+0000 7f6656796740 1 -- 172.21.15.139:0/266269837 >> 172.21.15.139:0/266269837 conn(0x555e49cfcf10 msgr2=0x555e49d455c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:34.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.886+0000 7f6656796740 1 -- 172.21.15.139:0/266269837 shutdown_connections 2024-04-25T09:57:34.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.886+0000 7f6656796740 1 -- 172.21.15.139:0/266269837 wait complete. 2024-04-25T09:57:34.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.886+0000 7f6656796740 20 client.7134 trim_cache size 0 max 16384 2024-04-25T09:57:34.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.901+0000 7f6656796740 1 Processor -- start 2024-04-25T09:57:34.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.901+0000 7f6656796740 1 -- start start 2024-04-25T09:57:34.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6656796740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d27230 0x555e49d5bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6656796740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d299f0 0x555e49d452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6656796740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 0x555e49d457e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6656796740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c1ffea0 con 0x555e49d44ad0 2024-04-25T09:57:34.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6656796740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c1f7fb0 con 0x555e49d27230 2024-04-25T09:57:34.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6656796740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c1ff8d0 con 0x555e49d299f0 2024-04-25T09:57:34.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6651f54640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 0x555e49d457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6651f54640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 0x555e49d457e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35658/0 (socket says 172.21.15.139:35658) 2024-04-25T09:57:34.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6651f54640 1 -- 172.21.15.139:0/3750904240 learned_addr learned my addr 172.21.15.139:0/3750904240 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:34.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6651f54640 1 -- 172.21.15.139:0/3750904240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d299f0 msgr2=0x555e49d452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6651f54640 1 --2- 172.21.15.139:0/3750904240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d299f0 0x555e49d452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6651f54640 1 -- 172.21.15.139:0/3750904240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d27230 msgr2=0x555e49d5bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6651f54640 1 --2- 172.21.15.139:0/3750904240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d27230 0x555e49d5bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6651f54640 1 -- 172.21.15.139:0/3750904240 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e49d395d0 con 0x555e49d44ad0 2024-04-25T09:57:34.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f6651f54640 1 --2- 172.21.15.139:0/3750904240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 0x555e49d457e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f6644088970 tx=0x7f6644015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e946e0f4339da3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f66367fc640 1 -- 172.21.15.139:0/3750904240 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664405cd10 con 0x555e49d44ad0 2024-04-25T09:57:34.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f66367fc640 1 -- 172.21.15.139:0/3750904240 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66440612d0 con 0x555e49d44ad0 2024-04-25T09:57:34.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.902+0000 7f66367fc640 1 -- 172.21.15.139:0/3750904240 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644055230 con 0x555e49d44ad0 2024-04-25T09:57:34.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.903+0000 7f6656796740 1 -- 172.21.15.139:0/3750904240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 msgr2=0x555e49d457e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.903+0000 7f6656796740 1 --2- 172.21.15.139:0/3750904240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 0x555e49d457e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f6644088970 tx=0x7f6644015040 comp rx=0 tx=0).stop 2024-04-25T09:57:34.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.903+0000 7f6656796740 1 -- 172.21.15.139:0/3750904240 shutdown_connections 2024-04-25T09:57:34.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.903+0000 7f6656796740 1 --2- 172.21.15.139:0/3750904240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 0x555e49d457e0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.903+0000 7f6656796740 1 --2- 172.21.15.139:0/3750904240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d299f0 0x555e49d452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.903+0000 7f6656796740 1 --2- 172.21.15.139:0/3750904240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d27230 0x555e49d5bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.903+0000 7f6656796740 1 -- 172.21.15.139:0/3750904240 >> 172.21.15.139:0/3750904240 conn(0x555e49cfcf10 msgr2=0x555e49d64a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:34.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.903+0000 7f6656796740 1 -- 172.21.15.139:0/3750904240 shutdown_connections 2024-04-25T09:57:34.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.903+0000 7f6656796740 1 -- 172.21.15.139:0/3750904240 wait complete. 2024-04-25T09:57:34.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.903+0000 7f6656796740 1 -- start start 2024-04-25T09:57:34.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.904+0000 7f6656796740 1 Processor -- start 2024-04-25T09:57:34.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.904+0000 7f6656796740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d27230 0x555e49da7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.904+0000 7f6656796740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d299f0 0x555e49d12eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.904+0000 7f6656796740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 0x555e49d133f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.904+0000 7f6656796740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e49d392b0 con 0x555e49d44ad0 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.904+0000 7f6656796740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e49d28d90 con 0x555e49d27230 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.904+0000 7f6656796740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e49d52670 con 0x555e49d299f0 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.904+0000 7f6656796740 10 client.0 _ll_register_callbacks cb 0x555e49cf7c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6651f54640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 0x555e49d133f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6651f54640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 0x555e49d133f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35666/0 (socket says 172.21.15.139:35666) 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6651f54640 1 -- 172.21.15.139:0/3122089915 learned_addr learned my addr 172.21.15.139:0/3122089915 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6651f54640 1 -- 172.21.15.139:0/3122089915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d299f0 msgr2=0x555e49d12eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6651f54640 1 --2- 172.21.15.139:0/3122089915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d299f0 0x555e49d12eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6651f54640 1 -- 172.21.15.139:0/3122089915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d27230 msgr2=0x555e49da7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6651f54640 1 --2- 172.21.15.139:0/3122089915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d27230 0x555e49da7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6651f54640 1 -- 172.21.15.139:0/3122089915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e49da7950 con 0x555e49d44ad0 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6651f54640 1 --2- 172.21.15.139:0/3122089915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 0x555e49d133f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f6644001f70 tx=0x7f664408a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde13927e35d71a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f662bfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f662bfff640 1 -- 172.21.15.139:0/3122089915 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644060e40 con 0x555e49d44ad0 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f662bfff640 1 -- 172.21.15.139:0/3122089915 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6644060fe0 con 0x555e49d44ad0 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6656796740 1 -- 172.21.15.139:0/3122089915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66440612d0 con 0x555e49d44ad0 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f662bfff640 1 -- 172.21.15.139:0/3122089915 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f664406cb80 con 0x555e49d44ad0 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6656796740 10 client.7140 fetch_fsmap learned FSMap version 42 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6656796740 1 -- 172.21.15.139:0/3122089915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x555e49d43320 con 0x555e49d44ad0 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f662bfff640 1 -- 172.21.15.139:0/3122089915 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66440612d0 con 0x555e49d44ad0 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f662bfff640 1 -- 172.21.15.139:0/3122089915 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6644058010 con 0x555e49d44ad0 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6656796740 10 client.7140 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6656796740 10 client.7140 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6656796740 20 client.7140 populate_metadata read hostname 'smithi139' 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6656796740 1 --2- 172.21.15.139:0/3122089915 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x555e49cf6c40 0x555e49dad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6656796740 4 client.7140 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.905+0000 7f6656796740 1 -- 172.21.15.139:0/3122089915 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555e49c6cf50 con 0x555e49cf6c40 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6651753640 1 --2- 172.21.15.139:0/3122089915 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x555e49cf6c40 0x555e49dad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6651753640 1 --2- 172.21.15.139:0/3122089915 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x555e49cf6c40 0x555e49dad600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec6c62cf4de8417b server_cookie=715df2e5d875a504 in_seq=0 out_seq=0 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f662bfff640 10 client.7140 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f662bfff640 1 -- 172.21.15.139:0/3122089915 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x555e49c6cf50 con 0x555e49cf6c40 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f662bfff640 10 client.7140 handle_command_reply: tid=0 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6656796740 1 client.7140 shutdown 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6656796740 1 -- 172.21.15.139:0/3122089915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 msgr2=0x555e49d133f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6656796740 1 --2- 172.21.15.139:0/3122089915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 0x555e49d133f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f6644001f70 tx=0x7f664408a040 comp rx=0 tx=0).stop 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6656796740 1 -- 172.21.15.139:0/3122089915 shutdown_connections 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6656796740 1 --2- 172.21.15.139:0/3122089915 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x555e49cf6c40 0x555e49dad600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6656796740 1 --2- 172.21.15.139:0/3122089915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e49d44ad0 0x555e49d133f0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6656796740 1 --2- 172.21.15.139:0/3122089915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e49d299f0 0x555e49d12eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f662bfff640 0 client.7140 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6656796740 1 --2- 172.21.15.139:0/3122089915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e49d27230 0x555e49da7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6656796740 1 -- 172.21.15.139:0/3122089915 >> 172.21.15.139:0/3122089915 conn(0x555e49cfcf10 msgr2=0x555e49d64a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6656796740 1 -- 172.21.15.139:0/3122089915 shutdown_connections 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6656796740 1 -- 172.21.15.139:0/3122089915 wait complete. 2024-04-25T09:57:34.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.906+0000 7f6656796740 20 client.7140 trim_cache size 0 max 16384 2024-04-25T09:57:34.918 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:57:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T09:57:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T09:57:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 6876, 2024-04-25T09:57:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T09:57:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T09:57:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T09:57:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 4.3722348430000002, 2024-04-25T09:57:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 42, 2024-04-25T09:57:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 77, 2024-04-25T09:57:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 77, 2024-04-25T09:57:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 18.888517582999999 2024-04-25T09:57:34.919 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:57:34.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.907+0000 7f66539b7640 1 -- 172.21.15.139:0/3117249231 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f663006b500 msgr2=0x7f663006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.907+0000 7f66539b7640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f663006b500 0x7f663006d9c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6648009a90 tx=0x7f6648008040 comp rx=0 tx=0).stop 2024-04-25T09:57:34.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.907+0000 7f66539b7640 1 -- 172.21.15.139:0/3117249231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c0b4a20 msgr2=0x7f664c0ca0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.907+0000 7f66539b7640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c0b4a20 0x7f664c0ca0a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f664c072620 tx=0x7f6644007bc0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.907+0000 7f66539b7640 1 -- 172.21.15.139:0/3117249231 shutdown_connections 2024-04-25T09:57:34.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.907+0000 7f66539b7640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c0b4a20 0x7f664c0ca0a0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.907+0000 7f66539b7640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f663006b500 0x7f663006d9c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.907+0000 7f66539b7640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f664c06bc70 0x7f664c0b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.907+0000 7f66539b7640 1 --2- 172.21.15.139:0/3117249231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6638004800 0x7f664c15ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.907+0000 7f66539b7640 1 -- 172.21.15.139:0/3117249231 >> 172.21.15.139:0/3117249231 conn(0x7f664c0b31f0 msgr2=0x7f664c0c5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:34.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.907+0000 7f66539b7640 1 -- 172.21.15.139:0/3117249231 shutdown_connections 2024-04-25T09:57:34.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:34.907+0000 7f66539b7640 1 -- 172.21.15.139:0/3117249231 wait complete. 2024-04-25T09:57:34.921 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:57:35.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.014+0000 7f871382a640 1 Processor -- start 2024-04-25T09:57:35.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.014+0000 7f871382a640 1 -- start start 2024-04-25T09:57:35.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.015+0000 7f871382a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f870c0cc2a0 0x7f870c09a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.015+0000 7f871382a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f870c0cf360 0x7f870c09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.015+0000 7f871382a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870c15d1c0 con 0x7f870c15cde0 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.015+0000 7f871382a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870c0cc720 con 0x7f870c0cc2a0 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.015+0000 7f871382a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870c0cd460 con 0x7f870c0cf360 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.015+0000 7f8710dc5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f870c0cc2a0 0x7f870c09a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.015+0000 7f8710dc5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f870c0cc2a0 0x7f870c09a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35670/0 (socket says 172.21.15.139:35670) 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.015+0000 7f8710dc5640 1 -- 172.21.15.139:0/2887488503 learned_addr learned my addr 172.21.15.139:0/2887488503 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.016+0000 7f8710dc5640 1 -- 172.21.15.139:0/2887488503 >> v2:172.21.15.150:3300/0 conn(0x7f870c0cf360 msgr2=0x7f870c09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.016+0000 7f8711dc7640 1 --2- 172.21.15.139:0/2887488503 >> v2:172.21.15.150:3300/0 conn(0x7f870c0cf360 0x7f870c09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.016+0000 7f8710dc5640 1 --2- 172.21.15.139:0/2887488503 >> v2:172.21.15.150:3300/0 conn(0x7f870c0cf360 0x7f870c09a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.016+0000 7f8710dc5640 1 -- 172.21.15.139:0/2887488503 >> v1:172.21.15.150:6789/0 conn(0x7f870c15cde0 legacy=0x7f870c15d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.016+0000 7f8710dc5640 1 -- 172.21.15.139:0/2887488503 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c099bf0 con 0x7f870c0cc2a0 2024-04-25T09:57:35.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.016+0000 7f8710dc5640 1 --2- 172.21.15.139:0/2887488503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f870c0cc2a0 0x7f870c09a3f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f870c0c5260 tx=0x7f86fc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c29f57ffcddb8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.016+0000 7f8703fff640 1 -- 172.21.15.139:0/2887488503 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc037070 con 0x7f870c0cc2a0 2024-04-25T09:57:35.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.016+0000 7f8703fff640 1 -- 172.21.15.139:0/2887488503 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc032440 con 0x7f870c0cc2a0 2024-04-25T09:57:35.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.017+0000 7f8703fff640 1 -- 172.21.15.139:0/2887488503 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc032810 con 0x7f870c0cc2a0 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.017+0000 7f871382a640 1 -- 172.21.15.139:0/2887488503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f870c0cc2a0 msgr2=0x7f870c09a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.017+0000 7f871382a640 1 --2- 172.21.15.139:0/2887488503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f870c0cc2a0 0x7f870c09a3f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f870c0c5260 tx=0x7f86fc02a230 comp rx=0 tx=0).stop 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.017+0000 7f871382a640 1 -- 172.21.15.139:0/2887488503 shutdown_connections 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.017+0000 7f871382a640 1 --2- 172.21.15.139:0/2887488503 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f870c0cc2a0 0x7f870c09a3f0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.017+0000 7f871382a640 1 --2- 172.21.15.139:0/2887488503 >> v2:172.21.15.150:3300/0 conn(0x7f870c0cf360 0x7f870c09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.017+0000 7f871382a640 1 -- 172.21.15.139:0/2887488503 >> 172.21.15.139:0/2887488503 conn(0x7f870c0d1790 msgr2=0x7f870c0d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.017+0000 7f871382a640 1 -- 172.21.15.139:0/2887488503 shutdown_connections 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.017+0000 7f871382a640 1 -- 172.21.15.139:0/2887488503 wait complete. 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.018+0000 7f871382a640 1 Processor -- start 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.018+0000 7f871382a640 1 -- start start 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f871382a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f870c0cc2a0 0x7f870c148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f871382a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f870c0cf360 0x7f870c09c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f871382a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f870c15cde0 0x7f870c1e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f871382a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c0cf810 con 0x7f870c0cc2a0 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f871382a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c15dd20 con 0x7f870c0cf360 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f871382a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c0cf990 con 0x7f870c15cde0 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f87115c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f870c0cc2a0 0x7f870c148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f87115c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f870c0cc2a0 0x7f870c148bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35676/0 (socket says 172.21.15.139:35676) 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f87115c6640 1 -- 172.21.15.139:0/3589822030 learned_addr learned my addr 172.21.15.139:0/3589822030 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f8710dc5640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f870c0cf360 0x7f870c09c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f8711dc7640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f870c15cde0 0x7f870c1e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f87115c6640 1 -- 172.21.15.139:0/3589822030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f870c15cde0 msgr2=0x7f870c1e43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f87115c6640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f870c15cde0 0x7f870c1e43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f87115c6640 1 -- 172.21.15.139:0/3589822030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f870c0cf360 msgr2=0x7f870c09c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f87115c6640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f870c0cf360 0x7f870c09c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f87115c6640 1 -- 172.21.15.139:0/3589822030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c149560 con 0x7f870c0cc2a0 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.019+0000 7f8710dc5640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f870c0cf360 0x7f870c09c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:35.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.020+0000 7f87115c6640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f870c0cc2a0 0x7f870c148bf0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f870800ba50 tx=0x7f870800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.020+0000 7f8701ffb640 1 -- 172.21.15.139:0/3589822030 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708019070 con 0x7f870c0cc2a0 2024-04-25T09:57:35.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.020+0000 7f8701ffb640 1 -- 172.21.15.139:0/3589822030 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870800cc20 con 0x7f870c0cc2a0 2024-04-25T09:57:35.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.020+0000 7f8701ffb640 1 -- 172.21.15.139:0/3589822030 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708014680 con 0x7f870c0cc2a0 2024-04-25T09:57:35.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.020+0000 7f871382a640 1 -- 172.21.15.139:0/3589822030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870c149840 con 0x7f870c0cc2a0 2024-04-25T09:57:35.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.020+0000 7f871382a640 1 -- 172.21.15.139:0/3589822030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870c149e00 con 0x7f870c0cc2a0 2024-04-25T09:57:35.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.021+0000 7f871382a640 1 -- 172.21.15.139:0/3589822030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870c074520 con 0x7f870c0cc2a0 2024-04-25T09:57:35.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.021+0000 7f8701ffb640 1 -- 172.21.15.139:0/3589822030 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f870801c070 con 0x7f870c0cc2a0 2024-04-25T09:57:35.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.022+0000 7f8701ffb640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f86f006b4d0 0x7f86f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.022+0000 7f8701ffb640 1 -- 172.21.15.139:0/3589822030 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f870808a020 con 0x7f870c0cc2a0 2024-04-25T09:57:35.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.022+0000 7f8710dc5640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f86f006b4d0 0x7f86f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.022+0000 7f8710dc5640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f86f006b4d0 0x7f86f006d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f86fc034000 tx=0x7f86fc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.026+0000 7f8701ffb640 1 -- 172.21.15.139:0/3589822030 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8708058700 con 0x7f870c0cc2a0 2024-04-25T09:57:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.233+0000 7f871382a640 1 -- 172.21.15.139:0/3589822030 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f870c071270 con 0x7f870c0cc2a0 2024-04-25T09:57:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.233+0000 7f8701ffb640 1 -- 172.21.15.139:0/3589822030 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7f870805c7a0 con 0x7f870c0cc2a0 2024-04-25T09:57:35.237 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:57:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.235+0000 7f871382a640 1 -- 172.21.15.139:0/3589822030 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f86f006b4d0 msgr2=0x7f86f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.235+0000 7f871382a640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f86f006b4d0 0x7f86f006d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f86fc034000 tx=0x7f86fc02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.235+0000 7f871382a640 1 -- 172.21.15.139:0/3589822030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f870c0cc2a0 msgr2=0x7f870c148bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.235+0000 7f871382a640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f870c0cc2a0 0x7f870c148bf0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f870800ba50 tx=0x7f870800bf10 comp rx=0 tx=0).stop 2024-04-25T09:57:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.235+0000 7f871382a640 1 -- 172.21.15.139:0/3589822030 shutdown_connections 2024-04-25T09:57:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.235+0000 7f871382a640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f86f006b4d0 0x7f86f006d990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.236+0000 7f871382a640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f870c15cde0 0x7f870c1e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.236+0000 7f871382a640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f870c0cf360 0x7f870c09c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.236+0000 7f871382a640 1 --2- 172.21.15.139:0/3589822030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f870c0cc2a0 0x7f870c148bf0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.236+0000 7f871382a640 1 -- 172.21.15.139:0/3589822030 >> 172.21.15.139:0/3589822030 conn(0x7f870c0d1790 msgr2=0x7f870c15d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.236+0000 7f871382a640 1 -- 172.21.15.139:0/3589822030 shutdown_connections 2024-04-25T09:57:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:35.236+0000 7f871382a640 1 -- 172.21.15.139:0/3589822030 wait complete. 2024-04-25T09:57:35.249 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:57:35.250 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:57:35.250 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T09:57:35.250 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:57:35.250 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:35.250 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:57:35.250 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:57:35.250 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:57:35.250 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:57:35.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:35.250 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T09:57:35.265 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:57:35.265 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:57:35.266 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:57:35.266 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:35.266 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:57:35.266 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:35.266 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:57:35.266 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:35.266 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T09:57:35.266 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:57:35.266 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:35.266 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T09:57:35.266 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42443sec preferred_lft 42443sec 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:57:35.267 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:57:35.268 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T09:57:35.268 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:35.268 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:57:35.268 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:57:35.268 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T09:57:35.268 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:35.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:35.268 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:57:35.328 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:57:35.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:35.329 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T09:57:35.385 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T09:57:35.385 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:57:35.385 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:57:35.385 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:57:35.385 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:57:35.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:35.385 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:57:35.385 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:57:35.386 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:57:35.386 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:57:35.386 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:57:35.386 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:57:35.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:35.468 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:57:35.524 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:35.524 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T09:57:35.588 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:57:35.588 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:35.643 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:57:35.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:57:35.643 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:35.697 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:57:35.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:35.697 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:57:35.777 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:57:35.778 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:57:35.785 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:57:35.785 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:57:35.785 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T09:57:35.785 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T09:57:35.785 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T09:57:35.785 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:57:35.786 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:57:35.786 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:57:35.786 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:57:35.786 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:57:35.787 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:57:35.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:35.787 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:35.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:35.852 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:57:35.908 INFO:teuthology.orchestra.run.smithi139.stdout:client7149 2024-04-25T09:57:35.908 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:57:35.909 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:35.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:35.963 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-25T09:57:36.020 INFO:teuthology.orchestra.run.smithi139.stdout:file_undamaged 2024-04-25T09:57:36.020 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> import json 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> import sys 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> try: 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> except OSError as e: 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> sys.exit(e.errno) 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> print(json.dumps( 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> indent=2)) 2024-04-25T09:57:36.021 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T09:57:36.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T09:57:36.135 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T09:57:36.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:36.136 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T09:57:36.192 INFO:teuthology.orchestra.run.smithi139.stdout:2 2024-04-25T09:57:36.192 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:57:36.192 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:36.247 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:57:36.247 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:36.302 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:36.355 INFO:tasks.ceph.mds.c.smithi139.stderr:2024-04-25T09:57:36.352+0000 7fe979902640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:57:36.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.436+0000 7f3eed18b640 1 Processor -- start 2024-04-25T09:57:36.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.436+0000 7f3eed18b640 1 -- start start 2024-04-25T09:57:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.437+0000 7f3eed18b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ee815a860 0x7f3ee80cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.437+0000 7f3eed18b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ee806ea20 0x7f3ee80b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.437+0000 7f3eed18b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee80af6d0 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.437+0000 7f3eed18b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee80b02e0 con 0x7f3ee806ea20 2024-04-25T09:57:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.437+0000 7f3eed18b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80af9b0 con 0x7f3ee815a860 2024-04-25T09:57:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.438+0000 7f3ee6d9d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ee815a860 0x7f3ee80cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.438+0000 7f3ee759e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ee806ea20 0x7f3ee80b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.438+0000 7f3ee659c640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f3ee80cd1f0 0x7f3ee806e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:57472/0 (socket says 172.21.15.139:57472) 2024-04-25T09:57:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.438+0000 7f3ee759e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ee806ea20 0x7f3ee80b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:52274/0 (socket says 172.21.15.139:52274) 2024-04-25T09:57:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.438+0000 7f3ee6d9d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ee815a860 0x7f3ee80cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:35488/0 (socket says 172.21.15.139:35488) 2024-04-25T09:57:36.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.438+0000 7f3ee659c640 1 -- 172.21.15.139:0/3106615286 learned_addr learned my addr 172.21.15.139:0/3106615286 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.438+0000 7f3ee6d9d640 1 -- 172.21.15.139:0/3106615286 >> v2:172.21.15.150:3300/0 conn(0x7f3ee806ea20 msgr2=0x7f3ee80b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.438+0000 7f3ee6d9d640 1 --2- 172.21.15.139:0/3106615286 >> v2:172.21.15.150:3300/0 conn(0x7f3ee806ea20 0x7f3ee80b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.438+0000 7f3ee6d9d640 1 -- 172.21.15.139:0/3106615286 >> v1:172.21.15.150:6789/0 conn(0x7f3ee80cd1f0 legacy=0x7f3ee806e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.438+0000 7f3ee6d9d640 1 -- 172.21.15.139:0/3106615286 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80afc90 con 0x7f3ee815a860 2024-04-25T09:57:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.438+0000 7f3ee759e640 1 --2- 172.21.15.139:0/3106615286 >> v2:172.21.15.150:3300/0 conn(0x7f3ee806ea20 0x7f3ee80b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:36.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.439+0000 7f3ee6d9d640 1 --2- 172.21.15.139:0/3106615286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ee815a860 0x7f3ee80cccb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80097a0 tx=0x7f3ed802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3d32a5b09b994f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.439+0000 7f3ee5d9b640 1 -- 172.21.15.139:0/3106615286 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8037070 con 0x7f3ee815a860 2024-04-25T09:57:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.439+0000 7f3ee5d9b640 1 -- 172.21.15.139:0/3106615286 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed8032440 con 0x7f3ee815a860 2024-04-25T09:57:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.439+0000 7f3ee5d9b640 1 -- 172.21.15.139:0/3106615286 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8032810 con 0x7f3ee815a860 2024-04-25T09:57:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.439+0000 7f3eed18b640 1 -- 172.21.15.139:0/3106615286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ee815a860 msgr2=0x7f3ee80cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.439+0000 7f3eed18b640 1 --2- 172.21.15.139:0/3106615286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ee815a860 0x7f3ee80cccb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80097a0 tx=0x7f3ed802a180 comp rx=0 tx=0).stop 2024-04-25T09:57:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.439+0000 7f3eed18b640 1 -- 172.21.15.139:0/3106615286 shutdown_connections 2024-04-25T09:57:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.439+0000 7f3eed18b640 1 --2- 172.21.15.139:0/3106615286 >> v2:172.21.15.150:3300/0 conn(0x7f3ee806ea20 0x7f3ee80b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.439+0000 7f3eed18b640 1 --2- 172.21.15.139:0/3106615286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ee815a860 0x7f3ee80cccb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.439+0000 7f3eed18b640 1 -- 172.21.15.139:0/3106615286 >> 172.21.15.139:0/3106615286 conn(0x7f3ee8066f20 msgr2=0x7f3ee806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.440+0000 7f3eed18b640 1 -- 172.21.15.139:0/3106615286 shutdown_connections 2024-04-25T09:57:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.440+0000 7f3eed18b640 1 -- 172.21.15.139:0/3106615286 wait complete. 2024-04-25T09:57:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.441+0000 7f3eed18b640 1 Processor -- start 2024-04-25T09:57:36.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.441+0000 7f3eed18b640 1 -- start start 2024-04-25T09:57:36.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3eed18b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee806ea20 0x7f3ee80ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3eed18b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ee80cd1f0 0x7f3ee81f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3eed18b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ee815a860 0x7f3ee8149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3eed18b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80af420 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3eed18b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80af2a0 con 0x7f3ee806ea20 2024-04-25T09:57:36.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3eed18b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee815ac40 con 0x7f3ee815a860 2024-04-25T09:57:36.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3ee659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ee80cd1f0 0x7f3ee81f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3ee659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ee80cd1f0 0x7f3ee81f47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38870/0 (socket says 172.21.15.139:38870) 2024-04-25T09:57:36.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3ee659c640 1 -- 172.21.15.139:0/1771261784 learned_addr learned my addr 172.21.15.139:0/1771261784 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:36.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3ee6d9d640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee806ea20 0x7f3ee80ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3ee759e640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ee815a860 0x7f3ee8149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3ee659c640 1 -- 172.21.15.139:0/1771261784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ee815a860 msgr2=0x7f3ee8149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3ee659c640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ee815a860 0x7f3ee8149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3ee659c640 1 -- 172.21.15.139:0/1771261784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee806ea20 msgr2=0x7f3ee80ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3ee659c640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee806ea20 0x7f3ee80ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3ee659c640 1 -- 172.21.15.139:0/1771261784 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee81f06d0 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.442+0000 7f3ee659c640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ee80cd1f0 0x7f3ee81f47e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000c400 tx=0x7f3ed000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.443+0000 7f3edf7fe640 1 -- 172.21.15.139:0/1771261784 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0017070 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.443+0000 7f3edf7fe640 1 -- 172.21.15.139:0/1771261784 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed0005ce0 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.443+0000 7f3edf7fe640 1 -- 172.21.15.139:0/1771261784 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed00072f0 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.443+0000 7f3eed18b640 1 -- 172.21.15.139:0/1771261784 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee80ad970 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.443+0000 7f3eed18b640 1 -- 172.21.15.139:0/1771261784 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee80adf30 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.444+0000 7f3eed18b640 1 -- 172.21.15.139:0/1771261784 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee8073780 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.444+0000 7f3edf7fe640 1 -- 172.21.15.139:0/1771261784 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ed001a070 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.445+0000 7f3edf7fe640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3ec806b4d0 0x7f3ec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.445+0000 7f3edf7fe640 1 -- 172.21.15.139:0/1771261784 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3ed0089770 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.445+0000 7f3ee6d9d640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3ec806b4d0 0x7f3ec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.445+0000 7f3ee6d9d640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3ec806b4d0 0x7f3ec806d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8034000 tx=0x7f3ed802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.449+0000 7f3edf7fe640 1 -- 172.21.15.139:0/1771261784 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ee8073780 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.620+0000 7f3eed18b640 1 -- 172.21.15.139:0/1771261784 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ee8074520 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.621+0000 7f3edf7fe640 1 -- 172.21.15.139:0/1771261784 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+17124 (secure 0 0 0) 0x7f3ed0057dc0 con 0x7f3ee80cd1f0 2024-04-25T09:57:36.624 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:57:36.625 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":77,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:57:29.527441+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:18.681554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:19.409382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6834/1932986598":"2024-04-26T09:57:29.527412+0000","172.21.15.150:6832/1932986598":"2024-04-26T09:57:29.527412+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:36.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.623+0000 7f3eed18b640 1 -- 172.21.15.139:0/1771261784 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3ec806b4d0 msgr2=0x7f3ec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.623+0000 7f3eed18b640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3ec806b4d0 0x7f3ec806d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8034000 tx=0x7f3ed802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:36.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.623+0000 7f3eed18b640 1 -- 172.21.15.139:0/1771261784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ee80cd1f0 msgr2=0x7f3ee81f47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.623+0000 7f3eed18b640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ee80cd1f0 0x7f3ee81f47e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000c400 tx=0x7f3ed000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.623+0000 7f3eed18b640 1 -- 172.21.15.139:0/1771261784 shutdown_connections 2024-04-25T09:57:36.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.623+0000 7f3eed18b640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3ee80cd1f0 0x7f3ee81f47e0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.623+0000 7f3eed18b640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3ec806b4d0 0x7f3ec806d990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.623+0000 7f3eed18b640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ee815a860 0x7f3ee8149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.623+0000 7f3eed18b640 1 --2- 172.21.15.139:0/1771261784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee806ea20 0x7f3ee80ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.623+0000 7f3eed18b640 1 -- 172.21.15.139:0/1771261784 >> 172.21.15.139:0/1771261784 conn(0x7f3ee8066f20 msgr2=0x7f3ee8159660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.623+0000 7f3eed18b640 1 -- 172.21.15.139:0/1771261784 shutdown_connections 2024-04-25T09:57:36.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.623+0000 7f3eed18b640 1 -- 172.21.15.139:0/1771261784 wait complete. 2024-04-25T09:57:36.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:57:18.681554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:57:19.409382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:36.638 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:36.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.773+0000 7f4f661b0640 1 Processor -- start 2024-04-25T09:57:36.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.773+0000 7f4f661b0640 1 -- start start 2024-04-25T09:57:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f661b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f60144090 0x7f4f600b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f661b0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4f6006ea40 0x7f4f6015e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f661b0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f600b2a50 con 0x7f4f600cc9e0 2024-04-25T09:57:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f661b0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6015b090 con 0x7f4f6006ea40 2024-04-25T09:57:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f661b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6015b370 con 0x7f4f60144090 2024-04-25T09:57:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f5ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4f6006ea40 0x7f4f6015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f5effd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4f600cc9e0 0x7f4f6006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:36210/0 (socket says 172.21.15.139:36210) 2024-04-25T09:57:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f5effd640 1 -- 172.21.15.139:0/4209575852 learned_addr learned my addr 172.21.15.139:0/4209575852 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f5f7fe640 1 --2- 172.21.15.139:0/4209575852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f60144090 0x7f4f600b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f5ffff640 1 -- 172.21.15.139:0/4209575852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f60144090 msgr2=0x7f4f600b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f5ffff640 1 --2- 172.21.15.139:0/4209575852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f60144090 0x7f4f600b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f5ffff640 1 -- 172.21.15.139:0/4209575852 >> v1:172.21.15.139:6789/0 conn(0x7f4f600cc9e0 legacy=0x7f4f6006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f5ffff640 1 -- 172.21.15.139:0/4209575852 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6015b650 con 0x7f4f6006ea40 2024-04-25T09:57:36.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.774+0000 7f4f5e7fc640 1 -- 172.21.15.139:0/4209575852 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673447609 0 0) 0x7f4f54001110 con 0x7f4f600cc9e0 2024-04-25T09:57:36.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.775+0000 7f4f5ffff640 1 --2- 172.21.15.139:0/4209575852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f6006ea40 0x7f4f6015e610 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009d90 tx=0x7f4f5802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e0fe975d53ea26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.775+0000 7f4f5e7fc640 1 -- 172.21.15.139:0/4209575852 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58031600 con 0x7f4f6006ea40 2024-04-25T09:57:36.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.775+0000 7f4f5e7fc640 1 -- 172.21.15.139:0/4209575852 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f580317a0 con 0x7f4f6006ea40 2024-04-25T09:57:36.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.775+0000 7f4f5e7fc640 1 -- 172.21.15.139:0/4209575852 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58031b70 con 0x7f4f6006ea40 2024-04-25T09:57:36.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.776+0000 7f4f661b0640 1 -- 172.21.15.139:0/4209575852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f6006ea40 msgr2=0x7f4f6015e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.776+0000 7f4f661b0640 1 --2- 172.21.15.139:0/4209575852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f6006ea40 0x7f4f6015e610 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009d90 tx=0x7f4f5802a5d0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.776+0000 7f4f661b0640 1 -- 172.21.15.139:0/4209575852 shutdown_connections 2024-04-25T09:57:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.776+0000 7f4f661b0640 1 --2- 172.21.15.139:0/4209575852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f6006ea40 0x7f4f6015e610 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.776+0000 7f4f661b0640 1 --2- 172.21.15.139:0/4209575852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f60144090 0x7f4f600b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.776+0000 7f4f661b0640 1 -- 172.21.15.139:0/4209575852 >> 172.21.15.139:0/4209575852 conn(0x7f4f60067000 msgr2=0x7f4f6006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.776+0000 7f4f661b0640 1 -- 172.21.15.139:0/4209575852 shutdown_connections 2024-04-25T09:57:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.776+0000 7f4f661b0640 1 -- 172.21.15.139:0/4209575852 wait complete. 2024-04-25T09:57:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.777+0000 7f4f661b0640 1 Processor -- start 2024-04-25T09:57:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.777+0000 7f4f661b0640 1 -- start start 2024-04-25T09:57:36.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f661b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f6006ea40 0x7f4f601f0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f661b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f600cc9e0 0x7f4f6013ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f661b0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f60144090 0x7f4f600a9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f661b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f60144470 con 0x7f4f600cc9e0 2024-04-25T09:57:36.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f661b0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6015bc10 con 0x7f4f60144090 2024-04-25T09:57:36.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f661b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6015ad80 con 0x7f4f6006ea40 2024-04-25T09:57:36.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f5effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f600cc9e0 0x7f4f6013ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f5effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f600cc9e0 0x7f4f6013ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38886/0 (socket says 172.21.15.139:38886) 2024-04-25T09:57:36.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f5effd640 1 -- 172.21.15.139:0/4149948204 learned_addr learned my addr 172.21.15.139:0/4149948204 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:36.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f5ffff640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f60144090 0x7f4f600a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f5f7fe640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f6006ea40 0x7f4f601f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f5effd640 1 -- 172.21.15.139:0/4149948204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f6006ea40 msgr2=0x7f4f601f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f5effd640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f6006ea40 0x7f4f601f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f5effd640 1 -- 172.21.15.139:0/4149948204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f60144090 msgr2=0x7f4f600a9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f5effd640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f60144090 0x7f4f600a9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.778+0000 7f4f5effd640 1 -- 172.21.15.139:0/4149948204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f600a9d40 con 0x7f4f600cc9e0 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.779+0000 7f4f5effd640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f600cc9e0 0x7f4f6013ebd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400c430 tx=0x7f4f5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.779+0000 7f4f47fff640 1 -- 172.21.15.139:0/4149948204 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f540074e0 con 0x7f4f600cc9e0 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.779+0000 7f4f47fff640 1 -- 172.21.15.139:0/4149948204 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f540047e0 con 0x7f4f600cc9e0 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.779+0000 7f4f661b0640 1 -- 172.21.15.139:0/4149948204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f600aa2c0 con 0x7f4f600cc9e0 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.779+0000 7f4f47fff640 1 -- 172.21.15.139:0/4149948204 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54004bd0 con 0x7f4f600cc9e0 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.779+0000 7f4f661b0640 1 -- 172.21.15.139:0/4149948204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f600aa880 con 0x7f4f600cc9e0 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.780+0000 7f4f661b0640 1 -- 172.21.15.139:0/4149948204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f60074520 con 0x7f4f600cc9e0 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.780+0000 7f4f47fff640 1 -- 172.21.15.139:0/4149948204 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f54011070 con 0x7f4f600cc9e0 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.781+0000 7f4f47fff640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4f4c06b4d0 0x7f4f4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.781+0000 7f4f47fff640 1 -- 172.21.15.139:0/4149948204 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4f5408a0a0 con 0x7f4f600cc9e0 2024-04-25T09:57:36.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.781+0000 7f4f5f7fe640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4f4c06b4d0 0x7f4f4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.781+0000 7f4f5f7fe640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4f4c06b4d0 0x7f4f4c06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4f600b1ed0 tx=0x7f4f50008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.785+0000 7f4f47fff640 1 -- 172.21.15.139:0/4149948204 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f600b2a50 con 0x7f4f600cc9e0 2024-04-25T09:57:36.818 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T09:57:36.815+0000 7f73a902d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T09:57:36.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.953+0000 7f4f661b0640 1 -- 172.21.15.139:0/4149948204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f6006c1a0 con 0x7f4f600cc9e0 2024-04-25T09:57:36.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.953+0000 7f4f47fff640 1 -- 172.21.15.139:0/4149948204 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f54058d40 con 0x7f4f600cc9e0 2024-04-25T09:57:36.957 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:57:36.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.955+0000 7f4f661b0640 1 -- 172.21.15.139:0/4149948204 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4f4c06b4d0 msgr2=0x7f4f4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.955+0000 7f4f661b0640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4f4c06b4d0 0x7f4f4c06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4f600b1ed0 tx=0x7f4f50008040 comp rx=0 tx=0).stop 2024-04-25T09:57:36.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.955+0000 7f4f661b0640 1 -- 172.21.15.139:0/4149948204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f600cc9e0 msgr2=0x7f4f6013ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.955+0000 7f4f661b0640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f600cc9e0 0x7f4f6013ebd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400c430 tx=0x7f4f5400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.955+0000 7f4f661b0640 1 -- 172.21.15.139:0/4149948204 shutdown_connections 2024-04-25T09:57:36.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.955+0000 7f4f661b0640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f600cc9e0 0x7f4f6013ebd0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.955+0000 7f4f661b0640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4f4c06b4d0 0x7f4f4c06d990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.955+0000 7f4f661b0640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f60144090 0x7f4f600a9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.955+0000 7f4f661b0640 1 --2- 172.21.15.139:0/4149948204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f6006ea40 0x7f4f601f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.955+0000 7f4f661b0640 1 -- 172.21.15.139:0/4149948204 >> 172.21.15.139:0/4149948204 conn(0x7f4f60067000 msgr2=0x7f4f6015d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.956+0000 7f4f661b0640 1 -- 172.21.15.139:0/4149948204 shutdown_connections 2024-04-25T09:57:36.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:36.956+0000 7f4f661b0640 1 -- 172.21.15.139:0/4149948204 wait complete. 2024-04-25T09:57:36.970 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.067+0000 7f26e63b7640 1 Processor -- start 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.067+0000 7f26e63b7640 1 -- start start 2024-04-25T09:57:37.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26e63b7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f26e015fca0 0x7f26e00cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26e63b7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e0099bf0 con 0x7f26e015f8c0 2024-04-25T09:57:37.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26e63b7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e0099ed0 con 0x7f26e00d0220 2024-04-25T09:57:37.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26e63b7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e00d06c0 con 0x7f26e015fca0 2024-04-25T09:57:37.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26dffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f26e015f8c0 0x7f26e009a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:36226/0 (socket says 172.21.15.139:36226) 2024-04-25T09:57:37.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26dffff640 1 -- 172.21.15.139:0/3389571789 learned_addr learned my addr 172.21.15.139:0/3389571789 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:37.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26df7fe640 1 --2- 172.21.15.139:0/3389571789 >> v2:172.21.15.150:3300/0 conn(0x7f26e015fca0 0x7f26e00cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741651279 0 0) 0x7f26e0099bf0 con 0x7f26e015f8c0 2024-04-25T09:57:37.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26cc003520 con 0x7f26e015f8c0 2024-04-25T09:57:37.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1543508716 0 0) 0x7f26cc003520 con 0x7f26e015f8c0 2024-04-25T09:57:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 >> v2:172.21.15.150:3300/0 conn(0x7f26e015fca0 msgr2=0x7f26e00cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 --2- 172.21.15.139:0/3389571789 >> v2:172.21.15.150:3300/0 conn(0x7f26e015fca0 0x7f26e00cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 >> v1:172.21.15.150:6789/0 conn(0x7f26e00d0220 legacy=0x7f26e015b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e009a1b0 con 0x7f26e015f8c0 2024-04-25T09:57:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f26d0001670 con 0x7f26e015f8c0 2024-04-25T09:57:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 >> v1:172.21.15.139:6789/0 conn(0x7f26e015f8c0 legacy=0x7f26e009a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 --2- 172.21.15.139:0/3389571789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26cc0039b0 0x7f26cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 --2- 172.21.15.139:0/3389571789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26cc004800 0x7f26cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 --2- 172.21.15.139:0/3389571789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26cc0051c0 0x7f26cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e00d06c0 con 0x7f26cc004800 2024-04-25T09:57:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc00c060 con 0x7f26cc0051c0 2024-04-25T09:57:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc00c260 con 0x7f26cc0039b0 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.068+0000 7f26df7fe640 1 --2- 172.21.15.139:0/3389571789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26cc004800 0x7f26cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.069+0000 7f26dffff640 1 --2- 172.21.15.139:0/3389571789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26cc0039b0 0x7f26cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.069+0000 7f26e4954640 1 --2- 172.21.15.139:0/3389571789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26cc0051c0 0x7f26cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.069+0000 7f26df7fe640 1 -- 172.21.15.139:0/3389571789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26cc0039b0 msgr2=0x7f26cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.069+0000 7f26df7fe640 1 --2- 172.21.15.139:0/3389571789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26cc0039b0 0x7f26cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.069+0000 7f26df7fe640 1 -- 172.21.15.139:0/3389571789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26cc0051c0 msgr2=0x7f26cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.069+0000 7f26df7fe640 1 --2- 172.21.15.139:0/3389571789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26cc0051c0 0x7f26cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.069+0000 7f26e4954640 1 -- 172.21.15.139:0/3389571789 reap_dead start 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.069+0000 7f26df7fe640 1 -- 172.21.15.139:0/3389571789 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc003520 con 0x7f26cc004800 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.069+0000 7f26df7fe640 1 --2- 172.21.15.139:0/3389571789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26cc004800 0x7f26cc004c80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009ac0 tx=0x7f26cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbfdaae065d02ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.070+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8036070 con 0x7f26cc004800 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.070+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c8031400 con 0x7f26cc004800 2024-04-25T09:57:37.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.070+0000 7f26deffd640 1 -- 172.21.15.139:0/3389571789 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8009690 con 0x7f26cc004800 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.070+0000 7f26e63b7640 1 -- 172.21.15.139:0/3389571789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26cc004800 msgr2=0x7f26cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.070+0000 7f26e63b7640 1 --2- 172.21.15.139:0/3389571789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26cc004800 0x7f26cc004c80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009ac0 tx=0x7f26cc005830 comp rx=0 tx=0).stop 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.070+0000 7f26e63b7640 1 -- 172.21.15.139:0/3389571789 shutdown_connections 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.070+0000 7f26e63b7640 1 --2- 172.21.15.139:0/3389571789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26cc004800 0x7f26cc004c80 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.070+0000 7f26e63b7640 1 -- 172.21.15.139:0/3389571789 >> 172.21.15.139:0/3389571789 conn(0x7f26e00b31f0 msgr2=0x7f26e00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.070+0000 7f26e63b7640 1 -- 172.21.15.139:0/3389571789 shutdown_connections 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.070+0000 7f26e63b7640 1 -- 172.21.15.139:0/3389571789 wait complete. 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.071+0000 7f26e63b7640 1 Processor -- start 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.071+0000 7f26e63b7640 1 -- start start 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.072+0000 7f26e63b7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26cc004800 0x7f26e00b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.072+0000 7f26e63b7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e01e5d80 0x7f26e01f1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.072+0000 7f26e63b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e01f1760 0x7f26e00b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.072+0000 7f26e63b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e0160450 con 0x7f26e01f1760 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.072+0000 7f26e63b7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e01602d0 con 0x7f26cc004800 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.072+0000 7f26e63b7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e00cfce0 con 0x7f26e01e5d80 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.072+0000 7f26e4954640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e01f1760 0x7f26e00b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.072+0000 7f26e4954640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e01f1760 0x7f26e00b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38904/0 (socket says 172.21.15.139:38904) 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.072+0000 7f26e4954640 1 -- 172.21.15.139:0/995840181 learned_addr learned my addr 172.21.15.139:0/995840181 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.072+0000 7f26dffff640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26cc004800 0x7f26e00b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.072+0000 7f26df7fe640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e01e5d80 0x7f26e01f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.073+0000 7f26e4954640 1 -- 172.21.15.139:0/995840181 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e01e5d80 msgr2=0x7f26e01f1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.073+0000 7f26e4954640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e01e5d80 0x7f26e01f1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.073+0000 7f26e4954640 1 -- 172.21.15.139:0/995840181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26cc004800 msgr2=0x7f26e00b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.073+0000 7f26e4954640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26cc004800 0x7f26e00b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.073+0000 7f26e4954640 1 -- 172.21.15.139:0/995840181 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e01e6200 con 0x7f26e01f1760 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.073+0000 7f26e4954640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e01f1760 0x7f26e00b6090 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f26e00725c0 tx=0x7f26d800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.073+0000 7f26dcff9640 1 -- 172.21.15.139:0/995840181 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8019070 con 0x7f26e01f1760 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.073+0000 7f26dcff9640 1 -- 172.21.15.139:0/995840181 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d8006d60 con 0x7f26e01f1760 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.073+0000 7f26e63b7640 1 -- 172.21.15.139:0/995840181 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e01e6480 con 0x7f26e01f1760 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.073+0000 7f26dcff9640 1 -- 172.21.15.139:0/995840181 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8003b90 con 0x7f26e01f1760 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.073+0000 7f26e63b7640 1 -- 172.21.15.139:0/995840181 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e01e6a40 con 0x7f26e01f1760 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.074+0000 7f26e63b7640 1 -- 172.21.15.139:0/995840181 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e0074520 con 0x7f26e01f1760 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.074+0000 7f26dcff9640 1 -- 172.21.15.139:0/995840181 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26d801c070 con 0x7f26e01f1760 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.075+0000 7f26dcff9640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26c006b4d0 0x7f26c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.075+0000 7f26dffff640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26c006b4d0 0x7f26c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.075+0000 7f26dcff9640 1 -- 172.21.15.139:0/995840181 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f26d80897a0 con 0x7f26e01f1760 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.075+0000 7f26dffff640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26c006b4d0 0x7f26c006d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f26d0001eb0 tx=0x7f26d0001c30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.079+0000 7f26dcff9640 1 -- 172.21.15.139:0/995840181 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26e01e6200 con 0x7f26e01f1760 2024-04-25T09:57:37.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.253+0000 7f26e63b7640 1 -- 172.21.15.139:0/995840181 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26e00cdc30 con 0x7f26e01f1760 2024-04-25T09:57:37.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.253+0000 7f26dcff9640 1 -- 172.21.15.139:0/995840181 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26d8057e80 con 0x7f26e01f1760 2024-04-25T09:57:37.257 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:57:37.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.255+0000 7f26e63b7640 1 -- 172.21.15.139:0/995840181 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26c006b4d0 msgr2=0x7f26c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.255+0000 7f26e63b7640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26c006b4d0 0x7f26c006d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f26d0001eb0 tx=0x7f26d0001c30 comp rx=0 tx=0).stop 2024-04-25T09:57:37.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.255+0000 7f26e63b7640 1 -- 172.21.15.139:0/995840181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e01f1760 msgr2=0x7f26e00b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.255+0000 7f26e63b7640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e01f1760 0x7f26e00b6090 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f26e00725c0 tx=0x7f26d800e960 comp rx=0 tx=0).stop 2024-04-25T09:57:37.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.255+0000 7f26e63b7640 1 -- 172.21.15.139:0/995840181 shutdown_connections 2024-04-25T09:57:37.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.255+0000 7f26e63b7640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26e01f1760 0x7f26e00b6090 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.255+0000 7f26e63b7640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f26c006b4d0 0x7f26c006d990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.255+0000 7f26e63b7640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26e01e5d80 0x7f26e01f1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.255+0000 7f26e63b7640 1 --2- 172.21.15.139:0/995840181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26cc004800 0x7f26e00b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.255+0000 7f26e63b7640 1 -- 172.21.15.139:0/995840181 >> 172.21.15.139:0/995840181 conn(0x7f26e00b31f0 msgr2=0x7f26e00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.255+0000 7f26e63b7640 1 -- 172.21.15.139:0/995840181 shutdown_connections 2024-04-25T09:57:37.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.255+0000 7f26e63b7640 1 -- 172.21.15.139:0/995840181 wait complete. 2024-04-25T09:57:37.269 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:57:37.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.365+0000 7fb8a495c640 1 Processor -- start 2024-04-25T09:57:37.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.365+0000 7fb8a495c640 1 -- start start 2024-04-25T09:57:37.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.366+0000 7fb8a495c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb8a00711c0 0x7fb8a0071650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.366+0000 7fb8a495c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a015c960 con 0x7fb8a015da30 2024-04-25T09:57:37.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.366+0000 7fb8a495c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a015cc40 con 0x7fb8a0145230 2024-04-25T09:57:37.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.366+0000 7fb8a495c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a015d230 con 0x7fb8a00711c0 2024-04-25T09:57:37.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.366+0000 7fb89ed9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb8a015da30 0x7fb8a015de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:36234/0 (socket says 172.21.15.139:36234) 2024-04-25T09:57:37.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.366+0000 7fb89ed9d640 1 -- 172.21.15.139:0/4258408726 learned_addr learned my addr 172.21.15.139:0/4258408726 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:37.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.366+0000 7fb89dd9b640 1 --2- 172.21.15.139:0/4258408726 >> v2:172.21.15.150:3300/0 conn(0x7fb8a00711c0 0x7fb8a0071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.366+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116882200 0 0) 0x7fb8a015c960 con 0x7fb8a015da30 2024-04-25T09:57:37.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.366+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb888003520 con 0x7fb8a015da30 2024-04-25T09:57:37.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 463165919 0 0) 0x7fb888003520 con 0x7fb8a015da30 2024-04-25T09:57:37.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 >> v2:172.21.15.150:3300/0 conn(0x7fb8a00711c0 msgr2=0x7fb8a0071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 --2- 172.21.15.139:0/4258408726 >> v2:172.21.15.150:3300/0 conn(0x7fb8a00711c0 0x7fb8a0071650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 >> v1:172.21.15.150:6789/0 conn(0x7fb8a0145230 legacy=0x7fb8a015be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a015d510 con 0x7fb8a015da30 2024-04-25T09:57:37.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fb898001670 con 0x7fb8a015da30 2024-04-25T09:57:37.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 >> v1:172.21.15.139:6789/0 conn(0x7fb8a015da30 legacy=0x7fb8a015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 --2- 172.21.15.139:0/4258408726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880039b0 0x7fb888003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 --2- 172.21.15.139:0/4258408726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb888004800 0x7fb888004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 --2- 172.21.15.139:0/4258408726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8880051c0 0x7fb88800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a015d230 con 0x7fb888004800 2024-04-25T09:57:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88800c060 con 0x7fb8880051c0 2024-04-25T09:57:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88800c260 con 0x7fb8880039b0 2024-04-25T09:57:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89dd9b640 1 --2- 172.21.15.139:0/4258408726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb888004800 0x7fb888004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89e59c640 1 --2- 172.21.15.139:0/4258408726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880039b0 0x7fb888003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89ed9d640 1 --2- 172.21.15.139:0/4258408726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8880051c0 0x7fb88800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89dd9b640 1 -- 172.21.15.139:0/4258408726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880039b0 msgr2=0x7fb888003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89dd9b640 1 --2- 172.21.15.139:0/4258408726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880039b0 0x7fb888003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89dd9b640 1 -- 172.21.15.139:0/4258408726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8880051c0 msgr2=0x7fb88800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89dd9b640 1 --2- 172.21.15.139:0/4258408726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8880051c0 0x7fb88800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89dd9b640 1 -- 172.21.15.139:0/4258408726 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888003520 con 0x7fb888004800 2024-04-25T09:57:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.367+0000 7fb89ed9d640 1 -- 172.21.15.139:0/4258408726 reap_dead start 2024-04-25T09:57:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.368+0000 7fb89dd9b640 1 --2- 172.21.15.139:0/4258408726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb888004800 0x7fb888004c80 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb894009ac0 tx=0x7fb888005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4297ac6fdd1631a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.368+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894036070 con 0x7fb888004800 2024-04-25T09:57:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.368+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb894031400 con 0x7fb888004800 2024-04-25T09:57:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.368+0000 7fb89d59a640 1 -- 172.21.15.139:0/4258408726 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894009690 con 0x7fb888004800 2024-04-25T09:57:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.368+0000 7fb8a495c640 1 -- 172.21.15.139:0/4258408726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb888004800 msgr2=0x7fb888004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.368+0000 7fb8a495c640 1 --2- 172.21.15.139:0/4258408726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb888004800 0x7fb888004c80 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb894009ac0 tx=0x7fb888005830 comp rx=0 tx=0).stop 2024-04-25T09:57:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.368+0000 7fb8a495c640 1 -- 172.21.15.139:0/4258408726 shutdown_connections 2024-04-25T09:57:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.368+0000 7fb8a495c640 1 --2- 172.21.15.139:0/4258408726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb888004800 0x7fb888004c80 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.368+0000 7fb8a495c640 1 -- 172.21.15.139:0/4258408726 >> 172.21.15.139:0/4258408726 conn(0x7fb8a0067000 msgr2=0x7fb8a006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.369+0000 7fb8a495c640 1 -- 172.21.15.139:0/4258408726 shutdown_connections 2024-04-25T09:57:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.369+0000 7fb8a495c640 1 -- 172.21.15.139:0/4258408726 wait complete. 2024-04-25T09:57:37.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.370+0000 7fb8a495c640 1 Processor -- start 2024-04-25T09:57:37.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.370+0000 7fb8a495c640 1 -- start start 2024-04-25T09:57:37.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb8a495c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb888004800 0x7fb8a013e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb8a495c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8a013eb60 0x7fb8a01e4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb8a495c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a00a9460 0x7fb8a00a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb8a495c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a00717d0 con 0x7fb8a013eb60 2024-04-25T09:57:37.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb8a495c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a0071650 con 0x7fb8a00a9460 2024-04-25T09:57:37.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb8a495c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a0071950 con 0x7fb888004800 2024-04-25T09:57:37.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb89dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8a013eb60 0x7fb8a01e4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb89dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8a013eb60 0x7fb8a01e4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38916/0 (socket says 172.21.15.139:38916) 2024-04-25T09:57:37.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb89dd9b640 1 -- 172.21.15.139:0/2855483720 learned_addr learned my addr 172.21.15.139:0/2855483720 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:37.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb89ed9d640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a00a9460 0x7fb8a00a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb89e59c640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb888004800 0x7fb8a013e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb89dd9b640 1 -- 172.21.15.139:0/2855483720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb888004800 msgr2=0x7fb8a013e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb89dd9b640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb888004800 0x7fb8a013e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb89dd9b640 1 -- 172.21.15.139:0/2855483720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a00a9460 msgr2=0x7fb8a00a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb89dd9b640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a00a9460 0x7fb8a00a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb89dd9b640 1 -- 172.21.15.139:0/2855483720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a01e5090 con 0x7fb8a013eb60 2024-04-25T09:57:37.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb89dd9b640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8a013eb60 0x7fb8a01e4720 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb894009be0 tx=0x7fb894004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb88effd640 1 -- 172.21.15.139:0/2855483720 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89403f020 con 0x7fb8a013eb60 2024-04-25T09:57:37.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb88effd640 1 -- 172.21.15.139:0/2855483720 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb894037040 con 0x7fb8a013eb60 2024-04-25T09:57:37.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb8a495c640 1 -- 172.21.15.139:0/2855483720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a01e5370 con 0x7fb8a013eb60 2024-04-25T09:57:37.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb88effd640 1 -- 172.21.15.139:0/2855483720 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894036070 con 0x7fb8a013eb60 2024-04-25T09:57:37.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.371+0000 7fb8a495c640 1 -- 172.21.15.139:0/2855483720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a00ab5a0 con 0x7fb8a013eb60 2024-04-25T09:57:37.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.372+0000 7fb8a495c640 1 -- 172.21.15.139:0/2855483720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a006baf0 con 0x7fb8a013eb60 2024-04-25T09:57:37.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.373+0000 7fb88effd640 1 -- 172.21.15.139:0/2855483720 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb894042020 con 0x7fb8a013eb60 2024-04-25T09:57:37.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.373+0000 7fb88effd640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb88006b500 0x7fb88006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.373+0000 7fb88effd640 1 -- 172.21.15.139:0/2855483720 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb8940a9a50 con 0x7fb8a013eb60 2024-04-25T09:57:37.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.373+0000 7fb89e59c640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb88006b500 0x7fb88006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.374+0000 7fb89e59c640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb88006b500 0x7fb88006d9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb890000f80 tx=0x7fb890009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.377+0000 7fb88effd640 1 -- 172.21.15.139:0/2855483720 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb894078130 con 0x7fb8a013eb60 2024-04-25T09:57:37.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.546+0000 7fb8a495c640 1 -- 172.21.15.139:0/2855483720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8a00cd7e0 con 0x7fb8a013eb60 2024-04-25T09:57:37.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.547+0000 7fb88effd640 1 -- 172.21.15.139:0/2855483720 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb89407c1d0 con 0x7fb8a013eb60 2024-04-25T09:57:37.550 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:57:37.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.548+0000 7fb8a495c640 1 -- 172.21.15.139:0/2855483720 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb88006b500 msgr2=0x7fb88006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.548+0000 7fb8a495c640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb88006b500 0x7fb88006d9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb890000f80 tx=0x7fb890009040 comp rx=0 tx=0).stop 2024-04-25T09:57:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.548+0000 7fb8a495c640 1 -- 172.21.15.139:0/2855483720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8a013eb60 msgr2=0x7fb8a01e4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.548+0000 7fb8a495c640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8a013eb60 0x7fb8a01e4720 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb894009be0 tx=0x7fb894004730 comp rx=0 tx=0).stop 2024-04-25T09:57:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.548+0000 7fb8a495c640 1 -- 172.21.15.139:0/2855483720 shutdown_connections 2024-04-25T09:57:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.548+0000 7fb8a495c640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8a013eb60 0x7fb8a01e4720 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.548+0000 7fb8a495c640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb88006b500 0x7fb88006d9c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.548+0000 7fb8a495c640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a00a9460 0x7fb8a00a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.549+0000 7fb8a495c640 1 --2- 172.21.15.139:0/2855483720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb888004800 0x7fb8a013e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.549+0000 7fb8a495c640 1 -- 172.21.15.139:0/2855483720 >> 172.21.15.139:0/2855483720 conn(0x7fb8a0067000 msgr2=0x7fb8a00b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.549+0000 7fb8a495c640 1 -- 172.21.15.139:0/2855483720 shutdown_connections 2024-04-25T09:57:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.549+0000 7fb8a495c640 1 -- 172.21.15.139:0/2855483720 wait complete. 2024-04-25T09:57:37.563 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:57:37.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.657+0000 7f7ceed29640 1 Processor -- start 2024-04-25T09:57:37.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.657+0000 7f7ceed29640 1 -- start start 2024-04-25T09:57:37.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.657+0000 7f7ceed29640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7ce815dbf0 0x7f7ce81617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.657+0000 7f7ceed29640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ce80b9370 0x7f7ce80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.657+0000 7f7ceed29640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce8161dc0 con 0x7f7ce80cec80 2024-04-25T09:57:37.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.657+0000 7f7ceed29640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce80d1ee0 con 0x7f7ce815dbf0 2024-04-25T09:57:37.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.657+0000 7f7ceed29640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce81620a0 con 0x7f7ce80b9370 2024-04-25T09:57:37.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.658+0000 7f7ce7fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ce80b9370 0x7f7ce80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.658+0000 7f7cecac5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7ce815dbf0 0x7f7ce81617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.658+0000 7f7cecac5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7ce815dbf0 0x7f7ce81617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:52374/0 (socket says 172.21.15.139:52374) 2024-04-25T09:57:37.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.658+0000 7f7ced2c6640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f7ce80cec80 0x7f7ce806c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:57498/0 (socket says 172.21.15.139:57498) 2024-04-25T09:57:37.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.658+0000 7f7cecac5640 1 -- 172.21.15.139:0/4056655725 learned_addr learned my addr 172.21.15.139:0/4056655725 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:37.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.658+0000 7f7ce7fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ce80b9370 0x7f7ce80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:35572/0 (socket says 172.21.15.139:35572) 2024-04-25T09:57:37.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.658+0000 7f7ce7fff640 1 -- 172.21.15.139:0/4056655725 >> v2:172.21.15.150:3300/0 conn(0x7f7ce815dbf0 msgr2=0x7f7ce81617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.658+0000 7f7ce7fff640 1 --2- 172.21.15.139:0/4056655725 >> v2:172.21.15.150:3300/0 conn(0x7f7ce815dbf0 0x7f7ce81617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.658+0000 7f7ce7fff640 1 -- 172.21.15.139:0/4056655725 >> v1:172.21.15.150:6789/0 conn(0x7f7ce80cec80 legacy=0x7f7ce806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.658+0000 7f7ce7fff640 1 -- 172.21.15.139:0/4056655725 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce8162380 con 0x7f7ce80b9370 2024-04-25T09:57:37.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.659+0000 7f7ce7fff640 1 --2- 172.21.15.139:0/4056655725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ce80b9370 0x7f7ce80ce740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40097a0 tx=0x7f7cd402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeb5be2000d71c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.659+0000 7f7ce77fe640 1 -- 172.21.15.139:0/4056655725 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4037070 con 0x7f7ce80b9370 2024-04-25T09:57:37.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.659+0000 7f7ce77fe640 1 -- 172.21.15.139:0/4056655725 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd4032440 con 0x7f7ce80b9370 2024-04-25T09:57:37.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.659+0000 7f7ce77fe640 1 -- 172.21.15.139:0/4056655725 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4032810 con 0x7f7ce80b9370 2024-04-25T09:57:37.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.659+0000 7f7ceed29640 1 -- 172.21.15.139:0/4056655725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ce80b9370 msgr2=0x7f7ce80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.659+0000 7f7ceed29640 1 --2- 172.21.15.139:0/4056655725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ce80b9370 0x7f7ce80ce740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40097a0 tx=0x7f7cd402a180 comp rx=0 tx=0).stop 2024-04-25T09:57:37.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.659+0000 7f7ceed29640 1 -- 172.21.15.139:0/4056655725 shutdown_connections 2024-04-25T09:57:37.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.659+0000 7f7ceed29640 1 --2- 172.21.15.139:0/4056655725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ce80b9370 0x7f7ce80ce740 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.659+0000 7f7ceed29640 1 --2- 172.21.15.139:0/4056655725 >> v2:172.21.15.150:3300/0 conn(0x7f7ce815dbf0 0x7f7ce81617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.659+0000 7f7ceed29640 1 -- 172.21.15.139:0/4056655725 >> 172.21.15.139:0/4056655725 conn(0x7f7ce80b5480 msgr2=0x7f7ce80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.660+0000 7f7ceed29640 1 -- 172.21.15.139:0/4056655725 shutdown_connections 2024-04-25T09:57:37.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.660+0000 7f7ceed29640 1 -- 172.21.15.139:0/4056655725 wait complete. 2024-04-25T09:57:37.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.661+0000 7f7ceed29640 1 Processor -- start 2024-04-25T09:57:37.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.661+0000 7f7ceed29640 1 -- start start 2024-04-25T09:57:37.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.661+0000 7f7ceed29640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ce80b9370 0x7f7ce814be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.661+0000 7f7ceed29640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ce80cec80 0x7f7ce8159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ceed29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ce815dbf0 0x7f7ce8159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ceed29640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce815dfd0 con 0x7f7ce815dbf0 2024-04-25T09:57:37.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ceed29640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80b97d0 con 0x7f7ce80b9370 2024-04-25T09:57:37.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ceed29640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce81f3730 con 0x7f7ce80cec80 2024-04-25T09:57:37.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ced2c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ce815dbf0 0x7f7ce8159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ced2c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ce815dbf0 0x7f7ce8159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38928/0 (socket says 172.21.15.139:38928) 2024-04-25T09:57:37.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ced2c6640 1 -- 172.21.15.139:0/3937870515 learned_addr learned my addr 172.21.15.139:0/3937870515 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:37.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7cecac5640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ce80b9370 0x7f7ce814be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ce7fff640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ce80cec80 0x7f7ce8159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ced2c6640 1 -- 172.21.15.139:0/3937870515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ce80cec80 msgr2=0x7f7ce8159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ced2c6640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ce80cec80 0x7f7ce8159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ced2c6640 1 -- 172.21.15.139:0/3937870515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ce80b9370 msgr2=0x7f7ce814be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ced2c6640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ce80b9370 0x7f7ce814be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ced2c6640 1 -- 172.21.15.139:0/3937870515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce815a5d0 con 0x7f7ce815dbf0 2024-04-25T09:57:37.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7cecac5640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ce80b9370 0x7f7ce814be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:37.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ced2c6640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ce815dbf0 0x7f7ce8159fa0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000c7b0 tx=0x7f7ce000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ce57fa640 1 -- 172.21.15.139:0/3937870515 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0017070 con 0x7f7ce815dbf0 2024-04-25T09:57:37.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ce57fa640 1 -- 172.21.15.139:0/3937870515 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce000a640 con 0x7f7ce815dbf0 2024-04-25T09:57:37.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ceed29640 1 -- 172.21.15.139:0/3937870515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce815a8b0 con 0x7f7ce815dbf0 2024-04-25T09:57:37.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ce57fa640 1 -- 172.21.15.139:0/3937870515 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce000a9f0 con 0x7f7ce815dbf0 2024-04-25T09:57:37.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.662+0000 7f7ceed29640 1 -- 172.21.15.139:0/3937870515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce815ae70 con 0x7f7ce815dbf0 2024-04-25T09:57:37.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.663+0000 7f7ceed29640 1 -- 172.21.15.139:0/3937870515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce8074520 con 0x7f7ce815dbf0 2024-04-25T09:57:37.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.664+0000 7f7ce57fa640 1 -- 172.21.15.139:0/3937870515 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ce001a070 con 0x7f7ce815dbf0 2024-04-25T09:57:37.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.664+0000 7f7ce57fa640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ccc06b4d0 0x7f7ccc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.664+0000 7f7ce57fa640 1 -- 172.21.15.139:0/3937870515 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7ce00898b0 con 0x7f7ce815dbf0 2024-04-25T09:57:37.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.664+0000 7f7cecac5640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ccc06b4d0 0x7f7ccc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.665+0000 7f7cecac5640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ccc06b4d0 0x7f7ccc06d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc009fc0 tx=0x7f7cdc009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.668+0000 7f7ce57fa640 1 -- 172.21.15.139:0/3937870515 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ce0057f90 con 0x7f7ce815dbf0 2024-04-25T09:57:37.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.853+0000 7f7ceed29640 1 -- 172.21.15.139:0/3937870515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7ce80d1680 con 0x7f7ce815dbf0 2024-04-25T09:57:37.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.853+0000 7f7ce57fa640 1 -- 172.21.15.139:0/3937870515 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v77) v1 ==== 80+0+124 (secure 0 0 0) 0x7f7ce005c030 con 0x7f7ce815dbf0 2024-04-25T09:57:37.857 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.150:6834/1932986598 2024-04-26T09:57:29.527412+0000 2024-04-25T09:57:37.857 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.150:6832/1932986598 2024-04-26T09:57:29.527412+0000 2024-04-25T09:57:37.858 INFO:teuthology.orchestra.run.smithi139.stderr:listed 2 entries 2024-04-25T09:57:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.855+0000 7f7ceed29640 1 -- 172.21.15.139:0/3937870515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ccc06b4d0 msgr2=0x7f7ccc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.855+0000 7f7ceed29640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ccc06b4d0 0x7f7ccc06d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc009fc0 tx=0x7f7cdc009240 comp rx=0 tx=0).stop 2024-04-25T09:57:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.855+0000 7f7ceed29640 1 -- 172.21.15.139:0/3937870515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ce815dbf0 msgr2=0x7f7ce8159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.855+0000 7f7ceed29640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ce815dbf0 0x7f7ce8159fa0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000c7b0 tx=0x7f7ce000cc70 comp rx=0 tx=0).stop 2024-04-25T09:57:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.856+0000 7f7ceed29640 1 -- 172.21.15.139:0/3937870515 shutdown_connections 2024-04-25T09:57:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.856+0000 7f7ceed29640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ce815dbf0 0x7f7ce8159fa0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.856+0000 7f7ceed29640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ccc06b4d0 0x7f7ccc06d990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.856+0000 7f7ceed29640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7ce80cec80 0x7f7ce8159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.856+0000 7f7ceed29640 1 --2- 172.21.15.139:0/3937870515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ce80b9370 0x7f7ce814be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.856+0000 7f7ceed29640 1 -- 172.21.15.139:0/3937870515 >> 172.21.15.139:0/3937870515 conn(0x7f7ce80b5480 msgr2=0x7f7ce80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.856+0000 7f7ceed29640 1 -- 172.21.15.139:0/3937870515 shutdown_connections 2024-04-25T09:57:37.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:37.856+0000 7f7ceed29640 1 -- 172.21.15.139:0/3937870515 wait complete. 2024-04-25T09:57:37.870 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:57:37.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:37.870 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:37.896 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:57:37.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:37.897 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:57:37.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:37.952 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-25T09:57:38.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.091+0000 7f54cf46b640 1 Processor -- start 2024-04-25T09:57:38.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.091+0000 7f54cf46b640 1 -- start start 2024-04-25T09:57:38.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cf46b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c81617e0 0x7f54c80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cf46b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f54c80cec80 0x7f54c80d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cf46b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c8161dc0 con 0x7f54c815dbf0 2024-04-25T09:57:38.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cf46b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c80d1ee0 con 0x7f54c80cec80 2024-04-25T09:57:38.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cf46b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c81620a0 con 0x7f54c81617e0 2024-04-25T09:57:38.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cca06640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c81617e0 0x7f54c80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cda08640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f54c80cec80 0x7f54c80d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cda08640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f54c80cec80 0x7f54c80d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:52390/0 (socket says 172.21.15.139:52390) 2024-04-25T09:57:38.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cca06640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c81617e0 0x7f54c80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:35588/0 (socket says 172.21.15.139:35588) 2024-04-25T09:57:38.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cda08640 1 -- 172.21.15.139:0/206309982 learned_addr learned my addr 172.21.15.139:0/206309982 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:38.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cd207640 1 --1- 172.21.15.139:0/206309982 >> v1:172.21.15.150:6789/0 conn(0x7f54c815dbf0 0x7f54c80b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:57510/0 (socket says 172.21.15.139:57510) 2024-04-25T09:57:38.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cca06640 1 -- 172.21.15.139:0/206309982 >> v2:172.21.15.150:3300/0 conn(0x7f54c80cec80 msgr2=0x7f54c80d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cca06640 1 --2- 172.21.15.139:0/206309982 >> v2:172.21.15.150:3300/0 conn(0x7f54c80cec80 0x7f54c80d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cca06640 1 -- 172.21.15.139:0/206309982 >> v1:172.21.15.150:6789/0 conn(0x7f54c815dbf0 legacy=0x7f54c80b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cca06640 1 -- 172.21.15.139:0/206309982 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c8162380 con 0x7f54c81617e0 2024-04-25T09:57:38.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.092+0000 7f54cda08640 1 --2- 172.21.15.139:0/206309982 >> v2:172.21.15.150:3300/0 conn(0x7f54c80cec80 0x7f54c80d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:38.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.093+0000 7f54cca06640 1 --2- 172.21.15.139:0/206309982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c81617e0 0x7f54c80ce740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f54b80097a0 tx=0x7f54b802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d57a5839547355 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.093+0000 7f54bffff640 1 -- 172.21.15.139:0/206309982 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8037070 con 0x7f54c81617e0 2024-04-25T09:57:38.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.093+0000 7f54bffff640 1 -- 172.21.15.139:0/206309982 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b802ae50 con 0x7f54c81617e0 2024-04-25T09:57:38.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.093+0000 7f54bffff640 1 -- 172.21.15.139:0/206309982 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8032660 con 0x7f54c81617e0 2024-04-25T09:57:38.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.094+0000 7f54cf46b640 1 -- 172.21.15.139:0/206309982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c81617e0 msgr2=0x7f54c80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.094+0000 7f54cf46b640 1 --2- 172.21.15.139:0/206309982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c81617e0 0x7f54c80ce740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f54b80097a0 tx=0x7f54b802a100 comp rx=0 tx=0).stop 2024-04-25T09:57:38.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.094+0000 7f54cf46b640 1 -- 172.21.15.139:0/206309982 shutdown_connections 2024-04-25T09:57:38.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.094+0000 7f54cf46b640 1 --2- 172.21.15.139:0/206309982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c81617e0 0x7f54c80ce740 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.094+0000 7f54cf46b640 1 --2- 172.21.15.139:0/206309982 >> v2:172.21.15.150:3300/0 conn(0x7f54c80cec80 0x7f54c80d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.094+0000 7f54cf46b640 1 -- 172.21.15.139:0/206309982 >> 172.21.15.139:0/206309982 conn(0x7f54c80b5480 msgr2=0x7f54c80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:38.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.094+0000 7f54cf46b640 1 -- 172.21.15.139:0/206309982 shutdown_connections 2024-04-25T09:57:38.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.094+0000 7f54cf46b640 1 -- 172.21.15.139:0/206309982 wait complete. 2024-04-25T09:57:38.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.095+0000 7f54cf46b640 1 Processor -- start 2024-04-25T09:57:38.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.095+0000 7f54cf46b640 1 -- start start 2024-04-25T09:57:38.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cf46b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54c80cec80 0x7f54c80c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cf46b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c815dbf0 0x7f54c8156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cf46b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54c81617e0 0x7f54c8157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cf46b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c815dfd0 con 0x7f54c81617e0 2024-04-25T09:57:38.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cf46b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c80d28b0 con 0x7f54c80cec80 2024-04-25T09:57:38.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cf46b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8153020 con 0x7f54c815dbf0 2024-04-25T09:57:38.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cda08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54c81617e0 0x7f54c8157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cda08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54c81617e0 0x7f54c8157340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38936/0 (socket says 172.21.15.139:38936) 2024-04-25T09:57:38.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cda08640 1 -- 172.21.15.139:0/1857805902 learned_addr learned my addr 172.21.15.139:0/1857805902 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:38.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cca06640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c815dbf0 0x7f54c8156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cd207640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54c80cec80 0x7f54c80c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cda08640 1 -- 172.21.15.139:0/1857805902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c815dbf0 msgr2=0x7f54c8156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cda08640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c815dbf0 0x7f54c8156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cda08640 1 -- 172.21.15.139:0/1857805902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54c80cec80 msgr2=0x7f54c80c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cda08640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54c80cec80 0x7f54c80c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cda08640 1 -- 172.21.15.139:0/1857805902 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c80c9b10 con 0x7f54c81617e0 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cca06640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c815dbf0 0x7f54c8156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cda08640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54c81617e0 0x7f54c8157340 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f54c000efc0 tx=0x7f54c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54bdffb640 1 -- 172.21.15.139:0/1857805902 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c000cbe0 con 0x7f54c81617e0 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54bdffb640 1 -- 172.21.15.139:0/1857805902 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c000cd80 con 0x7f54c81617e0 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.096+0000 7f54cf46b640 1 -- 172.21.15.139:0/1857805902 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c8160a20 con 0x7f54c81617e0 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.097+0000 7f54bdffb640 1 -- 172.21.15.139:0/1857805902 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c0003bb0 con 0x7f54c81617e0 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.097+0000 7f54cf46b640 1 -- 172.21.15.139:0/1857805902 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c8152cb0 con 0x7f54c81617e0 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.098+0000 7f54bdffb640 1 -- 172.21.15.139:0/1857805902 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f54c001b030 con 0x7f54c81617e0 2024-04-25T09:57:38.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.098+0000 7f54bdffb640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f54ac06b500 0x7f54ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.099+0000 7f54bdffb640 1 -- 172.21.15.139:0/1857805902 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f54c008a910 con 0x7f54c81617e0 2024-04-25T09:57:38.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.099+0000 7f54cd207640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f54ac06b500 0x7f54ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.099+0000 7f54cd207640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f54ac06b500 0x7f54ac06d9c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f54c4000f60 tx=0x7f54c4007cc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.099+0000 7f54d216f740 1 Processor -- start 2024-04-25T09:57:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.099+0000 7f54d216f740 1 -- start start 2024-04-25T09:57:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54d216f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x56375807d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54d216f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56375807daa0 0x56375804d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54d216f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375804d740 0x563758075500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54d216f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x563757fb60a0 con 0x56375807d080 2024-04-25T09:57:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54d216f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563757f5ed40 con 0x56375804d740 2024-04-25T09:57:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54d216f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563757fb2140 con 0x56375807daa0 2024-04-25T09:57:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54cd207640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x56375807d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54cd207640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x56375807d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38946/0 (socket says 172.21.15.139:38946) 2024-04-25T09:57:38.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54cd207640 1 -- 172.21.15.139:0/742048693 learned_addr learned my addr 172.21.15.139:0/742048693 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54cd207640 1 -- 172.21.15.139:0/742048693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56375807daa0 msgr2=0x56375804d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54cd207640 1 --2- 172.21.15.139:0/742048693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56375807daa0 0x56375804d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54cd207640 1 -- 172.21.15.139:0/742048693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375804d740 msgr2=0x563758075500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54cd207640 1 --2- 172.21.15.139:0/742048693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375804d740 0x563758075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.100+0000 7f54cd207640 1 -- 172.21.15.139:0/742048693 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563757d96a20 con 0x56375807d080 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54cd207640 1 --2- 172.21.15.139:0/742048693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x56375807d460 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f54c40076e0 tx=0x7f54c4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877a0ee99eec2281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54b2ffd640 1 -- 172.21.15.139:0/742048693 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4001c30 con 0x56375807d080 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54b2ffd640 1 -- 172.21.15.139:0/742048693 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c40047e0 con 0x56375807d080 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54b2ffd640 1 -- 172.21.15.139:0/742048693 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4004b90 con 0x56375807d080 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54d216f740 1 -- 172.21.15.139:0/742048693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 msgr2=0x56375807d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54d216f740 1 --2- 172.21.15.139:0/742048693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x56375807d460 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f54c40076e0 tx=0x7f54c4011000 comp rx=0 tx=0).stop 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54d216f740 1 -- 172.21.15.139:0/742048693 shutdown_connections 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54d216f740 1 --2- 172.21.15.139:0/742048693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375804d740 0x563758075500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54d216f740 1 --2- 172.21.15.139:0/742048693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56375807daa0 0x56375804d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54d216f740 1 --2- 172.21.15.139:0/742048693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x56375807d460 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54d216f740 1 -- 172.21.15.139:0/742048693 >> 172.21.15.139:0/742048693 conn(0x56375804f470 msgr2=0x563758097e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54d216f740 1 -- 172.21.15.139:0/742048693 shutdown_connections 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.101+0000 7f54d216f740 1 -- 172.21.15.139:0/742048693 wait complete. 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.102+0000 7f54d216f740 1 -- start start 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.102+0000 7f54d216f740 1 Processor -- start 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54d216f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x56375806e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54d216f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375807daa0 0x56375807e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54d216f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563758070f90 0x56375806fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54d216f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637580797d0 con 0x56375807d080 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54d216f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563757f543d0 con 0x56375807daa0 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54d216f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563758079a70 con 0x563758070f90 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54d216f740 10 client.0 _ll_register_callbacks cb 0x563757cc9e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54cd207640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x56375806e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54cd207640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x56375806e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38948/0 (socket says 172.21.15.139:38948) 2024-04-25T09:57:38.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54cd207640 1 -- 172.21.15.139:0/2582596214 learned_addr learned my addr 172.21.15.139:0/2582596214 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54cca06640 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375807daa0 0x56375807e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54cda08640 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563758070f90 0x56375806fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54cd207640 1 -- 172.21.15.139:0/2582596214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563758070f90 msgr2=0x56375806fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54cd207640 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563758070f90 0x56375806fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54cd207640 1 -- 172.21.15.139:0/2582596214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375807daa0 msgr2=0x56375807e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54cd207640 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375807daa0 0x56375807e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54cd207640 1 -- 172.21.15.139:0/2582596214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563757c8a310 con 0x56375807d080 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54cd207640 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x56375806e1c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f54c4005fd0 tx=0x7f54c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab496faf17639279 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.103+0000 7f54a7fff640 1 -- 172.21.15.139:0/2582596214 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4010040 con 0x56375807d080 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54a7fff640 1 -- 172.21.15.139:0/2582596214 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f54c4021450 con 0x56375807d080 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54d216f740 1 -- 172.21.15.139:0/2582596214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54c40047e0 con 0x56375807d080 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54a7fff640 1 -- 172.21.15.139:0/2582596214 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54c402d050 con 0x56375807d080 2024-04-25T09:57:38.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54d216f740 10 client.7185 fetch_fsmap learned FSMap version 43 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54d216f740 1 -- 172.21.15.139:0/2582596214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x563757df71b0 con 0x56375807d080 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54a7fff640 1 -- 172.21.15.139:0/2582596214 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c40047e0 con 0x56375807d080 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54a7fff640 1 -- 172.21.15.139:0/2582596214 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f54c4030020 con 0x56375807d080 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54d216f740 10 client.7185 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54d216f740 10 client.7185 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54d216f740 20 client.7185 populate_metadata read hostname 'smithi139' 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54d216f740 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x563758066bd0 0x563758066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54d216f740 4 client.7185 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54d216f740 1 -- 172.21.15.139:0/2582596214 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563757fc35e0 con 0x563758066bd0 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54cca06640 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x563758066bd0 0x563758066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54cca06640 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x563758066bd0 0x563758066fb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12c77409fe08fdce server_cookie=ab764dec9a1548ee in_seq=0 out_seq=0 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.104+0000 7f54a7fff640 10 client.7185 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:57:38.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54a7fff640 1 -- 172.21.15.139:0/2582596214 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563757fc35e0 con 0x563758066bd0 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54a7fff640 10 client.7185 handle_command_reply: tid=0 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54d216f740 1 client.7185 shutdown 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54d216f740 1 -- 172.21.15.139:0/2582596214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 msgr2=0x56375806e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54d216f740 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x56375806e1c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f54c4005fd0 tx=0x7f54c4000bc0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54d216f740 1 -- 172.21.15.139:0/2582596214 shutdown_connections 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54d216f740 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x563758066bd0 0x563758066fb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54d216f740 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563758070f90 0x56375806fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54d216f740 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375807daa0 0x56375807e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54d216f740 1 --2- 172.21.15.139:0/2582596214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x56375806e1c0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54a7fff640 0 client.7185 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54d216f740 1 -- 172.21.15.139:0/2582596214 >> 172.21.15.139:0/2582596214 conn(0x56375804d740 msgr2=0x563758050c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54d216f740 1 -- 172.21.15.139:0/2582596214 shutdown_connections 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54d216f740 1 -- 172.21.15.139:0/2582596214 wait complete. 2024-04-25T09:57:38.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.105+0000 7f54d216f740 20 client.7185 trim_cache size 0 max 16384 2024-04-25T09:57:38.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.120+0000 7f54d216f740 1 Processor -- start 2024-04-25T09:57:38.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.120+0000 7f54d216f740 1 -- start start 2024-04-25T09:57:38.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54d216f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563758070f90 0x5637580bbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54d216f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375807d080 0x5637580714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54d216f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807daa0 0x56375806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54d216f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c82004f0 con 0x56375807daa0 2024-04-25T09:57:38.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54d216f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c81ff790 con 0x56375807d080 2024-04-25T09:57:38.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54d216f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c81f8110 con 0x563758070f90 2024-04-25T09:57:38.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54cda08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807daa0 0x56375806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54cda08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807daa0 0x56375806dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38950/0 (socket says 172.21.15.139:38950) 2024-04-25T09:57:38.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54cda08640 1 -- 172.21.15.139:0/1815721158 learned_addr learned my addr 172.21.15.139:0/1815721158 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:38.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54cd207640 1 --2- 172.21.15.139:0/1815721158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563758070f90 0x5637580bbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54cca06640 1 --2- 172.21.15.139:0/1815721158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375807d080 0x5637580714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54cda08640 1 -- 172.21.15.139:0/1815721158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563758070f90 msgr2=0x5637580bbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54cda08640 1 --2- 172.21.15.139:0/1815721158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563758070f90 0x5637580bbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54cda08640 1 -- 172.21.15.139:0/1815721158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375807d080 msgr2=0x5637580714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54cda08640 1 --2- 172.21.15.139:0/1815721158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375807d080 0x5637580714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54cda08640 1 -- 172.21.15.139:0/1815721158 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563758067320 con 0x56375807daa0 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54cda08640 1 --2- 172.21.15.139:0/1815721158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807daa0 0x56375806dda0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f54c0088c50 tx=0x7f54c0059520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ed9be69976bcb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54b27fc640 1 -- 172.21.15.139:0/1815721158 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c005cf70 con 0x56375807daa0 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54b27fc640 1 -- 172.21.15.139:0/1815721158 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c0061530 con 0x56375807daa0 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.121+0000 7f54b27fc640 1 -- 172.21.15.139:0/1815721158 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c00092e0 con 0x56375807daa0 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.122+0000 7f54d216f740 1 -- 172.21.15.139:0/1815721158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807daa0 msgr2=0x56375806dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.122+0000 7f54d216f740 1 --2- 172.21.15.139:0/1815721158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807daa0 0x56375806dda0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f54c0088c50 tx=0x7f54c0059520 comp rx=0 tx=0).stop 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.122+0000 7f54d216f740 1 -- 172.21.15.139:0/1815721158 shutdown_connections 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.122+0000 7f54d216f740 1 --2- 172.21.15.139:0/1815721158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807daa0 0x56375806dda0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.122+0000 7f54d216f740 1 --2- 172.21.15.139:0/1815721158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56375807d080 0x5637580714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.122+0000 7f54d216f740 1 --2- 172.21.15.139:0/1815721158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563758070f90 0x5637580bbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.122+0000 7f54d216f740 1 -- 172.21.15.139:0/1815721158 >> 172.21.15.139:0/1815721158 conn(0x56375804d740 msgr2=0x56375807c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:38.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.122+0000 7f54d216f740 1 -- 172.21.15.139:0/1815721158 shutdown_connections 2024-04-25T09:57:38.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.122+0000 7f54d216f740 1 -- 172.21.15.139:0/1815721158 wait complete. 2024-04-25T09:57:38.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.123+0000 7f54d216f740 1 -- start start 2024-04-25T09:57:38.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.123+0000 7f54d216f740 1 Processor -- start 2024-04-25T09:57:38.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.123+0000 7f54d216f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563758070f90 0x56375806f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.123+0000 7f54d216f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x563758071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.123+0000 7f54d216f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56375807daa0 0x5637580718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.123+0000 7f54d216f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8154080 con 0x56375807d080 2024-04-25T09:57:38.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.123+0000 7f54d216f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56375807e830 con 0x563758070f90 2024-04-25T09:57:38.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.123+0000 7f54d216f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637580726a0 con 0x56375807daa0 2024-04-25T09:57:38.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.123+0000 7f54d216f740 10 client.0 _ll_register_callbacks cb 0x7f54c4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:38.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54cca06640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x563758071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54cca06640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x563758071370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38966/0 (socket says 172.21.15.139:38966) 2024-04-25T09:57:38.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54cca06640 1 -- 172.21.15.139:0/2829721125 learned_addr learned my addr 172.21.15.139:0/2829721125 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:38.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54cca06640 1 -- 172.21.15.139:0/2829721125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56375807daa0 msgr2=0x5637580718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54cca06640 1 --2- 172.21.15.139:0/2829721125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56375807daa0 0x5637580718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54cd207640 1 --2- 172.21.15.139:0/2829721125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563758070f90 0x56375806f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54cca06640 1 -- 172.21.15.139:0/2829721125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563758070f90 msgr2=0x56375806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54cca06640 1 --2- 172.21.15.139:0/2829721125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563758070f90 0x56375806f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54cca06640 1 -- 172.21.15.139:0/2829721125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563757c63490 con 0x56375807d080 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54cca06640 1 --2- 172.21.15.139:0/2829721125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x563758071370 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f54b80097a0 tx=0x7f54b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a7bcb92412f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54a7fff640 1 -- 172.21.15.139:0/2829721125 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8037040 con 0x56375807d080 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54a7fff640 1 -- 172.21.15.139:0/2829721125 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f54b8045320 con 0x56375807d080 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54d216f740 1 -- 172.21.15.139:0/2829721125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54c0061530 con 0x56375807d080 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54a7fff640 1 -- 172.21.15.139:0/2829721125 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54b8057050 con 0x56375807d080 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54d216f740 10 client.7191 fetch_fsmap learned FSMap version 43 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54d216f740 1 -- 172.21.15.139:0/2829721125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x5637580f0f30 con 0x56375807d080 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54a7fff640 1 -- 172.21.15.139:0/2829721125 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c0061530 con 0x56375807d080 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54a7fff640 1 -- 172.21.15.139:0/2829721125 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f54b805a020 con 0x56375807d080 2024-04-25T09:57:38.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54d216f740 10 client.7191 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:57:38.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54d216f740 10 client.7191 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:57:38.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54d216f740 20 client.7191 populate_metadata read hostname 'smithi139' 2024-04-25T09:57:38.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54d216f740 1 --2- 172.21.15.139:0/2829721125 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x56375810b550 0x56375810b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54d216f740 4 client.7191 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-25T09:57:38.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.124+0000 7f54d216f740 1 -- 172.21.15.139:0/2829721125 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) v1 -- 0x563757fbd930 con 0x56375810b550 2024-04-25T09:57:38.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.125+0000 7f54cd207640 1 --2- 172.21.15.139:0/2829721125 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x56375810b550 0x56375810b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.125+0000 7f54a7fff640 10 client.7191 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:57:38.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.125+0000 7f54cd207640 1 --2- 172.21.15.139:0/2829721125 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x56375810b550 0x56375810b930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3230ee7c0c2f1aa0 server_cookie=4c2c72224341b88f in_seq=0 out_seq=0 2024-04-25T09:57:38.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.125+0000 7f54a7fff640 1 -- 172.21.15.139:0/2829721125 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563757fbd930 con 0x56375810b550 2024-04-25T09:57:38.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.125+0000 7f54a7fff640 10 client.7191 handle_command_reply: tid=0 2024-04-25T09:57:38.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.125+0000 7f54d216f740 1 client.7191 shutdown 2024-04-25T09:57:38.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54d216f740 1 -- 172.21.15.139:0/2829721125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 msgr2=0x563758071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54d216f740 1 --2- 172.21.15.139:0/2829721125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x563758071370 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f54b80097a0 tx=0x7f54b8004730 comp rx=0 tx=0).stop 2024-04-25T09:57:38.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54d216f740 1 -- 172.21.15.139:0/2829721125 shutdown_connections 2024-04-25T09:57:38.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54d216f740 1 --2- 172.21.15.139:0/2829721125 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x56375810b550 0x56375810b930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54d216f740 1 --2- 172.21.15.139:0/2829721125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56375807d080 0x563758071370 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54d216f740 1 --2- 172.21.15.139:0/2829721125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56375807daa0 0x5637580718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54d216f740 1 --2- 172.21.15.139:0/2829721125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563758070f90 0x56375806f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54a7fff640 0 client.7191 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:57:38.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54d216f740 1 -- 172.21.15.139:0/2829721125 >> 172.21.15.139:0/2829721125 conn(0x56375804d740 msgr2=0x56375806a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:38.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54d216f740 1 -- 172.21.15.139:0/2829721125 shutdown_connections 2024-04-25T09:57:38.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54d216f740 1 -- 172.21.15.139:0/2829721125 wait complete. 2024-04-25T09:57:38.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54d216f740 20 client.7191 trim_cache size 0 max 16384 2024-04-25T09:57:38.136 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:57:38.136 INFO:teuthology.orchestra.run.smithi139.stdout: "return_code": 0, 2024-04-25T09:57:38.136 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_tag": "fd8af296-353e-4f0c-a9a7-3cd5c5fedc9f", 2024-04-25T09:57:38.136 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": "asynchronous" 2024-04-25T09:57:38.136 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:57:38.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54cf46b640 1 -- 172.21.15.139:0/1857805902 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f54ac06b500 msgr2=0x7f54ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54cf46b640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f54ac06b500 0x7f54ac06d9c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f54c4000f60 tx=0x7f54c4007cc0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54cf46b640 1 -- 172.21.15.139:0/1857805902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54c81617e0 msgr2=0x7f54c8157340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.126+0000 7f54cf46b640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54c81617e0 0x7f54c8157340 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f54c000efc0 tx=0x7f54c000c490 comp rx=0 tx=0).stop 2024-04-25T09:57:38.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.127+0000 7f54cf46b640 1 -- 172.21.15.139:0/1857805902 shutdown_connections 2024-04-25T09:57:38.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.127+0000 7f54cf46b640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54c81617e0 0x7f54c8157340 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.127+0000 7f54cf46b640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f54ac06b500 0x7f54ac06d9c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.127+0000 7f54cf46b640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54c815dbf0 0x7f54c8156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.127+0000 7f54cf46b640 1 --2- 172.21.15.139:0/1857805902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54c80cec80 0x7f54c80c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.127+0000 7f54cf46b640 1 -- 172.21.15.139:0/1857805902 >> 172.21.15.139:0/1857805902 conn(0x7f54c80b5480 msgr2=0x7f54c80d2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:38.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.127+0000 7f54cf46b640 1 -- 172.21.15.139:0/1857805902 shutdown_connections 2024-04-25T09:57:38.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.127+0000 7f54cf46b640 1 -- 172.21.15.139:0/1857805902 wait complete. 2024-04-25T09:57:38.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:38.141 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T09:57:38.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.235+0000 7f737f37e640 1 Processor -- start 2024-04-25T09:57:38.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.235+0000 7f737f37e640 1 -- start start 2024-04-25T09:57:38.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.236+0000 7f737f37e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73780712f0 0x7f73780b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.236+0000 7f737f37e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f73780719d0 0x7f73780b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.236+0000 7f737f37e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73780b1100 con 0x7f7378145250 2024-04-25T09:57:38.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.236+0000 7f737f37e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73780ceed0 con 0x7f73780719d0 2024-04-25T09:57:38.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.236+0000 7f737f37e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73780ce2c0 con 0x7f73780712f0 2024-04-25T09:57:38.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.237+0000 7f737d91b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f73780719d0 0x7f73780b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.237+0000 7f737c919640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73780712f0 0x7f73780b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.237+0000 7f737d11a640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f7378145250 0x7f7378070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:57516/0 (socket says 172.21.15.139:57516) 2024-04-25T09:57:38.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.237+0000 7f737d11a640 1 -- 172.21.15.139:0/4152680646 learned_addr learned my addr 172.21.15.139:0/4152680646 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:57:38.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.237+0000 7f737c919640 1 -- 172.21.15.139:0/4152680646 >> v2:172.21.15.150:3300/0 conn(0x7f73780719d0 msgr2=0x7f73780b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.237+0000 7f737c919640 1 --2- 172.21.15.139:0/4152680646 >> v2:172.21.15.150:3300/0 conn(0x7f73780719d0 0x7f73780b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.237+0000 7f737c919640 1 -- 172.21.15.139:0/4152680646 >> v1:172.21.15.150:6789/0 conn(0x7f7378145250 legacy=0x7f7378070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.237+0000 7f737c919640 1 -- 172.21.15.139:0/4152680646 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73780ce5a0 con 0x7f73780712f0 2024-04-25T09:57:38.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.238+0000 7f737c919640 1 --2- 172.21.15.139:0/4152680646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73780712f0 0x7f73780b4680 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f73680097a0 tx=0x7f736802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a65060cd34c79a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.238+0000 7f736ffff640 1 -- 172.21.15.139:0/4152680646 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368037070 con 0x7f73780712f0 2024-04-25T09:57:38.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.238+0000 7f736ffff640 1 -- 172.21.15.139:0/4152680646 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736802ae50 con 0x7f73780712f0 2024-04-25T09:57:38.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.238+0000 7f736ffff640 1 -- 172.21.15.139:0/4152680646 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368032660 con 0x7f73780712f0 2024-04-25T09:57:38.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.238+0000 7f737f37e640 1 -- 172.21.15.139:0/4152680646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73780712f0 msgr2=0x7f73780b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.238+0000 7f737f37e640 1 --2- 172.21.15.139:0/4152680646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73780712f0 0x7f73780b4680 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f73680097a0 tx=0x7f736802a100 comp rx=0 tx=0).stop 2024-04-25T09:57:38.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.238+0000 7f737f37e640 1 -- 172.21.15.139:0/4152680646 shutdown_connections 2024-04-25T09:57:38.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.239+0000 7f737f37e640 1 --2- 172.21.15.139:0/4152680646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73780712f0 0x7f73780b4680 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.239+0000 7f737f37e640 1 --2- 172.21.15.139:0/4152680646 >> v2:172.21.15.150:3300/0 conn(0x7f73780719d0 0x7f73780b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.239+0000 7f737f37e640 1 -- 172.21.15.139:0/4152680646 >> 172.21.15.139:0/4152680646 conn(0x7f737806f460 msgr2=0x7f737806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:38.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.239+0000 7f737f37e640 1 -- 172.21.15.139:0/4152680646 shutdown_connections 2024-04-25T09:57:38.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.239+0000 7f737f37e640 1 -- 172.21.15.139:0/4152680646 wait complete. 2024-04-25T09:57:38.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.240+0000 7f737f37e640 1 Processor -- start 2024-04-25T09:57:38.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.240+0000 7f737f37e640 1 -- start start 2024-04-25T09:57:38.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.240+0000 7f737f37e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73780712f0 0x7f73781f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.240+0000 7f737f37e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73780719d0 0x7f737813c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.240+0000 7f737f37e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7378145250 0x7f73781d69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.240+0000 7f737f37e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7378145630 con 0x7f7378145250 2024-04-25T09:57:38.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.240+0000 7f737f37e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73780ceb60 con 0x7f73780719d0 2024-04-25T09:57:38.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.240+0000 7f737f37e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73780d11a0 con 0x7f73780712f0 2024-04-25T09:57:38.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737d91b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7378145250 0x7f73781d69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737d91b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7378145250 0x7f73781d69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38980/0 (socket says 172.21.15.139:38980) 2024-04-25T09:57:38.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737d91b640 1 -- 172.21.15.139:0/439528634 learned_addr learned my addr 172.21.15.139:0/439528634 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:38.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737c919640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73780719d0 0x7f737813c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737d11a640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73780712f0 0x7f73781f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737d91b640 1 -- 172.21.15.139:0/439528634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73780712f0 msgr2=0x7f73781f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737d91b640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73780712f0 0x7f73781f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737d91b640 1 -- 172.21.15.139:0/439528634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73780719d0 msgr2=0x7f737813c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737d91b640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73780719d0 0x7f737813c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737d91b640 1 -- 172.21.15.139:0/439528634 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73781f3520 con 0x7f7378145250 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737d91b640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7378145250 0x7f73781d69c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f737000e960 tx=0x7f737000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f736dffb640 1 -- 172.21.15.139:0/439528634 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370017070 con 0x7f7378145250 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f736dffb640 1 -- 172.21.15.139:0/439528634 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737000c920 con 0x7f7378145250 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f736dffb640 1 -- 172.21.15.139:0/439528634 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737000cd10 con 0x7f7378145250 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737f37e640 1 -- 172.21.15.139:0/439528634 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737813cfb0 con 0x7f7378145250 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.241+0000 7f737f37e640 1 -- 172.21.15.139:0/439528634 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737813d570 con 0x7f7378145250 2024-04-25T09:57:38.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.243+0000 7f736dffb640 1 -- 172.21.15.139:0/439528634 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f737001a070 con 0x7f7378145250 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.243+0000 7f736dffb640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f735c06b4d0 0x7f735c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.244+0000 7f737d11a640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f735c06b4d0 0x7f735c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.244+0000 7f736dffb640 1 -- 172.21.15.139:0/439528634 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f737008a310 con 0x7f7378145250 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.244+0000 7f737d11a640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f735c06b4d0 0x7f735c06d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f73740012e0 tx=0x7f7374001990 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.244+0000 7f7382166740 1 Processor -- start 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.245+0000 7f7382166740 1 -- start start 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.245+0000 7f7382166740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a93b10 0x5558c4a96320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.245+0000 7f7382166740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a96860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.245+0000 7f7382166740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a66770 0x5558c4a69f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.245+0000 7f7382166740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c49cfda0 con 0x5558c4a93ef0 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.245+0000 7f7382166740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c4978b80 con 0x5558c4a93b10 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.245+0000 7f7382166740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558c49cbe40 con 0x5558c4a66770 2024-04-25T09:57:38.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.245+0000 7f737c919640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a96860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.245+0000 7f737c919640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a96860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38994/0 (socket says 172.21.15.139:38994) 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.245+0000 7f737c919640 1 -- 172.21.15.139:0/4112593226 learned_addr learned my addr 172.21.15.139:0/4112593226 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.246+0000 7f737c919640 1 -- 172.21.15.139:0/4112593226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a66770 msgr2=0x5558c4a69f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.246+0000 7f737c919640 1 --2- 172.21.15.139:0/4112593226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a66770 0x5558c4a69f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.246+0000 7f737c919640 1 -- 172.21.15.139:0/4112593226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a93b10 msgr2=0x5558c4a96320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.246+0000 7f737d11a640 1 --2- 172.21.15.139:0/4112593226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a93b10 0x5558c4a96320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.246+0000 7f737c919640 1 --2- 172.21.15.139:0/4112593226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a93b10 0x5558c4a96320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.246+0000 7f737c919640 1 -- 172.21.15.139:0/4112593226 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558c4982f10 con 0x5558c4a93ef0 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.246+0000 7f737c919640 1 --2- 172.21.15.139:0/4112593226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a96860 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f73680097a0 tx=0x7f736802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562cb0bdc75dafea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.246+0000 7f735affd640 1 -- 172.21.15.139:0/4112593226 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368037040 con 0x5558c4a93ef0 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.246+0000 7f735affd640 1 -- 172.21.15.139:0/4112593226 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7368032780 con 0x5558c4a93ef0 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.246+0000 7f735affd640 1 -- 172.21.15.139:0/4112593226 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368032b30 con 0x5558c4a93ef0 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.246+0000 7f7382166740 1 -- 172.21.15.139:0/4112593226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 msgr2=0x5558c4a96860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.246+0000 7f7382166740 1 --2- 172.21.15.139:0/4112593226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a96860 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f73680097a0 tx=0x7f736802c000 comp rx=0 tx=0).stop 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.247+0000 7f7382166740 1 -- 172.21.15.139:0/4112593226 shutdown_connections 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.247+0000 7f7382166740 1 --2- 172.21.15.139:0/4112593226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a96860 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.247+0000 7f7382166740 1 --2- 172.21.15.139:0/4112593226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a66770 0x5558c4a69f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.247+0000 7f7382166740 1 --2- 172.21.15.139:0/4112593226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a93b10 0x5558c4a96320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.247+0000 7f7382166740 1 -- 172.21.15.139:0/4112593226 >> 172.21.15.139:0/4112593226 conn(0x5558c4a689e0 msgr2=0x5558c4ab1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.247+0000 7f7382166740 1 -- 172.21.15.139:0/4112593226 shutdown_connections 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.247+0000 7f7382166740 1 -- 172.21.15.139:0/4112593226 wait complete. 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.247+0000 7f7382166740 1 -- start start 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.247+0000 7f7382166740 1 Processor -- start 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f7382166740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 0x5558c4aad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f7382166740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a6af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f7382166740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 0x5558c4aab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f7382166740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c4a6bb90 con 0x5558c4a93ef0 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f7382166740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c496e4b0 con 0x5558c4a6b4d0 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f7382166740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558c4a6be70 con 0x5558c4a93b10 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f7382166740 10 client.0 _ll_register_callbacks cb 0x5558c49ae180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f737c919640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a6af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f737c919640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a6af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39006/0 (socket says 172.21.15.139:39006) 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f737c919640 1 -- 172.21.15.139:0/3770082126 learned_addr learned my addr 172.21.15.139:0/3770082126 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f737d91b640 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 0x5558c4aab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f737d11a640 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 0x5558c4aad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f737c919640 1 -- 172.21.15.139:0/3770082126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 msgr2=0x5558c4aad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f737c919640 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 0x5558c4aad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f737c919640 1 -- 172.21.15.139:0/3770082126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 msgr2=0x5558c4aab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f737c919640 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 0x5558c4aab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f737c919640 1 -- 172.21.15.139:0/3770082126 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558c47ae7d0 con 0x5558c4a93ef0 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f737d91b640 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 0x5558c4aab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.248+0000 7f737d11a640 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 0x5558c4aad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f737c919640 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a6af90 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f73680098c0 tx=0x7f7368004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c02239630dff3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7353fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7353fff640 1 -- 172.21.15.139:0/3770082126 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368037070 con 0x5558c4a93ef0 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7353fff640 1 -- 172.21.15.139:0/3770082126 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7368032d40 con 0x5558c4a93ef0 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7382166740 1 -- 172.21.15.139:0/3770082126 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7368032780 con 0x5558c4a93ef0 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7353fff640 1 -- 172.21.15.139:0/3770082126 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7368055050 con 0x5558c4a93ef0 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7382166740 10 client.7200 fetch_fsmap learned FSMap version 43 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7382166740 1 -- 172.21.15.139:0/3770082126 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x5558c46e43f0 con 0x5558c4a93ef0 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7353fff640 1 -- 172.21.15.139:0/3770082126 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7368032780 con 0x5558c4a93ef0 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7353fff640 1 -- 172.21.15.139:0/3770082126 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7368058020 con 0x5558c4a93ef0 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7382166740 10 client.7200 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7382166740 10 client.7200 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:57:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7382166740 20 client.7200 populate_metadata read hostname 'smithi139' 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7382166740 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5558c4aac360 0x5558c4ac2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7382166740 4 client.7200 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f7382166740 1 -- 172.21.15.139:0/3770082126 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558c49dd2e0 con 0x5558c4aac360 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.249+0000 7f737d11a640 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5558c4aac360 0x5558c4ac2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7353fff640 10 client.7200 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f737d11a640 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5558c4aac360 0x5558c4ac2c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ce81288bc92ece9 server_cookie=45a2e5217c57ae36 in_seq=0 out_seq=0 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7353fff640 1 -- 172.21.15.139:0/3770082126 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558c49dd2e0 con 0x5558c4aac360 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7353fff640 10 client.7200 handle_command_reply: tid=0 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7382166740 1 client.7200 shutdown 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7382166740 1 -- 172.21.15.139:0/3770082126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 msgr2=0x5558c4a6af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7382166740 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a6af90 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f73680098c0 tx=0x7f7368004730 comp rx=0 tx=0).stop 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7382166740 1 -- 172.21.15.139:0/3770082126 shutdown_connections 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7382166740 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5558c4aac360 0x5558c4ac2c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7382166740 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a6af90 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7382166740 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 0x5558c4aab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7353fff640 0 client.7200 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:57:38.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7382166740 1 --2- 172.21.15.139:0/3770082126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 0x5558c4aad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7382166740 1 -- 172.21.15.139:0/3770082126 >> 172.21.15.139:0/3770082126 conn(0x5558c4a66770 msgr2=0x5558c4a9a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:38.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7382166740 1 -- 172.21.15.139:0/3770082126 shutdown_connections 2024-04-25T09:57:38.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.250+0000 7f7382166740 1 -- 172.21.15.139:0/3770082126 wait complete. 2024-04-25T09:57:38.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.251+0000 7f7382166740 20 client.7200 trim_cache size 0 max 16384 2024-04-25T09:57:38.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.265+0000 7f7382166740 1 Processor -- start 2024-04-25T09:57:38.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.265+0000 7f7382166740 1 -- start start 2024-04-25T09:57:38.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f7382166740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 0x5558c4aae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f7382166740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 0x5558c4a65600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f7382166740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a65b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f7382166740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7378200460 con 0x5558c4a93ef0 2024-04-25T09:57:38.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f7382166740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73781ff5c0 con 0x5558c4a6b4d0 2024-04-25T09:57:38.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f7382166740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73781fea20 con 0x5558c4a93b10 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f737d91b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a65b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f737d91b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a65b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39022/0 (socket says 172.21.15.139:39022) 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f737d91b640 1 -- 172.21.15.139:0/1064173864 learned_addr learned my addr 172.21.15.139:0/1064173864 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f737d91b640 1 -- 172.21.15.139:0/1064173864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 msgr2=0x5558c4a65600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f737d91b640 1 --2- 172.21.15.139:0/1064173864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 0x5558c4a65600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f737d91b640 1 -- 172.21.15.139:0/1064173864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 msgr2=0x5558c4aae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f737d91b640 1 --2- 172.21.15.139:0/1064173864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 0x5558c4aae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f737d91b640 1 -- 172.21.15.139:0/1064173864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558c4aacbc0 con 0x5558c4a93ef0 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.266+0000 7f737d91b640 1 --2- 172.21.15.139:0/1064173864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a65b40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7370060a90 tx=0x7f737005ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9c135ecd099343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.267+0000 7f735a7fc640 1 -- 172.21.15.139:0/1064173864 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370056c60 con 0x5558c4a93ef0 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.267+0000 7f735a7fc640 1 -- 172.21.15.139:0/1064173864 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737008a070 con 0x5558c4a93ef0 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.267+0000 7f735a7fc640 1 -- 172.21.15.139:0/1064173864 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370059be0 con 0x5558c4a93ef0 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.267+0000 7f7382166740 1 -- 172.21.15.139:0/1064173864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 msgr2=0x5558c4a65b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.267+0000 7f7382166740 1 --2- 172.21.15.139:0/1064173864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a65b40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7370060a90 tx=0x7f737005ca00 comp rx=0 tx=0).stop 2024-04-25T09:57:38.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.267+0000 7f7382166740 1 -- 172.21.15.139:0/1064173864 shutdown_connections 2024-04-25T09:57:38.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.267+0000 7f7382166740 1 --2- 172.21.15.139:0/1064173864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4a65b40 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.267+0000 7f7382166740 1 --2- 172.21.15.139:0/1064173864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 0x5558c4a65600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.267+0000 7f7382166740 1 --2- 172.21.15.139:0/1064173864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 0x5558c4aae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.267+0000 7f7382166740 1 -- 172.21.15.139:0/1064173864 >> 172.21.15.139:0/1064173864 conn(0x5558c4a66770 msgr2=0x5558c4aad270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:38.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.267+0000 7f7382166740 1 -- 172.21.15.139:0/1064173864 shutdown_connections 2024-04-25T09:57:38.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.267+0000 7f7382166740 1 -- 172.21.15.139:0/1064173864 wait complete. 2024-04-25T09:57:38.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.268+0000 7f7382166740 1 -- start start 2024-04-25T09:57:38.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.268+0000 7f7382166740 1 Processor -- start 2024-04-25T09:57:38.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f7382166740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 0x5558c4ad3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f7382166740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 0x5558c4aad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f7382166740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4aad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f7382166740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c4ab1430 con 0x5558c4a93ef0 2024-04-25T09:57:38.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f7382166740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73781feff0 con 0x5558c4a6b4d0 2024-04-25T09:57:38.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f7382166740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558c4a67480 con 0x5558c4a93b10 2024-04-25T09:57:38.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f7382166740 10 client.0 _ll_register_callbacks cb 0x7f7370009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:38.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f737d91b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4aad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f737d91b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4aad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39036/0 (socket says 172.21.15.139:39036) 2024-04-25T09:57:38.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f737d91b640 1 -- 172.21.15.139:0/691148517 learned_addr learned my addr 172.21.15.139:0/691148517 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:57:38.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f737d11a640 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 0x5558c4ad3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f737c919640 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 0x5558c4aad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f737d91b640 1 -- 172.21.15.139:0/691148517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 msgr2=0x5558c4aad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f737d91b640 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 0x5558c4aad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f737d91b640 1 -- 172.21.15.139:0/691148517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 msgr2=0x5558c4ad3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:38.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f737d91b640 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 0x5558c4ad3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f737d91b640 1 -- 172.21.15.139:0/691148517 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558c46a40f0 con 0x5558c4a93ef0 2024-04-25T09:57:38.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f737d91b640 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4aad710 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f737000e930 tx=0x7f737000efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c1bf867b06ec23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:38.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f7353fff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:57:38.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f7353fff640 1 -- 172.21.15.139:0/691148517 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370059be0 con 0x5558c4a93ef0 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f7353fff640 1 -- 172.21.15.139:0/691148517 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7370059d80 con 0x5558c4a93ef0 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f7382166740 1 -- 172.21.15.139:0/691148517 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f737008a070 con 0x5558c4a93ef0 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.269+0000 7f7353fff640 1 -- 172.21.15.139:0/691148517 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7370070170 con 0x5558c4a93ef0 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7382166740 10 client.7206 fetch_fsmap learned FSMap version 43 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7382166740 1 -- 172.21.15.139:0/691148517 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x5558c4a91a60 con 0x5558c4a93ef0 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7353fff640 1 -- 172.21.15.139:0/691148517 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737008a070 con 0x5558c4a93ef0 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7353fff640 1 -- 172.21.15.139:0/691148517 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7370024060 con 0x5558c4a93ef0 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7382166740 10 client.7206 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7382166740 10 client.7206 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7382166740 20 client.7206 populate_metadata read hostname 'smithi139' 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7382166740 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5558c4aa7a10 0x5558c4b1b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7382166740 4 client.7206 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7382166740 1 -- 172.21.15.139:0/691148517 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5558c49d7630 con 0x5558c4aa7a10 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f737d11a640 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5558c4aa7a10 0x5558c4b1b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:38.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7353fff640 10 client.7206 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f737d11a640 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5558c4aa7a10 0x5558c4b1b5b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b03373a74310e6d9 server_cookie=a06added0e769c0 in_seq=0 out_seq=0 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7353fff640 1 -- 172.21.15.139:0/691148517 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5558c49d7630 con 0x5558c4aa7a10 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7353fff640 10 client.7206 handle_command_reply: tid=0 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.270+0000 7f7382166740 1 client.7206 shutdown 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f7382166740 1 -- 172.21.15.139:0/691148517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 msgr2=0x5558c4aad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f7382166740 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4aad710 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f737000e930 tx=0x7f737000efb0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f7382166740 1 -- 172.21.15.139:0/691148517 shutdown_connections 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f7382166740 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5558c4aa7a10 0x5558c4b1b5b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f7382166740 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5558c4a93ef0 0x5558c4aad710 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f7382166740 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5558c4a93b10 0x5558c4aad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f7353fff640 0 client.7206 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f7382166740 1 --2- 172.21.15.139:0/691148517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5558c4a6b4d0 0x5558c4ad3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f7382166740 1 -- 172.21.15.139:0/691148517 >> 172.21.15.139:0/691148517 conn(0x5558c4a66770 msgr2=0x5558c4aae400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f7382166740 1 -- 172.21.15.139:0/691148517 shutdown_connections 2024-04-25T09:57:38.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f7382166740 1 -- 172.21.15.139:0/691148517 wait complete. 2024-04-25T09:57:38.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f7382166740 20 client.7206 trim_cache size 0 max 16384 2024-04-25T09:57:38.281 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:57:38.281 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:57:38.281 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": { 2024-04-25T09:57:38.281 INFO:teuthology.orchestra.run.smithi139.stdout: "fd8af296-353e-4f0c-a9a7-3cd5c5fedc9f": { 2024-04-25T09:57:38.281 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/subdir", 2024-04-25T09:57:38.281 INFO:teuthology.orchestra.run.smithi139.stdout: "tag": "fd8af296-353e-4f0c-a9a7-3cd5c5fedc9f", 2024-04-25T09:57:38.281 INFO:teuthology.orchestra.run.smithi139.stdout: "options": "recursive" 2024-04-25T09:57:38.281 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:38.281 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:57:38.282 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:57:38.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f737f37e640 1 -- 172.21.15.139:0/439528634 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f735c06b4d0 msgr2=0x7f735c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f737f37e640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f735c06b4d0 0x7f735c06d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f73740012e0 tx=0x7f7374001990 comp rx=0 tx=0).stop 2024-04-25T09:57:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f737f37e640 1 -- 172.21.15.139:0/439528634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7378145250 msgr2=0x7f73781d69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f737f37e640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7378145250 0x7f73781d69c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f737000e960 tx=0x7f737000ee20 comp rx=0 tx=0).stop 2024-04-25T09:57:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f737f37e640 1 -- 172.21.15.139:0/439528634 shutdown_connections 2024-04-25T09:57:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f737f37e640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7378145250 0x7f73781d69c0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f737f37e640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f735c06b4d0 0x7f735c06d990 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f737f37e640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73780719d0 0x7f737813c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f737f37e640 1 --2- 172.21.15.139:0/439528634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73780712f0 0x7f73781f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f737f37e640 1 -- 172.21.15.139:0/439528634 >> 172.21.15.139:0/439528634 conn(0x7f737806f460 msgr2=0x7f7378070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f737f37e640 1 -- 172.21.15.139:0/439528634 shutdown_connections 2024-04-25T09:57:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:57:38.271+0000 7f737f37e640 1 -- 172.21.15.139:0/439528634 wait complete. 2024-04-25T09:57:38.285 INFO:tasks.cephfs.filesystem:scrub status for tag:fd8af296-353e-4f0c-a9a7-3cd5c5fedc9f - {'path': '/subdir', 'tag': 'fd8af296-353e-4f0c-a9a7-3cd5c5fedc9f', 'options': 'recursive'} 2024-04-25T09:57:46.072 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:57:46.075 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:08.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:08.287 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T09:58:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.394+0000 7f66fe6e1640 1 Processor -- start 2024-04-25T09:58:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.394+0000 7f66fe6e1640 1 -- start start 2024-04-25T09:58:08.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.395+0000 7f66fe6e1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f66f815dbf0 0x7f66f81617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.395+0000 7f66fe6e1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f80cee50 0x7f66f80d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.395+0000 7f66fe6e1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f8161dc0 con 0x7f66f80b9370 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.395+0000 7f66fe6e1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f80d1ee0 con 0x7f66f815dbf0 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.395+0000 7f66fe6e1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f81620a0 con 0x7f66f80cee50 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.395+0000 7f66f7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f66f815dbf0 0x7f66f81617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.395+0000 7f66f7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f66f815dbf0 0x7f66f81617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36496/0 (socket says 172.21.15.139:36496) 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.395+0000 7f66f7fff640 1 -- 172.21.15.139:0/333965061 learned_addr learned my addr 172.21.15.139:0/333965061 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.396+0000 7f66fcc7e640 1 --2- 172.21.15.139:0/333965061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f80cee50 0x7f66f80d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.396+0000 7f66f7fff640 1 -- 172.21.15.139:0/333965061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f80cee50 msgr2=0x7f66f80d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.396+0000 7f66f7fff640 1 --2- 172.21.15.139:0/333965061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f80cee50 0x7f66f80d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.396+0000 7f66f7fff640 1 -- 172.21.15.139:0/333965061 >> v1:172.21.15.150:6789/0 conn(0x7f66f80b9370 legacy=0x7f66f80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.396+0000 7f66f7fff640 1 -- 172.21.15.139:0/333965061 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f8162380 con 0x7f66f815dbf0 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.396+0000 7f66f7fff640 1 --2- 172.21.15.139:0/333965061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66f815dbf0 0x7f66f81617e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009870 tx=0x7f66e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d522ec80fb21cb67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.397+0000 7f66f6ffd640 1 -- 172.21.15.139:0/333965061 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8037070 con 0x7f66f815dbf0 2024-04-25T09:58:08.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.397+0000 7f66f6ffd640 1 -- 172.21.15.139:0/333965061 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e8032440 con 0x7f66f815dbf0 2024-04-25T09:58:08.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.397+0000 7f66f6ffd640 1 -- 172.21.15.139:0/333965061 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8032810 con 0x7f66f815dbf0 2024-04-25T09:58:08.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.397+0000 7f66fe6e1640 1 -- 172.21.15.139:0/333965061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66f815dbf0 msgr2=0x7f66f81617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.397+0000 7f66fe6e1640 1 --2- 172.21.15.139:0/333965061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66f815dbf0 0x7f66f81617e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009870 tx=0x7f66e802a270 comp rx=0 tx=0).stop 2024-04-25T09:58:08.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.397+0000 7f66fe6e1640 1 -- 172.21.15.139:0/333965061 shutdown_connections 2024-04-25T09:58:08.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.397+0000 7f66fe6e1640 1 --2- 172.21.15.139:0/333965061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f80cee50 0x7f66f80d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.397+0000 7f66fe6e1640 1 --2- 172.21.15.139:0/333965061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66f815dbf0 0x7f66f81617e0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.397+0000 7f66fe6e1640 1 -- 172.21.15.139:0/333965061 >> 172.21.15.139:0/333965061 conn(0x7f66f80b5480 msgr2=0x7f66f80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.397+0000 7f66fe6e1640 1 -- 172.21.15.139:0/333965061 shutdown_connections 2024-04-25T09:58:08.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.397+0000 7f66fe6e1640 1 -- 172.21.15.139:0/333965061 wait complete. 2024-04-25T09:58:08.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.398+0000 7f66fe6e1640 1 Processor -- start 2024-04-25T09:58:08.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.399+0000 7f66fe6e1640 1 -- start start 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.399+0000 7f66fe6e1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f80b9370 0x7f66f8160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.399+0000 7f66fe6e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66f80cee50 0x7f66f814bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.399+0000 7f66fe6e1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f815dbf0 0x7f66f815f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.399+0000 7f66fe6e1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f815dfd0 con 0x7f66f80cee50 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.399+0000 7f66fe6e1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f80b97d0 con 0x7f66f815dbf0 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.399+0000 7f66fe6e1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f815ada0 con 0x7f66f80b9370 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.399+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66f80cee50 0x7f66f814bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.399+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66f80cee50 0x7f66f814bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36498/0 (socket says 172.21.15.139:36498) 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.399+0000 7f66f77fe640 1 -- 172.21.15.139:0/3550534127 learned_addr learned my addr 172.21.15.139:0/3550534127 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66f7fff640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f80b9370 0x7f66f8160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66fcc7e640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f815dbf0 0x7f66f815f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66f77fe640 1 -- 172.21.15.139:0/3550534127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f80b9370 msgr2=0x7f66f8160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66f77fe640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f80b9370 0x7f66f8160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66f77fe640 1 -- 172.21.15.139:0/3550534127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f815dbf0 msgr2=0x7f66f815f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66f77fe640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f815dbf0 0x7f66f815f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66f77fe640 1 -- 172.21.15.139:0/3550534127 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f815e460 con 0x7f66f80cee50 2024-04-25T09:58:08.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66f77fe640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66f80cee50 0x7f66f814bd20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00c400 tx=0x7f66ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66f4ff9640 1 -- 172.21.15.139:0/3550534127 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0052d0 con 0x7f66f80cee50 2024-04-25T09:58:08.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66f4ff9640 1 -- 172.21.15.139:0/3550534127 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec005490 con 0x7f66f80cee50 2024-04-25T09:58:08.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66fe6e1640 1 -- 172.21.15.139:0/3550534127 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f815e740 con 0x7f66f80cee50 2024-04-25T09:58:08.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66f4ff9640 1 -- 172.21.15.139:0/3550534127 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec007260 con 0x7f66f80cee50 2024-04-25T09:58:08.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.400+0000 7f66fe6e1640 1 -- 172.21.15.139:0/3550534127 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f815ed00 con 0x7f66f80cee50 2024-04-25T09:58:08.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.402+0000 7f66f4ff9640 1 -- 172.21.15.139:0/3550534127 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66ec007400 con 0x7f66f80cee50 2024-04-25T09:58:08.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.402+0000 7f66f4ff9640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f66e406b4d0 0x7f66e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.402+0000 7f66f4ff9640 1 -- 172.21.15.139:0/3550534127 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f66ec089880 con 0x7f66f80cee50 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.402+0000 7f66f7fff640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f66e406b4d0 0x7f66e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.403+0000 7f66f7fff640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f66e406b4d0 0x7f66e406d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f66e8034000 tx=0x7f66e802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.403+0000 7f67013e9740 1 Processor -- start 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.403+0000 7f67013e9740 1 -- start start 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f67013e9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 0x55c4aa0e0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f67013e9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 0x55c4aa0e0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f67013e9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0b0770 0x55c4aa0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f67013e9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4aa019da0 con 0x55c4aa0ddef0 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f67013e9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a9fc2b80 con 0x55c4aa0ddb10 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f67013e9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4aa015e40 con 0x55c4aa0b0770 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 0x55c4aa0e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 0x55c4aa0e0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36500/0 (socket says 172.21.15.139:36500) 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66f77fe640 1 -- 172.21.15.139:0/2246060344 learned_addr learned my addr 172.21.15.139:0/2246060344 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:08.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66f7fff640 1 --2- 172.21.15.139:0/2246060344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 0x55c4aa0e0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66fcc7e640 1 --2- 172.21.15.139:0/2246060344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0b0770 0x55c4aa0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66f77fe640 1 -- 172.21.15.139:0/2246060344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0b0770 msgr2=0x55c4aa0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66f77fe640 1 --2- 172.21.15.139:0/2246060344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0b0770 0x55c4aa0b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66f77fe640 1 -- 172.21.15.139:0/2246060344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 msgr2=0x55c4aa0e0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66f77fe640 1 --2- 172.21.15.139:0/2246060344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 0x55c4aa0e0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66f77fe640 1 -- 172.21.15.139:0/2246060344 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4a9fccf10 con 0x55c4aa0ddef0 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66fcc7e640 1 --2- 172.21.15.139:0/2246060344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0b0770 0x55c4aa0b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66f7fff640 1 --2- 172.21.15.139:0/2246060344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 0x55c4aa0e0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66f77fe640 1 --2- 172.21.15.139:0/2246060344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 0x55c4aa0e0860 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f66ec087bd0 tx=0x7f66ec00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ee4106c226bfc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66ddffb640 1 -- 172.21.15.139:0/2246060344 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0561d0 con 0x55c4aa0ddef0 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66ddffb640 1 -- 172.21.15.139:0/2246060344 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec054540 con 0x55c4aa0ddef0 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.404+0000 7f66ddffb640 1 -- 172.21.15.139:0/2246060344 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec057320 con 0x55c4aa0ddef0 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.405+0000 7f67013e9740 1 -- 172.21.15.139:0/2246060344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 msgr2=0x55c4aa0e0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.405+0000 7f67013e9740 1 --2- 172.21.15.139:0/2246060344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 0x55c4aa0e0860 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f66ec087bd0 tx=0x7f66ec00a000 comp rx=0 tx=0).stop 2024-04-25T09:58:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.405+0000 7f67013e9740 1 -- 172.21.15.139:0/2246060344 shutdown_connections 2024-04-25T09:58:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.405+0000 7f67013e9740 1 --2- 172.21.15.139:0/2246060344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 0x55c4aa0e0860 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.405+0000 7f67013e9740 1 --2- 172.21.15.139:0/2246060344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0b0770 0x55c4aa0b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.405+0000 7f67013e9740 1 --2- 172.21.15.139:0/2246060344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 0x55c4aa0e0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.405+0000 7f67013e9740 1 -- 172.21.15.139:0/2246060344 >> 172.21.15.139:0/2246060344 conn(0x55c4aa0b29e0 msgr2=0x55c4aa0fb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.405+0000 7f67013e9740 1 -- 172.21.15.139:0/2246060344 shutdown_connections 2024-04-25T09:58:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.405+0000 7f67013e9740 1 -- 172.21.15.139:0/2246060344 wait complete. 2024-04-25T09:58:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.406+0000 7f67013e9740 1 -- start start 2024-04-25T09:58:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.406+0000 7f67013e9740 1 Processor -- start 2024-04-25T09:58:08.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f67013e9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 0x55c4aa0d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f67013e9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddef0 0x55c4aa0dc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f67013e9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 0x55c4aa0dcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f67013e9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4aa019da0 con 0x55c4aa0d4620 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f67013e9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4a9fb84b0 con 0x55c4aa0ddb10 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f67013e9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4aa0b5b90 con 0x55c4aa0ddef0 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f67013e9740 10 client.0 _ll_register_callbacks cb 0x55c4a9ff8180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66fcc7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 0x55c4aa0dcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66fcc7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 0x55c4aa0dcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36510/0 (socket says 172.21.15.139:36510) 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66fcc7e640 1 -- 172.21.15.139:0/3405798739 learned_addr learned my addr 172.21.15.139:0/3405798739 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66f7fff640 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 0x55c4aa0d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66f77fe640 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddef0 0x55c4aa0dc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66fcc7e640 1 -- 172.21.15.139:0/3405798739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddef0 msgr2=0x55c4aa0dc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66fcc7e640 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddef0 0x55c4aa0dc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66fcc7e640 1 -- 172.21.15.139:0/3405798739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 msgr2=0x55c4aa0d60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66fcc7e640 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 0x55c4aa0d60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66fcc7e640 1 -- 172.21.15.139:0/3405798739 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4a9df87d0 con 0x55c4aa0d4620 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66fcc7e640 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 0x55c4aa0dcde0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f66f00040d0 tx=0x7f66f0004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5faeff7638b46e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66d6ffd640 1 -- 172.21.15.139:0/3405798739 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f001a020 con 0x55c4aa0d4620 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66d6ffd640 1 -- 172.21.15.139:0/3405798739 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f66f0007910 con 0x55c4aa0d4620 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f67013e9740 1 -- 172.21.15.139:0/3405798739 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66ec054540 con 0x55c4aa0d4620 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f66d6ffd640 1 -- 172.21.15.139:0/3405798739 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66f0024050 con 0x55c4aa0d4620 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.407+0000 7f67013e9740 10 client.7218 fetch_fsmap learned FSMap version 43 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.408+0000 7f67013e9740 1 -- 172.21.15.139:0/3405798739 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x55c4a9d2e3f0 con 0x55c4aa0d4620 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.408+0000 7f66d6ffd640 1 -- 172.21.15.139:0/3405798739 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec054540 con 0x55c4aa0d4620 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.408+0000 7f66d6ffd640 1 -- 172.21.15.139:0/3405798739 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f66f0027020 con 0x55c4aa0d4620 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.408+0000 7f67013e9740 10 client.7218 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.408+0000 7f67013e9740 10 client.7218 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:58:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.408+0000 7f67013e9740 20 client.7218 populate_metadata read hostname 'smithi139' 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.408+0000 7f67013e9740 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55c4aa0fe7a0 0x55c4aa0febd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.408+0000 7f67013e9740 4 client.7218 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.408+0000 7f67013e9740 1 -- 172.21.15.139:0/3405798739 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4aa0272e0 con 0x55c4aa0fe7a0 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.408+0000 7f66f7fff640 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55c4aa0fe7a0 0x55c4aa0febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.408+0000 7f66d6ffd640 10 client.7218 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.408+0000 7f66f7fff640 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55c4aa0fe7a0 0x55c4aa0febd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=898ae6f8565a0450 server_cookie=e6b52583e292e14b in_seq=0 out_seq=0 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f66d6ffd640 1 -- 172.21.15.139:0/3405798739 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c4aa0272e0 con 0x55c4aa0fe7a0 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f66d6ffd640 10 client.7218 handle_command_reply: tid=0 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f67013e9740 1 client.7218 shutdown 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f67013e9740 1 -- 172.21.15.139:0/3405798739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 msgr2=0x55c4aa0dcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f67013e9740 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 0x55c4aa0dcde0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f66f00040d0 tx=0x7f66f0004590 comp rx=0 tx=0).stop 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f67013e9740 1 -- 172.21.15.139:0/3405798739 shutdown_connections 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f67013e9740 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55c4aa0fe7a0 0x55c4aa0febd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f67013e9740 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 0x55c4aa0dcde0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f67013e9740 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddef0 0x55c4aa0dc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f67013e9740 1 --2- 172.21.15.139:0/3405798739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 0x55c4aa0d60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f66d6ffd640 0 client.7218 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f67013e9740 1 -- 172.21.15.139:0/3405798739 >> 172.21.15.139:0/3405798739 conn(0x55c4aa0b0770 msgr2=0x55c4aa0dc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f67013e9740 1 -- 172.21.15.139:0/3405798739 shutdown_connections 2024-04-25T09:58:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f67013e9740 1 -- 172.21.15.139:0/3405798739 wait complete. 2024-04-25T09:58:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.409+0000 7f67013e9740 20 client.7218 trim_cache size 0 max 16384 2024-04-25T09:58:08.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.424+0000 7f67013e9740 1 Processor -- start 2024-04-25T09:58:08.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.424+0000 7f67013e9740 1 -- start start 2024-04-25T09:58:08.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f67013e9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 0x55c4aa0f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f67013e9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 0x55c4aa0f8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f67013e9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddef0 0x55c4aa0d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f67013e9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f8200570 con 0x55c4aa0d4620 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f67013e9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f81ff760 con 0x55c4aa0ddb10 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f67013e9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f815a3c0 con 0x55c4aa0ddef0 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 0x55c4aa0f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 0x55c4aa0f7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36524/0 (socket says 172.21.15.139:36524) 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f66f7fff640 1 -- 172.21.15.139:0/2214500572 learned_addr learned my addr 172.21.15.139:0/2214500572 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f66f7fff640 1 -- 172.21.15.139:0/2214500572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddef0 msgr2=0x55c4aa0d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f66f7fff640 1 --2- 172.21.15.139:0/2214500572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddef0 0x55c4aa0d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f66f7fff640 1 -- 172.21.15.139:0/2214500572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 msgr2=0x55c4aa0f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f66f7fff640 1 --2- 172.21.15.139:0/2214500572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 0x55c4aa0f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f66f7fff640 1 -- 172.21.15.139:0/2214500572 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4a9cee0f0 con 0x55c4aa0d4620 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.425+0000 7f66f7fff640 1 --2- 172.21.15.139:0/2214500572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 0x55c4aa0f7e30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009870 tx=0x7f66e8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7395f96f283437e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.426+0000 7f66dd7fa640 1 -- 172.21.15.139:0/2214500572 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8037040 con 0x55c4aa0d4620 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.426+0000 7f66dd7fa640 1 -- 172.21.15.139:0/2214500572 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e8052030 con 0x55c4aa0d4620 2024-04-25T09:58:08.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.426+0000 7f66dd7fa640 1 -- 172.21.15.139:0/2214500572 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e804ee10 con 0x55c4aa0d4620 2024-04-25T09:58:08.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.426+0000 7f67013e9740 1 -- 172.21.15.139:0/2214500572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 msgr2=0x55c4aa0f7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.426+0000 7f67013e9740 1 --2- 172.21.15.139:0/2214500572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 0x55c4aa0f7e30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009870 tx=0x7f66e8008030 comp rx=0 tx=0).stop 2024-04-25T09:58:08.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.426+0000 7f67013e9740 1 -- 172.21.15.139:0/2214500572 shutdown_connections 2024-04-25T09:58:08.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.426+0000 7f67013e9740 1 --2- 172.21.15.139:0/2214500572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddef0 0x55c4aa0d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.426+0000 7f67013e9740 1 --2- 172.21.15.139:0/2214500572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0ddb10 0x55c4aa0f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.426+0000 7f67013e9740 1 --2- 172.21.15.139:0/2214500572 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0d4620 0x55c4aa0f7e30 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.426+0000 7f67013e9740 1 -- 172.21.15.139:0/2214500572 >> 172.21.15.139:0/2214500572 conn(0x55c4aa0b0770 msgr2=0x55c4aa0e02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.426+0000 7f67013e9740 1 -- 172.21.15.139:0/2214500572 shutdown_connections 2024-04-25T09:58:08.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.426+0000 7f67013e9740 1 -- 172.21.15.139:0/2214500572 wait complete. 2024-04-25T09:58:08.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.427+0000 7f67013e9740 1 -- start start 2024-04-25T09:58:08.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.427+0000 7f67013e9740 1 Processor -- start 2024-04-25T09:58:08.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f67013e9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0d4620 0x55c4aa0b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f67013e9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddb10 0x55c4aa0f8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f67013e9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 0x55c4aa0f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f67013e9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4aa0d70f0 con 0x55c4aa0ddef0 2024-04-25T09:58:08.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f67013e9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f81ff230 con 0x55c4aa0d4620 2024-04-25T09:58:08.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f67013e9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4aa0f1a40 con 0x55c4aa0ddb10 2024-04-25T09:58:08.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f67013e9740 10 client.0 _ll_register_callbacks cb 0x7f66e8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:08.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66fcc7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 0x55c4aa0f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66fcc7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 0x55c4aa0f0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36528/0 (socket says 172.21.15.139:36528) 2024-04-25T09:58:08.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66fcc7e640 1 -- 172.21.15.139:0/2148772100 learned_addr learned my addr 172.21.15.139:0/2148772100 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66f7fff640 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0d4620 0x55c4aa0b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66f77fe640 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddb10 0x55c4aa0f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66fcc7e640 1 -- 172.21.15.139:0/2148772100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddb10 msgr2=0x55c4aa0f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66fcc7e640 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddb10 0x55c4aa0f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66fcc7e640 1 -- 172.21.15.139:0/2148772100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0d4620 msgr2=0x55c4aa0b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66fcc7e640 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0d4620 0x55c4aa0b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66fcc7e640 1 -- 172.21.15.139:0/2148772100 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4aa11caa0 con 0x55c4aa0ddef0 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66fcc7e640 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 0x55c4aa0f0cb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f66f000b000 tx=0x7f66f0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a35552fde028c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66d6ffd640 1 -- 172.21.15.139:0/2148772100 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f002c020 con 0x55c4aa0ddef0 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f66d6ffd640 1 -- 172.21.15.139:0/2148772100 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f66f0007700 con 0x55c4aa0ddef0 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.428+0000 7f67013e9740 1 -- 172.21.15.139:0/2148772100 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66e8052030 con 0x55c4aa0ddef0 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f66d6ffd640 1 -- 172.21.15.139:0/2148772100 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66f001a070 con 0x55c4aa0ddef0 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f67013e9740 10 client.7224 fetch_fsmap learned FSMap version 43 2024-04-25T09:58:08.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f67013e9740 1 -- 172.21.15.139:0/2148772100 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x55c4aa0f1810 con 0x55c4aa0ddef0 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f66d6ffd640 1 -- 172.21.15.139:0/2148772100 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e8052030 con 0x55c4aa0ddef0 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f66d6ffd640 1 -- 172.21.15.139:0/2148772100 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f66f0027070 con 0x55c4aa0ddef0 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f67013e9740 10 client.7224 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f67013e9740 10 client.7224 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f67013e9740 20 client.7224 populate_metadata read hostname 'smithi139' 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f67013e9740 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55c4aa11dc70 0x55c4aa0f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f67013e9740 4 client.7224 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f67013e9740 1 -- 172.21.15.139:0/2148772100 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c4aa021630 con 0x55c4aa11dc70 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f66f7fff640 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55c4aa11dc70 0x55c4aa0f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f66d6ffd640 10 client.7224 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f66f7fff640 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55c4aa11dc70 0x55c4aa0f11f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7984c1fc8fa60d61 server_cookie=2806c7ce0a34d7e7 in_seq=0 out_seq=0 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f66d6ffd640 1 -- 172.21.15.139:0/2148772100 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c4aa021630 con 0x55c4aa11dc70 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.429+0000 7f66d6ffd640 10 client.7224 handle_command_reply: tid=0 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f67013e9740 1 client.7224 shutdown 2024-04-25T09:58:08.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f67013e9740 1 -- 172.21.15.139:0/2148772100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 msgr2=0x55c4aa0f0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f67013e9740 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 0x55c4aa0f0cb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f66f000b000 tx=0x7f66f0004ab0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f67013e9740 1 -- 172.21.15.139:0/2148772100 shutdown_connections 2024-04-25T09:58:08.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f67013e9740 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55c4aa11dc70 0x55c4aa0f11f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f67013e9740 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4aa0ddef0 0x55c4aa0f0cb0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f67013e9740 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c4aa0ddb10 0x55c4aa0f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f67013e9740 1 --2- 172.21.15.139:0/2148772100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4aa0d4620 0x55c4aa0b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f67013e9740 1 -- 172.21.15.139:0/2148772100 >> 172.21.15.139:0/2148772100 conn(0x55c4aa0b0770 msgr2=0x55c4aa0fb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f66d6ffd640 0 client.7224 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:08.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f67013e9740 1 -- 172.21.15.139:0/2148772100 shutdown_connections 2024-04-25T09:58:08.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f67013e9740 1 -- 172.21.15.139:0/2148772100 wait complete. 2024-04-25T09:58:08.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f67013e9740 20 client.7224 trim_cache size 0 max 16384 2024-04-25T09:58:08.440 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:58:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "no active scrubs running", 2024-04-25T09:58:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": {} 2024-04-25T09:58:08.440 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:58:08.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f66fe6e1640 1 -- 172.21.15.139:0/3550534127 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f66e406b4d0 msgr2=0x7f66e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f66fe6e1640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f66e406b4d0 0x7f66e406d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f66e8034000 tx=0x7f66e802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:08.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f66fe6e1640 1 -- 172.21.15.139:0/3550534127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66f80cee50 msgr2=0x7f66f814bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f66fe6e1640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66f80cee50 0x7f66f814bd20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00c400 tx=0x7f66ec00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f66fe6e1640 1 -- 172.21.15.139:0/3550534127 shutdown_connections 2024-04-25T09:58:08.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f66fe6e1640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66f80cee50 0x7f66f814bd20 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f66fe6e1640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f66e406b4d0 0x7f66e406d990 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f66fe6e1640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f815dbf0 0x7f66f815f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f66fe6e1640 1 --2- 172.21.15.139:0/3550534127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f80b9370 0x7f66f8160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.430+0000 7f66fe6e1640 1 -- 172.21.15.139:0/3550534127 >> 172.21.15.139:0/3550534127 conn(0x7f66f80b5480 msgr2=0x7f66f80b7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.431+0000 7f66fe6e1640 1 -- 172.21.15.139:0/3550534127 shutdown_connections 2024-04-25T09:58:08.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.431+0000 7f66fe6e1640 1 -- 172.21.15.139:0/3550534127 wait complete. 2024-04-25T09:58:08.445 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:58:08.445 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:08.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.539+0000 7f89d437e640 1 Processor -- start 2024-04-25T09:58:08.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.539+0000 7f89d437e640 1 -- start start 2024-04-25T09:58:08.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.540+0000 7f89d437e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f89cc15a860 0x7f89cc0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.540+0000 7f89d437e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f89cc06ea20 0x7f89cc0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.540+0000 7f89d437e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89cc0af6d0 con 0x7f89cc0cd1f0 2024-04-25T09:58:08.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.540+0000 7f89d437e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89cc0b02e0 con 0x7f89cc15a860 2024-04-25T09:58:08.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.540+0000 7f89d437e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89cc0af9b0 con 0x7f89cc06ea20 2024-04-25T09:58:08.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.540+0000 7f89d211a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f89cc15a860 0x7f89cc0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.540+0000 7f89d1919640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f89cc0cd1f0 0x7f89cc06e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:50812/0 (socket says 172.21.15.139:50812) 2024-04-25T09:58:08.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.540+0000 7f89d1919640 1 -- 172.21.15.139:0/1494676136 learned_addr learned my addr 172.21.15.139:0/1494676136 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:08.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.540+0000 7f89d211a640 1 -- 172.21.15.139:0/1494676136 >> v2:172.21.15.150:3300/0 conn(0x7f89cc06ea20 msgr2=0x7f89cc0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.540+0000 7f89d211a640 1 --2- 172.21.15.139:0/1494676136 >> v2:172.21.15.150:3300/0 conn(0x7f89cc06ea20 0x7f89cc0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.540+0000 7f89d211a640 1 -- 172.21.15.139:0/1494676136 >> v1:172.21.15.139:6789/0 conn(0x7f89cc0cd1f0 legacy=0x7f89cc06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.540+0000 7f89d211a640 1 -- 172.21.15.139:0/1494676136 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc0afc90 con 0x7f89cc15a860 2024-04-25T09:58:08.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.541+0000 7f89d211a640 1 --2- 172.21.15.139:0/1494676136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89cc15a860 0x7f89cc0cccb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0b0460 tx=0x7f89c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a903bcebc4cf635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.541+0000 7f89d1118640 1 -- 172.21.15.139:0/1494676136 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8037070 con 0x7f89cc15a860 2024-04-25T09:58:08.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.541+0000 7f89d1118640 1 -- 172.21.15.139:0/1494676136 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c8032440 con 0x7f89cc15a860 2024-04-25T09:58:08.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.541+0000 7f89d1118640 1 -- 172.21.15.139:0/1494676136 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8032810 con 0x7f89cc15a860 2024-04-25T09:58:08.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.541+0000 7f89d437e640 1 -- 172.21.15.139:0/1494676136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89cc15a860 msgr2=0x7f89cc0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.541+0000 7f89d437e640 1 --2- 172.21.15.139:0/1494676136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89cc15a860 0x7f89cc0cccb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0b0460 tx=0x7f89c802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.541+0000 7f89d437e640 1 -- 172.21.15.139:0/1494676136 shutdown_connections 2024-04-25T09:58:08.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.541+0000 7f89d437e640 1 --2- 172.21.15.139:0/1494676136 >> v2:172.21.15.150:3300/0 conn(0x7f89cc06ea20 0x7f89cc0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.541+0000 7f89d437e640 1 --2- 172.21.15.139:0/1494676136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89cc15a860 0x7f89cc0cccb0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.541+0000 7f89d437e640 1 -- 172.21.15.139:0/1494676136 >> 172.21.15.139:0/1494676136 conn(0x7f89cc066f20 msgr2=0x7f89cc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.541+0000 7f89d437e640 1 -- 172.21.15.139:0/1494676136 shutdown_connections 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.542+0000 7f89d437e640 1 -- 172.21.15.139:0/1494676136 wait complete. 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.543+0000 7f89d437e640 1 Processor -- start 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.543+0000 7f89d437e640 1 -- start start 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.543+0000 7f89d437e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89cc06ea20 0x7f89cc150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.543+0000 7f89d437e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89cc0cd1f0 0x7f89cc1f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.543+0000 7f89d437e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89cc15a860 0x7f89cc0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.543+0000 7f89d437e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc0af420 con 0x7f89cc06ea20 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.543+0000 7f89d437e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc0af2a0 con 0x7f89cc15a860 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.543+0000 7f89d437e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc15ac40 con 0x7f89cc0cd1f0 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.543+0000 7f89d211a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89cc06ea20 0x7f89cc150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.543+0000 7f89d211a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89cc06ea20 0x7f89cc150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36532/0 (socket says 172.21.15.139:36532) 2024-04-25T09:58:08.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.543+0000 7f89d211a640 1 -- 172.21.15.139:0/1114671593 learned_addr learned my addr 172.21.15.139:0/1114671593 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:08.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.544+0000 7f89d211a640 1 -- 172.21.15.139:0/1114671593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89cc0cd1f0 msgr2=0x7f89cc1f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.544+0000 7f89d211a640 1 --2- 172.21.15.139:0/1114671593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89cc0cd1f0 0x7f89cc1f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.544+0000 7f89d211a640 1 -- 172.21.15.139:0/1114671593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89cc15a860 msgr2=0x7f89cc0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.544+0000 7f89d211a640 1 --2- 172.21.15.139:0/1114671593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89cc15a860 0x7f89cc0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.544+0000 7f89d211a640 1 -- 172.21.15.139:0/1114671593 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc0b2fa0 con 0x7f89cc06ea20 2024-04-25T09:58:08.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.544+0000 7f89d211a640 1 --2- 172.21.15.139:0/1114671593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89cc06ea20 0x7f89cc150dc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f89c8000c00 tx=0x7f89c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.544+0000 7f89c2ffd640 1 -- 172.21.15.139:0/1114671593 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8041070 con 0x7f89cc06ea20 2024-04-25T09:58:08.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.544+0000 7f89c2ffd640 1 -- 172.21.15.139:0/1114671593 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c803c950 con 0x7f89cc06ea20 2024-04-25T09:58:08.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.544+0000 7f89d437e640 1 -- 172.21.15.139:0/1114671593 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89cc0b3280 con 0x7f89cc06ea20 2024-04-25T09:58:08.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.544+0000 7f89c2ffd640 1 -- 172.21.15.139:0/1114671593 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8037040 con 0x7f89cc06ea20 2024-04-25T09:58:08.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.544+0000 7f89d437e640 1 -- 172.21.15.139:0/1114671593 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc0b3840 con 0x7f89cc06ea20 2024-04-25T09:58:08.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.545+0000 7f89d437e640 1 -- 172.21.15.139:0/1114671593 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89cc074520 con 0x7f89cc06ea20 2024-04-25T09:58:08.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.545+0000 7f89c2ffd640 1 -- 172.21.15.139:0/1114671593 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89c8044070 con 0x7f89cc06ea20 2024-04-25T09:58:08.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.546+0000 7f89c2ffd640 1 --2- 172.21.15.139:0/1114671593 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f89b006b500 0x7f89b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.546+0000 7f89c2ffd640 1 -- 172.21.15.139:0/1114671593 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f89c80ab070 con 0x7f89cc06ea20 2024-04-25T09:58:08.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.546+0000 7f89d1919640 1 --2- 172.21.15.139:0/1114671593 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f89b006b500 0x7f89b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.546+0000 7f89d1919640 1 --2- 172.21.15.139:0/1114671593 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f89b006b500 0x7f89b006d9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f89bc000f60 tx=0x7f89bc007a80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.550+0000 7f89c2ffd640 1 -- 172.21.15.139:0/1114671593 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89c8079ec0 con 0x7f89cc06ea20 2024-04-25T09:58:08.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.757+0000 7f89d437e640 1 -- 172.21.15.139:0/1114671593 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89cc06ae40 con 0x7f89cc06ea20 2024-04-25T09:58:08.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.758+0000 7f89c2ffd640 1 -- 172.21.15.139:0/1114671593 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f89c80797c0 con 0x7f89cc06ea20 2024-04-25T09:58:08.761 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:08.761 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/982251530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":982251530},{"type":"v1","addr":"172.21.15.150:6834","nonce":982251530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:20.444843+0000","modified":"2024-04-25T09:57:36.538707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":8,"addr":"172.21.15.139:6837/1384380923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":1384380923},{"type":"v1","addr":"172.21.15.139:6837","nonce":1384380923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:58:08.762 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 43 2024-04-25T09:58:08.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.759+0000 7f89d437e640 1 -- 172.21.15.139:0/1114671593 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f89b006b500 msgr2=0x7f89b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.759+0000 7f89d437e640 1 --2- 172.21.15.139:0/1114671593 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f89b006b500 0x7f89b006d9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f89bc000f60 tx=0x7f89bc007a80 comp rx=0 tx=0).stop 2024-04-25T09:58:08.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.759+0000 7f89d437e640 1 -- 172.21.15.139:0/1114671593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89cc06ea20 msgr2=0x7f89cc150dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.759+0000 7f89d437e640 1 --2- 172.21.15.139:0/1114671593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89cc06ea20 0x7f89cc150dc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f89c8000c00 tx=0x7f89c80046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.760+0000 7f89d437e640 1 -- 172.21.15.139:0/1114671593 shutdown_connections 2024-04-25T09:58:08.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.760+0000 7f89d437e640 1 --2- 172.21.15.139:0/1114671593 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f89b006b500 0x7f89b006d9c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.760+0000 7f89d437e640 1 --2- 172.21.15.139:0/1114671593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89cc15a860 0x7f89cc0b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.760+0000 7f89d437e640 1 --2- 172.21.15.139:0/1114671593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f89cc0cd1f0 0x7f89cc1f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.760+0000 7f89d437e640 1 --2- 172.21.15.139:0/1114671593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89cc06ea20 0x7f89cc150dc0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.760+0000 7f89d437e640 1 -- 172.21.15.139:0/1114671593 >> 172.21.15.139:0/1114671593 conn(0x7f89cc066f20 msgr2=0x7f89cc1576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.760+0000 7f89d437e640 1 -- 172.21.15.139:0/1114671593 shutdown_connections 2024-04-25T09:58:08.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.760+0000 7f89d437e640 1 -- 172.21.15.139:0/1114671593 wait complete. 2024-04-25T09:58:08.774 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T09:58:08.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.870+0000 7f59a7646640 1 Processor -- start 2024-04-25T09:58:08.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.870+0000 7f59a7646640 1 -- start start 2024-04-25T09:58:08.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f59a7646640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f59a01617e0 0x7f59a00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f59a7646640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a0161dc0 con 0x7f59a015dbf0 2024-04-25T09:58:08.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f59a7646640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a01620a0 con 0x7f59a00cec80 2024-04-25T09:58:08.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f59a7646640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a00cf0d0 con 0x7f59a01617e0 2024-04-25T09:58:08.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f59a53e2640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f59a015dbf0 0x7f59a00b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:50818/0 (socket says 172.21.15.139:50818) 2024-04-25T09:58:08.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f59a53e2640 1 -- 172.21.15.139:0/1025718468 learned_addr learned my addr 172.21.15.139:0/1025718468 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f59a4be1640 1 --2- 172.21.15.139:0/1025718468 >> v2:172.21.15.150:3300/0 conn(0x7f59a01617e0 0x7f59a00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61134427 0 0) 0x7f59a0161dc0 con 0x7f59a015dbf0 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f598c003520 con 0x7f59a015dbf0 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4192912530 0 0) 0x7f598c003520 con 0x7f59a015dbf0 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 >> v2:172.21.15.150:3300/0 conn(0x7f59a01617e0 msgr2=0x7f59a00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f5997fff640 1 --2- 172.21.15.139:0/1025718468 >> v2:172.21.15.150:3300/0 conn(0x7f59a01617e0 0x7f59a00ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 >> v1:172.21.15.150:6789/0 conn(0x7f59a00cec80 legacy=0x7f59a00d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a0162380 con 0x7f59a015dbf0 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f599c001670 con 0x7f59a015dbf0 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.871+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 >> v1:172.21.15.139:6789/0 conn(0x7f59a015dbf0 legacy=0x7f59a00b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f5997fff640 1 --2- 172.21.15.139:0/1025718468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 0x7f598c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f5997fff640 1 --2- 172.21.15.139:0/1025718468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f598c004800 0x7f598c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f5997fff640 1 --2- 172.21.15.139:0/1025718468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f598c0051c0 0x7f598c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00cf0d0 con 0x7f598c0039b0 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c00c160 con 0x7f598c0051c0 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c00c360 con 0x7f598c004800 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1025718468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 0x7f598c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f59a4be1640 1 --2- 172.21.15.139:0/1025718468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f598c004800 0x7f598c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f59a5be3640 1 --2- 172.21.15.139:0/1025718468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f598c0051c0 0x7f598c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f59a53e2640 1 -- 172.21.15.139:0/1025718468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f598c004800 msgr2=0x7f598c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1025718468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f598c004800 0x7f598c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f59a53e2640 1 -- 172.21.15.139:0/1025718468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f598c0051c0 msgr2=0x7f598c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1025718468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f598c0051c0 0x7f598c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f59a5be3640 1 -- 172.21.15.139:0/1025718468 reap_dead start 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f59a53e2640 1 -- 172.21.15.139:0/1025718468 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c003520 con 0x7f598c0039b0 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.872+0000 7f59a4be1640 1 --2- 172.21.15.139:0/1025718468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f598c004800 0x7f598c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.873+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1025718468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 0x7f598c003dd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f599c00c240 tx=0x7f598c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c966a11f3c20cf09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.873+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c007d10 con 0x7f598c0039b0 2024-04-25T09:58:08.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.873+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c005ad0 con 0x7f598c0039b0 2024-04-25T09:58:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.873+0000 7f5997fff640 1 -- 172.21.15.139:0/1025718468 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c007070 con 0x7f598c0039b0 2024-04-25T09:58:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.873+0000 7f59a7646640 1 -- 172.21.15.139:0/1025718468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 msgr2=0x7f598c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.873+0000 7f59a7646640 1 --2- 172.21.15.139:0/1025718468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 0x7f598c003dd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f599c00c240 tx=0x7f598c005830 comp rx=0 tx=0).stop 2024-04-25T09:58:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.873+0000 7f59a7646640 1 -- 172.21.15.139:0/1025718468 shutdown_connections 2024-04-25T09:58:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.873+0000 7f59a7646640 1 --2- 172.21.15.139:0/1025718468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 0x7f598c003dd0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.873+0000 7f59a7646640 1 -- 172.21.15.139:0/1025718468 >> 172.21.15.139:0/1025718468 conn(0x7f59a00b5480 msgr2=0x7f59a00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.873+0000 7f59a7646640 1 -- 172.21.15.139:0/1025718468 shutdown_connections 2024-04-25T09:58:08.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.873+0000 7f59a7646640 1 -- 172.21.15.139:0/1025718468 wait complete. 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.875+0000 7f59a7646640 1 Processor -- start 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.875+0000 7f59a7646640 1 -- start start 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.875+0000 7f59a7646640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 0x7f59a014ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.875+0000 7f59a7646640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59a0160ee0 0x7f59a015e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.875+0000 7f59a7646640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59a00cc580 0x7f59a00cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.875+0000 7f59a7646640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a015dfd0 con 0x7f598c0039b0 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.875+0000 7f59a7646640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00d1ee0 con 0x7f59a00cc580 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.875+0000 7f59a7646640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00c8540 con 0x7f59a0160ee0 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.875+0000 7f59a53e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 0x7f59a014ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.875+0000 7f59a53e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 0x7f59a014ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36544/0 (socket says 172.21.15.139:36544) 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.875+0000 7f59a53e2640 1 -- 172.21.15.139:0/1059714519 learned_addr learned my addr 172.21.15.139:0/1059714519 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.876+0000 7f59a5be3640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59a00cc580 0x7f59a00cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.876+0000 7f59a53e2640 1 -- 172.21.15.139:0/1059714519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59a0160ee0 msgr2=0x7f59a015e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.876+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59a0160ee0 0x7f59a015e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.876+0000 7f59a53e2640 1 -- 172.21.15.139:0/1059714519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59a00cc580 msgr2=0x7f59a00cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.876+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59a00cc580 0x7f59a00cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.876+0000 7f59a53e2640 1 -- 172.21.15.139:0/1059714519 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a015f0b0 con 0x7f598c0039b0 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.876+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 0x7f59a014ebe0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f599c034040 tx=0x7f59a00d1e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.876+0000 7f5995ffb640 1 -- 172.21.15.139:0/1059714519 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c041070 con 0x7f598c0039b0 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.876+0000 7f5995ffb640 1 -- 172.21.15.139:0/1059714519 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c004e40 con 0x7f598c0039b0 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.876+0000 7f59a7646640 1 -- 172.21.15.139:0/1059714519 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a015f390 con 0x7f598c0039b0 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.876+0000 7f5995ffb640 1 -- 172.21.15.139:0/1059714519 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c007d10 con 0x7f598c0039b0 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.876+0000 7f59a7646640 1 -- 172.21.15.139:0/1059714519 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a014f120 con 0x7f598c0039b0 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.878+0000 7f5995ffb640 1 -- 172.21.15.139:0/1059714519 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f599c044070 con 0x7f598c0039b0 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.878+0000 7f5995ffb640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f598406b4d0 0x7f598406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.878+0000 7f5995ffb640 1 -- 172.21.15.139:0/1059714519 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f599c0aa790 con 0x7f598c0039b0 2024-04-25T09:58:08.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.878+0000 7f59a4be1640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f598406b4d0 0x7f598406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.879+0000 7f59a4be1640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f598406b4d0 0x7f598406d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f59a0158410 tx=0x7f5990008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.879+0000 7f59aa422740 1 Processor -- start 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.879+0000 7f59aa422740 1 -- start start 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59aa422740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b2650590 0x5609b26526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59aa422740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b268cc60 0x5609b2650970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59aa422740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b267b050 0x5609b267b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59aa422740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b25b3c60 con 0x5609b267b050 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59aa422740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b2552d80 con 0x5609b2650590 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59aa422740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609b25afd00 con 0x5609b268cc60 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59a5be3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b267b050 0x5609b267b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59a5be3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b267b050 0x5609b267b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36556/0 (socket says 172.21.15.139:36556) 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59a5be3640 1 -- 172.21.15.139:0/998321121 learned_addr learned my addr 172.21.15.139:0/998321121 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59a53e2640 1 --2- 172.21.15.139:0/998321121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b2650590 0x5609b26526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59a4be1640 1 --2- 172.21.15.139:0/998321121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b268cc60 0x5609b2650970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59a5be3640 1 -- 172.21.15.139:0/998321121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b268cc60 msgr2=0x5609b2650970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59a5be3640 1 --2- 172.21.15.139:0/998321121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b268cc60 0x5609b2650970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59a5be3640 1 -- 172.21.15.139:0/998321121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b2650590 msgr2=0x5609b26526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59a5be3640 1 --2- 172.21.15.139:0/998321121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b2650590 0x5609b26526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59a5be3640 1 -- 172.21.15.139:0/998321121 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609b23947c0 con 0x5609b267b050 2024-04-25T09:58:08.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.880+0000 7f59a5be3640 1 --2- 172.21.15.139:0/998321121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b267b050 0x5609b267b4d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f59a00d4a30 tx=0x7f5998001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6781be5950138c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.881+0000 7f598affd640 1 -- 172.21.15.139:0/998321121 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599801a070 con 0x5609b267b050 2024-04-25T09:58:08.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.881+0000 7f598affd640 1 -- 172.21.15.139:0/998321121 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5998005070 con 0x5609b267b050 2024-04-25T09:58:08.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.881+0000 7f598affd640 1 -- 172.21.15.139:0/998321121 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998005460 con 0x5609b267b050 2024-04-25T09:58:08.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.882+0000 7f59aa422740 1 -- 172.21.15.139:0/998321121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b267b050 msgr2=0x5609b267b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.882+0000 7f59aa422740 1 --2- 172.21.15.139:0/998321121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b267b050 0x5609b267b4d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f59a00d4a30 tx=0x7f5998001610 comp rx=0 tx=0).stop 2024-04-25T09:58:08.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.882+0000 7f59aa422740 1 -- 172.21.15.139:0/998321121 shutdown_connections 2024-04-25T09:58:08.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.882+0000 7f59aa422740 1 --2- 172.21.15.139:0/998321121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b267b050 0x5609b267b4d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.882+0000 7f59aa422740 1 --2- 172.21.15.139:0/998321121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b268cc60 0x5609b2650970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.882+0000 7f59aa422740 1 --2- 172.21.15.139:0/998321121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b2650590 0x5609b26526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.882+0000 7f59aa422740 1 -- 172.21.15.139:0/998321121 >> 172.21.15.139:0/998321121 conn(0x5609b26186e0 msgr2=0x5609b2696050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.882+0000 7f59aa422740 1 -- 172.21.15.139:0/998321121 shutdown_connections 2024-04-25T09:58:08.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.882+0000 7f59aa422740 1 -- 172.21.15.139:0/998321121 wait complete. 2024-04-25T09:58:08.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.882+0000 7f59aa422740 1 -- start start 2024-04-25T09:58:08.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.883+0000 7f59aa422740 1 Processor -- start 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.883+0000 7f59aa422740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b2650590 0x5609b267b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.883+0000 7f59aa422740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b267b050 0x5609b26a9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.883+0000 7f59aa422740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b2698970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.883+0000 7f59aa422740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b2550840 con 0x5609b268cc60 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.883+0000 7f59aa422740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b2678ba0 con 0x5609b2650590 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.883+0000 7f59aa422740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609b267e2c0 con 0x5609b267b050 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.883+0000 7f59aa422740 10 client.0 _ll_register_callbacks cb 0x5609b256a980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59a5be3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b2698970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59a5be3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b2698970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36562/0 (socket says 172.21.15.139:36562) 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59a5be3640 1 -- 172.21.15.139:0/1522216391 learned_addr learned my addr 172.21.15.139:0/1522216391 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b2650590 0x5609b267b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59a4be1640 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b267b050 0x5609b26a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59a5be3640 1 -- 172.21.15.139:0/1522216391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b267b050 msgr2=0x5609b26a9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59a5be3640 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b267b050 0x5609b26a9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59a5be3640 1 -- 172.21.15.139:0/1522216391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b2650590 msgr2=0x5609b267b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59a5be3640 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b2650590 0x5609b267b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59a5be3640 1 -- 172.21.15.139:0/1522216391 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609b23f4f50 con 0x5609b268cc60 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59a5be3640 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b2698970 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5998009010 tx=0x7f5998001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab86d5ac145384da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f597ffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f597ffff640 1 -- 172.21.15.139:0/1522216391 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599801a040 con 0x5609b268cc60 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f597ffff640 1 -- 172.21.15.139:0/1522216391 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f59980057d0 con 0x5609b268cc60 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59aa422740 1 -- 172.21.15.139:0/1522216391 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5998005070 con 0x5609b268cc60 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f597ffff640 1 -- 172.21.15.139:0/1522216391 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f599802c050 con 0x5609b268cc60 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59aa422740 10 client.7245 fetch_fsmap learned FSMap version 43 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f59aa422740 1 -- 172.21.15.139:0/1522216391 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x5609b22880b0 con 0x5609b268cc60 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.884+0000 7f597ffff640 1 -- 172.21.15.139:0/1522216391 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5998005070 con 0x5609b268cc60 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.885+0000 7f597ffff640 1 -- 172.21.15.139:0/1522216391 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f599802f020 con 0x5609b268cc60 2024-04-25T09:58:08.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.885+0000 7f59aa422740 10 client.7245 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.885+0000 7f59aa422740 10 client.7245 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.885+0000 7f59aa422740 20 client.7245 populate_metadata read hostname 'smithi139' 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.885+0000 7f59aa422740 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5609b267cf80 0x5609b267d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.885+0000 7f59aa422740 4 client.7245 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.885+0000 7f59aa422740 1 -- 172.21.15.139:0/1522216391 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609b2573e60 con 0x5609b267cf80 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.885+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5609b267cf80 0x5609b267d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.885+0000 7f597ffff640 10 client.7245 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.885+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5609b267cf80 0x5609b267d360 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8980aad4969e779 server_cookie=bc3896395d3da761 in_seq=0 out_seq=0 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f597ffff640 1 -- 172.21.15.139:0/1522216391 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5609b2573e60 con 0x5609b267cf80 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f597ffff640 10 client.7245 handle_command_reply: tid=0 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f59aa422740 1 client.7245 shutdown 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f59aa422740 1 -- 172.21.15.139:0/1522216391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 msgr2=0x5609b2698970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f59aa422740 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b2698970 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5998009010 tx=0x7f5998001510 comp rx=0 tx=0).stop 2024-04-25T09:58:08.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f59aa422740 1 -- 172.21.15.139:0/1522216391 shutdown_connections 2024-04-25T09:58:08.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f59aa422740 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5609b267cf80 0x5609b267d360 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f59aa422740 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b2698970 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f597ffff640 0 client.7245 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:08.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f59aa422740 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b267b050 0x5609b26a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f59aa422740 1 --2- 172.21.15.139:0/1522216391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b2650590 0x5609b267b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f59aa422740 1 -- 172.21.15.139:0/1522216391 >> 172.21.15.139:0/1522216391 conn(0x5609b26186e0 msgr2=0x5609b264ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f59aa422740 1 -- 172.21.15.139:0/1522216391 shutdown_connections 2024-04-25T09:58:08.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f59aa422740 1 -- 172.21.15.139:0/1522216391 wait complete. 2024-04-25T09:58:08.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.886+0000 7f59aa422740 20 client.7245 trim_cache size 0 max 16384 2024-04-25T09:58:08.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.898+0000 7f59aa422740 1 Processor -- start 2024-04-25T09:58:08.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.898+0000 7f59aa422740 1 -- start start 2024-04-25T09:58:08.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.898+0000 7f59aa422740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b2650590 0x5609b2698320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.898+0000 7f59aa422740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b267b050 0x5609b2698860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.898+0000 7f59aa422740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b26908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.898+0000 7f59aa422740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01ffdf0 con 0x5609b268cc60 2024-04-25T09:58:08.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.898+0000 7f59aa422740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00c9cb0 con 0x5609b267b050 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.898+0000 7f59aa422740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01ff7f0 con 0x5609b2650590 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.898+0000 7f59a5be3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b26908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.898+0000 7f59a5be3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b26908e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36574/0 (socket says 172.21.15.139:36574) 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.898+0000 7f59a5be3640 1 -- 172.21.15.139:0/607476787 learned_addr learned my addr 172.21.15.139:0/607476787 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f59a4be1640 1 --2- 172.21.15.139:0/607476787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b267b050 0x5609b2698860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f59a53e2640 1 --2- 172.21.15.139:0/607476787 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b2650590 0x5609b2698320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f59a5be3640 1 -- 172.21.15.139:0/607476787 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b2650590 msgr2=0x5609b2698320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f59a5be3640 1 --2- 172.21.15.139:0/607476787 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b2650590 0x5609b2698320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f59a5be3640 1 -- 172.21.15.139:0/607476787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b267b050 msgr2=0x5609b2698860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f59a5be3640 1 --2- 172.21.15.139:0/607476787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b267b050 0x5609b2698860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f59a5be3640 1 -- 172.21.15.139:0/607476787 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609b2678d60 con 0x5609b268cc60 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f59a53e2640 1 --2- 172.21.15.139:0/607476787 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b2650590 0x5609b2698320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f59a5be3640 1 --2- 172.21.15.139:0/607476787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b26908e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5998003b60 tx=0x7f599801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72044c762860394e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f598a7fc640 1 -- 172.21.15.139:0/607476787 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599801a020 con 0x5609b268cc60 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f598a7fc640 1 -- 172.21.15.139:0/607476787 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5998024070 con 0x5609b268cc60 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f598a7fc640 1 -- 172.21.15.139:0/607476787 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998005650 con 0x5609b268cc60 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f59aa422740 1 -- 172.21.15.139:0/607476787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 msgr2=0x5609b26908e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f59aa422740 1 --2- 172.21.15.139:0/607476787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b26908e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5998003b60 tx=0x7f599801e000 comp rx=0 tx=0).stop 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.899+0000 7f59aa422740 1 -- 172.21.15.139:0/607476787 shutdown_connections 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.900+0000 7f59aa422740 1 --2- 172.21.15.139:0/607476787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b26908e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.900+0000 7f59aa422740 1 --2- 172.21.15.139:0/607476787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b267b050 0x5609b2698860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.900+0000 7f59aa422740 1 --2- 172.21.15.139:0/607476787 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b2650590 0x5609b2698320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.900+0000 7f59aa422740 1 -- 172.21.15.139:0/607476787 >> 172.21.15.139:0/607476787 conn(0x5609b26186e0 msgr2=0x5609b2674380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.900+0000 7f59aa422740 1 -- 172.21.15.139:0/607476787 shutdown_connections 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.900+0000 7f59aa422740 1 -- 172.21.15.139:0/607476787 wait complete. 2024-04-25T09:58:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.900+0000 7f59aa422740 1 -- start start 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.900+0000 7f59aa422740 1 Processor -- start 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59aa422740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b2650590 0x5609b26982f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59aa422740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b267b050 0x5609b2698830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59aa422740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b2704690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59aa422740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b2691860 con 0x5609b268cc60 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59aa422740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b267e000 con 0x5609b267b050 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59aa422740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609b26a99f0 con 0x5609b2650590 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59aa422740 10 client.0 _ll_register_callbacks cb 0x7f5998018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59a5be3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b2704690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59a5be3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b2704690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36588/0 (socket says 172.21.15.139:36588) 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59a5be3640 1 -- 172.21.15.139:0/1152074047 learned_addr learned my addr 172.21.15.139:0/1152074047 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b2650590 0x5609b26982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59a4be1640 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b267b050 0x5609b2698830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59a5be3640 1 -- 172.21.15.139:0/1152074047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b2650590 msgr2=0x5609b26982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59a5be3640 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b2650590 0x5609b26982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59a5be3640 1 -- 172.21.15.139:0/1152074047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b267b050 msgr2=0x5609b2698830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59a5be3640 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b267b050 0x5609b2698830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.901+0000 7f59a5be3640 1 -- 172.21.15.139:0/1152074047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609b26ea1f0 con 0x5609b268cc60 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f59a5be3640 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b2704690 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5998016040 tx=0x7f599800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4466ef20746b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f597ffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f597ffff640 1 -- 172.21.15.139:0/1152074047 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998041060 con 0x5609b268cc60 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f597ffff640 1 -- 172.21.15.139:0/1152074047 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f599801fd80 con 0x5609b268cc60 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f59aa422740 1 -- 172.21.15.139:0/1152074047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5998024070 con 0x5609b268cc60 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f597ffff640 1 -- 172.21.15.139:0/1152074047 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5998047050 con 0x5609b268cc60 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f59aa422740 10 client.7251 fetch_fsmap learned FSMap version 43 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f59aa422740 1 -- 172.21.15.139:0/1152074047 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x5609b2697110 con 0x5609b268cc60 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f597ffff640 1 -- 172.21.15.139:0/1152074047 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5998024070 con 0x5609b268cc60 2024-04-25T09:58:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f597ffff640 1 -- 172.21.15.139:0/1152074047 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f599804a020 con 0x5609b268cc60 2024-04-25T09:58:08.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f59aa422740 10 client.7251 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:58:08.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f59aa422740 10 client.7251 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:58:08.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f59aa422740 20 client.7251 populate_metadata read hostname 'smithi139' 2024-04-25T09:58:08.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f59aa422740 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5609b2704bd0 0x5609b268b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:08.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f59aa422740 4 client.7251 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T09:58:08.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f59aa422740 1 -- 172.21.15.139:0/1152074047 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5609b25c11a0 con 0x5609b2704bd0 2024-04-25T09:58:08.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.902+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5609b2704bd0 0x5609b268b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:08.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f597ffff640 10 client.7251 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:08.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59a53e2640 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5609b2704bd0 0x5609b268b9e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebd2ceab8ab5441b server_cookie=1517b3e1b9e6fc9e in_seq=0 out_seq=0 2024-04-25T09:58:08.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f597ffff640 1 -- 172.21.15.139:0/1152074047 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+237 (crc 0 0 0) 0x5609b25c11a0 con 0x5609b2704bd0 2024-04-25T09:58:08.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f597ffff640 10 client.7251 handle_command_reply: tid=0 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59aa422740 1 client.7251 shutdown 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59aa422740 1 -- 172.21.15.139:0/1152074047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 msgr2=0x5609b2704690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59aa422740 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b2704690 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5998016040 tx=0x7f599800c000 comp rx=0 tx=0).stop 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59aa422740 1 -- 172.21.15.139:0/1152074047 shutdown_connections 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59aa422740 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5609b2704bd0 0x5609b268b9e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59aa422740 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5609b268cc60 0x5609b2704690 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f597ffff640 0 client.7251 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59aa422740 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609b267b050 0x5609b2698830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59aa422740 1 --2- 172.21.15.139:0/1152074047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5609b2650590 0x5609b26982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59aa422740 1 -- 172.21.15.139:0/1152074047 >> 172.21.15.139:0/1152074047 conn(0x5609b26186e0 msgr2=0x5609b2691c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59aa422740 1 -- 172.21.15.139:0/1152074047 shutdown_connections 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59aa422740 1 -- 172.21.15.139:0/1152074047 wait complete. 2024-04-25T09:58:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.903+0000 7f59aa422740 20 client.7251 trim_cache size 0 max 16384 2024-04-25T09:58:08.915 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:08.915 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-04-25T09:58:08.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:08.915 INFO:teuthology.orchestra.run.smithi139.stdout: "damage_type": "dentry", 2024-04-25T09:58:08.915 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 2347612915, 2024-04-25T09:58:08.915 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 1099511627776, 2024-04-25T09:58:08.916 INFO:teuthology.orchestra.run.smithi139.stdout: "frag": "*", 2024-04-25T09:58:08.916 INFO:teuthology.orchestra.run.smithi139.stdout: "dname": "file_to_be_damaged", 2024-04-25T09:58:08.916 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_id": "head", 2024-04-25T09:58:08.916 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-25T09:58:08.916 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:08.916 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-04-25T09:58:08.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.904+0000 7f59a7646640 1 -- 172.21.15.139:0/1059714519 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f598406b4d0 msgr2=0x7f598406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.904+0000 7f59a7646640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f598406b4d0 0x7f598406d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f59a0158410 tx=0x7f5990008040 comp rx=0 tx=0).stop 2024-04-25T09:58:08.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.904+0000 7f59a7646640 1 -- 172.21.15.139:0/1059714519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 msgr2=0x7f59a014ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:08.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.904+0000 7f59a7646640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 0x7f59a014ebe0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f599c034040 tx=0x7f59a00d1e50 comp rx=0 tx=0).stop 2024-04-25T09:58:08.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.904+0000 7f59a7646640 1 -- 172.21.15.139:0/1059714519 shutdown_connections 2024-04-25T09:58:08.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.904+0000 7f59a7646640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f598406b4d0 0x7f598406d990 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.904+0000 7f59a7646640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59a00cc580 0x7f59a00cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.904+0000 7f59a7646640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59a0160ee0 0x7f59a015e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.904+0000 7f59a7646640 1 --2- 172.21.15.139:0/1059714519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f598c0039b0 0x7f59a014ebe0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:08.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.904+0000 7f59a7646640 1 -- 172.21.15.139:0/1059714519 >> 172.21.15.139:0/1059714519 conn(0x7f59a00b5480 msgr2=0x7f59a00caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:08.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.904+0000 7f59a7646640 1 -- 172.21.15.139:0/1059714519 shutdown_connections 2024-04-25T09:58:08.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:08.904+0000 7f59a7646640 1 -- 172.21.15.139:0/1059714519 wait complete. 2024-04-25T09:58:08.919 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T09:58:09.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.010+0000 7f3a42353640 1 Processor -- start 2024-04-25T09:58:09.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.011+0000 7f3a42353640 1 -- start start 2024-04-25T09:58:09.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.011+0000 7f3a42353640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c0b37d0 0x7f3a3c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.011+0000 7f3a42353640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3c0b3e10 con 0x7f3a3c158260 2024-04-25T09:58:09.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.011+0000 7f3a42353640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3c1615c0 con 0x7f3a3c06da20 2024-04-25T09:58:09.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.011+0000 7f3a42353640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c1618a0 con 0x7f3a3c0b37d0 2024-04-25T09:58:09.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.012+0000 7f3a3bfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3a3c158260 0x7f3a3c0b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:50828/0 (socket says 172.21.15.139:50828) 2024-04-25T09:58:09.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.012+0000 7f3a3bfff640 1 -- 172.21.15.139:0/6927005 learned_addr learned my addr 172.21.15.139:0/6927005 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:09.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.012+0000 7f3a3b7fe640 1 --2- 172.21.15.139:0/6927005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c0b37d0 0x7f3a3c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.012+0000 7f3a3affd640 1 -- 172.21.15.139:0/6927005 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335627498 0 0) 0x7f3a2c001110 con 0x7f3a3c158260 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.012+0000 7f3a3affd640 1 -- 172.21.15.139:0/6927005 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a24003520 con 0x7f3a3c158260 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.012+0000 7f3a3affd640 1 -- 172.21.15.139:0/6927005 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940280838 0 0) 0x7f3a3c1615c0 con 0x7f3a3c06da20 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.012+0000 7f3a3affd640 1 -- 172.21.15.139:0/6927005 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a2c001110 con 0x7f3a3c06da20 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.012+0000 7f3a3b7fe640 1 -- 172.21.15.139:0/6927005 >> v1:172.21.15.150:6789/0 conn(0x7f3a3c06da20 legacy=0x7f3a3c06ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.012+0000 7f3a3b7fe640 1 -- 172.21.15.139:0/6927005 >> v1:172.21.15.139:6789/0 conn(0x7f3a3c158260 legacy=0x7f3a3c0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.012+0000 7f3a3b7fe640 1 -- 172.21.15.139:0/6927005 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c161b80 con 0x7f3a3c0b37d0 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.013+0000 7f3a3b7fe640 1 --2- 172.21.15.139:0/6927005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c0b37d0 0x7f3a3c15df80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3a300097d0 tx=0x7f3a3002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16dcb1818aa400e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.013+0000 7f3a3affd640 1 -- 172.21.15.139:0/6927005 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a30037070 con 0x7f3a3c0b37d0 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.013+0000 7f3a3affd640 1 -- 172.21.15.139:0/6927005 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a30032440 con 0x7f3a3c0b37d0 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.013+0000 7f3a3affd640 1 -- 172.21.15.139:0/6927005 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300327f0 con 0x7f3a3c0b37d0 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.013+0000 7f3a42353640 1 -- 172.21.15.139:0/6927005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c0b37d0 msgr2=0x7f3a3c15df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.013+0000 7f3a42353640 1 --2- 172.21.15.139:0/6927005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c0b37d0 0x7f3a3c15df80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3a300097d0 tx=0x7f3a3002a210 comp rx=0 tx=0).stop 2024-04-25T09:58:09.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.013+0000 7f3a42353640 1 -- 172.21.15.139:0/6927005 shutdown_connections 2024-04-25T09:58:09.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.013+0000 7f3a42353640 1 --2- 172.21.15.139:0/6927005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c0b37d0 0x7f3a3c15df80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.013+0000 7f3a42353640 1 -- 172.21.15.139:0/6927005 >> 172.21.15.139:0/6927005 conn(0x7f3a3c0b23d0 msgr2=0x7f3a3c0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.014+0000 7f3a42353640 1 -- 172.21.15.139:0/6927005 shutdown_connections 2024-04-25T09:58:09.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.014+0000 7f3a42353640 1 -- 172.21.15.139:0/6927005 wait complete. 2024-04-25T09:58:09.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.015+0000 7f3a42353640 1 Processor -- start 2024-04-25T09:58:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.015+0000 7f3a42353640 1 -- start start 2024-04-25T09:58:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a42353640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a3c06da20 0x7f3a3c1445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a42353640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3c0b37d0 0x7f3a3c144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a42353640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c158260 0x7f3a3c145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a42353640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c162420 con 0x7f3a3c06da20 2024-04-25T09:58:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a42353640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c162140 con 0x7f3a3c0b37d0 2024-04-25T09:58:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a42353640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c06b4b0 con 0x7f3a3c158260 2024-04-25T09:58:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a3bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a3c06da20 0x7f3a3c1445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a3bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a3c06da20 0x7f3a3c1445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36598/0 (socket says 172.21.15.139:36598) 2024-04-25T09:58:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a3bfff640 1 -- 172.21.15.139:0/2767716415 learned_addr learned my addr 172.21.15.139:0/2767716415 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a3b7fe640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3c0b37d0 0x7f3a3c144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a408f0640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c158260 0x7f3a3c145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a3bfff640 1 -- 172.21.15.139:0/2767716415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c158260 msgr2=0x7f3a3c145060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a3bfff640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c158260 0x7f3a3c145060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a3bfff640 1 -- 172.21.15.139:0/2767716415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3c0b37d0 msgr2=0x7f3a3c144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a3bfff640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3c0b37d0 0x7f3a3c144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a3bfff640 1 -- 172.21.15.139:0/2767716415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c1f37d0 con 0x7f3a3c06da20 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a408f0640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c158260 0x7f3a3c145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a3bfff640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a3c06da20 0x7f3a3c1445e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00c730 tx=0x7f3a2c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a38ff9640 1 -- 172.21.15.139:0/2767716415 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c017070 con 0x7f3a3c06da20 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a38ff9640 1 -- 172.21.15.139:0/2767716415 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2c005ab0 con 0x7f3a3c06da20 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a42353640 1 -- 172.21.15.139:0/2767716415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c0ceae0 con 0x7f3a3c06da20 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a38ff9640 1 -- 172.21.15.139:0/2767716415 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c007070 con 0x7f3a3c06da20 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.016+0000 7f3a42353640 1 -- 172.21.15.139:0/2767716415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3c0cf0a0 con 0x7f3a3c06da20 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.017+0000 7f3a42353640 1 -- 172.21.15.139:0/2767716415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3c0cf2e0 con 0x7f3a3c06da20 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.018+0000 7f3a38ff9640 1 -- 172.21.15.139:0/2767716415 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a2c01a070 con 0x7f3a3c06da20 2024-04-25T09:58:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.018+0000 7f3a38ff9640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a2806b4d0 0x7f3a2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.018+0000 7f3a38ff9640 1 -- 172.21.15.139:0/2767716415 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3a2c089840 con 0x7f3a3c06da20 2024-04-25T09:58:09.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.018+0000 7f3a3b7fe640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a2806b4d0 0x7f3a2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.019+0000 7f3a3b7fe640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a2806b4d0 0x7f3a2806d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3a30034000 tx=0x7f3a3002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.023+0000 7f3a38ff9640 1 -- 172.21.15.139:0/2767716415 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a3c0cf2e0 con 0x7f3a3c06da20 2024-04-25T09:58:09.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.234+0000 7f3a42353640 1 -- 172.21.15.139:0/2767716415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3a3c0cf510 con 0x7f3a3c06da20 2024-04-25T09:58:09.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.234+0000 7f3a38ff9640 1 -- 172.21.15.139:0/2767716415 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f3a3c1f37d0 con 0x7f3a3c06da20 2024-04-25T09:58:09.238 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:09.238 INFO:teuthology.orchestra.run.smithi139.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T09:58:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.236+0000 7f3a42353640 1 -- 172.21.15.139:0/2767716415 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a2806b4d0 msgr2=0x7f3a2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.236+0000 7f3a42353640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a2806b4d0 0x7f3a2806d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3a30034000 tx=0x7f3a3002e000 comp rx=0 tx=0).stop 2024-04-25T09:58:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.236+0000 7f3a42353640 1 -- 172.21.15.139:0/2767716415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a3c06da20 msgr2=0x7f3a3c1445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.236+0000 7f3a42353640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a3c06da20 0x7f3a3c1445e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00c730 tx=0x7f3a2c00cbf0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.236+0000 7f3a42353640 1 -- 172.21.15.139:0/2767716415 shutdown_connections 2024-04-25T09:58:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.236+0000 7f3a42353640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3a2806b4d0 0x7f3a2806d990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.236+0000 7f3a42353640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c158260 0x7f3a3c145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.236+0000 7f3a42353640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3c0b37d0 0x7f3a3c144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.236+0000 7f3a42353640 1 --2- 172.21.15.139:0/2767716415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3a3c06da20 0x7f3a3c1445e0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.236+0000 7f3a42353640 1 -- 172.21.15.139:0/2767716415 >> 172.21.15.139:0/2767716415 conn(0x7f3a3c0b23d0 msgr2=0x7f3a3c0b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.236+0000 7f3a42353640 1 -- 172.21.15.139:0/2767716415 shutdown_connections 2024-04-25T09:58:09.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.236+0000 7f3a42353640 1 -- 172.21.15.139:0/2767716415 wait complete. 2024-04-25T09:58:09.250 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T09:58:09.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:09.251 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-25T09:58:09.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.347+0000 7f2ff737e640 1 Processor -- start 2024-04-25T09:58:09.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.347+0000 7f2ff737e640 1 -- start start 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.348+0000 7f2ff737e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015dbf0 0x7f2ff01617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.348+0000 7f2ff737e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2ff00b9370 0x7f2ff00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.348+0000 7f2ff737e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff0161dc0 con 0x7f2ff00cec80 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.348+0000 7f2ff737e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff00d1ee0 con 0x7f2ff00b9370 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.348+0000 7f2ff737e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff01620a0 con 0x7f2ff015dbf0 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.348+0000 7f2ff591b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2ff00cec80 0x7f2ff006c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:50834/0 (socket says 172.21.15.139:50834) 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.348+0000 7f2ff591b640 1 -- 172.21.15.139:0/1257367774 learned_addr learned my addr 172.21.15.139:0/1257367774 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2ff511a640 1 --2- 172.21.15.139:0/1257367774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015dbf0 0x7f2ff01617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2ff4919640 1 --2- 172.21.15.139:0/1257367774 >> v2:172.21.15.150:3300/0 conn(0x7f2ff00b9370 0x7f2ff00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321117417 0 0) 0x7f2ff0161dc0 con 0x7f2ff00cec80 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fdc003520 con 0x7f2ff00cec80 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3092859605 0 0) 0x7f2fdc003520 con 0x7f2ff00cec80 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015dbf0 msgr2=0x7f2ff01617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 --2- 172.21.15.139:0/1257367774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015dbf0 0x7f2ff01617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 >> v2:172.21.15.150:3300/0 conn(0x7f2ff00b9370 msgr2=0x7f2ff00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 --2- 172.21.15.139:0/1257367774 >> v2:172.21.15.150:3300/0 conn(0x7f2ff00b9370 0x7f2ff00ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff0162380 con 0x7f2ff00cec80 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f2fe8001670 con 0x7f2ff00cec80 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 >> v1:172.21.15.139:6789/0 conn(0x7f2ff00cec80 legacy=0x7f2ff006c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 --2- 172.21.15.139:0/1257367774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015dbf0 0x7f2fdc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 --2- 172.21.15.139:0/1257367774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fdc003f30 0x7f2fdc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 --2- 172.21.15.139:0/1257367774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fdc004d40 0x7f2fdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00d1ee0 con 0x7f2fdc004d40 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff01620a0 con 0x7f2fdc003f30 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc00b9c0 con 0x7f2ff015dbf0 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2ff4919640 1 --2- 172.21.15.139:0/1257367774 >> v2:172.21.15.150:3300/0 conn(0x7f2ff00b9370 0x7f2ff00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2ff591b640 1 --2- 172.21.15.139:0/1257367774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fdc004d40 0x7f2fdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2ff511a640 1 --2- 172.21.15.139:0/1257367774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015dbf0 0x7f2fdc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2ff4919640 1 --2- 172.21.15.139:0/1257367774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fdc003f30 0x7f2fdc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2ff591b640 1 -- 172.21.15.139:0/1257367774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015dbf0 msgr2=0x7f2fdc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2ff591b640 1 --2- 172.21.15.139:0/1257367774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015dbf0 0x7f2fdc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2ff591b640 1 -- 172.21.15.139:0/1257367774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fdc003f30 msgr2=0x7f2fdc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2ff591b640 1 --2- 172.21.15.139:0/1257367774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fdc003f30 0x7f2fdc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.349+0000 7f2ff591b640 1 -- 172.21.15.139:0/1257367774 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc003520 con 0x7f2fdc004d40 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.350+0000 7f2ff591b640 1 --2- 172.21.15.139:0/1257367774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fdc004d40 0x7f2fdc0051c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c210 tx=0x7f2fdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81af0e001101161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.350+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8007d50 con 0x7f2fdc004d40 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.350+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe802e070 con 0x7f2fdc004d40 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.350+0000 7f2fe7fff640 1 -- 172.21.15.139:0/1257367774 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8005d30 con 0x7f2fdc004d40 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.351+0000 7f2ff737e640 1 -- 172.21.15.139:0/1257367774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fdc004d40 msgr2=0x7f2fdc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.351+0000 7f2ff737e640 1 --2- 172.21.15.139:0/1257367774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fdc004d40 0x7f2fdc0051c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c210 tx=0x7f2fdc005830 comp rx=0 tx=0).stop 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.351+0000 7f2ff591b640 1 -- 172.21.15.139:0/1257367774 reap_dead start 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.351+0000 7f2ff737e640 1 -- 172.21.15.139:0/1257367774 shutdown_connections 2024-04-25T09:58:09.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.351+0000 7f2ff737e640 1 -- 172.21.15.139:0/1257367774 >> 172.21.15.139:0/1257367774 conn(0x7f2ff00b5480 msgr2=0x7f2ff00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.351+0000 7f2ff737e640 1 -- 172.21.15.139:0/1257367774 shutdown_connections 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.351+0000 7f2ff737e640 1 -- 172.21.15.139:0/1257367774 wait complete. 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.352+0000 7f2ff737e640 1 Processor -- start 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.352+0000 7f2ff737e640 1 -- start start 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff737e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fdc004d40 0x7f2ff00b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff737e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ff01604b0 0x7f2ff0160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff737e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015af20 0x7f2ff015b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff737e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff015dfd0 con 0x7f2ff01604b0 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff737e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00b97d0 con 0x7f2fdc004d40 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff737e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff01f3380 con 0x7f2ff015af20 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff4919640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ff01604b0 0x7f2ff0160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff4919640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ff01604b0 0x7f2ff0160930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36624/0 (socket says 172.21.15.139:36624) 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff4919640 1 -- 172.21.15.139:0/2508373726 learned_addr learned my addr 172.21.15.139:0/2508373726 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff511a640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fdc004d40 0x7f2ff00b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff591b640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015af20 0x7f2ff015b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff4919640 1 -- 172.21.15.139:0/2508373726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015af20 msgr2=0x7f2ff015b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff4919640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015af20 0x7f2ff015b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff4919640 1 -- 172.21.15.139:0/2508373726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fdc004d40 msgr2=0x7f2ff00b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff4919640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fdc004d40 0x7f2ff00b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff4919640 1 -- 172.21.15.139:0/2508373726 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff015bc50 con 0x7f2ff01604b0 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff4919640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ff01604b0 0x7f2ff0160930 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0011930 tx=0x7f2fe0011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2fe5ffb640 1 -- 172.21.15.139:0/2508373726 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe000fbd0 con 0x7f2ff01604b0 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2fe5ffb640 1 -- 172.21.15.139:0/2508373726 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe000fd70 con 0x7f2ff01604b0 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2ff737e640 1 -- 172.21.15.139:0/2508373726 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff015bf30 con 0x7f2ff01604b0 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.353+0000 7f2fe5ffb640 1 -- 172.21.15.139:0/2508373726 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0015660 con 0x7f2ff01604b0 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.354+0000 7f2ff737e640 1 -- 172.21.15.139:0/2508373726 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff015c4f0 con 0x7f2ff01604b0 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.355+0000 7f2fe5ffb640 1 -- 172.21.15.139:0/2508373726 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2fe0015800 con 0x7f2ff01604b0 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.355+0000 7f2fe5ffb640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2fd406b4d0 0x7f2fd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.356+0000 7f2fe5ffb640 1 -- 172.21.15.139:0/2508373726 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2fe008ae50 con 0x7f2ff01604b0 2024-04-25T09:58:09.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.356+0000 7f2ff511a640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2fd406b4d0 0x7f2fd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.356+0000 7f2ff511a640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2fd406b4d0 0x7f2fd406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2fec014510 tx=0x7f2fec00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.356+0000 7f2ffa084740 1 Processor -- start 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.356+0000 7f2ffa084740 1 -- start start 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ffa084740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7acf70 0x55bbcb7ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ffa084740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 0x55bbcb77cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ffa084740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb77d420 0x55bbcb7a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ffa084740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbcb6e5c90 con 0x55bbcb77d420 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ffa084740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbcb68ef50 con 0x55bbcb7ad990 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ffa084740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bbcb6e1d30 con 0x55bbcb7acf70 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ff591b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb77d420 0x55bbcb7a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ff591b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb77d420 0x55bbcb7a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36626/0 (socket says 172.21.15.139:36626) 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ff591b640 1 -- 172.21.15.139:0/1994380054 learned_addr learned my addr 172.21.15.139:0/1994380054 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ff591b640 1 -- 172.21.15.139:0/1994380054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7acf70 msgr2=0x55bbcb7ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ff591b640 1 --2- 172.21.15.139:0/1994380054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7acf70 0x55bbcb7ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ff591b640 1 -- 172.21.15.139:0/1994380054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 msgr2=0x55bbcb77cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ff4919640 1 --2- 172.21.15.139:0/1994380054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 0x55bbcb77cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.357+0000 7f2ff591b640 1 --2- 172.21.15.139:0/1994380054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 0x55bbcb77cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2ff591b640 1 -- 172.21.15.139:0/1994380054 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbcb4c2a50 con 0x55bbcb77d420 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2ff591b640 1 --2- 172.21.15.139:0/1994380054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb77d420 0x55bbcb7a53f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c330 tx=0x7f2fe8009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca92728b7f16708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2fdaffd640 1 -- 172.21.15.139:0/1994380054 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8045070 con 0x55bbcb77d420 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2fdaffd640 1 -- 172.21.15.139:0/1994380054 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe8001630 con 0x55bbcb77d420 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2fdaffd640 1 -- 172.21.15.139:0/1994380054 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80049e0 con 0x55bbcb77d420 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2ffa084740 1 -- 172.21.15.139:0/1994380054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb77d420 msgr2=0x55bbcb7a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2ffa084740 1 --2- 172.21.15.139:0/1994380054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb77d420 0x55bbcb7a53f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c330 tx=0x7f2fe8009030 comp rx=0 tx=0).stop 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2ffa084740 1 -- 172.21.15.139:0/1994380054 shutdown_connections 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2ffa084740 1 --2- 172.21.15.139:0/1994380054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb77d420 0x55bbcb7a53f0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2ffa084740 1 --2- 172.21.15.139:0/1994380054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 0x55bbcb77cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2ffa084740 1 --2- 172.21.15.139:0/1994380054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7acf70 0x55bbcb7ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2ffa084740 1 -- 172.21.15.139:0/1994380054 >> 172.21.15.139:0/1994380054 conn(0x55bbcb77f360 msgr2=0x55bbcb7c7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2ffa084740 1 -- 172.21.15.139:0/1994380054 shutdown_connections 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.358+0000 7f2ffa084740 1 -- 172.21.15.139:0/1994380054 wait complete. 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.359+0000 7f2ffa084740 1 -- start start 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.359+0000 7f2ffa084740 1 Processor -- start 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ffa084740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 0x55bbcb7cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ffa084740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 0x55bbcb7b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ffa084740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7ccb60 0x55bbcb7ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ffa084740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbcb7a96c0 con 0x55bbcb7acf70 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ffa084740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbcb684700 con 0x55bbcb7ad990 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ffa084740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bbcb7a9960 con 0x55bbcb7ccb60 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ffa084740 10 client.0 _ll_register_callbacks cb 0x55bbcb6dc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ff511a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 0x55bbcb7cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ff511a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 0x55bbcb7cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36630/0 (socket says 172.21.15.139:36630) 2024-04-25T09:58:09.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ff511a640 1 -- 172.21.15.139:0/3658331286 learned_addr learned my addr 172.21.15.139:0/3658331286 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ff511a640 1 -- 172.21.15.139:0/3658331286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7ccb60 msgr2=0x55bbcb7ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ff4919640 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 0x55bbcb7b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ff591b640 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7ccb60 0x55bbcb7ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ff511a640 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7ccb60 0x55bbcb7ae5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ff511a640 1 -- 172.21.15.139:0/3658331286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 msgr2=0x55bbcb7b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ff511a640 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 0x55bbcb7b0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.360+0000 7f2ff511a640 1 -- 172.21.15.139:0/3658331286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbcb3ba3e0 con 0x55bbcb7acf70 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2ff511a640 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 0x55bbcb7cf9e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2fec016ef0 tx=0x7f2fec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcf2af36ce0666f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2fcffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2fcffff640 1 -- 172.21.15.139:0/3658331286 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec026070 con 0x55bbcb7acf70 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2fcffff640 1 -- 172.21.15.139:0/3658331286 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2fec02b660 con 0x55bbcb7acf70 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2ffa084740 1 -- 172.21.15.139:0/3658331286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fe8001630 con 0x55bbcb7acf70 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2fcffff640 1 -- 172.21.15.139:0/3658331286 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fec02b860 con 0x55bbcb7acf70 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2ffa084740 10 client.7269 fetch_fsmap learned FSMap version 43 2024-04-25T09:58:09.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2ffa084740 1 -- 172.21.15.139:0/3658331286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x55bbcb5231e0 con 0x55bbcb7acf70 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2fcffff640 1 -- 172.21.15.139:0/3658331286 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe8001630 con 0x55bbcb7acf70 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2fcffff640 1 -- 172.21.15.139:0/3658331286 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2fec036080 con 0x55bbcb7acf70 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2ffa084740 10 client.7269 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2ffa084740 10 client.7269 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2ffa084740 20 client.7269 populate_metadata read hostname 'smithi139' 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2ffa084740 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55bbcb7a31f0 0x55bbcb7a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2ffa084740 4 client.7269 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2ffa084740 1 -- 172.21.15.139:0/3658331286 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbcb6f31d0 con 0x55bbcb7a31f0 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2ff4919640 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55bbcb7a31f0 0x55bbcb7a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.361+0000 7f2fcffff640 10 client.7269 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2ff4919640 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55bbcb7a31f0 0x55bbcb7a35d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8edcf85a5f2599e server_cookie=24cd344625119748 in_seq=0 out_seq=0 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2fcffff640 1 -- 172.21.15.139:0/3658331286 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bbcb6f31d0 con 0x55bbcb7a31f0 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2fcffff640 10 client.7269 handle_command_reply: tid=0 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2ffa084740 1 client.7269 shutdown 2024-04-25T09:58:09.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2ffa084740 1 -- 172.21.15.139:0/3658331286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 msgr2=0x55bbcb7cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2ffa084740 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 0x55bbcb7cf9e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2fec016ef0 tx=0x7f2fec009750 comp rx=0 tx=0).stop 2024-04-25T09:58:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2ffa084740 1 -- 172.21.15.139:0/3658331286 shutdown_connections 2024-04-25T09:58:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2ffa084740 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55bbcb7a31f0 0x55bbcb7a35d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2ffa084740 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7ccb60 0x55bbcb7ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2ffa084740 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 0x55bbcb7b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2fcffff640 0 client.7269 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2ffa084740 1 --2- 172.21.15.139:0/3658331286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 0x55bbcb7cf9e0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.362+0000 7f2ffa084740 1 -- 172.21.15.139:0/3658331286 >> 172.21.15.139:0/3658331286 conn(0x55bbcb77d420 msgr2=0x55bbcb7c7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.363+0000 7f2ffa084740 1 -- 172.21.15.139:0/3658331286 shutdown_connections 2024-04-25T09:58:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.363+0000 7f2ffa084740 1 -- 172.21.15.139:0/3658331286 wait complete. 2024-04-25T09:58:09.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.363+0000 7f2ffa084740 20 client.7269 trim_cache size 0 max 16384 2024-04-25T09:58:09.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.378+0000 7f2ffa084740 1 Processor -- start 2024-04-25T09:58:09.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.378+0000 7f2ffa084740 1 -- start start 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.378+0000 7f2ffa084740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 0x55bbcb7add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.378+0000 7f2ffa084740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 0x55bbcb7834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.378+0000 7f2ffa084740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7ccb60 0x55bbcb7ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.378+0000 7f2ffa084740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0200a60 con 0x55bbcb7acf70 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.378+0000 7f2ffa084740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff01ffc10 con 0x55bbcb7ad990 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.378+0000 7f2ffa084740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff01f4880 con 0x55bbcb7ccb60 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.378+0000 7f2ff511a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 0x55bbcb7add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.378+0000 7f2ff511a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 0x55bbcb7add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36634/0 (socket says 172.21.15.139:36634) 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.378+0000 7f2ff511a640 1 -- 172.21.15.139:0/1690894885 learned_addr learned my addr 172.21.15.139:0/1690894885 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.378+0000 7f2ff4919640 1 --2- 172.21.15.139:0/1690894885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 0x55bbcb7834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.379+0000 7f2ff591b640 1 --2- 172.21.15.139:0/1690894885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7ccb60 0x55bbcb7ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.379+0000 7f2ff511a640 1 -- 172.21.15.139:0/1690894885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7ccb60 msgr2=0x55bbcb7ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.379+0000 7f2ff511a640 1 --2- 172.21.15.139:0/1690894885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7ccb60 0x55bbcb7ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.379+0000 7f2ff511a640 1 -- 172.21.15.139:0/1690894885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 msgr2=0x55bbcb7834e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.379+0000 7f2ff511a640 1 --2- 172.21.15.139:0/1690894885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 0x55bbcb7834e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.379+0000 7f2ff511a640 1 -- 172.21.15.139:0/1690894885 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbcb7a3940 con 0x55bbcb7acf70 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.379+0000 7f2ff511a640 1 --2- 172.21.15.139:0/1690894885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 0x55bbcb7add70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0143d0 tx=0x7f2fec01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b5c07788088a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.379+0000 7f2fda7fc640 1 -- 172.21.15.139:0/1690894885 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec01a040 con 0x55bbcb7acf70 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.379+0000 7f2fda7fc640 1 -- 172.21.15.139:0/1690894885 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec02baa0 con 0x55bbcb7acf70 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.379+0000 7f2fda7fc640 1 -- 172.21.15.139:0/1690894885 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec02b6c0 con 0x55bbcb7acf70 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.380+0000 7f2ffa084740 1 -- 172.21.15.139:0/1690894885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 msgr2=0x55bbcb7add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.380+0000 7f2ffa084740 1 --2- 172.21.15.139:0/1690894885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 0x55bbcb7add70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0143d0 tx=0x7f2fec01e000 comp rx=0 tx=0).stop 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.380+0000 7f2ffa084740 1 -- 172.21.15.139:0/1690894885 shutdown_connections 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.380+0000 7f2ffa084740 1 --2- 172.21.15.139:0/1690894885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7ccb60 0x55bbcb7ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.380+0000 7f2ffa084740 1 --2- 172.21.15.139:0/1690894885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ad990 0x55bbcb7834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.380+0000 7f2ffa084740 1 --2- 172.21.15.139:0/1690894885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7acf70 0x55bbcb7add70 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.380+0000 7f2ffa084740 1 -- 172.21.15.139:0/1690894885 >> 172.21.15.139:0/1690894885 conn(0x55bbcb77d420 msgr2=0x55bbcb7b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.380+0000 7f2ffa084740 1 -- 172.21.15.139:0/1690894885 shutdown_connections 2024-04-25T09:58:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.380+0000 7f2ffa084740 1 -- 172.21.15.139:0/1690894885 wait complete. 2024-04-25T09:58:09.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.380+0000 7f2ffa084740 1 -- start start 2024-04-25T09:58:09.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.380+0000 7f2ffa084740 1 Processor -- start 2024-04-25T09:58:09.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ffa084740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7acf70 0x55bbcb833fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ffa084740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7ad990 0x55bbcb7cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ffa084740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ccb60 0x55bbcb7c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ffa084740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbcb7c6e40 con 0x55bbcb7ad990 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ffa084740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff01ff610 con 0x55bbcb7ccb60 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ffa084740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bbcb6e8130 con 0x55bbcb7acf70 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ffa084740 10 client.0 _ll_register_callbacks cb 0x7f2fec0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ff4919640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7ad990 0x55bbcb7cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ff4919640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7ad990 0x55bbcb7cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36648/0 (socket says 172.21.15.139:36648) 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ff4919640 1 -- 172.21.15.139:0/2242571706 learned_addr learned my addr 172.21.15.139:0/2242571706 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ff591b640 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ccb60 0x55bbcb7c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ff511a640 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7acf70 0x55bbcb833fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ff4919640 1 -- 172.21.15.139:0/2242571706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7acf70 msgr2=0x55bbcb833fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ff4919640 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7acf70 0x55bbcb833fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ff4919640 1 -- 172.21.15.139:0/2242571706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ccb60 msgr2=0x55bbcb7c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ff4919640 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ccb60 0x55bbcb7c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.381+0000 7f2ff4919640 1 -- 172.21.15.139:0/2242571706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbcb3934a0 con 0x55bbcb7ad990 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2ff511a640 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7acf70 0x55bbcb833fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2ff4919640 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7ad990 0x55bbcb7cfd50 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000f5b0 tx=0x7f2fe005d600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f4c64c983c2b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2fcffff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2fcffff640 1 -- 172.21.15.139:0/2242571706 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0087060 con 0x55bbcb7ad990 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2fcffff640 1 -- 172.21.15.139:0/2242571706 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2fe0055b10 con 0x55bbcb7ad990 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2ffa084740 1 -- 172.21.15.139:0/2242571706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fec02baa0 con 0x55bbcb7ad990 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2fcffff640 1 -- 172.21.15.139:0/2242571706 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fe0077050 con 0x55bbcb7ad990 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2ffa084740 10 client.7275 fetch_fsmap learned FSMap version 43 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2ffa084740 1 -- 172.21.15.139:0/2242571706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x55bbcb833ab0 con 0x55bbcb7ad990 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2fcffff640 1 -- 172.21.15.139:0/2242571706 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe0072020 con 0x55bbcb7ad990 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2fcffff640 1 -- 172.21.15.139:0/2242571706 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2fe005d0b0 con 0x55bbcb7ad990 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2ffa084740 10 client.7275 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2ffa084740 10 client.7275 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2ffa084740 20 client.7275 populate_metadata read hostname 'smithi139' 2024-04-25T09:58:09.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2ffa084740 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55bbcb781260 0x55bbcb82d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2ffa084740 4 client.7275 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2ffa084740 1 -- 172.21.15.139:0/2242571706 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x55bbcb6ed520 con 0x55bbcb781260 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.382+0000 7f2ff511a640 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55bbcb781260 0x55bbcb82d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2ff511a640 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55bbcb781260 0x55bbcb82d080 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=548958903806e895 server_cookie=2ba0ba75dbe0f829 in_seq=0 out_seq=0 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2fcffff640 10 client.7275 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2fcffff640 1 -- 172.21.15.139:0/2242571706 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bbcb6ed520 con 0x55bbcb781260 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2fcffff640 10 client.7275 handle_command_reply: tid=0 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2ffa084740 1 client.7275 shutdown 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2ffa084740 1 -- 172.21.15.139:0/2242571706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7ad990 msgr2=0x55bbcb7cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2ffa084740 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7ad990 0x55bbcb7cfd50 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000f5b0 tx=0x7f2fe005d600 comp rx=0 tx=0).stop 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2ffa084740 1 -- 172.21.15.139:0/2242571706 shutdown_connections 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2ffa084740 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55bbcb781260 0x55bbcb82d080 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2ffa084740 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bbcb7ad990 0x55bbcb7cfd50 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2ffa084740 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbcb7ccb60 0x55bbcb7c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2fcffff640 0 client.7275 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:09.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2ffa084740 1 --2- 172.21.15.139:0/2242571706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bbcb7acf70 0x55bbcb833fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.383+0000 7f2ffa084740 1 -- 172.21.15.139:0/2242571706 >> 172.21.15.139:0/2242571706 conn(0x55bbcb77d420 msgr2=0x55bbcb780b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ffa084740 1 -- 172.21.15.139:0/2242571706 shutdown_connections 2024-04-25T09:58:09.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ffa084740 1 -- 172.21.15.139:0/2242571706 wait complete. 2024-04-25T09:58:09.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ffa084740 20 client.7275 trim_cache size 0 max 16384 2024-04-25T09:58:09.393 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:58:09.393 INFO:teuthology.orchestra.run.smithi139.stdout: "return_code": 0, 2024-04-25T09:58:09.393 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_tag": "8a620024-088e-4d27-8070-ea44464dba19", 2024-04-25T09:58:09.393 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": "asynchronous" 2024-04-25T09:58:09.393 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:58:09.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ff737e640 1 -- 172.21.15.139:0/2508373726 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2fd406b4d0 msgr2=0x7f2fd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ff737e640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2fd406b4d0 0x7f2fd406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2fec014510 tx=0x7f2fec00b040 comp rx=0 tx=0).stop 2024-04-25T09:58:09.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ff737e640 1 -- 172.21.15.139:0/2508373726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ff01604b0 msgr2=0x7f2ff0160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ff737e640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ff01604b0 0x7f2ff0160930 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0011930 tx=0x7f2fe0011df0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ff737e640 1 -- 172.21.15.139:0/2508373726 shutdown_connections 2024-04-25T09:58:09.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ff737e640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ff01604b0 0x7f2ff0160930 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ff737e640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2fd406b4d0 0x7f2fd406d990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ff737e640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff015af20 0x7f2ff015b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ff737e640 1 --2- 172.21.15.139:0/2508373726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fdc004d40 0x7f2ff00b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ff737e640 1 -- 172.21.15.139:0/2508373726 >> 172.21.15.139:0/2508373726 conn(0x7f2ff00b5480 msgr2=0x7f2ff00cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ff737e640 1 -- 172.21.15.139:0/2508373726 shutdown_connections 2024-04-25T09:58:09.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.384+0000 7f2ff737e640 1 -- 172.21.15.139:0/2508373726 wait complete. 2024-04-25T09:58:09.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:09.399 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T09:58:09.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.494+0000 7f738a85d640 1 Processor -- start 2024-04-25T09:58:09.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.494+0000 7f738a85d640 1 -- start start 2024-04-25T09:58:09.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.495+0000 7f738a85d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73840cd610 0x7f73840d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.495+0000 7f738a85d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7384099ca0 con 0x7f738409ad40 2024-04-25T09:58:09.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.495+0000 7f738a85d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7384099f80 con 0x7f7384161fd0 2024-04-25T09:58:09.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.495+0000 7f738a85d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7384099a50 con 0x7f73840cd610 2024-04-25T09:58:09.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.495+0000 7f73837fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f738409ad40 0x7f73841618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:50850/0 (socket says 172.21.15.139:50850) 2024-04-25T09:58:09.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.495+0000 7f73837fe640 1 -- 172.21.15.139:0/1426460948 learned_addr learned my addr 172.21.15.139:0/1426460948 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:09.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.495+0000 7f7383fff640 1 --2- 172.21.15.139:0/1426460948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73840cd610 0x7f73840d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.495+0000 7f7382ffd640 1 -- 172.21.15.139:0/1426460948 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691573086 0 0) 0x7f7384099ca0 con 0x7f738409ad40 2024-04-25T09:58:09.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.495+0000 7f7382ffd640 1 -- 172.21.15.139:0/1426460948 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f736c003520 con 0x7f738409ad40 2024-04-25T09:58:09.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.495+0000 7f7382ffd640 1 -- 172.21.15.139:0/1426460948 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158023759 0 0) 0x7f7384099f80 con 0x7f7384161fd0 2024-04-25T09:58:09.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.495+0000 7f7382ffd640 1 -- 172.21.15.139:0/1426460948 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7384099ca0 con 0x7f7384161fd0 2024-04-25T09:58:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.496+0000 7f7383fff640 1 -- 172.21.15.139:0/1426460948 >> v1:172.21.15.150:6789/0 conn(0x7f7384161fd0 legacy=0x7f73840cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.496+0000 7f7383fff640 1 -- 172.21.15.139:0/1426460948 >> v1:172.21.15.139:6789/0 conn(0x7f738409ad40 legacy=0x7f73841618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.496+0000 7f7383fff640 1 -- 172.21.15.139:0/1426460948 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738409a260 con 0x7f73840cd610 2024-04-25T09:58:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.496+0000 7f7383fff640 1 --2- 172.21.15.139:0/1426460948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73840cd610 0x7f73840d0370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73740097d0 tx=0x7f737402a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f4c5420bb1ead4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.496+0000 7f7382ffd640 1 -- 172.21.15.139:0/1426460948 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374037070 con 0x7f73840cd610 2024-04-25T09:58:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.496+0000 7f7382ffd640 1 -- 172.21.15.139:0/1426460948 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7374032440 con 0x7f73840cd610 2024-04-25T09:58:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.496+0000 7f7382ffd640 1 -- 172.21.15.139:0/1426460948 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73740327f0 con 0x7f73840cd610 2024-04-25T09:58:09.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.497+0000 7f738a85d640 1 -- 172.21.15.139:0/1426460948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73840cd610 msgr2=0x7f73840d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.497+0000 7f738a85d640 1 --2- 172.21.15.139:0/1426460948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73840cd610 0x7f73840d0370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73740097d0 tx=0x7f737402a290 comp rx=0 tx=0).stop 2024-04-25T09:58:09.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.497+0000 7f738a85d640 1 -- 172.21.15.139:0/1426460948 shutdown_connections 2024-04-25T09:58:09.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.497+0000 7f738a85d640 1 --2- 172.21.15.139:0/1426460948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73840cd610 0x7f73840d0370 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.497+0000 7f738a85d640 1 -- 172.21.15.139:0/1426460948 >> 172.21.15.139:0/1426460948 conn(0x7f73840b31f0 msgr2=0x7f73840b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.497+0000 7f738a85d640 1 -- 172.21.15.139:0/1426460948 shutdown_connections 2024-04-25T09:58:09.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.497+0000 7f738a85d640 1 -- 172.21.15.139:0/1426460948 wait complete. 2024-04-25T09:58:09.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.498+0000 7f738a85d640 1 Processor -- start 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.498+0000 7f738a85d640 1 -- start start 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.499+0000 7f738a85d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738409ad40 0x7f73840c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.499+0000 7f738a85d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73840cd610 0x7f738414d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.499+0000 7f738a85d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7384161fd0 0x7f738414db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.499+0000 7f738a85d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840cd9f0 con 0x7f738409ad40 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.499+0000 7f738a85d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738409a820 con 0x7f73840cd610 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.499+0000 7f738a85d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73840cdb70 con 0x7f7384161fd0 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.499+0000 7f7383fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738409ad40 0x7f73840c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.499+0000 7f7383fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738409ad40 0x7f73840c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36660/0 (socket says 172.21.15.139:36660) 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.499+0000 7f7383fff640 1 -- 172.21.15.139:0/2030137792 learned_addr learned my addr 172.21.15.139:0/2030137792 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.499+0000 7f73837fe640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73840cd610 0x7f738414d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.499+0000 7f7388dfa640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7384161fd0 0x7f738414db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.500+0000 7f7383fff640 1 -- 172.21.15.139:0/2030137792 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7384161fd0 msgr2=0x7f738414db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.500+0000 7f7383fff640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7384161fd0 0x7f738414db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.500+0000 7f7383fff640 1 -- 172.21.15.139:0/2030137792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73840cd610 msgr2=0x7f738414d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.500+0000 7f7383fff640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73840cd610 0x7f738414d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.500+0000 7f7383fff640 1 -- 172.21.15.139:0/2030137792 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73841f3d40 con 0x7f738409ad40 2024-04-25T09:58:09.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.500+0000 7f73837fe640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73840cd610 0x7f738414d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:09.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.500+0000 7f7383fff640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738409ad40 0x7f73840c7690 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f737402a2d0 tx=0x7f73740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.500+0000 7f7380ff9640 1 -- 172.21.15.139:0/2030137792 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374041030 con 0x7f738409ad40 2024-04-25T09:58:09.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.500+0000 7f7380ff9640 1 -- 172.21.15.139:0/2030137792 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737403d4e0 con 0x7f738409ad40 2024-04-25T09:58:09.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.500+0000 7f738a85d640 1 -- 172.21.15.139:0/2030137792 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73840c61d0 con 0x7f738409ad40 2024-04-25T09:58:09.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.500+0000 7f7380ff9640 1 -- 172.21.15.139:0/2030137792 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374037040 con 0x7f738409ad40 2024-04-25T09:58:09.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.500+0000 7f738a85d640 1 -- 172.21.15.139:0/2030137792 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73840c6790 con 0x7f738409ad40 2024-04-25T09:58:09.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.501+0000 7f7380ff9640 1 -- 172.21.15.139:0/2030137792 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7374044070 con 0x7f738409ad40 2024-04-25T09:58:09.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.502+0000 7f7380ff9640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f737006b4d0 0x7f737006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.502+0000 7f7380ff9640 1 -- 172.21.15.139:0/2030137792 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f73740aa640 con 0x7f738409ad40 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.502+0000 7f73837fe640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f737006b4d0 0x7f737006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.502+0000 7f73837fe640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f737006b4d0 0x7f737006d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f73840b7370 tx=0x7f7378009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.503+0000 7f738d564740 1 Processor -- start 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.503+0000 7f738d564740 1 -- start start 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.503+0000 7f738d564740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 0x55df69e3a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.503+0000 7f738d564740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37ef0 0x55df69e3a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f738d564740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e0a770 0x55df69e0df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f738d564740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df69d73da0 con 0x55df69e37ef0 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f738d564740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df69d1cb80 con 0x55df69e0a770 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f738d564740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df69d6fe40 con 0x55df69e37b10 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f73837fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37ef0 0x55df69e3a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f73837fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37ef0 0x55df69e3a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36666/0 (socket says 172.21.15.139:36666) 2024-04-25T09:58:09.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f73837fe640 1 -- 172.21.15.139:0/1619626827 learned_addr learned my addr 172.21.15.139:0/1619626827 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f7383fff640 1 --2- 172.21.15.139:0/1619626827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 0x55df69e3a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f7388dfa640 1 --2- 172.21.15.139:0/1619626827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e0a770 0x55df69e0df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f73837fe640 1 -- 172.21.15.139:0/1619626827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 msgr2=0x55df69e3a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f73837fe640 1 --2- 172.21.15.139:0/1619626827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 0x55df69e3a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f73837fe640 1 -- 172.21.15.139:0/1619626827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e0a770 msgr2=0x55df69e0df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f73837fe640 1 --2- 172.21.15.139:0/1619626827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e0a770 0x55df69e0df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f73837fe640 1 -- 172.21.15.139:0/1619626827 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df69d26f10 con 0x55df69e37ef0 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f73837fe640 1 --2- 172.21.15.139:0/1619626827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37ef0 0x55df69e3a860 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f7378000f60 tx=0x7f7378010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177a377bd3b8d794 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f7369ffb640 1 -- 172.21.15.139:0/1619626827 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737800dbb0 con 0x55df69e37ef0 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f7369ffb640 1 -- 172.21.15.139:0/1619626827 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737800dd50 con 0x55df69e37ef0 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.504+0000 7f7369ffb640 1 -- 172.21.15.139:0/1619626827 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7378020650 con 0x55df69e37ef0 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.505+0000 7f738d564740 1 -- 172.21.15.139:0/1619626827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37ef0 msgr2=0x55df69e3a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.505+0000 7f738d564740 1 --2- 172.21.15.139:0/1619626827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37ef0 0x55df69e3a860 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f7378000f60 tx=0x7f7378010000 comp rx=0 tx=0).stop 2024-04-25T09:58:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.505+0000 7f738d564740 1 -- 172.21.15.139:0/1619626827 shutdown_connections 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.505+0000 7f738d564740 1 --2- 172.21.15.139:0/1619626827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37ef0 0x55df69e3a860 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.505+0000 7f738d564740 1 --2- 172.21.15.139:0/1619626827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e0a770 0x55df69e0df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.505+0000 7f738d564740 1 --2- 172.21.15.139:0/1619626827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 0x55df69e3a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.505+0000 7f738d564740 1 -- 172.21.15.139:0/1619626827 >> 172.21.15.139:0/1619626827 conn(0x55df69e0c9e0 msgr2=0x55df69e55320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.505+0000 7f738d564740 1 -- 172.21.15.139:0/1619626827 shutdown_connections 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.505+0000 7f738d564740 1 -- 172.21.15.139:0/1619626827 wait complete. 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.505+0000 7f738d564740 1 -- start start 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f738d564740 1 Processor -- start 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f738d564740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37b10 0x55df69e27d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f738d564740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37ef0 0x55df69e5c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f738d564740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e599f0 0x55df69e59e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f738d564740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df69e0fb90 con 0x55df69e37b10 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f738d564740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df69d124b0 con 0x55df69e599f0 2024-04-25T09:58:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f738d564740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df69e0fe70 con 0x55df69e37ef0 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f738d564740 10 client.0 _ll_register_callbacks cb 0x55df69d52180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f7383fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37b10 0x55df69e27d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f7383fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37b10 0x55df69e27d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36670/0 (socket says 172.21.15.139:36670) 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f7383fff640 1 -- 172.21.15.139:0/2018298983 learned_addr learned my addr 172.21.15.139:0/2018298983 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f73837fe640 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37ef0 0x55df69e5c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.506+0000 7f7388dfa640 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e599f0 0x55df69e59e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f7383fff640 1 -- 172.21.15.139:0/2018298983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37ef0 msgr2=0x55df69e5c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f7383fff640 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37ef0 0x55df69e5c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f7383fff640 1 -- 172.21.15.139:0/2018298983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e599f0 msgr2=0x55df69e59e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f7383fff640 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e599f0 0x55df69e59e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f7383fff640 1 -- 172.21.15.139:0/2018298983 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df69b527d0 con 0x55df69e37b10 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f7383fff640 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37b10 0x55df69e27d30 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f7374078860 tx=0x7f7374044f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66db9c641c8412a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f7362ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f7362ffd640 1 -- 172.21.15.139:0/2018298983 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737407ddd0 con 0x55df69e37b10 2024-04-25T09:58:09.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f7362ffd640 1 -- 172.21.15.139:0/2018298983 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7374047ca0 con 0x55df69e37b10 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f738d564740 1 -- 172.21.15.139:0/2018298983 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f737800dd50 con 0x55df69e37b10 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f7362ffd640 1 -- 172.21.15.139:0/2018298983 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7374081270 con 0x55df69e37b10 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f738d564740 10 client.7287 fetch_fsmap learned FSMap version 43 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f738d564740 1 -- 172.21.15.139:0/2018298983 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x55df69a883f0 con 0x55df69e37b10 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f7362ffd640 1 -- 172.21.15.139:0/2018298983 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737800dd50 con 0x55df69e37b10 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f7362ffd640 1 -- 172.21.15.139:0/2018298983 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f737403a030 con 0x55df69e37b10 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f738d564740 10 client.7287 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f738d564740 10 client.7287 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f738d564740 20 client.7287 populate_metadata read hostname 'smithi139' 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f738d564740 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55df69e3c340 0x55df69e589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f738d564740 4 client.7287 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.507+0000 7f738d564740 1 -- 172.21.15.139:0/2018298983 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df69d812e0 con 0x55df69e3c340 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.508+0000 7f73837fe640 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55df69e3c340 0x55df69e589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.508+0000 7f7362ffd640 10 client.7287 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.508+0000 7f73837fe640 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55df69e3c340 0x55df69e589b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c850dc65484faf9 server_cookie=448e296a904ac534 in_seq=0 out_seq=0 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.508+0000 7f7362ffd640 1 -- 172.21.15.139:0/2018298983 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df69d812e0 con 0x55df69e3c340 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.508+0000 7f7362ffd640 10 client.7287 handle_command_reply: tid=0 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.508+0000 7f738d564740 1 client.7287 shutdown 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.508+0000 7f738d564740 1 -- 172.21.15.139:0/2018298983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37b10 msgr2=0x55df69e27d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.508+0000 7f738d564740 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37b10 0x55df69e27d30 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f7374078860 tx=0x7f7374044f80 comp rx=0 tx=0).stop 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.509+0000 7f738d564740 1 -- 172.21.15.139:0/2018298983 shutdown_connections 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.509+0000 7f738d564740 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55df69e3c340 0x55df69e589b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.509+0000 7f738d564740 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e599f0 0x55df69e59e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.509+0000 7f738d564740 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37ef0 0x55df69e5c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.509+0000 7f738d564740 1 --2- 172.21.15.139:0/2018298983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e37b10 0x55df69e27d30 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.509+0000 7f7362ffd640 0 client.7287 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.509+0000 7f738d564740 1 -- 172.21.15.139:0/2018298983 >> 172.21.15.139:0/2018298983 conn(0x55df69e0a770 msgr2=0x55df69e09600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.509+0000 7f738d564740 1 -- 172.21.15.139:0/2018298983 shutdown_connections 2024-04-25T09:58:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.509+0000 7f738d564740 1 -- 172.21.15.139:0/2018298983 wait complete. 2024-04-25T09:58:09.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.509+0000 7f738d564740 20 client.7287 trim_cache size 0 max 16384 2024-04-25T09:58:09.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.523+0000 7f738d564740 1 Processor -- start 2024-04-25T09:58:09.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.523+0000 7f738d564740 1 -- start start 2024-04-25T09:58:09.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f738d564740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 0x55df69e523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f738d564740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e37ef0 0x55df69e3a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f738d564740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 0x55df69e37060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f738d564740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73842003b0 con 0x55df69e599f0 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f738d564740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73841ff570 con 0x55df69e37ef0 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f738d564740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738415a230 con 0x55df69e37b10 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f7388dfa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 0x55df69e37060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f7388dfa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 0x55df69e37060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36672/0 (socket says 172.21.15.139:36672) 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f7388dfa640 1 -- 172.21.15.139:0/1625963777 learned_addr learned my addr 172.21.15.139:0/1625963777 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f7388dfa640 1 -- 172.21.15.139:0/1625963777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 msgr2=0x55df69e523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f7388dfa640 1 --2- 172.21.15.139:0/1625963777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 0x55df69e523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f7388dfa640 1 -- 172.21.15.139:0/1625963777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e37ef0 msgr2=0x55df69e3a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.524+0000 7f7388dfa640 1 --2- 172.21.15.139:0/1625963777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e37ef0 0x55df69e3a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f7388dfa640 1 -- 172.21.15.139:0/1625963777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df69e5a1e0 con 0x55df69e599f0 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f7388dfa640 1 --2- 172.21.15.139:0/1625963777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 0x55df69e37060 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f737c0050b0 tx=0x7f737c005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7164aedf4c98a7aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f73697fa640 1 -- 172.21.15.139:0/1625963777 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c0057c0 con 0x55df69e599f0 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f73697fa640 1 -- 172.21.15.139:0/1625963777 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c001af0 con 0x55df69e599f0 2024-04-25T09:58:09.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f73697fa640 1 -- 172.21.15.139:0/1625963777 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c001ea0 con 0x55df69e599f0 2024-04-25T09:58:09.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f738d564740 1 -- 172.21.15.139:0/1625963777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 msgr2=0x55df69e37060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f738d564740 1 --2- 172.21.15.139:0/1625963777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 0x55df69e37060 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f737c0050b0 tx=0x7f737c005570 comp rx=0 tx=0).stop 2024-04-25T09:58:09.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f738d564740 1 -- 172.21.15.139:0/1625963777 shutdown_connections 2024-04-25T09:58:09.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f738d564740 1 --2- 172.21.15.139:0/1625963777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 0x55df69e37060 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f738d564740 1 --2- 172.21.15.139:0/1625963777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e37ef0 0x55df69e3a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f738d564740 1 --2- 172.21.15.139:0/1625963777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 0x55df69e523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f738d564740 1 -- 172.21.15.139:0/1625963777 >> 172.21.15.139:0/1625963777 conn(0x55df69e0a770 msgr2=0x55df69e27b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f738d564740 1 -- 172.21.15.139:0/1625963777 shutdown_connections 2024-04-25T09:58:09.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.525+0000 7f738d564740 1 -- 172.21.15.139:0/1625963777 wait complete. 2024-04-25T09:58:09.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.526+0000 7f738d564740 1 -- start start 2024-04-25T09:58:09.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.526+0000 7f738d564740 1 Processor -- start 2024-04-25T09:58:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f738d564740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 0x55df69e0df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f738d564740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e37ef0 0x55df69ea51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f738d564740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 0x55df69e09600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f738d564740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df69e561c0 con 0x55df69e599f0 2024-04-25T09:58:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f738d564740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738415b230 con 0x55df69e37ef0 2024-04-25T09:58:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f738d564740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df69d140a0 con 0x55df69e37b10 2024-04-25T09:58:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f738d564740 10 client.0 _ll_register_callbacks cb 0x7f7378001430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f7388dfa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 0x55df69e09600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f7388dfa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 0x55df69e09600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:36678/0 (socket says 172.21.15.139:36678) 2024-04-25T09:58:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f7388dfa640 1 -- 172.21.15.139:0/40938877 learned_addr learned my addr 172.21.15.139:0/40938877 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:09.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f73837fe640 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e37ef0 0x55df69ea51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f7383fff640 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 0x55df69e0df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f7388dfa640 1 -- 172.21.15.139:0/40938877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 msgr2=0x55df69e0df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f7388dfa640 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 0x55df69e0df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f7388dfa640 1 -- 172.21.15.139:0/40938877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e37ef0 msgr2=0x55df69ea51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f7388dfa640 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e37ef0 0x55df69ea51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f7388dfa640 1 -- 172.21.15.139:0/40938877 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df69a480f0 con 0x55df69e599f0 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f7388dfa640 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 0x55df69e09600 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f737c003fd0 tx=0x7f737c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db52d588c946953 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.527+0000 7f7362ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f7362ffd640 1 -- 172.21.15.139:0/40938877 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c021950 con 0x55df69e599f0 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f7362ffd640 1 -- 172.21.15.139:0/40938877 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f737c021af0 con 0x55df69e599f0 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f738d564740 1 -- 172.21.15.139:0/40938877 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f737c001af0 con 0x55df69e599f0 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f7362ffd640 1 -- 172.21.15.139:0/40938877 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f737c021cf0 con 0x55df69e599f0 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f738d564740 10 client.7293 fetch_fsmap learned FSMap version 43 2024-04-25T09:58:09.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f738d564740 1 -- 172.21.15.139:0/40938877 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x55df69e0ec30 con 0x55df69e599f0 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f7362ffd640 1 -- 172.21.15.139:0/40938877 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c001af0 con 0x55df69e599f0 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f7362ffd640 1 -- 172.21.15.139:0/40938877 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f737c016450 con 0x55df69e599f0 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f738d564740 10 client.7293 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f738d564740 10 client.7293 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f738d564740 20 client.7293 populate_metadata read hostname 'smithi139' 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f738d564740 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55df69e3bf40 0x55df69e09b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f738d564740 4 client.7293 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f738d564740 1 -- 172.21.15.139:0/40938877 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55df69d7b630 con 0x55df69e3bf40 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f7383fff640 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55df69e3bf40 0x55df69e09b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f7362ffd640 10 client.7293 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.528+0000 7f7383fff640 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55df69e3bf40 0x55df69e09b40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ae0561d11012d0 server_cookie=e0b281d5cc0d80d5 in_seq=0 out_seq=0 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f7362ffd640 1 -- 172.21.15.139:0/40938877 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55df69d7b630 con 0x55df69e3bf40 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f7362ffd640 10 client.7293 handle_command_reply: tid=0 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738d564740 1 client.7293 shutdown 2024-04-25T09:58:09.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738d564740 1 -- 172.21.15.139:0/40938877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 msgr2=0x55df69e09600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738d564740 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 0x55df69e09600 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f737c003fd0 tx=0x7f737c001250 comp rx=0 tx=0).stop 2024-04-25T09:58:09.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738d564740 1 -- 172.21.15.139:0/40938877 shutdown_connections 2024-04-25T09:58:09.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738d564740 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55df69e3bf40 0x55df69e09b40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738d564740 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55df69e599f0 0x55df69e09600 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738d564740 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df69e37ef0 0x55df69ea51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f7362ffd640 0 client.7293 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:09.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738d564740 1 --2- 172.21.15.139:0/40938877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55df69e37b10 0x55df69e0df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738d564740 1 -- 172.21.15.139:0/40938877 >> 172.21.15.139:0/40938877 conn(0x55df69e0a770 msgr2=0x55df69e27b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738d564740 1 -- 172.21.15.139:0/40938877 shutdown_connections 2024-04-25T09:58:09.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738d564740 1 -- 172.21.15.139:0/40938877 wait complete. 2024-04-25T09:58:09.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738d564740 20 client.7293 trim_cache size 0 max 16384 2024-04-25T09:58:09.539 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:58:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:58:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": { 2024-04-25T09:58:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: "8a620024-088e-4d27-8070-ea44464dba19": { 2024-04-25T09:58:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/subdir", 2024-04-25T09:58:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: "tag": "8a620024-088e-4d27-8070-ea44464dba19", 2024-04-25T09:58:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: "options": "recursive,repair" 2024-04-25T09:58:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:09.539 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:58:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738a85d640 1 -- 172.21.15.139:0/2030137792 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f737006b4d0 msgr2=0x7f737006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738a85d640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f737006b4d0 0x7f737006d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f73840b7370 tx=0x7f7378009040 comp rx=0 tx=0).stop 2024-04-25T09:58:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738a85d640 1 -- 172.21.15.139:0/2030137792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738409ad40 msgr2=0x7f73840c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738a85d640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738409ad40 0x7f73840c7690 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f737402a2d0 tx=0x7f73740046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738a85d640 1 -- 172.21.15.139:0/2030137792 shutdown_connections 2024-04-25T09:58:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738a85d640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f737006b4d0 0x7f737006d990 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738a85d640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7384161fd0 0x7f738414db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738a85d640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73840cd610 0x7f738414d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738a85d640 1 --2- 172.21.15.139:0/2030137792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738409ad40 0x7f73840c7690 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.529+0000 7f738a85d640 1 -- 172.21.15.139:0/2030137792 >> 172.21.15.139:0/2030137792 conn(0x7f73840b31f0 msgr2=0x7f7384073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.530+0000 7f738a85d640 1 -- 172.21.15.139:0/2030137792 shutdown_connections 2024-04-25T09:58:09.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:09.530+0000 7f738a85d640 1 -- 172.21.15.139:0/2030137792 wait complete. 2024-04-25T09:58:09.544 INFO:tasks.cephfs.filesystem:scrub status for tag:8a620024-088e-4d27-8070-ea44464dba19 - {'path': '/subdir', 'tag': '8a620024-088e-4d27-8070-ea44464dba19', 'options': 'recursive,repair'} 2024-04-25T09:58:16.137 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:16.140 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:39.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:39.545 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T09:58:39.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.653+0000 7f73ed369640 1 Processor -- start 2024-04-25T09:58:39.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.653+0000 7f73ed369640 1 -- start start 2024-04-25T09:58:39.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73ed369640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e815dbf0 0x7f73e81617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73ed369640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f73e80b9370 0x7f73e80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73ed369640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f73e80cec80 0x7f73e806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73ed369640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e80d1ee0 con 0x7f73e80cec80 2024-04-25T09:58:39.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73ed369640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e8161dc0 con 0x7f73e80b9370 2024-04-25T09:58:39.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73ed369640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e81620a0 con 0x7f73e815dbf0 2024-04-25T09:58:39.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73e77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f73e80cec80 0x7f73e806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73e77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f73e80cec80 0x7f73e806c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38002/0 (socket says 172.21.15.139:38002) 2024-04-25T09:58:39.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73e77fe640 1 -- 172.21.15.139:0/1354645674 learned_addr learned my addr 172.21.15.139:0/1354645674 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:39.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/1354645674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e815dbf0 0x7f73e81617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73e67fc640 1 --2- 172.21.15.139:0/1354645674 >> v2:172.21.15.150:3300/0 conn(0x7f73e80b9370 0x7f73e80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73e77fe640 1 -- 172.21.15.139:0/1354645674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e815dbf0 msgr2=0x7f73e81617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:39.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73e77fe640 1 --2- 172.21.15.139:0/1354645674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e815dbf0 0x7f73e81617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73e77fe640 1 -- 172.21.15.139:0/1354645674 >> v2:172.21.15.150:3300/0 conn(0x7f73e80b9370 msgr2=0x7f73e80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:39.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73e77fe640 1 --2- 172.21.15.139:0/1354645674 >> v2:172.21.15.150:3300/0 conn(0x7f73e80b9370 0x7f73e80ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73e77fe640 1 -- 172.21.15.139:0/1354645674 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e8162380 con 0x7f73e80cec80 2024-04-25T09:58:39.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.654+0000 7f73e67fc640 1 --2- 172.21.15.139:0/1354645674 >> v2:172.21.15.150:3300/0 conn(0x7f73e80b9370 0x7f73e80ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:39.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.655+0000 7f73e77fe640 1 --2- 172.21.15.139:0/1354645674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80cec80 0x7f73e806c210 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f73e8068310 tx=0x7f73e002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cff6baab5ee103d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.655+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1354645674 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0036070 con 0x7f73e80cec80 2024-04-25T09:58:39.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.655+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1354645674 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e0031600 con 0x7f73e80cec80 2024-04-25T09:58:39.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.655+0000 7f73e5ffb640 1 -- 172.21.15.139:0/1354645674 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e00319d0 con 0x7f73e80cec80 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.656+0000 7f73ed369640 1 -- 172.21.15.139:0/1354645674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80cec80 msgr2=0x7f73e806c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.656+0000 7f73ed369640 1 --2- 172.21.15.139:0/1354645674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80cec80 0x7f73e806c210 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f73e8068310 tx=0x7f73e002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.656+0000 7f73ed369640 1 -- 172.21.15.139:0/1354645674 shutdown_connections 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.656+0000 7f73ed369640 1 --2- 172.21.15.139:0/1354645674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80cec80 0x7f73e806c210 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.656+0000 7f73ed369640 1 --2- 172.21.15.139:0/1354645674 >> v2:172.21.15.150:3300/0 conn(0x7f73e80b9370 0x7f73e80ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.656+0000 7f73ed369640 1 --2- 172.21.15.139:0/1354645674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e815dbf0 0x7f73e81617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.656+0000 7f73ed369640 1 -- 172.21.15.139:0/1354645674 >> 172.21.15.139:0/1354645674 conn(0x7f73e80b5480 msgr2=0x7f73e80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.656+0000 7f73ed369640 1 -- 172.21.15.139:0/1354645674 shutdown_connections 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.656+0000 7f73ed369640 1 -- 172.21.15.139:0/1354645674 wait complete. 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.657+0000 7f73ed369640 1 Processor -- start 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.657+0000 7f73ed369640 1 -- start start 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.658+0000 7f73ed369640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e80b9370 0x7f73e8158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.658+0000 7f73ed369640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80cec80 0x7f73e81604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.658+0000 7f73ed369640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73e815dbf0 0x7f73e81609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.658+0000 7f73ed369640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e815dfd0 con 0x7f73e80cec80 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.658+0000 7f73ed369640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80b97d0 con 0x7f73e815dbf0 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.658+0000 7f73ed369640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e815bc80 con 0x7f73e80b9370 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.658+0000 7f73e67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80cec80 0x7f73e81604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.658+0000 7f73e67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80cec80 0x7f73e81604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38018/0 (socket says 172.21.15.139:38018) 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.658+0000 7f73e67fc640 1 -- 172.21.15.139:0/175349809 learned_addr learned my addr 172.21.15.139:0/175349809 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.658+0000 7f73e77fe640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73e815dbf0 0x7f73e81609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.658+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e80b9370 0x7f73e8158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.659+0000 7f73e67fc640 1 -- 172.21.15.139:0/175349809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e80b9370 msgr2=0x7f73e8158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.659+0000 7f73e67fc640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e80b9370 0x7f73e8158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.659+0000 7f73e67fc640 1 -- 172.21.15.139:0/175349809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73e815dbf0 msgr2=0x7f73e81609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.659+0000 7f73e67fc640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73e815dbf0 0x7f73e81609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.659+0000 7f73e67fc640 1 -- 172.21.15.139:0/175349809 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e815bf40 con 0x7f73e80cec80 2024-04-25T09:58:39.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.659+0000 7f73e67fc640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80cec80 0x7f73e81604b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00d830 tx=0x7f73dc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.659+0000 7f73cf7fe640 1 -- 172.21.15.139:0/175349809 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc018070 con 0x7f73e80cec80 2024-04-25T09:58:39.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.659+0000 7f73cf7fe640 1 -- 172.21.15.139:0/175349809 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc00be10 con 0x7f73e80cec80 2024-04-25T09:58:39.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.659+0000 7f73cf7fe640 1 -- 172.21.15.139:0/175349809 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc004250 con 0x7f73e80cec80 2024-04-25T09:58:39.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.659+0000 7f73ed369640 1 -- 172.21.15.139:0/175349809 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e815c220 con 0x7f73e80cec80 2024-04-25T09:58:39.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.659+0000 7f73ed369640 1 -- 172.21.15.139:0/175349809 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e8159270 con 0x7f73e80cec80 2024-04-25T09:58:39.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.660+0000 7f73cf7fe640 1 -- 172.21.15.139:0/175349809 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73dc0098b0 con 0x7f73e80cec80 2024-04-25T09:58:39.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.661+0000 7f73cf7fe640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f73d406b4d0 0x7f73d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.661+0000 7f73cf7fe640 1 -- 172.21.15.139:0/175349809 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f73dc08a4a0 con 0x7f73e80cec80 2024-04-25T09:58:39.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.661+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f73d406b4d0 0x7f73d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.661+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f73d406b4d0 0x7f73d406d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009b40 tx=0x7f73d8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.662+0000 7f73f006a740 1 Processor -- start 2024-04-25T09:58:39.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.662+0000 7f73f006a740 1 -- start start 2024-04-25T09:58:39.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73f006a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de835e0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73f006a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835e0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73f006a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835b0770 0x55de835b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73f006a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de83519da0 con 0x55de835ddb10 2024-04-25T09:58:39.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73f006a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de834c2b80 con 0x55de835ddef0 2024-04-25T09:58:39.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73f006a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de83515e40 con 0x55de835b0770 2024-04-25T09:58:39.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73e6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de835e0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73e6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de835e0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38026/0 (socket says 172.21.15.139:38026) 2024-04-25T09:58:39.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73e6ffd640 1 -- 172.21.15.139:0/2419594542 learned_addr learned my addr 172.21.15.139:0/2419594542 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:39.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73e67fc640 1 --2- 172.21.15.139:0/2419594542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73e77fe640 1 --2- 172.21.15.139:0/2419594542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835b0770 0x55de835b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73e6ffd640 1 -- 172.21.15.139:0/2419594542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835b0770 msgr2=0x55de835b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/2419594542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835b0770 0x55de835b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73e6ffd640 1 -- 172.21.15.139:0/2419594542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 msgr2=0x55de835e0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/2419594542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835e0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73e6ffd640 1 -- 172.21.15.139:0/2419594542 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de834ccf10 con 0x55de835ddb10 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.663+0000 7f73e67fc640 1 --2- 172.21.15.139:0/2419594542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835e0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.664+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/2419594542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de835e0320 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006a90 tx=0x7f73d8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3607e79ba9fc274 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.664+0000 7f73ccff9640 1 -- 172.21.15.139:0/2419594542 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d801b070 con 0x55de835ddb10 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.664+0000 7f73ccff9640 1 -- 172.21.15.139:0/2419594542 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d8027070 con 0x55de835ddb10 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.664+0000 7f73ccff9640 1 -- 172.21.15.139:0/2419594542 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8003a40 con 0x55de835ddb10 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.664+0000 7f73f006a740 1 -- 172.21.15.139:0/2419594542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 msgr2=0x55de835e0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.664+0000 7f73f006a740 1 --2- 172.21.15.139:0/2419594542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de835e0320 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006a90 tx=0x7f73d8014000 comp rx=0 tx=0).stop 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.664+0000 7f73f006a740 1 -- 172.21.15.139:0/2419594542 shutdown_connections 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.664+0000 7f73f006a740 1 --2- 172.21.15.139:0/2419594542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835b0770 0x55de835b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.664+0000 7f73f006a740 1 --2- 172.21.15.139:0/2419594542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835e0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.664+0000 7f73f006a740 1 --2- 172.21.15.139:0/2419594542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de835e0320 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.664+0000 7f73f006a740 1 -- 172.21.15.139:0/2419594542 >> 172.21.15.139:0/2419594542 conn(0x55de835b29e0 msgr2=0x55de835fb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:39.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.665+0000 7f73f006a740 1 -- 172.21.15.139:0/2419594542 shutdown_connections 2024-04-25T09:58:39.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.665+0000 7f73f006a740 1 -- 172.21.15.139:0/2419594542 wait complete. 2024-04-25T09:58:39.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.665+0000 7f73f006a740 1 -- start start 2024-04-25T09:58:39.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.665+0000 7f73f006a740 1 Processor -- start 2024-04-25T09:58:39.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73f006a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835ddb10 0x55de835fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73f006a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddef0 0x55de835f7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73f006a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835d4c00 0x55de835f7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73f006a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de835b5b90 con 0x55de835ddef0 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73f006a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de834b84b0 con 0x55de835d4c00 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73f006a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de835b5e70 con 0x55de835ddb10 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73f006a740 10 client.0 _ll_register_callbacks cb 0x55de834f8180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73e67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddef0 0x55de835f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73e67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddef0 0x55de835f7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38036/0 (socket says 172.21.15.139:38036) 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73e67fc640 1 -- 172.21.15.139:0/4199666111 learned_addr learned my addr 172.21.15.139:0/4199666111 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73e77fe640 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835d4c00 0x55de835f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835ddb10 0x55de835fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73e67fc640 1 -- 172.21.15.139:0/4199666111 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835ddb10 msgr2=0x55de835fd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73e67fc640 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835ddb10 0x55de835fd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73e67fc640 1 -- 172.21.15.139:0/4199666111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835d4c00 msgr2=0x55de835f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73e67fc640 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835d4c00 0x55de835f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73e67fc640 1 -- 172.21.15.139:0/4199666111 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de832f87d0 con 0x55de835ddef0 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73e67fc640 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddef0 0x55de835f7120 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0887f0 tx=0x7f73dc05cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86736cb6ee01675b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.666+0000 7f73c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73c5ffb640 1 -- 172.21.15.139:0/4199666111 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc070030 con 0x55de835ddef0 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73c5ffb640 1 -- 172.21.15.139:0/4199666111 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f73dc0551f0 con 0x55de835ddef0 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73f006a740 1 -- 172.21.15.139:0/4199666111 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73d8027070 con 0x55de835ddef0 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73c5ffb640 1 -- 172.21.15.139:0/4199666111 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73dc056f40 con 0x55de835ddef0 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73f006a740 10 client.7305 fetch_fsmap learned FSMap version 44 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73f006a740 1 -- 172.21.15.139:0/4199666111 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55de8322e3f0 con 0x55de835ddef0 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73c5ffb640 1 -- 172.21.15.139:0/4199666111 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d8027070 con 0x55de835ddef0 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73c5ffb640 1 -- 172.21.15.139:0/4199666111 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f73dc013050 con 0x55de835ddef0 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73f006a740 10 client.7305 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73f006a740 10 client.7305 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73f006a740 20 client.7305 populate_metadata read hostname 'smithi139' 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73f006a740 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55de835d4380 0x55de83618590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73f006a740 4 client.7305 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73f006a740 1 -- 172.21.15.139:0/4199666111 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de835272e0 con 0x55de835d4380 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55de835d4380 0x55de83618590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73c5ffb640 10 client.7305 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.667+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55de835d4380 0x55de83618590 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b554d534d3a28783 server_cookie=43e486a890f87272 in_seq=0 out_seq=0 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73c5ffb640 1 -- 172.21.15.139:0/4199666111 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de835272e0 con 0x55de835d4380 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73c5ffb640 10 client.7305 handle_command_reply: tid=0 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73f006a740 1 client.7305 shutdown 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73f006a740 1 -- 172.21.15.139:0/4199666111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddef0 msgr2=0x55de835f7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73f006a740 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddef0 0x55de835f7120 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0887f0 tx=0x7f73dc05cc20 comp rx=0 tx=0).stop 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73f006a740 1 -- 172.21.15.139:0/4199666111 shutdown_connections 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73f006a740 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55de835d4380 0x55de83618590 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73f006a740 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddef0 0x55de835f7120 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73f006a740 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835d4c00 0x55de835f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73c5ffb640 0 client.7305 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:39.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73f006a740 1 --2- 172.21.15.139:0/4199666111 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835ddb10 0x55de835fd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73f006a740 1 -- 172.21.15.139:0/4199666111 >> 172.21.15.139:0/4199666111 conn(0x55de835b0770 msgr2=0x55de835b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:39.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73f006a740 1 -- 172.21.15.139:0/4199666111 shutdown_connections 2024-04-25T09:58:39.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73f006a740 1 -- 172.21.15.139:0/4199666111 wait complete. 2024-04-25T09:58:39.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.668+0000 7f73f006a740 20 client.7305 trim_cache size 0 max 16384 2024-04-25T09:58:39.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.683+0000 7f73f006a740 1 Processor -- start 2024-04-25T09:58:39.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.683+0000 7f73f006a740 1 -- start start 2024-04-25T09:58:39.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73f006a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835d4c00 0x55de835b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73f006a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de8365d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73f006a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73f006a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8200530 con 0x55de835ddb10 2024-04-25T09:58:39.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73f006a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e81ff6b0 con 0x55de835ddef0 2024-04-25T09:58:39.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73f006a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e81f5560 con 0x55de835d4c00 2024-04-25T09:58:39.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73e67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de8365d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73e67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de8365d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38038/0 (socket says 172.21.15.139:38038) 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73e67fc640 1 -- 172.21.15.139:0/3965228296 learned_addr learned my addr 172.21.15.139:0/3965228296 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73e67fc640 1 -- 172.21.15.139:0/3965228296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835d4c00 msgr2=0x55de835b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73e67fc640 1 --2- 172.21.15.139:0/3965228296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835d4c00 0x55de835b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73e67fc640 1 -- 172.21.15.139:0/3965228296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 msgr2=0x55de835d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73e67fc640 1 --2- 172.21.15.139:0/3965228296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73e67fc640 1 -- 172.21.15.139:0/3965228296 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de831ee0f0 con 0x55de835ddb10 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73e67fc640 1 --2- 172.21.15.139:0/3965228296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de8365d9d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f73dc054fe0 tx=0x7f73dc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49335ade709093f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73c7fff640 1 -- 172.21.15.139:0/3965228296 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc070040 con 0x55de835ddb10 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73c7fff640 1 -- 172.21.15.139:0/3965228296 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc0551f0 con 0x55de835ddb10 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.684+0000 7f73c7fff640 1 -- 172.21.15.139:0/3965228296 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc009ba0 con 0x55de835ddb10 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.685+0000 7f73f006a740 1 -- 172.21.15.139:0/3965228296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 msgr2=0x55de8365d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.685+0000 7f73f006a740 1 --2- 172.21.15.139:0/3965228296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de8365d9d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f73dc054fe0 tx=0x7f73dc015000 comp rx=0 tx=0).stop 2024-04-25T09:58:39.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.685+0000 7f73f006a740 1 -- 172.21.15.139:0/3965228296 shutdown_connections 2024-04-25T09:58:39.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.685+0000 7f73f006a740 1 --2- 172.21.15.139:0/3965228296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de8365d9d0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.685+0000 7f73f006a740 1 --2- 172.21.15.139:0/3965228296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835d21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.685+0000 7f73f006a740 1 --2- 172.21.15.139:0/3965228296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835d4c00 0x55de835b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.685+0000 7f73f006a740 1 -- 172.21.15.139:0/3965228296 >> 172.21.15.139:0/3965228296 conn(0x55de835b0770 msgr2=0x55de8361cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:39.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.685+0000 7f73f006a740 1 -- 172.21.15.139:0/3965228296 shutdown_connections 2024-04-25T09:58:39.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.685+0000 7f73f006a740 1 -- 172.21.15.139:0/3965228296 wait complete. 2024-04-25T09:58:39.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.685+0000 7f73f006a740 1 -- start start 2024-04-25T09:58:39.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.686+0000 7f73f006a740 1 Processor -- start 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.686+0000 7f73f006a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835d4c00 0x55de8365abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.686+0000 7f73f006a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de835d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.686+0000 7f73f006a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.686+0000 7f73f006a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8361c880 con 0x55de835ddb10 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.686+0000 7f73f006a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e81ff0b0 con 0x55de835ddef0 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.686+0000 7f73f006a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de835b4c50 con 0x55de835d4c00 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.686+0000 7f73f006a740 10 client.0 _ll_register_callbacks cb 0x7f73d80095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.686+0000 7f73e67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de835d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.686+0000 7f73e67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de835d21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38050/0 (socket says 172.21.15.139:38050) 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.686+0000 7f73e67fc640 1 -- 172.21.15.139:0/513490289 learned_addr learned my addr 172.21.15.139:0/513490289 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.686+0000 7f73e77fe640 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835d4c00 0x55de8365abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73e67fc640 1 -- 172.21.15.139:0/513490289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835d4c00 msgr2=0x55de8365abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73e67fc640 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835d4c00 0x55de8365abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73e67fc640 1 -- 172.21.15.139:0/513490289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 msgr2=0x55de835d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73e67fc640 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835d2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73e67fc640 1 -- 172.21.15.139:0/513490289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de8365b110 con 0x55de835ddb10 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73e77fe640 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73e67fc640 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de835d21c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f73dc004650 tx=0x7f73dc08a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3d7be1aa2c3e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73c5ffb640 1 -- 172.21.15.139:0/513490289 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc070040 con 0x55de835ddb10 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73c5ffb640 1 -- 172.21.15.139:0/513490289 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f73dc081ce0 con 0x55de835ddb10 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73f006a740 1 -- 172.21.15.139:0/513490289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73dc0551f0 con 0x55de835ddb10 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73c5ffb640 1 -- 172.21.15.139:0/513490289 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73dc0584e0 con 0x55de835ddb10 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73f006a740 10 client.7311 fetch_fsmap learned FSMap version 44 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73f006a740 1 -- 172.21.15.139:0/513490289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55de835b5220 con 0x55de835ddb10 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73c5ffb640 1 -- 172.21.15.139:0/513490289 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc0551f0 con 0x55de835ddb10 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73c5ffb640 1 -- 172.21.15.139:0/513490289 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f73dc05db40 con 0x55de835ddb10 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73f006a740 10 client.7311 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73f006a740 10 client.7311 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73f006a740 20 client.7311 populate_metadata read hostname 'smithi139' 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73f006a740 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55de835e4490 0x55de835fb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73f006a740 4 client.7311 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73f006a740 1 -- 172.21.15.139:0/513490289 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55de835ff040 con 0x55de835e4490 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.687+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55de835e4490 0x55de835fb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73c5ffb640 10 client.7311 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73e6ffd640 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55de835e4490 0x55de835fb320 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48eddd60cd0bb0cc server_cookie=4b1e8a6e57abe318 in_seq=0 out_seq=0 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73c5ffb640 1 -- 172.21.15.139:0/513490289 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55de835ff040 con 0x55de835e4490 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73c5ffb640 10 client.7311 handle_command_reply: tid=0 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73f006a740 1 client.7311 shutdown 2024-04-25T09:58:39.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73f006a740 1 -- 172.21.15.139:0/513490289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 msgr2=0x55de835d21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73f006a740 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de835d21c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f73dc004650 tx=0x7f73dc08a050 comp rx=0 tx=0).stop 2024-04-25T09:58:39.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73f006a740 1 -- 172.21.15.139:0/513490289 shutdown_connections 2024-04-25T09:58:39.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73f006a740 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x55de835e4490 0x55de835fb320 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73f006a740 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55de835ddb10 0x55de835d21c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73c5ffb640 0 client.7311 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:39.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73f006a740 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55de835ddef0 0x55de835d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73f006a740 1 --2- 172.21.15.139:0/513490289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55de835d4c00 0x55de8365abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73f006a740 1 -- 172.21.15.139:0/513490289 >> 172.21.15.139:0/513490289 conn(0x55de835b0770 msgr2=0x55de835b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:39.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73f006a740 1 -- 172.21.15.139:0/513490289 shutdown_connections 2024-04-25T09:58:39.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73f006a740 1 -- 172.21.15.139:0/513490289 wait complete. 2024-04-25T09:58:39.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.688+0000 7f73f006a740 20 client.7311 trim_cache size 0 max 16384 2024-04-25T09:58:39.699 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:58:39.699 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "no active scrubs running", 2024-04-25T09:58:39.699 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": {} 2024-04-25T09:58:39.699 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:58:39.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.689+0000 7f73ed369640 1 -- 172.21.15.139:0/175349809 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f73d406b4d0 msgr2=0x7f73d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.689+0000 7f73ed369640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f73d406b4d0 0x7f73d406d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009b40 tx=0x7f73d8009190 comp rx=0 tx=0).stop 2024-04-25T09:58:39.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.689+0000 7f73ed369640 1 -- 172.21.15.139:0/175349809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80cec80 msgr2=0x7f73e81604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.689+0000 7f73ed369640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80cec80 0x7f73e81604b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00d830 tx=0x7f73dc00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.689+0000 7f73ed369640 1 -- 172.21.15.139:0/175349809 shutdown_connections 2024-04-25T09:58:39.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.689+0000 7f73ed369640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80cec80 0x7f73e81604b0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.689+0000 7f73ed369640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f73d406b4d0 0x7f73d406d990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.689+0000 7f73ed369640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73e815dbf0 0x7f73e81609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.689+0000 7f73ed369640 1 --2- 172.21.15.139:0/175349809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e80b9370 0x7f73e8158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.689+0000 7f73ed369640 1 -- 172.21.15.139:0/175349809 >> 172.21.15.139:0/175349809 conn(0x7f73e80b5480 msgr2=0x7f73e80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:39.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.689+0000 7f73ed369640 1 -- 172.21.15.139:0/175349809 shutdown_connections 2024-04-25T09:58:39.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.689+0000 7f73ed369640 1 -- 172.21.15.139:0/175349809 wait complete. 2024-04-25T09:58:39.703 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:58:39.703 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:39.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.796+0000 7fd75cde2640 1 Processor -- start 2024-04-25T09:58:39.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.796+0000 7fd75cde2640 1 -- start start 2024-04-25T09:58:39.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd75cde2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd75815a860 0x7fd7580cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd75cde2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7580cd1f0 0x7fd75806e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd75cde2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7580af6d0 con 0x7fd75815ac40 2024-04-25T09:58:39.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd75cde2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7580b02e0 con 0x7fd75815a860 2024-04-25T09:58:39.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd75cde2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7580af9b0 con 0x7fd7580cd1f0 2024-04-25T09:58:39.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd74ffff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7580cd1f0 0x7fd75806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd75659c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd75815a860 0x7fd7580cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd756d9d640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fd75815ac40 0x7fd75815e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:40134/0 (socket says 172.21.15.139:40134) 2024-04-25T09:58:39.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd74ffff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7580cd1f0 0x7fd75806e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:48560/0 (socket says 172.21.15.139:48560) 2024-04-25T09:58:39.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd756d9d640 1 -- 172.21.15.139:0/1127755637 learned_addr learned my addr 172.21.15.139:0/1127755637 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:39.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd75659c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd75815a860 0x7fd7580cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:37382/0 (socket says 172.21.15.139:37382) 2024-04-25T09:58:39.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd74ffff640 1 -- 172.21.15.139:0/1127755637 >> v2:172.21.15.150:3300/0 conn(0x7fd75815a860 msgr2=0x7fd7580cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:39.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd74ffff640 1 --2- 172.21.15.139:0/1127755637 >> v2:172.21.15.150:3300/0 conn(0x7fd75815a860 0x7fd7580cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd74ffff640 1 -- 172.21.15.139:0/1127755637 >> v1:172.21.15.150:6789/0 conn(0x7fd75815ac40 legacy=0x7fd75815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd74ffff640 1 -- 172.21.15.139:0/1127755637 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7580afc90 con 0x7fd7580cd1f0 2024-04-25T09:58:39.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd75659c640 1 --2- 172.21.15.139:0/1127755637 >> v2:172.21.15.150:3300/0 conn(0x7fd75815a860 0x7fd7580cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:39.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.797+0000 7fd755d9b640 1 -- 172.21.15.139:0/1127755637 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419016480 0 0) 0x7fd7580af6d0 con 0x7fd75815ac40 2024-04-25T09:58:39.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.798+0000 7fd74ffff640 1 --2- 172.21.15.139:0/1127755637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7580cd1f0 0x7fd75806e310 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd7400097a0 tx=0x7fd74002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64de4c32e94317c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.798+0000 7fd755d9b640 1 -- 172.21.15.139:0/1127755637 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740037070 con 0x7fd7580cd1f0 2024-04-25T09:58:39.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.798+0000 7fd755d9b640 1 -- 172.21.15.139:0/1127755637 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd740032440 con 0x7fd7580cd1f0 2024-04-25T09:58:39.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.798+0000 7fd755d9b640 1 -- 172.21.15.139:0/1127755637 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740032810 con 0x7fd7580cd1f0 2024-04-25T09:58:39.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.799+0000 7fd75cde2640 1 -- 172.21.15.139:0/1127755637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7580cd1f0 msgr2=0x7fd75806e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.799+0000 7fd75cde2640 1 --2- 172.21.15.139:0/1127755637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7580cd1f0 0x7fd75806e310 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd7400097a0 tx=0x7fd74002a180 comp rx=0 tx=0).stop 2024-04-25T09:58:39.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.799+0000 7fd75cde2640 1 -- 172.21.15.139:0/1127755637 shutdown_connections 2024-04-25T09:58:39.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.799+0000 7fd75cde2640 1 --2- 172.21.15.139:0/1127755637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7580cd1f0 0x7fd75806e310 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.799+0000 7fd75cde2640 1 --2- 172.21.15.139:0/1127755637 >> v2:172.21.15.150:3300/0 conn(0x7fd75815a860 0x7fd7580cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.799+0000 7fd75cde2640 1 -- 172.21.15.139:0/1127755637 >> 172.21.15.139:0/1127755637 conn(0x7fd758066f20 msgr2=0x7fd75806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:39.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.799+0000 7fd75cde2640 1 -- 172.21.15.139:0/1127755637 shutdown_connections 2024-04-25T09:58:39.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.799+0000 7fd75cde2640 1 -- 172.21.15.139:0/1127755637 wait complete. 2024-04-25T09:58:39.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.800+0000 7fd75cde2640 1 Processor -- start 2024-04-25T09:58:39.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.800+0000 7fd75cde2640 1 -- start start 2024-04-25T09:58:39.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.801+0000 7fd75cde2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7580cd1f0 0x7fd758144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.801+0000 7fd75cde2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd75815a860 0x7fd7581eca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.801+0000 7fd75cde2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75815ac40 0x7fd7581ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.801+0000 7fd75cde2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7580af420 con 0x7fd7580cd1f0 2024-04-25T09:58:39.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.801+0000 7fd75cde2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7580af2a0 con 0x7fd75815ac40 2024-04-25T09:58:39.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.801+0000 7fd75cde2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75806e850 con 0x7fd75815a860 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.801+0000 7fd75659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7580cd1f0 0x7fd758144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.801+0000 7fd75659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7580cd1f0 0x7fd758144730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38066/0 (socket says 172.21.15.139:38066) 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.801+0000 7fd75659c640 1 -- 172.21.15.139:0/1011723518 learned_addr learned my addr 172.21.15.139:0/1011723518 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.801+0000 7fd756d9d640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75815ac40 0x7fd7581ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.801+0000 7fd74ffff640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd75815a860 0x7fd7581eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.801+0000 7fd75659c640 1 -- 172.21.15.139:0/1011723518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd75815a860 msgr2=0x7fd7581eca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.802+0000 7fd75659c640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd75815a860 0x7fd7581eca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.802+0000 7fd75659c640 1 -- 172.21.15.139:0/1011723518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75815ac40 msgr2=0x7fd7581ecfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.802+0000 7fd75659c640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75815ac40 0x7fd7581ecfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.802+0000 7fd75659c640 1 -- 172.21.15.139:0/1011723518 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7581fbec0 con 0x7fd7580cd1f0 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.802+0000 7fd756d9d640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75815ac40 0x7fd7581ecfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.802+0000 7fd74ffff640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd75815a860 0x7fd7581eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.802+0000 7fd75659c640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7580cd1f0 0x7fd758144730 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd74800ec10 tx=0x7fd74800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.802+0000 7fd74effd640 1 -- 172.21.15.139:0/1011723518 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748017070 con 0x7fd7580cd1f0 2024-04-25T09:58:39.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.802+0000 7fd74effd640 1 -- 172.21.15.139:0/1011723518 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74800cc30 con 0x7fd7580cd1f0 2024-04-25T09:58:39.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.802+0000 7fd75cde2640 1 -- 172.21.15.139:0/1011723518 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7581fc1a0 con 0x7fd7580cd1f0 2024-04-25T09:58:39.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.802+0000 7fd74effd640 1 -- 172.21.15.139:0/1011723518 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748013470 con 0x7fd7580cd1f0 2024-04-25T09:58:39.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.802+0000 7fd75cde2640 1 -- 172.21.15.139:0/1011723518 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7580a98f0 con 0x7fd7580cd1f0 2024-04-25T09:58:39.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.803+0000 7fd75cde2640 1 -- 172.21.15.139:0/1011723518 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd758074520 con 0x7fd7580cd1f0 2024-04-25T09:58:39.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.803+0000 7fd74effd640 1 -- 172.21.15.139:0/1011723518 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd74801a070 con 0x7fd7580cd1f0 2024-04-25T09:58:39.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.804+0000 7fd74effd640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd73806b4d0 0x7fd73806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.804+0000 7fd74effd640 1 -- 172.21.15.139:0/1011723518 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd74808a730 con 0x7fd7580cd1f0 2024-04-25T09:58:39.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.804+0000 7fd74ffff640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd73806b4d0 0x7fd73806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.804+0000 7fd74ffff640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd73806b4d0 0x7fd73806d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd740034000 tx=0x7fd74002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:39.808+0000 7fd74effd640 1 -- 172.21.15.139:0/1011723518 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7480593d0 con 0x7fd7580cd1f0 2024-04-25T09:58:40.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.014+0000 7fd75cde2640 1 -- 172.21.15.139:0/1011723518 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7580ce0b0 con 0x7fd7580cd1f0 2024-04-25T09:58:40.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.015+0000 7fd74effd640 1 -- 172.21.15.139:0/1011723518 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fd748058cd0 con 0x7fd7580cd1f0 2024-04-25T09:58:40.019 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:40.019 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/982251530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":982251530},{"type":"v1","addr":"172.21.15.150:6834","nonce":982251530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:20.444843+0000","modified":"2024-04-25T09:58:12.541265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":17,"addr":"172.21.15.139:6837/1384380923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":1384380923},{"type":"v1","addr":"172.21.15.139:6837","nonce":1384380923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:58:40.019 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 44 2024-04-25T09:58:40.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.017+0000 7fd75cde2640 1 -- 172.21.15.139:0/1011723518 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd73806b4d0 msgr2=0x7fd73806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.017+0000 7fd75cde2640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd73806b4d0 0x7fd73806d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd740034000 tx=0x7fd74002e000 comp rx=0 tx=0).stop 2024-04-25T09:58:40.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.017+0000 7fd75cde2640 1 -- 172.21.15.139:0/1011723518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7580cd1f0 msgr2=0x7fd758144730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.017+0000 7fd75cde2640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7580cd1f0 0x7fd758144730 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd74800ec10 tx=0x7fd74800c520 comp rx=0 tx=0).stop 2024-04-25T09:58:40.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.017+0000 7fd75cde2640 1 -- 172.21.15.139:0/1011723518 shutdown_connections 2024-04-25T09:58:40.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.017+0000 7fd75cde2640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd73806b4d0 0x7fd73806d990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.017+0000 7fd75cde2640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd75815ac40 0x7fd7581ecfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.017+0000 7fd75cde2640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd75815a860 0x7fd7581eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.017+0000 7fd75cde2640 1 --2- 172.21.15.139:0/1011723518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7580cd1f0 0x7fd758144730 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.017+0000 7fd75cde2640 1 -- 172.21.15.139:0/1011723518 >> 172.21.15.139:0/1011723518 conn(0x7fd758066f20 msgr2=0x7fd75806dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.017+0000 7fd75cde2640 1 -- 172.21.15.139:0/1011723518 shutdown_connections 2024-04-25T09:58:40.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.017+0000 7fd75cde2640 1 -- 172.21.15.139:0/1011723518 wait complete. 2024-04-25T09:58:40.032 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T09:58:40.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.128+0000 7f64ac322640 1 Processor -- start 2024-04-25T09:58:40.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.128+0000 7f64ac322640 1 -- start start 2024-04-25T09:58:40.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.129+0000 7f64ac322640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f64a40af8a0 0x7f64a40b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.129+0000 7f64ac322640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f64a40afc80 0x7f64a40cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.129+0000 7f64ac322640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a40c6cc0 con 0x7f64a406b700 2024-04-25T09:58:40.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.129+0000 7f64ac322640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a40c6fa0 con 0x7f64a40af8a0 2024-04-25T09:58:40.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.129+0000 7f64ac322640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a40c7280 con 0x7f64a40afc80 2024-04-25T09:58:40.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.129+0000 7f64aa0be640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f64a40af8a0 0x7f64a40b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.129+0000 7f64aa0be640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f64a40af8a0 0x7f64a40b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38068/0 (socket says 172.21.15.139:38068) 2024-04-25T09:58:40.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.129+0000 7f64aa0be640 1 -- 172.21.15.139:0/2456069970 learned_addr learned my addr 172.21.15.139:0/2456069970 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:40.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.129+0000 7f64a98bd640 1 --2- 172.21.15.139:0/2456069970 >> v2:172.21.15.150:3300/0 conn(0x7f64a40afc80 0x7f64a40cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.130+0000 7f64aa0be640 1 -- 172.21.15.139:0/2456069970 >> v2:172.21.15.150:3300/0 conn(0x7f64a40afc80 msgr2=0x7f64a40cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:40.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.130+0000 7f64aa0be640 1 --2- 172.21.15.139:0/2456069970 >> v2:172.21.15.150:3300/0 conn(0x7f64a40afc80 0x7f64a40cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.130+0000 7f64aa0be640 1 -- 172.21.15.139:0/2456069970 >> v1:172.21.15.139:6789/0 conn(0x7f64a406b700 legacy=0x7f64a406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.130+0000 7f64a90bc640 1 -- 172.21.15.139:0/2456069970 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1226672084 0 0) 0x7f64a40c6cc0 con 0x7f64a406b700 2024-04-25T09:58:40.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.130+0000 7f64aa0be640 1 -- 172.21.15.139:0/2456069970 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a40cd090 con 0x7f64a40af8a0 2024-04-25T09:58:40.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.130+0000 7f64aa0be640 1 --2- 172.21.15.139:0/2456069970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64a40af8a0 0x7f64a40b2d50 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f64a40b3290 tx=0x7f64a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd78e33ce1406e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.130+0000 7f64a90bc640 1 -- 172.21.15.139:0/2456069970 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a0037070 con 0x7f64a40af8a0 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.130+0000 7f64a90bc640 1 -- 172.21.15.139:0/2456069970 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a0032440 con 0x7f64a40af8a0 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.130+0000 7f64a90bc640 1 -- 172.21.15.139:0/2456069970 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a0032810 con 0x7f64a40af8a0 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.131+0000 7f64ac322640 1 -- 172.21.15.139:0/2456069970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64a40af8a0 msgr2=0x7f64a40b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.131+0000 7f64ac322640 1 --2- 172.21.15.139:0/2456069970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64a40af8a0 0x7f64a40b2d50 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f64a40b3290 tx=0x7f64a002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.131+0000 7f64ac322640 1 -- 172.21.15.139:0/2456069970 shutdown_connections 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.131+0000 7f64ac322640 1 --2- 172.21.15.139:0/2456069970 >> v2:172.21.15.150:3300/0 conn(0x7f64a40afc80 0x7f64a40cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.131+0000 7f64ac322640 1 --2- 172.21.15.139:0/2456069970 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64a40af8a0 0x7f64a40b2d50 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.131+0000 7f64ac322640 1 -- 172.21.15.139:0/2456069970 >> 172.21.15.139:0/2456069970 conn(0x7f64a40b2230 msgr2=0x7f64a40b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.131+0000 7f64ac322640 1 -- 172.21.15.139:0/2456069970 shutdown_connections 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.131+0000 7f64ac322640 1 -- 172.21.15.139:0/2456069970 wait complete. 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.132+0000 7f64ac322640 1 Processor -- start 2024-04-25T09:58:40.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.132+0000 7f64ac322640 1 -- start start 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64ac322640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64a406b700 0x7f64a41f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64ac322640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64a40af8a0 0x7f64a40b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64ac322640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64a40afc80 0x7f64a409cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64ac322640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a406b270 con 0x7f64a40afc80 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64ac322640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a40cddd0 con 0x7f64a406b700 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64ac322640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a406b3f0 con 0x7f64a40af8a0 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64aa8bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64a40afc80 0x7f64a409cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64aa8bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64a40afc80 0x7f64a409cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38070/0 (socket says 172.21.15.139:38070) 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64aa8bf640 1 -- 172.21.15.139:0/1312726498 learned_addr learned my addr 172.21.15.139:0/1312726498 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64aa0be640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64a406b700 0x7f64a41f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64a98bd640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64a40af8a0 0x7f64a40b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64aa8bf640 1 -- 172.21.15.139:0/1312726498 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64a40af8a0 msgr2=0x7f64a40b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64a40af8a0 0x7f64a40b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64aa8bf640 1 -- 172.21.15.139:0/1312726498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64a406b700 msgr2=0x7f64a41f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64a406b700 0x7f64a41f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64aa8bf640 1 -- 172.21.15.139:0/1312726498 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a40b5910 con 0x7f64a40afc80 2024-04-25T09:58:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64a98bd640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64a40af8a0 0x7f64a40b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.133+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64a40afc80 0x7f64a409cd60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f649c00c760 tx=0x7f649c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.134+0000 7f649affd640 1 -- 172.21.15.139:0/1312726498 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c003da0 con 0x7f64a40afc80 2024-04-25T09:58:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.134+0000 7f649affd640 1 -- 172.21.15.139:0/1312726498 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c005070 con 0x7f64a40afc80 2024-04-25T09:58:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.134+0000 7f64ac322640 1 -- 172.21.15.139:0/1312726498 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a40b5bf0 con 0x7f64a40afc80 2024-04-25T09:58:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.134+0000 7f649affd640 1 -- 172.21.15.139:0/1312726498 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c005460 con 0x7f64a40afc80 2024-04-25T09:58:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.134+0000 7f64ac322640 1 -- 172.21.15.139:0/1312726498 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a409a900 con 0x7f64a40afc80 2024-04-25T09:58:40.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.135+0000 7f649affd640 1 -- 172.21.15.139:0/1312726498 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f649c005600 con 0x7f64a40afc80 2024-04-25T09:58:40.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.136+0000 7f649affd640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f648806b500 0x7f648806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.136+0000 7f649affd640 1 -- 172.21.15.139:0/1312726498 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f649c08a190 con 0x7f64a40afc80 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.136+0000 7f64aa0be640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f648806b500 0x7f648806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.136+0000 7f64aa0be640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f648806b500 0x7f648806d9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f64a0034000 tx=0x7f64a002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.137+0000 7f64af024740 1 Processor -- start 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.137+0000 7f64af024740 1 -- start start 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.137+0000 7f64af024740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x5571687916f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64af024740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687cbc60 0x55716878f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64af024740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687ba050 0x5571687ba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64af024740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571686f2c60 con 0x5571687ba050 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64af024740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557168691d80 con 0x5571687cbc60 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64af024740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571686eed00 con 0x55716878f590 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64aa8bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687ba050 0x5571687ba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64aa8bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687ba050 0x5571687ba4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38080/0 (socket says 172.21.15.139:38080) 2024-04-25T09:58:40.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64aa8bf640 1 -- 172.21.15.139:0/2092225904 learned_addr learned my addr 172.21.15.139:0/2092225904 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64aa0be640 1 --2- 172.21.15.139:0/2092225904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x5571687916f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64a98bd640 1 --2- 172.21.15.139:0/2092225904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687cbc60 0x55716878f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64aa8bf640 1 -- 172.21.15.139:0/2092225904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 msgr2=0x5571687916f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/2092225904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x5571687916f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64aa8bf640 1 -- 172.21.15.139:0/2092225904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687cbc60 msgr2=0x55716878f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/2092225904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687cbc60 0x55716878f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64aa8bf640 1 -- 172.21.15.139:0/2092225904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571684d37c0 con 0x5571687ba050 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/2092225904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687ba050 0x5571687ba4d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f649c0609a0 tx=0x7f649c0586c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447cc42e24758c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f6487fff640 1 -- 172.21.15.139:0/2092225904 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c086070 con 0x5571687ba050 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f6487fff640 1 -- 172.21.15.139:0/2092225904 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c0587c0 con 0x5571687ba050 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.138+0000 7f6487fff640 1 -- 172.21.15.139:0/2092225904 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c05c910 con 0x5571687ba050 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.139+0000 7f64af024740 1 -- 172.21.15.139:0/2092225904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687ba050 msgr2=0x5571687ba4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.139+0000 7f64af024740 1 --2- 172.21.15.139:0/2092225904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687ba050 0x5571687ba4d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f649c0609a0 tx=0x7f649c0586c0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.139+0000 7f64af024740 1 -- 172.21.15.139:0/2092225904 shutdown_connections 2024-04-25T09:58:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.139+0000 7f64af024740 1 --2- 172.21.15.139:0/2092225904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687ba050 0x5571687ba4d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.139+0000 7f64af024740 1 --2- 172.21.15.139:0/2092225904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687cbc60 0x55716878f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.139+0000 7f64af024740 1 --2- 172.21.15.139:0/2092225904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x5571687916f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.139+0000 7f64af024740 1 -- 172.21.15.139:0/2092225904 >> 172.21.15.139:0/2092225904 conn(0x5571687576e0 msgr2=0x5571687d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.139+0000 7f64af024740 1 -- 172.21.15.139:0/2092225904 shutdown_connections 2024-04-25T09:58:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.139+0000 7f64af024740 1 -- 172.21.15.139:0/2092225904 wait complete. 2024-04-25T09:58:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.140+0000 7f64af024740 1 -- start start 2024-04-25T09:58:40.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.140+0000 7f64af024740 1 Processor -- start 2024-04-25T09:58:40.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64af024740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x5571687bd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64af024740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x5571687b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64af024740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x5571687a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64af024740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716868f840 con 0x5571687cbc60 2024-04-25T09:58:40.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64af024740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571687b7ba0 con 0x5571687ba050 2024-04-25T09:58:40.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64af024740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571687ba6e0 con 0x55716878f590 2024-04-25T09:58:40.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64af024740 10 client.0 _ll_register_callbacks cb 0x5571686a9980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:40.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64aa8bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x5571687a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64aa8bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x5571687a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38086/0 (socket says 172.21.15.139:38086) 2024-04-25T09:58:40.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64aa8bf640 1 -- 172.21.15.139:0/3834340674 learned_addr learned my addr 172.21.15.139:0/3834340674 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:40.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64aa0be640 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x5571687bd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64a98bd640 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x5571687b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64aa8bf640 1 -- 172.21.15.139:0/3834340674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 msgr2=0x5571687bd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:40.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x5571687bd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64aa8bf640 1 -- 172.21.15.139:0/3834340674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 msgr2=0x5571687b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:40.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x5571687b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64aa8bf640 1 -- 172.21.15.139:0/3834340674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557168533f50 con 0x5571687cbc60 2024-04-25T09:58:40.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64a98bd640 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x5571687b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:40.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x5571687a17a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f649c0885c0 tx=0x7f649c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a91b4c49dd88c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64857fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:58:40.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64857fa640 1 -- 172.21.15.139:0/3834340674 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c075060 con 0x5571687cbc60 2024-04-25T09:58:40.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64857fa640 1 -- 172.21.15.139:0/3834340674 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f649c060a40 con 0x5571687cbc60 2024-04-25T09:58:40.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64af024740 1 -- 172.21.15.139:0/3834340674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f649c0587c0 con 0x5571687cbc60 2024-04-25T09:58:40.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64857fa640 1 -- 172.21.15.139:0/3834340674 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f649c07c050 con 0x5571687cbc60 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64af024740 10 client.7329 fetch_fsmap learned FSMap version 44 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64af024740 1 -- 172.21.15.139:0/3834340674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x5571683c70b0 con 0x5571687cbc60 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.141+0000 7f64857fa640 1 -- 172.21.15.139:0/3834340674 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c0587c0 con 0x5571687cbc60 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.142+0000 7f64857fa640 1 -- 172.21.15.139:0/3834340674 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f649c057c10 con 0x5571687cbc60 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.142+0000 7f64af024740 10 client.7329 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.142+0000 7f64af024740 10 client.7329 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.142+0000 7f64af024740 20 client.7329 populate_metadata read hostname 'smithi139' 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.142+0000 7f64af024740 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5571687e9680 0x5571687bb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.142+0000 7f64af024740 4 client.7329 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.142+0000 7f64af024740 1 -- 172.21.15.139:0/3834340674 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571686b2e60 con 0x5571687e9680 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.142+0000 7f64aa0be640 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5571687e9680 0x5571687bb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.142+0000 7f64aa0be640 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5571687e9680 0x5571687bb9d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac27b16413788e3 server_cookie=d1152244c0a89de7 in_seq=0 out_seq=0 2024-04-25T09:58:40.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.142+0000 7f64857fa640 10 client.7329 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:40.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.142+0000 7f64857fa640 1 -- 172.21.15.139:0/3834340674 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5571686b2e60 con 0x5571687e9680 2024-04-25T09:58:40.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64857fa640 10 client.7329 handle_command_reply: tid=0 2024-04-25T09:58:40.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64af024740 1 client.7329 shutdown 2024-04-25T09:58:40.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64af024740 1 -- 172.21.15.139:0/3834340674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 msgr2=0x5571687a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64af024740 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x5571687a17a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f649c0885c0 tx=0x7f649c001250 comp rx=0 tx=0).stop 2024-04-25T09:58:40.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64af024740 1 -- 172.21.15.139:0/3834340674 shutdown_connections 2024-04-25T09:58:40.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64af024740 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5571687e9680 0x5571687bb9d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64af024740 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x5571687a17a0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64af024740 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x5571687b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64af024740 1 --2- 172.21.15.139:0/3834340674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x5571687bd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64857fa640 0 client.7329 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:40.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64af024740 1 -- 172.21.15.139:0/3834340674 >> 172.21.15.139:0/3834340674 conn(0x5571687576e0 msgr2=0x5571687b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64af024740 1 -- 172.21.15.139:0/3834340674 shutdown_connections 2024-04-25T09:58:40.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64af024740 1 -- 172.21.15.139:0/3834340674 wait complete. 2024-04-25T09:58:40.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.143+0000 7f64af024740 20 client.7329 trim_cache size 0 max 16384 2024-04-25T09:58:40.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.154+0000 7f64af024740 1 Processor -- start 2024-04-25T09:58:40.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.155+0000 7f64af024740 1 -- start start 2024-04-25T09:58:40.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.155+0000 7f64af024740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x5571687b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.155+0000 7f64af024740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x5571687e3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.155+0000 7f64af024740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x5571687e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.155+0000 7f64af024740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a41ff630 con 0x5571687cbc60 2024-04-25T09:58:40.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.155+0000 7f64af024740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a409c120 con 0x5571687ba050 2024-04-25T09:58:40.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.155+0000 7f64af024740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a41ff030 con 0x55716878f590 2024-04-25T09:58:40.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.155+0000 7f64aa8bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x5571687e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.155+0000 7f64aa8bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x5571687e3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38094/0 (socket says 172.21.15.139:38094) 2024-04-25T09:58:40.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.155+0000 7f64aa8bf640 1 -- 172.21.15.139:0/1257981403 learned_addr learned my addr 172.21.15.139:0/1257981403 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:40.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64aa8bf640 1 -- 172.21.15.139:0/1257981403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 msgr2=0x5571687b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:40.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64aa0be640 1 --2- 172.21.15.139:0/1257981403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x5571687b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64a98bd640 1 --2- 172.21.15.139:0/1257981403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x5571687e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/1257981403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x5571687b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64aa8bf640 1 -- 172.21.15.139:0/1257981403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 msgr2=0x5571687e3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:40.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/1257981403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x5571687e3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64aa8bf640 1 -- 172.21.15.139:0/1257981403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571687ba490 con 0x5571687cbc60 2024-04-25T09:58:40.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/1257981403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x5571687e3cb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f649c056a70 tx=0x7f649c05b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863b323a97d2c9a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64877fe640 1 -- 172.21.15.139:0/1257981403 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c086070 con 0x5571687cbc60 2024-04-25T09:58:40.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64877fe640 1 -- 172.21.15.139:0/1257981403 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c0587c0 con 0x5571687cbc60 2024-04-25T09:58:40.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64877fe640 1 -- 172.21.15.139:0/1257981403 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c058120 con 0x5571687cbc60 2024-04-25T09:58:40.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64af024740 1 -- 172.21.15.139:0/1257981403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 msgr2=0x5571687e3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64af024740 1 --2- 172.21.15.139:0/1257981403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x5571687e3cb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f649c056a70 tx=0x7f649c05b000 comp rx=0 tx=0).stop 2024-04-25T09:58:40.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64af024740 1 -- 172.21.15.139:0/1257981403 shutdown_connections 2024-04-25T09:58:40.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64af024740 1 --2- 172.21.15.139:0/1257981403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x5571687e3cb0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64af024740 1 --2- 172.21.15.139:0/1257981403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x5571687e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64af024740 1 --2- 172.21.15.139:0/1257981403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x5571687b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.156+0000 7f64af024740 1 -- 172.21.15.139:0/1257981403 >> 172.21.15.139:0/1257981403 conn(0x5571687576e0 msgr2=0x557168789490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.157+0000 7f64af024740 1 -- 172.21.15.139:0/1257981403 shutdown_connections 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.157+0000 7f64af024740 1 -- 172.21.15.139:0/1257981403 wait complete. 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.157+0000 7f64af024740 1 -- start start 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.157+0000 7f64af024740 1 Processor -- start 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64af024740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x55716878e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64af024740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x557168824a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64af024740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x557168842e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64af024740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571687bca50 con 0x5571687cbc60 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64af024740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716868f840 con 0x5571687ba050 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64af024740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571687e8ad0 con 0x55716878f590 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64af024740 10 client.0 _ll_register_callbacks cb 0x7f649c001870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64aa8bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x557168842e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64aa8bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x557168842e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38096/0 (socket says 172.21.15.139:38096) 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64aa8bf640 1 -- 172.21.15.139:0/1719460454 learned_addr learned my addr 172.21.15.139:0/1719460454 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64a98bd640 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x557168824a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64aa0be640 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x55716878e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64aa8bf640 1 -- 172.21.15.139:0/1719460454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 msgr2=0x55716878e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x55716878e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64aa8bf640 1 -- 172.21.15.139:0/1719460454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 msgr2=0x557168824a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x557168824a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64aa8bf640 1 -- 172.21.15.139:0/1719460454 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571687b0b40 con 0x5571687cbc60 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64aa8bf640 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x557168842e70 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f649c058610 tx=0x7f649c08a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6415fda5af28b13f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.158+0000 7f64857fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64857fa640 1 -- 172.21.15.139:0/1719460454 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c058120 con 0x5571687cbc60 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64857fa640 1 -- 172.21.15.139:0/1719460454 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f649c060a40 con 0x5571687cbc60 2024-04-25T09:58:40.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64af024740 1 -- 172.21.15.139:0/1719460454 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f649c0587c0 con 0x5571687cbc60 2024-04-25T09:58:40.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64857fa640 1 -- 172.21.15.139:0/1719460454 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f649c05c910 con 0x5571687cbc60 2024-04-25T09:58:40.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64af024740 10 client.7335 fetch_fsmap learned FSMap version 44 2024-04-25T09:58:40.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64af024740 1 -- 172.21.15.139:0/1719460454 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55716878e130 con 0x5571687cbc60 2024-04-25T09:58:40.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64857fa640 1 -- 172.21.15.139:0/1719460454 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c0587c0 con 0x5571687cbc60 2024-04-25T09:58:40.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64857fa640 1 -- 172.21.15.139:0/1719460454 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f649c066030 con 0x5571687cbc60 2024-04-25T09:58:40.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64af024740 10 client.7335 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T09:58:40.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64af024740 10 client.7335 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:58:40.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64af024740 20 client.7335 populate_metadata read hostname 'smithi139' 2024-04-25T09:58:40.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64af024740 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5571687ca9e0 0x5571687cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64af024740 4 client.7335 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T09:58:40.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64af024740 1 -- 172.21.15.139:0/1719460454 --> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5571687001a0 con 0x5571687ca9e0 2024-04-25T09:58:40.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64aa0be640 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5571687ca9e0 0x5571687cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64857fa640 10 client.7335 ms_handle_connect on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:40.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.159+0000 7f64aa0be640 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5571687ca9e0 0x5571687cadc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef92f5ea7455dd2a server_cookie=ed5a9001f3e2ade in_seq=0 out_seq=0 2024-04-25T09:58:40.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64857fa640 1 -- 172.21.15.139:0/1719460454 <== mds.0 v2:172.21.15.139:6836/1384380923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5571687001a0 con 0x5571687ca9e0 2024-04-25T09:58:40.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64857fa640 10 client.7335 handle_command_reply: tid=0 2024-04-25T09:58:40.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64af024740 1 client.7335 shutdown 2024-04-25T09:58:40.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64af024740 1 -- 172.21.15.139:0/1719460454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 msgr2=0x557168842e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64af024740 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x557168842e70 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f649c058610 tx=0x7f649c08a0d0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64af024740 1 -- 172.21.15.139:0/1719460454 shutdown_connections 2024-04-25T09:58:40.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64af024740 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.139:6836/1384380923,v1:172.21.15.139:6837/1384380923] conn(0x5571687ca9e0 0x5571687cadc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64af024740 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5571687cbc60 0x557168842e70 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64af024740 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5571687ba050 0x557168824a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64af024740 1 --2- 172.21.15.139:0/1719460454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55716878f590 0x55716878e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64857fa640 0 client.7335 ms_handle_reset on v2:172.21.15.139:6836/1384380923 2024-04-25T09:58:40.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64af024740 1 -- 172.21.15.139:0/1719460454 >> 172.21.15.139:0/1719460454 conn(0x5571687576e0 msgr2=0x5571687a11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64af024740 1 -- 172.21.15.139:0/1719460454 shutdown_connections 2024-04-25T09:58:40.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64af024740 1 -- 172.21.15.139:0/1719460454 wait complete. 2024-04-25T09:58:40.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64af024740 20 client.7335 trim_cache size 0 max 16384 2024-04-25T09:58:40.175 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:40.175 INFO:teuthology.orchestra.run.smithi139.stdout:[] 2024-04-25T09:58:40.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64ac322640 1 -- 172.21.15.139:0/1312726498 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f648806b500 msgr2=0x7f648806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64ac322640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f648806b500 0x7f648806d9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f64a0034000 tx=0x7f64a002e000 comp rx=0 tx=0).stop 2024-04-25T09:58:40.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64ac322640 1 -- 172.21.15.139:0/1312726498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64a40afc80 msgr2=0x7f64a409cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64ac322640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64a40afc80 0x7f64a409cd60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f649c00c760 tx=0x7f649c00cc20 comp rx=0 tx=0).stop 2024-04-25T09:58:40.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64ac322640 1 -- 172.21.15.139:0/1312726498 shutdown_connections 2024-04-25T09:58:40.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64ac322640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64a40afc80 0x7f64a409cd60 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64ac322640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f648806b500 0x7f648806d9c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64ac322640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64a40af8a0 0x7f64a40b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64ac322640 1 --2- 172.21.15.139:0/1312726498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64a406b700 0x7f64a41f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.160+0000 7f64ac322640 1 -- 172.21.15.139:0/1312726498 >> 172.21.15.139:0/1312726498 conn(0x7f64a40b2230 msgr2=0x7f64a40aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.161+0000 7f64ac322640 1 -- 172.21.15.139:0/1312726498 shutdown_connections 2024-04-25T09:58:40.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.161+0000 7f64ac322640 1 -- 172.21.15.139:0/1312726498 wait complete. 2024-04-25T09:58:40.178 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T09:58:40.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.271+0000 7f46ddcb3640 1 Processor -- start 2024-04-25T09:58:40.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.271+0000 7f46ddcb3640 1 -- start start 2024-04-25T09:58:40.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.271+0000 7f46ddcb3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d815ac40 0x7f46d806e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.271+0000 7f46ddcb3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d80af6d0 con 0x7f46d806e850 2024-04-25T09:58:40.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.271+0000 7f46ddcb3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d80af9b0 con 0x7f46d815a860 2024-04-25T09:58:40.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.271+0000 7f46ddcb3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d80b02e0 con 0x7f46d815ac40 2024-04-25T09:58:40.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.272+0000 7f46d7fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f46d806e850 0x7f46d80b0770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58968/0 (socket says 172.21.15.139:58968) 2024-04-25T09:58:40.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.272+0000 7f46d7fff640 1 -- 172.21.15.139:0/2236960447 learned_addr learned my addr 172.21.15.139:0/2236960447 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:40.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.272+0000 7f46d6ffd640 1 --2- 172.21.15.139:0/2236960447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d815ac40 0x7f46d806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.272+0000 7f46d67fc640 1 -- 172.21.15.139:0/2236960447 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182923964 0 0) 0x7f46d80af6d0 con 0x7f46d806e850 2024-04-25T09:58:40.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.272+0000 7f46d67fc640 1 -- 172.21.15.139:0/2236960447 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46c0003520 con 0x7f46d806e850 2024-04-25T09:58:40.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.272+0000 7f46d67fc640 1 -- 172.21.15.139:0/2236960447 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895793558 0 0) 0x7f46d80af9b0 con 0x7f46d815a860 2024-04-25T09:58:40.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.272+0000 7f46d67fc640 1 -- 172.21.15.139:0/2236960447 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46d80af6d0 con 0x7f46d815a860 2024-04-25T09:58:40.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.272+0000 7f46d67fc640 1 -- 172.21.15.139:0/2236960447 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1017603461 0 0) 0x7f46c0003520 con 0x7f46d806e850 2024-04-25T09:58:40.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.272+0000 7f46d6ffd640 1 -- 172.21.15.139:0/2236960447 >> v1:172.21.15.150:6789/0 conn(0x7f46d815a860 legacy=0x7f46d80cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.272+0000 7f46d6ffd640 1 -- 172.21.15.139:0/2236960447 >> v1:172.21.15.139:6789/0 conn(0x7f46d806e850 legacy=0x7f46d80b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.272+0000 7f46d6ffd640 1 -- 172.21.15.139:0/2236960447 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d80afc90 con 0x7f46d815ac40 2024-04-25T09:58:40.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.273+0000 7f46d6ffd640 1 --2- 172.21.15.139:0/2236960447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d815ac40 0x7f46d806e310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0097d0 tx=0x7f46cc02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1031cecbab383b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.273+0000 7f46d67fc640 1 -- 172.21.15.139:0/2236960447 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc037070 con 0x7f46d815ac40 2024-04-25T09:58:40.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.273+0000 7f46d67fc640 1 -- 172.21.15.139:0/2236960447 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc02ac30 con 0x7f46d815ac40 2024-04-25T09:58:40.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.273+0000 7f46d67fc640 1 -- 172.21.15.139:0/2236960447 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d00019f0 con 0x7f46d815ac40 2024-04-25T09:58:40.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.274+0000 7f46ddcb3640 1 -- 172.21.15.139:0/2236960447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d815ac40 msgr2=0x7f46d806e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.274+0000 7f46ddcb3640 1 --2- 172.21.15.139:0/2236960447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d815ac40 0x7f46d806e310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0097d0 tx=0x7f46cc02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.274+0000 7f46ddcb3640 1 -- 172.21.15.139:0/2236960447 shutdown_connections 2024-04-25T09:58:40.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.274+0000 7f46ddcb3640 1 --2- 172.21.15.139:0/2236960447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d815ac40 0x7f46d806e310 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.274+0000 7f46ddcb3640 1 -- 172.21.15.139:0/2236960447 >> 172.21.15.139:0/2236960447 conn(0x7f46d8066f20 msgr2=0x7f46d806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.274+0000 7f46ddcb3640 1 -- 172.21.15.139:0/2236960447 shutdown_connections 2024-04-25T09:58:40.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.274+0000 7f46ddcb3640 1 -- 172.21.15.139:0/2236960447 wait complete. 2024-04-25T09:58:40.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.275+0000 7f46ddcb3640 1 Processor -- start 2024-04-25T09:58:40.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.275+0000 7f46ddcb3640 1 -- start start 2024-04-25T09:58:40.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46ddcb3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d806e850 0x7f46d81eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46ddcb3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d815a860 0x7f46d8159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46ddcb3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46d815ac40 0x7f46d814dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46ddcb3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d80af2a0 con 0x7f46d815ac40 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46ddcb3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d80cd3c0 con 0x7f46d806e850 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46ddcb3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d80af420 con 0x7f46d815a860 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46d7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46d815ac40 0x7f46d814dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46d7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46d815ac40 0x7f46d814dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38112/0 (socket says 172.21.15.139:38112) 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46d7fff640 1 -- 172.21.15.139:0/3778172385 learned_addr learned my addr 172.21.15.139:0/3778172385 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46d77fe640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d806e850 0x7f46d81eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46d6ffd640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d815a860 0x7f46d8159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46d7fff640 1 -- 172.21.15.139:0/3778172385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d815a860 msgr2=0x7f46d8159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46d7fff640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d815a860 0x7f46d8159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46d7fff640 1 -- 172.21.15.139:0/3778172385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d806e850 msgr2=0x7f46d81eebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46d7fff640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d806e850 0x7f46d81eebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46d7fff640 1 -- 172.21.15.139:0/3778172385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d81575b0 con 0x7f46d815ac40 2024-04-25T09:58:40.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46d6ffd640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d815a860 0x7f46d8159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:40.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46d7fff640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46d815ac40 0x7f46d814dfd0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f46d0001f60 tx=0x7f46d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46bffff640 1 -- 172.21.15.139:0/3778172385 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d0017070 con 0x7f46d815ac40 2024-04-25T09:58:40.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46bffff640 1 -- 172.21.15.139:0/3778172385 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d0012400 con 0x7f46d815ac40 2024-04-25T09:58:40.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46bffff640 1 -- 172.21.15.139:0/3778172385 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d00127f0 con 0x7f46d815ac40 2024-04-25T09:58:40.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46ddcb3640 1 -- 172.21.15.139:0/3778172385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d8156a30 con 0x7f46d815ac40 2024-04-25T09:58:40.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.276+0000 7f46ddcb3640 1 -- 172.21.15.139:0/3778172385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d8156ff0 con 0x7f46d815ac40 2024-04-25T09:58:40.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.277+0000 7f46ddcb3640 1 -- 172.21.15.139:0/3778172385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d8074520 con 0x7f46d815ac40 2024-04-25T09:58:40.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.278+0000 7f46bffff640 1 -- 172.21.15.139:0/3778172385 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46d001a070 con 0x7f46d815ac40 2024-04-25T09:58:40.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.278+0000 7f46bffff640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f46c406b4d0 0x7f46c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.278+0000 7f46bffff640 1 -- 172.21.15.139:0/3778172385 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f46d008a180 con 0x7f46d815ac40 2024-04-25T09:58:40.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.278+0000 7f46d77fe640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f46c406b4d0 0x7f46c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.279+0000 7f46d77fe640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f46c406b4d0 0x7f46c406d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f46c80011e0 tx=0x7f46c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.283+0000 7f46bffff640 1 -- 172.21.15.139:0/3778172385 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46d00587d0 con 0x7f46d815ac40 2024-04-25T09:58:40.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.486+0000 7f46ddcb3640 1 -- 172.21.15.139:0/3778172385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f46d806c1a0 con 0x7f46d815ac40 2024-04-25T09:58:40.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.486+0000 7f46bffff640 1 -- 172.21.15.139:0/3778172385 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f46d005c870 con 0x7f46d815ac40 2024-04-25T09:58:40.490 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:40.490 INFO:teuthology.orchestra.run.smithi139.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T09:58:40.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.488+0000 7f46ddcb3640 1 -- 172.21.15.139:0/3778172385 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f46c406b4d0 msgr2=0x7f46c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.488+0000 7f46ddcb3640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f46c406b4d0 0x7f46c406d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f46c80011e0 tx=0x7f46c8009040 comp rx=0 tx=0).stop 2024-04-25T09:58:40.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.488+0000 7f46ddcb3640 1 -- 172.21.15.139:0/3778172385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46d815ac40 msgr2=0x7f46d814dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.488+0000 7f46ddcb3640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46d815ac40 0x7f46d814dfd0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f46d0001f60 tx=0x7f46d000cf90 comp rx=0 tx=0).stop 2024-04-25T09:58:40.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.488+0000 7f46ddcb3640 1 -- 172.21.15.139:0/3778172385 shutdown_connections 2024-04-25T09:58:40.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.488+0000 7f46ddcb3640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46d815ac40 0x7f46d814dfd0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.488+0000 7f46ddcb3640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f46c406b4d0 0x7f46c406d990 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.488+0000 7f46ddcb3640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46d815a860 0x7f46d8159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.488+0000 7f46ddcb3640 1 --2- 172.21.15.139:0/3778172385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46d806e850 0x7f46d81eebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.488+0000 7f46ddcb3640 1 -- 172.21.15.139:0/3778172385 >> 172.21.15.139:0/3778172385 conn(0x7f46d8066f20 msgr2=0x7f46d80b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.488+0000 7f46ddcb3640 1 -- 172.21.15.139:0/3778172385 shutdown_connections 2024-04-25T09:58:40.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.488+0000 7f46ddcb3640 1 -- 172.21.15.139:0/3778172385 wait complete. 2024-04-25T09:58:40.502 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T09:58:40.503 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:58:40.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.596+0000 7f2bab522640 1 Processor -- start 2024-04-25T09:58:40.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.596+0000 7f2bab522640 1 -- start start 2024-04-25T09:58:40.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2bab522640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2ba4160e90 0x7f2ba40afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2bab522640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ba406ab60 0x7f2ba406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2bab522640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba40d0400 con 0x7f2ba406fa70 2024-04-25T09:58:40.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2bab522640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba40d14a0 con 0x7f2ba4160e90 2024-04-25T09:58:40.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2bab522640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba40d06e0 con 0x7f2ba406ab60 2024-04-25T09:58:40.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2ba92be640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2ba4160e90 0x7f2ba40afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2ba9abf640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2ba406fa70 0x7f2ba406fec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58974/0 (socket says 172.21.15.139:58974) 2024-04-25T09:58:40.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2ba92be640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2ba4160e90 0x7f2ba40afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38124/0 (socket says 172.21.15.139:38124) 2024-04-25T09:58:40.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2ba9abf640 1 -- 172.21.15.139:0/3143286976 learned_addr learned my addr 172.21.15.139:0/3143286976 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:40.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2ba8abd640 1 --2- 172.21.15.139:0/3143286976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ba406ab60 0x7f2ba406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2ba92be640 1 -- 172.21.15.139:0/3143286976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ba406ab60 msgr2=0x7f2ba406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:40.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2ba92be640 1 --2- 172.21.15.139:0/3143286976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ba406ab60 0x7f2ba406aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2ba92be640 1 -- 172.21.15.139:0/3143286976 >> v1:172.21.15.139:6789/0 conn(0x7f2ba406fa70 legacy=0x7f2ba406fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.597+0000 7f2ba92be640 1 -- 172.21.15.139:0/3143286976 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba40d09c0 con 0x7f2ba4160e90 2024-04-25T09:58:40.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.598+0000 7f2ba92be640 1 --2- 172.21.15.139:0/3143286976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ba4160e90 0x7f2ba40afac0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f2ba00097a0 tx=0x7f2ba002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560f7ae2cc5e977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.598+0000 7f2b9bfff640 1 -- 172.21.15.139:0/3143286976 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0037070 con 0x7f2ba4160e90 2024-04-25T09:58:40.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.598+0000 7f2b9bfff640 1 -- 172.21.15.139:0/3143286976 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba0032440 con 0x7f2ba4160e90 2024-04-25T09:58:40.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.598+0000 7f2b9bfff640 1 -- 172.21.15.139:0/3143286976 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0032810 con 0x7f2ba4160e90 2024-04-25T09:58:40.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.599+0000 7f2bab522640 1 -- 172.21.15.139:0/3143286976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ba4160e90 msgr2=0x7f2ba40afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.599+0000 7f2bab522640 1 --2- 172.21.15.139:0/3143286976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ba4160e90 0x7f2ba40afac0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f2ba00097a0 tx=0x7f2ba002a180 comp rx=0 tx=0).stop 2024-04-25T09:58:40.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.599+0000 7f2bab522640 1 -- 172.21.15.139:0/3143286976 shutdown_connections 2024-04-25T09:58:40.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.599+0000 7f2bab522640 1 --2- 172.21.15.139:0/3143286976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ba406ab60 0x7f2ba406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.599+0000 7f2bab522640 1 --2- 172.21.15.139:0/3143286976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ba4160e90 0x7f2ba40afac0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.599+0000 7f2bab522640 1 -- 172.21.15.139:0/3143286976 >> 172.21.15.139:0/3143286976 conn(0x7f2ba40b23f0 msgr2=0x7f2ba40b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.599+0000 7f2bab522640 1 -- 172.21.15.139:0/3143286976 shutdown_connections 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.599+0000 7f2bab522640 1 -- 172.21.15.139:0/3143286976 wait complete. 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.600+0000 7f2bab522640 1 Processor -- start 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.600+0000 7f2bab522640 1 -- start start 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2bab522640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ba406ab60 0x7f2ba40acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2bab522640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba406fa70 0x7f2ba40aae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2bab522640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ba4160e90 0x7f2ba40ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2bab522640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba40b0000 con 0x7f2ba406ab60 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2bab522640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba40d0f80 con 0x7f2ba406fa70 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2bab522640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba4161270 con 0x7f2ba4160e90 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba92be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ba406ab60 0x7f2ba40acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba92be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ba406ab60 0x7f2ba40acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38132/0 (socket says 172.21.15.139:38132) 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba92be640 1 -- 172.21.15.139:0/3007696945 learned_addr learned my addr 172.21.15.139:0/3007696945 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba9abf640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ba4160e90 0x7f2ba40ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba8abd640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba406fa70 0x7f2ba40aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba92be640 1 -- 172.21.15.139:0/3007696945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ba4160e90 msgr2=0x7f2ba40ab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba92be640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ba4160e90 0x7f2ba40ab3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba92be640 1 -- 172.21.15.139:0/3007696945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba406fa70 msgr2=0x7f2ba40aae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba92be640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba406fa70 0x7f2ba40aae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba92be640 1 -- 172.21.15.139:0/3007696945 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba4144060 con 0x7f2ba406ab60 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba8abd640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba406fa70 0x7f2ba40aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba9abf640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ba4160e90 0x7f2ba40ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2ba92be640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ba406ab60 0x7f2ba40acee0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0035040 tx=0x7f2ba00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2b99ffb640 1 -- 172.21.15.139:0/3007696945 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0037040 con 0x7f2ba406ab60 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2b99ffb640 1 -- 172.21.15.139:0/3007696945 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba0030770 con 0x7f2ba406ab60 2024-04-25T09:58:40.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2bab522640 1 -- 172.21.15.139:0/3007696945 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba4144340 con 0x7f2ba406ab60 2024-04-25T09:58:40.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.601+0000 7f2b99ffb640 1 -- 172.21.15.139:0/3007696945 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0030b80 con 0x7f2ba406ab60 2024-04-25T09:58:40.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.602+0000 7f2bab522640 1 -- 172.21.15.139:0/3007696945 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba4144900 con 0x7f2ba406ab60 2024-04-25T09:58:40.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.602+0000 7f2bab522640 1 -- 172.21.15.139:0/3007696945 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba4074520 con 0x7f2ba406ab60 2024-04-25T09:58:40.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.603+0000 7f2b99ffb640 1 -- 172.21.15.139:0/3007696945 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ba0044020 con 0x7f2ba406ab60 2024-04-25T09:58:40.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.603+0000 7f2b99ffb640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b8806b4d0 0x7f2b8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.603+0000 7f2b99ffb640 1 -- 172.21.15.139:0/3007696945 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2ba00aa9e0 con 0x7f2ba406ab60 2024-04-25T09:58:40.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.603+0000 7f2ba8abd640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b8806b4d0 0x7f2b8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.604+0000 7f2ba8abd640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b8806b4d0 0x7f2b8806d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2ba41f6e00 tx=0x7f2b94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.608+0000 7f2b99ffb640 1 -- 172.21.15.139:0/3007696945 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ba0032e20 con 0x7f2ba406ab60 2024-04-25T09:58:40.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.815+0000 7f2bab522640 1 -- 172.21.15.139:0/3007696945 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2ba40728b0 con 0x7f2ba406ab60 2024-04-25T09:58:40.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.816+0000 7f2b99ffb640 1 -- 172.21.15.139:0/3007696945 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2ba00795f0 con 0x7f2ba406ab60 2024-04-25T09:58:40.820 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:58:40.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.818+0000 7f2bab522640 1 -- 172.21.15.139:0/3007696945 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b8806b4d0 msgr2=0x7f2b8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.818+0000 7f2bab522640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b8806b4d0 0x7f2b8806d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2ba41f6e00 tx=0x7f2b94008040 comp rx=0 tx=0).stop 2024-04-25T09:58:40.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.818+0000 7f2bab522640 1 -- 172.21.15.139:0/3007696945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ba406ab60 msgr2=0x7f2ba40acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.818+0000 7f2bab522640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ba406ab60 0x7f2ba40acee0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0035040 tx=0x7f2ba00046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.818+0000 7f2bab522640 1 -- 172.21.15.139:0/3007696945 shutdown_connections 2024-04-25T09:58:40.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.818+0000 7f2bab522640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b8806b4d0 0x7f2b8806d990 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.818+0000 7f2bab522640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ba4160e90 0x7f2ba40ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.818+0000 7f2bab522640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba406fa70 0x7f2ba40aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.818+0000 7f2bab522640 1 --2- 172.21.15.139:0/3007696945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ba406ab60 0x7f2ba40acee0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.818+0000 7f2bab522640 1 -- 172.21.15.139:0/3007696945 >> 172.21.15.139:0/3007696945 conn(0x7f2ba40b23f0 msgr2=0x7f2ba40cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.818+0000 7f2bab522640 1 -- 172.21.15.139:0/3007696945 shutdown_connections 2024-04-25T09:58:40.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:40.818+0000 7f2bab522640 1 -- 172.21.15.139:0/3007696945 wait complete. 2024-04-25T09:58:40.832 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:58:40.832 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:58:40.832 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T09:58:40.832 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:58:40.832 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:40.833 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:58:40.833 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:58:40.833 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:58:40.833 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:58:40.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:40.833 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T09:58:40.848 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:58:40.848 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:58:40.848 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:58:40.848 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:40.848 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:58:40.848 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:40.848 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:58:40.848 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:40.848 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T09:58:40.848 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42378sec preferred_lft 42378sec 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:40.849 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:58:40.850 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:40.850 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T09:58:40.850 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:40.850 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:58:40.850 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:58:40.850 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T09:58:40.850 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:40.850 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:58:40.850 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:58:40.850 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T09:58:40.850 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:40.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:40.851 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:58:40.911 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:58:40.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:40.912 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T09:58:40.968 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T09:58:40.968 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:58:40.968 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:58:40.968 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:58:40.968 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:58:40.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:40.969 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:58:40.969 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:58:40.969 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:58:40.969 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:58:40.969 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:58:40.969 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:58:41.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:41.053 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:58:41.068 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:58:41.068 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:58:41.068 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:58:41.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:41.068 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T09:58:41.132 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:58:41.132 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:41.187 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:58:41.188 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:58:41.188 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:41.242 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:58:41.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:41.242 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:58:41.323 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:58:41.323 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:58:41.334 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:58:41.334 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:58:41.334 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T09:58:41.334 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T09:58:41.335 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T09:58:41.335 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:58:41.335 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:58:41.335 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:58:41.335 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:58:41.335 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:58:41.336 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:58:41.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:41.337 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:41.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:41.401 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:58:41.457 INFO:teuthology.orchestra.run.smithi139.stdout:client5954 2024-04-25T09:58:41.458 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:58:41.458 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:41.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:41.513 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T09:58:41.568 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:58:41.568 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T09:58:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.648+0000 7f5e44d20f40 1 Processor -- start 2024-04-25T09:58:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.648+0000 7f5e44d20f40 1 -- start start 2024-04-25T09:58:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.648+0000 7f5e44d20f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb9b57f3b0 0x55bb9b5c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.648+0000 7f5e44d20f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bb9b5c4430 0x55bb9b6373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.648+0000 7f5e44d20f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b64e530 0x55bb9b57a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.648+0000 7f5e44d20f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9b57f7c0 con 0x55bb9b64e530 2024-04-25T09:58:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.648+0000 7f5e44d20f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9b523df0 con 0x55bb9b57f3b0 2024-04-25T09:58:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.648+0000 7f5e44d20f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb9b5c4b40 con 0x55bb9b5c4430 2024-04-25T09:58:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.648+0000 7f5e42707640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b64e530 0x55bb9b57a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.648+0000 7f5e42707640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b64e530 0x55bb9b57a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38142/0 (socket says 172.21.15.139:38142) 2024-04-25T09:58:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.648+0000 7f5e42707640 1 -- 172.21.15.139:0/3639429607 learned_addr learned my addr 172.21.15.139:0/3639429607 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e41f06640 1 --2- 172.21.15.139:0/3639429607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb9b57f3b0 0x55bb9b5c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e41705640 1 --2- 172.21.15.139:0/3639429607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bb9b5c4430 0x55bb9b6373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e42707640 1 -- 172.21.15.139:0/3639429607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bb9b5c4430 msgr2=0x55bb9b6373a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e42707640 1 --2- 172.21.15.139:0/3639429607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bb9b5c4430 0x55bb9b6373a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e42707640 1 -- 172.21.15.139:0/3639429607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb9b57f3b0 msgr2=0x55bb9b5c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e42707640 1 --2- 172.21.15.139:0/3639429607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb9b57f3b0 0x55bb9b5c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e42707640 1 -- 172.21.15.139:0/3639429607 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb9b4a9c50 con 0x55bb9b64e530 2024-04-25T09:58:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e41f06640 1 --2- 172.21.15.139:0/3639429607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb9b57f3b0 0x55bb9b5c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e41705640 1 --2- 172.21.15.139:0/3639429607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bb9b5c4430 0x55bb9b6373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e42707640 1 --2- 172.21.15.139:0/3639429607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b64e530 0x55bb9b57a6b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c001e40 tx=0x7f5e3c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a668f70650c2319e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e33fff640 1 -- 172.21.15.139:0/3639429607 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c008070 con 0x55bb9b64e530 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e33fff640 1 -- 172.21.15.139:0/3639429607 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c00d500 con 0x55bb9b64e530 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e33fff640 1 -- 172.21.15.139:0/3639429607 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00be60 con 0x55bb9b64e530 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3639429607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b64e530 msgr2=0x55bb9b57a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e44d20f40 1 --2- 172.21.15.139:0/3639429607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b64e530 0x55bb9b57a6b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c001e40 tx=0x7f5e3c001e70 comp rx=0 tx=0).stop 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3639429607 shutdown_connections 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e44d20f40 1 --2- 172.21.15.139:0/3639429607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b64e530 0x55bb9b57a6b0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e44d20f40 1 --2- 172.21.15.139:0/3639429607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bb9b5c4430 0x55bb9b6373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e44d20f40 1 --2- 172.21.15.139:0/3639429607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb9b57f3b0 0x55bb9b5c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3639429607 >> 172.21.15.139:0/3639429607 conn(0x55bb9b57cbe0 msgr2=0x55bb9b6531f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3639429607 shutdown_connections 2024-04-25T09:58:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.649+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3639429607 wait complete. 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.650+0000 7f5e44d20f40 1 Processor -- start 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.650+0000 7f5e44d20f40 1 -- start start 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e44d20f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b57f3b0 0x55bb9b6e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e44d20f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bb9b5c4430 0x55bb9b6362a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e44d20f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb9b64e530 0x55bb9b632640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e44d20f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9b64f390 con 0x55bb9b57f3b0 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e44d20f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9b637e10 con 0x55bb9b64e530 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e44d20f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb9b57b430 con 0x55bb9b5c4430 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e41f06640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b57f3b0 0x55bb9b6e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e41f06640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b57f3b0 0x55bb9b6e80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38156/0 (socket says 172.21.15.139:38156) 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e41f06640 1 -- 172.21.15.139:0/3203443474 learned_addr learned my addr 172.21.15.139:0/3203443474 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e42707640 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb9b64e530 0x55bb9b632640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e41705640 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bb9b5c4430 0x55bb9b6362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e41f06640 1 -- 172.21.15.139:0/3203443474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bb9b5c4430 msgr2=0x55bb9b6362a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e41f06640 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bb9b5c4430 0x55bb9b6362a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e41f06640 1 -- 172.21.15.139:0/3203443474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb9b64e530 msgr2=0x55bb9b632640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e41f06640 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb9b64e530 0x55bb9b632640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e41f06640 1 -- 172.21.15.139:0/3203443474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb9b636c80 con 0x55bb9b57f3b0 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e41f06640 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b57f3b0 0x55bb9b6e80c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5e34046100 tx=0x7f5e340033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e32ffd640 1 -- 172.21.15.139:0/3203443474 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34063460 con 0x55bb9b57f3b0 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e32ffd640 1 -- 172.21.15.139:0/3203443474 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e34068070 con 0x55bb9b57f3b0 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3203443474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bb9b634990 con 0x55bb9b57f3b0 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e32ffd640 1 -- 172.21.15.139:0/3203443474 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34063800 con 0x55bb9b57f3b0 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.651+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3203443474 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb9b6e8600 con 0x55bb9b57f3b0 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.653+0000 7f5e32ffd640 1 -- 172.21.15.139:0/3203443474 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e34061030 con 0x55bb9b57f3b0 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.653+0000 7f5e32ffd640 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5e24069eb0 0x7f5e2406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.653+0000 7f5e32ffd640 1 -- 172.21.15.139:0/3203443474 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5e340cd5c0 con 0x55bb9b57f3b0 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.653+0000 7f5e41705640 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5e24069eb0 0x7f5e2406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.653+0000 7f5e44d20f40 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55bb9b6367e0 0x55bb9b635980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.653+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3203443474 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77) v9 -- 0x55bb9b635ec0 con 0x55bb9b6367e0 2024-04-25T09:58:41.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.653+0000 7f5e42707640 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55bb9b6367e0 0x55bb9b635980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.654+0000 7f5e41705640 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5e24069eb0 0x7f5e2406c370 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55bb9b5c3580 tx=0x7f5e38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:41.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.654+0000 7f5e42707640 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55bb9b6367e0 0x55bb9b635980 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:41.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.655+0000 7f5e42707640 1 -- 172.21.15.139:0/3203443474 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v77'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5e3c00d900 con 0x55bb9b6367e0 2024-04-25T09:58:41.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.655+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55bb9b6367e0 msgr2=0x55bb9b635980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.655+0000 7f5e44d20f40 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55bb9b6367e0 0x55bb9b635980 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.655+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5e24069eb0 msgr2=0x7f5e2406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.655+0000 7f5e44d20f40 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5e24069eb0 0x7f5e2406c370 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55bb9b5c3580 tx=0x7f5e38009040 comp rx=0 tx=0).stop 2024-04-25T09:58:41.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.655+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b57f3b0 msgr2=0x55bb9b6e80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.655+0000 7f5e44d20f40 1 --2- 172.21.15.139:0/3203443474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bb9b57f3b0 0x55bb9b6e80c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5e34046100 tx=0x7f5e340033a0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.655+0000 7f5e42707640 1 -- 172.21.15.139:0/3203443474 reap_dead start 2024-04-25T09:58:41.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.655+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3203443474 shutdown_connections 2024-04-25T09:58:41.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.655+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3203443474 >> 172.21.15.139:0/3203443474 conn(0x55bb9b57cbe0 msgr2=0x55bb9b5bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:41.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.655+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3203443474 shutdown_connections 2024-04-25T09:58:41.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.655+0000 7f5e44d20f40 1 -- 172.21.15.139:0/3203443474 wait complete. 2024-04-25T09:58:41.665 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:58:41.665 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:58:41.748 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:58:41.787 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:58:41.787 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:58:41.787 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:41.803 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:58:41.803 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:41.858 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:41.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.992+0000 7f90fcaf5640 1 Processor -- start 2024-04-25T09:58:41.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.992+0000 7f90fcaf5640 1 -- start start 2024-04-25T09:58:41.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.992+0000 7f90fcaf5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f90f80d1080 0x7f90f80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.992+0000 7f90fcaf5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f80cf250 con 0x7f90f815dc60 2024-04-25T09:58:41.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.992+0000 7f90fcaf5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f80ce9b0 con 0x7f90f8161850 2024-04-25T09:58:41.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.992+0000 7f90fcaf5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f80d3730 con 0x7f90f80d1080 2024-04-25T09:58:41.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.993+0000 7f90effff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f90f80d1080 0x7f90f80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.993+0000 7f90f659c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f90f815dc60 0x7f90f80d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58992/0 (socket says 172.21.15.139:58992) 2024-04-25T09:58:41.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.993+0000 7f90f659c640 1 -- 172.21.15.139:0/1612039467 learned_addr learned my addr 172.21.15.139:0/1612039467 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:41.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.993+0000 7f90effff640 1 -- 172.21.15.139:0/1612039467 >> v1:172.21.15.150:6789/0 conn(0x7f90f8161850 legacy=0x7f90f80d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:41.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.993+0000 7f90effff640 1 -- 172.21.15.139:0/1612039467 >> v1:172.21.15.139:6789/0 conn(0x7f90f815dc60 legacy=0x7f90f80d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.993+0000 7f90effff640 1 -- 172.21.15.139:0/1612039467 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f80cec90 con 0x7f90f80d1080 2024-04-25T09:58:41.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.993+0000 7f90f5d9b640 1 -- 172.21.15.139:0/1612039467 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187428081 0 0) 0x7f90e8001110 con 0x7f90f815dc60 2024-04-25T09:58:41.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.994+0000 7f90effff640 1 --2- 172.21.15.139:0/1612039467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90f80d1080 0x7f90f80b6c10 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f90e00097a0 tx=0x7f90e002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10db5a931ec1ba6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:41.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.994+0000 7f90f5d9b640 1 -- 172.21.15.139:0/1612039467 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e0037070 con 0x7f90f80d1080 2024-04-25T09:58:41.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.994+0000 7f90f5d9b640 1 -- 172.21.15.139:0/1612039467 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e0032440 con 0x7f90f80d1080 2024-04-25T09:58:41.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.994+0000 7f90f5d9b640 1 -- 172.21.15.139:0/1612039467 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e0032810 con 0x7f90f80d1080 2024-04-25T09:58:41.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.994+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1612039467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90f80d1080 msgr2=0x7f90f80b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.994+0000 7f90fcaf5640 1 --2- 172.21.15.139:0/1612039467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90f80d1080 0x7f90f80b6c10 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f90e00097a0 tx=0x7f90e002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.994+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1612039467 shutdown_connections 2024-04-25T09:58:42.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.994+0000 7f90fcaf5640 1 --2- 172.21.15.139:0/1612039467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90f80d1080 0x7f90f80b6c10 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.994+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1612039467 >> 172.21.15.139:0/1612039467 conn(0x7f90f80b5910 msgr2=0x7f90f80b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:42.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.994+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1612039467 shutdown_connections 2024-04-25T09:58:42.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.994+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1612039467 wait complete. 2024-04-25T09:58:42.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.996+0000 7f90fcaf5640 1 Processor -- start 2024-04-25T09:58:42.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.996+0000 7f90fcaf5640 1 -- start start 2024-04-25T09:58:42.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.996+0000 7f90fcaf5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90f80d1080 0x7f90f815ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.996+0000 7f90fcaf5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f815dc60 0x7f90f806c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.996+0000 7f90fcaf5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90f8161850 0x7f90f806c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.996+0000 7f90fcaf5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f815e040 con 0x7f90f8161850 2024-04-25T09:58:42.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.996+0000 7f90fcaf5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f80d1fc0 con 0x7f90f815dc60 2024-04-25T09:58:42.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.996+0000 7f90fcaf5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f815bd90 con 0x7f90f80d1080 2024-04-25T09:58:42.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.996+0000 7f90f6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90f8161850 0x7f90f806c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.996+0000 7f90f6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90f8161850 0x7f90f806c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38168/0 (socket says 172.21.15.139:38168) 2024-04-25T09:58:42.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.996+0000 7f90f6d9d640 1 -- 172.21.15.139:0/1260204979 learned_addr learned my addr 172.21.15.139:0/1260204979 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:42.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.996+0000 7f90f659c640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90f80d1080 0x7f90f815ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90effff640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f815dc60 0x7f90f806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90f6d9d640 1 -- 172.21.15.139:0/1260204979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90f80d1080 msgr2=0x7f90f815ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90f6d9d640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90f80d1080 0x7f90f815ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90f6d9d640 1 -- 172.21.15.139:0/1260204979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f815dc60 msgr2=0x7f90f806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90f6d9d640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f815dc60 0x7f90f806c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90f6d9d640 1 -- 172.21.15.139:0/1260204979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f815ef50 con 0x7f90f8161850 2024-04-25T09:58:42.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90effff640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f815dc60 0x7f90f806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:42.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90f6d9d640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90f8161850 0x7f90f806c5b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f90f000b970 tx=0x7f90f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90eeffd640 1 -- 172.21.15.139:0/1260204979 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0019070 con 0x7f90f8161850 2024-04-25T09:58:42.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90eeffd640 1 -- 172.21.15.139:0/1260204979 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f000cb40 con 0x7f90f8161850 2024-04-25T09:58:42.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1260204979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f815b460 con 0x7f90f8161850 2024-04-25T09:58:42.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90eeffd640 1 -- 172.21.15.139:0/1260204979 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0014440 con 0x7f90f8161850 2024-04-25T09:58:42.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.997+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1260204979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f815ba20 con 0x7f90f8161850 2024-04-25T09:58:42.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.998+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1260204979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f8074520 con 0x7f90f8161850 2024-04-25T09:58:42.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.998+0000 7f90eeffd640 1 -- 172.21.15.139:0/1260204979 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90f001c070 con 0x7f90f8161850 2024-04-25T09:58:42.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.999+0000 7f90eeffd640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f90d806b4d0 0x7f90d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.999+0000 7f90eeffd640 1 -- 172.21.15.139:0/1260204979 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f90f0089f10 con 0x7f90f8161850 2024-04-25T09:58:42.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.999+0000 7f90f659c640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f90d806b4d0 0x7f90d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:41.999+0000 7f90f659c640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f90d806b4d0 0x7f90d806d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f90e8000f60 tx=0x7f90e8007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.003+0000 7f90eeffd640 1 -- 172.21.15.139:0/1260204979 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90f0058560 con 0x7f90f8161850 2024-04-25T09:58:42.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.173+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1260204979 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90f815bf10 con 0x7f90f8161850 2024-04-25T09:58:42.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.174+0000 7f90eeffd640 1 -- 172.21.15.139:0/1260204979 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+17124 (secure 0 0 0) 0x7f90f005c600 con 0x7f90f8161850 2024-04-25T09:58:42.178 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:42.178 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":77,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:57:29.527441+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:18.681554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:19.409382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6834/1932986598":"2024-04-26T09:57:29.527412+0000","172.21.15.150:6832/1932986598":"2024-04-26T09:57:29.527412+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:42.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.176+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1260204979 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f90d806b4d0 msgr2=0x7f90d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.176+0000 7f90fcaf5640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f90d806b4d0 0x7f90d806d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f90e8000f60 tx=0x7f90e8007b50 comp rx=0 tx=0).stop 2024-04-25T09:58:42.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.176+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1260204979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90f8161850 msgr2=0x7f90f806c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.176+0000 7f90fcaf5640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90f8161850 0x7f90f806c5b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f90f000b970 tx=0x7f90f000be30 comp rx=0 tx=0).stop 2024-04-25T09:58:42.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.176+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1260204979 shutdown_connections 2024-04-25T09:58:42.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.176+0000 7f90fcaf5640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90f8161850 0x7f90f806c5b0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.176+0000 7f90fcaf5640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f90d806b4d0 0x7f90d806d990 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.176+0000 7f90fcaf5640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f815dc60 0x7f90f806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.176+0000 7f90fcaf5640 1 --2- 172.21.15.139:0/1260204979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90f80d1080 0x7f90f815ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.176+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1260204979 >> 172.21.15.139:0/1260204979 conn(0x7f90f80b5910 msgr2=0x7f90f8073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:42.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.176+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1260204979 shutdown_connections 2024-04-25T09:58:42.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.176+0000 7f90fcaf5640 1 -- 172.21.15.139:0/1260204979 wait complete. 2024-04-25T09:58:42.191 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:57:18.681554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:57:19.409382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:58:42.192 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:58:42.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.289+0000 7f1b5a29e640 1 Processor -- start 2024-04-25T09:58:42.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.289+0000 7f1b5a29e640 1 -- start start 2024-04-25T09:58:42.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.289+0000 7f1b5a29e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b54145230 0x7f1b5415be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.289+0000 7f1b5a29e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1b540718d0 0x7f1b5415da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.289+0000 7f1b5a29e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5415c960 con 0x7f1b5415c3d0 2024-04-25T09:58:42.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.289+0000 7f1b5a29e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5415cc40 con 0x7f1b540718d0 2024-04-25T09:58:42.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.289+0000 7f1b5a29e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5415d230 con 0x7f1b54145230 2024-04-25T09:58:42.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.290+0000 7f1b53fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b54145230 0x7f1b5415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.290+0000 7f1b5883b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1b540718d0 0x7f1b5415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.290+0000 7f1b53fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b54145230 0x7f1b5415be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:48692/0 (socket says 172.21.15.139:48692) 2024-04-25T09:58:42.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.290+0000 7f1b537fe640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f1b5415c3d0 0x7f1b540711c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:40158/0 (socket says 172.21.15.139:40158) 2024-04-25T09:58:42.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.290+0000 7f1b53fff640 1 -- 172.21.15.139:0/2198306627 learned_addr learned my addr 172.21.15.139:0/2198306627 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:42.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.290+0000 7f1b5883b640 1 -- 172.21.15.139:0/2198306627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b54145230 msgr2=0x7f1b5415be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:42.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.290+0000 7f1b5883b640 1 --2- 172.21.15.139:0/2198306627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b54145230 0x7f1b5415be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.290+0000 7f1b5883b640 1 -- 172.21.15.139:0/2198306627 >> v1:172.21.15.150:6789/0 conn(0x7f1b5415c3d0 legacy=0x7f1b540711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.290+0000 7f1b5883b640 1 -- 172.21.15.139:0/2198306627 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5415d510 con 0x7f1b540718d0 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.290+0000 7f1b53fff640 1 --2- 172.21.15.139:0/2198306627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b54145230 0x7f1b5415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.291+0000 7f1b5883b640 1 --2- 172.21.15.139:0/2198306627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b540718d0 0x7f1b5415da30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009d90 tx=0x7f1b4c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a509382c6c6bb8d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.291+0000 7f1b52ffd640 1 -- 172.21.15.139:0/2198306627 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c031600 con 0x7f1b540718d0 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.291+0000 7f1b52ffd640 1 -- 172.21.15.139:0/2198306627 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4c0317a0 con 0x7f1b540718d0 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.291+0000 7f1b52ffd640 1 -- 172.21.15.139:0/2198306627 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c031b70 con 0x7f1b540718d0 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.292+0000 7f1b5a29e640 1 -- 172.21.15.139:0/2198306627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b540718d0 msgr2=0x7f1b5415da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.292+0000 7f1b5a29e640 1 --2- 172.21.15.139:0/2198306627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b540718d0 0x7f1b5415da30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009d90 tx=0x7f1b4c02a5d0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.292+0000 7f1b5a29e640 1 -- 172.21.15.139:0/2198306627 shutdown_connections 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.292+0000 7f1b5a29e640 1 --2- 172.21.15.139:0/2198306627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b540718d0 0x7f1b5415da30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.292+0000 7f1b5a29e640 1 --2- 172.21.15.139:0/2198306627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b54145230 0x7f1b5415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.292+0000 7f1b5a29e640 1 -- 172.21.15.139:0/2198306627 >> 172.21.15.139:0/2198306627 conn(0x7f1b54067000 msgr2=0x7f1b5406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.292+0000 7f1b5a29e640 1 -- 172.21.15.139:0/2198306627 shutdown_connections 2024-04-25T09:58:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.292+0000 7f1b5a29e640 1 -- 172.21.15.139:0/2198306627 wait complete. 2024-04-25T09:58:42.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.293+0000 7f1b5a29e640 1 Processor -- start 2024-04-25T09:58:42.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.293+0000 7f1b5a29e640 1 -- start start 2024-04-25T09:58:42.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b5a29e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b540718d0 0x7f1b5415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b5a29e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b54145230 0x7f1b540ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b5a29e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b5415c3d0 0x7f1b541324e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b5a29e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5415e0f0 con 0x7f1b540718d0 2024-04-25T09:58:42.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b5a29e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5415df70 con 0x7f1b54145230 2024-04-25T09:58:42.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b5a29e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5415e270 con 0x7f1b5415c3d0 2024-04-25T09:58:42.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b53fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b540718d0 0x7f1b5415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b53fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b540718d0 0x7f1b5415b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38172/0 (socket says 172.21.15.139:38172) 2024-04-25T09:58:42.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b53fff640 1 -- 172.21.15.139:0/4007533461 learned_addr learned my addr 172.21.15.139:0/4007533461 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:42.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b537fe640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b54145230 0x7f1b540ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b5883b640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b5415c3d0 0x7f1b541324e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b53fff640 1 -- 172.21.15.139:0/4007533461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b5415c3d0 msgr2=0x7f1b541324e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b53fff640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b5415c3d0 0x7f1b541324e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b53fff640 1 -- 172.21.15.139:0/4007533461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b54145230 msgr2=0x7f1b540ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b53fff640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b54145230 0x7f1b540ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b53fff640 1 -- 172.21.15.139:0/4007533461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b54132ec0 con 0x7f1b540718d0 2024-04-25T09:58:42.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b537fe640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b54145230 0x7f1b540ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:42.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b53fff640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b540718d0 0x7f1b5415b0c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400e9d0 tx=0x7f1b4400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b50ff9640 1 -- 172.21.15.139:0/4007533461 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44017070 con 0x7f1b540718d0 2024-04-25T09:58:42.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b50ff9640 1 -- 172.21.15.139:0/4007533461 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4400ca60 con 0x7f1b540718d0 2024-04-25T09:58:42.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b5a29e640 1 -- 172.21.15.139:0/4007533461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b541331a0 con 0x7f1b540718d0 2024-04-25T09:58:42.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b50ff9640 1 -- 172.21.15.139:0/4007533461 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400ce10 con 0x7f1b540718d0 2024-04-25T09:58:42.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.294+0000 7f1b5a29e640 1 -- 172.21.15.139:0/4007533461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b540acee0 con 0x7f1b540718d0 2024-04-25T09:58:42.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.295+0000 7f1b5a29e640 1 -- 172.21.15.139:0/4007533461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5406af00 con 0x7f1b540718d0 2024-04-25T09:58:42.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.296+0000 7f1b50ff9640 1 -- 172.21.15.139:0/4007533461 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b4401a070 con 0x7f1b540718d0 2024-04-25T09:58:42.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.296+0000 7f1b50ff9640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b4006b4d0 0x7f1b4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.296+0000 7f1b50ff9640 1 -- 172.21.15.139:0/4007533461 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1b4408a700 con 0x7f1b540718d0 2024-04-25T09:58:42.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.296+0000 7f1b537fe640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b4006b4d0 0x7f1b4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.297+0000 7f1b537fe640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b4006b4d0 0x7f1b4006d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1b48007fd0 tx=0x7f1b48007d40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.301+0000 7f1b50ff9640 1 -- 172.21.15.139:0/4007533461 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b5406af00 con 0x7f1b540718d0 2024-04-25T09:58:42.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.468+0000 7f1b5a29e640 1 -- 172.21.15.139:0/4007533461 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b54074520 con 0x7f1b540718d0 2024-04-25T09:58:42.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.469+0000 7f1b50ff9640 1 -- 172.21.15.139:0/4007533461 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b440593a0 con 0x7f1b540718d0 2024-04-25T09:58:42.473 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:58:42.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.471+0000 7f1b5a29e640 1 -- 172.21.15.139:0/4007533461 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b4006b4d0 msgr2=0x7f1b4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.471+0000 7f1b5a29e640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b4006b4d0 0x7f1b4006d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1b48007fd0 tx=0x7f1b48007d40 comp rx=0 tx=0).stop 2024-04-25T09:58:42.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.471+0000 7f1b5a29e640 1 -- 172.21.15.139:0/4007533461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b540718d0 msgr2=0x7f1b5415b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.471+0000 7f1b5a29e640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b540718d0 0x7f1b5415b0c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400e9d0 tx=0x7f1b4400ee90 comp rx=0 tx=0).stop 2024-04-25T09:58:42.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.471+0000 7f1b5a29e640 1 -- 172.21.15.139:0/4007533461 shutdown_connections 2024-04-25T09:58:42.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.471+0000 7f1b5a29e640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b4006b4d0 0x7f1b4006d990 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.471+0000 7f1b5a29e640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b5415c3d0 0x7f1b541324e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.471+0000 7f1b5a29e640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b54145230 0x7f1b540ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.471+0000 7f1b5a29e640 1 --2- 172.21.15.139:0/4007533461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b540718d0 0x7f1b5415b0c0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.471+0000 7f1b5a29e640 1 -- 172.21.15.139:0/4007533461 >> 172.21.15.139:0/4007533461 conn(0x7f1b54067000 msgr2=0x7f1b540b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:42.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.471+0000 7f1b5a29e640 1 -- 172.21.15.139:0/4007533461 shutdown_connections 2024-04-25T09:58:42.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.471+0000 7f1b5a29e640 1 -- 172.21.15.139:0/4007533461 wait complete. 2024-04-25T09:58:42.485 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:58:42.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.582+0000 7f2d8f41b640 1 Processor -- start 2024-04-25T09:58:42.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.582+0000 7f2d8f41b640 1 -- start start 2024-04-25T09:58:42.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8f41b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d880afc00 0x7f2d8806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8f41b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2d8815cb20 0x7f2d880cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8f41b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2d880cfeb0 0x7f2d8806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8f41b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d880d0320 con 0x7f2d8815cb20 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8f41b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d880703c0 con 0x7f2d880cfeb0 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8f41b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d880cd820 con 0x7f2d880afc00 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8c9b6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2d8815cb20 0x7f2d880cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8c9b6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2d8815cb20 0x7f2d880cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38184/0 (socket says 172.21.15.139:38184) 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8c9b6640 1 -- 172.21.15.139:0/1976781109 learned_addr learned my addr 172.21.15.139:0/1976781109 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8d9b8640 1 --2- 172.21.15.139:0/1976781109 >> v2:172.21.15.150:3300/0 conn(0x7f2d880cfeb0 0x7f2d8806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8d1b7640 1 --2- 172.21.15.139:0/1976781109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d880afc00 0x7f2d8806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8c9b6640 1 -- 172.21.15.139:0/1976781109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d880afc00 msgr2=0x7f2d8806fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8c9b6640 1 --2- 172.21.15.139:0/1976781109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d880afc00 0x7f2d8806fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8c9b6640 1 -- 172.21.15.139:0/1976781109 >> v2:172.21.15.150:3300/0 conn(0x7f2d880cfeb0 msgr2=0x7f2d8806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8c9b6640 1 --2- 172.21.15.139:0/1976781109 >> v2:172.21.15.150:3300/0 conn(0x7f2d880cfeb0 0x7f2d8806ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.583+0000 7f2d8c9b6640 1 -- 172.21.15.139:0/1976781109 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d880ccef0 con 0x7f2d8815cb20 2024-04-25T09:58:42.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.584+0000 7f2d8c9b6640 1 --2- 172.21.15.139:0/1976781109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d8815cb20 0x7f2d880cf970 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2d8806cb20 tx=0x7f2d7802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedaad33d1db221d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.584+0000 7f2d7ffff640 1 -- 172.21.15.139:0/1976781109 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78037070 con 0x7f2d8815cb20 2024-04-25T09:58:42.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.584+0000 7f2d7ffff640 1 -- 172.21.15.139:0/1976781109 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d78032440 con 0x7f2d8815cb20 2024-04-25T09:58:42.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.584+0000 7f2d7ffff640 1 -- 172.21.15.139:0/1976781109 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78032810 con 0x7f2d8815cb20 2024-04-25T09:58:42.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.584+0000 7f2d8f41b640 1 -- 172.21.15.139:0/1976781109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d8815cb20 msgr2=0x7f2d880cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.584+0000 7f2d8f41b640 1 --2- 172.21.15.139:0/1976781109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d8815cb20 0x7f2d880cf970 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2d8806cb20 tx=0x7f2d7802a140 comp rx=0 tx=0).stop 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.585+0000 7f2d8f41b640 1 -- 172.21.15.139:0/1976781109 shutdown_connections 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.585+0000 7f2d8f41b640 1 --2- 172.21.15.139:0/1976781109 >> v2:172.21.15.150:3300/0 conn(0x7f2d880cfeb0 0x7f2d8806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.585+0000 7f2d8f41b640 1 --2- 172.21.15.139:0/1976781109 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d8815cb20 0x7f2d880cf970 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.585+0000 7f2d8f41b640 1 --2- 172.21.15.139:0/1976781109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d880afc00 0x7f2d8806fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.585+0000 7f2d8f41b640 1 -- 172.21.15.139:0/1976781109 >> 172.21.15.139:0/1976781109 conn(0x7f2d880b2230 msgr2=0x7f2d880b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.585+0000 7f2d8f41b640 1 -- 172.21.15.139:0/1976781109 shutdown_connections 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.585+0000 7f2d8f41b640 1 -- 172.21.15.139:0/1976781109 wait complete. 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.586+0000 7f2d8f41b640 1 Processor -- start 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.586+0000 7f2d8f41b640 1 -- start start 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.586+0000 7f2d8f41b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d880afc00 0x7f2d8814eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.586+0000 7f2d8f41b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d880cfeb0 0x7f2d881585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8f41b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d8815cb20 0x7f2d8814e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8f41b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d880cd4b0 con 0x7f2d8815cb20 2024-04-25T09:58:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8f41b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d880affe0 con 0x7f2d880afc00 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8f41b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d88070060 con 0x7f2d880cfeb0 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8d9b8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d8815cb20 0x7f2d8814e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8d9b8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d8815cb20 0x7f2d8814e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38196/0 (socket says 172.21.15.139:38196) 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8d9b8640 1 -- 172.21.15.139:0/470184836 learned_addr learned my addr 172.21.15.139:0/470184836 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8d1b7640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d880afc00 0x7f2d8814eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8c9b6640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d880cfeb0 0x7f2d881585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8d9b8640 1 -- 172.21.15.139:0/470184836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d880cfeb0 msgr2=0x7f2d881585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8d9b8640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d880cfeb0 0x7f2d881585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8d9b8640 1 -- 172.21.15.139:0/470184836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d880afc00 msgr2=0x7f2d8814eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8d9b8640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d880afc00 0x7f2d8814eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8d9b8640 1 -- 172.21.15.139:0/470184836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d880b5290 con 0x7f2d8815cb20 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8c9b6640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d880cfeb0 0x7f2d881585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8d1b7640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d880afc00 0x7f2d8814eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8d9b8640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d8815cb20 0x7f2d8814e810 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f2d80007e40 tx=0x7f2d8000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d7dffb640 1 -- 172.21.15.139:0/470184836 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80014620 con 0x7f2d8815cb20 2024-04-25T09:58:42.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d7dffb640 1 -- 172.21.15.139:0/470184836 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d80019070 con 0x7f2d8815cb20 2024-04-25T09:58:42.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8f41b640 1 -- 172.21.15.139:0/470184836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8815b270 con 0x7f2d8815cb20 2024-04-25T09:58:42.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d7dffb640 1 -- 172.21.15.139:0/470184836 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80014a20 con 0x7f2d8815cb20 2024-04-25T09:58:42.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.587+0000 7f2d8f41b640 1 -- 172.21.15.139:0/470184836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8815b830 con 0x7f2d8815cb20 2024-04-25T09:58:42.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.588+0000 7f2d8f41b640 1 -- 172.21.15.139:0/470184836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d88074520 con 0x7f2d8815cb20 2024-04-25T09:58:42.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.589+0000 7f2d7dffb640 1 -- 172.21.15.139:0/470184836 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d80014bc0 con 0x7f2d8815cb20 2024-04-25T09:58:42.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.589+0000 7f2d7dffb640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d6c06b500 0x7f2d6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.589+0000 7f2d7dffb640 1 -- 172.21.15.139:0/470184836 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2d8008ac10 con 0x7f2d8815cb20 2024-04-25T09:58:42.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.589+0000 7f2d8d1b7640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d6c06b500 0x7f2d6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.590+0000 7f2d8d1b7640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d6c06b500 0x7f2d6c06d9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2d84006fd0 tx=0x7f2d84008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.594+0000 7f2d7dffb640 1 -- 172.21.15.139:0/470184836 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d880b5290 con 0x7f2d8815cb20 2024-04-25T09:58:42.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.761+0000 7f2d8f41b640 1 -- 172.21.15.139:0/470184836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d880cba50 con 0x7f2d8815cb20 2024-04-25T09:58:42.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.761+0000 7f2d7dffb640 1 -- 172.21.15.139:0/470184836 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d800598c0 con 0x7f2d8815cb20 2024-04-25T09:58:42.765 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:58:42.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.763+0000 7f2d8f41b640 1 -- 172.21.15.139:0/470184836 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d6c06b500 msgr2=0x7f2d6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.763+0000 7f2d8f41b640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d6c06b500 0x7f2d6c06d9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2d84006fd0 tx=0x7f2d84008040 comp rx=0 tx=0).stop 2024-04-25T09:58:42.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.763+0000 7f2d8f41b640 1 -- 172.21.15.139:0/470184836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d8815cb20 msgr2=0x7f2d8814e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.764+0000 7f2d8f41b640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d8815cb20 0x7f2d8814e810 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f2d80007e40 tx=0x7f2d8000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.764+0000 7f2d8f41b640 1 -- 172.21.15.139:0/470184836 shutdown_connections 2024-04-25T09:58:42.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.764+0000 7f2d8f41b640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d8815cb20 0x7f2d8814e810 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.764+0000 7f2d8f41b640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d6c06b500 0x7f2d6c06d9c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.764+0000 7f2d8f41b640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d880cfeb0 0x7f2d881585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.764+0000 7f2d8f41b640 1 --2- 172.21.15.139:0/470184836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d880afc00 0x7f2d8814eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.764+0000 7f2d8f41b640 1 -- 172.21.15.139:0/470184836 >> 172.21.15.139:0/470184836 conn(0x7f2d880b2230 msgr2=0x7f2d8806f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:42.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.764+0000 7f2d8f41b640 1 -- 172.21.15.139:0/470184836 shutdown_connections 2024-04-25T09:58:42.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.764+0000 7f2d8f41b640 1 -- 172.21.15.139:0/470184836 wait complete. 2024-04-25T09:58:42.778 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:58:42.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.873+0000 7fc902932640 1 Processor -- start 2024-04-25T09:58:42.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.873+0000 7fc902932640 1 -- start start 2024-04-25T09:58:42.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc902932640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8fc0cd610 0x7fc8fc0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc902932640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc8fc09ad40 0x7fc8fc0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc902932640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8fc099ca0 con 0x7fc8fc1618c0 2024-04-25T09:58:42.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc902932640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8fc099a50 con 0x7fc8fc09ad40 2024-04-25T09:58:42.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc902932640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc099f80 con 0x7fc8fc0cd610 2024-04-25T09:58:42.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc8fb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc8fc09ad40 0x7fc8fc0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc900ecf640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc8fc1618c0 0x7fc8fc06be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59008/0 (socket says 172.21.15.139:59008) 2024-04-25T09:58:42.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc8fb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc8fc09ad40 0x7fc8fc0d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38208/0 (socket says 172.21.15.139:38208) 2024-04-25T09:58:42.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc900ecf640 1 -- 172.21.15.139:0/1002025783 learned_addr learned my addr 172.21.15.139:0/1002025783 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:42.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc8fbfff640 1 --2- 172.21.15.139:0/1002025783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8fc0cd610 0x7fc8fc0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc8fb7fe640 1 -- 172.21.15.139:0/1002025783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8fc0cd610 msgr2=0x7fc8fc0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:42.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc8fb7fe640 1 --2- 172.21.15.139:0/1002025783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8fc0cd610 0x7fc8fc0d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc8fb7fe640 1 -- 172.21.15.139:0/1002025783 >> v1:172.21.15.139:6789/0 conn(0x7fc8fc1618c0 legacy=0x7fc8fc06be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.874+0000 7fc8fb7fe640 1 -- 172.21.15.139:0/1002025783 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc09a260 con 0x7fc8fc09ad40 2024-04-25T09:58:42.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.875+0000 7fc8fb7fe640 1 --2- 172.21.15.139:0/1002025783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc09ad40 0x7fc8fc0d08b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00097a0 tx=0x7fc8f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7735409ab57708a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.875+0000 7fc8faffd640 1 -- 172.21.15.139:0/1002025783 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0037070 con 0x7fc8fc09ad40 2024-04-25T09:58:42.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.875+0000 7fc8faffd640 1 -- 172.21.15.139:0/1002025783 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f0032440 con 0x7fc8fc09ad40 2024-04-25T09:58:42.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.875+0000 7fc8faffd640 1 -- 172.21.15.139:0/1002025783 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0032810 con 0x7fc8fc09ad40 2024-04-25T09:58:42.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.875+0000 7fc902932640 1 -- 172.21.15.139:0/1002025783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc09ad40 msgr2=0x7fc8fc0d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.875+0000 7fc902932640 1 --2- 172.21.15.139:0/1002025783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc09ad40 0x7fc8fc0d08b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00097a0 tx=0x7fc8f002a210 comp rx=0 tx=0).stop 2024-04-25T09:58:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.875+0000 7fc902932640 1 -- 172.21.15.139:0/1002025783 shutdown_connections 2024-04-25T09:58:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.875+0000 7fc902932640 1 --2- 172.21.15.139:0/1002025783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc09ad40 0x7fc8fc0d08b0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.875+0000 7fc902932640 1 --2- 172.21.15.139:0/1002025783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8fc0cd610 0x7fc8fc0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.875+0000 7fc902932640 1 -- 172.21.15.139:0/1002025783 >> 172.21.15.139:0/1002025783 conn(0x7fc8fc0b31f0 msgr2=0x7fc8fc0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.875+0000 7fc902932640 1 -- 172.21.15.139:0/1002025783 shutdown_connections 2024-04-25T09:58:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.875+0000 7fc902932640 1 -- 172.21.15.139:0/1002025783 wait complete. 2024-04-25T09:58:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.877+0000 7fc902932640 1 Processor -- start 2024-04-25T09:58:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.877+0000 7fc902932640 1 -- start start 2024-04-25T09:58:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.877+0000 7fc902932640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8fc09ad40 0x7fc8fc1496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.877+0000 7fc902932640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc0cd610 0x7fc8fc159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.877+0000 7fc902932640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8fc1618c0 0x7fc8fc1e7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.877+0000 7fc902932640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc0cd9f0 con 0x7fc8fc0cd610 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.877+0000 7fc902932640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc09a820 con 0x7fc8fc1618c0 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.877+0000 7fc902932640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc0cdb70 con 0x7fc8fc09ad40 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8fb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc0cd610 0x7fc8fc159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8fb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc0cd610 0x7fc8fc159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38216/0 (socket says 172.21.15.139:38216) 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8fb7fe640 1 -- 172.21.15.139:0/2791271124 learned_addr learned my addr 172.21.15.139:0/2791271124 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc900ecf640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8fc1618c0 0x7fc8fc1e7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8fbfff640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8fc09ad40 0x7fc8fc1496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8fb7fe640 1 -- 172.21.15.139:0/2791271124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8fc09ad40 msgr2=0x7fc8fc1496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8fb7fe640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8fc09ad40 0x7fc8fc1496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8fb7fe640 1 -- 172.21.15.139:0/2791271124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8fc1618c0 msgr2=0x7fc8fc1e7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8fb7fe640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8fc1618c0 0x7fc8fc1e7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8fb7fe640 1 -- 172.21.15.139:0/2791271124 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc1e6d90 con 0x7fc8fc0cd610 2024-04-25T09:58:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8fb7fe640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc0cd610 0x7fc8fc159770 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00098c0 tx=0x7fc8f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8f8ff9640 1 -- 172.21.15.139:0/2791271124 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0041070 con 0x7fc8fc0cd610 2024-04-25T09:58:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8f8ff9640 1 -- 172.21.15.139:0/2791271124 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f003c870 con 0x7fc8fc0cd610 2024-04-25T09:58:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc902932640 1 -- 172.21.15.139:0/2791271124 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8fc15a070 con 0x7fc8fc0cd610 2024-04-25T09:58:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc8f8ff9640 1 -- 172.21.15.139:0/2791271124 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0037040 con 0x7fc8fc0cd610 2024-04-25T09:58:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.878+0000 7fc902932640 1 -- 172.21.15.139:0/2791271124 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8fc15a630 con 0x7fc8fc0cd610 2024-04-25T09:58:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.879+0000 7fc902932640 1 -- 172.21.15.139:0/2791271124 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8fc1e95e0 con 0x7fc8fc0cd610 2024-04-25T09:58:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.880+0000 7fc8f8ff9640 1 -- 172.21.15.139:0/2791271124 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8f0044070 con 0x7fc8fc0cd610 2024-04-25T09:58:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.880+0000 7fc8f8ff9640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc8e806b500 0x7fc8e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.880+0000 7fc8f8ff9640 1 -- 172.21.15.139:0/2791271124 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc8f00aaf50 con 0x7fc8fc0cd610 2024-04-25T09:58:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.880+0000 7fc8fbfff640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc8e806b500 0x7fc8e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.880+0000 7fc8fbfff640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc8e806b500 0x7fc8e806d9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00a970 tx=0x7fc8ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:42.884+0000 7fc8f8ff9640 1 -- 172.21.15.139:0/2791271124 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8f0079d10 con 0x7fc8fc0cd610 2024-04-25T09:58:43.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.058+0000 7fc902932640 1 -- 172.21.15.139:0/2791271124 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8fc074520 con 0x7fc8fc0cd610 2024-04-25T09:58:43.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.058+0000 7fc8f8ff9640 1 -- 172.21.15.139:0/2791271124 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8f0079610 con 0x7fc8fc0cd610 2024-04-25T09:58:43.063 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:58:43.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.060+0000 7fc902932640 1 -- 172.21.15.139:0/2791271124 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc8e806b500 msgr2=0x7fc8e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.060+0000 7fc902932640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc8e806b500 0x7fc8e806d9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00a970 tx=0x7fc8ec008040 comp rx=0 tx=0).stop 2024-04-25T09:58:43.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.060+0000 7fc902932640 1 -- 172.21.15.139:0/2791271124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc0cd610 msgr2=0x7fc8fc159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.060+0000 7fc902932640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc0cd610 0x7fc8fc159770 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00098c0 tx=0x7fc8f00046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.060+0000 7fc902932640 1 -- 172.21.15.139:0/2791271124 shutdown_connections 2024-04-25T09:58:43.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.060+0000 7fc902932640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8fc0cd610 0x7fc8fc159770 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.060+0000 7fc902932640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc8e806b500 0x7fc8e806d9c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.060+0000 7fc902932640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8fc1618c0 0x7fc8fc1e7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.060+0000 7fc902932640 1 --2- 172.21.15.139:0/2791271124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8fc09ad40 0x7fc8fc1496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.060+0000 7fc902932640 1 -- 172.21.15.139:0/2791271124 >> 172.21.15.139:0/2791271124 conn(0x7fc8fc0b31f0 msgr2=0x7fc8fc0b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:43.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.060+0000 7fc902932640 1 -- 172.21.15.139:0/2791271124 shutdown_connections 2024-04-25T09:58:43.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.060+0000 7fc902932640 1 -- 172.21.15.139:0/2791271124 wait complete. 2024-04-25T09:58:43.076 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:58:43.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.169+0000 7ff15855c640 1 Processor -- start 2024-04-25T09:58:43.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.169+0000 7ff15855c640 1 -- start start 2024-04-25T09:58:43.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.170+0000 7ff15855c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff150145250 0x7ff150070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.170+0000 7ff15855c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1500b1100 con 0x7ff1500b5690 2024-04-25T09:58:43.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.170+0000 7ff15855c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff150160200 con 0x7ff1500719d0 2024-04-25T09:58:43.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.170+0000 7ff15855c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500ceed0 con 0x7ff150145250 2024-04-25T09:58:43.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.170+0000 7ff156af9640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff1500b5690 0x7ff15015faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59012/0 (socket says 172.21.15.139:59012) 2024-04-25T09:58:43.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.170+0000 7ff156af9640 1 -- 172.21.15.139:0/591735810 learned_addr learned my addr 172.21.15.139:0/591735810 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:43.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1562f8640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff150145250 0x7ff150070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238056871 0 0) 0x7ff1500b1100 con 0x7ff1500b5690 2024-04-25T09:58:43.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff13c003520 con 0x7ff1500b5690 2024-04-25T09:58:43.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984675808 0 0) 0x7ff150160200 con 0x7ff1500719d0 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1500b1100 con 0x7ff1500719d0 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2572318711 0 0) 0x7ff13c003520 con 0x7ff1500b5690 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff150145250 msgr2=0x7ff150070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff150145250 0x7ff150070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 >> v1:172.21.15.150:6789/0 conn(0x7ff1500719d0 legacy=0x7ff15015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1500ce2c0 con 0x7ff1500b5690 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7ff148001670 con 0x7ff1500b5690 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 >> v1:172.21.15.139:6789/0 conn(0x7ff1500b5690 legacy=0x7ff15015faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff150145250 0x7ff13c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c0040d0 0x7ff13c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff13c004d40 0x7ff13c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500ceed0 con 0x7ff13c0040d0 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c00bb50 con 0x7ff13c004d40 2024-04-25T09:58:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c00bd20 con 0x7ff150145250 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff155af7640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c0040d0 0x7ff13c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff1562f8640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff150145250 0x7ff13c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff156af9640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff13c004d40 0x7ff13c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff155af7640 1 -- 172.21.15.139:0/591735810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff150145250 msgr2=0x7ff13c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff155af7640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff150145250 0x7ff13c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff155af7640 1 -- 172.21.15.139:0/591735810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff13c004d40 msgr2=0x7ff13c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff155af7640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff13c004d40 0x7ff13c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff155af7640 1 -- 172.21.15.139:0/591735810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c003520 con 0x7ff13c0040d0 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.171+0000 7ff156af9640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff13c004d40 0x7ff13c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.172+0000 7ff155af7640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c0040d0 0x7ff13c004800 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff1400018d0 tx=0x7ff13c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f982596fd3784b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.172+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1400014f0 con 0x7ff13c0040d0 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.172+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1400052d0 con 0x7ff13c0040d0 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.172+0000 7ff1552f6640 1 -- 172.21.15.139:0/591735810 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14002aca0 con 0x7ff13c0040d0 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.173+0000 7ff15855c640 1 -- 172.21.15.139:0/591735810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c0040d0 msgr2=0x7ff13c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.173+0000 7ff15855c640 1 --2- 172.21.15.139:0/591735810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff13c0040d0 0x7ff13c004800 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff1400018d0 tx=0x7ff13c005830 comp rx=0 tx=0).stop 2024-04-25T09:58:43.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.173+0000 7ff156af9640 1 -- 172.21.15.139:0/591735810 reap_dead start 2024-04-25T09:58:43.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.173+0000 7ff15855c640 1 -- 172.21.15.139:0/591735810 shutdown_connections 2024-04-25T09:58:43.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.173+0000 7ff15855c640 1 -- 172.21.15.139:0/591735810 >> 172.21.15.139:0/591735810 conn(0x7ff15006f460 msgr2=0x7ff15006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:43.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.173+0000 7ff15855c640 1 -- 172.21.15.139:0/591735810 shutdown_connections 2024-04-25T09:58:43.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.173+0000 7ff15855c640 1 -- 172.21.15.139:0/591735810 wait complete. 2024-04-25T09:58:43.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.174+0000 7ff15855c640 1 Processor -- start 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.174+0000 7ff15855c640 1 -- start start 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff15855c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff13c0040d0 0x7ff150158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff15855c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1501563e0 0x7ff150155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff15855c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15015b1e0 0x7ff15015b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff15855c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500ceb60 con 0x7ff15015b1e0 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff15855c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500ce880 con 0x7ff13c0040d0 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff15855c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff150145630 con 0x7ff1501563e0 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff156af9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15015b1e0 0x7ff15015b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff156af9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15015b1e0 0x7ff15015b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38224/0 (socket says 172.21.15.139:38224) 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff156af9640 1 -- 172.21.15.139:0/1794774559 learned_addr learned my addr 172.21.15.139:0/1794774559 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff1562f8640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff13c0040d0 0x7ff150158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff155af7640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1501563e0 0x7ff150155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff156af9640 1 -- 172.21.15.139:0/1794774559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1501563e0 msgr2=0x7ff150155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff156af9640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1501563e0 0x7ff150155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff156af9640 1 -- 172.21.15.139:0/1794774559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff13c0040d0 msgr2=0x7ff150158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff156af9640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff13c0040d0 0x7ff150158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff156af9640 1 -- 172.21.15.139:0/1794774559 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150156860 con 0x7ff15015b1e0 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff156af9640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15015b1e0 0x7ff15015b660 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7ff1500b5b20 tx=0x7ff14800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff146ffd640 1 -- 172.21.15.139:0/1794774559 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148010510 con 0x7ff15015b1e0 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff146ffd640 1 -- 172.21.15.139:0/1794774559 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1480106d0 con 0x7ff15015b1e0 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff15855c640 1 -- 172.21.15.139:0/1794774559 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff150156b10 con 0x7ff15015b1e0 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff146ffd640 1 -- 172.21.15.139:0/1794774559 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148010ac0 con 0x7ff15015b1e0 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.175+0000 7ff15855c640 1 -- 172.21.15.139:0/1794774559 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1501570d0 con 0x7ff15015b1e0 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.176+0000 7ff15855c640 1 -- 172.21.15.139:0/1794774559 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff150074520 con 0x7ff15015b1e0 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.177+0000 7ff146ffd640 1 -- 172.21.15.139:0/1794774559 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff14801b020 con 0x7ff15015b1e0 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.177+0000 7ff146ffd640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff13406b500 0x7ff13406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.177+0000 7ff146ffd640 1 -- 172.21.15.139:0/1794774559 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff148089a60 con 0x7ff15015b1e0 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.177+0000 7ff1562f8640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff13406b500 0x7ff13406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.177+0000 7ff1562f8640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff13406b500 0x7ff13406d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0133c0 tx=0x7ff14c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.181+0000 7ff146ffd640 1 -- 172.21.15.139:0/1794774559 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff148058790 con 0x7ff15015b1e0 2024-04-25T09:58:43.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.361+0000 7ff15855c640 1 -- 172.21.15.139:0/1794774559 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff1501572a0 con 0x7ff15015b1e0 2024-04-25T09:58:43.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.362+0000 7ff146ffd640 1 -- 172.21.15.139:0/1794774559 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v77) v1 ==== 80+0+124 (secure 0 0 0) 0x7ff148058090 con 0x7ff15015b1e0 2024-04-25T09:58:43.367 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.150:6834/1932986598 2024-04-26T09:57:29.527412+0000 2024-04-25T09:58:43.367 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.150:6832/1932986598 2024-04-26T09:57:29.527412+0000 2024-04-25T09:58:43.367 INFO:teuthology.orchestra.run.smithi139.stderr:listed 2 entries 2024-04-25T09:58:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.364+0000 7ff15855c640 1 -- 172.21.15.139:0/1794774559 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff13406b500 msgr2=0x7ff13406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.364+0000 7ff15855c640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff13406b500 0x7ff13406d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0133c0 tx=0x7ff14c00b040 comp rx=0 tx=0).stop 2024-04-25T09:58:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.364+0000 7ff15855c640 1 -- 172.21.15.139:0/1794774559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15015b1e0 msgr2=0x7ff15015b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.364+0000 7ff15855c640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15015b1e0 0x7ff15015b660 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7ff1500b5b20 tx=0x7ff14800e960 comp rx=0 tx=0).stop 2024-04-25T09:58:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.364+0000 7ff15855c640 1 -- 172.21.15.139:0/1794774559 shutdown_connections 2024-04-25T09:58:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.364+0000 7ff15855c640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff15015b1e0 0x7ff15015b660 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.364+0000 7ff15855c640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff13406b500 0x7ff13406d9c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.364+0000 7ff15855c640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1501563e0 0x7ff150155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.364+0000 7ff15855c640 1 --2- 172.21.15.139:0/1794774559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff13c0040d0 0x7ff150158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.364+0000 7ff15855c640 1 -- 172.21.15.139:0/1794774559 >> 172.21.15.139:0/1794774559 conn(0x7ff15006f460 msgr2=0x7ff1500b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.364+0000 7ff15855c640 1 -- 172.21.15.139:0/1794774559 shutdown_connections 2024-04-25T09:58:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.364+0000 7ff15855c640 1 -- 172.21.15.139:0/1794774559 wait complete. 2024-04-25T09:58:43.379 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:58:43.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:43.379 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:43.406 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:58:43.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:43.406 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:58:43.461 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:58:43.462 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:43.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:58:43.480 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:58:43.480 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:43.516 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:58:43.517 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:58:43.517 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:43.538 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:58:43.539 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:43.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.648+0000 7f0acdb1d640 1 Processor -- start 2024-04-25T09:58:43.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.648+0000 7f0acdb1d640 1 -- start start 2024-04-25T09:58:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.649+0000 7f0acdb1d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ac80b0210 0x7f0ac80b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.649+0000 7f0acdb1d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ac80cefe0 0x7f0ac806d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.649+0000 7f0acdb1d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac80ce220 con 0x7f0ac815ae40 2024-04-25T09:58:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.649+0000 7f0acdb1d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac80b1070 con 0x7f0ac80cefe0 2024-04-25T09:58:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.649+0000 7f0acdb1d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80cdff0 con 0x7f0ac80b0210 2024-04-25T09:58:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.649+0000 7f0ac77fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ac80b0210 0x7f0ac80b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.649+0000 7f0ac7fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ac80cefe0 0x7f0ac806d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.649+0000 7f0ac77fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ac80b0210 0x7f0ac80b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:48750/0 (socket says 172.21.15.139:48750) 2024-04-25T09:58:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.649+0000 7f0ac7fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ac80cefe0 0x7f0ac806d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:37572/0 (socket says 172.21.15.139:37572) 2024-04-25T09:58:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.649+0000 7f0ac77fe640 1 -- 172.21.15.139:0/530468734 learned_addr learned my addr 172.21.15.139:0/530468734 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.649+0000 7f0ac6ffd640 1 --1- 172.21.15.139:0/530468734 >> v1:172.21.15.150:6789/0 conn(0x7f0ac815ae40 0x7f0ac80b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:40184/0 (socket says 172.21.15.139:40184) 2024-04-25T09:58:43.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.650+0000 7f0ac77fe640 1 -- 172.21.15.139:0/530468734 >> v2:172.21.15.150:3300/0 conn(0x7f0ac80cefe0 msgr2=0x7f0ac806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:43.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.650+0000 7f0ac77fe640 1 --2- 172.21.15.139:0/530468734 >> v2:172.21.15.150:3300/0 conn(0x7f0ac80cefe0 0x7f0ac806d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.650+0000 7f0ac77fe640 1 -- 172.21.15.139:0/530468734 >> v1:172.21.15.150:6789/0 conn(0x7f0ac815ae40 legacy=0x7f0ac80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.650+0000 7f0ac77fe640 1 -- 172.21.15.139:0/530468734 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac80ce500 con 0x7f0ac80b0210 2024-04-25T09:58:43.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.650+0000 7f0ac7fff640 1 --2- 172.21.15.139:0/530468734 >> v2:172.21.15.150:3300/0 conn(0x7f0ac80cefe0 0x7f0ac806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:43.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.651+0000 7f0ac77fe640 1 --2- 172.21.15.139:0/530468734 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ac80b0210 0x7f0ac80b1960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8008120 tx=0x7f0ab802a500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68ad34e3e423a986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.651+0000 7f0ac67fc640 1 -- 172.21.15.139:0/530468734 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8036070 con 0x7f0ac80b0210 2024-04-25T09:58:43.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.651+0000 7f0ac67fc640 1 -- 172.21.15.139:0/530468734 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab8004230 con 0x7f0ac80b0210 2024-04-25T09:58:43.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.651+0000 7f0ac67fc640 1 -- 172.21.15.139:0/530468734 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800b040 con 0x7f0ac80b0210 2024-04-25T09:58:43.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.651+0000 7f0acdb1d640 1 -- 172.21.15.139:0/530468734 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ac80b0210 msgr2=0x7f0ac80b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.651+0000 7f0acdb1d640 1 --2- 172.21.15.139:0/530468734 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ac80b0210 0x7f0ac80b1960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8008120 tx=0x7f0ab802a500 comp rx=0 tx=0).stop 2024-04-25T09:58:43.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.651+0000 7f0acdb1d640 1 -- 172.21.15.139:0/530468734 shutdown_connections 2024-04-25T09:58:43.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.651+0000 7f0acdb1d640 1 --2- 172.21.15.139:0/530468734 >> v2:172.21.15.150:3300/0 conn(0x7f0ac80cefe0 0x7f0ac806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.651+0000 7f0acdb1d640 1 --2- 172.21.15.139:0/530468734 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ac80b0210 0x7f0ac80b1960 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.651+0000 7f0acdb1d640 1 -- 172.21.15.139:0/530468734 >> 172.21.15.139:0/530468734 conn(0x7f0ac8066f20 msgr2=0x7f0ac806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:43.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.651+0000 7f0acdb1d640 1 -- 172.21.15.139:0/530468734 shutdown_connections 2024-04-25T09:58:43.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.651+0000 7f0acdb1d640 1 -- 172.21.15.139:0/530468734 wait complete. 2024-04-25T09:58:43.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.652+0000 7f0acdb1d640 1 Processor -- start 2024-04-25T09:58:43.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.653+0000 7f0acdb1d640 1 -- start start 2024-04-25T09:58:43.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.653+0000 7f0acdb1d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ac80b0210 0x7f0ac806add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.653+0000 7f0acdb1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac80cefe0 0x7f0ac8151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.653+0000 7f0acdb1d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ac815ae40 0x7f0ac8145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.653+0000 7f0acdb1d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac815b3e0 con 0x7f0ac80cefe0 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.653+0000 7f0acdb1d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80b1ea0 con 0x7f0ac815ae40 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.653+0000 7f0acdb1d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80ceb50 con 0x7f0ac80b0210 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0ac6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac80cefe0 0x7f0ac8151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0ac6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac80cefe0 0x7f0ac8151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38232/0 (socket says 172.21.15.139:38232) 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0ac6ffd640 1 -- 172.21.15.139:0/2388878336 learned_addr learned my addr 172.21.15.139:0/2388878336 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0ac7fff640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ac815ae40 0x7f0ac8145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0ac77fe640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ac80b0210 0x7f0ac806add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0ac6ffd640 1 -- 172.21.15.139:0/2388878336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ac80b0210 msgr2=0x7f0ac806add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0ac6ffd640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ac80b0210 0x7f0ac806add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0ac6ffd640 1 -- 172.21.15.139:0/2388878336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ac815ae40 msgr2=0x7f0ac8145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0ac6ffd640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ac815ae40 0x7f0ac8145cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0ac6ffd640 1 -- 172.21.15.139:0/2388878336 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac8157df0 con 0x7f0ac80cefe0 2024-04-25T09:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0ac6ffd640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac80cefe0 0x7f0ac8151610 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00c3c0 tx=0x7f0abc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0aaffff640 1 -- 172.21.15.139:0/2388878336 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0074e0 con 0x7f0ac80cefe0 2024-04-25T09:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0aaffff640 1 -- 172.21.15.139:0/2388878336 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc007680 con 0x7f0ac80cefe0 2024-04-25T09:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0aaffff640 1 -- 172.21.15.139:0/2388878336 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc004a00 con 0x7f0ac80cefe0 2024-04-25T09:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0acdb1d640 1 -- 172.21.15.139:0/2388878336 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac8157270 con 0x7f0ac80cefe0 2024-04-25T09:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.654+0000 7f0acdb1d640 1 -- 172.21.15.139:0/2388878336 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac8157830 con 0x7f0ac80cefe0 2024-04-25T09:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.655+0000 7f0acdb1d640 1 -- 172.21.15.139:0/2388878336 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac8074520 con 0x7f0ac80cefe0 2024-04-25T09:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.656+0000 7f0aaffff640 1 -- 172.21.15.139:0/2388878336 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0abc01a020 con 0x7f0ac80cefe0 2024-04-25T09:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.656+0000 7f0aaffff640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0ab406b4d0 0x7f0ab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.656+0000 7f0aaffff640 1 -- 172.21.15.139:0/2388878336 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0abc089b50 con 0x7f0ac80cefe0 2024-04-25T09:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.656+0000 7f0ac77fe640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0ab406b4d0 0x7f0ab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.657+0000 7f0ac77fe640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0ab406b4d0 0x7f0ab406d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8033000 tx=0x7f0ab802f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.660+0000 7f0aaffff640 1 -- 172.21.15.139:0/2388878336 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0abc0587f0 con 0x7f0ac80cefe0 2024-04-25T09:58:43.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.862+0000 7f0acdb1d640 1 -- 172.21.15.139:0/2388878336 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ac80b2040 con 0x7f0ac80cefe0 2024-04-25T09:58:43.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.863+0000 7f0aaffff640 1 -- 172.21.15.139:0/2388878336 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f0abc0580f0 con 0x7f0ac80cefe0 2024-04-25T09:58:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:43.868 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/982251530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":982251530},{"type":"v1","addr":"172.21.15.150:6834","nonce":982251530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:20.444843+0000","modified":"2024-04-25T09:58:12.541265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":17,"addr":"172.21.15.139:6837/1384380923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":1384380923},{"type":"v1","addr":"172.21.15.139:6837","nonce":1384380923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:58:43.868 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 44 2024-04-25T09:58:43.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.865+0000 7f0acdb1d640 1 -- 172.21.15.139:0/2388878336 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0ab406b4d0 msgr2=0x7f0ab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.865+0000 7f0acdb1d640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0ab406b4d0 0x7f0ab406d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8033000 tx=0x7f0ab802f000 comp rx=0 tx=0).stop 2024-04-25T09:58:43.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.865+0000 7f0acdb1d640 1 -- 172.21.15.139:0/2388878336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac80cefe0 msgr2=0x7f0ac8151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.865+0000 7f0acdb1d640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac80cefe0 0x7f0ac8151610 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00c3c0 tx=0x7f0abc00c880 comp rx=0 tx=0).stop 2024-04-25T09:58:43.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.865+0000 7f0acdb1d640 1 -- 172.21.15.139:0/2388878336 shutdown_connections 2024-04-25T09:58:43.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.865+0000 7f0acdb1d640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac80cefe0 0x7f0ac8151610 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.865+0000 7f0acdb1d640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0ab406b4d0 0x7f0ab406d990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.865+0000 7f0acdb1d640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ac815ae40 0x7f0ac8145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.865+0000 7f0acdb1d640 1 --2- 172.21.15.139:0/2388878336 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ac80b0210 0x7f0ac806add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.865+0000 7f0acdb1d640 1 -- 172.21.15.139:0/2388878336 >> 172.21.15.139:0/2388878336 conn(0x7f0ac8066f20 msgr2=0x7f0ac8073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:43.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.865+0000 7f0acdb1d640 1 -- 172.21.15.139:0/2388878336 shutdown_connections 2024-04-25T09:58:43.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.865+0000 7f0acdb1d640 1 -- 172.21.15.139:0/2388878336 wait complete. 2024-04-25T09:58:43.881 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:43.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.974+0000 7f56c2ff4640 1 Processor -- start 2024-04-25T09:58:43.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.974+0000 7f56c2ff4640 1 -- start start 2024-04-25T09:58:43.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.975+0000 7f56c2ff4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f56bc15ae40 0x7f56bc0b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.975+0000 7f56c2ff4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f56bc0cefe0 0x7f56bc06d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.975+0000 7f56c2ff4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc0ce220 con 0x7f56bc0b0210 2024-04-25T09:58:43.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.975+0000 7f56c2ff4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56bc0b1070 con 0x7f56bc0cefe0 2024-04-25T09:58:43.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.975+0000 7f56c2ff4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56bc0cdff0 con 0x7f56bc15ae40 2024-04-25T09:58:43.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.975+0000 7f56c0d90640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f56bc0b0210 0x7f56bc0b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59016/0 (socket says 172.21.15.139:59016) 2024-04-25T09:58:43.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.975+0000 7f56c0d90640 1 -- 172.21.15.139:0/386696356 learned_addr learned my addr 172.21.15.139:0/386696356 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:43.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.975+0000 7f56c1591640 1 --2- 172.21.15.139:0/386696356 >> v2:172.21.15.139:3300/0 conn(0x7f56bc0cefe0 0x7f56bc06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.976+0000 7f56bbfff640 1 --2- 172.21.15.139:0/386696356 >> v2:172.21.15.150:3300/0 conn(0x7f56bc15ae40 0x7f56bc0b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.976+0000 7f56c1591640 1 -- 172.21.15.139:0/386696356 >> v2:172.21.15.150:3300/0 conn(0x7f56bc15ae40 msgr2=0x7f56bc0b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:43.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.976+0000 7f56c1591640 1 --2- 172.21.15.139:0/386696356 >> v2:172.21.15.150:3300/0 conn(0x7f56bc15ae40 0x7f56bc0b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.976+0000 7f56c1591640 1 -- 172.21.15.139:0/386696356 >> v1:172.21.15.139:6789/0 conn(0x7f56bc0b0210 legacy=0x7f56bc0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.976+0000 7f56c1591640 1 -- 172.21.15.139:0/386696356 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc0ce500 con 0x7f56bc0cefe0 2024-04-25T09:58:43.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.976+0000 7f56bb7fe640 1 -- 172.21.15.139:0/386696356 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166641090 0 0) 0x7f56bc0ce220 con 0x7f56bc0b0210 2024-04-25T09:58:43.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.976+0000 7f56bbfff640 1 --2- 172.21.15.139:0/386696356 >> v2:172.21.15.150:3300/0 conn(0x7f56bc15ae40 0x7f56bc0b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:43.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.976+0000 7f56c1591640 1 --2- 172.21.15.139:0/386696356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56bc0cefe0 0x7f56bc06d200 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f56bc06f330 tx=0x7f56b402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9000573fcceba7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.976+0000 7f56bb7fe640 1 -- 172.21.15.139:0/386696356 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4007740 con 0x7f56bc0cefe0 2024-04-25T09:58:43.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.977+0000 7f56bb7fe640 1 -- 172.21.15.139:0/386696356 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b40078e0 con 0x7f56bc0cefe0 2024-04-25T09:58:43.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.977+0000 7f56bb7fe640 1 -- 172.21.15.139:0/386696356 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4007cb0 con 0x7f56bc0cefe0 2024-04-25T09:58:43.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.977+0000 7f56c2ff4640 1 -- 172.21.15.139:0/386696356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56bc0cefe0 msgr2=0x7f56bc06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.977+0000 7f56c2ff4640 1 --2- 172.21.15.139:0/386696356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56bc0cefe0 0x7f56bc06d200 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f56bc06f330 tx=0x7f56b402a680 comp rx=0 tx=0).stop 2024-04-25T09:58:43.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.977+0000 7f56c2ff4640 1 -- 172.21.15.139:0/386696356 shutdown_connections 2024-04-25T09:58:43.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.977+0000 7f56c2ff4640 1 --2- 172.21.15.139:0/386696356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56bc0cefe0 0x7f56bc06d200 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.977+0000 7f56c2ff4640 1 --2- 172.21.15.139:0/386696356 >> v2:172.21.15.150:3300/0 conn(0x7f56bc15ae40 0x7f56bc0b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.977+0000 7f56c2ff4640 1 -- 172.21.15.139:0/386696356 >> 172.21.15.139:0/386696356 conn(0x7f56bc066f20 msgr2=0x7f56bc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:43.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.977+0000 7f56c2ff4640 1 -- 172.21.15.139:0/386696356 shutdown_connections 2024-04-25T09:58:43.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.977+0000 7f56c2ff4640 1 -- 172.21.15.139:0/386696356 wait complete. 2024-04-25T09:58:43.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.978+0000 7f56c2ff4640 1 Processor -- start 2024-04-25T09:58:43.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.978+0000 7f56c2ff4640 1 -- start start 2024-04-25T09:58:43.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c2ff4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56bc0b0210 0x7f56bc14de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c2ff4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56bc0cefe0 0x7f56bc14a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c2ff4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56bc15ae40 0x7f56bc14a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c2ff4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc15b3e0 con 0x7f56bc0b0210 2024-04-25T09:58:43.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c2ff4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0b1ea0 con 0x7f56bc15ae40 2024-04-25T09:58:43.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c2ff4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0ceb50 con 0x7f56bc0cefe0 2024-04-25T09:58:43.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c0d90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56bc0b0210 0x7f56bc14de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c0d90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56bc0b0210 0x7f56bc14de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38250/0 (socket says 172.21.15.139:38250) 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c0d90640 1 -- 172.21.15.139:0/2385794308 learned_addr learned my addr 172.21.15.139:0/2385794308 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c1591640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56bc15ae40 0x7f56bc14a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56bbfff640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56bc0cefe0 0x7f56bc14a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c0d90640 1 -- 172.21.15.139:0/2385794308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56bc0cefe0 msgr2=0x7f56bc14a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c0d90640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56bc0cefe0 0x7f56bc14a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c0d90640 1 -- 172.21.15.139:0/2385794308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56bc15ae40 msgr2=0x7f56bc14a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.979+0000 7f56c0d90640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56bc15ae40 0x7f56bc14a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.980+0000 7f56c0d90640 1 -- 172.21.15.139:0/2385794308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc14ae20 con 0x7f56bc0b0210 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.980+0000 7f56c0d90640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56bc0b0210 0x7f56bc14de50 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00c4d0 tx=0x7f56ac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.980+0000 7f56b97fa640 1 -- 172.21.15.139:0/2385794308 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac018070 con 0x7f56bc0b0210 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.980+0000 7f56b97fa640 1 -- 172.21.15.139:0/2385794308 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac007500 con 0x7f56bc0b0210 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.980+0000 7f56c2ff4640 1 -- 172.21.15.139:0/2385794308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc14b440 con 0x7f56bc0b0210 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.980+0000 7f56b97fa640 1 -- 172.21.15.139:0/2385794308 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac004a20 con 0x7f56bc0b0210 2024-04-25T09:58:43.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.980+0000 7f56c2ff4640 1 -- 172.21.15.139:0/2385794308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc147320 con 0x7f56bc0b0210 2024-04-25T09:58:43.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.981+0000 7f56c2ff4640 1 -- 172.21.15.139:0/2385794308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc14b230 con 0x7f56bc0b0210 2024-04-25T09:58:43.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.981+0000 7f56b97fa640 1 -- 172.21.15.139:0/2385794308 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56ac004bc0 con 0x7f56bc0b0210 2024-04-25T09:58:43.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.982+0000 7f56b97fa640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56a806b4d0 0x7f56a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.982+0000 7f56b97fa640 1 -- 172.21.15.139:0/2385794308 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f56ac0896c0 con 0x7f56bc0b0210 2024-04-25T09:58:43.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.982+0000 7f56bbfff640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56a806b4d0 0x7f56a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.982+0000 7f56bbfff640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56a806b4d0 0x7f56a806d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f56bc159790 tx=0x7f56b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:43.986+0000 7f56b97fa640 1 -- 172.21.15.139:0/2385794308 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56ac058370 con 0x7f56bc0b0210 2024-04-25T09:58:44.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.156+0000 7f56c2ff4640 1 -- 172.21.15.139:0/2385794308 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56bc074520 con 0x7f56bc0b0210 2024-04-25T09:58:44.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.157+0000 7f56b97fa640 1 -- 172.21.15.139:0/2385794308 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+17124 (secure 0 0 0) 0x7f56ac057c70 con 0x7f56bc0b0210 2024-04-25T09:58:44.162 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:44.162 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":77,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:57:29.527441+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:18.681554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:19.409382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6834/1932986598":"2024-04-26T09:57:29.527412+0000","172.21.15.150:6832/1932986598":"2024-04-26T09:57:29.527412+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:44.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.159+0000 7f56c2ff4640 1 -- 172.21.15.139:0/2385794308 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56a806b4d0 msgr2=0x7f56a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.159+0000 7f56c2ff4640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56a806b4d0 0x7f56a806d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f56bc159790 tx=0x7f56b0008040 comp rx=0 tx=0).stop 2024-04-25T09:58:44.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.159+0000 7f56c2ff4640 1 -- 172.21.15.139:0/2385794308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56bc0b0210 msgr2=0x7f56bc14de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.159+0000 7f56c2ff4640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56bc0b0210 0x7f56bc14de50 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00c4d0 tx=0x7f56ac00c990 comp rx=0 tx=0).stop 2024-04-25T09:58:44.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.159+0000 7f56c2ff4640 1 -- 172.21.15.139:0/2385794308 shutdown_connections 2024-04-25T09:58:44.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.159+0000 7f56c2ff4640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56a806b4d0 0x7f56a806d990 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.159+0000 7f56c2ff4640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56bc15ae40 0x7f56bc14a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.159+0000 7f56c2ff4640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56bc0cefe0 0x7f56bc14a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.159+0000 7f56c2ff4640 1 --2- 172.21.15.139:0/2385794308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56bc0b0210 0x7f56bc14de50 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.159+0000 7f56c2ff4640 1 -- 172.21.15.139:0/2385794308 >> 172.21.15.139:0/2385794308 conn(0x7f56bc066f20 msgr2=0x7f56bc15e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:44.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.159+0000 7f56c2ff4640 1 -- 172.21.15.139:0/2385794308 shutdown_connections 2024-04-25T09:58:44.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.159+0000 7f56c2ff4640 1 -- 172.21.15.139:0/2385794308 wait complete. 2024-04-25T09:58:44.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:57:18.681554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:57:19.409382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:58:44.175 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:58:44.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.269+0000 7f8c576e1640 1 Processor -- start 2024-04-25T09:58:44.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.269+0000 7f8c576e1640 1 -- start start 2024-04-25T09:58:44.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c576e1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c501617e0 0x7f8c500ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c576e1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8c500cec80 0x7f8c500d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c576e1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c50161dc0 con 0x7f8c5015dbf0 2024-04-25T09:58:44.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c576e1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c500d1ee0 con 0x7f8c500cec80 2024-04-25T09:58:44.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c576e1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c501620a0 con 0x7f8c501617e0 2024-04-25T09:58:44.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c55c7e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8c500cec80 0x7f8c500d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c54c7c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c501617e0 0x7f8c500ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c5547d640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f8c5015dbf0 0x7f8c500b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:40194/0 (socket says 172.21.15.139:40194) 2024-04-25T09:58:44.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c5547d640 1 -- 172.21.15.139:0/1022042201 learned_addr learned my addr 172.21.15.139:0/1022042201 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:44.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c54c7c640 1 -- 172.21.15.139:0/1022042201 >> v2:172.21.15.150:3300/0 conn(0x7f8c500cec80 msgr2=0x7f8c500d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:44.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c54c7c640 1 --2- 172.21.15.139:0/1022042201 >> v2:172.21.15.150:3300/0 conn(0x7f8c500cec80 0x7f8c500d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c54c7c640 1 -- 172.21.15.139:0/1022042201 >> v1:172.21.15.150:6789/0 conn(0x7f8c5015dbf0 legacy=0x7f8c500b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c54c7c640 1 -- 172.21.15.139:0/1022042201 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c50162380 con 0x7f8c501617e0 2024-04-25T09:58:44.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c47fff640 1 -- 172.21.15.139:0/1022042201 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747994662 0 0) 0x7f8c50161dc0 con 0x7f8c5015dbf0 2024-04-25T09:58:44.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.270+0000 7f8c55c7e640 1 --2- 172.21.15.139:0/1022042201 >> v2:172.21.15.150:3300/0 conn(0x7f8c500cec80 0x7f8c500d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.271+0000 7f8c54c7c640 1 --2- 172.21.15.139:0/1022042201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c501617e0 0x7f8c500ce740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c400097a0 tx=0x7f8c4002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d47c73229ae759eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.271+0000 7f8c47fff640 1 -- 172.21.15.139:0/1022042201 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40037070 con 0x7f8c501617e0 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.271+0000 7f8c47fff640 1 -- 172.21.15.139:0/1022042201 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4002ae50 con 0x7f8c501617e0 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.271+0000 7f8c47fff640 1 -- 172.21.15.139:0/1022042201 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40032660 con 0x7f8c501617e0 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.272+0000 7f8c576e1640 1 -- 172.21.15.139:0/1022042201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c501617e0 msgr2=0x7f8c500ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.272+0000 7f8c576e1640 1 --2- 172.21.15.139:0/1022042201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c501617e0 0x7f8c500ce740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c400097a0 tx=0x7f8c4002a100 comp rx=0 tx=0).stop 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.272+0000 7f8c576e1640 1 -- 172.21.15.139:0/1022042201 shutdown_connections 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.272+0000 7f8c576e1640 1 --2- 172.21.15.139:0/1022042201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c501617e0 0x7f8c500ce740 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.272+0000 7f8c576e1640 1 --2- 172.21.15.139:0/1022042201 >> v2:172.21.15.150:3300/0 conn(0x7f8c500cec80 0x7f8c500d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.272+0000 7f8c576e1640 1 -- 172.21.15.139:0/1022042201 >> 172.21.15.139:0/1022042201 conn(0x7f8c500b5480 msgr2=0x7f8c500b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.272+0000 7f8c576e1640 1 -- 172.21.15.139:0/1022042201 shutdown_connections 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.272+0000 7f8c576e1640 1 -- 172.21.15.139:0/1022042201 wait complete. 2024-04-25T09:58:44.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.273+0000 7f8c576e1640 1 Processor -- start 2024-04-25T09:58:44.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.273+0000 7f8c576e1640 1 -- start start 2024-04-25T09:58:44.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c576e1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c500cec80 0x7f8c5014f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c576e1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5015dbf0 0x7f8c50143b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c576e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c501617e0 0x7f8c5015fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c576e1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5015dfd0 con 0x7f8c501617e0 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c576e1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500d28b0 con 0x7f8c500cec80 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c576e1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5014fb10 con 0x7f8c5015dbf0 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c55c7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c501617e0 0x7f8c5015fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c55c7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c501617e0 0x7f8c5015fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38258/0 (socket says 172.21.15.139:38258) 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c55c7e640 1 -- 172.21.15.139:0/3877280236 learned_addr learned my addr 172.21.15.139:0/3877280236 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c5547d640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c500cec80 0x7f8c5014f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c54c7c640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5015dbf0 0x7f8c50143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c55c7e640 1 -- 172.21.15.139:0/3877280236 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5015dbf0 msgr2=0x7f8c50143b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c55c7e640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5015dbf0 0x7f8c50143b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c55c7e640 1 -- 172.21.15.139:0/3877280236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c500cec80 msgr2=0x7f8c5014f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c55c7e640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c500cec80 0x7f8c5014f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c55c7e640 1 -- 172.21.15.139:0/3877280236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5015e2e0 con 0x7f8c501617e0 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c5547d640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c500cec80 0x7f8c5014f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:44.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.274+0000 7f8c55c7e640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c501617e0 0x7f8c5015fdb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800efc0 tx=0x7f8c48009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.275+0000 7f8c45ffb640 1 -- 172.21.15.139:0/3877280236 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48018030 con 0x7f8c501617e0 2024-04-25T09:58:44.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.275+0000 7f8c45ffb640 1 -- 172.21.15.139:0/3877280236 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4800cde0 con 0x7f8c501617e0 2024-04-25T09:58:44.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.275+0000 7f8c576e1640 1 -- 172.21.15.139:0/3877280236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c50144090 con 0x7f8c501617e0 2024-04-25T09:58:44.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.275+0000 7f8c45ffb640 1 -- 172.21.15.139:0/3877280236 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48003bb0 con 0x7f8c501617e0 2024-04-25T09:58:44.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.275+0000 7f8c576e1640 1 -- 172.21.15.139:0/3877280236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c50151bd0 con 0x7f8c501617e0 2024-04-25T09:58:44.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.276+0000 7f8c576e1640 1 -- 172.21.15.139:0/3877280236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c50074520 con 0x7f8c501617e0 2024-04-25T09:58:44.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.276+0000 7f8c45ffb640 1 -- 172.21.15.139:0/3877280236 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c48005d10 con 0x7f8c501617e0 2024-04-25T09:58:44.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.276+0000 7f8c45ffb640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c3406b4d0 0x7f8c3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.277+0000 7f8c45ffb640 1 -- 172.21.15.139:0/3877280236 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8c4808ae50 con 0x7f8c501617e0 2024-04-25T09:58:44.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.277+0000 7f8c5547d640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c3406b4d0 0x7f8c3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.277+0000 7f8c5547d640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c3406b4d0 0x7f8c3406d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c000f60 tx=0x7f8c4c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.281+0000 7f8c45ffb640 1 -- 172.21.15.139:0/3877280236 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c48022070 con 0x7f8c501617e0 2024-04-25T09:58:44.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.450+0000 7f8c576e1640 1 -- 172.21.15.139:0/3877280236 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c500d1680 con 0x7f8c501617e0 2024-04-25T09:58:44.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.450+0000 7f8c45ffb640 1 -- 172.21.15.139:0/3877280236 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c48010040 con 0x7f8c501617e0 2024-04-25T09:58:44.455 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:58:44.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.452+0000 7f8c576e1640 1 -- 172.21.15.139:0/3877280236 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c3406b4d0 msgr2=0x7f8c3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.452+0000 7f8c576e1640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c3406b4d0 0x7f8c3406d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c000f60 tx=0x7f8c4c009040 comp rx=0 tx=0).stop 2024-04-25T09:58:44.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.452+0000 7f8c576e1640 1 -- 172.21.15.139:0/3877280236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c501617e0 msgr2=0x7f8c5015fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.452+0000 7f8c576e1640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c501617e0 0x7f8c5015fdb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800efc0 tx=0x7f8c48009f90 comp rx=0 tx=0).stop 2024-04-25T09:58:44.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.452+0000 7f8c576e1640 1 -- 172.21.15.139:0/3877280236 shutdown_connections 2024-04-25T09:58:44.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.452+0000 7f8c576e1640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c501617e0 0x7f8c5015fdb0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.452+0000 7f8c576e1640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c3406b4d0 0x7f8c3406d990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.452+0000 7f8c576e1640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5015dbf0 0x7f8c50143b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.452+0000 7f8c576e1640 1 --2- 172.21.15.139:0/3877280236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c500cec80 0x7f8c5014f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.452+0000 7f8c576e1640 1 -- 172.21.15.139:0/3877280236 >> 172.21.15.139:0/3877280236 conn(0x7f8c500b5480 msgr2=0x7f8c500d2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:44.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.452+0000 7f8c576e1640 1 -- 172.21.15.139:0/3877280236 shutdown_connections 2024-04-25T09:58:44.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.452+0000 7f8c576e1640 1 -- 172.21.15.139:0/3877280236 wait complete. 2024-04-25T09:58:44.467 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:58:44.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.563+0000 7fdb48efc640 1 Processor -- start 2024-04-25T09:58:44.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.564+0000 7fdb48efc640 1 -- start start 2024-04-25T09:58:44.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.564+0000 7fdb48efc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdb440b31a0 0x7fdb4406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb48efc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4406e870 0x7fdb4415e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb48efc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb440b2a50 con 0x7fdb44144090 2024-04-25T09:58:44.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb48efc640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4406ece0 con 0x7fdb440b31a0 2024-04-25T09:58:44.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb48efc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440b3610 con 0x7fdb4406e870 2024-04-25T09:58:44.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb4259c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdb44144090 0x7fdb440cc9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59032/0 (socket says 172.21.15.139:59032) 2024-04-25T09:58:44.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb4259c640 1 -- 172.21.15.139:0/1686679551 learned_addr learned my addr 172.21.15.139:0/1686679551 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:44.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb41d9b640 1 --2- 172.21.15.139:0/1686679551 >> v2:172.21.15.150:3300/0 conn(0x7fdb440b31a0 0x7fdb4406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb42d9d640 1 --2- 172.21.15.139:0/1686679551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4406e870 0x7fdb4415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb4159a640 1 -- 172.21.15.139:0/1686679551 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869830248 0 0) 0x7fdb34001110 con 0x7fdb44144090 2024-04-25T09:58:44.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb4159a640 1 -- 172.21.15.139:0/1686679551 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb2c003520 con 0x7fdb44144090 2024-04-25T09:58:44.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb41d9b640 1 -- 172.21.15.139:0/1686679551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4406e870 msgr2=0x7fdb4415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:44.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb41d9b640 1 --2- 172.21.15.139:0/1686679551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4406e870 0x7fdb4415e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb41d9b640 1 -- 172.21.15.139:0/1686679551 >> v1:172.21.15.139:6789/0 conn(0x7fdb44144090 legacy=0x7fdb440cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb4159a640 1 -- 172.21.15.139:0/1686679551 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 857416038 0 0) 0x7fdb2c003520 con 0x7fdb44144090 2024-04-25T09:58:44.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.565+0000 7fdb41d9b640 1 -- 172.21.15.139:0/1686679551 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440cd0f0 con 0x7fdb440b31a0 2024-04-25T09:58:44.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.566+0000 7fdb41d9b640 1 --2- 172.21.15.139:0/1686679551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb440b31a0 0x7fdb4406e330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdb380097a0 tx=0x7fdb3802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=370abd0267d5f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.566+0000 7fdb4159a640 1 -- 172.21.15.139:0/1686679551 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38037070 con 0x7fdb440b31a0 2024-04-25T09:58:44.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.566+0000 7fdb4159a640 1 -- 172.21.15.139:0/1686679551 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3802ae50 con 0x7fdb440b31a0 2024-04-25T09:58:44.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.566+0000 7fdb4159a640 1 -- 172.21.15.139:0/1686679551 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38032640 con 0x7fdb440b31a0 2024-04-25T09:58:44.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.567+0000 7fdb48efc640 1 -- 172.21.15.139:0/1686679551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb440b31a0 msgr2=0x7fdb4406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.567+0000 7fdb48efc640 1 --2- 172.21.15.139:0/1686679551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb440b31a0 0x7fdb4406e330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdb380097a0 tx=0x7fdb3802a100 comp rx=0 tx=0).stop 2024-04-25T09:58:44.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.567+0000 7fdb48efc640 1 -- 172.21.15.139:0/1686679551 shutdown_connections 2024-04-25T09:58:44.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.567+0000 7fdb48efc640 1 --2- 172.21.15.139:0/1686679551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb4406e870 0x7fdb4415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.567+0000 7fdb48efc640 1 --2- 172.21.15.139:0/1686679551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb440b31a0 0x7fdb4406e330 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.567+0000 7fdb48efc640 1 -- 172.21.15.139:0/1686679551 >> 172.21.15.139:0/1686679551 conn(0x7fdb44067000 msgr2=0x7fdb4406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:44.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.567+0000 7fdb48efc640 1 -- 172.21.15.139:0/1686679551 shutdown_connections 2024-04-25T09:58:44.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.567+0000 7fdb48efc640 1 -- 172.21.15.139:0/1686679551 wait complete. 2024-04-25T09:58:44.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.568+0000 7fdb48efc640 1 Processor -- start 2024-04-25T09:58:44.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.568+0000 7fdb48efc640 1 -- start start 2024-04-25T09:58:44.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb48efc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4406e870 0x7fdb44159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb48efc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb440b31a0 0x7fdb440c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb48efc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb44144090 0x7fdb44133580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb48efc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4415b650 con 0x7fdb4406e870 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb48efc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4415b370 con 0x7fdb44144090 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb48efc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4415b930 con 0x7fdb440b31a0 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb4259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4406e870 0x7fdb44159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb4259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4406e870 0x7fdb44159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38274/0 (socket says 172.21.15.139:38274) 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb4259c640 1 -- 172.21.15.139:0/609652834 learned_addr learned my addr 172.21.15.139:0/609652834 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb41d9b640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb440b31a0 0x7fdb440c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb42d9d640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb44144090 0x7fdb44133580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb4259c640 1 -- 172.21.15.139:0/609652834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb440b31a0 msgr2=0x7fdb440c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb4259c640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb440b31a0 0x7fdb440c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb4259c640 1 -- 172.21.15.139:0/609652834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb44144090 msgr2=0x7fdb44133580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb4259c640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb44144090 0x7fdb44133580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb4259c640 1 -- 172.21.15.139:0/609652834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440c4600 con 0x7fdb4406e870 2024-04-25T09:58:44.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb4259c640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4406e870 0x7fdb44159f40 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400c4d0 tx=0x7fdb3400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb32ffd640 1 -- 172.21.15.139:0/609652834 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34005ce0 con 0x7fdb4406e870 2024-04-25T09:58:44.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.569+0000 7fdb32ffd640 1 -- 172.21.15.139:0/609652834 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34007070 con 0x7fdb4406e870 2024-04-25T09:58:44.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.570+0000 7fdb32ffd640 1 -- 172.21.15.139:0/609652834 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34007460 con 0x7fdb4406e870 2024-04-25T09:58:44.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.570+0000 7fdb48efc640 1 -- 172.21.15.139:0/609652834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb440c48a0 con 0x7fdb4406e870 2024-04-25T09:58:44.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.570+0000 7fdb48efc640 1 -- 172.21.15.139:0/609652834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb440c4e60 con 0x7fdb4406e870 2024-04-25T09:58:44.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.570+0000 7fdb48efc640 1 -- 172.21.15.139:0/609652834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb44074520 con 0x7fdb4406e870 2024-04-25T09:58:44.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.571+0000 7fdb32ffd640 1 -- 172.21.15.139:0/609652834 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb34007600 con 0x7fdb4406e870 2024-04-25T09:58:44.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.571+0000 7fdb32ffd640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdb2406b4d0 0x7fdb2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.571+0000 7fdb32ffd640 1 -- 172.21.15.139:0/609652834 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdb3408a0d0 con 0x7fdb4406e870 2024-04-25T09:58:44.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.571+0000 7fdb41d9b640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdb2406b4d0 0x7fdb2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.572+0000 7fdb41d9b640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdb2406b4d0 0x7fdb2406d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdb38034000 tx=0x7fdb3802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.575+0000 7fdb32ffd640 1 -- 172.21.15.139:0/609652834 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb440b2a50 con 0x7fdb4406e870 2024-04-25T09:58:44.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.745+0000 7fdb48efc640 1 -- 172.21.15.139:0/609652834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb440ced20 con 0x7fdb4406e870 2024-04-25T09:58:44.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.745+0000 7fdb32ffd640 1 -- 172.21.15.139:0/609652834 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb34058d70 con 0x7fdb4406e870 2024-04-25T09:58:44.750 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:58:44.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.747+0000 7fdb48efc640 1 -- 172.21.15.139:0/609652834 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdb2406b4d0 msgr2=0x7fdb2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.747+0000 7fdb48efc640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdb2406b4d0 0x7fdb2406d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdb38034000 tx=0x7fdb3802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:44.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.748+0000 7fdb48efc640 1 -- 172.21.15.139:0/609652834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4406e870 msgr2=0x7fdb44159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.748+0000 7fdb48efc640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4406e870 0x7fdb44159f40 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400c4d0 tx=0x7fdb3400c990 comp rx=0 tx=0).stop 2024-04-25T09:58:44.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.748+0000 7fdb48efc640 1 -- 172.21.15.139:0/609652834 shutdown_connections 2024-04-25T09:58:44.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.748+0000 7fdb48efc640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdb2406b4d0 0x7fdb2406d990 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.748+0000 7fdb48efc640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb44144090 0x7fdb44133580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.748+0000 7fdb48efc640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb440b31a0 0x7fdb440c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.748+0000 7fdb48efc640 1 --2- 172.21.15.139:0/609652834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb4406e870 0x7fdb44159f40 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.748+0000 7fdb48efc640 1 -- 172.21.15.139:0/609652834 >> 172.21.15.139:0/609652834 conn(0x7fdb44067000 msgr2=0x7fdb440b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:44.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.748+0000 7fdb48efc640 1 -- 172.21.15.139:0/609652834 shutdown_connections 2024-04-25T09:58:44.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.748+0000 7fdb48efc640 1 -- 172.21.15.139:0/609652834 wait complete. 2024-04-25T09:58:44.763 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:58:44.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.858+0000 7fac9f4ac640 1 Processor -- start 2024-04-25T09:58:44.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.859+0000 7fac9f4ac640 1 -- start start 2024-04-25T09:58:44.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.859+0000 7fac9f4ac640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fac980712f0 0x7fac980b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.859+0000 7fac9f4ac640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980b1100 con 0x7fac980719d0 2024-04-25T09:58:44.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.859+0000 7fac9f4ac640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980ce2c0 con 0x7fac98145250 2024-04-25T09:58:44.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.859+0000 7fac9f4ac640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980ceed0 con 0x7fac980712f0 2024-04-25T09:58:44.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac9da49640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fac980719d0 0x7fac980b4bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59038/0 (socket says 172.21.15.139:59038) 2024-04-25T09:58:44.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac9da49640 1 -- 172.21.15.139:0/3712318752 learned_addr learned my addr 172.21.15.139:0/3712318752 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac9ca47640 1 --2- 172.21.15.139:0/3712318752 >> v2:172.21.15.150:3300/0 conn(0x7fac980712f0 0x7fac980b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427402595 0 0) 0x7fac980b1100 con 0x7fac980719d0 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac84003520 con 0x7fac980719d0 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2254627947 0 0) 0x7fac84003520 con 0x7fac980719d0 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 >> v2:172.21.15.150:3300/0 conn(0x7fac980712f0 msgr2=0x7fac980b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 --2- 172.21.15.139:0/3712318752 >> v2:172.21.15.150:3300/0 conn(0x7fac980712f0 0x7fac980b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 >> v1:172.21.15.150:6789/0 conn(0x7fac98145250 legacy=0x7fac98070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980ce5a0 con 0x7fac980719d0 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fac900019f0 con 0x7fac980719d0 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 >> v1:172.21.15.139:6789/0 conn(0x7fac980719d0 legacy=0x7fac980b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 --2- 172.21.15.139:0/3712318752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac840039b0 0x7fac84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 --2- 172.21.15.139:0/3712318752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84004800 0x7fac84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 --2- 172.21.15.139:0/3712318752 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac840051c0 0x7fac8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980ceed0 con 0x7fac84004800 2024-04-25T09:58:44.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8400c160 con 0x7fac840039b0 2024-04-25T09:58:44.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8400c360 con 0x7fac840051c0 2024-04-25T09:58:44.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac9ca47640 1 --2- 172.21.15.139:0/3712318752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84004800 0x7fac84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac9d248640 1 --2- 172.21.15.139:0/3712318752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac840039b0 0x7fac84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac9da49640 1 --2- 172.21.15.139:0/3712318752 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac840051c0 0x7fac8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.860+0000 7fac9ca47640 1 -- 172.21.15.139:0/3712318752 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac840051c0 msgr2=0x7fac8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:44.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.861+0000 7fac9ca47640 1 --2- 172.21.15.139:0/3712318752 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac840051c0 0x7fac8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.861+0000 7fac9ca47640 1 -- 172.21.15.139:0/3712318752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac840039b0 msgr2=0x7fac84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:44.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.861+0000 7fac9ca47640 1 --2- 172.21.15.139:0/3712318752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac840039b0 0x7fac84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.861+0000 7fac9da49640 1 -- 172.21.15.139:0/3712318752 reap_dead start 2024-04-25T09:58:44.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.861+0000 7fac9ca47640 1 -- 172.21.15.139:0/3712318752 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac84003520 con 0x7fac84004800 2024-04-25T09:58:44.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.861+0000 7fac9ca47640 1 --2- 172.21.15.139:0/3712318752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84004800 0x7fac84004c80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fac94009ac0 tx=0x7fac84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6220a35111035550 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.861+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94036070 con 0x7fac84004800 2024-04-25T09:58:44.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.861+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac94031400 con 0x7fac84004800 2024-04-25T09:58:44.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.861+0000 7fac8bfff640 1 -- 172.21.15.139:0/3712318752 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94009690 con 0x7fac84004800 2024-04-25T09:58:44.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.862+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3712318752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84004800 msgr2=0x7fac84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.862+0000 7fac9f4ac640 1 --2- 172.21.15.139:0/3712318752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84004800 0x7fac84004c80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fac94009ac0 tx=0x7fac84005830 comp rx=0 tx=0).stop 2024-04-25T09:58:44.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.862+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3712318752 shutdown_connections 2024-04-25T09:58:44.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.862+0000 7fac9f4ac640 1 --2- 172.21.15.139:0/3712318752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac84004800 0x7fac84004c80 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.862+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3712318752 >> 172.21.15.139:0/3712318752 conn(0x7fac9806f460 msgr2=0x7fac9806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:44.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.862+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3712318752 shutdown_connections 2024-04-25T09:58:44.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.862+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3712318752 wait complete. 2024-04-25T09:58:44.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.863+0000 7fac9f4ac640 1 Processor -- start 2024-04-25T09:58:44.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.863+0000 7fac9f4ac640 1 -- start start 2024-04-25T09:58:44.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9f4ac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac84004800 0x7fac981e6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9f4ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac980c6fc0 0x7fac981f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9f4ac640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac98143f50 0x7fac981443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9f4ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac98145630 con 0x7fac980c6fc0 2024-04-25T09:58:44.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9f4ac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980ceb60 con 0x7fac84004800 2024-04-25T09:58:44.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9f4ac640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac980d11a0 con 0x7fac98143f50 2024-04-25T09:58:44.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9ca47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac980c6fc0 0x7fac981f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9ca47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac980c6fc0 0x7fac981f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38294/0 (socket says 172.21.15.139:38294) 2024-04-25T09:58:44.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9ca47640 1 -- 172.21.15.139:0/3087416688 learned_addr learned my addr 172.21.15.139:0/3087416688 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:44.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9d248640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac84004800 0x7fac981e6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9da49640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac98143f50 0x7fac981443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9ca47640 1 -- 172.21.15.139:0/3087416688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac98143f50 msgr2=0x7fac981443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9ca47640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac98143f50 0x7fac981443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9ca47640 1 -- 172.21.15.139:0/3087416688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac84004800 msgr2=0x7fac981e6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9ca47640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac84004800 0x7fac981e6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9ca47640 1 -- 172.21.15.139:0/3087416688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980c77c0 con 0x7fac980c6fc0 2024-04-25T09:58:44.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9da49640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac98143f50 0x7fac981443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:44.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9ca47640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac980c6fc0 0x7fac981f4c40 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fac94009be0 tx=0x7fac94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac89ffb640 1 -- 172.21.15.139:0/3087416688 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9403f020 con 0x7fac980c6fc0 2024-04-25T09:58:44.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac89ffb640 1 -- 172.21.15.139:0/3087416688 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac94037040 con 0x7fac980c6fc0 2024-04-25T09:58:44.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3087416688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac980c7aa0 con 0x7fac980c6fc0 2024-04-25T09:58:44.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.864+0000 7fac89ffb640 1 -- 172.21.15.139:0/3087416688 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94036070 con 0x7fac980c6fc0 2024-04-25T09:58:44.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.865+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3087416688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac980c8060 con 0x7fac980c6fc0 2024-04-25T09:58:44.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.866+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3087416688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac98074520 con 0x7fac980c6fc0 2024-04-25T09:58:44.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.866+0000 7fac89ffb640 1 -- 172.21.15.139:0/3087416688 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac94042020 con 0x7fac980c6fc0 2024-04-25T09:58:44.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.866+0000 7fac89ffb640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fac7c06b500 0x7fac7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.866+0000 7fac89ffb640 1 -- 172.21.15.139:0/3087416688 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fac940aa3c0 con 0x7fac980c6fc0 2024-04-25T09:58:44.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.867+0000 7fac9d248640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fac7c06b500 0x7fac7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.867+0000 7fac9d248640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fac7c06b500 0x7fac7c06d9c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fac9806a410 tx=0x7fac8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:44.871+0000 7fac89ffb640 1 -- 172.21.15.139:0/3087416688 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac94049030 con 0x7fac980c6fc0 2024-04-25T09:58:45.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.042+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3087416688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac980701f0 con 0x7fac980c6fc0 2024-04-25T09:58:45.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.042+0000 7fac89ffb640 1 -- 172.21.15.139:0/3087416688 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac94079120 con 0x7fac980c6fc0 2024-04-25T09:58:45.047 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:58:45.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.044+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3087416688 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fac7c06b500 msgr2=0x7fac7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.044+0000 7fac9f4ac640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fac7c06b500 0x7fac7c06d9c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fac9806a410 tx=0x7fac8c009040 comp rx=0 tx=0).stop 2024-04-25T09:58:45.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.044+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3087416688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac980c6fc0 msgr2=0x7fac981f4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.044+0000 7fac9f4ac640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac980c6fc0 0x7fac981f4c40 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fac94009be0 tx=0x7fac94004730 comp rx=0 tx=0).stop 2024-04-25T09:58:45.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.044+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3087416688 shutdown_connections 2024-04-25T09:58:45.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.044+0000 7fac9f4ac640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac980c6fc0 0x7fac981f4c40 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.044+0000 7fac9f4ac640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fac7c06b500 0x7fac7c06d9c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.044+0000 7fac9f4ac640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac98143f50 0x7fac981443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.044+0000 7fac9f4ac640 1 --2- 172.21.15.139:0/3087416688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac84004800 0x7fac981e6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.045+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3087416688 >> 172.21.15.139:0/3087416688 conn(0x7fac9806f460 msgr2=0x7fac98141810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:45.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.045+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3087416688 shutdown_connections 2024-04-25T09:58:45.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.045+0000 7fac9f4ac640 1 -- 172.21.15.139:0/3087416688 wait complete. 2024-04-25T09:58:45.060 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:45.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.152+0000 7f583e9b7640 1 Processor -- start 2024-04-25T09:58:45.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.152+0000 7f583e9b7640 1 -- start start 2024-04-25T09:58:45.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f583e9b7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58381611e0 0x7f58380b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f583e9b7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58380cea00 0x7f583806c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f583e9b7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58380d4740 con 0x7f58380cee60 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f583e9b7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58380d1fa0 con 0x7f58381611e0 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f583e9b7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58380d2560 con 0x7f58380cea00 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f5837fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58381611e0 0x7f58380b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f5837fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58381611e0 0x7f58380b6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38298/0 (socket says 172.21.15.139:38298) 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f5837fff640 1 -- 172.21.15.139:0/2029497237 learned_addr learned my addr 172.21.15.139:0/2029497237 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f58377fe640 1 --2- 172.21.15.139:0/2029497237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58380cea00 0x7f583806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f5837fff640 1 -- 172.21.15.139:0/2029497237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58380cea00 msgr2=0x7f583806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f5837fff640 1 --2- 172.21.15.139:0/2029497237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58380cea00 0x7f583806c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f5837fff640 1 -- 172.21.15.139:0/2029497237 >> v1:172.21.15.150:6789/0 conn(0x7f58380cee60 legacy=0x7f58380d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f5837fff640 1 -- 172.21.15.139:0/2029497237 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58380b2770 con 0x7f58381611e0 2024-04-25T09:58:45.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.153+0000 7f58377fe640 1 --2- 172.21.15.139:0/2029497237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58380cea00 0x7f583806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:45.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.154+0000 7f5837fff640 1 --2- 172.21.15.139:0/2029497237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58381611e0 0x7f58380b6af0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f58280098a0 tx=0x7f582802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30932556d6a50546 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:45.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.154+0000 7f5836ffd640 1 -- 172.21.15.139:0/2029497237 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828037070 con 0x7f58381611e0 2024-04-25T09:58:45.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.154+0000 7f5836ffd640 1 -- 172.21.15.139:0/2029497237 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5828032440 con 0x7f58381611e0 2024-04-25T09:58:45.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.154+0000 7f5836ffd640 1 -- 172.21.15.139:0/2029497237 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828032810 con 0x7f58381611e0 2024-04-25T09:58:45.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.155+0000 7f583e9b7640 1 -- 172.21.15.139:0/2029497237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58381611e0 msgr2=0x7f58380b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.155+0000 7f583e9b7640 1 --2- 172.21.15.139:0/2029497237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58381611e0 0x7f58380b6af0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f58280098a0 tx=0x7f582802a290 comp rx=0 tx=0).stop 2024-04-25T09:58:45.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.155+0000 7f583e9b7640 1 -- 172.21.15.139:0/2029497237 shutdown_connections 2024-04-25T09:58:45.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.155+0000 7f583e9b7640 1 --2- 172.21.15.139:0/2029497237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58380cea00 0x7f583806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.155+0000 7f583e9b7640 1 --2- 172.21.15.139:0/2029497237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58381611e0 0x7f58380b6af0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.155+0000 7f583e9b7640 1 -- 172.21.15.139:0/2029497237 >> 172.21.15.139:0/2029497237 conn(0x7f58380b5b00 msgr2=0x7f58380d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:45.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.155+0000 7f583e9b7640 1 -- 172.21.15.139:0/2029497237 shutdown_connections 2024-04-25T09:58:45.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.155+0000 7f583e9b7640 1 -- 172.21.15.139:0/2029497237 wait complete. 2024-04-25T09:58:45.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.156+0000 7f583e9b7640 1 Processor -- start 2024-04-25T09:58:45.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.156+0000 7f583e9b7640 1 -- start start 2024-04-25T09:58:45.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f583e9b7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58380cea00 0x7f5838149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f583e9b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58380cee60 0x7f583806c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f583e9b7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58381611e0 0x7f583806c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f583e9b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58380d21f0 con 0x7f58380cee60 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f583e9b7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58380b2d30 con 0x7f58381611e0 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f583e9b7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58381615c0 con 0x7f58380cea00 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f58377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58380cee60 0x7f583806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f58377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58380cee60 0x7f583806c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38302/0 (socket says 172.21.15.139:38302) 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f58377fe640 1 -- 172.21.15.139:0/217457403 learned_addr learned my addr 172.21.15.139:0/217457403 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f5837fff640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58380cea00 0x7f5838149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f583cf54640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58381611e0 0x7f583806c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f58377fe640 1 -- 172.21.15.139:0/217457403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58380cea00 msgr2=0x7f5838149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f58377fe640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58380cea00 0x7f5838149600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f58377fe640 1 -- 172.21.15.139:0/217457403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58381611e0 msgr2=0x7f583806c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f58377fe640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58381611e0 0x7f583806c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f58377fe640 1 -- 172.21.15.139:0/217457403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58380cb7a0 con 0x7f58380cee60 2024-04-25T09:58:45.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f5837fff640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58380cea00 0x7f5838149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:45.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f583cf54640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58381611e0 0x7f583806c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:45.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f58377fe640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58380cee60 0x7f583806c070 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f582c00e810 tx=0x7f582c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:45.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.157+0000 7f5834ff9640 1 -- 172.21.15.139:0/217457403 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c00f040 con 0x7f58380cee60 2024-04-25T09:58:45.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.158+0000 7f5834ff9640 1 -- 172.21.15.139:0/217457403 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c004030 con 0x7f58380cee60 2024-04-25T09:58:45.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.158+0000 7f583e9b7640 1 -- 172.21.15.139:0/217457403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5838149d10 con 0x7f58380cee60 2024-04-25T09:58:45.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.158+0000 7f5834ff9640 1 -- 172.21.15.139:0/217457403 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c004400 con 0x7f58380cee60 2024-04-25T09:58:45.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.158+0000 7f583e9b7640 1 -- 172.21.15.139:0/217457403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583814a2d0 con 0x7f58380cee60 2024-04-25T09:58:45.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.159+0000 7f583e9b7640 1 -- 172.21.15.139:0/217457403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5838074520 con 0x7f58380cee60 2024-04-25T09:58:45.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.159+0000 7f5834ff9640 1 -- 172.21.15.139:0/217457403 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f582c00c670 con 0x7f58380cee60 2024-04-25T09:58:45.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.159+0000 7f5834ff9640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582406b4d0 0x7f582406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.159+0000 7f5834ff9640 1 -- 172.21.15.139:0/217457403 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f582c08a220 con 0x7f58380cee60 2024-04-25T09:58:45.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.159+0000 7f5837fff640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582406b4d0 0x7f582406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.160+0000 7f5837fff640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582406b4d0 0x7f582406d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5828034000 tx=0x7f582802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:45.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.164+0000 7f5834ff9640 1 -- 172.21.15.139:0/217457403 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f582c017020 con 0x7f58380cee60 2024-04-25T09:58:45.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.375+0000 7f583e9b7640 1 -- 172.21.15.139:0/217457403 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f583806ceb0 con 0x7f58380cee60 2024-04-25T09:58:45.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.376+0000 7f5834ff9640 1 -- 172.21.15.139:0/217457403 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f582c058e30 con 0x7f58380cee60 2024-04-25T09:58:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:45.381 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/982251530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":982251530},{"type":"v1","addr":"172.21.15.150:6834","nonce":982251530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:20.444843+0000","modified":"2024-04-25T09:58:12.541265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":17,"addr":"172.21.15.139:6837/1384380923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":1384380923},{"type":"v1","addr":"172.21.15.139:6837","nonce":1384380923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:58:45.381 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 44 2024-04-25T09:58:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.378+0000 7f583e9b7640 1 -- 172.21.15.139:0/217457403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582406b4d0 msgr2=0x7f582406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.378+0000 7f583e9b7640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582406b4d0 0x7f582406d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5828034000 tx=0x7f582802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.378+0000 7f583e9b7640 1 -- 172.21.15.139:0/217457403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58380cee60 msgr2=0x7f583806c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.378+0000 7f583e9b7640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58380cee60 0x7f583806c070 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f582c00e810 tx=0x7f582c00ecd0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.378+0000 7f583e9b7640 1 -- 172.21.15.139:0/217457403 shutdown_connections 2024-04-25T09:58:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.378+0000 7f583e9b7640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58380cee60 0x7f583806c070 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.378+0000 7f583e9b7640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582406b4d0 0x7f582406d990 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.378+0000 7f583e9b7640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58381611e0 0x7f583806c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.378+0000 7f583e9b7640 1 --2- 172.21.15.139:0/217457403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58380cea00 0x7f5838149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.378+0000 7f583e9b7640 1 -- 172.21.15.139:0/217457403 >> 172.21.15.139:0/217457403 conn(0x7f58380b5b00 msgr2=0x7f58380b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:45.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.378+0000 7f583e9b7640 1 -- 172.21.15.139:0/217457403 shutdown_connections 2024-04-25T09:58:45.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.378+0000 7f583e9b7640 1 -- 172.21.15.139:0/217457403 wait complete. 2024-04-25T09:58:45.394 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:45.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.485+0000 7f003abd5640 1 Processor -- start 2024-04-25T09:58:45.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.485+0000 7f003abd5640 1 -- start start 2024-04-25T09:58:45.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f003abd5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f00340cc2a0 0x7f003409a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f003abd5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f00340cf360 0x7f003409a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f003abd5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003415d1c0 con 0x7f003415cde0 2024-04-25T09:58:45.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f003abd5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00340cc720 con 0x7f00340cf360 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f003abd5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00340cd460 con 0x7f00340cc2a0 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f0039172640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f00340cf360 0x7f003409a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f0039172640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f00340cf360 0x7f003409a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38306/0 (socket says 172.21.15.139:38306) 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f0039172640 1 -- 172.21.15.139:0/1401007147 learned_addr learned my addr 172.21.15.139:0/1401007147 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f0033fff640 1 --2- 172.21.15.139:0/1401007147 >> v2:172.21.15.150:3300/0 conn(0x7f00340cc2a0 0x7f003409a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f0039172640 1 -- 172.21.15.139:0/1401007147 >> v2:172.21.15.150:3300/0 conn(0x7f00340cc2a0 msgr2=0x7f003409a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f0039172640 1 --2- 172.21.15.139:0/1401007147 >> v2:172.21.15.150:3300/0 conn(0x7f00340cc2a0 0x7f003409a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f0039172640 1 -- 172.21.15.139:0/1401007147 >> v1:172.21.15.150:6789/0 conn(0x7f003415cde0 legacy=0x7f003415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f0039172640 1 -- 172.21.15.139:0/1401007147 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0034099bf0 con 0x7f00340cf360 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.486+0000 7f0033fff640 1 --2- 172.21.15.139:0/1401007147 >> v2:172.21.15.150:3300/0 conn(0x7f00340cc2a0 0x7f003409a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.487+0000 7f0039172640 1 --2- 172.21.15.139:0/1401007147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340cf360 0x7f003409a930 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0034072c70 tx=0x7f002c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becdb7546c4a0cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.487+0000 7f00337fe640 1 -- 172.21.15.139:0/1401007147 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c036070 con 0x7f00340cf360 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.487+0000 7f00337fe640 1 -- 172.21.15.139:0/1401007147 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c031600 con 0x7f00340cf360 2024-04-25T09:58:45.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.487+0000 7f00337fe640 1 -- 172.21.15.139:0/1401007147 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c0319d0 con 0x7f00340cf360 2024-04-25T09:58:45.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.487+0000 7f003abd5640 1 -- 172.21.15.139:0/1401007147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340cf360 msgr2=0x7f003409a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.487+0000 7f003abd5640 1 --2- 172.21.15.139:0/1401007147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340cf360 0x7f003409a930 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0034072c70 tx=0x7f002c02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.487+0000 7f003abd5640 1 -- 172.21.15.139:0/1401007147 shutdown_connections 2024-04-25T09:58:45.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.487+0000 7f003abd5640 1 --2- 172.21.15.139:0/1401007147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340cf360 0x7f003409a930 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.487+0000 7f003abd5640 1 --2- 172.21.15.139:0/1401007147 >> v2:172.21.15.150:3300/0 conn(0x7f00340cc2a0 0x7f003409a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.487+0000 7f003abd5640 1 -- 172.21.15.139:0/1401007147 >> 172.21.15.139:0/1401007147 conn(0x7f00340d1790 msgr2=0x7f00340d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:45.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.488+0000 7f003abd5640 1 -- 172.21.15.139:0/1401007147 shutdown_connections 2024-04-25T09:58:45.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.488+0000 7f003abd5640 1 -- 172.21.15.139:0/1401007147 wait complete. 2024-04-25T09:58:45.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.489+0000 7f003abd5640 1 Processor -- start 2024-04-25T09:58:45.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.489+0000 7f003abd5640 1 -- start start 2024-04-25T09:58:45.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.489+0000 7f003abd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00340cc2a0 0x7f00340b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.489+0000 7f003abd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340cf360 0x7f00340b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.489+0000 7f003abd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f003415cde0 0x7f00340b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.489+0000 7f003abd5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00340cf810 con 0x7f00340cf360 2024-04-25T09:58:45.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f003abd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003415dd20 con 0x7f003415cde0 2024-04-25T09:58:45.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f003abd5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00340cf990 con 0x7f00340cc2a0 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f0033fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340cf360 0x7f00340b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f0033fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340cf360 0x7f00340b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38316/0 (socket says 172.21.15.139:38316) 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f0033fff640 1 -- 172.21.15.139:0/226111652 learned_addr learned my addr 172.21.15.139:0/226111652 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f0039172640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f003415cde0 0x7f00340b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f0038971640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00340cc2a0 0x7f00340b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f0033fff640 1 -- 172.21.15.139:0/226111652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00340cc2a0 msgr2=0x7f00340b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f0033fff640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00340cc2a0 0x7f00340b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f0033fff640 1 -- 172.21.15.139:0/226111652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f003415cde0 msgr2=0x7f00340b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f0033fff640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f003415cde0 0x7f00340b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f0033fff640 1 -- 172.21.15.139:0/226111652 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003406c9a0 con 0x7f00340cf360 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f0039172640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f003415cde0 0x7f00340b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f0033fff640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340cf360 0x7f00340b46a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f002000c7f0 tx=0x7f002000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f00317fa640 1 -- 172.21.15.139:0/226111652 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020015070 con 0x7f00340cf360 2024-04-25T09:58:45.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f00317fa640 1 -- 172.21.15.139:0/226111652 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0020007a30 con 0x7f00340cf360 2024-04-25T09:58:45.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f003abd5640 1 -- 172.21.15.139:0/226111652 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00341f4e10 con 0x7f00340cf360 2024-04-25T09:58:45.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f00317fa640 1 -- 172.21.15.139:0/226111652 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020007e00 con 0x7f00340cf360 2024-04-25T09:58:45.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.490+0000 7f003abd5640 1 -- 172.21.15.139:0/226111652 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00341f53d0 con 0x7f00340cf360 2024-04-25T09:58:45.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.491+0000 7f003abd5640 1 -- 172.21.15.139:0/226111652 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00341f55d0 con 0x7f00340cf360 2024-04-25T09:58:45.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.492+0000 7f00317fa640 1 -- 172.21.15.139:0/226111652 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0020018070 con 0x7f00340cf360 2024-04-25T09:58:45.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.492+0000 7f00317fa640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f001806b500 0x7f001806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.492+0000 7f0038971640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f001806b500 0x7f001806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.492+0000 7f00317fa640 1 -- 172.21.15.139:0/226111652 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f002008a420 con 0x7f00340cf360 2024-04-25T09:58:45.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.492+0000 7f0038971640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f001806b500 0x7f001806d9c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0028006fd0 tx=0x7f0028008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:45.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.496+0000 7f00317fa640 1 -- 172.21.15.139:0/226111652 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00341f55d0 con 0x7f00340cf360 2024-04-25T09:58:45.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.667+0000 7f003abd5640 1 -- 172.21.15.139:0/226111652 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0034074520 con 0x7f00340cf360 2024-04-25T09:58:45.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.668+0000 7f00317fa640 1 -- 172.21.15.139:0/226111652 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+17124 (secure 0 0 0) 0x7f0020059030 con 0x7f00340cf360 2024-04-25T09:58:45.673 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:45.673 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":77,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:57:29.527441+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:18.681554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:19.409382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6834/1932986598":"2024-04-26T09:57:29.527412+0000","172.21.15.150:6832/1932986598":"2024-04-26T09:57:29.527412+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:45.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.670+0000 7f003abd5640 1 -- 172.21.15.139:0/226111652 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f001806b500 msgr2=0x7f001806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.670+0000 7f003abd5640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f001806b500 0x7f001806d9c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0028006fd0 tx=0x7f0028008040 comp rx=0 tx=0).stop 2024-04-25T09:58:45.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.670+0000 7f003abd5640 1 -- 172.21.15.139:0/226111652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340cf360 msgr2=0x7f00340b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.670+0000 7f003abd5640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340cf360 0x7f00340b46a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f002000c7f0 tx=0x7f002000cbf0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.670+0000 7f003abd5640 1 -- 172.21.15.139:0/226111652 shutdown_connections 2024-04-25T09:58:45.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.670+0000 7f003abd5640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340cf360 0x7f00340b46a0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.670+0000 7f003abd5640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f001806b500 0x7f001806d9c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.670+0000 7f003abd5640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f003415cde0 0x7f00340b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.670+0000 7f003abd5640 1 --2- 172.21.15.139:0/226111652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00340cc2a0 0x7f00340b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.670+0000 7f003abd5640 1 -- 172.21.15.139:0/226111652 >> 172.21.15.139:0/226111652 conn(0x7f00340d1790 msgr2=0x7f003415bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:45.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.670+0000 7f003abd5640 1 -- 172.21.15.139:0/226111652 shutdown_connections 2024-04-25T09:58:45.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.670+0000 7f003abd5640 1 -- 172.21.15.139:0/226111652 wait complete. 2024-04-25T09:58:45.686 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:58:45.686 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:45.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.778+0000 7f720234f640 1 Processor -- start 2024-04-25T09:58:45.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.778+0000 7f720234f640 1 -- start start 2024-04-25T09:58:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.779+0000 7f720234f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71fc06ab60 0x7f71fc06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.779+0000 7f720234f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f71fc0b36f0 0x7f71fc0b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.779+0000 7f720234f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc0d0400 con 0x7f71fc160e90 2024-04-25T09:58:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.779+0000 7f720234f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc0d14a0 con 0x7f71fc0b36f0 2024-04-25T09:58:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.779+0000 7f720234f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0d06e0 con 0x7f71fc06ab60 2024-04-25T09:58:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.779+0000 7f72008ec640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f71fc0b36f0 0x7f71fc0b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.779+0000 7f72008ec640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f71fc0b36f0 0x7f71fc0b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38328/0 (socket says 172.21.15.139:38328) 2024-04-25T09:58:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.779+0000 7f72008ec640 1 -- 172.21.15.139:0/1035199380 learned_addr learned my addr 172.21.15.139:0/1035199380 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.779+0000 7f71fb7fe640 1 --2- 172.21.15.139:0/1035199380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71fc06ab60 0x7f71fc06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.779+0000 7f72008ec640 1 -- 172.21.15.139:0/1035199380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71fc06ab60 msgr2=0x7f71fc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.779+0000 7f72008ec640 1 --2- 172.21.15.139:0/1035199380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71fc06ab60 0x7f71fc06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.780+0000 7f72008ec640 1 -- 172.21.15.139:0/1035199380 >> v1:172.21.15.150:6789/0 conn(0x7f71fc160e90 legacy=0x7f71fc0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.780+0000 7f72008ec640 1 -- 172.21.15.139:0/1035199380 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc0d09c0 con 0x7f71fc0b36f0 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.780+0000 7f71fb7fe640 1 --2- 172.21.15.139:0/1035199380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71fc06ab60 0x7f71fc06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.780+0000 7f72008ec640 1 --2- 172.21.15.139:0/1035199380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71fc0b36f0 0x7f71fc0b3b40 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009e50 tx=0x7f71f402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4abef001503d2b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.780+0000 7f71faffd640 1 -- 172.21.15.139:0/1035199380 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4036070 con 0x7f71fc0b36f0 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.780+0000 7f71faffd640 1 -- 172.21.15.139:0/1035199380 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f4031600 con 0x7f71fc0b36f0 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.780+0000 7f71faffd640 1 -- 172.21.15.139:0/1035199380 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f40319d0 con 0x7f71fc0b36f0 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.781+0000 7f720234f640 1 -- 172.21.15.139:0/1035199380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71fc0b36f0 msgr2=0x7f71fc0b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.781+0000 7f720234f640 1 --2- 172.21.15.139:0/1035199380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71fc0b36f0 0x7f71fc0b3b40 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009e50 tx=0x7f71f402a680 comp rx=0 tx=0).stop 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.781+0000 7f720234f640 1 -- 172.21.15.139:0/1035199380 shutdown_connections 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.781+0000 7f720234f640 1 --2- 172.21.15.139:0/1035199380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71fc0b36f0 0x7f71fc0b3b40 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.781+0000 7f720234f640 1 --2- 172.21.15.139:0/1035199380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71fc06ab60 0x7f71fc06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.781+0000 7f720234f640 1 -- 172.21.15.139:0/1035199380 >> 172.21.15.139:0/1035199380 conn(0x7f71fc0b23f0 msgr2=0x7f71fc0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.781+0000 7f720234f640 1 -- 172.21.15.139:0/1035199380 shutdown_connections 2024-04-25T09:58:45.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.781+0000 7f720234f640 1 -- 172.21.15.139:0/1035199380 wait complete. 2024-04-25T09:58:45.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.782+0000 7f720234f640 1 Processor -- start 2024-04-25T09:58:45.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.782+0000 7f720234f640 1 -- start start 2024-04-25T09:58:45.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f720234f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71fc06ab60 0x7f71fc144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f720234f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71fc0b36f0 0x7f71fc1e8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f720234f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71fc160e90 0x7f71fc1e8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f720234f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc161270 con 0x7f71fc160e90 2024-04-25T09:58:45.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f720234f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0d0f80 con 0x7f71fc06ab60 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f720234f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc152ca0 con 0x7f71fc0b36f0 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f72008ec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71fc160e90 0x7f71fc1e8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f72008ec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71fc160e90 0x7f71fc1e8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38332/0 (socket says 172.21.15.139:38332) 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f72008ec640 1 -- 172.21.15.139:0/3328144379 learned_addr learned my addr 172.21.15.139:0/3328144379 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f71fbfff640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71fc06ab60 0x7f71fc144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f71fb7fe640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71fc0b36f0 0x7f71fc1e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f72008ec640 1 -- 172.21.15.139:0/3328144379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71fc0b36f0 msgr2=0x7f71fc1e8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f72008ec640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71fc0b36f0 0x7f71fc1e8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f72008ec640 1 -- 172.21.15.139:0/3328144379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71fc06ab60 msgr2=0x7f71fc144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f72008ec640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71fc06ab60 0x7f71fc144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f72008ec640 1 -- 172.21.15.139:0/3328144379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc1526e0 con 0x7f71fc160e90 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.783+0000 7f71fbfff640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71fc06ab60 0x7f71fc144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.784+0000 7f72008ec640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71fc160e90 0x7f71fc1e8c80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009f70 tx=0x7f71f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.784+0000 7f71f8ff9640 1 -- 172.21.15.139:0/3328144379 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f403f020 con 0x7f71fc160e90 2024-04-25T09:58:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.784+0000 7f71f8ff9640 1 -- 172.21.15.139:0/3328144379 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f4037040 con 0x7f71fc160e90 2024-04-25T09:58:45.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.784+0000 7f720234f640 1 -- 172.21.15.139:0/3328144379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71fc1529c0 con 0x7f71fc160e90 2024-04-25T09:58:45.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.784+0000 7f71f8ff9640 1 -- 172.21.15.139:0/3328144379 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4036070 con 0x7f71fc160e90 2024-04-25T09:58:45.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.784+0000 7f720234f640 1 -- 172.21.15.139:0/3328144379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc158b10 con 0x7f71fc160e90 2024-04-25T09:58:45.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.785+0000 7f720234f640 1 -- 172.21.15.139:0/3328144379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71fc074520 con 0x7f71fc160e90 2024-04-25T09:58:45.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.785+0000 7f71f8ff9640 1 -- 172.21.15.139:0/3328144379 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71f4042020 con 0x7f71fc160e90 2024-04-25T09:58:45.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.785+0000 7f71f8ff9640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f71e806b4d0 0x7f71e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.786+0000 7f71f8ff9640 1 -- 172.21.15.139:0/3328144379 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f71f40aa8b0 con 0x7f71fc160e90 2024-04-25T09:58:45.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.786+0000 7f71fbfff640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f71e806b4d0 0x7f71e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.786+0000 7f71fbfff640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f71e806b4d0 0x7f71e806d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f71fc1f6dc0 tx=0x7f71ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:45.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.790+0000 7f71f8ff9640 1 -- 172.21.15.139:0/3328144379 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71f4049030 con 0x7f71fc160e90 2024-04-25T09:58:45.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.994+0000 7f720234f640 1 -- 172.21.15.139:0/3328144379 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71fc0728b0 con 0x7f71fc160e90 2024-04-25T09:58:45.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.995+0000 7f71f8ff9640 1 -- 172.21.15.139:0/3328144379 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f71f4078f90 con 0x7f71fc160e90 2024-04-25T09:58:46.000 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:46.000 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/982251530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":982251530},{"type":"v1","addr":"172.21.15.150:6834","nonce":982251530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:20.444843+0000","modified":"2024-04-25T09:58:12.541265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":17,"addr":"172.21.15.139:6837/1384380923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":1384380923},{"type":"v1","addr":"172.21.15.139:6837","nonce":1384380923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:58:46.000 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 44 2024-04-25T09:58:46.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.997+0000 7f720234f640 1 -- 172.21.15.139:0/3328144379 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f71e806b4d0 msgr2=0x7f71e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.997+0000 7f720234f640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f71e806b4d0 0x7f71e806d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f71fc1f6dc0 tx=0x7f71ec009040 comp rx=0 tx=0).stop 2024-04-25T09:58:46.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.997+0000 7f720234f640 1 -- 172.21.15.139:0/3328144379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71fc160e90 msgr2=0x7f71fc1e8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.997+0000 7f720234f640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71fc160e90 0x7f71fc1e8c80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009f70 tx=0x7f71f4004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:46.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.997+0000 7f720234f640 1 -- 172.21.15.139:0/3328144379 shutdown_connections 2024-04-25T09:58:46.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.997+0000 7f720234f640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71fc160e90 0x7f71fc1e8c80 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.997+0000 7f720234f640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f71e806b4d0 0x7f71e806d990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.997+0000 7f720234f640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71fc0b36f0 0x7f71fc1e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.997+0000 7f720234f640 1 --2- 172.21.15.139:0/3328144379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71fc06ab60 0x7f71fc144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.997+0000 7f720234f640 1 -- 172.21.15.139:0/3328144379 >> 172.21.15.139:0/3328144379 conn(0x7f71fc0b23f0 msgr2=0x7f71fc0cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.997+0000 7f720234f640 1 -- 172.21.15.139:0/3328144379 shutdown_connections 2024-04-25T09:58:46.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:45.997+0000 7f720234f640 1 -- 172.21.15.139:0/3328144379 wait complete. 2024-04-25T09:58:46.013 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:46.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.104+0000 7f4c70c9e640 1 Processor -- start 2024-04-25T09:58:46.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.104+0000 7f4c70c9e640 1 -- start start 2024-04-25T09:58:46.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.105+0000 7f4c70c9e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c6c15a860 0x7f4c6c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.105+0000 7f4c70c9e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4c6c06ea20 0x7f4c6c0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.105+0000 7f4c70c9e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6c0af6d0 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.105+0000 7f4c70c9e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6c0b02e0 con 0x7f4c6c15a860 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.105+0000 7f4c70c9e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6c0af9b0 con 0x7f4c6c06ea20 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.106+0000 7f4c6a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c6c15a860 0x7f4c6c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.106+0000 7f4c6a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c6c15a860 0x7f4c6c0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38346/0 (socket says 172.21.15.139:38346) 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.106+0000 7f4c6a59c640 1 -- 172.21.15.139:0/2807235870 learned_addr learned my addr 172.21.15.139:0/2807235870 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.106+0000 7f4c6ad9d640 1 --2- 172.21.15.139:0/2807235870 >> v2:172.21.15.150:3300/0 conn(0x7f4c6c06ea20 0x7f4c6c0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.106+0000 7f4c6a59c640 1 -- 172.21.15.139:0/2807235870 >> v2:172.21.15.150:3300/0 conn(0x7f4c6c06ea20 msgr2=0x7f4c6c0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.106+0000 7f4c6a59c640 1 --2- 172.21.15.139:0/2807235870 >> v2:172.21.15.150:3300/0 conn(0x7f4c6c06ea20 0x7f4c6c0b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.106+0000 7f4c6a59c640 1 -- 172.21.15.139:0/2807235870 >> v1:172.21.15.150:6789/0 conn(0x7f4c6c0cd1f0 legacy=0x7f4c6c06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.106+0000 7f4c6a59c640 1 -- 172.21.15.139:0/2807235870 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c0afc90 con 0x7f4c6c15a860 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.106+0000 7f4c6ad9d640 1 --2- 172.21.15.139:0/2807235870 >> v2:172.21.15.150:3300/0 conn(0x7f4c6c06ea20 0x7f4c6c0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.106+0000 7f4c6a59c640 1 --2- 172.21.15.139:0/2807235870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c15a860 0x7f4c6c0cccb0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0b0460 tx=0x7f4c6402c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ed75e8a3fb6e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.107+0000 7f4c6959a640 1 -- 172.21.15.139:0/2807235870 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c640325a0 con 0x7f4c6c15a860 2024-04-25T09:58:46.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.107+0000 7f4c6959a640 1 -- 172.21.15.139:0/2807235870 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c64038070 con 0x7f4c6c15a860 2024-04-25T09:58:46.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.107+0000 7f4c6959a640 1 -- 172.21.15.139:0/2807235870 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64032960 con 0x7f4c6c15a860 2024-04-25T09:58:46.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.107+0000 7f4c70c9e640 1 -- 172.21.15.139:0/2807235870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c15a860 msgr2=0x7f4c6c0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.107+0000 7f4c70c9e640 1 --2- 172.21.15.139:0/2807235870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c15a860 0x7f4c6c0cccb0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0b0460 tx=0x7f4c6402c8e0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.107+0000 7f4c70c9e640 1 -- 172.21.15.139:0/2807235870 shutdown_connections 2024-04-25T09:58:46.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.107+0000 7f4c70c9e640 1 --2- 172.21.15.139:0/2807235870 >> v2:172.21.15.150:3300/0 conn(0x7f4c6c06ea20 0x7f4c6c0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.107+0000 7f4c70c9e640 1 --2- 172.21.15.139:0/2807235870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c15a860 0x7f4c6c0cccb0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.107+0000 7f4c70c9e640 1 -- 172.21.15.139:0/2807235870 >> 172.21.15.139:0/2807235870 conn(0x7f4c6c066f20 msgr2=0x7f4c6c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.107+0000 7f4c70c9e640 1 -- 172.21.15.139:0/2807235870 shutdown_connections 2024-04-25T09:58:46.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.107+0000 7f4c70c9e640 1 -- 172.21.15.139:0/2807235870 wait complete. 2024-04-25T09:58:46.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.109+0000 7f4c70c9e640 1 Processor -- start 2024-04-25T09:58:46.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.109+0000 7f4c70c9e640 1 -- start start 2024-04-25T09:58:46.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.109+0000 7f4c70c9e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c6c06ea20 0x7f4c6c14aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.109+0000 7f4c70c9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c0cd1f0 0x7f4c6c148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.109+0000 7f4c70c9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c6c15a860 0x7f4c6c151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.109+0000 7f4c70c9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0af420 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.109+0000 7f4c70c9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0af2a0 con 0x7f4c6c15a860 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.109+0000 7f4c70c9e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c15ac40 con 0x7f4c6c06ea20 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.109+0000 7f4c69d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c0cd1f0 0x7f4c6c148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.109+0000 7f4c69d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c0cd1f0 0x7f4c6c148bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38360/0 (socket says 172.21.15.139:38360) 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.109+0000 7f4c69d9b640 1 -- 172.21.15.139:0/13707653 learned_addr learned my addr 172.21.15.139:0/13707653 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c6ad9d640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c6c15a860 0x7f4c6c151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c6a59c640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c6c06ea20 0x7f4c6c14aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c69d9b640 1 -- 172.21.15.139:0/13707653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c6c06ea20 msgr2=0x7f4c6c14aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c69d9b640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c6c06ea20 0x7f4c6c14aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c69d9b640 1 -- 172.21.15.139:0/13707653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c6c15a860 msgr2=0x7f4c6c151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c69d9b640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c6c15a860 0x7f4c6c151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c69d9b640 1 -- 172.21.15.139:0/13707653 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c1529e0 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c6a59c640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c6c06ea20 0x7f4c6c14aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c69d9b640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c0cd1f0 0x7f4c6c148bf0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c400 tx=0x7f4c5400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c62ffd640 1 -- 172.21.15.139:0/13707653 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54011790 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c62ffd640 1 -- 172.21.15.139:0/13707653 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c54017070 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c62ffd640 1 -- 172.21.15.139:0/13707653 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54011bb0 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c70c9e640 1 -- 172.21.15.139:0/13707653 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6c152cc0 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.110+0000 7f4c70c9e640 1 -- 172.21.15.139:0/13707653 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6c153280 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.111+0000 7f4c70c9e640 1 -- 172.21.15.139:0/13707653 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6c074520 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.111+0000 7f4c62ffd640 1 -- 172.21.15.139:0/13707653 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c5401c460 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.112+0000 7f4c62ffd640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.112+0000 7f4c62ffd640 1 -- 172.21.15.139:0/13707653 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4c5408a1d0 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.112+0000 7f4c6a59c640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.112+0000 7f4c6a59c640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4c64037000 tx=0x7f4c64032000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.116+0000 7f4c62ffd640 1 -- 172.21.15.139:0/13707653 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c54058e70 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.277 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:46.279 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:46.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.323+0000 7f4c70c9e640 1 -- 172.21.15.139:0/13707653 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c6c0b08a0 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.324+0000 7f4c62ffd640 1 -- 172.21.15.139:0/13707653 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f4c54058770 con 0x7f4c6c0cd1f0 2024-04-25T09:58:46.328 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:46.329 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/982251530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":982251530},{"type":"v1","addr":"172.21.15.150:6834","nonce":982251530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:20.444843+0000","modified":"2024-04-25T09:58:12.541265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":77,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":17,"addr":"172.21.15.139:6837/1384380923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":1384380923},{"type":"v1","addr":"172.21.15.139:6837","nonce":1384380923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:58:46.329 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 44 2024-04-25T09:58:46.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.326+0000 7f4c70c9e640 1 -- 172.21.15.139:0/13707653 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c4c06b4d0 msgr2=0x7f4c4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.326+0000 7f4c70c9e640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4c64037000 tx=0x7f4c64032000 comp rx=0 tx=0).stop 2024-04-25T09:58:46.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.326+0000 7f4c70c9e640 1 -- 172.21.15.139:0/13707653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c0cd1f0 msgr2=0x7f4c6c148bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.326+0000 7f4c70c9e640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c0cd1f0 0x7f4c6c148bf0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c400 tx=0x7f4c5400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.326+0000 7f4c70c9e640 1 -- 172.21.15.139:0/13707653 shutdown_connections 2024-04-25T09:58:46.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.326+0000 7f4c70c9e640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c6c0cd1f0 0x7f4c6c148bf0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.326+0000 7f4c70c9e640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.326+0000 7f4c70c9e640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c6c15a860 0x7f4c6c151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.326+0000 7f4c70c9e640 1 --2- 172.21.15.139:0/13707653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c6c06ea20 0x7f4c6c14aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.326+0000 7f4c70c9e640 1 -- 172.21.15.139:0/13707653 >> 172.21.15.139:0/13707653 conn(0x7f4c6c066f20 msgr2=0x7f4c6c071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.326+0000 7f4c70c9e640 1 -- 172.21.15.139:0/13707653 shutdown_connections 2024-04-25T09:58:46.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.326+0000 7f4c70c9e640 1 -- 172.21.15.139:0/13707653 wait complete. 2024-04-25T09:58:46.341 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:46.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.433+0000 7eff9f1e2640 1 Processor -- start 2024-04-25T09:58:46.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.433+0000 7eff9f1e2640 1 -- start start 2024-04-25T09:58:46.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.434+0000 7eff9f1e2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7eff9815a860 0x7eff980cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.434+0000 7eff9f1e2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7eff9806ea20 0x7eff980b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.434+0000 7eff9f1e2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff980af6d0 con 0x7eff980cd1f0 2024-04-25T09:58:46.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.434+0000 7eff9f1e2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff980b02e0 con 0x7eff9815a860 2024-04-25T09:58:46.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.434+0000 7eff9f1e2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff980af9b0 con 0x7eff9806ea20 2024-04-25T09:58:46.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.434+0000 7eff9cf7e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7eff9815a860 0x7eff980cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.434+0000 7eff97fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7eff980cd1f0 0x7eff9806e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55052/0 (socket says 172.21.15.139:55052) 2024-04-25T09:58:46.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.434+0000 7eff97fff640 1 -- 172.21.15.139:0/3449826790 learned_addr learned my addr 172.21.15.139:0/3449826790 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:46.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.434+0000 7eff9d77f640 1 --2- 172.21.15.139:0/3449826790 >> v2:172.21.15.150:3300/0 conn(0x7eff9806ea20 0x7eff980b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.435+0000 7eff9cf7e640 1 -- 172.21.15.139:0/3449826790 >> v2:172.21.15.150:3300/0 conn(0x7eff9806ea20 msgr2=0x7eff980b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:46.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.435+0000 7eff9cf7e640 1 --2- 172.21.15.139:0/3449826790 >> v2:172.21.15.150:3300/0 conn(0x7eff9806ea20 0x7eff980b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.435+0000 7eff9cf7e640 1 -- 172.21.15.139:0/3449826790 >> v1:172.21.15.139:6789/0 conn(0x7eff980cd1f0 legacy=0x7eff9806e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.435+0000 7eff9cf7e640 1 -- 172.21.15.139:0/3449826790 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff980afc90 con 0x7eff9815a860 2024-04-25T09:58:46.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.435+0000 7eff977fe640 1 -- 172.21.15.139:0/3449826790 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321134282 0 0) 0x7eff980af6d0 con 0x7eff980cd1f0 2024-04-25T09:58:46.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.435+0000 7eff9cf7e640 1 --2- 172.21.15.139:0/3449826790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff9815a860 0x7eff980cccb0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7eff980b0460 tx=0x7eff8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2b60e6e323fa1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.435+0000 7eff977fe640 1 -- 172.21.15.139:0/3449826790 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff88037070 con 0x7eff9815a860 2024-04-25T09:58:46.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.435+0000 7eff977fe640 1 -- 172.21.15.139:0/3449826790 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff88032440 con 0x7eff9815a860 2024-04-25T09:58:46.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.435+0000 7eff977fe640 1 -- 172.21.15.139:0/3449826790 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff88032810 con 0x7eff9815a860 2024-04-25T09:58:46.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.436+0000 7eff9f1e2640 1 -- 172.21.15.139:0/3449826790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff9815a860 msgr2=0x7eff980cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.436+0000 7eff9f1e2640 1 --2- 172.21.15.139:0/3449826790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff9815a860 0x7eff980cccb0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7eff980b0460 tx=0x7eff8802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.436+0000 7eff9f1e2640 1 -- 172.21.15.139:0/3449826790 shutdown_connections 2024-04-25T09:58:46.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.436+0000 7eff9f1e2640 1 --2- 172.21.15.139:0/3449826790 >> v2:172.21.15.150:3300/0 conn(0x7eff9806ea20 0x7eff980b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.436+0000 7eff9f1e2640 1 --2- 172.21.15.139:0/3449826790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff9815a860 0x7eff980cccb0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.436+0000 7eff9f1e2640 1 -- 172.21.15.139:0/3449826790 >> 172.21.15.139:0/3449826790 conn(0x7eff98066f20 msgr2=0x7eff9806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.436+0000 7eff9f1e2640 1 -- 172.21.15.139:0/3449826790 shutdown_connections 2024-04-25T09:58:46.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.436+0000 7eff9f1e2640 1 -- 172.21.15.139:0/3449826790 wait complete. 2024-04-25T09:58:46.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.437+0000 7eff9f1e2640 1 Processor -- start 2024-04-25T09:58:46.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.437+0000 7eff9f1e2640 1 -- start start 2024-04-25T09:58:46.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9f1e2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff9806ea20 0x7eff981582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9f1e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff980cd1f0 0x7eff98159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9f1e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff9815a860 0x7eff981e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9f1e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff980af420 con 0x7eff9815a860 2024-04-25T09:58:46.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9f1e2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff980af2a0 con 0x7eff9806ea20 2024-04-25T09:58:46.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9f1e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9815ac40 con 0x7eff980cd1f0 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9d77f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff9815a860 0x7eff981e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9d77f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff9815a860 0x7eff981e4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47984/0 (socket says 172.21.15.139:47984) 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9d77f640 1 -- 172.21.15.139:0/1902176550 learned_addr learned my addr 172.21.15.139:0/1902176550 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9cf7e640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff9806ea20 0x7eff981582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9d77f640 1 -- 172.21.15.139:0/1902176550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff980cd1f0 msgr2=0x7eff98159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9d77f640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff980cd1f0 0x7eff98159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9d77f640 1 -- 172.21.15.139:0/1902176550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff9806ea20 msgr2=0x7eff981582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9d77f640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff9806ea20 0x7eff981582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.438+0000 7eff9d77f640 1 -- 172.21.15.139:0/1902176550 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff981595c0 con 0x7eff9815a860 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.439+0000 7eff9d77f640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff9815a860 0x7eff981e4a90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7eff90007e60 tx=0x7eff9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.439+0000 7eff957fa640 1 -- 172.21.15.139:0/1902176550 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90014620 con 0x7eff9815a860 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.439+0000 7eff957fa640 1 -- 172.21.15.139:0/1902176550 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff90019070 con 0x7eff9815a860 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.439+0000 7eff957fa640 1 -- 172.21.15.139:0/1902176550 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90014a00 con 0x7eff9815a860 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.439+0000 7eff9f1e2640 1 -- 172.21.15.139:0/1902176550 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff981f6c10 con 0x7eff9815a860 2024-04-25T09:58:46.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.439+0000 7eff9f1e2640 1 -- 172.21.15.139:0/1902176550 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff981f71d0 con 0x7eff9815a860 2024-04-25T09:58:46.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.440+0000 7eff9f1e2640 1 -- 172.21.15.139:0/1902176550 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff98074520 con 0x7eff9815a860 2024-04-25T09:58:46.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.440+0000 7eff957fa640 1 -- 172.21.15.139:0/1902176550 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff90014ba0 con 0x7eff9815a860 2024-04-25T09:58:46.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.441+0000 7eff957fa640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7eff8406b500 0x7eff8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.441+0000 7eff957fa640 1 -- 172.21.15.139:0/1902176550 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7eff9008ac80 con 0x7eff9815a860 2024-04-25T09:58:46.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.441+0000 7eff9cf7e640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7eff8406b500 0x7eff8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.441+0000 7eff9cf7e640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7eff8406b500 0x7eff8406d9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7eff88034000 tx=0x7eff8802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.445+0000 7eff957fa640 1 -- 172.21.15.139:0/1902176550 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff90059920 con 0x7eff9815a860 2024-04-25T09:58:46.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.616+0000 7eff9f1e2640 1 -- 172.21.15.139:0/1902176550 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff980b3990 con 0x7eff9815a860 2024-04-25T09:58:46.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.617+0000 7eff957fa640 1 -- 172.21.15.139:0/1902176550 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+17124 (secure 0 0 0) 0x7eff90059220 con 0x7eff9815a860 2024-04-25T09:58:46.622 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:46.622 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":77,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:57:29.527441+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:18.681554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:19.409382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6834/1932986598":"2024-04-26T09:57:29.527412+0000","172.21.15.150:6832/1932986598":"2024-04-26T09:57:29.527412+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:46.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.619+0000 7eff9f1e2640 1 -- 172.21.15.139:0/1902176550 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7eff8406b500 msgr2=0x7eff8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.619+0000 7eff9f1e2640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7eff8406b500 0x7eff8406d9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7eff88034000 tx=0x7eff8802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:46.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.619+0000 7eff9f1e2640 1 -- 172.21.15.139:0/1902176550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff9815a860 msgr2=0x7eff981e4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.619+0000 7eff9f1e2640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff9815a860 0x7eff981e4a90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7eff90007e60 tx=0x7eff9000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.619+0000 7eff9f1e2640 1 -- 172.21.15.139:0/1902176550 shutdown_connections 2024-04-25T09:58:46.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.619+0000 7eff9f1e2640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff9815a860 0x7eff981e4a90 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.619+0000 7eff9f1e2640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7eff8406b500 0x7eff8406d9c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.619+0000 7eff9f1e2640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff980cd1f0 0x7eff98159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.619+0000 7eff9f1e2640 1 --2- 172.21.15.139:0/1902176550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff9806ea20 0x7eff981582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.619+0000 7eff9f1e2640 1 -- 172.21.15.139:0/1902176550 >> 172.21.15.139:0/1902176550 conn(0x7eff98066f20 msgr2=0x7eff98156f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.619+0000 7eff9f1e2640 1 -- 172.21.15.139:0/1902176550 shutdown_connections 2024-04-25T09:58:46.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.619+0000 7eff9f1e2640 1 -- 172.21.15.139:0/1902176550 wait complete. 2024-04-25T09:58:46.635 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:58:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.728+0000 7fab2b438640 1 Processor -- start 2024-04-25T09:58:46.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.728+0000 7fab2b438640 1 -- start start 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab2b438640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab240afd00 0x7fab2415cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab2b438640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab240ccef0 con 0x7fab240d0080 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab2b438640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab240cd1d0 con 0x7fab2406fb20 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab2b438640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab240703c0 con 0x7fab240afd00 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab299d5640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fab240d0080 0x7fab2406e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55062/0 (socket says 172.21.15.139:55062) 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab299d5640 1 -- 172.21.15.139:0/76808551 learned_addr learned my addr 172.21.15.139:0/76808551 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab291d4640 1 --2- 172.21.15.139:0/76808551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab240afd00 0x7fab2415cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab23fff640 1 -- 172.21.15.139:0/76808551 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174476077 0 0) 0x7fab240ccef0 con 0x7fab240d0080 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab23fff640 1 -- 172.21.15.139:0/76808551 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab0c003520 con 0x7fab240d0080 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab23fff640 1 -- 172.21.15.139:0/76808551 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953947539 0 0) 0x7fab240cd1d0 con 0x7fab2406fb20 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab23fff640 1 -- 172.21.15.139:0/76808551 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab240ccef0 con 0x7fab2406fb20 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab23fff640 1 -- 172.21.15.139:0/76808551 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 284701827 0 0) 0x7fab0c003520 con 0x7fab240d0080 2024-04-25T09:58:46.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab291d4640 1 -- 172.21.15.139:0/76808551 >> v1:172.21.15.150:6789/0 conn(0x7fab2406fb20 legacy=0x7fab240cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab291d4640 1 -- 172.21.15.139:0/76808551 >> v1:172.21.15.139:6789/0 conn(0x7fab240d0080 legacy=0x7fab2406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.729+0000 7fab291d4640 1 -- 172.21.15.139:0/76808551 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab240cd4b0 con 0x7fab240afd00 2024-04-25T09:58:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.730+0000 7fab291d4640 1 --2- 172.21.15.139:0/76808551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab240afd00 0x7fab2415cad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0097d0 tx=0x7fab1c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b920626795ba3380 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.730+0000 7fab23fff640 1 -- 172.21.15.139:0/76808551 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c037070 con 0x7fab240afd00 2024-04-25T09:58:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.730+0000 7fab23fff640 1 -- 172.21.15.139:0/76808551 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c02ac30 con 0x7fab240afd00 2024-04-25T09:58:46.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.730+0000 7fab23fff640 1 -- 172.21.15.139:0/76808551 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab140019f0 con 0x7fab240afd00 2024-04-25T09:58:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.731+0000 7fab2b438640 1 -- 172.21.15.139:0/76808551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab240afd00 msgr2=0x7fab2415cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.731+0000 7fab2b438640 1 --2- 172.21.15.139:0/76808551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab240afd00 0x7fab2415cad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0097d0 tx=0x7fab1c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.731+0000 7fab2b438640 1 -- 172.21.15.139:0/76808551 shutdown_connections 2024-04-25T09:58:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.731+0000 7fab2b438640 1 --2- 172.21.15.139:0/76808551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab240afd00 0x7fab2415cad0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.731+0000 7fab2b438640 1 -- 172.21.15.139:0/76808551 >> 172.21.15.139:0/76808551 conn(0x7fab240b2230 msgr2=0x7fab240b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.731+0000 7fab2b438640 1 -- 172.21.15.139:0/76808551 shutdown_connections 2024-04-25T09:58:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.731+0000 7fab2b438640 1 -- 172.21.15.139:0/76808551 wait complete. 2024-04-25T09:58:46.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.732+0000 7fab2b438640 1 Processor -- start 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.732+0000 7fab2b438640 1 -- start start 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab2b438640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab2406fb20 0x7fab24152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab2b438640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab240afd00 0x7fab24154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab2b438640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab240d0080 0x7fab241551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab2b438640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab240cdcb0 con 0x7fab240afd00 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab2b438640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab240cd820 con 0x7fab2406fb20 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab2b438640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab240cde30 con 0x7fab240d0080 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab289d3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab240afd00 0x7fab24154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab289d3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab240afd00 0x7fab24154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47998/0 (socket says 172.21.15.139:47998) 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab289d3640 1 -- 172.21.15.139:0/890084727 learned_addr learned my addr 172.21.15.139:0/890084727 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab291d4640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab2406fb20 0x7fab24152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab299d5640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab240d0080 0x7fab241551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab289d3640 1 -- 172.21.15.139:0/890084727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab240d0080 msgr2=0x7fab241551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab289d3640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab240d0080 0x7fab241551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab289d3640 1 -- 172.21.15.139:0/890084727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab2406fb20 msgr2=0x7fab24152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab289d3640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab2406fb20 0x7fab24152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab289d3640 1 -- 172.21.15.139:0/890084727 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab24155860 con 0x7fab240afd00 2024-04-25T09:58:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab291d4640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab2406fb20 0x7fab24152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab289d3640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab240afd00 0x7fab24154ca0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fab1800c430 tx=0x7fab1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab21ffb640 1 -- 172.21.15.139:0/890084727 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18017070 con 0x7fab240afd00 2024-04-25T09:58:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.733+0000 7fab21ffb640 1 -- 172.21.15.139:0/890084727 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab180052f0 con 0x7fab240afd00 2024-04-25T09:58:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.734+0000 7fab2b438640 1 -- 172.21.15.139:0/890084727 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab24155b40 con 0x7fab240afd00 2024-04-25T09:58:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.734+0000 7fab21ffb640 1 -- 172.21.15.139:0/890084727 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18003830 con 0x7fab240afd00 2024-04-25T09:58:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.734+0000 7fab2b438640 1 -- 172.21.15.139:0/890084727 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab24156100 con 0x7fab240afd00 2024-04-25T09:58:46.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.735+0000 7fab2b438640 1 -- 172.21.15.139:0/890084727 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab24074520 con 0x7fab240afd00 2024-04-25T09:58:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.735+0000 7fab21ffb640 1 -- 172.21.15.139:0/890084727 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab1801a070 con 0x7fab240afd00 2024-04-25T09:58:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.735+0000 7fab21ffb640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fab1006b4d0 0x7fab1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.735+0000 7fab21ffb640 1 -- 172.21.15.139:0/890084727 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fab18089830 con 0x7fab240afd00 2024-04-25T09:58:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.735+0000 7fab291d4640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fab1006b4d0 0x7fab1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.736+0000 7fab291d4640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fab1006b4d0 0x7fab1006d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fab1c034000 tx=0x7fab1c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.740+0000 7fab21ffb640 1 -- 172.21.15.139:0/890084727 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab18058450 con 0x7fab240afd00 2024-04-25T09:58:46.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.942+0000 7fab2b438640 1 -- 172.21.15.139:0/890084727 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fab240cc1a0 con 0x7fab240afd00 2024-04-25T09:58:46.947 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T09:58:46.942+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:58:46.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.943+0000 7fab21ffb640 1 -- 172.21.15.139:0/890084727 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7fab18057d50 con 0x7fab240afd00 2024-04-25T09:58:46.948 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:58:46.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.946+0000 7fab2b438640 1 -- 172.21.15.139:0/890084727 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fab1006b4d0 msgr2=0x7fab1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.946+0000 7fab2b438640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fab1006b4d0 0x7fab1006d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fab1c034000 tx=0x7fab1c02e000 comp rx=0 tx=0).stop 2024-04-25T09:58:46.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.946+0000 7fab2b438640 1 -- 172.21.15.139:0/890084727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab240afd00 msgr2=0x7fab24154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.946+0000 7fab2b438640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab240afd00 0x7fab24154ca0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fab1800c430 tx=0x7fab1800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.946+0000 7fab2b438640 1 -- 172.21.15.139:0/890084727 shutdown_connections 2024-04-25T09:58:46.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.946+0000 7fab2b438640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab240afd00 0x7fab24154ca0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.946+0000 7fab2b438640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fab1006b4d0 0x7fab1006d990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.946+0000 7fab2b438640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab240d0080 0x7fab241551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.946+0000 7fab2b438640 1 --2- 172.21.15.139:0/890084727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab2406fb20 0x7fab24152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.946+0000 7fab2b438640 1 -- 172.21.15.139:0/890084727 >> 172.21.15.139:0/890084727 conn(0x7fab240b2230 msgr2=0x7fab2415b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.947+0000 7fab2b438640 1 -- 172.21.15.139:0/890084727 shutdown_connections 2024-04-25T09:58:46.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:46.947+0000 7fab2b438640 1 -- 172.21.15.139:0/890084727 wait complete. 2024-04-25T09:58:46.954 INFO:tasks.ceph.mds.c.smithi139.stderr: -3465> 2024-04-25T09:57:36.352+0000 7fe979902640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:58:46.963 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:58:46.982 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-04-25T09:58:47.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.059+0000 7f07df316640 1 Processor -- start 2024-04-25T09:58:47.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.059+0000 7f07df316640 1 -- start start 2024-04-25T09:58:47.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07df316640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d80cd610 0x7f07d80d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07df316640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f07d8161fd0 0x7f07d80d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07df316640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d8099ca0 con 0x7f07d809ad40 2024-04-25T09:58:47.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07df316640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d8099a50 con 0x7f07d8161fd0 2024-04-25T09:58:47.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07df316640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8099f80 con 0x7f07d80cd610 2024-04-25T09:58:47.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07dc8b1640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f07d809ad40 0x7f07d81618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55070/0 (socket says 172.21.15.139:55070) 2024-04-25T09:58:47.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07dc8b1640 1 -- 172.21.15.139:0/1586975189 learned_addr learned my addr 172.21.15.139:0/1586975189 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:47.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07dd0b2640 1 --2- 172.21.15.139:0/1586975189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d80cd610 0x7f07d80d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07dd8b3640 1 --2- 172.21.15.139:0/1586975189 >> v2:172.21.15.150:3300/0 conn(0x7f07d8161fd0 0x7f07d80d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147623089 0 0) 0x7f07d8099ca0 con 0x7f07d809ad40 2024-04-25T09:58:47.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07c4003520 con 0x7f07d809ad40 2024-04-25T09:58:47.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3214245052 0 0) 0x7f07c4003520 con 0x7f07d809ad40 2024-04-25T09:58:47.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d80cd610 msgr2=0x7f07d80d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:47.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 --2- 172.21.15.139:0/1586975189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d80cd610 0x7f07d80d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 >> v2:172.21.15.150:3300/0 conn(0x7f07d8161fd0 msgr2=0x7f07d80d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:47.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 --2- 172.21.15.139:0/1586975189 >> v2:172.21.15.150:3300/0 conn(0x7f07d8161fd0 0x7f07d80d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d809a260 con 0x7f07d809ad40 2024-04-25T09:58:47.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f07c8001670 con 0x7f07d809ad40 2024-04-25T09:58:47.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 >> v1:172.21.15.139:6789/0 conn(0x7f07d809ad40 legacy=0x7f07d81618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 --2- 172.21.15.139:0/1586975189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07c40039b0 0x7f07c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 --2- 172.21.15.139:0/1586975189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07c4004800 0x7f07c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 --2- 172.21.15.139:0/1586975189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d80cd610 0x7f07c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8099a50 con 0x7f07c4004800 2024-04-25T09:58:47.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8099f80 con 0x7f07c40039b0 2024-04-25T09:58:47.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c400b9c0 con 0x7f07d80cd610 2024-04-25T09:58:47.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.060+0000 7f07dc8b1640 1 --2- 172.21.15.139:0/1586975189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07c4004800 0x7f07c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.061+0000 7f07dd0b2640 1 --2- 172.21.15.139:0/1586975189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07c40039b0 0x7f07c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.061+0000 7f07dd8b3640 1 --2- 172.21.15.139:0/1586975189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d80cd610 0x7f07c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.061+0000 7f07dc8b1640 1 -- 172.21.15.139:0/1586975189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d80cd610 msgr2=0x7f07c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:47.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.061+0000 7f07dc8b1640 1 --2- 172.21.15.139:0/1586975189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d80cd610 0x7f07c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.061+0000 7f07dc8b1640 1 -- 172.21.15.139:0/1586975189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07c40039b0 msgr2=0x7f07c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:47.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.061+0000 7f07dc8b1640 1 --2- 172.21.15.139:0/1586975189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07c40039b0 0x7f07c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.061+0000 7f07dc8b1640 1 -- 172.21.15.139:0/1586975189 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c4003520 con 0x7f07c4004800 2024-04-25T09:58:47.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.061+0000 7f07dc8b1640 1 --2- 172.21.15.139:0/1586975189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07c4004800 0x7f07c4004c80 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f07c8001050 tx=0x7f07c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5315c344a737db1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:47.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.061+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c802e070 con 0x7f07c4004800 2024-04-25T09:58:47.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.062+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c8005af0 con 0x7f07c4004800 2024-04-25T09:58:47.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.062+0000 7f07cffff640 1 -- 172.21.15.139:0/1586975189 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8007d70 con 0x7f07c4004800 2024-04-25T09:58:47.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.062+0000 7f07df316640 1 -- 172.21.15.139:0/1586975189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07c4004800 msgr2=0x7f07c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.062+0000 7f07df316640 1 --2- 172.21.15.139:0/1586975189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07c4004800 0x7f07c4004c80 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f07c8001050 tx=0x7f07c4005830 comp rx=0 tx=0).stop 2024-04-25T09:58:47.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.062+0000 7f07dd8b3640 1 -- 172.21.15.139:0/1586975189 reap_dead start 2024-04-25T09:58:47.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.062+0000 7f07df316640 1 -- 172.21.15.139:0/1586975189 shutdown_connections 2024-04-25T09:58:47.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.062+0000 7f07df316640 1 -- 172.21.15.139:0/1586975189 >> 172.21.15.139:0/1586975189 conn(0x7f07d80b31f0 msgr2=0x7f07d80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:47.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.062+0000 7f07df316640 1 -- 172.21.15.139:0/1586975189 shutdown_connections 2024-04-25T09:58:47.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.062+0000 7f07df316640 1 -- 172.21.15.139:0/1586975189 wait complete. 2024-04-25T09:58:47.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.063+0000 7f07df316640 1 Processor -- start 2024-04-25T09:58:47.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.063+0000 7f07df316640 1 -- start start 2024-04-25T09:58:47.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07df316640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07c4004800 0x7f07d815cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07df316640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07d815e260 0x7f07d806f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07df316640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d806fd00 0x7f07d80c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07df316640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80cd9f0 con 0x7f07d815e260 2024-04-25T09:58:47.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07df316640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d809a820 con 0x7f07c4004800 2024-04-25T09:58:47.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07df316640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80cdb70 con 0x7f07d806fd00 2024-04-25T09:58:47.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07dc8b1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07d815e260 0x7f07d806f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07dc8b1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07d815e260 0x7f07d806f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48050/0 (socket says 172.21.15.139:48050) 2024-04-25T09:58:47.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07dc8b1640 1 -- 172.21.15.139:0/1477920165 learned_addr learned my addr 172.21.15.139:0/1477920165 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:47.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07dd0b2640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07c4004800 0x7f07d815cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07dd8b3640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d806fd00 0x7f07d80c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07dc8b1640 1 -- 172.21.15.139:0/1477920165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d806fd00 msgr2=0x7f07d80c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07dc8b1640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d806fd00 0x7f07d80c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07dc8b1640 1 -- 172.21.15.139:0/1477920165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07c4004800 msgr2=0x7f07d815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07dc8b1640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07c4004800 0x7f07d815cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07dc8b1640 1 -- 172.21.15.139:0/1477920165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d80c7400 con 0x7f07d815e260 2024-04-25T09:58:47.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.064+0000 7f07dc8b1640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07d815e260 0x7f07d806f7c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f07c800c300 tx=0x7f07c8000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:47.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.065+0000 7f07cdffb640 1 -- 172.21.15.139:0/1477920165 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8042070 con 0x7f07d815e260 2024-04-25T09:58:47.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.065+0000 7f07cdffb640 1 -- 172.21.15.139:0/1477920165 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c8007480 con 0x7f07d815e260 2024-04-25T09:58:47.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.065+0000 7f07df316640 1 -- 172.21.15.139:0/1477920165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d80c76e0 con 0x7f07d815e260 2024-04-25T09:58:47.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.065+0000 7f07cdffb640 1 -- 172.21.15.139:0/1477920165 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8001cc0 con 0x7f07d815e260 2024-04-25T09:58:47.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.065+0000 7f07df316640 1 -- 172.21.15.139:0/1477920165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d80c7ca0 con 0x7f07d815e260 2024-04-25T09:58:47.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.066+0000 7f07df316640 1 -- 172.21.15.139:0/1477920165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d8074520 con 0x7f07d815e260 2024-04-25T09:58:47.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.066+0000 7f07cdffb640 1 -- 172.21.15.139:0/1477920165 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07c8045070 con 0x7f07d815e260 2024-04-25T09:58:47.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.067+0000 7f07cdffb640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f07bc06b500 0x7f07bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.067+0000 7f07cdffb640 1 -- 172.21.15.139:0/1477920165 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f07c80aadc0 con 0x7f07d815e260 2024-04-25T09:58:47.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.067+0000 7f07dd0b2640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f07bc06b500 0x7f07bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.067+0000 7f07dd0b2640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f07bc06b500 0x7f07bc06d9c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f07d40098a0 tx=0x7f07d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:47.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.071+0000 7f07cdffb640 1 -- 172.21.15.139:0/1477920165 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07c8005ac0 con 0x7f07d815e260 2024-04-25T09:58:47.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.279+0000 7f07df316640 1 -- 172.21.15.139:0/1477920165 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f07d80700e0 con 0x7f07d815e260 2024-04-25T09:58:47.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.960+0000 7f07cdffb640 1 -- 172.21.15.139:0/1477920165 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) v1 ==== 104+0+0 (secure 0 0 0) 0x7f07c8079b20 con 0x7f07d815e260 2024-04-25T09:58:47.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.962+0000 7f07df316640 1 -- 172.21.15.139:0/1477920165 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f07bc06b500 msgr2=0x7f07bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.962+0000 7f07df316640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f07bc06b500 0x7f07bc06d9c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f07d40098a0 tx=0x7f07d4008040 comp rx=0 tx=0).stop 2024-04-25T09:58:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.962+0000 7f07df316640 1 -- 172.21.15.139:0/1477920165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07d815e260 msgr2=0x7f07d806f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.962+0000 7f07df316640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07d815e260 0x7f07d806f7c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f07c800c300 tx=0x7f07c8000fe0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.963+0000 7f07df316640 1 -- 172.21.15.139:0/1477920165 shutdown_connections 2024-04-25T09:58:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.963+0000 7f07df316640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07d815e260 0x7f07d806f7c0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.963+0000 7f07df316640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f07bc06b500 0x7f07bc06d9c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.963+0000 7f07df316640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07d806fd00 0x7f07d80c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.963+0000 7f07df316640 1 --2- 172.21.15.139:0/1477920165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07c4004800 0x7f07d815cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.963+0000 7f07df316640 1 -- 172.21.15.139:0/1477920165 >> 172.21.15.139:0/1477920165 conn(0x7f07d80b31f0 msgr2=0x7f07d8160310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.963+0000 7f07df316640 1 -- 172.21.15.139:0/1477920165 shutdown_connections 2024-04-25T09:58:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:47.963+0000 7f07df316640 1 -- 172.21.15.139:0/1477920165 wait complete. 2024-04-25T09:58:47.981 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:58:48.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.124+0000 7f021fec4640 1 Processor -- start 2024-04-25T09:58:48.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.124+0000 7f021fec4640 1 -- start start 2024-04-25T09:58:48.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.125+0000 7f021fec4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f021815dbf0 0x7f02181617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:48.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.125+0000 7f021fec4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f02180cee50 0x7f02180d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:48.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.125+0000 7f021fec4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0218161dc0 con 0x7f02180b9370 2024-04-25T09:58:48.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.125+0000 7f021fec4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02180d1ee0 con 0x7f02180cee50 2024-04-25T09:58:48.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.125+0000 7f021fec4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02181620a0 con 0x7f021815dbf0 2024-04-25T09:58:48.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.125+0000 7f021dc60640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f021815dbf0 0x7f02181617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:48.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.125+0000 7f021e461640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f02180cee50 0x7f02180d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:48.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.125+0000 7f021d45f640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f02180b9370 0x7f02180ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:50428/0 (socket says 172.21.15.139:50428) 2024-04-25T09:58:48.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.125+0000 7f021d45f640 1 -- 172.21.15.139:0/970431610 learned_addr learned my addr 172.21.15.139:0/970431610 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:48.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.126+0000 7f021dc60640 1 -- 172.21.15.139:0/970431610 >> v2:172.21.15.150:3300/0 conn(0x7f02180cee50 msgr2=0x7f02180d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:48.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.126+0000 7f021dc60640 1 --2- 172.21.15.139:0/970431610 >> v2:172.21.15.150:3300/0 conn(0x7f02180cee50 0x7f02180d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.126+0000 7f021dc60640 1 -- 172.21.15.139:0/970431610 >> v1:172.21.15.150:6789/0 conn(0x7f02180b9370 legacy=0x7f02180ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:48.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.126+0000 7f021dc60640 1 -- 172.21.15.139:0/970431610 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0218162380 con 0x7f021815dbf0 2024-04-25T09:58:48.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.126+0000 7f021e461640 1 --2- 172.21.15.139:0/970431610 >> v2:172.21.15.150:3300/0 conn(0x7f02180cee50 0x7f02180d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:48.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.126+0000 7f021dc60640 1 --2- 172.21.15.139:0/970431610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f021815dbf0 0x7f02181617e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f02140097a0 tx=0x7f021402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb533f8bbf98d667 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:48.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.127+0000 7f021cc5e640 1 -- 172.21.15.139:0/970431610 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214037070 con 0x7f021815dbf0 2024-04-25T09:58:48.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.127+0000 7f021cc5e640 1 -- 172.21.15.139:0/970431610 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0214032440 con 0x7f021815dbf0 2024-04-25T09:58:48.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.127+0000 7f021cc5e640 1 -- 172.21.15.139:0/970431610 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214032810 con 0x7f021815dbf0 2024-04-25T09:58:48.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.127+0000 7f021fec4640 1 -- 172.21.15.139:0/970431610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f021815dbf0 msgr2=0x7f02181617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:48.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.127+0000 7f021fec4640 1 --2- 172.21.15.139:0/970431610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f021815dbf0 0x7f02181617e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f02140097a0 tx=0x7f021402a180 comp rx=0 tx=0).stop 2024-04-25T09:58:48.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.127+0000 7f021fec4640 1 -- 172.21.15.139:0/970431610 shutdown_connections 2024-04-25T09:58:48.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.127+0000 7f021fec4640 1 --2- 172.21.15.139:0/970431610 >> v2:172.21.15.150:3300/0 conn(0x7f02180cee50 0x7f02180d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.127+0000 7f021fec4640 1 --2- 172.21.15.139:0/970431610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f021815dbf0 0x7f02181617e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.127+0000 7f021fec4640 1 -- 172.21.15.139:0/970431610 >> 172.21.15.139:0/970431610 conn(0x7f02180b5480 msgr2=0x7f02180b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:48.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.127+0000 7f021fec4640 1 -- 172.21.15.139:0/970431610 shutdown_connections 2024-04-25T09:58:48.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.127+0000 7f021fec4640 1 -- 172.21.15.139:0/970431610 wait complete. 2024-04-25T09:58:48.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.128+0000 7f021fec4640 1 Processor -- start 2024-04-25T09:58:48.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.128+0000 7f021fec4640 1 -- start start 2024-04-25T09:58:48.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021fec4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02180b9370 0x7f0218151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:48.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021fec4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02180cee50 0x7f0218153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:48.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021fec4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f021815dbf0 0x7f02181540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:48.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021fec4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021815dfd0 con 0x7f02180b9370 2024-04-25T09:58:48.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021fec4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02180b97d0 con 0x7f02180cee50 2024-04-25T09:58:48.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021fec4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02181546d0 con 0x7f021815dbf0 2024-04-25T09:58:48.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021dc60640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02180b9370 0x7f0218151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:48.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021dc60640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02180b9370 0x7f0218151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48054/0 (socket says 172.21.15.139:48054) 2024-04-25T09:58:48.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021dc60640 1 -- 172.21.15.139:0/3787775301 learned_addr learned my addr 172.21.15.139:0/3787775301 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:48.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021d45f640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02180cee50 0x7f0218153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:48.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021e461640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f021815dbf0 0x7f02181540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021dc60640 1 -- 172.21.15.139:0/3787775301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f021815dbf0 msgr2=0x7f02181540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.129+0000 7f021dc60640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f021815dbf0 0x7f02181540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.130+0000 7f021dc60640 1 -- 172.21.15.139:0/3787775301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02180cee50 msgr2=0x7f0218153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.130+0000 7f021dc60640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02180cee50 0x7f0218153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.130+0000 7f021dc60640 1 -- 172.21.15.139:0/3787775301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02181549b0 con 0x7f02180b9370 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.130+0000 7f021d45f640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02180cee50 0x7f0218153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.130+0000 7f021dc60640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02180b9370 0x7f0218151d60 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0214035040 tx=0x7f02140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.130+0000 7f020e7fc640 1 -- 172.21.15.139:0/3787775301 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214037040 con 0x7f02180b9370 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.130+0000 7f020e7fc640 1 -- 172.21.15.139:0/3787775301 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0214030500 con 0x7f02180b9370 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.130+0000 7f020e7fc640 1 -- 172.21.15.139:0/3787775301 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214030910 con 0x7f02180b9370 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.130+0000 7f021fec4640 1 -- 172.21.15.139:0/3787775301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0218154c90 con 0x7f02180b9370 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.130+0000 7f021fec4640 1 -- 172.21.15.139:0/3787775301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021815e590 con 0x7f02180b9370 2024-04-25T09:58:48.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.131+0000 7f021fec4640 1 -- 172.21.15.139:0/3787775301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0218074520 con 0x7f02180b9370 2024-04-25T09:58:48.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.131+0000 7f020e7fc640 1 -- 172.21.15.139:0/3787775301 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0214044020 con 0x7f02180b9370 2024-04-25T09:58:48.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.132+0000 7f020e7fc640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f01fc06b4d0 0x7f01fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:48.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.132+0000 7f021d45f640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f01fc06b4d0 0x7f01fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:48.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.132+0000 7f020e7fc640 1 -- 172.21.15.139:0/3787775301 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f02140aa720 con 0x7f02180b9370 2024-04-25T09:58:48.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.132+0000 7f021d45f640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f01fc06b4d0 0x7f01fc06d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f021815cd90 tx=0x7f0208009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:48.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.136+0000 7f020e7fc640 1 -- 172.21.15.139:0/3787775301 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02140792e0 con 0x7f02180b9370 2024-04-25T09:58:48.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.308+0000 7f021fec4640 1 -- 172.21.15.139:0/3787775301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f02180d1680 con 0x7f02180b9370 2024-04-25T09:58:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.965+0000 7f020e7fc640 1 -- 172.21.15.139:0/3787775301 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0214078be0 con 0x7f02180b9370 2024-04-25T09:58:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:58:48.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.968+0000 7f021fec4640 1 -- 172.21.15.139:0/3787775301 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f01fc06b4d0 msgr2=0x7f01fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:48.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.968+0000 7f021fec4640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f01fc06b4d0 0x7f01fc06d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f021815cd90 tx=0x7f0208009040 comp rx=0 tx=0).stop 2024-04-25T09:58:48.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.968+0000 7f021fec4640 1 -- 172.21.15.139:0/3787775301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02180b9370 msgr2=0x7f0218151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:48.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.968+0000 7f021fec4640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02180b9370 0x7f0218151d60 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0214035040 tx=0x7f02140046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.968+0000 7f021fec4640 1 -- 172.21.15.139:0/3787775301 shutdown_connections 2024-04-25T09:58:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.969+0000 7f021fec4640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f01fc06b4d0 0x7f01fc06d990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.969+0000 7f021fec4640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f021815dbf0 0x7f02181540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.969+0000 7f021fec4640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02180cee50 0x7f0218153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.969+0000 7f021fec4640 1 --2- 172.21.15.139:0/3787775301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02180b9370 0x7f0218151d60 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.969+0000 7f021fec4640 1 -- 172.21.15.139:0/3787775301 >> 172.21.15.139:0/3787775301 conn(0x7f02180b5480 msgr2=0x7f02180b80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.969+0000 7f021fec4640 1 -- 172.21.15.139:0/3787775301 shutdown_connections 2024-04-25T09:58:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:48.969+0000 7f021fec4640 1 -- 172.21.15.139:0/3787775301 wait complete. 2024-04-25T09:58:48.988 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:58:49.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.088+0000 7f7577d89640 1 Processor -- start 2024-04-25T09:58:49.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.089+0000 7f7577d89640 1 -- start start 2024-04-25T09:58:49.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.089+0000 7f7577d89640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7570144090 0x7f75700b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:49.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.089+0000 7f7577d89640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f757006ea40 0x7f757015e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:49.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.089+0000 7f7577d89640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75700b2a50 con 0x7f75700cc9e0 2024-04-25T09:58:49.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.089+0000 7f7577d89640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757015b090 con 0x7f757006ea40 2024-04-25T09:58:49.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.089+0000 7f7577d89640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757015b370 con 0x7f7570144090 2024-04-25T09:58:49.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.090+0000 7f7576326640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f757006ea40 0x7f757015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:49.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.090+0000 7f7576326640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f757006ea40 0x7f757015e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48070/0 (socket says 172.21.15.139:48070) 2024-04-25T09:58:49.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.090+0000 7f7575324640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f75700cc9e0 0x7f757006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55086/0 (socket says 172.21.15.139:55086) 2024-04-25T09:58:49.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.090+0000 7f7576326640 1 -- 172.21.15.139:0/1602769033 learned_addr learned my addr 172.21.15.139:0/1602769033 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:49.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.090+0000 7f7575b25640 1 --2- 172.21.15.139:0/1602769033 >> v2:172.21.15.150:3300/0 conn(0x7f7570144090 0x7f75700b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:49.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.090+0000 7f7576326640 1 -- 172.21.15.139:0/1602769033 >> v2:172.21.15.150:3300/0 conn(0x7f7570144090 msgr2=0x7f75700b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:49.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.090+0000 7f7576326640 1 --2- 172.21.15.139:0/1602769033 >> v2:172.21.15.150:3300/0 conn(0x7f7570144090 0x7f75700b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.090+0000 7f7576326640 1 -- 172.21.15.139:0/1602769033 >> v1:172.21.15.139:6789/0 conn(0x7f75700cc9e0 legacy=0x7f757006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.090+0000 7f7574b23640 1 -- 172.21.15.139:0/1602769033 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938025543 0 0) 0x7f7560001110 con 0x7f75700cc9e0 2024-04-25T09:58:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.090+0000 7f7576326640 1 -- 172.21.15.139:0/1602769033 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757015b650 con 0x7f757006ea40 2024-04-25T09:58:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.091+0000 7f7576326640 1 --2- 172.21.15.139:0/1602769033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f757006ea40 0x7f757015e610 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f75700b4ce0 tx=0x7f756802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2da67ed765357cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.091+0000 7f7574b23640 1 -- 172.21.15.139:0/1602769033 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568037070 con 0x7f757006ea40 2024-04-25T09:58:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.091+0000 7f7574b23640 1 -- 172.21.15.139:0/1602769033 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7568032440 con 0x7f757006ea40 2024-04-25T09:58:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.091+0000 7f7574b23640 1 -- 172.21.15.139:0/1602769033 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568032810 con 0x7f757006ea40 2024-04-25T09:58:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.091+0000 7f7577d89640 1 -- 172.21.15.139:0/1602769033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f757006ea40 msgr2=0x7f757015e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.091+0000 7f7577d89640 1 --2- 172.21.15.139:0/1602769033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f757006ea40 0x7f757015e610 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f75700b4ce0 tx=0x7f756802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.091+0000 7f7577d89640 1 -- 172.21.15.139:0/1602769033 shutdown_connections 2024-04-25T09:58:49.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.091+0000 7f7577d89640 1 --2- 172.21.15.139:0/1602769033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f757006ea40 0x7f757015e610 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.091+0000 7f7577d89640 1 --2- 172.21.15.139:0/1602769033 >> v2:172.21.15.150:3300/0 conn(0x7f7570144090 0x7f75700b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.091+0000 7f7577d89640 1 -- 172.21.15.139:0/1602769033 >> 172.21.15.139:0/1602769033 conn(0x7f7570067000 msgr2=0x7f757006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:49.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.091+0000 7f7577d89640 1 -- 172.21.15.139:0/1602769033 shutdown_connections 2024-04-25T09:58:49.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.091+0000 7f7577d89640 1 -- 172.21.15.139:0/1602769033 wait complete. 2024-04-25T09:58:49.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.093+0000 7f7577d89640 1 Processor -- start 2024-04-25T09:58:49.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.093+0000 7f7577d89640 1 -- start start 2024-04-25T09:58:49.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.093+0000 7f7577d89640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f757006ea40 0x7f757015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:49.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.093+0000 7f7577d89640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75700cc9e0 0x7f75701f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:49.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.093+0000 7f7577d89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7570144090 0x7f757013fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:49.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.093+0000 7f7577d89640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7570144470 con 0x7f7570144090 2024-04-25T09:58:49.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.093+0000 7f7577d89640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757015bc10 con 0x7f757006ea40 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.093+0000 7f7577d89640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757015ad80 con 0x7f75700cc9e0 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.093+0000 7f7576326640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7570144090 0x7f757013fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.093+0000 7f7576326640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7570144090 0x7f757013fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48084/0 (socket says 172.21.15.139:48084) 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.093+0000 7f7576326640 1 -- 172.21.15.139:0/633399421 learned_addr learned my addr 172.21.15.139:0/633399421 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.094+0000 7f7576326640 1 -- 172.21.15.139:0/633399421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75700cc9e0 msgr2=0x7f75701f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.094+0000 7f7576326640 1 --2- 172.21.15.139:0/633399421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75700cc9e0 0x7f75701f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.094+0000 7f7576326640 1 -- 172.21.15.139:0/633399421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f757006ea40 msgr2=0x7f757015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.094+0000 7f7576326640 1 --2- 172.21.15.139:0/633399421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f757006ea40 0x7f757015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.094+0000 7f7576326640 1 -- 172.21.15.139:0/633399421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75701f57a0 con 0x7f7570144090 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.094+0000 7f7576326640 1 --2- 172.21.15.139:0/633399421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7570144090 0x7f757013fa80 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f756802a220 tx=0x7f75680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.094+0000 7f75667fc640 1 -- 172.21.15.139:0/633399421 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568037040 con 0x7f7570144090 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.094+0000 7f75667fc640 1 -- 172.21.15.139:0/633399421 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7568042070 con 0x7f7570144090 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.094+0000 7f7577d89640 1 -- 172.21.15.139:0/633399421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757015c080 con 0x7f7570144090 2024-04-25T09:58:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.094+0000 7f75667fc640 1 -- 172.21.15.139:0/633399421 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568030920 con 0x7f7570144090 2024-04-25T09:58:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.094+0000 7f7577d89640 1 -- 172.21.15.139:0/633399421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757015c600 con 0x7f7570144090 2024-04-25T09:58:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.095+0000 7f7577d89640 1 -- 172.21.15.139:0/633399421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7570074520 con 0x7f7570144090 2024-04-25T09:58:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.095+0000 7f75667fc640 1 -- 172.21.15.139:0/633399421 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7568030ac0 con 0x7f7570144090 2024-04-25T09:58:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.096+0000 7f75667fc640 1 --2- 172.21.15.139:0/633399421 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f755406b4d0 0x7f755406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.096+0000 7f75667fc640 1 -- 172.21.15.139:0/633399421 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f75680aab30 con 0x7f7570144090 2024-04-25T09:58:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.096+0000 7f7575b25640 1 --2- 172.21.15.139:0/633399421 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f755406b4d0 0x7f755406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.096+0000 7f7575b25640 1 --2- 172.21.15.139:0/633399421 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f755406b4d0 0x7f755406d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f756c0070b0 tx=0x7f756c007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:49.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.100+0000 7f75667fc640 1 -- 172.21.15.139:0/633399421 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75701f57a0 con 0x7f7570144090 2024-04-25T09:58:49.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.274+0000 7f7577d89640 1 -- 172.21.15.139:0/633399421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f75700cdfb0 con 0x7f7570144090 2024-04-25T09:58:49.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.980+0000 7f75667fc640 1 -- 172.21.15.139:0/633399421 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) v1 ==== 169+0+0 (secure 0 0 0) 0x7f75680792e0 con 0x7f7570144090 2024-04-25T09:58:49.985 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' removed 2024-04-25T09:58:49.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.983+0000 7f7577d89640 1 -- 172.21.15.139:0/633399421 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f755406b4d0 msgr2=0x7f755406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:49.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.983+0000 7f7577d89640 1 --2- 172.21.15.139:0/633399421 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f755406b4d0 0x7f755406d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f756c0070b0 tx=0x7f756c007040 comp rx=0 tx=0).stop 2024-04-25T09:58:49.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.983+0000 7f7577d89640 1 -- 172.21.15.139:0/633399421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7570144090 msgr2=0x7f757013fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:49.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.983+0000 7f7577d89640 1 --2- 172.21.15.139:0/633399421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7570144090 0x7f757013fa80 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f756802a220 tx=0x7f75680046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.983+0000 7f7577d89640 1 -- 172.21.15.139:0/633399421 shutdown_connections 2024-04-25T09:58:49.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.983+0000 7f7577d89640 1 --2- 172.21.15.139:0/633399421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7570144090 0x7f757013fa80 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.983+0000 7f7577d89640 1 --2- 172.21.15.139:0/633399421 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f755406b4d0 0x7f755406d990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.983+0000 7f7577d89640 1 --2- 172.21.15.139:0/633399421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75700cc9e0 0x7f75701f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.983+0000 7f7577d89640 1 --2- 172.21.15.139:0/633399421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f757006ea40 0x7f757015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.983+0000 7f7577d89640 1 -- 172.21.15.139:0/633399421 >> 172.21.15.139:0/633399421 conn(0x7f7570067000 msgr2=0x7f75700b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:49.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.983+0000 7f7577d89640 1 -- 172.21.15.139:0/633399421 shutdown_connections 2024-04-25T09:58:49.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:49.983+0000 7f7577d89640 1 -- 172.21.15.139:0/633399421 wait complete. 2024-04-25T09:58:50.001 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:58:50.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.097+0000 7f0120510640 1 Processor -- start 2024-04-25T09:58:50.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.097+0000 7f0120510640 1 -- start start 2024-04-25T09:58:50.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.098+0000 7f0120510640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f011815a870 0x7f01180b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.098+0000 7f0120510640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0118070970 0x7f01180d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.098+0000 7f0120510640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01180b45f0 con 0x7f01180b4190 2024-04-25T09:58:50.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.098+0000 7f0120510640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01180d1020 con 0x7f011815a870 2024-04-25T09:58:50.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.098+0000 7f0120510640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01180b5190 con 0x7f0118070970 2024-04-25T09:58:50.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.098+0000 7f011daab640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f01180b4190 0x7f0118070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55088/0 (socket says 172.21.15.139:55088) 2024-04-25T09:58:50.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.098+0000 7f011daab640 1 -- 172.21.15.139:0/1465904179 learned_addr learned my addr 172.21.15.139:0/1465904179 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:50.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138422327 0 0) 0x7f01180b45f0 con 0x7f01180b4190 2024-04-25T09:58:50.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011eaad640 1 --2- 172.21.15.139:0/1465904179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0118070970 0x7f01180d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0104003520 con 0x7f01180b4190 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011e2ac640 1 --2- 172.21.15.139:0/1465904179 >> v2:172.21.15.150:3300/0 conn(0x7f011815a870 0x7f01180b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3929843609 0 0) 0x7f0104003520 con 0x7f01180b4190 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0118070970 msgr2=0x7f01180d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 --2- 172.21.15.139:0/1465904179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0118070970 0x7f01180d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 >> v2:172.21.15.150:3300/0 conn(0x7f011815a870 msgr2=0x7f01180b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 --2- 172.21.15.139:0/1465904179 >> v2:172.21.15.150:3300/0 conn(0x7f011815a870 0x7f01180b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01180d0d40 con 0x7f01180b4190 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f0108001670 con 0x7f01180b4190 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 >> v1:172.21.15.139:6789/0 conn(0x7f01180b4190 legacy=0x7f0118070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 --2- 172.21.15.139:0/1465904179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0118070970 0x7f01040039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 --2- 172.21.15.139:0/1465904179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0104003f30 0x7f0104004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 --2- 172.21.15.139:0/1465904179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0104004d40 0x7f01040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01180d1020 con 0x7f0104003f30 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01180b5190 con 0x7f0104004d40 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010400baa0 con 0x7f0118070970 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011daab640 1 --2- 172.21.15.139:0/1465904179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0104003f30 0x7f0104004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011eaad640 1 --2- 172.21.15.139:0/1465904179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0104004d40 0x7f01040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011e2ac640 1 --2- 172.21.15.139:0/1465904179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0118070970 0x7f01040039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011daab640 1 -- 172.21.15.139:0/1465904179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0118070970 msgr2=0x7f01040039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011daab640 1 --2- 172.21.15.139:0/1465904179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0118070970 0x7f01040039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011daab640 1 -- 172.21.15.139:0/1465904179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0104004d40 msgr2=0x7f01040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011daab640 1 --2- 172.21.15.139:0/1465904179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0104004d40 0x7f01040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.099+0000 7f011daab640 1 -- 172.21.15.139:0/1465904179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0104003520 con 0x7f0104003f30 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.100+0000 7f011daab640 1 --2- 172.21.15.139:0/1465904179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0104003f30 0x7f0104004800 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f010800c240 tx=0x7f0104005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a52a4f9e6643d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.100+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010802e070 con 0x7f0104003f30 2024-04-25T09:58:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.100+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0108005af0 con 0x7f0104003f30 2024-04-25T09:58:50.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.100+0000 7f011d2aa640 1 -- 172.21.15.139:0/1465904179 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108007d30 con 0x7f0104003f30 2024-04-25T09:58:50.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.101+0000 7f0120510640 1 -- 172.21.15.139:0/1465904179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0104003f30 msgr2=0x7f0104004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.101+0000 7f0120510640 1 --2- 172.21.15.139:0/1465904179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0104003f30 0x7f0104004800 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f010800c240 tx=0x7f0104005830 comp rx=0 tx=0).stop 2024-04-25T09:58:50.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.101+0000 7f011eaad640 1 -- 172.21.15.139:0/1465904179 reap_dead start 2024-04-25T09:58:50.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.101+0000 7f0120510640 1 -- 172.21.15.139:0/1465904179 shutdown_connections 2024-04-25T09:58:50.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.101+0000 7f0120510640 1 -- 172.21.15.139:0/1465904179 >> 172.21.15.139:0/1465904179 conn(0x7f01180b23f0 msgr2=0x7f01180b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.101+0000 7f0120510640 1 -- 172.21.15.139:0/1465904179 shutdown_connections 2024-04-25T09:58:50.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.101+0000 7f0120510640 1 -- 172.21.15.139:0/1465904179 wait complete. 2024-04-25T09:58:50.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.102+0000 7f0120510640 1 Processor -- start 2024-04-25T09:58:50.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.102+0000 7f0120510640 1 -- start start 2024-04-25T09:58:50.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f0120510640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0104003f30 0x7f011813f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f0120510640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0118142eb0 0x7f01180b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f0120510640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0118143330 0x7f01180b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f0120510640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01180d05f0 con 0x7f0118142eb0 2024-04-25T09:58:50.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f0120510640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01180b4e20 con 0x7f0118143330 2024-04-25T09:58:50.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f0120510640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01180d0770 con 0x7f0104003f30 2024-04-25T09:58:50.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f011daab640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0118142eb0 0x7f01180b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f011daab640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0118142eb0 0x7f01180b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48110/0 (socket says 172.21.15.139:48110) 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f011daab640 1 -- 172.21.15.139:0/1924884938 learned_addr learned my addr 172.21.15.139:0/1924884938 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f011e2ac640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0104003f30 0x7f011813f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f011eaad640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0118143330 0x7f01180b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f011daab640 1 -- 172.21.15.139:0/1924884938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0104003f30 msgr2=0x7f011813f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f011daab640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0104003f30 0x7f011813f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f011daab640 1 -- 172.21.15.139:0/1924884938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0118143330 msgr2=0x7f01180b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f011daab640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0118143330 0x7f01180b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f011daab640 1 -- 172.21.15.139:0/1924884938 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011815c160 con 0x7f0118142eb0 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f011daab640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0118142eb0 0x7f01180b0d40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f010802ae30 tx=0x7f01180d0f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f010effd640 1 -- 172.21.15.139:0/1924884938 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108042070 con 0x7f0118142eb0 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f010effd640 1 -- 172.21.15.139:0/1924884938 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0108007480 con 0x7f0118142eb0 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f0120510640 1 -- 172.21.15.139:0/1924884938 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011815c440 con 0x7f0118142eb0 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f010effd640 1 -- 172.21.15.139:0/1924884938 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108007d30 con 0x7f0118142eb0 2024-04-25T09:58:50.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.103+0000 7f0120510640 1 -- 172.21.15.139:0/1924884938 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011815ca00 con 0x7f0118142eb0 2024-04-25T09:58:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.104+0000 7f0120510640 1 -- 172.21.15.139:0/1924884938 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0118074520 con 0x7f0118142eb0 2024-04-25T09:58:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.105+0000 7f010effd640 1 -- 172.21.15.139:0/1924884938 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0108045070 con 0x7f0118142eb0 2024-04-25T09:58:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.105+0000 7f010effd640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f00fc06b500 0x7f00fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.105+0000 7f010effd640 1 -- 172.21.15.139:0/1924884938 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f01080aad10 con 0x7f0118142eb0 2024-04-25T09:58:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.105+0000 7f011e2ac640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f00fc06b500 0x7f00fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.106+0000 7f011e2ac640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f00fc06b500 0x7f00fc06d9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0114007520 tx=0x7f011400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.109+0000 7f010effd640 1 -- 172.21.15.139:0/1924884938 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0108005ac0 con 0x7f0118142eb0 2024-04-25T09:58:50.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.280+0000 7f0120510640 1 -- 172.21.15.139:0/1924884938 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0118072280 con 0x7f0118142eb0 2024-04-25T09:58:50.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.283+0000 7f010effd640 1 -- 172.21.15.139:0/1924884938 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0108079dd0 con 0x7f0118142eb0 2024-04-25T09:58:50.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.285+0000 7f0120510640 1 -- 172.21.15.139:0/1924884938 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f00fc06b500 msgr2=0x7f00fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.285+0000 7f0120510640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f00fc06b500 0x7f00fc06d9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0114007520 tx=0x7f011400b040 comp rx=0 tx=0).stop 2024-04-25T09:58:50.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.285+0000 7f0120510640 1 -- 172.21.15.139:0/1924884938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0118142eb0 msgr2=0x7f01180b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.285+0000 7f0120510640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0118142eb0 0x7f01180b0d40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f010802ae30 tx=0x7f01180d0f10 comp rx=0 tx=0).stop 2024-04-25T09:58:50.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.285+0000 7f0120510640 1 -- 172.21.15.139:0/1924884938 shutdown_connections 2024-04-25T09:58:50.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.285+0000 7f0120510640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0118142eb0 0x7f01180b0d40 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.285+0000 7f0120510640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f00fc06b500 0x7f00fc06d9c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.285+0000 7f0120510640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0118143330 0x7f01180b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.285+0000 7f0120510640 1 --2- 172.21.15.139:0/1924884938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0104003f30 0x7f011813f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.285+0000 7f0120510640 1 -- 172.21.15.139:0/1924884938 >> 172.21.15.139:0/1924884938 conn(0x7f01180b23f0 msgr2=0x7f01180b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.285+0000 7f0120510640 1 -- 172.21.15.139:0/1924884938 shutdown_connections 2024-04-25T09:58:50.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.285+0000 7f0120510640 1 -- 172.21.15.139:0/1924884938 wait complete. 2024-04-25T09:58:50.300 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T09:58:50.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.392+0000 7f0dd71d5640 1 Processor -- start 2024-04-25T09:58:50.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.392+0000 7f0dd71d5640 1 -- start start 2024-04-25T09:58:50.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.393+0000 7f0dd71d5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0dd006aff0 0x7f0dd00b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.393+0000 7f0dd71d5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd006b480 con 0x7f0dd0160e90 2024-04-25T09:58:50.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.393+0000 7f0dd71d5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd00d0400 con 0x7f0dd006ab60 2024-04-25T09:58:50.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.393+0000 7f0dd71d5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd00d14a0 con 0x7f0dd006aff0 2024-04-25T09:58:50.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.393+0000 7f0dd5772640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0dd006aff0 0x7f0dd00b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.393+0000 7f0dd4f71640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0dd0160e90 0x7f0dd00afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55104/0 (socket says 172.21.15.139:55104) 2024-04-25T09:58:50.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.393+0000 7f0dd5772640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0dd006aff0 0x7f0dd00b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48116/0 (socket says 172.21.15.139:48116) 2024-04-25T09:58:50.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.393+0000 7f0dd4f71640 1 -- 172.21.15.139:0/4043890246 learned_addr learned my addr 172.21.15.139:0/4043890246 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:50.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.394+0000 7f0dd5772640 1 -- 172.21.15.139:0/4043890246 >> v1:172.21.15.150:6789/0 conn(0x7f0dd006ab60 legacy=0x7f0dd00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.394+0000 7f0dd5772640 1 -- 172.21.15.139:0/4043890246 >> v1:172.21.15.139:6789/0 conn(0x7f0dd0160e90 legacy=0x7f0dd00afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.394+0000 7f0dcf7fe640 1 -- 172.21.15.139:0/4043890246 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809760975 0 0) 0x7f0dd006b480 con 0x7f0dd0160e90 2024-04-25T09:58:50.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.394+0000 7f0dd5772640 1 -- 172.21.15.139:0/4043890246 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd00d06e0 con 0x7f0dd006aff0 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.394+0000 7f0dd5772640 1 --2- 172.21.15.139:0/4043890246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd006aff0 0x7f0dd00b3e00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009dc0 tx=0x7f0dc802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14ca9b3fb03f3ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.394+0000 7f0dcf7fe640 1 -- 172.21.15.139:0/4043890246 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8007740 con 0x7f0dd006aff0 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.394+0000 7f0dcf7fe640 1 -- 172.21.15.139:0/4043890246 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc80078e0 con 0x7f0dd006aff0 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.394+0000 7f0dcf7fe640 1 -- 172.21.15.139:0/4043890246 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8007cb0 con 0x7f0dd006aff0 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.395+0000 7f0dd71d5640 1 -- 172.21.15.139:0/4043890246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd006aff0 msgr2=0x7f0dd00b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.395+0000 7f0dd71d5640 1 --2- 172.21.15.139:0/4043890246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd006aff0 0x7f0dd00b3e00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009dc0 tx=0x7f0dc802a640 comp rx=0 tx=0).stop 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.395+0000 7f0dd71d5640 1 -- 172.21.15.139:0/4043890246 shutdown_connections 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.395+0000 7f0dd71d5640 1 --2- 172.21.15.139:0/4043890246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd006aff0 0x7f0dd00b3e00 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.395+0000 7f0dd71d5640 1 -- 172.21.15.139:0/4043890246 >> 172.21.15.139:0/4043890246 conn(0x7f0dd00b23f0 msgr2=0x7f0dd00b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.395+0000 7f0dd71d5640 1 -- 172.21.15.139:0/4043890246 shutdown_connections 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.395+0000 7f0dd71d5640 1 -- 172.21.15.139:0/4043890246 wait complete. 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.396+0000 7f0dd71d5640 1 Processor -- start 2024-04-25T09:58:50.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.396+0000 7f0dd71d5640 1 -- start start 2024-04-25T09:58:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd71d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0dd006ab60 0x7f0dd015d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd71d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0dd006aff0 0x7f0dd01560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd71d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd0160e90 0x7f0dd0156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd71d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd00d0f80 con 0x7f0dd0160e90 2024-04-25T09:58:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd71d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd00d0ca0 con 0x7f0dd006aff0 2024-04-25T09:58:50.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd71d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0161270 con 0x7f0dd006ab60 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd5772640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd0160e90 0x7f0dd0156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd5772640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd0160e90 0x7f0dd0156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48128/0 (socket says 172.21.15.139:48128) 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd5772640 1 -- 172.21.15.139:0/2588447085 learned_addr learned my addr 172.21.15.139:0/2588447085 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd4f71640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0dd006ab60 0x7f0dd015d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dcffff640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0dd006aff0 0x7f0dd01560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd5772640 1 -- 172.21.15.139:0/2588447085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0dd006ab60 msgr2=0x7f0dd015d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd5772640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0dd006ab60 0x7f0dd015d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd5772640 1 -- 172.21.15.139:0/2588447085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0dd006aff0 msgr2=0x7f0dd01560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd5772640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0dd006aff0 0x7f0dd01560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd5772640 1 -- 172.21.15.139:0/2588447085 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd0158c80 con 0x7f0dd0160e90 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.397+0000 7f0dd4f71640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0dd006ab60 0x7f0dd015d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.398+0000 7f0dd5772640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd0160e90 0x7f0dd0156620 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f0dc802ac30 tx=0x7f0dc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.398+0000 7f0dcd7fa640 1 -- 172.21.15.139:0/2588447085 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8040070 con 0x7f0dd0160e90 2024-04-25T09:58:50.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.398+0000 7f0dcd7fa640 1 -- 172.21.15.139:0/2588447085 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc8031bc0 con 0x7f0dd0160e90 2024-04-25T09:58:50.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.398+0000 7f0dd71d5640 1 -- 172.21.15.139:0/2588447085 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd0158f60 con 0x7f0dd0160e90 2024-04-25T09:58:50.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.398+0000 7f0dcd7fa640 1 -- 172.21.15.139:0/2588447085 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8045460 con 0x7f0dd0160e90 2024-04-25T09:58:50.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.398+0000 7f0dd71d5640 1 -- 172.21.15.139:0/2588447085 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd0159520 con 0x7f0dd0160e90 2024-04-25T09:58:50.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.399+0000 7f0dd71d5640 1 -- 172.21.15.139:0/2588447085 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd0162380 con 0x7f0dd0160e90 2024-04-25T09:58:50.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.399+0000 7f0dcd7fa640 1 -- 172.21.15.139:0/2588447085 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0dc8039040 con 0x7f0dd0160e90 2024-04-25T09:58:50.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.400+0000 7f0dcd7fa640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0dbc06b4d0 0x7f0dbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.400+0000 7f0dcd7fa640 1 -- 172.21.15.139:0/2588447085 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f0dc80ab9f0 con 0x7f0dd0160e90 2024-04-25T09:58:50.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.400+0000 7f0dd4f71640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0dbc06b4d0 0x7f0dbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.400+0000 7f0dd4f71640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0dbc06b4d0 0x7f0dbc06d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0000f60 tx=0x7f0dc0007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.404+0000 7f0dcd7fa640 1 -- 172.21.15.139:0/2588447085 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0dd0162380 con 0x7f0dd0160e90 2024-04-25T09:58:50.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.640+0000 7f0dd71d5640 1 -- 172.21.15.139:0/2588447085 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f0dd0074520 con 0x7f0dd0160e90 2024-04-25T09:58:50.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.981+0000 7f0dcd7fa640 1 -- 172.21.15.139:0/2588447085 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v278) v1 ==== 166+0+0 (secure 0 0 0) 0x7f0dc802ae20 con 0x7f0dd0160e90 2024-04-25T09:58:50.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.983+0000 7f0dd71d5640 1 -- 172.21.15.139:0/2588447085 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0dbc06b4d0 msgr2=0x7f0dbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.983+0000 7f0dd71d5640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0dbc06b4d0 0x7f0dbc06d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0000f60 tx=0x7f0dc0007c10 comp rx=0 tx=0).stop 2024-04-25T09:58:50.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.983+0000 7f0dd71d5640 1 -- 172.21.15.139:0/2588447085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd0160e90 msgr2=0x7f0dd0156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.983+0000 7f0dd71d5640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd0160e90 0x7f0dd0156620 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f0dc802ac30 tx=0x7f0dc8004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.983+0000 7f0dd71d5640 1 -- 172.21.15.139:0/2588447085 shutdown_connections 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.983+0000 7f0dd71d5640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dd0160e90 0x7f0dd0156620 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.983+0000 7f0dd71d5640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0dbc06b4d0 0x7f0dbc06d990 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.983+0000 7f0dd71d5640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0dd006aff0 0x7f0dd01560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.983+0000 7f0dd71d5640 1 --2- 172.21.15.139:0/2588447085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0dd006ab60 0x7f0dd015d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.983+0000 7f0dd71d5640 1 -- 172.21.15.139:0/2588447085 >> 172.21.15.139:0/2588447085 conn(0x7f0dd00b23f0 msgr2=0x7f0dd006fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.984+0000 7f0dd71d5640 1 -- 172.21.15.139:0/2588447085 shutdown_connections 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:50.984+0000 7f0dd71d5640 1 -- 172.21.15.139:0/2588447085 wait complete. 2024-04-25T09:58:51.000 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T09:58:51.000 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T09:58:51.000 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T09:58:51.000 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:51.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.095+0000 7f50c484a640 1 Processor -- start 2024-04-25T09:58:51.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.095+0000 7f50c484a640 1 -- start start 2024-04-25T09:58:51.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50c484a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c015efb0 0x7f50c00d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50c484a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f50c00cf900 0x7f50c006c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50c484a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c00cfe70 con 0x7f50c006cd80 2024-04-25T09:58:51.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50c484a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c00d0a80 con 0x7f50c00cf900 2024-04-25T09:58:51.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50c484a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c00d0150 con 0x7f50c015efb0 2024-04-25T09:58:51.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50be59c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c015efb0 0x7f50c00d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50b7fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f50c00cf900 0x7f50c006c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50be59c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c015efb0 0x7f50c00d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:46832/0 (socket says 172.21.15.139:46832) 2024-04-25T09:58:51.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50be59c640 1 -- 172.21.15.139:0/903645284 learned_addr learned my addr 172.21.15.139:0/903645284 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:51.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50bed9d640 1 --1- 172.21.15.139:0/903645284 >> v1:172.21.15.150:6789/0 conn(0x7f50c006cd80 0x7f50c0099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:50456/0 (socket says 172.21.15.139:50456) 2024-04-25T09:58:51.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50be59c640 1 -- 172.21.15.139:0/903645284 >> v2:172.21.15.150:3300/0 conn(0x7f50c00cf900 msgr2=0x7f50c006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50be59c640 1 --2- 172.21.15.139:0/903645284 >> v2:172.21.15.150:3300/0 conn(0x7f50c00cf900 0x7f50c006c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50be59c640 1 -- 172.21.15.139:0/903645284 >> v1:172.21.15.150:6789/0 conn(0x7f50c006cd80 legacy=0x7f50c0099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50be59c640 1 -- 172.21.15.139:0/903645284 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00d0430 con 0x7f50c015efb0 2024-04-25T09:58:51.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.096+0000 7f50b7fff640 1 --2- 172.21.15.139:0/903645284 >> v2:172.21.15.150:3300/0 conn(0x7f50c00cf900 0x7f50c006c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.097+0000 7f50be59c640 1 --2- 172.21.15.139:0/903645284 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c015efb0 0x7f50c00d0f10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009870 tx=0x7f50b002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9564ad17b3e447b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.097+0000 7f50bdd9b640 1 -- 172.21.15.139:0/903645284 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0037070 con 0x7f50c015efb0 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.097+0000 7f50bdd9b640 1 -- 172.21.15.139:0/903645284 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b0032440 con 0x7f50c015efb0 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.097+0000 7f50bdd9b640 1 -- 172.21.15.139:0/903645284 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0032810 con 0x7f50c015efb0 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.098+0000 7f50c484a640 1 -- 172.21.15.139:0/903645284 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c015efb0 msgr2=0x7f50c00d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.098+0000 7f50c484a640 1 --2- 172.21.15.139:0/903645284 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c015efb0 0x7f50c00d0f10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009870 tx=0x7f50b002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.098+0000 7f50c484a640 1 -- 172.21.15.139:0/903645284 shutdown_connections 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.098+0000 7f50c484a640 1 --2- 172.21.15.139:0/903645284 >> v2:172.21.15.150:3300/0 conn(0x7f50c00cf900 0x7f50c006c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.098+0000 7f50c484a640 1 --2- 172.21.15.139:0/903645284 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c015efb0 0x7f50c00d0f10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.098+0000 7f50c484a640 1 -- 172.21.15.139:0/903645284 >> 172.21.15.139:0/903645284 conn(0x7f50c00d1790 msgr2=0x7f50c00d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.098+0000 7f50c484a640 1 -- 172.21.15.139:0/903645284 shutdown_connections 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.098+0000 7f50c484a640 1 -- 172.21.15.139:0/903645284 wait complete. 2024-04-25T09:58:51.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.099+0000 7f50c484a640 1 Processor -- start 2024-04-25T09:58:51.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.099+0000 7f50c484a640 1 -- start start 2024-04-25T09:58:51.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50c484a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50c006cd80 0x7f50c009f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50c484a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c00cf900 0x7f50c014c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50c484a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50c015efb0 0x7f50c014a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50c484a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c009a2e0 con 0x7f50c015efb0 2024-04-25T09:58:51.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50c484a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c009a160 con 0x7f50c006cd80 2024-04-25T09:58:51.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50c484a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c00cbde0 con 0x7f50c00cf900 2024-04-25T09:58:51.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50bed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50c015efb0 0x7f50c014a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50bed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50c015efb0 0x7f50c014a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48132/0 (socket says 172.21.15.139:48132) 2024-04-25T09:58:51.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50bed9d640 1 -- 172.21.15.139:0/4066000222 learned_addr learned my addr 172.21.15.139:0/4066000222 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:51.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50be59c640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50c006cd80 0x7f50c009f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50b7fff640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c00cf900 0x7f50c014c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50bed9d640 1 -- 172.21.15.139:0/4066000222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c00cf900 msgr2=0x7f50c014c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.100+0000 7f50bed9d640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c00cf900 0x7f50c014c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.101+0000 7f50bed9d640 1 -- 172.21.15.139:0/4066000222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50c006cd80 msgr2=0x7f50c009f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.101+0000 7f50bed9d640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50c006cd80 0x7f50c009f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.101+0000 7f50bed9d640 1 -- 172.21.15.139:0/4066000222 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c014c5e0 con 0x7f50c015efb0 2024-04-25T09:58:51.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.101+0000 7f50bed9d640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50c015efb0 0x7f50c014a7b0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f50b800c730 tx=0x7f50b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.101+0000 7f50b6ffd640 1 -- 172.21.15.139:0/4066000222 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8017070 con 0x7f50c015efb0 2024-04-25T09:58:51.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.101+0000 7f50b6ffd640 1 -- 172.21.15.139:0/4066000222 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b800ce40 con 0x7f50c015efb0 2024-04-25T09:58:51.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.101+0000 7f50b6ffd640 1 -- 172.21.15.139:0/4066000222 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b800a840 con 0x7f50c015efb0 2024-04-25T09:58:51.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.101+0000 7f50c484a640 1 -- 172.21.15.139:0/4066000222 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c014b000 con 0x7f50c015efb0 2024-04-25T09:58:51.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.101+0000 7f50c484a640 1 -- 172.21.15.139:0/4066000222 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c014b5c0 con 0x7f50c015efb0 2024-04-25T09:58:51.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.102+0000 7f50c484a640 1 -- 172.21.15.139:0/4066000222 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c014a4d0 con 0x7f50c015efb0 2024-04-25T09:58:51.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.102+0000 7f50b6ffd640 1 -- 172.21.15.139:0/4066000222 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50b801a070 con 0x7f50c015efb0 2024-04-25T09:58:51.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.103+0000 7f50b6ffd640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50a006b4d0 0x7f50a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.103+0000 7f50b6ffd640 1 -- 172.21.15.139:0/4066000222 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f50b8089540 con 0x7f50c015efb0 2024-04-25T09:58:51.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.103+0000 7f50be59c640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50a006b4d0 0x7f50a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.103+0000 7f50be59c640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50a006b4d0 0x7f50a006d990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f50b0034000 tx=0x7f50b002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.107+0000 7f50b6ffd640 1 -- 172.21.15.139:0/4066000222 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50c014a4d0 con 0x7f50c015efb0 2024-04-25T09:58:51.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.287+0000 7f50c484a640 1 -- 172.21.15.139:0/4066000222 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50c0074520 con 0x7f50c015efb0 2024-04-25T09:58:51.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.287+0000 7f50b6ffd640 1 -- 172.21.15.139:0/4066000222 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+13476 (secure 0 0 0) 0x7f50b8057f20 con 0x7f50c015efb0 2024-04-25T09:58:51.291 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:51.291 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":80,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:58:49.976255+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6837/1384380923":"2024-04-26T09:58:46.944284+0000","172.21.15.139:6836/1384380923":"2024-04-26T09:58:46.944284+0000","172.21.15.150:6834/1932986598":"2024-04-26T09:57:29.527412+0000","172.21.15.150:6832/1932986598":"2024-04-26T09:57:29.527412+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:51.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.289+0000 7f50c484a640 1 -- 172.21.15.139:0/4066000222 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50a006b4d0 msgr2=0x7f50a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.289+0000 7f50c484a640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50a006b4d0 0x7f50a006d990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f50b0034000 tx=0x7f50b002e000 comp rx=0 tx=0).stop 2024-04-25T09:58:51.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.289+0000 7f50c484a640 1 -- 172.21.15.139:0/4066000222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50c015efb0 msgr2=0x7f50c014a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.289+0000 7f50c484a640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50c015efb0 0x7f50c014a7b0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f50b800c730 tx=0x7f50b800cbf0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.289+0000 7f50c484a640 1 -- 172.21.15.139:0/4066000222 shutdown_connections 2024-04-25T09:58:51.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.289+0000 7f50c484a640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50c015efb0 0x7f50c014a7b0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.289+0000 7f50c484a640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50a006b4d0 0x7f50a006d990 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.290+0000 7f50c484a640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c00cf900 0x7f50c014c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.290+0000 7f50c484a640 1 --2- 172.21.15.139:0/4066000222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50c006cd80 0x7f50c009f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.290+0000 7f50c484a640 1 -- 172.21.15.139:0/4066000222 >> 172.21.15.139:0/4066000222 conn(0x7f50c00d1790 msgr2=0x7f50c009a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.290+0000 7f50c484a640 1 -- 172.21.15.139:0/4066000222 shutdown_connections 2024-04-25T09:58:51.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.290+0000 7f50c484a640 1 -- 172.21.15.139:0/4066000222 wait complete. 2024-04-25T09:58:51.304 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:58:51.305 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:58:51.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.400+0000 7f9357330640 1 Processor -- start 2024-04-25T09:58:51.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.400+0000 7f9357330640 1 -- start start 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.401+0000 7f9357330640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500af8a0 0x7f93500b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.401+0000 7f9357330640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93500c6cc0 con 0x7f93500afc80 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.401+0000 7f9357330640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93500c6fa0 con 0x7f935006b700 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.401+0000 7f9357330640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500c7280 con 0x7f93500af8a0 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.401+0000 7f93548cb640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f93500afc80 0x7f93500cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55110/0 (socket says 172.21.15.139:55110) 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.401+0000 7f93548cb640 1 -- 172.21.15.139:0/573236958 learned_addr learned my addr 172.21.15.139:0/573236958 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.401+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355282356 0 0) 0x7f93500c6cc0 con 0x7f93500afc80 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.401+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f933c003520 con 0x7f93500afc80 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.401+0000 7f93550cc640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500af8a0 0x7f93500b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2414724469 0 0) 0x7f933c003520 con 0x7f93500afc80 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500af8a0 msgr2=0x7f93500b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500af8a0 0x7f93500b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 >> v1:172.21.15.150:6789/0 conn(0x7f935006b700 legacy=0x7f935006ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93500cd090 con 0x7f93500afc80 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f9340001670 con 0x7f93500afc80 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 >> v1:172.21.15.139:6789/0 conn(0x7f93500afc80 legacy=0x7f93500cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f933c0039e0 0x7f933c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c004390 0x7f933c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500af8a0 0x7f933c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500c7280 con 0x7f933c004390 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c00b9c0 con 0x7f933c0039e0 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c00bbc0 con 0x7f93500af8a0 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f93548cb640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c004390 0x7f933c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f93550cc640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f933c0039e0 0x7f933c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f93558cd640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500af8a0 0x7f933c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f93548cb640 1 -- 172.21.15.139:0/573236958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500af8a0 msgr2=0x7f933c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f93548cb640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500af8a0 0x7f933c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f93548cb640 1 -- 172.21.15.139:0/573236958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f933c0039e0 msgr2=0x7f933c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f93548cb640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f933c0039e0 0x7f933c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f93548cb640 1 -- 172.21.15.139:0/573236958 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c003520 con 0x7f933c004390 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.402+0000 7f93558cd640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500af8a0 0x7f933c0050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.403+0000 7f93548cb640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c004390 0x7f933c004860 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f934000c240 tx=0x7f933c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711faeec69ad5225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.403+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340007d30 con 0x7f933c004390 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.403+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9340005af0 con 0x7f933c004390 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.403+0000 7f9347fff640 1 -- 172.21.15.139:0/573236958 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340007070 con 0x7f933c004390 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.403+0000 7f9357330640 1 -- 172.21.15.139:0/573236958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c004390 msgr2=0x7f933c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.403+0000 7f9357330640 1 --2- 172.21.15.139:0/573236958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c004390 0x7f933c004860 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f934000c240 tx=0x7f933c005830 comp rx=0 tx=0).stop 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.403+0000 7f93558cd640 1 -- 172.21.15.139:0/573236958 reap_dead start 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.403+0000 7f9357330640 1 -- 172.21.15.139:0/573236958 shutdown_connections 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.403+0000 7f9357330640 1 -- 172.21.15.139:0/573236958 >> 172.21.15.139:0/573236958 conn(0x7f93500b2230 msgr2=0x7f93500b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.404+0000 7f9357330640 1 -- 172.21.15.139:0/573236958 shutdown_connections 2024-04-25T09:58:51.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.404+0000 7f9357330640 1 -- 172.21.15.139:0/573236958 wait complete. 2024-04-25T09:58:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.405+0000 7f9357330640 1 Processor -- start 2024-04-25T09:58:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.405+0000 7f9357330640 1 -- start start 2024-04-25T09:58:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.405+0000 7f9357330640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f933c004390 0x7f9350152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.405+0000 7f9357330640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93500b59b0 0x7f93500b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.405+0000 7f9357330640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93500b33c0 0x7f93500b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.405+0000 7f9357330640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935006b3f0 con 0x7f93500b59b0 2024-04-25T09:58:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.405+0000 7f9357330640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935006b270 con 0x7f93500b33c0 2024-04-25T09:58:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.405+0000 7f9357330640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b2d50 con 0x7f933c004390 2024-04-25T09:58:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.405+0000 7f93548cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93500b59b0 0x7f93500b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.405+0000 7f93548cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93500b59b0 0x7f93500b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48140/0 (socket says 172.21.15.139:48140) 2024-04-25T09:58:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.405+0000 7f93548cb640 1 -- 172.21.15.139:0/1643289323 learned_addr learned my addr 172.21.15.139:0/1643289323 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.406+0000 7f93548cb640 1 -- 172.21.15.139:0/1643289323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f933c004390 msgr2=0x7f9350152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.406+0000 7f93548cb640 1 --2- 172.21.15.139:0/1643289323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f933c004390 0x7f9350152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.406+0000 7f93548cb640 1 -- 172.21.15.139:0/1643289323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93500b33c0 msgr2=0x7f93500b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.406+0000 7f93548cb640 1 --2- 172.21.15.139:0/1643289323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93500b33c0 0x7f93500b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.406+0000 7f93548cb640 1 -- 172.21.15.139:0/1643289323 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93500b4140 con 0x7f93500b59b0 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.406+0000 7f93548cb640 1 --2- 172.21.15.139:0/1643289323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93500b59b0 0x7f93500b5e30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f934000c360 tx=0x7f93500c6e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.406+0000 7f9345ffb640 1 -- 172.21.15.139:0/1643289323 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340042020 con 0x7f93500b59b0 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.406+0000 7f9345ffb640 1 -- 172.21.15.139:0/1643289323 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9340004c10 con 0x7f93500b59b0 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.406+0000 7f9345ffb640 1 -- 172.21.15.139:0/1643289323 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340007d30 con 0x7f93500b59b0 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.406+0000 7f9357330640 1 -- 172.21.15.139:0/1643289323 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93500b4420 con 0x7f93500b59b0 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.406+0000 7f9357330640 1 -- 172.21.15.139:0/1643289323 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93500b49e0 con 0x7f93500b59b0 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.407+0000 7f9357330640 1 -- 172.21.15.139:0/1643289323 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9350074520 con 0x7f93500b59b0 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.408+0000 7f9345ffb640 1 -- 172.21.15.139:0/1643289323 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9340045020 con 0x7f93500b59b0 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.408+0000 7f9345ffb640 1 --2- 172.21.15.139:0/1643289323 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f933406b500 0x7f933406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.408+0000 7f9345ffb640 1 -- 172.21.15.139:0/1643289323 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f93400aa590 con 0x7f93500b59b0 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.408+0000 7f93550cc640 1 --2- 172.21.15.139:0/1643289323 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f933406b500 0x7f933406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.409+0000 7f93550cc640 1 --2- 172.21.15.139:0/1643289323 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f933406b500 0x7f933406d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9348009ab0 tx=0x7f93480095d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.412+0000 7f9345ffb640 1 -- 172.21.15.139:0/1643289323 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f934004c070 con 0x7f93500b59b0 2024-04-25T09:58:51.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.581+0000 7f9357330640 1 -- 172.21.15.139:0/1643289323 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93500af350 con 0x7f93500b59b0 2024-04-25T09:58:51.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.582+0000 7f9345ffb640 1 -- 172.21.15.139:0/1643289323 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9340031e40 con 0x7f93500b59b0 2024-04-25T09:58:51.585 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:58:51.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.583+0000 7f9357330640 1 -- 172.21.15.139:0/1643289323 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f933406b500 msgr2=0x7f933406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.583+0000 7f9357330640 1 --2- 172.21.15.139:0/1643289323 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f933406b500 0x7f933406d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9348009ab0 tx=0x7f93480095d0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.583+0000 7f9357330640 1 -- 172.21.15.139:0/1643289323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93500b59b0 msgr2=0x7f93500b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.583+0000 7f9357330640 1 --2- 172.21.15.139:0/1643289323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93500b59b0 0x7f93500b5e30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f934000c360 tx=0x7f93500c6e90 comp rx=0 tx=0).stop 2024-04-25T09:58:51.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.584+0000 7f9357330640 1 -- 172.21.15.139:0/1643289323 shutdown_connections 2024-04-25T09:58:51.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.584+0000 7f9357330640 1 --2- 172.21.15.139:0/1643289323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93500b59b0 0x7f93500b5e30 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.584+0000 7f9357330640 1 --2- 172.21.15.139:0/1643289323 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f933406b500 0x7f933406d9c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.584+0000 7f9357330640 1 --2- 172.21.15.139:0/1643289323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93500b33c0 0x7f93500b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.584+0000 7f9357330640 1 --2- 172.21.15.139:0/1643289323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f933c004390 0x7f9350152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.584+0000 7f9357330640 1 -- 172.21.15.139:0/1643289323 >> 172.21.15.139:0/1643289323 conn(0x7f93500b2230 msgr2=0x7f935006faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.584+0000 7f9357330640 1 -- 172.21.15.139:0/1643289323 shutdown_connections 2024-04-25T09:58:51.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.584+0000 7f9357330640 1 -- 172.21.15.139:0/1643289323 wait complete. 2024-04-25T09:58:51.598 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T09:58:51.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.692+0000 7fd44e89a640 1 Processor -- start 2024-04-25T09:58:51.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.692+0000 7fd44e89a640 1 -- start start 2024-04-25T09:58:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd44e89a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd44815cde0 0x7fd44815d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd44e89a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44815db50 0x7fd4480cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd44e89a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44815d1c0 con 0x7fd4480cc2a0 2024-04-25T09:58:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd44e89a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4480cc720 con 0x7fd44815cde0 2024-04-25T09:58:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd44e89a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480cd460 con 0x7fd44815db50 2024-04-25T09:58:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd447fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd44815cde0 0x7fd44815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd447fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd44815cde0 0x7fd44815d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48152/0 (socket says 172.21.15.139:48152) 2024-04-25T09:58:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd447fff640 1 -- 172.21.15.139:0/1311832263 learned_addr learned my addr 172.21.15.139:0/1311832263 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:51.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd447fff640 1 -- 172.21.15.139:0/1311832263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44815db50 msgr2=0x7fd4480cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd44ce37640 1 --2- 172.21.15.139:0/1311832263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44815db50 0x7fd4480cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd447fff640 1 --2- 172.21.15.139:0/1311832263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44815db50 0x7fd4480cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd447fff640 1 -- 172.21.15.139:0/1311832263 >> v1:172.21.15.150:6789/0 conn(0x7fd4480cc2a0 legacy=0x7fd44809a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd447fff640 1 -- 172.21.15.139:0/1311832263 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd448099bf0 con 0x7fd44815cde0 2024-04-25T09:58:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.693+0000 7fd44ce37640 1 --2- 172.21.15.139:0/1311832263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44815db50 0x7fd4480cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.694+0000 7fd447fff640 1 --2- 172.21.15.139:0/1311832263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd44815cde0 0x7fd44815d610 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fd438009870 tx=0x7fd43802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3210ca05103cb1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.694+0000 7fd446ffd640 1 -- 172.21.15.139:0/1311832263 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438037070 con 0x7fd44815cde0 2024-04-25T09:58:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.694+0000 7fd446ffd640 1 -- 172.21.15.139:0/1311832263 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd438032440 con 0x7fd44815cde0 2024-04-25T09:58:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.694+0000 7fd446ffd640 1 -- 172.21.15.139:0/1311832263 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438032810 con 0x7fd44815cde0 2024-04-25T09:58:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.695+0000 7fd44e89a640 1 -- 172.21.15.139:0/1311832263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd44815cde0 msgr2=0x7fd44815d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.695+0000 7fd44e89a640 1 --2- 172.21.15.139:0/1311832263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd44815cde0 0x7fd44815d610 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fd438009870 tx=0x7fd43802a270 comp rx=0 tx=0).stop 2024-04-25T09:58:51.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.695+0000 7fd44e89a640 1 -- 172.21.15.139:0/1311832263 shutdown_connections 2024-04-25T09:58:51.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.695+0000 7fd44e89a640 1 --2- 172.21.15.139:0/1311832263 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44815db50 0x7fd4480cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.695+0000 7fd44e89a640 1 --2- 172.21.15.139:0/1311832263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd44815cde0 0x7fd44815d610 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.695+0000 7fd44e89a640 1 -- 172.21.15.139:0/1311832263 >> 172.21.15.139:0/1311832263 conn(0x7fd4480d1790 msgr2=0x7fd4480d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.695+0000 7fd44e89a640 1 -- 172.21.15.139:0/1311832263 shutdown_connections 2024-04-25T09:58:51.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.695+0000 7fd44e89a640 1 -- 172.21.15.139:0/1311832263 wait complete. 2024-04-25T09:58:51.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.696+0000 7fd44e89a640 1 Processor -- start 2024-04-25T09:58:51.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.696+0000 7fd44e89a640 1 -- start start 2024-04-25T09:58:51.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd44e89a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4480cc2a0 0x7fd44806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd44e89a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44815cde0 0x7fd448151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd44e89a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44815db50 0x7fd448158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd44e89a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480cf4e0 con 0x7fd4480cc2a0 2024-04-25T09:58:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd44e89a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480cf360 con 0x7fd44815cde0 2024-04-25T09:58:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd44e89a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd448152400 con 0x7fd44815db50 2024-04-25T09:58:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd447fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4480cc2a0 0x7fd44806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd447fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4480cc2a0 0x7fd44806d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48168/0 (socket says 172.21.15.139:48168) 2024-04-25T09:58:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd447fff640 1 -- 172.21.15.139:0/464549331 learned_addr learned my addr 172.21.15.139:0/464549331 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd4477fe640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44815cde0 0x7fd448151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd44ce37640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44815db50 0x7fd448158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd447fff640 1 -- 172.21.15.139:0/464549331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44815db50 msgr2=0x7fd448158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd447fff640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44815db50 0x7fd448158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd447fff640 1 -- 172.21.15.139:0/464549331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44815cde0 msgr2=0x7fd448151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd447fff640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44815cde0 0x7fd448151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd447fff640 1 -- 172.21.15.139:0/464549331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44806d8d0 con 0x7fd4480cc2a0 2024-04-25T09:58:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd447fff640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4480cc2a0 0x7fd44806d2c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fd43802a2b0 tx=0x7fd4380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd444ff9640 1 -- 172.21.15.139:0/464549331 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438037040 con 0x7fd4480cc2a0 2024-04-25T09:58:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd444ff9640 1 -- 172.21.15.139:0/464549331 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd438042070 con 0x7fd4480cc2a0 2024-04-25T09:58:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd44e89a640 1 -- 172.21.15.139:0/464549331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44806dbb0 con 0x7fd4480cc2a0 2024-04-25T09:58:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd444ff9640 1 -- 172.21.15.139:0/464549331 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438030bd0 con 0x7fd4480cc2a0 2024-04-25T09:58:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.697+0000 7fd44e89a640 1 -- 172.21.15.139:0/464549331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4480b59b0 con 0x7fd4480cc2a0 2024-04-25T09:58:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.698+0000 7fd44e89a640 1 -- 172.21.15.139:0/464549331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd448074520 con 0x7fd4480cc2a0 2024-04-25T09:58:51.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.699+0000 7fd444ff9640 1 -- 172.21.15.139:0/464549331 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd438047460 con 0x7fd4480cc2a0 2024-04-25T09:58:51.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.699+0000 7fd444ff9640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd43406b4d0 0x7fd43406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:51.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.699+0000 7fd444ff9640 1 -- 172.21.15.139:0/464549331 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd4380aac50 con 0x7fd4480cc2a0 2024-04-25T09:58:51.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.699+0000 7fd4477fe640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd43406b4d0 0x7fd43406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:51.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.700+0000 7fd4477fe640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd43406b4d0 0x7fd43406d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd44806d800 tx=0x7fd43c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:51.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.703+0000 7fd444ff9640 1 -- 172.21.15.139:0/464549331 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4380b0020 con 0x7fd4480cc2a0 2024-04-25T09:58:51.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.942+0000 7fd44e89a640 1 -- 172.21.15.139:0/464549331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7fd4480d25f0 con 0x7fd4480cc2a0 2024-04-25T09:58:51.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.987+0000 7fd444ff9640 1 -- 172.21.15.139:0/464549331 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v279) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd438045360 con 0x7fd4480cc2a0 2024-04-25T09:58:51.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.989+0000 7fd44e89a640 1 -- 172.21.15.139:0/464549331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd43406b4d0 msgr2=0x7fd43406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.989+0000 7fd44e89a640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd43406b4d0 0x7fd43406d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd44806d800 tx=0x7fd43c009040 comp rx=0 tx=0).stop 2024-04-25T09:58:51.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.989+0000 7fd44e89a640 1 -- 172.21.15.139:0/464549331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4480cc2a0 msgr2=0x7fd44806d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.989+0000 7fd44e89a640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4480cc2a0 0x7fd44806d2c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fd43802a2b0 tx=0x7fd4380046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.989+0000 7fd44e89a640 1 -- 172.21.15.139:0/464549331 shutdown_connections 2024-04-25T09:58:51.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.989+0000 7fd44e89a640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd43406b4d0 0x7fd43406d990 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.989+0000 7fd44e89a640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44815db50 0x7fd448158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.989+0000 7fd44e89a640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44815cde0 0x7fd448151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.989+0000 7fd44e89a640 1 --2- 172.21.15.139:0/464549331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4480cc2a0 0x7fd44806d2c0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.989+0000 7fd44e89a640 1 -- 172.21.15.139:0/464549331 >> 172.21.15.139:0/464549331 conn(0x7fd4480d1790 msgr2=0x7fd4480c5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.989+0000 7fd44e89a640 1 -- 172.21.15.139:0/464549331 shutdown_connections 2024-04-25T09:58:51.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:51.989+0000 7fd44e89a640 1 -- 172.21.15.139:0/464549331 wait complete. 2024-04-25T09:58:52.004 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:58:52.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.101+0000 7fa6a564c640 1 Processor -- start 2024-04-25T09:58:52.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.101+0000 7fa6a564c640 1 -- start start 2024-04-25T09:58:52.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.101+0000 7fa6a564c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6a00cee50 0x7fa6a0168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.101+0000 7fa6a564c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a0161dc0 con 0x7fa6a01617e0 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.101+0000 7fa6a564c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a01620a0 con 0x7fa6a015dbf0 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.101+0000 7fa6a564c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00d1ee0 con 0x7fa6a00cee50 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69e7fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa6a01617e0 0x7fa6a00ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55112/0 (socket says 172.21.15.139:55112) 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69e7fc640 1 -- 172.21.15.139:0/701993278 learned_addr learned my addr 172.21.15.139:0/701993278 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69f7fe640 1 --2- 172.21.15.139:0/701993278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6a00cee50 0x7fa6a0168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660174828 0 0) 0x7fa6a0161dc0 con 0x7fa6a01617e0 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa688003520 con 0x7fa6a01617e0 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366306621 0 0) 0x7fa6a01620a0 con 0x7fa6a015dbf0 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a0161dc0 con 0x7fa6a015dbf0 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4278184376 0 0) 0x7fa688003520 con 0x7fa6a01617e0 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6a00cee50 msgr2=0x7fa6a0168b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 --2- 172.21.15.139:0/701993278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6a00cee50 0x7fa6a0168b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 >> v1:172.21.15.150:6789/0 conn(0x7fa6a015dbf0 legacy=0x7fa6a00b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a0162380 con 0x7fa6a01617e0 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fa694001670 con 0x7fa6a01617e0 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 >> v1:172.21.15.139:6789/0 conn(0x7fa6a01617e0 legacy=0x7fa6a00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 --2- 172.21.15.139:0/701993278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa688003b70 0x7fa688003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 --2- 172.21.15.139:0/701993278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6a00cee50 0x7fa688004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 --2- 172.21.15.139:0/701993278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa688004d40 0x7fa6880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00d1ee0 con 0x7fa688004d40 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68800bb20 con 0x7fa688003b70 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68800bd20 con 0x7fa6a00cee50 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69f7fe640 1 --2- 172.21.15.139:0/701993278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa688004d40 0x7fa6880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69e7fc640 1 --2- 172.21.15.139:0/701993278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6a00cee50 0x7fa688004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69effd640 1 --2- 172.21.15.139:0/701993278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa688003b70 0x7fa688003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69f7fe640 1 -- 172.21.15.139:0/701993278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6a00cee50 msgr2=0x7fa688004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.102+0000 7fa69f7fe640 1 --2- 172.21.15.139:0/701993278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6a00cee50 0x7fa688004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.103+0000 7fa69f7fe640 1 -- 172.21.15.139:0/701993278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa688003b70 msgr2=0x7fa688003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.103+0000 7fa69f7fe640 1 --2- 172.21.15.139:0/701993278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa688003b70 0x7fa688003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.103+0000 7fa69f7fe640 1 -- 172.21.15.139:0/701993278 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa688003520 con 0x7fa688004d40 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.103+0000 7fa69f7fe640 1 --2- 172.21.15.139:0/701993278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa688004d40 0x7fa6880051c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00ce3a0 tx=0x7fa688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7186a190b69221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.103+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698037070 con 0x7fa688004d40 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.103+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69802e050 con 0x7fa688004d40 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.103+0000 7fa69dffb640 1 -- 172.21.15.139:0/701993278 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698009c10 con 0x7fa688004d40 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.104+0000 7fa6a564c640 1 -- 172.21.15.139:0/701993278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa688004d40 msgr2=0x7fa6880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.104+0000 7fa6a564c640 1 --2- 172.21.15.139:0/701993278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa688004d40 0x7fa6880051c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00ce3a0 tx=0x7fa688005830 comp rx=0 tx=0).stop 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.104+0000 7fa69f7fe640 1 -- 172.21.15.139:0/701993278 reap_dead start 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.104+0000 7fa6a564c640 1 -- 172.21.15.139:0/701993278 shutdown_connections 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.104+0000 7fa6a564c640 1 -- 172.21.15.139:0/701993278 >> 172.21.15.139:0/701993278 conn(0x7fa6a00b5480 msgr2=0x7fa6a00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.104+0000 7fa6a564c640 1 -- 172.21.15.139:0/701993278 shutdown_connections 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.104+0000 7fa6a564c640 1 -- 172.21.15.139:0/701993278 wait complete. 2024-04-25T09:58:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.105+0000 7fa6a564c640 1 Processor -- start 2024-04-25T09:58:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.105+0000 7fa6a564c640 1 -- start start 2024-04-25T09:58:52.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa6a564c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa688004d40 0x7fa6a00cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa6a564c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6a015f550 0x7fa6a015f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa6a564c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6a014e800 0x7fa6a014ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa6a564c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a015dfd0 con 0x7fa688004d40 2024-04-25T09:58:52.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa6a564c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00d3380 con 0x7fa6a015f550 2024-04-25T09:58:52.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa6a564c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a015be60 con 0x7fa6a014e800 2024-04-25T09:58:52.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa69effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa688004d40 0x7fa6a00cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa69effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa688004d40 0x7fa6a00cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48192/0 (socket says 172.21.15.139:48192) 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa69effd640 1 -- 172.21.15.139:0/2532029985 learned_addr learned my addr 172.21.15.139:0/2532029985 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa69effd640 1 -- 172.21.15.139:0/2532029985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6a014e800 msgr2=0x7fa6a014ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa69effd640 1 --2- 172.21.15.139:0/2532029985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6a014e800 0x7fa6a014ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa69effd640 1 -- 172.21.15.139:0/2532029985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6a015f550 msgr2=0x7fa6a015f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa69effd640 1 --2- 172.21.15.139:0/2532029985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6a015f550 0x7fa6a015f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa69effd640 1 -- 172.21.15.139:0/2532029985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a015c140 con 0x7fa688004d40 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.106+0000 7fa69effd640 1 --2- 172.21.15.139:0/2532029985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa688004d40 0x7fa6a00cbd00 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa69000f7a0 tx=0x7fa69000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.107+0000 7fa6877fe640 1 -- 172.21.15.139:0/2532029985 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000fe50 con 0x7fa688004d40 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.107+0000 7fa6877fe640 1 -- 172.21.15.139:0/2532029985 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa690018070 con 0x7fa688004d40 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.107+0000 7fa6a564c640 1 -- 172.21.15.139:0/2532029985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a015c420 con 0x7fa688004d40 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.107+0000 7fa6877fe640 1 -- 172.21.15.139:0/2532029985 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000d6b0 con 0x7fa688004d40 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.107+0000 7fa6a564c640 1 -- 172.21.15.139:0/2532029985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a015ba80 con 0x7fa688004d40 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.108+0000 7fa6a564c640 1 -- 172.21.15.139:0/2532029985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a015c940 con 0x7fa688004d40 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.108+0000 7fa6877fe640 1 -- 172.21.15.139:0/2532029985 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa69000d850 con 0x7fa688004d40 2024-04-25T09:58:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.108+0000 7fa6877fe640 1 --2- 172.21.15.139:0/2532029985 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa68c06b500 0x7fa68c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.108+0000 7fa6877fe640 1 -- 172.21.15.139:0/2532029985 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa69008a090 con 0x7fa688004d40 2024-04-25T09:58:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.109+0000 7fa69e7fc640 1 --2- 172.21.15.139:0/2532029985 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa68c06b500 0x7fa68c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.109+0000 7fa69e7fc640 1 --2- 172.21.15.139:0/2532029985 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa68c06b500 0x7fa68c06d9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa694000f60 tx=0x7fa694000ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.113+0000 7fa6877fe640 1 -- 172.21.15.139:0/2532029985 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6a015c940 con 0x7fa688004d40 2024-04-25T09:58:52.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.283+0000 7fa6a564c640 1 -- 172.21.15.139:0/2532029985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa6a0074520 con 0x7fa688004d40 2024-04-25T09:58:52.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.286+0000 7fa6877fe640 1 -- 172.21.15.139:0/2532029985 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa690058f90 con 0x7fa688004d40 2024-04-25T09:58:52.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.288+0000 7fa6a564c640 1 -- 172.21.15.139:0/2532029985 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa68c06b500 msgr2=0x7fa68c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.288+0000 7fa6a564c640 1 --2- 172.21.15.139:0/2532029985 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa68c06b500 0x7fa68c06d9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa694000f60 tx=0x7fa694000ed0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.288+0000 7fa6a564c640 1 -- 172.21.15.139:0/2532029985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa688004d40 msgr2=0x7fa6a00cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.288+0000 7fa6a564c640 1 --2- 172.21.15.139:0/2532029985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa688004d40 0x7fa6a00cbd00 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa69000f7a0 tx=0x7fa69000fc60 comp rx=0 tx=0).stop 2024-04-25T09:58:52.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.288+0000 7fa6a564c640 1 -- 172.21.15.139:0/2532029985 shutdown_connections 2024-04-25T09:58:52.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.288+0000 7fa6a564c640 1 --2- 172.21.15.139:0/2532029985 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa68c06b500 0x7fa68c06d9c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.288+0000 7fa6a564c640 1 --2- 172.21.15.139:0/2532029985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6a014e800 0x7fa6a014ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.288+0000 7fa6a564c640 1 --2- 172.21.15.139:0/2532029985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6a015f550 0x7fa6a015f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.288+0000 7fa6a564c640 1 --2- 172.21.15.139:0/2532029985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa688004d40 0x7fa6a00cbd00 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.288+0000 7fa6a564c640 1 -- 172.21.15.139:0/2532029985 >> 172.21.15.139:0/2532029985 conn(0x7fa6a00b5480 msgr2=0x7fa6a0158a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.288+0000 7fa6a564c640 1 -- 172.21.15.139:0/2532029985 shutdown_connections 2024-04-25T09:58:52.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.288+0000 7fa6a564c640 1 -- 172.21.15.139:0/2532029985 wait complete. 2024-04-25T09:58:52.302 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:58:52.303 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:58:52.341 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:58:52.398 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:58:52.399 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:58:52.399 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:52.413 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:58:52.413 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:58:52.413 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:52.431 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:58:52.431 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:58:52.431 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:52.468 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:58:52.468 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T09:58:52.468 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:52.488 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:58:52.488 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:58:52.489 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:52.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.602+0000 7fc8d6fb6640 1 Processor -- start 2024-04-25T09:58:52.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.602+0000 7fc8d6fb6640 1 -- start start 2024-04-25T09:58:52.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8d6fb6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc8d015ad60 0x7fc8d006eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8d6fb6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc8d015b850 0x7fc8d00aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8d6fb6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d006da20 con 0x7fc8d00aeb20 2024-04-25T09:58:52.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8d6fb6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d0155230 con 0x7fc8d015b850 2024-04-25T09:58:52.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8d6fb6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d006dd00 con 0x7fc8d015ad60 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8d5553640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc8d015b850 0x7fc8d00aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8cffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc8d00aeb20 0x7fc8d015b140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55120/0 (socket says 172.21.15.139:55120) 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8cffff640 1 -- 172.21.15.139:0/3341709057 learned_addr learned my addr 172.21.15.139:0/3341709057 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8d4d52640 1 --2- 172.21.15.139:0/3341709057 >> v2:172.21.15.150:3300/0 conn(0x7fc8d015ad60 0x7fc8d006eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8d5553640 1 -- 172.21.15.139:0/3341709057 >> v2:172.21.15.150:3300/0 conn(0x7fc8d015ad60 msgr2=0x7fc8d006eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8d5553640 1 --2- 172.21.15.139:0/3341709057 >> v2:172.21.15.150:3300/0 conn(0x7fc8d015ad60 0x7fc8d006eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8d5553640 1 -- 172.21.15.139:0/3341709057 >> v1:172.21.15.139:6789/0 conn(0x7fc8d00aeb20 legacy=0x7fc8d015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8d5553640 1 -- 172.21.15.139:0/3341709057 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d006dfe0 con 0x7fc8d015b850 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.603+0000 7fc8cf7fe640 1 -- 172.21.15.139:0/3341709057 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235286113 0 0) 0x7fc8d006da20 con 0x7fc8d00aeb20 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.604+0000 7fc8d5553640 1 --2- 172.21.15.139:0/3341709057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d015b850 0x7fc8d00aefb0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00b54a0 tx=0x7fc8c002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e628a0fb523800a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.604+0000 7fc8cf7fe640 1 -- 172.21.15.139:0/3341709057 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0037070 con 0x7fc8d015b850 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.604+0000 7fc8cf7fe640 1 -- 172.21.15.139:0/3341709057 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c0032440 con 0x7fc8d015b850 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.604+0000 7fc8cf7fe640 1 -- 172.21.15.139:0/3341709057 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0032810 con 0x7fc8d015b850 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.604+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/3341709057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d015b850 msgr2=0x7fc8d00aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.604+0000 7fc8d6fb6640 1 --2- 172.21.15.139:0/3341709057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d015b850 0x7fc8d00aefb0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00b54a0 tx=0x7fc8c002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.604+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/3341709057 shutdown_connections 2024-04-25T09:58:52.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.604+0000 7fc8d6fb6640 1 --2- 172.21.15.139:0/3341709057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d015b850 0x7fc8d00aefb0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.604+0000 7fc8d6fb6640 1 --2- 172.21.15.139:0/3341709057 >> v2:172.21.15.150:3300/0 conn(0x7fc8d015ad60 0x7fc8d006eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.604+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/3341709057 >> 172.21.15.139:0/3341709057 conn(0x7fc8d00b23d0 msgr2=0x7fc8d00cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.604+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/3341709057 shutdown_connections 2024-04-25T09:58:52.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.605+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/3341709057 wait complete. 2024-04-25T09:58:52.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.606+0000 7fc8d6fb6640 1 Processor -- start 2024-04-25T09:58:52.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.606+0000 7fc8d6fb6640 1 -- start start 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.606+0000 7fc8d6fb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d00aeb20 0x7fc8d0149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.606+0000 7fc8d6fb6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8d015ad60 0x7fc8d014bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.606+0000 7fc8d6fb6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8d015b850 0x7fc8d00b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.606+0000 7fc8d6fb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d006e880 con 0x7fc8d00aeb20 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.606+0000 7fc8d6fb6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d006e5a0 con 0x7fc8d015b850 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d6fb6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d006f2e0 con 0x7fc8d015ad60 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d4d52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d00aeb20 0x7fc8d0149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d4d52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d00aeb20 0x7fc8d0149860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48214/0 (socket says 172.21.15.139:48214) 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d4d52640 1 -- 172.21.15.139:0/2870137034 learned_addr learned my addr 172.21.15.139:0/2870137034 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d5553640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8d015b850 0x7fc8d00b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8cffff640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8d015ad60 0x7fc8d014bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d4d52640 1 -- 172.21.15.139:0/2870137034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8d015ad60 msgr2=0x7fc8d014bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d4d52640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8d015ad60 0x7fc8d014bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d4d52640 1 -- 172.21.15.139:0/2870137034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8d015b850 msgr2=0x7fc8d00b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d4d52640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8d015b850 0x7fc8d00b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d4d52640 1 -- 172.21.15.139:0/2870137034 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d014e6f0 con 0x7fc8d00aeb20 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d5553640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8d015b850 0x7fc8d00b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d4d52640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d00aeb20 0x7fc8d0149860 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800ce10 tx=0x7fc8c800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8cd7fa640 1 -- 172.21.15.139:0/2870137034 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8003b90 con 0x7fc8d00aeb20 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8cd7fa640 1 -- 172.21.15.139:0/2870137034 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c8003d30 con 0x7fc8d00aeb20 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/2870137034 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d014db70 con 0x7fc8d00aeb20 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8cd7fa640 1 -- 172.21.15.139:0/2870137034 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8004240 con 0x7fc8d00aeb20 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.607+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/2870137034 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d014e130 con 0x7fc8d00aeb20 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.608+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/2870137034 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d0074520 con 0x7fc8d00aeb20 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.609+0000 7fc8cd7fa640 1 -- 172.21.15.139:0/2870137034 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8c80043e0 con 0x7fc8d00aeb20 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.609+0000 7fc8cd7fa640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc8b006b4d0 0x7fc8b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.609+0000 7fc8cd7fa640 1 -- 172.21.15.139:0/2870137034 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fc8c808ab70 con 0x7fc8d00aeb20 2024-04-25T09:58:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.609+0000 7fc8cffff640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc8b006b4d0 0x7fc8b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.609+0000 7fc8cffff640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc8b006b4d0 0x7fc8b006d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc8d014d170 tx=0x7fc8b8007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.613+0000 7fc8cd7fa640 1 -- 172.21.15.139:0/2870137034 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8c801d340 con 0x7fc8d00aeb20 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.817+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/2870137034 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8d014a230 con 0x7fc8d00aeb20 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.818+0000 7fc8cd7fa640 1 -- 172.21.15.139:0/2870137034 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fc8c8059ba0 con 0x7fc8d00aeb20 2024-04-25T09:58:52.822 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:52.822 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5777,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/982251530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":982251530},{"type":"v1","addr":"172.21.15.150:6834","nonce":982251530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6028,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2532107941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2532107941},{"type":"v1","addr":"172.21.15.150:6835","nonce":2532107941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/233624690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":233624690},{"type":"v1","addr":"172.21.15.139:6835","nonce":233624690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":7470,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3052246867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":3052246867},{"type":"v1","addr":"172.21.15.139:6837","nonce":3052246867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[]} 2024-04-25T09:58:52.822 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 46 2024-04-25T09:58:52.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.820+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/2870137034 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc8b006b4d0 msgr2=0x7fc8b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.820+0000 7fc8d6fb6640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc8b006b4d0 0x7fc8b006d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc8d014d170 tx=0x7fc8b8007b50 comp rx=0 tx=0).stop 2024-04-25T09:58:52.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.820+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/2870137034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d00aeb20 msgr2=0x7fc8d0149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.820+0000 7fc8d6fb6640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d00aeb20 0x7fc8d0149860 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800ce10 tx=0x7fc8c800a660 comp rx=0 tx=0).stop 2024-04-25T09:58:52.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.820+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/2870137034 shutdown_connections 2024-04-25T09:58:52.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.820+0000 7fc8d6fb6640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc8b006b4d0 0x7fc8b006d990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.820+0000 7fc8d6fb6640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8d015b850 0x7fc8d00b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.820+0000 7fc8d6fb6640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8d015ad60 0x7fc8d014bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.820+0000 7fc8d6fb6640 1 --2- 172.21.15.139:0/2870137034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d00aeb20 0x7fc8d0149860 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.820+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/2870137034 >> 172.21.15.139:0/2870137034 conn(0x7fc8d00b23d0 msgr2=0x7fc8d015be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.820+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/2870137034 shutdown_connections 2024-04-25T09:58:52.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.820+0000 7fc8d6fb6640 1 -- 172.21.15.139:0/2870137034 wait complete. 2024-04-25T09:58:52.834 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:58:52.835 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:58:52.835 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:58:52.835 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:58:52.835 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:58:52.835 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:58:52.835 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:58:52.835 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:58:52.836 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:58:52.836 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:58:52.836 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:58:52.836 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:58:52.836 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:58:52.836 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:58:52.836 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:58:52.836 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:58:52.846 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:58:52.846 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:58:52.847 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:58:52.847 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:58:52.848 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:58:52.848 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:58:52.849 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:58:52.849 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:58:52.889 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:58:52.890 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:52.891 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:58:52.892 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:52.892 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:58:52.893 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:52.894 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:58:52.894 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:52.894 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:58:52.962 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T09:58:52.962 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-04-25T09:58:52.973 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-04-25T09:58:52.973 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-04-25T09:58:52.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.995+0000 7fad7a70a640 1 Processor -- start 2024-04-25T09:58:52.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.995+0000 7fad7a70a640 1 -- start start 2024-04-25T09:58:53.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.996+0000 7fad7a70a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fad74072a40 0x7fad7415d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.996+0000 7fad7a70a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad740adf20 con 0x7fad740b3020 2024-04-25T09:58:53.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.996+0000 7fad7a70a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad740cdef0 con 0x7fad740669d0 2024-04-25T09:58:53.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.996+0000 7fad7a70a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad740cdca0 con 0x7fad74072a40 2024-04-25T09:58:53.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.996+0000 7fad73fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fad740b3020 0x7fad740ad810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55148/0 (socket says 172.21.15.139:55148) 2024-04-25T09:58:53.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.996+0000 7fad73fff640 1 -- 172.21.15.139:0/2288307009 learned_addr learned my addr 172.21.15.139:0/2288307009 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:53.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.996+0000 7fad737fe640 1 --2- 172.21.15.139:0/2288307009 >> v2:172.21.15.150:3300/0 conn(0x7fad74072a40 0x7fad7415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.997+0000 7fad737fe640 1 -- 172.21.15.139:0/2288307009 >> v1:172.21.15.150:6789/0 conn(0x7fad740669d0 legacy=0x7fad740cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.997+0000 7fad737fe640 1 -- 172.21.15.139:0/2288307009 >> v1:172.21.15.139:6789/0 conn(0x7fad740b3020 legacy=0x7fad740ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.997+0000 7fad737fe640 1 -- 172.21.15.139:0/2288307009 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad740ce1d0 con 0x7fad74072a40 2024-04-25T09:58:53.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.997+0000 7fad737fe640 1 --2- 172.21.15.139:0/2288307009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad74072a40 0x7fad7415d790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fad740cde20 tx=0x7fad6802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1db62e825076995a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.998+0000 7fad72ffd640 1 -- 172.21.15.139:0/2288307009 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68037070 con 0x7fad74072a40 2024-04-25T09:58:53.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.998+0000 7fad72ffd640 1 -- 172.21.15.139:0/2288307009 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad68032440 con 0x7fad74072a40 2024-04-25T09:58:53.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.998+0000 7fad72ffd640 1 -- 172.21.15.139:0/2288307009 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68032810 con 0x7fad74072a40 2024-04-25T09:58:53.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.998+0000 7fad7a70a640 1 -- 172.21.15.139:0/2288307009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad74072a40 msgr2=0x7fad7415d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.998+0000 7fad7a70a640 1 --2- 172.21.15.139:0/2288307009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad74072a40 0x7fad7415d790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fad740cde20 tx=0x7fad6802a210 comp rx=0 tx=0).stop 2024-04-25T09:58:53.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.998+0000 7fad7a70a640 1 -- 172.21.15.139:0/2288307009 shutdown_connections 2024-04-25T09:58:53.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.998+0000 7fad7a70a640 1 --2- 172.21.15.139:0/2288307009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad74072a40 0x7fad7415d790 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.998+0000 7fad7a70a640 1 -- 172.21.15.139:0/2288307009 >> 172.21.15.139:0/2288307009 conn(0x7fad740cc820 msgr2=0x7fad740ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:53.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.998+0000 7fad7a70a640 1 -- 172.21.15.139:0/2288307009 shutdown_connections 2024-04-25T09:58:53.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:52.998+0000 7fad7a70a640 1 -- 172.21.15.139:0/2288307009 wait complete. 2024-04-25T09:58:53.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.000+0000 7fad7a70a640 1 Processor -- start 2024-04-25T09:58:53.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.000+0000 7fad7a70a640 1 -- start start 2024-04-25T09:58:53.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad7a70a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad740669d0 0x7fad740b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad7a70a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad740b3020 0x7fad740afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad7a70a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7409ca80 0x7fad7409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad7a70a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad740b4570 con 0x7fad740b3020 2024-04-25T09:58:53.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad7a70a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad740ce7d0 con 0x7fad7409ca80 2024-04-25T09:58:53.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad7a70a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad740b46f0 con 0x7fad740669d0 2024-04-25T09:58:53.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad73fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad740669d0 0x7fad740b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad73fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad740669d0 0x7fad740b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:46996/0 (socket says 172.21.15.139:46996) 2024-04-25T09:58:53.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad73fff640 1 -- 172.21.15.139:0/2313038595 learned_addr learned my addr 172.21.15.139:0/2313038595 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:53.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad78ca7640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7409ca80 0x7fad7409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad737fe640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad740b3020 0x7fad740afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad737fe640 1 -- 172.21.15.139:0/2313038595 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad740669d0 msgr2=0x7fad740b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad737fe640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad740669d0 0x7fad740b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad737fe640 1 -- 172.21.15.139:0/2313038595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7409ca80 msgr2=0x7fad7409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad737fe640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7409ca80 0x7fad7409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.001+0000 7fad737fe640 1 -- 172.21.15.139:0/2313038595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad74143810 con 0x7fad740b3020 2024-04-25T09:58:53.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.002+0000 7fad737fe640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad740b3020 0x7fad740afb40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fad68031000 tx=0x7fad680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.003+0000 7fad70ff9640 1 -- 172.21.15.139:0/2313038595 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68032e60 con 0x7fad740b3020 2024-04-25T09:58:53.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.004+0000 7fad7a70a640 1 -- 172.21.15.139:0/2313038595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad74148a00 con 0x7fad740b3020 2024-04-25T09:58:53.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.004+0000 7fad70ff9640 1 -- 172.21.15.139:0/2313038595 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad68030770 con 0x7fad740b3020 2024-04-25T09:58:53.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.004+0000 7fad70ff9640 1 -- 172.21.15.139:0/2313038595 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68037040 con 0x7fad740b3020 2024-04-25T09:58:53.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.004+0000 7fad7a70a640 1 -- 172.21.15.139:0/2313038595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad74148fc0 con 0x7fad740b3020 2024-04-25T09:58:53.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.005+0000 7fad70ff9640 1 -- 172.21.15.139:0/2313038595 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad68044020 con 0x7fad740b3020 2024-04-25T09:58:53.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.006+0000 7fad7a70a640 1 -- 172.21.15.139:0/2313038595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad74148790 con 0x7fad740b3020 2024-04-25T09:58:53.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.006+0000 7fad70ff9640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad6006b4d0 0x7fad6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.006+0000 7fad70ff9640 1 -- 172.21.15.139:0/2313038595 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fad680abc40 con 0x7fad740b3020 2024-04-25T09:58:53.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.006+0000 7fad73fff640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad6006b4d0 0x7fad6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.006+0000 7fad73fff640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad6006b4d0 0x7fad6006d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fad64005880 tx=0x7fad64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.010+0000 7fad70ff9640 1 -- 172.21.15.139:0/2313038595 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad68078420 con 0x7fad740b3020 2024-04-25T09:58:53.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:53.195+0000 7fad7a70a640 1 -- 172.21.15.139:0/2313038595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fad74154550 con 0x7fad740b3020 2024-04-25T09:58:54.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.006+0000 7fad70ff9640 1 -- 172.21.15.139:0/2313038595 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v85) v1 ==== 97+0+0 (secure 0 0 0) 0x7fad6807c4c0 con 0x7fad740b3020 2024-04-25T09:58:54.010 INFO:teuthology.orchestra.run.smithi139.stderr: removed all blocklist entries 2024-04-25T09:58:54.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.009+0000 7fad7a70a640 1 -- 172.21.15.139:0/2313038595 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad6006b4d0 msgr2=0x7fad6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.009+0000 7fad7a70a640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad6006b4d0 0x7fad6006d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fad64005880 tx=0x7fad64009040 comp rx=0 tx=0).stop 2024-04-25T09:58:54.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.009+0000 7fad7a70a640 1 -- 172.21.15.139:0/2313038595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad740b3020 msgr2=0x7fad740afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.009+0000 7fad7a70a640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad740b3020 0x7fad740afb40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fad68031000 tx=0x7fad680046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.009+0000 7fad7a70a640 1 -- 172.21.15.139:0/2313038595 shutdown_connections 2024-04-25T09:58:54.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.009+0000 7fad7a70a640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad740b3020 0x7fad740afb40 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.009+0000 7fad7a70a640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad6006b4d0 0x7fad6006d990 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.009+0000 7fad7a70a640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7409ca80 0x7fad7409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.009+0000 7fad7a70a640 1 --2- 172.21.15.139:0/2313038595 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad740669d0 0x7fad740b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.009+0000 7fad7a70a640 1 -- 172.21.15.139:0/2313038595 >> 172.21.15.139:0/2313038595 conn(0x7fad740cc820 msgr2=0x7fad7406e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.010+0000 7fad7a70a640 1 -- 172.21.15.139:0/2313038595 shutdown_connections 2024-04-25T09:58:54.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.010+0000 7fad7a70a640 1 -- 172.21.15.139:0/2313038595 wait complete. 2024-04-25T09:58:54.028 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:58:54.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.126+0000 7f980fd3b640 1 Processor -- start 2024-04-25T09:58:54.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.127+0000 7f980fd3b640 1 -- start start 2024-04-25T09:58:54.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.127+0000 7f980fd3b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f98080d1080 0x7f98080b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.127+0000 7f980fd3b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9808161850 0x7f98080ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.127+0000 7f980fd3b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98080cf250 con 0x7f980815dc60 2024-04-25T09:58:54.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.127+0000 7f980fd3b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98080ce780 con 0x7f98080d1080 2024-04-25T09:58:54.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.127+0000 7f980fd3b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98080d3730 con 0x7f9808161850 2024-04-25T09:58:54.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.128+0000 7f980e2d8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9808161850 0x7f98080ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.128+0000 7f980d2d6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f98080d1080 0x7f98080b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.128+0000 7f980e2d8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9808161850 0x7f98080ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:47000/0 (socket says 172.21.15.139:47000) 2024-04-25T09:58:54.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.128+0000 7f980e2d8640 1 -- 172.21.15.139:0/3952601961 learned_addr learned my addr 172.21.15.139:0/3952601961 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:54.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.128+0000 7f980e2d8640 1 -- 172.21.15.139:0/3952601961 >> v2:172.21.15.150:3300/0 conn(0x7f98080d1080 msgr2=0x7f98080b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.128+0000 7f980e2d8640 1 --2- 172.21.15.139:0/3952601961 >> v2:172.21.15.150:3300/0 conn(0x7f98080d1080 0x7f98080b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.128+0000 7f980e2d8640 1 -- 172.21.15.139:0/3952601961 >> v1:172.21.15.150:6789/0 conn(0x7f980815dc60 legacy=0x7f98080d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.128+0000 7f980e2d8640 1 -- 172.21.15.139:0/3952601961 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98080ce9b0 con 0x7f9808161850 2024-04-25T09:58:54.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.128+0000 7f980d2d6640 1 --2- 172.21.15.139:0/3952601961 >> v2:172.21.15.150:3300/0 conn(0x7f98080d1080 0x7f98080b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:54.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.129+0000 7f980e2d8640 1 --2- 172.21.15.139:0/3952601961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9808161850 0x7f98080ce240 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9800009e50 tx=0x7f980002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=282fba360a162221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.129+0000 7f980cad5640 1 -- 172.21.15.139:0/3952601961 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800036070 con 0x7f9808161850 2024-04-25T09:58:54.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.129+0000 7f980cad5640 1 -- 172.21.15.139:0/3952601961 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9800031600 con 0x7f9808161850 2024-04-25T09:58:54.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.129+0000 7f980cad5640 1 -- 172.21.15.139:0/3952601961 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98000319d0 con 0x7f9808161850 2024-04-25T09:58:54.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.130+0000 7f980fd3b640 1 -- 172.21.15.139:0/3952601961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9808161850 msgr2=0x7f98080ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.130+0000 7f980fd3b640 1 --2- 172.21.15.139:0/3952601961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9808161850 0x7f98080ce240 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9800009e50 tx=0x7f980002a710 comp rx=0 tx=0).stop 2024-04-25T09:58:54.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.130+0000 7f980fd3b640 1 -- 172.21.15.139:0/3952601961 shutdown_connections 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.130+0000 7f980fd3b640 1 --2- 172.21.15.139:0/3952601961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9808161850 0x7f98080ce240 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.130+0000 7f980fd3b640 1 --2- 172.21.15.139:0/3952601961 >> v2:172.21.15.150:3300/0 conn(0x7f98080d1080 0x7f98080b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.130+0000 7f980fd3b640 1 -- 172.21.15.139:0/3952601961 >> 172.21.15.139:0/3952601961 conn(0x7f98080b5910 msgr2=0x7f98080b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.130+0000 7f980fd3b640 1 -- 172.21.15.139:0/3952601961 shutdown_connections 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.130+0000 7f980fd3b640 1 -- 172.21.15.139:0/3952601961 wait complete. 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.131+0000 7f980fd3b640 1 Processor -- start 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.131+0000 7f980fd3b640 1 -- start start 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980fd3b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98080d1080 0x7f980814ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980fd3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f980815dc60 0x7f980815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980fd3b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9808161850 0x7f98080c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980fd3b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98080d1fc0 con 0x7f980815dc60 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980fd3b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98080cef70 con 0x7f9808161850 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980fd3b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980815e040 con 0x7f98080d1080 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980d2d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f980815dc60 0x7f980815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980d2d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f980815dc60 0x7f980815ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48242/0 (socket says 172.21.15.139:48242) 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980d2d6640 1 -- 172.21.15.139:0/1967192366 learned_addr learned my addr 172.21.15.139:0/1967192366 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980e2d8640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9808161850 0x7f98080c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980dad7640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98080d1080 0x7f980814ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980d2d6640 1 -- 172.21.15.139:0/1967192366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98080d1080 msgr2=0x7f980814ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980d2d6640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98080d1080 0x7f980814ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980d2d6640 1 -- 172.21.15.139:0/1967192366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9808161850 msgr2=0x7f98080c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980d2d6640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9808161850 0x7f98080c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980d2d6640 1 -- 172.21.15.139:0/1967192366 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980814d4f0 con 0x7f980815dc60 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980e2d8640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9808161850 0x7f98080c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980dad7640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98080d1080 0x7f980814ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980d2d6640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f980815dc60 0x7f980815ecc0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00e960 tx=0x7f97fc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f97fa7fc640 1 -- 172.21.15.139:0/1967192366 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc017070 con 0x7f980815dc60 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f97fa7fc640 1 -- 172.21.15.139:0/1967192366 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97fc00ca00 con 0x7f980815dc60 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980fd3b640 1 -- 172.21.15.139:0/1967192366 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980814d790 con 0x7f980815dc60 2024-04-25T09:58:54.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f97fa7fc640 1 -- 172.21.15.139:0/1967192366 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc00cdd0 con 0x7f980815dc60 2024-04-25T09:58:54.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.132+0000 7f980fd3b640 1 -- 172.21.15.139:0/1967192366 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980814dd50 con 0x7f980815dc60 2024-04-25T09:58:54.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.133+0000 7f980fd3b640 1 -- 172.21.15.139:0/1967192366 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9808074520 con 0x7f980815dc60 2024-04-25T09:58:54.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.134+0000 7f97fa7fc640 1 -- 172.21.15.139:0/1967192366 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97fc01a070 con 0x7f980815dc60 2024-04-25T09:58:54.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.134+0000 7f97fa7fc640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f97ec06b500 0x7f97ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.134+0000 7f980dad7640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f97ec06b500 0x7f97ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.134+0000 7f97fa7fc640 1 -- 172.21.15.139:0/1967192366 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f97fc089ee0 con 0x7f980815dc60 2024-04-25T09:58:54.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.135+0000 7f980dad7640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f97ec06b500 0x7f97ec06d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f980815f900 tx=0x7f9804009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.139+0000 7f97fa7fc640 1 -- 172.21.15.139:0/1967192366 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97fc058f30 con 0x7f980815dc60 2024-04-25T09:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.372+0000 7f980fd3b640 1 -- 172.21.15.139:0/1967192366 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f98080d16f0 con 0x7f980815dc60 2024-04-25T09:58:54.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.373+0000 7f97fa7fc640 1 -- 172.21.15.139:0/1967192366 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f97fc058830 con 0x7f980815dc60 2024-04-25T09:58:54.377 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:54.378 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:58:54.378 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_dump": [ 2024-04-25T09:58:54.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.378 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.a", 2024-04-25T09:58:54.378 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypm0xdaHhAA6nupxCTQhucepW2iEWNUoQ==", 2024-04-25T09:58:54.378 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.379 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:58:54.379 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:58:54.379 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:58:54.379 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.379 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.380 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.b", 2024-04-25T09:58:54.380 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmB9GDKRAA/lmU4OiKO5dFgayYx09GQg==", 2024-04-25T09:58:54.380 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.380 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:58:54.380 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:58:54.381 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:58:54.381 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.381 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.381 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.c", 2024-04-25T09:58:54.382 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmvYgaJRAA/YIE6ralp2WuTK993D7H0Q==", 2024-04-25T09:58:54.382 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.382 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:58:54.382 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:58:54.382 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:58:54.383 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.383 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.383 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.d", 2024-04-25T09:58:54.383 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmg49WMBAAlPY9ZSeOoV296xSdl/m1gg==", 2024-04-25T09:58:54.384 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.384 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:58:54.384 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:58:54.384 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:58:54.384 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.384 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.385 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.0", 2024-04-25T09:58:54.385 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDVJypmG7TpDhAAo9CKuZBw8BsQgB2F+BR5gw==", 2024-04-25T09:58:54.385 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.385 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:54.386 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:58:54.386 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.386 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.387 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.1", 2024-04-25T09:58:54.387 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDYJypmgR62DBAAotfnvS47WDunP9Ic03gA6Q==", 2024-04-25T09:58:54.387 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.387 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:54.387 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:58:54.387 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.388 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.388 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.2", 2024-04-25T09:58:54.388 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDbJypmzyqGChAA6wrfIdqpiLm3RO36oorihQ==", 2024-04-25T09:58:54.388 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.388 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:54.389 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:58:54.389 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.389 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.389 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.3", 2024-04-25T09:58:54.389 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDeJypmt72cCBAAgcLrFC3MUU7wlFySAfLWig==", 2024-04-25T09:58:54.389 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.389 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:54.390 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:58:54.390 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.390 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.390 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.4", 2024-04-25T09:58:54.390 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDkJypmxJLpDBAAvROZ9AvYdWMqOALEKIAFCw==", 2024-04-25T09:58:54.390 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.391 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:54.391 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:58:54.391 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.391 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.391 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.5", 2024-04-25T09:58:54.391 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDnJypmTPXEChAA20BghY39PGw6zFyu9yWbBQ==", 2024-04-25T09:58:54.391 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.392 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:54.392 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:58:54.392 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.392 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.392 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.6", 2024-04-25T09:58:54.392 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDqJypmaBoYCRAAy9eHgUI/WTkE2XdT1RKLsw==", 2024-04-25T09:58:54.393 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.393 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:54.393 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:58:54.393 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.393 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.393 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.7", 2024-04-25T09:58:54.393 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDtJypmViJlCBAANJSDWVHVbWR03Y1SMDH++Q==", 2024-04-25T09:58:54.393 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.393 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.0", 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg==", 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow r", 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.1", 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypmOncYARAA6gYjZbRDkhqFX0mpSDbH5Q==", 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.2", 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmQs2wORAArzyV70nGDX4486LlFjM70Q==", 2024-04-25T09:58:54.395 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.3", 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypm90x9BhAA41rZPs0O5mwmuHy1bkPCvw==", 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T09:58:54.396 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.admin", 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmOWQcAxAA/ZVXEHYJQZf9UtgZ5g25Ag==", 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.x", 2024-04-25T09:58:54.397 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmt+RVGxAAi1djVoqnoYCDMzJ9A0dmNA==", 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.y", 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmuJ4bFxAAw6a53AgThU/ut+Mk5j0gqQ==", 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T09:58:54.398 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.399 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:54.399 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-04-25T09:58:54.399 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:58:54.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.375+0000 7f980fd3b640 1 -- 172.21.15.139:0/1967192366 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f97ec06b500 msgr2=0x7f97ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.375+0000 7f980fd3b640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f97ec06b500 0x7f97ec06d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f980815f900 tx=0x7f9804009040 comp rx=0 tx=0).stop 2024-04-25T09:58:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.375+0000 7f980fd3b640 1 -- 172.21.15.139:0/1967192366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f980815dc60 msgr2=0x7f980815ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.375+0000 7f980fd3b640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f980815dc60 0x7f980815ecc0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00e960 tx=0x7f97fc00ee20 comp rx=0 tx=0).stop 2024-04-25T09:58:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.375+0000 7f980fd3b640 1 -- 172.21.15.139:0/1967192366 shutdown_connections 2024-04-25T09:58:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.375+0000 7f980fd3b640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f980815dc60 0x7f980815ecc0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.375+0000 7f980fd3b640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f97ec06b500 0x7f97ec06d9c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.375+0000 7f980fd3b640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9808161850 0x7f98080c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.375+0000 7f980fd3b640 1 --2- 172.21.15.139:0/1967192366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98080d1080 0x7f980814ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.375+0000 7f980fd3b640 1 -- 172.21.15.139:0/1967192366 >> 172.21.15.139:0/1967192366 conn(0x7f98080b5910 msgr2=0x7f980806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.375+0000 7f980fd3b640 1 -- 172.21.15.139:0/1967192366 shutdown_connections 2024-04-25T09:58:54.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.375+0000 7f980fd3b640 1 -- 172.21.15.139:0/1967192366 wait complete. 2024-04-25T09:58:54.402 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:54.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.495+0000 7fd0f3989640 1 Processor -- start 2024-04-25T09:58:54.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.495+0000 7fd0f3989640 1 -- start start 2024-04-25T09:58:54.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0f3989640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec0cf900 0x7fd0ec0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0f3989640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ec0cfe70 con 0x7fd0ec099a50 2024-04-25T09:58:54.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0f3989640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ec0d0150 con 0x7fd0ec15efb0 2024-04-25T09:58:54.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0f3989640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec0d0a80 con 0x7fd0ec0cf900 2024-04-25T09:58:54.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0f1f26640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd0ec099a50 0x7fd0ec0cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55162/0 (socket says 172.21.15.139:55162) 2024-04-25T09:58:54.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0f1f26640 1 -- 172.21.15.139:0/3589158330 learned_addr learned my addr 172.21.15.139:0/3589158330 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:54.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0f0f24640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec0cf900 0x7fd0ec0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521186673 0 0) 0x7fd0ec0cfe70 con 0x7fd0ec099a50 2024-04-25T09:58:54.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d8003520 con 0x7fd0ec099a50 2024-04-25T09:58:54.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1701218749 0 0) 0x7fd0d8003520 con 0x7fd0ec099a50 2024-04-25T09:58:54.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec0cf900 msgr2=0x7fd0ec0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec0cf900 0x7fd0ec0d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 >> v1:172.21.15.150:6789/0 conn(0x7fd0ec15efb0 legacy=0x7fd0ec06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec0d0430 con 0x7fd0ec099a50 2024-04-25T09:58:54.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fd0e40019f0 con 0x7fd0ec099a50 2024-04-25T09:58:54.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0f0f24640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec0cf900 0x7fd0ec0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:54.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 >> v1:172.21.15.139:6789/0 conn(0x7fd0ec099a50 legacy=0x7fd0ec0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d80039e0 0x7fd0d8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec0cf900 0x7fd0d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d8004d40 0x7fd0d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec0d0a80 con 0x7fd0d80039e0 2024-04-25T09:58:54.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d800ba10 con 0x7fd0d8004d40 2024-04-25T09:58:54.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.496+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d800bc10 con 0x7fd0ec0cf900 2024-04-25T09:58:54.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.497+0000 7fd0f1725640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d80039e0 0x7fd0d8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.497+0000 7fd0f0f24640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec0cf900 0x7fd0d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.497+0000 7fd0f1f26640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d8004d40 0x7fd0d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.497+0000 7fd0f1725640 1 -- 172.21.15.139:0/3589158330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec0cf900 msgr2=0x7fd0d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.497+0000 7fd0f1725640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec0cf900 0x7fd0d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.497+0000 7fd0f1725640 1 -- 172.21.15.139:0/3589158330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d8004d40 msgr2=0x7fd0d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.497+0000 7fd0f1725640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d8004d40 0x7fd0d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.497+0000 7fd0f1725640 1 -- 172.21.15.139:0/3589158330 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d8003520 con 0x7fd0d80039e0 2024-04-25T09:58:54.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.497+0000 7fd0f1725640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d80039e0 0x7fd0d8003dc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8001860 tx=0x7fd0d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51193bfe5affecda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.498+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8037070 con 0x7fd0d80039e0 2024-04-25T09:58:54.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.498+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e802ac30 con 0x7fd0d80039e0 2024-04-25T09:58:54.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.498+0000 7fd0e3fff640 1 -- 172.21.15.139:0/3589158330 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8001430 con 0x7fd0d80039e0 2024-04-25T09:58:54.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.498+0000 7fd0f3989640 1 -- 172.21.15.139:0/3589158330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d80039e0 msgr2=0x7fd0d8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.498+0000 7fd0f3989640 1 --2- 172.21.15.139:0/3589158330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d80039e0 0x7fd0d8003dc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8001860 tx=0x7fd0d8005830 comp rx=0 tx=0).stop 2024-04-25T09:58:54.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.498+0000 7fd0f1f26640 1 -- 172.21.15.139:0/3589158330 reap_dead start 2024-04-25T09:58:54.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.498+0000 7fd0f3989640 1 -- 172.21.15.139:0/3589158330 shutdown_connections 2024-04-25T09:58:54.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.498+0000 7fd0f3989640 1 -- 172.21.15.139:0/3589158330 >> 172.21.15.139:0/3589158330 conn(0x7fd0ec0d1790 msgr2=0x7fd0ec0d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.498+0000 7fd0f3989640 1 -- 172.21.15.139:0/3589158330 shutdown_connections 2024-04-25T09:58:54.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.498+0000 7fd0f3989640 1 -- 172.21.15.139:0/3589158330 wait complete. 2024-04-25T09:58:54.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.499+0000 7fd0f3989640 1 Processor -- start 2024-04-25T09:58:54.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.499+0000 7fd0f3989640 1 -- start start 2024-04-25T09:58:54.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f3989640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d80039e0 0x7fd0ec1e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f3989640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec15bcd0 0x7fd0ec156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f3989640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0ec0b65c0 0x7fd0ec0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f3989640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec0d0f10 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f3989640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec06cf50 con 0x7fd0d80039e0 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f3989640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec0d1090 con 0x7fd0ec15bcd0 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f1f26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0ec0b65c0 0x7fd0ec0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f1f26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0ec0b65c0 0x7fd0ec0b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48260/0 (socket says 172.21.15.139:48260) 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f1f26640 1 -- 172.21.15.139:0/3945843905 learned_addr learned my addr 172.21.15.139:0/3945843905 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f1725640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d80039e0 0x7fd0ec1e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f0f24640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec15bcd0 0x7fd0ec156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f1f26640 1 -- 172.21.15.139:0/3945843905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec15bcd0 msgr2=0x7fd0ec156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f1f26640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec15bcd0 0x7fd0ec156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f1f26640 1 -- 172.21.15.139:0/3945843905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d80039e0 msgr2=0x7fd0ec1e64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.500+0000 7fd0f1f26640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d80039e0 0x7fd0ec1e64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.501+0000 7fd0f1f26640 1 -- 172.21.15.139:0/3945843905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec15c290 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.501+0000 7fd0f1f26640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0ec0b65c0 0x7fd0ec0b6a40 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0c5290 tx=0x7fd0e400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.501+0000 7fd0e1ffb640 1 -- 172.21.15.139:0/3945843905 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e400ee20 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.501+0000 7fd0e1ffb640 1 -- 172.21.15.139:0/3945843905 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e4014430 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.501+0000 7fd0f3989640 1 -- 172.21.15.139:0/3945843905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ec15c570 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.501+0000 7fd0e1ffb640 1 -- 172.21.15.139:0/3945843905 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4014820 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.501+0000 7fd0f3989640 1 -- 172.21.15.139:0/3945843905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ec15cb30 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.502+0000 7fd0f3989640 1 -- 172.21.15.139:0/3945843905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0ec074520 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.502+0000 7fd0e1ffb640 1 -- 172.21.15.139:0/3945843905 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0e40149c0 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.503+0000 7fd0e1ffb640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd0d006b500 0x7fd0d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.503+0000 7fd0e1ffb640 1 -- 172.21.15.139:0/3945843905 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd0e4089e50 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.503+0000 7fd0f1725640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd0d006b500 0x7fd0d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.503+0000 7fd0f1725640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd0d006b500 0x7fd0d006d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8034000 tx=0x7fd0e8009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.507+0000 7fd0e1ffb640 1 -- 172.21.15.139:0/3945843905 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0e4058e90 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.678+0000 7fd0f3989640 1 -- 172.21.15.139:0/3945843905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0ec09ae50 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.679+0000 7fd0e1ffb640 1 -- 172.21.15.139:0/3945843905 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+13213 (secure 0 0 0) 0x7fd0e4058790 con 0x7fd0ec0b65c0 2024-04-25T09:58:54.687 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:54.687 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":85,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:58:54.003320+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.681+0000 7fd0f3989640 1 -- 172.21.15.139:0/3945843905 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd0d006b500 msgr2=0x7fd0d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.681+0000 7fd0f3989640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd0d006b500 0x7fd0d006d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8034000 tx=0x7fd0e8009030 comp rx=0 tx=0).stop 2024-04-25T09:58:54.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.681+0000 7fd0f3989640 1 -- 172.21.15.139:0/3945843905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0ec0b65c0 msgr2=0x7fd0ec0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.681+0000 7fd0f3989640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0ec0b65c0 0x7fd0ec0b6a40 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0c5290 tx=0x7fd0e400e530 comp rx=0 tx=0).stop 2024-04-25T09:58:54.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.681+0000 7fd0f3989640 1 -- 172.21.15.139:0/3945843905 shutdown_connections 2024-04-25T09:58:54.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.681+0000 7fd0f3989640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0ec0b65c0 0x7fd0ec0b6a40 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.681+0000 7fd0f3989640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd0d006b500 0x7fd0d006d9c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.681+0000 7fd0f3989640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0ec15bcd0 0x7fd0ec156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.681+0000 7fd0f3989640 1 --2- 172.21.15.139:0/3945843905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d80039e0 0x7fd0ec1e64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.681+0000 7fd0f3989640 1 -- 172.21.15.139:0/3945843905 >> 172.21.15.139:0/3945843905 conn(0x7fd0ec0d1790 msgr2=0x7fd0ec0cd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.681+0000 7fd0f3989640 1 -- 172.21.15.139:0/3945843905 shutdown_connections 2024-04-25T09:58:54.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.681+0000 7fd0f3989640 1 -- 172.21.15.139:0/3945843905 wait complete. 2024-04-25T09:58:54.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:58:54.696 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:58:54.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.792+0000 7fae05291640 1 Processor -- start 2024-04-25T09:58:54.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.792+0000 7fae05291640 1 -- start start 2024-04-25T09:58:54.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.793+0000 7fae05291640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae000cee50 0x7fae00168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.793+0000 7fae05291640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae00161dc0 con 0x7fae001617e0 2024-04-25T09:58:54.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.793+0000 7fae05291640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae001620a0 con 0x7fae0015dbf0 2024-04-25T09:58:54.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.793+0000 7fae05291640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae000d1ee0 con 0x7fae000cee50 2024-04-25T09:58:54.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.793+0000 7fadfe7fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fae001617e0 0x7fae000ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55166/0 (socket says 172.21.15.139:55166) 2024-04-25T09:58:54.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.793+0000 7fadfe7fc640 1 -- 172.21.15.139:0/3658146490 learned_addr learned my addr 172.21.15.139:0/3658146490 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.793+0000 7fadff7fe640 1 --2- 172.21.15.139:0/3658146490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae000cee50 0x7fae00168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.793+0000 7fadfdffb640 1 -- 172.21.15.139:0/3658146490 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062268780 0 0) 0x7fae00161dc0 con 0x7fae001617e0 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.793+0000 7fadfdffb640 1 -- 172.21.15.139:0/3658146490 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade8003520 con 0x7fae001617e0 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.794+0000 7fadfdffb640 1 -- 172.21.15.139:0/3658146490 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429631092 0 0) 0x7fae001620a0 con 0x7fae0015dbf0 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.794+0000 7fadfdffb640 1 -- 172.21.15.139:0/3658146490 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae00161dc0 con 0x7fae0015dbf0 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.794+0000 7fadfdffb640 1 -- 172.21.15.139:0/3658146490 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4171571091 0 0) 0x7fade8003520 con 0x7fae001617e0 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.794+0000 7fadff7fe640 1 -- 172.21.15.139:0/3658146490 >> v1:172.21.15.150:6789/0 conn(0x7fae0015dbf0 legacy=0x7fae000b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.794+0000 7fadff7fe640 1 -- 172.21.15.139:0/3658146490 >> v1:172.21.15.139:6789/0 conn(0x7fae001617e0 legacy=0x7fae000ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.794+0000 7fadff7fe640 1 -- 172.21.15.139:0/3658146490 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae00162380 con 0x7fae000cee50 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.794+0000 7fadff7fe640 1 --2- 172.21.15.139:0/3658146490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae000cee50 0x7fae00168b10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fae000d2060 tx=0x7fadf802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcdd678623fd336c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.794+0000 7fadfdffb640 1 -- 172.21.15.139:0/3658146490 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8037070 con 0x7fae000cee50 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.795+0000 7fadfdffb640 1 -- 172.21.15.139:0/3658146490 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf802ab90 con 0x7fae000cee50 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.795+0000 7fadfdffb640 1 -- 172.21.15.139:0/3658146490 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4001670 con 0x7fae000cee50 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.795+0000 7fae05291640 1 -- 172.21.15.139:0/3658146490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae000cee50 msgr2=0x7fae00168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.795+0000 7fae05291640 1 --2- 172.21.15.139:0/3658146490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae000cee50 0x7fae00168b10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fae000d2060 tx=0x7fadf802a100 comp rx=0 tx=0).stop 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.795+0000 7fae05291640 1 -- 172.21.15.139:0/3658146490 shutdown_connections 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.795+0000 7fae05291640 1 --2- 172.21.15.139:0/3658146490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae000cee50 0x7fae00168b10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.795+0000 7fae05291640 1 -- 172.21.15.139:0/3658146490 >> 172.21.15.139:0/3658146490 conn(0x7fae000b5480 msgr2=0x7fae000b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.795+0000 7fae05291640 1 -- 172.21.15.139:0/3658146490 shutdown_connections 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.795+0000 7fae05291640 1 -- 172.21.15.139:0/3658146490 wait complete. 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.796+0000 7fae05291640 1 Processor -- start 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.796+0000 7fae05291640 1 -- start start 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.797+0000 7fae05291640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae000cee50 0x7fae00160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.797+0000 7fae05291640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0015dbf0 0x7fae00147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.797+0000 7fae05291640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae001617e0 0x7fae00159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.797+0000 7fae05291640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015dfd0 con 0x7fae0015dbf0 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.797+0000 7fae05291640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae000d3380 con 0x7fae001617e0 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.797+0000 7fae05291640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015f8e0 con 0x7fae000cee50 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.797+0000 7fadfe7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0015dbf0 0x7fae00147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.797+0000 7fadfe7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0015dbf0 0x7fae00147c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48262/0 (socket says 172.21.15.139:48262) 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.797+0000 7fadfe7fc640 1 -- 172.21.15.139:0/3137020584 learned_addr learned my addr 172.21.15.139:0/3137020584 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fadfe7fc640 1 -- 172.21.15.139:0/3137020584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae000cee50 msgr2=0x7fae00160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fadfeffd640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae000cee50 0x7fae00160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fadff7fe640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae001617e0 0x7fae00159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fadfe7fc640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae000cee50 0x7fae00160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fadfe7fc640 1 -- 172.21.15.139:0/3137020584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae001617e0 msgr2=0x7fae00159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fadfe7fc640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae001617e0 0x7fae00159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fadfe7fc640 1 -- 172.21.15.139:0/3137020584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0015f040 con 0x7fae0015dbf0 2024-04-25T09:58:54.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fadff7fe640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae001617e0 0x7fae00159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:54.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fadfeffd640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae000cee50 0x7fae00160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:54.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fadfe7fc640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0015dbf0 0x7fae00147c70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fadf400c780 tx=0x7fadf400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fade77fe640 1 -- 172.21.15.139:0/3137020584 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4005ad0 con 0x7fae0015dbf0 2024-04-25T09:58:54.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fade77fe640 1 -- 172.21.15.139:0/3137020584 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf4005c70 con 0x7fae0015dbf0 2024-04-25T09:58:54.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fae05291640 1 -- 172.21.15.139:0/3137020584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0015f320 con 0x7fae0015dbf0 2024-04-25T09:58:54.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fade77fe640 1 -- 172.21.15.139:0/3137020584 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf40072b0 con 0x7fae0015dbf0 2024-04-25T09:58:54.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.798+0000 7fae05291640 1 -- 172.21.15.139:0/3137020584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae00154a60 con 0x7fae0015dbf0 2024-04-25T09:58:54.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.799+0000 7fae05291640 1 -- 172.21.15.139:0/3137020584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae00074520 con 0x7fae0015dbf0 2024-04-25T09:58:54.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.799+0000 7fade77fe640 1 -- 172.21.15.139:0/3137020584 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadf4007450 con 0x7fae0015dbf0 2024-04-25T09:58:54.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.800+0000 7fade77fe640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fadec06b4d0 0x7fadec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.800+0000 7fade77fe640 1 -- 172.21.15.139:0/3137020584 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fadf4089e50 con 0x7fae0015dbf0 2024-04-25T09:58:54.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.800+0000 7fadfeffd640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fadec06b4d0 0x7fadec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.800+0000 7fadfeffd640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fadec06b4d0 0x7fadec06d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fadf0007fd0 tx=0x7fadf0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.804+0000 7fade77fe640 1 -- 172.21.15.139:0/3137020584 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fadf4005e20 con 0x7fae0015dbf0 2024-04-25T09:58:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.975+0000 7fae05291640 1 -- 172.21.15.139:0/3137020584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae000ca2d0 con 0x7fae0015dbf0 2024-04-25T09:58:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.975+0000 7fade77fe640 1 -- 172.21.15.139:0/3137020584 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fadf4058f60 con 0x7fae0015dbf0 2024-04-25T09:58:54.979 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:58:54.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.977+0000 7fae05291640 1 -- 172.21.15.139:0/3137020584 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fadec06b4d0 msgr2=0x7fadec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.977+0000 7fae05291640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fadec06b4d0 0x7fadec06d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fadf0007fd0 tx=0x7fadf0009040 comp rx=0 tx=0).stop 2024-04-25T09:58:54.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.977+0000 7fae05291640 1 -- 172.21.15.139:0/3137020584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0015dbf0 msgr2=0x7fae00147c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.977+0000 7fae05291640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0015dbf0 0x7fae00147c70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fadf400c780 tx=0x7fadf400cc40 comp rx=0 tx=0).stop 2024-04-25T09:58:54.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.977+0000 7fae05291640 1 -- 172.21.15.139:0/3137020584 shutdown_connections 2024-04-25T09:58:54.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.977+0000 7fae05291640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae0015dbf0 0x7fae00147c70 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.977+0000 7fae05291640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fadec06b4d0 0x7fadec06d990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.977+0000 7fae05291640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae001617e0 0x7fae00159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.977+0000 7fae05291640 1 --2- 172.21.15.139:0/3137020584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae000cee50 0x7fae00160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.977+0000 7fae05291640 1 -- 172.21.15.139:0/3137020584 >> 172.21.15.139:0/3137020584 conn(0x7fae000b5480 msgr2=0x7fae000d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.977+0000 7fae05291640 1 -- 172.21.15.139:0/3137020584 shutdown_connections 2024-04-25T09:58:54.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:54.977+0000 7fae05291640 1 -- 172.21.15.139:0/3137020584 wait complete. 2024-04-25T09:58:54.992 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:58:55.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.087+0000 7fac70f10640 1 Processor -- start 2024-04-25T09:58:55.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.087+0000 7fac70f10640 1 -- start start 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.088+0000 7fac70f10640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fac6c06ab60 0x7fac6c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.088+0000 7fac70f10640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6c0d0400 con 0x7fac6c0b36f0 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.088+0000 7fac70f10640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6c0d06e0 con 0x7fac6c160e90 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.088+0000 7fac70f10640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6c0d14a0 con 0x7fac6c06ab60 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.088+0000 7fac63fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fac6c06ab60 0x7fac6c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.088+0000 7fac63fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fac6c06ab60 0x7fac6c06aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48264/0 (socket says 172.21.15.139:48264) 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.088+0000 7fac63fff640 1 -- 172.21.15.139:0/1358710298 learned_addr learned my addr 172.21.15.139:0/1358710298 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.088+0000 7fac6ad9d640 1 --1- 172.21.15.139:0/1358710298 >> v1:172.21.15.139:6789/0 conn(0x7fac6c0b36f0 0x7fac6c0b3b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55170/0 (socket says 172.21.15.139:55170) 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.088+0000 7fac63fff640 1 -- 172.21.15.139:0/1358710298 >> v1:172.21.15.150:6789/0 conn(0x7fac6c160e90 legacy=0x7fac6c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.088+0000 7fac63fff640 1 -- 172.21.15.139:0/1358710298 >> v1:172.21.15.139:6789/0 conn(0x7fac6c0b36f0 legacy=0x7fac6c0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.088+0000 7fac69d9b640 1 -- 172.21.15.139:0/1358710298 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980734960 0 0) 0x7fac6c0d0400 con 0x7fac6c0b36f0 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.088+0000 7fac63fff640 1 -- 172.21.15.139:0/1358710298 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c0d09c0 con 0x7fac6c06ab60 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.089+0000 7fac63fff640 1 --2- 172.21.15.139:0/1358710298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac6c06ab60 0x7fac6c06aff0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fac540098a0 tx=0x7fac5402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9250aa6092830305 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.089+0000 7fac69d9b640 1 -- 172.21.15.139:0/1358710298 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54037070 con 0x7fac6c06ab60 2024-04-25T09:58:55.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.089+0000 7fac69d9b640 1 -- 172.21.15.139:0/1358710298 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac54032440 con 0x7fac6c06ab60 2024-04-25T09:58:55.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.089+0000 7fac69d9b640 1 -- 172.21.15.139:0/1358710298 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54032810 con 0x7fac6c06ab60 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.089+0000 7fac70f10640 1 -- 172.21.15.139:0/1358710298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac6c06ab60 msgr2=0x7fac6c06aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.089+0000 7fac70f10640 1 --2- 172.21.15.139:0/1358710298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac6c06ab60 0x7fac6c06aff0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fac540098a0 tx=0x7fac5402a290 comp rx=0 tx=0).stop 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.090+0000 7fac70f10640 1 -- 172.21.15.139:0/1358710298 shutdown_connections 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.090+0000 7fac70f10640 1 --2- 172.21.15.139:0/1358710298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac6c06ab60 0x7fac6c06aff0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.090+0000 7fac70f10640 1 -- 172.21.15.139:0/1358710298 >> 172.21.15.139:0/1358710298 conn(0x7fac6c0b23f0 msgr2=0x7fac6c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.090+0000 7fac70f10640 1 -- 172.21.15.139:0/1358710298 shutdown_connections 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.090+0000 7fac70f10640 1 -- 172.21.15.139:0/1358710298 wait complete. 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.091+0000 7fac70f10640 1 Processor -- start 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.091+0000 7fac70f10640 1 -- start start 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac70f10640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac6c06ab60 0x7fac6c158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac70f10640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac6c0b36f0 0x7fac6c0cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac70f10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac6c160e90 0x7fac6c15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac70f10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c161270 con 0x7fac6c160e90 2024-04-25T09:58:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac70f10640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c0d0f80 con 0x7fac6c0b36f0 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac70f10640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c0c9b90 con 0x7fac6c06ab60 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac6ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac6c160e90 0x7fac6c15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac6ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac6c160e90 0x7fac6c15b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48278/0 (socket says 172.21.15.139:48278) 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac6ad9d640 1 -- 172.21.15.139:0/4153118254 learned_addr learned my addr 172.21.15.139:0/4153118254 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac63fff640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac6c0b36f0 0x7fac6c0cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac6a59c640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac6c06ab60 0x7fac6c158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac6ad9d640 1 -- 172.21.15.139:0/4153118254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac6c06ab60 msgr2=0x7fac6c158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac6ad9d640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac6c06ab60 0x7fac6c158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac6ad9d640 1 -- 172.21.15.139:0/4153118254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac6c0b36f0 msgr2=0x7fac6c0cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac6ad9d640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac6c0b36f0 0x7fac6c0cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac6ad9d640 1 -- 172.21.15.139:0/4153118254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c0ca0d0 con 0x7fac6c160e90 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac6ad9d640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac6c160e90 0x7fac6c15b7e0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fac6400c7a0 tx=0x7fac6400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac62ffd640 1 -- 172.21.15.139:0/4153118254 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64018070 con 0x7fac6c160e90 2024-04-25T09:58:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac62ffd640 1 -- 172.21.15.139:0/4153118254 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6400a640 con 0x7fac6c160e90 2024-04-25T09:58:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac70f10640 1 -- 172.21.15.139:0/4153118254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6c0ca3b0 con 0x7fac6c160e90 2024-04-25T09:58:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac62ffd640 1 -- 172.21.15.139:0/4153118254 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6400aab0 con 0x7fac6c160e90 2024-04-25T09:58:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.092+0000 7fac70f10640 1 -- 172.21.15.139:0/4153118254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6c158880 con 0x7fac6c160e90 2024-04-25T09:58:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.093+0000 7fac70f10640 1 -- 172.21.15.139:0/4153118254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6c074520 con 0x7fac6c160e90 2024-04-25T09:58:55.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.094+0000 7fac62ffd640 1 -- 172.21.15.139:0/4153118254 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac6401b070 con 0x7fac6c160e90 2024-04-25T09:58:55.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.094+0000 7fac62ffd640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fac4c06b4d0 0x7fac4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.094+0000 7fac62ffd640 1 -- 172.21.15.139:0/4153118254 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fac64089540 con 0x7fac6c160e90 2024-04-25T09:58:55.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.094+0000 7fac6a59c640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fac4c06b4d0 0x7fac4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.095+0000 7fac6a59c640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fac4c06b4d0 0x7fac4c06d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fac5c000f60 tx=0x7fac5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.098+0000 7fac62ffd640 1 -- 172.21.15.139:0/4153118254 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac64057fc0 con 0x7fac6c160e90 2024-04-25T09:58:55.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.271+0000 7fac70f10640 1 -- 172.21.15.139:0/4153118254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fac6c0af840 con 0x7fac6c160e90 2024-04-25T09:58:55.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.272+0000 7fac62ffd640 1 -- 172.21.15.139:0/4153118254 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) v1 ==== 84+0+65 (secure 0 0 0) 0x7fac6405c060 con 0x7fac6c160e90 2024-04-25T09:58:55.275 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:55.276 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-04-25T09:58:55.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:58:55.276 INFO:teuthology.orchestra.run.smithi139.stdout: "poolnum": 1, 2024-04-25T09:58:55.276 INFO:teuthology.orchestra.run.smithi139.stdout: "poolname": ".mgr" 2024-04-25T09:58:55.276 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:58:55.276 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-04-25T09:58:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.274+0000 7fac70f10640 1 -- 172.21.15.139:0/4153118254 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fac4c06b4d0 msgr2=0x7fac4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.274+0000 7fac70f10640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fac4c06b4d0 0x7fac4c06d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fac5c000f60 tx=0x7fac5c009040 comp rx=0 tx=0).stop 2024-04-25T09:58:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.274+0000 7fac70f10640 1 -- 172.21.15.139:0/4153118254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac6c160e90 msgr2=0x7fac6c15b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.274+0000 7fac70f10640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac6c160e90 0x7fac6c15b7e0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fac6400c7a0 tx=0x7fac6400cc60 comp rx=0 tx=0).stop 2024-04-25T09:58:55.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.274+0000 7fac70f10640 1 -- 172.21.15.139:0/4153118254 shutdown_connections 2024-04-25T09:58:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.274+0000 7fac70f10640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac6c160e90 0x7fac6c15b7e0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.274+0000 7fac70f10640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fac4c06b4d0 0x7fac4c06d990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.274+0000 7fac70f10640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac6c0b36f0 0x7fac6c0cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.274+0000 7fac70f10640 1 --2- 172.21.15.139:0/4153118254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac6c06ab60 0x7fac6c158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.274+0000 7fac70f10640 1 -- 172.21.15.139:0/4153118254 >> 172.21.15.139:0/4153118254 conn(0x7fac6c0b23f0 msgr2=0x7fac6c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.274+0000 7fac70f10640 1 -- 172.21.15.139:0/4153118254 shutdown_connections 2024-04-25T09:58:55.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.274+0000 7fac70f10640 1 -- 172.21.15.139:0/4153118254 wait complete. 2024-04-25T09:58:55.288 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:58:55.289 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:58:55.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.386+0000 7fa5e78cb640 1 Processor -- start 2024-04-25T09:58:55.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.386+0000 7fa5e78cb640 1 -- start start 2024-04-25T09:58:55.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e78cb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa5e015f8c0 0x7fa5e009a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e78cb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5e015b560 0x7fa5e015b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e78cb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e00d03f0 con 0x7fa5e015fca0 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e78cb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e00d06c0 con 0x7fa5e015f8c0 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e78cb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e0099bf0 con 0x7fa5e015b560 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e5667640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa5e015f8c0 0x7fa5e009a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e5667640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa5e015f8c0 0x7fa5e009a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48292/0 (socket says 172.21.15.139:48292) 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e5667640 1 -- 172.21.15.139:0/1378643467 learned_addr learned my addr 172.21.15.139:0/1378643467 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e5e68640 1 --2- 172.21.15.139:0/1378643467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5e015b560 0x7fa5e015b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e5667640 1 -- 172.21.15.139:0/1378643467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5e015b560 msgr2=0x7fa5e015b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e5667640 1 --2- 172.21.15.139:0/1378643467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5e015b560 0x7fa5e015b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e5667640 1 -- 172.21.15.139:0/1378643467 >> v1:172.21.15.150:6789/0 conn(0x7fa5e015fca0 legacy=0x7fa5e00cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.387+0000 7fa5e5667640 1 -- 172.21.15.139:0/1378643467 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e0099ed0 con 0x7fa5e015f8c0 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.388+0000 7fa5e5667640 1 --2- 172.21.15.139:0/1378643467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5e015f8c0 0x7fa5e009a9b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009870 tx=0x7fa5dc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1018681e2455c4d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.388+0000 7fa5d7fff640 1 -- 172.21.15.139:0/1378643467 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc037070 con 0x7fa5e015f8c0 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.388+0000 7fa5d7fff640 1 -- 172.21.15.139:0/1378643467 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc02ae50 con 0x7fa5e015f8c0 2024-04-25T09:58:55.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.388+0000 7fa5d7fff640 1 -- 172.21.15.139:0/1378643467 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc032860 con 0x7fa5e015f8c0 2024-04-25T09:58:55.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.388+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1378643467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5e015f8c0 msgr2=0x7fa5e009a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.388+0000 7fa5e78cb640 1 --2- 172.21.15.139:0/1378643467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5e015f8c0 0x7fa5e009a9b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009870 tx=0x7fa5dc02a300 comp rx=0 tx=0).stop 2024-04-25T09:58:55.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.388+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1378643467 shutdown_connections 2024-04-25T09:58:55.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.388+0000 7fa5e78cb640 1 --2- 172.21.15.139:0/1378643467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5e015b560 0x7fa5e015b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.388+0000 7fa5e78cb640 1 --2- 172.21.15.139:0/1378643467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5e015f8c0 0x7fa5e009a9b0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.388+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1378643467 >> 172.21.15.139:0/1378643467 conn(0x7fa5e00b31f0 msgr2=0x7fa5e00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:55.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.389+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1378643467 shutdown_connections 2024-04-25T09:58:55.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.389+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1378643467 wait complete. 2024-04-25T09:58:55.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.390+0000 7fa5e78cb640 1 Processor -- start 2024-04-25T09:58:55.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.390+0000 7fa5e78cb640 1 -- start start 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.390+0000 7fa5e78cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5e015b560 0x7fa5e0151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.390+0000 7fa5e78cb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5e015f8c0 0x7fa5e00b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.390+0000 7fa5e78cb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5e015fca0 0x7fa5e01f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.390+0000 7fa5e78cb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e009aef0 con 0x7fa5e015b560 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.390+0000 7fa5e78cb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e009a490 con 0x7fa5e015f8c0 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.390+0000 7fa5e78cb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e015bed0 con 0x7fa5e015fca0 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.390+0000 7fa5e5667640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5e015b560 0x7fa5e0151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.390+0000 7fa5e5667640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5e015b560 0x7fa5e0151a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48306/0 (socket says 172.21.15.139:48306) 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.390+0000 7fa5e5667640 1 -- 172.21.15.139:0/1113597916 learned_addr learned my addr 172.21.15.139:0/1113597916 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5e5e68640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5e015fca0 0x7fa5e01f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5e4e66640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5e015f8c0 0x7fa5e00b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5e5667640 1 -- 172.21.15.139:0/1113597916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5e015fca0 msgr2=0x7fa5e01f16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5e5667640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5e015fca0 0x7fa5e01f16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5e5667640 1 -- 172.21.15.139:0/1113597916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5e015f8c0 msgr2=0x7fa5e00b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5e5667640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5e015f8c0 0x7fa5e00b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5e5667640 1 -- 172.21.15.139:0/1113597916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e0151f40 con 0x7fa5e015b560 2024-04-25T09:58:55.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5e5667640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5e015b560 0x7fa5e0151a00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc000c00 tx=0x7fa5dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5d5ffb640 1 -- 172.21.15.139:0/1113597916 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc041020 con 0x7fa5e015b560 2024-04-25T09:58:55.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5d5ffb640 1 -- 172.21.15.139:0/1113597916 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc03c670 con 0x7fa5e015b560 2024-04-25T09:58:55.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1113597916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e015ce00 con 0x7fa5e015b560 2024-04-25T09:58:55.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5d5ffb640 1 -- 172.21.15.139:0/1113597916 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc037040 con 0x7fa5e015b560 2024-04-25T09:58:55.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.391+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1113597916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e015d3c0 con 0x7fa5e015b560 2024-04-25T09:58:55.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.392+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1113597916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e0074520 con 0x7fa5e015b560 2024-04-25T09:58:55.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.393+0000 7fa5d5ffb640 1 -- 172.21.15.139:0/1113597916 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5dc044020 con 0x7fa5e015b560 2024-04-25T09:58:55.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.393+0000 7fa5d5ffb640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa5c406b4d0 0x7fa5c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.393+0000 7fa5d5ffb640 1 -- 172.21.15.139:0/1113597916 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa5dc0aa710 con 0x7fa5e015b560 2024-04-25T09:58:55.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.393+0000 7fa5e4e66640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa5c406b4d0 0x7fa5c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.394+0000 7fa5e4e66640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa5c406b4d0 0x7fa5c406d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0073980 tx=0x7fa5d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.397+0000 7fa5d5ffb640 1 -- 172.21.15.139:0/1113597916 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5dc04b030 con 0x7fa5e015b560 2024-04-25T09:58:55.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:55.566+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1113597916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa5e00cdc30 con 0x7fa5e015b560 2024-04-25T09:58:56.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.012+0000 7fa5d5ffb640 1 -- 172.21.15.139:0/1113597916 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa5dc079750 con 0x7fa5e015b560 2024-04-25T09:58:56.016 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' created 2024-04-25T09:58:56.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.014+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1113597916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa5c406b4d0 msgr2=0x7fa5c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:56.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.014+0000 7fa5e78cb640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa5c406b4d0 0x7fa5c406d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0073980 tx=0x7fa5d0009040 comp rx=0 tx=0).stop 2024-04-25T09:58:56.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.014+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1113597916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5e015b560 msgr2=0x7fa5e0151a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:56.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.014+0000 7fa5e78cb640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5e015b560 0x7fa5e0151a00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc000c00 tx=0x7fa5dc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.015+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1113597916 shutdown_connections 2024-04-25T09:58:56.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.015+0000 7fa5e78cb640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa5c406b4d0 0x7fa5c406d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.015+0000 7fa5e78cb640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5e015fca0 0x7fa5e01f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.015+0000 7fa5e78cb640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5e015f8c0 0x7fa5e00b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.015+0000 7fa5e78cb640 1 --2- 172.21.15.139:0/1113597916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5e015b560 0x7fa5e0151a00 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.015+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1113597916 >> 172.21.15.139:0/1113597916 conn(0x7fa5e00b31f0 msgr2=0x7fa5e015ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:56.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.015+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1113597916 shutdown_connections 2024-04-25T09:58:56.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.015+0000 7fa5e78cb640 1 -- 172.21.15.139:0/1113597916 wait complete. 2024-04-25T09:58:56.033 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:58:56.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.133+0000 7f4913f9e640 1 Processor -- start 2024-04-25T09:58:56.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.133+0000 7f4913f9e640 1 -- start start 2024-04-25T09:58:56.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.134+0000 7f4913f9e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f490c15dbf0 0x7f490c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:56.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.134+0000 7f4913f9e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c161dc0 con 0x7f490c0cee50 2024-04-25T09:58:56.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.134+0000 7f4913f9e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c1620a0 con 0x7f490c0b9370 2024-04-25T09:58:56.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.134+0000 7f4913f9e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c0d1ee0 con 0x7f490c15dbf0 2024-04-25T09:58:56.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.134+0000 7f4911d3a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f490c15dbf0 0x7f490c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:56.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.134+0000 7f491253b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f490c0cee50 0x7f490c0d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55174/0 (socket says 172.21.15.139:55174) 2024-04-25T09:58:56.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.134+0000 7f491253b640 1 -- 172.21.15.139:0/285115221 learned_addr learned my addr 172.21.15.139:0/285115221 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:56.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.135+0000 7f4911d3a640 1 -- 172.21.15.139:0/285115221 >> v1:172.21.15.150:6789/0 conn(0x7f490c0b9370 legacy=0x7f490c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:56.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.135+0000 7f4911d3a640 1 -- 172.21.15.139:0/285115221 >> v1:172.21.15.139:6789/0 conn(0x7f490c0cee50 legacy=0x7f490c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:56.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.135+0000 7f4911d3a640 1 -- 172.21.15.139:0/285115221 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c162380 con 0x7f490c15dbf0 2024-04-25T09:58:56.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.135+0000 7f4910d38640 1 -- 172.21.15.139:0/285115221 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373840058 0 0) 0x7f490c161dc0 con 0x7f490c0cee50 2024-04-25T09:58:56.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.135+0000 7f4911d3a640 1 --2- 172.21.15.139:0/285115221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490c15dbf0 0x7f490c1617e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f49080097a0 tx=0x7f490802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8aea9a743508b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:56.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.135+0000 7f4910d38640 1 -- 172.21.15.139:0/285115221 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908037070 con 0x7f490c15dbf0 2024-04-25T09:58:56.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.135+0000 7f4910d38640 1 -- 172.21.15.139:0/285115221 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908032440 con 0x7f490c15dbf0 2024-04-25T09:58:56.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.135+0000 7f4910d38640 1 -- 172.21.15.139:0/285115221 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908032810 con 0x7f490c15dbf0 2024-04-25T09:58:56.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.136+0000 7f4913f9e640 1 -- 172.21.15.139:0/285115221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490c15dbf0 msgr2=0x7f490c1617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:56.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.136+0000 7f4913f9e640 1 --2- 172.21.15.139:0/285115221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490c15dbf0 0x7f490c1617e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f49080097a0 tx=0x7f490802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.136+0000 7f4913f9e640 1 -- 172.21.15.139:0/285115221 shutdown_connections 2024-04-25T09:58:56.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.136+0000 7f4913f9e640 1 --2- 172.21.15.139:0/285115221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490c15dbf0 0x7f490c1617e0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.136+0000 7f4913f9e640 1 -- 172.21.15.139:0/285115221 >> 172.21.15.139:0/285115221 conn(0x7f490c0b5480 msgr2=0x7f490c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:56.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.136+0000 7f4913f9e640 1 -- 172.21.15.139:0/285115221 shutdown_connections 2024-04-25T09:58:56.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.136+0000 7f4913f9e640 1 -- 172.21.15.139:0/285115221 wait complete. 2024-04-25T09:58:56.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.137+0000 7f4913f9e640 1 Processor -- start 2024-04-25T09:58:56.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.137+0000 7f4913f9e640 1 -- start start 2024-04-25T09:58:56.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4913f9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490c0b9370 0x7f490c1f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:56.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4913f9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f490c0cee50 0x7f490c1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:56.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4913f9e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c15dbf0 0x7f490c1520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:56.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4913f9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c15dfd0 con 0x7f490c0b9370 2024-04-25T09:58:56.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4913f9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0b97d0 con 0x7f490c0cee50 2024-04-25T09:58:56.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4913f9e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c152630 con 0x7f490c15dbf0 2024-04-25T09:58:56.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4911d3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490c0b9370 0x7f490c1f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:56.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4911d3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490c0b9370 0x7f490c1f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48322/0 (socket says 172.21.15.139:48322) 2024-04-25T09:58:56.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4911d3a640 1 -- 172.21.15.139:0/3968417916 learned_addr learned my addr 172.21.15.139:0/3968417916 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:56.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4911d3a640 1 -- 172.21.15.139:0/3968417916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c15dbf0 msgr2=0x7f490c1520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:56.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4911d3a640 1 --2- 172.21.15.139:0/3968417916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c15dbf0 0x7f490c1520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4911d3a640 1 -- 172.21.15.139:0/3968417916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f490c0cee50 msgr2=0x7f490c1548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:56.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4911d3a640 1 --2- 172.21.15.139:0/3968417916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f490c0cee50 0x7f490c1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:56.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4911d3a640 1 -- 172.21.15.139:0/3968417916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c1f82e0 con 0x7f490c0b9370 2024-04-25T09:58:56.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.138+0000 7f4911d3a640 1 --2- 172.21.15.139:0/3968417916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490c0b9370 0x7f490c1f7c50 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f49080097a0 tx=0x7f49080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:56.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.139+0000 7f49027fc640 1 -- 172.21.15.139:0/3968417916 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908037040 con 0x7f490c0b9370 2024-04-25T09:58:56.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.139+0000 7f49027fc640 1 -- 172.21.15.139:0/3968417916 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908004250 con 0x7f490c0b9370 2024-04-25T09:58:56.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.139+0000 7f4913f9e640 1 -- 172.21.15.139:0/3968417916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c1f85c0 con 0x7f490c0b9370 2024-04-25T09:58:56.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.139+0000 7f49027fc640 1 -- 172.21.15.139:0/3968417916 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908032e20 con 0x7f490c0b9370 2024-04-25T09:58:56.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.139+0000 7f4913f9e640 1 -- 172.21.15.139:0/3968417916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c15a1c0 con 0x7f490c0b9370 2024-04-25T09:58:56.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.140+0000 7f4913f9e640 1 -- 172.21.15.139:0/3968417916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c15a450 con 0x7f490c0b9370 2024-04-25T09:58:56.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.140+0000 7f49027fc640 1 -- 172.21.15.139:0/3968417916 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4908030770 con 0x7f490c0b9370 2024-04-25T09:58:56.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.141+0000 7f49027fc640 1 --2- 172.21.15.139:0/3968417916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f48f006b500 0x7f48f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:56.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.141+0000 7f49027fc640 1 -- 172.21.15.139:0/3968417916 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f49080ab560 con 0x7f490c0b9370 2024-04-25T09:58:56.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.141+0000 7f4911539640 1 --2- 172.21.15.139:0/3968417916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f48f006b500 0x7f48f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:56.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.141+0000 7f4911539640 1 --2- 172.21.15.139:0/3968417916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f48f006b500 0x7f48f006d9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f490c15cef0 tx=0x7f48fc006ba0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:56.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.145+0000 7f49027fc640 1 -- 172.21.15.139:0/3968417916 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f490c15a450 con 0x7f490c0b9370 2024-04-25T09:58:56.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:56.316+0000 7f4913f9e640 1 -- 172.21.15.139:0/3968417916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f490c161b40 con 0x7f490c0b9370 2024-04-25T09:58:57.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.020+0000 7f49027fc640 1 -- 172.21.15.139:0/3968417916 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) v1 ==== 170+0+0 (secure 0 0 0) 0x7f490c161b40 con 0x7f490c0b9370 2024-04-25T09:58:57.024 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' created 2024-04-25T09:58:57.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.022+0000 7f4913f9e640 1 -- 172.21.15.139:0/3968417916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f48f006b500 msgr2=0x7f48f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:57.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.022+0000 7f4913f9e640 1 --2- 172.21.15.139:0/3968417916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f48f006b500 0x7f48f006d9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f490c15cef0 tx=0x7f48fc006ba0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.022+0000 7f4913f9e640 1 -- 172.21.15.139:0/3968417916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490c0b9370 msgr2=0x7f490c1f7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:57.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.022+0000 7f4913f9e640 1 --2- 172.21.15.139:0/3968417916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490c0b9370 0x7f490c1f7c50 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f49080097a0 tx=0x7f49080046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.023+0000 7f4913f9e640 1 -- 172.21.15.139:0/3968417916 shutdown_connections 2024-04-25T09:58:57.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.023+0000 7f4913f9e640 1 --2- 172.21.15.139:0/3968417916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f48f006b500 0x7f48f006d9c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.023+0000 7f4913f9e640 1 --2- 172.21.15.139:0/3968417916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c15dbf0 0x7f490c1520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.023+0000 7f4913f9e640 1 --2- 172.21.15.139:0/3968417916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f490c0cee50 0x7f490c1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.023+0000 7f4913f9e640 1 --2- 172.21.15.139:0/3968417916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f490c0b9370 0x7f490c1f7c50 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.023+0000 7f4913f9e640 1 -- 172.21.15.139:0/3968417916 >> 172.21.15.139:0/3968417916 conn(0x7f490c0b5480 msgr2=0x7f490c1583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:57.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.023+0000 7f4913f9e640 1 -- 172.21.15.139:0/3968417916 shutdown_connections 2024-04-25T09:58:57.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.023+0000 7f4913f9e640 1 -- 172.21.15.139:0/3968417916 wait complete. 2024-04-25T09:58:57.053 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:58:57.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.154+0000 7f266fb28640 1 Processor -- start 2024-04-25T09:58:57.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.155+0000 7f266fb28640 1 -- start start 2024-04-25T09:58:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.155+0000 7f266fb28640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2668145250 0x7f2668070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.155+0000 7f266fb28640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26680719d0 0x7f2668071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.155+0000 7f266fb28640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26680b1100 con 0x7f26680d11a0 2024-04-25T09:58:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.155+0000 7f266fb28640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26680ceed0 con 0x7f2668145250 2024-04-25T09:58:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.155+0000 7f266fb28640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26680ce2c0 con 0x7f26680719d0 2024-04-25T09:58:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.155+0000 7f266d8c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2668145250 0x7f2668070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.156+0000 7f266d8c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2668145250 0x7f2668070be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58082/0 (socket says 172.21.15.139:58082) 2024-04-25T09:58:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.156+0000 7f266d8c4640 1 -- 172.21.15.139:0/3341073030 learned_addr learned my addr 172.21.15.139:0/3341073030 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.156+0000 7f266d8c4640 1 -- 172.21.15.139:0/3341073030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26680719d0 msgr2=0x7f2668071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.156+0000 7f266d0c3640 1 --2- 172.21.15.139:0/3341073030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26680719d0 0x7f2668071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.156+0000 7f266d8c4640 1 --2- 172.21.15.139:0/3341073030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26680719d0 0x7f2668071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.156+0000 7f266d8c4640 1 -- 172.21.15.139:0/3341073030 >> v1:172.21.15.150:6789/0 conn(0x7f26680d11a0 legacy=0x7f266815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.156+0000 7f266d8c4640 1 -- 172.21.15.139:0/3341073030 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26680ce5a0 con 0x7f2668145250 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.156+0000 7f266d8c4640 1 --2- 172.21.15.139:0/3341073030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2668145250 0x7f2668070be0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2664009e50 tx=0x7f266402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8071007af4238ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.157+0000 7f266c8c2640 1 -- 172.21.15.139:0/3341073030 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664036070 con 0x7f2668145250 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.157+0000 7f266c8c2640 1 -- 172.21.15.139:0/3341073030 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2664031600 con 0x7f2668145250 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.157+0000 7f266c8c2640 1 -- 172.21.15.139:0/3341073030 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26640319d0 con 0x7f2668145250 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.157+0000 7f266fb28640 1 -- 172.21.15.139:0/3341073030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2668145250 msgr2=0x7f2668070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.157+0000 7f266fb28640 1 --2- 172.21.15.139:0/3341073030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2668145250 0x7f2668070be0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2664009e50 tx=0x7f266402a5b0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.157+0000 7f266fb28640 1 -- 172.21.15.139:0/3341073030 shutdown_connections 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.157+0000 7f266fb28640 1 --2- 172.21.15.139:0/3341073030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26680719d0 0x7f2668071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.157+0000 7f266fb28640 1 --2- 172.21.15.139:0/3341073030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2668145250 0x7f2668070be0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.157+0000 7f266fb28640 1 -- 172.21.15.139:0/3341073030 >> 172.21.15.139:0/3341073030 conn(0x7f266806f460 msgr2=0x7f266806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.157+0000 7f266fb28640 1 -- 172.21.15.139:0/3341073030 shutdown_connections 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.157+0000 7f266fb28640 1 -- 172.21.15.139:0/3341073030 wait complete. 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.158+0000 7f266fb28640 1 Processor -- start 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.158+0000 7f266fb28640 1 -- start start 2024-04-25T09:58:57.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.159+0000 7f266fb28640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26680719d0 0x7f266813c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.159+0000 7f266fb28640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26680d11a0 0x7f26681e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.159+0000 7f266fb28640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2668145250 0x7f26681510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.159+0000 7f266fb28640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2668145630 con 0x7f26680719d0 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.159+0000 7f266fb28640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26680ceb60 con 0x7f26680d11a0 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.159+0000 7f266fb28640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26680b5690 con 0x7f2668145250 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.159+0000 7f266d8c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26680719d0 0x7f266813c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.159+0000 7f266d8c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26680719d0 0x7f266813c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58096/0 (socket says 172.21.15.139:58096) 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.159+0000 7f266d8c4640 1 -- 172.21.15.139:0/570035237 learned_addr learned my addr 172.21.15.139:0/570035237 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.159+0000 7f266e0c5640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2668145250 0x7f26681510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.159+0000 7f266d0c3640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26680d11a0 0x7f26681e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.160+0000 7f266d8c4640 1 -- 172.21.15.139:0/570035237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2668145250 msgr2=0x7f26681510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.160+0000 7f266d8c4640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2668145250 0x7f26681510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.160+0000 7f266d8c4640 1 -- 172.21.15.139:0/570035237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26680d11a0 msgr2=0x7f26681e6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.160+0000 7f266d8c4640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26680d11a0 0x7f26681e6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:57.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.160+0000 7f266d8c4640 1 -- 172.21.15.139:0/570035237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266813d0d0 con 0x7f26680719d0 2024-04-25T09:58:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.160+0000 7f266d8c4640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26680719d0 0x7f266813c750 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2664034040 tx=0x7f2664004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.160+0000 7f265e7fc640 1 -- 172.21.15.139:0/570035237 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664036050 con 0x7f26680719d0 2024-04-25T09:58:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.160+0000 7f265e7fc640 1 -- 172.21.15.139:0/570035237 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2664042070 con 0x7f26680719d0 2024-04-25T09:58:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.160+0000 7f266fb28640 1 -- 172.21.15.139:0/570035237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266813d3b0 con 0x7f26680719d0 2024-04-25T09:58:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.160+0000 7f265e7fc640 1 -- 172.21.15.139:0/570035237 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664009510 con 0x7f26680719d0 2024-04-25T09:58:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.160+0000 7f266fb28640 1 -- 172.21.15.139:0/570035237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266813dc50 con 0x7f26680719d0 2024-04-25T09:58:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.161+0000 7f266fb28640 1 -- 172.21.15.139:0/570035237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26680b41e0 con 0x7f26680719d0 2024-04-25T09:58:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.161+0000 7f265e7fc640 1 -- 172.21.15.139:0/570035237 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2664007c50 con 0x7f26680719d0 2024-04-25T09:58:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.162+0000 7f265e7fc640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f264c06b4d0 0x7f264c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.162+0000 7f265e7fc640 1 -- 172.21.15.139:0/570035237 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f26640ab390 con 0x7f26680719d0 2024-04-25T09:58:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.162+0000 7f266d0c3640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f264c06b4d0 0x7f264c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.162+0000 7f266d0c3640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f264c06b4d0 0x7f264c06d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2658006fd0 tx=0x7f2658008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.166+0000 7f265e7fc640 1 -- 172.21.15.139:0/570035237 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26680b41e0 con 0x7f26680719d0 2024-04-25T09:58:57.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:57.373+0000 7f266fb28640 1 -- 172.21.15.139:0/570035237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2668067bb0 con 0x7f26680719d0 2024-04-25T09:58:58.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.021+0000 7f265e7fc640 1 -- 172.21.15.139:0/570035237 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v48) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2668067bb0 con 0x7f26680719d0 2024-04-25T09:58:58.025 INFO:teuthology.orchestra.run.smithi139.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T09:58:58.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.024+0000 7f266fb28640 1 -- 172.21.15.139:0/570035237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f264c06b4d0 msgr2=0x7f264c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.024+0000 7f266fb28640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f264c06b4d0 0x7f264c06d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2658006fd0 tx=0x7f2658008040 comp rx=0 tx=0).stop 2024-04-25T09:58:58.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.024+0000 7f266fb28640 1 -- 172.21.15.139:0/570035237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26680719d0 msgr2=0x7f266813c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.024+0000 7f266fb28640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26680719d0 0x7f266813c750 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2664034040 tx=0x7f2664004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:58.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.024+0000 7f266fb28640 1 -- 172.21.15.139:0/570035237 shutdown_connections 2024-04-25T09:58:58.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.024+0000 7f266fb28640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f264c06b4d0 0x7f264c06d990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.024+0000 7f266fb28640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2668145250 0x7f26681510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.024+0000 7f266fb28640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26680d11a0 0x7f26681e6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.024+0000 7f266fb28640 1 --2- 172.21.15.139:0/570035237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26680719d0 0x7f266813c750 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.024+0000 7f266fb28640 1 -- 172.21.15.139:0/570035237 >> 172.21.15.139:0/570035237 conn(0x7f266806f460 msgr2=0x7f2668141a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:58.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.024+0000 7f266fb28640 1 -- 172.21.15.139:0/570035237 shutdown_connections 2024-04-25T09:58:58.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.024+0000 7f266fb28640 1 -- 172.21.15.139:0/570035237 wait complete. 2024-04-25T09:58:58.052 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:58.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.152+0000 7f670fdb7640 1 Processor -- start 2024-04-25T09:58:58.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.152+0000 7f670fdb7640 1 -- start start 2024-04-25T09:58:58.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670fdb7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f67081611e0 0x7f67080b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670fdb7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f670806c780 0x7f67080d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670fdb7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67080d4740 con 0x7f67080cea00 2024-04-25T09:58:58.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670fdb7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67080d1fa0 con 0x7f670806c780 2024-04-25T09:58:58.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670fdb7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67080d2560 con 0x7f67081611e0 2024-04-25T09:58:58.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670e354640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f670806c780 0x7f67080d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670e354640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f670806c780 0x7f67080d29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58112/0 (socket says 172.21.15.139:58112) 2024-04-25T09:58:58.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670e354640 1 -- 172.21.15.139:0/1041565437 learned_addr learned my addr 172.21.15.139:0/1041565437 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:58.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670db53640 1 --2- 172.21.15.139:0/1041565437 >> v2:172.21.15.150:3300/0 conn(0x7f67081611e0 0x7f67080b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670e354640 1 -- 172.21.15.139:0/1041565437 >> v2:172.21.15.150:3300/0 conn(0x7f67081611e0 msgr2=0x7f67080b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:58.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670e354640 1 --2- 172.21.15.139:0/1041565437 >> v2:172.21.15.150:3300/0 conn(0x7f67081611e0 0x7f67080b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670e354640 1 -- 172.21.15.139:0/1041565437 >> v1:172.21.15.150:6789/0 conn(0x7f67080cea00 legacy=0x7f670806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:58.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.153+0000 7f670e354640 1 -- 172.21.15.139:0/1041565437 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67080b2770 con 0x7f670806c780 2024-04-25T09:58:58.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.154+0000 7f670e354640 1 --2- 172.21.15.139:0/1041565437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f670806c780 0x7f67080d29f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f670808c420 tx=0x7f670002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bef1c054e8d115 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:58.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.154+0000 7f670cb51640 1 -- 172.21.15.139:0/1041565437 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700036070 con 0x7f670806c780 2024-04-25T09:58:58.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.154+0000 7f670cb51640 1 -- 172.21.15.139:0/1041565437 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6700031600 con 0x7f670806c780 2024-04-25T09:58:58.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.154+0000 7f670cb51640 1 -- 172.21.15.139:0/1041565437 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67000319d0 con 0x7f670806c780 2024-04-25T09:58:58.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.155+0000 7f670fdb7640 1 -- 172.21.15.139:0/1041565437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f670806c780 msgr2=0x7f67080d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.155+0000 7f670fdb7640 1 --2- 172.21.15.139:0/1041565437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f670806c780 0x7f67080d29f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f670808c420 tx=0x7f670002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.155+0000 7f670fdb7640 1 -- 172.21.15.139:0/1041565437 shutdown_connections 2024-04-25T09:58:58.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.155+0000 7f670fdb7640 1 --2- 172.21.15.139:0/1041565437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f670806c780 0x7f67080d29f0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.155+0000 7f670fdb7640 1 --2- 172.21.15.139:0/1041565437 >> v2:172.21.15.150:3300/0 conn(0x7f67081611e0 0x7f67080b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.155+0000 7f670fdb7640 1 -- 172.21.15.139:0/1041565437 >> 172.21.15.139:0/1041565437 conn(0x7f67080b5b00 msgr2=0x7f67080d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:58.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.155+0000 7f670fdb7640 1 -- 172.21.15.139:0/1041565437 shutdown_connections 2024-04-25T09:58:58.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.155+0000 7f670fdb7640 1 -- 172.21.15.139:0/1041565437 wait complete. 2024-04-25T09:58:58.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.156+0000 7f670fdb7640 1 Processor -- start 2024-04-25T09:58:58.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.156+0000 7f670fdb7640 1 -- start start 2024-04-25T09:58:58.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670fdb7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f670806c780 0x7f67081f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670fdb7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f67080cea00 0x7f67081f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670fdb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67081611e0 0x7f6708161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670fdb7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67080d21f0 con 0x7f67081611e0 2024-04-25T09:58:58.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670fdb7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67080b2d30 con 0x7f670806c780 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670fdb7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67081615c0 con 0x7f67080cea00 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670e354640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67081611e0 0x7f6708161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670e354640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67081611e0 0x7f6708161ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58118/0 (socket says 172.21.15.139:58118) 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670e354640 1 -- 172.21.15.139:0/2218610618 learned_addr learned my addr 172.21.15.139:0/2218610618 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670db53640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f670806c780 0x7f67081f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670d352640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f67080cea00 0x7f67081f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670e354640 1 -- 172.21.15.139:0/2218610618 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f67080cea00 msgr2=0x7f67081f7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670e354640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f67080cea00 0x7f67081f7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670e354640 1 -- 172.21.15.139:0/2218610618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f670806c780 msgr2=0x7f67081f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670e354640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f670806c780 0x7f67081f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670e354640 1 -- 172.21.15.139:0/2218610618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67081f47a0 con 0x7f67081611e0 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670e354640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67081611e0 0x7f6708161ad0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6700034040 tx=0x7f6700004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f66fe7fc640 1 -- 172.21.15.139:0/2218610618 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700036050 con 0x7f67081611e0 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f66fe7fc640 1 -- 172.21.15.139:0/2218610618 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6700042070 con 0x7f67081611e0 2024-04-25T09:58:58.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670fdb7640 1 -- 172.21.15.139:0/2218610618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67081f3f00 con 0x7f67081611e0 2024-04-25T09:58:58.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f66fe7fc640 1 -- 172.21.15.139:0/2218610618 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700009510 con 0x7f67081611e0 2024-04-25T09:58:58.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.157+0000 7f670fdb7640 1 -- 172.21.15.139:0/2218610618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67081f44c0 con 0x7f67081611e0 2024-04-25T09:58:58.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.158+0000 7f670fdb7640 1 -- 172.21.15.139:0/2218610618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6708074520 con 0x7f67081611e0 2024-04-25T09:58:58.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.159+0000 7f66fe7fc640 1 -- 172.21.15.139:0/2218610618 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6700007bd0 con 0x7f67081611e0 2024-04-25T09:58:58.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.159+0000 7f66fe7fc640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f66ec06b4d0 0x7f66ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.159+0000 7f670db53640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f66ec06b4d0 0x7f66ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.159+0000 7f66fe7fc640 1 -- 172.21.15.139:0/2218610618 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67000ab0f0 con 0x7f67081611e0 2024-04-25T09:58:58.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.160+0000 7f670db53640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f66ec06b4d0 0x7f66ec06d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6704006fd0 tx=0x7f6704008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:58.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.164+0000 7f66fe7fc640 1 -- 172.21.15.139:0/2218610618 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6700079e70 con 0x7f67081611e0 2024-04-25T09:58:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.336+0000 7f670fdb7640 1 -- 172.21.15.139:0/2218610618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f670806ceb0 con 0x7f67081611e0 2024-04-25T09:58:58.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.337+0000 7f66fe7fc640 1 -- 172.21.15.139:0/2218610618 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16946 (secure 0 0 0) 0x7f6700079770 con 0x7f67081611e0 2024-04-25T09:58:58.341 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:58.341 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":88,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:58:57.374279+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:55.585380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:56.336230+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:58.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.339+0000 7f670fdb7640 1 -- 172.21.15.139:0/2218610618 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f66ec06b4d0 msgr2=0x7f66ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.339+0000 7f670fdb7640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f66ec06b4d0 0x7f66ec06d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6704006fd0 tx=0x7f6704008040 comp rx=0 tx=0).stop 2024-04-25T09:58:58.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.339+0000 7f670fdb7640 1 -- 172.21.15.139:0/2218610618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67081611e0 msgr2=0x7f6708161ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.339+0000 7f670fdb7640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67081611e0 0x7f6708161ad0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6700034040 tx=0x7f6700004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:58.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.339+0000 7f670fdb7640 1 -- 172.21.15.139:0/2218610618 shutdown_connections 2024-04-25T09:58:58.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.339+0000 7f670fdb7640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67081611e0 0x7f6708161ad0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.339+0000 7f670fdb7640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f66ec06b4d0 0x7f66ec06d990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.339+0000 7f670fdb7640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f67080cea00 0x7f67081f7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.339+0000 7f670fdb7640 1 --2- 172.21.15.139:0/2218610618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f670806c780 0x7f67081f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.339+0000 7f670fdb7640 1 -- 172.21.15.139:0/2218610618 >> 172.21.15.139:0/2218610618 conn(0x7f67080b5b00 msgr2=0x7f67080b93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:58.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.339+0000 7f670fdb7640 1 -- 172.21.15.139:0/2218610618 shutdown_connections 2024-04-25T09:58:58.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.339+0000 7f670fdb7640 1 -- 172.21.15.139:0/2218610618 wait complete. 2024-04-25T09:58:58.355 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:58.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.448+0000 7fe8ab59e640 1 Processor -- start 2024-04-25T09:58:58.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.448+0000 7fe8ab59e640 1 -- start start 2024-04-25T09:58:58.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.449+0000 7fe8ab59e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe8a406fe60 0x7fe8a40d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.449+0000 7fe8ab59e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8a40d05d0 0x7fe8a415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.449+0000 7fe8ab59e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a40b3590 con 0x7fe8a40af8d0 2024-04-25T09:58:58.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.449+0000 7fe8ab59e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a40afcb0 con 0x7fe8a406fe60 2024-04-25T09:58:58.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.449+0000 7fe8ab59e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40b3340 con 0x7fe8a40d05d0 2024-04-25T09:58:58.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.449+0000 7fe8aa59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe8a40af8d0 0x7fe8a40b5020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53474/0 (socket says 172.21.15.139:53474) 2024-04-25T09:58:58.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.449+0000 7fe8aa59c640 1 -- 172.21.15.139:0/2091091001 learned_addr learned my addr 172.21.15.139:0/2091091001 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:58.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154238902 0 0) 0x7fe8a40b3590 con 0x7fe8a40af8d0 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe890003520 con 0x7fe8a40af8d0 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a9d9b640 1 --2- 172.21.15.139:0/2091091001 >> v2:172.21.15.150:3300/0 conn(0x7fe8a406fe60 0x7fe8a40d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8aad9d640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8a40d05d0 0x7fe8a415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2939865967 0 0) 0x7fe890003520 con 0x7fe8a40af8d0 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8a40d05d0 msgr2=0x7fe8a415e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8a40d05d0 0x7fe8a415e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 >> v2:172.21.15.150:3300/0 conn(0x7fe8a406fe60 msgr2=0x7fe8a40d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 --2- 172.21.15.139:0/2091091001 >> v2:172.21.15.150:3300/0 conn(0x7fe8a406fe60 0x7fe8a40d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40b3870 con 0x7fe8a40af8d0 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fe898001670 con 0x7fe8a40af8d0 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a9d9b640 1 --2- 172.21.15.139:0/2091091001 >> v2:172.21.15.150:3300/0 conn(0x7fe8a406fe60 0x7fe8a40d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 >> v1:172.21.15.139:6789/0 conn(0x7fe8a40af8d0 legacy=0x7fe8a40b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8a40d05d0 0x7fe8900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe890003fe0 0x7fe890004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe890004d40 0x7fe8900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40afcb0 con 0x7fe890003fe0 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40b3340 con 0x7fe890004d40 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89000bc30 con 0x7fe8a40d05d0 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a9d9b640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe890003fe0 0x7fe890004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8aa59c640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8a40d05d0 0x7fe8900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8aad9d640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe890004d40 0x7fe8900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a9d9b640 1 -- 172.21.15.139:0/2091091001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8a40d05d0 msgr2=0x7fe8900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a9d9b640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8a40d05d0 0x7fe8900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a9d9b640 1 -- 172.21.15.139:0/2091091001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe890004d40 msgr2=0x7fe8900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a9d9b640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe890004d40 0x7fe8900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8a9d9b640 1 -- 172.21.15.139:0/2091091001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe890003520 con 0x7fe890003fe0 2024-04-25T09:58:58.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.450+0000 7fe8aad9d640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe890004d40 0x7fe8900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:58.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.451+0000 7fe8a9d9b640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe890003fe0 0x7fe890004800 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0098a0 tx=0x7fe890005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea18724aad90eb4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:58.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.451+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c00f040 con 0x7fe890003fe0 2024-04-25T09:58:58.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.451+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c02ad20 con 0x7fe890003fe0 2024-04-25T09:58:58.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.451+0000 7fe8a959a640 1 -- 172.21.15.139:0/2091091001 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c004250 con 0x7fe890003fe0 2024-04-25T09:58:58.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.452+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2091091001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe890003fe0 msgr2=0x7fe890004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.452+0000 7fe8ab59e640 1 --2- 172.21.15.139:0/2091091001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe890003fe0 0x7fe890004800 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0098a0 tx=0x7fe890005830 comp rx=0 tx=0).stop 2024-04-25T09:58:58.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.452+0000 7fe8aad9d640 1 -- 172.21.15.139:0/2091091001 reap_dead start 2024-04-25T09:58:58.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.452+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2091091001 shutdown_connections 2024-04-25T09:58:58.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.452+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2091091001 >> 172.21.15.139:0/2091091001 conn(0x7fe8a40b5a80 msgr2=0x7fe8a4099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:58.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.452+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2091091001 shutdown_connections 2024-04-25T09:58:58.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.452+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2091091001 wait complete. 2024-04-25T09:58:58.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.453+0000 7fe8ab59e640 1 Processor -- start 2024-04-25T09:58:58.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.453+0000 7fe8ab59e640 1 -- start start 2024-04-25T09:58:58.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8ab59e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890003fe0 0x7fe8a40afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8ab59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a409a620 0x7fe8a4148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8ab59e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8a415ca80 0x7fe8a415cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8ab59e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a415e4d0 con 0x7fe8a409a620 2024-04-25T09:58:58.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8ab59e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40b3b50 con 0x7fe8a415ca80 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8ab59e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a415e650 con 0x7fe890003fe0 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8a9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a409a620 0x7fe8a4148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8a9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a409a620 0x7fe8a4148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58138/0 (socket says 172.21.15.139:58138) 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8a9d9b640 1 -- 172.21.15.139:0/2706908063 learned_addr learned my addr 172.21.15.139:0/2706908063 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8aa59c640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890003fe0 0x7fe8a40afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8aad9d640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8a415ca80 0x7fe8a415cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8a9d9b640 1 -- 172.21.15.139:0/2706908063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890003fe0 msgr2=0x7fe8a40afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8a9d9b640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890003fe0 0x7fe8a40afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8a9d9b640 1 -- 172.21.15.139:0/2706908063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8a415ca80 msgr2=0x7fe8a415cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8a9d9b640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8a415ca80 0x7fe8a415cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8a9d9b640 1 -- 172.21.15.139:0/2706908063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40b0700 con 0x7fe8a409a620 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe8a9d9b640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a409a620 0x7fe8a4148f00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0099c0 tx=0x7fe89c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.454+0000 7fe896ffd640 1 -- 172.21.15.139:0/2706908063 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c041070 con 0x7fe8a409a620 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.455+0000 7fe896ffd640 1 -- 172.21.15.139:0/2706908063 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c02ab20 con 0x7fe8a409a620 2024-04-25T09:58:58.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.455+0000 7fe896ffd640 1 -- 172.21.15.139:0/2706908063 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c03d430 con 0x7fe8a409a620 2024-04-25T09:58:58.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.455+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2706908063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a40b09e0 con 0x7fe8a409a620 2024-04-25T09:58:58.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.455+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2706908063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a40b03f0 con 0x7fe8a409a620 2024-04-25T09:58:58.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.456+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2706908063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a404ef90 con 0x7fe8a409a620 2024-04-25T09:58:58.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.456+0000 7fe896ffd640 1 -- 172.21.15.139:0/2706908063 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe89c044070 con 0x7fe8a409a620 2024-04-25T09:58:58.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.456+0000 7fe896ffd640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe88806b500 0x7fe88806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.457+0000 7fe896ffd640 1 -- 172.21.15.139:0/2706908063 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe89c0aaa50 con 0x7fe8a409a620 2024-04-25T09:58:58.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.457+0000 7fe8aa59c640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe88806b500 0x7fe88806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.457+0000 7fe8aa59c640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe88806b500 0x7fe88806d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe898001d30 tx=0x7fe898007970 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:58.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.461+0000 7fe896ffd640 1 -- 172.21.15.139:0/2706908063 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe89c0790f0 con 0x7fe8a409a620 2024-04-25T09:58:58.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.631+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2706908063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8a406b800 con 0x7fe8a409a620 2024-04-25T09:58:58.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.633+0000 7fe896ffd640 1 -- 172.21.15.139:0/2706908063 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16946 (secure 0 0 0) 0x7fe89c07d190 con 0x7fe8a409a620 2024-04-25T09:58:58.637 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:58:58.637 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":88,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:58:57.374279+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:55.585380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:56.336230+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:58.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.635+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2706908063 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe88806b500 msgr2=0x7fe88806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.635+0000 7fe8ab59e640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe88806b500 0x7fe88806d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe898001d30 tx=0x7fe898007970 comp rx=0 tx=0).stop 2024-04-25T09:58:58.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.635+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2706908063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a409a620 msgr2=0x7fe8a4148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.635+0000 7fe8ab59e640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a409a620 0x7fe8a4148f00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0099c0 tx=0x7fe89c004730 comp rx=0 tx=0).stop 2024-04-25T09:58:58.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.635+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2706908063 shutdown_connections 2024-04-25T09:58:58.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.635+0000 7fe8ab59e640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a409a620 0x7fe8a4148f00 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.635+0000 7fe8ab59e640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe88806b500 0x7fe88806d9c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.635+0000 7fe8ab59e640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8a415ca80 0x7fe8a415cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.635+0000 7fe8ab59e640 1 --2- 172.21.15.139:0/2706908063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890003fe0 0x7fe8a40afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.635+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2706908063 >> 172.21.15.139:0/2706908063 conn(0x7fe8a40b5a80 msgr2=0x7fe8a40b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:58.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.635+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2706908063 shutdown_connections 2024-04-25T09:58:58.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.635+0000 7fe8ab59e640 1 -- 172.21.15.139:0/2706908063 wait complete. 2024-04-25T09:58:58.650 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:58:58.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.745+0000 7f72fdfc1640 1 Processor -- start 2024-04-25T09:58:58.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.745+0000 7f72fdfc1640 1 -- start start 2024-04-25T09:58:58.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.746+0000 7f72fdfc1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f72f815da30 0x7f72f80704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.746+0000 7f72fdfc1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f80731a0 con 0x7f72f8072d60 2024-04-25T09:58:58.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.746+0000 7f72fdfc1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f815de10 con 0x7f72f806b270 2024-04-25T09:58:58.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.746+0000 7f72fdfc1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f80cd540 con 0x7f72f815da30 2024-04-25T09:58:58.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.746+0000 7f72f6ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f72f8072d60 0x7f72f806ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53480/0 (socket says 172.21.15.139:53480) 2024-04-25T09:58:58.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.746+0000 7f72f6ffd640 1 -- 172.21.15.139:0/3533605845 learned_addr learned my addr 172.21.15.139:0/3533605845 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:58:58.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.747+0000 7f72f77fe640 1 --2- 172.21.15.139:0/3533605845 >> v2:172.21.15.150:3300/0 conn(0x7f72f815da30 0x7f72f80704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.747+0000 7f72f67fc640 1 -- 172.21.15.139:0/3533605845 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378254621 0 0) 0x7f72f80731a0 con 0x7f72f8072d60 2024-04-25T09:58:58.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.747+0000 7f72f67fc640 1 -- 172.21.15.139:0/3533605845 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72e0003520 con 0x7f72f8072d60 2024-04-25T09:58:58.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.747+0000 7f72f67fc640 1 -- 172.21.15.139:0/3533605845 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 780564189 0 0) 0x7f72e0003520 con 0x7f72f8072d60 2024-04-25T09:58:58.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.747+0000 7f72f77fe640 1 -- 172.21.15.139:0/3533605845 >> v1:172.21.15.150:6789/0 conn(0x7f72f806b270 legacy=0x7f72f8167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.747+0000 7f72f77fe640 1 -- 172.21.15.139:0/3533605845 >> v1:172.21.15.139:6789/0 conn(0x7f72f8072d60 legacy=0x7f72f806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.747+0000 7f72f77fe640 1 -- 172.21.15.139:0/3533605845 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f80d1470 con 0x7f72f815da30 2024-04-25T09:58:58.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.748+0000 7f72f77fe640 1 --2- 172.21.15.139:0/3533605845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f815da30 0x7f72f80704a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f72e80097a0 tx=0x7f72e802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ec59514806903d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:58.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.748+0000 7f72f67fc640 1 -- 172.21.15.139:0/3533605845 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8037070 con 0x7f72f815da30 2024-04-25T09:58:58.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.748+0000 7f72f67fc640 1 -- 172.21.15.139:0/3533605845 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e802ac50 con 0x7f72f815da30 2024-04-25T09:58:58.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.748+0000 7f72f67fc640 1 -- 172.21.15.139:0/3533605845 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec001690 con 0x7f72f815da30 2024-04-25T09:58:58.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.748+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3533605845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f815da30 msgr2=0x7f72f80704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.748+0000 7f72fdfc1640 1 --2- 172.21.15.139:0/3533605845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f815da30 0x7f72f80704a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f72e80097a0 tx=0x7f72e802a1c0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.748+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3533605845 shutdown_connections 2024-04-25T09:58:58.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.748+0000 7f72fdfc1640 1 --2- 172.21.15.139:0/3533605845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f815da30 0x7f72f80704a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.748+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3533605845 >> 172.21.15.139:0/3533605845 conn(0x7f72f80b23f0 msgr2=0x7f72f80b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:58.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.749+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3533605845 shutdown_connections 2024-04-25T09:58:58.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.749+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3533605845 wait complete. 2024-04-25T09:58:58.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.750+0000 7f72fdfc1640 1 Processor -- start 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.750+0000 7f72fdfc1640 1 -- start start 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.750+0000 7f72fdfc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f806b270 0x7f72f8151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.750+0000 7f72fdfc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f8072d60 0x7f72f8150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72fdfc1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72f815da30 0x7f72f81607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72fdfc1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f80b3b00 con 0x7f72f806b270 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72fdfc1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f80b3de0 con 0x7f72f8072d60 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72fdfc1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f80b42a0 con 0x7f72f815da30 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f806b270 0x7f72f8151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f806b270 0x7f72f8151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58142/0 (socket says 172.21.15.139:58142) 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f77fe640 1 -- 172.21.15.139:0/3629582597 learned_addr learned my addr 172.21.15.139:0/3629582597 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f6ffd640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f8072d60 0x7f72f8150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f7fff640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72f815da30 0x7f72f81607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f77fe640 1 -- 172.21.15.139:0/3629582597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72f815da30 msgr2=0x7f72f81607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f77fe640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72f815da30 0x7f72f81607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f77fe640 1 -- 172.21.15.139:0/3629582597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f8072d60 msgr2=0x7f72f8150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f77fe640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f8072d60 0x7f72f8150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f77fe640 1 -- 172.21.15.139:0/3629582597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f8150da0 con 0x7f72f806b270 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f7fff640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72f815da30 0x7f72f81607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f6ffd640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f8072d60 0x7f72f8150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72f77fe640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f806b270 0x7f72f8151180 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f72e80098c0 tx=0x7f72e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72dffff640 1 -- 172.21.15.139:0/3629582597 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8041020 con 0x7f72f806b270 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72dffff640 1 -- 172.21.15.139:0/3629582597 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e80044c0 con 0x7f72f806b270 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72dffff640 1 -- 172.21.15.139:0/3629582597 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8037040 con 0x7f72f806b270 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3629582597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f8154990 con 0x7f72f806b270 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.751+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3629582597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f8154f50 con 0x7f72f806b270 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.752+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3629582597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f8074520 con 0x7f72f806b270 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.753+0000 7f72dffff640 1 -- 172.21.15.139:0/3629582597 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72e8044020 con 0x7f72f806b270 2024-04-25T09:58:58.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.753+0000 7f72dffff640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f72e406b4d0 0x7f72e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:58.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.753+0000 7f72dffff640 1 -- 172.21.15.139:0/3629582597 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72e80aa7e0 con 0x7f72f806b270 2024-04-25T09:58:58.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.753+0000 7f72f6ffd640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f72e406b4d0 0x7f72e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:58.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.754+0000 7f72f6ffd640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f72e406b4d0 0x7f72e406d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f72ec001eb0 tx=0x7f72ec001d00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:58.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.757+0000 7f72dffff640 1 -- 172.21.15.139:0/3629582597 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72e804b030 con 0x7f72f806b270 2024-04-25T09:58:58.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:58.966+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3629582597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f72f806d2f0 con 0x7f72f806b270 2024-04-25T09:58:59.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.031+0000 7f72dffff640 1 -- 172.21.15.139:0/3629582597 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v49) v1 ==== 118+0+0 (secure 0 0 0) 0x7f72f8150da0 con 0x7f72f806b270 2024-04-25T09:58:59.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.033+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3629582597 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f72e406b4d0 msgr2=0x7f72e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.033+0000 7f72fdfc1640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f72e406b4d0 0x7f72e406d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f72ec001eb0 tx=0x7f72ec001d00 comp rx=0 tx=0).stop 2024-04-25T09:58:59.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.033+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3629582597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f806b270 msgr2=0x7f72f8151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.033+0000 7f72fdfc1640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f806b270 0x7f72f8151180 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f72e80098c0 tx=0x7f72e80046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.033+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3629582597 shutdown_connections 2024-04-25T09:58:59.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.033+0000 7f72fdfc1640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f72e406b4d0 0x7f72e406d990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.033+0000 7f72fdfc1640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72f815da30 0x7f72f81607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.033+0000 7f72fdfc1640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f8072d60 0x7f72f8150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.033+0000 7f72fdfc1640 1 --2- 172.21.15.139:0/3629582597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72f806b270 0x7f72f8151180 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.033+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3629582597 >> 172.21.15.139:0/3629582597 conn(0x7f72f80b23f0 msgr2=0x7f72f80af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:59.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.034+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3629582597 shutdown_connections 2024-04-25T09:58:59.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.034+0000 7f72fdfc1640 1 -- 172.21.15.139:0/3629582597 wait complete. 2024-04-25T09:58:59.050 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:58:59.050 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:58:59.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.147+0000 7fbb309f4640 1 Processor -- start 2024-04-25T09:58:59.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.147+0000 7fbb309f4640 1 -- start start 2024-04-25T09:58:59.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.147+0000 7fbb309f4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb2c144090 0x7fbb2c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.147+0000 7fbb309f4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbb2c0cc9e0 0x7fbb2c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.147+0000 7fbb309f4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbb2c0cce20 0x7fbb2c06e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.147+0000 7fbb309f4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2c0b2a50 con 0x7fbb2c0cce20 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.147+0000 7fbb309f4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2c15b090 con 0x7fbb2c0cc9e0 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.147+0000 7fbb309f4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c15b370 con 0x7fbb2c144090 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.148+0000 7fbb2ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbb2c0cce20 0x7fbb2c06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.148+0000 7fbb2ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbb2c0cce20 0x7fbb2c06e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58148/0 (socket says 172.21.15.139:58148) 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.148+0000 7fbb2ad9d640 1 -- 172.21.15.139:0/4230320981 learned_addr learned my addr 172.21.15.139:0/4230320981 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.148+0000 7fbb23fff640 1 --2- 172.21.15.139:0/4230320981 >> v2:172.21.15.150:3300/0 conn(0x7fbb2c0cc9e0 0x7fbb2c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.148+0000 7fbb2a59c640 1 --2- 172.21.15.139:0/4230320981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb2c144090 0x7fbb2c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.148+0000 7fbb2ad9d640 1 -- 172.21.15.139:0/4230320981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb2c144090 msgr2=0x7fbb2c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.148+0000 7fbb2ad9d640 1 --2- 172.21.15.139:0/4230320981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb2c144090 0x7fbb2c0b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.148+0000 7fbb2ad9d640 1 -- 172.21.15.139:0/4230320981 >> v2:172.21.15.150:3300/0 conn(0x7fbb2c0cc9e0 msgr2=0x7fbb2c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.148+0000 7fbb2ad9d640 1 --2- 172.21.15.139:0/4230320981 >> v2:172.21.15.150:3300/0 conn(0x7fbb2c0cc9e0 0x7fbb2c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.148+0000 7fbb2ad9d640 1 -- 172.21.15.139:0/4230320981 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c15b650 con 0x7fbb2c0cce20 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.148+0000 7fbb2ad9d640 1 --2- 172.21.15.139:0/4230320981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb2c0cce20 0x7fbb2c06e870 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c06b930 tx=0x7fbb2402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676f226a44bde2fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:59.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.149+0000 7fbb29d9b640 1 -- 172.21.15.139:0/4230320981 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24036070 con 0x7fbb2c0cce20 2024-04-25T09:58:59.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.149+0000 7fbb29d9b640 1 -- 172.21.15.139:0/4230320981 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb24031600 con 0x7fbb2c0cce20 2024-04-25T09:58:59.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.149+0000 7fbb29d9b640 1 -- 172.21.15.139:0/4230320981 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb240319d0 con 0x7fbb2c0cce20 2024-04-25T09:58:59.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.149+0000 7fbb309f4640 1 -- 172.21.15.139:0/4230320981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb2c0cce20 msgr2=0x7fbb2c06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.149+0000 7fbb309f4640 1 --2- 172.21.15.139:0/4230320981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb2c0cce20 0x7fbb2c06e870 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c06b930 tx=0x7fbb2402a610 comp rx=0 tx=0).stop 2024-04-25T09:58:59.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.149+0000 7fbb309f4640 1 -- 172.21.15.139:0/4230320981 shutdown_connections 2024-04-25T09:58:59.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.149+0000 7fbb309f4640 1 --2- 172.21.15.139:0/4230320981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb2c0cce20 0x7fbb2c06e870 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.149+0000 7fbb309f4640 1 --2- 172.21.15.139:0/4230320981 >> v2:172.21.15.150:3300/0 conn(0x7fbb2c0cc9e0 0x7fbb2c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.149+0000 7fbb309f4640 1 --2- 172.21.15.139:0/4230320981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb2c144090 0x7fbb2c0b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.149+0000 7fbb309f4640 1 -- 172.21.15.139:0/4230320981 >> 172.21.15.139:0/4230320981 conn(0x7fbb2c067000 msgr2=0x7fbb2c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:59.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.149+0000 7fbb309f4640 1 -- 172.21.15.139:0/4230320981 shutdown_connections 2024-04-25T09:58:59.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.149+0000 7fbb309f4640 1 -- 172.21.15.139:0/4230320981 wait complete. 2024-04-25T09:58:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.151+0000 7fbb309f4640 1 Processor -- start 2024-04-25T09:58:59.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.151+0000 7fbb309f4640 1 -- start start 2024-04-25T09:58:59.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.151+0000 7fbb309f4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb2c0cc9e0 0x7fbb2c15a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.151+0000 7fbb309f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb2c0cce20 0x7fbb2c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.151+0000 7fbb309f4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb2c144090 0x7fbb2c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.151+0000 7fbb309f4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c144470 con 0x7fbb2c0cce20 2024-04-25T09:58:59.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.151+0000 7fbb309f4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c15bc10 con 0x7fbb2c0cc9e0 2024-04-25T09:58:59.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.151+0000 7fbb309f4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c15ad80 con 0x7fbb2c144090 2024-04-25T09:58:59.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.151+0000 7fbb23fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb2c0cce20 0x7fbb2c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.151+0000 7fbb23fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb2c0cce20 0x7fbb2c06d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58154/0 (socket says 172.21.15.139:58154) 2024-04-25T09:58:59.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.151+0000 7fbb23fff640 1 -- 172.21.15.139:0/3227311880 learned_addr learned my addr 172.21.15.139:0/3227311880 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:58:59.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.152+0000 7fbb2a59c640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb2c0cc9e0 0x7fbb2c15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.152+0000 7fbb23fff640 1 -- 172.21.15.139:0/3227311880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb2c144090 msgr2=0x7fbb2c06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.152+0000 7fbb23fff640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb2c144090 0x7fbb2c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.152+0000 7fbb23fff640 1 -- 172.21.15.139:0/3227311880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb2c0cc9e0 msgr2=0x7fbb2c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:59.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.152+0000 7fbb23fff640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb2c0cc9e0 0x7fbb2c15a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:59.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.152+0000 7fbb23fff640 1 -- 172.21.15.139:0/3227311880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c0ada60 con 0x7fbb2c0cce20 2024-04-25T09:58:59.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.152+0000 7fbb23fff640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb2c0cce20 0x7fbb2c06d220 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400d800 tx=0x7fbb1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:59.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.152+0000 7fbb22ffd640 1 -- 172.21.15.139:0/3227311880 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14018030 con 0x7fbb2c0cce20 2024-04-25T09:58:59.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.152+0000 7fbb22ffd640 1 -- 172.21.15.139:0/3227311880 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb140096a0 con 0x7fbb2c0cce20 2024-04-25T09:58:59.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.152+0000 7fbb22ffd640 1 -- 172.21.15.139:0/3227311880 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14009870 con 0x7fbb2c0cce20 2024-04-25T09:58:59.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.152+0000 7fbb309f4640 1 -- 172.21.15.139:0/3227311880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c0add40 con 0x7fbb2c0cce20 2024-04-25T09:58:59.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.152+0000 7fbb309f4640 1 -- 172.21.15.139:0/3227311880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2c142880 con 0x7fbb2c0cce20 2024-04-25T09:58:59.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.153+0000 7fbb309f4640 1 -- 172.21.15.139:0/3227311880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2c073d80 con 0x7fbb2c0cce20 2024-04-25T09:58:59.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.153+0000 7fbb22ffd640 1 -- 172.21.15.139:0/3227311880 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb1401b070 con 0x7fbb2c0cce20 2024-04-25T09:58:59.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.154+0000 7fbb22ffd640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb0c06b500 0x7fbb0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:59.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.154+0000 7fbb2a59c640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb0c06b500 0x7fbb0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:59.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.154+0000 7fbb22ffd640 1 -- 172.21.15.139:0/3227311880 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb1408a570 con 0x7fbb2c0cce20 2024-04-25T09:58:59.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.154+0000 7fbb2a59c640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb0c06b500 0x7fbb0c06d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c006fd0 tx=0x7fbb1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:59.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.158+0000 7fbb22ffd640 1 -- 172.21.15.139:0/3227311880 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb2c073d80 con 0x7fbb2c0cce20 2024-04-25T09:58:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:58:59.364+0000 7fbb309f4640 1 -- 172.21.15.139:0/3227311880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbb2c074520 con 0x7fbb2c0cce20 2024-04-25T09:59:00.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.042+0000 7fbb22ffd640 1 -- 172.21.15.139:0/3227311880 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v50) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbb140592f0 con 0x7fbb2c0cce20 2024-04-25T09:59:00.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.045+0000 7fbb309f4640 1 -- 172.21.15.139:0/3227311880 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb0c06b500 msgr2=0x7fbb0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.045+0000 7fbb309f4640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb0c06b500 0x7fbb0c06d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c006fd0 tx=0x7fbb1c008040 comp rx=0 tx=0).stop 2024-04-25T09:59:00.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.045+0000 7fbb309f4640 1 -- 172.21.15.139:0/3227311880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb2c0cce20 msgr2=0x7fbb2c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.045+0000 7fbb309f4640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb2c0cce20 0x7fbb2c06d220 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400d800 tx=0x7fbb1400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.045+0000 7fbb309f4640 1 -- 172.21.15.139:0/3227311880 shutdown_connections 2024-04-25T09:59:00.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.045+0000 7fbb309f4640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb2c0cce20 0x7fbb2c06d220 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.045+0000 7fbb309f4640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb0c06b500 0x7fbb0c06d9c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.045+0000 7fbb309f4640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb2c144090 0x7fbb2c06d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.045+0000 7fbb309f4640 1 --2- 172.21.15.139:0/3227311880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb2c0cc9e0 0x7fbb2c15a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.045+0000 7fbb309f4640 1 -- 172.21.15.139:0/3227311880 >> 172.21.15.139:0/3227311880 conn(0x7fbb2c067000 msgr2=0x7fbb2c0b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.045+0000 7fbb309f4640 1 -- 172.21.15.139:0/3227311880 shutdown_connections 2024-04-25T09:59:00.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.045+0000 7fbb309f4640 1 -- 172.21.15.139:0/3227311880 wait complete. 2024-04-25T09:59:00.062 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:59:00.063 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:00.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.162+0000 7f09b8328640 1 Processor -- start 2024-04-25T09:59:00.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.162+0000 7f09b8328640 1 -- start start 2024-04-25T09:59:00.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b8328640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b006aff0 0x7f09b00b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b8328640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b006b480 con 0x7f09b0160e90 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b8328640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b00d0400 con 0x7f09b006ab60 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b8328640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00d14a0 con 0x7f09b006aff0 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b60c4640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f09b0160e90 0x7f09b00afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53494/0 (socket says 172.21.15.139:53494) 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b60c4640 1 -- 172.21.15.139:0/1891476512 learned_addr learned my addr 172.21.15.139:0/1891476512 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b68c5640 1 --2- 172.21.15.139:0/1891476512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b006aff0 0x7f09b00b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458316491 0 0) 0x7f09b006b480 con 0x7f09b0160e90 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f099c003520 con 0x7f09b0160e90 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330610191 0 0) 0x7f09b00d0400 con 0x7f09b006ab60 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09b006b480 con 0x7f09b006ab60 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2481420435 0 0) 0x7f099c003520 con 0x7f09b0160e90 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b006aff0 msgr2=0x7f09b00b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b50c2640 1 --2- 172.21.15.139:0/1891476512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b006aff0 0x7f09b00b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 >> v1:172.21.15.150:6789/0 conn(0x7f09b006ab60 legacy=0x7f09b00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00d06e0 con 0x7f09b0160e90 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.163+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f09ac001670 con 0x7f09b0160e90 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 >> v1:172.21.15.139:6789/0 conn(0x7f09b0160e90 legacy=0x7f09b00afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b50c2640 1 --2- 172.21.15.139:0/1891476512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f099c003b70 0x7f099c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b50c2640 1 --2- 172.21.15.139:0/1891476512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099c004800 0x7f099c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b50c2640 1 --2- 172.21.15.139:0/1891476512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b006aff0 0x7f099c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00d14a0 con 0x7f099c003b70 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c00bb50 con 0x7f099c004800 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c00bd20 con 0x7f09b006aff0 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b60c4640 1 --2- 172.21.15.139:0/1891476512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f099c003b70 0x7f099c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b58c3640 1 --2- 172.21.15.139:0/1891476512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099c004800 0x7f099c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b68c5640 1 --2- 172.21.15.139:0/1891476512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b006aff0 0x7f099c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b60c4640 1 -- 172.21.15.139:0/1891476512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b006aff0 msgr2=0x7f099c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b60c4640 1 --2- 172.21.15.139:0/1891476512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b006aff0 0x7f099c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b60c4640 1 -- 172.21.15.139:0/1891476512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099c004800 msgr2=0x7f099c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b60c4640 1 --2- 172.21.15.139:0/1891476512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099c004800 0x7f099c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.164+0000 7f09b60c4640 1 -- 172.21.15.139:0/1891476512 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c003520 con 0x7f099c003b70 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.165+0000 7f09b60c4640 1 --2- 172.21.15.139:0/1891476512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f099c003b70 0x7f099c003f70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00c110 tx=0x7f099c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfad6b8f2631ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.165+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac007be0 con 0x7f099c003b70 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.165+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ac005af0 con 0x7f099c003b70 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.165+0000 7f09b50c2640 1 -- 172.21.15.139:0/1891476512 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac007070 con 0x7f099c003b70 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.165+0000 7f09b8328640 1 -- 172.21.15.139:0/1891476512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f099c003b70 msgr2=0x7f099c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.165+0000 7f09b8328640 1 --2- 172.21.15.139:0/1891476512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f099c003b70 0x7f099c003f70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00c110 tx=0x7f099c005830 comp rx=0 tx=0).stop 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.165+0000 7f09b68c5640 1 -- 172.21.15.139:0/1891476512 reap_dead start 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.165+0000 7f09b8328640 1 -- 172.21.15.139:0/1891476512 shutdown_connections 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.165+0000 7f09b8328640 1 -- 172.21.15.139:0/1891476512 >> 172.21.15.139:0/1891476512 conn(0x7f09b00b23f0 msgr2=0x7f09b00b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.165+0000 7f09b8328640 1 -- 172.21.15.139:0/1891476512 shutdown_connections 2024-04-25T09:59:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.165+0000 7f09b8328640 1 -- 172.21.15.139:0/1891476512 wait complete. 2024-04-25T09:59:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.167+0000 7f09b8328640 1 Processor -- start 2024-04-25T09:59:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.167+0000 7f09b8328640 1 -- start start 2024-04-25T09:59:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.167+0000 7f09b8328640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099c003b70 0x7f09b01f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.167+0000 7f09b8328640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09b00cbf80 0x7f09b00cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.167+0000 7f09b8328640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b00c9ce0 0x7f09b00ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.167+0000 7f09b8328640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00d0f80 con 0x7f09b00cbf80 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.167+0000 7f09b8328640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00d0ca0 con 0x7f099c003b70 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.167+0000 7f09b8328640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0161270 con 0x7f09b00c9ce0 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.167+0000 7f09b58c3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09b00cbf80 0x7f09b00cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.167+0000 7f09b58c3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09b00cbf80 0x7f09b00cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58178/0 (socket says 172.21.15.139:58178) 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.167+0000 7f09b58c3640 1 -- 172.21.15.139:0/2114971568 learned_addr learned my addr 172.21.15.139:0/2114971568 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09b60c4640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099c003b70 0x7f09b01f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09b68c5640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b00c9ce0 0x7f09b00ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09b58c3640 1 -- 172.21.15.139:0/2114971568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b00c9ce0 msgr2=0x7f09b00ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09b58c3640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b00c9ce0 0x7f09b00ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09b58c3640 1 -- 172.21.15.139:0/2114971568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099c003b70 msgr2=0x7f09b01f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09b58c3640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099c003b70 0x7f09b01f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09b58c3640 1 -- 172.21.15.139:0/2114971568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00caa40 con 0x7f09b00cbf80 2024-04-25T09:59:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09b58c3640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09b00cbf80 0x7f09b00cc3e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f09a000f7d0 tx=0x7f09a000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09a6ffd640 1 -- 172.21.15.139:0/2114971568 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a001a070 con 0x7f09b00cbf80 2024-04-25T09:59:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09a6ffd640 1 -- 172.21.15.139:0/2114971568 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a000fe80 con 0x7f09b00cbf80 2024-04-25T09:59:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09b8328640 1 -- 172.21.15.139:0/2114971568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b00cad20 con 0x7f09b00cbf80 2024-04-25T09:59:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09a6ffd640 1 -- 172.21.15.139:0/2114971568 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0007050 con 0x7f09b00cbf80 2024-04-25T09:59:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.168+0000 7f09b8328640 1 -- 172.21.15.139:0/2114971568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b00cb2e0 con 0x7f09b00cbf80 2024-04-25T09:59:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.169+0000 7f09b8328640 1 -- 172.21.15.139:0/2114971568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b0074520 con 0x7f09b00cbf80 2024-04-25T09:59:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.169+0000 7f09a6ffd640 1 -- 172.21.15.139:0/2114971568 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09a001d070 con 0x7f09b00cbf80 2024-04-25T09:59:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.170+0000 7f09a6ffd640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f099406b500 0x7f099406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.170+0000 7f09b60c4640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f099406b500 0x7f099406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.170+0000 7f09a6ffd640 1 -- 172.21.15.139:0/2114971568 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09a0089b40 con 0x7f09b00cbf80 2024-04-25T09:59:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.170+0000 7f09b60c4640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f099406b500 0x7f099406d9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f09ac032000 tx=0x7f09ac02d000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.174+0000 7f09a6ffd640 1 -- 172.21.15.139:0/2114971568 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09a0058830 con 0x7f09b00cbf80 2024-04-25T09:59:00.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.380+0000 7f09b8328640 1 -- 172.21.15.139:0/2114971568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09b00728b0 con 0x7f09b00cbf80 2024-04-25T09:59:00.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.380+0000 7f09a6ffd640 1 -- 172.21.15.139:0/2114971568 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f09a0058130 con 0x7f09b00cbf80 2024-04-25T09:59:00.384 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:00.384 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7545,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1239243954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1239243954},{"type":"v1","addr":"172.21.15.139:6836","nonce":1239243954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T09:58:59.365616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7557":{"gid":7557,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2266713846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2266713846},{"type":"v1","addr":"172.21.15.150:6835","nonce":2266713846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:00.384 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 50 2024-04-25T09:59:00.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.382+0000 7f09b8328640 1 -- 172.21.15.139:0/2114971568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f099406b500 msgr2=0x7f099406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.382+0000 7f09b8328640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f099406b500 0x7f099406d9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f09ac032000 tx=0x7f09ac02d000 comp rx=0 tx=0).stop 2024-04-25T09:59:00.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.382+0000 7f09b8328640 1 -- 172.21.15.139:0/2114971568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09b00cbf80 msgr2=0x7f09b00cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.382+0000 7f09b8328640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09b00cbf80 0x7f09b00cc3e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f09a000f7d0 tx=0x7f09a000fc90 comp rx=0 tx=0).stop 2024-04-25T09:59:00.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.382+0000 7f09b8328640 1 -- 172.21.15.139:0/2114971568 shutdown_connections 2024-04-25T09:59:00.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.382+0000 7f09b8328640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09b00cbf80 0x7f09b00cc3e0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.382+0000 7f09b8328640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f099406b500 0x7f099406d9c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.382+0000 7f09b8328640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09b00c9ce0 0x7f09b00ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.382+0000 7f09b8328640 1 --2- 172.21.15.139:0/2114971568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f099c003b70 0x7f09b01f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.382+0000 7f09b8328640 1 -- 172.21.15.139:0/2114971568 >> 172.21.15.139:0/2114971568 conn(0x7f09b00b23f0 msgr2=0x7f09b00b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.382+0000 7f09b8328640 1 -- 172.21.15.139:0/2114971568 shutdown_connections 2024-04-25T09:59:00.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.382+0000 7f09b8328640 1 -- 172.21.15.139:0/2114971568 wait complete. 2024-04-25T09:59:00.397 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:00.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.491+0000 7f3659c4a640 1 Processor -- start 2024-04-25T09:59:00.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.491+0000 7f3659c4a640 1 -- start start 2024-04-25T09:59:00.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f3659c4a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f365415c9c0 0x7f36540ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f3659c4a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f36540ad570 0x7f365406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f3659c4a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36540af590 con 0x7f365406c1f0 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f3659c4a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36540af870 con 0x7f36540ad570 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f3659c4a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365415c530 con 0x7f365415c9c0 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36537fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f365406c1f0 0x7f365415fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53506/0 (socket says 172.21.15.139:53506) 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36537fe640 1 -- 172.21.15.139:0/3788846290 learned_addr learned my addr 172.21.15.139:0/3788846290 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f3653fff640 1 --2- 172.21.15.139:0/3788846290 >> v2:172.21.15.150:3300/0 conn(0x7f36540ad570 0x7f365406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f3652ffd640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f365415c9c0 0x7f36540ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360185722 0 0) 0x7f36540af590 con 0x7f365406c1f0 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f363c003520 con 0x7f365406c1f0 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 240446395 0 0) 0x7f363c003520 con 0x7f365406c1f0 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f365415c9c0 msgr2=0x7f36540ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36527fc640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f365415c9c0 0x7f36540ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 >> v2:172.21.15.150:3300/0 conn(0x7f36540ad570 msgr2=0x7f365406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36527fc640 1 --2- 172.21.15.139:0/3788846290 >> v2:172.21.15.150:3300/0 conn(0x7f36540ad570 0x7f365406ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365415bee0 con 0x7f365406c1f0 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f3644001670 con 0x7f365406c1f0 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 >> v1:172.21.15.139:6789/0 conn(0x7f365406c1f0 legacy=0x7f365415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.492+0000 7f36527fc640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c0039b0 0x7f363c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f36527fc640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363c004800 0x7f363c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f36527fc640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f365415c9c0 0x7f363c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36540af870 con 0x7f363c0039b0 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365415c530 con 0x7f363c004800 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c00b9c0 con 0x7f365415c9c0 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f36537fe640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c0039b0 0x7f363c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f3652ffd640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363c004800 0x7f363c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f3653fff640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f365415c9c0 0x7f363c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f36537fe640 1 -- 172.21.15.139:0/3788846290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f365415c9c0 msgr2=0x7f363c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f36537fe640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f365415c9c0 0x7f363c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f36537fe640 1 -- 172.21.15.139:0/3788846290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363c004800 msgr2=0x7f363c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f36537fe640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363c004800 0x7f363c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f36537fe640 1 -- 172.21.15.139:0/3788846290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c003520 con 0x7f363c0039b0 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.493+0000 7f3652ffd640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363c004800 0x7f363c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.494+0000 7f36537fe640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c0039b0 0x7f363c003dd0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f364400e8b0 tx=0x7f363c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16c4f5015e78fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.494+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364400e4a0 con 0x7f363c0039b0 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.494+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3644033070 con 0x7f363c0039b0 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.494+0000 7f36527fc640 1 -- 172.21.15.139:0/3788846290 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364402fcf0 con 0x7f363c0039b0 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.494+0000 7f3659c4a640 1 -- 172.21.15.139:0/3788846290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c0039b0 msgr2=0x7f363c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.494+0000 7f3659c4a640 1 --2- 172.21.15.139:0/3788846290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c0039b0 0x7f363c003dd0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f364400e8b0 tx=0x7f363c005830 comp rx=0 tx=0).stop 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.494+0000 7f3653fff640 1 -- 172.21.15.139:0/3788846290 reap_dead start 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.494+0000 7f3659c4a640 1 -- 172.21.15.139:0/3788846290 shutdown_connections 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.494+0000 7f3659c4a640 1 -- 172.21.15.139:0/3788846290 >> 172.21.15.139:0/3788846290 conn(0x7f36540b23d0 msgr2=0x7f36540cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.494+0000 7f3659c4a640 1 -- 172.21.15.139:0/3788846290 shutdown_connections 2024-04-25T09:59:00.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.494+0000 7f3659c4a640 1 -- 172.21.15.139:0/3788846290 wait complete. 2024-04-25T09:59:00.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.496+0000 7f3659c4a640 1 Processor -- start 2024-04-25T09:59:00.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.496+0000 7f3659c4a640 1 -- start start 2024-04-25T09:59:00.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.496+0000 7f3659c4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c0039b0 0x7f36541520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.496+0000 7f3659c4a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36541ef000 0x7f365414b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.496+0000 7f3659c4a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f365415e0e0 0x7f365415e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.496+0000 7f3659c4a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365406f7b0 con 0x7f363c0039b0 2024-04-25T09:59:00.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.496+0000 7f3659c4a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365406f630 con 0x7f365415e0e0 2024-04-25T09:59:00.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.496+0000 7f3659c4a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36540af160 con 0x7f36541ef000 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.496+0000 7f36537fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c0039b0 0x7f36541520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.496+0000 7f36537fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c0039b0 0x7f36541520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58210/0 (socket says 172.21.15.139:58210) 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.496+0000 7f36537fe640 1 -- 172.21.15.139:0/1234747647 learned_addr learned my addr 172.21.15.139:0/1234747647 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f3653fff640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f365415e0e0 0x7f365415e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f3652ffd640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36541ef000 0x7f365414b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f36537fe640 1 -- 172.21.15.139:0/1234747647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36541ef000 msgr2=0x7f365414b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f36537fe640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36541ef000 0x7f365414b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f36537fe640 1 -- 172.21.15.139:0/1234747647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f365415e0e0 msgr2=0x7f365415e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f36537fe640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f365415e0e0 0x7f365415e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f36537fe640 1 -- 172.21.15.139:0/1234747647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365414b890 con 0x7f363c0039b0 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f3652ffd640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36541ef000 0x7f365414b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f36537fe640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c0039b0 0x7f36541520e0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f364400e8b0 tx=0x7f365415c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f363bfff640 1 -- 172.21.15.139:0/1234747647 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364400e4a0 con 0x7f363c0039b0 2024-04-25T09:59:00.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f363bfff640 1 -- 172.21.15.139:0/1234747647 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36440388a0 con 0x7f363c0039b0 2024-04-25T09:59:00.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f3659c4a640 1 -- 172.21.15.139:0/1234747647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365414bb30 con 0x7f363c0039b0 2024-04-25T09:59:00.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f363bfff640 1 -- 172.21.15.139:0/1234747647 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644001c70 con 0x7f363c0039b0 2024-04-25T09:59:00.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.497+0000 7f3659c4a640 1 -- 172.21.15.139:0/1234747647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36540b33c0 con 0x7f363c0039b0 2024-04-25T09:59:00.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.498+0000 7f3659c4a640 1 -- 172.21.15.139:0/1234747647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3654074520 con 0x7f363c0039b0 2024-04-25T09:59:00.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.499+0000 7f363bfff640 1 -- 172.21.15.139:0/1234747647 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3644038c90 con 0x7f363c0039b0 2024-04-25T09:59:00.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.499+0000 7f363bfff640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f364006b4d0 0x7f364006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.499+0000 7f363bfff640 1 -- 172.21.15.139:0/1234747647 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f36440aed90 con 0x7f363c0039b0 2024-04-25T09:59:00.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.499+0000 7f3652ffd640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f364006b4d0 0x7f364006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.499+0000 7f3652ffd640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f364006b4d0 0x7f364006d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3654151be0 tx=0x7f3648008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.503+0000 7f363bfff640 1 -- 172.21.15.139:0/1234747647 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f364407da80 con 0x7f363c0039b0 2024-04-25T09:59:00.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.673+0000 7f3659c4a640 1 -- 172.21.15.139:0/1234747647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36540d09c0 con 0x7f363c0039b0 2024-04-25T09:59:00.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.674+0000 7f363bfff640 1 -- 172.21.15.139:0/1234747647 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f364407d380 con 0x7f363c0039b0 2024-04-25T09:59:00.678 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:00.678 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":89,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:58:59.031596+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:55.585380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:56.336230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:00.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.676+0000 7f3659c4a640 1 -- 172.21.15.139:0/1234747647 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f364006b4d0 msgr2=0x7f364006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.676+0000 7f3659c4a640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f364006b4d0 0x7f364006d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3654151be0 tx=0x7f3648008040 comp rx=0 tx=0).stop 2024-04-25T09:59:00.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.676+0000 7f3659c4a640 1 -- 172.21.15.139:0/1234747647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c0039b0 msgr2=0x7f36541520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.676+0000 7f3659c4a640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c0039b0 0x7f36541520e0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f364400e8b0 tx=0x7f365415c4a0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.676+0000 7f3659c4a640 1 -- 172.21.15.139:0/1234747647 shutdown_connections 2024-04-25T09:59:00.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.676+0000 7f3659c4a640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f364006b4d0 0x7f364006d990 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.676+0000 7f3659c4a640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f365415e0e0 0x7f365415e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.676+0000 7f3659c4a640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36541ef000 0x7f365414b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.676+0000 7f3659c4a640 1 --2- 172.21.15.139:0/1234747647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363c0039b0 0x7f36541520e0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.676+0000 7f3659c4a640 1 -- 172.21.15.139:0/1234747647 >> 172.21.15.139:0/1234747647 conn(0x7f36540b23d0 msgr2=0x7f365406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.676+0000 7f3659c4a640 1 -- 172.21.15.139:0/1234747647 shutdown_connections 2024-04-25T09:59:00.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.676+0000 7f3659c4a640 1 -- 172.21.15.139:0/1234747647 wait complete. 2024-04-25T09:59:00.691 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:59:00.691 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:59:00.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.784+0000 7f1e27f2b640 1 Processor -- start 2024-04-25T09:59:00.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.784+0000 7f1e27f2b640 1 -- start start 2024-04-25T09:59:00.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.785+0000 7f1e27f2b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1e20161850 0x7f1e200b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.785+0000 7f1e27f2b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e200cf250 con 0x7f1e2015dc60 2024-04-25T09:59:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.785+0000 7f1e27f2b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e200ce9b0 con 0x7f1e200d1080 2024-04-25T09:59:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.785+0000 7f1e27f2b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e200d3730 con 0x7f1e20161850 2024-04-25T09:59:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.785+0000 7f1e25cc7640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1e2015dc60 0x7f1e200d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53514/0 (socket says 172.21.15.139:53514) 2024-04-25T09:59:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.785+0000 7f1e264c8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1e20161850 0x7f1e200b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.785+0000 7f1e25cc7640 1 -- 172.21.15.139:0/37144563 learned_addr learned my addr 172.21.15.139:0/37144563 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.786+0000 7f1e264c8640 1 -- 172.21.15.139:0/37144563 >> v1:172.21.15.150:6789/0 conn(0x7f1e200d1080 legacy=0x7f1e200d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.786+0000 7f1e264c8640 1 -- 172.21.15.139:0/37144563 >> v1:172.21.15.139:6789/0 conn(0x7f1e2015dc60 legacy=0x7f1e200d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.786+0000 7f1e24cc5640 1 -- 172.21.15.139:0/37144563 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837292228 0 0) 0x7f1e1c001490 con 0x7f1e2015dc60 2024-04-25T09:59:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.786+0000 7f1e264c8640 1 -- 172.21.15.139:0/37144563 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e200cec90 con 0x7f1e20161850 2024-04-25T09:59:00.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.786+0000 7f1e264c8640 1 --2- 172.21.15.139:0/37144563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e20161850 0x7f1e200b6c10 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f1e180097a0 tx=0x7f1e1802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e6bd7e89aff82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.787+0000 7f1e24cc5640 1 -- 172.21.15.139:0/37144563 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18037070 con 0x7f1e20161850 2024-04-25T09:59:00.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.787+0000 7f1e24cc5640 1 -- 172.21.15.139:0/37144563 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e18032440 con 0x7f1e20161850 2024-04-25T09:59:00.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.787+0000 7f1e24cc5640 1 -- 172.21.15.139:0/37144563 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18032810 con 0x7f1e20161850 2024-04-25T09:59:00.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.787+0000 7f1e27f2b640 1 -- 172.21.15.139:0/37144563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e20161850 msgr2=0x7f1e200b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.787+0000 7f1e27f2b640 1 --2- 172.21.15.139:0/37144563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e20161850 0x7f1e200b6c10 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f1e180097a0 tx=0x7f1e1802a230 comp rx=0 tx=0).stop 2024-04-25T09:59:00.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.787+0000 7f1e27f2b640 1 -- 172.21.15.139:0/37144563 shutdown_connections 2024-04-25T09:59:00.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.787+0000 7f1e27f2b640 1 --2- 172.21.15.139:0/37144563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e20161850 0x7f1e200b6c10 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.787+0000 7f1e27f2b640 1 -- 172.21.15.139:0/37144563 >> 172.21.15.139:0/37144563 conn(0x7f1e200b5910 msgr2=0x7f1e200b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.787+0000 7f1e27f2b640 1 -- 172.21.15.139:0/37144563 shutdown_connections 2024-04-25T09:59:00.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.787+0000 7f1e27f2b640 1 -- 172.21.15.139:0/37144563 wait complete. 2024-04-25T09:59:00.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.788+0000 7f1e27f2b640 1 Processor -- start 2024-04-25T09:59:00.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.788+0000 7f1e27f2b640 1 -- start start 2024-04-25T09:59:00.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e27f2b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e200d1080 0x7f1e2014ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e27f2b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e2015dc60 0x7f1e20152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e27f2b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e20161850 0x7f1e201534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e27f2b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2015e040 con 0x7f1e200d1080 2024-04-25T09:59:00.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e27f2b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e200d1fc0 con 0x7f1e2015dc60 2024-04-25T09:59:00.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e27f2b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2015d3e0 con 0x7f1e20161850 2024-04-25T09:59:00.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e25cc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e200d1080 0x7f1e2014ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e25cc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e200d1080 0x7f1e2014ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58236/0 (socket says 172.21.15.139:58236) 2024-04-25T09:59:00.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e25cc7640 1 -- 172.21.15.139:0/2766857001 learned_addr learned my addr 172.21.15.139:0/2766857001 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:00.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e25cc7640 1 -- 172.21.15.139:0/2766857001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e20161850 msgr2=0x7f1e201534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e25cc7640 1 --2- 172.21.15.139:0/2766857001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e20161850 0x7f1e201534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e25cc7640 1 -- 172.21.15.139:0/2766857001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e2015dc60 msgr2=0x7f1e20152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e25cc7640 1 --2- 172.21.15.139:0/2766857001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e2015dc60 0x7f1e20152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.789+0000 7f1e25cc7640 1 -- 172.21.15.139:0/2766857001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2015c7d0 con 0x7f1e200d1080 2024-04-25T09:59:00.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.790+0000 7f1e25cc7640 1 --2- 172.21.15.139:0/2766857001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e200d1080 0x7f1e2014ad60 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00c810 tx=0x7f1e1c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.790+0000 7f1e167fc640 1 -- 172.21.15.139:0/2766857001 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c018070 con 0x7f1e200d1080 2024-04-25T09:59:00.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.790+0000 7f1e167fc640 1 -- 172.21.15.139:0/2766857001 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1c00a640 con 0x7f1e200d1080 2024-04-25T09:59:00.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.790+0000 7f1e167fc640 1 -- 172.21.15.139:0/2766857001 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c00aa50 con 0x7f1e200d1080 2024-04-25T09:59:00.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.790+0000 7f1e27f2b640 1 -- 172.21.15.139:0/2766857001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2015cab0 con 0x7f1e200d1080 2024-04-25T09:59:00.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.790+0000 7f1e27f2b640 1 -- 172.21.15.139:0/2766857001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2015d070 con 0x7f1e200d1080 2024-04-25T09:59:00.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.791+0000 7f1e27f2b640 1 -- 172.21.15.139:0/2766857001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e20074520 con 0x7f1e200d1080 2024-04-25T09:59:00.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.791+0000 7f1e167fc640 1 -- 172.21.15.139:0/2766857001 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e1c01b070 con 0x7f1e200d1080 2024-04-25T09:59:00.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.792+0000 7f1e167fc640 1 --2- 172.21.15.139:0/2766857001 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1e0406b4d0 0x7f1e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:00.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.792+0000 7f1e167fc640 1 -- 172.21.15.139:0/2766857001 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e1c089990 con 0x7f1e200d1080 2024-04-25T09:59:00.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.792+0000 7f1e254c6640 1 --2- 172.21.15.139:0/2766857001 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1e0406b4d0 0x7f1e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:00.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.792+0000 7f1e254c6640 1 --2- 172.21.15.139:0/2766857001 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1e0406b4d0 0x7f1e0406d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1e10000f60 tx=0x7f1e100079d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:00.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.796+0000 7f1e167fc640 1 -- 172.21.15.139:0/2766857001 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e200cf250 con 0x7f1e200d1080 2024-04-25T09:59:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.963+0000 7f1e27f2b640 1 -- 172.21.15.139:0/2766857001 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e200d16f0 con 0x7f1e0406b4d0 2024-04-25T09:59:00.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.969+0000 7f1e167fc640 1 -- 172.21.15.139:0/2766857001 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246767 (secure 0 0 0) 0x7f1e200d16f0 con 0x7f1e0406b4d0 2024-04-25T09:59:00.976 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:00.977 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:59:00.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.972+0000 7f1e27f2b640 1 -- 172.21.15.139:0/2766857001 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1e0406b4d0 msgr2=0x7f1e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.972+0000 7f1e27f2b640 1 --2- 172.21.15.139:0/2766857001 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1e0406b4d0 0x7f1e0406d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1e10000f60 tx=0x7f1e100079d0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.972+0000 7f1e27f2b640 1 -- 172.21.15.139:0/2766857001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e200d1080 msgr2=0x7f1e2014ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:00.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.972+0000 7f1e27f2b640 1 --2- 172.21.15.139:0/2766857001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e200d1080 0x7f1e2014ad60 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00c810 tx=0x7f1e1c00ccd0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.972+0000 7f1e27f2b640 1 -- 172.21.15.139:0/2766857001 shutdown_connections 2024-04-25T09:59:00.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.972+0000 7f1e27f2b640 1 --2- 172.21.15.139:0/2766857001 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1e0406b4d0 0x7f1e0406d990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.972+0000 7f1e27f2b640 1 --2- 172.21.15.139:0/2766857001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e20161850 0x7f1e201534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.972+0000 7f1e27f2b640 1 --2- 172.21.15.139:0/2766857001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e2015dc60 0x7f1e20152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.972+0000 7f1e27f2b640 1 --2- 172.21.15.139:0/2766857001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e200d1080 0x7f1e2014ad60 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:00.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.972+0000 7f1e27f2b640 1 -- 172.21.15.139:0/2766857001 >> 172.21.15.139:0/2766857001 conn(0x7f1e200b5910 msgr2=0x7f1e200b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:00.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.972+0000 7f1e27f2b640 1 -- 172.21.15.139:0/2766857001 shutdown_connections 2024-04-25T09:59:00.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:00.972+0000 7f1e27f2b640 1 -- 172.21.15.139:0/2766857001 wait complete. 2024-04-25T09:59:00.986 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-04-25T09:58:59.044341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259032,"kb_used_data":9544,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715688,"statfs":{"total":773094113280,"available":772828864512,"internally_reserved":0,"allocated":9773056,"data_stored":5097512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000729"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029464+0000","last_change":"2024-04-25T09:58:57.029464+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029464+0000","last_undegraded":"2024-04-25T09:58:57.029464+0000","last_fullsized":"2024-04-25T09:58:57.029464+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.034310+0000","last_change":"2024-04-25T09:58:57.034310+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.034310+0000","last_undegraded":"2024-04-25T09:58:57.034310+0000","last_fullsized":"2024-04-25T09:58:57.034310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029376+0000","last_change":"2024-04-25T09:58:57.029376+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029376+0000","last_undegraded":"2024-04-25T09:58:57.029376+0000","last_fullsized":"2024-04-25T09:58:57.029376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.034971+0000","last_change":"2024-04-25T09:58:57.034971+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.034971+0000","last_undegraded":"2024-04-25T09:58:57.034971+0000","last_fullsized":"2024-04-25T09:58:57.034971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028683+0000","last_change":"2024-04-25T09:58:57.028683+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028683+0000","last_undegraded":"2024-04-25T09:58:57.028683+0000","last_fullsized":"2024-04-25T09:58:57.028683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.036041+0000","last_change":"2024-04-25T09:58:57.036041+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.036041+0000","last_undegraded":"2024-04-25T09:58:57.036041+0000","last_fullsized":"2024-04-25T09:58:57.036041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024527+0000","last_change":"2024-04-25T09:58:57.024527+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024527+0000","last_undegraded":"2024-04-25T09:58:57.024527+0000","last_fullsized":"2024-04-25T09:58:57.024527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.034283+0000","last_change":"2024-04-25T09:58:57.034283+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.034283+0000","last_undegraded":"2024-04-25T09:58:57.034283+0000","last_fullsized":"2024-04-25T09:58:57.034283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026906+0000","last_change":"2024-04-25T09:58:57.026906+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026906+0000","last_undegraded":"2024-04-25T09:58:57.026906+0000","last_fullsized":"2024-04-25T09:58:57.026906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032805+0000","last_change":"2024-04-25T09:58:57.032805+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032805+0000","last_undegraded":"2024-04-25T09:58:57.032805+0000","last_fullsized":"2024-04-25T09:58:57.032805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029222+0000","last_change":"2024-04-25T09:58:57.029222+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029222+0000","last_undegraded":"2024-04-25T09:58:57.029222+0000","last_fullsized":"2024-04-25T09:58:57.029222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032544+0000","last_change":"2024-04-25T09:58:57.032544+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032544+0000","last_undegraded":"2024-04-25T09:58:57.032544+0000","last_fullsized":"2024-04-25T09:58:57.032544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033778+0000","last_change":"2024-04-25T09:58:57.033778+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033778+0000","last_undegraded":"2024-04-25T09:58:57.033778+0000","last_fullsized":"2024-04-25T09:58:57.033778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029419+0000","last_change":"2024-04-25T09:58:57.029419+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029419+0000","last_undegraded":"2024-04-25T09:58:57.029419+0000","last_fullsized":"2024-04-25T09:58:57.029419+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033181+0000","last_change":"2024-04-25T09:58:57.033181+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033181+0000","last_undegraded":"2024-04-25T09:58:57.033181+0000","last_fullsized":"2024-04-25T09:58:57.033181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.037221+0000","last_change":"2024-04-25T09:58:57.037221+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.037221+0000","last_undegraded":"2024-04-25T09:58:57.037221+0000","last_fullsized":"2024-04-25T09:58:57.037221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.037999+0000","last_change":"2024-04-25T09:58:57.037999+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.037999+0000","last_undegraded":"2024-04-25T09:58:57.037999+0000","last_fullsized":"2024-04-25T09:58:57.037999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029302+0000","last_change":"2024-04-25T09:58:57.029302+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029302+0000","last_undegraded":"2024-04-25T09:58:57.029302+0000","last_fullsized":"2024-04-25T09:58:57.029302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028752+0000","last_change":"2024-04-25T09:58:57.028752+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028752+0000","last_undegraded":"2024-04-25T09:58:57.028752+0000","last_fullsized":"2024-04-25T09:58:57.028752+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032721+0000","last_change":"2024-04-25T09:58:57.032721+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032721+0000","last_undegraded":"2024-04-25T09:58:57.032721+0000","last_fullsized":"2024-04-25T09:58:57.032721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029156+0000","last_change":"2024-04-25T09:58:57.029156+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029156+0000","last_undegraded":"2024-04-25T09:58:57.029156+0000","last_fullsized":"2024-04-25T09:58:57.029156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028607+0000","last_change":"2024-04-25T09:58:57.028607+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028607+0000","last_undegraded":"2024-04-25T09:58:57.028607+0000","last_fullsized":"2024-04-25T09:58:57.028607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032825+0000","last_change":"2024-04-25T09:58:57.032825+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032825+0000","last_undegraded":"2024-04-25T09:58:57.032825+0000","last_fullsized":"2024-04-25T09:58:57.032825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028484+0000","last_change":"2024-04-25T09:58:57.028484+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028484+0000","last_undegraded":"2024-04-25T09:58:57.028484+0000","last_fullsized":"2024-04-25T09:58:57.028484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033413+0000","last_change":"2024-04-25T09:58:57.033413+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033413+0000","last_undegraded":"2024-04-25T09:58:57.033413+0000","last_fullsized":"2024-04-25T09:58:57.033413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029507+0000","last_change":"2024-04-25T09:58:57.029507+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029507+0000","last_undegraded":"2024-04-25T09:58:57.029507+0000","last_fullsized":"2024-04-25T09:58:57.029507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032369+0000","last_change":"2024-04-25T09:58:57.032369+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032369+0000","last_undegraded":"2024-04-25T09:58:57.032369+0000","last_fullsized":"2024-04-25T09:58:57.032369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032287+0000","last_change":"2024-04-25T09:58:57.032287+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032287+0000","last_undegraded":"2024-04-25T09:58:57.032287+0000","last_fullsized":"2024-04-25T09:58:57.032287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024530+0000","last_change":"2024-04-25T09:58:57.024530+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024530+0000","last_undegraded":"2024-04-25T09:58:57.024530+0000","last_fullsized":"2024-04-25T09:58:57.024530+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026740+0000","last_change":"2024-04-25T09:58:57.026740+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026740+0000","last_undegraded":"2024-04-25T09:58:57.026740+0000","last_fullsized":"2024-04-25T09:58:57.026740+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032961+0000","last_change":"2024-04-25T09:58:57.032961+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032961+0000","last_undegraded":"2024-04-25T09:58:57.032961+0000","last_fullsized":"2024-04-25T09:58:57.032961+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032779+0000","last_change":"2024-04-25T09:58:57.032779+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032779+0000","last_undegraded":"2024-04-25T09:58:57.032779+0000","last_fullsized":"2024-04-25T09:58:57.032779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.036398+0000","last_change":"2024-04-25T09:58:57.036398+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.036398+0000","last_undegraded":"2024-04-25T09:58:57.036398+0000","last_fullsized":"2024-04-25T09:58:57.036398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046934+0000","last_change":"2024-04-25T09:58:57.043628+0000","last_active":"2024-04-25T09:58:57.046934+0000","last_peered":"2024-04-25T09:58:57.046934+0000","last_clean":"2024-04-25T09:58:57.046934+0000","last_became_active":"2024-04-25T09:58:57.043322+0000","last_became_peered":"2024-04-25T09:58:57.043322+0000","last_unstale":"2024-04-25T09:58:57.046934+0000","last_undegraded":"2024-04-25T09:58:57.046934+0000","last_fullsized":"2024-04-25T09:58:57.046934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:56.130588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.038701+0000","last_change":"2024-04-25T09:58:57.038701+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.038701+0000","last_undegraded":"2024-04-25T09:58:57.038701+0000","last_fullsized":"2024-04-25T09:58:57.038701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046492+0000","last_change":"2024-04-25T09:58:57.041488+0000","last_active":"2024-04-25T09:58:57.046492+0000","last_peered":"2024-04-25T09:58:57.046492+0000","last_clean":"2024-04-25T09:58:57.046492+0000","last_became_active":"2024-04-25T09:58:57.041117+0000","last_became_peered":"2024-04-25T09:58:57.041117+0000","last_unstale":"2024-04-25T09:58:57.046492+0000","last_undegraded":"2024-04-25T09:58:57.046492+0000","last_fullsized":"2024-04-25T09:58:57.046492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:59.594081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024524+0000","last_change":"2024-04-25T09:58:57.024524+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024524+0000","last_undegraded":"2024-04-25T09:58:57.024524+0000","last_fullsized":"2024-04-25T09:58:57.024524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.038904+0000","last_change":"2024-04-25T09:58:57.034001+0000","last_active":"2024-04-25T09:58:57.038904+0000","last_peered":"2024-04-25T09:58:57.038904+0000","last_clean":"2024-04-25T09:58:57.038904+0000","last_became_active":"2024-04-25T09:58:57.033646+0000","last_became_peered":"2024-04-25T09:58:57.033646+0000","last_unstale":"2024-04-25T09:58:57.038904+0000","last_undegraded":"2024-04-25T09:58:57.038904+0000","last_fullsized":"2024-04-25T09:58:57.038904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:20.730136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.035703+0000","last_change":"2024-04-25T09:58:57.035703+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.035703+0000","last_undegraded":"2024-04-25T09:58:57.035703+0000","last_fullsized":"2024-04-25T09:58:57.035703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047225+0000","last_change":"2024-04-25T09:58:57.044955+0000","last_active":"2024-04-25T09:58:57.047225+0000","last_peered":"2024-04-25T09:58:57.047225+0000","last_clean":"2024-04-25T09:58:57.047225+0000","last_became_active":"2024-04-25T09:58:57.044389+0000","last_became_peered":"2024-04-25T09:58:57.044389+0000","last_unstale":"2024-04-25T09:58:57.047225+0000","last_undegraded":"2024-04-25T09:58:57.047225+0000","last_fullsized":"2024-04-25T09:58:57.047225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:55.535171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031396+0000","last_change":"2024-04-25T09:58:57.031396+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031396+0000","last_undegraded":"2024-04-25T09:58:57.031396+0000","last_fullsized":"2024-04-25T09:58:57.031396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.038695+0000","last_change":"2024-04-25T09:58:57.034395+0000","last_active":"2024-04-25T09:58:57.038695+0000","last_peered":"2024-04-25T09:58:57.038695+0000","last_clean":"2024-04-25T09:58:57.038695+0000","last_became_active":"2024-04-25T09:58:57.034079+0000","last_became_peered":"2024-04-25T09:58:57.034079+0000","last_unstale":"2024-04-25T09:58:57.038695+0000","last_undegraded":"2024-04-25T09:58:57.038695+0000","last_fullsized":"2024-04-25T09:58:57.038695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:03.751385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032373+0000","last_change":"2024-04-25T09:58:57.032373+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032373+0000","last_undegraded":"2024-04-25T09:58:57.032373+0000","last_fullsized":"2024-04-25T09:58:57.032373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043740+0000","last_change":"2024-04-25T09:58:57.038771+0000","last_active":"2024-04-25T09:58:57.043740+0000","last_peered":"2024-04-25T09:58:57.043740+0000","last_clean":"2024-04-25T09:58:57.043740+0000","last_became_active":"2024-04-25T09:58:57.038480+0000","last_became_peered":"2024-04-25T09:58:57.038480+0000","last_unstale":"2024-04-25T09:58:57.043740+0000","last_undegraded":"2024-04-25T09:58:57.043740+0000","last_fullsized":"2024-04-25T09:58:57.043740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:58.574435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033026+0000","last_change":"2024-04-25T09:58:57.033026+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033026+0000","last_undegraded":"2024-04-25T09:58:57.033026+0000","last_fullsized":"2024-04-25T09:58:57.033026+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.035188+0000","last_change":"2024-04-25T09:58:57.034137+0000","last_active":"2024-04-25T09:58:57.035188+0000","last_peered":"2024-04-25T09:58:57.035188+0000","last_clean":"2024-04-25T09:58:57.035188+0000","last_became_active":"2024-04-25T09:58:57.033509+0000","last_became_peered":"2024-04-25T09:58:57.033509+0000","last_unstale":"2024-04-25T09:58:57.035188+0000","last_undegraded":"2024-04-25T09:58:57.035188+0000","last_fullsized":"2024-04-25T09:58:57.035188+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:51.854104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031838+0000","last_change":"2024-04-25T09:58:57.031838+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031838+0000","last_undegraded":"2024-04-25T09:58:57.031838+0000","last_fullsized":"2024-04-25T09:58:57.031838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047142+0000","last_change":"2024-04-25T09:58:57.045529+0000","last_active":"2024-04-25T09:58:57.047142+0000","last_peered":"2024-04-25T09:58:57.047142+0000","last_clean":"2024-04-25T09:58:57.047142+0000","last_became_active":"2024-04-25T09:58:57.045229+0000","last_became_peered":"2024-04-25T09:58:57.045229+0000","last_unstale":"2024-04-25T09:58:57.047142+0000","last_undegraded":"2024-04-25T09:58:57.047142+0000","last_fullsized":"2024-04-25T09:58:57.047142+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:34.875415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026908+0000","last_change":"2024-04-25T09:58:57.026908+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026908+0000","last_undegraded":"2024-04-25T09:58:57.026908+0000","last_fullsized":"2024-04-25T09:58:57.026908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.041384+0000","last_change":"2024-04-25T09:58:57.039169+0000","last_active":"2024-04-25T09:58:57.041384+0000","last_peered":"2024-04-25T09:58:57.041384+0000","last_clean":"2024-04-25T09:58:57.041384+0000","last_became_active":"2024-04-25T09:58:57.038892+0000","last_became_peered":"2024-04-25T09:58:57.038892+0000","last_unstale":"2024-04-25T09:58:57.041384+0000","last_undegraded":"2024-04-25T09:58:57.041384+0000","last_fullsized":"2024-04-25T09:58:57.041384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:58.402797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024520+0000","last_change":"2024-04-25T09:58:57.024520+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024520+0000","last_undegraded":"2024-04-25T09:58:57.024520+0000","last_fullsized":"2024-04-25T09:58:57.024520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.041964+0000","last_change":"2024-04-25T09:58:57.040183+0000","last_active":"2024-04-25T09:58:57.041964+0000","last_peered":"2024-04-25T09:58:57.041964+0000","last_clean":"2024-04-25T09:58:57.041964+0000","last_became_active":"2024-04-25T09:58:57.039924+0000","last_became_peered":"2024-04-25T09:58:57.039924+0000","last_unstale":"2024-04-25T09:58:57.041964+0000","last_undegraded":"2024-04-25T09:58:57.041964+0000","last_fullsized":"2024-04-25T09:58:57.041964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:22.066654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028488+0000","last_change":"2024-04-25T09:58:57.028488+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028488+0000","last_undegraded":"2024-04-25T09:58:57.028488+0000","last_fullsized":"2024-04-25T09:58:57.028488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047039+0000","last_change":"2024-04-25T09:58:57.045075+0000","last_active":"2024-04-25T09:58:57.047039+0000","last_peered":"2024-04-25T09:58:57.047039+0000","last_clean":"2024-04-25T09:58:57.047039+0000","last_became_active":"2024-04-25T09:58:57.044812+0000","last_became_peered":"2024-04-25T09:58:57.044812+0000","last_unstale":"2024-04-25T09:58:57.047039+0000","last_undegraded":"2024-04-25T09:58:57.047039+0000","last_fullsized":"2024-04-25T09:58:57.047039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:51.303697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.030942+0000","last_change":"2024-04-25T09:58:57.030942+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.030942+0000","last_undegraded":"2024-04-25T09:58:57.030942+0000","last_fullsized":"2024-04-25T09:58:57.030942+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047888+0000","last_change":"2024-04-25T09:58:57.044690+0000","last_active":"2024-04-25T09:58:57.047888+0000","last_peered":"2024-04-25T09:58:57.047888+0000","last_clean":"2024-04-25T09:58:57.047888+0000","last_became_active":"2024-04-25T09:58:57.044405+0000","last_became_peered":"2024-04-25T09:58:57.044405+0000","last_unstale":"2024-04-25T09:58:57.047888+0000","last_undegraded":"2024-04-25T09:58:57.047888+0000","last_fullsized":"2024-04-25T09:58:57.047888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:03.790236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.030477+0000","last_change":"2024-04-25T09:58:57.030477+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.030477+0000","last_undegraded":"2024-04-25T09:58:57.030477+0000","last_fullsized":"2024-04-25T09:58:57.030477+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047136+0000","last_change":"2024-04-25T09:58:57.045507+0000","last_active":"2024-04-25T09:58:57.047136+0000","last_peered":"2024-04-25T09:58:57.047136+0000","last_clean":"2024-04-25T09:58:57.047136+0000","last_became_active":"2024-04-25T09:58:57.045211+0000","last_became_peered":"2024-04-25T09:58:57.045211+0000","last_unstale":"2024-04-25T09:58:57.047136+0000","last_undegraded":"2024-04-25T09:58:57.047136+0000","last_fullsized":"2024-04-25T09:58:57.047136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:03.790236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026747+0000","last_change":"2024-04-25T09:58:57.026747+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026747+0000","last_undegraded":"2024-04-25T09:58:57.026747+0000","last_fullsized":"2024-04-25T09:58:57.026747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043240+0000","last_change":"2024-04-25T09:58:57.042334+0000","last_active":"2024-04-25T09:58:57.043240+0000","last_peered":"2024-04-25T09:58:57.043240+0000","last_clean":"2024-04-25T09:58:57.043240+0000","last_became_active":"2024-04-25T09:58:57.042071+0000","last_became_peered":"2024-04-25T09:58:57.042071+0000","last_unstale":"2024-04-25T09:58:57.043240+0000","last_undegraded":"2024-04-25T09:58:57.043240+0000","last_fullsized":"2024-04-25T09:58:57.043240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:35.349212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031415+0000","last_change":"2024-04-25T09:58:57.031415+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031415+0000","last_undegraded":"2024-04-25T09:58:57.031415+0000","last_fullsized":"2024-04-25T09:58:57.031415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046805+0000","last_change":"2024-04-25T09:58:57.046367+0000","last_active":"2024-04-25T09:58:57.046805+0000","last_peered":"2024-04-25T09:58:57.046805+0000","last_clean":"2024-04-25T09:58:57.046805+0000","last_became_active":"2024-04-25T09:58:57.046089+0000","last_became_peered":"2024-04-25T09:58:57.046089+0000","last_unstale":"2024-04-25T09:58:57.046805+0000","last_undegraded":"2024-04-25T09:58:57.046805+0000","last_fullsized":"2024-04-25T09:58:57.046805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:20.908778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032127+0000","last_change":"2024-04-25T09:58:57.032127+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032127+0000","last_undegraded":"2024-04-25T09:58:57.032127+0000","last_fullsized":"2024-04-25T09:58:57.032127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043706+0000","last_change":"2024-04-25T09:58:57.038473+0000","last_active":"2024-04-25T09:58:57.043706+0000","last_peered":"2024-04-25T09:58:57.043706+0000","last_clean":"2024-04-25T09:58:57.043706+0000","last_became_active":"2024-04-25T09:58:57.038170+0000","last_became_peered":"2024-04-25T09:58:57.038170+0000","last_unstale":"2024-04-25T09:58:57.043706+0000","last_undegraded":"2024-04-25T09:58:57.043706+0000","last_fullsized":"2024-04-25T09:58:57.043706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:51.303697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.035966+0000","last_change":"2024-04-25T09:58:57.035966+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.035966+0000","last_undegraded":"2024-04-25T09:58:57.035966+0000","last_fullsized":"2024-04-25T09:58:57.035966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043997+0000","last_change":"2024-04-25T09:58:57.040136+0000","last_active":"2024-04-25T09:58:57.043997+0000","last_peered":"2024-04-25T09:58:57.043997+0000","last_clean":"2024-04-25T09:58:57.043997+0000","last_became_active":"2024-04-25T09:58:57.039832+0000","last_became_peered":"2024-04-25T09:58:57.039832+0000","last_unstale":"2024-04-25T09:58:57.043997+0000","last_undegraded":"2024-04-25T09:58:57.043997+0000","last_fullsized":"2024-04-25T09:58:57.043997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:44.391697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031726+0000","last_change":"2024-04-25T09:58:57.031726+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031726+0000","last_undegraded":"2024-04-25T09:58:57.031726+0000","last_fullsized":"2024-04-25T09:58:57.031726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.041588+0000","last_change":"2024-04-25T09:58:57.039566+0000","last_active":"2024-04-25T09:58:57.041588+0000","last_peered":"2024-04-25T09:58:57.041588+0000","last_clean":"2024-04-25T09:58:57.041588+0000","last_became_active":"2024-04-25T09:58:57.039279+0000","last_became_peered":"2024-04-25T09:58:57.039279+0000","last_unstale":"2024-04-25T09:58:57.041588+0000","last_undegraded":"2024-04-25T09:58:57.041588+0000","last_fullsized":"2024-04-25T09:58:57.041588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:35.349212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028874+0000","last_change":"2024-04-25T09:58:57.028874+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028874+0000","last_undegraded":"2024-04-25T09:58:57.028874+0000","last_fullsized":"2024-04-25T09:58:57.028874+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046543+0000","last_change":"2024-04-25T09:58:57.043428+0000","last_active":"2024-04-25T09:58:57.046543+0000","last_peered":"2024-04-25T09:58:57.046543+0000","last_clean":"2024-04-25T09:58:57.046543+0000","last_became_active":"2024-04-25T09:58:57.043158+0000","last_became_peered":"2024-04-25T09:58:57.043158+0000","last_unstale":"2024-04-25T09:58:57.046543+0000","last_undegraded":"2024-04-25T09:58:57.046543+0000","last_fullsized":"2024-04-25T09:58:57.046543+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:50.057945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032298+0000","last_change":"2024-04-25T09:58:57.032298+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032298+0000","last_undegraded":"2024-04-25T09:58:57.032298+0000","last_fullsized":"2024-04-25T09:58:57.032298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.039320+0000","last_change":"2024-04-25T09:58:57.036704+0000","last_active":"2024-04-25T09:58:57.039320+0000","last_peered":"2024-04-25T09:58:57.039320+0000","last_clean":"2024-04-25T09:58:57.039320+0000","last_became_active":"2024-04-25T09:58:57.036421+0000","last_became_peered":"2024-04-25T09:58:57.036421+0000","last_unstale":"2024-04-25T09:58:57.039320+0000","last_undegraded":"2024-04-25T09:58:57.039320+0000","last_fullsized":"2024-04-25T09:58:57.039320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:23.461435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032498+0000","last_change":"2024-04-25T09:58:57.032498+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032498+0000","last_undegraded":"2024-04-25T09:58:57.032498+0000","last_fullsized":"2024-04-25T09:58:57.032498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.042350+0000","last_change":"2024-04-25T09:58:57.037903+0000","last_active":"2024-04-25T09:58:57.042350+0000","last_peered":"2024-04-25T09:58:57.042350+0000","last_clean":"2024-04-25T09:58:57.042350+0000","last_became_active":"2024-04-25T09:58:57.037640+0000","last_became_peered":"2024-04-25T09:58:57.037640+0000","last_unstale":"2024-04-25T09:58:57.042350+0000","last_undegraded":"2024-04-25T09:58:57.042350+0000","last_fullsized":"2024-04-25T09:58:57.042350+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:04.149145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031438+0000","last_change":"2024-04-25T09:58:57.031438+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031438+0000","last_undegraded":"2024-04-25T09:58:57.031438+0000","last_fullsized":"2024-04-25T09:58:57.031438+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046749+0000","last_change":"2024-04-25T09:58:57.043679+0000","last_active":"2024-04-25T09:58:57.046749+0000","last_peered":"2024-04-25T09:58:57.046749+0000","last_clean":"2024-04-25T09:58:57.046749+0000","last_became_active":"2024-04-25T09:58:57.043368+0000","last_became_peered":"2024-04-25T09:58:57.043368+0000","last_unstale":"2024-04-25T09:58:57.046749+0000","last_undegraded":"2024-04-25T09:58:57.046749+0000","last_fullsized":"2024-04-25T09:58:57.046749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:54.216572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.035284+0000","last_change":"2024-04-25T09:58:57.035284+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.035284+0000","last_undegraded":"2024-04-25T09:58:57.035284+0000","last_fullsized":"2024-04-25T09:58:57.035284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.042343+0000","last_change":"2024-04-25T09:58:57.037357+0000","last_active":"2024-04-25T09:58:57.042343+0000","last_peered":"2024-04-25T09:58:57.042343+0000","last_clean":"2024-04-25T09:58:57.042343+0000","last_became_active":"2024-04-25T09:58:57.037087+0000","last_became_peered":"2024-04-25T09:58:57.037087+0000","last_unstale":"2024-04-25T09:58:57.042343+0000","last_undegraded":"2024-04-25T09:58:57.042343+0000","last_fullsized":"2024-04-25T09:58:57.042343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:58.457158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032863+0000","last_change":"2024-04-25T09:58:57.032863+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032863+0000","last_undegraded":"2024-04-25T09:58:57.032863+0000","last_fullsized":"2024-04-25T09:58:57.032863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.044146+0000","last_change":"2024-04-25T09:58:57.040746+0000","last_active":"2024-04-25T09:58:57.044146+0000","last_peered":"2024-04-25T09:58:57.044146+0000","last_clean":"2024-04-25T09:58:57.044146+0000","last_became_active":"2024-04-25T09:58:57.040447+0000","last_became_peered":"2024-04-25T09:58:57.040447+0000","last_unstale":"2024-04-25T09:58:57.044146+0000","last_undegraded":"2024-04-25T09:58:57.044146+0000","last_fullsized":"2024-04-25T09:58:57.044146+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:04.360841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.038293+0000","last_change":"2024-04-25T09:58:57.038293+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.038293+0000","last_undegraded":"2024-04-25T09:58:57.038293+0000","last_fullsized":"2024-04-25T09:58:57.038293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.044039+0000","last_change":"2024-04-25T09:58:57.039890+0000","last_active":"2024-04-25T09:58:57.044039+0000","last_peered":"2024-04-25T09:58:57.044039+0000","last_clean":"2024-04-25T09:58:57.044039+0000","last_became_active":"2024-04-25T09:58:57.039428+0000","last_became_peered":"2024-04-25T09:58:57.039428+0000","last_unstale":"2024-04-25T09:58:57.044039+0000","last_undegraded":"2024-04-25T09:58:57.044039+0000","last_fullsized":"2024-04-25T09:58:57.044039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:34.094773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031608+0000","last_change":"2024-04-25T09:58:57.031608+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031608+0000","last_undegraded":"2024-04-25T09:58:57.031608+0000","last_fullsized":"2024-04-25T09:58:57.031608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047026+0000","last_change":"2024-04-25T09:58:57.045052+0000","last_active":"2024-04-25T09:58:57.047026+0000","last_peered":"2024-04-25T09:58:57.047026+0000","last_clean":"2024-04-25T09:58:57.047026+0000","last_became_active":"2024-04-25T09:58:57.044719+0000","last_became_peered":"2024-04-25T09:58:57.044719+0000","last_unstale":"2024-04-25T09:58:57.047026+0000","last_undegraded":"2024-04-25T09:58:57.047026+0000","last_fullsized":"2024-04-25T09:58:57.047026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:20.749313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032975+0000","last_change":"2024-04-25T09:58:57.032975+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032975+0000","last_undegraded":"2024-04-25T09:58:57.032975+0000","last_fullsized":"2024-04-25T09:58:57.032975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046867+0000","last_change":"2024-04-25T09:58:57.042212+0000","last_active":"2024-04-25T09:58:57.046867+0000","last_peered":"2024-04-25T09:58:57.046867+0000","last_clean":"2024-04-25T09:58:57.046867+0000","last_became_active":"2024-04-25T09:58:57.041931+0000","last_became_peered":"2024-04-25T09:58:57.041931+0000","last_unstale":"2024-04-25T09:58:57.046867+0000","last_undegraded":"2024-04-25T09:58:57.046867+0000","last_fullsized":"2024-04-25T09:58:57.046867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:39.639654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.030927+0000","last_change":"2024-04-25T09:58:57.030927+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.030927+0000","last_undegraded":"2024-04-25T09:58:57.030927+0000","last_fullsized":"2024-04-25T09:58:57.030927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.045305+0000","last_change":"2024-04-25T09:58:57.043114+0000","last_active":"2024-04-25T09:58:57.045305+0000","last_peered":"2024-04-25T09:58:57.045305+0000","last_clean":"2024-04-25T09:58:57.045305+0000","last_became_active":"2024-04-25T09:58:57.042800+0000","last_became_peered":"2024-04-25T09:58:57.042800+0000","last_unstale":"2024-04-25T09:58:57.045305+0000","last_undegraded":"2024-04-25T09:58:57.045305+0000","last_fullsized":"2024-04-25T09:58:57.045305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:04.360841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028484+0000","last_change":"2024-04-25T09:58:57.028484+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028484+0000","last_undegraded":"2024-04-25T09:58:57.028484+0000","last_fullsized":"2024-04-25T09:58:57.028484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.039332+0000","last_change":"2024-04-25T09:58:57.036703+0000","last_active":"2024-04-25T09:58:57.039332+0000","last_peered":"2024-04-25T09:58:57.039332+0000","last_clean":"2024-04-25T09:58:57.039332+0000","last_became_active":"2024-04-25T09:58:57.036409+0000","last_became_peered":"2024-04-25T09:58:57.036409+0000","last_unstale":"2024-04-25T09:58:57.039332+0000","last_undegraded":"2024-04-25T09:58:57.039332+0000","last_fullsized":"2024-04-25T09:58:57.039332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:26.837285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031019+0000","last_change":"2024-04-25T09:58:57.031019+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031019+0000","last_undegraded":"2024-04-25T09:58:57.031019+0000","last_fullsized":"2024-04-25T09:58:57.031019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.044055+0000","last_change":"2024-04-25T09:58:57.039451+0000","last_active":"2024-04-25T09:58:57.044055+0000","last_peered":"2024-04-25T09:58:57.044055+0000","last_clean":"2024-04-25T09:58:57.044055+0000","last_became_active":"2024-04-25T09:58:57.039193+0000","last_became_peered":"2024-04-25T09:58:57.039193+0000","last_unstale":"2024-04-25T09:58:57.044055+0000","last_undegraded":"2024-04-25T09:58:57.044055+0000","last_fullsized":"2024-04-25T09:58:57.044055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:35.986283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032671+0000","last_change":"2024-04-25T09:58:57.032671+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032671+0000","last_undegraded":"2024-04-25T09:58:57.032671+0000","last_fullsized":"2024-04-25T09:58:57.032671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.039001+0000","last_change":"2024-04-25T09:58:57.035195+0000","last_active":"2024-04-25T09:58:57.039001+0000","last_peered":"2024-04-25T09:58:57.039001+0000","last_clean":"2024-04-25T09:58:57.039001+0000","last_became_active":"2024-04-25T09:58:57.034665+0000","last_became_peered":"2024-04-25T09:58:57.034665+0000","last_unstale":"2024-04-25T09:58:57.039001+0000","last_undegraded":"2024-04-25T09:58:57.039001+0000","last_fullsized":"2024-04-25T09:58:57.039001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:03.668577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024524+0000","last_change":"2024-04-25T09:58:57.024524+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024524+0000","last_undegraded":"2024-04-25T09:58:57.024524+0000","last_fullsized":"2024-04-25T09:58:57.024524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.045377+0000","last_change":"2024-04-25T09:58:57.042156+0000","last_active":"2024-04-25T09:58:57.045377+0000","last_peered":"2024-04-25T09:58:57.045377+0000","last_clean":"2024-04-25T09:58:57.045377+0000","last_became_active":"2024-04-25T09:58:57.040423+0000","last_became_peered":"2024-04-25T09:58:57.040423+0000","last_unstale":"2024-04-25T09:58:57.045377+0000","last_undegraded":"2024-04-25T09:58:57.045377+0000","last_fullsized":"2024-04-25T09:58:57.045377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:27.846107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":293,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.025542+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:58:57.025542+0000","last_peered":"2024-04-25T09:58:57.025542+0000","last_clean":"2024-04-25T09:58:57.025542+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:58:57.025542+0000","last_undegraded":"2024-04-25T09:58:57.025542+0000","last_fullsized":"2024-04-25T09:58:57.025542+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1667072,"data_stored":1079965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1667072,"data_stored":1079965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:59:00.991 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:59:01.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.122+0000 7fd59d332640 1 Processor -- start 2024-04-25T09:59:01.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.122+0000 7fd59d332640 1 -- start start 2024-04-25T09:59:01.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.123+0000 7fd59d332640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd59815bdc0 0x7fd59815c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.123+0000 7fd59d332640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59815c880 con 0x7fd5980b1740 2024-04-25T09:59:01.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.123+0000 7fd59d332640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59815cb60 con 0x7fd598155e30 2024-04-25T09:59:01.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.123+0000 7fd59d332640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59815d690 con 0x7fd59815bdc0 2024-04-25T09:59:01.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.123+0000 7fd5967fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd59815bdc0 0x7fd59815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.123+0000 7fd5977fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd5980b1740 0x7fd59806db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53526/0 (socket says 172.21.15.139:53526) 2024-04-25T09:59:01.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.123+0000 7fd5977fe640 1 -- 172.21.15.139:0/256261447 learned_addr learned my addr 172.21.15.139:0/256261447 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:01.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.123+0000 7fd5967fc640 1 -- 172.21.15.139:0/256261447 >> v1:172.21.15.150:6789/0 conn(0x7fd598155e30 legacy=0x7fd59815cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:01.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.123+0000 7fd5967fc640 1 -- 172.21.15.139:0/256261447 >> v1:172.21.15.139:6789/0 conn(0x7fd5980b1740 legacy=0x7fd59806db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.123+0000 7fd5967fc640 1 -- 172.21.15.139:0/256261447 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59806d380 con 0x7fd59815bdc0 2024-04-25T09:59:01.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.124+0000 7fd5967fc640 1 --2- 172.21.15.139:0/256261447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd59815bdc0 0x7fd59815c240 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0097a0 tx=0x7fd58c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486aad2705725327 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:01.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.124+0000 7fd595ffb640 1 -- 172.21.15.139:0/256261447 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c037070 con 0x7fd59815bdc0 2024-04-25T09:59:01.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.124+0000 7fd595ffb640 1 -- 172.21.15.139:0/256261447 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c032440 con 0x7fd59815bdc0 2024-04-25T09:59:01.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.124+0000 7fd595ffb640 1 -- 172.21.15.139:0/256261447 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c032810 con 0x7fd59815bdc0 2024-04-25T09:59:01.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.125+0000 7fd59d332640 1 -- 172.21.15.139:0/256261447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd59815bdc0 msgr2=0x7fd59815c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.125+0000 7fd59d332640 1 --2- 172.21.15.139:0/256261447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd59815bdc0 0x7fd59815c240 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0097a0 tx=0x7fd58c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.125+0000 7fd59d332640 1 -- 172.21.15.139:0/256261447 shutdown_connections 2024-04-25T09:59:01.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.125+0000 7fd59d332640 1 --2- 172.21.15.139:0/256261447 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd59815bdc0 0x7fd59815c240 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.125+0000 7fd59d332640 1 -- 172.21.15.139:0/256261447 >> 172.21.15.139:0/256261447 conn(0x7fd598066f20 msgr2=0x7fd59806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:01.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.125+0000 7fd59d332640 1 -- 172.21.15.139:0/256261447 shutdown_connections 2024-04-25T09:59:01.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.125+0000 7fd59d332640 1 -- 172.21.15.139:0/256261447 wait complete. 2024-04-25T09:59:01.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.126+0000 7fd59d332640 1 Processor -- start 2024-04-25T09:59:01.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.126+0000 7fd59d332640 1 -- start start 2024-04-25T09:59:01.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd59d332640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5980b1740 0x7fd598156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd59d332640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd598155e30 0x7fd5981590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd59d332640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd59815bdc0 0x7fd59806db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd59d332640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980af0e0 con 0x7fd59815bdc0 2024-04-25T09:59:01.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd59d332640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59806d6f0 con 0x7fd5980b1740 2024-04-25T09:59:01.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd59d332640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980af260 con 0x7fd598155e30 2024-04-25T09:59:01.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd5977fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd59815bdc0 0x7fd59806db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd5977fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd59815bdc0 0x7fd59806db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58250/0 (socket says 172.21.15.139:58250) 2024-04-25T09:59:01.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd5977fe640 1 -- 172.21.15.139:0/3635622175 learned_addr learned my addr 172.21.15.139:0/3635622175 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:01.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd596ffd640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5980b1740 0x7fd598156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd5967fc640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd598155e30 0x7fd5981590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd5977fe640 1 -- 172.21.15.139:0/3635622175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd598155e30 msgr2=0x7fd5981590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd5977fe640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd598155e30 0x7fd5981590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd5977fe640 1 -- 172.21.15.139:0/3635622175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5980b1740 msgr2=0x7fd598156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd5977fe640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5980b1740 0x7fd598156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd5977fe640 1 -- 172.21.15.139:0/3635622175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980ad7d0 con 0x7fd59815bdc0 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.127+0000 7fd5977fe640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd59815bdc0 0x7fd59806db80 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c420 tx=0x7fd59000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.128+0000 7fd57f7fe640 1 -- 172.21.15.139:0/3635622175 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590017070 con 0x7fd59815bdc0 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.128+0000 7fd57f7fe640 1 -- 172.21.15.139:0/3635622175 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590005ce0 con 0x7fd59815bdc0 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.128+0000 7fd59d332640 1 -- 172.21.15.139:0/3635622175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5980af5d0 con 0x7fd59815bdc0 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.128+0000 7fd57f7fe640 1 -- 172.21.15.139:0/3635622175 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900072d0 con 0x7fd59815bdc0 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.128+0000 7fd59d332640 1 -- 172.21.15.139:0/3635622175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5980afb90 con 0x7fd59815bdc0 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.129+0000 7fd59d332640 1 -- 172.21.15.139:0/3635622175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5980cd650 con 0x7fd59815bdc0 2024-04-25T09:59:01.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.129+0000 7fd57f7fe640 1 -- 172.21.15.139:0/3635622175 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd59001a070 con 0x7fd59815bdc0 2024-04-25T09:59:01.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.129+0000 7fd57f7fe640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd58406b4d0 0x7fd58406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.130+0000 7fd57f7fe640 1 -- 172.21.15.139:0/3635622175 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd590089620 con 0x7fd59815bdc0 2024-04-25T09:59:01.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.130+0000 7fd596ffd640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd58406b4d0 0x7fd58406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.130+0000 7fd596ffd640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd58406b4d0 0x7fd58406d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd588005520 tx=0x7fd588009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:01.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.134+0000 7fd57f7fe640 1 -- 172.21.15.139:0/3635622175 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5980cd650 con 0x7fd59815bdc0 2024-04-25T09:59:01.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.302+0000 7fd59d332640 1 -- 172.21.15.139:0/3635622175 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd598074520 con 0x7fd58406b4d0 2024-04-25T09:59:01.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.308+0000 7fd57f7fe640 1 -- 172.21.15.139:0/3635622175 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246767 (secure 0 0 0) 0x7fd598074520 con 0x7fd58406b4d0 2024-04-25T09:59:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:01.317 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:59:01.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.311+0000 7fd59d332640 1 -- 172.21.15.139:0/3635622175 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd58406b4d0 msgr2=0x7fd58406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.311+0000 7fd59d332640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd58406b4d0 0x7fd58406d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd588005520 tx=0x7fd588009040 comp rx=0 tx=0).stop 2024-04-25T09:59:01.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.311+0000 7fd59d332640 1 -- 172.21.15.139:0/3635622175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd59815bdc0 msgr2=0x7fd59806db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.311+0000 7fd59d332640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd59815bdc0 0x7fd59806db80 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c420 tx=0x7fd59000c8e0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.311+0000 7fd59d332640 1 -- 172.21.15.139:0/3635622175 shutdown_connections 2024-04-25T09:59:01.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.311+0000 7fd59d332640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd59815bdc0 0x7fd59806db80 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.311+0000 7fd59d332640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd58406b4d0 0x7fd58406d990 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.311+0000 7fd59d332640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd598155e30 0x7fd5981590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.311+0000 7fd59d332640 1 --2- 172.21.15.139:0/3635622175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5980b1740 0x7fd598156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.311+0000 7fd59d332640 1 -- 172.21.15.139:0/3635622175 >> 172.21.15.139:0/3635622175 conn(0x7fd598066f20 msgr2=0x7fd59815a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:01.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.311+0000 7fd59d332640 1 -- 172.21.15.139:0/3635622175 shutdown_connections 2024-04-25T09:59:01.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.311+0000 7fd59d332640 1 -- 172.21.15.139:0/3635622175 wait complete. 2024-04-25T09:59:01.326 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-04-25T09:59:01.044642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259032,"kb_used_data":9544,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715688,"statfs":{"total":773094113280,"available":772828864512,"internally_reserved":0,"allocated":9773056,"data_stored":5097512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.031004"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029464+0000","last_change":"2024-04-25T09:58:57.029464+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029464+0000","last_undegraded":"2024-04-25T09:58:57.029464+0000","last_fullsized":"2024-04-25T09:58:57.029464+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.034310+0000","last_change":"2024-04-25T09:58:57.034310+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.034310+0000","last_undegraded":"2024-04-25T09:58:57.034310+0000","last_fullsized":"2024-04-25T09:58:57.034310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029376+0000","last_change":"2024-04-25T09:58:57.029376+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029376+0000","last_undegraded":"2024-04-25T09:58:57.029376+0000","last_fullsized":"2024-04-25T09:58:57.029376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.034971+0000","last_change":"2024-04-25T09:58:57.034971+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.034971+0000","last_undegraded":"2024-04-25T09:58:57.034971+0000","last_fullsized":"2024-04-25T09:58:57.034971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028683+0000","last_change":"2024-04-25T09:58:57.028683+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028683+0000","last_undegraded":"2024-04-25T09:58:57.028683+0000","last_fullsized":"2024-04-25T09:58:57.028683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.036041+0000","last_change":"2024-04-25T09:58:57.036041+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.036041+0000","last_undegraded":"2024-04-25T09:58:57.036041+0000","last_fullsized":"2024-04-25T09:58:57.036041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024527+0000","last_change":"2024-04-25T09:58:57.024527+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024527+0000","last_undegraded":"2024-04-25T09:58:57.024527+0000","last_fullsized":"2024-04-25T09:58:57.024527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.034283+0000","last_change":"2024-04-25T09:58:57.034283+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.034283+0000","last_undegraded":"2024-04-25T09:58:57.034283+0000","last_fullsized":"2024-04-25T09:58:57.034283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026906+0000","last_change":"2024-04-25T09:58:57.026906+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026906+0000","last_undegraded":"2024-04-25T09:58:57.026906+0000","last_fullsized":"2024-04-25T09:58:57.026906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032805+0000","last_change":"2024-04-25T09:58:57.032805+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032805+0000","last_undegraded":"2024-04-25T09:58:57.032805+0000","last_fullsized":"2024-04-25T09:58:57.032805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029222+0000","last_change":"2024-04-25T09:58:57.029222+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029222+0000","last_undegraded":"2024-04-25T09:58:57.029222+0000","last_fullsized":"2024-04-25T09:58:57.029222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032544+0000","last_change":"2024-04-25T09:58:57.032544+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032544+0000","last_undegraded":"2024-04-25T09:58:57.032544+0000","last_fullsized":"2024-04-25T09:58:57.032544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033778+0000","last_change":"2024-04-25T09:58:57.033778+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033778+0000","last_undegraded":"2024-04-25T09:58:57.033778+0000","last_fullsized":"2024-04-25T09:58:57.033778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029419+0000","last_change":"2024-04-25T09:58:57.029419+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029419+0000","last_undegraded":"2024-04-25T09:58:57.029419+0000","last_fullsized":"2024-04-25T09:58:57.029419+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033181+0000","last_change":"2024-04-25T09:58:57.033181+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033181+0000","last_undegraded":"2024-04-25T09:58:57.033181+0000","last_fullsized":"2024-04-25T09:58:57.033181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.037221+0000","last_change":"2024-04-25T09:58:57.037221+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.037221+0000","last_undegraded":"2024-04-25T09:58:57.037221+0000","last_fullsized":"2024-04-25T09:58:57.037221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.037999+0000","last_change":"2024-04-25T09:58:57.037999+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.037999+0000","last_undegraded":"2024-04-25T09:58:57.037999+0000","last_fullsized":"2024-04-25T09:58:57.037999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029302+0000","last_change":"2024-04-25T09:58:57.029302+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029302+0000","last_undegraded":"2024-04-25T09:58:57.029302+0000","last_fullsized":"2024-04-25T09:58:57.029302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028752+0000","last_change":"2024-04-25T09:58:57.028752+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028752+0000","last_undegraded":"2024-04-25T09:58:57.028752+0000","last_fullsized":"2024-04-25T09:58:57.028752+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032721+0000","last_change":"2024-04-25T09:58:57.032721+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032721+0000","last_undegraded":"2024-04-25T09:58:57.032721+0000","last_fullsized":"2024-04-25T09:58:57.032721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029156+0000","last_change":"2024-04-25T09:58:57.029156+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029156+0000","last_undegraded":"2024-04-25T09:58:57.029156+0000","last_fullsized":"2024-04-25T09:58:57.029156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028607+0000","last_change":"2024-04-25T09:58:57.028607+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028607+0000","last_undegraded":"2024-04-25T09:58:57.028607+0000","last_fullsized":"2024-04-25T09:58:57.028607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032825+0000","last_change":"2024-04-25T09:58:57.032825+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032825+0000","last_undegraded":"2024-04-25T09:58:57.032825+0000","last_fullsized":"2024-04-25T09:58:57.032825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028484+0000","last_change":"2024-04-25T09:58:57.028484+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028484+0000","last_undegraded":"2024-04-25T09:58:57.028484+0000","last_fullsized":"2024-04-25T09:58:57.028484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033413+0000","last_change":"2024-04-25T09:58:57.033413+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033413+0000","last_undegraded":"2024-04-25T09:58:57.033413+0000","last_fullsized":"2024-04-25T09:58:57.033413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029507+0000","last_change":"2024-04-25T09:58:57.029507+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029507+0000","last_undegraded":"2024-04-25T09:58:57.029507+0000","last_fullsized":"2024-04-25T09:58:57.029507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032369+0000","last_change":"2024-04-25T09:58:57.032369+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032369+0000","last_undegraded":"2024-04-25T09:58:57.032369+0000","last_fullsized":"2024-04-25T09:58:57.032369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032287+0000","last_change":"2024-04-25T09:58:57.032287+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032287+0000","last_undegraded":"2024-04-25T09:58:57.032287+0000","last_fullsized":"2024-04-25T09:58:57.032287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024530+0000","last_change":"2024-04-25T09:58:57.024530+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024530+0000","last_undegraded":"2024-04-25T09:58:57.024530+0000","last_fullsized":"2024-04-25T09:58:57.024530+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026740+0000","last_change":"2024-04-25T09:58:57.026740+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026740+0000","last_undegraded":"2024-04-25T09:58:57.026740+0000","last_fullsized":"2024-04-25T09:58:57.026740+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032961+0000","last_change":"2024-04-25T09:58:57.032961+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032961+0000","last_undegraded":"2024-04-25T09:58:57.032961+0000","last_fullsized":"2024-04-25T09:58:57.032961+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032779+0000","last_change":"2024-04-25T09:58:57.032779+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032779+0000","last_undegraded":"2024-04-25T09:58:57.032779+0000","last_fullsized":"2024-04-25T09:58:57.032779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.036398+0000","last_change":"2024-04-25T09:58:57.036398+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.036398+0000","last_undegraded":"2024-04-25T09:58:57.036398+0000","last_fullsized":"2024-04-25T09:58:57.036398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046934+0000","last_change":"2024-04-25T09:58:57.043628+0000","last_active":"2024-04-25T09:58:57.046934+0000","last_peered":"2024-04-25T09:58:57.046934+0000","last_clean":"2024-04-25T09:58:57.046934+0000","last_became_active":"2024-04-25T09:58:57.043322+0000","last_became_peered":"2024-04-25T09:58:57.043322+0000","last_unstale":"2024-04-25T09:58:57.046934+0000","last_undegraded":"2024-04-25T09:58:57.046934+0000","last_fullsized":"2024-04-25T09:58:57.046934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:56.130588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.038701+0000","last_change":"2024-04-25T09:58:57.038701+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.038701+0000","last_undegraded":"2024-04-25T09:58:57.038701+0000","last_fullsized":"2024-04-25T09:58:57.038701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046492+0000","last_change":"2024-04-25T09:58:57.041488+0000","last_active":"2024-04-25T09:58:57.046492+0000","last_peered":"2024-04-25T09:58:57.046492+0000","last_clean":"2024-04-25T09:58:57.046492+0000","last_became_active":"2024-04-25T09:58:57.041117+0000","last_became_peered":"2024-04-25T09:58:57.041117+0000","last_unstale":"2024-04-25T09:58:57.046492+0000","last_undegraded":"2024-04-25T09:58:57.046492+0000","last_fullsized":"2024-04-25T09:58:57.046492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:59.594081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024524+0000","last_change":"2024-04-25T09:58:57.024524+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024524+0000","last_undegraded":"2024-04-25T09:58:57.024524+0000","last_fullsized":"2024-04-25T09:58:57.024524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.038904+0000","last_change":"2024-04-25T09:58:57.034001+0000","last_active":"2024-04-25T09:58:57.038904+0000","last_peered":"2024-04-25T09:58:57.038904+0000","last_clean":"2024-04-25T09:58:57.038904+0000","last_became_active":"2024-04-25T09:58:57.033646+0000","last_became_peered":"2024-04-25T09:58:57.033646+0000","last_unstale":"2024-04-25T09:58:57.038904+0000","last_undegraded":"2024-04-25T09:58:57.038904+0000","last_fullsized":"2024-04-25T09:58:57.038904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:20.730136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.035703+0000","last_change":"2024-04-25T09:58:57.035703+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.035703+0000","last_undegraded":"2024-04-25T09:58:57.035703+0000","last_fullsized":"2024-04-25T09:58:57.035703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047225+0000","last_change":"2024-04-25T09:58:57.044955+0000","last_active":"2024-04-25T09:58:57.047225+0000","last_peered":"2024-04-25T09:58:57.047225+0000","last_clean":"2024-04-25T09:58:57.047225+0000","last_became_active":"2024-04-25T09:58:57.044389+0000","last_became_peered":"2024-04-25T09:58:57.044389+0000","last_unstale":"2024-04-25T09:58:57.047225+0000","last_undegraded":"2024-04-25T09:58:57.047225+0000","last_fullsized":"2024-04-25T09:58:57.047225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:55.535171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031396+0000","last_change":"2024-04-25T09:58:57.031396+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031396+0000","last_undegraded":"2024-04-25T09:58:57.031396+0000","last_fullsized":"2024-04-25T09:58:57.031396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.038695+0000","last_change":"2024-04-25T09:58:57.034395+0000","last_active":"2024-04-25T09:58:57.038695+0000","last_peered":"2024-04-25T09:58:57.038695+0000","last_clean":"2024-04-25T09:58:57.038695+0000","last_became_active":"2024-04-25T09:58:57.034079+0000","last_became_peered":"2024-04-25T09:58:57.034079+0000","last_unstale":"2024-04-25T09:58:57.038695+0000","last_undegraded":"2024-04-25T09:58:57.038695+0000","last_fullsized":"2024-04-25T09:58:57.038695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:03.751385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032373+0000","last_change":"2024-04-25T09:58:57.032373+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032373+0000","last_undegraded":"2024-04-25T09:58:57.032373+0000","last_fullsized":"2024-04-25T09:58:57.032373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043740+0000","last_change":"2024-04-25T09:58:57.038771+0000","last_active":"2024-04-25T09:58:57.043740+0000","last_peered":"2024-04-25T09:58:57.043740+0000","last_clean":"2024-04-25T09:58:57.043740+0000","last_became_active":"2024-04-25T09:58:57.038480+0000","last_became_peered":"2024-04-25T09:58:57.038480+0000","last_unstale":"2024-04-25T09:58:57.043740+0000","last_undegraded":"2024-04-25T09:58:57.043740+0000","last_fullsized":"2024-04-25T09:58:57.043740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:58.574435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033026+0000","last_change":"2024-04-25T09:58:57.033026+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033026+0000","last_undegraded":"2024-04-25T09:58:57.033026+0000","last_fullsized":"2024-04-25T09:58:57.033026+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.035188+0000","last_change":"2024-04-25T09:58:57.034137+0000","last_active":"2024-04-25T09:58:57.035188+0000","last_peered":"2024-04-25T09:58:57.035188+0000","last_clean":"2024-04-25T09:58:57.035188+0000","last_became_active":"2024-04-25T09:58:57.033509+0000","last_became_peered":"2024-04-25T09:58:57.033509+0000","last_unstale":"2024-04-25T09:58:57.035188+0000","last_undegraded":"2024-04-25T09:58:57.035188+0000","last_fullsized":"2024-04-25T09:58:57.035188+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:51.854104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031838+0000","last_change":"2024-04-25T09:58:57.031838+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031838+0000","last_undegraded":"2024-04-25T09:58:57.031838+0000","last_fullsized":"2024-04-25T09:58:57.031838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047142+0000","last_change":"2024-04-25T09:58:57.045529+0000","last_active":"2024-04-25T09:58:57.047142+0000","last_peered":"2024-04-25T09:58:57.047142+0000","last_clean":"2024-04-25T09:58:57.047142+0000","last_became_active":"2024-04-25T09:58:57.045229+0000","last_became_peered":"2024-04-25T09:58:57.045229+0000","last_unstale":"2024-04-25T09:58:57.047142+0000","last_undegraded":"2024-04-25T09:58:57.047142+0000","last_fullsized":"2024-04-25T09:58:57.047142+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:34.875415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026908+0000","last_change":"2024-04-25T09:58:57.026908+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026908+0000","last_undegraded":"2024-04-25T09:58:57.026908+0000","last_fullsized":"2024-04-25T09:58:57.026908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.041384+0000","last_change":"2024-04-25T09:58:57.039169+0000","last_active":"2024-04-25T09:58:57.041384+0000","last_peered":"2024-04-25T09:58:57.041384+0000","last_clean":"2024-04-25T09:58:57.041384+0000","last_became_active":"2024-04-25T09:58:57.038892+0000","last_became_peered":"2024-04-25T09:58:57.038892+0000","last_unstale":"2024-04-25T09:58:57.041384+0000","last_undegraded":"2024-04-25T09:58:57.041384+0000","last_fullsized":"2024-04-25T09:58:57.041384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:58.402797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024520+0000","last_change":"2024-04-25T09:58:57.024520+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024520+0000","last_undegraded":"2024-04-25T09:58:57.024520+0000","last_fullsized":"2024-04-25T09:58:57.024520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.041964+0000","last_change":"2024-04-25T09:58:57.040183+0000","last_active":"2024-04-25T09:58:57.041964+0000","last_peered":"2024-04-25T09:58:57.041964+0000","last_clean":"2024-04-25T09:58:57.041964+0000","last_became_active":"2024-04-25T09:58:57.039924+0000","last_became_peered":"2024-04-25T09:58:57.039924+0000","last_unstale":"2024-04-25T09:58:57.041964+0000","last_undegraded":"2024-04-25T09:58:57.041964+0000","last_fullsized":"2024-04-25T09:58:57.041964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:22.066654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028488+0000","last_change":"2024-04-25T09:58:57.028488+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028488+0000","last_undegraded":"2024-04-25T09:58:57.028488+0000","last_fullsized":"2024-04-25T09:58:57.028488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047039+0000","last_change":"2024-04-25T09:58:57.045075+0000","last_active":"2024-04-25T09:58:57.047039+0000","last_peered":"2024-04-25T09:58:57.047039+0000","last_clean":"2024-04-25T09:58:57.047039+0000","last_became_active":"2024-04-25T09:58:57.044812+0000","last_became_peered":"2024-04-25T09:58:57.044812+0000","last_unstale":"2024-04-25T09:58:57.047039+0000","last_undegraded":"2024-04-25T09:58:57.047039+0000","last_fullsized":"2024-04-25T09:58:57.047039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:51.303697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.030942+0000","last_change":"2024-04-25T09:58:57.030942+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.030942+0000","last_undegraded":"2024-04-25T09:58:57.030942+0000","last_fullsized":"2024-04-25T09:58:57.030942+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047888+0000","last_change":"2024-04-25T09:58:57.044690+0000","last_active":"2024-04-25T09:58:57.047888+0000","last_peered":"2024-04-25T09:58:57.047888+0000","last_clean":"2024-04-25T09:58:57.047888+0000","last_became_active":"2024-04-25T09:58:57.044405+0000","last_became_peered":"2024-04-25T09:58:57.044405+0000","last_unstale":"2024-04-25T09:58:57.047888+0000","last_undegraded":"2024-04-25T09:58:57.047888+0000","last_fullsized":"2024-04-25T09:58:57.047888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:03.790236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.030477+0000","last_change":"2024-04-25T09:58:57.030477+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.030477+0000","last_undegraded":"2024-04-25T09:58:57.030477+0000","last_fullsized":"2024-04-25T09:58:57.030477+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047136+0000","last_change":"2024-04-25T09:58:57.045507+0000","last_active":"2024-04-25T09:58:57.047136+0000","last_peered":"2024-04-25T09:58:57.047136+0000","last_clean":"2024-04-25T09:58:57.047136+0000","last_became_active":"2024-04-25T09:58:57.045211+0000","last_became_peered":"2024-04-25T09:58:57.045211+0000","last_unstale":"2024-04-25T09:58:57.047136+0000","last_undegraded":"2024-04-25T09:58:57.047136+0000","last_fullsized":"2024-04-25T09:58:57.047136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:03.790236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026747+0000","last_change":"2024-04-25T09:58:57.026747+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026747+0000","last_undegraded":"2024-04-25T09:58:57.026747+0000","last_fullsized":"2024-04-25T09:58:57.026747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043240+0000","last_change":"2024-04-25T09:58:57.042334+0000","last_active":"2024-04-25T09:58:57.043240+0000","last_peered":"2024-04-25T09:58:57.043240+0000","last_clean":"2024-04-25T09:58:57.043240+0000","last_became_active":"2024-04-25T09:58:57.042071+0000","last_became_peered":"2024-04-25T09:58:57.042071+0000","last_unstale":"2024-04-25T09:58:57.043240+0000","last_undegraded":"2024-04-25T09:58:57.043240+0000","last_fullsized":"2024-04-25T09:58:57.043240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:35.349212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031415+0000","last_change":"2024-04-25T09:58:57.031415+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031415+0000","last_undegraded":"2024-04-25T09:58:57.031415+0000","last_fullsized":"2024-04-25T09:58:57.031415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046805+0000","last_change":"2024-04-25T09:58:57.046367+0000","last_active":"2024-04-25T09:58:57.046805+0000","last_peered":"2024-04-25T09:58:57.046805+0000","last_clean":"2024-04-25T09:58:57.046805+0000","last_became_active":"2024-04-25T09:58:57.046089+0000","last_became_peered":"2024-04-25T09:58:57.046089+0000","last_unstale":"2024-04-25T09:58:57.046805+0000","last_undegraded":"2024-04-25T09:58:57.046805+0000","last_fullsized":"2024-04-25T09:58:57.046805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:20.908778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032127+0000","last_change":"2024-04-25T09:58:57.032127+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032127+0000","last_undegraded":"2024-04-25T09:58:57.032127+0000","last_fullsized":"2024-04-25T09:58:57.032127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043706+0000","last_change":"2024-04-25T09:58:57.038473+0000","last_active":"2024-04-25T09:58:57.043706+0000","last_peered":"2024-04-25T09:58:57.043706+0000","last_clean":"2024-04-25T09:58:57.043706+0000","last_became_active":"2024-04-25T09:58:57.038170+0000","last_became_peered":"2024-04-25T09:58:57.038170+0000","last_unstale":"2024-04-25T09:58:57.043706+0000","last_undegraded":"2024-04-25T09:58:57.043706+0000","last_fullsized":"2024-04-25T09:58:57.043706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:51.303697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.035966+0000","last_change":"2024-04-25T09:58:57.035966+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.035966+0000","last_undegraded":"2024-04-25T09:58:57.035966+0000","last_fullsized":"2024-04-25T09:58:57.035966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043997+0000","last_change":"2024-04-25T09:58:57.040136+0000","last_active":"2024-04-25T09:58:57.043997+0000","last_peered":"2024-04-25T09:58:57.043997+0000","last_clean":"2024-04-25T09:58:57.043997+0000","last_became_active":"2024-04-25T09:58:57.039832+0000","last_became_peered":"2024-04-25T09:58:57.039832+0000","last_unstale":"2024-04-25T09:58:57.043997+0000","last_undegraded":"2024-04-25T09:58:57.043997+0000","last_fullsized":"2024-04-25T09:58:57.043997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:44.391697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031726+0000","last_change":"2024-04-25T09:58:57.031726+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031726+0000","last_undegraded":"2024-04-25T09:58:57.031726+0000","last_fullsized":"2024-04-25T09:58:57.031726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.041588+0000","last_change":"2024-04-25T09:58:57.039566+0000","last_active":"2024-04-25T09:58:57.041588+0000","last_peered":"2024-04-25T09:58:57.041588+0000","last_clean":"2024-04-25T09:58:57.041588+0000","last_became_active":"2024-04-25T09:58:57.039279+0000","last_became_peered":"2024-04-25T09:58:57.039279+0000","last_unstale":"2024-04-25T09:58:57.041588+0000","last_undegraded":"2024-04-25T09:58:57.041588+0000","last_fullsized":"2024-04-25T09:58:57.041588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:35.349212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028874+0000","last_change":"2024-04-25T09:58:57.028874+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028874+0000","last_undegraded":"2024-04-25T09:58:57.028874+0000","last_fullsized":"2024-04-25T09:58:57.028874+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046543+0000","last_change":"2024-04-25T09:58:57.043428+0000","last_active":"2024-04-25T09:58:57.046543+0000","last_peered":"2024-04-25T09:58:57.046543+0000","last_clean":"2024-04-25T09:58:57.046543+0000","last_became_active":"2024-04-25T09:58:57.043158+0000","last_became_peered":"2024-04-25T09:58:57.043158+0000","last_unstale":"2024-04-25T09:58:57.046543+0000","last_undegraded":"2024-04-25T09:58:57.046543+0000","last_fullsized":"2024-04-25T09:58:57.046543+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:50.057945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032298+0000","last_change":"2024-04-25T09:58:57.032298+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032298+0000","last_undegraded":"2024-04-25T09:58:57.032298+0000","last_fullsized":"2024-04-25T09:58:57.032298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.039320+0000","last_change":"2024-04-25T09:58:57.036704+0000","last_active":"2024-04-25T09:58:57.039320+0000","last_peered":"2024-04-25T09:58:57.039320+0000","last_clean":"2024-04-25T09:58:57.039320+0000","last_became_active":"2024-04-25T09:58:57.036421+0000","last_became_peered":"2024-04-25T09:58:57.036421+0000","last_unstale":"2024-04-25T09:58:57.039320+0000","last_undegraded":"2024-04-25T09:58:57.039320+0000","last_fullsized":"2024-04-25T09:58:57.039320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:23.461435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032498+0000","last_change":"2024-04-25T09:58:57.032498+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032498+0000","last_undegraded":"2024-04-25T09:58:57.032498+0000","last_fullsized":"2024-04-25T09:58:57.032498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.042350+0000","last_change":"2024-04-25T09:58:57.037903+0000","last_active":"2024-04-25T09:58:57.042350+0000","last_peered":"2024-04-25T09:58:57.042350+0000","last_clean":"2024-04-25T09:58:57.042350+0000","last_became_active":"2024-04-25T09:58:57.037640+0000","last_became_peered":"2024-04-25T09:58:57.037640+0000","last_unstale":"2024-04-25T09:58:57.042350+0000","last_undegraded":"2024-04-25T09:58:57.042350+0000","last_fullsized":"2024-04-25T09:58:57.042350+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:04.149145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031438+0000","last_change":"2024-04-25T09:58:57.031438+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031438+0000","last_undegraded":"2024-04-25T09:58:57.031438+0000","last_fullsized":"2024-04-25T09:58:57.031438+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046749+0000","last_change":"2024-04-25T09:58:57.043679+0000","last_active":"2024-04-25T09:58:57.046749+0000","last_peered":"2024-04-25T09:58:57.046749+0000","last_clean":"2024-04-25T09:58:57.046749+0000","last_became_active":"2024-04-25T09:58:57.043368+0000","last_became_peered":"2024-04-25T09:58:57.043368+0000","last_unstale":"2024-04-25T09:58:57.046749+0000","last_undegraded":"2024-04-25T09:58:57.046749+0000","last_fullsized":"2024-04-25T09:58:57.046749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:54.216572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.035284+0000","last_change":"2024-04-25T09:58:57.035284+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.035284+0000","last_undegraded":"2024-04-25T09:58:57.035284+0000","last_fullsized":"2024-04-25T09:58:57.035284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.042343+0000","last_change":"2024-04-25T09:58:57.037357+0000","last_active":"2024-04-25T09:58:57.042343+0000","last_peered":"2024-04-25T09:58:57.042343+0000","last_clean":"2024-04-25T09:58:57.042343+0000","last_became_active":"2024-04-25T09:58:57.037087+0000","last_became_peered":"2024-04-25T09:58:57.037087+0000","last_unstale":"2024-04-25T09:58:57.042343+0000","last_undegraded":"2024-04-25T09:58:57.042343+0000","last_fullsized":"2024-04-25T09:58:57.042343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:58.457158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032863+0000","last_change":"2024-04-25T09:58:57.032863+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032863+0000","last_undegraded":"2024-04-25T09:58:57.032863+0000","last_fullsized":"2024-04-25T09:58:57.032863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.044146+0000","last_change":"2024-04-25T09:58:57.040746+0000","last_active":"2024-04-25T09:58:57.044146+0000","last_peered":"2024-04-25T09:58:57.044146+0000","last_clean":"2024-04-25T09:58:57.044146+0000","last_became_active":"2024-04-25T09:58:57.040447+0000","last_became_peered":"2024-04-25T09:58:57.040447+0000","last_unstale":"2024-04-25T09:58:57.044146+0000","last_undegraded":"2024-04-25T09:58:57.044146+0000","last_fullsized":"2024-04-25T09:58:57.044146+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:04.360841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.038293+0000","last_change":"2024-04-25T09:58:57.038293+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.038293+0000","last_undegraded":"2024-04-25T09:58:57.038293+0000","last_fullsized":"2024-04-25T09:58:57.038293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.044039+0000","last_change":"2024-04-25T09:58:57.039890+0000","last_active":"2024-04-25T09:58:57.044039+0000","last_peered":"2024-04-25T09:58:57.044039+0000","last_clean":"2024-04-25T09:58:57.044039+0000","last_became_active":"2024-04-25T09:58:57.039428+0000","last_became_peered":"2024-04-25T09:58:57.039428+0000","last_unstale":"2024-04-25T09:58:57.044039+0000","last_undegraded":"2024-04-25T09:58:57.044039+0000","last_fullsized":"2024-04-25T09:58:57.044039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:34.094773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031608+0000","last_change":"2024-04-25T09:58:57.031608+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031608+0000","last_undegraded":"2024-04-25T09:58:57.031608+0000","last_fullsized":"2024-04-25T09:58:57.031608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047026+0000","last_change":"2024-04-25T09:58:57.045052+0000","last_active":"2024-04-25T09:58:57.047026+0000","last_peered":"2024-04-25T09:58:57.047026+0000","last_clean":"2024-04-25T09:58:57.047026+0000","last_became_active":"2024-04-25T09:58:57.044719+0000","last_became_peered":"2024-04-25T09:58:57.044719+0000","last_unstale":"2024-04-25T09:58:57.047026+0000","last_undegraded":"2024-04-25T09:58:57.047026+0000","last_fullsized":"2024-04-25T09:58:57.047026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:20.749313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032975+0000","last_change":"2024-04-25T09:58:57.032975+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032975+0000","last_undegraded":"2024-04-25T09:58:57.032975+0000","last_fullsized":"2024-04-25T09:58:57.032975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046867+0000","last_change":"2024-04-25T09:58:57.042212+0000","last_active":"2024-04-25T09:58:57.046867+0000","last_peered":"2024-04-25T09:58:57.046867+0000","last_clean":"2024-04-25T09:58:57.046867+0000","last_became_active":"2024-04-25T09:58:57.041931+0000","last_became_peered":"2024-04-25T09:58:57.041931+0000","last_unstale":"2024-04-25T09:58:57.046867+0000","last_undegraded":"2024-04-25T09:58:57.046867+0000","last_fullsized":"2024-04-25T09:58:57.046867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:39.639654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.030927+0000","last_change":"2024-04-25T09:58:57.030927+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.030927+0000","last_undegraded":"2024-04-25T09:58:57.030927+0000","last_fullsized":"2024-04-25T09:58:57.030927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.045305+0000","last_change":"2024-04-25T09:58:57.043114+0000","last_active":"2024-04-25T09:58:57.045305+0000","last_peered":"2024-04-25T09:58:57.045305+0000","last_clean":"2024-04-25T09:58:57.045305+0000","last_became_active":"2024-04-25T09:58:57.042800+0000","last_became_peered":"2024-04-25T09:58:57.042800+0000","last_unstale":"2024-04-25T09:58:57.045305+0000","last_undegraded":"2024-04-25T09:58:57.045305+0000","last_fullsized":"2024-04-25T09:58:57.045305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:04.360841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028484+0000","last_change":"2024-04-25T09:58:57.028484+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028484+0000","last_undegraded":"2024-04-25T09:58:57.028484+0000","last_fullsized":"2024-04-25T09:58:57.028484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.039332+0000","last_change":"2024-04-25T09:58:57.036703+0000","last_active":"2024-04-25T09:58:57.039332+0000","last_peered":"2024-04-25T09:58:57.039332+0000","last_clean":"2024-04-25T09:58:57.039332+0000","last_became_active":"2024-04-25T09:58:57.036409+0000","last_became_peered":"2024-04-25T09:58:57.036409+0000","last_unstale":"2024-04-25T09:58:57.039332+0000","last_undegraded":"2024-04-25T09:58:57.039332+0000","last_fullsized":"2024-04-25T09:58:57.039332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:26.837285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031019+0000","last_change":"2024-04-25T09:58:57.031019+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031019+0000","last_undegraded":"2024-04-25T09:58:57.031019+0000","last_fullsized":"2024-04-25T09:58:57.031019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.044055+0000","last_change":"2024-04-25T09:58:57.039451+0000","last_active":"2024-04-25T09:58:57.044055+0000","last_peered":"2024-04-25T09:58:57.044055+0000","last_clean":"2024-04-25T09:58:57.044055+0000","last_became_active":"2024-04-25T09:58:57.039193+0000","last_became_peered":"2024-04-25T09:58:57.039193+0000","last_unstale":"2024-04-25T09:58:57.044055+0000","last_undegraded":"2024-04-25T09:58:57.044055+0000","last_fullsized":"2024-04-25T09:58:57.044055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:35.986283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032671+0000","last_change":"2024-04-25T09:58:57.032671+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032671+0000","last_undegraded":"2024-04-25T09:58:57.032671+0000","last_fullsized":"2024-04-25T09:58:57.032671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.039001+0000","last_change":"2024-04-25T09:58:57.035195+0000","last_active":"2024-04-25T09:58:57.039001+0000","last_peered":"2024-04-25T09:58:57.039001+0000","last_clean":"2024-04-25T09:58:57.039001+0000","last_became_active":"2024-04-25T09:58:57.034665+0000","last_became_peered":"2024-04-25T09:58:57.034665+0000","last_unstale":"2024-04-25T09:58:57.039001+0000","last_undegraded":"2024-04-25T09:58:57.039001+0000","last_fullsized":"2024-04-25T09:58:57.039001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:03.668577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024524+0000","last_change":"2024-04-25T09:58:57.024524+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024524+0000","last_undegraded":"2024-04-25T09:58:57.024524+0000","last_fullsized":"2024-04-25T09:58:57.024524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.045377+0000","last_change":"2024-04-25T09:58:57.042156+0000","last_active":"2024-04-25T09:58:57.045377+0000","last_peered":"2024-04-25T09:58:57.045377+0000","last_clean":"2024-04-25T09:58:57.045377+0000","last_became_active":"2024-04-25T09:58:57.040423+0000","last_became_peered":"2024-04-25T09:58:57.040423+0000","last_unstale":"2024-04-25T09:58:57.045377+0000","last_undegraded":"2024-04-25T09:58:57.045377+0000","last_fullsized":"2024-04-25T09:58:57.045377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:27.846107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":293,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.025542+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:58:57.025542+0000","last_peered":"2024-04-25T09:58:57.025542+0000","last_clean":"2024-04-25T09:58:57.025542+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:58:57.025542+0000","last_undegraded":"2024-04-25T09:58:57.025542+0000","last_fullsized":"2024-04-25T09:58:57.025542+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1667072,"data_stored":1079965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1667072,"data_stored":1079965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:59:01.330 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:59:01.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.460+0000 7fc4d042a640 1 Processor -- start 2024-04-25T09:59:01.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.460+0000 7fc4d042a640 1 -- start start 2024-04-25T09:59:01.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.461+0000 7fc4d042a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4c8072990 0x7fc4c80b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.461+0000 7fc4d042a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c815e500 con 0x7fc4c815aa00 2024-04-25T09:59:01.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.461+0000 7fc4d042a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c80b5660 con 0x7fc4c80b5110 2024-04-25T09:59:01.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.461+0000 7fc4d042a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80b5c20 con 0x7fc4c8072990 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.461+0000 7fc4ce1c6640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc4c815aa00 0x7fc4c815ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53542/0 (socket says 172.21.15.139:53542) 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.461+0000 7fc4ce1c6640 1 -- 172.21.15.139:0/3166079017 learned_addr learned my addr 172.21.15.139:0/3166079017 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4ce9c7640 1 --2- 172.21.15.139:0/3166079017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4c8072990 0x7fc4c80b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477156948 0 0) 0x7fc4c815e500 con 0x7fc4c815aa00 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4b4003520 con 0x7fc4c815aa00 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 165694517 0 0) 0x7fc4b4003520 con 0x7fc4c815aa00 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4c8072990 msgr2=0x7fc4c80b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 --2- 172.21.15.139:0/3166079017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4c8072990 0x7fc4c80b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 >> v1:172.21.15.150:6789/0 conn(0x7fc4c80b5110 legacy=0x7fc4c8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c80b5940 con 0x7fc4c815aa00 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fc4c40019f0 con 0x7fc4c815aa00 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 >> v1:172.21.15.139:6789/0 conn(0x7fc4c815aa00 legacy=0x7fc4c815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 --2- 172.21.15.139:0/3166079017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b40039e0 0x7fc4b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 --2- 172.21.15.139:0/3166079017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4b4004390 0x7fc4b4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 --2- 172.21.15.139:0/3166079017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4c8072990 0x7fc4b4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80b5c20 con 0x7fc4b40039e0 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b400baa0 con 0x7fc4b4004390 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b400bca0 con 0x7fc4c8072990 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4ce1c6640 1 --2- 172.21.15.139:0/3166079017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b40039e0 0x7fc4b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4cd9c5640 1 --2- 172.21.15.139:0/3166079017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4b4004390 0x7fc4b4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4ce9c7640 1 --2- 172.21.15.139:0/3166079017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4c8072990 0x7fc4b4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4ce1c6640 1 -- 172.21.15.139:0/3166079017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4c8072990 msgr2=0x7fc4b4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4ce1c6640 1 --2- 172.21.15.139:0/3166079017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4c8072990 0x7fc4b4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4ce1c6640 1 -- 172.21.15.139:0/3166079017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4b4004390 msgr2=0x7fc4b4004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4ce1c6640 1 --2- 172.21.15.139:0/3166079017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4b4004390 0x7fc4b4004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.462+0000 7fc4ce1c6640 1 -- 172.21.15.139:0/3166079017 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b4003520 con 0x7fc4b40039e0 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.463+0000 7fc4ce1c6640 1 --2- 172.21.15.139:0/3166079017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b40039e0 0x7fc4b4003dc0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400a920 tx=0x7fc4b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c6fe594baf1160 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.463+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400a4c0 con 0x7fc4b40039e0 2024-04-25T09:59:01.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.463+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c402f070 con 0x7fc4b40039e0 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.463+0000 7fc4cd1c4640 1 -- 172.21.15.139:0/3166079017 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c402bd00 con 0x7fc4b40039e0 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.464+0000 7fc4d042a640 1 -- 172.21.15.139:0/3166079017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b40039e0 msgr2=0x7fc4b4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.464+0000 7fc4d042a640 1 --2- 172.21.15.139:0/3166079017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4b40039e0 0x7fc4b4003dc0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400a920 tx=0x7fc4b4005830 comp rx=0 tx=0).stop 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.464+0000 7fc4ce9c7640 1 -- 172.21.15.139:0/3166079017 reap_dead start 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.464+0000 7fc4d042a640 1 -- 172.21.15.139:0/3166079017 shutdown_connections 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.464+0000 7fc4d042a640 1 -- 172.21.15.139:0/3166079017 >> 172.21.15.139:0/3166079017 conn(0x7fc4c806e280 msgr2=0x7fc4c806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.464+0000 7fc4d042a640 1 -- 172.21.15.139:0/3166079017 shutdown_connections 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.464+0000 7fc4d042a640 1 -- 172.21.15.139:0/3166079017 wait complete. 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.465+0000 7fc4d042a640 1 Processor -- start 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.465+0000 7fc4d042a640 1 -- start start 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4d042a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4b40039e0 0x7fc4c8158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4d042a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c81521f0 0x7fc4c80a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4d042a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4c815d000 0x7fc4c815d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4d042a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80b4210 con 0x7fc4c81521f0 2024-04-25T09:59:01.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4d042a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80b4090 con 0x7fc4c815d000 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4d042a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80b4390 con 0x7fc4b40039e0 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4cd9c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c81521f0 0x7fc4c80a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4cd9c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c81521f0 0x7fc4c80a91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58274/0 (socket says 172.21.15.139:58274) 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4cd9c5640 1 -- 172.21.15.139:0/2484090053 learned_addr learned my addr 172.21.15.139:0/2484090053 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4ce9c7640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4c815d000 0x7fc4c815d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4ce1c6640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4b40039e0 0x7fc4c8158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4cd9c5640 1 -- 172.21.15.139:0/2484090053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4b40039e0 msgr2=0x7fc4c8158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4cd9c5640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4b40039e0 0x7fc4c8158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4cd9c5640 1 -- 172.21.15.139:0/2484090053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4c815d000 msgr2=0x7fc4c815d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4cd9c5640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4c815d000 0x7fc4c815d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4cd9c5640 1 -- 172.21.15.139:0/2484090053 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c8159090 con 0x7fc4c81521f0 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4cd9c5640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c81521f0 0x7fc4c80a91a0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800f7e0 tx=0x7fc4b800fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:01.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4beffd640 1 -- 172.21.15.139:0/2484090053 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80050d0 con 0x7fc4c81521f0 2024-04-25T09:59:01.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4beffd640 1 -- 172.21.15.139:0/2484090053 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8018070 con 0x7fc4c81521f0 2024-04-25T09:59:01.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4d042a640 1 -- 172.21.15.139:0/2484090053 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c8159370 con 0x7fc4c81521f0 2024-04-25T09:59:01.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.466+0000 7fc4beffd640 1 -- 172.21.15.139:0/2484090053 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80054b0 con 0x7fc4c81521f0 2024-04-25T09:59:01.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.467+0000 7fc4d042a640 1 -- 172.21.15.139:0/2484090053 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c8159930 con 0x7fc4c81521f0 2024-04-25T09:59:01.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.468+0000 7fc4d042a640 1 -- 172.21.15.139:0/2484090053 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c8074520 con 0x7fc4c81521f0 2024-04-25T09:59:01.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.468+0000 7fc4beffd640 1 -- 172.21.15.139:0/2484090053 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4b800d730 con 0x7fc4c81521f0 2024-04-25T09:59:01.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.468+0000 7fc4beffd640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc4ac06b4d0 0x7fc4ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.468+0000 7fc4beffd640 1 -- 172.21.15.139:0/2484090053 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc4b808a230 con 0x7fc4c81521f0 2024-04-25T09:59:01.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.468+0000 7fc4ce1c6640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc4ac06b4d0 0x7fc4ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.469+0000 7fc4ce1c6640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc4ac06b4d0 0x7fc4ac06d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4033000 tx=0x7fc4c4009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:01.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.473+0000 7fc4beffd640 1 -- 172.21.15.139:0/2484090053 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4b8058f20 con 0x7fc4c81521f0 2024-04-25T09:59:01.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.637+0000 7fc4d042a640 1 -- 172.21.15.139:0/2484090053 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc4c8149790 con 0x7fc4ac06b4d0 2024-04-25T09:59:01.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.643+0000 7fc4beffd640 1 -- 172.21.15.139:0/2484090053 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246767 (secure 0 0 0) 0x7fc4c40017b0 con 0x7fc4ac06b4d0 2024-04-25T09:59:01.650 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:01.651 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:59:01.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.646+0000 7fc4d042a640 1 -- 172.21.15.139:0/2484090053 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc4ac06b4d0 msgr2=0x7fc4ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.646+0000 7fc4d042a640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc4ac06b4d0 0x7fc4ac06d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4033000 tx=0x7fc4c4009030 comp rx=0 tx=0).stop 2024-04-25T09:59:01.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.646+0000 7fc4d042a640 1 -- 172.21.15.139:0/2484090053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c81521f0 msgr2=0x7fc4c80a91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.646+0000 7fc4d042a640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c81521f0 0x7fc4c80a91a0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800f7e0 tx=0x7fc4b800fbe0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.646+0000 7fc4d042a640 1 -- 172.21.15.139:0/2484090053 shutdown_connections 2024-04-25T09:59:01.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.646+0000 7fc4d042a640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c81521f0 0x7fc4c80a91a0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.646+0000 7fc4d042a640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc4ac06b4d0 0x7fc4ac06d990 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.646+0000 7fc4d042a640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4c815d000 0x7fc4c815d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.646+0000 7fc4d042a640 1 --2- 172.21.15.139:0/2484090053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4b40039e0 0x7fc4c8158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.646+0000 7fc4d042a640 1 -- 172.21.15.139:0/2484090053 >> 172.21.15.139:0/2484090053 conn(0x7fc4c806e280 msgr2=0x7fc4c80b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:01.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.646+0000 7fc4d042a640 1 -- 172.21.15.139:0/2484090053 shutdown_connections 2024-04-25T09:59:01.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.646+0000 7fc4d042a640 1 -- 172.21.15.139:0/2484090053 wait complete. 2024-04-25T09:59:01.661 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-04-25T09:59:01.044642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259032,"kb_used_data":9544,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715688,"statfs":{"total":773094113280,"available":772828864512,"internally_reserved":0,"allocated":9773056,"data_stored":5097512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.031004"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029464+0000","last_change":"2024-04-25T09:58:57.029464+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029464+0000","last_undegraded":"2024-04-25T09:58:57.029464+0000","last_fullsized":"2024-04-25T09:58:57.029464+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.034310+0000","last_change":"2024-04-25T09:58:57.034310+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.034310+0000","last_undegraded":"2024-04-25T09:58:57.034310+0000","last_fullsized":"2024-04-25T09:58:57.034310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029376+0000","last_change":"2024-04-25T09:58:57.029376+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029376+0000","last_undegraded":"2024-04-25T09:58:57.029376+0000","last_fullsized":"2024-04-25T09:58:57.029376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.034971+0000","last_change":"2024-04-25T09:58:57.034971+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.034971+0000","last_undegraded":"2024-04-25T09:58:57.034971+0000","last_fullsized":"2024-04-25T09:58:57.034971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028683+0000","last_change":"2024-04-25T09:58:57.028683+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028683+0000","last_undegraded":"2024-04-25T09:58:57.028683+0000","last_fullsized":"2024-04-25T09:58:57.028683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.036041+0000","last_change":"2024-04-25T09:58:57.036041+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.036041+0000","last_undegraded":"2024-04-25T09:58:57.036041+0000","last_fullsized":"2024-04-25T09:58:57.036041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024527+0000","last_change":"2024-04-25T09:58:57.024527+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024527+0000","last_undegraded":"2024-04-25T09:58:57.024527+0000","last_fullsized":"2024-04-25T09:58:57.024527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.034283+0000","last_change":"2024-04-25T09:58:57.034283+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.034283+0000","last_undegraded":"2024-04-25T09:58:57.034283+0000","last_fullsized":"2024-04-25T09:58:57.034283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026906+0000","last_change":"2024-04-25T09:58:57.026906+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026906+0000","last_undegraded":"2024-04-25T09:58:57.026906+0000","last_fullsized":"2024-04-25T09:58:57.026906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032805+0000","last_change":"2024-04-25T09:58:57.032805+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032805+0000","last_undegraded":"2024-04-25T09:58:57.032805+0000","last_fullsized":"2024-04-25T09:58:57.032805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029222+0000","last_change":"2024-04-25T09:58:57.029222+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029222+0000","last_undegraded":"2024-04-25T09:58:57.029222+0000","last_fullsized":"2024-04-25T09:58:57.029222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032544+0000","last_change":"2024-04-25T09:58:57.032544+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032544+0000","last_undegraded":"2024-04-25T09:58:57.032544+0000","last_fullsized":"2024-04-25T09:58:57.032544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033778+0000","last_change":"2024-04-25T09:58:57.033778+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033778+0000","last_undegraded":"2024-04-25T09:58:57.033778+0000","last_fullsized":"2024-04-25T09:58:57.033778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029419+0000","last_change":"2024-04-25T09:58:57.029419+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029419+0000","last_undegraded":"2024-04-25T09:58:57.029419+0000","last_fullsized":"2024-04-25T09:58:57.029419+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033181+0000","last_change":"2024-04-25T09:58:57.033181+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033181+0000","last_undegraded":"2024-04-25T09:58:57.033181+0000","last_fullsized":"2024-04-25T09:58:57.033181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.037221+0000","last_change":"2024-04-25T09:58:57.037221+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.037221+0000","last_undegraded":"2024-04-25T09:58:57.037221+0000","last_fullsized":"2024-04-25T09:58:57.037221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.037999+0000","last_change":"2024-04-25T09:58:57.037999+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.037999+0000","last_undegraded":"2024-04-25T09:58:57.037999+0000","last_fullsized":"2024-04-25T09:58:57.037999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029302+0000","last_change":"2024-04-25T09:58:57.029302+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029302+0000","last_undegraded":"2024-04-25T09:58:57.029302+0000","last_fullsized":"2024-04-25T09:58:57.029302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028752+0000","last_change":"2024-04-25T09:58:57.028752+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028752+0000","last_undegraded":"2024-04-25T09:58:57.028752+0000","last_fullsized":"2024-04-25T09:58:57.028752+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032721+0000","last_change":"2024-04-25T09:58:57.032721+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032721+0000","last_undegraded":"2024-04-25T09:58:57.032721+0000","last_fullsized":"2024-04-25T09:58:57.032721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029156+0000","last_change":"2024-04-25T09:58:57.029156+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029156+0000","last_undegraded":"2024-04-25T09:58:57.029156+0000","last_fullsized":"2024-04-25T09:58:57.029156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028607+0000","last_change":"2024-04-25T09:58:57.028607+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028607+0000","last_undegraded":"2024-04-25T09:58:57.028607+0000","last_fullsized":"2024-04-25T09:58:57.028607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032825+0000","last_change":"2024-04-25T09:58:57.032825+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032825+0000","last_undegraded":"2024-04-25T09:58:57.032825+0000","last_fullsized":"2024-04-25T09:58:57.032825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028484+0000","last_change":"2024-04-25T09:58:57.028484+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028484+0000","last_undegraded":"2024-04-25T09:58:57.028484+0000","last_fullsized":"2024-04-25T09:58:57.028484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033413+0000","last_change":"2024-04-25T09:58:57.033413+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033413+0000","last_undegraded":"2024-04-25T09:58:57.033413+0000","last_fullsized":"2024-04-25T09:58:57.033413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029507+0000","last_change":"2024-04-25T09:58:57.029507+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029507+0000","last_undegraded":"2024-04-25T09:58:57.029507+0000","last_fullsized":"2024-04-25T09:58:57.029507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032369+0000","last_change":"2024-04-25T09:58:57.032369+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032369+0000","last_undegraded":"2024-04-25T09:58:57.032369+0000","last_fullsized":"2024-04-25T09:58:57.032369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032287+0000","last_change":"2024-04-25T09:58:57.032287+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032287+0000","last_undegraded":"2024-04-25T09:58:57.032287+0000","last_fullsized":"2024-04-25T09:58:57.032287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024530+0000","last_change":"2024-04-25T09:58:57.024530+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024530+0000","last_undegraded":"2024-04-25T09:58:57.024530+0000","last_fullsized":"2024-04-25T09:58:57.024530+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026740+0000","last_change":"2024-04-25T09:58:57.026740+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026740+0000","last_undegraded":"2024-04-25T09:58:57.026740+0000","last_fullsized":"2024-04-25T09:58:57.026740+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032961+0000","last_change":"2024-04-25T09:58:57.032961+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032961+0000","last_undegraded":"2024-04-25T09:58:57.032961+0000","last_fullsized":"2024-04-25T09:58:57.032961+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032779+0000","last_change":"2024-04-25T09:58:57.032779+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032779+0000","last_undegraded":"2024-04-25T09:58:57.032779+0000","last_fullsized":"2024-04-25T09:58:57.032779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.036398+0000","last_change":"2024-04-25T09:58:57.036398+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.036398+0000","last_undegraded":"2024-04-25T09:58:57.036398+0000","last_fullsized":"2024-04-25T09:58:57.036398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046934+0000","last_change":"2024-04-25T09:58:57.043628+0000","last_active":"2024-04-25T09:58:57.046934+0000","last_peered":"2024-04-25T09:58:57.046934+0000","last_clean":"2024-04-25T09:58:57.046934+0000","last_became_active":"2024-04-25T09:58:57.043322+0000","last_became_peered":"2024-04-25T09:58:57.043322+0000","last_unstale":"2024-04-25T09:58:57.046934+0000","last_undegraded":"2024-04-25T09:58:57.046934+0000","last_fullsized":"2024-04-25T09:58:57.046934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:56.130588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.038701+0000","last_change":"2024-04-25T09:58:57.038701+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.038701+0000","last_undegraded":"2024-04-25T09:58:57.038701+0000","last_fullsized":"2024-04-25T09:58:57.038701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046492+0000","last_change":"2024-04-25T09:58:57.041488+0000","last_active":"2024-04-25T09:58:57.046492+0000","last_peered":"2024-04-25T09:58:57.046492+0000","last_clean":"2024-04-25T09:58:57.046492+0000","last_became_active":"2024-04-25T09:58:57.041117+0000","last_became_peered":"2024-04-25T09:58:57.041117+0000","last_unstale":"2024-04-25T09:58:57.046492+0000","last_undegraded":"2024-04-25T09:58:57.046492+0000","last_fullsized":"2024-04-25T09:58:57.046492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:59.594081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024524+0000","last_change":"2024-04-25T09:58:57.024524+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024524+0000","last_undegraded":"2024-04-25T09:58:57.024524+0000","last_fullsized":"2024-04-25T09:58:57.024524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.038904+0000","last_change":"2024-04-25T09:58:57.034001+0000","last_active":"2024-04-25T09:58:57.038904+0000","last_peered":"2024-04-25T09:58:57.038904+0000","last_clean":"2024-04-25T09:58:57.038904+0000","last_became_active":"2024-04-25T09:58:57.033646+0000","last_became_peered":"2024-04-25T09:58:57.033646+0000","last_unstale":"2024-04-25T09:58:57.038904+0000","last_undegraded":"2024-04-25T09:58:57.038904+0000","last_fullsized":"2024-04-25T09:58:57.038904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:20.730136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.035703+0000","last_change":"2024-04-25T09:58:57.035703+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.035703+0000","last_undegraded":"2024-04-25T09:58:57.035703+0000","last_fullsized":"2024-04-25T09:58:57.035703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047225+0000","last_change":"2024-04-25T09:58:57.044955+0000","last_active":"2024-04-25T09:58:57.047225+0000","last_peered":"2024-04-25T09:58:57.047225+0000","last_clean":"2024-04-25T09:58:57.047225+0000","last_became_active":"2024-04-25T09:58:57.044389+0000","last_became_peered":"2024-04-25T09:58:57.044389+0000","last_unstale":"2024-04-25T09:58:57.047225+0000","last_undegraded":"2024-04-25T09:58:57.047225+0000","last_fullsized":"2024-04-25T09:58:57.047225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:55.535171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031396+0000","last_change":"2024-04-25T09:58:57.031396+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031396+0000","last_undegraded":"2024-04-25T09:58:57.031396+0000","last_fullsized":"2024-04-25T09:58:57.031396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.038695+0000","last_change":"2024-04-25T09:58:57.034395+0000","last_active":"2024-04-25T09:58:57.038695+0000","last_peered":"2024-04-25T09:58:57.038695+0000","last_clean":"2024-04-25T09:58:57.038695+0000","last_became_active":"2024-04-25T09:58:57.034079+0000","last_became_peered":"2024-04-25T09:58:57.034079+0000","last_unstale":"2024-04-25T09:58:57.038695+0000","last_undegraded":"2024-04-25T09:58:57.038695+0000","last_fullsized":"2024-04-25T09:58:57.038695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:03.751385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032373+0000","last_change":"2024-04-25T09:58:57.032373+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032373+0000","last_undegraded":"2024-04-25T09:58:57.032373+0000","last_fullsized":"2024-04-25T09:58:57.032373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043740+0000","last_change":"2024-04-25T09:58:57.038771+0000","last_active":"2024-04-25T09:58:57.043740+0000","last_peered":"2024-04-25T09:58:57.043740+0000","last_clean":"2024-04-25T09:58:57.043740+0000","last_became_active":"2024-04-25T09:58:57.038480+0000","last_became_peered":"2024-04-25T09:58:57.038480+0000","last_unstale":"2024-04-25T09:58:57.043740+0000","last_undegraded":"2024-04-25T09:58:57.043740+0000","last_fullsized":"2024-04-25T09:58:57.043740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:58.574435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033026+0000","last_change":"2024-04-25T09:58:57.033026+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033026+0000","last_undegraded":"2024-04-25T09:58:57.033026+0000","last_fullsized":"2024-04-25T09:58:57.033026+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.035188+0000","last_change":"2024-04-25T09:58:57.034137+0000","last_active":"2024-04-25T09:58:57.035188+0000","last_peered":"2024-04-25T09:58:57.035188+0000","last_clean":"2024-04-25T09:58:57.035188+0000","last_became_active":"2024-04-25T09:58:57.033509+0000","last_became_peered":"2024-04-25T09:58:57.033509+0000","last_unstale":"2024-04-25T09:58:57.035188+0000","last_undegraded":"2024-04-25T09:58:57.035188+0000","last_fullsized":"2024-04-25T09:58:57.035188+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:51.854104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031838+0000","last_change":"2024-04-25T09:58:57.031838+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031838+0000","last_undegraded":"2024-04-25T09:58:57.031838+0000","last_fullsized":"2024-04-25T09:58:57.031838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047142+0000","last_change":"2024-04-25T09:58:57.045529+0000","last_active":"2024-04-25T09:58:57.047142+0000","last_peered":"2024-04-25T09:58:57.047142+0000","last_clean":"2024-04-25T09:58:57.047142+0000","last_became_active":"2024-04-25T09:58:57.045229+0000","last_became_peered":"2024-04-25T09:58:57.045229+0000","last_unstale":"2024-04-25T09:58:57.047142+0000","last_undegraded":"2024-04-25T09:58:57.047142+0000","last_fullsized":"2024-04-25T09:58:57.047142+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:34.875415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026908+0000","last_change":"2024-04-25T09:58:57.026908+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026908+0000","last_undegraded":"2024-04-25T09:58:57.026908+0000","last_fullsized":"2024-04-25T09:58:57.026908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.041384+0000","last_change":"2024-04-25T09:58:57.039169+0000","last_active":"2024-04-25T09:58:57.041384+0000","last_peered":"2024-04-25T09:58:57.041384+0000","last_clean":"2024-04-25T09:58:57.041384+0000","last_became_active":"2024-04-25T09:58:57.038892+0000","last_became_peered":"2024-04-25T09:58:57.038892+0000","last_unstale":"2024-04-25T09:58:57.041384+0000","last_undegraded":"2024-04-25T09:58:57.041384+0000","last_fullsized":"2024-04-25T09:58:57.041384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:58.402797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024520+0000","last_change":"2024-04-25T09:58:57.024520+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024520+0000","last_undegraded":"2024-04-25T09:58:57.024520+0000","last_fullsized":"2024-04-25T09:58:57.024520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.041964+0000","last_change":"2024-04-25T09:58:57.040183+0000","last_active":"2024-04-25T09:58:57.041964+0000","last_peered":"2024-04-25T09:58:57.041964+0000","last_clean":"2024-04-25T09:58:57.041964+0000","last_became_active":"2024-04-25T09:58:57.039924+0000","last_became_peered":"2024-04-25T09:58:57.039924+0000","last_unstale":"2024-04-25T09:58:57.041964+0000","last_undegraded":"2024-04-25T09:58:57.041964+0000","last_fullsized":"2024-04-25T09:58:57.041964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:22.066654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028488+0000","last_change":"2024-04-25T09:58:57.028488+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028488+0000","last_undegraded":"2024-04-25T09:58:57.028488+0000","last_fullsized":"2024-04-25T09:58:57.028488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047039+0000","last_change":"2024-04-25T09:58:57.045075+0000","last_active":"2024-04-25T09:58:57.047039+0000","last_peered":"2024-04-25T09:58:57.047039+0000","last_clean":"2024-04-25T09:58:57.047039+0000","last_became_active":"2024-04-25T09:58:57.044812+0000","last_became_peered":"2024-04-25T09:58:57.044812+0000","last_unstale":"2024-04-25T09:58:57.047039+0000","last_undegraded":"2024-04-25T09:58:57.047039+0000","last_fullsized":"2024-04-25T09:58:57.047039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:51.303697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.030942+0000","last_change":"2024-04-25T09:58:57.030942+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.030942+0000","last_undegraded":"2024-04-25T09:58:57.030942+0000","last_fullsized":"2024-04-25T09:58:57.030942+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047888+0000","last_change":"2024-04-25T09:58:57.044690+0000","last_active":"2024-04-25T09:58:57.047888+0000","last_peered":"2024-04-25T09:58:57.047888+0000","last_clean":"2024-04-25T09:58:57.047888+0000","last_became_active":"2024-04-25T09:58:57.044405+0000","last_became_peered":"2024-04-25T09:58:57.044405+0000","last_unstale":"2024-04-25T09:58:57.047888+0000","last_undegraded":"2024-04-25T09:58:57.047888+0000","last_fullsized":"2024-04-25T09:58:57.047888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:03.790236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.030477+0000","last_change":"2024-04-25T09:58:57.030477+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.030477+0000","last_undegraded":"2024-04-25T09:58:57.030477+0000","last_fullsized":"2024-04-25T09:58:57.030477+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047136+0000","last_change":"2024-04-25T09:58:57.045507+0000","last_active":"2024-04-25T09:58:57.047136+0000","last_peered":"2024-04-25T09:58:57.047136+0000","last_clean":"2024-04-25T09:58:57.047136+0000","last_became_active":"2024-04-25T09:58:57.045211+0000","last_became_peered":"2024-04-25T09:58:57.045211+0000","last_unstale":"2024-04-25T09:58:57.047136+0000","last_undegraded":"2024-04-25T09:58:57.047136+0000","last_fullsized":"2024-04-25T09:58:57.047136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:03.790236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026747+0000","last_change":"2024-04-25T09:58:57.026747+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026747+0000","last_undegraded":"2024-04-25T09:58:57.026747+0000","last_fullsized":"2024-04-25T09:58:57.026747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043240+0000","last_change":"2024-04-25T09:58:57.042334+0000","last_active":"2024-04-25T09:58:57.043240+0000","last_peered":"2024-04-25T09:58:57.043240+0000","last_clean":"2024-04-25T09:58:57.043240+0000","last_became_active":"2024-04-25T09:58:57.042071+0000","last_became_peered":"2024-04-25T09:58:57.042071+0000","last_unstale":"2024-04-25T09:58:57.043240+0000","last_undegraded":"2024-04-25T09:58:57.043240+0000","last_fullsized":"2024-04-25T09:58:57.043240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:35.349212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031415+0000","last_change":"2024-04-25T09:58:57.031415+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031415+0000","last_undegraded":"2024-04-25T09:58:57.031415+0000","last_fullsized":"2024-04-25T09:58:57.031415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046805+0000","last_change":"2024-04-25T09:58:57.046367+0000","last_active":"2024-04-25T09:58:57.046805+0000","last_peered":"2024-04-25T09:58:57.046805+0000","last_clean":"2024-04-25T09:58:57.046805+0000","last_became_active":"2024-04-25T09:58:57.046089+0000","last_became_peered":"2024-04-25T09:58:57.046089+0000","last_unstale":"2024-04-25T09:58:57.046805+0000","last_undegraded":"2024-04-25T09:58:57.046805+0000","last_fullsized":"2024-04-25T09:58:57.046805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:20.908778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032127+0000","last_change":"2024-04-25T09:58:57.032127+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032127+0000","last_undegraded":"2024-04-25T09:58:57.032127+0000","last_fullsized":"2024-04-25T09:58:57.032127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043706+0000","last_change":"2024-04-25T09:58:57.038473+0000","last_active":"2024-04-25T09:58:57.043706+0000","last_peered":"2024-04-25T09:58:57.043706+0000","last_clean":"2024-04-25T09:58:57.043706+0000","last_became_active":"2024-04-25T09:58:57.038170+0000","last_became_peered":"2024-04-25T09:58:57.038170+0000","last_unstale":"2024-04-25T09:58:57.043706+0000","last_undegraded":"2024-04-25T09:58:57.043706+0000","last_fullsized":"2024-04-25T09:58:57.043706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:51.303697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.035966+0000","last_change":"2024-04-25T09:58:57.035966+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.035966+0000","last_undegraded":"2024-04-25T09:58:57.035966+0000","last_fullsized":"2024-04-25T09:58:57.035966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043997+0000","last_change":"2024-04-25T09:58:57.040136+0000","last_active":"2024-04-25T09:58:57.043997+0000","last_peered":"2024-04-25T09:58:57.043997+0000","last_clean":"2024-04-25T09:58:57.043997+0000","last_became_active":"2024-04-25T09:58:57.039832+0000","last_became_peered":"2024-04-25T09:58:57.039832+0000","last_unstale":"2024-04-25T09:58:57.043997+0000","last_undegraded":"2024-04-25T09:58:57.043997+0000","last_fullsized":"2024-04-25T09:58:57.043997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:44.391697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031726+0000","last_change":"2024-04-25T09:58:57.031726+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031726+0000","last_undegraded":"2024-04-25T09:58:57.031726+0000","last_fullsized":"2024-04-25T09:58:57.031726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.041588+0000","last_change":"2024-04-25T09:58:57.039566+0000","last_active":"2024-04-25T09:58:57.041588+0000","last_peered":"2024-04-25T09:58:57.041588+0000","last_clean":"2024-04-25T09:58:57.041588+0000","last_became_active":"2024-04-25T09:58:57.039279+0000","last_became_peered":"2024-04-25T09:58:57.039279+0000","last_unstale":"2024-04-25T09:58:57.041588+0000","last_undegraded":"2024-04-25T09:58:57.041588+0000","last_fullsized":"2024-04-25T09:58:57.041588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:35.349212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028874+0000","last_change":"2024-04-25T09:58:57.028874+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028874+0000","last_undegraded":"2024-04-25T09:58:57.028874+0000","last_fullsized":"2024-04-25T09:58:57.028874+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046543+0000","last_change":"2024-04-25T09:58:57.043428+0000","last_active":"2024-04-25T09:58:57.046543+0000","last_peered":"2024-04-25T09:58:57.046543+0000","last_clean":"2024-04-25T09:58:57.046543+0000","last_became_active":"2024-04-25T09:58:57.043158+0000","last_became_peered":"2024-04-25T09:58:57.043158+0000","last_unstale":"2024-04-25T09:58:57.046543+0000","last_undegraded":"2024-04-25T09:58:57.046543+0000","last_fullsized":"2024-04-25T09:58:57.046543+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:50.057945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032298+0000","last_change":"2024-04-25T09:58:57.032298+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032298+0000","last_undegraded":"2024-04-25T09:58:57.032298+0000","last_fullsized":"2024-04-25T09:58:57.032298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.039320+0000","last_change":"2024-04-25T09:58:57.036704+0000","last_active":"2024-04-25T09:58:57.039320+0000","last_peered":"2024-04-25T09:58:57.039320+0000","last_clean":"2024-04-25T09:58:57.039320+0000","last_became_active":"2024-04-25T09:58:57.036421+0000","last_became_peered":"2024-04-25T09:58:57.036421+0000","last_unstale":"2024-04-25T09:58:57.039320+0000","last_undegraded":"2024-04-25T09:58:57.039320+0000","last_fullsized":"2024-04-25T09:58:57.039320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:23.461435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032498+0000","last_change":"2024-04-25T09:58:57.032498+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032498+0000","last_undegraded":"2024-04-25T09:58:57.032498+0000","last_fullsized":"2024-04-25T09:58:57.032498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.042350+0000","last_change":"2024-04-25T09:58:57.037903+0000","last_active":"2024-04-25T09:58:57.042350+0000","last_peered":"2024-04-25T09:58:57.042350+0000","last_clean":"2024-04-25T09:58:57.042350+0000","last_became_active":"2024-04-25T09:58:57.037640+0000","last_became_peered":"2024-04-25T09:58:57.037640+0000","last_unstale":"2024-04-25T09:58:57.042350+0000","last_undegraded":"2024-04-25T09:58:57.042350+0000","last_fullsized":"2024-04-25T09:58:57.042350+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:04.149145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031438+0000","last_change":"2024-04-25T09:58:57.031438+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031438+0000","last_undegraded":"2024-04-25T09:58:57.031438+0000","last_fullsized":"2024-04-25T09:58:57.031438+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046749+0000","last_change":"2024-04-25T09:58:57.043679+0000","last_active":"2024-04-25T09:58:57.046749+0000","last_peered":"2024-04-25T09:58:57.046749+0000","last_clean":"2024-04-25T09:58:57.046749+0000","last_became_active":"2024-04-25T09:58:57.043368+0000","last_became_peered":"2024-04-25T09:58:57.043368+0000","last_unstale":"2024-04-25T09:58:57.046749+0000","last_undegraded":"2024-04-25T09:58:57.046749+0000","last_fullsized":"2024-04-25T09:58:57.046749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:54.216572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.035284+0000","last_change":"2024-04-25T09:58:57.035284+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.035284+0000","last_undegraded":"2024-04-25T09:58:57.035284+0000","last_fullsized":"2024-04-25T09:58:57.035284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.042343+0000","last_change":"2024-04-25T09:58:57.037357+0000","last_active":"2024-04-25T09:58:57.042343+0000","last_peered":"2024-04-25T09:58:57.042343+0000","last_clean":"2024-04-25T09:58:57.042343+0000","last_became_active":"2024-04-25T09:58:57.037087+0000","last_became_peered":"2024-04-25T09:58:57.037087+0000","last_unstale":"2024-04-25T09:58:57.042343+0000","last_undegraded":"2024-04-25T09:58:57.042343+0000","last_fullsized":"2024-04-25T09:58:57.042343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:58.457158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032863+0000","last_change":"2024-04-25T09:58:57.032863+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032863+0000","last_undegraded":"2024-04-25T09:58:57.032863+0000","last_fullsized":"2024-04-25T09:58:57.032863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.044146+0000","last_change":"2024-04-25T09:58:57.040746+0000","last_active":"2024-04-25T09:58:57.044146+0000","last_peered":"2024-04-25T09:58:57.044146+0000","last_clean":"2024-04-25T09:58:57.044146+0000","last_became_active":"2024-04-25T09:58:57.040447+0000","last_became_peered":"2024-04-25T09:58:57.040447+0000","last_unstale":"2024-04-25T09:58:57.044146+0000","last_undegraded":"2024-04-25T09:58:57.044146+0000","last_fullsized":"2024-04-25T09:58:57.044146+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:04.360841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.038293+0000","last_change":"2024-04-25T09:58:57.038293+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.038293+0000","last_undegraded":"2024-04-25T09:58:57.038293+0000","last_fullsized":"2024-04-25T09:58:57.038293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.044039+0000","last_change":"2024-04-25T09:58:57.039890+0000","last_active":"2024-04-25T09:58:57.044039+0000","last_peered":"2024-04-25T09:58:57.044039+0000","last_clean":"2024-04-25T09:58:57.044039+0000","last_became_active":"2024-04-25T09:58:57.039428+0000","last_became_peered":"2024-04-25T09:58:57.039428+0000","last_unstale":"2024-04-25T09:58:57.044039+0000","last_undegraded":"2024-04-25T09:58:57.044039+0000","last_fullsized":"2024-04-25T09:58:57.044039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:34.094773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031608+0000","last_change":"2024-04-25T09:58:57.031608+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031608+0000","last_undegraded":"2024-04-25T09:58:57.031608+0000","last_fullsized":"2024-04-25T09:58:57.031608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047026+0000","last_change":"2024-04-25T09:58:57.045052+0000","last_active":"2024-04-25T09:58:57.047026+0000","last_peered":"2024-04-25T09:58:57.047026+0000","last_clean":"2024-04-25T09:58:57.047026+0000","last_became_active":"2024-04-25T09:58:57.044719+0000","last_became_peered":"2024-04-25T09:58:57.044719+0000","last_unstale":"2024-04-25T09:58:57.047026+0000","last_undegraded":"2024-04-25T09:58:57.047026+0000","last_fullsized":"2024-04-25T09:58:57.047026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:20.749313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032975+0000","last_change":"2024-04-25T09:58:57.032975+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032975+0000","last_undegraded":"2024-04-25T09:58:57.032975+0000","last_fullsized":"2024-04-25T09:58:57.032975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046867+0000","last_change":"2024-04-25T09:58:57.042212+0000","last_active":"2024-04-25T09:58:57.046867+0000","last_peered":"2024-04-25T09:58:57.046867+0000","last_clean":"2024-04-25T09:58:57.046867+0000","last_became_active":"2024-04-25T09:58:57.041931+0000","last_became_peered":"2024-04-25T09:58:57.041931+0000","last_unstale":"2024-04-25T09:58:57.046867+0000","last_undegraded":"2024-04-25T09:58:57.046867+0000","last_fullsized":"2024-04-25T09:58:57.046867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:39.639654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.030927+0000","last_change":"2024-04-25T09:58:57.030927+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.030927+0000","last_undegraded":"2024-04-25T09:58:57.030927+0000","last_fullsized":"2024-04-25T09:58:57.030927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.045305+0000","last_change":"2024-04-25T09:58:57.043114+0000","last_active":"2024-04-25T09:58:57.045305+0000","last_peered":"2024-04-25T09:58:57.045305+0000","last_clean":"2024-04-25T09:58:57.045305+0000","last_became_active":"2024-04-25T09:58:57.042800+0000","last_became_peered":"2024-04-25T09:58:57.042800+0000","last_unstale":"2024-04-25T09:58:57.045305+0000","last_undegraded":"2024-04-25T09:58:57.045305+0000","last_fullsized":"2024-04-25T09:58:57.045305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:04.360841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028484+0000","last_change":"2024-04-25T09:58:57.028484+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028484+0000","last_undegraded":"2024-04-25T09:58:57.028484+0000","last_fullsized":"2024-04-25T09:58:57.028484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.039332+0000","last_change":"2024-04-25T09:58:57.036703+0000","last_active":"2024-04-25T09:58:57.039332+0000","last_peered":"2024-04-25T09:58:57.039332+0000","last_clean":"2024-04-25T09:58:57.039332+0000","last_became_active":"2024-04-25T09:58:57.036409+0000","last_became_peered":"2024-04-25T09:58:57.036409+0000","last_unstale":"2024-04-25T09:58:57.039332+0000","last_undegraded":"2024-04-25T09:58:57.039332+0000","last_fullsized":"2024-04-25T09:58:57.039332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:26.837285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031019+0000","last_change":"2024-04-25T09:58:57.031019+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031019+0000","last_undegraded":"2024-04-25T09:58:57.031019+0000","last_fullsized":"2024-04-25T09:58:57.031019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.044055+0000","last_change":"2024-04-25T09:58:57.039451+0000","last_active":"2024-04-25T09:58:57.044055+0000","last_peered":"2024-04-25T09:58:57.044055+0000","last_clean":"2024-04-25T09:58:57.044055+0000","last_became_active":"2024-04-25T09:58:57.039193+0000","last_became_peered":"2024-04-25T09:58:57.039193+0000","last_unstale":"2024-04-25T09:58:57.044055+0000","last_undegraded":"2024-04-25T09:58:57.044055+0000","last_fullsized":"2024-04-25T09:58:57.044055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:35.986283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032671+0000","last_change":"2024-04-25T09:58:57.032671+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032671+0000","last_undegraded":"2024-04-25T09:58:57.032671+0000","last_fullsized":"2024-04-25T09:58:57.032671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.039001+0000","last_change":"2024-04-25T09:58:57.035195+0000","last_active":"2024-04-25T09:58:57.039001+0000","last_peered":"2024-04-25T09:58:57.039001+0000","last_clean":"2024-04-25T09:58:57.039001+0000","last_became_active":"2024-04-25T09:58:57.034665+0000","last_became_peered":"2024-04-25T09:58:57.034665+0000","last_unstale":"2024-04-25T09:58:57.039001+0000","last_undegraded":"2024-04-25T09:58:57.039001+0000","last_fullsized":"2024-04-25T09:58:57.039001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:03.668577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024524+0000","last_change":"2024-04-25T09:58:57.024524+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024524+0000","last_undegraded":"2024-04-25T09:58:57.024524+0000","last_fullsized":"2024-04-25T09:58:57.024524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.045377+0000","last_change":"2024-04-25T09:58:57.042156+0000","last_active":"2024-04-25T09:58:57.045377+0000","last_peered":"2024-04-25T09:58:57.045377+0000","last_clean":"2024-04-25T09:58:57.045377+0000","last_became_active":"2024-04-25T09:58:57.040423+0000","last_became_peered":"2024-04-25T09:58:57.040423+0000","last_unstale":"2024-04-25T09:58:57.045377+0000","last_undegraded":"2024-04-25T09:58:57.045377+0000","last_fullsized":"2024-04-25T09:58:57.045377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:27.846107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":293,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.025542+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:58:57.025542+0000","last_peered":"2024-04-25T09:58:57.025542+0000","last_clean":"2024-04-25T09:58:57.025542+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:58:57.025542+0000","last_undegraded":"2024-04-25T09:58:57.025542+0000","last_fullsized":"2024-04-25T09:58:57.025542+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1667072,"data_stored":1079965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1667072,"data_stored":1079965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:59:01.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3f is not active+clean 2024-04-25T09:59:01.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.029464+0000', 'last_change': '2024-04-25T09:58:57.029464+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.029464+0000', 'last_undegraded': '2024-04-25T09:58:57.029464+0000', 'last_fullsized': '2024-04-25T09:58:57.029464+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:59:01.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3e is not active+clean 2024-04-25T09:59:01.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.034310+0000', 'last_change': '2024-04-25T09:58:57.034310+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.034310+0000', 'last_undegraded': '2024-04-25T09:58:57.034310+0000', 'last_fullsized': '2024-04-25T09:58:57.034310+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:59:01.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3d is not active+clean 2024-04-25T09:59:01.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.029376+0000', 'last_change': '2024-04-25T09:58:57.029376+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.029376+0000', 'last_undegraded': '2024-04-25T09:58:57.029376+0000', 'last_fullsized': '2024-04-25T09:58:57.029376+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:59:01.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3c is not active+clean 2024-04-25T09:59:01.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.034971+0000', 'last_change': '2024-04-25T09:58:57.034971+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.034971+0000', 'last_undegraded': '2024-04-25T09:58:57.034971+0000', 'last_fullsized': '2024-04-25T09:58:57.034971+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:59:01.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3b is not active+clean 2024-04-25T09:59:01.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.028683+0000', 'last_change': '2024-04-25T09:58:57.028683+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.028683+0000', 'last_undegraded': '2024-04-25T09:58:57.028683+0000', 'last_fullsized': '2024-04-25T09:58:57.028683+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:59:01.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3a is not active+clean 2024-04-25T09:59:01.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.036041+0000', 'last_change': '2024-04-25T09:58:57.036041+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.036041+0000', 'last_undegraded': '2024-04-25T09:58:57.036041+0000', 'last_fullsized': '2024-04-25T09:58:57.036041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.39 is not active+clean 2024-04-25T09:59:01.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.024527+0000', 'last_change': '2024-04-25T09:58:57.024527+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.024527+0000', 'last_undegraded': '2024-04-25T09:58:57.024527+0000', 'last_fullsized': '2024-04-25T09:58:57.024527+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:59:01.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.38 is not active+clean 2024-04-25T09:59:01.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.034283+0000', 'last_change': '2024-04-25T09:58:57.034283+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.034283+0000', 'last_undegraded': '2024-04-25T09:58:57.034283+0000', 'last_fullsized': '2024-04-25T09:58:57.034283+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:59:01.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.37 is not active+clean 2024-04-25T09:59:01.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.026906+0000', 'last_change': '2024-04-25T09:58:57.026906+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.026906+0000', 'last_undegraded': '2024-04-25T09:58:57.026906+0000', 'last_fullsized': '2024-04-25T09:58:57.026906+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:59:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.36 is not active+clean 2024-04-25T09:59:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032805+0000', 'last_change': '2024-04-25T09:58:57.032805+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032805+0000', 'last_undegraded': '2024-04-25T09:58:57.032805+0000', 'last_fullsized': '2024-04-25T09:58:57.032805+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:59:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.35 is not active+clean 2024-04-25T09:59:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.029222+0000', 'last_change': '2024-04-25T09:58:57.029222+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.029222+0000', 'last_undegraded': '2024-04-25T09:58:57.029222+0000', 'last_fullsized': '2024-04-25T09:58:57.029222+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:59:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.34 is not active+clean 2024-04-25T09:59:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032544+0000', 'last_change': '2024-04-25T09:58:57.032544+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032544+0000', 'last_undegraded': '2024-04-25T09:58:57.032544+0000', 'last_fullsized': '2024-04-25T09:58:57.032544+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:59:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.33 is not active+clean 2024-04-25T09:59:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.033778+0000', 'last_change': '2024-04-25T09:58:57.033778+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.033778+0000', 'last_undegraded': '2024-04-25T09:58:57.033778+0000', 'last_fullsized': '2024-04-25T09:58:57.033778+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:59:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.32 is not active+clean 2024-04-25T09:59:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.029419+0000', 'last_change': '2024-04-25T09:58:57.029419+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.029419+0000', 'last_undegraded': '2024-04-25T09:58:57.029419+0000', 'last_fullsized': '2024-04-25T09:58:57.029419+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:59:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.31 is not active+clean 2024-04-25T09:59:01.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.033181+0000', 'last_change': '2024-04-25T09:58:57.033181+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.033181+0000', 'last_undegraded': '2024-04-25T09:58:57.033181+0000', 'last_fullsized': '2024-04-25T09:58:57.033181+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:59:01.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.30 is not active+clean 2024-04-25T09:59:01.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.037221+0000', 'last_change': '2024-04-25T09:58:57.037221+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.037221+0000', 'last_undegraded': '2024-04-25T09:58:57.037221+0000', 'last_fullsized': '2024-04-25T09:58:57.037221+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2f is not active+clean 2024-04-25T09:59:01.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.037999+0000', 'last_change': '2024-04-25T09:58:57.037999+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.037999+0000', 'last_undegraded': '2024-04-25T09:58:57.037999+0000', 'last_fullsized': '2024-04-25T09:58:57.037999+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2e is not active+clean 2024-04-25T09:59:01.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.029302+0000', 'last_change': '2024-04-25T09:58:57.029302+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.029302+0000', 'last_undegraded': '2024-04-25T09:58:57.029302+0000', 'last_fullsized': '2024-04-25T09:58:57.029302+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:59:01.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2d is not active+clean 2024-04-25T09:59:01.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.028752+0000', 'last_change': '2024-04-25T09:58:57.028752+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.028752+0000', 'last_undegraded': '2024-04-25T09:58:57.028752+0000', 'last_fullsized': '2024-04-25T09:58:57.028752+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:59:01.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2c is not active+clean 2024-04-25T09:59:01.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032721+0000', 'last_change': '2024-04-25T09:58:57.032721+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032721+0000', 'last_undegraded': '2024-04-25T09:58:57.032721+0000', 'last_fullsized': '2024-04-25T09:58:57.032721+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:59:01.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2b is not active+clean 2024-04-25T09:59:01.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.029156+0000', 'last_change': '2024-04-25T09:58:57.029156+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.029156+0000', 'last_undegraded': '2024-04-25T09:58:57.029156+0000', 'last_fullsized': '2024-04-25T09:58:57.029156+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:59:01.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2a is not active+clean 2024-04-25T09:59:01.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.028607+0000', 'last_change': '2024-04-25T09:58:57.028607+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.028607+0000', 'last_undegraded': '2024-04-25T09:58:57.028607+0000', 'last_fullsized': '2024-04-25T09:58:57.028607+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:59:01.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2024-04-25T09:59:01.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032825+0000', 'last_change': '2024-04-25T09:58:57.032825+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032825+0000', 'last_undegraded': '2024-04-25T09:58:57.032825+0000', 'last_fullsized': '2024-04-25T09:58:57.032825+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:59:01.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.28 is not active+clean 2024-04-25T09:59:01.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.028484+0000', 'last_change': '2024-04-25T09:58:57.028484+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.028484+0000', 'last_undegraded': '2024-04-25T09:58:57.028484+0000', 'last_fullsized': '2024-04-25T09:58:57.028484+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:59:01.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.27 is not active+clean 2024-04-25T09:59:01.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.033413+0000', 'last_change': '2024-04-25T09:58:57.033413+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.033413+0000', 'last_undegraded': '2024-04-25T09:58:57.033413+0000', 'last_fullsized': '2024-04-25T09:58:57.033413+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:59:01.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.26 is not active+clean 2024-04-25T09:59:01.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.029507+0000', 'last_change': '2024-04-25T09:58:57.029507+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.029507+0000', 'last_undegraded': '2024-04-25T09:58:57.029507+0000', 'last_fullsized': '2024-04-25T09:58:57.029507+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:59:01.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.25 is not active+clean 2024-04-25T09:59:01.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032369+0000', 'last_change': '2024-04-25T09:58:57.032369+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032369+0000', 'last_undegraded': '2024-04-25T09:58:57.032369+0000', 'last_fullsized': '2024-04-25T09:58:57.032369+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:59:01.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.24 is not active+clean 2024-04-25T09:59:01.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032287+0000', 'last_change': '2024-04-25T09:58:57.032287+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032287+0000', 'last_undegraded': '2024-04-25T09:58:57.032287+0000', 'last_fullsized': '2024-04-25T09:58:57.032287+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:59:01.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.23 is not active+clean 2024-04-25T09:59:01.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.024530+0000', 'last_change': '2024-04-25T09:58:57.024530+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.024530+0000', 'last_undegraded': '2024-04-25T09:58:57.024530+0000', 'last_fullsized': '2024-04-25T09:58:57.024530+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:59:01.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.22 is not active+clean 2024-04-25T09:59:01.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.026740+0000', 'last_change': '2024-04-25T09:58:57.026740+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.026740+0000', 'last_undegraded': '2024-04-25T09:58:57.026740+0000', 'last_fullsized': '2024-04-25T09:58:57.026740+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:59:01.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.21 is not active+clean 2024-04-25T09:59:01.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032961+0000', 'last_change': '2024-04-25T09:58:57.032961+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032961+0000', 'last_undegraded': '2024-04-25T09:58:57.032961+0000', 'last_fullsized': '2024-04-25T09:58:57.032961+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:59:01.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.20 is not active+clean 2024-04-25T09:59:01.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032779+0000', 'last_change': '2024-04-25T09:58:57.032779+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032779+0000', 'last_undegraded': '2024-04-25T09:58:57.032779+0000', 'last_fullsized': '2024-04-25T09:58:57.032779+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-04-25T09:59:01.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.036398+0000', 'last_change': '2024-04-25T09:58:57.036398+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.036398+0000', 'last_undegraded': '2024-04-25T09:58:57.036398+0000', 'last_fullsized': '2024-04-25T09:58:57.036398+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-04-25T09:59:01.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.038701+0000', 'last_change': '2024-04-25T09:58:57.038701+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.038701+0000', 'last_undegraded': '2024-04-25T09:58:57.038701+0000', 'last_fullsized': '2024-04-25T09:58:57.038701+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-04-25T09:59:01.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.024524+0000', 'last_change': '2024-04-25T09:58:57.024524+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.024524+0000', 'last_undegraded': '2024-04-25T09:58:57.024524+0000', 'last_fullsized': '2024-04-25T09:58:57.024524+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:59:01.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-04-25T09:59:01.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.035703+0000', 'last_change': '2024-04-25T09:58:57.035703+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.035703+0000', 'last_undegraded': '2024-04-25T09:58:57.035703+0000', 'last_fullsized': '2024-04-25T09:58:57.035703+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2024-04-25T09:59:01.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.031396+0000', 'last_change': '2024-04-25T09:58:57.031396+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.031396+0000', 'last_undegraded': '2024-04-25T09:58:57.031396+0000', 'last_fullsized': '2024-04-25T09:58:57.031396+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:59:01.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-04-25T09:59:01.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032373+0000', 'last_change': '2024-04-25T09:58:57.032373+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032373+0000', 'last_undegraded': '2024-04-25T09:58:57.032373+0000', 'last_fullsized': '2024-04-25T09:58:57.032373+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:59:01.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-04-25T09:59:01.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.033026+0000', 'last_change': '2024-04-25T09:58:57.033026+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.033026+0000', 'last_undegraded': '2024-04-25T09:58:57.033026+0000', 'last_fullsized': '2024-04-25T09:58:57.033026+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2024-04-25T09:59:01.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.031838+0000', 'last_change': '2024-04-25T09:58:57.031838+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.031838+0000', 'last_undegraded': '2024-04-25T09:58:57.031838+0000', 'last_fullsized': '2024-04-25T09:58:57.031838+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:59:01.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-04-25T09:59:01.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.026908+0000', 'last_change': '2024-04-25T09:58:57.026908+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.026908+0000', 'last_undegraded': '2024-04-25T09:58:57.026908+0000', 'last_fullsized': '2024-04-25T09:58:57.026908+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:59:01.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-04-25T09:59:01.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.024520+0000', 'last_change': '2024-04-25T09:58:57.024520+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.024520+0000', 'last_undegraded': '2024-04-25T09:58:57.024520+0000', 'last_fullsized': '2024-04-25T09:58:57.024520+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:59:01.673 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-04-25T09:59:01.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.028488+0000', 'last_change': '2024-04-25T09:58:57.028488+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.028488+0000', 'last_undegraded': '2024-04-25T09:58:57.028488+0000', 'last_fullsized': '2024-04-25T09:58:57.028488+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:59:01.673 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2024-04-25T09:59:01.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.030942+0000', 'last_change': '2024-04-25T09:58:57.030942+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.030942+0000', 'last_undegraded': '2024-04-25T09:58:57.030942+0000', 'last_fullsized': '2024-04-25T09:58:57.030942+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:59:01.673 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2024-04-25T09:59:01.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.030477+0000', 'last_change': '2024-04-25T09:58:57.030477+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.030477+0000', 'last_undegraded': '2024-04-25T09:58:57.030477+0000', 'last_fullsized': '2024-04-25T09:58:57.030477+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:59:01.673 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-04-25T09:59:01.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.026747+0000', 'last_change': '2024-04-25T09:58:57.026747+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.026747+0000', 'last_undegraded': '2024-04-25T09:58:57.026747+0000', 'last_fullsized': '2024-04-25T09:58:57.026747+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:59:01.673 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-04-25T09:59:01.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.031415+0000', 'last_change': '2024-04-25T09:58:57.031415+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.031415+0000', 'last_undegraded': '2024-04-25T09:58:57.031415+0000', 'last_fullsized': '2024-04-25T09:58:57.031415+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:59:01.673 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-04-25T09:59:01.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032127+0000', 'last_change': '2024-04-25T09:58:57.032127+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032127+0000', 'last_undegraded': '2024-04-25T09:58:57.032127+0000', 'last_fullsized': '2024-04-25T09:58:57.032127+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:59:01.674 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-04-25T09:59:01.674 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.035966+0000', 'last_change': '2024-04-25T09:58:57.035966+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.035966+0000', 'last_undegraded': '2024-04-25T09:58:57.035966+0000', 'last_fullsized': '2024-04-25T09:58:57.035966+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.674 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-04-25T09:59:01.674 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.031726+0000', 'last_change': '2024-04-25T09:58:57.031726+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.031726+0000', 'last_undegraded': '2024-04-25T09:58:57.031726+0000', 'last_fullsized': '2024-04-25T09:58:57.031726+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:59:01.674 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-04-25T09:59:01.674 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.028874+0000', 'last_change': '2024-04-25T09:58:57.028874+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.028874+0000', 'last_undegraded': '2024-04-25T09:58:57.028874+0000', 'last_fullsized': '2024-04-25T09:58:57.028874+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:59:01.674 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-04-25T09:59:01.674 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032298+0000', 'last_change': '2024-04-25T09:58:57.032298+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032298+0000', 'last_undegraded': '2024-04-25T09:58:57.032298+0000', 'last_fullsized': '2024-04-25T09:58:57.032298+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.674 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-04-25T09:59:01.674 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032498+0000', 'last_change': '2024-04-25T09:58:57.032498+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032498+0000', 'last_undegraded': '2024-04-25T09:58:57.032498+0000', 'last_fullsized': '2024-04-25T09:58:57.032498+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:59:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-04-25T09:59:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.031438+0000', 'last_change': '2024-04-25T09:58:57.031438+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.031438+0000', 'last_undegraded': '2024-04-25T09:58:57.031438+0000', 'last_fullsized': '2024-04-25T09:58:57.031438+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:59:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-04-25T09:59:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.035284+0000', 'last_change': '2024-04-25T09:58:57.035284+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.035284+0000', 'last_undegraded': '2024-04-25T09:58:57.035284+0000', 'last_fullsized': '2024-04-25T09:58:57.035284+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-04-25T09:59:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032863+0000', 'last_change': '2024-04-25T09:58:57.032863+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032863+0000', 'last_undegraded': '2024-04-25T09:58:57.032863+0000', 'last_fullsized': '2024-04-25T09:58:57.032863+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:59:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-04-25T09:59:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.038293+0000', 'last_change': '2024-04-25T09:58:57.038293+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.038293+0000', 'last_undegraded': '2024-04-25T09:58:57.038293+0000', 'last_fullsized': '2024-04-25T09:58:57.038293+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-04-25T09:59:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.031608+0000', 'last_change': '2024-04-25T09:58:57.031608+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.031608+0000', 'last_undegraded': '2024-04-25T09:58:57.031608+0000', 'last_fullsized': '2024-04-25T09:58:57.031608+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-04-25T09:59:01.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032975+0000', 'last_change': '2024-04-25T09:58:57.032975+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032975+0000', 'last_undegraded': '2024-04-25T09:58:57.032975+0000', 'last_fullsized': '2024-04-25T09:58:57.032975+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:59:01.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-04-25T09:59:01.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.030927+0000', 'last_change': '2024-04-25T09:58:57.030927+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.030927+0000', 'last_undegraded': '2024-04-25T09:58:57.030927+0000', 'last_fullsized': '2024-04-25T09:58:57.030927+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:59:01.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-04-25T09:59:01.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.028484+0000', 'last_change': '2024-04-25T09:58:57.028484+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.028484+0000', 'last_undegraded': '2024-04-25T09:58:57.028484+0000', 'last_fullsized': '2024-04-25T09:58:57.028484+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:59:01.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-04-25T09:59:01.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.031019+0000', 'last_change': '2024-04-25T09:58:57.031019+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.031019+0000', 'last_undegraded': '2024-04-25T09:58:57.031019+0000', 'last_fullsized': '2024-04-25T09:58:57.031019+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:59:01.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-04-25T09:59:01.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.032671+0000', 'last_change': '2024-04-25T09:58:57.032671+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.032671+0000', 'last_undegraded': '2024-04-25T09:58:57.032671+0000', 'last_fullsized': '2024-04-25T09:58:57.032671+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:59:01.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-04-25T09:59:01.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 87, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:58:57.024524+0000', 'last_change': '2024-04-25T09:58:57.024524+0000', 'last_active': '2024-04-25T09:58:57.013386+0000', 'last_peered': '2024-04-25T09:58:57.013386+0000', 'last_clean': '2024-04-25T09:58:57.013386+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:57.024524+0000', 'last_undegraded': '2024-04-25T09:58:57.024524+0000', 'last_fullsized': '2024-04-25T09:58:57.024524+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:57.013386+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:59:01.677 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:59:01.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.795+0000 7fe745a7c640 1 Processor -- start 2024-04-25T09:59:01.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.795+0000 7fe745a7c640 1 -- start start 2024-04-25T09:59:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe745a7c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe74015efb0 0x7fe7400d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe745a7c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7400cf900 0x7fe74006c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe745a7c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7400cfe70 con 0x7fe74006cd80 2024-04-25T09:59:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe745a7c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7400d0a80 con 0x7fe74015efb0 2024-04-25T09:59:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe745a7c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400d0150 con 0x7fe7400cf900 2024-04-25T09:59:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe74006cd80 0x7fe740099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53556/0 (socket says 172.21.15.139:53556) 2024-04-25T09:59:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73ffff640 1 -- 172.21.15.139:0/2928486025 learned_addr learned my addr 172.21.15.139:0/2928486025 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73f7fe640 1 --2- 172.21.15.139:0/2928486025 >> v2:172.21.15.150:3300/0 conn(0x7fe74015efb0 0x7fe7400d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120735356 0 0) 0x7fe7400cfe70 con 0x7fe74006cd80 2024-04-25T09:59:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe728003520 con 0x7fe74006cd80 2024-04-25T09:59:01.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73effd640 1 --2- 172.21.15.139:0/2928486025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7400cf900 0x7fe74006c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3693625570 0 0) 0x7fe728003520 con 0x7fe74006cd80 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7400cf900 msgr2=0x7fe74006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 --2- 172.21.15.139:0/2928486025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7400cf900 0x7fe74006c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 >> v2:172.21.15.150:3300/0 conn(0x7fe74015efb0 msgr2=0x7fe7400d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 --2- 172.21.15.139:0/2928486025 >> v2:172.21.15.150:3300/0 conn(0x7fe74015efb0 0x7fe7400d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7400d0430 con 0x7fe74006cd80 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fe738001670 con 0x7fe74006cd80 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 >> v1:172.21.15.139:6789/0 conn(0x7fe74006cd80 legacy=0x7fe740099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 --2- 172.21.15.139:0/2928486025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7280039b0 0x7fe728003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 --2- 172.21.15.139:0/2928486025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7400cf900 0x7fe728004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.796+0000 7fe73e7fc640 1 --2- 172.21.15.139:0/2928486025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe728004d40 0x7fe7280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.797+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400d0a80 con 0x7fe7280039b0 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.797+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400d0150 con 0x7fe728004d40 2024-04-25T09:59:01.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.797+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72800b9c0 con 0x7fe7400cf900 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.797+0000 7fe73f7fe640 1 --2- 172.21.15.139:0/2928486025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7280039b0 0x7fe728003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.797+0000 7fe73ffff640 1 --2- 172.21.15.139:0/2928486025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe728004d40 0x7fe7280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.797+0000 7fe73effd640 1 --2- 172.21.15.139:0/2928486025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7400cf900 0x7fe728004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.797+0000 7fe73f7fe640 1 -- 172.21.15.139:0/2928486025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7400cf900 msgr2=0x7fe728004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.797+0000 7fe73f7fe640 1 --2- 172.21.15.139:0/2928486025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7400cf900 0x7fe728004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.797+0000 7fe73f7fe640 1 -- 172.21.15.139:0/2928486025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe728004d40 msgr2=0x7fe7280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.797+0000 7fe73f7fe640 1 --2- 172.21.15.139:0/2928486025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe728004d40 0x7fe7280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.797+0000 7fe73f7fe640 1 -- 172.21.15.139:0/2928486025 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe728003520 con 0x7fe7280039b0 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.798+0000 7fe73f7fe640 1 --2- 172.21.15.139:0/2928486025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7280039b0 0x7fe728003dd0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe730009fc0 tx=0x7fe728005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a30dca4730a0cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.798+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730009c10 con 0x7fe7280039b0 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.798+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe730036070 con 0x7fe7280039b0 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.798+0000 7fe73e7fc640 1 -- 172.21.15.139:0/2928486025 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73002d070 con 0x7fe7280039b0 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.798+0000 7fe745a7c640 1 -- 172.21.15.139:0/2928486025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7280039b0 msgr2=0x7fe728003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.798+0000 7fe745a7c640 1 --2- 172.21.15.139:0/2928486025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7280039b0 0x7fe728003dd0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe730009fc0 tx=0x7fe728005830 comp rx=0 tx=0).stop 2024-04-25T09:59:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.798+0000 7fe73ffff640 1 -- 172.21.15.139:0/2928486025 reap_dead start 2024-04-25T09:59:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.798+0000 7fe745a7c640 1 -- 172.21.15.139:0/2928486025 shutdown_connections 2024-04-25T09:59:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.798+0000 7fe745a7c640 1 -- 172.21.15.139:0/2928486025 >> 172.21.15.139:0/2928486025 conn(0x7fe7400d1790 msgr2=0x7fe7400d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.798+0000 7fe745a7c640 1 -- 172.21.15.139:0/2928486025 shutdown_connections 2024-04-25T09:59:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.798+0000 7fe745a7c640 1 -- 172.21.15.139:0/2928486025 wait complete. 2024-04-25T09:59:01.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.799+0000 7fe745a7c640 1 Processor -- start 2024-04-25T09:59:01.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.800+0000 7fe745a7c640 1 -- start start 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.800+0000 7fe745a7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7280039b0 0x7fe740155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.800+0000 7fe745a7c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7401e7930 0x7fe7400b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.800+0000 7fe745a7c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7401e7db0 0x7fe740142610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.800+0000 7fe745a7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74009a2e0 con 0x7fe7280039b0 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.800+0000 7fe745a7c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74009a160 con 0x7fe7401e7930 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.800+0000 7fe745a7c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74006c840 con 0x7fe7401e7db0 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.800+0000 7fe73f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7280039b0 0x7fe740155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.800+0000 7fe73f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7280039b0 0x7fe740155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58294/0 (socket says 172.21.15.139:58294) 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.800+0000 7fe73f7fe640 1 -- 172.21.15.139:0/422970887 learned_addr learned my addr 172.21.15.139:0/422970887 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe73ffff640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7401e7db0 0x7fe740142610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe73effd640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7401e7930 0x7fe7400b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe73f7fe640 1 -- 172.21.15.139:0/422970887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7401e7db0 msgr2=0x7fe740142610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe73f7fe640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7401e7db0 0x7fe740142610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe73f7fe640 1 -- 172.21.15.139:0/422970887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7401e7930 msgr2=0x7fe7400b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe73f7fe640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7401e7930 0x7fe7400b6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe73f7fe640 1 -- 172.21.15.139:0/422970887 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7400b4f70 con 0x7fe7280039b0 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe73f7fe640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7280039b0 0x7fe740155000 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe73002af10 tx=0x7fe730009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe727fff640 1 -- 172.21.15.139:0/422970887 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730003b90 con 0x7fe7280039b0 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe727fff640 1 -- 172.21.15.139:0/422970887 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe730003d30 con 0x7fe7280039b0 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe745a7c640 1 -- 172.21.15.139:0/422970887 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7400b5250 con 0x7fe7280039b0 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe727fff640 1 -- 172.21.15.139:0/422970887 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730004220 con 0x7fe7280039b0 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.801+0000 7fe745a7c640 1 -- 172.21.15.139:0/422970887 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7400b5810 con 0x7fe7280039b0 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.802+0000 7fe745a7c640 1 -- 172.21.15.139:0/422970887 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe740074520 con 0x7fe7280039b0 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.802+0000 7fe727fff640 1 -- 172.21.15.139:0/422970887 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe730005d10 con 0x7fe7280039b0 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.803+0000 7fe727fff640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe72c06b4d0 0x7fe72c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.803+0000 7fe727fff640 1 -- 172.21.15.139:0/422970887 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7300ab910 con 0x7fe7280039b0 2024-04-25T09:59:01.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.803+0000 7fe73effd640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe72c06b4d0 0x7fe72c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.803+0000 7fe73effd640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe72c06b4d0 0x7fe72c06d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe74006d530 tx=0x7fe734008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:01.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:01.807+0000 7fe727fff640 1 -- 172.21.15.139:0/422970887 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe73007a720 con 0x7fe7280039b0 2024-04-25T09:59:02.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.043+0000 7fe745a7c640 1 -- 172.21.15.139:0/422970887 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe7400d11d0 con 0x7fe7280039b0 2024-04-25T09:59:02.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.043+0000 7fe727fff640 1 -- 172.21.15.139:0/422970887 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1035 (secure 0 0 0) 0x7fe730044e80 con 0x7fe7280039b0 2024-04-25T09:59:02.047 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:02.047 INFO:teuthology.orchestra.run.smithi139.stdout:{"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":371,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":89,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038782,"num_in_osds":8,"osd_in_since":1714038773,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":265248768,"bytes_avail":772828864512,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":50,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7557}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T09:58:47.042363+0000","services":{}},"progress_events":{}} 2024-04-25T09:59:02.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.045+0000 7fe745a7c640 1 -- 172.21.15.139:0/422970887 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe72c06b4d0 msgr2=0x7fe72c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.045+0000 7fe745a7c640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe72c06b4d0 0x7fe72c06d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe74006d530 tx=0x7fe734008040 comp rx=0 tx=0).stop 2024-04-25T09:59:02.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.045+0000 7fe745a7c640 1 -- 172.21.15.139:0/422970887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7280039b0 msgr2=0x7fe740155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.045+0000 7fe745a7c640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7280039b0 0x7fe740155000 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe73002af10 tx=0x7fe730009750 comp rx=0 tx=0).stop 2024-04-25T09:59:02.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.045+0000 7fe745a7c640 1 -- 172.21.15.139:0/422970887 shutdown_connections 2024-04-25T09:59:02.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.045+0000 7fe745a7c640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe72c06b4d0 0x7fe72c06d990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.045+0000 7fe745a7c640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7401e7db0 0x7fe740142610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.045+0000 7fe745a7c640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7401e7930 0x7fe7400b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.045+0000 7fe745a7c640 1 --2- 172.21.15.139:0/422970887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7280039b0 0x7fe740155000 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.045+0000 7fe745a7c640 1 -- 172.21.15.139:0/422970887 >> 172.21.15.139:0/422970887 conn(0x7fe7400d1790 msgr2=0x7fe7400cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:02.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.045+0000 7fe745a7c640 1 -- 172.21.15.139:0/422970887 shutdown_connections 2024-04-25T09:59:02.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.045+0000 7fe745a7c640 1 -- 172.21.15.139:0/422970887 wait complete. 2024-04-25T09:59:02.060 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:59:02.060 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:59:02.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.154+0000 7f0e01934640 1 Processor -- start 2024-04-25T09:59:02.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.154+0000 7f0e01934640 1 -- start start 2024-04-25T09:59:02.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0e01934640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0dfc0d1080 0x7f0dfc0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0e01934640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0dfc161850 0x7f0dfc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0e01934640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dfc0cf250 con 0x7f0dfc15dc60 2024-04-25T09:59:02.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0e01934640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dfc0ce780 con 0x7f0dfc161850 2024-04-25T09:59:02.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0e01934640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dfc0d3730 con 0x7f0dfc0d1080 2024-04-25T09:59:02.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0dfaffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0dfc15dc60 0x7f0dfc0d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53568/0 (socket says 172.21.15.139:53568) 2024-04-25T09:59:02.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0dfaffd640 1 -- 172.21.15.139:0/225184498 learned_addr learned my addr 172.21.15.139:0/225184498 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:02.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0dfb7fe640 1 --2- 172.21.15.139:0/225184498 >> v2:172.21.15.139:3300/0 conn(0x7f0dfc161850 0x7f0dfc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0df3fff640 1 --2- 172.21.15.139:0/225184498 >> v2:172.21.15.150:3300/0 conn(0x7f0dfc0d1080 0x7f0dfc0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0dfa7fc640 1 -- 172.21.15.139:0/225184498 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271795379 0 0) 0x7f0dec001110 con 0x7f0dfc15dc60 2024-04-25T09:59:02.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0dfa7fc640 1 -- 172.21.15.139:0/225184498 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0de8003520 con 0x7f0dfc15dc60 2024-04-25T09:59:02.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0dfb7fe640 1 -- 172.21.15.139:0/225184498 >> v2:172.21.15.150:3300/0 conn(0x7f0dfc0d1080 msgr2=0x7f0dfc0b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:02.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0dfb7fe640 1 --2- 172.21.15.139:0/225184498 >> v2:172.21.15.150:3300/0 conn(0x7f0dfc0d1080 0x7f0dfc0b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0dfb7fe640 1 -- 172.21.15.139:0/225184498 >> v1:172.21.15.139:6789/0 conn(0x7f0dfc15dc60 legacy=0x7f0dfc0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.155+0000 7f0dfb7fe640 1 -- 172.21.15.139:0/225184498 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc0ce9b0 con 0x7f0dfc161850 2024-04-25T09:59:02.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.156+0000 7f0dfb7fe640 1 --2- 172.21.15.139:0/225184498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dfc161850 0x7f0dfc0ce240 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc068750 tx=0x7f0df402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3da246dddb63b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:02.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.156+0000 7f0dfa7fc640 1 -- 172.21.15.139:0/225184498 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4007740 con 0x7f0dfc161850 2024-04-25T09:59:02.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.156+0000 7f0dfa7fc640 1 -- 172.21.15.139:0/225184498 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df40078e0 con 0x7f0dfc161850 2024-04-25T09:59:02.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.156+0000 7f0dfa7fc640 1 -- 172.21.15.139:0/225184498 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4007cb0 con 0x7f0dfc161850 2024-04-25T09:59:02.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.157+0000 7f0e01934640 1 -- 172.21.15.139:0/225184498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dfc161850 msgr2=0x7f0dfc0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.157+0000 7f0e01934640 1 --2- 172.21.15.139:0/225184498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dfc161850 0x7f0dfc0ce240 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc068750 tx=0x7f0df402a680 comp rx=0 tx=0).stop 2024-04-25T09:59:02.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.157+0000 7f0e01934640 1 -- 172.21.15.139:0/225184498 shutdown_connections 2024-04-25T09:59:02.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.157+0000 7f0e01934640 1 --2- 172.21.15.139:0/225184498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dfc161850 0x7f0dfc0ce240 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.157+0000 7f0e01934640 1 --2- 172.21.15.139:0/225184498 >> v2:172.21.15.150:3300/0 conn(0x7f0dfc0d1080 0x7f0dfc0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.157+0000 7f0e01934640 1 -- 172.21.15.139:0/225184498 >> 172.21.15.139:0/225184498 conn(0x7f0dfc0b5910 msgr2=0x7f0dfc0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:02.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.157+0000 7f0e01934640 1 -- 172.21.15.139:0/225184498 shutdown_connections 2024-04-25T09:59:02.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.157+0000 7f0e01934640 1 -- 172.21.15.139:0/225184498 wait complete. 2024-04-25T09:59:02.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.158+0000 7f0e01934640 1 Processor -- start 2024-04-25T09:59:02.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.158+0000 7f0e01934640 1 -- start start 2024-04-25T09:59:02.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.158+0000 7f0e01934640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dfc0d1080 0x7f0dfc0d3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.158+0000 7f0e01934640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0dfc15dc60 0x7f0dfc0c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0e01934640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0dfc161850 0x7f0dfc0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0e01934640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc0d1fc0 con 0x7f0dfc0d1080 2024-04-25T09:59:02.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0e01934640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc0cef70 con 0x7f0dfc15dc60 2024-04-25T09:59:02.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0e01934640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc15e040 con 0x7f0dfc161850 2024-04-25T09:59:02.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0dfaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dfc0d1080 0x7f0dfc0d3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0dfaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dfc0d1080 0x7f0dfc0d3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58308/0 (socket says 172.21.15.139:58308) 2024-04-25T09:59:02.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0dfaffd640 1 -- 172.21.15.139:0/1258731015 learned_addr learned my addr 172.21.15.139:0/1258731015 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0df3fff640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0dfc15dc60 0x7f0dfc0c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0dfb7fe640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0dfc161850 0x7f0dfc0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0dfaffd640 1 -- 172.21.15.139:0/1258731015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0dfc161850 msgr2=0x7f0dfc0c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0dfaffd640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0dfc161850 0x7f0dfc0c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0dfaffd640 1 -- 172.21.15.139:0/1258731015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0dfc15dc60 msgr2=0x7f0dfc0c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0dfaffd640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0dfc15dc60 0x7f0dfc0c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0dfaffd640 1 -- 172.21.15.139:0/1258731015 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc0cd8e0 con 0x7f0dfc0d1080 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0dfaffd640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dfc0d1080 0x7f0dfc0d3e80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00c730 tx=0x7f0dec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0df8ff9640 1 -- 172.21.15.139:0/1258731015 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec017070 con 0x7f0dfc0d1080 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0df8ff9640 1 -- 172.21.15.139:0/1258731015 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec005ad0 con 0x7f0dfc0d1080 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0df8ff9640 1 -- 172.21.15.139:0/1258731015 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec007e20 con 0x7f0dfc0d1080 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0e01934640 1 -- 172.21.15.139:0/1258731015 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dfc0d43c0 con 0x7f0dfc0d1080 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.159+0000 7f0e01934640 1 -- 172.21.15.139:0/1258731015 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dfc0ccb20 con 0x7f0dfc0d1080 2024-04-25T09:59:02.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.160+0000 7f0e01934640 1 -- 172.21.15.139:0/1258731015 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dfc074520 con 0x7f0dfc0d1080 2024-04-25T09:59:02.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.161+0000 7f0df8ff9640 1 -- 172.21.15.139:0/1258731015 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0dec01a070 con 0x7f0dfc0d1080 2024-04-25T09:59:02.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.161+0000 7f0df8ff9640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0ddc06b500 0x7f0ddc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:02.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.161+0000 7f0df8ff9640 1 -- 172.21.15.139:0/1258731015 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0dec089710 con 0x7f0dfc0d1080 2024-04-25T09:59:02.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.161+0000 7f0df3fff640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0ddc06b500 0x7f0ddc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:02.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.161+0000 7f0df3fff640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0ddc06b500 0x7f0ddc06d9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0de4006fd0 tx=0x7f0de4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:02.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.165+0000 7f0df8ff9640 1 -- 172.21.15.139:0/1258731015 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0dfc0cf250 con 0x7f0dfc0d1080 2024-04-25T09:59:02.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.330+0000 7f0e01934640 1 -- 172.21.15.139:0/1258731015 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0dfc0d16f0 con 0x7f0ddc06b500 2024-04-25T09:59:02.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.336+0000 7f0df8ff9640 1 -- 172.21.15.139:0/1258731015 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246767 (secure 0 0 0) 0x7f0dfc0d16f0 con 0x7f0ddc06b500 2024-04-25T09:59:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:02.344 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:59:02.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.339+0000 7f0e01934640 1 -- 172.21.15.139:0/1258731015 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0ddc06b500 msgr2=0x7f0ddc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.339+0000 7f0e01934640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0ddc06b500 0x7f0ddc06d9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0de4006fd0 tx=0x7f0de4008040 comp rx=0 tx=0).stop 2024-04-25T09:59:02.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.339+0000 7f0e01934640 1 -- 172.21.15.139:0/1258731015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dfc0d1080 msgr2=0x7f0dfc0d3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:02.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.339+0000 7f0e01934640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dfc0d1080 0x7f0dfc0d3e80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00c730 tx=0x7f0dec00cbf0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.339+0000 7f0e01934640 1 -- 172.21.15.139:0/1258731015 shutdown_connections 2024-04-25T09:59:02.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.339+0000 7f0e01934640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0ddc06b500 0x7f0ddc06d9c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.339+0000 7f0e01934640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0dfc161850 0x7f0dfc0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.339+0000 7f0e01934640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0dfc15dc60 0x7f0dfc0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.339+0000 7f0e01934640 1 --2- 172.21.15.139:0/1258731015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0dfc0d1080 0x7f0dfc0d3e80 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:02.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.339+0000 7f0e01934640 1 -- 172.21.15.139:0/1258731015 >> 172.21.15.139:0/1258731015 conn(0x7f0dfc0b5910 msgr2=0x7f0dfc157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:02.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.339+0000 7f0e01934640 1 -- 172.21.15.139:0/1258731015 shutdown_connections 2024-04-25T09:59:02.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:02.339+0000 7f0e01934640 1 -- 172.21.15.139:0/1258731015 wait complete. 2024-04-25T09:59:02.354 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-04-25T09:59:01.044642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259032,"kb_used_data":9544,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715688,"statfs":{"total":773094113280,"available":772828864512,"internally_reserved":0,"allocated":9773056,"data_stored":5097512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.031004"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029464+0000","last_change":"2024-04-25T09:58:57.029464+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029464+0000","last_undegraded":"2024-04-25T09:58:57.029464+0000","last_fullsized":"2024-04-25T09:58:57.029464+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.034310+0000","last_change":"2024-04-25T09:58:57.034310+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.034310+0000","last_undegraded":"2024-04-25T09:58:57.034310+0000","last_fullsized":"2024-04-25T09:58:57.034310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029376+0000","last_change":"2024-04-25T09:58:57.029376+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029376+0000","last_undegraded":"2024-04-25T09:58:57.029376+0000","last_fullsized":"2024-04-25T09:58:57.029376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.034971+0000","last_change":"2024-04-25T09:58:57.034971+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.034971+0000","last_undegraded":"2024-04-25T09:58:57.034971+0000","last_fullsized":"2024-04-25T09:58:57.034971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028683+0000","last_change":"2024-04-25T09:58:57.028683+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028683+0000","last_undegraded":"2024-04-25T09:58:57.028683+0000","last_fullsized":"2024-04-25T09:58:57.028683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.036041+0000","last_change":"2024-04-25T09:58:57.036041+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.036041+0000","last_undegraded":"2024-04-25T09:58:57.036041+0000","last_fullsized":"2024-04-25T09:58:57.036041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024527+0000","last_change":"2024-04-25T09:58:57.024527+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024527+0000","last_undegraded":"2024-04-25T09:58:57.024527+0000","last_fullsized":"2024-04-25T09:58:57.024527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.034283+0000","last_change":"2024-04-25T09:58:57.034283+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.034283+0000","last_undegraded":"2024-04-25T09:58:57.034283+0000","last_fullsized":"2024-04-25T09:58:57.034283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026906+0000","last_change":"2024-04-25T09:58:57.026906+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026906+0000","last_undegraded":"2024-04-25T09:58:57.026906+0000","last_fullsized":"2024-04-25T09:58:57.026906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032805+0000","last_change":"2024-04-25T09:58:57.032805+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032805+0000","last_undegraded":"2024-04-25T09:58:57.032805+0000","last_fullsized":"2024-04-25T09:58:57.032805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029222+0000","last_change":"2024-04-25T09:58:57.029222+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029222+0000","last_undegraded":"2024-04-25T09:58:57.029222+0000","last_fullsized":"2024-04-25T09:58:57.029222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032544+0000","last_change":"2024-04-25T09:58:57.032544+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032544+0000","last_undegraded":"2024-04-25T09:58:57.032544+0000","last_fullsized":"2024-04-25T09:58:57.032544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033778+0000","last_change":"2024-04-25T09:58:57.033778+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033778+0000","last_undegraded":"2024-04-25T09:58:57.033778+0000","last_fullsized":"2024-04-25T09:58:57.033778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029419+0000","last_change":"2024-04-25T09:58:57.029419+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029419+0000","last_undegraded":"2024-04-25T09:58:57.029419+0000","last_fullsized":"2024-04-25T09:58:57.029419+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033181+0000","last_change":"2024-04-25T09:58:57.033181+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033181+0000","last_undegraded":"2024-04-25T09:58:57.033181+0000","last_fullsized":"2024-04-25T09:58:57.033181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.037221+0000","last_change":"2024-04-25T09:58:57.037221+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.037221+0000","last_undegraded":"2024-04-25T09:58:57.037221+0000","last_fullsized":"2024-04-25T09:58:57.037221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.037999+0000","last_change":"2024-04-25T09:58:57.037999+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.037999+0000","last_undegraded":"2024-04-25T09:58:57.037999+0000","last_fullsized":"2024-04-25T09:58:57.037999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029302+0000","last_change":"2024-04-25T09:58:57.029302+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029302+0000","last_undegraded":"2024-04-25T09:58:57.029302+0000","last_fullsized":"2024-04-25T09:58:57.029302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028752+0000","last_change":"2024-04-25T09:58:57.028752+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028752+0000","last_undegraded":"2024-04-25T09:58:57.028752+0000","last_fullsized":"2024-04-25T09:58:57.028752+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032721+0000","last_change":"2024-04-25T09:58:57.032721+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032721+0000","last_undegraded":"2024-04-25T09:58:57.032721+0000","last_fullsized":"2024-04-25T09:58:57.032721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029156+0000","last_change":"2024-04-25T09:58:57.029156+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029156+0000","last_undegraded":"2024-04-25T09:58:57.029156+0000","last_fullsized":"2024-04-25T09:58:57.029156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028607+0000","last_change":"2024-04-25T09:58:57.028607+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028607+0000","last_undegraded":"2024-04-25T09:58:57.028607+0000","last_fullsized":"2024-04-25T09:58:57.028607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032825+0000","last_change":"2024-04-25T09:58:57.032825+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032825+0000","last_undegraded":"2024-04-25T09:58:57.032825+0000","last_fullsized":"2024-04-25T09:58:57.032825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028484+0000","last_change":"2024-04-25T09:58:57.028484+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028484+0000","last_undegraded":"2024-04-25T09:58:57.028484+0000","last_fullsized":"2024-04-25T09:58:57.028484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033413+0000","last_change":"2024-04-25T09:58:57.033413+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033413+0000","last_undegraded":"2024-04-25T09:58:57.033413+0000","last_fullsized":"2024-04-25T09:58:57.033413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.029507+0000","last_change":"2024-04-25T09:58:57.029507+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.029507+0000","last_undegraded":"2024-04-25T09:58:57.029507+0000","last_fullsized":"2024-04-25T09:58:57.029507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032369+0000","last_change":"2024-04-25T09:58:57.032369+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032369+0000","last_undegraded":"2024-04-25T09:58:57.032369+0000","last_fullsized":"2024-04-25T09:58:57.032369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032287+0000","last_change":"2024-04-25T09:58:57.032287+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032287+0000","last_undegraded":"2024-04-25T09:58:57.032287+0000","last_fullsized":"2024-04-25T09:58:57.032287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024530+0000","last_change":"2024-04-25T09:58:57.024530+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024530+0000","last_undegraded":"2024-04-25T09:58:57.024530+0000","last_fullsized":"2024-04-25T09:58:57.024530+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026740+0000","last_change":"2024-04-25T09:58:57.026740+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026740+0000","last_undegraded":"2024-04-25T09:58:57.026740+0000","last_fullsized":"2024-04-25T09:58:57.026740+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032961+0000","last_change":"2024-04-25T09:58:57.032961+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032961+0000","last_undegraded":"2024-04-25T09:58:57.032961+0000","last_fullsized":"2024-04-25T09:58:57.032961+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032779+0000","last_change":"2024-04-25T09:58:57.032779+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032779+0000","last_undegraded":"2024-04-25T09:58:57.032779+0000","last_fullsized":"2024-04-25T09:58:57.032779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.036398+0000","last_change":"2024-04-25T09:58:57.036398+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.036398+0000","last_undegraded":"2024-04-25T09:58:57.036398+0000","last_fullsized":"2024-04-25T09:58:57.036398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046934+0000","last_change":"2024-04-25T09:58:57.043628+0000","last_active":"2024-04-25T09:58:57.046934+0000","last_peered":"2024-04-25T09:58:57.046934+0000","last_clean":"2024-04-25T09:58:57.046934+0000","last_became_active":"2024-04-25T09:58:57.043322+0000","last_became_peered":"2024-04-25T09:58:57.043322+0000","last_unstale":"2024-04-25T09:58:57.046934+0000","last_undegraded":"2024-04-25T09:58:57.046934+0000","last_fullsized":"2024-04-25T09:58:57.046934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:56.130588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.038701+0000","last_change":"2024-04-25T09:58:57.038701+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.038701+0000","last_undegraded":"2024-04-25T09:58:57.038701+0000","last_fullsized":"2024-04-25T09:58:57.038701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046492+0000","last_change":"2024-04-25T09:58:57.041488+0000","last_active":"2024-04-25T09:58:57.046492+0000","last_peered":"2024-04-25T09:58:57.046492+0000","last_clean":"2024-04-25T09:58:57.046492+0000","last_became_active":"2024-04-25T09:58:57.041117+0000","last_became_peered":"2024-04-25T09:58:57.041117+0000","last_unstale":"2024-04-25T09:58:57.046492+0000","last_undegraded":"2024-04-25T09:58:57.046492+0000","last_fullsized":"2024-04-25T09:58:57.046492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:59.594081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024524+0000","last_change":"2024-04-25T09:58:57.024524+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024524+0000","last_undegraded":"2024-04-25T09:58:57.024524+0000","last_fullsized":"2024-04-25T09:58:57.024524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.038904+0000","last_change":"2024-04-25T09:58:57.034001+0000","last_active":"2024-04-25T09:58:57.038904+0000","last_peered":"2024-04-25T09:58:57.038904+0000","last_clean":"2024-04-25T09:58:57.038904+0000","last_became_active":"2024-04-25T09:58:57.033646+0000","last_became_peered":"2024-04-25T09:58:57.033646+0000","last_unstale":"2024-04-25T09:58:57.038904+0000","last_undegraded":"2024-04-25T09:58:57.038904+0000","last_fullsized":"2024-04-25T09:58:57.038904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:20.730136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.035703+0000","last_change":"2024-04-25T09:58:57.035703+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.035703+0000","last_undegraded":"2024-04-25T09:58:57.035703+0000","last_fullsized":"2024-04-25T09:58:57.035703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047225+0000","last_change":"2024-04-25T09:58:57.044955+0000","last_active":"2024-04-25T09:58:57.047225+0000","last_peered":"2024-04-25T09:58:57.047225+0000","last_clean":"2024-04-25T09:58:57.047225+0000","last_became_active":"2024-04-25T09:58:57.044389+0000","last_became_peered":"2024-04-25T09:58:57.044389+0000","last_unstale":"2024-04-25T09:58:57.047225+0000","last_undegraded":"2024-04-25T09:58:57.047225+0000","last_fullsized":"2024-04-25T09:58:57.047225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:55.535171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031396+0000","last_change":"2024-04-25T09:58:57.031396+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031396+0000","last_undegraded":"2024-04-25T09:58:57.031396+0000","last_fullsized":"2024-04-25T09:58:57.031396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.038695+0000","last_change":"2024-04-25T09:58:57.034395+0000","last_active":"2024-04-25T09:58:57.038695+0000","last_peered":"2024-04-25T09:58:57.038695+0000","last_clean":"2024-04-25T09:58:57.038695+0000","last_became_active":"2024-04-25T09:58:57.034079+0000","last_became_peered":"2024-04-25T09:58:57.034079+0000","last_unstale":"2024-04-25T09:58:57.038695+0000","last_undegraded":"2024-04-25T09:58:57.038695+0000","last_fullsized":"2024-04-25T09:58:57.038695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:03.751385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032373+0000","last_change":"2024-04-25T09:58:57.032373+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032373+0000","last_undegraded":"2024-04-25T09:58:57.032373+0000","last_fullsized":"2024-04-25T09:58:57.032373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043740+0000","last_change":"2024-04-25T09:58:57.038771+0000","last_active":"2024-04-25T09:58:57.043740+0000","last_peered":"2024-04-25T09:58:57.043740+0000","last_clean":"2024-04-25T09:58:57.043740+0000","last_became_active":"2024-04-25T09:58:57.038480+0000","last_became_peered":"2024-04-25T09:58:57.038480+0000","last_unstale":"2024-04-25T09:58:57.043740+0000","last_undegraded":"2024-04-25T09:58:57.043740+0000","last_fullsized":"2024-04-25T09:58:57.043740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:58.574435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.033026+0000","last_change":"2024-04-25T09:58:57.033026+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.033026+0000","last_undegraded":"2024-04-25T09:58:57.033026+0000","last_fullsized":"2024-04-25T09:58:57.033026+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.035188+0000","last_change":"2024-04-25T09:58:57.034137+0000","last_active":"2024-04-25T09:58:57.035188+0000","last_peered":"2024-04-25T09:58:57.035188+0000","last_clean":"2024-04-25T09:58:57.035188+0000","last_became_active":"2024-04-25T09:58:57.033509+0000","last_became_peered":"2024-04-25T09:58:57.033509+0000","last_unstale":"2024-04-25T09:58:57.035188+0000","last_undegraded":"2024-04-25T09:58:57.035188+0000","last_fullsized":"2024-04-25T09:58:57.035188+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:51.854104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031838+0000","last_change":"2024-04-25T09:58:57.031838+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031838+0000","last_undegraded":"2024-04-25T09:58:57.031838+0000","last_fullsized":"2024-04-25T09:58:57.031838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047142+0000","last_change":"2024-04-25T09:58:57.045529+0000","last_active":"2024-04-25T09:58:57.047142+0000","last_peered":"2024-04-25T09:58:57.047142+0000","last_clean":"2024-04-25T09:58:57.047142+0000","last_became_active":"2024-04-25T09:58:57.045229+0000","last_became_peered":"2024-04-25T09:58:57.045229+0000","last_unstale":"2024-04-25T09:58:57.047142+0000","last_undegraded":"2024-04-25T09:58:57.047142+0000","last_fullsized":"2024-04-25T09:58:57.047142+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:34.875415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026908+0000","last_change":"2024-04-25T09:58:57.026908+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026908+0000","last_undegraded":"2024-04-25T09:58:57.026908+0000","last_fullsized":"2024-04-25T09:58:57.026908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.041384+0000","last_change":"2024-04-25T09:58:57.039169+0000","last_active":"2024-04-25T09:58:57.041384+0000","last_peered":"2024-04-25T09:58:57.041384+0000","last_clean":"2024-04-25T09:58:57.041384+0000","last_became_active":"2024-04-25T09:58:57.038892+0000","last_became_peered":"2024-04-25T09:58:57.038892+0000","last_unstale":"2024-04-25T09:58:57.041384+0000","last_undegraded":"2024-04-25T09:58:57.041384+0000","last_fullsized":"2024-04-25T09:58:57.041384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:58.402797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024520+0000","last_change":"2024-04-25T09:58:57.024520+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024520+0000","last_undegraded":"2024-04-25T09:58:57.024520+0000","last_fullsized":"2024-04-25T09:58:57.024520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.041964+0000","last_change":"2024-04-25T09:58:57.040183+0000","last_active":"2024-04-25T09:58:57.041964+0000","last_peered":"2024-04-25T09:58:57.041964+0000","last_clean":"2024-04-25T09:58:57.041964+0000","last_became_active":"2024-04-25T09:58:57.039924+0000","last_became_peered":"2024-04-25T09:58:57.039924+0000","last_unstale":"2024-04-25T09:58:57.041964+0000","last_undegraded":"2024-04-25T09:58:57.041964+0000","last_fullsized":"2024-04-25T09:58:57.041964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:22.066654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028488+0000","last_change":"2024-04-25T09:58:57.028488+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028488+0000","last_undegraded":"2024-04-25T09:58:57.028488+0000","last_fullsized":"2024-04-25T09:58:57.028488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047039+0000","last_change":"2024-04-25T09:58:57.045075+0000","last_active":"2024-04-25T09:58:57.047039+0000","last_peered":"2024-04-25T09:58:57.047039+0000","last_clean":"2024-04-25T09:58:57.047039+0000","last_became_active":"2024-04-25T09:58:57.044812+0000","last_became_peered":"2024-04-25T09:58:57.044812+0000","last_unstale":"2024-04-25T09:58:57.047039+0000","last_undegraded":"2024-04-25T09:58:57.047039+0000","last_fullsized":"2024-04-25T09:58:57.047039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:51.303697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.030942+0000","last_change":"2024-04-25T09:58:57.030942+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.030942+0000","last_undegraded":"2024-04-25T09:58:57.030942+0000","last_fullsized":"2024-04-25T09:58:57.030942+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047888+0000","last_change":"2024-04-25T09:58:57.044690+0000","last_active":"2024-04-25T09:58:57.047888+0000","last_peered":"2024-04-25T09:58:57.047888+0000","last_clean":"2024-04-25T09:58:57.047888+0000","last_became_active":"2024-04-25T09:58:57.044405+0000","last_became_peered":"2024-04-25T09:58:57.044405+0000","last_unstale":"2024-04-25T09:58:57.047888+0000","last_undegraded":"2024-04-25T09:58:57.047888+0000","last_fullsized":"2024-04-25T09:58:57.047888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:03.790236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.030477+0000","last_change":"2024-04-25T09:58:57.030477+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.030477+0000","last_undegraded":"2024-04-25T09:58:57.030477+0000","last_fullsized":"2024-04-25T09:58:57.030477+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047136+0000","last_change":"2024-04-25T09:58:57.045507+0000","last_active":"2024-04-25T09:58:57.047136+0000","last_peered":"2024-04-25T09:58:57.047136+0000","last_clean":"2024-04-25T09:58:57.047136+0000","last_became_active":"2024-04-25T09:58:57.045211+0000","last_became_peered":"2024-04-25T09:58:57.045211+0000","last_unstale":"2024-04-25T09:58:57.047136+0000","last_undegraded":"2024-04-25T09:58:57.047136+0000","last_fullsized":"2024-04-25T09:58:57.047136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:03.790236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.026747+0000","last_change":"2024-04-25T09:58:57.026747+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.026747+0000","last_undegraded":"2024-04-25T09:58:57.026747+0000","last_fullsized":"2024-04-25T09:58:57.026747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043240+0000","last_change":"2024-04-25T09:58:57.042334+0000","last_active":"2024-04-25T09:58:57.043240+0000","last_peered":"2024-04-25T09:58:57.043240+0000","last_clean":"2024-04-25T09:58:57.043240+0000","last_became_active":"2024-04-25T09:58:57.042071+0000","last_became_peered":"2024-04-25T09:58:57.042071+0000","last_unstale":"2024-04-25T09:58:57.043240+0000","last_undegraded":"2024-04-25T09:58:57.043240+0000","last_fullsized":"2024-04-25T09:58:57.043240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:35.349212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031415+0000","last_change":"2024-04-25T09:58:57.031415+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031415+0000","last_undegraded":"2024-04-25T09:58:57.031415+0000","last_fullsized":"2024-04-25T09:58:57.031415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046805+0000","last_change":"2024-04-25T09:58:57.046367+0000","last_active":"2024-04-25T09:58:57.046805+0000","last_peered":"2024-04-25T09:58:57.046805+0000","last_clean":"2024-04-25T09:58:57.046805+0000","last_became_active":"2024-04-25T09:58:57.046089+0000","last_became_peered":"2024-04-25T09:58:57.046089+0000","last_unstale":"2024-04-25T09:58:57.046805+0000","last_undegraded":"2024-04-25T09:58:57.046805+0000","last_fullsized":"2024-04-25T09:58:57.046805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:20.908778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032127+0000","last_change":"2024-04-25T09:58:57.032127+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032127+0000","last_undegraded":"2024-04-25T09:58:57.032127+0000","last_fullsized":"2024-04-25T09:58:57.032127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043706+0000","last_change":"2024-04-25T09:58:57.038473+0000","last_active":"2024-04-25T09:58:57.043706+0000","last_peered":"2024-04-25T09:58:57.043706+0000","last_clean":"2024-04-25T09:58:57.043706+0000","last_became_active":"2024-04-25T09:58:57.038170+0000","last_became_peered":"2024-04-25T09:58:57.038170+0000","last_unstale":"2024-04-25T09:58:57.043706+0000","last_undegraded":"2024-04-25T09:58:57.043706+0000","last_fullsized":"2024-04-25T09:58:57.043706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:51.303697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.035966+0000","last_change":"2024-04-25T09:58:57.035966+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.035966+0000","last_undegraded":"2024-04-25T09:58:57.035966+0000","last_fullsized":"2024-04-25T09:58:57.035966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.043997+0000","last_change":"2024-04-25T09:58:57.040136+0000","last_active":"2024-04-25T09:58:57.043997+0000","last_peered":"2024-04-25T09:58:57.043997+0000","last_clean":"2024-04-25T09:58:57.043997+0000","last_became_active":"2024-04-25T09:58:57.039832+0000","last_became_peered":"2024-04-25T09:58:57.039832+0000","last_unstale":"2024-04-25T09:58:57.043997+0000","last_undegraded":"2024-04-25T09:58:57.043997+0000","last_fullsized":"2024-04-25T09:58:57.043997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:44.391697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031726+0000","last_change":"2024-04-25T09:58:57.031726+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031726+0000","last_undegraded":"2024-04-25T09:58:57.031726+0000","last_fullsized":"2024-04-25T09:58:57.031726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.041588+0000","last_change":"2024-04-25T09:58:57.039566+0000","last_active":"2024-04-25T09:58:57.041588+0000","last_peered":"2024-04-25T09:58:57.041588+0000","last_clean":"2024-04-25T09:58:57.041588+0000","last_became_active":"2024-04-25T09:58:57.039279+0000","last_became_peered":"2024-04-25T09:58:57.039279+0000","last_unstale":"2024-04-25T09:58:57.041588+0000","last_undegraded":"2024-04-25T09:58:57.041588+0000","last_fullsized":"2024-04-25T09:58:57.041588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:35.349212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028874+0000","last_change":"2024-04-25T09:58:57.028874+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028874+0000","last_undegraded":"2024-04-25T09:58:57.028874+0000","last_fullsized":"2024-04-25T09:58:57.028874+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046543+0000","last_change":"2024-04-25T09:58:57.043428+0000","last_active":"2024-04-25T09:58:57.046543+0000","last_peered":"2024-04-25T09:58:57.046543+0000","last_clean":"2024-04-25T09:58:57.046543+0000","last_became_active":"2024-04-25T09:58:57.043158+0000","last_became_peered":"2024-04-25T09:58:57.043158+0000","last_unstale":"2024-04-25T09:58:57.046543+0000","last_undegraded":"2024-04-25T09:58:57.046543+0000","last_fullsized":"2024-04-25T09:58:57.046543+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:50.057945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032298+0000","last_change":"2024-04-25T09:58:57.032298+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032298+0000","last_undegraded":"2024-04-25T09:58:57.032298+0000","last_fullsized":"2024-04-25T09:58:57.032298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.039320+0000","last_change":"2024-04-25T09:58:57.036704+0000","last_active":"2024-04-25T09:58:57.039320+0000","last_peered":"2024-04-25T09:58:57.039320+0000","last_clean":"2024-04-25T09:58:57.039320+0000","last_became_active":"2024-04-25T09:58:57.036421+0000","last_became_peered":"2024-04-25T09:58:57.036421+0000","last_unstale":"2024-04-25T09:58:57.039320+0000","last_undegraded":"2024-04-25T09:58:57.039320+0000","last_fullsized":"2024-04-25T09:58:57.039320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:23.461435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032498+0000","last_change":"2024-04-25T09:58:57.032498+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032498+0000","last_undegraded":"2024-04-25T09:58:57.032498+0000","last_fullsized":"2024-04-25T09:58:57.032498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.042350+0000","last_change":"2024-04-25T09:58:57.037903+0000","last_active":"2024-04-25T09:58:57.042350+0000","last_peered":"2024-04-25T09:58:57.042350+0000","last_clean":"2024-04-25T09:58:57.042350+0000","last_became_active":"2024-04-25T09:58:57.037640+0000","last_became_peered":"2024-04-25T09:58:57.037640+0000","last_unstale":"2024-04-25T09:58:57.042350+0000","last_undegraded":"2024-04-25T09:58:57.042350+0000","last_fullsized":"2024-04-25T09:58:57.042350+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:04.149145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031438+0000","last_change":"2024-04-25T09:58:57.031438+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031438+0000","last_undegraded":"2024-04-25T09:58:57.031438+0000","last_fullsized":"2024-04-25T09:58:57.031438+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046749+0000","last_change":"2024-04-25T09:58:57.043679+0000","last_active":"2024-04-25T09:58:57.046749+0000","last_peered":"2024-04-25T09:58:57.046749+0000","last_clean":"2024-04-25T09:58:57.046749+0000","last_became_active":"2024-04-25T09:58:57.043368+0000","last_became_peered":"2024-04-25T09:58:57.043368+0000","last_unstale":"2024-04-25T09:58:57.046749+0000","last_undegraded":"2024-04-25T09:58:57.046749+0000","last_fullsized":"2024-04-25T09:58:57.046749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:54.216572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.035284+0000","last_change":"2024-04-25T09:58:57.035284+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.035284+0000","last_undegraded":"2024-04-25T09:58:57.035284+0000","last_fullsized":"2024-04-25T09:58:57.035284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.042343+0000","last_change":"2024-04-25T09:58:57.037357+0000","last_active":"2024-04-25T09:58:57.042343+0000","last_peered":"2024-04-25T09:58:57.042343+0000","last_clean":"2024-04-25T09:58:57.042343+0000","last_became_active":"2024-04-25T09:58:57.037087+0000","last_became_peered":"2024-04-25T09:58:57.037087+0000","last_unstale":"2024-04-25T09:58:57.042343+0000","last_undegraded":"2024-04-25T09:58:57.042343+0000","last_fullsized":"2024-04-25T09:58:57.042343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:58.457158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032863+0000","last_change":"2024-04-25T09:58:57.032863+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032863+0000","last_undegraded":"2024-04-25T09:58:57.032863+0000","last_fullsized":"2024-04-25T09:58:57.032863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.044146+0000","last_change":"2024-04-25T09:58:57.040746+0000","last_active":"2024-04-25T09:58:57.044146+0000","last_peered":"2024-04-25T09:58:57.044146+0000","last_clean":"2024-04-25T09:58:57.044146+0000","last_became_active":"2024-04-25T09:58:57.040447+0000","last_became_peered":"2024-04-25T09:58:57.040447+0000","last_unstale":"2024-04-25T09:58:57.044146+0000","last_undegraded":"2024-04-25T09:58:57.044146+0000","last_fullsized":"2024-04-25T09:58:57.044146+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:04.360841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.038293+0000","last_change":"2024-04-25T09:58:57.038293+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.038293+0000","last_undegraded":"2024-04-25T09:58:57.038293+0000","last_fullsized":"2024-04-25T09:58:57.038293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.044039+0000","last_change":"2024-04-25T09:58:57.039890+0000","last_active":"2024-04-25T09:58:57.044039+0000","last_peered":"2024-04-25T09:58:57.044039+0000","last_clean":"2024-04-25T09:58:57.044039+0000","last_became_active":"2024-04-25T09:58:57.039428+0000","last_became_peered":"2024-04-25T09:58:57.039428+0000","last_unstale":"2024-04-25T09:58:57.044039+0000","last_undegraded":"2024-04-25T09:58:57.044039+0000","last_fullsized":"2024-04-25T09:58:57.044039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:34.094773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031608+0000","last_change":"2024-04-25T09:58:57.031608+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031608+0000","last_undegraded":"2024-04-25T09:58:57.031608+0000","last_fullsized":"2024-04-25T09:58:57.031608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.047026+0000","last_change":"2024-04-25T09:58:57.045052+0000","last_active":"2024-04-25T09:58:57.047026+0000","last_peered":"2024-04-25T09:58:57.047026+0000","last_clean":"2024-04-25T09:58:57.047026+0000","last_became_active":"2024-04-25T09:58:57.044719+0000","last_became_peered":"2024-04-25T09:58:57.044719+0000","last_unstale":"2024-04-25T09:58:57.047026+0000","last_undegraded":"2024-04-25T09:58:57.047026+0000","last_fullsized":"2024-04-25T09:58:57.047026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:20.749313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032975+0000","last_change":"2024-04-25T09:58:57.032975+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032975+0000","last_undegraded":"2024-04-25T09:58:57.032975+0000","last_fullsized":"2024-04-25T09:58:57.032975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.046867+0000","last_change":"2024-04-25T09:58:57.042212+0000","last_active":"2024-04-25T09:58:57.046867+0000","last_peered":"2024-04-25T09:58:57.046867+0000","last_clean":"2024-04-25T09:58:57.046867+0000","last_became_active":"2024-04-25T09:58:57.041931+0000","last_became_peered":"2024-04-25T09:58:57.041931+0000","last_unstale":"2024-04-25T09:58:57.046867+0000","last_undegraded":"2024-04-25T09:58:57.046867+0000","last_fullsized":"2024-04-25T09:58:57.046867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:39.639654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.030927+0000","last_change":"2024-04-25T09:58:57.030927+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.030927+0000","last_undegraded":"2024-04-25T09:58:57.030927+0000","last_fullsized":"2024-04-25T09:58:57.030927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.045305+0000","last_change":"2024-04-25T09:58:57.043114+0000","last_active":"2024-04-25T09:58:57.045305+0000","last_peered":"2024-04-25T09:58:57.045305+0000","last_clean":"2024-04-25T09:58:57.045305+0000","last_became_active":"2024-04-25T09:58:57.042800+0000","last_became_peered":"2024-04-25T09:58:57.042800+0000","last_unstale":"2024-04-25T09:58:57.045305+0000","last_undegraded":"2024-04-25T09:58:57.045305+0000","last_fullsized":"2024-04-25T09:58:57.045305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:04.360841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.028484+0000","last_change":"2024-04-25T09:58:57.028484+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.028484+0000","last_undegraded":"2024-04-25T09:58:57.028484+0000","last_fullsized":"2024-04-25T09:58:57.028484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.039332+0000","last_change":"2024-04-25T09:58:57.036703+0000","last_active":"2024-04-25T09:58:57.039332+0000","last_peered":"2024-04-25T09:58:57.039332+0000","last_clean":"2024-04-25T09:58:57.039332+0000","last_became_active":"2024-04-25T09:58:57.036409+0000","last_became_peered":"2024-04-25T09:58:57.036409+0000","last_unstale":"2024-04-25T09:58:57.039332+0000","last_undegraded":"2024-04-25T09:58:57.039332+0000","last_fullsized":"2024-04-25T09:58:57.039332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:26.837285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.031019+0000","last_change":"2024-04-25T09:58:57.031019+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.031019+0000","last_undegraded":"2024-04-25T09:58:57.031019+0000","last_fullsized":"2024-04-25T09:58:57.031019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.044055+0000","last_change":"2024-04-25T09:58:57.039451+0000","last_active":"2024-04-25T09:58:57.044055+0000","last_peered":"2024-04-25T09:58:57.044055+0000","last_clean":"2024-04-25T09:58:57.044055+0000","last_became_active":"2024-04-25T09:58:57.039193+0000","last_became_peered":"2024-04-25T09:58:57.039193+0000","last_unstale":"2024-04-25T09:58:57.044055+0000","last_undegraded":"2024-04-25T09:58:57.044055+0000","last_fullsized":"2024-04-25T09:58:57.044055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:35.986283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.032671+0000","last_change":"2024-04-25T09:58:57.032671+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.032671+0000","last_undegraded":"2024-04-25T09:58:57.032671+0000","last_fullsized":"2024-04-25T09:58:57.032671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.039001+0000","last_change":"2024-04-25T09:58:57.035195+0000","last_active":"2024-04-25T09:58:57.039001+0000","last_peered":"2024-04-25T09:58:57.039001+0000","last_clean":"2024-04-25T09:58:57.039001+0000","last_became_active":"2024-04-25T09:58:57.034665+0000","last_became_peered":"2024-04-25T09:58:57.034665+0000","last_unstale":"2024-04-25T09:58:57.039001+0000","last_undegraded":"2024-04-25T09:58:57.039001+0000","last_fullsized":"2024-04-25T09:58:57.039001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:03.668577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T09:58:57.024524+0000","last_change":"2024-04-25T09:58:57.024524+0000","last_active":"2024-04-25T09:58:57.013386+0000","last_peered":"2024-04-25T09:58:57.013386+0000","last_clean":"2024-04-25T09:58:57.013386+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:57.024524+0000","last_undegraded":"2024-04-25T09:58:57.024524+0000","last_fullsized":"2024-04-25T09:58:57.024524+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.045377+0000","last_change":"2024-04-25T09:58:57.042156+0000","last_active":"2024-04-25T09:58:57.045377+0000","last_peered":"2024-04-25T09:58:57.045377+0000","last_clean":"2024-04-25T09:58:57.045377+0000","last_became_active":"2024-04-25T09:58:57.040423+0000","last_became_peered":"2024-04-25T09:58:57.040423+0000","last_unstale":"2024-04-25T09:58:57.045377+0000","last_undegraded":"2024-04-25T09:58:57.045377+0000","last_fullsized":"2024-04-25T09:58:57.045377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:27.846107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":293,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T09:58:57.025542+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:58:57.025542+0000","last_peered":"2024-04-25T09:58:57.025542+0000","last_clean":"2024-04-25T09:58:57.025542+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:58:57.025542+0000","last_undegraded":"2024-04-25T09:58:57.025542+0000","last_fullsized":"2024-04-25T09:58:57.025542+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1667072,"data_stored":1079965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1073152,"data_stored":489597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1667072,"data_stored":1079965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:59:03.296 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-04-25T09:59:03.292+0000 7f75cb771640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:59:05.359 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:59:05.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.465+0000 7effd4f63640 1 Processor -- start 2024-04-25T09:59:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.465+0000 7effd4f63640 1 -- start start 2024-04-25T09:59:05.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.465+0000 7effd4f63640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7effd015b980 0x7effd006fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.465+0000 7effd4f63640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effd006eda0 0x7effd006ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.465+0000 7effd4f63640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7effd006b0a0 0x7effd015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.465+0000 7effd4f63640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd00cd030 con 0x7effd006b0a0 2024-04-25T09:59:05.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.465+0000 7effd4f63640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd0070580 con 0x7effd015b980 2024-04-25T09:59:05.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.465+0000 7effd4f63640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd0070860 con 0x7effd006eda0 2024-04-25T09:59:05.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.466+0000 7effced9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7effd006b0a0 0x7effd015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.466+0000 7effced9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7effd006b0a0 0x7effd015bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58314/0 (socket says 172.21.15.139:58314) 2024-04-25T09:59:05.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.466+0000 7effced9d640 1 -- 172.21.15.139:0/613452331 learned_addr learned my addr 172.21.15.139:0/613452331 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:05.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.466+0000 7effce59c640 1 --2- 172.21.15.139:0/613452331 >> v2:172.21.15.150:3300/0 conn(0x7effd015b980 0x7effd006fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.466+0000 7effced9d640 1 -- 172.21.15.139:0/613452331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effd006eda0 msgr2=0x7effd006ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:05.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.466+0000 7effc5d9b640 1 --2- 172.21.15.139:0/613452331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effd006eda0 0x7effd006ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.466+0000 7effced9d640 1 --2- 172.21.15.139:0/613452331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effd006eda0 0x7effd006ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.466+0000 7effced9d640 1 -- 172.21.15.139:0/613452331 >> v2:172.21.15.150:3300/0 conn(0x7effd015b980 msgr2=0x7effd006fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:05.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.466+0000 7effced9d640 1 --2- 172.21.15.139:0/613452331 >> v2:172.21.15.150:3300/0 conn(0x7effd015b980 0x7effd006fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.466+0000 7effced9d640 1 -- 172.21.15.139:0/613452331 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd0070b40 con 0x7effd006b0a0 2024-04-25T09:59:05.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effced9d640 1 --2- 172.21.15.139:0/613452331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effd006b0a0 0x7effd015bd60 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7effd004f610 tx=0x7effc802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1ce5889fcfd211 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effcdd9b640 1 -- 172.21.15.139:0/613452331 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc8036070 con 0x7effd006b0a0 2024-04-25T09:59:05.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effcdd9b640 1 -- 172.21.15.139:0/613452331 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc8031600 con 0x7effd006b0a0 2024-04-25T09:59:05.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effcdd9b640 1 -- 172.21.15.139:0/613452331 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc80319d0 con 0x7effd006b0a0 2024-04-25T09:59:05.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effd4f63640 1 -- 172.21.15.139:0/613452331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effd006b0a0 msgr2=0x7effd015bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effd4f63640 1 --2- 172.21.15.139:0/613452331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effd006b0a0 0x7effd015bd60 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7effd004f610 tx=0x7effc802a610 comp rx=0 tx=0).stop 2024-04-25T09:59:05.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effd4f63640 1 -- 172.21.15.139:0/613452331 shutdown_connections 2024-04-25T09:59:05.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effd4f63640 1 --2- 172.21.15.139:0/613452331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effd006b0a0 0x7effd015bd60 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effd4f63640 1 --2- 172.21.15.139:0/613452331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effd006eda0 0x7effd006ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effd4f63640 1 --2- 172.21.15.139:0/613452331 >> v2:172.21.15.150:3300/0 conn(0x7effd015b980 0x7effd006fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effd4f63640 1 -- 172.21.15.139:0/613452331 >> 172.21.15.139:0/613452331 conn(0x7effd006e280 msgr2=0x7effd006e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:05.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effd4f63640 1 -- 172.21.15.139:0/613452331 shutdown_connections 2024-04-25T09:59:05.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.467+0000 7effd4f63640 1 -- 172.21.15.139:0/613452331 wait complete. 2024-04-25T09:59:05.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.469+0000 7effd4f63640 1 Processor -- start 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.469+0000 7effd4f63640 1 -- start start 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effd4f63640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effd006b0a0 0x7effd01518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effd4f63640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effd006eda0 0x7effd0143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effd4f63640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effd015b980 0x7effd015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effd4f63640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd015c2a0 con 0x7effd006eda0 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effd4f63640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd0070270 con 0x7effd015b980 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effd4f63640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd015c420 con 0x7effd006b0a0 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effc5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effd006eda0 0x7effd0143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effc5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effd006eda0 0x7effd0143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58324/0 (socket says 172.21.15.139:58324) 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effc5d9b640 1 -- 172.21.15.139:0/455662304 learned_addr learned my addr 172.21.15.139:0/455662304 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effced9d640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effd015b980 0x7effd015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effce59c640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effd006b0a0 0x7effd01518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effc5d9b640 1 -- 172.21.15.139:0/455662304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effd006b0a0 msgr2=0x7effd01518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effc5d9b640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effd006b0a0 0x7effd01518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effc5d9b640 1 -- 172.21.15.139:0/455662304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effd015b980 msgr2=0x7effd015a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effc5d9b640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effd015b980 0x7effd015a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effc5d9b640 1 -- 172.21.15.139:0/455662304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd0159680 con 0x7effd006eda0 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effc5d9b640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effd006eda0 0x7effd0143f00 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7effb800c7f0 tx=0x7effb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effc77fe640 1 -- 172.21.15.139:0/455662304 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8015070 con 0x7effd006eda0 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effc77fe640 1 -- 172.21.15.139:0/455662304 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb8004220 con 0x7effd006eda0 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effc77fe640 1 -- 172.21.15.139:0/455662304 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8007620 con 0x7effd006eda0 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effd4f63640 1 -- 172.21.15.139:0/455662304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd0150de0 con 0x7effd006eda0 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.470+0000 7effd4f63640 1 -- 172.21.15.139:0/455662304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd0151e00 con 0x7effd006eda0 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.471+0000 7effd4f63640 1 -- 172.21.15.139:0/455662304 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd0074520 con 0x7effd006eda0 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.472+0000 7effc77fe640 1 -- 172.21.15.139:0/455662304 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7effb80079d0 con 0x7effd006eda0 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.472+0000 7effc77fe640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7effb006b500 0x7effb006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.472+0000 7effc77fe640 1 -- 172.21.15.139:0/455662304 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7effb808a3e0 con 0x7effd006eda0 2024-04-25T09:59:05.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.472+0000 7effce59c640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7effb006b500 0x7effb006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.473+0000 7effce59c640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7effb006b500 0x7effb006d9c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7effc000aa20 tx=0x7effc0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.477+0000 7effc77fe640 1 -- 172.21.15.139:0/455662304 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7effb80589f0 con 0x7effd006eda0 2024-04-25T09:59:05.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.643+0000 7effd4f63640 1 -- 172.21.15.139:0/455662304 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7effd0150c30 con 0x7effb006b500 2024-04-25T09:59:05.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.649+0000 7effc77fe640 1 -- 172.21.15.139:0/455662304 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252075 (secure 0 0 0) 0x7effd0150c30 con 0x7effb006b500 2024-04-25T09:59:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:05.656 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T09:59:05.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.651+0000 7effd4f63640 1 -- 172.21.15.139:0/455662304 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7effb006b500 msgr2=0x7effb006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.651+0000 7effd4f63640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7effb006b500 0x7effb006d9c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7effc000aa20 tx=0x7effc0008040 comp rx=0 tx=0).stop 2024-04-25T09:59:05.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.652+0000 7effd4f63640 1 -- 172.21.15.139:0/455662304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effd006eda0 msgr2=0x7effd0143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.652+0000 7effd4f63640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effd006eda0 0x7effd0143f00 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7effb800c7f0 tx=0x7effb800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.652+0000 7effd4f63640 1 -- 172.21.15.139:0/455662304 shutdown_connections 2024-04-25T09:59:05.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.652+0000 7effd4f63640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effd006eda0 0x7effd0143f00 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.652+0000 7effd4f63640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7effb006b500 0x7effb006d9c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.652+0000 7effd4f63640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effd015b980 0x7effd015a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.652+0000 7effd4f63640 1 --2- 172.21.15.139:0/455662304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effd006b0a0 0x7effd01518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.652+0000 7effd4f63640 1 -- 172.21.15.139:0/455662304 >> 172.21.15.139:0/455662304 conn(0x7effd006e280 msgr2=0x7effd00cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:05.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.652+0000 7effd4f63640 1 -- 172.21.15.139:0/455662304 shutdown_connections 2024-04-25T09:59:05.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.652+0000 7effd4f63640 1 -- 172.21.15.139:0/455662304 wait complete. 2024-04-25T09:59:05.666 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-04-25T09:59:05.045648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259244,"kb_used_data":9800,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715476,"statfs":{"total":773094113280,"available":772828647424,"internally_reserved":0,"allocated":10035200,"data_stored":5213204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001760"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.446102+0000","last_change":"2024-04-25T09:58:58.052725+0000","last_active":"2024-04-25T09:59:00.446102+0000","last_peered":"2024-04-25T09:59:00.446102+0000","last_clean":"2024-04-25T09:59:00.446102+0000","last_became_active":"2024-04-25T09:58:58.052473+0000","last_became_peered":"2024-04-25T09:58:58.052473+0000","last_unstale":"2024-04-25T09:59:00.446102+0000","last_undegraded":"2024-04-25T09:59:00.446102+0000","last_fullsized":"2024-04-25T09:59:00.446102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:26.149630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.029647+0000","last_change":"2024-04-25T09:58:58.046196+0000","last_active":"2024-04-25T09:59:00.029647+0000","last_peered":"2024-04-25T09:59:00.029647+0000","last_clean":"2024-04-25T09:59:00.029647+0000","last_became_active":"2024-04-25T09:58:58.045898+0000","last_became_peered":"2024-04-25T09:58:58.045898+0000","last_unstale":"2024-04-25T09:59:00.029647+0000","last_undegraded":"2024-04-25T09:59:00.029647+0000","last_fullsized":"2024-04-25T09:59:00.029647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:04.673014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.446076+0000","last_change":"2024-04-25T09:58:58.045399+0000","last_active":"2024-04-25T09:59:00.446076+0000","last_peered":"2024-04-25T09:59:00.446076+0000","last_clean":"2024-04-25T09:59:00.446076+0000","last_became_active":"2024-04-25T09:58:58.045099+0000","last_became_peered":"2024-04-25T09:58:58.045099+0000","last_unstale":"2024-04-25T09:59:00.446076+0000","last_undegraded":"2024-04-25T09:59:00.446076+0000","last_fullsized":"2024-04-25T09:59:00.446076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:29.740524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445326+0000","last_change":"2024-04-25T09:58:58.051321+0000","last_active":"2024-04-25T09:59:00.445326+0000","last_peered":"2024-04-25T09:59:00.445326+0000","last_clean":"2024-04-25T09:59:00.445326+0000","last_became_active":"2024-04-25T09:58:58.051039+0000","last_became_peered":"2024-04-25T09:58:58.051039+0000","last_unstale":"2024-04-25T09:59:00.445326+0000","last_undegraded":"2024-04-25T09:59:00.445326+0000","last_fullsized":"2024-04-25T09:59:00.445326+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:50.032371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.039781+0000","last_change":"2024-04-25T09:58:58.038211+0000","last_active":"2024-04-25T09:58:59.039781+0000","last_peered":"2024-04-25T09:58:59.039781+0000","last_clean":"2024-04-25T09:58:59.039781+0000","last_became_active":"2024-04-25T09:58:58.037895+0000","last_became_peered":"2024-04-25T09:58:58.037895+0000","last_unstale":"2024-04-25T09:58:59.039781+0000","last_undegraded":"2024-04-25T09:58:59.039781+0000","last_fullsized":"2024-04-25T09:58:59.039781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:50.032371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.037841+0000","last_change":"2024-04-25T09:58:58.059562+0000","last_active":"2024-04-25T09:58:59.037841+0000","last_peered":"2024-04-25T09:58:59.037841+0000","last_clean":"2024-04-25T09:58:59.037841+0000","last_became_active":"2024-04-25T09:58:58.059289+0000","last_became_peered":"2024-04-25T09:58:58.059289+0000","last_unstale":"2024-04-25T09:58:59.037841+0000","last_undegraded":"2024-04-25T09:58:59.037841+0000","last_fullsized":"2024-04-25T09:58:59.037841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:37.120842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445463+0000","last_change":"2024-04-25T09:58:58.038921+0000","last_active":"2024-04-25T09:59:00.445463+0000","last_peered":"2024-04-25T09:59:00.445463+0000","last_clean":"2024-04-25T09:59:00.445463+0000","last_became_active":"2024-04-25T09:58:58.038504+0000","last_became_peered":"2024-04-25T09:58:58.038504+0000","last_unstale":"2024-04-25T09:59:00.445463+0000","last_undegraded":"2024-04-25T09:59:00.445463+0000","last_fullsized":"2024-04-25T09:59:00.445463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:40.300219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445278+0000","last_change":"2024-04-25T09:58:58.051389+0000","last_active":"2024-04-25T09:59:00.445278+0000","last_peered":"2024-04-25T09:59:00.445278+0000","last_clean":"2024-04-25T09:59:00.445278+0000","last_became_active":"2024-04-25T09:58:58.051092+0000","last_became_peered":"2024-04-25T09:58:58.051092+0000","last_unstale":"2024-04-25T09:59:00.445278+0000","last_undegraded":"2024-04-25T09:59:00.445278+0000","last_fullsized":"2024-04-25T09:59:00.445278+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:51.275902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.040070+0000","last_change":"2024-04-25T09:58:58.040000+0000","last_active":"2024-04-25T09:58:59.040070+0000","last_peered":"2024-04-25T09:58:59.040070+0000","last_clean":"2024-04-25T09:58:59.040070+0000","last_became_active":"2024-04-25T09:58:58.039692+0000","last_became_peered":"2024-04-25T09:58:58.039692+0000","last_unstale":"2024-04-25T09:58:59.040070+0000","last_undegraded":"2024-04-25T09:58:59.040070+0000","last_fullsized":"2024-04-25T09:58:59.040070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:28.118176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.446338+0000","last_change":"2024-04-25T09:58:58.049568+0000","last_active":"2024-04-25T09:59:00.446338+0000","last_peered":"2024-04-25T09:59:00.446338+0000","last_clean":"2024-04-25T09:59:00.446338+0000","last_became_active":"2024-04-25T09:58:58.049289+0000","last_became_peered":"2024-04-25T09:58:58.049289+0000","last_unstale":"2024-04-25T09:59:00.446338+0000","last_undegraded":"2024-04-25T09:59:00.446338+0000","last_fullsized":"2024-04-25T09:59:00.446338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:02.799046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.040026+0000","last_change":"2024-04-25T09:58:58.048512+0000","last_active":"2024-04-25T09:58:59.040026+0000","last_peered":"2024-04-25T09:58:59.040026+0000","last_clean":"2024-04-25T09:58:59.040026+0000","last_became_active":"2024-04-25T09:58:58.048238+0000","last_became_peered":"2024-04-25T09:58:58.048238+0000","last_unstale":"2024-04-25T09:58:59.040026+0000","last_undegraded":"2024-04-25T09:58:59.040026+0000","last_fullsized":"2024-04-25T09:58:59.040026+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:20.106413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445513+0000","last_change":"2024-04-25T09:58:58.038623+0000","last_active":"2024-04-25T09:59:00.445513+0000","last_peered":"2024-04-25T09:59:00.445513+0000","last_clean":"2024-04-25T09:59:00.445513+0000","last_became_active":"2024-04-25T09:58:58.038355+0000","last_became_peered":"2024-04-25T09:58:58.038355+0000","last_unstale":"2024-04-25T09:59:00.445513+0000","last_undegraded":"2024-04-25T09:59:00.445513+0000","last_fullsized":"2024-04-25T09:59:00.445513+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:41.316803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445167+0000","last_change":"2024-04-25T09:58:58.043618+0000","last_active":"2024-04-25T09:59:00.445167+0000","last_peered":"2024-04-25T09:59:00.445167+0000","last_clean":"2024-04-25T09:59:00.445167+0000","last_became_active":"2024-04-25T09:58:58.043327+0000","last_became_peered":"2024-04-25T09:58:58.043327+0000","last_unstale":"2024-04-25T09:59:00.445167+0000","last_undegraded":"2024-04-25T09:59:00.445167+0000","last_fullsized":"2024-04-25T09:59:00.445167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:55.221010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.039510+0000","last_change":"2024-04-25T09:58:58.046972+0000","last_active":"2024-04-25T09:58:59.039510+0000","last_peered":"2024-04-25T09:58:59.039510+0000","last_clean":"2024-04-25T09:58:59.039510+0000","last_became_active":"2024-04-25T09:58:58.045481+0000","last_became_peered":"2024-04-25T09:58:58.045481+0000","last_unstale":"2024-04-25T09:58:59.039510+0000","last_undegraded":"2024-04-25T09:58:59.039510+0000","last_fullsized":"2024-04-25T09:58:59.039510+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:55.221010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445060+0000","last_change":"2024-04-25T09:58:58.054799+0000","last_active":"2024-04-25T09:59:00.445060+0000","last_peered":"2024-04-25T09:59:00.445060+0000","last_clean":"2024-04-25T09:59:00.445060+0000","last_became_active":"2024-04-25T09:58:58.053784+0000","last_became_peered":"2024-04-25T09:58:58.053784+0000","last_unstale":"2024-04-25T09:59:00.445060+0000","last_undegraded":"2024-04-25T09:59:00.445060+0000","last_fullsized":"2024-04-25T09:59:00.445060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:24.465873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.037697+0000","last_change":"2024-04-25T09:58:58.052392+0000","last_active":"2024-04-25T09:58:59.037697+0000","last_peered":"2024-04-25T09:58:59.037697+0000","last_clean":"2024-04-25T09:58:59.037697+0000","last_became_active":"2024-04-25T09:58:58.052078+0000","last_became_peered":"2024-04-25T09:58:58.052078+0000","last_unstale":"2024-04-25T09:58:59.037697+0000","last_undegraded":"2024-04-25T09:58:59.037697+0000","last_fullsized":"2024-04-25T09:58:59.037697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:08.549430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.038271+0000","last_change":"2024-04-25T09:58:58.052263+0000","last_active":"2024-04-25T09:58:59.038271+0000","last_peered":"2024-04-25T09:58:59.038271+0000","last_clean":"2024-04-25T09:58:59.038271+0000","last_became_active":"2024-04-25T09:58:58.051993+0000","last_became_peered":"2024-04-25T09:58:58.051993+0000","last_unstale":"2024-04-25T09:58:59.038271+0000","last_undegraded":"2024-04-25T09:58:59.038271+0000","last_fullsized":"2024-04-25T09:58:59.038271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:07.418641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445711+0000","last_change":"2024-04-25T09:58:58.039194+0000","last_active":"2024-04-25T09:59:00.445711+0000","last_peered":"2024-04-25T09:59:00.445711+0000","last_clean":"2024-04-25T09:59:00.445711+0000","last_became_active":"2024-04-25T09:58:58.038899+0000","last_became_peered":"2024-04-25T09:58:58.038899+0000","last_unstale":"2024-04-25T09:59:00.445711+0000","last_undegraded":"2024-04-25T09:59:00.445711+0000","last_fullsized":"2024-04-25T09:59:00.445711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:02.594951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.040109+0000","last_change":"2024-04-25T09:58:58.045179+0000","last_active":"2024-04-25T09:58:59.040109+0000","last_peered":"2024-04-25T09:58:59.040109+0000","last_clean":"2024-04-25T09:58:59.040109+0000","last_became_active":"2024-04-25T09:58:58.044636+0000","last_became_peered":"2024-04-25T09:58:58.044636+0000","last_unstale":"2024-04-25T09:58:59.040109+0000","last_undegraded":"2024-04-25T09:58:59.040109+0000","last_fullsized":"2024-04-25T09:58:59.040109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:27.776323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445647+0000","last_change":"2024-04-25T09:58:58.054071+0000","last_active":"2024-04-25T09:59:00.445647+0000","last_peered":"2024-04-25T09:59:00.445647+0000","last_clean":"2024-04-25T09:59:00.445647+0000","last_became_active":"2024-04-25T09:58:58.053795+0000","last_became_peered":"2024-04-25T09:58:58.053795+0000","last_unstale":"2024-04-25T09:59:00.445647+0000","last_undegraded":"2024-04-25T09:59:00.445647+0000","last_fullsized":"2024-04-25T09:59:00.445647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:28.118176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.039586+0000","last_change":"2024-04-25T09:58:58.037730+0000","last_active":"2024-04-25T09:58:59.039586+0000","last_peered":"2024-04-25T09:58:59.039586+0000","last_clean":"2024-04-25T09:58:59.039586+0000","last_became_active":"2024-04-25T09:58:58.037445+0000","last_became_peered":"2024-04-25T09:58:58.037445+0000","last_unstale":"2024-04-25T09:58:59.039586+0000","last_undegraded":"2024-04-25T09:58:59.039586+0000","last_fullsized":"2024-04-25T09:58:59.039586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:51.275902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.039518+0000","last_change":"2024-04-25T09:58:58.045221+0000","last_active":"2024-04-25T09:58:59.039518+0000","last_peered":"2024-04-25T09:58:59.039518+0000","last_clean":"2024-04-25T09:58:59.039518+0000","last_became_active":"2024-04-25T09:58:58.044740+0000","last_became_peered":"2024-04-25T09:58:58.044740+0000","last_unstale":"2024-04-25T09:58:59.039518+0000","last_undegraded":"2024-04-25T09:58:59.039518+0000","last_fullsized":"2024-04-25T09:58:59.039518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:24.465873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445253+0000","last_change":"2024-04-25T09:58:58.044567+0000","last_active":"2024-04-25T09:59:00.445253+0000","last_peered":"2024-04-25T09:59:00.445253+0000","last_clean":"2024-04-25T09:59:00.445253+0000","last_became_active":"2024-04-25T09:58:58.044256+0000","last_became_peered":"2024-04-25T09:58:58.044256+0000","last_unstale":"2024-04-25T09:59:00.445253+0000","last_undegraded":"2024-04-25T09:59:00.445253+0000","last_fullsized":"2024-04-25T09:59:00.445253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:59.407235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445201+0000","last_change":"2024-04-25T09:58:58.048489+0000","last_active":"2024-04-25T09:59:00.445201+0000","last_peered":"2024-04-25T09:59:00.445201+0000","last_clean":"2024-04-25T09:59:00.445201+0000","last_became_active":"2024-04-25T09:58:58.048221+0000","last_became_peered":"2024-04-25T09:58:58.048221+0000","last_unstale":"2024-04-25T09:59:00.445201+0000","last_undegraded":"2024-04-25T09:59:00.445201+0000","last_fullsized":"2024-04-25T09:59:00.445201+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:35.879852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.029698+0000","last_change":"2024-04-25T09:58:58.036701+0000","last_active":"2024-04-25T09:59:00.029698+0000","last_peered":"2024-04-25T09:59:00.029698+0000","last_clean":"2024-04-25T09:59:00.029698+0000","last_became_active":"2024-04-25T09:58:58.036432+0000","last_became_peered":"2024-04-25T09:58:58.036432+0000","last_unstale":"2024-04-25T09:59:00.029698+0000","last_undegraded":"2024-04-25T09:59:00.029698+0000","last_fullsized":"2024-04-25T09:59:00.029698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:36.353650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.040174+0000","last_change":"2024-04-25T09:58:58.045614+0000","last_active":"2024-04-25T09:58:59.040174+0000","last_peered":"2024-04-25T09:58:59.040174+0000","last_clean":"2024-04-25T09:58:59.040174+0000","last_became_active":"2024-04-25T09:58:58.045119+0000","last_became_peered":"2024-04-25T09:58:58.045119+0000","last_unstale":"2024-04-25T09:58:59.040174+0000","last_undegraded":"2024-04-25T09:58:59.040174+0000","last_fullsized":"2024-04-25T09:58:59.040174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:14.155390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445741+0000","last_change":"2024-04-25T09:58:58.033790+0000","last_active":"2024-04-25T09:59:00.445741+0000","last_peered":"2024-04-25T09:59:00.445741+0000","last_clean":"2024-04-25T09:59:00.445741+0000","last_became_active":"2024-04-25T09:58:58.033522+0000","last_became_peered":"2024-04-25T09:58:58.033522+0000","last_unstale":"2024-04-25T09:59:00.445741+0000","last_undegraded":"2024-04-25T09:59:00.445741+0000","last_fullsized":"2024-04-25T09:59:00.445741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:12.333982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445476+0000","last_change":"2024-04-25T09:58:58.047558+0000","last_active":"2024-04-25T09:59:00.445476+0000","last_peered":"2024-04-25T09:59:00.445476+0000","last_clean":"2024-04-25T09:59:00.445476+0000","last_became_active":"2024-04-25T09:58:58.047217+0000","last_became_peered":"2024-04-25T09:58:58.047217+0000","last_unstale":"2024-04-25T09:59:00.445476+0000","last_undegraded":"2024-04-25T09:59:00.445476+0000","last_fullsized":"2024-04-25T09:59:00.445476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:20.106413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445605+0000","last_change":"2024-04-25T09:58:58.050019+0000","last_active":"2024-04-25T09:59:00.445605+0000","last_peered":"2024-04-25T09:59:00.445605+0000","last_clean":"2024-04-25T09:59:00.445605+0000","last_became_active":"2024-04-25T09:58:58.048341+0000","last_became_peered":"2024-04-25T09:58:58.048341+0000","last_unstale":"2024-04-25T09:59:00.445605+0000","last_undegraded":"2024-04-25T09:59:00.445605+0000","last_fullsized":"2024-04-25T09:59:00.445605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:45.862333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.039741+0000","last_change":"2024-04-25T09:58:58.040790+0000","last_active":"2024-04-25T09:58:59.039741+0000","last_peered":"2024-04-25T09:58:59.039741+0000","last_clean":"2024-04-25T09:58:59.039741+0000","last_became_active":"2024-04-25T09:58:58.040499+0000","last_became_peered":"2024-04-25T09:58:58.040499+0000","last_unstale":"2024-04-25T09:58:59.039741+0000","last_undegraded":"2024-04-25T09:58:59.039741+0000","last_fullsized":"2024-04-25T09:58:59.039741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.363976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.029464+0000","last_change":"2024-04-25T09:58:58.058410+0000","last_active":"2024-04-25T09:59:00.029464+0000","last_peered":"2024-04-25T09:59:00.029464+0000","last_clean":"2024-04-25T09:59:00.029464+0000","last_became_active":"2024-04-25T09:58:58.058071+0000","last_became_peered":"2024-04-25T09:58:58.058071+0000","last_unstale":"2024-04-25T09:59:00.029464+0000","last_undegraded":"2024-04-25T09:59:00.029464+0000","last_fullsized":"2024-04-25T09:59:00.029464+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:47.970921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.037696+0000","last_change":"2024-04-25T09:58:58.047719+0000","last_active":"2024-04-25T09:58:59.037696+0000","last_peered":"2024-04-25T09:58:59.037696+0000","last_clean":"2024-04-25T09:58:59.037696+0000","last_became_active":"2024-04-25T09:58:58.046979+0000","last_became_peered":"2024-04-25T09:58:58.046979+0000","last_unstale":"2024-04-25T09:58:59.037696+0000","last_undegraded":"2024-04-25T09:58:59.037696+0000","last_fullsized":"2024-04-25T09:58:59.037696+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:11.063136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.037885+0000","last_change":"2024-04-25T09:58:58.051843+0000","last_active":"2024-04-25T09:58:59.037885+0000","last_peered":"2024-04-25T09:58:59.037885+0000","last_clean":"2024-04-25T09:58:59.037885+0000","last_became_active":"2024-04-25T09:58:58.050845+0000","last_became_peered":"2024-04-25T09:58:58.050845+0000","last_unstale":"2024-04-25T09:58:59.037885+0000","last_undegraded":"2024-04-25T09:58:59.037885+0000","last_fullsized":"2024-04-25T09:58:59.037885+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:22.711829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"88'4","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445700+0000","last_change":"2024-04-25T09:58:57.043628+0000","last_active":"2024-04-25T09:59:00.445700+0000","last_peered":"2024-04-25T09:59:00.445700+0000","last_clean":"2024-04-25T09:59:00.445700+0000","last_became_active":"2024-04-25T09:58:57.043322+0000","last_became_peered":"2024-04-25T09:58:57.043322+0000","last_unstale":"2024-04-25T09:59:00.445700+0000","last_undegraded":"2024-04-25T09:59:00.445700+0000","last_fullsized":"2024-04-25T09:59:00.445700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:43.020784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.038001+0000","last_change":"2024-04-25T09:58:58.054806+0000","last_active":"2024-04-25T09:58:59.038001+0000","last_peered":"2024-04-25T09:58:59.038001+0000","last_clean":"2024-04-25T09:58:59.038001+0000","last_became_active":"2024-04-25T09:58:58.054532+0000","last_became_peered":"2024-04-25T09:58:58.054532+0000","last_unstale":"2024-04-25T09:58:59.038001+0000","last_undegraded":"2024-04-25T09:58:59.038001+0000","last_fullsized":"2024-04-25T09:58:59.038001+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:32.076860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.040302+0000","last_change":"2024-04-25T09:58:57.041488+0000","last_active":"2024-04-25T09:58:59.040302+0000","last_peered":"2024-04-25T09:58:59.040302+0000","last_clean":"2024-04-25T09:58:59.040302+0000","last_became_active":"2024-04-25T09:58:57.041117+0000","last_became_peered":"2024-04-25T09:58:57.041117+0000","last_unstale":"2024-04-25T09:58:59.040302+0000","last_undegraded":"2024-04-25T09:58:59.040302+0000","last_fullsized":"2024-04-25T09:58:59.040302+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:20.908778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445197+0000","last_change":"2024-04-25T09:58:58.049196+0000","last_active":"2024-04-25T09:59:00.445197+0000","last_peered":"2024-04-25T09:59:00.445197+0000","last_clean":"2024-04-25T09:59:00.445197+0000","last_became_active":"2024-04-25T09:58:58.048912+0000","last_became_peered":"2024-04-25T09:58:58.048912+0000","last_unstale":"2024-04-25T09:59:00.445197+0000","last_undegraded":"2024-04-25T09:59:00.445197+0000","last_fullsized":"2024-04-25T09:59:00.445197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:13.122375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445176+0000","last_change":"2024-04-25T09:58:57.034001+0000","last_active":"2024-04-25T09:59:00.445176+0000","last_peered":"2024-04-25T09:59:00.445176+0000","last_clean":"2024-04-25T09:59:00.445176+0000","last_became_active":"2024-04-25T09:58:57.033646+0000","last_became_peered":"2024-04-25T09:58:57.033646+0000","last_unstale":"2024-04-25T09:59:00.445176+0000","last_undegraded":"2024-04-25T09:59:00.445176+0000","last_fullsized":"2024-04-25T09:59:00.445176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:14.385296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.038308+0000","last_change":"2024-04-25T09:58:58.050325+0000","last_active":"2024-04-25T09:58:59.038308+0000","last_peered":"2024-04-25T09:58:59.038308+0000","last_clean":"2024-04-25T09:58:59.038308+0000","last_became_active":"2024-04-25T09:58:58.050028+0000","last_became_peered":"2024-04-25T09:58:58.050028+0000","last_unstale":"2024-04-25T09:58:59.038308+0000","last_undegraded":"2024-04-25T09:58:59.038308+0000","last_fullsized":"2024-04-25T09:58:59.038308+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:55.784890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445888+0000","last_change":"2024-04-25T09:58:57.044955+0000","last_active":"2024-04-25T09:59:00.445888+0000","last_peered":"2024-04-25T09:59:00.445888+0000","last_clean":"2024-04-25T09:59:00.445888+0000","last_became_active":"2024-04-25T09:58:57.044389+0000","last_became_peered":"2024-04-25T09:58:57.044389+0000","last_unstale":"2024-04-25T09:59:00.445888+0000","last_undegraded":"2024-04-25T09:59:00.445888+0000","last_fullsized":"2024-04-25T09:59:00.445888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:33.372378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445412+0000","last_change":"2024-04-25T09:58:58.055017+0000","last_active":"2024-04-25T09:59:00.445412+0000","last_peered":"2024-04-25T09:59:00.445412+0000","last_clean":"2024-04-25T09:59:00.445412+0000","last_became_active":"2024-04-25T09:58:58.054600+0000","last_became_peered":"2024-04-25T09:58:58.054600+0000","last_unstale":"2024-04-25T09:59:00.445412+0000","last_undegraded":"2024-04-25T09:59:00.445412+0000","last_fullsized":"2024-04-25T09:59:00.445412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:24.191210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.039699+0000","last_change":"2024-04-25T09:58:57.034395+0000","last_active":"2024-04-25T09:58:59.039699+0000","last_peered":"2024-04-25T09:58:59.039699+0000","last_clean":"2024-04-25T09:58:59.039699+0000","last_became_active":"2024-04-25T09:58:57.034079+0000","last_became_peered":"2024-04-25T09:58:57.034079+0000","last_unstale":"2024-04-25T09:58:59.039699+0000","last_undegraded":"2024-04-25T09:58:59.039699+0000","last_fullsized":"2024-04-25T09:58:59.039699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:58.574435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445434+0000","last_change":"2024-04-25T09:58:58.042642+0000","last_active":"2024-04-25T09:59:00.445434+0000","last_peered":"2024-04-25T09:59:00.445434+0000","last_clean":"2024-04-25T09:59:00.445434+0000","last_became_active":"2024-04-25T09:58:58.042155+0000","last_became_peered":"2024-04-25T09:58:58.042155+0000","last_unstale":"2024-04-25T09:59:00.445434+0000","last_undegraded":"2024-04-25T09:59:00.445434+0000","last_fullsized":"2024-04-25T09:59:00.445434+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:51.275902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445984+0000","last_change":"2024-04-25T09:58:57.038771+0000","last_active":"2024-04-25T09:59:00.445984+0000","last_peered":"2024-04-25T09:59:00.445984+0000","last_clean":"2024-04-25T09:59:00.445984+0000","last_became_active":"2024-04-25T09:58:57.038480+0000","last_became_peered":"2024-04-25T09:58:57.038480+0000","last_unstale":"2024-04-25T09:59:00.445984+0000","last_undegraded":"2024-04-25T09:59:00.445984+0000","last_fullsized":"2024-04-25T09:59:00.445984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:44.391697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.038420+0000","last_change":"2024-04-25T09:58:58.059401+0000","last_active":"2024-04-25T09:58:59.038420+0000","last_peered":"2024-04-25T09:58:59.038420+0000","last_clean":"2024-04-25T09:58:59.038420+0000","last_became_active":"2024-04-25T09:58:58.059027+0000","last_became_peered":"2024-04-25T09:58:58.059027+0000","last_unstale":"2024-04-25T09:58:59.038420+0000","last_undegraded":"2024-04-25T09:58:59.038420+0000","last_fullsized":"2024-04-25T09:58:59.038420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:47.517757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445494+0000","last_change":"2024-04-25T09:58:57.034137+0000","last_active":"2024-04-25T09:59:00.445494+0000","last_peered":"2024-04-25T09:59:00.445494+0000","last_clean":"2024-04-25T09:59:00.445494+0000","last_became_active":"2024-04-25T09:58:57.033509+0000","last_became_peered":"2024-04-25T09:58:57.033509+0000","last_unstale":"2024-04-25T09:59:00.445494+0000","last_undegraded":"2024-04-25T09:59:00.445494+0000","last_fullsized":"2024-04-25T09:59:00.445494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:15.716440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445315+0000","last_change":"2024-04-25T09:58:58.051381+0000","last_active":"2024-04-25T09:59:00.445315+0000","last_peered":"2024-04-25T09:59:00.445315+0000","last_clean":"2024-04-25T09:59:00.445315+0000","last_became_active":"2024-04-25T09:58:58.051007+0000","last_became_peered":"2024-04-25T09:58:58.051007+0000","last_unstale":"2024-04-25T09:59:00.445315+0000","last_undegraded":"2024-04-25T09:59:00.445315+0000","last_fullsized":"2024-04-25T09:59:00.445315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.363976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445146+0000","last_change":"2024-04-25T09:58:57.045529+0000","last_active":"2024-04-25T09:59:00.445146+0000","last_peered":"2024-04-25T09:59:00.445146+0000","last_clean":"2024-04-25T09:59:00.445146+0000","last_became_active":"2024-04-25T09:58:57.045229+0000","last_became_peered":"2024-04-25T09:58:57.045229+0000","last_unstale":"2024-04-25T09:59:00.445146+0000","last_undegraded":"2024-04-25T09:59:00.445146+0000","last_fullsized":"2024-04-25T09:59:00.445146+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:01.590514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.039971+0000","last_change":"2024-04-25T09:58:58.041026+0000","last_active":"2024-04-25T09:58:59.039971+0000","last_peered":"2024-04-25T09:58:59.039971+0000","last_clean":"2024-04-25T09:58:59.039971+0000","last_became_active":"2024-04-25T09:58:58.040669+0000","last_became_peered":"2024-04-25T09:58:58.040669+0000","last_unstale":"2024-04-25T09:58:59.039971+0000","last_undegraded":"2024-04-25T09:58:59.039971+0000","last_fullsized":"2024-04-25T09:58:59.039971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:24.191210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.029819+0000","last_change":"2024-04-25T09:58:57.039169+0000","last_active":"2024-04-25T09:59:00.029819+0000","last_peered":"2024-04-25T09:59:00.029819+0000","last_clean":"2024-04-25T09:59:00.029819+0000","last_became_active":"2024-04-25T09:58:57.038892+0000","last_became_peered":"2024-04-25T09:58:57.038892+0000","last_unstale":"2024-04-25T09:59:00.029819+0000","last_undegraded":"2024-04-25T09:59:00.029819+0000","last_fullsized":"2024-04-25T09:59:00.029819+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:50.271465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445236+0000","last_change":"2024-04-25T09:58:58.043163+0000","last_active":"2024-04-25T09:59:00.445236+0000","last_peered":"2024-04-25T09:59:00.445236+0000","last_clean":"2024-04-25T09:59:00.445236+0000","last_became_active":"2024-04-25T09:58:58.042875+0000","last_became_peered":"2024-04-25T09:58:58.042875+0000","last_unstale":"2024-04-25T09:59:00.445236+0000","last_undegraded":"2024-04-25T09:59:00.445236+0000","last_fullsized":"2024-04-25T09:59:00.445236+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:22.769305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445933+0000","last_change":"2024-04-25T09:58:57.040183+0000","last_active":"2024-04-25T09:59:00.445933+0000","last_peered":"2024-04-25T09:59:00.445933+0000","last_clean":"2024-04-25T09:59:00.445933+0000","last_became_active":"2024-04-25T09:58:57.039924+0000","last_became_peered":"2024-04-25T09:58:57.039924+0000","last_unstale":"2024-04-25T09:59:00.445933+0000","last_undegraded":"2024-04-25T09:59:00.445933+0000","last_fullsized":"2024-04-25T09:59:00.445933+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:23.691243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445848+0000","last_change":"2024-04-25T09:58:58.044602+0000","last_active":"2024-04-25T09:59:00.445848+0000","last_peered":"2024-04-25T09:59:00.445848+0000","last_clean":"2024-04-25T09:59:00.445848+0000","last_became_active":"2024-04-25T09:58:58.044335+0000","last_became_peered":"2024-04-25T09:58:58.044335+0000","last_unstale":"2024-04-25T09:59:00.445848+0000","last_undegraded":"2024-04-25T09:59:00.445848+0000","last_fullsized":"2024-04-25T09:59:00.445848+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:19.968532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"88'3","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.040202+0000","last_change":"2024-04-25T09:58:57.045075+0000","last_active":"2024-04-25T09:58:59.040202+0000","last_peered":"2024-04-25T09:58:59.040202+0000","last_clean":"2024-04-25T09:58:59.040202+0000","last_became_active":"2024-04-25T09:58:57.044812+0000","last_became_peered":"2024-04-25T09:58:57.044812+0000","last_unstale":"2024-04-25T09:58:59.040202+0000","last_undegraded":"2024-04-25T09:58:59.040202+0000","last_fullsized":"2024-04-25T09:58:59.040202+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:01.022645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445121+0000","last_change":"2024-04-25T09:58:58.046576+0000","last_active":"2024-04-25T09:59:00.445121+0000","last_peered":"2024-04-25T09:59:00.445121+0000","last_clean":"2024-04-25T09:59:00.445121+0000","last_became_active":"2024-04-25T09:58:58.046304+0000","last_became_peered":"2024-04-25T09:58:58.046304+0000","last_unstale":"2024-04-25T09:59:00.445121+0000","last_undegraded":"2024-04-25T09:59:00.445121+0000","last_fullsized":"2024-04-25T09:59:00.445121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:59.407235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"88'6","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.037968+0000","last_change":"2024-04-25T09:58:57.044690+0000","last_active":"2024-04-25T09:58:59.037968+0000","last_peered":"2024-04-25T09:58:59.037968+0000","last_clean":"2024-04-25T09:58:59.037968+0000","last_became_active":"2024-04-25T09:58:57.044405+0000","last_became_peered":"2024-04-25T09:58:57.044405+0000","last_unstale":"2024-04-25T09:58:59.037968+0000","last_undegraded":"2024-04-25T09:58:59.037968+0000","last_fullsized":"2024-04-25T09:58:59.037968+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:18.964094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445795+0000","last_change":"2024-04-25T09:58:58.044254+0000","last_active":"2024-04-25T09:59:00.445795+0000","last_peered":"2024-04-25T09:59:00.445795+0000","last_clean":"2024-04-25T09:59:00.445795+0000","last_became_active":"2024-04-25T09:58:58.044024+0000","last_became_peered":"2024-04-25T09:58:58.044024+0000","last_unstale":"2024-04-25T09:59:00.445795+0000","last_undegraded":"2024-04-25T09:59:00.445795+0000","last_fullsized":"2024-04-25T09:59:00.445795+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:27.776323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445687+0000","last_change":"2024-04-25T09:58:57.045507+0000","last_active":"2024-04-25T09:59:00.445687+0000","last_peered":"2024-04-25T09:59:00.445687+0000","last_clean":"2024-04-25T09:59:00.445687+0000","last_became_active":"2024-04-25T09:58:57.045211+0000","last_became_peered":"2024-04-25T09:58:57.045211+0000","last_unstale":"2024-04-25T09:59:00.445687+0000","last_undegraded":"2024-04-25T09:59:00.445687+0000","last_fullsized":"2024-04-25T09:59:00.445687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.402934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.039508+0000","last_change":"2024-04-25T09:58:58.036859+0000","last_active":"2024-04-25T09:58:59.039508+0000","last_peered":"2024-04-25T09:58:59.039508+0000","last_clean":"2024-04-25T09:58:59.039508+0000","last_became_active":"2024-04-25T09:58:58.036605+0000","last_became_peered":"2024-04-25T09:58:58.036605+0000","last_unstale":"2024-04-25T09:58:59.039508+0000","last_undegraded":"2024-04-25T09:58:59.039508+0000","last_fullsized":"2024-04-25T09:58:59.039508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:59.407235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.037942+0000","last_change":"2024-04-25T09:58:57.042334+0000","last_active":"2024-04-25T09:58:59.037942+0000","last_peered":"2024-04-25T09:58:59.037942+0000","last_clean":"2024-04-25T09:58:59.037942+0000","last_became_active":"2024-04-25T09:58:57.042071+0000","last_became_peered":"2024-04-25T09:58:57.042071+0000","last_unstale":"2024-04-25T09:58:59.037942+0000","last_undegraded":"2024-04-25T09:58:59.037942+0000","last_fullsized":"2024-04-25T09:58:59.037942+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:51.372657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445962+0000","last_change":"2024-04-25T09:58:58.048529+0000","last_active":"2024-04-25T09:59:00.445962+0000","last_peered":"2024-04-25T09:59:00.445962+0000","last_clean":"2024-04-25T09:59:00.445962+0000","last_became_active":"2024-04-25T09:58:58.048290+0000","last_became_peered":"2024-04-25T09:58:58.048290+0000","last_unstale":"2024-04-25T09:59:00.445962+0000","last_undegraded":"2024-04-25T09:59:00.445962+0000","last_fullsized":"2024-04-25T09:59:00.445962+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:32.420521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445172+0000","last_change":"2024-04-25T09:58:57.046367+0000","last_active":"2024-04-25T09:59:00.445172+0000","last_peered":"2024-04-25T09:59:00.445172+0000","last_clean":"2024-04-25T09:59:00.445172+0000","last_became_active":"2024-04-25T09:58:57.046089+0000","last_became_peered":"2024-04-25T09:58:57.046089+0000","last_unstale":"2024-04-25T09:59:00.445172+0000","last_undegraded":"2024-04-25T09:59:00.445172+0000","last_fullsized":"2024-04-25T09:59:00.445172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:35.986283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445169+0000","last_change":"2024-04-25T09:58:58.036629+0000","last_active":"2024-04-25T09:59:00.445169+0000","last_peered":"2024-04-25T09:59:00.445169+0000","last_clean":"2024-04-25T09:59:00.445169+0000","last_became_active":"2024-04-25T09:58:58.036363+0000","last_became_peered":"2024-04-25T09:58:58.036363+0000","last_unstale":"2024-04-25T09:59:00.445169+0000","last_undegraded":"2024-04-25T09:59:00.445169+0000","last_fullsized":"2024-04-25T09:59:00.445169+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:08.345907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"88'8","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445904+0000","last_change":"2024-04-25T09:58:57.038473+0000","last_active":"2024-04-25T09:59:00.445904+0000","last_peered":"2024-04-25T09:59:00.445904+0000","last_clean":"2024-04-25T09:59:00.445904+0000","last_became_active":"2024-04-25T09:58:57.038170+0000","last_became_peered":"2024-04-25T09:58:57.038170+0000","last_unstale":"2024-04-25T09:59:00.445904+0000","last_undegraded":"2024-04-25T09:59:00.445904+0000","last_fullsized":"2024-04-25T09:59:00.445904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:01.022645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.038002+0000","last_change":"2024-04-25T09:58:58.050508+0000","last_active":"2024-04-25T09:58:59.038002+0000","last_peered":"2024-04-25T09:58:59.038002+0000","last_clean":"2024-04-25T09:58:59.038002+0000","last_became_active":"2024-04-25T09:58:58.050185+0000","last_became_peered":"2024-04-25T09:58:58.050185+0000","last_unstale":"2024-04-25T09:58:59.038002+0000","last_undegraded":"2024-04-25T09:58:59.038002+0000","last_fullsized":"2024-04-25T09:58:59.038002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:39.736489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445759+0000","last_change":"2024-04-25T09:58:57.040136+0000","last_active":"2024-04-25T09:59:00.445759+0000","last_peered":"2024-04-25T09:59:00.445759+0000","last_clean":"2024-04-25T09:59:00.445759+0000","last_became_active":"2024-04-25T09:58:57.039832+0000","last_became_peered":"2024-04-25T09:58:57.039832+0000","last_unstale":"2024-04-25T09:59:00.445759+0000","last_undegraded":"2024-04-25T09:59:00.445759+0000","last_fullsized":"2024-04-25T09:59:00.445759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:40.097607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445792+0000","last_change":"2024-04-25T09:58:58.036701+0000","last_active":"2024-04-25T09:59:00.445792+0000","last_peered":"2024-04-25T09:59:00.445792+0000","last_clean":"2024-04-25T09:59:00.445792+0000","last_became_active":"2024-04-25T09:58:58.036433+0000","last_became_peered":"2024-04-25T09:58:58.036433+0000","last_unstale":"2024-04-25T09:59:00.445792+0000","last_undegraded":"2024-04-25T09:59:00.445792+0000","last_fullsized":"2024-04-25T09:59:00.445792+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:12.363976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445317+0000","last_change":"2024-04-25T09:58:57.039566+0000","last_active":"2024-04-25T09:59:00.445317+0000","last_peered":"2024-04-25T09:59:00.445317+0000","last_clean":"2024-04-25T09:59:00.445317+0000","last_became_active":"2024-04-25T09:58:57.039279+0000","last_became_peered":"2024-04-25T09:58:57.039279+0000","last_unstale":"2024-04-25T09:59:00.445317+0000","last_undegraded":"2024-04-25T09:59:00.445317+0000","last_fullsized":"2024-04-25T09:59:00.445317+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:25.145193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.040229+0000","last_change":"2024-04-25T09:58:58.043472+0000","last_active":"2024-04-25T09:58:59.040229+0000","last_peered":"2024-04-25T09:58:59.040229+0000","last_clean":"2024-04-25T09:58:59.040229+0000","last_became_active":"2024-04-25T09:58:58.042833+0000","last_became_peered":"2024-04-25T09:58:58.042833+0000","last_unstale":"2024-04-25T09:58:59.040229+0000","last_undegraded":"2024-04-25T09:58:59.040229+0000","last_fullsized":"2024-04-25T09:58:59.040229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:52.858541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.040285+0000","last_change":"2024-04-25T09:58:57.043428+0000","last_active":"2024-04-25T09:58:59.040285+0000","last_peered":"2024-04-25T09:58:59.040285+0000","last_clean":"2024-04-25T09:58:59.040285+0000","last_became_active":"2024-04-25T09:58:57.043158+0000","last_became_peered":"2024-04-25T09:58:57.043158+0000","last_unstale":"2024-04-25T09:58:59.040285+0000","last_undegraded":"2024-04-25T09:58:59.040285+0000","last_fullsized":"2024-04-25T09:58:59.040285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:35.986283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.038544+0000","last_change":"2024-04-25T09:58:58.059604+0000","last_active":"2024-04-25T09:58:59.038544+0000","last_peered":"2024-04-25T09:58:59.038544+0000","last_clean":"2024-04-25T09:58:59.038544+0000","last_became_active":"2024-04-25T09:58:58.059309+0000","last_became_peered":"2024-04-25T09:58:58.059309+0000","last_unstale":"2024-04-25T09:58:59.038544+0000","last_undegraded":"2024-04-25T09:58:59.038544+0000","last_fullsized":"2024-04-25T09:58:59.038544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:46.285108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.029369+0000","last_change":"2024-04-25T09:58:57.036704+0000","last_active":"2024-04-25T09:59:00.029369+0000","last_peered":"2024-04-25T09:59:00.029369+0000","last_clean":"2024-04-25T09:59:00.029369+0000","last_became_active":"2024-04-25T09:58:57.036421+0000","last_became_peered":"2024-04-25T09:58:57.036421+0000","last_unstale":"2024-04-25T09:59:00.029369+0000","last_undegraded":"2024-04-25T09:59:00.029369+0000","last_fullsized":"2024-04-25T09:59:00.029369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:49.027934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.029482+0000","last_change":"2024-04-25T09:58:58.045857+0000","last_active":"2024-04-25T09:59:00.029482+0000","last_peered":"2024-04-25T09:59:00.029482+0000","last_clean":"2024-04-25T09:59:00.029482+0000","last_became_active":"2024-04-25T09:58:58.045538+0000","last_became_peered":"2024-04-25T09:58:58.045538+0000","last_unstale":"2024-04-25T09:59:00.029482+0000","last_undegraded":"2024-04-25T09:59:00.029482+0000","last_fullsized":"2024-04-25T09:59:00.029482+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:21.734573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.446115+0000","last_change":"2024-04-25T09:58:57.037903+0000","last_active":"2024-04-25T09:59:00.446115+0000","last_peered":"2024-04-25T09:59:00.446115+0000","last_clean":"2024-04-25T09:59:00.446115+0000","last_became_active":"2024-04-25T09:58:57.037640+0000","last_became_peered":"2024-04-25T09:58:57.037640+0000","last_unstale":"2024-04-25T09:59:00.446115+0000","last_undegraded":"2024-04-25T09:59:00.446115+0000","last_fullsized":"2024-04-25T09:59:00.446115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:34.625977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.029458+0000","last_change":"2024-04-25T09:58:58.057929+0000","last_active":"2024-04-25T09:59:00.029458+0000","last_peered":"2024-04-25T09:59:00.029458+0000","last_clean":"2024-04-25T09:59:00.029458+0000","last_became_active":"2024-04-25T09:58:58.057630+0000","last_became_peered":"2024-04-25T09:58:58.057630+0000","last_unstale":"2024-04-25T09:59:00.029458+0000","last_undegraded":"2024-04-25T09:59:00.029458+0000","last_fullsized":"2024-04-25T09:59:00.029458+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:24.695680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.029618+0000","last_change":"2024-04-25T09:58:57.043679+0000","last_active":"2024-04-25T09:59:00.029618+0000","last_peered":"2024-04-25T09:59:00.029618+0000","last_clean":"2024-04-25T09:59:00.029618+0000","last_became_active":"2024-04-25T09:58:57.043368+0000","last_became_peered":"2024-04-25T09:58:57.043368+0000","last_unstale":"2024-04-25T09:59:00.029618+0000","last_undegraded":"2024-04-25T09:59:00.029618+0000","last_fullsized":"2024-04-25T09:59:00.029618+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:11.359539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.037809+0000","last_change":"2024-04-25T09:58:58.055481+0000","last_active":"2024-04-25T09:58:59.037809+0000","last_peered":"2024-04-25T09:58:59.037809+0000","last_clean":"2024-04-25T09:58:59.037809+0000","last_became_active":"2024-04-25T09:58:58.055215+0000","last_became_peered":"2024-04-25T09:58:58.055215+0000","last_unstale":"2024-04-25T09:58:59.037809+0000","last_undegraded":"2024-04-25T09:58:59.037809+0000","last_fullsized":"2024-04-25T09:58:59.037809+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:03.004338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.444993+0000","last_change":"2024-04-25T09:58:57.037357+0000","last_active":"2024-04-25T09:59:00.444993+0000","last_peered":"2024-04-25T09:59:00.444993+0000","last_clean":"2024-04-25T09:59:00.444993+0000","last_became_active":"2024-04-25T09:58:57.037087+0000","last_became_peered":"2024-04-25T09:58:57.037087+0000","last_unstale":"2024-04-25T09:59:00.444993+0000","last_undegraded":"2024-04-25T09:59:00.444993+0000","last_fullsized":"2024-04-25T09:59:00.444993+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:19.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445919+0000","last_change":"2024-04-25T09:58:58.039350+0000","last_active":"2024-04-25T09:59:00.445919+0000","last_peered":"2024-04-25T09:59:00.445919+0000","last_clean":"2024-04-25T09:59:00.445919+0000","last_became_active":"2024-04-25T09:58:58.039086+0000","last_became_peered":"2024-04-25T09:58:58.039086+0000","last_unstale":"2024-04-25T09:59:00.445919+0000","last_undegraded":"2024-04-25T09:59:00.445919+0000","last_fullsized":"2024-04-25T09:59:00.445919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:50.032371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445509+0000","last_change":"2024-04-25T09:58:57.040746+0000","last_active":"2024-04-25T09:59:00.445509+0000","last_peered":"2024-04-25T09:59:00.445509+0000","last_clean":"2024-04-25T09:59:00.445509+0000","last_became_active":"2024-04-25T09:58:57.040447+0000","last_became_peered":"2024-04-25T09:58:57.040447+0000","last_unstale":"2024-04-25T09:59:00.445509+0000","last_undegraded":"2024-04-25T09:59:00.445509+0000","last_fullsized":"2024-04-25T09:59:00.445509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:28.736087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.038140+0000","last_change":"2024-04-25T09:58:58.048379+0000","last_active":"2024-04-25T09:58:59.038140+0000","last_peered":"2024-04-25T09:58:59.038140+0000","last_clean":"2024-04-25T09:58:59.038140+0000","last_became_active":"2024-04-25T09:58:58.048104+0000","last_became_peered":"2024-04-25T09:58:58.048104+0000","last_unstale":"2024-04-25T09:58:59.038140+0000","last_undegraded":"2024-04-25T09:58:59.038140+0000","last_fullsized":"2024-04-25T09:58:59.038140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:28.078357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.040409+0000","last_change":"2024-04-25T09:58:57.039890+0000","last_active":"2024-04-25T09:58:59.040409+0000","last_peered":"2024-04-25T09:58:59.040409+0000","last_clean":"2024-04-25T09:58:59.040409+0000","last_became_active":"2024-04-25T09:58:57.039428+0000","last_became_peered":"2024-04-25T09:58:57.039428+0000","last_unstale":"2024-04-25T09:58:59.040409+0000","last_undegraded":"2024-04-25T09:58:59.040409+0000","last_fullsized":"2024-04-25T09:58:59.040409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:27.846107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.038167+0000","last_change":"2024-04-25T09:58:58.054292+0000","last_active":"2024-04-25T09:58:59.038167+0000","last_peered":"2024-04-25T09:58:59.038167+0000","last_clean":"2024-04-25T09:58:59.038167+0000","last_became_active":"2024-04-25T09:58:58.054000+0000","last_became_peered":"2024-04-25T09:58:58.054000+0000","last_unstale":"2024-04-25T09:58:59.038167+0000","last_undegraded":"2024-04-25T09:58:59.038167+0000","last_fullsized":"2024-04-25T09:58:59.038167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:47.432997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.040287+0000","last_change":"2024-04-25T09:58:57.045052+0000","last_active":"2024-04-25T09:58:59.040287+0000","last_peered":"2024-04-25T09:58:59.040287+0000","last_clean":"2024-04-25T09:58:59.040287+0000","last_became_active":"2024-04-25T09:58:57.044719+0000","last_became_peered":"2024-04-25T09:58:57.044719+0000","last_unstale":"2024-04-25T09:58:59.040287+0000","last_undegraded":"2024-04-25T09:58:59.040287+0000","last_fullsized":"2024-04-25T09:58:59.040287+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:44.391697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.029751+0000","last_change":"2024-04-25T09:58:58.049048+0000","last_active":"2024-04-25T09:59:00.029751+0000","last_peered":"2024-04-25T09:59:00.029751+0000","last_clean":"2024-04-25T09:59:00.029751+0000","last_became_active":"2024-04-25T09:58:58.048767+0000","last_became_peered":"2024-04-25T09:58:58.048767+0000","last_unstale":"2024-04-25T09:59:00.029751+0000","last_undegraded":"2024-04-25T09:59:00.029751+0000","last_fullsized":"2024-04-25T09:59:00.029751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:05.365279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445981+0000","last_change":"2024-04-25T09:58:57.042212+0000","last_active":"2024-04-25T09:59:00.445981+0000","last_peered":"2024-04-25T09:59:00.445981+0000","last_clean":"2024-04-25T09:59:00.445981+0000","last_became_active":"2024-04-25T09:58:57.041931+0000","last_became_peered":"2024-04-25T09:58:57.041931+0000","last_unstale":"2024-04-25T09:59:00.445981+0000","last_undegraded":"2024-04-25T09:59:00.445981+0000","last_fullsized":"2024-04-25T09:59:00.445981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:19.563627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.444823+0000","last_change":"2024-04-25T09:58:58.052734+0000","last_active":"2024-04-25T09:59:00.444823+0000","last_peered":"2024-04-25T09:59:00.444823+0000","last_clean":"2024-04-25T09:59:00.444823+0000","last_became_active":"2024-04-25T09:58:58.052484+0000","last_became_peered":"2024-04-25T09:58:58.052484+0000","last_unstale":"2024-04-25T09:59:00.444823+0000","last_undegraded":"2024-04-25T09:59:00.444823+0000","last_fullsized":"2024-04-25T09:59:00.444823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:22.767767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.038029+0000","last_change":"2024-04-25T09:58:57.043114+0000","last_active":"2024-04-25T09:58:59.038029+0000","last_peered":"2024-04-25T09:58:59.038029+0000","last_clean":"2024-04-25T09:58:59.038029+0000","last_became_active":"2024-04-25T09:58:57.042800+0000","last_became_peered":"2024-04-25T09:58:57.042800+0000","last_unstale":"2024-04-25T09:58:59.038029+0000","last_undegraded":"2024-04-25T09:58:59.038029+0000","last_fullsized":"2024-04-25T09:58:59.038029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:01.590514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445274+0000","last_change":"2024-04-25T09:58:58.057965+0000","last_active":"2024-04-25T09:59:00.445274+0000","last_peered":"2024-04-25T09:59:00.445274+0000","last_clean":"2024-04-25T09:59:00.445274+0000","last_became_active":"2024-04-25T09:58:58.057678+0000","last_became_peered":"2024-04-25T09:58:58.057678+0000","last_unstale":"2024-04-25T09:59:00.445274+0000","last_undegraded":"2024-04-25T09:59:00.445274+0000","last_fullsized":"2024-04-25T09:59:00.445274+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:52.377095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.029637+0000","last_change":"2024-04-25T09:58:57.036703+0000","last_active":"2024-04-25T09:59:00.029637+0000","last_peered":"2024-04-25T09:59:00.029637+0000","last_clean":"2024-04-25T09:59:00.029637+0000","last_became_active":"2024-04-25T09:58:57.036409+0000","last_became_peered":"2024-04-25T09:58:57.036409+0000","last_unstale":"2024-04-25T09:59:00.029637+0000","last_undegraded":"2024-04-25T09:59:00.029637+0000","last_fullsized":"2024-04-25T09:59:00.029637+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:23.186772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445277+0000","last_change":"2024-04-25T09:58:58.058026+0000","last_active":"2024-04-25T09:59:00.445277+0000","last_peered":"2024-04-25T09:59:00.445277+0000","last_clean":"2024-04-25T09:59:00.445277+0000","last_became_active":"2024-04-25T09:58:58.057666+0000","last_became_peered":"2024-04-25T09:58:58.057666+0000","last_unstale":"2024-04-25T09:59:00.445277+0000","last_undegraded":"2024-04-25T09:59:00.445277+0000","last_fullsized":"2024-04-25T09:59:00.445277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:55.221010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445399+0000","last_change":"2024-04-25T09:58:57.039451+0000","last_active":"2024-04-25T09:59:00.445399+0000","last_peered":"2024-04-25T09:59:00.445399+0000","last_clean":"2024-04-25T09:59:00.445399+0000","last_became_active":"2024-04-25T09:58:57.039193+0000","last_became_peered":"2024-04-25T09:58:57.039193+0000","last_unstale":"2024-04-25T09:59:00.445399+0000","last_undegraded":"2024-04-25T09:59:00.445399+0000","last_fullsized":"2024-04-25T09:59:00.445399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:53.809246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:58:59.037834+0000","last_change":"2024-04-25T09:58:58.053260+0000","last_active":"2024-04-25T09:58:59.037834+0000","last_peered":"2024-04-25T09:58:59.037834+0000","last_clean":"2024-04-25T09:58:59.037834+0000","last_became_active":"2024-04-25T09:58:58.052035+0000","last_became_peered":"2024-04-25T09:58:58.052035+0000","last_unstale":"2024-04-25T09:58:59.037834+0000","last_undegraded":"2024-04-25T09:58:59.037834+0000","last_fullsized":"2024-04-25T09:58:59.037834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:04.394566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445106+0000","last_change":"2024-04-25T09:58:57.035195+0000","last_active":"2024-04-25T09:59:00.445106+0000","last_peered":"2024-04-25T09:59:00.445106+0000","last_clean":"2024-04-25T09:59:00.445106+0000","last_became_active":"2024-04-25T09:58:57.034665+0000","last_became_peered":"2024-04-25T09:58:57.034665+0000","last_unstale":"2024-04-25T09:59:00.445106+0000","last_undegraded":"2024-04-25T09:59:00.445106+0000","last_fullsized":"2024-04-25T09:59:00.445106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:18.964094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445408+0000","last_change":"2024-04-25T09:58:58.039906+0000","last_active":"2024-04-25T09:59:00.445408+0000","last_peered":"2024-04-25T09:59:00.445408+0000","last_clean":"2024-04-25T09:59:00.445408+0000","last_became_active":"2024-04-25T09:58:58.039538+0000","last_became_peered":"2024-04-25T09:58:58.039538+0000","last_unstale":"2024-04-25T09:59:00.445408+0000","last_undegraded":"2024-04-25T09:59:00.445408+0000","last_fullsized":"2024-04-25T09:59:00.445408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:57.013386+0000","last_clean_scrub_stamp":"2024-04-25T09:58:57.013386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:37.444639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445560+0000","last_change":"2024-04-25T09:58:57.042156+0000","last_active":"2024-04-25T09:59:00.445560+0000","last_peered":"2024-04-25T09:59:00.445560+0000","last_clean":"2024-04-25T09:59:00.445560+0000","last_became_active":"2024-04-25T09:58:57.040423+0000","last_became_peered":"2024-04-25T09:58:57.040423+0000","last_unstale":"2024-04-25T09:59:00.445560+0000","last_undegraded":"2024-04-25T09:59:00.445560+0000","last_fullsized":"2024-04-25T09:59:00.445560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:56.008948+0000","last_clean_scrub_stamp":"2024-04-25T09:58:56.008948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:39.639654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":297,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T09:59:00.445272+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T09:59:00.445272+0000","last_peered":"2024-04-25T09:59:00.445272+0000","last_clean":"2024-04-25T09:59:00.445272+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T09:59:00.445272+0000","last_undegraded":"2024-04-25T09:59:00.445272+0000","last_fullsized":"2024-04-25T09:59:00.445272+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339000,"statfs":{"total":96636764160,"available":96603136000,"internally_reserved":0,"allocated":1699840,"data_stored":1093880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1118208,"data_stored":505672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1105920,"data_stored":503616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574925,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":503456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":503526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1716224,"data_stored":1096142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:59:05.670 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:59:05.670 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:59:05.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.807+0000 7fa5a8bdb640 1 Processor -- start 2024-04-25T09:59:05.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.807+0000 7fa5a8bdb640 1 -- start start 2024-04-25T09:59:05.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa5a8bdb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa5a415dbf0 0x7fa5a41617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa5a8bdb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa5a40b9370 0x7fa5a40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa5a8bdb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a4161dc0 con 0x7fa5a40cec80 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa5a8bdb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a40cf0d0 con 0x7fa5a40b9370 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa5a8bdb640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a40d1ee0 con 0x7fa5a415dbf0 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa59bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa5a40b9370 0x7fa5a40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa59bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa5a40b9370 0x7fa5a40ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58328/0 (socket says 172.21.15.139:58328) 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa59bfff640 1 -- 172.21.15.139:0/186909025 learned_addr learned my addr 172.21.15.139:0/186909025 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa5a259c640 1 --2- 172.21.15.139:0/186909025 >> v2:172.21.15.150:3300/0 conn(0x7fa5a415dbf0 0x7fa5a41617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa59bfff640 1 -- 172.21.15.139:0/186909025 >> v2:172.21.15.150:3300/0 conn(0x7fa5a415dbf0 msgr2=0x7fa5a41617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa59bfff640 1 --2- 172.21.15.139:0/186909025 >> v2:172.21.15.150:3300/0 conn(0x7fa5a415dbf0 0x7fa5a41617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa59bfff640 1 -- 172.21.15.139:0/186909025 >> v1:172.21.15.150:6789/0 conn(0x7fa5a40cec80 legacy=0x7fa5a406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.808+0000 7fa59bfff640 1 -- 172.21.15.139:0/186909025 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a41620a0 con 0x7fa5a40b9370 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.809+0000 7fa59bfff640 1 --2- 172.21.15.139:0/186909025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a40b9370 0x7fa5a40ce740 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa5a406d330 tx=0x7fa58c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ea73100ea93a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.809+0000 7fa5a1d9b640 1 -- 172.21.15.139:0/186909025 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c037070 con 0x7fa5a40b9370 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.809+0000 7fa5a1d9b640 1 -- 172.21.15.139:0/186909025 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c032440 con 0x7fa5a40b9370 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.809+0000 7fa5a1d9b640 1 -- 172.21.15.139:0/186909025 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c032810 con 0x7fa5a40b9370 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.809+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/186909025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a40b9370 msgr2=0x7fa5a40ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.809+0000 7fa5a8bdb640 1 --2- 172.21.15.139:0/186909025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a40b9370 0x7fa5a40ce740 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa5a406d330 tx=0x7fa58c02a230 comp rx=0 tx=0).stop 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.809+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/186909025 shutdown_connections 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.809+0000 7fa5a8bdb640 1 --2- 172.21.15.139:0/186909025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a40b9370 0x7fa5a40ce740 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.809+0000 7fa5a8bdb640 1 --2- 172.21.15.139:0/186909025 >> v2:172.21.15.150:3300/0 conn(0x7fa5a415dbf0 0x7fa5a41617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.810+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/186909025 >> 172.21.15.139:0/186909025 conn(0x7fa5a40b5480 msgr2=0x7fa5a40b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.810+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/186909025 shutdown_connections 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.810+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/186909025 wait complete. 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.811+0000 7fa5a8bdb640 1 Processor -- start 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.811+0000 7fa5a8bdb640 1 -- start start 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a8bdb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5a40b9370 0x7fa5a40c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a8bdb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a40cec80 0x7fa5a4153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a8bdb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415dbf0 0x7fa5a4155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a8bdb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a415dfd0 con 0x7fa5a415dbf0 2024-04-25T09:59:05.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a8bdb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a4162660 con 0x7fa5a40b9370 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a8bdb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a4154290 con 0x7fa5a40cec80 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a2d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415dbf0 0x7fa5a4155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a2d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415dbf0 0x7fa5a4155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58344/0 (socket says 172.21.15.139:58344) 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a2d9d640 1 -- 172.21.15.139:0/549404961 learned_addr learned my addr 172.21.15.139:0/549404961 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a259c640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5a40b9370 0x7fa5a40c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa59bfff640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a40cec80 0x7fa5a4153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a2d9d640 1 -- 172.21.15.139:0/549404961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a40cec80 msgr2=0x7fa5a4153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a2d9d640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a40cec80 0x7fa5a4153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a2d9d640 1 -- 172.21.15.139:0/549404961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5a40b9370 msgr2=0x7fa5a40c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a2d9d640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5a40b9370 0x7fa5a40c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a2d9d640 1 -- 172.21.15.139:0/549404961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a4154570 con 0x7fa5a415dbf0 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a2d9d640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415dbf0 0x7fa5a4155b50 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00c780 tx=0x7fa59c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa59affd640 1 -- 172.21.15.139:0/549404961 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c004b60 con 0x7fa5a415dbf0 2024-04-25T09:59:05.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa59affd640 1 -- 172.21.15.139:0/549404961 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c004d00 con 0x7fa5a415dbf0 2024-04-25T09:59:05.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/549404961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a4154850 con 0x7fa5a415dbf0 2024-04-25T09:59:05.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa59affd640 1 -- 172.21.15.139:0/549404961 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c00aa90 con 0x7fa5a415dbf0 2024-04-25T09:59:05.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.812+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/549404961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a4154e10 con 0x7fa5a415dbf0 2024-04-25T09:59:05.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.813+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/549404961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a4074520 con 0x7fa5a415dbf0 2024-04-25T09:59:05.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.814+0000 7fa59affd640 1 -- 172.21.15.139:0/549404961 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa59c00ac30 con 0x7fa5a415dbf0 2024-04-25T09:59:05.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.814+0000 7fa59affd640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa58406b4d0 0x7fa58406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:05.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.814+0000 7fa59affd640 1 -- 172.21.15.139:0/549404961 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa59c08a990 con 0x7fa5a415dbf0 2024-04-25T09:59:05.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.814+0000 7fa5a259c640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa58406b4d0 0x7fa58406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:05.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.814+0000 7fa5a259c640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa58406b4d0 0x7fa58406d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa594006fd0 tx=0x7fa594008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:05.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:05.818+0000 7fa59affd640 1 -- 172.21.15.139:0/549404961 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa59c059680 con 0x7fa5a415dbf0 2024-04-25T09:59:06.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.054+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/549404961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa5a415d410 con 0x7fa5a415dbf0 2024-04-25T09:59:06.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.056+0000 7fa59affd640 1 -- 172.21.15.139:0/549404961 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa59c058f80 con 0x7fa5a415dbf0 2024-04-25T09:59:06.060 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2024-04-25T09:59:06.060 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg== 2024-04-25T09:59:06.060 INFO:teuthology.orchestra.run.smithi139.stdout: caps mds = "allow" 2024-04-25T09:59:06.060 INFO:teuthology.orchestra.run.smithi139.stdout: caps mon = "allow r" 2024-04-25T09:59:06.060 INFO:teuthology.orchestra.run.smithi139.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:59:06.061 INFO:teuthology.orchestra.run.smithi139.stderr:updated caps for client.0 2024-04-25T09:59:06.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.059+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/549404961 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa58406b4d0 msgr2=0x7fa58406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.059+0000 7fa5a8bdb640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa58406b4d0 0x7fa58406d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa594006fd0 tx=0x7fa594008040 comp rx=0 tx=0).stop 2024-04-25T09:59:06.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.059+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/549404961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415dbf0 msgr2=0x7fa5a4155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.059+0000 7fa5a8bdb640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415dbf0 0x7fa5a4155b50 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00c780 tx=0x7fa59c00cc40 comp rx=0 tx=0).stop 2024-04-25T09:59:06.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.059+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/549404961 shutdown_connections 2024-04-25T09:59:06.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.059+0000 7fa5a8bdb640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415dbf0 0x7fa5a4155b50 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.059+0000 7fa5a8bdb640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa58406b4d0 0x7fa58406d990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.059+0000 7fa5a8bdb640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a40cec80 0x7fa5a4153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.059+0000 7fa5a8bdb640 1 --2- 172.21.15.139:0/549404961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5a40b9370 0x7fa5a40c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.059+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/549404961 >> 172.21.15.139:0/549404961 conn(0x7fa5a40b5480 msgr2=0x7fa5a40b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:06.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.059+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/549404961 shutdown_connections 2024-04-25T09:59:06.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.059+0000 7fa5a8bdb640 1 -- 172.21.15.139:0/549404961 wait complete. 2024-04-25T09:59:06.074 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:06.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.168+0000 7f4c286fc640 1 Processor -- start 2024-04-25T09:59:06.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.168+0000 7f4c286fc640 1 -- start start 2024-04-25T09:59:06.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.169+0000 7f4c286fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c200cf900 0x7f4c200d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.169+0000 7f4c286fc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c200cfe70 con 0x7f4c2015efb0 2024-04-25T09:59:06.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.169+0000 7f4c286fc640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c200d0150 con 0x7f4c20099a50 2024-04-25T09:59:06.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.169+0000 7f4c286fc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c200d0a80 con 0x7f4c200cf900 2024-04-25T09:59:06.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.169+0000 7f4c25c97640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c200cf900 0x7f4c200d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.169+0000 7f4c26498640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4c2015efb0 0x7f4c2006c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53574/0 (socket says 172.21.15.139:53574) 2024-04-25T09:59:06.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.169+0000 7f4c26498640 1 -- 172.21.15.139:0/11146530 learned_addr learned my addr 172.21.15.139:0/11146530 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:06.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.169+0000 7f4c25496640 1 -- 172.21.15.139:0/11146530 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819329831 0 0) 0x7f4c200cfe70 con 0x7f4c2015efb0 2024-04-25T09:59:06.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.169+0000 7f4c25496640 1 -- 172.21.15.139:0/11146530 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c0c003520 con 0x7f4c2015efb0 2024-04-25T09:59:06.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.169+0000 7f4c25c97640 1 -- 172.21.15.139:0/11146530 >> v1:172.21.15.150:6789/0 conn(0x7f4c20099a50 legacy=0x7f4c200cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.169+0000 7f4c25c97640 1 -- 172.21.15.139:0/11146530 >> v1:172.21.15.139:6789/0 conn(0x7f4c2015efb0 legacy=0x7f4c2006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.169+0000 7f4c25c97640 1 -- 172.21.15.139:0/11146530 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c200d0430 con 0x7f4c200cf900 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.170+0000 7f4c25c97640 1 --2- 172.21.15.139:0/11146530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c200cf900 0x7f4c200d0f10 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f4c100097a0 tx=0x7f4c1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf4e18418338260 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.170+0000 7f4c25496640 1 -- 172.21.15.139:0/11146530 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10037070 con 0x7f4c200cf900 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.170+0000 7f4c25496640 1 -- 172.21.15.139:0/11146530 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c10032440 con 0x7f4c200cf900 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.170+0000 7f4c25496640 1 -- 172.21.15.139:0/11146530 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10032810 con 0x7f4c200cf900 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.170+0000 7f4c286fc640 1 -- 172.21.15.139:0/11146530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c200cf900 msgr2=0x7f4c200d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.170+0000 7f4c286fc640 1 --2- 172.21.15.139:0/11146530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c200cf900 0x7f4c200d0f10 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f4c100097a0 tx=0x7f4c1002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.171+0000 7f4c286fc640 1 -- 172.21.15.139:0/11146530 shutdown_connections 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.171+0000 7f4c286fc640 1 --2- 172.21.15.139:0/11146530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c200cf900 0x7f4c200d0f10 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.171+0000 7f4c286fc640 1 -- 172.21.15.139:0/11146530 >> 172.21.15.139:0/11146530 conn(0x7f4c200d1790 msgr2=0x7f4c200d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.171+0000 7f4c286fc640 1 -- 172.21.15.139:0/11146530 shutdown_connections 2024-04-25T09:59:06.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.171+0000 7f4c286fc640 1 -- 172.21.15.139:0/11146530 wait complete. 2024-04-25T09:59:06.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.172+0000 7f4c286fc640 1 Processor -- start 2024-04-25T09:59:06.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.172+0000 7f4c286fc640 1 -- start start 2024-04-25T09:59:06.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.172+0000 7f4c286fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c20099a50 0x7f4c201f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.172+0000 7f4c286fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c200cf900 0x7f4c20159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.172+0000 7f4c286fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c2015efb0 0x7f4c20159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.172+0000 7f4c286fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c20099e80 con 0x7f4c200cf900 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.172+0000 7f4c286fc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2006cf50 con 0x7f4c2015efb0 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.172+0000 7f4c286fc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2009a000 con 0x7f4c20099a50 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.172+0000 7f4c25c97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c200cf900 0x7f4c20159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c25c97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c200cf900 0x7f4c20159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58362/0 (socket says 172.21.15.139:58362) 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c25c97640 1 -- 172.21.15.139:0/1366661346 learned_addr learned my addr 172.21.15.139:0/1366661346 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c25c97640 1 -- 172.21.15.139:0/1366661346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c20099a50 msgr2=0x7f4c201f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c26c99640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c2015efb0 0x7f4c20159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c25c97640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c20099a50 0x7f4c201f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c25c97640 1 -- 172.21.15.139:0/1366661346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c2015efb0 msgr2=0x7f4c20159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c25c97640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c2015efb0 0x7f4c20159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c25c97640 1 -- 172.21.15.139:0/1366661346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c201f4ec0 con 0x7f4c200cf900 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c25c97640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c200cf900 0x7f4c20159610 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f4c10035040 tx=0x7f4c100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c16ffd640 1 -- 172.21.15.139:0/1366661346 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10037040 con 0x7f4c200cf900 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c16ffd640 1 -- 172.21.15.139:0/1366661346 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c10030500 con 0x7f4c200cf900 2024-04-25T09:59:06.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c286fc640 1 -- 172.21.15.139:0/1366661346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c201f7690 con 0x7f4c200cf900 2024-04-25T09:59:06.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c16ffd640 1 -- 172.21.15.139:0/1366661346 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10030910 con 0x7f4c200cf900 2024-04-25T09:59:06.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.173+0000 7f4c286fc640 1 -- 172.21.15.139:0/1366661346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c201f7c50 con 0x7f4c200cf900 2024-04-25T09:59:06.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.174+0000 7f4c286fc640 1 -- 172.21.15.139:0/1366661346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c20074520 con 0x7f4c200cf900 2024-04-25T09:59:06.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.175+0000 7f4c16ffd640 1 -- 172.21.15.139:0/1366661346 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c10044020 con 0x7f4c200cf900 2024-04-25T09:59:06.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.175+0000 7f4c16ffd640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c0406b4d0 0x7f4c0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.175+0000 7f4c16ffd640 1 -- 172.21.15.139:0/1366661346 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c100aa600 con 0x7f4c200cf900 2024-04-25T09:59:06.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.175+0000 7f4c26498640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c0406b4d0 0x7f4c0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.175+0000 7f4c26498640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c0406b4d0 0x7f4c0406d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4c2015abf0 tx=0x7f4c1c007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.179+0000 7f4c16ffd640 1 -- 172.21.15.139:0/1366661346 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c201f4ec0 con 0x7f4c200cf900 2024-04-25T09:59:06.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.386+0000 7f4c286fc640 1 -- 172.21.15.139:0/1366661346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c200d25f0 con 0x7f4c200cf900 2024-04-25T09:59:06.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.386+0000 7f4c16ffd640 1 -- 172.21.15.139:0/1366661346 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f4c10079380 con 0x7f4c200cf900 2024-04-25T09:59:06.390 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:06.390 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7545,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1239243954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1239243954},{"type":"v1","addr":"172.21.15.139:6836","nonce":1239243954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T09:58:59.365616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7557":{"gid":7557,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2266713846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2266713846},{"type":"v1","addr":"172.21.15.150:6835","nonce":2266713846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:06.391 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 50 2024-04-25T09:59:06.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.388+0000 7f4c286fc640 1 -- 172.21.15.139:0/1366661346 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c0406b4d0 msgr2=0x7f4c0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.388+0000 7f4c286fc640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c0406b4d0 0x7f4c0406d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4c2015abf0 tx=0x7f4c1c007930 comp rx=0 tx=0).stop 2024-04-25T09:59:06.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.388+0000 7f4c286fc640 1 -- 172.21.15.139:0/1366661346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c200cf900 msgr2=0x7f4c20159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.388+0000 7f4c286fc640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c200cf900 0x7f4c20159610 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f4c10035040 tx=0x7f4c100046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.388+0000 7f4c286fc640 1 -- 172.21.15.139:0/1366661346 shutdown_connections 2024-04-25T09:59:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.388+0000 7f4c286fc640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c200cf900 0x7f4c20159610 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.388+0000 7f4c286fc640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4c0406b4d0 0x7f4c0406d990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.388+0000 7f4c286fc640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c2015efb0 0x7f4c20159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.388+0000 7f4c286fc640 1 --2- 172.21.15.139:0/1366661346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c20099a50 0x7f4c201f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.388+0000 7f4c286fc640 1 -- 172.21.15.139:0/1366661346 >> 172.21.15.139:0/1366661346 conn(0x7f4c200d1790 msgr2=0x7f4c200ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.388+0000 7f4c286fc640 1 -- 172.21.15.139:0/1366661346 shutdown_connections 2024-04-25T09:59:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.388+0000 7f4c286fc640 1 -- 172.21.15.139:0/1366661346 wait complete. 2024-04-25T09:59:06.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:57.374595+0000', 'modified': '2024-04-25T09:58:59.365616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7557}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7557': {'gid': 7557, 'name': 'd', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6835/2266713846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6833', 'nonce': 2266713846}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 2266713846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:59:06.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:59:06.806 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T09:59:06.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.903+0000 7f3cc2e09640 1 Processor -- start 2024-04-25T09:59:06.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.903+0000 7f3cc2e09640 1 -- start start 2024-04-25T09:59:06.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.904+0000 7f3cc2e09640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3cbc1617e0 0x7f3cbc0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.904+0000 7f3cc2e09640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cbc0cec80 0x7f3cbc0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.904+0000 7f3cc2e09640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cbc161dc0 con 0x7f3cbc15dbf0 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.904+0000 7f3cc2e09640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cbc0d1ee0 con 0x7f3cbc1617e0 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.904+0000 7f3cc2e09640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc1620a0 con 0x7f3cbc0cec80 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.904+0000 7f3cbbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3cbc1617e0 0x7f3cbc0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.905+0000 7f3cbbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3cbc1617e0 0x7f3cbc0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48876/0 (socket says 172.21.15.139:48876) 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.905+0000 7f3cbbfff640 1 -- 172.21.15.139:0/112088273 learned_addr learned my addr 172.21.15.139:0/112088273 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.905+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/112088273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cbc0cec80 0x7f3cbc0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.905+0000 7f3cbbfff640 1 -- 172.21.15.139:0/112088273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cbc0cec80 msgr2=0x7f3cbc0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.905+0000 7f3cbbfff640 1 --2- 172.21.15.139:0/112088273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cbc0cec80 0x7f3cbc0d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.905+0000 7f3cbbfff640 1 -- 172.21.15.139:0/112088273 >> v1:172.21.15.150:6789/0 conn(0x7f3cbc15dbf0 legacy=0x7f3cbc0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.905+0000 7f3cbb7fe640 1 -- 172.21.15.139:0/112088273 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271522895 0 0) 0x7f3cbc161dc0 con 0x7f3cbc15dbf0 2024-04-25T09:59:06.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.905+0000 7f3cbbfff640 1 -- 172.21.15.139:0/112088273 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc162380 con 0x7f3cbc1617e0 2024-04-25T09:59:06.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.906+0000 7f3cbbfff640 1 --2- 172.21.15.139:0/112088273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cbc1617e0 0x7f3cbc0ce740 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009870 tx=0x7f3cb002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d164cd489c32d7bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.906+0000 7f3cbb7fe640 1 -- 172.21.15.139:0/112088273 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0037070 con 0x7f3cbc1617e0 2024-04-25T09:59:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.906+0000 7f3cbb7fe640 1 -- 172.21.15.139:0/112088273 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb0032440 con 0x7f3cbc1617e0 2024-04-25T09:59:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.906+0000 7f3cbb7fe640 1 -- 172.21.15.139:0/112088273 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0032810 con 0x7f3cbc1617e0 2024-04-25T09:59:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.906+0000 7f3cc2e09640 1 -- 172.21.15.139:0/112088273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cbc1617e0 msgr2=0x7f3cbc0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.906+0000 7f3cc2e09640 1 --2- 172.21.15.139:0/112088273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cbc1617e0 0x7f3cbc0ce740 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009870 tx=0x7f3cb002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.906+0000 7f3cc2e09640 1 -- 172.21.15.139:0/112088273 shutdown_connections 2024-04-25T09:59:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.906+0000 7f3cc2e09640 1 --2- 172.21.15.139:0/112088273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cbc0cec80 0x7f3cbc0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.906+0000 7f3cc2e09640 1 --2- 172.21.15.139:0/112088273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cbc1617e0 0x7f3cbc0ce740 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.906+0000 7f3cc2e09640 1 -- 172.21.15.139:0/112088273 >> 172.21.15.139:0/112088273 conn(0x7f3cbc0b5480 msgr2=0x7f3cbc0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.906+0000 7f3cc2e09640 1 -- 172.21.15.139:0/112088273 shutdown_connections 2024-04-25T09:59:06.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.906+0000 7f3cc2e09640 1 -- 172.21.15.139:0/112088273 wait complete. 2024-04-25T09:59:06.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.907+0000 7f3cc2e09640 1 Processor -- start 2024-04-25T09:59:06.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.907+0000 7f3cc2e09640 1 -- start start 2024-04-25T09:59:06.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.908+0000 7f3cc2e09640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cbc0cec80 0x7f3cbc0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.908+0000 7f3cc2e09640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cbc15dbf0 0x7f3cbc15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.908+0000 7f3cc2e09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cbc1617e0 0x7f3cbc15fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.908+0000 7f3cc2e09640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc15dfd0 con 0x7f3cbc1617e0 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.908+0000 7f3cc2e09640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc0d28b0 con 0x7f3cbc0cec80 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.908+0000 7f3cc2e09640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc0c93b0 con 0x7f3cbc15dbf0 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.908+0000 7f3cc13a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cbc1617e0 0x7f3cbc15fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.908+0000 7f3cc13a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cbc1617e0 0x7f3cbc15fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48888/0 (socket says 172.21.15.139:48888) 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.908+0000 7f3cc13a6640 1 -- 172.21.15.139:0/2363988467 learned_addr learned my addr 172.21.15.139:0/2363988467 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cbc0cec80 0x7f3cbc0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cbbfff640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cbc15dbf0 0x7f3cbc15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cc13a6640 1 -- 172.21.15.139:0/2363988467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cbc15dbf0 msgr2=0x7f3cbc15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cbc15dbf0 0x7f3cbc15e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cc13a6640 1 -- 172.21.15.139:0/2363988467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cbc0cec80 msgr2=0x7f3cbc0c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cbc0cec80 0x7f3cbc0c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cc13a6640 1 -- 172.21.15.139:0/2363988467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc14f090 con 0x7f3cbc1617e0 2024-04-25T09:59:06.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cbc0cec80 0x7f3cbc0c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:06.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cbc1617e0 0x7f3cbc15fcb0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009dd0 tx=0x7f3cb400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cb97fa640 1 -- 172.21.15.139:0/2363988467 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4018020 con 0x7f3cbc1617e0 2024-04-25T09:59:06.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cb97fa640 1 -- 172.21.15.139:0/2363988467 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb4014480 con 0x7f3cbc1617e0 2024-04-25T09:59:06.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cc2e09640 1 -- 172.21.15.139:0/2363988467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cbc15e820 con 0x7f3cbc1617e0 2024-04-25T09:59:06.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cb97fa640 1 -- 172.21.15.139:0/2363988467 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4014870 con 0x7f3cbc1617e0 2024-04-25T09:59:06.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.909+0000 7f3cc2e09640 1 -- 172.21.15.139:0/2363988467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cbc0ccfd0 con 0x7f3cbc1617e0 2024-04-25T09:59:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.911+0000 7f3cb97fa640 1 -- 172.21.15.139:0/2363988467 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3cb4005d10 con 0x7f3cbc1617e0 2024-04-25T09:59:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.911+0000 7f3cb97fa640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3ca806b500 0x7f3ca806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.911+0000 7f3cb97fa640 1 -- 172.21.15.139:0/2363988467 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3cb408ae50 con 0x7f3cbc1617e0 2024-04-25T09:59:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.911+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3ca806b500 0x7f3ca806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.911+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3ca806b500 0x7f3ca806d9c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3cac000f60 tx=0x7f3cac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.912+0000 7f3cc5bdf740 1 Processor -- start 2024-04-25T09:59:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.912+0000 7f3cc5bdf740 1 -- start start 2024-04-25T09:59:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.912+0000 7f3cc5bdf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0349230 0x557dd0349610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc5bdf740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0366ad0 0x557dd0367020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc5bdf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd031ef10 0x557dd0365190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc5bdf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd0281a10 con 0x557dd031ef10 2024-04-25T09:59:06.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc5bdf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd022ab30 con 0x557dd0349230 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc5bdf740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557dd027dc50 con 0x557dd0366ad0 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc13a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd031ef10 0x557dd0365190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc13a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd031ef10 0x557dd0365190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48900/0 (socket says 172.21.15.139:48900) 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc13a6640 1 -- 172.21.15.139:0/1776977342 learned_addr learned my addr 172.21.15.139:0/1776977342 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cbbfff640 1 --2- 172.21.15.139:0/1776977342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0366ad0 0x557dd0367020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/1776977342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0349230 0x557dd0349610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc13a6640 1 -- 172.21.15.139:0/1776977342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0366ad0 msgr2=0x557dd0367020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/1776977342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0366ad0 0x557dd0367020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc13a6640 1 -- 172.21.15.139:0/1776977342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0349230 msgr2=0x557dd0349610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/1776977342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0349230 0x557dd0349610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc13a6640 1 -- 172.21.15.139:0/1776977342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dd023fe10 con 0x557dd031ef10 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/1776977342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd031ef10 0x557dd0365190 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40558a0 tx=0x7f3cb4059500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba76ceaab221af3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3ca27fc640 1 -- 172.21.15.139:0/1776977342 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4087070 con 0x557dd031ef10 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3ca27fc640 1 -- 172.21.15.139:0/1776977342 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb4057740 con 0x557dd031ef10 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.913+0000 7f3ca27fc640 1 -- 172.21.15.139:0/1776977342 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4055ab0 con 0x557dd031ef10 2024-04-25T09:59:06.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.914+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1776977342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd031ef10 msgr2=0x557dd0365190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.914+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/1776977342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd031ef10 0x557dd0365190 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40558a0 tx=0x7f3cb4059500 comp rx=0 tx=0).stop 2024-04-25T09:59:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.914+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1776977342 shutdown_connections 2024-04-25T09:59:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.914+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/1776977342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd031ef10 0x557dd0365190 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.914+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/1776977342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0366ad0 0x557dd0367020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.914+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/1776977342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0349230 0x557dd0349610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.914+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1776977342 >> 172.21.15.139:0/1776977342 conn(0x557dd031b840 msgr2=0x557dd0364040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.914+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1776977342 shutdown_connections 2024-04-25T09:59:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.914+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1776977342 wait complete. 2024-04-25T09:59:06.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.914+0000 7f3cc5bdf740 1 -- start start 2024-04-25T09:59:06.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.915+0000 7f3cc5bdf740 1 Processor -- start 2024-04-25T09:59:06.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.915+0000 7f3cc5bdf740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 0x557dd031d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.915+0000 7f3cc5bdf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd031d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.915+0000 7f3cc5bdf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd0335ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.915+0000 7f3cc5bdf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd031fcb0 con 0x557dd0366ad0 2024-04-25T09:59:06.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.915+0000 7f3cc5bdf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd023e0c0 con 0x557dd0335660 2024-04-25T09:59:06.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.915+0000 7f3cc5bdf740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557dd0319770 con 0x557dd0349230 2024-04-25T09:59:06.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.915+0000 7f3cc5bdf740 10 client.0 _ll_register_callbacks cb 0x557dd0231ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:06.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.915+0000 7f3cbbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd031d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cbbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd031d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48912/0 (socket says 172.21.15.139:48912) 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cbbfff640 1 -- 172.21.15.139:0/1125223689 learned_addr learned my addr 172.21.15.139:0/1125223689 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 0x557dd031d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd0335ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cbbfff640 1 -- 172.21.15.139:0/1125223689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 msgr2=0x557dd031d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cbbfff640 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 0x557dd031d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cbbfff640 1 -- 172.21.15.139:0/1125223689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 msgr2=0x557dd0335ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cbbfff640 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd0335ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cbbfff640 1 -- 172.21.15.139:0/1125223689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dcff962d0 con 0x557dd0366ad0 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cbbfff640 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd031d7c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00090b0 tx=0x7f3cb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc8080d83f50064 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3c9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3c9b7fe640 1 -- 172.21.15.139:0/1125223689 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0044070 con 0x557dd0366ad0 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3c9b7fe640 1 -- 172.21.15.139:0/1125223689 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3cb0003610 con 0x557dd0366ad0 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1125223689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cb4057740 con 0x557dd0366ad0 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3c9b7fe640 1 -- 172.21.15.139:0/1125223689 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cb004f050 con 0x557dd0366ad0 2024-04-25T09:59:06.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cc5bdf740 10 client.7713 fetch_fsmap learned FSMap version 50 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1125223689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x557dd00606b0 con 0x557dd0366ad0 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3c9b7fe640 1 -- 172.21.15.139:0/1125223689 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb4057740 con 0x557dd0366ad0 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3c9b7fe640 1 -- 172.21.15.139:0/1125223689 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3cb0052020 con 0x557dd0366ad0 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cc5bdf740 10 client.7713 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cc5bdf740 10 client.7713 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cc5bdf740 20 client.7713 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.150:6833/2266713846,v1:172.21.15.150:6835/2266713846] conn(0x557dd034c610 0x557dd034c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cc5bdf740 4 client.7713 mds_command: new command op to 7557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.916+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1125223689 --> [v2:172.21.15.150:6833/2266713846,v1:172.21.15.150:6835/2266713846] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557dd02e0920 con 0x557dd034c610 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.917+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.150:6833/2266713846,v1:172.21.15.150:6835/2266713846] conn(0x557dd034c610 0x557dd034c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.917+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.150:6833/2266713846,v1:172.21.15.150:6835/2266713846] conn(0x557dd034c610 0x557dd034c9f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd38f6d558cd00a server_cookie=90c2d0a88918ea3d in_seq=0 out_seq=0 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.917+0000 7f3c9b7fe640 10 client.7713 ms_handle_connect on v2:172.21.15.150:6833/2266713846 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3c9b7fe640 1 -- 172.21.15.139:0/1125223689 <== mds.0 v2:172.21.15.150:6833/2266713846 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557dd02e0920 con 0x557dd034c610 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3c9b7fe640 10 client.7713 handle_command_reply: tid=0 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3cc5bdf740 1 client.7713 shutdown 2024-04-25T09:59:06.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1125223689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 msgr2=0x557dd031d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd031d7c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00090b0 tx=0x7f3cb0004730 comp rx=0 tx=0).stop 2024-04-25T09:59:06.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1125223689 shutdown_connections 2024-04-25T09:59:06.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.150:6833/2266713846,v1:172.21.15.150:6835/2266713846] conn(0x557dd034c610 0x557dd034c9f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd031d7c0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd0335ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/1125223689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 0x557dd031d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3c9b7fe640 0 client.7713 ms_handle_reset on v2:172.21.15.150:6833/2266713846 2024-04-25T09:59:06.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1125223689 >> 172.21.15.139:0/1125223689 conn(0x557dd031ef10 msgr2=0x557dd03675c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:06.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1125223689 shutdown_connections 2024-04-25T09:59:06.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/1125223689 wait complete. 2024-04-25T09:59:06.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.918+0000 7f3cc5bdf740 20 client.7713 trim_cache size 0 max 16384 2024-04-25T09:59:06.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.934+0000 7f3cc5bdf740 1 Processor -- start 2024-04-25T09:59:06.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.934+0000 7f3cc5bdf740 1 -- start start 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.934+0000 7f3cc5bdf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd03c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.934+0000 7f3cc5bdf740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 0x557dd035b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.934+0000 7f3cc5bdf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd035bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.934+0000 7f3cc5bdf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc200250 con 0x557dd0366ad0 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.934+0000 7f3cc5bdf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc1ff4a0 con 0x557dd0335660 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.934+0000 7f3cc5bdf740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc150140 con 0x557dd0349230 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3cc13a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd035bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3cc13a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd035bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48928/0 (socket says 172.21.15.139:48928) 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3cc13a6640 1 -- 172.21.15.139:0/3199437874 learned_addr learned my addr 172.21.15.139:0/3199437874 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3cc13a6640 1 -- 172.21.15.139:0/3199437874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 msgr2=0x557dd035b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/3199437874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 0x557dd035b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/3199437874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd03c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3cc13a6640 1 -- 172.21.15.139:0/3199437874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 msgr2=0x557dd03c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:06.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/3199437874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd03c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3cc13a6640 1 -- 172.21.15.139:0/3199437874 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dd03668f0 con 0x557dd0366ad0 2024-04-25T09:59:06.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/3199437874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd03c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:06.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/3199437874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd035bad0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4059140 tx=0x7f3cb408a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c79442d692a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3ca1ffb640 1 -- 172.21.15.139:0/3199437874 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb407f070 con 0x557dd0366ad0 2024-04-25T09:59:06.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3ca1ffb640 1 -- 172.21.15.139:0/3199437874 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb4088040 con 0x557dd0366ad0 2024-04-25T09:59:06.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.935+0000 7f3ca1ffb640 1 -- 172.21.15.139:0/3199437874 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4087050 con 0x557dd0366ad0 2024-04-25T09:59:06.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.936+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/3199437874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 msgr2=0x557dd035bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.936+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/3199437874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd035bad0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4059140 tx=0x7f3cb408a000 comp rx=0 tx=0).stop 2024-04-25T09:59:06.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.936+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/3199437874 shutdown_connections 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.936+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/3199437874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd035bad0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.936+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/3199437874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 0x557dd035b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.936+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/3199437874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd03c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.936+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/3199437874 >> 172.21.15.139:0/3199437874 conn(0x557dd031ef10 msgr2=0x557dd0335ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.936+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/3199437874 shutdown_connections 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.936+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/3199437874 wait complete. 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.936+0000 7f3cc5bdf740 1 -- start start 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.937+0000 7f3cc5bdf740 1 Processor -- start 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.937+0000 7f3cc5bdf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd037d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.937+0000 7f3cc5bdf740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 0x557dd0359e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.937+0000 7f3cc5bdf740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd035a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.937+0000 7f3cc5bdf740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd034bcb0 con 0x557dd0366ad0 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.937+0000 7f3cc5bdf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc1506f0 con 0x557dd0335660 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.937+0000 7f3cc5bdf740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557dd034b940 con 0x557dd0349230 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.937+0000 7f3cc5bdf740 10 client.0 _ll_register_callbacks cb 0x557dd0319c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:06.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.937+0000 7f3cc13a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd035a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.937+0000 7f3cc13a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd035a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48944/0 (socket says 172.21.15.139:48944) 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.937+0000 7f3cc13a6640 1 -- 172.21.15.139:0/676679785 learned_addr learned my addr 172.21.15.139:0/676679785 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.937+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd037d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cbbfff640 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 0x557dd0359e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc13a6640 1 -- 172.21.15.139:0/676679785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 msgr2=0x557dd0359e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 0x557dd0359e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc13a6640 1 -- 172.21.15.139:0/676679785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 msgr2=0x557dd037d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd037d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc13a6640 1 -- 172.21.15.139:0/676679785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dd037e690 con 0x557dd0366ad0 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc13a6640 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd035a370 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f3cb405d750 tx=0x7f3cb401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc692c42e82c289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3c9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3c9b7fe640 1 -- 172.21.15.139:0/676679785 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb407f070 con 0x557dd0366ad0 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3c9b7fe640 1 -- 172.21.15.139:0/676679785 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3cb4055ab0 con 0x557dd0366ad0 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/676679785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cb4088040 con 0x557dd0366ad0 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3c9b7fe640 1 -- 172.21.15.139:0/676679785 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cb4069050 con 0x557dd0366ad0 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc5bdf740 10 client.7719 fetch_fsmap learned FSMap version 50 2024-04-25T09:59:06.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/676679785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x557dd0319120 con 0x557dd0366ad0 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3c9b7fe640 1 -- 172.21.15.139:0/676679785 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb4088040 con 0x557dd0366ad0 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3c9b7fe640 1 -- 172.21.15.139:0/676679785 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3cb406c020 con 0x557dd0366ad0 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc5bdf740 10 client.7719 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc5bdf740 10 client.7719 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc5bdf740 20 client.7719 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.150:6833/2266713846,v1:172.21.15.150:6835/2266713846] conn(0x557dd03a99a0 0x557dd03a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc5bdf740 4 client.7719 mds_command: new command op to 7557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.938+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/676679785 --> [v2:172.21.15.150:6833/2266713846,v1:172.21.15.150:6835/2266713846] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557dd035cd50 con 0x557dd03a99a0 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.150:6833/2266713846,v1:172.21.15.150:6835/2266713846] conn(0x557dd03a99a0 0x557dd03a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3c9b7fe640 10 client.7719 ms_handle_connect on v2:172.21.15.150:6833/2266713846 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc0ba5640 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.150:6833/2266713846,v1:172.21.15.150:6835/2266713846] conn(0x557dd03a99a0 0x557dd03a9d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7017a3397f5d5658 server_cookie=f1ac484804cd86d8 in_seq=0 out_seq=0 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3c9b7fe640 1 -- 172.21.15.139:0/676679785 <== mds.0 v2:172.21.15.150:6833/2266713846 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x557dd035cd50 con 0x557dd03a99a0 2024-04-25T09:59:06.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3c9b7fe640 10 client.7719 handle_command_reply: tid=0 2024-04-25T09:59:06.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc5bdf740 1 client.7719 shutdown 2024-04-25T09:59:06.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/676679785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 msgr2=0x557dd035a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd035a370 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f3cb405d750 tx=0x7f3cb401f040 comp rx=0 tx=0).stop 2024-04-25T09:59:06.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/676679785 shutdown_connections 2024-04-25T09:59:06.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.150:6833/2266713846,v1:172.21.15.150:6835/2266713846] conn(0x557dd03a99a0 0x557dd03a9d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x557dd0366ad0 0x557dd035a370 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3c9b7fe640 0 client.7719 ms_handle_reset on v2:172.21.15.150:6833/2266713846 2024-04-25T09:59:06.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557dd0349230 0x557dd0359e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc5bdf740 1 --2- 172.21.15.139:0/676679785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557dd0335660 0x557dd037d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/676679785 >> 172.21.15.139:0/676679785 conn(0x557dd031ef10 msgr2=0x557dd031cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:06.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/676679785 shutdown_connections 2024-04-25T09:59:06.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.939+0000 7f3cc5bdf740 1 -- 172.21.15.139:0/676679785 wait complete. 2024-04-25T09:59:06.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc5bdf740 20 client.7719 trim_cache size 0 max 16384 2024-04-25T09:59:06.949 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:59:06.949 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T09:59:06.949 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T09:59:06.949 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 7557, 2024-04-25T09:59:06.950 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T09:59:06.950 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T09:59:06.950 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T09:59:06.950 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 8.9174258640000001, 2024-04-25T09:59:06.950 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 50, 2024-04-25T09:59:06.950 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 89, 2024-04-25T09:59:06.950 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T09:59:06.951 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 13.968388394 2024-04-25T09:59:06.951 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:59:06.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc2e09640 1 -- 172.21.15.139:0/2363988467 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3ca806b500 msgr2=0x7f3ca806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc2e09640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3ca806b500 0x7f3ca806d9c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3cac000f60 tx=0x7f3cac009040 comp rx=0 tx=0).stop 2024-04-25T09:59:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc2e09640 1 -- 172.21.15.139:0/2363988467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cbc1617e0 msgr2=0x7f3cbc15fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc2e09640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cbc1617e0 0x7f3cbc15fcb0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009dd0 tx=0x7f3cb400c490 comp rx=0 tx=0).stop 2024-04-25T09:59:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc2e09640 1 -- 172.21.15.139:0/2363988467 shutdown_connections 2024-04-25T09:59:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc2e09640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cbc1617e0 0x7f3cbc15fcb0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc2e09640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3ca806b500 0x7f3ca806d9c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc2e09640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cbc15dbf0 0x7f3cbc15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc2e09640 1 --2- 172.21.15.139:0/2363988467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cbc0cec80 0x7f3cbc0c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:06.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc2e09640 1 -- 172.21.15.139:0/2363988467 >> 172.21.15.139:0/2363988467 conn(0x7f3cbc0b5480 msgr2=0x7f3cbc0d2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:06.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc2e09640 1 -- 172.21.15.139:0/2363988467 shutdown_connections 2024-04-25T09:59:06.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:06.940+0000 7f3cc2e09640 1 -- 172.21.15.139:0/2363988467 wait complete. 2024-04-25T09:59:06.955 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:59:07.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.048+0000 7ff1a4b70640 1 Processor -- start 2024-04-25T09:59:07.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.048+0000 7ff1a4b70640 1 -- start start 2024-04-25T09:59:07.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.049+0000 7ff1a4b70640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff1a00d1080 0x7ff1a00b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:07.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.049+0000 7ff1a4b70640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a00cf250 con 0x7ff1a0161850 2024-04-25T09:59:07.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.049+0000 7ff1a4b70640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a00ce9b0 con 0x7ff1a015dc60 2024-04-25T09:59:07.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.049+0000 7ff1a4b70640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a00d3730 con 0x7ff1a00d1080 2024-04-25T09:59:07.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.049+0000 7ff195d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff1a00d1080 0x7ff1a00b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:07.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.049+0000 7ff195d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff1a00d1080 0x7ff1a00b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48946/0 (socket says 172.21.15.139:48946) 2024-04-25T09:59:07.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.049+0000 7ff195d9b640 1 -- 172.21.15.139:0/1704707917 learned_addr learned my addr 172.21.15.139:0/1704707917 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:07.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.049+0000 7ff19ed9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff1a0161850 0x7ff1a00d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42448/0 (socket says 172.21.15.139:42448) 2024-04-25T09:59:07.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.049+0000 7ff195d9b640 1 -- 172.21.15.139:0/1704707917 >> v1:172.21.15.150:6789/0 conn(0x7ff1a015dc60 legacy=0x7ff1a00d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:07.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.050+0000 7ff195d9b640 1 -- 172.21.15.139:0/1704707917 >> v1:172.21.15.139:6789/0 conn(0x7ff1a0161850 legacy=0x7ff1a00d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:07.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.050+0000 7ff19dd9b640 1 -- 172.21.15.139:0/1704707917 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516759970 0 0) 0x7ff198001490 con 0x7ff1a0161850 2024-04-25T09:59:07.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.050+0000 7ff195d9b640 1 -- 172.21.15.139:0/1704707917 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a00cec90 con 0x7ff1a00d1080 2024-04-25T09:59:07.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.050+0000 7ff195d9b640 1 --2- 172.21.15.139:0/1704707917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1a00d1080 0x7ff1a00b6c10 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff188009870 tx=0x7ff18802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca62ff0c29243d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:07.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.050+0000 7ff19dd9b640 1 -- 172.21.15.139:0/1704707917 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188037070 con 0x7ff1a00d1080 2024-04-25T09:59:07.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.050+0000 7ff19dd9b640 1 -- 172.21.15.139:0/1704707917 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff188032440 con 0x7ff1a00d1080 2024-04-25T09:59:07.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.050+0000 7ff19dd9b640 1 -- 172.21.15.139:0/1704707917 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188032810 con 0x7ff1a00d1080 2024-04-25T09:59:07.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.051+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1704707917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1a00d1080 msgr2=0x7ff1a00b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:07.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.051+0000 7ff1a4b70640 1 --2- 172.21.15.139:0/1704707917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1a00d1080 0x7ff1a00b6c10 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff188009870 tx=0x7ff18802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.051+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1704707917 shutdown_connections 2024-04-25T09:59:07.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.051+0000 7ff1a4b70640 1 --2- 172.21.15.139:0/1704707917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1a00d1080 0x7ff1a00b6c10 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.051+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1704707917 >> 172.21.15.139:0/1704707917 conn(0x7ff1a00b5910 msgr2=0x7ff1a00b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:07.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.051+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1704707917 shutdown_connections 2024-04-25T09:59:07.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.051+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1704707917 wait complete. 2024-04-25T09:59:07.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.052+0000 7ff1a4b70640 1 Processor -- start 2024-04-25T09:59:07.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.052+0000 7ff1a4b70640 1 -- start start 2024-04-25T09:59:07.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff1a4b70640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1a00d1080 0x7ff1a01f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:07.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff1a4b70640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a015dc60 0x7ff1a00c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:07.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff1a4b70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1a0161850 0x7ff1a00b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:07.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff1a4b70640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a015e040 con 0x7ff1a0161850 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff1a4b70640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a00d1fc0 con 0x7ff1a015dc60 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff1a4b70640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a00c7e60 con 0x7ff1a00d1080 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff19ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1a0161850 0x7ff1a00b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff19ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1a0161850 0x7ff1a00b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48952/0 (socket says 172.21.15.139:48952) 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff19ed9d640 1 -- 172.21.15.139:0/1254069693 learned_addr learned my addr 172.21.15.139:0/1254069693 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff19ed9d640 1 -- 172.21.15.139:0/1254069693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1a00d1080 msgr2=0x7ff1a01f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff19ed9d640 1 --2- 172.21.15.139:0/1254069693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1a00d1080 0x7ff1a01f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff19ed9d640 1 -- 172.21.15.139:0/1254069693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a015dc60 msgr2=0x7ff1a00c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff19ed9d640 1 --2- 172.21.15.139:0/1254069693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a015dc60 0x7ff1a00c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.053+0000 7ff19ed9d640 1 -- 172.21.15.139:0/1254069693 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a00c8140 con 0x7ff1a0161850 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.054+0000 7ff19ed9d640 1 --2- 172.21.15.139:0/1254069693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1a0161850 0x7ff1a00b3460 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c730 tx=0x7ff19800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.054+0000 7ff1977fe640 1 -- 172.21.15.139:0/1254069693 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198018070 con 0x7ff1a0161850 2024-04-25T09:59:07.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.054+0000 7ff1977fe640 1 -- 172.21.15.139:0/1254069693 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19800ce40 con 0x7ff1a0161850 2024-04-25T09:59:07.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.054+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1254069693 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a00c8420 con 0x7ff1a0161850 2024-04-25T09:59:07.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.054+0000 7ff1977fe640 1 -- 172.21.15.139:0/1254069693 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19800a860 con 0x7ff1a0161850 2024-04-25T09:59:07.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.054+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1254069693 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a00c7a50 con 0x7ff1a0161850 2024-04-25T09:59:07.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.055+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1254069693 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a0074520 con 0x7ff1a0161850 2024-04-25T09:59:07.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.055+0000 7ff1977fe640 1 -- 172.21.15.139:0/1254069693 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff19801b070 con 0x7ff1a0161850 2024-04-25T09:59:07.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.056+0000 7ff1977fe640 1 --2- 172.21.15.139:0/1254069693 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff18006b4d0 0x7ff18006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:07.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.056+0000 7ff19e59c640 1 --2- 172.21.15.139:0/1254069693 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff18006b4d0 0x7ff18006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:07.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.056+0000 7ff1977fe640 1 -- 172.21.15.139:0/1254069693 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff1980898b0 con 0x7ff1a0161850 2024-04-25T09:59:07.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.056+0000 7ff19e59c640 1 --2- 172.21.15.139:0/1254069693 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff18006b4d0 0x7ff18006d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff190003cc0 tx=0x7ff1900079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:07.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.060+0000 7ff1977fe640 1 -- 172.21.15.139:0/1254069693 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff1a00cf250 con 0x7ff1a0161850 2024-04-25T09:59:07.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.266+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1254069693 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff1a00b98c0 con 0x7ff1a0161850 2024-04-25T09:59:07.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.266+0000 7ff1977fe640 1 -- 172.21.15.139:0/1254069693 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff198058630 con 0x7ff1a0161850 2024-04-25T09:59:07.270 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:59:07.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.268+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1254069693 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff18006b4d0 msgr2=0x7ff18006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:07.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.268+0000 7ff1a4b70640 1 --2- 172.21.15.139:0/1254069693 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff18006b4d0 0x7ff18006d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff190003cc0 tx=0x7ff1900079b0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.268+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1254069693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1a0161850 msgr2=0x7ff1a00b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:07.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.268+0000 7ff1a4b70640 1 --2- 172.21.15.139:0/1254069693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1a0161850 0x7ff1a00b3460 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c730 tx=0x7ff19800cbf0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.268+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1254069693 shutdown_connections 2024-04-25T09:59:07.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.268+0000 7ff1a4b70640 1 --2- 172.21.15.139:0/1254069693 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1a0161850 0x7ff1a00b3460 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.268+0000 7ff1a4b70640 1 --2- 172.21.15.139:0/1254069693 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff18006b4d0 0x7ff18006d990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.268+0000 7ff1a4b70640 1 --2- 172.21.15.139:0/1254069693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a015dc60 0x7ff1a00c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.268+0000 7ff1a4b70640 1 --2- 172.21.15.139:0/1254069693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1a00d1080 0x7ff1a01f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:07.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.268+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1254069693 >> 172.21.15.139:0/1254069693 conn(0x7ff1a00b5910 msgr2=0x7ff1a006caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:07.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.268+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1254069693 shutdown_connections 2024-04-25T09:59:07.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:07.268+0000 7ff1a4b70640 1 -- 172.21.15.139:0/1254069693 wait complete. 2024-04-25T09:59:07.283 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:59:07.283 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:59:07.283 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T09:59:07.283 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:59:07.283 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:07.283 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:59:07.283 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:59:07.283 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:59:07.283 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:59:07.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:07.284 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T09:59:07.299 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:59:07.299 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:07.300 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:59:07.301 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42351sec preferred_lft 42351sec 2024-04-25T09:59:07.301 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T09:59:07.301 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:07.301 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:59:07.301 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:07.301 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:59:07.301 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:07.301 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T09:59:07.301 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:07.301 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:59:07.301 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:59:07.301 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T09:59:07.302 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:07.302 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:59:07.302 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:59:07.302 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T09:59:07.302 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:07.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:07.302 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:59:07.361 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:59:07.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:07.364 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T09:59:07.419 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T09:59:07.419 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:59:07.419 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:59:07.419 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:59:07.419 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:59:07.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:07.420 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:59:07.420 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:59:07.420 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:59:07.420 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:59:07.420 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:59:07.420 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:59:07.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:07.503 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:59:07.558 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:59:07.558 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:59:07.559 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:59:07.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:07.559 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T09:59:07.622 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:59:07.622 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:07.678 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:59:07.678 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:59:07.678 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:07.733 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:59:07.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:07.733 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:59:07.815 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:59:07.815 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:59:07.826 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:59:07.826 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:59:07.826 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T09:59:07.826 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T09:59:07.826 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T09:59:07.826 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:59:07.826 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:59:07.827 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:59:07.827 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:59:07.827 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:59:07.828 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:59:07.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:07.828 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:07.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:07.893 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:59:07.953 INFO:teuthology.orchestra.run.smithi139.stdout:client6658 2024-04-25T09:59:07.954 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:59:07.954 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:08.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:08.009 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-25T09:59:08.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:08.067 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-25T09:59:08.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:08.124 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-25T09:59:08.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:08.181 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-25T09:59:08.243 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T09:59:08.244 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T09:59:08.244 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T09:59:08.244 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T09:59:08.244 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-25T09:59:08.244 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T09:59:08.347 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627776 2024-04-25T09:59:08.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:59:08.357 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:59:08.357 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:08.413 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:59:08.413 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:08.469 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:59:08.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.601+0000 7efd692b3640 1 Processor -- start 2024-04-25T09:59:08.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.601+0000 7efd692b3640 1 -- start start 2024-04-25T09:59:08.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.602+0000 7efd692b3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efd640d1080 0x7efd640b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.602+0000 7efd692b3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd640cf250 con 0x7efd6415dc60 2024-04-25T09:59:08.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.602+0000 7efd692b3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd640ce9b0 con 0x7efd64161850 2024-04-25T09:59:08.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.602+0000 7efd692b3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd640d3730 con 0x7efd640d1080 2024-04-25T09:59:08.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.602+0000 7efd62ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7efd6415dc60 0x7efd640d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42458/0 (socket says 172.21.15.139:42458) 2024-04-25T09:59:08.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.602+0000 7efd627fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efd640d1080 0x7efd640b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.602+0000 7efd62ffd640 1 -- 172.21.15.139:0/4060734861 learned_addr learned my addr 172.21.15.139:0/4060734861 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:08.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.602+0000 7efd627fc640 1 -- 172.21.15.139:0/4060734861 >> v1:172.21.15.150:6789/0 conn(0x7efd64161850 legacy=0x7efd640d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:08.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.602+0000 7efd627fc640 1 -- 172.21.15.139:0/4060734861 >> v1:172.21.15.139:6789/0 conn(0x7efd6415dc60 legacy=0x7efd640d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.602+0000 7efd627fc640 1 -- 172.21.15.139:0/4060734861 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd640cec90 con 0x7efd640d1080 2024-04-25T09:59:08.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.602+0000 7efd61ffb640 1 -- 172.21.15.139:0/4060734861 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692880288 0 0) 0x7efd54001490 con 0x7efd6415dc60 2024-04-25T09:59:08.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.603+0000 7efd627fc640 1 --2- 172.21.15.139:0/4060734861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd640d1080 0x7efd640b6c10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7efd580097a0 tx=0x7efd5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ab36345e3d42e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:08.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.603+0000 7efd61ffb640 1 -- 172.21.15.139:0/4060734861 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58037070 con 0x7efd640d1080 2024-04-25T09:59:08.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.603+0000 7efd61ffb640 1 -- 172.21.15.139:0/4060734861 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd58032440 con 0x7efd640d1080 2024-04-25T09:59:08.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.603+0000 7efd61ffb640 1 -- 172.21.15.139:0/4060734861 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58032810 con 0x7efd640d1080 2024-04-25T09:59:08.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.604+0000 7efd692b3640 1 -- 172.21.15.139:0/4060734861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd640d1080 msgr2=0x7efd640b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.604+0000 7efd692b3640 1 --2- 172.21.15.139:0/4060734861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd640d1080 0x7efd640b6c10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7efd580097a0 tx=0x7efd5802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.604+0000 7efd692b3640 1 -- 172.21.15.139:0/4060734861 shutdown_connections 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.604+0000 7efd692b3640 1 --2- 172.21.15.139:0/4060734861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd640d1080 0x7efd640b6c10 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.604+0000 7efd692b3640 1 -- 172.21.15.139:0/4060734861 >> 172.21.15.139:0/4060734861 conn(0x7efd640b5910 msgr2=0x7efd640b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.604+0000 7efd692b3640 1 -- 172.21.15.139:0/4060734861 shutdown_connections 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.604+0000 7efd692b3640 1 -- 172.21.15.139:0/4060734861 wait complete. 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.605+0000 7efd692b3640 1 Processor -- start 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.605+0000 7efd692b3640 1 -- start start 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd692b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd640d1080 0x7efd6414f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd692b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd6415dc60 0x7efd641e7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd692b3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd64161850 0x7efd640b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd692b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6415e040 con 0x7efd6415dc60 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd692b3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd640d1fc0 con 0x7efd64161850 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd692b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd641e9cd0 con 0x7efd640d1080 2024-04-25T09:59:08.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd627fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd6415dc60 0x7efd641e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd627fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd6415dc60 0x7efd641e7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48980/0 (socket says 172.21.15.139:48980) 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd627fc640 1 -- 172.21.15.139:0/1943093280 learned_addr learned my addr 172.21.15.139:0/1943093280 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd637fe640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd64161850 0x7efd640b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd627fc640 1 -- 172.21.15.139:0/1943093280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd640d1080 msgr2=0x7efd6414f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd627fc640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd640d1080 0x7efd6414f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd627fc640 1 -- 172.21.15.139:0/1943093280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd64161850 msgr2=0x7efd640b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd627fc640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd64161850 0x7efd640b88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd627fc640 1 -- 172.21.15.139:0/1943093280 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd641e7430 con 0x7efd6415dc60 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd627fc640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd6415dc60 0x7efd641e7930 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7efd58000c00 tx=0x7efd580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd4b7fe640 1 -- 172.21.15.139:0/1943093280 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58037040 con 0x7efd6415dc60 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd4b7fe640 1 -- 172.21.15.139:0/1943093280 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd58004230 con 0x7efd6415dc60 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd4b7fe640 1 -- 172.21.15.139:0/1943093280 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58032e40 con 0x7efd6415dc60 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd692b3640 1 -- 172.21.15.139:0/1943093280 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd641e9430 con 0x7efd6415dc60 2024-04-25T09:59:08.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.606+0000 7efd692b3640 1 -- 172.21.15.139:0/1943093280 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd641e99f0 con 0x7efd6415dc60 2024-04-25T09:59:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.607+0000 7efd692b3640 1 -- 172.21.15.139:0/1943093280 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd641e7600 con 0x7efd6415dc60 2024-04-25T09:59:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.608+0000 7efd4b7fe640 1 -- 172.21.15.139:0/1943093280 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd58030770 con 0x7efd6415dc60 2024-04-25T09:59:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.608+0000 7efd4b7fe640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd5006b4d0 0x7efd5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.608+0000 7efd4b7fe640 1 -- 172.21.15.139:0/1943093280 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd580ab6e0 con 0x7efd6415dc60 2024-04-25T09:59:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.608+0000 7efd62ffd640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd5006b4d0 0x7efd5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.609+0000 7efd62ffd640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd5006b4d0 0x7efd5006d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efd54007fd0 tx=0x7efd54001690 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:08.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.612+0000 7efd4b7fe640 1 -- 172.21.15.139:0/1943093280 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd5807a4f0 con 0x7efd6415dc60 2024-04-25T09:59:08.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.784+0000 7efd692b3640 1 -- 172.21.15.139:0/1943093280 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd64074520 con 0x7efd6415dc60 2024-04-25T09:59:08.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.785+0000 7efd4b7fe640 1 -- 172.21.15.139:0/1943093280 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16933 (secure 0 0 0) 0x7efd58079df0 con 0x7efd6415dc60 2024-04-25T09:59:08.789 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:08.790 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":89,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:58:59.031596+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:55.585380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:56.336230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:59:08.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.787+0000 7efd692b3640 1 -- 172.21.15.139:0/1943093280 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd5006b4d0 msgr2=0x7efd5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.787+0000 7efd692b3640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd5006b4d0 0x7efd5006d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efd54007fd0 tx=0x7efd54001690 comp rx=0 tx=0).stop 2024-04-25T09:59:08.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.787+0000 7efd692b3640 1 -- 172.21.15.139:0/1943093280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd6415dc60 msgr2=0x7efd641e7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.787+0000 7efd692b3640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd6415dc60 0x7efd641e7930 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7efd58000c00 tx=0x7efd580046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.787+0000 7efd692b3640 1 -- 172.21.15.139:0/1943093280 shutdown_connections 2024-04-25T09:59:08.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.787+0000 7efd692b3640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd6415dc60 0x7efd641e7930 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.787+0000 7efd692b3640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd5006b4d0 0x7efd5006d990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.787+0000 7efd692b3640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd64161850 0x7efd640b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.787+0000 7efd692b3640 1 --2- 172.21.15.139:0/1943093280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd640d1080 0x7efd6414f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.787+0000 7efd692b3640 1 -- 172.21.15.139:0/1943093280 >> 172.21.15.139:0/1943093280 conn(0x7efd640b5910 msgr2=0x7efd64073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:08.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.787+0000 7efd692b3640 1 -- 172.21.15.139:0/1943093280 shutdown_connections 2024-04-25T09:59:08.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.787+0000 7efd692b3640 1 -- 172.21.15.139:0/1943093280 wait complete. 2024-04-25T09:59:08.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:58:55.585380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:58:56.336230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:59:08.803 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:59:08.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.899+0000 7fb1d918b640 1 Processor -- start 2024-04-25T09:59:08.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.899+0000 7fb1d918b640 1 -- start start 2024-04-25T09:59:08.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.900+0000 7fb1d918b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb1d415bf90 0x7fb1d40cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.900+0000 7fb1d918b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d415c490 con 0x7fb1d40ce040 2024-04-25T09:59:08.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.900+0000 7fb1d918b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d415c770 con 0x7fb1d4071a50 2024-04-25T09:59:08.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.900+0000 7fb1d918b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d415ca50 con 0x7fb1d415bf90 2024-04-25T09:59:08.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.900+0000 7fb1d259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb1d415bf90 0x7fb1d40cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.900+0000 7fb1d259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb1d415bf90 0x7fb1d40cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:48994/0 (socket says 172.21.15.139:48994) 2024-04-25T09:59:08.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.900+0000 7fb1d259c640 1 -- 172.21.15.139:0/68827648 learned_addr learned my addr 172.21.15.139:0/68827648 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:08.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.900+0000 7fb1d359e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb1d40ce040 0x7fb1d40ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42474/0 (socket says 172.21.15.139:42474) 2024-04-25T09:59:08.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.900+0000 7fb1d259c640 1 -- 172.21.15.139:0/68827648 >> v1:172.21.15.150:6789/0 conn(0x7fb1d4071a50 legacy=0x7fb1d4071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.900+0000 7fb1d259c640 1 -- 172.21.15.139:0/68827648 >> v1:172.21.15.139:6789/0 conn(0x7fb1d40ce040 legacy=0x7fb1d40ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.900+0000 7fb1d259c640 1 -- 172.21.15.139:0/68827648 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d415cd30 con 0x7fb1d415bf90 2024-04-25T09:59:08.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.900+0000 7fb1d1d9b640 1 -- 172.21.15.139:0/68827648 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360647231 0 0) 0x7fb1d415c490 con 0x7fb1d40ce040 2024-04-25T09:59:08.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.901+0000 7fb1d259c640 1 --2- 172.21.15.139:0/68827648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d415bf90 0x7fb1d40cc2c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009870 tx=0x7fb1c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129dd5bf2600daf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:08.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.901+0000 7fb1d1d9b640 1 -- 172.21.15.139:0/68827648 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8037070 con 0x7fb1d415bf90 2024-04-25T09:59:08.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.901+0000 7fb1d1d9b640 1 -- 172.21.15.139:0/68827648 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c8032440 con 0x7fb1d415bf90 2024-04-25T09:59:08.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.901+0000 7fb1d1d9b640 1 -- 172.21.15.139:0/68827648 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8032810 con 0x7fb1d415bf90 2024-04-25T09:59:08.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.902+0000 7fb1d918b640 1 -- 172.21.15.139:0/68827648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d415bf90 msgr2=0x7fb1d40cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.902+0000 7fb1d918b640 1 --2- 172.21.15.139:0/68827648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d415bf90 0x7fb1d40cc2c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009870 tx=0x7fb1c802a270 comp rx=0 tx=0).stop 2024-04-25T09:59:08.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.902+0000 7fb1d918b640 1 -- 172.21.15.139:0/68827648 shutdown_connections 2024-04-25T09:59:08.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.902+0000 7fb1d918b640 1 --2- 172.21.15.139:0/68827648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d415bf90 0x7fb1d40cc2c0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.902+0000 7fb1d918b640 1 -- 172.21.15.139:0/68827648 >> 172.21.15.139:0/68827648 conn(0x7fb1d40cfd80 msgr2=0x7fb1d40d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:08.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.902+0000 7fb1d918b640 1 -- 172.21.15.139:0/68827648 shutdown_connections 2024-04-25T09:59:08.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.902+0000 7fb1d918b640 1 -- 172.21.15.139:0/68827648 wait complete. 2024-04-25T09:59:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.903+0000 7fb1d918b640 1 Processor -- start 2024-04-25T09:59:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.903+0000 7fb1d918b640 1 -- start start 2024-04-25T09:59:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d918b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4071a50 0x7fb1d409cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d918b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d40ce040 0x7fb1d40b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d918b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1d415bf90 0x7fb1d41583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d918b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d4156280 con 0x7fb1d40ce040 2024-04-25T09:59:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d918b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d40728b0 con 0x7fb1d4071a50 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d918b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d4156400 con 0x7fb1d415bf90 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d40ce040 0x7fb1d40b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d40ce040 0x7fb1d40b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49000/0 (socket says 172.21.15.139:49000) 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d259c640 1 -- 172.21.15.139:0/2834887035 learned_addr learned my addr 172.21.15.139:0/2834887035 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d359e640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1d415bf90 0x7fb1d41583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d2d9d640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4071a50 0x7fb1d409cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d259c640 1 -- 172.21.15.139:0/2834887035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1d415bf90 msgr2=0x7fb1d41583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d259c640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1d415bf90 0x7fb1d41583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d259c640 1 -- 172.21.15.139:0/2834887035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4071a50 msgr2=0x7fb1d409cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d259c640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4071a50 0x7fb1d409cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d259c640 1 -- 172.21.15.139:0/2834887035 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d415a790 con 0x7fb1d40ce040 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d2d9d640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4071a50 0x7fb1d409cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:08.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d359e640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1d415bf90 0x7fb1d41583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1d259c640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d40ce040 0x7fb1d40b0050 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80090b0 tx=0x7fb1c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1bb7fe640 1 -- 172.21.15.139:0/2834887035 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8041020 con 0x7fb1d40ce040 2024-04-25T09:59:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.904+0000 7fb1bb7fe640 1 -- 172.21.15.139:0/2834887035 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c8030990 con 0x7fb1d40ce040 2024-04-25T09:59:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.905+0000 7fb1d918b640 1 -- 172.21.15.139:0/2834887035 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d415aa30 con 0x7fb1d40ce040 2024-04-25T09:59:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.905+0000 7fb1bb7fe640 1 -- 172.21.15.139:0/2834887035 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8037040 con 0x7fb1d40ce040 2024-04-25T09:59:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.905+0000 7fb1d918b640 1 -- 172.21.15.139:0/2834887035 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d415aff0 con 0x7fb1d40ce040 2024-04-25T09:59:08.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.905+0000 7fb1d918b640 1 -- 172.21.15.139:0/2834887035 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d4074520 con 0x7fb1d40ce040 2024-04-25T09:59:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.906+0000 7fb1bb7fe640 1 -- 172.21.15.139:0/2834887035 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1c8044020 con 0x7fb1d40ce040 2024-04-25T09:59:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.906+0000 7fb1bb7fe640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb1c006b500 0x7fb1c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.906+0000 7fb1d2d9d640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb1c006b500 0x7fb1c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.906+0000 7fb1bb7fe640 1 -- 172.21.15.139:0/2834887035 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb1c80aaae0 con 0x7fb1d40ce040 2024-04-25T09:59:08.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.907+0000 7fb1d2d9d640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb1c006b500 0x7fb1c006d9c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4007fd0 tx=0x7fb1c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:08.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:08.911+0000 7fb1bb7fe640 1 -- 172.21.15.139:0/2834887035 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1c804b070 con 0x7fb1d40ce040 2024-04-25T09:59:09.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.080+0000 7fb1d918b640 1 -- 172.21.15.139:0/2834887035 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1d40b0fb0 con 0x7fb1d40ce040 2024-04-25T09:59:09.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.081+0000 7fb1bb7fe640 1 -- 172.21.15.139:0/2834887035 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1c80797d0 con 0x7fb1d40ce040 2024-04-25T09:59:09.085 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T09:59:09.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.083+0000 7fb1d918b640 1 -- 172.21.15.139:0/2834887035 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb1c006b500 msgr2=0x7fb1c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.083+0000 7fb1d918b640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb1c006b500 0x7fb1c006d9c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4007fd0 tx=0x7fb1c4009040 comp rx=0 tx=0).stop 2024-04-25T09:59:09.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.083+0000 7fb1d918b640 1 -- 172.21.15.139:0/2834887035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d40ce040 msgr2=0x7fb1d40b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.083+0000 7fb1d918b640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d40ce040 0x7fb1d40b0050 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80090b0 tx=0x7fb1c80046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.083+0000 7fb1d918b640 1 -- 172.21.15.139:0/2834887035 shutdown_connections 2024-04-25T09:59:09.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.083+0000 7fb1d918b640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1d40ce040 0x7fb1d40b0050 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.083+0000 7fb1d918b640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb1c006b500 0x7fb1c006d9c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.083+0000 7fb1d918b640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1d415bf90 0x7fb1d41583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.083+0000 7fb1d918b640 1 --2- 172.21.15.139:0/2834887035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4071a50 0x7fb1d409cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.083+0000 7fb1d918b640 1 -- 172.21.15.139:0/2834887035 >> 172.21.15.139:0/2834887035 conn(0x7fb1d40cfd80 msgr2=0x7fb1d415d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:09.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.083+0000 7fb1d918b640 1 -- 172.21.15.139:0/2834887035 shutdown_connections 2024-04-25T09:59:09.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.083+0000 7fb1d918b640 1 -- 172.21.15.139:0/2834887035 wait complete. 2024-04-25T09:59:09.097 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:59:09.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.194+0000 7f574fd41640 1 Processor -- start 2024-04-25T09:59:09.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.194+0000 7f574fd41640 1 -- start start 2024-04-25T09:59:09.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574fd41640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574806ebf0 0x7f574806f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574fd41640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f57480b2db0 0x7f57480b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574fd41640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57480cf940 con 0x7f574815dd90 2024-04-25T09:59:09.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574fd41640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574806ab60 con 0x7f57480b2db0 2024-04-25T09:59:09.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574fd41640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574806adb0 con 0x7f574806ebf0 2024-04-25T09:59:09.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574e2de640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f57480b2db0 0x7f57480b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574e2de640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f57480b2db0 0x7f57480b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49004/0 (socket says 172.21.15.139:49004) 2024-04-25T09:59:09.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574e2de640 1 -- 172.21.15.139:0/913839846 learned_addr learned my addr 172.21.15.139:0/913839846 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:09.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574d2dc640 1 --2- 172.21.15.139:0/913839846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574806ebf0 0x7f574806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574e2de640 1 -- 172.21.15.139:0/913839846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574806ebf0 msgr2=0x7f574806f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:09.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574e2de640 1 --2- 172.21.15.139:0/913839846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574806ebf0 0x7f574806f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574e2de640 1 -- 172.21.15.139:0/913839846 >> v1:172.21.15.150:6789/0 conn(0x7f574815dd90 legacy=0x7f57480ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.195+0000 7f574e2de640 1 -- 172.21.15.139:0/913839846 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574806b090 con 0x7f57480b2db0 2024-04-25T09:59:09.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.196+0000 7f574e2de640 1 --2- 172.21.15.139:0/913839846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57480b2db0 0x7f57480b3240 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f5740009e50 tx=0x7f574002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcfaa2b221c35c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.196+0000 7f574cadb640 1 -- 172.21.15.139:0/913839846 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740036070 con 0x7f57480b2db0 2024-04-25T09:59:09.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.196+0000 7f574cadb640 1 -- 172.21.15.139:0/913839846 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5740031600 con 0x7f57480b2db0 2024-04-25T09:59:09.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.196+0000 7f574cadb640 1 -- 172.21.15.139:0/913839846 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57400319d0 con 0x7f57480b2db0 2024-04-25T09:59:09.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.197+0000 7f574fd41640 1 -- 172.21.15.139:0/913839846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57480b2db0 msgr2=0x7f57480b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.197+0000 7f574fd41640 1 --2- 172.21.15.139:0/913839846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57480b2db0 0x7f57480b3240 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f5740009e50 tx=0x7f574002a710 comp rx=0 tx=0).stop 2024-04-25T09:59:09.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.197+0000 7f574fd41640 1 -- 172.21.15.139:0/913839846 shutdown_connections 2024-04-25T09:59:09.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.197+0000 7f574fd41640 1 --2- 172.21.15.139:0/913839846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57480b2db0 0x7f57480b3240 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.197+0000 7f574fd41640 1 --2- 172.21.15.139:0/913839846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574806ebf0 0x7f574806f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.197+0000 7f574fd41640 1 -- 172.21.15.139:0/913839846 >> 172.21.15.139:0/913839846 conn(0x7f57480cfd80 msgr2=0x7f57480d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:09.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.197+0000 7f574fd41640 1 -- 172.21.15.139:0/913839846 shutdown_connections 2024-04-25T09:59:09.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.197+0000 7f574fd41640 1 -- 172.21.15.139:0/913839846 wait complete. 2024-04-25T09:59:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.198+0000 7f574fd41640 1 Processor -- start 2024-04-25T09:59:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.198+0000 7f574fd41640 1 -- start start 2024-04-25T09:59:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.198+0000 7f574fd41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f574806ebf0 0x7f5748148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.198+0000 7f574fd41640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57480b2db0 0x7f57481e8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574fd41640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574815dd90 0x7f57481e8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574fd41640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5748072620 con 0x7f574806ebf0 2024-04-25T09:59:09.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574fd41640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574806f570 con 0x7f57480b2db0 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574fd41640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57480cf5b0 con 0x7f574815dd90 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574dadd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f574806ebf0 0x7f5748148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574dadd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f574806ebf0 0x7f5748148a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49020/0 (socket says 172.21.15.139:49020) 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574dadd640 1 -- 172.21.15.139:0/2013072037 learned_addr learned my addr 172.21.15.139:0/2013072037 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574d2dc640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57480b2db0 0x7f57481e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574e2de640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574815dd90 0x7f57481e8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574dadd640 1 -- 172.21.15.139:0/2013072037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574815dd90 msgr2=0x7f57481e8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574dadd640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574815dd90 0x7f57481e8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574dadd640 1 -- 172.21.15.139:0/2013072037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57480b2db0 msgr2=0x7f57481e8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574dadd640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57480b2db0 0x7f57481e8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574dadd640 1 -- 172.21.15.139:0/2013072037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57481e6ce0 con 0x7f574806ebf0 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574dadd640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f574806ebf0 0x7f5748148a20 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f574400c3c0 tx=0x7f574400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f573e7fc640 1 -- 172.21.15.139:0/2013072037 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57440052d0 con 0x7f574806ebf0 2024-04-25T09:59:09.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f573e7fc640 1 -- 172.21.15.139:0/2013072037 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5744005490 con 0x7f574806ebf0 2024-04-25T09:59:09.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574fd41640 1 -- 172.21.15.139:0/2013072037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57481e6fc0 con 0x7f574806ebf0 2024-04-25T09:59:09.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f573e7fc640 1 -- 172.21.15.139:0/2013072037 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744007260 con 0x7f574806ebf0 2024-04-25T09:59:09.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.199+0000 7f574fd41640 1 -- 172.21.15.139:0/2013072037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57481e7580 con 0x7f574806ebf0 2024-04-25T09:59:09.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.200+0000 7f574fd41640 1 -- 172.21.15.139:0/2013072037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5748074520 con 0x7f574806ebf0 2024-04-25T09:59:09.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.201+0000 7f573e7fc640 1 -- 172.21.15.139:0/2013072037 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5744007400 con 0x7f574806ebf0 2024-04-25T09:59:09.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.201+0000 7f573e7fc640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f572c06b4d0 0x7f572c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.201+0000 7f573e7fc640 1 -- 172.21.15.139:0/2013072037 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57440897c0 con 0x7f574806ebf0 2024-04-25T09:59:09.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.201+0000 7f574d2dc640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f572c06b4d0 0x7f572c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.202+0000 7f574d2dc640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f572c06b4d0 0x7f572c06d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f57481f6da0 tx=0x7f5738008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.205+0000 7f573e7fc640 1 -- 172.21.15.139:0/2013072037 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5744057e60 con 0x7f574806ebf0 2024-04-25T09:59:09.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.377+0000 7f574fd41640 1 -- 172.21.15.139:0/2013072037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5748073cc0 con 0x7f574806ebf0 2024-04-25T09:59:09.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.377+0000 7f573e7fc640 1 -- 172.21.15.139:0/2013072037 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f574405ce20 con 0x7f574806ebf0 2024-04-25T09:59:09.381 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T09:59:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.379+0000 7f574fd41640 1 -- 172.21.15.139:0/2013072037 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f572c06b4d0 msgr2=0x7f572c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.379+0000 7f574fd41640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f572c06b4d0 0x7f572c06d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f57481f6da0 tx=0x7f5738008040 comp rx=0 tx=0).stop 2024-04-25T09:59:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.380+0000 7f574fd41640 1 -- 172.21.15.139:0/2013072037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f574806ebf0 msgr2=0x7f5748148a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.380+0000 7f574fd41640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f574806ebf0 0x7f5748148a20 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f574400c3c0 tx=0x7f574400c880 comp rx=0 tx=0).stop 2024-04-25T09:59:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.380+0000 7f574fd41640 1 -- 172.21.15.139:0/2013072037 shutdown_connections 2024-04-25T09:59:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.380+0000 7f574fd41640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f572c06b4d0 0x7f572c06d990 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.380+0000 7f574fd41640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574815dd90 0x7f57481e8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.380+0000 7f574fd41640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57480b2db0 0x7f57481e8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.380+0000 7f574fd41640 1 --2- 172.21.15.139:0/2013072037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f574806ebf0 0x7f5748148a20 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.380+0000 7f574fd41640 1 -- 172.21.15.139:0/2013072037 >> 172.21.15.139:0/2013072037 conn(0x7f57480cfd80 msgr2=0x7f57480ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.380+0000 7f574fd41640 1 -- 172.21.15.139:0/2013072037 shutdown_connections 2024-04-25T09:59:09.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.380+0000 7f574fd41640 1 -- 172.21.15.139:0/2013072037 wait complete. 2024-04-25T09:59:09.394 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:59:09.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.491+0000 7f5d53f53640 1 Processor -- start 2024-04-25T09:59:09.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.491+0000 7f5d53f53640 1 -- start start 2024-04-25T09:59:09.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.491+0000 7f5d53f53640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5d4c15f8c0 0x7f5d4c09a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d53f53640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5d4c15fca0 0x7f5d4c0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d53f53640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d4c099bf0 con 0x7f5d4c0d0220 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d53f53640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c0d06c0 con 0x7f5d4c15fca0 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d53f53640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c099ed0 con 0x7f5d4c15f8c0 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d514ee640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5d4c15fca0 0x7f5d4c0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d524f0640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5d4c0d0220 0x7f5d4c15b560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42484/0 (socket says 172.21.15.139:42484) 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d524f0640 1 -- 172.21.15.139:0/3075836059 learned_addr learned my addr 172.21.15.139:0/3075836059 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d51cef640 1 --2- 172.21.15.139:0/3075836059 >> v2:172.21.15.150:3300/0 conn(0x7f5d4c15f8c0 0x7f5d4c09a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d50ced640 1 -- 172.21.15.139:0/3075836059 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437408176 0 0) 0x7f5d4c099bf0 con 0x7f5d4c0d0220 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d50ced640 1 -- 172.21.15.139:0/3075836059 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d38003520 con 0x7f5d4c0d0220 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d514ee640 1 -- 172.21.15.139:0/3075836059 >> v2:172.21.15.150:3300/0 conn(0x7f5d4c15f8c0 msgr2=0x7f5d4c09a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d514ee640 1 --2- 172.21.15.139:0/3075836059 >> v2:172.21.15.150:3300/0 conn(0x7f5d4c15f8c0 0x7f5d4c09a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d514ee640 1 -- 172.21.15.139:0/3075836059 >> v1:172.21.15.139:6789/0 conn(0x7f5d4c0d0220 legacy=0x7f5d4c15b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d514ee640 1 -- 172.21.15.139:0/3075836059 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c09a1b0 con 0x7f5d4c15fca0 2024-04-25T09:59:09.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.492+0000 7f5d51cef640 1 --2- 172.21.15.139:0/3075836059 >> v2:172.21.15.150:3300/0 conn(0x7f5d4c15f8c0 0x7f5d4c09a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:09.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.493+0000 7f5d514ee640 1 --2- 172.21.15.139:0/3075836059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d4c15fca0 0x7f5d4c0cfce0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5d480097a0 tx=0x7f5d4802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9f49b568c3e009 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.493+0000 7f5d50ced640 1 -- 172.21.15.139:0/3075836059 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48037070 con 0x7f5d4c15fca0 2024-04-25T09:59:09.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.493+0000 7f5d50ced640 1 -- 172.21.15.139:0/3075836059 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d48032440 con 0x7f5d4c15fca0 2024-04-25T09:59:09.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.493+0000 7f5d50ced640 1 -- 172.21.15.139:0/3075836059 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48032810 con 0x7f5d4c15fca0 2024-04-25T09:59:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.493+0000 7f5d53f53640 1 -- 172.21.15.139:0/3075836059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d4c15fca0 msgr2=0x7f5d4c0cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.493+0000 7f5d53f53640 1 --2- 172.21.15.139:0/3075836059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d4c15fca0 0x7f5d4c0cfce0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5d480097a0 tx=0x7f5d4802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.493+0000 7f5d53f53640 1 -- 172.21.15.139:0/3075836059 shutdown_connections 2024-04-25T09:59:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.493+0000 7f5d53f53640 1 --2- 172.21.15.139:0/3075836059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d4c15fca0 0x7f5d4c0cfce0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.493+0000 7f5d53f53640 1 --2- 172.21.15.139:0/3075836059 >> v2:172.21.15.150:3300/0 conn(0x7f5d4c15f8c0 0x7f5d4c09a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.493+0000 7f5d53f53640 1 -- 172.21.15.139:0/3075836059 >> 172.21.15.139:0/3075836059 conn(0x7f5d4c0b31f0 msgr2=0x7f5d4c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.494+0000 7f5d53f53640 1 -- 172.21.15.139:0/3075836059 shutdown_connections 2024-04-25T09:59:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.494+0000 7f5d53f53640 1 -- 172.21.15.139:0/3075836059 wait complete. 2024-04-25T09:59:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.495+0000 7f5d53f53640 1 Processor -- start 2024-04-25T09:59:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.495+0000 7f5d53f53640 1 -- start start 2024-04-25T09:59:09.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.495+0000 7f5d53f53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d4c0d0220 0x7f5d4c146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.495+0000 7f5d53f53640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d4c15f8c0 0x7f5d4c156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.495+0000 7f5d53f53640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d4c15fca0 0x7f5d4c157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.495+0000 7f5d53f53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c1602d0 con 0x7f5d4c0d0220 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.495+0000 7f5d53f53640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c09aef0 con 0x7f5d4c15f8c0 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.495+0000 7f5d53f53640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c160450 con 0x7f5d4c15fca0 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.495+0000 7f5d51cef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d4c0d0220 0x7f5d4c146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.495+0000 7f5d51cef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d4c0d0220 0x7f5d4c146b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49032/0 (socket says 172.21.15.139:49032) 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.495+0000 7f5d51cef640 1 -- 172.21.15.139:0/194473920 learned_addr learned my addr 172.21.15.139:0/194473920 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d514ee640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d4c15f8c0 0x7f5d4c156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d524f0640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d4c15fca0 0x7f5d4c157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d51cef640 1 -- 172.21.15.139:0/194473920 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d4c15fca0 msgr2=0x7f5d4c157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d51cef640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d4c15fca0 0x7f5d4c157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d51cef640 1 -- 172.21.15.139:0/194473920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d4c15f8c0 msgr2=0x7f5d4c156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d51cef640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d4c15f8c0 0x7f5d4c156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d51cef640 1 -- 172.21.15.139:0/194473920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c0ca140 con 0x7f5d4c0d0220 2024-04-25T09:59:09.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d51cef640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d4c0d0220 0x7f5d4c146b60 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000e810 tx=0x7f5d4000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d3e7fc640 1 -- 172.21.15.139:0/194473920 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000f040 con 0x7f5d4c0d0220 2024-04-25T09:59:09.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d3e7fc640 1 -- 172.21.15.139:0/194473920 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d40004030 con 0x7f5d4c0d0220 2024-04-25T09:59:09.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d53f53640 1 -- 172.21.15.139:0/194473920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c159070 con 0x7f5d4c0d0220 2024-04-25T09:59:09.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d3e7fc640 1 -- 172.21.15.139:0/194473920 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40004400 con 0x7f5d4c0d0220 2024-04-25T09:59:09.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.496+0000 7f5d53f53640 1 -- 172.21.15.139:0/194473920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c159630 con 0x7f5d4c0d0220 2024-04-25T09:59:09.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.497+0000 7f5d53f53640 1 -- 172.21.15.139:0/194473920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c074520 con 0x7f5d4c0d0220 2024-04-25T09:59:09.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.497+0000 7f5d3e7fc640 1 -- 172.21.15.139:0/194473920 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d4000c6d0 con 0x7f5d4c0d0220 2024-04-25T09:59:09.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.498+0000 7f5d3e7fc640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5d3006b4d0 0x7f5d3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.498+0000 7f5d3e7fc640 1 -- 172.21.15.139:0/194473920 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5d4008a170 con 0x7f5d4c0d0220 2024-04-25T09:59:09.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.498+0000 7f5d514ee640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5d3006b4d0 0x7f5d3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.498+0000 7f5d514ee640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5d3006b4d0 0x7f5d3006d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5d48034000 tx=0x7f5d4802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.502+0000 7f5d3e7fc640 1 -- 172.21.15.139:0/194473920 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d40017020 con 0x7f5d4c0d0220 2024-04-25T09:59:09.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.674+0000 7f5d53f53640 1 -- 172.21.15.139:0/194473920 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d4c0cdc30 con 0x7f5d4c0d0220 2024-04-25T09:59:09.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.674+0000 7f5d3e7fc640 1 -- 172.21.15.139:0/194473920 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d40058e60 con 0x7f5d4c0d0220 2024-04-25T09:59:09.678 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T09:59:09.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.677+0000 7f5d53f53640 1 -- 172.21.15.139:0/194473920 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5d3006b4d0 msgr2=0x7f5d3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.677+0000 7f5d53f53640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5d3006b4d0 0x7f5d3006d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5d48034000 tx=0x7f5d4802e000 comp rx=0 tx=0).stop 2024-04-25T09:59:09.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.677+0000 7f5d53f53640 1 -- 172.21.15.139:0/194473920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d4c0d0220 msgr2=0x7f5d4c146b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.677+0000 7f5d53f53640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d4c0d0220 0x7f5d4c146b60 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000e810 tx=0x7f5d4000ecd0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.677+0000 7f5d53f53640 1 -- 172.21.15.139:0/194473920 shutdown_connections 2024-04-25T09:59:09.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.677+0000 7f5d53f53640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5d3006b4d0 0x7f5d3006d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.677+0000 7f5d53f53640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d4c15fca0 0x7f5d4c157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.677+0000 7f5d53f53640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d4c15f8c0 0x7f5d4c156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.677+0000 7f5d53f53640 1 --2- 172.21.15.139:0/194473920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d4c0d0220 0x7f5d4c146b60 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.677+0000 7f5d53f53640 1 -- 172.21.15.139:0/194473920 >> 172.21.15.139:0/194473920 conn(0x7f5d4c0b31f0 msgr2=0x7f5d4c0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:09.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.677+0000 7f5d53f53640 1 -- 172.21.15.139:0/194473920 shutdown_connections 2024-04-25T09:59:09.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.677+0000 7f5d53f53640 1 -- 172.21.15.139:0/194473920 wait complete. 2024-04-25T09:59:09.692 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:59:09.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.785+0000 7ff82be12640 1 Processor -- start 2024-04-25T09:59:09.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.785+0000 7ff82be12640 1 -- start start 2024-04-25T09:59:09.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.786+0000 7ff82be12640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff82415da30 0x7ff8240704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.786+0000 7ff82be12640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff824072d60 0x7ff82406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.786+0000 7ff82be12640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8240731a0 con 0x7ff82406b0a0 2024-04-25T09:59:09.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.786+0000 7ff82be12640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82406b4f0 con 0x7ff824072d60 2024-04-25T09:59:09.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.786+0000 7ff82be12640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8240cd540 con 0x7ff82415da30 2024-04-25T09:59:09.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.786+0000 7ff8293ad640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff824072d60 0x7ff82406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.786+0000 7ff8293ad640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff824072d60 0x7ff82406ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49038/0 (socket says 172.21.15.139:49038) 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.786+0000 7ff82a3af640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff82406b0a0 0x7ff8240d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42498/0 (socket says 172.21.15.139:42498) 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.786+0000 7ff8293ad640 1 -- 172.21.15.139:0/3032691793 learned_addr learned my addr 172.21.15.139:0/3032691793 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.787+0000 7ff8293ad640 1 -- 172.21.15.139:0/3032691793 >> v2:172.21.15.150:3300/0 conn(0x7ff82415da30 msgr2=0x7ff8240704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.787+0000 7ff829bae640 1 --2- 172.21.15.139:0/3032691793 >> v2:172.21.15.150:3300/0 conn(0x7ff82415da30 0x7ff8240704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.787+0000 7ff8293ad640 1 --2- 172.21.15.139:0/3032691793 >> v2:172.21.15.150:3300/0 conn(0x7ff82415da30 0x7ff8240704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.787+0000 7ff8293ad640 1 -- 172.21.15.139:0/3032691793 >> v1:172.21.15.139:6789/0 conn(0x7ff82406b0a0 legacy=0x7ff8240d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.787+0000 7ff828bac640 1 -- 172.21.15.139:0/3032691793 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368104098 0 0) 0x7ff8240731a0 con 0x7ff82406b0a0 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.787+0000 7ff8293ad640 1 -- 172.21.15.139:0/3032691793 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82415de10 con 0x7ff824072d60 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.787+0000 7ff829bae640 1 --2- 172.21.15.139:0/3032691793 >> v2:172.21.15.150:3300/0 conn(0x7ff82415da30 0x7ff8240704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.787+0000 7ff8293ad640 1 --2- 172.21.15.139:0/3032691793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff824072d60 0x7ff82406ab60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff8240b5bc0 tx=0x7ff81402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c25c0e674a63c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.788+0000 7ff828bac640 1 -- 172.21.15.139:0/3032691793 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814037070 con 0x7ff824072d60 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.788+0000 7ff828bac640 1 -- 172.21.15.139:0/3032691793 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff814032440 con 0x7ff824072d60 2024-04-25T09:59:09.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.788+0000 7ff828bac640 1 -- 172.21.15.139:0/3032691793 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814032810 con 0x7ff824072d60 2024-04-25T09:59:09.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.788+0000 7ff82be12640 1 -- 172.21.15.139:0/3032691793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff824072d60 msgr2=0x7ff82406ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.788+0000 7ff82be12640 1 --2- 172.21.15.139:0/3032691793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff824072d60 0x7ff82406ab60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff8240b5bc0 tx=0x7ff81402a270 comp rx=0 tx=0).stop 2024-04-25T09:59:09.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.788+0000 7ff82be12640 1 -- 172.21.15.139:0/3032691793 shutdown_connections 2024-04-25T09:59:09.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.788+0000 7ff82be12640 1 --2- 172.21.15.139:0/3032691793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff824072d60 0x7ff82406ab60 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.788+0000 7ff82be12640 1 --2- 172.21.15.139:0/3032691793 >> v2:172.21.15.150:3300/0 conn(0x7ff82415da30 0x7ff8240704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.788+0000 7ff82be12640 1 -- 172.21.15.139:0/3032691793 >> 172.21.15.139:0/3032691793 conn(0x7ff8240b23f0 msgr2=0x7ff8240b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.788+0000 7ff82be12640 1 -- 172.21.15.139:0/3032691793 shutdown_connections 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.788+0000 7ff82be12640 1 -- 172.21.15.139:0/3032691793 wait complete. 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.789+0000 7ff82be12640 1 Processor -- start 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.789+0000 7ff82be12640 1 -- start start 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82be12640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff82406b0a0 0x7ff824159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82be12640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff824072d60 0x7ff8240cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82be12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff82415da30 0x7ff824160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82be12640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240b3b00 con 0x7ff82415da30 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82be12640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240b3de0 con 0x7ff824072d60 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82be12640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240b42a0 con 0x7ff82406b0a0 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82a3af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff82415da30 0x7ff824160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82a3af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff82415da30 0x7ff824160950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49048/0 (socket says 172.21.15.139:49048) 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82a3af640 1 -- 172.21.15.139:0/217286800 learned_addr learned my addr 172.21.15.139:0/217286800 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:09.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82a3af640 1 -- 172.21.15.139:0/217286800 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff82406b0a0 msgr2=0x7ff824159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82a3af640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff82406b0a0 0x7ff824159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82a3af640 1 -- 172.21.15.139:0/217286800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff824072d60 msgr2=0x7ff8240cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff8293ad640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff824072d60 0x7ff8240cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82a3af640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff824072d60 0x7ff8240cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.790+0000 7ff82a3af640 1 -- 172.21.15.139:0/217286800 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8240cc1d0 con 0x7ff82415da30 2024-04-25T09:59:09.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.791+0000 7ff82a3af640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff82415da30 0x7ff824160950 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00c440 tx=0x7ff81c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.791+0000 7ff81a7fc640 1 -- 172.21.15.139:0/217286800 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0074e0 con 0x7ff82415da30 2024-04-25T09:59:09.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.791+0000 7ff81a7fc640 1 -- 172.21.15.139:0/217286800 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c0047e0 con 0x7ff82415da30 2024-04-25T09:59:09.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.791+0000 7ff82be12640 1 -- 172.21.15.139:0/217286800 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8240cc4b0 con 0x7ff82415da30 2024-04-25T09:59:09.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.791+0000 7ff81a7fc640 1 -- 172.21.15.139:0/217286800 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004bd0 con 0x7ff82415da30 2024-04-25T09:59:09.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.791+0000 7ff82be12640 1 -- 172.21.15.139:0/217286800 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff824160e90 con 0x7ff82415da30 2024-04-25T09:59:09.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.792+0000 7ff82be12640 1 -- 172.21.15.139:0/217286800 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff824074520 con 0x7ff82415da30 2024-04-25T09:59:09.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.792+0000 7ff81a7fc640 1 -- 172.21.15.139:0/217286800 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff81c01a020 con 0x7ff82415da30 2024-04-25T09:59:09.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.793+0000 7ff81a7fc640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff80806b4d0 0x7ff80806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:09.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.793+0000 7ff81a7fc640 1 -- 172.21.15.139:0/217286800 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff81c089b60 con 0x7ff82415da30 2024-04-25T09:59:09.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.793+0000 7ff829bae640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff80806b4d0 0x7ff80806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:09.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.793+0000 7ff829bae640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff80806b4d0 0x7ff80806d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff820008ea0 tx=0x7ff8200083d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:09.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.797+0000 7ff81a7fc640 1 -- 172.21.15.139:0/217286800 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff81c058910 con 0x7ff82415da30 2024-04-25T09:59:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.977+0000 7ff82be12640 1 -- 172.21.15.139:0/217286800 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff8240d1160 con 0x7ff82415da30 2024-04-25T09:59:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.978+0000 7ff81a7fc640 1 -- 172.21.15.139:0/217286800 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff81c058210 con 0x7ff82415da30 2024-04-25T09:59:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T09:59:09.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.979+0000 7ff82be12640 1 -- 172.21.15.139:0/217286800 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff80806b4d0 msgr2=0x7ff80806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.979+0000 7ff82be12640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff80806b4d0 0x7ff80806d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff820008ea0 tx=0x7ff8200083d0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.979+0000 7ff82be12640 1 -- 172.21.15.139:0/217286800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff82415da30 msgr2=0x7ff824160950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:09.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.979+0000 7ff82be12640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff82415da30 0x7ff824160950 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00c440 tx=0x7ff81c00c900 comp rx=0 tx=0).stop 2024-04-25T09:59:09.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.979+0000 7ff82be12640 1 -- 172.21.15.139:0/217286800 shutdown_connections 2024-04-25T09:59:09.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.980+0000 7ff82be12640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff82415da30 0x7ff824160950 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.980+0000 7ff82be12640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff80806b4d0 0x7ff80806d990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.980+0000 7ff82be12640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff824072d60 0x7ff8240cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.980+0000 7ff82be12640 1 --2- 172.21.15.139:0/217286800 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff82406b0a0 0x7ff824159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:09.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.980+0000 7ff82be12640 1 -- 172.21.15.139:0/217286800 >> 172.21.15.139:0/217286800 conn(0x7ff8240b23f0 msgr2=0x7ff82406e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:09.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.980+0000 7ff82be12640 1 -- 172.21.15.139:0/217286800 shutdown_connections 2024-04-25T09:59:09.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:09.980+0000 7ff82be12640 1 -- 172.21.15.139:0/217286800 wait complete. 2024-04-25T09:59:09.994 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:59:09.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:09.994 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:10.049 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T09:59:10.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:10.050 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:59:10.073 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:10.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.207+0000 7f0f21ecb640 1 Processor -- start 2024-04-25T09:59:10.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.207+0000 7f0f21ecb640 1 -- start start 2024-04-25T09:59:10.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.207+0000 7f0f21ecb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0f1c15ba70 0x7f0f1c15be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.207+0000 7f0f21ecb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0f1c0cf7b0 0x7f0f1c0cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.207+0000 7f0f21ecb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1c15f420 con 0x7f0f1c0cbd20 2024-04-25T09:59:10.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.207+0000 7f0f21ecb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f1c0d0100 con 0x7f0f1c15ba70 2024-04-25T09:59:10.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.207+0000 7f0f21ecb640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f1c15c710 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.208+0000 7f0f1b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0f1c15ba70 0x7f0f1c15be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.208+0000 7f0f1affd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0f1c0cbd20 0x7f0f1c15fc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42514/0 (socket says 172.21.15.139:42514) 2024-04-25T09:59:10.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.208+0000 7f0f1affd640 1 -- 172.21.15.139:0/1085981441 learned_addr learned my addr 172.21.15.139:0/1085981441 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:10.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.208+0000 7f0f1b7fe640 1 -- 172.21.15.139:0/1085981441 >> v2:172.21.15.150:3300/0 conn(0x7f0f1c0cf7b0 msgr2=0x7f0f1c0cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:10.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.208+0000 7f0f1b7fe640 1 --2- 172.21.15.139:0/1085981441 >> v2:172.21.15.150:3300/0 conn(0x7f0f1c0cf7b0 0x7f0f1c0cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.208+0000 7f0f1b7fe640 1 -- 172.21.15.139:0/1085981441 >> v1:172.21.15.139:6789/0 conn(0x7f0f1c0cbd20 legacy=0x7f0f1c15fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.208+0000 7f0f1b7fe640 1 -- 172.21.15.139:0/1085981441 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c15c9f0 con 0x7f0f1c15ba70 2024-04-25T09:59:10.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.208+0000 7f0f1b7fe640 1 --2- 172.21.15.139:0/1085981441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f1c15ba70 0x7f0f1c15be50 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0b76f0 tx=0x7f0f0c02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562d799f2e63259a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.209+0000 7f0f1a7fc640 1 -- 172.21.15.139:0/1085981441 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c037070 con 0x7f0f1c15ba70 2024-04-25T09:59:10.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.209+0000 7f0f1a7fc640 1 -- 172.21.15.139:0/1085981441 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c02ad30 con 0x7f0f1c15ba70 2024-04-25T09:59:10.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.209+0000 7f0f1a7fc640 1 -- 172.21.15.139:0/1085981441 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c032660 con 0x7f0f1c15ba70 2024-04-25T09:59:10.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.209+0000 7f0f21ecb640 1 -- 172.21.15.139:0/1085981441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f1c15ba70 msgr2=0x7f0f1c15be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.209+0000 7f0f21ecb640 1 --2- 172.21.15.139:0/1085981441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f1c15ba70 0x7f0f1c15be50 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0b76f0 tx=0x7f0f0c02a0a0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.209+0000 7f0f21ecb640 1 -- 172.21.15.139:0/1085981441 shutdown_connections 2024-04-25T09:59:10.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.209+0000 7f0f21ecb640 1 --2- 172.21.15.139:0/1085981441 >> v2:172.21.15.150:3300/0 conn(0x7f0f1c0cf7b0 0x7f0f1c0cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.209+0000 7f0f21ecb640 1 --2- 172.21.15.139:0/1085981441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f1c15ba70 0x7f0f1c15be50 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.209+0000 7f0f21ecb640 1 -- 172.21.15.139:0/1085981441 >> 172.21.15.139:0/1085981441 conn(0x7f0f1c0b3380 msgr2=0x7f0f1c0b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.209+0000 7f0f21ecb640 1 -- 172.21.15.139:0/1085981441 shutdown_connections 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.209+0000 7f0f21ecb640 1 -- 172.21.15.139:0/1085981441 wait complete. 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.210+0000 7f0f21ecb640 1 Processor -- start 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.211+0000 7f0f21ecb640 1 -- start start 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.211+0000 7f0f21ecb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f1c0cbd20 0x7f0f1c146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.211+0000 7f0f21ecb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f1c0cf7b0 0x7f0f1c0cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.211+0000 7f0f21ecb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f1c15ba70 0x7f0f1c0ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.211+0000 7f0f21ecb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c160310 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.211+0000 7f0f21ecb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c15f6e0 con 0x7f0f1c15ba70 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.211+0000 7f0f21ecb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c15c390 con 0x7f0f1c0cbd20 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.211+0000 7f0f1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f1c0cf7b0 0x7f0f1c0cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.211+0000 7f0f1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f1c0cf7b0 0x7f0f1c0cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49062/0 (socket says 172.21.15.139:49062) 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.211+0000 7f0f1affd640 1 -- 172.21.15.139:0/2441521775 learned_addr learned my addr 172.21.15.139:0/2441521775 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:10.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f1bfff640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f1c15ba70 0x7f0f1c0ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f1b7fe640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f1c0cbd20 0x7f0f1c146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f1affd640 1 -- 172.21.15.139:0/2441521775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f1c0cbd20 msgr2=0x7f0f1c146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f1affd640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f1c0cbd20 0x7f0f1c146010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f1affd640 1 -- 172.21.15.139:0/2441521775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f1c15ba70 msgr2=0x7f0f1c0ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f1affd640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f1c15ba70 0x7f0f1c0ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f1affd640 1 -- 172.21.15.139:0/2441521775 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c1519f0 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f1bfff640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f1c15ba70 0x7f0f1c0ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f1affd640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f1c0cf7b0 0x7f0f1c0cdf30 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000c780 tx=0x7f0f1000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f03fff640 1 -- 172.21.15.139:0/2441521775 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10018070 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f03fff640 1 -- 172.21.15.139:0/2441521775 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f100072f0 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f03fff640 1 -- 172.21.15.139:0/2441521775 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f100047e0 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f21ecb640 1 -- 172.21.15.139:0/2441521775 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1c151cd0 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.212+0000 7f0f21ecb640 1 -- 172.21.15.139:0/2441521775 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1c152290 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.213+0000 7f0f21ecb640 1 -- 172.21.15.139:0/2441521775 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1c152670 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.213+0000 7f0f03fff640 1 -- 172.21.15.139:0/2441521775 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f10004980 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.214+0000 7f0f03fff640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0f0806b4d0 0x7f0f0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.214+0000 7f0f1b7fe640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0f0806b4d0 0x7f0f0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.214+0000 7f0f03fff640 1 -- 172.21.15.139:0/2441521775 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f10089870 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.214+0000 7f0f1b7fe640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0f0806b4d0 0x7f0f0806d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c034000 tx=0x7f0f0c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.218+0000 7f0f03fff640 1 -- 172.21.15.139:0/2441521775 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f1c152670 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.427+0000 7f0f21ecb640 1 -- 172.21.15.139:0/2441521775 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f1c074520 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.427+0000 7f0f03fff640 1 -- 172.21.15.139:0/2441521775 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f0f10058560 con 0x7f0f1c0cf7b0 2024-04-25T09:59:10.431 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:10.431 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7545,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1239243954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1239243954},{"type":"v1","addr":"172.21.15.139:6836","nonce":1239243954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T09:58:59.365616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7557":{"gid":7557,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2266713846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2266713846},{"type":"v1","addr":"172.21.15.150:6835","nonce":2266713846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:10.431 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 50 2024-04-25T09:59:10.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.429+0000 7f0f21ecb640 1 -- 172.21.15.139:0/2441521775 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0f0806b4d0 msgr2=0x7f0f0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.429+0000 7f0f21ecb640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0f0806b4d0 0x7f0f0806d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c034000 tx=0x7f0f0c02e000 comp rx=0 tx=0).stop 2024-04-25T09:59:10.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.429+0000 7f0f21ecb640 1 -- 172.21.15.139:0/2441521775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f1c0cf7b0 msgr2=0x7f0f1c0cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.429+0000 7f0f21ecb640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f1c0cf7b0 0x7f0f1c0cdf30 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000c780 tx=0x7f0f1000cc40 comp rx=0 tx=0).stop 2024-04-25T09:59:10.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.429+0000 7f0f21ecb640 1 -- 172.21.15.139:0/2441521775 shutdown_connections 2024-04-25T09:59:10.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.429+0000 7f0f21ecb640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f1c0cf7b0 0x7f0f1c0cdf30 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.429+0000 7f0f21ecb640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0f0806b4d0 0x7f0f0806d990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.429+0000 7f0f21ecb640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f1c15ba70 0x7f0f1c0ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.429+0000 7f0f21ecb640 1 --2- 172.21.15.139:0/2441521775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f1c0cbd20 0x7f0f1c146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.429+0000 7f0f21ecb640 1 -- 172.21.15.139:0/2441521775 >> 172.21.15.139:0/2441521775 conn(0x7f0f1c0b3380 msgr2=0x7f0f1c156690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:10.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.429+0000 7f0f21ecb640 1 -- 172.21.15.139:0/2441521775 shutdown_connections 2024-04-25T09:59:10.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.429+0000 7f0f21ecb640 1 -- 172.21.15.139:0/2441521775 wait complete. 2024-04-25T09:59:10.444 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T09:59:10.581 INFO:teuthology.orchestra.run.smithi150.stdout:{"message":"","return_code":0} 2024-04-25T09:59:10.581 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:59:10.582 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:59:10.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.676+0000 7f32cef34640 1 Processor -- start 2024-04-25T09:59:10.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.676+0000 7f32cef34640 1 -- start start 2024-04-25T09:59:10.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.676+0000 7f32cef34640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f32c81617e0 0x7f32c80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.676+0000 7f32cef34640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c80cec80 0x7f32c80d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.677+0000 7f32cef34640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c8161dc0 con 0x7f32c815dbf0 2024-04-25T09:59:10.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.677+0000 7f32cef34640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c80d1ee0 con 0x7f32c81617e0 2024-04-25T09:59:10.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.677+0000 7f32cef34640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81620a0 con 0x7f32c80cec80 2024-04-25T09:59:10.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.677+0000 7f32c7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f32c81617e0 0x7f32c80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.677+0000 7f32c7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f32c81617e0 0x7f32c80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49070/0 (socket says 172.21.15.139:49070) 2024-04-25T09:59:10.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.677+0000 7f32c7fff640 1 -- 172.21.15.139:0/1326205328 learned_addr learned my addr 172.21.15.139:0/1326205328 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:10.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.677+0000 7f32c7fff640 1 -- 172.21.15.139:0/1326205328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c80cec80 msgr2=0x7f32c80d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:10.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.677+0000 7f32c7fff640 1 --2- 172.21.15.139:0/1326205328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c80cec80 0x7f32c80d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.677+0000 7f32c7fff640 1 -- 172.21.15.139:0/1326205328 >> v1:172.21.15.150:6789/0 conn(0x7f32c815dbf0 legacy=0x7f32c80b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:10.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.677+0000 7f32c7fff640 1 -- 172.21.15.139:0/1326205328 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c8162380 con 0x7f32c81617e0 2024-04-25T09:59:10.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.678+0000 7f32c7fff640 1 --2- 172.21.15.139:0/1326205328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c81617e0 0x7f32c80ce740 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009870 tx=0x7f32b402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4505534e2d2c4bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.678+0000 7f32c77fe640 1 -- 172.21.15.139:0/1326205328 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4037070 con 0x7f32c81617e0 2024-04-25T09:59:10.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.678+0000 7f32c77fe640 1 -- 172.21.15.139:0/1326205328 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b4032440 con 0x7f32c81617e0 2024-04-25T09:59:10.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.678+0000 7f32c77fe640 1 -- 172.21.15.139:0/1326205328 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4032810 con 0x7f32c81617e0 2024-04-25T09:59:10.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.678+0000 7f32cef34640 1 -- 172.21.15.139:0/1326205328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c81617e0 msgr2=0x7f32c80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.678+0000 7f32cef34640 1 --2- 172.21.15.139:0/1326205328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c81617e0 0x7f32c80ce740 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009870 tx=0x7f32b402a180 comp rx=0 tx=0).stop 2024-04-25T09:59:10.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.678+0000 7f32cef34640 1 -- 172.21.15.139:0/1326205328 shutdown_connections 2024-04-25T09:59:10.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.678+0000 7f32cef34640 1 --2- 172.21.15.139:0/1326205328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c80cec80 0x7f32c80d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.678+0000 7f32cef34640 1 --2- 172.21.15.139:0/1326205328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c81617e0 0x7f32c80ce740 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.678+0000 7f32cef34640 1 -- 172.21.15.139:0/1326205328 >> 172.21.15.139:0/1326205328 conn(0x7f32c80b5480 msgr2=0x7f32c80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:10.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.678+0000 7f32cef34640 1 -- 172.21.15.139:0/1326205328 shutdown_connections 2024-04-25T09:59:10.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.678+0000 7f32cef34640 1 -- 172.21.15.139:0/1326205328 wait complete. 2024-04-25T09:59:10.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.679+0000 7f32cef34640 1 Processor -- start 2024-04-25T09:59:10.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.679+0000 7f32cef34640 1 -- start start 2024-04-25T09:59:10.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.680+0000 7f32cef34640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c80cec80 0x7f32c81e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.680+0000 7f32cef34640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c815dbf0 0x7f32c815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.680+0000 7f32cef34640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c81617e0 0x7f32c80cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.680+0000 7f32cef34640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c815dfd0 con 0x7f32c815dbf0 2024-04-25T09:59:10.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.680+0000 7f32cef34640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80d28b0 con 0x7f32c81617e0 2024-04-25T09:59:10.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.680+0000 7f32cef34640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c815c6f0 con 0x7f32c80cec80 2024-04-25T09:59:10.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.680+0000 7f32c7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c815dbf0 0x7f32c815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.680+0000 7f32c7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c815dbf0 0x7f32c815c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49076/0 (socket says 172.21.15.139:49076) 2024-04-25T09:59:10.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.680+0000 7f32c7fff640 1 -- 172.21.15.139:0/3969733426 learned_addr learned my addr 172.21.15.139:0/3969733426 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:10.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32cccd0640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c80cec80 0x7f32c81e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32c7fff640 1 -- 172.21.15.139:0/3969733426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c80cec80 msgr2=0x7f32c81e82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32cd4d1640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c81617e0 0x7f32c80cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32c7fff640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c80cec80 0x7f32c81e82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32c7fff640 1 -- 172.21.15.139:0/3969733426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c81617e0 msgr2=0x7f32c80cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32c7fff640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c81617e0 0x7f32c80cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32c7fff640 1 -- 172.21.15.139:0/3969733426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c815b5a0 con 0x7f32c815dbf0 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32cd4d1640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c81617e0 0x7f32c80cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32c7fff640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c815dbf0 0x7f32c815c1b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f32b40090b0 tx=0x7f32b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32c57fa640 1 -- 172.21.15.139:0/3969733426 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4041070 con 0x7f32c815dbf0 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32c57fa640 1 -- 172.21.15.139:0/3969733426 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b403c890 con 0x7f32c815dbf0 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32cef34640 1 -- 172.21.15.139:0/3969733426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c815b880 con 0x7f32c815dbf0 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32c57fa640 1 -- 172.21.15.139:0/3969733426 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4037040 con 0x7f32c815dbf0 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.681+0000 7f32cef34640 1 -- 172.21.15.139:0/3969733426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c81e7850 con 0x7f32c815dbf0 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.682+0000 7f32cef34640 1 -- 172.21.15.139:0/3969733426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c80cd560 con 0x7f32c815dbf0 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.682+0000 7f32c57fa640 1 -- 172.21.15.139:0/3969733426 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32b4044070 con 0x7f32c815dbf0 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.683+0000 7f32c57fa640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f32ac06b4d0 0x7f32ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.683+0000 7f32c57fa640 1 -- 172.21.15.139:0/3969733426 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32b40ab0f0 con 0x7f32c815dbf0 2024-04-25T09:59:10.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.683+0000 7f32cccd0640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f32ac06b4d0 0x7f32ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.683+0000 7f32cccd0640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f32ac06b4d0 0x7f32ac06d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f32bc003d90 tx=0x7f32bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.687+0000 7f32c57fa640 1 -- 172.21.15.139:0/3969733426 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32b4079f90 con 0x7f32c815dbf0 2024-04-25T09:59:10.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.895+0000 7f32cef34640 1 -- 172.21.15.139:0/3969733426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f32c8074520 con 0x7f32c815dbf0 2024-04-25T09:59:10.900 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T09:59:10.896+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:59:10.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.897+0000 7f32c57fa640 1 -- 172.21.15.139:0/3969733426 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7f32b4079890 con 0x7f32c815dbf0 2024-04-25T09:59:10.901 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:59:10.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.900+0000 7f32cef34640 1 -- 172.21.15.139:0/3969733426 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f32ac06b4d0 msgr2=0x7f32ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.900+0000 7f32cef34640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f32ac06b4d0 0x7f32ac06d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f32bc003d90 tx=0x7f32bc009040 comp rx=0 tx=0).stop 2024-04-25T09:59:10.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.900+0000 7f32cef34640 1 -- 172.21.15.139:0/3969733426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c815dbf0 msgr2=0x7f32c815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.900+0000 7f32cef34640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c815dbf0 0x7f32c815c1b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f32b40090b0 tx=0x7f32b40046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.900+0000 7f32cef34640 1 -- 172.21.15.139:0/3969733426 shutdown_connections 2024-04-25T09:59:10.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.900+0000 7f32cef34640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c815dbf0 0x7f32c815c1b0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.900+0000 7f32cef34640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f32ac06b4d0 0x7f32ac06d990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.900+0000 7f32cef34640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c81617e0 0x7f32c80cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.900+0000 7f32cef34640 1 --2- 172.21.15.139:0/3969733426 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c80cec80 0x7f32c81e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.900+0000 7f32cef34640 1 -- 172.21.15.139:0/3969733426 >> 172.21.15.139:0/3969733426 conn(0x7f32c80b5480 msgr2=0x7f32c80b90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:10.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.900+0000 7f32cef34640 1 -- 172.21.15.139:0/3969733426 shutdown_connections 2024-04-25T09:59:10.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.900+0000 7f32cef34640 1 -- 172.21.15.139:0/3969733426 wait complete. 2024-04-25T09:59:10.906 INFO:tasks.ceph.mds.d.smithi150.stderr: -4535> 2024-04-25T09:59:03.292+0000 7f75cb771640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:59:10.915 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-25T09:59:10.935 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-04-25T09:59:10.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.952+0000 7f7c9eb84f40 1 Processor -- start 2024-04-25T09:59:10.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.952+0000 7f7c9eb84f40 1 -- start start 2024-04-25T09:59:10.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.952+0000 7f7c9eb84f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555dfb0df580 0x555dfb1acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.952+0000 7f7c9eb84f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb124430 0x555dfb1b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.952+0000 7f7c9eb84f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555dfb1ae530 0x555dfb0ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.952+0000 7f7c9eb84f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfb124b40 con 0x555dfb124430 2024-04-25T09:59:10.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.952+0000 7f7c9eb84f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfb083df0 con 0x555dfb0df580 2024-04-25T09:59:10.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.952+0000 7f7c9eb84f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555dfb124de0 con 0x555dfb1ae530 2024-04-25T09:59:10.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.952+0000 7f7c9be84640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb124430 0x555dfb1b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.952+0000 7f7c9be84640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb124430 0x555dfb1b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49094/0 (socket says 172.21.15.139:49094) 2024-04-25T09:59:10.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9be84640 1 -- 172.21.15.139:0/3119876234 learned_addr learned my addr 172.21.15.139:0/3119876234 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:10.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9c685640 1 --2- 172.21.15.139:0/3119876234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555dfb0df580 0x555dfb1acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9ce86640 1 --2- 172.21.15.139:0/3119876234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555dfb1ae530 0x555dfb0ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9be84640 1 -- 172.21.15.139:0/3119876234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555dfb1ae530 msgr2=0x555dfb0ddfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9be84640 1 --2- 172.21.15.139:0/3119876234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555dfb1ae530 0x555dfb0ddfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9be84640 1 -- 172.21.15.139:0/3119876234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555dfb0df580 msgr2=0x555dfb1acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9be84640 1 --2- 172.21.15.139:0/3119876234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555dfb0df580 0x555dfb1acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9be84640 1 -- 172.21.15.139:0/3119876234 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dfb009c50 con 0x555dfb124430 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9c685640 1 --2- 172.21.15.139:0/3119876234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555dfb0df580 0x555dfb1acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9be84640 1 --2- 172.21.15.139:0/3119876234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb124430 0x555dfb1b3530 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000a770 tx=0x7f7c9000a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8baf67e522e09205 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9ae82640 1 -- 172.21.15.139:0/3119876234 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90019070 con 0x555dfb124430 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9ae82640 1 -- 172.21.15.139:0/3119876234 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c90007600 con 0x555dfb124430 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9ae82640 1 -- 172.21.15.139:0/3119876234 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c900079d0 con 0x555dfb124430 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3119876234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb124430 msgr2=0x555dfb1b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.953+0000 7f7c9eb84f40 1 --2- 172.21.15.139:0/3119876234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb124430 0x555dfb1b3530 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000a770 tx=0x7f7c9000a860 comp rx=0 tx=0).stop 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.954+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3119876234 shutdown_connections 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.954+0000 7f7c9eb84f40 1 --2- 172.21.15.139:0/3119876234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb124430 0x555dfb1b3530 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.954+0000 7f7c9eb84f40 1 --2- 172.21.15.139:0/3119876234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555dfb1ae530 0x555dfb0ddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.954+0000 7f7c9eb84f40 1 --2- 172.21.15.139:0/3119876234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555dfb0df580 0x555dfb1acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.954+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3119876234 >> 172.21.15.139:0/3119876234 conn(0x555dfb0dcbe0 msgr2=0x555dfb10b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.954+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3119876234 shutdown_connections 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.954+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3119876234 wait complete. 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.954+0000 7f7c9eb84f40 1 Processor -- start 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.954+0000 7f7c9eb84f40 1 -- start start 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9eb84f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb0df580 0x555dfb1b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9eb84f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555dfb124430 0x555dfb11f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9eb84f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555dfb1ae530 0x555dfb10a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9eb84f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfb1b0750 con 0x555dfb0df580 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9eb84f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfb0df170 con 0x555dfb124430 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9eb84f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555dfb10c260 con 0x555dfb1ae530 2024-04-25T09:59:10.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9c685640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb0df580 0x555dfb1b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9c685640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb0df580 0x555dfb1b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49106/0 (socket says 172.21.15.139:49106) 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9c685640 1 -- 172.21.15.139:0/3859717657 learned_addr learned my addr 172.21.15.139:0/3859717657 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9be84640 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555dfb124430 0x555dfb11f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9ce86640 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555dfb1ae530 0x555dfb10a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9c685640 1 -- 172.21.15.139:0/3859717657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555dfb1ae530 msgr2=0x555dfb10a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9c685640 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555dfb1ae530 0x555dfb10a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9c685640 1 -- 172.21.15.139:0/3859717657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555dfb124430 msgr2=0x555dfb11f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9c685640 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555dfb124430 0x555dfb11f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9c685640 1 -- 172.21.15.139:0/3859717657 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dfb0de980 con 0x555dfb0df580 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.955+0000 7f7c9c685640 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb0df580 0x555dfb1b37b0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c006fb0 tx=0x7f7c8c00d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.956+0000 7f7c99e80640 1 -- 172.21.15.139:0/3859717657 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c018070 con 0x555dfb0df580 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.956+0000 7f7c99e80640 1 -- 172.21.15.139:0/3859717657 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8c0060d0 con 0x555dfb0df580 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.956+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3859717657 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555dfb0dec60 con 0x555dfb0df580 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.956+0000 7f7c99e80640 1 -- 172.21.15.139:0/3859717657 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c004520 con 0x555dfb0df580 2024-04-25T09:59:10.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.956+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3859717657 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555dfb1b3cf0 con 0x555dfb0df580 2024-04-25T09:59:10.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.957+0000 7f7c99e80640 1 -- 172.21.15.139:0/3859717657 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c8c01b070 con 0x555dfb0df580 2024-04-25T09:59:10.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.957+0000 7f7c99e80640 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7c88069eb0 0x7f7c8806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.957+0000 7f7c99e80640 1 -- 172.21.15.139:0/3859717657 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7c8c0887f0 con 0x555dfb0df580 2024-04-25T09:59:10.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.957+0000 7f7c9be84640 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7c88069eb0 0x7f7c8806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.958+0000 7f7c9eb84f40 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555dfb10ab80 0x555dfb23e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:10.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.958+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3859717657 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e90) v9 -- 0x555dfb1aa6b0 con 0x555dfb10ab80 2024-04-25T09:59:10.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.958+0000 7f7c9ce86640 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555dfb10ab80 0x555dfb23e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:10.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.958+0000 7f7c9be84640 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7c88069eb0 0x7f7c8806c370 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000afd0 tx=0x7f7c9000c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.958+0000 7f7c9ce86640 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555dfb10ab80 0x555dfb23e840 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:10.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.961+0000 7f7c9ce86640 1 -- 172.21.15.139:0/3859717657 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v90'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7c9405c040 con 0x555dfb10ab80 2024-04-25T09:59:10.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.961+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555dfb10ab80 msgr2=0x555dfb23e840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.961+0000 7f7c9eb84f40 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555dfb10ab80 0x555dfb23e840 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.961+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7c88069eb0 msgr2=0x7f7c8806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.961+0000 7f7c9eb84f40 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7c88069eb0 0x7f7c8806c370 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000afd0 tx=0x7f7c9000c000 comp rx=0 tx=0).stop 2024-04-25T09:59:10.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.961+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb0df580 msgr2=0x555dfb1b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:10.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.961+0000 7f7c9eb84f40 1 --2- 172.21.15.139:0/3859717657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555dfb0df580 0x555dfb1b37b0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c006fb0 tx=0x7f7c8c00d5f0 comp rx=0 tx=0).stop 2024-04-25T09:59:10.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.961+0000 7f7c9ce86640 1 -- 172.21.15.139:0/3859717657 reap_dead start 2024-04-25T09:59:10.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.962+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3859717657 shutdown_connections 2024-04-25T09:59:10.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.962+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3859717657 >> 172.21.15.139:0/3859717657 conn(0x555dfb0dcbe0 msgr2=0x555dfb1b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:10.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.962+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3859717657 shutdown_connections 2024-04-25T09:59:10.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:10.962+0000 7f7c9eb84f40 1 -- 172.21.15.139:0/3859717657 wait complete. 2024-04-25T09:59:10.969 DEBUG:teuthology.orchestra.run.smithi139:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-25T09:59:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.044+0000 7f9213a1a280 1 -- start start 2024-04-25T09:59:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.044+0000 7f9213a1a280 1 Processor -- start 2024-04-25T09:59:11.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f9213a1a280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4c980c00 0x555a4c9f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f9213a1a280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4c981800 0x555a4c9f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f9213a1a280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4c981c00 0x555a4c9f1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f9213a1a280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4d762780 con 0x555a4c980c00 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f9213a1a280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4d762600 con 0x555a4c981800 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f9213a1a280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a4d762480 con 0x555a4c981c00 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f92114fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4c980c00 0x555a4c9f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f92114fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4c980c00 0x555a4c9f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49118/0 (socket says 172.21.15.139:49118) 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f92114fc640 1 -- 172.21.15.139:0/2777572313 learned_addr learned my addr 172.21.15.139:0/2777572313 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f9210cfb640 1 --2- 172.21.15.139:0/2777572313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4c981800 0x555a4c9f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f9211cfd640 1 --2- 172.21.15.139:0/2777572313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4c981c00 0x555a4c9f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f92114fc640 1 -- 172.21.15.139:0/2777572313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4c981c00 msgr2=0x555a4c9f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f92114fc640 1 --2- 172.21.15.139:0/2777572313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4c981c00 0x555a4c9f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f92114fc640 1 -- 172.21.15.139:0/2777572313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4c981800 msgr2=0x555a4c9f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f92114fc640 1 --2- 172.21.15.139:0/2777572313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4c981800 0x555a4c9f0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f92114fc640 1 -- 172.21.15.139:0/2777572313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a4c9b6780 con 0x555a4c980c00 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.045+0000 7f9211cfd640 1 --2- 172.21.15.139:0/2777572313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4c981c00 0x555a4c9f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.046+0000 7f92114fc640 1 --2- 172.21.15.139:0/2777572313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4c980c00 0x555a4c9f0580 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x555a4d7a1380 tx=0x555a4d74a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6f5c2e485d96d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.046+0000 7f920ecf7640 1 -- 172.21.15.139:0/2777572313 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a4c9a6680 con 0x555a4c980c00 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.046+0000 7f920ecf7640 1 -- 172.21.15.139:0/2777572313 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555a4c9ac700 con 0x555a4c980c00 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.046+0000 7f9213a1a280 1 -- 172.21.15.139:0/2777572313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a4c9b63c0 con 0x555a4c980c00 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.046+0000 7f920ecf7640 1 -- 172.21.15.139:0/2777572313 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a4c9a6b60 con 0x555a4c980c00 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.046+0000 7f9213a1a280 1 -- 172.21.15.139:0/2777572313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555a4c9b7680 con 0x555a4c980c00 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.046+0000 7f9213a1a280 4 waiting for MDS map... 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.046+0000 7f920ecf7640 1 -- 172.21.15.139:0/2777572313 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x555a4d850600 con 0x555a4c980c00 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.046+0000 7f920ecf7640 1 -- 172.21.15.139:0/2777572313 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 51) v1 ==== 2723+0+0 (secure 0 0 0) 0x555a4c9db180 con 0x555a4c980c00 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.046+0000 7f9213a1a280 4 Got MDS map 51 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.046+0000 7f9213a1a280 10 main: JournalTool::main 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.046+0000 7f9213a1a280 4 main: JournalTool: connecting to RADOS... 2024-04-25T09:59:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.047+0000 7f9213a1a280 1 Processor -- start 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.047+0000 7f9213a1a280 1 -- start start 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.047+0000 7f9213a1a280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4d777000 0x555a4c9f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.047+0000 7f9213a1a280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 0x555a4c9f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.047+0000 7f9213a1a280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4d777800 0x555a4c9f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.047+0000 7f9213a1a280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4d762300 con 0x555a4d777400 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.047+0000 7f9213a1a280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4d762000 con 0x555a4d777000 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.047+0000 7f9213a1a280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a4c9e2900 con 0x555a4d777800 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.047+0000 7f92114fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 0x555a4c9f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.047+0000 7f92114fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 0x555a4c9f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49130/0 (socket says 172.21.15.139:49130) 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.047+0000 7f92114fc640 1 -- 172.21.15.139:0/2041691613 learned_addr learned my addr 172.21.15.139:0/2041691613 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.047+0000 7f9211cfd640 1 --2- 172.21.15.139:0/2041691613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4d777000 0x555a4c9f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f9210cfb640 1 --2- 172.21.15.139:0/2041691613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4d777800 0x555a4c9f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f92114fc640 1 -- 172.21.15.139:0/2041691613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4d777800 msgr2=0x555a4c9f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f92114fc640 1 --2- 172.21.15.139:0/2041691613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4d777800 0x555a4c9f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f92114fc640 1 -- 172.21.15.139:0/2041691613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4d777000 msgr2=0x555a4c9f1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f92114fc640 1 --2- 172.21.15.139:0/2041691613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4d777000 0x555a4c9f1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f92114fc640 1 -- 172.21.15.139:0/2041691613 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a4c9b74a0 con 0x555a4d777400 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f92114fc640 1 --2- 172.21.15.139:0/2041691613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 0x555a4c9f1b80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x555a4d848630 tx=0x555a4d705020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e194e4fde63ba34d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f920ccf3640 1 -- 172.21.15.139:0/2041691613 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a4c9a6820 con 0x555a4d777400 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f920ccf3640 1 -- 172.21.15.139:0/2041691613 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555a4c9acfc0 con 0x555a4d777400 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f920ccf3640 1 -- 172.21.15.139:0/2041691613 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a4c9a6680 con 0x555a4d777400 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f9213a1a280 1 -- 172.21.15.139:0/2041691613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 msgr2=0x555a4c9f1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2041691613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 0x555a4c9f1b80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x555a4d848630 tx=0x555a4d705020 comp rx=0 tx=0).stop 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f9213a1a280 1 -- 172.21.15.139:0/2041691613 shutdown_connections 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2041691613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 0x555a4c9f1b80 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2041691613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4d777800 0x555a4c9f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2041691613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4d777000 0x555a4c9f1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f9213a1a280 1 -- 172.21.15.139:0/2041691613 >> 172.21.15.139:0/2041691613 conn(0x555a4d776c00 msgr2=0x555a4ca7b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f9213a1a280 1 -- 172.21.15.139:0/2041691613 shutdown_connections 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.048+0000 7f9213a1a280 1 -- 172.21.15.139:0/2041691613 wait complete. 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.049+0000 7f9213a1a280 1 Processor -- start 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.049+0000 7f9213a1a280 1 -- start start 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.049+0000 7f9213a1a280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4d777000 0x555a4c9f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.049+0000 7f9213a1a280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 0x555a4c9f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.049+0000 7f9213a1a280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4d777800 0x555a4c9f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.049+0000 7f9213a1a280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4d763380 con 0x555a4d777400 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.049+0000 7f9213a1a280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4d763500 con 0x555a4d777000 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.049+0000 7f9213a1a280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a4d762000 con 0x555a4d777800 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f92114fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 0x555a4c9f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f92114fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 0x555a4c9f2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49138/0 (socket says 172.21.15.139:49138) 2024-04-25T09:59:11.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f9211cfd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4d777000 0x555a4c9f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f92114fc640 1 -- 172.21.15.139:0/1659658017 learned_addr learned my addr 172.21.15.139:0/1659658017 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f9210cfb640 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4d777800 0x555a4c9f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f9211cfd640 1 -- 172.21.15.139:0/1659658017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4d777800 msgr2=0x555a4c9f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f9211cfd640 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4d777800 0x555a4c9f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f9211cfd640 1 -- 172.21.15.139:0/1659658017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 msgr2=0x555a4c9f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f9211cfd640 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 0x555a4c9f2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f9211cfd640 1 -- 172.21.15.139:0/1659658017 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a4c9b6780 con 0x555a4d777000 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f92114fc640 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 0x555a4c9f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f9211cfd640 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4d777000 0x555a4c9f3180 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x555a4d849320 tx=0x555a4d860e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f920b4f0640 1 -- 172.21.15.139:0/1659658017 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a4c9a6820 con 0x555a4d777000 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f920b4f0640 1 -- 172.21.15.139:0/1659658017 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555a4c9acfc0 con 0x555a4d777000 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f920b4f0640 1 -- 172.21.15.139:0/1659658017 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a4c9a6680 con 0x555a4d777000 2024-04-25T09:59:11.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f9213a1a280 1 -- 172.21.15.139:0/1659658017 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555a4c9b63c0 con 0x555a4d777000 2024-04-25T09:59:11.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f9213a1a280 1 -- 172.21.15.139:0/1659658017 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a4c9b7680 con 0x555a4d777000 2024-04-25T09:59:11.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.050+0000 7f9213a1a280 4 main: JournalTool: resolving pool 13 2024-04-25T09:59:11.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.052+0000 7f920b4f0640 1 -- 172.21.15.139:0/1659658017 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x555a4d8b6c00 con 0x555a4d777000 2024-04-25T09:59:11.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.052+0000 7f920b4f0640 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x555a4d8cc800 0x555a4c9f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.052+0000 7f920b4f0640 1 -- 172.21.15.139:0/1659658017 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x555a4d8caa00 con 0x555a4d777000 2024-04-25T09:59:11.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.052+0000 7f92114fc640 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x555a4d8cc800 0x555a4c9f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.052+0000 7f9213a1a280 4 main: JournalTool: creating IoCtx.. 2024-04-25T09:59:11.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.052+0000 7f9213a1a280 4 main: Executing for rank 0 2024-04-25T09:59:11.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.052+0000 7f92114fc640 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x555a4d8cc800 0x555a4c9f3700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x555a4d848840 tx=0x555a4d860c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.052+0000 7f9213a1a280 1 -- start start 2024-04-25T09:59:11.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.052+0000 7f9213a1a280 1 Processor -- start 2024-04-25T09:59:11.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9213a1a280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4cd48000 0x555a4c9f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9213a1a280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4cd48400 0x555a4c9f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9213a1a280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4cd48800 0x555a4c9f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9213a1a280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4d763680 con 0x555a4cd48800 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9213a1a280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4d763800 con 0x555a4cd48000 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9213a1a280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a4d763980 con 0x555a4cd48400 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9210cfb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4cd48800 0x555a4c9f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9210cfb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4cd48800 0x555a4c9f1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49142/0 (socket says 172.21.15.139:49142) 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9210cfb640 1 -- 172.21.15.139:0/2389723492 learned_addr learned my addr 172.21.15.139:0/2389723492 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9210cfb640 1 -- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4cd48400 msgr2=0x555a4c9f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9210cfb640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4cd48400 0x555a4c9f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9211cfd640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4cd48000 0x555a4c9f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9210cfb640 1 -- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4cd48000 msgr2=0x555a4c9f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:11.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9210cfb640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4cd48000 0x555a4c9f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9210cfb640 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a4c9b6000 con 0x555a4cd48800 2024-04-25T09:59:11.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.053+0000 7f9211cfd640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4cd48000 0x555a4c9f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:11.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f9210cfb640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4cd48800 0x555a4c9f1600 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x555a4ccad770 tx=0x555a4d712ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6defd7cd86b38e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f92084ea640 1 -- 172.21.15.139:0/2389723492 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a4c9a69c0 con 0x555a4cd48800 2024-04-25T09:59:11.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f92084ea640 1 -- 172.21.15.139:0/2389723492 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555a4c9ace00 con 0x555a4cd48800 2024-04-25T09:59:11.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f92084ea640 1 -- 172.21.15.139:0/2389723492 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a4d8ffa00 con 0x555a4cd48800 2024-04-25T09:59:11.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a4c9b6780 con 0x555a4cd48800 2024-04-25T09:59:11.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555a4c9b63c0 con 0x555a4cd48800 2024-04-25T09:59:11.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f9213a1a280 4 waiting for MDS map... 2024-04-25T09:59:11.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f92084ea640 1 -- 172.21.15.139:0/2389723492 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x555a4cd52400 con 0x555a4cd48800 2024-04-25T09:59:11.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f92084ea640 1 -- 172.21.15.139:0/2389723492 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 51) v1 ==== 2723+0+0 (secure 0 0 0) 0x555a4cd5a000 con 0x555a4cd48800 2024-04-25T09:59:11.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f9213a1a280 4 Got MDS map 51 2024-04-25T09:59:11.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x555a4cd48c00 0x555a4cd6a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) v9 -- 0x555a4cd49000 con 0x555a4cd48c00 2024-04-25T09:59:11.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.054+0000 7f9211cfd640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x555a4cd48c00 0x555a4cd6a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.055+0000 7f9211cfd640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x555a4cd48c00 0x555a4cd6a000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=ed9038f9f8dc2c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.057+0000 7f9211cfd640 1 -- 172.21.15.139:0/2389723492 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x555a4d700000 con 0x555a4cd48c00 2024-04-25T09:59:11.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.057+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x555a4cd49400 0x555a4cd6a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.057+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) v9 -- 0x555a4cd49c00 con 0x555a4cd49400 2024-04-25T09:59:11.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.057+0000 7f92114fc640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x555a4cd49400 0x555a4cd6a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.057+0000 7f92114fc640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x555a4cd49400 0x555a4cd6a580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=6ace1efa5970d9b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.059+0000 7f92114fc640 1 -- 172.21.15.139:0/2389723492 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv21 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x555a4d700500 con 0x555a4cd49400 2024-04-25T09:59:11.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.059+0000 7f9206ce7640 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90) v9 -- 0x555a4cd78000 con 0x555a4cd49400 2024-04-25T09:59:11.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.059+0000 7f9206ce7640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x555a4cd78400 0x555a4cd6c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.059+0000 7f9206ce7640 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90) v9 -- 0x555a4cd78800 con 0x555a4cd78400 2024-04-25T09:59:11.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.059+0000 7f92114fc640 1 -- 172.21.15.139:0/2389723492 <== osd.0 v2:172.21.15.139:6808/1167038101 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv18 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x555a4d700500 con 0x555a4cd49400 2024-04-25T09:59:11.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.059+0000 7f9211cfd640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x555a4cd78400 0x555a4cd6c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.060+0000 7f9211cfd640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x555a4cd78400 0x555a4cd6c680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=47b18a50131a3b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.060+0000 7f9211cfd640 1 -- 172.21.15.139:0/2389723492 <== osd.4 v2:172.21.15.150:6800/2810531857 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555a4d700000 con 0x555a4cd78400 2024-04-25T09:59:11.072 INFO:teuthology.orchestra.run.smithi139.stdout:old journal was 4212724~2605 2024-04-25T09:59:11.072 INFO:teuthology.orchestra.run.smithi139.stdout:new journal start will be 8388608 (4173279 bytes past old end) 2024-04-25T09:59:11.072 INFO:teuthology.orchestra.run.smithi139.stdout:writing journal head 2024-04-25T09:59:11.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.060+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555a4cd78c00 con 0x555a4cd49400 2024-04-25T09:59:11.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f92114fc640 1 -- 172.21.15.139:0/2389723492 <== osd.0 v2:172.21.15.139:6808/1167038101 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v90'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x555a4d700500 con 0x555a4cd49400 2024-04-25T09:59:11.073 INFO:teuthology.orchestra.run.smithi139.stdout:writing EResetJournal entry 2024-04-25T09:59:11.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555a4cd79000 con 0x555a4cd78400 2024-04-25T09:59:11.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x555a4cd79400 0x555a4cd6ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555a4cd79800 con 0x555a4cd79400 2024-04-25T09:59:11.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555a4cd79c00 con 0x555a4cd48c00 2024-04-25T09:59:11.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555a4cd78000 0x555a4cd6b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555a4cd78c00 con 0x555a4cd78000 2024-04-25T09:59:11.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555a4cd49c00 con 0x555a4cd78400 2024-04-25T09:59:11.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x555a4d8cd800 0x555a4cd6b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555a4d8cd400 con 0x555a4d8cd800 2024-04-25T09:59:11.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f9210cfb640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555a4cd78000 0x555a4cd6b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f92114fc640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x555a4cd79400 0x555a4cd6ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f9211cfd640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x555a4d8cd800 0x555a4cd6b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.064+0000 7f9210cfb640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555a4cd78000 0x555a4cd6b080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=cfef3162816d13c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.065+0000 7f9211cfd640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x555a4d8cd800 0x555a4cd6b600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=25375fe298134872 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.065+0000 7f92114fc640 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x555a4cd79400 0x555a4cd6ab00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=1d191a9cc1e067e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.065+0000 7f9211cfd640 1 -- 172.21.15.139:0/2389723492 <== osd.4 v2:172.21.15.150:6800/2810531857 2 ==== osd_op_reply(6 200.00000002 [delete] v90'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555a4d700000 con 0x555a4cd78400 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.066+0000 7f9211cfd640 1 -- 172.21.15.139:0/2389723492 <== osd.4 v2:172.21.15.150:6800/2810531857 3 ==== osd_op_reply(10 200.00000006 [delete] v90'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555a4d700000 con 0x555a4cd78400 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.066+0000 7f9210cfb640 1 -- 172.21.15.139:0/2389723492 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(9 200.00000005 [delete] v90'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555a4d700780 con 0x555a4cd78000 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.066+0000 7f9211cfd640 1 -- 172.21.15.139:0/2389723492 <== osd.6 v2:172.21.15.150:6809/3369628189 2 ==== osd_op_reply(8 200.00000004 [delete] v90'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555a4d700000 con 0x555a4cd48c00 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.068+0000 7f92114fc640 1 -- 172.21.15.139:0/2389723492 <== osd.5 v2:172.21.15.150:6824/2184499879 1 ==== osd_op_reply(7 200.00000003 [delete] v90'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555a4d700500 con 0x555a4cd79400 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.068+0000 7f9206ce7640 1 -- 172.21.15.139:0/2389723492 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555a4cd78800 con 0x555a4cd78400 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.068+0000 7f9211cfd640 1 -- 172.21.15.139:0/2389723492 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== osd_op_reply(11 200.00000007 [delete] v90'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555a4d700000 con 0x555a4d8cd800 2024-04-25T09:59:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9211cfd640 1 -- 172.21.15.139:0/2389723492 <== osd.4 v2:172.21.15.150:6800/2810531857 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v90'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x555a4d700000 con 0x555a4cd78400 2024-04-25T09:59:11.077 INFO:teuthology.orchestra.run.smithi139.stdout:done 2024-04-25T09:59:11.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x555a4cd49400 msgr2=0x555a4cd6a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x555a4cd49400 0x555a4cd6a580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555a4cd78000 msgr2=0x555a4cd6b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555a4cd78000 0x555a4cd6b080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x555a4cd78400 msgr2=0x555a4cd6c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x555a4cd78400 0x555a4cd6c680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x555a4cd79400 msgr2=0x555a4cd6ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x555a4cd79400 0x555a4cd6ab00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x555a4cd48c00 msgr2=0x555a4cd6a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x555a4cd48c00 0x555a4cd6a000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x555a4d8cd800 msgr2=0x555a4cd6b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x555a4d8cd800 0x555a4cd6b600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9210cfb640 1 -- 172.21.15.139:0/2389723492 reap_dead start 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9210cfb640 1 -- 172.21.15.139:0/2389723492 reap_dead start 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4cd48800 msgr2=0x555a4c9f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4cd48800 0x555a4c9f1600 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x555a4ccad770 tx=0x555a4d712ae0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 shutdown_connections 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2389723492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4cd48800 0x555a4c9f1600 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 >> 172.21.15.139:0/2389723492 conn(0x555a4d8cdc00 msgr2=0x555a4ca7b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.071+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 shutdown_connections 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 -- 172.21.15.139:0/2389723492 wait complete. 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 -- 172.21.15.139:0/1659658017 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x555a4d8cc800 msgr2=0x555a4c9f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x555a4d8cc800 0x555a4c9f3700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x555a4d848840 tx=0x555a4d860c00 comp rx=0 tx=0).stop 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 -- 172.21.15.139:0/1659658017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4d777000 msgr2=0x555a4c9f3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4d777000 0x555a4c9f3180 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x555a4d849320 tx=0x555a4d860e40 comp rx=0 tx=0).stop 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 -- 172.21.15.139:0/1659658017 shutdown_connections 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4d777400 0x555a4c9f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x555a4d8cc800 0x555a4c9f3700 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4d777800 0x555a4c9f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 --2- 172.21.15.139:0/1659658017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4d777000 0x555a4c9f3180 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 -- 172.21.15.139:0/1659658017 >> 172.21.15.139:0/1659658017 conn(0x555a4d776c00 msgr2=0x555a4ca7b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:11.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 -- 172.21.15.139:0/1659658017 shutdown_connections 2024-04-25T09:59:11.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 -- 172.21.15.139:0/1659658017 wait complete. 2024-04-25T09:59:11.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 -- 172.21.15.139:0/2777572313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4c980c00 msgr2=0x555a4c9f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2777572313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4c980c00 0x555a4c9f0580 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x555a4d7a1380 tx=0x555a4d74a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 -- 172.21.15.139:0/2777572313 shutdown_connections 2024-04-25T09:59:11.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2777572313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555a4c981c00 0x555a4c9f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2777572313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a4c981800 0x555a4c9f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 --2- 172.21.15.139:0/2777572313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555a4c980c00 0x555a4c9f0580 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 -- 172.21.15.139:0/2777572313 >> 172.21.15.139:0/2777572313 conn(0x555a4c981000 msgr2=0x555a4ca7a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:11.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 -- 172.21.15.139:0/2777572313 shutdown_connections 2024-04-25T09:59:11.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.072+0000 7f9213a1a280 1 -- 172.21.15.139:0/2777572313 wait complete. 2024-04-25T09:59:11.082 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.113633, result: old journal was 4212724~2605 new journal start will be 8388608 (4173279 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-25T09:59:11.083 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T09:59:11.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.215+0000 7f7330f30640 1 Processor -- start 2024-04-25T09:59:11.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.215+0000 7f7330f30640 1 -- start start 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.216+0000 7f7330f30640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f732c15dc90 0x7f732c0ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.216+0000 7f7330f30640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732c0cf980 con 0x7f732c06f300 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.216+0000 7f7330f30640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732c06ab60 con 0x7f732c0cf3e0 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.216+0000 7f7330f30640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732c06ae40 con 0x7f732c15dc90 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.216+0000 7f732ad9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f732c06f300 0x7f732c0b2db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42528/0 (socket says 172.21.15.139:42528) 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.216+0000 7f732ad9d640 1 -- 172.21.15.139:0/2996995788 learned_addr learned my addr 172.21.15.139:0/2996995788 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732a59c640 1 --2- 172.21.15.139:0/2996995788 >> v2:172.21.15.150:3300/0 conn(0x7f732c15dc90 0x7f732c0ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016983820 0 0) 0x7f732c0cf980 con 0x7f732c06f300 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7314003520 con 0x7f732c06f300 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2443834121 0 0) 0x7f7314003520 con 0x7f732c06f300 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 >> v2:172.21.15.150:3300/0 conn(0x7f732c15dc90 msgr2=0x7f732c0ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 --2- 172.21.15.139:0/2996995788 >> v2:172.21.15.150:3300/0 conn(0x7f732c15dc90 0x7f732c0ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 >> v1:172.21.15.150:6789/0 conn(0x7f732c0cf3e0 legacy=0x7f732c06ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c06b120 con 0x7f732c06f300 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f7324001670 con 0x7f732c06f300 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 >> v1:172.21.15.139:6789/0 conn(0x7f732c06f300 legacy=0x7f732c0b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 --2- 172.21.15.139:0/2996995788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73140039b0 0x7f7314003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 --2- 172.21.15.139:0/2996995788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7314004800 0x7f7314004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 --2- 172.21.15.139:0/2996995788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73140051c0 0x7f731400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c06ae40 con 0x7f7314004800 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731400c060 con 0x7f73140039b0 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731400c260 con 0x7f73140051c0 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f7329d9b640 1 --2- 172.21.15.139:0/2996995788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7314004800 0x7f7314004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732a59c640 1 --2- 172.21.15.139:0/2996995788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73140039b0 0x7f7314003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732ad9d640 1 --2- 172.21.15.139:0/2996995788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73140051c0 0x7f731400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f7329d9b640 1 -- 172.21.15.139:0/2996995788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73140051c0 msgr2=0x7f731400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f7329d9b640 1 --2- 172.21.15.139:0/2996995788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73140051c0 0x7f731400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f7329d9b640 1 -- 172.21.15.139:0/2996995788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73140039b0 msgr2=0x7f7314003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:11.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f7329d9b640 1 --2- 172.21.15.139:0/2996995788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73140039b0 0x7f7314003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f7329d9b640 1 -- 172.21.15.139:0/2996995788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7314003520 con 0x7f7314004800 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.217+0000 7f732ad9d640 1 -- 172.21.15.139:0/2996995788 reap_dead start 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.218+0000 7f7329d9b640 1 --2- 172.21.15.139:0/2996995788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7314004800 0x7f7314004c80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f7320001750 tx=0x7f7314005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a367705faa6a0ee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.218+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320001370 con 0x7f7314004800 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.218+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73200052d0 con 0x7f7314004800 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.218+0000 7f732959a640 1 -- 172.21.15.139:0/2996995788 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732002abc0 con 0x7f7314004800 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.219+0000 7f7330f30640 1 -- 172.21.15.139:0/2996995788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7314004800 msgr2=0x7f7314004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.219+0000 7f7330f30640 1 --2- 172.21.15.139:0/2996995788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7314004800 0x7f7314004c80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f7320001750 tx=0x7f7314005830 comp rx=0 tx=0).stop 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.219+0000 7f7330f30640 1 -- 172.21.15.139:0/2996995788 shutdown_connections 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.219+0000 7f7330f30640 1 --2- 172.21.15.139:0/2996995788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7314004800 0x7f7314004c80 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.219+0000 7f7330f30640 1 -- 172.21.15.139:0/2996995788 >> 172.21.15.139:0/2996995788 conn(0x7f732c0cfd80 msgr2=0x7f732c0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.219+0000 7f7330f30640 1 -- 172.21.15.139:0/2996995788 shutdown_connections 2024-04-25T09:59:11.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.219+0000 7f7330f30640 1 -- 172.21.15.139:0/2996995788 wait complete. 2024-04-25T09:59:11.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.220+0000 7f7330f30640 1 Processor -- start 2024-04-25T09:59:11.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.220+0000 7f7330f30640 1 -- start start 2024-04-25T09:59:11.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f7330f30640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7314004800 0x7f732c147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f7330f30640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f732c071a50 0x7f732c071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f7330f30640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c1486a0 0x7f732c148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f7330f30640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c06b920 con 0x7f732c1486a0 2024-04-25T09:59:11.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f7330f30640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c072620 con 0x7f732c071a50 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f7330f30640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0ceea0 con 0x7f7314004800 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f732ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c1486a0 0x7f732c148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f732ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c1486a0 0x7f732c148b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49160/0 (socket says 172.21.15.139:49160) 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f732ad9d640 1 -- 172.21.15.139:0/3119959333 learned_addr learned my addr 172.21.15.139:0/3119959333 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f732a59c640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7314004800 0x7f732c147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f7329d9b640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f732c071a50 0x7f732c071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f732ad9d640 1 -- 172.21.15.139:0/3119959333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7314004800 msgr2=0x7f732c147cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f732ad9d640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7314004800 0x7f732c147cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f732ad9d640 1 -- 172.21.15.139:0/3119959333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f732c071a50 msgr2=0x7f732c071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f732ad9d640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f732c071a50 0x7f732c071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f732ad9d640 1 -- 172.21.15.139:0/3119959333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c149440 con 0x7f732c1486a0 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f732ad9d640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c1486a0 0x7f732c148b20 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f732c0af990 tx=0x7f7324007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.221+0000 7f731affd640 1 -- 172.21.15.139:0/3119959333 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732401a070 con 0x7f732c1486a0 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.222+0000 7f731affd640 1 -- 172.21.15.139:0/3119959333 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732400f6a0 con 0x7f732c1486a0 2024-04-25T09:59:11.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.222+0000 7f7330f30640 1 -- 172.21.15.139:0/3119959333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c149720 con 0x7f732c1486a0 2024-04-25T09:59:11.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.222+0000 7f731affd640 1 -- 172.21.15.139:0/3119959333 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732400fa70 con 0x7f732c1486a0 2024-04-25T09:59:11.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.222+0000 7f7330f30640 1 -- 172.21.15.139:0/3119959333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c149ce0 con 0x7f732c1486a0 2024-04-25T09:59:11.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.223+0000 7f7330f30640 1 -- 172.21.15.139:0/3119959333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732c074520 con 0x7f732c1486a0 2024-04-25T09:59:11.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.223+0000 7f731affd640 1 -- 172.21.15.139:0/3119959333 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f732401d070 con 0x7f732c1486a0 2024-04-25T09:59:11.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.223+0000 7f731affd640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f730c06b4d0 0x7f730c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:11.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.223+0000 7f731affd640 1 -- 172.21.15.139:0/3119959333 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7324089720 con 0x7f732c1486a0 2024-04-25T09:59:11.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.224+0000 7f732a59c640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f730c06b4d0 0x7f730c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:11.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.224+0000 7f732a59c640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f730c06b4d0 0x7f730c06d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f731c014450 tx=0x7f731c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:11.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.228+0000 7f731affd640 1 -- 172.21.15.139:0/3119959333 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7324057d70 con 0x7f732c1486a0 2024-04-25T09:59:11.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.438+0000 7f7330f30640 1 -- 172.21.15.139:0/3119959333 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f732c0d03b0 con 0x7f732c1486a0 2024-04-25T09:59:11.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.902+0000 7f731affd640 1 -- 172.21.15.139:0/3119959333 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v52) v1 ==== 162+0+0 (secure 0 0 0) 0x7f732405be10 con 0x7f732c1486a0 2024-04-25T09:59:11.906 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T09:59:11.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.904+0000 7f7330f30640 1 -- 172.21.15.139:0/3119959333 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f730c06b4d0 msgr2=0x7f730c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.904+0000 7f7330f30640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f730c06b4d0 0x7f730c06d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f731c014450 tx=0x7f731c00b040 comp rx=0 tx=0).stop 2024-04-25T09:59:11.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.904+0000 7f7330f30640 1 -- 172.21.15.139:0/3119959333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c1486a0 msgr2=0x7f732c148b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:11.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.904+0000 7f7330f30640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c1486a0 0x7f732c148b20 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f732c0af990 tx=0x7f7324007430 comp rx=0 tx=0).stop 2024-04-25T09:59:11.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.904+0000 7f7330f30640 1 -- 172.21.15.139:0/3119959333 shutdown_connections 2024-04-25T09:59:11.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.905+0000 7f7330f30640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732c1486a0 0x7f732c148b20 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.905+0000 7f7330f30640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f730c06b4d0 0x7f730c06d990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.905+0000 7f7330f30640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f732c071a50 0x7f732c071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.905+0000 7f7330f30640 1 --2- 172.21.15.139:0/3119959333 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7314004800 0x7f732c147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:11.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.905+0000 7f7330f30640 1 -- 172.21.15.139:0/3119959333 >> 172.21.15.139:0/3119959333 conn(0x7f732c0cfd80 msgr2=0x7f732c0b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:11.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.905+0000 7f7330f30640 1 -- 172.21.15.139:0/3119959333 shutdown_connections 2024-04-25T09:59:11.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:11.905+0000 7f7330f30640 1 -- 172.21.15.139:0/3119959333 wait complete. 2024-04-25T09:59:11.921 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:12.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.016+0000 7fbc6885d640 1 Processor -- start 2024-04-25T09:59:12.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.016+0000 7fbc6885d640 1 -- start start 2024-04-25T09:59:12.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc6885d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbc6415efb0 0x7fbc640d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc6885d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbc640cf900 0x7fbc6406c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc6885d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc640cfe70 con 0x7fbc6406cd80 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc6885d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc640d0a80 con 0x7fbc6415efb0 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc6885d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc640d0150 con 0x7fbc640cf900 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc62d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbc6406cd80 0x7fbc64099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42530/0 (socket says 172.21.15.139:42530) 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc6259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbc6415efb0 0x7fbc640d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc62d9d640 1 -- 172.21.15.139:0/394210561 learned_addr learned my addr 172.21.15.139:0/394210561 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc6159a640 1 -- 172.21.15.139:0/394210561 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912358027 0 0) 0x7fbc640cfe70 con 0x7fbc6406cd80 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc6159a640 1 -- 172.21.15.139:0/394210561 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc4c003520 con 0x7fbc6406cd80 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc61d9b640 1 --2- 172.21.15.139:0/394210561 >> v2:172.21.15.150:3300/0 conn(0x7fbc640cf900 0x7fbc6406c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc6259c640 1 -- 172.21.15.139:0/394210561 >> v2:172.21.15.150:3300/0 conn(0x7fbc640cf900 msgr2=0x7fbc6406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc6259c640 1 --2- 172.21.15.139:0/394210561 >> v2:172.21.15.150:3300/0 conn(0x7fbc640cf900 0x7fbc6406c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc6259c640 1 -- 172.21.15.139:0/394210561 >> v1:172.21.15.139:6789/0 conn(0x7fbc6406cd80 legacy=0x7fbc64099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc6259c640 1 -- 172.21.15.139:0/394210561 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640d0430 con 0x7fbc6415efb0 2024-04-25T09:59:12.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.017+0000 7fbc61d9b640 1 --2- 172.21.15.139:0/394210561 >> v2:172.21.15.150:3300/0 conn(0x7fbc640cf900 0x7fbc6406c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:12.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.018+0000 7fbc6259c640 1 --2- 172.21.15.139:0/394210561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc6415efb0 0x7fbc640d0f10 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbc6409aad0 tx=0x7fbc5402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c356343393c03aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:12.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.018+0000 7fbc6159a640 1 -- 172.21.15.139:0/394210561 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc54037070 con 0x7fbc6415efb0 2024-04-25T09:59:12.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.018+0000 7fbc6159a640 1 -- 172.21.15.139:0/394210561 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc54032440 con 0x7fbc6415efb0 2024-04-25T09:59:12.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.018+0000 7fbc6159a640 1 -- 172.21.15.139:0/394210561 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5402ae60 con 0x7fbc6415efb0 2024-04-25T09:59:12.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.019+0000 7fbc6885d640 1 -- 172.21.15.139:0/394210561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc6415efb0 msgr2=0x7fbc640d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.019+0000 7fbc6885d640 1 --2- 172.21.15.139:0/394210561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc6415efb0 0x7fbc640d0f10 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbc6409aad0 tx=0x7fbc5402a110 comp rx=0 tx=0).stop 2024-04-25T09:59:12.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.019+0000 7fbc6885d640 1 -- 172.21.15.139:0/394210561 shutdown_connections 2024-04-25T09:59:12.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.019+0000 7fbc6885d640 1 --2- 172.21.15.139:0/394210561 >> v2:172.21.15.150:3300/0 conn(0x7fbc640cf900 0x7fbc6406c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.019+0000 7fbc6885d640 1 --2- 172.21.15.139:0/394210561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc6415efb0 0x7fbc640d0f10 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.019+0000 7fbc6885d640 1 -- 172.21.15.139:0/394210561 >> 172.21.15.139:0/394210561 conn(0x7fbc640d1790 msgr2=0x7fbc640d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:12.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.019+0000 7fbc6885d640 1 -- 172.21.15.139:0/394210561 shutdown_connections 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.019+0000 7fbc6885d640 1 -- 172.21.15.139:0/394210561 wait complete. 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.020+0000 7fbc6885d640 1 Processor -- start 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.020+0000 7fbc6885d640 1 -- start start 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc6885d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc6406cd80 0x7fbc64161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc6885d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc640cf900 0x7fbc64154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc6885d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc6415efb0 0x7fbc641e9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc6885d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6409a2e0 con 0x7fbc6415efb0 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc6885d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6409a160 con 0x7fbc6406cd80 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc6885d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc640cbde0 con 0x7fbc640cf900 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc62d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc6415efb0 0x7fbc641e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc62d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc6415efb0 0x7fbc641e9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49186/0 (socket says 172.21.15.139:49186) 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc62d9d640 1 -- 172.21.15.139:0/180238516 learned_addr learned my addr 172.21.15.139:0/180238516 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:12.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc6259c640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc6406cd80 0x7fbc64161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc61d9b640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc640cf900 0x7fbc64154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc62d9d640 1 -- 172.21.15.139:0/180238516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc640cf900 msgr2=0x7fbc64154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc62d9d640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc640cf900 0x7fbc64154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc62d9d640 1 -- 172.21.15.139:0/180238516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc6406cd80 msgr2=0x7fbc64161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc62d9d640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc6406cd80 0x7fbc64161dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc62d9d640 1 -- 172.21.15.139:0/180238516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc64160280 con 0x7fbc6415efb0 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc62d9d640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc6415efb0 0x7fbc641e9110 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00cbe0 tx=0x7fbc5c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc52ffd640 1 -- 172.21.15.139:0/180238516 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c017070 con 0x7fbc6415efb0 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc52ffd640 1 -- 172.21.15.139:0/180238516 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c00ad40 con 0x7fbc6415efb0 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc6885d640 1 -- 172.21.15.139:0/180238516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc64160560 con 0x7fbc6415efb0 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc52ffd640 1 -- 172.21.15.139:0/180238516 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c012600 con 0x7fbc6415efb0 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.021+0000 7fbc6885d640 1 -- 172.21.15.139:0/180238516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc64160b20 con 0x7fbc6415efb0 2024-04-25T09:59:12.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.023+0000 7fbc6885d640 1 -- 172.21.15.139:0/180238516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc64074520 con 0x7fbc6415efb0 2024-04-25T09:59:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.023+0000 7fbc52ffd640 1 -- 172.21.15.139:0/180238516 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc5c01a070 con 0x7fbc6415efb0 2024-04-25T09:59:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.023+0000 7fbc52ffd640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbc4406b4d0 0x7fbc4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.023+0000 7fbc52ffd640 1 -- 172.21.15.139:0/180238516 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbc5c08a0d0 con 0x7fbc6415efb0 2024-04-25T09:59:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.023+0000 7fbc6259c640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbc4406b4d0 0x7fbc4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:12.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.024+0000 7fbc6259c640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbc4406b4d0 0x7fbc4406d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbc54034000 tx=0x7fbc5402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:12.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.028+0000 7fbc52ffd640 1 -- 172.21.15.139:0/180238516 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc5c058720 con 0x7fbc6415efb0 2024-04-25T09:59:12.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.233+0000 7fbc6885d640 1 -- 172.21.15.139:0/180238516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc64161ba0 con 0x7fbc6415efb0 2024-04-25T09:59:12.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.233+0000 7fbc52ffd640 1 -- 172.21.15.139:0/180238516 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4953 (secure 0 0 0) 0x7fbc5c05c7c0 con 0x7fbc6415efb0 2024-04-25T09:59:12.237 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:12.238 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T09:59:11.904883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":53,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6836/1239243954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1239243954},{"type":"v1","addr":"172.21.15.139:6836","nonce":1239243954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:12.238 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 53 2024-04-25T09:59:12.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.235+0000 7fbc6885d640 1 -- 172.21.15.139:0/180238516 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbc4406b4d0 msgr2=0x7fbc4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.235+0000 7fbc6885d640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbc4406b4d0 0x7fbc4406d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbc54034000 tx=0x7fbc5402e000 comp rx=0 tx=0).stop 2024-04-25T09:59:12.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.235+0000 7fbc6885d640 1 -- 172.21.15.139:0/180238516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc6415efb0 msgr2=0x7fbc641e9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:12.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.235+0000 7fbc6885d640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc6415efb0 0x7fbc641e9110 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00cbe0 tx=0x7fbc5c00a430 comp rx=0 tx=0).stop 2024-04-25T09:59:12.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.235+0000 7fbc6885d640 1 -- 172.21.15.139:0/180238516 shutdown_connections 2024-04-25T09:59:12.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.235+0000 7fbc6885d640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc6415efb0 0x7fbc641e9110 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.235+0000 7fbc6885d640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbc4406b4d0 0x7fbc4406d990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.235+0000 7fbc6885d640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc640cf900 0x7fbc64154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.235+0000 7fbc6885d640 1 --2- 172.21.15.139:0/180238516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc6406cd80 0x7fbc64161dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:12.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.235+0000 7fbc6885d640 1 -- 172.21.15.139:0/180238516 >> 172.21.15.139:0/180238516 conn(0x7fbc640d1790 msgr2=0x7fbc6409a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:12.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.235+0000 7fbc6885d640 1 -- 172.21.15.139:0/180238516 shutdown_connections 2024-04-25T09:59:12.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:12.235+0000 7fbc6885d640 1 -- 172.21.15.139:0/180238516 wait complete. 2024-04-25T09:59:12.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:57.374595+0000', 'modified': '2024-04-25T09:59:11.904883+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7545': {'gid': 7545, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.139:6836/1239243954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1239243954}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 1239243954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:59:12.652 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7545:up:replay 2024-04-25T09:59:13.653 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:13.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.759+0000 7fd577916640 1 Processor -- start 2024-04-25T09:59:13.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.759+0000 7fd577916640 1 -- start start 2024-04-25T09:59:13.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.760+0000 7fd577916640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd57006c780 0x7fd5700d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.760+0000 7fd577916640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5700cf110 con 0x7fd5700b6af0 2024-04-25T09:59:13.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.760+0000 7fd577916640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5700b6f50 con 0x7fd5701611e0 2024-04-25T09:59:13.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.760+0000 7fd577916640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5700d1fa0 con 0x7fd57006c780 2024-04-25T09:59:13.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.760+0000 7fd574eb1640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd5700b6af0 0x7fd57006c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42546/0 (socket says 172.21.15.139:42546) 2024-04-25T09:59:13.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.760+0000 7fd574eb1640 1 -- 172.21.15.139:0/1673543229 learned_addr learned my addr 172.21.15.139:0/1673543229 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:13.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.760+0000 7fd575eb3640 1 --2- 172.21.15.139:0/1673543229 >> v2:172.21.15.139:3300/0 conn(0x7fd57006c780 0x7fd5700d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.761+0000 7fd575eb3640 1 -- 172.21.15.139:0/1673543229 >> v1:172.21.15.150:6789/0 conn(0x7fd5701611e0 legacy=0x7fd5700cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:13.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.761+0000 7fd575eb3640 1 -- 172.21.15.139:0/1673543229 >> v1:172.21.15.139:6789/0 conn(0x7fd5700b6af0 legacy=0x7fd57006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.761+0000 7fd567fff640 1 -- 172.21.15.139:0/1673543229 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882700720 0 0) 0x7fd5700cf110 con 0x7fd5700b6af0 2024-04-25T09:59:13.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.761+0000 7fd575eb3640 1 -- 172.21.15.139:0/1673543229 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5700d4740 con 0x7fd57006c780 2024-04-25T09:59:13.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.761+0000 7fd575eb3640 1 --2- 172.21.15.139:0/1673543229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd57006c780 0x7fd5700d29f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd568009d90 tx=0x7fd56802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350a8c21bac2a189 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.761+0000 7fd567fff640 1 -- 172.21.15.139:0/1673543229 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568007740 con 0x7fd57006c780 2024-04-25T09:59:13.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.761+0000 7fd567fff640 1 -- 172.21.15.139:0/1673543229 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5680078e0 con 0x7fd57006c780 2024-04-25T09:59:13.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.761+0000 7fd567fff640 1 -- 172.21.15.139:0/1673543229 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568007cb0 con 0x7fd57006c780 2024-04-25T09:59:13.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.762+0000 7fd577916640 1 -- 172.21.15.139:0/1673543229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd57006c780 msgr2=0x7fd5700d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.762+0000 7fd577916640 1 --2- 172.21.15.139:0/1673543229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd57006c780 0x7fd5700d29f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd568009d90 tx=0x7fd56802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.762+0000 7fd577916640 1 -- 172.21.15.139:0/1673543229 shutdown_connections 2024-04-25T09:59:13.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.762+0000 7fd577916640 1 --2- 172.21.15.139:0/1673543229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd57006c780 0x7fd5700d29f0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.762+0000 7fd577916640 1 -- 172.21.15.139:0/1673543229 >> 172.21.15.139:0/1673543229 conn(0x7fd5700b5b00 msgr2=0x7fd5700d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:13.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.762+0000 7fd577916640 1 -- 172.21.15.139:0/1673543229 shutdown_connections 2024-04-25T09:59:13.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.762+0000 7fd577916640 1 -- 172.21.15.139:0/1673543229 wait complete. 2024-04-25T09:59:13.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.763+0000 7fd577916640 1 Processor -- start 2024-04-25T09:59:13.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.763+0000 7fd577916640 1 -- start start 2024-04-25T09:59:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd577916640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd57006c780 0x7fd57015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd577916640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5700b6af0 0x7fd570151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd577916640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5701611e0 0x7fd5701f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd577916640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700b2a50 con 0x7fd5700b6af0 2024-04-25T09:59:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd577916640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700d2560 con 0x7fd5701611e0 2024-04-25T09:59:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd577916640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700b2d30 con 0x7fd57006c780 2024-04-25T09:59:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd574eb1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5700b6af0 0x7fd570151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd574eb1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5700b6af0 0x7fd570151ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49214/0 (socket says 172.21.15.139:49214) 2024-04-25T09:59:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd574eb1640 1 -- 172.21.15.139:0/2624717534 learned_addr learned my addr 172.21.15.139:0/2624717534 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:13.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd575eb3640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5701611e0 0x7fd5701f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd574eb1640 1 -- 172.21.15.139:0/2624717534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd57006c780 msgr2=0x7fd57015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd5756b2640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd57006c780 0x7fd57015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd574eb1640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd57006c780 0x7fd57015ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd574eb1640 1 -- 172.21.15.139:0/2624717534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5701611e0 msgr2=0x7fd5701f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:13.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd574eb1640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5701611e0 0x7fd5701f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:13.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd574eb1640 1 -- 172.21.15.139:0/2624717534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5700caa90 con 0x7fd5700b6af0 2024-04-25T09:59:13.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd575eb3640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5701611e0 0x7fd5701f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:13.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.764+0000 7fd5756b2640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd57006c780 0x7fd57015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:13.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.765+0000 7fd574eb1640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5700b6af0 0x7fd570151ed0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fd56000c4a0 tx=0x7fd56000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.765+0000 7fd565ffb640 1 -- 172.21.15.139:0/2624717534 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560018070 con 0x7fd5700b6af0 2024-04-25T09:59:13.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.765+0000 7fd565ffb640 1 -- 172.21.15.139:0/2624717534 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5600074e0 con 0x7fd5700b6af0 2024-04-25T09:59:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.765+0000 7fd577916640 1 -- 172.21.15.139:0/2624717534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5701f41f0 con 0x7fd5700b6af0 2024-04-25T09:59:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.765+0000 7fd565ffb640 1 -- 172.21.15.139:0/2624717534 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560004a20 con 0x7fd5700b6af0 2024-04-25T09:59:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.765+0000 7fd577916640 1 -- 172.21.15.139:0/2624717534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5701f47b0 con 0x7fd5700b6af0 2024-04-25T09:59:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.766+0000 7fd577916640 1 -- 172.21.15.139:0/2624717534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd570074520 con 0x7fd5700b6af0 2024-04-25T09:59:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.766+0000 7fd565ffb640 1 -- 172.21.15.139:0/2624717534 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd56001b070 con 0x7fd5700b6af0 2024-04-25T09:59:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.767+0000 7fd565ffb640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd55406b4d0 0x7fd55406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.767+0000 7fd565ffb640 1 -- 172.21.15.139:0/2624717534 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd5600896a0 con 0x7fd5700b6af0 2024-04-25T09:59:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.767+0000 7fd5756b2640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd55406b4d0 0x7fd55406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.767+0000 7fd5756b2640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd55406b4d0 0x7fd55406d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd56c000f60 tx=0x7fd56c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:13.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.771+0000 7fd565ffb640 1 -- 172.21.15.139:0/2624717534 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd560058340 con 0x7fd5700b6af0 2024-04-25T09:59:13.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.978+0000 7fd577916640 1 -- 172.21.15.139:0/2624717534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd57006ceb0 con 0x7fd5700b6af0 2024-04-25T09:59:14.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.978+0000 7fd565ffb640 1 -- 172.21.15.139:0/2624717534 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4953 (secure 0 0 0) 0x7fd560057c40 con 0x7fd5700b6af0 2024-04-25T09:59:14.006 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:14.006 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T09:59:12.910129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":53,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.139:6836/1239243954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1239243954},{"type":"v1","addr":"172.21.15.139:6836","nonce":1239243954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:14.008 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 55 2024-04-25T09:59:14.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.981+0000 7fd577916640 1 -- 172.21.15.139:0/2624717534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd55406b4d0 msgr2=0x7fd55406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:14.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.981+0000 7fd577916640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd55406b4d0 0x7fd55406d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd56c000f60 tx=0x7fd56c009040 comp rx=0 tx=0).stop 2024-04-25T09:59:14.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.981+0000 7fd577916640 1 -- 172.21.15.139:0/2624717534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5700b6af0 msgr2=0x7fd570151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:14.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.981+0000 7fd577916640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5700b6af0 0x7fd570151ed0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fd56000c4a0 tx=0x7fd56000c960 comp rx=0 tx=0).stop 2024-04-25T09:59:14.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.981+0000 7fd577916640 1 -- 172.21.15.139:0/2624717534 shutdown_connections 2024-04-25T09:59:14.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.981+0000 7fd577916640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5700b6af0 0x7fd570151ed0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:14.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.981+0000 7fd577916640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd55406b4d0 0x7fd55406d990 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.981+0000 7fd577916640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5701611e0 0x7fd5701f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.981+0000 7fd577916640 1 --2- 172.21.15.139:0/2624717534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd57006c780 0x7fd57015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.981+0000 7fd577916640 1 -- 172.21.15.139:0/2624717534 >> 172.21.15.139:0/2624717534 conn(0x7fd5700b5b00 msgr2=0x7fd570161a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.981+0000 7fd577916640 1 -- 172.21.15.139:0/2624717534 shutdown_connections 2024-04-25T09:59:14.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:13.981+0000 7fd577916640 1 -- 172.21.15.139:0/2624717534 wait complete. 2024-04-25T09:59:14.414 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:57.374595+0000', 'modified': '2024-04-25T09:59:12.910129+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7545': {'gid': 7545, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.139:6836/1239243954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1239243954}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 1239243954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:59:14.414 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7545:up:rejoin 2024-04-25T09:59:15.415 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:15.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.520+0000 7f4f04291640 1 Processor -- start 2024-04-25T09:59:15.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.520+0000 7f4f04291640 1 -- start start 2024-04-25T09:59:15.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.521+0000 7f4f04291640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4efc15ec30 0x7f4efc0aec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:15.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.521+0000 7f4f04291640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4efc0af180 0x7f4efc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:15.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.521+0000 7f4f04291640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4efc147570 con 0x7f4efc147190 2024-04-25T09:59:15.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.521+0000 7f4f04291640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4efc0b24b0 con 0x7f4efc0af180 2024-04-25T09:59:15.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.521+0000 7f4f04291640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0b16f0 con 0x7f4efc15ec30 2024-04-25T09:59:15.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.521+0000 7f4f0282e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4efc0af180 0x7f4efc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:15.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.521+0000 7f4f0282e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4efc0af180 0x7f4efc06d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49230/0 (socket says 172.21.15.139:49230) 2024-04-25T09:59:15.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.521+0000 7f4f0282e640 1 -- 172.21.15.139:0/282656410 learned_addr learned my addr 172.21.15.139:0/282656410 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:15.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.522+0000 7f4f0282e640 1 -- 172.21.15.139:0/282656410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4efc15ec30 msgr2=0x7f4efc0aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:15.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.522+0000 7f4f0182c640 1 --2- 172.21.15.139:0/282656410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4efc15ec30 0x7f4efc0aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:15.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.522+0000 7f4f0282e640 1 --2- 172.21.15.139:0/282656410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4efc15ec30 0x7f4efc0aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:15.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.522+0000 7f4f0282e640 1 -- 172.21.15.139:0/282656410 >> v1:172.21.15.150:6789/0 conn(0x7f4efc147190 legacy=0x7f4efc0b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:15.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.522+0000 7f4f0282e640 1 -- 172.21.15.139:0/282656410 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc0b1930 con 0x7f4efc0af180 2024-04-25T09:59:15.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.522+0000 7f4f0182c640 1 --2- 172.21.15.139:0/282656410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4efc15ec30 0x7f4efc0aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:15.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.522+0000 7f4f0282e640 1 --2- 172.21.15.139:0/282656410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4efc0af180 0x7f4efc06d5e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40098a0 tx=0x7f4ef402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11306cc6bfa999cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:15.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.522+0000 7f4f0102b640 1 -- 172.21.15.139:0/282656410 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4037070 con 0x7f4efc0af180 2024-04-25T09:59:15.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.523+0000 7f4f0102b640 1 -- 172.21.15.139:0/282656410 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef4032440 con 0x7f4efc0af180 2024-04-25T09:59:15.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.523+0000 7f4f0102b640 1 -- 172.21.15.139:0/282656410 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4032810 con 0x7f4efc0af180 2024-04-25T09:59:15.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.523+0000 7f4f04291640 1 -- 172.21.15.139:0/282656410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4efc0af180 msgr2=0x7f4efc06d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:15.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.523+0000 7f4f04291640 1 --2- 172.21.15.139:0/282656410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4efc0af180 0x7f4efc06d5e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40098a0 tx=0x7f4ef402a290 comp rx=0 tx=0).stop 2024-04-25T09:59:15.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.523+0000 7f4f04291640 1 -- 172.21.15.139:0/282656410 shutdown_connections 2024-04-25T09:59:15.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.523+0000 7f4f04291640 1 --2- 172.21.15.139:0/282656410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4efc0af180 0x7f4efc06d5e0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:15.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.523+0000 7f4f04291640 1 --2- 172.21.15.139:0/282656410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4efc15ec30 0x7f4efc0aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:15.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.523+0000 7f4f04291640 1 -- 172.21.15.139:0/282656410 >> 172.21.15.139:0/282656410 conn(0x7f4efc066f20 msgr2=0x7f4efc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:15.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.523+0000 7f4f04291640 1 -- 172.21.15.139:0/282656410 shutdown_connections 2024-04-25T09:59:15.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.523+0000 7f4f04291640 1 -- 172.21.15.139:0/282656410 wait complete. 2024-04-25T09:59:15.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.524+0000 7f4f04291640 1 Processor -- start 2024-04-25T09:59:15.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.524+0000 7f4f04291640 1 -- start start 2024-04-25T09:59:15.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.525+0000 7f4f04291640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4efc0af180 0x7f4efc15d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:15.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.525+0000 7f4f04291640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4efc147190 0x7f4efc142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:15.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.525+0000 7f4f04291640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4efc15ec30 0x7f4efc13e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:15.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.525+0000 7f4f04291640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0b21d0 con 0x7f4efc0af180 2024-04-25T09:59:15.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.525+0000 7f4f04291640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0b1ef0 con 0x7f4efc15ec30 2024-04-25T09:59:15.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.525+0000 7f4f04291640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc06db20 con 0x7f4efc147190 2024-04-25T09:59:15.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.525+0000 7f4f0202d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4efc0af180 0x7f4efc15d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:15.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.525+0000 7f4f0202d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4efc0af180 0x7f4efc15d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49240/0 (socket says 172.21.15.139:49240) 2024-04-25T09:59:15.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.525+0000 7f4f0202d640 1 -- 172.21.15.139:0/2657147593 learned_addr learned my addr 172.21.15.139:0/2657147593 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.525+0000 7f4f0182c640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4efc147190 0x7f4efc142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.525+0000 7f4f0282e640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4efc15ec30 0x7f4efc13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.526+0000 7f4f0202d640 1 -- 172.21.15.139:0/2657147593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4efc147190 msgr2=0x7f4efc142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.526+0000 7f4f0202d640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4efc147190 0x7f4efc142df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.526+0000 7f4f0202d640 1 -- 172.21.15.139:0/2657147593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4efc15ec30 msgr2=0x7f4efc13e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.526+0000 7f4f0202d640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4efc15ec30 0x7f4efc13e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.526+0000 7f4f0202d640 1 -- 172.21.15.139:0/2657147593 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc13f1d0 con 0x7f4efc0af180 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.526+0000 7f4f0202d640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4efc0af180 0x7f4efc15d260 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800c790 tx=0x7f4ef800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.526+0000 7f4ef2ffd640 1 -- 172.21.15.139:0/2657147593 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8016020 con 0x7f4efc0af180 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.526+0000 7f4ef2ffd640 1 -- 172.21.15.139:0/2657147593 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef8006d50 con 0x7f4efc0af180 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.526+0000 7f4f04291640 1 -- 172.21.15.139:0/2657147593 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4efc0cd060 con 0x7f4efc0af180 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.526+0000 7f4ef2ffd640 1 -- 172.21.15.139:0/2657147593 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8003bb0 con 0x7f4efc0af180 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.526+0000 7f4f04291640 1 -- 172.21.15.139:0/2657147593 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4efc0cd620 con 0x7f4efc0af180 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.527+0000 7f4f04291640 1 -- 172.21.15.139:0/2657147593 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4efc074520 con 0x7f4efc0af180 2024-04-25T09:59:15.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.527+0000 7f4ef2ffd640 1 -- 172.21.15.139:0/2657147593 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ef800a850 con 0x7f4efc0af180 2024-04-25T09:59:15.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.528+0000 7f4ef2ffd640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ee006b4d0 0x7f4ee006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:15.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.528+0000 7f4f0182c640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ee006b4d0 0x7f4ee006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:15.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.528+0000 7f4ef2ffd640 1 -- 172.21.15.139:0/2657147593 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4ef8089f30 con 0x7f4efc0af180 2024-04-25T09:59:15.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.528+0000 7f4f0182c640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ee006b4d0 0x7f4ee006d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4efc142b40 tx=0x7f4eec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:15.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.532+0000 7f4ef2ffd640 1 -- 172.21.15.139:0/2657147593 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ef8020020 con 0x7f4efc0af180 2024-04-25T09:59:15.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.744+0000 7f4f04291640 1 -- 172.21.15.139:0/2657147593 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4efc145920 con 0x7f4efc0af180 2024-04-25T09:59:15.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.744+0000 7f4ef2ffd640 1 -- 172.21.15.139:0/2657147593 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f4ef8058610 con 0x7f4efc0af180 2024-04-25T09:59:15.748 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:15.748 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T09:59:13.914754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":8,"addr":"172.21.15.139:6836/1239243954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1239243954},{"type":"v1","addr":"172.21.15.139:6836","nonce":1239243954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:15.749 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 56 2024-04-25T09:59:15.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.746+0000 7f4f04291640 1 -- 172.21.15.139:0/2657147593 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ee006b4d0 msgr2=0x7f4ee006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:15.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.746+0000 7f4f04291640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ee006b4d0 0x7f4ee006d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4efc142b40 tx=0x7f4eec008040 comp rx=0 tx=0).stop 2024-04-25T09:59:15.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.746+0000 7f4f04291640 1 -- 172.21.15.139:0/2657147593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4efc0af180 msgr2=0x7f4efc15d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:15.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.746+0000 7f4f04291640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4efc0af180 0x7f4efc15d260 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800c790 tx=0x7f4ef800cc50 comp rx=0 tx=0).stop 2024-04-25T09:59:15.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.746+0000 7f4f04291640 1 -- 172.21.15.139:0/2657147593 shutdown_connections 2024-04-25T09:59:15.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.746+0000 7f4f04291640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ee006b4d0 0x7f4ee006d990 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:15.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.746+0000 7f4f04291640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4efc15ec30 0x7f4efc13e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:15.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.746+0000 7f4f04291640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4efc147190 0x7f4efc142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:15.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.746+0000 7f4f04291640 1 --2- 172.21.15.139:0/2657147593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4efc0af180 0x7f4efc15d260 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:15.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.746+0000 7f4f04291640 1 -- 172.21.15.139:0/2657147593 >> 172.21.15.139:0/2657147593 conn(0x7f4efc066f20 msgr2=0x7f4efc15f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:15.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.746+0000 7f4f04291640 1 -- 172.21.15.139:0/2657147593 shutdown_connections 2024-04-25T09:59:15.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:15.746+0000 7f4f04291640 1 -- 172.21.15.139:0/2657147593 wait complete. 2024-04-25T09:59:16.164 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:57.374595+0000', 'modified': '2024-04-25T09:59:13.914754+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7545': {'gid': 7545, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.139:6836/1239243954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1239243954}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 1239243954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:59:16.164 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:59:16.165 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:59:16.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.260+0000 7f4da6407640 1 Processor -- start 2024-04-25T09:59:16.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.260+0000 7f4da6407640 1 -- start start 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.260+0000 7f4da6407640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4da01617e0 0x7f4da00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.260+0000 7f4da6407640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4da00cec80 0x7f4da00d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.260+0000 7f4da6407640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da0161dc0 con 0x7f4da015dbf0 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.260+0000 7f4da6407640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da00d1ee0 con 0x7f4da00cec80 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.260+0000 7f4da6407640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da01620a0 con 0x7f4da01617e0 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.261+0000 7f4da49a4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4da00cec80 0x7f4da00d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.261+0000 7f4d9f7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4da01617e0 0x7f4da00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.261+0000 7f4da49a4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4da00cec80 0x7f4da00d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:40370/0 (socket says 172.21.15.139:40370) 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.261+0000 7f4da49a4640 1 -- 172.21.15.139:0/658901732 learned_addr learned my addr 172.21.15.139:0/658901732 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:16.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.261+0000 7f4da49a4640 1 -- 172.21.15.139:0/658901732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4da01617e0 msgr2=0x7f4da00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:16.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.261+0000 7f4da49a4640 1 --2- 172.21.15.139:0/658901732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4da01617e0 0x7f4da00ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.261+0000 7f4da49a4640 1 -- 172.21.15.139:0/658901732 >> v1:172.21.15.150:6789/0 conn(0x7f4da015dbf0 legacy=0x7f4da00b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.261+0000 7f4da49a4640 1 -- 172.21.15.139:0/658901732 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da0162380 con 0x7f4da00cec80 2024-04-25T09:59:16.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.262+0000 7f4da49a4640 1 --2- 172.21.15.139:0/658901732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4da00cec80 0x7f4da00d2370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009e50 tx=0x7f4d9802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da6ad60c4e6db5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:16.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.262+0000 7f4d9effd640 1 -- 172.21.15.139:0/658901732 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98036070 con 0x7f4da00cec80 2024-04-25T09:59:16.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.262+0000 7f4d9effd640 1 -- 172.21.15.139:0/658901732 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d98031600 con 0x7f4da00cec80 2024-04-25T09:59:16.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.262+0000 7f4d9effd640 1 -- 172.21.15.139:0/658901732 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d980319d0 con 0x7f4da00cec80 2024-04-25T09:59:16.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.263+0000 7f4da6407640 1 -- 172.21.15.139:0/658901732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4da00cec80 msgr2=0x7f4da00d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.263+0000 7f4da6407640 1 --2- 172.21.15.139:0/658901732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4da00cec80 0x7f4da00d2370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009e50 tx=0x7f4d9802a680 comp rx=0 tx=0).stop 2024-04-25T09:59:16.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.263+0000 7f4da6407640 1 -- 172.21.15.139:0/658901732 shutdown_connections 2024-04-25T09:59:16.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.263+0000 7f4da6407640 1 --2- 172.21.15.139:0/658901732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4da01617e0 0x7f4da00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.263+0000 7f4da6407640 1 --2- 172.21.15.139:0/658901732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4da00cec80 0x7f4da00d2370 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.263+0000 7f4da6407640 1 -- 172.21.15.139:0/658901732 >> 172.21.15.139:0/658901732 conn(0x7f4da00b5480 msgr2=0x7f4da00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:16.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.263+0000 7f4da6407640 1 -- 172.21.15.139:0/658901732 shutdown_connections 2024-04-25T09:59:16.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.263+0000 7f4da6407640 1 -- 172.21.15.139:0/658901732 wait complete. 2024-04-25T09:59:16.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.264+0000 7f4da6407640 1 Processor -- start 2024-04-25T09:59:16.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.264+0000 7f4da6407640 1 -- start start 2024-04-25T09:59:16.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4da6407640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4da00cec80 0x7f4da00c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4da6407640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da015dbf0 0x7f4da015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4da6407640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4da01617e0 0x7f4da00cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4da6407640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da015dfd0 con 0x7f4da015dbf0 2024-04-25T09:59:16.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4da6407640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00d28b0 con 0x7f4da00cec80 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4da6407640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da0152830 con 0x7f4da01617e0 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4d9f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da015dbf0 0x7f4da015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4d9f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da015dbf0 0x7f4da015a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49244/0 (socket says 172.21.15.139:49244) 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4d9f7fe640 1 -- 172.21.15.139:0/3345262876 learned_addr learned my addr 172.21.15.139:0/3345262876 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4da00cec80 0x7f4da00c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4da49a4640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4da01617e0 0x7f4da00cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4d9f7fe640 1 -- 172.21.15.139:0/3345262876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4da01617e0 msgr2=0x7f4da00cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4d9f7fe640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4da01617e0 0x7f4da00cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4d9f7fe640 1 -- 172.21.15.139:0/3345262876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4da00cec80 msgr2=0x7f4da00c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4d9f7fe640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4da00cec80 0x7f4da00c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4d9f7fe640 1 -- 172.21.15.139:0/3345262876 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da0152b10 con 0x7f4da015dbf0 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4da49a4640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4da01617e0 0x7f4da00cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4d9f7fe640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da015dbf0 0x7f4da015a100 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400e870 tx=0x7f4d9400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4d9cff9640 1 -- 172.21.15.139:0/3345262876 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94017070 con 0x7f4da015dbf0 2024-04-25T09:59:16.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.265+0000 7f4d9cff9640 1 -- 172.21.15.139:0/3345262876 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9400c940 con 0x7f4da015dbf0 2024-04-25T09:59:16.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.266+0000 7f4d9cff9640 1 -- 172.21.15.139:0/3345262876 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9400cd30 con 0x7f4da015dbf0 2024-04-25T09:59:16.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.266+0000 7f4da6407640 1 -- 172.21.15.139:0/3345262876 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da0152df0 con 0x7f4da015dbf0 2024-04-25T09:59:16.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.266+0000 7f4da6407640 1 -- 172.21.15.139:0/3345262876 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da01533b0 con 0x7f4da015dbf0 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.267+0000 7f4d9cff9640 1 -- 172.21.15.139:0/3345262876 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d9401a070 con 0x7f4da015dbf0 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.268+0000 7f4d9cff9640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d8c06b4d0 0x7f4d8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.268+0000 7f4d9cff9640 1 -- 172.21.15.139:0/3345262876 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4d9408a660 con 0x7f4da015dbf0 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.268+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d8c06b4d0 0x7f4d8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.268+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d8c06b4d0 0x7f4d8c06d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4d90007fd0 tx=0x7f4d90009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.268+0000 7f4da91e1740 1 Processor -- start 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.269+0000 7f4da91e1740 1 -- start start 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.269+0000 7f4da91e1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8275f6230 0x55d8275f6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.269+0000 7f4da91e1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 0x55d827614020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.269+0000 7f4da91e1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275cbf10 0x55d827612190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.269+0000 7f4da91e1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d82752ea10 con 0x55d8275cbf10 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.269+0000 7f4da91e1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8274d7b30 con 0x55d827613ad0 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.269+0000 7f4da91e1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d82752ac50 con 0x55d8275f6230 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.269+0000 7f4da49a4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275cbf10 0x55d827612190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.269+0000 7f4da49a4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275cbf10 0x55d827612190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49254/0 (socket says 172.21.15.139:49254) 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.269+0000 7f4da49a4640 1 -- 172.21.15.139:0/2557803954 learned_addr learned my addr 172.21.15.139:0/2557803954 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.269+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/2557803954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8275f6230 0x55d8275f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.270+0000 7f4d9f7fe640 1 --2- 172.21.15.139:0/2557803954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 0x55d827614020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.270+0000 7f4da49a4640 1 -- 172.21.15.139:0/2557803954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8275f6230 msgr2=0x55d8275f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.270+0000 7f4da49a4640 1 --2- 172.21.15.139:0/2557803954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8275f6230 0x55d8275f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.270+0000 7f4da49a4640 1 -- 172.21.15.139:0/2557803954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 msgr2=0x55d827614020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.270+0000 7f4da49a4640 1 --2- 172.21.15.139:0/2557803954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 0x55d827614020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.270+0000 7f4da49a4640 1 -- 172.21.15.139:0/2557803954 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8274ece10 con 0x55d8275cbf10 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.270+0000 7f4d9f7fe640 1 --2- 172.21.15.139:0/2557803954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 0x55d827614020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.270+0000 7f4da49a4640 1 --2- 172.21.15.139:0/2557803954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275cbf10 0x55d827612190 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f4d9802a6c0 tx=0x7f4d9802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea751edab4a87d4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.270+0000 7f4d85ffb640 1 -- 172.21.15.139:0/2557803954 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98036040 con 0x55d8275cbf10 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.270+0000 7f4d85ffb640 1 -- 172.21.15.139:0/2557803954 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9803c640 con 0x55d8275cbf10 2024-04-25T09:59:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.270+0000 7f4d85ffb640 1 -- 172.21.15.139:0/2557803954 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9803c9f0 con 0x55d8275cbf10 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.271+0000 7f4da91e1740 1 -- 172.21.15.139:0/2557803954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275cbf10 msgr2=0x55d827612190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.271+0000 7f4da91e1740 1 --2- 172.21.15.139:0/2557803954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275cbf10 0x55d827612190 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f4d9802a6c0 tx=0x7f4d9802e000 comp rx=0 tx=0).stop 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.271+0000 7f4da91e1740 1 -- 172.21.15.139:0/2557803954 shutdown_connections 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.271+0000 7f4da91e1740 1 --2- 172.21.15.139:0/2557803954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275cbf10 0x55d827612190 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.271+0000 7f4da91e1740 1 --2- 172.21.15.139:0/2557803954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 0x55d827614020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.271+0000 7f4da91e1740 1 --2- 172.21.15.139:0/2557803954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d8275f6230 0x55d8275f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.271+0000 7f4da91e1740 1 -- 172.21.15.139:0/2557803954 >> 172.21.15.139:0/2557803954 conn(0x55d8275c8840 msgr2=0x55d827611040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.271+0000 7f4da91e1740 1 -- 172.21.15.139:0/2557803954 shutdown_connections 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.271+0000 7f4da91e1740 1 -- 172.21.15.139:0/2557803954 wait complete. 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.271+0000 7f4da91e1740 1 -- start start 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.271+0000 7f4da91e1740 1 Processor -- start 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.272+0000 7f4da91e1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d827609250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.272+0000 7f4da91e1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827613ad0 0x55d827609a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.272+0000 7f4da91e1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8275f8690 0x55d827609fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.272+0000 7f4da91e1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8275cccb0 con 0x55d8275f6230 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.272+0000 7f4da91e1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8274eb0c0 con 0x55d8275f8690 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.272+0000 7f4da91e1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8275c6770 con 0x55d827613ad0 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.272+0000 7f4da91e1740 10 client.0 _ll_register_callbacks cb 0x55d8274deec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.272+0000 7f4d9ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d827609250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.272+0000 7f4d9ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d827609250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49266/0 (socket says 172.21.15.139:49266) 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.272+0000 7f4d9ffff640 1 -- 172.21.15.139:0/419672931 learned_addr learned my addr 172.21.15.139:0/419672931 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.272+0000 7f4da49a4640 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8275f8690 0x55d827609fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.272+0000 7f4d9f7fe640 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827613ad0 0x55d827609a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d9ffff640 1 -- 172.21.15.139:0/419672931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827613ad0 msgr2=0x55d827609a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827613ad0 0x55d827609a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d9ffff640 1 -- 172.21.15.139:0/419672931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8275f8690 msgr2=0x55d827609fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8275f8690 0x55d827609fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d9ffff640 1 -- 172.21.15.139:0/419672931 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8272432d0 con 0x55d8275f6230 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d827609250 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f4d90003950 tx=0x7f4d90000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa24971467e5e524 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d7effd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d7effd640 1 -- 172.21.15.139:0/419672931 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90027060 con 0x55d8275f6230 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d7effd640 1 -- 172.21.15.139:0/419672931 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4d90010040 con 0x55d8275f6230 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4da91e1740 1 -- 172.21.15.139:0/419672931 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d9803c640 con 0x55d8275f6230 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d7effd640 1 -- 172.21.15.139:0/419672931 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d9000ec90 con 0x55d8275f6230 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4da91e1740 10 client.7854 fetch_fsmap learned FSMap version 56 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4da91e1740 1 -- 172.21.15.139:0/419672931 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55d82730d6b0 con 0x55d8275f6230 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d7effd640 1 -- 172.21.15.139:0/419672931 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9803c640 con 0x55d8275f6230 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d7effd640 1 -- 172.21.15.139:0/419672931 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4d90013420 con 0x55d8275f6230 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4da91e1740 10 client.7854 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4da91e1740 10 client.7854 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4da91e1740 20 client.7854 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4da91e1740 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55d8275e5570 0x55d8275f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4da91e1740 4 client.7854 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4da91e1740 1 -- 172.21.15.139:0/419672931 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d82758d920 con 0x55d8275e5570 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.273+0000 7f4d9f7fe640 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55d8275e5570 0x55d8275f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.274+0000 7f4d7effd640 10 client.7854 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.274+0000 7f4d9f7fe640 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55d8275e5570 0x55d8275f9d60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb90125ef11878b server_cookie=a64e2fc3150d4f25 in_seq=0 out_seq=0 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.274+0000 7f4d7effd640 1 -- 172.21.15.139:0/419672931 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d82758d920 con 0x55d8275e5570 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.274+0000 7f4d7effd640 10 client.7854 handle_command_reply: tid=0 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.274+0000 7f4da91e1740 1 client.7854 shutdown 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.274+0000 7f4da91e1740 1 -- 172.21.15.139:0/419672931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 msgr2=0x55d827609250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.274+0000 7f4da91e1740 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d827609250 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f4d90003950 tx=0x7f4d90000ed0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.275+0000 7f4da91e1740 1 -- 172.21.15.139:0/419672931 shutdown_connections 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.275+0000 7f4da91e1740 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55d8275e5570 0x55d8275f9d60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.275+0000 7f4da91e1740 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8275f8690 0x55d827609fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.275+0000 7f4da91e1740 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827613ad0 0x55d827609a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.275+0000 7f4d7effd640 0 client.7854 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:16.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.275+0000 7f4da91e1740 1 --2- 172.21.15.139:0/419672931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d827609250 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.275+0000 7f4da91e1740 1 -- 172.21.15.139:0/419672931 >> 172.21.15.139:0/419672931 conn(0x55d8275cbf10 msgr2=0x55d827614e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:16.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.275+0000 7f4da91e1740 1 -- 172.21.15.139:0/419672931 shutdown_connections 2024-04-25T09:59:16.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.275+0000 7f4da91e1740 1 -- 172.21.15.139:0/419672931 wait complete. 2024-04-25T09:59:16.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.275+0000 7f4da91e1740 20 client.7854 trim_cache size 0 max 16384 2024-04-25T09:59:16.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.290+0000 7f4da91e1740 1 Processor -- start 2024-04-25T09:59:16.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.290+0000 7f4da91e1740 1 -- start start 2024-04-25T09:59:16.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4da91e1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d8275c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4da91e1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 0x55d8275cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4da91e1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827609cb0 0x55d8275cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4da91e1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da01ff7e0 con 0x55d8275f6230 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4da91e1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da0154840 con 0x55d827613ad0 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4da91e1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da0154e40 con 0x55d827609cb0 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4d9ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d8275c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4d9ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d8275c7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49272/0 (socket says 172.21.15.139:49272) 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4d9ffff640 1 -- 172.21.15.139:0/3291599962 learned_addr learned my addr 172.21.15.139:0/3291599962 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4d9ffff640 1 -- 172.21.15.139:0/3291599962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827609cb0 msgr2=0x55d8275cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/3291599962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827609cb0 0x55d8275cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4d9ffff640 1 -- 172.21.15.139:0/3291599962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 msgr2=0x55d8275cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/3291599962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 0x55d8275cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.291+0000 7f4d9ffff640 1 -- 172.21.15.139:0/3291599962 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8275c7d00 con 0x55d8275f6230 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/3291599962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d8275c7640 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f4d90001420 tx=0x7f4d9001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731505b586014541 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4d857fa640 1 -- 172.21.15.139:0/3291599962 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9000ee30 con 0x55d8275f6230 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4d857fa640 1 -- 172.21.15.139:0/3291599962 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d90013420 con 0x55d8275f6230 2024-04-25T09:59:16.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4d857fa640 1 -- 172.21.15.139:0/3291599962 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90005070 con 0x55d8275f6230 2024-04-25T09:59:16.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4da91e1740 1 -- 172.21.15.139:0/3291599962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 msgr2=0x55d8275c7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4da91e1740 1 --2- 172.21.15.139:0/3291599962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d8275c7640 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f4d90001420 tx=0x7f4d9001a000 comp rx=0 tx=0).stop 2024-04-25T09:59:16.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4da91e1740 1 -- 172.21.15.139:0/3291599962 shutdown_connections 2024-04-25T09:59:16.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4da91e1740 1 --2- 172.21.15.139:0/3291599962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827609cb0 0x55d8275cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4da91e1740 1 --2- 172.21.15.139:0/3291599962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 0x55d8275cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4da91e1740 1 --2- 172.21.15.139:0/3291599962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d8275c7640 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4da91e1740 1 -- 172.21.15.139:0/3291599962 >> 172.21.15.139:0/3291599962 conn(0x55d8275cbf10 msgr2=0x55d82760b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:16.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4da91e1740 1 -- 172.21.15.139:0/3291599962 shutdown_connections 2024-04-25T09:59:16.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.292+0000 7f4da91e1740 1 -- 172.21.15.139:0/3291599962 wait complete. 2024-04-25T09:59:16.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.293+0000 7f4da91e1740 1 -- start start 2024-04-25T09:59:16.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.293+0000 7f4da91e1740 1 Processor -- start 2024-04-25T09:59:16.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4da91e1740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d827630e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4da91e1740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827609cb0 0x55d827608c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4da91e1740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 0x55d8276146c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4da91e1740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8275e53f0 con 0x55d8275f6230 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4da91e1740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8275f7ef0 con 0x55d827613ad0 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4da91e1740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8275e5990 con 0x55d827609cb0 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4da91e1740 10 client.0 _ll_register_callbacks cb 0x55d8275c6c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4d9ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d827630e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4d9ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d827630e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49286/0 (socket says 172.21.15.139:49286) 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4d9ffff640 1 -- 172.21.15.139:0/1721740375 learned_addr learned my addr 172.21.15.139:0/1721740375 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4d9f7fe640 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827609cb0 0x55d827608c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4d9ffff640 1 -- 172.21.15.139:0/1721740375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827609cb0 msgr2=0x55d827608c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827609cb0 0x55d827608c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4da49a4640 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 0x55d8276146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4d9ffff640 1 -- 172.21.15.139:0/1721740375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 msgr2=0x55d8276146c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 0x55d8276146c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4d9ffff640 1 -- 172.21.15.139:0/1721740375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d827634110 con 0x55d8275f6230 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4d9ffff640 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d827630e90 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009010 tx=0x7f4d90013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacdeaa93e9e12d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.294+0000 7f4d7effd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4d7effd640 1 -- 172.21.15.139:0/1721740375 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90027040 con 0x55d8275f6230 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4d7effd640 1 -- 172.21.15.139:0/1721740375 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4d90036680 con 0x55d8275f6230 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4da91e1740 1 -- 172.21.15.139:0/1721740375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d90013420 con 0x55d8275f6230 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4d7effd640 1 -- 172.21.15.139:0/1721740375 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d90045050 con 0x55d8275f6230 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4da91e1740 10 client.7860 fetch_fsmap learned FSMap version 56 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4da91e1740 1 -- 172.21.15.139:0/1721740375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55d827633cc0 con 0x55d8275f6230 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4d7effd640 1 -- 172.21.15.139:0/1721740375 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d90013420 con 0x55d8275f6230 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4d7effd640 1 -- 172.21.15.139:0/1721740375 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4d90048020 con 0x55d8275f6230 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4da91e1740 10 client.7860 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4da91e1740 10 client.7860 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:59:16.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4da91e1740 20 client.7860 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4da91e1740 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55d827614c00 0x55d8276121f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4da91e1740 4 client.7860 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4da91e1740 1 -- 172.21.15.139:0/1721740375 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d82753bf50 con 0x55d827614c00 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4d9f7fe640 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55d827614c00 0x55d8276121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4d7effd640 10 client.7860 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.295+0000 7f4d9f7fe640 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55d827614c00 0x55d8276121f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c99a20fc2581c26 server_cookie=b18ca5c5cb7f008c in_seq=0 out_seq=0 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4d7effd640 1 -- 172.21.15.139:0/1721740375 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x55d82753bf50 con 0x55d827614c00 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4d7effd640 10 client.7860 handle_command_reply: tid=0 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da91e1740 1 client.7860 shutdown 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da91e1740 1 -- 172.21.15.139:0/1721740375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 msgr2=0x55d827630e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da91e1740 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d827630e90 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009010 tx=0x7f4d90013000 comp rx=0 tx=0).stop 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da91e1740 1 -- 172.21.15.139:0/1721740375 shutdown_connections 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da91e1740 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55d827614c00 0x55d8276121f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da91e1740 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d827613ad0 0x55d8276146c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da91e1740 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d827609cb0 0x55d827608c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4d7effd640 0 client.7860 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:16.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da91e1740 1 --2- 172.21.15.139:0/1721740375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d8275f6230 0x55d827630e90 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da91e1740 1 -- 172.21.15.139:0/1721740375 >> 172.21.15.139:0/1721740375 conn(0x55d8275cbf10 msgr2=0x55d827620a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:16.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da91e1740 1 -- 172.21.15.139:0/1721740375 shutdown_connections 2024-04-25T09:59:16.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da91e1740 1 -- 172.21.15.139:0/1721740375 wait complete. 2024-04-25T09:59:16.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da91e1740 20 client.7860 trim_cache size 0 max 16384 2024-04-25T09:59:16.306 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:59:16.306 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T09:59:16.306 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T09:59:16.307 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 7545, 2024-04-25T09:59:16.307 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T09:59:16.307 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T09:59:16.307 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T09:59:16.307 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 4.39031167, 2024-04-25T09:59:16.307 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 56, 2024-04-25T09:59:16.307 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 90, 2024-04-25T09:59:16.307 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T09:59:16.307 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 23.336025830000001 2024-04-25T09:59:16.307 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:59:16.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da6407640 1 -- 172.21.15.139:0/3345262876 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d8c06b4d0 msgr2=0x7f4d8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da6407640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d8c06b4d0 0x7f4d8c06d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4d90007fd0 tx=0x7f4d90009040 comp rx=0 tx=0).stop 2024-04-25T09:59:16.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da6407640 1 -- 172.21.15.139:0/3345262876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da015dbf0 msgr2=0x7f4da015a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da6407640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da015dbf0 0x7f4da015a100 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400e870 tx=0x7f4d9400ed30 comp rx=0 tx=0).stop 2024-04-25T09:59:16.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da6407640 1 -- 172.21.15.139:0/3345262876 shutdown_connections 2024-04-25T09:59:16.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da6407640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4da015dbf0 0x7f4da015a100 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da6407640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d8c06b4d0 0x7f4d8c06d990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da6407640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4da01617e0 0x7f4da00cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da6407640 1 --2- 172.21.15.139:0/3345262876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4da00cec80 0x7f4da00c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.296+0000 7f4da6407640 1 -- 172.21.15.139:0/3345262876 >> 172.21.15.139:0/3345262876 conn(0x7f4da00b5480 msgr2=0x7f4da00d2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:16.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.297+0000 7f4da6407640 1 -- 172.21.15.139:0/3345262876 shutdown_connections 2024-04-25T09:59:16.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.297+0000 7f4da6407640 1 -- 172.21.15.139:0/3345262876 wait complete. 2024-04-25T09:59:16.311 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:59:16.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.403+0000 7f1539390640 1 Processor -- start 2024-04-25T09:59:16.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.403+0000 7f1539390640 1 -- start start 2024-04-25T09:59:16.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.404+0000 7f1539390640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f153415aa00 0x7f15340b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.404+0000 7f1539390640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f15340b5110 0x7f153415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.404+0000 7f1539390640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1534072280 0x7f1534072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.404+0000 7f1539390640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15340b5c20 con 0x7f1534072280 2024-04-25T09:59:16.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.404+0000 7f1539390640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15340b45d0 con 0x7f15340b5110 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.404+0000 7f1539390640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153415e500 con 0x7f153415aa00 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.404+0000 7f15337fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1534072280 0x7f1534072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.404+0000 7f15337fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1534072280 0x7f1534072710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51166/0 (socket says 172.21.15.139:51166) 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.404+0000 7f15337fe640 1 -- 172.21.15.139:0/1113099042 learned_addr learned my addr 172.21.15.139:0/1113099042 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.405+0000 7f15337fe640 1 -- 172.21.15.139:0/1113099042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f153415aa00 msgr2=0x7f15340b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.405+0000 7f15327fc640 1 --2- 172.21.15.139:0/1113099042 >> v2:172.21.15.150:3300/0 conn(0x7f15340b5110 0x7f153415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.405+0000 7f1532ffd640 1 --2- 172.21.15.139:0/1113099042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f153415aa00 0x7f15340b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.405+0000 7f15337fe640 1 --2- 172.21.15.139:0/1113099042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f153415aa00 0x7f15340b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.405+0000 7f15337fe640 1 -- 172.21.15.139:0/1113099042 >> v2:172.21.15.150:3300/0 conn(0x7f15340b5110 msgr2=0x7f153415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.405+0000 7f15337fe640 1 --2- 172.21.15.139:0/1113099042 >> v2:172.21.15.150:3300/0 conn(0x7f15340b5110 0x7f153415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.405+0000 7f15337fe640 1 -- 172.21.15.139:0/1113099042 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15340b5660 con 0x7f1534072280 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.405+0000 7f15327fc640 1 --2- 172.21.15.139:0/1113099042 >> v2:172.21.15.150:3300/0 conn(0x7f15340b5110 0x7f153415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:16.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.405+0000 7f1532ffd640 1 --2- 172.21.15.139:0/1113099042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f153415aa00 0x7f15340b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:16.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.405+0000 7f15337fe640 1 --2- 172.21.15.139:0/1113099042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1534072280 0x7f1534072710 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f153406dbb0 tx=0x7f152402c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee6a5a5f573491d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:16.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.406+0000 7f1531ffb640 1 -- 172.21.15.139:0/1113099042 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524032790 con 0x7f1534072280 2024-04-25T09:59:16.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.406+0000 7f1531ffb640 1 -- 172.21.15.139:0/1113099042 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1524038070 con 0x7f1534072280 2024-04-25T09:59:16.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.406+0000 7f1531ffb640 1 -- 172.21.15.139:0/1113099042 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524032b50 con 0x7f1534072280 2024-04-25T09:59:16.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.406+0000 7f1539390640 1 -- 172.21.15.139:0/1113099042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1534072280 msgr2=0x7f1534072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.406+0000 7f1539390640 1 --2- 172.21.15.139:0/1113099042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1534072280 0x7f1534072710 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f153406dbb0 tx=0x7f152402c8d0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.406+0000 7f1539390640 1 -- 172.21.15.139:0/1113099042 shutdown_connections 2024-04-25T09:59:16.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.406+0000 7f1539390640 1 --2- 172.21.15.139:0/1113099042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1534072280 0x7f1534072710 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.406+0000 7f1539390640 1 --2- 172.21.15.139:0/1113099042 >> v2:172.21.15.150:3300/0 conn(0x7f15340b5110 0x7f153415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.406+0000 7f1539390640 1 --2- 172.21.15.139:0/1113099042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f153415aa00 0x7f15340b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.406+0000 7f1539390640 1 -- 172.21.15.139:0/1113099042 >> 172.21.15.139:0/1113099042 conn(0x7f153406e280 msgr2=0x7f153406e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:16.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.406+0000 7f1539390640 1 -- 172.21.15.139:0/1113099042 shutdown_connections 2024-04-25T09:59:16.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.406+0000 7f1539390640 1 -- 172.21.15.139:0/1113099042 wait complete. 2024-04-25T09:59:16.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.407+0000 7f1539390640 1 Processor -- start 2024-04-25T09:59:16.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.408+0000 7f1539390640 1 -- start start 2024-04-25T09:59:16.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.408+0000 7f1539390640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534072280 0x7f153415c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.408+0000 7f1539390640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340b5110 0x7f153414d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.408+0000 7f1539390640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f153414dcf0 0x7f153415b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.408+0000 7f1539390640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153415f200 con 0x7f153414dcf0 2024-04-25T09:59:16.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.408+0000 7f1539390640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153415e7a0 con 0x7f15340b5110 2024-04-25T09:59:16.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.408+0000 7f1539390640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15340ce610 con 0x7f1534072280 2024-04-25T09:59:16.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.408+0000 7f15337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f153414dcf0 0x7f153415b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.408+0000 7f15337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f153414dcf0 0x7f153415b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51174/0 (socket says 172.21.15.139:51174) 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.408+0000 7f15337fe640 1 -- 172.21.15.139:0/3242243971 learned_addr learned my addr 172.21.15.139:0/3242243971 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.408+0000 7f15327fc640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340b5110 0x7f153414d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f1532ffd640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534072280 0x7f153415c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f15337fe640 1 -- 172.21.15.139:0/3242243971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534072280 msgr2=0x7f153415c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f15337fe640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534072280 0x7f153415c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f15337fe640 1 -- 172.21.15.139:0/3242243971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340b5110 msgr2=0x7f153414d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f15337fe640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340b5110 0x7f153414d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f15337fe640 1 -- 172.21.15.139:0/3242243971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153414dae0 con 0x7f153414dcf0 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f1532ffd640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534072280 0x7f153415c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f15337fe640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f153414dcf0 0x7f153415b690 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f152400b2c0 tx=0x7f15240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f151b7fe640 1 -- 172.21.15.139:0/3242243971 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524033620 con 0x7f153414dcf0 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f151b7fe640 1 -- 172.21.15.139:0/3242243971 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15240337c0 con 0x7f153414dcf0 2024-04-25T09:59:16.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f1539390640 1 -- 172.21.15.139:0/3242243971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153415d190 con 0x7f153414dcf0 2024-04-25T09:59:16.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f151b7fe640 1 -- 172.21.15.139:0/3242243971 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524033bb0 con 0x7f153414dcf0 2024-04-25T09:59:16.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.409+0000 7f1539390640 1 -- 172.21.15.139:0/3242243971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153415ceb0 con 0x7f153414dcf0 2024-04-25T09:59:16.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.410+0000 7f1539390640 1 -- 172.21.15.139:0/3242243971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1534074520 con 0x7f153414dcf0 2024-04-25T09:59:16.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.410+0000 7f151b7fe640 1 -- 172.21.15.139:0/3242243971 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f152404d060 con 0x7f153414dcf0 2024-04-25T09:59:16.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.411+0000 7f151b7fe640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f152006b4d0 0x7f152006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:16.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.411+0000 7f151b7fe640 1 -- 172.21.15.139:0/3242243971 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f15240acc20 con 0x7f153414dcf0 2024-04-25T09:59:16.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.411+0000 7f1532ffd640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f152006b4d0 0x7f152006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:16.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.411+0000 7f1532ffd640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f152006b4d0 0x7f152006d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f152c006fd0 tx=0x7f152c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:16.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.415+0000 7f151b7fe640 1 -- 172.21.15.139:0/3242243971 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1524041070 con 0x7f153414dcf0 2024-04-25T09:59:16.425 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:59:16.427 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:59:16.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.625+0000 7f1539390640 1 -- 172.21.15.139:0/3242243971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1534154d00 con 0x7f153414dcf0 2024-04-25T09:59:16.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.625+0000 7f151b7fe640 1 -- 172.21.15.139:0/3242243971 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f152407b8c0 con 0x7f153414dcf0 2024-04-25T09:59:16.629 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:59:16.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.627+0000 7f1539390640 1 -- 172.21.15.139:0/3242243971 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f152006b4d0 msgr2=0x7f152006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.627+0000 7f1539390640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f152006b4d0 0x7f152006d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f152c006fd0 tx=0x7f152c008040 comp rx=0 tx=0).stop 2024-04-25T09:59:16.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.627+0000 7f1539390640 1 -- 172.21.15.139:0/3242243971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f153414dcf0 msgr2=0x7f153415b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:16.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.627+0000 7f1539390640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f153414dcf0 0x7f153415b690 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f152400b2c0 tx=0x7f15240046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.627+0000 7f1539390640 1 -- 172.21.15.139:0/3242243971 shutdown_connections 2024-04-25T09:59:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.627+0000 7f1539390640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f153414dcf0 0x7f153415b690 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.627+0000 7f1539390640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f152006b4d0 0x7f152006d990 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.627+0000 7f1539390640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340b5110 0x7f153414d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.627+0000 7f1539390640 1 --2- 172.21.15.139:0/3242243971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534072280 0x7f153415c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.627+0000 7f1539390640 1 -- 172.21.15.139:0/3242243971 >> 172.21.15.139:0/3242243971 conn(0x7f153406e280 msgr2=0x7f1534154250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.627+0000 7f1539390640 1 -- 172.21.15.139:0/3242243971 shutdown_connections 2024-04-25T09:59:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:16.627+0000 7f1539390640 1 -- 172.21.15.139:0/3242243971 wait complete. 2024-04-25T09:59:16.642 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:59:16.642 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:59:16.642 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T09:59:16.642 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:59:16.642 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:16.642 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:59:16.642 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:59:16.642 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:59:16.642 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:59:16.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:16.643 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T09:59:16.659 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42342sec preferred_lft 42342sec 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:16.660 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T09:59:16.661 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:16.661 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:59:16.661 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:59:16.661 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T09:59:16.661 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:16.661 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:59:16.661 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:59:16.661 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T09:59:16.661 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:59:16.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:16.662 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:59:16.721 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T09:59:16.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:16.723 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T09:59:16.778 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T09:59:16.778 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:59:16.778 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:59:16.778 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:59:16.778 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:59:16.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:16.779 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T09:59:16.779 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T09:59:16.779 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:59:16.779 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:59:16.779 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:59:16.779 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T09:59:16.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:16.863 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T09:59:16.918 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:59:16.919 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:59:16.919 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:59:16.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:16.919 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T09:59:16.982 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:59:16.982 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:17.038 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:59:17.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:59:17.039 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:17.093 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:59:17.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:17.093 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:59:17.174 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:59:17.174 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:59:17.188 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:59:17.188 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:59:17.188 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T09:59:17.188 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T09:59:17.188 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T09:59:17.189 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:59:17.189 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:59:17.189 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:59:17.189 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:59:17.189 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:59:17.190 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:59:17.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:17.191 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:59:17.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:17.254 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:59:17.311 INFO:teuthology.orchestra.run.smithi139.stdout:client6745 2024-04-25T09:59:17.311 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T09:59:17.311 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:59:17.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:17.366 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-25T09:59:17.422 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T09:59:17.419+0000 7efdd1538640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T09:59:17.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:59:17.424 INFO:teuthology.orchestra.run.smithi139.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-25T09:59:17.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:17.424 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-25T09:59:17.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.561+0000 7fc60c2dd640 1 Processor -- start 2024-04-25T09:59:17.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.561+0000 7fc60c2dd640 1 -- start start 2024-04-25T09:59:17.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.562+0000 7fc60c2dd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc60415da30 0x7fc6040704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.562+0000 7fc60c2dd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc60406b270 0x7fc604167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.562+0000 7fc60c2dd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6040731a0 con 0x7fc604072d60 2024-04-25T09:59:17.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.562+0000 7fc60c2dd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6040cd540 con 0x7fc60415da30 2024-04-25T09:59:17.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.562+0000 7fc60c2dd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6040b3de0 con 0x7fc60406b270 2024-04-25T09:59:17.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.562+0000 7fc60a079640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc60415da30 0x7fc6040704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.562+0000 7fc60a079640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc60415da30 0x7fc6040704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51180/0 (socket says 172.21.15.139:51180) 2024-04-25T09:59:17.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.562+0000 7fc60a079640 1 -- 172.21.15.139:0/235790241 learned_addr learned my addr 172.21.15.139:0/235790241 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:17.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.562+0000 7fc609878640 1 --1- 172.21.15.139:0/235790241 >> v1:172.21.15.139:6789/0 conn(0x7fc604072d60 0x7fc60406ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:36568/0 (socket says 172.21.15.139:36568) 2024-04-25T09:59:17.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.563+0000 7fc60a079640 1 -- 172.21.15.139:0/235790241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc60406b270 msgr2=0x7fc604167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.563+0000 7fc60a079640 1 --2- 172.21.15.139:0/235790241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc60406b270 0x7fc604167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.563+0000 7fc60a079640 1 -- 172.21.15.139:0/235790241 >> v1:172.21.15.139:6789/0 conn(0x7fc604072d60 legacy=0x7fc60406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.563+0000 7fc609077640 1 -- 172.21.15.139:0/235790241 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044495004 0 0) 0x7fc6040731a0 con 0x7fc604072d60 2024-04-25T09:59:17.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.563+0000 7fc60a079640 1 -- 172.21.15.139:0/235790241 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60415de10 con 0x7fc60415da30 2024-04-25T09:59:17.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.563+0000 7fc60a079640 1 --2- 172.21.15.139:0/235790241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60415da30 0x7fc6040704a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc6040cd6c0 tx=0x7fc60002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80a2e7e88bf89bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.563+0000 7fc609077640 1 -- 172.21.15.139:0/235790241 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600036070 con 0x7fc60415da30 2024-04-25T09:59:17.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.564+0000 7fc609077640 1 -- 172.21.15.139:0/235790241 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600031600 con 0x7fc60415da30 2024-04-25T09:59:17.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.564+0000 7fc609077640 1 -- 172.21.15.139:0/235790241 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6000319d0 con 0x7fc60415da30 2024-04-25T09:59:17.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.564+0000 7fc60c2dd640 1 -- 172.21.15.139:0/235790241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60415da30 msgr2=0x7fc6040704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.564+0000 7fc60c2dd640 1 --2- 172.21.15.139:0/235790241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60415da30 0x7fc6040704a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc6040cd6c0 tx=0x7fc60002a730 comp rx=0 tx=0).stop 2024-04-25T09:59:17.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.564+0000 7fc60c2dd640 1 -- 172.21.15.139:0/235790241 shutdown_connections 2024-04-25T09:59:17.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.564+0000 7fc60c2dd640 1 --2- 172.21.15.139:0/235790241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc60406b270 0x7fc604167550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.564+0000 7fc60c2dd640 1 --2- 172.21.15.139:0/235790241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60415da30 0x7fc6040704a0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.564+0000 7fc60c2dd640 1 -- 172.21.15.139:0/235790241 >> 172.21.15.139:0/235790241 conn(0x7fc6040b23f0 msgr2=0x7fc6040b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:17.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.564+0000 7fc60c2dd640 1 -- 172.21.15.139:0/235790241 shutdown_connections 2024-04-25T09:59:17.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.564+0000 7fc60c2dd640 1 -- 172.21.15.139:0/235790241 wait complete. 2024-04-25T09:59:17.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.565+0000 7fc60c2dd640 1 Processor -- start 2024-04-25T09:59:17.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.565+0000 7fc60c2dd640 1 -- start start 2024-04-25T09:59:17.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60c2dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60406b270 0x7fc6040cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60c2dd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc604072d60 0x7fc6041552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60c2dd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc60415da30 0x7fc6041527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60c2dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6040b3b00 con 0x7fc60406b270 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60c2dd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6040b3820 con 0x7fc60415da30 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60c2dd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6040b42a0 con 0x7fc604072d60 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60a079640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60406b270 0x7fc6040cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60a079640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60406b270 0x7fc6040cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51190/0 (socket says 172.21.15.139:51190) 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60a079640 1 -- 172.21.15.139:0/2621651551 learned_addr learned my addr 172.21.15.139:0/2621651551 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc609878640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc604072d60 0x7fc6041552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60a87a640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc60415da30 0x7fc6041527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60a079640 1 -- 172.21.15.139:0/2621651551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc604072d60 msgr2=0x7fc6041552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60a079640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc604072d60 0x7fc6041552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.566+0000 7fc60a079640 1 -- 172.21.15.139:0/2621651551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc60415da30 msgr2=0x7fc6041527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.567+0000 7fc60a079640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc60415da30 0x7fc6041527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.567+0000 7fc60a079640 1 -- 172.21.15.139:0/2621651551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60415e1e0 con 0x7fc60406b270 2024-04-25T09:59:17.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.567+0000 7fc609878640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc604072d60 0x7fc6041552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:17.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.567+0000 7fc60a87a640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc60415da30 0x7fc6041527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:17.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.567+0000 7fc60a079640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60406b270 0x7fc6040cc300 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fc600000f80 tx=0x7fc600004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.567+0000 7fc5faffd640 1 -- 172.21.15.139:0/2621651551 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600041020 con 0x7fc60406b270 2024-04-25T09:59:17.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.567+0000 7fc5faffd640 1 -- 172.21.15.139:0/2621651551 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600007820 con 0x7fc60406b270 2024-04-25T09:59:17.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.567+0000 7fc5faffd640 1 -- 172.21.15.139:0/2621651551 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600036040 con 0x7fc60406b270 2024-04-25T09:59:17.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.567+0000 7fc60c2dd640 1 -- 172.21.15.139:0/2621651551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6041544f0 con 0x7fc60406b270 2024-04-25T09:59:17.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.567+0000 7fc60c2dd640 1 -- 172.21.15.139:0/2621651551 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc604154ab0 con 0x7fc60406b270 2024-04-25T09:59:17.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.568+0000 7fc5faffd640 1 -- 172.21.15.139:0/2621651551 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc600044020 con 0x7fc60406b270 2024-04-25T09:59:17.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.569+0000 7fc5faffd640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc5e806b4d0 0x7fc5e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.569+0000 7fc5faffd640 1 -- 172.21.15.139:0/2621651551 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc6000ab0d0 con 0x7fc60406b270 2024-04-25T09:59:17.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.569+0000 7fc609878640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc5e806b4d0 0x7fc5e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.569+0000 7fc609878640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc5e806b4d0 0x7fc5e806d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc6040687d0 tx=0x7fc5f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.570+0000 7fc60efda740 1 Processor -- start 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.570+0000 7fc60efda740 1 -- start start 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc60efda740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b080 0x5580db41d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc60efda740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b460 0x5580db41ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc60efda740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db3edce0 0x5580db3f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc60efda740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580db356af0 con 0x5580db41b460 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc60efda740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580db2ffde0 con 0x5580db3edce0 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc60efda740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5580db352b90 con 0x5580db41b080 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc609878640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b460 0x5580db41ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc609878640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b460 0x5580db41ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51194/0 (socket says 172.21.15.139:51194) 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc609878640 1 -- 172.21.15.139:0/4009570361 learned_addr learned my addr 172.21.15.139:0/4009570361 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc60a87a640 1 --2- 172.21.15.139:0/4009570361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db3edce0 0x5580db3f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc60a079640 1 --2- 172.21.15.139:0/4009570361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b080 0x5580db41d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc609878640 1 -- 172.21.15.139:0/4009570361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b080 msgr2=0x5580db41d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc609878640 1 --2- 172.21.15.139:0/4009570361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b080 0x5580db41d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc609878640 1 -- 172.21.15.139:0/4009570361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db3edce0 msgr2=0x5580db3f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc609878640 1 --2- 172.21.15.139:0/4009570361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db3edce0 0x5580db3f14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc609878640 1 -- 172.21.15.139:0/4009570361 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580db1377b0 con 0x5580db41b460 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc60a079640 1 --2- 172.21.15.139:0/4009570361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b080 0x5580db41d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc60a87a640 1 --2- 172.21.15.139:0/4009570361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db3edce0 0x5580db3f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc609878640 1 --2- 172.21.15.139:0/4009570361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b460 0x5580db41ddd0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4007360 tx=0x7fc5f4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d42eab57384eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc5e7fff640 1 -- 172.21.15.139:0/4009570361 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f400d9a0 con 0x5580db41b460 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc5e7fff640 1 -- 172.21.15.139:0/4009570361 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f400db60 con 0x5580db41b460 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.571+0000 7fc5e7fff640 1 -- 172.21.15.139:0/4009570361 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4020450 con 0x5580db41b460 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.572+0000 7fc60efda740 1 -- 172.21.15.139:0/4009570361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b460 msgr2=0x5580db41ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.572+0000 7fc60efda740 1 --2- 172.21.15.139:0/4009570361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b460 0x5580db41ddd0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4007360 tx=0x7fc5f4010000 comp rx=0 tx=0).stop 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.572+0000 7fc60efda740 1 -- 172.21.15.139:0/4009570361 shutdown_connections 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.572+0000 7fc60efda740 1 --2- 172.21.15.139:0/4009570361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b460 0x5580db41ddd0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.572+0000 7fc60efda740 1 --2- 172.21.15.139:0/4009570361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db3edce0 0x5580db3f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.572+0000 7fc60efda740 1 --2- 172.21.15.139:0/4009570361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b080 0x5580db41d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.572+0000 7fc60efda740 1 -- 172.21.15.139:0/4009570361 >> 172.21.15.139:0/4009570361 conn(0x5580db3eff50 msgr2=0x5580db438890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.572+0000 7fc60efda740 1 -- 172.21.15.139:0/4009570361 shutdown_connections 2024-04-25T09:59:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.572+0000 7fc60efda740 1 -- 172.21.15.139:0/4009570361 wait complete. 2024-04-25T09:59:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.573+0000 7fc60efda740 1 -- start start 2024-04-25T09:59:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.573+0000 7fc60efda740 1 Processor -- start 2024-04-25T09:59:17.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60efda740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db4116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60efda740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db41b460 0x5580db4140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60efda740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db412250 0x5580db4145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60efda740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580db3f3100 con 0x5580db41b080 2024-04-25T09:59:17.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60efda740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580db2f53e0 con 0x5580db41b460 2024-04-25T09:59:17.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60efda740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5580db3f33e0 con 0x5580db412250 2024-04-25T09:59:17.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60efda740 10 client.0 _ll_register_callbacks cb 0x5580db312d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:17.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60a079640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db4116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60a079640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db4116b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51196/0 (socket says 172.21.15.139:51196) 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60a079640 1 -- 172.21.15.139:0/472971946 learned_addr learned my addr 172.21.15.139:0/472971946 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc609878640 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db41b460 0x5580db4140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60a87a640 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db412250 0x5580db4145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60a079640 1 -- 172.21.15.139:0/472971946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db412250 msgr2=0x5580db4145f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60a079640 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db412250 0x5580db4145f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60a079640 1 -- 172.21.15.139:0/472971946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db41b460 msgr2=0x5580db4140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60a079640 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db41b460 0x5580db4140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60a079640 1 -- 172.21.15.139:0/472971946 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580db197f40 con 0x5580db41b080 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc609878640 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db41b460 0x5580db4140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60a079640 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db4116b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fc600007090 tx=0x7fc6000095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e9b558235cd7ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc5e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc5e57fa640 1 -- 172.21.15.139:0/472971946 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6000a6020 con 0x5580db41b080 2024-04-25T09:59:17.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc5e57fa640 1 -- 172.21.15.139:0/472971946 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc600081c40 con 0x5580db41b080 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc60efda740 1 -- 172.21.15.139:0/472971946 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5f400db60 con 0x5580db41b080 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.574+0000 7fc5e57fa640 1 -- 172.21.15.139:0/472971946 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc600098050 con 0x5580db41b080 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc60efda740 10 client.7881 fetch_fsmap learned FSMap version 56 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc60efda740 1 -- 172.21.15.139:0/472971946 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5580db02b0a0 con 0x5580db41b080 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc5e57fa640 1 -- 172.21.15.139:0/472971946 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f400db60 con 0x5580db41b080 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc5e57fa640 1 -- 172.21.15.139:0/472971946 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc600078c10 con 0x5580db41b080 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc60efda740 10 client.7881 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc60efda740 10 client.7881 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc60efda740 20 client.7881 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc60efda740 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5580db4399c0 0x5580db439e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc60efda740 4 client.7881 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc60efda740 1 -- 172.21.15.139:0/472971946 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580db364030 con 0x5580db4399c0 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc609878640 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5580db4399c0 0x5580db439e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc5e57fa640 10 client.7881 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:17.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.575+0000 7fc609878640 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5580db4399c0 0x5580db439e20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662901fd83e0fcd0 server_cookie=75f9e5da7add01eb in_seq=0 out_seq=0 2024-04-25T09:59:17.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc5e57fa640 1 -- 172.21.15.139:0/472971946 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5580db364030 con 0x5580db4399c0 2024-04-25T09:59:17.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc5e57fa640 10 client.7881 handle_command_reply: tid=0 2024-04-25T09:59:17.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc60efda740 1 client.7881 shutdown 2024-04-25T09:59:17.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc60efda740 1 -- 172.21.15.139:0/472971946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 msgr2=0x5580db4116b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc60efda740 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db4116b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fc600007090 tx=0x7fc6000095d0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc60efda740 1 -- 172.21.15.139:0/472971946 shutdown_connections 2024-04-25T09:59:17.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc60efda740 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5580db4399c0 0x5580db439e20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc60efda740 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db412250 0x5580db4145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc60efda740 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db41b460 0x5580db4140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc5e57fa640 0 client.7881 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:17.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc60efda740 1 --2- 172.21.15.139:0/472971946 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db4116b0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc60efda740 1 -- 172.21.15.139:0/472971946 >> 172.21.15.139:0/472971946 conn(0x5580db3edce0 msgr2=0x5580db419700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:17.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc60efda740 1 -- 172.21.15.139:0/472971946 shutdown_connections 2024-04-25T09:59:17.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc60efda740 1 -- 172.21.15.139:0/472971946 wait complete. 2024-04-25T09:59:17.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.576+0000 7fc60efda740 20 client.7881 trim_cache size 0 max 16384 2024-04-25T09:59:17.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.591+0000 7fc60efda740 1 Processor -- start 2024-04-25T09:59:17.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.591+0000 7fc60efda740 1 -- start start 2024-04-25T09:59:17.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc60efda740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db412250 0x5580db4140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc60efda740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db414610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc60efda740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b460 0x5580db419910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc60efda740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc604200470 con 0x5580db41b080 2024-04-25T09:59:17.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc60efda740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6041ff630 con 0x5580db412250 2024-04-25T09:59:17.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc60efda740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60415f6b0 con 0x5580db41b460 2024-04-25T09:59:17.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc609878640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db414610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc609878640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db414610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51208/0 (socket says 172.21.15.139:51208) 2024-04-25T09:59:17.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc609878640 1 -- 172.21.15.139:0/242472220 learned_addr learned my addr 172.21.15.139:0/242472220 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:17.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc60a079640 1 --2- 172.21.15.139:0/242472220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db412250 0x5580db4140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc60a87a640 1 --2- 172.21.15.139:0/242472220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b460 0x5580db419910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc609878640 1 -- 172.21.15.139:0/242472220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b460 msgr2=0x5580db419910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc609878640 1 --2- 172.21.15.139:0/242472220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b460 0x5580db419910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc609878640 1 -- 172.21.15.139:0/242472220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db412250 msgr2=0x5580db4140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc609878640 1 --2- 172.21.15.139:0/242472220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db412250 0x5580db4140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc609878640 1 -- 172.21.15.139:0/242472220 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580db41da80 con 0x5580db41b080 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc60a87a640 1 --2- 172.21.15.139:0/242472220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b460 0x5580db419910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.592+0000 7fc609878640 1 --2- 172.21.15.139:0/242472220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db414610 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4005be0 tx=0x7fc5f4007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1c00f888981384 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.593+0000 7fc5e77fe640 1 -- 172.21.15.139:0/242472220 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4035070 con 0x5580db41b080 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.593+0000 7fc5e77fe640 1 -- 172.21.15.139:0/242472220 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f400d9a0 con 0x5580db41b080 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.593+0000 7fc5e77fe640 1 -- 172.21.15.139:0/242472220 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f402f250 con 0x5580db41b080 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.593+0000 7fc60efda740 1 -- 172.21.15.139:0/242472220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 msgr2=0x5580db414610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.593+0000 7fc60efda740 1 --2- 172.21.15.139:0/242472220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db414610 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4005be0 tx=0x7fc5f4007000 comp rx=0 tx=0).stop 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.593+0000 7fc60efda740 1 -- 172.21.15.139:0/242472220 shutdown_connections 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.593+0000 7fc60efda740 1 --2- 172.21.15.139:0/242472220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db414610 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.593+0000 7fc60efda740 1 --2- 172.21.15.139:0/242472220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b460 0x5580db419910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.593+0000 7fc60efda740 1 --2- 172.21.15.139:0/242472220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db412250 0x5580db4140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.593+0000 7fc60efda740 1 -- 172.21.15.139:0/242472220 >> 172.21.15.139:0/242472220 conn(0x5580db3edce0 msgr2=0x5580db4114c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.593+0000 7fc60efda740 1 -- 172.21.15.139:0/242472220 shutdown_connections 2024-04-25T09:59:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.593+0000 7fc60efda740 1 -- 172.21.15.139:0/242472220 wait complete. 2024-04-25T09:59:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.594+0000 7fc60efda740 1 -- start start 2024-04-25T09:59:17.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.594+0000 7fc60efda740 1 Processor -- start 2024-04-25T09:59:17.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60efda740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db412250 0x5580db4196e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60efda740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db4140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60efda740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b460 0x5580db414610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60efda740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580db3f2af0 con 0x5580db41b080 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60efda740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6041ff030 con 0x5580db412250 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60efda740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5580db3e9790 con 0x5580db41b460 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60efda740 10 client.0 _ll_register_callbacks cb 0x7fc600009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc609878640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db4140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc609878640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db4140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51220/0 (socket says 172.21.15.139:51220) 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc609878640 1 -- 172.21.15.139:0/2444734836 learned_addr learned my addr 172.21.15.139:0/2444734836 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60a079640 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db412250 0x5580db4196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60a87a640 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b460 0x5580db414610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc609878640 1 -- 172.21.15.139:0/2444734836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b460 msgr2=0x5580db414610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc609878640 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b460 0x5580db414610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc609878640 1 -- 172.21.15.139:0/2444734836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db412250 msgr2=0x5580db4196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc609878640 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db412250 0x5580db4196e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc609878640 1 -- 172.21.15.139:0/2444734836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580db4215d0 con 0x5580db41b080 2024-04-25T09:59:17.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60a87a640 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b460 0x5580db414610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc609878640 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db4140d0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4033040 tx=0x7fc5f40012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7525f6e5ca52d0e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc5e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc5e57fa640 1 -- 172.21.15.139:0/2444734836 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f403e060 con 0x5580db41b080 2024-04-25T09:59:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc5e57fa640 1 -- 172.21.15.139:0/2444734836 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc5f400d7a0 con 0x5580db41b080 2024-04-25T09:59:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60efda740 1 -- 172.21.15.139:0/2444734836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5f400d9a0 con 0x5580db41b080 2024-04-25T09:59:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc5e57fa640 1 -- 172.21.15.139:0/2444734836 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5f4044050 con 0x5580db41b080 2024-04-25T09:59:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60efda740 10 client.7887 fetch_fsmap learned FSMap version 56 2024-04-25T09:59:17.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.595+0000 7fc60efda740 1 -- 172.21.15.139:0/2444734836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5580db421310 con 0x5580db41b080 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc5e57fa640 1 -- 172.21.15.139:0/2444734836 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f400d9a0 con 0x5580db41b080 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc5e57fa640 1 -- 172.21.15.139:0/2444734836 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc5f4047020 con 0x5580db41b080 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc60efda740 10 client.7887 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc60efda740 10 client.7887 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc60efda740 20 client.7887 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc60efda740 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5580db4a69c0 0x5580db439da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc60efda740 4 client.7887 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc60efda740 1 -- 172.21.15.139:0/2444734836 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) v1 -- 0x5580db35e380 con 0x5580db4a69c0 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc60a079640 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5580db4a69c0 0x5580db439da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc5e57fa640 10 client.7887 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc60a079640 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5580db4a69c0 0x5580db439da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dc5b3a20612957d server_cookie=be5f4a1f4a7c5c12 in_seq=0 out_seq=0 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc5e57fa640 1 -- 172.21.15.139:0/2444734836 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5580db35e380 con 0x5580db4a69c0 2024-04-25T09:59:17.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc5e57fa640 10 client.7887 handle_command_reply: tid=0 2024-04-25T09:59:17.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.596+0000 7fc60efda740 1 client.7887 shutdown 2024-04-25T09:59:17.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60efda740 1 -- 172.21.15.139:0/2444734836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 msgr2=0x5580db4140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60efda740 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db4140d0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4033040 tx=0x7fc5f40012b0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60efda740 1 -- 172.21.15.139:0/2444734836 shutdown_connections 2024-04-25T09:59:17.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60efda740 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5580db4a69c0 0x5580db439da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60efda740 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5580db41b080 0x5580db4140d0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60efda740 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5580db41b460 0x5580db414610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc5e57fa640 0 client.7887 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:17.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60efda740 1 --2- 172.21.15.139:0/2444734836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5580db412250 0x5580db4196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60efda740 1 -- 172.21.15.139:0/2444734836 >> 172.21.15.139:0/2444734836 conn(0x5580db3edce0 msgr2=0x5580db410910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:17.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60efda740 1 -- 172.21.15.139:0/2444734836 shutdown_connections 2024-04-25T09:59:17.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60efda740 1 -- 172.21.15.139:0/2444734836 wait complete. 2024-04-25T09:59:17.608 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T09:59:17.597+0000 7efdd1538640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-25T09:59:08.119440+0000 1=1+0) n(v0 rc2024-04-25T09:59:08.238437+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x557cea233b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T09:59:08.119440+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T09:59:08.238437+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T09:59:17.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60efda740 20 client.7887 trim_cache size 0 max 16384 2024-04-25T09:59:17.608 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:59:17.609 INFO:teuthology.orchestra.run.smithi139.stdout: "return_code": 0, 2024-04-25T09:59:17.609 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_tag": "bd10579d-d66c-4161-b9c0-3723ce1af2e1", 2024-04-25T09:59:17.609 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": "asynchronous" 2024-04-25T09:59:17.609 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60c2dd640 1 -- 172.21.15.139:0/2621651551 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc5e806b4d0 msgr2=0x7fc5e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60c2dd640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc5e806b4d0 0x7fc5e806d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc6040687d0 tx=0x7fc5f4009040 comp rx=0 tx=0).stop 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60c2dd640 1 -- 172.21.15.139:0/2621651551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60406b270 msgr2=0x7fc6040cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60c2dd640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60406b270 0x7fc6040cc300 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fc600000f80 tx=0x7fc600004a40 comp rx=0 tx=0).stop 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60c2dd640 1 -- 172.21.15.139:0/2621651551 shutdown_connections 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60c2dd640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc5e806b4d0 0x7fc5e806d990 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60c2dd640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc60415da30 0x7fc6041527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60c2dd640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc604072d60 0x7fc6041552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60c2dd640 1 --2- 172.21.15.139:0/2621651551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc60406b270 0x7fc6040cc300 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60c2dd640 1 -- 172.21.15.139:0/2621651551 >> 172.21.15.139:0/2621651551 conn(0x7fc6040b23f0 msgr2=0x7fc6040cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60c2dd640 1 -- 172.21.15.139:0/2621651551 shutdown_connections 2024-04-25T09:59:17.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.597+0000 7fc60c2dd640 1 -- 172.21.15.139:0/2621651551 wait complete. 2024-04-25T09:59:17.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:17.612 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:59:17.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.707+0000 7f1f3e7c1640 1 Processor -- start 2024-04-25T09:59:17.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.707+0000 7f1f3e7c1640 1 -- start start 2024-04-25T09:59:17.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.708+0000 7f1f3e7c1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1f3815dbf0 0x7f1f381617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.708+0000 7f1f3e7c1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1f380b9370 0x7f1f380ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.708+0000 7f1f3e7c1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f380cec80 0x7f1f3806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.708+0000 7f1f3e7c1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f380d1ee0 con 0x7f1f3815dbf0 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.708+0000 7f1f3e7c1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f38161dc0 con 0x7f1f380b9370 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.708+0000 7f1f3e7c1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f381620a0 con 0x7f1f380cec80 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.708+0000 7f1f37fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1f3815dbf0 0x7f1f381617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.708+0000 7f1f37fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1f3815dbf0 0x7f1f381617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51224/0 (socket says 172.21.15.139:51224) 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.708+0000 7f1f37fff640 1 -- 172.21.15.139:0/1468980507 learned_addr learned my addr 172.21.15.139:0/1468980507 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.708+0000 7f1f377fe640 1 --2- 172.21.15.139:0/1468980507 >> v2:172.21.15.150:3300/0 conn(0x7f1f380b9370 0x7f1f380ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.709+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/1468980507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f380cec80 0x7f1f3806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.709+0000 7f1f37fff640 1 -- 172.21.15.139:0/1468980507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f380cec80 msgr2=0x7f1f3806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.709+0000 7f1f37fff640 1 --2- 172.21.15.139:0/1468980507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f380cec80 0x7f1f3806c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.709+0000 7f1f37fff640 1 -- 172.21.15.139:0/1468980507 >> v2:172.21.15.150:3300/0 conn(0x7f1f380b9370 msgr2=0x7f1f380ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.709+0000 7f1f37fff640 1 --2- 172.21.15.139:0/1468980507 >> v2:172.21.15.150:3300/0 conn(0x7f1f380b9370 0x7f1f380ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.709+0000 7f1f37fff640 1 -- 172.21.15.139:0/1468980507 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f38162380 con 0x7f1f3815dbf0 2024-04-25T09:59:17.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.709+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/1468980507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f380cec80 0x7f1f3806c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:17.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.709+0000 7f1f37fff640 1 --2- 172.21.15.139:0/1468980507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f3815dbf0 0x7f1f381617e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f1f3808c420 tx=0x7f1f2802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ad2e00809d5edc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.709+0000 7f1f36ffd640 1 -- 172.21.15.139:0/1468980507 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28037070 con 0x7f1f3815dbf0 2024-04-25T09:59:17.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.709+0000 7f1f36ffd640 1 -- 172.21.15.139:0/1468980507 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f28032440 con 0x7f1f3815dbf0 2024-04-25T09:59:17.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.709+0000 7f1f36ffd640 1 -- 172.21.15.139:0/1468980507 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28032810 con 0x7f1f3815dbf0 2024-04-25T09:59:17.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.710+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/1468980507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f3815dbf0 msgr2=0x7f1f381617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.710+0000 7f1f3e7c1640 1 --2- 172.21.15.139:0/1468980507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f3815dbf0 0x7f1f381617e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f1f3808c420 tx=0x7f1f2802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.710+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/1468980507 shutdown_connections 2024-04-25T09:59:17.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.710+0000 7f1f3e7c1640 1 --2- 172.21.15.139:0/1468980507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f380cec80 0x7f1f3806c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.710+0000 7f1f3e7c1640 1 --2- 172.21.15.139:0/1468980507 >> v2:172.21.15.150:3300/0 conn(0x7f1f380b9370 0x7f1f380ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.710+0000 7f1f3e7c1640 1 --2- 172.21.15.139:0/1468980507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f3815dbf0 0x7f1f381617e0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.710+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/1468980507 >> 172.21.15.139:0/1468980507 conn(0x7f1f380b5480 msgr2=0x7f1f380b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:17.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.710+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/1468980507 shutdown_connections 2024-04-25T09:59:17.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.710+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/1468980507 wait complete. 2024-04-25T09:59:17.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.711+0000 7f1f3e7c1640 1 Processor -- start 2024-04-25T09:59:17.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.711+0000 7f1f3e7c1640 1 -- start start 2024-04-25T09:59:17.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f3e7c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f380b9370 0x7f1f3814e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f3e7c1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f380cec80 0x7f1f3815c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f3e7c1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3815dbf0 0x7f1f3815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f3e7c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3815dfd0 con 0x7f1f380b9370 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f3e7c1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380b97d0 con 0x7f1f3815dbf0 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f3e7c1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380aa350 con 0x7f1f380cec80 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f380b9370 0x7f1f3814e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f380b9370 0x7f1f3814e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51236/0 (socket says 172.21.15.139:51236) 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f37fff640 1 -- 172.21.15.139:0/4141241831 learned_addr learned my addr 172.21.15.139:0/4141241831 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3815dbf0 0x7f1f3815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f377fe640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f380cec80 0x7f1f3815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f37fff640 1 -- 172.21.15.139:0/4141241831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f380cec80 msgr2=0x7f1f3815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f37fff640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f380cec80 0x7f1f3815c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f37fff640 1 -- 172.21.15.139:0/4141241831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3815dbf0 msgr2=0x7f1f3815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f37fff640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3815dbf0 0x7f1f3815c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.712+0000 7f1f37fff640 1 -- 172.21.15.139:0/4141241831 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f380aa5a0 con 0x7f1f380b9370 2024-04-25T09:59:17.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.713+0000 7f1f37fff640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f380b9370 0x7f1f3814e690 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1f2802a220 tx=0x7f1f280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.713+0000 7f1f34ff9640 1 -- 172.21.15.139:0/4141241831 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28041020 con 0x7f1f380b9370 2024-04-25T09:59:17.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.713+0000 7f1f34ff9640 1 -- 172.21.15.139:0/4141241831 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f280306c0 con 0x7f1f380b9370 2024-04-25T09:59:17.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.713+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/4141241831 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f380aa880 con 0x7f1f380b9370 2024-04-25T09:59:17.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.713+0000 7f1f34ff9640 1 -- 172.21.15.139:0/4141241831 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28037040 con 0x7f1f380b9370 2024-04-25T09:59:17.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.713+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/4141241831 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f380aae40 con 0x7f1f380b9370 2024-04-25T09:59:17.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.714+0000 7f1f34ff9640 1 -- 172.21.15.139:0/4141241831 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f28044020 con 0x7f1f380b9370 2024-04-25T09:59:17.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.715+0000 7f1f34ff9640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1f2406b4d0 0x7f1f2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.715+0000 7f1f34ff9640 1 -- 172.21.15.139:0/4141241831 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1f280aaa30 con 0x7f1f380b9370 2024-04-25T09:59:17.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.715+0000 7f1f377fe640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1f2406b4d0 0x7f1f2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.715+0000 7f1f377fe640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1f2406b4d0 0x7f1f2406d990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f381597d0 tx=0x7f1f2c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.716+0000 7f1f4159f740 1 Processor -- start 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.716+0000 7f1f4159f740 1 -- start start 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.716+0000 7f1f4159f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7b10 0x55c017aaa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.716+0000 7f1f4159f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7ef0 0x55c017aaa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.716+0000 7f1f4159f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a7a770 0x55c017a7df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.716+0000 7f1f4159f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0179e3da0 con 0x55c017a7a770 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.716+0000 7f1f4159f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01798cb80 con 0x55c017aa7b10 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.716+0000 7f1f4159f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0179dfe40 con 0x55c017aa7ef0 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.717+0000 7f1f3cd5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a7a770 0x55c017a7df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.717+0000 7f1f3cd5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a7a770 0x55c017a7df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51240/0 (socket says 172.21.15.139:51240) 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.717+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/3770832036 learned_addr learned my addr 172.21.15.139:0/3770832036 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.717+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/3770832036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7ef0 msgr2=0x55c017aaa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.717+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/3770832036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7ef0 0x55c017aaa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.717+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/3770832036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7b10 msgr2=0x55c017aaa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.717+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/3770832036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7b10 0x55c017aaa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.717+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/3770832036 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c017996f10 con 0x55c017a7a770 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.717+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/3770832036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a7a770 0x55c017a7df80 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000f4a0 tx=0x7f1f3000f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a8238e84ce833b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.717+0000 7f1f1dffb640 1 -- 172.21.15.139:0/3770832036 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3001c070 con 0x55c017a7a770 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.717+0000 7f1f1dffb640 1 -- 172.21.15.139:0/3770832036 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3000d6c0 con 0x55c017a7a770 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.717+0000 7f1f1dffb640 1 -- 172.21.15.139:0/3770832036 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3000da70 con 0x55c017a7a770 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.718+0000 7f1f4159f740 1 -- 172.21.15.139:0/3770832036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a7a770 msgr2=0x55c017a7df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.718+0000 7f1f4159f740 1 --2- 172.21.15.139:0/3770832036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a7a770 0x55c017a7df80 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000f4a0 tx=0x7f1f3000f960 comp rx=0 tx=0).stop 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.718+0000 7f1f4159f740 1 -- 172.21.15.139:0/3770832036 shutdown_connections 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.718+0000 7f1f4159f740 1 --2- 172.21.15.139:0/3770832036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a7a770 0x55c017a7df80 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.718+0000 7f1f4159f740 1 --2- 172.21.15.139:0/3770832036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7ef0 0x55c017aaa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.718+0000 7f1f4159f740 1 --2- 172.21.15.139:0/3770832036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7b10 0x55c017aaa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.718+0000 7f1f4159f740 1 -- 172.21.15.139:0/3770832036 >> 172.21.15.139:0/3770832036 conn(0x55c017a7c9e0 msgr2=0x55c017ac5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:17.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.718+0000 7f1f4159f740 1 -- 172.21.15.139:0/3770832036 shutdown_connections 2024-04-25T09:59:17.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.718+0000 7f1f4159f740 1 -- 172.21.15.139:0/3770832036 wait complete. 2024-04-25T09:59:17.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.718+0000 7f1f4159f740 1 -- start start 2024-04-25T09:59:17.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.719+0000 7f1f4159f740 1 Processor -- start 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.719+0000 7f1f4159f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7b10 0x55c017aa1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.719+0000 7f1f4159f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7ef0 0x55c017ac77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.719+0000 7f1f4159f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a9e830 0x55c017ac7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.719+0000 7f1f4159f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c017a7fb90 con 0x55c017a9e830 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.719+0000 7f1f4159f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0179824b0 con 0x55c017aa7ef0 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.719+0000 7f1f4159f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c017a7fe70 con 0x55c017aa7b10 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.719+0000 7f1f4159f740 10 client.0 _ll_register_callbacks cb 0x55c0179c2180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.719+0000 7f1f3cd5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a9e830 0x55c017ac7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.719+0000 7f1f3cd5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a9e830 0x55c017ac7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51250/0 (socket says 172.21.15.139:51250) 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.719+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/415483530 learned_addr learned my addr 172.21.15.139:0/415483530 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f37fff640 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7b10 0x55c017aa1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f377fe640 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7ef0 0x55c017ac77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/415483530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7b10 msgr2=0x55c017aa1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7b10 0x55c017aa1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/415483530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7ef0 msgr2=0x55c017ac77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7ef0 0x55c017ac77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/415483530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0177c27d0 con 0x55c017a9e830 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a9e830 0x55c017ac7d30 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x55c017a82070 tx=0x7f1f30004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a375d7518d41a6eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f16ffd640 1 -- 172.21.15.139:0/415483530 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3001c040 con 0x55c017a9e830 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f4159f740 1 -- 172.21.15.139:0/415483530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f3000d6c0 con 0x55c017a9e830 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f16ffd640 1 -- 172.21.15.139:0/415483530 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1f30003990 con 0x55c017a9e830 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f16ffd640 1 -- 172.21.15.139:0/415483530 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f3002e050 con 0x55c017a9e830 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f4159f740 10 client.7899 fetch_fsmap learned FSMap version 56 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f4159f740 1 -- 172.21.15.139:0/415483530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55c0176f83f0 con 0x55c017a9e830 2024-04-25T09:59:17.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f16ffd640 1 -- 172.21.15.139:0/415483530 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3000d6c0 con 0x55c017a9e830 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f16ffd640 1 -- 172.21.15.139:0/415483530 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1f30031020 con 0x55c017a9e830 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f4159f740 10 client.7899 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f4159f740 10 client.7899 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f4159f740 20 client.7899 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f4159f740 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55c017a9df70 0x55c017ae2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f4159f740 4 client.7899 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.720+0000 7f1f4159f740 1 -- 172.21.15.139:0/415483530 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0179f12e0 con 0x55c017a9df70 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.721+0000 7f1f37fff640 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55c017a9df70 0x55c017ae2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.721+0000 7f1f16ffd640 10 client.7899 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.721+0000 7f1f37fff640 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55c017a9df70 0x55c017ae2700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa62aa02dd8880d6 server_cookie=43ced6ff71d03082 in_seq=0 out_seq=0 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.721+0000 7f1f16ffd640 1 -- 172.21.15.139:0/415483530 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c0179f12e0 con 0x55c017a9df70 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.721+0000 7f1f16ffd640 10 client.7899 handle_command_reply: tid=0 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.721+0000 7f1f4159f740 1 client.7899 shutdown 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.722+0000 7f1f4159f740 1 -- 172.21.15.139:0/415483530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a9e830 msgr2=0x55c017ac7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.722+0000 7f1f4159f740 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a9e830 0x55c017ac7d30 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x55c017a82070 tx=0x7f1f30004ab0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.722+0000 7f1f4159f740 1 -- 172.21.15.139:0/415483530 shutdown_connections 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.722+0000 7f1f4159f740 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55c017a9df70 0x55c017ae2700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.722+0000 7f1f4159f740 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017a9e830 0x55c017ac7d30 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.722+0000 7f1f4159f740 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7ef0 0x55c017ac77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.722+0000 7f1f4159f740 1 --2- 172.21.15.139:0/415483530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7b10 0x55c017aa1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.722+0000 7f1f4159f740 1 -- 172.21.15.139:0/415483530 >> 172.21.15.139:0/415483530 conn(0x55c017a7a770 msgr2=0x55c017a7b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.722+0000 7f1f16ffd640 0 client.7899 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.722+0000 7f1f4159f740 1 -- 172.21.15.139:0/415483530 shutdown_connections 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.722+0000 7f1f4159f740 1 -- 172.21.15.139:0/415483530 wait complete. 2024-04-25T09:59:17.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.722+0000 7f1f4159f740 20 client.7899 trim_cache size 0 max 16384 2024-04-25T09:59:17.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.737+0000 7f1f4159f740 1 Processor -- start 2024-04-25T09:59:17.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.737+0000 7f1f4159f740 1 -- start start 2024-04-25T09:59:17.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.737+0000 7f1f4159f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017a9e830 0x55c017aa6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.737+0000 7f1f4159f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7b10 0x55c017aa7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.737+0000 7f1f4159f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 0x55c017a79600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.737+0000 7f1f4159f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f38200610 con 0x55c017aa7ef0 2024-04-25T09:59:17.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.737+0000 7f1f4159f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f381ff7b0 con 0x55c017aa7b10 2024-04-25T09:59:17.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.737+0000 7f1f4159f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f381febb0 con 0x55c017a9e830 2024-04-25T09:59:17.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.738+0000 7f1f3cd5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 0x55c017a79600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.738+0000 7f1f3cd5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 0x55c017a79600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51264/0 (socket says 172.21.15.139:51264) 2024-04-25T09:59:17.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.738+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/2091445594 learned_addr learned my addr 172.21.15.139:0/2091445594 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:17.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.738+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/2091445594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017a9e830 msgr2=0x55c017aa6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.738+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/2091445594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017a9e830 0x55c017aa6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.738+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/2091445594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7b10 msgr2=0x55c017aa7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.738+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/2091445594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7b10 0x55c017aa7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.738+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/2091445594 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0176b80f0 con 0x55c017aa7ef0 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.738+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/2091445594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 0x55c017a79600 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f1f30000f80 tx=0x7f1f30020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173a91d02f8c3140 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.738+0000 7f1f1d7fa640 1 -- 172.21.15.139:0/2091445594 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3001c030 con 0x55c017aa7ef0 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.738+0000 7f1f1d7fa640 1 -- 172.21.15.139:0/2091445594 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f30026070 con 0x55c017aa7ef0 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.738+0000 7f1f1d7fa640 1 -- 172.21.15.139:0/2091445594 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3000d6c0 con 0x55c017aa7ef0 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.739+0000 7f1f4159f740 1 -- 172.21.15.139:0/2091445594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 msgr2=0x55c017a79600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.739+0000 7f1f4159f740 1 --2- 172.21.15.139:0/2091445594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 0x55c017a79600 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f1f30000f80 tx=0x7f1f30020000 comp rx=0 tx=0).stop 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.739+0000 7f1f4159f740 1 -- 172.21.15.139:0/2091445594 shutdown_connections 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.739+0000 7f1f4159f740 1 --2- 172.21.15.139:0/2091445594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 0x55c017a79600 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.739+0000 7f1f4159f740 1 --2- 172.21.15.139:0/2091445594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017aa7b10 0x55c017aa7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.739+0000 7f1f4159f740 1 --2- 172.21.15.139:0/2091445594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017a9e830 0x55c017aa6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.739+0000 7f1f4159f740 1 -- 172.21.15.139:0/2091445594 >> 172.21.15.139:0/2091445594 conn(0x55c017a7a770 msgr2=0x55c017aa1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.739+0000 7f1f4159f740 1 -- 172.21.15.139:0/2091445594 shutdown_connections 2024-04-25T09:59:17.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.739+0000 7f1f4159f740 1 -- 172.21.15.139:0/2091445594 wait complete. 2024-04-25T09:59:17.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.739+0000 7f1f4159f740 1 -- start start 2024-04-25T09:59:17.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.739+0000 7f1f4159f740 1 Processor -- start 2024-04-25T09:59:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.740+0000 7f1f4159f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017a9e830 0x55c017a7ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.740+0000 7f1f4159f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7b10 0x55c017a7ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.740+0000 7f1f4159f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 0x55c017aa6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.740+0000 7f1f4159f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f381ff1b0 con 0x55c017aa7ef0 2024-04-25T09:59:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.740+0000 7f1f4159f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c017a9ff20 con 0x55c017a9e830 2024-04-25T09:59:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.740+0000 7f1f4159f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c017ac5940 con 0x55c017aa7b10 2024-04-25T09:59:17.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.740+0000 7f1f4159f740 10 client.0 _ll_register_callbacks cb 0x7f1f30007a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.740+0000 7f1f3cd5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 0x55c017aa6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.740+0000 7f1f3cd5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 0x55c017aa6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:51278/0 (socket says 172.21.15.139:51278) 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.740+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/3427586005 learned_addr learned my addr 172.21.15.139:0/3427586005 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/3427586005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7b10 msgr2=0x55c017a7ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/3427586005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7b10 0x55c017a7ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/3427586005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017a9e830 msgr2=0x55c017a7ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/3427586005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017a9e830 0x55c017a7ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f3cd5e640 1 -- 172.21.15.139:0/3427586005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c017b2f370 con 0x55c017aa7ef0 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f3cd5e640 1 --2- 172.21.15.139:0/3427586005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 0x55c017aa6d30 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000dc40 tx=0x7f1f300076e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e9f39be54ba04f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f16ffd640 1 -- 172.21.15.139:0/3427586005 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30042060 con 0x55c017aa7ef0 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f16ffd640 1 -- 172.21.15.139:0/3427586005 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1f30021620 con 0x55c017aa7ef0 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f4159f740 1 -- 172.21.15.139:0/3427586005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f30026070 con 0x55c017aa7ef0 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f16ffd640 1 -- 172.21.15.139:0/3427586005 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f30048050 con 0x55c017aa7ef0 2024-04-25T09:59:17.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f4159f740 10 client.7905 fetch_fsmap learned FSMap version 56 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f4159f740 1 -- 172.21.15.139:0/3427586005 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55c017a80090 con 0x55c017aa7ef0 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f16ffd640 1 -- 172.21.15.139:0/3427586005 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f30026070 con 0x55c017aa7ef0 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f16ffd640 1 -- 172.21.15.139:0/3427586005 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1f3004b020 con 0x55c017aa7ef0 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f4159f740 10 client.7905 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f4159f740 10 client.7905 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f4159f740 20 client.7905 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f4159f740 1 --2- 172.21.15.139:0/3427586005 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55c017aabbf0 0x55c017aa7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f4159f740 4 client.7905 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f4159f740 1 -- 172.21.15.139:0/3427586005 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c0179eb630 con 0x55c017aabbf0 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.741+0000 7f1f37fff640 1 --2- 172.21.15.139:0/3427586005 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55c017aabbf0 0x55c017aa7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f16ffd640 10 client.7905 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f37fff640 1 --2- 172.21.15.139:0/3427586005 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55c017aabbf0 0x55c017aa7270 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8124a1e41370c7eb server_cookie=681830cbf7f831fd in_seq=0 out_seq=0 2024-04-25T09:59:17.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f16ffd640 1 -- 172.21.15.139:0/3427586005 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0x55c0179eb630 con 0x55c017aabbf0 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f16ffd640 10 client.7905 handle_command_reply: tid=0 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f4159f740 1 client.7905 shutdown 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f4159f740 1 -- 172.21.15.139:0/3427586005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 msgr2=0x55c017aa6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f4159f740 1 --2- 172.21.15.139:0/3427586005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 0x55c017aa6d30 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000dc40 tx=0x7f1f300076e0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f4159f740 1 -- 172.21.15.139:0/3427586005 shutdown_connections 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f4159f740 1 --2- 172.21.15.139:0/3427586005 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55c017aabbf0 0x55c017aa7270 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f4159f740 1 --2- 172.21.15.139:0/3427586005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c017aa7ef0 0x55c017aa6d30 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f4159f740 1 --2- 172.21.15.139:0/3427586005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55c017aa7b10 0x55c017a7ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f4159f740 1 --2- 172.21.15.139:0/3427586005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c017a9e830 0x55c017a7ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f16ffd640 0 client.7905 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f4159f740 1 -- 172.21.15.139:0/3427586005 >> 172.21.15.139:0/3427586005 conn(0x55c017a7a770 msgr2=0x55c017ac2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f4159f740 1 -- 172.21.15.139:0/3427586005 shutdown_connections 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f4159f740 1 -- 172.21.15.139:0/3427586005 wait complete. 2024-04-25T09:59:17.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.742+0000 7f1f4159f740 20 client.7905 trim_cache size 0 max 16384 2024-04-25T09:59:17.752 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:59:17.752 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:59:17.752 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": { 2024-04-25T09:59:17.752 INFO:teuthology.orchestra.run.smithi139.stdout: "bd10579d-d66c-4161-b9c0-3723ce1af2e1": { 2024-04-25T09:59:17.752 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/dir", 2024-04-25T09:59:17.752 INFO:teuthology.orchestra.run.smithi139.stdout: "tag": "bd10579d-d66c-4161-b9c0-3723ce1af2e1", 2024-04-25T09:59:17.752 INFO:teuthology.orchestra.run.smithi139.stdout: "options": "recursive" 2024-04-25T09:59:17.752 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:59:17.752 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:59:17.752 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:59:17.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.743+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/4141241831 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1f2406b4d0 msgr2=0x7f1f2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.743+0000 7f1f3e7c1640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1f2406b4d0 0x7f1f2406d990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f381597d0 tx=0x7f1f2c008040 comp rx=0 tx=0).stop 2024-04-25T09:59:17.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.743+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/4141241831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f380b9370 msgr2=0x7f1f3814e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:17.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.743+0000 7f1f3e7c1640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f380b9370 0x7f1f3814e690 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1f2802a220 tx=0x7f1f280046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.743+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/4141241831 shutdown_connections 2024-04-25T09:59:17.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.743+0000 7f1f3e7c1640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1f2406b4d0 0x7f1f2406d990 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.743+0000 7f1f3e7c1640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f3815dbf0 0x7f1f3815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.743+0000 7f1f3e7c1640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f380cec80 0x7f1f3815c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.743+0000 7f1f3e7c1640 1 --2- 172.21.15.139:0/4141241831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f380b9370 0x7f1f3814e690 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:17.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.743+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/4141241831 >> 172.21.15.139:0/4141241831 conn(0x7f1f380b5480 msgr2=0x7f1f380d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:17.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.743+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/4141241831 shutdown_connections 2024-04-25T09:59:17.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:17.743+0000 7f1f3e7c1640 1 -- 172.21.15.139:0/4141241831 wait complete. 2024-04-25T09:59:17.757 INFO:tasks.cephfs.filesystem:scrub status for tag:bd10579d-d66c-4161-b9c0-3723ce1af2e1 - {'path': '/dir', 'tag': 'bd10579d-d66c-4161-b9c0-3723ce1af2e1', 'options': 'recursive'} 2024-04-25T09:59:18.296 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T09:59:18.292+0000 7efdd5540640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:59:21.934 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T09:59:21.930+0000 7f73a902d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T09:59:46.516 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:59:46.523 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:59:47.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:47.758 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:59:47.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.867+0000 7fc9ee53b640 1 Processor -- start 2024-04-25T09:59:47.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.867+0000 7fc9ee53b640 1 -- start start 2024-04-25T09:59:47.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.867+0000 7fc9ee53b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc9e80cee50 0x7fc9e8168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.867+0000 7fc9ee53b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e8161dc0 con 0x7fc9e815dbf0 2024-04-25T09:59:47.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.867+0000 7fc9ee53b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e81620a0 con 0x7fc9e81617e0 2024-04-25T09:59:47.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.867+0000 7fc9ee53b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e80d1ee0 con 0x7fc9e80cee50 2024-04-25T09:59:47.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.868+0000 7fc9ecad8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc9e80cee50 0x7fc9e8168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.868+0000 7fc9e7fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc9e815dbf0 0x7fc9e80b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48970/0 (socket says 172.21.15.139:48970) 2024-04-25T09:59:47.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.868+0000 7fc9e7fff640 1 -- 172.21.15.139:0/1100970372 learned_addr learned my addr 172.21.15.139:0/1100970372 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:47.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.868+0000 7fc9ecad8640 1 -- 172.21.15.139:0/1100970372 >> v1:172.21.15.150:6789/0 conn(0x7fc9e81617e0 legacy=0x7fc9e80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:47.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.868+0000 7fc9ecad8640 1 -- 172.21.15.139:0/1100970372 >> v1:172.21.15.139:6789/0 conn(0x7fc9e815dbf0 legacy=0x7fc9e80b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.868+0000 7fc9ecad8640 1 -- 172.21.15.139:0/1100970372 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e8162380 con 0x7fc9e80cee50 2024-04-25T09:59:47.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.868+0000 7fc9e6ffd640 1 -- 172.21.15.139:0/1100970372 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092626636 0 0) 0x7fc9e8161dc0 con 0x7fc9e815dbf0 2024-04-25T09:59:47.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.869+0000 7fc9ecad8640 1 --2- 172.21.15.139:0/1100970372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e80cee50 0x7fc9e8168b10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009d90 tx=0x7fc9e002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22f14d8d94a79d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:47.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.869+0000 7fc9e6ffd640 1 -- 172.21.15.139:0/1100970372 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0031600 con 0x7fc9e80cee50 2024-04-25T09:59:47.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.869+0000 7fc9e6ffd640 1 -- 172.21.15.139:0/1100970372 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e00317a0 con 0x7fc9e80cee50 2024-04-25T09:59:47.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.869+0000 7fc9e6ffd640 1 -- 172.21.15.139:0/1100970372 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0031b70 con 0x7fc9e80cee50 2024-04-25T09:59:47.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.869+0000 7fc9ee53b640 1 -- 172.21.15.139:0/1100970372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e80cee50 msgr2=0x7fc9e8168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.869+0000 7fc9ee53b640 1 --2- 172.21.15.139:0/1100970372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e80cee50 0x7fc9e8168b10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009d90 tx=0x7fc9e002a610 comp rx=0 tx=0).stop 2024-04-25T09:59:47.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.869+0000 7fc9ee53b640 1 -- 172.21.15.139:0/1100970372 shutdown_connections 2024-04-25T09:59:47.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.869+0000 7fc9ee53b640 1 --2- 172.21.15.139:0/1100970372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e80cee50 0x7fc9e8168b10 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.869+0000 7fc9ee53b640 1 -- 172.21.15.139:0/1100970372 >> 172.21.15.139:0/1100970372 conn(0x7fc9e80b5480 msgr2=0x7fc9e80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:47.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.869+0000 7fc9ee53b640 1 -- 172.21.15.139:0/1100970372 shutdown_connections 2024-04-25T09:59:47.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.869+0000 7fc9ee53b640 1 -- 172.21.15.139:0/1100970372 wait complete. 2024-04-25T09:59:47.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.871+0000 7fc9ee53b640 1 Processor -- start 2024-04-25T09:59:47.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.871+0000 7fc9ee53b640 1 -- start start 2024-04-25T09:59:47.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9ee53b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9e80cee50 0x7fc9e80cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9ee53b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e815dbf0 0x7fc9e8160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9ee53b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9e81617e0 0x7fc9e8152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9ee53b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e815dfd0 con 0x7fc9e815dbf0 2024-04-25T09:59:47.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9ee53b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80d3380 con 0x7fc9e80cee50 2024-04-25T09:59:47.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9ee53b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8153830 con 0x7fc9e81617e0 2024-04-25T09:59:47.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e815dbf0 0x7fc9e8160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e815dbf0 0x7fc9e8160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44752/0 (socket says 172.21.15.139:44752) 2024-04-25T09:59:47.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e77fe640 1 -- 172.21.15.139:0/3950030747 learned_addr learned my addr 172.21.15.139:0/3950030747 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:47.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e7fff640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9e80cee50 0x7fc9e80cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9ecad8640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9e81617e0 0x7fc9e8152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e77fe640 1 -- 172.21.15.139:0/3950030747 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9e81617e0 msgr2=0x7fc9e8152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e77fe640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9e81617e0 0x7fc9e8152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e77fe640 1 -- 172.21.15.139:0/3950030747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9e80cee50 msgr2=0x7fc9e80cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e77fe640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9e80cee50 0x7fc9e80cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e77fe640 1 -- 172.21.15.139:0/3950030747 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e8160e90 con 0x7fc9e815dbf0 2024-04-25T09:59:47.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e7fff640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9e80cee50 0x7fc9e80cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:47.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e77fe640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e815dbf0 0x7fc9e8160380 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00b3f0 tx=0x7fc9dc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:47.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e4ff9640 1 -- 172.21.15.139:0/3950030747 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc015020 con 0x7fc9e815dbf0 2024-04-25T09:59:47.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e4ff9640 1 -- 172.21.15.139:0/3950030747 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc011830 con 0x7fc9e815dbf0 2024-04-25T09:59:47.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9ee53b640 1 -- 172.21.15.139:0/3950030747 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e8153d80 con 0x7fc9e815dbf0 2024-04-25T09:59:47.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9e4ff9640 1 -- 172.21.15.139:0/3950030747 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc011c20 con 0x7fc9e815dbf0 2024-04-25T09:59:47.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.872+0000 7fc9ee53b640 1 -- 172.21.15.139:0/3950030747 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e80cd3c0 con 0x7fc9e815dbf0 2024-04-25T09:59:47.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.874+0000 7fc9e4ff9640 1 -- 172.21.15.139:0/3950030747 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9dc018020 con 0x7fc9e815dbf0 2024-04-25T09:59:47.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.874+0000 7fc9e4ff9640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9d406b4d0 0x7fc9d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.874+0000 7fc9e4ff9640 1 -- 172.21.15.139:0/3950030747 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc9dc08a090 con 0x7fc9e815dbf0 2024-04-25T09:59:47.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.874+0000 7fc9e7fff640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9d406b4d0 0x7fc9d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.875+0000 7fc9e7fff640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9d406b4d0 0x7fc9d406d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8000f60 tx=0x7fc9d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:47.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.875+0000 7fc9f1242740 1 Processor -- start 2024-04-25T09:59:47.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.875+0000 7fc9f1242740 1 -- start start 2024-04-25T09:59:47.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9f1242740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 0x55a455b68320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9f1242740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65ef0 0x55a455b68860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9f1242740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b38770 0x55a455b3bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9f1242740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a455aa1da0 con 0x55a455b38770 2024-04-25T09:59:47.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9f1242740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a455a4ab80 con 0x55a455b65ef0 2024-04-25T09:59:47.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9f1242740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a455a9de40 con 0x55a455b65b10 2024-04-25T09:59:47.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9ecad8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b38770 0x55a455b3bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9ecad8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b38770 0x55a455b3bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44758/0 (socket says 172.21.15.139:44758) 2024-04-25T09:59:47.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9ecad8640 1 -- 172.21.15.139:0/3492329614 learned_addr learned my addr 172.21.15.139:0/3492329614 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:47.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9ecad8640 1 -- 172.21.15.139:0/3492329614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 msgr2=0x55a455b68320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:47.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9e77fe640 1 --2- 172.21.15.139:0/3492329614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65ef0 0x55a455b68860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9e7fff640 1 --2- 172.21.15.139:0/3492329614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 0x55a455b68320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9ecad8640 1 --2- 172.21.15.139:0/3492329614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 0x55a455b68320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9ecad8640 1 -- 172.21.15.139:0/3492329614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65ef0 msgr2=0x55a455b68860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:47.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9ecad8640 1 --2- 172.21.15.139:0/3492329614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65ef0 0x55a455b68860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9ecad8640 1 -- 172.21.15.139:0/3492329614 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a455a54f10 con 0x55a455b38770 2024-04-25T09:59:47.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.876+0000 7fc9ecad8640 1 --2- 172.21.15.139:0/3492329614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b38770 0x55a455b3bf80 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc9e002ac00 tx=0x7fc9e002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb83ce8326f668e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:47.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.877+0000 7fc9cdffb640 1 -- 172.21.15.139:0/3492329614 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00318c0 con 0x55a455b38770 2024-04-25T09:59:47.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.877+0000 7fc9cdffb640 1 -- 172.21.15.139:0/3492329614 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e0031a60 con 0x55a455b38770 2024-04-25T09:59:47.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.877+0000 7fc9cdffb640 1 -- 172.21.15.139:0/3492329614 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0031e30 con 0x55a455b38770 2024-04-25T09:59:47.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.877+0000 7fc9f1242740 1 -- 172.21.15.139:0/3492329614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b38770 msgr2=0x55a455b3bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.877+0000 7fc9f1242740 1 --2- 172.21.15.139:0/3492329614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b38770 0x55a455b3bf80 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc9e002ac00 tx=0x7fc9e002e000 comp rx=0 tx=0).stop 2024-04-25T09:59:47.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.877+0000 7fc9f1242740 1 -- 172.21.15.139:0/3492329614 shutdown_connections 2024-04-25T09:59:47.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.877+0000 7fc9f1242740 1 --2- 172.21.15.139:0/3492329614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b38770 0x55a455b3bf80 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.877+0000 7fc9f1242740 1 --2- 172.21.15.139:0/3492329614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65ef0 0x55a455b68860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.877+0000 7fc9f1242740 1 --2- 172.21.15.139:0/3492329614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 0x55a455b68320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.877+0000 7fc9f1242740 1 -- 172.21.15.139:0/3492329614 >> 172.21.15.139:0/3492329614 conn(0x55a455b3a9e0 msgr2=0x55a455b83320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:47.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.877+0000 7fc9f1242740 1 -- 172.21.15.139:0/3492329614 shutdown_connections 2024-04-25T09:59:47.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.877+0000 7fc9f1242740 1 -- 172.21.15.139:0/3492329614 wait complete. 2024-04-25T09:59:47.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.878+0000 7fc9f1242740 1 -- start start 2024-04-25T09:59:47.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.878+0000 7fc9f1242740 1 Processor -- start 2024-04-25T09:59:47.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9f1242740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 0x55a455b691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9f1242740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b65ef0 0x55a455b64590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9f1242740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b86a70 0x55a455b86ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9f1242740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a455b3db90 con 0x55a455b65ef0 2024-04-25T09:59:47.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9f1242740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a455a404b0 con 0x55a455b86a70 2024-04-25T09:59:47.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9f1242740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a455b3de70 con 0x55a455b65b10 2024-04-25T09:59:47.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9f1242740 10 client.0 _ll_register_callbacks cb 0x55a455a80180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:47.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b65ef0 0x55a455b64590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9e77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b65ef0 0x55a455b64590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44764/0 (socket says 172.21.15.139:44764) 2024-04-25T09:59:47.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9e77fe640 1 -- 172.21.15.139:0/2475798352 learned_addr learned my addr 172.21.15.139:0/2475798352 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:47.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9e7fff640 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 0x55a455b691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9ecad8640 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b86a70 0x55a455b86ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9e77fe640 1 -- 172.21.15.139:0/2475798352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 msgr2=0x55a455b691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:47.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9e77fe640 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 0x55a455b691f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9e77fe640 1 -- 172.21.15.139:0/2475798352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b86a70 msgr2=0x55a455b86ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:47.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9e77fe640 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b86a70 0x55a455b86ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9e77fe640 1 -- 172.21.15.139:0/2475798352 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4558807d0 con 0x55a455b65ef0 2024-04-25T09:59:47.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9e77fe640 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b65ef0 0x55a455b64590 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc060810 tx=0x7fc9dc05c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1e497d9e5e3100 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:47.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.879+0000 7fc9c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:47.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9c6ffd640 1 -- 172.21.15.139:0/2475798352 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc070170 con 0x55a455b65ef0 2024-04-25T09:59:47.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9c6ffd640 1 -- 172.21.15.139:0/2475798352 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc9dc054c10 con 0x55a455b65ef0 2024-04-25T09:59:47.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9f1242740 1 -- 172.21.15.139:0/2475798352 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9e0031a60 con 0x55a455b65ef0 2024-04-25T09:59:47.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9c6ffd640 1 -- 172.21.15.139:0/2475798352 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9dc088050 con 0x55a455b65ef0 2024-04-25T09:59:47.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9f1242740 10 client.7920 fetch_fsmap learned FSMap version 57 2024-04-25T09:59:47.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9f1242740 1 -- 172.21.15.139:0/2475798352 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55a4557b63f0 con 0x55a455b65ef0 2024-04-25T09:59:47.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9c6ffd640 1 -- 172.21.15.139:0/2475798352 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e0031a60 con 0x55a455b65ef0 2024-04-25T09:59:47.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9c6ffd640 1 -- 172.21.15.139:0/2475798352 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc9dc0579f0 con 0x55a455b65ef0 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9f1242740 10 client.7920 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9f1242740 10 client.7920 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9f1242740 20 client.7920 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9f1242740 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55a455b62070 0x55a455b84450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9f1242740 4 client.7920 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9f1242740 1 -- 172.21.15.139:0/2475798352 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a455aaf2e0 con 0x55a455b62070 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9e7fff640 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55a455b62070 0x55a455b84450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9e7fff640 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55a455b62070 0x55a455b84450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc4bf7e7edd753a server_cookie=d0282ec3bb8ef504 in_seq=0 out_seq=0 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.880+0000 7fc9c6ffd640 10 client.7920 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9c6ffd640 1 -- 172.21.15.139:0/2475798352 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a455aaf2e0 con 0x55a455b62070 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9c6ffd640 10 client.7920 handle_command_reply: tid=0 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9f1242740 1 client.7920 shutdown 2024-04-25T09:59:47.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9f1242740 1 -- 172.21.15.139:0/2475798352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b65ef0 msgr2=0x55a455b64590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9f1242740 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b65ef0 0x55a455b64590 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc060810 tx=0x7fc9dc05c6f0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9f1242740 1 -- 172.21.15.139:0/2475798352 shutdown_connections 2024-04-25T09:59:47.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9f1242740 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55a455b62070 0x55a455b84450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9f1242740 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b65ef0 0x55a455b64590 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9f1242740 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b86a70 0x55a455b86ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9c6ffd640 0 client.7920 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:47.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9f1242740 1 --2- 172.21.15.139:0/2475798352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 0x55a455b691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9f1242740 1 -- 172.21.15.139:0/2475798352 >> 172.21.15.139:0/2475798352 conn(0x55a455b38770 msgr2=0x55a455b39420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:47.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9f1242740 1 -- 172.21.15.139:0/2475798352 shutdown_connections 2024-04-25T09:59:47.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9f1242740 1 -- 172.21.15.139:0/2475798352 wait complete. 2024-04-25T09:59:47.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.881+0000 7fc9f1242740 20 client.7920 trim_cache size 0 max 16384 2024-04-25T09:59:47.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.896+0000 7fc9f1242740 1 Processor -- start 2024-04-25T09:59:47.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.896+0000 7fc9f1242740 1 -- start start 2024-04-25T09:59:47.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.897+0000 7fc9f1242740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 0x55a455b5fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.897+0000 7fc9f1242740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65ef0 0x55a455be7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.897+0000 7fc9f1242740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 0x55a455b37600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.897+0000 7fc9f1242740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a455b385f0 con 0x55a455b86a70 2024-04-25T09:59:47.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.897+0000 7fc9f1242740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8200310 con 0x55a455b65ef0 2024-04-25T09:59:47.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.897+0000 7fc9f1242740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e81ff590 con 0x55a455b65b10 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.897+0000 7fc9ecad8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 0x55a455b37600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.897+0000 7fc9ecad8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 0x55a455b37600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44772/0 (socket says 172.21.15.139:44772) 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.897+0000 7fc9ecad8640 1 -- 172.21.15.139:0/4121580234 learned_addr learned my addr 172.21.15.139:0/4121580234 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.897+0000 7fc9e77fe640 1 --2- 172.21.15.139:0/4121580234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65ef0 0x55a455be7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.897+0000 7fc9e7fff640 1 --2- 172.21.15.139:0/4121580234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 0x55a455b5fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9ecad8640 1 -- 172.21.15.139:0/4121580234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 msgr2=0x55a455b5fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9ecad8640 1 --2- 172.21.15.139:0/4121580234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 0x55a455b5fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9ecad8640 1 -- 172.21.15.139:0/4121580234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65ef0 msgr2=0x55a455be7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9ecad8640 1 --2- 172.21.15.139:0/4121580234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65ef0 0x55a455be7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9ecad8640 1 -- 172.21.15.139:0/4121580234 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4557760f0 con 0x55a455b86a70 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9ecad8640 1 --2- 172.21.15.139:0/4121580234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 0x55a455b37600 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009e80 tx=0x7fc9e0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca20e4dcc2a25d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:47.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9cd7fa640 1 -- 172.21.15.139:0/4121580234 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0003dc0 con 0x55a455b86a70 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9cd7fa640 1 -- 172.21.15.139:0/4121580234 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e00316c0 con 0x55a455b86a70 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9cd7fa640 1 -- 172.21.15.139:0/4121580234 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00318b0 con 0x55a455b86a70 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9f1242740 1 -- 172.21.15.139:0/4121580234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 msgr2=0x55a455b37600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9f1242740 1 --2- 172.21.15.139:0/4121580234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 0x55a455b37600 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009e80 tx=0x7fc9e0030000 comp rx=0 tx=0).stop 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9f1242740 1 -- 172.21.15.139:0/4121580234 shutdown_connections 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9f1242740 1 --2- 172.21.15.139:0/4121580234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 0x55a455b37600 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9f1242740 1 --2- 172.21.15.139:0/4121580234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65ef0 0x55a455be7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9f1242740 1 --2- 172.21.15.139:0/4121580234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65b10 0x55a455b5fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9f1242740 1 -- 172.21.15.139:0/4121580234 >> 172.21.15.139:0/4121580234 conn(0x55a455b38770 msgr2=0x55a455b834b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9f1242740 1 -- 172.21.15.139:0/4121580234 shutdown_connections 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.898+0000 7fc9f1242740 1 -- 172.21.15.139:0/4121580234 wait complete. 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.899+0000 7fc9f1242740 1 -- start start 2024-04-25T09:59:47.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.899+0000 7fc9f1242740 1 Processor -- start 2024-04-25T09:59:47.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9f1242740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65b10 0x55a455b84f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9f1242740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65ef0 0x55a455b62880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9f1242740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 0x55a455be6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9f1242740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e81551b0 con 0x55a455b86a70 2024-04-25T09:59:47.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9f1242740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8154bb0 con 0x55a455b65b10 2024-04-25T09:59:47.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9f1242740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a455ba5930 con 0x55a455b65ef0 2024-04-25T09:59:47.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9f1242740 10 client.0 _ll_register_callbacks cb 0x7fc9e0009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:47.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9ecad8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 0x55a455be6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9ecad8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 0x55a455be6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44778/0 (socket says 172.21.15.139:44778) 2024-04-25T09:59:47.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9ecad8640 1 -- 172.21.15.139:0/2904668985 learned_addr learned my addr 172.21.15.139:0/2904668985 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:47.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9e7fff640 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65b10 0x55a455b84f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9e77fe640 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65ef0 0x55a455b62880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9ecad8640 1 -- 172.21.15.139:0/2904668985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65ef0 msgr2=0x55a455b62880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:47.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9ecad8640 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65ef0 0x55a455b62880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9ecad8640 1 -- 172.21.15.139:0/2904668985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65b10 msgr2=0x55a455b84f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:47.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9ecad8640 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65b10 0x55a455b84f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.900+0000 7fc9ecad8640 1 -- 172.21.15.139:0/2904668985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a455b844f0 con 0x55a455b86a70 2024-04-25T09:59:47.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9ecad8640 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 0x55a455be6d80 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0048040 tx=0x7fc9e0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2025dc432aa898a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:47.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:47.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9c6ffd640 1 -- 172.21.15.139:0/2904668985 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e005c020 con 0x55a455b86a70 2024-04-25T09:59:47.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9c6ffd640 1 -- 172.21.15.139:0/2904668985 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc9e0053040 con 0x55a455b86a70 2024-04-25T09:59:47.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9f1242740 1 -- 172.21.15.139:0/2904668985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9e00316c0 con 0x55a455b86a70 2024-04-25T09:59:47.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9c6ffd640 1 -- 172.21.15.139:0/2904668985 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9e0031a80 con 0x55a455b86a70 2024-04-25T09:59:47.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9f1242740 10 client.7926 fetch_fsmap learned FSMap version 57 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9f1242740 1 -- 172.21.15.139:0/2904668985 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55a455b3c5e0 con 0x55a455b86a70 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9c6ffd640 1 -- 172.21.15.139:0/2904668985 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e00316c0 con 0x55a455b86a70 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9c6ffd640 1 -- 172.21.15.139:0/2904668985 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc9e0061400 con 0x55a455b86a70 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9f1242740 10 client.7926 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9f1242740 10 client.7926 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9f1242740 20 client.7926 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9f1242740 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55a455b89930 0x55a455b38c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9f1242740 4 client.7926 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9f1242740 1 -- 172.21.15.139:0/2904668985 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a455b83110 con 0x55a455b89930 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.901+0000 7fc9e7fff640 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55a455b89930 0x55a455b38c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9c6ffd640 10 client.7926 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9e7fff640 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55a455b89930 0x55a455b38c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=801421e4d32c0878 server_cookie=701ab392919d9b15 in_seq=0 out_seq=0 2024-04-25T09:59:47.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9c6ffd640 1 -- 172.21.15.139:0/2904668985 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a455b83110 con 0x55a455b89930 2024-04-25T09:59:47.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9c6ffd640 10 client.7926 handle_command_reply: tid=0 2024-04-25T09:59:47.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9f1242740 1 client.7926 shutdown 2024-04-25T09:59:47.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9f1242740 1 -- 172.21.15.139:0/2904668985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 msgr2=0x55a455be6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9f1242740 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 0x55a455be6d80 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0048040 tx=0x7fc9e0004ab0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9f1242740 1 -- 172.21.15.139:0/2904668985 shutdown_connections 2024-04-25T09:59:47.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9f1242740 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55a455b89930 0x55a455b38c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9f1242740 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a455b86a70 0x55a455be6d80 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9f1242740 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a455b65ef0 0x55a455b62880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9c6ffd640 0 client.7926 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:47.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9f1242740 1 --2- 172.21.15.139:0/2904668985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a455b65b10 0x55a455b84f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9f1242740 1 -- 172.21.15.139:0/2904668985 >> 172.21.15.139:0/2904668985 conn(0x55a455b38770 msgr2=0x55a455b37690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:47.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9f1242740 1 -- 172.21.15.139:0/2904668985 shutdown_connections 2024-04-25T09:59:47.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9f1242740 1 -- 172.21.15.139:0/2904668985 wait complete. 2024-04-25T09:59:47.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.902+0000 7fc9f1242740 20 client.7926 trim_cache size 0 max 16384 2024-04-25T09:59:47.914 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:59:47.914 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "no active scrubs running", 2024-04-25T09:59:47.914 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": {} 2024-04-25T09:59:47.914 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:59:47.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.903+0000 7fc9ee53b640 1 -- 172.21.15.139:0/3950030747 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9d406b4d0 msgr2=0x7fc9d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.903+0000 7fc9ee53b640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9d406b4d0 0x7fc9d406d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8000f60 tx=0x7fc9d8009040 comp rx=0 tx=0).stop 2024-04-25T09:59:47.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.903+0000 7fc9ee53b640 1 -- 172.21.15.139:0/3950030747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e815dbf0 msgr2=0x7fc9e8160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:47.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.903+0000 7fc9ee53b640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e815dbf0 0x7fc9e8160380 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00b3f0 tx=0x7fc9dc00b8b0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.903+0000 7fc9ee53b640 1 -- 172.21.15.139:0/3950030747 shutdown_connections 2024-04-25T09:59:47.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.903+0000 7fc9ee53b640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9e815dbf0 0x7fc9e8160380 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.903+0000 7fc9ee53b640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9d406b4d0 0x7fc9d406d990 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.903+0000 7fc9ee53b640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9e81617e0 0x7fc9e8152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.903+0000 7fc9ee53b640 1 --2- 172.21.15.139:0/3950030747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9e80cee50 0x7fc9e80cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:47.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.903+0000 7fc9ee53b640 1 -- 172.21.15.139:0/3950030747 >> 172.21.15.139:0/3950030747 conn(0x7fc9e80b5480 msgr2=0x7fc9e80b71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:47.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.903+0000 7fc9ee53b640 1 -- 172.21.15.139:0/3950030747 shutdown_connections 2024-04-25T09:59:47.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:47.903+0000 7fc9ee53b640 1 -- 172.21.15.139:0/3950030747 wait complete. 2024-04-25T09:59:47.918 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:59:47.918 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:59:48.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.011+0000 7fdf3d03a640 1 Processor -- start 2024-04-25T09:59:48.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.011+0000 7fdf3d03a640 1 -- start start 2024-04-25T09:59:48.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.012+0000 7fdf3d03a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380b0d00 0x7fdf3815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.012+0000 7fdf3d03a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf3815e610 0x7fdf380d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.012+0000 7fdf3d03a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf380b10e0 con 0x7fdf380d12c0 2024-04-25T09:59:48.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.012+0000 7fdf3d03a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf380d0350 con 0x7fdf3815e610 2024-04-25T09:59:48.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.012+0000 7fdf3d03a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3815f070 con 0x7fdf380b0d00 2024-04-25T09:59:48.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.012+0000 7fdf3759e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdf380d12c0 0x7fdf3806ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48972/0 (socket says 172.21.15.139:48972) 2024-04-25T09:59:48.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.012+0000 7fdf3759e640 1 -- 172.21.15.139:0/2939112376 learned_addr learned my addr 172.21.15.139:0/2939112376 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T09:59:48.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.012+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434353120 0 0) 0x7fdf380b10e0 con 0x7fdf380d12c0 2024-04-25T09:59:48.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf20003520 con 0x7fdf380d12c0 2024-04-25T09:59:48.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf3659c640 1 --2- 172.21.15.139:0/2939112376 >> v2:172.21.15.150:3300/0 conn(0x7fdf3815e610 0x7fdf380d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf36d9d640 1 --2- 172.21.15.139:0/2939112376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380b0d00 0x7fdf3815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 861119955 0 0) 0x7fdf20003520 con 0x7fdf380d12c0 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380b0d00 msgr2=0x7fdf3815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 --2- 172.21.15.139:0/2939112376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380b0d00 0x7fdf3815e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 >> v2:172.21.15.150:3300/0 conn(0x7fdf3815e610 msgr2=0x7fdf380d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 --2- 172.21.15.139:0/2939112376 >> v2:172.21.15.150:3300/0 conn(0x7fdf3815e610 0x7fdf380d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3815eab0 con 0x7fdf380d12c0 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fdf30001670 con 0x7fdf380d12c0 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf3659c640 1 --2- 172.21.15.139:0/2939112376 >> v2:172.21.15.150:3300/0 conn(0x7fdf3815e610 0x7fdf380d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 >> v1:172.21.15.139:6789/0 conn(0x7fdf380d12c0 legacy=0x7fdf3806ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 --2- 172.21.15.139:0/2939112376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380b0d00 0x7fdf200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 --2- 172.21.15.139:0/2939112376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20003fe0 0x7fdf20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 --2- 172.21.15.139:0/2939112376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf20004d40 0x7fdf200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3815f070 con 0x7fdf20004d40 2024-04-25T09:59:48.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2000bb00 con 0x7fdf20003fe0 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2000bd00 con 0x7fdf380b0d00 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf3759e640 1 --2- 172.21.15.139:0/2939112376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf20004d40 0x7fdf200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf36d9d640 1 --2- 172.21.15.139:0/2939112376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380b0d00 0x7fdf200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf3659c640 1 --2- 172.21.15.139:0/2939112376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20003fe0 0x7fdf20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf3759e640 1 -- 172.21.15.139:0/2939112376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380b0d00 msgr2=0x7fdf200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf3759e640 1 --2- 172.21.15.139:0/2939112376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380b0d00 0x7fdf200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf3759e640 1 -- 172.21.15.139:0/2939112376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20003fe0 msgr2=0x7fdf20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf3759e640 1 --2- 172.21.15.139:0/2939112376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf20003fe0 0x7fdf20004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.013+0000 7fdf3759e640 1 -- 172.21.15.139:0/2939112376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf20003520 con 0x7fdf20004d40 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.014+0000 7fdf3759e640 1 --2- 172.21.15.139:0/2939112376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf20004d40 0x7fdf200051c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fdf380b7940 tx=0x7fdf20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f501031856ea63a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.014+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30007d50 con 0x7fdf20004d40 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.014+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf300058d0 con 0x7fdf20004d40 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.014+0000 7fdf35d9b640 1 -- 172.21.15.139:0/2939112376 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30005cc0 con 0x7fdf20004d40 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.015+0000 7fdf3d03a640 1 -- 172.21.15.139:0/2939112376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf20004d40 msgr2=0x7fdf200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.015+0000 7fdf3d03a640 1 --2- 172.21.15.139:0/2939112376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf20004d40 0x7fdf200051c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fdf380b7940 tx=0x7fdf20005830 comp rx=0 tx=0).stop 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.015+0000 7fdf3759e640 1 -- 172.21.15.139:0/2939112376 reap_dead start 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.015+0000 7fdf3d03a640 1 -- 172.21.15.139:0/2939112376 shutdown_connections 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.015+0000 7fdf3d03a640 1 -- 172.21.15.139:0/2939112376 >> 172.21.15.139:0/2939112376 conn(0x7fdf380b3380 msgr2=0x7fdf380b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.015+0000 7fdf3d03a640 1 -- 172.21.15.139:0/2939112376 shutdown_connections 2024-04-25T09:59:48.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.015+0000 7fdf3d03a640 1 -- 172.21.15.139:0/2939112376 wait complete. 2024-04-25T09:59:48.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.016+0000 7fdf3d03a640 1 Processor -- start 2024-04-25T09:59:48.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.016+0000 7fdf3d03a640 1 -- start start 2024-04-25T09:59:48.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf3d03a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf38153ff0 0x7fdf3815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf3d03a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf381543d0 0x7fdf3815d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf3d03a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf381f35a0 0x7fdf381f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf3d03a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3806d510 con 0x7fdf38153ff0 2024-04-25T09:59:48.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf3d03a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380d0580 con 0x7fdf381543d0 2024-04-25T09:59:48.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf3d03a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3806d690 con 0x7fdf381f35a0 2024-04-25T09:59:48.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf36d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf38153ff0 0x7fdf3815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf36d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf38153ff0 0x7fdf3815cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44792/0 (socket says 172.21.15.139:44792) 2024-04-25T09:59:48.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf36d9d640 1 -- 172.21.15.139:0/3319263231 learned_addr learned my addr 172.21.15.139:0/3319263231 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf3759e640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf381f35a0 0x7fdf381f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf3659c640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf381543d0 0x7fdf3815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf36d9d640 1 -- 172.21.15.139:0/3319263231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf381f35a0 msgr2=0x7fdf381f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf36d9d640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf381f35a0 0x7fdf381f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf36d9d640 1 -- 172.21.15.139:0/3319263231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf381543d0 msgr2=0x7fdf3815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf36d9d640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf381543d0 0x7fdf3815d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf36d9d640 1 -- 172.21.15.139:0/3319263231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf381f42d0 con 0x7fdf38153ff0 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf3759e640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf381f35a0 0x7fdf381f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf3659c640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf381543d0 0x7fdf3815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf36d9d640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf38153ff0 0x7fdf3815cb50 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fdf280046a0 tx=0x7fdf28007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf1f7fe640 1 -- 172.21.15.139:0/3319263231 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28015420 con 0x7fdf38153ff0 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf1f7fe640 1 -- 172.21.15.139:0/3319263231 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2801a070 con 0x7fdf38153ff0 2024-04-25T09:59:48.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf3d03a640 1 -- 172.21.15.139:0/3319263231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf381f6a00 con 0x7fdf38153ff0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.017+0000 7fdf1f7fe640 1 -- 172.21.15.139:0/3319263231 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28015800 con 0x7fdf38153ff0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.018+0000 7fdf3d03a640 1 -- 172.21.15.139:0/3319263231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf381f6fc0 con 0x7fdf38153ff0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.019+0000 7fdf3d03a640 1 -- 172.21.15.139:0/3319263231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf38074520 con 0x7fdf38153ff0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.019+0000 7fdf1f7fe640 1 -- 172.21.15.139:0/3319263231 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf280159a0 con 0x7fdf38153ff0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.019+0000 7fdf1f7fe640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf2406b4d0 0x7fdf2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.019+0000 7fdf1f7fe640 1 -- 172.21.15.139:0/3319263231 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdf2808aeb0 con 0x7fdf38153ff0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.019+0000 7fdf3659c640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf2406b4d0 0x7fdf2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.020+0000 7fdf3659c640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf2406b4d0 0x7fdf2406d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdf381547b0 tx=0x7fdf2c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.024+0000 7fdf1f7fe640 1 -- 172.21.15.139:0/3319263231 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf28024020 con 0x7fdf38153ff0 2024-04-25T09:59:48.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.231+0000 7fdf3d03a640 1 -- 172.21.15.139:0/3319263231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf380b2ac0 con 0x7fdf38153ff0 2024-04-25T09:59:48.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.232+0000 7fdf1f7fe640 1 -- 172.21.15.139:0/3319263231 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4954 (secure 0 0 0) 0x7fdf28059b50 con 0x7fdf38153ff0 2024-04-25T09:59:48.236 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:48.236 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T09:59:21.913360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":10,"addr":"172.21.15.139:6836/1239243954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1239243954},{"type":"v1","addr":"172.21.15.139:6836","nonce":1239243954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:59:48.236 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 57 2024-04-25T09:59:48.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.234+0000 7fdf3d03a640 1 -- 172.21.15.139:0/3319263231 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf2406b4d0 msgr2=0x7fdf2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.234+0000 7fdf3d03a640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf2406b4d0 0x7fdf2406d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdf381547b0 tx=0x7fdf2c008040 comp rx=0 tx=0).stop 2024-04-25T09:59:48.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.234+0000 7fdf3d03a640 1 -- 172.21.15.139:0/3319263231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf38153ff0 msgr2=0x7fdf3815cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.234+0000 7fdf3d03a640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf38153ff0 0x7fdf3815cb50 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fdf280046a0 tx=0x7fdf28007700 comp rx=0 tx=0).stop 2024-04-25T09:59:48.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.234+0000 7fdf3d03a640 1 -- 172.21.15.139:0/3319263231 shutdown_connections 2024-04-25T09:59:48.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.234+0000 7fdf3d03a640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf2406b4d0 0x7fdf2406d990 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.234+0000 7fdf3d03a640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf381f35a0 0x7fdf381f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.234+0000 7fdf3d03a640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf381543d0 0x7fdf3815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.234+0000 7fdf3d03a640 1 --2- 172.21.15.139:0/3319263231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf38153ff0 0x7fdf3815cb50 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.234+0000 7fdf3d03a640 1 -- 172.21.15.139:0/3319263231 >> 172.21.15.139:0/3319263231 conn(0x7fdf380b3380 msgr2=0x7fdf38070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.234+0000 7fdf3d03a640 1 -- 172.21.15.139:0/3319263231 shutdown_connections 2024-04-25T09:59:48.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.234+0000 7fdf3d03a640 1 -- 172.21.15.139:0/3319263231 wait complete. 2024-04-25T09:59:48.249 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T09:59:48.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.344+0000 7f5c5dbe0640 1 Processor -- start 2024-04-25T09:59:48.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.344+0000 7f5c5dbe0640 1 -- start start 2024-04-25T09:59:48.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.344+0000 7f5c5dbe0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5c580cd610 0x7f5c580d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.344+0000 7f5c5dbe0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5c5809ad40 0x7f5c580d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c5dbe0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c58099ca0 con 0x7f5c581618c0 2024-04-25T09:59:48.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c5dbe0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c58099a50 con 0x7f5c5809ad40 2024-04-25T09:59:48.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c5dbe0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c58099f80 con 0x7f5c580cd610 2024-04-25T09:59:48.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c56ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5c5809ad40 0x7f5c580d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c56ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5c5809ad40 0x7f5c580d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44794/0 (socket says 172.21.15.139:44794) 2024-04-25T09:59:48.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c56ffd640 1 -- 172.21.15.139:0/3248745042 learned_addr learned my addr 172.21.15.139:0/3248745042 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c57fff640 1 --1- 172.21.15.139:0/3248745042 >> v1:172.21.15.139:6789/0 conn(0x7f5c581618c0 0x7f5c5806be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48976/0 (socket says 172.21.15.139:48976) 2024-04-25T09:59:48.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c577fe640 1 --2- 172.21.15.139:0/3248745042 >> v2:172.21.15.150:3300/0 conn(0x7f5c580cd610 0x7f5c580d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c56ffd640 1 -- 172.21.15.139:0/3248745042 >> v2:172.21.15.150:3300/0 conn(0x7f5c580cd610 msgr2=0x7f5c580d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/3248745042 >> v2:172.21.15.150:3300/0 conn(0x7f5c580cd610 0x7f5c580d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c56ffd640 1 -- 172.21.15.139:0/3248745042 >> v1:172.21.15.139:6789/0 conn(0x7f5c581618c0 legacy=0x7f5c5806be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c567fc640 1 -- 172.21.15.139:0/3248745042 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839621699 0 0) 0x7f5c58099ca0 con 0x7f5c581618c0 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.345+0000 7f5c56ffd640 1 -- 172.21.15.139:0/3248745042 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5809a260 con 0x7f5c5809ad40 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.346+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/3248745042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c5809ad40 0x7f5c580d08b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f5c5806f210 tx=0x7f5c4c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853e48d5e348a26e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.346+0000 7f5c567fc640 1 -- 172.21.15.139:0/3248745042 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c037070 con 0x7f5c5809ad40 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.346+0000 7f5c567fc640 1 -- 172.21.15.139:0/3248745042 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4c02ae50 con 0x7f5c5809ad40 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.346+0000 7f5c567fc640 1 -- 172.21.15.139:0/3248745042 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c032860 con 0x7f5c5809ad40 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.346+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/3248745042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c5809ad40 msgr2=0x7f5c580d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.346+0000 7f5c5dbe0640 1 --2- 172.21.15.139:0/3248745042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c5809ad40 0x7f5c580d08b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f5c5806f210 tx=0x7f5c4c02a300 comp rx=0 tx=0).stop 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.346+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/3248745042 shutdown_connections 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.346+0000 7f5c5dbe0640 1 --2- 172.21.15.139:0/3248745042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c5809ad40 0x7f5c580d08b0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.346+0000 7f5c5dbe0640 1 --2- 172.21.15.139:0/3248745042 >> v2:172.21.15.150:3300/0 conn(0x7f5c580cd610 0x7f5c580d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.346+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/3248745042 >> 172.21.15.139:0/3248745042 conn(0x7f5c580b31f0 msgr2=0x7f5c580b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.346+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/3248745042 shutdown_connections 2024-04-25T09:59:48.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.346+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/3248745042 wait complete. 2024-04-25T09:59:48.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.348+0000 7f5c5dbe0640 1 Processor -- start 2024-04-25T09:59:48.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.348+0000 7f5c5dbe0640 1 -- start start 2024-04-25T09:59:48.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.348+0000 7f5c5dbe0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c5809ad40 0x7f5c58156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.348+0000 7f5c5dbe0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c580cd610 0x7f5c5815fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.348+0000 7f5c5dbe0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c581618c0 0x7f5c58157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.348+0000 7f5c5dbe0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580cd9f0 con 0x7f5c580cd610 2024-04-25T09:59:48.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.348+0000 7f5c5dbe0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5809a820 con 0x7f5c581618c0 2024-04-25T09:59:48.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.348+0000 7f5c5dbe0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580cdb70 con 0x7f5c5809ad40 2024-04-25T09:59:48.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c580cd610 0x7f5c5815fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c580cd610 0x7f5c5815fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44806/0 (socket says 172.21.15.139:44806) 2024-04-25T09:59:48.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c56ffd640 1 -- 172.21.15.139:0/215267205 learned_addr learned my addr 172.21.15.139:0/215267205 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c56ffd640 1 -- 172.21.15.139:0/215267205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c5809ad40 msgr2=0x7f5c58156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/215267205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c5809ad40 0x7f5c58156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c56ffd640 1 -- 172.21.15.139:0/215267205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c581618c0 msgr2=0x7f5c58157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/215267205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c581618c0 0x7f5c58157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c56ffd640 1 -- 172.21.15.139:0/215267205 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580b5b00 con 0x7f5c580cd610 2024-04-25T09:59:48.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/215267205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c580cd610 0x7f5c5815fe00 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c02a340 tx=0x7f5c4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c3ffff640 1 -- 172.21.15.139:0/215267205 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c041070 con 0x7f5c580cd610 2024-04-25T09:59:48.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c3ffff640 1 -- 172.21.15.139:0/215267205 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4c03c670 con 0x7f5c580cd610 2024-04-25T09:59:48.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/215267205 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c580b5de0 con 0x7f5c580cd610 2024-04-25T09:59:48.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c3ffff640 1 -- 172.21.15.139:0/215267205 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c037040 con 0x7f5c580cd610 2024-04-25T09:59:48.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.349+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/215267205 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c580b63a0 con 0x7f5c580cd610 2024-04-25T09:59:48.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.351+0000 7f5c3ffff640 1 -- 172.21.15.139:0/215267205 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c4c044070 con 0x7f5c580cd610 2024-04-25T09:59:48.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.351+0000 7f5c3ffff640 1 --2- 172.21.15.139:0/215267205 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5c4406b4d0 0x7f5c4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.351+0000 7f5c3ffff640 1 -- 172.21.15.139:0/215267205 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5c4c0aac50 con 0x7f5c580cd610 2024-04-25T09:59:48.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.351+0000 7f5c577fe640 1 --2- 172.21.15.139:0/215267205 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5c4406b4d0 0x7f5c4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.351+0000 7f5c577fe640 1 --2- 172.21.15.139:0/215267205 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5c4406b4d0 0x7f5c4406d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5c480098f0 tx=0x7f5c48008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.352+0000 7f5c609c0740 1 Processor -- start 2024-04-25T09:59:48.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.352+0000 7f5c609c0740 1 -- start start 2024-04-25T09:59:48.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c609c0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af9196f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c609c0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 0x5584af917970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c609c0740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 0x5584af9424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c609c0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584af87ac60 con 0x5584af917590 2024-04-25T09:59:48.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c609c0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584af819d80 con 0x5584af953c60 2024-04-25T09:59:48.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c609c0740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584af876d00 con 0x5584af942050 2024-04-25T09:59:48.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af9196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af9196f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44816/0 (socket says 172.21.15.139:44816) 2024-04-25T09:59:48.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c577fe640 1 -- 172.21.15.139:0/3105684346 learned_addr learned my addr 172.21.15.139:0/3105684346 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c57fff640 1 --2- 172.21.15.139:0/3105684346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 0x5584af9424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/3105684346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 0x5584af917970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c577fe640 1 -- 172.21.15.139:0/3105684346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 msgr2=0x5584af9424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c577fe640 1 --2- 172.21.15.139:0/3105684346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 0x5584af9424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c577fe640 1 -- 172.21.15.139:0/3105684346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 msgr2=0x5584af917970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c577fe640 1 --2- 172.21.15.139:0/3105684346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 0x5584af917970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c577fe640 1 -- 172.21.15.139:0/3105684346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584af65b7c0 con 0x5584af917590 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c577fe640 1 --2- 172.21.15.139:0/3105684346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af9196f0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800d910 tx=0x7f5c48010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef32a7081059d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c3d7fa640 1 -- 172.21.15.139:0/3105684346 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48024070 con 0x5584af917590 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c3d7fa640 1 -- 172.21.15.139:0/3105684346 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48009100 con 0x5584af917590 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.353+0000 7f5c3d7fa640 1 -- 172.21.15.139:0/3105684346 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48014dc0 con 0x5584af917590 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.354+0000 7f5c609c0740 1 -- 172.21.15.139:0/3105684346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 msgr2=0x5584af9196f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.354+0000 7f5c609c0740 1 --2- 172.21.15.139:0/3105684346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af9196f0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800d910 tx=0x7f5c48010000 comp rx=0 tx=0).stop 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.354+0000 7f5c609c0740 1 -- 172.21.15.139:0/3105684346 shutdown_connections 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.354+0000 7f5c609c0740 1 --2- 172.21.15.139:0/3105684346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 0x5584af9424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.354+0000 7f5c609c0740 1 --2- 172.21.15.139:0/3105684346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 0x5584af917970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.354+0000 7f5c609c0740 1 --2- 172.21.15.139:0/3105684346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af9196f0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.354+0000 7f5c609c0740 1 -- 172.21.15.139:0/3105684346 >> 172.21.15.139:0/3105684346 conn(0x5584af8df6e0 msgr2=0x5584af95d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.354+0000 7f5c609c0740 1 -- 172.21.15.139:0/3105684346 shutdown_connections 2024-04-25T09:59:48.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.354+0000 7f5c609c0740 1 -- 172.21.15.139:0/3105684346 wait complete. 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.355+0000 7f5c609c0740 1 -- start start 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.355+0000 7f5c609c0740 1 Processor -- start 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c609c0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af9348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c609c0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af942050 0x5584af943a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c609c0740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af953c60 0x5584af955bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c609c0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584af817840 con 0x5584af917590 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c609c0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584af93fba0 con 0x5584af942050 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c609c0740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584af96d680 con 0x5584af953c60 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c609c0740 10 client.0 _ll_register_callbacks cb 0x5584af831980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af9348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af9348f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44822/0 (socket says 172.21.15.139:44822) 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c577fe640 1 -- 172.21.15.139:0/3877745367 learned_addr learned my addr 172.21.15.139:0/3877745367 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c57fff640 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af953c60 0x5584af955bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af942050 0x5584af943a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c577fe640 1 -- 172.21.15.139:0/3877745367 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af953c60 msgr2=0x5584af955bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c577fe640 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af953c60 0x5584af955bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c577fe640 1 -- 172.21.15.139:0/3877745367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af942050 msgr2=0x5584af943a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c577fe640 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af942050 0x5584af943a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c577fe640 1 -- 172.21.15.139:0/3877745367 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584af6bbf50 con 0x5584af917590 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c577fe640 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af9348f0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009530 tx=0x7f5c48014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b37a546d7ab8648 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c367fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c367fc640 1 -- 172.21.15.139:0/3877745367 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48024040 con 0x5584af917590 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c367fc640 1 -- 172.21.15.139:0/3877745367 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5c48005dd0 con 0x5584af917590 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c609c0740 1 -- 172.21.15.139:0/3877745367 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584af83ae60 con 0x5584af917590 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c367fc640 1 -- 172.21.15.139:0/3877745367 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c48006040 con 0x5584af917590 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c609c0740 10 client.7947 fetch_fsmap learned FSMap version 57 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.356+0000 7f5c609c0740 1 -- 172.21.15.139:0/3877745367 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5584af54f0b0 con 0x5584af917590 2024-04-25T09:59:48.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.357+0000 7f5c367fc640 1 -- 172.21.15.139:0/3877745367 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5584af83ae60 con 0x5584af917590 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.357+0000 7f5c367fc640 1 -- 172.21.15.139:0/3877745367 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5c48013070 con 0x5584af917590 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.357+0000 7f5c609c0740 10 client.7947 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.357+0000 7f5c609c0740 10 client.7947 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.357+0000 7f5c609c0740 20 client.7947 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.357+0000 7f5c609c0740 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5584af957d60 0x5584af958140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.357+0000 7f5c609c0740 4 client.7947 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.357+0000 7f5c609c0740 1 -- 172.21.15.139:0/3877745367 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584af8881a0 con 0x5584af957d60 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.357+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5584af957d60 0x5584af958140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.357+0000 7f5c367fc640 10 client.7947 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.357+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5584af957d60 0x5584af958140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752ae62ad23049dc server_cookie=4b8239897717524c in_seq=0 out_seq=0 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c367fc640 1 -- 172.21.15.139:0/3877745367 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5584af8881a0 con 0x5584af957d60 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c367fc640 10 client.7947 handle_command_reply: tid=0 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c609c0740 1 client.7947 shutdown 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c609c0740 1 -- 172.21.15.139:0/3877745367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 msgr2=0x5584af9348f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c609c0740 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af9348f0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009530 tx=0x7f5c48014000 comp rx=0 tx=0).stop 2024-04-25T09:59:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c609c0740 1 -- 172.21.15.139:0/3877745367 shutdown_connections 2024-04-25T09:59:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c609c0740 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5584af957d60 0x5584af958140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c609c0740 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af953c60 0x5584af955bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c609c0740 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af942050 0x5584af943a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c367fc640 0 client.7947 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c609c0740 1 --2- 172.21.15.139:0/3877745367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af9348f0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c609c0740 1 -- 172.21.15.139:0/3877745367 >> 172.21.15.139:0/3877745367 conn(0x5584af8df6e0 msgr2=0x5584af93fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c609c0740 1 -- 172.21.15.139:0/3877745367 shutdown_connections 2024-04-25T09:59:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c609c0740 1 -- 172.21.15.139:0/3877745367 wait complete. 2024-04-25T09:59:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.358+0000 7f5c609c0740 20 client.7947 trim_cache size 0 max 16384 2024-04-25T09:59:48.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.369+0000 7f5c609c0740 1 Processor -- start 2024-04-25T09:59:48.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.370+0000 7f5c609c0740 1 -- start start 2024-04-25T09:59:48.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.370+0000 7f5c609c0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af933f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.370+0000 7f5c609c0740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 0x5584af90dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.370+0000 7f5c609c0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 0x5584af96fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.370+0000 7f5c609c0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c581ff620 con 0x5584af917590 2024-04-25T09:59:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.370+0000 7f5c609c0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b46e0 con 0x5584af953c60 2024-04-25T09:59:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.370+0000 7f5c609c0740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b4ce0 con 0x5584af942050 2024-04-25T09:59:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.370+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af933f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.370+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af933f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44824/0 (socket says 172.21.15.139:44824) 2024-04-25T09:59:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.370+0000 7f5c577fe640 1 -- 172.21.15.139:0/1908469660 learned_addr learned my addr 172.21.15.139:0/1908469660 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c57fff640 1 --2- 172.21.15.139:0/1908469660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 0x5584af96fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/1908469660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 0x5584af90dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c577fe640 1 -- 172.21.15.139:0/1908469660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 msgr2=0x5584af90dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c577fe640 1 --2- 172.21.15.139:0/1908469660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 0x5584af90dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c577fe640 1 -- 172.21.15.139:0/1908469660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 msgr2=0x5584af96fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c577fe640 1 --2- 172.21.15.139:0/1908469660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 0x5584af96fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c577fe640 1 -- 172.21.15.139:0/1908469660 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584af9156c0 con 0x5584af917590 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c577fe640 1 --2- 172.21.15.139:0/1908469660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af933f80 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009500 tx=0x7f5c4801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df60f42f2f108ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c3cff9640 1 -- 172.21.15.139:0/1908469660 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48014800 con 0x5584af917590 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c3cff9640 1 -- 172.21.15.139:0/1908469660 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c480066b0 con 0x5584af917590 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c3cff9640 1 -- 172.21.15.139:0/1908469660 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480068a0 con 0x5584af917590 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c609c0740 1 -- 172.21.15.139:0/1908469660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 msgr2=0x5584af933f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.371+0000 7f5c609c0740 1 --2- 172.21.15.139:0/1908469660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af933f80 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009500 tx=0x7f5c4801d000 comp rx=0 tx=0).stop 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.372+0000 7f5c609c0740 1 -- 172.21.15.139:0/1908469660 shutdown_connections 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.372+0000 7f5c609c0740 1 --2- 172.21.15.139:0/1908469660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 0x5584af96fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.372+0000 7f5c609c0740 1 --2- 172.21.15.139:0/1908469660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 0x5584af90dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.372+0000 7f5c609c0740 1 --2- 172.21.15.139:0/1908469660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af933f80 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.372+0000 7f5c609c0740 1 -- 172.21.15.139:0/1908469660 >> 172.21.15.139:0/1908469660 conn(0x5584af8df6e0 msgr2=0x5584af919570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.372+0000 7f5c609c0740 1 -- 172.21.15.139:0/1908469660 shutdown_connections 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.372+0000 7f5c609c0740 1 -- 172.21.15.139:0/1908469660 wait complete. 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.372+0000 7f5c609c0740 1 -- start start 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.372+0000 7f5c609c0740 1 Processor -- start 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c609c0740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af96ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c609c0740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 0x5584af940790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c609c0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 0x5584af96cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c609c0740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584af9562c0 con 0x5584af917590 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c609c0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584af944e80 con 0x5584af953c60 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c609c0740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584af970a20 con 0x5584af942050 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c609c0740 10 client.0 _ll_register_callbacks cb 0x7f5c48009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af96ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af96ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44840/0 (socket says 172.21.15.139:44840) 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c577fe640 1 -- 172.21.15.139:0/2564525072 learned_addr learned my addr 172.21.15.139:0/2564525072 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 0x5584af940790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c57fff640 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 0x5584af96cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c577fe640 1 -- 172.21.15.139:0/2564525072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 msgr2=0x5584af940790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c577fe640 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 0x5584af940790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c577fe640 1 -- 172.21.15.139:0/2564525072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 msgr2=0x5584af96cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c577fe640 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 0x5584af96cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c577fe640 1 -- 172.21.15.139:0/2564525072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584af96d3e0 con 0x5584af917590 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c577fe640 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af96ffb0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006dc0 tx=0x7f5c48006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ecb003bc7f6700 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c367fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.373+0000 7f5c367fc640 1 -- 172.21.15.139:0/2564525072 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48044060 con 0x5584af917590 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c367fc640 1 -- 172.21.15.139:0/2564525072 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5c4802aa60 con 0x5584af917590 2024-04-25T09:59:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c609c0740 1 -- 172.21.15.139:0/2564525072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c480066b0 con 0x5584af917590 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c367fc640 1 -- 172.21.15.139:0/2564525072 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c4804a050 con 0x5584af917590 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c609c0740 10 client.7953 fetch_fsmap learned FSMap version 57 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c609c0740 1 -- 172.21.15.139:0/2564525072 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5584af9cac50 con 0x5584af917590 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c367fc640 1 -- 172.21.15.139:0/2564525072 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c480066b0 con 0x5584af917590 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c367fc640 1 -- 172.21.15.139:0/2564525072 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5c4804d020 con 0x5584af917590 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c609c0740 10 client.7953 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c609c0740 10 client.7953 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c609c0740 20 client.7953 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c609c0740 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5584af9c7d20 0x5584af942430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c609c0740 4 client.7953 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c609c0740 1 -- 172.21.15.139:0/2564525072 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5584af8824f0 con 0x5584af9c7d20 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5584af9c7d20 0x5584af942430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c367fc640 10 client.7953 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.374+0000 7f5c56ffd640 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5584af9c7d20 0x5584af942430 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57127cbf212f2c1f server_cookie=ebd30686a75a7a69 in_seq=0 out_seq=0 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c367fc640 1 -- 172.21.15.139:0/2564525072 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+420 (crc 0 0 0) 0x5584af8824f0 con 0x5584af9c7d20 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c367fc640 10 client.7953 handle_command_reply: tid=0 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c609c0740 1 client.7953 shutdown 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c609c0740 1 -- 172.21.15.139:0/2564525072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 msgr2=0x5584af96ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c609c0740 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af96ffb0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006dc0 tx=0x7f5c48006e10 comp rx=0 tx=0).stop 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c609c0740 1 -- 172.21.15.139:0/2564525072 shutdown_connections 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c609c0740 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5584af9c7d20 0x5584af942430 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c609c0740 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584af953c60 0x5584af96cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c367fc640 0 client.7953 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c609c0740 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5584af942050 0x5584af940790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c609c0740 1 --2- 172.21.15.139:0/2564525072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5584af917590 0x5584af96ffb0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c609c0740 1 -- 172.21.15.139:0/2564525072 >> 172.21.15.139:0/2564525072 conn(0x5584af8df6e0 msgr2=0x5584af97fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c609c0740 1 -- 172.21.15.139:0/2564525072 shutdown_connections 2024-04-25T09:59:48.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c609c0740 1 -- 172.21.15.139:0/2564525072 wait complete. 2024-04-25T09:59:48.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c609c0740 20 client.7953 trim_cache size 0 max 16384 2024-04-25T09:59:48.388 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:48.388 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-04-25T09:59:48.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:59:48.388 INFO:teuthology.orchestra.run.smithi139.stdout: "damage_type": "dir_frag", 2024-04-25T09:59:48.389 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 2205095346, 2024-04-25T09:59:48.389 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 1099511627776, 2024-04-25T09:59:48.389 INFO:teuthology.orchestra.run.smithi139.stdout: "frag": "*", 2024-04-25T09:59:48.389 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/dir" 2024-04-25T09:59:48.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:59:48.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:59:48.389 INFO:teuthology.orchestra.run.smithi139.stdout: "damage_type": "backtrace", 2024-04-25T09:59:48.389 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 2335747626, 2024-04-25T09:59:48.389 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 1099511627777, 2024-04-25T09:59:48.390 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/testdir/hardlink" 2024-04-25T09:59:48.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T09:59:48.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T09:59:48.390 INFO:teuthology.orchestra.run.smithi139.stdout: "damage_type": "backtrace", 2024-04-25T09:59:48.390 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 3538552845, 2024-04-25T09:59:48.390 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 1099511627776, 2024-04-25T09:59:48.390 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/dir" 2024-04-25T09:59:48.390 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:59:48.390 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-04-25T09:59:48.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/215267205 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5c4406b4d0 msgr2=0x7f5c4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c5dbe0640 1 --2- 172.21.15.139:0/215267205 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5c4406b4d0 0x7f5c4406d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5c480098f0 tx=0x7f5c48008040 comp rx=0 tx=0).stop 2024-04-25T09:59:48.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/215267205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c580cd610 msgr2=0x7f5c5815fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c5dbe0640 1 --2- 172.21.15.139:0/215267205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c580cd610 0x7f5c5815fe00 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c02a340 tx=0x7f5c4c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/215267205 shutdown_connections 2024-04-25T09:59:48.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c5dbe0640 1 --2- 172.21.15.139:0/215267205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c580cd610 0x7f5c5815fe00 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c5dbe0640 1 --2- 172.21.15.139:0/215267205 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5c4406b4d0 0x7f5c4406d990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c5dbe0640 1 --2- 172.21.15.139:0/215267205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c581618c0 0x7f5c58157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c5dbe0640 1 --2- 172.21.15.139:0/215267205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c5809ad40 0x7f5c58156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/215267205 >> 172.21.15.139:0/215267205 conn(0x7f5c580b31f0 msgr2=0x7f5c58070b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.375+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/215267205 shutdown_connections 2024-04-25T09:59:48.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.376+0000 7f5c5dbe0640 1 -- 172.21.15.139:0/215267205 wait complete. 2024-04-25T09:59:48.393 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T09:59:48.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.484+0000 7f3458691640 1 Processor -- start 2024-04-25T09:59:48.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.484+0000 7f3458691640 1 -- start start 2024-04-25T09:59:48.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.485+0000 7f3458691640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f345015dc60 0x7f34500b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.485+0000 7f3458691640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34500d1080 0x7f34500ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.485+0000 7f3458691640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34500cf250 con 0x7f3450161850 2024-04-25T09:59:48.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.485+0000 7f3458691640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34500ce780 con 0x7f345015dc60 2024-04-25T09:59:48.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.485+0000 7f3458691640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34500d3730 con 0x7f34500d1080 2024-04-25T09:59:48.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.485+0000 7f345642d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f345015dc60 0x7f34500b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.485+0000 7f345642d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f345015dc60 0x7f34500b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44852/0 (socket says 172.21.15.139:44852) 2024-04-25T09:59:48.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.485+0000 7f345642d640 1 -- 172.21.15.139:0/3478226666 learned_addr learned my addr 172.21.15.139:0/3478226666 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.486+0000 7f3456c2e640 1 --2- 172.21.15.139:0/3478226666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34500d1080 0x7f34500ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.486+0000 7f345642d640 1 -- 172.21.15.139:0/3478226666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34500d1080 msgr2=0x7f34500ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.486+0000 7f345642d640 1 --2- 172.21.15.139:0/3478226666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34500d1080 0x7f34500ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.486+0000 7f345642d640 1 -- 172.21.15.139:0/3478226666 >> v1:172.21.15.150:6789/0 conn(0x7f3450161850 legacy=0x7f34500d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.486+0000 7f345642d640 1 -- 172.21.15.139:0/3478226666 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500ce9b0 con 0x7f345015dc60 2024-04-25T09:59:48.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.486+0000 7f345642d640 1 --2- 172.21.15.139:0/3478226666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f345015dc60 0x7f34500b6c10 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f344c009870 tx=0x7f344c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28db0d91e77f4a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.486+0000 7f345542b640 1 -- 172.21.15.139:0/3478226666 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c037070 con 0x7f345015dc60 2024-04-25T09:59:48.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.487+0000 7f345542b640 1 -- 172.21.15.139:0/3478226666 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c032440 con 0x7f345015dc60 2024-04-25T09:59:48.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.487+0000 7f345542b640 1 -- 172.21.15.139:0/3478226666 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c032810 con 0x7f345015dc60 2024-04-25T09:59:48.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.487+0000 7f3458691640 1 -- 172.21.15.139:0/3478226666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f345015dc60 msgr2=0x7f34500b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.487+0000 7f3458691640 1 --2- 172.21.15.139:0/3478226666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f345015dc60 0x7f34500b6c10 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f344c009870 tx=0x7f344c02a270 comp rx=0 tx=0).stop 2024-04-25T09:59:48.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.487+0000 7f3458691640 1 -- 172.21.15.139:0/3478226666 shutdown_connections 2024-04-25T09:59:48.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.487+0000 7f3458691640 1 --2- 172.21.15.139:0/3478226666 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34500d1080 0x7f34500ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.487+0000 7f3458691640 1 --2- 172.21.15.139:0/3478226666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f345015dc60 0x7f34500b6c10 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.487+0000 7f3458691640 1 -- 172.21.15.139:0/3478226666 >> 172.21.15.139:0/3478226666 conn(0x7f34500b5910 msgr2=0x7f34500b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.487+0000 7f3458691640 1 -- 172.21.15.139:0/3478226666 shutdown_connections 2024-04-25T09:59:48.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.487+0000 7f3458691640 1 -- 172.21.15.139:0/3478226666 wait complete. 2024-04-25T09:59:48.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.488+0000 7f3458691640 1 Processor -- start 2024-04-25T09:59:48.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.488+0000 7f3458691640 1 -- start start 2024-04-25T09:59:48.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3458691640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34500d1080 0x7f34501f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3458691640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f345015dc60 0x7f3450145070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3458691640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3450161850 0x7f34500d3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3458691640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34500d1fc0 con 0x7f345015dc60 2024-04-25T09:59:48.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3458691640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34500cef70 con 0x7f34500d1080 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3458691640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345015e040 con 0x7f3450161850 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3455c2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f345015dc60 0x7f3450145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3455c2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f345015dc60 0x7f3450145070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44866/0 (socket says 172.21.15.139:44866) 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3455c2c640 1 -- 172.21.15.139:0/2928030094 learned_addr learned my addr 172.21.15.139:0/2928030094 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f345642d640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34500d1080 0x7f34501f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3456c2e640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3450161850 0x7f34500d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3455c2c640 1 -- 172.21.15.139:0/2928030094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3450161850 msgr2=0x7f34500d3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3455c2c640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3450161850 0x7f34500d3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3455c2c640 1 -- 172.21.15.139:0/2928030094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34500d1080 msgr2=0x7f34501f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.489+0000 7f3455c2c640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34500d1080 0x7f34501f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.490+0000 7f3455c2c640 1 -- 172.21.15.139:0/2928030094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34501f43d0 con 0x7f345015dc60 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.490+0000 7f345642d640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34500d1080 0x7f34501f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.490+0000 7f3455c2c640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f345015dc60 0x7f3450145070 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f344000c3f0 tx=0x7f344000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.490+0000 7f3446ffd640 1 -- 172.21.15.139:0/2928030094 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34400052d0 con 0x7f345015dc60 2024-04-25T09:59:48.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.490+0000 7f3446ffd640 1 -- 172.21.15.139:0/2928030094 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3440005490 con 0x7f345015dc60 2024-04-25T09:59:48.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.490+0000 7f3446ffd640 1 -- 172.21.15.139:0/2928030094 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440007260 con 0x7f345015dc60 2024-04-25T09:59:48.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.490+0000 7f3458691640 1 -- 172.21.15.139:0/2928030094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34501f46b0 con 0x7f345015dc60 2024-04-25T09:59:48.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.490+0000 7f3458691640 1 -- 172.21.15.139:0/2928030094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3450142b00 con 0x7f345015dc60 2024-04-25T09:59:48.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.491+0000 7f3458691640 1 -- 172.21.15.139:0/2928030094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34500b8820 con 0x7f345015dc60 2024-04-25T09:59:48.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.491+0000 7f3446ffd640 1 -- 172.21.15.139:0/2928030094 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3440007400 con 0x7f345015dc60 2024-04-25T09:59:48.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.492+0000 7f3446ffd640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f343406b4d0 0x7f343406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.492+0000 7f3446ffd640 1 -- 172.21.15.139:0/2928030094 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f34400898c0 con 0x7f345015dc60 2024-04-25T09:59:48.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.492+0000 7f345642d640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f343406b4d0 0x7f343406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.492+0000 7f345642d640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f343406b4d0 0x7f343406d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f344c034000 tx=0x7f344c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.496+0000 7f3446ffd640 1 -- 172.21.15.139:0/2928030094 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34500b8820 con 0x7f345015dc60 2024-04-25T09:59:48.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.704+0000 7f3458691640 1 -- 172.21.15.139:0/2928030094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3450074520 con 0x7f345015dc60 2024-04-25T09:59:48.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.705+0000 7f3446ffd640 1 -- 172.21.15.139:0/2928030094 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f34500cf250 con 0x7f345015dc60 2024-04-25T09:59:48.709 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T09:59:48.709 INFO:teuthology.orchestra.run.smithi139.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T09:59:48.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.707+0000 7f3458691640 1 -- 172.21.15.139:0/2928030094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f343406b4d0 msgr2=0x7f343406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.707+0000 7f3458691640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f343406b4d0 0x7f343406d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f344c034000 tx=0x7f344c02e000 comp rx=0 tx=0).stop 2024-04-25T09:59:48.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.707+0000 7f3458691640 1 -- 172.21.15.139:0/2928030094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f345015dc60 msgr2=0x7f3450145070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.707+0000 7f3458691640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f345015dc60 0x7f3450145070 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f344000c3f0 tx=0x7f344000c8b0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.707+0000 7f3458691640 1 -- 172.21.15.139:0/2928030094 shutdown_connections 2024-04-25T09:59:48.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.707+0000 7f3458691640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f345015dc60 0x7f3450145070 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.707+0000 7f3458691640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f343406b4d0 0x7f343406d990 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.707+0000 7f3458691640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3450161850 0x7f34500d3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.707+0000 7f3458691640 1 --2- 172.21.15.139:0/2928030094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34500d1080 0x7f34501f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.707+0000 7f3458691640 1 -- 172.21.15.139:0/2928030094 >> 172.21.15.139:0/2928030094 conn(0x7f34500b5910 msgr2=0x7f34500b75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.707+0000 7f3458691640 1 -- 172.21.15.139:0/2928030094 shutdown_connections 2024-04-25T09:59:48.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.707+0000 7f3458691640 1 -- 172.21.15.139:0/2928030094 wait complete. 2024-04-25T09:59:48.721 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T09:59:48.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:48.721 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T09:59:48.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.818+0000 7f445ab09640 1 Processor -- start 2024-04-25T09:59:48.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.818+0000 7f445ab09640 1 -- start start 2024-04-25T09:59:48.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.818+0000 7f445ab09640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f44540af8a0 0x7f44540b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.818+0000 7f445ab09640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f44540afc80 0x7f44540cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.818+0000 7f445ab09640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f445406b700 0x7f445406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.818+0000 7f445ab09640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44540c6cc0 con 0x7f44540afc80 2024-04-25T09:59:48.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.818+0000 7f445ab09640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44540c6fa0 con 0x7f44540af8a0 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.818+0000 7f445ab09640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540c7280 con 0x7f445406b700 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.819+0000 7f4453fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f44540afc80 0x7f44540cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.819+0000 7f4453fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f44540afc80 0x7f44540cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44874/0 (socket says 172.21.15.139:44874) 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.819+0000 7f4453fff640 1 -- 172.21.15.139:0/2270251735 learned_addr learned my addr 172.21.15.139:0/2270251735 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.819+0000 7f44590a6640 1 --2- 172.21.15.139:0/2270251735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f445406b700 0x7f445406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.819+0000 7f44588a5640 1 --2- 172.21.15.139:0/2270251735 >> v2:172.21.15.150:3300/0 conn(0x7f44540af8a0 0x7f44540b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.819+0000 7f4453fff640 1 -- 172.21.15.139:0/2270251735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f445406b700 msgr2=0x7f445406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.819+0000 7f4453fff640 1 --2- 172.21.15.139:0/2270251735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f445406b700 0x7f445406ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.819+0000 7f4453fff640 1 -- 172.21.15.139:0/2270251735 >> v2:172.21.15.150:3300/0 conn(0x7f44540af8a0 msgr2=0x7f44540b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.819+0000 7f4453fff640 1 --2- 172.21.15.139:0/2270251735 >> v2:172.21.15.150:3300/0 conn(0x7f44540af8a0 0x7f44540b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.819+0000 7f4453fff640 1 -- 172.21.15.139:0/2270251735 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540cd090 con 0x7f44540afc80 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.819+0000 7f44588a5640 1 --2- 172.21.15.139:0/2270251735 >> v2:172.21.15.150:3300/0 conn(0x7f44540af8a0 0x7f44540b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.819+0000 7f4453fff640 1 --2- 172.21.15.139:0/2270251735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44540afc80 0x7f44540cd890 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f4454070aa0 tx=0x7f444002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a93b812a5eeb80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.820+0000 7f44537fe640 1 -- 172.21.15.139:0/2270251735 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440037070 con 0x7f44540afc80 2024-04-25T09:59:48.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.820+0000 7f44537fe640 1 -- 172.21.15.139:0/2270251735 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4440032440 con 0x7f44540afc80 2024-04-25T09:59:48.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.820+0000 7f44537fe640 1 -- 172.21.15.139:0/2270251735 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440032810 con 0x7f44540afc80 2024-04-25T09:59:48.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.820+0000 7f445ab09640 1 -- 172.21.15.139:0/2270251735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44540afc80 msgr2=0x7f44540cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.820+0000 7f445ab09640 1 --2- 172.21.15.139:0/2270251735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44540afc80 0x7f44540cd890 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f4454070aa0 tx=0x7f444002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.820+0000 7f445ab09640 1 -- 172.21.15.139:0/2270251735 shutdown_connections 2024-04-25T09:59:48.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.820+0000 7f445ab09640 1 --2- 172.21.15.139:0/2270251735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f445406b700 0x7f445406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.820+0000 7f445ab09640 1 --2- 172.21.15.139:0/2270251735 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44540afc80 0x7f44540cd890 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.820+0000 7f445ab09640 1 --2- 172.21.15.139:0/2270251735 >> v2:172.21.15.150:3300/0 conn(0x7f44540af8a0 0x7f44540b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.820+0000 7f445ab09640 1 -- 172.21.15.139:0/2270251735 >> 172.21.15.139:0/2270251735 conn(0x7f44540b2230 msgr2=0x7f44540b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.820+0000 7f445ab09640 1 -- 172.21.15.139:0/2270251735 shutdown_connections 2024-04-25T09:59:48.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.820+0000 7f445ab09640 1 -- 172.21.15.139:0/2270251735 wait complete. 2024-04-25T09:59:48.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.821+0000 7f445ab09640 1 Processor -- start 2024-04-25T09:59:48.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.821+0000 7f445ab09640 1 -- start start 2024-04-25T09:59:48.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.822+0000 7f445ab09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f445406b700 0x7f44541f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.822+0000 7f445ab09640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44540af8a0 0x7f445415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.822+0000 7f445ab09640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44540afc80 0x7f44540adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.822+0000 7f445ab09640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445406b0a0 con 0x7f445406b700 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.822+0000 7f445ab09640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540cddd0 con 0x7f44540af8a0 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.822+0000 7f445ab09640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445406b220 con 0x7f44540afc80 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.822+0000 7f44588a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f445406b700 0x7f44541f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.822+0000 7f44588a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f445406b700 0x7f44541f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44886/0 (socket says 172.21.15.139:44886) 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f44588a5640 1 -- 172.21.15.139:0/2480077896 learned_addr learned my addr 172.21.15.139:0/2480077896 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f4453fff640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44540af8a0 0x7f445415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f44590a6640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44540afc80 0x7f44540adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f44588a5640 1 -- 172.21.15.139:0/2480077896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44540afc80 msgr2=0x7f44540adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f44588a5640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44540afc80 0x7f44540adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f44588a5640 1 -- 172.21.15.139:0/2480077896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44540af8a0 msgr2=0x7f445415b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f44588a5640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44540af8a0 0x7f445415b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f44588a5640 1 -- 172.21.15.139:0/2480077896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445409b5f0 con 0x7f445406b700 2024-04-25T09:59:48.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f44588a5640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f445406b700 0x7f44541f4400 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f444800c7f0 tx=0x7f444800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f44517fa640 1 -- 172.21.15.139:0/2480077896 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448015070 con 0x7f445406b700 2024-04-25T09:59:48.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f44517fa640 1 -- 172.21.15.139:0/2480077896 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4448007a30 con 0x7f445406b700 2024-04-25T09:59:48.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f445ab09640 1 -- 172.21.15.139:0/2480077896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445409e350 con 0x7f445406b700 2024-04-25T09:59:48.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f44517fa640 1 -- 172.21.15.139:0/2480077896 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448007e00 con 0x7f445406b700 2024-04-25T09:59:48.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.823+0000 7f445ab09640 1 -- 172.21.15.139:0/2480077896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445409bb70 con 0x7f445406b700 2024-04-25T09:59:48.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.825+0000 7f44517fa640 1 -- 172.21.15.139:0/2480077896 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4448018070 con 0x7f445406b700 2024-04-25T09:59:48.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.825+0000 7f44517fa640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f443806b4d0 0x7f443806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.825+0000 7f44517fa640 1 -- 172.21.15.139:0/2480077896 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f444808a350 con 0x7f445406b700 2024-04-25T09:59:48.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.825+0000 7f4453fff640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f443806b4d0 0x7f443806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.826+0000 7f4453fff640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f443806b4d0 0x7f443806d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4440034000 tx=0x7f444002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.826+0000 7f445d811740 1 Processor -- start 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.826+0000 7f445d811740 1 -- start start 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f445d811740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc5abb0 0x55fe4cc5af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f445d811740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc55830 0x55fe4cc597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f445d811740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc59cf0 0x55fe4cc5a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f445d811740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4cbbeac0 con 0x55fe4cc55830 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f445d811740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4cb67d80 con 0x55fe4cc59cf0 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f445d811740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe4cbbab60 con 0x55fe4cc5abb0 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f4453fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc55830 0x55fe4cc597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f4453fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc55830 0x55fe4cc597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44888/0 (socket says 172.21.15.139:44888) 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f4453fff640 1 -- 172.21.15.139:0/2076018248 learned_addr learned my addr 172.21.15.139:0/2076018248 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f44590a6640 1 --2- 172.21.15.139:0/2076018248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc59cf0 0x55fe4cc5a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f44588a5640 1 --2- 172.21.15.139:0/2076018248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc5abb0 0x55fe4cc5af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f4453fff640 1 -- 172.21.15.139:0/2076018248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc5abb0 msgr2=0x55fe4cc5af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f4453fff640 1 --2- 172.21.15.139:0/2076018248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc5abb0 0x55fe4cc5af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f4453fff640 1 -- 172.21.15.139:0/2076018248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc59cf0 msgr2=0x55fe4cc5a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f4453fff640 1 --2- 172.21.15.139:0/2076018248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc59cf0 0x55fe4cc5a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f4453fff640 1 -- 172.21.15.139:0/2076018248 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe4c99b7e0 con 0x55fe4cc55830 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f4453fff640 1 --2- 172.21.15.139:0/2076018248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc55830 0x55fe4cc597b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f4440000c00 tx=0x7f444002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835363842209a268 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f443e7fc640 1 -- 172.21.15.139:0/2076018248 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444004e070 con 0x55fe4cc55830 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f443e7fc640 1 -- 172.21.15.139:0/2076018248 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4440032e00 con 0x55fe4cc55830 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.827+0000 7f443e7fc640 1 -- 172.21.15.139:0/2076018248 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440037040 con 0x55fe4cc55830 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.828+0000 7f445d811740 1 -- 172.21.15.139:0/2076018248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc55830 msgr2=0x55fe4cc597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.828+0000 7f445d811740 1 --2- 172.21.15.139:0/2076018248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc55830 0x55fe4cc597b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f4440000c00 tx=0x7f444002c000 comp rx=0 tx=0).stop 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.828+0000 7f445d811740 1 -- 172.21.15.139:0/2076018248 shutdown_connections 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.828+0000 7f445d811740 1 --2- 172.21.15.139:0/2076018248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc55830 0x55fe4cc597b0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.828+0000 7f445d811740 1 --2- 172.21.15.139:0/2076018248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc59cf0 0x55fe4cc5a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.828+0000 7f445d811740 1 --2- 172.21.15.139:0/2076018248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc5abb0 0x55fe4cc5af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.828+0000 7f445d811740 1 -- 172.21.15.139:0/2076018248 >> 172.21.15.139:0/2076018248 conn(0x55fe4cc57cb0 msgr2=0x55fe4cca0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.828+0000 7f445d811740 1 -- 172.21.15.139:0/2076018248 shutdown_connections 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.828+0000 7f445d811740 1 -- 172.21.15.139:0/2076018248 wait complete. 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.828+0000 7f445d811740 1 -- start start 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.829+0000 7f445d811740 1 Processor -- start 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.829+0000 7f445d811740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc59cf0 0x55fe4cca22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.829+0000 7f445d811740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc5abb0 0x55fe4cca3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.829+0000 7f445d811740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc71b00 0x55fe4cca4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.829+0000 7f445d811740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4cc86740 con 0x55fe4cc71b00 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.829+0000 7f445d811740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4cb5d530 con 0x55fe4cc5abb0 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.829+0000 7f445d811740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe4cc85bc0 con 0x55fe4cc59cf0 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f445d811740 10 client.0 _ll_register_callbacks cb 0x55fe4cbb5130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44590a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc71b00 0x55fe4cca4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44590a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc71b00 0x55fe4cca4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44896/0 (socket says 172.21.15.139:44896) 2024-04-25T09:59:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44590a6640 1 -- 172.21.15.139:0/1745266499 learned_addr learned my addr 172.21.15.139:0/1745266499 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44588a5640 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc59cf0 0x55fe4cca22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f4453fff640 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc5abb0 0x55fe4cca3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44590a6640 1 -- 172.21.15.139:0/1745266499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc59cf0 msgr2=0x55fe4cca22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44590a6640 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc59cf0 0x55fe4cca22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44590a6640 1 -- 172.21.15.139:0/1745266499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc5abb0 msgr2=0x55fe4cca3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44590a6640 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc5abb0 0x55fe4cca3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44590a6640 1 -- 172.21.15.139:0/1745266499 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe4c9fbf70 con 0x55fe4cc71b00 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44588a5640 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc59cf0 0x55fe4cca22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44590a6640 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc71b00 0x55fe4cca4280 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f444c007a00 tx=0x7f444c007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a254ea6e6084c17f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44337fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44337fe640 1 -- 172.21.15.139:0/1745266499 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c01b020 con 0x55fe4cc71b00 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44337fe640 1 -- 172.21.15.139:0/1745266499 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f444c011a80 con 0x55fe4cc71b00 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f445d811740 1 -- 172.21.15.139:0/1745266499 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4440032e00 con 0x55fe4cc71b00 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44337fe640 1 -- 172.21.15.139:0/1745266499 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f444c025050 con 0x55fe4cc71b00 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f445d811740 10 client.7971 fetch_fsmap learned FSMap version 57 2024-04-25T09:59:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f445d811740 1 -- 172.21.15.139:0/1745266499 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55fe4c893170 con 0x55fe4cc71b00 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.830+0000 7f44337fe640 1 -- 172.21.15.139:0/1745266499 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4440032e00 con 0x55fe4cc71b00 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f44337fe640 1 -- 172.21.15.139:0/1745266499 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f444c028020 con 0x55fe4cc71b00 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f445d811740 10 client.7971 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f445d811740 10 client.7971 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f445d811740 20 client.7971 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f445d811740 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55fe4cc77a50 0x55fe4cc77ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f445d811740 4 client.7971 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f445d811740 1 -- 172.21.15.139:0/1745266499 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe4cbcc000 con 0x55fe4cc77a50 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f44588a5640 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55fe4cc77a50 0x55fe4cc77ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f44337fe640 10 client.7971 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f44588a5640 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55fe4cc77a50 0x55fe4cc77ea0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f59ade4f3e9db99 server_cookie=ce8b47b679f8829b in_seq=0 out_seq=0 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f44337fe640 1 -- 172.21.15.139:0/1745266499 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe4cbcc000 con 0x55fe4cc77a50 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f44337fe640 10 client.7971 handle_command_reply: tid=0 2024-04-25T09:59:48.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.831+0000 7f445d811740 1 client.7971 shutdown 2024-04-25T09:59:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.832+0000 7f445d811740 1 -- 172.21.15.139:0/1745266499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc71b00 msgr2=0x55fe4cca4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.832+0000 7f445d811740 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc71b00 0x55fe4cca4280 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f444c007a00 tx=0x7f444c007ec0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.832+0000 7f445d811740 1 -- 172.21.15.139:0/1745266499 shutdown_connections 2024-04-25T09:59:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.832+0000 7f445d811740 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55fe4cc77a50 0x55fe4cc77ea0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.832+0000 7f445d811740 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc71b00 0x55fe4cca4280 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.832+0000 7f445d811740 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc5abb0 0x55fe4cca3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.832+0000 7f44337fe640 0 client.7971 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.832+0000 7f445d811740 1 --2- 172.21.15.139:0/1745266499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc59cf0 0x55fe4cca22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.832+0000 7f445d811740 1 -- 172.21.15.139:0/1745266499 >> 172.21.15.139:0/1745266499 conn(0x55fe4cc55830 msgr2=0x55fe4cc5b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.832+0000 7f445d811740 1 -- 172.21.15.139:0/1745266499 shutdown_connections 2024-04-25T09:59:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.832+0000 7f445d811740 1 -- 172.21.15.139:0/1745266499 wait complete. 2024-04-25T09:59:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.832+0000 7f445d811740 20 client.7971 trim_cache size 0 max 16384 2024-04-25T09:59:48.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.847+0000 7f445d811740 1 Processor -- start 2024-04-25T09:59:48.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.847+0000 7f445d811740 1 -- start start 2024-04-25T09:59:48.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.847+0000 7f445d811740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc59cf0 0x55fe4cca1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.847+0000 7f445d811740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 0x55fe4cca1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.847+0000 7f445d811740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc71b00 0x55fe4cc77e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.847+0000 7f445d811740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454200460 con 0x55fe4cc5abb0 2024-04-25T09:59:48.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.847+0000 7f445d811740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44541ff5e0 con 0x55fe4cc71b00 2024-04-25T09:59:48.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.847+0000 7f445d811740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44541fea80 con 0x55fe4cc59cf0 2024-04-25T09:59:48.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.847+0000 7f4453fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 0x55fe4cca1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.847+0000 7f4453fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 0x55fe4cca1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44898/0 (socket says 172.21.15.139:44898) 2024-04-25T09:59:48.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f4453fff640 1 -- 172.21.15.139:0/4206947961 learned_addr learned my addr 172.21.15.139:0/4206947961 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f44588a5640 1 --2- 172.21.15.139:0/4206947961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc59cf0 0x55fe4cca1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f44590a6640 1 --2- 172.21.15.139:0/4206947961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc71b00 0x55fe4cc77e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f4453fff640 1 -- 172.21.15.139:0/4206947961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc59cf0 msgr2=0x55fe4cca1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f4453fff640 1 --2- 172.21.15.139:0/4206947961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc59cf0 0x55fe4cca1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f4453fff640 1 -- 172.21.15.139:0/4206947961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc71b00 msgr2=0x55fe4cc77e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f4453fff640 1 --2- 172.21.15.139:0/4206947961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc71b00 0x55fe4cc77e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f4453fff640 1 -- 172.21.15.139:0/4206947961 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe4cc51750 con 0x55fe4cc5abb0 2024-04-25T09:59:48.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f44588a5640 1 --2- 172.21.15.139:0/4206947961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc59cf0 0x55fe4cca1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:48.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f4453fff640 1 --2- 172.21.15.139:0/4206947961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 0x55fe4cca1660 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f444002a7d0 tx=0x7f444003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ecf49df7ade497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f443dffb640 1 -- 172.21.15.139:0/4206947961 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444004e070 con 0x55fe4cc5abb0 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f443dffb640 1 -- 172.21.15.139:0/4206947961 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444003e4e0 con 0x55fe4cc5abb0 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.848+0000 7f443dffb640 1 -- 172.21.15.139:0/4206947961 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440037040 con 0x55fe4cc5abb0 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.849+0000 7f445d811740 1 -- 172.21.15.139:0/4206947961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 msgr2=0x55fe4cca1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.849+0000 7f445d811740 1 --2- 172.21.15.139:0/4206947961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 0x55fe4cca1660 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f444002a7d0 tx=0x7f444003b000 comp rx=0 tx=0).stop 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.849+0000 7f445d811740 1 -- 172.21.15.139:0/4206947961 shutdown_connections 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.849+0000 7f445d811740 1 --2- 172.21.15.139:0/4206947961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 0x55fe4cca1660 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.849+0000 7f445d811740 1 --2- 172.21.15.139:0/4206947961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc71b00 0x55fe4cc77e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.849+0000 7f445d811740 1 --2- 172.21.15.139:0/4206947961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc59cf0 0x55fe4cca1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.849+0000 7f445d811740 1 -- 172.21.15.139:0/4206947961 >> 172.21.15.139:0/4206947961 conn(0x55fe4cc55830 msgr2=0x55fe4ccba9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.849+0000 7f445d811740 1 -- 172.21.15.139:0/4206947961 shutdown_connections 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.849+0000 7f445d811740 1 -- 172.21.15.139:0/4206947961 wait complete. 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.849+0000 7f445d811740 1 -- start start 2024-04-25T09:59:48.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.849+0000 7f445d811740 1 Processor -- start 2024-04-25T09:59:48.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.850+0000 7f445d811740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc59cf0 0x55fe4cd0aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.850+0000 7f445d811740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 0x55fe4cc5af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.850+0000 7f445d811740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc71b00 0x55fe4cc5b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.850+0000 7f445d811740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4cc7e720 con 0x55fe4cc5abb0 2024-04-25T09:59:48.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.850+0000 7f445d811740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44541fefe0 con 0x55fe4cc59cf0 2024-04-25T09:59:48.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.850+0000 7f445d811740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe4cc71910 con 0x55fe4cc71b00 2024-04-25T09:59:48.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.850+0000 7f445d811740 10 client.0 _ll_register_callbacks cb 0x7f4440035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:48.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.850+0000 7f4453fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 0x55fe4cc5af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.850+0000 7f4453fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 0x55fe4cc5af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44908/0 (socket says 172.21.15.139:44908) 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.850+0000 7f4453fff640 1 -- 172.21.15.139:0/2710826839 learned_addr learned my addr 172.21.15.139:0/2710826839 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.850+0000 7f44590a6640 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc71b00 0x55fe4cc5b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.850+0000 7f44588a5640 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc59cf0 0x55fe4cd0aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f4453fff640 1 -- 172.21.15.139:0/2710826839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc71b00 msgr2=0x55fe4cc5b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f4453fff640 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc71b00 0x55fe4cc5b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f4453fff640 1 -- 172.21.15.139:0/2710826839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc59cf0 msgr2=0x55fe4cd0aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f4453fff640 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc59cf0 0x55fe4cd0aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f4453fff640 1 -- 172.21.15.139:0/2710826839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe4cc54df0 con 0x55fe4cc5abb0 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f4453fff640 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 0x55fe4cc5af90 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f444002a7d0 tx=0x7f4440004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8ca6ae59f63820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f44337fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f44337fe640 1 -- 172.21.15.139:0/2710826839 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440058020 con 0x55fe4cc5abb0 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f44337fe640 1 -- 172.21.15.139:0/2710826839 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4440032e00 con 0x55fe4cc5abb0 2024-04-25T09:59:48.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f445d811740 1 -- 172.21.15.139:0/2710826839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f444003e4e0 con 0x55fe4cc5abb0 2024-04-25T09:59:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f44337fe640 1 -- 172.21.15.139:0/2710826839 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4440062050 con 0x55fe4cc5abb0 2024-04-25T09:59:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f445d811740 10 client.7977 fetch_fsmap learned FSMap version 57 2024-04-25T09:59:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f445d811740 1 -- 172.21.15.139:0/2710826839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55fe4cca3250 con 0x55fe4cc5abb0 2024-04-25T09:59:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f44337fe640 1 -- 172.21.15.139:0/2710826839 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444003e4e0 con 0x55fe4cc5abb0 2024-04-25T09:59:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f44337fe640 1 -- 172.21.15.139:0/2710826839 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4440065020 con 0x55fe4cc5abb0 2024-04-25T09:59:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f445d811740 10 client.7977 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T09:59:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f445d811740 10 client.7977 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:59:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f445d811740 20 client.7977 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f445d811740 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55fe4cc7e260 0x55fe4cca1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f445d811740 4 client.7977 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T09:59:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f445d811740 1 -- 172.21.15.139:0/2710826839 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55fe4cc75c80 con 0x55fe4cc7e260 2024-04-25T09:59:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.851+0000 7f44588a5640 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55fe4cc7e260 0x55fe4cca1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f44337fe640 10 client.7977 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:48.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f44588a5640 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55fe4cc7e260 0x55fe4cca1fe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88714babea7b3b38 server_cookie=1b5b1fc6899a2097 in_seq=0 out_seq=0 2024-04-25T09:59:48.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f44337fe640 1 -- 172.21.15.139:0/2710826839 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fe4cc75c80 con 0x55fe4cc7e260 2024-04-25T09:59:48.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f44337fe640 10 client.7977 handle_command_reply: tid=0 2024-04-25T09:59:48.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f445d811740 1 client.7977 shutdown 2024-04-25T09:59:48.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f445d811740 1 -- 172.21.15.139:0/2710826839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 msgr2=0x55fe4cc5af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f445d811740 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 0x55fe4cc5af90 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f444002a7d0 tx=0x7f4440004730 comp rx=0 tx=0).stop 2024-04-25T09:59:48.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f445d811740 1 -- 172.21.15.139:0/2710826839 shutdown_connections 2024-04-25T09:59:48.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f445d811740 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55fe4cc7e260 0x55fe4cca1fe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f445d811740 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fe4cc5abb0 0x55fe4cc5af90 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f445d811740 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fe4cc71b00 0x55fe4cc5b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f445d811740 1 --2- 172.21.15.139:0/2710826839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4cc59cf0 0x55fe4cd0aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f445d811740 1 -- 172.21.15.139:0/2710826839 >> 172.21.15.139:0/2710826839 conn(0x55fe4cc55830 msgr2=0x55fe4cc77e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f44337fe640 0 client.7977 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:48.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f445d811740 1 -- 172.21.15.139:0/2710826839 shutdown_connections 2024-04-25T09:59:48.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.852+0000 7f445d811740 1 -- 172.21.15.139:0/2710826839 wait complete. 2024-04-25T09:59:48.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445d811740 20 client.7977 trim_cache size 0 max 16384 2024-04-25T09:59:48.864 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:59:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: "return_code": 0, 2024-04-25T09:59:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_tag": "af3104de-09d1-486c-8538-db1125d1c955", 2024-04-25T09:59:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": "asynchronous" 2024-04-25T09:59:48.865 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:59:48.865 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T09:59:48.853+0000 7efdd1538640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-04-25T09:59:08.119440+0000 1=1+0)->f(v1 m2024-04-25T09:59:08.119440+0000) n(v0 rc2024-04-25T09:59:08.238437+0000 2=1+1)->n(v0 rc2024-04-25T09:59:08.238437+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x557cea233b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T09:59:08.119440+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T09:59:08.238437+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T09:59:48.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445ab09640 1 -- 172.21.15.139:0/2480077896 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f443806b4d0 msgr2=0x7f443806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445ab09640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f443806b4d0 0x7f443806d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4440034000 tx=0x7f444002e000 comp rx=0 tx=0).stop 2024-04-25T09:59:48.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445ab09640 1 -- 172.21.15.139:0/2480077896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f445406b700 msgr2=0x7f44541f4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445ab09640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f445406b700 0x7f44541f4400 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f444800c7f0 tx=0x7f444800cbf0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445ab09640 1 -- 172.21.15.139:0/2480077896 shutdown_connections 2024-04-25T09:59:48.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445ab09640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f443806b4d0 0x7f443806d990 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445ab09640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44540afc80 0x7f44540adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445ab09640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44540af8a0 0x7f445415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445ab09640 1 --2- 172.21.15.139:0/2480077896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f445406b700 0x7f44541f4400 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445ab09640 1 -- 172.21.15.139:0/2480077896 >> 172.21.15.139:0/2480077896 conn(0x7f44540b2230 msgr2=0x7f44540ad510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445ab09640 1 -- 172.21.15.139:0/2480077896 shutdown_connections 2024-04-25T09:59:48.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.853+0000 7f445ab09640 1 -- 172.21.15.139:0/2480077896 wait complete. 2024-04-25T09:59:48.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:48.868 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:59:48.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.963+0000 7f2df362a640 1 Processor -- start 2024-04-25T09:59:48.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.964+0000 7f2df362a640 1 -- start start 2024-04-25T09:59:48.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.964+0000 7f2df362a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2dec1617e0 0x7f2dec0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.964+0000 7f2df362a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2dec0cec80 0x7f2dec0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.964+0000 7f2df362a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dec161dc0 con 0x7f2dec15dbf0 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.964+0000 7f2df362a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dec0cf0d0 con 0x7f2dec1617e0 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.964+0000 7f2df362a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dec0d1ee0 con 0x7f2dec0cec80 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.965+0000 7f2df0bc5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2dec1617e0 0x7f2dec0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.965+0000 7f2df0bc5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2dec1617e0 0x7f2dec0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44920/0 (socket says 172.21.15.139:44920) 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.965+0000 7f2df0bc5640 1 -- 172.21.15.139:0/3694299145 learned_addr learned my addr 172.21.15.139:0/3694299145 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.965+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/3694299145 >> v2:172.21.15.150:3300/0 conn(0x7f2dec0cec80 0x7f2dec0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.965+0000 7f2df0bc5640 1 -- 172.21.15.139:0/3694299145 >> v2:172.21.15.150:3300/0 conn(0x7f2dec0cec80 msgr2=0x7f2dec0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.965+0000 7f2df0bc5640 1 --2- 172.21.15.139:0/3694299145 >> v2:172.21.15.150:3300/0 conn(0x7f2dec0cec80 0x7f2dec0d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.965+0000 7f2df0bc5640 1 -- 172.21.15.139:0/3694299145 >> v1:172.21.15.150:6789/0 conn(0x7f2dec15dbf0 legacy=0x7f2dec0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.965+0000 7f2df0bc5640 1 -- 172.21.15.139:0/3694299145 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec1620a0 con 0x7f2dec1617e0 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.966+0000 7f2df0bc5640 1 --2- 172.21.15.139:0/3694299145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dec1617e0 0x7f2dec0ce740 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2dec08ce00 tx=0x7f2de802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c490c1bae4fdc986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.966+0000 7f2de3fff640 1 -- 172.21.15.139:0/3694299145 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8037070 con 0x7f2dec1617e0 2024-04-25T09:59:48.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.966+0000 7f2de3fff640 1 -- 172.21.15.139:0/3694299145 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de8032440 con 0x7f2dec1617e0 2024-04-25T09:59:48.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.966+0000 7f2de3fff640 1 -- 172.21.15.139:0/3694299145 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8032810 con 0x7f2dec1617e0 2024-04-25T09:59:48.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.966+0000 7f2df362a640 1 -- 172.21.15.139:0/3694299145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dec1617e0 msgr2=0x7f2dec0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.966+0000 7f2df362a640 1 --2- 172.21.15.139:0/3694299145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dec1617e0 0x7f2dec0ce740 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2dec08ce00 tx=0x7f2de802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.966+0000 7f2df362a640 1 -- 172.21.15.139:0/3694299145 shutdown_connections 2024-04-25T09:59:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.966+0000 7f2df362a640 1 --2- 172.21.15.139:0/3694299145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dec1617e0 0x7f2dec0ce740 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.966+0000 7f2df362a640 1 --2- 172.21.15.139:0/3694299145 >> v2:172.21.15.150:3300/0 conn(0x7f2dec0cec80 0x7f2dec0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.966+0000 7f2df362a640 1 -- 172.21.15.139:0/3694299145 >> 172.21.15.139:0/3694299145 conn(0x7f2dec0b5480 msgr2=0x7f2dec0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.966+0000 7f2df362a640 1 -- 172.21.15.139:0/3694299145 shutdown_connections 2024-04-25T09:59:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.966+0000 7f2df362a640 1 -- 172.21.15.139:0/3694299145 wait complete. 2024-04-25T09:59:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.967+0000 7f2df362a640 1 Processor -- start 2024-04-25T09:59:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.968+0000 7f2df362a640 1 -- start start 2024-04-25T09:59:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.968+0000 7f2df362a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dec0cec80 0x7f2dec0c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.968+0000 7f2df362a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dec15dbf0 0x7f2dec15ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.968+0000 7f2df362a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dec1617e0 0x7f2dec155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.968+0000 7f2df362a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec15dfd0 con 0x7f2dec15dbf0 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.968+0000 7f2df362a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec162660 con 0x7f2dec1617e0 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.968+0000 7f2df362a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec0c9aa0 con 0x7f2dec0cec80 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.968+0000 7f2df0bc5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dec15dbf0 0x7f2dec15ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.968+0000 7f2df0bc5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dec15dbf0 0x7f2dec15ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44922/0 (socket says 172.21.15.139:44922) 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.968+0000 7f2df0bc5640 1 -- 172.21.15.139:0/1498338420 learned_addr learned my addr 172.21.15.139:0/1498338420 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.969+0000 7f2df0bc5640 1 -- 172.21.15.139:0/1498338420 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dec0cec80 msgr2=0x7f2dec0c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.969+0000 7f2df0bc5640 1 --2- 172.21.15.139:0/1498338420 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dec0cec80 0x7f2dec0c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.969+0000 7f2df0bc5640 1 -- 172.21.15.139:0/1498338420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dec1617e0 msgr2=0x7f2dec155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.969+0000 7f2df0bc5640 1 --2- 172.21.15.139:0/1498338420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dec1617e0 0x7f2dec155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.969+0000 7f2df0bc5640 1 -- 172.21.15.139:0/1498338420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec1543f0 con 0x7f2dec15dbf0 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.969+0000 7f2df0bc5640 1 --2- 172.21.15.139:0/1498338420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dec15dbf0 0x7f2dec15ca20 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f2de802a220 tx=0x7f2de80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.969+0000 7f2de1ffb640 1 -- 172.21.15.139:0/1498338420 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8041020 con 0x7f2dec15dbf0 2024-04-25T09:59:48.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.969+0000 7f2de1ffb640 1 -- 172.21.15.139:0/1498338420 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de80306c0 con 0x7f2dec15dbf0 2024-04-25T09:59:48.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.969+0000 7f2df362a640 1 -- 172.21.15.139:0/1498338420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dec1546d0 con 0x7f2dec15dbf0 2024-04-25T09:59:48.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.969+0000 7f2de1ffb640 1 -- 172.21.15.139:0/1498338420 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8037040 con 0x7f2dec15dbf0 2024-04-25T09:59:48.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.969+0000 7f2df362a640 1 -- 172.21.15.139:0/1498338420 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dec160bc0 con 0x7f2dec15dbf0 2024-04-25T09:59:48.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.970+0000 7f2de1ffb640 1 -- 172.21.15.139:0/1498338420 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2de8044020 con 0x7f2dec15dbf0 2024-04-25T09:59:48.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.971+0000 7f2de1ffb640 1 --2- 172.21.15.139:0/1498338420 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2dd006b4d0 0x7f2dd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.971+0000 7f2df13c6640 1 --2- 172.21.15.139:0/1498338420 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2dd006b4d0 0x7f2dd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.971+0000 7f2de1ffb640 1 -- 172.21.15.139:0/1498338420 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2de80aabf0 con 0x7f2dec15dbf0 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.971+0000 7f2df13c6640 1 --2- 172.21.15.139:0/1498338420 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2dd006b4d0 0x7f2dd006d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc000f60 tx=0x7f2ddc0079d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.972+0000 7f2df6409740 1 Processor -- start 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.972+0000 7f2df6409740 1 -- start start 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df6409740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869dfb10 0x5575869e2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df6409740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfef0 0x5575869e2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df6409740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b2770 0x5575869b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df6409740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758691bda0 con 0x5575869b2770 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df6409740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575868c4b80 con 0x5575869dfef0 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df6409740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557586917e40 con 0x5575869dfb10 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df1bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b2770 0x5575869b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df1bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b2770 0x5575869b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44928/0 (socket says 172.21.15.139:44928) 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df1bc7640 1 -- 172.21.15.139:0/4110837442 learned_addr learned my addr 172.21.15.139:0/4110837442 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df13c6640 1 --2- 172.21.15.139:0/4110837442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869dfb10 0x5575869e2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df0bc5640 1 --2- 172.21.15.139:0/4110837442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfef0 0x5575869e2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df1bc7640 1 -- 172.21.15.139:0/4110837442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869dfb10 msgr2=0x5575869e2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/4110837442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869dfb10 0x5575869e2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df1bc7640 1 -- 172.21.15.139:0/4110837442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfef0 msgr2=0x5575869e2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/4110837442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfef0 0x5575869e2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df1bc7640 1 -- 172.21.15.139:0/4110837442 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575868cef10 con 0x5575869b2770 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df13c6640 1 --2- 172.21.15.139:0/4110837442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869dfb10 0x5575869e2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/4110837442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b2770 0x5575869b5f80 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009fb0 tx=0x7f2de40077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475e9a110a308bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2dd6ffd640 1 -- 172.21.15.139:0/4110837442 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4019070 con 0x5575869b2770 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2dd6ffd640 1 -- 172.21.15.139:0/4110837442 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de4012710 con 0x5575869b2770 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.973+0000 7f2dd6ffd640 1 -- 172.21.15.139:0/4110837442 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4012ae0 con 0x5575869b2770 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.974+0000 7f2df6409740 1 -- 172.21.15.139:0/4110837442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b2770 msgr2=0x5575869b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.974+0000 7f2df6409740 1 --2- 172.21.15.139:0/4110837442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b2770 0x5575869b5f80 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009fb0 tx=0x7f2de40077c0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.974+0000 7f2df6409740 1 -- 172.21.15.139:0/4110837442 shutdown_connections 2024-04-25T09:59:48.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.974+0000 7f2df6409740 1 --2- 172.21.15.139:0/4110837442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b2770 0x5575869b5f80 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.974+0000 7f2df6409740 1 --2- 172.21.15.139:0/4110837442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfef0 0x5575869e2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.974+0000 7f2df6409740 1 --2- 172.21.15.139:0/4110837442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869dfb10 0x5575869e2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.974+0000 7f2df6409740 1 -- 172.21.15.139:0/4110837442 >> 172.21.15.139:0/4110837442 conn(0x5575869b49e0 msgr2=0x5575869fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.974+0000 7f2df6409740 1 -- 172.21.15.139:0/4110837442 shutdown_connections 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.974+0000 7f2df6409740 1 -- 172.21.15.139:0/4110837442 wait complete. 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.975+0000 7f2df6409740 1 -- start start 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.975+0000 7f2df6409740 1 Processor -- start 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.975+0000 7f2df6409740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869dfb10 0x557586a10f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.975+0000 7f2df6409740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfef0 0x5575869d64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.975+0000 7f2df6409740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b6f90 0x5575869b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df6409740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575869b7b90 con 0x5575869b6f90 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df6409740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575868ba4b0 con 0x5575869dfef0 2024-04-25T09:59:48.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df6409740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575869b7e70 con 0x5575869dfb10 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df6409740 10 client.0 _ll_register_callbacks cb 0x5575868fa180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df1bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b6f90 0x5575869b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df1bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b6f90 0x5575869b73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44940/0 (socket says 172.21.15.139:44940) 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df1bc7640 1 -- 172.21.15.139:0/1116125398 learned_addr learned my addr 172.21.15.139:0/1116125398 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df13c6640 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869dfb10 0x557586a10f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df0bc5640 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfef0 0x5575869d64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df1bc7640 1 -- 172.21.15.139:0/1116125398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869dfb10 msgr2=0x557586a10f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869dfb10 0x557586a10f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df1bc7640 1 -- 172.21.15.139:0/1116125398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfef0 msgr2=0x5575869d64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfef0 0x5575869d64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df1bc7640 1 -- 172.21.15.139:0/1116125398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575866fa7d0 con 0x5575869b6f90 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2dcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:48.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b6f90 0x5575869b73f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x5575869ba070 tx=0x7f2de4009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512f10f58d49e156 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2dcbfff640 1 -- 172.21.15.139:0/1116125398 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4024020 con 0x5575869b6f90 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2dcbfff640 1 -- 172.21.15.139:0/1116125398 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2de40128c0 con 0x5575869b6f90 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df6409740 1 -- 172.21.15.139:0/1116125398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2de4012710 con 0x5575869b6f90 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2dcbfff640 1 -- 172.21.15.139:0/1116125398 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de4019040 con 0x5575869b6f90 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df6409740 10 client.7989 fetch_fsmap learned FSMap version 57 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.976+0000 7f2df6409740 1 -- 172.21.15.139:0/1116125398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5575866303f0 con 0x5575869b6f90 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.977+0000 7f2dcbfff640 1 -- 172.21.15.139:0/1116125398 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de4012710 con 0x5575869b6f90 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.977+0000 7f2dcbfff640 1 -- 172.21.15.139:0/1116125398 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2de4012ac0 con 0x5575869b6f90 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.977+0000 7f2df6409740 10 client.7989 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.977+0000 7f2df6409740 10 client.7989 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.977+0000 7f2df6409740 20 client.7989 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.977+0000 7f2df6409740 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5575869e36c0 0x5575869e3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:48.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.977+0000 7f2df6409740 4 client.7989 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.977+0000 7f2df6409740 1 -- 172.21.15.139:0/1116125398 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575869292e0 con 0x5575869e36c0 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.977+0000 7f2df13c6640 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5575869e36c0 0x5575869e3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.977+0000 7f2dcbfff640 10 client.7989 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.977+0000 7f2df13c6640 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5575869e36c0 0x5575869e3ad0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce5c79aef927049 server_cookie=d5843cb5e34e1a1d in_seq=0 out_seq=0 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2dcbfff640 1 -- 172.21.15.139:0/1116125398 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5575869292e0 con 0x5575869e36c0 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2dcbfff640 10 client.7989 handle_command_reply: tid=0 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2df6409740 1 client.7989 shutdown 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2df6409740 1 -- 172.21.15.139:0/1116125398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b6f90 msgr2=0x5575869b73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2df6409740 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b6f90 0x5575869b73f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x5575869ba070 tx=0x7f2de4009450 comp rx=0 tx=0).stop 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2df6409740 1 -- 172.21.15.139:0/1116125398 shutdown_connections 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2df6409740 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5575869e36c0 0x5575869e3ad0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2df6409740 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869b6f90 0x5575869b73f0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2df6409740 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfef0 0x5575869d64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2dcbfff640 0 client.7989 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:48.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2df6409740 1 --2- 172.21.15.139:0/1116125398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869dfb10 0x557586a10f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:48.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2df6409740 1 -- 172.21.15.139:0/1116125398 >> 172.21.15.139:0/1116125398 conn(0x5575869b2770 msgr2=0x5575869de190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:48.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2df6409740 1 -- 172.21.15.139:0/1116125398 shutdown_connections 2024-04-25T09:59:48.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2df6409740 1 -- 172.21.15.139:0/1116125398 wait complete. 2024-04-25T09:59:48.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.978+0000 7f2df6409740 20 client.7989 trim_cache size 0 max 16384 2024-04-25T09:59:48.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.993+0000 7f2df6409740 1 Processor -- start 2024-04-25T09:59:48.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.993+0000 7f2df6409740 1 -- start start 2024-04-25T09:59:49.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df6409740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869b6f90 0x5575869fd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df6409740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfb10 0x5575869b9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df6409740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 0x5575869dd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df6409740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec200640 con 0x5575869dfef0 2024-04-25T09:59:49.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df6409740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec1ff820 con 0x5575869dfb10 2024-04-25T09:59:49.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df6409740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec15f9f0 con 0x5575869b6f90 2024-04-25T09:59:49.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df1bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 0x5575869dd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df1bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 0x5575869dd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44950/0 (socket says 172.21.15.139:44950) 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df1bc7640 1 -- 172.21.15.139:0/2087448118 learned_addr learned my addr 172.21.15.139:0/2087448118 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df0bc5640 1 --2- 172.21.15.139:0/2087448118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfb10 0x5575869b9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df13c6640 1 --2- 172.21.15.139:0/2087448118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869b6f90 0x5575869fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df1bc7640 1 -- 172.21.15.139:0/2087448118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869b6f90 msgr2=0x5575869fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/2087448118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869b6f90 0x5575869fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df1bc7640 1 -- 172.21.15.139:0/2087448118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfb10 msgr2=0x5575869b9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/2087448118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfb10 0x5575869b9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df1bc7640 1 -- 172.21.15.139:0/2087448118 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575865f00f0 con 0x5575869dfef0 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/2087448118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 0x5575869dd3d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f2de4002230 tx=0x7f2de401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acba909565f05f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2dd67fc640 1 -- 172.21.15.139:0/2087448118 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4024070 con 0x5575869dfef0 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2dd67fc640 1 -- 172.21.15.139:0/2087448118 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de4009ac0 con 0x5575869dfef0 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.994+0000 7f2dd67fc640 1 -- 172.21.15.139:0/2087448118 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de40126d0 con 0x5575869dfef0 2024-04-25T09:59:49.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.995+0000 7f2df6409740 1 -- 172.21.15.139:0/2087448118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 msgr2=0x5575869dd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.995+0000 7f2df6409740 1 --2- 172.21.15.139:0/2087448118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 0x5575869dd3d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f2de4002230 tx=0x7f2de401d000 comp rx=0 tx=0).stop 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.995+0000 7f2df6409740 1 -- 172.21.15.139:0/2087448118 shutdown_connections 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.995+0000 7f2df6409740 1 --2- 172.21.15.139:0/2087448118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 0x5575869dd3d0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.995+0000 7f2df6409740 1 --2- 172.21.15.139:0/2087448118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfb10 0x5575869b9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.995+0000 7f2df6409740 1 --2- 172.21.15.139:0/2087448118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869b6f90 0x5575869fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.995+0000 7f2df6409740 1 -- 172.21.15.139:0/2087448118 >> 172.21.15.139:0/2087448118 conn(0x5575869b2770 msgr2=0x5575869e3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.995+0000 7f2df6409740 1 -- 172.21.15.139:0/2087448118 shutdown_connections 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.995+0000 7f2df6409740 1 -- 172.21.15.139:0/2087448118 wait complete. 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.995+0000 7f2df6409740 1 -- start start 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.996+0000 7f2df6409740 1 Processor -- start 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.996+0000 7f2df6409740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869b6f90 0x5575869ff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.996+0000 7f2df6409740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfb10 0x5575869ff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.996+0000 7f2df6409740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 0x5575869dd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.996+0000 7f2df6409740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575868dd520 con 0x5575869dfef0 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.996+0000 7f2df6409740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec1ff220 con 0x5575869dfb10 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.996+0000 7f2df6409740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575869fdcc0 con 0x5575869b6f90 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.996+0000 7f2df6409740 10 client.0 _ll_register_callbacks cb 0x7f2de4009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df1bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 0x5575869dd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df1bc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 0x5575869dd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44960/0 (socket says 172.21.15.139:44960) 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df1bc7640 1 -- 172.21.15.139:0/1025570524 learned_addr learned my addr 172.21.15.139:0/1025570524 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df0bc5640 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfb10 0x5575869ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df13c6640 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869b6f90 0x5575869ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df1bc7640 1 -- 172.21.15.139:0/1025570524 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869b6f90 msgr2=0x5575869ff420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869b6f90 0x5575869ff420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df1bc7640 1 -- 172.21.15.139:0/1025570524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfb10 msgr2=0x5575869ff960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfb10 0x5575869ff960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df1bc7640 1 -- 172.21.15.139:0/1025570524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575869b1f60 con 0x5575869dfef0 2024-04-25T09:59:49.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df1bc7640 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 0x5575869dd3d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f2de4007800 tx=0x7f2de4009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84938ad3d2df4e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:49.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2dcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T09:59:49.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2dcbfff640 1 -- 172.21.15.139:0/1025570524 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4024070 con 0x5575869dfef0 2024-04-25T09:59:49.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2dcbfff640 1 -- 172.21.15.139:0/1025570524 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2de4012ae0 con 0x5575869dfef0 2024-04-25T09:59:49.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df6409740 1 -- 172.21.15.139:0/1025570524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2de4009ac0 con 0x5575869dfef0 2024-04-25T09:59:49.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2dcbfff640 1 -- 172.21.15.139:0/1025570524 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de403f050 con 0x5575869dfef0 2024-04-25T09:59:49.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df6409740 10 client.7995 fetch_fsmap learned FSMap version 57 2024-04-25T09:59:49.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2df6409740 1 -- 172.21.15.139:0/1025570524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5575869deb40 con 0x5575869dfef0 2024-04-25T09:59:49.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.997+0000 7f2dcbfff640 1 -- 172.21.15.139:0/1025570524 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de4009ac0 con 0x5575869dfef0 2024-04-25T09:59:49.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2dcbfff640 1 -- 172.21.15.139:0/1025570524 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2de4042020 con 0x5575869dfef0 2024-04-25T09:59:49.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 10 client.7995 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T09:59:49.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 10 client.7995 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:59:49.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 20 client.7995 populate_metadata read hostname 'smithi139' 2024-04-25T09:59:49.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5575869b1600 0x5575869dd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:49.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 4 client.7995 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:59:49.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 -- 172.21.15.139:0/1025570524 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557586a00760 con 0x5575869b1600 2024-04-25T09:59:49.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df13c6640 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5575869b1600 0x5575869dd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:49.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2dcbfff640 10 client.7995 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:49.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df13c6640 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5575869b1600 0x5575869dd910 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6592976ddfde7e95 server_cookie=e7af72d1c0912d in_seq=0 out_seq=0 2024-04-25T09:59:49.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2dcbfff640 1 -- 172.21.15.139:0/1025570524 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x557586a00760 con 0x5575869b1600 2024-04-25T09:59:49.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2dcbfff640 10 client.7995 handle_command_reply: tid=0 2024-04-25T09:59:49.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 client.7995 shutdown 2024-04-25T09:59:49.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 -- 172.21.15.139:0/1025570524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 msgr2=0x5575869dd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 0x5575869dd3d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f2de4007800 tx=0x7f2de4009910 comp rx=0 tx=0).stop 2024-04-25T09:59:49.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 -- 172.21.15.139:0/1025570524 shutdown_connections 2024-04-25T09:59:49.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x5575869b1600 0x5575869dd910 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5575869dfef0 0x5575869dd3d0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575869dfb10 0x5575869ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 --2- 172.21.15.139:0/1025570524 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5575869b6f90 0x5575869ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2dcbfff640 0 client.7995 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T09:59:49.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 -- 172.21.15.139:0/1025570524 >> 172.21.15.139:0/1025570524 conn(0x5575869b2770 msgr2=0x5575869b7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:49.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 -- 172.21.15.139:0/1025570524 shutdown_connections 2024-04-25T09:59:49.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 1 -- 172.21.15.139:0/1025570524 wait complete. 2024-04-25T09:59:49.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.998+0000 7f2df6409740 20 client.7995 trim_cache size 0 max 16384 2024-04-25T09:59:49.011 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T09:59:49.011 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:59:49.011 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": { 2024-04-25T09:59:49.011 INFO:teuthology.orchestra.run.smithi139.stdout: "af3104de-09d1-486c-8538-db1125d1c955": { 2024-04-25T09:59:49.011 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/dir", 2024-04-25T09:59:49.011 INFO:teuthology.orchestra.run.smithi139.stdout: "tag": "af3104de-09d1-486c-8538-db1125d1c955", 2024-04-25T09:59:49.011 INFO:teuthology.orchestra.run.smithi139.stdout: "options": "recursive,repair" 2024-04-25T09:59:49.011 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:59:49.011 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T09:59:49.011 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T09:59:49.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.999+0000 7f2df362a640 1 -- 172.21.15.139:0/1498338420 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2dd006b4d0 msgr2=0x7f2dd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.999+0000 7f2df362a640 1 --2- 172.21.15.139:0/1498338420 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2dd006b4d0 0x7f2dd006d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc000f60 tx=0x7f2ddc0079d0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.999+0000 7f2df362a640 1 -- 172.21.15.139:0/1498338420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dec15dbf0 msgr2=0x7f2dec15ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:49.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.999+0000 7f2df362a640 1 --2- 172.21.15.139:0/1498338420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dec15dbf0 0x7f2dec15ca20 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f2de802a220 tx=0x7f2de80046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.999+0000 7f2df362a640 1 -- 172.21.15.139:0/1498338420 shutdown_connections 2024-04-25T09:59:49.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.999+0000 7f2df362a640 1 --2- 172.21.15.139:0/1498338420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dec15dbf0 0x7f2dec15ca20 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.999+0000 7f2df362a640 1 --2- 172.21.15.139:0/1498338420 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2dd006b4d0 0x7f2dd006d990 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.999+0000 7f2df362a640 1 --2- 172.21.15.139:0/1498338420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dec1617e0 0x7f2dec155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.999+0000 7f2df362a640 1 --2- 172.21.15.139:0/1498338420 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dec0cec80 0x7f2dec0c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:49.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.999+0000 7f2df362a640 1 -- 172.21.15.139:0/1498338420 >> 172.21.15.139:0/1498338420 conn(0x7f2dec0b5480 msgr2=0x7f2dec0caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:49.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.999+0000 7f2df362a640 1 -- 172.21.15.139:0/1498338420 shutdown_connections 2024-04-25T09:59:49.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T09:59:48.999+0000 7f2df362a640 1 -- 172.21.15.139:0/1498338420 wait complete. 2024-04-25T09:59:49.014 INFO:tasks.cephfs.filesystem:scrub status for tag:af3104de-09d1-486c-8538-db1125d1c955 - {'path': '/dir', 'tag': 'af3104de-09d1-486c-8538-db1125d1c955', 'options': 'recursive,repair'} 2024-04-25T10:00:00.004 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T09:59:59.999+0000 7f73a902d640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-04-25T10:00:16.646 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:16.650 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:19.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:19.016 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:00:19.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.119+0000 7f5da6f16640 1 Processor -- start 2024-04-25T10:00:19.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.119+0000 7f5da6f16640 1 -- start start 2024-04-25T10:00:19.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.120+0000 7f5da6f16640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5da0144090 0x7f5da00b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.120+0000 7f5da6f16640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da00b2a50 con 0x7f5da006ea40 2024-04-25T10:00:19.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.120+0000 7f5da6f16640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da015b090 con 0x7f5da00cc9e0 2024-04-25T10:00:19.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.120+0000 7f5da6f16640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015b370 con 0x7f5da0144090 2024-04-25T10:00:19.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.120+0000 7f5da54b3640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5da006ea40 0x7f5da015e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41562/0 (socket says 172.21.15.139:41562) 2024-04-25T10:00:19.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.120+0000 7f5da54b3640 1 -- 172.21.15.139:0/449320468 learned_addr learned my addr 172.21.15.139:0/449320468 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:19.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664008093 0 0) 0x7f5d98001490 con 0x7f5da006ea40 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5da4cb2640 1 --2- 172.21.15.139:0/449320468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5da0144090 0x7f5da00b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d88003520 con 0x7f5da006ea40 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 30867817 0 0) 0x7f5d88003520 con 0x7f5da006ea40 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5da0144090 msgr2=0x7f5da00b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 --2- 172.21.15.139:0/449320468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5da0144090 0x7f5da00b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 >> v1:172.21.15.150:6789/0 conn(0x7f5da00cc9e0 legacy=0x7f5da006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da015b650 con 0x7f5da006ea40 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f5d98001bc0 con 0x7f5da006ea40 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 >> v1:172.21.15.139:6789/0 conn(0x7f5da006ea40 legacy=0x7f5da015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 --2- 172.21.15.139:0/449320468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5da0144090 0x7f5d880039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 --2- 172.21.15.139:0/449320468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d88003f60 0x7f5d88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 --2- 172.21.15.139:0/449320468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d88004d40 0x7f5d880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015b370 con 0x7f5d88004d40 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8800bb50 con 0x7f5d88003f60 2024-04-25T10:00:19.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8800df60 con 0x7f5da0144090 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5da54b3640 1 --2- 172.21.15.139:0/449320468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d88004d40 0x7f5d880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5da4cb2640 1 --2- 172.21.15.139:0/449320468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5da0144090 0x7f5d880039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5d9ffff640 1 --2- 172.21.15.139:0/449320468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d88003f60 0x7f5d88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5da54b3640 1 -- 172.21.15.139:0/449320468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5da0144090 msgr2=0x7f5d880039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5da54b3640 1 --2- 172.21.15.139:0/449320468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5da0144090 0x7f5d880039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5da54b3640 1 -- 172.21.15.139:0/449320468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d88003f60 msgr2=0x7f5d88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5da54b3640 1 --2- 172.21.15.139:0/449320468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d88003f60 0x7f5d88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.121+0000 7f5da54b3640 1 -- 172.21.15.139:0/449320468 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d88003520 con 0x7f5d88004d40 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.122+0000 7f5da54b3640 1 --2- 172.21.15.139:0/449320468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d88004d40 0x7f5d880051c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f5da00b27a0 tx=0x7f5d88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ec540c2554c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.122+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9800a650 con 0x7f5d88004d40 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.122+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9802f070 con 0x7f5d88004d40 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.122+0000 7f5d9f7fe640 1 -- 172.21.15.139:0/449320468 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9802be40 con 0x7f5d88004d40 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.123+0000 7f5da6f16640 1 -- 172.21.15.139:0/449320468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d88004d40 msgr2=0x7f5d880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.123+0000 7f5da6f16640 1 --2- 172.21.15.139:0/449320468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5d88004d40 0x7f5d880051c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f5da00b27a0 tx=0x7f5d88005830 comp rx=0 tx=0).stop 2024-04-25T10:00:19.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.123+0000 7f5da54b3640 1 -- 172.21.15.139:0/449320468 reap_dead start 2024-04-25T10:00:19.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.123+0000 7f5da6f16640 1 -- 172.21.15.139:0/449320468 shutdown_connections 2024-04-25T10:00:19.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.123+0000 7f5da6f16640 1 -- 172.21.15.139:0/449320468 >> 172.21.15.139:0/449320468 conn(0x7f5da0067000 msgr2=0x7f5da006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.123+0000 7f5da6f16640 1 -- 172.21.15.139:0/449320468 shutdown_connections 2024-04-25T10:00:19.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.123+0000 7f5da6f16640 1 -- 172.21.15.139:0/449320468 wait complete. 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.124+0000 7f5da6f16640 1 Processor -- start 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.124+0000 7f5da6f16640 1 -- start start 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5da6f16640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d88004d40 0x7f5da00a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5da6f16640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5da013e930 0x7f5da00ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5da6f16640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5da006d470 0x7f5da013ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5da6f16640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0144470 con 0x7f5da013e930 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5da6f16640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015bc10 con 0x7f5da006d470 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5da6f16640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015ad80 con 0x7f5d88004d40 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5d9ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5da013e930 0x7f5da00ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5d9ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5da013e930 0x7f5da00ab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35310/0 (socket says 172.21.15.139:35310) 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5d9ffff640 1 -- 172.21.15.139:0/363126459 learned_addr learned my addr 172.21.15.139:0/363126459 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5da4cb2640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d88004d40 0x7f5da00a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5da54b3640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5da006d470 0x7f5da013ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5d9ffff640 1 -- 172.21.15.139:0/363126459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d88004d40 msgr2=0x7f5da00a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5d9ffff640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d88004d40 0x7f5da00a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5d9ffff640 1 -- 172.21.15.139:0/363126459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5da006d470 msgr2=0x7f5da013ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5d9ffff640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5da006d470 0x7f5da013ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5d9ffff640 1 -- 172.21.15.139:0/363126459 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da013f160 con 0x7f5da013e930 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5da54b3640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5da006d470 0x7f5da013ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.125+0000 7f5d9ffff640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5da013e930 0x7f5da00ab090 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400f850 tx=0x7f5d9400fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.126+0000 7f5d9d7fa640 1 -- 172.21.15.139:0/363126459 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d940050d0 con 0x7f5da013e930 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.126+0000 7f5d9d7fa640 1 -- 172.21.15.139:0/363126459 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d94018070 con 0x7f5da013e930 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.126+0000 7f5d9d7fa640 1 -- 172.21.15.139:0/363126459 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400d6d0 con 0x7f5da013e930 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.126+0000 7f5da6f16640 1 -- 172.21.15.139:0/363126459 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da00a8550 con 0x7f5da013e930 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.126+0000 7f5da6f16640 1 -- 172.21.15.139:0/363126459 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da00a8b10 con 0x7f5da013e930 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.127+0000 7f5d9d7fa640 1 -- 172.21.15.139:0/363126459 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d9400d870 con 0x7f5da013e930 2024-04-25T10:00:19.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.128+0000 7f5d9d7fa640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5d8c06b500 0x7f5d8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.128+0000 7f5d9d7fa640 1 -- 172.21.15.139:0/363126459 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5d9408a160 con 0x7f5da013e930 2024-04-25T10:00:19.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.128+0000 7f5da4cb2640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5d8c06b500 0x7f5d8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.128+0000 7f5da4cb2640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5d8c06b500 0x7f5d8c06d9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009bc0 tx=0x7f5d90009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.129+0000 7f5da9c16740 1 Processor -- start 2024-04-25T10:00:19.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.129+0000 7f5da9c16740 1 -- start start 2024-04-25T10:00:19.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.129+0000 7f5da9c16740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 0x55988d4bf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.129+0000 7f5da9c16740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcef0 0x55988d4bf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.129+0000 7f5da9c16740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d48f770 0x55988d492f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.129+0000 7f5da9c16740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988d3f8da0 con 0x55988d48f770 2024-04-25T10:00:19.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.129+0000 7f5da9c16740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988d3a1b80 con 0x55988d4bcb10 2024-04-25T10:00:19.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.129+0000 7f5da9c16740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55988d3f4e40 con 0x55988d4bcef0 2024-04-25T10:00:19.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.129+0000 7f5da54b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d48f770 0x55988d492f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.129+0000 7f5da54b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d48f770 0x55988d492f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35326/0 (socket says 172.21.15.139:35326) 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.129+0000 7f5da54b3640 1 -- 172.21.15.139:0/802129689 learned_addr learned my addr 172.21.15.139:0/802129689 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.130+0000 7f5d9ffff640 1 --2- 172.21.15.139:0/802129689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcef0 0x55988d4bf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.130+0000 7f5da4cb2640 1 --2- 172.21.15.139:0/802129689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 0x55988d4bf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.130+0000 7f5da54b3640 1 -- 172.21.15.139:0/802129689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcef0 msgr2=0x55988d4bf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.130+0000 7f5da54b3640 1 --2- 172.21.15.139:0/802129689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcef0 0x55988d4bf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.130+0000 7f5da54b3640 1 -- 172.21.15.139:0/802129689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 msgr2=0x55988d4bf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.130+0000 7f5da54b3640 1 --2- 172.21.15.139:0/802129689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 0x55988d4bf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.130+0000 7f5da54b3640 1 -- 172.21.15.139:0/802129689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55988d3abf10 con 0x55988d48f770 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.130+0000 7f5da54b3640 1 --2- 172.21.15.139:0/802129689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d48f770 0x55988d492f80 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800aaf0 tx=0x7f5d98007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7137de465111a88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.130+0000 7f5d867fc640 1 -- 172.21.15.139:0/802129689 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d980019c0 con 0x55988d48f770 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.130+0000 7f5d867fc640 1 -- 172.21.15.139:0/802129689 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9802f040 con 0x55988d48f770 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.130+0000 7f5d867fc640 1 -- 172.21.15.139:0/802129689 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9802b9e0 con 0x55988d48f770 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.131+0000 7f5da9c16740 1 -- 172.21.15.139:0/802129689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d48f770 msgr2=0x55988d492f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.131+0000 7f5da9c16740 1 --2- 172.21.15.139:0/802129689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d48f770 0x55988d492f80 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800aaf0 tx=0x7f5d98007000 comp rx=0 tx=0).stop 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.131+0000 7f5da9c16740 1 -- 172.21.15.139:0/802129689 shutdown_connections 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.131+0000 7f5da9c16740 1 --2- 172.21.15.139:0/802129689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d48f770 0x55988d492f80 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.131+0000 7f5da9c16740 1 --2- 172.21.15.139:0/802129689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcef0 0x55988d4bf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.131+0000 7f5da9c16740 1 --2- 172.21.15.139:0/802129689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 0x55988d4bf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.131+0000 7f5da9c16740 1 -- 172.21.15.139:0/802129689 >> 172.21.15.139:0/802129689 conn(0x55988d4919e0 msgr2=0x55988d4da320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.131+0000 7f5da9c16740 1 -- 172.21.15.139:0/802129689 shutdown_connections 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.131+0000 7f5da9c16740 1 -- 172.21.15.139:0/802129689 wait complete. 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.131+0000 7f5da9c16740 1 -- start start 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.132+0000 7f5da9c16740 1 Processor -- start 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.132+0000 7f5da9c16740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 0x55988d4dd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.132+0000 7f5da9c16740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcef0 0x55988d4d4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.132+0000 7f5da9c16740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4b3790 0x55988d4dc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.132+0000 7f5da9c16740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988d494b90 con 0x55988d4b3790 2024-04-25T10:00:19.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.132+0000 7f5da9c16740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988d3974b0 con 0x55988d4bcb10 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.132+0000 7f5da9c16740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55988d494e70 con 0x55988d4bcef0 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.132+0000 7f5da9c16740 10 client.0 _ll_register_callbacks cb 0x55988d3d7180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.132+0000 7f5da54b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4b3790 0x55988d4dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.132+0000 7f5da54b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4b3790 0x55988d4dc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35334/0 (socket says 172.21.15.139:35334) 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.132+0000 7f5da54b3640 1 -- 172.21.15.139:0/1739052763 learned_addr learned my addr 172.21.15.139:0/1739052763 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5da4cb2640 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 0x55988d4dd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5d9ffff640 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcef0 0x55988d4d4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5da54b3640 1 -- 172.21.15.139:0/1739052763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcef0 msgr2=0x55988d4d4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5da54b3640 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcef0 0x55988d4d4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5da54b3640 1 -- 172.21.15.139:0/1739052763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 msgr2=0x55988d4dd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5da54b3640 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 0x55988d4dd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5da54b3640 1 -- 172.21.15.139:0/1739052763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55988d1d77d0 con 0x55988d4b3790 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5da54b3640 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4b3790 0x55988d4dc700 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x55988d497070 tx=0x7f5d9802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fb6bff4af37042 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5d7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5d7f7fe640 1 -- 172.21.15.139:0/1739052763 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9804f060 con 0x55988d4b3790 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5d7f7fe640 1 -- 172.21.15.139:0/1739052763 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5d9802bbf0 con 0x55988d4b3790 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5da9c16740 1 -- 172.21.15.139:0/1739052763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d9802f040 con 0x55988d4b3790 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5d7f7fe640 1 -- 172.21.15.139:0/1739052763 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d98055050 con 0x55988d4b3790 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5da9c16740 10 client.8007 fetch_fsmap learned FSMap version 57 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5da9c16740 1 -- 172.21.15.139:0/1739052763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55988d10d3f0 con 0x55988d4b3790 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5d7f7fe640 1 -- 172.21.15.139:0/1739052763 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9802f040 con 0x55988d4b3790 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5d7f7fe640 1 -- 172.21.15.139:0/1739052763 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5d98058020 con 0x55988d4b3790 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5da9c16740 10 client.8007 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.133+0000 7f5da9c16740 10 client.8007 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.134+0000 7f5da9c16740 20 client.8007 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.134+0000 7f5da9c16740 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55988d4d4e90 0x55988d4f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.134+0000 7f5da9c16740 4 client.8007 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:19.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.134+0000 7f5da9c16740 1 -- 172.21.15.139:0/1739052763 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55988d4062e0 con 0x55988d4d4e90 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.134+0000 7f5da4cb2640 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55988d4d4e90 0x55988d4f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.134+0000 7f5d7f7fe640 10 client.8007 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.134+0000 7f5da4cb2640 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55988d4d4e90 0x55988d4f7600 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f1f23246dfd3e38 server_cookie=fa290c7f5146ae18 in_seq=0 out_seq=0 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.134+0000 7f5d7f7fe640 1 -- 172.21.15.139:0/1739052763 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55988d4062e0 con 0x55988d4d4e90 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.134+0000 7f5d7f7fe640 10 client.8007 handle_command_reply: tid=0 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.134+0000 7f5da9c16740 1 client.8007 shutdown 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.135+0000 7f5da9c16740 1 -- 172.21.15.139:0/1739052763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4b3790 msgr2=0x55988d4dc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.135+0000 7f5da9c16740 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4b3790 0x55988d4dc700 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x55988d497070 tx=0x7f5d9802d070 comp rx=0 tx=0).stop 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.135+0000 7f5da9c16740 1 -- 172.21.15.139:0/1739052763 shutdown_connections 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.135+0000 7f5da9c16740 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55988d4d4e90 0x55988d4f7600 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.135+0000 7f5da9c16740 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4b3790 0x55988d4dc700 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.135+0000 7f5da9c16740 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcef0 0x55988d4d4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.135+0000 7f5d7f7fe640 0 client.8007 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T10:00:19.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.135+0000 7f5da9c16740 1 --2- 172.21.15.139:0/1739052763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 0x55988d4dd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.135+0000 7f5da9c16740 1 -- 172.21.15.139:0/1739052763 >> 172.21.15.139:0/1739052763 conn(0x55988d48f770 msgr2=0x55988d4bbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.135+0000 7f5da9c16740 1 -- 172.21.15.139:0/1739052763 shutdown_connections 2024-04-25T10:00:19.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.135+0000 7f5da9c16740 1 -- 172.21.15.139:0/1739052763 wait complete. 2024-04-25T10:00:19.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.135+0000 7f5da9c16740 20 client.8007 trim_cache size 0 max 16384 2024-04-25T10:00:19.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.150+0000 7f5da9c16740 1 Processor -- start 2024-04-25T10:00:19.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.150+0000 7f5da9c16740 1 -- start start 2024-04-25T10:00:19.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.150+0000 7f5da9c16740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4b3790 0x55988d4bf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.150+0000 7f5da9c16740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 0x55988d4bf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.150+0000 7f5da9c16740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 0x55988d48e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.150+0000 7f5da9c16740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01fcc60 con 0x55988d4bcef0 2024-04-25T10:00:19.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.150+0000 7f5da9c16740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01fbd80 con 0x55988d4bcb10 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.150+0000 7f5da9c16740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0140080 con 0x55988d4b3790 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5da54b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 0x55988d48e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5da54b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 0x55988d48e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35340/0 (socket says 172.21.15.139:35340) 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5da54b3640 1 -- 172.21.15.139:0/1074450191 learned_addr learned my addr 172.21.15.139:0/1074450191 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5d9ffff640 1 --2- 172.21.15.139:0/1074450191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 0x55988d4bf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5da4cb2640 1 --2- 172.21.15.139:0/1074450191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4b3790 0x55988d4bf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5da54b3640 1 -- 172.21.15.139:0/1074450191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4b3790 msgr2=0x55988d4bf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5da54b3640 1 --2- 172.21.15.139:0/1074450191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4b3790 0x55988d4bf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5da54b3640 1 -- 172.21.15.139:0/1074450191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 msgr2=0x55988d4bf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5da54b3640 1 --2- 172.21.15.139:0/1074450191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 0x55988d4bf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5da54b3640 1 -- 172.21.15.139:0/1074450191 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55988d4dd810 con 0x55988d4bcef0 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5da54b3640 1 --2- 172.21.15.139:0/1074450191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 0x55988d48e600 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f5d98005c20 tx=0x7f5d98046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b10a469171c514 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5d85ffb640 1 -- 172.21.15.139:0/1074450191 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d980415f0 con 0x55988d4bcef0 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5d85ffb640 1 -- 172.21.15.139:0/1074450191 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9802f030 con 0x55988d4bcef0 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.151+0000 7f5d85ffb640 1 -- 172.21.15.139:0/1074450191 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98041990 con 0x55988d4bcef0 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.152+0000 7f5da9c16740 1 -- 172.21.15.139:0/1074450191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 msgr2=0x55988d48e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.152+0000 7f5da9c16740 1 --2- 172.21.15.139:0/1074450191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 0x55988d48e600 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f5d98005c20 tx=0x7f5d98046000 comp rx=0 tx=0).stop 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.152+0000 7f5da9c16740 1 -- 172.21.15.139:0/1074450191 shutdown_connections 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.152+0000 7f5da9c16740 1 --2- 172.21.15.139:0/1074450191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 0x55988d48e600 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.152+0000 7f5da9c16740 1 --2- 172.21.15.139:0/1074450191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4bcb10 0x55988d4bf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.152+0000 7f5da9c16740 1 --2- 172.21.15.139:0/1074450191 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4b3790 0x55988d4bf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.152+0000 7f5da9c16740 1 -- 172.21.15.139:0/1074450191 >> 172.21.15.139:0/1074450191 conn(0x55988d48f770 msgr2=0x55988d490840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.152+0000 7f5da9c16740 1 -- 172.21.15.139:0/1074450191 shutdown_connections 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.152+0000 7f5da9c16740 1 -- 172.21.15.139:0/1074450191 wait complete. 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.152+0000 7f5da9c16740 1 -- start start 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5da9c16740 1 Processor -- start 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5da9c16740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4b3790 0x55988d4bf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5da9c16740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcb10 0x55988d54b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5da9c16740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 0x55988d4bb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5da9c16740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01fb760 con 0x55988d4bcef0 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5da9c16740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988d494590 con 0x55988d4b3790 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5da9c16740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55988d4d7980 con 0x55988d4bcb10 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5da9c16740 10 client.0 _ll_register_callbacks cb 0x7f5d880036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5da54b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 0x55988d4bb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5da54b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 0x55988d4bb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35350/0 (socket says 172.21.15.139:35350) 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5da54b3640 1 -- 172.21.15.139:0/2142039073 learned_addr learned my addr 172.21.15.139:0/2142039073 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5da4cb2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4b3790 0x55988d4bf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.153+0000 7f5d9ffff640 1 --2- 172.21.15.139:0/2142039073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcb10 0x55988d54b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da54b3640 1 -- 172.21.15.139:0/2142039073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcb10 msgr2=0x55988d54b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da54b3640 1 --2- 172.21.15.139:0/2142039073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcb10 0x55988d54b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da54b3640 1 -- 172.21.15.139:0/2142039073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4b3790 msgr2=0x55988d4bf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da54b3640 1 --2- 172.21.15.139:0/2142039073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4b3790 0x55988d4bf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da54b3640 1 -- 172.21.15.139:0/2142039073 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55988d0cd0f0 con 0x55988d4bcef0 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da54b3640 1 --2- 172.21.15.139:0/2142039073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 0x55988d4bb170 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f5d98002030 tx=0x7f5d98009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8a305e20b36b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5d7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5d7f7fe640 1 -- 172.21.15.139:0/2142039073 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98041d90 con 0x55988d4bcef0 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5d7f7fe640 1 -- 172.21.15.139:0/2142039073 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5d9804fa70 con 0x55988d4bcef0 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da9c16740 1 -- 172.21.15.139:0/2142039073 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d9802f030 con 0x55988d4bcef0 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5d7f7fe640 1 -- 172.21.15.139:0/2142039073 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d9804fc70 con 0x55988d4bcef0 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da9c16740 10 client.8013 fetch_fsmap learned FSMap version 57 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da9c16740 1 -- 172.21.15.139:0/2142039073 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55988d0cd0f0 con 0x55988d4bcef0 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5d7f7fe640 1 -- 172.21.15.139:0/2142039073 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9802f030 con 0x55988d4bcef0 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5d7f7fe640 1 -- 172.21.15.139:0/2142039073 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5d98058050 con 0x55988d4bcef0 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da9c16740 10 client.8013 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da9c16740 10 client.8013 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da9c16740 20 client.8013 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da9c16740 1 --2- 172.21.15.139:0/2142039073 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55988d4d6c00 0x55988d48e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da9c16740 4 client.8013 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da9c16740 1 -- 172.21.15.139:0/2142039073 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55988d4b4f30 con 0x55988d4d6c00 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.154+0000 7f5da4cb2640 1 --2- 172.21.15.139:0/2142039073 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55988d4d6c00 0x55988d48e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5d7f7fe640 10 client.8013 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da4cb2640 1 --2- 172.21.15.139:0/2142039073 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55988d4d6c00 0x55988d48e600 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a24db6e6af36319 server_cookie=9f87678867213623 in_seq=0 out_seq=0 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5d7f7fe640 1 -- 172.21.15.139:0/2142039073 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55988d4b4f30 con 0x55988d4d6c00 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5d7f7fe640 10 client.8013 handle_command_reply: tid=0 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da9c16740 1 client.8013 shutdown 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da9c16740 1 -- 172.21.15.139:0/2142039073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 msgr2=0x55988d4bb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da9c16740 1 --2- 172.21.15.139:0/2142039073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 0x55988d4bb170 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f5d98002030 tx=0x7f5d98009030 comp rx=0 tx=0).stop 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da9c16740 1 -- 172.21.15.139:0/2142039073 shutdown_connections 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da9c16740 1 --2- 172.21.15.139:0/2142039073 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55988d4d6c00 0x55988d48e600 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da9c16740 1 --2- 172.21.15.139:0/2142039073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55988d4bcef0 0x55988d4bb170 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da9c16740 1 --2- 172.21.15.139:0/2142039073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55988d4bcb10 0x55988d54b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5d7f7fe640 0 client.8013 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da9c16740 1 --2- 172.21.15.139:0/2142039073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d4b3790 0x55988d4bf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da9c16740 1 -- 172.21.15.139:0/2142039073 >> 172.21.15.139:0/2142039073 conn(0x55988d48f770 msgr2=0x55988d4e2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da9c16740 1 -- 172.21.15.139:0/2142039073 shutdown_connections 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da9c16740 1 -- 172.21.15.139:0/2142039073 wait complete. 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.155+0000 7f5da9c16740 20 client.8013 trim_cache size 0 max 16384 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:00:19.168 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "no active scrubs running", 2024-04-25T10:00:19.168 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": {} 2024-04-25T10:00:19.168 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:00:19.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.156+0000 7f5da6f16640 1 -- 172.21.15.139:0/363126459 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5d8c06b500 msgr2=0x7f5d8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.156+0000 7f5da6f16640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5d8c06b500 0x7f5d8c06d9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009bc0 tx=0x7f5d90009190 comp rx=0 tx=0).stop 2024-04-25T10:00:19.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.156+0000 7f5da6f16640 1 -- 172.21.15.139:0/363126459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5da013e930 msgr2=0x7f5da00ab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.156+0000 7f5da6f16640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5da013e930 0x7f5da00ab090 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400f850 tx=0x7f5d9400fc50 comp rx=0 tx=0).stop 2024-04-25T10:00:19.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.156+0000 7f5da6f16640 1 -- 172.21.15.139:0/363126459 shutdown_connections 2024-04-25T10:00:19.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.156+0000 7f5da6f16640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5da013e930 0x7f5da00ab090 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.156+0000 7f5da6f16640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5d8c06b500 0x7f5d8c06d9c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.156+0000 7f5da6f16640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5da006d470 0x7f5da013ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.156+0000 7f5da6f16640 1 --2- 172.21.15.139:0/363126459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d88004d40 0x7f5da00a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.156+0000 7f5da6f16640 1 -- 172.21.15.139:0/363126459 >> 172.21.15.139:0/363126459 conn(0x7f5da0067000 msgr2=0x7f5da00b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.156+0000 7f5da6f16640 1 -- 172.21.15.139:0/363126459 shutdown_connections 2024-04-25T10:00:19.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.156+0000 7f5da6f16640 1 -- 172.21.15.139:0/363126459 wait complete. 2024-04-25T10:00:19.171 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:00:19.172 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:19.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.263+0000 7fbed5b9b640 1 Processor -- start 2024-04-25T10:00:19.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.263+0000 7fbed5b9b640 1 -- start start 2024-04-25T10:00:19.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbed5b9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbed0147660 0x7fbed006f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbed5b9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbed0072620 0x7fbed00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbed5b9b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed00b26b0 con 0x7fbed00715e0 2024-04-25T10:00:19.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbed5b9b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed00b3ad0 con 0x7fbed0147660 2024-04-25T10:00:19.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbed5b9b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed006e050 con 0x7fbed0072620 2024-04-25T10:00:19.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbecf7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbed0147660 0x7fbed006f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbecf7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbed0147660 0x7fbed006f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35354/0 (socket says 172.21.15.139:35354) 2024-04-25T10:00:19.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbecf7fe640 1 -- 172.21.15.139:0/1714113008 learned_addr learned my addr 172.21.15.139:0/1714113008 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbeceffd640 1 --2- 172.21.15.139:0/1714113008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbed0072620 0x7fbed00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbecf7fe640 1 -- 172.21.15.139:0/1714113008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbed0072620 msgr2=0x7fbed00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbecf7fe640 1 --2- 172.21.15.139:0/1714113008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbed0072620 0x7fbed00b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbecf7fe640 1 -- 172.21.15.139:0/1714113008 >> v1:172.21.15.150:6789/0 conn(0x7fbed00715e0 legacy=0x7fbed0161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.264+0000 7fbecf7fe640 1 -- 172.21.15.139:0/1714113008 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed006e330 con 0x7fbed0147660 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.265+0000 7fbecf7fe640 1 --2- 172.21.15.139:0/1714113008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed0147660 0x7fbed006f0f0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009870 tx=0x7fbec002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a7f1c3bf45a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.265+0000 7fbece7fc640 1 -- 172.21.15.139:0/1714113008 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0037070 con 0x7fbed0147660 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.265+0000 7fbece7fc640 1 -- 172.21.15.139:0/1714113008 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec0032440 con 0x7fbed0147660 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.265+0000 7fbece7fc640 1 -- 172.21.15.139:0/1714113008 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0032810 con 0x7fbed0147660 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.266+0000 7fbed5b9b640 1 -- 172.21.15.139:0/1714113008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed0147660 msgr2=0x7fbed006f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.266+0000 7fbed5b9b640 1 --2- 172.21.15.139:0/1714113008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed0147660 0x7fbed006f0f0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009870 tx=0x7fbec002a180 comp rx=0 tx=0).stop 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.266+0000 7fbed5b9b640 1 -- 172.21.15.139:0/1714113008 shutdown_connections 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.266+0000 7fbed5b9b640 1 --2- 172.21.15.139:0/1714113008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbed0072620 0x7fbed00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.266+0000 7fbed5b9b640 1 --2- 172.21.15.139:0/1714113008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed0147660 0x7fbed006f0f0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.266+0000 7fbed5b9b640 1 -- 172.21.15.139:0/1714113008 >> 172.21.15.139:0/1714113008 conn(0x7fbed00b17a0 msgr2=0x7fbed00b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.266+0000 7fbed5b9b640 1 -- 172.21.15.139:0/1714113008 shutdown_connections 2024-04-25T10:00:19.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.266+0000 7fbed5b9b640 1 -- 172.21.15.139:0/1714113008 wait complete. 2024-04-25T10:00:19.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.267+0000 7fbed5b9b640 1 Processor -- start 2024-04-25T10:00:19.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.267+0000 7fbed5b9b640 1 -- start start 2024-04-25T10:00:19.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbed5b9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbed00715e0 0x7fbed014afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbed5b9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed0072620 0x7fbed014b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbed5b9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbed0147660 0x7fbed013b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbed5b9b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed006ebd0 con 0x7fbed0072620 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbed5b9b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed006e8f0 con 0x7fbed0147660 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbed5b9b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed0071a10 con 0x7fbed00715e0 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbeceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed0072620 0x7fbed014b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbeceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed0072620 0x7fbed014b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35368/0 (socket says 172.21.15.139:35368) 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbeceffd640 1 -- 172.21.15.139:0/3053123904 learned_addr learned my addr 172.21.15.139:0/3053123904 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbecf7fe640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbed00715e0 0x7fbed014afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbecffff640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbed0147660 0x7fbed013b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbeceffd640 1 -- 172.21.15.139:0/3053123904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbed00715e0 msgr2=0x7fbed014afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbeceffd640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbed00715e0 0x7fbed014afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbeceffd640 1 -- 172.21.15.139:0/3053123904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbed0147660 msgr2=0x7fbed013b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbeceffd640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbed0147660 0x7fbed013b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbeceffd640 1 -- 172.21.15.139:0/3053123904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed013a2b0 con 0x7fbed0072620 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbecffff640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbed0147660 0x7fbed013b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbeceffd640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed0072620 0x7fbed014b520 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fbec400d830 tx=0x7fbec400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbeb7fff640 1 -- 172.21.15.139:0/3053123904 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec400b6c0 con 0x7fbed0072620 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbeb7fff640 1 -- 172.21.15.139:0/3053123904 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec4019070 con 0x7fbed0072620 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.268+0000 7fbed5b9b640 1 -- 172.21.15.139:0/3053123904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed013a590 con 0x7fbed0072620 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.269+0000 7fbeb7fff640 1 -- 172.21.15.139:0/3053123904 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec400ba80 con 0x7fbed0072620 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.269+0000 7fbed5b9b640 1 -- 172.21.15.139:0/3053123904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed013ab50 con 0x7fbed0072620 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.269+0000 7fbed5b9b640 1 -- 172.21.15.139:0/3053123904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed0074520 con 0x7fbed0072620 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.270+0000 7fbeb7fff640 1 -- 172.21.15.139:0/3053123904 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbec401e460 con 0x7fbed0072620 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.270+0000 7fbeb7fff640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbebc06b4d0 0x7fbebc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.270+0000 7fbeb7fff640 1 -- 172.21.15.139:0/3053123904 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fbec408aa40 con 0x7fbed0072620 2024-04-25T10:00:19.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.270+0000 7fbecf7fe640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbebc06b4d0 0x7fbebc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.271+0000 7fbecf7fe640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbebc06b4d0 0x7fbebc06d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbec0034000 tx=0x7fbec002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.275+0000 7fbeb7fff640 1 -- 172.21.15.139:0/3053123904 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbec4059670 con 0x7fbed0072620 2024-04-25T10:00:19.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.478+0000 7fbed5b9b640 1 -- 172.21.15.139:0/3053123904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbed00b3160 con 0x7fbed0072620 2024-04-25T10:00:19.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.478+0000 7fbeb7fff640 1 -- 172.21.15.139:0/3053123904 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4954 (secure 0 0 0) 0x7fbec4058f70 con 0x7fbed0072620 2024-04-25T10:00:19.483 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:00:19.483 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T09:59:21.913360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":10,"addr":"172.21.15.139:6836/1239243954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1239243954},{"type":"v1","addr":"172.21.15.139:6836","nonce":1239243954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:19.484 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 57 2024-04-25T10:00:19.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.480+0000 7fbed5b9b640 1 -- 172.21.15.139:0/3053123904 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbebc06b4d0 msgr2=0x7fbebc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.480+0000 7fbed5b9b640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbebc06b4d0 0x7fbebc06d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbec0034000 tx=0x7fbec002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:19.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.480+0000 7fbed5b9b640 1 -- 172.21.15.139:0/3053123904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed0072620 msgr2=0x7fbed014b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.480+0000 7fbed5b9b640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed0072620 0x7fbed014b520 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fbec400d830 tx=0x7fbec400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.480+0000 7fbed5b9b640 1 -- 172.21.15.139:0/3053123904 shutdown_connections 2024-04-25T10:00:19.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.480+0000 7fbed5b9b640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed0072620 0x7fbed014b520 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.480+0000 7fbed5b9b640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbebc06b4d0 0x7fbebc06d990 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.480+0000 7fbed5b9b640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbed0147660 0x7fbed013b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.480+0000 7fbed5b9b640 1 --2- 172.21.15.139:0/3053123904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbed00715e0 0x7fbed014afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.480+0000 7fbed5b9b640 1 -- 172.21.15.139:0/3053123904 >> 172.21.15.139:0/3053123904 conn(0x7fbed00b17a0 msgr2=0x7fbed00aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.480+0000 7fbed5b9b640 1 -- 172.21.15.139:0/3053123904 shutdown_connections 2024-04-25T10:00:19.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.480+0000 7fbed5b9b640 1 -- 172.21.15.139:0/3053123904 wait complete. 2024-04-25T10:00:19.496 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:00:19.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.590+0000 7f50d8709640 1 Processor -- start 2024-04-25T10:00:19.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.590+0000 7f50d8709640 1 -- start start 2024-04-25T10:00:19.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.591+0000 7f50d8709640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f50d00af8a0 0x7f50d00b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.591+0000 7f50d8709640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d00c6cc0 con 0x7f50d006b700 2024-04-25T10:00:19.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.591+0000 7f50d8709640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d00c6fa0 con 0x7f50d00afc80 2024-04-25T10:00:19.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.591+0000 7f50d8709640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d00c7280 con 0x7f50d00af8a0 2024-04-25T10:00:19.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.591+0000 7f50d6ca6640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f50d006b700 0x7f50d006ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41566/0 (socket says 172.21.15.139:41566) 2024-04-25T10:00:19.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.591+0000 7f50d6ca6640 1 -- 172.21.15.139:0/2993087292 learned_addr learned my addr 172.21.15.139:0/2993087292 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:19.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.591+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1576582794 0 0) 0x7f50d00c6cc0 con 0x7f50d006b700 2024-04-25T10:00:19.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.591+0000 7f50d64a5640 1 --2- 172.21.15.139:0/2993087292 >> v2:172.21.15.150:3300/0 conn(0x7f50d00af8a0 0x7f50d00b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50bc003520 con 0x7f50d006b700 2024-04-25T10:00:19.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1809571830 0 0) 0x7f50bc003520 con 0x7f50d006b700 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 >> v2:172.21.15.150:3300/0 conn(0x7f50d00af8a0 msgr2=0x7f50d00b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 --2- 172.21.15.139:0/2993087292 >> v2:172.21.15.150:3300/0 conn(0x7f50d00af8a0 0x7f50d00b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 >> v1:172.21.15.150:6789/0 conn(0x7f50d00afc80 legacy=0x7f50d00cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d00cd090 con 0x7f50d006b700 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f50c80019f0 con 0x7f50d006b700 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 >> v1:172.21.15.139:6789/0 conn(0x7f50d006b700 legacy=0x7f50d006ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 --2- 172.21.15.139:0/2993087292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50bc0039b0 0x7f50bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 --2- 172.21.15.139:0/2993087292 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50bc004800 0x7f50bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 --2- 172.21.15.139:0/2993087292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 0x7f50bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00c7280 con 0x7f50bc0051c0 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc00c160 con 0x7f50bc0039b0 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc00c360 con 0x7f50bc004800 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d6ca6640 1 --2- 172.21.15.139:0/2993087292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 0x7f50bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d64a5640 1 --2- 172.21.15.139:0/2993087292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50bc0039b0 0x7f50bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/2993087292 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50bc004800 0x7f50bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d6ca6640 1 -- 172.21.15.139:0/2993087292 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50bc004800 msgr2=0x7f50bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d6ca6640 1 --2- 172.21.15.139:0/2993087292 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50bc004800 0x7f50bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d6ca6640 1 -- 172.21.15.139:0/2993087292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50bc0039b0 msgr2=0x7f50bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d6ca6640 1 --2- 172.21.15.139:0/2993087292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50bc0039b0 0x7f50bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.592+0000 7f50d6ca6640 1 -- 172.21.15.139:0/2993087292 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc003520 con 0x7f50bc0051c0 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.593+0000 7f50d6ca6640 1 -- 172.21.15.139:0/2993087292 reap_dead start 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.593+0000 7f50d6ca6640 1 --2- 172.21.15.139:0/2993087292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 0x7f50bc00b9c0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f50c800a860 tx=0x7f50bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2c318d4c25913b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.593+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c800a480 con 0x7f50bc0051c0 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.593+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c802f070 con 0x7f50bc0051c0 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.593+0000 7f50d54a3640 1 -- 172.21.15.139:0/2993087292 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c802bcc0 con 0x7f50bc0051c0 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.593+0000 7f50d8709640 1 -- 172.21.15.139:0/2993087292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 msgr2=0x7f50bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.593+0000 7f50d8709640 1 --2- 172.21.15.139:0/2993087292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 0x7f50bc00b9c0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f50c800a860 tx=0x7f50bc005830 comp rx=0 tx=0).stop 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.594+0000 7f50d8709640 1 -- 172.21.15.139:0/2993087292 shutdown_connections 2024-04-25T10:00:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.594+0000 7f50d8709640 1 --2- 172.21.15.139:0/2993087292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 0x7f50bc00b9c0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.594+0000 7f50d8709640 1 -- 172.21.15.139:0/2993087292 >> 172.21.15.139:0/2993087292 conn(0x7f50d00b2230 msgr2=0x7f50d00b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.594+0000 7f50d8709640 1 -- 172.21.15.139:0/2993087292 shutdown_connections 2024-04-25T10:00:19.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.594+0000 7f50d8709640 1 -- 172.21.15.139:0/2993087292 wait complete. 2024-04-25T10:00:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.595+0000 7f50d8709640 1 Processor -- start 2024-04-25T10:00:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.595+0000 7f50d8709640 1 -- start start 2024-04-25T10:00:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.595+0000 7f50d8709640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 0x7f50d00b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d8709640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50d014ea80 0x7f50d0156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d8709640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50d0156890 0x7f50d015a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d8709640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00b2ed0 con 0x7f50bc0051c0 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d8709640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00b2d50 con 0x7f50d014ea80 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d8709640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00b3050 con 0x7f50d0156890 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d64a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 0x7f50d00b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d64a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 0x7f50d00b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35386/0 (socket says 172.21.15.139:35386) 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d64a5640 1 -- 172.21.15.139:0/2770191515 learned_addr learned my addr 172.21.15.139:0/2770191515 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d6ca6640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50d0156890 0x7f50d015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d64a5640 1 -- 172.21.15.139:0/2770191515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50d0156890 msgr2=0x7f50d015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d64a5640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50d0156890 0x7f50d015a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50d014ea80 0x7f50d0156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d64a5640 1 -- 172.21.15.139:0/2770191515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50d014ea80 msgr2=0x7f50d0156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d64a5640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50d014ea80 0x7f50d0156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d64a5640 1 -- 172.21.15.139:0/2770191515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d00b38a0 con 0x7f50bc0051c0 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d64a5640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 0x7f50d00b3230 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f50cc011940 tx=0x7f50cc011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50c6ffd640 1 -- 172.21.15.139:0/2770191515 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00faf0 con 0x7f50bc0051c0 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50c6ffd640 1 -- 172.21.15.139:0/2770191515 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc00fc90 con 0x7f50bc0051c0 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d8709640 1 -- 172.21.15.139:0/2770191515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d00b3b80 con 0x7f50bc0051c0 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50c6ffd640 1 -- 172.21.15.139:0/2770191515 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc014440 con 0x7f50bc0051c0 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.596+0000 7f50d8709640 1 -- 172.21.15.139:0/2770191515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d00b4140 con 0x7f50bc0051c0 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.598+0000 7f50c6ffd640 1 -- 172.21.15.139:0/2770191515 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50cc012040 con 0x7f50bc0051c0 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.598+0000 7f50c6ffd640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50b406b4d0 0x7f50b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.598+0000 7f50c6ffd640 1 -- 172.21.15.139:0/2770191515 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f50cc08ab10 con 0x7f50bc0051c0 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.598+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50b406b4d0 0x7f50b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.599+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50b406b4d0 0x7f50b406d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f50c0000c00 tx=0x7f50c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.599+0000 7f50db405740 1 Processor -- start 2024-04-25T10:00:19.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.599+0000 7f50db405740 1 -- start start 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50db405740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 0x55dc3255d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50db405740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32597c60 0x55dc3255b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50db405740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32586050 0x55dc325864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50db405740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc324bec60 con 0x55dc32597c60 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50db405740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3245dd80 con 0x55dc32586050 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50db405740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc324bad00 con 0x55dc3255b590 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50d5ca4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32597c60 0x55dc3255b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50d5ca4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32597c60 0x55dc3255b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35392/0 (socket says 172.21.15.139:35392) 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50d5ca4640 1 -- 172.21.15.139:0/3023890046 learned_addr learned my addr 172.21.15.139:0/3023890046 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50d5ca4640 1 -- 172.21.15.139:0/3023890046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 msgr2=0x55dc3255d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/3023890046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 0x55dc3255d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50d5ca4640 1 -- 172.21.15.139:0/3023890046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32586050 msgr2=0x55dc325864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/3023890046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32586050 0x55dc325864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.600+0000 7f50d5ca4640 1 -- 172.21.15.139:0/3023890046 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc3229f7c0 con 0x55dc32597c60 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/3023890046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32597c60 0x55dc3255b970 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f50c0001590 tx=0x7f50c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11be24876da83985 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50b3fff640 1 -- 172.21.15.139:0/3023890046 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0027070 con 0x55dc32597c60 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50b3fff640 1 -- 172.21.15.139:0/3023890046 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0014740 con 0x55dc32597c60 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50b3fff640 1 -- 172.21.15.139:0/3023890046 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0014af0 con 0x55dc32597c60 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50db405740 1 -- 172.21.15.139:0/3023890046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32597c60 msgr2=0x55dc3255b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50db405740 1 --2- 172.21.15.139:0/3023890046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32597c60 0x55dc3255b970 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f50c0001590 tx=0x7f50c0013000 comp rx=0 tx=0).stop 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50db405740 1 -- 172.21.15.139:0/3023890046 shutdown_connections 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50db405740 1 --2- 172.21.15.139:0/3023890046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32597c60 0x55dc3255b970 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50db405740 1 --2- 172.21.15.139:0/3023890046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32586050 0x55dc325864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50db405740 1 --2- 172.21.15.139:0/3023890046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 0x55dc3255d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50db405740 1 -- 172.21.15.139:0/3023890046 >> 172.21.15.139:0/3023890046 conn(0x55dc325236e0 msgr2=0x55dc325a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50db405740 1 -- 172.21.15.139:0/3023890046 shutdown_connections 2024-04-25T10:00:19.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.601+0000 7f50db405740 1 -- 172.21.15.139:0/3023890046 wait complete. 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.602+0000 7f50db405740 1 -- start start 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.602+0000 7f50db405740 1 Processor -- start 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50db405740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 0x55dc325882c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50db405740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc325b5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50db405740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32597c60 0x55dc32598080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50db405740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3245b840 con 0x55dc32586050 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50db405740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc32583ba0 con 0x55dc32597c60 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50db405740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc32598c20 con 0x55dc3255b590 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50db405740 10 client.0 _ll_register_callbacks cb 0x55dc32475980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50d5ca4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc325b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50d5ca4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc325b5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35404/0 (socket says 172.21.15.139:35404) 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50d5ca4640 1 -- 172.21.15.139:0/2263595397 learned_addr learned my addr 172.21.15.139:0/2263595397 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50d64a5640 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 0x55dc325882c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50d6ca6640 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32597c60 0x55dc32598080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50d5ca4640 1 -- 172.21.15.139:0/2263595397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 msgr2=0x55dc325882c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 0x55dc325882c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50d5ca4640 1 -- 172.21.15.139:0/2263595397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32597c60 msgr2=0x55dc32598080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32597c60 0x55dc32598080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50d5ca4640 1 -- 172.21.15.139:0/2263595397 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc322fff50 con 0x55dc32586050 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.603+0000 7f50d6ca6640 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32597c60 0x55dc32598080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc325b5040 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f50c0025040 tx=0x7f50c0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bae57895744d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50b17fa640 1 -- 172.21.15.139:0/2263595397 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0027040 con 0x55dc32586050 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50b17fa640 1 -- 172.21.15.139:0/2263595397 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f50c0021620 con 0x55dc32586050 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50db405740 1 -- 172.21.15.139:0/2263595397 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50c0014740 con 0x55dc32586050 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50b17fa640 1 -- 172.21.15.139:0/2263595397 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50c0039050 con 0x55dc32586050 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50db405740 10 client.8031 fetch_fsmap learned FSMap version 57 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50db405740 1 -- 172.21.15.139:0/2263595397 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55dc321930b0 con 0x55dc32586050 2024-04-25T10:00:19.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50b17fa640 1 -- 172.21.15.139:0/2263595397 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0014740 con 0x55dc32586050 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50b17fa640 1 -- 172.21.15.139:0/2263595397 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f50c003c020 con 0x55dc32586050 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50db405740 10 client.8031 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50db405740 10 client.8031 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50db405740 20 client.8031 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50db405740 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55dc325864d0 0x55dc325868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50db405740 4 client.8031 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50db405740 1 -- 172.21.15.139:0/2263595397 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc3247ee60 con 0x55dc325864d0 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.604+0000 7f50d64a5640 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55dc325864d0 0x55dc325868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50b17fa640 10 client.8031 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50d64a5640 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55dc325864d0 0x55dc325868b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1182c89cfc76a7ff server_cookie=4aec4c499358fd32 in_seq=0 out_seq=0 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50b17fa640 1 -- 172.21.15.139:0/2263595397 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc3247ee60 con 0x55dc325864d0 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50b17fa640 10 client.8031 handle_command_reply: tid=0 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50db405740 1 client.8031 shutdown 2024-04-25T10:00:19.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50db405740 1 -- 172.21.15.139:0/2263595397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 msgr2=0x55dc325b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50db405740 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc325b5040 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f50c0025040 tx=0x7f50c0001120 comp rx=0 tx=0).stop 2024-04-25T10:00:19.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50db405740 1 -- 172.21.15.139:0/2263595397 shutdown_connections 2024-04-25T10:00:19.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50db405740 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55dc325864d0 0x55dc325868b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50db405740 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc325b5040 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50db405740 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32597c60 0x55dc32598080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50b17fa640 0 client.8031 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T10:00:19.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50db405740 1 --2- 172.21.15.139:0/2263595397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 0x55dc325882c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50db405740 1 -- 172.21.15.139:0/2263595397 >> 172.21.15.139:0/2263595397 conn(0x55dc325236e0 msgr2=0x55dc32583fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50db405740 1 -- 172.21.15.139:0/2263595397 shutdown_connections 2024-04-25T10:00:19.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.605+0000 7f50db405740 1 -- 172.21.15.139:0/2263595397 wait complete. 2024-04-25T10:00:19.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.606+0000 7f50db405740 20 client.8031 trim_cache size 0 max 16384 2024-04-25T10:00:19.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.617+0000 7f50db405740 1 Processor -- start 2024-04-25T10:00:19.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.617+0000 7f50db405740 1 -- start start 2024-04-25T10:00:19.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50db405740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 0x55dc3259a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50db405740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc3259ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50db405740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32597c60 0x55dc325868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50db405740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d0200060 con 0x55dc32586050 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50db405740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d01ff590 con 0x55dc32597c60 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50db405740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d01ffa90 con 0x55dc3255b590 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50d5ca4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc3259ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50d5ca4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc3259ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35410/0 (socket says 172.21.15.139:35410) 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50d5ca4640 1 -- 172.21.15.139:0/3727488777 learned_addr learned my addr 172.21.15.139:0/3727488777 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50d64a5640 1 --2- 172.21.15.139:0/3727488777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 0x55dc3259a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50d5ca4640 1 -- 172.21.15.139:0/3727488777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 msgr2=0x55dc3259a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/3727488777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 0x55dc3259a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50d5ca4640 1 -- 172.21.15.139:0/3727488777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32597c60 msgr2=0x55dc325868b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/3727488777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32597c60 0x55dc325868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50d5ca4640 1 -- 172.21.15.139:0/3727488777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc325c4410 con 0x55dc32586050 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/3727488777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc3259ac80 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f50c00145f0 tx=0x7f50c002b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6e5ffe710959bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50b37fe640 1 -- 172.21.15.139:0/3727488777 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0041020 con 0x55dc32586050 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50b37fe640 1 -- 172.21.15.139:0/3727488777 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0031070 con 0x55dc32586050 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.618+0000 7f50b37fe640 1 -- 172.21.15.139:0/3727488777 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0014720 con 0x55dc32586050 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.619+0000 7f50db405740 1 -- 172.21.15.139:0/3727488777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 msgr2=0x55dc3259ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.619+0000 7f50db405740 1 --2- 172.21.15.139:0/3727488777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc3259ac80 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f50c00145f0 tx=0x7f50c002b000 comp rx=0 tx=0).stop 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.619+0000 7f50db405740 1 -- 172.21.15.139:0/3727488777 shutdown_connections 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.619+0000 7f50db405740 1 --2- 172.21.15.139:0/3727488777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc3259ac80 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.619+0000 7f50db405740 1 --2- 172.21.15.139:0/3727488777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc32597c60 0x55dc325868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.619+0000 7f50db405740 1 --2- 172.21.15.139:0/3727488777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc3255b590 0x55dc3259a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.619+0000 7f50db405740 1 -- 172.21.15.139:0/3727488777 >> 172.21.15.139:0/3727488777 conn(0x55dc325236e0 msgr2=0x55dc32556090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.619+0000 7f50db405740 1 -- 172.21.15.139:0/3727488777 shutdown_connections 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.619+0000 7f50db405740 1 -- 172.21.15.139:0/3727488777 wait complete. 2024-04-25T10:00:19.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.619+0000 7f50db405740 1 -- start start 2024-04-25T10:00:19.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.620+0000 7f50db405740 1 Processor -- start 2024-04-25T10:00:19.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.620+0000 7f50db405740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc3255b590 0x55dc3259a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.620+0000 7f50db405740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc3259ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.620+0000 7f50db405740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc32597c60 0x55dc325b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.620+0000 7f50db405740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc325985c0 con 0x55dc32586050 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.620+0000 7f50db405740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc325b5630 con 0x55dc3255b590 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.620+0000 7f50db405740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc3259a200 con 0x55dc32597c60 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.620+0000 7f50db405740 10 client.0 _ll_register_callbacks cb 0x7f50c00051d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.620+0000 7f50d5ca4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc3259ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50d5ca4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc3259ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35420/0 (socket says 172.21.15.139:35420) 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50d5ca4640 1 -- 172.21.15.139:0/463097832 learned_addr learned my addr 172.21.15.139:0/463097832 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50d6ca6640 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc32597c60 0x55dc325b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50d64a5640 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc3255b590 0x55dc3259a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50d5ca4640 1 -- 172.21.15.139:0/463097832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc32597c60 msgr2=0x55dc325b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc32597c60 0x55dc325b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50d5ca4640 1 -- 172.21.15.139:0/463097832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc3255b590 msgr2=0x55dc3259a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc3255b590 0x55dc3259a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50d5ca4640 1 -- 172.21.15.139:0/463097832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc32586d80 con 0x55dc32586050 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50d64a5640 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc3255b590 0x55dc3259a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50d5ca4640 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc3259ac80 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f50c0007920 tx=0x7f50c000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00c2321797fc27d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50b17fa640 1 -- 172.21.15.139:0/463097832 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c004c060 con 0x55dc32586050 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50b17fa640 1 -- 172.21.15.139:0/463097832 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f50c0034550 con 0x55dc32586050 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50db405740 1 -- 172.21.15.139:0/463097832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50c0031070 con 0x55dc32586050 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50b17fa640 1 -- 172.21.15.139:0/463097832 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50c0027030 con 0x55dc32586050 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50db405740 10 client.8037 fetch_fsmap learned FSMap version 57 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50db405740 1 -- 172.21.15.139:0/463097832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55dc32586d80 con 0x55dc32586050 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50b17fa640 1 -- 172.21.15.139:0/463097832 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0031070 con 0x55dc32586050 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.621+0000 7f50b17fa640 1 -- 172.21.15.139:0/463097832 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f50c003c070 con 0x55dc32586050 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 10 client.8037 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 10 client.8037 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:00:19.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 20 client.8037 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55dc32596a50 0x55dc325b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 4 client.8037 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 -- 172.21.15.139:0/463097832 --> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55dc324cc1a0 con 0x55dc32596a50 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50d64a5640 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55dc32596a50 0x55dc325b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50b17fa640 10 client.8037 ms_handle_connect on v2:172.21.15.139:6834/1239243954 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50d64a5640 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55dc32596a50 0x55dc325b3ce0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc7796902ca14c60 server_cookie=9824a235399aab92 in_seq=0 out_seq=0 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50b17fa640 1 -- 172.21.15.139:0/463097832 <== mds.0 v2:172.21.15.139:6834/1239243954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+144 (crc 0 0 0) 0x55dc324cc1a0 con 0x55dc32596a50 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50b17fa640 10 client.8037 handle_command_reply: tid=0 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 client.8037 shutdown 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 -- 172.21.15.139:0/463097832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 msgr2=0x55dc3259ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc3259ac80 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f50c0007920 tx=0x7f50c000c000 comp rx=0 tx=0).stop 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 -- 172.21.15.139:0/463097832 shutdown_connections 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.139:6834/1239243954,v1:172.21.15.139:6836/1239243954] conn(0x55dc32596a50 0x55dc325b3ce0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dc32586050 0x55dc3259ac80 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50b17fa640 0 client.8037 ms_handle_reset on v2:172.21.15.139:6834/1239243954 2024-04-25T10:00:19.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55dc32597c60 0x55dc325b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 --2- 172.21.15.139:0/463097832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc3255b590 0x55dc3259a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 -- 172.21.15.139:0/463097832 >> 172.21.15.139:0/463097832 conn(0x55dc325236e0 msgr2=0x55dc3255bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 -- 172.21.15.139:0/463097832 shutdown_connections 2024-04-25T10:00:19.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.622+0000 7f50db405740 1 -- 172.21.15.139:0/463097832 wait complete. 2024-04-25T10:00:19.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50db405740 20 client.8037 trim_cache size 0 max 16384 2024-04-25T10:00:19.634 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:00:19.634 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-04-25T10:00:19.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:00:19.635 INFO:teuthology.orchestra.run.smithi139.stdout: "damage_type": "backtrace", 2024-04-25T10:00:19.635 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 2335747626, 2024-04-25T10:00:19.635 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 1099511627777, 2024-04-25T10:00:19.635 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/testdir/hardlink" 2024-04-25T10:00:19.635 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:00:19.635 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-04-25T10:00:19.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50d8709640 1 -- 172.21.15.139:0/2770191515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50b406b4d0 msgr2=0x7f50b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50d8709640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50b406b4d0 0x7f50b406d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f50c0000c00 tx=0x7f50c0009040 comp rx=0 tx=0).stop 2024-04-25T10:00:19.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50d8709640 1 -- 172.21.15.139:0/2770191515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 msgr2=0x7f50d00b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50d8709640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 0x7f50d00b3230 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f50cc011940 tx=0x7f50cc011e00 comp rx=0 tx=0).stop 2024-04-25T10:00:19.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50d8709640 1 -- 172.21.15.139:0/2770191515 shutdown_connections 2024-04-25T10:00:19.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50d8709640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f50b406b4d0 0x7f50b406d990 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50d8709640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50d0156890 0x7f50d015a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50d8709640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50d014ea80 0x7f50d0156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50d8709640 1 --2- 172.21.15.139:0/2770191515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50bc0051c0 0x7f50d00b3230 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50d8709640 1 -- 172.21.15.139:0/2770191515 >> 172.21.15.139:0/2770191515 conn(0x7f50d00b2230 msgr2=0x7f50d00c65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50d8709640 1 -- 172.21.15.139:0/2770191515 shutdown_connections 2024-04-25T10:00:19.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.623+0000 7f50d8709640 1 -- 172.21.15.139:0/2770191515 wait complete. 2024-04-25T10:00:19.638 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:00:19.638 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:19.653 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:00:19.654 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:19.709 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:19.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.841+0000 7f2f4d7f5640 1 Processor -- start 2024-04-25T10:00:19.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.841+0000 7f2f4d7f5640 1 -- start start 2024-04-25T10:00:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.842+0000 7f2f4d7f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f480d1860 0x7f2f480cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.842+0000 7f2f4d7f5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480cbf80 con 0x7f2f480cfba0 2024-04-25T10:00:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.842+0000 7f2f4d7f5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480cd2a0 con 0x7f2f48158240 2024-04-25T10:00:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.842+0000 7f2f4d7f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48158620 con 0x7f2f480d1860 2024-04-25T10:00:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.842+0000 7f2f477fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2f480cfba0 0x7f2f480cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41580/0 (socket says 172.21.15.139:41580) 2024-04-25T10:00:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.842+0000 7f2f477fe640 1 -- 172.21.15.139:0/2605338159 learned_addr learned my addr 172.21.15.139:0/2605338159 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.842+0000 7f2f467fc640 1 --2- 172.21.15.139:0/2605338159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f480d1860 0x7f2f480cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.843+0000 7f2f45ffb640 1 -- 172.21.15.139:0/2605338159 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445972319 0 0) 0x7f2f40001110 con 0x7f2f480cfba0 2024-04-25T10:00:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.843+0000 7f2f45ffb640 1 -- 172.21.15.139:0/2605338159 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f30003520 con 0x7f2f480cfba0 2024-04-25T10:00:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.843+0000 7f2f467fc640 1 -- 172.21.15.139:0/2605338159 >> v1:172.21.15.150:6789/0 conn(0x7f2f48158240 legacy=0x7f2f480d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.843+0000 7f2f467fc640 1 -- 172.21.15.139:0/2605338159 >> v1:172.21.15.139:6789/0 conn(0x7f2f480cfba0 legacy=0x7f2f480cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.843+0000 7f2f467fc640 1 -- 172.21.15.139:0/2605338159 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480cc200 con 0x7f2f480d1860 2024-04-25T10:00:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.843+0000 7f2f467fc640 1 --2- 172.21.15.139:0/2605338159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f480d1860 0x7f2f480cf660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0097d0 tx=0x7f2f3c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ada151383d33e17a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.844+0000 7f2f45ffb640 1 -- 172.21.15.139:0/2605338159 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c037070 con 0x7f2f480d1860 2024-04-25T10:00:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.844+0000 7f2f45ffb640 1 -- 172.21.15.139:0/2605338159 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c032440 con 0x7f2f480d1860 2024-04-25T10:00:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.844+0000 7f2f45ffb640 1 -- 172.21.15.139:0/2605338159 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c0327f0 con 0x7f2f480d1860 2024-04-25T10:00:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.844+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/2605338159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f480d1860 msgr2=0x7f2f480cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.844+0000 7f2f4d7f5640 1 --2- 172.21.15.139:0/2605338159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f480d1860 0x7f2f480cf660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0097d0 tx=0x7f2f3c02a210 comp rx=0 tx=0).stop 2024-04-25T10:00:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.844+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/2605338159 shutdown_connections 2024-04-25T10:00:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.844+0000 7f2f4d7f5640 1 --2- 172.21.15.139:0/2605338159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f480d1860 0x7f2f480cf660 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.844+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/2605338159 >> 172.21.15.139:0/2605338159 conn(0x7f2f480b3380 msgr2=0x7f2f480b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.844+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/2605338159 shutdown_connections 2024-04-25T10:00:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.844+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/2605338159 wait complete. 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.845+0000 7f2f4d7f5640 1 Processor -- start 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.846+0000 7f2f4d7f5640 1 -- start start 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.846+0000 7f2f4d7f5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f480cfba0 0x7f2f481f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.846+0000 7f2f4d7f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f480d1860 0x7f2f480b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.846+0000 7f2f4d7f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f48158240 0x7f2f481f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.846+0000 7f2f4d7f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480cc7c0 con 0x7f2f48158240 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.846+0000 7f2f4d7f5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480cce10 con 0x7f2f480cfba0 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.846+0000 7f2f4d7f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480ccaa0 con 0x7f2f480d1860 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.846+0000 7f2f477fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f48158240 0x7f2f481f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.846+0000 7f2f477fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f48158240 0x7f2f481f05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35430/0 (socket says 172.21.15.139:35430) 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.846+0000 7f2f477fe640 1 -- 172.21.15.139:0/4227374296 learned_addr learned my addr 172.21.15.139:0/4227374296 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f467fc640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f480d1860 0x7f2f480b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f46ffd640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f480cfba0 0x7f2f481f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f477fe640 1 -- 172.21.15.139:0/4227374296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f480d1860 msgr2=0x7f2f480b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f477fe640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f480d1860 0x7f2f480b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f477fe640 1 -- 172.21.15.139:0/4227374296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f480cfba0 msgr2=0x7f2f481f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f477fe640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f480cfba0 0x7f2f481f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f477fe640 1 -- 172.21.15.139:0/4227374296 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f481f7c50 con 0x7f2f48158240 2024-04-25T10:00:19.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f477fe640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f48158240 0x7f2f481f05a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000c730 tx=0x7f2f4000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f2f7fe640 1 -- 172.21.15.139:0/4227374296 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40017070 con 0x7f2f48158240 2024-04-25T10:00:19.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f2f7fe640 1 -- 172.21.15.139:0/4227374296 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f400058b0 con 0x7f2f48158240 2024-04-25T10:00:19.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/4227374296 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f480c5590 con 0x7f2f48158240 2024-04-25T10:00:19.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f2f7fe640 1 -- 172.21.15.139:0/4227374296 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40005ce0 con 0x7f2f48158240 2024-04-25T10:00:19.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.847+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/4227374296 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f480c5b50 con 0x7f2f48158240 2024-04-25T10:00:19.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.848+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/4227374296 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f48074520 con 0x7f2f48158240 2024-04-25T10:00:19.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.849+0000 7f2f2f7fe640 1 -- 172.21.15.139:0/4227374296 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f4001a070 con 0x7f2f48158240 2024-04-25T10:00:19.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.849+0000 7f2f2f7fe640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2f3406b4d0 0x7f2f3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.849+0000 7f2f2f7fe640 1 -- 172.21.15.139:0/4227374296 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f2f400896e0 con 0x7f2f48158240 2024-04-25T10:00:19.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.849+0000 7f2f46ffd640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2f3406b4d0 0x7f2f3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.849+0000 7f2f46ffd640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2f3406b4d0 0x7f2f3406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2f38001e70 tx=0x7f2f38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:19.853+0000 7f2f2f7fe640 1 -- 172.21.15.139:0/4227374296 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f481f7c50 con 0x7f2f48158240 2024-04-25T10:00:20.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.023+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/4227374296 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f4815db20 con 0x7f2f48158240 2024-04-25T10:00:20.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.024+0000 7f2f2f7fe640 1 -- 172.21.15.139:0/4227374296 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17257 (secure 0 0 0) 0x7f2f480cbf80 con 0x7f2f48158240 2024-04-25T10:00:20.029 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:00:20.030 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":92,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T09:59:58.154612+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:55.585380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:56.336230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6835/2266713846":"2024-04-26T09:59:10.897062+0000","172.21.15.150:6833/2266713846":"2024-04-26T09:59:10.897062+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:20.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.026+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/4227374296 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2f3406b4d0 msgr2=0x7f2f3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.026+0000 7f2f4d7f5640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2f3406b4d0 0x7f2f3406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2f38001e70 tx=0x7f2f38009040 comp rx=0 tx=0).stop 2024-04-25T10:00:20.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.026+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/4227374296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f48158240 msgr2=0x7f2f481f05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.026+0000 7f2f4d7f5640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f48158240 0x7f2f481f05a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000c730 tx=0x7f2f4000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.026+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/4227374296 shutdown_connections 2024-04-25T10:00:20.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.026+0000 7f2f4d7f5640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f48158240 0x7f2f481f05a0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.026+0000 7f2f4d7f5640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2f3406b4d0 0x7f2f3406d990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.026+0000 7f2f4d7f5640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f480d1860 0x7f2f480b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.026+0000 7f2f4d7f5640 1 --2- 172.21.15.139:0/4227374296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f480cfba0 0x7f2f481f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.026+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/4227374296 >> 172.21.15.139:0/4227374296 conn(0x7f2f480b3380 msgr2=0x7f2f4815bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.026+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/4227374296 shutdown_connections 2024-04-25T10:00:20.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.026+0000 7f2f4d7f5640 1 -- 172.21.15.139:0/4227374296 wait complete. 2024-04-25T10:00:20.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:58:55.585380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:58:56.336230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:00:20.043 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:00:20.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.137+0000 7f1a19ff9640 1 Processor -- start 2024-04-25T10:00:20.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.137+0000 7f1a19ff9640 1 -- start start 2024-04-25T10:00:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a19ff9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1a140712f0 0x7f1a140b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a19ff9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1a140719d0 0x7f1a140b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a19ff9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a140b1100 con 0x7f1a14145250 2024-04-25T10:00:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a19ff9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a140ceed0 con 0x7f1a140719d0 2024-04-25T10:00:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a19ff9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a140ce2c0 con 0x7f1a140712f0 2024-04-25T10:00:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a137fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1a14145250 0x7f1a14070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41582/0 (socket says 172.21.15.139:41582) 2024-04-25T10:00:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a137fe640 1 -- 172.21.15.139:0/4170551362 learned_addr learned my addr 172.21.15.139:0/4170551362 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a13fff640 1 --2- 172.21.15.139:0/4170551362 >> v2:172.21.15.139:3300/0 conn(0x7f1a140719d0 0x7f1a140b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a12ffd640 1 --2- 172.21.15.139:0/4170551362 >> v2:172.21.15.150:3300/0 conn(0x7f1a140712f0 0x7f1a140b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a13fff640 1 -- 172.21.15.139:0/4170551362 >> v2:172.21.15.150:3300/0 conn(0x7f1a140712f0 msgr2=0x7f1a140b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a13fff640 1 --2- 172.21.15.139:0/4170551362 >> v2:172.21.15.150:3300/0 conn(0x7f1a140712f0 0x7f1a140b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a13fff640 1 -- 172.21.15.139:0/4170551362 >> v1:172.21.15.139:6789/0 conn(0x7f1a14145250 legacy=0x7f1a14070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a127fc640 1 -- 172.21.15.139:0/4170551362 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272489081 0 0) 0x7f1a140b1100 con 0x7f1a14145250 2024-04-25T10:00:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a13fff640 1 -- 172.21.15.139:0/4170551362 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a140ce5a0 con 0x7f1a140719d0 2024-04-25T10:00:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.138+0000 7f1a12ffd640 1 --2- 172.21.15.139:0/4170551362 >> v2:172.21.15.150:3300/0 conn(0x7f1a140712f0 0x7f1a140b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.139+0000 7f1a13fff640 1 --2- 172.21.15.139:0/4170551362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a140719d0 0x7f1a140b4bc0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f1a14071980 tx=0x7f1a0c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d755873a02ef3222 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.139+0000 7f1a127fc640 1 -- 172.21.15.139:0/4170551362 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c007740 con 0x7f1a140719d0 2024-04-25T10:00:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.139+0000 7f1a127fc640 1 -- 172.21.15.139:0/4170551362 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c0078e0 con 0x7f1a140719d0 2024-04-25T10:00:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.139+0000 7f1a127fc640 1 -- 172.21.15.139:0/4170551362 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c007cb0 con 0x7f1a140719d0 2024-04-25T10:00:20.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.140+0000 7f1a19ff9640 1 -- 172.21.15.139:0/4170551362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a140719d0 msgr2=0x7f1a140b4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.140+0000 7f1a19ff9640 1 --2- 172.21.15.139:0/4170551362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a140719d0 0x7f1a140b4bc0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f1a14071980 tx=0x7f1a0c02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.140+0000 7f1a19ff9640 1 -- 172.21.15.139:0/4170551362 shutdown_connections 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.140+0000 7f1a19ff9640 1 --2- 172.21.15.139:0/4170551362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a140719d0 0x7f1a140b4bc0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.140+0000 7f1a19ff9640 1 --2- 172.21.15.139:0/4170551362 >> v2:172.21.15.150:3300/0 conn(0x7f1a140712f0 0x7f1a140b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.140+0000 7f1a19ff9640 1 -- 172.21.15.139:0/4170551362 >> 172.21.15.139:0/4170551362 conn(0x7f1a1406f460 msgr2=0x7f1a1406f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.140+0000 7f1a19ff9640 1 -- 172.21.15.139:0/4170551362 shutdown_connections 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.140+0000 7f1a19ff9640 1 -- 172.21.15.139:0/4170551362 wait complete. 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.141+0000 7f1a19ff9640 1 Processor -- start 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.141+0000 7f1a19ff9640 1 -- start start 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a19ff9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a140712f0 0x7f1a1415b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a19ff9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a140719d0 0x7f1a141e6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a19ff9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a14145250 0x7f1a14157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a19ff9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a14145630 con 0x7f1a140719d0 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a19ff9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140ceb60 con 0x7f1a140712f0 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a19ff9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140d11a0 con 0x7f1a14145250 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a12ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a140719d0 0x7f1a141e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a12ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a140719d0 0x7f1a141e6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35448/0 (socket says 172.21.15.139:35448) 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a12ffd640 1 -- 172.21.15.139:0/1224284894 learned_addr learned my addr 172.21.15.139:0/1224284894 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a137fe640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a140712f0 0x7f1a1415b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a13fff640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a14145250 0x7f1a14157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a12ffd640 1 -- 172.21.15.139:0/1224284894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a14145250 msgr2=0x7f1a14157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a12ffd640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a14145250 0x7f1a14157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a12ffd640 1 -- 172.21.15.139:0/1224284894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a140712f0 msgr2=0x7f1a1415b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a12ffd640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a140712f0 0x7f1a1415b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a12ffd640 1 -- 172.21.15.139:0/1224284894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a14159bb0 con 0x7f1a140719d0 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a13fff640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a14145250 0x7f1a14157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f1a12ffd640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a140719d0 0x7f1a141e6070 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800c7f0 tx=0x7f1a0800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.142+0000 7f19fbfff640 1 -- 172.21.15.139:0/1224284894 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a08015070 con 0x7f1a140719d0 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.143+0000 7f19fbfff640 1 -- 172.21.15.139:0/1224284894 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a08007a30 con 0x7f1a140719d0 2024-04-25T10:00:20.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.143+0000 7f19fbfff640 1 -- 172.21.15.139:0/1224284894 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a08007e00 con 0x7f1a140719d0 2024-04-25T10:00:20.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.143+0000 7f1a19ff9640 1 -- 172.21.15.139:0/1224284894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a14159e90 con 0x7f1a140719d0 2024-04-25T10:00:20.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.143+0000 7f1a19ff9640 1 -- 172.21.15.139:0/1224284894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1415a450 con 0x7f1a140719d0 2024-04-25T10:00:20.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.144+0000 7f1a19ff9640 1 -- 172.21.15.139:0/1224284894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a140b0cf0 con 0x7f1a140719d0 2024-04-25T10:00:20.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.144+0000 7f19fbfff640 1 -- 172.21.15.139:0/1224284894 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a08018070 con 0x7f1a140719d0 2024-04-25T10:00:20.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.144+0000 7f19fbfff640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1a0006b500 0x7f1a0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.144+0000 7f19fbfff640 1 -- 172.21.15.139:0/1224284894 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1a0808a400 con 0x7f1a140719d0 2024-04-25T10:00:20.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.144+0000 7f1a137fe640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1a0006b500 0x7f1a0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.145+0000 7f1a137fe640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1a0006b500 0x7f1a0006d9c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1a04000f60 tx=0x7f1a04007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.149+0000 7f19fbfff640 1 -- 172.21.15.139:0/1224284894 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a140b0cf0 con 0x7f1a140719d0 2024-04-25T10:00:20.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.321+0000 7f1a19ff9640 1 -- 172.21.15.139:0/1224284894 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a14074520 con 0x7f1a140719d0 2024-04-25T10:00:20.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.321+0000 7f19fbfff640 1 -- 172.21.15.139:0/1224284894 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a08059030 con 0x7f1a140719d0 2024-04-25T10:00:20.327 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:00:20.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.323+0000 7f1a19ff9640 1 -- 172.21.15.139:0/1224284894 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1a0006b500 msgr2=0x7f1a0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.323+0000 7f1a19ff9640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1a0006b500 0x7f1a0006d9c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1a04000f60 tx=0x7f1a04007c00 comp rx=0 tx=0).stop 2024-04-25T10:00:20.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.323+0000 7f1a19ff9640 1 -- 172.21.15.139:0/1224284894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a140719d0 msgr2=0x7f1a141e6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.323+0000 7f1a19ff9640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a140719d0 0x7f1a141e6070 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800c7f0 tx=0x7f1a0800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.324+0000 7f1a19ff9640 1 -- 172.21.15.139:0/1224284894 shutdown_connections 2024-04-25T10:00:20.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.324+0000 7f1a19ff9640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a140719d0 0x7f1a141e6070 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.324+0000 7f1a19ff9640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1a0006b500 0x7f1a0006d9c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.324+0000 7f1a19ff9640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a14145250 0x7f1a14157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.324+0000 7f1a19ff9640 1 --2- 172.21.15.139:0/1224284894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a140712f0 0x7f1a1415b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.324+0000 7f1a19ff9640 1 -- 172.21.15.139:0/1224284894 >> 172.21.15.139:0/1224284894 conn(0x7f1a1406f460 msgr2=0x7f1a140d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.324+0000 7f1a19ff9640 1 -- 172.21.15.139:0/1224284894 shutdown_connections 2024-04-25T10:00:20.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.324+0000 7f1a19ff9640 1 -- 172.21.15.139:0/1224284894 wait complete. 2024-04-25T10:00:20.340 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:00:20.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.434+0000 7fad31a5a640 1 Processor -- start 2024-04-25T10:00:20.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.434+0000 7fad31a5a640 1 -- start start 2024-04-25T10:00:20.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.435+0000 7fad31a5a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad2c15dba0 0x7fad2c161800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.435+0000 7fad31a5a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2c0b93b0 0x7fad2c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.435+0000 7fad31a5a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2c0d1e60 con 0x7fad2c0ced30 2024-04-25T10:00:20.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.435+0000 7fad31a5a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c0cf180 con 0x7fad2c15dba0 2024-04-25T10:00:20.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.435+0000 7fad31a5a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0d21b0 con 0x7fad2c0b93b0 2024-04-25T10:00:20.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.435+0000 7fad2affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad2c15dba0 0x7fad2c161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.435+0000 7fad2affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad2c15dba0 0x7fad2c161800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35458/0 (socket says 172.21.15.139:35458) 2024-04-25T10:00:20.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.435+0000 7fad2affd640 1 -- 172.21.15.139:0/3679318526 learned_addr learned my addr 172.21.15.139:0/3679318526 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:20.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.435+0000 7fad2a7fc640 1 --2- 172.21.15.139:0/3679318526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2c0b93b0 0x7fad2c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.435+0000 7fad2affd640 1 -- 172.21.15.139:0/3679318526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2c0b93b0 msgr2=0x7fad2c0ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:20.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.436+0000 7fad2affd640 1 --2- 172.21.15.139:0/3679318526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2c0b93b0 0x7fad2c0ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.436+0000 7fad2affd640 1 -- 172.21.15.139:0/3679318526 >> v1:172.21.15.150:6789/0 conn(0x7fad2c0ced30 legacy=0x7fad2c06c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.436+0000 7fad2affd640 1 -- 172.21.15.139:0/3679318526 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c162050 con 0x7fad2c15dba0 2024-04-25T10:00:20.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.436+0000 7fad2affd640 1 --2- 172.21.15.139:0/3679318526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2c15dba0 0x7fad2c161800 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009e50 tx=0x7fad1c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef516c0e57d86f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.436+0000 7fad29ffb640 1 -- 172.21.15.139:0/3679318526 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c036070 con 0x7fad2c15dba0 2024-04-25T10:00:20.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.436+0000 7fad29ffb640 1 -- 172.21.15.139:0/3679318526 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c031600 con 0x7fad2c15dba0 2024-04-25T10:00:20.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.436+0000 7fad29ffb640 1 -- 172.21.15.139:0/3679318526 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c0319d0 con 0x7fad2c15dba0 2024-04-25T10:00:20.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.437+0000 7fad31a5a640 1 -- 172.21.15.139:0/3679318526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2c15dba0 msgr2=0x7fad2c161800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.437+0000 7fad31a5a640 1 --2- 172.21.15.139:0/3679318526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2c15dba0 0x7fad2c161800 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009e50 tx=0x7fad1c02a710 comp rx=0 tx=0).stop 2024-04-25T10:00:20.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.437+0000 7fad31a5a640 1 -- 172.21.15.139:0/3679318526 shutdown_connections 2024-04-25T10:00:20.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.437+0000 7fad31a5a640 1 --2- 172.21.15.139:0/3679318526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2c0b93b0 0x7fad2c0ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.437+0000 7fad31a5a640 1 --2- 172.21.15.139:0/3679318526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2c15dba0 0x7fad2c161800 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.437+0000 7fad31a5a640 1 -- 172.21.15.139:0/3679318526 >> 172.21.15.139:0/3679318526 conn(0x7fad2c0d04a0 msgr2=0x7fad2c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.437+0000 7fad31a5a640 1 -- 172.21.15.139:0/3679318526 shutdown_connections 2024-04-25T10:00:20.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.437+0000 7fad31a5a640 1 -- 172.21.15.139:0/3679318526 wait complete. 2024-04-25T10:00:20.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.438+0000 7fad31a5a640 1 Processor -- start 2024-04-25T10:00:20.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.438+0000 7fad31a5a640 1 -- start start 2024-04-25T10:00:20.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad31a5a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2c0b93b0 0x7fad2c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad31a5a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad2c0ced30 0x7fad2c15efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad31a5a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2c15dba0 0x7fad2c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad31a5a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c161d40 con 0x7fad2c15dba0 2024-04-25T10:00:20.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad31a5a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c162610 con 0x7fad2c0ced30 2024-04-25T10:00:20.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad31a5a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0b6bc0 con 0x7fad2c0b93b0 2024-04-25T10:00:20.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad2b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2c15dba0 0x7fad2c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad2b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2c15dba0 0x7fad2c0cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35470/0 (socket says 172.21.15.139:35470) 2024-04-25T10:00:20.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad2b7fe640 1 -- 172.21.15.139:0/588332029 learned_addr learned my addr 172.21.15.139:0/588332029 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad2affd640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2c0b93b0 0x7fad2c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad2b7fe640 1 -- 172.21.15.139:0/588332029 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2c0b93b0 msgr2=0x7fad2c0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad2a7fc640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad2c0ced30 0x7fad2c15efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad2b7fe640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2c0b93b0 0x7fad2c0b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad2b7fe640 1 -- 172.21.15.139:0/588332029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad2c0ced30 msgr2=0x7fad2c15efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad2b7fe640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad2c0ced30 0x7fad2c15efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.439+0000 7fad2b7fe640 1 -- 172.21.15.139:0/588332029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c15f990 con 0x7fad2c15dba0 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.440+0000 7fad2b7fe640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2c15dba0 0x7fad2c0cce00 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fad24001eb0 tx=0x7fad2400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.440+0000 7fad13fff640 1 -- 172.21.15.139:0/588332029 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24017070 con 0x7fad2c15dba0 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.440+0000 7fad13fff640 1 -- 172.21.15.139:0/588332029 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad240052d0 con 0x7fad2c15dba0 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.440+0000 7fad31a5a640 1 -- 172.21.15.139:0/588332029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2c15fc70 con 0x7fad2c15dba0 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.440+0000 7fad13fff640 1 -- 172.21.15.139:0/588332029 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24003810 con 0x7fad2c15dba0 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.440+0000 7fad31a5a640 1 -- 172.21.15.139:0/588332029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c160230 con 0x7fad2c15dba0 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.441+0000 7fad31a5a640 1 -- 172.21.15.139:0/588332029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2c0c7bb0 con 0x7fad2c15dba0 2024-04-25T10:00:20.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.441+0000 7fad13fff640 1 -- 172.21.15.139:0/588332029 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad2401a070 con 0x7fad2c15dba0 2024-04-25T10:00:20.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.442+0000 7fad13fff640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad1806b4d0 0x7fad1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.442+0000 7fad2affd640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad1806b4d0 0x7fad1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.442+0000 7fad13fff640 1 -- 172.21.15.139:0/588332029 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fad240897d0 con 0x7fad2c15dba0 2024-04-25T10:00:20.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.442+0000 7fad2affd640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad1806b4d0 0x7fad1806d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fad1c033000 tx=0x7fad1c009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.446+0000 7fad13fff640 1 -- 172.21.15.139:0/588332029 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad2c0c7bb0 con 0x7fad2c15dba0 2024-04-25T10:00:20.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.613+0000 7fad31a5a640 1 -- 172.21.15.139:0/588332029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad2c160710 con 0x7fad2c15dba0 2024-04-25T10:00:20.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.614+0000 7fad13fff640 1 -- 172.21.15.139:0/588332029 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad24058400 con 0x7fad2c15dba0 2024-04-25T10:00:20.619 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T10:00:20.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.616+0000 7fad31a5a640 1 -- 172.21.15.139:0/588332029 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad1806b4d0 msgr2=0x7fad1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.616+0000 7fad31a5a640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad1806b4d0 0x7fad1806d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fad1c033000 tx=0x7fad1c009000 comp rx=0 tx=0).stop 2024-04-25T10:00:20.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.616+0000 7fad31a5a640 1 -- 172.21.15.139:0/588332029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2c15dba0 msgr2=0x7fad2c0cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.616+0000 7fad31a5a640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2c15dba0 0x7fad2c0cce00 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fad24001eb0 tx=0x7fad2400c890 comp rx=0 tx=0).stop 2024-04-25T10:00:20.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.616+0000 7fad31a5a640 1 -- 172.21.15.139:0/588332029 shutdown_connections 2024-04-25T10:00:20.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.616+0000 7fad31a5a640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad2c15dba0 0x7fad2c0cce00 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.616+0000 7fad31a5a640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fad1806b4d0 0x7fad1806d990 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.616+0000 7fad31a5a640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad2c0ced30 0x7fad2c15efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.616+0000 7fad31a5a640 1 --2- 172.21.15.139:0/588332029 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad2c0b93b0 0x7fad2c0b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.616+0000 7fad31a5a640 1 -- 172.21.15.139:0/588332029 >> 172.21.15.139:0/588332029 conn(0x7fad2c0d04a0 msgr2=0x7fad2c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.616+0000 7fad31a5a640 1 -- 172.21.15.139:0/588332029 shutdown_connections 2024-04-25T10:00:20.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.616+0000 7fad31a5a640 1 -- 172.21.15.139:0/588332029 wait complete. 2024-04-25T10:00:20.632 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:00:20.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.726+0000 7f5a0c77e640 1 Processor -- start 2024-04-25T10:00:20.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.726+0000 7f5a0c77e640 1 -- start start 2024-04-25T10:00:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.727+0000 7f5a0c77e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a040d0a80 0x7f5a041618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.727+0000 7f5a0c77e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a04099ca0 con 0x7f5a0409ad40 2024-04-25T10:00:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.727+0000 7f5a0c77e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a04099f80 con 0x7f5a040cd610 2024-04-25T10:00:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.727+0000 7f5a0c77e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04099a50 con 0x7f5a040d0a80 2024-04-25T10:00:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.727+0000 7f5a0ad1b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5a0409ad40 0x7f5a040cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41586/0 (socket says 172.21.15.139:41586) 2024-04-25T10:00:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.727+0000 7f5a0ad1b640 1 -- 172.21.15.139:0/2693573770 learned_addr learned my addr 172.21.15.139:0/2693573770 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.727+0000 7f5a09d19640 1 --2- 172.21.15.139:0/2693573770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a040d0a80 0x7f5a041618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.727+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278263166 0 0) 0x7f5a04099ca0 con 0x7f5a0409ad40 2024-04-25T10:00:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.727+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59f0003520 con 0x7f5a0409ad40 2024-04-25T10:00:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3849761774 0 0) 0x7f59f0003520 con 0x7f5a0409ad40 2024-04-25T10:00:20.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a040d0a80 msgr2=0x7f5a041618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 --2- 172.21.15.139:0/2693573770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a040d0a80 0x7f5a041618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 >> v1:172.21.15.150:6789/0 conn(0x7f5a040cd610 legacy=0x7f5a040d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0409a260 con 0x7f5a0409ad40 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f59fc0019f0 con 0x7f5a0409ad40 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 >> v1:172.21.15.139:6789/0 conn(0x7f5a0409ad40 legacy=0x7f5a040cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 --2- 172.21.15.139:0/2693573770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f00039e0 0x7f59f0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 --2- 172.21.15.139:0/2693573770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f0004390 0x7f59f0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 --2- 172.21.15.139:0/2693573770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a040d0a80 0x7f59f00050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04099a50 con 0x7f59f00039e0 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f000b9c0 con 0x7f59f0004390 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f000bbc0 con 0x7f5a040d0a80 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a0a51a640 1 --2- 172.21.15.139:0/2693573770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f00039e0 0x7f59f0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a0ad1b640 1 --2- 172.21.15.139:0/2693573770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a040d0a80 0x7f59f00050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a09d19640 1 --2- 172.21.15.139:0/2693573770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f0004390 0x7f59f0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a0a51a640 1 -- 172.21.15.139:0/2693573770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a040d0a80 msgr2=0x7f59f00050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:20.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a0a51a640 1 --2- 172.21.15.139:0/2693573770 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a040d0a80 0x7f59f00050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a0a51a640 1 -- 172.21.15.139:0/2693573770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f0004390 msgr2=0x7f59f0004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a0a51a640 1 --2- 172.21.15.139:0/2693573770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f0004390 0x7f59f0004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.728+0000 7f5a0a51a640 1 -- 172.21.15.139:0/2693573770 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f0003520 con 0x7f59f00039e0 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.729+0000 7f5a0a51a640 1 --2- 172.21.15.139:0/2693573770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f00039e0 0x7f59f0003dc0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f5a00001860 tx=0x7f59f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84831769631b027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.729+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00037070 con 0x7f59f00039e0 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.729+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0002ac30 con 0x7f59f00039e0 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.729+0000 7f5a09518640 1 -- 172.21.15.139:0/2693573770 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00001430 con 0x7f59f00039e0 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.729+0000 7f5a0c77e640 1 -- 172.21.15.139:0/2693573770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f00039e0 msgr2=0x7f59f0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.729+0000 7f5a0c77e640 1 --2- 172.21.15.139:0/2693573770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f00039e0 0x7f59f0003dc0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f5a00001860 tx=0x7f59f0005830 comp rx=0 tx=0).stop 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.729+0000 7f5a0ad1b640 1 -- 172.21.15.139:0/2693573770 reap_dead start 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.729+0000 7f5a0c77e640 1 -- 172.21.15.139:0/2693573770 shutdown_connections 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.729+0000 7f5a0c77e640 1 -- 172.21.15.139:0/2693573770 >> 172.21.15.139:0/2693573770 conn(0x7f5a040b31f0 msgr2=0x7f5a040b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.729+0000 7f5a0c77e640 1 -- 172.21.15.139:0/2693573770 shutdown_connections 2024-04-25T10:00:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.729+0000 7f5a0c77e640 1 -- 172.21.15.139:0/2693573770 wait complete. 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.731+0000 7f5a0c77e640 1 Processor -- start 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.731+0000 7f5a0c77e640 1 -- start start 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.731+0000 7f5a0c77e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f00039e0 0x7f5a041542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.731+0000 7f5a0c77e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a040706f0 0x7f5a04070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.731+0000 7f5a0c77e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a040b6a30 0x7f5a040b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.731+0000 7f5a0c77e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a040cd9f0 con 0x7f59f00039e0 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.731+0000 7f5a0c77e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0409a820 con 0x7f5a040b6a30 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.731+0000 7f5a0c77e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a040cdb70 con 0x7f5a040706f0 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.731+0000 7f5a0a51a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f00039e0 0x7f5a041542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.731+0000 7f5a0a51a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f00039e0 0x7f5a041542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35488/0 (socket says 172.21.15.139:35488) 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.731+0000 7f5a0a51a640 1 -- 172.21.15.139:0/150590915 learned_addr learned my addr 172.21.15.139:0/150590915 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.732+0000 7f5a0a51a640 1 -- 172.21.15.139:0/150590915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a040706f0 msgr2=0x7f5a04070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.732+0000 7f5a0a51a640 1 --2- 172.21.15.139:0/150590915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a040706f0 0x7f5a04070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.732+0000 7f5a0a51a640 1 -- 172.21.15.139:0/150590915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a040b6a30 msgr2=0x7f5a040b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.732+0000 7f5a0a51a640 1 --2- 172.21.15.139:0/150590915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a040b6a30 0x7f5a040b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.732+0000 7f5a0a51a640 1 -- 172.21.15.139:0/150590915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a040b73f0 con 0x7f59f00039e0 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.732+0000 7f5a0a51a640 1 --2- 172.21.15.139:0/150590915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f00039e0 0x7f5a041542a0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f5a000075a0 tx=0x7f5a00001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.732+0000 7f59faffd640 1 -- 172.21.15.139:0/150590915 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00040020 con 0x7f59f00039e0 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.732+0000 7f59faffd640 1 -- 172.21.15.139:0/150590915 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a00005270 con 0x7f59f00039e0 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.732+0000 7f59faffd640 1 -- 172.21.15.139:0/150590915 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00037040 con 0x7f59f00039e0 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.732+0000 7f5a0c77e640 1 -- 172.21.15.139:0/150590915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a040c79b0 con 0x7f59f00039e0 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.732+0000 7f5a0c77e640 1 -- 172.21.15.139:0/150590915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a040c6ed0 con 0x7f59f00039e0 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.733+0000 7f5a0c77e640 1 -- 172.21.15.139:0/150590915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a04074520 con 0x7f59f00039e0 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.733+0000 7f59faffd640 1 -- 172.21.15.139:0/150590915 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a00043020 con 0x7f59f00039e0 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.734+0000 7f59faffd640 1 --2- 172.21.15.139:0/150590915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f59e806b4d0 0x7f59e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.734+0000 7f59faffd640 1 -- 172.21.15.139:0/150590915 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5a000aa1d0 con 0x7f59f00039e0 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.734+0000 7f5a09d19640 1 --2- 172.21.15.139:0/150590915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f59e806b4d0 0x7f59e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.734+0000 7f5a09d19640 1 --2- 172.21.15.139:0/150590915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f59e806b4d0 0x7f59e806d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5a041f6960 tx=0x7f59f4006c30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.738+0000 7f59faffd640 1 -- 172.21.15.139:0/150590915 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a0004a030 con 0x7f59f00039e0 2024-04-25T10:00:20.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.906+0000 7f5a0c77e640 1 -- 172.21.15.139:0/150590915 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a040b3f70 con 0x7f59f00039e0 2024-04-25T10:00:20.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.907+0000 7f59faffd640 1 -- 172.21.15.139:0/150590915 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a00078e90 con 0x7f59f00039e0 2024-04-25T10:00:20.912 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T10:00:20.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.910+0000 7f5a0c77e640 1 -- 172.21.15.139:0/150590915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f59e806b4d0 msgr2=0x7f59e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.910+0000 7f5a0c77e640 1 --2- 172.21.15.139:0/150590915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f59e806b4d0 0x7f59e806d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5a041f6960 tx=0x7f59f4006c30 comp rx=0 tx=0).stop 2024-04-25T10:00:20.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.910+0000 7f5a0c77e640 1 -- 172.21.15.139:0/150590915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f00039e0 msgr2=0x7f5a041542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.910+0000 7f5a0c77e640 1 --2- 172.21.15.139:0/150590915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f00039e0 0x7f5a041542a0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f5a000075a0 tx=0x7f5a00001230 comp rx=0 tx=0).stop 2024-04-25T10:00:20.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.910+0000 7f5a0c77e640 1 -- 172.21.15.139:0/150590915 shutdown_connections 2024-04-25T10:00:20.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.910+0000 7f5a0c77e640 1 --2- 172.21.15.139:0/150590915 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f59e806b4d0 0x7f59e806d990 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.910+0000 7f5a0c77e640 1 --2- 172.21.15.139:0/150590915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a040b6a30 0x7f5a040b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.910+0000 7f5a0c77e640 1 --2- 172.21.15.139:0/150590915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a040706f0 0x7f5a04070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.910+0000 7f5a0c77e640 1 --2- 172.21.15.139:0/150590915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59f00039e0 0x7f5a041542a0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.910+0000 7f5a0c77e640 1 -- 172.21.15.139:0/150590915 >> 172.21.15.139:0/150590915 conn(0x7f5a040b31f0 msgr2=0x7f5a040b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.910+0000 7f5a0c77e640 1 -- 172.21.15.139:0/150590915 shutdown_connections 2024-04-25T10:00:20.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:20.910+0000 7f5a0c77e640 1 -- 172.21.15.139:0/150590915 wait complete. 2024-04-25T10:00:20.925 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:00:21.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.018+0000 7fe74ec07640 1 Processor -- start 2024-04-25T10:00:21.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.018+0000 7fe74ec07640 1 -- start start 2024-04-25T10:00:21.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.019+0000 7fe74ec07640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe74815fca0 0x7fe7480cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.019+0000 7fe74ec07640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe748099bf0 con 0x7fe74815f8c0 2024-04-25T10:00:21.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.019+0000 7fe74ec07640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe748099ed0 con 0x7fe7480d0220 2024-04-25T10:00:21.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.019+0000 7fe74ec07640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7480d06c0 con 0x7fe74815fca0 2024-04-25T10:00:21.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.019+0000 7fe74c9a3640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe74815f8c0 0x7fe74809a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41590/0 (socket says 172.21.15.139:41590) 2024-04-25T10:00:21.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.019+0000 7fe747fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe74815fca0 0x7fe7480cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.019+0000 7fe74c9a3640 1 -- 172.21.15.139:0/4004767444 learned_addr learned my addr 172.21.15.139:0/4004767444 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:21.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.019+0000 7fe7477fe640 1 -- 172.21.15.139:0/4004767444 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748624015 0 0) 0x7fe748099bf0 con 0x7fe74815f8c0 2024-04-25T10:00:21.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.019+0000 7fe7477fe640 1 -- 172.21.15.139:0/4004767444 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe738003520 con 0x7fe74815f8c0 2024-04-25T10:00:21.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.019+0000 7fe747fff640 1 -- 172.21.15.139:0/4004767444 >> v1:172.21.15.150:6789/0 conn(0x7fe7480d0220 legacy=0x7fe74815b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:21.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.019+0000 7fe747fff640 1 -- 172.21.15.139:0/4004767444 >> v1:172.21.15.139:6789/0 conn(0x7fe74815f8c0 legacy=0x7fe74809a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.019+0000 7fe747fff640 1 -- 172.21.15.139:0/4004767444 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74809a1b0 con 0x7fe74815fca0 2024-04-25T10:00:21.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.020+0000 7fe747fff640 1 --2- 172.21.15.139:0/4004767444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe74815fca0 0x7fe7480cfce0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fe7340097d0 tx=0x7fe73402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdd1de75438cbb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.020+0000 7fe7477fe640 1 -- 172.21.15.139:0/4004767444 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734037070 con 0x7fe74815fca0 2024-04-25T10:00:21.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.020+0000 7fe7477fe640 1 -- 172.21.15.139:0/4004767444 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe734032440 con 0x7fe74815fca0 2024-04-25T10:00:21.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.020+0000 7fe7477fe640 1 -- 172.21.15.139:0/4004767444 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7340327f0 con 0x7fe74815fca0 2024-04-25T10:00:21.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.021+0000 7fe74ec07640 1 -- 172.21.15.139:0/4004767444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe74815fca0 msgr2=0x7fe7480cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.021+0000 7fe74ec07640 1 --2- 172.21.15.139:0/4004767444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe74815fca0 0x7fe7480cfce0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fe7340097d0 tx=0x7fe73402a210 comp rx=0 tx=0).stop 2024-04-25T10:00:21.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.021+0000 7fe74ec07640 1 -- 172.21.15.139:0/4004767444 shutdown_connections 2024-04-25T10:00:21.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.021+0000 7fe74ec07640 1 --2- 172.21.15.139:0/4004767444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe74815fca0 0x7fe7480cfce0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.021+0000 7fe74ec07640 1 -- 172.21.15.139:0/4004767444 >> 172.21.15.139:0/4004767444 conn(0x7fe7480b31f0 msgr2=0x7fe7480b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:21.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.021+0000 7fe74ec07640 1 -- 172.21.15.139:0/4004767444 shutdown_connections 2024-04-25T10:00:21.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.021+0000 7fe74ec07640 1 -- 172.21.15.139:0/4004767444 wait complete. 2024-04-25T10:00:21.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.022+0000 7fe74ec07640 1 Processor -- start 2024-04-25T10:00:21.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.022+0000 7fe74ec07640 1 -- start start 2024-04-25T10:00:21.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74ec07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7480d0220 0x7fe7480cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74ec07640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe74815f8c0 0x7fe748150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74ec07640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe74815fca0 0x7fe7481f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74ec07640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe748160450 con 0x7fe7480d0220 2024-04-25T10:00:21.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74ec07640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7481602d0 con 0x7fe74815fca0 2024-04-25T10:00:21.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74ec07640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74815bc70 con 0x7fe74815f8c0 2024-04-25T10:00:21.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74c9a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7480d0220 0x7fe7480cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74c9a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7480d0220 0x7fe7480cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35510/0 (socket says 172.21.15.139:35510) 2024-04-25T10:00:21.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74c9a3640 1 -- 172.21.15.139:0/3806491696 learned_addr learned my addr 172.21.15.139:0/3806491696 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:21.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe747fff640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe74815f8c0 0x7fe748150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74d1a4640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe74815fca0 0x7fe7481f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74c9a3640 1 -- 172.21.15.139:0/3806491696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe74815f8c0 msgr2=0x7fe748150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74c9a3640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe74815f8c0 0x7fe748150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74c9a3640 1 -- 172.21.15.139:0/3806491696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe74815fca0 msgr2=0x7fe7481f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74c9a3640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe74815fca0 0x7fe7481f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74c9a3640 1 -- 172.21.15.139:0/3806491696 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe748152630 con 0x7fe7480d0220 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74d1a4640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe74815fca0 0x7fe7481f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe747fff640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe74815f8c0 0x7fe748150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74c9a3640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7480d0220 0x7fe7480cb130 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00c430 tx=0x7fe73c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe7457fa640 1 -- 172.21.15.139:0/3806491696 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c017070 con 0x7fe7480d0220 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe7457fa640 1 -- 172.21.15.139:0/3806491696 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73c005ce0 con 0x7fe7480d0220 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74ec07640 1 -- 172.21.15.139:0/3806491696 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7480b4d40 con 0x7fe7480d0220 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe7457fa640 1 -- 172.21.15.139:0/3806491696 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c0072f0 con 0x7fe7480d0220 2024-04-25T10:00:21.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.023+0000 7fe74ec07640 1 -- 172.21.15.139:0/3806491696 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7480b52c0 con 0x7fe7480d0220 2024-04-25T10:00:21.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.025+0000 7fe74ec07640 1 -- 172.21.15.139:0/3806491696 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7480d0010 con 0x7fe7480d0220 2024-04-25T10:00:21.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.025+0000 7fe7457fa640 1 -- 172.21.15.139:0/3806491696 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe73c01a070 con 0x7fe7480d0220 2024-04-25T10:00:21.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.025+0000 7fe7457fa640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe72c06b4d0 0x7fe72c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.026+0000 7fe747fff640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe72c06b4d0 0x7fe72c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.026+0000 7fe7457fa640 1 -- 172.21.15.139:0/3806491696 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe73c089900 con 0x7fe7480d0220 2024-04-25T10:00:21.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.026+0000 7fe747fff640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe72c06b4d0 0x7fe72c06d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe734034000 tx=0x7fe73402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.031+0000 7fe7457fa640 1 -- 172.21.15.139:0/3806491696 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7480d0010 con 0x7fe7480d0220 2024-04-25T10:00:21.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.212+0000 7fe74ec07640 1 -- 172.21.15.139:0/3806491696 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe748073dd0 con 0x7fe7480d0220 2024-04-25T10:00:21.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.212+0000 7fe7457fa640 1 -- 172.21.15.139:0/3806491696 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v92) v1 ==== 80+0+124 (secure 0 0 0) 0x7fe748073dd0 con 0x7fe7480d0220 2024-04-25T10:00:21.218 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.150:6835/2266713846 2024-04-26T09:59:10.897062+0000 2024-04-25T10:00:21.218 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.150:6833/2266713846 2024-04-26T09:59:10.897062+0000 2024-04-25T10:00:21.218 INFO:teuthology.orchestra.run.smithi139.stderr:listed 2 entries 2024-04-25T10:00:21.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.214+0000 7fe74ec07640 1 -- 172.21.15.139:0/3806491696 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe72c06b4d0 msgr2=0x7fe72c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.214+0000 7fe74ec07640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe72c06b4d0 0x7fe72c06d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe734034000 tx=0x7fe73402e000 comp rx=0 tx=0).stop 2024-04-25T10:00:21.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.214+0000 7fe74ec07640 1 -- 172.21.15.139:0/3806491696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7480d0220 msgr2=0x7fe7480cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.214+0000 7fe74ec07640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7480d0220 0x7fe7480cb130 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00c430 tx=0x7fe73c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.214+0000 7fe74ec07640 1 -- 172.21.15.139:0/3806491696 shutdown_connections 2024-04-25T10:00:21.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.214+0000 7fe74ec07640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe72c06b4d0 0x7fe72c06d990 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.214+0000 7fe74ec07640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe74815fca0 0x7fe7481f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.214+0000 7fe74ec07640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe74815f8c0 0x7fe748150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.214+0000 7fe74ec07640 1 --2- 172.21.15.139:0/3806491696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7480d0220 0x7fe7480cb130 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.214+0000 7fe74ec07640 1 -- 172.21.15.139:0/3806491696 >> 172.21.15.139:0/3806491696 conn(0x7fe7480b31f0 msgr2=0x7fe7480c5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:21.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.215+0000 7fe74ec07640 1 -- 172.21.15.139:0/3806491696 shutdown_connections 2024-04-25T10:00:21.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.215+0000 7fe74ec07640 1 -- 172.21.15.139:0/3806491696 wait complete. 2024-04-25T10:00:21.230 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:00:21.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:21.231 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:21.258 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:00:21.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:21.258 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:00:21.313 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:21.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.444+0000 7f9d72b69640 1 Processor -- start 2024-04-25T10:00:21.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.444+0000 7f9d72b69640 1 -- start start 2024-04-25T10:00:21.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.444+0000 7f9d72b69640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9d6c15a860 0x7f9d6c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.444+0000 7f9d72b69640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9d6c0cd1f0 0x7f9d6c06e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.444+0000 7f9d72b69640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6c0af6d0 con 0x7f9d6c15ac40 2024-04-25T10:00:21.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.444+0000 7f9d72b69640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6c0b02e0 con 0x7f9d6c15a860 2024-04-25T10:00:21.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.444+0000 7f9d72b69640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6c0af9b0 con 0x7f9d6c0cd1f0 2024-04-25T10:00:21.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.445+0000 7f9d70905640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9d6c15a860 0x7f9d6c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.445+0000 7f9d71106640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9d6c15ac40 0x7f9d6c15e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41594/0 (socket says 172.21.15.139:41594) 2024-04-25T10:00:21.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.445+0000 7f9d71106640 1 -- 172.21.15.139:0/2001355466 learned_addr learned my addr 172.21.15.139:0/2001355466 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:21.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.445+0000 7f9d70905640 1 -- 172.21.15.139:0/2001355466 >> v2:172.21.15.150:3300/0 conn(0x7f9d6c0cd1f0 msgr2=0x7f9d6c06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:21.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.445+0000 7f9d70905640 1 --2- 172.21.15.139:0/2001355466 >> v2:172.21.15.150:3300/0 conn(0x7f9d6c0cd1f0 0x7f9d6c06e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.445+0000 7f9d70905640 1 -- 172.21.15.139:0/2001355466 >> v1:172.21.15.139:6789/0 conn(0x7f9d6c15ac40 legacy=0x7f9d6c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.445+0000 7f9d70905640 1 -- 172.21.15.139:0/2001355466 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0afc90 con 0x7f9d6c15a860 2024-04-25T10:00:21.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.445+0000 7f9d6b7fe640 1 -- 172.21.15.139:0/2001355466 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2061397427 0 0) 0x7f9d6c0af6d0 con 0x7f9d6c15ac40 2024-04-25T10:00:21.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.446+0000 7f9d70905640 1 --2- 172.21.15.139:0/2001355466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d6c15a860 0x7f9d6c0cccb0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0b2610 tx=0x7f9d6402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5b86dda87c191b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.446+0000 7f9d6b7fe640 1 -- 172.21.15.139:0/2001355466 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64007740 con 0x7f9d6c15a860 2024-04-25T10:00:21.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.446+0000 7f9d6b7fe640 1 -- 172.21.15.139:0/2001355466 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d640078e0 con 0x7f9d6c15a860 2024-04-25T10:00:21.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.446+0000 7f9d6b7fe640 1 -- 172.21.15.139:0/2001355466 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64007cb0 con 0x7f9d6c15a860 2024-04-25T10:00:21.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.446+0000 7f9d72b69640 1 -- 172.21.15.139:0/2001355466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d6c15a860 msgr2=0x7f9d6c0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.446+0000 7f9d72b69640 1 --2- 172.21.15.139:0/2001355466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d6c15a860 0x7f9d6c0cccb0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0b2610 tx=0x7f9d6402a680 comp rx=0 tx=0).stop 2024-04-25T10:00:21.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.446+0000 7f9d72b69640 1 -- 172.21.15.139:0/2001355466 shutdown_connections 2024-04-25T10:00:21.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.446+0000 7f9d72b69640 1 --2- 172.21.15.139:0/2001355466 >> v2:172.21.15.150:3300/0 conn(0x7f9d6c0cd1f0 0x7f9d6c06e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.446+0000 7f9d72b69640 1 --2- 172.21.15.139:0/2001355466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d6c15a860 0x7f9d6c0cccb0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.446+0000 7f9d72b69640 1 -- 172.21.15.139:0/2001355466 >> 172.21.15.139:0/2001355466 conn(0x7f9d6c066f20 msgr2=0x7f9d6c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:21.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.446+0000 7f9d72b69640 1 -- 172.21.15.139:0/2001355466 shutdown_connections 2024-04-25T10:00:21.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.446+0000 7f9d72b69640 1 -- 172.21.15.139:0/2001355466 wait complete. 2024-04-25T10:00:21.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.447+0000 7f9d72b69640 1 Processor -- start 2024-04-25T10:00:21.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.447+0000 7f9d72b69640 1 -- start start 2024-04-25T10:00:21.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.448+0000 7f9d72b69640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d6c0cd1f0 0x7f9d6c06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.448+0000 7f9d72b69640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d6c15a860 0x7f9d6c0ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.448+0000 7f9d72b69640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6c15ac40 0x7f9d6c1f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.448+0000 7f9d72b69640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0af420 con 0x7f9d6c15a860 2024-04-25T10:00:21.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.448+0000 7f9d72b69640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0af2a0 con 0x7f9d6c0cd1f0 2024-04-25T10:00:21.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.448+0000 7f9d72b69640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c06e850 con 0x7f9d6c15ac40 2024-04-25T10:00:21.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.448+0000 7f9d6bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d6c15a860 0x7f9d6c0ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.448+0000 7f9d6bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d6c15a860 0x7f9d6c0ad070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35536/0 (socket says 172.21.15.139:35536) 2024-04-25T10:00:21.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.448+0000 7f9d6bfff640 1 -- 172.21.15.139:0/3107204625 learned_addr learned my addr 172.21.15.139:0/3107204625 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:21.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.449+0000 7f9d70905640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d6c0cd1f0 0x7f9d6c06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.449+0000 7f9d6bfff640 1 -- 172.21.15.139:0/3107204625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6c15ac40 msgr2=0x7f9d6c1f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.449+0000 7f9d6bfff640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6c15ac40 0x7f9d6c1f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.449+0000 7f9d6bfff640 1 -- 172.21.15.139:0/3107204625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d6c0cd1f0 msgr2=0x7f9d6c06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.449+0000 7f9d6bfff640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d6c0cd1f0 0x7f9d6c06d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.449+0000 7f9d6bfff640 1 -- 172.21.15.139:0/3107204625 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c157fb0 con 0x7f9d6c15a860 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.449+0000 7f9d6bfff640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d6c15a860 0x7f9d6c0ad070 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400b450 tx=0x7f9d5400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.449+0000 7f9d697fa640 1 -- 172.21.15.139:0/3107204625 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54015020 con 0x7f9d6c15a860 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.449+0000 7f9d697fa640 1 -- 172.21.15.139:0/3107204625 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d54011890 con 0x7f9d6c15a860 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.449+0000 7f9d697fa640 1 -- 172.21.15.139:0/3107204625 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54011c80 con 0x7f9d6c15a860 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.449+0000 7f9d72b69640 1 -- 172.21.15.139:0/3107204625 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c158290 con 0x7f9d6c15a860 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.449+0000 7f9d72b69640 1 -- 172.21.15.139:0/3107204625 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c158850 con 0x7f9d6c15a860 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.450+0000 7f9d72b69640 1 -- 172.21.15.139:0/3107204625 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c074520 con 0x7f9d6c15a860 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.450+0000 7f9d697fa640 1 -- 172.21.15.139:0/3107204625 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d54018020 con 0x7f9d6c15a860 2024-04-25T10:00:21.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.451+0000 7f9d697fa640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9d4c06b500 0x7f9d4c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.451+0000 7f9d697fa640 1 -- 172.21.15.139:0/3107204625 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f9d5408a590 con 0x7f9d6c15a860 2024-04-25T10:00:21.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.451+0000 7f9d70905640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9d4c06b500 0x7f9d4c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.451+0000 7f9d70905640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9d4c06b500 0x7f9d4c06d9c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9d64034000 tx=0x7f9d6402f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.456+0000 7f9d697fa640 1 -- 172.21.15.139:0/3107204625 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d5401f020 con 0x7f9d6c15a860 2024-04-25T10:00:21.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.664+0000 7f9d72b69640 1 -- 172.21.15.139:0/3107204625 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d6c0b34e0 con 0x7f9d6c15a860 2024-04-25T10:00:21.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.665+0000 7f9d697fa640 1 -- 172.21.15.139:0/3107204625 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4954 (secure 0 0 0) 0x7f9d5400d040 con 0x7f9d6c15a860 2024-04-25T10:00:21.669 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:00:21.669 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T09:59:21.913360+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":10,"addr":"172.21.15.139:6836/1239243954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1239243954},{"type":"v1","addr":"172.21.15.139:6836","nonce":1239243954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:21.670 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 57 2024-04-25T10:00:21.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.667+0000 7f9d72b69640 1 -- 172.21.15.139:0/3107204625 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9d4c06b500 msgr2=0x7f9d4c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.667+0000 7f9d72b69640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9d4c06b500 0x7f9d4c06d9c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9d64034000 tx=0x7f9d6402f000 comp rx=0 tx=0).stop 2024-04-25T10:00:21.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.667+0000 7f9d72b69640 1 -- 172.21.15.139:0/3107204625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d6c15a860 msgr2=0x7f9d6c0ad070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.667+0000 7f9d72b69640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d6c15a860 0x7f9d6c0ad070 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400b450 tx=0x7f9d5400b910 comp rx=0 tx=0).stop 2024-04-25T10:00:21.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.667+0000 7f9d72b69640 1 -- 172.21.15.139:0/3107204625 shutdown_connections 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.667+0000 7f9d72b69640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d6c15a860 0x7f9d6c0ad070 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.667+0000 7f9d72b69640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9d4c06b500 0x7f9d4c06d9c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.667+0000 7f9d72b69640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d6c15ac40 0x7f9d6c1f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.667+0000 7f9d72b69640 1 --2- 172.21.15.139:0/3107204625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d6c0cd1f0 0x7f9d6c06d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.667+0000 7f9d72b69640 1 -- 172.21.15.139:0/3107204625 >> 172.21.15.139:0/3107204625 conn(0x7f9d6c066f20 msgr2=0x7f9d6c0b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.667+0000 7f9d72b69640 1 -- 172.21.15.139:0/3107204625 shutdown_connections 2024-04-25T10:00:21.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.667+0000 7f9d72b69640 1 -- 172.21.15.139:0/3107204625 wait complete. 2024-04-25T10:00:21.682 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:00:21.804 INFO:teuthology.orchestra.run.smithi139.stdout:{"message":"","return_code":0} 2024-04-25T10:00:21.804 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:00:21.805 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:00:21.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.899+0000 7fdb18b34640 1 Processor -- start 2024-04-25T10:00:21.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.899+0000 7fdb18b34640 1 -- start start 2024-04-25T10:00:21.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.899+0000 7fdb18b34640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb141617e0 0x7fdb140ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.899+0000 7fdb18b34640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb14161dc0 con 0x7fdb140cec80 2024-04-25T10:00:21.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.899+0000 7fdb18b34640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb141620a0 con 0x7fdb1415dbf0 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.899+0000 7fdb18b34640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140d1ee0 con 0x7fdb141617e0 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb12d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdb140cec80 0x7fdb140d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41604/0 (socket says 172.21.15.139:41604) 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb12d9d640 1 -- 172.21.15.139:0/1174601788 learned_addr learned my addr 172.21.15.139:0/1174601788 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb0bfff640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb141617e0 0x7fdb140ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582994197 0 0) 0x7fdb14161dc0 con 0x7fdb140cec80 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb00003520 con 0x7fdb140cec80 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4006411833 0 0) 0x7fdb00003520 con 0x7fdb140cec80 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb141617e0 msgr2=0x7fdb140ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb141617e0 0x7fdb140ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 >> v1:172.21.15.150:6789/0 conn(0x7fdb1415dbf0 legacy=0x7fdb140b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb14162380 con 0x7fdb140cec80 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fdb0c0019f0 con 0x7fdb140cec80 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 >> v1:172.21.15.139:6789/0 conn(0x7fdb140cec80 legacy=0x7fdb140d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb141617e0 0x7fdb000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb00003f60 0x7fdb00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb00004d40 0x7fdb000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140d1ee0 con 0x7fdb00004d40 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0000bb50 con 0x7fdb00003f60 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0000bd50 con 0x7fdb141617e0 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb12d9d640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb00004d40 0x7fdb000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb0bfff640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb00003f60 0x7fdb00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb1259c640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb141617e0 0x7fdb000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb12d9d640 1 -- 172.21.15.139:0/1174601788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb141617e0 msgr2=0x7fdb000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb12d9d640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb141617e0 0x7fdb000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.900+0000 7fdb12d9d640 1 -- 172.21.15.139:0/1174601788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb00003f60 msgr2=0x7fdb00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.901+0000 7fdb12d9d640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb00003f60 0x7fdb00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.901+0000 7fdb12d9d640 1 -- 172.21.15.139:0/1174601788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb00003520 con 0x7fdb00004d40 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.901+0000 7fdb0bfff640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb00003f60 0x7fdb00004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:21.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.901+0000 7fdb12d9d640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb00004d40 0x7fdb000051c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00a860 tx=0x7fdb00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9114ef89a6a2fa9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.901+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c00a480 con 0x7fdb00004d40 2024-04-25T10:00:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.901+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c02f070 con 0x7fdb00004d40 2024-04-25T10:00:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.901+0000 7fdb11d9b640 1 -- 172.21.15.139:0/1174601788 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c02bd00 con 0x7fdb00004d40 2024-04-25T10:00:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.902+0000 7fdb18b34640 1 -- 172.21.15.139:0/1174601788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb00004d40 msgr2=0x7fdb000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.902+0000 7fdb18b34640 1 --2- 172.21.15.139:0/1174601788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb00004d40 0x7fdb000051c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00a860 tx=0x7fdb00005830 comp rx=0 tx=0).stop 2024-04-25T10:00:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.902+0000 7fdb12d9d640 1 -- 172.21.15.139:0/1174601788 reap_dead start 2024-04-25T10:00:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.902+0000 7fdb18b34640 1 -- 172.21.15.139:0/1174601788 shutdown_connections 2024-04-25T10:00:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.902+0000 7fdb18b34640 1 -- 172.21.15.139:0/1174601788 >> 172.21.15.139:0/1174601788 conn(0x7fdb140b5480 msgr2=0x7fdb140b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.902+0000 7fdb18b34640 1 -- 172.21.15.139:0/1174601788 shutdown_connections 2024-04-25T10:00:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.902+0000 7fdb18b34640 1 -- 172.21.15.139:0/1174601788 wait complete. 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.903+0000 7fdb18b34640 1 Processor -- start 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.903+0000 7fdb18b34640 1 -- start start 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb18b34640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb1414e400 0x7fdb140cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb18b34640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb140c7480 0x7fdb140cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb18b34640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb1414ea20 0x7fdb141f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb18b34640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1415dfd0 con 0x7fdb1414e400 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb18b34640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140ce740 con 0x7fdb140c7480 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb18b34640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140c8570 con 0x7fdb1414ea20 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb1259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb1414e400 0x7fdb140cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb1259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb1414e400 0x7fdb140cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35548/0 (socket says 172.21.15.139:35548) 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb1259c640 1 -- 172.21.15.139:0/1491589546 learned_addr learned my addr 172.21.15.139:0/1491589546 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb1259c640 1 -- 172.21.15.139:0/1491589546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb1414ea20 msgr2=0x7fdb141f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb0bfff640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb140c7480 0x7fdb140cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb1259c640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb1414ea20 0x7fdb141f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb1259c640 1 -- 172.21.15.139:0/1491589546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb140c7480 msgr2=0x7fdb140cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb1259c640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb140c7480 0x7fdb140cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb1259c640 1 -- 172.21.15.139:0/1491589546 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1414e7e0 con 0x7fdb1414e400 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.904+0000 7fdb0bfff640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb140c7480 0x7fdb140cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.905+0000 7fdb1259c640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb1414e400 0x7fdb140cc540 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400e4d0 tx=0x7fdb0400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.905+0000 7fdb0affd640 1 -- 172.21.15.139:0/1491589546 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb040052e0 con 0x7fdb1414e400 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.905+0000 7fdb0affd640 1 -- 172.21.15.139:0/1491589546 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb04018070 con 0x7fdb1414e400 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.905+0000 7fdb18b34640 1 -- 172.21.15.139:0/1491589546 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb140c79f0 con 0x7fdb1414e400 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.905+0000 7fdb0affd640 1 -- 172.21.15.139:0/1491589546 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04003810 con 0x7fdb1414e400 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.905+0000 7fdb18b34640 1 -- 172.21.15.139:0/1491589546 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb140c7fb0 con 0x7fdb1414e400 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.906+0000 7fdb18b34640 1 -- 172.21.15.139:0/1491589546 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb14074520 con 0x7fdb1414e400 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.906+0000 7fdb0affd640 1 -- 172.21.15.139:0/1491589546 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb040039b0 con 0x7fdb1414e400 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.907+0000 7fdb0affd640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdaf406b500 0x7fdaf406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.907+0000 7fdb0bfff640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdaf406b500 0x7fdaf406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.907+0000 7fdb0affd640 1 -- 172.21.15.139:0/1491589546 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fdb04089f20 con 0x7fdb1414e400 2024-04-25T10:00:21.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.907+0000 7fdb0bfff640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdaf406b500 0x7fdaf406d9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdb141f6ff0 tx=0x7fdafc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:21.911+0000 7fdb0affd640 1 -- 172.21.15.139:0/1491589546 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb04058b50 con 0x7fdb1414e400 2024-04-25T10:00:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.117+0000 7fdb18b34640 1 -- 172.21.15.139:0/1491589546 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdb140c88f0 con 0x7fdb1414e400 2024-04-25T10:00:22.122 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T10:00:22.118+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:00:22.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.120+0000 7fdb0affd640 1 -- 172.21.15.139:0/1491589546 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdb04058450 con 0x7fdb1414e400 2024-04-25T10:00:22.124 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:00:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.122+0000 7fdb18b34640 1 -- 172.21.15.139:0/1491589546 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdaf406b500 msgr2=0x7fdaf406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.122+0000 7fdb18b34640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdaf406b500 0x7fdaf406d9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdb141f6ff0 tx=0x7fdafc008040 comp rx=0 tx=0).stop 2024-04-25T10:00:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.122+0000 7fdb18b34640 1 -- 172.21.15.139:0/1491589546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb1414e400 msgr2=0x7fdb140cc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.122+0000 7fdb18b34640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb1414e400 0x7fdb140cc540 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400e4d0 tx=0x7fdb0400e990 comp rx=0 tx=0).stop 2024-04-25T10:00:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.123+0000 7fdb18b34640 1 -- 172.21.15.139:0/1491589546 shutdown_connections 2024-04-25T10:00:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.123+0000 7fdb18b34640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdaf406b500 0x7fdaf406d9c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.123+0000 7fdb18b34640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb1414ea20 0x7fdb141f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.123+0000 7fdb18b34640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb140c7480 0x7fdb140cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.123+0000 7fdb18b34640 1 --2- 172.21.15.139:0/1491589546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb1414e400 0x7fdb140cc540 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.123+0000 7fdb18b34640 1 -- 172.21.15.139:0/1491589546 >> 172.21.15.139:0/1491589546 conn(0x7fdb140b5480 msgr2=0x7fdb14158bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.123+0000 7fdb18b34640 1 -- 172.21.15.139:0/1491589546 shutdown_connections 2024-04-25T10:00:22.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.123+0000 7fdb18b34640 1 -- 172.21.15.139:0/1491589546 wait complete. 2024-04-25T10:00:22.129 INFO:tasks.ceph.mds.a.smithi139.stderr: -4191> 2024-04-25T09:59:17.419+0000 7efdd1538640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T10:00:22.130 INFO:tasks.ceph.mds.a.smithi139.stderr: -4029> 2024-04-25T09:59:17.597+0000 7efdd1538640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-25T09:59:08.119440+0000 1=1+0) n(v0 rc2024-04-25T09:59:08.238437+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x557cea233b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T09:59:08.119440+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T09:59:08.238437+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T10:00:22.130 INFO:tasks.ceph.mds.a.smithi139.stderr: -3917> 2024-04-25T09:59:18.292+0000 7efdd5540640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:00:22.131 INFO:tasks.ceph.mds.a.smithi139.stderr: -2218> 2024-04-25T09:59:48.853+0000 7efdd1538640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-04-25T09:59:08.119440+0000 1=1+0)->f(v1 m2024-04-25T09:59:08.119440+0000) n(v0 rc2024-04-25T09:59:08.238437+0000 2=1+1)->n(v0 rc2024-04-25T09:59:08.238437+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x557cea233b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T09:59:08.119440+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T09:59:08.238437+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T10:00:22.138 DEBUG:teuthology.orchestra.run.smithi139:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T10:00:22.157 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T10:00:22.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.173+0000 7f7709a38740 1 -- start start 2024-04-25T10:00:22.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.173+0000 7f7709a38740 1 Processor -- start 2024-04-25T10:00:22.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7709a38740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602a54400 0x555601cf5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7709a38740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555601c85c00 0x555601cf4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7709a38740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602a54800 0x555601cf5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7709a38740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602a5a000 con 0x555602a54800 2024-04-25T10:00:22.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7709a38740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602a5a180 con 0x555601c85c00 2024-04-25T10:00:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7709a38740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555602a5a300 con 0x555602a54400 2024-04-25T10:00:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7707b40640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602a54800 0x555601cf5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7707b40640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602a54800 0x555601cf5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35586/0 (socket says 172.21.15.139:35586) 2024-04-25T10:00:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7707b40640 1 -- 172.21.15.139:0/3768358351 learned_addr learned my addr 172.21.15.139:0/3768358351 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7706b3e640 1 --2- 172.21.15.139:0/3768358351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555601c85c00 0x555601cf4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f770733f640 1 --2- 172.21.15.139:0/3768358351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602a54400 0x555601cf5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7707b40640 1 -- 172.21.15.139:0/3768358351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602a54400 msgr2=0x555601cf5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7707b40640 1 --2- 172.21.15.139:0/3768358351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602a54400 0x555601cf5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7707b40640 1 -- 172.21.15.139:0/3768358351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555601c85c00 msgr2=0x555601cf4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7707b40640 1 --2- 172.21.15.139:0/3768358351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555601c85c00 0x555601cf4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7707b40640 1 -- 172.21.15.139:0/3768358351 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555601cb1680 con 0x555602a54800 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7706b3e640 1 --2- 172.21.15.139:0/3768358351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555601c85c00 0x555601cf4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.174+0000 7f7707b40640 1 --2- 172.21.15.139:0/3768358351 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602a54800 0x555601cf5b80 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x555602aadaa0 tx=0x555602a17320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e47dd8607223d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.175+0000 7f7704b3a640 1 -- 172.21.15.139:0/3768358351 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555601caad00 con 0x555602a54800 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.175+0000 7f7704b3a640 1 -- 172.21.15.139:0/3768358351 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555601ccec40 con 0x555602a54800 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.175+0000 7f7704b3a640 1 -- 172.21.15.139:0/3768358351 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555601caa680 con 0x555602a54800 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.175+0000 7f7709a38740 1 -- 172.21.15.139:0/3768358351 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555601cb03c0 con 0x555602a54800 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.175+0000 7f7709a38740 1 -- 172.21.15.139:0/3768358351 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555601cb1a40 con 0x555602a54800 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.175+0000 7f7709a38740 4 waiting for MDS map... 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.175+0000 7f7704b3a640 1 -- 172.21.15.139:0/3768358351 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x555602b56400 con 0x555602a54800 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.175+0000 7f7704b3a640 1 -- 172.21.15.139:0/3768358351 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 58) v1 ==== 2723+0+0 (secure 0 0 0) 0x555601cdd180 con 0x555602a54800 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.175+0000 7f7709a38740 4 Got MDS map 58 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.175+0000 7f7709a38740 4 datascan.main: Using metadata pool output 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.175+0000 7f7709a38740 4 datascan.main: connecting to RADOS... 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.175+0000 7f7709a38740 1 Processor -- start 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7709a38740 1 -- start start 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7709a38740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555602b58800 0x555601cf5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7709a38740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602b58c00 0x555601cf4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7709a38740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 0x555601cf6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7709a38740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602a5a480 con 0x555602b59000 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7709a38740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602a5a600 con 0x555602b58800 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7709a38740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555602a5a780 con 0x555602b58c00 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7706b3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 0x555601cf6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7706b3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 0x555601cf6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35590/0 (socket says 172.21.15.139:35590) 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7706b3e640 1 -- 172.21.15.139:0/1426920982 learned_addr learned my addr 172.21.15.139:0/1426920982 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f770733f640 1 --2- 172.21.15.139:0/1426920982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602b58c00 0x555601cf4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1426920982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555602b58800 0x555601cf5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7706b3e640 1 -- 172.21.15.139:0/1426920982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602b58c00 msgr2=0x555601cf4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1426920982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602b58c00 0x555601cf4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7706b3e640 1 -- 172.21.15.139:0/1426920982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555602b58800 msgr2=0x555601cf5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1426920982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555602b58800 0x555601cf5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.176+0000 7f7706b3e640 1 -- 172.21.15.139:0/1426920982 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555601cb14a0 con 0x555602b59000 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1426920982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 0x555601cf6100 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x555602aad590 tx=0x555602a08a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283608eb462f8179 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7702b36640 1 -- 172.21.15.139:0/1426920982 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555601caab60 con 0x555602b59000 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7702b36640 1 -- 172.21.15.139:0/1426920982 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555601ccf340 con 0x555602b59000 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7702b36640 1 -- 172.21.15.139:0/1426920982 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555601caad00 con 0x555602b59000 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7709a38740 1 -- 172.21.15.139:0/1426920982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 msgr2=0x555601cf6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7709a38740 1 --2- 172.21.15.139:0/1426920982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 0x555601cf6100 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x555602aad590 tx=0x555602a08a80 comp rx=0 tx=0).stop 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7709a38740 1 -- 172.21.15.139:0/1426920982 shutdown_connections 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7709a38740 1 --2- 172.21.15.139:0/1426920982 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 0x555601cf6100 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7709a38740 1 --2- 172.21.15.139:0/1426920982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602b58c00 0x555601cf4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7709a38740 1 --2- 172.21.15.139:0/1426920982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555602b58800 0x555601cf5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7709a38740 1 -- 172.21.15.139:0/1426920982 >> 172.21.15.139:0/1426920982 conn(0x555602a54000 msgr2=0x555601d7d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7709a38740 1 -- 172.21.15.139:0/1426920982 shutdown_connections 2024-04-25T10:00:22.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.177+0000 7f7709a38740 1 -- 172.21.15.139:0/1426920982 wait complete. 2024-04-25T10:00:22.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f7709a38740 1 Processor -- start 2024-04-25T10:00:22.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f7709a38740 1 -- start start 2024-04-25T10:00:22.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f7709a38740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602b58800 0x555601cf7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f7709a38740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555602b58c00 0x555601cf6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f7709a38740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 0x555601cf6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f7709a38740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602a5bc80 con 0x555602b59000 2024-04-25T10:00:22.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f7709a38740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602a5bb00 con 0x555602b58c00 2024-04-25T10:00:22.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f7709a38740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555602a5b980 con 0x555602b58800 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f7706b3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 0x555601cf6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f7706b3e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 0x555601cf6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35606/0 (socket says 172.21.15.139:35606) 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 learned_addr learned my addr 172.21.15.139:0/1707056211 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602b58800 0x555601cf7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.178+0000 7f770733f640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555602b58c00 0x555601cf6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.179+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602b58800 msgr2=0x555601cf7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.179+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602b58800 0x555601cf7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.179+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555602b58c00 msgr2=0x555601cf6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.179+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555602b58c00 0x555601cf6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.179+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555601cb1860 con 0x555602b59000 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.179+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 0x555601cf6680 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x555602b71770 tx=0x555602a08900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.179+0000 7f7701333640 1 -- 172.21.15.139:0/1707056211 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555601caa680 con 0x555602b59000 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.179+0000 7f7701333640 1 -- 172.21.15.139:0/1707056211 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555601ccf500 con 0x555602b59000 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.179+0000 7f7701333640 1 -- 172.21.15.139:0/1707056211 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555601caa820 con 0x555602b59000 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.179+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555601cb10e0 con 0x555602b59000 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.179+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555601cb1680 con 0x555602b59000 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.179+0000 7f7709a38740 4 datascan.init: resolving metadata pool 13 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.180+0000 7f7701333640 1 -- 172.21.15.139:0/1707056211 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x555602bccc00 con 0x555602b59000 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.180+0000 7f7701333640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x555602b59c00 0x555601cf7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.180+0000 7f7701333640 1 -- 172.21.15.139:0/1707056211 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x555601d55600 con 0x555602b59000 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.180+0000 7f7709a38740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.180+0000 7f7709a38740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.180+0000 7f7709a38740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.180+0000 7f770733f640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x555602b59c00 0x555601cf7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.180+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x555602b59400 0x555601cf6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.181+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602b59800 con 0x555602b59400 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.181+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x555602b59400 0x555601cf6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.181+0000 7f770733f640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x555602b59c00 0x555601cf7700 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x555602a19110 tx=0x555602b77f80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.181+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x555602b59400 0x555601cf6100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.183+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.2 v2:172.21.15.139:6800/1982336775 1 ==== osd_op_reply(1 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602b59400 2024-04-25T10:00:22.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.183+0000 7f7702335640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x555602006c00 0x555602014000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.183+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602007000 con 0x555602006c00 2024-04-25T10:00:22.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.183+0000 7f7707b40640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x555602006c00 0x555602014000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.183+0000 7f7707b40640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x555602006c00 0x555602014000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.183+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== osd_op_reply(2 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.183+0000 7f7702b36640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555602007800 0x555602014580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.183+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602007c00 con 0x555602007800 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.183+0000 7f770733f640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555602007800 0x555602014580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.184+0000 7f770733f640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555602007800 0x555602014580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.185+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(3 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602007800 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.185+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602007400 con 0x555602006c00 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.186+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 2 ==== osd_op_reply(4 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.186+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020000 con 0x555602006c00 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.186+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 3 ==== osd_op_reply(5 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.186+0000 7f7702335640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x555602007000 0x555602014b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.186+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602007400 con 0x555602007000 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.186+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x555602007000 0x555602014b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.186+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x555602007000 0x555602014b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.188+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.3 v2:172.21.15.139:6824/1040931291 1 ==== osd_op_reply(6 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602007000 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.188+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020400 con 0x555602007800 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.188+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.1 v2:172.21.15.139:6814/315640110 2 ==== osd_op_reply(7 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602007800 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.188+0000 7f7702335640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x555602b58400 0x555602015080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.188+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020000 con 0x555602b58400 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.188+0000 7f7707b40640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x555602b58400 0x555602015080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.189+0000 7f7707b40640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x555602b58400 0x555602015080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.191+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.5 v2:172.21.15.150:6824/2184499879 1 ==== osd_op_reply(8 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602b58400 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.191+0000 7f7702b36640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x555602020800 0x555602015600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.191+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602021800 con 0x555602020800 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.191+0000 7f770733f640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x555602020800 0x555602015600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.191+0000 7f770733f640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x555602020800 0x555602015600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.193+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(9 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602020800 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.193+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020c00 con 0x555602007800 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.194+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.1 v2:172.21.15.139:6814/315640110 3 ==== osd_op_reply(10 [pgnls start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602007800 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.194+0000 7f7702b36640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x555602021c00 0x555602015b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.194+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020400 con 0x555602021c00 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.194+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x555602021c00 0x555602015b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.194+0000 7f7706b3e640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x555602021c00 0x555602015b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.194+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.4 v2:172.21.15.150:6800/2810531857 1 ==== osd_op_reply(11 [pgnls start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602021c00 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.194+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602021000 con 0x555602b59400 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.195+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.2 v2:172.21.15.139:6800/1982336775 2 ==== osd_op_reply(12 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602b59400 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.195+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602021800 con 0x555602006c00 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.195+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 4 ==== osd_op_reply(13 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.195+0000 7f7702335640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x555602021400 0x555602016100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.195+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202e000 con 0x555602021400 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.195+0000 7f7707b40640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x555602021400 0x555602016100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.196+0000 7f7707b40640 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x555602021400 0x555602016100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.198+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== osd_op_reply(14 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602021400 2024-04-25T10:00:22.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.198+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602006800 con 0x555602006c00 2024-04-25T10:00:22.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.198+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 5 ==== osd_op_reply(15 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.198+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202e400 con 0x555602007800 2024-04-25T10:00:22.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.198+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.1 v2:172.21.15.139:6814/315640110 4 ==== osd_op_reply(16 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602007800 2024-04-25T10:00:22.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.198+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602007c00 con 0x555602020800 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.198+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.6 v2:172.21.15.150:6809/3369628189 2 ==== osd_op_reply(17 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602020800 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.199+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202e800 con 0x555602021c00 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.199+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.4 v2:172.21.15.150:6800/2810531857 2 ==== osd_op_reply(18 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602021c00 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.199+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202e800 con 0x555602b59400 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.199+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.2 v2:172.21.15.139:6800/1982336775 3 ==== osd_op_reply(19 [pgnls start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602b59400 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.199+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202ec00 con 0x555602021c00 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.199+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.4 v2:172.21.15.150:6800/2810531857 3 ==== osd_op_reply(20 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602021c00 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.199+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202f400 con 0x555602b58400 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.200+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.5 v2:172.21.15.150:6824/2184499879 2 ==== osd_op_reply(21 [pgnls start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602b58400 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.200+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202f000 con 0x555602021c00 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.200+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.4 v2:172.21.15.150:6800/2810531857 4 ==== osd_op_reply(22 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602021c00 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.200+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020400 con 0x555602006c00 2024-04-25T10:00:22.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.200+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 6 ==== osd_op_reply(23 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.200+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202e000 con 0x555602006c00 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.200+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 7 ==== osd_op_reply(24 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.200+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602021000 con 0x555602006c00 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.201+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 8 ==== osd_op_reply(25 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.201+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202f400 con 0x555602021c00 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.201+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.4 v2:172.21.15.150:6800/2810531857 5 ==== osd_op_reply(26 [pgnls start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602021c00 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.201+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602007400 con 0x555602007000 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.201+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.3 v2:172.21.15.139:6824/1040931291 2 ==== osd_op_reply(27 [pgnls start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602007000 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.201+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202f800 con 0x555602b59400 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.201+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.2 v2:172.21.15.139:6800/1982336775 4 ==== osd_op_reply(28 [pgnls start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602b59400 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.201+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602b59800 con 0x555602021c00 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.202+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.4 v2:172.21.15.150:6800/2810531857 6 ==== osd_op_reply(29 [pgnls start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602021c00 2024-04-25T10:00:22.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.202+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602006800 con 0x555602007000 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.202+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.3 v2:172.21.15.139:6824/1040931291 3 ==== osd_op_reply(30 [pgnls start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602007000 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.202+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202e800 con 0x555602021400 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.202+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.7 v2:172.21.15.150:6808/2122423536 2 ==== osd_op_reply(31 [pgnls start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602021400 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.202+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020000 con 0x555602b58400 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.203+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.5 v2:172.21.15.150:6824/2184499879 3 ==== osd_op_reply(32 [pgnls start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602b58400 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.203+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202ec00 con 0x555602007000 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.203+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.3 v2:172.21.15.139:6824/1040931291 4 ==== osd_op_reply(33 [pgnls start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602007000 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.203+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602021800 con 0x555602b58400 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.203+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.5 v2:172.21.15.150:6824/2184499879 4 ==== osd_op_reply(34 [pgnls start_epoch 93 out=49b] v89'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602b58400 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.203+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202f000 con 0x555602021c00 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.203+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.4 v2:172.21.15.150:6800/2810531857 7 ==== osd_op_reply(35 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602021c00 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.204+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020400 con 0x555602007800 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.204+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.1 v2:172.21.15.139:6814/315640110 5 ==== osd_op_reply(36 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602007800 2024-04-25T10:00:22.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.204+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202f400 con 0x555602b58400 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.204+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.5 v2:172.21.15.150:6824/2184499879 5 ==== osd_op_reply(37 [pgnls start_epoch 93 out=49b] v89'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602b58400 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.204+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602021000 con 0x555602020800 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.204+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.6 v2:172.21.15.150:6809/3369628189 3 ==== osd_op_reply(38 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602020800 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.204+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202f800 con 0x555602020800 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.205+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.6 v2:172.21.15.150:6809/3369628189 4 ==== osd_op_reply(39 [pgnls start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602020800 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.205+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020000 con 0x555602021400 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.205+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.7 v2:172.21.15.150:6808/2122423536 3 ==== osd_op_reply(40 [pgnls start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602021400 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.205+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202fc00 con 0x555602006c00 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.205+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 9 ==== osd_op_reply(41 [pgnls start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.205+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602021800 con 0x555602b59400 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.206+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.2 v2:172.21.15.139:6800/1982336775 5 ==== osd_op_reply(42 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602b59400 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.206+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602007400 con 0x555602021400 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.206+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.7 v2:172.21.15.150:6808/2122423536 4 ==== osd_op_reply(43 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602021400 2024-04-25T10:00:22.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.206+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202e000 con 0x555602021c00 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.206+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.4 v2:172.21.15.150:6800/2810531857 8 ==== osd_op_reply(44 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602021c00 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.206+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020c00 con 0x555602020800 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.207+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.6 v2:172.21.15.150:6809/3369628189 5 ==== osd_op_reply(45 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602020800 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.207+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202e800 con 0x555602021c00 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.207+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.4 v2:172.21.15.150:6800/2810531857 9 ==== osd_op_reply(46 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602021c00 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.207+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020400 con 0x555602006c00 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.207+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 10 ==== osd_op_reply(47 [pgnls start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.207+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202ec00 con 0x555602007000 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.208+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.3 v2:172.21.15.139:6824/1040931291 5 ==== osd_op_reply(48 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602007000 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.208+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602021000 con 0x555602006c00 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.208+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 11 ==== osd_op_reply(49 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.208+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202f000 con 0x555602021400 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.208+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.7 v2:172.21.15.150:6808/2122423536 5 ==== osd_op_reply(50 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602021400 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.208+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602007c00 con 0x555602b59400 2024-04-25T10:00:22.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.209+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.2 v2:172.21.15.139:6800/1982336775 6 ==== osd_op_reply(51 [pgnls start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602b59400 2024-04-25T10:00:22.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.209+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202e000 con 0x555602007800 2024-04-25T10:00:22.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.209+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.1 v2:172.21.15.139:6814/315640110 6 ==== osd_op_reply(52 [pgnls start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602007800 2024-04-25T10:00:22.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.209+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202e400 con 0x555602006c00 2024-04-25T10:00:22.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.209+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 12 ==== osd_op_reply(53 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.209+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202e800 con 0x555602021c00 2024-04-25T10:00:22.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.209+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.4 v2:172.21.15.150:6800/2810531857 10 ==== osd_op_reply(54 [pgnls start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602021c00 2024-04-25T10:00:22.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.209+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202f400 con 0x555602007800 2024-04-25T10:00:22.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.210+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.1 v2:172.21.15.139:6814/315640110 7 ==== osd_op_reply(55 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602007800 2024-04-25T10:00:22.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.210+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602021800 con 0x555602021c00 2024-04-25T10:00:22.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.210+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.4 v2:172.21.15.150:6800/2810531857 11 ==== osd_op_reply(56 [pgnls start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602021c00 2024-04-25T10:00:22.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.210+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202fc00 con 0x555602b58400 2024-04-25T10:00:22.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.210+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.5 v2:172.21.15.150:6824/2184499879 6 ==== osd_op_reply(57 [pgnls start_epoch 93 out=81b] v89'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555602a04500 con 0x555602b58400 2024-04-25T10:00:22.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.210+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560203e000 con 0x555602b58400 2024-04-25T10:00:22.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.211+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.5 v2:172.21.15.150:6824/2184499879 7 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555602a04500 con 0x555602b58400 2024-04-25T10:00:22.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.211+0000 7f7709a38740 10 datascan.operator(): handling object 1099511627777.0 2024-04-25T10:00:22.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.211+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560203e400 con 0x555602b58400 2024-04-25T10:00:22.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.212+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.5 v2:172.21.15.150:6824/2184499879 8 ==== osd_op_reply(59 10000000001.00000000 [call] v93'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555602a04500 con 0x555602b58400 2024-04-25T10:00:22.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.212+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560203e800 con 0x555602b58400 2024-04-25T10:00:22.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.212+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.5 v2:172.21.15.150:6824/2184499879 9 ==== osd_op_reply(60 [pgnls start_epoch 93 out=49b] v89'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602b58400 2024-04-25T10:00:22.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.212+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560202f000 con 0x555602007800 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.213+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 <== osd.1 v2:172.21.15.139:6814/315640110 8 ==== osd_op_reply(61 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04780 con 0x555602007800 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.213+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602006800 con 0x555602021c00 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.213+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.4 v2:172.21.15.150:6800/2810531857 12 ==== osd_op_reply(62 [pgnls start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602021c00 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.213+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020000 con 0x555602021400 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.213+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.7 v2:172.21.15.150:6808/2122423536 6 ==== osd_op_reply(63 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602021400 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.213+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602b59800 con 0x555602006c00 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 13 ==== osd_op_reply(64 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7702b36640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x555602020400 con 0x555602006c00 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7707b40640 1 -- 172.21.15.139:0/1707056211 <== osd.0 v2:172.21.15.139:6808/1167038101 14 ==== osd_op_reply(65 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04500 con 0x555602006c00 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7702335640 1 -- 172.21.15.139:0/1707056211 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55560203e000 con 0x555602007000 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7706b3e640 1 -- 172.21.15.139:0/1707056211 <== osd.3 v2:172.21.15.139:6824/1040931291 6 ==== osd_op_reply(66 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555602a04000 con 0x555602007000 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x555602006c00 msgr2=0x555602014000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x555602006c00 0x555602014000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555602007800 msgr2=0x555602014580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x555602007800 0x555602014580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x555602b59400 msgr2=0x555601cf6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x555602b59400 0x555601cf6100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x555602007000 msgr2=0x555602014b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x555602007000 0x555602014b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x555602021c00 msgr2=0x555602015b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x555602021c00 0x555602015b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x555602b58400 msgr2=0x555602015080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x555602b58400 0x555602015080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 reap_dead start 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x555602020800 msgr2=0x555602015600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x555602020800 0x555602015600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x555602021400 msgr2=0x555602016100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x555602021400 0x555602016100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 reap_dead start 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 reap_dead start 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f770733f640 1 -- 172.21.15.139:0/1707056211 reap_dead start 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x555602b59c00 msgr2=0x555601cf7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x555602b59c00 0x555601cf7700 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x555602a19110 tx=0x555602b77f80 comp rx=0 tx=0).stop 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 msgr2=0x555601cf6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 0x555601cf6680 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x555602b71770 tx=0x555602a08900 comp rx=0 tx=0).stop 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 shutdown_connections 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602b59000 0x555601cf6680 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 --2- 172.21.15.139:0/1707056211 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x555602b59c00 0x555601cf7700 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.214+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 >> 172.21.15.139:0/1707056211 conn(0x555602a54000 msgr2=0x555601d7d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.215+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 shutdown_connections 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.215+0000 7f7709a38740 1 -- 172.21.15.139:0/1707056211 wait complete. 2024-04-25T10:00:22.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.215+0000 7f7709a38740 1 -- 172.21.15.139:0/3768358351 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602a54800 msgr2=0x555601cf5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.215+0000 7f7709a38740 1 --2- 172.21.15.139:0/3768358351 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602a54800 0x555601cf5b80 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x555602aadaa0 tx=0x555602a17320 comp rx=0 tx=0).stop 2024-04-25T10:00:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.215+0000 7f7709a38740 1 -- 172.21.15.139:0/3768358351 shutdown_connections 2024-04-25T10:00:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.215+0000 7f7709a38740 1 --2- 172.21.15.139:0/3768358351 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555602a54800 0x555601cf5b80 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.215+0000 7f7709a38740 1 --2- 172.21.15.139:0/3768358351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555601c85c00 0x555601cf4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.215+0000 7f7709a38740 1 --2- 172.21.15.139:0/3768358351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555602a54400 0x555601cf5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.215+0000 7f7709a38740 1 -- 172.21.15.139:0/3768358351 >> 172.21.15.139:0/3768358351 conn(0x555601c85000 msgr2=0x555601d7e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.215+0000 7f7709a38740 1 -- 172.21.15.139:0/3768358351 shutdown_connections 2024-04-25T10:00:22.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.215+0000 7f7709a38740 1 -- 172.21.15.139:0/3768358351 wait complete. 2024-04-25T10:00:22.229 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.091152, result: 2024-04-25T10:00:22.229 DEBUG:teuthology.orchestra.run.smithi139:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-25T10:00:22.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.301+0000 7f4cfeb22740 1 -- start start 2024-04-25T10:00:22.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.301+0000 7f4cfeb22740 1 Processor -- start 2024-04-25T10:00:22.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.301+0000 7f4cfeb22740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7ee2cc00 0x556d7ee9c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.301+0000 7f4cfeb22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7ee2d800 0x556d7ee9cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.301+0000 7f4cfeb22740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7ee2dc00 0x556d7ee9d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.301+0000 7f4cfeb22740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7fbbc780 con 0x556d7ee2d800 2024-04-25T10:00:22.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.301+0000 7f4cfeb22740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7fbbc600 con 0x556d7ee2dc00 2024-04-25T10:00:22.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.301+0000 7f4cfeb22740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d7fbbc480 con 0x556d7ee2cc00 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfc2fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7ee2d800 0x556d7ee9cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfc2fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7ee2d800 0x556d7ee9cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35622/0 (socket says 172.21.15.139:35622) 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4056056104 learned_addr learned my addr 172.21.15.139:0/4056056104 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfcafd640 1 --2- 172.21.15.139:0/4056056104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7ee2cc00 0x556d7ee9c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfd2fe640 1 --2- 172.21.15.139:0/4056056104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7ee2dc00 0x556d7ee9d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4056056104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7ee2cc00 msgr2=0x556d7ee9c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfc2fc640 1 --2- 172.21.15.139:0/4056056104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7ee2cc00 0x556d7ee9c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4056056104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7ee2dc00 msgr2=0x556d7ee9d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfc2fc640 1 --2- 172.21.15.139:0/4056056104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7ee2dc00 0x556d7ee9d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4056056104 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7ee59680 con 0x556d7ee2d800 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfcafd640 1 --2- 172.21.15.139:0/4056056104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7ee2cc00 0x556d7ee9c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfc2fc640 1 --2- 172.21.15.139:0/4056056104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7ee2d800 0x556d7ee9cb00 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x556d7fc43c80 tx=0x556d7efa04e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db75dab4bf232fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfa2f8640 1 -- 172.21.15.139:0/4056056104 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556d7ee52680 con 0x556d7ee2d800 2024-04-25T10:00:22.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfa2f8640 1 -- 172.21.15.139:0/4056056104 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d7ee76e00 con 0x556d7ee2d800 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfa2f8640 1 -- 172.21.15.139:0/4056056104 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556d7ee52b60 con 0x556d7ee2d800 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4056056104 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d7ee594a0 con 0x556d7ee2d800 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4056056104 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556d7ee59a40 con 0x556d7ee2d800 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.302+0000 7f4cfeb22740 4 waiting for MDS map... 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.303+0000 7f4cfa2f8640 1 -- 172.21.15.139:0/4056056104 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x556d7fd06600 con 0x556d7ee2d800 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.303+0000 7f4cfa2f8640 1 -- 172.21.15.139:0/4056056104 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 58) v1 ==== 2723+0+0 (secure 0 0 0) 0x556d7ee85180 con 0x556d7ee2d800 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.303+0000 7f4cfeb22740 4 Got MDS map 58 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.303+0000 7f4cfeb22740 4 datascan.main: Using metadata pool output 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.303+0000 7f4cfeb22740 4 datascan.main: connecting to RADOS... 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.303+0000 7f4cfeb22740 1 Processor -- start 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.303+0000 7f4cfeb22740 1 -- start start 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfeb22740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7fc00c00 0x556d7ee9d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfeb22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc01000 0x556d7ee9db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfeb22740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7fc01400 0x556d7ee9e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfeb22740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7fbbc300 con 0x556d7fc01000 2024-04-25T10:00:22.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfeb22740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7fbbc000 con 0x556d7fc00c00 2024-04-25T10:00:22.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfeb22740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d7ee8c900 con 0x556d7fc01400 2024-04-25T10:00:22.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfcafd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc01000 0x556d7ee9db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfcafd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc01000 0x556d7ee9db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35634/0 (socket says 172.21.15.139:35634) 2024-04-25T10:00:22.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfcafd640 1 -- 172.21.15.139:0/2082997654 learned_addr learned my addr 172.21.15.139:0/2082997654 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:22.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfd2fe640 1 --2- 172.21.15.139:0/2082997654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7fc00c00 0x556d7ee9d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfcafd640 1 -- 172.21.15.139:0/2082997654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7fc01400 msgr2=0x556d7ee9e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfcafd640 1 --2- 172.21.15.139:0/2082997654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7fc01400 0x556d7ee9e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfcafd640 1 -- 172.21.15.139:0/2082997654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7fc00c00 msgr2=0x556d7ee9d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfcafd640 1 --2- 172.21.15.139:0/2082997654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7fc00c00 0x556d7ee9d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfcafd640 1 -- 172.21.15.139:0/2082997654 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7ee59860 con 0x556d7fc01000 2024-04-25T10:00:22.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfd2fe640 1 --2- 172.21.15.139:0/2082997654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7fc00c00 0x556d7ee9d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:22.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cfcafd640 1 --2- 172.21.15.139:0/2082997654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc01000 0x556d7ee9db80 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x556d7fd1aab0 tx=0x556d7fd22c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfda35185e4258b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cf82f4640 1 -- 172.21.15.139:0/2082997654 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556d7ee52680 con 0x556d7fc01000 2024-04-25T10:00:22.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cf82f4640 1 -- 172.21.15.139:0/2082997654 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d7ee76540 con 0x556d7fc01000 2024-04-25T10:00:22.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.304+0000 7f4cf82f4640 1 -- 172.21.15.139:0/2082997654 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556d7ee52b60 con 0x556d7fc01000 2024-04-25T10:00:22.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.305+0000 7f4cfeb22740 1 -- 172.21.15.139:0/2082997654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc01000 msgr2=0x556d7ee9db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.305+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/2082997654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc01000 0x556d7ee9db80 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x556d7fd1aab0 tx=0x556d7fd22c00 comp rx=0 tx=0).stop 2024-04-25T10:00:22.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.305+0000 7f4cfeb22740 1 -- 172.21.15.139:0/2082997654 shutdown_connections 2024-04-25T10:00:22.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.305+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/2082997654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc01000 0x556d7ee9db80 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.305+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/2082997654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7fc01400 0x556d7ee9e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.305+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/2082997654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7fc00c00 0x556d7ee9d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.305+0000 7f4cfeb22740 1 -- 172.21.15.139:0/2082997654 >> 172.21.15.139:0/2082997654 conn(0x556d7fc00800 msgr2=0x556d7ef27000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.305+0000 7f4cfeb22740 1 -- 172.21.15.139:0/2082997654 shutdown_connections 2024-04-25T10:00:22.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.305+0000 7f4cfeb22740 1 -- 172.21.15.139:0/2082997654 wait complete. 2024-04-25T10:00:22.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.305+0000 7f4cfeb22740 1 Processor -- start 2024-04-25T10:00:22.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.305+0000 7f4cfeb22740 1 -- start start 2024-04-25T10:00:22.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfeb22740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc00c00 0x556d7ee9f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfeb22740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7fc01000 0x556d7ee9ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfeb22740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7fc01400 0x556d7ee9e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfeb22740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7fbbc300 con 0x556d7fc00c00 2024-04-25T10:00:22.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfeb22740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7fbbc000 con 0x556d7fc01400 2024-04-25T10:00:22.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfeb22740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d7fbbd500 con 0x556d7fc01000 2024-04-25T10:00:22.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfd2fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc00c00 0x556d7ee9f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfd2fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc00c00 0x556d7ee9f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35650/0 (socket says 172.21.15.139:35650) 2024-04-25T10:00:22.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 learned_addr learned my addr 172.21.15.139:0/4184450091 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:22.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfcafd640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7fc01000 0x556d7ee9ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfc2fc640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7fc01400 0x556d7ee9e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7fc01000 msgr2=0x556d7ee9ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfd2fe640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7fc01000 0x556d7ee9ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7fc01400 msgr2=0x556d7ee9e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfd2fe640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7fc01400 0x556d7ee9e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7ee59680 con 0x556d7fc00c00 2024-04-25T10:00:22.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.306+0000 7f4cfd2fe640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc00c00 0x556d7ee9f180 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x556d7fc43200 tx=0x556d7fbbe8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.307+0000 7f4cf6af1640 1 -- 172.21.15.139:0/4184450091 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556d7ee529c0 con 0x556d7fc00c00 2024-04-25T10:00:22.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.307+0000 7f4cf6af1640 1 -- 172.21.15.139:0/4184450091 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d7ee76c40 con 0x556d7fc00c00 2024-04-25T10:00:22.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.307+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556d7ee594a0 con 0x556d7fc00c00 2024-04-25T10:00:22.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.307+0000 7f4cf6af1640 1 -- 172.21.15.139:0/4184450091 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556d7ee52680 con 0x556d7fc00c00 2024-04-25T10:00:22.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.307+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d7ee592c0 con 0x556d7fc00c00 2024-04-25T10:00:22.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.307+0000 7f4cfeb22740 4 datascan.init: resolving metadata pool 13 2024-04-25T10:00:22.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.308+0000 7f4cf6af1640 1 -- 172.21.15.139:0/4184450091 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x556d7fd6ac00 con 0x556d7fc00c00 2024-04-25T10:00:22.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.308+0000 7f4cf6af1640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x556d7f192000 0x556d7ee9f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.308+0000 7f4cf6af1640 1 -- 172.21.15.139:0/4184450091 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x556d7fd68a00 con 0x556d7fc00c00 2024-04-25T10:00:22.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.308+0000 7f4cfeb22740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:00:22.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.308+0000 7f4cfcafd640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x556d7f192000 0x556d7ee9f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.308+0000 7f4cfeb22740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-25T10:00:22.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.308+0000 7f4cfeb22740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:00:22.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.308+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x556d7f192400 0x556d7ee9e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.308+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f192800 con 0x556d7f192400 2024-04-25T10:00:22.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.308+0000 7f4cfc2fc640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x556d7f192400 0x556d7ee9e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.308+0000 7f4cfcafd640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x556d7f192000 0x556d7ee9f700 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x556d7fd1aab0 tx=0x556d7fbb0f60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.309+0000 7f4cfc2fc640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x556d7f192400 0x556d7ee9e100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.309+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv26 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.309+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x556d7f192c00 0x556d7ee9db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.309+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f193000 con 0x556d7f192c00 2024-04-25T10:00:22.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.309+0000 7f4cfd2fe640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x556d7f192c00 0x556d7ee9db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.309+0000 7f4cfd2fe640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x556d7f192c00 0x556d7ee9db80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.310+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f192c00 2024-04-25T10:00:22.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.310+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x556d7f193400 0x556d7ee9d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.310+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f193800 con 0x556d7f193400 2024-04-25T10:00:22.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.310+0000 7f4cfcafd640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x556d7f193400 0x556d7ee9d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.310+0000 7f4cfcafd640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x556d7f193400 0x556d7ee9d600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.310+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.2 v2:172.21.15.139:6800/1982336775 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f193400 2024-04-25T10:00:22.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.310+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ae000 con 0x556d7f192400 2024-04-25T10:00:22.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.311+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.311+0000 7f4cf82f4640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x556d7f1ae400 0x556d7f1b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.311+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ae800 con 0x556d7f1ae400 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.311+0000 7f4cfc2fc640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x556d7f1ae400 0x556d7f1b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.311+0000 7f4cfc2fc640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x556d7f1ae400 0x556d7f1b4000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.311+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.311+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1af000 con 0x556d7f192400 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.312+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.312+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1aec00 con 0x556d7f192400 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.312+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.312+0000 7f4cf7af3640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x556d7f1af400 0x556d7f1b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.312+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ae000 con 0x556d7f1af400 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.312+0000 7f4cfd2fe640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x556d7f1af400 0x556d7f1b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.312+0000 7f4cfd2fe640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x556d7f1af400 0x556d7f1b4580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.313+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.3 v2:172.21.15.139:6824/1040931291 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1af400 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.313+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1af000 con 0x556d7f1ae400 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.313+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.313+0000 7f4cf7af3640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x556d7f1ae800 0x556d7f1b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.313+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f192800 con 0x556d7f1ae800 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.313+0000 7f4cfcafd640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x556d7f1ae800 0x556d7f1b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.313+0000 7f4cfcafd640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x556d7f1ae800 0x556d7f1b4b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.314+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 1 ==== osd_op_reply(10 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.314+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1af800 con 0x556d7f192c00 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.314+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.6 v2:172.21.15.150:6809/3369628189 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f192c00 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.314+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4000 con 0x556d7f1ae400 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.314+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.314+0000 7f4cf82f4640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x556d7f1afc00 0x556d7f1b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.314+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5000 con 0x556d7f1afc00 2024-04-25T10:00:22.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.315+0000 7f4cfc2fc640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x556d7f1afc00 0x556d7f1b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.315+0000 7f4cfc2fc640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x556d7f1afc00 0x556d7f1b5600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.315+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.315+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4400 con 0x556d7f193400 2024-04-25T10:00:22.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.315+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.2 v2:172.21.15.139:6800/1982336775 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f193400 2024-04-25T10:00:22.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.315+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5400 con 0x556d7f192400 2024-04-25T10:00:22.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.316+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.316+0000 7f4cf7af3640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x556d7f1c4800 0x556d7f1b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.316+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4c00 con 0x556d7f1c4800 2024-04-25T10:00:22.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.316+0000 7f4cfd2fe640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x556d7f1c4800 0x556d7f1b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.316+0000 7f4cfd2fe640 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x556d7f1c4800 0x556d7f1b5080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.316+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1c4800 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.317+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5800 con 0x556d7f192400 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.317+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.317+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ae000 con 0x556d7f1ae400 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.317+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.317+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5c00 con 0x556d7f192c00 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.317+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.6 v2:172.21.15.150:6809/3369628189 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f192c00 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.317+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1af800 con 0x556d7f1afc00 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.318+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.318+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4c00 con 0x556d7f193400 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.318+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.2 v2:172.21.15.139:6800/1982336775 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f193400 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.318+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1aec00 con 0x556d7f1afc00 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.318+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.318+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4000 con 0x556d7f1ae800 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.318+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 2 ==== osd_op_reply(23 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.318+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f193000 con 0x556d7f1afc00 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.319+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.319+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5000 con 0x556d7f192400 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.319+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.319+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7fc0f400 con 0x556d7f192400 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.319+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.319+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5400 con 0x556d7f192400 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.319+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.319+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f193000 con 0x556d7f1afc00 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.320+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.320+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1af000 con 0x556d7f1af400 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.320+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.3 v2:172.21.15.139:6824/1040931291 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1af400 2024-04-25T10:00:22.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.320+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f193800 con 0x556d7f193400 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.320+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.2 v2:172.21.15.139:6800/1982336775 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f193400 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.320+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5800 con 0x556d7f1afc00 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.321+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.321+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f192800 con 0x556d7f1af400 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.321+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.3 v2:172.21.15.139:6824/1040931291 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1af400 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.321+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5000 con 0x556d7f1c4800 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.321+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.7 v2:172.21.15.150:6808/2122423536 2 ==== osd_op_reply(33 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1c4800 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.321+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ae000 con 0x556d7f1ae800 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.322+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 3 ==== osd_op_reply(34 [pgnls-filter start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.322+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4400 con 0x556d7f1af400 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.322+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.3 v2:172.21.15.139:6824/1040931291 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1af400 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.322+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1af800 con 0x556d7f1ae800 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.322+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 93 out=49b] v89'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.322+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4c00 con 0x556d7f1afc00 2024-04-25T10:00:22.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.322+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.323+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1aec00 con 0x556d7f1ae400 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.323+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.323+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8000 con 0x556d7f1ae800 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.323+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 93 out=49b] v89'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.323+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8000 con 0x556d7f192c00 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.323+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.6 v2:172.21.15.150:6809/3369628189 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f192c00 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.323+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8400 con 0x556d7f192c00 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.324+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.6 v2:172.21.15.150:6809/3369628189 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f192c00 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.324+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9c00 con 0x556d7f1c4800 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.324+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.7 v2:172.21.15.150:6808/2122423536 3 ==== osd_op_reply(42 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1c4800 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.324+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8800 con 0x556d7f192400 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.324+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.324+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ae000 con 0x556d7f193400 2024-04-25T10:00:22.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.325+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.2 v2:172.21.15.139:6800/1982336775 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f193400 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.325+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8c00 con 0x556d7f1c4800 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.325+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.7 v2:172.21.15.150:6808/2122423536 4 ==== osd_op_reply(45 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1c4800 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.325+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1af800 con 0x556d7f1afc00 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.325+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.325+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9000 con 0x556d7f192c00 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.326+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.6 v2:172.21.15.150:6809/3369628189 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f192c00 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.326+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f193800 con 0x556d7f1afc00 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.326+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.326+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9400 con 0x556d7f192400 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.326+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.326+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4000 con 0x556d7f1af400 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.326+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.3 v2:172.21.15.139:6824/1040931291 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1af400 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.326+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9800 con 0x556d7f192400 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.327+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.327+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5400 con 0x556d7f1c4800 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.327+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.7 v2:172.21.15.150:6808/2122423536 5 ==== osd_op_reply(52 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1c4800 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.327+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8800 con 0x556d7f193400 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.327+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.2 v2:172.21.15.139:6800/1982336775 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f193400 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.327+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5800 con 0x556d7f1ae400 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.327+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 6 ==== osd_op_reply(54 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.327+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8000 con 0x556d7f192400 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.328+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.328+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4c00 con 0x556d7f1afc00 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.328+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.328+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5c00 con 0x556d7f1ae400 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.328+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 7 ==== osd_op_reply(57 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.328+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f193000 con 0x556d7f1afc00 2024-04-25T10:00:22.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.328+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.328+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5000 con 0x556d7f1ae800 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.329+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 6 ==== osd_op_reply(59 [pgnls-filter start_epoch 93 out=81b] v93'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.329+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f193c00 con 0x556d7f193400 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.329+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.2 v2:172.21.15.139:6800/1982336775 7 ==== osd_op_reply(60 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f193400 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.329+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4400 con 0x556d7f192400 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.329+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.329+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f192800 con 0x556d7f1ae400 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.330+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 8 ==== osd_op_reply(62 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.330+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5800 con 0x556d7f192400 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.330+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.330+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1aec00 con 0x556d7f192400 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.330+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.330+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4c00 con 0x556d7f1af400 2024-04-25T10:00:22.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.330+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.3 v2:172.21.15.139:6824/1040931291 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1af400 2024-04-25T10:00:22.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.330+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1af000 con 0x556d7f1ae400 2024-04-25T10:00:22.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.330+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 9 ==== osd_op_reply(66 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.330+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5c00 con 0x556d7f1ae800 2024-04-25T10:00:22.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.331+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.331+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7fc0f400 con 0x556d7f192c00 2024-04-25T10:00:22.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.331+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.6 v2:172.21.15.150:6809/3369628189 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f192c00 2024-04-25T10:00:22.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.331+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9400 con 0x556d7f1ae400 2024-04-25T10:00:22.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.331+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 10 ==== osd_op_reply(69 [pgnls-filter start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.331+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4000 con 0x556d7f1afc00 2024-04-25T10:00:22.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.332+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.332+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9800 con 0x556d7f193400 2024-04-25T10:00:22.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.332+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.2 v2:172.21.15.139:6800/1982336775 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f193400 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.332+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5400 con 0x556d7f192400 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.332+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.332+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8000 con 0x556d7f1c4800 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.332+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.7 v2:172.21.15.150:6808/2122423536 6 ==== osd_op_reply(73 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1c4800 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.332+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4c00 con 0x556d7f192400 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.333+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.333+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f193800 con 0x556d7f1ae400 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.333+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 11 ==== osd_op_reply(75 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.333+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8400 con 0x556d7f192c00 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.333+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.6 v2:172.21.15.150:6809/3369628189 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f192c00 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.333+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1af800 con 0x556d7f1afc00 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.333+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.333+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9c00 con 0x556d7f193400 2024-04-25T10:00:22.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.334+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.2 v2:172.21.15.139:6800/1982336775 9 ==== osd_op_reply(78 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f193400 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.334+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4400 con 0x556d7f1afc00 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.334+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.334+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8c00 con 0x556d7f1ae800 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.334+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 8 ==== osd_op_reply(80 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.334+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5800 con 0x556d7f1afc00 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.334+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.334+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9000 con 0x556d7f192400 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.335+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.335+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4000 con 0x556d7f192400 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.335+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.335+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9400 con 0x556d7f192400 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.335+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.335+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5400 con 0x556d7f1afc00 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.335+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.335+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8800 con 0x556d7f1af400 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.336+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.3 v2:172.21.15.139:6824/1040931291 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1af400 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.336+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f193000 con 0x556d7f193400 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.336+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.2 v2:172.21.15.139:6800/1982336775 10 ==== osd_op_reply(87 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f193400 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.336+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9800 con 0x556d7f1afc00 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.336+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.336+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f192800 con 0x556d7f1af400 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.336+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.3 v2:172.21.15.139:6824/1040931291 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1af400 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.337+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5000 con 0x556d7f1c4800 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.337+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.7 v2:172.21.15.150:6808/2122423536 7 ==== osd_op_reply(90 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1c4800 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.337+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f193c00 con 0x556d7f1ae800 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.337+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 9 ==== osd_op_reply(91 [pgnls-filter start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.337+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5c00 con 0x556d7f1af400 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.337+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.3 v2:172.21.15.139:6824/1040931291 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1af400 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.337+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1aec00 con 0x556d7f1ae800 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.338+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 10 ==== osd_op_reply(93 [pgnls-filter start_epoch 93 out=49b] v89'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.338+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9000 con 0x556d7f1afc00 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.338+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.338+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1af000 con 0x556d7f1ae400 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.338+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 12 ==== osd_op_reply(95 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.338+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9400 con 0x556d7f1ae800 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.338+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 11 ==== osd_op_reply(96 [pgnls-filter start_epoch 93 out=49b] v89'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.338+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ae000 con 0x556d7f192c00 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.339+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.6 v2:172.21.15.150:6809/3369628189 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f192c00 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.339+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9800 con 0x556d7f192c00 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.339+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.6 v2:172.21.15.150:6809/3369628189 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f192c00 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.339+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ae000 con 0x556d7f1c4800 2024-04-25T10:00:22.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.339+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.7 v2:172.21.15.150:6808/2122423536 8 ==== osd_op_reply(99 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1c4800 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.339+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4c00 con 0x556d7f192400 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.340+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.340+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5000 con 0x556d7f193400 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.340+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.2 v2:172.21.15.139:6800/1982336775 11 ==== osd_op_reply(101 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f193400 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.340+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4400 con 0x556d7f1c4800 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.340+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.7 v2:172.21.15.150:6808/2122423536 9 ==== osd_op_reply(102 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1c4800 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.340+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5c00 con 0x556d7f1afc00 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.340+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.340+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5800 con 0x556d7f192c00 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.341+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.6 v2:172.21.15.150:6809/3369628189 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f192c00 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.341+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f192800 con 0x556d7f1afc00 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.341+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.341+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4000 con 0x556d7f192400 2024-04-25T10:00:22.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.341+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.342+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8000 con 0x556d7f1af400 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.342+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.3 v2:172.21.15.139:6824/1040931291 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1af400 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.342+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5400 con 0x556d7f192400 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.342+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.342+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8400 con 0x556d7f1c4800 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.342+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.7 v2:172.21.15.150:6808/2122423536 10 ==== osd_op_reply(109 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1c4800 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.342+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1af800 con 0x556d7f193400 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.343+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.2 v2:172.21.15.139:6800/1982336775 12 ==== osd_op_reply(110 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f193400 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.343+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8800 con 0x556d7f1ae400 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.343+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 13 ==== osd_op_reply(111 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.343+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f193800 con 0x556d7f192400 2024-04-25T10:00:22.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.343+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.343+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7fc0f400 con 0x556d7f1afc00 2024-04-25T10:00:22.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.344+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.344+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5000 con 0x556d7f1ae400 2024-04-25T10:00:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.344+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 14 ==== osd_op_reply(114 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.344+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4c00 con 0x556d7f1afc00 2024-04-25T10:00:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.344+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.344+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9c00 con 0x556d7f1ae800 2024-04-25T10:00:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.344+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 12 ==== osd_op_reply(116 [pgnls-filter start_epoch 93 out=81b] v93'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.344+0000 7f4cfeb22740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-25T10:00:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.344+0000 7f4cfeb22740 10 datascan.operator(): handling object 10000000001.0 2024-04-25T10:00:22.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.344+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ee400 con 0x556d7f1ae800 2024-04-25T10:00:22.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.345+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 13 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.345+0000 7f4cfeb22740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-25T10:00:22.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.345+0000 7f4cfeb22740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-25T10:00:22.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.345+0000 7f4cfeb22740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-25T10:00:22.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.345+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ee800 con 0x556d7f1ae400 2024-04-25T10:00:22.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.345+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 15 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+59 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.345+0000 7f4cfeb22740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-25T10:00:22.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.345+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1eec00 con 0x556d7f192c00 2024-04-25T10:00:22.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.6 v2:172.21.15.150:6809/3369628189 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+581 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f192c00 2024-04-25T10:00:22.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfeb22740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-25T10:00:22.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfeb22740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T10:00:22.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfeb22740 20 datascan.get_frag_of: frag is * 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ef000 con 0x556d7f1ae400 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 16 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfeb22740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfeb22740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ef400 con 0x556d7f1ae400 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 17 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfeb22740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfeb22740 20 datascan.read_dentry: file_head not found in result 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfeb22740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.346+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1ef800 con 0x556d7f1ae400 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.347+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 18 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v93'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.347+0000 7f4cfeb22740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:00:22.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.347+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1efc00 con 0x556d7f1ae800 2024-04-25T10:00:22.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.348+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 <== osd.5 v2:172.21.15.150:6824/2184499879 14 ==== osd_op_reply(123 [pgnls-filter start_epoch 93 out=49b] v89'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac780 con 0x556d7f1ae800 2024-04-25T10:00:22.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.348+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d8c00 con 0x556d7f1ae400 2024-04-25T10:00:22.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.348+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.1 v2:172.21.15.139:6814/315640110 19 ==== osd_op_reply(124 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1ae400 2024-04-25T10:00:22.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.348+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5c00 con 0x556d7f1afc00 2024-04-25T10:00:22.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.348+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.4 v2:172.21.15.150:6800/2810531857 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f1afc00 2024-04-25T10:00:22.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.348+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1d9400 con 0x556d7f1c4800 2024-04-25T10:00:22.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.7 v2:172.21.15.150:6808/2122423536 11 ==== osd_op_reply(126 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1c4800 2024-04-25T10:00:22.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c4000 con 0x556d7f192400 2024-04-25T10:00:22.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cf7af3640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1aec00 con 0x556d7f192400 2024-04-25T10:00:22.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfc2fc640 1 -- 172.21.15.139:0/4184450091 <== osd.0 v2:172.21.15.139:6808/1167038101 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac000 con 0x556d7f192400 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cf82f4640 1 -- 172.21.15.139:0/4184450091 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x556d7f1c5400 con 0x556d7f1af400 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfd2fe640 1 -- 172.21.15.139:0/4184450091 <== osd.3 v2:172.21.15.139:6824/1040931291 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7fbac500 con 0x556d7f1af400 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x556d7f192400 msgr2=0x556d7ee9e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x556d7f192400 0x556d7ee9e100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x556d7f1ae400 msgr2=0x556d7f1b4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x556d7f1ae400 0x556d7f1b4000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x556d7f193400 msgr2=0x556d7ee9d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x556d7f193400 0x556d7ee9d600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x556d7f1af400 msgr2=0x556d7f1b4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x556d7f1af400 0x556d7f1b4580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 reap_dead start 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x556d7f1afc00 msgr2=0x556d7f1b5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x556d7f1afc00 0x556d7f1b5600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.349+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x556d7f1ae800 msgr2=0x556d7f1b4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x556d7f1ae800 0x556d7f1b4b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x556d7f192c00 msgr2=0x556d7ee9db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfcafd640 1 -- 172.21.15.139:0/4184450091 reap_dead start 2024-04-25T10:00:22.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x556d7f192c00 0x556d7ee9db80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x556d7f1c4800 msgr2=0x556d7f1b5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x556d7f1c4800 0x556d7f1b5080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x556d7f192000 msgr2=0x556d7ee9f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x556d7f192000 0x556d7ee9f700 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x556d7fd1aab0 tx=0x556d7fbb0f60 comp rx=0 tx=0).stop 2024-04-25T10:00:22.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc00c00 msgr2=0x556d7ee9f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc00c00 0x556d7ee9f180 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x556d7fc43200 tx=0x556d7fbbe8a0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 shutdown_connections 2024-04-25T10:00:22.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x556d7f1c4800 0x556d7f1b5080 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x556d7f192c00 0x556d7ee9db80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x556d7f192000 0x556d7ee9f700 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4184450091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7fc00c00 0x556d7ee9f180 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 >> 172.21.15.139:0/4184450091 conn(0x556d7fc00800 msgr2=0x556d7ef27000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 shutdown_connections 2024-04-25T10:00:22.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4184450091 wait complete. 2024-04-25T10:00:22.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4056056104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7ee2d800 msgr2=0x556d7ee9cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4056056104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7ee2d800 0x556d7ee9cb00 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x556d7fc43c80 tx=0x556d7efa04e0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4056056104 shutdown_connections 2024-04-25T10:00:22.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4056056104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556d7ee2d800 0x556d7ee9cb00 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4056056104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d7ee2dc00 0x556d7ee9d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 --2- 172.21.15.139:0/4056056104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x556d7ee2cc00 0x556d7ee9c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4056056104 >> 172.21.15.139:0/4056056104 conn(0x556d7ee2d000 msgr2=0x556d7ef26800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4056056104 shutdown_connections 2024-04-25T10:00:22.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.350+0000 7f4cfeb22740 1 -- 172.21.15.139:0/4056056104 wait complete. 2024-04-25T10:00:22.381 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.151973, result: 2024-04-25T10:00:22.382 DEBUG:teuthology.orchestra.run.smithi139:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T10:00:22.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.456+0000 7fdd64a1a740 1 -- start start 2024-04-25T10:00:22.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.456+0000 7fdd64a1a740 1 Processor -- start 2024-04-25T10:00:22.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.456+0000 7fdd64a1a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4da890c00 0x55f4da900580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.456+0000 7fdd64a1a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4da891800 0x55f4da900b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.456+0000 7fdd64a1a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4da891c00 0x55f4da901080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.456+0000 7fdd64a1a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4db620600 con 0x55f4da891800 2024-04-25T10:00:22.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.456+0000 7fdd64a1a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4db620480 con 0x55f4da890c00 2024-04-25T10:00:22.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.456+0000 7fdd64a1a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4db620300 con 0x55f4da891c00 2024-04-25T10:00:22.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.456+0000 7fdd61e58640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4da891800 0x55f4da900b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.456+0000 7fdd61e58640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4da891800 0x55f4da900b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35654/0 (socket says 172.21.15.139:35654) 2024-04-25T10:00:22.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.456+0000 7fdd61e58640 1 -- 172.21.15.139:0/3214970471 learned_addr learned my addr 172.21.15.139:0/3214970471 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:22.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/3214970471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4da891c00 0x55f4da901080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd62659640 1 --2- 172.21.15.139:0/3214970471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4da890c00 0x55f4da900580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd61e58640 1 -- 172.21.15.139:0/3214970471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4da891c00 msgr2=0x55f4da901080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd61e58640 1 --2- 172.21.15.139:0/3214970471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4da891c00 0x55f4da901080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd61e58640 1 -- 172.21.15.139:0/3214970471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4da890c00 msgr2=0x55f4da900580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd61e58640 1 --2- 172.21.15.139:0/3214970471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4da890c00 0x55f4da900580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd61e58640 1 -- 172.21.15.139:0/3214970471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4da8bd680 con 0x55f4da891800 2024-04-25T10:00:22.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/3214970471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4da891c00 0x55f4da901080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:22.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd61e58640 1 --2- 172.21.15.139:0/3214970471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4da891800 0x55f4da900b00 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x55f4db6a7c80 tx=0x55f4daa042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80beb7e64f3e2a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd5fe54640 1 -- 172.21.15.139:0/3214970471 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f4da8b6680 con 0x55f4da891800 2024-04-25T10:00:22.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd5fe54640 1 -- 172.21.15.139:0/3214970471 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f4da8dac40 con 0x55f4da891800 2024-04-25T10:00:22.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd5fe54640 1 -- 172.21.15.139:0/3214970471 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f4da8b69c0 con 0x55f4da891800 2024-04-25T10:00:22.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd64a1a740 1 -- 172.21.15.139:0/3214970471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4da8bd4a0 con 0x55f4da891800 2024-04-25T10:00:22.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd64a1a740 1 -- 172.21.15.139:0/3214970471 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f4da8bc3c0 con 0x55f4da891800 2024-04-25T10:00:22.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd64a1a740 4 waiting for MDS map... 2024-04-25T10:00:22.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd5fe54640 1 -- 172.21.15.139:0/3214970471 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x55f4db758400 con 0x55f4da891800 2024-04-25T10:00:22.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd5fe54640 1 -- 172.21.15.139:0/3214970471 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 58) v1 ==== 2723+0+0 (secure 0 0 0) 0x55f4da8e9180 con 0x55f4da891800 2024-04-25T10:00:22.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd64a1a740 4 Got MDS map 58 2024-04-25T10:00:22.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd64a1a740 4 datascan.main: Using metadata pool output 2024-04-25T10:00:22.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.457+0000 7fdd64a1a740 4 datascan.main: connecting to RADOS... 2024-04-25T10:00:22.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.458+0000 7fdd64a1a740 1 Processor -- start 2024-04-25T10:00:22.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.458+0000 7fdd64a1a740 1 -- start start 2024-04-25T10:00:22.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.458+0000 7fdd64a1a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b000 0x55f4da901600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.458+0000 7fdd64a1a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4db75b400 0x55f4da901b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.458+0000 7fdd64a1a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4db75b800 0x55f4da902100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.458+0000 7fdd64a1a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4db620180 con 0x55f4db75b000 2024-04-25T10:00:22.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.458+0000 7fdd64a1a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4da8f0480 con 0x55f4db75b800 2024-04-25T10:00:22.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.458+0000 7fdd64a1a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4da8f0900 con 0x55f4db75b400 2024-04-25T10:00:22.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd62e5a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b000 0x55f4da901600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd62e5a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b000 0x55f4da901600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35670/0 (socket says 172.21.15.139:35670) 2024-04-25T10:00:22.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd62e5a640 1 -- 172.21.15.139:0/592306464 learned_addr learned my addr 172.21.15.139:0/592306464 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:22.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd62659640 1 --2- 172.21.15.139:0/592306464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4db75b400 0x55f4da901b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd61e58640 1 --2- 172.21.15.139:0/592306464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4db75b800 0x55f4da902100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd62e5a640 1 -- 172.21.15.139:0/592306464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4db75b400 msgr2=0x55f4da901b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/592306464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4db75b400 0x55f4da901b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd62e5a640 1 -- 172.21.15.139:0/592306464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4db75b800 msgr2=0x55f4da902100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/592306464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4db75b800 0x55f4da902100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd62e5a640 1 -- 172.21.15.139:0/592306464 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4da8bda40 con 0x55f4db75b000 2024-04-25T10:00:22.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/592306464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b000 0x55f4da901600 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x55f4db662ab0 tx=0x55f4db776c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63679f63428569d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd5de50640 1 -- 172.21.15.139:0/592306464 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f4da8b6680 con 0x55f4db75b000 2024-04-25T10:00:22.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd5de50640 1 -- 172.21.15.139:0/592306464 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f4da8daa80 con 0x55f4db75b000 2024-04-25T10:00:22.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.459+0000 7fdd5de50640 1 -- 172.21.15.139:0/592306464 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f4da8b69c0 con 0x55f4db75b000 2024-04-25T10:00:22.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.460+0000 7fdd64a1a740 1 -- 172.21.15.139:0/592306464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b000 msgr2=0x55f4da901600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.460+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/592306464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b000 0x55f4da901600 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x55f4db662ab0 tx=0x55f4db776c00 comp rx=0 tx=0).stop 2024-04-25T10:00:22.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.460+0000 7fdd64a1a740 1 -- 172.21.15.139:0/592306464 shutdown_connections 2024-04-25T10:00:22.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.460+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/592306464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4db75b800 0x55f4da902100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.460+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/592306464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4db75b400 0x55f4da901b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.460+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/592306464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b000 0x55f4da901600 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.460+0000 7fdd64a1a740 1 -- 172.21.15.139:0/592306464 >> 172.21.15.139:0/592306464 conn(0x55f4db75ac00 msgr2=0x55f4da988800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.460+0000 7fdd64a1a740 1 -- 172.21.15.139:0/592306464 shutdown_connections 2024-04-25T10:00:22.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.460+0000 7fdd64a1a740 1 -- 172.21.15.139:0/592306464 wait complete. 2024-04-25T10:00:22.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.460+0000 7fdd64a1a740 1 Processor -- start 2024-04-25T10:00:22.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.460+0000 7fdd64a1a740 1 -- start start 2024-04-25T10:00:22.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd64a1a740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4db75b000 0x55f4da903180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd64a1a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4db75b400 0x55f4da902c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd64a1a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b800 0x55f4da902680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd64a1a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4db621380 con 0x55f4db75b800 2024-04-25T10:00:22.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd64a1a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4db620180 con 0x55f4db75b400 2024-04-25T10:00:22.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd64a1a740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4db621500 con 0x55f4db75b000 2024-04-25T10:00:22.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd61e58640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b800 0x55f4da902680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd61e58640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b800 0x55f4da902680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35684/0 (socket says 172.21.15.139:35684) 2024-04-25T10:00:22.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd62e5a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4db75b000 0x55f4da903180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 learned_addr learned my addr 172.21.15.139:0/2126967317 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:22.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd62659640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4db75b400 0x55f4da902c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4db75b000 msgr2=0x55f4da903180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd61e58640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4db75b000 0x55f4da903180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4db75b400 msgr2=0x55f4da902c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd61e58640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4db75b400 0x55f4da902c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4da8bd2c0 con 0x55f4db75b800 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd61e58640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b800 0x55f4da902680 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x55f4db660180 tx=0x55f4db614fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd5c64d640 1 -- 172.21.15.139:0/2126967317 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f4da8b6680 con 0x55f4db75b800 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd5c64d640 1 -- 172.21.15.139:0/2126967317 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f4da8da540 con 0x55f4db75b800 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f4da8bd680 con 0x55f4db75b800 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd5c64d640 1 -- 172.21.15.139:0/2126967317 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f4da8b69c0 con 0x55f4db75b800 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4da8bd4a0 con 0x55f4db75b800 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.461+0000 7fdd64a1a740 4 datascan.init: resolving metadata pool 13 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.463+0000 7fdd5c64d640 1 -- 172.21.15.139:0/2126967317 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55f4db688c00 con 0x55f4db75b800 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.463+0000 7fdd5c64d640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55f4db681800 0x55f4da903700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.463+0000 7fdd5c64d640 1 -- 172.21.15.139:0/2126967317 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x55f4db682e00 con 0x55f4db75b800 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.463+0000 7fdd64a1a740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:00:22.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.463+0000 7fdd64a1a740 4 datascan.main: resolving metadata pool 13 2024-04-25T10:00:22.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.463+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55f4db681800 0x55f4da903700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.463+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55f4db681c00 0x55f4da902100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.463+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac04000 con 0x55f4db681c00 2024-04-25T10:00:22.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.463+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55f4db681800 0x55f4da903700 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x55f4db662ab0 tx=0x55f4daa04060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.463+0000 7fdd62659640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55f4db681c00 0x55f4da902100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.464+0000 7fdd62659640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55f4db681c00 0x55f4da902100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.464+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== osd_op_reply(1 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.464+0000 7fdd62659640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55f4dac04000 0x55f4dac06000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.464+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db669400 con 0x55f4dac04000 2024-04-25T10:00:22.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.464+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55f4dac04000 0x55f4dac06000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.464+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55f4dac04000 0x55f4dac06000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.465+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(2 [pgnls start_epoch 93 out=177b] v93'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.465+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x55f4db669400 0x55f4dac06580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.465+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db680400 con 0x55f4db669400 2024-04-25T10:00:22.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.465+0000 7fdd62659640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x55f4db669400 0x55f4dac06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.465+0000 7fdd62659640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x55f4db669400 0x55f4dac06580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.465+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.2 v2:172.21.15.139:6800/1982336775 1 ==== osd_op_reply(3 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610000 con 0x55f4db669400 2024-04-25T10:00:22.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.465+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db680400 con 0x55f4db681c00 2024-04-25T10:00:22.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.466+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 2 ==== osd_op_reply(4 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.466+0000 7fdd62659640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55f4db680400 0x55f4dac06b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.466+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db669000 con 0x55f4db680400 2024-04-25T10:00:22.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.466+0000 7fdd61e58640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55f4db680400 0x55f4dac06b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.466+0000 7fdd61e58640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55f4db680400 0x55f4dac06b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.467+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(5 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.467+0000 7fdd61e58640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55f4db669000 0x55f4dac07600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.467+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db680c00 con 0x55f4db669000 2024-04-25T10:00:22.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.467+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55f4db669000 0x55f4dac07600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.467+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55f4db669000 0x55f4dac07600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.467+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== osd_op_reply(6 [pgnls start_epoch 93 out=149b] v92'29 uv29 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55f4db610500 con 0x55f4db669000 2024-04-25T10:00:22.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.467+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db680c00 con 0x55f4dac04000 2024-04-25T10:00:22.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.468+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 2 ==== osd_op_reply(7 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.468+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db680c00 con 0x55f4db680400 2024-04-25T10:00:22.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.468+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 2 ==== osd_op_reply(8 [pgnls start_epoch 93 out=73b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.468+0000 7fdd61e58640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55f4db680c00 0x55f4dac07b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.468+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681000 con 0x55f4db680c00 2024-04-25T10:00:22.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.468+0000 7fdd62659640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55f4db680c00 0x55f4dac07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.469+0000 7fdd62659640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55f4db680c00 0x55f4dac07b80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.469+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.5 v2:172.21.15.150:6824/2184499879 1 ==== osd_op_reply(9 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610000 con 0x55f4db680c00 2024-04-25T10:00:22.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.469+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681000 con 0x55f4db669000 2024-04-25T10:00:22.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.469+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.0 v2:172.21.15.139:6808/1167038101 2 ==== osd_op_reply(10 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4db669000 2024-04-25T10:00:22.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.469+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681000 con 0x55f4db669400 2024-04-25T10:00:22.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.469+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.2 v2:172.21.15.139:6800/1982336775 2 ==== osd_op_reply(11 [pgnls start_epoch 93 out=81b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f4db610000 con 0x55f4db669400 2024-04-25T10:00:22.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.469+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681000 con 0x55f4dac04000 2024-04-25T10:00:22.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.470+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 3 ==== osd_op_reply(12 [pgnls start_epoch 93 out=49b] v90'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.470+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55f4db681000 0x55f4dac08680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.470+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac05800 con 0x55f4db681000 2024-04-25T10:00:22.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.470+0000 7fdd61e58640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55f4db681000 0x55f4dac08680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.470+0000 7fdd61e58640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55f4db681000 0x55f4dac08680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.470+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 1 ==== osd_op_reply(13 [pgnls start_epoch 93 out=77b] v92'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.470+0000 7fdd61e58640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55f4dac05800 0x55f4dac08100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.470+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4dac05800 2024-04-25T10:00:22.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.471+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55f4dac05800 0x55f4dac08100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.471+0000 7fdd62e5a640 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55f4dac05800 0x55f4dac08100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.471+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.3 v2:172.21.15.139:6824/1040931291 1 ==== osd_op_reply(14 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac05800 2024-04-25T10:00:22.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.471+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db681c00 2024-04-25T10:00:22.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.471+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 3 ==== osd_op_reply(15 [pgnls start_epoch 93 out=73b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.471+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db681000 2024-04-25T10:00:22.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.472+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 2 ==== osd_op_reply(16 [pgnls start_epoch 93 out=49b] v90'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.472+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db680400 2024-04-25T10:00:22.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.472+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 3 ==== osd_op_reply(17 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.472+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4dac04000 2024-04-25T10:00:22.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.472+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 4 ==== osd_op_reply(18 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.472+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db680c00 2024-04-25T10:00:22.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.473+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.5 v2:172.21.15.150:6824/2184499879 2 ==== osd_op_reply(19 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610000 con 0x55f4db680c00 2024-04-25T10:00:22.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.473+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4dac05800 2024-04-25T10:00:22.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.473+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.3 v2:172.21.15.139:6824/1040931291 2 ==== osd_op_reply(20 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac05800 2024-04-25T10:00:22.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.473+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db681000 2024-04-25T10:00:22.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.473+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 3 ==== osd_op_reply(21 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.473+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db681000 2024-04-25T10:00:22.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.473+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 4 ==== osd_op_reply(22 [pgnls start_epoch 93 out=97b] v92'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.473+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db681000 2024-04-25T10:00:22.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.474+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 5 ==== osd_op_reply(23 [pgnls start_epoch 93 out=74b] v88'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.474+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db681c00 2024-04-25T10:00:22.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.474+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 4 ==== osd_op_reply(24 [pgnls start_epoch 93 out=73b] v92'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.474+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db669000 2024-04-25T10:00:22.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.474+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.0 v2:172.21.15.139:6808/1167038101 3 ==== osd_op_reply(25 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4db669000 2024-04-25T10:00:22.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.474+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db681c00 2024-04-25T10:00:22.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.475+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 5 ==== osd_op_reply(26 [pgnls start_epoch 93 out=98b] v92'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.475+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db680c00 2024-04-25T10:00:22.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.475+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.5 v2:172.21.15.150:6824/2184499879 3 ==== osd_op_reply(27 [pgnls start_epoch 93 out=76b] v92'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55f4db610000 con 0x55f4db680c00 2024-04-25T10:00:22.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.475+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db681000 2024-04-25T10:00:22.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.475+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 6 ==== osd_op_reply(28 [pgnls start_epoch 93 out=73b] v92'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.475+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db681c00 2024-04-25T10:00:22.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.476+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 6 ==== osd_op_reply(29 [pgnls start_epoch 93 out=73b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.476+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db680c00 2024-04-25T10:00:22.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.476+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.5 v2:172.21.15.150:6824/2184499879 4 ==== osd_op_reply(30 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610000 con 0x55f4db680c00 2024-04-25T10:00:22.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.476+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db680400 2024-04-25T10:00:22.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.476+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 4 ==== osd_op_reply(31 [pgnls start_epoch 93 out=73b] v88'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.476+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db680400 2024-04-25T10:00:22.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.477+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 5 ==== osd_op_reply(32 [pgnls start_epoch 93 out=101b] v92'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.477+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T10:00:22.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.477+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac04400 con 0x55f4dac04000 2024-04-25T10:00:22.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.477+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.477+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T10:00:22.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.477+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac04800 con 0x55f4dac04000 2024-04-25T10:00:22.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.477+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.477+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:00:22.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.477+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac04c00 con 0x55f4dac04000 2024-04-25T10:00:22.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 20 datascan.scan_links: injected ino detected 2024-04-25T10:00:22.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac05000 con 0x55f4dac04000 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac05400 con 0x55f4dac04000 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T10:00:22.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac05c00 con 0x55f4db669000 2024-04-25T10:00:22.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.0 v2:172.21.15.139:6808/1167038101 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv27 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610500 con 0x55f4db669000 2024-04-25T10:00:22.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T10:00:22.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.478+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac04400 con 0x55f4db680400 2024-04-25T10:00:22.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.479+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.479+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-25T10:00:22.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.479+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac04800 con 0x55f4db669400 2024-04-25T10:00:22.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.479+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.2 v2:172.21.15.139:6800/1982336775 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55f4db610000 con 0x55f4db669400 2024-04-25T10:00:22.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.479+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T10:00:22.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.479+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:00:22.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.479+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T10:00:22.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.479+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac04c00 con 0x55f4db681c00 2024-04-25T10:00:22.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T10:00:22.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac05000 con 0x55f4db681000 2024-04-25T10:00:22.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T10:00:22.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:00:22.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:00:22.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:00:22.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T10:00:22.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac32000 con 0x55f4db681c00 2024-04-25T10:00:22.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:00:22.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:00:22.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:00:22.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.480+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac32400 con 0x55f4db681c00 2024-04-25T10:00:22.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:00:22.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:00:22.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-25T10:00:22.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T10:00:22.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T10:00:22.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac32800 con 0x55f4db681c00 2024-04-25T10:00:22.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T10:00:22.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:00:22.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:00:22.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:00:22.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd64a1a740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.481+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac32c00 con 0x55f4db680400 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.482+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1163 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.482+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681c00 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.482+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 11 ==== osd_op_reply(47 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.482+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4dac04000 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.482+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 10 ==== osd_op_reply(48 [pgnls start_epoch 93 out=177b] v93'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.482+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db669400 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.483+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.2 v2:172.21.15.139:6800/1982336775 4 ==== osd_op_reply(49 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610000 con 0x55f4db669400 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.483+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681c00 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.483+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 12 ==== osd_op_reply(50 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.483+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db680400 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.483+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 8 ==== osd_op_reply(51 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.483+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db669000 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.483+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.0 v2:172.21.15.139:6808/1167038101 5 ==== osd_op_reply(52 [pgnls start_epoch 93 out=149b] v92'29 uv29 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55f4db610500 con 0x55f4db669000 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.483+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4dac04000 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.484+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 11 ==== osd_op_reply(53 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.484+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db680400 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.484+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 9 ==== osd_op_reply(54 [pgnls start_epoch 93 out=73b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.484+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db680c00 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.485+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.5 v2:172.21.15.150:6824/2184499879 5 ==== osd_op_reply(55 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610000 con 0x55f4db680c00 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.485+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db669000 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.485+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.0 v2:172.21.15.139:6808/1167038101 6 ==== osd_op_reply(56 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4db669000 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.485+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db669400 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.485+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.2 v2:172.21.15.139:6800/1982336775 5 ==== osd_op_reply(57 [pgnls start_epoch 93 out=81b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f4db610000 con 0x55f4db669400 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.485+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4dac04000 2024-04-25T10:00:22.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.485+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 12 ==== osd_op_reply(58 [pgnls start_epoch 93 out=49b] v90'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.486+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681000 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.486+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 8 ==== osd_op_reply(59 [pgnls start_epoch 93 out=77b] v92'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.486+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4dac05800 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.486+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.3 v2:172.21.15.139:6824/1040931291 3 ==== osd_op_reply(60 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac05800 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.486+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681c00 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.487+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 13 ==== osd_op_reply(61 [pgnls start_epoch 93 out=73b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.487+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681000 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.487+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 9 ==== osd_op_reply(62 [pgnls start_epoch 93 out=49b] v90'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.487+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db680400 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.487+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 10 ==== osd_op_reply(63 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.487+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4dac04000 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.487+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 13 ==== osd_op_reply(64 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.487+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db680c00 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.488+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.5 v2:172.21.15.150:6824/2184499879 6 ==== osd_op_reply(65 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610000 con 0x55f4db680c00 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.488+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4dac05800 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.488+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.3 v2:172.21.15.139:6824/1040931291 4 ==== osd_op_reply(66 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac05800 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.488+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681000 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.488+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 10 ==== osd_op_reply(67 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.488+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681000 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.489+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 11 ==== osd_op_reply(68 [pgnls start_epoch 93 out=97b] v92'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.489+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681000 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.489+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 12 ==== osd_op_reply(69 [pgnls start_epoch 93 out=74b] v88'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.489+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681c00 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.489+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 14 ==== osd_op_reply(70 [pgnls start_epoch 93 out=73b] v92'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.489+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db669000 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.490+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.0 v2:172.21.15.139:6808/1167038101 7 ==== osd_op_reply(71 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610500 con 0x55f4db669000 2024-04-25T10:00:22.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.490+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681c00 2024-04-25T10:00:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.490+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 15 ==== osd_op_reply(72 [pgnls start_epoch 93 out=98b] v92'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.490+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db680c00 2024-04-25T10:00:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.490+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.5 v2:172.21.15.150:6824/2184499879 7 ==== osd_op_reply(73 [pgnls start_epoch 93 out=76b] v92'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55f4db610000 con 0x55f4db680c00 2024-04-25T10:00:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.490+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681000 2024-04-25T10:00:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.491+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 13 ==== osd_op_reply(74 [pgnls start_epoch 93 out=73b] v92'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.491+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681c00 2024-04-25T10:00:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.491+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 16 ==== osd_op_reply(75 [pgnls start_epoch 93 out=73b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.491+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db680c00 2024-04-25T10:00:22.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.492+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.5 v2:172.21.15.150:6824/2184499879 8 ==== osd_op_reply(76 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f4db610000 con 0x55f4db680c00 2024-04-25T10:00:22.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.492+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db680400 2024-04-25T10:00:22.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.492+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 11 ==== osd_op_reply(77 [pgnls start_epoch 93 out=73b] v88'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.492+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db680400 2024-04-25T10:00:22.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.492+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 12 ==== osd_op_reply(78 [pgnls start_epoch 93 out=101b] v92'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.492+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.492+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33400 con 0x55f4dac04000 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.492+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.492+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.492+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33800 con 0x55f4dac04000 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.493+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.493+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.493+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33c00 con 0x55f4dac04000 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.493+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.493+0000 7fdd64a1a740 20 datascan.scan_links: injected ino detected 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.493+0000 7fdd64a1a740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.493+0000 7fdd64a1a740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-25T10:00:22.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.493+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.493+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac32000 con 0x55f4dac04000 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.493+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.493+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.493+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac32400 con 0x55f4dac04000 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac32800 con 0x55f4db669000 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.0 v2:172.21.15.139:6808/1167038101 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv27 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610500 con 0x55f4db669000 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac04800 con 0x55f4db680400 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.494+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac04c00 con 0x55f4db669400 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.495+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.2 v2:172.21.15.139:6800/1982336775 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55f4db610000 con 0x55f4db669400 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.495+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.495+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.495+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.495+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db668c00 con 0x55f4db681c00 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.495+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.495+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T10:00:22.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.495+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db668800 con 0x55f4db681000 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac32c00 con 0x55f4db681c00 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33000 con 0x55f4db681c00 2024-04-25T10:00:22.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.496+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33400 con 0x55f4db681c00 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33800 con 0x55f4db680400 2024-04-25T10:00:22.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.6 v2:172.21.15.150:6809/3369628189 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1163 (crc 0 0 0) 0x55f4db610780 con 0x55f4db680400 2024-04-25T10:00:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-25T10:00:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:00:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-25T10:00:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-25T10:00:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac04400 con 0x55f4dac04000 2024-04-25T10:00:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:00:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.497+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac05000 con 0x55f4dac04000 2024-04-25T10:00:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.498+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v93'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.498+0000 7fdd64a1a740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:00:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.498+0000 7fdd64a1a740 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T10:00:22.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.498+0000 7fdd64a1a740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.498+0000 7fdd64a1a740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.498+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac04800 con 0x55f4dac04000 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.499+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.499+0000 7fdd64a1a740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.499+0000 7fdd64a1a740 20 datascan.scan_links: first is now 2 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.499+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac05400 con 0x55f4dac04000 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.500+0000 7fdd62e5a640 1 -- 172.21.15.139:0/2126967317 <== osd.1 v2:172.21.15.139:6814/315640110 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v93'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f4db610500 con 0x55f4dac04000 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.500+0000 7fdd64a1a740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.500+0000 7fdd64a1a740 10 datascan.scan_links: updating inotable 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.500+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac05c00 con 0x55f4db681c00 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.500+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.500+0000 7fdd64a1a740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.500+0000 7fdd64a1a740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.500+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4db681400 con 0x55f4db681c00 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.501+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 <== osd.7 v2:172.21.15.150:6808/2122423536 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v93'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55f4db610000 con 0x55f4db681c00 2024-04-25T10:00:22.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.501+0000 7fdd64a1a740 10 datascan.scan_links: updating snaptable 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.501+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f4dac33c00 con 0x55f4db681000 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd61e58640 1 -- 172.21.15.139:0/2126967317 <== osd.4 v2:172.21.15.150:6800/2810531857 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55f4db610780 con 0x55f4db681000 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55f4db669000 msgr2=0x55f4dac07600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55f4db669000 0x55f4dac07600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55f4dac04000 msgr2=0x55f4dac06000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55f4dac04000 0x55f4dac06000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x55f4db669400 msgr2=0x55f4dac06580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x55f4db669400 0x55f4dac06580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55f4dac05800 msgr2=0x55f4dac08100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55f4dac05800 0x55f4dac08100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 reap_dead start 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55f4db681000 msgr2=0x55f4dac08680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55f4db681000 0x55f4dac08680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55f4db680c00 msgr2=0x55f4dac07b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55f4db680c00 0x55f4dac07b80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55f4db680400 msgr2=0x55f4dac06b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55f4db680400 0x55f4dac06b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55f4db681c00 msgr2=0x55f4da902100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55f4db681c00 0x55f4da902100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd62659640 1 -- 172.21.15.139:0/2126967317 reap_dead start 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55f4db681800 msgr2=0x55f4da903700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55f4db681800 0x55f4da903700 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x55f4db662ab0 tx=0x55f4daa04060 comp rx=0 tx=0).stop 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b800 msgr2=0x55f4da902680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b800 0x55f4da902680 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x55f4db660180 tx=0x55f4db614fc0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 shutdown_connections 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4db75b800 0x55f4da902680 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/2126967317 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55f4db681800 0x55f4da903700 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 >> 172.21.15.139:0/2126967317 conn(0x55f4db75ac00 msgr2=0x55f4da988800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 shutdown_connections 2024-04-25T10:00:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/2126967317 wait complete. 2024-04-25T10:00:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/3214970471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4da891800 msgr2=0x55f4da900b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/3214970471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4da891800 0x55f4da900b00 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x55f4db6a7c80 tx=0x55f4daa042a0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/3214970471 shutdown_connections 2024-04-25T10:00:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/3214970471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f4da891800 0x55f4da900b00 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/3214970471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f4da891c00 0x55f4da901080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 --2- 172.21.15.139:0/3214970471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4da890c00 0x55f4da900580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.502+0000 7fdd64a1a740 1 -- 172.21.15.139:0/3214970471 >> 172.21.15.139:0/3214970471 conn(0x55f4da891000 msgr2=0x55f4da989000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.503+0000 7fdd64a1a740 1 -- 172.21.15.139:0/3214970471 shutdown_connections 2024-04-25T10:00:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.503+0000 7fdd64a1a740 1 -- 172.21.15.139:0/3214970471 wait complete. 2024-04-25T10:00:22.533 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.150986, result: 2024-04-25T10:00:22.533 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:00:22.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.658+0000 7f83dac15640 1 Processor -- start 2024-04-25T10:00:22.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.658+0000 7f83dac15640 1 -- start start 2024-04-25T10:00:22.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.659+0000 7f83dac15640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f83d40cee50 0x7f83d4168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.659+0000 7f83dac15640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d4161dc0 con 0x7f83d41617e0 2024-04-25T10:00:22.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.659+0000 7f83dac15640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d41620a0 con 0x7f83d415dbf0 2024-04-25T10:00:22.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.659+0000 7f83dac15640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d40d1ee0 con 0x7f83d40cee50 2024-04-25T10:00:22.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.659+0000 7f83d3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f83d41617e0 0x7f83d40ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41634/0 (socket says 172.21.15.139:41634) 2024-04-25T10:00:22.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.659+0000 7f83d3fff640 1 -- 172.21.15.139:0/4227684992 learned_addr learned my addr 172.21.15.139:0/4227684992 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:22.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910520121 0 0) 0x7f83d4161dc0 con 0x7f83d41617e0 2024-04-25T10:00:22.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d91b2640 1 --2- 172.21.15.139:0/4227684992 >> v2:172.21.15.150:3300/0 conn(0x7f83d40cee50 0x7f83d4168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83bc003520 con 0x7f83d41617e0 2024-04-25T10:00:22.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 55702176 0 0) 0x7f83bc003520 con 0x7f83d41617e0 2024-04-25T10:00:22.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 >> v2:172.21.15.150:3300/0 conn(0x7f83d40cee50 msgr2=0x7f83d4168b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 --2- 172.21.15.139:0/4227684992 >> v2:172.21.15.150:3300/0 conn(0x7f83d40cee50 0x7f83d4168b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 >> v1:172.21.15.150:6789/0 conn(0x7f83d415dbf0 legacy=0x7f83d40b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d4162380 con 0x7f83d41617e0 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f83c8001670 con 0x7f83d41617e0 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 >> v1:172.21.15.139:6789/0 conn(0x7f83d41617e0 legacy=0x7f83d40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 --2- 172.21.15.139:0/4227684992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83bc0039b0 0x7f83bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 --2- 172.21.15.139:0/4227684992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 0x7f83bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 --2- 172.21.15.139:0/4227684992 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83bc0051c0 0x7f83bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40d1ee0 con 0x7f83bc004800 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc00c160 con 0x7f83bc0039b0 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc00c360 con 0x7f83bc0051c0 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d3fff640 1 --2- 172.21.15.139:0/4227684992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 0x7f83bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d89b1640 1 --2- 172.21.15.139:0/4227684992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83bc0039b0 0x7f83bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d91b2640 1 --2- 172.21.15.139:0/4227684992 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83bc0051c0 0x7f83bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d3fff640 1 -- 172.21.15.139:0/4227684992 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83bc0051c0 msgr2=0x7f83bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d3fff640 1 --2- 172.21.15.139:0/4227684992 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83bc0051c0 0x7f83bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d3fff640 1 -- 172.21.15.139:0/4227684992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83bc0039b0 msgr2=0x7f83bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d3fff640 1 --2- 172.21.15.139:0/4227684992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83bc0039b0 0x7f83bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d3fff640 1 -- 172.21.15.139:0/4227684992 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc003520 con 0x7f83bc004800 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d91b2640 1 --2- 172.21.15.139:0/4227684992 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83bc0051c0 0x7f83bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.660+0000 7f83d91b2640 1 -- 172.21.15.139:0/4227684992 reap_dead start 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.661+0000 7f83d3fff640 1 --2- 172.21.15.139:0/4227684992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 0x7f83bc004c80 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f83c800c240 tx=0x7f83bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c234f8373fe642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.661+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8007cf0 con 0x7f83bc004800 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.661+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c8005af0 con 0x7f83bc004800 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.661+0000 7f83d37fe640 1 -- 172.21.15.139:0/4227684992 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8007070 con 0x7f83bc004800 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.662+0000 7f83dac15640 1 -- 172.21.15.139:0/4227684992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 msgr2=0x7f83bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.662+0000 7f83dac15640 1 --2- 172.21.15.139:0/4227684992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 0x7f83bc004c80 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f83c800c240 tx=0x7f83bc005830 comp rx=0 tx=0).stop 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.662+0000 7f83dac15640 1 -- 172.21.15.139:0/4227684992 shutdown_connections 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.662+0000 7f83dac15640 1 --2- 172.21.15.139:0/4227684992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 0x7f83bc004c80 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.662+0000 7f83dac15640 1 -- 172.21.15.139:0/4227684992 >> 172.21.15.139:0/4227684992 conn(0x7f83d40b5480 msgr2=0x7f83d40b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.662+0000 7f83dac15640 1 -- 172.21.15.139:0/4227684992 shutdown_connections 2024-04-25T10:00:22.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.662+0000 7f83dac15640 1 -- 172.21.15.139:0/4227684992 wait complete. 2024-04-25T10:00:22.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.663+0000 7f83dac15640 1 Processor -- start 2024-04-25T10:00:22.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.663+0000 7f83dac15640 1 -- start start 2024-04-25T10:00:22.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83dac15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 0x7f83d4159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83dac15640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d40cccc0 0x7f83d4151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83dac15640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83d40cd120 0x7f83d41517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83dac15640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d415dfd0 con 0x7f83bc004800 2024-04-25T10:00:22.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83dac15640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40b9370 con 0x7f83d40cd120 2024-04-25T10:00:22.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83dac15640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d415ec10 con 0x7f83d40cccc0 2024-04-25T10:00:22.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d89b1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 0x7f83d4159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d89b1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 0x7f83d4159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35698/0 (socket says 172.21.15.139:35698) 2024-04-25T10:00:22.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d89b1640 1 -- 172.21.15.139:0/2659501788 learned_addr learned my addr 172.21.15.139:0/2659501788 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:22.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d3fff640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d40cccc0 0x7f83d4151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d91b2640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83d40cd120 0x7f83d41517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d89b1640 1 -- 172.21.15.139:0/2659501788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d40cccc0 msgr2=0x7f83d4151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d89b1640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d40cccc0 0x7f83d4151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d89b1640 1 -- 172.21.15.139:0/2659501788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83d40cd120 msgr2=0x7f83d41517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d89b1640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83d40cd120 0x7f83d41517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d89b1640 1 -- 172.21.15.139:0/2659501788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d4151fd0 con 0x7f83bc004800 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d89b1640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 0x7f83d4159960 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f83c400f5f0 tx=0x7f83c400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d17fa640 1 -- 172.21.15.139:0/2659501788 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40052e0 con 0x7f83bc004800 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d17fa640 1 -- 172.21.15.139:0/2659501788 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c4018070 con 0x7f83bc004800 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83d17fa640 1 -- 172.21.15.139:0/2659501788 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4003810 con 0x7f83bc004800 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83dac15640 1 -- 172.21.15.139:0/2659501788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d415e2e0 con 0x7f83bc004800 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.664+0000 7f83dac15640 1 -- 172.21.15.139:0/2659501788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d415e8a0 con 0x7f83bc004800 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.665+0000 7f83dac15640 1 -- 172.21.15.139:0/2659501788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d4074520 con 0x7f83bc004800 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.666+0000 7f83d17fa640 1 -- 172.21.15.139:0/2659501788 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83c40039b0 con 0x7f83bc004800 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.666+0000 7f83d17fa640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f83c006b500 0x7f83c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.666+0000 7f83d17fa640 1 -- 172.21.15.139:0/2659501788 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f83c408a180 con 0x7f83bc004800 2024-04-25T10:00:22.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.666+0000 7f83d3fff640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f83c006b500 0x7f83c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.667+0000 7f83d3fff640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f83c006b500 0x7f83c006d9c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f83c8032000 tx=0x7f83c8009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.670+0000 7f83d17fa640 1 -- 172.21.15.139:0/2659501788 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83c4058670 con 0x7f83bc004800 2024-04-25T10:00:22.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:22.875+0000 7f83dac15640 1 -- 172.21.15.139:0/2659501788 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f83d40d1680 con 0x7f83bc004800 2024-04-25T10:00:23.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.121+0000 7f83d17fa640 1 -- 172.21.15.139:0/2659501788 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v59) v1 ==== 162+0+0 (secure 0 0 0) 0x7f83c405c710 con 0x7f83bc004800 2024-04-25T10:00:23.125 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:00:23.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.124+0000 7f83dac15640 1 -- 172.21.15.139:0/2659501788 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f83c006b500 msgr2=0x7f83c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.124+0000 7f83dac15640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f83c006b500 0x7f83c006d9c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f83c8032000 tx=0x7f83c8009030 comp rx=0 tx=0).stop 2024-04-25T10:00:23.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.124+0000 7f83dac15640 1 -- 172.21.15.139:0/2659501788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 msgr2=0x7f83d4159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.124+0000 7f83dac15640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 0x7f83d4159960 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f83c400f5f0 tx=0x7f83c400fab0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.124+0000 7f83dac15640 1 -- 172.21.15.139:0/2659501788 shutdown_connections 2024-04-25T10:00:23.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.124+0000 7f83dac15640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f83c006b500 0x7f83c006d9c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.124+0000 7f83dac15640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83d40cd120 0x7f83d41517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.124+0000 7f83dac15640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d40cccc0 0x7f83d4151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.124+0000 7f83dac15640 1 --2- 172.21.15.139:0/2659501788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83bc004800 0x7f83d4159960 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.124+0000 7f83dac15640 1 -- 172.21.15.139:0/2659501788 >> 172.21.15.139:0/2659501788 conn(0x7f83d40b5480 msgr2=0x7f83d40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:23.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.124+0000 7f83dac15640 1 -- 172.21.15.139:0/2659501788 shutdown_connections 2024-04-25T10:00:23.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.124+0000 7f83dac15640 1 -- 172.21.15.139:0/2659501788 wait complete. 2024-04-25T10:00:23.141 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.235+0000 7fa2ef329640 1 Processor -- start 2024-04-25T10:00:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.235+0000 7fa2ef329640 1 -- start start 2024-04-25T10:00:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.236+0000 7fa2ef329640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa2e815ba70 0x7fa2e815be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.236+0000 7fa2ef329640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2e80cbd20 0x7fa2e815fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.236+0000 7fa2ef329640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e815f420 con 0x7fa2e8160140 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.236+0000 7fa2ef329640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e815c710 con 0x7fa2e80cbd20 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.236+0000 7fa2ef329640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e815c9f0 con 0x7fa2e815ba70 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.236+0000 7fa2ec8c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2e80cbd20 0x7fa2e815fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.236+0000 7fa2ed8c6640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa2e8160140 0x7fa2e80cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41638/0 (socket says 172.21.15.139:41638) 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.236+0000 7fa2ed8c6640 1 -- 172.21.15.139:0/1608193784 learned_addr learned my addr 172.21.15.139:0/1608193784 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.237+0000 7fa2ed0c5640 1 --2- 172.21.15.139:0/1608193784 >> v2:172.21.15.150:3300/0 conn(0x7fa2e815ba70 0x7fa2e815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.237+0000 7fa2ec8c4640 1 -- 172.21.15.139:0/1608193784 >> v2:172.21.15.150:3300/0 conn(0x7fa2e815ba70 msgr2=0x7fa2e815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.237+0000 7fa2ec8c4640 1 --2- 172.21.15.139:0/1608193784 >> v2:172.21.15.150:3300/0 conn(0x7fa2e815ba70 0x7fa2e815be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.237+0000 7fa2ec8c4640 1 -- 172.21.15.139:0/1608193784 >> v1:172.21.15.139:6789/0 conn(0x7fa2e8160140 legacy=0x7fa2e80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.237+0000 7fa2ec8c4640 1 -- 172.21.15.139:0/1608193784 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e815ccd0 con 0x7fa2e80cbd20 2024-04-25T10:00:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.237+0000 7fa2dffff640 1 -- 172.21.15.139:0/1608193784 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361176453 0 0) 0x7fa2e815f420 con 0x7fa2e8160140 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.237+0000 7fa2ed0c5640 1 --2- 172.21.15.139:0/1608193784 >> v2:172.21.15.150:3300/0 conn(0x7fa2e815ba70 0x7fa2e815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.237+0000 7fa2ec8c4640 1 --2- 172.21.15.139:0/1608193784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80cbd20 0x7fa2e815fc00 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8072db0 tx=0x7fa2d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8385e90d0af95736 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.238+0000 7fa2dffff640 1 -- 172.21.15.139:0/1608193784 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8037070 con 0x7fa2e80cbd20 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.238+0000 7fa2dffff640 1 -- 172.21.15.139:0/1608193784 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d8032440 con 0x7fa2e80cbd20 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.238+0000 7fa2dffff640 1 -- 172.21.15.139:0/1608193784 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8032810 con 0x7fa2e80cbd20 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.238+0000 7fa2ef329640 1 -- 172.21.15.139:0/1608193784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80cbd20 msgr2=0x7fa2e815fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.238+0000 7fa2ef329640 1 --2- 172.21.15.139:0/1608193784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80cbd20 0x7fa2e815fc00 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8072db0 tx=0x7fa2d802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.238+0000 7fa2ef329640 1 -- 172.21.15.139:0/1608193784 shutdown_connections 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.238+0000 7fa2ef329640 1 --2- 172.21.15.139:0/1608193784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80cbd20 0x7fa2e815fc00 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.238+0000 7fa2ef329640 1 --2- 172.21.15.139:0/1608193784 >> v2:172.21.15.150:3300/0 conn(0x7fa2e815ba70 0x7fa2e815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.238+0000 7fa2ef329640 1 -- 172.21.15.139:0/1608193784 >> 172.21.15.139:0/1608193784 conn(0x7fa2e80b3380 msgr2=0x7fa2e80b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.238+0000 7fa2ef329640 1 -- 172.21.15.139:0/1608193784 shutdown_connections 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.238+0000 7fa2ef329640 1 -- 172.21.15.139:0/1608193784 wait complete. 2024-04-25T10:00:23.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.239+0000 7fa2ef329640 1 Processor -- start 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.239+0000 7fa2ef329640 1 -- start start 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ef329640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80cbd20 0x7fa2e81480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ef329640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2e815ba70 0x7fa2e81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ef329640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e8160140 0x7fa2e815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ef329640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80cfec0 con 0x7fa2e80cbd20 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ef329640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e815c390 con 0x7fa2e815ba70 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ef329640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80d0040 con 0x7fa2e8160140 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ed0c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80cbd20 0x7fa2e81480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ed0c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80cbd20 0x7fa2e81480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35710/0 (socket says 172.21.15.139:35710) 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ed0c5640 1 -- 172.21.15.139:0/3817662726 learned_addr learned my addr 172.21.15.139:0/3817662726 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ed0c5640 1 -- 172.21.15.139:0/3817662726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e8160140 msgr2=0x7fa2e815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ed0c5640 1 --2- 172.21.15.139:0/3817662726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e8160140 0x7fa2e815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ed0c5640 1 -- 172.21.15.139:0/3817662726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2e815ba70 msgr2=0x7fa2e81562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ed0c5640 1 --2- 172.21.15.139:0/3817662726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2e815ba70 0x7fa2e81562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.240+0000 7fa2ed0c5640 1 -- 172.21.15.139:0/3817662726 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e81f20c0 con 0x7fa2e80cbd20 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.241+0000 7fa2ed0c5640 1 --2- 172.21.15.139:0/3817662726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80cbd20 0x7fa2e81480f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400c7f0 tx=0x7fa2e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.241+0000 7fa2ddffb640 1 -- 172.21.15.139:0/3817662726 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4015070 con 0x7fa2e80cbd20 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.241+0000 7fa2ddffb640 1 -- 172.21.15.139:0/3817662726 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e4007a30 con 0x7fa2e80cbd20 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.241+0000 7fa2ef329640 1 -- 172.21.15.139:0/3817662726 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e81f23a0 con 0x7fa2e80cbd20 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.241+0000 7fa2ddffb640 1 -- 172.21.15.139:0/3817662726 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4007e00 con 0x7fa2e80cbd20 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.241+0000 7fa2ef329640 1 -- 172.21.15.139:0/3817662726 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e81f2960 con 0x7fa2e80cbd20 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.242+0000 7fa2ef329640 1 -- 172.21.15.139:0/3817662726 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e8074520 con 0x7fa2e80cbd20 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.242+0000 7fa2ddffb640 1 -- 172.21.15.139:0/3817662726 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa2e4018070 con 0x7fa2e80cbd20 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.243+0000 7fa2ddffb640 1 --2- 172.21.15.139:0/3817662726 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa2cc06b4d0 0x7fa2cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.243+0000 7fa2ddffb640 1 -- 172.21.15.139:0/3817662726 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fa2e408a450 con 0x7fa2e80cbd20 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.243+0000 7fa2ec8c4640 1 --2- 172.21.15.139:0/3817662726 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa2cc06b4d0 0x7fa2cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.243+0000 7fa2ec8c4640 1 --2- 172.21.15.139:0/3817662726 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa2cc06b4d0 0x7fa2cc06d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8034000 tx=0x7fa2d802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:23.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.247+0000 7fa2ddffb640 1 -- 172.21.15.139:0/3817662726 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2e4059030 con 0x7fa2e80cbd20 2024-04-25T10:00:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.458+0000 7fa2ef329640 1 -- 172.21.15.139:0/3817662726 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2e8158dc0 con 0x7fa2e80cbd20 2024-04-25T10:00:23.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.459+0000 7fa2ddffb640 1 -- 172.21.15.139:0/3817662726 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4953 (secure 0 0 0) 0x7fa2e4058930 con 0x7fa2e80cbd20 2024-04-25T10:00:23.464 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:00:23.464 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T10:00:23.125067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":60,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6836/1457343244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1457343244},{"type":"v1","addr":"172.21.15.139:6836","nonce":1457343244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:23.466 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 60 2024-04-25T10:00:23.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.461+0000 7fa2ef329640 1 -- 172.21.15.139:0/3817662726 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa2cc06b4d0 msgr2=0x7fa2cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.461+0000 7fa2ef329640 1 --2- 172.21.15.139:0/3817662726 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa2cc06b4d0 0x7fa2cc06d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8034000 tx=0x7fa2d802c000 comp rx=0 tx=0).stop 2024-04-25T10:00:23.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.461+0000 7fa2ef329640 1 -- 172.21.15.139:0/3817662726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80cbd20 msgr2=0x7fa2e81480f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.461+0000 7fa2ef329640 1 --2- 172.21.15.139:0/3817662726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80cbd20 0x7fa2e81480f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400c7f0 tx=0x7fa2e400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.461+0000 7fa2ef329640 1 -- 172.21.15.139:0/3817662726 shutdown_connections 2024-04-25T10:00:23.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.461+0000 7fa2ef329640 1 --2- 172.21.15.139:0/3817662726 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa2cc06b4d0 0x7fa2cc06d990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.461+0000 7fa2ef329640 1 --2- 172.21.15.139:0/3817662726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2e8160140 0x7fa2e815a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.461+0000 7fa2ef329640 1 --2- 172.21.15.139:0/3817662726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2e815ba70 0x7fa2e81562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.461+0000 7fa2ef329640 1 --2- 172.21.15.139:0/3817662726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80cbd20 0x7fa2e81480f0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.461+0000 7fa2ef329640 1 -- 172.21.15.139:0/3817662726 >> 172.21.15.139:0/3817662726 conn(0x7fa2e80b3380 msgr2=0x7fa2e80d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:23.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.461+0000 7fa2ef329640 1 -- 172.21.15.139:0/3817662726 shutdown_connections 2024-04-25T10:00:23.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:23.461+0000 7fa2ef329640 1 -- 172.21.15.139:0/3817662726 wait complete. 2024-04-25T10:00:23.881 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:57.374595+0000', 'modified': '2024-04-25T10:00:23.125067+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8094}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8094': {'gid': 8094, 'name': 'a', 'rank': 0, 'incarnation': 60, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.139:6836/1457343244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1457343244}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 1457343244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:00:23.881 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8094:up:replay 2024-04-25T10:00:24.883 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:24.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.987+0000 7f8a124a3640 1 Processor -- start 2024-04-25T10:00:24.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.987+0000 7f8a124a3640 1 -- start start 2024-04-25T10:00:24.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.988+0000 7f8a124a3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8a0c15efb0 0x7f8a0c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.988+0000 7f8a124a3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0cfe70 con 0x7f8a0c0cf900 2024-04-25T10:00:24.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.988+0000 7f8a124a3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0d0150 con 0x7f8a0c099a50 2024-04-25T10:00:24.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.988+0000 7f8a124a3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c0d0a80 con 0x7f8a0c15efb0 2024-04-25T10:00:24.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.988+0000 7f8a0bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8a0c15efb0 0x7f8a0c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.988+0000 7f8a0b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8a0c0cf900 0x7f8a0c06c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41650/0 (socket says 172.21.15.139:41650) 2024-04-25T10:00:24.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.988+0000 7f8a0b7fe640 1 -- 172.21.15.139:0/1881587527 learned_addr learned my addr 172.21.15.139:0/1881587527 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:24.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.988+0000 7f8a0bfff640 1 -- 172.21.15.139:0/1881587527 >> v1:172.21.15.150:6789/0 conn(0x7f8a0c099a50 legacy=0x7f8a0c0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:24.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.988+0000 7f8a0bfff640 1 -- 172.21.15.139:0/1881587527 >> v1:172.21.15.139:6789/0 conn(0x7f8a0c0cf900 legacy=0x7f8a0c06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.988+0000 7f8a0bfff640 1 -- 172.21.15.139:0/1881587527 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0d0430 con 0x7f8a0c15efb0 2024-04-25T10:00:24.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.988+0000 7f8a0affd640 1 -- 172.21.15.139:0/1881587527 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539027794 0 0) 0x7f8a0c0cfe70 con 0x7f8a0c0cf900 2024-04-25T10:00:24.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.989+0000 7f8a0bfff640 1 --2- 172.21.15.139:0/1881587527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a0c15efb0 0x7f8a0c0d0f10 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0097a0 tx=0x7f89fc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d21cb4196c9a1be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:24.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.989+0000 7f8a0affd640 1 -- 172.21.15.139:0/1881587527 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc037070 con 0x7f8a0c15efb0 2024-04-25T10:00:24.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.989+0000 7f8a0affd640 1 -- 172.21.15.139:0/1881587527 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc02ae50 con 0x7f8a0c15efb0 2024-04-25T10:00:24.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.989+0000 7f8a0affd640 1 -- 172.21.15.139:0/1881587527 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc032660 con 0x7f8a0c15efb0 2024-04-25T10:00:24.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.989+0000 7f8a124a3640 1 -- 172.21.15.139:0/1881587527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a0c15efb0 msgr2=0x7f8a0c0d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.989+0000 7f8a124a3640 1 --2- 172.21.15.139:0/1881587527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a0c15efb0 0x7f8a0c0d0f10 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0097a0 tx=0x7f89fc02a140 comp rx=0 tx=0).stop 2024-04-25T10:00:24.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.990+0000 7f8a124a3640 1 -- 172.21.15.139:0/1881587527 shutdown_connections 2024-04-25T10:00:24.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.990+0000 7f8a124a3640 1 --2- 172.21.15.139:0/1881587527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a0c15efb0 0x7f8a0c0d0f10 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.990+0000 7f8a124a3640 1 -- 172.21.15.139:0/1881587527 >> 172.21.15.139:0/1881587527 conn(0x7f8a0c0d1790 msgr2=0x7f8a0c0d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:24.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.990+0000 7f8a124a3640 1 -- 172.21.15.139:0/1881587527 shutdown_connections 2024-04-25T10:00:24.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.990+0000 7f8a124a3640 1 -- 172.21.15.139:0/1881587527 wait complete. 2024-04-25T10:00:24.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.991+0000 7f8a124a3640 1 Processor -- start 2024-04-25T10:00:24.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.991+0000 7f8a124a3640 1 -- start start 2024-04-25T10:00:24.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a124a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a0c099a50 0x7f8a0c0b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a124a3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a0c0cf900 0x7f8a0c158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a124a3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a0c15efb0 0x7f8a0c156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a124a3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c099e80 con 0x7f8a0c099a50 2024-04-25T10:00:24.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a124a3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c06cf50 con 0x7f8a0c15efb0 2024-04-25T10:00:24.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a124a3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c09a000 con 0x7f8a0c0cf900 2024-04-25T10:00:24.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a0bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a0c099a50 0x7f8a0c0b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a0bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a0c099a50 0x7f8a0c0b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35722/0 (socket says 172.21.15.139:35722) 2024-04-25T10:00:24.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a0bfff640 1 -- 172.21.15.139:0/666095344 learned_addr learned my addr 172.21.15.139:0/666095344 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:24.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a10a40640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a0c15efb0 0x7f8a0c156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a0b7fe640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a0c0cf900 0x7f8a0c158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a0bfff640 1 -- 172.21.15.139:0/666095344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a0c0cf900 msgr2=0x7f8a0c158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a0bfff640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a0c0cf900 0x7f8a0c158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a0bfff640 1 -- 172.21.15.139:0/666095344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a0c15efb0 msgr2=0x7f8a0c156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a0bfff640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a0c15efb0 0x7f8a0c156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a0bfff640 1 -- 172.21.15.139:0/666095344 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0c53f0 con 0x7f8a0c099a50 2024-04-25T10:00:24.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a0bfff640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a0c099a50 0x7f8a0c0b4a60 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0098c0 tx=0x7f89fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:24.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.992+0000 7f8a08ff9640 1 -- 172.21.15.139:0/666095344 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc042070 con 0x7f8a0c099a50 2024-04-25T10:00:24.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.993+0000 7f8a08ff9640 1 -- 172.21.15.139:0/666095344 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc03e670 con 0x7f8a0c099a50 2024-04-25T10:00:24.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.993+0000 7f8a124a3640 1 -- 172.21.15.139:0/666095344 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c0c56d0 con 0x7f8a0c099a50 2024-04-25T10:00:24.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.993+0000 7f8a08ff9640 1 -- 172.21.15.139:0/666095344 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc037040 con 0x7f8a0c099a50 2024-04-25T10:00:24.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.993+0000 7f8a124a3640 1 -- 172.21.15.139:0/666095344 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c0c5c90 con 0x7f8a0c099a50 2024-04-25T10:00:24.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.994+0000 7f8a124a3640 1 -- 172.21.15.139:0/666095344 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c074520 con 0x7f8a0c099a50 2024-04-25T10:00:25.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.994+0000 7f8a08ff9640 1 -- 172.21.15.139:0/666095344 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89fc045070 con 0x7f8a0c099a50 2024-04-25T10:00:25.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.995+0000 7f8a08ff9640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f89f806b4d0 0x7f89f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.995+0000 7f8a0b7fe640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f89f806b4d0 0x7f89f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.995+0000 7f8a08ff9640 1 -- 172.21.15.139:0/666095344 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f89fc0aaac0 con 0x7f8a0c099a50 2024-04-25T10:00:25.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.995+0000 7f8a0b7fe640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f89f806b4d0 0x7f89f806d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c1f7560 tx=0x7f8a00007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:25.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:24.999+0000 7f8a08ff9640 1 -- 172.21.15.139:0/666095344 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89fc079720 con 0x7f8a0c099a50 2024-04-25T10:00:25.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.207+0000 7f8a124a3640 1 -- 172.21.15.139:0/666095344 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a0c09ae50 con 0x7f8a0c099a50 2024-04-25T10:00:25.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.207+0000 7f8a08ff9640 1 -- 172.21.15.139:0/666095344 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f89fc079020 con 0x7f8a0c099a50 2024-04-25T10:00:25.212 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:00:25.212 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T10:00:24.130115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":60,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.139:6836/1457343244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1457343244},{"type":"v1","addr":"172.21.15.139:6836","nonce":1457343244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:25.212 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 62 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.209+0000 7f8a124a3640 1 -- 172.21.15.139:0/666095344 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f89f806b4d0 msgr2=0x7f89f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.209+0000 7f8a124a3640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f89f806b4d0 0x7f89f806d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c1f7560 tx=0x7f8a00007b50 comp rx=0 tx=0).stop 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.209+0000 7f8a124a3640 1 -- 172.21.15.139:0/666095344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a0c099a50 msgr2=0x7f8a0c0b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.209+0000 7f8a124a3640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a0c099a50 0x7f8a0c0b4a60 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0098c0 tx=0x7f89fc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.209+0000 7f8a124a3640 1 -- 172.21.15.139:0/666095344 shutdown_connections 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.209+0000 7f8a124a3640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f89f806b4d0 0x7f89f806d990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.209+0000 7f8a124a3640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a0c15efb0 0x7f8a0c156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.209+0000 7f8a124a3640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a0c0cf900 0x7f8a0c158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.209+0000 7f8a124a3640 1 --2- 172.21.15.139:0/666095344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a0c099a50 0x7f8a0c0b4a60 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.209+0000 7f8a124a3640 1 -- 172.21.15.139:0/666095344 >> 172.21.15.139:0/666095344 conn(0x7f8a0c0d1790 msgr2=0x7f8a0c06fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.210+0000 7f8a124a3640 1 -- 172.21.15.139:0/666095344 shutdown_connections 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:25.210+0000 7f8a124a3640 1 -- 172.21.15.139:0/666095344 wait complete. 2024-04-25T10:00:25.627 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:57.374595+0000', 'modified': '2024-04-25T10:00:24.130115+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8094}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8094': {'gid': 8094, 'name': 'a', 'rank': 0, 'incarnation': 60, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.139:6836/1457343244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1457343244}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 1457343244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:00:25.628 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8094:up:rejoin 2024-04-25T10:00:26.629 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:26.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.733+0000 7f43761a9640 1 Processor -- start 2024-04-25T10:00:26.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.734+0000 7f43761a9640 1 -- start start 2024-04-25T10:00:26.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.734+0000 7f43761a9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f43700d1080 0x7f43700b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.734+0000 7f43761a9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4370161850 0x7f43700ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.734+0000 7f43761a9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43700cf250 con 0x7f437015dc60 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.734+0000 7f43761a9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43700ce780 con 0x7f43700d1080 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.734+0000 7f43761a9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43700d3730 con 0x7f4370161850 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.734+0000 7f436f7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f437015dc60 0x7f43700d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57006/0 (socket says 172.21.15.139:57006) 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.734+0000 7f436effd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f43700d1080 0x7f43700b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.735+0000 7f436f7fe640 1 -- 172.21.15.139:0/3913139249 learned_addr learned my addr 172.21.15.139:0/3913139249 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.735+0000 7f436e7fc640 1 -- 172.21.15.139:0/3913139249 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622615443 0 0) 0x7f4360001110 con 0x7f437015dc60 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.735+0000 7f436e7fc640 1 -- 172.21.15.139:0/3913139249 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4358003520 con 0x7f437015dc60 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.735+0000 7f436ffff640 1 --2- 172.21.15.139:0/3913139249 >> v2:172.21.15.150:3300/0 conn(0x7f4370161850 0x7f43700ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.735+0000 7f436effd640 1 -- 172.21.15.139:0/3913139249 >> v2:172.21.15.150:3300/0 conn(0x7f4370161850 msgr2=0x7f43700ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.735+0000 7f436effd640 1 --2- 172.21.15.139:0/3913139249 >> v2:172.21.15.150:3300/0 conn(0x7f4370161850 0x7f43700ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.735+0000 7f436effd640 1 -- 172.21.15.139:0/3913139249 >> v1:172.21.15.139:6789/0 conn(0x7f437015dc60 legacy=0x7f43700d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.735+0000 7f436effd640 1 -- 172.21.15.139:0/3913139249 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700ce9b0 con 0x7f43700d1080 2024-04-25T10:00:26.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.736+0000 7f436effd640 1 --2- 172.21.15.139:0/3913139249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43700d1080 0x7f43700b6c10 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f43700b5480 tx=0x7f436402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab552b5a1be7eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.736+0000 7f436e7fc640 1 -- 172.21.15.139:0/3913139249 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364037070 con 0x7f43700d1080 2024-04-25T10:00:26.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.736+0000 7f436e7fc640 1 -- 172.21.15.139:0/3913139249 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4364032440 con 0x7f43700d1080 2024-04-25T10:00:26.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.736+0000 7f436e7fc640 1 -- 172.21.15.139:0/3913139249 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364032810 con 0x7f43700d1080 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.736+0000 7f43761a9640 1 -- 172.21.15.139:0/3913139249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43700d1080 msgr2=0x7f43700b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.736+0000 7f43761a9640 1 --2- 172.21.15.139:0/3913139249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43700d1080 0x7f43700b6c10 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f43700b5480 tx=0x7f436402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.736+0000 7f43761a9640 1 -- 172.21.15.139:0/3913139249 shutdown_connections 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.736+0000 7f43761a9640 1 --2- 172.21.15.139:0/3913139249 >> v2:172.21.15.150:3300/0 conn(0x7f4370161850 0x7f43700ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.736+0000 7f43761a9640 1 --2- 172.21.15.139:0/3913139249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43700d1080 0x7f43700b6c10 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.736+0000 7f43761a9640 1 -- 172.21.15.139:0/3913139249 >> 172.21.15.139:0/3913139249 conn(0x7f43700b5910 msgr2=0x7f43700b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.736+0000 7f43761a9640 1 -- 172.21.15.139:0/3913139249 shutdown_connections 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.736+0000 7f43761a9640 1 -- 172.21.15.139:0/3913139249 wait complete. 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.737+0000 7f43761a9640 1 Processor -- start 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.738+0000 7f43761a9640 1 -- start start 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.738+0000 7f43761a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43700d1080 0x7f437015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.738+0000 7f43761a9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437015dc60 0x7f43701482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.738+0000 7f43761a9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4370161850 0x7f43701605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.738+0000 7f43761a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43700d1fc0 con 0x7f43700d1080 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.738+0000 7f43761a9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43700cef70 con 0x7f437015dc60 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.738+0000 7f43761a9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437015e040 con 0x7f4370161850 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.738+0000 7f436f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43700d1080 0x7f437015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f436f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43700d1080 0x7f437015ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56916/0 (socket says 172.21.15.139:56916) 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f436f7fe640 1 -- 172.21.15.139:0/26315415 learned_addr learned my addr 172.21.15.139:0/26315415 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f436effd640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437015dc60 0x7f43701482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f436ffff640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4370161850 0x7f43701605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f436f7fe640 1 -- 172.21.15.139:0/26315415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4370161850 msgr2=0x7f43701605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f436f7fe640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4370161850 0x7f43701605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f436f7fe640 1 -- 172.21.15.139:0/26315415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437015dc60 msgr2=0x7f43701482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f436f7fe640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437015dc60 0x7f43701482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f436f7fe640 1 -- 172.21.15.139:0/26315415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43701530f0 con 0x7f43700d1080 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f436effd640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437015dc60 0x7f43701482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f436f7fe640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43700d1080 0x7f437015ec90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f436000c760 tx=0x7f436000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f4357fff640 1 -- 172.21.15.139:0/26315415 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360017070 con 0x7f43700d1080 2024-04-25T10:00:26.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f4357fff640 1 -- 172.21.15.139:0/26315415 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4360005ad0 con 0x7f43700d1080 2024-04-25T10:00:26.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f43761a9640 1 -- 172.21.15.139:0/26315415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43700c86c0 con 0x7f43700d1080 2024-04-25T10:00:26.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f4357fff640 1 -- 172.21.15.139:0/26315415 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360007e20 con 0x7f43700d1080 2024-04-25T10:00:26.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.739+0000 7f43761a9640 1 -- 172.21.15.139:0/26315415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43700c8c80 con 0x7f43700d1080 2024-04-25T10:00:26.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.740+0000 7f43761a9640 1 -- 172.21.15.139:0/26315415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4370074520 con 0x7f43700d1080 2024-04-25T10:00:26.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.741+0000 7f4357fff640 1 -- 172.21.15.139:0/26315415 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f436001a070 con 0x7f43700d1080 2024-04-25T10:00:26.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.741+0000 7f4357fff640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f435c06b500 0x7f435c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.741+0000 7f436effd640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f435c06b500 0x7f435c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.741+0000 7f4357fff640 1 -- 172.21.15.139:0/26315415 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f4360089830 con 0x7f43700d1080 2024-04-25T10:00:26.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.742+0000 7f436effd640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f435c06b500 0x7f435c06d9c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4364034000 tx=0x7f436402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.746+0000 7f4357fff640 1 -- 172.21.15.139:0/26315415 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43700cf250 con 0x7f43700d1080 2024-04-25T10:00:26.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.951+0000 7f43761a9640 1 -- 172.21.15.139:0/26315415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43700d16f0 con 0x7f43700d1080 2024-04-25T10:00:26.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.952+0000 7f4357fff640 1 -- 172.21.15.139:0/26315415 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f4360058400 con 0x7f43700d1080 2024-04-25T10:00:26.956 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:00:26.956 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T10:00:25.131250+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6836/1457343244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1457343244},{"type":"v1","addr":"172.21.15.139:6836","nonce":1457343244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:26.957 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 63 2024-04-25T10:00:26.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.954+0000 7f43761a9640 1 -- 172.21.15.139:0/26315415 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f435c06b500 msgr2=0x7f435c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.954+0000 7f43761a9640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f435c06b500 0x7f435c06d9c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4364034000 tx=0x7f436402e000 comp rx=0 tx=0).stop 2024-04-25T10:00:26.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.954+0000 7f43761a9640 1 -- 172.21.15.139:0/26315415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43700d1080 msgr2=0x7f437015ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.954+0000 7f43761a9640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43700d1080 0x7f437015ec90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f436000c760 tx=0x7f436000cc20 comp rx=0 tx=0).stop 2024-04-25T10:00:26.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.954+0000 7f43761a9640 1 -- 172.21.15.139:0/26315415 shutdown_connections 2024-04-25T10:00:26.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.954+0000 7f43761a9640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f435c06b500 0x7f435c06d9c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.954+0000 7f43761a9640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4370161850 0x7f43701605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.954+0000 7f43761a9640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437015dc60 0x7f43701482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.954+0000 7f43761a9640 1 --2- 172.21.15.139:0/26315415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43700d1080 0x7f437015ec90 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.954+0000 7f43761a9640 1 -- 172.21.15.139:0/26315415 >> 172.21.15.139:0/26315415 conn(0x7f43700b5910 msgr2=0x7f43700d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.954+0000 7f43761a9640 1 -- 172.21.15.139:0/26315415 shutdown_connections 2024-04-25T10:00:26.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:26.954+0000 7f43761a9640 1 -- 172.21.15.139:0/26315415 wait complete. 2024-04-25T10:00:27.372 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:57.374595+0000', 'modified': '2024-04-25T10:00:25.131250+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8094}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8094': {'gid': 8094, 'name': 'a', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.139:6836/1457343244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1457343244}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 1457343244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:00:27.372 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:00:27.372 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:00:27.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.468+0000 7f723109a640 1 Processor -- start 2024-04-25T10:00:27.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.468+0000 7f723109a640 1 -- start start 2024-04-25T10:00:27.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f723109a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f722c144090 0x7f722c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f723109a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c06ea40 0x7f722c15e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f723109a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c0b2a50 con 0x7f722c0cc9e0 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f723109a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722c15b090 con 0x7f722c144090 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f723109a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c15b370 con 0x7f722c06ea40 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f722ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f722c144090 0x7f722c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f722ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f722c144090 0x7f722c0b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56926/0 (socket says 172.21.15.139:56926) 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f722ad9d640 1 -- 172.21.15.139:0/944397931 learned_addr learned my addr 172.21.15.139:0/944397931 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f722b59e640 1 --2- 172.21.15.139:0/944397931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c06ea40 0x7f722c15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f722ad9d640 1 -- 172.21.15.139:0/944397931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c06ea40 msgr2=0x7f722c15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f722ad9d640 1 --2- 172.21.15.139:0/944397931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c06ea40 0x7f722c15e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f722ad9d640 1 -- 172.21.15.139:0/944397931 >> v1:172.21.15.150:6789/0 conn(0x7f722c0cc9e0 legacy=0x7f722c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.469+0000 7f722ad9d640 1 -- 172.21.15.139:0/944397931 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c15b650 con 0x7f722c144090 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.470+0000 7f722ad9d640 1 --2- 172.21.15.139:0/944397931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c144090 0x7f722c0b31a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f721c00bb10 tx=0x7f721c02c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3302db2535306a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.470+0000 7f7229d9b640 1 -- 172.21.15.139:0/944397931 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c03a070 con 0x7f722c144090 2024-04-25T10:00:27.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.470+0000 7f7229d9b640 1 -- 172.21.15.139:0/944397931 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c0326f0 con 0x7f722c144090 2024-04-25T10:00:27.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.470+0000 7f7229d9b640 1 -- 172.21.15.139:0/944397931 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c032ac0 con 0x7f722c144090 2024-04-25T10:00:27.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.470+0000 7f723109a640 1 -- 172.21.15.139:0/944397931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c144090 msgr2=0x7f722c0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.470+0000 7f723109a640 1 --2- 172.21.15.139:0/944397931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c144090 0x7f722c0b31a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f721c00bb10 tx=0x7f721c02c830 comp rx=0 tx=0).stop 2024-04-25T10:00:27.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.470+0000 7f723109a640 1 -- 172.21.15.139:0/944397931 shutdown_connections 2024-04-25T10:00:27.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.470+0000 7f723109a640 1 --2- 172.21.15.139:0/944397931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c06ea40 0x7f722c15e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.470+0000 7f723109a640 1 --2- 172.21.15.139:0/944397931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c144090 0x7f722c0b31a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.470+0000 7f723109a640 1 -- 172.21.15.139:0/944397931 >> 172.21.15.139:0/944397931 conn(0x7f722c067000 msgr2=0x7f722c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.470+0000 7f723109a640 1 -- 172.21.15.139:0/944397931 shutdown_connections 2024-04-25T10:00:27.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.470+0000 7f723109a640 1 -- 172.21.15.139:0/944397931 wait complete. 2024-04-25T10:00:27.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.472+0000 7f723109a640 1 Processor -- start 2024-04-25T10:00:27.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.472+0000 7f723109a640 1 -- start start 2024-04-25T10:00:27.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f723109a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c06ea40 0x7f722c1f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f723109a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0cc9e0 0x7f722c13fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f723109a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c144090 0x7f722c15e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f723109a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c144470 con 0x7f722c0cc9e0 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f723109a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c15bc10 con 0x7f722c144090 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f723109a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c15ad80 con 0x7f722c06ea40 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f722a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0cc9e0 0x7f722c13fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f722a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0cc9e0 0x7f722c13fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56928/0 (socket says 172.21.15.139:56928) 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f722a59c640 1 -- 172.21.15.139:0/3545718706 learned_addr learned my addr 172.21.15.139:0/3545718706 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f722b59e640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c144090 0x7f722c15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f722ad9d640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c06ea40 0x7f722c1f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f722a59c640 1 -- 172.21.15.139:0/3545718706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c06ea40 msgr2=0x7f722c1f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f722a59c640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c06ea40 0x7f722c1f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f722a59c640 1 -- 172.21.15.139:0/3545718706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c144090 msgr2=0x7f722c15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f722a59c640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c144090 0x7f722c15e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f722a59c640 1 -- 172.21.15.139:0/3545718706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c15ce60 con 0x7f722c0cc9e0 2024-04-25T10:00:27.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f722a59c640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0cc9e0 0x7f722c13fa60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f722000b420 tx=0x7f722000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f72137fe640 1 -- 172.21.15.139:0/3545718706 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7220011830 con 0x7f722c0cc9e0 2024-04-25T10:00:27.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f72137fe640 1 -- 172.21.15.139:0/3545718706 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7220016070 con 0x7f722c0cc9e0 2024-04-25T10:00:27.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f723109a640 1 -- 172.21.15.139:0/3545718706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722c15d140 con 0x7f722c0cc9e0 2024-04-25T10:00:27.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f72137fe640 1 -- 172.21.15.139:0/3545718706 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7220011c10 con 0x7f722c0cc9e0 2024-04-25T10:00:27.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.473+0000 7f723109a640 1 -- 172.21.15.139:0/3545718706 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722c15d700 con 0x7f722c0cc9e0 2024-04-25T10:00:27.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.475+0000 7f72137fe640 1 -- 172.21.15.139:0/3545718706 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f722001b460 con 0x7f722c0cc9e0 2024-04-25T10:00:27.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.475+0000 7f72137fe640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f721806b4d0 0x7f721806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.475+0000 7f72137fe640 1 -- 172.21.15.139:0/3545718706 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f722008a0d0 con 0x7f722c0cc9e0 2024-04-25T10:00:27.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.475+0000 7f722ad9d640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f721806b4d0 0x7f721806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.476+0000 7f722ad9d640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f721806b4d0 0x7f721806d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f721c037000 tx=0x7f721c032000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.476+0000 7f7233d96740 1 Processor -- start 2024-04-25T10:00:27.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f7233d96740 1 -- start start 2024-04-25T10:00:27.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f7233d96740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72a6230 0x5632f72a6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f7233d96740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 0x5632f72c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f7233d96740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f727bf10 0x5632f72c2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f7233d96740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f71dea10 con 0x5632f72c3ad0 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f7233d96740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f7187b30 con 0x5632f727bf10 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f7233d96740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632f71dac50 con 0x5632f72a6230 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f722a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 0x5632f72c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f722a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 0x5632f72c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56938/0 (socket says 172.21.15.139:56938) 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f722a59c640 1 -- 172.21.15.139:0/1997941594 learned_addr learned my addr 172.21.15.139:0/1997941594 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f722ad9d640 1 --2- 172.21.15.139:0/1997941594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72a6230 0x5632f72a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f722b59e640 1 --2- 172.21.15.139:0/1997941594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f727bf10 0x5632f72c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.477+0000 7f722a59c640 1 -- 172.21.15.139:0/1997941594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72a6230 msgr2=0x5632f72a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f722a59c640 1 --2- 172.21.15.139:0/1997941594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72a6230 0x5632f72a6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f722a59c640 1 -- 172.21.15.139:0/1997941594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f727bf10 msgr2=0x5632f72c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f722a59c640 1 --2- 172.21.15.139:0/1997941594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f727bf10 0x5632f72c2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f722a59c640 1 -- 172.21.15.139:0/1997941594 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632f719ce10 con 0x5632f72c3ad0 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f722a59c640 1 --2- 172.21.15.139:0/1997941594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 0x5632f72c4020 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f72200882f0 tx=0x7f7220058740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b5392d64a586f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f7210ff9640 1 -- 172.21.15.139:0/1997941594 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7220070070 con 0x5632f72c3ad0 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f7210ff9640 1 -- 172.21.15.139:0/1997941594 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7220054c30 con 0x5632f72c3ad0 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f7210ff9640 1 -- 172.21.15.139:0/1997941594 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722005c3b0 con 0x5632f72c3ad0 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f7233d96740 1 -- 172.21.15.139:0/1997941594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 msgr2=0x5632f72c4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f7233d96740 1 --2- 172.21.15.139:0/1997941594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 0x5632f72c4020 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f72200882f0 tx=0x7f7220058740 comp rx=0 tx=0).stop 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f7233d96740 1 -- 172.21.15.139:0/1997941594 shutdown_connections 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f7233d96740 1 --2- 172.21.15.139:0/1997941594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 0x5632f72c4020 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f7233d96740 1 --2- 172.21.15.139:0/1997941594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f727bf10 0x5632f72c2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f7233d96740 1 --2- 172.21.15.139:0/1997941594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72a6230 0x5632f72a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.478+0000 7f7233d96740 1 -- 172.21.15.139:0/1997941594 >> 172.21.15.139:0/1997941594 conn(0x5632f7278840 msgr2=0x5632f72c1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.479+0000 7f7233d96740 1 -- 172.21.15.139:0/1997941594 shutdown_connections 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.479+0000 7f7233d96740 1 -- 172.21.15.139:0/1997941594 wait complete. 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.479+0000 7f7233d96740 1 -- start start 2024-04-25T10:00:27.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.479+0000 7f7233d96740 1 Processor -- start 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f7233d96740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72a6230 0x5632f72a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f7233d96740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 0x5632f72a9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f7233d96740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 0x5632f72d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f7233d96740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f727ccb0 con 0x5632f72c3ad0 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f7233d96740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f719b0c0 con 0x5632f72a9aa0 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f7233d96740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632f7276770 con 0x5632f72a6230 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f7233d96740 10 client.0 _ll_register_callbacks cb 0x5632f718eec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f722a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 0x5632f72a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f722a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 0x5632f72a9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56950/0 (socket says 172.21.15.139:56950) 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f722a59c640 1 -- 172.21.15.139:0/1132042651 learned_addr learned my addr 172.21.15.139:0/1132042651 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f722b59e640 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 0x5632f72d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f722ad9d640 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72a6230 0x5632f72a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f722a59c640 1 -- 172.21.15.139:0/1132042651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72a6230 msgr2=0x5632f72a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f722a59c640 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72a6230 0x5632f72a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f722a59c640 1 -- 172.21.15.139:0/1132042651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 msgr2=0x5632f72d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f722a59c640 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 0x5632f72d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f722a59c640 1 -- 172.21.15.139:0/1132042651 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632f6ef32d0 con 0x5632f72c3ad0 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f722a59c640 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 0x5632f72a9560 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f72200883a0 tx=0x7f722000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e877a0f969f7f536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f7209ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.480+0000 7f7209ffb640 1 -- 172.21.15.139:0/1132042651 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7220070040 con 0x5632f72c3ad0 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7209ffb640 1 -- 172.21.15.139:0/1132042651 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f72200568c0 con 0x5632f72c3ad0 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7233d96740 1 -- 172.21.15.139:0/1132042651 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7220054c30 con 0x5632f72c3ad0 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7209ffb640 1 -- 172.21.15.139:0/1132042651 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7220077050 con 0x5632f72c3ad0 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7233d96740 10 client.8172 fetch_fsmap learned FSMap version 63 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7233d96740 1 -- 172.21.15.139:0/1132042651 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x5632f6fbd6b0 con 0x5632f72c3ad0 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7209ffb640 1 -- 172.21.15.139:0/1132042651 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7220054c30 con 0x5632f72c3ad0 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7209ffb640 1 -- 172.21.15.139:0/1132042651 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7220054310 con 0x5632f72c3ad0 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7233d96740 10 client.8172 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:00:27.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7233d96740 10 client.8172 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7233d96740 20 client.8172 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7233d96740 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x5632f72dccb0 0x5632f72df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7233d96740 4 client.8172 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7233d96740 1 -- 172.21.15.139:0/1132042651 --> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632f723d920 con 0x5632f72dccb0 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f722ad9d640 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x5632f72dccb0 0x5632f72df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f7209ffb640 10 client.8172 ms_handle_connect on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.481+0000 7f722ad9d640 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x5632f72dccb0 0x5632f72df0a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9daeeee2193b0a server_cookie=ae4860586a3be30b in_seq=0 out_seq=0 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7209ffb640 1 -- 172.21.15.139:0/1132042651 <== mds.0 v2:172.21.15.139:6834/1457343244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632f723d920 con 0x5632f72dccb0 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7209ffb640 10 client.8172 handle_command_reply: tid=0 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7233d96740 1 client.8172 shutdown 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7233d96740 1 -- 172.21.15.139:0/1132042651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 msgr2=0x5632f72a9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7233d96740 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 0x5632f72a9560 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f72200883a0 tx=0x7f722000bf90 comp rx=0 tx=0).stop 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7233d96740 1 -- 172.21.15.139:0/1132042651 shutdown_connections 2024-04-25T10:00:27.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7233d96740 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x5632f72dccb0 0x5632f72df0a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7233d96740 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72c3ad0 0x5632f72a9560 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7209ffb640 0 client.8172 ms_handle_reset on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:27.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7233d96740 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 0x5632f72d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7233d96740 1 --2- 172.21.15.139:0/1132042651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72a6230 0x5632f72a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7233d96740 1 -- 172.21.15.139:0/1132042651 >> 172.21.15.139:0/1132042651 conn(0x5632f727bf10 msgr2=0x5632f72c4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7233d96740 1 -- 172.21.15.139:0/1132042651 shutdown_connections 2024-04-25T10:00:27.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7233d96740 1 -- 172.21.15.139:0/1132042651 wait complete. 2024-04-25T10:00:27.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.482+0000 7f7233d96740 20 client.8172 trim_cache size 0 max 16384 2024-04-25T10:00:27.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.498+0000 7f7233d96740 1 Processor -- start 2024-04-25T10:00:27.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.498+0000 7f7233d96740 1 -- start start 2024-04-25T10:00:27.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.498+0000 7f7233d96740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 0x5632f73238c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.498+0000 7f7233d96740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 0x5632f72d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.498+0000 7f7233d96740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72c3ad0 0x5632f72d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.498+0000 7f7233d96740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c1ffd20 con 0x5632f72a6230 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.498+0000 7f7233d96740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c1f3e00 con 0x5632f72a9aa0 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.498+0000 7f7233d96740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c1f4400 con 0x5632f72c3ad0 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.498+0000 7f722ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 0x5632f73238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.498+0000 7f722ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 0x5632f73238c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56958/0 (socket says 172.21.15.139:56958) 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.498+0000 7f722ad9d640 1 -- 172.21.15.139:0/1006955187 learned_addr learned my addr 172.21.15.139:0/1006955187 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f722ad9d640 1 -- 172.21.15.139:0/1006955187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72c3ad0 msgr2=0x5632f72d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f722ad9d640 1 --2- 172.21.15.139:0/1006955187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72c3ad0 0x5632f72d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f722ad9d640 1 -- 172.21.15.139:0/1006955187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 msgr2=0x5632f72d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f722ad9d640 1 --2- 172.21.15.139:0/1006955187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 0x5632f72d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f722ad9d640 1 -- 172.21.15.139:0/1006955187 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632f72e1970 con 0x5632f72a6230 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f722ad9d640 1 --2- 172.21.15.139:0/1006955187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 0x5632f73238c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f721c00bb10 tx=0x7f721c031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54305b03948e7677 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f720bfff640 1 -- 172.21.15.139:0/1006955187 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c03a040 con 0x5632f72a6230 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f720bfff640 1 -- 172.21.15.139:0/1006955187 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c055070 con 0x5632f72a6230 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f720bfff640 1 -- 172.21.15.139:0/1006955187 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c035db0 con 0x5632f72a6230 2024-04-25T10:00:27.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f7233d96740 1 -- 172.21.15.139:0/1006955187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 msgr2=0x5632f73238c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f7233d96740 1 --2- 172.21.15.139:0/1006955187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 0x5632f73238c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f721c00bb10 tx=0x7f721c031000 comp rx=0 tx=0).stop 2024-04-25T10:00:27.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f7233d96740 1 -- 172.21.15.139:0/1006955187 shutdown_connections 2024-04-25T10:00:27.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f7233d96740 1 --2- 172.21.15.139:0/1006955187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72c3ad0 0x5632f72d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f7233d96740 1 --2- 172.21.15.139:0/1006955187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 0x5632f72d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f7233d96740 1 --2- 172.21.15.139:0/1006955187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 0x5632f73238c0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.499+0000 7f7233d96740 1 -- 172.21.15.139:0/1006955187 >> 172.21.15.139:0/1006955187 conn(0x5632f727bf10 msgr2=0x5632f7271bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.500+0000 7f7233d96740 1 -- 172.21.15.139:0/1006955187 shutdown_connections 2024-04-25T10:00:27.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.500+0000 7f7233d96740 1 -- 172.21.15.139:0/1006955187 wait complete. 2024-04-25T10:00:27.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.500+0000 7f7233d96740 1 -- start start 2024-04-25T10:00:27.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.500+0000 7f7233d96740 1 Processor -- start 2024-04-25T10:00:27.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f7233d96740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 0x5632f73227e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f7233d96740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 0x5632f72b7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f7233d96740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72c3ad0 0x5632f732ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f7233d96740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f727ccb0 con 0x5632f72a6230 2024-04-25T10:00:27.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f7233d96740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f7276770 con 0x5632f72a9aa0 2024-04-25T10:00:27.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f7233d96740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632f719b0c0 con 0x5632f72c3ad0 2024-04-25T10:00:27.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f7233d96740 10 client.0 _ll_register_callbacks cb 0x5632f7276c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:27.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f722ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 0x5632f73227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f722ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 0x5632f73227e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56960/0 (socket says 172.21.15.139:56960) 2024-04-25T10:00:27.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f722ad9d640 1 -- 172.21.15.139:0/1516168769 learned_addr learned my addr 172.21.15.139:0/1516168769 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:27.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f722a59c640 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 0x5632f72b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f722b59e640 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72c3ad0 0x5632f732ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f722ad9d640 1 -- 172.21.15.139:0/1516168769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72c3ad0 msgr2=0x5632f732ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f722ad9d640 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72c3ad0 0x5632f732ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f722ad9d640 1 -- 172.21.15.139:0/1516168769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 msgr2=0x5632f72b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f722ad9d640 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 0x5632f72b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f722ad9d640 1 -- 172.21.15.139:0/1516168769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632f72e0780 con 0x5632f72a6230 2024-04-25T10:00:27.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f722a59c640 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 0x5632f72b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:27.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f722ad9d640 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 0x5632f73227e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f721c02ce20 tx=0x7f721c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d9e15b074f060b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f7209ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:27.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.501+0000 7f7209ffb640 1 -- 172.21.15.139:0/1516168769 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c03a040 con 0x5632f72a6230 2024-04-25T10:00:27.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7209ffb640 1 -- 172.21.15.139:0/1516168769 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f721c035dd0 con 0x5632f72a6230 2024-04-25T10:00:27.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7233d96740 1 -- 172.21.15.139:0/1516168769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f721c055070 con 0x5632f72a6230 2024-04-25T10:00:27.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7209ffb640 1 -- 172.21.15.139:0/1516168769 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f721c050120 con 0x5632f72a6230 2024-04-25T10:00:27.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7233d96740 10 client.8178 fetch_fsmap learned FSMap version 63 2024-04-25T10:00:27.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7233d96740 1 -- 172.21.15.139:0/1516168769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x5632f72a5920 con 0x5632f72a6230 2024-04-25T10:00:27.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7209ffb640 1 -- 172.21.15.139:0/1516168769 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c055070 con 0x5632f72a6230 2024-04-25T10:00:27.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7209ffb640 1 -- 172.21.15.139:0/1516168769 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f721c069080 con 0x5632f72a6230 2024-04-25T10:00:27.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7233d96740 10 client.8178 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:00:27.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7233d96740 10 client.8178 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:00:27.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7233d96740 20 client.8178 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:27.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7233d96740 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x5632f727b670 0x5632f7279c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7233d96740 4 client.8178 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:00:27.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7233d96740 1 -- 172.21.15.139:0/1516168769 --> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5632f71ebf50 con 0x5632f727b670 2024-04-25T10:00:27.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f722a59c640 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x5632f727b670 0x5632f7279c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f7209ffb640 10 client.8178 ms_handle_connect on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:27.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.502+0000 7f722a59c640 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x5632f727b670 0x5632f7279c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=538319d7ba205cb3 server_cookie=b1e7b37bfdd51985 in_seq=0 out_seq=0 2024-04-25T10:00:27.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7209ffb640 1 -- 172.21.15.139:0/1516168769 <== mds.0 v2:172.21.15.139:6834/1457343244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5632f71ebf50 con 0x5632f727b670 2024-04-25T10:00:27.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7209ffb640 10 client.8178 handle_command_reply: tid=0 2024-04-25T10:00:27.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7233d96740 1 client.8178 shutdown 2024-04-25T10:00:27.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7233d96740 1 -- 172.21.15.139:0/1516168769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 msgr2=0x5632f73227e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7233d96740 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 0x5632f73227e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f721c02ce20 tx=0x7f721c004730 comp rx=0 tx=0).stop 2024-04-25T10:00:27.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7233d96740 1 -- 172.21.15.139:0/1516168769 shutdown_connections 2024-04-25T10:00:27.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7233d96740 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x5632f727b670 0x5632f7279c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7233d96740 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632f72c3ad0 0x5632f732ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7209ffb640 0 client.8178 ms_handle_reset on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:27.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7233d96740 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5632f72a9aa0 0x5632f72b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7233d96740 1 --2- 172.21.15.139:0/1516168769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632f72a6230 0x5632f73227e0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7233d96740 1 -- 172.21.15.139:0/1516168769 >> 172.21.15.139:0/1516168769 conn(0x5632f727bf10 msgr2=0x5632f7322f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7233d96740 1 -- 172.21.15.139:0/1516168769 shutdown_connections 2024-04-25T10:00:27.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7233d96740 1 -- 172.21.15.139:0/1516168769 wait complete. 2024-04-25T10:00:27.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f7233d96740 20 client.8178 trim_cache size 0 max 16384 2024-04-25T10:00:27.515 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:00:27.515 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T10:00:27.515 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T10:00:27.516 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 8094, 2024-04-25T10:00:27.516 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T10:00:27.516 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T10:00:27.516 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T10:00:27.516 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 4.3745665709999999, 2024-04-25T10:00:27.516 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 63, 2024-04-25T10:00:27.516 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 93, 2024-04-25T10:00:27.516 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 93, 2024-04-25T10:00:27.516 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 5.3483880099999999 2024-04-25T10:00:27.516 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:00:27.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f723109a640 1 -- 172.21.15.139:0/3545718706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f721806b4d0 msgr2=0x7f721806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f723109a640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f721806b4d0 0x7f721806d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f721c037000 tx=0x7f721c032000 comp rx=0 tx=0).stop 2024-04-25T10:00:27.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f723109a640 1 -- 172.21.15.139:0/3545718706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0cc9e0 msgr2=0x7f722c13fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f723109a640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0cc9e0 0x7f722c13fa60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f722000b420 tx=0x7f722000b8e0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f723109a640 1 -- 172.21.15.139:0/3545718706 shutdown_connections 2024-04-25T10:00:27.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f723109a640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f722c0cc9e0 0x7f722c13fa60 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f723109a640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f721806b4d0 0x7f721806d990 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f723109a640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f722c144090 0x7f722c15e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f723109a640 1 --2- 172.21.15.139:0/3545718706 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f722c06ea40 0x7f722c1f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f723109a640 1 -- 172.21.15.139:0/3545718706 >> 172.21.15.139:0/3545718706 conn(0x7f722c067000 msgr2=0x7f722c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f723109a640 1 -- 172.21.15.139:0/3545718706 shutdown_connections 2024-04-25T10:00:27.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.503+0000 7f723109a640 1 -- 172.21.15.139:0/3545718706 wait complete. 2024-04-25T10:00:27.519 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:00:27.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.612+0000 7efdbda88640 1 Processor -- start 2024-04-25T10:00:27.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.613+0000 7efdbda88640 1 -- start start 2024-04-25T10:00:27.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.613+0000 7efdbda88640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efdb8158240 0x7efdb80d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.613+0000 7efdbda88640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efdb80d1860 0x7efdb80cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.613+0000 7efdbda88640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb80cbf80 con 0x7efdb80cfba0 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.613+0000 7efdbda88640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb8158620 con 0x7efdb8158240 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.613+0000 7efdbda88640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb80cce10 con 0x7efdb80d1860 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.613+0000 7efdb77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efdb8158240 0x7efdb80d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.613+0000 7efdb77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efdb8158240 0x7efdb80d1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56966/0 (socket says 172.21.15.139:56966) 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.613+0000 7efdb77fe640 1 -- 172.21.15.139:0/3842231937 learned_addr learned my addr 172.21.15.139:0/3842231937 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.614+0000 7efdaffff640 1 --2- 172.21.15.139:0/3842231937 >> v2:172.21.15.150:3300/0 conn(0x7efdb80d1860 0x7efdb80cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.614+0000 7efdb77fe640 1 -- 172.21.15.139:0/3842231937 >> v2:172.21.15.150:3300/0 conn(0x7efdb80d1860 msgr2=0x7efdb80cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.614+0000 7efdb77fe640 1 --2- 172.21.15.139:0/3842231937 >> v2:172.21.15.150:3300/0 conn(0x7efdb80d1860 0x7efdb80cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.614+0000 7efdb77fe640 1 -- 172.21.15.139:0/3842231937 >> v1:172.21.15.150:6789/0 conn(0x7efdb80cfba0 legacy=0x7efdb81616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.614+0000 7efdb77fe640 1 -- 172.21.15.139:0/3842231937 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb80ceb50 con 0x7efdb8158240 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.614+0000 7efdb77fe640 1 --2- 172.21.15.139:0/3842231937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8158240 0x7efdb80d1f00 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7efdb80cf250 tx=0x7efda802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169b60720b448fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.615+0000 7efdb6ffd640 1 -- 172.21.15.139:0/3842231937 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8037070 con 0x7efdb8158240 2024-04-25T10:00:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.615+0000 7efdb6ffd640 1 -- 172.21.15.139:0/3842231937 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda8032440 con 0x7efdb8158240 2024-04-25T10:00:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.615+0000 7efdb6ffd640 1 -- 172.21.15.139:0/3842231937 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8032810 con 0x7efdb8158240 2024-04-25T10:00:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.615+0000 7efdbda88640 1 -- 172.21.15.139:0/3842231937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8158240 msgr2=0x7efdb80d1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.615+0000 7efdbda88640 1 --2- 172.21.15.139:0/3842231937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8158240 0x7efdb80d1f00 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7efdb80cf250 tx=0x7efda802a270 comp rx=0 tx=0).stop 2024-04-25T10:00:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.615+0000 7efdbda88640 1 -- 172.21.15.139:0/3842231937 shutdown_connections 2024-04-25T10:00:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.615+0000 7efdbda88640 1 --2- 172.21.15.139:0/3842231937 >> v2:172.21.15.150:3300/0 conn(0x7efdb80d1860 0x7efdb80cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.615+0000 7efdbda88640 1 --2- 172.21.15.139:0/3842231937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8158240 0x7efdb80d1f00 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.615+0000 7efdbda88640 1 -- 172.21.15.139:0/3842231937 >> 172.21.15.139:0/3842231937 conn(0x7efdb80b3380 msgr2=0x7efdb80b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.615+0000 7efdbda88640 1 -- 172.21.15.139:0/3842231937 shutdown_connections 2024-04-25T10:00:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.615+0000 7efdbda88640 1 -- 172.21.15.139:0/3842231937 wait complete. 2024-04-25T10:00:27.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.616+0000 7efdbda88640 1 Processor -- start 2024-04-25T10:00:27.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.616+0000 7efdbda88640 1 -- start start 2024-04-25T10:00:27.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.617+0000 7efdbda88640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdb80cfba0 0x7efdb815a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.617+0000 7efdbda88640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdb80d1860 0x7efdb8153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.617+0000 7efdbda88640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8158240 0x7efdb80b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.617+0000 7efdbda88640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80cc200 con 0x7efdb8158240 2024-04-25T10:00:27.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.617+0000 7efdbda88640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80cffd0 con 0x7efdb80d1860 2024-04-25T10:00:27.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.617+0000 7efdbda88640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80cc4e0 con 0x7efdb80cfba0 2024-04-25T10:00:27.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.617+0000 7efdb7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8158240 0x7efdb80b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.617+0000 7efdb7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8158240 0x7efdb80b20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56978/0 (socket says 172.21.15.139:56978) 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.617+0000 7efdb7fff640 1 -- 172.21.15.139:0/3468715350 learned_addr learned my addr 172.21.15.139:0/3468715350 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.617+0000 7efdaffff640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdb80d1860 0x7efdb8153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.617+0000 7efdb77fe640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdb80cfba0 0x7efdb815a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.618+0000 7efdb7fff640 1 -- 172.21.15.139:0/3468715350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdb80cfba0 msgr2=0x7efdb815a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.618+0000 7efdb7fff640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdb80cfba0 0x7efdb815a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.618+0000 7efdb7fff640 1 -- 172.21.15.139:0/3468715350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdb80d1860 msgr2=0x7efdb8153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.618+0000 7efdb7fff640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdb80d1860 0x7efdb8153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.618+0000 7efdb7fff640 1 -- 172.21.15.139:0/3468715350 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb80c9800 con 0x7efdb8158240 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.618+0000 7efdb7fff640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8158240 0x7efdb80b20d0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c7a0 tx=0x7efdb000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.618+0000 7efdb4ff9640 1 -- 172.21.15.139:0/3468715350 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0003b70 con 0x7efdb8158240 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.618+0000 7efdb4ff9640 1 -- 172.21.15.139:0/3468715350 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb0003d30 con 0x7efdb8158240 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.618+0000 7efdbda88640 1 -- 172.21.15.139:0/3468715350 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb80c8c80 con 0x7efdb8158240 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.618+0000 7efdb4ff9640 1 -- 172.21.15.139:0/3468715350 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb000aa90 con 0x7efdb8158240 2024-04-25T10:00:27.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.618+0000 7efdbda88640 1 -- 172.21.15.139:0/3468715350 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb80c9240 con 0x7efdb8158240 2024-04-25T10:00:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.619+0000 7efdbda88640 1 -- 172.21.15.139:0/3468715350 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb8074520 con 0x7efdb8158240 2024-04-25T10:00:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.619+0000 7efdb4ff9640 1 -- 172.21.15.139:0/3468715350 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efdb000ac30 con 0x7efdb8158240 2024-04-25T10:00:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.620+0000 7efdb4ff9640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd9806b4d0 0x7efd9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.620+0000 7efdb77fe640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd9806b4d0 0x7efd9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.620+0000 7efdb4ff9640 1 -- 172.21.15.139:0/3468715350 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7efdb008a2b0 con 0x7efdb8158240 2024-04-25T10:00:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.620+0000 7efdb77fe640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd9806b4d0 0x7efd9806d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efda8034000 tx=0x7efda802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.624+0000 7efdb4ff9640 1 -- 172.21.15.139:0/3468715350 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efdb80cbf80 con 0x7efdb8158240 2024-04-25T10:00:27.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.830+0000 7efdbda88640 1 -- 172.21.15.139:0/3468715350 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efdb80c95e0 con 0x7efdb8158240 2024-04-25T10:00:27.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.831+0000 7efdb4ff9640 1 -- 172.21.15.139:0/3468715350 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) v1 ==== 53+0+73 (secure 0 0 0) 0x7efdb0058830 con 0x7efdb8158240 2024-04-25T10:00:27.835 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:00:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.832+0000 7efdbda88640 1 -- 172.21.15.139:0/3468715350 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd9806b4d0 msgr2=0x7efd9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.832+0000 7efdbda88640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd9806b4d0 0x7efd9806d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efda8034000 tx=0x7efda802e000 comp rx=0 tx=0).stop 2024-04-25T10:00:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.832+0000 7efdbda88640 1 -- 172.21.15.139:0/3468715350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8158240 msgr2=0x7efdb80b20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.832+0000 7efdbda88640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8158240 0x7efdb80b20d0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c7a0 tx=0x7efdb000cc60 comp rx=0 tx=0).stop 2024-04-25T10:00:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.832+0000 7efdbda88640 1 -- 172.21.15.139:0/3468715350 shutdown_connections 2024-04-25T10:00:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.832+0000 7efdbda88640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efdb8158240 0x7efdb80b20d0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.833+0000 7efdbda88640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd9806b4d0 0x7efd9806d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.833+0000 7efdbda88640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdb80d1860 0x7efdb8153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.833+0000 7efdbda88640 1 --2- 172.21.15.139:0/3468715350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdb80cfba0 0x7efdb815a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.833+0000 7efdbda88640 1 -- 172.21.15.139:0/3468715350 >> 172.21.15.139:0/3468715350 conn(0x7efdb80b3380 msgr2=0x7efdb80d1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.833+0000 7efdbda88640 1 -- 172.21.15.139:0/3468715350 shutdown_connections 2024-04-25T10:00:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:27.833+0000 7efdbda88640 1 -- 172.21.15.139:0/3468715350 wait complete. 2024-04-25T10:00:27.847 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:00:27.847 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:00:27.847 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T10:00:27.848 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:00:27.848 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:27.848 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:00:27.848 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:00:27.848 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:00:27.848 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:00:27.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:27.848 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T10:00:27.863 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:00:27.863 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:00:27.863 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:00:27.863 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:27.863 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T10:00:27.863 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42271sec preferred_lft 42271sec 2024-04-25T10:00:27.864 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:00:27.865 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T10:00:27.866 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:27.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:27.866 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:00:27.926 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T10:00:27.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:27.928 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T10:00:27.984 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T10:00:27.985 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:00:27.985 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:00:27.985 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:00:27.985 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:00:27.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:27.985 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T10:00:27.985 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T10:00:27.985 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:00:27.985 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:00:27.986 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:00:27.986 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T10:00:28.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:28.069 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T10:00:28.125 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:00:28.125 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:00:28.125 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:00:28.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:28.125 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T10:00:28.133 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T10:00:28.129+0000 7fbdccb06640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:00:28.155 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:00:28.156 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:28.211 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:00:28.211 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:00:28.212 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:28.267 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:00:28.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:28.268 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:00:28.352 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:00:28.352 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:00:28.360 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:00:28.360 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:00:28.360 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T10:00:28.360 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T10:00:28.360 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T10:00:28.360 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:00:28.361 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:00:28.361 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:00:28.361 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:00:28.361 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:00:28.362 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:00:28.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:28.363 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:28.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:28.387 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:00:28.442 INFO:teuthology.orchestra.run.smithi139.stdout:client8187 2024-04-25T10:00:28.442 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:00:28.443 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:28.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:28.498 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T10:00:28.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.635+0000 7f01b0f68640 1 Processor -- start 2024-04-25T10:00:28.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.635+0000 7f01b0f68640 1 -- start start 2024-04-25T10:00:28.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.636+0000 7f01b0f68640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f01ac15fca0 0x7f01ac0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.636+0000 7f01b0f68640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f01ac0d0220 0x7f01ac15b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.636+0000 7f01b0f68640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ac099bf0 con 0x7f01ac15f8c0 2024-04-25T10:00:28.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.636+0000 7f01b0f68640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac0d06c0 con 0x7f01ac0d0220 2024-04-25T10:00:28.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.636+0000 7f01b0f68640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac099ed0 con 0x7f01ac15fca0 2024-04-25T10:00:28.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.636+0000 7f01aad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f01ac0d0220 0x7f01ac15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.636+0000 7f01aad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f01ac0d0220 0x7f01ac15b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56988/0 (socket says 172.21.15.139:56988) 2024-04-25T10:00:28.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.636+0000 7f01aad9d640 1 -- 172.21.15.139:0/2652826326 learned_addr learned my addr 172.21.15.139:0/2652826326 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:28.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.636+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/2652826326 >> v2:172.21.15.150:3300/0 conn(0x7f01ac15fca0 0x7f01ac0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.637+0000 7f01aad9d640 1 -- 172.21.15.139:0/2652826326 >> v2:172.21.15.150:3300/0 conn(0x7f01ac15fca0 msgr2=0x7f01ac0cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.637+0000 7f01aad9d640 1 --2- 172.21.15.139:0/2652826326 >> v2:172.21.15.150:3300/0 conn(0x7f01ac15fca0 0x7f01ac0cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.637+0000 7f01aad9d640 1 -- 172.21.15.139:0/2652826326 >> v1:172.21.15.150:6789/0 conn(0x7f01ac15f8c0 legacy=0x7f01ac09a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.637+0000 7f01aad9d640 1 -- 172.21.15.139:0/2652826326 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac09a1b0 con 0x7f01ac0d0220 2024-04-25T10:00:28.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.637+0000 7f01aad9d640 1 --2- 172.21.15.139:0/2652826326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01ac0d0220 0x7f01ac15b560 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009870 tx=0x7f01a402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebce3be4b251438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.637+0000 7f01a959a640 1 -- 172.21.15.139:0/2652826326 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4037070 con 0x7f01ac0d0220 2024-04-25T10:00:28.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.637+0000 7f01a959a640 1 -- 172.21.15.139:0/2652826326 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4032440 con 0x7f01ac0d0220 2024-04-25T10:00:28.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.637+0000 7f01a959a640 1 -- 172.21.15.139:0/2652826326 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4032810 con 0x7f01ac0d0220 2024-04-25T10:00:28.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.638+0000 7f01b0f68640 1 -- 172.21.15.139:0/2652826326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01ac0d0220 msgr2=0x7f01ac15b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.638+0000 7f01b0f68640 1 --2- 172.21.15.139:0/2652826326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01ac0d0220 0x7f01ac15b560 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009870 tx=0x7f01a402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.638+0000 7f01b0f68640 1 -- 172.21.15.139:0/2652826326 shutdown_connections 2024-04-25T10:00:28.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.638+0000 7f01b0f68640 1 --2- 172.21.15.139:0/2652826326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01ac0d0220 0x7f01ac15b560 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.638+0000 7f01b0f68640 1 --2- 172.21.15.139:0/2652826326 >> v2:172.21.15.150:3300/0 conn(0x7f01ac15fca0 0x7f01ac0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.638+0000 7f01b0f68640 1 -- 172.21.15.139:0/2652826326 >> 172.21.15.139:0/2652826326 conn(0x7f01ac0b31f0 msgr2=0x7f01ac0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.638+0000 7f01b0f68640 1 -- 172.21.15.139:0/2652826326 shutdown_connections 2024-04-25T10:00:28.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.638+0000 7f01b0f68640 1 -- 172.21.15.139:0/2652826326 wait complete. 2024-04-25T10:00:28.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.639+0000 7f01b0f68640 1 Processor -- start 2024-04-25T10:00:28.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.640+0000 7f01b0f68640 1 -- start start 2024-04-25T10:00:28.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.640+0000 7f01b0f68640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01ac0d0220 0x7f01ac1f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.640+0000 7f01b0f68640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac15f8c0 0x7f01ac154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.640+0000 7f01b0f68640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01ac15fca0 0x7f01ac0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.640+0000 7f01b0f68640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac160450 con 0x7f01ac15fca0 2024-04-25T10:00:28.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.640+0000 7f01b0f68640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac1602d0 con 0x7f01ac15f8c0 2024-04-25T10:00:28.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.640+0000 7f01b0f68640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac15baa0 con 0x7f01ac0d0220 2024-04-25T10:00:28.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.640+0000 7f01aad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01ac15fca0 0x7f01ac0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.640+0000 7f01aad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01ac15fca0 0x7f01ac0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56994/0 (socket says 172.21.15.139:56994) 2024-04-25T10:00:28.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.640+0000 7f01aad9d640 1 -- 172.21.15.139:0/3115175052 learned_addr learned my addr 172.21.15.139:0/3115175052 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:28.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.640+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac15f8c0 0x7f01ac154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.640+0000 7f01aa59c640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01ac0d0220 0x7f01ac1f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.641+0000 7f01aad9d640 1 -- 172.21.15.139:0/3115175052 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01ac0d0220 msgr2=0x7f01ac1f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.641+0000 7f01aad9d640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01ac0d0220 0x7f01ac1f4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.641+0000 7f01aad9d640 1 -- 172.21.15.139:0/3115175052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac15f8c0 msgr2=0x7f01ac154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.641+0000 7f01aad9d640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac15f8c0 0x7f01ac154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.641+0000 7f01aad9d640 1 -- 172.21.15.139:0/3115175052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac154ab0 con 0x7f01ac15fca0 2024-04-25T10:00:28.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.641+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac15f8c0 0x7f01ac154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:28.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.641+0000 7f01aad9d640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01ac15fca0 0x7f01ac0707a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009870 tx=0x7f01a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.641+0000 7f019affd640 1 -- 172.21.15.139:0/3115175052 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4037040 con 0x7f01ac15fca0 2024-04-25T10:00:28.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.641+0000 7f019affd640 1 -- 172.21.15.139:0/3115175052 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4042070 con 0x7f01ac15fca0 2024-04-25T10:00:28.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.641+0000 7f019affd640 1 -- 172.21.15.139:0/3115175052 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4030920 con 0x7f01ac15fca0 2024-04-25T10:00:28.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.641+0000 7f01b0f68640 1 -- 172.21.15.139:0/3115175052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac156e70 con 0x7f01ac15fca0 2024-04-25T10:00:28.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.641+0000 7f01b0f68640 1 -- 172.21.15.139:0/3115175052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac157430 con 0x7f01ac15fca0 2024-04-25T10:00:28.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.642+0000 7f019affd640 1 -- 172.21.15.139:0/3115175052 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01a4030ac0 con 0x7f01ac15fca0 2024-04-25T10:00:28.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.643+0000 7f019affd640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f018c06b4d0 0x7f018c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.643+0000 7f019affd640 1 -- 172.21.15.139:0/3115175052 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f01a40aadd0 con 0x7f01ac15fca0 2024-04-25T10:00:28.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.643+0000 7f01aa59c640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f018c06b4d0 0x7f018c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.643+0000 7f01aa59c640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f018c06b4d0 0x7f018c06d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f019c001d40 tx=0x7f019c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.644+0000 7f01b3c62740 1 Processor -- start 2024-04-25T10:00:28.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.644+0000 7f01b3c62740 1 -- start start 2024-04-25T10:00:28.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01b3c62740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b9bb0 0x55a0972b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01b3c62740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b4830 0x55a0972b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01b3c62740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972b8cf0 0x55a0972b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01b3c62740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a09721dac0 con 0x55a0972b4830 2024-04-25T10:00:28.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01b3c62740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0971c6d80 con 0x55a0972b9bb0 2024-04-25T10:00:28.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01b3c62740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a097219b60 con 0x55a0972b8cf0 2024-04-25T10:00:28.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01a9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b4830 0x55a0972b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01a9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b4830 0x55a0972b87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56996/0 (socket says 172.21.15.139:56996) 2024-04-25T10:00:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01a9d9b640 1 -- 172.21.15.139:0/2800464442 learned_addr learned my addr 172.21.15.139:0/2800464442 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01aa59c640 1 --2- 172.21.15.139:0/2800464442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b9bb0 0x55a0972b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01aad9d640 1 --2- 172.21.15.139:0/2800464442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972b8cf0 0x55a0972b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01a9d9b640 1 -- 172.21.15.139:0/2800464442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972b8cf0 msgr2=0x55a0972b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/2800464442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972b8cf0 0x55a0972b9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01a9d9b640 1 -- 172.21.15.139:0/2800464442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b9bb0 msgr2=0x55a0972b9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/2800464442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b9bb0 0x55a0972b9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01a9d9b640 1 -- 172.21.15.139:0/2800464442 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a096ffa7e0 con 0x55a0972b4830 2024-04-25T10:00:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/2800464442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b4830 0x55a0972b87b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f01a000b8f0 tx=0x7f01a000bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a0c1f661ba9643 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f018bfff640 1 -- 172.21.15.139:0/2800464442 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0018020 con 0x55a0972b4830 2024-04-25T10:00:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f018bfff640 1 -- 172.21.15.139:0/2800464442 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a000f9c0 con 0x55a0972b4830 2024-04-25T10:00:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.645+0000 7f018bfff640 1 -- 172.21.15.139:0/2800464442 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a000fd70 con 0x55a0972b4830 2024-04-25T10:00:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.646+0000 7f01b3c62740 1 -- 172.21.15.139:0/2800464442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b4830 msgr2=0x55a0972b87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.646+0000 7f01b3c62740 1 --2- 172.21.15.139:0/2800464442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b4830 0x55a0972b87b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f01a000b8f0 tx=0x7f01a000bdb0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.646+0000 7f01b3c62740 1 -- 172.21.15.139:0/2800464442 shutdown_connections 2024-04-25T10:00:28.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.646+0000 7f01b3c62740 1 --2- 172.21.15.139:0/2800464442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b4830 0x55a0972b87b0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.646+0000 7f01b3c62740 1 --2- 172.21.15.139:0/2800464442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972b8cf0 0x55a0972b9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.646+0000 7f01b3c62740 1 --2- 172.21.15.139:0/2800464442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b9bb0 0x55a0972b9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.646+0000 7f01b3c62740 1 -- 172.21.15.139:0/2800464442 >> 172.21.15.139:0/2800464442 conn(0x55a0972b6cb0 msgr2=0x55a0972ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.646+0000 7f01b3c62740 1 -- 172.21.15.139:0/2800464442 shutdown_connections 2024-04-25T10:00:28.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.646+0000 7f01b3c62740 1 -- 172.21.15.139:0/2800464442 wait complete. 2024-04-25T10:00:28.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.647+0000 7f01b3c62740 1 -- start start 2024-04-25T10:00:28.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.647+0000 7f01b3c62740 1 Processor -- start 2024-04-25T10:00:28.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.647+0000 7f01b3c62740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b8cf0 0x55a0972e1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.647+0000 7f01b3c62740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 0x55a0972da0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01b3c62740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972dada0 0x55a097303e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01b3c62740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0972e5740 con 0x55a0972b9bb0 2024-04-25T10:00:28.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01b3c62740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0971bc530 con 0x55a0972b8cf0 2024-04-25T10:00:28.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01b3c62740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0972e4bc0 con 0x55a0972dada0 2024-04-25T10:00:28.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01b3c62740 10 client.0 _ll_register_callbacks cb 0x55a097214130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:28.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01a9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 0x55a0972da0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01a9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 0x55a0972da0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57008/0 (socket says 172.21.15.139:57008) 2024-04-25T10:00:28.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01a9d9b640 1 -- 172.21.15.139:0/2455628615 learned_addr learned my addr 172.21.15.139:0/2455628615 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:28.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01aa59c640 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b8cf0 0x55a0972e1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01aad9d640 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972dada0 0x55a097303e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01a9d9b640 1 -- 172.21.15.139:0/2455628615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972dada0 msgr2=0x55a097303e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972dada0 0x55a097303e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01a9d9b640 1 -- 172.21.15.139:0/2455628615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b8cf0 msgr2=0x55a0972e1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b8cf0 0x55a0972e1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01a9d9b640 1 -- 172.21.15.139:0/2455628615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a09705af70 con 0x55a0972b9bb0 2024-04-25T10:00:28.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 0x55a0972da0e0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f01a00146c0 tx=0x7f01a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c749e5d46d67f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01897fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:28.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01897fa640 1 -- 172.21.15.139:0/2455628615 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0018040 con 0x55a0972b9bb0 2024-04-25T10:00:28.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01897fa640 1 -- 172.21.15.139:0/2455628615 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f01a000f760 con 0x55a0972b9bb0 2024-04-25T10:00:28.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01b3c62740 1 -- 172.21.15.139:0/2455628615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01a000f9c0 con 0x55a0972b9bb0 2024-04-25T10:00:28.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01897fa640 1 -- 172.21.15.139:0/2455628615 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01a002f050 con 0x55a0972b9bb0 2024-04-25T10:00:28.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01b3c62740 10 client.8199 fetch_fsmap learned FSMap version 63 2024-04-25T10:00:28.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01b3c62740 1 -- 172.21.15.139:0/2455628615 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55a096ef2170 con 0x55a0972b9bb0 2024-04-25T10:00:28.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.648+0000 7f01897fa640 1 -- 172.21.15.139:0/2455628615 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a000f9c0 con 0x55a0972b9bb0 2024-04-25T10:00:28.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.649+0000 7f01897fa640 1 -- 172.21.15.139:0/2455628615 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f01a0032020 con 0x55a0972b9bb0 2024-04-25T10:00:28.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.649+0000 7f01b3c62740 10 client.8199 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:00:28.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.649+0000 7f01b3c62740 10 client.8199 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:00:28.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.649+0000 7f01b3c62740 20 client.8199 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:28.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.649+0000 7f01b3c62740 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55a097306c80 0x55a097307060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.649+0000 7f01b3c62740 4 client.8199 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:28.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.649+0000 7f01b3c62740 1 -- 172.21.15.139:0/2455628615 --> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a09722b000 con 0x55a097306c80 2024-04-25T10:00:28.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.649+0000 7f01aa59c640 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55a097306c80 0x55a097307060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.649+0000 7f01897fa640 10 client.8199 ms_handle_connect on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:28.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.649+0000 7f01aa59c640 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55a097306c80 0x55a097307060 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a86b2819896b4 server_cookie=41f7d8d36599f041 in_seq=0 out_seq=0 2024-04-25T10:00:28.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01897fa640 1 -- 172.21.15.139:0/2455628615 <== mds.0 v2:172.21.15.139:6834/1457343244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a09722b000 con 0x55a097306c80 2024-04-25T10:00:28.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01897fa640 10 client.8199 handle_command_reply: tid=0 2024-04-25T10:00:28.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01b3c62740 1 client.8199 shutdown 2024-04-25T10:00:28.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01b3c62740 1 -- 172.21.15.139:0/2455628615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 msgr2=0x55a0972da0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01b3c62740 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 0x55a0972da0e0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f01a00146c0 tx=0x7f01a0004730 comp rx=0 tx=0).stop 2024-04-25T10:00:28.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01b3c62740 1 -- 172.21.15.139:0/2455628615 shutdown_connections 2024-04-25T10:00:28.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01b3c62740 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55a097306c80 0x55a097307060 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01b3c62740 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 0x55a0972da0e0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01897fa640 0 client.8199 ms_handle_reset on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:28.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01b3c62740 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972dada0 0x55a097303e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01b3c62740 1 --2- 172.21.15.139:0/2455628615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b8cf0 0x55a0972e1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01b3c62740 1 -- 172.21.15.139:0/2455628615 >> 172.21.15.139:0/2455628615 conn(0x55a0972b4830 msgr2=0x55a0972ba6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01b3c62740 1 -- 172.21.15.139:0/2455628615 shutdown_connections 2024-04-25T10:00:28.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01b3c62740 1 -- 172.21.15.139:0/2455628615 wait complete. 2024-04-25T10:00:28.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.650+0000 7f01b3c62740 20 client.8199 trim_cache size 0 max 16384 2024-04-25T10:00:28.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.665+0000 7f01b3c62740 1 Processor -- start 2024-04-25T10:00:28.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.665+0000 7f01b3c62740 1 -- start start 2024-04-25T10:00:28.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01b3c62740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972b8cf0 0x55a0972ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01b3c62740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b9bb0 0x55a097303e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01b3c62740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972dada0 0x55a0972e5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01b3c62740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac1fe560 con 0x55a0972dada0 2024-04-25T10:00:28.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01b3c62740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac1fd6e0 con 0x55a0972b9bb0 2024-04-25T10:00:28.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01b3c62740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0b57c0 con 0x55a0972b8cf0 2024-04-25T10:00:28.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01aad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972dada0 0x55a0972e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01aad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972dada0 0x55a0972e5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57016/0 (socket says 172.21.15.139:57016) 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01aad9d640 1 -- 172.21.15.139:0/681198053 learned_addr learned my addr 172.21.15.139:0/681198053 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/681198053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b9bb0 0x55a097303e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01aa59c640 1 --2- 172.21.15.139:0/681198053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972b8cf0 0x55a0972ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01aad9d640 1 -- 172.21.15.139:0/681198053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972b8cf0 msgr2=0x55a0972ffc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01aad9d640 1 --2- 172.21.15.139:0/681198053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972b8cf0 0x55a0972ffc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01aad9d640 1 -- 172.21.15.139:0/681198053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b9bb0 msgr2=0x55a097303e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01aad9d640 1 --2- 172.21.15.139:0/681198053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b9bb0 0x55a097303e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01aad9d640 1 -- 172.21.15.139:0/681198053 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0972b90d0 con 0x55a0972dada0 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/681198053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b9bb0 0x55a097303e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f01aad9d640 1 --2- 172.21.15.139:0/681198053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972dada0 0x55a0972e5140 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f01a4079270 tx=0x7f01a4049040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48669748ed1366f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f018b7fe640 1 -- 172.21.15.139:0/681198053 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4032440 con 0x55a0972dada0 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f018b7fe640 1 -- 172.21.15.139:0/681198053 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4079330 con 0x55a0972dada0 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.666+0000 7f018b7fe640 1 -- 172.21.15.139:0/681198053 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a407d3d0 con 0x55a0972dada0 2024-04-25T10:00:28.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.667+0000 7f01b3c62740 1 -- 172.21.15.139:0/681198053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972dada0 msgr2=0x55a0972e5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.667+0000 7f01b3c62740 1 --2- 172.21.15.139:0/681198053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972dada0 0x55a0972e5140 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f01a4079270 tx=0x7f01a4049040 comp rx=0 tx=0).stop 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.667+0000 7f01b3c62740 1 -- 172.21.15.139:0/681198053 shutdown_connections 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.667+0000 7f01b3c62740 1 --2- 172.21.15.139:0/681198053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972dada0 0x55a0972e5140 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.667+0000 7f01b3c62740 1 --2- 172.21.15.139:0/681198053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b9bb0 0x55a097303e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.667+0000 7f01b3c62740 1 --2- 172.21.15.139:0/681198053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972b8cf0 0x55a0972ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.667+0000 7f01b3c62740 1 -- 172.21.15.139:0/681198053 >> 172.21.15.139:0/681198053 conn(0x55a0972b4830 msgr2=0x55a0972d9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.667+0000 7f01b3c62740 1 -- 172.21.15.139:0/681198053 shutdown_connections 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.667+0000 7f01b3c62740 1 -- 172.21.15.139:0/681198053 wait complete. 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.667+0000 7f01b3c62740 1 -- start start 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.668+0000 7f01b3c62740 1 Processor -- start 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.668+0000 7f01b3c62740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b8cf0 0x55a097353c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.668+0000 7f01b3c62740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 0x55a097302740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.668+0000 7f01b3c62740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972dada0 0x55a097307060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.668+0000 7f01b3c62740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0b5dc0 con 0x55a0972b9bb0 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.668+0000 7f01b3c62740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0972e4bf0 con 0x55a0972b8cf0 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.668+0000 7f01b3c62740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a09736bd20 con 0x55a0972dada0 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01b3c62740 10 client.0 _ll_register_callbacks cb 0x7f01a000b320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01a9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 0x55a097302740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01a9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 0x55a097302740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57030/0 (socket says 172.21.15.139:57030) 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01a9d9b640 1 -- 172.21.15.139:0/56816124 learned_addr learned my addr 172.21.15.139:0/56816124 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01a9d9b640 1 -- 172.21.15.139:0/56816124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972dada0 msgr2=0x55a097307060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01aa59c640 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b8cf0 0x55a097353c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01aad9d640 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972dada0 0x55a097307060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972dada0 0x55a097307060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01a9d9b640 1 -- 172.21.15.139:0/56816124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b8cf0 msgr2=0x55a097353c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b8cf0 0x55a097353c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01a9d9b640 1 -- 172.21.15.139:0/56816124 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0972dd9a0 con 0x55a0972b9bb0 2024-04-25T10:00:28.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01a9d9b640 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 0x55a097302740 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f01a00146c0 tx=0x7f01a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc138cc39eee70a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01897fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01897fa640 1 -- 172.21.15.139:0/56816124 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0037070 con 0x55a0972b9bb0 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01897fa640 1 -- 172.21.15.139:0/56816124 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f01a0014900 con 0x55a0972b9bb0 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01b3c62740 1 -- 172.21.15.139:0/56816124 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01a4079330 con 0x55a0972b9bb0 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01897fa640 1 -- 172.21.15.139:0/56816124 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01a0018050 con 0x55a0972b9bb0 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01b3c62740 10 client.8205 fetch_fsmap learned FSMap version 63 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01b3c62740 1 -- 172.21.15.139:0/56816124 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55a0972baa80 con 0x55a0972b9bb0 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.669+0000 7f01897fa640 1 -- 172.21.15.139:0/56816124 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4079330 con 0x55a0972b9bb0 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01897fa640 1 -- 172.21.15.139:0/56816124 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f01a0045080 con 0x55a0972b9bb0 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01b3c62740 10 client.8205 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01b3c62740 10 client.8205 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01b3c62740 20 client.8205 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01b3c62740 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55a0973075f0 0x55a097312170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01b3c62740 4 client.8205 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:00:28.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01b3c62740 1 -- 172.21.15.139:0/56816124 --> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55a097225350 con 0x55a0973075f0 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01aa59c640 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55a0973075f0 0x55a097312170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01897fa640 10 client.8205 ms_handle_connect on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01aa59c640 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55a0973075f0 0x55a097312170 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eb8dc07da0a1041 server_cookie=82e17495b3d68582 in_seq=0 out_seq=0 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01897fa640 1 -- 172.21.15.139:0/56816124 <== mds.0 v2:172.21.15.139:6834/1457343244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a097225350 con 0x55a0973075f0 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01897fa640 10 client.8205 handle_command_reply: tid=0 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.670+0000 7f01b3c62740 1 client.8205 shutdown 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b3c62740 1 -- 172.21.15.139:0/56816124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 msgr2=0x55a097302740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b3c62740 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 0x55a097302740 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f01a00146c0 tx=0x7f01a0008040 comp rx=0 tx=0).stop 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b3c62740 1 -- 172.21.15.139:0/56816124 shutdown_connections 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b3c62740 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55a0973075f0 0x55a097312170 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b3c62740 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a0972b9bb0 0x55a097302740 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b3c62740 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a0972dada0 0x55a097307060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b3c62740 1 --2- 172.21.15.139:0/56816124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a0972b8cf0 0x55a097353c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01897fa640 0 client.8205 ms_handle_reset on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:28.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b3c62740 1 -- 172.21.15.139:0/56816124 >> 172.21.15.139:0/56816124 conn(0x55a0972b4830 msgr2=0x55a0972b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b3c62740 1 -- 172.21.15.139:0/56816124 shutdown_connections 2024-04-25T10:00:28.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b3c62740 1 -- 172.21.15.139:0/56816124 wait complete. 2024-04-25T10:00:28.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b3c62740 20 client.8205 trim_cache size 0 max 16384 2024-04-25T10:00:28.699 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:00:28.699 INFO:teuthology.orchestra.run.smithi139.stdout: "return_code": 0, 2024-04-25T10:00:28.699 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_tag": "72e76e60-7056-4ee2-82f4-b87de061421e", 2024-04-25T10:00:28.699 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": "asynchronous" 2024-04-25T10:00:28.699 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:00:28.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b0f68640 1 -- 172.21.15.139:0/3115175052 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f018c06b4d0 msgr2=0x7f018c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b0f68640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f018c06b4d0 0x7f018c06d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f019c001d40 tx=0x7f019c009040 comp rx=0 tx=0).stop 2024-04-25T10:00:28.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b0f68640 1 -- 172.21.15.139:0/3115175052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01ac15fca0 msgr2=0x7f01ac0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b0f68640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01ac15fca0 0x7f01ac0707a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009870 tx=0x7f01a40046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b0f68640 1 -- 172.21.15.139:0/3115175052 shutdown_connections 2024-04-25T10:00:28.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b0f68640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01ac15fca0 0x7f01ac0707a0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.671+0000 7f01b0f68640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f018c06b4d0 0x7f018c06d990 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.672+0000 7f01b0f68640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01ac15f8c0 0x7f01ac154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.672+0000 7f01b0f68640 1 --2- 172.21.15.139:0/3115175052 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01ac0d0220 0x7f01ac1f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.672+0000 7f01b0f68640 1 -- 172.21.15.139:0/3115175052 >> 172.21.15.139:0/3115175052 conn(0x7f01ac0b31f0 msgr2=0x7f01ac0cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.672+0000 7f01b0f68640 1 -- 172.21.15.139:0/3115175052 shutdown_connections 2024-04-25T10:00:28.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.672+0000 7f01b0f68640 1 -- 172.21.15.139:0/3115175052 wait complete. 2024-04-25T10:00:28.702 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T10:00:28.672+0000 7fbdc8afe640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55a777d86100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:00:28.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:28.703 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:00:28.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.798+0000 7fce6a24a640 1 Processor -- start 2024-04-25T10:00:28.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.798+0000 7fce6a24a640 1 -- start start 2024-04-25T10:00:28.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce6a24a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fce640b31a0 0x7fce6406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce6a24a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fce6406e870 0x7fce6415e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce6a24a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce640b2a50 con 0x7fce64144090 2024-04-25T10:00:28.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce6a24a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce640b3610 con 0x7fce6406e870 2024-04-25T10:00:28.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce6a24a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6406eca0 con 0x7fce640b31a0 2024-04-25T10:00:28.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce63fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fce6406e870 0x7fce6415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce63fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fce6406e870 0x7fce6415e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57034/0 (socket says 172.21.15.139:57034) 2024-04-25T10:00:28.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce63fff640 1 -- 172.21.15.139:0/2123598696 learned_addr learned my addr 172.21.15.139:0/2123598696 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:28.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce62ffd640 1 --2- 172.21.15.139:0/2123598696 >> v2:172.21.15.150:3300/0 conn(0x7fce640b31a0 0x7fce6406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce63fff640 1 -- 172.21.15.139:0/2123598696 >> v2:172.21.15.150:3300/0 conn(0x7fce640b31a0 msgr2=0x7fce6406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce63fff640 1 --2- 172.21.15.139:0/2123598696 >> v2:172.21.15.150:3300/0 conn(0x7fce640b31a0 0x7fce6406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce63fff640 1 -- 172.21.15.139:0/2123598696 >> v1:172.21.15.150:6789/0 conn(0x7fce64144090 legacy=0x7fce640cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.799+0000 7fce63fff640 1 -- 172.21.15.139:0/2123598696 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce640cd0f0 con 0x7fce6406e870 2024-04-25T10:00:28.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.800+0000 7fce63fff640 1 --2- 172.21.15.139:0/2123598696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce6406e870 0x7fce6415e610 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fce640b3790 tx=0x7fce5c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a45f233634ca0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.800+0000 7fce627fc640 1 -- 172.21.15.139:0/2123598696 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c036070 con 0x7fce6406e870 2024-04-25T10:00:28.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.800+0000 7fce627fc640 1 -- 172.21.15.139:0/2123598696 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c031600 con 0x7fce6406e870 2024-04-25T10:00:28.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.800+0000 7fce627fc640 1 -- 172.21.15.139:0/2123598696 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c0319d0 con 0x7fce6406e870 2024-04-25T10:00:28.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.800+0000 7fce6a24a640 1 -- 172.21.15.139:0/2123598696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce6406e870 msgr2=0x7fce6415e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.800+0000 7fce6a24a640 1 --2- 172.21.15.139:0/2123598696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce6406e870 0x7fce6415e610 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fce640b3790 tx=0x7fce5c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.801+0000 7fce6a24a640 1 -- 172.21.15.139:0/2123598696 shutdown_connections 2024-04-25T10:00:28.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.801+0000 7fce6a24a640 1 --2- 172.21.15.139:0/2123598696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce6406e870 0x7fce6415e610 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.801+0000 7fce6a24a640 1 --2- 172.21.15.139:0/2123598696 >> v2:172.21.15.150:3300/0 conn(0x7fce640b31a0 0x7fce6406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.801+0000 7fce6a24a640 1 -- 172.21.15.139:0/2123598696 >> 172.21.15.139:0/2123598696 conn(0x7fce64067000 msgr2=0x7fce6406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.801+0000 7fce6a24a640 1 -- 172.21.15.139:0/2123598696 shutdown_connections 2024-04-25T10:00:28.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.801+0000 7fce6a24a640 1 -- 172.21.15.139:0/2123598696 wait complete. 2024-04-25T10:00:28.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.802+0000 7fce6a24a640 1 Processor -- start 2024-04-25T10:00:28.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.802+0000 7fce6a24a640 1 -- start start 2024-04-25T10:00:28.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce6a24a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce6406e870 0x7fce64140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce6a24a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce640b31a0 0x7fce6415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce6a24a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce64144090 0x7fce64161c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce6a24a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6415b650 con 0x7fce64144090 2024-04-25T10:00:28.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce6a24a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6415b370 con 0x7fce6406e870 2024-04-25T10:00:28.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce6a24a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6415b930 con 0x7fce640b31a0 2024-04-25T10:00:28.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce63fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce64144090 0x7fce64161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce63fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce64144090 0x7fce64161c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57038/0 (socket says 172.21.15.139:57038) 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce63fff640 1 -- 172.21.15.139:0/4168768179 learned_addr learned my addr 172.21.15.139:0/4168768179 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce637fe640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce6406e870 0x7fce64140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce62ffd640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce640b31a0 0x7fce6415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce63fff640 1 -- 172.21.15.139:0/4168768179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce640b31a0 msgr2=0x7fce6415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce63fff640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce640b31a0 0x7fce6415cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce63fff640 1 -- 172.21.15.139:0/4168768179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce6406e870 msgr2=0x7fce64140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce63fff640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce6406e870 0x7fce64140e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce63fff640 1 -- 172.21.15.139:0/4168768179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce64160ec0 con 0x7fce64144090 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce62ffd640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce640b31a0 0x7fce6415cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce63fff640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce64144090 0x7fce64161c40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fce5c034040 tx=0x7fce5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce4bfff640 1 -- 172.21.15.139:0/4168768179 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c036050 con 0x7fce64144090 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce4bfff640 1 -- 172.21.15.139:0/4168768179 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c042070 con 0x7fce64144090 2024-04-25T10:00:28.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce6a24a640 1 -- 172.21.15.139:0/4168768179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6415dc20 con 0x7fce64144090 2024-04-25T10:00:28.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce4bfff640 1 -- 172.21.15.139:0/4168768179 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c009510 con 0x7fce64144090 2024-04-25T10:00:28.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.803+0000 7fce6a24a640 1 -- 172.21.15.139:0/4168768179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce64161440 con 0x7fce64144090 2024-04-25T10:00:28.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.805+0000 7fce4bfff640 1 -- 172.21.15.139:0/4168768179 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce5c007bf0 con 0x7fce64144090 2024-04-25T10:00:28.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.805+0000 7fce4bfff640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce5006b4d0 0x7fce5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.805+0000 7fce4bfff640 1 -- 172.21.15.139:0/4168768179 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fce5c0ab250 con 0x7fce64144090 2024-04-25T10:00:28.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.805+0000 7fce637fe640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce5006b4d0 0x7fce5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.806+0000 7fce637fe640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce5006b4d0 0x7fce5006d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fce54001640 tx=0x7fce54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.806+0000 7fce6d028740 1 Processor -- start 2024-04-25T10:00:28.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.806+0000 7fce6d028740 1 -- start start 2024-04-25T10:00:28.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce6d028740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73b10 0x558755e76320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce6d028740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 0x558755e76860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce6d028740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e46770 0x558755e49f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce6d028740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x558755dafda0 con 0x558755e73ef0 2024-04-25T10:00:28.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce6d028740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558755d58b80 con 0x558755e46770 2024-04-25T10:00:28.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce6d028740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558755dabe40 con 0x558755e73b10 2024-04-25T10:00:28.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce62ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 0x558755e76860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce62ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 0x558755e76860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57046/0 (socket says 172.21.15.139:57046) 2024-04-25T10:00:28.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce62ffd640 1 -- 172.21.15.139:0/964343521 learned_addr learned my addr 172.21.15.139:0/964343521 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:28.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce62ffd640 1 -- 172.21.15.139:0/964343521 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73b10 msgr2=0x558755e76320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce62ffd640 1 --2- 172.21.15.139:0/964343521 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73b10 0x558755e76320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce62ffd640 1 -- 172.21.15.139:0/964343521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e46770 msgr2=0x558755e49f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce62ffd640 1 --2- 172.21.15.139:0/964343521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e46770 0x558755e49f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.807+0000 7fce62ffd640 1 -- 172.21.15.139:0/964343521 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558755d62f10 con 0x558755e73ef0 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.808+0000 7fce62ffd640 1 --2- 172.21.15.139:0/964343521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 0x558755e76860 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fce58009bb0 tx=0x7fce58009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbd7a30fa4ed075 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.808+0000 7fce497fa640 1 -- 172.21.15.139:0/964343521 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5800f040 con 0x558755e73ef0 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.808+0000 7fce497fa640 1 -- 172.21.15.139:0/964343521 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce58010e00 con 0x558755e73ef0 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.808+0000 7fce497fa640 1 -- 172.21.15.139:0/964343521 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58004270 con 0x558755e73ef0 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.808+0000 7fce6d028740 1 -- 172.21.15.139:0/964343521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 msgr2=0x558755e76860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.808+0000 7fce6d028740 1 --2- 172.21.15.139:0/964343521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 0x558755e76860 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fce58009bb0 tx=0x7fce58009190 comp rx=0 tx=0).stop 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.808+0000 7fce6d028740 1 -- 172.21.15.139:0/964343521 shutdown_connections 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.808+0000 7fce6d028740 1 --2- 172.21.15.139:0/964343521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 0x558755e76860 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.808+0000 7fce6d028740 1 --2- 172.21.15.139:0/964343521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e46770 0x558755e49f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.808+0000 7fce6d028740 1 --2- 172.21.15.139:0/964343521 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73b10 0x558755e76320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.808+0000 7fce6d028740 1 -- 172.21.15.139:0/964343521 >> 172.21.15.139:0/964343521 conn(0x558755e489e0 msgr2=0x558755e91320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.808+0000 7fce6d028740 1 -- 172.21.15.139:0/964343521 shutdown_connections 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.809+0000 7fce6d028740 1 -- 172.21.15.139:0/964343521 wait complete. 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.809+0000 7fce6d028740 1 -- start start 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.809+0000 7fce6d028740 1 Processor -- start 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce6d028740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73b10 0x558755e699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce6d028740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73ef0 0x558755e99400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce6d028740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e755a0 0x558755e75a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce6d028740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x558755e4bb90 con 0x558755e73b10 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce6d028740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558755d4e4b0 con 0x558755e755a0 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce6d028740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558755e4be70 con 0x558755e73ef0 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce6d028740 10 client.0 _ll_register_callbacks cb 0x558755d8e180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73b10 0x558755e699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73b10 0x558755e699f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57060/0 (socket says 172.21.15.139:57060) 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce637fe640 1 -- 172.21.15.139:0/3012684998 learned_addr learned my addr 172.21.15.139:0/3012684998 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:28.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce63fff640 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e755a0 0x558755e75a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce62ffd640 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73ef0 0x558755e99400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce637fe640 1 -- 172.21.15.139:0/3012684998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73ef0 msgr2=0x558755e99400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce637fe640 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73ef0 0x558755e99400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce637fe640 1 -- 172.21.15.139:0/3012684998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e755a0 msgr2=0x558755e75a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce637fe640 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e755a0 0x558755e75a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce637fe640 1 -- 172.21.15.139:0/3012684998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558755b8e7d0 con 0x558755e73b10 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce63fff640 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e755a0 0x558755e75a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce637fe640 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73b10 0x558755e699f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fce54003870 tx=0x7fce54003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f766a1a608667be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce427fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce427fc640 1 -- 172.21.15.139:0/3012684998 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5400ea30 con 0x558755e73b10 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce427fc640 1 -- 172.21.15.139:0/3012684998 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fce54023d80 con 0x558755e73b10 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.810+0000 7fce6d028740 1 -- 172.21.15.139:0/3012684998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce58010e00 con 0x558755e73b10 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce427fc640 1 -- 172.21.15.139:0/3012684998 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce5400ebd0 con 0x558755e73b10 2024-04-25T10:00:28.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce6d028740 10 client.8217 fetch_fsmap learned FSMap version 63 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce6d028740 1 -- 172.21.15.139:0/3012684998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x558755ac43f0 con 0x558755e73b10 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce427fc640 1 -- 172.21.15.139:0/3012684998 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce58010e00 con 0x558755e73b10 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce427fc640 1 -- 172.21.15.139:0/3012684998 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fce5402a050 con 0x558755e73b10 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce6d028740 10 client.8217 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce6d028740 10 client.8217 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce6d028740 20 client.8217 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce6d028740 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x558755e97800 0x558755e97be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce6d028740 4 client.8217 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce6d028740 1 -- 172.21.15.139:0/3012684998 --> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558755dbd2e0 con 0x558755e97800 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce62ffd640 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x558755e97800 0x558755e97be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce427fc640 10 client.8217 ms_handle_connect on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.811+0000 7fce62ffd640 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x558755e97800 0x558755e97be0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ecf35939068008 server_cookie=87b68831a277098f in_seq=0 out_seq=0 2024-04-25T10:00:28.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce427fc640 1 -- 172.21.15.139:0/3012684998 <== mds.0 v2:172.21.15.139:6834/1457343244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558755dbd2e0 con 0x558755e97800 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce427fc640 10 client.8217 handle_command_reply: tid=0 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce6d028740 1 client.8217 shutdown 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce6d028740 1 -- 172.21.15.139:0/3012684998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73b10 msgr2=0x558755e699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce6d028740 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73b10 0x558755e699f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fce54003870 tx=0x7fce54003c70 comp rx=0 tx=0).stop 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce6d028740 1 -- 172.21.15.139:0/3012684998 shutdown_connections 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce6d028740 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x558755e97800 0x558755e97be0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce6d028740 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e755a0 0x558755e75a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce6d028740 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73ef0 0x558755e99400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce427fc640 0 client.8217 ms_handle_reset on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce6d028740 1 --2- 172.21.15.139:0/3012684998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73b10 0x558755e699f0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce6d028740 1 -- 172.21.15.139:0/3012684998 >> 172.21.15.139:0/3012684998 conn(0x558755e46770 msgr2=0x558755e47420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce6d028740 1 -- 172.21.15.139:0/3012684998 shutdown_connections 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce6d028740 1 -- 172.21.15.139:0/3012684998 wait complete. 2024-04-25T10:00:28.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.812+0000 7fce6d028740 20 client.8217 trim_cache size 0 max 16384 2024-04-25T10:00:28.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.827+0000 7fce6d028740 1 Processor -- start 2024-04-25T10:00:28.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.827+0000 7fce6d028740 1 -- start start 2024-04-25T10:00:28.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce6d028740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e73b10 0x558755e476c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce6d028740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 0x558755e72dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce6d028740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e755a0 0x558755e73300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce6d028740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce641fe540 con 0x558755e73ef0 2024-04-25T10:00:28.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce6d028740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce641fd6c0 con 0x558755e73b10 2024-04-25T10:00:28.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce6d028740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce641fcbc0 con 0x558755e755a0 2024-04-25T10:00:28.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce62ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 0x558755e72dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce62ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 0x558755e72dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57076/0 (socket says 172.21.15.139:57076) 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce62ffd640 1 -- 172.21.15.139:0/588999983 learned_addr learned my addr 172.21.15.139:0/588999983 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce62ffd640 1 -- 172.21.15.139:0/588999983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e755a0 msgr2=0x558755e73300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce62ffd640 1 --2- 172.21.15.139:0/588999983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e755a0 0x558755e73300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce62ffd640 1 -- 172.21.15.139:0/588999983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e73b10 msgr2=0x558755e476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce637fe640 1 --2- 172.21.15.139:0/588999983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e73b10 0x558755e476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce62ffd640 1 --2- 172.21.15.139:0/588999983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e73b10 0x558755e476c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce62ffd640 1 -- 172.21.15.139:0/588999983 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558755a840f0 con 0x558755e73ef0 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce637fe640 1 --2- 172.21.15.139:0/588999983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e73b10 0x558755e476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce62ffd640 1 --2- 172.21.15.139:0/588999983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 0x558755e72dc0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fce58004000 tx=0x7fce58015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a7508dbe975495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce48ff9640 1 -- 172.21.15.139:0/588999983 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58004230 con 0x558755e73ef0 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce48ff9640 1 -- 172.21.15.139:0/588999983 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce580043f0 con 0x558755e73ef0 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.828+0000 7fce48ff9640 1 -- 172.21.15.139:0/588999983 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5802b310 con 0x558755e73ef0 2024-04-25T10:00:28.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.829+0000 7fce6d028740 1 -- 172.21.15.139:0/588999983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 msgr2=0x558755e72dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.829+0000 7fce6d028740 1 --2- 172.21.15.139:0/588999983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 0x558755e72dc0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fce58004000 tx=0x7fce58015000 comp rx=0 tx=0).stop 2024-04-25T10:00:28.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.829+0000 7fce6d028740 1 -- 172.21.15.139:0/588999983 shutdown_connections 2024-04-25T10:00:28.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.829+0000 7fce6d028740 1 --2- 172.21.15.139:0/588999983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e73ef0 0x558755e72dc0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.829+0000 7fce6d028740 1 --2- 172.21.15.139:0/588999983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e755a0 0x558755e73300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.829+0000 7fce6d028740 1 --2- 172.21.15.139:0/588999983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e73b10 0x558755e476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.829+0000 7fce6d028740 1 -- 172.21.15.139:0/588999983 >> 172.21.15.139:0/588999983 conn(0x558755e46770 msgr2=0x558755e4c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.829+0000 7fce6d028740 1 -- 172.21.15.139:0/588999983 shutdown_connections 2024-04-25T10:00:28.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.829+0000 7fce6d028740 1 -- 172.21.15.139:0/588999983 wait complete. 2024-04-25T10:00:28.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.830+0000 7fce6d028740 1 -- start start 2024-04-25T10:00:28.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.830+0000 7fce6d028740 1 Processor -- start 2024-04-25T10:00:28.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.830+0000 7fce6d028740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e73b10 0x558755e72970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.830+0000 7fce6d028740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73ef0 0x558755e72eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.830+0000 7fce6d028740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e755a0 0x558755e4a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.830+0000 7fce6d028740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x558755e944f0 con 0x558755e755a0 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.830+0000 7fce6d028740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce641fd0c0 con 0x558755e73b10 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.830+0000 7fce6d028740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558755efef70 con 0x558755e73ef0 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce6d028740 10 client.0 _ll_register_callbacks cb 0x7fce58009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce63fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e755a0 0x558755e4a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce63fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e755a0 0x558755e4a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:57080/0 (socket says 172.21.15.139:57080) 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce63fff640 1 -- 172.21.15.139:0/3679047860 learned_addr learned my addr 172.21.15.139:0/3679047860 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce62ffd640 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73ef0 0x558755e72eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce637fe640 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e73b10 0x558755e72970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce63fff640 1 -- 172.21.15.139:0/3679047860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73ef0 msgr2=0x558755e72eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce63fff640 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73ef0 0x558755e72eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce63fff640 1 -- 172.21.15.139:0/3679047860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e73b10 msgr2=0x558755e72970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce63fff640 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e73b10 0x558755e72970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce63fff640 1 -- 172.21.15.139:0/3679047860 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558755e79540 con 0x558755e755a0 2024-04-25T10:00:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce63fff640 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e755a0 0x558755e4a130 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0a9500 tx=0x7fce5c079370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c0fd290b5580d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce427fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce427fc640 1 -- 172.21.15.139:0/3679047860 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c079eb0 con 0x558755e755a0 2024-04-25T10:00:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce427fc640 1 -- 172.21.15.139:0/3679047860 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fce5c078c00 con 0x558755e755a0 2024-04-25T10:00:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce6d028740 1 -- 172.21.15.139:0/3679047860 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce580043f0 con 0x558755e755a0 2024-04-25T10:00:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce427fc640 1 -- 172.21.15.139:0/3679047860 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce5c0797b0 con 0x558755e755a0 2024-04-25T10:00:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce6d028740 10 client.8223 fetch_fsmap learned FSMap version 63 2024-04-25T10:00:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce6d028740 1 -- 172.21.15.139:0/3679047860 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x558755e79da0 con 0x558755e755a0 2024-04-25T10:00:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce427fc640 1 -- 172.21.15.139:0/3679047860 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce580043f0 con 0x558755e755a0 2024-04-25T10:00:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce427fc640 1 -- 172.21.15.139:0/3679047860 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fce5c0a9170 con 0x558755e755a0 2024-04-25T10:00:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce6d028740 10 client.8223 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:00:28.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce6d028740 10 client.8223 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce6d028740 20 client.8223 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.831+0000 7fce6d028740 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x558755e4b090 0x558755e4a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 4 client.8223 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 1 -- 172.21.15.139:0/3679047860 --> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558755e95e00 con 0x558755e4b090 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce637fe640 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x558755e4b090 0x558755e4a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce427fc640 10 client.8223 ms_handle_connect on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce637fe640 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x558755e4b090 0x558755e4a670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=300b0c058a58feab server_cookie=91abb54af96a29a5 in_seq=0 out_seq=0 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce427fc640 1 -- 172.21.15.139:0/3679047860 <== mds.0 v2:172.21.15.139:6834/1457343244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x558755e95e00 con 0x558755e4b090 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce427fc640 10 client.8223 handle_command_reply: tid=0 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 1 client.8223 shutdown 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 1 -- 172.21.15.139:0/3679047860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e755a0 msgr2=0x558755e4a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e755a0 0x558755e4a130 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0a9500 tx=0x7fce5c079370 comp rx=0 tx=0).stop 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 1 -- 172.21.15.139:0/3679047860 shutdown_connections 2024-04-25T10:00:28.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x558755e4b090 0x558755e4a670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558755e755a0 0x558755e4a130 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce427fc640 0 client.8223 ms_handle_reset on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:28.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558755e73ef0 0x558755e72eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 1 --2- 172.21.15.139:0/3679047860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558755e73b10 0x558755e72970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 1 -- 172.21.15.139:0/3679047860 >> 172.21.15.139:0/3679047860 conn(0x558755e46770 msgr2=0x558755e4c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 1 -- 172.21.15.139:0/3679047860 shutdown_connections 2024-04-25T10:00:28.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 1 -- 172.21.15.139:0/3679047860 wait complete. 2024-04-25T10:00:28.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.832+0000 7fce6d028740 20 client.8223 trim_cache size 0 max 16384 2024-04-25T10:00:28.844 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:00:28.844 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:00:28.845 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": { 2024-04-25T10:00:28.845 INFO:teuthology.orchestra.run.smithi139.stdout: "72e76e60-7056-4ee2-82f4-b87de061421e": { 2024-04-25T10:00:28.845 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/dir", 2024-04-25T10:00:28.845 INFO:teuthology.orchestra.run.smithi139.stdout: "tag": "72e76e60-7056-4ee2-82f4-b87de061421e", 2024-04-25T10:00:28.845 INFO:teuthology.orchestra.run.smithi139.stdout: "options": "recursive,repair" 2024-04-25T10:00:28.845 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:00:28.845 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:00:28.845 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:00:28.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.833+0000 7fce6a24a640 1 -- 172.21.15.139:0/4168768179 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce5006b4d0 msgr2=0x7fce5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.833+0000 7fce6a24a640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce5006b4d0 0x7fce5006d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fce54001640 tx=0x7fce54009040 comp rx=0 tx=0).stop 2024-04-25T10:00:28.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.833+0000 7fce6a24a640 1 -- 172.21.15.139:0/4168768179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce64144090 msgr2=0x7fce64161c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.833+0000 7fce6a24a640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce64144090 0x7fce64161c40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fce5c034040 tx=0x7fce5c004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:28.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.833+0000 7fce6a24a640 1 -- 172.21.15.139:0/4168768179 shutdown_connections 2024-04-25T10:00:28.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.833+0000 7fce6a24a640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce64144090 0x7fce64161c40 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.833+0000 7fce6a24a640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce5006b4d0 0x7fce5006d990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.833+0000 7fce6a24a640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce640b31a0 0x7fce6415cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.833+0000 7fce6a24a640 1 --2- 172.21.15.139:0/4168768179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce6406e870 0x7fce64140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.833+0000 7fce6a24a640 1 -- 172.21.15.139:0/4168768179 >> 172.21.15.139:0/4168768179 conn(0x7fce64067000 msgr2=0x7fce64071a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.833+0000 7fce6a24a640 1 -- 172.21.15.139:0/4168768179 shutdown_connections 2024-04-25T10:00:28.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:28.833+0000 7fce6a24a640 1 -- 172.21.15.139:0/4168768179 wait complete. 2024-04-25T10:00:28.848 INFO:tasks.cephfs.filesystem:scrub status for tag:72e76e60-7056-4ee2-82f4-b87de061421e - {'path': '/dir', 'tag': '72e76e60-7056-4ee2-82f4-b87de061421e', 'options': 'recursive,repair'} 2024-04-25T10:00:46.791 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:46.795 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:58.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:58.849 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:00:58.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.956+0000 7fe639bc1640 1 Processor -- start 2024-04-25T10:00:58.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.956+0000 7fe639bc1640 1 -- start start 2024-04-25T10:00:58.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.957+0000 7fe639bc1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6340afc80 0x7fe6340b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.957+0000 7fe639bc1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6340c6cc0 con 0x7fe6340af8a0 2024-04-25T10:00:58.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.957+0000 7fe639bc1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6340c6fa0 con 0x7fe63406b700 2024-04-25T10:00:58.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.957+0000 7fe639bc1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340c7280 con 0x7fe6340afc80 2024-04-25T10:00:58.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.957+0000 7fe6337fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe6340af8a0 0x7fe6340cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37152/0 (socket says 172.21.15.139:37152) 2024-04-25T10:00:58.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.957+0000 7fe6337fe640 1 -- 172.21.15.139:0/2674922937 learned_addr learned my addr 172.21.15.139:0/2674922937 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.957+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329561750 0 0) 0x7fe6340c6cc0 con 0x7fe6340af8a0 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe61c003520 con 0x7fe6340af8a0 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe632ffd640 1 --2- 172.21.15.139:0/2674922937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6340afc80 0x7fe6340b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1929998879 0 0) 0x7fe61c003520 con 0x7fe6340af8a0 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6340afc80 msgr2=0x7fe6340b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 --2- 172.21.15.139:0/2674922937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6340afc80 0x7fe6340b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 >> v1:172.21.15.150:6789/0 conn(0x7fe63406b700 legacy=0x7fe63406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6340cd090 con 0x7fe6340af8a0 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fe624001670 con 0x7fe6340af8a0 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 >> v1:172.21.15.139:6789/0 conn(0x7fe6340af8a0 legacy=0x7fe6340cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 --2- 172.21.15.139:0/2674922937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe61c0039e0 0x7fe61c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 --2- 172.21.15.139:0/2674922937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6340afc80 0x7fe61c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 --2- 172.21.15.139:0/2674922937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c004d40 0x7fe61c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340c7280 con 0x7fe61c004d40 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c00ba70 con 0x7fe61c0039e0 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c00bc70 con 0x7fe6340afc80 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe633fff640 1 --2- 172.21.15.139:0/2674922937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c004d40 0x7fe61c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe6337fe640 1 --2- 172.21.15.139:0/2674922937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe61c0039e0 0x7fe61c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe632ffd640 1 --2- 172.21.15.139:0/2674922937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6340afc80 0x7fe61c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe633fff640 1 -- 172.21.15.139:0/2674922937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6340afc80 msgr2=0x7fe61c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe633fff640 1 --2- 172.21.15.139:0/2674922937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6340afc80 0x7fe61c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe633fff640 1 -- 172.21.15.139:0/2674922937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe61c0039e0 msgr2=0x7fe61c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe633fff640 1 --2- 172.21.15.139:0/2674922937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe61c0039e0 0x7fe61c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.958+0000 7fe633fff640 1 -- 172.21.15.139:0/2674922937 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c003520 con 0x7fe61c004d40 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.959+0000 7fe633fff640 1 --2- 172.21.15.139:0/2674922937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c004d40 0x7fe61c0051c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fe62c001e20 tx=0x7fe61c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70f413618ca3429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.959+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c001a70 con 0x7fe61c004d40 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.959+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c02ae40 con 0x7fe61c004d40 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.959+0000 7fe6327fc640 1 -- 172.21.15.139:0/2674922937 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c003bb0 con 0x7fe61c004d40 2024-04-25T10:00:58.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.960+0000 7fe639bc1640 1 -- 172.21.15.139:0/2674922937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c004d40 msgr2=0x7fe61c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.960+0000 7fe639bc1640 1 --2- 172.21.15.139:0/2674922937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c004d40 0x7fe61c0051c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fe62c001e20 tx=0x7fe61c005830 comp rx=0 tx=0).stop 2024-04-25T10:00:58.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.960+0000 7fe633fff640 1 -- 172.21.15.139:0/2674922937 reap_dead start 2024-04-25T10:00:58.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.960+0000 7fe639bc1640 1 -- 172.21.15.139:0/2674922937 shutdown_connections 2024-04-25T10:00:58.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.960+0000 7fe639bc1640 1 -- 172.21.15.139:0/2674922937 >> 172.21.15.139:0/2674922937 conn(0x7fe6340b2230 msgr2=0x7fe6340b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.960+0000 7fe639bc1640 1 -- 172.21.15.139:0/2674922937 shutdown_connections 2024-04-25T10:00:58.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.960+0000 7fe639bc1640 1 -- 172.21.15.139:0/2674922937 wait complete. 2024-04-25T10:00:58.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.961+0000 7fe639bc1640 1 Processor -- start 2024-04-25T10:00:58.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.961+0000 7fe639bc1640 1 -- start start 2024-04-25T10:00:58.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe639bc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6341555a0 0x7fe63409df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe639bc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe634155980 0x7fe6340b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe639bc1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe63415a3d0 0x7fe63415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe639bc1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63406b3f0 con 0x7fe6341555a0 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe639bc1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63406b270 con 0x7fe634155980 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe639bc1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340b2d50 con 0x7fe63415a3d0 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe6337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6341555a0 0x7fe63409df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe6337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6341555a0 0x7fe63409df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54310/0 (socket says 172.21.15.139:54310) 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe6337fe640 1 -- 172.21.15.139:0/2989197609 learned_addr learned my addr 172.21.15.139:0/2989197609 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe632ffd640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe634155980 0x7fe6340b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe6337fe640 1 -- 172.21.15.139:0/2989197609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe63415a3d0 msgr2=0x7fe63415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe6337fe640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe63415a3d0 0x7fe63415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe6337fe640 1 -- 172.21.15.139:0/2989197609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe634155980 msgr2=0x7fe6340b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe6337fe640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe634155980 0x7fe6340b3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe6337fe640 1 -- 172.21.15.139:0/2989197609 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6340b38a0 con 0x7fe6341555a0 2024-04-25T10:00:58.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.962+0000 7fe6337fe640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6341555a0 0x7fe63409df40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fe624007090 tx=0x7fe624007550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.963+0000 7fe61bfff640 1 -- 172.21.15.139:0/2989197609 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62400f940 con 0x7fe6341555a0 2024-04-25T10:00:58.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.963+0000 7fe61bfff640 1 -- 172.21.15.139:0/2989197609 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe624018070 con 0x7fe6341555a0 2024-04-25T10:00:58.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.963+0000 7fe639bc1640 1 -- 172.21.15.139:0/2989197609 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6340b3b80 con 0x7fe6341555a0 2024-04-25T10:00:58.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.963+0000 7fe61bfff640 1 -- 172.21.15.139:0/2989197609 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62400fd60 con 0x7fe6341555a0 2024-04-25T10:00:58.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.963+0000 7fe639bc1640 1 -- 172.21.15.139:0/2989197609 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6340b4140 con 0x7fe6341555a0 2024-04-25T10:00:58.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.964+0000 7fe61bfff640 1 -- 172.21.15.139:0/2989197609 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe62401d460 con 0x7fe6341555a0 2024-04-25T10:00:58.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.964+0000 7fe61bfff640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe62006b4d0 0x7fe62006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.965+0000 7fe61bfff640 1 -- 172.21.15.139:0/2989197609 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe62408a5c0 con 0x7fe6341555a0 2024-04-25T10:00:58.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.965+0000 7fe632ffd640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe62006b4d0 0x7fe62006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.965+0000 7fe632ffd640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe62006b4d0 0x7fe62006d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe6341f6f10 tx=0x7fe628008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.966+0000 7fe63c99f740 1 Processor -- start 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.966+0000 7fe63c99f740 1 -- start start 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe63c99f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x562874046320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe63c99f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 0x562874046860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe63c99f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874016770 0x562874019f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe63c99f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562873f7fda0 con 0x562874043ef0 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe63c99f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562873f28b80 con 0x562874043b10 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe63c99f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562873f7be40 con 0x562874016770 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe632ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 0x562874046860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe632ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 0x562874046860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54326/0 (socket says 172.21.15.139:54326) 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe632ffd640 1 -- 172.21.15.139:0/3547507262 learned_addr learned my addr 172.21.15.139:0/3547507262 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe633fff640 1 --2- 172.21.15.139:0/3547507262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874016770 0x562874019f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe6337fe640 1 --2- 172.21.15.139:0/3547507262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x562874046320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe632ffd640 1 -- 172.21.15.139:0/3547507262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874016770 msgr2=0x562874019f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe632ffd640 1 --2- 172.21.15.139:0/3547507262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874016770 0x562874019f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe632ffd640 1 -- 172.21.15.139:0/3547507262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 msgr2=0x562874046320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe632ffd640 1 --2- 172.21.15.139:0/3547507262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x562874046320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe632ffd640 1 -- 172.21.15.139:0/3547507262 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562873f32f10 con 0x562874043ef0 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe6337fe640 1 --2- 172.21.15.139:0/3547507262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x562874046320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe632ffd640 1 --2- 172.21.15.139:0/3547507262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 0x562874046860 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fe628013d50 tx=0x7fe628016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd85c199dabf27d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe6197fa640 1 -- 172.21.15.139:0/3547507262 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe628028070 con 0x562874043ef0 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe6197fa640 1 -- 172.21.15.139:0/3547507262 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62801c030 con 0x562874043ef0 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.967+0000 7fe6197fa640 1 -- 172.21.15.139:0/3547507262 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe628004030 con 0x562874043ef0 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.968+0000 7fe63c99f740 1 -- 172.21.15.139:0/3547507262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 msgr2=0x562874046860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.968+0000 7fe63c99f740 1 --2- 172.21.15.139:0/3547507262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 0x562874046860 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fe628013d50 tx=0x7fe628016000 comp rx=0 tx=0).stop 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.968+0000 7fe63c99f740 1 -- 172.21.15.139:0/3547507262 shutdown_connections 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.968+0000 7fe63c99f740 1 --2- 172.21.15.139:0/3547507262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 0x562874046860 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.968+0000 7fe63c99f740 1 --2- 172.21.15.139:0/3547507262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874016770 0x562874019f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.968+0000 7fe63c99f740 1 --2- 172.21.15.139:0/3547507262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x562874046320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.968+0000 7fe63c99f740 1 -- 172.21.15.139:0/3547507262 >> 172.21.15.139:0/3547507262 conn(0x5628740189e0 msgr2=0x562874061320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.968+0000 7fe63c99f740 1 -- 172.21.15.139:0/3547507262 shutdown_connections 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.968+0000 7fe63c99f740 1 -- 172.21.15.139:0/3547507262 wait complete. 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.969+0000 7fe63c99f740 1 -- start start 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.969+0000 7fe63c99f740 1 Processor -- start 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe63c99f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x56287405d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe63c99f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874043ef0 0x56287403a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe63c99f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 0x56287401af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe63c99f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56287401bb90 con 0x56287403a990 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe63c99f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562873f1e4b0 con 0x562874043b10 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe63c99f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56287401be70 con 0x562874043ef0 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe63c99f740 10 client.0 _ll_register_callbacks cb 0x562873f5e180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe633fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 0x56287401af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe633fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 0x56287401af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54332/0 (socket says 172.21.15.139:54332) 2024-04-25T10:00:58.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe633fff640 1 -- 172.21.15.139:0/3490662281 learned_addr learned my addr 172.21.15.139:0/3490662281 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe632ffd640 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874043ef0 0x56287403a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe6337fe640 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x56287405d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe633fff640 1 -- 172.21.15.139:0/3490662281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874043ef0 msgr2=0x56287403a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe633fff640 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874043ef0 0x56287403a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe633fff640 1 -- 172.21.15.139:0/3490662281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 msgr2=0x56287405d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe633fff640 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x56287405d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe633fff640 1 -- 172.21.15.139:0/3490662281 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562873d5e7d0 con 0x56287403a990 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe633fff640 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 0x56287401af90 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe62c005040 tx=0x7fe62c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b8efb2781efa7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe6127fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe6127fc640 1 -- 172.21.15.139:0/3490662281 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c037040 con 0x56287403a990 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe6127fc640 1 -- 172.21.15.139:0/3490662281 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe62c0050d0 con 0x56287403a990 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe63c99f740 1 -- 172.21.15.139:0/3490662281 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe62801c030 con 0x56287403a990 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.970+0000 7fe6127fc640 1 -- 172.21.15.139:0/3490662281 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe62c007db0 con 0x56287403a990 2024-04-25T10:00:58.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe63c99f740 10 client.8235 fetch_fsmap learned FSMap version 63 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe63c99f740 1 -- 172.21.15.139:0/3490662281 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x562873c943f0 con 0x56287403a990 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe6127fc640 1 -- 172.21.15.139:0/3490662281 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62801c030 con 0x56287403a990 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe6127fc640 1 -- 172.21.15.139:0/3490662281 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe62c0052d0 con 0x56287403a990 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe63c99f740 10 client.8235 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe63c99f740 10 client.8235 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe63c99f740 20 client.8235 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe63c99f740 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x56287405be20 0x562874062450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe63c99f740 4 client.8235 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe63c99f740 1 -- 172.21.15.139:0/3490662281 --> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562873f8d2e0 con 0x56287405be20 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe6337fe640 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x56287405be20 0x562874062450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe6127fc640 10 client.8235 ms_handle_connect on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.971+0000 7fe6337fe640 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x56287405be20 0x562874062450 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=915d580b490bb67a server_cookie=518cbdb697c49480 in_seq=0 out_seq=0 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe6127fc640 1 -- 172.21.15.139:0/3490662281 <== mds.0 v2:172.21.15.139:6834/1457343244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562873f8d2e0 con 0x56287405be20 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe6127fc640 10 client.8235 handle_command_reply: tid=0 2024-04-25T10:00:58.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe63c99f740 1 client.8235 shutdown 2024-04-25T10:00:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe63c99f740 1 -- 172.21.15.139:0/3490662281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 msgr2=0x56287401af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe63c99f740 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 0x56287401af90 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe62c005040 tx=0x7fe62c0014a0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe63c99f740 1 -- 172.21.15.139:0/3490662281 shutdown_connections 2024-04-25T10:00:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe63c99f740 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x56287405be20 0x562874062450 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe63c99f740 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 0x56287401af90 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe63c99f740 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874043ef0 0x56287403a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe63c99f740 1 --2- 172.21.15.139:0/3490662281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x56287405d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe6127fc640 0 client.8235 ms_handle_reset on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe63c99f740 1 -- 172.21.15.139:0/3490662281 >> 172.21.15.139:0/3490662281 conn(0x562874016770 msgr2=0x562874017760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe63c99f740 1 -- 172.21.15.139:0/3490662281 shutdown_connections 2024-04-25T10:00:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe63c99f740 1 -- 172.21.15.139:0/3490662281 wait complete. 2024-04-25T10:00:58.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.972+0000 7fe63c99f740 20 client.8235 trim_cache size 0 max 16384 2024-04-25T10:00:58.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.987+0000 7fe63c99f740 1 Processor -- start 2024-04-25T10:00:58.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.987+0000 7fe63c99f740 1 -- start start 2024-04-25T10:00:58.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe63c99f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56287403a990 0x56287405d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe63c99f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x5628740492f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe63c99f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 0x5628740157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe63c99f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6342013d0 con 0x562874043ef0 2024-04-25T10:00:58.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe63c99f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6342004f0 con 0x562874043b10 2024-04-25T10:00:58.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe63c99f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6341575c0 con 0x56287403a990 2024-04-25T10:00:58.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe633fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 0x5628740157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe633fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 0x5628740157f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54348/0 (socket says 172.21.15.139:54348) 2024-04-25T10:00:58.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe633fff640 1 -- 172.21.15.139:0/2046554914 learned_addr learned my addr 172.21.15.139:0/2046554914 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe632ffd640 1 --2- 172.21.15.139:0/2046554914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x5628740492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe6337fe640 1 --2- 172.21.15.139:0/2046554914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56287403a990 0x56287405d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe633fff640 1 -- 172.21.15.139:0/2046554914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56287403a990 msgr2=0x56287405d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe633fff640 1 --2- 172.21.15.139:0/2046554914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56287403a990 0x56287405d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe633fff640 1 -- 172.21.15.139:0/2046554914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 msgr2=0x5628740492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe633fff640 1 --2- 172.21.15.139:0/2046554914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x5628740492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.988+0000 7fe633fff640 1 -- 172.21.15.139:0/2046554914 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562873c540f0 con 0x562874043ef0 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.989+0000 7fe633fff640 1 --2- 172.21.15.139:0/2046554914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 0x5628740157f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0018b0 tx=0x7fe62c038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa82c110529b135e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.989+0000 7fe618ff9640 1 -- 172.21.15.139:0/2046554914 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c005070 con 0x562874043ef0 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.989+0000 7fe618ff9640 1 -- 172.21.15.139:0/2046554914 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c044070 con 0x562874043ef0 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.989+0000 7fe618ff9640 1 -- 172.21.15.139:0/2046554914 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c005450 con 0x562874043ef0 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.989+0000 7fe63c99f740 1 -- 172.21.15.139:0/2046554914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 msgr2=0x5628740157f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.989+0000 7fe63c99f740 1 --2- 172.21.15.139:0/2046554914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 0x5628740157f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0018b0 tx=0x7fe62c038000 comp rx=0 tx=0).stop 2024-04-25T10:00:58.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.989+0000 7fe63c99f740 1 -- 172.21.15.139:0/2046554914 shutdown_connections 2024-04-25T10:00:58.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.989+0000 7fe63c99f740 1 --2- 172.21.15.139:0/2046554914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562874043ef0 0x5628740157f0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.990+0000 7fe63c99f740 1 --2- 172.21.15.139:0/2046554914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043b10 0x5628740492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.990+0000 7fe63c99f740 1 --2- 172.21.15.139:0/2046554914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56287403a990 0x56287405d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.990+0000 7fe63c99f740 1 -- 172.21.15.139:0/2046554914 >> 172.21.15.139:0/2046554914 conn(0x562874016770 msgr2=0x562874083c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.990+0000 7fe63c99f740 1 -- 172.21.15.139:0/2046554914 shutdown_connections 2024-04-25T10:00:58.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.990+0000 7fe63c99f740 1 -- 172.21.15.139:0/2046554914 wait complete. 2024-04-25T10:00:58.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.990+0000 7fe63c99f740 1 -- start start 2024-04-25T10:00:58.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.990+0000 7fe63c99f740 1 Processor -- start 2024-04-25T10:00:59.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe63c99f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 0x5628740cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe63c99f740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874043b10 0x562874039e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe63c99f740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043ef0 0x562874056cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe63c99f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56287407b3d0 con 0x56287403a990 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe63c99f740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6341fff00 con 0x562874043ef0 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe63c99f740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562874046e30 con 0x562874043b10 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe63c99f740 10 client.0 _ll_register_callbacks cb 0x7fe6280092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe6337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 0x5628740cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe6337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 0x5628740cb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54364/0 (socket says 172.21.15.139:54364) 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe6337fe640 1 -- 172.21.15.139:0/612759201 learned_addr learned my addr 172.21.15.139:0/612759201 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe633fff640 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043ef0 0x562874056cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe6337fe640 1 -- 172.21.15.139:0/612759201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874043b10 msgr2=0x562874039e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe632ffd640 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874043b10 0x562874039e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe6337fe640 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874043b10 0x562874039e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe6337fe640 1 -- 172.21.15.139:0/612759201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043ef0 msgr2=0x562874056cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe6337fe640 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043ef0 0x562874056cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe6337fe640 1 -- 172.21.15.139:0/612759201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56287405db90 con 0x56287403a990 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe633fff640 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043ef0 0x562874056cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe6337fe640 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 0x5628740cb460 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe624058790 tx=0x7fe624088710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e217c5ca85198616 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe6127fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.991+0000 7fe6127fc640 1 -- 172.21.15.139:0/612759201 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6240611a0 con 0x56287403a990 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe6127fc640 1 -- 172.21.15.139:0/612759201 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe624055100 con 0x56287403a990 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe63c99f740 1 -- 172.21.15.139:0/612759201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe62c044070 con 0x56287403a990 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe6127fc640 1 -- 172.21.15.139:0/612759201 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe624060d10 con 0x56287403a990 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe63c99f740 10 client.8241 fetch_fsmap learned FSMap version 63 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe63c99f740 1 -- 172.21.15.139:0/612759201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x562874057320 con 0x56287403a990 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe6127fc640 1 -- 172.21.15.139:0/612759201 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c044070 con 0x56287403a990 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe6127fc640 1 -- 172.21.15.139:0/612759201 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe624057ee0 con 0x56287403a990 2024-04-25T10:00:59.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe63c99f740 10 client.8241 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe63c99f740 10 client.8241 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe63c99f740 20 client.8241 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe63c99f740 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x56287403a4d0 0x56287401a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe63c99f740 4 client.8241 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe63c99f740 1 -- 172.21.15.139:0/612759201 --> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562873f87630 con 0x56287403a4d0 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe632ffd640 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x56287403a4d0 0x56287401a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe6127fc640 10 client.8241 ms_handle_connect on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe632ffd640 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x56287403a4d0 0x56287401a1c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f86c2fbaabb64e03 server_cookie=7c46db022e8f5df2 in_seq=0 out_seq=0 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe6127fc640 1 -- 172.21.15.139:0/612759201 <== mds.0 v2:172.21.15.139:6834/1457343244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562873f87630 con 0x56287403a4d0 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.992+0000 7fe6127fc640 10 client.8241 handle_command_reply: tid=0 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe63c99f740 1 client.8241 shutdown 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe63c99f740 1 -- 172.21.15.139:0/612759201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 msgr2=0x5628740cb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe63c99f740 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 0x5628740cb460 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe624058790 tx=0x7fe624088710 comp rx=0 tx=0).stop 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe63c99f740 1 -- 172.21.15.139:0/612759201 shutdown_connections 2024-04-25T10:00:59.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe63c99f740 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x56287403a4d0 0x56287401a1c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe63c99f740 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562874043ef0 0x562874056cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe63c99f740 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562874043b10 0x562874039e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe6127fc640 0 client.8241 ms_handle_reset on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:59.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe63c99f740 1 --2- 172.21.15.139:0/612759201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56287403a990 0x5628740cb460 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe63c99f740 1 -- 172.21.15.139:0/612759201 >> 172.21.15.139:0/612759201 conn(0x562874016770 msgr2=0x562874017710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe63c99f740 1 -- 172.21.15.139:0/612759201 shutdown_connections 2024-04-25T10:00:59.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe63c99f740 1 -- 172.21.15.139:0/612759201 wait complete. 2024-04-25T10:00:59.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe63c99f740 20 client.8241 trim_cache size 0 max 16384 2024-04-25T10:00:59.004 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:00:59.004 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "no active scrubs running", 2024-04-25T10:00:59.005 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": {} 2024-04-25T10:00:59.005 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:00:59.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe639bc1640 1 -- 172.21.15.139:0/2989197609 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe62006b4d0 msgr2=0x7fe62006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe639bc1640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe62006b4d0 0x7fe62006d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe6341f6f10 tx=0x7fe628008040 comp rx=0 tx=0).stop 2024-04-25T10:00:59.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe639bc1640 1 -- 172.21.15.139:0/2989197609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6341555a0 msgr2=0x7fe63409df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe639bc1640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6341555a0 0x7fe63409df40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fe624007090 tx=0x7fe624007550 comp rx=0 tx=0).stop 2024-04-25T10:00:59.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe639bc1640 1 -- 172.21.15.139:0/2989197609 shutdown_connections 2024-04-25T10:00:59.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe639bc1640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe62006b4d0 0x7fe62006d990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe639bc1640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe63415a3d0 0x7fe63415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe639bc1640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe634155980 0x7fe6340b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe639bc1640 1 --2- 172.21.15.139:0/2989197609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6341555a0 0x7fe63409df40 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.993+0000 7fe639bc1640 1 -- 172.21.15.139:0/2989197609 >> 172.21.15.139:0/2989197609 conn(0x7fe6340b2230 msgr2=0x7fe6340c54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.994+0000 7fe639bc1640 1 -- 172.21.15.139:0/2989197609 shutdown_connections 2024-04-25T10:00:59.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:58.994+0000 7fe639bc1640 1 -- 172.21.15.139:0/2989197609 wait complete. 2024-04-25T10:00:59.010 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:00:59.011 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.103+0000 7f14ece9a640 1 Processor -- start 2024-04-25T10:00:59.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.103+0000 7f14ece9a640 1 -- start start 2024-04-25T10:00:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.104+0000 7f14ece9a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f14e80d1080 0x7f14e80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.104+0000 7f14ece9a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14e8161850 0x7f14e80ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.104+0000 7f14ece9a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e80cf250 con 0x7f14e815dc60 2024-04-25T10:00:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.104+0000 7f14ece9a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e80ce780 con 0x7f14e8161850 2024-04-25T10:00:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.104+0000 7f14ece9a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e80d3730 con 0x7f14e80d1080 2024-04-25T10:00:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.104+0000 7f14e6d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14e8161850 0x7f14e80ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.104+0000 7f14e6d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14e8161850 0x7f14e80ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54368/0 (socket says 172.21.15.139:54368) 2024-04-25T10:00:59.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.104+0000 7f14e6d9d640 1 -- 172.21.15.139:0/1241797864 learned_addr learned my addr 172.21.15.139:0/1241797864 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:59.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.105+0000 7f14e5d9b640 1 --2- 172.21.15.139:0/1241797864 >> v2:172.21.15.150:3300/0 conn(0x7f14e80d1080 0x7f14e80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.105+0000 7f14e6d9d640 1 -- 172.21.15.139:0/1241797864 >> v2:172.21.15.150:3300/0 conn(0x7f14e80d1080 msgr2=0x7f14e80b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.105+0000 7f14e6d9d640 1 --2- 172.21.15.139:0/1241797864 >> v2:172.21.15.150:3300/0 conn(0x7f14e80d1080 0x7f14e80b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.105+0000 7f14e6d9d640 1 -- 172.21.15.139:0/1241797864 >> v1:172.21.15.150:6789/0 conn(0x7f14e815dc60 legacy=0x7f14e80d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.105+0000 7f14e6d9d640 1 -- 172.21.15.139:0/1241797864 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e80ce9b0 con 0x7f14e8161850 2024-04-25T10:00:59.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.105+0000 7f14e6d9d640 1 --2- 172.21.15.139:0/1241797864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e8161850 0x7f14e80ce240 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f14e8068750 tx=0x7f14e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb8aa0a6067fb7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.106+0000 7f14e559a640 1 -- 172.21.15.139:0/1241797864 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0037070 con 0x7f14e8161850 2024-04-25T10:00:59.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.106+0000 7f14e559a640 1 -- 172.21.15.139:0/1241797864 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e0032440 con 0x7f14e8161850 2024-04-25T10:00:59.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.106+0000 7f14e559a640 1 -- 172.21.15.139:0/1241797864 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0032810 con 0x7f14e8161850 2024-04-25T10:00:59.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.106+0000 7f14ece9a640 1 -- 172.21.15.139:0/1241797864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e8161850 msgr2=0x7f14e80ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.106+0000 7f14ece9a640 1 --2- 172.21.15.139:0/1241797864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e8161850 0x7f14e80ce240 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f14e8068750 tx=0x7f14e002a230 comp rx=0 tx=0).stop 2024-04-25T10:00:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.106+0000 7f14ece9a640 1 -- 172.21.15.139:0/1241797864 shutdown_connections 2024-04-25T10:00:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.106+0000 7f14ece9a640 1 --2- 172.21.15.139:0/1241797864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e8161850 0x7f14e80ce240 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.106+0000 7f14ece9a640 1 --2- 172.21.15.139:0/1241797864 >> v2:172.21.15.150:3300/0 conn(0x7f14e80d1080 0x7f14e80b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.106+0000 7f14ece9a640 1 -- 172.21.15.139:0/1241797864 >> 172.21.15.139:0/1241797864 conn(0x7f14e80b5910 msgr2=0x7f14e80b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.106+0000 7f14ece9a640 1 -- 172.21.15.139:0/1241797864 shutdown_connections 2024-04-25T10:00:59.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.106+0000 7f14ece9a640 1 -- 172.21.15.139:0/1241797864 wait complete. 2024-04-25T10:00:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.107+0000 7f14ece9a640 1 Processor -- start 2024-04-25T10:00:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.108+0000 7f14ece9a640 1 -- start start 2024-04-25T10:00:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.108+0000 7f14ece9a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14e80d1080 0x7f14e8149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.108+0000 7f14ece9a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14e815dc60 0x7f14e80c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.108+0000 7f14ece9a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e8161850 0x7f14e80c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.108+0000 7f14ece9a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e80d1fc0 con 0x7f14e8161850 2024-04-25T10:00:59.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.108+0000 7f14ece9a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e80cef70 con 0x7f14e80d1080 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.108+0000 7f14ece9a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e815e040 con 0x7f14e815dc60 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.108+0000 7f14e6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e8161850 0x7f14e80c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.108+0000 7f14e6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e8161850 0x7f14e80c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54384/0 (socket says 172.21.15.139:54384) 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.108+0000 7f14e6d9d640 1 -- 172.21.15.139:0/3998631923 learned_addr learned my addr 172.21.15.139:0/3998631923 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14e5d9b640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14e815dc60 0x7f14e80c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14e659c640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14e80d1080 0x7f14e8149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14e6d9d640 1 -- 172.21.15.139:0/3998631923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14e815dc60 msgr2=0x7f14e80c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14e6d9d640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14e815dc60 0x7f14e80c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14e6d9d640 1 -- 172.21.15.139:0/3998631923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14e80d1080 msgr2=0x7f14e8149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14e6d9d640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14e80d1080 0x7f14e8149d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14e6d9d640 1 -- 172.21.15.139:0/3998631923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e815e620 con 0x7f14e8161850 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14e659c640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14e80d1080 0x7f14e8149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14e5d9b640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14e815dc60 0x7f14e80c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14e6d9d640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e8161850 0x7f14e80c8aa0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f14e00090b0 tx=0x7f14e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14d6ffd640 1 -- 172.21.15.139:0/3998631923 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0041020 con 0x7f14e8161850 2024-04-25T10:00:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14d6ffd640 1 -- 172.21.15.139:0/3998631923 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e00306c0 con 0x7f14e8161850 2024-04-25T10:00:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14ece9a640 1 -- 172.21.15.139:0/3998631923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e815e900 con 0x7f14e8161850 2024-04-25T10:00:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14d6ffd640 1 -- 172.21.15.139:0/3998631923 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0037040 con 0x7f14e8161850 2024-04-25T10:00:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.109+0000 7f14ece9a640 1 -- 172.21.15.139:0/3998631923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e80cb120 con 0x7f14e8161850 2024-04-25T10:00:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.110+0000 7f14ece9a640 1 -- 172.21.15.139:0/3998631923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e8074520 con 0x7f14e8161850 2024-04-25T10:00:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.111+0000 7f14d6ffd640 1 -- 172.21.15.139:0/3998631923 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14e0044020 con 0x7f14e8161850 2024-04-25T10:00:59.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.111+0000 7f14d6ffd640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f14c806b500 0x7f14c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.111+0000 7f14d6ffd640 1 -- 172.21.15.139:0/3998631923 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f14e00aac70 con 0x7f14e8161850 2024-04-25T10:00:59.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.111+0000 7f14e659c640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f14c806b500 0x7f14c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.111+0000 7f14e659c640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f14c806b500 0x7f14c806d9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f14d8004520 tx=0x7f14d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.115+0000 7f14d6ffd640 1 -- 172.21.15.139:0/3998631923 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14e004b030 con 0x7f14e8161850 2024-04-25T10:00:59.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.322+0000 7f14ece9a640 1 -- 172.21.15.139:0/3998631923 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14e80d16f0 con 0x7f14e8161850 2024-04-25T10:00:59.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.322+0000 7f14d6ffd640 1 -- 172.21.15.139:0/3998631923 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f14e007f020 con 0x7f14e8161850 2024-04-25T10:00:59.328 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:00:59.328 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T10:00:25.131250+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6836/1457343244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1457343244},{"type":"v1","addr":"172.21.15.139:6836","nonce":1457343244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:59.329 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 63 2024-04-25T10:00:59.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.324+0000 7f14ece9a640 1 -- 172.21.15.139:0/3998631923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f14c806b500 msgr2=0x7f14c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.324+0000 7f14ece9a640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f14c806b500 0x7f14c806d9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f14d8004520 tx=0x7f14d8009040 comp rx=0 tx=0).stop 2024-04-25T10:00:59.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.324+0000 7f14ece9a640 1 -- 172.21.15.139:0/3998631923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e8161850 msgr2=0x7f14e80c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.324+0000 7f14ece9a640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e8161850 0x7f14e80c8aa0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f14e00090b0 tx=0x7f14e00046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.324+0000 7f14ece9a640 1 -- 172.21.15.139:0/3998631923 shutdown_connections 2024-04-25T10:00:59.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.324+0000 7f14ece9a640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e8161850 0x7f14e80c8aa0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.324+0000 7f14ece9a640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f14c806b500 0x7f14c806d9c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.324+0000 7f14ece9a640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14e815dc60 0x7f14e80c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.324+0000 7f14ece9a640 1 --2- 172.21.15.139:0/3998631923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14e80d1080 0x7f14e8149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.324+0000 7f14ece9a640 1 -- 172.21.15.139:0/3998631923 >> 172.21.15.139:0/3998631923 conn(0x7f14e80b5910 msgr2=0x7f14e806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.325+0000 7f14ece9a640 1 -- 172.21.15.139:0/3998631923 shutdown_connections 2024-04-25T10:00:59.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.325+0000 7f14ece9a640 1 -- 172.21.15.139:0/3998631923 wait complete. 2024-04-25T10:00:59.341 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:00:59.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.434+0000 7fde21eac640 1 Processor -- start 2024-04-25T10:00:59.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.434+0000 7fde21eac640 1 -- start start 2024-04-25T10:00:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.435+0000 7fde21eac640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1c0cd610 0x7fde1c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.435+0000 7fde21eac640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fde1c161fd0 0x7fde1c0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.435+0000 7fde21eac640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1c099ca0 con 0x7fde1c09ad40 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.435+0000 7fde21eac640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1c099a50 con 0x7fde1c161fd0 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.435+0000 7fde21eac640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c099f80 con 0x7fde1c0cd610 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.435+0000 7fde1bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fde1c161fd0 0x7fde1c0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.435+0000 7fde1bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fde1c161fd0 0x7fde1c0d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54400/0 (socket says 172.21.15.139:54400) 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.435+0000 7fde1affd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fde1c09ad40 0x7fde1c1618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37166/0 (socket says 172.21.15.139:37166) 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.435+0000 7fde1bfff640 1 -- 172.21.15.139:0/1490521940 learned_addr learned my addr 172.21.15.139:0/1490521940 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.435+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/1490521940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1c0cd610 0x7fde1c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.436+0000 7fde1bfff640 1 -- 172.21.15.139:0/1490521940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1c0cd610 msgr2=0x7fde1c0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.436+0000 7fde1bfff640 1 --2- 172.21.15.139:0/1490521940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1c0cd610 0x7fde1c0d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.436+0000 7fde1bfff640 1 -- 172.21.15.139:0/1490521940 >> v1:172.21.15.139:6789/0 conn(0x7fde1c09ad40 legacy=0x7fde1c1618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.436+0000 7fde1a7fc640 1 -- 172.21.15.139:0/1490521940 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943914571 0 0) 0x7fde1c099ca0 con 0x7fde1c09ad40 2024-04-25T10:00:59.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.436+0000 7fde1bfff640 1 -- 172.21.15.139:0/1490521940 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c09a260 con 0x7fde1c161fd0 2024-04-25T10:00:59.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.436+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/1490521940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1c0cd610 0x7fde1c0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:59.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.436+0000 7fde1bfff640 1 --2- 172.21.15.139:0/1490521940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde1c161fd0 0x7fde1c0d08b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fde14009e50 tx=0x7fde1402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab900be24dbad27f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.436+0000 7fde1a7fc640 1 -- 172.21.15.139:0/1490521940 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14036070 con 0x7fde1c161fd0 2024-04-25T10:00:59.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.436+0000 7fde1a7fc640 1 -- 172.21.15.139:0/1490521940 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde14031600 con 0x7fde1c161fd0 2024-04-25T10:00:59.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.436+0000 7fde1a7fc640 1 -- 172.21.15.139:0/1490521940 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde140319d0 con 0x7fde1c161fd0 2024-04-25T10:00:59.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.437+0000 7fde21eac640 1 -- 172.21.15.139:0/1490521940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde1c161fd0 msgr2=0x7fde1c0d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.437+0000 7fde21eac640 1 --2- 172.21.15.139:0/1490521940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde1c161fd0 0x7fde1c0d08b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fde14009e50 tx=0x7fde1402a7a0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.437+0000 7fde21eac640 1 -- 172.21.15.139:0/1490521940 shutdown_connections 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.437+0000 7fde21eac640 1 --2- 172.21.15.139:0/1490521940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde1c161fd0 0x7fde1c0d08b0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.437+0000 7fde21eac640 1 --2- 172.21.15.139:0/1490521940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1c0cd610 0x7fde1c0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.437+0000 7fde21eac640 1 -- 172.21.15.139:0/1490521940 >> 172.21.15.139:0/1490521940 conn(0x7fde1c0b31f0 msgr2=0x7fde1c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.437+0000 7fde21eac640 1 -- 172.21.15.139:0/1490521940 shutdown_connections 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.437+0000 7fde21eac640 1 -- 172.21.15.139:0/1490521940 wait complete. 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.438+0000 7fde21eac640 1 Processor -- start 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.438+0000 7fde21eac640 1 -- start start 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde21eac640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1c09ad40 0x7fde1c14dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde21eac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde1c0cd610 0x7fde1c0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde21eac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde1c161fd0 0x7fde1c0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde21eac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0cd9f0 con 0x7fde1c0cd610 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde21eac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c09a820 con 0x7fde1c161fd0 2024-04-25T10:00:59.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde21eac640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0cdb70 con 0x7fde1c09ad40 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde1c0cd610 0x7fde1c0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde1c0cd610 0x7fde1c0b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54406/0 (socket says 172.21.15.139:54406) 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde1affd640 1 -- 172.21.15.139:0/335893859 learned_addr learned my addr 172.21.15.139:0/335893859 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1c09ad40 0x7fde1c14dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde1bfff640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde1c161fd0 0x7fde1c0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde1affd640 1 -- 172.21.15.139:0/335893859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1c09ad40 msgr2=0x7fde1c14dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde1affd640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1c09ad40 0x7fde1c14dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde1affd640 1 -- 172.21.15.139:0/335893859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde1c161fd0 msgr2=0x7fde1c0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde1affd640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde1c161fd0 0x7fde1c0af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde1affd640 1 -- 172.21.15.139:0/335893859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c14e1a0 con 0x7fde1c0cd610 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1c09ad40 0x7fde1c14dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.439+0000 7fde1bfff640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde1c161fd0 0x7fde1c0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.440+0000 7fde1affd640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde1c0cd610 0x7fde1c0b14c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fde1000c4d0 tx=0x7fde1000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.440+0000 7fde03fff640 1 -- 172.21.15.139:0/335893859 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10018070 con 0x7fde1c0cd610 2024-04-25T10:00:59.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.440+0000 7fde03fff640 1 -- 172.21.15.139:0/335893859 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde10007500 con 0x7fde1c0cd610 2024-04-25T10:00:59.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.440+0000 7fde21eac640 1 -- 172.21.15.139:0/335893859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c1f4b70 con 0x7fde1c0cd610 2024-04-25T10:00:59.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.440+0000 7fde03fff640 1 -- 172.21.15.139:0/335893859 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10004a20 con 0x7fde1c0cd610 2024-04-25T10:00:59.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.440+0000 7fde21eac640 1 -- 172.21.15.139:0/335893859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c1f5130 con 0x7fde1c0cd610 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.441+0000 7fde03fff640 1 -- 172.21.15.139:0/335893859 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde1001b070 con 0x7fde1c0cd610 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.442+0000 7fde03fff640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fde0806b500 0x7fde0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.442+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fde0806b500 0x7fde0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.442+0000 7fde03fff640 1 -- 172.21.15.139:0/335893859 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fde10089810 con 0x7fde1c0cd610 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.442+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fde0806b500 0x7fde0806d9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0b0220 tx=0x7fde0c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.443+0000 7fde24bac740 1 Processor -- start 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.443+0000 7fde24bac740 1 -- start start 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.443+0000 7fde24bac740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b66e590 0x55759b6706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.443+0000 7fde24bac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 0x55759b66e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.443+0000 7fde24bac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b699050 0x55759b6994d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde24bac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759b5d1c60 con 0x55759b6aac60 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde24bac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759b570d80 con 0x55759b699050 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde24bac740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55759b5cdd00 con 0x55759b66e590 2024-04-25T10:00:59.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 0x55759b66e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 0x55759b66e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54418/0 (socket says 172.21.15.139:54418) 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde1affd640 1 -- 172.21.15.139:0/491461019 learned_addr learned my addr 172.21.15.139:0/491461019 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde1bfff640 1 --2- 172.21.15.139:0/491461019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b699050 0x55759b6994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/491461019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b66e590 0x55759b6706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde1affd640 1 -- 172.21.15.139:0/491461019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b66e590 msgr2=0x55759b6706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde1affd640 1 --2- 172.21.15.139:0/491461019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b66e590 0x55759b6706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde1affd640 1 -- 172.21.15.139:0/491461019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b699050 msgr2=0x55759b6994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde1affd640 1 --2- 172.21.15.139:0/491461019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b699050 0x55759b6994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde1affd640 1 -- 172.21.15.139:0/491461019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55759b3b27c0 con 0x55759b6aac60 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde1bfff640 1 --2- 172.21.15.139:0/491461019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b699050 0x55759b6994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde1affd640 1 --2- 172.21.15.139:0/491461019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 0x55759b66e970 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fde10087930 tx=0x7fde10057c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625102bbdfd269ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde017fa640 1 -- 172.21.15.139:0/491461019 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1005e070 con 0x55759b6aac60 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde017fa640 1 -- 172.21.15.139:0/491461019 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde10087e50 con 0x55759b6aac60 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.444+0000 7fde017fa640 1 -- 172.21.15.139:0/491461019 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10060350 con 0x55759b6aac60 2024-04-25T10:00:59.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.445+0000 7fde24bac740 1 -- 172.21.15.139:0/491461019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 msgr2=0x55759b66e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.445+0000 7fde24bac740 1 --2- 172.21.15.139:0/491461019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 0x55759b66e970 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fde10087930 tx=0x7fde10057c40 comp rx=0 tx=0).stop 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.445+0000 7fde24bac740 1 -- 172.21.15.139:0/491461019 shutdown_connections 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.445+0000 7fde24bac740 1 --2- 172.21.15.139:0/491461019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 0x55759b66e970 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.445+0000 7fde24bac740 1 --2- 172.21.15.139:0/491461019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b699050 0x55759b6994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.445+0000 7fde24bac740 1 --2- 172.21.15.139:0/491461019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b66e590 0x55759b6706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.445+0000 7fde24bac740 1 -- 172.21.15.139:0/491461019 >> 172.21.15.139:0/491461019 conn(0x55759b6366e0 msgr2=0x55759b6b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.445+0000 7fde24bac740 1 -- 172.21.15.139:0/491461019 shutdown_connections 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.445+0000 7fde24bac740 1 -- 172.21.15.139:0/491461019 wait complete. 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.446+0000 7fde24bac740 1 -- start start 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.446+0000 7fde24bac740 1 Processor -- start 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.446+0000 7fde24bac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b66e590 0x55759b68de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.446+0000 7fde24bac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 0x55759b6c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.446+0000 7fde24bac740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b6aac60 0x55759b69b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.446+0000 7fde24bac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759b56e840 con 0x55759b699050 2024-04-25T10:00:59.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.446+0000 7fde24bac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759b696ba0 con 0x55759b66e590 2024-04-25T10:00:59.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.446+0000 7fde24bac740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55759b68e3a0 con 0x55759b6aac60 2024-04-25T10:00:59.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.446+0000 7fde1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 0x55759b6c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.446+0000 7fde24bac740 10 client.0 _ll_register_callbacks cb 0x55759b588980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:59.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.446+0000 7fde1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 0x55759b6c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54426/0 (socket says 172.21.15.139:54426) 2024-04-25T10:00:59.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.446+0000 7fde1affd640 1 -- 172.21.15.139:0/3833097600 learned_addr learned my addr 172.21.15.139:0/3833097600 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:59.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b66e590 0x55759b68de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde1bfff640 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b6aac60 0x55759b69b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde1affd640 1 -- 172.21.15.139:0/3833097600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b6aac60 msgr2=0x55759b69b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde1affd640 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b6aac60 0x55759b69b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde1affd640 1 -- 172.21.15.139:0/3833097600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b66e590 msgr2=0x55759b68de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde1affd640 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b66e590 0x55759b68de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde1affd640 1 -- 172.21.15.139:0/3833097600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55759b412f50 con 0x55759b699050 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde1affd640 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 0x55759b6c40c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fde1005a010 tx=0x7fde1005a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399aecdfedffd723 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fddfa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fddfa7fc640 1 -- 172.21.15.139:0/3833097600 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10004dc0 con 0x55759b699050 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fddfa7fc640 1 -- 172.21.15.139:0/3833097600 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fde10057090 con 0x55759b699050 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde24bac740 1 -- 172.21.15.139:0/3833097600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde10087e50 con 0x55759b699050 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fddfa7fc640 1 -- 172.21.15.139:0/3833097600 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde10060350 con 0x55759b699050 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde24bac740 10 client.8262 fetch_fsmap learned FSMap version 63 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde24bac740 1 -- 172.21.15.139:0/3833097600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55759b2a60b0 con 0x55759b699050 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fddfa7fc640 1 -- 172.21.15.139:0/3833097600 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde10087e50 con 0x55759b699050 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fddfa7fc640 1 -- 172.21.15.139:0/3833097600 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde10053990 con 0x55759b699050 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde24bac740 10 client.8262 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:00:59.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde24bac740 10 client.8262 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde24bac740 20 client.8262 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde24bac740 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55759b69e810 0x55759b6d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde24bac740 4 client.8262 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.447+0000 7fde24bac740 1 -- 172.21.15.139:0/3833097600 --> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55759b591e60 con 0x55759b69e810 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55759b69e810 0x55759b6d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fddfa7fc640 10 client.8262 ms_handle_connect on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55759b69e810 0x55759b6d17d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8018cfd6b9892671 server_cookie=1b17ca0501429a3 in_seq=0 out_seq=0 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fddfa7fc640 1 -- 172.21.15.139:0/3833097600 <== mds.0 v2:172.21.15.139:6834/1457343244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55759b591e60 con 0x55759b69e810 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fddfa7fc640 10 client.8262 handle_command_reply: tid=0 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fde24bac740 1 client.8262 shutdown 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fde24bac740 1 -- 172.21.15.139:0/3833097600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 msgr2=0x55759b6c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fde24bac740 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 0x55759b6c40c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fde1005a010 tx=0x7fde1005a040 comp rx=0 tx=0).stop 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fde24bac740 1 -- 172.21.15.139:0/3833097600 shutdown_connections 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fde24bac740 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55759b69e810 0x55759b6d17d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fde24bac740 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 0x55759b6c40c0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fde24bac740 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b6aac60 0x55759b69b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fddfa7fc640 0 client.8262 ms_handle_reset on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:59.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fde24bac740 1 --2- 172.21.15.139:0/3833097600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b66e590 0x55759b68de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.448+0000 7fde24bac740 1 -- 172.21.15.139:0/3833097600 >> 172.21.15.139:0/3833097600 conn(0x55759b6366e0 msgr2=0x55759b696d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.449+0000 7fde24bac740 1 -- 172.21.15.139:0/3833097600 shutdown_connections 2024-04-25T10:00:59.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.449+0000 7fde24bac740 1 -- 172.21.15.139:0/3833097600 wait complete. 2024-04-25T10:00:59.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.449+0000 7fde24bac740 20 client.8262 trim_cache size 0 max 16384 2024-04-25T10:00:59.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.460+0000 7fde24bac740 1 Processor -- start 2024-04-25T10:00:59.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.460+0000 7fde24bac740 1 -- start start 2024-04-25T10:00:59.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.461+0000 7fde24bac740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b66e590 0x55759b6c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.461+0000 7fde24bac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b699050 0x55759b68cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.461+0000 7fde24bac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 0x55759b68d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.461+0000 7fde24bac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c1ff750 con 0x55759b6aac60 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.461+0000 7fde24bac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c1f6880 con 0x55759b699050 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.461+0000 7fde24bac740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c1ff150 con 0x55759b66e590 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.461+0000 7fde1bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 0x55759b68d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.461+0000 7fde1bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 0x55759b68d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54428/0 (socket says 172.21.15.139:54428) 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.461+0000 7fde1bfff640 1 -- 172.21.15.139:0/3756780520 learned_addr learned my addr 172.21.15.139:0/3756780520 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde1bfff640 1 -- 172.21.15.139:0/3756780520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b66e590 msgr2=0x55759b6c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/3756780520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b66e590 0x55759b6c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde1affd640 1 --2- 172.21.15.139:0/3756780520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b699050 0x55759b68cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde1bfff640 1 --2- 172.21.15.139:0/3756780520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b66e590 0x55759b6c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde1bfff640 1 -- 172.21.15.139:0/3756780520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b699050 msgr2=0x55759b68cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde1bfff640 1 --2- 172.21.15.139:0/3756780520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b699050 0x55759b68cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde1bfff640 1 -- 172.21.15.139:0/3756780520 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55759b6aaa90 con 0x55759b6aac60 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/3756780520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b66e590 0x55759b6c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde1affd640 1 --2- 172.21.15.139:0/3756780520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b699050 0x55759b68cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde1bfff640 1 --2- 172.21.15.139:0/3756780520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 0x55759b68d440 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fde14034040 tx=0x7fde1402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479fda4600e83a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde00ff9640 1 -- 172.21.15.139:0/3756780520 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14036040 con 0x55759b6aac60 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde00ff9640 1 -- 172.21.15.139:0/3756780520 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde14031740 con 0x55759b6aac60 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.462+0000 7fde00ff9640 1 -- 172.21.15.139:0/3756780520 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14031b30 con 0x55759b6aac60 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.463+0000 7fde24bac740 1 -- 172.21.15.139:0/3756780520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 msgr2=0x55759b68d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.463+0000 7fde24bac740 1 --2- 172.21.15.139:0/3756780520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 0x55759b68d440 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fde14034040 tx=0x7fde1402e000 comp rx=0 tx=0).stop 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.463+0000 7fde24bac740 1 -- 172.21.15.139:0/3756780520 shutdown_connections 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.463+0000 7fde24bac740 1 --2- 172.21.15.139:0/3756780520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b6aac60 0x55759b68d440 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.463+0000 7fde24bac740 1 --2- 172.21.15.139:0/3756780520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b699050 0x55759b68cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.463+0000 7fde24bac740 1 --2- 172.21.15.139:0/3756780520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b66e590 0x55759b6c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.463+0000 7fde24bac740 1 -- 172.21.15.139:0/3756780520 >> 172.21.15.139:0/3756780520 conn(0x55759b6366e0 msgr2=0x55759b68dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.463+0000 7fde24bac740 1 -- 172.21.15.139:0/3756780520 shutdown_connections 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.463+0000 7fde24bac740 1 -- 172.21.15.139:0/3756780520 wait complete. 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.463+0000 7fde24bac740 1 -- start start 2024-04-25T10:00:59.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.463+0000 7fde24bac740 1 Processor -- start 2024-04-25T10:00:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde24bac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b66e590 0x55759b6b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde24bac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 0x55759b694c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde24bac740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b6aac60 0x55759b68cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde24bac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759b56e840 con 0x55759b699050 2024-04-25T10:00:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde24bac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759b696ba0 con 0x55759b66e590 2024-04-25T10:00:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde24bac740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55759b69bc30 con 0x55759b6aac60 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde24bac740 10 client.0 _ll_register_callbacks cb 0x7fde10053ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 0x55759b694c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 0x55759b694c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54440/0 (socket says 172.21.15.139:54440) 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde1affd640 1 -- 172.21.15.139:0/4169097301 learned_addr learned my addr 172.21.15.139:0/4169097301 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde1bfff640 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b6aac60 0x55759b68cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b66e590 0x55759b6b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde1affd640 1 -- 172.21.15.139:0/4169097301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b6aac60 msgr2=0x55759b68cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde1affd640 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b6aac60 0x55759b68cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde1affd640 1 -- 172.21.15.139:0/4169097301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b66e590 msgr2=0x55759b6b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde1affd640 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b66e590 0x55759b6b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde1affd640 1 -- 172.21.15.139:0/4169097301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55759b6b80c0 con 0x55759b699050 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.464+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b66e590 0x55759b6b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fde1affd640 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 0x55759b694c30 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fde1005fe20 tx=0x7fde10053f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c489c5134fcfdda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fddfa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fddfa7fc640 1 -- 172.21.15.139:0/4169097301 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde100573b0 con 0x55759b699050 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fddfa7fc640 1 -- 172.21.15.139:0/4169097301 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fde10056b00 con 0x55759b699050 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fde24bac740 1 -- 172.21.15.139:0/4169097301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde14031740 con 0x55759b699050 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fddfa7fc640 1 -- 172.21.15.139:0/4169097301 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde10056d00 con 0x55759b699050 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fde24bac740 10 client.8268 fetch_fsmap learned FSMap version 63 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fde24bac740 1 -- 172.21.15.139:0/4169097301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55759b670210 con 0x55759b699050 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fddfa7fc640 1 -- 172.21.15.139:0/4169097301 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde14031740 con 0x55759b699050 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fddfa7fc640 1 -- 172.21.15.139:0/4169097301 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde10076b10 con 0x55759b699050 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fde24bac740 10 client.8268 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fde24bac740 10 client.8268 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fde24bac740 20 client.8268 populate_metadata read hostname 'smithi139' 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fde24bac740 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55759b68d440 0x55759b697790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fde24bac740 4 client.8268 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fde24bac740 1 -- 172.21.15.139:0/4169097301 --> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55759b5df1a0 con 0x55759b68d440 2024-04-25T10:00:59.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.465+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55759b68d440 0x55759b697790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fddfa7fc640 10 client.8268 ms_handle_connect on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde1b7fe640 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55759b68d440 0x55759b697790 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f64f141a2a292df3 server_cookie=3731858ccba87dac in_seq=0 out_seq=0 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fddfa7fc640 1 -- 172.21.15.139:0/4169097301 <== mds.0 v2:172.21.15.139:6834/1457343244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55759b5df1a0 con 0x55759b68d440 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fddfa7fc640 10 client.8268 handle_command_reply: tid=0 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde24bac740 1 client.8268 shutdown 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde24bac740 1 -- 172.21.15.139:0/4169097301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 msgr2=0x55759b694c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde24bac740 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 0x55759b694c30 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fde1005fe20 tx=0x7fde10053f90 comp rx=0 tx=0).stop 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde24bac740 1 -- 172.21.15.139:0/4169097301 shutdown_connections 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde24bac740 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.139:6834/1457343244,v1:172.21.15.139:6836/1457343244] conn(0x55759b68d440 0x55759b697790 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde24bac740 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55759b699050 0x55759b694c30 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde24bac740 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55759b6aac60 0x55759b68cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fddfa7fc640 0 client.8268 ms_handle_reset on v2:172.21.15.139:6834/1457343244 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde24bac740 1 --2- 172.21.15.139:0/4169097301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55759b66e590 0x55759b6b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde24bac740 1 -- 172.21.15.139:0/4169097301 >> 172.21.15.139:0/4169097301 conn(0x55759b6366e0 msgr2=0x55759b6a12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde24bac740 1 -- 172.21.15.139:0/4169097301 shutdown_connections 2024-04-25T10:00:59.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde24bac740 1 -- 172.21.15.139:0/4169097301 wait complete. 2024-04-25T10:00:59.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde24bac740 20 client.8268 trim_cache size 0 max 16384 2024-04-25T10:00:59.478 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:00:59.478 INFO:teuthology.orchestra.run.smithi139.stdout:[] 2024-04-25T10:00:59.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde21eac640 1 -- 172.21.15.139:0/335893859 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fde0806b500 msgr2=0x7fde0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde21eac640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fde0806b500 0x7fde0806d9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0b0220 tx=0x7fde0c008040 comp rx=0 tx=0).stop 2024-04-25T10:00:59.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde21eac640 1 -- 172.21.15.139:0/335893859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde1c0cd610 msgr2=0x7fde1c0b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.466+0000 7fde21eac640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde1c0cd610 0x7fde1c0b14c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fde1000c4d0 tx=0x7fde1000c990 comp rx=0 tx=0).stop 2024-04-25T10:00:59.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.467+0000 7fde21eac640 1 -- 172.21.15.139:0/335893859 shutdown_connections 2024-04-25T10:00:59.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.467+0000 7fde21eac640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde1c0cd610 0x7fde1c0b14c0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.467+0000 7fde21eac640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fde0806b500 0x7fde0806d9c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.467+0000 7fde21eac640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde1c161fd0 0x7fde1c0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.467+0000 7fde21eac640 1 --2- 172.21.15.139:0/335893859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde1c09ad40 0x7fde1c14dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.467+0000 7fde21eac640 1 -- 172.21.15.139:0/335893859 >> 172.21.15.139:0/335893859 conn(0x7fde1c0b31f0 msgr2=0x7fde1c06f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.467+0000 7fde21eac640 1 -- 172.21.15.139:0/335893859 shutdown_connections 2024-04-25T10:00:59.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.467+0000 7fde21eac640 1 -- 172.21.15.139:0/335893859 wait complete. 2024-04-25T10:00:59.482 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.573+0000 7f3d3495c640 1 Processor -- start 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.573+0000 7f3d3495c640 1 -- start start 2024-04-25T10:00:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.574+0000 7f3d3495c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3015efb0 0x7f3d300d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.574+0000 7f3d3495c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3d300cf900 0x7f3d3006c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.574+0000 7f3d3495c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d300cfe70 con 0x7f3d3006cd80 2024-04-25T10:00:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.574+0000 7f3d3495c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d300d0a80 con 0x7f3d300cf900 2024-04-25T10:00:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.574+0000 7f3d3495c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d300d0150 con 0x7f3d3015efb0 2024-04-25T10:00:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.574+0000 7f3d2ed9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3d3006cd80 0x7f3d30099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37172/0 (socket says 172.21.15.139:37172) 2024-04-25T10:00:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.574+0000 7f3d2ed9d640 1 -- 172.21.15.139:0/1469578981 learned_addr learned my addr 172.21.15.139:0/1469578981 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:00:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.575+0000 7f3d2e59c640 1 --2- 172.21.15.139:0/1469578981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3015efb0 0x7f3d300d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.575+0000 7f3d27fff640 1 --2- 172.21.15.139:0/1469578981 >> v2:172.21.15.150:3300/0 conn(0x7f3d300cf900 0x7f3d3006c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.575+0000 7f3d2dd9b640 1 -- 172.21.15.139:0/1469578981 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276798226 0 0) 0x7f3d300cfe70 con 0x7f3d3006cd80 2024-04-25T10:00:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.575+0000 7f3d2dd9b640 1 -- 172.21.15.139:0/1469578981 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d1c003520 con 0x7f3d3006cd80 2024-04-25T10:00:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.575+0000 7f3d2e59c640 1 -- 172.21.15.139:0/1469578981 >> v2:172.21.15.150:3300/0 conn(0x7f3d300cf900 msgr2=0x7f3d3006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.575+0000 7f3d2e59c640 1 --2- 172.21.15.139:0/1469578981 >> v2:172.21.15.150:3300/0 conn(0x7f3d300cf900 0x7f3d3006c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.575+0000 7f3d2e59c640 1 -- 172.21.15.139:0/1469578981 >> v1:172.21.15.139:6789/0 conn(0x7f3d3006cd80 legacy=0x7f3d30099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.575+0000 7f3d2e59c640 1 -- 172.21.15.139:0/1469578981 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d300d0430 con 0x7f3d3015efb0 2024-04-25T10:00:59.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.576+0000 7f3d2e59c640 1 --2- 172.21.15.139:0/1469578981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3015efb0 0x7f3d300d0f10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d200097a0 tx=0x7f3d2002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73cba3baada7826e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.576+0000 7f3d2dd9b640 1 -- 172.21.15.139:0/1469578981 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d20037070 con 0x7f3d3015efb0 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.576+0000 7f3d2dd9b640 1 -- 172.21.15.139:0/1469578981 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2002ae50 con 0x7f3d3015efb0 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.576+0000 7f3d2dd9b640 1 -- 172.21.15.139:0/1469578981 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d20032660 con 0x7f3d3015efb0 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.576+0000 7f3d3495c640 1 -- 172.21.15.139:0/1469578981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3015efb0 msgr2=0x7f3d300d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.576+0000 7f3d3495c640 1 --2- 172.21.15.139:0/1469578981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3015efb0 0x7f3d300d0f10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d200097a0 tx=0x7f3d2002a100 comp rx=0 tx=0).stop 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.576+0000 7f3d3495c640 1 -- 172.21.15.139:0/1469578981 shutdown_connections 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.576+0000 7f3d3495c640 1 --2- 172.21.15.139:0/1469578981 >> v2:172.21.15.150:3300/0 conn(0x7f3d300cf900 0x7f3d3006c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.576+0000 7f3d3495c640 1 --2- 172.21.15.139:0/1469578981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3015efb0 0x7f3d300d0f10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.576+0000 7f3d3495c640 1 -- 172.21.15.139:0/1469578981 >> 172.21.15.139:0/1469578981 conn(0x7f3d300d1790 msgr2=0x7f3d300d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.576+0000 7f3d3495c640 1 -- 172.21.15.139:0/1469578981 shutdown_connections 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.576+0000 7f3d3495c640 1 -- 172.21.15.139:0/1469578981 wait complete. 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.578+0000 7f3d3495c640 1 Processor -- start 2024-04-25T10:00:59.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.578+0000 7f3d3495c640 1 -- start start 2024-04-25T10:00:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.578+0000 7f3d3495c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d3006cd80 0x7f3d300b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.578+0000 7f3d3495c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d300cf900 0x7f3d300cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.578+0000 7f3d3495c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3015efb0 0x7f3d300cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.578+0000 7f3d3495c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3009a2e0 con 0x7f3d3006cd80 2024-04-25T10:00:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.578+0000 7f3d3495c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3009a160 con 0x7f3d3015efb0 2024-04-25T10:00:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.578+0000 7f3d3495c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d300cbde0 con 0x7f3d300cf900 2024-04-25T10:00:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.578+0000 7f3d2e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d3006cd80 0x7f3d300b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.578+0000 7f3d2e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d3006cd80 0x7f3d300b6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54442/0 (socket says 172.21.15.139:54442) 2024-04-25T10:00:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.578+0000 7f3d2e59c640 1 -- 172.21.15.139:0/1972126514 learned_addr learned my addr 172.21.15.139:0/1972126514 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:00:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.579+0000 7f3d2e59c640 1 -- 172.21.15.139:0/1972126514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d300cf900 msgr2=0x7f3d300cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.579+0000 7f3d2e59c640 1 --2- 172.21.15.139:0/1972126514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d300cf900 0x7f3d300cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.579+0000 7f3d2e59c640 1 -- 172.21.15.139:0/1972126514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3015efb0 msgr2=0x7f3d300cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.579+0000 7f3d2e59c640 1 --2- 172.21.15.139:0/1972126514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3015efb0 0x7f3d300cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.579+0000 7f3d2e59c640 1 -- 172.21.15.139:0/1972126514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d300b53c0 con 0x7f3d3006cd80 2024-04-25T10:00:59.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.579+0000 7f3d2e59c640 1 --2- 172.21.15.139:0/1972126514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d3006cd80 0x7f3d300b6b30 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f3d200098c0 tx=0x7f3d200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.579+0000 7f3d26ffd640 1 -- 172.21.15.139:0/1972126514 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d20042070 con 0x7f3d3006cd80 2024-04-25T10:00:59.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.579+0000 7f3d26ffd640 1 -- 172.21.15.139:0/1972126514 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2003e460 con 0x7f3d3006cd80 2024-04-25T10:00:59.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.579+0000 7f3d26ffd640 1 -- 172.21.15.139:0/1972126514 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d20037040 con 0x7f3d3006cd80 2024-04-25T10:00:59.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.579+0000 7f3d3495c640 1 -- 172.21.15.139:0/1972126514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d300b56a0 con 0x7f3d3006cd80 2024-04-25T10:00:59.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.579+0000 7f3d3495c640 1 -- 172.21.15.139:0/1972126514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d300b5c60 con 0x7f3d3006cd80 2024-04-25T10:00:59.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.580+0000 7f3d3495c640 1 -- 172.21.15.139:0/1972126514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d30074520 con 0x7f3d3006cd80 2024-04-25T10:00:59.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.581+0000 7f3d26ffd640 1 -- 172.21.15.139:0/1972126514 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d20045070 con 0x7f3d3006cd80 2024-04-25T10:00:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.581+0000 7f3d26ffd640 1 --2- 172.21.15.139:0/1972126514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3d1006b4d0 0x7f3d1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.581+0000 7f3d26ffd640 1 -- 172.21.15.139:0/1972126514 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f3d200aab10 con 0x7f3d3006cd80 2024-04-25T10:00:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.581+0000 7f3d27fff640 1 --2- 172.21.15.139:0/1972126514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3d1006b4d0 0x7f3d1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.581+0000 7f3d27fff640 1 --2- 172.21.15.139:0/1972126514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3d1006b4d0 0x7f3d1006d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3d18006a90 tx=0x7f3d18006a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.585+0000 7f3d26ffd640 1 -- 172.21.15.139:0/1972126514 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d200796e0 con 0x7f3d3006cd80 2024-04-25T10:00:59.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.792+0000 7f3d3495c640 1 -- 172.21.15.139:0/1972126514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3d300d25f0 con 0x7f3d3006cd80 2024-04-25T10:00:59.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.792+0000 7f3d26ffd640 1 -- 172.21.15.139:0/1972126514 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3d200aa040 con 0x7f3d3006cd80 2024-04-25T10:00:59.797 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:00:59.797 INFO:teuthology.orchestra.run.smithi139.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:00:59.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.794+0000 7f3d3495c640 1 -- 172.21.15.139:0/1972126514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3d1006b4d0 msgr2=0x7f3d1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.794+0000 7f3d3495c640 1 --2- 172.21.15.139:0/1972126514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3d1006b4d0 0x7f3d1006d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3d18006a90 tx=0x7f3d18006a20 comp rx=0 tx=0).stop 2024-04-25T10:00:59.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.794+0000 7f3d3495c640 1 -- 172.21.15.139:0/1972126514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d3006cd80 msgr2=0x7f3d300b6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.794+0000 7f3d3495c640 1 --2- 172.21.15.139:0/1972126514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d3006cd80 0x7f3d300b6b30 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f3d200098c0 tx=0x7f3d200046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.794+0000 7f3d3495c640 1 -- 172.21.15.139:0/1972126514 shutdown_connections 2024-04-25T10:00:59.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.794+0000 7f3d3495c640 1 --2- 172.21.15.139:0/1972126514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3d1006b4d0 0x7f3d1006d990 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.794+0000 7f3d3495c640 1 --2- 172.21.15.139:0/1972126514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3015efb0 0x7f3d300cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.794+0000 7f3d3495c640 1 --2- 172.21.15.139:0/1972126514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d300cf900 0x7f3d300cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.794+0000 7f3d3495c640 1 --2- 172.21.15.139:0/1972126514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d3006cd80 0x7f3d300b6b30 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.794+0000 7f3d3495c640 1 -- 172.21.15.139:0/1972126514 >> 172.21.15.139:0/1972126514 conn(0x7f3d300d1790 msgr2=0x7f3d300702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.794+0000 7f3d3495c640 1 -- 172.21.15.139:0/1972126514 shutdown_connections 2024-04-25T10:00:59.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:00:59.794+0000 7f3d3495c640 1 -- 172.21.15.139:0/1972126514 wait complete. 2024-04-25T10:00:59.811 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:00:59.811 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:00:59.811 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:00:59.857 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:00:59.904 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:00:59.904 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:00:59.904 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:59.919 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:00:59.919 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:59.974 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:00.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.106+0000 7f1d8fee1640 1 Processor -- start 2024-04-25T10:01:00.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.106+0000 7f1d8fee1640 1 -- start start 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8fee1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d88160e90 0x7f1d880afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8fee1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1d8806afc0 0x7f1d88070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8fee1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d880d0400 con 0x7f1d8806ab60 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8fee1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d880d14a0 con 0x7f1d8806afc0 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8fee1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880d06e0 con 0x7f1d88160e90 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8e47e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1d8806afc0 0x7f1d88070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8e47e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1d8806afc0 0x7f1d88070180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54452/0 (socket says 172.21.15.139:54452) 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8e47e640 1 -- 172.21.15.139:0/1746111504 learned_addr learned my addr 172.21.15.139:0/1746111504 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8dc7d640 1 --2- 172.21.15.139:0/1746111504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d88160e90 0x7f1d880afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8e47e640 1 -- 172.21.15.139:0/1746111504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d88160e90 msgr2=0x7f1d880afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8e47e640 1 --2- 172.21.15.139:0/1746111504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d88160e90 0x7f1d880afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8e47e640 1 -- 172.21.15.139:0/1746111504 >> v1:172.21.15.150:6789/0 conn(0x7f1d8806ab60 legacy=0x7f1d8806fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.107+0000 7f1d8e47e640 1 -- 172.21.15.139:0/1746111504 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d880d09c0 con 0x7f1d8806afc0 2024-04-25T10:01:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.108+0000 7f1d8e47e640 1 --2- 172.21.15.139:0/1746111504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8806afc0 0x7f1d88070180 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009870 tx=0x7f1d8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eae51e0f24c2bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.108+0000 7f1d8cc7b640 1 -- 172.21.15.139:0/1746111504 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80037070 con 0x7f1d8806afc0 2024-04-25T10:01:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.108+0000 7f1d8cc7b640 1 -- 172.21.15.139:0/1746111504 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d80032440 con 0x7f1d8806afc0 2024-04-25T10:01:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.108+0000 7f1d8cc7b640 1 -- 172.21.15.139:0/1746111504 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80032810 con 0x7f1d8806afc0 2024-04-25T10:01:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.109+0000 7f1d8fee1640 1 -- 172.21.15.139:0/1746111504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8806afc0 msgr2=0x7f1d88070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.109+0000 7f1d8fee1640 1 --2- 172.21.15.139:0/1746111504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8806afc0 0x7f1d88070180 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009870 tx=0x7f1d8002a270 comp rx=0 tx=0).stop 2024-04-25T10:01:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.109+0000 7f1d8fee1640 1 -- 172.21.15.139:0/1746111504 shutdown_connections 2024-04-25T10:01:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.109+0000 7f1d8fee1640 1 --2- 172.21.15.139:0/1746111504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8806afc0 0x7f1d88070180 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.109+0000 7f1d8fee1640 1 --2- 172.21.15.139:0/1746111504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d88160e90 0x7f1d880afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.109+0000 7f1d8fee1640 1 -- 172.21.15.139:0/1746111504 >> 172.21.15.139:0/1746111504 conn(0x7f1d880b23f0 msgr2=0x7f1d880b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.109+0000 7f1d8fee1640 1 -- 172.21.15.139:0/1746111504 shutdown_connections 2024-04-25T10:01:00.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.109+0000 7f1d8fee1640 1 -- 172.21.15.139:0/1746111504 wait complete. 2024-04-25T10:01:00.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.110+0000 7f1d8fee1640 1 Processor -- start 2024-04-25T10:01:00.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.110+0000 7f1d8fee1640 1 -- start start 2024-04-25T10:01:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8fee1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8806ab60 0x7f1d8814c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8fee1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806afc0 0x7f1d881f3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8fee1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d88160e90 0x7f1d880cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8fee1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880b0000 con 0x7f1d8806ab60 2024-04-25T10:01:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8fee1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880d0f80 con 0x7f1d88160e90 2024-04-25T10:01:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8fee1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8806b410 con 0x7f1d8806afc0 2024-04-25T10:01:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8dc7d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8806ab60 0x7f1d8814c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8dc7d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8806ab60 0x7f1d8814c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54462/0 (socket says 172.21.15.139:54462) 2024-04-25T10:01:00.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8dc7d640 1 -- 172.21.15.139:0/74608136 learned_addr learned my addr 172.21.15.139:0/74608136 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8d47c640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806afc0 0x7f1d881f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8e47e640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d88160e90 0x7f1d880cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8dc7d640 1 -- 172.21.15.139:0/74608136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806afc0 msgr2=0x7f1d881f3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8dc7d640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806afc0 0x7f1d881f3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8dc7d640 1 -- 172.21.15.139:0/74608136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d88160e90 msgr2=0x7f1d880cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8dc7d640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d88160e90 0x7f1d880cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8dc7d640 1 -- 172.21.15.139:0/74608136 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d88157500 con 0x7f1d8806ab60 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8e47e640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d88160e90 0x7f1d880cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8d47c640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806afc0 0x7f1d881f3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d8dc7d640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8806ab60 0x7f1d8814c220 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1d84007e70 tx=0x7f1d8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.111+0000 7f1d7e7fc640 1 -- 172.21.15.139:0/74608136 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84014420 con 0x7f1d8806ab60 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.112+0000 7f1d7e7fc640 1 -- 172.21.15.139:0/74608136 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d84019070 con 0x7f1d8806ab60 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.112+0000 7f1d7e7fc640 1 -- 172.21.15.139:0/74608136 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84014800 con 0x7f1d8806ab60 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.112+0000 7f1d8fee1640 1 -- 172.21.15.139:0/74608136 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d88158d00 con 0x7f1d8806ab60 2024-04-25T10:01:00.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.112+0000 7f1d8fee1640 1 -- 172.21.15.139:0/74608136 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d881592c0 con 0x7f1d8806ab60 2024-04-25T10:01:00.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.112+0000 7f1d8fee1640 1 -- 172.21.15.139:0/74608136 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d88074520 con 0x7f1d8806ab60 2024-04-25T10:01:00.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.113+0000 7f1d7e7fc640 1 -- 172.21.15.139:0/74608136 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d840149a0 con 0x7f1d8806ab60 2024-04-25T10:01:00.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.113+0000 7f1d7e7fc640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d6c06b4d0 0x7f1d6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.113+0000 7f1d7e7fc640 1 -- 172.21.15.139:0/74608136 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f1d8408ac60 con 0x7f1d8806ab60 2024-04-25T10:01:00.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.114+0000 7f1d8d47c640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d6c06b4d0 0x7f1d6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.114+0000 7f1d8d47c640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d6c06b4d0 0x7f1d6c06d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1d880b1ff0 tx=0x7f1d78009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.118+0000 7f1d7e7fc640 1 -- 172.21.15.139:0/74608136 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d84059830 con 0x7f1d8806ab60 2024-04-25T10:01:00.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.289+0000 7f1d8fee1640 1 -- 172.21.15.139:0/74608136 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d8806d2b0 con 0x7f1d8806ab60 2024-04-25T10:01:00.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.290+0000 7f1d7e7fc640 1 -- 172.21.15.139:0/74608136 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17389 (secure 0 0 0) 0x7f1d84059130 con 0x7f1d8806ab60 2024-04-25T10:01:00.296 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:00.296 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":93,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:00:22.118995+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:55.585380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:56.336230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6836/1239243954":"2024-04-26T10:00:22.118970+0000","172.21.15.139:6834/1239243954":"2024-04-26T10:00:22.118970+0000","172.21.15.150:6835/2266713846":"2024-04-26T09:59:10.897062+0000","172.21.15.150:6833/2266713846":"2024-04-26T09:59:10.897062+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:00.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.292+0000 7f1d8fee1640 1 -- 172.21.15.139:0/74608136 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d6c06b4d0 msgr2=0x7f1d6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.292+0000 7f1d8fee1640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d6c06b4d0 0x7f1d6c06d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1d880b1ff0 tx=0x7f1d78009040 comp rx=0 tx=0).stop 2024-04-25T10:01:00.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.292+0000 7f1d8fee1640 1 -- 172.21.15.139:0/74608136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8806ab60 msgr2=0x7f1d8814c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.292+0000 7f1d8fee1640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8806ab60 0x7f1d8814c220 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1d84007e70 tx=0x7f1d8400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.292+0000 7f1d8fee1640 1 -- 172.21.15.139:0/74608136 shutdown_connections 2024-04-25T10:01:00.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.292+0000 7f1d8fee1640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d6c06b4d0 0x7f1d6c06d990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.292+0000 7f1d8fee1640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d88160e90 0x7f1d880cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.292+0000 7f1d8fee1640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806afc0 0x7f1d881f3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.292+0000 7f1d8fee1640 1 --2- 172.21.15.139:0/74608136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8806ab60 0x7f1d8814c220 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.292+0000 7f1d8fee1640 1 -- 172.21.15.139:0/74608136 >> 172.21.15.139:0/74608136 conn(0x7f1d880b23f0 msgr2=0x7f1d880b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.292+0000 7f1d8fee1640 1 -- 172.21.15.139:0/74608136 shutdown_connections 2024-04-25T10:01:00.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.292+0000 7f1d8fee1640 1 -- 172.21.15.139:0/74608136 wait complete. 2024-04-25T10:01:00.308 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:58:55.585380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:58:56.336230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:00.309 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:00.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.403+0000 7f5f06f96640 1 Processor -- start 2024-04-25T10:01:00.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.403+0000 7f5f06f96640 1 -- start start 2024-04-25T10:01:00.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f06f96640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5f00144090 0x7f5f000b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f06f96640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5f000cc9e0 0x7f5f0006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f06f96640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f000cce50 0x7f5f0006e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f06f96640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f000b2a50 con 0x7f5f00144090 2024-04-25T10:01:00.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f06f96640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0015b090 con 0x7f5f000cc9e0 2024-04-25T10:01:00.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f06f96640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0015b370 con 0x7f5f000cce50 2024-04-25T10:01:00.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f04d32640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5f00144090 0x7f5f000b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f04d32640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5f00144090 0x7f5f000b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54470/0 (socket says 172.21.15.139:54470) 2024-04-25T10:01:00.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f04d32640 1 -- 172.21.15.139:0/1753769166 learned_addr learned my addr 172.21.15.139:0/1753769166 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:00.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f05533640 1 --2- 172.21.15.139:0/1753769166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f000cce50 0x7f5f0006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5efffff640 1 --2- 172.21.15.139:0/1753769166 >> v2:172.21.15.150:3300/0 conn(0x7f5f000cc9e0 0x7f5f0006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f04d32640 1 -- 172.21.15.139:0/1753769166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f000cce50 msgr2=0x7f5f0006e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:00.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f04d32640 1 --2- 172.21.15.139:0/1753769166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f000cce50 0x7f5f0006e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f04d32640 1 -- 172.21.15.139:0/1753769166 >> v2:172.21.15.150:3300/0 conn(0x7f5f000cc9e0 msgr2=0x7f5f0006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:00.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f04d32640 1 --2- 172.21.15.139:0/1753769166 >> v2:172.21.15.150:3300/0 conn(0x7f5f000cc9e0 0x7f5f0006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.404+0000 7f5f04d32640 1 -- 172.21.15.139:0/1753769166 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0015b650 con 0x7f5f00144090 2024-04-25T10:01:00.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5f04d32640 1 --2- 172.21.15.139:0/1753769166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f00144090 0x7f5f000b31a0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5f000b2780 tx=0x7f5ef402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b35015dfd3cff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5eff7fe640 1 -- 172.21.15.139:0/1753769166 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4037070 con 0x7f5f00144090 2024-04-25T10:01:00.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5eff7fe640 1 -- 172.21.15.139:0/1753769166 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef4032440 con 0x7f5f00144090 2024-04-25T10:01:00.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5eff7fe640 1 -- 172.21.15.139:0/1753769166 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4032810 con 0x7f5f00144090 2024-04-25T10:01:00.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5f06f96640 1 -- 172.21.15.139:0/1753769166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f00144090 msgr2=0x7f5f000b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5f06f96640 1 --2- 172.21.15.139:0/1753769166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f00144090 0x7f5f000b31a0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5f000b2780 tx=0x7f5ef402a140 comp rx=0 tx=0).stop 2024-04-25T10:01:00.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5f06f96640 1 -- 172.21.15.139:0/1753769166 shutdown_connections 2024-04-25T10:01:00.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5f06f96640 1 --2- 172.21.15.139:0/1753769166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f000cce50 0x7f5f0006e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5f06f96640 1 --2- 172.21.15.139:0/1753769166 >> v2:172.21.15.150:3300/0 conn(0x7f5f000cc9e0 0x7f5f0006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5f06f96640 1 --2- 172.21.15.139:0/1753769166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f00144090 0x7f5f000b31a0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5f06f96640 1 -- 172.21.15.139:0/1753769166 >> 172.21.15.139:0/1753769166 conn(0x7f5f00067000 msgr2=0x7f5f0006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5f06f96640 1 -- 172.21.15.139:0/1753769166 shutdown_connections 2024-04-25T10:01:00.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.405+0000 7f5f06f96640 1 -- 172.21.15.139:0/1753769166 wait complete. 2024-04-25T10:01:00.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.407+0000 7f5f06f96640 1 Processor -- start 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.407+0000 7f5f06f96640 1 -- start start 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.407+0000 7f5f06f96640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f000cc9e0 0x7f5f0015d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.407+0000 7f5f06f96640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000cce50 0x7f5f00138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.407+0000 7f5f06f96640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f00144090 0x7f5f00159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.407+0000 7f5f06f96640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f00144470 con 0x7f5f000cce50 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.407+0000 7f5f06f96640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0015bc10 con 0x7f5f000cc9e0 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.407+0000 7f5f06f96640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0015ad80 con 0x7f5f00144090 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5efffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000cce50 0x7f5f00138630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5efffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000cce50 0x7f5f00138630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54476/0 (socket says 172.21.15.139:54476) 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5efffff640 1 -- 172.21.15.139:0/4097990492 learned_addr learned my addr 172.21.15.139:0/4097990492 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5f04d32640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f000cc9e0 0x7f5f0015d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5f05533640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f00144090 0x7f5f00159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5efffff640 1 -- 172.21.15.139:0/4097990492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f00144090 msgr2=0x7f5f00159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5efffff640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f00144090 0x7f5f00159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5efffff640 1 -- 172.21.15.139:0/4097990492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f000cc9e0 msgr2=0x7f5f0015d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5efffff640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f000cc9e0 0x7f5f0015d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5efffff640 1 -- 172.21.15.139:0/4097990492 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f00140640 con 0x7f5f000cce50 2024-04-25T10:01:00.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5f04d32640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f000cc9e0 0x7f5f0015d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:00.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5efffff640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000cce50 0x7f5f00138630 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00e8a0 tx=0x7f5eec00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5efd7fa640 1 -- 172.21.15.139:0/4097990492 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec017070 con 0x7f5f000cce50 2024-04-25T10:01:00.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5efd7fa640 1 -- 172.21.15.139:0/4097990492 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec00ee90 con 0x7f5f000cce50 2024-04-25T10:01:00.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5f06f96640 1 -- 172.21.15.139:0/4097990492 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f00140920 con 0x7f5f000cce50 2024-04-25T10:01:00.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5efd7fa640 1 -- 172.21.15.139:0/4097990492 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec00cb80 con 0x7f5f000cce50 2024-04-25T10:01:00.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.408+0000 7f5f06f96640 1 -- 172.21.15.139:0/4097990492 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0015c8a0 con 0x7f5f000cce50 2024-04-25T10:01:00.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.409+0000 7f5f06f96640 1 -- 172.21.15.139:0/4097990492 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f00074520 con 0x7f5f000cce50 2024-04-25T10:01:00.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.410+0000 7f5efd7fa640 1 -- 172.21.15.139:0/4097990492 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5eec01a070 con 0x7f5f000cce50 2024-04-25T10:01:00.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.410+0000 7f5efd7fa640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5ee406b500 0x7f5ee406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.410+0000 7f5efd7fa640 1 -- 172.21.15.139:0/4097990492 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f5eec08a640 con 0x7f5f000cce50 2024-04-25T10:01:00.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.410+0000 7f5f04d32640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5ee406b500 0x7f5ee406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.411+0000 7f5f04d32640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5ee406b500 0x7f5ee406d9c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4034000 tx=0x7f5ef402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.414+0000 7f5efd7fa640 1 -- 172.21.15.139:0/4097990492 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5eec059210 con 0x7f5f000cce50 2024-04-25T10:01:00.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.583+0000 7f5f06f96640 1 -- 172.21.15.139:0/4097990492 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f0006c1a0 con 0x7f5f000cce50 2024-04-25T10:01:00.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.583+0000 7f5efd7fa640 1 -- 172.21.15.139:0/4097990492 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5eec058b10 con 0x7f5f000cce50 2024-04-25T10:01:00.588 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:01:00.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.585+0000 7f5f06f96640 1 -- 172.21.15.139:0/4097990492 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5ee406b500 msgr2=0x7f5ee406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.585+0000 7f5f06f96640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5ee406b500 0x7f5ee406d9c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4034000 tx=0x7f5ef402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:00.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.585+0000 7f5f06f96640 1 -- 172.21.15.139:0/4097990492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000cce50 msgr2=0x7f5f00138630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.585+0000 7f5f06f96640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000cce50 0x7f5f00138630 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00e8a0 tx=0x7f5eec00ed60 comp rx=0 tx=0).stop 2024-04-25T10:01:00.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.585+0000 7f5f06f96640 1 -- 172.21.15.139:0/4097990492 shutdown_connections 2024-04-25T10:01:00.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.585+0000 7f5f06f96640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000cce50 0x7f5f00138630 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.585+0000 7f5f06f96640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5ee406b500 0x7f5ee406d9c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.585+0000 7f5f06f96640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f00144090 0x7f5f00159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.585+0000 7f5f06f96640 1 --2- 172.21.15.139:0/4097990492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f000cc9e0 0x7f5f0015d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.585+0000 7f5f06f96640 1 -- 172.21.15.139:0/4097990492 >> 172.21.15.139:0/4097990492 conn(0x7f5f00067000 msgr2=0x7f5f00143340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.585+0000 7f5f06f96640 1 -- 172.21.15.139:0/4097990492 shutdown_connections 2024-04-25T10:01:00.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.585+0000 7f5f06f96640 1 -- 172.21.15.139:0/4097990492 wait complete. 2024-04-25T10:01:00.601 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:01:00.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.695+0000 7f4dbc653640 1 Processor -- start 2024-04-25T10:01:00.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.695+0000 7f4dbc653640 1 -- start start 2024-04-25T10:01:00.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.696+0000 7f4dbc653640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4db4071a50 0x7f4db4071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.696+0000 7f4dbc653640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db406b980 con 0x7f4db40726c0 2024-04-25T10:01:00.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.696+0000 7f4dbc653640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db4072370 con 0x7f4db40cf0e0 2024-04-25T10:01:00.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.696+0000 7f4dbc653640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db415c500 con 0x7f4db4071a50 2024-04-25T10:01:00.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.697+0000 7f4db9bee640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4db40726c0 0x7f4db406ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37178/0 (socket says 172.21.15.139:37178) 2024-04-25T10:01:00.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.697+0000 7f4db9bee640 1 -- 172.21.15.139:0/2576121100 learned_addr learned my addr 172.21.15.139:0/2576121100 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:00.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.697+0000 7f4dba3ef640 1 --2- 172.21.15.139:0/2576121100 >> v2:172.21.15.150:3300/0 conn(0x7f4db4071a50 0x7f4db4071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.697+0000 7f4db93ed640 1 -- 172.21.15.139:0/2576121100 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385026051 0 0) 0x7f4db406b980 con 0x7f4db40726c0 2024-04-25T10:01:00.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.697+0000 7f4db93ed640 1 -- 172.21.15.139:0/2576121100 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4da0003520 con 0x7f4db40726c0 2024-04-25T10:01:00.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.697+0000 7f4dba3ef640 1 -- 172.21.15.139:0/2576121100 >> v1:172.21.15.150:6789/0 conn(0x7f4db40cf0e0 legacy=0x7f4db406b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.697+0000 7f4dba3ef640 1 -- 172.21.15.139:0/2576121100 >> v1:172.21.15.139:6789/0 conn(0x7f4db40726c0 legacy=0x7f4db406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.697+0000 7f4db93ed640 1 -- 172.21.15.139:0/2576121100 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3663651144 0 0) 0x7f4da0003520 con 0x7f4db40726c0 2024-04-25T10:01:00.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.697+0000 7f4dba3ef640 1 -- 172.21.15.139:0/2576121100 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db415c7e0 con 0x7f4db4071a50 2024-04-25T10:01:00.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.698+0000 7f4dba3ef640 1 --2- 172.21.15.139:0/2576121100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db4071a50 0x7f4db4071e30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4db00097a0 tx=0x7f4db002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5a05d5c5024bddb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.698+0000 7f4db93ed640 1 -- 172.21.15.139:0/2576121100 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0037070 con 0x7f4db4071a50 2024-04-25T10:01:00.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.698+0000 7f4db93ed640 1 -- 172.21.15.139:0/2576121100 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db002ac50 con 0x7f4db4071a50 2024-04-25T10:01:00.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.698+0000 7f4db93ed640 1 -- 172.21.15.139:0/2576121100 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da4001670 con 0x7f4db4071a50 2024-04-25T10:01:00.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.699+0000 7f4dbc653640 1 -- 172.21.15.139:0/2576121100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db4071a50 msgr2=0x7f4db4071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.699+0000 7f4dbc653640 1 --2- 172.21.15.139:0/2576121100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db4071a50 0x7f4db4071e30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4db00097a0 tx=0x7f4db002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.699+0000 7f4dbc653640 1 -- 172.21.15.139:0/2576121100 shutdown_connections 2024-04-25T10:01:00.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.699+0000 7f4dbc653640 1 --2- 172.21.15.139:0/2576121100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db4071a50 0x7f4db4071e30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.699+0000 7f4dbc653640 1 -- 172.21.15.139:0/2576121100 >> 172.21.15.139:0/2576121100 conn(0x7f4db40cfd80 msgr2=0x7f4db40d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.699+0000 7f4dbc653640 1 -- 172.21.15.139:0/2576121100 shutdown_connections 2024-04-25T10:01:00.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.699+0000 7f4dbc653640 1 -- 172.21.15.139:0/2576121100 wait complete. 2024-04-25T10:01:00.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.700+0000 7f4dbc653640 1 Processor -- start 2024-04-25T10:01:00.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.700+0000 7f4dbc653640 1 -- start start 2024-04-25T10:01:00.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbc653640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db4071a50 0x7f4db4154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbc653640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db40cf0e0 0x7f4db4146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbc653640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db409d210 0x7f4db4148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbc653640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4156280 con 0x7f4db409d210 2024-04-25T10:01:00.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbc653640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db415cda0 con 0x7f4db4071a50 2024-04-25T10:01:00.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbc653640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4156400 con 0x7f4db40cf0e0 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbabf0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db409d210 0x7f4db4148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbabf0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db409d210 0x7f4db4148130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54492/0 (socket says 172.21.15.139:54492) 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbabf0640 1 -- 172.21.15.139:0/478610094 learned_addr learned my addr 172.21.15.139:0/478610094 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dba3ef640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db4071a50 0x7f4db4154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4db9bee640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db40cf0e0 0x7f4db4146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbabf0640 1 -- 172.21.15.139:0/478610094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db40cf0e0 msgr2=0x7f4db4146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbabf0640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db40cf0e0 0x7f4db4146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbabf0640 1 -- 172.21.15.139:0/478610094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db4071a50 msgr2=0x7f4db4154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbabf0640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db4071a50 0x7f4db4154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbabf0640 1 -- 172.21.15.139:0/478610094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db4147300 con 0x7f4db409d210 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbabf0640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db409d210 0x7f4db4148130 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00c730 tx=0x7f4dac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4daaffd640 1 -- 172.21.15.139:0/478610094 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac003b70 con 0x7f4db409d210 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4daaffd640 1 -- 172.21.15.139:0/478610094 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac003d30 con 0x7f4db409d210 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbc653640 1 -- 172.21.15.139:0/478610094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db41475e0 con 0x7f4db409d210 2024-04-25T10:01:00.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4daaffd640 1 -- 172.21.15.139:0/478610094 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac005310 con 0x7f4db409d210 2024-04-25T10:01:00.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.701+0000 7f4dbc653640 1 -- 172.21.15.139:0/478610094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db4147ba0 con 0x7f4db409d210 2024-04-25T10:01:00.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.702+0000 7f4dbc653640 1 -- 172.21.15.139:0/478610094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db40b37d0 con 0x7f4db409d210 2024-04-25T10:01:00.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.703+0000 7f4daaffd640 1 -- 172.21.15.139:0/478610094 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4dac0054b0 con 0x7f4db409d210 2024-04-25T10:01:00.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.703+0000 7f4daaffd640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d9806b500 0x7f4d9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.703+0000 7f4daaffd640 1 -- 172.21.15.139:0/478610094 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f4dac08a1d0 con 0x7f4db409d210 2024-04-25T10:01:00.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.703+0000 7f4dba3ef640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d9806b500 0x7f4d9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.704+0000 7f4dba3ef640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d9806b500 0x7f4d9806d9c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4db0034000 tx=0x7f4db002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.707+0000 7f4daaffd640 1 -- 172.21.15.139:0/478610094 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4db40b37d0 con 0x7f4db409d210 2024-04-25T10:01:00.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.875+0000 7f4dbc653640 1 -- 172.21.15.139:0/478610094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4db4074520 con 0x7f4db409d210 2024-04-25T10:01:00.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.876+0000 7f4daaffd640 1 -- 172.21.15.139:0/478610094 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4dac058750 con 0x7f4db409d210 2024-04-25T10:01:00.881 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T10:01:00.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.877+0000 7f4dbc653640 1 -- 172.21.15.139:0/478610094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d9806b500 msgr2=0x7f4d9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.877+0000 7f4dbc653640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d9806b500 0x7f4d9806d9c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4db0034000 tx=0x7f4db002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:00.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.877+0000 7f4dbc653640 1 -- 172.21.15.139:0/478610094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db409d210 msgr2=0x7f4db4148130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.877+0000 7f4dbc653640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db409d210 0x7f4db4148130 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00c730 tx=0x7f4dac00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.878+0000 7f4dbc653640 1 -- 172.21.15.139:0/478610094 shutdown_connections 2024-04-25T10:01:00.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.878+0000 7f4dbc653640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db409d210 0x7f4db4148130 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.878+0000 7f4dbc653640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d9806b500 0x7f4d9806d9c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.878+0000 7f4dbc653640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db40cf0e0 0x7f4db4146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.878+0000 7f4dbc653640 1 --2- 172.21.15.139:0/478610094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db4071a50 0x7f4db4154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.878+0000 7f4dbc653640 1 -- 172.21.15.139:0/478610094 >> 172.21.15.139:0/478610094 conn(0x7f4db40cfd80 msgr2=0x7f4db40afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.878+0000 7f4dbc653640 1 -- 172.21.15.139:0/478610094 shutdown_connections 2024-04-25T10:01:00.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.878+0000 7f4dbc653640 1 -- 172.21.15.139:0/478610094 wait complete. 2024-04-25T10:01:00.894 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:01:00.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.989+0000 7f7b79f63640 1 Processor -- start 2024-04-25T10:01:00.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.989+0000 7f7b79f63640 1 -- start start 2024-04-25T10:01:00.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.990+0000 7f7b79f63640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b740b5620 0x7f7b740b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.990+0000 7f7b79f63640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b740d0d40 con 0x7f7b7415a870 2024-04-25T10:01:00.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.990+0000 7f7b79f63640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b740b4b40 con 0x7f7b74070970 2024-04-25T10:01:00.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.990+0000 7f7b79f63640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740b5a70 con 0x7f7b740b5620 2024-04-25T10:01:00.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.990+0000 7f7b737fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f7b7415a870 0x7f7b74070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37184/0 (socket says 172.21.15.139:37184) 2024-04-25T10:01:00.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.990+0000 7f7b737fe640 1 -- 172.21.15.139:0/1659811271 learned_addr learned my addr 172.21.15.139:0/1659811271 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:00.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.990+0000 7f7b73fff640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b740b5620 0x7f7b740b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.990+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507955784 0 0) 0x7f7b740d0d40 con 0x7f7b7415a870 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.990+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b5c003520 con 0x7f7b7415a870 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 504036513 0 0) 0x7f7b5c003520 con 0x7f7b7415a870 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b740b5620 msgr2=0x7f7b740b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b740b5620 0x7f7b740b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 >> v1:172.21.15.150:6789/0 conn(0x7f7b74070970 legacy=0x7f7b740ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b740b4e20 con 0x7f7b7415a870 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f7b64001690 con 0x7f7b7415a870 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 >> v1:172.21.15.139:6789/0 conn(0x7f7b7415a870 legacy=0x7f7b74070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b740b5620 0x7f7b5c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b5c003f20 0x7f7b5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b5c004d40 0x7f7b5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740b5a70 con 0x7f7b5c003f20 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c00bad0 con 0x7f7b5c004d40 2024-04-25T10:01:00.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c00bcd0 con 0x7f7b740b5620 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b72ffd640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b5c003f20 0x7f7b5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b737fe640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b740b5620 0x7f7b5c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b73fff640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b5c004d40 0x7f7b5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b72ffd640 1 -- 172.21.15.139:0/1659811271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b740b5620 msgr2=0x7f7b5c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b72ffd640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b740b5620 0x7f7b5c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b72ffd640 1 -- 172.21.15.139:0/1659811271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b5c004d40 msgr2=0x7f7b5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b72ffd640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b5c004d40 0x7f7b5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b72ffd640 1 -- 172.21.15.139:0/1659811271 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c003520 con 0x7f7b5c003f20 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.991+0000 7f7b73fff640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b5c004d40 0x7f7b5c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.992+0000 7f7b72ffd640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b5c003f20 0x7f7b5c004800 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f7b68001860 tx=0x7f7b5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25301c295e51c84b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.992+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b68037070 con 0x7f7b5c003f20 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.992+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6802abf0 con 0x7f7b5c003f20 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.992+0000 7f7b727fc640 1 -- 172.21.15.139:0/1659811271 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b68001430 con 0x7f7b5c003f20 2024-04-25T10:01:01.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.993+0000 7f7b79f63640 1 -- 172.21.15.139:0/1659811271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b5c003f20 msgr2=0x7f7b5c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.993+0000 7f7b79f63640 1 --2- 172.21.15.139:0/1659811271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b5c003f20 0x7f7b5c004800 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f7b68001860 tx=0x7f7b5c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:01.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.993+0000 7f7b73fff640 1 -- 172.21.15.139:0/1659811271 reap_dead start 2024-04-25T10:01:01.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.993+0000 7f7b79f63640 1 -- 172.21.15.139:0/1659811271 shutdown_connections 2024-04-25T10:01:01.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.993+0000 7f7b79f63640 1 -- 172.21.15.139:0/1659811271 >> 172.21.15.139:0/1659811271 conn(0x7f7b740b23f0 msgr2=0x7f7b740b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:01.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.993+0000 7f7b79f63640 1 -- 172.21.15.139:0/1659811271 shutdown_connections 2024-04-25T10:01:01.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.993+0000 7f7b79f63640 1 -- 172.21.15.139:0/1659811271 wait complete. 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.994+0000 7f7b79f63640 1 Processor -- start 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.994+0000 7f7b79f63640 1 -- start start 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b79f63640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b5c003f20 0x7f7b7414dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b79f63640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b7415bd60 0x7f7b74155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b79f63640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b7415d2e0 0x7f7b7415d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b79f63640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740b4520 con 0x7f7b5c003f20 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b79f63640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740b43a0 con 0x7f7b7415bd60 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b79f63640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740d1020 con 0x7f7b7415d2e0 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b737fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b5c003f20 0x7f7b7414dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b737fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b5c003f20 0x7f7b7414dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54510/0 (socket says 172.21.15.139:54510) 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b737fe640 1 -- 172.21.15.139:0/1445837783 learned_addr learned my addr 172.21.15.139:0/1445837783 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b72ffd640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b7415bd60 0x7f7b74155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b737fe640 1 -- 172.21.15.139:0/1445837783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b7415d2e0 msgr2=0x7f7b7415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b737fe640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b7415d2e0 0x7f7b7415d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b737fe640 1 -- 172.21.15.139:0/1445837783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b7415bd60 msgr2=0x7f7b74155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b737fe640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b7415bd60 0x7f7b74155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b737fe640 1 -- 172.21.15.139:0/1445837783 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7414e1c0 con 0x7f7b5c003f20 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b72ffd640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b7415bd60 0x7f7b74155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b737fe640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b5c003f20 0x7f7b7414dc80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f7b64007fd0 tx=0x7f7b64007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b5bfff640 1 -- 172.21.15.139:0/1445837783 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400f730 con 0x7f7b5c003f20 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b5bfff640 1 -- 172.21.15.139:0/1445837783 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b64018070 con 0x7f7b5c003f20 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.995+0000 7f7b79f63640 1 -- 172.21.15.139:0/1445837783 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7415c240 con 0x7f7b5c003f20 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.996+0000 7f7b5bfff640 1 -- 172.21.15.139:0/1445837783 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400fb50 con 0x7f7b5c003f20 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.996+0000 7f7b79f63640 1 -- 172.21.15.139:0/1445837783 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7415c800 con 0x7f7b5c003f20 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.996+0000 7f7b79f63640 1 -- 172.21.15.139:0/1445837783 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b74074520 con 0x7f7b5c003f20 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.997+0000 7f7b5bfff640 1 -- 172.21.15.139:0/1445837783 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b6400fcf0 con 0x7f7b5c003f20 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.997+0000 7f7b5bfff640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7b6006b4d0 0x7f7b6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.997+0000 7f7b5bfff640 1 -- 172.21.15.139:0/1445837783 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f7b6408a3d0 con 0x7f7b5c003f20 2024-04-25T10:01:01.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.997+0000 7f7b72ffd640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7b6006b4d0 0x7f7b6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:00.998+0000 7f7b72ffd640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7b6006b4d0 0x7f7b6006d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7b68034000 tx=0x7f7b68009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.002+0000 7f7b5bfff640 1 -- 172.21.15.139:0/1445837783 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b64059030 con 0x7f7b5c003f20 2024-04-25T10:01:01.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.174+0000 7f7b79f63640 1 -- 172.21.15.139:0/1445837783 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b74157740 con 0x7f7b5c003f20 2024-04-25T10:01:01.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.174+0000 7f7b5bfff640 1 -- 172.21.15.139:0/1445837783 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b64058930 con 0x7f7b5c003f20 2024-04-25T10:01:01.179 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T10:01:01.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.176+0000 7f7b79f63640 1 -- 172.21.15.139:0/1445837783 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7b6006b4d0 msgr2=0x7f7b6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.176+0000 7f7b79f63640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7b6006b4d0 0x7f7b6006d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7b68034000 tx=0x7f7b68009030 comp rx=0 tx=0).stop 2024-04-25T10:01:01.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.176+0000 7f7b79f63640 1 -- 172.21.15.139:0/1445837783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b5c003f20 msgr2=0x7f7b7414dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.176+0000 7f7b79f63640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b5c003f20 0x7f7b7414dc80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f7b64007fd0 tx=0x7f7b64007430 comp rx=0 tx=0).stop 2024-04-25T10:01:01.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.176+0000 7f7b79f63640 1 -- 172.21.15.139:0/1445837783 shutdown_connections 2024-04-25T10:01:01.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.176+0000 7f7b79f63640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7b6006b4d0 0x7f7b6006d990 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.176+0000 7f7b79f63640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b7415d2e0 0x7f7b7415d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.176+0000 7f7b79f63640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b7415bd60 0x7f7b74155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.176+0000 7f7b79f63640 1 --2- 172.21.15.139:0/1445837783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b5c003f20 0x7f7b7414dc80 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.176+0000 7f7b79f63640 1 -- 172.21.15.139:0/1445837783 >> 172.21.15.139:0/1445837783 conn(0x7f7b740b23f0 msgr2=0x7f7b74072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:01.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.176+0000 7f7b79f63640 1 -- 172.21.15.139:0/1445837783 shutdown_connections 2024-04-25T10:01:01.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.177+0000 7f7b79f63640 1 -- 172.21.15.139:0/1445837783 wait complete. 2024-04-25T10:01:01.192 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:01:01.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.285+0000 7f1cbe1f9640 1 Processor -- start 2024-04-25T10:01:01.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.285+0000 7f1cbe1f9640 1 -- start start 2024-04-25T10:01:01.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.286+0000 7f1cbe1f9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1cb8145250 0x7f1cb8070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.286+0000 7f1cbe1f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb80b5690 0x7f1cb8071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.286+0000 7f1cbe1f9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb80b1100 con 0x7f1cb80719d0 2024-04-25T10:01:01.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.286+0000 7f1cbe1f9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb80ceed0 con 0x7f1cb8145250 2024-04-25T10:01:01.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.286+0000 7f1cbe1f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb80ce2c0 con 0x7f1cb80b5690 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.286+0000 7f1cb6ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1cb80719d0 0x7f1cb815f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37188/0 (socket says 172.21.15.139:37188) 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.286+0000 7f1cb6ffd640 1 -- 172.21.15.139:0/895423609 learned_addr learned my addr 172.21.15.139:0/895423609 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb7fff640 1 --2- 172.21.15.139:0/895423609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb80b5690 0x7f1cb8071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875381038 0 0) 0x7f1cb80b1100 con 0x7f1cb80719d0 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb77fe640 1 --2- 172.21.15.139:0/895423609 >> v2:172.21.15.150:3300/0 conn(0x7f1cb8145250 0x7f1cb8070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ca0003520 con 0x7f1cb80719d0 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1816415295 0 0) 0x7f1ca0003520 con 0x7f1cb80719d0 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb80b5690 msgr2=0x7f1cb8071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 --2- 172.21.15.139:0/895423609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb80b5690 0x7f1cb8071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 >> v2:172.21.15.150:3300/0 conn(0x7f1cb8145250 msgr2=0x7f1cb8070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 --2- 172.21.15.139:0/895423609 >> v2:172.21.15.150:3300/0 conn(0x7f1cb8145250 0x7f1cb8070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb80ce5a0 con 0x7f1cb80719d0 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f1cac001670 con 0x7f1cb80719d0 2024-04-25T10:01:01.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 >> v1:172.21.15.139:6789/0 conn(0x7f1cb80719d0 legacy=0x7f1cb815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 --2- 172.21.15.139:0/895423609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca00039b0 0x7f1ca0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 --2- 172.21.15.139:0/895423609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ca0004800 0x7f1ca0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 --2- 172.21.15.139:0/895423609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb80b5690 0x7f1ca00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb80ceed0 con 0x7f1ca0004800 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb80ce2c0 con 0x7f1ca00039b0 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca000b9f0 con 0x7f1cb80b5690 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb6ffd640 1 --2- 172.21.15.139:0/895423609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ca0004800 0x7f1ca0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb77fe640 1 --2- 172.21.15.139:0/895423609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca00039b0 0x7f1ca0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb7fff640 1 --2- 172.21.15.139:0/895423609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb80b5690 0x7f1ca00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb6ffd640 1 -- 172.21.15.139:0/895423609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb80b5690 msgr2=0x7f1ca00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb6ffd640 1 --2- 172.21.15.139:0/895423609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb80b5690 0x7f1ca00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb6ffd640 1 -- 172.21.15.139:0/895423609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca00039b0 msgr2=0x7f1ca0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb6ffd640 1 --2- 172.21.15.139:0/895423609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca00039b0 0x7f1ca0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.287+0000 7f1cb6ffd640 1 -- 172.21.15.139:0/895423609 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca0003520 con 0x7f1ca0004800 2024-04-25T10:01:01.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.288+0000 7f1cb6ffd640 1 --2- 172.21.15.139:0/895423609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ca0004800 0x7f1ca0004c80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00c240 tx=0x7f1ca0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7a00a7499a564 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.288+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac007d30 con 0x7f1ca0004800 2024-04-25T10:01:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.288+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac005ad0 con 0x7f1ca0004800 2024-04-25T10:01:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.288+0000 7f1cb67fc640 1 -- 172.21.15.139:0/895423609 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac007070 con 0x7f1ca0004800 2024-04-25T10:01:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.289+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/895423609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ca0004800 msgr2=0x7f1ca0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.289+0000 7f1cbe1f9640 1 --2- 172.21.15.139:0/895423609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ca0004800 0x7f1ca0004c80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00c240 tx=0x7f1ca0005830 comp rx=0 tx=0).stop 2024-04-25T10:01:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.289+0000 7f1cb7fff640 1 -- 172.21.15.139:0/895423609 reap_dead start 2024-04-25T10:01:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.289+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/895423609 shutdown_connections 2024-04-25T10:01:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.289+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/895423609 >> 172.21.15.139:0/895423609 conn(0x7f1cb806f460 msgr2=0x7f1cb806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.289+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/895423609 shutdown_connections 2024-04-25T10:01:01.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.289+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/895423609 wait complete. 2024-04-25T10:01:01.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.290+0000 7f1cbe1f9640 1 Processor -- start 2024-04-25T10:01:01.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.290+0000 7f1cbe1f9640 1 -- start start 2024-04-25T10:01:01.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cbe1f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ca0004800 0x7f1cb80b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cbe1f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb806cc80 0x7f1cb80a8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cbe1f9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb80a8a60 0x7f1cb81e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cbe1f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb8145630 con 0x7f1ca0004800 2024-04-25T10:01:01.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cbe1f9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb80ceb60 con 0x7f1cb80a8a60 2024-04-25T10:01:01.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cbe1f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb80d11a0 con 0x7f1cb806cc80 2024-04-25T10:01:01.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cb77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ca0004800 0x7f1cb80b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cb77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ca0004800 0x7f1cb80b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54520/0 (socket says 172.21.15.139:54520) 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cb77fe640 1 -- 172.21.15.139:0/2674486069 learned_addr learned my addr 172.21.15.139:0/2674486069 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cb6ffd640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb806cc80 0x7f1cb80a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cb7fff640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb80a8a60 0x7f1cb81e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cb77fe640 1 -- 172.21.15.139:0/2674486069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb806cc80 msgr2=0x7f1cb80a8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cb77fe640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb806cc80 0x7f1cb80a8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cb77fe640 1 -- 172.21.15.139:0/2674486069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb80a8a60 msgr2=0x7f1cb81e2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cb77fe640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb80a8a60 0x7f1cb81e2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cb77fe640 1 -- 172.21.15.139:0/2674486069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb806bec0 con 0x7f1ca0004800 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cb7fff640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb80a8a60 0x7f1cb81e2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.291+0000 7f1cb77fe640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ca0004800 0x7f1cb80b1560 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80119c0 tx=0x7f1ca8011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.292+0000 7f1c9ffff640 1 -- 172.21.15.139:0/2674486069 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca800fb50 con 0x7f1ca0004800 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.292+0000 7f1c9ffff640 1 -- 172.21.15.139:0/2674486069 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca800fcf0 con 0x7f1ca0004800 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.292+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/2674486069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb806c1a0 con 0x7f1ca0004800 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.292+0000 7f1c9ffff640 1 -- 172.21.15.139:0/2674486069 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca800fea0 con 0x7f1ca0004800 2024-04-25T10:01:01.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.292+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/2674486069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb806c760 con 0x7f1ca0004800 2024-04-25T10:01:01.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.293+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/2674486069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb8074520 con 0x7f1ca0004800 2024-04-25T10:01:01.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.293+0000 7f1c9ffff640 1 -- 172.21.15.139:0/2674486069 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ca8012040 con 0x7f1ca0004800 2024-04-25T10:01:01.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.293+0000 7f1c9ffff640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1ca406b4d0 0x7f1ca406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.293+0000 7f1c9ffff640 1 -- 172.21.15.139:0/2674486069 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f1ca808ab40 con 0x7f1ca0004800 2024-04-25T10:01:01.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.293+0000 7f1cb6ffd640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1ca406b4d0 0x7f1ca406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.294+0000 7f1cb6ffd640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1ca406b4d0 0x7f1ca406d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1cac033000 tx=0x7f1cac009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.298+0000 7f1c9ffff640 1 -- 172.21.15.139:0/2674486069 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ca8019030 con 0x7f1ca0004800 2024-04-25T10:01:01.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.478+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/2674486069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1cb80701f0 con 0x7f1ca0004800 2024-04-25T10:01:01.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.479+0000 7f1c9ffff640 1 -- 172.21.15.139:0/2674486069 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v93) v1 ==== 80+0+248 (secure 0 0 0) 0x7f1ca80597a0 con 0x7f1ca0004800 2024-04-25T10:01:01.484 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6836/1239243954 2024-04-26T10:00:22.118970+0000 2024-04-25T10:01:01.484 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6834/1239243954 2024-04-26T10:00:22.118970+0000 2024-04-25T10:01:01.484 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.150:6835/2266713846 2024-04-26T09:59:10.897062+0000 2024-04-25T10:01:01.485 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.150:6833/2266713846 2024-04-26T09:59:10.897062+0000 2024-04-25T10:01:01.485 INFO:teuthology.orchestra.run.smithi139.stderr:listed 4 entries 2024-04-25T10:01:01.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.481+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/2674486069 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1ca406b4d0 msgr2=0x7f1ca406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.481+0000 7f1cbe1f9640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1ca406b4d0 0x7f1ca406d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1cac033000 tx=0x7f1cac009030 comp rx=0 tx=0).stop 2024-04-25T10:01:01.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.481+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/2674486069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ca0004800 msgr2=0x7f1cb80b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.481+0000 7f1cbe1f9640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ca0004800 0x7f1cb80b1560 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80119c0 tx=0x7f1ca8011e80 comp rx=0 tx=0).stop 2024-04-25T10:01:01.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.481+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/2674486069 shutdown_connections 2024-04-25T10:01:01.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.481+0000 7f1cbe1f9640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1ca406b4d0 0x7f1ca406d990 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.481+0000 7f1cbe1f9640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cb80a8a60 0x7f1cb81e2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.481+0000 7f1cbe1f9640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cb806cc80 0x7f1cb80a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.481+0000 7f1cbe1f9640 1 --2- 172.21.15.139:0/2674486069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ca0004800 0x7f1cb80b1560 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.481+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/2674486069 >> 172.21.15.139:0/2674486069 conn(0x7f1cb806f460 msgr2=0x7f1cb8144340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:01.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.481+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/2674486069 shutdown_connections 2024-04-25T10:01:01.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.481+0000 7f1cbe1f9640 1 -- 172.21.15.139:0/2674486069 wait complete. 2024-04-25T10:01:01.497 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:01:01.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:01.497 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:01.526 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:01:01.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:01.526 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:01:01.581 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T10:01:01.581 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:01.600 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:01.600 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:01.600 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:01.636 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:01:01.636 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T10:01:01.636 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:01.657 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:01.657 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:01.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.767+0000 7fd7e2f53640 1 Processor -- start 2024-04-25T10:01:01.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.767+0000 7fd7e2f53640 1 -- start start 2024-04-25T10:01:01.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.768+0000 7fd7e2f53640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd7dc158240 0x7fd7dc0d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.768+0000 7fd7e2f53640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7dc0d1860 0x7fd7dc0cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.768+0000 7fd7e2f53640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd7dc0cfba0 0x7fd7dc1616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.768+0000 7fd7e2f53640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7dc158620 con 0x7fd7dc0cfba0 2024-04-25T10:01:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.768+0000 7fd7e2f53640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7dc0cce10 con 0x7fd7dc158240 2024-04-25T10:01:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.768+0000 7fd7e2f53640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc0cbf80 con 0x7fd7dc0d1860 2024-04-25T10:01:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.768+0000 7fd7e14f0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd7dc0cfba0 0x7fd7dc1616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.768+0000 7fd7e14f0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd7dc0cfba0 0x7fd7dc1616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54534/0 (socket says 172.21.15.139:54534) 2024-04-25T10:01:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.768+0000 7fd7e14f0640 1 -- 172.21.15.139:0/4043536352 learned_addr learned my addr 172.21.15.139:0/4043536352 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.769+0000 7fd7e0cef640 1 --2- 172.21.15.139:0/4043536352 >> v2:172.21.15.150:3300/0 conn(0x7fd7dc158240 0x7fd7dc0d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.769+0000 7fd7dbfff640 1 --2- 172.21.15.139:0/4043536352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7dc0d1860 0x7fd7dc0cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.769+0000 7fd7e14f0640 1 -- 172.21.15.139:0/4043536352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7dc0d1860 msgr2=0x7fd7dc0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.769+0000 7fd7e14f0640 1 --2- 172.21.15.139:0/4043536352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7dc0d1860 0x7fd7dc0cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.769+0000 7fd7e14f0640 1 -- 172.21.15.139:0/4043536352 >> v2:172.21.15.150:3300/0 conn(0x7fd7dc158240 msgr2=0x7fd7dc0d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.769+0000 7fd7e14f0640 1 --2- 172.21.15.139:0/4043536352 >> v2:172.21.15.150:3300/0 conn(0x7fd7dc158240 0x7fd7dc0d1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.769+0000 7fd7e14f0640 1 -- 172.21.15.139:0/4043536352 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc0cd2a0 con 0x7fd7dc0cfba0 2024-04-25T10:01:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.769+0000 7fd7dbfff640 1 --2- 172.21.15.139:0/4043536352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7dc0d1860 0x7fd7dc0cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.769+0000 7fd7e14f0640 1 --2- 172.21.15.139:0/4043536352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7dc0cfba0 0x7fd7dc1616d0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc072d20 tx=0x7fd7d402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8edfe8c3e114b97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.770+0000 7fd7db7fe640 1 -- 172.21.15.139:0/4043536352 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4036070 con 0x7fd7dc0cfba0 2024-04-25T10:01:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.770+0000 7fd7db7fe640 1 -- 172.21.15.139:0/4043536352 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d4031600 con 0x7fd7dc0cfba0 2024-04-25T10:01:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.770+0000 7fd7db7fe640 1 -- 172.21.15.139:0/4043536352 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d40319d0 con 0x7fd7dc0cfba0 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.770+0000 7fd7e2f53640 1 -- 172.21.15.139:0/4043536352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7dc0cfba0 msgr2=0x7fd7dc1616d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.770+0000 7fd7e2f53640 1 --2- 172.21.15.139:0/4043536352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7dc0cfba0 0x7fd7dc1616d0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc072d20 tx=0x7fd7d402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.770+0000 7fd7e2f53640 1 -- 172.21.15.139:0/4043536352 shutdown_connections 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.770+0000 7fd7e2f53640 1 --2- 172.21.15.139:0/4043536352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7dc0cfba0 0x7fd7dc1616d0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.770+0000 7fd7e2f53640 1 --2- 172.21.15.139:0/4043536352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7dc0d1860 0x7fd7dc0cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.770+0000 7fd7e2f53640 1 --2- 172.21.15.139:0/4043536352 >> v2:172.21.15.150:3300/0 conn(0x7fd7dc158240 0x7fd7dc0d1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.770+0000 7fd7e2f53640 1 -- 172.21.15.139:0/4043536352 >> 172.21.15.139:0/4043536352 conn(0x7fd7dc0b3380 msgr2=0x7fd7dc0b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.770+0000 7fd7e2f53640 1 -- 172.21.15.139:0/4043536352 shutdown_connections 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.770+0000 7fd7e2f53640 1 -- 172.21.15.139:0/4043536352 wait complete. 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.771+0000 7fd7e2f53640 1 Processor -- start 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.771+0000 7fd7e2f53640 1 -- start start 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.772+0000 7fd7e2f53640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7dc0cfba0 0x7fd7dc0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.772+0000 7fd7e2f53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7dc0d1860 0x7fd7dc0b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.772+0000 7fd7e2f53640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7dc158240 0x7fd7dc0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.772+0000 7fd7e2f53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc0cc7c0 con 0x7fd7dc0d1860 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.772+0000 7fd7e2f53640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc0cc4e0 con 0x7fd7dc0cfba0 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.772+0000 7fd7e2f53640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc0ccaa0 con 0x7fd7dc158240 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.772+0000 7fd7dbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7dc0d1860 0x7fd7dc0b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.772+0000 7fd7dbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7dc0d1860 0x7fd7dc0b6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54540/0 (socket says 172.21.15.139:54540) 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.772+0000 7fd7dbfff640 1 -- 172.21.15.139:0/2180655216 learned_addr learned my addr 172.21.15.139:0/2180655216 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7e14f0640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7dc158240 0x7fd7dc0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7e0cef640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7dc0cfba0 0x7fd7dc0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7dbfff640 1 -- 172.21.15.139:0/2180655216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7dc158240 msgr2=0x7fd7dc0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7dbfff640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7dc158240 0x7fd7dc0cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7dbfff640 1 -- 172.21.15.139:0/2180655216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7dc0cfba0 msgr2=0x7fd7dc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7dbfff640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7dc0cfba0 0x7fd7dc0c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7dbfff640 1 -- 172.21.15.139:0/2180655216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc0cb9c0 con 0x7fd7dc0d1860 2024-04-25T10:01:01.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7dbfff640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7dc0d1860 0x7fd7dc0b6eb0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000e810 tx=0x7fd7d000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7d97fa640 1 -- 172.21.15.139:0/2180655216 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d000f040 con 0x7fd7dc0d1860 2024-04-25T10:01:01.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7d97fa640 1 -- 172.21.15.139:0/2180655216 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d0004030 con 0x7fd7dc0d1860 2024-04-25T10:01:01.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7e2f53640 1 -- 172.21.15.139:0/2180655216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7dc0cb400 con 0x7fd7dc0d1860 2024-04-25T10:01:01.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7d97fa640 1 -- 172.21.15.139:0/2180655216 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0004400 con 0x7fd7dc0d1860 2024-04-25T10:01:01.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.773+0000 7fd7e2f53640 1 -- 172.21.15.139:0/2180655216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7dc159850 con 0x7fd7dc0d1860 2024-04-25T10:01:01.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.774+0000 7fd7e2f53640 1 -- 172.21.15.139:0/2180655216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7dc074520 con 0x7fd7dc0d1860 2024-04-25T10:01:01.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.775+0000 7fd7d97fa640 1 -- 172.21.15.139:0/2180655216 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7d000c6d0 con 0x7fd7dc0d1860 2024-04-25T10:01:01.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.775+0000 7fd7d97fa640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7c806b4d0 0x7fd7c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.775+0000 7fd7e0cef640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7c806b4d0 0x7fd7c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.775+0000 7fd7d97fa640 1 -- 172.21.15.139:0/2180655216 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd7d008a180 con 0x7fd7dc0d1860 2024-04-25T10:01:01.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.775+0000 7fd7e0cef640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7c806b4d0 0x7fd7c806d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0d2130 tx=0x7fd7cc009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.779+0000 7fd7d97fa640 1 -- 172.21.15.139:0/2180655216 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7d0017020 con 0x7fd7dc0d1860 2024-04-25T10:01:01.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.986+0000 7fd7e2f53640 1 -- 172.21.15.139:0/2180655216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7dc15d1d0 con 0x7fd7dc0d1860 2024-04-25T10:01:01.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.986+0000 7fd7d97fa640 1 -- 172.21.15.139:0/2180655216 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4953 (secure 0 0 0) 0x7fd7d0058790 con 0x7fd7dc0d1860 2024-04-25T10:01:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:01.992 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T10:00:25.131250+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6836/1457343244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1457343244},{"type":"v1","addr":"172.21.15.139:6836","nonce":1457343244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:01:01.992 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 63 2024-04-25T10:01:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.988+0000 7fd7e2f53640 1 -- 172.21.15.139:0/2180655216 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7c806b4d0 msgr2=0x7fd7c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.988+0000 7fd7e2f53640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7c806b4d0 0x7fd7c806d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0d2130 tx=0x7fd7cc009190 comp rx=0 tx=0).stop 2024-04-25T10:01:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.988+0000 7fd7e2f53640 1 -- 172.21.15.139:0/2180655216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7dc0d1860 msgr2=0x7fd7dc0b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.988+0000 7fd7e2f53640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7dc0d1860 0x7fd7dc0b6eb0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000e810 tx=0x7fd7d000ecd0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.988+0000 7fd7e2f53640 1 -- 172.21.15.139:0/2180655216 shutdown_connections 2024-04-25T10:01:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.988+0000 7fd7e2f53640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7dc0d1860 0x7fd7dc0b6eb0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.988+0000 7fd7e2f53640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7c806b4d0 0x7fd7c806d990 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.988+0000 7fd7e2f53640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7dc158240 0x7fd7dc0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.988+0000 7fd7e2f53640 1 --2- 172.21.15.139:0/2180655216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7dc0cfba0 0x7fd7dc0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.988+0000 7fd7e2f53640 1 -- 172.21.15.139:0/2180655216 >> 172.21.15.139:0/2180655216 conn(0x7fd7dc0b3380 msgr2=0x7fd7dc15e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.988+0000 7fd7e2f53640 1 -- 172.21.15.139:0/2180655216 shutdown_connections 2024-04-25T10:01:01.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:01.988+0000 7fd7e2f53640 1 -- 172.21.15.139:0/2180655216 wait complete. 2024-04-25T10:01:02.005 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:02.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.097+0000 7f5850407640 1 Processor -- start 2024-04-25T10:01:02.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.097+0000 7f5850407640 1 -- start start 2024-04-25T10:01:02.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.098+0000 7f5850407640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58480cf900 0x7f58480d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.098+0000 7f5850407640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5848099a50 0x7f5848099ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.098+0000 7f5850407640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58480cfe70 con 0x7f584815efb0 2024-04-25T10:01:02.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.098+0000 7f5850407640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58480d0a80 con 0x7f5848099a50 2024-04-25T10:01:02.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.098+0000 7f5850407640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58480d0150 con 0x7f58480cf900 2024-04-25T10:01:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.098+0000 7f584e9a4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5848099a50 0x7f5848099ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.098+0000 7f584e9a4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5848099a50 0x7f5848099ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54544/0 (socket says 172.21.15.139:54544) 2024-04-25T10:01:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.098+0000 7f584e9a4640 1 -- 172.21.15.139:0/2933203324 learned_addr learned my addr 172.21.15.139:0/2933203324 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.098+0000 7f584e9a4640 1 -- 172.21.15.139:0/2933203324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58480cf900 msgr2=0x7f58480d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.098+0000 7f584e9a4640 1 --2- 172.21.15.139:0/2933203324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58480cf900 0x7f58480d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.098+0000 7f584e9a4640 1 -- 172.21.15.139:0/2933203324 >> v1:172.21.15.150:6789/0 conn(0x7f584815efb0 legacy=0x7f584806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.098+0000 7f584e9a4640 1 -- 172.21.15.139:0/2933203324 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480d0430 con 0x7f5848099a50 2024-04-25T10:01:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.099+0000 7f584e9a4640 1 --2- 172.21.15.139:0/2933203324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5848099a50 0x7f5848099ee0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5840009e50 tx=0x7f584002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0d6eb1ef8a794d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.099+0000 7f584d1a1640 1 -- 172.21.15.139:0/2933203324 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840036070 con 0x7f5848099a50 2024-04-25T10:01:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.099+0000 7f584d1a1640 1 -- 172.21.15.139:0/2933203324 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5840031600 con 0x7f5848099a50 2024-04-25T10:01:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.099+0000 7f584d1a1640 1 -- 172.21.15.139:0/2933203324 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58400319d0 con 0x7f5848099a50 2024-04-25T10:01:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.099+0000 7f5850407640 1 -- 172.21.15.139:0/2933203324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5848099a50 msgr2=0x7f5848099ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.099+0000 7f5850407640 1 --2- 172.21.15.139:0/2933203324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5848099a50 0x7f5848099ee0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5840009e50 tx=0x7f584002a630 comp rx=0 tx=0).stop 2024-04-25T10:01:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.099+0000 7f5850407640 1 -- 172.21.15.139:0/2933203324 shutdown_connections 2024-04-25T10:01:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.099+0000 7f5850407640 1 --2- 172.21.15.139:0/2933203324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5848099a50 0x7f5848099ee0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.099+0000 7f5850407640 1 --2- 172.21.15.139:0/2933203324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58480cf900 0x7f58480d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.099+0000 7f5850407640 1 -- 172.21.15.139:0/2933203324 >> 172.21.15.139:0/2933203324 conn(0x7f58480d1790 msgr2=0x7f58480d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.100+0000 7f5850407640 1 -- 172.21.15.139:0/2933203324 shutdown_connections 2024-04-25T10:01:02.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.100+0000 7f5850407640 1 -- 172.21.15.139:0/2933203324 wait complete. 2024-04-25T10:01:02.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.101+0000 7f5850407640 1 Processor -- start 2024-04-25T10:01:02.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.101+0000 7f5850407640 1 -- start start 2024-04-25T10:01:02.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.101+0000 7f5850407640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5848099a50 0x7f58480b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.101+0000 7f5850407640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58480cf900 0x7f58480b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.101+0000 7f5850407640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f584815efb0 0x7f5848158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.101+0000 7f5850407640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480cbde0 con 0x7f5848099a50 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.101+0000 7f5850407640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584806cf50 con 0x7f58480cf900 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.101+0000 7f5850407640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58480cbf60 con 0x7f584815efb0 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.101+0000 7f584e1a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5848099a50 0x7f58480b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.101+0000 7f584e1a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5848099a50 0x7f58480b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54546/0 (socket says 172.21.15.139:54546) 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.101+0000 7f584e1a3640 1 -- 172.21.15.139:0/368070361 learned_addr learned my addr 172.21.15.139:0/368070361 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f584e9a4640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f584815efb0 0x7f5848158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f584d9a2640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58480cf900 0x7f58480b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f584e1a3640 1 -- 172.21.15.139:0/368070361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f584815efb0 msgr2=0x7f5848158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f584e1a3640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f584815efb0 0x7f5848158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f584e1a3640 1 -- 172.21.15.139:0/368070361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58480cf900 msgr2=0x7f58480b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f584e1a3640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58480cf900 0x7f58480b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f584e1a3640 1 -- 172.21.15.139:0/368070361 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480b6820 con 0x7f5848099a50 2024-04-25T10:01:02.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f584e1a3640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5848099a50 0x7f58480b5630 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f584400b420 tx=0x7f584400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f583effd640 1 -- 172.21.15.139:0/368070361 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844011830 con 0x7f5848099a50 2024-04-25T10:01:02.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f583effd640 1 -- 172.21.15.139:0/368070361 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5844016070 con 0x7f5848099a50 2024-04-25T10:01:02.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f5850407640 1 -- 172.21.15.139:0/368070361 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58480b5c10 con 0x7f5848099a50 2024-04-25T10:01:02.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f583effd640 1 -- 172.21.15.139:0/368070361 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844011c10 con 0x7f5848099a50 2024-04-25T10:01:02.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.102+0000 7f5850407640 1 -- 172.21.15.139:0/368070361 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58480b64b0 con 0x7f5848099a50 2024-04-25T10:01:02.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.103+0000 7f5850407640 1 -- 172.21.15.139:0/368070361 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58480b6000 con 0x7f5848099a50 2024-04-25T10:01:02.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.103+0000 7f583effd640 1 -- 172.21.15.139:0/368070361 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f584401b460 con 0x7f5848099a50 2024-04-25T10:01:02.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.104+0000 7f583effd640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582c06b4d0 0x7f582c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.104+0000 7f584d9a2640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582c06b4d0 0x7f582c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.104+0000 7f583effd640 1 -- 172.21.15.139:0/368070361 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f5844089fd0 con 0x7f5848099a50 2024-04-25T10:01:02.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.104+0000 7f584d9a2640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582c06b4d0 0x7f582c06d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f58480c9300 tx=0x7f5838008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.108+0000 7f583effd640 1 -- 172.21.15.139:0/368070361 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5844020020 con 0x7f5848099a50 2024-04-25T10:01:02.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.282+0000 7f5850407640 1 -- 172.21.15.139:0/368070361 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5848074520 con 0x7f5848099a50 2024-04-25T10:01:02.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.284+0000 7f583effd640 1 -- 172.21.15.139:0/368070361 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17389 (secure 0 0 0) 0x7f58480b6820 con 0x7f5848099a50 2024-04-25T10:01:02.292 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:02.292 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":93,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:00:22.118995+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:55.585380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:56.336230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6836/1239243954":"2024-04-26T10:00:22.118970+0000","172.21.15.139:6834/1239243954":"2024-04-26T10:00:22.118970+0000","172.21.15.150:6835/2266713846":"2024-04-26T09:59:10.897062+0000","172.21.15.150:6833/2266713846":"2024-04-26T09:59:10.897062+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:02.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.285+0000 7f5850407640 1 -- 172.21.15.139:0/368070361 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582c06b4d0 msgr2=0x7f582c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.285+0000 7f5850407640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582c06b4d0 0x7f582c06d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f58480c9300 tx=0x7f5838008040 comp rx=0 tx=0).stop 2024-04-25T10:01:02.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.286+0000 7f5850407640 1 -- 172.21.15.139:0/368070361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5848099a50 msgr2=0x7f58480b5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.286+0000 7f5850407640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5848099a50 0x7f58480b5630 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f584400b420 tx=0x7f584400b8e0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.286+0000 7f5850407640 1 -- 172.21.15.139:0/368070361 shutdown_connections 2024-04-25T10:01:02.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.286+0000 7f5850407640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582c06b4d0 0x7f582c06d990 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.286+0000 7f5850407640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f584815efb0 0x7f5848158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.286+0000 7f5850407640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58480cf900 0x7f58480b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.286+0000 7f5850407640 1 --2- 172.21.15.139:0/368070361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5848099a50 0x7f58480b5630 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.286+0000 7f5850407640 1 -- 172.21.15.139:0/368070361 >> 172.21.15.139:0/368070361 conn(0x7f58480d1790 msgr2=0x7f584815a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.286+0000 7f5850407640 1 -- 172.21.15.139:0/368070361 shutdown_connections 2024-04-25T10:01:02.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.286+0000 7f5850407640 1 -- 172.21.15.139:0/368070361 wait complete. 2024-04-25T10:01:02.302 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:58:55.585380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:58:56.336230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:02.302 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:02.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.397+0000 7fdf3d86f640 1 Processor -- start 2024-04-25T10:01:02.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.397+0000 7fdf3d86f640 1 -- start start 2024-04-25T10:01:02.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.398+0000 7fdf3d86f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380d0a80 0x7fdf381618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.398+0000 7fdf3d86f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdf3809ad40 0x7fdf38161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.398+0000 7fdf3d86f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf38099ca0 con 0x7fdf380cd610 2024-04-25T10:01:02.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.398+0000 7fdf3d86f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf38099a50 con 0x7fdf3809ad40 2024-04-25T10:01:02.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.398+0000 7fdf3d86f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38099f80 con 0x7fdf380d0a80 2024-04-25T10:01:02.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.398+0000 7fdf377fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdf3809ad40 0x7fdf38161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.398+0000 7fdf377fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdf3809ad40 0x7fdf38161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54558/0 (socket says 172.21.15.139:54558) 2024-04-25T10:01:02.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.398+0000 7fdf377fe640 1 -- 172.21.15.139:0/3358244906 learned_addr learned my addr 172.21.15.139:0/3358244906 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:02.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.398+0000 7fdf377fe640 1 -- 172.21.15.139:0/3358244906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380d0a80 msgr2=0x7fdf381618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.398+0000 7fdf377fe640 1 --2- 172.21.15.139:0/3358244906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380d0a80 0x7fdf381618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.398+0000 7fdf377fe640 1 -- 172.21.15.139:0/3358244906 >> v1:172.21.15.150:6789/0 conn(0x7fdf380cd610 legacy=0x7fdf380d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.398+0000 7fdf377fe640 1 -- 172.21.15.139:0/3358244906 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3809a260 con 0x7fdf3809ad40 2024-04-25T10:01:02.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.399+0000 7fdf377fe640 1 --2- 172.21.15.139:0/3358244906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf3809ad40 0x7fdf38161e00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009870 tx=0x7fdf3002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38689dddd7077cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.399+0000 7fdf35ffb640 1 -- 172.21.15.139:0/3358244906 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30037070 con 0x7fdf3809ad40 2024-04-25T10:01:02.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.399+0000 7fdf35ffb640 1 -- 172.21.15.139:0/3358244906 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf30032440 con 0x7fdf3809ad40 2024-04-25T10:01:02.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.399+0000 7fdf35ffb640 1 -- 172.21.15.139:0/3358244906 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30032810 con 0x7fdf3809ad40 2024-04-25T10:01:02.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.400+0000 7fdf3d86f640 1 -- 172.21.15.139:0/3358244906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf3809ad40 msgr2=0x7fdf38161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.400+0000 7fdf3d86f640 1 --2- 172.21.15.139:0/3358244906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf3809ad40 0x7fdf38161e00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009870 tx=0x7fdf3002a2a0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.400+0000 7fdf3d86f640 1 -- 172.21.15.139:0/3358244906 shutdown_connections 2024-04-25T10:01:02.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.400+0000 7fdf3d86f640 1 --2- 172.21.15.139:0/3358244906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf3809ad40 0x7fdf38161e00 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.400+0000 7fdf3d86f640 1 --2- 172.21.15.139:0/3358244906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380d0a80 0x7fdf381618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.400+0000 7fdf3d86f640 1 -- 172.21.15.139:0/3358244906 >> 172.21.15.139:0/3358244906 conn(0x7fdf380b31f0 msgr2=0x7fdf380b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.400+0000 7fdf3d86f640 1 -- 172.21.15.139:0/3358244906 shutdown_connections 2024-04-25T10:01:02.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.400+0000 7fdf3d86f640 1 -- 172.21.15.139:0/3358244906 wait complete. 2024-04-25T10:01:02.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.401+0000 7fdf3d86f640 1 Processor -- start 2024-04-25T10:01:02.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.401+0000 7fdf3d86f640 1 -- start start 2024-04-25T10:01:02.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf3d86f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3809ad40 0x7fdf380b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf3d86f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf380cd610 0x7fdf380b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf3d86f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380d0a80 0x7fdf381572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf3d86f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380cd9f0 con 0x7fdf380cd610 2024-04-25T10:01:02.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf3d86f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3809a820 con 0x7fdf3809ad40 2024-04-25T10:01:02.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf3d86f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380cdb70 con 0x7fdf380d0a80 2024-04-25T10:01:02.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf367fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf380cd610 0x7fdf380b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf367fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf380cd610 0x7fdf380b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54560/0 (socket says 172.21.15.139:54560) 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf367fc640 1 -- 172.21.15.139:0/1567454892 learned_addr learned my addr 172.21.15.139:0/1567454892 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf36ffd640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3809ad40 0x7fdf380b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf377fe640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380d0a80 0x7fdf381572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf367fc640 1 -- 172.21.15.139:0/1567454892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380d0a80 msgr2=0x7fdf381572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf367fc640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380d0a80 0x7fdf381572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf367fc640 1 -- 172.21.15.139:0/1567454892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3809ad40 msgr2=0x7fdf380b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf367fc640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3809ad40 0x7fdf380b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf367fc640 1 -- 172.21.15.139:0/1567454892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf380b4a20 con 0x7fdf380cd610 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf36ffd640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3809ad40 0x7fdf380b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf377fe640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380d0a80 0x7fdf381572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.402+0000 7fdf367fc640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf380cd610 0x7fdf380b4320 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00b410 tx=0x7fdf2c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.403+0000 7fdf1ffff640 1 -- 172.21.15.139:0/1567454892 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c015020 con 0x7fdf380cd610 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.403+0000 7fdf1ffff640 1 -- 172.21.15.139:0/1567454892 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c011850 con 0x7fdf380cd610 2024-04-25T10:01:02.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.403+0000 7fdf1ffff640 1 -- 172.21.15.139:0/1567454892 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c011c40 con 0x7fdf380cd610 2024-04-25T10:01:02.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.403+0000 7fdf3d86f640 1 -- 172.21.15.139:0/1567454892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf380b4d00 con 0x7fdf380cd610 2024-04-25T10:01:02.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.403+0000 7fdf3d86f640 1 -- 172.21.15.139:0/1567454892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf380b52c0 con 0x7fdf380cd610 2024-04-25T10:01:02.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.404+0000 7fdf3d86f640 1 -- 172.21.15.139:0/1567454892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf38074520 con 0x7fdf380cd610 2024-04-25T10:01:02.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.404+0000 7fdf1ffff640 1 -- 172.21.15.139:0/1567454892 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf2c018020 con 0x7fdf380cd610 2024-04-25T10:01:02.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.404+0000 7fdf1ffff640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf2406b4d0 0x7fdf2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.405+0000 7fdf1ffff640 1 -- 172.21.15.139:0/1567454892 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fdf2c08a3c0 con 0x7fdf380cd610 2024-04-25T10:01:02.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.405+0000 7fdf36ffd640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf2406b4d0 0x7fdf2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.405+0000 7fdf36ffd640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf2406b4d0 0x7fdf2406d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdf380c73c0 tx=0x7fdf28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.409+0000 7fdf1ffff640 1 -- 172.21.15.139:0/1567454892 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf2c01f020 con 0x7fdf380cd610 2024-04-25T10:01:02.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.578+0000 7fdf3d86f640 1 -- 172.21.15.139:0/1567454892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf381581c0 con 0x7fdf380cd610 2024-04-25T10:01:02.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.579+0000 7fdf1ffff640 1 -- 172.21.15.139:0/1567454892 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf2c0172c0 con 0x7fdf380cd610 2024-04-25T10:01:02.584 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:01:02.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.581+0000 7fdf3d86f640 1 -- 172.21.15.139:0/1567454892 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf2406b4d0 msgr2=0x7fdf2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.581+0000 7fdf3d86f640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf2406b4d0 0x7fdf2406d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdf380c73c0 tx=0x7fdf28009040 comp rx=0 tx=0).stop 2024-04-25T10:01:02.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.581+0000 7fdf3d86f640 1 -- 172.21.15.139:0/1567454892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf380cd610 msgr2=0x7fdf380b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.581+0000 7fdf3d86f640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf380cd610 0x7fdf380b4320 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00b410 tx=0x7fdf2c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.581+0000 7fdf3d86f640 1 -- 172.21.15.139:0/1567454892 shutdown_connections 2024-04-25T10:01:02.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.581+0000 7fdf3d86f640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf380cd610 0x7fdf380b4320 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.581+0000 7fdf3d86f640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdf2406b4d0 0x7fdf2406d990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.581+0000 7fdf3d86f640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf380d0a80 0x7fdf381572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.581+0000 7fdf3d86f640 1 --2- 172.21.15.139:0/1567454892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3809ad40 0x7fdf380b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.581+0000 7fdf3d86f640 1 -- 172.21.15.139:0/1567454892 >> 172.21.15.139:0/1567454892 conn(0x7fdf380b31f0 msgr2=0x7fdf380b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.581+0000 7fdf3d86f640 1 -- 172.21.15.139:0/1567454892 shutdown_connections 2024-04-25T10:01:02.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.581+0000 7fdf3d86f640 1 -- 172.21.15.139:0/1567454892 wait complete. 2024-04-25T10:01:02.597 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:01:02.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.691+0000 7faf4fc6b640 1 Processor -- start 2024-04-25T10:01:02.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.691+0000 7faf4fc6b640 1 -- start start 2024-04-25T10:01:02.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.692+0000 7faf4fc6b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faf4815dbf0 0x7faf481617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.692+0000 7faf4fc6b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faf480cee50 0x7faf480d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.692+0000 7faf4fc6b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf48161dc0 con 0x7faf480b9370 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.692+0000 7faf4fc6b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf480d1ee0 con 0x7faf4815dbf0 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.692+0000 7faf4fc6b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf481620a0 con 0x7faf480cee50 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.692+0000 7faf4da07640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faf4815dbf0 0x7faf481617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.692+0000 7faf4da07640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faf4815dbf0 0x7faf481617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54562/0 (socket says 172.21.15.139:54562) 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.692+0000 7faf4da07640 1 -- 172.21.15.139:0/2916510570 learned_addr learned my addr 172.21.15.139:0/2916510570 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.692+0000 7faf4da07640 1 -- 172.21.15.139:0/2916510570 >> v2:172.21.15.150:3300/0 conn(0x7faf480cee50 msgr2=0x7faf480d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.692+0000 7faf4da07640 1 --2- 172.21.15.139:0/2916510570 >> v2:172.21.15.150:3300/0 conn(0x7faf480cee50 0x7faf480d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.693+0000 7faf4da07640 1 -- 172.21.15.139:0/2916510570 >> v1:172.21.15.150:6789/0 conn(0x7faf480b9370 legacy=0x7faf480ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.693+0000 7faf4da07640 1 -- 172.21.15.139:0/2916510570 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf48162380 con 0x7faf4815dbf0 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.693+0000 7faf4da07640 1 --2- 172.21.15.139:0/2916510570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf4815dbf0 0x7faf481617e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7faf4808ce00 tx=0x7faf4402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a117621187f3fffd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.693+0000 7faf4ca05640 1 -- 172.21.15.139:0/2916510570 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44037070 con 0x7faf4815dbf0 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.693+0000 7faf4ca05640 1 -- 172.21.15.139:0/2916510570 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf44032440 con 0x7faf4815dbf0 2024-04-25T10:01:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.693+0000 7faf4ca05640 1 -- 172.21.15.139:0/2916510570 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44032810 con 0x7faf4815dbf0 2024-04-25T10:01:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.694+0000 7faf4fc6b640 1 -- 172.21.15.139:0/2916510570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf4815dbf0 msgr2=0x7faf481617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.694+0000 7faf4fc6b640 1 --2- 172.21.15.139:0/2916510570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf4815dbf0 0x7faf481617e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7faf4808ce00 tx=0x7faf4402a230 comp rx=0 tx=0).stop 2024-04-25T10:01:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.694+0000 7faf4fc6b640 1 -- 172.21.15.139:0/2916510570 shutdown_connections 2024-04-25T10:01:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.694+0000 7faf4fc6b640 1 --2- 172.21.15.139:0/2916510570 >> v2:172.21.15.150:3300/0 conn(0x7faf480cee50 0x7faf480d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.694+0000 7faf4fc6b640 1 --2- 172.21.15.139:0/2916510570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf4815dbf0 0x7faf481617e0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.694+0000 7faf4fc6b640 1 -- 172.21.15.139:0/2916510570 >> 172.21.15.139:0/2916510570 conn(0x7faf480b5480 msgr2=0x7faf480b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.694+0000 7faf4fc6b640 1 -- 172.21.15.139:0/2916510570 shutdown_connections 2024-04-25T10:01:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.694+0000 7faf4fc6b640 1 -- 172.21.15.139:0/2916510570 wait complete. 2024-04-25T10:01:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.695+0000 7faf4fc6b640 1 Processor -- start 2024-04-25T10:01:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.695+0000 7faf4fc6b640 1 -- start start 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4fc6b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf480b9370 0x7faf4814ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4fc6b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf480cee50 0x7faf480d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4fc6b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf4815dbf0 0x7faf480d2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4fc6b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4815dfd0 con 0x7faf480cee50 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4fc6b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf480b97d0 con 0x7faf4815dbf0 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4fc6b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf481f46f0 con 0x7faf480b9370 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4d206640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf480cee50 0x7faf480d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4d206640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf480cee50 0x7faf480d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54572/0 (socket says 172.21.15.139:54572) 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4d206640 1 -- 172.21.15.139:0/1804712733 learned_addr learned my addr 172.21.15.139:0/1804712733 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4da07640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf480b9370 0x7faf4814ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4e208640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf4815dbf0 0x7faf480d2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4d206640 1 -- 172.21.15.139:0/1804712733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf480b9370 msgr2=0x7faf4814ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4d206640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf480b9370 0x7faf4814ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4d206640 1 -- 172.21.15.139:0/1804712733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf4815dbf0 msgr2=0x7faf480d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4d206640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf4815dbf0 0x7faf480d2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4d206640 1 -- 172.21.15.139:0/1804712733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf48150970 con 0x7faf480cee50 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4e208640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf4815dbf0 0x7faf480d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf4d206640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf480cee50 0x7faf480d27c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7faf3800b3e0 tx=0x7faf3800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.696+0000 7faf3e7fc640 1 -- 172.21.15.139:0/1804712733 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38015020 con 0x7faf480cee50 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.697+0000 7faf3e7fc640 1 -- 172.21.15.139:0/1804712733 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf38011850 con 0x7faf480cee50 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.697+0000 7faf4fc6b640 1 -- 172.21.15.139:0/1804712733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf48150c50 con 0x7faf480cee50 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.697+0000 7faf3e7fc640 1 -- 172.21.15.139:0/1804712733 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38011c40 con 0x7faf480cee50 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.697+0000 7faf4fc6b640 1 -- 172.21.15.139:0/1804712733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf481f4af0 con 0x7faf480cee50 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.698+0000 7faf4fc6b640 1 -- 172.21.15.139:0/1804712733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf48150790 con 0x7faf480cee50 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.698+0000 7faf3e7fc640 1 -- 172.21.15.139:0/1804712733 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf38018020 con 0x7faf480cee50 2024-04-25T10:01:02.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.698+0000 7faf3e7fc640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faf2c06b4d0 0x7faf2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.698+0000 7faf3e7fc640 1 -- 172.21.15.139:0/1804712733 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7faf3808a380 con 0x7faf480cee50 2024-04-25T10:01:02.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.698+0000 7faf4da07640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faf2c06b4d0 0x7faf2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.699+0000 7faf4da07640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faf2c06b4d0 0x7faf2c06d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faf44034000 tx=0x7faf4402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.703+0000 7faf3e7fc640 1 -- 172.21.15.139:0/1804712733 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf48150790 con 0x7faf480cee50 2024-04-25T10:01:02.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.872+0000 7faf4fc6b640 1 -- 172.21.15.139:0/1804712733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faf48074520 con 0x7faf480cee50 2024-04-25T10:01:02.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.872+0000 7faf3e7fc640 1 -- 172.21.15.139:0/1804712733 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7faf3801f020 con 0x7faf480cee50 2024-04-25T10:01:02.877 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T10:01:02.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.874+0000 7faf4fc6b640 1 -- 172.21.15.139:0/1804712733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faf2c06b4d0 msgr2=0x7faf2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.874+0000 7faf4fc6b640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faf2c06b4d0 0x7faf2c06d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faf44034000 tx=0x7faf4402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:02.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.874+0000 7faf4fc6b640 1 -- 172.21.15.139:0/1804712733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf480cee50 msgr2=0x7faf480d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.874+0000 7faf4fc6b640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf480cee50 0x7faf480d27c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7faf3800b3e0 tx=0x7faf3800b8a0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.874+0000 7faf4fc6b640 1 -- 172.21.15.139:0/1804712733 shutdown_connections 2024-04-25T10:01:02.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.874+0000 7faf4fc6b640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf480cee50 0x7faf480d27c0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.874+0000 7faf4fc6b640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faf2c06b4d0 0x7faf2c06d990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.874+0000 7faf4fc6b640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf4815dbf0 0x7faf480d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.874+0000 7faf4fc6b640 1 --2- 172.21.15.139:0/1804712733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf480b9370 0x7faf4814ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.874+0000 7faf4fc6b640 1 -- 172.21.15.139:0/1804712733 >> 172.21.15.139:0/1804712733 conn(0x7faf480b5480 msgr2=0x7faf480b7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.874+0000 7faf4fc6b640 1 -- 172.21.15.139:0/1804712733 shutdown_connections 2024-04-25T10:01:02.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.874+0000 7faf4fc6b640 1 -- 172.21.15.139:0/1804712733 wait complete. 2024-04-25T10:01:02.890 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:01:02.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.985+0000 7f3b7d81f640 1 Processor -- start 2024-04-25T10:01:02.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.985+0000 7f3b7d81f640 1 -- start start 2024-04-25T10:01:02.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.985+0000 7f3b7d81f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3b7815dbf0 0x7f3b781617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.985+0000 7f3b7d81f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b780b9370 0x7f3b780ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.985+0000 7f3b7d81f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b78161dc0 con 0x7f3b780cec80 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.985+0000 7f3b7d81f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b780d1ee0 con 0x7f3b7815dbf0 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.985+0000 7f3b7d81f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b781620a0 con 0x7f3b780b9370 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.986+0000 7f3b767fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b780b9370 0x7f3b780ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.986+0000 7f3b76ffd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3b7815dbf0 0x7f3b781617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.986+0000 7f3b777fe640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f3b780cec80 0x7f3b7806c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:40010/0 (socket says 172.21.15.139:40010) 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.986+0000 7f3b767fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b780b9370 0x7f3b780ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:39426/0 (socket says 172.21.15.139:39426) 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.986+0000 7f3b76ffd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3b7815dbf0 0x7f3b781617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:51114/0 (socket says 172.21.15.139:51114) 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.986+0000 7f3b777fe640 1 -- 172.21.15.139:0/2762964716 learned_addr learned my addr 172.21.15.139:0/2762964716 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.986+0000 7f3b767fc640 1 -- 172.21.15.139:0/2762964716 >> v2:172.21.15.150:3300/0 conn(0x7f3b7815dbf0 msgr2=0x7f3b781617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.986+0000 7f3b767fc640 1 --2- 172.21.15.139:0/2762964716 >> v2:172.21.15.150:3300/0 conn(0x7f3b7815dbf0 0x7f3b781617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.986+0000 7f3b767fc640 1 -- 172.21.15.139:0/2762964716 >> v1:172.21.15.150:6789/0 conn(0x7f3b780cec80 legacy=0x7f3b7806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.986+0000 7f3b767fc640 1 -- 172.21.15.139:0/2762964716 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b78162380 con 0x7f3b780b9370 2024-04-25T10:01:02.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.986+0000 7f3b76ffd640 1 --2- 172.21.15.139:0/2762964716 >> v2:172.21.15.150:3300/0 conn(0x7f3b7815dbf0 0x7f3b781617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.987+0000 7f3b767fc640 1 --2- 172.21.15.139:0/2762964716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b780b9370 0x7f3b780ce740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0097a0 tx=0x7f3b6c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d579eb8118d0d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.987+0000 7f3b75ffb640 1 -- 172.21.15.139:0/2762964716 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c037070 con 0x7f3b780b9370 2024-04-25T10:01:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.987+0000 7f3b75ffb640 1 -- 172.21.15.139:0/2762964716 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c032440 con 0x7f3b780b9370 2024-04-25T10:01:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.987+0000 7f3b75ffb640 1 -- 172.21.15.139:0/2762964716 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c032810 con 0x7f3b780b9370 2024-04-25T10:01:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.988+0000 7f3b7d81f640 1 -- 172.21.15.139:0/2762964716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b780b9370 msgr2=0x7f3b780ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.988+0000 7f3b7d81f640 1 --2- 172.21.15.139:0/2762964716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b780b9370 0x7f3b780ce740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0097a0 tx=0x7f3b6c02a180 comp rx=0 tx=0).stop 2024-04-25T10:01:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.988+0000 7f3b7d81f640 1 -- 172.21.15.139:0/2762964716 shutdown_connections 2024-04-25T10:01:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.988+0000 7f3b7d81f640 1 --2- 172.21.15.139:0/2762964716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b780b9370 0x7f3b780ce740 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.988+0000 7f3b7d81f640 1 --2- 172.21.15.139:0/2762964716 >> v2:172.21.15.150:3300/0 conn(0x7f3b7815dbf0 0x7f3b781617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.988+0000 7f3b7d81f640 1 -- 172.21.15.139:0/2762964716 >> 172.21.15.139:0/2762964716 conn(0x7f3b780b5480 msgr2=0x7f3b780b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.988+0000 7f3b7d81f640 1 -- 172.21.15.139:0/2762964716 shutdown_connections 2024-04-25T10:01:02.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.988+0000 7f3b7d81f640 1 -- 172.21.15.139:0/2762964716 wait complete. 2024-04-25T10:01:02.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.989+0000 7f3b7d81f640 1 Processor -- start 2024-04-25T10:01:02.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.989+0000 7f3b7d81f640 1 -- start start 2024-04-25T10:01:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b7d81f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b780b9370 0x7f3b7814ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b7d81f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b780cec80 0x7f3b780ab960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b7d81f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b7815dbf0 0x7f3b7815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b7d81f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7815dfd0 con 0x7f3b780cec80 2024-04-25T10:01:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b7d81f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b780b97d0 con 0x7f3b780b9370 2024-04-25T10:01:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b7d81f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b780aa8c0 con 0x7f3b7815dbf0 2024-04-25T10:01:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b767fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b780cec80 0x7f3b780ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b76ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b780b9370 0x7f3b7814ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b767fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b780cec80 0x7f3b780ab960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54582/0 (socket says 172.21.15.139:54582) 2024-04-25T10:01:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b767fc640 1 -- 172.21.15.139:0/814808153 learned_addr learned my addr 172.21.15.139:0/814808153 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b777fe640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b7815dbf0 0x7f3b7815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b767fc640 1 -- 172.21.15.139:0/814808153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b7815dbf0 msgr2=0x7f3b7815bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b767fc640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b7815dbf0 0x7f3b7815bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b767fc640 1 -- 172.21.15.139:0/814808153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b780b9370 msgr2=0x7f3b7814ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b767fc640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b780b9370 0x7f3b7814ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b767fc640 1 -- 172.21.15.139:0/814808153 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b780aaba0 con 0x7f3b780cec80 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b767fc640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b780cec80 0x7f3b780ab960 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0097a0 tx=0x7f3b6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b5ffff640 1 -- 172.21.15.139:0/814808153 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c041070 con 0x7f3b780cec80 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b5ffff640 1 -- 172.21.15.139:0/814808153 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c03c870 con 0x7f3b780cec80 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b7d81f640 1 -- 172.21.15.139:0/814808153 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b780aae80 con 0x7f3b780cec80 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b5ffff640 1 -- 172.21.15.139:0/814808153 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c037040 con 0x7f3b780cec80 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.990+0000 7f3b7d81f640 1 -- 172.21.15.139:0/814808153 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b780ab440 con 0x7f3b780cec80 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.991+0000 7f3b7d81f640 1 -- 172.21.15.139:0/814808153 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b78073dd0 con 0x7f3b780cec80 2024-04-25T10:01:02.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.992+0000 7f3b5ffff640 1 -- 172.21.15.139:0/814808153 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b6c044070 con 0x7f3b780cec80 2024-04-25T10:01:02.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.992+0000 7f3b5ffff640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b6406b4d0 0x7f3b6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.992+0000 7f3b5ffff640 1 -- 172.21.15.139:0/814808153 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f3b6c0ab0d0 con 0x7f3b780cec80 2024-04-25T10:01:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.992+0000 7f3b76ffd640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b6406b4d0 0x7f3b6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.993+0000 7f3b76ffd640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b6406b4d0 0x7f3b6406d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009560 tx=0x7f3b68009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:02.997+0000 7f3b5ffff640 1 -- 172.21.15.139:0/814808153 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b78073dd0 con 0x7f3b780cec80 2024-04-25T10:01:03.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.171+0000 7f3b7d81f640 1 -- 172.21.15.139:0/814808153 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b78067420 con 0x7f3b780cec80 2024-04-25T10:01:03.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.171+0000 7f3b5ffff640 1 -- 172.21.15.139:0/814808153 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b78067420 con 0x7f3b780cec80 2024-04-25T10:01:03.176 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T10:01:03.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.173+0000 7f3b7d81f640 1 -- 172.21.15.139:0/814808153 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b6406b4d0 msgr2=0x7f3b6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.173+0000 7f3b7d81f640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b6406b4d0 0x7f3b6406d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009560 tx=0x7f3b68009190 comp rx=0 tx=0).stop 2024-04-25T10:01:03.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.173+0000 7f3b7d81f640 1 -- 172.21.15.139:0/814808153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b780cec80 msgr2=0x7f3b780ab960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.173+0000 7f3b7d81f640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b780cec80 0x7f3b780ab960 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0097a0 tx=0x7f3b6c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.173+0000 7f3b7d81f640 1 -- 172.21.15.139:0/814808153 shutdown_connections 2024-04-25T10:01:03.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.173+0000 7f3b7d81f640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b780cec80 0x7f3b780ab960 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.173+0000 7f3b7d81f640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3b6406b4d0 0x7f3b6406d990 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.173+0000 7f3b7d81f640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3b7815dbf0 0x7f3b7815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.173+0000 7f3b7d81f640 1 --2- 172.21.15.139:0/814808153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b780b9370 0x7f3b7814ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.173+0000 7f3b7d81f640 1 -- 172.21.15.139:0/814808153 >> 172.21.15.139:0/814808153 conn(0x7f3b780b5480 msgr2=0x7f3b7806c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:03.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.173+0000 7f3b7d81f640 1 -- 172.21.15.139:0/814808153 shutdown_connections 2024-04-25T10:01:03.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.173+0000 7f3b7d81f640 1 -- 172.21.15.139:0/814808153 wait complete. 2024-04-25T10:01:03.189 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.280+0000 7efe7b85d640 1 Processor -- start 2024-04-25T10:01:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.280+0000 7efe7b85d640 1 -- start start 2024-04-25T10:01:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe7b85d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efe74155e30 0x7efe7415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe7b85d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efe7415bdc0 0x7efe7415c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe7b85d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7415c880 con 0x7efe740b1740 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe7b85d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7406d6f0 con 0x7efe74155e30 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe7b85d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7415cb60 con 0x7efe7415bdc0 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe795f9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efe74155e30 0x7efe7415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe795f9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efe74155e30 0x7efe7415cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54598/0 (socket says 172.21.15.139:54598) 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe795f9640 1 -- 172.21.15.139:0/3001567158 learned_addr learned my addr 172.21.15.139:0/3001567158 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe79dfa640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7efe740b1740 0x7efe7406db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37196/0 (socket says 172.21.15.139:37196) 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe78df8640 1 --2- 172.21.15.139:0/3001567158 >> v2:172.21.15.150:3300/0 conn(0x7efe7415bdc0 0x7efe7415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe795f9640 1 -- 172.21.15.139:0/3001567158 >> v2:172.21.15.150:3300/0 conn(0x7efe7415bdc0 msgr2=0x7efe7415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe795f9640 1 --2- 172.21.15.139:0/3001567158 >> v2:172.21.15.150:3300/0 conn(0x7efe7415bdc0 0x7efe7415c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe795f9640 1 -- 172.21.15.139:0/3001567158 >> v1:172.21.15.139:6789/0 conn(0x7efe740b1740 legacy=0x7efe7406db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe6bfff640 1 -- 172.21.15.139:0/3001567158 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535750366 0 0) 0x7efe7415c880 con 0x7efe740b1740 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.281+0000 7efe795f9640 1 -- 172.21.15.139:0/3001567158 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7406d380 con 0x7efe74155e30 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.282+0000 7efe795f9640 1 --2- 172.21.15.139:0/3001567158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe74155e30 0x7efe7415cf80 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7efe740b4e10 tx=0x7efe7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0de555ae5f60683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.282+0000 7efe6bfff640 1 -- 172.21.15.139:0/3001567158 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70037070 con 0x7efe74155e30 2024-04-25T10:01:03.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.282+0000 7efe6bfff640 1 -- 172.21.15.139:0/3001567158 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe70032440 con 0x7efe74155e30 2024-04-25T10:01:03.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.282+0000 7efe6bfff640 1 -- 172.21.15.139:0/3001567158 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70032810 con 0x7efe74155e30 2024-04-25T10:01:03.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.283+0000 7efe7b85d640 1 -- 172.21.15.139:0/3001567158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe74155e30 msgr2=0x7efe7415cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.283+0000 7efe7b85d640 1 --2- 172.21.15.139:0/3001567158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe74155e30 0x7efe7415cf80 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7efe740b4e10 tx=0x7efe7002a270 comp rx=0 tx=0).stop 2024-04-25T10:01:03.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.283+0000 7efe7b85d640 1 -- 172.21.15.139:0/3001567158 shutdown_connections 2024-04-25T10:01:03.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.283+0000 7efe7b85d640 1 --2- 172.21.15.139:0/3001567158 >> v2:172.21.15.150:3300/0 conn(0x7efe7415bdc0 0x7efe7415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.283+0000 7efe7b85d640 1 --2- 172.21.15.139:0/3001567158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe74155e30 0x7efe7415cf80 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.283+0000 7efe7b85d640 1 -- 172.21.15.139:0/3001567158 >> 172.21.15.139:0/3001567158 conn(0x7efe74066f20 msgr2=0x7efe7406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:03.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.283+0000 7efe7b85d640 1 -- 172.21.15.139:0/3001567158 shutdown_connections 2024-04-25T10:01:03.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.283+0000 7efe7b85d640 1 -- 172.21.15.139:0/3001567158 wait complete. 2024-04-25T10:01:03.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.284+0000 7efe7b85d640 1 Processor -- start 2024-04-25T10:01:03.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.284+0000 7efe7b85d640 1 -- start start 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe7b85d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe740b1740 0x7efe7414f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe7b85d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe74155e30 0x7efe7413f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe7b85d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe7415bdc0 0x7efe740b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe7b85d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe740af260 con 0x7efe74155e30 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe7b85d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe740af0e0 con 0x7efe7415bdc0 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe7b85d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7415d4c0 con 0x7efe740b1740 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe78df8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe74155e30 0x7efe7413f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe78df8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe74155e30 0x7efe7413f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54600/0 (socket says 172.21.15.139:54600) 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe78df8640 1 -- 172.21.15.139:0/427266916 learned_addr learned my addr 172.21.15.139:0/427266916 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe79dfa640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe7415bdc0 0x7efe740b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe795f9640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe740b1740 0x7efe7414f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe78df8640 1 -- 172.21.15.139:0/427266916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe740b1740 msgr2=0x7efe7414f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe78df8640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe740b1740 0x7efe7414f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe78df8640 1 -- 172.21.15.139:0/427266916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe7415bdc0 msgr2=0x7efe740b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe78df8640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe7415bdc0 0x7efe740b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe78df8640 1 -- 172.21.15.139:0/427266916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7414e7c0 con 0x7efe74155e30 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe79dfa640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe7415bdc0 0x7efe740b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe795f9640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe740b1740 0x7efe7414f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe78df8640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe74155e30 0x7efe7413f110 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7efe6400d5f0 tx=0x7efe6400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe69ffb640 1 -- 172.21.15.139:0/427266916 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe640125f0 con 0x7efe74155e30 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe69ffb640 1 -- 172.21.15.139:0/427266916 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe64019070 con 0x7efe74155e30 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe7b85d640 1 -- 172.21.15.139:0/427266916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7414eaa0 con 0x7efe74155e30 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.285+0000 7efe69ffb640 1 -- 172.21.15.139:0/427266916 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe640129d0 con 0x7efe74155e30 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.286+0000 7efe7b85d640 1 -- 172.21.15.139:0/427266916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7414f060 con 0x7efe74155e30 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.287+0000 7efe7b85d640 1 -- 172.21.15.139:0/427266916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe74074520 con 0x7efe74155e30 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.287+0000 7efe69ffb640 1 -- 172.21.15.139:0/427266916 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe64012b70 con 0x7efe74155e30 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.287+0000 7efe69ffb640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe5806b4d0 0x7efe5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.287+0000 7efe69ffb640 1 -- 172.21.15.139:0/427266916 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7efe6408a870 con 0x7efe74155e30 2024-04-25T10:01:03.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.287+0000 7efe795f9640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe5806b4d0 0x7efe5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.288+0000 7efe795f9640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe5806b4d0 0x7efe5806d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efe70034000 tx=0x7efe7002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.291+0000 7efe69ffb640 1 -- 172.21.15.139:0/427266916 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe64059440 con 0x7efe74155e30 2024-04-25T10:01:03.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.499+0000 7efe7b85d640 1 -- 172.21.15.139:0/427266916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe7406b830 con 0x7efe74155e30 2024-04-25T10:01:03.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.500+0000 7efe69ffb640 1 -- 172.21.15.139:0/427266916 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4953 (secure 0 0 0) 0x7efe64058d40 con 0x7efe74155e30 2024-04-25T10:01:03.505 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:03.506 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T10:00:25.131250+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6836/1457343244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1457343244},{"type":"v1","addr":"172.21.15.139:6836","nonce":1457343244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:01:03.506 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 63 2024-04-25T10:01:03.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.502+0000 7efe7b85d640 1 -- 172.21.15.139:0/427266916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe5806b4d0 msgr2=0x7efe5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.502+0000 7efe7b85d640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe5806b4d0 0x7efe5806d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efe70034000 tx=0x7efe7002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.502+0000 7efe7b85d640 1 -- 172.21.15.139:0/427266916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe74155e30 msgr2=0x7efe7413f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.502+0000 7efe7b85d640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe74155e30 0x7efe7413f110 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7efe6400d5f0 tx=0x7efe6400dab0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.502+0000 7efe7b85d640 1 -- 172.21.15.139:0/427266916 shutdown_connections 2024-04-25T10:01:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.502+0000 7efe7b85d640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe74155e30 0x7efe7413f110 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.502+0000 7efe7b85d640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe5806b4d0 0x7efe5806d990 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.502+0000 7efe7b85d640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe7415bdc0 0x7efe740b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.502+0000 7efe7b85d640 1 --2- 172.21.15.139:0/427266916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe740b1740 0x7efe7414f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.502+0000 7efe7b85d640 1 -- 172.21.15.139:0/427266916 >> 172.21.15.139:0/427266916 conn(0x7efe74066f20 msgr2=0x7efe7415ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.502+0000 7efe7b85d640 1 -- 172.21.15.139:0/427266916 shutdown_connections 2024-04-25T10:01:03.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.502+0000 7efe7b85d640 1 -- 172.21.15.139:0/427266916 wait complete. 2024-04-25T10:01:03.518 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:03.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.610+0000 7f3755cbc640 1 Processor -- start 2024-04-25T10:01:03.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.610+0000 7f3755cbc640 1 -- start start 2024-04-25T10:01:03.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.610+0000 7f3755cbc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f375015dc60 0x7f37500b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.610+0000 7f3755cbc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3750161850 0x7f37500ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.610+0000 7f3755cbc640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37500cf250 con 0x7f37500d0970 2024-04-25T10:01:03.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.610+0000 7f3755cbc640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37500ce780 con 0x7f375015dc60 2024-04-25T10:01:03.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.610+0000 7f3755cbc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37500d3730 con 0x7f3750161850 2024-04-25T10:01:03.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.611+0000 7f374f7fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f375015dc60 0x7f37500b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.611+0000 7f374effd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3750161850 0x7f37500ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.611+0000 7f374ffff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f37500d0970 0x7f37500d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:40024/0 (socket says 172.21.15.139:40024) 2024-04-25T10:01:03.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.611+0000 7f374ffff640 1 -- 172.21.15.139:0/456265818 learned_addr learned my addr 172.21.15.139:0/456265818 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:03.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.611+0000 7f374e7fc640 1 -- 172.21.15.139:0/456265818 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848120062 0 0) 0x7f3748001490 con 0x7f37500d0970 2024-04-25T10:01:03.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.611+0000 7f374e7fc640 1 -- 172.21.15.139:0/456265818 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3738003520 con 0x7f37500d0970 2024-04-25T10:01:03.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.611+0000 7f374f7fe640 1 -- 172.21.15.139:0/456265818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3750161850 msgr2=0x7f37500ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:03.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.611+0000 7f374f7fe640 1 --2- 172.21.15.139:0/456265818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3750161850 0x7f37500ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.611+0000 7f374f7fe640 1 -- 172.21.15.139:0/456265818 >> v1:172.21.15.150:6789/0 conn(0x7f37500d0970 legacy=0x7f37500d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.611+0000 7f374f7fe640 1 -- 172.21.15.139:0/456265818 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37500ce9b0 con 0x7f375015dc60 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.611+0000 7f374effd640 1 --2- 172.21.15.139:0/456265818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3750161850 0x7f37500ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.612+0000 7f374f7fe640 1 --2- 172.21.15.139:0/456265818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f375015dc60 0x7f37500b6c10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f37400097d0 tx=0x7f374002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e79c0125c7b2450c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.612+0000 7f374e7fc640 1 -- 172.21.15.139:0/456265818 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740037070 con 0x7f375015dc60 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.612+0000 7f374e7fc640 1 -- 172.21.15.139:0/456265818 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3740032440 con 0x7f375015dc60 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.612+0000 7f374e7fc640 1 -- 172.21.15.139:0/456265818 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37400327f0 con 0x7f375015dc60 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.613+0000 7f3755cbc640 1 -- 172.21.15.139:0/456265818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f375015dc60 msgr2=0x7f37500b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.613+0000 7f3755cbc640 1 --2- 172.21.15.139:0/456265818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f375015dc60 0x7f37500b6c10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f37400097d0 tx=0x7f374002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.613+0000 7f3755cbc640 1 -- 172.21.15.139:0/456265818 shutdown_connections 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.613+0000 7f3755cbc640 1 --2- 172.21.15.139:0/456265818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3750161850 0x7f37500ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.613+0000 7f3755cbc640 1 --2- 172.21.15.139:0/456265818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f375015dc60 0x7f37500b6c10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.613+0000 7f3755cbc640 1 -- 172.21.15.139:0/456265818 >> 172.21.15.139:0/456265818 conn(0x7f37500b5910 msgr2=0x7f37500b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.613+0000 7f3755cbc640 1 -- 172.21.15.139:0/456265818 shutdown_connections 2024-04-25T10:01:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.613+0000 7f3755cbc640 1 -- 172.21.15.139:0/456265818 wait complete. 2024-04-25T10:01:03.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.614+0000 7f3755cbc640 1 Processor -- start 2024-04-25T10:01:03.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.614+0000 7f3755cbc640 1 -- start start 2024-04-25T10:01:03.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f3755cbc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37500d0970 0x7f3750152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f3755cbc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f375015dc60 0x7f37500b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f3755cbc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3750161850 0x7f37500b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f3755cbc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37500d1fc0 con 0x7f375015dc60 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f3755cbc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37500cef70 con 0x7f3750161850 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f3755cbc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375015e040 con 0x7f37500d0970 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f374effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f375015dc60 0x7f37500b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f374effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f375015dc60 0x7f37500b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54608/0 (socket says 172.21.15.139:54608) 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f374effd640 1 -- 172.21.15.139:0/1895881154 learned_addr learned my addr 172.21.15.139:0/1895881154 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f374f7fe640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37500d0970 0x7f3750152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f374ffff640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3750161850 0x7f37500b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f374effd640 1 -- 172.21.15.139:0/1895881154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37500d0970 msgr2=0x7f3750152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f374effd640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37500d0970 0x7f3750152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f374effd640 1 -- 172.21.15.139:0/1895881154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3750161850 msgr2=0x7f37500b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f374effd640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3750161850 0x7f37500b7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.615+0000 7f374effd640 1 -- 172.21.15.139:0/1895881154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375015b890 con 0x7f375015dc60 2024-04-25T10:01:03.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.616+0000 7f374effd640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f375015dc60 0x7f37500b6c10 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f374400e960 tx=0x7f374400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.616+0000 7f3737fff640 1 -- 172.21.15.139:0/1895881154 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744017070 con 0x7f375015dc60 2024-04-25T10:01:03.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.616+0000 7f3755cbc640 1 -- 172.21.15.139:0/1895881154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375015bb70 con 0x7f375015dc60 2024-04-25T10:01:03.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.616+0000 7f3737fff640 1 -- 172.21.15.139:0/1895881154 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374400ca00 con 0x7f375015dc60 2024-04-25T10:01:03.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.616+0000 7f3737fff640 1 -- 172.21.15.139:0/1895881154 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400cdd0 con 0x7f375015dc60 2024-04-25T10:01:03.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.616+0000 7f3755cbc640 1 -- 172.21.15.139:0/1895881154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375015c130 con 0x7f375015dc60 2024-04-25T10:01:03.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.617+0000 7f3755cbc640 1 -- 172.21.15.139:0/1895881154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3750066db0 con 0x7f375015dc60 2024-04-25T10:01:03.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.617+0000 7f3737fff640 1 -- 172.21.15.139:0/1895881154 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f374401a070 con 0x7f375015dc60 2024-04-25T10:01:03.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.617+0000 7f3737fff640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f373c06b500 0x7f373c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.618+0000 7f3737fff640 1 -- 172.21.15.139:0/1895881154 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f374408a6b0 con 0x7f375015dc60 2024-04-25T10:01:03.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.618+0000 7f374f7fe640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f373c06b500 0x7f373c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.618+0000 7f374f7fe640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f373c06b500 0x7f373c06d9c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3740034000 tx=0x7f374002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.622+0000 7f3737fff640 1 -- 172.21.15.139:0/1895881154 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3750066db0 con 0x7f375015dc60 2024-04-25T10:01:03.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.791+0000 7f3755cbc640 1 -- 172.21.15.139:0/1895881154 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3750074520 con 0x7f375015dc60 2024-04-25T10:01:03.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.793+0000 7f3737fff640 1 -- 172.21.15.139:0/1895881154 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17389 (secure 0 0 0) 0x7f3744058bb0 con 0x7f375015dc60 2024-04-25T10:01:03.798 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:03.799 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":93,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:00:22.118995+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:55.585380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:56.336230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6836/1239243954":"2024-04-26T10:00:22.118970+0000","172.21.15.139:6834/1239243954":"2024-04-26T10:00:22.118970+0000","172.21.15.150:6835/2266713846":"2024-04-26T09:59:10.897062+0000","172.21.15.150:6833/2266713846":"2024-04-26T09:59:10.897062+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:03.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.795+0000 7f3755cbc640 1 -- 172.21.15.139:0/1895881154 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f373c06b500 msgr2=0x7f373c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.795+0000 7f3755cbc640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f373c06b500 0x7f373c06d9c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3740034000 tx=0x7f374002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:03.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.795+0000 7f3755cbc640 1 -- 172.21.15.139:0/1895881154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f375015dc60 msgr2=0x7f37500b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.795+0000 7f3755cbc640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f375015dc60 0x7f37500b6c10 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f374400e960 tx=0x7f374400ee20 comp rx=0 tx=0).stop 2024-04-25T10:01:03.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.795+0000 7f3755cbc640 1 -- 172.21.15.139:0/1895881154 shutdown_connections 2024-04-25T10:01:03.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.795+0000 7f3755cbc640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f375015dc60 0x7f37500b6c10 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.795+0000 7f3755cbc640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f373c06b500 0x7f373c06d9c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.795+0000 7f3755cbc640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3750161850 0x7f37500b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.795+0000 7f3755cbc640 1 --2- 172.21.15.139:0/1895881154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37500d0970 0x7f3750152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.795+0000 7f3755cbc640 1 -- 172.21.15.139:0/1895881154 >> 172.21.15.139:0/1895881154 conn(0x7f37500b5910 msgr2=0x7f37500c74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:03.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.795+0000 7f3755cbc640 1 -- 172.21.15.139:0/1895881154 shutdown_connections 2024-04-25T10:01:03.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.795+0000 7f3755cbc640 1 -- 172.21.15.139:0/1895881154 wait complete. 2024-04-25T10:01:03.812 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:01:03.812 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:03.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.904+0000 7ff6e2370640 1 Processor -- start 2024-04-25T10:01:03.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.904+0000 7ff6e2370640 1 -- start start 2024-04-25T10:01:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.904+0000 7ff6e2370640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff6dc15dc60 0x7ff6dc0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.904+0000 7ff6e2370640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6dc161850 0x7ff6dc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.904+0000 7ff6e2370640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff6dc0d0970 0x7ff6dc0d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.904+0000 7ff6e2370640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6dc0ce780 con 0x7ff6dc15dc60 2024-04-25T10:01:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.904+0000 7ff6e2370640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6dc0d3730 con 0x7ff6dc0d0970 2024-04-25T10:01:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.904+0000 7ff6e2370640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0cf250 con 0x7ff6dc161850 2024-04-25T10:01:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.904+0000 7ff6dbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff6dc15dc60 0x7ff6dc0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.905+0000 7ff6dbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff6dc15dc60 0x7ff6dc0b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54616/0 (socket says 172.21.15.139:54616) 2024-04-25T10:01:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.905+0000 7ff6dbfff640 1 -- 172.21.15.139:0/1409468732 learned_addr learned my addr 172.21.15.139:0/1409468732 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.905+0000 7ff6db7fe640 1 --2- 172.21.15.139:0/1409468732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6dc161850 0x7ff6dc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.905+0000 7ff6e090d640 1 --2- 172.21.15.139:0/1409468732 >> v2:172.21.15.150:3300/0 conn(0x7ff6dc0d0970 0x7ff6dc0d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.905+0000 7ff6dbfff640 1 -- 172.21.15.139:0/1409468732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6dc161850 msgr2=0x7ff6dc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.905+0000 7ff6dbfff640 1 --2- 172.21.15.139:0/1409468732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6dc161850 0x7ff6dc0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.905+0000 7ff6dbfff640 1 -- 172.21.15.139:0/1409468732 >> v2:172.21.15.150:3300/0 conn(0x7ff6dc0d0970 msgr2=0x7ff6dc0d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.905+0000 7ff6dbfff640 1 --2- 172.21.15.139:0/1409468732 >> v2:172.21.15.150:3300/0 conn(0x7ff6dc0d0970 0x7ff6dc0d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.905+0000 7ff6dbfff640 1 -- 172.21.15.139:0/1409468732 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc0ce9b0 con 0x7ff6dc15dc60 2024-04-25T10:01:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.905+0000 7ff6e090d640 1 --2- 172.21.15.139:0/1409468732 >> v2:172.21.15.150:3300/0 conn(0x7ff6dc0d0970 0x7ff6dc0d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.905+0000 7ff6db7fe640 1 --2- 172.21.15.139:0/1409468732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6dc161850 0x7ff6dc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6dbfff640 1 --2- 172.21.15.139:0/1409468732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6dc15dc60 0x7ff6dc0b6c10 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc06d250 tx=0x7ff6cc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9c83de10bc7f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6daffd640 1 -- 172.21.15.139:0/1409468732 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc037070 con 0x7ff6dc15dc60 2024-04-25T10:01:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6daffd640 1 -- 172.21.15.139:0/1409468732 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6cc032440 con 0x7ff6dc15dc60 2024-04-25T10:01:03.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6daffd640 1 -- 172.21.15.139:0/1409468732 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc032810 con 0x7ff6dc15dc60 2024-04-25T10:01:03.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6e2370640 1 -- 172.21.15.139:0/1409468732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6dc15dc60 msgr2=0x7ff6dc0b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6e2370640 1 --2- 172.21.15.139:0/1409468732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6dc15dc60 0x7ff6dc0b6c10 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc06d250 tx=0x7ff6cc02a210 comp rx=0 tx=0).stop 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6e2370640 1 -- 172.21.15.139:0/1409468732 shutdown_connections 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6e2370640 1 --2- 172.21.15.139:0/1409468732 >> v2:172.21.15.150:3300/0 conn(0x7ff6dc0d0970 0x7ff6dc0d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6e2370640 1 --2- 172.21.15.139:0/1409468732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6dc161850 0x7ff6dc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6e2370640 1 --2- 172.21.15.139:0/1409468732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6dc15dc60 0x7ff6dc0b6c10 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6e2370640 1 -- 172.21.15.139:0/1409468732 >> 172.21.15.139:0/1409468732 conn(0x7ff6dc0b5910 msgr2=0x7ff6dc0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6e2370640 1 -- 172.21.15.139:0/1409468732 shutdown_connections 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.906+0000 7ff6e2370640 1 -- 172.21.15.139:0/1409468732 wait complete. 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.907+0000 7ff6e2370640 1 Processor -- start 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.908+0000 7ff6e2370640 1 -- start start 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.908+0000 7ff6e2370640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6dc0d0970 0x7ff6dc1f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.908+0000 7ff6e2370640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6dc15dc60 0x7ff6dc0c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.908+0000 7ff6e2370640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6dc161850 0x7ff6dc0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.908+0000 7ff6e2370640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0d1fc0 con 0x7ff6dc15dc60 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.908+0000 7ff6e2370640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0cef70 con 0x7ff6dc0d0970 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.908+0000 7ff6e2370640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc15e040 con 0x7ff6dc161850 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.908+0000 7ff6db7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6dc15dc60 0x7ff6dc0c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.908+0000 7ff6db7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6dc15dc60 0x7ff6dc0c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54622/0 (socket says 172.21.15.139:54622) 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.908+0000 7ff6db7fe640 1 -- 172.21.15.139:0/3888302012 learned_addr learned my addr 172.21.15.139:0/3888302012 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.909+0000 7ff6db7fe640 1 -- 172.21.15.139:0/3888302012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6dc161850 msgr2=0x7ff6dc0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.909+0000 7ff6db7fe640 1 --2- 172.21.15.139:0/3888302012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6dc161850 0x7ff6dc0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.909+0000 7ff6db7fe640 1 -- 172.21.15.139:0/3888302012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6dc0d0970 msgr2=0x7ff6dc1f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.909+0000 7ff6db7fe640 1 --2- 172.21.15.139:0/3888302012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6dc0d0970 0x7ff6dc1f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.909+0000 7ff6db7fe640 1 -- 172.21.15.139:0/3888302012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc0c8ed0 con 0x7ff6dc15dc60 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.909+0000 7ff6db7fe640 1 --2- 172.21.15.139:0/3888302012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6dc15dc60 0x7ff6dc0c9130 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000c820 tx=0x7ff6d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.909+0000 7ff6d8ff9640 1 -- 172.21.15.139:0/3888302012 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0015070 con 0x7ff6dc15dc60 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.909+0000 7ff6d8ff9640 1 -- 172.21.15.139:0/3888302012 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d0007a30 con 0x7ff6dc15dc60 2024-04-25T10:01:03.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.909+0000 7ff6e2370640 1 -- 172.21.15.139:0/3888302012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6dc0c8bf0 con 0x7ff6dc15dc60 2024-04-25T10:01:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.909+0000 7ff6d8ff9640 1 -- 172.21.15.139:0/3888302012 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0007de0 con 0x7ff6dc15dc60 2024-04-25T10:01:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.909+0000 7ff6e2370640 1 -- 172.21.15.139:0/3888302012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6dc0cae30 con 0x7ff6dc15dc60 2024-04-25T10:01:03.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.910+0000 7ff6e2370640 1 -- 172.21.15.139:0/3888302012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6dc074520 con 0x7ff6dc15dc60 2024-04-25T10:01:03.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.910+0000 7ff6d8ff9640 1 -- 172.21.15.139:0/3888302012 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6d0018070 con 0x7ff6dc15dc60 2024-04-25T10:01:03.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.911+0000 7ff6d8ff9640 1 --2- 172.21.15.139:0/3888302012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff6c806b4d0 0x7ff6c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.911+0000 7ff6d8ff9640 1 -- 172.21.15.139:0/3888302012 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7ff6d008a360 con 0x7ff6dc15dc60 2024-04-25T10:01:03.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.911+0000 7ff6dbfff640 1 --2- 172.21.15.139:0/3888302012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff6c806b4d0 0x7ff6c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.911+0000 7ff6dbfff640 1 --2- 172.21.15.139:0/3888302012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff6c806b4d0 0x7ff6c806d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc034000 tx=0x7ff6cc02c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:03.915+0000 7ff6d8ff9640 1 -- 172.21.15.139:0/3888302012 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6d0058f30 con 0x7ff6dc15dc60 2024-04-25T10:01:04.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.121+0000 7ff6e2370640 1 -- 172.21.15.139:0/3888302012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6dc0d16f0 con 0x7ff6dc15dc60 2024-04-25T10:01:04.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.121+0000 7ff6d8ff9640 1 -- 172.21.15.139:0/3888302012 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4953 (secure 0 0 0) 0x7ff6d0058830 con 0x7ff6dc15dc60 2024-04-25T10:01:04.127 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:04.127 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T10:00:25.131250+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6836/1457343244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1457343244},{"type":"v1","addr":"172.21.15.139:6836","nonce":1457343244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:01:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 63 2024-04-25T10:01:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.123+0000 7ff6e2370640 1 -- 172.21.15.139:0/3888302012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff6c806b4d0 msgr2=0x7ff6c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.123+0000 7ff6e2370640 1 --2- 172.21.15.139:0/3888302012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff6c806b4d0 0x7ff6c806d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc034000 tx=0x7ff6cc02c000 comp rx=0 tx=0).stop 2024-04-25T10:01:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.123+0000 7ff6e2370640 1 -- 172.21.15.139:0/3888302012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6dc15dc60 msgr2=0x7ff6dc0c9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.123+0000 7ff6e2370640 1 --2- 172.21.15.139:0/3888302012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6dc15dc60 0x7ff6dc0c9130 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000c820 tx=0x7ff6d000cce0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.123+0000 7ff6e2370640 1 -- 172.21.15.139:0/3888302012 shutdown_connections 2024-04-25T10:01:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.123+0000 7ff6e2370640 1 --2- 172.21.15.139:0/3888302012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff6dc15dc60 0x7ff6dc0c9130 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.123+0000 7ff6e2370640 1 --2- 172.21.15.139:0/3888302012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff6c806b4d0 0x7ff6c806d990 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.123+0000 7ff6e2370640 1 --2- 172.21.15.139:0/3888302012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6dc161850 0x7ff6dc0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.123+0000 7ff6e2370640 1 --2- 172.21.15.139:0/3888302012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6dc0d0970 0x7ff6dc1f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.123+0000 7ff6e2370640 1 -- 172.21.15.139:0/3888302012 >> 172.21.15.139:0/3888302012 conn(0x7ff6dc0b5910 msgr2=0x7ff6dc153de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.123+0000 7ff6e2370640 1 -- 172.21.15.139:0/3888302012 shutdown_connections 2024-04-25T10:01:04.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.123+0000 7ff6e2370640 1 -- 172.21.15.139:0/3888302012 wait complete. 2024-04-25T10:01:04.139 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:04.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.231+0000 7f5c87241640 1 Processor -- start 2024-04-25T10:01:04.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.231+0000 7f5c87241640 1 -- start start 2024-04-25T10:01:04.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.232+0000 7f5c87241640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5c800b0210 0x7f5c800b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.232+0000 7f5c87241640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c800ce220 con 0x7f5c8015ae40 2024-04-25T10:01:04.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.232+0000 7f5c87241640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c800ce500 con 0x7f5c800b1960 2024-04-25T10:01:04.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.232+0000 7f5c87241640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c800b1070 con 0x7f5c800b0210 2024-04-25T10:01:04.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.232+0000 7f5c857de640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5c800b0210 0x7f5c800b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.232+0000 7f5c84fdd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5c8015ae40 0x7f5c800cefe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37212/0 (socket says 172.21.15.139:37212) 2024-04-25T10:01:04.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.232+0000 7f5c857de640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5c800b0210 0x7f5c800b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54632/0 (socket says 172.21.15.139:54632) 2024-04-25T10:01:04.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.232+0000 7f5c84fdd640 1 -- 172.21.15.139:0/2001367736 learned_addr learned my addr 172.21.15.139:0/2001367736 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:04.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.232+0000 7f5c857de640 1 -- 172.21.15.139:0/2001367736 >> v1:172.21.15.150:6789/0 conn(0x7f5c800b1960 legacy=0x7f5c8006b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:04.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.232+0000 7f5c857de640 1 -- 172.21.15.139:0/2001367736 >> v1:172.21.15.139:6789/0 conn(0x7f5c8015ae40 legacy=0x7f5c800cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.232+0000 7f5c7f7fe640 1 -- 172.21.15.139:0/2001367736 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954801199 0 0) 0x7f5c800ce220 con 0x7f5c8015ae40 2024-04-25T10:01:04.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.232+0000 7f5c857de640 1 -- 172.21.15.139:0/2001367736 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c800ce7e0 con 0x7f5c800b0210 2024-04-25T10:01:04.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.233+0000 7f5c857de640 1 --2- 172.21.15.139:0/2001367736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c800b0210 0x7f5c800b06a0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009d90 tx=0x7f5c7802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7830d00bb894d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.233+0000 7f5c7f7fe640 1 -- 172.21.15.139:0/2001367736 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78007740 con 0x7f5c800b0210 2024-04-25T10:01:04.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.233+0000 7f5c7f7fe640 1 -- 172.21.15.139:0/2001367736 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c780078e0 con 0x7f5c800b0210 2024-04-25T10:01:04.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.233+0000 7f5c7f7fe640 1 -- 172.21.15.139:0/2001367736 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78007cb0 con 0x7f5c800b0210 2024-04-25T10:01:04.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.233+0000 7f5c87241640 1 -- 172.21.15.139:0/2001367736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c800b0210 msgr2=0x7f5c800b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.233+0000 7f5c87241640 1 --2- 172.21.15.139:0/2001367736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c800b0210 0x7f5c800b06a0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009d90 tx=0x7f5c7802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.234+0000 7f5c87241640 1 -- 172.21.15.139:0/2001367736 shutdown_connections 2024-04-25T10:01:04.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.234+0000 7f5c87241640 1 --2- 172.21.15.139:0/2001367736 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c800b0210 0x7f5c800b06a0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.234+0000 7f5c87241640 1 -- 172.21.15.139:0/2001367736 >> 172.21.15.139:0/2001367736 conn(0x7f5c80066f20 msgr2=0x7f5c8006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:04.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.234+0000 7f5c87241640 1 -- 172.21.15.139:0/2001367736 shutdown_connections 2024-04-25T10:01:04.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.234+0000 7f5c87241640 1 -- 172.21.15.139:0/2001367736 wait complete. 2024-04-25T10:01:04.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.235+0000 7f5c87241640 1 Processor -- start 2024-04-25T10:01:04.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.235+0000 7f5c87241640 1 -- start start 2024-04-25T10:01:04.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.235+0000 7f5c87241640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c800b0210 0x7f5c801ef090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.235+0000 7f5c87241640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c800b1960 0x7f5c801f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c87241640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c8015ae40 0x7f5c801f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c87241640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8006b980 con 0x7f5c800b0210 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c87241640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8015b3e0 con 0x7f5c8015ae40 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c87241640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c801484c0 con 0x7f5c800b1960 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c84fdd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c800b0210 0x7f5c801ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c84fdd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c800b0210 0x7f5c801ef090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54644/0 (socket says 172.21.15.139:54644) 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c84fdd640 1 -- 172.21.15.139:0/1410777301 learned_addr learned my addr 172.21.15.139:0/1410777301 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c84fdd640 1 -- 172.21.15.139:0/1410777301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c800b1960 msgr2=0x7f5c801f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c84fdd640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c800b1960 0x7f5c801f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c857de640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c8015ae40 0x7f5c801f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c84fdd640 1 -- 172.21.15.139:0/1410777301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c8015ae40 msgr2=0x7f5c801f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c84fdd640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c8015ae40 0x7f5c801f3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c84fdd640 1 -- 172.21.15.139:0/1410777301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c801f39c0 con 0x7f5c800b0210 2024-04-25T10:01:04.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c84fdd640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c800b0210 0x7f5c801ef090 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400c430 tx=0x7f5c7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c7d7fa640 1 -- 172.21.15.139:0/1410777301 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c740074e0 con 0x7f5c800b0210 2024-04-25T10:01:04.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c7d7fa640 1 -- 172.21.15.139:0/1410777301 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c740047e0 con 0x7f5c800b0210 2024-04-25T10:01:04.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c87241640 1 -- 172.21.15.139:0/1410777301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c801f3ca0 con 0x7f5c800b0210 2024-04-25T10:01:04.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c7d7fa640 1 -- 172.21.15.139:0/1410777301 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74004bd0 con 0x7f5c800b0210 2024-04-25T10:01:04.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.236+0000 7f5c87241640 1 -- 172.21.15.139:0/1410777301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c800a8790 con 0x7f5c800b0210 2024-04-25T10:01:04.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.237+0000 7f5c87241640 1 -- 172.21.15.139:0/1410777301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c800cf510 con 0x7f5c800b0210 2024-04-25T10:01:04.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.238+0000 7f5c7d7fa640 1 -- 172.21.15.139:0/1410777301 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c74011070 con 0x7f5c800b0210 2024-04-25T10:01:04.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.238+0000 7f5c7d7fa640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5c6406b4d0 0x7f5c6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.238+0000 7f5c7d7fa640 1 -- 172.21.15.139:0/1410777301 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f5c7408a170 con 0x7f5c800b0210 2024-04-25T10:01:04.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.238+0000 7f5c7ffff640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5c6406b4d0 0x7f5c6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.239+0000 7f5c7ffff640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5c6406b4d0 0x7f5c6406d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5c8006b220 tx=0x7f5c6c0079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.242+0000 7f5c7d7fa640 1 -- 172.21.15.139:0/1410777301 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c800cf510 con 0x7f5c800b0210 2024-04-25T10:01:04.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.449+0000 7f5c87241640 1 -- 172.21.15.139:0/1410777301 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c80074520 con 0x7f5c800b0210 2024-04-25T10:01:04.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.449+0000 7f5c7d7fa640 1 -- 172.21.15.139:0/1410777301 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f5c74058d40 con 0x7f5c800b0210 2024-04-25T10:01:04.455 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:04.455 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:57.374595+0000","modified":"2024-04-25T10:00:25.131250+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6836/1457343244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1457343244},{"type":"v1","addr":"172.21.15.139:6836","nonce":1457343244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:01:04.455 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 63 2024-04-25T10:01:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.451+0000 7f5c87241640 1 -- 172.21.15.139:0/1410777301 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5c6406b4d0 msgr2=0x7f5c6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.451+0000 7f5c87241640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5c6406b4d0 0x7f5c6406d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5c8006b220 tx=0x7f5c6c0079b0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.451+0000 7f5c87241640 1 -- 172.21.15.139:0/1410777301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c800b0210 msgr2=0x7f5c801ef090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.451+0000 7f5c87241640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c800b0210 0x7f5c801ef090 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400c430 tx=0x7f5c7400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.451+0000 7f5c87241640 1 -- 172.21.15.139:0/1410777301 shutdown_connections 2024-04-25T10:01:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.451+0000 7f5c87241640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5c6406b4d0 0x7f5c6406d990 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.451+0000 7f5c87241640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c8015ae40 0x7f5c801f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.451+0000 7f5c87241640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c800b1960 0x7f5c801f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.451+0000 7f5c87241640 1 --2- 172.21.15.139:0/1410777301 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c800b0210 0x7f5c801ef090 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.451+0000 7f5c87241640 1 -- 172.21.15.139:0/1410777301 >> 172.21.15.139:0/1410777301 conn(0x7f5c80066f20 msgr2=0x7f5c800b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.451+0000 7f5c87241640 1 -- 172.21.15.139:0/1410777301 shutdown_connections 2024-04-25T10:01:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.452+0000 7f5c87241640 1 -- 172.21.15.139:0/1410777301 wait complete. 2024-04-25T10:01:04.468 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:04.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.559+0000 7f56b904c640 1 Processor -- start 2024-04-25T10:01:04.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.559+0000 7f56b904c640 1 -- start start 2024-04-25T10:01:04.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.560+0000 7f56b904c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56b415aa00 0x7f56b40b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.560+0000 7f56b904c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f56b40b5110 0x7f56b415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.560+0000 7f56b904c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b415e500 con 0x7f56b4072280 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.560+0000 7f56b904c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b40b5c20 con 0x7f56b40b5110 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.560+0000 7f56b904c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40b45d0 con 0x7f56b415aa00 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.560+0000 7f56b359e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f56b4072280 0x7f56b40ce610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37214/0 (socket says 172.21.15.139:37214) 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.560+0000 7f56b359e640 1 -- 172.21.15.139:0/1122657766 learned_addr learned my addr 172.21.15.139:0/1122657766 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.560+0000 7f56b259c640 1 --2- 172.21.15.139:0/1122657766 >> v2:172.21.15.150:3300/0 conn(0x7f56b40b5110 0x7f56b415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.560+0000 7f56b2d9d640 1 --2- 172.21.15.139:0/1122657766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56b415aa00 0x7f56b40b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.561+0000 7f56b1d9b640 1 -- 172.21.15.139:0/1122657766 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194509901 0 0) 0x7f56b415e500 con 0x7f56b4072280 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.561+0000 7f56b1d9b640 1 -- 172.21.15.139:0/1122657766 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f569c003520 con 0x7f56b4072280 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.561+0000 7f56b2d9d640 1 -- 172.21.15.139:0/1122657766 >> v2:172.21.15.150:3300/0 conn(0x7f56b40b5110 msgr2=0x7f56b415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.561+0000 7f56b2d9d640 1 --2- 172.21.15.139:0/1122657766 >> v2:172.21.15.150:3300/0 conn(0x7f56b40b5110 0x7f56b415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.561+0000 7f56b2d9d640 1 -- 172.21.15.139:0/1122657766 >> v1:172.21.15.139:6789/0 conn(0x7f56b4072280 legacy=0x7f56b40ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.561+0000 7f56b1d9b640 1 -- 172.21.15.139:0/1122657766 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3305285276 0 0) 0x7f569c003520 con 0x7f56b4072280 2024-04-25T10:01:04.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.561+0000 7f56b2d9d640 1 -- 172.21.15.139:0/1122657766 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b40b5660 con 0x7f56b415aa00 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.561+0000 7f56b259c640 1 --2- 172.21.15.139:0/1122657766 >> v2:172.21.15.150:3300/0 conn(0x7f56b40b5110 0x7f56b415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.562+0000 7f56b2d9d640 1 --2- 172.21.15.139:0/1122657766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56b415aa00 0x7f56b40b4090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009d90 tx=0x7f56a402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f8432e76244e80d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.562+0000 7f56b1d9b640 1 -- 172.21.15.139:0/1122657766 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac001670 con 0x7f56b415aa00 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.562+0000 7f56b1d9b640 1 -- 172.21.15.139:0/1122657766 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a4007540 con 0x7f56b415aa00 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.562+0000 7f56b1d9b640 1 -- 172.21.15.139:0/1122657766 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a4007910 con 0x7f56b415aa00 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.562+0000 7f56b904c640 1 -- 172.21.15.139:0/1122657766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56b415aa00 msgr2=0x7f56b40b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.562+0000 7f56b904c640 1 --2- 172.21.15.139:0/1122657766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56b415aa00 0x7f56b40b4090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009d90 tx=0x7f56a402a5f0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.562+0000 7f56b904c640 1 -- 172.21.15.139:0/1122657766 shutdown_connections 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.562+0000 7f56b904c640 1 --2- 172.21.15.139:0/1122657766 >> v2:172.21.15.150:3300/0 conn(0x7f56b40b5110 0x7f56b415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.562+0000 7f56b904c640 1 --2- 172.21.15.139:0/1122657766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56b415aa00 0x7f56b40b4090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.562+0000 7f56b904c640 1 -- 172.21.15.139:0/1122657766 >> 172.21.15.139:0/1122657766 conn(0x7f56b406e280 msgr2=0x7f56b406e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.562+0000 7f56b904c640 1 -- 172.21.15.139:0/1122657766 shutdown_connections 2024-04-25T10:01:04.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.562+0000 7f56b904c640 1 -- 172.21.15.139:0/1122657766 wait complete. 2024-04-25T10:01:04.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.564+0000 7f56b904c640 1 Processor -- start 2024-04-25T10:01:04.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.564+0000 7f56b904c640 1 -- start start 2024-04-25T10:01:04.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.564+0000 7f56b904c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56b4072280 0x7f56b415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.564+0000 7f56b904c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b40b5110 0x7f56b415d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.564+0000 7f56b904c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56b41492a0 0x7f56b41f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.564+0000 7f56b904c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b415f200 con 0x7f56b40b5110 2024-04-25T10:01:04.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.564+0000 7f56b904c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b415e7a0 con 0x7f56b41492a0 2024-04-25T10:01:04.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.564+0000 7f56b904c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4072710 con 0x7f56b4072280 2024-04-25T10:01:04.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b40b5110 0x7f56b415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b40b5110 0x7f56b415d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54658/0 (socket says 172.21.15.139:54658) 2024-04-25T10:01:04.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b259c640 1 -- 172.21.15.139:0/3134368975 learned_addr learned my addr 172.21.15.139:0/3134368975 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:04.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b2d9d640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56b4072280 0x7f56b415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b259c640 1 -- 172.21.15.139:0/3134368975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56b4072280 msgr2=0x7f56b415b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b259c640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56b4072280 0x7f56b415b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b359e640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56b41492a0 0x7f56b41f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b259c640 1 -- 172.21.15.139:0/3134368975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56b41492a0 msgr2=0x7f56b41f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b259c640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56b41492a0 0x7f56b41f7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b259c640 1 -- 172.21.15.139:0/3134368975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b415afe0 con 0x7f56b40b5110 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b259c640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b40b5110 0x7f56b415d820 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f56a800ebe0 tx=0x7f56a800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f569b7fe640 1 -- 172.21.15.139:0/3134368975 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8017070 con 0x7f56b40b5110 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f569b7fe640 1 -- 172.21.15.139:0/3134368975 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a800cbd0 con 0x7f56b40b5110 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b904c640 1 -- 172.21.15.139:0/3134368975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b415b280 con 0x7f56b40b5110 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f569b7fe640 1 -- 172.21.15.139:0/3134368975 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8013470 con 0x7f56b40b5110 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.565+0000 7f56b904c640 1 -- 172.21.15.139:0/3134368975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b40af660 con 0x7f56b40b5110 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.566+0000 7f56b904c640 1 -- 172.21.15.139:0/3134368975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b4067bb0 con 0x7f56b40b5110 2024-04-25T10:01:04.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.567+0000 7f569b7fe640 1 -- 172.21.15.139:0/3134368975 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56a801a070 con 0x7f56b40b5110 2024-04-25T10:01:04.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.567+0000 7f569b7fe640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56a006b4d0 0x7f56a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.567+0000 7f56b2d9d640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56a006b4d0 0x7f56a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.567+0000 7f569b7fe640 1 -- 172.21.15.139:0/3134368975 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f56a808a780 con 0x7f56b40b5110 2024-04-25T10:01:04.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.568+0000 7f56b2d9d640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56a006b4d0 0x7f56a006d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f56a4034000 tx=0x7f56a4033040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.572+0000 7f569b7fe640 1 -- 172.21.15.139:0/3134368975 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56b4067bb0 con 0x7f56b40b5110 2024-04-25T10:01:04.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.743+0000 7f56b904c640 1 -- 172.21.15.139:0/3134368975 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56b4074520 con 0x7f56b40b5110 2024-04-25T10:01:04.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.744+0000 7f569b7fe640 1 -- 172.21.15.139:0/3134368975 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17389 (secure 0 0 0) 0x7f56a8088e20 con 0x7f56b40b5110 2024-04-25T10:01:04.749 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:04.749 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":93,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:00:22.118995+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:55.585380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:56.336230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6836/1239243954":"2024-04-26T10:00:22.118970+0000","172.21.15.139:6834/1239243954":"2024-04-26T10:00:22.118970+0000","172.21.15.150:6835/2266713846":"2024-04-26T09:59:10.897062+0000","172.21.15.150:6833/2266713846":"2024-04-26T09:59:10.897062+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:04.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.746+0000 7f56b904c640 1 -- 172.21.15.139:0/3134368975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56a006b4d0 msgr2=0x7f56a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.746+0000 7f56b904c640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56a006b4d0 0x7f56a006d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f56a4034000 tx=0x7f56a4033040 comp rx=0 tx=0).stop 2024-04-25T10:01:04.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.746+0000 7f56b904c640 1 -- 172.21.15.139:0/3134368975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b40b5110 msgr2=0x7f56b415d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.746+0000 7f56b904c640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b40b5110 0x7f56b415d820 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f56a800ebe0 tx=0x7f56a800c520 comp rx=0 tx=0).stop 2024-04-25T10:01:04.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.746+0000 7f56b904c640 1 -- 172.21.15.139:0/3134368975 shutdown_connections 2024-04-25T10:01:04.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.746+0000 7f56b904c640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b40b5110 0x7f56b415d820 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.746+0000 7f56b904c640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56a006b4d0 0x7f56a006d990 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.746+0000 7f56b904c640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56b41492a0 0x7f56b41f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.746+0000 7f56b904c640 1 --2- 172.21.15.139:0/3134368975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56b4072280 0x7f56b415b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.746+0000 7f56b904c640 1 -- 172.21.15.139:0/3134368975 >> 172.21.15.139:0/3134368975 conn(0x7f56b406e280 msgr2=0x7f56b40b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:04.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.746+0000 7f56b904c640 1 -- 172.21.15.139:0/3134368975 shutdown_connections 2024-04-25T10:01:04.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.746+0000 7f56b904c640 1 -- 172.21.15.139:0/3134368975 wait complete. 2024-04-25T10:01:04.762 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:01:04.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.855+0000 7f47a0ff4640 1 Processor -- start 2024-04-25T10:01:04.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.855+0000 7f47a0ff4640 1 -- start start 2024-04-25T10:01:04.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.856+0000 7f47a0ff4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f479c15a870 0x7f479c0b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.856+0000 7f47a0ff4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f479c0b4190 0x7f479c0d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.856+0000 7f47a0ff4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f479c0b45f0 con 0x7f479c070260 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.856+0000 7f47a0ff4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479c0d1020 con 0x7f479c15a870 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.856+0000 7f47a0ff4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479c0b5190 con 0x7f479c0b4190 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.856+0000 7f479a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f479c15a870 0x7f479c0b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.856+0000 7f479a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f479c15a870 0x7f479c0b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54668/0 (socket says 172.21.15.139:54668) 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.856+0000 7f479a59c640 1 -- 172.21.15.139:0/2282159344 learned_addr learned my addr 172.21.15.139:0/2282159344 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.857+0000 7f479a59c640 1 -- 172.21.15.139:0/2282159344 >> v2:172.21.15.150:3300/0 conn(0x7f479c0b4190 msgr2=0x7f479c0d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.857+0000 7f4793fff640 1 --2- 172.21.15.139:0/2282159344 >> v2:172.21.15.150:3300/0 conn(0x7f479c0b4190 0x7f479c0d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.857+0000 7f479a59c640 1 --2- 172.21.15.139:0/2282159344 >> v2:172.21.15.150:3300/0 conn(0x7f479c0b4190 0x7f479c0d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.857+0000 7f479a59c640 1 -- 172.21.15.139:0/2282159344 >> v1:172.21.15.150:6789/0 conn(0x7f479c070260 legacy=0x7f479c0ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.857+0000 7f479a59c640 1 -- 172.21.15.139:0/2282159344 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c0d0d40 con 0x7f479c15a870 2024-04-25T10:01:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.857+0000 7f479a59c640 1 --2- 172.21.15.139:0/2282159344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f479c15a870 0x7f479c0b5620 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f479c0afed0 tx=0x7f478c02c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a771f3e6b6c22d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.857+0000 7f4799d9b640 1 -- 172.21.15.139:0/2282159344 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c031710 con 0x7f479c15a870 2024-04-25T10:01:04.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.858+0000 7f4799d9b640 1 -- 172.21.15.139:0/2282159344 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c0318b0 con 0x7f479c15a870 2024-04-25T10:01:04.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.858+0000 7f4799d9b640 1 -- 172.21.15.139:0/2282159344 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c031c80 con 0x7f479c15a870 2024-04-25T10:01:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.858+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2282159344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f479c15a870 msgr2=0x7f479c0b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.858+0000 7f47a0ff4640 1 --2- 172.21.15.139:0/2282159344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f479c15a870 0x7f479c0b5620 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f479c0afed0 tx=0x7f478c02c7d0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.858+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2282159344 shutdown_connections 2024-04-25T10:01:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.858+0000 7f47a0ff4640 1 --2- 172.21.15.139:0/2282159344 >> v2:172.21.15.150:3300/0 conn(0x7f479c0b4190 0x7f479c0d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.858+0000 7f47a0ff4640 1 --2- 172.21.15.139:0/2282159344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f479c15a870 0x7f479c0b5620 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.858+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2282159344 >> 172.21.15.139:0/2282159344 conn(0x7f479c0b23f0 msgr2=0x7f479c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.858+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2282159344 shutdown_connections 2024-04-25T10:01:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.858+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2282159344 wait complete. 2024-04-25T10:01:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.859+0000 7f47a0ff4640 1 Processor -- start 2024-04-25T10:01:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.859+0000 7f47a0ff4640 1 -- start start 2024-04-25T10:01:04.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.860+0000 7f47a0ff4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f479c070260 0x7f479c0cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.860+0000 7f47a0ff4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f479c0b4190 0x7f479c158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.860+0000 7f47a0ff4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f479c15a870 0x7f479c159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.860+0000 7f47a0ff4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c070690 con 0x7f479c15a870 2024-04-25T10:01:04.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.860+0000 7f47a0ff4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c0b4e20 con 0x7f479c0b4190 2024-04-25T10:01:04.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.860+0000 7f47a0ff4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c070810 con 0x7f479c070260 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.860+0000 7f479ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f479c15a870 0x7f479c159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.860+0000 7f479ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f479c15a870 0x7f479c159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54676/0 (socket says 172.21.15.139:54676) 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.860+0000 7f479ad9d640 1 -- 172.21.15.139:0/2435865832 learned_addr learned my addr 172.21.15.139:0/2435865832 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f4793fff640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f479c0b4190 0x7f479c158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f479a59c640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f479c070260 0x7f479c0cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f479ad9d640 1 -- 172.21.15.139:0/2435865832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f479c070260 msgr2=0x7f479c0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f479ad9d640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f479c070260 0x7f479c0cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f479ad9d640 1 -- 172.21.15.139:0/2435865832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f479c0b4190 msgr2=0x7f479c158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f479ad9d640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f479c0b4190 0x7f479c158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f479ad9d640 1 -- 172.21.15.139:0/2435865832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c0c58b0 con 0x7f479c15a870 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f4793fff640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f479c0b4190 0x7f479c158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f479a59c640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f479c070260 0x7f479c0cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f479ad9d640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f479c15a870 0x7f479c159190 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f479400c7b0 tx=0x7f479400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f4792ffd640 1 -- 172.21.15.139:0/2435865832 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794017070 con 0x7f479c15a870 2024-04-25T10:01:04.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f4792ffd640 1 -- 172.21.15.139:0/2435865832 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4794003da0 con 0x7f479c15a870 2024-04-25T10:01:04.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2435865832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479c0c5b90 con 0x7f479c15a870 2024-04-25T10:01:04.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f4792ffd640 1 -- 172.21.15.139:0/2435865832 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47940052b0 con 0x7f479c15a870 2024-04-25T10:01:04.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.861+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2435865832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479c0c6150 con 0x7f479c15a870 2024-04-25T10:01:04.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.862+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2435865832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479c074520 con 0x7f479c15a870 2024-04-25T10:01:04.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.862+0000 7f4792ffd640 1 -- 172.21.15.139:0/2435865832 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f479401a070 con 0x7f479c15a870 2024-04-25T10:01:04.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.863+0000 7f4792ffd640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f477c06b4d0 0x7f477c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.863+0000 7f479a59c640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f477c06b4d0 0x7f477c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.863+0000 7f4792ffd640 1 -- 172.21.15.139:0/2435865832 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f4794089a60 con 0x7f479c15a870 2024-04-25T10:01:04.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.863+0000 7f479a59c640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f477c06b4d0 0x7f477c06d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f478c036000 tx=0x7f478c035040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:04.867+0000 7f4792ffd640 1 -- 172.21.15.139:0/2435865832 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47940586c0 con 0x7f479c15a870 2024-04-25T10:01:05.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.076+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2435865832 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f479c0cdc60 con 0x7f479c15a870 2024-04-25T10:01:05.081 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T10:01:05.076+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:01:05.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.077+0000 7f4792ffd640 1 -- 172.21.15.139:0/2435865832 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4794057fc0 con 0x7f479c15a870 2024-04-25T10:01:05.083 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:01:05.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.080+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2435865832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f477c06b4d0 msgr2=0x7f477c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.080+0000 7f47a0ff4640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f477c06b4d0 0x7f477c06d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f478c036000 tx=0x7f478c035040 comp rx=0 tx=0).stop 2024-04-25T10:01:05.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.080+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2435865832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f479c15a870 msgr2=0x7f479c159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.080+0000 7f47a0ff4640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f479c15a870 0x7f479c159190 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f479400c7b0 tx=0x7f479400cc70 comp rx=0 tx=0).stop 2024-04-25T10:01:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.081+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2435865832 shutdown_connections 2024-04-25T10:01:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.081+0000 7f47a0ff4640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f479c15a870 0x7f479c159190 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.081+0000 7f47a0ff4640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f477c06b4d0 0x7f477c06d990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.081+0000 7f47a0ff4640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f479c0b4190 0x7f479c158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.081+0000 7f47a0ff4640 1 --2- 172.21.15.139:0/2435865832 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f479c070260 0x7f479c0cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.081+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2435865832 >> 172.21.15.139:0/2435865832 conn(0x7f479c0b23f0 msgr2=0x7f479c0b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.081+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2435865832 shutdown_connections 2024-04-25T10:01:05.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.081+0000 7f47a0ff4640 1 -- 172.21.15.139:0/2435865832 wait complete. 2024-04-25T10:01:05.088 INFO:tasks.ceph.mds.a.smithi139.stderr: -2273> 2024-04-25T10:00:28.129+0000 7fbdccb06640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:01:05.088 INFO:tasks.ceph.mds.a.smithi139.stderr: -1879> 2024-04-25T10:00:28.672+0000 7fbdc8afe640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55a777d86100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:01:05.099 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:01:05.116 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T10:01:05.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.193+0000 7fb79766f640 1 Processor -- start 2024-04-25T10:01:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.193+0000 7fb79766f640 1 -- start start 2024-04-25T10:01:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.194+0000 7fb79766f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb79015dbf0 0x7fb7901617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.194+0000 7fb79766f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7900b9370 0x7fb7900ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.194+0000 7fb79766f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb7900cec80 0x7fb79006c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.194+0000 7fb79766f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7900d1ee0 con 0x7fb79015dbf0 2024-04-25T10:01:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.194+0000 7fb79766f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb790161dc0 con 0x7fb7900cec80 2024-04-25T10:01:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.194+0000 7fb79766f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7901620a0 con 0x7fb7900b9370 2024-04-25T10:01:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.194+0000 7fb79540b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb79015dbf0 0x7fb7901617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.194+0000 7fb79540b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb79015dbf0 0x7fb7901617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54694/0 (socket says 172.21.15.139:54694) 2024-04-25T10:01:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.194+0000 7fb79540b640 1 -- 172.21.15.139:0/1600128429 learned_addr learned my addr 172.21.15.139:0/1600128429 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.194+0000 7fb794c0a640 1 --2- 172.21.15.139:0/1600128429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7900b9370 0x7fb7900ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.194+0000 7fb795c0c640 1 --2- 172.21.15.139:0/1600128429 >> v2:172.21.15.150:3300/0 conn(0x7fb7900cec80 0x7fb79006c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.195+0000 7fb79540b640 1 -- 172.21.15.139:0/1600128429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7900b9370 msgr2=0x7fb7900ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.195+0000 7fb79540b640 1 --2- 172.21.15.139:0/1600128429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7900b9370 0x7fb7900ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.195+0000 7fb79540b640 1 -- 172.21.15.139:0/1600128429 >> v2:172.21.15.150:3300/0 conn(0x7fb7900cec80 msgr2=0x7fb79006c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.195+0000 7fb79540b640 1 --2- 172.21.15.139:0/1600128429 >> v2:172.21.15.150:3300/0 conn(0x7fb7900cec80 0x7fb79006c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.195+0000 7fb79540b640 1 -- 172.21.15.139:0/1600128429 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb790162380 con 0x7fb79015dbf0 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.195+0000 7fb795c0c640 1 --2- 172.21.15.139:0/1600128429 >> v2:172.21.15.150:3300/0 conn(0x7fb7900cec80 0x7fb79006c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.195+0000 7fb79540b640 1 --2- 172.21.15.139:0/1600128429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb79015dbf0 0x7fb7901617e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fb79008c170 tx=0x7fb78002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c94725639a8b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.195+0000 7fb787fff640 1 -- 172.21.15.139:0/1600128429 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780037070 con 0x7fb79015dbf0 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.195+0000 7fb787fff640 1 -- 172.21.15.139:0/1600128429 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb780032440 con 0x7fb79015dbf0 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.195+0000 7fb787fff640 1 -- 172.21.15.139:0/1600128429 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780032810 con 0x7fb79015dbf0 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.196+0000 7fb79766f640 1 -- 172.21.15.139:0/1600128429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb79015dbf0 msgr2=0x7fb7901617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.196+0000 7fb79766f640 1 --2- 172.21.15.139:0/1600128429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb79015dbf0 0x7fb7901617e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fb79008c170 tx=0x7fb78002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.196+0000 7fb79766f640 1 -- 172.21.15.139:0/1600128429 shutdown_connections 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.196+0000 7fb79766f640 1 --2- 172.21.15.139:0/1600128429 >> v2:172.21.15.150:3300/0 conn(0x7fb7900cec80 0x7fb79006c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.196+0000 7fb79766f640 1 --2- 172.21.15.139:0/1600128429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7900b9370 0x7fb7900ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.196+0000 7fb79766f640 1 --2- 172.21.15.139:0/1600128429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb79015dbf0 0x7fb7901617e0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.196+0000 7fb79766f640 1 -- 172.21.15.139:0/1600128429 >> 172.21.15.139:0/1600128429 conn(0x7fb7900b5480 msgr2=0x7fb7900b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:05.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.196+0000 7fb79766f640 1 -- 172.21.15.139:0/1600128429 shutdown_connections 2024-04-25T10:01:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.196+0000 7fb79766f640 1 -- 172.21.15.139:0/1600128429 wait complete. 2024-04-25T10:01:05.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.197+0000 7fb79766f640 1 Processor -- start 2024-04-25T10:01:05.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.197+0000 7fb79766f640 1 -- start start 2024-04-25T10:01:05.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb79766f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7900b9370 0x7fb79014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb79766f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7900cec80 0x7fb7900aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb79766f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79015dbf0 0x7fb7900d2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb79766f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79015dfd0 con 0x7fb7900cec80 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb79766f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7900b97d0 con 0x7fb79015dbf0 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb79766f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7900aada0 con 0x7fb7900b9370 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb794c0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7900cec80 0x7fb7900aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb794c0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7900cec80 0x7fb7900aa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54698/0 (socket says 172.21.15.139:54698) 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb794c0a640 1 -- 172.21.15.139:0/746313811 learned_addr learned my addr 172.21.15.139:0/746313811 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb794c0a640 1 -- 172.21.15.139:0/746313811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7900b9370 msgr2=0x7fb79014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb794c0a640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7900b9370 0x7fb79014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb794c0a640 1 -- 172.21.15.139:0/746313811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79015dbf0 msgr2=0x7fb7900d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb795c0c640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79015dbf0 0x7fb7900d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb794c0a640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79015dbf0 0x7fb7900d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.198+0000 7fb794c0a640 1 -- 172.21.15.139:0/746313811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7900ab080 con 0x7fb7900cec80 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.199+0000 7fb794c0a640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7900cec80 0x7fb7900aa350 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00d800 tx=0x7fb78c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.199+0000 7fb785ffb640 1 -- 172.21.15.139:0/746313811 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c018030 con 0x7fb7900cec80 2024-04-25T10:01:05.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.199+0000 7fb785ffb640 1 -- 172.21.15.139:0/746313811 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c0096a0 con 0x7fb7900cec80 2024-04-25T10:01:05.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.199+0000 7fb79766f640 1 -- 172.21.15.139:0/746313811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7900ab360 con 0x7fb7900cec80 2024-04-25T10:01:05.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.199+0000 7fb785ffb640 1 -- 172.21.15.139:0/746313811 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c009870 con 0x7fb7900cec80 2024-04-25T10:01:05.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.199+0000 7fb79766f640 1 -- 172.21.15.139:0/746313811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7900ab920 con 0x7fb7900cec80 2024-04-25T10:01:05.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.200+0000 7fb79766f640 1 -- 172.21.15.139:0/746313811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb790074520 con 0x7fb7900cec80 2024-04-25T10:01:05.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.200+0000 7fb785ffb640 1 -- 172.21.15.139:0/746313811 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb78c01b070 con 0x7fb7900cec80 2024-04-25T10:01:05.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.201+0000 7fb785ffb640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb77406b4d0 0x7fb77406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.201+0000 7fb785ffb640 1 -- 172.21.15.139:0/746313811 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6475+0+0 (secure 0 0 0) 0x7fb78c08a820 con 0x7fb7900cec80 2024-04-25T10:01:05.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.201+0000 7fb79540b640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb77406b4d0 0x7fb77406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.201+0000 7fb79540b640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb77406b4d0 0x7fb77406d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb780034000 tx=0x7fb78002c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.205+0000 7fb785ffb640 1 -- 172.21.15.139:0/746313811 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb78c059310 con 0x7fb7900cec80 2024-04-25T10:01:05.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:05.411+0000 7fb79766f640 1 -- 172.21.15.139:0/746313811 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb7900d1680 con 0x7fb7900cec80 2024-04-25T10:01:06.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.084+0000 7fb785ffb640 1 -- 172.21.15.139:0/746313811 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb78c058c10 con 0x7fb7900cec80 2024-04-25T10:01:06.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.087+0000 7fb79766f640 1 -- 172.21.15.139:0/746313811 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb77406b4d0 msgr2=0x7fb77406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.087+0000 7fb79766f640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb77406b4d0 0x7fb77406d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb780034000 tx=0x7fb78002c000 comp rx=0 tx=0).stop 2024-04-25T10:01:06.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.087+0000 7fb79766f640 1 -- 172.21.15.139:0/746313811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7900cec80 msgr2=0x7fb7900aa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.087+0000 7fb79766f640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7900cec80 0x7fb7900aa350 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00d800 tx=0x7fb78c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.087+0000 7fb79766f640 1 -- 172.21.15.139:0/746313811 shutdown_connections 2024-04-25T10:01:06.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.087+0000 7fb79766f640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7900cec80 0x7fb7900aa350 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.087+0000 7fb79766f640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb77406b4d0 0x7fb77406d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.087+0000 7fb79766f640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79015dbf0 0x7fb7900d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.087+0000 7fb79766f640 1 --2- 172.21.15.139:0/746313811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7900b9370 0x7fb79014ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.087+0000 7fb79766f640 1 -- 172.21.15.139:0/746313811 >> 172.21.15.139:0/746313811 conn(0x7fb7900b5480 msgr2=0x7fb7900d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:06.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.087+0000 7fb79766f640 1 -- 172.21.15.139:0/746313811 shutdown_connections 2024-04-25T10:01:06.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.087+0000 7fb79766f640 1 -- 172.21.15.139:0/746313811 wait complete. 2024-04-25T10:01:06.104 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:01:06.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.240+0000 7fedb1b5c640 1 Processor -- start 2024-04-25T10:01:06.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.240+0000 7fedb1b5c640 1 -- start start 2024-04-25T10:01:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedb1b5c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fedac150ed0 0x7fedac0b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedb1b5c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fedac0cdd20 0x7fedac071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedb1b5c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedac15a480 con 0x7fedac0b4740 2024-04-25T10:01:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedb1b5c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedac15a760 con 0x7fedac150ed0 2024-04-25T10:01:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedb1b5c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedac15aa40 con 0x7fedac0cdd20 2024-04-25T10:01:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedab7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fedac150ed0 0x7fedac0b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedab7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fedac150ed0 0x7fedac0b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54702/0 (socket says 172.21.15.139:54702) 2024-04-25T10:01:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedabfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fedac0b4740 0x7fedac06d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37238/0 (socket says 172.21.15.139:37238) 2024-04-25T10:01:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedab7fe640 1 -- 172.21.15.139:0/1928103287 learned_addr learned my addr 172.21.15.139:0/1928103287 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7feda3fff640 1 --2- 172.21.15.139:0/1928103287 >> v2:172.21.15.150:3300/0 conn(0x7fedac0cdd20 0x7fedac071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedab7fe640 1 -- 172.21.15.139:0/1928103287 >> v2:172.21.15.150:3300/0 conn(0x7fedac0cdd20 msgr2=0x7fedac071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedab7fe640 1 --2- 172.21.15.139:0/1928103287 >> v2:172.21.15.150:3300/0 conn(0x7fedac0cdd20 0x7fedac071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedab7fe640 1 -- 172.21.15.139:0/1928103287 >> v1:172.21.15.139:6789/0 conn(0x7fedac0b4740 legacy=0x7fedac06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedaaffd640 1 -- 172.21.15.139:0/1928103287 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918789859 0 0) 0x7fedac15a480 con 0x7fedac0b4740 2024-04-25T10:01:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.241+0000 7fedab7fe640 1 -- 172.21.15.139:0/1928103287 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac15ad20 con 0x7fedac150ed0 2024-04-25T10:01:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.242+0000 7fedab7fe640 1 --2- 172.21.15.139:0/1928103287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedac150ed0 0x7fedac0b4200 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00bb10 tx=0x7fed9c02c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3241f2bcc647987c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.242+0000 7fedaaffd640 1 -- 172.21.15.139:0/1928103287 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c032790 con 0x7fedac150ed0 2024-04-25T10:01:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.242+0000 7fedaaffd640 1 -- 172.21.15.139:0/1928103287 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c038070 con 0x7fedac150ed0 2024-04-25T10:01:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.242+0000 7fedaaffd640 1 -- 172.21.15.139:0/1928103287 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c032b50 con 0x7fedac150ed0 2024-04-25T10:01:06.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.243+0000 7fedb1b5c640 1 -- 172.21.15.139:0/1928103287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedac150ed0 msgr2=0x7fedac0b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.243+0000 7fedb1b5c640 1 --2- 172.21.15.139:0/1928103287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedac150ed0 0x7fedac0b4200 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00bb10 tx=0x7fed9c02c8d0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.243+0000 7fedb1b5c640 1 -- 172.21.15.139:0/1928103287 shutdown_connections 2024-04-25T10:01:06.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.243+0000 7fedb1b5c640 1 --2- 172.21.15.139:0/1928103287 >> v2:172.21.15.150:3300/0 conn(0x7fedac0cdd20 0x7fedac071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.243+0000 7fedb1b5c640 1 --2- 172.21.15.139:0/1928103287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedac150ed0 0x7fedac0b4200 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.243+0000 7fedb1b5c640 1 -- 172.21.15.139:0/1928103287 >> 172.21.15.139:0/1928103287 conn(0x7fedac067000 msgr2=0x7fedac06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:06.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.243+0000 7fedb1b5c640 1 -- 172.21.15.139:0/1928103287 shutdown_connections 2024-04-25T10:01:06.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.243+0000 7fedb1b5c640 1 -- 172.21.15.139:0/1928103287 wait complete. 2024-04-25T10:01:06.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.244+0000 7fedb1b5c640 1 Processor -- start 2024-04-25T10:01:06.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.244+0000 7fedb1b5c640 1 -- start start 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedb1b5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedac0b4740 0x7fedac14ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedb1b5c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedac0cdd20 0x7fedac15c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedb1b5c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedac150ed0 0x7fedac14be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedb1b5c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac1512b0 con 0x7fedac0b4740 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedb1b5c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0b3960 con 0x7fedac150ed0 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedb1b5c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac071f90 con 0x7fedac0cdd20 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedab7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedac0b4740 0x7fedac14ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedab7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedac0b4740 0x7fedac14ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54712/0 (socket says 172.21.15.139:54712) 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedab7fe640 1 -- 172.21.15.139:0/187430391 learned_addr learned my addr 172.21.15.139:0/187430391 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7feda3fff640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedac0cdd20 0x7fedac15c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedabfff640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedac150ed0 0x7fedac14be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedab7fe640 1 -- 172.21.15.139:0/187430391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedac0cdd20 msgr2=0x7fedac15c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedab7fe640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedac0cdd20 0x7fedac15c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedab7fe640 1 -- 172.21.15.139:0/187430391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedac150ed0 msgr2=0x7fedac14be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedab7fe640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedac150ed0 0x7fedac14be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedab7fe640 1 -- 172.21.15.139:0/187430391 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac14f1d0 con 0x7fedac0b4740 2024-04-25T10:01:06.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedab7fe640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedac0b4740 0x7fedac14ec90 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00b2c0 tx=0x7fed9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7feda8ff9640 1 -- 172.21.15.139:0/187430391 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c033620 con 0x7fedac0b4740 2024-04-25T10:01:06.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7feda8ff9640 1 -- 172.21.15.139:0/187430391 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c0337c0 con 0x7fedac0b4740 2024-04-25T10:01:06.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedb1b5c640 1 -- 172.21.15.139:0/187430391 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac15d1f0 con 0x7fedac0b4740 2024-04-25T10:01:06.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7feda8ff9640 1 -- 172.21.15.139:0/187430391 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c033bb0 con 0x7fedac0b4740 2024-04-25T10:01:06.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.245+0000 7fedb1b5c640 1 -- 172.21.15.139:0/187430391 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac15d7b0 con 0x7fedac0b4740 2024-04-25T10:01:06.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.246+0000 7fedb1b5c640 1 -- 172.21.15.139:0/187430391 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac0722c0 con 0x7fedac0b4740 2024-04-25T10:01:06.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.247+0000 7feda8ff9640 1 -- 172.21.15.139:0/187430391 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed9c04d060 con 0x7fedac0b4740 2024-04-25T10:01:06.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.247+0000 7feda8ff9640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed8c06b4d0 0x7fed8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.247+0000 7feda8ff9640 1 -- 172.21.15.139:0/187430391 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6475+0+0 (secure 0 0 0) 0x7fed9c0ace00 con 0x7fedac0b4740 2024-04-25T10:01:06.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.247+0000 7feda3fff640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed8c06b4d0 0x7fed8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.248+0000 7feda3fff640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed8c06b4d0 0x7fed8c06d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fedac148c60 tx=0x7fed94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.251+0000 7feda8ff9640 1 -- 172.21.15.139:0/187430391 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fedac0722c0 con 0x7fedac0b4740 2024-04-25T10:01:06.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:06.421+0000 7fedb1b5c640 1 -- 172.21.15.139:0/187430391 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fedac15dda0 con 0x7fedac0b4740 2024-04-25T10:01:07.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.087+0000 7feda8ff9640 1 -- 172.21.15.139:0/187430391 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) v1 ==== 181+0+0 (secure 0 0 0) 0x7fedac14f1d0 con 0x7fedac0b4740 2024-04-25T10:01:07.091 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:01:07.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.090+0000 7fedb1b5c640 1 -- 172.21.15.139:0/187430391 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed8c06b4d0 msgr2=0x7fed8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.090+0000 7fedb1b5c640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed8c06b4d0 0x7fed8c06d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fedac148c60 tx=0x7fed94008040 comp rx=0 tx=0).stop 2024-04-25T10:01:07.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.090+0000 7fedb1b5c640 1 -- 172.21.15.139:0/187430391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedac0b4740 msgr2=0x7fedac14ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.090+0000 7fedb1b5c640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedac0b4740 0x7fedac14ec90 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00b2c0 tx=0x7fed9c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.090+0000 7fedb1b5c640 1 -- 172.21.15.139:0/187430391 shutdown_connections 2024-04-25T10:01:07.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.090+0000 7fedb1b5c640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed8c06b4d0 0x7fed8c06d990 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.090+0000 7fedb1b5c640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedac150ed0 0x7fedac14be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.090+0000 7fedb1b5c640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedac0cdd20 0x7fedac15c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.090+0000 7fedb1b5c640 1 --2- 172.21.15.139:0/187430391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedac0b4740 0x7fedac14ec90 secure :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00b2c0 tx=0x7fed9c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.090+0000 7fedb1b5c640 1 -- 172.21.15.139:0/187430391 >> 172.21.15.139:0/187430391 conn(0x7fedac067000 msgr2=0x7fedac1480e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:07.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.091+0000 7fedb1b5c640 1 -- 172.21.15.139:0/187430391 shutdown_connections 2024-04-25T10:01:07.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.091+0000 7fedb1b5c640 1 -- 172.21.15.139:0/187430391 wait complete. 2024-04-25T10:01:07.108 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:01:07.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.208+0000 7ff149ecb640 1 Processor -- start 2024-04-25T10:01:07.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.208+0000 7ff149ecb640 1 -- start start 2024-04-25T10:01:07.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.209+0000 7ff149ecb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff1440afc80 0x7ff1440b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.209+0000 7ff149ecb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff14406b700 0x7ff14406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.209+0000 7ff149ecb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1440c6cc0 con 0x7ff1440af8a0 2024-04-25T10:01:07.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.209+0000 7ff149ecb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1440c6fa0 con 0x7ff1440afc80 2024-04-25T10:01:07.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.209+0000 7ff149ecb640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1440c7280 con 0x7ff14406b700 2024-04-25T10:01:07.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.209+0000 7ff142ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff1440afc80 0x7ff1440b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.209+0000 7ff1437fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff1440af8a0 0x7ff1440cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:47074/0 (socket says 172.21.15.139:47074) 2024-04-25T10:01:07.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.209+0000 7ff142ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff1440afc80 0x7ff1440b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43130/0 (socket says 172.21.15.139:43130) 2024-04-25T10:01:07.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.209+0000 7ff1437fe640 1 -- 172.21.15.139:0/3398678688 learned_addr learned my addr 172.21.15.139:0/3398678688 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:07.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.210+0000 7ff142ffd640 1 -- 172.21.15.139:0/3398678688 >> v2:172.21.15.150:3300/0 conn(0x7ff14406b700 msgr2=0x7ff14406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:07.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.210+0000 7ff142ffd640 1 --2- 172.21.15.139:0/3398678688 >> v2:172.21.15.150:3300/0 conn(0x7ff14406b700 0x7ff14406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.210+0000 7ff142ffd640 1 -- 172.21.15.139:0/3398678688 >> v1:172.21.15.139:6789/0 conn(0x7ff1440af8a0 legacy=0x7ff1440cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.210+0000 7ff142ffd640 1 -- 172.21.15.139:0/3398678688 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1440cd090 con 0x7ff1440afc80 2024-04-25T10:01:07.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.210+0000 7ff1427fc640 1 -- 172.21.15.139:0/3398678688 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799561991 0 0) 0x7ff1440c6cc0 con 0x7ff1440af8a0 2024-04-25T10:01:07.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.210+0000 7ff142ffd640 1 --2- 172.21.15.139:0/3398678688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1440afc80 0x7ff1440b2d50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff14406f060 tx=0x7ff13802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110f8d91e3f0f55b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:07.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.210+0000 7ff1427fc640 1 -- 172.21.15.139:0/3398678688 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138037070 con 0x7ff1440afc80 2024-04-25T10:01:07.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.210+0000 7ff1427fc640 1 -- 172.21.15.139:0/3398678688 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff138032440 con 0x7ff1440afc80 2024-04-25T10:01:07.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.210+0000 7ff1427fc640 1 -- 172.21.15.139:0/3398678688 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13802ae60 con 0x7ff1440afc80 2024-04-25T10:01:07.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.211+0000 7ff149ecb640 1 -- 172.21.15.139:0/3398678688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1440afc80 msgr2=0x7ff1440b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.211+0000 7ff149ecb640 1 --2- 172.21.15.139:0/3398678688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1440afc80 0x7ff1440b2d50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff14406f060 tx=0x7ff13802a110 comp rx=0 tx=0).stop 2024-04-25T10:01:07.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.211+0000 7ff149ecb640 1 -- 172.21.15.139:0/3398678688 shutdown_connections 2024-04-25T10:01:07.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.211+0000 7ff149ecb640 1 --2- 172.21.15.139:0/3398678688 >> v2:172.21.15.150:3300/0 conn(0x7ff14406b700 0x7ff14406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.211+0000 7ff149ecb640 1 --2- 172.21.15.139:0/3398678688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1440afc80 0x7ff1440b2d50 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.211+0000 7ff149ecb640 1 -- 172.21.15.139:0/3398678688 >> 172.21.15.139:0/3398678688 conn(0x7ff1440b2230 msgr2=0x7ff1440b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:07.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.211+0000 7ff149ecb640 1 -- 172.21.15.139:0/3398678688 shutdown_connections 2024-04-25T10:01:07.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.211+0000 7ff149ecb640 1 -- 172.21.15.139:0/3398678688 wait complete. 2024-04-25T10:01:07.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.212+0000 7ff149ecb640 1 Processor -- start 2024-04-25T10:01:07.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.212+0000 7ff149ecb640 1 -- start start 2024-04-25T10:01:07.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff149ecb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff14406b700 0x7ff144147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff149ecb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1440af8a0 0x7ff144155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff149ecb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1440afc80 0x7ff144155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff149ecb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14406b220 con 0x7ff1440af8a0 2024-04-25T10:01:07.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff149ecb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14406b0a0 con 0x7ff14406b700 2024-04-25T10:01:07.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff149ecb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14406b3a0 con 0x7ff1440afc80 2024-04-25T10:01:07.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff142ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1440af8a0 0x7ff144155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff142ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1440af8a0 0x7ff144155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43142/0 (socket says 172.21.15.139:43142) 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff142ffd640 1 -- 172.21.15.139:0/1657802123 learned_addr learned my addr 172.21.15.139:0/1657802123 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff142ffd640 1 -- 172.21.15.139:0/1657802123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1440afc80 msgr2=0x7ff144155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff142ffd640 1 --2- 172.21.15.139:0/1657802123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1440afc80 0x7ff144155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff142ffd640 1 -- 172.21.15.139:0/1657802123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff14406b700 msgr2=0x7ff144147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff142ffd640 1 --2- 172.21.15.139:0/1657802123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff14406b700 0x7ff144147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.213+0000 7ff142ffd640 1 -- 172.21.15.139:0/1657802123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1441542c0 con 0x7ff1440af8a0 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.214+0000 7ff142ffd640 1 --2- 172.21.15.139:0/1657802123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1440af8a0 0x7ff144155100 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff138000c00 tx=0x7ff1380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.214+0000 7ff12bfff640 1 -- 172.21.15.139:0/1657802123 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138041070 con 0x7ff1440af8a0 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.214+0000 7ff12bfff640 1 -- 172.21.15.139:0/1657802123 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13803c950 con 0x7ff1440af8a0 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.214+0000 7ff149ecb640 1 -- 172.21.15.139:0/1657802123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1441545a0 con 0x7ff1440af8a0 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.214+0000 7ff12bfff640 1 -- 172.21.15.139:0/1657802123 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138037040 con 0x7ff1440af8a0 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.214+0000 7ff149ecb640 1 -- 172.21.15.139:0/1657802123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff144156a40 con 0x7ff1440af8a0 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.215+0000 7ff149ecb640 1 -- 172.21.15.139:0/1657802123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff144074520 con 0x7ff1440af8a0 2024-04-25T10:01:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.215+0000 7ff12bfff640 1 -- 172.21.15.139:0/1657802123 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff138044070 con 0x7ff1440af8a0 2024-04-25T10:01:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.216+0000 7ff12bfff640 1 --2- 172.21.15.139:0/1657802123 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff13006b500 0x7ff13006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.216+0000 7ff1437fe640 1 --2- 172.21.15.139:0/1657802123 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff13006b500 0x7ff13006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.216+0000 7ff12bfff640 1 -- 172.21.15.139:0/1657802123 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6037+0+0 (secure 0 0 0) 0x7ff1380ab080 con 0x7ff1440af8a0 2024-04-25T10:01:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.216+0000 7ff1437fe640 1 --2- 172.21.15.139:0/1657802123 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff13006b500 0x7ff13006d9c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff134007fd0 tx=0x7ff134007920 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.220+0000 7ff12bfff640 1 -- 172.21.15.139:0/1657802123 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff138079f60 con 0x7ff1440af8a0 2024-04-25T10:01:07.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:07.391+0000 7ff149ecb640 1 -- 172.21.15.139:0/1657802123 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff144162060 con 0x7ff1440af8a0 2024-04-25T10:01:08.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.090+0000 7ff12bfff640 1 -- 172.21.15.139:0/1657802123 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff138079860 con 0x7ff1440af8a0 2024-04-25T10:01:08.095 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' removed 2024-04-25T10:01:08.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.093+0000 7ff149ecb640 1 -- 172.21.15.139:0/1657802123 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff13006b500 msgr2=0x7ff13006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.093+0000 7ff149ecb640 1 --2- 172.21.15.139:0/1657802123 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff13006b500 0x7ff13006d9c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff134007fd0 tx=0x7ff134007920 comp rx=0 tx=0).stop 2024-04-25T10:01:08.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.093+0000 7ff149ecb640 1 -- 172.21.15.139:0/1657802123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1440af8a0 msgr2=0x7ff144155100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.093+0000 7ff149ecb640 1 --2- 172.21.15.139:0/1657802123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1440af8a0 0x7ff144155100 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff138000c00 tx=0x7ff1380046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.093+0000 7ff149ecb640 1 -- 172.21.15.139:0/1657802123 shutdown_connections 2024-04-25T10:01:08.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.093+0000 7ff149ecb640 1 --2- 172.21.15.139:0/1657802123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1440af8a0 0x7ff144155100 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.093+0000 7ff149ecb640 1 --2- 172.21.15.139:0/1657802123 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff13006b500 0x7ff13006d9c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.093+0000 7ff149ecb640 1 --2- 172.21.15.139:0/1657802123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1440afc80 0x7ff144155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.093+0000 7ff149ecb640 1 --2- 172.21.15.139:0/1657802123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff14406b700 0x7ff144147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.093+0000 7ff149ecb640 1 -- 172.21.15.139:0/1657802123 >> 172.21.15.139:0/1657802123 conn(0x7ff1440b2230 msgr2=0x7ff14415d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:08.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.093+0000 7ff149ecb640 1 -- 172.21.15.139:0/1657802123 shutdown_connections 2024-04-25T10:01:08.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.093+0000 7ff149ecb640 1 -- 172.21.15.139:0/1657802123 wait complete. 2024-04-25T10:01:08.115 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:01:08.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.213+0000 7fd61efa5640 1 Processor -- start 2024-04-25T10:01:08.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.213+0000 7fd61efa5640 1 -- start start 2024-04-25T10:01:08.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.214+0000 7fd61efa5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd61806ebf0 0x7fd61806f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.214+0000 7fd61efa5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd6180b2db0 0x7fd6180b3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.214+0000 7fd61efa5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6180cf980 con 0x7fd61815dc90 2024-04-25T10:01:08.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.214+0000 7fd61efa5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61806ab60 con 0x7fd6180b2db0 2024-04-25T10:01:08.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.214+0000 7fd61efa5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61806ae40 con 0x7fd61806ebf0 2024-04-25T10:01:08.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.214+0000 7fd61cd41640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd61815dc90 0x7fd6180ceea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:47090/0 (socket says 172.21.15.139:47090) 2024-04-25T10:01:08.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.214+0000 7fd61cd41640 1 -- 172.21.15.139:0/4105525407 learned_addr learned my addr 172.21.15.139:0/4105525407 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:08.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.214+0000 7fd617fff640 1 --2- 172.21.15.139:0/4105525407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd61806ebf0 0x7fd61806f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.214+0000 7fd61d542640 1 --2- 172.21.15.139:0/4105525407 >> v2:172.21.15.150:3300/0 conn(0x7fd6180b2db0 0x7fd6180b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.214+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021906594 0 0) 0x7fd6180cf980 con 0x7fd61815dc90 2024-04-25T10:01:08.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.214+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd600003520 con 0x7fd61815dc90 2024-04-25T10:01:08.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1655530575 0 0) 0x7fd600003520 con 0x7fd61815dc90 2024-04-25T10:01:08.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd61806ebf0 msgr2=0x7fd61806f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:08.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 --2- 172.21.15.139:0/4105525407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd61806ebf0 0x7fd61806f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 >> v2:172.21.15.150:3300/0 conn(0x7fd6180b2db0 msgr2=0x7fd6180b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:08.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 --2- 172.21.15.139:0/4105525407 >> v2:172.21.15.150:3300/0 conn(0x7fd6180b2db0 0x7fd6180b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61806b120 con 0x7fd61815dc90 2024-04-25T10:01:08.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fd608001670 con 0x7fd61815dc90 2024-04-25T10:01:08.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 >> v1:172.21.15.139:6789/0 conn(0x7fd61815dc90 legacy=0x7fd6180ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 --2- 172.21.15.139:0/4105525407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6000039b0 0x7fd600003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 --2- 172.21.15.139:0/4105525407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd600004800 0x7fd600004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 --2- 172.21.15.139:0/4105525407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd61806ebf0 0x7fd6000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61806ab60 con 0x7fd6000039b0 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61806ae40 con 0x7fd600004800 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60000b9c0 con 0x7fd61806ebf0 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd61cd41640 1 --2- 172.21.15.139:0/4105525407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6000039b0 0x7fd600003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd617fff640 1 --2- 172.21.15.139:0/4105525407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd600004800 0x7fd600004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd61d542640 1 --2- 172.21.15.139:0/4105525407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd61806ebf0 0x7fd6000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd61cd41640 1 -- 172.21.15.139:0/4105525407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd61806ebf0 msgr2=0x7fd6000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd61cd41640 1 --2- 172.21.15.139:0/4105525407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd61806ebf0 0x7fd6000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd61cd41640 1 -- 172.21.15.139:0/4105525407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd600004800 msgr2=0x7fd600004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd61cd41640 1 --2- 172.21.15.139:0/4105525407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd600004800 0x7fd600004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.215+0000 7fd61cd41640 1 -- 172.21.15.139:0/4105525407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd600003520 con 0x7fd6000039b0 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.216+0000 7fd61cd41640 1 --2- 172.21.15.139:0/4105525407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6000039b0 0x7fd600003dd0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fd60800c240 tx=0x7fd600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b53f2d5fb88639a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.216+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60802e070 con 0x7fd6000039b0 2024-04-25T10:01:08.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.216+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd608005af0 con 0x7fd6000039b0 2024-04-25T10:01:08.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.216+0000 7fd6177fe640 1 -- 172.21.15.139:0/4105525407 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608007d30 con 0x7fd6000039b0 2024-04-25T10:01:08.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.216+0000 7fd61efa5640 1 -- 172.21.15.139:0/4105525407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6000039b0 msgr2=0x7fd600003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.216+0000 7fd61efa5640 1 --2- 172.21.15.139:0/4105525407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6000039b0 0x7fd600003dd0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fd60800c240 tx=0x7fd600005830 comp rx=0 tx=0).stop 2024-04-25T10:01:08.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.216+0000 7fd61d542640 1 -- 172.21.15.139:0/4105525407 reap_dead start 2024-04-25T10:01:08.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.216+0000 7fd61efa5640 1 -- 172.21.15.139:0/4105525407 shutdown_connections 2024-04-25T10:01:08.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.216+0000 7fd61efa5640 1 -- 172.21.15.139:0/4105525407 >> 172.21.15.139:0/4105525407 conn(0x7fd6180cfd80 msgr2=0x7fd6180d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:08.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.217+0000 7fd61efa5640 1 -- 172.21.15.139:0/4105525407 shutdown_connections 2024-04-25T10:01:08.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.217+0000 7fd61efa5640 1 -- 172.21.15.139:0/4105525407 wait complete. 2024-04-25T10:01:08.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.218+0000 7fd61efa5640 1 Processor -- start 2024-04-25T10:01:08.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.218+0000 7fd61efa5640 1 -- start start 2024-04-25T10:01:08.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.218+0000 7fd61efa5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6000039b0 0x7fd618147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.218+0000 7fd61efa5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61815ffe0 0x7fd618160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.218+0000 7fd61efa5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd618159b70 0x7fd618159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.218+0000 7fd61efa5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61806b920 con 0x7fd61815ffe0 2024-04-25T10:01:08.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.218+0000 7fd61efa5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618072620 con 0x7fd6000039b0 2024-04-25T10:01:08.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.218+0000 7fd61efa5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180cf5b0 con 0x7fd618159b70 2024-04-25T10:01:08.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd617fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61815ffe0 0x7fd618160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd617fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61815ffe0 0x7fd618160440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43150/0 (socket says 172.21.15.139:43150) 2024-04-25T10:01:08.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd617fff640 1 -- 172.21.15.139:0/3488525738 learned_addr learned my addr 172.21.15.139:0/3488525738 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:08.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd61cd41640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6000039b0 0x7fd618147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd617fff640 1 -- 172.21.15.139:0/3488525738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd618159b70 msgr2=0x7fd618159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd61d542640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd618159b70 0x7fd618159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd617fff640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd618159b70 0x7fd618159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd617fff640 1 -- 172.21.15.139:0/3488525738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6000039b0 msgr2=0x7fd618147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd617fff640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6000039b0 0x7fd618147190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd617fff640 1 -- 172.21.15.139:0/3488525738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6181479f0 con 0x7fd61815ffe0 2024-04-25T10:01:08.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd617fff640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61815ffe0 0x7fd618160440 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fd60c011940 tx=0x7fd60c011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd6157fa640 1 -- 172.21.15.139:0/3488525738 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00fa90 con 0x7fd61815ffe0 2024-04-25T10:01:08.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd6157fa640 1 -- 172.21.15.139:0/3488525738 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c00fc30 con 0x7fd61815ffe0 2024-04-25T10:01:08.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd6157fa640 1 -- 172.21.15.139:0/3488525738 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c014440 con 0x7fd61815ffe0 2024-04-25T10:01:08.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd61efa5640 1 -- 172.21.15.139:0/3488525738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd618147c60 con 0x7fd61815ffe0 2024-04-25T10:01:08.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.219+0000 7fd61efa5640 1 -- 172.21.15.139:0/3488525738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd618148220 con 0x7fd61815ffe0 2024-04-25T10:01:08.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.220+0000 7fd61efa5640 1 -- 172.21.15.139:0/3488525738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd618074520 con 0x7fd61815ffe0 2024-04-25T10:01:08.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.221+0000 7fd6157fa640 1 -- 172.21.15.139:0/3488525738 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd60c012040 con 0x7fd61815ffe0 2024-04-25T10:01:08.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.221+0000 7fd6157fa640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd60406b4d0 0x7fd60406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.221+0000 7fd6157fa640 1 -- 172.21.15.139:0/3488525738 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fd60c08a760 con 0x7fd61815ffe0 2024-04-25T10:01:08.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.221+0000 7fd61cd41640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd60406b4d0 0x7fd60406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.222+0000 7fd61cd41640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd60406b4d0 0x7fd60406d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd608033000 tx=0x7fd608009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.225+0000 7fd6157fa640 1 -- 172.21.15.139:0/3488525738 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd60c019030 con 0x7fd61815ffe0 2024-04-25T10:01:08.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.398+0000 7fd61efa5640 1 -- 172.21.15.139:0/3488525738 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd6180ced30 con 0x7fd61815ffe0 2024-04-25T10:01:08.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.401+0000 7fd6157fa640 1 -- 172.21.15.139:0/3488525738 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd60c059610 con 0x7fd61815ffe0 2024-04-25T10:01:08.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.403+0000 7fd61efa5640 1 -- 172.21.15.139:0/3488525738 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd60406b4d0 msgr2=0x7fd60406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.403+0000 7fd61efa5640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd60406b4d0 0x7fd60406d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd608033000 tx=0x7fd608009030 comp rx=0 tx=0).stop 2024-04-25T10:01:08.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.403+0000 7fd61efa5640 1 -- 172.21.15.139:0/3488525738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61815ffe0 msgr2=0x7fd618160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.403+0000 7fd61efa5640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61815ffe0 0x7fd618160440 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fd60c011940 tx=0x7fd60c011e00 comp rx=0 tx=0).stop 2024-04-25T10:01:08.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.403+0000 7fd61efa5640 1 -- 172.21.15.139:0/3488525738 shutdown_connections 2024-04-25T10:01:08.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.403+0000 7fd61efa5640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd61815ffe0 0x7fd618160440 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.403+0000 7fd61efa5640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd60406b4d0 0x7fd60406d990 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.403+0000 7fd61efa5640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd618159b70 0x7fd618159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.403+0000 7fd61efa5640 1 --2- 172.21.15.139:0/3488525738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6000039b0 0x7fd618147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.403+0000 7fd61efa5640 1 -- 172.21.15.139:0/3488525738 >> 172.21.15.139:0/3488525738 conn(0x7fd6180cfd80 msgr2=0x7fd6180b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:08.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.403+0000 7fd61efa5640 1 -- 172.21.15.139:0/3488525738 shutdown_connections 2024-04-25T10:01:08.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.403+0000 7fd61efa5640 1 -- 172.21.15.139:0/3488525738 wait complete. 2024-04-25T10:01:08.418 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T10:01:08.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.514+0000 7fc446a93640 1 Processor -- start 2024-04-25T10:01:08.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.514+0000 7fc446a93640 1 -- start start 2024-04-25T10:01:08.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.515+0000 7fc446a93640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc44015a860 0x7fc4400cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.515+0000 7fc446a93640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4400af6d0 con 0x7fc4400cd1f0 2024-04-25T10:01:08.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.515+0000 7fc446a93640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4400af9b0 con 0x7fc44006ea20 2024-04-25T10:01:08.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.515+0000 7fc446a93640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4400b02e0 con 0x7fc44015a860 2024-04-25T10:01:08.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.515+0000 7fc44482f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc44015a860 0x7fc4400cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.515+0000 7fc43ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc4400cd1f0 0x7fc44006e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:47096/0 (socket says 172.21.15.139:47096) 2024-04-25T10:01:08.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.515+0000 7fc43ffff640 1 -- 172.21.15.139:0/1899831405 learned_addr learned my addr 172.21.15.139:0/1899831405 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:08.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.515+0000 7fc44482f640 1 -- 172.21.15.139:0/1899831405 >> v1:172.21.15.150:6789/0 conn(0x7fc44006ea20 legacy=0x7fc4400b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:08.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.515+0000 7fc44482f640 1 -- 172.21.15.139:0/1899831405 >> v1:172.21.15.139:6789/0 conn(0x7fc4400cd1f0 legacy=0x7fc44006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.515+0000 7fc43f7fe640 1 -- 172.21.15.139:0/1899831405 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797369947 0 0) 0x7fc4400af6d0 con 0x7fc4400cd1f0 2024-04-25T10:01:08.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.515+0000 7fc44482f640 1 -- 172.21.15.139:0/1899831405 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4400afc90 con 0x7fc44015a860 2024-04-25T10:01:08.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.516+0000 7fc44482f640 1 --2- 172.21.15.139:0/1899831405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44015a860 0x7fc4400cccb0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fc4340097a0 tx=0x7fc43402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a36344d5086bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.516+0000 7fc43f7fe640 1 -- 172.21.15.139:0/1899831405 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434037070 con 0x7fc44015a860 2024-04-25T10:01:08.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.516+0000 7fc43f7fe640 1 -- 172.21.15.139:0/1899831405 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc434032440 con 0x7fc44015a860 2024-04-25T10:01:08.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.516+0000 7fc43f7fe640 1 -- 172.21.15.139:0/1899831405 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434032810 con 0x7fc44015a860 2024-04-25T10:01:08.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.516+0000 7fc446a93640 1 -- 172.21.15.139:0/1899831405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44015a860 msgr2=0x7fc4400cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.516+0000 7fc446a93640 1 --2- 172.21.15.139:0/1899831405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44015a860 0x7fc4400cccb0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fc4340097a0 tx=0x7fc43402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.516+0000 7fc446a93640 1 -- 172.21.15.139:0/1899831405 shutdown_connections 2024-04-25T10:01:08.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.516+0000 7fc446a93640 1 --2- 172.21.15.139:0/1899831405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44015a860 0x7fc4400cccb0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.517+0000 7fc446a93640 1 -- 172.21.15.139:0/1899831405 >> 172.21.15.139:0/1899831405 conn(0x7fc440066f20 msgr2=0x7fc44006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:08.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.517+0000 7fc446a93640 1 -- 172.21.15.139:0/1899831405 shutdown_connections 2024-04-25T10:01:08.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.517+0000 7fc446a93640 1 -- 172.21.15.139:0/1899831405 wait complete. 2024-04-25T10:01:08.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.518+0000 7fc446a93640 1 Processor -- start 2024-04-25T10:01:08.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.518+0000 7fc446a93640 1 -- start start 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.518+0000 7fc446a93640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc44006ea20 0x7fc4401e4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.518+0000 7fc446a93640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4400cd1f0 0x7fc440142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.518+0000 7fc446a93640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44015a860 0x7fc4401e5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.518+0000 7fc446a93640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4400af420 con 0x7fc44015a860 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.518+0000 7fc446a93640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4400af2a0 con 0x7fc4400cd1f0 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.518+0000 7fc446a93640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44015ac40 con 0x7fc44006ea20 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc445030640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44015a860 0x7fc4401e5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc445030640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44015a860 0x7fc4401e5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43162/0 (socket says 172.21.15.139:43162) 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc445030640 1 -- 172.21.15.139:0/671718175 learned_addr learned my addr 172.21.15.139:0/671718175 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc43ffff640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4400cd1f0 0x7fc440142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc44482f640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc44006ea20 0x7fc4401e4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc445030640 1 -- 172.21.15.139:0/671718175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc44006ea20 msgr2=0x7fc4401e4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc445030640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc44006ea20 0x7fc4401e4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc445030640 1 -- 172.21.15.139:0/671718175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4400cd1f0 msgr2=0x7fc440142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc445030640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4400cd1f0 0x7fc440142020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc445030640 1 -- 172.21.15.139:0/671718175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4401e48b0 con 0x7fc44015a860 2024-04-25T10:01:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc445030640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44015a860 0x7fc4401e5950 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fc43800c730 tx=0x7fc43800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc43d7fa640 1 -- 172.21.15.139:0/671718175 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438016020 con 0x7fc44015a860 2024-04-25T10:01:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc43d7fa640 1 -- 172.21.15.139:0/671718175 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43800aa50 con 0x7fc44015a860 2024-04-25T10:01:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc43d7fa640 1 -- 172.21.15.139:0/671718175 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43800ce40 con 0x7fc44015a860 2024-04-25T10:01:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc446a93640 1 -- 172.21.15.139:0/671718175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4401e4b90 con 0x7fc44015a860 2024-04-25T10:01:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.519+0000 7fc446a93640 1 -- 172.21.15.139:0/671718175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4401e5150 con 0x7fc44015a860 2024-04-25T10:01:08.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.520+0000 7fc446a93640 1 -- 172.21.15.139:0/671718175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc440074520 con 0x7fc44015a860 2024-04-25T10:01:08.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.521+0000 7fc43d7fa640 1 -- 172.21.15.139:0/671718175 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc438019020 con 0x7fc44015a860 2024-04-25T10:01:08.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.521+0000 7fc43d7fa640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc42406b4d0 0x7fc42406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.521+0000 7fc43d7fa640 1 -- 172.21.15.139:0/671718175 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc43808a120 con 0x7fc44015a860 2024-04-25T10:01:08.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.521+0000 7fc44482f640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc42406b4d0 0x7fc42406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.522+0000 7fc44482f640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc42406b4d0 0x7fc42406d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc434034000 tx=0x7fc43402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.525+0000 7fc43d7fa640 1 -- 172.21.15.139:0/671718175 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc438020030 con 0x7fc44015a860 2024-04-25T10:01:08.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:08.759+0000 7fc446a93640 1 -- 172.21.15.139:0/671718175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7fc4400b1300 con 0x7fc44015a860 2024-04-25T10:01:09.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.103+0000 7fc43d7fa640 1 -- 172.21.15.139:0/671718175 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v371) v1 ==== 167+0+0 (secure 0 0 0) 0x7fc4380212b0 con 0x7fc44015a860 2024-04-25T10:01:09.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.105+0000 7fc446a93640 1 -- 172.21.15.139:0/671718175 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc42406b4d0 msgr2=0x7fc42406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.105+0000 7fc446a93640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc42406b4d0 0x7fc42406d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc434034000 tx=0x7fc43402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:09.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.105+0000 7fc446a93640 1 -- 172.21.15.139:0/671718175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44015a860 msgr2=0x7fc4401e5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.105+0000 7fc446a93640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44015a860 0x7fc4401e5950 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fc43800c730 tx=0x7fc43800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.105+0000 7fc446a93640 1 -- 172.21.15.139:0/671718175 shutdown_connections 2024-04-25T10:01:09.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.105+0000 7fc446a93640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc44015a860 0x7fc4401e5950 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.105+0000 7fc446a93640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc42406b4d0 0x7fc42406d990 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.105+0000 7fc446a93640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4400cd1f0 0x7fc440142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.105+0000 7fc446a93640 1 --2- 172.21.15.139:0/671718175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc44006ea20 0x7fc4401e4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.105+0000 7fc446a93640 1 -- 172.21.15.139:0/671718175 >> 172.21.15.139:0/671718175 conn(0x7fc440066f20 msgr2=0x7fc4400b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.105+0000 7fc446a93640 1 -- 172.21.15.139:0/671718175 shutdown_connections 2024-04-25T10:01:09.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.105+0000 7fc446a93640 1 -- 172.21.15.139:0/671718175 wait complete. 2024-04-25T10:01:09.121 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:01:09.121 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:01:09.122 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:09.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.214+0000 7ff9ea4fc640 1 Processor -- start 2024-04-25T10:01:09.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.214+0000 7ff9ea4fc640 1 -- start start 2024-04-25T10:01:09.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.215+0000 7ff9ea4fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e40cf900 0x7ff9e40d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.215+0000 7ff9ea4fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff9e4099a50 0x7ff9e4099ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.215+0000 7ff9ea4fc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e40cfe70 con 0x7ff9e415efb0 2024-04-25T10:01:09.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.215+0000 7ff9ea4fc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e40d0a80 con 0x7ff9e4099a50 2024-04-25T10:01:09.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.215+0000 7ff9ea4fc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e40d0150 con 0x7ff9e40cf900 2024-04-25T10:01:09.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.215+0000 7ff9e8a99640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff9e4099a50 0x7ff9e4099ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.215+0000 7ff9e3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff9e415efb0 0x7ff9e406c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:47098/0 (socket says 172.21.15.139:47098) 2024-04-25T10:01:09.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.215+0000 7ff9e8a99640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff9e4099a50 0x7ff9e4099ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43172/0 (socket says 172.21.15.139:43172) 2024-04-25T10:01:09.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.215+0000 7ff9e3fff640 1 -- 172.21.15.139:0/2759837272 learned_addr learned my addr 172.21.15.139:0/2759837272 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:09.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.216+0000 7ff9e37fe640 1 --2- 172.21.15.139:0/2759837272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e40cf900 0x7ff9e40d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.216+0000 7ff9e8a99640 1 -- 172.21.15.139:0/2759837272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e40cf900 msgr2=0x7ff9e40d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.216+0000 7ff9e8a99640 1 --2- 172.21.15.139:0/2759837272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e40cf900 0x7ff9e40d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.216+0000 7ff9e8a99640 1 -- 172.21.15.139:0/2759837272 >> v1:172.21.15.139:6789/0 conn(0x7ff9e415efb0 legacy=0x7ff9e406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.216+0000 7ff9e8a99640 1 -- 172.21.15.139:0/2759837272 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e40d0430 con 0x7ff9e4099a50 2024-04-25T10:01:09.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.216+0000 7ff9e2ffd640 1 -- 172.21.15.139:0/2759837272 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916450863 0 0) 0x7ff9e40cfe70 con 0x7ff9e415efb0 2024-04-25T10:01:09.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.216+0000 7ff9e8a99640 1 --2- 172.21.15.139:0/2759837272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e4099a50 0x7ff9e4099ee0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009d90 tx=0x7ff9dc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b773156c6b3d803a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.216+0000 7ff9e2ffd640 1 -- 172.21.15.139:0/2759837272 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc031600 con 0x7ff9e4099a50 2024-04-25T10:01:09.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.217+0000 7ff9e2ffd640 1 -- 172.21.15.139:0/2759837272 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc0317a0 con 0x7ff9e4099a50 2024-04-25T10:01:09.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.217+0000 7ff9e2ffd640 1 -- 172.21.15.139:0/2759837272 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc031b70 con 0x7ff9e4099a50 2024-04-25T10:01:09.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.217+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/2759837272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e4099a50 msgr2=0x7ff9e4099ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.217+0000 7ff9ea4fc640 1 --2- 172.21.15.139:0/2759837272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e4099a50 0x7ff9e4099ee0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009d90 tx=0x7ff9dc02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.217+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/2759837272 shutdown_connections 2024-04-25T10:01:09.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.217+0000 7ff9ea4fc640 1 --2- 172.21.15.139:0/2759837272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e4099a50 0x7ff9e4099ee0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.217+0000 7ff9ea4fc640 1 --2- 172.21.15.139:0/2759837272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e40cf900 0x7ff9e40d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.217+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/2759837272 >> 172.21.15.139:0/2759837272 conn(0x7ff9e40d1790 msgr2=0x7ff9e40d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.217+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/2759837272 shutdown_connections 2024-04-25T10:01:09.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.217+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/2759837272 wait complete. 2024-04-25T10:01:09.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.218+0000 7ff9ea4fc640 1 Processor -- start 2024-04-25T10:01:09.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.218+0000 7ff9ea4fc640 1 -- start start 2024-04-25T10:01:09.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.219+0000 7ff9ea4fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e4099a50 0x7ff9e414dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.219+0000 7ff9ea4fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e40cf900 0x7ff9e40c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.219+0000 7ff9ea4fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e415efb0 0x7ff9e406bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.219+0000 7ff9ea4fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e40cbde0 con 0x7ff9e415efb0 2024-04-25T10:01:09.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.219+0000 7ff9ea4fc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e406cf50 con 0x7ff9e40cf900 2024-04-25T10:01:09.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.219+0000 7ff9ea4fc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e40cbf60 con 0x7ff9e4099a50 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.219+0000 7ff9e8a99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e415efb0 0x7ff9e406bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.219+0000 7ff9e8a99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e415efb0 0x7ff9e406bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43184/0 (socket says 172.21.15.139:43184) 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.219+0000 7ff9e8a99640 1 -- 172.21.15.139:0/1153084999 learned_addr learned my addr 172.21.15.139:0/1153084999 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.219+0000 7ff9e3fff640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e4099a50 0x7ff9e414dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.219+0000 7ff9e37fe640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e40cf900 0x7ff9e40c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.220+0000 7ff9e8a99640 1 -- 172.21.15.139:0/1153084999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e4099a50 msgr2=0x7ff9e414dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.220+0000 7ff9e8a99640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e4099a50 0x7ff9e414dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.220+0000 7ff9e8a99640 1 -- 172.21.15.139:0/1153084999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e40cf900 msgr2=0x7ff9e40c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.220+0000 7ff9e8a99640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e40cf900 0x7ff9e40c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.220+0000 7ff9e8a99640 1 -- 172.21.15.139:0/1153084999 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e40c81b0 con 0x7ff9e415efb0 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.220+0000 7ff9e37fe640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e40cf900 0x7ff9e40c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.220+0000 7ff9e8a99640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e415efb0 0x7ff9e406bc70 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009eb0 tx=0x7ff9dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.220+0000 7ff9e0ff9640 1 -- 172.21.15.139:0/1153084999 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc040070 con 0x7ff9e415efb0 2024-04-25T10:01:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.220+0000 7ff9e0ff9640 1 -- 172.21.15.139:0/1153084999 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc030b60 con 0x7ff9e415efb0 2024-04-25T10:01:09.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.220+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/1153084999 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e40c7110 con 0x7ff9e415efb0 2024-04-25T10:01:09.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.220+0000 7ff9e0ff9640 1 -- 172.21.15.139:0/1153084999 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc009450 con 0x7ff9e415efb0 2024-04-25T10:01:09.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.220+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/1153084999 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e40c76d0 con 0x7ff9e415efb0 2024-04-25T10:01:09.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.221+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/1153084999 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e40b5760 con 0x7ff9e415efb0 2024-04-25T10:01:09.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.221+0000 7ff9e0ff9640 1 -- 172.21.15.139:0/1153084999 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9dc039040 con 0x7ff9e415efb0 2024-04-25T10:01:09.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.222+0000 7ff9e0ff9640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff9d006b500 0x7ff9d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.222+0000 7ff9e0ff9640 1 -- 172.21.15.139:0/1153084999 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ff9dc0ab8f0 con 0x7ff9e415efb0 2024-04-25T10:01:09.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.222+0000 7ff9e3fff640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff9d006b500 0x7ff9d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.222+0000 7ff9e3fff640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff9d006b500 0x7ff9d006d9c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4000f60 tx=0x7ff9d4007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.226+0000 7ff9e0ff9640 1 -- 172.21.15.139:0/1153084999 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9dc07a8c0 con 0x7ff9e415efb0 2024-04-25T10:01:09.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.397+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/1153084999 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9e4074520 con 0x7ff9e415efb0 2024-04-25T10:01:09.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.398+0000 7ff9e0ff9640 1 -- 172.21.15.139:0/1153084999 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+13608 (secure 0 0 0) 0x7ff9dc07a1c0 con 0x7ff9e415efb0 2024-04-25T10:01:09.402 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:09.402 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":96,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:01:08.087935+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6836/1457343244":"2024-04-26T10:01:05.078161+0000","172.21.15.139:6834/1457343244":"2024-04-26T10:01:05.078161+0000","172.21.15.139:6836/1239243954":"2024-04-26T10:00:22.118970+0000","172.21.15.139:6834/1239243954":"2024-04-26T10:00:22.118970+0000","172.21.15.150:6835/2266713846":"2024-04-26T09:59:10.897062+0000","172.21.15.150:6833/2266713846":"2024-04-26T09:59:10.897062+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:09.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.400+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/1153084999 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff9d006b500 msgr2=0x7ff9d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.400+0000 7ff9ea4fc640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff9d006b500 0x7ff9d006d9c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4000f60 tx=0x7ff9d4007c00 comp rx=0 tx=0).stop 2024-04-25T10:01:09.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.400+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/1153084999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e415efb0 msgr2=0x7ff9e406bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.400+0000 7ff9ea4fc640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e415efb0 0x7ff9e406bc70 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009eb0 tx=0x7ff9dc004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:09.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.400+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/1153084999 shutdown_connections 2024-04-25T10:01:09.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.400+0000 7ff9ea4fc640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9e415efb0 0x7ff9e406bc70 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.400+0000 7ff9ea4fc640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff9d006b500 0x7ff9d006d9c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.400+0000 7ff9ea4fc640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e40cf900 0x7ff9e40c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.400+0000 7ff9ea4fc640 1 --2- 172.21.15.139:0/1153084999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e4099a50 0x7ff9e414dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.400+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/1153084999 >> 172.21.15.139:0/1153084999 conn(0x7ff9e40d1790 msgr2=0x7ff9e4156c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.400+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/1153084999 shutdown_connections 2024-04-25T10:01:09.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.400+0000 7ff9ea4fc640 1 -- 172.21.15.139:0/1153084999 wait complete. 2024-04-25T10:01:09.416 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:09.416 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:09.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.511+0000 7f06abb5c640 1 Processor -- start 2024-04-25T10:01:09.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.511+0000 7f06abb5c640 1 -- start start 2024-04-25T10:01:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06abb5c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f06a40d0a80 0x7f06a41618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06abb5c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f06a409ad40 0x7f06a4161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06abb5c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a4099ca0 con 0x7f06a40cd610 2024-04-25T10:01:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06abb5c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a4099a50 con 0x7f06a40d0a80 2024-04-25T10:01:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06abb5c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a4099f80 con 0x7f06a409ad40 2024-04-25T10:01:09.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06a90f7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f06a40d0a80 0x7f06a41618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06a90f7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f06a40d0a80 0x7f06a41618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43200/0 (socket says 172.21.15.139:43200) 2024-04-25T10:01:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06a90f7640 1 -- 172.21.15.139:0/2885904626 learned_addr learned my addr 172.21.15.139:0/2885904626 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06aa0f9640 1 --2- 172.21.15.139:0/2885904626 >> v2:172.21.15.150:3300/0 conn(0x7f06a409ad40 0x7f06a4161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06a90f7640 1 -- 172.21.15.139:0/2885904626 >> v2:172.21.15.150:3300/0 conn(0x7f06a409ad40 msgr2=0x7f06a4161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06a90f7640 1 --2- 172.21.15.139:0/2885904626 >> v2:172.21.15.150:3300/0 conn(0x7f06a409ad40 0x7f06a4161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06a90f7640 1 -- 172.21.15.139:0/2885904626 >> v1:172.21.15.150:6789/0 conn(0x7f06a40cd610 legacy=0x7f06a40d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.512+0000 7f06a90f7640 1 -- 172.21.15.139:0/2885904626 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a409a260 con 0x7f06a40d0a80 2024-04-25T10:01:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.513+0000 7f06a90f7640 1 --2- 172.21.15.139:0/2885904626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a40d0a80 0x7f06a41618c0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f06a40c51e0 tx=0x7f069402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d54c8ba5d3be256 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.513+0000 7f06a88f6640 1 -- 172.21.15.139:0/2885904626 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694037070 con 0x7f06a40d0a80 2024-04-25T10:01:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.513+0000 7f06a88f6640 1 -- 172.21.15.139:0/2885904626 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694032440 con 0x7f06a40d0a80 2024-04-25T10:01:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.513+0000 7f06a88f6640 1 -- 172.21.15.139:0/2885904626 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694032810 con 0x7f06a40d0a80 2024-04-25T10:01:09.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.514+0000 7f06abb5c640 1 -- 172.21.15.139:0/2885904626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a40d0a80 msgr2=0x7f06a41618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.514+0000 7f06abb5c640 1 --2- 172.21.15.139:0/2885904626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a40d0a80 0x7f06a41618c0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f06a40c51e0 tx=0x7f069402a2c0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.514+0000 7f06abb5c640 1 -- 172.21.15.139:0/2885904626 shutdown_connections 2024-04-25T10:01:09.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.514+0000 7f06abb5c640 1 --2- 172.21.15.139:0/2885904626 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a40d0a80 0x7f06a41618c0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.514+0000 7f06abb5c640 1 --2- 172.21.15.139:0/2885904626 >> v2:172.21.15.150:3300/0 conn(0x7f06a409ad40 0x7f06a4161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.514+0000 7f06abb5c640 1 -- 172.21.15.139:0/2885904626 >> 172.21.15.139:0/2885904626 conn(0x7f06a40b31f0 msgr2=0x7f06a40b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.514+0000 7f06abb5c640 1 -- 172.21.15.139:0/2885904626 shutdown_connections 2024-04-25T10:01:09.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.514+0000 7f06abb5c640 1 -- 172.21.15.139:0/2885904626 wait complete. 2024-04-25T10:01:09.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.515+0000 7f06abb5c640 1 Processor -- start 2024-04-25T10:01:09.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.515+0000 7f06abb5c640 1 -- start start 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06abb5c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a409ad40 0x7f06a41485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06abb5c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a40cd610 0x7f06a40b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06abb5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a40d0a80 0x7f06a414a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06abb5c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40cd9f0 con 0x7f06a40d0a80 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06abb5c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a409a820 con 0x7f06a409ad40 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06abb5c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40cdb70 con 0x7f06a40cd610 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06aa0f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a40d0a80 0x7f06a414a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06aa0f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a40d0a80 0x7f06a414a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43208/0 (socket says 172.21.15.139:43208) 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06aa0f9640 1 -- 172.21.15.139:0/518952782 learned_addr learned my addr 172.21.15.139:0/518952782 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06a98f8640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a409ad40 0x7f06a41485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06a90f7640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a40cd610 0x7f06a40b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06aa0f9640 1 -- 172.21.15.139:0/518952782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a40cd610 msgr2=0x7f06a40b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06aa0f9640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a40cd610 0x7f06a40b6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06aa0f9640 1 -- 172.21.15.139:0/518952782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a409ad40 msgr2=0x7f06a41485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06aa0f9640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a409ad40 0x7f06a41485e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06aa0f9640 1 -- 172.21.15.139:0/518952782 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a414b030 con 0x7f06a40d0a80 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06aa0f9640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a40d0a80 0x7f06a414a670 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f069c007e70 tx=0x7f069c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f069a7fc640 1 -- 172.21.15.139:0/518952782 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c014620 con 0x7f06a40d0a80 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f069a7fc640 1 -- 172.21.15.139:0/518952782 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c019070 con 0x7f06a40d0a80 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06abb5c640 1 -- 172.21.15.139:0/518952782 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a414b310 con 0x7f06a40d0a80 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f069a7fc640 1 -- 172.21.15.139:0/518952782 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c014a20 con 0x7f06a40d0a80 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.516+0000 7f06abb5c640 1 -- 172.21.15.139:0/518952782 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a415be60 con 0x7f06a40d0a80 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.517+0000 7f06abb5c640 1 -- 172.21.15.139:0/518952782 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a415d9d0 con 0x7f06a40d0a80 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.518+0000 7f069a7fc640 1 -- 172.21.15.139:0/518952782 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f069c014bc0 con 0x7f06a40d0a80 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.518+0000 7f069a7fc640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f068806b4d0 0x7f068806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.518+0000 7f069a7fc640 1 -- 172.21.15.139:0/518952782 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f069c08a900 con 0x7f06a40d0a80 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.518+0000 7f06a98f8640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f068806b4d0 0x7f068806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.519+0000 7f06a98f8640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f068806b4d0 0x7f068806d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f06a0004f60 tx=0x7f06a0007e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.522+0000 7f069a7fc640 1 -- 172.21.15.139:0/518952782 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f069c0591f0 con 0x7f06a40d0a80 2024-04-25T10:01:09.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.699+0000 7f06abb5c640 1 -- 172.21.15.139:0/518952782 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06a4074520 con 0x7f06a40d0a80 2024-04-25T10:01:09.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.699+0000 7f069a7fc640 1 -- 172.21.15.139:0/518952782 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f069c05f070 con 0x7f06a40d0a80 2024-04-25T10:01:09.704 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:01:09.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.701+0000 7f06abb5c640 1 -- 172.21.15.139:0/518952782 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f068806b4d0 msgr2=0x7f068806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.701+0000 7f06abb5c640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f068806b4d0 0x7f068806d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f06a0004f60 tx=0x7f06a0007e30 comp rx=0 tx=0).stop 2024-04-25T10:01:09.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.701+0000 7f06abb5c640 1 -- 172.21.15.139:0/518952782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a40d0a80 msgr2=0x7f06a414a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.701+0000 7f06abb5c640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a40d0a80 0x7f06a414a670 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f069c007e70 tx=0x7f069c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.701+0000 7f06abb5c640 1 -- 172.21.15.139:0/518952782 shutdown_connections 2024-04-25T10:01:09.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.701+0000 7f06abb5c640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06a40d0a80 0x7f06a414a670 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.701+0000 7f06abb5c640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f068806b4d0 0x7f068806d990 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.701+0000 7f06abb5c640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06a40cd610 0x7f06a40b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.701+0000 7f06abb5c640 1 --2- 172.21.15.139:0/518952782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06a409ad40 0x7f06a41485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.701+0000 7f06abb5c640 1 -- 172.21.15.139:0/518952782 >> 172.21.15.139:0/518952782 conn(0x7f06a40b31f0 msgr2=0x7f06a4160d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.701+0000 7f06abb5c640 1 -- 172.21.15.139:0/518952782 shutdown_connections 2024-04-25T10:01:09.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.701+0000 7f06abb5c640 1 -- 172.21.15.139:0/518952782 wait complete. 2024-04-25T10:01:09.716 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T10:01:09.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.809+0000 7fe5d9bc4640 1 Processor -- start 2024-04-25T10:01:09.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.809+0000 7fe5d9bc4640 1 -- start start 2024-04-25T10:01:09.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.810+0000 7fe5d9bc4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe5d40b37d0 0x7fe5d415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.810+0000 7fe5d9bc4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d40b3e10 con 0x7fe5d406da20 2024-04-25T10:01:09.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.810+0000 7fe5d9bc4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d41615c0 con 0x7fe5d4158260 2024-04-25T10:01:09.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.810+0000 7fe5d9bc4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d41618a0 con 0x7fe5d40b37d0 2024-04-25T10:01:09.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.810+0000 7fe5d2ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe5d40b37d0 0x7fe5d415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.810+0000 7fe5d3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe5d406da20 0x7fe5d406ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:47104/0 (socket says 172.21.15.139:47104) 2024-04-25T10:01:09.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.810+0000 7fe5d3fff640 1 -- 172.21.15.139:0/351332668 learned_addr learned my addr 172.21.15.139:0/351332668 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:09.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.810+0000 7fe5d2ffd640 1 -- 172.21.15.139:0/351332668 >> v1:172.21.15.150:6789/0 conn(0x7fe5d4158260 legacy=0x7fe5d40b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.810+0000 7fe5d2ffd640 1 -- 172.21.15.139:0/351332668 >> v1:172.21.15.139:6789/0 conn(0x7fe5d406da20 legacy=0x7fe5d406ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.810+0000 7fe5d2ffd640 1 -- 172.21.15.139:0/351332668 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d4161b80 con 0x7fe5d40b37d0 2024-04-25T10:01:09.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.810+0000 7fe5d27fc640 1 -- 172.21.15.139:0/351332668 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626341097 0 0) 0x7fe5cc001490 con 0x7fe5d406da20 2024-04-25T10:01:09.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.811+0000 7fe5d2ffd640 1 --2- 172.21.15.139:0/351332668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5d40b37d0 0x7fe5d415df80 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80097a0 tx=0x7fe5c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78d6766cacc102f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.811+0000 7fe5d27fc640 1 -- 172.21.15.139:0/351332668 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8037070 con 0x7fe5d40b37d0 2024-04-25T10:01:09.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.811+0000 7fe5d27fc640 1 -- 172.21.15.139:0/351332668 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8032440 con 0x7fe5d40b37d0 2024-04-25T10:01:09.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.811+0000 7fe5d27fc640 1 -- 172.21.15.139:0/351332668 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8032810 con 0x7fe5d40b37d0 2024-04-25T10:01:09.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.812+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/351332668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5d40b37d0 msgr2=0x7fe5d415df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.812+0000 7fe5d9bc4640 1 --2- 172.21.15.139:0/351332668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5d40b37d0 0x7fe5d415df80 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80097a0 tx=0x7fe5c802a230 comp rx=0 tx=0).stop 2024-04-25T10:01:09.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.812+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/351332668 shutdown_connections 2024-04-25T10:01:09.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.812+0000 7fe5d9bc4640 1 --2- 172.21.15.139:0/351332668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5d40b37d0 0x7fe5d415df80 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.812+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/351332668 >> 172.21.15.139:0/351332668 conn(0x7fe5d40b23d0 msgr2=0x7fe5d40cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.812+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/351332668 shutdown_connections 2024-04-25T10:01:09.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.812+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/351332668 wait complete. 2024-04-25T10:01:09.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.813+0000 7fe5d9bc4640 1 Processor -- start 2024-04-25T10:01:09.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.813+0000 7fe5d9bc4640 1 -- start start 2024-04-25T10:01:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d9bc4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5d406da20 0x7fe5d41e7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d9bc4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5d40b37d0 0x7fe5d4154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d9bc4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5d4158260 0x7fe5d4146730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d9bc4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4162420 con 0x7fe5d406da20 2024-04-25T10:01:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d9bc4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4162140 con 0x7fe5d4158260 2024-04-25T10:01:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d9bc4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d406b4b0 con 0x7fe5d40b37d0 2024-04-25T10:01:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5d406da20 0x7fe5d41e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5d406da20 0x7fe5d41e7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43222/0 (socket says 172.21.15.139:43222) 2024-04-25T10:01:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d37fe640 1 -- 172.21.15.139:0/1004379849 learned_addr learned my addr 172.21.15.139:0/1004379849 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d37fe640 1 -- 172.21.15.139:0/1004379849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5d40b37d0 msgr2=0x7fe5d4154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d3fff640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5d4158260 0x7fe5d4146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d2ffd640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5d40b37d0 0x7fe5d4154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d37fe640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5d40b37d0 0x7fe5d4154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d37fe640 1 -- 172.21.15.139:0/1004379849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5d4158260 msgr2=0x7fe5d4146730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d37fe640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5d4158260 0x7fe5d4146730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d37fe640 1 -- 172.21.15.139:0/1004379849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d4154fd0 con 0x7fe5d406da20 2024-04-25T10:01:09.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d3fff640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5d4158260 0x7fe5d4146730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:09.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d37fe640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5d406da20 0x7fe5d41e7540 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c430 tx=0x7fe5c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5bbfff640 1 -- 172.21.15.139:0/1004379849 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4017070 con 0x7fe5d406da20 2024-04-25T10:01:09.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5bbfff640 1 -- 172.21.15.139:0/1004379849 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c40052d0 con 0x7fe5d406da20 2024-04-25T10:01:09.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/1004379849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d41552b0 con 0x7fe5d406da20 2024-04-25T10:01:09.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.814+0000 7fe5bbfff640 1 -- 172.21.15.139:0/1004379849 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4003830 con 0x7fe5d406da20 2024-04-25T10:01:09.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.815+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/1004379849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d4155870 con 0x7fe5d406da20 2024-04-25T10:01:09.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.815+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/1004379849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d4074520 con 0x7fe5d406da20 2024-04-25T10:01:09.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.816+0000 7fe5bbfff640 1 -- 172.21.15.139:0/1004379849 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5c401a070 con 0x7fe5d406da20 2024-04-25T10:01:09.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.816+0000 7fe5bbfff640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe5c006b500 0x7fe5c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.816+0000 7fe5bbfff640 1 -- 172.21.15.139:0/1004379849 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe5c4089550 con 0x7fe5d406da20 2024-04-25T10:01:09.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.816+0000 7fe5d2ffd640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe5c006b500 0x7fe5c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.817+0000 7fe5d2ffd640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe5c006b500 0x7fe5c006d9c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8034000 tx=0x7fe5c802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:09.820+0000 7fe5bbfff640 1 -- 172.21.15.139:0/1004379849 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5c4058400 con 0x7fe5d406da20 2024-04-25T10:01:10.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.055+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/1004379849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7fe5d406e300 con 0x7fe5d406da20 2024-04-25T10:01:10.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.104+0000 7fe5bbfff640 1 -- 172.21.15.139:0/1004379849 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v372) v1 ==== 149+0+0 (secure 0 0 0) 0x7fe5c4057d00 con 0x7fe5d406da20 2024-04-25T10:01:10.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.106+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/1004379849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe5c006b500 msgr2=0x7fe5c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.106+0000 7fe5d9bc4640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe5c006b500 0x7fe5c006d9c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8034000 tx=0x7fe5c802e000 comp rx=0 tx=0).stop 2024-04-25T10:01:10.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.106+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/1004379849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5d406da20 msgr2=0x7fe5d41e7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.106+0000 7fe5d9bc4640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5d406da20 0x7fe5d41e7540 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c430 tx=0x7fe5c400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.106+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/1004379849 shutdown_connections 2024-04-25T10:01:10.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.106+0000 7fe5d9bc4640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe5c006b500 0x7fe5c006d9c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.106+0000 7fe5d9bc4640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5d4158260 0x7fe5d4146730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.106+0000 7fe5d9bc4640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5d40b37d0 0x7fe5d4154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.106+0000 7fe5d9bc4640 1 --2- 172.21.15.139:0/1004379849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5d406da20 0x7fe5d41e7540 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.106+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/1004379849 >> 172.21.15.139:0/1004379849 conn(0x7fe5d40b23d0 msgr2=0x7fe5d40b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.106+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/1004379849 shutdown_connections 2024-04-25T10:01:10.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.106+0000 7fe5d9bc4640 1 -- 172.21.15.139:0/1004379849 wait complete. 2024-04-25T10:01:10.121 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:01:10.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.217+0000 7fad0882a640 1 Processor -- start 2024-04-25T10:01:10.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.217+0000 7fad0882a640 1 -- start start 2024-04-25T10:01:10.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.218+0000 7fad0882a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad041617e0 0x7fad040ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.218+0000 7fad0882a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad040cec80 0x7fad040d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.218+0000 7fad0882a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad04161dc0 con 0x7fad0415dbf0 2024-04-25T10:01:10.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.218+0000 7fad0882a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad040d1ee0 con 0x7fad040cec80 2024-04-25T10:01:10.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.218+0000 7fad0882a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad041620a0 con 0x7fad041617e0 2024-04-25T10:01:10.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.218+0000 7fad02d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad040cec80 0x7fad040d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.218+0000 7fad02d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad040cec80 0x7fad040d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43232/0 (socket says 172.21.15.139:43232) 2024-04-25T10:01:10.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.218+0000 7fad02d9d640 1 -- 172.21.15.139:0/1412081091 learned_addr learned my addr 172.21.15.139:0/1412081091 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:10.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.218+0000 7fad01d9b640 1 --2- 172.21.15.139:0/1412081091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad041617e0 0x7fad040ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.219+0000 7fad02d9d640 1 -- 172.21.15.139:0/1412081091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad041617e0 msgr2=0x7fad040ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:10.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.219+0000 7fad02d9d640 1 --2- 172.21.15.139:0/1412081091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad041617e0 0x7fad040ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.219+0000 7fad02d9d640 1 -- 172.21.15.139:0/1412081091 >> v1:172.21.15.150:6789/0 conn(0x7fad0415dbf0 legacy=0x7fad040b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.219+0000 7fad02d9d640 1 -- 172.21.15.139:0/1412081091 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad04162380 con 0x7fad040cec80 2024-04-25T10:01:10.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.219+0000 7fad02d9d640 1 --2- 172.21.15.139:0/1412081091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad040cec80 0x7fad040d2370 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7facfc009e50 tx=0x7facfc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69877668d2c16832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.219+0000 7fad0159a640 1 -- 172.21.15.139:0/1412081091 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc036070 con 0x7fad040cec80 2024-04-25T10:01:10.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.219+0000 7fad0159a640 1 -- 172.21.15.139:0/1412081091 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc031600 con 0x7fad040cec80 2024-04-25T10:01:10.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.219+0000 7fad0159a640 1 -- 172.21.15.139:0/1412081091 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0319d0 con 0x7fad040cec80 2024-04-25T10:01:10.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.220+0000 7fad0882a640 1 -- 172.21.15.139:0/1412081091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad040cec80 msgr2=0x7fad040d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.220+0000 7fad0882a640 1 --2- 172.21.15.139:0/1412081091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad040cec80 0x7fad040d2370 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7facfc009e50 tx=0x7facfc02a680 comp rx=0 tx=0).stop 2024-04-25T10:01:10.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.220+0000 7fad0882a640 1 -- 172.21.15.139:0/1412081091 shutdown_connections 2024-04-25T10:01:10.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.220+0000 7fad0882a640 1 --2- 172.21.15.139:0/1412081091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad040cec80 0x7fad040d2370 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.220+0000 7fad0882a640 1 --2- 172.21.15.139:0/1412081091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad041617e0 0x7fad040ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.220+0000 7fad0882a640 1 -- 172.21.15.139:0/1412081091 >> 172.21.15.139:0/1412081091 conn(0x7fad040b5480 msgr2=0x7fad040b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.220+0000 7fad0882a640 1 -- 172.21.15.139:0/1412081091 shutdown_connections 2024-04-25T10:01:10.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.220+0000 7fad0882a640 1 -- 172.21.15.139:0/1412081091 wait complete. 2024-04-25T10:01:10.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.221+0000 7fad0882a640 1 Processor -- start 2024-04-25T10:01:10.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.221+0000 7fad0882a640 1 -- start start 2024-04-25T10:01:10.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.222+0000 7fad0882a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad040cec80 0x7fad041557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.222+0000 7fad0882a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad0415dbf0 0x7fad041609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.222+0000 7fad0882a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041617e0 0x7fad04152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.222+0000 7fad0882a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0415dfd0 con 0x7fad041617e0 2024-04-25T10:01:10.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.222+0000 7fad0882a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad040d28b0 con 0x7fad040cec80 2024-04-25T10:01:10.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.222+0000 7fad0882a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad04153620 con 0x7fad0415dbf0 2024-04-25T10:01:10.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.222+0000 7fad02d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041617e0 0x7fad04152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.222+0000 7fad02d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041617e0 0x7fad04152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43246/0 (socket says 172.21.15.139:43246) 2024-04-25T10:01:10.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.222+0000 7fad02d9d640 1 -- 172.21.15.139:0/2618552204 learned_addr learned my addr 172.21.15.139:0/2618552204 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:10.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.222+0000 7fad01d9b640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad0415dbf0 0x7fad041609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.222+0000 7fad0259c640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad040cec80 0x7fad041557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.223+0000 7fad02d9d640 1 -- 172.21.15.139:0/2618552204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad0415dbf0 msgr2=0x7fad041609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.223+0000 7fad02d9d640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad0415dbf0 0x7fad041609d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.223+0000 7fad02d9d640 1 -- 172.21.15.139:0/2618552204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad040cec80 msgr2=0x7fad041557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.223+0000 7fad02d9d640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad040cec80 0x7fad041557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.223+0000 7fad02d9d640 1 -- 172.21.15.139:0/2618552204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad04153870 con 0x7fad041617e0 2024-04-25T10:01:10.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.223+0000 7fad0259c640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad040cec80 0x7fad041557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:10.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.223+0000 7fad02d9d640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041617e0 0x7fad04152a70 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7facfc009e50 tx=0x7facfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.223+0000 7facf2ffd640 1 -- 172.21.15.139:0/2618552204 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc041020 con 0x7fad041617e0 2024-04-25T10:01:10.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.223+0000 7facf2ffd640 1 -- 172.21.15.139:0/2618552204 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc007780 con 0x7fad041617e0 2024-04-25T10:01:10.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.223+0000 7fad0882a640 1 -- 172.21.15.139:0/2618552204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad04153b50 con 0x7fad041617e0 2024-04-25T10:01:10.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.223+0000 7facf2ffd640 1 -- 172.21.15.139:0/2618552204 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc036040 con 0x7fad041617e0 2024-04-25T10:01:10.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.223+0000 7fad0882a640 1 -- 172.21.15.139:0/2618552204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad04154110 con 0x7fad041617e0 2024-04-25T10:01:10.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.224+0000 7fad0882a640 1 -- 172.21.15.139:0/2618552204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0415e150 con 0x7fad041617e0 2024-04-25T10:01:10.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.224+0000 7facf2ffd640 1 -- 172.21.15.139:0/2618552204 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7facfc044020 con 0x7fad041617e0 2024-04-25T10:01:10.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.225+0000 7facf2ffd640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7face406b4d0 0x7face406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.225+0000 7fad0259c640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7face406b4d0 0x7face406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.225+0000 7facf2ffd640 1 -- 172.21.15.139:0/2618552204 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7facfc0aadc0 con 0x7fad041617e0 2024-04-25T10:01:10.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.225+0000 7fad0259c640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7face406b4d0 0x7face406d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7facf8001680 tx=0x7facf8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.229+0000 7facf2ffd640 1 -- 172.21.15.139:0/2618552204 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7facfc04b030 con 0x7fad041617e0 2024-04-25T10:01:10.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.399+0000 7fad0882a640 1 -- 172.21.15.139:0/2618552204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fad04074520 con 0x7fad041617e0 2024-04-25T10:01:10.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.402+0000 7facf2ffd640 1 -- 172.21.15.139:0/2618552204 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7facfc079e50 con 0x7fad041617e0 2024-04-25T10:01:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.404+0000 7fad0882a640 1 -- 172.21.15.139:0/2618552204 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7face406b4d0 msgr2=0x7face406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.404+0000 7fad0882a640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7face406b4d0 0x7face406d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7facf8001680 tx=0x7facf8009040 comp rx=0 tx=0).stop 2024-04-25T10:01:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.404+0000 7fad0882a640 1 -- 172.21.15.139:0/2618552204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041617e0 msgr2=0x7fad04152a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.404+0000 7fad0882a640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041617e0 0x7fad04152a70 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7facfc009e50 tx=0x7facfc004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.404+0000 7fad0882a640 1 -- 172.21.15.139:0/2618552204 shutdown_connections 2024-04-25T10:01:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.404+0000 7fad0882a640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad041617e0 0x7fad04152a70 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.404+0000 7fad0882a640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7face406b4d0 0x7face406d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.404+0000 7fad0882a640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad0415dbf0 0x7fad041609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.404+0000 7fad0882a640 1 --2- 172.21.15.139:0/2618552204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad040cec80 0x7fad041557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.405+0000 7fad0882a640 1 -- 172.21.15.139:0/2618552204 >> 172.21.15.139:0/2618552204 conn(0x7fad040b5480 msgr2=0x7fad040d2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.405+0000 7fad0882a640 1 -- 172.21.15.139:0/2618552204 shutdown_connections 2024-04-25T10:01:10.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.405+0000 7fad0882a640 1 -- 172.21.15.139:0/2618552204 wait complete. 2024-04-25T10:01:10.420 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:01:10.420 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:01:10.471 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:01:10.525 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:01:10.525 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:10.525 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:10.542 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:01:10.542 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T10:01:10.542 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:10.560 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:10.560 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:10.560 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:10.597 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:01:10.597 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T10:01:10.597 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:10.617 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:10.618 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:01:10.618 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:10.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.729+0000 7f4f9241f640 1 Processor -- start 2024-04-25T10:01:10.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.729+0000 7f4f9241f640 1 -- start start 2024-04-25T10:01:10.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f9241f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4f8c06c1f0 0x7f4f8c15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f9241f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8c160280 con 0x7f4f8c0ad740 2024-04-25T10:01:10.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f9241f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8c0af590 con 0x7f4f8c15c9c0 2024-04-25T10:01:10.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f9241f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f8c0af870 con 0x7f4f8c06c1f0 2024-04-25T10:01:10.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f909bc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4f8c0ad740 0x7f4f8c06ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:47118/0 (socket says 172.21.15.139:47118) 2024-04-25T10:01:10.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f909bc640 1 -- 172.21.15.139:0/1038415187 learned_addr learned my addr 172.21.15.139:0/1038415187 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:10.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842390010 0 0) 0x7f4f84001490 con 0x7f4f8c0ad740 2024-04-25T10:01:10.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f8bfff640 1 --2- 172.21.15.139:0/1038415187 >> v2:172.21.15.150:3300/0 conn(0x7f4f8c06c1f0 0x7f4f8c15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f74003520 con 0x7f4f8c0ad740 2024-04-25T10:01:10.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 105741484 0 0) 0x7f4f74003520 con 0x7f4f8c0ad740 2024-04-25T10:01:10.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 >> v2:172.21.15.150:3300/0 conn(0x7f4f8c06c1f0 msgr2=0x7f4f8c15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:10.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f8affd640 1 --2- 172.21.15.139:0/1038415187 >> v2:172.21.15.150:3300/0 conn(0x7f4f8c06c1f0 0x7f4f8c15fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 >> v1:172.21.15.150:6789/0 conn(0x7f4f8c15c9c0 legacy=0x7f4f8c0ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c15bee0 con 0x7f4f8c0ad740 2024-04-25T10:01:10.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f4f84001bc0 con 0x7f4f8c0ad740 2024-04-25T10:01:10.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 >> v1:172.21.15.139:6789/0 conn(0x7f4f8c0ad740 legacy=0x7f4f8c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.730+0000 7f4f8affd640 1 --2- 172.21.15.139:0/1038415187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f740039b0 0x7f4f74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8affd640 1 --2- 172.21.15.139:0/1038415187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f74004800 0x7f4f74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8affd640 1 --2- 172.21.15.139:0/1038415187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f740051c0 0x7f4f7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c0af870 con 0x7f4f740039b0 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7400c160 con 0x7f4f74004800 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7400c360 con 0x7f4f740051c0 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8bfff640 1 --2- 172.21.15.139:0/1038415187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f740039b0 0x7f4f74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8b7fe640 1 --2- 172.21.15.139:0/1038415187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f74004800 0x7f4f74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f909bc640 1 --2- 172.21.15.139:0/1038415187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f740051c0 0x7f4f7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8bfff640 1 -- 172.21.15.139:0/1038415187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f740051c0 msgr2=0x7f4f7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8bfff640 1 --2- 172.21.15.139:0/1038415187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f740051c0 0x7f4f7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8bfff640 1 -- 172.21.15.139:0/1038415187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f74004800 msgr2=0x7f4f74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8bfff640 1 --2- 172.21.15.139:0/1038415187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f74004800 0x7f4f74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f909bc640 1 -- 172.21.15.139:0/1038415187 reap_dead start 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8bfff640 1 -- 172.21.15.139:0/1038415187 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f74003520 con 0x7f4f740039b0 2024-04-25T10:01:10.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.731+0000 7f4f8b7fe640 1 --2- 172.21.15.139:0/1038415187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f74004800 0x7f4f74004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:10.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.732+0000 7f4f8bfff640 1 --2- 172.21.15.139:0/1038415187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f740039b0 0x7f4f74003dd0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009ac0 tx=0x7f4f74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e5ae7b3cb0bc09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.732+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c036070 con 0x7f4f740039b0 2024-04-25T10:01:10.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.732+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7c031400 con 0x7f4f740039b0 2024-04-25T10:01:10.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.732+0000 7f4f8affd640 1 -- 172.21.15.139:0/1038415187 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c009690 con 0x7f4f740039b0 2024-04-25T10:01:10.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.732+0000 7f4f9241f640 1 -- 172.21.15.139:0/1038415187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f740039b0 msgr2=0x7f4f74003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.732+0000 7f4f9241f640 1 --2- 172.21.15.139:0/1038415187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f740039b0 0x7f4f74003dd0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009ac0 tx=0x7f4f74005830 comp rx=0 tx=0).stop 2024-04-25T10:01:10.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.732+0000 7f4f9241f640 1 -- 172.21.15.139:0/1038415187 shutdown_connections 2024-04-25T10:01:10.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.732+0000 7f4f9241f640 1 --2- 172.21.15.139:0/1038415187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f740039b0 0x7f4f74003dd0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.732+0000 7f4f9241f640 1 -- 172.21.15.139:0/1038415187 >> 172.21.15.139:0/1038415187 conn(0x7f4f8c0b23d0 msgr2=0x7f4f8c0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.732+0000 7f4f9241f640 1 -- 172.21.15.139:0/1038415187 shutdown_connections 2024-04-25T10:01:10.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.732+0000 7f4f9241f640 1 -- 172.21.15.139:0/1038415187 wait complete. 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.733+0000 7f4f9241f640 1 Processor -- start 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.734+0000 7f4f9241f640 1 -- start start 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.734+0000 7f4f9241f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f740039b0 0x7f4f8c145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.734+0000 7f4f9241f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f8c0c9130 0x7f4f8c1e5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.734+0000 7f4f9241f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f8c1e56b0 0x7f4f8c147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.734+0000 7f4f9241f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c06f630 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.734+0000 7f4f9241f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c15c530 con 0x7f4f8c0c9130 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.734+0000 7f4f9241f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c06f7b0 con 0x7f4f740039b0 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.734+0000 7f4f909bc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f8c1e56b0 0x7f4f8c147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.734+0000 7f4f909bc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f8c1e56b0 0x7f4f8c147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43270/0 (socket says 172.21.15.139:43270) 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.734+0000 7f4f909bc640 1 -- 172.21.15.139:0/884159285 learned_addr learned my addr 172.21.15.139:0/884159285 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f8b7fe640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f8c0c9130 0x7f4f8c1e5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f8bfff640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f740039b0 0x7f4f8c145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f909bc640 1 -- 172.21.15.139:0/884159285 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f740039b0 msgr2=0x7f4f8c145130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f909bc640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f740039b0 0x7f4f8c145130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f909bc640 1 -- 172.21.15.139:0/884159285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f8c0c9130 msgr2=0x7f4f8c1e5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f909bc640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f8c0c9130 0x7f4f8c1e5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f909bc640 1 -- 172.21.15.139:0/884159285 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c145e60 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f909bc640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f8c1e56b0 0x7f4f8c147b10 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0cc520 tx=0x7f4f8400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f88ff9640 1 -- 172.21.15.139:0/884159285 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f84019070 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f88ff9640 1 -- 172.21.15.139:0/884159285 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f84001c00 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f88ff9640 1 -- 172.21.15.139:0/884159285 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f84014480 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f9241f640 1 -- 172.21.15.139:0/884159285 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8c146140 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.735+0000 7f4f9241f640 1 -- 172.21.15.139:0/884159285 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8c146700 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.736+0000 7f4f9241f640 1 -- 172.21.15.139:0/884159285 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8c074520 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.736+0000 7f4f88ff9640 1 -- 172.21.15.139:0/884159285 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f8401c070 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.737+0000 7f4f88ff9640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4f7806b500 0x7f4f7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.737+0000 7f4f88ff9640 1 -- 172.21.15.139:0/884159285 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f4f84089da0 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.737+0000 7f4f8bfff640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4f7806b500 0x7f4f7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.737+0000 7f4f8bfff640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4f7806b500 0x7f4f7806d9c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c033000 tx=0x7f4f7c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.741+0000 7f4f88ff9640 1 -- 172.21.15.139:0/884159285 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f8c160280 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.944+0000 7f4f9241f640 1 -- 172.21.15.139:0/884159285 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f8c0d09c0 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.944+0000 7f4f88ff9640 1 -- 172.21.15.139:0/884159285 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f4f84058ce0 con 0x7f4f8c1e56b0 2024-04-25T10:01:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:10.949 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":65,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6074,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2911215694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2911215694},{"type":"v1","addr":"172.21.15.150:6834","nonce":2911215694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4021849384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4021849384},{"type":"v1","addr":"172.21.15.150:6835","nonce":4021849384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7542,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/1331237155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":1331237155},{"type":"v1","addr":"172.21.15.139:6837","nonce":1331237155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":8379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2202268752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2202268752},{"type":"v1","addr":"172.21.15.139:6836","nonce":2202268752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[]} 2024-04-25T10:01:10.949 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 65 2024-04-25T10:01:10.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.946+0000 7f4f9241f640 1 -- 172.21.15.139:0/884159285 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4f7806b500 msgr2=0x7f4f7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.946+0000 7f4f9241f640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4f7806b500 0x7f4f7806d9c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c033000 tx=0x7f4f7c008030 comp rx=0 tx=0).stop 2024-04-25T10:01:10.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.946+0000 7f4f9241f640 1 -- 172.21.15.139:0/884159285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f8c1e56b0 msgr2=0x7f4f8c147b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.946+0000 7f4f9241f640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f8c1e56b0 0x7f4f8c147b10 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0cc520 tx=0x7f4f8400e530 comp rx=0 tx=0).stop 2024-04-25T10:01:10.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.946+0000 7f4f9241f640 1 -- 172.21.15.139:0/884159285 shutdown_connections 2024-04-25T10:01:10.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.946+0000 7f4f9241f640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f8c1e56b0 0x7f4f8c147b10 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.946+0000 7f4f9241f640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4f7806b500 0x7f4f7806d9c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.946+0000 7f4f9241f640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f8c0c9130 0x7f4f8c1e5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.946+0000 7f4f9241f640 1 --2- 172.21.15.139:0/884159285 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4f740039b0 0x7f4f8c145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.946+0000 7f4f9241f640 1 -- 172.21.15.139:0/884159285 >> 172.21.15.139:0/884159285 conn(0x7f4f8c0b23d0 msgr2=0x7f4f8c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.946+0000 7f4f9241f640 1 -- 172.21.15.139:0/884159285 shutdown_connections 2024-04-25T10:01:10.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:10.946+0000 7f4f9241f640 1 -- 172.21.15.139:0/884159285 wait complete. 2024-04-25T10:01:10.962 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:01:10.962 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:01:10.962 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:01:10.962 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:10.963 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:01:10.963 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:01:10.963 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:01:10.963 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:10.963 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:01:10.963 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:01:10.964 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:01:10.964 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:10.964 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:01:10.964 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:01:10.964 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:01:10.964 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:10.972 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:01:10.972 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:01:10.973 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:01:10.973 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:01:10.977 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:01:10.977 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:01:10.978 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:01:10.978 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:01:11.015 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:01:11.015 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:11.016 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:01:11.016 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:11.019 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:01:11.020 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:11.020 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:01:11.020 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:11.021 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:01:11.087 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T10:01:11.088 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-04-25T10:01:11.100 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-04-25T10:01:11.101 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-04-25T10:01:11.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.122+0000 7fe78f61f640 1 Processor -- start 2024-04-25T10:01:11.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.122+0000 7fe78f61f640 1 -- start start 2024-04-25T10:01:11.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.123+0000 7fe78f61f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 0x7fe7880b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.123+0000 7fe78f61f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7880c6cc0 con 0x7fe7880af8a0 2024-04-25T10:01:11.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.123+0000 7fe78f61f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7880c6fa0 con 0x7fe7880afc80 2024-04-25T10:01:11.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.123+0000 7fe78f61f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7880c7280 con 0x7fe78806b270 2024-04-25T10:01:11.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.123+0000 7fe78d3bb640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe7880af8a0 0x7fe7880cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:47132/0 (socket says 172.21.15.139:47132) 2024-04-25T10:01:11.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.123+0000 7fe78d3bb640 1 -- 172.21.15.139:0/4163240316 learned_addr learned my addr 172.21.15.139:0/4163240316 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:11.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.123+0000 7fe78dbbc640 1 --2- 172.21.15.139:0/4163240316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 0x7fe7880b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.123+0000 7fe78dbbc640 1 -- 172.21.15.139:0/4163240316 >> v1:172.21.15.150:6789/0 conn(0x7fe7880afc80 legacy=0x7fe78806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.123+0000 7fe78dbbc640 1 -- 172.21.15.139:0/4163240316 >> v1:172.21.15.139:6789/0 conn(0x7fe7880af8a0 legacy=0x7fe7880cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.123+0000 7fe78dbbc640 1 -- 172.21.15.139:0/4163240316 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7880cd090 con 0x7fe78806b270 2024-04-25T10:01:11.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.124+0000 7fe78dbbc640 1 --2- 172.21.15.139:0/4163240316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 0x7fe7880b2d50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe780009df0 tx=0x7fe78002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83f6f48384c153c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.124+0000 7fe77ffff640 1 -- 172.21.15.139:0/4163240316 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780007740 con 0x7fe78806b270 2024-04-25T10:01:11.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.124+0000 7fe77ffff640 1 -- 172.21.15.139:0/4163240316 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7800078e0 con 0x7fe78806b270 2024-04-25T10:01:11.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.124+0000 7fe77ffff640 1 -- 172.21.15.139:0/4163240316 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780007cb0 con 0x7fe78806b270 2024-04-25T10:01:11.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.125+0000 7fe78f61f640 1 -- 172.21.15.139:0/4163240316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 msgr2=0x7fe7880b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.125+0000 7fe78f61f640 1 --2- 172.21.15.139:0/4163240316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 0x7fe7880b2d50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe780009df0 tx=0x7fe78002a710 comp rx=0 tx=0).stop 2024-04-25T10:01:11.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.125+0000 7fe78f61f640 1 -- 172.21.15.139:0/4163240316 shutdown_connections 2024-04-25T10:01:11.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.125+0000 7fe78f61f640 1 --2- 172.21.15.139:0/4163240316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 0x7fe7880b2d50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.125+0000 7fe78f61f640 1 -- 172.21.15.139:0/4163240316 >> 172.21.15.139:0/4163240316 conn(0x7fe7880b2230 msgr2=0x7fe7880b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.125+0000 7fe78f61f640 1 -- 172.21.15.139:0/4163240316 shutdown_connections 2024-04-25T10:01:11.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.125+0000 7fe78f61f640 1 -- 172.21.15.139:0/4163240316 wait complete. 2024-04-25T10:01:11.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.127+0000 7fe78f61f640 1 Processor -- start 2024-04-25T10:01:11.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.127+0000 7fe78f61f640 1 -- start start 2024-04-25T10:01:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78f61f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 0x7fe788158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78f61f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880af8a0 0x7fe7881eaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78f61f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7880afc80 0x7fe7881e90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78f61f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78806b880 con 0x7fe7880af8a0 2024-04-25T10:01:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78f61f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78806b700 con 0x7fe7880afc80 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78f61f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7880c54a0 con 0x7fe78806b270 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78cbba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880af8a0 0x7fe7881eaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78d3bb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 0x7fe788158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78cbba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880af8a0 0x7fe7881eaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43308/0 (socket says 172.21.15.139:43308) 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78d3bb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 0x7fe788158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:60180/0 (socket says 172.21.15.139:60180) 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78cbba640 1 -- 172.21.15.139:0/2099910012 learned_addr learned my addr 172.21.15.139:0/2099910012 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78dbbc640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7880afc80 0x7fe7881e90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78d3bb640 1 -- 172.21.15.139:0/2099910012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7880afc80 msgr2=0x7fe7881e90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78d3bb640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7880afc80 0x7fe7881e90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78d3bb640 1 -- 172.21.15.139:0/2099910012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880af8a0 msgr2=0x7fe7881eaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78d3bb640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880af8a0 0x7fe7881eaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78d3bb640 1 -- 172.21.15.139:0/2099910012 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788156050 con 0x7fe78806b270 2024-04-25T10:01:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.128+0000 7fe78d3bb640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 0x7fe788158db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c440 tx=0x7fe784001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.129+0000 7fe77dffb640 1 -- 172.21.15.139:0/2099910012 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784017070 con 0x7fe78806b270 2024-04-25T10:01:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.129+0000 7fe77dffb640 1 -- 172.21.15.139:0/2099910012 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe784005ce0 con 0x7fe78806b270 2024-04-25T10:01:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.129+0000 7fe78f61f640 1 -- 172.21.15.139:0/2099910012 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe788156330 con 0x7fe78806b270 2024-04-25T10:01:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.129+0000 7fe77dffb640 1 -- 172.21.15.139:0/2099910012 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7840072d0 con 0x7fe78806b270 2024-04-25T10:01:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.129+0000 7fe78f61f640 1 -- 172.21.15.139:0/2099910012 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7881568f0 con 0x7fe78806b270 2024-04-25T10:01:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.130+0000 7fe78f61f640 1 -- 172.21.15.139:0/2099910012 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78806b090 con 0x7fe78806b270 2024-04-25T10:01:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.130+0000 7fe77dffb640 1 -- 172.21.15.139:0/2099910012 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe78401a070 con 0x7fe78806b270 2024-04-25T10:01:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.130+0000 7fe77dffb640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe76c06b4d0 0x7fe76c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.130+0000 7fe77dffb640 1 -- 172.21.15.139:0/2099910012 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe784089330 con 0x7fe78806b270 2024-04-25T10:01:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.130+0000 7fe78cbba640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe76c06b4d0 0x7fe76c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.131+0000 7fe78cbba640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe76c06b4d0 0x7fe76c06d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe7881623e0 tx=0x7fe778009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.135+0000 7fe77dffb640 1 -- 172.21.15.139:0/2099910012 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe78806b090 con 0x7fe78806b270 2024-04-25T10:01:11.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:11.317+0000 7fe78f61f640 1 -- 172.21.15.139:0/2099910012 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe7881558a0 con 0x7fe78806b270 2024-04-25T10:01:12.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.126+0000 7fe77dffb640 1 -- 172.21.15.139:0/2099910012 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v101) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe784057cb0 con 0x7fe78806b270 2024-04-25T10:01:12.130 INFO:teuthology.orchestra.run.smithi139.stderr: removed all blocklist entries 2024-04-25T10:01:12.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.129+0000 7fe78f61f640 1 -- 172.21.15.139:0/2099910012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe76c06b4d0 msgr2=0x7fe76c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.129+0000 7fe78f61f640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe76c06b4d0 0x7fe76c06d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe7881623e0 tx=0x7fe778009040 comp rx=0 tx=0).stop 2024-04-25T10:01:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.129+0000 7fe78f61f640 1 -- 172.21.15.139:0/2099910012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 msgr2=0x7fe788158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.129+0000 7fe78f61f640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 0x7fe788158db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c440 tx=0x7fe784001e70 comp rx=0 tx=0).stop 2024-04-25T10:01:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.129+0000 7fe78f61f640 1 -- 172.21.15.139:0/2099910012 shutdown_connections 2024-04-25T10:01:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.129+0000 7fe78f61f640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7880af8a0 0x7fe7881eaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.129+0000 7fe78f61f640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe76c06b4d0 0x7fe76c06d990 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.129+0000 7fe78f61f640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7880afc80 0x7fe7881e90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.129+0000 7fe78f61f640 1 --2- 172.21.15.139:0/2099910012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78806b270 0x7fe788158db0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.129+0000 7fe78f61f640 1 -- 172.21.15.139:0/2099910012 >> 172.21.15.139:0/2099910012 conn(0x7fe7880b2230 msgr2=0x7fe7880c6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.129+0000 7fe78f61f640 1 -- 172.21.15.139:0/2099910012 shutdown_connections 2024-04-25T10:01:12.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.129+0000 7fe78f61f640 1 -- 172.21.15.139:0/2099910012 wait complete. 2024-04-25T10:01:12.146 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:01:12.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.244+0000 7faffa0fc640 1 Processor -- start 2024-04-25T10:01:12.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.244+0000 7faffa0fc640 1 -- start start 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.245+0000 7faffa0fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faff40b0d00 0x7faff415e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.245+0000 7faffa0fc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faff415e610 0x7faff40d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.245+0000 7faffa0fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff40d12c0 0x7faff406ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.245+0000 7faffa0fc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff40d0350 con 0x7faff40b0d00 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.245+0000 7faffa0fc640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff415f070 con 0x7faff415e610 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.245+0000 7faffa0fc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff40b10e0 con 0x7faff40d12c0 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.245+0000 7faff37fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faff40b0d00 0x7faff415e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.245+0000 7faff37fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faff40b0d00 0x7faff415e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43320/0 (socket says 172.21.15.139:43320) 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.245+0000 7faff37fe640 1 -- 172.21.15.139:0/3030252297 learned_addr learned my addr 172.21.15.139:0/3030252297 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.245+0000 7faff3fff640 1 --2- 172.21.15.139:0/3030252297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff40d12c0 0x7faff406ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.245+0000 7faff2ffd640 1 --2- 172.21.15.139:0/3030252297 >> v2:172.21.15.150:3300/0 conn(0x7faff415e610 0x7faff40d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.246+0000 7faff37fe640 1 -- 172.21.15.139:0/3030252297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff40d12c0 msgr2=0x7faff406ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:12.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.246+0000 7faff37fe640 1 --2- 172.21.15.139:0/3030252297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff40d12c0 0x7faff406ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.246+0000 7faff37fe640 1 -- 172.21.15.139:0/3030252297 >> v2:172.21.15.150:3300/0 conn(0x7faff415e610 msgr2=0x7faff40d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.246+0000 7faff37fe640 1 --2- 172.21.15.139:0/3030252297 >> v2:172.21.15.150:3300/0 conn(0x7faff415e610 0x7faff40d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.246+0000 7faff37fe640 1 -- 172.21.15.139:0/3030252297 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff415eab0 con 0x7faff40b0d00 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.246+0000 7faff3fff640 1 --2- 172.21.15.139:0/3030252297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff40d12c0 0x7faff406ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.246+0000 7faff2ffd640 1 --2- 172.21.15.139:0/3030252297 >> v2:172.21.15.150:3300/0 conn(0x7faff415e610 0x7faff40d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.246+0000 7faff37fe640 1 --2- 172.21.15.139:0/3030252297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faff40b0d00 0x7faff415e0d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7faff40cef00 tx=0x7fafe802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea34728c89cdd85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.246+0000 7faff27fc640 1 -- 172.21.15.139:0/3030252297 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8037070 con 0x7faff40b0d00 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.246+0000 7faff27fc640 1 -- 172.21.15.139:0/3030252297 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe8032440 con 0x7faff40b0d00 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.246+0000 7faff27fc640 1 -- 172.21.15.139:0/3030252297 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8032810 con 0x7faff40b0d00 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.247+0000 7faffa0fc640 1 -- 172.21.15.139:0/3030252297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faff40b0d00 msgr2=0x7faff415e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.247+0000 7faffa0fc640 1 --2- 172.21.15.139:0/3030252297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faff40b0d00 0x7faff415e0d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7faff40cef00 tx=0x7fafe802a140 comp rx=0 tx=0).stop 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.247+0000 7faffa0fc640 1 -- 172.21.15.139:0/3030252297 shutdown_connections 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.247+0000 7faffa0fc640 1 --2- 172.21.15.139:0/3030252297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff40d12c0 0x7faff406ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.247+0000 7faffa0fc640 1 --2- 172.21.15.139:0/3030252297 >> v2:172.21.15.150:3300/0 conn(0x7faff415e610 0x7faff40d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.247+0000 7faffa0fc640 1 --2- 172.21.15.139:0/3030252297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faff40b0d00 0x7faff415e0d0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.247+0000 7faffa0fc640 1 -- 172.21.15.139:0/3030252297 >> 172.21.15.139:0/3030252297 conn(0x7faff40b3380 msgr2=0x7faff40b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.247+0000 7faffa0fc640 1 -- 172.21.15.139:0/3030252297 shutdown_connections 2024-04-25T10:01:12.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.247+0000 7faffa0fc640 1 -- 172.21.15.139:0/3030252297 wait complete. 2024-04-25T10:01:12.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.248+0000 7faffa0fc640 1 Processor -- start 2024-04-25T10:01:12.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.248+0000 7faffa0fc640 1 -- start start 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faffa0fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff40b0d00 0x7faff40b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faffa0fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faff40d12c0 0x7faff41f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faffa0fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff415e610 0x7faff40b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faffa0fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff406c8a0 con 0x7faff40d12c0 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faffa0fc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff40d0580 con 0x7faff415e610 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faffa0fc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff406ca20 con 0x7faff40b0d00 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faff2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faff40d12c0 0x7faff41f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faff2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faff40d12c0 0x7faff41f3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43334/0 (socket says 172.21.15.139:43334) 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faff2ffd640 1 -- 172.21.15.139:0/1874419703 learned_addr learned my addr 172.21.15.139:0/1874419703 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faff3fff640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff415e610 0x7faff40b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faff37fe640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff40b0d00 0x7faff40b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faff2ffd640 1 -- 172.21.15.139:0/1874419703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff40b0d00 msgr2=0x7faff40b6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faff2ffd640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff40b0d00 0x7faff40b6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faff2ffd640 1 -- 172.21.15.139:0/1874419703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff415e610 msgr2=0x7faff40b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faff2ffd640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff415e610 0x7faff40b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faff2ffd640 1 -- 172.21.15.139:0/1874419703 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff41f6fe0 con 0x7faff40d12c0 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faff3fff640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff415e610 0x7faff40b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.249+0000 7faff37fe640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff40b0d00 0x7faff40b6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.250+0000 7faff2ffd640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faff40d12c0 0x7faff41f3f20 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c7f0 tx=0x7fafe400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.250+0000 7fafdbfff640 1 -- 172.21.15.139:0/1874419703 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4015070 con 0x7faff40d12c0 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.250+0000 7fafdbfff640 1 -- 172.21.15.139:0/1874419703 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe4007820 con 0x7faff40d12c0 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.250+0000 7faffa0fc640 1 -- 172.21.15.139:0/1874419703 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff41f72c0 con 0x7faff40d12c0 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.250+0000 7fafdbfff640 1 -- 172.21.15.139:0/1874419703 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4007c30 con 0x7faff40d12c0 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.250+0000 7faffa0fc640 1 -- 172.21.15.139:0/1874419703 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff41f7880 con 0x7faff40d12c0 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.251+0000 7faffa0fc640 1 -- 172.21.15.139:0/1874419703 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff406de00 con 0x7faff40d12c0 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.251+0000 7fafdbfff640 1 -- 172.21.15.139:0/1874419703 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafe4018070 con 0x7faff40d12c0 2024-04-25T10:01:12.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.252+0000 7fafdbfff640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fafe006b4d0 0x7fafe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.252+0000 7fafdbfff640 1 -- 172.21.15.139:0/1874419703 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fafe4089e10 con 0x7faff40d12c0 2024-04-25T10:01:12.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.252+0000 7faff37fe640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fafe006b4d0 0x7fafe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.252+0000 7faff37fe640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fafe006b4d0 0x7fafe006d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fafe8034000 tx=0x7fafe802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.256+0000 7fafdbfff640 1 -- 172.21.15.139:0/1874419703 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fafe4058ee0 con 0x7faff40d12c0 2024-04-25T10:01:12.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.492+0000 7faffa0fc640 1 -- 172.21.15.139:0/1874419703 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7faff4074520 con 0x7faff40d12c0 2024-04-25T10:01:12.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.492+0000 7fafdbfff640 1 -- 172.21.15.139:0/1874419703 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fafe40587e0 con 0x7faff40d12c0 2024-04-25T10:01:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:12.496 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_dump": [ 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.a", 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypm0xdaHhAA6nupxCTQhucepW2iEWNUoQ==", 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.b", 2024-04-25T10:01:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmB9GDKRAA/lmU4OiKO5dFgayYx09GQg==", 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.c", 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmvYgaJRAA/YIE6ralp2WuTK993D7H0Q==", 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T10:01:12.498 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.d", 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmg49WMBAAlPY9ZSeOoV296xSdl/m1gg==", 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.0", 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDVJypmG7TpDhAAo9CKuZBw8BsQgB2F+BR5gw==", 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.1", 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDYJypmgR62DBAAotfnvS47WDunP9Ic03gA6Q==", 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:01:12.500 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.501 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.501 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.2", 2024-04-25T10:01:12.501 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDbJypmzyqGChAA6wrfIdqpiLm3RO36oorihQ==", 2024-04-25T10:01:12.501 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.501 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:12.501 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:01:12.501 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.501 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.3", 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDeJypmt72cCBAAgcLrFC3MUU7wlFySAfLWig==", 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.4", 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDkJypmxJLpDBAAvROZ9AvYdWMqOALEKIAFCw==", 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.502 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.5", 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDnJypmTPXEChAA20BghY39PGw6zFyu9yWbBQ==", 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.6", 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDqJypmaBoYCRAAy9eHgUI/WTkE2XdT1RKLsw==", 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.7", 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDtJypmViJlCBAANJSDWVHVbWR03Y1SMDH++Q==", 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:01:12.504 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.0", 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg==", 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow r", 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.1", 2024-04-25T10:01:12.505 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypmOncYARAA6gYjZbRDkhqFX0mpSDbH5Q==", 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.2", 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmQs2wORAArzyV70nGDX4486LlFjM70Q==", 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T10:01:12.506 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.3", 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypm90x9BhAA41rZPs0O5mwmuHy1bkPCvw==", 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.admin", 2024-04-25T10:01:12.507 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmOWQcAxAA/ZVXEHYJQZf9UtgZ5g25Ag==", 2024-04-25T10:01:12.508 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.508 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T10:01:12.508 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T10:01:12.508 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T10:01:12.508 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.508 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.508 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.x", 2024-04-25T10:01:12.508 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmt+RVGxAAi1djVoqnoYCDMzJ9A0dmNA==", 2024-04-25T10:01:12.508 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.y", 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmuJ4bFxAAw6a53AgThU/ut+Mk5j0gqQ==", 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:12.509 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-04-25T10:01:12.510 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:01:12.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.494+0000 7faffa0fc640 1 -- 172.21.15.139:0/1874419703 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fafe006b4d0 msgr2=0x7fafe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.494+0000 7faffa0fc640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fafe006b4d0 0x7fafe006d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fafe8034000 tx=0x7fafe802e000 comp rx=0 tx=0).stop 2024-04-25T10:01:12.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.494+0000 7faffa0fc640 1 -- 172.21.15.139:0/1874419703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faff40d12c0 msgr2=0x7faff41f3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.494+0000 7faffa0fc640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faff40d12c0 0x7faff41f3f20 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c7f0 tx=0x7fafe400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.494+0000 7faffa0fc640 1 -- 172.21.15.139:0/1874419703 shutdown_connections 2024-04-25T10:01:12.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.494+0000 7faffa0fc640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faff40d12c0 0x7faff41f3f20 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.494+0000 7faffa0fc640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fafe006b4d0 0x7fafe006d990 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.494+0000 7faffa0fc640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff415e610 0x7faff40b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.494+0000 7faffa0fc640 1 --2- 172.21.15.139:0/1874419703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff40b0d00 0x7faff40b6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.494+0000 7faffa0fc640 1 -- 172.21.15.139:0/1874419703 >> 172.21.15.139:0/1874419703 conn(0x7faff40b3380 msgr2=0x7faff40b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.494+0000 7faffa0fc640 1 -- 172.21.15.139:0/1874419703 shutdown_connections 2024-04-25T10:01:12.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.494+0000 7faffa0fc640 1 -- 172.21.15.139:0/1874419703 wait complete. 2024-04-25T10:01:12.513 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:12.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.605+0000 7f4ae22f5640 1 Processor -- start 2024-04-25T10:01:12.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.605+0000 7f4ae22f5640 1 -- start start 2024-04-25T10:01:12.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.606+0000 7f4ae22f5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4adc06b270 0x7f4adc168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.606+0000 7f4ae22f5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc1625d0 con 0x7f4adc0afe40 2024-04-25T10:01:12.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.606+0000 7f4ae22f5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc161810 con 0x7f4adc15daa0 2024-04-25T10:01:12.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.606+0000 7f4ae22f5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4adc161af0 con 0x7f4adc06b270 2024-04-25T10:01:12.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.606+0000 7f4adb7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4adc0afe40 0x7f4adc06ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:47146/0 (socket says 172.21.15.139:47146) 2024-04-25T10:01:12.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.606+0000 7f4ae0892640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4adc06b270 0x7f4adc168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.606+0000 7f4adb7fe640 1 -- 172.21.15.139:0/3283600590 learned_addr learned my addr 172.21.15.139:0/3283600590 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:12.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.607+0000 7f4ae0892640 1 -- 172.21.15.139:0/3283600590 >> v1:172.21.15.150:6789/0 conn(0x7f4adc15daa0 legacy=0x7f4adc0af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.607+0000 7f4ae0892640 1 -- 172.21.15.139:0/3283600590 >> v1:172.21.15.139:6789/0 conn(0x7f4adc0afe40 legacy=0x7f4adc06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.607+0000 7f4ae0892640 1 -- 172.21.15.139:0/3283600590 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc161dd0 con 0x7f4adc06b270 2024-04-25T10:01:12.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.607+0000 7f4adaffd640 1 -- 172.21.15.139:0/3283600590 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203731942 0 0) 0x7f4adc1625d0 con 0x7f4adc0afe40 2024-04-25T10:01:12.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.607+0000 7f4ae0892640 1 --2- 172.21.15.139:0/3283600590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc06b270 0x7f4adc168a00 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009d90 tx=0x7f4ad402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c1f5c114d59571 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.608+0000 7f4adaffd640 1 -- 172.21.15.139:0/3283600590 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4007740 con 0x7f4adc06b270 2024-04-25T10:01:12.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.608+0000 7f4adaffd640 1 -- 172.21.15.139:0/3283600590 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad40078e0 con 0x7f4adc06b270 2024-04-25T10:01:12.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.608+0000 7f4adaffd640 1 -- 172.21.15.139:0/3283600590 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4007cb0 con 0x7f4adc06b270 2024-04-25T10:01:12.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.608+0000 7f4ae22f5640 1 -- 172.21.15.139:0/3283600590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc06b270 msgr2=0x7f4adc168a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.608+0000 7f4ae22f5640 1 --2- 172.21.15.139:0/3283600590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc06b270 0x7f4adc168a00 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009d90 tx=0x7f4ad402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.608+0000 7f4ae22f5640 1 -- 172.21.15.139:0/3283600590 shutdown_connections 2024-04-25T10:01:12.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.608+0000 7f4ae22f5640 1 --2- 172.21.15.139:0/3283600590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc06b270 0x7f4adc168a00 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.608+0000 7f4ae22f5640 1 -- 172.21.15.139:0/3283600590 >> 172.21.15.139:0/3283600590 conn(0x7f4adc06e280 msgr2=0x7f4adc06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.608+0000 7f4ae22f5640 1 -- 172.21.15.139:0/3283600590 shutdown_connections 2024-04-25T10:01:12.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.608+0000 7f4ae22f5640 1 -- 172.21.15.139:0/3283600590 wait complete. 2024-04-25T10:01:12.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.609+0000 7f4ae22f5640 1 Processor -- start 2024-04-25T10:01:12.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.609+0000 7f4ae22f5640 1 -- start start 2024-04-25T10:01:12.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.610+0000 7f4ae22f5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4adc06b270 0x7f4adc14fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.610+0000 7f4ae22f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc0afe40 0x7f4adc14d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.610+0000 7f4ae22f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4adc15daa0 0x7f4adc153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.610+0000 7f4ae22f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0b4690 con 0x7f4adc0afe40 2024-04-25T10:01:12.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.610+0000 7f4ae22f5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc15de80 con 0x7f4adc06b270 2024-04-25T10:01:12.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.610+0000 7f4ae22f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc168f40 con 0x7f4adc15daa0 2024-04-25T10:01:12.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.610+0000 7f4adb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc0afe40 0x7f4adc14d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.610+0000 7f4adb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc0afe40 0x7f4adc14d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43340/0 (socket says 172.21.15.139:43340) 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.610+0000 7f4adb7fe640 1 -- 172.21.15.139:0/2660365708 learned_addr learned my addr 172.21.15.139:0/2660365708 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.610+0000 7f4adbfff640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4adc06b270 0x7f4adc14fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.610+0000 7f4ae0892640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4adc15daa0 0x7f4adc153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.611+0000 7f4adb7fe640 1 -- 172.21.15.139:0/2660365708 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4adc15daa0 msgr2=0x7f4adc153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.611+0000 7f4adb7fe640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4adc15daa0 0x7f4adc153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.611+0000 7f4adb7fe640 1 -- 172.21.15.139:0/2660365708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4adc06b270 msgr2=0x7f4adc14fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.611+0000 7f4adb7fe640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4adc06b270 0x7f4adc14fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.611+0000 7f4adb7fe640 1 -- 172.21.15.139:0/2660365708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc15e000 con 0x7f4adc0afe40 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.611+0000 7f4adbfff640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4adc06b270 0x7f4adc14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.611+0000 7f4adb7fe640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc0afe40 0x7f4adc14d3e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c4a0 tx=0x7f4ad000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.611+0000 7f4ad8ff9640 1 -- 172.21.15.139:0/2660365708 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0018070 con 0x7f4adc0afe40 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.611+0000 7f4ad8ff9640 1 -- 172.21.15.139:0/2660365708 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad00074e0 con 0x7f4adc0afe40 2024-04-25T10:01:12.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.611+0000 7f4ad8ff9640 1 -- 172.21.15.139:0/2660365708 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0004a20 con 0x7f4adc0afe40 2024-04-25T10:01:12.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.611+0000 7f4ae22f5640 1 -- 172.21.15.139:0/2660365708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4adc15e2e0 con 0x7f4adc0afe40 2024-04-25T10:01:12.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.611+0000 7f4ae22f5640 1 -- 172.21.15.139:0/2660365708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4adc15e8a0 con 0x7f4adc0afe40 2024-04-25T10:01:12.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.612+0000 7f4ae22f5640 1 -- 172.21.15.139:0/2660365708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4adc0b6970 con 0x7f4adc0afe40 2024-04-25T10:01:12.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.612+0000 7f4ad8ff9640 1 -- 172.21.15.139:0/2660365708 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ad001b070 con 0x7f4adc0afe40 2024-04-25T10:01:12.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.613+0000 7f4ad8ff9640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ac806b4d0 0x7f4ac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.613+0000 7f4adbfff640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ac806b4d0 0x7f4ac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.613+0000 7f4ad8ff9640 1 -- 172.21.15.139:0/2660365708 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4ad00892c0 con 0x7f4adc0afe40 2024-04-25T10:01:12.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.613+0000 7f4adbfff640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ac806b4d0 0x7f4ac806d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4adc153440 tx=0x7f4acc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.617+0000 7f4ad8ff9640 1 -- 172.21.15.139:0/2660365708 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4adc0b6970 con 0x7f4adc0afe40 2024-04-25T10:01:12.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.788+0000 7f4ae22f5640 1 -- 172.21.15.139:0/2660365708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4adc168d20 con 0x7f4adc0afe40 2024-04-25T10:01:12.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.788+0000 7f4ad8ff9640 1 -- 172.21.15.139:0/2660365708 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+13214 (secure 0 0 0) 0x7f4ad0058390 con 0x7f4adc0afe40 2024-04-25T10:01:12.793 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:12.793 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":101,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:01:12.122704+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:12.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.790+0000 7f4ae22f5640 1 -- 172.21.15.139:0/2660365708 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ac806b4d0 msgr2=0x7f4ac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.790+0000 7f4ae22f5640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ac806b4d0 0x7f4ac806d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4adc153440 tx=0x7f4acc009040 comp rx=0 tx=0).stop 2024-04-25T10:01:12.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.790+0000 7f4ae22f5640 1 -- 172.21.15.139:0/2660365708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc0afe40 msgr2=0x7f4adc14d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.790+0000 7f4ae22f5640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc0afe40 0x7f4adc14d3e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c4a0 tx=0x7f4ad000c960 comp rx=0 tx=0).stop 2024-04-25T10:01:12.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.790+0000 7f4ae22f5640 1 -- 172.21.15.139:0/2660365708 shutdown_connections 2024-04-25T10:01:12.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.790+0000 7f4ae22f5640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc0afe40 0x7f4adc14d3e0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.790+0000 7f4ae22f5640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4ac806b4d0 0x7f4ac806d990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.790+0000 7f4ae22f5640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4adc15daa0 0x7f4adc153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.790+0000 7f4ae22f5640 1 --2- 172.21.15.139:0/2660365708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4adc06b270 0x7f4adc14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.791+0000 7f4ae22f5640 1 -- 172.21.15.139:0/2660365708 >> 172.21.15.139:0/2660365708 conn(0x7f4adc06e280 msgr2=0x7f4adc06fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.791+0000 7f4ae22f5640 1 -- 172.21.15.139:0/2660365708 shutdown_connections 2024-04-25T10:01:12.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.791+0000 7f4ae22f5640 1 -- 172.21.15.139:0/2660365708 wait complete. 2024-04-25T10:01:12.806 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:12.807 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:12.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.902+0000 7f800c74c640 1 Processor -- start 2024-04-25T10:01:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.902+0000 7f800c74c640 1 -- start start 2024-04-25T10:01:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f800c74c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040b31a0 0x7f800406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f800c74c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80040b2a50 con 0x7f8004144090 2024-04-25T10:01:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f800c74c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80040b35e0 con 0x7f800406e870 2024-04-25T10:01:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f800c74c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80040cd0f0 con 0x7f80040b31a0 2024-04-25T10:01:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f800a4e8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8004144090 0x7f80040cc9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:47150/0 (socket says 172.21.15.139:47150) 2024-04-25T10:01:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f800a4e8640 1 -- 172.21.15.139:0/2665196891 learned_addr learned my addr 172.21.15.139:0/2665196891 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f8009ce7640 1 --2- 172.21.15.139:0/2665196891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040b31a0 0x7f800406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3068384818 0 0) 0x7f8000001110 con 0x7f8004144090 2024-04-25T10:01:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ff0003520 con 0x7f8004144090 2024-04-25T10:01:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979548284 0 0) 0x7f7ffc001490 con 0x7f800406e870 2024-04-25T10:01:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8000001110 con 0x7f800406e870 2024-04-25T10:01:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3380379561 0 0) 0x7f7ff0003520 con 0x7f8004144090 2024-04-25T10:01:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040b31a0 msgr2=0x7f800406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 --2- 172.21.15.139:0/2665196891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040b31a0 0x7f800406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 >> v1:172.21.15.150:6789/0 conn(0x7f800406e870 legacy=0x7f800415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800415b090 con 0x7f8004144090 2024-04-25T10:01:12.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f8000001840 con 0x7f8004144090 2024-04-25T10:01:12.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 >> v1:172.21.15.139:6789/0 conn(0x7f8004144090 legacy=0x7f80040cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 --2- 172.21.15.139:0/2665196891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040b31a0 0x7f7ff0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 --2- 172.21.15.139:0/2665196891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ff00040d0 0x7f7ff0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 --2- 172.21.15.139:0/2665196891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff0004d40 0x7f7ff00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80040cd0f0 con 0x7f7ff0004d40 2024-04-25T10:01:12.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff000bb50 con 0x7f7ff00040d0 2024-04-25T10:01:12.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff000bd20 con 0x7f80040b31a0 2024-04-25T10:01:12.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.903+0000 7f800ace9640 1 --2- 172.21.15.139:0/2665196891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff0004d40 0x7f7ff00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.904+0000 7f800a4e8640 1 --2- 172.21.15.139:0/2665196891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040b31a0 0x7f7ff0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.904+0000 7f8009ce7640 1 --2- 172.21.15.139:0/2665196891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ff00040d0 0x7f7ff0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.904+0000 7f800ace9640 1 -- 172.21.15.139:0/2665196891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040b31a0 msgr2=0x7f7ff0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:12.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.904+0000 7f800ace9640 1 --2- 172.21.15.139:0/2665196891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040b31a0 0x7f7ff0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.904+0000 7f800ace9640 1 -- 172.21.15.139:0/2665196891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ff00040d0 msgr2=0x7f7ff0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:12.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.904+0000 7f800ace9640 1 --2- 172.21.15.139:0/2665196891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ff00040d0 0x7f7ff0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.904+0000 7f800ace9640 1 -- 172.21.15.139:0/2665196891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff0003520 con 0x7f7ff0004d40 2024-04-25T10:01:12.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.904+0000 7f800ace9640 1 --2- 172.21.15.139:0/2665196891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff0004d40 0x7f7ff00051c0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0021a0 tx=0x7f7ff0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79366ec1e46b0ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.905+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc001dc0 con 0x7f7ff0004d40 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.905+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc003b70 con 0x7f7ff0004d40 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.905+0000 7f80094e6640 1 -- 172.21.15.139:0/2665196891 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0050d0 con 0x7f7ff0004d40 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.905+0000 7f800c74c640 1 -- 172.21.15.139:0/2665196891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff0004d40 msgr2=0x7f7ff00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.905+0000 7f800c74c640 1 --2- 172.21.15.139:0/2665196891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff0004d40 0x7f7ff00051c0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0021a0 tx=0x7f7ff0005830 comp rx=0 tx=0).stop 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.905+0000 7f800ace9640 1 -- 172.21.15.139:0/2665196891 reap_dead start 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.905+0000 7f800c74c640 1 -- 172.21.15.139:0/2665196891 shutdown_connections 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.905+0000 7f800c74c640 1 -- 172.21.15.139:0/2665196891 >> 172.21.15.139:0/2665196891 conn(0x7f8004067000 msgr2=0x7f800406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.905+0000 7f800c74c640 1 -- 172.21.15.139:0/2665196891 shutdown_connections 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.905+0000 7f800c74c640 1 -- 172.21.15.139:0/2665196891 wait complete. 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.906+0000 7f800c74c640 1 Processor -- start 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.906+0000 7f800c74c640 1 -- start start 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800c74c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff0004d40 0x7f8004159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800c74c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8004139930 0x7f80040b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800c74c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8004139db0 0x7f80040b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800c74c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800415b930 con 0x7f7ff0004d40 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800c74c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800415b650 con 0x7f8004139db0 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800c74c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800415bc10 con 0x7f8004139930 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800a4e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff0004d40 0x7f8004159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800a4e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff0004d40 0x7f8004159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43366/0 (socket says 172.21.15.139:43366) 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800a4e8640 1 -- 172.21.15.139:0/103512219 learned_addr learned my addr 172.21.15.139:0/103512219 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800ace9640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8004139db0 0x7f80040b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f8009ce7640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8004139930 0x7f80040b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800a4e8640 1 -- 172.21.15.139:0/103512219 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8004139930 msgr2=0x7f80040b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800a4e8640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8004139930 0x7f80040b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800a4e8640 1 -- 172.21.15.139:0/103512219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8004139db0 msgr2=0x7f80040b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800a4e8640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8004139db0 0x7f80040b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.907+0000 7f800a4e8640 1 -- 172.21.15.139:0/103512219 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800413a2f0 con 0x7f7ff0004d40 2024-04-25T10:01:12.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.908+0000 7f800a4e8640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff0004d40 0x7f8004159a60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f8000004fd0 tx=0x7f80000077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.908+0000 7f7ffaffd640 1 -- 172.21.15.139:0/103512219 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800000f990 con 0x7f7ff0004d40 2024-04-25T10:01:12.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.908+0000 7f7ffaffd640 1 -- 172.21.15.139:0/103512219 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8000018070 con 0x7f7ff0004d40 2024-04-25T10:01:12.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.908+0000 7f800c74c640 1 -- 172.21.15.139:0/103512219 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800413a5d0 con 0x7f7ff0004d40 2024-04-25T10:01:12.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.908+0000 7f7ffaffd640 1 -- 172.21.15.139:0/103512219 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800000fd30 con 0x7f7ff0004d40 2024-04-25T10:01:12.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.908+0000 7f800c74c640 1 -- 172.21.15.139:0/103512219 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800413ab90 con 0x7f7ff0004d40 2024-04-25T10:01:12.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.909+0000 7f800c74c640 1 -- 172.21.15.139:0/103512219 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8004071ff0 con 0x7f7ff0004d40 2024-04-25T10:01:12.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.909+0000 7f7ffaffd640 1 -- 172.21.15.139:0/103512219 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f800001d460 con 0x7f7ff0004d40 2024-04-25T10:01:12.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.909+0000 7f7ffaffd640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7fe806b500 0x7f7fe806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.910+0000 7f7ffaffd640 1 -- 172.21.15.139:0/103512219 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f800008a2a0 con 0x7f7ff0004d40 2024-04-25T10:01:12.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.910+0000 7f8009ce7640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7fe806b500 0x7f7fe806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.910+0000 7f8009ce7640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7fe806b500 0x7f7fe806d9c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f800406e530 tx=0x7f7ff4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:12.914+0000 7f7ffaffd640 1 -- 172.21.15.139:0/103512219 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8004071ff0 con 0x7f7ff0004d40 2024-04-25T10:01:13.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.086+0000 7f800c74c640 1 -- 172.21.15.139:0/103512219 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8004074520 con 0x7f7ff0004d40 2024-04-25T10:01:13.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.087+0000 7f7ffaffd640 1 -- 172.21.15.139:0/103512219 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80040b2a50 con 0x7f7ff0004d40 2024-04-25T10:01:13.091 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:01:13.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.089+0000 7f800c74c640 1 -- 172.21.15.139:0/103512219 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7fe806b500 msgr2=0x7f7fe806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.089+0000 7f800c74c640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7fe806b500 0x7f7fe806d9c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f800406e530 tx=0x7f7ff4008040 comp rx=0 tx=0).stop 2024-04-25T10:01:13.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.089+0000 7f800c74c640 1 -- 172.21.15.139:0/103512219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff0004d40 msgr2=0x7f8004159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.089+0000 7f800c74c640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff0004d40 0x7f8004159a60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f8000004fd0 tx=0x7f80000077a0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.089+0000 7f800c74c640 1 -- 172.21.15.139:0/103512219 shutdown_connections 2024-04-25T10:01:13.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.089+0000 7f800c74c640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7fe806b500 0x7f7fe806d9c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.089+0000 7f800c74c640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8004139db0 0x7f80040b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.089+0000 7f800c74c640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8004139930 0x7f80040b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.089+0000 7f800c74c640 1 --2- 172.21.15.139:0/103512219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff0004d40 0x7f8004159a60 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.089+0000 7f800c74c640 1 -- 172.21.15.139:0/103512219 >> 172.21.15.139:0/103512219 conn(0x7f8004067000 msgr2=0x7f800415d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.089+0000 7f800c74c640 1 -- 172.21.15.139:0/103512219 shutdown_connections 2024-04-25T10:01:13.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.089+0000 7f800c74c640 1 -- 172.21.15.139:0/103512219 wait complete. 2024-04-25T10:01:13.104 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:01:13.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.197+0000 7f125c24a640 1 Processor -- start 2024-04-25T10:01:13.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.197+0000 7f125c24a640 1 -- start start 2024-04-25T10:01:13.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.198+0000 7f125c24a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12541611e0 0x7f12540b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.198+0000 7f125c24a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f125406c780 0x7f12540d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.198+0000 7f125c24a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12540d4740 con 0x7f12540cea00 2024-04-25T10:01:13.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.198+0000 7f125c24a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12540d1fa0 con 0x7f125406c780 2024-04-25T10:01:13.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.198+0000 7f125c24a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12540d2560 con 0x7f12541611e0 2024-04-25T10:01:13.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.198+0000 7f125a7e7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f125406c780 0x7f12540d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.198+0000 7f1259fe6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12541611e0 0x7f12540b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.198+0000 7f125a7e7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f125406c780 0x7f12540d29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:54288/0 (socket says 172.21.15.139:54288) 2024-04-25T10:01:13.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.198+0000 7f12597e5640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f12540cea00 0x7f125406c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:42644/0 (socket says 172.21.15.139:42644) 2024-04-25T10:01:13.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.198+0000 7f125a7e7640 1 -- 172.21.15.139:0/4121789277 learned_addr learned my addr 172.21.15.139:0/4121789277 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:13.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.199+0000 7f125a7e7640 1 -- 172.21.15.139:0/4121789277 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12541611e0 msgr2=0x7f12540b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.199+0000 7f125a7e7640 1 --2- 172.21.15.139:0/4121789277 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12541611e0 0x7f12540b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.199+0000 7f125a7e7640 1 -- 172.21.15.139:0/4121789277 >> v1:172.21.15.150:6789/0 conn(0x7f12540cea00 legacy=0x7f125406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.199+0000 7f125a7e7640 1 -- 172.21.15.139:0/4121789277 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12540b2770 con 0x7f125406c780 2024-04-25T10:01:13.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.199+0000 7f125a7e7640 1 --2- 172.21.15.139:0/4121789277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f125406c780 0x7f12540d29f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f124c009e50 tx=0x7f124c02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6272760b077f6d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.199+0000 7f1258fe4640 1 -- 172.21.15.139:0/4121789277 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c036070 con 0x7f125406c780 2024-04-25T10:01:13.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.200+0000 7f1258fe4640 1 -- 172.21.15.139:0/4121789277 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c031600 con 0x7f125406c780 2024-04-25T10:01:13.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.200+0000 7f1258fe4640 1 -- 172.21.15.139:0/4121789277 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c0319d0 con 0x7f125406c780 2024-04-25T10:01:13.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.200+0000 7f125c24a640 1 -- 172.21.15.139:0/4121789277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f125406c780 msgr2=0x7f12540d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.200+0000 7f125c24a640 1 --2- 172.21.15.139:0/4121789277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f125406c780 0x7f12540d29f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f124c009e50 tx=0x7f124c02a710 comp rx=0 tx=0).stop 2024-04-25T10:01:13.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.200+0000 7f125c24a640 1 -- 172.21.15.139:0/4121789277 shutdown_connections 2024-04-25T10:01:13.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.200+0000 7f125c24a640 1 --2- 172.21.15.139:0/4121789277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f125406c780 0x7f12540d29f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.200+0000 7f125c24a640 1 --2- 172.21.15.139:0/4121789277 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12541611e0 0x7f12540b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.200+0000 7f125c24a640 1 -- 172.21.15.139:0/4121789277 >> 172.21.15.139:0/4121789277 conn(0x7f12540b5b00 msgr2=0x7f12540d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.200+0000 7f125c24a640 1 -- 172.21.15.139:0/4121789277 shutdown_connections 2024-04-25T10:01:13.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.200+0000 7f125c24a640 1 -- 172.21.15.139:0/4121789277 wait complete. 2024-04-25T10:01:13.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.201+0000 7f125c24a640 1 Processor -- start 2024-04-25T10:01:13.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.201+0000 7f125c24a640 1 -- start start 2024-04-25T10:01:13.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.202+0000 7f125c24a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f125406c780 0x7f125415fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.202+0000 7f125c24a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12540cea00 0x7f125415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.202+0000 7f125c24a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12541611e0 0x7f12541f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.202+0000 7f125c24a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12540d21f0 con 0x7f12541611e0 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.202+0000 7f125c24a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12540b2d30 con 0x7f12540cea00 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.202+0000 7f125c24a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12541615c0 con 0x7f125406c780 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.202+0000 7f125a7e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12541611e0 0x7f12541f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.202+0000 7f125a7e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12541611e0 0x7f12541f4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43372/0 (socket says 172.21.15.139:43372) 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.202+0000 7f125a7e7640 1 -- 172.21.15.139:0/2515397795 learned_addr learned my addr 172.21.15.139:0/2515397795 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.202+0000 7f1259fe6640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f125406c780 0x7f125415fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.202+0000 7f12597e5640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12540cea00 0x7f125415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.203+0000 7f125a7e7640 1 -- 172.21.15.139:0/2515397795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f125406c780 msgr2=0x7f125415fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.203+0000 7f125a7e7640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f125406c780 0x7f125415fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.203+0000 7f125a7e7640 1 -- 172.21.15.139:0/2515397795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12540cea00 msgr2=0x7f125415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.203+0000 7f125a7e7640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12540cea00 0x7f125415af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.203+0000 7f125a7e7640 1 -- 172.21.15.139:0/2515397795 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12541f51b0 con 0x7f12541611e0 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.203+0000 7f125a7e7640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12541611e0 0x7f12541f4a60 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f124c009e50 tx=0x7f124c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.203+0000 7f124a7fc640 1 -- 172.21.15.139:0/2515397795 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c036050 con 0x7f12541611e0 2024-04-25T10:01:13.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.203+0000 7f124a7fc640 1 -- 172.21.15.139:0/2515397795 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c042070 con 0x7f12541611e0 2024-04-25T10:01:13.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.203+0000 7f125c24a640 1 -- 172.21.15.139:0/2515397795 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12541f5490 con 0x7f12541611e0 2024-04-25T10:01:13.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.203+0000 7f124a7fc640 1 -- 172.21.15.139:0/2515397795 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c009510 con 0x7f12541611e0 2024-04-25T10:01:13.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.203+0000 7f125c24a640 1 -- 172.21.15.139:0/2515397795 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12541f5a50 con 0x7f12541611e0 2024-04-25T10:01:13.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.204+0000 7f125c24a640 1 -- 172.21.15.139:0/2515397795 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1254074520 con 0x7f12541611e0 2024-04-25T10:01:13.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.204+0000 7f124a7fc640 1 -- 172.21.15.139:0/2515397795 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f124c007bf0 con 0x7f12541611e0 2024-04-25T10:01:13.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.205+0000 7f124a7fc640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f123806b4d0 0x7f123806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.205+0000 7f124a7fc640 1 -- 172.21.15.139:0/2515397795 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f124c0aace0 con 0x7f12541611e0 2024-04-25T10:01:13.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.205+0000 7f1259fe6640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f123806b4d0 0x7f123806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.205+0000 7f1259fe6640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f123806b4d0 0x7f123806d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f12540cf370 tx=0x7f1250009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.209+0000 7f124a7fc640 1 -- 172.21.15.139:0/2515397795 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f124c0796d0 con 0x7f12541611e0 2024-04-25T10:01:13.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.379+0000 7f125c24a640 1 -- 172.21.15.139:0/2515397795 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f125406ceb0 con 0x7f12541611e0 2024-04-25T10:01:13.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.380+0000 7f124a7fc640 1 -- 172.21.15.139:0/2515397795 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) v1 ==== 84+0+65 (secure 0 0 0) 0x7f124c07d770 con 0x7f12541611e0 2024-04-25T10:01:13.384 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:13.384 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-04-25T10:01:13.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:01:13.384 INFO:teuthology.orchestra.run.smithi139.stdout: "poolnum": 1, 2024-04-25T10:01:13.384 INFO:teuthology.orchestra.run.smithi139.stdout: "poolname": ".mgr" 2024-04-25T10:01:13.384 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:13.384 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-04-25T10:01:13.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.382+0000 7f125c24a640 1 -- 172.21.15.139:0/2515397795 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f123806b4d0 msgr2=0x7f123806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.382+0000 7f125c24a640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f123806b4d0 0x7f123806d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f12540cf370 tx=0x7f1250009190 comp rx=0 tx=0).stop 2024-04-25T10:01:13.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.382+0000 7f125c24a640 1 -- 172.21.15.139:0/2515397795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12541611e0 msgr2=0x7f12541f4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.382+0000 7f125c24a640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12541611e0 0x7f12541f4a60 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f124c009e50 tx=0x7f124c004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:13.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.382+0000 7f125c24a640 1 -- 172.21.15.139:0/2515397795 shutdown_connections 2024-04-25T10:01:13.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.382+0000 7f125c24a640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12541611e0 0x7f12541f4a60 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.382+0000 7f125c24a640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f123806b4d0 0x7f123806d990 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.382+0000 7f125c24a640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12540cea00 0x7f125415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.382+0000 7f125c24a640 1 --2- 172.21.15.139:0/2515397795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f125406c780 0x7f125415fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.382+0000 7f125c24a640 1 -- 172.21.15.139:0/2515397795 >> 172.21.15.139:0/2515397795 conn(0x7f12540b5b00 msgr2=0x7f1254161f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.382+0000 7f125c24a640 1 -- 172.21.15.139:0/2515397795 shutdown_connections 2024-04-25T10:01:13.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.382+0000 7f125c24a640 1 -- 172.21.15.139:0/2515397795 wait complete. 2024-04-25T10:01:13.397 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:01:13.397 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:01:13.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.495+0000 7f909a06b640 1 Processor -- start 2024-04-25T10:01:13.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.495+0000 7f909a06b640 1 -- start start 2024-04-25T10:01:13.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.496+0000 7f909a06b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9094161fd0 0x7f909415d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.496+0000 7f909a06b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9094099ca0 con 0x7f90940d0a80 2024-04-25T10:01:13.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.496+0000 7f909a06b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9094099f80 con 0x7f90940cd610 2024-04-25T10:01:13.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.496+0000 7f909a06b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9094099a50 con 0x7f9094161fd0 2024-04-25T10:01:13.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.496+0000 7f9093fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9094161fd0 0x7f909415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.496+0000 7f9092ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f90940d0a80 0x7f90941618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:47162/0 (socket says 172.21.15.139:47162) 2024-04-25T10:01:13.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.496+0000 7f9093fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9094161fd0 0x7f909415d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43380/0 (socket says 172.21.15.139:43380) 2024-04-25T10:01:13.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.496+0000 7f9092ffd640 1 -- 172.21.15.139:0/2653506633 learned_addr learned my addr 172.21.15.139:0/2653506633 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:13.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.496+0000 7f9093fff640 1 -- 172.21.15.139:0/2653506633 >> v1:172.21.15.150:6789/0 conn(0x7f90940cd610 legacy=0x7f90940d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.496+0000 7f90927fc640 1 -- 172.21.15.139:0/2653506633 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973638023 0 0) 0x7f9094099ca0 con 0x7f90940d0a80 2024-04-25T10:01:13.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.496+0000 7f9093fff640 1 -- 172.21.15.139:0/2653506633 >> v1:172.21.15.139:6789/0 conn(0x7f90940d0a80 legacy=0x7f90941618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.496+0000 7f9093fff640 1 -- 172.21.15.139:0/2653506633 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909409a260 con 0x7f9094161fd0 2024-04-25T10:01:13.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.497+0000 7f9093fff640 1 --2- 172.21.15.139:0/2653506633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9094161fd0 0x7f909415d370 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f908c009d90 tx=0x7f908c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff1923ff9aecec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.497+0000 7f90927fc640 1 -- 172.21.15.139:0/2653506633 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c007740 con 0x7f9094161fd0 2024-04-25T10:01:13.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.497+0000 7f90927fc640 1 -- 172.21.15.139:0/2653506633 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c0078e0 con 0x7f9094161fd0 2024-04-25T10:01:13.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.497+0000 7f90927fc640 1 -- 172.21.15.139:0/2653506633 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c007cb0 con 0x7f9094161fd0 2024-04-25T10:01:13.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.498+0000 7f909a06b640 1 -- 172.21.15.139:0/2653506633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9094161fd0 msgr2=0x7f909415d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.498+0000 7f909a06b640 1 --2- 172.21.15.139:0/2653506633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9094161fd0 0x7f909415d370 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f908c009d90 tx=0x7f908c02a730 comp rx=0 tx=0).stop 2024-04-25T10:01:13.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.498+0000 7f909a06b640 1 -- 172.21.15.139:0/2653506633 shutdown_connections 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.498+0000 7f909a06b640 1 --2- 172.21.15.139:0/2653506633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9094161fd0 0x7f909415d370 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.498+0000 7f909a06b640 1 -- 172.21.15.139:0/2653506633 >> 172.21.15.139:0/2653506633 conn(0x7f90940b31f0 msgr2=0x7f90940b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.498+0000 7f909a06b640 1 -- 172.21.15.139:0/2653506633 shutdown_connections 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.498+0000 7f909a06b640 1 -- 172.21.15.139:0/2653506633 wait complete. 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.499+0000 7f909a06b640 1 Processor -- start 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.499+0000 7f909a06b640 1 -- start start 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f909a06b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90940cd610 0x7f90940b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f909a06b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90940d0a80 0x7f90940b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f909a06b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9094161fd0 0x7f90940b6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f909a06b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90940cd9f0 con 0x7f90940cd610 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f909a06b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909409a820 con 0x7f90940d0a80 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f909a06b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90940cdb70 con 0x7f9094161fd0 2024-04-25T10:01:13.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f90937fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90940cd610 0x7f90940b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f90937fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90940cd610 0x7f90940b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43386/0 (socket says 172.21.15.139:43386) 2024-04-25T10:01:13.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f90937fe640 1 -- 172.21.15.139:0/3905348411 learned_addr learned my addr 172.21.15.139:0/3905348411 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:13.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f9092ffd640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90940d0a80 0x7f90940b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f9093fff640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9094161fd0 0x7f90940b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f90937fe640 1 -- 172.21.15.139:0/3905348411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9094161fd0 msgr2=0x7f90940b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f90937fe640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9094161fd0 0x7f90940b6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f90937fe640 1 -- 172.21.15.139:0/3905348411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90940d0a80 msgr2=0x7f90940b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f90937fe640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90940d0a80 0x7f90940b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f90937fe640 1 -- 172.21.15.139:0/3905348411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90940b5c10 con 0x7f90940cd610 2024-04-25T10:01:13.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f9092ffd640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90940d0a80 0x7f90940b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:13.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f90937fe640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90940cd610 0x7f90940b4190 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f908400c430 tx=0x7f908400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f907bfff640 1 -- 172.21.15.139:0/3905348411 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084011790 con 0x7f90940cd610 2024-04-25T10:01:13.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f907bfff640 1 -- 172.21.15.139:0/3905348411 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9084017070 con 0x7f90940cd610 2024-04-25T10:01:13.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f907bfff640 1 -- 172.21.15.139:0/3905348411 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084011bb0 con 0x7f90940cd610 2024-04-25T10:01:13.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f909a06b640 1 -- 172.21.15.139:0/3905348411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90940b4b70 con 0x7f90940cd610 2024-04-25T10:01:13.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.500+0000 7f909a06b640 1 -- 172.21.15.139:0/3905348411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90940b5130 con 0x7f90940cd610 2024-04-25T10:01:13.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.501+0000 7f909a06b640 1 -- 172.21.15.139:0/3905348411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9094074520 con 0x7f90940cd610 2024-04-25T10:01:13.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.502+0000 7f907bfff640 1 -- 172.21.15.139:0/3905348411 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f908401c460 con 0x7f90940cd610 2024-04-25T10:01:13.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.502+0000 7f907bfff640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f908006b4d0 0x7f908006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.503+0000 7f9092ffd640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f908006b4d0 0x7f908006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.503+0000 7f907bfff640 1 -- 172.21.15.139:0/3905348411 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9084089f00 con 0x7f90940cd610 2024-04-25T10:01:13.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.503+0000 7f9092ffd640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f908006b4d0 0x7f908006d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f90940d0f80 tx=0x7f9088009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.507+0000 7f907bfff640 1 -- 172.21.15.139:0/3905348411 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90940b5c10 con 0x7f90940cd610 2024-04-25T10:01:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:13.676+0000 7f909a06b640 1 -- 172.21.15.139:0/3905348411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f90940b3f70 con 0x7f90940cd610 2024-04-25T10:01:14.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.133+0000 7f907bfff640 1 -- 172.21.15.139:0/3905348411 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9084058f40 con 0x7f90940cd610 2024-04-25T10:01:14.138 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' created 2024-04-25T10:01:14.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.136+0000 7f909a06b640 1 -- 172.21.15.139:0/3905348411 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f908006b4d0 msgr2=0x7f908006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.136+0000 7f909a06b640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f908006b4d0 0x7f908006d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f90940d0f80 tx=0x7f9088009040 comp rx=0 tx=0).stop 2024-04-25T10:01:14.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.136+0000 7f909a06b640 1 -- 172.21.15.139:0/3905348411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90940cd610 msgr2=0x7f90940b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.136+0000 7f909a06b640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90940cd610 0x7f90940b4190 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f908400c430 tx=0x7f908400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.136+0000 7f909a06b640 1 -- 172.21.15.139:0/3905348411 shutdown_connections 2024-04-25T10:01:14.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.136+0000 7f909a06b640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f908006b4d0 0x7f908006d990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.136+0000 7f909a06b640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9094161fd0 0x7f90940b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.136+0000 7f909a06b640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90940d0a80 0x7f90940b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.136+0000 7f909a06b640 1 --2- 172.21.15.139:0/3905348411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90940cd610 0x7f90940b4190 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.136+0000 7f909a06b640 1 -- 172.21.15.139:0/3905348411 >> 172.21.15.139:0/3905348411 conn(0x7f90940b31f0 msgr2=0x7f90940ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:14.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.136+0000 7f909a06b640 1 -- 172.21.15.139:0/3905348411 shutdown_connections 2024-04-25T10:01:14.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.136+0000 7f909a06b640 1 -- 172.21.15.139:0/3905348411 wait complete. 2024-04-25T10:01:14.155 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:01:14.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.256+0000 7f4057ce1640 1 Processor -- start 2024-04-25T10:01:14.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.256+0000 7f4057ce1640 1 -- start start 2024-04-25T10:01:14.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.257+0000 7f4057ce1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f40500711c0 0x7f4050071650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.257+0000 7f4057ce1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f405015da30 0x7f405015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.257+0000 7f4057ce1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f405015c960 con 0x7f4050145230 2024-04-25T10:01:14.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.257+0000 7f4057ce1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405015cc40 con 0x7f40500711c0 2024-04-25T10:01:14.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.257+0000 7f4057ce1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405015d230 con 0x7f405015da30 2024-04-25T10:01:14.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.257+0000 7f405527c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f40500711c0 0x7f4050071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:14.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.257+0000 7f405627e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f405015da30 0x7f405015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:14.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.257+0000 7f4055a7d640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f4050145230 0x7f405015be90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:42654/0 (socket says 172.21.15.139:42654) 2024-04-25T10:01:14.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.257+0000 7f4055a7d640 1 -- 172.21.15.139:0/369148991 learned_addr learned my addr 172.21.15.139:0/369148991 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:14.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.258+0000 7f405527c640 1 -- 172.21.15.139:0/369148991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f405015da30 msgr2=0x7f405015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:14.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.258+0000 7f405527c640 1 --2- 172.21.15.139:0/369148991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f405015da30 0x7f405015de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.258+0000 7f405527c640 1 -- 172.21.15.139:0/369148991 >> v1:172.21.15.150:6789/0 conn(0x7f4050145230 legacy=0x7f405015be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.258+0000 7f405527c640 1 -- 172.21.15.139:0/369148991 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405015d510 con 0x7f40500711c0 2024-04-25T10:01:14.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.258+0000 7f4054a7b640 1 -- 172.21.15.139:0/369148991 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314612347 0 0) 0x7f405015c960 con 0x7f4050145230 2024-04-25T10:01:14.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.259+0000 7f405527c640 1 --2- 172.21.15.139:0/369148991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40500711c0 0x7f4050071650 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f40400097a0 tx=0x7f404002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9f6cd79ac7ff146 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:14.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.259+0000 7f4054a7b640 1 -- 172.21.15.139:0/369148991 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040037070 con 0x7f40500711c0 2024-04-25T10:01:14.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.259+0000 7f4054a7b640 1 -- 172.21.15.139:0/369148991 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040032440 con 0x7f40500711c0 2024-04-25T10:01:14.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.259+0000 7f4054a7b640 1 -- 172.21.15.139:0/369148991 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040032810 con 0x7f40500711c0 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.259+0000 7f4057ce1640 1 -- 172.21.15.139:0/369148991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40500711c0 msgr2=0x7f4050071650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.259+0000 7f4057ce1640 1 --2- 172.21.15.139:0/369148991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40500711c0 0x7f4050071650 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f40400097a0 tx=0x7f404002a180 comp rx=0 tx=0).stop 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.259+0000 7f4057ce1640 1 -- 172.21.15.139:0/369148991 shutdown_connections 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.259+0000 7f4057ce1640 1 --2- 172.21.15.139:0/369148991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f405015da30 0x7f405015de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.259+0000 7f4057ce1640 1 --2- 172.21.15.139:0/369148991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40500711c0 0x7f4050071650 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.259+0000 7f4057ce1640 1 -- 172.21.15.139:0/369148991 >> 172.21.15.139:0/369148991 conn(0x7f4050067000 msgr2=0x7f405006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.259+0000 7f4057ce1640 1 -- 172.21.15.139:0/369148991 shutdown_connections 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.259+0000 7f4057ce1640 1 -- 172.21.15.139:0/369148991 wait complete. 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.260+0000 7f4057ce1640 1 Processor -- start 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.260+0000 7f4057ce1640 1 -- start start 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.261+0000 7f4057ce1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40500711c0 0x7f4050139b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.261+0000 7f4057ce1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4050145230 0x7f405013a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.261+0000 7f4057ce1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f405015da30 0x7f40501525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.261+0000 7f4057ce1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405015c720 con 0x7f4050145230 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.261+0000 7f4057ce1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405015c5a0 con 0x7f40500711c0 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.261+0000 7f4057ce1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405015e3a0 con 0x7f405015da30 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.261+0000 7f405527c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4050145230 0x7f405013a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.261+0000 7f405527c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4050145230 0x7f405013a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43394/0 (socket says 172.21.15.139:43394) 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.261+0000 7f405527c640 1 -- 172.21.15.139:0/1608238675 learned_addr learned my addr 172.21.15.139:0/1608238675 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.261+0000 7f4055a7d640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40500711c0 0x7f4050139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f405627e640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f405015da30 0x7f40501525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f405527c640 1 -- 172.21.15.139:0/1608238675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f405015da30 msgr2=0x7f40501525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f405527c640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f405015da30 0x7f40501525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f405527c640 1 -- 172.21.15.139:0/1608238675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40500711c0 msgr2=0x7f4050139b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f405527c640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40500711c0 0x7f4050139b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f405527c640 1 -- 172.21.15.139:0/1608238675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40500a8d50 con 0x7f4050145230 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f4055a7d640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40500711c0 0x7f4050139b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:14.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f405627e640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f405015da30 0x7f40501525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:14.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f405527c640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4050145230 0x7f405013a0b0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f4040035040 tx=0x7f40400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:14.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f40467fc640 1 -- 172.21.15.139:0/1608238675 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040037050 con 0x7f4050145230 2024-04-25T10:01:14.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f40467fc640 1 -- 172.21.15.139:0/1608238675 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040038040 con 0x7f4050145230 2024-04-25T10:01:14.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f4057ce1640 1 -- 172.21.15.139:0/1608238675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40500a8a70 con 0x7f4050145230 2024-04-25T10:01:14.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f40467fc640 1 -- 172.21.15.139:0/1608238675 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040004490 con 0x7f4050145230 2024-04-25T10:01:14.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.262+0000 7f4057ce1640 1 -- 172.21.15.139:0/1608238675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405013aa00 con 0x7f4050145230 2024-04-25T10:01:14.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.263+0000 7f4057ce1640 1 -- 172.21.15.139:0/1608238675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405015e160 con 0x7f4050145230 2024-04-25T10:01:14.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.263+0000 7f40467fc640 1 -- 172.21.15.139:0/1608238675 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f404003c6d0 con 0x7f4050145230 2024-04-25T10:01:14.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.264+0000 7f40467fc640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f403406b4d0 0x7f403406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:14.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.264+0000 7f40467fc640 1 -- 172.21.15.139:0/1608238675 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f40400aab40 con 0x7f4050145230 2024-04-25T10:01:14.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.264+0000 7f4055a7d640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f403406b4d0 0x7f403406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:14.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.264+0000 7f4055a7d640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f403406b4d0 0x7f403406d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f404c0012e0 tx=0x7f404c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:14.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.268+0000 7f40467fc640 1 -- 172.21.15.139:0/1608238675 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f405015e160 con 0x7f4050145230 2024-04-25T10:01:14.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:14.441+0000 7f4057ce1640 1 -- 172.21.15.139:0/1608238675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4050074520 con 0x7f4050145230 2024-04-25T10:01:15.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.140+0000 7f40467fc640 1 -- 172.21.15.139:0/1608238675 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) v1 ==== 170+0+0 (secure 0 0 0) 0x7f40500a8d50 con 0x7f4050145230 2024-04-25T10:01:15.144 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' created 2024-04-25T10:01:15.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.143+0000 7f4057ce1640 1 -- 172.21.15.139:0/1608238675 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f403406b4d0 msgr2=0x7f403406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.143+0000 7f4057ce1640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f403406b4d0 0x7f403406d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f404c0012e0 tx=0x7f404c009040 comp rx=0 tx=0).stop 2024-04-25T10:01:15.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.143+0000 7f4057ce1640 1 -- 172.21.15.139:0/1608238675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4050145230 msgr2=0x7f405013a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.143+0000 7f4057ce1640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4050145230 0x7f405013a0b0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f4040035040 tx=0x7f40400046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.143+0000 7f4057ce1640 1 -- 172.21.15.139:0/1608238675 shutdown_connections 2024-04-25T10:01:15.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.143+0000 7f4057ce1640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4050145230 0x7f405013a0b0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.143+0000 7f4057ce1640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f403406b4d0 0x7f403406d990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.143+0000 7f4057ce1640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f405015da30 0x7f40501525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.143+0000 7f4057ce1640 1 --2- 172.21.15.139:0/1608238675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40500711c0 0x7f4050139b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.143+0000 7f4057ce1640 1 -- 172.21.15.139:0/1608238675 >> 172.21.15.139:0/1608238675 conn(0x7f4050067000 msgr2=0x7f405006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:15.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.144+0000 7f4057ce1640 1 -- 172.21.15.139:0/1608238675 shutdown_connections 2024-04-25T10:01:15.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.146+0000 7f4057ce1640 1 -- 172.21.15.139:0/1608238675 wait complete. 2024-04-25T10:01:15.175 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:01:15.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.273+0000 7ff8e684a640 1 Processor -- start 2024-04-25T10:01:15.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.274+0000 7ff8e684a640 1 -- start start 2024-04-25T10:01:15.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.274+0000 7ff8e684a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff8e01617e0 0x7ff8e00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.274+0000 7ff8e684a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff8e00cec80 0x7ff8e00d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.274+0000 7ff8e684a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e0161dc0 con 0x7ff8e015dbf0 2024-04-25T10:01:15.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.275+0000 7ff8e684a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e00cf0d0 con 0x7ff8e00cec80 2024-04-25T10:01:15.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.275+0000 7ff8e4de7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff8e00cec80 0x7ff8e00d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.275+0000 7ff8e684a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e00d1ee0 con 0x7ff8e01617e0 2024-04-25T10:01:15.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.275+0000 7ff8df7fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff8e01617e0 0x7ff8e00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.275+0000 7ff8e4de7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff8e00cec80 0x7ff8e00d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43398/0 (socket says 172.21.15.139:43398) 2024-04-25T10:01:15.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.275+0000 7ff8e4de7640 1 -- 172.21.15.139:0/3980033361 learned_addr learned my addr 172.21.15.139:0/3980033361 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:15.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.275+0000 7ff8e4de7640 1 -- 172.21.15.139:0/3980033361 >> v2:172.21.15.150:3300/0 conn(0x7ff8e01617e0 msgr2=0x7ff8e00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:15.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.275+0000 7ff8e4de7640 1 --2- 172.21.15.139:0/3980033361 >> v2:172.21.15.150:3300/0 conn(0x7ff8e01617e0 0x7ff8e00ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.275+0000 7ff8e4de7640 1 -- 172.21.15.139:0/3980033361 >> v1:172.21.15.150:6789/0 conn(0x7ff8e015dbf0 legacy=0x7ff8e00b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.275+0000 7ff8e4de7640 1 -- 172.21.15.139:0/3980033361 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e01620a0 con 0x7ff8e00cec80 2024-04-25T10:01:15.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.275+0000 7ff8e4de7640 1 --2- 172.21.15.139:0/3980033361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8e00cec80 0x7ff8e00d2370 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff8e008c420 tx=0x7ff8d802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cdd41ad90b2fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:15.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.276+0000 7ff8deffd640 1 -- 172.21.15.139:0/3980033361 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8036070 con 0x7ff8e00cec80 2024-04-25T10:01:15.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.276+0000 7ff8deffd640 1 -- 172.21.15.139:0/3980033361 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8031600 con 0x7ff8e00cec80 2024-04-25T10:01:15.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.276+0000 7ff8deffd640 1 -- 172.21.15.139:0/3980033361 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d80319d0 con 0x7ff8e00cec80 2024-04-25T10:01:15.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.276+0000 7ff8e684a640 1 -- 172.21.15.139:0/3980033361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8e00cec80 msgr2=0x7ff8e00d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.276+0000 7ff8e684a640 1 --2- 172.21.15.139:0/3980033361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8e00cec80 0x7ff8e00d2370 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff8e008c420 tx=0x7ff8d802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.276+0000 7ff8e684a640 1 -- 172.21.15.139:0/3980033361 shutdown_connections 2024-04-25T10:01:15.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.276+0000 7ff8e684a640 1 --2- 172.21.15.139:0/3980033361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8e00cec80 0x7ff8e00d2370 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.276+0000 7ff8e684a640 1 --2- 172.21.15.139:0/3980033361 >> v2:172.21.15.150:3300/0 conn(0x7ff8e01617e0 0x7ff8e00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.276+0000 7ff8e684a640 1 -- 172.21.15.139:0/3980033361 >> 172.21.15.139:0/3980033361 conn(0x7ff8e00b5480 msgr2=0x7ff8e00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:15.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.276+0000 7ff8e684a640 1 -- 172.21.15.139:0/3980033361 shutdown_connections 2024-04-25T10:01:15.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.276+0000 7ff8e684a640 1 -- 172.21.15.139:0/3980033361 wait complete. 2024-04-25T10:01:15.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.277+0000 7ff8e684a640 1 Processor -- start 2024-04-25T10:01:15.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.278+0000 7ff8e684a640 1 -- start start 2024-04-25T10:01:15.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.278+0000 7ff8e684a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8e00cec80 0x7ff8e01546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.278+0000 7ff8e684a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8e015dbf0 0x7ff8e01518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.278+0000 7ff8e684a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8e01617e0 0x7ff8e015f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.278+0000 7ff8e684a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e015dfd0 con 0x7ff8e015dbf0 2024-04-25T10:01:15.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.278+0000 7ff8e684a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0162660 con 0x7ff8e00cec80 2024-04-25T10:01:15.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.278+0000 7ff8e684a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0151e30 con 0x7ff8e01617e0 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.278+0000 7ff8df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8e015dbf0 0x7ff8e01518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.278+0000 7ff8df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8e015dbf0 0x7ff8e01518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43406/0 (socket says 172.21.15.139:43406) 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.278+0000 7ff8df7fe640 1 -- 172.21.15.139:0/1613417702 learned_addr learned my addr 172.21.15.139:0/1613417702 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8df7fe640 1 -- 172.21.15.139:0/1613417702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8e01617e0 msgr2=0x7ff8e015f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8dffff640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8e00cec80 0x7ff8e01546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8e4de7640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8e01617e0 0x7ff8e015f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8df7fe640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8e01617e0 0x7ff8e015f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8df7fe640 1 -- 172.21.15.139:0/1613417702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8e00cec80 msgr2=0x7ff8e01546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8df7fe640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8e00cec80 0x7ff8e01546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8df7fe640 1 -- 172.21.15.139:0/1613417702 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e0152a00 con 0x7ff8e015dbf0 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8df7fe640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8e015dbf0 0x7ff8e01518f0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400e870 tx=0x7ff8d400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8dcff9640 1 -- 172.21.15.139:0/1613417702 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4017070 con 0x7ff8e015dbf0 2024-04-25T10:01:15.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8dcff9640 1 -- 172.21.15.139:0/1613417702 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d400c8e0 con 0x7ff8e015dbf0 2024-04-25T10:01:15.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8e684a640 1 -- 172.21.15.139:0/1613417702 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e015f8c0 con 0x7ff8e015dbf0 2024-04-25T10:01:15.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8dcff9640 1 -- 172.21.15.139:0/1613417702 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d400ccd0 con 0x7ff8e015dbf0 2024-04-25T10:01:15.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.279+0000 7ff8e684a640 1 -- 172.21.15.139:0/1613417702 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e0152ca0 con 0x7ff8e015dbf0 2024-04-25T10:01:15.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.280+0000 7ff8e684a640 1 -- 172.21.15.139:0/1613417702 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e00cf8e0 con 0x7ff8e015dbf0 2024-04-25T10:01:15.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.280+0000 7ff8dcff9640 1 -- 172.21.15.139:0/1613417702 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8d401a070 con 0x7ff8e015dbf0 2024-04-25T10:01:15.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.281+0000 7ff8dcff9640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8cc06b4d0 0x7ff8cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:15.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.281+0000 7ff8dcff9640 1 -- 172.21.15.139:0/1613417702 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6001+0+0 (secure 0 0 0) 0x7ff8d408a0d0 con 0x7ff8e015dbf0 2024-04-25T10:01:15.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.281+0000 7ff8dffff640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8cc06b4d0 0x7ff8cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:15.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.281+0000 7ff8dffff640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8cc06b4d0 0x7ff8cc06d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0154130 tx=0x7ff8d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:15.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.285+0000 7ff8dcff9640 1 -- 172.21.15.139:0/1613417702 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8e00cf8e0 con 0x7ff8e015dbf0 2024-04-25T10:01:15.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:15.496+0000 7ff8e684a640 1 -- 172.21.15.139:0/1613417702 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff8e0074520 con 0x7ff8e015dbf0 2024-04-25T10:01:16.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.141+0000 7ff8dcff9640 1 -- 172.21.15.139:0/1613417702 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v67) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff8d4058eb0 con 0x7ff8e015dbf0 2024-04-25T10:01:16.145 INFO:teuthology.orchestra.run.smithi139.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.146+0000 7ff8e684a640 1 -- 172.21.15.139:0/1613417702 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8cc06b4d0 msgr2=0x7ff8cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.146+0000 7ff8e684a640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8cc06b4d0 0x7ff8cc06d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0154130 tx=0x7ff8d0009040 comp rx=0 tx=0).stop 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.146+0000 7ff8e684a640 1 -- 172.21.15.139:0/1613417702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8e015dbf0 msgr2=0x7ff8e01518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.146+0000 7ff8e684a640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8e015dbf0 0x7ff8e01518f0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400e870 tx=0x7ff8d400ed30 comp rx=0 tx=0).stop 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.146+0000 7ff8e684a640 1 -- 172.21.15.139:0/1613417702 shutdown_connections 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.146+0000 7ff8e684a640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8e015dbf0 0x7ff8e01518f0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.146+0000 7ff8e684a640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8cc06b4d0 0x7ff8cc06d990 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.146+0000 7ff8e684a640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8e01617e0 0x7ff8e015f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.146+0000 7ff8e684a640 1 --2- 172.21.15.139:0/1613417702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8e00cec80 0x7ff8e01546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.146+0000 7ff8e684a640 1 -- 172.21.15.139:0/1613417702 >> 172.21.15.139:0/1613417702 conn(0x7ff8e00b5480 msgr2=0x7ff8e00d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.146+0000 7ff8e684a640 1 -- 172.21.15.139:0/1613417702 shutdown_connections 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.147+0000 7ff8e684a640 1 -- 172.21.15.139:0/1613417702 wait complete. 2024-04-25T10:01:16.173 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:16.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.269+0000 7fc9c55c8640 1 Processor -- start 2024-04-25T10:01:16.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.269+0000 7fc9c55c8640 1 -- start start 2024-04-25T10:01:16.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9c55c8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc9c00d1080 0x7fc9c00b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9c55c8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c0161850 0x7fc9c00ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9c55c8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c00cf250 con 0x7fc9c015dc60 2024-04-25T10:01:16.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9c55c8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c00ce780 con 0x7fc9c00d1080 2024-04-25T10:01:16.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9c55c8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00d3730 con 0x7fc9c0161850 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9b7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc9c00d1080 0x7fc9c00b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9beffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc9c015dc60 0x7fc9c00d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:47170/0 (socket says 172.21.15.139:47170) 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9b7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc9c00d1080 0x7fc9c00b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43422/0 (socket says 172.21.15.139:43422) 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9beffd640 1 -- 172.21.15.139:0/2920666583 learned_addr learned my addr 172.21.15.139:0/2920666583 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9bf7fe640 1 --2- 172.21.15.139:0/2920666583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c0161850 0x7fc9c00ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9b7fff640 1 -- 172.21.15.139:0/2920666583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c0161850 msgr2=0x7fc9c00ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9b7fff640 1 --2- 172.21.15.139:0/2920666583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c0161850 0x7fc9c00ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9b7fff640 1 -- 172.21.15.139:0/2920666583 >> v1:172.21.15.139:6789/0 conn(0x7fc9c015dc60 legacy=0x7fc9c00d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9be7fc640 1 -- 172.21.15.139:0/2920666583 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281175633 0 0) 0x7fc9b0001110 con 0x7fc9c015dc60 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9b7fff640 1 -- 172.21.15.139:0/2920666583 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c00ce9b0 con 0x7fc9c00d1080 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.270+0000 7fc9bf7fe640 1 --2- 172.21.15.139:0/2920666583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c0161850 0x7fc9c00ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.271+0000 7fc9b7fff640 1 --2- 172.21.15.139:0/2920666583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9c00d1080 0x7fc9c00b6c10 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80097d0 tx=0x7fc9a802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30ce6d8fbdf1e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.271+0000 7fc9be7fc640 1 -- 172.21.15.139:0/2920666583 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8037070 con 0x7fc9c00d1080 2024-04-25T10:01:16.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.271+0000 7fc9be7fc640 1 -- 172.21.15.139:0/2920666583 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a8032440 con 0x7fc9c00d1080 2024-04-25T10:01:16.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.271+0000 7fc9be7fc640 1 -- 172.21.15.139:0/2920666583 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8032810 con 0x7fc9c00d1080 2024-04-25T10:01:16.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.272+0000 7fc9c55c8640 1 -- 172.21.15.139:0/2920666583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9c00d1080 msgr2=0x7fc9c00b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.272+0000 7fc9c55c8640 1 --2- 172.21.15.139:0/2920666583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9c00d1080 0x7fc9c00b6c10 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80097d0 tx=0x7fc9a802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.272+0000 7fc9c55c8640 1 -- 172.21.15.139:0/2920666583 shutdown_connections 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.272+0000 7fc9c55c8640 1 --2- 172.21.15.139:0/2920666583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c0161850 0x7fc9c00ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.272+0000 7fc9c55c8640 1 --2- 172.21.15.139:0/2920666583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9c00d1080 0x7fc9c00b6c10 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.272+0000 7fc9c55c8640 1 -- 172.21.15.139:0/2920666583 >> 172.21.15.139:0/2920666583 conn(0x7fc9c00b5910 msgr2=0x7fc9c00b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.272+0000 7fc9c55c8640 1 -- 172.21.15.139:0/2920666583 shutdown_connections 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.272+0000 7fc9c55c8640 1 -- 172.21.15.139:0/2920666583 wait complete. 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.273+0000 7fc9c55c8640 1 Processor -- start 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.273+0000 7fc9c55c8640 1 -- start start 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9c55c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9c00d1080 0x7fc9c0150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9c55c8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c015dc60 0x7fc9c0160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9c55c8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c0161850 0x7fc9c00ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9c55c8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00d1fc0 con 0x7fc9c00d1080 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9c55c8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00cef70 con 0x7fc9c015dc60 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9c55c8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c015e040 con 0x7fc9c0161850 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9beffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9c00d1080 0x7fc9c0150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9beffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9c00d1080 0x7fc9c0150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43428/0 (socket says 172.21.15.139:43428) 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9beffd640 1 -- 172.21.15.139:0/1353659050 learned_addr learned my addr 172.21.15.139:0/1353659050 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9bf7fe640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c0161850 0x7fc9c00ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9b7fff640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c015dc60 0x7fc9c0160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9beffd640 1 -- 172.21.15.139:0/1353659050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c0161850 msgr2=0x7fc9c00ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9beffd640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c0161850 0x7fc9c00ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9beffd640 1 -- 172.21.15.139:0/1353659050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c015dc60 msgr2=0x7fc9c0160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9beffd640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c015dc60 0x7fc9c0160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9beffd640 1 -- 172.21.15.139:0/1353659050 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c015f1a0 con 0x7fc9c00d1080 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9beffd640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9c00d1080 0x7fc9c0150d20 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000c7b0 tx=0x7fc9b000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9b77fe640 1 -- 172.21.15.139:0/1353659050 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0018070 con 0x7fc9c00d1080 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9b77fe640 1 -- 172.21.15.139:0/1353659050 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b00072d0 con 0x7fc9c00d1080 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9c55c8640 1 -- 172.21.15.139:0/1353659050 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c015f480 con 0x7fc9c00d1080 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9b77fe640 1 -- 172.21.15.139:0/1353659050 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b00047e0 con 0x7fc9c00d1080 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.274+0000 7fc9c55c8640 1 -- 172.21.15.139:0/1353659050 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c015fa40 con 0x7fc9c00d1080 2024-04-25T10:01:16.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.275+0000 7fc9c55c8640 1 -- 172.21.15.139:0/1353659050 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c0074520 con 0x7fc9c00d1080 2024-04-25T10:01:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.276+0000 7fc9b77fe640 1 -- 172.21.15.139:0/1353659050 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9b001b070 con 0x7fc9c00d1080 2024-04-25T10:01:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.276+0000 7fc9b77fe640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9a006b4d0 0x7fc9a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.276+0000 7fc9b77fe640 1 -- 172.21.15.139:0/1353659050 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc9b00896d0 con 0x7fc9c00d1080 2024-04-25T10:01:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.276+0000 7fc9b7fff640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9a006b4d0 0x7fc9a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.277+0000 7fc9b7fff640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9a006b4d0 0x7fc9a006d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8034000 tx=0x7fc9a802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.281+0000 7fc9b77fe640 1 -- 172.21.15.139:0/1353659050 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc9c00cf250 con 0x7fc9c00d1080 2024-04-25T10:01:16.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.453+0000 7fc9c55c8640 1 -- 172.21.15.139:0/1353659050 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9c00d16f0 con 0x7fc9c00d1080 2024-04-25T10:01:16.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.454+0000 7fc9b77fe640 1 -- 172.21.15.139:0/1353659050 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16945 (secure 0 0 0) 0x7fc9b00583c0 con 0x7fc9c00d1080 2024-04-25T10:01:16.460 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:16.460 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":104,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:01:15.497594+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:13.695577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:14.460042+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:16.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.456+0000 7fc9c55c8640 1 -- 172.21.15.139:0/1353659050 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9a006b4d0 msgr2=0x7fc9a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.456+0000 7fc9c55c8640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9a006b4d0 0x7fc9a006d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8034000 tx=0x7fc9a802e000 comp rx=0 tx=0).stop 2024-04-25T10:01:16.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.456+0000 7fc9c55c8640 1 -- 172.21.15.139:0/1353659050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9c00d1080 msgr2=0x7fc9c0150d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.456+0000 7fc9c55c8640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9c00d1080 0x7fc9c0150d20 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000c7b0 tx=0x7fc9b000cc70 comp rx=0 tx=0).stop 2024-04-25T10:01:16.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.456+0000 7fc9c55c8640 1 -- 172.21.15.139:0/1353659050 shutdown_connections 2024-04-25T10:01:16.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.456+0000 7fc9c55c8640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc9a006b4d0 0x7fc9a006d990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.456+0000 7fc9c55c8640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c0161850 0x7fc9c00ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.456+0000 7fc9c55c8640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c015dc60 0x7fc9c0160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.456+0000 7fc9c55c8640 1 --2- 172.21.15.139:0/1353659050 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc9c00d1080 0x7fc9c0150d20 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.456+0000 7fc9c55c8640 1 -- 172.21.15.139:0/1353659050 >> 172.21.15.139:0/1353659050 conn(0x7fc9c00b5910 msgr2=0x7fc9c015bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:16.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.457+0000 7fc9c55c8640 1 -- 172.21.15.139:0/1353659050 shutdown_connections 2024-04-25T10:01:16.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.457+0000 7fc9c55c8640 1 -- 172.21.15.139:0/1353659050 wait complete. 2024-04-25T10:01:16.472 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:16.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.565+0000 7f16d1231640 1 Processor -- start 2024-04-25T10:01:16.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.565+0000 7f16d1231640 1 -- start start 2024-04-25T10:01:16.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16d1231640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f16cc15a860 0x7f16cc0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16d1231640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc06ea20 0x7f16cc0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16d1231640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16cc0af6d0 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16d1231640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16cc0b02e0 con 0x7f16cc15a860 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16d1231640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0af9b0 con 0x7f16cc06ea20 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16cad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f16cc15a860 0x7f16cc0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16cad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f16cc15a860 0x7f16cc0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44044/0 (socket says 172.21.15.139:44044) 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16cad9d640 1 -- 172.21.15.139:0/2965727079 learned_addr learned my addr 172.21.15.139:0/2965727079 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16cb59e640 1 --2- 172.21.15.139:0/2965727079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc06ea20 0x7f16cc0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16cad9d640 1 -- 172.21.15.139:0/2965727079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc06ea20 msgr2=0x7f16cc0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16cad9d640 1 --2- 172.21.15.139:0/2965727079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc06ea20 0x7f16cc0b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16cad9d640 1 -- 172.21.15.139:0/2965727079 >> v1:172.21.15.139:6789/0 conn(0x7f16cc0cd1f0 legacy=0x7f16cc06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16cad9d640 1 -- 172.21.15.139:0/2965727079 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc0afc90 con 0x7f16cc15a860 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.566+0000 7f16cb59e640 1 --2- 172.21.15.139:0/2965727079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc06ea20 0x7f16cc0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.567+0000 7f16cad9d640 1 --2- 172.21.15.139:0/2965727079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc15a860 0x7f16cc0cccb0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009870 tx=0x7f16bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a242b7560b1cb47d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.567+0000 7f16ca59c640 1 -- 172.21.15.139:0/2965727079 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc037070 con 0x7f16cc15a860 2024-04-25T10:01:16.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.567+0000 7f16ca59c640 1 -- 172.21.15.139:0/2965727079 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc032440 con 0x7f16cc15a860 2024-04-25T10:01:16.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.567+0000 7f16ca59c640 1 -- 172.21.15.139:0/2965727079 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc032810 con 0x7f16cc15a860 2024-04-25T10:01:16.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.567+0000 7f16d1231640 1 -- 172.21.15.139:0/2965727079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc15a860 msgr2=0x7f16cc0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.567+0000 7f16d1231640 1 --2- 172.21.15.139:0/2965727079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc15a860 0x7f16cc0cccb0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009870 tx=0x7f16bc02a270 comp rx=0 tx=0).stop 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.568+0000 7f16d1231640 1 -- 172.21.15.139:0/2965727079 shutdown_connections 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.568+0000 7f16d1231640 1 --2- 172.21.15.139:0/2965727079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc06ea20 0x7f16cc0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.568+0000 7f16d1231640 1 --2- 172.21.15.139:0/2965727079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc15a860 0x7f16cc0cccb0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.568+0000 7f16d1231640 1 -- 172.21.15.139:0/2965727079 >> 172.21.15.139:0/2965727079 conn(0x7f16cc066f20 msgr2=0x7f16cc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.568+0000 7f16d1231640 1 -- 172.21.15.139:0/2965727079 shutdown_connections 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.568+0000 7f16d1231640 1 -- 172.21.15.139:0/2965727079 wait complete. 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.569+0000 7f16d1231640 1 Processor -- start 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.569+0000 7f16d1231640 1 -- start start 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16d1231640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc06ea20 0x7f16cc145070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16d1231640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc0cd1f0 0x7f16cc152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16d1231640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16cc15a860 0x7f16cc071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16d1231640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0af420 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16d1231640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0af2a0 con 0x7f16cc15a860 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16d1231640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc15ac40 con 0x7f16cc06ea20 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc0cd1f0 0x7f16cc152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc0cd1f0 0x7f16cc152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44054/0 (socket says 172.21.15.139:44054) 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16c3fff640 1 -- 172.21.15.139:0/3776717690 learned_addr learned my addr 172.21.15.139:0/3776717690 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16cad9d640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc06ea20 0x7f16cc145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16cb59e640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16cc15a860 0x7f16cc071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16c3fff640 1 -- 172.21.15.139:0/3776717690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc06ea20 msgr2=0x7f16cc145070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16c3fff640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc06ea20 0x7f16cc145070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16c3fff640 1 -- 172.21.15.139:0/3776717690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16cc15a860 msgr2=0x7f16cc071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16c3fff640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16cc15a860 0x7f16cc071170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16c3fff640 1 -- 172.21.15.139:0/3776717690 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc0a8630 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16cad9d640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc06ea20 0x7f16cc145070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:16.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16c3fff640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc0cd1f0 0x7f16cc152aa0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f16b400c440 tx=0x7f16b4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16c37fe640 1 -- 172.21.15.139:0/3776717690 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b40052d0 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16c37fe640 1 -- 172.21.15.139:0/3776717690 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b4005470 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16d1231640 1 -- 172.21.15.139:0/3776717690 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc0a8910 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16c37fe640 1 -- 172.21.15.139:0/3776717690 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4007260 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.570+0000 7f16d1231640 1 -- 172.21.15.139:0/3776717690 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16cc0a8ed0 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.571+0000 7f16d1231640 1 -- 172.21.15.139:0/3776717690 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16cc074520 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.572+0000 7f16c37fe640 1 -- 172.21.15.139:0/3776717690 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16b4007400 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.572+0000 7f16c37fe640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f16ac06b500 0x7f16ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.572+0000 7f16c37fe640 1 -- 172.21.15.139:0/3776717690 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16b40897f0 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.572+0000 7f16cad9d640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f16ac06b500 0x7f16ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.573+0000 7f16cad9d640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f16ac06b500 0x7f16ac06d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f16bc034000 tx=0x7f16bc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.576+0000 7f16c37fe640 1 -- 172.21.15.139:0/3776717690 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16b4058570 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.745+0000 7f16d1231640 1 -- 172.21.15.139:0/3776717690 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16cc06d580 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.746+0000 7f16c37fe640 1 -- 172.21.15.139:0/3776717690 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16945 (secure 0 0 0) 0x7f16b4057e70 con 0x7f16cc0cd1f0 2024-04-25T10:01:16.751 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:16.751 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":104,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:01:15.497594+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:13.695577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:14.460042+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.748+0000 7f16d1231640 1 -- 172.21.15.139:0/3776717690 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f16ac06b500 msgr2=0x7f16ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.748+0000 7f16d1231640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f16ac06b500 0x7f16ac06d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f16bc034000 tx=0x7f16bc02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.748+0000 7f16d1231640 1 -- 172.21.15.139:0/3776717690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc0cd1f0 msgr2=0x7f16cc152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.748+0000 7f16d1231640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc0cd1f0 0x7f16cc152aa0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f16b400c440 tx=0x7f16b4001e70 comp rx=0 tx=0).stop 2024-04-25T10:01:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.748+0000 7f16d1231640 1 -- 172.21.15.139:0/3776717690 shutdown_connections 2024-04-25T10:01:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.748+0000 7f16d1231640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16cc0cd1f0 0x7f16cc152aa0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.748+0000 7f16d1231640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f16ac06b500 0x7f16ac06d9c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.748+0000 7f16d1231640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16cc15a860 0x7f16cc071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.748+0000 7f16d1231640 1 --2- 172.21.15.139:0/3776717690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc06ea20 0x7f16cc145070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.748+0000 7f16d1231640 1 -- 172.21.15.139:0/3776717690 >> 172.21.15.139:0/3776717690 conn(0x7f16cc066f20 msgr2=0x7f16cc158c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.748+0000 7f16d1231640 1 -- 172.21.15.139:0/3776717690 shutdown_connections 2024-04-25T10:01:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.748+0000 7f16d1231640 1 -- 172.21.15.139:0/3776717690 wait complete. 2024-04-25T10:01:16.764 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:01:16.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.859+0000 7fce56dd5640 1 Processor -- start 2024-04-25T10:01:16.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.859+0000 7fce56dd5640 1 -- start start 2024-04-25T10:01:16.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.860+0000 7fce56dd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce500cd610 0x7fce500d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.860+0000 7fce56dd5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fce5009ad40 0x7fce500d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.860+0000 7fce56dd5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce50099ca0 con 0x7fce501618c0 2024-04-25T10:01:16.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.860+0000 7fce56dd5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce50099a50 con 0x7fce5009ad40 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.860+0000 7fce56dd5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce50099f80 con 0x7fce500cd610 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.860+0000 7fce55372640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fce501618c0 0x7fce5006be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38346/0 (socket says 172.21.15.139:38346) 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.860+0000 7fce55372640 1 -- 172.21.15.139:0/3609867225 learned_addr learned my addr 172.21.15.139:0/3609867225 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.860+0000 7fce54b71640 1 --2- 172.21.15.139:0/3609867225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce500cd610 0x7fce500d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.860+0000 7fce4ffff640 1 --2- 172.21.15.139:0/3609867225 >> v2:172.21.15.150:3300/0 conn(0x7fce5009ad40 0x7fce500d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.860+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438260720 0 0) 0x7fce50099ca0 con 0x7fce501618c0 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce38003520 con 0x7fce501618c0 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1082446254 0 0) 0x7fce38003520 con 0x7fce501618c0 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce500cd610 msgr2=0x7fce500d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 --2- 172.21.15.139:0/3609867225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce500cd610 0x7fce500d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 >> v2:172.21.15.150:3300/0 conn(0x7fce5009ad40 msgr2=0x7fce500d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 --2- 172.21.15.139:0/3609867225 >> v2:172.21.15.150:3300/0 conn(0x7fce5009ad40 0x7fce500d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5009a260 con 0x7fce501618c0 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fce480019f0 con 0x7fce501618c0 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 >> v1:172.21.15.139:6789/0 conn(0x7fce501618c0 legacy=0x7fce5006be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 --2- 172.21.15.139:0/3609867225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce500cd610 0x7fce380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 --2- 172.21.15.139:0/3609867225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce38003f30 0x7fce38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 --2- 172.21.15.139:0/3609867225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce38004d40 0x7fce380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce50099a50 con 0x7fce38004d40 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce50099f80 con 0x7fce38003f30 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3800b9f0 con 0x7fce500cd610 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce55372640 1 --2- 172.21.15.139:0/3609867225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce38004d40 0x7fce380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce54b71640 1 --2- 172.21.15.139:0/3609867225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce500cd610 0x7fce380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce4ffff640 1 --2- 172.21.15.139:0/3609867225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce38003f30 0x7fce38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce55372640 1 -- 172.21.15.139:0/3609867225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce500cd610 msgr2=0x7fce380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce55372640 1 --2- 172.21.15.139:0/3609867225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce500cd610 0x7fce380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce55372640 1 -- 172.21.15.139:0/3609867225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce38003f30 msgr2=0x7fce38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce55372640 1 --2- 172.21.15.139:0/3609867225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce38003f30 0x7fce38004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.861+0000 7fce55372640 1 -- 172.21.15.139:0/3609867225 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce38003520 con 0x7fce38004d40 2024-04-25T10:01:16.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.862+0000 7fce55372640 1 --2- 172.21.15.139:0/3609867225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce38004d40 0x7fce380051c0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fce480013d0 tx=0x7fce38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7fd468fca7de75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.862+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4800a4e0 con 0x7fce38004d40 2024-04-25T10:01:16.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.862+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4802f070 con 0x7fce38004d40 2024-04-25T10:01:16.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.862+0000 7fce4f7fe640 1 -- 172.21.15.139:0/3609867225 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4802bce0 con 0x7fce38004d40 2024-04-25T10:01:16.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.863+0000 7fce56dd5640 1 -- 172.21.15.139:0/3609867225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce38004d40 msgr2=0x7fce380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.863+0000 7fce56dd5640 1 --2- 172.21.15.139:0/3609867225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce38004d40 0x7fce380051c0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fce480013d0 tx=0x7fce38005830 comp rx=0 tx=0).stop 2024-04-25T10:01:16.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.863+0000 7fce55372640 1 -- 172.21.15.139:0/3609867225 reap_dead start 2024-04-25T10:01:16.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.863+0000 7fce56dd5640 1 -- 172.21.15.139:0/3609867225 shutdown_connections 2024-04-25T10:01:16.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.863+0000 7fce56dd5640 1 -- 172.21.15.139:0/3609867225 >> 172.21.15.139:0/3609867225 conn(0x7fce500b31f0 msgr2=0x7fce500b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:16.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.863+0000 7fce56dd5640 1 -- 172.21.15.139:0/3609867225 shutdown_connections 2024-04-25T10:01:16.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.863+0000 7fce56dd5640 1 -- 172.21.15.139:0/3609867225 wait complete. 2024-04-25T10:01:16.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.864+0000 7fce56dd5640 1 Processor -- start 2024-04-25T10:01:16.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.864+0000 7fce56dd5640 1 -- start start 2024-04-25T10:01:16.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce56dd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce50155c30 0x7fce5015b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce56dd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce50156010 0x7fce50159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce56dd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce501597a0 0x7fce50157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce56dd5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500cd9f0 con 0x7fce50155c30 2024-04-25T10:01:16.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce56dd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5009a820 con 0x7fce50156010 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce56dd5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce500cdb70 con 0x7fce501597a0 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce54b71640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce50155c30 0x7fce5015b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce54b71640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce50155c30 0x7fce5015b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44062/0 (socket says 172.21.15.139:44062) 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce54b71640 1 -- 172.21.15.139:0/1202916493 learned_addr learned my addr 172.21.15.139:0/1202916493 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce55372640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce501597a0 0x7fce50157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce4ffff640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce50156010 0x7fce50159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce54b71640 1 -- 172.21.15.139:0/1202916493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce501597a0 msgr2=0x7fce50157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce54b71640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce501597a0 0x7fce50157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce54b71640 1 -- 172.21.15.139:0/1202916493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce50156010 msgr2=0x7fce50159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce54b71640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce50156010 0x7fce50159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce54b71640 1 -- 172.21.15.139:0/1202916493 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce500c52d0 con 0x7fce50155c30 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce54b71640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce50155c30 0x7fce5015b750 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fce400119d0 tx=0x7fce40011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce4d7fa640 1 -- 172.21.15.139:0/1202916493 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4000fb50 con 0x7fce50155c30 2024-04-25T10:01:16.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce4d7fa640 1 -- 172.21.15.139:0/1202916493 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4000fcf0 con 0x7fce50155c30 2024-04-25T10:01:16.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce4d7fa640 1 -- 172.21.15.139:0/1202916493 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4000fea0 con 0x7fce50155c30 2024-04-25T10:01:16.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce56dd5640 1 -- 172.21.15.139:0/1202916493 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce500c55b0 con 0x7fce50155c30 2024-04-25T10:01:16.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.865+0000 7fce56dd5640 1 -- 172.21.15.139:0/1202916493 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce50158a50 con 0x7fce50155c30 2024-04-25T10:01:16.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.866+0000 7fce56dd5640 1 -- 172.21.15.139:0/1202916493 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5015a0d0 con 0x7fce50155c30 2024-04-25T10:01:16.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.867+0000 7fce4d7fa640 1 -- 172.21.15.139:0/1202916493 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce40012040 con 0x7fce50155c30 2024-04-25T10:01:16.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.867+0000 7fce4d7fa640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce3c06b4d0 0x7fce3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:16.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.867+0000 7fce4d7fa640 1 -- 172.21.15.139:0/1202916493 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fce4008a9b0 con 0x7fce50155c30 2024-04-25T10:01:16.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.867+0000 7fce4ffff640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce3c06b4d0 0x7fce3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:16.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.868+0000 7fce4ffff640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce3c06b4d0 0x7fce3c06d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fce5015dee0 tx=0x7fce44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:16.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:16.871+0000 7fce4d7fa640 1 -- 172.21.15.139:0/1202916493 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce40019030 con 0x7fce50155c30 2024-04-25T10:01:16.906 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:16.908 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:17.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.084+0000 7fce56dd5640 1 -- 172.21.15.139:0/1202916493 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fce50159bb0 con 0x7fce50155c30 2024-04-25T10:01:17.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.162+0000 7fce4d7fa640 1 -- 172.21.15.139:0/1202916493 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7fce50159bb0 con 0x7fce50155c30 2024-04-25T10:01:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.164+0000 7fce56dd5640 1 -- 172.21.15.139:0/1202916493 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce3c06b4d0 msgr2=0x7fce3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.164+0000 7fce56dd5640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce3c06b4d0 0x7fce3c06d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fce5015dee0 tx=0x7fce44008040 comp rx=0 tx=0).stop 2024-04-25T10:01:17.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.164+0000 7fce56dd5640 1 -- 172.21.15.139:0/1202916493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce50155c30 msgr2=0x7fce5015b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.164+0000 7fce56dd5640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce50155c30 0x7fce5015b750 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fce400119d0 tx=0x7fce40011e90 comp rx=0 tx=0).stop 2024-04-25T10:01:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.164+0000 7fce56dd5640 1 -- 172.21.15.139:0/1202916493 shutdown_connections 2024-04-25T10:01:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.164+0000 7fce56dd5640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce3c06b4d0 0x7fce3c06d990 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.164+0000 7fce56dd5640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce501597a0 0x7fce50157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.164+0000 7fce56dd5640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce50156010 0x7fce50159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.164+0000 7fce56dd5640 1 --2- 172.21.15.139:0/1202916493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce50155c30 0x7fce5015b750 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.164+0000 7fce56dd5640 1 -- 172.21.15.139:0/1202916493 >> 172.21.15.139:0/1202916493 conn(0x7fce500b31f0 msgr2=0x7fce500b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.164+0000 7fce56dd5640 1 -- 172.21.15.139:0/1202916493 shutdown_connections 2024-04-25T10:01:17.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.164+0000 7fce56dd5640 1 -- 172.21.15.139:0/1202916493 wait complete. 2024-04-25T10:01:17.179 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:01:17.179 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:01:17.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.274+0000 7fabc8128640 1 Processor -- start 2024-04-25T10:01:17.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.274+0000 7fabc8128640 1 -- start start 2024-04-25T10:01:17.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc8128640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc0145230 0x7fabc015be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc8128640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fabc015c3d0 0x7fabc00711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc8128640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc015c960 con 0x7fabc0071700 2024-04-25T10:01:17.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc8128640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc0071b50 con 0x7fabc015c3d0 2024-04-25T10:01:17.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc8128640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc015cc40 con 0x7fabc0145230 2024-04-25T10:01:17.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc56c3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fabc015c3d0 0x7fabc00711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc56c3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fabc015c3d0 0x7fabc00711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44076/0 (socket says 172.21.15.139:44076) 2024-04-25T10:01:17.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc56c3640 1 -- 172.21.15.139:0/1655748530 learned_addr learned my addr 172.21.15.139:0/1655748530 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:17.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc5ec4640 1 --2- 172.21.15.139:0/1655748530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc0145230 0x7fabc015be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc56c3640 1 -- 172.21.15.139:0/1655748530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc0145230 msgr2=0x7fabc015be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:17.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc56c3640 1 --2- 172.21.15.139:0/1655748530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc0145230 0x7fabc015be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc56c3640 1 -- 172.21.15.139:0/1655748530 >> v1:172.21.15.150:6789/0 conn(0x7fabc0071700 legacy=0x7fabc006d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.275+0000 7fabc56c3640 1 -- 172.21.15.139:0/1655748530 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc015d230 con 0x7fabc015c3d0 2024-04-25T10:01:17.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.276+0000 7fabc56c3640 1 --2- 172.21.15.139:0/1655748530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc015c3d0 0x7fabc00711c0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009870 tx=0x7fabb002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87962936888123e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:17.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.276+0000 7fabc4ec2640 1 -- 172.21.15.139:0/1655748530 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0037070 con 0x7fabc015c3d0 2024-04-25T10:01:17.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.276+0000 7fabc4ec2640 1 -- 172.21.15.139:0/1655748530 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb0032440 con 0x7fabc015c3d0 2024-04-25T10:01:17.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.276+0000 7fabc4ec2640 1 -- 172.21.15.139:0/1655748530 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0032810 con 0x7fabc015c3d0 2024-04-25T10:01:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.276+0000 7fabc8128640 1 -- 172.21.15.139:0/1655748530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc015c3d0 msgr2=0x7fabc00711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.276+0000 7fabc8128640 1 --2- 172.21.15.139:0/1655748530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc015c3d0 0x7fabc00711c0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009870 tx=0x7fabb002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.277+0000 7fabc8128640 1 -- 172.21.15.139:0/1655748530 shutdown_connections 2024-04-25T10:01:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.277+0000 7fabc8128640 1 --2- 172.21.15.139:0/1655748530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc015c3d0 0x7fabc00711c0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.277+0000 7fabc8128640 1 --2- 172.21.15.139:0/1655748530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc0145230 0x7fabc015be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.277+0000 7fabc8128640 1 -- 172.21.15.139:0/1655748530 >> 172.21.15.139:0/1655748530 conn(0x7fabc0067000 msgr2=0x7fabc006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.277+0000 7fabc8128640 1 -- 172.21.15.139:0/1655748530 shutdown_connections 2024-04-25T10:01:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.277+0000 7fabc8128640 1 -- 172.21.15.139:0/1655748530 wait complete. 2024-04-25T10:01:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.278+0000 7fabc8128640 1 Processor -- start 2024-04-25T10:01:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.278+0000 7fabc8128640 1 -- start start 2024-04-25T10:01:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.278+0000 7fabc8128640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc0071700 0x7fabc0139d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.278+0000 7fabc8128640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc0145230 0x7fabc00adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.278+0000 7fabc8128640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc015c3d0 0x7fabc00ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.278+0000 7fabc8128640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc015ea40 con 0x7fabc015c3d0 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.278+0000 7fabc8128640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc006e4b0 con 0x7fabc0071700 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.278+0000 7fabc8128640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc015ebc0 con 0x7fabc0145230 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc66c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc015c3d0 0x7fabc00ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc66c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc015c3d0 0x7fabc00ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44088/0 (socket says 172.21.15.139:44088) 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc66c5640 1 -- 172.21.15.139:0/323245267 learned_addr learned my addr 172.21.15.139:0/323245267 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc5ec4640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc0071700 0x7fabc0139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc56c3640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc0145230 0x7fabc00adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc66c5640 1 -- 172.21.15.139:0/323245267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc0145230 msgr2=0x7fabc00adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc66c5640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc0145230 0x7fabc00adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc66c5640 1 -- 172.21.15.139:0/323245267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc0071700 msgr2=0x7fabc0139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc66c5640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc0071700 0x7fabc0139d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc66c5640 1 -- 172.21.15.139:0/323245267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc013a270 con 0x7fabc015c3d0 2024-04-25T10:01:17.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc56c3640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc0145230 0x7fabc00adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:17.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc5ec4640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc0071700 0x7fabc0139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:17.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc66c5640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc015c3d0 0x7fabc00ae060 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fabb800c7a0 tx=0x7fabb800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:17.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabb67fc640 1 -- 172.21.15.139:0/323245267 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8003b70 con 0x7fabc015c3d0 2024-04-25T10:01:17.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabb67fc640 1 -- 172.21.15.139:0/323245267 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb8003d30 con 0x7fabc015c3d0 2024-04-25T10:01:17.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabb67fc640 1 -- 172.21.15.139:0/323245267 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb800aa90 con 0x7fabc015c3d0 2024-04-25T10:01:17.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc8128640 1 -- 172.21.15.139:0/323245267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc00ac540 con 0x7fabc015c3d0 2024-04-25T10:01:17.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.279+0000 7fabc8128640 1 -- 172.21.15.139:0/323245267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc00acb00 con 0x7fabc015c3d0 2024-04-25T10:01:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.280+0000 7fabc8128640 1 -- 172.21.15.139:0/323245267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc006baf0 con 0x7fabc015c3d0 2024-04-25T10:01:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.281+0000 7fabb67fc640 1 -- 172.21.15.139:0/323245267 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabb800ac30 con 0x7fabc015c3d0 2024-04-25T10:01:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.281+0000 7fabb67fc640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faba406b4d0 0x7faba406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.281+0000 7fabb67fc640 1 -- 172.21.15.139:0/323245267 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fabb808a360 con 0x7fabc015c3d0 2024-04-25T10:01:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.281+0000 7fabc5ec4640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faba406b4d0 0x7faba406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.281+0000 7fabc5ec4640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faba406b4d0 0x7faba406d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fabbc006fd0 tx=0x7fabbc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:17.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.285+0000 7fabb67fc640 1 -- 172.21.15.139:0/323245267 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fabb8058fc0 con 0x7fabc015c3d0 2024-04-25T10:01:17.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:17.494+0000 7fabc8128640 1 -- 172.21.15.139:0/323245267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fabc00aefc0 con 0x7fabc015c3d0 2024-04-25T10:01:18.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.167+0000 7fabb67fc640 1 -- 172.21.15.139:0/323245267 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v69) v1 ==== 122+0+0 (secure 0 0 0) 0x7fabb80588c0 con 0x7fabc015c3d0 2024-04-25T10:01:18.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.170+0000 7fabc8128640 1 -- 172.21.15.139:0/323245267 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faba406b4d0 msgr2=0x7faba406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.170+0000 7fabc8128640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faba406b4d0 0x7faba406d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fabbc006fd0 tx=0x7fabbc008040 comp rx=0 tx=0).stop 2024-04-25T10:01:18.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.170+0000 7fabc8128640 1 -- 172.21.15.139:0/323245267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc015c3d0 msgr2=0x7fabc00ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.170+0000 7fabc8128640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc015c3d0 0x7fabc00ae060 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fabb800c7a0 tx=0x7fabb800cc60 comp rx=0 tx=0).stop 2024-04-25T10:01:18.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.170+0000 7fabc8128640 1 -- 172.21.15.139:0/323245267 shutdown_connections 2024-04-25T10:01:18.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.170+0000 7fabc8128640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc015c3d0 0x7fabc00ae060 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.170+0000 7fabc8128640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faba406b4d0 0x7faba406d990 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.170+0000 7fabc8128640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc0145230 0x7fabc00adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.170+0000 7fabc8128640 1 --2- 172.21.15.139:0/323245267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc0071700 0x7fabc0139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.170+0000 7fabc8128640 1 -- 172.21.15.139:0/323245267 >> 172.21.15.139:0/323245267 conn(0x7fabc0067000 msgr2=0x7fabc01440c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:18.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.170+0000 7fabc8128640 1 -- 172.21.15.139:0/323245267 shutdown_connections 2024-04-25T10:01:18.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.170+0000 7fabc8128640 1 -- 172.21.15.139:0/323245267 wait complete. 2024-04-25T10:01:18.187 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:01:18.188 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:18.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.287+0000 7f68e283a640 1 Processor -- start 2024-04-25T10:01:18.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.287+0000 7f68e283a640 1 -- start start 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68e283a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68dc0d1080 0x7f68dc0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68e283a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f68dc161850 0x7f68dc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68e283a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68dc0cf250 con 0x7f68dc15dc60 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68e283a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68dc0ce780 con 0x7f68dc161850 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68e283a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc0d3730 con 0x7f68dc0d1080 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68e0dd7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f68dc161850 0x7f68dc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68dbfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f68dc15dc60 0x7f68dc0d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38358/0 (socket says 172.21.15.139:38358) 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68dbfff640 1 -- 172.21.15.139:0/3866000021 learned_addr learned my addr 172.21.15.139:0/3866000021 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68db7fe640 1 --2- 172.21.15.139:0/3866000021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68dc0d1080 0x7f68dc0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68e0dd7640 1 -- 172.21.15.139:0/3866000021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68dc0d1080 msgr2=0x7f68dc0b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68e0dd7640 1 --2- 172.21.15.139:0/3866000021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68dc0d1080 0x7f68dc0b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68e0dd7640 1 -- 172.21.15.139:0/3866000021 >> v1:172.21.15.139:6789/0 conn(0x7f68dc15dc60 legacy=0x7f68dc0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68daffd640 1 -- 172.21.15.139:0/3866000021 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185523956 0 0) 0x7f68cc001110 con 0x7f68dc15dc60 2024-04-25T10:01:18.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.288+0000 7f68e0dd7640 1 -- 172.21.15.139:0/3866000021 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68dc0ce9b0 con 0x7f68dc161850 2024-04-25T10:01:18.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.289+0000 7f68e0dd7640 1 --2- 172.21.15.139:0/3866000021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68dc161850 0x7f68dc0ce240 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009dc0 tx=0x7f68d402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934cc90adf1520d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:18.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.289+0000 7f68daffd640 1 -- 172.21.15.139:0/3866000021 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d402ae60 con 0x7f68dc161850 2024-04-25T10:01:18.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.289+0000 7f68daffd640 1 -- 172.21.15.139:0/3866000021 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d4007740 con 0x7f68dc161850 2024-04-25T10:01:18.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.289+0000 7f68daffd640 1 -- 172.21.15.139:0/3866000021 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4007b10 con 0x7f68dc161850 2024-04-25T10:01:18.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.290+0000 7f68e283a640 1 -- 172.21.15.139:0/3866000021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68dc161850 msgr2=0x7f68dc0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.290+0000 7f68e283a640 1 --2- 172.21.15.139:0/3866000021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68dc161850 0x7f68dc0ce240 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009dc0 tx=0x7f68d402a680 comp rx=0 tx=0).stop 2024-04-25T10:01:18.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.290+0000 7f68e283a640 1 -- 172.21.15.139:0/3866000021 shutdown_connections 2024-04-25T10:01:18.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.290+0000 7f68e283a640 1 --2- 172.21.15.139:0/3866000021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68dc161850 0x7f68dc0ce240 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.290+0000 7f68e283a640 1 --2- 172.21.15.139:0/3866000021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68dc0d1080 0x7f68dc0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.290+0000 7f68e283a640 1 -- 172.21.15.139:0/3866000021 >> 172.21.15.139:0/3866000021 conn(0x7f68dc0b5910 msgr2=0x7f68dc0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:18.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.290+0000 7f68e283a640 1 -- 172.21.15.139:0/3866000021 shutdown_connections 2024-04-25T10:01:18.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.290+0000 7f68e283a640 1 -- 172.21.15.139:0/3866000021 wait complete. 2024-04-25T10:01:18.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.291+0000 7f68e283a640 1 Processor -- start 2024-04-25T10:01:18.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.291+0000 7f68e283a640 1 -- start start 2024-04-25T10:01:18.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e283a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68dc0d1080 0x7f68dc15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e283a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68dc15dc60 0x7f68dc0b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e283a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68dc161850 0x7f68dc0b8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e283a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc0d1fc0 con 0x7f68dc161850 2024-04-25T10:01:18.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e283a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc0cef70 con 0x7f68dc0d1080 2024-04-25T10:01:18.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e283a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc15e040 con 0x7f68dc15dc60 2024-04-25T10:01:18.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e0dd7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68dc161850 0x7f68dc0b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e0dd7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68dc161850 0x7f68dc0b8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44114/0 (socket says 172.21.15.139:44114) 2024-04-25T10:01:18.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e0dd7640 1 -- 172.21.15.139:0/2066952115 learned_addr learned my addr 172.21.15.139:0/2066952115 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:18.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68db7fe640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68dc15dc60 0x7f68dc0b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68dbfff640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68dc0d1080 0x7f68dc15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e0dd7640 1 -- 172.21.15.139:0/2066952115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68dc15dc60 msgr2=0x7f68dc0b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e0dd7640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68dc15dc60 0x7f68dc0b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e0dd7640 1 -- 172.21.15.139:0/2066952115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68dc0d1080 msgr2=0x7f68dc15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e0dd7640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68dc0d1080 0x7f68dc15f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68e0dd7640 1 -- 172.21.15.139:0/2066952115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68dc15e670 con 0x7f68dc161850 2024-04-25T10:01:18.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.292+0000 7f68db7fe640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68dc15dc60 0x7f68dc0b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:18.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.293+0000 7f68e0dd7640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68dc161850 0x7f68dc0b8160 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f68d402ac70 tx=0x7f68d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:18.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.293+0000 7f68d8ff9640 1 -- 172.21.15.139:0/2066952115 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4040070 con 0x7f68dc161850 2024-04-25T10:01:18.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.293+0000 7f68d8ff9640 1 -- 172.21.15.139:0/2066952115 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d402ae60 con 0x7f68dc161850 2024-04-25T10:01:18.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.293+0000 7f68d8ff9640 1 -- 172.21.15.139:0/2066952115 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4045460 con 0x7f68dc161850 2024-04-25T10:01:18.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.293+0000 7f68e283a640 1 -- 172.21.15.139:0/2066952115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68dc15e950 con 0x7f68dc161850 2024-04-25T10:01:18.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.293+0000 7f68e283a640 1 -- 172.21.15.139:0/2066952115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68dc15ef10 con 0x7f68dc161850 2024-04-25T10:01:18.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.294+0000 7f68e283a640 1 -- 172.21.15.139:0/2066952115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68dc074520 con 0x7f68dc161850 2024-04-25T10:01:18.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.294+0000 7f68d8ff9640 1 -- 172.21.15.139:0/2066952115 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68d4039040 con 0x7f68dc161850 2024-04-25T10:01:18.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.295+0000 7f68d8ff9640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f68c806b4d0 0x7f68c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.295+0000 7f68dbfff640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f68c806b4d0 0x7f68c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.295+0000 7f68d8ff9640 1 -- 172.21.15.139:0/2066952115 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68d40abd30 con 0x7f68dc161850 2024-04-25T10:01:18.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.295+0000 7f68dbfff640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f68c806b4d0 0x7f68c806d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f68cc000f60 tx=0x7f68cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:18.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.299+0000 7f68d8ff9640 1 -- 172.21.15.139:0/2066952115 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68d404c020 con 0x7f68dc161850 2024-04-25T10:01:18.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.508+0000 7f68e283a640 1 -- 172.21.15.139:0/2066952115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68dc0d16f0 con 0x7f68dc161850 2024-04-25T10:01:18.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.509+0000 7f68d8ff9640 1 -- 172.21.15.139:0/2066952115 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f68d407aab0 con 0x7f68dc161850 2024-04-25T10:01:18.513 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:18.513 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":69,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:15.497940+0000","modified":"2024-04-25T10:01:17.495730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8466":{"gid":8466,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/4074693991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4074693991},{"type":"v1","addr":"172.21.15.139:6837","nonce":4074693991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:18.513 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 69 2024-04-25T10:01:18.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.510+0000 7f68e283a640 1 -- 172.21.15.139:0/2066952115 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f68c806b4d0 msgr2=0x7f68c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.510+0000 7f68e283a640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f68c806b4d0 0x7f68c806d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f68cc000f60 tx=0x7f68cc009040 comp rx=0 tx=0).stop 2024-04-25T10:01:18.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.511+0000 7f68e283a640 1 -- 172.21.15.139:0/2066952115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68dc161850 msgr2=0x7f68dc0b8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.511+0000 7f68e283a640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68dc161850 0x7f68dc0b8160 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f68d402ac70 tx=0x7f68d4004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:18.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.511+0000 7f68e283a640 1 -- 172.21.15.139:0/2066952115 shutdown_connections 2024-04-25T10:01:18.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.511+0000 7f68e283a640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f68dc161850 0x7f68dc0b8160 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.511+0000 7f68e283a640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f68c806b4d0 0x7f68c806d990 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.511+0000 7f68e283a640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68dc15dc60 0x7f68dc0b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.511+0000 7f68e283a640 1 --2- 172.21.15.139:0/2066952115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68dc0d1080 0x7f68dc15f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.511+0000 7f68e283a640 1 -- 172.21.15.139:0/2066952115 >> 172.21.15.139:0/2066952115 conn(0x7f68dc0b5910 msgr2=0x7f68dc15b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:18.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.511+0000 7f68e283a640 1 -- 172.21.15.139:0/2066952115 shutdown_connections 2024-04-25T10:01:18.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.511+0000 7f68e283a640 1 -- 172.21.15.139:0/2066952115 wait complete. 2024-04-25T10:01:18.526 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:18.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.619+0000 7f79c04ac640 1 Processor -- start 2024-04-25T10:01:18.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.619+0000 7f79c04ac640 1 -- start start 2024-04-25T10:01:18.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.620+0000 7f79c04ac640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79b8155e30 0x7f79b815cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.620+0000 7f79c04ac640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f79b80b1e50 0x7f79b815c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.620+0000 7f79c04ac640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b815c880 con 0x7f79b815bdc0 2024-04-25T10:01:18.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.620+0000 7f79c04ac640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b806d6f0 con 0x7f79b80b1e50 2024-04-25T10:01:18.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.620+0000 7f79c04ac640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b815cb60 con 0x7f79b8155e30 2024-04-25T10:01:18.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.620+0000 7f79bea49640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f79b80b1e50 0x7f79b815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.620+0000 7f79bea49640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f79b80b1e50 0x7f79b815c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44116/0 (socket says 172.21.15.139:44116) 2024-04-25T10:01:18.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.620+0000 7f79bea49640 1 -- 172.21.15.139:0/3625422433 learned_addr learned my addr 172.21.15.139:0/3625422433 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:18.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.620+0000 7f79be248640 1 --2- 172.21.15.139:0/3625422433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79b8155e30 0x7f79b815cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.621+0000 7f79bea49640 1 -- 172.21.15.139:0/3625422433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79b8155e30 msgr2=0x7f79b815cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:18.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.621+0000 7f79bea49640 1 --2- 172.21.15.139:0/3625422433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79b8155e30 0x7f79b815cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.621+0000 7f79bea49640 1 -- 172.21.15.139:0/3625422433 >> v1:172.21.15.150:6789/0 conn(0x7f79b815bdc0 legacy=0x7f79b80b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.621+0000 7f79bea49640 1 -- 172.21.15.139:0/3625422433 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b806d380 con 0x7f79b80b1e50 2024-04-25T10:01:18.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.621+0000 7f79be248640 1 --2- 172.21.15.139:0/3625422433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79b8155e30 0x7f79b815cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:18.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.621+0000 7f79bea49640 1 --2- 172.21.15.139:0/3625422433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b80b1e50 0x7f79b815c240 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009870 tx=0x7f79b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febd2e2af15330dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:18.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.621+0000 7f79bd246640 1 -- 172.21.15.139:0/3625422433 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0037070 con 0x7f79b80b1e50 2024-04-25T10:01:18.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.622+0000 7f79bd246640 1 -- 172.21.15.139:0/3625422433 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b0032440 con 0x7f79b80b1e50 2024-04-25T10:01:18.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.622+0000 7f79bd246640 1 -- 172.21.15.139:0/3625422433 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0032810 con 0x7f79b80b1e50 2024-04-25T10:01:18.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.622+0000 7f79c04ac640 1 -- 172.21.15.139:0/3625422433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b80b1e50 msgr2=0x7f79b815c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.622+0000 7f79c04ac640 1 --2- 172.21.15.139:0/3625422433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b80b1e50 0x7f79b815c240 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009870 tx=0x7f79b002a270 comp rx=0 tx=0).stop 2024-04-25T10:01:18.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.622+0000 7f79c04ac640 1 -- 172.21.15.139:0/3625422433 shutdown_connections 2024-04-25T10:01:18.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.622+0000 7f79c04ac640 1 --2- 172.21.15.139:0/3625422433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b80b1e50 0x7f79b815c240 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.622+0000 7f79c04ac640 1 --2- 172.21.15.139:0/3625422433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79b8155e30 0x7f79b815cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.622+0000 7f79c04ac640 1 -- 172.21.15.139:0/3625422433 >> 172.21.15.139:0/3625422433 conn(0x7f79b8066f20 msgr2=0x7f79b806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:18.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.622+0000 7f79c04ac640 1 -- 172.21.15.139:0/3625422433 shutdown_connections 2024-04-25T10:01:18.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.622+0000 7f79c04ac640 1 -- 172.21.15.139:0/3625422433 wait complete. 2024-04-25T10:01:18.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.623+0000 7f79c04ac640 1 Processor -- start 2024-04-25T10:01:18.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.623+0000 7f79c04ac640 1 -- start start 2024-04-25T10:01:18.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79c04ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b80b1e50 0x7f79b80aa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79c04ac640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79b8155e30 0x7f79b815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79c04ac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79b815bdc0 0x7f79b8152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79c04ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80af260 con 0x7f79b80b1e50 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79c04ac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80af0e0 con 0x7f79b815bdc0 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79c04ac640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b815d4c0 con 0x7f79b8155e30 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79be248640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b80b1e50 0x7f79b80aa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79be248640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b80b1e50 0x7f79b80aa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44130/0 (socket says 172.21.15.139:44130) 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79be248640 1 -- 172.21.15.139:0/3030825595 learned_addr learned my addr 172.21.15.139:0/3030825595 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79bda47640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79b8155e30 0x7f79b815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79bea49640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79b815bdc0 0x7f79b8152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79be248640 1 -- 172.21.15.139:0/3030825595 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79b8155e30 msgr2=0x7f79b815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79be248640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79b8155e30 0x7f79b815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79be248640 1 -- 172.21.15.139:0/3030825595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79b815bdc0 msgr2=0x7f79b8152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79be248640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79b815bdc0 0x7f79b8152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.624+0000 7f79be248640 1 -- 172.21.15.139:0/3030825595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b80a9290 con 0x7f79b80b1e50 2024-04-25T10:01:18.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.625+0000 7f79be248640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b80b1e50 0x7f79b80aa010 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009fc0 tx=0x7f79b400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:18.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.625+0000 7f79aeffd640 1 -- 172.21.15.139:0/3030825595 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b401a070 con 0x7f79b80b1e50 2024-04-25T10:01:18.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.625+0000 7f79aeffd640 1 -- 172.21.15.139:0/3030825595 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4003bb0 con 0x7f79b80b1e50 2024-04-25T10:01:18.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.625+0000 7f79aeffd640 1 -- 172.21.15.139:0/3030825595 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4004030 con 0x7f79b80b1e50 2024-04-25T10:01:18.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.625+0000 7f79c04ac640 1 -- 172.21.15.139:0/3030825595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b80a9530 con 0x7f79b80b1e50 2024-04-25T10:01:18.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.625+0000 7f79c04ac640 1 -- 172.21.15.139:0/3030825595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b80aa550 con 0x7f79b80b1e50 2024-04-25T10:01:18.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.626+0000 7f79c04ac640 1 -- 172.21.15.139:0/3030825595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b8074520 con 0x7f79b80b1e50 2024-04-25T10:01:18.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.626+0000 7f79aeffd640 1 -- 172.21.15.139:0/3030825595 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79b401d070 con 0x7f79b80b1e50 2024-04-25T10:01:18.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.627+0000 7f79aeffd640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f799c06b500 0x7f799c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.627+0000 7f79aeffd640 1 -- 172.21.15.139:0/3030825595 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79b408a570 con 0x7f79b80b1e50 2024-04-25T10:01:18.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.627+0000 7f79bda47640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f799c06b500 0x7f799c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.627+0000 7f79bda47640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f799c06b500 0x7f799c06d9c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f79b8153d00 tx=0x7f79a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:18.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.631+0000 7f79aeffd640 1 -- 172.21.15.139:0/3030825595 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79b4010070 con 0x7f79b80b1e50 2024-04-25T10:01:18.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.800+0000 7f79c04ac640 1 -- 172.21.15.139:0/3030825595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79b806b830 con 0x7f79b80b1e50 2024-04-25T10:01:18.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.801+0000 7f79aeffd640 1 -- 172.21.15.139:0/3030825595 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f79b4058c10 con 0x7f79b80b1e50 2024-04-25T10:01:18.806 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:18.806 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":105,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:01:17.142273+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:13.695577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:14.460042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:18.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.803+0000 7f79c04ac640 1 -- 172.21.15.139:0/3030825595 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f799c06b500 msgr2=0x7f799c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.803+0000 7f79c04ac640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f799c06b500 0x7f799c06d9c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f79b8153d00 tx=0x7f79a8009040 comp rx=0 tx=0).stop 2024-04-25T10:01:18.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.803+0000 7f79c04ac640 1 -- 172.21.15.139:0/3030825595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b80b1e50 msgr2=0x7f79b80aa010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.803+0000 7f79c04ac640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b80b1e50 0x7f79b80aa010 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009fc0 tx=0x7f79b400d560 comp rx=0 tx=0).stop 2024-04-25T10:01:18.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.803+0000 7f79c04ac640 1 -- 172.21.15.139:0/3030825595 shutdown_connections 2024-04-25T10:01:18.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.803+0000 7f79c04ac640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f799c06b500 0x7f799c06d9c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.803+0000 7f79c04ac640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79b815bdc0 0x7f79b8152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.803+0000 7f79c04ac640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79b8155e30 0x7f79b815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.803+0000 7f79c04ac640 1 --2- 172.21.15.139:0/3030825595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b80b1e50 0x7f79b80aa010 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.803+0000 7f79c04ac640 1 -- 172.21.15.139:0/3030825595 >> 172.21.15.139:0/3030825595 conn(0x7f79b8066f20 msgr2=0x7f79b8154790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:18.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.803+0000 7f79c04ac640 1 -- 172.21.15.139:0/3030825595 shutdown_connections 2024-04-25T10:01:18.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.803+0000 7f79c04ac640 1 -- 172.21.15.139:0/3030825595 wait complete. 2024-04-25T10:01:18.819 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:01:18.819 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:18.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.911+0000 7f8dc7db7640 1 Processor -- start 2024-04-25T10:01:18.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.912+0000 7f8dc7db7640 1 -- start start 2024-04-25T10:01:18.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.912+0000 7f8dc7db7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015dc60 0x7f8dc00b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.912+0000 7f8dc7db7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc00cf250 con 0x7f8dc00d1080 2024-04-25T10:01:18.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.912+0000 7f8dc7db7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc00ce9b0 con 0x7f8dc0161850 2024-04-25T10:01:18.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.912+0000 7f8dc7db7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00d3730 con 0x7f8dc015dc60 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc6354640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8dc00d1080 0x7f8dc00d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38360/0 (socket says 172.21.15.139:38360) 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc6354640 1 -- 172.21.15.139:0/1016333959 learned_addr learned my addr 172.21.15.139:0/1016333959 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc5b53640 1 --2- 172.21.15.139:0/1016333959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015dc60 0x7f8dc00b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674055370 0 0) 0x7f8dbc001110 con 0x7f8dc00d1080 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dac003520 con 0x7f8dc00d1080 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3686268138 0 0) 0x7f8dac003520 con 0x7f8dc00d1080 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015dc60 msgr2=0x7f8dc00b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 --2- 172.21.15.139:0/1016333959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015dc60 0x7f8dc00b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 >> v1:172.21.15.150:6789/0 conn(0x7f8dc0161850 legacy=0x7f8dc00d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc00cec90 con 0x7f8dc00d1080 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f8dbc001840 con 0x7f8dc00d1080 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 >> v1:172.21.15.139:6789/0 conn(0x7f8dc00d1080 legacy=0x7f8dc00d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 --2- 172.21.15.139:0/1016333959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dac0039e0 0x7f8dac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 --2- 172.21.15.139:0/1016333959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015dc60 0x7f8dac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 --2- 172.21.15.139:0/1016333959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dac004d40 0x7f8dac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00d3730 con 0x7f8dac004d40 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac00ba70 con 0x7f8dac0039e0 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac00bc70 con 0x7f8dc015dc60 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc6354640 1 --2- 172.21.15.139:0/1016333959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dac004d40 0x7f8dac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc5b53640 1 --2- 172.21.15.139:0/1016333959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dac0039e0 0x7f8dac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.913+0000 7f8dc5352640 1 --2- 172.21.15.139:0/1016333959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015dc60 0x7f8dac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.914+0000 7f8dc6354640 1 -- 172.21.15.139:0/1016333959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015dc60 msgr2=0x7f8dac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.914+0000 7f8dc6354640 1 --2- 172.21.15.139:0/1016333959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015dc60 0x7f8dac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.914+0000 7f8dc6354640 1 -- 172.21.15.139:0/1016333959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dac0039e0 msgr2=0x7f8dac003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.914+0000 7f8dc6354640 1 --2- 172.21.15.139:0/1016333959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dac0039e0 0x7f8dac003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.914+0000 7f8dc6354640 1 -- 172.21.15.139:0/1016333959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac003520 con 0x7f8dac004d40 2024-04-25T10:01:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.914+0000 7f8dc6354640 1 --2- 172.21.15.139:0/1016333959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dac004d40 0x7f8dac0051c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00c490 tx=0x7f8dac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbec09e41b4e9e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.914+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00c0e0 con 0x7f8dac004d40 2024-04-25T10:01:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.914+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc034070 con 0x7f8dac004d40 2024-04-25T10:01:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.914+0000 7f8dc4b51640 1 -- 172.21.15.139:0/1016333959 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc005d30 con 0x7f8dac004d40 2024-04-25T10:01:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.915+0000 7f8dc7db7640 1 -- 172.21.15.139:0/1016333959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dac004d40 msgr2=0x7f8dac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.915+0000 7f8dc7db7640 1 --2- 172.21.15.139:0/1016333959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dac004d40 0x7f8dac0051c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00c490 tx=0x7f8dac005830 comp rx=0 tx=0).stop 2024-04-25T10:01:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.915+0000 7f8dc6354640 1 -- 172.21.15.139:0/1016333959 reap_dead start 2024-04-25T10:01:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.915+0000 7f8dc7db7640 1 -- 172.21.15.139:0/1016333959 shutdown_connections 2024-04-25T10:01:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.915+0000 7f8dc7db7640 1 -- 172.21.15.139:0/1016333959 >> 172.21.15.139:0/1016333959 conn(0x7f8dc00b5910 msgr2=0x7f8dc00b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.915+0000 7f8dc7db7640 1 -- 172.21.15.139:0/1016333959 shutdown_connections 2024-04-25T10:01:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.915+0000 7f8dc7db7640 1 -- 172.21.15.139:0/1016333959 wait complete. 2024-04-25T10:01:18.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.916+0000 7f8dc7db7640 1 Processor -- start 2024-04-25T10:01:18.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.916+0000 7f8dc7db7640 1 -- start start 2024-04-25T10:01:18.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc7db7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dac004d40 0x7f8dc0158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc7db7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dc015a0b0 0x7f8dc00c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc7db7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015a510 0x7f8dc00caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc7db7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc015e040 con 0x7f8dac004d40 2024-04-25T10:01:18.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc7db7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00d1fc0 con 0x7f8dc015a0b0 2024-04-25T10:01:18.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc7db7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc015abc0 con 0x7f8dc015a510 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc5b53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dac004d40 0x7f8dc0158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc5b53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dac004d40 0x7f8dc0158030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44160/0 (socket says 172.21.15.139:44160) 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc5b53640 1 -- 172.21.15.139:0/197215501 learned_addr learned my addr 172.21.15.139:0/197215501 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc6354640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015a510 0x7f8dc00caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc5352640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dc015a0b0 0x7f8dc00c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc5b53640 1 -- 172.21.15.139:0/197215501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015a510 msgr2=0x7f8dc00caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc5b53640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015a510 0x7f8dc00caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc5b53640 1 -- 172.21.15.139:0/197215501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dc015a0b0 msgr2=0x7f8dc00c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc5b53640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dc015a0b0 0x7f8dc00c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc5b53640 1 -- 172.21.15.139:0/197215501 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc015ae80 con 0x7f8dac004d40 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc6354640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015a510 0x7f8dc00caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.917+0000 7f8dc5b53640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dac004d40 0x7f8dc0158030 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f8db8011e30 tx=0x7f8db800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.918+0000 7f8db67fc640 1 -- 172.21.15.139:0/197215501 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8015630 con 0x7f8dac004d40 2024-04-25T10:01:18.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.918+0000 7f8db67fc640 1 -- 172.21.15.139:0/197215501 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db80157d0 con 0x7f8dac004d40 2024-04-25T10:01:18.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.918+0000 7f8dc7db7640 1 -- 172.21.15.139:0/197215501 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc00c75b0 con 0x7f8dac004d40 2024-04-25T10:01:18.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.918+0000 7f8db67fc640 1 -- 172.21.15.139:0/197215501 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8015ba0 con 0x7f8dac004d40 2024-04-25T10:01:18.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.918+0000 7f8dc7db7640 1 -- 172.21.15.139:0/197215501 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc015b160 con 0x7f8dac004d40 2024-04-25T10:01:18.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.919+0000 7f8dc7db7640 1 -- 172.21.15.139:0/197215501 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc0074520 con 0x7f8dac004d40 2024-04-25T10:01:18.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.919+0000 7f8db67fc640 1 -- 172.21.15.139:0/197215501 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8db801d400 con 0x7f8dac004d40 2024-04-25T10:01:18.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.919+0000 7f8db67fc640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8da406b4d0 0x7f8da406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:18.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.920+0000 7f8db67fc640 1 -- 172.21.15.139:0/197215501 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8db808b6b0 con 0x7f8dac004d40 2024-04-25T10:01:18.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.920+0000 7f8dc5352640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8da406b4d0 0x7f8da406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:18.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.920+0000 7f8dc5352640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8da406b4d0 0x7f8da406d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00d4400 tx=0x7f8db0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:18.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:18.924+0000 7f8db67fc640 1 -- 172.21.15.139:0/197215501 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8db8019030 con 0x7f8dac004d40 2024-04-25T10:01:19.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.090+0000 7f8dc7db7640 1 -- 172.21.15.139:0/197215501 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8dc00ce0f0 con 0x7f8da406b4d0 2024-04-25T10:01:19.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.096+0000 7f8db67fc640 1 -- 172.21.15.139:0/197215501 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252481 (secure 0 0 0) 0x7f8dc00ce0f0 con 0x7f8da406b4d0 2024-04-25T10:01:19.103 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:19.104 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T10:01:19.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.099+0000 7f8dc7db7640 1 -- 172.21.15.139:0/197215501 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8da406b4d0 msgr2=0x7f8da406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.099+0000 7f8dc7db7640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8da406b4d0 0x7f8da406d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00d4400 tx=0x7f8db0009040 comp rx=0 tx=0).stop 2024-04-25T10:01:19.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.099+0000 7f8dc7db7640 1 -- 172.21.15.139:0/197215501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dac004d40 msgr2=0x7f8dc0158030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.099+0000 7f8dc7db7640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dac004d40 0x7f8dc0158030 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f8db8011e30 tx=0x7f8db800f580 comp rx=0 tx=0).stop 2024-04-25T10:01:19.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.099+0000 7f8dc7db7640 1 -- 172.21.15.139:0/197215501 shutdown_connections 2024-04-25T10:01:19.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.099+0000 7f8dc7db7640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8da406b4d0 0x7f8da406d990 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.099+0000 7f8dc7db7640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc015a510 0x7f8dc00caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.099+0000 7f8dc7db7640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dc015a0b0 0x7f8dc00c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.099+0000 7f8dc7db7640 1 --2- 172.21.15.139:0/197215501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8dac004d40 0x7f8dc0158030 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.099+0000 7f8dc7db7640 1 -- 172.21.15.139:0/197215501 >> 172.21.15.139:0/197215501 conn(0x7f8dc00b5910 msgr2=0x7f8dc00d38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:19.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.099+0000 7f8dc7db7640 1 -- 172.21.15.139:0/197215501 shutdown_connections 2024-04-25T10:01:19.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.099+0000 7f8dc7db7640 1 -- 172.21.15.139:0/197215501 wait complete. 2024-04-25T10:01:19.114 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-04-25T10:01:19.078141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260832,"kb_used_data":11352,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713888,"statfs":{"total":773094113280,"available":772827021312,"internally_reserved":0,"allocated":11624448,"data_stored":5995756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.942624"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148151+0000","last_change":"2024-04-25T10:01:16.163785+0000","last_active":"2024-04-25T10:01:17.148151+0000","last_peered":"2024-04-25T10:01:17.148151+0000","last_clean":"2024-04-25T10:01:17.148151+0000","last_became_active":"2024-04-25T10:01:16.162777+0000","last_became_peered":"2024-04-25T10:01:16.162777+0000","last_unstale":"2024-04-25T10:01:17.148151+0000","last_undegraded":"2024-04-25T10:01:17.148151+0000","last_fullsized":"2024-04-25T10:01:17.148151+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:07.399739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649541+0000","last_change":"2024-04-25T10:01:16.158502+0000","last_active":"2024-04-25T10:01:17.649541+0000","last_peered":"2024-04-25T10:01:17.649541+0000","last_clean":"2024-04-25T10:01:17.649541+0000","last_became_active":"2024-04-25T10:01:16.158218+0000","last_became_peered":"2024-04-25T10:01:16.158218+0000","last_unstale":"2024-04-25T10:01:17.649541+0000","last_undegraded":"2024-04-25T10:01:17.649541+0000","last_fullsized":"2024-04-25T10:01:17.649541+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.767089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148223+0000","last_change":"2024-04-25T10:01:16.164784+0000","last_active":"2024-04-25T10:01:17.148223+0000","last_peered":"2024-04-25T10:01:17.148223+0000","last_clean":"2024-04-25T10:01:17.148223+0000","last_became_active":"2024-04-25T10:01:16.164517+0000","last_became_peered":"2024-04-25T10:01:16.164517+0000","last_unstale":"2024-04-25T10:01:17.148223+0000","last_undegraded":"2024-04-25T10:01:17.148223+0000","last_fullsized":"2024-04-25T10:01:17.148223+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:22.840434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648996+0000","last_change":"2024-04-25T10:01:16.158585+0000","last_active":"2024-04-25T10:01:17.648996+0000","last_peered":"2024-04-25T10:01:17.648996+0000","last_clean":"2024-04-25T10:01:17.648996+0000","last_became_active":"2024-04-25T10:01:16.158240+0000","last_became_peered":"2024-04-25T10:01:16.158240+0000","last_unstale":"2024-04-25T10:01:17.648996+0000","last_undegraded":"2024-04-25T10:01:17.648996+0000","last_fullsized":"2024-04-25T10:01:17.648996+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:29.994543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648782+0000","last_change":"2024-04-25T10:01:16.168633+0000","last_active":"2024-04-25T10:01:17.648782+0000","last_peered":"2024-04-25T10:01:17.648782+0000","last_clean":"2024-04-25T10:01:17.648782+0000","last_became_active":"2024-04-25T10:01:16.168356+0000","last_became_peered":"2024-04-25T10:01:16.168356+0000","last_unstale":"2024-04-25T10:01:17.648782+0000","last_undegraded":"2024-04-25T10:01:17.648782+0000","last_fullsized":"2024-04-25T10:01:17.648782+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:13.378908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648532+0000","last_change":"2024-04-25T10:01:16.170253+0000","last_active":"2024-04-25T10:01:17.648532+0000","last_peered":"2024-04-25T10:01:17.648532+0000","last_clean":"2024-04-25T10:01:17.648532+0000","last_became_active":"2024-04-25T10:01:16.170000+0000","last_became_peered":"2024-04-25T10:01:16.170000+0000","last_unstale":"2024-04-25T10:01:17.648532+0000","last_undegraded":"2024-04-25T10:01:17.648532+0000","last_fullsized":"2024-04-25T10:01:17.648532+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:14.322434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648348+0000","last_change":"2024-04-25T10:01:16.164765+0000","last_active":"2024-04-25T10:01:17.648348+0000","last_peered":"2024-04-25T10:01:17.648348+0000","last_clean":"2024-04-25T10:01:17.648348+0000","last_became_active":"2024-04-25T10:01:16.164467+0000","last_became_peered":"2024-04-25T10:01:16.164467+0000","last_unstale":"2024-04-25T10:01:17.648348+0000","last_undegraded":"2024-04-25T10:01:17.648348+0000","last_fullsized":"2024-04-25T10:01:17.648348+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:13.378908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.647902+0000","last_change":"2024-04-25T10:01:16.173608+0000","last_active":"2024-04-25T10:01:17.647902+0000","last_peered":"2024-04-25T10:01:17.647902+0000","last_clean":"2024-04-25T10:01:17.647902+0000","last_became_active":"2024-04-25T10:01:16.173217+0000","last_became_peered":"2024-04-25T10:01:16.173217+0000","last_unstale":"2024-04-25T10:01:17.647902+0000","last_undegraded":"2024-04-25T10:01:17.647902+0000","last_fullsized":"2024-04-25T10:01:17.647902+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:55.566324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649152+0000","last_change":"2024-04-25T10:01:16.164769+0000","last_active":"2024-04-25T10:01:17.649152+0000","last_peered":"2024-04-25T10:01:17.649152+0000","last_clean":"2024-04-25T10:01:17.649152+0000","last_became_active":"2024-04-25T10:01:16.164505+0000","last_became_peered":"2024-04-25T10:01:16.164505+0000","last_unstale":"2024-04-25T10:01:17.649152+0000","last_undegraded":"2024-04-25T10:01:17.649152+0000","last_fullsized":"2024-04-25T10:01:17.649152+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:21.703614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648559+0000","last_change":"2024-04-25T10:01:16.165186+0000","last_active":"2024-04-25T10:01:17.648559+0000","last_peered":"2024-04-25T10:01:17.648559+0000","last_clean":"2024-04-25T10:01:17.648559+0000","last_became_active":"2024-04-25T10:01:16.164911+0000","last_became_peered":"2024-04-25T10:01:16.164911+0000","last_unstale":"2024-04-25T10:01:17.648559+0000","last_undegraded":"2024-04-25T10:01:17.648559+0000","last_fullsized":"2024-04-25T10:01:17.648559+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:02.786410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649068+0000","last_change":"2024-04-25T10:01:16.163787+0000","last_active":"2024-04-25T10:01:17.649068+0000","last_peered":"2024-04-25T10:01:17.649068+0000","last_clean":"2024-04-25T10:01:17.649068+0000","last_became_active":"2024-04-25T10:01:16.162437+0000","last_became_peered":"2024-04-25T10:01:16.162437+0000","last_unstale":"2024-04-25T10:01:17.649068+0000","last_undegraded":"2024-04-25T10:01:17.649068+0000","last_fullsized":"2024-04-25T10:01:17.649068+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:19.369840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147472+0000","last_change":"2024-04-25T10:01:16.160696+0000","last_active":"2024-04-25T10:01:17.147472+0000","last_peered":"2024-04-25T10:01:17.147472+0000","last_clean":"2024-04-25T10:01:17.147472+0000","last_became_active":"2024-04-25T10:01:16.160445+0000","last_became_peered":"2024-04-25T10:01:16.160445+0000","last_unstale":"2024-04-25T10:01:17.147472+0000","last_undegraded":"2024-04-25T10:01:17.147472+0000","last_fullsized":"2024-04-25T10:01:17.147472+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:51.790891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648843+0000","last_change":"2024-04-25T10:01:16.170430+0000","last_active":"2024-04-25T10:01:17.648843+0000","last_peered":"2024-04-25T10:01:17.648843+0000","last_clean":"2024-04-25T10:01:17.648843+0000","last_became_active":"2024-04-25T10:01:16.170187+0000","last_became_peered":"2024-04-25T10:01:16.170187+0000","last_unstale":"2024-04-25T10:01:17.648843+0000","last_undegraded":"2024-04-25T10:01:17.648843+0000","last_fullsized":"2024-04-25T10:01:17.648843+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:34.259379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168898+0000","last_change":"2024-04-25T10:01:16.159772+0000","last_active":"2024-04-25T10:01:17.168898+0000","last_peered":"2024-04-25T10:01:17.168898+0000","last_clean":"2024-04-25T10:01:17.168898+0000","last_became_active":"2024-04-25T10:01:16.159492+0000","last_became_peered":"2024-04-25T10:01:16.159492+0000","last_unstale":"2024-04-25T10:01:17.168898+0000","last_undegraded":"2024-04-25T10:01:17.168898+0000","last_fullsized":"2024-04-25T10:01:17.168898+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:24.536283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168918+0000","last_change":"2024-04-25T10:01:16.164618+0000","last_active":"2024-04-25T10:01:17.168918+0000","last_peered":"2024-04-25T10:01:17.168918+0000","last_clean":"2024-04-25T10:01:17.168918+0000","last_became_active":"2024-04-25T10:01:16.164313+0000","last_became_peered":"2024-04-25T10:01:16.164313+0000","last_unstale":"2024-04-25T10:01:17.168918+0000","last_undegraded":"2024-04-25T10:01:17.168918+0000","last_fullsized":"2024-04-25T10:01:17.168918+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:25.170660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168815+0000","last_change":"2024-04-25T10:01:16.158552+0000","last_active":"2024-04-25T10:01:17.168815+0000","last_peered":"2024-04-25T10:01:17.168815+0000","last_clean":"2024-04-25T10:01:17.168815+0000","last_became_active":"2024-04-25T10:01:16.158307+0000","last_became_peered":"2024-04-25T10:01:16.158307+0000","last_unstale":"2024-04-25T10:01:17.168815+0000","last_undegraded":"2024-04-25T10:01:17.168815+0000","last_fullsized":"2024-04-25T10:01:17.168815+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:16.513283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148274+0000","last_change":"2024-04-25T10:01:16.165783+0000","last_active":"2024-04-25T10:01:17.148274+0000","last_peered":"2024-04-25T10:01:17.148274+0000","last_clean":"2024-04-25T10:01:17.148274+0000","last_became_active":"2024-04-25T10:01:16.165221+0000","last_became_peered":"2024-04-25T10:01:16.165221+0000","last_unstale":"2024-04-25T10:01:17.148274+0000","last_undegraded":"2024-04-25T10:01:17.148274+0000","last_fullsized":"2024-04-25T10:01:17.148274+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:33.757510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649541+0000","last_change":"2024-04-25T10:01:16.162082+0000","last_active":"2024-04-25T10:01:17.649541+0000","last_peered":"2024-04-25T10:01:17.649541+0000","last_clean":"2024-04-25T10:01:17.649541+0000","last_became_active":"2024-04-25T10:01:16.161809+0000","last_became_peered":"2024-04-25T10:01:16.161809+0000","last_unstale":"2024-04-25T10:01:17.649541+0000","last_undegraded":"2024-04-25T10:01:17.649541+0000","last_fullsized":"2024-04-25T10:01:17.649541+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:22.855842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148494+0000","last_change":"2024-04-25T10:01:16.162360+0000","last_active":"2024-04-25T10:01:17.148494+0000","last_peered":"2024-04-25T10:01:17.148494+0000","last_clean":"2024-04-25T10:01:17.148494+0000","last_became_active":"2024-04-25T10:01:16.162079+0000","last_became_peered":"2024-04-25T10:01:16.162079+0000","last_unstale":"2024-04-25T10:01:17.148494+0000","last_undegraded":"2024-04-25T10:01:17.148494+0000","last_fullsized":"2024-04-25T10:01:17.148494+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:07.663121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147555+0000","last_change":"2024-04-25T10:01:16.152957+0000","last_active":"2024-04-25T10:01:17.147555+0000","last_peered":"2024-04-25T10:01:17.147555+0000","last_clean":"2024-04-25T10:01:17.147555+0000","last_became_active":"2024-04-25T10:01:16.152669+0000","last_became_peered":"2024-04-25T10:01:16.152669+0000","last_unstale":"2024-04-25T10:01:17.147555+0000","last_undegraded":"2024-04-25T10:01:17.147555+0000","last_fullsized":"2024-04-25T10:01:17.147555+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:45.068283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648828+0000","last_change":"2024-04-25T10:01:16.167706+0000","last_active":"2024-04-25T10:01:17.648828+0000","last_peered":"2024-04-25T10:01:17.648828+0000","last_clean":"2024-04-25T10:01:17.648828+0000","last_became_active":"2024-04-25T10:01:16.167465+0000","last_became_peered":"2024-04-25T10:01:16.167465+0000","last_unstale":"2024-04-25T10:01:17.648828+0000","last_undegraded":"2024-04-25T10:01:17.648828+0000","last_fullsized":"2024-04-25T10:01:17.648828+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:50.198545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648655+0000","last_change":"2024-04-25T10:01:16.164411+0000","last_active":"2024-04-25T10:01:17.648655+0000","last_peered":"2024-04-25T10:01:17.648655+0000","last_clean":"2024-04-25T10:01:17.648655+0000","last_became_active":"2024-04-25T10:01:16.164162+0000","last_became_peered":"2024-04-25T10:01:16.164162+0000","last_unstale":"2024-04-25T10:01:17.648655+0000","last_undegraded":"2024-04-25T10:01:17.648655+0000","last_fullsized":"2024-04-25T10:01:17.648655+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:57.633684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147966+0000","last_change":"2024-04-25T10:01:16.152924+0000","last_active":"2024-04-25T10:01:17.147966+0000","last_peered":"2024-04-25T10:01:17.147966+0000","last_clean":"2024-04-25T10:01:17.147966+0000","last_became_active":"2024-04-25T10:01:16.152646+0000","last_became_peered":"2024-04-25T10:01:16.152646+0000","last_unstale":"2024-04-25T10:01:17.147966+0000","last_undegraded":"2024-04-25T10:01:17.147966+0000","last_fullsized":"2024-04-25T10:01:17.147966+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:42.718579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648220+0000","last_change":"2024-04-25T10:01:16.168058+0000","last_active":"2024-04-25T10:01:17.648220+0000","last_peered":"2024-04-25T10:01:17.648220+0000","last_clean":"2024-04-25T10:01:17.648220+0000","last_became_active":"2024-04-25T10:01:16.167791+0000","last_became_peered":"2024-04-25T10:01:16.167791+0000","last_unstale":"2024-04-25T10:01:17.648220+0000","last_undegraded":"2024-04-25T10:01:17.648220+0000","last_fullsized":"2024-04-25T10:01:17.648220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:24.529960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147585+0000","last_change":"2024-04-25T10:01:16.167296+0000","last_active":"2024-04-25T10:01:17.147585+0000","last_peered":"2024-04-25T10:01:17.147585+0000","last_clean":"2024-04-25T10:01:17.147585+0000","last_became_active":"2024-04-25T10:01:16.167008+0000","last_became_peered":"2024-04-25T10:01:16.167008+0000","last_unstale":"2024-04-25T10:01:17.147585+0000","last_undegraded":"2024-04-25T10:01:17.147585+0000","last_fullsized":"2024-04-25T10:01:17.147585+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:22.855842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147580+0000","last_change":"2024-04-25T10:01:16.161419+0000","last_active":"2024-04-25T10:01:17.147580+0000","last_peered":"2024-04-25T10:01:17.147580+0000","last_clean":"2024-04-25T10:01:17.147580+0000","last_became_active":"2024-04-25T10:01:16.161157+0000","last_became_peered":"2024-04-25T10:01:16.161157+0000","last_unstale":"2024-04-25T10:01:17.147580+0000","last_undegraded":"2024-04-25T10:01:17.147580+0000","last_fullsized":"2024-04-25T10:01:17.147580+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.767089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649086+0000","last_change":"2024-04-25T10:01:16.156875+0000","last_active":"2024-04-25T10:01:17.649086+0000","last_peered":"2024-04-25T10:01:17.649086+0000","last_clean":"2024-04-25T10:01:17.649086+0000","last_became_active":"2024-04-25T10:01:16.156334+0000","last_became_peered":"2024-04-25T10:01:16.156334+0000","last_unstale":"2024-04-25T10:01:17.649086+0000","last_undegraded":"2024-04-25T10:01:17.649086+0000","last_fullsized":"2024-04-25T10:01:17.649086+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:21.703614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648637+0000","last_change":"2024-04-25T10:01:16.166538+0000","last_active":"2024-04-25T10:01:17.648637+0000","last_peered":"2024-04-25T10:01:17.648637+0000","last_clean":"2024-04-25T10:01:17.648637+0000","last_became_active":"2024-04-25T10:01:16.166252+0000","last_became_peered":"2024-04-25T10:01:16.166252+0000","last_unstale":"2024-04-25T10:01:17.648637+0000","last_undegraded":"2024-04-25T10:01:17.648637+0000","last_fullsized":"2024-04-25T10:01:17.648637+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:11.683435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648958+0000","last_change":"2024-04-25T10:01:16.165276+0000","last_active":"2024-04-25T10:01:17.648958+0000","last_peered":"2024-04-25T10:01:17.648958+0000","last_clean":"2024-04-25T10:01:17.648958+0000","last_became_active":"2024-04-25T10:01:16.164998+0000","last_became_peered":"2024-04-25T10:01:16.164998+0000","last_unstale":"2024-04-25T10:01:17.648958+0000","last_undegraded":"2024-04-25T10:01:17.648958+0000","last_fullsized":"2024-04-25T10:01:17.648958+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:46.200041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147000+0000","last_change":"2024-04-25T10:01:16.166890+0000","last_active":"2024-04-25T10:01:17.147000+0000","last_peered":"2024-04-25T10:01:17.147000+0000","last_clean":"2024-04-25T10:01:17.147000+0000","last_became_active":"2024-04-25T10:01:16.166576+0000","last_became_peered":"2024-04-25T10:01:16.166576+0000","last_unstale":"2024-04-25T10:01:17.147000+0000","last_undegraded":"2024-04-25T10:01:17.147000+0000","last_fullsized":"2024-04-25T10:01:17.147000+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:19.369840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649405+0000","last_change":"2024-04-25T10:01:16.160703+0000","last_active":"2024-04-25T10:01:17.649405+0000","last_peered":"2024-04-25T10:01:17.649405+0000","last_clean":"2024-04-25T10:01:17.649405+0000","last_became_active":"2024-04-25T10:01:16.160389+0000","last_became_peered":"2024-04-25T10:01:16.160389+0000","last_unstale":"2024-04-25T10:01:17.649405+0000","last_undegraded":"2024-04-25T10:01:17.649405+0000","last_fullsized":"2024-04-25T10:01:17.649405+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:45.068283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648172+0000","last_change":"2024-04-25T10:01:16.165352+0000","last_active":"2024-04-25T10:01:17.648172+0000","last_peered":"2024-04-25T10:01:17.648172+0000","last_clean":"2024-04-25T10:01:17.648172+0000","last_became_active":"2024-04-25T10:01:16.164980+0000","last_became_peered":"2024-04-25T10:01:16.164980+0000","last_unstale":"2024-04-25T10:01:17.648172+0000","last_undegraded":"2024-04-25T10:01:17.648172+0000","last_fullsized":"2024-04-25T10:01:17.648172+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:03.984018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148074+0000","last_change":"2024-04-25T10:01:16.157210+0000","last_active":"2024-04-25T10:01:17.148074+0000","last_peered":"2024-04-25T10:01:17.148074+0000","last_clean":"2024-04-25T10:01:17.148074+0000","last_became_active":"2024-04-25T10:01:16.156635+0000","last_became_peered":"2024-04-25T10:01:16.156635+0000","last_unstale":"2024-04-25T10:01:17.148074+0000","last_undegraded":"2024-04-25T10:01:17.148074+0000","last_fullsized":"2024-04-25T10:01:17.148074+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:57.589958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"104'4","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649106+0000","last_change":"2024-04-25T10:01:15.161914+0000","last_active":"2024-04-25T10:01:17.649106+0000","last_peered":"2024-04-25T10:01:17.649106+0000","last_clean":"2024-04-25T10:01:17.649106+0000","last_became_active":"2024-04-25T10:01:15.161644+0000","last_became_peered":"2024-04-25T10:01:15.161644+0000","last_unstale":"2024-04-25T10:01:17.649106+0000","last_undegraded":"2024-04-25T10:01:17.649106+0000","last_fullsized":"2024-04-25T10:01:17.649106+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:18.364153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648378+0000","last_change":"2024-04-25T10:01:16.174563+0000","last_active":"2024-04-25T10:01:17.648378+0000","last_peered":"2024-04-25T10:01:17.648378+0000","last_clean":"2024-04-25T10:01:17.648378+0000","last_became_active":"2024-04-25T10:01:16.174290+0000","last_became_peered":"2024-04-25T10:01:16.174290+0000","last_unstale":"2024-04-25T10:01:17.648378+0000","last_undegraded":"2024-04-25T10:01:17.648378+0000","last_fullsized":"2024-04-25T10:01:17.648378+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:21.126022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648333+0000","last_change":"2024-04-25T10:01:15.158147+0000","last_active":"2024-04-25T10:01:17.648333+0000","last_peered":"2024-04-25T10:01:17.648333+0000","last_clean":"2024-04-25T10:01:17.648333+0000","last_became_active":"2024-04-25T10:01:15.157874+0000","last_became_peered":"2024-04-25T10:01:15.157874+0000","last_unstale":"2024-04-25T10:01:17.648333+0000","last_undegraded":"2024-04-25T10:01:17.648333+0000","last_fullsized":"2024-04-25T10:01:17.648333+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:52.995850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147183+0000","last_change":"2024-04-25T10:01:16.159799+0000","last_active":"2024-04-25T10:01:17.147183+0000","last_peered":"2024-04-25T10:01:17.147183+0000","last_clean":"2024-04-25T10:01:17.147183+0000","last_became_active":"2024-04-25T10:01:16.159555+0000","last_became_peered":"2024-04-25T10:01:16.159555+0000","last_unstale":"2024-04-25T10:01:17.147183+0000","last_undegraded":"2024-04-25T10:01:17.147183+0000","last_fullsized":"2024-04-25T10:01:17.147183+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:21.703614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147548+0000","last_change":"2024-04-25T10:01:15.156417+0000","last_active":"2024-04-25T10:01:17.147548+0000","last_peered":"2024-04-25T10:01:17.147548+0000","last_clean":"2024-04-25T10:01:17.147548+0000","last_became_active":"2024-04-25T10:01:15.156144+0000","last_became_peered":"2024-04-25T10:01:15.156144+0000","last_unstale":"2024-04-25T10:01:17.147548+0000","last_undegraded":"2024-04-25T10:01:17.147548+0000","last_fullsized":"2024-04-25T10:01:17.147548+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:33.937222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649161+0000","last_change":"2024-04-25T10:01:16.155803+0000","last_active":"2024-04-25T10:01:17.649161+0000","last_peered":"2024-04-25T10:01:17.649161+0000","last_clean":"2024-04-25T10:01:17.649161+0000","last_became_active":"2024-04-25T10:01:16.155462+0000","last_became_peered":"2024-04-25T10:01:16.155462+0000","last_unstale":"2024-04-25T10:01:17.649161+0000","last_undegraded":"2024-04-25T10:01:17.649161+0000","last_fullsized":"2024-04-25T10:01:17.649161+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:51.790891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648491+0000","last_change":"2024-04-25T10:01:15.154241+0000","last_active":"2024-04-25T10:01:17.648491+0000","last_peered":"2024-04-25T10:01:17.648491+0000","last_clean":"2024-04-25T10:01:17.648491+0000","last_became_active":"2024-04-25T10:01:15.153838+0000","last_became_peered":"2024-04-25T10:01:15.153838+0000","last_unstale":"2024-04-25T10:01:17.648491+0000","last_undegraded":"2024-04-25T10:01:17.648491+0000","last_fullsized":"2024-04-25T10:01:17.648491+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:26.356231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649166+0000","last_change":"2024-04-25T10:01:16.169953+0000","last_active":"2024-04-25T10:01:17.649166+0000","last_peered":"2024-04-25T10:01:17.649166+0000","last_clean":"2024-04-25T10:01:17.649166+0000","last_became_active":"2024-04-25T10:01:16.169692+0000","last_became_peered":"2024-04-25T10:01:16.169692+0000","last_unstale":"2024-04-25T10:01:17.649166+0000","last_undegraded":"2024-04-25T10:01:17.649166+0000","last_fullsized":"2024-04-25T10:01:17.649166+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:28.394593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168909+0000","last_change":"2024-04-25T10:01:15.161688+0000","last_active":"2024-04-25T10:01:17.168909+0000","last_peered":"2024-04-25T10:01:17.168909+0000","last_clean":"2024-04-25T10:01:17.168909+0000","last_became_active":"2024-04-25T10:01:15.161376+0000","last_became_peered":"2024-04-25T10:01:15.161376+0000","last_unstale":"2024-04-25T10:01:17.168909+0000","last_undegraded":"2024-04-25T10:01:17.168909+0000","last_fullsized":"2024-04-25T10:01:17.168909+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:27.388906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148470+0000","last_change":"2024-04-25T10:01:16.157485+0000","last_active":"2024-04-25T10:01:17.148470+0000","last_peered":"2024-04-25T10:01:17.148470+0000","last_clean":"2024-04-25T10:01:17.148470+0000","last_became_active":"2024-04-25T10:01:16.156967+0000","last_became_peered":"2024-04-25T10:01:16.156967+0000","last_unstale":"2024-04-25T10:01:17.148470+0000","last_undegraded":"2024-04-25T10:01:17.148470+0000","last_fullsized":"2024-04-25T10:01:17.148470+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:43.158723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649025+0000","last_change":"2024-04-25T10:01:15.159782+0000","last_active":"2024-04-25T10:01:17.649025+0000","last_peered":"2024-04-25T10:01:17.649025+0000","last_clean":"2024-04-25T10:01:17.649025+0000","last_became_active":"2024-04-25T10:01:15.159489+0000","last_became_peered":"2024-04-25T10:01:15.159489+0000","last_unstale":"2024-04-25T10:01:17.649025+0000","last_undegraded":"2024-04-25T10:01:17.649025+0000","last_fullsized":"2024-04-25T10:01:17.649025+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:57.232174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649655+0000","last_change":"2024-04-25T10:01:16.163364+0000","last_active":"2024-04-25T10:01:17.649655+0000","last_peered":"2024-04-25T10:01:17.649655+0000","last_clean":"2024-04-25T10:01:17.649655+0000","last_became_active":"2024-04-25T10:01:16.163019+0000","last_became_peered":"2024-04-25T10:01:16.163019+0000","last_unstale":"2024-04-25T10:01:17.649655+0000","last_undegraded":"2024-04-25T10:01:17.649655+0000","last_fullsized":"2024-04-25T10:01:17.649655+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:35.487895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.146769+0000","last_change":"2024-04-25T10:01:15.156689+0000","last_active":"2024-04-25T10:01:17.146769+0000","last_peered":"2024-04-25T10:01:17.146769+0000","last_clean":"2024-04-25T10:01:17.146769+0000","last_became_active":"2024-04-25T10:01:15.155419+0000","last_became_peered":"2024-04-25T10:01:15.155419+0000","last_unstale":"2024-04-25T10:01:17.146769+0000","last_undegraded":"2024-04-25T10:01:17.146769+0000","last_fullsized":"2024-04-25T10:01:17.146769+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:57.232174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649139+0000","last_change":"2024-04-25T10:01:16.167548+0000","last_active":"2024-04-25T10:01:17.649139+0000","last_peered":"2024-04-25T10:01:17.649139+0000","last_clean":"2024-04-25T10:01:17.649139+0000","last_became_active":"2024-04-25T10:01:16.167295+0000","last_became_peered":"2024-04-25T10:01:16.167295+0000","last_unstale":"2024-04-25T10:01:17.649139+0000","last_undegraded":"2024-04-25T10:01:17.649139+0000","last_fullsized":"2024-04-25T10:01:17.649139+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:22.532819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.647898+0000","last_change":"2024-04-25T10:01:15.156256+0000","last_active":"2024-04-25T10:01:17.647898+0000","last_peered":"2024-04-25T10:01:17.647898+0000","last_clean":"2024-04-25T10:01:17.647898+0000","last_became_active":"2024-04-25T10:01:15.155956+0000","last_became_peered":"2024-04-25T10:01:15.155956+0000","last_unstale":"2024-04-25T10:01:17.647898+0000","last_undegraded":"2024-04-25T10:01:17.647898+0000","last_fullsized":"2024-04-25T10:01:17.647898+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:10.677748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147930+0000","last_change":"2024-04-25T10:01:16.157814+0000","last_active":"2024-04-25T10:01:17.147930+0000","last_peered":"2024-04-25T10:01:17.147930+0000","last_clean":"2024-04-25T10:01:17.147930+0000","last_became_active":"2024-04-25T10:01:16.157032+0000","last_became_peered":"2024-04-25T10:01:16.157032+0000","last_unstale":"2024-04-25T10:01:17.147930+0000","last_undegraded":"2024-04-25T10:01:17.147930+0000","last_fullsized":"2024-04-25T10:01:17.147930+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:29.459037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147589+0000","last_change":"2024-04-25T10:01:15.156903+0000","last_active":"2024-04-25T10:01:17.147589+0000","last_peered":"2024-04-25T10:01:17.147589+0000","last_clean":"2024-04-25T10:01:17.147589+0000","last_became_active":"2024-04-25T10:01:15.156618+0000","last_became_peered":"2024-04-25T10:01:15.156618+0000","last_unstale":"2024-04-25T10:01:17.147589+0000","last_undegraded":"2024-04-25T10:01:17.147589+0000","last_fullsized":"2024-04-25T10:01:17.147589+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:57.768353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648111+0000","last_change":"2024-04-25T10:01:16.173197+0000","last_active":"2024-04-25T10:01:17.648111+0000","last_peered":"2024-04-25T10:01:17.648111+0000","last_clean":"2024-04-25T10:01:17.648111+0000","last_became_active":"2024-04-25T10:01:16.172938+0000","last_became_peered":"2024-04-25T10:01:16.172938+0000","last_unstale":"2024-04-25T10:01:17.648111+0000","last_undegraded":"2024-04-25T10:01:17.648111+0000","last_fullsized":"2024-04-25T10:01:17.648111+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:14.346871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148015+0000","last_change":"2024-04-25T10:01:15.151057+0000","last_active":"2024-04-25T10:01:17.148015+0000","last_peered":"2024-04-25T10:01:17.148015+0000","last_clean":"2024-04-25T10:01:17.148015+0000","last_became_active":"2024-04-25T10:01:15.150785+0000","last_became_peered":"2024-04-25T10:01:15.150785+0000","last_unstale":"2024-04-25T10:01:17.148015+0000","last_undegraded":"2024-04-25T10:01:17.148015+0000","last_fullsized":"2024-04-25T10:01:17.148015+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:19.036399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648434+0000","last_change":"2024-04-25T10:01:16.177934+0000","last_active":"2024-04-25T10:01:17.648434+0000","last_peered":"2024-04-25T10:01:17.648434+0000","last_clean":"2024-04-25T10:01:17.648434+0000","last_became_active":"2024-04-25T10:01:16.177646+0000","last_became_peered":"2024-04-25T10:01:16.177646+0000","last_unstale":"2024-04-25T10:01:17.648434+0000","last_undegraded":"2024-04-25T10:01:17.648434+0000","last_fullsized":"2024-04-25T10:01:17.648434+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:22.516250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"104'3","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168983+0000","last_change":"2024-04-25T10:01:15.162131+0000","last_active":"2024-04-25T10:01:17.168983+0000","last_peered":"2024-04-25T10:01:17.168983+0000","last_clean":"2024-04-25T10:01:17.168983+0000","last_became_active":"2024-04-25T10:01:15.161850+0000","last_became_peered":"2024-04-25T10:01:15.161850+0000","last_unstale":"2024-04-25T10:01:17.168983+0000","last_undegraded":"2024-04-25T10:01:17.168983+0000","last_fullsized":"2024-04-25T10:01:17.168983+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:28.988856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648926+0000","last_change":"2024-04-25T10:01:16.173895+0000","last_active":"2024-04-25T10:01:17.648926+0000","last_peered":"2024-04-25T10:01:17.648926+0000","last_clean":"2024-04-25T10:01:17.648926+0000","last_became_active":"2024-04-25T10:01:16.173617+0000","last_became_peered":"2024-04-25T10:01:16.173617+0000","last_unstale":"2024-04-25T10:01:17.648926+0000","last_undegraded":"2024-04-25T10:01:17.648926+0000","last_fullsized":"2024-04-25T10:01:17.648926+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:11.683435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"104'6","reported_seq":20,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649336+0000","last_change":"2024-04-25T10:01:15.162469+0000","last_active":"2024-04-25T10:01:17.649336+0000","last_peered":"2024-04-25T10:01:17.649336+0000","last_clean":"2024-04-25T10:01:17.649336+0000","last_became_active":"2024-04-25T10:01:15.162150+0000","last_became_peered":"2024-04-25T10:01:15.162150+0000","last_unstale":"2024-04-25T10:01:17.649336+0000","last_undegraded":"2024-04-25T10:01:17.649336+0000","last_fullsized":"2024-04-25T10:01:17.649336+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:44.732558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649086+0000","last_change":"2024-04-25T10:01:16.170184+0000","last_active":"2024-04-25T10:01:17.649086+0000","last_peered":"2024-04-25T10:01:17.649086+0000","last_clean":"2024-04-25T10:01:17.649086+0000","last_became_active":"2024-04-25T10:01:16.169881+0000","last_became_peered":"2024-04-25T10:01:16.169881+0000","last_unstale":"2024-04-25T10:01:17.649086+0000","last_undegraded":"2024-04-25T10:01:17.649086+0000","last_fullsized":"2024-04-25T10:01:17.649086+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:11.683435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648979+0000","last_change":"2024-04-25T10:01:15.163189+0000","last_active":"2024-04-25T10:01:17.648979+0000","last_peered":"2024-04-25T10:01:17.648979+0000","last_clean":"2024-04-25T10:01:17.648979+0000","last_became_active":"2024-04-25T10:01:15.162938+0000","last_became_peered":"2024-04-25T10:01:15.162938+0000","last_unstale":"2024-04-25T10:01:17.648979+0000","last_undegraded":"2024-04-25T10:01:17.648979+0000","last_fullsized":"2024-04-25T10:01:17.648979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:39.351545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649087+0000","last_change":"2024-04-25T10:01:16.160505+0000","last_active":"2024-04-25T10:01:17.649087+0000","last_peered":"2024-04-25T10:01:17.649087+0000","last_clean":"2024-04-25T10:01:17.649087+0000","last_became_active":"2024-04-25T10:01:16.160221+0000","last_became_peered":"2024-04-25T10:01:16.160221+0000","last_unstale":"2024-04-25T10:01:17.649087+0000","last_undegraded":"2024-04-25T10:01:17.649087+0000","last_fullsized":"2024-04-25T10:01:17.649087+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:19.369840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648312+0000","last_change":"2024-04-25T10:01:15.155989+0000","last_active":"2024-04-25T10:01:17.648312+0000","last_peered":"2024-04-25T10:01:17.648312+0000","last_clean":"2024-04-25T10:01:17.648312+0000","last_became_active":"2024-04-25T10:01:15.155666+0000","last_became_peered":"2024-04-25T10:01:15.155666+0000","last_unstale":"2024-04-25T10:01:17.648312+0000","last_undegraded":"2024-04-25T10:01:17.648312+0000","last_fullsized":"2024-04-25T10:01:17.648312+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:13.316747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648231+0000","last_change":"2024-04-25T10:01:16.164745+0000","last_active":"2024-04-25T10:01:17.648231+0000","last_peered":"2024-04-25T10:01:17.648231+0000","last_clean":"2024-04-25T10:01:17.648231+0000","last_became_active":"2024-04-25T10:01:16.164129+0000","last_became_peered":"2024-04-25T10:01:16.164129+0000","last_unstale":"2024-04-25T10:01:17.648231+0000","last_undegraded":"2024-04-25T10:01:17.648231+0000","last_fullsized":"2024-04-25T10:01:17.648231+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:26.671115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648043+0000","last_change":"2024-04-25T10:01:15.154866+0000","last_active":"2024-04-25T10:01:17.648043+0000","last_peered":"2024-04-25T10:01:17.648043+0000","last_clean":"2024-04-25T10:01:17.648043+0000","last_became_active":"2024-04-25T10:01:15.154568+0000","last_became_peered":"2024-04-25T10:01:15.154568+0000","last_unstale":"2024-04-25T10:01:17.648043+0000","last_undegraded":"2024-04-25T10:01:17.648043+0000","last_fullsized":"2024-04-25T10:01:17.648043+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:21.789012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148397+0000","last_change":"2024-04-25T10:01:16.165952+0000","last_active":"2024-04-25T10:01:17.148397+0000","last_peered":"2024-04-25T10:01:17.148397+0000","last_clean":"2024-04-25T10:01:17.148397+0000","last_became_active":"2024-04-25T10:01:16.165530+0000","last_became_peered":"2024-04-25T10:01:16.165530+0000","last_unstale":"2024-04-25T10:01:17.148397+0000","last_undegraded":"2024-04-25T10:01:17.148397+0000","last_fullsized":"2024-04-25T10:01:17.148397+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:24.218627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"104'8","reported_seq":22,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.146797+0000","last_change":"2024-04-25T10:01:15.155813+0000","last_active":"2024-04-25T10:01:17.146797+0000","last_peered":"2024-04-25T10:01:17.146797+0000","last_clean":"2024-04-25T10:01:17.146797+0000","last_became_active":"2024-04-25T10:01:15.155138+0000","last_became_peered":"2024-04-25T10:01:15.155138+0000","last_unstale":"2024-04-25T10:01:17.146797+0000","last_undegraded":"2024-04-25T10:01:17.146797+0000","last_fullsized":"2024-04-25T10:01:17.146797+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:47.363045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648091+0000","last_change":"2024-04-25T10:01:16.168349+0000","last_active":"2024-04-25T10:01:17.648091+0000","last_peered":"2024-04-25T10:01:17.648091+0000","last_clean":"2024-04-25T10:01:17.648091+0000","last_became_active":"2024-04-25T10:01:16.168104+0000","last_became_peered":"2024-04-25T10:01:16.168104+0000","last_unstale":"2024-04-25T10:01:17.648091+0000","last_undegraded":"2024-04-25T10:01:17.648091+0000","last_fullsized":"2024-04-25T10:01:17.648091+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:58.421904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649469+0000","last_change":"2024-04-25T10:01:15.160924+0000","last_active":"2024-04-25T10:01:17.649469+0000","last_peered":"2024-04-25T10:01:17.649469+0000","last_clean":"2024-04-25T10:01:17.649469+0000","last_became_active":"2024-04-25T10:01:15.160638+0000","last_became_peered":"2024-04-25T10:01:15.160638+0000","last_unstale":"2024-04-25T10:01:17.649469+0000","last_undegraded":"2024-04-25T10:01:17.649469+0000","last_fullsized":"2024-04-25T10:01:17.649469+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:10.215320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649330+0000","last_change":"2024-04-25T10:01:16.168180+0000","last_active":"2024-04-25T10:01:17.649330+0000","last_peered":"2024-04-25T10:01:17.649330+0000","last_clean":"2024-04-25T10:01:17.649330+0000","last_became_active":"2024-04-25T10:01:16.167904+0000","last_became_peered":"2024-04-25T10:01:16.167904+0000","last_unstale":"2024-04-25T10:01:17.649330+0000","last_undegraded":"2024-04-25T10:01:17.649330+0000","last_fullsized":"2024-04-25T10:01:17.649330+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:51.790891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168974+0000","last_change":"2024-04-25T10:01:15.159510+0000","last_active":"2024-04-25T10:01:17.168974+0000","last_peered":"2024-04-25T10:01:17.168974+0000","last_clean":"2024-04-25T10:01:17.168974+0000","last_became_active":"2024-04-25T10:01:15.159244+0000","last_became_peered":"2024-04-25T10:01:15.159244+0000","last_unstale":"2024-04-25T10:01:17.168974+0000","last_undegraded":"2024-04-25T10:01:17.168974+0000","last_fullsized":"2024-04-25T10:01:17.168974+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:21.527132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648878+0000","last_change":"2024-04-25T10:01:16.172708+0000","last_active":"2024-04-25T10:01:17.648878+0000","last_peered":"2024-04-25T10:01:17.648878+0000","last_clean":"2024-04-25T10:01:17.648878+0000","last_became_active":"2024-04-25T10:01:16.172449+0000","last_became_peered":"2024-04-25T10:01:16.172449+0000","last_unstale":"2024-04-25T10:01:17.648878+0000","last_undegraded":"2024-04-25T10:01:17.648878+0000","last_fullsized":"2024-04-25T10:01:17.648878+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:14.322434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649053+0000","last_change":"2024-04-25T10:01:15.161768+0000","last_active":"2024-04-25T10:01:17.649053+0000","last_peered":"2024-04-25T10:01:17.649053+0000","last_clean":"2024-04-25T10:01:17.649053+0000","last_became_active":"2024-04-25T10:01:15.161484+0000","last_became_peered":"2024-04-25T10:01:15.161484+0000","last_unstale":"2024-04-25T10:01:17.649053+0000","last_undegraded":"2024-04-25T10:01:17.649053+0000","last_fullsized":"2024-04-25T10:01:17.649053+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:33.937222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147948+0000","last_change":"2024-04-25T10:01:16.160443+0000","last_active":"2024-04-25T10:01:17.147948+0000","last_peered":"2024-04-25T10:01:17.147948+0000","last_clean":"2024-04-25T10:01:17.147948+0000","last_became_active":"2024-04-25T10:01:16.159853+0000","last_became_peered":"2024-04-25T10:01:16.159853+0000","last_unstale":"2024-04-25T10:01:17.147948+0000","last_undegraded":"2024-04-25T10:01:17.147948+0000","last_fullsized":"2024-04-25T10:01:17.147948+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:47.775162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649030+0000","last_change":"2024-04-25T10:01:15.162292+0000","last_active":"2024-04-25T10:01:17.649030+0000","last_peered":"2024-04-25T10:01:17.649030+0000","last_clean":"2024-04-25T10:01:17.649030+0000","last_became_active":"2024-04-25T10:01:15.161924+0000","last_became_peered":"2024-04-25T10:01:15.161924+0000","last_unstale":"2024-04-25T10:01:17.649030+0000","last_undegraded":"2024-04-25T10:01:17.649030+0000","last_fullsized":"2024-04-25T10:01:17.649030+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:09.493092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148349+0000","last_change":"2024-04-25T10:01:16.160164+0000","last_active":"2024-04-25T10:01:17.148349+0000","last_peered":"2024-04-25T10:01:17.148349+0000","last_clean":"2024-04-25T10:01:17.148349+0000","last_became_active":"2024-04-25T10:01:16.159716+0000","last_became_peered":"2024-04-25T10:01:16.159716+0000","last_unstale":"2024-04-25T10:01:17.148349+0000","last_undegraded":"2024-04-25T10:01:17.148349+0000","last_fullsized":"2024-04-25T10:01:17.148349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:57.627511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648288+0000","last_change":"2024-04-25T10:01:15.162064+0000","last_active":"2024-04-25T10:01:17.648288+0000","last_peered":"2024-04-25T10:01:17.648288+0000","last_clean":"2024-04-25T10:01:17.648288+0000","last_became_active":"2024-04-25T10:01:15.161708+0000","last_became_peered":"2024-04-25T10:01:15.161708+0000","last_unstale":"2024-04-25T10:01:17.648288+0000","last_undegraded":"2024-04-25T10:01:17.648288+0000","last_fullsized":"2024-04-25T10:01:17.648288+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:19.710949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648597+0000","last_change":"2024-04-25T10:01:16.166536+0000","last_active":"2024-04-25T10:01:17.648597+0000","last_peered":"2024-04-25T10:01:17.648597+0000","last_clean":"2024-04-25T10:01:17.648597+0000","last_became_active":"2024-04-25T10:01:16.166239+0000","last_became_peered":"2024-04-25T10:01:16.166239+0000","last_unstale":"2024-04-25T10:01:17.648597+0000","last_undegraded":"2024-04-25T10:01:17.648597+0000","last_fullsized":"2024-04-25T10:01:17.648597+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:33.261097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648446+0000","last_change":"2024-04-25T10:01:15.162917+0000","last_active":"2024-04-25T10:01:17.648446+0000","last_peered":"2024-04-25T10:01:17.648446+0000","last_clean":"2024-04-25T10:01:17.648446+0000","last_became_active":"2024-04-25T10:01:15.162662+0000","last_became_peered":"2024-04-25T10:01:15.162662+0000","last_unstale":"2024-04-25T10:01:17.648446+0000","last_undegraded":"2024-04-25T10:01:17.648446+0000","last_fullsized":"2024-04-25T10:01:17.648446+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:26.356231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648543+0000","last_change":"2024-04-25T10:01:16.172338+0000","last_active":"2024-04-25T10:01:17.648543+0000","last_peered":"2024-04-25T10:01:17.648543+0000","last_clean":"2024-04-25T10:01:17.648543+0000","last_became_active":"2024-04-25T10:01:16.172069+0000","last_became_peered":"2024-04-25T10:01:16.172069+0000","last_unstale":"2024-04-25T10:01:17.648543+0000","last_undegraded":"2024-04-25T10:01:17.648543+0000","last_fullsized":"2024-04-25T10:01:17.648543+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:55.242526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648184+0000","last_change":"2024-04-25T10:01:15.162662+0000","last_active":"2024-04-25T10:01:17.648184+0000","last_peered":"2024-04-25T10:01:17.648184+0000","last_clean":"2024-04-25T10:01:17.648184+0000","last_became_active":"2024-04-25T10:01:15.162249+0000","last_became_peered":"2024-04-25T10:01:15.162249+0000","last_unstale":"2024-04-25T10:01:17.648184+0000","last_undegraded":"2024-04-25T10:01:17.648184+0000","last_fullsized":"2024-04-25T10:01:17.648184+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:53.469647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147930+0000","last_change":"2024-04-25T10:01:16.159845+0000","last_active":"2024-04-25T10:01:17.147930+0000","last_peered":"2024-04-25T10:01:17.147930+0000","last_clean":"2024-04-25T10:01:17.147930+0000","last_became_active":"2024-04-25T10:01:16.159504+0000","last_became_peered":"2024-04-25T10:01:16.159504+0000","last_unstale":"2024-04-25T10:01:17.147930+0000","last_undegraded":"2024-04-25T10:01:17.147930+0000","last_fullsized":"2024-04-25T10:01:17.147930+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:46.644548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648505+0000","last_change":"2024-04-25T10:01:15.162772+0000","last_active":"2024-04-25T10:01:17.648505+0000","last_peered":"2024-04-25T10:01:17.648505+0000","last_clean":"2024-04-25T10:01:17.648505+0000","last_became_active":"2024-04-25T10:01:15.162489+0000","last_became_peered":"2024-04-25T10:01:15.162489+0000","last_unstale":"2024-04-25T10:01:17.648505+0000","last_undegraded":"2024-04-25T10:01:17.648505+0000","last_fullsized":"2024-04-25T10:01:17.648505+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:10.215320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.146798+0000","last_change":"2024-04-25T10:01:16.161385+0000","last_active":"2024-04-25T10:01:17.146798+0000","last_peered":"2024-04-25T10:01:17.146798+0000","last_clean":"2024-04-25T10:01:17.146798+0000","last_became_active":"2024-04-25T10:01:16.158536+0000","last_became_peered":"2024-04-25T10:01:16.158536+0000","last_unstale":"2024-04-25T10:01:17.146798+0000","last_undegraded":"2024-04-25T10:01:17.146798+0000","last_fullsized":"2024-04-25T10:01:17.146798+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:11.683435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148034+0000","last_change":"2024-04-25T10:01:15.157299+0000","last_active":"2024-04-25T10:01:17.148034+0000","last_peered":"2024-04-25T10:01:17.148034+0000","last_clean":"2024-04-25T10:01:17.148034+0000","last_became_active":"2024-04-25T10:01:15.157020+0000","last_became_peered":"2024-04-25T10:01:15.157020+0000","last_unstale":"2024-04-25T10:01:17.148034+0000","last_undegraded":"2024-04-25T10:01:17.148034+0000","last_fullsized":"2024-04-25T10:01:17.148034+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:40.320762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648224+0000","last_change":"2024-04-25T10:01:16.161304+0000","last_active":"2024-04-25T10:01:17.648224+0000","last_peered":"2024-04-25T10:01:17.648224+0000","last_clean":"2024-04-25T10:01:17.648224+0000","last_became_active":"2024-04-25T10:01:16.161063+0000","last_became_peered":"2024-04-25T10:01:16.161063+0000","last_unstale":"2024-04-25T10:01:17.648224+0000","last_undegraded":"2024-04-25T10:01:17.648224+0000","last_fullsized":"2024-04-25T10:01:17.648224+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:39.490172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649205+0000","last_change":"2024-04-25T10:01:15.162934+0000","last_active":"2024-04-25T10:01:17.649205+0000","last_peered":"2024-04-25T10:01:17.649205+0000","last_clean":"2024-04-25T10:01:17.649205+0000","last_became_active":"2024-04-25T10:01:15.162620+0000","last_became_peered":"2024-04-25T10:01:15.162620+0000","last_unstale":"2024-04-25T10:01:17.649205+0000","last_undegraded":"2024-04-25T10:01:17.649205+0000","last_fullsized":"2024-04-25T10:01:17.649205+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649360+0000","last_change":"2024-04-25T10:01:16.164602+0000","last_active":"2024-04-25T10:01:17.649360+0000","last_peered":"2024-04-25T10:01:17.649360+0000","last_clean":"2024-04-25T10:01:17.649360+0000","last_became_active":"2024-04-25T10:01:16.164330+0000","last_became_peered":"2024-04-25T10:01:16.164330+0000","last_unstale":"2024-04-25T10:01:17.649360+0000","last_undegraded":"2024-04-25T10:01:17.649360+0000","last_fullsized":"2024-04-25T10:01:17.649360+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.767089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648562+0000","last_change":"2024-04-25T10:01:15.161411+0000","last_active":"2024-04-25T10:01:17.648562+0000","last_peered":"2024-04-25T10:01:17.648562+0000","last_clean":"2024-04-25T10:01:17.648562+0000","last_became_active":"2024-04-25T10:01:15.161145+0000","last_became_peered":"2024-04-25T10:01:15.161145+0000","last_unstale":"2024-04-25T10:01:17.648562+0000","last_undegraded":"2024-04-25T10:01:17.648562+0000","last_fullsized":"2024-04-25T10:01:17.648562+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648689+0000","last_change":"2024-04-25T10:01:16.173871+0000","last_active":"2024-04-25T10:01:17.648689+0000","last_peered":"2024-04-25T10:01:17.648689+0000","last_clean":"2024-04-25T10:01:17.648689+0000","last_became_active":"2024-04-25T10:01:16.173358+0000","last_became_peered":"2024-04-25T10:01:16.173358+0000","last_unstale":"2024-04-25T10:01:17.648689+0000","last_undegraded":"2024-04-25T10:01:17.648689+0000","last_fullsized":"2024-04-25T10:01:17.648689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:40.833514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147294+0000","last_change":"2024-04-25T10:01:15.161182+0000","last_active":"2024-04-25T10:01:17.147294+0000","last_peered":"2024-04-25T10:01:17.147294+0000","last_clean":"2024-04-25T10:01:17.147294+0000","last_became_active":"2024-04-25T10:01:15.160798+0000","last_became_peered":"2024-04-25T10:01:15.160798+0000","last_unstale":"2024-04-25T10:01:17.147294+0000","last_undegraded":"2024-04-25T10:01:17.147294+0000","last_fullsized":"2024-04-25T10:01:17.147294+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:35.385701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648297+0000","last_change":"2024-04-25T10:01:16.172361+0000","last_active":"2024-04-25T10:01:17.648297+0000","last_peered":"2024-04-25T10:01:17.648297+0000","last_clean":"2024-04-25T10:01:17.648297+0000","last_became_active":"2024-04-25T10:01:16.172096+0000","last_became_peered":"2024-04-25T10:01:16.172096+0000","last_unstale":"2024-04-25T10:01:17.648297+0000","last_undegraded":"2024-04-25T10:01:17.648297+0000","last_fullsized":"2024-04-25T10:01:17.648297+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:29.184821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648521+0000","last_change":"2024-04-25T10:01:15.154998+0000","last_active":"2024-04-25T10:01:17.648521+0000","last_peered":"2024-04-25T10:01:17.648521+0000","last_clean":"2024-04-25T10:01:17.648521+0000","last_became_active":"2024-04-25T10:01:15.154730+0000","last_became_peered":"2024-04-25T10:01:15.154730+0000","last_unstale":"2024-04-25T10:01:17.648521+0000","last_undegraded":"2024-04-25T10:01:17.648521+0000","last_fullsized":"2024-04-25T10:01:17.648521+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:37.084529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.169012+0000","last_change":"2024-04-25T10:01:16.164851+0000","last_active":"2024-04-25T10:01:17.169012+0000","last_peered":"2024-04-25T10:01:17.169012+0000","last_clean":"2024-04-25T10:01:17.169012+0000","last_became_active":"2024-04-25T10:01:16.164575+0000","last_became_peered":"2024-04-25T10:01:16.164575+0000","last_unstale":"2024-04-25T10:01:17.169012+0000","last_undegraded":"2024-04-25T10:01:17.169012+0000","last_fullsized":"2024-04-25T10:01:17.169012+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:59.616054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648571+0000","last_change":"2024-04-25T10:01:15.164883+0000","last_active":"2024-04-25T10:01:17.648571+0000","last_peered":"2024-04-25T10:01:17.648571+0000","last_clean":"2024-04-25T10:01:17.648571+0000","last_became_active":"2024-04-25T10:01:15.164658+0000","last_became_peered":"2024-04-25T10:01:15.164658+0000","last_unstale":"2024-04-25T10:01:17.648571+0000","last_undegraded":"2024-04-25T10:01:17.648571+0000","last_fullsized":"2024-04-25T10:01:17.648571+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:39.351545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649302+0000","last_change":"2024-04-25T10:01:16.164389+0000","last_active":"2024-04-25T10:01:17.649302+0000","last_peered":"2024-04-25T10:01:17.649302+0000","last_clean":"2024-04-25T10:01:17.649302+0000","last_became_active":"2024-04-25T10:01:16.164070+0000","last_became_peered":"2024-04-25T10:01:16.164070+0000","last_unstale":"2024-04-25T10:01:17.649302+0000","last_undegraded":"2024-04-25T10:01:17.649302+0000","last_fullsized":"2024-04-25T10:01:17.649302+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:45.068283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648264+0000","last_change":"2024-04-25T10:01:15.162486+0000","last_active":"2024-04-25T10:01:17.648264+0000","last_peered":"2024-04-25T10:01:17.648264+0000","last_clean":"2024-04-25T10:01:17.648264+0000","last_became_active":"2024-04-25T10:01:15.161762+0000","last_became_peered":"2024-04-25T10:01:15.161762+0000","last_unstale":"2024-04-25T10:01:17.648264+0000","last_undegraded":"2024-04-25T10:01:17.648264+0000","last_fullsized":"2024-04-25T10:01:17.648264+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:22.481276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648796+0000","last_change":"2024-04-25T10:01:16.169200+0000","last_active":"2024-04-25T10:01:17.648796+0000","last_peered":"2024-04-25T10:01:17.648796+0000","last_clean":"2024-04-25T10:01:17.648796+0000","last_became_active":"2024-04-25T10:01:16.168923+0000","last_became_peered":"2024-04-25T10:01:16.168923+0000","last_unstale":"2024-04-25T10:01:17.648796+0000","last_undegraded":"2024-04-25T10:01:17.648796+0000","last_fullsized":"2024-04-25T10:01:17.648796+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:14.322434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648083+0000","last_change":"2024-04-25T10:01:15.162678+0000","last_active":"2024-04-25T10:01:17.648083+0000","last_peered":"2024-04-25T10:01:17.648083+0000","last_clean":"2024-04-25T10:01:17.648083+0000","last_became_active":"2024-04-25T10:01:15.162090+0000","last_became_peered":"2024-04-25T10:01:15.162090+0000","last_unstale":"2024-04-25T10:01:17.648083+0000","last_undegraded":"2024-04-25T10:01:17.648083+0000","last_fullsized":"2024-04-25T10:01:17.648083+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:21.910671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":328,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168610+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T10:01:17.168610+0000","last_peered":"2024-04-25T10:01:17.168610+0000","last_clean":"2024-04-25T10:01:17.168610+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T10:01:17.168610+0000","last_undegraded":"2024-04-25T10:01:17.168610+0000","last_fullsized":"2024-04-25T10:01:17.168610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574951,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1904640,"data_stored":1195149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1310720,"data_stored":600145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574950,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1302528,"data_stored":604621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1298432,"data_stored":598021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574951,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":598511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":604655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":597963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1912832,"data_stored":1196691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:19.119 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.249+0000 7fb6e1c12640 1 Processor -- start 2024-04-25T10:01:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.249+0000 7fb6e1c12640 1 -- start start 2024-04-25T10:01:19.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6e1c12640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc15ac40 0x7fb6dc06e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6e1c12640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6dc0af6d0 con 0x7fb6dc06e850 2024-04-25T10:01:19.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6e1c12640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6dc0af9b0 con 0x7fb6dc15a860 2024-04-25T10:01:19.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6e1c12640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc0b02e0 con 0x7fb6dc15ac40 2024-04-25T10:01:19.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6dbfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb6dc06e850 0x7fb6dc0b0770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38376/0 (socket says 172.21.15.139:38376) 2024-04-25T10:01:19.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6dbfff640 1 -- 172.21.15.139:0/2416670090 learned_addr learned my addr 172.21.15.139:0/2416670090 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:19.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6daffd640 1 --2- 172.21.15.139:0/2416670090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc15ac40 0x7fb6dc06e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3270745260 0 0) 0x7fb6dc0af6d0 con 0x7fb6dc06e850 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6c4005730 con 0x7fb6dc06e850 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184340665 0 0) 0x7fb6dc0af9b0 con 0x7fb6dc15a860 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6dc0af6d0 con 0x7fb6dc15a860 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1671550916 0 0) 0x7fb6c4005730 con 0x7fb6dc06e850 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc15ac40 msgr2=0x7fb6dc06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6da7fc640 1 --2- 172.21.15.139:0/2416670090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc15ac40 0x7fb6dc06e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 >> v1:172.21.15.150:6789/0 conn(0x7fb6dc15a860 legacy=0x7fb6dc0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc0afc90 con 0x7fb6dc06e850 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fb6d40019f0 con 0x7fb6dc06e850 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.250+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 >> v1:172.21.15.139:6789/0 conn(0x7fb6dc06e850 legacy=0x7fb6dc0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6da7fc640 1 --2- 172.21.15.139:0/2416670090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc15ac40 0x7fb6c4005d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6da7fc640 1 --2- 172.21.15.139:0/2416670090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6c4006330 0x7fb6c4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6da7fc640 1 --2- 172.21.15.139:0/2416670090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c4006f50 0x7fb6c40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc0b02e0 con 0x7fb6c4006330 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c400dd10 con 0x7fb6c4006f50 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c400df10 con 0x7fb6dc15ac40 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6daffd640 1 --2- 172.21.15.139:0/2416670090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6c4006330 0x7fb6c4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6db7fe640 1 --2- 172.21.15.139:0/2416670090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc15ac40 0x7fb6c4005d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6dbfff640 1 --2- 172.21.15.139:0/2416670090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c4006f50 0x7fb6c40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6daffd640 1 -- 172.21.15.139:0/2416670090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc15ac40 msgr2=0x7fb6c4005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6daffd640 1 --2- 172.21.15.139:0/2416670090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc15ac40 0x7fb6c4005d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6daffd640 1 -- 172.21.15.139:0/2416670090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c4006f50 msgr2=0x7fb6c40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6daffd640 1 --2- 172.21.15.139:0/2416670090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c4006f50 0x7fb6c40073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.251+0000 7fb6daffd640 1 -- 172.21.15.139:0/2416670090 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c4005730 con 0x7fb6c4006330 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.252+0000 7fb6daffd640 1 --2- 172.21.15.139:0/2416670090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6c4006330 0x7fb6c4006a10 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009ac0 tx=0x7fb6c4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16098b1b412c189 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.252+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0036070 con 0x7fb6c4006330 2024-04-25T10:01:19.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.252+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d0031400 con 0x7fb6c4006330 2024-04-25T10:01:19.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.252+0000 7fb6da7fc640 1 -- 172.21.15.139:0/2416670090 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0009690 con 0x7fb6c4006330 2024-04-25T10:01:19.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.252+0000 7fb6e1c12640 1 -- 172.21.15.139:0/2416670090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6c4006330 msgr2=0x7fb6c4006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.252+0000 7fb6e1c12640 1 --2- 172.21.15.139:0/2416670090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6c4006330 0x7fb6c4006a10 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009ac0 tx=0x7fb6c4007a40 comp rx=0 tx=0).stop 2024-04-25T10:01:19.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.252+0000 7fb6dbfff640 1 -- 172.21.15.139:0/2416670090 reap_dead start 2024-04-25T10:01:19.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.252+0000 7fb6e1c12640 1 -- 172.21.15.139:0/2416670090 shutdown_connections 2024-04-25T10:01:19.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.252+0000 7fb6e1c12640 1 -- 172.21.15.139:0/2416670090 >> 172.21.15.139:0/2416670090 conn(0x7fb6dc066f20 msgr2=0x7fb6dc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:19.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.252+0000 7fb6e1c12640 1 -- 172.21.15.139:0/2416670090 shutdown_connections 2024-04-25T10:01:19.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.253+0000 7fb6e1c12640 1 -- 172.21.15.139:0/2416670090 wait complete. 2024-04-25T10:01:19.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.254+0000 7fb6e1c12640 1 Processor -- start 2024-04-25T10:01:19.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.254+0000 7fb6e1c12640 1 -- start start 2024-04-25T10:01:19.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.254+0000 7fb6e1c12640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c4006330 0x7fb6dc1f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6e1c12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6dc0a9420 0x7fb6dc071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6e1c12640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc0716b0 0x7fb6dc1ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6e1c12640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc0af2a0 con 0x7fb6dc0a9420 2024-04-25T10:01:19.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6e1c12640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc0cd3c0 con 0x7fb6c4006330 2024-04-25T10:01:19.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6e1c12640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc0af420 con 0x7fb6dc0716b0 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6daffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6dc0a9420 0x7fb6dc071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6daffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6dc0a9420 0x7fb6dc071170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44166/0 (socket says 172.21.15.139:44166) 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6daffd640 1 -- 172.21.15.139:0/987666423 learned_addr learned my addr 172.21.15.139:0/987666423 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6db7fe640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c4006330 0x7fb6dc1f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6dbfff640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc0716b0 0x7fb6dc1ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6daffd640 1 -- 172.21.15.139:0/987666423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc0716b0 msgr2=0x7fb6dc1ef6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6daffd640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc0716b0 0x7fb6dc1ef6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6daffd640 1 -- 172.21.15.139:0/987666423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c4006330 msgr2=0x7fb6dc1f04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6daffd640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c4006330 0x7fb6dc1f04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6daffd640 1 -- 172.21.15.139:0/987666423 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc1501e0 con 0x7fb6dc0a9420 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6dbfff640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc0716b0 0x7fb6dc1ef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6daffd640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6dc0a9420 0x7fb6dc071170 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0000c00 tx=0x7fb6d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6c3fff640 1 -- 172.21.15.139:0/987666423 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0040020 con 0x7fb6dc0a9420 2024-04-25T10:01:19.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6c3fff640 1 -- 172.21.15.139:0/987666423 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d0003c60 con 0x7fb6dc0a9420 2024-04-25T10:01:19.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6e1c12640 1 -- 172.21.15.139:0/987666423 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6dc150480 con 0x7fb6dc0a9420 2024-04-25T10:01:19.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6c3fff640 1 -- 172.21.15.139:0/987666423 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0036040 con 0x7fb6dc0a9420 2024-04-25T10:01:19.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.255+0000 7fb6e1c12640 1 -- 172.21.15.139:0/987666423 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6dc150a40 con 0x7fb6dc0a9420 2024-04-25T10:01:19.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.256+0000 7fb6e1c12640 1 -- 172.21.15.139:0/987666423 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6dc071ac0 con 0x7fb6dc0a9420 2024-04-25T10:01:19.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.257+0000 7fb6c3fff640 1 -- 172.21.15.139:0/987666423 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6d0043020 con 0x7fb6dc0a9420 2024-04-25T10:01:19.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.257+0000 7fb6c3fff640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb6c806b500 0x7fb6c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.257+0000 7fb6c3fff640 1 -- 172.21.15.139:0/987666423 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb6d00aa3c0 con 0x7fb6dc0a9420 2024-04-25T10:01:19.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.257+0000 7fb6db7fe640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb6c806b500 0x7fb6c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.258+0000 7fb6db7fe640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb6c806b500 0x7fb6c806d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc155620 tx=0x7fb6cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:19.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.261+0000 7fb6c3fff640 1 -- 172.21.15.139:0/987666423 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6dc071ac0 con 0x7fb6dc0a9420 2024-04-25T10:01:19.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.429+0000 7fb6e1c12640 1 -- 172.21.15.139:0/987666423 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6dc074520 con 0x7fb6c806b500 2024-04-25T10:01:19.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.434+0000 7fb6c3fff640 1 -- 172.21.15.139:0/987666423 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252481 (secure 0 0 0) 0x7fb6dc074520 con 0x7fb6c806b500 2024-04-25T10:01:19.441 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:19.442 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T10:01:19.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.437+0000 7fb6e1c12640 1 -- 172.21.15.139:0/987666423 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb6c806b500 msgr2=0x7fb6c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.437+0000 7fb6e1c12640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb6c806b500 0x7fb6c806d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc155620 tx=0x7fb6cc009040 comp rx=0 tx=0).stop 2024-04-25T10:01:19.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.437+0000 7fb6e1c12640 1 -- 172.21.15.139:0/987666423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6dc0a9420 msgr2=0x7fb6dc071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.437+0000 7fb6e1c12640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6dc0a9420 0x7fb6dc071170 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0000c00 tx=0x7fb6d0004730 comp rx=0 tx=0).stop 2024-04-25T10:01:19.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.437+0000 7fb6e1c12640 1 -- 172.21.15.139:0/987666423 shutdown_connections 2024-04-25T10:01:19.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.437+0000 7fb6e1c12640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6dc0a9420 0x7fb6dc071170 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.437+0000 7fb6e1c12640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb6c806b500 0x7fb6c806d9c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.437+0000 7fb6e1c12640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6dc0716b0 0x7fb6dc1ef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.437+0000 7fb6e1c12640 1 --2- 172.21.15.139:0/987666423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c4006330 0x7fb6dc1f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.437+0000 7fb6e1c12640 1 -- 172.21.15.139:0/987666423 >> 172.21.15.139:0/987666423 conn(0x7fb6dc066f20 msgr2=0x7fb6dc0cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:19.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.437+0000 7fb6e1c12640 1 -- 172.21.15.139:0/987666423 shutdown_connections 2024-04-25T10:01:19.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.437+0000 7fb6e1c12640 1 -- 172.21.15.139:0/987666423 wait complete. 2024-04-25T10:01:19.453 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-04-25T10:01:19.078141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260832,"kb_used_data":11352,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713888,"statfs":{"total":773094113280,"available":772827021312,"internally_reserved":0,"allocated":11624448,"data_stored":5995756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.942624"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148151+0000","last_change":"2024-04-25T10:01:16.163785+0000","last_active":"2024-04-25T10:01:17.148151+0000","last_peered":"2024-04-25T10:01:17.148151+0000","last_clean":"2024-04-25T10:01:17.148151+0000","last_became_active":"2024-04-25T10:01:16.162777+0000","last_became_peered":"2024-04-25T10:01:16.162777+0000","last_unstale":"2024-04-25T10:01:17.148151+0000","last_undegraded":"2024-04-25T10:01:17.148151+0000","last_fullsized":"2024-04-25T10:01:17.148151+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:07.399739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649541+0000","last_change":"2024-04-25T10:01:16.158502+0000","last_active":"2024-04-25T10:01:17.649541+0000","last_peered":"2024-04-25T10:01:17.649541+0000","last_clean":"2024-04-25T10:01:17.649541+0000","last_became_active":"2024-04-25T10:01:16.158218+0000","last_became_peered":"2024-04-25T10:01:16.158218+0000","last_unstale":"2024-04-25T10:01:17.649541+0000","last_undegraded":"2024-04-25T10:01:17.649541+0000","last_fullsized":"2024-04-25T10:01:17.649541+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.767089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148223+0000","last_change":"2024-04-25T10:01:16.164784+0000","last_active":"2024-04-25T10:01:17.148223+0000","last_peered":"2024-04-25T10:01:17.148223+0000","last_clean":"2024-04-25T10:01:17.148223+0000","last_became_active":"2024-04-25T10:01:16.164517+0000","last_became_peered":"2024-04-25T10:01:16.164517+0000","last_unstale":"2024-04-25T10:01:17.148223+0000","last_undegraded":"2024-04-25T10:01:17.148223+0000","last_fullsized":"2024-04-25T10:01:17.148223+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:22.840434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648996+0000","last_change":"2024-04-25T10:01:16.158585+0000","last_active":"2024-04-25T10:01:17.648996+0000","last_peered":"2024-04-25T10:01:17.648996+0000","last_clean":"2024-04-25T10:01:17.648996+0000","last_became_active":"2024-04-25T10:01:16.158240+0000","last_became_peered":"2024-04-25T10:01:16.158240+0000","last_unstale":"2024-04-25T10:01:17.648996+0000","last_undegraded":"2024-04-25T10:01:17.648996+0000","last_fullsized":"2024-04-25T10:01:17.648996+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:29.994543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648782+0000","last_change":"2024-04-25T10:01:16.168633+0000","last_active":"2024-04-25T10:01:17.648782+0000","last_peered":"2024-04-25T10:01:17.648782+0000","last_clean":"2024-04-25T10:01:17.648782+0000","last_became_active":"2024-04-25T10:01:16.168356+0000","last_became_peered":"2024-04-25T10:01:16.168356+0000","last_unstale":"2024-04-25T10:01:17.648782+0000","last_undegraded":"2024-04-25T10:01:17.648782+0000","last_fullsized":"2024-04-25T10:01:17.648782+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:13.378908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648532+0000","last_change":"2024-04-25T10:01:16.170253+0000","last_active":"2024-04-25T10:01:17.648532+0000","last_peered":"2024-04-25T10:01:17.648532+0000","last_clean":"2024-04-25T10:01:17.648532+0000","last_became_active":"2024-04-25T10:01:16.170000+0000","last_became_peered":"2024-04-25T10:01:16.170000+0000","last_unstale":"2024-04-25T10:01:17.648532+0000","last_undegraded":"2024-04-25T10:01:17.648532+0000","last_fullsized":"2024-04-25T10:01:17.648532+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:14.322434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648348+0000","last_change":"2024-04-25T10:01:16.164765+0000","last_active":"2024-04-25T10:01:17.648348+0000","last_peered":"2024-04-25T10:01:17.648348+0000","last_clean":"2024-04-25T10:01:17.648348+0000","last_became_active":"2024-04-25T10:01:16.164467+0000","last_became_peered":"2024-04-25T10:01:16.164467+0000","last_unstale":"2024-04-25T10:01:17.648348+0000","last_undegraded":"2024-04-25T10:01:17.648348+0000","last_fullsized":"2024-04-25T10:01:17.648348+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:13.378908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.647902+0000","last_change":"2024-04-25T10:01:16.173608+0000","last_active":"2024-04-25T10:01:17.647902+0000","last_peered":"2024-04-25T10:01:17.647902+0000","last_clean":"2024-04-25T10:01:17.647902+0000","last_became_active":"2024-04-25T10:01:16.173217+0000","last_became_peered":"2024-04-25T10:01:16.173217+0000","last_unstale":"2024-04-25T10:01:17.647902+0000","last_undegraded":"2024-04-25T10:01:17.647902+0000","last_fullsized":"2024-04-25T10:01:17.647902+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:55.566324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649152+0000","last_change":"2024-04-25T10:01:16.164769+0000","last_active":"2024-04-25T10:01:17.649152+0000","last_peered":"2024-04-25T10:01:17.649152+0000","last_clean":"2024-04-25T10:01:17.649152+0000","last_became_active":"2024-04-25T10:01:16.164505+0000","last_became_peered":"2024-04-25T10:01:16.164505+0000","last_unstale":"2024-04-25T10:01:17.649152+0000","last_undegraded":"2024-04-25T10:01:17.649152+0000","last_fullsized":"2024-04-25T10:01:17.649152+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:21.703614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648559+0000","last_change":"2024-04-25T10:01:16.165186+0000","last_active":"2024-04-25T10:01:17.648559+0000","last_peered":"2024-04-25T10:01:17.648559+0000","last_clean":"2024-04-25T10:01:17.648559+0000","last_became_active":"2024-04-25T10:01:16.164911+0000","last_became_peered":"2024-04-25T10:01:16.164911+0000","last_unstale":"2024-04-25T10:01:17.648559+0000","last_undegraded":"2024-04-25T10:01:17.648559+0000","last_fullsized":"2024-04-25T10:01:17.648559+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:02.786410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649068+0000","last_change":"2024-04-25T10:01:16.163787+0000","last_active":"2024-04-25T10:01:17.649068+0000","last_peered":"2024-04-25T10:01:17.649068+0000","last_clean":"2024-04-25T10:01:17.649068+0000","last_became_active":"2024-04-25T10:01:16.162437+0000","last_became_peered":"2024-04-25T10:01:16.162437+0000","last_unstale":"2024-04-25T10:01:17.649068+0000","last_undegraded":"2024-04-25T10:01:17.649068+0000","last_fullsized":"2024-04-25T10:01:17.649068+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:19.369840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147472+0000","last_change":"2024-04-25T10:01:16.160696+0000","last_active":"2024-04-25T10:01:17.147472+0000","last_peered":"2024-04-25T10:01:17.147472+0000","last_clean":"2024-04-25T10:01:17.147472+0000","last_became_active":"2024-04-25T10:01:16.160445+0000","last_became_peered":"2024-04-25T10:01:16.160445+0000","last_unstale":"2024-04-25T10:01:17.147472+0000","last_undegraded":"2024-04-25T10:01:17.147472+0000","last_fullsized":"2024-04-25T10:01:17.147472+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:51.790891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648843+0000","last_change":"2024-04-25T10:01:16.170430+0000","last_active":"2024-04-25T10:01:17.648843+0000","last_peered":"2024-04-25T10:01:17.648843+0000","last_clean":"2024-04-25T10:01:17.648843+0000","last_became_active":"2024-04-25T10:01:16.170187+0000","last_became_peered":"2024-04-25T10:01:16.170187+0000","last_unstale":"2024-04-25T10:01:17.648843+0000","last_undegraded":"2024-04-25T10:01:17.648843+0000","last_fullsized":"2024-04-25T10:01:17.648843+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:34.259379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168898+0000","last_change":"2024-04-25T10:01:16.159772+0000","last_active":"2024-04-25T10:01:17.168898+0000","last_peered":"2024-04-25T10:01:17.168898+0000","last_clean":"2024-04-25T10:01:17.168898+0000","last_became_active":"2024-04-25T10:01:16.159492+0000","last_became_peered":"2024-04-25T10:01:16.159492+0000","last_unstale":"2024-04-25T10:01:17.168898+0000","last_undegraded":"2024-04-25T10:01:17.168898+0000","last_fullsized":"2024-04-25T10:01:17.168898+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:24.536283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168918+0000","last_change":"2024-04-25T10:01:16.164618+0000","last_active":"2024-04-25T10:01:17.168918+0000","last_peered":"2024-04-25T10:01:17.168918+0000","last_clean":"2024-04-25T10:01:17.168918+0000","last_became_active":"2024-04-25T10:01:16.164313+0000","last_became_peered":"2024-04-25T10:01:16.164313+0000","last_unstale":"2024-04-25T10:01:17.168918+0000","last_undegraded":"2024-04-25T10:01:17.168918+0000","last_fullsized":"2024-04-25T10:01:17.168918+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:25.170660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168815+0000","last_change":"2024-04-25T10:01:16.158552+0000","last_active":"2024-04-25T10:01:17.168815+0000","last_peered":"2024-04-25T10:01:17.168815+0000","last_clean":"2024-04-25T10:01:17.168815+0000","last_became_active":"2024-04-25T10:01:16.158307+0000","last_became_peered":"2024-04-25T10:01:16.158307+0000","last_unstale":"2024-04-25T10:01:17.168815+0000","last_undegraded":"2024-04-25T10:01:17.168815+0000","last_fullsized":"2024-04-25T10:01:17.168815+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:16.513283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148274+0000","last_change":"2024-04-25T10:01:16.165783+0000","last_active":"2024-04-25T10:01:17.148274+0000","last_peered":"2024-04-25T10:01:17.148274+0000","last_clean":"2024-04-25T10:01:17.148274+0000","last_became_active":"2024-04-25T10:01:16.165221+0000","last_became_peered":"2024-04-25T10:01:16.165221+0000","last_unstale":"2024-04-25T10:01:17.148274+0000","last_undegraded":"2024-04-25T10:01:17.148274+0000","last_fullsized":"2024-04-25T10:01:17.148274+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:33.757510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649541+0000","last_change":"2024-04-25T10:01:16.162082+0000","last_active":"2024-04-25T10:01:17.649541+0000","last_peered":"2024-04-25T10:01:17.649541+0000","last_clean":"2024-04-25T10:01:17.649541+0000","last_became_active":"2024-04-25T10:01:16.161809+0000","last_became_peered":"2024-04-25T10:01:16.161809+0000","last_unstale":"2024-04-25T10:01:17.649541+0000","last_undegraded":"2024-04-25T10:01:17.649541+0000","last_fullsized":"2024-04-25T10:01:17.649541+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:22.855842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148494+0000","last_change":"2024-04-25T10:01:16.162360+0000","last_active":"2024-04-25T10:01:17.148494+0000","last_peered":"2024-04-25T10:01:17.148494+0000","last_clean":"2024-04-25T10:01:17.148494+0000","last_became_active":"2024-04-25T10:01:16.162079+0000","last_became_peered":"2024-04-25T10:01:16.162079+0000","last_unstale":"2024-04-25T10:01:17.148494+0000","last_undegraded":"2024-04-25T10:01:17.148494+0000","last_fullsized":"2024-04-25T10:01:17.148494+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:07.663121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147555+0000","last_change":"2024-04-25T10:01:16.152957+0000","last_active":"2024-04-25T10:01:17.147555+0000","last_peered":"2024-04-25T10:01:17.147555+0000","last_clean":"2024-04-25T10:01:17.147555+0000","last_became_active":"2024-04-25T10:01:16.152669+0000","last_became_peered":"2024-04-25T10:01:16.152669+0000","last_unstale":"2024-04-25T10:01:17.147555+0000","last_undegraded":"2024-04-25T10:01:17.147555+0000","last_fullsized":"2024-04-25T10:01:17.147555+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:45.068283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648828+0000","last_change":"2024-04-25T10:01:16.167706+0000","last_active":"2024-04-25T10:01:17.648828+0000","last_peered":"2024-04-25T10:01:17.648828+0000","last_clean":"2024-04-25T10:01:17.648828+0000","last_became_active":"2024-04-25T10:01:16.167465+0000","last_became_peered":"2024-04-25T10:01:16.167465+0000","last_unstale":"2024-04-25T10:01:17.648828+0000","last_undegraded":"2024-04-25T10:01:17.648828+0000","last_fullsized":"2024-04-25T10:01:17.648828+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:50.198545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648655+0000","last_change":"2024-04-25T10:01:16.164411+0000","last_active":"2024-04-25T10:01:17.648655+0000","last_peered":"2024-04-25T10:01:17.648655+0000","last_clean":"2024-04-25T10:01:17.648655+0000","last_became_active":"2024-04-25T10:01:16.164162+0000","last_became_peered":"2024-04-25T10:01:16.164162+0000","last_unstale":"2024-04-25T10:01:17.648655+0000","last_undegraded":"2024-04-25T10:01:17.648655+0000","last_fullsized":"2024-04-25T10:01:17.648655+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:57.633684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147966+0000","last_change":"2024-04-25T10:01:16.152924+0000","last_active":"2024-04-25T10:01:17.147966+0000","last_peered":"2024-04-25T10:01:17.147966+0000","last_clean":"2024-04-25T10:01:17.147966+0000","last_became_active":"2024-04-25T10:01:16.152646+0000","last_became_peered":"2024-04-25T10:01:16.152646+0000","last_unstale":"2024-04-25T10:01:17.147966+0000","last_undegraded":"2024-04-25T10:01:17.147966+0000","last_fullsized":"2024-04-25T10:01:17.147966+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:42.718579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648220+0000","last_change":"2024-04-25T10:01:16.168058+0000","last_active":"2024-04-25T10:01:17.648220+0000","last_peered":"2024-04-25T10:01:17.648220+0000","last_clean":"2024-04-25T10:01:17.648220+0000","last_became_active":"2024-04-25T10:01:16.167791+0000","last_became_peered":"2024-04-25T10:01:16.167791+0000","last_unstale":"2024-04-25T10:01:17.648220+0000","last_undegraded":"2024-04-25T10:01:17.648220+0000","last_fullsized":"2024-04-25T10:01:17.648220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:24.529960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147585+0000","last_change":"2024-04-25T10:01:16.167296+0000","last_active":"2024-04-25T10:01:17.147585+0000","last_peered":"2024-04-25T10:01:17.147585+0000","last_clean":"2024-04-25T10:01:17.147585+0000","last_became_active":"2024-04-25T10:01:16.167008+0000","last_became_peered":"2024-04-25T10:01:16.167008+0000","last_unstale":"2024-04-25T10:01:17.147585+0000","last_undegraded":"2024-04-25T10:01:17.147585+0000","last_fullsized":"2024-04-25T10:01:17.147585+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:22.855842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147580+0000","last_change":"2024-04-25T10:01:16.161419+0000","last_active":"2024-04-25T10:01:17.147580+0000","last_peered":"2024-04-25T10:01:17.147580+0000","last_clean":"2024-04-25T10:01:17.147580+0000","last_became_active":"2024-04-25T10:01:16.161157+0000","last_became_peered":"2024-04-25T10:01:16.161157+0000","last_unstale":"2024-04-25T10:01:17.147580+0000","last_undegraded":"2024-04-25T10:01:17.147580+0000","last_fullsized":"2024-04-25T10:01:17.147580+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.767089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649086+0000","last_change":"2024-04-25T10:01:16.156875+0000","last_active":"2024-04-25T10:01:17.649086+0000","last_peered":"2024-04-25T10:01:17.649086+0000","last_clean":"2024-04-25T10:01:17.649086+0000","last_became_active":"2024-04-25T10:01:16.156334+0000","last_became_peered":"2024-04-25T10:01:16.156334+0000","last_unstale":"2024-04-25T10:01:17.649086+0000","last_undegraded":"2024-04-25T10:01:17.649086+0000","last_fullsized":"2024-04-25T10:01:17.649086+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:21.703614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648637+0000","last_change":"2024-04-25T10:01:16.166538+0000","last_active":"2024-04-25T10:01:17.648637+0000","last_peered":"2024-04-25T10:01:17.648637+0000","last_clean":"2024-04-25T10:01:17.648637+0000","last_became_active":"2024-04-25T10:01:16.166252+0000","last_became_peered":"2024-04-25T10:01:16.166252+0000","last_unstale":"2024-04-25T10:01:17.648637+0000","last_undegraded":"2024-04-25T10:01:17.648637+0000","last_fullsized":"2024-04-25T10:01:17.648637+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:11.683435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648958+0000","last_change":"2024-04-25T10:01:16.165276+0000","last_active":"2024-04-25T10:01:17.648958+0000","last_peered":"2024-04-25T10:01:17.648958+0000","last_clean":"2024-04-25T10:01:17.648958+0000","last_became_active":"2024-04-25T10:01:16.164998+0000","last_became_peered":"2024-04-25T10:01:16.164998+0000","last_unstale":"2024-04-25T10:01:17.648958+0000","last_undegraded":"2024-04-25T10:01:17.648958+0000","last_fullsized":"2024-04-25T10:01:17.648958+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:46.200041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147000+0000","last_change":"2024-04-25T10:01:16.166890+0000","last_active":"2024-04-25T10:01:17.147000+0000","last_peered":"2024-04-25T10:01:17.147000+0000","last_clean":"2024-04-25T10:01:17.147000+0000","last_became_active":"2024-04-25T10:01:16.166576+0000","last_became_peered":"2024-04-25T10:01:16.166576+0000","last_unstale":"2024-04-25T10:01:17.147000+0000","last_undegraded":"2024-04-25T10:01:17.147000+0000","last_fullsized":"2024-04-25T10:01:17.147000+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:19.369840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649405+0000","last_change":"2024-04-25T10:01:16.160703+0000","last_active":"2024-04-25T10:01:17.649405+0000","last_peered":"2024-04-25T10:01:17.649405+0000","last_clean":"2024-04-25T10:01:17.649405+0000","last_became_active":"2024-04-25T10:01:16.160389+0000","last_became_peered":"2024-04-25T10:01:16.160389+0000","last_unstale":"2024-04-25T10:01:17.649405+0000","last_undegraded":"2024-04-25T10:01:17.649405+0000","last_fullsized":"2024-04-25T10:01:17.649405+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:45.068283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648172+0000","last_change":"2024-04-25T10:01:16.165352+0000","last_active":"2024-04-25T10:01:17.648172+0000","last_peered":"2024-04-25T10:01:17.648172+0000","last_clean":"2024-04-25T10:01:17.648172+0000","last_became_active":"2024-04-25T10:01:16.164980+0000","last_became_peered":"2024-04-25T10:01:16.164980+0000","last_unstale":"2024-04-25T10:01:17.648172+0000","last_undegraded":"2024-04-25T10:01:17.648172+0000","last_fullsized":"2024-04-25T10:01:17.648172+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:03.984018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148074+0000","last_change":"2024-04-25T10:01:16.157210+0000","last_active":"2024-04-25T10:01:17.148074+0000","last_peered":"2024-04-25T10:01:17.148074+0000","last_clean":"2024-04-25T10:01:17.148074+0000","last_became_active":"2024-04-25T10:01:16.156635+0000","last_became_peered":"2024-04-25T10:01:16.156635+0000","last_unstale":"2024-04-25T10:01:17.148074+0000","last_undegraded":"2024-04-25T10:01:17.148074+0000","last_fullsized":"2024-04-25T10:01:17.148074+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:57.589958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"104'4","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649106+0000","last_change":"2024-04-25T10:01:15.161914+0000","last_active":"2024-04-25T10:01:17.649106+0000","last_peered":"2024-04-25T10:01:17.649106+0000","last_clean":"2024-04-25T10:01:17.649106+0000","last_became_active":"2024-04-25T10:01:15.161644+0000","last_became_peered":"2024-04-25T10:01:15.161644+0000","last_unstale":"2024-04-25T10:01:17.649106+0000","last_undegraded":"2024-04-25T10:01:17.649106+0000","last_fullsized":"2024-04-25T10:01:17.649106+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:18.364153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648378+0000","last_change":"2024-04-25T10:01:16.174563+0000","last_active":"2024-04-25T10:01:17.648378+0000","last_peered":"2024-04-25T10:01:17.648378+0000","last_clean":"2024-04-25T10:01:17.648378+0000","last_became_active":"2024-04-25T10:01:16.174290+0000","last_became_peered":"2024-04-25T10:01:16.174290+0000","last_unstale":"2024-04-25T10:01:17.648378+0000","last_undegraded":"2024-04-25T10:01:17.648378+0000","last_fullsized":"2024-04-25T10:01:17.648378+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:21.126022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648333+0000","last_change":"2024-04-25T10:01:15.158147+0000","last_active":"2024-04-25T10:01:17.648333+0000","last_peered":"2024-04-25T10:01:17.648333+0000","last_clean":"2024-04-25T10:01:17.648333+0000","last_became_active":"2024-04-25T10:01:15.157874+0000","last_became_peered":"2024-04-25T10:01:15.157874+0000","last_unstale":"2024-04-25T10:01:17.648333+0000","last_undegraded":"2024-04-25T10:01:17.648333+0000","last_fullsized":"2024-04-25T10:01:17.648333+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:52.995850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147183+0000","last_change":"2024-04-25T10:01:16.159799+0000","last_active":"2024-04-25T10:01:17.147183+0000","last_peered":"2024-04-25T10:01:17.147183+0000","last_clean":"2024-04-25T10:01:17.147183+0000","last_became_active":"2024-04-25T10:01:16.159555+0000","last_became_peered":"2024-04-25T10:01:16.159555+0000","last_unstale":"2024-04-25T10:01:17.147183+0000","last_undegraded":"2024-04-25T10:01:17.147183+0000","last_fullsized":"2024-04-25T10:01:17.147183+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:21.703614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147548+0000","last_change":"2024-04-25T10:01:15.156417+0000","last_active":"2024-04-25T10:01:17.147548+0000","last_peered":"2024-04-25T10:01:17.147548+0000","last_clean":"2024-04-25T10:01:17.147548+0000","last_became_active":"2024-04-25T10:01:15.156144+0000","last_became_peered":"2024-04-25T10:01:15.156144+0000","last_unstale":"2024-04-25T10:01:17.147548+0000","last_undegraded":"2024-04-25T10:01:17.147548+0000","last_fullsized":"2024-04-25T10:01:17.147548+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:33.937222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649161+0000","last_change":"2024-04-25T10:01:16.155803+0000","last_active":"2024-04-25T10:01:17.649161+0000","last_peered":"2024-04-25T10:01:17.649161+0000","last_clean":"2024-04-25T10:01:17.649161+0000","last_became_active":"2024-04-25T10:01:16.155462+0000","last_became_peered":"2024-04-25T10:01:16.155462+0000","last_unstale":"2024-04-25T10:01:17.649161+0000","last_undegraded":"2024-04-25T10:01:17.649161+0000","last_fullsized":"2024-04-25T10:01:17.649161+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:51.790891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648491+0000","last_change":"2024-04-25T10:01:15.154241+0000","last_active":"2024-04-25T10:01:17.648491+0000","last_peered":"2024-04-25T10:01:17.648491+0000","last_clean":"2024-04-25T10:01:17.648491+0000","last_became_active":"2024-04-25T10:01:15.153838+0000","last_became_peered":"2024-04-25T10:01:15.153838+0000","last_unstale":"2024-04-25T10:01:17.648491+0000","last_undegraded":"2024-04-25T10:01:17.648491+0000","last_fullsized":"2024-04-25T10:01:17.648491+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:26.356231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649166+0000","last_change":"2024-04-25T10:01:16.169953+0000","last_active":"2024-04-25T10:01:17.649166+0000","last_peered":"2024-04-25T10:01:17.649166+0000","last_clean":"2024-04-25T10:01:17.649166+0000","last_became_active":"2024-04-25T10:01:16.169692+0000","last_became_peered":"2024-04-25T10:01:16.169692+0000","last_unstale":"2024-04-25T10:01:17.649166+0000","last_undegraded":"2024-04-25T10:01:17.649166+0000","last_fullsized":"2024-04-25T10:01:17.649166+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:28.394593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168909+0000","last_change":"2024-04-25T10:01:15.161688+0000","last_active":"2024-04-25T10:01:17.168909+0000","last_peered":"2024-04-25T10:01:17.168909+0000","last_clean":"2024-04-25T10:01:17.168909+0000","last_became_active":"2024-04-25T10:01:15.161376+0000","last_became_peered":"2024-04-25T10:01:15.161376+0000","last_unstale":"2024-04-25T10:01:17.168909+0000","last_undegraded":"2024-04-25T10:01:17.168909+0000","last_fullsized":"2024-04-25T10:01:17.168909+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:27.388906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148470+0000","last_change":"2024-04-25T10:01:16.157485+0000","last_active":"2024-04-25T10:01:17.148470+0000","last_peered":"2024-04-25T10:01:17.148470+0000","last_clean":"2024-04-25T10:01:17.148470+0000","last_became_active":"2024-04-25T10:01:16.156967+0000","last_became_peered":"2024-04-25T10:01:16.156967+0000","last_unstale":"2024-04-25T10:01:17.148470+0000","last_undegraded":"2024-04-25T10:01:17.148470+0000","last_fullsized":"2024-04-25T10:01:17.148470+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:43.158723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649025+0000","last_change":"2024-04-25T10:01:15.159782+0000","last_active":"2024-04-25T10:01:17.649025+0000","last_peered":"2024-04-25T10:01:17.649025+0000","last_clean":"2024-04-25T10:01:17.649025+0000","last_became_active":"2024-04-25T10:01:15.159489+0000","last_became_peered":"2024-04-25T10:01:15.159489+0000","last_unstale":"2024-04-25T10:01:17.649025+0000","last_undegraded":"2024-04-25T10:01:17.649025+0000","last_fullsized":"2024-04-25T10:01:17.649025+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:57.232174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649655+0000","last_change":"2024-04-25T10:01:16.163364+0000","last_active":"2024-04-25T10:01:17.649655+0000","last_peered":"2024-04-25T10:01:17.649655+0000","last_clean":"2024-04-25T10:01:17.649655+0000","last_became_active":"2024-04-25T10:01:16.163019+0000","last_became_peered":"2024-04-25T10:01:16.163019+0000","last_unstale":"2024-04-25T10:01:17.649655+0000","last_undegraded":"2024-04-25T10:01:17.649655+0000","last_fullsized":"2024-04-25T10:01:17.649655+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:35.487895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.146769+0000","last_change":"2024-04-25T10:01:15.156689+0000","last_active":"2024-04-25T10:01:17.146769+0000","last_peered":"2024-04-25T10:01:17.146769+0000","last_clean":"2024-04-25T10:01:17.146769+0000","last_became_active":"2024-04-25T10:01:15.155419+0000","last_became_peered":"2024-04-25T10:01:15.155419+0000","last_unstale":"2024-04-25T10:01:17.146769+0000","last_undegraded":"2024-04-25T10:01:17.146769+0000","last_fullsized":"2024-04-25T10:01:17.146769+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:57.232174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649139+0000","last_change":"2024-04-25T10:01:16.167548+0000","last_active":"2024-04-25T10:01:17.649139+0000","last_peered":"2024-04-25T10:01:17.649139+0000","last_clean":"2024-04-25T10:01:17.649139+0000","last_became_active":"2024-04-25T10:01:16.167295+0000","last_became_peered":"2024-04-25T10:01:16.167295+0000","last_unstale":"2024-04-25T10:01:17.649139+0000","last_undegraded":"2024-04-25T10:01:17.649139+0000","last_fullsized":"2024-04-25T10:01:17.649139+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:22.532819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.647898+0000","last_change":"2024-04-25T10:01:15.156256+0000","last_active":"2024-04-25T10:01:17.647898+0000","last_peered":"2024-04-25T10:01:17.647898+0000","last_clean":"2024-04-25T10:01:17.647898+0000","last_became_active":"2024-04-25T10:01:15.155956+0000","last_became_peered":"2024-04-25T10:01:15.155956+0000","last_unstale":"2024-04-25T10:01:17.647898+0000","last_undegraded":"2024-04-25T10:01:17.647898+0000","last_fullsized":"2024-04-25T10:01:17.647898+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:10.677748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147930+0000","last_change":"2024-04-25T10:01:16.157814+0000","last_active":"2024-04-25T10:01:17.147930+0000","last_peered":"2024-04-25T10:01:17.147930+0000","last_clean":"2024-04-25T10:01:17.147930+0000","last_became_active":"2024-04-25T10:01:16.157032+0000","last_became_peered":"2024-04-25T10:01:16.157032+0000","last_unstale":"2024-04-25T10:01:17.147930+0000","last_undegraded":"2024-04-25T10:01:17.147930+0000","last_fullsized":"2024-04-25T10:01:17.147930+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:29.459037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147589+0000","last_change":"2024-04-25T10:01:15.156903+0000","last_active":"2024-04-25T10:01:17.147589+0000","last_peered":"2024-04-25T10:01:17.147589+0000","last_clean":"2024-04-25T10:01:17.147589+0000","last_became_active":"2024-04-25T10:01:15.156618+0000","last_became_peered":"2024-04-25T10:01:15.156618+0000","last_unstale":"2024-04-25T10:01:17.147589+0000","last_undegraded":"2024-04-25T10:01:17.147589+0000","last_fullsized":"2024-04-25T10:01:17.147589+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:57.768353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648111+0000","last_change":"2024-04-25T10:01:16.173197+0000","last_active":"2024-04-25T10:01:17.648111+0000","last_peered":"2024-04-25T10:01:17.648111+0000","last_clean":"2024-04-25T10:01:17.648111+0000","last_became_active":"2024-04-25T10:01:16.172938+0000","last_became_peered":"2024-04-25T10:01:16.172938+0000","last_unstale":"2024-04-25T10:01:17.648111+0000","last_undegraded":"2024-04-25T10:01:17.648111+0000","last_fullsized":"2024-04-25T10:01:17.648111+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:14.346871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148015+0000","last_change":"2024-04-25T10:01:15.151057+0000","last_active":"2024-04-25T10:01:17.148015+0000","last_peered":"2024-04-25T10:01:17.148015+0000","last_clean":"2024-04-25T10:01:17.148015+0000","last_became_active":"2024-04-25T10:01:15.150785+0000","last_became_peered":"2024-04-25T10:01:15.150785+0000","last_unstale":"2024-04-25T10:01:17.148015+0000","last_undegraded":"2024-04-25T10:01:17.148015+0000","last_fullsized":"2024-04-25T10:01:17.148015+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:19.036399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648434+0000","last_change":"2024-04-25T10:01:16.177934+0000","last_active":"2024-04-25T10:01:17.648434+0000","last_peered":"2024-04-25T10:01:17.648434+0000","last_clean":"2024-04-25T10:01:17.648434+0000","last_became_active":"2024-04-25T10:01:16.177646+0000","last_became_peered":"2024-04-25T10:01:16.177646+0000","last_unstale":"2024-04-25T10:01:17.648434+0000","last_undegraded":"2024-04-25T10:01:17.648434+0000","last_fullsized":"2024-04-25T10:01:17.648434+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:22.516250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"104'3","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168983+0000","last_change":"2024-04-25T10:01:15.162131+0000","last_active":"2024-04-25T10:01:17.168983+0000","last_peered":"2024-04-25T10:01:17.168983+0000","last_clean":"2024-04-25T10:01:17.168983+0000","last_became_active":"2024-04-25T10:01:15.161850+0000","last_became_peered":"2024-04-25T10:01:15.161850+0000","last_unstale":"2024-04-25T10:01:17.168983+0000","last_undegraded":"2024-04-25T10:01:17.168983+0000","last_fullsized":"2024-04-25T10:01:17.168983+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:28.988856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648926+0000","last_change":"2024-04-25T10:01:16.173895+0000","last_active":"2024-04-25T10:01:17.648926+0000","last_peered":"2024-04-25T10:01:17.648926+0000","last_clean":"2024-04-25T10:01:17.648926+0000","last_became_active":"2024-04-25T10:01:16.173617+0000","last_became_peered":"2024-04-25T10:01:16.173617+0000","last_unstale":"2024-04-25T10:01:17.648926+0000","last_undegraded":"2024-04-25T10:01:17.648926+0000","last_fullsized":"2024-04-25T10:01:17.648926+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:11.683435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"104'6","reported_seq":20,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649336+0000","last_change":"2024-04-25T10:01:15.162469+0000","last_active":"2024-04-25T10:01:17.649336+0000","last_peered":"2024-04-25T10:01:17.649336+0000","last_clean":"2024-04-25T10:01:17.649336+0000","last_became_active":"2024-04-25T10:01:15.162150+0000","last_became_peered":"2024-04-25T10:01:15.162150+0000","last_unstale":"2024-04-25T10:01:17.649336+0000","last_undegraded":"2024-04-25T10:01:17.649336+0000","last_fullsized":"2024-04-25T10:01:17.649336+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:44.732558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649086+0000","last_change":"2024-04-25T10:01:16.170184+0000","last_active":"2024-04-25T10:01:17.649086+0000","last_peered":"2024-04-25T10:01:17.649086+0000","last_clean":"2024-04-25T10:01:17.649086+0000","last_became_active":"2024-04-25T10:01:16.169881+0000","last_became_peered":"2024-04-25T10:01:16.169881+0000","last_unstale":"2024-04-25T10:01:17.649086+0000","last_undegraded":"2024-04-25T10:01:17.649086+0000","last_fullsized":"2024-04-25T10:01:17.649086+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:11.683435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648979+0000","last_change":"2024-04-25T10:01:15.163189+0000","last_active":"2024-04-25T10:01:17.648979+0000","last_peered":"2024-04-25T10:01:17.648979+0000","last_clean":"2024-04-25T10:01:17.648979+0000","last_became_active":"2024-04-25T10:01:15.162938+0000","last_became_peered":"2024-04-25T10:01:15.162938+0000","last_unstale":"2024-04-25T10:01:17.648979+0000","last_undegraded":"2024-04-25T10:01:17.648979+0000","last_fullsized":"2024-04-25T10:01:17.648979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:39.351545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649087+0000","last_change":"2024-04-25T10:01:16.160505+0000","last_active":"2024-04-25T10:01:17.649087+0000","last_peered":"2024-04-25T10:01:17.649087+0000","last_clean":"2024-04-25T10:01:17.649087+0000","last_became_active":"2024-04-25T10:01:16.160221+0000","last_became_peered":"2024-04-25T10:01:16.160221+0000","last_unstale":"2024-04-25T10:01:17.649087+0000","last_undegraded":"2024-04-25T10:01:17.649087+0000","last_fullsized":"2024-04-25T10:01:17.649087+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:19.369840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648312+0000","last_change":"2024-04-25T10:01:15.155989+0000","last_active":"2024-04-25T10:01:17.648312+0000","last_peered":"2024-04-25T10:01:17.648312+0000","last_clean":"2024-04-25T10:01:17.648312+0000","last_became_active":"2024-04-25T10:01:15.155666+0000","last_became_peered":"2024-04-25T10:01:15.155666+0000","last_unstale":"2024-04-25T10:01:17.648312+0000","last_undegraded":"2024-04-25T10:01:17.648312+0000","last_fullsized":"2024-04-25T10:01:17.648312+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:13.316747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648231+0000","last_change":"2024-04-25T10:01:16.164745+0000","last_active":"2024-04-25T10:01:17.648231+0000","last_peered":"2024-04-25T10:01:17.648231+0000","last_clean":"2024-04-25T10:01:17.648231+0000","last_became_active":"2024-04-25T10:01:16.164129+0000","last_became_peered":"2024-04-25T10:01:16.164129+0000","last_unstale":"2024-04-25T10:01:17.648231+0000","last_undegraded":"2024-04-25T10:01:17.648231+0000","last_fullsized":"2024-04-25T10:01:17.648231+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:26.671115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648043+0000","last_change":"2024-04-25T10:01:15.154866+0000","last_active":"2024-04-25T10:01:17.648043+0000","last_peered":"2024-04-25T10:01:17.648043+0000","last_clean":"2024-04-25T10:01:17.648043+0000","last_became_active":"2024-04-25T10:01:15.154568+0000","last_became_peered":"2024-04-25T10:01:15.154568+0000","last_unstale":"2024-04-25T10:01:17.648043+0000","last_undegraded":"2024-04-25T10:01:17.648043+0000","last_fullsized":"2024-04-25T10:01:17.648043+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:21.789012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148397+0000","last_change":"2024-04-25T10:01:16.165952+0000","last_active":"2024-04-25T10:01:17.148397+0000","last_peered":"2024-04-25T10:01:17.148397+0000","last_clean":"2024-04-25T10:01:17.148397+0000","last_became_active":"2024-04-25T10:01:16.165530+0000","last_became_peered":"2024-04-25T10:01:16.165530+0000","last_unstale":"2024-04-25T10:01:17.148397+0000","last_undegraded":"2024-04-25T10:01:17.148397+0000","last_fullsized":"2024-04-25T10:01:17.148397+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:24.218627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"104'8","reported_seq":22,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.146797+0000","last_change":"2024-04-25T10:01:15.155813+0000","last_active":"2024-04-25T10:01:17.146797+0000","last_peered":"2024-04-25T10:01:17.146797+0000","last_clean":"2024-04-25T10:01:17.146797+0000","last_became_active":"2024-04-25T10:01:15.155138+0000","last_became_peered":"2024-04-25T10:01:15.155138+0000","last_unstale":"2024-04-25T10:01:17.146797+0000","last_undegraded":"2024-04-25T10:01:17.146797+0000","last_fullsized":"2024-04-25T10:01:17.146797+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:47.363045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648091+0000","last_change":"2024-04-25T10:01:16.168349+0000","last_active":"2024-04-25T10:01:17.648091+0000","last_peered":"2024-04-25T10:01:17.648091+0000","last_clean":"2024-04-25T10:01:17.648091+0000","last_became_active":"2024-04-25T10:01:16.168104+0000","last_became_peered":"2024-04-25T10:01:16.168104+0000","last_unstale":"2024-04-25T10:01:17.648091+0000","last_undegraded":"2024-04-25T10:01:17.648091+0000","last_fullsized":"2024-04-25T10:01:17.648091+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:58.421904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649469+0000","last_change":"2024-04-25T10:01:15.160924+0000","last_active":"2024-04-25T10:01:17.649469+0000","last_peered":"2024-04-25T10:01:17.649469+0000","last_clean":"2024-04-25T10:01:17.649469+0000","last_became_active":"2024-04-25T10:01:15.160638+0000","last_became_peered":"2024-04-25T10:01:15.160638+0000","last_unstale":"2024-04-25T10:01:17.649469+0000","last_undegraded":"2024-04-25T10:01:17.649469+0000","last_fullsized":"2024-04-25T10:01:17.649469+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:10.215320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649330+0000","last_change":"2024-04-25T10:01:16.168180+0000","last_active":"2024-04-25T10:01:17.649330+0000","last_peered":"2024-04-25T10:01:17.649330+0000","last_clean":"2024-04-25T10:01:17.649330+0000","last_became_active":"2024-04-25T10:01:16.167904+0000","last_became_peered":"2024-04-25T10:01:16.167904+0000","last_unstale":"2024-04-25T10:01:17.649330+0000","last_undegraded":"2024-04-25T10:01:17.649330+0000","last_fullsized":"2024-04-25T10:01:17.649330+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:51.790891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168974+0000","last_change":"2024-04-25T10:01:15.159510+0000","last_active":"2024-04-25T10:01:17.168974+0000","last_peered":"2024-04-25T10:01:17.168974+0000","last_clean":"2024-04-25T10:01:17.168974+0000","last_became_active":"2024-04-25T10:01:15.159244+0000","last_became_peered":"2024-04-25T10:01:15.159244+0000","last_unstale":"2024-04-25T10:01:17.168974+0000","last_undegraded":"2024-04-25T10:01:17.168974+0000","last_fullsized":"2024-04-25T10:01:17.168974+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:21.527132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648878+0000","last_change":"2024-04-25T10:01:16.172708+0000","last_active":"2024-04-25T10:01:17.648878+0000","last_peered":"2024-04-25T10:01:17.648878+0000","last_clean":"2024-04-25T10:01:17.648878+0000","last_became_active":"2024-04-25T10:01:16.172449+0000","last_became_peered":"2024-04-25T10:01:16.172449+0000","last_unstale":"2024-04-25T10:01:17.648878+0000","last_undegraded":"2024-04-25T10:01:17.648878+0000","last_fullsized":"2024-04-25T10:01:17.648878+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:14.322434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649053+0000","last_change":"2024-04-25T10:01:15.161768+0000","last_active":"2024-04-25T10:01:17.649053+0000","last_peered":"2024-04-25T10:01:17.649053+0000","last_clean":"2024-04-25T10:01:17.649053+0000","last_became_active":"2024-04-25T10:01:15.161484+0000","last_became_peered":"2024-04-25T10:01:15.161484+0000","last_unstale":"2024-04-25T10:01:17.649053+0000","last_undegraded":"2024-04-25T10:01:17.649053+0000","last_fullsized":"2024-04-25T10:01:17.649053+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:33.937222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147948+0000","last_change":"2024-04-25T10:01:16.160443+0000","last_active":"2024-04-25T10:01:17.147948+0000","last_peered":"2024-04-25T10:01:17.147948+0000","last_clean":"2024-04-25T10:01:17.147948+0000","last_became_active":"2024-04-25T10:01:16.159853+0000","last_became_peered":"2024-04-25T10:01:16.159853+0000","last_unstale":"2024-04-25T10:01:17.147948+0000","last_undegraded":"2024-04-25T10:01:17.147948+0000","last_fullsized":"2024-04-25T10:01:17.147948+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:47.775162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649030+0000","last_change":"2024-04-25T10:01:15.162292+0000","last_active":"2024-04-25T10:01:17.649030+0000","last_peered":"2024-04-25T10:01:17.649030+0000","last_clean":"2024-04-25T10:01:17.649030+0000","last_became_active":"2024-04-25T10:01:15.161924+0000","last_became_peered":"2024-04-25T10:01:15.161924+0000","last_unstale":"2024-04-25T10:01:17.649030+0000","last_undegraded":"2024-04-25T10:01:17.649030+0000","last_fullsized":"2024-04-25T10:01:17.649030+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:09.493092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148349+0000","last_change":"2024-04-25T10:01:16.160164+0000","last_active":"2024-04-25T10:01:17.148349+0000","last_peered":"2024-04-25T10:01:17.148349+0000","last_clean":"2024-04-25T10:01:17.148349+0000","last_became_active":"2024-04-25T10:01:16.159716+0000","last_became_peered":"2024-04-25T10:01:16.159716+0000","last_unstale":"2024-04-25T10:01:17.148349+0000","last_undegraded":"2024-04-25T10:01:17.148349+0000","last_fullsized":"2024-04-25T10:01:17.148349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:57.627511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648288+0000","last_change":"2024-04-25T10:01:15.162064+0000","last_active":"2024-04-25T10:01:17.648288+0000","last_peered":"2024-04-25T10:01:17.648288+0000","last_clean":"2024-04-25T10:01:17.648288+0000","last_became_active":"2024-04-25T10:01:15.161708+0000","last_became_peered":"2024-04-25T10:01:15.161708+0000","last_unstale":"2024-04-25T10:01:17.648288+0000","last_undegraded":"2024-04-25T10:01:17.648288+0000","last_fullsized":"2024-04-25T10:01:17.648288+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:19.710949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648597+0000","last_change":"2024-04-25T10:01:16.166536+0000","last_active":"2024-04-25T10:01:17.648597+0000","last_peered":"2024-04-25T10:01:17.648597+0000","last_clean":"2024-04-25T10:01:17.648597+0000","last_became_active":"2024-04-25T10:01:16.166239+0000","last_became_peered":"2024-04-25T10:01:16.166239+0000","last_unstale":"2024-04-25T10:01:17.648597+0000","last_undegraded":"2024-04-25T10:01:17.648597+0000","last_fullsized":"2024-04-25T10:01:17.648597+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:33.261097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648446+0000","last_change":"2024-04-25T10:01:15.162917+0000","last_active":"2024-04-25T10:01:17.648446+0000","last_peered":"2024-04-25T10:01:17.648446+0000","last_clean":"2024-04-25T10:01:17.648446+0000","last_became_active":"2024-04-25T10:01:15.162662+0000","last_became_peered":"2024-04-25T10:01:15.162662+0000","last_unstale":"2024-04-25T10:01:17.648446+0000","last_undegraded":"2024-04-25T10:01:17.648446+0000","last_fullsized":"2024-04-25T10:01:17.648446+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:26.356231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648543+0000","last_change":"2024-04-25T10:01:16.172338+0000","last_active":"2024-04-25T10:01:17.648543+0000","last_peered":"2024-04-25T10:01:17.648543+0000","last_clean":"2024-04-25T10:01:17.648543+0000","last_became_active":"2024-04-25T10:01:16.172069+0000","last_became_peered":"2024-04-25T10:01:16.172069+0000","last_unstale":"2024-04-25T10:01:17.648543+0000","last_undegraded":"2024-04-25T10:01:17.648543+0000","last_fullsized":"2024-04-25T10:01:17.648543+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:55.242526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648184+0000","last_change":"2024-04-25T10:01:15.162662+0000","last_active":"2024-04-25T10:01:17.648184+0000","last_peered":"2024-04-25T10:01:17.648184+0000","last_clean":"2024-04-25T10:01:17.648184+0000","last_became_active":"2024-04-25T10:01:15.162249+0000","last_became_peered":"2024-04-25T10:01:15.162249+0000","last_unstale":"2024-04-25T10:01:17.648184+0000","last_undegraded":"2024-04-25T10:01:17.648184+0000","last_fullsized":"2024-04-25T10:01:17.648184+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:53.469647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147930+0000","last_change":"2024-04-25T10:01:16.159845+0000","last_active":"2024-04-25T10:01:17.147930+0000","last_peered":"2024-04-25T10:01:17.147930+0000","last_clean":"2024-04-25T10:01:17.147930+0000","last_became_active":"2024-04-25T10:01:16.159504+0000","last_became_peered":"2024-04-25T10:01:16.159504+0000","last_unstale":"2024-04-25T10:01:17.147930+0000","last_undegraded":"2024-04-25T10:01:17.147930+0000","last_fullsized":"2024-04-25T10:01:17.147930+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:46.644548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648505+0000","last_change":"2024-04-25T10:01:15.162772+0000","last_active":"2024-04-25T10:01:17.648505+0000","last_peered":"2024-04-25T10:01:17.648505+0000","last_clean":"2024-04-25T10:01:17.648505+0000","last_became_active":"2024-04-25T10:01:15.162489+0000","last_became_peered":"2024-04-25T10:01:15.162489+0000","last_unstale":"2024-04-25T10:01:17.648505+0000","last_undegraded":"2024-04-25T10:01:17.648505+0000","last_fullsized":"2024-04-25T10:01:17.648505+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:10.215320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.146798+0000","last_change":"2024-04-25T10:01:16.161385+0000","last_active":"2024-04-25T10:01:17.146798+0000","last_peered":"2024-04-25T10:01:17.146798+0000","last_clean":"2024-04-25T10:01:17.146798+0000","last_became_active":"2024-04-25T10:01:16.158536+0000","last_became_peered":"2024-04-25T10:01:16.158536+0000","last_unstale":"2024-04-25T10:01:17.146798+0000","last_undegraded":"2024-04-25T10:01:17.146798+0000","last_fullsized":"2024-04-25T10:01:17.146798+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:11.683435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.148034+0000","last_change":"2024-04-25T10:01:15.157299+0000","last_active":"2024-04-25T10:01:17.148034+0000","last_peered":"2024-04-25T10:01:17.148034+0000","last_clean":"2024-04-25T10:01:17.148034+0000","last_became_active":"2024-04-25T10:01:15.157020+0000","last_became_peered":"2024-04-25T10:01:15.157020+0000","last_unstale":"2024-04-25T10:01:17.148034+0000","last_undegraded":"2024-04-25T10:01:17.148034+0000","last_fullsized":"2024-04-25T10:01:17.148034+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:40.320762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648224+0000","last_change":"2024-04-25T10:01:16.161304+0000","last_active":"2024-04-25T10:01:17.648224+0000","last_peered":"2024-04-25T10:01:17.648224+0000","last_clean":"2024-04-25T10:01:17.648224+0000","last_became_active":"2024-04-25T10:01:16.161063+0000","last_became_peered":"2024-04-25T10:01:16.161063+0000","last_unstale":"2024-04-25T10:01:17.648224+0000","last_undegraded":"2024-04-25T10:01:17.648224+0000","last_fullsized":"2024-04-25T10:01:17.648224+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:39.490172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649205+0000","last_change":"2024-04-25T10:01:15.162934+0000","last_active":"2024-04-25T10:01:17.649205+0000","last_peered":"2024-04-25T10:01:17.649205+0000","last_clean":"2024-04-25T10:01:17.649205+0000","last_became_active":"2024-04-25T10:01:15.162620+0000","last_became_peered":"2024-04-25T10:01:15.162620+0000","last_unstale":"2024-04-25T10:01:17.649205+0000","last_undegraded":"2024-04-25T10:01:17.649205+0000","last_fullsized":"2024-04-25T10:01:17.649205+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649360+0000","last_change":"2024-04-25T10:01:16.164602+0000","last_active":"2024-04-25T10:01:17.649360+0000","last_peered":"2024-04-25T10:01:17.649360+0000","last_clean":"2024-04-25T10:01:17.649360+0000","last_became_active":"2024-04-25T10:01:16.164330+0000","last_became_peered":"2024-04-25T10:01:16.164330+0000","last_unstale":"2024-04-25T10:01:17.649360+0000","last_undegraded":"2024-04-25T10:01:17.649360+0000","last_fullsized":"2024-04-25T10:01:17.649360+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.767089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648562+0000","last_change":"2024-04-25T10:01:15.161411+0000","last_active":"2024-04-25T10:01:17.648562+0000","last_peered":"2024-04-25T10:01:17.648562+0000","last_clean":"2024-04-25T10:01:17.648562+0000","last_became_active":"2024-04-25T10:01:15.161145+0000","last_became_peered":"2024-04-25T10:01:15.161145+0000","last_unstale":"2024-04-25T10:01:17.648562+0000","last_undegraded":"2024-04-25T10:01:17.648562+0000","last_fullsized":"2024-04-25T10:01:17.648562+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:40.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648689+0000","last_change":"2024-04-25T10:01:16.173871+0000","last_active":"2024-04-25T10:01:17.648689+0000","last_peered":"2024-04-25T10:01:17.648689+0000","last_clean":"2024-04-25T10:01:17.648689+0000","last_became_active":"2024-04-25T10:01:16.173358+0000","last_became_peered":"2024-04-25T10:01:16.173358+0000","last_unstale":"2024-04-25T10:01:17.648689+0000","last_undegraded":"2024-04-25T10:01:17.648689+0000","last_fullsized":"2024-04-25T10:01:17.648689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:40.833514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.147294+0000","last_change":"2024-04-25T10:01:15.161182+0000","last_active":"2024-04-25T10:01:17.147294+0000","last_peered":"2024-04-25T10:01:17.147294+0000","last_clean":"2024-04-25T10:01:17.147294+0000","last_became_active":"2024-04-25T10:01:15.160798+0000","last_became_peered":"2024-04-25T10:01:15.160798+0000","last_unstale":"2024-04-25T10:01:17.147294+0000","last_undegraded":"2024-04-25T10:01:17.147294+0000","last_fullsized":"2024-04-25T10:01:17.147294+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:35.385701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648297+0000","last_change":"2024-04-25T10:01:16.172361+0000","last_active":"2024-04-25T10:01:17.648297+0000","last_peered":"2024-04-25T10:01:17.648297+0000","last_clean":"2024-04-25T10:01:17.648297+0000","last_became_active":"2024-04-25T10:01:16.172096+0000","last_became_peered":"2024-04-25T10:01:16.172096+0000","last_unstale":"2024-04-25T10:01:17.648297+0000","last_undegraded":"2024-04-25T10:01:17.648297+0000","last_fullsized":"2024-04-25T10:01:17.648297+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:29.184821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648521+0000","last_change":"2024-04-25T10:01:15.154998+0000","last_active":"2024-04-25T10:01:17.648521+0000","last_peered":"2024-04-25T10:01:17.648521+0000","last_clean":"2024-04-25T10:01:17.648521+0000","last_became_active":"2024-04-25T10:01:15.154730+0000","last_became_peered":"2024-04-25T10:01:15.154730+0000","last_unstale":"2024-04-25T10:01:17.648521+0000","last_undegraded":"2024-04-25T10:01:17.648521+0000","last_fullsized":"2024-04-25T10:01:17.648521+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:37.084529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.169012+0000","last_change":"2024-04-25T10:01:16.164851+0000","last_active":"2024-04-25T10:01:17.169012+0000","last_peered":"2024-04-25T10:01:17.169012+0000","last_clean":"2024-04-25T10:01:17.169012+0000","last_became_active":"2024-04-25T10:01:16.164575+0000","last_became_peered":"2024-04-25T10:01:16.164575+0000","last_unstale":"2024-04-25T10:01:17.169012+0000","last_undegraded":"2024-04-25T10:01:17.169012+0000","last_fullsized":"2024-04-25T10:01:17.169012+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:59.616054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648571+0000","last_change":"2024-04-25T10:01:15.164883+0000","last_active":"2024-04-25T10:01:17.648571+0000","last_peered":"2024-04-25T10:01:17.648571+0000","last_clean":"2024-04-25T10:01:17.648571+0000","last_became_active":"2024-04-25T10:01:15.164658+0000","last_became_peered":"2024-04-25T10:01:15.164658+0000","last_unstale":"2024-04-25T10:01:17.648571+0000","last_undegraded":"2024-04-25T10:01:17.648571+0000","last_fullsized":"2024-04-25T10:01:17.648571+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:39.351545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.649302+0000","last_change":"2024-04-25T10:01:16.164389+0000","last_active":"2024-04-25T10:01:17.649302+0000","last_peered":"2024-04-25T10:01:17.649302+0000","last_clean":"2024-04-25T10:01:17.649302+0000","last_became_active":"2024-04-25T10:01:16.164070+0000","last_became_peered":"2024-04-25T10:01:16.164070+0000","last_unstale":"2024-04-25T10:01:17.649302+0000","last_undegraded":"2024-04-25T10:01:17.649302+0000","last_fullsized":"2024-04-25T10:01:17.649302+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:45.068283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648264+0000","last_change":"2024-04-25T10:01:15.162486+0000","last_active":"2024-04-25T10:01:17.648264+0000","last_peered":"2024-04-25T10:01:17.648264+0000","last_clean":"2024-04-25T10:01:17.648264+0000","last_became_active":"2024-04-25T10:01:15.161762+0000","last_became_peered":"2024-04-25T10:01:15.161762+0000","last_unstale":"2024-04-25T10:01:17.648264+0000","last_undegraded":"2024-04-25T10:01:17.648264+0000","last_fullsized":"2024-04-25T10:01:17.648264+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:22.481276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648796+0000","last_change":"2024-04-25T10:01:16.169200+0000","last_active":"2024-04-25T10:01:17.648796+0000","last_peered":"2024-04-25T10:01:17.648796+0000","last_clean":"2024-04-25T10:01:17.648796+0000","last_became_active":"2024-04-25T10:01:16.168923+0000","last_became_peered":"2024-04-25T10:01:16.168923+0000","last_unstale":"2024-04-25T10:01:17.648796+0000","last_undegraded":"2024-04-25T10:01:17.648796+0000","last_fullsized":"2024-04-25T10:01:17.648796+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:15.135070+0000","last_clean_scrub_stamp":"2024-04-25T10:01:15.135070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:14.322434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.648083+0000","last_change":"2024-04-25T10:01:15.162678+0000","last_active":"2024-04-25T10:01:17.648083+0000","last_peered":"2024-04-25T10:01:17.648083+0000","last_clean":"2024-04-25T10:01:17.648083+0000","last_became_active":"2024-04-25T10:01:15.162090+0000","last_became_peered":"2024-04-25T10:01:15.162090+0000","last_unstale":"2024-04-25T10:01:17.648083+0000","last_undegraded":"2024-04-25T10:01:17.648083+0000","last_fullsized":"2024-04-25T10:01:17.648083+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:14.129383+0000","last_clean_scrub_stamp":"2024-04-25T10:01:14.129383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:21.910671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":328,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:01:17.168610+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T10:01:17.168610+0000","last_peered":"2024-04-25T10:01:17.168610+0000","last_clean":"2024-04-25T10:01:17.168610+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T10:01:17.168610+0000","last_undegraded":"2024-04-25T10:01:17.168610+0000","last_fullsized":"2024-04-25T10:01:17.168610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574951,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1904640,"data_stored":1195149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1310720,"data_stored":600145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574950,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1302528,"data_stored":604621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1298432,"data_stored":598021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574951,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":598511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":604655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":597963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1912832,"data_stored":1196691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:19.456 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:01:19.456 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:01:19.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.593+0000 7f1b76046640 1 Processor -- start 2024-04-25T10:01:19.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.593+0000 7f1b76046640 1 -- start start 2024-04-25T10:01:19.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b76046640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1b700afc00 0x7f1b7006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b76046640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b700d0080 0x7f1b7006e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b76046640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b700ccef0 con 0x7f1b7015cb20 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b76046640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b700703c0 con 0x7f1b700afc00 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b76046640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b700cd820 con 0x7f1b700d0080 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b6f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1b700afc00 0x7f1b7006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b6f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1b700afc00 0x7f1b7006fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44178/0 (socket says 172.21.15.139:44178) 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b6f7fe640 1 -- 172.21.15.139:0/1815576051 learned_addr learned my addr 172.21.15.139:0/1815576051 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b6ffff640 1 --2- 172.21.15.139:0/1815576051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b700d0080 0x7f1b7006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b6f7fe640 1 -- 172.21.15.139:0/1815576051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b700d0080 msgr2=0x7f1b7006e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b6f7fe640 1 --2- 172.21.15.139:0/1815576051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b700d0080 0x7f1b7006e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b6f7fe640 1 -- 172.21.15.139:0/1815576051 >> v1:172.21.15.150:6789/0 conn(0x7f1b7015cb20 legacy=0x7f1b700cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.594+0000 7f1b6f7fe640 1 -- 172.21.15.139:0/1815576051 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b700cd1d0 con 0x7f1b700afc00 2024-04-25T10:01:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.595+0000 7f1b6f7fe640 1 --2- 172.21.15.139:0/1815576051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b700afc00 0x7f1b7006fb20 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f1b600098d0 tx=0x7f1b6002e560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce60b35d23da7f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.595+0000 7f1b6e7fc640 1 -- 172.21.15.139:0/1815576051 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6003a070 con 0x7f1b700afc00 2024-04-25T10:01:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.595+0000 7f1b6e7fc640 1 -- 172.21.15.139:0/1815576051 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b60033710 con 0x7f1b700afc00 2024-04-25T10:01:19.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.595+0000 7f1b6e7fc640 1 -- 172.21.15.139:0/1815576051 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60033ae0 con 0x7f1b700afc00 2024-04-25T10:01:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.596+0000 7f1b76046640 1 -- 172.21.15.139:0/1815576051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b700afc00 msgr2=0x7f1b7006fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.596+0000 7f1b76046640 1 --2- 172.21.15.139:0/1815576051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b700afc00 0x7f1b7006fb20 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f1b600098d0 tx=0x7f1b6002e560 comp rx=0 tx=0).stop 2024-04-25T10:01:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.596+0000 7f1b76046640 1 -- 172.21.15.139:0/1815576051 shutdown_connections 2024-04-25T10:01:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.596+0000 7f1b76046640 1 --2- 172.21.15.139:0/1815576051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b700d0080 0x7f1b7006e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.596+0000 7f1b76046640 1 --2- 172.21.15.139:0/1815576051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b700afc00 0x7f1b7006fb20 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.596+0000 7f1b76046640 1 -- 172.21.15.139:0/1815576051 >> 172.21.15.139:0/1815576051 conn(0x7f1b700b2230 msgr2=0x7f1b700b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.596+0000 7f1b76046640 1 -- 172.21.15.139:0/1815576051 shutdown_connections 2024-04-25T10:01:19.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.596+0000 7f1b76046640 1 -- 172.21.15.139:0/1815576051 wait complete. 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.597+0000 7f1b76046640 1 Processor -- start 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.597+0000 7f1b76046640 1 -- start start 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b76046640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b700afc00 0x7f1b700af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b76046640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b700d0080 0x7f1b7015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b76046640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b7015cb20 0x7f1b701583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b76046640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b70070060 con 0x7f1b700d0080 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b76046640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b700affe0 con 0x7f1b7015cb20 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b76046640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b700cdcb0 con 0x7f1b700afc00 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b6effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b700d0080 0x7f1b7015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b6effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b700d0080 0x7f1b7015b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44190/0 (socket says 172.21.15.139:44190) 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b6effd640 1 -- 172.21.15.139:0/970252760 learned_addr learned my addr 172.21.15.139:0/970252760 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b6f7fe640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b700afc00 0x7f1b700af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b6ffff640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b7015cb20 0x7f1b701583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b6effd640 1 -- 172.21.15.139:0/970252760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b700afc00 msgr2=0x7f1b700af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b6effd640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b700afc00 0x7f1b700af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b6effd640 1 -- 172.21.15.139:0/970252760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b7015cb20 msgr2=0x7f1b701583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b6effd640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b7015cb20 0x7f1b701583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b6effd640 1 -- 172.21.15.139:0/970252760 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b700c5a60 con 0x7f1b700d0080 2024-04-25T10:01:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.598+0000 7f1b6f7fe640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b700afc00 0x7f1b700af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.599+0000 7f1b6effd640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b700d0080 0x7f1b7015b4d0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400c400 tx=0x7f1b6400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.599+0000 7f1b57fff640 1 -- 172.21.15.139:0/970252760 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b640052d0 con 0x7f1b700d0080 2024-04-25T10:01:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.599+0000 7f1b57fff640 1 -- 172.21.15.139:0/970252760 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b64005490 con 0x7f1b700d0080 2024-04-25T10:01:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.599+0000 7f1b76046640 1 -- 172.21.15.139:0/970252760 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b700c5d40 con 0x7f1b700d0080 2024-04-25T10:01:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.599+0000 7f1b57fff640 1 -- 172.21.15.139:0/970252760 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64007260 con 0x7f1b700d0080 2024-04-25T10:01:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.599+0000 7f1b76046640 1 -- 172.21.15.139:0/970252760 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b700c6300 con 0x7f1b700d0080 2024-04-25T10:01:19.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.600+0000 7f1b76046640 1 -- 172.21.15.139:0/970252760 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b70074520 con 0x7f1b700d0080 2024-04-25T10:01:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.600+0000 7f1b57fff640 1 -- 172.21.15.139:0/970252760 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b64007400 con 0x7f1b700d0080 2024-04-25T10:01:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.601+0000 7f1b57fff640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b5c06b4d0 0x7f1b5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.601+0000 7f1b6f7fe640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b5c06b4d0 0x7f1b5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.601+0000 7f1b57fff640 1 -- 172.21.15.139:0/970252760 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1b64089910 con 0x7f1b700d0080 2024-04-25T10:01:19.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.601+0000 7f1b6f7fe640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b5c06b4d0 0x7f1b5c06d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1b60037000 tx=0x7f1b60008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:19.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.605+0000 7f1b57fff640 1 -- 172.21.15.139:0/970252760 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b64058600 con 0x7f1b700d0080 2024-04-25T10:01:19.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.840+0000 7f1b76046640 1 -- 172.21.15.139:0/970252760 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1b70157e90 con 0x7f1b700d0080 2024-04-25T10:01:19.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.843+0000 7f1b57fff640 1 -- 172.21.15.139:0/970252760 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1b64057f00 con 0x7f1b700d0080 2024-04-25T10:01:19.847 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2024-04-25T10:01:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg== 2024-04-25T10:01:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: caps mds = "allow" 2024-04-25T10:01:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: caps mon = "allow r" 2024-04-25T10:01:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:01:19.847 INFO:teuthology.orchestra.run.smithi139.stderr:updated caps for client.0 2024-04-25T10:01:19.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.845+0000 7f1b76046640 1 -- 172.21.15.139:0/970252760 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b5c06b4d0 msgr2=0x7f1b5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.845+0000 7f1b76046640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b5c06b4d0 0x7f1b5c06d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1b60037000 tx=0x7f1b60008030 comp rx=0 tx=0).stop 2024-04-25T10:01:19.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.845+0000 7f1b76046640 1 -- 172.21.15.139:0/970252760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b700d0080 msgr2=0x7f1b7015b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.845+0000 7f1b76046640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b700d0080 0x7f1b7015b4d0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400c400 tx=0x7f1b6400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.845+0000 7f1b76046640 1 -- 172.21.15.139:0/970252760 shutdown_connections 2024-04-25T10:01:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.845+0000 7f1b76046640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b700d0080 0x7f1b7015b4d0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.845+0000 7f1b76046640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b5c06b4d0 0x7f1b5c06d990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.845+0000 7f1b76046640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b7015cb20 0x7f1b701583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.845+0000 7f1b76046640 1 --2- 172.21.15.139:0/970252760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b700afc00 0x7f1b700af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.845+0000 7f1b76046640 1 -- 172.21.15.139:0/970252760 >> 172.21.15.139:0/970252760 conn(0x7f1b700b2230 msgr2=0x7f1b700cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.845+0000 7f1b76046640 1 -- 172.21.15.139:0/970252760 shutdown_connections 2024-04-25T10:01:19.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.845+0000 7f1b76046640 1 -- 172.21.15.139:0/970252760 wait complete. 2024-04-25T10:01:19.860 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:19.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.953+0000 7fd639eee640 1 Processor -- start 2024-04-25T10:01:19.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.953+0000 7fd639eee640 1 -- start start 2024-04-25T10:01:19.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd639eee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6340cf900 0x7fd6340d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd639eee640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6340cfe70 con 0x7fd63415efb0 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd639eee640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6340d0150 con 0x7fd634099a50 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd639eee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340d0a80 con 0x7fd6340cf900 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd6337fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd63415efb0 0x7fd63406c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38388/0 (socket says 172.21.15.139:38388) 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd6337fe640 1 -- 172.21.15.139:0/933196225 learned_addr learned my addr 172.21.15.139:0/933196225 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd632ffd640 1 --2- 172.21.15.139:0/933196225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6340cf900 0x7fd6340d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd6327fc640 1 -- 172.21.15.139:0/933196225 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248541650 0 0) 0x7fd6340cfe70 con 0x7fd63415efb0 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd6327fc640 1 -- 172.21.15.139:0/933196225 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd61c003520 con 0x7fd63415efb0 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd6327fc640 1 -- 172.21.15.139:0/933196225 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718252825 0 0) 0x7fd6340d0150 con 0x7fd634099a50 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd6327fc640 1 -- 172.21.15.139:0/933196225 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6340cfe70 con 0x7fd634099a50 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd6327fc640 1 -- 172.21.15.139:0/933196225 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3932805099 0 0) 0x7fd61c003520 con 0x7fd63415efb0 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd632ffd640 1 -- 172.21.15.139:0/933196225 >> v1:172.21.15.150:6789/0 conn(0x7fd634099a50 legacy=0x7fd6340cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd632ffd640 1 -- 172.21.15.139:0/933196225 >> v1:172.21.15.139:6789/0 conn(0x7fd63415efb0 legacy=0x7fd63406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.954+0000 7fd632ffd640 1 -- 172.21.15.139:0/933196225 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6340d0430 con 0x7fd6340cf900 2024-04-25T10:01:19.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.955+0000 7fd632ffd640 1 --2- 172.21.15.139:0/933196225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6340cf900 0x7fd6340d0f10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd6280097d0 tx=0x7fd62802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab2c649bb2536438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:19.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.955+0000 7fd6327fc640 1 -- 172.21.15.139:0/933196225 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628037070 con 0x7fd6340cf900 2024-04-25T10:01:19.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.955+0000 7fd6327fc640 1 -- 172.21.15.139:0/933196225 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62802ab90 con 0x7fd6340cf900 2024-04-25T10:01:19.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.955+0000 7fd6327fc640 1 -- 172.21.15.139:0/933196225 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624001670 con 0x7fd6340cf900 2024-04-25T10:01:19.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.956+0000 7fd639eee640 1 -- 172.21.15.139:0/933196225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6340cf900 msgr2=0x7fd6340d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.956+0000 7fd639eee640 1 --2- 172.21.15.139:0/933196225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6340cf900 0x7fd6340d0f10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd6280097d0 tx=0x7fd62802a100 comp rx=0 tx=0).stop 2024-04-25T10:01:19.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.956+0000 7fd639eee640 1 -- 172.21.15.139:0/933196225 shutdown_connections 2024-04-25T10:01:19.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.956+0000 7fd639eee640 1 --2- 172.21.15.139:0/933196225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6340cf900 0x7fd6340d0f10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.956+0000 7fd639eee640 1 -- 172.21.15.139:0/933196225 >> 172.21.15.139:0/933196225 conn(0x7fd6340d1790 msgr2=0x7fd6340d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.956+0000 7fd639eee640 1 -- 172.21.15.139:0/933196225 shutdown_connections 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.956+0000 7fd639eee640 1 -- 172.21.15.139:0/933196225 wait complete. 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.957+0000 7fd639eee640 1 Processor -- start 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.957+0000 7fd639eee640 1 -- start start 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd639eee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd634099a50 0x7fd6340b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd639eee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6340cf900 0x7fd634160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd639eee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63415efb0 0x7fd6341e7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd639eee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd634099ee0 con 0x7fd6340cf900 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd639eee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63406cf50 con 0x7fd63415efb0 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd639eee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63409a060 con 0x7fd634099a50 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd632ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6340cf900 0x7fd634160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd632ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6340cf900 0x7fd634160ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44196/0 (socket says 172.21.15.139:44196) 2024-04-25T10:01:19.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd632ffd640 1 -- 172.21.15.139:0/3664166523 learned_addr learned my addr 172.21.15.139:0/3664166523 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd6337fe640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd634099a50 0x7fd6340b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd633fff640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63415efb0 0x7fd6341e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd632ffd640 1 -- 172.21.15.139:0/3664166523 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd634099a50 msgr2=0x7fd6340b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd632ffd640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd634099a50 0x7fd6340b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd632ffd640 1 -- 172.21.15.139:0/3664166523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63415efb0 msgr2=0x7fd6341e7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd632ffd640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63415efb0 0x7fd6341e7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd632ffd640 1 -- 172.21.15.139:0/3664166523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6341591e0 con 0x7fd6340cf900 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.958+0000 7fd632ffd640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6340cf900 0x7fd634160ed0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd628035040 tx=0x7fd6280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.959+0000 7fd61bfff640 1 -- 172.21.15.139:0/3664166523 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628037050 con 0x7fd6340cf900 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.959+0000 7fd61bfff640 1 -- 172.21.15.139:0/3664166523 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd628043070 con 0x7fd6340cf900 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.959+0000 7fd639eee640 1 -- 172.21.15.139:0/3664166523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd634145ab0 con 0x7fd6340cf900 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.959+0000 7fd61bfff640 1 -- 172.21.15.139:0/3664166523 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628006e10 con 0x7fd6340cf900 2024-04-25T10:01:19.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.959+0000 7fd639eee640 1 -- 172.21.15.139:0/3664166523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd634146070 con 0x7fd6340cf900 2024-04-25T10:01:19.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.960+0000 7fd639eee640 1 -- 172.21.15.139:0/3664166523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd634074520 con 0x7fd6340cf900 2024-04-25T10:01:19.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.960+0000 7fd61bfff640 1 -- 172.21.15.139:0/3664166523 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd62803ca10 con 0x7fd6340cf900 2024-04-25T10:01:19.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.961+0000 7fd61bfff640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd62006b500 0x7fd62006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.961+0000 7fd61bfff640 1 -- 172.21.15.139:0/3664166523 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd6280aaf00 con 0x7fd6340cf900 2024-04-25T10:01:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.961+0000 7fd6337fe640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd62006b500 0x7fd62006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:19.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.961+0000 7fd6337fe640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd62006b500 0x7fd62006d9c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd6240011e0 tx=0x7fd624001710 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:19.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:19.965+0000 7fd61bfff640 1 -- 172.21.15.139:0/3664166523 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6341591e0 con 0x7fd6340cf900 2024-04-25T10:01:20.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.175+0000 7fd639eee640 1 -- 172.21.15.139:0/3664166523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd63409ae50 con 0x7fd6340cf900 2024-04-25T10:01:20.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.175+0000 7fd61bfff640 1 -- 172.21.15.139:0/3664166523 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fd6280796c0 con 0x7fd6340cf900 2024-04-25T10:01:20.180 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:20.180 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":69,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:15.497940+0000","modified":"2024-04-25T10:01:17.495730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8466":{"gid":8466,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/4074693991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4074693991},{"type":"v1","addr":"172.21.15.139:6837","nonce":4074693991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:20.180 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 69 2024-04-25T10:01:20.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.178+0000 7fd639eee640 1 -- 172.21.15.139:0/3664166523 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd62006b500 msgr2=0x7fd62006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.178+0000 7fd639eee640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd62006b500 0x7fd62006d9c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd6240011e0 tx=0x7fd624001710 comp rx=0 tx=0).stop 2024-04-25T10:01:20.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.178+0000 7fd639eee640 1 -- 172.21.15.139:0/3664166523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6340cf900 msgr2=0x7fd634160ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.178+0000 7fd639eee640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6340cf900 0x7fd634160ed0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd628035040 tx=0x7fd6280046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.178+0000 7fd639eee640 1 -- 172.21.15.139:0/3664166523 shutdown_connections 2024-04-25T10:01:20.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.178+0000 7fd639eee640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6340cf900 0x7fd634160ed0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.178+0000 7fd639eee640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd62006b500 0x7fd62006d9c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.178+0000 7fd639eee640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63415efb0 0x7fd6341e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.178+0000 7fd639eee640 1 --2- 172.21.15.139:0/3664166523 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd634099a50 0x7fd6340b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.178+0000 7fd639eee640 1 -- 172.21.15.139:0/3664166523 >> 172.21.15.139:0/3664166523 conn(0x7fd6340d1790 msgr2=0x7fd6340cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:20.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.178+0000 7fd639eee640 1 -- 172.21.15.139:0/3664166523 shutdown_connections 2024-04-25T10:01:20.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.178+0000 7fd639eee640 1 -- 172.21.15.139:0/3664166523 wait complete. 2024-04-25T10:01:20.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:01:15.497940+0000', 'modified': '2024-04-25T10:01:17.495730+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8466}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8466': {'gid': 8466, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6837/4074693991', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6835', 'nonce': 4074693991}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 4074693991}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:01:20.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:01:20.596 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:01:20.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.694+0000 7f828fe76640 1 Processor -- start 2024-04-25T10:01:20.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.695+0000 7f828fe76640 1 -- start start 2024-04-25T10:01:20.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.695+0000 7f828fe76640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f82880b5170 0x7f82880b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.695+0000 7f828fe76640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f828806bf50 0x7f828806c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.695+0000 7f828fe76640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8288070f50 con 0x7f828815db40 2024-04-25T10:01:20.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.695+0000 7f828fe76640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82880b5b30 con 0x7f82880b5170 2024-04-25T10:01:20.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.695+0000 7f828fe76640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8288071230 con 0x7f828806bf50 2024-04-25T10:01:20.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.695+0000 7f828dc12640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f828815db40 0x7f82880b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38392/0 (socket says 172.21.15.139:38392) 2024-04-25T10:01:20.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.696+0000 7f828dc12640 1 -- 172.21.15.139:0/3806874234 learned_addr learned my addr 172.21.15.139:0/3806874234 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:20.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.696+0000 7f828d411640 1 --2- 172.21.15.139:0/3806874234 >> v2:172.21.15.139:3300/0 conn(0x7f82880b5170 0x7f82880b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.696+0000 7f828e413640 1 --2- 172.21.15.139:0/3806874234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f828806bf50 0x7f828806c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.696+0000 7f828d411640 1 -- 172.21.15.139:0/3806874234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f828806bf50 msgr2=0x7f828806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:20.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.696+0000 7f828d411640 1 --2- 172.21.15.139:0/3806874234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f828806bf50 0x7f828806c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.696+0000 7f828d411640 1 -- 172.21.15.139:0/3806874234 >> v1:172.21.15.139:6789/0 conn(0x7f828815db40 legacy=0x7f82880b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.696+0000 7f828d411640 1 -- 172.21.15.139:0/3806874234 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288071510 con 0x7f82880b5170 2024-04-25T10:01:20.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.696+0000 7f828e413640 1 --2- 172.21.15.139:0/3806874234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f828806bf50 0x7f828806c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.696+0000 7f828d411640 1 --2- 172.21.15.139:0/3806874234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82880b5170 0x7f82880b55f0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f82780097a0 tx=0x7f827802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f8f9936db26b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.697+0000 7f828cc10640 1 -- 172.21.15.139:0/3806874234 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278037070 con 0x7f82880b5170 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.697+0000 7f828cc10640 1 -- 172.21.15.139:0/3806874234 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8278032440 con 0x7f82880b5170 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.697+0000 7f828cc10640 1 -- 172.21.15.139:0/3806874234 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278032810 con 0x7f82880b5170 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.697+0000 7f828fe76640 1 -- 172.21.15.139:0/3806874234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82880b5170 msgr2=0x7f82880b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.697+0000 7f828fe76640 1 --2- 172.21.15.139:0/3806874234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82880b5170 0x7f82880b55f0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f82780097a0 tx=0x7f827802a180 comp rx=0 tx=0).stop 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.697+0000 7f828fe76640 1 -- 172.21.15.139:0/3806874234 shutdown_connections 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.697+0000 7f828fe76640 1 --2- 172.21.15.139:0/3806874234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f828806bf50 0x7f828806c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.697+0000 7f828fe76640 1 --2- 172.21.15.139:0/3806874234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f82880b5170 0x7f82880b55f0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.697+0000 7f828fe76640 1 -- 172.21.15.139:0/3806874234 >> 172.21.15.139:0/3806874234 conn(0x7f828806f460 msgr2=0x7f828806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.697+0000 7f828fe76640 1 -- 172.21.15.139:0/3806874234 shutdown_connections 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.697+0000 7f828fe76640 1 -- 172.21.15.139:0/3806874234 wait complete. 2024-04-25T10:01:20.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.698+0000 7f828fe76640 1 Processor -- start 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.698+0000 7f828fe76640 1 -- start start 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.699+0000 7f828fe76640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f828806bf50 0x7f82881f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.699+0000 7f828fe76640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82880b5170 0x7f82880cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.699+0000 7f828fe76640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f828815db40 0x7f82881569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.699+0000 7f828fe76640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82880b4d90 con 0x7f828815db40 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.699+0000 7f828fe76640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8288071ad0 con 0x7f82880b5170 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.699+0000 7f828fe76640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8288070be0 con 0x7f828806bf50 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.699+0000 7f828e413640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f828815db40 0x7f82881569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.699+0000 7f828e413640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f828815db40 0x7f82881569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44220/0 (socket says 172.21.15.139:44220) 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.699+0000 7f828e413640 1 -- 172.21.15.139:0/2350157117 learned_addr learned my addr 172.21.15.139:0/2350157117 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.700+0000 7f828dc12640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f828806bf50 0x7f82881f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.700+0000 7f828e413640 1 -- 172.21.15.139:0/2350157117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f828806bf50 msgr2=0x7f82881f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.700+0000 7f828e413640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f828806bf50 0x7f82881f3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.700+0000 7f828e413640 1 -- 172.21.15.139:0/2350157117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82880b5170 msgr2=0x7f82880cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.700+0000 7f828e413640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82880b5170 0x7f82880cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.700+0000 7f828e413640 1 -- 172.21.15.139:0/2350157117 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82880cb950 con 0x7f828815db40 2024-04-25T10:01:20.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.700+0000 7f828e413640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f828815db40 0x7f82881569e0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f828000cef0 tx=0x7f828000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:20.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.700+0000 7f827e7fc640 1 -- 172.21.15.139:0/2350157117 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828001a070 con 0x7f828815db40 2024-04-25T10:01:20.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.700+0000 7f827e7fc640 1 -- 172.21.15.139:0/2350157117 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828000d860 con 0x7f828815db40 2024-04-25T10:01:20.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.700+0000 7f828fe76640 1 -- 172.21.15.139:0/2350157117 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82881e7660 con 0x7f828815db40 2024-04-25T10:01:20.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.700+0000 7f827e7fc640 1 -- 172.21.15.139:0/2350157117 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828000dc50 con 0x7f828815db40 2024-04-25T10:01:20.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.700+0000 7f828fe76640 1 -- 172.21.15.139:0/2350157117 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82881e7c20 con 0x7f828815db40 2024-04-25T10:01:20.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.702+0000 7f827e7fc640 1 -- 172.21.15.139:0/2350157117 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f828001d070 con 0x7f828815db40 2024-04-25T10:01:20.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.702+0000 7f827e7fc640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f826c06b4d0 0x7f826c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.702+0000 7f827e7fc640 1 -- 172.21.15.139:0/2350157117 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f828008ac00 con 0x7f828815db40 2024-04-25T10:01:20.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.702+0000 7f828dc12640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f826c06b4d0 0x7f826c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.703+0000 7f828dc12640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f826c06b4d0 0x7f826c06d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8284000f60 tx=0x7f8284009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:20.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.703+0000 7f8292b7c740 1 Processor -- start 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.703+0000 7f8292b7c740 1 -- start start 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f8292b7c740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c956230 0x55618c956610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f8292b7c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c974020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f8292b7c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c92bf10 0x55618c972190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f8292b7c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618c88ea10 con 0x55618c973ad0 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f8292b7c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618c837b30 con 0x55618c92bf10 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f8292b7c740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55618c88ac50 con 0x55618c956230 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f828d411640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c974020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f828d411640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c974020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44232/0 (socket says 172.21.15.139:44232) 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f828d411640 1 -- 172.21.15.139:0/466248203 learned_addr learned my addr 172.21.15.139:0/466248203 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f828dc12640 1 --2- 172.21.15.139:0/466248203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c956230 0x55618c956610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f828e413640 1 --2- 172.21.15.139:0/466248203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c92bf10 0x55618c972190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f828d411640 1 -- 172.21.15.139:0/466248203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c956230 msgr2=0x55618c956610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:20.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f828d411640 1 --2- 172.21.15.139:0/466248203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c956230 0x55618c956610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f828d411640 1 -- 172.21.15.139:0/466248203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c92bf10 msgr2=0x55618c972190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f828d411640 1 --2- 172.21.15.139:0/466248203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c92bf10 0x55618c972190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f828d411640 1 -- 172.21.15.139:0/466248203 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55618c84ce10 con 0x55618c973ad0 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f828e413640 1 --2- 172.21.15.139:0/466248203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c92bf10 0x55618c972190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f828d411640 1 --2- 172.21.15.139:0/466248203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c974020 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f8278035040 tx=0x7f827802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a10d0cdc3b63259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f826b7fe640 1 -- 172.21.15.139:0/466248203 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278037040 con 0x55618c973ad0 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f826b7fe640 1 -- 172.21.15.139:0/466248203 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8278032500 con 0x55618c973ad0 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.704+0000 7f826b7fe640 1 -- 172.21.15.139:0/466248203 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82780328f0 con 0x55618c973ad0 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.705+0000 7f8292b7c740 1 -- 172.21.15.139:0/466248203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 msgr2=0x55618c974020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.705+0000 7f8292b7c740 1 --2- 172.21.15.139:0/466248203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c974020 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f8278035040 tx=0x7f827802c070 comp rx=0 tx=0).stop 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.705+0000 7f8292b7c740 1 -- 172.21.15.139:0/466248203 shutdown_connections 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.705+0000 7f8292b7c740 1 --2- 172.21.15.139:0/466248203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c974020 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.705+0000 7f8292b7c740 1 --2- 172.21.15.139:0/466248203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c92bf10 0x55618c972190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.705+0000 7f8292b7c740 1 --2- 172.21.15.139:0/466248203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c956230 0x55618c956610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.705+0000 7f8292b7c740 1 -- 172.21.15.139:0/466248203 >> 172.21.15.139:0/466248203 conn(0x55618c928840 msgr2=0x55618c971040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.705+0000 7f8292b7c740 1 -- 172.21.15.139:0/466248203 shutdown_connections 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.705+0000 7f8292b7c740 1 -- 172.21.15.139:0/466248203 wait complete. 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.706+0000 7f8292b7c740 1 -- start start 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.706+0000 7f8292b7c740 1 Processor -- start 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8292b7c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c956230 0x55618c982300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8292b7c740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c973ad0 0x55618c96fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8292b7c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c9680f0 0x55618c96a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8292b7c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618c92ccb0 con 0x55618c9680f0 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8292b7c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618c84b0c0 con 0x55618c956230 2024-04-25T10:01:20.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8292b7c740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55618c926770 con 0x55618c973ad0 2024-04-25T10:01:20.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8292b7c740 10 client.0 _ll_register_callbacks cb 0x55618c83eec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:20.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828e413640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c9680f0 0x55618c96a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828e413640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c9680f0 0x55618c96a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44248/0 (socket says 172.21.15.139:44248) 2024-04-25T10:01:20.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828e413640 1 -- 172.21.15.139:0/570786416 learned_addr learned my addr 172.21.15.139:0/570786416 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:20.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828dc12640 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c956230 0x55618c982300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828d411640 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c973ad0 0x55618c96fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828e413640 1 -- 172.21.15.139:0/570786416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c973ad0 msgr2=0x55618c96fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:20.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828e413640 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c973ad0 0x55618c96fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828e413640 1 -- 172.21.15.139:0/570786416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c956230 msgr2=0x55618c982300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:20.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828e413640 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c956230 0x55618c982300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828e413640 1 -- 172.21.15.139:0/570786416 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55618c5a32d0 con 0x55618c9680f0 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828dc12640 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c956230 0x55618c982300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828d411640 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c973ad0 0x55618c96fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f828e413640 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c9680f0 0x55618c96a3e0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f8280059180 tx=0x7f828005d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b001f2365f84a8fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8268ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8268ff9640 1 -- 172.21.15.139:0/570786416 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280061470 con 0x55618c9680f0 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8268ff9640 1 -- 172.21.15.139:0/570786416 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8280061610 con 0x55618c9680f0 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8292b7c740 1 -- 172.21.15.139:0/570786416 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8278032500 con 0x55618c9680f0 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8268ff9640 1 -- 172.21.15.139:0/570786416 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8280058640 con 0x55618c9680f0 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8292b7c740 10 client.8607 fetch_fsmap learned FSMap version 69 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8292b7c740 1 -- 172.21.15.139:0/570786416 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55618c66d6b0 con 0x55618c9680f0 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.707+0000 7f8268ff9640 1 -- 172.21.15.139:0/570786416 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8278032500 con 0x55618c9680f0 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.708+0000 7f8268ff9640 1 -- 172.21.15.139:0/570786416 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f82800587e0 con 0x55618c9680f0 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.708+0000 7f8292b7c740 10 client.8607 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:01:20.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.708+0000 7f8292b7c740 10 client.8607 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.708+0000 7f8292b7c740 20 client.8607 populate_metadata read hostname 'smithi139' 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.708+0000 7f8292b7c740 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55618c98d560 0x55618c98f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.708+0000 7f8292b7c740 4 client.8607 mds_command: new command op to 8466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.708+0000 7f8292b7c740 1 -- 172.21.15.139:0/570786416 --> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55618c8ed920 con 0x55618c98d560 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.708+0000 7f828dc12640 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55618c98d560 0x55618c98f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.708+0000 7f828dc12640 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55618c98d560 0x55618c98f950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d6b4dd64a10b56 server_cookie=7b60d133b50bdbd5 in_seq=0 out_seq=0 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.708+0000 7f8268ff9640 10 client.8607 ms_handle_connect on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8268ff9640 1 -- 172.21.15.139:0/570786416 <== mds.0 v2:172.21.15.139:6835/4074693991 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55618c8ed920 con 0x55618c98d560 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8268ff9640 10 client.8607 handle_command_reply: tid=0 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8292b7c740 1 client.8607 shutdown 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8292b7c740 1 -- 172.21.15.139:0/570786416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c9680f0 msgr2=0x55618c96a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8292b7c740 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c9680f0 0x55618c96a3e0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f8280059180 tx=0x7f828005d340 comp rx=0 tx=0).stop 2024-04-25T10:01:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8292b7c740 1 -- 172.21.15.139:0/570786416 shutdown_connections 2024-04-25T10:01:20.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8292b7c740 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55618c98d560 0x55618c98f950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8292b7c740 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c9680f0 0x55618c96a3e0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8292b7c740 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c973ad0 0x55618c96fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8292b7c740 1 --2- 172.21.15.139:0/570786416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c956230 0x55618c982300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8292b7c740 1 -- 172.21.15.139:0/570786416 >> 172.21.15.139:0/570786416 conn(0x55618c92bf10 msgr2=0x55618c9745c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:20.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8268ff9640 0 client.8607 ms_handle_reset on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:20.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8292b7c740 1 -- 172.21.15.139:0/570786416 shutdown_connections 2024-04-25T10:01:20.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8292b7c740 1 -- 172.21.15.139:0/570786416 wait complete. 2024-04-25T10:01:20.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.709+0000 7f8292b7c740 20 client.8607 trim_cache size 0 max 16384 2024-04-25T10:01:20.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.724+0000 7f8292b7c740 1 Processor -- start 2024-04-25T10:01:20.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.724+0000 7f8292b7c740 1 -- start start 2024-04-25T10:01:20.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f8292b7c740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c956230 0x55618c9748c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f8292b7c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c9680f0 0x55618c993970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f8292b7c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c993eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f8292b7c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82881f7ca0 con 0x55618c973ad0 2024-04-25T10:01:20.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f8292b7c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82881e9420 con 0x55618c9680f0 2024-04-25T10:01:20.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f8292b7c740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82881e8820 con 0x55618c956230 2024-04-25T10:01:20.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f828e413640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c993eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f828e413640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c993eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44250/0 (socket says 172.21.15.139:44250) 2024-04-25T10:01:20.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f828e413640 1 -- 172.21.15.139:0/3601417719 learned_addr learned my addr 172.21.15.139:0/3601417719 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f828e413640 1 -- 172.21.15.139:0/3601417719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c956230 msgr2=0x55618c9748c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f828e413640 1 --2- 172.21.15.139:0/3601417719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c956230 0x55618c9748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f828e413640 1 -- 172.21.15.139:0/3601417719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c9680f0 msgr2=0x55618c993970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f828e413640 1 --2- 172.21.15.139:0/3601417719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c9680f0 0x55618c993970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.725+0000 7f828e413640 1 -- 172.21.15.139:0/3601417719 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55618c956980 con 0x55618c973ad0 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f828e413640 1 --2- 172.21.15.139:0/3601417719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c993eb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f82800613d0 tx=0x7f8280017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b3e89de20a7df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f826affd640 1 -- 172.21.15.139:0/3601417719 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280055490 con 0x55618c973ad0 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f826affd640 1 -- 172.21.15.139:0/3601417719 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82800587e0 con 0x55618c973ad0 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f826affd640 1 -- 172.21.15.139:0/3601417719 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280061470 con 0x55618c973ad0 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f8292b7c740 1 -- 172.21.15.139:0/3601417719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 msgr2=0x55618c993eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f8292b7c740 1 --2- 172.21.15.139:0/3601417719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c993eb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f82800613d0 tx=0x7f8280017000 comp rx=0 tx=0).stop 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f8292b7c740 1 -- 172.21.15.139:0/3601417719 shutdown_connections 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f8292b7c740 1 --2- 172.21.15.139:0/3601417719 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c993eb0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f8292b7c740 1 --2- 172.21.15.139:0/3601417719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c9680f0 0x55618c993970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f8292b7c740 1 --2- 172.21.15.139:0/3601417719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c956230 0x55618c9748c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f8292b7c740 1 -- 172.21.15.139:0/3601417719 >> 172.21.15.139:0/3601417719 conn(0x55618c92bf10 msgr2=0x55618c96b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f8292b7c740 1 -- 172.21.15.139:0/3601417719 shutdown_connections 2024-04-25T10:01:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.726+0000 7f8292b7c740 1 -- 172.21.15.139:0/3601417719 wait complete. 2024-04-25T10:01:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.727+0000 7f8292b7c740 1 -- start start 2024-04-25T10:01:20.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.727+0000 7f8292b7c740 1 Processor -- start 2024-04-25T10:01:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f8292b7c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c956230 0x55618c9df0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f8292b7c740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c9680f0 0x55618c97dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f8292b7c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c97e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f8292b7c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618c957330 con 0x55618c973ad0 2024-04-25T10:01:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f8292b7c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82881e8e20 con 0x55618c956230 2024-04-25T10:01:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f8292b7c740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55618c957780 con 0x55618c9680f0 2024-04-25T10:01:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f8292b7c740 10 client.0 _ll_register_callbacks cb 0x55618c926c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f828e413640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c97e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f828e413640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c97e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44260/0 (socket says 172.21.15.139:44260) 2024-04-25T10:01:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f828e413640 1 -- 172.21.15.139:0/1935903581 learned_addr learned my addr 172.21.15.139:0/1935903581 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f828e413640 1 -- 172.21.15.139:0/1935903581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c9680f0 msgr2=0x55618c97dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:20.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f828e413640 1 --2- 172.21.15.139:0/1935903581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c9680f0 0x55618c97dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f828e413640 1 -- 172.21.15.139:0/1935903581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c956230 msgr2=0x55618c9df0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f828e413640 1 --2- 172.21.15.139:0/1935903581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c956230 0x55618c9df0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f828e413640 1 -- 172.21.15.139:0/1935903581 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55618c92d620 con 0x55618c973ad0 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f828e413640 1 --2- 172.21.15.139:0/1935903581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c97e120 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f82800613d0 tx=0x7f8280088e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2cf924b4817464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f8268ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f8268ff9640 1 -- 172.21.15.139:0/1935903581 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280066060 con 0x55618c973ad0 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f8268ff9640 1 -- 172.21.15.139:0/1935903581 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8280061b50 con 0x55618c973ad0 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.728+0000 7f8292b7c740 1 -- 172.21.15.139:0/1935903581 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82800587e0 con 0x55618c973ad0 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8268ff9640 1 -- 172.21.15.139:0/1935903581 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f828006c050 con 0x55618c973ad0 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8292b7c740 10 client.8613 fetch_fsmap learned FSMap version 69 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8292b7c740 1 -- 172.21.15.139:0/1935903581 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55618c92d620 con 0x55618c973ad0 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8268ff9640 1 -- 172.21.15.139:0/1935903581 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82800587e0 con 0x55618c973ad0 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8268ff9640 1 -- 172.21.15.139:0/1935903581 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f828007d070 con 0x55618c973ad0 2024-04-25T10:01:20.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8292b7c740 10 client.8613 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8292b7c740 10 client.8613 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8292b7c740 20 client.8613 populate_metadata read hostname 'smithi139' 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8292b7c740 1 --2- 172.21.15.139:0/1935903581 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55618c993ba0 0x55618c993f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8292b7c740 4 client.8613 mds_command: new command op to 8466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8292b7c740 1 -- 172.21.15.139:0/1935903581 --> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55618c89bf50 con 0x55618c993ba0 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f828dc12640 1 --2- 172.21.15.139:0/1935903581 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55618c993ba0 0x55618c993f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8268ff9640 10 client.8613 ms_handle_connect on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f828dc12640 1 --2- 172.21.15.139:0/1935903581 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55618c993ba0 0x55618c993f80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b14ffeff30b1efd server_cookie=fcd11d1a965b1045 in_seq=0 out_seq=0 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8268ff9640 1 -- 172.21.15.139:0/1935903581 <== mds.0 v2:172.21.15.139:6835/4074693991 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55618c89bf50 con 0x55618c993ba0 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8268ff9640 10 client.8613 handle_command_reply: tid=0 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8292b7c740 1 client.8613 shutdown 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8292b7c740 1 -- 172.21.15.139:0/1935903581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 msgr2=0x55618c97e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.729+0000 7f8292b7c740 1 --2- 172.21.15.139:0/1935903581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c97e120 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f82800613d0 tx=0x7f8280088e70 comp rx=0 tx=0).stop 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f8292b7c740 1 -- 172.21.15.139:0/1935903581 shutdown_connections 2024-04-25T10:01:20.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f8292b7c740 1 --2- 172.21.15.139:0/1935903581 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55618c993ba0 0x55618c993f80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f8292b7c740 1 --2- 172.21.15.139:0/1935903581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55618c973ad0 0x55618c97e120 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f8292b7c740 1 --2- 172.21.15.139:0/1935903581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55618c9680f0 0x55618c97dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f8268ff9640 0 client.8613 ms_handle_reset on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:20.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f8292b7c740 1 --2- 172.21.15.139:0/1935903581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55618c956230 0x55618c9df0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f8292b7c740 1 -- 172.21.15.139:0/1935903581 >> 172.21.15.139:0/1935903581 conn(0x55618c92bf10 msgr2=0x55618c92acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:20.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f8292b7c740 1 -- 172.21.15.139:0/1935903581 shutdown_connections 2024-04-25T10:01:20.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f8292b7c740 1 -- 172.21.15.139:0/1935903581 wait complete. 2024-04-25T10:01:20.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f8292b7c740 20 client.8613 trim_cache size 0 max 16384 2024-04-25T10:01:20.740 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:01:20.740 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T10:01:20.741 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T10:01:20.741 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 8466, 2024-04-25T10:01:20.741 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T10:01:20.741 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T10:01:20.741 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T10:01:20.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 4.5877090840000001, 2024-04-25T10:01:20.741 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 69, 2024-04-25T10:01:20.741 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 105, 2024-04-25T10:01:20.741 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 105, 2024-04-25T10:01:20.741 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 9.6447576189999999 2024-04-25T10:01:20.741 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:01:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f828fe76640 1 -- 172.21.15.139:0/2350157117 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f826c06b4d0 msgr2=0x7f826c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f828fe76640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f826c06b4d0 0x7f826c06d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8284000f60 tx=0x7f8284009040 comp rx=0 tx=0).stop 2024-04-25T10:01:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f828fe76640 1 -- 172.21.15.139:0/2350157117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f828815db40 msgr2=0x7f82881569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f828fe76640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f828815db40 0x7f82881569e0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f828000cef0 tx=0x7f828000a710 comp rx=0 tx=0).stop 2024-04-25T10:01:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f828fe76640 1 -- 172.21.15.139:0/2350157117 shutdown_connections 2024-04-25T10:01:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f828fe76640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f828815db40 0x7f82881569e0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f828fe76640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f826c06b4d0 0x7f826c06d990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f828fe76640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82880b5170 0x7f82880cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f828fe76640 1 --2- 172.21.15.139:0/2350157117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f828806bf50 0x7f82881f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f828fe76640 1 -- 172.21.15.139:0/2350157117 >> 172.21.15.139:0/2350157117 conn(0x7f828806f460 msgr2=0x7f828815c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f828fe76640 1 -- 172.21.15.139:0/2350157117 shutdown_connections 2024-04-25T10:01:20.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.730+0000 7f828fe76640 1 -- 172.21.15.139:0/2350157117 wait complete. 2024-04-25T10:01:20.745 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:01:20.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.837+0000 7fc83a5bd640 1 Processor -- start 2024-04-25T10:01:20.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.837+0000 7fc83a5bd640 1 -- start start 2024-04-25T10:01:20.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc83a5bd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc8340b37d0 0x7fc83415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc83a5bd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc83406da20 0x7fc83406ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc83a5bd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8340b3e10 con 0x7fc834158260 2024-04-25T10:01:20.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc83a5bd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8341615c0 con 0x7fc83406da20 2024-04-25T10:01:20.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc83a5bd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8341618a0 con 0x7fc8340b37d0 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc838b5a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc83406da20 0x7fc83406ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc838b5a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc83406da20 0x7fc83406ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44264/0 (socket says 172.21.15.139:44264) 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc838b5a640 1 -- 172.21.15.139:0/2709805519 learned_addr learned my addr 172.21.15.139:0/2709805519 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc8337fe640 1 --2- 172.21.15.139:0/2709805519 >> v2:172.21.15.150:3300/0 conn(0x7fc8340b37d0 0x7fc83415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc838b5a640 1 -- 172.21.15.139:0/2709805519 >> v2:172.21.15.150:3300/0 conn(0x7fc8340b37d0 msgr2=0x7fc83415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc838b5a640 1 --2- 172.21.15.139:0/2709805519 >> v2:172.21.15.150:3300/0 conn(0x7fc8340b37d0 0x7fc83415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc838b5a640 1 -- 172.21.15.139:0/2709805519 >> v1:172.21.15.150:6789/0 conn(0x7fc834158260 legacy=0x7fc8340b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.838+0000 7fc838b5a640 1 -- 172.21.15.139:0/2709805519 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc834161b80 con 0x7fc83406da20 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.839+0000 7fc838b5a640 1 --2- 172.21.15.139:0/2709805519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc83406da20 0x7fc83406ada0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc834072d10 tx=0x7fc82c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c76de582e5913a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.839+0000 7fc832ffd640 1 -- 172.21.15.139:0/2709805519 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c036070 con 0x7fc83406da20 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.839+0000 7fc832ffd640 1 -- 172.21.15.139:0/2709805519 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82c031600 con 0x7fc83406da20 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.839+0000 7fc832ffd640 1 -- 172.21.15.139:0/2709805519 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0319d0 con 0x7fc83406da20 2024-04-25T10:01:20.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.840+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2709805519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc83406da20 msgr2=0x7fc83406ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.840+0000 7fc83a5bd640 1 --2- 172.21.15.139:0/2709805519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc83406da20 0x7fc83406ada0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc834072d10 tx=0x7fc82c02a730 comp rx=0 tx=0).stop 2024-04-25T10:01:20.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.840+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2709805519 shutdown_connections 2024-04-25T10:01:20.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.840+0000 7fc83a5bd640 1 --2- 172.21.15.139:0/2709805519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc83406da20 0x7fc83406ada0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.840+0000 7fc83a5bd640 1 --2- 172.21.15.139:0/2709805519 >> v2:172.21.15.150:3300/0 conn(0x7fc8340b37d0 0x7fc83415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.840+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2709805519 >> 172.21.15.139:0/2709805519 conn(0x7fc8340b23d0 msgr2=0x7fc8340cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:20.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.840+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2709805519 shutdown_connections 2024-04-25T10:01:20.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.840+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2709805519 wait complete. 2024-04-25T10:01:20.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.841+0000 7fc83a5bd640 1 Processor -- start 2024-04-25T10:01:20.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.841+0000 7fc83a5bd640 1 -- start start 2024-04-25T10:01:20.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc83a5bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc83406da20 0x7fc834153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc83a5bd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8340b37d0 0x7fc834155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc83a5bd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc834158260 0x7fc834156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc83a5bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc834162420 con 0x7fc83406da20 2024-04-25T10:01:20.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc83a5bd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc834162140 con 0x7fc8340b37d0 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc83a5bd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83406b2e0 con 0x7fc834158260 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc833fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc83406da20 0x7fc834153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc833fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc83406da20 0x7fc834153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44276/0 (socket says 172.21.15.139:44276) 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc833fff640 1 -- 172.21.15.139:0/2534436753 learned_addr learned my addr 172.21.15.139:0/2534436753 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc8337fe640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8340b37d0 0x7fc834155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc838b5a640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc834158260 0x7fc834156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc833fff640 1 -- 172.21.15.139:0/2534436753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc834158260 msgr2=0x7fc834156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc833fff640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc834158260 0x7fc834156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc833fff640 1 -- 172.21.15.139:0/2534436753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8340b37d0 msgr2=0x7fc834155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc833fff640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8340b37d0 0x7fc834155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc833fff640 1 -- 172.21.15.139:0/2534436753 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc834154650 con 0x7fc83406da20 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc833fff640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc83406da20 0x7fc834153f90 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fc82400c3c0 tx=0x7fc82400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc830ff9640 1 -- 172.21.15.139:0/2534436753 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8240052d0 con 0x7fc83406da20 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc830ff9640 1 -- 172.21.15.139:0/2534436753 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc824005490 con 0x7fc83406da20 2024-04-25T10:01:20.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc830ff9640 1 -- 172.21.15.139:0/2534436753 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824007260 con 0x7fc83406da20 2024-04-25T10:01:20.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2534436753 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc834154930 con 0x7fc83406da20 2024-04-25T10:01:20.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.842+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2534436753 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc834154ef0 con 0x7fc83406da20 2024-04-25T10:01:20.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.843+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2534436753 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8340b0a40 con 0x7fc83406da20 2024-04-25T10:01:20.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.844+0000 7fc830ff9640 1 -- 172.21.15.139:0/2534436753 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc824007400 con 0x7fc83406da20 2024-04-25T10:01:20.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.844+0000 7fc830ff9640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc82006b500 0x7fc82006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:20.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.844+0000 7fc830ff9640 1 -- 172.21.15.139:0/2534436753 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc8240897c0 con 0x7fc83406da20 2024-04-25T10:01:20.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.844+0000 7fc8337fe640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc82006b500 0x7fc82006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:20.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.845+0000 7fc8337fe640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc82006b500 0x7fc82006d9c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc8341574c0 tx=0x7fc828009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:20.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:20.848+0000 7fc830ff9640 1 -- 172.21.15.139:0/2534436753 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8340b0a40 con 0x7fc83406da20 2024-04-25T10:01:21.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.054+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2534436753 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc8340d0aa0 con 0x7fc83406da20 2024-04-25T10:01:21.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.055+0000 7fc830ff9640 1 -- 172.21.15.139:0/2534436753 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc8340b3e10 con 0x7fc83406da20 2024-04-25T10:01:21.059 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:01:21.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.057+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2534436753 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc82006b500 msgr2=0x7fc82006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:21.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.057+0000 7fc83a5bd640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc82006b500 0x7fc82006d9c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc8341574c0 tx=0x7fc828009190 comp rx=0 tx=0).stop 2024-04-25T10:01:21.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.057+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2534436753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc83406da20 msgr2=0x7fc834153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:21.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.057+0000 7fc83a5bd640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc83406da20 0x7fc834153f90 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fc82400c3c0 tx=0x7fc82400c880 comp rx=0 tx=0).stop 2024-04-25T10:01:21.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.057+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2534436753 shutdown_connections 2024-04-25T10:01:21.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.057+0000 7fc83a5bd640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc82006b500 0x7fc82006d9c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:21.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.057+0000 7fc83a5bd640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc834158260 0x7fc834156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:21.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.057+0000 7fc83a5bd640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8340b37d0 0x7fc834155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:21.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.057+0000 7fc83a5bd640 1 --2- 172.21.15.139:0/2534436753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc83406da20 0x7fc834153f90 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:21.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.057+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2534436753 >> 172.21.15.139:0/2534436753 conn(0x7fc8340b23d0 msgr2=0x7fc8340b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:21.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.057+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2534436753 shutdown_connections 2024-04-25T10:01:21.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:21.057+0000 7fc83a5bd640 1 -- 172.21.15.139:0/2534436753 wait complete. 2024-04-25T10:01:21.072 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:01:21.072 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:01:21.072 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T10:01:21.072 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:01:21.073 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:21.073 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:01:21.073 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:01:21.073 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:01:21.073 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:01:21.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.073 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T10:01:21.088 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:01:21.088 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:01:21.088 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:01:21.088 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:21.088 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T10:01:21.088 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:21.088 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42217sec preferred_lft 42217sec 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:01:21.090 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:01:21.091 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T10:01:21.091 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:21.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.091 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:01:21.150 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T10:01:21.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.151 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T10:01:21.208 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T10:01:21.208 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:01:21.208 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:01:21.208 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:01:21.208 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:01:21.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.208 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T10:01:21.209 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T10:01:21.209 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:01:21.209 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:01:21.209 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:01:21.209 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T10:01:21.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.292 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T10:01:21.347 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:01:21.347 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:01:21.347 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:01:21.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.347 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T10:01:21.411 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:01:21.411 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:21.415 INFO:tasks.ceph.mds.c.smithi139.stderr:2024-04-25T10:01:21.411+0000 7f1f0fc2b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:01:21.429 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:01:21.430 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:01:21.430 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:21.485 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:01:21.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.485 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:01:21.566 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:01:21.566 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:01:21.573 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:01:21.573 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:01:21.573 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T10:01:21.573 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T10:01:21.573 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T10:01:21.573 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:01:21.574 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:01:21.574 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:01:21.574 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:01:21.574 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:01:21.576 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:01:21.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.576 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:21.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.641 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:01:21.702 INFO:teuthology.orchestra.run.smithi139.stdout:client8622 2024-04-25T10:01:21.702 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:21.702 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:21.757 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:21.757 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:21.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.812 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:01:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:6+0 records in 2024-04-25T10:01:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:6+0 records out 2024-04-25T10:01:21.909 INFO:teuthology.orchestra.run.smithi139.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.037767 s, 167 MB/s 2024-04-25T10:01:21.909 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:21.910 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:21.924 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:21.925 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:21.980 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:22.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.112+0000 7fe72d0bd640 1 Processor -- start 2024-04-25T10:01:22.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.112+0000 7fe72d0bd640 1 -- start start 2024-04-25T10:01:22.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72d0bd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe728158260 0x7fe72815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72d0bd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe72806da20 0x7fe7280b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72d0bd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7280b3e10 con 0x7fe7280b3600 2024-04-25T10:01:22.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72d0bd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7280b3a70 con 0x7fe72806da20 2024-04-25T10:01:22.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72d0bd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7281615c0 con 0x7fe728158260 2024-04-25T10:01:22.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72659c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe72806da20 0x7fe7280b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe726d9d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe728158260 0x7fe72815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe726d9d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe728158260 0x7fe72815df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:57320/0 (socket says 172.21.15.139:57320) 2024-04-25T10:01:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72759e640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fe7280b3600 0x7fe72806def0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:48986/0 (socket says 172.21.15.139:48986) 2024-04-25T10:01:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72659c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe72806da20 0x7fe7280b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:40140/0 (socket says 172.21.15.139:40140) 2024-04-25T10:01:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72759e640 1 -- 172.21.15.139:0/4062854651 learned_addr learned my addr 172.21.15.139:0/4062854651 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe725d9b640 1 -- 172.21.15.139:0/4062854651 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014425440 0 0) 0x7fe720001110 con 0x7fe7280b3600 2024-04-25T10:01:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe725d9b640 1 -- 172.21.15.139:0/4062854651 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe710003520 con 0x7fe7280b3600 2024-04-25T10:01:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72659c640 1 -- 172.21.15.139:0/4062854651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe728158260 msgr2=0x7fe72815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72659c640 1 --2- 172.21.15.139:0/4062854651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe728158260 0x7fe72815df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72659c640 1 -- 172.21.15.139:0/4062854651 >> v1:172.21.15.150:6789/0 conn(0x7fe7280b3600 legacy=0x7fe72806def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe72659c640 1 -- 172.21.15.139:0/4062854651 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7281618a0 con 0x7fe72806da20 2024-04-25T10:01:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.113+0000 7fe726d9d640 1 --2- 172.21.15.139:0/4062854651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe728158260 0x7fe72815df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:22.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.114+0000 7fe72659c640 1 --2- 172.21.15.139:0/4062854651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe72806da20 0x7fe7280b30c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0097a0 tx=0x7fe71c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e90bb21459a5851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.114+0000 7fe725d9b640 1 -- 172.21.15.139:0/4062854651 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c037070 con 0x7fe72806da20 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.114+0000 7fe725d9b640 1 -- 172.21.15.139:0/4062854651 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c032440 con 0x7fe72806da20 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.114+0000 7fe725d9b640 1 -- 172.21.15.139:0/4062854651 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0327f0 con 0x7fe72806da20 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.115+0000 7fe72d0bd640 1 -- 172.21.15.139:0/4062854651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe72806da20 msgr2=0x7fe7280b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.115+0000 7fe72d0bd640 1 --2- 172.21.15.139:0/4062854651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe72806da20 0x7fe7280b30c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0097a0 tx=0x7fe71c02a180 comp rx=0 tx=0).stop 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.115+0000 7fe72d0bd640 1 -- 172.21.15.139:0/4062854651 shutdown_connections 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.115+0000 7fe72d0bd640 1 --2- 172.21.15.139:0/4062854651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe72806da20 0x7fe7280b30c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.115+0000 7fe72d0bd640 1 --2- 172.21.15.139:0/4062854651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe728158260 0x7fe72815df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.115+0000 7fe72d0bd640 1 -- 172.21.15.139:0/4062854651 >> 172.21.15.139:0/4062854651 conn(0x7fe7280b23d0 msgr2=0x7fe7280cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.115+0000 7fe72d0bd640 1 -- 172.21.15.139:0/4062854651 shutdown_connections 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.115+0000 7fe72d0bd640 1 -- 172.21.15.139:0/4062854651 wait complete. 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.116+0000 7fe72d0bd640 1 Processor -- start 2024-04-25T10:01:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.116+0000 7fe72d0bd640 1 -- start start 2024-04-25T10:01:22.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe72d0bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe72806da20 0x7fe7281e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe72d0bd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7280b3600 0x7fe72814f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe72d0bd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe728158260 0x7fe7281554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe72d0bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe728162140 con 0x7fe72806da20 2024-04-25T10:01:22.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe72d0bd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe728161e60 con 0x7fe728158260 2024-04-25T10:01:22.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe72d0bd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe728162420 con 0x7fe7280b3600 2024-04-25T10:01:22.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe726d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe72806da20 0x7fe7281e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe726d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe72806da20 0x7fe7281e7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44286/0 (socket says 172.21.15.139:44286) 2024-04-25T10:01:22.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe726d9d640 1 -- 172.21.15.139:0/1144996224 learned_addr learned my addr 172.21.15.139:0/1144996224 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:22.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe72659c640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7280b3600 0x7fe72814f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe72759e640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe728158260 0x7fe7281554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe726d9d640 1 -- 172.21.15.139:0/1144996224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7280b3600 msgr2=0x7fe72814f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe726d9d640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7280b3600 0x7fe72814f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe726d9d640 1 -- 172.21.15.139:0/1144996224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe728158260 msgr2=0x7fe7281554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe726d9d640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe728158260 0x7fe7281554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.117+0000 7fe726d9d640 1 -- 172.21.15.139:0/1144996224 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7281f3cd0 con 0x7fe72806da20 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.118+0000 7fe726d9d640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe72806da20 0x7fe7281e7ec0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe71800efc0 tx=0x7fe71800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.118+0000 7fe70f7fe640 1 -- 172.21.15.139:0/1144996224 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718018020 con 0x7fe72806da20 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.118+0000 7fe70f7fe640 1 -- 172.21.15.139:0/1144996224 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe718014680 con 0x7fe72806da20 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.118+0000 7fe70f7fe640 1 -- 172.21.15.139:0/1144996224 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718014a50 con 0x7fe72806da20 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.118+0000 7fe72d0bd640 1 -- 172.21.15.139:0/1144996224 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7281f3fb0 con 0x7fe72806da20 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.118+0000 7fe72d0bd640 1 -- 172.21.15.139:0/1144996224 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7281f4570 con 0x7fe72806da20 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.119+0000 7fe72d0bd640 1 -- 172.21.15.139:0/1144996224 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe728074520 con 0x7fe72806da20 2024-04-25T10:01:22.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.119+0000 7fe70f7fe640 1 -- 172.21.15.139:0/1144996224 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe718005d10 con 0x7fe72806da20 2024-04-25T10:01:22.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.120+0000 7fe70f7fe640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe71406b4d0 0x7fe71406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.120+0000 7fe70f7fe640 1 -- 172.21.15.139:0/1144996224 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe71808afe0 con 0x7fe72806da20 2024-04-25T10:01:22.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.120+0000 7fe72659c640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe71406b4d0 0x7fe71406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.120+0000 7fe72659c640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe71406b4d0 0x7fe71406d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe71c034000 tx=0x7fe71c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:22.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.124+0000 7fe70f7fe640 1 -- 172.21.15.139:0/1144996224 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe718022030 con 0x7fe72806da20 2024-04-25T10:01:22.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.297+0000 7fe72d0bd640 1 -- 172.21.15.139:0/1144996224 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7280d0aa0 con 0x7fe72806da20 2024-04-25T10:01:22.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.298+0000 7fe70f7fe640 1 -- 172.21.15.139:0/1144996224 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16932 (secure 0 0 0) 0x7fe718010040 con 0x7fe72806da20 2024-04-25T10:01:22.302 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:22.302 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":105,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:01:17.142273+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:13.695577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:14.460042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.300+0000 7fe72d0bd640 1 -- 172.21.15.139:0/1144996224 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe71406b4d0 msgr2=0x7fe71406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.300+0000 7fe72d0bd640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe71406b4d0 0x7fe71406d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe71c034000 tx=0x7fe71c02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.300+0000 7fe72d0bd640 1 -- 172.21.15.139:0/1144996224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe72806da20 msgr2=0x7fe7281e7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.300+0000 7fe72d0bd640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe72806da20 0x7fe7281e7ec0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe71800efc0 tx=0x7fe71800c490 comp rx=0 tx=0).stop 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.300+0000 7fe72d0bd640 1 -- 172.21.15.139:0/1144996224 shutdown_connections 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.300+0000 7fe72d0bd640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe71406b4d0 0x7fe71406d990 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.300+0000 7fe72d0bd640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe728158260 0x7fe7281554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.300+0000 7fe72d0bd640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7280b3600 0x7fe72814f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.300+0000 7fe72d0bd640 1 --2- 172.21.15.139:0/1144996224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe72806da20 0x7fe7281e7ec0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.300+0000 7fe72d0bd640 1 -- 172.21.15.139:0/1144996224 >> 172.21.15.139:0/1144996224 conn(0x7fe7280b23d0 msgr2=0x7fe7280b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.300+0000 7fe72d0bd640 1 -- 172.21.15.139:0/1144996224 shutdown_connections 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.300+0000 7fe72d0bd640 1 -- 172.21.15.139:0/1144996224 wait complete. 2024-04-25T10:01:22.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:13.695577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:14.460042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:22.318 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.413+0000 7fc687df9640 1 Processor -- start 2024-04-25T10:01:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.414+0000 7fc687df9640 1 -- start start 2024-04-25T10:01:22.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.414+0000 7fc687df9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc6800cdd20 0x7fc680071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.414+0000 7fc687df9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc680071f90 0x7fc68015da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.414+0000 7fc687df9640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68015a480 con 0x7fc680150ed0 2024-04-25T10:01:22.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.414+0000 7fc687df9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68015a760 con 0x7fc680071f90 2024-04-25T10:01:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.414+0000 7fc687df9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68015aa40 con 0x7fc6800cdd20 2024-04-25T10:01:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.415+0000 7fc686396640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc680071f90 0x7fc68015da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.415+0000 7fc686396640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc680071f90 0x7fc68015da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44290/0 (socket says 172.21.15.139:44290) 2024-04-25T10:01:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.415+0000 7fc686396640 1 -- 172.21.15.139:0/1619492025 learned_addr learned my addr 172.21.15.139:0/1619492025 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.415+0000 7fc685394640 1 --2- 172.21.15.139:0/1619492025 >> v2:172.21.15.150:3300/0 conn(0x7fc6800cdd20 0x7fc680071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.415+0000 7fc686396640 1 -- 172.21.15.139:0/1619492025 >> v2:172.21.15.150:3300/0 conn(0x7fc6800cdd20 msgr2=0x7fc680071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.415+0000 7fc686396640 1 --2- 172.21.15.139:0/1619492025 >> v2:172.21.15.150:3300/0 conn(0x7fc6800cdd20 0x7fc680071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.415+0000 7fc686396640 1 -- 172.21.15.139:0/1619492025 >> v1:172.21.15.150:6789/0 conn(0x7fc680150ed0 legacy=0x7fc6800b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.415+0000 7fc686396640 1 -- 172.21.15.139:0/1619492025 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68015ad20 con 0x7fc680071f90 2024-04-25T10:01:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.415+0000 7fc685394640 1 --2- 172.21.15.139:0/1619492025 >> v2:172.21.15.150:3300/0 conn(0x7fc6800cdd20 0x7fc680071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:22.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.416+0000 7fc686396640 1 --2- 172.21.15.139:0/1619492025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc680071f90 0x7fc68015da70 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fc67800bb10 tx=0x7fc67802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3816d568e5cf282f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.416+0000 7fc684b93640 1 -- 172.21.15.139:0/1619492025 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678032750 con 0x7fc680071f90 2024-04-25T10:01:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.416+0000 7fc684b93640 1 -- 172.21.15.139:0/1619492025 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678030050 con 0x7fc680071f90 2024-04-25T10:01:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.416+0000 7fc684b93640 1 -- 172.21.15.139:0/1619492025 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678032b10 con 0x7fc680071f90 2024-04-25T10:01:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.416+0000 7fc687df9640 1 -- 172.21.15.139:0/1619492025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc680071f90 msgr2=0x7fc68015da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.416+0000 7fc687df9640 1 --2- 172.21.15.139:0/1619492025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc680071f90 0x7fc68015da70 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fc67800bb10 tx=0x7fc67802c890 comp rx=0 tx=0).stop 2024-04-25T10:01:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.416+0000 7fc687df9640 1 -- 172.21.15.139:0/1619492025 shutdown_connections 2024-04-25T10:01:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.416+0000 7fc687df9640 1 --2- 172.21.15.139:0/1619492025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc680071f90 0x7fc68015da70 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.416+0000 7fc687df9640 1 --2- 172.21.15.139:0/1619492025 >> v2:172.21.15.150:3300/0 conn(0x7fc6800cdd20 0x7fc680071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.416+0000 7fc687df9640 1 -- 172.21.15.139:0/1619492025 >> 172.21.15.139:0/1619492025 conn(0x7fc680067000 msgr2=0x7fc68006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.416+0000 7fc687df9640 1 -- 172.21.15.139:0/1619492025 shutdown_connections 2024-04-25T10:01:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.416+0000 7fc687df9640 1 -- 172.21.15.139:0/1619492025 wait complete. 2024-04-25T10:01:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.418+0000 7fc687df9640 1 Processor -- start 2024-04-25T10:01:22.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.418+0000 7fc687df9640 1 -- start start 2024-04-25T10:01:22.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.418+0000 7fc687df9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc680071f90 0x7fc6800aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.418+0000 7fc687df9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6800cdd20 0x7fc68014c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.418+0000 7fc687df9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc680150ed0 0x7fc6800711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.418+0000 7fc687df9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6801512b0 con 0x7fc680150ed0 2024-04-25T10:01:22.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.418+0000 7fc687df9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6800b3960 con 0x7fc6800cdd20 2024-04-25T10:01:22.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.418+0000 7fc687df9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6800b4910 con 0x7fc680071f90 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.418+0000 7fc686396640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc680150ed0 0x7fc6800711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.418+0000 7fc686396640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc680150ed0 0x7fc6800711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44304/0 (socket says 172.21.15.139:44304) 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.418+0000 7fc686396640 1 -- 172.21.15.139:0/17892777 learned_addr learned my addr 172.21.15.139:0/17892777 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.418+0000 7fc685394640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6800cdd20 0x7fc68014c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc685b95640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc680071f90 0x7fc6800aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc686396640 1 -- 172.21.15.139:0/17892777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc680071f90 msgr2=0x7fc6800aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc686396640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc680071f90 0x7fc6800aae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc686396640 1 -- 172.21.15.139:0/17892777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6800cdd20 msgr2=0x7fc68014c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc686396640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6800cdd20 0x7fc68014c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc686396640 1 -- 172.21.15.139:0/17892777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6800a8890 con 0x7fc680150ed0 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc685b95640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc680071f90 0x7fc6800aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc686396640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc680150ed0 0x7fc6800711c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc67802c8d0 tx=0x7fc6780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc6767fc640 1 -- 172.21.15.139:0/17892777 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678034b30 con 0x7fc680150ed0 2024-04-25T10:01:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc6767fc640 1 -- 172.21.15.139:0/17892777 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678034cf0 con 0x7fc680150ed0 2024-04-25T10:01:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc6767fc640 1 -- 172.21.15.139:0/17892777 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678048650 con 0x7fc680150ed0 2024-04-25T10:01:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc687df9640 1 -- 172.21.15.139:0/17892777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6800a8bd0 con 0x7fc680150ed0 2024-04-25T10:01:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.419+0000 7fc687df9640 1 -- 172.21.15.139:0/17892777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6800a9410 con 0x7fc680150ed0 2024-04-25T10:01:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.420+0000 7fc687df9640 1 -- 172.21.15.139:0/17892777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc680074520 con 0x7fc680150ed0 2024-04-25T10:01:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.420+0000 7fc6767fc640 1 -- 172.21.15.139:0/17892777 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc678046020 con 0x7fc680150ed0 2024-04-25T10:01:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.421+0000 7fc6767fc640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc66406b4d0 0x7fc66406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.421+0000 7fc685b95640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc66406b4d0 0x7fc66406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.421+0000 7fc6767fc640 1 -- 172.21.15.139:0/17892777 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc6780ad8f0 con 0x7fc680150ed0 2024-04-25T10:01:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.421+0000 7fc685b95640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc66406b4d0 0x7fc66406d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009fc0 tx=0x7fc67c009550 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:22.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.425+0000 7fc6767fc640 1 -- 172.21.15.139:0/17892777 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6800a8bd0 con 0x7fc680150ed0 2024-04-25T10:01:22.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.597+0000 7fc687df9640 1 -- 172.21.15.139:0/17892777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc68006f150 con 0x7fc680150ed0 2024-04-25T10:01:22.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.597+0000 7fc6767fc640 1 -- 172.21.15.139:0/17892777 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc67807c140 con 0x7fc680150ed0 2024-04-25T10:01:22.602 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:01:22.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.599+0000 7fc687df9640 1 -- 172.21.15.139:0/17892777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc66406b4d0 msgr2=0x7fc66406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.599+0000 7fc687df9640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc66406b4d0 0x7fc66406d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009fc0 tx=0x7fc67c009550 comp rx=0 tx=0).stop 2024-04-25T10:01:22.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.599+0000 7fc687df9640 1 -- 172.21.15.139:0/17892777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc680150ed0 msgr2=0x7fc6800711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.599+0000 7fc687df9640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc680150ed0 0x7fc6800711c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc67802c8d0 tx=0x7fc6780046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.599+0000 7fc687df9640 1 -- 172.21.15.139:0/17892777 shutdown_connections 2024-04-25T10:01:22.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.599+0000 7fc687df9640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc680150ed0 0x7fc6800711c0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.599+0000 7fc687df9640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc66406b4d0 0x7fc66406d990 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.599+0000 7fc687df9640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6800cdd20 0x7fc68014c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.599+0000 7fc687df9640 1 --2- 172.21.15.139:0/17892777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc680071f90 0x7fc6800aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.599+0000 7fc687df9640 1 -- 172.21.15.139:0/17892777 >> 172.21.15.139:0/17892777 conn(0x7fc680067000 msgr2=0x7fc6800b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:22.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.599+0000 7fc687df9640 1 -- 172.21.15.139:0/17892777 shutdown_connections 2024-04-25T10:01:22.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.599+0000 7fc687df9640 1 -- 172.21.15.139:0/17892777 wait complete. 2024-04-25T10:01:22.614 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:01:22.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.709+0000 7f0cadf7e640 1 Processor -- start 2024-04-25T10:01:22.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.709+0000 7f0cadf7e640 1 -- start start 2024-04-25T10:01:22.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.710+0000 7f0cadf7e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0ca8161fd0 0x7f0ca815d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.710+0000 7f0cadf7e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca8099ca0 con 0x7f0ca80d0a80 2024-04-25T10:01:22.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.710+0000 7f0cadf7e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca8099f80 con 0x7f0ca80cd610 2024-04-25T10:01:22.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.710+0000 7f0cadf7e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca8099a50 con 0x7f0ca8161fd0 2024-04-25T10:01:22.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.710+0000 7f0ca7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0ca8161fd0 0x7f0ca815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.710+0000 7f0ca6ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0ca80d0a80 0x7f0ca81618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38402/0 (socket says 172.21.15.139:38402) 2024-04-25T10:01:22.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.710+0000 7f0ca6ffd640 1 -- 172.21.15.139:0/3129493877 learned_addr learned my addr 172.21.15.139:0/3129493877 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:22.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.710+0000 7f0ca7fff640 1 -- 172.21.15.139:0/3129493877 >> v1:172.21.15.150:6789/0 conn(0x7f0ca80cd610 legacy=0x7f0ca80d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:22.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.710+0000 7f0ca7fff640 1 -- 172.21.15.139:0/3129493877 >> v1:172.21.15.139:6789/0 conn(0x7f0ca80d0a80 legacy=0x7f0ca81618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.710+0000 7f0ca7fff640 1 -- 172.21.15.139:0/3129493877 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca809a260 con 0x7f0ca8161fd0 2024-04-25T10:01:22.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.710+0000 7f0ca67fc640 1 -- 172.21.15.139:0/3129493877 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687662077 0 0) 0x7f0ca8099ca0 con 0x7f0ca80d0a80 2024-04-25T10:01:22.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.711+0000 7f0ca7fff640 1 --2- 172.21.15.139:0/3129493877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ca8161fd0 0x7f0ca815d370 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009d90 tx=0x7f0ca002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aeb22b89abf1eee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:22.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.711+0000 7f0ca67fc640 1 -- 172.21.15.139:0/3129493877 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0007740 con 0x7f0ca8161fd0 2024-04-25T10:01:22.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.711+0000 7f0ca67fc640 1 -- 172.21.15.139:0/3129493877 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca00078e0 con 0x7f0ca8161fd0 2024-04-25T10:01:22.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.711+0000 7f0ca67fc640 1 -- 172.21.15.139:0/3129493877 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0007cb0 con 0x7f0ca8161fd0 2024-04-25T10:01:22.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.712+0000 7f0cadf7e640 1 -- 172.21.15.139:0/3129493877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ca8161fd0 msgr2=0x7f0ca815d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.712+0000 7f0cadf7e640 1 --2- 172.21.15.139:0/3129493877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ca8161fd0 0x7f0ca815d370 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009d90 tx=0x7f0ca002a730 comp rx=0 tx=0).stop 2024-04-25T10:01:22.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.712+0000 7f0cadf7e640 1 -- 172.21.15.139:0/3129493877 shutdown_connections 2024-04-25T10:01:22.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.712+0000 7f0cadf7e640 1 --2- 172.21.15.139:0/3129493877 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ca8161fd0 0x7f0ca815d370 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.712+0000 7f0cadf7e640 1 -- 172.21.15.139:0/3129493877 >> 172.21.15.139:0/3129493877 conn(0x7f0ca80b31f0 msgr2=0x7f0ca80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:22.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.712+0000 7f0cadf7e640 1 -- 172.21.15.139:0/3129493877 shutdown_connections 2024-04-25T10:01:22.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.712+0000 7f0cadf7e640 1 -- 172.21.15.139:0/3129493877 wait complete. 2024-04-25T10:01:22.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.713+0000 7f0cadf7e640 1 Processor -- start 2024-04-25T10:01:22.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.713+0000 7f0cadf7e640 1 -- start start 2024-04-25T10:01:22.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0cadf7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca80cd610 0x7f0ca81e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0cadf7e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca80d0a80 0x7f0ca814aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0cadf7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ca8161fd0 0x7f0ca815ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0cadf7e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca80cd9f0 con 0x7f0ca8161fd0 2024-04-25T10:01:22.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0cadf7e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca809a820 con 0x7f0ca80d0a80 2024-04-25T10:01:22.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0cadf7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca80cdb70 con 0x7f0ca80cd610 2024-04-25T10:01:22.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0ca7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ca8161fd0 0x7f0ca815ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0ca7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ca8161fd0 0x7f0ca815ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44328/0 (socket says 172.21.15.139:44328) 2024-04-25T10:01:22.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0ca7fff640 1 -- 172.21.15.139:0/4184868891 learned_addr learned my addr 172.21.15.139:0/4184868891 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:22.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0ca6ffd640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca80d0a80 0x7f0ca814aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0ca77fe640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca80cd610 0x7f0ca81e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0ca7fff640 1 -- 172.21.15.139:0/4184868891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca80cd610 msgr2=0x7f0ca81e7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0ca7fff640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca80cd610 0x7f0ca81e7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0ca7fff640 1 -- 172.21.15.139:0/4184868891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca80d0a80 msgr2=0x7f0ca814aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0ca7fff640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca80d0a80 0x7f0ca814aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0ca7fff640 1 -- 172.21.15.139:0/4184868891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca815bb50 con 0x7f0ca8161fd0 2024-04-25T10:01:22.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0ca7fff640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ca8161fd0 0x7f0ca815ec60 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009eb0 tx=0x7f0ca0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:22.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0c8ffff640 1 -- 172.21.15.139:0/4184868891 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0040070 con 0x7f0ca8161fd0 2024-04-25T10:01:22.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0c8ffff640 1 -- 172.21.15.139:0/4184868891 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca0031c90 con 0x7f0ca8161fd0 2024-04-25T10:01:22.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.714+0000 7f0c8ffff640 1 -- 172.21.15.139:0/4184868891 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0045680 con 0x7f0ca8161fd0 2024-04-25T10:01:22.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.715+0000 7f0cadf7e640 1 -- 172.21.15.139:0/4184868891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca815aab0 con 0x7f0ca8161fd0 2024-04-25T10:01:22.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.715+0000 7f0cadf7e640 1 -- 172.21.15.139:0/4184868891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca815b070 con 0x7f0ca8161fd0 2024-04-25T10:01:22.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.716+0000 7f0cadf7e640 1 -- 172.21.15.139:0/4184868891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca8074520 con 0x7f0ca8161fd0 2024-04-25T10:01:22.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.716+0000 7f0c8ffff640 1 -- 172.21.15.139:0/4184868891 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ca0039040 con 0x7f0ca8161fd0 2024-04-25T10:01:22.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.716+0000 7f0c8ffff640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0c9406b4d0 0x7f0c9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:22.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.717+0000 7f0c8ffff640 1 -- 172.21.15.139:0/4184868891 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0ca00abd10 con 0x7f0ca8161fd0 2024-04-25T10:01:22.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.717+0000 7f0ca77fe640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0c9406b4d0 0x7f0c9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:22.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.717+0000 7f0ca77fe640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0c9406b4d0 0x7f0c9406d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0c98004ea0 tx=0x7f0c98009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:22.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.721+0000 7f0c8ffff640 1 -- 172.21.15.139:0/4184868891 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ca007ab50 con 0x7f0ca8161fd0 2024-04-25T10:01:22.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.892+0000 7f0cadf7e640 1 -- 172.21.15.139:0/4184868891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ca80ce8a0 con 0x7f0ca8161fd0 2024-04-25T10:01:22.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.892+0000 7f0c8ffff640 1 -- 172.21.15.139:0/4184868891 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ca007a450 con 0x7f0ca8161fd0 2024-04-25T10:01:22.897 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T10:01:22.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.894+0000 7f0cadf7e640 1 -- 172.21.15.139:0/4184868891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0c9406b4d0 msgr2=0x7f0c9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.894+0000 7f0cadf7e640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0c9406b4d0 0x7f0c9406d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0c98004ea0 tx=0x7f0c98009040 comp rx=0 tx=0).stop 2024-04-25T10:01:22.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.894+0000 7f0cadf7e640 1 -- 172.21.15.139:0/4184868891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ca8161fd0 msgr2=0x7f0ca815ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:22.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.894+0000 7f0cadf7e640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ca8161fd0 0x7f0ca815ec60 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009eb0 tx=0x7f0ca0004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:22.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.894+0000 7f0cadf7e640 1 -- 172.21.15.139:0/4184868891 shutdown_connections 2024-04-25T10:01:22.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.894+0000 7f0cadf7e640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ca8161fd0 0x7f0ca815ec60 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.894+0000 7f0cadf7e640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0c9406b4d0 0x7f0c9406d990 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.894+0000 7f0cadf7e640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca80d0a80 0x7f0ca814aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.894+0000 7f0cadf7e640 1 --2- 172.21.15.139:0/4184868891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca80cd610 0x7f0ca81e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:22.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.894+0000 7f0cadf7e640 1 -- 172.21.15.139:0/4184868891 >> 172.21.15.139:0/4184868891 conn(0x7f0ca80b31f0 msgr2=0x7f0ca80cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:22.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.894+0000 7f0cadf7e640 1 -- 172.21.15.139:0/4184868891 shutdown_connections 2024-04-25T10:01:22.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:22.894+0000 7f0cadf7e640 1 -- 172.21.15.139:0/4184868891 wait complete. 2024-04-25T10:01:22.909 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:01:23.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.004+0000 7f81bb353640 1 Processor -- start 2024-04-25T10:01:23.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.005+0000 7f81bb353640 1 -- start start 2024-04-25T10:01:23.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.005+0000 7f81bb353640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f81b40cee50 0x7f81b4168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.005+0000 7f81bb353640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b4161dc0 con 0x7f81b415dbf0 2024-04-25T10:01:23.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.005+0000 7f81bb353640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b41620a0 con 0x7f81b41617e0 2024-04-25T10:01:23.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.005+0000 7f81bb353640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b40d1ee0 con 0x7f81b40cee50 2024-04-25T10:01:23.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.006+0000 7f81b90ef640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f81b415dbf0 0x7f81b40b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38412/0 (socket says 172.21.15.139:38412) 2024-04-25T10:01:23.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.006+0000 7f81b98f0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f81b40cee50 0x7f81b4168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.006+0000 7f81b90ef640 1 -- 172.21.15.139:0/2516374432 learned_addr learned my addr 172.21.15.139:0/2516374432 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:23.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.006+0000 7f81abfff640 1 -- 172.21.15.139:0/2516374432 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939784545 0 0) 0x7f81b4161dc0 con 0x7f81b415dbf0 2024-04-25T10:01:23.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.006+0000 7f81abfff640 1 -- 172.21.15.139:0/2516374432 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a0003520 con 0x7f81b415dbf0 2024-04-25T10:01:23.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.006+0000 7f81b98f0640 1 -- 172.21.15.139:0/2516374432 >> v1:172.21.15.150:6789/0 conn(0x7f81b41617e0 legacy=0x7f81b40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:23.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.006+0000 7f81b98f0640 1 -- 172.21.15.139:0/2516374432 >> v1:172.21.15.139:6789/0 conn(0x7f81b415dbf0 legacy=0x7f81b40b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.006+0000 7f81b98f0640 1 -- 172.21.15.139:0/2516374432 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b4162380 con 0x7f81b40cee50 2024-04-25T10:01:23.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.007+0000 7f81b98f0640 1 --2- 172.21.15.139:0/2516374432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81b40cee50 0x7f81b4168b10 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0097a0 tx=0x7f81ac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ceb6f6c364bf7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:23.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.007+0000 7f81abfff640 1 -- 172.21.15.139:0/2516374432 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac037070 con 0x7f81b40cee50 2024-04-25T10:01:23.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.007+0000 7f81abfff640 1 -- 172.21.15.139:0/2516374432 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac02ae50 con 0x7f81b40cee50 2024-04-25T10:01:23.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.007+0000 7f81abfff640 1 -- 172.21.15.139:0/2516374432 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac032660 con 0x7f81b40cee50 2024-04-25T10:01:23.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.007+0000 7f81bb353640 1 -- 172.21.15.139:0/2516374432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81b40cee50 msgr2=0x7f81b4168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.007+0000 7f81bb353640 1 --2- 172.21.15.139:0/2516374432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81b40cee50 0x7f81b4168b10 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0097a0 tx=0x7f81ac02a140 comp rx=0 tx=0).stop 2024-04-25T10:01:23.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.007+0000 7f81bb353640 1 -- 172.21.15.139:0/2516374432 shutdown_connections 2024-04-25T10:01:23.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.007+0000 7f81bb353640 1 --2- 172.21.15.139:0/2516374432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81b40cee50 0x7f81b4168b10 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.007+0000 7f81bb353640 1 -- 172.21.15.139:0/2516374432 >> 172.21.15.139:0/2516374432 conn(0x7f81b40b5480 msgr2=0x7f81b40b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:23.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.007+0000 7f81bb353640 1 -- 172.21.15.139:0/2516374432 shutdown_connections 2024-04-25T10:01:23.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.007+0000 7f81bb353640 1 -- 172.21.15.139:0/2516374432 wait complete. 2024-04-25T10:01:23.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.008+0000 7f81bb353640 1 Processor -- start 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.009+0000 7f81bb353640 1 -- start start 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.009+0000 7f81bb353640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81b40cee50 0x7f81b415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.009+0000 7f81bb353640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81b415dbf0 0x7f81b40b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.009+0000 7f81bb353640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81b41617e0 0x7f81b40b7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.009+0000 7f81bb353640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b415dfd0 con 0x7f81b415dbf0 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.009+0000 7f81bb353640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b40d3380 con 0x7f81b41617e0 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.009+0000 7f81bb353640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b415a8b0 con 0x7f81b40cee50 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.009+0000 7f81b88ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81b415dbf0 0x7f81b40b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.009+0000 7f81b88ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81b415dbf0 0x7f81b40b6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44348/0 (socket says 172.21.15.139:44348) 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.009+0000 7f81b88ee640 1 -- 172.21.15.139:0/259037853 learned_addr learned my addr 172.21.15.139:0/259037853 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81b88ee640 1 -- 172.21.15.139:0/259037853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81b40cee50 msgr2=0x7f81b415e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81b90ef640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81b40cee50 0x7f81b415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81b98f0640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81b41617e0 0x7f81b40b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81b88ee640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81b40cee50 0x7f81b415e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81b88ee640 1 -- 172.21.15.139:0/259037853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81b41617e0 msgr2=0x7f81b40b7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81b88ee640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81b41617e0 0x7f81b40b7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81b88ee640 1 -- 172.21.15.139:0/259037853 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b415c580 con 0x7f81b415dbf0 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81b90ef640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81b40cee50 0x7f81b415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81b88ee640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81b415dbf0 0x7f81b40b6ad0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f81a400b3f0 tx=0x7f81a400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81a9ffb640 1 -- 172.21.15.139:0/259037853 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4015020 con 0x7f81b415dbf0 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81a9ffb640 1 -- 172.21.15.139:0/259037853 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a4011830 con 0x7f81b415dbf0 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81a9ffb640 1 -- 172.21.15.139:0/259037853 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4011c20 con 0x7f81b415dbf0 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81bb353640 1 -- 172.21.15.139:0/259037853 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b415edb0 con 0x7f81b415dbf0 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.010+0000 7f81bb353640 1 -- 172.21.15.139:0/259037853 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b415f370 con 0x7f81b415dbf0 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.011+0000 7f81bb353640 1 -- 172.21.15.139:0/259037853 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b4074520 con 0x7f81b415dbf0 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.011+0000 7f81a9ffb640 1 -- 172.21.15.139:0/259037853 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81a4018020 con 0x7f81b415dbf0 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.012+0000 7f81a9ffb640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f819806b4d0 0x7f819806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.012+0000 7f81a9ffb640 1 -- 172.21.15.139:0/259037853 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81a408a060 con 0x7f81b415dbf0 2024-04-25T10:01:23.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.012+0000 7f81b90ef640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f819806b4d0 0x7f819806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.012+0000 7f81b90ef640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f819806b4d0 0x7f819806d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f81b0005c00 tx=0x7f81b0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:23.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.016+0000 7f81a9ffb640 1 -- 172.21.15.139:0/259037853 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81a401f030 con 0x7f81b415dbf0 2024-04-25T10:01:23.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.190+0000 7f81bb353640 1 -- 172.21.15.139:0/259037853 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81b40d1680 con 0x7f81b415dbf0 2024-04-25T10:01:23.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.190+0000 7f81a9ffb640 1 -- 172.21.15.139:0/259037853 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7f81a40172c0 con 0x7f81b415dbf0 2024-04-25T10:01:23.195 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T10:01:23.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.192+0000 7f81bb353640 1 -- 172.21.15.139:0/259037853 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f819806b4d0 msgr2=0x7f819806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.192+0000 7f81bb353640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f819806b4d0 0x7f819806d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f81b0005c00 tx=0x7f81b0009040 comp rx=0 tx=0).stop 2024-04-25T10:01:23.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.192+0000 7f81bb353640 1 -- 172.21.15.139:0/259037853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81b415dbf0 msgr2=0x7f81b40b6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.192+0000 7f81bb353640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81b415dbf0 0x7f81b40b6ad0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f81a400b3f0 tx=0x7f81a400b8b0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.193+0000 7f81bb353640 1 -- 172.21.15.139:0/259037853 shutdown_connections 2024-04-25T10:01:23.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.193+0000 7f81bb353640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81b415dbf0 0x7f81b40b6ad0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.193+0000 7f81bb353640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f819806b4d0 0x7f819806d990 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.193+0000 7f81bb353640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81b41617e0 0x7f81b40b7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.193+0000 7f81bb353640 1 --2- 172.21.15.139:0/259037853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81b40cee50 0x7f81b415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.193+0000 7f81bb353640 1 -- 172.21.15.139:0/259037853 >> 172.21.15.139:0/259037853 conn(0x7f81b40b5480 msgr2=0x7f81b40d3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:23.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.193+0000 7f81bb353640 1 -- 172.21.15.139:0/259037853 shutdown_connections 2024-04-25T10:01:23.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.193+0000 7f81bb353640 1 -- 172.21.15.139:0/259037853 wait complete. 2024-04-25T10:01:23.208 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:01:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.301+0000 7f279245a640 1 Processor -- start 2024-04-25T10:01:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.302+0000 7f279245a640 1 -- start start 2024-04-25T10:01:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.302+0000 7f279245a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f278c15dba0 0x7f278c161800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.302+0000 7f279245a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f278c0b93b0 0x7f278c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.302+0000 7f279245a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f278c0ced30 0x7f278c06c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.302+0000 7f279245a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c0cf180 con 0x7f278c0b93b0 2024-04-25T10:01:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.302+0000 7f279245a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c0d21b0 con 0x7f278c0ced30 2024-04-25T10:01:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.302+0000 7f279245a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0d1e60 con 0x7f278c15dba0 2024-04-25T10:01:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.302+0000 7f278b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f278c0b93b0 0x7f278c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.303+0000 7f278b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f278c0b93b0 0x7f278c0ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44356/0 (socket says 172.21.15.139:44356) 2024-04-25T10:01:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.303+0000 7f278b7fe640 1 -- 172.21.15.139:0/2192861015 learned_addr learned my addr 172.21.15.139:0/2192861015 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.303+0000 7f27909f7640 1 --2- 172.21.15.139:0/2192861015 >> v2:172.21.15.150:3300/0 conn(0x7f278c0ced30 0x7f278c06c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.303+0000 7f278bfff640 1 --2- 172.21.15.139:0/2192861015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f278c15dba0 0x7f278c161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.303+0000 7f278b7fe640 1 -- 172.21.15.139:0/2192861015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f278c15dba0 msgr2=0x7f278c161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.303+0000 7f278b7fe640 1 --2- 172.21.15.139:0/2192861015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f278c15dba0 0x7f278c161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.303+0000 7f278b7fe640 1 -- 172.21.15.139:0/2192861015 >> v2:172.21.15.150:3300/0 conn(0x7f278c0ced30 msgr2=0x7f278c06c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.303+0000 7f278b7fe640 1 --2- 172.21.15.139:0/2192861015 >> v2:172.21.15.150:3300/0 conn(0x7f278c0ced30 0x7f278c06c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.303+0000 7f278b7fe640 1 -- 172.21.15.139:0/2192861015 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c162050 con 0x7f278c0b93b0 2024-04-25T10:01:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.303+0000 7f27909f7640 1 --2- 172.21.15.139:0/2192861015 >> v2:172.21.15.150:3300/0 conn(0x7f278c0ced30 0x7f278c06c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.303+0000 7f278b7fe640 1 --2- 172.21.15.139:0/2192861015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f278c0b93b0 0x7f278c0ce7f0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f278c0b6990 tx=0x7f278002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce26facf099fd0c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.304+0000 7f278affd640 1 -- 172.21.15.139:0/2192861015 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780037070 con 0x7f278c0b93b0 2024-04-25T10:01:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.304+0000 7f278affd640 1 -- 172.21.15.139:0/2192861015 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2780032440 con 0x7f278c0b93b0 2024-04-25T10:01:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.304+0000 7f278affd640 1 -- 172.21.15.139:0/2192861015 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780032810 con 0x7f278c0b93b0 2024-04-25T10:01:23.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.304+0000 7f279245a640 1 -- 172.21.15.139:0/2192861015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f278c0b93b0 msgr2=0x7f278c0ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.304+0000 7f279245a640 1 --2- 172.21.15.139:0/2192861015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f278c0b93b0 0x7f278c0ce7f0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f278c0b6990 tx=0x7f278002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.304+0000 7f279245a640 1 -- 172.21.15.139:0/2192861015 shutdown_connections 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.304+0000 7f279245a640 1 --2- 172.21.15.139:0/2192861015 >> v2:172.21.15.150:3300/0 conn(0x7f278c0ced30 0x7f278c06c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.304+0000 7f279245a640 1 --2- 172.21.15.139:0/2192861015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f278c0b93b0 0x7f278c0ce7f0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.304+0000 7f279245a640 1 --2- 172.21.15.139:0/2192861015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f278c15dba0 0x7f278c161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.304+0000 7f279245a640 1 -- 172.21.15.139:0/2192861015 >> 172.21.15.139:0/2192861015 conn(0x7f278c0d04a0 msgr2=0x7f278c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.304+0000 7f279245a640 1 -- 172.21.15.139:0/2192861015 shutdown_connections 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.304+0000 7f279245a640 1 -- 172.21.15.139:0/2192861015 wait complete. 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.305+0000 7f279245a640 1 Processor -- start 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.306+0000 7f279245a640 1 -- start start 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.306+0000 7f279245a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f278c0b93b0 0x7f278c149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.306+0000 7f279245a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f278c0ced30 0x7f278c1f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.306+0000 7f279245a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f278c15dba0 0x7f278c1f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.306+0000 7f279245a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c161d40 con 0x7f278c15dba0 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.306+0000 7f279245a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c162610 con 0x7f278c0ced30 2024-04-25T10:01:23.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.306+0000 7f279245a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0b6bc0 con 0x7f278c0b93b0 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.306+0000 7f27909f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f278c15dba0 0x7f278c1f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.306+0000 7f27909f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f278c15dba0 0x7f278c1f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44360/0 (socket says 172.21.15.139:44360) 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.306+0000 7f27909f7640 1 -- 172.21.15.139:0/1704655237 learned_addr learned my addr 172.21.15.139:0/1704655237 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.306+0000 7f278b7fe640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f278c0ced30 0x7f278c1f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.307+0000 7f278bfff640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f278c0b93b0 0x7f278c149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.307+0000 7f27909f7640 1 -- 172.21.15.139:0/1704655237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f278c0b93b0 msgr2=0x7f278c149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.307+0000 7f27909f7640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f278c0b93b0 0x7f278c149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.307+0000 7f27909f7640 1 -- 172.21.15.139:0/1704655237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f278c0ced30 msgr2=0x7f278c1f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.307+0000 7f27909f7640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f278c0ced30 0x7f278c1f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.307+0000 7f27909f7640 1 -- 172.21.15.139:0/1704655237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c1f7be0 con 0x7f278c15dba0 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.307+0000 7f27909f7640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f278c15dba0 0x7f278c1f4600 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f2784007e10 tx=0x7f278400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.307+0000 7f2788ff9640 1 -- 172.21.15.139:0/1704655237 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278401a070 con 0x7f278c15dba0 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.307+0000 7f2788ff9640 1 -- 172.21.15.139:0/1704655237 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2784003b90 con 0x7f278c15dba0 2024-04-25T10:01:23.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.307+0000 7f2788ff9640 1 -- 172.21.15.139:0/1704655237 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784004030 con 0x7f278c15dba0 2024-04-25T10:01:23.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.307+0000 7f279245a640 1 -- 172.21.15.139:0/1704655237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c1f7ec0 con 0x7f278c15dba0 2024-04-25T10:01:23.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.307+0000 7f279245a640 1 -- 172.21.15.139:0/1704655237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c1f40e0 con 0x7f278c15dba0 2024-04-25T10:01:23.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.308+0000 7f279245a640 1 -- 172.21.15.139:0/1704655237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c0cdad0 con 0x7f278c15dba0 2024-04-25T10:01:23.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.308+0000 7f2788ff9640 1 -- 172.21.15.139:0/1704655237 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f278401d070 con 0x7f278c15dba0 2024-04-25T10:01:23.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.309+0000 7f2788ff9640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f277806b4d0 0x7f277806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.309+0000 7f2788ff9640 1 -- 172.21.15.139:0/1704655237 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f278408a550 con 0x7f278c15dba0 2024-04-25T10:01:23.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.309+0000 7f278bfff640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f277806b4d0 0x7f277806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.309+0000 7f278bfff640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f277806b4d0 0x7f277806d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f277c0079a0 tx=0x7f277c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:23.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.313+0000 7f2788ff9640 1 -- 172.21.15.139:0/1704655237 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2784010070 con 0x7f278c15dba0 2024-04-25T10:01:23.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.496+0000 7f279245a640 1 -- 172.21.15.139:0/1704655237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f278c0c6960 con 0x7f278c15dba0 2024-04-25T10:01:23.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.496+0000 7f2788ff9640 1 -- 172.21.15.139:0/1704655237 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2784058c80 con 0x7f278c15dba0 2024-04-25T10:01:23.500 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T10:01:23.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.498+0000 7f279245a640 1 -- 172.21.15.139:0/1704655237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f277806b4d0 msgr2=0x7f277806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.498+0000 7f279245a640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f277806b4d0 0x7f277806d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f277c0079a0 tx=0x7f277c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:23.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.498+0000 7f279245a640 1 -- 172.21.15.139:0/1704655237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f278c15dba0 msgr2=0x7f278c1f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.498+0000 7f279245a640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f278c15dba0 0x7f278c1f4600 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f2784007e10 tx=0x7f278400d4a0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.498+0000 7f279245a640 1 -- 172.21.15.139:0/1704655237 shutdown_connections 2024-04-25T10:01:23.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.498+0000 7f279245a640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f278c15dba0 0x7f278c1f4600 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.498+0000 7f279245a640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f277806b4d0 0x7f277806d990 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.498+0000 7f279245a640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f278c0ced30 0x7f278c1f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.498+0000 7f279245a640 1 --2- 172.21.15.139:0/1704655237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f278c0b93b0 0x7f278c149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.498+0000 7f279245a640 1 -- 172.21.15.139:0/1704655237 >> 172.21.15.139:0/1704655237 conn(0x7f278c0d04a0 msgr2=0x7f278c0d2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:23.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.498+0000 7f279245a640 1 -- 172.21.15.139:0/1704655237 shutdown_connections 2024-04-25T10:01:23.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.498+0000 7f279245a640 1 -- 172.21.15.139:0/1704655237 wait complete. 2024-04-25T10:01:23.513 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:01:23.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:23.514 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:23.543 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:01:23.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:23.543 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:01:23.598 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:23.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.732+0000 7efd7a122640 1 Processor -- start 2024-04-25T10:01:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.732+0000 7efd7a122640 1 -- start start 2024-04-25T10:01:23.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.733+0000 7efd7a122640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efd7415ec30 0x7efd740aec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.733+0000 7efd7a122640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd740af580 con 0x7efd74147190 2024-04-25T10:01:23.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.733+0000 7efd7a122640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd74147570 con 0x7efd740af180 2024-04-25T10:01:23.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.733+0000 7efd7a122640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd740b24b0 con 0x7efd7415ec30 2024-04-25T10:01:23.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.733+0000 7efd737fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7efd74147190 0x7efd740b0fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38414/0 (socket says 172.21.15.139:38414) 2024-04-25T10:01:23.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.733+0000 7efd737fe640 1 -- 172.21.15.139:0/1651640854 learned_addr learned my addr 172.21.15.139:0/1651640854 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:23.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.733+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224529407 0 0) 0x7efd740af580 con 0x7efd74147190 2024-04-25T10:01:23.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.733+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd5c003520 con 0x7efd74147190 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.733+0000 7efd72ffd640 1 --2- 172.21.15.139:0/1651640854 >> v2:172.21.15.150:3300/0 conn(0x7efd7415ec30 0x7efd740aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2253157369 0 0) 0x7efd5c003520 con 0x7efd74147190 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 >> v2:172.21.15.150:3300/0 conn(0x7efd7415ec30 msgr2=0x7efd740aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 --2- 172.21.15.139:0/1651640854 >> v2:172.21.15.150:3300/0 conn(0x7efd7415ec30 0x7efd740aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 >> v1:172.21.15.150:6789/0 conn(0x7efd740af180 legacy=0x7efd7406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd740b1930 con 0x7efd74147190 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7efd64001670 con 0x7efd74147190 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 >> v1:172.21.15.139:6789/0 conn(0x7efd74147190 legacy=0x7efd740b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 --2- 172.21.15.139:0/1651640854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd5c0039b0 0x7efd5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 --2- 172.21.15.139:0/1651640854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd5c004800 0x7efd5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 --2- 172.21.15.139:0/1651640854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd5c0051c0 0x7efd5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c00c030 con 0x7efd5c004800 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c00c1e0 con 0x7efd5c0039b0 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c00c3e0 con 0x7efd5c0051c0 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd72ffd640 1 --2- 172.21.15.139:0/1651640854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd5c004800 0x7efd5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd737fe640 1 --2- 172.21.15.139:0/1651640854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd5c0039b0 0x7efd5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd73fff640 1 --2- 172.21.15.139:0/1651640854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd5c0051c0 0x7efd5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd72ffd640 1 -- 172.21.15.139:0/1651640854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd5c0051c0 msgr2=0x7efd5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd72ffd640 1 --2- 172.21.15.139:0/1651640854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd5c0051c0 0x7efd5c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd72ffd640 1 -- 172.21.15.139:0/1651640854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd5c0039b0 msgr2=0x7efd5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd72ffd640 1 --2- 172.21.15.139:0/1651640854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd5c0039b0 0x7efd5c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd73fff640 1 -- 172.21.15.139:0/1651640854 reap_dead start 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.734+0000 7efd72ffd640 1 -- 172.21.15.139:0/1651640854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c003520 con 0x7efd5c004800 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.735+0000 7efd72ffd640 1 --2- 172.21.15.139:0/1651640854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd5c004800 0x7efd5c004c80 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7efd68009ac0 tx=0x7efd5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11c3f70a6a3aae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.735+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68036070 con 0x7efd5c004800 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.735+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd68031400 con 0x7efd5c004800 2024-04-25T10:01:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.735+0000 7efd727fc640 1 -- 172.21.15.139:0/1651640854 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68009690 con 0x7efd5c004800 2024-04-25T10:01:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.735+0000 7efd7a122640 1 -- 172.21.15.139:0/1651640854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd5c004800 msgr2=0x7efd5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.735+0000 7efd7a122640 1 --2- 172.21.15.139:0/1651640854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd5c004800 0x7efd5c004c80 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7efd68009ac0 tx=0x7efd5c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.735+0000 7efd7a122640 1 -- 172.21.15.139:0/1651640854 shutdown_connections 2024-04-25T10:01:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.735+0000 7efd7a122640 1 --2- 172.21.15.139:0/1651640854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd5c004800 0x7efd5c004c80 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.735+0000 7efd7a122640 1 -- 172.21.15.139:0/1651640854 >> 172.21.15.139:0/1651640854 conn(0x7efd74066f20 msgr2=0x7efd7406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.735+0000 7efd7a122640 1 -- 172.21.15.139:0/1651640854 shutdown_connections 2024-04-25T10:01:23.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.735+0000 7efd7a122640 1 -- 172.21.15.139:0/1651640854 wait complete. 2024-04-25T10:01:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.737+0000 7efd7a122640 1 Processor -- start 2024-04-25T10:01:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.737+0000 7efd7a122640 1 -- start start 2024-04-25T10:01:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.737+0000 7efd7a122640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd5c004800 0x7efd7415cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.737+0000 7efd7a122640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd740ce990 0x7efd740cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.737+0000 7efd7a122640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd7406de80 0x7efd7406e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.737+0000 7efd7a122640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740b21d0 con 0x7efd740ce990 2024-04-25T10:01:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.737+0000 7efd7a122640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740b1ef0 con 0x7efd7406de80 2024-04-25T10:01:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.737+0000 7efd7a122640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd740aec40 con 0x7efd5c004800 2024-04-25T10:01:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd72ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd740ce990 0x7efd740cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd72ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd740ce990 0x7efd740cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44384/0 (socket says 172.21.15.139:44384) 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd72ffd640 1 -- 172.21.15.139:0/403191754 learned_addr learned my addr 172.21.15.139:0/403191754 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd737fe640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd5c004800 0x7efd7415cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd73fff640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd7406de80 0x7efd7406e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd72ffd640 1 -- 172.21.15.139:0/403191754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd5c004800 msgr2=0x7efd7415cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd72ffd640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd5c004800 0x7efd7415cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd72ffd640 1 -- 172.21.15.139:0/403191754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd7406de80 msgr2=0x7efd7406e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd72ffd640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd7406de80 0x7efd7406e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd72ffd640 1 -- 172.21.15.139:0/403191754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd740cef00 con 0x7efd740ce990 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd73fff640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd7406de80 0x7efd7406e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd72ffd640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd740ce990 0x7efd740cc360 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7efd68034040 tx=0x7efd68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd5bfff640 1 -- 172.21.15.139:0/403191754 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6803f020 con 0x7efd740ce990 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd5bfff640 1 -- 172.21.15.139:0/403191754 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd68037040 con 0x7efd740ce990 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd5bfff640 1 -- 172.21.15.139:0/403191754 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68036070 con 0x7efd740ce990 2024-04-25T10:01:23.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd7a122640 1 -- 172.21.15.139:0/403191754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd740cf1e0 con 0x7efd740ce990 2024-04-25T10:01:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.738+0000 7efd7a122640 1 -- 172.21.15.139:0/403191754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7413c420 con 0x7efd740ce990 2024-04-25T10:01:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.739+0000 7efd7a122640 1 -- 172.21.15.139:0/403191754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd74074520 con 0x7efd740ce990 2024-04-25T10:01:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.740+0000 7efd5bfff640 1 -- 172.21.15.139:0/403191754 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd68042020 con 0x7efd740ce990 2024-04-25T10:01:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.740+0000 7efd5bfff640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd6006b500 0x7efd6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.740+0000 7efd5bfff640 1 -- 172.21.15.139:0/403191754 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd680aa390 con 0x7efd740ce990 2024-04-25T10:01:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.740+0000 7efd737fe640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd6006b500 0x7efd6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.741+0000 7efd737fe640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd6006b500 0x7efd6006d9c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efd7413e180 tx=0x7efd64000ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:23.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.744+0000 7efd5bfff640 1 -- 172.21.15.139:0/403191754 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd68049030 con 0x7efd740ce990 2024-04-25T10:01:23.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.950+0000 7efd7a122640 1 -- 172.21.15.139:0/403191754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd7406b850 con 0x7efd740ce990 2024-04-25T10:01:23.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.950+0000 7efd5bfff640 1 -- 172.21.15.139:0/403191754 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4952 (secure 0 0 0) 0x7efd68031400 con 0x7efd740ce990 2024-04-25T10:01:23.955 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:23.955 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":69,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:15.497940+0000","modified":"2024-04-25T10:01:17.495730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8466":{"gid":8466,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/4074693991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4074693991},{"type":"v1","addr":"172.21.15.139:6837","nonce":4074693991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:23.956 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 69 2024-04-25T10:01:23.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.952+0000 7efd7a122640 1 -- 172.21.15.139:0/403191754 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd6006b500 msgr2=0x7efd6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.952+0000 7efd7a122640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd6006b500 0x7efd6006d9c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efd7413e180 tx=0x7efd64000ed0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.952+0000 7efd7a122640 1 -- 172.21.15.139:0/403191754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd740ce990 msgr2=0x7efd740cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:23.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.952+0000 7efd7a122640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd740ce990 0x7efd740cc360 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7efd68034040 tx=0x7efd68004730 comp rx=0 tx=0).stop 2024-04-25T10:01:23.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.952+0000 7efd7a122640 1 -- 172.21.15.139:0/403191754 shutdown_connections 2024-04-25T10:01:23.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.952+0000 7efd7a122640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd740ce990 0x7efd740cc360 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.952+0000 7efd7a122640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efd6006b500 0x7efd6006d9c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.952+0000 7efd7a122640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd7406de80 0x7efd7406e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.952+0000 7efd7a122640 1 --2- 172.21.15.139:0/403191754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd5c004800 0x7efd7415cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:23.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.952+0000 7efd7a122640 1 -- 172.21.15.139:0/403191754 >> 172.21.15.139:0/403191754 conn(0x7efd74066f20 msgr2=0x7efd74146350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:23.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.953+0000 7efd7a122640 1 -- 172.21.15.139:0/403191754 shutdown_connections 2024-04-25T10:01:23.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:23.953+0000 7efd7a122640 1 -- 172.21.15.139:0/403191754 wait complete. 2024-04-25T10:01:23.968 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:01:24.092 INFO:teuthology.orchestra.run.smithi139.stdout:{"message":"","return_code":0} 2024-04-25T10:01:24.093 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:01:24.093 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.190+0000 7fbf4d838640 1 Processor -- start 2024-04-25T10:01:24.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.190+0000 7fbf4d838640 1 -- start start 2024-04-25T10:01:24.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.190+0000 7fbf4d838640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbf4815dc60 0x7fbf480b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.190+0000 7fbf4d838640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbf48161850 0x7fbf480ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.190+0000 7fbf4d838640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf480cf250 con 0x7fbf480d0970 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.190+0000 7fbf4d838640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf480ce780 con 0x7fbf4815dc60 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.190+0000 7fbf4d838640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf480d3730 con 0x7fbf48161850 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.191+0000 7fbf46ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbf4815dc60 0x7fbf480b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.191+0000 7fbf477fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbf480d0970 0x7fbf480d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38420/0 (socket says 172.21.15.139:38420) 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.191+0000 7fbf477fe640 1 -- 172.21.15.139:0/2764222658 learned_addr learned my addr 172.21.15.139:0/2764222658 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:24.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.191+0000 7fbf467fc640 1 --2- 172.21.15.139:0/2764222658 >> v2:172.21.15.150:3300/0 conn(0x7fbf48161850 0x7fbf480ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.191+0000 7fbf46ffd640 1 -- 172.21.15.139:0/2764222658 >> v2:172.21.15.150:3300/0 conn(0x7fbf48161850 msgr2=0x7fbf480ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.191+0000 7fbf46ffd640 1 --2- 172.21.15.139:0/2764222658 >> v2:172.21.15.150:3300/0 conn(0x7fbf48161850 0x7fbf480ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.191+0000 7fbf46ffd640 1 -- 172.21.15.139:0/2764222658 >> v1:172.21.15.139:6789/0 conn(0x7fbf480d0970 legacy=0x7fbf480d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.191+0000 7fbf45ffb640 1 -- 172.21.15.139:0/2764222658 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138968623 0 0) 0x7fbf40001490 con 0x7fbf480d0970 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.191+0000 7fbf46ffd640 1 -- 172.21.15.139:0/2764222658 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf480ce9b0 con 0x7fbf4815dc60 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.192+0000 7fbf46ffd640 1 --2- 172.21.15.139:0/2764222658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf4815dc60 0x7fbf480b6c10 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fbf48068750 tx=0x7fbf3802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb022a56ca61a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.192+0000 7fbf45ffb640 1 -- 172.21.15.139:0/2764222658 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38037070 con 0x7fbf4815dc60 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.192+0000 7fbf45ffb640 1 -- 172.21.15.139:0/2764222658 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf38032440 con 0x7fbf4815dc60 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.192+0000 7fbf45ffb640 1 -- 172.21.15.139:0/2764222658 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38032810 con 0x7fbf4815dc60 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.192+0000 7fbf4d838640 1 -- 172.21.15.139:0/2764222658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf4815dc60 msgr2=0x7fbf480b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.192+0000 7fbf4d838640 1 --2- 172.21.15.139:0/2764222658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf4815dc60 0x7fbf480b6c10 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fbf48068750 tx=0x7fbf3802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.192+0000 7fbf4d838640 1 -- 172.21.15.139:0/2764222658 shutdown_connections 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.192+0000 7fbf4d838640 1 --2- 172.21.15.139:0/2764222658 >> v2:172.21.15.150:3300/0 conn(0x7fbf48161850 0x7fbf480ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.192+0000 7fbf4d838640 1 --2- 172.21.15.139:0/2764222658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf4815dc60 0x7fbf480b6c10 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.192+0000 7fbf4d838640 1 -- 172.21.15.139:0/2764222658 >> 172.21.15.139:0/2764222658 conn(0x7fbf480b5910 msgr2=0x7fbf480b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.192+0000 7fbf4d838640 1 -- 172.21.15.139:0/2764222658 shutdown_connections 2024-04-25T10:01:24.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.192+0000 7fbf4d838640 1 -- 172.21.15.139:0/2764222658 wait complete. 2024-04-25T10:01:24.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.193+0000 7fbf4d838640 1 Processor -- start 2024-04-25T10:01:24.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.194+0000 7fbf4d838640 1 -- start start 2024-04-25T10:01:24.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.194+0000 7fbf4d838640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf480d0970 0x7fbf48146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.194+0000 7fbf4d838640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf4815dc60 0x7fbf480cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.194+0000 7fbf4d838640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf48161850 0x7fbf480d3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.194+0000 7fbf4d838640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf480d1fc0 con 0x7fbf4815dc60 2024-04-25T10:01:24.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.194+0000 7fbf4d838640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf480cef70 con 0x7fbf48161850 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.194+0000 7fbf4d838640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4815e040 con 0x7fbf480d0970 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.194+0000 7fbf467fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf4815dc60 0x7fbf480cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.194+0000 7fbf467fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf4815dc60 0x7fbf480cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44400/0 (socket says 172.21.15.139:44400) 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.194+0000 7fbf467fc640 1 -- 172.21.15.139:0/3351895390 learned_addr learned my addr 172.21.15.139:0/3351895390 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf46ffd640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf480d0970 0x7fbf48146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf477fe640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf48161850 0x7fbf480d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf467fc640 1 -- 172.21.15.139:0/3351895390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf480d0970 msgr2=0x7fbf48146240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf467fc640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf480d0970 0x7fbf48146240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf467fc640 1 -- 172.21.15.139:0/3351895390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf48161850 msgr2=0x7fbf480d3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf467fc640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf48161850 0x7fbf480d3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf467fc640 1 -- 172.21.15.139:0/3351895390 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf480ca030 con 0x7fbf4815dc60 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf467fc640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf4815dc60 0x7fbf480cb4c0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00e930 tx=0x7fbf3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf2ffff640 1 -- 172.21.15.139:0/3351895390 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c017070 con 0x7fbf4815dc60 2024-04-25T10:01:24.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf2ffff640 1 -- 172.21.15.139:0/3351895390 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3c00ca00 con 0x7fbf4815dc60 2024-04-25T10:01:24.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf2ffff640 1 -- 172.21.15.139:0/3351895390 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00cdf0 con 0x7fbf4815dc60 2024-04-25T10:01:24.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf4d838640 1 -- 172.21.15.139:0/3351895390 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf48146780 con 0x7fbf4815dc60 2024-04-25T10:01:24.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.195+0000 7fbf4d838640 1 -- 172.21.15.139:0/3351895390 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf480c9270 con 0x7fbf4815dc60 2024-04-25T10:01:24.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.196+0000 7fbf4d838640 1 -- 172.21.15.139:0/3351895390 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf48074520 con 0x7fbf4815dc60 2024-04-25T10:01:24.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.197+0000 7fbf2ffff640 1 -- 172.21.15.139:0/3351895390 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf3c01a070 con 0x7fbf4815dc60 2024-04-25T10:01:24.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.197+0000 7fbf2ffff640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf3406b4d0 0x7fbf3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.197+0000 7fbf2ffff640 1 -- 172.21.15.139:0/3351895390 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbf3c08a4f0 con 0x7fbf4815dc60 2024-04-25T10:01:24.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.197+0000 7fbf46ffd640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf3406b4d0 0x7fbf3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.197+0000 7fbf46ffd640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf3406b4d0 0x7fbf3406d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbf38034000 tx=0x7fbf3802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.201+0000 7fbf2ffff640 1 -- 172.21.15.139:0/3351895390 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf3c059210 con 0x7fbf4815dc60 2024-04-25T10:01:24.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.408+0000 7fbf4d838640 1 -- 172.21.15.139:0/3351895390 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf480d16f0 con 0x7fbf4815dc60 2024-04-25T10:01:24.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.408+0000 7fbf2ffff640 1 -- 172.21.15.139:0/3351895390 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fbf3c058b10 con 0x7fbf4815dc60 2024-04-25T10:01:24.413 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:24.413 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":69,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:15.497940+0000","modified":"2024-04-25T10:01:17.495730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8466":{"gid":8466,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/4074693991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4074693991},{"type":"v1","addr":"172.21.15.139:6837","nonce":4074693991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:24.413 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 69 2024-04-25T10:01:24.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.410+0000 7fbf4d838640 1 -- 172.21.15.139:0/3351895390 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf3406b4d0 msgr2=0x7fbf3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.410+0000 7fbf4d838640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf3406b4d0 0x7fbf3406d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbf38034000 tx=0x7fbf3802e000 comp rx=0 tx=0).stop 2024-04-25T10:01:24.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.410+0000 7fbf4d838640 1 -- 172.21.15.139:0/3351895390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf4815dc60 msgr2=0x7fbf480cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.410+0000 7fbf4d838640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf4815dc60 0x7fbf480cb4c0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00e930 tx=0x7fbf3c00edf0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.410+0000 7fbf4d838640 1 -- 172.21.15.139:0/3351895390 shutdown_connections 2024-04-25T10:01:24.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.410+0000 7fbf4d838640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf4815dc60 0x7fbf480cb4c0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.410+0000 7fbf4d838640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf3406b4d0 0x7fbf3406d990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.410+0000 7fbf4d838640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf48161850 0x7fbf480d3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.410+0000 7fbf4d838640 1 --2- 172.21.15.139:0/3351895390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf480d0970 0x7fbf48146240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.410+0000 7fbf4d838640 1 -- 172.21.15.139:0/3351895390 >> 172.21.15.139:0/3351895390 conn(0x7fbf480b5910 msgr2=0x7fbf48152bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.411+0000 7fbf4d838640 1 -- 172.21.15.139:0/3351895390 shutdown_connections 2024-04-25T10:01:24.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.411+0000 7fbf4d838640 1 -- 172.21.15.139:0/3351895390 wait complete. 2024-04-25T10:01:24.426 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-25T10:01:24.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.464+0000 7f5fdce65f40 1 Processor -- start 2024-04-25T10:01:24.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.464+0000 7f5fdce65f40 1 -- start start 2024-04-25T10:01:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fdce65f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 0x55ec196fb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fdce65f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec19704b20 0x55ec197023f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fdce65f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ec19702930 0x55ec19660a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fdce65f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec195d9df0 con 0x7f5fc80022a0 2024-04-25T10:01:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fdce65f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec195ddd50 con 0x55ec19704b20 2024-04-25T10:01:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fdce65f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec19707f20 con 0x55ec19702930 2024-04-25T10:01:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fda966640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 0x55ec196fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fda966640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 0x55ec196fb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44408/0 (socket says 172.21.15.139:44408) 2024-04-25T10:01:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fda966640 1 -- 172.21.15.139:0/3191104372 learned_addr learned my addr 172.21.15.139:0/3191104372 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fdb167640 1 --2- 172.21.15.139:0/3191104372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ec19702930 0x55ec19660a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fd2165640 1 --2- 172.21.15.139:0/3191104372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec19704b20 0x55ec197023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fda966640 1 -- 172.21.15.139:0/3191104372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ec19702930 msgr2=0x55ec19660a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fda966640 1 --2- 172.21.15.139:0/3191104372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ec19702930 0x55ec19660a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fda966640 1 -- 172.21.15.139:0/3191104372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec19704b20 msgr2=0x55ec197023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fda966640 1 --2- 172.21.15.139:0/3191104372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec19704b20 0x55ec197023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fda966640 1 -- 172.21.15.139:0/3191104372 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec19708a50 con 0x7f5fc80022a0 2024-04-25T10:01:24.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fda966640 1 --2- 172.21.15.139:0/3191104372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 0x55ec196fb180 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc057040 tx=0x7f5fcc050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458ba74e10fc947b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fd9964640 1 -- 172.21.15.139:0/3191104372 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc061450 con 0x7f5fc80022a0 2024-04-25T10:01:24.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fd9964640 1 -- 172.21.15.139:0/3191104372 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc066070 con 0x7f5fc80022a0 2024-04-25T10:01:24.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.465+0000 7f5fd9964640 1 -- 172.21.15.139:0/3191104372 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc061810 con 0x7f5fc80022a0 2024-04-25T10:01:24.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.466+0000 7f5fdce65f40 1 -- 172.21.15.139:0/3191104372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 msgr2=0x55ec196fb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.466+0000 7f5fdce65f40 1 --2- 172.21.15.139:0/3191104372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 0x55ec196fb180 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc057040 tx=0x7f5fcc050000 comp rx=0 tx=0).stop 2024-04-25T10:01:24.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.466+0000 7f5fdce65f40 1 -- 172.21.15.139:0/3191104372 shutdown_connections 2024-04-25T10:01:24.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.466+0000 7f5fdce65f40 1 --2- 172.21.15.139:0/3191104372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ec19702930 0x55ec19660a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.466+0000 7f5fdce65f40 1 --2- 172.21.15.139:0/3191104372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec19704b20 0x55ec197023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.466+0000 7f5fdce65f40 1 --2- 172.21.15.139:0/3191104372 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 0x55ec196fb180 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.466+0000 7f5fdce65f40 1 -- 172.21.15.139:0/3191104372 >> 172.21.15.139:0/3191104372 conn(0x55ec19632be0 msgr2=0x55ec19634520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.466+0000 7f5fdce65f40 1 -- 172.21.15.139:0/3191104372 shutdown_connections 2024-04-25T10:01:24.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.466+0000 7f5fdce65f40 1 -- 172.21.15.139:0/3191104372 wait complete. 2024-04-25T10:01:24.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.466+0000 7f5fdce65f40 1 Processor -- start 2024-04-25T10:01:24.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.467+0000 7f5fdce65f40 1 -- start start 2024-04-25T10:01:24.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.467+0000 7f5fdce65f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec19702930 0x55ec19701c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.467+0000 7f5fdce65f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ec19704b20 0x55ec196efc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.467+0000 7f5fdce65f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 0x55ec19660980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.467+0000 7f5fdce65f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec19660580 con 0x7f5fc80022a0 2024-04-25T10:01:24.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.467+0000 7f5fdce65f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec196fdb00 con 0x55ec19702930 2024-04-25T10:01:24.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.467+0000 7f5fdce65f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec196feac0 con 0x55ec19704b20 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.467+0000 7f5fdb167640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 0x55ec19660980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.467+0000 7f5fdb167640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 0x55ec19660980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44416/0 (socket says 172.21.15.139:44416) 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.467+0000 7f5fdb167640 1 -- 172.21.15.139:0/301974602 learned_addr learned my addr 172.21.15.139:0/301974602 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fda966640 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec19702930 0x55ec19701c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fd2165640 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ec19704b20 0x55ec196efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fdb167640 1 -- 172.21.15.139:0/301974602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ec19704b20 msgr2=0x55ec196efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fdb167640 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ec19704b20 0x55ec196efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fdb167640 1 -- 172.21.15.139:0/301974602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec19702930 msgr2=0x55ec19701c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fdb167640 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec19702930 0x55ec19701c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fdb167640 1 -- 172.21.15.139:0/301974602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec196f01a0 con 0x7f5fc80022a0 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fdb167640 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 0x55ec19660980 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x55ec1961e6f0 tx=0x7f5fd4002650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fd8962640 1 -- 172.21.15.139:0/301974602 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4018020 con 0x7f5fc80022a0 2024-04-25T10:01:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fd8962640 1 -- 172.21.15.139:0/301974602 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd4014480 con 0x7f5fc80022a0 2024-04-25T10:01:24.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fd8962640 1 -- 172.21.15.139:0/301974602 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4014850 con 0x7f5fc80022a0 2024-04-25T10:01:24.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fdce65f40 1 -- 172.21.15.139:0/301974602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ec196f8640 con 0x7f5fc80022a0 2024-04-25T10:01:24.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.468+0000 7f5fdce65f40 1 -- 172.21.15.139:0/301974602 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec196f8c00 con 0x7f5fc80022a0 2024-04-25T10:01:24.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.469+0000 7f5fd8962640 1 -- 172.21.15.139:0/301974602 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fd401b020 con 0x7f5fc80022a0 2024-04-25T10:01:24.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.469+0000 7f5fd8962640 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5fbc069eb0 0x7f5fbc06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.470+0000 7f5fd8962640 1 -- 172.21.15.139:0/301974602 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5fd40897a0 con 0x7f5fc80022a0 2024-04-25T10:01:24.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.470+0000 7f5fda966640 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5fbc069eb0 0x7f5fbc06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.470+0000 7f5fdce65f40 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55ec196fedf0 0x55ec196ff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.470+0000 7f5fdce65f40 1 -- 172.21.15.139:0/301974602 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) v9 -- 0x55ec196ff710 con 0x55ec196fedf0 2024-04-25T10:01:24.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.470+0000 7f5fd2165640 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55ec196fedf0 0x55ec196ff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.470+0000 7f5fda966640 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5fbc069eb0 0x7f5fbc06c370 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc057040 tx=0x7f5fcc052000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.470+0000 7f5fd2165640 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55ec196fedf0 0x55ec196ff1d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.470+0000 7f5fd2165640 1 -- 172.21.15.139:0/301974602 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7f5fc400e040 con 0x55ec196fedf0 2024-04-25T10:01:24.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.471+0000 7f5fdce65f40 1 -- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55ec196fedf0 msgr2=0x55ec196ff1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.471+0000 7f5fdce65f40 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55ec196fedf0 0x55ec196ff1d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.471+0000 7f5fdce65f40 1 -- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5fbc069eb0 msgr2=0x7f5fbc06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.471+0000 7f5fdce65f40 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f5fbc069eb0 0x7f5fbc06c370 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc057040 tx=0x7f5fcc052000 comp rx=0 tx=0).stop 2024-04-25T10:01:24.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.471+0000 7f5fdce65f40 1 -- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 msgr2=0x55ec19660980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.471+0000 7f5fdce65f40 1 --2- 172.21.15.139:0/301974602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fc80022a0 0x55ec19660980 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x55ec1961e6f0 tx=0x7f5fd4002650 comp rx=0 tx=0).stop 2024-04-25T10:01:24.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.471+0000 7f5fdb167640 1 -- 172.21.15.139:0/301974602 reap_dead start 2024-04-25T10:01:24.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.471+0000 7f5fdce65f40 1 -- 172.21.15.139:0/301974602 shutdown_connections 2024-04-25T10:01:24.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.471+0000 7f5fdce65f40 1 -- 172.21.15.139:0/301974602 >> 172.21.15.139:0/301974602 conn(0x55ec19632be0 msgr2=0x55ec19634520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.471+0000 7f5fdce65f40 1 -- 172.21.15.139:0/301974602 shutdown_connections 2024-04-25T10:01:24.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.471+0000 7f5fdce65f40 1 -- 172.21.15.139:0/301974602 wait complete. 2024-04-25T10:01:24.484 INFO:teuthology.orchestra.run.smithi139.stdout:����� 2024-04-25T10:01:24.484 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:01:24.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.615+0000 7f8508789640 1 Processor -- start 2024-04-25T10:01:24.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.615+0000 7f8508789640 1 -- start start 2024-04-25T10:01:24.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.616+0000 7f8508789640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f850015efb0 0x7f85000d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.616+0000 7f8508789640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85000cfe70 con 0x7f85000cf900 2024-04-25T10:01:24.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.616+0000 7f8508789640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85000d0150 con 0x7f8500099a50 2024-04-25T10:01:24.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.616+0000 7f8508789640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85000d0a80 con 0x7f850015efb0 2024-04-25T10:01:24.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.616+0000 7f8506525640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f850015efb0 0x7f85000d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.616+0000 7f8505d24640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f85000cf900 0x7f850006c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38434/0 (socket says 172.21.15.139:38434) 2024-04-25T10:01:24.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.616+0000 7f8505d24640 1 -- 172.21.15.139:0/2348807757 learned_addr learned my addr 172.21.15.139:0/2348807757 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:24.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.616+0000 7f8506525640 1 -- 172.21.15.139:0/2348807757 >> v1:172.21.15.150:6789/0 conn(0x7f8500099a50 legacy=0x7f85000cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.616+0000 7f8506525640 1 -- 172.21.15.139:0/2348807757 >> v1:172.21.15.139:6789/0 conn(0x7f85000cf900 legacy=0x7f850006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.616+0000 7f8505523640 1 -- 172.21.15.139:0/2348807757 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475872633 0 0) 0x7f85000cfe70 con 0x7f85000cf900 2024-04-25T10:01:24.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.616+0000 7f8506525640 1 -- 172.21.15.139:0/2348807757 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85000d0430 con 0x7f850015efb0 2024-04-25T10:01:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.617+0000 7f8506525640 1 --2- 172.21.15.139:0/2348807757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f850015efb0 0x7f85000d0f10 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0097a0 tx=0x7f84fc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25abbfea46dae194 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.617+0000 7f8505523640 1 -- 172.21.15.139:0/2348807757 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc037070 con 0x7f850015efb0 2024-04-25T10:01:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.617+0000 7f8505523640 1 -- 172.21.15.139:0/2348807757 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc02ae50 con 0x7f850015efb0 2024-04-25T10:01:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.617+0000 7f8505523640 1 -- 172.21.15.139:0/2348807757 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc032660 con 0x7f850015efb0 2024-04-25T10:01:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.618+0000 7f8508789640 1 -- 172.21.15.139:0/2348807757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f850015efb0 msgr2=0x7f85000d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.618+0000 7f8508789640 1 --2- 172.21.15.139:0/2348807757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f850015efb0 0x7f85000d0f10 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0097a0 tx=0x7f84fc02a140 comp rx=0 tx=0).stop 2024-04-25T10:01:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.618+0000 7f8508789640 1 -- 172.21.15.139:0/2348807757 shutdown_connections 2024-04-25T10:01:24.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.618+0000 7f8508789640 1 --2- 172.21.15.139:0/2348807757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f850015efb0 0x7f85000d0f10 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.618+0000 7f8508789640 1 -- 172.21.15.139:0/2348807757 >> 172.21.15.139:0/2348807757 conn(0x7f85000d1790 msgr2=0x7f85000d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.618+0000 7f8508789640 1 -- 172.21.15.139:0/2348807757 shutdown_connections 2024-04-25T10:01:24.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.618+0000 7f8508789640 1 -- 172.21.15.139:0/2348807757 wait complete. 2024-04-25T10:01:24.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.619+0000 7f8508789640 1 Processor -- start 2024-04-25T10:01:24.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.619+0000 7f8508789640 1 -- start start 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.619+0000 7f8508789640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8500099a50 0x7f8500156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8508789640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85000cf900 0x7f85000c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8508789640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f850015efb0 0x7f8500158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8508789640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8500099e80 con 0x7f850015efb0 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8508789640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850006cf50 con 0x7f8500099a50 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8508789640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850009a000 con 0x7f85000cf900 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8506d26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f850015efb0 0x7f8500158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8506d26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f850015efb0 0x7f8500158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:44424/0 (socket says 172.21.15.139:44424) 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8506d26640 1 -- 172.21.15.139:0/1006195694 learned_addr learned my addr 172.21.15.139:0/1006195694 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8506525640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8500099a50 0x7f8500156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8505d24640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85000cf900 0x7f85000c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8506d26640 1 -- 172.21.15.139:0/1006195694 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85000cf900 msgr2=0x7f85000c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8506d26640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85000cf900 0x7f85000c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8506d26640 1 -- 172.21.15.139:0/1006195694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8500099a50 msgr2=0x7f8500156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8506d26640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8500099a50 0x7f8500156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8506d26640 1 -- 172.21.15.139:0/1006195694 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8500161470 con 0x7f850015efb0 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8506d26640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f850015efb0 0x7f8500158b40 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c780 tx=0x7f84f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f84f6ffd640 1 -- 172.21.15.139:0/1006195694 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8017070 con 0x7f850015efb0 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f84f6ffd640 1 -- 172.21.15.139:0/1006195694 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f8003da0 con 0x7f850015efb0 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f84f6ffd640 1 -- 172.21.15.139:0/1006195694 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f80052b0 con 0x7f850015efb0 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8508789640 1 -- 172.21.15.139:0/1006195694 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8500161750 con 0x7f850015efb0 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.620+0000 7f8508789640 1 -- 172.21.15.139:0/1006195694 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8500161d10 con 0x7f850015efb0 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.621+0000 7f8508789640 1 -- 172.21.15.139:0/1006195694 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8500074520 con 0x7f850015efb0 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.622+0000 7f84f6ffd640 1 -- 172.21.15.139:0/1006195694 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84f8005450 con 0x7f850015efb0 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.622+0000 7f84f6ffd640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f84e406b500 0x7f84e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.622+0000 7f84f6ffd640 1 -- 172.21.15.139:0/1006195694 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f84f8089780 con 0x7f850015efb0 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.622+0000 7f8506525640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f84e406b500 0x7f84e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.623+0000 7f8506525640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f84e406b500 0x7f84e406d9c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f84fc034000 tx=0x7f84fc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.626+0000 7f84f6ffd640 1 -- 172.21.15.139:0/1006195694 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84f8057e20 con 0x7f850015efb0 2024-04-25T10:01:24.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.833+0000 7f8508789640 1 -- 172.21.15.139:0/1006195694 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f850009ae50 con 0x7f850015efb0 2024-04-25T10:01:24.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.834+0000 7f84f6ffd640 1 -- 172.21.15.139:0/1006195694 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7f84f805bec0 con 0x7f850015efb0 2024-04-25T10:01:24.838 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:01:24.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.836+0000 7f8508789640 1 -- 172.21.15.139:0/1006195694 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f84e406b500 msgr2=0x7f84e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.836+0000 7f8508789640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f84e406b500 0x7f84e406d9c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f84fc034000 tx=0x7f84fc02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:24.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.836+0000 7f8508789640 1 -- 172.21.15.139:0/1006195694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f850015efb0 msgr2=0x7f8500158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.836+0000 7f8508789640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f850015efb0 0x7f8500158b40 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c780 tx=0x7f84f800cc40 comp rx=0 tx=0).stop 2024-04-25T10:01:24.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.836+0000 7f8508789640 1 -- 172.21.15.139:0/1006195694 shutdown_connections 2024-04-25T10:01:24.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.836+0000 7f8508789640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f850015efb0 0x7f8500158b40 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.836+0000 7f8508789640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f84e406b500 0x7f84e406d9c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.836+0000 7f8508789640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85000cf900 0x7f85000c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.836+0000 7f8508789640 1 --2- 172.21.15.139:0/1006195694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8500099a50 0x7f8500156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.836+0000 7f8508789640 1 -- 172.21.15.139:0/1006195694 >> 172.21.15.139:0/1006195694 conn(0x7f85000d1790 msgr2=0x7f8500070280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.836+0000 7f8508789640 1 -- 172.21.15.139:0/1006195694 shutdown_connections 2024-04-25T10:01:24.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:24.836+0000 7f8508789640 1 -- 172.21.15.139:0/1006195694 wait complete. 2024-04-25T10:01:24.851 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:01:24.851 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:01:24.851 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T10:01:24.851 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:01:24.851 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:24.851 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:01:24.851 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:01:24.851 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:01:24.851 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:01:24.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:24.852 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T10:01:24.866 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:01:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:01:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:01:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T10:01:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:24.867 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:01:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T10:01:24.867 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:01:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T10:01:24.867 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42214sec preferred_lft 42214sec 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:01:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:01:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T10:01:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:24.869 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:01:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:01:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T10:01:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:24.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:24.869 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:01:24.928 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T10:01:24.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:24.930 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T10:01:24.985 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T10:01:24.985 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:01:24.986 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:01:24.986 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:01:24.986 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:01:24.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:24.986 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T10:01:24.986 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T10:01:24.986 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:01:24.986 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:01:24.986 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:01:24.986 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T10:01:25.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:25.070 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T10:01:25.125 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:01:25.125 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:01:25.125 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:01:25.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:25.125 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T10:01:25.188 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:01:25.189 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:25.244 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:01:25.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:01:25.244 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:25.298 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:01:25.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:25.299 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:01:25.380 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:01:25.380 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:01:25.386 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:01:25.387 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:01:25.387 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T10:01:25.387 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T10:01:25.387 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T10:01:25.387 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:01:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:01:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:01:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:01:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:01:25.389 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:01:25.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:25.389 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:25.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:25.453 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:01:25.509 INFO:teuthology.orchestra.run.smithi139.stdout:client8691 2024-04-25T10:01:25.509 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:25.509 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:25.564 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:25.564 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:25.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:25.619 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:01:25.712 INFO:teuthology.orchestra.run.smithi139.stderr:6+0 records in 2024-04-25T10:01:25.712 INFO:teuthology.orchestra.run.smithi139.stderr:6+0 records out 2024-04-25T10:01:25.712 INFO:teuthology.orchestra.run.smithi139.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0363496 s, 173 MB/s 2024-04-25T10:01:25.713 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:25.713 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:25.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:25.728 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:01:25.820 INFO:teuthology.orchestra.run.smithi139.stderr:6+0 records in 2024-04-25T10:01:25.820 INFO:teuthology.orchestra.run.smithi139.stderr:6+0 records out 2024-04-25T10:01:25.820 INFO:teuthology.orchestra.run.smithi139.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0359437 s, 175 MB/s 2024-04-25T10:01:25.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:25.820 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:01:25.836 INFO:teuthology.orchestra.run.smithi139.stdout:./ 2024-04-25T10:01:25.836 INFO:teuthology.orchestra.run.smithi139.stdout:./file2_sixmegs 2024-04-25T10:01:25.836 INFO:teuthology.orchestra.run.smithi139.stdout:./file3_sixmegs 2024-04-25T10:01:25.837 INFO:teuthology.orchestra.run.smithi139.stdout:./file1_sixmegs 2024-04-25T10:01:25.837 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:01:25.837 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:01:25.837 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:01:25.837 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:01:25.837 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:01:25.837 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:01:25.941 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T10:01:25.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:01:25.950 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:01:25.950 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:01:25.950 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:01:25.950 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:01:25.950 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-25T10:01:25.950 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:01:26.054 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627777 2024-04-25T10:01:26.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:01:26.063 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:01:26.063 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:01:26.064 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:01:26.064 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:01:26.064 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-25T10:01:26.064 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:01:26.167 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627778 2024-04-25T10:01:26.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:01:26.177 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:01:26.177 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:01:26.177 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:01:26.177 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:01:26.177 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-25T10:01:26.177 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:01:26.281 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627776 2024-04-25T10:01:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:01:26.290 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:26.290 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:26.347 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:26.347 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:26.402 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:26.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.533+0000 7f8ecb9cf640 1 Processor -- start 2024-04-25T10:01:26.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.533+0000 7f8ecb9cf640 1 -- start start 2024-04-25T10:01:26.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ecb9cf640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8ec4155e30 0x7f8ec415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ecb9cf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ec40b1e50 0x7f8ec415c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ecb9cf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec415c880 con 0x7f8ec415bdc0 2024-04-25T10:01:26.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ecb9cf640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec406d6f0 con 0x7f8ec4155e30 2024-04-25T10:01:26.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ecb9cf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec415cb60 con 0x7f8ec40b1e50 2024-04-25T10:01:26.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ec8f6a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8ec415bdc0 0x7f8ec40b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41880/0 (socket says 172.21.15.139:41880) 2024-04-25T10:01:26.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ec8f6a640 1 -- 172.21.15.139:0/3735365128 learned_addr learned my addr 172.21.15.139:0/3735365128 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:26.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090870541 0 0) 0x7f8ec415c880 con 0x7f8ec415bdc0 2024-04-25T10:01:26.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ec9f6c640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ec40b1e50 0x7f8ec415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ec976b640 1 --2- 172.21.15.139:0/3735365128 >> v2:172.21.15.150:3300/0 conn(0x7f8ec4155e30 0x7f8ec415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8eb0003520 con 0x7f8ec415bdc0 2024-04-25T10:01:26.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1159298158 0 0) 0x7f8eb0003520 con 0x7f8ec415bdc0 2024-04-25T10:01:26.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ec40b1e50 msgr2=0x7f8ec415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ec40b1e50 0x7f8ec415c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 >> v2:172.21.15.150:3300/0 conn(0x7f8ec4155e30 msgr2=0x7f8ec415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 --2- 172.21.15.139:0/3735365128 >> v2:172.21.15.150:3300/0 conn(0x7f8ec4155e30 0x7f8ec415cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec406d380 con 0x7f8ec415bdc0 2024-04-25T10:01:26.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f8eb4001690 con 0x7f8ec415bdc0 2024-04-25T10:01:26.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 >> v1:172.21.15.139:6789/0 conn(0x7f8ec415bdc0 legacy=0x7f8ec40b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00039b0 0x7f8eb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb0004800 0x7f8eb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ec40b1e50 0x7f8eb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec406d6f0 con 0x7f8eb00039b0 2024-04-25T10:01:26.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec415cb60 con 0x7f8eb0004800 2024-04-25T10:01:26.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.534+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb000b9c0 con 0x7f8ec40b1e50 2024-04-25T10:01:26.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.535+0000 7f8ec976b640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00039b0 0x7f8eb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.535+0000 7f8ec8f6a640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb0004800 0x7f8eb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.535+0000 7f8ec9f6c640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ec40b1e50 0x7f8eb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.535+0000 7f8ec976b640 1 -- 172.21.15.139:0/3735365128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ec40b1e50 msgr2=0x7f8eb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.535+0000 7f8ec976b640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ec40b1e50 0x7f8eb00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.535+0000 7f8ec976b640 1 -- 172.21.15.139:0/3735365128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb0004800 msgr2=0x7f8eb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.535+0000 7f8ec976b640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb0004800 0x7f8eb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.535+0000 7f8ec976b640 1 -- 172.21.15.139:0/3735365128 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb0003520 con 0x7f8eb00039b0 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.535+0000 7f8ec8f6a640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8eb0004800 0x7f8eb0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.535+0000 7f8ec976b640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00039b0 0x7f8eb0003dd0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009ac0 tx=0x7f8eb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea9ec8e112f240d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.536+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0036070 con 0x7f8eb00039b0 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.536+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec0031400 con 0x7f8eb00039b0 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.536+0000 7f8ebbfff640 1 -- 172.21.15.139:0/3735365128 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0009690 con 0x7f8eb00039b0 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.536+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/3735365128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00039b0 msgr2=0x7f8eb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.536+0000 7f8ecb9cf640 1 --2- 172.21.15.139:0/3735365128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00039b0 0x7f8eb0003dd0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009ac0 tx=0x7f8eb0005830 comp rx=0 tx=0).stop 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.536+0000 7f8ec9f6c640 1 -- 172.21.15.139:0/3735365128 reap_dead start 2024-04-25T10:01:26.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.536+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/3735365128 shutdown_connections 2024-04-25T10:01:26.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.536+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/3735365128 >> 172.21.15.139:0/3735365128 conn(0x7f8ec4066f20 msgr2=0x7f8ec406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.536+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/3735365128 shutdown_connections 2024-04-25T10:01:26.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.536+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/3735365128 wait complete. 2024-04-25T10:01:26.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.537+0000 7f8ecb9cf640 1 Processor -- start 2024-04-25T10:01:26.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.537+0000 7f8ecb9cf640 1 -- start start 2024-04-25T10:01:26.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ecb9cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00039b0 0x7f8ec4151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ecb9cf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec4158ce0 0x7f8ec41f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ecb9cf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ec41f3bb0 0x7f8ec40b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ecb9cf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40af260 con 0x7f8eb00039b0 2024-04-25T10:01:26.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ecb9cf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40af0e0 con 0x7f8ec4158ce0 2024-04-25T10:01:26.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ecb9cf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec415d4c0 con 0x7f8ec41f3bb0 2024-04-25T10:01:26.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ec976b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00039b0 0x7f8ec4151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ec976b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00039b0 0x7f8ec4151310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55892/0 (socket says 172.21.15.139:55892) 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ec976b640 1 -- 172.21.15.139:0/517421608 learned_addr learned my addr 172.21.15.139:0/517421608 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ec976b640 1 -- 172.21.15.139:0/517421608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ec41f3bb0 msgr2=0x7f8ec40b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ec976b640 1 --2- 172.21.15.139:0/517421608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ec41f3bb0 0x7f8ec40b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ec976b640 1 -- 172.21.15.139:0/517421608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec4158ce0 msgr2=0x7f8ec41f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ec976b640 1 --2- 172.21.15.139:0/517421608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec4158ce0 0x7f8ec41f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.538+0000 7f8ec976b640 1 -- 172.21.15.139:0/517421608 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec4158260 con 0x7f8eb00039b0 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.539+0000 7f8ec976b640 1 --2- 172.21.15.139:0/517421608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00039b0 0x7f8ec4151310 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009be0 tx=0x7f8ec0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.539+0000 7f8eb9ffb640 1 -- 172.21.15.139:0/517421608 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0041020 con 0x7f8eb00039b0 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.539+0000 7f8eb9ffb640 1 -- 172.21.15.139:0/517421608 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec003c470 con 0x7f8eb00039b0 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.539+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/517421608 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec41f4100 con 0x7f8eb00039b0 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.539+0000 7f8eb9ffb640 1 -- 172.21.15.139:0/517421608 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0036040 con 0x7f8eb00039b0 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.539+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/517421608 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec41584e0 con 0x7f8eb00039b0 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.540+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/517421608 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec4074520 con 0x7f8eb00039b0 2024-04-25T10:01:26.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.540+0000 7f8eb9ffb640 1 -- 172.21.15.139:0/517421608 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ec0044020 con 0x7f8eb00039b0 2024-04-25T10:01:26.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.541+0000 7f8eb9ffb640 1 --2- 172.21.15.139:0/517421608 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ea806b500 0x7f8ea806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.541+0000 7f8eb9ffb640 1 -- 172.21.15.139:0/517421608 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ec00aa140 con 0x7f8eb00039b0 2024-04-25T10:01:26.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.541+0000 7f8ec8f6a640 1 --2- 172.21.15.139:0/517421608 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ea806b500 0x7f8ea806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.541+0000 7f8ec8f6a640 1 --2- 172.21.15.139:0/517421608 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ea806b500 0x7f8ea806d9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4001eb0 tx=0x7f8eb4001b30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.545+0000 7f8eb9ffb640 1 -- 172.21.15.139:0/517421608 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ec4158260 con 0x7f8eb00039b0 2024-04-25T10:01:26.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.719+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/517421608 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ec406b830 con 0x7f8eb00039b0 2024-04-25T10:01:26.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.720+0000 7f8eb9ffb640 1 -- 172.21.15.139:0/517421608 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f8ec0078e30 con 0x7f8eb00039b0 2024-04-25T10:01:26.724 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:26.724 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":105,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:01:17.142273+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:13.695577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:14.460042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:26.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.722+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/517421608 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ea806b500 msgr2=0x7f8ea806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.722+0000 7f8ecb9cf640 1 --2- 172.21.15.139:0/517421608 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ea806b500 0x7f8ea806d9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4001eb0 tx=0x7f8eb4001b30 comp rx=0 tx=0).stop 2024-04-25T10:01:26.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.722+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/517421608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00039b0 msgr2=0x7f8ec4151310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.722+0000 7f8ecb9cf640 1 --2- 172.21.15.139:0/517421608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00039b0 0x7f8ec4151310 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009be0 tx=0x7f8ec0004730 comp rx=0 tx=0).stop 2024-04-25T10:01:26.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.722+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/517421608 shutdown_connections 2024-04-25T10:01:26.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.722+0000 7f8ecb9cf640 1 --2- 172.21.15.139:0/517421608 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ea806b500 0x7f8ea806d9c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.722+0000 7f8ecb9cf640 1 --2- 172.21.15.139:0/517421608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ec41f3bb0 0x7f8ec40b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.722+0000 7f8ecb9cf640 1 --2- 172.21.15.139:0/517421608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec4158ce0 0x7f8ec41f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.722+0000 7f8ecb9cf640 1 --2- 172.21.15.139:0/517421608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8eb00039b0 0x7f8ec4151310 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.722+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/517421608 >> 172.21.15.139:0/517421608 conn(0x7f8ec4066f20 msgr2=0x7f8ec406ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.722+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/517421608 shutdown_connections 2024-04-25T10:01:26.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.722+0000 7f8ecb9cf640 1 -- 172.21.15.139:0/517421608 wait complete. 2024-04-25T10:01:26.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:13.695577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:14.460042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:26.738 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:26.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.833+0000 7f6623d63640 1 Processor -- start 2024-04-25T10:01:26.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.833+0000 7f6623d63640 1 -- start start 2024-04-25T10:01:26.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.834+0000 7f6623d63640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f661c144090 0x7f661c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.834+0000 7f6623d63640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f661c0cc9e0 0x7f661c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.834+0000 7f6623d63640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f661c0cce20 0x7f661c06e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.834+0000 7f6623d63640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661c0b2a50 con 0x7f661c0cce20 2024-04-25T10:01:26.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.834+0000 7f6623d63640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661c15b090 con 0x7f661c0cc9e0 2024-04-25T10:01:26.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.834+0000 7f6623d63640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661c15b370 con 0x7f661c144090 2024-04-25T10:01:26.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.834+0000 7f6622300640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f661c0cce20 0x7f661c06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.834+0000 7f6622300640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f661c0cce20 0x7f661c06e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55908/0 (socket says 172.21.15.139:55908) 2024-04-25T10:01:26.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.834+0000 7f6622300640 1 -- 172.21.15.139:0/526480760 learned_addr learned my addr 172.21.15.139:0/526480760 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:26.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.835+0000 7f6621aff640 1 --2- 172.21.15.139:0/526480760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f661c144090 0x7f661c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.835+0000 7f66212fe640 1 --2- 172.21.15.139:0/526480760 >> v2:172.21.15.150:3300/0 conn(0x7f661c0cc9e0 0x7f661c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.835+0000 7f6622300640 1 -- 172.21.15.139:0/526480760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f661c144090 msgr2=0x7f661c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.835+0000 7f6622300640 1 --2- 172.21.15.139:0/526480760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f661c144090 0x7f661c0b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.835+0000 7f6622300640 1 -- 172.21.15.139:0/526480760 >> v2:172.21.15.150:3300/0 conn(0x7f661c0cc9e0 msgr2=0x7f661c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.835+0000 7f6622300640 1 --2- 172.21.15.139:0/526480760 >> v2:172.21.15.150:3300/0 conn(0x7f661c0cc9e0 0x7f661c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.835+0000 7f6622300640 1 -- 172.21.15.139:0/526480760 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c15b650 con 0x7f661c0cce20 2024-04-25T10:01:26.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.835+0000 7f6621aff640 1 --2- 172.21.15.139:0/526480760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f661c144090 0x7f661c0b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:26.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.835+0000 7f66212fe640 1 --2- 172.21.15.139:0/526480760 >> v2:172.21.15.150:3300/0 conn(0x7f661c0cc9e0 0x7f661c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:26.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.835+0000 7f6622300640 1 --2- 172.21.15.139:0/526480760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c0cce20 0x7f661c06e870 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f661c06b930 tx=0x7f661402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6554d90fb738bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.835+0000 7f6620afd640 1 -- 172.21.15.139:0/526480760 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614036070 con 0x7f661c0cce20 2024-04-25T10:01:26.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.835+0000 7f6620afd640 1 -- 172.21.15.139:0/526480760 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6614031600 con 0x7f661c0cce20 2024-04-25T10:01:26.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.836+0000 7f6620afd640 1 -- 172.21.15.139:0/526480760 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66140319d0 con 0x7f661c0cce20 2024-04-25T10:01:26.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.836+0000 7f6623d63640 1 -- 172.21.15.139:0/526480760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c0cce20 msgr2=0x7f661c06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.836+0000 7f6623d63640 1 --2- 172.21.15.139:0/526480760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c0cce20 0x7f661c06e870 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f661c06b930 tx=0x7f661402a610 comp rx=0 tx=0).stop 2024-04-25T10:01:26.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.836+0000 7f6623d63640 1 -- 172.21.15.139:0/526480760 shutdown_connections 2024-04-25T10:01:26.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.836+0000 7f6623d63640 1 --2- 172.21.15.139:0/526480760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c0cce20 0x7f661c06e870 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.836+0000 7f6623d63640 1 --2- 172.21.15.139:0/526480760 >> v2:172.21.15.150:3300/0 conn(0x7f661c0cc9e0 0x7f661c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.836+0000 7f6623d63640 1 --2- 172.21.15.139:0/526480760 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f661c144090 0x7f661c0b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.836+0000 7f6623d63640 1 -- 172.21.15.139:0/526480760 >> 172.21.15.139:0/526480760 conn(0x7f661c067000 msgr2=0x7f661c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.836+0000 7f6623d63640 1 -- 172.21.15.139:0/526480760 shutdown_connections 2024-04-25T10:01:26.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.836+0000 7f6623d63640 1 -- 172.21.15.139:0/526480760 wait complete. 2024-04-25T10:01:26.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.837+0000 7f6623d63640 1 Processor -- start 2024-04-25T10:01:26.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.837+0000 7f6623d63640 1 -- start start 2024-04-25T10:01:26.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6623d63640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f661c0cc9e0 0x7f661c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6623d63640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f661c0cce20 0x7f661c1f1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6623d63640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c144090 0x7f661c142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6623d63640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c144470 con 0x7f661c144090 2024-04-25T10:01:26.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6623d63640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c15bc10 con 0x7f661c0cce20 2024-04-25T10:01:26.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6623d63640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661c15ad80 con 0x7f661c0cc9e0 2024-04-25T10:01:26.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6622300640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c144090 0x7f661c142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6622300640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c144090 0x7f661c142bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55920/0 (socket says 172.21.15.139:55920) 2024-04-25T10:01:26.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6622300640 1 -- 172.21.15.139:0/3197255402 learned_addr learned my addr 172.21.15.139:0/3197255402 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:26.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6621aff640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f661c0cc9e0 0x7f661c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f66212fe640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f661c0cce20 0x7f661c1f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6622300640 1 -- 172.21.15.139:0/3197255402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f661c0cc9e0 msgr2=0x7f661c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6622300640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f661c0cc9e0 0x7f661c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6622300640 1 -- 172.21.15.139:0/3197255402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f661c0cce20 msgr2=0x7f661c1f1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6622300640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f661c0cce20 0x7f661c1f1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f6622300640 1 -- 172.21.15.139:0/3197255402 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c15fb20 con 0x7f661c144090 2024-04-25T10:01:26.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.838+0000 7f66212fe640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f661c0cce20 0x7f661c1f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:26.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.839+0000 7f6622300640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c144090 0x7f661c142bb0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6614034040 tx=0x7f6614004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.839+0000 7f66127fc640 1 -- 172.21.15.139:0/3197255402 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614041020 con 0x7f661c144090 2024-04-25T10:01:26.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.839+0000 7f66127fc640 1 -- 172.21.15.139:0/3197255402 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66140077a0 con 0x7f661c144090 2024-04-25T10:01:26.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.839+0000 7f6623d63640 1 -- 172.21.15.139:0/3197255402 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661c15fe00 con 0x7f661c144090 2024-04-25T10:01:26.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.839+0000 7f66127fc640 1 -- 172.21.15.139:0/3197255402 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614036040 con 0x7f661c144090 2024-04-25T10:01:26.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.839+0000 7f6623d63640 1 -- 172.21.15.139:0/3197255402 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661c1603c0 con 0x7f661c144090 2024-04-25T10:01:26.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.840+0000 7f6623d63640 1 -- 172.21.15.139:0/3197255402 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661c074520 con 0x7f661c144090 2024-04-25T10:01:26.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.840+0000 7f66127fc640 1 -- 172.21.15.139:0/3197255402 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6614044020 con 0x7f661c144090 2024-04-25T10:01:26.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.840+0000 7f66127fc640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f660006b4d0 0x7f660006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.841+0000 7f6621aff640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f660006b4d0 0x7f660006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.841+0000 7f66127fc640 1 -- 172.21.15.139:0/3197255402 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f66140ab140 con 0x7f661c144090 2024-04-25T10:01:26.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.841+0000 7f6621aff640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f660006b4d0 0x7f660006d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f661800aac0 tx=0x7f6618008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:26.845+0000 7f66127fc640 1 -- 172.21.15.139:0/3197255402 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f661404b030 con 0x7f661c144090 2024-04-25T10:01:27.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.013+0000 7f6623d63640 1 -- 172.21.15.139:0/3197255402 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f661c164830 con 0x7f661c144090 2024-04-25T10:01:27.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.013+0000 7f66127fc640 1 -- 172.21.15.139:0/3197255402 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6614079ec0 con 0x7f661c144090 2024-04-25T10:01:27.018 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:01:27.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.015+0000 7f6623d63640 1 -- 172.21.15.139:0/3197255402 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f660006b4d0 msgr2=0x7f660006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.015+0000 7f6623d63640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f660006b4d0 0x7f660006d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f661800aac0 tx=0x7f6618008040 comp rx=0 tx=0).stop 2024-04-25T10:01:27.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.015+0000 7f6623d63640 1 -- 172.21.15.139:0/3197255402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c144090 msgr2=0x7f661c142bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.015+0000 7f6623d63640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c144090 0x7f661c142bb0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6614034040 tx=0x7f6614004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:27.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.015+0000 7f6623d63640 1 -- 172.21.15.139:0/3197255402 shutdown_connections 2024-04-25T10:01:27.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.015+0000 7f6623d63640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f661c144090 0x7f661c142bb0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.015+0000 7f6623d63640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f660006b4d0 0x7f660006d990 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.015+0000 7f6623d63640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f661c0cce20 0x7f661c1f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.015+0000 7f6623d63640 1 --2- 172.21.15.139:0/3197255402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f661c0cc9e0 0x7f661c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.015+0000 7f6623d63640 1 -- 172.21.15.139:0/3197255402 >> 172.21.15.139:0/3197255402 conn(0x7f661c067000 msgr2=0x7f661c06d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.015+0000 7f6623d63640 1 -- 172.21.15.139:0/3197255402 shutdown_connections 2024-04-25T10:01:27.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.015+0000 7f6623d63640 1 -- 172.21.15.139:0/3197255402 wait complete. 2024-04-25T10:01:27.030 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:01:27.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.126+0000 7f283cbe7640 1 Processor -- start 2024-04-25T10:01:27.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.126+0000 7f283cbe7640 1 -- start start 2024-04-25T10:01:27.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.127+0000 7f283cbe7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f28380ae880 0x7f28380aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.127+0000 7f283cbe7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28380cdca0 0x7f28380ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.127+0000 7f283cbe7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28380cf930 con 0x7f283815c380 2024-04-25T10:01:27.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.127+0000 7f283cbe7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283815ce70 con 0x7f28380ae880 2024-04-25T10:01:27.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.127+0000 7f283cbe7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28380cfc10 con 0x7f28380cdca0 2024-04-25T10:01:27.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.127+0000 7f283659c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f283815c380 0x7f283815c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41892/0 (socket says 172.21.15.139:41892) 2024-04-25T10:01:27.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.127+0000 7f283659c640 1 -- 172.21.15.139:0/2712784512 learned_addr learned my addr 172.21.15.139:0/2712784512 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:27.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.127+0000 7f2835d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f28380ae880 0x7f28380aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.128+0000 7f2836d9d640 1 --2- 172.21.15.139:0/2712784512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28380cdca0 0x7f28380ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.128+0000 7f283559a640 1 -- 172.21.15.139:0/2712784512 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171525557 0 0) 0x7f28380cf930 con 0x7f283815c380 2024-04-25T10:01:27.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.128+0000 7f283559a640 1 -- 172.21.15.139:0/2712784512 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2820003520 con 0x7f283815c380 2024-04-25T10:01:27.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.128+0000 7f2835d9b640 1 -- 172.21.15.139:0/2712784512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28380cdca0 msgr2=0x7f28380ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:27.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.128+0000 7f2835d9b640 1 --2- 172.21.15.139:0/2712784512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28380cdca0 0x7f28380ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.128+0000 7f2835d9b640 1 -- 172.21.15.139:0/2712784512 >> v1:172.21.15.139:6789/0 conn(0x7f283815c380 legacy=0x7f283815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.128+0000 7f2835d9b640 1 -- 172.21.15.139:0/2712784512 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28380acf30 con 0x7f28380ae880 2024-04-25T10:01:27.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.128+0000 7f2835d9b640 1 --2- 172.21.15.139:0/2712784512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28380ae880 0x7f28380aecc0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f282c0097a0 tx=0x7f282c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c47f42efecc8b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.129+0000 7f283559a640 1 -- 172.21.15.139:0/2712784512 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c037070 con 0x7f28380ae880 2024-04-25T10:01:27.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.129+0000 7f283559a640 1 -- 172.21.15.139:0/2712784512 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c032440 con 0x7f28380ae880 2024-04-25T10:01:27.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.129+0000 7f283559a640 1 -- 172.21.15.139:0/2712784512 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c032810 con 0x7f28380ae880 2024-04-25T10:01:27.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.129+0000 7f283cbe7640 1 -- 172.21.15.139:0/2712784512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28380ae880 msgr2=0x7f28380aecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.129+0000 7f283cbe7640 1 --2- 172.21.15.139:0/2712784512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28380ae880 0x7f28380aecc0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f282c0097a0 tx=0x7f282c02a180 comp rx=0 tx=0).stop 2024-04-25T10:01:27.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.129+0000 7f283cbe7640 1 -- 172.21.15.139:0/2712784512 shutdown_connections 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.129+0000 7f283cbe7640 1 --2- 172.21.15.139:0/2712784512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28380cdca0 0x7f28380ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.129+0000 7f283cbe7640 1 --2- 172.21.15.139:0/2712784512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28380ae880 0x7f28380aecc0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.129+0000 7f283cbe7640 1 -- 172.21.15.139:0/2712784512 >> 172.21.15.139:0/2712784512 conn(0x7f28380cc820 msgr2=0x7f28380ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.129+0000 7f283cbe7640 1 -- 172.21.15.139:0/2712784512 shutdown_connections 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.129+0000 7f283cbe7640 1 -- 172.21.15.139:0/2712784512 wait complete. 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.131+0000 7f283cbe7640 1 Processor -- start 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.131+0000 7f283cbe7640 1 -- start start 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.131+0000 7f283cbe7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28380ae880 0x7f28381532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.131+0000 7f283cbe7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28380cdca0 0x7f2838153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.131+0000 7f283cbe7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f283815c380 0x7f283815a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.131+0000 7f283cbe7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28380cf620 con 0x7f28380ae880 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.131+0000 7f283cbe7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28380668f0 con 0x7f28380cdca0 2024-04-25T10:01:27.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.131+0000 7f283cbe7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28380af200 con 0x7f283815c380 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.131+0000 7f283659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28380ae880 0x7f28381532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.131+0000 7f283659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28380ae880 0x7f28381532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55926/0 (socket says 172.21.15.139:55926) 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f283659c640 1 -- 172.21.15.139:0/4179472893 learned_addr learned my addr 172.21.15.139:0/4179472893 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f2836d9d640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f283815c380 0x7f283815a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f2835d9b640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28380cdca0 0x7f2838153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f283659c640 1 -- 172.21.15.139:0/4179472893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f283815c380 msgr2=0x7f283815a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f283659c640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f283815c380 0x7f283815a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f283659c640 1 -- 172.21.15.139:0/4179472893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28380cdca0 msgr2=0x7f2838153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f283659c640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28380cdca0 0x7f2838153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f283659c640 1 -- 172.21.15.139:0/4179472893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283806dc80 con 0x7f28380ae880 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f2835d9b640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28380cdca0 0x7f2838153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f283659c640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28380ae880 0x7f28381532f0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f282800cad0 tx=0x7f282800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f2826ffd640 1 -- 172.21.15.139:0/4179472893 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828017070 con 0x7f28380ae880 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f2826ffd640 1 -- 172.21.15.139:0/4179472893 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282800abd0 con 0x7f28380ae880 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f283cbe7640 1 -- 172.21.15.139:0/4179472893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283806df60 con 0x7f28380ae880 2024-04-25T10:01:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f2826ffd640 1 -- 172.21.15.139:0/4179472893 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828012400 con 0x7f28380ae880 2024-04-25T10:01:27.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.132+0000 7f283cbe7640 1 -- 172.21.15.139:0/4179472893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2838153f50 con 0x7f28380ae880 2024-04-25T10:01:27.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.133+0000 7f283cbe7640 1 -- 172.21.15.139:0/4179472893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2838067bb0 con 0x7f28380ae880 2024-04-25T10:01:27.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.133+0000 7f2826ffd640 1 -- 172.21.15.139:0/4179472893 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f282801a070 con 0x7f28380ae880 2024-04-25T10:01:27.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.134+0000 7f2826ffd640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f281806b4d0 0x7f281806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.134+0000 7f2826ffd640 1 -- 172.21.15.139:0/4179472893 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f282808a1c0 con 0x7f28380ae880 2024-04-25T10:01:27.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.134+0000 7f2835d9b640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f281806b4d0 0x7f281806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.134+0000 7f2835d9b640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f281806b4d0 0x7f281806d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f282c034000 tx=0x7f282c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.138+0000 7f2826ffd640 1 -- 172.21.15.139:0/4179472893 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2838067bb0 con 0x7f28380ae880 2024-04-25T10:01:27.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.309+0000 7f283cbe7640 1 -- 172.21.15.139:0/4179472893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2838074520 con 0x7f28380ae880 2024-04-25T10:01:27.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.309+0000 7f2826ffd640 1 -- 172.21.15.139:0/4179472893 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2828058e20 con 0x7f28380ae880 2024-04-25T10:01:27.314 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T10:01:27.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.311+0000 7f283cbe7640 1 -- 172.21.15.139:0/4179472893 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f281806b4d0 msgr2=0x7f281806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.311+0000 7f283cbe7640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f281806b4d0 0x7f281806d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f282c034000 tx=0x7f282c02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:27.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.311+0000 7f283cbe7640 1 -- 172.21.15.139:0/4179472893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28380ae880 msgr2=0x7f28381532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.311+0000 7f283cbe7640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28380ae880 0x7f28381532f0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f282800cad0 tx=0x7f282800cf90 comp rx=0 tx=0).stop 2024-04-25T10:01:27.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.311+0000 7f283cbe7640 1 -- 172.21.15.139:0/4179472893 shutdown_connections 2024-04-25T10:01:27.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.311+0000 7f283cbe7640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f281806b4d0 0x7f281806d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.311+0000 7f283cbe7640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f283815c380 0x7f283815a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.311+0000 7f283cbe7640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28380cdca0 0x7f2838153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.311+0000 7f283cbe7640 1 --2- 172.21.15.139:0/4179472893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28380ae880 0x7f28381532f0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.311+0000 7f283cbe7640 1 -- 172.21.15.139:0/4179472893 >> 172.21.15.139:0/4179472893 conn(0x7f28380cc820 msgr2=0x7f283815fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.311+0000 7f283cbe7640 1 -- 172.21.15.139:0/4179472893 shutdown_connections 2024-04-25T10:01:27.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.311+0000 7f283cbe7640 1 -- 172.21.15.139:0/4179472893 wait complete. 2024-04-25T10:01:27.326 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:01:27.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.422+0000 7f6650b09640 1 Processor -- start 2024-04-25T10:01:27.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.422+0000 7f6650b09640 1 -- start start 2024-04-25T10:01:27.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f6650b09640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f664c145230 0x7f664c15be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f6650b09640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f664c15c3d0 0x7f664c0711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f6650b09640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c15c960 con 0x7f664c071700 2024-04-25T10:01:27.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f6650b09640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c071b30 con 0x7f664c145230 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f6650b09640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c15cc40 con 0x7f664c15c3d0 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f664a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f664c145230 0x7f664c15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f664ad9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f664c071700 0x7f664c06d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41906/0 (socket says 172.21.15.139:41906) 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f664ad9d640 1 -- 172.21.15.139:0/3426508688 learned_addr learned my addr 172.21.15.139:0/3426508688 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f664a59c640 1 -- 172.21.15.139:0/3426508688 >> v2:172.21.15.150:3300/0 conn(0x7f664c15c3d0 msgr2=0x7f664c0711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f664a59c640 1 --2- 172.21.15.139:0/3426508688 >> v2:172.21.15.150:3300/0 conn(0x7f664c15c3d0 0x7f664c0711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f664a59c640 1 -- 172.21.15.139:0/3426508688 >> v1:172.21.15.139:6789/0 conn(0x7f664c071700 legacy=0x7f664c06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f664a59c640 1 -- 172.21.15.139:0/3426508688 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c15d230 con 0x7f664c145230 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.423+0000 7f6649d9b640 1 -- 172.21.15.139:0/3426508688 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876233857 0 0) 0x7f664c15c960 con 0x7f664c071700 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.424+0000 7f664a59c640 1 --2- 172.21.15.139:0/3426508688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c145230 0x7f664c15be90 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f664c0cd540 tx=0x7f663c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917eb4836501d94c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.424+0000 7f6649d9b640 1 -- 172.21.15.139:0/3426508688 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c037070 con 0x7f664c145230 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.424+0000 7f6649d9b640 1 -- 172.21.15.139:0/3426508688 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c032440 con 0x7f664c145230 2024-04-25T10:01:27.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.424+0000 7f6649d9b640 1 -- 172.21.15.139:0/3426508688 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c032810 con 0x7f664c145230 2024-04-25T10:01:27.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.424+0000 7f6650b09640 1 -- 172.21.15.139:0/3426508688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c145230 msgr2=0x7f664c15be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.424+0000 7f6650b09640 1 --2- 172.21.15.139:0/3426508688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c145230 0x7f664c15be90 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f664c0cd540 tx=0x7f663c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.424+0000 7f6650b09640 1 -- 172.21.15.139:0/3426508688 shutdown_connections 2024-04-25T10:01:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.424+0000 7f6650b09640 1 --2- 172.21.15.139:0/3426508688 >> v2:172.21.15.150:3300/0 conn(0x7f664c15c3d0 0x7f664c0711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.424+0000 7f6650b09640 1 --2- 172.21.15.139:0/3426508688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c145230 0x7f664c15be90 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.425+0000 7f6650b09640 1 -- 172.21.15.139:0/3426508688 >> 172.21.15.139:0/3426508688 conn(0x7f664c067000 msgr2=0x7f664c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.425+0000 7f6650b09640 1 -- 172.21.15.139:0/3426508688 shutdown_connections 2024-04-25T10:01:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.425+0000 7f6650b09640 1 -- 172.21.15.139:0/3426508688 wait complete. 2024-04-25T10:01:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.426+0000 7f6650b09640 1 Processor -- start 2024-04-25T10:01:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.426+0000 7f6650b09640 1 -- start start 2024-04-25T10:01:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6650b09640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f664c071700 0x7f664c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6650b09640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f664c145230 0x7f664c138b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6650b09640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c15c3d0 0x7f664c1390c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6650b09640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c15ea40 con 0x7f664c15c3d0 2024-04-25T10:01:27.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6650b09640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c06e4b0 con 0x7f664c071700 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6650b09640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c15ebc0 con 0x7f664c145230 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f664ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c15c3d0 0x7f664c1390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f664ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c15c3d0 0x7f664c1390c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55944/0 (socket says 172.21.15.139:55944) 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f664ad9d640 1 -- 172.21.15.139:0/2433220786 learned_addr learned my addr 172.21.15.139:0/2433220786 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6643fff640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f664c145230 0x7f664c138b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f664a59c640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f664c071700 0x7f664c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f664ad9d640 1 -- 172.21.15.139:0/2433220786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f664c145230 msgr2=0x7f664c138b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f664ad9d640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f664c145230 0x7f664c138b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f664ad9d640 1 -- 172.21.15.139:0/2433220786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f664c071700 msgr2=0x7f664c15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f664ad9d640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f664c071700 0x7f664c15afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f664ad9d640 1 -- 172.21.15.139:0/2433220786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c0c5550 con 0x7f664c15c3d0 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6643fff640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f664c145230 0x7f664c138b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f664ad9d640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c15c3d0 0x7f664c1390c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f664400c730 tx=0x7f664400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6642ffd640 1 -- 172.21.15.139:0/2433220786 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644018070 con 0x7f664c15c3d0 2024-04-25T10:01:27.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6642ffd640 1 -- 172.21.15.139:0/2433220786 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664400ce40 con 0x7f664c15c3d0 2024-04-25T10:01:27.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6650b09640 1 -- 172.21.15.139:0/2433220786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c0c5830 con 0x7f664c15c3d0 2024-04-25T10:01:27.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6642ffd640 1 -- 172.21.15.139:0/2433220786 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664400a860 con 0x7f664c15c3d0 2024-04-25T10:01:27.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.427+0000 7f6650b09640 1 -- 172.21.15.139:0/2433220786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c0c5df0 con 0x7f664c15c3d0 2024-04-25T10:01:27.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.428+0000 7f6650b09640 1 -- 172.21.15.139:0/2433220786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664c074520 con 0x7f664c15c3d0 2024-04-25T10:01:27.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.429+0000 7f6642ffd640 1 -- 172.21.15.139:0/2433220786 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f664401b070 con 0x7f664c15c3d0 2024-04-25T10:01:27.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.429+0000 7f6642ffd640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f662c06b4d0 0x7f662c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.429+0000 7f664a59c640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f662c06b4d0 0x7f662c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.429+0000 7f6642ffd640 1 -- 172.21.15.139:0/2433220786 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6644089900 con 0x7f664c15c3d0 2024-04-25T10:01:27.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.430+0000 7f664a59c640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f662c06b4d0 0x7f662c06d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f663c034000 tx=0x7f663c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.434+0000 7f6642ffd640 1 -- 172.21.15.139:0/2433220786 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6644058680 con 0x7f664c15c3d0 2024-04-25T10:01:27.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.602+0000 7f6650b09640 1 -- 172.21.15.139:0/2433220786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f664c06c1a0 con 0x7f664c15c3d0 2024-04-25T10:01:27.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.603+0000 7f6642ffd640 1 -- 172.21.15.139:0/2433220786 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6644057f80 con 0x7f664c15c3d0 2024-04-25T10:01:27.607 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T10:01:27.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.604+0000 7f6650b09640 1 -- 172.21.15.139:0/2433220786 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f662c06b4d0 msgr2=0x7f662c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.604+0000 7f6650b09640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f662c06b4d0 0x7f662c06d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f663c034000 tx=0x7f663c02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:27.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.604+0000 7f6650b09640 1 -- 172.21.15.139:0/2433220786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c15c3d0 msgr2=0x7f664c1390c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.604+0000 7f6650b09640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c15c3d0 0x7f664c1390c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f664400c730 tx=0x7f664400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.605+0000 7f6650b09640 1 -- 172.21.15.139:0/2433220786 shutdown_connections 2024-04-25T10:01:27.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.605+0000 7f6650b09640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f664c15c3d0 0x7f664c1390c0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.605+0000 7f6650b09640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f662c06b4d0 0x7f662c06d990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.605+0000 7f6650b09640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f664c145230 0x7f664c138b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.605+0000 7f6650b09640 1 --2- 172.21.15.139:0/2433220786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f664c071700 0x7f664c15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.605+0000 7f6650b09640 1 -- 172.21.15.139:0/2433220786 >> 172.21.15.139:0/2433220786 conn(0x7f664c067000 msgr2=0x7f664c0ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.605+0000 7f6650b09640 1 -- 172.21.15.139:0/2433220786 shutdown_connections 2024-04-25T10:01:27.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.605+0000 7f6650b09640 1 -- 172.21.15.139:0/2433220786 wait complete. 2024-04-25T10:01:27.620 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:01:27.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.712+0000 7f151e6b3640 1 Processor -- start 2024-04-25T10:01:27.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.712+0000 7f151e6b3640 1 -- start start 2024-04-25T10:01:27.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.713+0000 7f151e6b3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1518145250 0x7f1518070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.713+0000 7f151e6b3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f15180719d0 0x7f1518071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.713+0000 7f151e6b3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15180b1100 con 0x7f15180d11a0 2024-04-25T10:01:27.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.713+0000 7f151e6b3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15180ceed0 con 0x7f15180719d0 2024-04-25T10:01:27.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.713+0000 7f151e6b3640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15180ce2c0 con 0x7f1518145250 2024-04-25T10:01:27.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.713+0000 7f15177fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f15180719d0 0x7f1518071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.714+0000 7f15177fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f15180719d0 0x7f1518071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55954/0 (socket says 172.21.15.139:55954) 2024-04-25T10:01:27.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.714+0000 7f15177fe640 1 -- 172.21.15.139:0/554678341 learned_addr learned my addr 172.21.15.139:0/554678341 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:27.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.714+0000 7f1517fff640 1 --2- 172.21.15.139:0/554678341 >> v2:172.21.15.150:3300/0 conn(0x7f1518145250 0x7f1518070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.714+0000 7f15177fe640 1 -- 172.21.15.139:0/554678341 >> v2:172.21.15.150:3300/0 conn(0x7f1518145250 msgr2=0x7f1518070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:27.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.714+0000 7f15177fe640 1 --2- 172.21.15.139:0/554678341 >> v2:172.21.15.150:3300/0 conn(0x7f1518145250 0x7f1518070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.714+0000 7f15177fe640 1 -- 172.21.15.139:0/554678341 >> v1:172.21.15.150:6789/0 conn(0x7f15180d11a0 legacy=0x7f151815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.714+0000 7f15177fe640 1 -- 172.21.15.139:0/554678341 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15180ce5a0 con 0x7f15180719d0 2024-04-25T10:01:27.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.714+0000 7f15177fe640 1 --2- 172.21.15.139:0/554678341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15180719d0 0x7f1518071120 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f151806efc0 tx=0x7f150c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae84ee92789afd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.715+0000 7f1516ffd640 1 -- 172.21.15.139:0/554678341 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c037070 con 0x7f15180719d0 2024-04-25T10:01:27.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.715+0000 7f1516ffd640 1 -- 172.21.15.139:0/554678341 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c032440 con 0x7f15180719d0 2024-04-25T10:01:27.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.715+0000 7f1516ffd640 1 -- 172.21.15.139:0/554678341 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c032810 con 0x7f15180719d0 2024-04-25T10:01:27.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.715+0000 7f151e6b3640 1 -- 172.21.15.139:0/554678341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15180719d0 msgr2=0x7f1518071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.715+0000 7f151e6b3640 1 --2- 172.21.15.139:0/554678341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15180719d0 0x7f1518071120 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f151806efc0 tx=0x7f150c02a150 comp rx=0 tx=0).stop 2024-04-25T10:01:27.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.715+0000 7f151e6b3640 1 -- 172.21.15.139:0/554678341 shutdown_connections 2024-04-25T10:01:27.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.715+0000 7f151e6b3640 1 --2- 172.21.15.139:0/554678341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15180719d0 0x7f1518071120 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.715+0000 7f151e6b3640 1 --2- 172.21.15.139:0/554678341 >> v2:172.21.15.150:3300/0 conn(0x7f1518145250 0x7f1518070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.715+0000 7f151e6b3640 1 -- 172.21.15.139:0/554678341 >> 172.21.15.139:0/554678341 conn(0x7f151806f460 msgr2=0x7f151806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.715+0000 7f151e6b3640 1 -- 172.21.15.139:0/554678341 shutdown_connections 2024-04-25T10:01:27.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.715+0000 7f151e6b3640 1 -- 172.21.15.139:0/554678341 wait complete. 2024-04-25T10:01:27.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.716+0000 7f151e6b3640 1 Processor -- start 2024-04-25T10:01:27.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.717+0000 7f151e6b3640 1 -- start start 2024-04-25T10:01:27.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.717+0000 7f151e6b3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15180719d0 0x7f15181d6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.717+0000 7f151e6b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15180d11a0 0x7f15180c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.717+0000 7f151e6b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1518145250 0x7f15181405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.717+0000 7f151e6b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1518145630 con 0x7f1518145250 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.717+0000 7f151e6b3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15180ceb60 con 0x7f15180719d0 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.717+0000 7f151e6b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15180b5690 con 0x7f15180d11a0 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.717+0000 7f151cc50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1518145250 0x7f15181405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.717+0000 7f151cc50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1518145250 0x7f15181405b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55960/0 (socket says 172.21.15.139:55960) 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.717+0000 7f151cc50640 1 -- 172.21.15.139:0/3791304237 learned_addr learned my addr 172.21.15.139:0/3791304237 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f15177fe640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15180d11a0 0x7f15180c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f1517fff640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15180719d0 0x7f15181d6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f151cc50640 1 -- 172.21.15.139:0/3791304237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15180d11a0 msgr2=0x7f15180c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f151cc50640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15180d11a0 0x7f15180c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f151cc50640 1 -- 172.21.15.139:0/3791304237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15180719d0 msgr2=0x7f15181d6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f151cc50640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15180719d0 0x7f15181d6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f151cc50640 1 -- 172.21.15.139:0/3791304237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15181f5a80 con 0x7f1518145250 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f1517fff640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15180719d0 0x7f15181d6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f151cc50640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1518145250 0x7f15181405b0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f151000c7a0 tx=0x7f151000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f1514ff9640 1 -- 172.21.15.139:0/3791304237 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510003b70 con 0x7f1518145250 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f1514ff9640 1 -- 172.21.15.139:0/3791304237 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1510003d30 con 0x7f1518145250 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f151e6b3640 1 -- 172.21.15.139:0/3791304237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15181f5d60 con 0x7f1518145250 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f1514ff9640 1 -- 172.21.15.139:0/3791304237 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151000aa90 con 0x7f1518145250 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.718+0000 7f151e6b3640 1 -- 172.21.15.139:0/3791304237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15181f6320 con 0x7f1518145250 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.719+0000 7f151e6b3640 1 -- 172.21.15.139:0/3791304237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1518074520 con 0x7f1518145250 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.719+0000 7f1514ff9640 1 -- 172.21.15.139:0/3791304237 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f151000ac30 con 0x7f1518145250 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.720+0000 7f1514ff9640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f150406b500 0x7f150406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.720+0000 7f1517fff640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f150406b500 0x7f150406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.720+0000 7f1514ff9640 1 -- 172.21.15.139:0/3791304237 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f151008a1e0 con 0x7f1518145250 2024-04-25T10:01:27.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.720+0000 7f1517fff640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f150406b500 0x7f150406d9c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1508009af0 tx=0x7f1508008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.724+0000 7f1514ff9640 1 -- 172.21.15.139:0/3791304237 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1510058ed0 con 0x7f1518145250 2024-04-25T10:01:27.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.903+0000 7f151e6b3640 1 -- 172.21.15.139:0/3791304237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f15180701f0 con 0x7f1518145250 2024-04-25T10:01:27.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.903+0000 7f1514ff9640 1 -- 172.21.15.139:0/3791304237 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) v1 ==== 80+0+0 (secure 0 0 0) 0x7f15100587d0 con 0x7f1518145250 2024-04-25T10:01:27.908 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T10:01:27.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.905+0000 7f151e6b3640 1 -- 172.21.15.139:0/3791304237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f150406b500 msgr2=0x7f150406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.905+0000 7f151e6b3640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f150406b500 0x7f150406d9c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1508009af0 tx=0x7f1508008040 comp rx=0 tx=0).stop 2024-04-25T10:01:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.905+0000 7f151e6b3640 1 -- 172.21.15.139:0/3791304237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1518145250 msgr2=0x7f15181405b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.905+0000 7f151e6b3640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1518145250 0x7f15181405b0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f151000c7a0 tx=0x7f151000cc60 comp rx=0 tx=0).stop 2024-04-25T10:01:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.905+0000 7f151e6b3640 1 -- 172.21.15.139:0/3791304237 shutdown_connections 2024-04-25T10:01:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.905+0000 7f151e6b3640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1518145250 0x7f15181405b0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.905+0000 7f151e6b3640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f150406b500 0x7f150406d9c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.905+0000 7f151e6b3640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15180d11a0 0x7f15180c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.905+0000 7f151e6b3640 1 --2- 172.21.15.139:0/3791304237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15180719d0 0x7f15181d6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.905+0000 7f151e6b3640 1 -- 172.21.15.139:0/3791304237 >> 172.21.15.139:0/3791304237 conn(0x7f151806f460 msgr2=0x7f15181e2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.906+0000 7f151e6b3640 1 -- 172.21.15.139:0/3791304237 shutdown_connections 2024-04-25T10:01:27.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:27.906+0000 7f151e6b3640 1 -- 172.21.15.139:0/3791304237 wait complete. 2024-04-25T10:01:27.921 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:01:27.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:27.921 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:27.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:01:27.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:27.966 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:01:27.993 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:28.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.125+0000 7fbf41d7e640 1 Processor -- start 2024-04-25T10:01:28.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.125+0000 7fbf41d7e640 1 -- start start 2024-04-25T10:01:28.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf41d7e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbf3c15b950 0x7fbf3c072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf41d7e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbf3c0d0970 0x7fbf3c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf41d7e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c070a40 con 0x7fbf3c06ab60 2024-04-25T10:01:28.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf41d7e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c06b500 con 0x7fbf3c15b950 2024-04-25T10:01:28.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf41d7e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c06fc80 con 0x7fbf3c0d0970 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf3b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbf3c15b950 0x7fbf3c072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf3b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbf3c15b950 0x7fbf3c072a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55972/0 (socket says 172.21.15.139:55972) 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf3b7fe640 1 -- 172.21.15.139:0/4123995151 learned_addr learned my addr 172.21.15.139:0/4123995151 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf3bfff640 1 --2- 172.21.15.139:0/4123995151 >> v2:172.21.15.150:3300/0 conn(0x7fbf3c0d0970 0x7fbf3c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf3b7fe640 1 -- 172.21.15.139:0/4123995151 >> v2:172.21.15.150:3300/0 conn(0x7fbf3c0d0970 msgr2=0x7fbf3c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf3b7fe640 1 --2- 172.21.15.139:0/4123995151 >> v2:172.21.15.150:3300/0 conn(0x7fbf3c0d0970 0x7fbf3c06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf3b7fe640 1 -- 172.21.15.139:0/4123995151 >> v1:172.21.15.139:6789/0 conn(0x7fbf3c06ab60 legacy=0x7fbf3c15f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf3b7fe640 1 -- 172.21.15.139:0/4123995151 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c06ff60 con 0x7fbf3c15b950 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.126+0000 7fbf3bfff640 1 --2- 172.21.15.139:0/4123995151 >> v2:172.21.15.150:3300/0 conn(0x7fbf3c0d0970 0x7fbf3c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.127+0000 7fbf3b7fe640 1 --2- 172.21.15.139:0/4123995151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf3c15b950 0x7fbf3c072a60 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c06bab0 tx=0x7fbf2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b289c4fd1ba32c75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.127+0000 7fbf3a7fc640 1 -- 172.21.15.139:0/4123995151 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c037070 con 0x7fbf3c15b950 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.127+0000 7fbf3a7fc640 1 -- 172.21.15.139:0/4123995151 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c032440 con 0x7fbf3c15b950 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.127+0000 7fbf3a7fc640 1 -- 172.21.15.139:0/4123995151 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c032810 con 0x7fbf3c15b950 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.128+0000 7fbf41d7e640 1 -- 172.21.15.139:0/4123995151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf3c15b950 msgr2=0x7fbf3c072a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.128+0000 7fbf41d7e640 1 --2- 172.21.15.139:0/4123995151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf3c15b950 0x7fbf3c072a60 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c06bab0 tx=0x7fbf2c02a270 comp rx=0 tx=0).stop 2024-04-25T10:01:28.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.128+0000 7fbf41d7e640 1 -- 172.21.15.139:0/4123995151 shutdown_connections 2024-04-25T10:01:28.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.128+0000 7fbf41d7e640 1 --2- 172.21.15.139:0/4123995151 >> v2:172.21.15.150:3300/0 conn(0x7fbf3c0d0970 0x7fbf3c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.128+0000 7fbf41d7e640 1 --2- 172.21.15.139:0/4123995151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf3c15b950 0x7fbf3c072a60 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.128+0000 7fbf41d7e640 1 -- 172.21.15.139:0/4123995151 >> 172.21.15.139:0/4123995151 conn(0x7fbf3c0b23f0 msgr2=0x7fbf3c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.128+0000 7fbf41d7e640 1 -- 172.21.15.139:0/4123995151 shutdown_connections 2024-04-25T10:01:28.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.128+0000 7fbf41d7e640 1 -- 172.21.15.139:0/4123995151 wait complete. 2024-04-25T10:01:28.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.129+0000 7fbf41d7e640 1 Processor -- start 2024-04-25T10:01:28.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.129+0000 7fbf41d7e640 1 -- start start 2024-04-25T10:01:28.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf41d7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf3c06ab60 0x7fbf3c1f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf41d7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf3c0d0970 0x7fbf3c15fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf41d7e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf3c15b950 0x7fbf3c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf41d7e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0cdce0 con 0x7fbf3c0d0970 2024-04-25T10:01:28.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf41d7e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0cda00 con 0x7fbf3c15b950 2024-04-25T10:01:28.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf41d7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0cdfc0 con 0x7fbf3c06ab60 2024-04-25T10:01:28.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf3affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf3c0d0970 0x7fbf3c15fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf3affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf3c0d0970 0x7fbf3c15fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55986/0 (socket says 172.21.15.139:55986) 2024-04-25T10:01:28.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf3affd640 1 -- 172.21.15.139:0/1674368138 learned_addr learned my addr 172.21.15.139:0/1674368138 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf3b7fe640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf3c06ab60 0x7fbf3c1f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf3bfff640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf3c15b950 0x7fbf3c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf3affd640 1 -- 172.21.15.139:0/1674368138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf3c06ab60 msgr2=0x7fbf3c1f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf3affd640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf3c06ab60 0x7fbf3c1f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf3affd640 1 -- 172.21.15.139:0/1674368138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf3c15b950 msgr2=0x7fbf3c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf3affd640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf3c15b950 0x7fbf3c15c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf3affd640 1 -- 172.21.15.139:0/1674368138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c15cd10 con 0x7fbf3c0d0970 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf3b7fe640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf3c06ab60 0x7fbf3c1f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf3affd640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf3c0d0970 0x7fbf3c15fe90 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000c400 tx=0x7fbf3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf23fff640 1 -- 172.21.15.139:0/1674368138 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300052d0 con 0x7fbf3c0d0970 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf23fff640 1 -- 172.21.15.139:0/1674368138 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf30005490 con 0x7fbf3c0d0970 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf41d7e640 1 -- 172.21.15.139:0/1674368138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c1f1ee0 con 0x7fbf3c0d0970 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.130+0000 7fbf23fff640 1 -- 172.21.15.139:0/1674368138 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30007260 con 0x7fbf3c0d0970 2024-04-25T10:01:28.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.131+0000 7fbf41d7e640 1 -- 172.21.15.139:0/1674368138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c1f24a0 con 0x7fbf3c0d0970 2024-04-25T10:01:28.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.131+0000 7fbf41d7e640 1 -- 172.21.15.139:0/1674368138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c0af8c0 con 0x7fbf3c0d0970 2024-04-25T10:01:28.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.132+0000 7fbf23fff640 1 -- 172.21.15.139:0/1674368138 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf30007400 con 0x7fbf3c0d0970 2024-04-25T10:01:28.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.132+0000 7fbf23fff640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf2806b4d0 0x7fbf2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.132+0000 7fbf23fff640 1 -- 172.21.15.139:0/1674368138 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbf300898a0 con 0x7fbf3c0d0970 2024-04-25T10:01:28.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.132+0000 7fbf3b7fe640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf2806b4d0 0x7fbf2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.133+0000 7fbf3b7fe640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf2806b4d0 0x7fbf2806d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c034000 tx=0x7fbf2c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.136+0000 7fbf23fff640 1 -- 172.21.15.139:0/1674368138 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf3c0af8c0 con 0x7fbf3c0d0970 2024-04-25T10:01:28.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.344+0000 7fbf41d7e640 1 -- 172.21.15.139:0/1674368138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf3c074520 con 0x7fbf3c0d0970 2024-04-25T10:01:28.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.345+0000 7fbf23fff640 1 -- 172.21.15.139:0/1674368138 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fbf30058590 con 0x7fbf3c0d0970 2024-04-25T10:01:28.349 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:28.349 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":69,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:15.497940+0000","modified":"2024-04-25T10:01:17.495730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8466":{"gid":8466,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6837/4074693991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":4074693991},{"type":"v1","addr":"172.21.15.139:6837","nonce":4074693991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:28.350 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 69 2024-04-25T10:01:28.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.347+0000 7fbf41d7e640 1 -- 172.21.15.139:0/1674368138 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf2806b4d0 msgr2=0x7fbf2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.347+0000 7fbf41d7e640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf2806b4d0 0x7fbf2806d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c034000 tx=0x7fbf2c02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:28.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.347+0000 7fbf41d7e640 1 -- 172.21.15.139:0/1674368138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf3c0d0970 msgr2=0x7fbf3c15fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.347+0000 7fbf41d7e640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf3c0d0970 0x7fbf3c15fe90 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000c400 tx=0x7fbf3000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.347+0000 7fbf41d7e640 1 -- 172.21.15.139:0/1674368138 shutdown_connections 2024-04-25T10:01:28.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.347+0000 7fbf41d7e640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf3c0d0970 0x7fbf3c15fe90 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.347+0000 7fbf41d7e640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf2806b4d0 0x7fbf2806d990 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.347+0000 7fbf41d7e640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf3c15b950 0x7fbf3c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.347+0000 7fbf41d7e640 1 --2- 172.21.15.139:0/1674368138 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf3c06ab60 0x7fbf3c1f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.347+0000 7fbf41d7e640 1 -- 172.21.15.139:0/1674368138 >> 172.21.15.139:0/1674368138 conn(0x7fbf3c0b23f0 msgr2=0x7fbf3c1576a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.347+0000 7fbf41d7e640 1 -- 172.21.15.139:0/1674368138 shutdown_connections 2024-04-25T10:01:28.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.347+0000 7fbf41d7e640 1 -- 172.21.15.139:0/1674368138 wait complete. 2024-04-25T10:01:28.362 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:01:28.481 INFO:teuthology.orchestra.run.smithi139.stdout:{"message":"","return_code":0} 2024-04-25T10:01:28.481 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:01:28.482 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:01:28.482 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:28.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:01:28.497 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:01:28.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:28.497 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:01:28.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:28.552 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:01:28.553 DEBUG:teuthology.orchestra.run.smithi139:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:01:28.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:28.596 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T10:01:28.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.694+0000 7f56f68c4640 1 Processor -- start 2024-04-25T10:01:28.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.694+0000 7f56f68c4640 1 -- start start 2024-04-25T10:01:28.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.694+0000 7f56f68c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56f01617e0 0x7f56f00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.694+0000 7f56f68c4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f56f00cec80 0x7f56f00d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.694+0000 7f56f68c4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f0161dc0 con 0x7f56f015dbf0 2024-04-25T10:01:28.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.694+0000 7f56f68c4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f00d1ee0 con 0x7f56f00cec80 2024-04-25T10:01:28.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.694+0000 7f56f68c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f01620a0 con 0x7f56f01617e0 2024-04-25T10:01:28.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.695+0000 7f56ef7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56f01617e0 0x7f56f00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.695+0000 7f56ef7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56f01617e0 0x7f56f00ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:39826/0 (socket says 172.21.15.139:39826) 2024-04-25T10:01:28.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.695+0000 7f56f4e61640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f56f00cec80 0x7f56f00d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.695+0000 7f56ef7fe640 1 -- 172.21.15.139:0/3056684977 learned_addr learned my addr 172.21.15.139:0/3056684977 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.695+0000 7f56effff640 1 --1- 172.21.15.139:0/3056684977 >> v1:172.21.15.150:6789/0 conn(0x7f56f015dbf0 0x7f56f00b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:50322/0 (socket says 172.21.15.139:50322) 2024-04-25T10:01:28.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.695+0000 7f56ef7fe640 1 -- 172.21.15.139:0/3056684977 >> v2:172.21.15.150:3300/0 conn(0x7f56f00cec80 msgr2=0x7f56f00d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.695+0000 7f56ef7fe640 1 --2- 172.21.15.139:0/3056684977 >> v2:172.21.15.150:3300/0 conn(0x7f56f00cec80 0x7f56f00d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.695+0000 7f56ef7fe640 1 -- 172.21.15.139:0/3056684977 >> v1:172.21.15.150:6789/0 conn(0x7f56f015dbf0 legacy=0x7f56f00b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.695+0000 7f56ef7fe640 1 -- 172.21.15.139:0/3056684977 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f0162380 con 0x7f56f01617e0 2024-04-25T10:01:28.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.695+0000 7f56f4e61640 1 --2- 172.21.15.139:0/3056684977 >> v2:172.21.15.150:3300/0 conn(0x7f56f00cec80 0x7f56f00d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:28.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.696+0000 7f56ef7fe640 1 --2- 172.21.15.139:0/3056684977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56f01617e0 0x7f56f00ce740 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009870 tx=0x7f56d802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87dfd94ad2e845f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.696+0000 7f56eeffd640 1 -- 172.21.15.139:0/3056684977 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8037070 con 0x7f56f01617e0 2024-04-25T10:01:28.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.696+0000 7f56eeffd640 1 -- 172.21.15.139:0/3056684977 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d8032440 con 0x7f56f01617e0 2024-04-25T10:01:28.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.696+0000 7f56eeffd640 1 -- 172.21.15.139:0/3056684977 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8032810 con 0x7f56f01617e0 2024-04-25T10:01:28.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.697+0000 7f56f68c4640 1 -- 172.21.15.139:0/3056684977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56f01617e0 msgr2=0x7f56f00ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.697+0000 7f56f68c4640 1 --2- 172.21.15.139:0/3056684977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56f01617e0 0x7f56f00ce740 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009870 tx=0x7f56d802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.697+0000 7f56f68c4640 1 -- 172.21.15.139:0/3056684977 shutdown_connections 2024-04-25T10:01:28.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.697+0000 7f56f68c4640 1 --2- 172.21.15.139:0/3056684977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56f01617e0 0x7f56f00ce740 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.697+0000 7f56f68c4640 1 --2- 172.21.15.139:0/3056684977 >> v2:172.21.15.150:3300/0 conn(0x7f56f00cec80 0x7f56f00d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.697+0000 7f56f68c4640 1 -- 172.21.15.139:0/3056684977 >> 172.21.15.139:0/3056684977 conn(0x7f56f00b5480 msgr2=0x7f56f00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.697+0000 7f56f68c4640 1 -- 172.21.15.139:0/3056684977 shutdown_connections 2024-04-25T10:01:28.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.697+0000 7f56f68c4640 1 -- 172.21.15.139:0/3056684977 wait complete. 2024-04-25T10:01:28.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.698+0000 7f56f68c4640 1 Processor -- start 2024-04-25T10:01:28.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.698+0000 7f56f68c4640 1 -- start start 2024-04-25T10:01:28.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56f68c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56f00cec80 0x7f56f00c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56f68c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56f015dbf0 0x7f56f0156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56f68c4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56f01617e0 0x7f56f0157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56f68c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f015dfd0 con 0x7f56f015dbf0 2024-04-25T10:01:28.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56f68c4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00d28b0 con 0x7f56f01617e0 2024-04-25T10:01:28.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56f68c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f0153020 con 0x7f56f00cec80 2024-04-25T10:01:28.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56ef7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56f015dbf0 0x7f56f0156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56ef7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56f015dbf0 0x7f56f0156e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55992/0 (socket says 172.21.15.139:55992) 2024-04-25T10:01:28.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56ef7fe640 1 -- 172.21.15.139:0/3399776252 learned_addr learned my addr 172.21.15.139:0/3399776252 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56effff640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56f00cec80 0x7f56f00c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56f4e61640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56f01617e0 0x7f56f0157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56ef7fe640 1 -- 172.21.15.139:0/3399776252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56f00cec80 msgr2=0x7f56f00c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56ef7fe640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56f00cec80 0x7f56f00c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56ef7fe640 1 -- 172.21.15.139:0/3399776252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56f01617e0 msgr2=0x7f56f0157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56ef7fe640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56f01617e0 0x7f56f0157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56ef7fe640 1 -- 172.21.15.139:0/3399776252 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00c9b10 con 0x7f56f015dbf0 2024-04-25T10:01:28.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.699+0000 7f56f4e61640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56f01617e0 0x7f56f0157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:28.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.700+0000 7f56ef7fe640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56f015dbf0 0x7f56f0156e00 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009870 tx=0x7f56d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.700+0000 7f56ecff9640 1 -- 172.21.15.139:0/3399776252 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8041070 con 0x7f56f015dbf0 2024-04-25T10:01:28.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.700+0000 7f56ecff9640 1 -- 172.21.15.139:0/3399776252 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d803c890 con 0x7f56f015dbf0 2024-04-25T10:01:28.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.700+0000 7f56f68c4640 1 -- 172.21.15.139:0/3399776252 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f0160a20 con 0x7f56f015dbf0 2024-04-25T10:01:28.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.700+0000 7f56ecff9640 1 -- 172.21.15.139:0/3399776252 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8037040 con 0x7f56f015dbf0 2024-04-25T10:01:28.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.700+0000 7f56f68c4640 1 -- 172.21.15.139:0/3399776252 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f0152cb0 con 0x7f56f015dbf0 2024-04-25T10:01:28.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.701+0000 7f56ecff9640 1 -- 172.21.15.139:0/3399776252 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56d8044070 con 0x7f56f015dbf0 2024-04-25T10:01:28.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.702+0000 7f56ecff9640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56d006b500 0x7f56d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.702+0000 7f56effff640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56d006b500 0x7f56d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.702+0000 7f56ecff9640 1 -- 172.21.15.139:0/3399776252 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56d80aad70 con 0x7f56f015dbf0 2024-04-25T10:01:28.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.703+0000 7f56effff640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56d006b500 0x7f56d006d9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f56e0000f60 tx=0x7f56e0007cc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.703+0000 7f56f95c6740 1 Processor -- start 2024-04-25T10:01:28.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.703+0000 7f56f95c6740 1 -- start start 2024-04-25T10:01:28.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f95c6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c3abb0 0x5556f9c3af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f95c6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c35830 0x5556f9c397b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f95c6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c39cf0 0x5556f9c3a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f95c6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f9b9eac0 con 0x5556f9c39cf0 2024-04-25T10:01:28.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f95c6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f9b47d80 con 0x5556f9c3abb0 2024-04-25T10:01:28.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f95c6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556f9b9ab60 con 0x5556f9c35830 2024-04-25T10:01:28.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f4e61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c39cf0 0x5556f9c3a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f4e61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c39cf0 0x5556f9c3a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56000/0 (socket says 172.21.15.139:56000) 2024-04-25T10:01:28.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f4e61640 1 -- 172.21.15.139:0/3828724714 learned_addr learned my addr 172.21.15.139:0/3828724714 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56ef7fe640 1 --2- 172.21.15.139:0/3828724714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c35830 0x5556f9c397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56effff640 1 --2- 172.21.15.139:0/3828724714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c3abb0 0x5556f9c3af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f4e61640 1 -- 172.21.15.139:0/3828724714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c35830 msgr2=0x5556f9c397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f4e61640 1 --2- 172.21.15.139:0/3828724714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c35830 0x5556f9c397b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f4e61640 1 -- 172.21.15.139:0/3828724714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c3abb0 msgr2=0x5556f9c3af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f4e61640 1 --2- 172.21.15.139:0/3828724714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c3abb0 0x5556f9c3af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f4e61640 1 -- 172.21.15.139:0/3828724714 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556f997b7e0 con 0x5556f9c39cf0 2024-04-25T10:01:28.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56f4e61640 1 --2- 172.21.15.139:0/3828724714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c39cf0 0x5556f9c3a130 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f56e80074f0 tx=0x7f56e8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de991f9fb3757912 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56e5ffb640 1 -- 172.21.15.139:0/3828724714 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e80147e0 con 0x5556f9c39cf0 2024-04-25T10:01:28.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56e5ffb640 1 -- 172.21.15.139:0/3828724714 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e801a070 con 0x5556f9c39cf0 2024-04-25T10:01:28.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.704+0000 7f56e5ffb640 1 -- 172.21.15.139:0/3828724714 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8014ba0 con 0x5556f9c39cf0 2024-04-25T10:01:28.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.705+0000 7f56f95c6740 1 -- 172.21.15.139:0/3828724714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c39cf0 msgr2=0x5556f9c3a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.705+0000 7f56f95c6740 1 --2- 172.21.15.139:0/3828724714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c39cf0 0x5556f9c3a130 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f56e80074f0 tx=0x7f56e8009750 comp rx=0 tx=0).stop 2024-04-25T10:01:28.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.705+0000 7f56f95c6740 1 -- 172.21.15.139:0/3828724714 shutdown_connections 2024-04-25T10:01:28.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.705+0000 7f56f95c6740 1 --2- 172.21.15.139:0/3828724714 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c39cf0 0x5556f9c3a130 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.705+0000 7f56f95c6740 1 --2- 172.21.15.139:0/3828724714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c35830 0x5556f9c397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.705+0000 7f56f95c6740 1 --2- 172.21.15.139:0/3828724714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c3abb0 0x5556f9c3af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.705+0000 7f56f95c6740 1 -- 172.21.15.139:0/3828724714 >> 172.21.15.139:0/3828724714 conn(0x5556f9c37cb0 msgr2=0x5556f9c80770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.705+0000 7f56f95c6740 1 -- 172.21.15.139:0/3828724714 shutdown_connections 2024-04-25T10:01:28.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.705+0000 7f56f95c6740 1 -- 172.21.15.139:0/3828724714 wait complete. 2024-04-25T10:01:28.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.706+0000 7f56f95c6740 1 -- start start 2024-04-25T10:01:28.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.706+0000 7f56f95c6740 1 Processor -- start 2024-04-25T10:01:28.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.706+0000 7f56f95c6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c39cf0 0x5556f9c91430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.706+0000 7f56f95c6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c3abb0 0x5556f9c5cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.706+0000 7f56f95c6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 0x5556f9c882a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.706+0000 7f56f95c6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f9c66740 con 0x5556f9c93f30 2024-04-25T10:01:28.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.706+0000 7f56f95c6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f9b3d530 con 0x5556f9c39cf0 2024-04-25T10:01:28.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.706+0000 7f56f95c6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556f9c65bc0 con 0x5556f9c3abb0 2024-04-25T10:01:28.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f95c6740 10 client.0 _ll_register_callbacks cb 0x5556f9b95130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:28.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f4e61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 0x5556f9c882a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f4e61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 0x5556f9c882a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56008/0 (socket says 172.21.15.139:56008) 2024-04-25T10:01:28.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f4e61640 1 -- 172.21.15.139:0/1683576964 learned_addr learned my addr 172.21.15.139:0/1683576964 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56effff640 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c39cf0 0x5556f9c91430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56ef7fe640 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c3abb0 0x5556f9c5cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f4e61640 1 -- 172.21.15.139:0/1683576964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c3abb0 msgr2=0x5556f9c5cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f4e61640 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c3abb0 0x5556f9c5cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f4e61640 1 -- 172.21.15.139:0/1683576964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c39cf0 msgr2=0x5556f9c91430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f4e61640 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c39cf0 0x5556f9c91430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f4e61640 1 -- 172.21.15.139:0/1683576964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556f99dbf70 con 0x5556f9c93f30 2024-04-25T10:01:28.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56ef7fe640 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c3abb0 0x5556f9c5cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:28.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56effff640 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c39cf0 0x5556f9c91430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:28.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f4e61640 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 0x5556f9c882a0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f56e8014cc0 tx=0x7f56e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472ea577f2298c4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:01:28.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56ceffd640 1 -- 172.21.15.139:0/1683576964 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8015880 con 0x5556f9c93f30 2024-04-25T10:01:28.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56ceffd640 1 -- 172.21.15.139:0/1683576964 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56e8015a20 con 0x5556f9c93f30 2024-04-25T10:01:28.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f95c6740 1 -- 172.21.15.139:0/1683576964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56e801a070 con 0x5556f9c93f30 2024-04-25T10:01:28.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56ceffd640 1 -- 172.21.15.139:0/1683576964 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56e8032050 con 0x5556f9c93f30 2024-04-25T10:01:28.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f95c6740 10 client.8745 fetch_fsmap learned FSMap version 69 2024-04-25T10:01:28.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.707+0000 7f56f95c6740 1 -- 172.21.15.139:0/1683576964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5556f9873170 con 0x5556f9c93f30 2024-04-25T10:01:28.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.708+0000 7f56ceffd640 1 -- 172.21.15.139:0/1683576964 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e801a070 con 0x5556f9c93f30 2024-04-25T10:01:28.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.708+0000 7f56ceffd640 1 -- 172.21.15.139:0/1683576964 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f56e8035020 con 0x5556f9c93f30 2024-04-25T10:01:28.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.708+0000 7f56f95c6740 10 client.8745 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:01:28.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.708+0000 7f56f95c6740 10 client.8745 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:01:28.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.708+0000 7f56f95c6740 20 client.8745 populate_metadata read hostname 'smithi139' 2024-04-25T10:01:28.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.708+0000 7f56f95c6740 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x5556f9c5fbc0 0x5556f9c5de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.708+0000 7f56f95c6740 4 client.8745 mds_command: new command op to 8466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:28.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.708+0000 7f56f95c6740 1 -- 172.21.15.139:0/1683576964 --> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556f9bac000 con 0x5556f9c5fbc0 2024-04-25T10:01:28.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.708+0000 7f56effff640 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x5556f9c5fbc0 0x5556f9c5de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.708+0000 7f56ceffd640 10 client.8745 ms_handle_connect on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:28.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.708+0000 7f56effff640 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x5556f9c5fbc0 0x5556f9c5de40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729719e2f1ee6267 server_cookie=4ba9e8e8d14da0ea in_seq=0 out_seq=0 2024-04-25T10:01:28.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56ceffd640 1 -- 172.21.15.139:0/1683576964 <== mds.0 v2:172.21.15.139:6835/4074693991 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5556f9bac000 con 0x5556f9c5fbc0 2024-04-25T10:01:28.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56ceffd640 10 client.8745 handle_command_reply: tid=0 2024-04-25T10:01:28.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56f95c6740 1 client.8745 shutdown 2024-04-25T10:01:28.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56f95c6740 1 -- 172.21.15.139:0/1683576964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 msgr2=0x5556f9c882a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56f95c6740 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 0x5556f9c882a0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f56e8014cc0 tx=0x7f56e8004ab0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56f95c6740 1 -- 172.21.15.139:0/1683576964 shutdown_connections 2024-04-25T10:01:28.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56f95c6740 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x5556f9c5fbc0 0x5556f9c5de40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56f95c6740 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 0x5556f9c882a0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56f95c6740 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c3abb0 0x5556f9c5cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56ceffd640 0 client.8745 ms_handle_reset on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:28.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56f95c6740 1 --2- 172.21.15.139:0/1683576964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c39cf0 0x5556f9c91430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56f95c6740 1 -- 172.21.15.139:0/1683576964 >> 172.21.15.139:0/1683576964 conn(0x5556f9c35830 msgr2=0x5556f9c34f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56f95c6740 1 -- 172.21.15.139:0/1683576964 shutdown_connections 2024-04-25T10:01:28.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56f95c6740 1 -- 172.21.15.139:0/1683576964 wait complete. 2024-04-25T10:01:28.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.709+0000 7f56f95c6740 20 client.8745 trim_cache size 0 max 16384 2024-04-25T10:01:28.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.712+0000 7fce5084a640 1 Processor -- start 2024-04-25T10:01:28.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.712+0000 7fce5084a640 1 -- start start 2024-04-25T10:01:28.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.713+0000 7fce5084a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce4c15b860 0x7fce4c15e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.713+0000 7fce5084a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fce4c0b9430 0x7fce4c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.713+0000 7fce5084a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fce4c0b5980 0x7fce4c06bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.713+0000 7fce5084a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4c0b6980 con 0x7fce4c0b9430 2024-04-25T10:01:28.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.713+0000 7fce5084a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4c15f090 con 0x7fce4c0b5980 2024-04-25T10:01:28.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.713+0000 7fce5084a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0b5db0 con 0x7fce4c15b860 2024-04-25T10:01:28.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.713+0000 7fce49d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fce4c0b9430 0x7fce4c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.713+0000 7fce49d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fce4c0b9430 0x7fce4c0b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56022/0 (socket says 172.21.15.139:56022) 2024-04-25T10:01:28.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.713+0000 7fce49d9b640 1 -- 172.21.15.139:0/1980814698 learned_addr learned my addr 172.21.15.139:0/1980814698 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.713+0000 7fce4a59c640 1 --2- 172.21.15.139:0/1980814698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce4c15b860 0x7fce4c15e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.713+0000 7fce4ad9d640 1 --2- 172.21.15.139:0/1980814698 >> v2:172.21.15.150:3300/0 conn(0x7fce4c0b5980 0x7fce4c06bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.714+0000 7fce49d9b640 1 -- 172.21.15.139:0/1980814698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce4c15b860 msgr2=0x7fce4c15e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.714+0000 7fce49d9b640 1 --2- 172.21.15.139:0/1980814698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce4c15b860 0x7fce4c15e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.714+0000 7fce49d9b640 1 -- 172.21.15.139:0/1980814698 >> v2:172.21.15.150:3300/0 conn(0x7fce4c0b5980 msgr2=0x7fce4c06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.714+0000 7fce49d9b640 1 --2- 172.21.15.139:0/1980814698 >> v2:172.21.15.150:3300/0 conn(0x7fce4c0b5980 0x7fce4c06bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.714+0000 7fce49d9b640 1 -- 172.21.15.139:0/1980814698 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c15e7f0 con 0x7fce4c0b9430 2024-04-25T10:01:28.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.714+0000 7fce49d9b640 1 --2- 172.21.15.139:0/1980814698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce4c0b9430 0x7fce4c0b5440 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0c7420 tx=0x7fce4002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf3f092d2a288af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.714+0000 7fce4959a640 1 -- 172.21.15.139:0/1980814698 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40037070 con 0x7fce4c0b9430 2024-04-25T10:01:28.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.714+0000 7fce4959a640 1 -- 172.21.15.139:0/1980814698 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce40032440 con 0x7fce4c0b9430 2024-04-25T10:01:28.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.714+0000 7fce4959a640 1 -- 172.21.15.139:0/1980814698 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40032810 con 0x7fce4c0b9430 2024-04-25T10:01:28.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.715+0000 7fce5084a640 1 -- 172.21.15.139:0/1980814698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce4c0b9430 msgr2=0x7fce4c0b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.715+0000 7fce5084a640 1 --2- 172.21.15.139:0/1980814698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce4c0b9430 0x7fce4c0b5440 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0c7420 tx=0x7fce4002a210 comp rx=0 tx=0).stop 2024-04-25T10:01:28.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.715+0000 7fce5084a640 1 -- 172.21.15.139:0/1980814698 shutdown_connections 2024-04-25T10:01:28.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.715+0000 7fce5084a640 1 --2- 172.21.15.139:0/1980814698 >> v2:172.21.15.150:3300/0 conn(0x7fce4c0b5980 0x7fce4c06bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.715+0000 7fce5084a640 1 --2- 172.21.15.139:0/1980814698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce4c0b9430 0x7fce4c0b5440 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.715+0000 7fce5084a640 1 --2- 172.21.15.139:0/1980814698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce4c15b860 0x7fce4c15e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.715+0000 7fce5084a640 1 -- 172.21.15.139:0/1980814698 >> 172.21.15.139:0/1980814698 conn(0x7fce4c0d0620 msgr2=0x7fce4c0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.715+0000 7fce5084a640 1 -- 172.21.15.139:0/1980814698 shutdown_connections 2024-04-25T10:01:28.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.715+0000 7fce5084a640 1 -- 172.21.15.139:0/1980814698 wait complete. 2024-04-25T10:01:28.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.716+0000 7fce5084a640 1 Processor -- start 2024-04-25T10:01:28.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.717+0000 7fce5084a640 1 -- start start 2024-04-25T10:01:28.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.717+0000 7fce5084a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce4c0b5980 0x7fce4c1494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.717+0000 7fce5084a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce4c0b9430 0x7fce4c0b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.717+0000 7fce5084a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce4c15b860 0x7fce4c149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.717+0000 7fce5084a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c15edb0 con 0x7fce4c0b5980 2024-04-25T10:01:28.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.717+0000 7fce5084a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c15ead0 con 0x7fce4c15b860 2024-04-25T10:01:28.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.717+0000 7fce5084a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0d1df0 con 0x7fce4c0b9430 2024-04-25T10:01:28.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.717+0000 7fce4a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce4c0b5980 0x7fce4c1494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.717+0000 7fce4a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce4c0b5980 0x7fce4c1494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56034/0 (socket says 172.21.15.139:56034) 2024-04-25T10:01:28.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.717+0000 7fce4a59c640 1 -- 172.21.15.139:0/3869143164 learned_addr learned my addr 172.21.15.139:0/3869143164 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.718+0000 7fce49d9b640 1 --2- 172.21.15.139:0/3869143164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce4c0b9430 0x7fce4c0b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.718+0000 7fce4a59c640 1 -- 172.21.15.139:0/3869143164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce4c0b9430 msgr2=0x7fce4c0b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.718+0000 7fce4a59c640 1 --2- 172.21.15.139:0/3869143164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce4c0b9430 0x7fce4c0b7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.718+0000 7fce4a59c640 1 -- 172.21.15.139:0/3869143164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce4c15b860 msgr2=0x7fce4c149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.718+0000 7fce4a59c640 1 --2- 172.21.15.139:0/3869143164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce4c15b860 0x7fce4c149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.718+0000 7fce4a59c640 1 -- 172.21.15.139:0/3869143164 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0b6ef0 con 0x7fce4c0b5980 2024-04-25T10:01:28.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.718+0000 7fce4a59c640 1 --2- 172.21.15.139:0/3869143164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce4c0b5980 0x7fce4c1494d0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00d5f0 tx=0x7fce3c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.718+0000 7fce3affd640 1 -- 172.21.15.139:0/3869143164 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0125f0 con 0x7fce4c0b5980 2024-04-25T10:01:28.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.718+0000 7fce3affd640 1 -- 172.21.15.139:0/3869143164 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c019070 con 0x7fce4c0b5980 2024-04-25T10:01:28.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.718+0000 7fce3affd640 1 -- 172.21.15.139:0/3869143164 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0129d0 con 0x7fce4c0b5980 2024-04-25T10:01:28.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.718+0000 7fce5084a640 1 -- 172.21.15.139:0/3869143164 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c0b71d0 con 0x7fce4c0b5980 2024-04-25T10:01:28.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.718+0000 7fce5084a640 1 -- 172.21.15.139:0/3869143164 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c0b7790 con 0x7fce4c0b5980 2024-04-25T10:01:28.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.719+0000 7fce5084a640 1 -- 172.21.15.139:0/3869143164 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fce4c074520 con 0x7fce4c0b5980 2024-04-25T10:01:28.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.719+0000 7fce3affd640 1 -- 172.21.15.139:0/3869143164 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce3c012b70 con 0x7fce4c0b5980 2024-04-25T10:01:28.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.720+0000 7fce3affd640 1 --2- 172.21.15.139:0/3869143164 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce2c06b4d0 0x7fce2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.720+0000 7fce3affd640 1 -- 172.21.15.139:0/3869143164 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fce3c08b240 con 0x7fce4c0b5980 2024-04-25T10:01:28.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.720+0000 7fce49d9b640 1 --2- 172.21.15.139:0/3869143164 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce2c06b4d0 0x7fce2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.720+0000 7fce3affd640 1 -- 172.21.15.139:0/3869143164 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+501 (secure 0 0 0) 0x7fce3c08b6f0 con 0x7fce4c0b5980 2024-04-25T10:01:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.720+0000 7fce5084a640 1 -- 172.21.15.139:0/3869143164 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fce4c0d0c50 con 0x7fce4c0b5980 2024-04-25T10:01:28.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.720+0000 7fce49d9b640 1 --2- 172.21.15.139:0/3869143164 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fce2c06b4d0 0x7fce2c06d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fce40034000 tx=0x7fce4002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.725+0000 7f56f95c6740 1 Processor -- start 2024-04-25T10:01:28.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.725+0000 7f56f95c6740 1 -- start start 2024-04-25T10:01:28.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56f95c6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c39cf0 0x5556f9c35060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56f95c6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c3abb0 0x5556f9c80aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56f95c6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c93f30 0x5556f9c3c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56f95c6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f0200570 con 0x5556f9c3abb0 2024-04-25T10:01:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56f95c6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f01ff790 con 0x5556f9c93f30 2024-04-25T10:01:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56f95c6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f01f8090 con 0x5556f9c39cf0 2024-04-25T10:01:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56ef7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c3abb0 0x5556f9c80aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56ef7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c3abb0 0x5556f9c80aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56038/0 (socket says 172.21.15.139:56038) 2024-04-25T10:01:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56ef7fe640 1 -- 172.21.15.139:0/1647769871 learned_addr learned my addr 172.21.15.139:0/1647769871 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56ef7fe640 1 -- 172.21.15.139:0/1647769871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c39cf0 msgr2=0x5556f9c35060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56ef7fe640 1 --2- 172.21.15.139:0/1647769871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c39cf0 0x5556f9c35060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56ef7fe640 1 -- 172.21.15.139:0/1647769871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c93f30 msgr2=0x5556f9c3c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56ef7fe640 1 --2- 172.21.15.139:0/1647769871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c93f30 0x5556f9c3c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.726+0000 7f56ef7fe640 1 -- 172.21.15.139:0/1647769871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556f9ca3690 con 0x5556f9c3abb0 2024-04-25T10:01:28.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.727+0000 7f56ef7fe640 1 --2- 172.21.15.139:0/1647769871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c3abb0 0x5556f9c80aa0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f56d802a200 tx=0x7f56d8079170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bf03f1de3d67e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.727+0000 7f56e57fa640 1 -- 172.21.15.139:0/1647769871 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d804b070 con 0x5556f9c3abb0 2024-04-25T10:01:28.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.727+0000 7f56e57fa640 1 -- 172.21.15.139:0/1647769871 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d80799e0 con 0x5556f9c3abb0 2024-04-25T10:01:28.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.727+0000 7f56e57fa640 1 -- 172.21.15.139:0/1647769871 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d80792e0 con 0x5556f9c3abb0 2024-04-25T10:01:28.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.727+0000 7f56f95c6740 1 -- 172.21.15.139:0/1647769871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c3abb0 msgr2=0x5556f9c80aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.727+0000 7f56f95c6740 1 --2- 172.21.15.139:0/1647769871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c3abb0 0x5556f9c80aa0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f56d802a200 tx=0x7f56d8079170 comp rx=0 tx=0).stop 2024-04-25T10:01:28.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.727+0000 7f56f95c6740 1 -- 172.21.15.139:0/1647769871 shutdown_connections 2024-04-25T10:01:28.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.727+0000 7f56f95c6740 1 --2- 172.21.15.139:0/1647769871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c3abb0 0x5556f9c80aa0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.727+0000 7f56f95c6740 1 --2- 172.21.15.139:0/1647769871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c93f30 0x5556f9c3c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.727+0000 7f56f95c6740 1 --2- 172.21.15.139:0/1647769871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c39cf0 0x5556f9c35060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.727+0000 7f56f95c6740 1 -- 172.21.15.139:0/1647769871 >> 172.21.15.139:0/1647769871 conn(0x5556f9c35830 msgr2=0x5556f9c5c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.728+0000 7f56f95c6740 1 -- 172.21.15.139:0/1647769871 shutdown_connections 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.728+0000 7f56f95c6740 1 -- 172.21.15.139:0/1647769871 wait complete. 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.728+0000 7f56f95c6740 1 -- start start 2024-04-25T10:01:28.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.728+0000 7f56f95c6740 1 Processor -- start 2024-04-25T10:01:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f95c6740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c39cf0 0x5556f9c65a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f95c6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c3abb0 0x5556f9ce1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f95c6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 0x5556f9ce1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f95c6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f9c398f0 con 0x5556f9c93f30 2024-04-25T10:01:28.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f95c6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f0153fe0 con 0x5556f9c3abb0 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f95c6740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556f9c39b40 con 0x5556f9c39cf0 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f95c6740 10 client.0 _ll_register_callbacks cb 0x7f56e8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f4e61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 0x5556f9ce1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f4e61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 0x5556f9ce1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56046/0 (socket says 172.21.15.139:56046) 2024-04-25T10:01:28.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f4e61640 1 -- 172.21.15.139:0/2079977069 learned_addr learned my addr 172.21.15.139:0/2079977069 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56effff640 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c39cf0 0x5556f9c65a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56ef7fe640 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c3abb0 0x5556f9ce1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f4e61640 1 -- 172.21.15.139:0/2079977069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c39cf0 msgr2=0x5556f9c65a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f4e61640 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c39cf0 0x5556f9c65a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f4e61640 1 -- 172.21.15.139:0/2079977069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c3abb0 msgr2=0x5556f9ce1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f4e61640 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c3abb0 0x5556f9ce1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.729+0000 7f56f4e61640 1 -- 172.21.15.139:0/2079977069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556f9c84060 con 0x5556f9c93f30 2024-04-25T10:01:28.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56f4e61640 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 0x5556f9ce1750 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f56e8026000 tx=0x7f56e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368e2f3b8745d4f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:01:28.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56ceffd640 1 -- 172.21.15.139:0/2079977069 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e803b060 con 0x5556f9c93f30 2024-04-25T10:01:28.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56ceffd640 1 -- 172.21.15.139:0/2079977069 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56e802a890 con 0x5556f9c93f30 2024-04-25T10:01:28.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56f95c6740 1 -- 172.21.15.139:0/2079977069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56d80799e0 con 0x5556f9c93f30 2024-04-25T10:01:28.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56ceffd640 1 -- 172.21.15.139:0/2079977069 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56e8041050 con 0x5556f9c93f30 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56f95c6740 10 client.8757 fetch_fsmap learned FSMap version 69 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56f95c6740 1 -- 172.21.15.139:0/2079977069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5556f9c83280 con 0x5556f9c93f30 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56ceffd640 1 -- 172.21.15.139:0/2079977069 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d80799e0 con 0x5556f9c93f30 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56ceffd640 1 -- 172.21.15.139:0/2079977069 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f56e8044020 con 0x5556f9c93f30 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56f95c6740 10 client.8757 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:01:28.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56f95c6740 10 client.8757 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56f95c6740 20 client.8757 populate_metadata read hostname 'smithi139' 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56f95c6740 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x5556f9c96f30 0x5556f9c5ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56f95c6740 4 client.8757 mds_command: new command op to 8466 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56f95c6740 1 -- 172.21.15.139:0/2079977069 --> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x5556f9ba6350 con 0x5556f9c96f30 2024-04-25T10:01:28.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56effff640 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x5556f9c96f30 0x5556f9c5ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56ceffd640 10 client.8757 ms_handle_connect on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.730+0000 7f56effff640 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x5556f9c96f30 0x5556f9c5ef90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11f67b405287d07 server_cookie=b4517e88d35174c1 in_seq=0 out_seq=0 2024-04-25T10:01:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56ceffd640 1 -- 172.21.15.139:0/2079977069 <== mds.0 v2:172.21.15.139:6835/4074693991 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5556f9ba6350 con 0x5556f9c96f30 2024-04-25T10:01:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56ceffd640 10 client.8757 handle_command_reply: tid=0 2024-04-25T10:01:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56f95c6740 1 client.8757 shutdown 2024-04-25T10:01:28.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56f95c6740 1 -- 172.21.15.139:0/2079977069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 msgr2=0x5556f9ce1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56f95c6740 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 0x5556f9ce1750 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f56e8026000 tx=0x7f56e8004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56f95c6740 1 -- 172.21.15.139:0/2079977069 shutdown_connections 2024-04-25T10:01:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56f95c6740 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x5556f9c96f30 0x5556f9c5ef90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56f95c6740 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5556f9c93f30 0x5556f9ce1750 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56f95c6740 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556f9c3abb0 0x5556f9ce1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56f95c6740 1 --2- 172.21.15.139:0/2079977069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5556f9c39cf0 0x5556f9c65a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56ceffd640 0 client.8757 ms_handle_reset on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56f95c6740 1 -- 172.21.15.139:0/2079977069 >> 172.21.15.139:0/2079977069 conn(0x5556f9c35830 msgr2=0x5556f9c35060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56f95c6740 1 -- 172.21.15.139:0/2079977069 shutdown_connections 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56f95c6740 1 -- 172.21.15.139:0/2079977069 wait complete. 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.731+0000 7f56f95c6740 20 client.8757 trim_cache size 0 max 16384 2024-04-25T10:01:28.780 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:01:28.780 INFO:teuthology.orchestra.run.smithi139.stdout: "return_code": 0, 2024-04-25T10:01:28.780 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_tag": "601dbbf1-f22e-424d-a1e4-89c8be56a391", 2024-04-25T10:01:28.780 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": "asynchronous" 2024-04-25T10:01:28.780 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.732+0000 7f56f68c4640 1 -- 172.21.15.139:0/3399776252 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56d006b500 msgr2=0x7f56d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.732+0000 7f56f68c4640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56d006b500 0x7f56d006d9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f56e0000f60 tx=0x7f56e0007cc0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.732+0000 7f56f68c4640 1 -- 172.21.15.139:0/3399776252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56f015dbf0 msgr2=0x7f56f0156e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.732+0000 7f56f68c4640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56f015dbf0 0x7f56f0156e00 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009870 tx=0x7f56d80046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.732+0000 7f56f68c4640 1 -- 172.21.15.139:0/3399776252 shutdown_connections 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.732+0000 7f56f68c4640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56f015dbf0 0x7f56f0156e00 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.732+0000 7f56f68c4640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f56d006b500 0x7f56d006d9c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.732+0000 7f56f68c4640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56f01617e0 0x7f56f0157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.732+0000 7f56f68c4640 1 --2- 172.21.15.139:0/3399776252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56f00cec80 0x7f56f00c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.732+0000 7f56f68c4640 1 -- 172.21.15.139:0/3399776252 >> 172.21.15.139:0/3399776252 conn(0x7f56f00b5480 msgr2=0x7f56f00d2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.732+0000 7f56f68c4640 1 -- 172.21.15.139:0/3399776252 shutdown_connections 2024-04-25T10:01:28.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.732+0000 7f56f68c4640 1 -- 172.21.15.139:0/3399776252 wait complete. 2024-04-25T10:01:28.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:28.783 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:01:28.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.878+0000 7fbb9b09e640 1 Processor -- start 2024-04-25T10:01:28.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.878+0000 7fbb9b09e640 1 -- start start 2024-04-25T10:01:28.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.879+0000 7fbb9b09e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbb940b5620 0x7fbb940b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.879+0000 7fbb9b09e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb940b5a20 con 0x7fbb9415a870 2024-04-25T10:01:28.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.879+0000 7fbb9b09e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb940d0d40 con 0x7fbb94070970 2024-04-25T10:01:28.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.879+0000 7fbb9b09e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb940d1020 con 0x7fbb940b5620 2024-04-25T10:01:28.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.879+0000 7fbb98e3a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbb9415a870 0x7fbb94070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41920/0 (socket says 172.21.15.139:41920) 2024-04-25T10:01:28.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.879+0000 7fbb98e3a640 1 -- 172.21.15.139:0/3543905887 learned_addr learned my addr 172.21.15.139:0/3543905887 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:28.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.879+0000 7fbb9963b640 1 --2- 172.21.15.139:0/3543905887 >> v2:172.21.15.150:3300/0 conn(0x7fbb940b5620 0x7fbb940b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4086937067 0 0) 0x7fbb940b5a20 con 0x7fbb9415a870 2024-04-25T10:01:28.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb7c003520 con 0x7fbb9415a870 2024-04-25T10:01:28.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4249807323 0 0) 0x7fbb7c003520 con 0x7fbb9415a870 2024-04-25T10:01:28.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 >> v2:172.21.15.150:3300/0 conn(0x7fbb940b5620 msgr2=0x7fbb940b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 --2- 172.21.15.139:0/3543905887 >> v2:172.21.15.150:3300/0 conn(0x7fbb940b5620 0x7fbb940b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 >> v1:172.21.15.150:6789/0 conn(0x7fbb94070970 legacy=0x7fbb940ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb940b4b40 con 0x7fbb9415a870 2024-04-25T10:01:28.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fbb84001670 con 0x7fbb9415a870 2024-04-25T10:01:28.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 >> v1:172.21.15.139:6789/0 conn(0x7fbb9415a870 legacy=0x7fbb94070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 --2- 172.21.15.139:0/3543905887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 0x7fbb7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 --2- 172.21.15.139:0/3543905887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb7c004800 0x7fbb7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 --2- 172.21.15.139:0/3543905887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb7c0051c0 0x7fbb7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940d1020 con 0x7fbb7c0039b0 2024-04-25T10:01:28.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c00c160 con 0x7fbb7c004800 2024-04-25T10:01:28.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c00e570 con 0x7fbb7c0051c0 2024-04-25T10:01:28.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/3543905887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 0x7fbb7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb93fff640 1 --2- 172.21.15.139:0/3543905887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb7c004800 0x7fbb7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb9963b640 1 --2- 172.21.15.139:0/3543905887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb7c0051c0 0x7fbb7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb98e3a640 1 -- 172.21.15.139:0/3543905887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb7c0051c0 msgr2=0x7fbb7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/3543905887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb7c0051c0 0x7fbb7c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb98e3a640 1 -- 172.21.15.139:0/3543905887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb7c004800 msgr2=0x7fbb7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/3543905887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb7c004800 0x7fbb7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb9963b640 1 -- 172.21.15.139:0/3543905887 reap_dead start 2024-04-25T10:01:28.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb98e3a640 1 -- 172.21.15.139:0/3543905887 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c003520 con 0x7fbb7c0039b0 2024-04-25T10:01:28.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.880+0000 7fbb93fff640 1 --2- 172.21.15.139:0/3543905887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb7c004800 0x7fbb7c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:28.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.881+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/3543905887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 0x7fbb7c003dd0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400c240 tx=0x7fbb7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d70316a0bed4dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.881+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84007cf0 con 0x7fbb7c0039b0 2024-04-25T10:01:28.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.881+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb84005af0 con 0x7fbb7c0039b0 2024-04-25T10:01:28.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.881+0000 7fbb937fe640 1 -- 172.21.15.139:0/3543905887 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84007070 con 0x7fbb7c0039b0 2024-04-25T10:01:28.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.882+0000 7fbb9b09e640 1 -- 172.21.15.139:0/3543905887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 msgr2=0x7fbb7c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.882+0000 7fbb9b09e640 1 --2- 172.21.15.139:0/3543905887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 0x7fbb7c003dd0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400c240 tx=0x7fbb7c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:28.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.882+0000 7fbb9b09e640 1 -- 172.21.15.139:0/3543905887 shutdown_connections 2024-04-25T10:01:28.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.882+0000 7fbb9b09e640 1 --2- 172.21.15.139:0/3543905887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 0x7fbb7c003dd0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.882+0000 7fbb9b09e640 1 -- 172.21.15.139:0/3543905887 >> 172.21.15.139:0/3543905887 conn(0x7fbb940b23f0 msgr2=0x7fbb940b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.882+0000 7fbb9b09e640 1 -- 172.21.15.139:0/3543905887 shutdown_connections 2024-04-25T10:01:28.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.882+0000 7fbb9b09e640 1 -- 172.21.15.139:0/3543905887 wait complete. 2024-04-25T10:01:28.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.883+0000 7fbb9b09e640 1 Processor -- start 2024-04-25T10:01:28.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.883+0000 7fbb9b09e640 1 -- start start 2024-04-25T10:01:28.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb9b09e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 0x7fbb941ef590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb9b09e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb941f2590 0x7fbb9414f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb9b09e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9414f5a0 0x7fbb94155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb9b09e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940b4190 con 0x7fbb7c0039b0 2024-04-25T10:01:28.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb9b09e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940b5190 con 0x7fbb9414f5a0 2024-04-25T10:01:28.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb9b09e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940b4310 con 0x7fbb941f2590 2024-04-25T10:01:28.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb98e3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 0x7fbb941ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb98e3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 0x7fbb941ef590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56070/0 (socket says 172.21.15.139:56070) 2024-04-25T10:01:28.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb98e3a640 1 -- 172.21.15.139:0/518467383 learned_addr learned my addr 172.21.15.139:0/518467383 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb93fff640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb941f2590 0x7fbb9414f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb9963b640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9414f5a0 0x7fbb94155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb98e3a640 1 -- 172.21.15.139:0/518467383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb941f2590 msgr2=0x7fbb9414f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb941f2590 0x7fbb9414f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb98e3a640 1 -- 172.21.15.139:0/518467383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9414f5a0 msgr2=0x7fbb94155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9414f5a0 0x7fbb94155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb98e3a640 1 -- 172.21.15.139:0/518467383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb941501a0 con 0x7fbb7c0039b0 2024-04-25T10:01:28.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb9963b640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9414f5a0 0x7fbb94155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:28.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 0x7fbb941ef590 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fbb8402ad80 tx=0x7fbb940b4d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb917fa640 1 -- 172.21.15.139:0/518467383 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84037040 con 0x7fbb7c0039b0 2024-04-25T10:01:28.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb917fa640 1 -- 172.21.15.139:0/518467383 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb84041070 con 0x7fbb7c0039b0 2024-04-25T10:01:28.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb9b09e640 1 -- 172.21.15.139:0/518467383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9414fbe0 con 0x7fbb7c0039b0 2024-04-25T10:01:28.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb917fa640 1 -- 172.21.15.139:0/518467383 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84001670 con 0x7fbb7c0039b0 2024-04-25T10:01:28.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.884+0000 7fbb9b09e640 1 -- 172.21.15.139:0/518467383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9415c0c0 con 0x7fbb7c0039b0 2024-04-25T10:01:28.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.886+0000 7fbb917fa640 1 -- 172.21.15.139:0/518467383 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb84004a00 con 0x7fbb7c0039b0 2024-04-25T10:01:28.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.886+0000 7fbb917fa640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb8006b500 0x7fbb8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.886+0000 7fbb917fa640 1 -- 172.21.15.139:0/518467383 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb840a9ef0 con 0x7fbb7c0039b0 2024-04-25T10:01:28.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.886+0000 7fbb93fff640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb8006b500 0x7fbb8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.887+0000 7fbb93fff640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb8006b500 0x7fbb8006d9c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbb88000c00 tx=0x7fbb88009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.887+0000 7fbb9dd9e740 1 Processor -- start 2024-04-25T10:01:28.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.887+0000 7fbb9dd9e740 1 -- start start 2024-04-25T10:01:28.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb9dd9e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 0x56532d1f5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb9dd9e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 0x56532d1f5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb9dd9e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1c5770 0x56532d1c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb9dd9e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56532d12eda0 con 0x56532d1f2b10 2024-04-25T10:01:28.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb9dd9e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56532d0d7b80 con 0x56532d1c5770 2024-04-25T10:01:28.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb9dd9e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56532d12ae40 con 0x56532d1f2ef0 2024-04-25T10:01:28.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb98e3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 0x56532d1f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb98e3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 0x56532d1f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56076/0 (socket says 172.21.15.139:56076) 2024-04-25T10:01:28.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb98e3a640 1 -- 172.21.15.139:0/1660745638 learned_addr learned my addr 172.21.15.139:0/1660745638 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb98e3a640 1 -- 172.21.15.139:0/1660745638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 msgr2=0x56532d1f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/1660745638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 0x56532d1f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb98e3a640 1 -- 172.21.15.139:0/1660745638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1c5770 msgr2=0x56532d1c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/1660745638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1c5770 0x56532d1c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb98e3a640 1 -- 172.21.15.139:0/1660745638 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56532d0e1f10 con 0x56532d1f2b10 2024-04-25T10:01:28.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/1660745638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 0x56532d1f5320 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fbb840a8290 tx=0x7fbb84000f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e2fe49bbbf208f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb7a7fc640 1 -- 172.21.15.139:0/1660745638 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84090020 con 0x56532d1f2b10 2024-04-25T10:01:28.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb7a7fc640 1 -- 172.21.15.139:0/1660745638 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8407c6c0 con 0x56532d1f2b10 2024-04-25T10:01:28.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.888+0000 7fbb7a7fc640 1 -- 172.21.15.139:0/1660745638 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84080c80 con 0x56532d1f2b10 2024-04-25T10:01:28.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.889+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1660745638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 msgr2=0x56532d1f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.889+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/1660745638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 0x56532d1f5320 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fbb840a8290 tx=0x7fbb84000f90 comp rx=0 tx=0).stop 2024-04-25T10:01:28.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.889+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1660745638 shutdown_connections 2024-04-25T10:01:28.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.889+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/1660745638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1c5770 0x56532d1c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.889+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/1660745638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 0x56532d1f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.889+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/1660745638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 0x56532d1f5320 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.889+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1660745638 >> 172.21.15.139:0/1660745638 conn(0x56532d1c79e0 msgr2=0x56532d210320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.889+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1660745638 shutdown_connections 2024-04-25T10:01:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.889+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1660745638 wait complete. 2024-04-25T10:01:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.890+0000 7fbb9dd9e740 1 -- start start 2024-04-25T10:01:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.890+0000 7fbb9dd9e740 1 Processor -- start 2024-04-25T10:01:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb9dd9e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 0x56532d1e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb9dd9e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 0x56532d1c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb9dd9e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1e4f00 0x56532d1c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb9dd9e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56532d0d7b80 con 0x56532d1f2b10 2024-04-25T10:01:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb9dd9e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56532d0cd4b0 con 0x56532d1e4f00 2024-04-25T10:01:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb9dd9e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56532d12eda0 con 0x56532d1f2ef0 2024-04-25T10:01:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb98e3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 0x56532d1e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb9dd9e740 10 client.0 _ll_register_callbacks cb 0x56532d10d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb98e3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 0x56532d1e96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56092/0 (socket says 172.21.15.139:56092) 2024-04-25T10:01:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb98e3a640 1 -- 172.21.15.139:0/2329027878 learned_addr learned my addr 172.21.15.139:0/2329027878 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb9963b640 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1e4f00 0x56532d1c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb93fff640 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 0x56532d1c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb98e3a640 1 -- 172.21.15.139:0/2329027878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 msgr2=0x56532d1c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 0x56532d1c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb98e3a640 1 -- 172.21.15.139:0/2329027878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1e4f00 msgr2=0x56532d1c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1e4f00 0x56532d1c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb98e3a640 1 -- 172.21.15.139:0/2329027878 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56532cf0d7d0 con 0x56532d1f2b10 2024-04-25T10:01:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 0x56532d1e96e0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbb840a8370 tx=0x7fbb84048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937ec883034361b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb737fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:01:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb737fe640 1 -- 172.21.15.139:0/2329027878 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84096020 con 0x56532d1f2b10 2024-04-25T10:01:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb737fe640 1 -- 172.21.15.139:0/2329027878 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb84004cf0 con 0x56532d1f2b10 2024-04-25T10:01:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/2329027878 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb8407c6c0 con 0x56532d1f2b10 2024-04-25T10:01:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb737fe640 1 -- 172.21.15.139:0/2329027878 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb84074be0 con 0x56532d1f2b10 2024-04-25T10:01:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb9dd9e740 10 client.8769 fetch_fsmap learned FSMap version 69 2024-04-25T10:01:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/2329027878 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x56532ce433f0 con 0x56532d1f2b10 2024-04-25T10:01:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.891+0000 7fbb737fe640 1 -- 172.21.15.139:0/2329027878 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8407c6c0 con 0x56532d1f2b10 2024-04-25T10:01:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.892+0000 7fbb737fe640 1 -- 172.21.15.139:0/2329027878 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbb840779c0 con 0x56532d1f2b10 2024-04-25T10:01:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.892+0000 7fbb9dd9e740 10 client.8769 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:01:28.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.892+0000 7fbb9dd9e740 10 client.8769 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:01:28.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.892+0000 7fbb9dd9e740 20 client.8769 populate_metadata read hostname 'smithi139' 2024-04-25T10:01:28.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.892+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x56532d1f7020 0x56532d1f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.892+0000 7fbb9dd9e740 4 client.8769 mds_command: new command op to 8466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:28.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.892+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/2329027878 --> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56532d13c2e0 con 0x56532d1f7020 2024-04-25T10:01:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.892+0000 7fbb93fff640 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x56532d1f7020 0x56532d1f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.892+0000 7fbb737fe640 10 client.8769 ms_handle_connect on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.892+0000 7fbb93fff640 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x56532d1f7020 0x56532d1f7430 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3048f812cc0a2352 server_cookie=f925e06a66cd2ffc in_seq=0 out_seq=0 2024-04-25T10:01:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb737fe640 1 -- 172.21.15.139:0/2329027878 <== mds.0 v2:172.21.15.139:6835/4074693991 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56532d13c2e0 con 0x56532d1f7020 2024-04-25T10:01:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb737fe640 10 client.8769 handle_command_reply: tid=0 2024-04-25T10:01:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb9dd9e740 1 client.8769 shutdown 2024-04-25T10:01:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/2329027878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 msgr2=0x56532d1e96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 0x56532d1e96e0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbb840a8370 tx=0x7fbb84048040 comp rx=0 tx=0).stop 2024-04-25T10:01:28.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/2329027878 shutdown_connections 2024-04-25T10:01:28.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x56532d1f7020 0x56532d1f7430 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1e4f00 0x56532d1c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 0x56532d1c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb737fe640 0 client.8769 ms_handle_reset on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:28.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/2329027878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1f2b10 0x56532d1e96e0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/2329027878 >> 172.21.15.139:0/2329027878 conn(0x56532d1c5770 msgr2=0x56532d1f5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/2329027878 shutdown_connections 2024-04-25T10:01:28.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/2329027878 wait complete. 2024-04-25T10:01:28.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.893+0000 7fbb9dd9e740 20 client.8769 trim_cache size 0 max 16384 2024-04-25T10:01:28.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.908+0000 7fbb9dd9e740 1 Processor -- start 2024-04-25T10:01:28.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.908+0000 7fbb9dd9e740 1 -- start start 2024-04-25T10:01:28.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb9dd9e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 0x56532d1e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb9dd9e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2b10 0x56532d1c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb9dd9e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1f2ef0 0x56532d1ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb9dd9e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb941fc7f0 con 0x56532d1e4f00 2024-04-25T10:01:28.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb9dd9e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb941fb9f0 con 0x56532d1f2ef0 2024-04-25T10:01:28.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb9dd9e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb941fadf0 con 0x56532d1f2b10 2024-04-25T10:01:28.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb98e3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 0x56532d1e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb98e3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 0x56532d1e44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56108/0 (socket says 172.21.15.139:56108) 2024-04-25T10:01:28.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb98e3a640 1 -- 172.21.15.139:0/3487025132 learned_addr learned my addr 172.21.15.139:0/3487025132 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb93fff640 1 --2- 172.21.15.139:0/3487025132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2b10 0x56532d1c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb9963b640 1 --2- 172.21.15.139:0/3487025132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1f2ef0 0x56532d1ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb98e3a640 1 -- 172.21.15.139:0/3487025132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2b10 msgr2=0x56532d1c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/3487025132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2b10 0x56532d1c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb98e3a640 1 -- 172.21.15.139:0/3487025132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1f2ef0 msgr2=0x56532d1ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/3487025132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1f2ef0 0x56532d1ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb98e3a640 1 -- 172.21.15.139:0/3487025132 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56532ce030f0 con 0x56532d1e4f00 2024-04-25T10:01:28.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb9963b640 1 --2- 172.21.15.139:0/3487025132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1f2ef0 0x56532d1ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:28.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/3487025132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 0x56532d1e44a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbb84001e40 tx=0x7fbb8404b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb41d3f5d081faf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb79ffb640 1 -- 172.21.15.139:0/3487025132 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84096070 con 0x56532d1e4f00 2024-04-25T10:01:28.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb79ffb640 1 -- 172.21.15.139:0/3487025132 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8407c6c0 con 0x56532d1e4f00 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.909+0000 7fbb79ffb640 1 -- 172.21.15.139:0/3487025132 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84074d80 con 0x56532d1e4f00 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.910+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/3487025132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 msgr2=0x56532d1e44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.910+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/3487025132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 0x56532d1e44a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbb84001e40 tx=0x7fbb8404b000 comp rx=0 tx=0).stop 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.910+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/3487025132 shutdown_connections 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.910+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/3487025132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1f2ef0 0x56532d1ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.910+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/3487025132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2b10 0x56532d1c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.910+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/3487025132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 0x56532d1e44a0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.910+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/3487025132 >> 172.21.15.139:0/3487025132 conn(0x56532d1c5770 msgr2=0x56532d2126c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.910+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/3487025132 shutdown_connections 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.910+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/3487025132 wait complete. 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.910+0000 7fbb9dd9e740 1 -- start start 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.911+0000 7fbb9dd9e740 1 Processor -- start 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.911+0000 7fbb9dd9e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 0x56532d2794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.911+0000 7fbb9dd9e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1f2b10 0x56532d1f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.911+0000 7fbb9dd9e740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 0x56532d1f0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.911+0000 7fbb9dd9e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb941fb3f0 con 0x56532d1e4f00 2024-04-25T10:01:28.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.911+0000 7fbb9dd9e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56532d1f2490 con 0x56532d1f2b10 2024-04-25T10:01:28.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.911+0000 7fbb9dd9e740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56532d1e8ba0 con 0x56532d1f2ef0 2024-04-25T10:01:28.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.911+0000 7fbb9dd9e740 10 client.0 _ll_register_callbacks cb 0x7fbb84074890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:28.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.911+0000 7fbb98e3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 0x56532d2794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.911+0000 7fbb98e3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 0x56532d2794e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56116/0 (socket says 172.21.15.139:56116) 2024-04-25T10:01:28.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.911+0000 7fbb98e3a640 1 -- 172.21.15.139:0/1693427785 learned_addr learned my addr 172.21.15.139:0/1693427785 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:28.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb98e3a640 1 -- 172.21.15.139:0/1693427785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 msgr2=0x56532d1f0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/1693427785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 0x56532d1f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb98e3a640 1 -- 172.21.15.139:0/1693427785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1f2b10 msgr2=0x56532d1f1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/1693427785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1f2b10 0x56532d1f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb98e3a640 1 -- 172.21.15.139:0/1693427785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56532d2185c0 con 0x56532d1e4f00 2024-04-25T10:01:28.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb98e3a640 1 --2- 172.21.15.139:0/1693427785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 0x56532d2794e0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fbb84001be0 tx=0x7fbb84074940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adaa263844d2ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb737fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:01:28.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb737fe640 1 -- 172.21.15.139:0/1693427785 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84096070 con 0x56532d1e4f00 2024-04-25T10:01:28.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb737fe640 1 -- 172.21.15.139:0/1693427785 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb8407c370 con 0x56532d1e4f00 2024-04-25T10:01:28.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1693427785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb8407c6c0 con 0x56532d1e4f00 2024-04-25T10:01:28.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb737fe640 1 -- 172.21.15.139:0/1693427785 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb84084030 con 0x56532d1e4f00 2024-04-25T10:01:28.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb9dd9e740 10 client.8775 fetch_fsmap learned FSMap version 69 2024-04-25T10:01:28.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1693427785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x56532d2185c0 con 0x56532d1e4f00 2024-04-25T10:01:28.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb737fe640 1 -- 172.21.15.139:0/1693427785 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8407c6c0 con 0x56532d1e4f00 2024-04-25T10:01:28.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb737fe640 1 -- 172.21.15.139:0/1693427785 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbb84087030 con 0x56532d1e4f00 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb9dd9e740 10 client.8775 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb9dd9e740 10 client.8775 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb9dd9e740 20 client.8775 populate_metadata read hostname 'smithi139' 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/1693427785 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x56532d1f0ab0 0x56532d1c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb9dd9e740 4 client.8775 mds_command: new command op to 8466 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1693427785 --> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56532d136630 con 0x56532d1f0ab0 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.912+0000 7fbb93fff640 1 --2- 172.21.15.139:0/1693427785 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x56532d1f0ab0 0x56532d1c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb737fe640 10 client.8775 ms_handle_connect on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb93fff640 1 --2- 172.21.15.139:0/1693427785 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x56532d1f0ab0 0x56532d1c9da0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af4fbc847a870cd1 server_cookie=90386bd90f819734 in_seq=0 out_seq=0 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb737fe640 1 -- 172.21.15.139:0/1693427785 <== mds.0 v2:172.21.15.139:6835/4074693991 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x56532d136630 con 0x56532d1f0ab0 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb737fe640 10 client.8775 handle_command_reply: tid=0 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb9dd9e740 1 client.8775 shutdown 2024-04-25T10:01:28.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1693427785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 msgr2=0x56532d2794e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/1693427785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 0x56532d2794e0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fbb84001be0 tx=0x7fbb84074940 comp rx=0 tx=0).stop 2024-04-25T10:01:28.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1693427785 shutdown_connections 2024-04-25T10:01:28.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/1693427785 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x56532d1f0ab0 0x56532d1c9da0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/1693427785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56532d1f2ef0 0x56532d1f0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb737fe640 0 client.8775 ms_handle_reset on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:28.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/1693427785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56532d1f2b10 0x56532d1f1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb9dd9e740 1 --2- 172.21.15.139:0/1693427785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56532d1e4f00 0x56532d2794e0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1693427785 >> 172.21.15.139:0/1693427785 conn(0x56532d1c5770 msgr2=0x56532d1c4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1693427785 shutdown_connections 2024-04-25T10:01:28.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb9dd9e740 1 -- 172.21.15.139:0/1693427785 wait complete. 2024-04-25T10:01:28.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.913+0000 7fbb9dd9e740 20 client.8775 trim_cache size 0 max 16384 2024-04-25T10:01:28.930 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:01:28.930 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:01:28.930 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": { 2024-04-25T10:01:28.930 INFO:teuthology.orchestra.run.smithi139.stdout: "601dbbf1-f22e-424d-a1e4-89c8be56a391": { 2024-04-25T10:01:28.930 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/", 2024-04-25T10:01:28.930 INFO:teuthology.orchestra.run.smithi139.stdout: "tag": "601dbbf1-f22e-424d-a1e4-89c8be56a391", 2024-04-25T10:01:28.930 INFO:teuthology.orchestra.run.smithi139.stdout: "options": "recursive,repair" 2024-04-25T10:01:28.931 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:28.931 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:01:28.931 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:01:28.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.914+0000 7fbb9b09e640 1 -- 172.21.15.139:0/518467383 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb8006b500 msgr2=0x7fbb8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.914+0000 7fbb9b09e640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb8006b500 0x7fbb8006d9c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbb88000c00 tx=0x7fbb88009040 comp rx=0 tx=0).stop 2024-04-25T10:01:28.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.914+0000 7fbb9b09e640 1 -- 172.21.15.139:0/518467383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 msgr2=0x7fbb941ef590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.914+0000 7fbb9b09e640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 0x7fbb941ef590 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fbb8402ad80 tx=0x7fbb940b4d10 comp rx=0 tx=0).stop 2024-04-25T10:01:28.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.914+0000 7fbb9b09e640 1 -- 172.21.15.139:0/518467383 shutdown_connections 2024-04-25T10:01:28.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.914+0000 7fbb9b09e640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb8006b500 0x7fbb8006d9c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.914+0000 7fbb9b09e640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb9414f5a0 0x7fbb94155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.914+0000 7fbb9b09e640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb941f2590 0x7fbb9414f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.914+0000 7fbb9b09e640 1 --2- 172.21.15.139:0/518467383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb7c0039b0 0x7fbb941ef590 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.914+0000 7fbb9b09e640 1 -- 172.21.15.139:0/518467383 >> 172.21.15.139:0/518467383 conn(0x7fbb940b23f0 msgr2=0x7fbb94072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.914+0000 7fbb9b09e640 1 -- 172.21.15.139:0/518467383 shutdown_connections 2024-04-25T10:01:28.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:28.914+0000 7fbb9b09e640 1 -- 172.21.15.139:0/518467383 wait complete. 2024-04-25T10:01:28.934 INFO:tasks.cephfs.filesystem:scrub status for tag:601dbbf1-f22e-424d-a1e4-89c8be56a391 - {'path': '/', 'tag': '601dbbf1-f22e-424d-a1e4-89c8be56a391', 'options': 'recursive,repair'} 2024-04-25T10:01:29.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:29.207+0000 7fce3affd640 1 -- 172.21.15.139:0/3869143164 <== mon.0 v2:172.21.15.139:3300/0 7 ==== log(4 entries from seq 1 at 2024-04-25T10:01:28.731976+0000) v1 ==== 695+0+0 (secure 0 0 0) 0x7fce3c0896f0 con 0x7fce4c0b5980 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: cluster: 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: id: 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: health: HEALTH_OK 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: services: 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: mgr: y(active, since 8m), standbys: x 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 8m) 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: data: 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: volumes: 1/1 healthy 2024-04-25T10:01:29.212 INFO:teuthology.orchestra.run.smithi139.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:01:29.213 INFO:teuthology.orchestra.run.smithi139.stdout: objects: 26 objects, 6.6 MiB 2024-04-25T10:01:29.213 INFO:teuthology.orchestra.run.smithi139.stdout: usage: 267 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:01:29.213 INFO:teuthology.orchestra.run.smithi139.stdout: pgs: 97 active+clean 2024-04-25T10:01:29.213 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:29.213 INFO:teuthology.orchestra.run.smithi139.stdout: io: 2024-04-25T10:01:29.213 INFO:teuthology.orchestra.run.smithi139.stdout: client: 617 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-04-25T10:01:29.213 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:29.213 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:01:29.213 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:01:28.731976+0000 mds.c [INF] scrub queued for path: / 2024-04-25T10:01:29.213 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:01:28.731979+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:01:29.213 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:01:28.731984+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-25T10:01:29.213 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:01:28.733164+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:01:32.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:32.213+0000 7fce3affd640 1 -- 172.21.15.139:0/3869143164 <== mon.0 v2:172.21.15.139:3300/0 8 ==== log(1 entries from seq 5 at 2024-04-25T10:01:31.088215+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7fce3c012e60 con 0x7fce4c0b5980 2024-04-25T10:01:32.302 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:01:31.088215+0000 mds.c [INF] scrub summary: idle 2024-04-25T10:01:47.023 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:47.027 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:57.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:57.811+0000 7fce3affd640 1 -- 172.21.15.139:0/3869143164 <== mon.0 v2:172.21.15.139:3300/0 9 ==== log(2 entries from seq 1153 at 2024-04-25T10:01:57.172242+0000) v1 ==== 608+0+0 (secure 0 0 0) 0x7fce3c057840 con 0x7fce4c0b5980 2024-04-25T10:01:58.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:58.817+0000 7fce3affd640 1 -- 172.21.15.139:0/3869143164 <== mon.0 v2:172.21.15.139:3300/0 10 ==== log(2 entries from seq 1155 at 2024-04-25T10:01:57.816469+0000) v1 ==== 612+0+0 (secure 0 0 0) 0x7fce3c057aa0 con 0x7fce4c0b5980 2024-04-25T10:01:58.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:58.935 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:01:59.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.041+0000 7f558d95c640 1 Processor -- start 2024-04-25T10:01:59.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.041+0000 7f558d95c640 1 -- start start 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.042+0000 7f558d95c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f558815dbf0 0x7f55881617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.042+0000 7f558d95c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5588161dc0 con 0x7f55880cee50 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.042+0000 7f558d95c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55881620a0 con 0x7f55880b9370 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.042+0000 7f558d95c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55880d1ee0 con 0x7f558815dbf0 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.042+0000 7f55877fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f55880cee50 0x7f55880d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57722/0 (socket says 172.21.15.139:57722) 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.042+0000 7f55877fe640 1 -- 172.21.15.139:0/2919086984 learned_addr learned my addr 172.21.15.139:0/2919086984 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f5586ffd640 1 --2- 172.21.15.139:0/2919086984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f558815dbf0 0x7f55881617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613562269 0 0) 0x7f5588161dc0 con 0x7f55880cee50 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5574003520 con 0x7f55880cee50 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2413125934 0 0) 0x7f5574003520 con 0x7f55880cee50 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f558815dbf0 msgr2=0x7f55881617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 --2- 172.21.15.139:0/2919086984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f558815dbf0 0x7f55881617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 >> v1:172.21.15.150:6789/0 conn(0x7f55880b9370 legacy=0x7f55880ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5588162380 con 0x7f55880cee50 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f55800019f0 con 0x7f55880cee50 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 >> v1:172.21.15.139:6789/0 conn(0x7f55880cee50 legacy=0x7f55880d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 --2- 172.21.15.139:0/2919086984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55740039e0 0x7f5574003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 --2- 172.21.15.139:0/2919086984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5574004390 0x7f5574004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 --2- 172.21.15.139:0/2919086984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f558815dbf0 0x7f55740050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55880d1ee0 con 0x7f5574004390 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557400b9c0 con 0x7f55740039e0 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557400bbc0 con 0x7f558815dbf0 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f557ffff640 1 --2- 172.21.15.139:0/2919086984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5574004390 0x7f5574004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f5586ffd640 1 --2- 172.21.15.139:0/2919086984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55740039e0 0x7f5574003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f55877fe640 1 --2- 172.21.15.139:0/2919086984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f558815dbf0 0x7f55740050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f557ffff640 1 -- 172.21.15.139:0/2919086984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f558815dbf0 msgr2=0x7f55740050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f557ffff640 1 --2- 172.21.15.139:0/2919086984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f558815dbf0 0x7f55740050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f557ffff640 1 -- 172.21.15.139:0/2919086984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55740039e0 msgr2=0x7f5574003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f557ffff640 1 --2- 172.21.15.139:0/2919086984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55740039e0 0x7f5574003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.043+0000 7f557ffff640 1 -- 172.21.15.139:0/2919086984 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5574003520 con 0x7f5574004390 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.044+0000 7f557ffff640 1 --2- 172.21.15.139:0/2919086984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5574004390 0x7f5574004860 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f5570001860 tx=0x7f5574005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74d5b9c7240498a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.044+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570037070 con 0x7f5574004390 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.044+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557002abf0 con 0x7f5574004390 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.044+0000 7f55867fc640 1 -- 172.21.15.139:0/2919086984 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570001430 con 0x7f5574004390 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.045+0000 7f558d95c640 1 -- 172.21.15.139:0/2919086984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5574004390 msgr2=0x7f5574004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.045+0000 7f558d95c640 1 --2- 172.21.15.139:0/2919086984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5574004390 0x7f5574004860 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f5570001860 tx=0x7f5574005830 comp rx=0 tx=0).stop 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.045+0000 7f55877fe640 1 -- 172.21.15.139:0/2919086984 reap_dead start 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.045+0000 7f558d95c640 1 -- 172.21.15.139:0/2919086984 shutdown_connections 2024-04-25T10:01:59.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.045+0000 7f558d95c640 1 -- 172.21.15.139:0/2919086984 >> 172.21.15.139:0/2919086984 conn(0x7f55880b5480 msgr2=0x7f55880b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.045+0000 7f558d95c640 1 -- 172.21.15.139:0/2919086984 shutdown_connections 2024-04-25T10:01:59.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.045+0000 7f558d95c640 1 -- 172.21.15.139:0/2919086984 wait complete. 2024-04-25T10:01:59.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.046+0000 7f558d95c640 1 Processor -- start 2024-04-25T10:01:59.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.046+0000 7f558d95c640 1 -- start start 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f558d95c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5574004390 0x7f558815e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f558d95c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5588150640 0x7f55880c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f558d95c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55880ccc10 0x7f55880cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f558d95c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558815dfd0 con 0x7f55880ccc10 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f558d95c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55880b97d0 con 0x7f5574004390 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f558d95c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55880cd600 con 0x7f5588150640 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f55877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55880ccc10 0x7f55880cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f55877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55880ccc10 0x7f55880cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46066/0 (socket says 172.21.15.139:46066) 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f55877fe640 1 -- 172.21.15.139:0/3852224546 learned_addr learned my addr 172.21.15.139:0/3852224546 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f5586ffd640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5574004390 0x7f558815e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f557ffff640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5588150640 0x7f55880c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f55877fe640 1 -- 172.21.15.139:0/3852224546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5588150640 msgr2=0x7f55880c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f55877fe640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5588150640 0x7f55880c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f55877fe640 1 -- 172.21.15.139:0/3852224546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5574004390 msgr2=0x7f558815e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f55877fe640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5574004390 0x7f558815e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f55877fe640 1 -- 172.21.15.139:0/3852224546 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558815e8c0 con 0x7f55880ccc10 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f55877fe640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55880ccc10 0x7f55880cd0c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f558808c170 tx=0x7f558000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.047+0000 7f5584ff9640 1 -- 172.21.15.139:0/3852224546 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558000ec10 con 0x7f55880ccc10 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.048+0000 7f5584ff9640 1 -- 172.21.15.139:0/3852224546 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5580018070 con 0x7f55880ccc10 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.048+0000 7f5584ff9640 1 -- 172.21.15.139:0/3852224546 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580013420 con 0x7f55880ccc10 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.048+0000 7f558d95c640 1 -- 172.21.15.139:0/3852224546 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558815eba0 con 0x7f55880ccc10 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.048+0000 7f558d95c640 1 -- 172.21.15.139:0/3852224546 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558815f160 con 0x7f55880ccc10 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.049+0000 7f5584ff9640 1 -- 172.21.15.139:0/3852224546 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55800135c0 con 0x7f55880ccc10 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.050+0000 7f5584ff9640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f556806b4d0 0x7f556806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.050+0000 7f5584ff9640 1 -- 172.21.15.139:0/3852224546 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f558008a710 con 0x7f55880ccc10 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.050+0000 7f5586ffd640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f556806b4d0 0x7f556806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.050+0000 7f5586ffd640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f556806b4d0 0x7f556806d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5588161490 tx=0x7f5578008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.051+0000 7f5590666740 1 Processor -- start 2024-04-25T10:01:59.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.051+0000 7f5590666740 1 -- start start 2024-04-25T10:01:59.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f5590666740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284b10 0x55db73287320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f5590666740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284ef0 0x55db73287860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f5590666740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73257770 0x55db7325af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f5590666740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db731c0da0 con 0x55db73284ef0 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f5590666740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db73169b80 con 0x55db73257770 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f5590666740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db731bce40 con 0x55db73284b10 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f557ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284ef0 0x55db73287860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f557ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284ef0 0x55db73287860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46068/0 (socket says 172.21.15.139:46068) 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f557ffff640 1 -- 172.21.15.139:0/1024784206 learned_addr learned my addr 172.21.15.139:0/1024784206 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f5586ffd640 1 --2- 172.21.15.139:0/1024784206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284b10 0x55db73287320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f55877fe640 1 --2- 172.21.15.139:0/1024784206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73257770 0x55db7325af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f557ffff640 1 -- 172.21.15.139:0/1024784206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284b10 msgr2=0x55db73287320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f557ffff640 1 --2- 172.21.15.139:0/1024784206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284b10 0x55db73287320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f557ffff640 1 -- 172.21.15.139:0/1024784206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73257770 msgr2=0x55db7325af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f557ffff640 1 --2- 172.21.15.139:0/1024784206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73257770 0x55db7325af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f557ffff640 1 -- 172.21.15.139:0/1024784206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db73173f10 con 0x55db73284ef0 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f557ffff640 1 --2- 172.21.15.139:0/1024784206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284ef0 0x55db73287860 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f5570008fd0 tx=0x7f557002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1f64be7da3fa62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f557d7fa640 1 -- 172.21.15.139:0/1024784206 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570042070 con 0x55db73284ef0 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f557d7fa640 1 -- 172.21.15.139:0/1024784206 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557002ac50 con 0x55db73284ef0 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.052+0000 7f557d7fa640 1 -- 172.21.15.139:0/1024784206 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570037040 con 0x55db73284ef0 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.053+0000 7f5590666740 1 -- 172.21.15.139:0/1024784206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284ef0 msgr2=0x55db73287860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.053+0000 7f5590666740 1 --2- 172.21.15.139:0/1024784206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284ef0 0x55db73287860 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f5570008fd0 tx=0x7f557002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.053+0000 7f5590666740 1 -- 172.21.15.139:0/1024784206 shutdown_connections 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.053+0000 7f5590666740 1 --2- 172.21.15.139:0/1024784206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284ef0 0x55db73287860 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.053+0000 7f5590666740 1 --2- 172.21.15.139:0/1024784206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73257770 0x55db7325af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.053+0000 7f5590666740 1 --2- 172.21.15.139:0/1024784206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284b10 0x55db73287320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.053+0000 7f5590666740 1 -- 172.21.15.139:0/1024784206 >> 172.21.15.139:0/1024784206 conn(0x55db732599e0 msgr2=0x55db732a2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.053+0000 7f5590666740 1 -- 172.21.15.139:0/1024784206 shutdown_connections 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.053+0000 7f5590666740 1 -- 172.21.15.139:0/1024784206 wait complete. 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.054+0000 7f5590666740 1 -- start start 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.054+0000 7f5590666740 1 Processor -- start 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f5590666740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284b10 0x55db732a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f5590666740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284ef0 0x55db73274b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f5590666740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284130 0x55db732a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f5590666740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7325cb90 con 0x55db73284130 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f5590666740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7315f4b0 con 0x55db73284b10 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f5590666740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db7325ce70 con 0x55db73284ef0 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f5590666740 10 client.0 _ll_register_callbacks cb 0x55db7319f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f55877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284130 0x55db732a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f55877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284130 0x55db732a69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46076/0 (socket says 172.21.15.139:46076) 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f55877fe640 1 -- 172.21.15.139:0/3481885689 learned_addr learned my addr 172.21.15.139:0/3481885689 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f557ffff640 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284ef0 0x55db73274b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f5586ffd640 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284b10 0x55db732a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f55877fe640 1 -- 172.21.15.139:0/3481885689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284ef0 msgr2=0x55db73274b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f55877fe640 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284ef0 0x55db73274b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f55877fe640 1 -- 172.21.15.139:0/3481885689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284b10 msgr2=0x55db732a9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f55877fe640 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284b10 0x55db732a9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f55877fe640 1 -- 172.21.15.139:0/3481885689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db72f9f7d0 con 0x55db73284130 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f557ffff640 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284ef0 0x55db73274b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f55877fe640 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284130 0x55db732a69f0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f5580002230 tx=0x7f5580058ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c197438e6103df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f55667fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f55667fc640 1 -- 172.21.15.139:0/3481885689 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580087060 con 0x55db73284130 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f55667fc640 1 -- 172.21.15.139:0/3481885689 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f55800553d0 con 0x55db73284130 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.055+0000 7f5590666740 1 -- 172.21.15.139:0/3481885689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f557002ac50 con 0x55db73284130 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f55667fc640 1 -- 172.21.15.139:0/3481885689 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5580070030 con 0x55db73284130 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f5590666740 10 client.8787 fetch_fsmap learned FSMap version 69 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f5590666740 1 -- 172.21.15.139:0/3481885689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55db72ed53f0 con 0x55db73284130 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f55667fc640 1 -- 172.21.15.139:0/3481885689 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557002ac50 con 0x55db73284130 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f55667fc640 1 -- 172.21.15.139:0/3481885689 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f55800581b0 con 0x55db73284130 2024-04-25T10:01:59.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f5590666740 10 client.8787 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f5590666740 10 client.8787 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f5590666740 20 client.8787 populate_metadata read hostname 'smithi139' 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f5590666740 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55db73282fc0 0x55db732833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f5590666740 4 client.8787 mds_command: new command op to 8466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f5590666740 1 -- 172.21.15.139:0/3481885689 --> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db731ce2e0 con 0x55db73282fc0 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f5586ffd640 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55db73282fc0 0x55db732833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f5586ffd640 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55db73282fc0 0x55db732833d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d650f7f6624e3d14 server_cookie=5fe458361126cc37 in_seq=0 out_seq=0 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.056+0000 7f55667fc640 10 client.8787 ms_handle_connect on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f55667fc640 1 -- 172.21.15.139:0/3481885689 <== mds.0 v2:172.21.15.139:6835/4074693991 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db731ce2e0 con 0x55db73282fc0 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f55667fc640 10 client.8787 handle_command_reply: tid=0 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f5590666740 1 client.8787 shutdown 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f5590666740 1 -- 172.21.15.139:0/3481885689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284130 msgr2=0x55db732a69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f5590666740 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284130 0x55db732a69f0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f5580002230 tx=0x7f5580058ea0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f5590666740 1 -- 172.21.15.139:0/3481885689 shutdown_connections 2024-04-25T10:01:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f5590666740 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55db73282fc0 0x55db732833d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f5590666740 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284130 0x55db732a69f0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f55667fc640 0 client.8787 ms_handle_reset on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f5590666740 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284ef0 0x55db73274b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f5590666740 1 --2- 172.21.15.139:0/3481885689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284b10 0x55db732a9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f5590666740 1 -- 172.21.15.139:0/3481885689 >> 172.21.15.139:0/3481885689 conn(0x55db73257770 msgr2=0x55db73258760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f5590666740 1 -- 172.21.15.139:0/3481885689 shutdown_connections 2024-04-25T10:01:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f5590666740 1 -- 172.21.15.139:0/3481885689 wait complete. 2024-04-25T10:01:59.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.057+0000 7f5590666740 20 client.8787 trim_cache size 0 max 16384 2024-04-25T10:01:59.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.072+0000 7f5590666740 1 Processor -- start 2024-04-25T10:01:59.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.072+0000 7f5590666740 1 -- start start 2024-04-25T10:01:59.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f5590666740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284130 0x55db732872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f5590666740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 0x55db7328a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f5590666740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284ef0 0x55db7329fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f5590666740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5588200950 con 0x55db73284b10 2024-04-25T10:01:59.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f5590666740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55881ffb20 con 0x55db73284130 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f5590666740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55881610b0 con 0x55db73284ef0 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f557ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 0x55db7328a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f557ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 0x55db7328a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46086/0 (socket says 172.21.15.139:46086) 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f557ffff640 1 -- 172.21.15.139:0/3422837692 learned_addr learned my addr 172.21.15.139:0/3422837692 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f5586ffd640 1 --2- 172.21.15.139:0/3422837692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284130 0x55db732872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f55877fe640 1 --2- 172.21.15.139:0/3422837692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284ef0 0x55db7329fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f557ffff640 1 -- 172.21.15.139:0/3422837692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284ef0 msgr2=0x55db7329fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f557ffff640 1 --2- 172.21.15.139:0/3422837692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284ef0 0x55db7329fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f557ffff640 1 -- 172.21.15.139:0/3422837692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284130 msgr2=0x55db732872e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f557ffff640 1 --2- 172.21.15.139:0/3422837692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284130 0x55db732872e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f557ffff640 1 -- 172.21.15.139:0/3422837692 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db72e950f0 con 0x55db73284b10 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f55877fe640 1 --2- 172.21.15.139:0/3422837692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284ef0 0x55db7329fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:59.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f5586ffd640 1 --2- 172.21.15.139:0/3422837692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284130 0x55db732872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f557ffff640 1 --2- 172.21.15.139:0/3422837692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 0x55db7328a7d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f5570001950 tx=0x7f557002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec91d1d100946bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.073+0000 7f557cff9640 1 -- 172.21.15.139:0/3422837692 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570037070 con 0x55db73284b10 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.074+0000 7f557cff9640 1 -- 172.21.15.139:0/3422837692 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5570005030 con 0x55db73284b10 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.074+0000 7f557cff9640 1 -- 172.21.15.139:0/3422837692 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570042070 con 0x55db73284b10 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.074+0000 7f5590666740 1 -- 172.21.15.139:0/3422837692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 msgr2=0x55db7328a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.074+0000 7f5590666740 1 --2- 172.21.15.139:0/3422837692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 0x55db7328a7d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f5570001950 tx=0x7f557002f000 comp rx=0 tx=0).stop 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.074+0000 7f5590666740 1 -- 172.21.15.139:0/3422837692 shutdown_connections 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.074+0000 7f5590666740 1 --2- 172.21.15.139:0/3422837692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 0x55db7328a7d0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.074+0000 7f5590666740 1 --2- 172.21.15.139:0/3422837692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284ef0 0x55db7329fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.074+0000 7f5590666740 1 --2- 172.21.15.139:0/3422837692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284130 0x55db732872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.074+0000 7f5590666740 1 -- 172.21.15.139:0/3422837692 >> 172.21.15.139:0/3422837692 conn(0x55db73257770 msgr2=0x55db7325d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.074+0000 7f5590666740 1 -- 172.21.15.139:0/3422837692 shutdown_connections 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.074+0000 7f5590666740 1 -- 172.21.15.139:0/3422837692 wait complete. 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.075+0000 7f5590666740 1 -- start start 2024-04-25T10:01:59.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.075+0000 7f5590666740 1 Processor -- start 2024-04-25T10:01:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f5590666740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284130 0x55db732bbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f5590666740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 0x55db732a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f5590666740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284ef0 0x55db73297cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f5590666740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db732826e0 con 0x55db73284b10 2024-04-25T10:01:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f5590666740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55881ff520 con 0x55db73284ef0 2024-04-25T10:01:59.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f5590666740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db732c49a0 con 0x55db73284130 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f5590666740 10 client.0 _ll_register_callbacks cb 0x7f5570035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f557ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 0x55db732a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f557ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 0x55db732a9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46092/0 (socket says 172.21.15.139:46092) 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f557ffff640 1 -- 172.21.15.139:0/3904442981 learned_addr learned my addr 172.21.15.139:0/3904442981 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f5586ffd640 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284130 0x55db732bbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f55877fe640 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284ef0 0x55db73297cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f557ffff640 1 -- 172.21.15.139:0/3904442981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284130 msgr2=0x55db732bbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f557ffff640 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284130 0x55db732bbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f557ffff640 1 -- 172.21.15.139:0/3904442981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284ef0 msgr2=0x55db73297cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f557ffff640 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284ef0 0x55db73297cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f557ffff640 1 -- 172.21.15.139:0/3904442981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db7325b100 con 0x55db73284b10 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f55877fe640 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284ef0 0x55db73297cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f5586ffd640 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284130 0x55db732bbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:59.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f557ffff640 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 0x55db732a9350 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5570033000 tx=0x7f557002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5378ef06034685f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f55667fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f55667fc640 1 -- 172.21.15.139:0/3904442981 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570042070 con 0x55db73284b10 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f55667fc640 1 -- 172.21.15.139:0/3904442981 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5570003a00 con 0x55db73284b10 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f5590666740 1 -- 172.21.15.139:0/3904442981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5570005030 con 0x55db73284b10 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f55667fc640 1 -- 172.21.15.139:0/3904442981 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5570055050 con 0x55db73284b10 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f5590666740 10 client.8793 fetch_fsmap learned FSMap version 69 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.076+0000 7f5590666740 1 -- 172.21.15.139:0/3904442981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55db732bb6b0 con 0x55db73284b10 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f55667fc640 1 -- 172.21.15.139:0/3904442981 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5570005030 con 0x55db73284b10 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f55667fc640 1 -- 172.21.15.139:0/3904442981 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5570058020 con 0x55db73284b10 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 10 client.8793 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 10 client.8793 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 20 client.8793 populate_metadata read hostname 'smithi139' 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55db732b1910 0x55db732a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 4 client.8793 mds_command: new command op to 8466 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:01:59.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 -- 172.21.15.139:0/3904442981 --> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55db731c8630 con 0x55db732b1910 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5586ffd640 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55db732b1910 0x55db732a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f55667fc640 10 client.8793 ms_handle_connect on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5586ffd640 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55db732b1910 0x55db732a22c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edb4782e8fae4ee7 server_cookie=7cad19ff1e590a7d in_seq=0 out_seq=0 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f55667fc640 1 -- 172.21.15.139:0/3904442981 <== mds.0 v2:172.21.15.139:6835/4074693991 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55db731c8630 con 0x55db732b1910 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f55667fc640 10 client.8793 handle_command_reply: tid=0 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 client.8793 shutdown 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 -- 172.21.15.139:0/3904442981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 msgr2=0x55db732a9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 0x55db732a9350 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5570033000 tx=0x7f557002c070 comp rx=0 tx=0).stop 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 -- 172.21.15.139:0/3904442981 shutdown_connections 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.139:6835/4074693991,v1:172.21.15.139:6837/4074693991] conn(0x55db732b1910 0x55db732a22c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55db73284b10 0x55db732a9350 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db73284ef0 0x55db73297cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f55667fc640 0 client.8793 ms_handle_reset on v2:172.21.15.139:6835/4074693991 2024-04-25T10:01:59.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 --2- 172.21.15.139:0/3904442981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db73284130 0x55db732bbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 -- 172.21.15.139:0/3904442981 >> 172.21.15.139:0/3904442981 conn(0x55db73257770 msgr2=0x55db7328a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 -- 172.21.15.139:0/3904442981 shutdown_connections 2024-04-25T10:01:59.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.077+0000 7f5590666740 1 -- 172.21.15.139:0/3904442981 wait complete. 2024-04-25T10:01:59.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f5590666740 20 client.8793 trim_cache size 0 max 16384 2024-04-25T10:01:59.089 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:01:59.089 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "no active scrubs running", 2024-04-25T10:01:59.089 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": {} 2024-04-25T10:01:59.089 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:01:59.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f558d95c640 1 -- 172.21.15.139:0/3852224546 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f556806b4d0 msgr2=0x7f556806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f558d95c640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f556806b4d0 0x7f556806d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5588161490 tx=0x7f5578008040 comp rx=0 tx=0).stop 2024-04-25T10:01:59.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f558d95c640 1 -- 172.21.15.139:0/3852224546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55880ccc10 msgr2=0x7f55880cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f558d95c640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55880ccc10 0x7f55880cd0c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f558808c170 tx=0x7f558000e530 comp rx=0 tx=0).stop 2024-04-25T10:01:59.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f558d95c640 1 -- 172.21.15.139:0/3852224546 shutdown_connections 2024-04-25T10:01:59.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f558d95c640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55880ccc10 0x7f55880cd0c0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f558d95c640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f556806b4d0 0x7f556806d990 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f558d95c640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5588150640 0x7f55880c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f558d95c640 1 --2- 172.21.15.139:0/3852224546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5574004390 0x7f558815e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f558d95c640 1 -- 172.21.15.139:0/3852224546 >> 172.21.15.139:0/3852224546 conn(0x7f55880b5480 msgr2=0x7f5588158710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f558d95c640 1 -- 172.21.15.139:0/3852224546 shutdown_connections 2024-04-25T10:01:59.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.078+0000 7f558d95c640 1 -- 172.21.15.139:0/3852224546 wait complete. 2024-04-25T10:01:59.093 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:01:59.194 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:01:59.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.287+0000 7f836a86f640 1 Processor -- start 2024-04-25T10:01:59.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.287+0000 7f836a86f640 1 -- start start 2024-04-25T10:01:59.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.288+0000 7f836a86f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f83640cee50 0x7f8364168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.288+0000 7f836a86f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8364161dc0 con 0x7f836415dbf0 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.288+0000 7f836a86f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83641620a0 con 0x7f83641617e0 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.288+0000 7f836a86f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83640d1ee0 con 0x7f83640cee50 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.288+0000 7f8368e0c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f83640cee50 0x7f8364168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.288+0000 7f8368e0c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f83640cee50 0x7f8364168b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46100/0 (socket says 172.21.15.139:46100) 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.288+0000 7f8368e0c640 1 -- 172.21.15.139:0/3779900221 learned_addr learned my addr 172.21.15.139:0/3779900221 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.288+0000 7f8363fff640 1 --1- 172.21.15.139:0/3779900221 >> v1:172.21.15.139:6789/0 conn(0x7f836415dbf0 0x7f83640b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57728/0 (socket says 172.21.15.139:57728) 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.289+0000 7f8368e0c640 1 -- 172.21.15.139:0/3779900221 >> v1:172.21.15.150:6789/0 conn(0x7f83641617e0 legacy=0x7f83640ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.289+0000 7f8368e0c640 1 -- 172.21.15.139:0/3779900221 >> v1:172.21.15.139:6789/0 conn(0x7f836415dbf0 legacy=0x7f83640b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.289+0000 7f8368e0c640 1 -- 172.21.15.139:0/3779900221 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8364162380 con 0x7f83640cee50 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.289+0000 7f8362ffd640 1 -- 172.21.15.139:0/3779900221 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583507929 0 0) 0x7f8364161dc0 con 0x7f836415dbf0 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.289+0000 7f8368e0c640 1 --2- 172.21.15.139:0/3779900221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83640cee50 0x7f8364168b10 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f835c009e50 tx=0x7f835c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef7c44d7d132aca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.289+0000 7f8362ffd640 1 -- 172.21.15.139:0/3779900221 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c036070 con 0x7f83640cee50 2024-04-25T10:01:59.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.290+0000 7f8362ffd640 1 -- 172.21.15.139:0/3779900221 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c031600 con 0x7f83640cee50 2024-04-25T10:01:59.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.290+0000 7f8362ffd640 1 -- 172.21.15.139:0/3779900221 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c0319d0 con 0x7f83640cee50 2024-04-25T10:01:59.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.290+0000 7f836a86f640 1 -- 172.21.15.139:0/3779900221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83640cee50 msgr2=0x7f8364168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.290+0000 7f836a86f640 1 --2- 172.21.15.139:0/3779900221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83640cee50 0x7f8364168b10 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f835c009e50 tx=0x7f835c02a6d0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.290+0000 7f836a86f640 1 -- 172.21.15.139:0/3779900221 shutdown_connections 2024-04-25T10:01:59.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.290+0000 7f836a86f640 1 --2- 172.21.15.139:0/3779900221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83640cee50 0x7f8364168b10 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.290+0000 7f836a86f640 1 -- 172.21.15.139:0/3779900221 >> 172.21.15.139:0/3779900221 conn(0x7f83640b5480 msgr2=0x7f83640b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.290+0000 7f836a86f640 1 -- 172.21.15.139:0/3779900221 shutdown_connections 2024-04-25T10:01:59.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.290+0000 7f836a86f640 1 -- 172.21.15.139:0/3779900221 wait complete. 2024-04-25T10:01:59.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.291+0000 7f836a86f640 1 Processor -- start 2024-04-25T10:01:59.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.291+0000 7f836a86f640 1 -- start start 2024-04-25T10:01:59.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.292+0000 7f836a86f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83640cee50 0x7f83640c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.292+0000 7f836a86f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f836415dbf0 0x7f836413d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.292+0000 7f836a86f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83641617e0 0x7f836413d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.292+0000 7f836a86f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836415dfd0 con 0x7f83641617e0 2024-04-25T10:01:59.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.292+0000 7f836a86f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83640d3380 con 0x7f836415dbf0 2024-04-25T10:01:59.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.292+0000 7f836a86f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836413df30 con 0x7f83640cee50 2024-04-25T10:01:59.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f8368e0c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83641617e0 0x7f836413d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f8368e0c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83641617e0 0x7f836413d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46112/0 (socket says 172.21.15.139:46112) 2024-04-25T10:01:59.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f8368e0c640 1 -- 172.21.15.139:0/711592368 learned_addr learned my addr 172.21.15.139:0/711592368 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f83637fe640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f836415dbf0 0x7f836413d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f8363fff640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83640cee50 0x7f83640c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f8368e0c640 1 -- 172.21.15.139:0/711592368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83640cee50 msgr2=0x7f83640c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f8368e0c640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83640cee50 0x7f83640c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f8368e0c640 1 -- 172.21.15.139:0/711592368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f836415dbf0 msgr2=0x7f836413d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f8368e0c640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f836415dbf0 0x7f836413d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f8368e0c640 1 -- 172.21.15.139:0/711592368 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83640cb4d0 con 0x7f83641617e0 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f83637fe640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f836415dbf0 0x7f836413d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f8368e0c640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83641617e0 0x7f836413d9f0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f835c034040 tx=0x7f835c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.293+0000 7f8360ff9640 1 -- 172.21.15.139:0/711592368 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c041020 con 0x7f83641617e0 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.294+0000 7f8360ff9640 1 -- 172.21.15.139:0/711592368 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c007780 con 0x7f83641617e0 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.294+0000 7f8360ff9640 1 -- 172.21.15.139:0/711592368 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c036040 con 0x7f83641617e0 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.294+0000 7f836a86f640 1 -- 172.21.15.139:0/711592368 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83640cb7b0 con 0x7f83641617e0 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.294+0000 7f836a86f640 1 -- 172.21.15.139:0/711592368 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83640cbd70 con 0x7f83641617e0 2024-04-25T10:01:59.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.295+0000 7f836a86f640 1 -- 172.21.15.139:0/711592368 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8364074520 con 0x7f83641617e0 2024-04-25T10:01:59.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.296+0000 7f8360ff9640 1 -- 172.21.15.139:0/711592368 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f835c044020 con 0x7f83641617e0 2024-04-25T10:01:59.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.296+0000 7f8360ff9640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f835006b520 0x7f835006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.296+0000 7f8360ff9640 1 -- 172.21.15.139:0/711592368 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f835c0aaf80 con 0x7f83641617e0 2024-04-25T10:01:59.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.296+0000 7f8363fff640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f835006b520 0x7f835006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.297+0000 7f8363fff640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f835006b520 0x7f835006d9e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8354000f60 tx=0x7f8354007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.300+0000 7f8360ff9640 1 -- 172.21.15.139:0/711592368 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f835c04b030 con 0x7f83641617e0 2024-04-25T10:01:59.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.506+0000 7f836a86f640 1 -- 172.21.15.139:0/711592368 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f83640d1680 con 0x7f83641617e0 2024-04-25T10:01:59.511 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T10:01:59.506+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:01:59.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.508+0000 7f8360ff9640 1 -- 172.21.15.139:0/711592368 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7f835c079790 con 0x7f83641617e0 2024-04-25T10:01:59.512 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:01:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.510+0000 7f836a86f640 1 -- 172.21.15.139:0/711592368 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f835006b520 msgr2=0x7f835006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.510+0000 7f836a86f640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f835006b520 0x7f835006d9e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8354000f60 tx=0x7f8354007d50 comp rx=0 tx=0).stop 2024-04-25T10:01:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.511+0000 7f836a86f640 1 -- 172.21.15.139:0/711592368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83641617e0 msgr2=0x7f836413d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.511+0000 7f836a86f640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83641617e0 0x7f836413d9f0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f835c034040 tx=0x7f835c004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.511+0000 7f836a86f640 1 -- 172.21.15.139:0/711592368 shutdown_connections 2024-04-25T10:01:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.511+0000 7f836a86f640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83641617e0 0x7f836413d9f0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.511+0000 7f836a86f640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f835006b520 0x7f835006d9e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.511+0000 7f836a86f640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f836415dbf0 0x7f836413d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.511+0000 7f836a86f640 1 --2- 172.21.15.139:0/711592368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83640cee50 0x7f83640c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.511+0000 7f836a86f640 1 -- 172.21.15.139:0/711592368 >> 172.21.15.139:0/711592368 conn(0x7f83640b5480 msgr2=0x7f83640b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.511+0000 7f836a86f640 1 -- 172.21.15.139:0/711592368 shutdown_connections 2024-04-25T10:01:59.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.511+0000 7f836a86f640 1 -- 172.21.15.139:0/711592368 wait complete. 2024-04-25T10:01:59.518 INFO:tasks.ceph.mds.c.smithi139.stderr: -6768> 2024-04-25T10:01:21.411+0000 7f1f0fc2b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:01:59.527 DEBUG:teuthology.orchestra.run.smithi139:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-25T10:01:59.548 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-04-25T10:01:59.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb499720280 1 -- start start 2024-04-25T10:01:59.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb499720280 1 Processor -- start 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb499720280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbe000 0x55e1e5fc5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb499720280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6cbf000 0x55e1e5fc4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb499720280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6cbf400 0x55e1e5fc5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb499720280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e6d2a780 con 0x55e1e6cbe000 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb499720280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e6d2a600 con 0x55e1e6cbf400 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb499720280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1e6d2a180 con 0x55e1e6cbf000 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb49740e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbe000 0x55e1e5fc5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb49740e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbe000 0x55e1e5fc5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46138/0 (socket says 172.21.15.139:46138) 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb49740e640 1 -- 172.21.15.139:0/374679161 learned_addr learned my addr 172.21.15.139:0/374679161 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb496c0d640 1 --2- 172.21.15.139:0/374679161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6cbf000 0x55e1e5fc4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.564+0000 7fb497c0f640 1 --2- 172.21.15.139:0/374679161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6cbf400 0x55e1e5fc5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb49740e640 1 -- 172.21.15.139:0/374679161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6cbf000 msgr2=0x55e1e5fc4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb49740e640 1 --2- 172.21.15.139:0/374679161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6cbf000 0x55e1e5fc4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb49740e640 1 -- 172.21.15.139:0/374679161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6cbf400 msgr2=0x55e1e5fc5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb49740e640 1 --2- 172.21.15.139:0/374679161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6cbf400 0x55e1e5fc5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb49740e640 1 -- 172.21.15.139:0/374679161 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1e5f8a780 con 0x55e1e6cbe000 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb49740e640 1 --2- 172.21.15.139:0/374679161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbe000 0x55e1e5fc5080 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x55e1e6d7b260 tx=0x55e1e6d184e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17ea92ebf3309e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb494c09640 1 -- 172.21.15.139:0/374679161 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e1e5f7a9c0 con 0x55e1e6cbe000 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb494c09640 1 -- 172.21.15.139:0/374679161 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1e5f80e00 con 0x55e1e6cbe000 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb494c09640 1 -- 172.21.15.139:0/374679161 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e1e5f7a680 con 0x55e1e6cbe000 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb499720280 1 -- 172.21.15.139:0/374679161 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1e5f8b4a0 con 0x55e1e6cbe000 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb499720280 1 -- 172.21.15.139:0/374679161 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55e1e5f8a960 con 0x55e1e6cbe000 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb499720280 4 waiting for MDS map... 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb494c09640 1 -- 172.21.15.139:0/374679161 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x55e1e6d4e000 con 0x55e1e6cbe000 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb494c09640 1 -- 172.21.15.139:0/374679161 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 70) v1 ==== 2723+0+0 (secure 0 0 0) 0x55e1e5faf180 con 0x55e1e6cbe000 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb499720280 4 Got MDS map 70 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb499720280 10 main: JournalTool::main 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.565+0000 7fb499720280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:01:59.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.566+0000 7fb499720280 1 Processor -- start 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.566+0000 7fb499720280 1 -- start start 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.566+0000 7fb499720280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbfc00 0x55e1e5fc4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.566+0000 7fb499720280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6e84000 0x55e1e5fc6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.566+0000 7fb499720280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6e84400 0x55e1e5fc6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.566+0000 7fb499720280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e5fb6480 con 0x55e1e6cbfc00 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.566+0000 7fb499720280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e5fb6900 con 0x55e1e6e84400 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.566+0000 7fb499720280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1e6d2b200 con 0x55e1e6e84000 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.566+0000 7fb497c0f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbfc00 0x55e1e5fc4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.566+0000 7fb497c0f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbfc00 0x55e1e5fc4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46154/0 (socket says 172.21.15.139:46154) 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb497c0f640 1 -- 172.21.15.139:0/2886863433 learned_addr learned my addr 172.21.15.139:0/2886863433 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb49740e640 1 --2- 172.21.15.139:0/2886863433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6e84000 0x55e1e5fc6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb496c0d640 1 --2- 172.21.15.139:0/2886863433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6e84400 0x55e1e5fc6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb497c0f640 1 -- 172.21.15.139:0/2886863433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6e84000 msgr2=0x55e1e5fc6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb497c0f640 1 --2- 172.21.15.139:0/2886863433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6e84000 0x55e1e5fc6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb497c0f640 1 -- 172.21.15.139:0/2886863433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6e84400 msgr2=0x55e1e5fc6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb497c0f640 1 --2- 172.21.15.139:0/2886863433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6e84400 0x55e1e5fc6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb497c0f640 1 -- 172.21.15.139:0/2886863433 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1e5f8a3c0 con 0x55e1e6cbfc00 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb496c0d640 1 --2- 172.21.15.139:0/2886863433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6e84400 0x55e1e5fc6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb497c0f640 1 --2- 172.21.15.139:0/2886863433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbfc00 0x55e1e5fc4580 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x55e1e6e265d0 tx=0x55e1e6e2cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24afb75a859ef897 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb492c05640 1 -- 172.21.15.139:0/2886863433 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e1e5f7a9c0 con 0x55e1e6cbfc00 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb492c05640 1 -- 172.21.15.139:0/2886863433 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1e5f80700 con 0x55e1e6cbfc00 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb492c05640 1 -- 172.21.15.139:0/2886863433 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e1e5f7a680 con 0x55e1e6cbfc00 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb499720280 1 -- 172.21.15.139:0/2886863433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbfc00 msgr2=0x55e1e5fc4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb499720280 1 --2- 172.21.15.139:0/2886863433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbfc00 0x55e1e5fc4580 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x55e1e6e265d0 tx=0x55e1e6e2cc00 comp rx=0 tx=0).stop 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.567+0000 7fb499720280 1 -- 172.21.15.139:0/2886863433 shutdown_connections 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.568+0000 7fb499720280 1 --2- 172.21.15.139:0/2886863433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6e84400 0x55e1e5fc6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.568+0000 7fb499720280 1 --2- 172.21.15.139:0/2886863433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6e84000 0x55e1e5fc6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.568+0000 7fb499720280 1 --2- 172.21.15.139:0/2886863433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbfc00 0x55e1e5fc4580 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.568+0000 7fb499720280 1 -- 172.21.15.139:0/2886863433 >> 172.21.15.139:0/2886863433 conn(0x55e1e6cbf800 msgr2=0x55e1e6cd3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.568+0000 7fb499720280 1 -- 172.21.15.139:0/2886863433 shutdown_connections 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.568+0000 7fb499720280 1 -- 172.21.15.139:0/2886863433 wait complete. 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.568+0000 7fb499720280 1 Processor -- start 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.568+0000 7fb499720280 1 -- start start 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb499720280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6cbf800 0x55e1e5fc7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb499720280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6e85000 0x55e1e5fc7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb499720280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6e84c00 0x55e1e5fc6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb499720280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e6d2b500 con 0x55e1e6e85000 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb499720280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e6d2b380 con 0x55e1e6e84c00 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb499720280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1e6d2b680 con 0x55e1e6cbf800 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb49740e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6e85000 0x55e1e5fc7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb49740e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6e85000 0x55e1e5fc7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46162/0 (socket says 172.21.15.139:46162) 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb49740e640 1 -- 172.21.15.139:0/1553667864 learned_addr learned my addr 172.21.15.139:0/1553667864 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb497c0f640 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6cbf800 0x55e1e5fc7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb496c0d640 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6e84c00 0x55e1e5fc6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb49740e640 1 -- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6cbf800 msgr2=0x55e1e5fc7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb49740e640 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6cbf800 0x55e1e5fc7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb49740e640 1 -- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6e84c00 msgr2=0x55e1e5fc6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb49740e640 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6e84c00 0x55e1e5fc6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb49740e640 1 -- 172.21.15.139:0/1553667864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1e5f8a780 con 0x55e1e6e85000 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb49740e640 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6e85000 0x55e1e5fc7180 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x55e1e6d7b650 tx=0x55e1e6d185a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb491402640 1 -- 172.21.15.139:0/1553667864 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e1e5f7a9c0 con 0x55e1e6e85000 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb491402640 1 -- 172.21.15.139:0/1553667864 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1e5f80c40 con 0x55e1e6e85000 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb491402640 1 -- 172.21.15.139:0/1553667864 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e1e5f7a680 con 0x55e1e6e85000 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1e5f8a000 con 0x55e1e6e85000 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1e5f8b680 con 0x55e1e6e85000 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.569+0000 7fb499720280 4 main: JournalTool: resolving pool 15 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.570+0000 7fb491402640 1 -- 172.21.15.139:0/1553667864 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55e1e6e9cc00 con 0x55e1e6e85000 2024-04-25T10:01:59.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.571+0000 7fb491402640 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55e1e6ea5000 0x55e1e62ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.571+0000 7fb491402640 1 -- 172.21.15.139:0/1553667864 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x55e1e6e9ae00 con 0x55e1e6e85000 2024-04-25T10:01:59.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.571+0000 7fb497c0f640 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55e1e6ea5000 0x55e1e62ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.571+0000 7fb499720280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:01:59.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.571+0000 7fb499720280 4 main: Executing for rank 0 2024-04-25T10:01:59.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.571+0000 7fb499720280 4 Filtering by inode '1099511627777' 2024-04-25T10:01:59.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.571+0000 7fb499720280 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55e1e6e84800 0x55e1e5fc6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.571+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x55e1e6e84400 con 0x55e1e6e84800 2024-04-25T10:01:59.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.571+0000 7fb497c0f640 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55e1e6ea5000 0x55e1e62ce000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55e1e6e265d0 tx=0x55e1e6d18300 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.571+0000 7fb496c0d640 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55e1e6e84800 0x55e1e5fc6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.572+0000 7fb496c0d640 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55e1e6e84800 0x55e1e5fc6680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.574+0000 7fb496c0d640 1 -- 172.21.15.139:0/1553667864 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55e1e6cd4000 con 0x55e1e6e84800 2024-04-25T10:01:59.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.574+0000 7fb499720280 4 Pointer 400.00000000 is readable 2024-04-25T10:01:59.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.574+0000 7fb499720280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T10:01:59.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.574+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x55e1e6e84000 con 0x55e1e6e84800 2024-04-25T10:01:59.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb496c0d640 1 -- 172.21.15.139:0/1553667864 <== osd.6 v2:172.21.15.150:6809/3369628189 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55e1e6cd4000 con 0x55e1e6e84800 2024-04-25T10:01:59.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb499720280 10 Header 0x400000 0x407d21 0x40809d 2024-04-25T10:01:59.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb499720280 10 Starting journal scan from offset 0x407d21 2024-04-25T10:01:59.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x55e1e6ea5400 con 0x55e1e6e84800 2024-04-25T10:01:59.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb496c0d640 1 -- 172.21.15.139:0/1553667864 <== osd.6 v2:172.21.15.150:6809/3369628189 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x55e1e6cd4000 con 0x55e1e6e84800 2024-04-25T10:01:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb499720280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T10:01:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb499720280 10 Parsing data, 0x37c bytes available 2024-04-25T10:01:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb499720280 10 Attempting decode at 0x407d21 2024-04-25T10:01:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb499720280 10 Consumed 0x37c bytes 2024-04-25T10:01:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb499720280 10 Valid entry at 0x407d21 2024-04-25T10:01:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb499720280 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55e1e6ea5800 0x55e1e5fc6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x55e1e6ea5c00 con 0x55e1e6ea5800 2024-04-25T10:01:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.575+0000 7fb497c0f640 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55e1e6ea5800 0x55e1e5fc6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.576+0000 7fb497c0f640 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55e1e6ea5800 0x55e1e5fc6100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.585 INFO:teuthology.orchestra.run.smithi139.stdout:Events by type: 2024-04-25T10:01:59.585 INFO:teuthology.orchestra.run.smithi139.stdout:Errors: 0 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb497c0f640 1 -- 172.21.15.139:0/1553667864 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55e1e6cd4500 con 0x55e1e6ea5800 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 4 Reached end of journal objects 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 4 Scanned objects, 0 missing, 1 valid 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 4 Events scanned, 0 gaps 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 4 Found 1 valid events 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 4 Selected 0 events events for processing 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 4 0 events 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55e1e6e84800 msgr2=0x55e1e5fc6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55e1e6e84800 0x55e1e5fc6680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55e1e6ea5800 msgr2=0x55e1e5fc6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55e1e6ea5800 0x55e1e5fc6100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55e1e6ea5000 msgr2=0x55e1e62ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55e1e6ea5000 0x55e1e62ce000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55e1e6e265d0 tx=0x55e1e6d18300 comp rx=0 tx=0).stop 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6e85000 msgr2=0x55e1e5fc7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 --2- 172.21.15.139:0/1553667864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6e85000 0x55e1e5fc7180 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x55e1e6d7b650 tx=0x55e1e6d185a0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb496c0d640 1 -- 172.21.15.139:0/1553667864 reap_dead start 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb496c0d640 1 -- 172.21.15.139:0/1553667864 reap_dead start 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 shutdown_connections 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 >> 172.21.15.139:0/1553667864 conn(0x55e1e6cbfc00 msgr2=0x55e1e6cd3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 shutdown_connections 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/1553667864 wait complete. 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/374679161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbe000 msgr2=0x55e1e5fc5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 --2- 172.21.15.139:0/374679161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbe000 0x55e1e5fc5080 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x55e1e6d7b260 tx=0x55e1e6d184e0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/374679161 shutdown_connections 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 --2- 172.21.15.139:0/374679161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e1e6cbf400 0x55e1e5fc5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 --2- 172.21.15.139:0/374679161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e1e6cbf000 0x55e1e5fc4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 --2- 172.21.15.139:0/374679161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e1e6cbe000 0x55e1e5fc5080 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/374679161 >> 172.21.15.139:0/374679161 conn(0x55e1e5f55000 msgr2=0x55e1e6cd2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/374679161 shutdown_connections 2024-04-25T10:01:59.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.578+0000 7fb499720280 1 -- 172.21.15.139:0/374679161 wait complete. 2024-04-25T10:01:59.590 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.063350, result: Events by type: Errors: 0 2024-04-25T10:01:59.590 DEBUG:teuthology.orchestra.run.smithi139:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-25T10:01:59.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.665+0000 7fe089be0280 1 -- start start 2024-04-25T10:01:59.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.665+0000 7fe089be0280 1 Processor -- start 2024-04-25T10:01:59.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe089be0280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b74815ac00 0x55b7481ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe089be0280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b74815b800 0x55b7481cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe089be0280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b74815bc00 0x55b7481cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe089be0280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b749012180 con 0x55b74815ac00 2024-04-25T10:01:59.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe089be0280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b749012300 con 0x55b74815b800 2024-04-25T10:01:59.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe089be0280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b749012480 con 0x55b74815bc00 2024-04-25T10:01:59.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe08838c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b74815ac00 0x55b7481ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe08838c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b74815ac00 0x55b7481ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46194/0 (socket says 172.21.15.139:46194) 2024-04-25T10:01:59.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe08838c640 1 -- 172.21.15.139:0/1068540529 learned_addr learned my addr 172.21.15.139:0/1068540529 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe088b8d640 1 --2- 172.21.15.139:0/1068540529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b74815bc00 0x55b7481cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe087b8b640 1 --2- 172.21.15.139:0/1068540529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b74815b800 0x55b7481cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe08838c640 1 -- 172.21.15.139:0/1068540529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b74815bc00 msgr2=0x55b7481cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe08838c640 1 --2- 172.21.15.139:0/1068540529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b74815bc00 0x55b7481cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe08838c640 1 -- 172.21.15.139:0/1068540529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b74815b800 msgr2=0x55b7481cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe08838c640 1 --2- 172.21.15.139:0/1068540529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b74815b800 0x55b7481cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe08838c640 1 -- 172.21.15.139:0/1068540529 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b748190780 con 0x55b74815ac00 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe08838c640 1 --2- 172.21.15.139:0/1068540529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b74815ac00 0x55b7481ca580 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x55b748f7b2c0 tx=0x55b74821ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5237d69bd6f73807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe085b87640 1 -- 172.21.15.139:0/1068540529 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b7481809c0 con 0x55b74815ac00 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe085b87640 1 -- 172.21.15.139:0/1068540529 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b748186fc0 con 0x55b74815ac00 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe085b87640 1 -- 172.21.15.139:0/1068540529 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b748180680 con 0x55b74815ac00 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe089be0280 1 -- 172.21.15.139:0/1068540529 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7481903c0 con 0x55b74815ac00 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe089be0280 1 -- 172.21.15.139:0/1068540529 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b748191680 con 0x55b74815ac00 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.666+0000 7fe089be0280 4 waiting for MDS map... 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.667+0000 7fe085b87640 1 -- 172.21.15.139:0/1068540529 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x55b748f28000 con 0x55b74815ac00 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.667+0000 7fe085b87640 1 -- 172.21.15.139:0/1068540529 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 70) v1 ==== 2723+0+0 (secure 0 0 0) 0x55b7481b5180 con 0x55b74815ac00 2024-04-25T10:01:59.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.667+0000 7fe089be0280 4 Got MDS map 70 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.667+0000 7fe089be0280 10 main: JournalTool::main 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.667+0000 7fe089be0280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.667+0000 7fe089be0280 1 Processor -- start 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.667+0000 7fe089be0280 1 -- start start 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe089be0280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b748f39c00 0x55b7481cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe089be0280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b749086000 0x55b7481cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe089be0280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b749086400 0x55b7481cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe089be0280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b749012600 con 0x55b748f39c00 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe089be0280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b749012780 con 0x55b749086400 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe089be0280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b749012900 con 0x55b749086000 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe088b8d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b748f39c00 0x55b7481cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe088b8d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b748f39c00 0x55b7481cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46204/0 (socket says 172.21.15.139:46204) 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe088b8d640 1 -- 172.21.15.139:0/343306007 learned_addr learned my addr 172.21.15.139:0/343306007 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe087b8b640 1 --2- 172.21.15.139:0/343306007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b749086400 0x55b7481cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe08838c640 1 --2- 172.21.15.139:0/343306007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b749086000 0x55b7481cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe088b8d640 1 -- 172.21.15.139:0/343306007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b749086000 msgr2=0x55b7481cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe088b8d640 1 --2- 172.21.15.139:0/343306007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b749086000 0x55b7481cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe088b8d640 1 -- 172.21.15.139:0/343306007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b749086400 msgr2=0x55b7481cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe088b8d640 1 --2- 172.21.15.139:0/343306007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b749086400 0x55b7481cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe088b8d640 1 -- 172.21.15.139:0/343306007 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7481914a0 con 0x55b748f39c00 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe088b8d640 1 --2- 172.21.15.139:0/343306007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b748f39c00 0x55b7481cb600 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x55b74901a8d0 tx=0x55b74901cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794126bde3aa318b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.668+0000 7fe083b83640 1 -- 172.21.15.139:0/343306007 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b7481809c0 con 0x55b748f39c00 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.669+0000 7fe083b83640 1 -- 172.21.15.139:0/343306007 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b748186c40 con 0x55b748f39c00 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.669+0000 7fe083b83640 1 -- 172.21.15.139:0/343306007 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b748180680 con 0x55b748f39c00 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.669+0000 7fe089be0280 1 -- 172.21.15.139:0/343306007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b748f39c00 msgr2=0x55b7481cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.669+0000 7fe089be0280 1 --2- 172.21.15.139:0/343306007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b748f39c00 0x55b7481cb600 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x55b74901a8d0 tx=0x55b74901cc00 comp rx=0 tx=0).stop 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.669+0000 7fe089be0280 1 -- 172.21.15.139:0/343306007 shutdown_connections 2024-04-25T10:01:59.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.669+0000 7fe089be0280 1 --2- 172.21.15.139:0/343306007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b749086400 0x55b7481cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.669+0000 7fe089be0280 1 --2- 172.21.15.139:0/343306007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b749086000 0x55b7481cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.669+0000 7fe089be0280 1 --2- 172.21.15.139:0/343306007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b748f39c00 0x55b7481cb600 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.669+0000 7fe089be0280 1 -- 172.21.15.139:0/343306007 >> 172.21.15.139:0/343306007 conn(0x55b748f39800 msgr2=0x55b748255000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.669+0000 7fe089be0280 1 -- 172.21.15.139:0/343306007 shutdown_connections 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.669+0000 7fe089be0280 1 -- 172.21.15.139:0/343306007 wait complete. 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe089be0280 1 Processor -- start 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe089be0280 1 -- start start 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe089be0280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b748f39800 0x55b7481cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe089be0280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b749087000 0x55b7481ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe089be0280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b749086c00 0x55b7481cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe089be0280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b749013c80 con 0x55b749086c00 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe089be0280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b749013b00 con 0x55b749087000 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe089be0280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b749013980 con 0x55b748f39800 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe087b8b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b749086c00 0x55b7481cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe087b8b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b749086c00 0x55b7481cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46210/0 (socket says 172.21.15.139:46210) 2024-04-25T10:01:59.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe087b8b640 1 -- 172.21.15.139:0/1019350610 learned_addr learned my addr 172.21.15.139:0/1019350610 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe08838c640 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b749087000 0x55b7481ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe088b8d640 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b748f39800 0x55b7481cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe087b8b640 1 -- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b748f39800 msgr2=0x55b7481cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe087b8b640 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b748f39800 0x55b7481cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe087b8b640 1 -- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b749087000 msgr2=0x55b7481ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe087b8b640 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b749087000 0x55b7481ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.670+0000 7fe087b8b640 1 -- 172.21.15.139:0/1019350610 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b748190000 con 0x55b749086c00 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.671+0000 7fe087b8b640 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b749086c00 0x55b7481cc680 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x55b74901b5f0 tx=0x55b74901cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.671+0000 7fe082380640 1 -- 172.21.15.139:0/1019350610 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b7481809c0 con 0x55b749086c00 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.671+0000 7fe082380640 1 -- 172.21.15.139:0/1019350610 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b748186a80 con 0x55b749086c00 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.671+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b748190960 con 0x55b749086c00 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.671+0000 7fe082380640 1 -- 172.21.15.139:0/1019350610 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b748180680 con 0x55b749086c00 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.671+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b748191860 con 0x55b749086c00 2024-04-25T10:01:59.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.671+0000 7fe089be0280 4 main: JournalTool: resolving pool 15 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.672+0000 7fe082380640 1 -- 172.21.15.139:0/1019350610 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55b749092c00 con 0x55b749086c00 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.672+0000 7fe082380640 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55b74909ec00 0x55b7481cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.672+0000 7fe082380640 1 -- 172.21.15.139:0/1019350610 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x55b749091000 con 0x55b749086c00 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.672+0000 7fe088b8d640 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55b74909ec00 0x55b7481cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.672+0000 7fe089be0280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.672+0000 7fe089be0280 4 main: Executing for rank 0 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.672+0000 7fe089be0280 4 Filtering by inode '1099511627778' 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.672+0000 7fe089be0280 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55b749086800 0x55b7481cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.672+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x55b749086400 con 0x55b749086800 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.673+0000 7fe08838c640 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55b749086800 0x55b7481cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.673+0000 7fe088b8d640 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55b74909ec00 0x55b7481cd700 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x55b74901a8d0 tx=0x55b748f22180 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.673+0000 7fe08838c640 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55b749086800 0x55b7481cc100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.673+0000 7fe08838c640 1 -- 172.21.15.139:0/1019350610 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55b748eda000 con 0x55b749086800 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.673+0000 7fe089be0280 4 Pointer 400.00000000 is readable 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.673+0000 7fe089be0280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T10:01:59.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.673+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x55b749086000 con 0x55b749086800 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.673+0000 7fe08838c640 1 -- 172.21.15.139:0/1019350610 <== osd.6 v2:172.21.15.150:6809/3369628189 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55b748eda000 con 0x55b749086800 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.673+0000 7fe089be0280 10 Header 0x400000 0x407d21 0x40809d 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.674+0000 7fe089be0280 10 Starting journal scan from offset 0x407d21 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.674+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x55b74909f000 con 0x55b749086800 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.674+0000 7fe08838c640 1 -- 172.21.15.139:0/1019350610 <== osd.6 v2:172.21.15.150:6809/3369628189 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x55b748eda000 con 0x55b749086800 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.674+0000 7fe089be0280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.674+0000 7fe089be0280 10 Parsing data, 0x37c bytes available 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.674+0000 7fe089be0280 10 Attempting decode at 0x407d21 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.674+0000 7fe089be0280 10 Consumed 0x37c bytes 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.674+0000 7fe089be0280 10 Valid entry at 0x407d21 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.674+0000 7fe089be0280 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55b74909f400 0x55b7481cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.674+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x55b74909f800 con 0x55b74909f400 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.674+0000 7fe088b8d640 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55b74909f400 0x55b7481cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.674+0000 7fe088b8d640 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55b74909f400 0x55b7481cbb80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe088b8d640 1 -- 172.21.15.139:0/1019350610 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b748eda500 con 0x55b74909f400 2024-04-25T10:01:59.687 INFO:teuthology.orchestra.run.smithi139.stdout:Events by type: 2024-04-25T10:01:59.687 INFO:teuthology.orchestra.run.smithi139.stdout:Errors: 0 2024-04-25T10:01:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 4 Reached end of journal objects 2024-04-25T10:01:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 4 Scanned objects, 0 missing, 1 valid 2024-04-25T10:01:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 4 Events scanned, 0 gaps 2024-04-25T10:01:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 4 Found 1 valid events 2024-04-25T10:01:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 4 Selected 0 events events for processing 2024-04-25T10:01:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 4 0 events 2024-04-25T10:01:59.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55b749086800 msgr2=0x55b7481cc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55b749086800 0x55b7481cc100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55b74909f400 msgr2=0x55b7481cbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55b74909f400 0x55b7481cbb80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55b74909ec00 msgr2=0x55b7481cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55b74909ec00 0x55b7481cd700 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x55b74901a8d0 tx=0x55b748f22180 comp rx=0 tx=0).stop 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b749086c00 msgr2=0x55b7481cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b749086c00 0x55b7481cc680 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x55b74901b5f0 tx=0x55b74901cd20 comp rx=0 tx=0).stop 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe087b8b640 1 -- 172.21.15.139:0/1019350610 reap_dead start 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 shutdown_connections 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 --2- 172.21.15.139:0/1019350610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b749086c00 0x55b7481cc680 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 >> 172.21.15.139:0/1019350610 conn(0x55b748f39c00 msgr2=0x55b748255000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 shutdown_connections 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1019350610 wait complete. 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1068540529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b74815ac00 msgr2=0x55b7481ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 --2- 172.21.15.139:0/1068540529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b74815ac00 0x55b7481ca580 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x55b748f7b2c0 tx=0x55b74821ee40 comp rx=0 tx=0).stop 2024-04-25T10:01:59.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1068540529 shutdown_connections 2024-04-25T10:01:59.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 --2- 172.21.15.139:0/1068540529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b74815bc00 0x55b7481cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 --2- 172.21.15.139:0/1068540529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b74815b800 0x55b7481cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 --2- 172.21.15.139:0/1068540529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b74815ac00 0x55b7481ca580 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1068540529 >> 172.21.15.139:0/1068540529 conn(0x55b74815b000 msgr2=0x55b748254800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1068540529 shutdown_connections 2024-04-25T10:01:59.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.675+0000 7fe089be0280 1 -- 172.21.15.139:0/1068540529 wait complete. 2024-04-25T10:01:59.691 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.101232, result: Events by type: Errors: 0 2024-04-25T10:01:59.691 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-25T10:01:59.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.767+0000 7fdc9d40cf40 1 Processor -- start 2024-04-25T10:01:59.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.767+0000 7fdc9d40cf40 1 -- start start 2024-04-25T10:01:59.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9d40cf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc840030f0 0x5621c077cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9d40cf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c077d490 0x5621c06d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9d40cf40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 0x5621c06d8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9d40cf40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c0653df0 con 0x5621c06d6680 2024-04-25T10:01:59.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9d40cf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c0657d50 con 0x5621c077d490 2024-04-25T10:01:59.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9d40cf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621c06f4b40 con 0x7fdc840030f0 2024-04-25T10:01:59.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9aafd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 0x5621c06d8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9aafd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 0x5621c06d8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46236/0 (socket says 172.21.15.139:46236) 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9aafd640 1 -- 172.21.15.139:0/1728290464 learned_addr learned my addr 172.21.15.139:0/1728290464 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9aafd640 1 -- 172.21.15.139:0/1728290464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc840030f0 msgr2=0x5621c077cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9aafd640 1 --2- 172.21.15.139:0/1728290464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc840030f0 0x5621c077cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9aafd640 1 -- 172.21.15.139:0/1728290464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c077d490 msgr2=0x5621c06d7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9aafd640 1 --2- 172.21.15.139:0/1728290464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c077d490 0x5621c06d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9aafd640 1 -- 172.21.15.139:0/1728290464 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621c077e0d0 con 0x5621c06d6680 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc9aafd640 1 --2- 172.21.15.139:0/1728290464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 0x5621c06d8170 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x5621c06f7110 tx=0x7fdc9400e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd528d69f863e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.768+0000 7fdc98af9640 1 -- 172.21.15.139:0/1728290464 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400ed90 con 0x5621c06d6680 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.769+0000 7fdc98af9640 1 -- 172.21.15.139:0/1728290464 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc94001ac0 con 0x5621c06d6680 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.769+0000 7fdc98af9640 1 -- 172.21.15.139:0/1728290464 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94001c90 con 0x5621c06d6680 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.769+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/1728290464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 msgr2=0x5621c06d8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.769+0000 7fdc9d40cf40 1 --2- 172.21.15.139:0/1728290464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 0x5621c06d8170 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x5621c06f7110 tx=0x7fdc9400e6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.769+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/1728290464 shutdown_connections 2024-04-25T10:01:59.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.769+0000 7fdc9d40cf40 1 --2- 172.21.15.139:0/1728290464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 0x5621c06d8170 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.769+0000 7fdc9d40cf40 1 --2- 172.21.15.139:0/1728290464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c077d490 0x5621c06d7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.769+0000 7fdc9d40cf40 1 --2- 172.21.15.139:0/1728290464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc840030f0 0x5621c077cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.769+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/1728290464 >> 172.21.15.139:0/1728290464 conn(0x5621c06acbe0 msgr2=0x5621c06ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.769+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/1728290464 shutdown_connections 2024-04-25T10:01:59.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.769+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/1728290464 wait complete. 2024-04-25T10:01:59.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.770+0000 7fdc9d40cf40 1 Processor -- start 2024-04-25T10:01:59.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.770+0000 7fdc9d40cf40 1 -- start start 2024-04-25T10:01:59.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.770+0000 7fdc9d40cf40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 0x5621c06d8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.770+0000 7fdc9d40cf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c077d490 0x5621c0781a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.770+0000 7fdc9d40cf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc840030f0 0x5621c0773690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.770+0000 7fdc9d40cf40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c06d6d10 con 0x5621c06d6680 2024-04-25T10:01:59.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.770+0000 7fdc9d40cf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c077de30 con 0x5621c077d490 2024-04-25T10:01:59.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.770+0000 7fdc9d40cf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621c06aed40 con 0x7fdc840030f0 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.770+0000 7fdc9a2fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 0x5621c06d8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.770+0000 7fdc9a2fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 0x5621c06d8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46240/0 (socket says 172.21.15.139:46240) 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.770+0000 7fdc9a2fc640 1 -- 172.21.15.139:0/781021647 learned_addr learned my addr 172.21.15.139:0/781021647 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc9aafd640 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc840030f0 0x5621c0773690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc99afb640 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c077d490 0x5621c0781a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc9a2fc640 1 -- 172.21.15.139:0/781021647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc840030f0 msgr2=0x5621c0773690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc9a2fc640 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc840030f0 0x5621c0773690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc9a2fc640 1 -- 172.21.15.139:0/781021647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c077d490 msgr2=0x5621c0781a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc9a2fc640 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c077d490 0x5621c0781a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc9a2fc640 1 -- 172.21.15.139:0/781021647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621c0783a20 con 0x5621c06d6680 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc99afb640 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c077d490 0x5621c0781a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc9a2fc640 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 0x5621c06d8140 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c010550 tx=0x7fdc8c010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc8b7fe640 1 -- 172.21.15.139:0/781021647 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c017030 con 0x5621c06d6680 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc8b7fe640 1 -- 172.21.15.139:0/781021647 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c013710 con 0x5621c06d6680 2024-04-25T10:01:59.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc8b7fe640 1 -- 172.21.15.139:0/781021647 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c013900 con 0x5621c06d6680 2024-04-25T10:01:59.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/781021647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5621c0774ee0 con 0x5621c06d6680 2024-04-25T10:01:59.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.771+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/781021647 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621c07754a0 con 0x5621c06d6680 2024-04-25T10:01:59.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.772+0000 7fdc8b7fe640 1 -- 172.21.15.139:0/781021647 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc8c01a030 con 0x5621c06d6680 2024-04-25T10:01:59.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.773+0000 7fdc8b7fe640 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdc7c069eb0 0x7fdc7c06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.773+0000 7fdc99afb640 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdc7c069eb0 0x7fdc7c06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.773+0000 7fdc8b7fe640 1 -- 172.21.15.139:0/781021647 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fdc8c0893c0 con 0x5621c06d6680 2024-04-25T10:01:59.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.773+0000 7fdc9d40cf40 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5621c06da960 0x5621c07759c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.773+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/781021647 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v9 -- 0x5621c0775f00 con 0x5621c06da960 2024-04-25T10:01:59.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.773+0000 7fdc9aafd640 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5621c06da960 0x5621c07759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.773+0000 7fdc99afb640 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdc7c069eb0 0x7fdc7c06c370 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdc90056000 tx=0x7fdc90050000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.773+0000 7fdc9aafd640 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5621c06da960 0x5621c07759c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.776+0000 7fdc9aafd640 1 -- 172.21.15.139:0/781021647 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v108'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdc94011450 con 0x5621c06da960 2024-04-25T10:01:59.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.776+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5621c06da960 msgr2=0x5621c07759c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.776+0000 7fdc9d40cf40 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5621c06da960 0x5621c07759c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.776+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdc7c069eb0 msgr2=0x7fdc7c06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.776+0000 7fdc9d40cf40 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdc7c069eb0 0x7fdc7c06c370 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdc90056000 tx=0x7fdc90050000 comp rx=0 tx=0).stop 2024-04-25T10:01:59.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.776+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 msgr2=0x5621c06d8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.776+0000 7fdc9d40cf40 1 --2- 172.21.15.139:0/781021647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5621c06d6680 0x5621c06d8140 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c010550 tx=0x7fdc8c010a10 comp rx=0 tx=0).stop 2024-04-25T10:01:59.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.776+0000 7fdc9aafd640 1 -- 172.21.15.139:0/781021647 reap_dead start 2024-04-25T10:01:59.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.776+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/781021647 shutdown_connections 2024-04-25T10:01:59.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.776+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/781021647 >> 172.21.15.139:0/781021647 conn(0x5621c06acbe0 msgr2=0x5621c06ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.776+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/781021647 shutdown_connections 2024-04-25T10:01:59.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.776+0000 7fdc9d40cf40 1 -- 172.21.15.139:0/781021647 wait complete. 2024-04-25T10:01:59.784 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:01:59.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.918+0000 7f0030bfe640 1 Processor -- start 2024-04-25T10:01:59.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.918+0000 7f0030bfe640 1 -- start start 2024-04-25T10:01:59.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.919+0000 7f0030bfe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f002c1617e0 0x7f002c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.919+0000 7f0030bfe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f002c0cec80 0x7f002c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.919+0000 7f0030bfe640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002c161dc0 con 0x7f002c15dbf0 2024-04-25T10:01:59.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.919+0000 7f0030bfe640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f002c0d1ee0 con 0x7f002c0cec80 2024-04-25T10:01:59.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.919+0000 7f0030bfe640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c1620a0 con 0x7f002c1617e0 2024-04-25T10:01:59.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.919+0000 7f002a59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f002c15dbf0 0x7f002c0b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57754/0 (socket says 172.21.15.139:57754) 2024-04-25T10:01:59.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.919+0000 7f002a59c640 1 -- 172.21.15.139:0/2482737500 learned_addr learned my addr 172.21.15.139:0/2482737500 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:01:59.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.919+0000 7f002ad9d640 1 --2- 172.21.15.139:0/2482737500 >> v2:172.21.15.139:3300/0 conn(0x7f002c0cec80 0x7f002c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.920+0000 7f0029d9b640 1 --2- 172.21.15.139:0/2482737500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f002c1617e0 0x7f002c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.920+0000 7f002ad9d640 1 -- 172.21.15.139:0/2482737500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f002c1617e0 msgr2=0x7f002c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.920+0000 7f002ad9d640 1 --2- 172.21.15.139:0/2482737500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f002c1617e0 0x7f002c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.920+0000 7f002ad9d640 1 -- 172.21.15.139:0/2482737500 >> v1:172.21.15.139:6789/0 conn(0x7f002c15dbf0 legacy=0x7f002c0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.920+0000 7f002959a640 1 -- 172.21.15.139:0/2482737500 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235028921 0 0) 0x7f002c161dc0 con 0x7f002c15dbf0 2024-04-25T10:01:59.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.920+0000 7f002ad9d640 1 -- 172.21.15.139:0/2482737500 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c162380 con 0x7f002c0cec80 2024-04-25T10:01:59.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.920+0000 7f002ad9d640 1 --2- 172.21.15.139:0/2482737500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f002c0cec80 0x7f002c0d2370 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f00240097a0 tx=0x7f002402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b010deac86d56527 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.921+0000 7f002959a640 1 -- 172.21.15.139:0/2482737500 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024037070 con 0x7f002c0cec80 2024-04-25T10:01:59.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.921+0000 7f002959a640 1 -- 172.21.15.139:0/2482737500 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002402ae50 con 0x7f002c0cec80 2024-04-25T10:01:59.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.921+0000 7f002959a640 1 -- 172.21.15.139:0/2482737500 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024032660 con 0x7f002c0cec80 2024-04-25T10:01:59.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.921+0000 7f0030bfe640 1 -- 172.21.15.139:0/2482737500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f002c0cec80 msgr2=0x7f002c0d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.921+0000 7f0030bfe640 1 --2- 172.21.15.139:0/2482737500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f002c0cec80 0x7f002c0d2370 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f00240097a0 tx=0x7f002402a100 comp rx=0 tx=0).stop 2024-04-25T10:01:59.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.921+0000 7f0030bfe640 1 -- 172.21.15.139:0/2482737500 shutdown_connections 2024-04-25T10:01:59.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.921+0000 7f0030bfe640 1 --2- 172.21.15.139:0/2482737500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f002c0cec80 0x7f002c0d2370 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.921+0000 7f0030bfe640 1 --2- 172.21.15.139:0/2482737500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f002c1617e0 0x7f002c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.921+0000 7f0030bfe640 1 -- 172.21.15.139:0/2482737500 >> 172.21.15.139:0/2482737500 conn(0x7f002c0b5480 msgr2=0x7f002c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.921+0000 7f0030bfe640 1 -- 172.21.15.139:0/2482737500 shutdown_connections 2024-04-25T10:01:59.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.921+0000 7f0030bfe640 1 -- 172.21.15.139:0/2482737500 wait complete. 2024-04-25T10:01:59.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.923+0000 7f0030bfe640 1 Processor -- start 2024-04-25T10:01:59.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.923+0000 7f0030bfe640 1 -- start start 2024-04-25T10:01:59.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.923+0000 7f0030bfe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f002c0cec80 0x7f002c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.923+0000 7f0030bfe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f002c15dbf0 0x7f002c15f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.923+0000 7f0030bfe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f002c1617e0 0x7f002c141500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.923+0000 7f0030bfe640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c15dfd0 con 0x7f002c15dbf0 2024-04-25T10:01:59.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.923+0000 7f0030bfe640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0d28b0 con 0x7f002c0cec80 2024-04-25T10:01:59.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.923+0000 7f0030bfe640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c152370 con 0x7f002c1617e0 2024-04-25T10:01:59.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.923+0000 7f0029d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f002c15dbf0 0x7f002c15f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.923+0000 7f0029d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f002c15dbf0 0x7f002c15f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46262/0 (socket says 172.21.15.139:46262) 2024-04-25T10:01:59.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.923+0000 7f0029d9b640 1 -- 172.21.15.139:0/599387335 learned_addr learned my addr 172.21.15.139:0/599387335 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:01:59.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.924+0000 7f0029d9b640 1 -- 172.21.15.139:0/599387335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f002c1617e0 msgr2=0x7f002c141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.924+0000 7f0029d9b640 1 --2- 172.21.15.139:0/599387335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f002c1617e0 0x7f002c141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.924+0000 7f0029d9b640 1 -- 172.21.15.139:0/599387335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f002c0cec80 msgr2=0x7f002c154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.924+0000 7f0029d9b640 1 --2- 172.21.15.139:0/599387335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f002c0cec80 0x7f002c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.924+0000 7f0029d9b640 1 -- 172.21.15.139:0/599387335 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c1501e0 con 0x7f002c15dbf0 2024-04-25T10:01:59.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.924+0000 7f0029d9b640 1 --2- 172.21.15.139:0/599387335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f002c15dbf0 0x7f002c15f8e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f002000d5f0 tx=0x7f002000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.924+0000 7f001affd640 1 -- 172.21.15.139:0/599387335 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00200125f0 con 0x7f002c15dbf0 2024-04-25T10:01:59.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.924+0000 7f001affd640 1 -- 172.21.15.139:0/599387335 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0020019070 con 0x7f002c15dbf0 2024-04-25T10:01:59.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.924+0000 7f0030bfe640 1 -- 172.21.15.139:0/599387335 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002c1504c0 con 0x7f002c15dbf0 2024-04-25T10:01:59.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.924+0000 7f001affd640 1 -- 172.21.15.139:0/599387335 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00200129d0 con 0x7f002c15dbf0 2024-04-25T10:01:59.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.924+0000 7f0030bfe640 1 -- 172.21.15.139:0/599387335 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c14fe70 con 0x7f002c15dbf0 2024-04-25T10:01:59.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.925+0000 7f0030bfe640 1 -- 172.21.15.139:0/599387335 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002c074520 con 0x7f002c15dbf0 2024-04-25T10:01:59.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.925+0000 7f001affd640 1 -- 172.21.15.139:0/599387335 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0020012b70 con 0x7f002c15dbf0 2024-04-25T10:01:59.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.926+0000 7f001affd640 1 --2- 172.21.15.139:0/599387335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f000c06b500 0x7f000c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.926+0000 7f002a59c640 1 --2- 172.21.15.139:0/599387335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f000c06b500 0x7f000c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.926+0000 7f001affd640 1 -- 172.21.15.139:0/599387335 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f002008a770 con 0x7f002c15dbf0 2024-04-25T10:01:59.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.926+0000 7f002a59c640 1 --2- 172.21.15.139:0/599387335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f000c06b500 0x7f000c06d9c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f001c001560 tx=0x7f001c007ca0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:01:59.930+0000 7f001affd640 1 -- 172.21.15.139:0/599387335 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00200593d0 con 0x7f002c15dbf0 2024-04-25T10:02:00.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.139+0000 7f0030bfe640 1 -- 172.21.15.139:0/599387335 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f002c0d1680 con 0x7f002c15dbf0 2024-04-25T10:02:00.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.509+0000 7f001affd640 1 -- 172.21.15.139:0/599387335 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v71) v1 ==== 162+0+0 (secure 0 0 0) 0x7f0020058cd0 con 0x7f002c15dbf0 2024-04-25T10:02:00.514 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.511+0000 7f0030bfe640 1 -- 172.21.15.139:0/599387335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f000c06b500 msgr2=0x7f000c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.511+0000 7f0030bfe640 1 --2- 172.21.15.139:0/599387335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f000c06b500 0x7f000c06d9c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f001c001560 tx=0x7f001c007ca0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.511+0000 7f0030bfe640 1 -- 172.21.15.139:0/599387335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f002c15dbf0 msgr2=0x7f002c15f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.511+0000 7f0030bfe640 1 --2- 172.21.15.139:0/599387335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f002c15dbf0 0x7f002c15f8e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f002000d5f0 tx=0x7f002000dab0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.512+0000 7f0030bfe640 1 -- 172.21.15.139:0/599387335 shutdown_connections 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.512+0000 7f0030bfe640 1 --2- 172.21.15.139:0/599387335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f002c15dbf0 0x7f002c15f8e0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.512+0000 7f0030bfe640 1 --2- 172.21.15.139:0/599387335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f000c06b500 0x7f000c06d9c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.512+0000 7f0030bfe640 1 --2- 172.21.15.139:0/599387335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f002c1617e0 0x7f002c141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.512+0000 7f0030bfe640 1 --2- 172.21.15.139:0/599387335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f002c0cec80 0x7f002c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.512+0000 7f0030bfe640 1 -- 172.21.15.139:0/599387335 >> 172.21.15.139:0/599387335 conn(0x7f002c0b5480 msgr2=0x7f002c0b6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.512+0000 7f0030bfe640 1 -- 172.21.15.139:0/599387335 shutdown_connections 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.512+0000 7f0030bfe640 1 -- 172.21.15.139:0/599387335 wait complete. 2024-04-25T10:02:00.529 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:00.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.621+0000 7f786acb5640 1 Processor -- start 2024-04-25T10:02:00.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.621+0000 7f786acb5640 1 -- start start 2024-04-25T10:02:00.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.622+0000 7f786acb5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7864158260 0x7f786415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.622+0000 7f786acb5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f786406da20 0x7f78640b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.622+0000 7f786acb5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f78640b3600 0x7f786406def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.622+0000 7f786acb5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78640b3a70 con 0x7f786406da20 2024-04-25T10:02:00.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.622+0000 7f786acb5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78640b3e10 con 0x7f78640b3600 2024-04-25T10:02:00.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.622+0000 7f786acb5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78641615c0 con 0x7f7864158260 2024-04-25T10:02:00.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.622+0000 7f7863fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f786406da20 0x7f78640b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.622+0000 7f7863fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f786406da20 0x7f78640b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46274/0 (socket says 172.21.15.139:46274) 2024-04-25T10:02:00.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.622+0000 7f7863fff640 1 -- 172.21.15.139:0/2699970944 learned_addr learned my addr 172.21.15.139:0/2699970944 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:00.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.622+0000 7f7868a51640 1 --2- 172.21.15.139:0/2699970944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7864158260 0x7f786415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.622+0000 7f7869252640 1 --2- 172.21.15.139:0/2699970944 >> v2:172.21.15.150:3300/0 conn(0x7f78640b3600 0x7f786406def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.623+0000 7f7863fff640 1 -- 172.21.15.139:0/2699970944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7864158260 msgr2=0x7f786415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:00.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.623+0000 7f7863fff640 1 --2- 172.21.15.139:0/2699970944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7864158260 0x7f786415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.623+0000 7f7863fff640 1 -- 172.21.15.139:0/2699970944 >> v2:172.21.15.150:3300/0 conn(0x7f78640b3600 msgr2=0x7f786406def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:00.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.623+0000 7f7863fff640 1 --2- 172.21.15.139:0/2699970944 >> v2:172.21.15.150:3300/0 conn(0x7f78640b3600 0x7f786406def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.623+0000 7f7863fff640 1 -- 172.21.15.139:0/2699970944 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78641618a0 con 0x7f786406da20 2024-04-25T10:02:00.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.623+0000 7f7868a51640 1 --2- 172.21.15.139:0/2699970944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7864158260 0x7f786415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:00.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.623+0000 7f7863fff640 1 --2- 172.21.15.139:0/2699970944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f786406da20 0x7f78640b30c0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f7864066a90 tx=0x7f785802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5839cc0114684f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.623+0000 7f78637fe640 1 -- 172.21.15.139:0/2699970944 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858037070 con 0x7f786406da20 2024-04-25T10:02:00.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.623+0000 7f78637fe640 1 -- 172.21.15.139:0/2699970944 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7858032440 con 0x7f786406da20 2024-04-25T10:02:00.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.624+0000 7f78637fe640 1 -- 172.21.15.139:0/2699970944 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858032810 con 0x7f786406da20 2024-04-25T10:02:00.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.624+0000 7f786acb5640 1 -- 172.21.15.139:0/2699970944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f786406da20 msgr2=0x7f78640b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.624+0000 7f786acb5640 1 --2- 172.21.15.139:0/2699970944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f786406da20 0x7f78640b30c0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f7864066a90 tx=0x7f785802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.624+0000 7f786acb5640 1 -- 172.21.15.139:0/2699970944 shutdown_connections 2024-04-25T10:02:00.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.624+0000 7f786acb5640 1 --2- 172.21.15.139:0/2699970944 >> v2:172.21.15.150:3300/0 conn(0x7f78640b3600 0x7f786406def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.624+0000 7f786acb5640 1 --2- 172.21.15.139:0/2699970944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f786406da20 0x7f78640b30c0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.624+0000 7f786acb5640 1 --2- 172.21.15.139:0/2699970944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7864158260 0x7f786415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.624+0000 7f786acb5640 1 -- 172.21.15.139:0/2699970944 >> 172.21.15.139:0/2699970944 conn(0x7f78640b23d0 msgr2=0x7f78640cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.624+0000 7f786acb5640 1 -- 172.21.15.139:0/2699970944 shutdown_connections 2024-04-25T10:02:00.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.624+0000 7f786acb5640 1 -- 172.21.15.139:0/2699970944 wait complete. 2024-04-25T10:02:00.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.625+0000 7f786acb5640 1 Processor -- start 2024-04-25T10:02:00.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.625+0000 7f786acb5640 1 -- start start 2024-04-25T10:02:00.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.626+0000 7f786acb5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f786406da20 0x7f7864153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.626+0000 7f786acb5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78640b3600 0x7f78641f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.626+0000 7f786acb5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7864158260 0x7f78641e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.626+0000 7f786acb5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7864162140 con 0x7f7864158260 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.626+0000 7f786acb5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7864161e60 con 0x7f78640b3600 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.626+0000 7f786acb5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7864162420 con 0x7f786406da20 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.626+0000 7f7869252640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7864158260 0x7f78641e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.626+0000 7f7869252640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7864158260 0x7f78641e78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46286/0 (socket says 172.21.15.139:46286) 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.626+0000 7f7869252640 1 -- 172.21.15.139:0/1174968568 learned_addr learned my addr 172.21.15.139:0/1174968568 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.626+0000 7f7868a51640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f786406da20 0x7f7864153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.626+0000 7f7863fff640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78640b3600 0x7f78641f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.627+0000 7f7869252640 1 -- 172.21.15.139:0/1174968568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f786406da20 msgr2=0x7f7864153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.627+0000 7f7869252640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f786406da20 0x7f7864153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.627+0000 7f7869252640 1 -- 172.21.15.139:0/1174968568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78640b3600 msgr2=0x7f78641f3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.627+0000 7f7869252640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78640b3600 0x7f78641f3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.627+0000 7f7869252640 1 -- 172.21.15.139:0/1174968568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864154540 con 0x7f7864158260 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.627+0000 7f7868a51640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f786406da20 0x7f7864153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:00.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.627+0000 7f7869252640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7864158260 0x7f78641e78e0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f785c007e90 tx=0x7f785c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.627+0000 7f78617fa640 1 -- 172.21.15.139:0/1174968568 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c014420 con 0x7f7864158260 2024-04-25T10:02:00.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.627+0000 7f78617fa640 1 -- 172.21.15.139:0/1174968568 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c019070 con 0x7f7864158260 2024-04-25T10:02:00.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.627+0000 7f786acb5640 1 -- 172.21.15.139:0/1174968568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7864154820 con 0x7f7864158260 2024-04-25T10:02:00.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.627+0000 7f78617fa640 1 -- 172.21.15.139:0/1174968568 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c014820 con 0x7f7864158260 2024-04-25T10:02:00.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.627+0000 7f786acb5640 1 -- 172.21.15.139:0/1174968568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7864154de0 con 0x7f7864158260 2024-04-25T10:02:00.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.628+0000 7f786acb5640 1 -- 172.21.15.139:0/1174968568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7864074520 con 0x7f7864158260 2024-04-25T10:02:00.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.628+0000 7f78617fa640 1 -- 172.21.15.139:0/1174968568 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f785c0149c0 con 0x7f7864158260 2024-04-25T10:02:00.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.629+0000 7f78617fa640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f785006b500 0x7f785006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.629+0000 7f78617fa640 1 -- 172.21.15.139:0/1174968568 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f785c08ac90 con 0x7f7864158260 2024-04-25T10:02:00.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.629+0000 7f7868a51640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f785006b500 0x7f785006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.629+0000 7f7868a51640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f785006b500 0x7f785006d9c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7854006fd0 tx=0x7f7854008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.633+0000 7f78617fa640 1 -- 172.21.15.139:0/1174968568 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f785c0598f0 con 0x7f7864158260 2024-04-25T10:02:00.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.839+0000 7f786acb5640 1 -- 172.21.15.139:0/1174968568 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78640b31d0 con 0x7f7864158260 2024-04-25T10:02:00.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.840+0000 7f78617fa640 1 -- 172.21.15.139:0/1174968568 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f785c0591f0 con 0x7f7864158260 2024-04-25T10:02:00.845 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:00.845 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":72,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:15.497940+0000","modified":"2024-04-25T10:02:00.512790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8808":{"gid":8808,"name":"c","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6837/359626718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":359626718},{"type":"v1","addr":"172.21.15.139:6837","nonce":359626718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:02:00.845 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 72 2024-04-25T10:02:00.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.842+0000 7f786acb5640 1 -- 172.21.15.139:0/1174968568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f785006b500 msgr2=0x7f785006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.842+0000 7f786acb5640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f785006b500 0x7f785006d9c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7854006fd0 tx=0x7f7854008040 comp rx=0 tx=0).stop 2024-04-25T10:02:00.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.842+0000 7f786acb5640 1 -- 172.21.15.139:0/1174968568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7864158260 msgr2=0x7f78641e78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.842+0000 7f786acb5640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7864158260 0x7f78641e78e0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f785c007e90 tx=0x7f785c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.842+0000 7f786acb5640 1 -- 172.21.15.139:0/1174968568 shutdown_connections 2024-04-25T10:02:00.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.842+0000 7f786acb5640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7864158260 0x7f78641e78e0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.842+0000 7f786acb5640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f785006b500 0x7f785006d9c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.842+0000 7f786acb5640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78640b3600 0x7f78641f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.842+0000 7f786acb5640 1 --2- 172.21.15.139:0/1174968568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f786406da20 0x7f7864153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.842+0000 7f786acb5640 1 -- 172.21.15.139:0/1174968568 >> 172.21.15.139:0/1174968568 conn(0x7f78640b23d0 msgr2=0x7f7864156d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.842+0000 7f786acb5640 1 -- 172.21.15.139:0/1174968568 shutdown_connections 2024-04-25T10:02:00.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:00.842+0000 7f786acb5640 1 -- 172.21.15.139:0/1174968568 wait complete. 2024-04-25T10:02:01.260 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:01:15.497940+0000', 'modified': '2024-04-25T10:02:00.512790+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8808}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8808': {'gid': 8808, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.139:6837/359626718', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6835', 'nonce': 359626718}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 359626718}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:02:01.260 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8808:up:replay 2024-04-25T10:02:02.261 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:02.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.366+0000 7fc1c0177640 1 Processor -- start 2024-04-25T10:02:02.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.367+0000 7fc1c0177640 1 -- start start 2024-04-25T10:02:02.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.367+0000 7fc1c0177640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1b80d0730 0x7fc1b80cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.367+0000 7fc1c0177640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc1b80d0b90 0x7fc1b80cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.367+0000 7fc1c0177640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b80cc8c0 con 0x7fc1b80b0750 2024-04-25T10:02:02.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.367+0000 7fc1c0177640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b80ccb70 con 0x7fc1b80d0b90 2024-04-25T10:02:02.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.367+0000 7fc1c0177640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80cf970 con 0x7fc1b80d0730 2024-04-25T10:02:02.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.368+0000 7fc1bd712640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1b80d0730 0x7fc1b80cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.368+0000 7fc1be714640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc1b80d0b90 0x7fc1b80cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.368+0000 7fc1bd712640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1b80d0730 0x7fc1b80cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:33550/0 (socket says 172.21.15.139:33550) 2024-04-25T10:02:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.368+0000 7fc1bd712640 1 -- 172.21.15.139:0/2252585852 learned_addr learned my addr 172.21.15.139:0/2252585852 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.368+0000 7fc1bdf13640 1 --1- 172.21.15.139:0/2252585852 >> v1:172.21.15.150:6789/0 conn(0x7fc1b80b0750 0x7fc1b80b0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:60592/0 (socket says 172.21.15.139:60592) 2024-04-25T10:02:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.368+0000 7fc1bd712640 1 -- 172.21.15.139:0/2252585852 >> v2:172.21.15.150:3300/0 conn(0x7fc1b80d0b90 msgr2=0x7fc1b80cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.368+0000 7fc1bd712640 1 --2- 172.21.15.139:0/2252585852 >> v2:172.21.15.150:3300/0 conn(0x7fc1b80d0b90 0x7fc1b80cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.368+0000 7fc1bd712640 1 -- 172.21.15.139:0/2252585852 >> v1:172.21.15.150:6789/0 conn(0x7fc1b80b0750 legacy=0x7fc1b80b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.368+0000 7fc1bd712640 1 -- 172.21.15.139:0/2252585852 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b80cfc50 con 0x7fc1b80d0730 2024-04-25T10:02:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.368+0000 7fc1be714640 1 --2- 172.21.15.139:0/2252585852 >> v2:172.21.15.150:3300/0 conn(0x7fc1b80d0b90 0x7fc1b80cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.369+0000 7fc1bd712640 1 --2- 172.21.15.139:0/2252585852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1b80d0730 0x7fc1b80cbe40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009870 tx=0x7fc1a802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8cb201d5c2a325d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.369+0000 7fc1bcf11640 1 -- 172.21.15.139:0/2252585852 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8037070 con 0x7fc1b80d0730 2024-04-25T10:02:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.369+0000 7fc1bcf11640 1 -- 172.21.15.139:0/2252585852 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a8032440 con 0x7fc1b80d0730 2024-04-25T10:02:02.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.369+0000 7fc1bcf11640 1 -- 172.21.15.139:0/2252585852 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8032810 con 0x7fc1b80d0730 2024-04-25T10:02:02.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.370+0000 7fc1c0177640 1 -- 172.21.15.139:0/2252585852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1b80d0730 msgr2=0x7fc1b80cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.370+0000 7fc1c0177640 1 --2- 172.21.15.139:0/2252585852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1b80d0730 0x7fc1b80cbe40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009870 tx=0x7fc1a802a270 comp rx=0 tx=0).stop 2024-04-25T10:02:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.370+0000 7fc1c0177640 1 -- 172.21.15.139:0/2252585852 shutdown_connections 2024-04-25T10:02:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.370+0000 7fc1c0177640 1 --2- 172.21.15.139:0/2252585852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1b80d0730 0x7fc1b80cbe40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.370+0000 7fc1c0177640 1 --2- 172.21.15.139:0/2252585852 >> v2:172.21.15.150:3300/0 conn(0x7fc1b80d0b90 0x7fc1b80cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.370+0000 7fc1c0177640 1 -- 172.21.15.139:0/2252585852 >> 172.21.15.139:0/2252585852 conn(0x7fc1b80b3380 msgr2=0x7fc1b80b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.370+0000 7fc1c0177640 1 -- 172.21.15.139:0/2252585852 shutdown_connections 2024-04-25T10:02:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.370+0000 7fc1c0177640 1 -- 172.21.15.139:0/2252585852 wait complete. 2024-04-25T10:02:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.371+0000 7fc1c0177640 1 Processor -- start 2024-04-25T10:02:02.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.371+0000 7fc1c0177640 1 -- start start 2024-04-25T10:02:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1c0177640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1b80b0750 0x7fc1b8150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1c0177640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b80d0730 0x7fc1b80ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1c0177640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1b80afc30 0x7fc1b806d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1c0177640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b815bcc0 con 0x7fc1b80d0730 2024-04-25T10:02:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1c0177640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80d0210 con 0x7fc1b80b0750 2024-04-25T10:02:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1c0177640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b815be40 con 0x7fc1b80afc30 2024-04-25T10:02:02.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1bd712640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b80d0730 0x7fc1b80ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1bd712640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b80d0730 0x7fc1b80ca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46290/0 (socket says 172.21.15.139:46290) 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1bd712640 1 -- 172.21.15.139:0/2873854977 learned_addr learned my addr 172.21.15.139:0/2873854977 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1bdf13640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1b80b0750 0x7fc1b8150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1be714640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1b80afc30 0x7fc1b806d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1bd712640 1 -- 172.21.15.139:0/2873854977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1b80afc30 msgr2=0x7fc1b806d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1bd712640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1b80afc30 0x7fc1b806d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1bd712640 1 -- 172.21.15.139:0/2873854977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1b80b0750 msgr2=0x7fc1b8150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1bd712640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1b80b0750 0x7fc1b8150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1bd712640 1 -- 172.21.15.139:0/2873854977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b80caba0 con 0x7fc1b80d0730 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.372+0000 7fc1bd712640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b80d0730 0x7fc1b80ca2a0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc1a802a2b0 tx=0x7fc1a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.373+0000 7fc1ae7fc640 1 -- 172.21.15.139:0/2873854977 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8037040 con 0x7fc1b80d0730 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.373+0000 7fc1ae7fc640 1 -- 172.21.15.139:0/2873854977 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a8042070 con 0x7fc1b80d0730 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.373+0000 7fc1c0177640 1 -- 172.21.15.139:0/2873854977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b806bc70 con 0x7fc1b80d0730 2024-04-25T10:02:02.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.373+0000 7fc1ae7fc640 1 -- 172.21.15.139:0/2873854977 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a80308a0 con 0x7fc1b80d0730 2024-04-25T10:02:02.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.373+0000 7fc1c0177640 1 -- 172.21.15.139:0/2873854977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b806c230 con 0x7fc1b80d0730 2024-04-25T10:02:02.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.374+0000 7fc1c0177640 1 -- 172.21.15.139:0/2873854977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b80cc6b0 con 0x7fc1b80d0730 2024-04-25T10:02:02.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.374+0000 7fc1ae7fc640 1 -- 172.21.15.139:0/2873854977 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1a8030a40 con 0x7fc1b80d0730 2024-04-25T10:02:02.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.374+0000 7fc1ae7fc640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc19c06b500 0x7fc19c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.374+0000 7fc1ae7fc640 1 -- 172.21.15.139:0/2873854977 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fc1a80aaac0 con 0x7fc1b80d0730 2024-04-25T10:02:02.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.375+0000 7fc1bdf13640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc19c06b500 0x7fc19c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.375+0000 7fc1bdf13640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc19c06b500 0x7fc19c06d9c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4007fd0 tx=0x7fc1b4001630 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.379+0000 7fc1ae7fc640 1 -- 172.21.15.139:0/2873854977 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1b80cc6b0 con 0x7fc1b80d0730 2024-04-25T10:02:02.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.585+0000 7fc1c0177640 1 -- 172.21.15.139:0/2873854977 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1b8074520 con 0x7fc1b80d0730 2024-04-25T10:02:02.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.585+0000 7fc1ae7fc640 1 -- 172.21.15.139:0/2873854977 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4951 (secure 0 0 0) 0x7fc1a8079730 con 0x7fc1b80d0730 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:15.497940+0000","modified":"2024-04-25T10:02:01.518660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8808":{"gid":8808,"name":"c","rank":0,"incarnation":72,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.139:6837/359626718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":359626718},{"type":"v1","addr":"172.21.15.139:6837","nonce":359626718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 74 2024-04-25T10:02:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.587+0000 7fc1c0177640 1 -- 172.21.15.139:0/2873854977 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc19c06b500 msgr2=0x7fc19c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.587+0000 7fc1c0177640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc19c06b500 0x7fc19c06d9c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4007fd0 tx=0x7fc1b4001630 comp rx=0 tx=0).stop 2024-04-25T10:02:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.587+0000 7fc1c0177640 1 -- 172.21.15.139:0/2873854977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b80d0730 msgr2=0x7fc1b80ca2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.587+0000 7fc1c0177640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b80d0730 0x7fc1b80ca2a0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc1a802a2b0 tx=0x7fc1a80046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.587+0000 7fc1c0177640 1 -- 172.21.15.139:0/2873854977 shutdown_connections 2024-04-25T10:02:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.587+0000 7fc1c0177640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1b80d0730 0x7fc1b80ca2a0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.587+0000 7fc1c0177640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc19c06b500 0x7fc19c06d9c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.587+0000 7fc1c0177640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1b80afc30 0x7fc1b806d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.587+0000 7fc1c0177640 1 --2- 172.21.15.139:0/2873854977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1b80b0750 0x7fc1b8150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.587+0000 7fc1c0177640 1 -- 172.21.15.139:0/2873854977 >> 172.21.15.139:0/2873854977 conn(0x7fc1b80b3380 msgr2=0x7fc1b806f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.587+0000 7fc1c0177640 1 -- 172.21.15.139:0/2873854977 shutdown_connections 2024-04-25T10:02:02.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:02.587+0000 7fc1c0177640 1 -- 172.21.15.139:0/2873854977 wait complete. 2024-04-25T10:02:03.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:01:15.497940+0000', 'modified': '2024-04-25T10:02:01.518660+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8808}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8808': {'gid': 8808, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.139:6837/359626718', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6835', 'nonce': 359626718}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 359626718}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:02:03.006 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8808:up:rejoin 2024-04-25T10:02:04.007 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:04.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.112+0000 7f6078541640 1 Processor -- start 2024-04-25T10:02:04.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.112+0000 7f6078541640 1 -- start start 2024-04-25T10:02:04.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.113+0000 7f6078541640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f607006ab60 0x7f607006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.113+0000 7f6078541640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f60700cf960 0x7f60700cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.113+0000 7f6078541640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6070070a40 con 0x7f607015b950 2024-04-25T10:02:04.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.113+0000 7f6078541640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6070073170 con 0x7f60700cf960 2024-04-25T10:02:04.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.113+0000 7f6078541640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607006b500 con 0x7f607006ab60 2024-04-25T10:02:04.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.113+0000 7f6076ade640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f60700cf960 0x7f60700cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.113+0000 7f60762dd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f607015b950 0x7f6070072a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57758/0 (socket says 172.21.15.139:57758) 2024-04-25T10:02:04.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.113+0000 7f60762dd640 1 -- 172.21.15.139:0/1937864280 learned_addr learned my addr 172.21.15.139:0/1937864280 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:04.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.113+0000 7f6076ade640 1 -- 172.21.15.139:0/1937864280 >> v2:172.21.15.150:3300/0 conn(0x7f607006ab60 msgr2=0x7f607006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.113+0000 7f6076ade640 1 --2- 172.21.15.139:0/1937864280 >> v2:172.21.15.150:3300/0 conn(0x7f607006ab60 0x7f607006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.113+0000 7f6076ade640 1 -- 172.21.15.139:0/1937864280 >> v1:172.21.15.139:6789/0 conn(0x7f607015b950 legacy=0x7f6070072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.113+0000 7f6076ade640 1 -- 172.21.15.139:0/1937864280 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607006fc80 con 0x7f60700cf960 2024-04-25T10:02:04.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.114+0000 7f6076ade640 1 --2- 172.21.15.139:0/1937864280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60700cf960 0x7f60700cfdf0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f60700d1680 tx=0x7f606802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e0bbd811182c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.114+0000 7f60752db640 1 -- 172.21.15.139:0/1937864280 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068037070 con 0x7f60700cf960 2024-04-25T10:02:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.114+0000 7f60752db640 1 -- 172.21.15.139:0/1937864280 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6068032440 con 0x7f60700cf960 2024-04-25T10:02:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.114+0000 7f60752db640 1 -- 172.21.15.139:0/1937864280 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068032810 con 0x7f60700cf960 2024-04-25T10:02:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.115+0000 7f6078541640 1 -- 172.21.15.139:0/1937864280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60700cf960 msgr2=0x7f60700cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.115+0000 7f6078541640 1 --2- 172.21.15.139:0/1937864280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60700cf960 0x7f60700cfdf0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f60700d1680 tx=0x7f606802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.115+0000 7f6078541640 1 -- 172.21.15.139:0/1937864280 shutdown_connections 2024-04-25T10:02:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.115+0000 7f6078541640 1 --2- 172.21.15.139:0/1937864280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60700cf960 0x7f60700cfdf0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.115+0000 7f6078541640 1 --2- 172.21.15.139:0/1937864280 >> v2:172.21.15.150:3300/0 conn(0x7f607006ab60 0x7f607006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.115+0000 7f6078541640 1 -- 172.21.15.139:0/1937864280 >> 172.21.15.139:0/1937864280 conn(0x7f60700b23f0 msgr2=0x7f60700b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.115+0000 7f6078541640 1 -- 172.21.15.139:0/1937864280 shutdown_connections 2024-04-25T10:02:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.115+0000 7f6078541640 1 -- 172.21.15.139:0/1937864280 wait complete. 2024-04-25T10:02:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.116+0000 7f6078541640 1 Processor -- start 2024-04-25T10:02:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.116+0000 7f6078541640 1 -- start start 2024-04-25T10:02:04.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6078541640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f607006ab60 0x7f6070154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6078541640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60700cf960 0x7f6070154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6078541640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f607015b950 0x7f607015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6078541640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60700cda00 con 0x7f607015b950 2024-04-25T10:02:04.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6078541640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6070070240 con 0x7f607006ab60 2024-04-25T10:02:04.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6078541640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60700cdce0 con 0x7f60700cf960 2024-04-25T10:02:04.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6076ade640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f607015b950 0x7f607015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6076ade640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f607015b950 0x7f607015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46314/0 (socket says 172.21.15.139:46314) 2024-04-25T10:02:04.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6076ade640 1 -- 172.21.15.139:0/222463988 learned_addr learned my addr 172.21.15.139:0/222463988 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:04.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6076ade640 1 -- 172.21.15.139:0/222463988 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60700cf960 msgr2=0x7f6070154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6076ade640 1 --2- 172.21.15.139:0/222463988 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60700cf960 0x7f6070154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6076ade640 1 -- 172.21.15.139:0/222463988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f607006ab60 msgr2=0x7f6070154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6076ade640 1 --2- 172.21.15.139:0/222463988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f607006ab60 0x7f6070154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.117+0000 7f6076ade640 1 -- 172.21.15.139:0/222463988 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60701e72b0 con 0x7f607015b950 2024-04-25T10:02:04.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.118+0000 7f6076ade640 1 --2- 172.21.15.139:0/222463988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f607015b950 0x7f607015d120 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f606802a1e0 tx=0x7f60680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.118+0000 7f6066ffd640 1 -- 172.21.15.139:0/222463988 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068041070 con 0x7f607015b950 2024-04-25T10:02:04.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.118+0000 7f6066ffd640 1 -- 172.21.15.139:0/222463988 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606803c890 con 0x7f607015b950 2024-04-25T10:02:04.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.118+0000 7f6078541640 1 -- 172.21.15.139:0/222463988 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60701e7590 con 0x7f607015b950 2024-04-25T10:02:04.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.118+0000 7f6066ffd640 1 -- 172.21.15.139:0/222463988 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068037040 con 0x7f607015b950 2024-04-25T10:02:04.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.118+0000 7f6078541640 1 -- 172.21.15.139:0/222463988 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60701e7b50 con 0x7f607015b950 2024-04-25T10:02:04.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.119+0000 7f6078541640 1 -- 172.21.15.139:0/222463988 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6070074520 con 0x7f607015b950 2024-04-25T10:02:04.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.119+0000 7f6066ffd640 1 -- 172.21.15.139:0/222463988 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6068044070 con 0x7f607015b950 2024-04-25T10:02:04.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.119+0000 7f6066ffd640 1 --2- 172.21.15.139:0/222463988 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f605406b500 0x7f605406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.120+0000 7f6066ffd640 1 -- 172.21.15.139:0/222463988 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f60680ab080 con 0x7f607015b950 2024-04-25T10:02:04.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.120+0000 7f60762dd640 1 --2- 172.21.15.139:0/222463988 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f605406b500 0x7f605406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.120+0000 7f60762dd640 1 --2- 172.21.15.139:0/222463988 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f605406b500 0x7f605406d9c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f606c0012e0 tx=0x7f606c001590 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.124+0000 7f6066ffd640 1 -- 172.21.15.139:0/222463988 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6068079e00 con 0x7f607015b950 2024-04-25T10:02:04.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.334+0000 7f6078541640 1 -- 172.21.15.139:0/222463988 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6070072460 con 0x7f607015b950 2024-04-25T10:02:04.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.334+0000 7f6066ffd640 1 -- 172.21.15.139:0/222463988 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f6068079700 con 0x7f607015b950 2024-04-25T10:02:04.339 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:04.339 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":75,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:15.497940+0000","modified":"2024-04-25T10:02:02.519505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8808":{"gid":8808,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6837/359626718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":359626718},{"type":"v1","addr":"172.21.15.139:6837","nonce":359626718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:02:04.340 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 75 2024-04-25T10:02:04.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.336+0000 7f6078541640 1 -- 172.21.15.139:0/222463988 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f605406b500 msgr2=0x7f605406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.336+0000 7f6078541640 1 --2- 172.21.15.139:0/222463988 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f605406b500 0x7f605406d9c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f606c0012e0 tx=0x7f606c001590 comp rx=0 tx=0).stop 2024-04-25T10:02:04.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.336+0000 7f6078541640 1 -- 172.21.15.139:0/222463988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f607015b950 msgr2=0x7f607015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.336+0000 7f6078541640 1 --2- 172.21.15.139:0/222463988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f607015b950 0x7f607015d120 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f606802a1e0 tx=0x7f60680046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.336+0000 7f6078541640 1 -- 172.21.15.139:0/222463988 shutdown_connections 2024-04-25T10:02:04.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.336+0000 7f6078541640 1 --2- 172.21.15.139:0/222463988 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f607015b950 0x7f607015d120 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.336+0000 7f6078541640 1 --2- 172.21.15.139:0/222463988 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f605406b500 0x7f605406d9c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.336+0000 7f6078541640 1 --2- 172.21.15.139:0/222463988 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60700cf960 0x7f6070154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.336+0000 7f6078541640 1 --2- 172.21.15.139:0/222463988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f607006ab60 0x7f6070154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.336+0000 7f6078541640 1 -- 172.21.15.139:0/222463988 >> 172.21.15.139:0/222463988 conn(0x7f60700b23f0 msgr2=0x7f607006e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.336+0000 7f6078541640 1 -- 172.21.15.139:0/222463988 shutdown_connections 2024-04-25T10:02:04.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.336+0000 7f6078541640 1 -- 172.21.15.139:0/222463988 wait complete. 2024-04-25T10:02:04.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:01:15.497940+0000', 'modified': '2024-04-25T10:02:02.519505+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8808}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8808': {'gid': 8808, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.139:6837/359626718', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6835', 'nonce': 359626718}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 359626718}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:02:04.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:04.755 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.855+0000 7f19ad2d2640 1 Processor -- start 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.855+0000 7f19ad2d2640 1 -- start start 2024-04-25T10:02:04.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.855+0000 7f19ad2d2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f19a80cee50 0x7f19a8168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.855+0000 7f19ad2d2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a8161dc0 con 0x7f19a81617e0 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.855+0000 7f19ad2d2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a81620a0 con 0x7f19a815dbf0 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.855+0000 7f19ad2d2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a80d1ee0 con 0x7f19a80cee50 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.856+0000 7f19a77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f19a80cee50 0x7f19a8168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.856+0000 7f19a67fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f19a81617e0 0x7f19a80ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57764/0 (socket says 172.21.15.139:57764) 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.856+0000 7f19a67fc640 1 -- 172.21.15.139:0/892455618 learned_addr learned my addr 172.21.15.139:0/892455618 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.856+0000 7f19a77fe640 1 -- 172.21.15.139:0/892455618 >> v1:172.21.15.150:6789/0 conn(0x7f19a815dbf0 legacy=0x7f19a80b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.856+0000 7f19a77fe640 1 -- 172.21.15.139:0/892455618 >> v1:172.21.15.139:6789/0 conn(0x7f19a81617e0 legacy=0x7f19a80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.856+0000 7f19a77fe640 1 -- 172.21.15.139:0/892455618 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a8162380 con 0x7f19a80cee50 2024-04-25T10:02:04.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.856+0000 7f19a77fe640 1 --2- 172.21.15.139:0/892455618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a80cee50 0x7f19a8168b10 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009d90 tx=0x7f19a002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13a79812716e545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.857+0000 7f19a5ffb640 1 -- 172.21.15.139:0/892455618 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0031600 con 0x7f19a80cee50 2024-04-25T10:02:04.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.857+0000 7f19a5ffb640 1 -- 172.21.15.139:0/892455618 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a00317a0 con 0x7f19a80cee50 2024-04-25T10:02:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.857+0000 7f19a5ffb640 1 -- 172.21.15.139:0/892455618 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0031b70 con 0x7f19a80cee50 2024-04-25T10:02:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.857+0000 7f19ad2d2640 1 -- 172.21.15.139:0/892455618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a80cee50 msgr2=0x7f19a8168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.857+0000 7f19ad2d2640 1 --2- 172.21.15.139:0/892455618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a80cee50 0x7f19a8168b10 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009d90 tx=0x7f19a002a610 comp rx=0 tx=0).stop 2024-04-25T10:02:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.857+0000 7f19ad2d2640 1 -- 172.21.15.139:0/892455618 shutdown_connections 2024-04-25T10:02:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.857+0000 7f19ad2d2640 1 --2- 172.21.15.139:0/892455618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a80cee50 0x7f19a8168b10 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.857+0000 7f19ad2d2640 1 -- 172.21.15.139:0/892455618 >> 172.21.15.139:0/892455618 conn(0x7f19a80b5480 msgr2=0x7f19a80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.857+0000 7f19ad2d2640 1 -- 172.21.15.139:0/892455618 shutdown_connections 2024-04-25T10:02:04.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.857+0000 7f19ad2d2640 1 -- 172.21.15.139:0/892455618 wait complete. 2024-04-25T10:02:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.858+0000 7f19ad2d2640 1 Processor -- start 2024-04-25T10:02:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.859+0000 7f19ad2d2640 1 -- start start 2024-04-25T10:02:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.859+0000 7f19ad2d2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19a80cee50 0x7f19a815ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.859+0000 7f19ad2d2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19a815dbf0 0x7f19a815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.859+0000 7f19ad2d2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a81617e0 0x7f19a81f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.859+0000 7f19ad2d2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a815dfd0 con 0x7f19a81617e0 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.859+0000 7f19ad2d2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a80d3380 con 0x7f19a815dbf0 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.859+0000 7f19ad2d2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a815f4e0 con 0x7f19a80cee50 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.859+0000 7f19a77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a81617e0 0x7f19a81f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.859+0000 7f19a77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a81617e0 0x7f19a81f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46318/0 (socket says 172.21.15.139:46318) 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.859+0000 7f19a77fe640 1 -- 172.21.15.139:0/2111426534 learned_addr learned my addr 172.21.15.139:0/2111426534 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f19a77fe640 1 -- 172.21.15.139:0/2111426534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19a80cee50 msgr2=0x7f19a815ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f19a67fc640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19a815dbf0 0x7f19a815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f19a6ffd640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19a80cee50 0x7f19a815ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f19a77fe640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19a80cee50 0x7f19a815ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f19a77fe640 1 -- 172.21.15.139:0/2111426534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19a815dbf0 msgr2=0x7f19a815a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f19a77fe640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19a815dbf0 0x7f19a815a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f19a77fe640 1 -- 172.21.15.139:0/2111426534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a8152050 con 0x7f19a81617e0 2024-04-25T10:02:04.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f19a77fe640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a81617e0 0x7f19a81f48c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009d90 tx=0x7f19a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f198f7fe640 1 -- 172.21.15.139:0/2111426534 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0040070 con 0x7f19a81617e0 2024-04-25T10:02:04.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f198f7fe640 1 -- 172.21.15.139:0/2111426534 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a0030ac0 con 0x7f19a81617e0 2024-04-25T10:02:04.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f19ad2d2640 1 -- 172.21.15.139:0/2111426534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a8152330 con 0x7f19a81617e0 2024-04-25T10:02:04.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f198f7fe640 1 -- 172.21.15.139:0/2111426534 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a003c420 con 0x7f19a81617e0 2024-04-25T10:02:04.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.860+0000 7f19ad2d2640 1 -- 172.21.15.139:0/2111426534 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a815c930 con 0x7f19a81617e0 2024-04-25T10:02:04.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.862+0000 7f198f7fe640 1 -- 172.21.15.139:0/2111426534 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19a0039040 con 0x7f19a81617e0 2024-04-25T10:02:04.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.862+0000 7f198f7fe640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f199406b4d0 0x7f199406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.862+0000 7f198f7fe640 1 -- 172.21.15.139:0/2111426534 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f19a00ab8d0 con 0x7f19a81617e0 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.862+0000 7f19a6ffd640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f199406b4d0 0x7f199406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.863+0000 7f19a6ffd640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f199406b4d0 0x7f199406d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f19980015d0 tx=0x7f1998009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.863+0000 7f19affd8740 1 Processor -- start 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.863+0000 7f19affd8740 1 -- start start 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19affd8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c04230 0x55e829c04610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19affd8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c21ad0 0x55e829c22020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19affd8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829bd9f10 0x55e829c20190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19affd8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e829b3ca10 con 0x55e829c21ad0 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19affd8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e829ae5b30 con 0x55e829bd9f10 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19affd8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e829b38c50 con 0x55e829c04230 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c21ad0 0x55e829c22020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c21ad0 0x55e829c22020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46328/0 (socket says 172.21.15.139:46328) 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a67fc640 1 -- 172.21.15.139:0/1939621883 learned_addr learned my addr 172.21.15.139:0/1939621883 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a6ffd640 1 --2- 172.21.15.139:0/1939621883 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c04230 0x55e829c04610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a77fe640 1 --2- 172.21.15.139:0/1939621883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829bd9f10 0x55e829c20190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a67fc640 1 -- 172.21.15.139:0/1939621883 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c04230 msgr2=0x55e829c04610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a67fc640 1 --2- 172.21.15.139:0/1939621883 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c04230 0x55e829c04610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a67fc640 1 -- 172.21.15.139:0/1939621883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829bd9f10 msgr2=0x55e829c20190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a67fc640 1 --2- 172.21.15.139:0/1939621883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829bd9f10 0x55e829c20190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a67fc640 1 -- 172.21.15.139:0/1939621883 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e829afae10 con 0x55e829c21ad0 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a6ffd640 1 --2- 172.21.15.139:0/1939621883 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c04230 0x55e829c04610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a77fe640 1 --2- 172.21.15.139:0/1939621883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829bd9f10 0x55e829c20190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f19a67fc640 1 --2- 172.21.15.139:0/1939621883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c21ad0 0x55e829c22020 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f199c001590 tx=0x7f199c001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a6499cafbe0c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f198cff9640 1 -- 172.21.15.139:0/1939621883 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c018070 con 0x55e829c21ad0 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f198cff9640 1 -- 172.21.15.139:0/1939621883 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c005ad0 con 0x55e829c21ad0 2024-04-25T10:02:04.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.864+0000 7f198cff9640 1 -- 172.21.15.139:0/1939621883 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c0070d0 con 0x55e829c21ad0 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.865+0000 7f19affd8740 1 -- 172.21.15.139:0/1939621883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c21ad0 msgr2=0x55e829c22020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.865+0000 7f19affd8740 1 --2- 172.21.15.139:0/1939621883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c21ad0 0x55e829c22020 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f199c001590 tx=0x7f199c001a50 comp rx=0 tx=0).stop 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.865+0000 7f19affd8740 1 -- 172.21.15.139:0/1939621883 shutdown_connections 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.865+0000 7f19affd8740 1 --2- 172.21.15.139:0/1939621883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c21ad0 0x55e829c22020 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.865+0000 7f19affd8740 1 --2- 172.21.15.139:0/1939621883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829bd9f10 0x55e829c20190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.865+0000 7f19affd8740 1 --2- 172.21.15.139:0/1939621883 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c04230 0x55e829c04610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.865+0000 7f19affd8740 1 -- 172.21.15.139:0/1939621883 >> 172.21.15.139:0/1939621883 conn(0x55e829bd6840 msgr2=0x55e829c1f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.865+0000 7f19affd8740 1 -- 172.21.15.139:0/1939621883 shutdown_connections 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.865+0000 7f19affd8740 1 -- 172.21.15.139:0/1939621883 wait complete. 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.866+0000 7f19affd8740 1 -- start start 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.866+0000 7f19affd8740 1 Processor -- start 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19affd8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c04230 0x55e829c1e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19affd8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c21ad0 0x55e829c06860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19affd8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c171d0 0x55e829c17630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19affd8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e829bdacb0 con 0x55e829c04230 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19affd8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e829af90c0 con 0x55e829c171d0 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19affd8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e829bd4770 con 0x55e829c21ad0 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19affd8740 10 client.0 _ll_register_callbacks cb 0x55e829aecec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c04230 0x55e829c1e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c04230 0x55e829c1e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46330/0 (socket says 172.21.15.139:46330) 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a6ffd640 1 -- 172.21.15.139:0/2807019065 learned_addr learned my addr 172.21.15.139:0/2807019065 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a67fc640 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c21ad0 0x55e829c06860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a77fe640 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c171d0 0x55e829c17630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a6ffd640 1 -- 172.21.15.139:0/2807019065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c21ad0 msgr2=0x55e829c06860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a6ffd640 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c21ad0 0x55e829c06860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a6ffd640 1 -- 172.21.15.139:0/2807019065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c171d0 msgr2=0x55e829c17630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a6ffd640 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c171d0 0x55e829c17630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a6ffd640 1 -- 172.21.15.139:0/2807019065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8298512d0 con 0x55e829c04230 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a67fc640 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c21ad0 0x55e829c06860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a77fe640 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c171d0 0x55e829c17630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19a6ffd640 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c04230 0x55e829c1e430 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1998003ce0 tx=0x7f1998000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9930926d22fac4d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f1985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f1985ffb640 1 -- 172.21.15.139:0/2807019065 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998021630 con 0x55e829c04230 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f1985ffb640 1 -- 172.21.15.139:0/2807019065 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f1998020ca0 con 0x55e829c04230 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19affd8740 1 -- 172.21.15.139:0/2807019065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f199c005ad0 con 0x55e829c04230 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f1985ffb640 1 -- 172.21.15.139:0/2807019065 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19980217d0 con 0x55e829c04230 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19affd8740 10 client.8886 fetch_fsmap learned FSMap version 75 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.867+0000 7f19affd8740 1 -- 172.21.15.139:0/2807019065 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55e82991b6b0 con 0x55e829c04230 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.868+0000 7f1985ffb640 1 -- 172.21.15.139:0/2807019065 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c005ad0 con 0x55e829c04230 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.868+0000 7f1985ffb640 1 -- 172.21.15.139:0/2807019065 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1998021970 con 0x55e829c04230 2024-04-25T10:02:04.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.868+0000 7f19affd8740 10 client.8886 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.868+0000 7f19affd8740 10 client.8886 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.868+0000 7f19affd8740 20 client.8886 populate_metadata read hostname 'smithi139' 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.868+0000 7f19affd8740 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55e829c3b3c0 0x55e829c3d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.868+0000 7f19affd8740 4 client.8886 mds_command: new command op to 8808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.868+0000 7f19affd8740 1 -- 172.21.15.139:0/2807019065 --> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e829b9b920 con 0x55e829c3b3c0 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.868+0000 7f19a67fc640 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55e829c3b3c0 0x55e829c3d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.868+0000 7f1985ffb640 10 client.8886 ms_handle_connect on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.868+0000 7f19a67fc640 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55e829c3b3c0 0x55e829c3d7b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44207accd03b7117 server_cookie=c28477d703a1012a in_seq=0 out_seq=0 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f1985ffb640 1 -- 172.21.15.139:0/2807019065 <== mds.0 v2:172.21.15.139:6835/359626718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e829b9b920 con 0x55e829c3b3c0 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f1985ffb640 10 client.8886 handle_command_reply: tid=0 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f19affd8740 1 client.8886 shutdown 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f19affd8740 1 -- 172.21.15.139:0/2807019065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c04230 msgr2=0x55e829c1e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f19affd8740 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c04230 0x55e829c1e430 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1998003ce0 tx=0x7f1998000fb0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f19affd8740 1 -- 172.21.15.139:0/2807019065 shutdown_connections 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f19affd8740 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55e829c3b3c0 0x55e829c3d7b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f19affd8740 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c171d0 0x55e829c17630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f19affd8740 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c21ad0 0x55e829c06860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f19affd8740 1 --2- 172.21.15.139:0/2807019065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c04230 0x55e829c1e430 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f19affd8740 1 -- 172.21.15.139:0/2807019065 >> 172.21.15.139:0/2807019065 conn(0x55e829bd9f10 msgr2=0x55e829c03710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f1985ffb640 0 client.8886 ms_handle_reset on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:04.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f19affd8740 1 -- 172.21.15.139:0/2807019065 shutdown_connections 2024-04-25T10:02:04.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f19affd8740 1 -- 172.21.15.139:0/2807019065 wait complete. 2024-04-25T10:02:04.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.869+0000 7f19affd8740 20 client.8886 trim_cache size 0 max 16384 2024-04-25T10:02:04.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.884+0000 7f19affd8740 1 Processor -- start 2024-04-25T10:02:04.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.885+0000 7f19affd8740 1 -- start start 2024-04-25T10:02:04.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.885+0000 7f19affd8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c04230 0x55e829c849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.885+0000 7f19affd8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 0x55e829c16690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.885+0000 7f19affd8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c21ad0 0x55e829c16bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.885+0000 7f19affd8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a80b6d70 con 0x55e829c171d0 2024-04-25T10:02:04.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.885+0000 7f19affd8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a814d590 con 0x55e829c21ad0 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.885+0000 7f19affd8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a814db90 con 0x55e829c04230 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.885+0000 7f19a67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 0x55e829c16690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.885+0000 7f19a67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 0x55e829c16690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46346/0 (socket says 172.21.15.139:46346) 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.885+0000 7f19a67fc640 1 -- 172.21.15.139:0/4235016725 learned_addr learned my addr 172.21.15.139:0/4235016725 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19a67fc640 1 -- 172.21.15.139:0/4235016725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c04230 msgr2=0x55e829c849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19a67fc640 1 --2- 172.21.15.139:0/4235016725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c04230 0x55e829c849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19a67fc640 1 -- 172.21.15.139:0/4235016725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c21ad0 msgr2=0x55e829c16bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19a67fc640 1 --2- 172.21.15.139:0/4235016725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c21ad0 0x55e829c16bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19a67fc640 1 -- 172.21.15.139:0/4235016725 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e829bd9740 con 0x55e829c171d0 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19a67fc640 1 --2- 172.21.15.139:0/4235016725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 0x55e829c16690 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f199c001a90 tx=0x7f199c00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b58b25e0b01bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f1987fff640 1 -- 172.21.15.139:0/4235016725 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c018070 con 0x55e829c171d0 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f1987fff640 1 -- 172.21.15.139:0/4235016725 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c02b070 con 0x55e829c171d0 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f1987fff640 1 -- 172.21.15.139:0/4235016725 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c004e50 con 0x55e829c171d0 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19affd8740 1 -- 172.21.15.139:0/4235016725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 msgr2=0x55e829c16690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19affd8740 1 --2- 172.21.15.139:0/4235016725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 0x55e829c16690 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f199c001a90 tx=0x7f199c00b000 comp rx=0 tx=0).stop 2024-04-25T10:02:04.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19affd8740 1 -- 172.21.15.139:0/4235016725 shutdown_connections 2024-04-25T10:02:04.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19affd8740 1 --2- 172.21.15.139:0/4235016725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 0x55e829c16690 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19affd8740 1 --2- 172.21.15.139:0/4235016725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c21ad0 0x55e829c16bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19affd8740 1 --2- 172.21.15.139:0/4235016725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c04230 0x55e829c849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19affd8740 1 -- 172.21.15.139:0/4235016725 >> 172.21.15.139:0/4235016725 conn(0x55e829bd9f10 msgr2=0x55e829bcfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19affd8740 1 -- 172.21.15.139:0/4235016725 shutdown_connections 2024-04-25T10:02:04.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.886+0000 7f19affd8740 1 -- 172.21.15.139:0/4235016725 wait complete. 2024-04-25T10:02:04.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.887+0000 7f19affd8740 1 -- start start 2024-04-25T10:02:04.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.887+0000 7f19affd8740 1 Processor -- start 2024-04-25T10:02:04.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19affd8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c04230 0x55e829c834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19affd8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 0x55e829c1cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19affd8740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c21ad0 0x55e829c1e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19affd8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e829c054a0 con 0x55e829c171d0 2024-04-25T10:02:04.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19affd8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e829c06340 con 0x55e829c04230 2024-04-25T10:02:04.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19affd8740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e829c18c80 con 0x55e829c21ad0 2024-04-25T10:02:04.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19affd8740 10 client.0 _ll_register_callbacks cb 0x55e829bd4c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:04.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19a67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 0x55e829c1cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19a67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 0x55e829c1cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46350/0 (socket says 172.21.15.139:46350) 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19a67fc640 1 -- 172.21.15.139:0/1563719133 learned_addr learned my addr 172.21.15.139:0/1563719133 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19a6ffd640 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c04230 0x55e829c834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19a77fe640 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c21ad0 0x55e829c1e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19a67fc640 1 -- 172.21.15.139:0/1563719133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c21ad0 msgr2=0x55e829c1e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19a67fc640 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c21ad0 0x55e829c1e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19a67fc640 1 -- 172.21.15.139:0/1563719133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c04230 msgr2=0x55e829c834e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19a67fc640 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c04230 0x55e829c834e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19a67fc640 1 -- 172.21.15.139:0/1563719133 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e829c42540 con 0x55e829c171d0 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19a77fe640 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c21ad0 0x55e829c1e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19a67fc640 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 0x55e829c1cee0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f199c001560 tx=0x7f199c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fa0b6fe2250c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f1985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f1985ffb640 1 -- 172.21.15.139:0/1563719133 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c018040 con 0x55e829c171d0 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f1985ffb640 1 -- 172.21.15.139:0/1563719133 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f199c004cc0 con 0x55e829c171d0 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19affd8740 1 -- 172.21.15.139:0/1563719133 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f199c02b070 con 0x55e829c171d0 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f1985ffb640 1 -- 172.21.15.139:0/1563719133 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f199c03c050 con 0x55e829c171d0 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19affd8740 10 client.8892 fetch_fsmap learned FSMap version 75 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.888+0000 7f19affd8740 1 -- 172.21.15.139:0/1563719133 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55e829c41810 con 0x55e829c171d0 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f1985ffb640 1 -- 172.21.15.139:0/1563719133 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c02b070 con 0x55e829c171d0 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f1985ffb640 1 -- 172.21.15.139:0/1563719133 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f199c03f020 con 0x55e829c171d0 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 10 client.8892 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 10 client.8892 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 20 client.8892 populate_metadata read hostname 'smithi139' 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55e829bda2f0 0x55e829bda6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 4 client.8892 mds_command: new command op to 8808 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 -- 172.21.15.139:0/1563719133 --> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e829b49f50 con 0x55e829bda2f0 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19a6ffd640 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55e829bda2f0 0x55e829bda6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f1985ffb640 10 client.8892 ms_handle_connect on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19a6ffd640 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55e829bda2f0 0x55e829bda6d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d776689be27c72 server_cookie=e40fe36a0dfab6b6 in_seq=0 out_seq=0 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f1985ffb640 1 -- 172.21.15.139:0/1563719133 <== mds.0 v2:172.21.15.139:6835/359626718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55e829b49f50 con 0x55e829bda2f0 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f1985ffb640 10 client.8892 handle_command_reply: tid=0 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 client.8892 shutdown 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 -- 172.21.15.139:0/1563719133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 msgr2=0x55e829c1cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 0x55e829c1cee0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f199c001560 tx=0x7f199c000bc0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 -- 172.21.15.139:0/1563719133 shutdown_connections 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55e829bda2f0 0x55e829bda6d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e829c171d0 0x55e829c1cee0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e829c21ad0 0x55e829c1e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 --2- 172.21.15.139:0/1563719133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e829c04230 0x55e829c834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f1985ffb640 0 client.8892 ms_handle_reset on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 -- 172.21.15.139:0/1563719133 >> 172.21.15.139:0/1563719133 conn(0x55e829bd9f10 msgr2=0x55e829c38540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 -- 172.21.15.139:0/1563719133 shutdown_connections 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.889+0000 7f19affd8740 1 -- 172.21.15.139:0/1563719133 wait complete. 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19affd8740 20 client.8892 trim_cache size 0 max 16384 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 8808, 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 4.3734144190000004, 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 75, 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 108, 2024-04-25T10:02:04.902 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 108, 2024-04-25T10:02:04.902 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 5.3444028350000004 2024-04-25T10:02:04.902 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:02:04.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19ad2d2640 1 -- 172.21.15.139:0/2111426534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f199406b4d0 msgr2=0x7f199406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19ad2d2640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f199406b4d0 0x7f199406d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f19980015d0 tx=0x7f1998009040 comp rx=0 tx=0).stop 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19ad2d2640 1 -- 172.21.15.139:0/2111426534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a81617e0 msgr2=0x7f19a81f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19ad2d2640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a81617e0 0x7f19a81f48c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009d90 tx=0x7f19a0004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19ad2d2640 1 -- 172.21.15.139:0/2111426534 shutdown_connections 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19ad2d2640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a81617e0 0x7f19a81f48c0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19ad2d2640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f199406b4d0 0x7f199406d990 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19ad2d2640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19a815dbf0 0x7f19a815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19ad2d2640 1 --2- 172.21.15.139:0/2111426534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19a80cee50 0x7f19a815ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19ad2d2640 1 -- 172.21.15.139:0/2111426534 >> 172.21.15.139:0/2111426534 conn(0x7f19a80b5480 msgr2=0x7f19a80b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19ad2d2640 1 -- 172.21.15.139:0/2111426534 shutdown_connections 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:04.890+0000 7f19ad2d2640 1 -- 172.21.15.139:0/2111426534 wait complete. 2024-04-25T10:02:04.905 DEBUG:teuthology.orchestra.run.smithi139:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:02:04.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:04.908 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T10:02:05.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.005+0000 7efe77c5d640 1 Processor -- start 2024-04-25T10:02:05.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.005+0000 7efe77c5d640 1 -- start start 2024-04-25T10:02:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.006+0000 7efe77c5d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7015dba0 0x7efe70161800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.006+0000 7efe77c5d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efe700b93b0 0x7efe700ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.006+0000 7efe77c5d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe700d1e60 con 0x7efe700ced30 2024-04-25T10:02:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.006+0000 7efe77c5d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe700cf180 con 0x7efe700b93b0 2024-04-25T10:02:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.006+0000 7efe77c5d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe700d21b0 con 0x7efe7015dba0 2024-04-25T10:02:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.006+0000 7efe759f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7015dba0 0x7efe70161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.006+0000 7efe751f8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efe700b93b0 0x7efe700ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.006+0000 7efe759f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7015dba0 0x7efe70161800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:33612/0 (socket says 172.21.15.139:33612) 2024-04-25T10:02:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.006+0000 7efe751f8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efe700b93b0 0x7efe700ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:34106/0 (socket says 172.21.15.139:34106) 2024-04-25T10:02:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.006+0000 7efe759f9640 1 -- 172.21.15.139:0/1446542782 learned_addr learned my addr 172.21.15.139:0/1446542782 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.006+0000 7efe761fa640 1 --1- 172.21.15.139:0/1446542782 >> v1:172.21.15.150:6789/0 conn(0x7efe700ced30 0x7efe7006c030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:60614/0 (socket says 172.21.15.139:60614) 2024-04-25T10:02:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.007+0000 7efe751f8640 1 -- 172.21.15.139:0/1446542782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7015dba0 msgr2=0x7efe70161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.007+0000 7efe751f8640 1 --2- 172.21.15.139:0/1446542782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7015dba0 0x7efe70161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.007+0000 7efe751f8640 1 -- 172.21.15.139:0/1446542782 >> v1:172.21.15.150:6789/0 conn(0x7efe700ced30 legacy=0x7efe7006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.007+0000 7efe749f7640 1 -- 172.21.15.139:0/1446542782 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782266567 0 0) 0x7efe700d1e60 con 0x7efe700ced30 2024-04-25T10:02:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.007+0000 7efe751f8640 1 -- 172.21.15.139:0/1446542782 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe70162050 con 0x7efe700b93b0 2024-04-25T10:02:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.007+0000 7efe751f8640 1 --2- 172.21.15.139:0/1446542782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe700b93b0 0x7efe700ce7f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efe600097a0 tx=0x7efe6002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec59ce47c687fbbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.008+0000 7efe749f7640 1 -- 172.21.15.139:0/1446542782 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60037070 con 0x7efe700b93b0 2024-04-25T10:02:05.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.008+0000 7efe749f7640 1 -- 172.21.15.139:0/1446542782 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe60032440 con 0x7efe700b93b0 2024-04-25T10:02:05.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.008+0000 7efe749f7640 1 -- 172.21.15.139:0/1446542782 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60032810 con 0x7efe700b93b0 2024-04-25T10:02:05.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.008+0000 7efe77c5d640 1 -- 172.21.15.139:0/1446542782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe700b93b0 msgr2=0x7efe700ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.008+0000 7efe77c5d640 1 --2- 172.21.15.139:0/1446542782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe700b93b0 0x7efe700ce7f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efe600097a0 tx=0x7efe6002a180 comp rx=0 tx=0).stop 2024-04-25T10:02:05.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.008+0000 7efe77c5d640 1 -- 172.21.15.139:0/1446542782 shutdown_connections 2024-04-25T10:02:05.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.008+0000 7efe77c5d640 1 --2- 172.21.15.139:0/1446542782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe700b93b0 0x7efe700ce7f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.008+0000 7efe77c5d640 1 --2- 172.21.15.139:0/1446542782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7015dba0 0x7efe70161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.008+0000 7efe77c5d640 1 -- 172.21.15.139:0/1446542782 >> 172.21.15.139:0/1446542782 conn(0x7efe700d04a0 msgr2=0x7efe700d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.008+0000 7efe77c5d640 1 -- 172.21.15.139:0/1446542782 shutdown_connections 2024-04-25T10:02:05.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.008+0000 7efe77c5d640 1 -- 172.21.15.139:0/1446542782 wait complete. 2024-04-25T10:02:05.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.009+0000 7efe77c5d640 1 Processor -- start 2024-04-25T10:02:05.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.010+0000 7efe77c5d640 1 -- start start 2024-04-25T10:02:05.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.010+0000 7efe77c5d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe700b93b0 0x7efe700c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.010+0000 7efe77c5d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe700ced30 0x7efe70160780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.010+0000 7efe77c5d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7015dba0 0x7efe70148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.010+0000 7efe77c5d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe70161d40 con 0x7efe700b93b0 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.010+0000 7efe77c5d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe70162610 con 0x7efe700ced30 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.010+0000 7efe77c5d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe700b6bc0 con 0x7efe7015dba0 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.010+0000 7efe759f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe700b93b0 0x7efe700c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.010+0000 7efe759f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe700b93b0 0x7efe700c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46354/0 (socket says 172.21.15.139:46354) 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.010+0000 7efe759f9640 1 -- 172.21.15.139:0/2900818921 learned_addr learned my addr 172.21.15.139:0/2900818921 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe761fa640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7015dba0 0x7efe70148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe751f8640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe700ced30 0x7efe70160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe759f9640 1 -- 172.21.15.139:0/2900818921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7015dba0 msgr2=0x7efe70148f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe759f9640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7015dba0 0x7efe70148f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe759f9640 1 -- 172.21.15.139:0/2900818921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe700ced30 msgr2=0x7efe70160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe759f9640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe700ced30 0x7efe70160780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe759f9640 1 -- 172.21.15.139:0/2900818921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe70149470 con 0x7efe700b93b0 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe759f9640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe700b93b0 0x7efe700c7530 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009cf0 tx=0x7efe6c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe667fc640 1 -- 172.21.15.139:0/2900818921 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c00cda0 con 0x7efe700b93b0 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe667fc640 1 -- 172.21.15.139:0/2900818921 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c019070 con 0x7efe700b93b0 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe667fc640 1 -- 172.21.15.139:0/2900818921 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0146a0 con 0x7efe700b93b0 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe77c5d640 1 -- 172.21.15.139:0/2900818921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe700c7c10 con 0x7efe700b93b0 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.011+0000 7efe77c5d640 1 -- 172.21.15.139:0/2900818921 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe700c81d0 con 0x7efe700b93b0 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.013+0000 7efe667fc640 1 -- 172.21.15.139:0/2900818921 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe6c005d10 con 0x7efe700b93b0 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.013+0000 7efe667fc640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe5406b4d0 0x7efe5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.013+0000 7efe667fc640 1 -- 172.21.15.139:0/2900818921 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7efe6c08a710 con 0x7efe700b93b0 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.013+0000 7efe751f8640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe5406b4d0 0x7efe5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.014+0000 7efe751f8640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe5406b4d0 0x7efe5406d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7efe60034000 tx=0x7efe6002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.014+0000 7efe7a5fd740 1 Processor -- start 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.014+0000 7efe7a5fd740 1 -- start start 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe7a5fd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e680cbb0 0x55f0e680cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe7a5fd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e6807830 0x55f0e680b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe7a5fd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e680c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe7a5fd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e6770ac0 con 0x55f0e680bcf0 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe7a5fd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e6719d80 con 0x55f0e680cbb0 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe7a5fd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0e676cb60 con 0x55f0e6807830 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe761fa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e680c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe761fa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e680c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46370/0 (socket says 172.21.15.139:46370) 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe761fa640 1 -- 172.21.15.139:0/6431066 learned_addr learned my addr 172.21.15.139:0/6431066 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe751f8640 1 --2- 172.21.15.139:0/6431066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e6807830 0x55f0e680b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe759f9640 1 --2- 172.21.15.139:0/6431066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e680cbb0 0x55f0e680cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe761fa640 1 -- 172.21.15.139:0/6431066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e6807830 msgr2=0x55f0e680b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe761fa640 1 --2- 172.21.15.139:0/6431066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e6807830 0x55f0e680b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe761fa640 1 -- 172.21.15.139:0/6431066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e680cbb0 msgr2=0x55f0e680cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe761fa640 1 --2- 172.21.15.139:0/6431066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e680cbb0 0x55f0e680cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe761fa640 1 -- 172.21.15.139:0/6431066 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0e654d7e0 con 0x55f0e680bcf0 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe759f9640 1 --2- 172.21.15.139:0/6431066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e680cbb0 0x55f0e680cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.015+0000 7efe761fa640 1 --2- 172.21.15.139:0/6431066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e680c130 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7efe68007610 tx=0x7efe68004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e6cd76cfdc560d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.016+0000 7efe537fe640 1 -- 172.21.15.139:0/6431066 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68001890 con 0x55f0e680bcf0 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.016+0000 7efe537fe640 1 -- 172.21.15.139:0/6431066 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe68018070 con 0x55f0e680bcf0 2024-04-25T10:02:05.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.016+0000 7efe537fe640 1 -- 172.21.15.139:0/6431066 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68001c70 con 0x55f0e680bcf0 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.016+0000 7efe7a5fd740 1 -- 172.21.15.139:0/6431066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 msgr2=0x55f0e680c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.016+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/6431066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e680c130 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7efe68007610 tx=0x7efe68004bb0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.016+0000 7efe7a5fd740 1 -- 172.21.15.139:0/6431066 shutdown_connections 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.016+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/6431066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e680c130 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.016+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/6431066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e6807830 0x55f0e680b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.016+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/6431066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e680cbb0 0x55f0e680cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.016+0000 7efe7a5fd740 1 -- 172.21.15.139:0/6431066 >> 172.21.15.139:0/6431066 conn(0x55f0e6809cb0 msgr2=0x55f0e6852770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.016+0000 7efe7a5fd740 1 -- 172.21.15.139:0/6431066 shutdown_connections 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.016+0000 7efe7a5fd740 1 -- 172.21.15.139:0/6431066 wait complete. 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.017+0000 7efe7a5fd740 1 -- start start 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.017+0000 7efe7a5fd740 1 Processor -- start 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe7a5fd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e6857d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe7a5fd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e680cbb0 0x55f0e6859b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe7a5fd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e681fea0 0x55f0e685a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe7a5fd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e6838740 con 0x55f0e680bcf0 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe7a5fd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e670f530 con 0x55f0e681fea0 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe7a5fd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0e6837bc0 con 0x55f0e680cbb0 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe7a5fd740 10 client.0 _ll_register_callbacks cb 0x55f0e6767130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe759f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e6857d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe759f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e6857d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46374/0 (socket says 172.21.15.139:46374) 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe759f9640 1 -- 172.21.15.139:0/1925304183 learned_addr learned my addr 172.21.15.139:0/1925304183 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe759f9640 1 -- 172.21.15.139:0/1925304183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e680cbb0 msgr2=0x55f0e6859b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe759f9640 1 --2- 172.21.15.139:0/1925304183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e680cbb0 0x55f0e6859b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe759f9640 1 -- 172.21.15.139:0/1925304183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e681fea0 msgr2=0x55f0e685a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe759f9640 1 --2- 172.21.15.139:0/1925304183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e681fea0 0x55f0e685a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe759f9640 1 -- 172.21.15.139:0/1925304183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0e65adf70 con 0x55f0e680bcf0 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe759f9640 1 --2- 172.21.15.139:0/1925304183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e6857d60 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7efe6c058ab0 tx=0x7efe6c058ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd88802a4344ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.018+0000 7efe50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:02:05.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe50ff9640 1 -- 172.21.15.139:0/1925304183 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c058200 con 0x55f0e680bcf0 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe50ff9640 1 -- 172.21.15.139:0/1925304183 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7efe6c0583a0 con 0x55f0e680bcf0 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe7a5fd740 1 -- 172.21.15.139:0/1925304183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe68018070 con 0x55f0e680bcf0 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe50ff9640 1 -- 172.21.15.139:0/1925304183 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe6c054b00 con 0x55f0e680bcf0 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe7a5fd740 10 client.8901 fetch_fsmap learned FSMap version 75 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe7a5fd740 1 -- 172.21.15.139:0/1925304183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55f0e6445170 con 0x55f0e680bcf0 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe50ff9640 1 -- 172.21.15.139:0/1925304183 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe68018070 con 0x55f0e680bcf0 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe50ff9640 1 -- 172.21.15.139:0/1925304183 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe6c057c70 con 0x55f0e680bcf0 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe7a5fd740 10 client.8901 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe7a5fd740 10 client.8901 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe7a5fd740 20 client.8901 populate_metadata read hostname 'smithi139' 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/1925304183 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55f0e6834400 0x55f0e6834890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe7a5fd740 4 client.8901 mds_command: new command op to 8808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe7a5fd740 1 -- 172.21.15.139:0/1925304183 --> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0e677e000 con 0x55f0e6834400 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.019+0000 7efe751f8640 1 --2- 172.21.15.139:0/1925304183 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55f0e6834400 0x55f0e6834890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.020+0000 7efe50ff9640 10 client.8901 ms_handle_connect on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.020+0000 7efe751f8640 1 --2- 172.21.15.139:0/1925304183 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55f0e6834400 0x55f0e6834890 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7a99f1363a37f9 server_cookie=f4e60c7082e1fe0 in_seq=0 out_seq=0 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.020+0000 7efe50ff9640 1 -- 172.21.15.139:0/1925304183 <== mds.0 v2:172.21.15.139:6835/359626718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f0e677e000 con 0x55f0e6834400 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.020+0000 7efe50ff9640 10 client.8901 handle_command_reply: tid=0 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe7a5fd740 1 client.8901 shutdown 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe7a5fd740 1 -- 172.21.15.139:0/1925304183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 msgr2=0x55f0e6857d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/1925304183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e6857d60 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7efe6c058ab0 tx=0x7efe6c058ef0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe7a5fd740 1 -- 172.21.15.139:0/1925304183 shutdown_connections 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/1925304183 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55f0e6834400 0x55f0e6834890 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/1925304183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e681fea0 0x55f0e685a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe50ff9640 0 client.8901 ms_handle_reset on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/1925304183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e680cbb0 0x55f0e6859b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/1925304183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e6857d60 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe7a5fd740 1 -- 172.21.15.139:0/1925304183 >> 172.21.15.139:0/1925304183 conn(0x55f0e6807830 msgr2=0x55f0e680d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe7a5fd740 1 -- 172.21.15.139:0/1925304183 shutdown_connections 2024-04-25T10:02:05.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe7a5fd740 1 -- 172.21.15.139:0/1925304183 wait complete. 2024-04-25T10:02:05.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.021+0000 7efe7a5fd740 20 client.8901 trim_cache size 0 max 16384 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.027+0000 7f263abe7640 1 Processor -- start 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.027+0000 7f263abe7640 1 -- start start 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.028+0000 7f263abe7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f263415b860 0x7f263415e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.028+0000 7f263abe7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263415e7f0 con 0x7f26340b9430 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.028+0000 7f263abe7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263415ead0 con 0x7f26340b5b50 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.028+0000 7f263abe7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26340b6980 con 0x7f263415b860 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.028+0000 7f2633fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f26340b9430 0x7f26340b5440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57768/0 (socket says 172.21.15.139:57768) 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.028+0000 7f2633fff640 1 -- 172.21.15.139:0/3675137039 learned_addr learned my addr 172.21.15.139:0/3675137039 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.028+0000 7f2638983640 1 --2- 172.21.15.139:0/3675137039 >> v2:172.21.15.150:3300/0 conn(0x7f263415b860 0x7f263415e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.028+0000 7f26337fe640 1 -- 172.21.15.139:0/3675137039 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313258848 0 0) 0x7f263415e7f0 con 0x7f26340b9430 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.028+0000 7f26337fe640 1 -- 172.21.15.139:0/3675137039 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2624003520 con 0x7f26340b9430 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.028+0000 7f2638983640 1 -- 172.21.15.139:0/3675137039 >> v1:172.21.15.150:6789/0 conn(0x7f26340b5b50 legacy=0x7f26340d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.028+0000 7f2638983640 1 -- 172.21.15.139:0/3675137039 >> v1:172.21.15.139:6789/0 conn(0x7f26340b9430 legacy=0x7f26340b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.028+0000 7f2638983640 1 -- 172.21.15.139:0/3675137039 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263415edb0 con 0x7f263415b860 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.029+0000 7f2638983640 1 --2- 172.21.15.139:0/3675137039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f263415b860 0x7f263415e0f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f26280097a0 tx=0x7f262802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4872d08bb22df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.029+0000 7f26337fe640 1 -- 172.21.15.139:0/3675137039 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628037070 con 0x7f263415b860 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.029+0000 7f26337fe640 1 -- 172.21.15.139:0/3675137039 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2628032440 con 0x7f263415b860 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.029+0000 7f26337fe640 1 -- 172.21.15.139:0/3675137039 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628032810 con 0x7f263415b860 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.030+0000 7f263abe7640 1 -- 172.21.15.139:0/3675137039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f263415b860 msgr2=0x7f263415e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.030+0000 7f263abe7640 1 --2- 172.21.15.139:0/3675137039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f263415b860 0x7f263415e0f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f26280097a0 tx=0x7f262802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.030+0000 7f263abe7640 1 -- 172.21.15.139:0/3675137039 shutdown_connections 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.030+0000 7f263abe7640 1 --2- 172.21.15.139:0/3675137039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f263415b860 0x7f263415e0f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.030+0000 7f263abe7640 1 -- 172.21.15.139:0/3675137039 >> 172.21.15.139:0/3675137039 conn(0x7f26340d0620 msgr2=0x7f26340d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.030+0000 7f263abe7640 1 -- 172.21.15.139:0/3675137039 shutdown_connections 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.030+0000 7f263abe7640 1 -- 172.21.15.139:0/3675137039 wait complete. 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.031+0000 7f263abe7640 1 Processor -- start 2024-04-25T10:02:05.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.031+0000 7f263abe7640 1 -- start start 2024-04-25T10:02:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f263abe7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26340b5b50 0x7f26340b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f263abe7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26340b9430 0x7f263414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f263abe7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f263415b860 0x7f26340b8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f263abe7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26340d1df0 con 0x7f263415b860 2024-04-25T10:02:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f263abe7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263415f090 con 0x7f26340b5b50 2024-04-25T10:02:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f263abe7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26340b98e0 con 0x7f26340b9430 2024-04-25T10:02:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f2639184640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f263415b860 0x7f26340b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f2639184640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f263415b860 0x7f26340b8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46378/0 (socket says 172.21.15.139:46378) 2024-04-25T10:02:05.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f2639184640 1 -- 172.21.15.139:0/3000531873 learned_addr learned my addr 172.21.15.139:0/3000531873 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f2638983640 1 --2- 172.21.15.139:0/3000531873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26340b5b50 0x7f26340b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f2633fff640 1 --2- 172.21.15.139:0/3000531873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26340b9430 0x7f263414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f2639184640 1 -- 172.21.15.139:0/3000531873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26340b9430 msgr2=0x7f263414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f2639184640 1 --2- 172.21.15.139:0/3000531873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26340b9430 0x7f263414fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f2639184640 1 -- 172.21.15.139:0/3000531873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26340b5b50 msgr2=0x7f26340b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f2639184640 1 --2- 172.21.15.139:0/3000531873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26340b5b50 0x7f26340b74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f2639184640 1 -- 172.21.15.139:0/3000531873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26340b7ae0 con 0x7f263415b860 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.032+0000 7f2639184640 1 --2- 172.21.15.139:0/3000531873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f263415b860 0x7f26340b8940 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c6b0 tx=0x7f262c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.033+0000 7f26317fa640 1 -- 172.21.15.139:0/3000531873 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c003b70 con 0x7f263415b860 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.033+0000 7f26317fa640 1 -- 172.21.15.139:0/3000531873 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c003d30 con 0x7f263415b860 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.033+0000 7f263abe7640 1 -- 172.21.15.139:0/3000531873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26340b7dc0 con 0x7f263415b860 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.033+0000 7f26317fa640 1 -- 172.21.15.139:0/3000531873 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c005310 con 0x7f263415b860 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.033+0000 7f263abe7640 1 -- 172.21.15.139:0/3000531873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26340b8380 con 0x7f263415b860 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.033+0000 7f263abe7640 1 -- 172.21.15.139:0/3000531873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f2634074520 con 0x7f263415b860 2024-04-25T10:02:05.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.034+0000 7f26317fa640 1 -- 172.21.15.139:0/3000531873 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f262c0054b0 con 0x7f263415b860 2024-04-25T10:02:05.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.034+0000 7f26317fa640 1 --2- 172.21.15.139:0/3000531873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f261806b500 0x7f261806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.035+0000 7f26317fa640 1 -- 172.21.15.139:0/3000531873 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f262c08ab20 con 0x7f263415b860 2024-04-25T10:02:05.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.035+0000 7f2638983640 1 --2- 172.21.15.139:0/3000531873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f261806b500 0x7f261806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.035+0000 7f26317fa640 1 -- 172.21.15.139:0/3000531873 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+549 (secure 0 0 0) 0x7f262c08b300 con 0x7f263415b860 2024-04-25T10:02:05.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.035+0000 7f263abe7640 1 -- 172.21.15.139:0/3000531873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f263406ce60 con 0x7f263415b860 2024-04-25T10:02:05.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.035+0000 7f2638983640 1 --2- 172.21.15.139:0/3000531873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f261806b500 0x7f261806d9c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2628034000 tx=0x7f262802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.037+0000 7efe7a5fd740 1 Processor -- start 2024-04-25T10:02:05.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.037+0000 7efe7a5fd740 1 -- start start 2024-04-25T10:02:05.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe7a5fd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e680d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe7a5fd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e680cbb0 0x55f0e68066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe7a5fd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e681fea0 0x55f0e6806c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe7a5fd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe702003b0 con 0x55f0e680bcf0 2024-04-25T10:02:05.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe7a5fd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe701ff5a0 con 0x55f0e680cbb0 2024-04-25T10:02:05.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe7a5fd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe700cd870 con 0x55f0e681fea0 2024-04-25T10:02:05.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe759f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e680d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe759f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e680d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46388/0 (socket says 172.21.15.139:46388) 2024-04-25T10:02:05.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe759f9640 1 -- 172.21.15.139:0/17071573 learned_addr learned my addr 172.21.15.139:0/17071573 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:05.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe759f9640 1 -- 172.21.15.139:0/17071573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e681fea0 msgr2=0x55f0e6806c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe759f9640 1 --2- 172.21.15.139:0/17071573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e681fea0 0x55f0e6806c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe759f9640 1 -- 172.21.15.139:0/17071573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e680cbb0 msgr2=0x55f0e68066c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe759f9640 1 --2- 172.21.15.139:0/17071573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e680cbb0 0x55f0e68066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe759f9640 1 -- 172.21.15.139:0/17071573 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0e6848dd0 con 0x55f0e680bcf0 2024-04-25T10:02:05.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.038+0000 7efe759f9640 1 --2- 172.21.15.139:0/17071573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e680d3d0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00c990 tx=0x7efe6c05f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8c479ef334c993 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.039+0000 7efe52ffd640 1 -- 172.21.15.139:0/17071573 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c057c70 con 0x55f0e680bcf0 2024-04-25T10:02:05.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.039+0000 7efe52ffd640 1 -- 172.21.15.139:0/17071573 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c077bd0 con 0x55f0e680bcf0 2024-04-25T10:02:05.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.039+0000 7efe52ffd640 1 -- 172.21.15.139:0/17071573 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c058200 con 0x55f0e680bcf0 2024-04-25T10:02:05.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.039+0000 7efe7a5fd740 1 -- 172.21.15.139:0/17071573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 msgr2=0x55f0e680d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.039+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/17071573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e680d3d0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00c990 tx=0x7efe6c05f050 comp rx=0 tx=0).stop 2024-04-25T10:02:05.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.039+0000 7efe7a5fd740 1 -- 172.21.15.139:0/17071573 shutdown_connections 2024-04-25T10:02:05.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.039+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/17071573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e681fea0 0x55f0e6806c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.039+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/17071573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e680cbb0 0x55f0e68066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.039+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/17071573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680bcf0 0x55f0e680d3d0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.039+0000 7efe7a5fd740 1 -- 172.21.15.139:0/17071573 >> 172.21.15.139:0/17071573 conn(0x55f0e6807830 msgr2=0x55f0e6836e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.039+0000 7efe7a5fd740 1 -- 172.21.15.139:0/17071573 shutdown_connections 2024-04-25T10:02:05.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.039+0000 7efe7a5fd740 1 -- 172.21.15.139:0/17071573 wait complete. 2024-04-25T10:02:05.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.040+0000 7efe7a5fd740 1 -- start start 2024-04-25T10:02:05.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.040+0000 7efe7a5fd740 1 Processor -- start 2024-04-25T10:02:05.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe7a5fd740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e680bcf0 0x55f0e68b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe7a5fd740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680cbb0 0x55f0e6831e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe7a5fd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e681fea0 0x55f0e68323a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe7a5fd740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7014a360 con 0x55f0e680cbb0 2024-04-25T10:02:05.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe7a5fd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e680d9e0 con 0x55f0e681fea0 2024-04-25T10:02:05.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe7a5fd740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0e68c1220 con 0x55f0e680bcf0 2024-04-25T10:02:05.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe7a5fd740 10 client.0 _ll_register_callbacks cb 0x7efe68007060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:05.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe751f8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680cbb0 0x55f0e6831e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe751f8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680cbb0 0x55f0e6831e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46392/0 (socket says 172.21.15.139:46392) 2024-04-25T10:02:05.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe751f8640 1 -- 172.21.15.139:0/4092739599 learned_addr learned my addr 172.21.15.139:0/4092739599 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:05.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe759f9640 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e680bcf0 0x55f0e68b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe761fa640 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e681fea0 0x55f0e68323a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe751f8640 1 -- 172.21.15.139:0/4092739599 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e680bcf0 msgr2=0x55f0e68b74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe751f8640 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e680bcf0 0x55f0e68b74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe751f8640 1 -- 172.21.15.139:0/4092739599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e681fea0 msgr2=0x55f0e68323a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe751f8640 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e681fea0 0x55f0e68323a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe751f8640 1 -- 172.21.15.139:0/4092739599 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0e6857570 con 0x55f0e680cbb0 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe751f8640 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680cbb0 0x55f0e6831e60 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7efe6002a770 tx=0x7efe600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f6090b75c1e42a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.041+0000 7efe50ff9640 1 -- 172.21.15.139:0/4092739599 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60053070 con 0x55f0e680cbb0 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe50ff9640 1 -- 172.21.15.139:0/4092739599 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7efe6004fd00 con 0x55f0e680cbb0 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe7a5fd740 1 -- 172.21.15.139:0/4092739599 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe6c077bd0 con 0x55f0e680cbb0 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe50ff9640 1 -- 172.21.15.139:0/4092739599 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe60060050 con 0x55f0e680cbb0 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe7a5fd740 10 client.8913 fetch_fsmap learned FSMap version 75 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe7a5fd740 1 -- 172.21.15.139:0/4092739599 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55f0e68c0bc0 con 0x55f0e680cbb0 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe50ff9640 1 -- 172.21.15.139:0/4092739599 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c077bd0 con 0x55f0e680cbb0 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe50ff9640 1 -- 172.21.15.139:0/4092739599 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe60047040 con 0x55f0e680cbb0 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe7a5fd740 10 client.8913 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:02:05.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe7a5fd740 10 client.8913 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:02:05.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe7a5fd740 20 client.8913 populate_metadata read hostname 'smithi139' 2024-04-25T10:02:05.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55f0e6859360 0x55f0e68bd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe7a5fd740 4 client.8913 mds_command: new command op to 8808 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:02:05.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe7a5fd740 1 -- 172.21.15.139:0/4092739599 --> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55f0e6778350 con 0x55f0e6859360 2024-04-25T10:02:05.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe759f9640 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55f0e6859360 0x55f0e68bd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.042+0000 7efe50ff9640 10 client.8913 ms_handle_connect on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:05.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe759f9640 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55f0e6859360 0x55f0e68bd9b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe89855bf30277d2 server_cookie=2ca352697a628015 in_seq=0 out_seq=0 2024-04-25T10:02:05.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe50ff9640 1 -- 172.21.15.139:0/4092739599 <== mds.0 v2:172.21.15.139:6835/359626718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f0e6778350 con 0x55f0e6859360 2024-04-25T10:02:05.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe50ff9640 10 client.8913 handle_command_reply: tid=0 2024-04-25T10:02:05.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe7a5fd740 1 client.8913 shutdown 2024-04-25T10:02:05.056 INFO:tasks.ceph.mds.c.smithi139.stderr:2024-04-25T10:02:05.043+0000 7f23fb033640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:02:05.056 INFO:tasks.ceph.mds.c.smithi139.stderr:2024-04-25T10:02:05.043+0000 7f23fb033640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T10:02:05.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe7a5fd740 1 -- 172.21.15.139:0/4092739599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680cbb0 msgr2=0x55f0e6831e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680cbb0 0x55f0e6831e60 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7efe6002a770 tx=0x7efe600046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe7a5fd740 1 -- 172.21.15.139:0/4092739599 shutdown_connections 2024-04-25T10:02:05.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55f0e6859360 0x55f0e68bd9b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f0e680cbb0 0x55f0e6831e60 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe50ff9640 0 client.8913 ms_handle_reset on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:05.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0e681fea0 0x55f0e68323a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe7a5fd740 1 --2- 172.21.15.139:0/4092739599 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f0e680bcf0 0x55f0e68b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.043+0000 7efe7a5fd740 1 -- 172.21.15.139:0/4092739599 >> 172.21.15.139:0/4092739599 conn(0x55f0e6807830 msgr2=0x55f0e68379a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.044+0000 7efe7a5fd740 1 -- 172.21.15.139:0/4092739599 shutdown_connections 2024-04-25T10:02:05.058 INFO:tasks.ceph.mds.c.smithi139.stderr:2024-04-25T10:02:05.044+0000 7f23fb033640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:02:05.058 INFO:tasks.ceph.mds.c.smithi139.stderr:2024-04-25T10:02:05.044+0000 7f23fb033640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T10:02:05.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.044+0000 7efe7a5fd740 1 -- 172.21.15.139:0/4092739599 wait complete. 2024-04-25T10:02:05.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.044+0000 7efe7a5fd740 20 client.8913 trim_cache size 0 max 16384 2024-04-25T10:02:05.058 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:02:05.059 INFO:teuthology.orchestra.run.smithi139.stdout: "return_code": 0, 2024-04-25T10:02:05.059 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_tag": "c3ea6cfd-328b-4eda-9ed9-87e67f893a67", 2024-04-25T10:02:05.059 INFO:teuthology.orchestra.run.smithi139.stdout: "mode": "asynchronous" 2024-04-25T10:02:05.059 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.045+0000 7efe77c5d640 1 -- 172.21.15.139:0/2900818921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe5406b4d0 msgr2=0x7efe5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.045+0000 7efe77c5d640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe5406b4d0 0x7efe5406d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7efe60034000 tx=0x7efe6002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.045+0000 7efe77c5d640 1 -- 172.21.15.139:0/2900818921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe700b93b0 msgr2=0x7efe700c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.045+0000 7efe77c5d640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe700b93b0 0x7efe700c7530 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009cf0 tx=0x7efe6c00c490 comp rx=0 tx=0).stop 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.045+0000 7efe77c5d640 1 -- 172.21.15.139:0/2900818921 shutdown_connections 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.045+0000 7efe77c5d640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7efe5406b4d0 0x7efe5406d990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.045+0000 7efe77c5d640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe7015dba0 0x7efe70148f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.045+0000 7efe77c5d640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe700ced30 0x7efe70160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.045+0000 7efe77c5d640 1 --2- 172.21.15.139:0/2900818921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe700b93b0 0x7efe700c7530 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.045+0000 7efe77c5d640 1 -- 172.21.15.139:0/2900818921 >> 172.21.15.139:0/2900818921 conn(0x7efe700d04a0 msgr2=0x7efe700b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.045+0000 7efe77c5d640 1 -- 172.21.15.139:0/2900818921 shutdown_connections 2024-04-25T10:02:05.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.045+0000 7efe77c5d640 1 -- 172.21.15.139:0/2900818921 wait complete. 2024-04-25T10:02:05.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:05.061 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:02:05.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.155+0000 7f6f7a7a5640 1 Processor -- start 2024-04-25T10:02:05.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.155+0000 7f6f7a7a5640 1 -- start start 2024-04-25T10:02:05.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.155+0000 7f6f7a7a5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6f740d0a80 0x7f6f741618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f7a7a5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f74099ca0 con 0x7f6f7409ad40 2024-04-25T10:02:05.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f7a7a5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f74099f80 con 0x7f6f740cd610 2024-04-25T10:02:05.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f7a7a5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f74099a50 con 0x7f6f740d0a80 2024-04-25T10:02:05.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f78d42640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f6f7409ad40 0x7f6f740cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57770/0 (socket says 172.21.15.139:57770) 2024-04-25T10:02:05.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f78d42640 1 -- 172.21.15.139:0/2548619884 learned_addr learned my addr 172.21.15.139:0/2548619884 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:05.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f737fe640 1 --2- 172.21.15.139:0/2548619884 >> v2:172.21.15.150:3300/0 conn(0x7f6f740d0a80 0x7f6f741618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331621320 0 0) 0x7f6f74099ca0 con 0x7f6f7409ad40 2024-04-25T10:02:05.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f5c003520 con 0x7f6f7409ad40 2024-04-25T10:02:05.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1121097389 0 0) 0x7f6f5c003520 con 0x7f6f7409ad40 2024-04-25T10:02:05.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 >> v2:172.21.15.150:3300/0 conn(0x7f6f740d0a80 msgr2=0x7f6f741618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 --2- 172.21.15.139:0/2548619884 >> v2:172.21.15.150:3300/0 conn(0x7f6f740d0a80 0x7f6f741618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 >> v1:172.21.15.150:6789/0 conn(0x7f6f740cd610 legacy=0x7f6f740d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7409a260 con 0x7f6f7409ad40 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f6f6c0019f0 con 0x7f6f7409ad40 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 >> v1:172.21.15.139:6789/0 conn(0x7f6f7409ad40 legacy=0x7f6f740cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f5c0039b0 0x7f6f5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f5c004800 0x7f6f5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f5c0051c0 0x7f6f5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f74099a50 con 0x7f6f5c0051c0 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c00c070 con 0x7f6f5c004800 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c00c270 con 0x7f6f5c0039b0 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f737fe640 1 --2- 172.21.15.139:0/2548619884 >> v2:172.21.15.150:3300/0 conn(0x7f6f740d0a80 0x7f6f741618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f78d42640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f5c0051c0 0x7f6f5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f73fff640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f5c0039b0 0x7f6f5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.156+0000 7f6f737fe640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f5c004800 0x7f6f5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.157+0000 7f6f78d42640 1 -- 172.21.15.139:0/2548619884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f5c0039b0 msgr2=0x7f6f5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.157+0000 7f6f78d42640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f5c0039b0 0x7f6f5c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.157+0000 7f6f78d42640 1 -- 172.21.15.139:0/2548619884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f5c004800 msgr2=0x7f6f5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.157+0000 7f6f78d42640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f5c004800 0x7f6f5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.157+0000 7f6f78d42640 1 -- 172.21.15.139:0/2548619884 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c003520 con 0x7f6f5c0051c0 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.157+0000 7f6f73fff640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f5c0039b0 0x7f6f5c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.157+0000 7f6f737fe640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f5c004800 0x7f6f5c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.157+0000 7f6f78d42640 1 -- 172.21.15.139:0/2548619884 reap_dead start 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.157+0000 7f6f78d42640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f5c0051c0 0x7f6f5c00b9c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00a850 tx=0x7f6f5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd4a72191023a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.157+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00a4a0 con 0x7f6f5c0051c0 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.157+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c02f070 con 0x7f6f5c0051c0 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.157+0000 7f6f72ffd640 1 -- 172.21.15.139:0/2548619884 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c02bce0 con 0x7f6f5c0051c0 2024-04-25T10:02:05.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.158+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2548619884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f5c0051c0 msgr2=0x7f6f5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.158+0000 7f6f7a7a5640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f5c0051c0 0x7f6f5c00b9c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00a850 tx=0x7f6f5c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:05.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.158+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2548619884 shutdown_connections 2024-04-25T10:02:05.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.158+0000 7f6f7a7a5640 1 --2- 172.21.15.139:0/2548619884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f5c0051c0 0x7f6f5c00b9c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.158+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2548619884 >> 172.21.15.139:0/2548619884 conn(0x7f6f740b31f0 msgr2=0x7f6f740b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.158+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2548619884 shutdown_connections 2024-04-25T10:02:05.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.158+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2548619884 wait complete. 2024-04-25T10:02:05.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.159+0000 7f6f7a7a5640 1 Processor -- start 2024-04-25T10:02:05.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.159+0000 7f6f7a7a5640 1 -- start start 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f7a7a5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f5c0051c0 0x7f6f74156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f7a7a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f7415a150 0x7f6f74155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f7a7a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f74158420 0x7f6f740b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f7a7a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740cd9f0 con 0x7f6f74158420 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f7a7a5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7409a820 con 0x7f6f7415a150 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f7a7a5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740cdb70 con 0x7f6f5c0051c0 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f78d42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f74158420 0x7f6f740b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f78d42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f74158420 0x7f6f740b6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46412/0 (socket says 172.21.15.139:46412) 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f78d42640 1 -- 172.21.15.139:0/2868334305 learned_addr learned my addr 172.21.15.139:0/2868334305 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f73fff640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f5c0051c0 0x7f6f74156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f737fe640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f7415a150 0x7f6f74155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f78d42640 1 -- 172.21.15.139:0/2868334305 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f5c0051c0 msgr2=0x7f6f74156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f78d42640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f5c0051c0 0x7f6f74156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f78d42640 1 -- 172.21.15.139:0/2868334305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f7415a150 msgr2=0x7f6f74155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f78d42640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f7415a150 0x7f6f74155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f78d42640 1 -- 172.21.15.139:0/2868334305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f74158d50 con 0x7f6f74158420 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f737fe640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f7415a150 0x7f6f74155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.160+0000 7f6f73fff640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f5c0051c0 0x7f6f74156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.161+0000 7f6f78d42640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f74158420 0x7f6f740b6bf0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c000f80 tx=0x7f6f6c02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.161+0000 7f6f70ff9640 1 -- 172.21.15.139:0/2868334305 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c02b880 con 0x7f6f74158420 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.161+0000 7f6f70ff9640 1 -- 172.21.15.139:0/2868334305 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c02ba20 con 0x7f6f74158420 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.161+0000 7f6f70ff9640 1 -- 172.21.15.139:0/2868334305 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c02be10 con 0x7f6f74158420 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.161+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2868334305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f740b43c0 con 0x7f6f74158420 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.161+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2868334305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f740b4980 con 0x7f6f74158420 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.162+0000 7f6f70ff9640 1 -- 172.21.15.139:0/2868334305 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f6c034470 con 0x7f6f74158420 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.163+0000 7f6f70ff9640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6f6006b500 0x7f6f6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.163+0000 7f6f70ff9640 1 -- 172.21.15.139:0/2868334305 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f6f6c0aa0b0 con 0x7f6f74158420 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.163+0000 7f6f73fff640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6f6006b500 0x7f6f6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.163+0000 7f6f73fff640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6f6006b500 0x7f6f6006d9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6f68007fd0 tx=0x7f6f68009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.164+0000 7f6f7d586740 1 Processor -- start 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.164+0000 7f6f7d586740 1 -- start start 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.164+0000 7f6f7d586740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecb10 0x55b94dbef320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.164+0000 7f6f7d586740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbecef0 0x55b94dbef860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.164+0000 7f6f7d586740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbbf770 0x55b94dbc2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.164+0000 7f6f7d586740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94db28da0 con 0x55b94dbbf770 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.164+0000 7f6f7d586740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94dad1b80 con 0x55b94dbecb10 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.164+0000 7f6f7d586740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b94db24e40 con 0x55b94dbecef0 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.164+0000 7f6f78d42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbbf770 0x55b94dbc2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.164+0000 7f6f78d42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbbf770 0x55b94dbc2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46428/0 (socket says 172.21.15.139:46428) 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.164+0000 7f6f78d42640 1 -- 172.21.15.139:0/3659990663 learned_addr learned my addr 172.21.15.139:0/3659990663 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:05.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f737fe640 1 --2- 172.21.15.139:0/3659990663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbecef0 0x55b94dbef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f73fff640 1 --2- 172.21.15.139:0/3659990663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecb10 0x55b94dbef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f78d42640 1 -- 172.21.15.139:0/3659990663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbecef0 msgr2=0x55b94dbef860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f78d42640 1 --2- 172.21.15.139:0/3659990663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbecef0 0x55b94dbef860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f78d42640 1 -- 172.21.15.139:0/3659990663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecb10 msgr2=0x55b94dbef320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f78d42640 1 --2- 172.21.15.139:0/3659990663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecb10 0x55b94dbef320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f78d42640 1 -- 172.21.15.139:0/3659990663 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b94dadbf10 con 0x55b94dbbf770 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f737fe640 1 --2- 172.21.15.139:0/3659990663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbecef0 0x55b94dbef860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f73fff640 1 --2- 172.21.15.139:0/3659990663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecb10 0x55b94dbef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f78d42640 1 --2- 172.21.15.139:0/3659990663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbbf770 0x55b94dbc2f80 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c004fd0 tx=0x7f6f6c00a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e911cb31fa2313a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f59ffb640 1 -- 172.21.15.139:0/3659990663 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c078610 con 0x55b94dbbf770 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f59ffb640 1 -- 172.21.15.139:0/3659990663 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c02f040 con 0x55b94dbbf770 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.165+0000 7f6f59ffb640 1 -- 172.21.15.139:0/3659990663 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c07c760 con 0x55b94dbbf770 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.166+0000 7f6f7d586740 1 -- 172.21.15.139:0/3659990663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbbf770 msgr2=0x55b94dbc2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.166+0000 7f6f7d586740 1 --2- 172.21.15.139:0/3659990663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbbf770 0x55b94dbc2f80 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c004fd0 tx=0x7f6f6c00a540 comp rx=0 tx=0).stop 2024-04-25T10:02:05.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.166+0000 7f6f7d586740 1 -- 172.21.15.139:0/3659990663 shutdown_connections 2024-04-25T10:02:05.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.166+0000 7f6f7d586740 1 --2- 172.21.15.139:0/3659990663 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbbf770 0x55b94dbc2f80 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.166+0000 7f6f7d586740 1 --2- 172.21.15.139:0/3659990663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbecef0 0x55b94dbef860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.166+0000 7f6f7d586740 1 --2- 172.21.15.139:0/3659990663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecb10 0x55b94dbef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.166+0000 7f6f7d586740 1 -- 172.21.15.139:0/3659990663 >> 172.21.15.139:0/3659990663 conn(0x55b94dbc19e0 msgr2=0x55b94dc0a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.166+0000 7f6f7d586740 1 -- 172.21.15.139:0/3659990663 shutdown_connections 2024-04-25T10:02:05.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.166+0000 7f6f7d586740 1 -- 172.21.15.139:0/3659990663 wait complete. 2024-04-25T10:02:05.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.166+0000 7f6f7d586740 1 -- start start 2024-04-25T10:02:05.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.166+0000 7f6f7d586740 1 Processor -- start 2024-04-25T10:02:05.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.167+0000 7f6f7d586740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 0x55b94dbe6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.167+0000 7f6f7d586740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecef0 0x55b94dbe4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.167+0000 7f6f7d586740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 0x55b94dbe8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.167+0000 7f6f7d586740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94dad1b80 con 0x55b94dbecb10 2024-04-25T10:02:05.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.167+0000 7f6f7d586740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94dac74b0 con 0x55b94dbecef0 2024-04-25T10:02:05.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.167+0000 7f6f7d586740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b94db28da0 con 0x55b94dbe3ea0 2024-04-25T10:02:05.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.167+0000 7f6f7d586740 10 client.0 _ll_register_callbacks cb 0x55b94db07180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:05.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.167+0000 7f6f73fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 0x55b94dbe6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.167+0000 7f6f73fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 0x55b94dbe6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46434/0 (socket says 172.21.15.139:46434) 2024-04-25T10:02:05.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.167+0000 7f6f73fff640 1 -- 172.21.15.139:0/3198322711 learned_addr learned my addr 172.21.15.139:0/3198322711 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:05.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.167+0000 7f6f737fe640 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecef0 0x55b94dbe4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.167+0000 7f6f78d42640 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 0x55b94dbe8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f73fff640 1 -- 172.21.15.139:0/3198322711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 msgr2=0x55b94dbe8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f73fff640 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 0x55b94dbe8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f73fff640 1 -- 172.21.15.139:0/3198322711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecef0 msgr2=0x55b94dbe4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f73fff640 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecef0 0x55b94dbe4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f73fff640 1 -- 172.21.15.139:0/3198322711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b94d9077d0 con 0x55b94dbecb10 2024-04-25T10:02:05.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f737fe640 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecef0 0x55b94dbe4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:05.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f78d42640 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 0x55b94dbe8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:05.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f73fff640 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 0x55b94dbe6310 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f6f68001c40 tx=0x7f6f68001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d5c7866ae6ddff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:02:05.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f52ffd640 1 -- 172.21.15.139:0/3198322711 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6801a070 con 0x55b94dbecb10 2024-04-25T10:02:05.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f52ffd640 1 -- 172.21.15.139:0/3198322711 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f6f68023940 con 0x55b94dbecb10 2024-04-25T10:02:05.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f7d586740 1 -- 172.21.15.139:0/3198322711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f6c02f040 con 0x55b94dbecb10 2024-04-25T10:02:05.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f52ffd640 1 -- 172.21.15.139:0/3198322711 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f68001d30 con 0x55b94dbecb10 2024-04-25T10:02:05.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f7d586740 10 client.8925 fetch_fsmap learned FSMap version 75 2024-04-25T10:02:05.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f7d586740 1 -- 172.21.15.139:0/3198322711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55b94d83d3f0 con 0x55b94dbecb10 2024-04-25T10:02:05.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f52ffd640 1 -- 172.21.15.139:0/3198322711 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c02f040 con 0x55b94dbecb10 2024-04-25T10:02:05.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f52ffd640 1 -- 172.21.15.139:0/3198322711 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6f68020070 con 0x55b94dbecb10 2024-04-25T10:02:05.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f7d586740 10 client.8925 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:02:05.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f7d586740 10 client.8925 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:02:05.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f7d586740 20 client.8925 populate_metadata read hostname 'smithi139' 2024-04-25T10:02:05.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f7d586740 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b94dbebe10 0x55b94dbe8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f7d586740 4 client.8925 mds_command: new command op to 8808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:05.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f7d586740 1 -- 172.21.15.139:0/3198322711 --> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b94db362e0 con 0x55b94dbebe10 2024-04-25T10:02:05.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.168+0000 7f6f737fe640 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b94dbebe10 0x55b94dbe8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f52ffd640 10 client.8925 ms_handle_connect on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:05.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f737fe640 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b94dbebe10 0x55b94dbe8670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1182f62813d1f32d server_cookie=4dac75db342a7893 in_seq=0 out_seq=0 2024-04-25T10:02:05.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f52ffd640 1 -- 172.21.15.139:0/3198322711 <== mds.0 v2:172.21.15.139:6835/359626718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b94db362e0 con 0x55b94dbebe10 2024-04-25T10:02:05.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f52ffd640 10 client.8925 handle_command_reply: tid=0 2024-04-25T10:02:05.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f7d586740 1 client.8925 shutdown 2024-04-25T10:02:05.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f7d586740 1 -- 172.21.15.139:0/3198322711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 msgr2=0x55b94dbe6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f7d586740 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 0x55b94dbe6310 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f6f68001c40 tx=0x7f6f68001120 comp rx=0 tx=0).stop 2024-04-25T10:02:05.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f7d586740 1 -- 172.21.15.139:0/3198322711 shutdown_connections 2024-04-25T10:02:05.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f7d586740 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b94dbebe10 0x55b94dbe8670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f7d586740 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 0x55b94dbe8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f7d586740 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecef0 0x55b94dbe4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f52ffd640 0 client.8925 ms_handle_reset on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:05.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f7d586740 1 --2- 172.21.15.139:0/3198322711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 0x55b94dbe6310 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.169+0000 7f6f7d586740 1 -- 172.21.15.139:0/3198322711 >> 172.21.15.139:0/3198322711 conn(0x55b94dbbf770 msgr2=0x55b94dbeb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.170+0000 7f6f7d586740 1 -- 172.21.15.139:0/3198322711 shutdown_connections 2024-04-25T10:02:05.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.170+0000 7f6f7d586740 1 -- 172.21.15.139:0/3198322711 wait complete. 2024-04-25T10:02:05.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.170+0000 7f6f7d586740 20 client.8925 trim_cache size 0 max 16384 2024-04-25T10:02:05.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.184+0000 7f6f7d586740 1 Processor -- start 2024-04-25T10:02:05.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.184+0000 7f6f7d586740 1 -- start start 2024-04-25T10:02:05.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f7d586740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 0x55b94dc0aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f7d586740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 0x55b94dc0e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f7d586740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecef0 0x55b94dbbe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f7d586740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f74200cc0 con 0x55b94dbecb10 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f7d586740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f741ffe40 con 0x55b94dbecef0 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f7d586740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f741ff240 con 0x55b94dbe3ea0 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f737fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 0x55b94dc0e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f737fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 0x55b94dc0e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46436/0 (socket says 172.21.15.139:46436) 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f737fe640 1 -- 172.21.15.139:0/2137208396 learned_addr learned my addr 172.21.15.139:0/2137208396 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f737fe640 1 -- 172.21.15.139:0/2137208396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 msgr2=0x55b94dc0aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f737fe640 1 --2- 172.21.15.139:0/2137208396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 0x55b94dc0aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f737fe640 1 -- 172.21.15.139:0/2137208396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecef0 msgr2=0x55b94dbbe600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f737fe640 1 --2- 172.21.15.139:0/2137208396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecef0 0x55b94dbbe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.185+0000 7f6f737fe640 1 -- 172.21.15.139:0/2137208396 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b94d7fd0f0 con 0x55b94dbecb10 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f737fe640 1 --2- 172.21.15.139:0/2137208396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 0x55b94dc0e150 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f6f740afe90 tx=0x7f6f64008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b57bb610c74e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f597fa640 1 -- 172.21.15.139:0/2137208396 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64017d20 con 0x55b94dbecb10 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f597fa640 1 -- 172.21.15.139:0/2137208396 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f64024070 con 0x55b94dbecb10 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f597fa640 1 -- 172.21.15.139:0/2137208396 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64004270 con 0x55b94dbecb10 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f7d586740 1 -- 172.21.15.139:0/2137208396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 msgr2=0x55b94dc0e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f7d586740 1 --2- 172.21.15.139:0/2137208396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 0x55b94dc0e150 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f6f740afe90 tx=0x7f6f64008040 comp rx=0 tx=0).stop 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f7d586740 1 -- 172.21.15.139:0/2137208396 shutdown_connections 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f7d586740 1 --2- 172.21.15.139:0/2137208396 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecb10 0x55b94dc0e150 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f7d586740 1 --2- 172.21.15.139:0/2137208396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecef0 0x55b94dbbe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f7d586740 1 --2- 172.21.15.139:0/2137208396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 0x55b94dc0aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f7d586740 1 -- 172.21.15.139:0/2137208396 >> 172.21.15.139:0/2137208396 conn(0x55b94dbbf770 msgr2=0x55b94dbe4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f7d586740 1 -- 172.21.15.139:0/2137208396 shutdown_connections 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.186+0000 7f6f7d586740 1 -- 172.21.15.139:0/2137208396 wait complete. 2024-04-25T10:02:05.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.187+0000 7f6f7d586740 1 -- start start 2024-04-25T10:02:05.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.187+0000 7f6f7d586740 1 Processor -- start 2024-04-25T10:02:05.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f7d586740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 0x55b94dc0aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f7d586740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecb10 0x55b94dbbe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f7d586740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecef0 0x55b94dbbeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f7d586740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b94dc0e690 con 0x55b94dbecef0 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f7d586740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f741ff840 con 0x55b94dbecb10 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f7d586740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b94dc0a650 con 0x55b94dbe3ea0 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f7d586740 10 client.0 _ll_register_callbacks cb 0x7f6f6c0748b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f78d42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecef0 0x55b94dbbeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f78d42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecef0 0x55b94dbbeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46440/0 (socket says 172.21.15.139:46440) 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f78d42640 1 -- 172.21.15.139:0/217877130 learned_addr learned my addr 172.21.15.139:0/217877130 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f73fff640 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 0x55b94dc0aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f737fe640 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecb10 0x55b94dbbe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f78d42640 1 -- 172.21.15.139:0/217877130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 msgr2=0x55b94dc0aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f78d42640 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 0x55b94dc0aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f78d42640 1 -- 172.21.15.139:0/217877130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecb10 msgr2=0x55b94dbbe600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f78d42640 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecb10 0x55b94dbbe600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f78d42640 1 -- 172.21.15.139:0/217877130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b94dc744d0 con 0x55b94dbecef0 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f78d42640 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecef0 0x55b94dbbeb40 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00a850 tx=0x7f6f6c0a8290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54965fe166bd047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f52ffd640 1 -- 172.21.15.139:0/217877130 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c0a6060 con 0x55b94dbecef0 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f52ffd640 1 -- 172.21.15.139:0/217877130 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f6f6c080890 con 0x55b94dbecef0 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.188+0000 7f6f7d586740 1 -- 172.21.15.139:0/217877130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f64024070 con 0x55b94dbecef0 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f52ffd640 1 -- 172.21.15.139:0/217877130 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f6c078610 con 0x55b94dbecef0 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f7d586740 10 client.8931 fetch_fsmap learned FSMap version 75 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f7d586740 1 -- 172.21.15.139:0/217877130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55b94dc01ef0 con 0x55b94dbecef0 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f52ffd640 1 -- 172.21.15.139:0/217877130 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f64024070 con 0x55b94dbecef0 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f52ffd640 1 -- 172.21.15.139:0/217877130 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6f6c097080 con 0x55b94dbecef0 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f7d586740 10 client.8931 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f7d586740 10 client.8931 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f7d586740 20 client.8931 populate_metadata read hostname 'smithi139' 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f7d586740 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b94dc74990 0x55b94dc74da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f7d586740 4 client.8931 mds_command: new command op to 8808 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f7d586740 1 -- 172.21.15.139:0/217877130 --> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b94dc0a110 con 0x55b94dc74990 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f73fff640 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b94dc74990 0x55b94dc74da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f52ffd640 10 client.8931 ms_handle_connect on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f73fff640 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b94dc74990 0x55b94dc74da0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddf36a86b7e3542e server_cookie=520c322c7534a11f in_seq=0 out_seq=0 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f52ffd640 1 -- 172.21.15.139:0/217877130 <== mds.0 v2:172.21.15.139:6835/359626718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55b94dc0a110 con 0x55b94dc74990 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f52ffd640 10 client.8931 handle_command_reply: tid=0 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.189+0000 7f6f7d586740 1 client.8931 shutdown 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7d586740 1 -- 172.21.15.139:0/217877130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecef0 msgr2=0x55b94dbbeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7d586740 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecef0 0x55b94dbbeb40 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00a850 tx=0x7f6f6c0a8290 comp rx=0 tx=0).stop 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7d586740 1 -- 172.21.15.139:0/217877130 shutdown_connections 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7d586740 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b94dc74990 0x55b94dc74da0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7d586740 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b94dbecef0 0x55b94dbbeb40 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7d586740 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b94dbecb10 0x55b94dbbe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7d586740 1 --2- 172.21.15.139:0/217877130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b94dbe3ea0 0x55b94dc0aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f52ffd640 0 client.8931 ms_handle_reset on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7d586740 1 -- 172.21.15.139:0/217877130 >> 172.21.15.139:0/217877130 conn(0x55b94dbbf770 msgr2=0x55b94dbc4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7d586740 1 -- 172.21.15.139:0/217877130 shutdown_connections 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7d586740 1 -- 172.21.15.139:0/217877130 wait complete. 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7d586740 20 client.8931 trim_cache size 0 max 16384 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": { 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi139.stdout: "c3ea6cfd-328b-4eda-9ed9-87e67f893a67": { 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi139.stdout: "path": "/", 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi139.stdout: "tag": "c3ea6cfd-328b-4eda-9ed9-87e67f893a67", 2024-04-25T10:02:05.203 INFO:teuthology.orchestra.run.smithi139.stdout: "options": "recursive,repair" 2024-04-25T10:02:05.203 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:05.203 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:05.203 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:02:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2868334305 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6f6006b500 msgr2=0x7f6f6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7a7a5640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6f6006b500 0x7f6f6006d9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6f68007fd0 tx=0x7f6f68009040 comp rx=0 tx=0).stop 2024-04-25T10:02:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2868334305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f74158420 msgr2=0x7f6f740b6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7a7a5640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f74158420 0x7f6f740b6bf0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c000f80 tx=0x7f6f6c02d070 comp rx=0 tx=0).stop 2024-04-25T10:02:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2868334305 shutdown_connections 2024-04-25T10:02:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7a7a5640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f74158420 0x7f6f740b6bf0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7a7a5640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f6f6006b500 0x7f6f6006d9c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7a7a5640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f7415a150 0x7f6f74155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7a7a5640 1 --2- 172.21.15.139:0/2868334305 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f5c0051c0 0x7f6f74156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2868334305 >> 172.21.15.139:0/2868334305 conn(0x7f6f740b31f0 msgr2=0x7f6f740706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2868334305 shutdown_connections 2024-04-25T10:02:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.190+0000 7f6f7a7a5640 1 -- 172.21.15.139:0/2868334305 wait complete. 2024-04-25T10:02:05.205 INFO:tasks.cephfs.filesystem:scrub status for tag:c3ea6cfd-328b-4eda-9ed9-87e67f893a67 - {'path': '/', 'tag': 'c3ea6cfd-328b-4eda-9ed9-87e67f893a67', 'options': 'recursive,repair'} 2024-04-25T10:02:05.521 INFO:tasks.ceph.mds.c.smithi139.stderr:2024-04-25T10:02:05.517+0000 7f23ff03b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:02:05.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:05.852+0000 7f26317fa640 1 -- 172.21.15.139:0/3000531873 <== mon.0 v2:172.21.15.139:3300/0 7 ==== log(8 entries from seq 1 at 2024-04-25T10:02:05.044026+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7f262c088f40 con 0x7f263415b860 2024-04-25T10:02:05.857 INFO:teuthology.orchestra.run.smithi139.stdout: cluster: 2024-04-25T10:02:05.857 INFO:teuthology.orchestra.run.smithi139.stdout: id: 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T10:02:05.857 INFO:teuthology.orchestra.run.smithi139.stdout: health: HEALTH_OK 2024-04-25T10:02:05.857 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:05.857 INFO:teuthology.orchestra.run.smithi139.stdout: services: 2024-04-25T10:02:05.857 INFO:teuthology.orchestra.run.smithi139.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-04-25T10:02:05.857 INFO:teuthology.orchestra.run.smithi139.stdout: mgr: y(active, since 9m), standbys: x 2024-04-25T10:02:05.857 INFO:teuthology.orchestra.run.smithi139.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:02:05.857 INFO:teuthology.orchestra.run.smithi139.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-04-25T10:02:05.857 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout: data: 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout: volumes: 1/1 healthy 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout: objects: 30 objects, 19 MiB 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout: pgs: 97 active+clean 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout: io: 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout: client: 3.0 KiB/s rd, 170 B/s wr, 3 op/s rd, 0 op/s wr 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout: recovery: 683 KiB/s, 0 objects/s 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:02:05.044026+0000 mds.c [INF] scrub queued for path: / 2024-04-25T10:02:05.858 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:02:05.044029+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:02:05.859 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:02:05.044034+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-25T10:02:05.859 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:02:05.044533+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:02:05.859 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:02:05.044541+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-04-25T10:02:05.859 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:02:05.045137+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:02:05.859 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:02:05.045143+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-04-25T10:02:05.859 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:02:05.045520+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:02:10.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:10.859+0000 7f26317fa640 1 -- 172.21.15.139:0/3000531873 <== mon.0 v2:172.21.15.139:3300/0 8 ==== log(1 entries from seq 9 at 2024-04-25T10:02:09.547961+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f262c0057a0 con 0x7f263415b860 2024-04-25T10:02:10.864 INFO:teuthology.orchestra.run.smithi139.stdout:2024-04-25T10:02:09.547961+0000 mds.c [INF] scrub summary: idle 2024-04-25T10:02:17.173 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:17.176 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:35.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:35.206 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:02:35.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.313+0000 7f39c085d640 1 Processor -- start 2024-04-25T10:02:35.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.313+0000 7f39c085d640 1 -- start start 2024-04-25T10:02:35.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.314+0000 7f39c085d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f39bc15cb20 0x7f39bc0cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.314+0000 7f39c085d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f39bc0cfeb0 0x7f39bc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.314+0000 7f39c085d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39bc0d02c0 con 0x7f39bc0afc00 2024-04-25T10:02:35.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.314+0000 7f39c085d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39bc0703c0 con 0x7f39bc0cfeb0 2024-04-25T10:02:35.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.314+0000 7f39c085d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39bc0cd820 con 0x7f39bc15cb20 2024-04-25T10:02:35.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.314+0000 7f39bad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f39bc0cfeb0 0x7f39bc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.314+0000 7f39bad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f39bc0cfeb0 0x7f39bc06e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41254/0 (socket says 172.21.15.139:41254) 2024-04-25T10:02:35.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.314+0000 7f39bad9d640 1 -- 172.21.15.139:0/4112235206 learned_addr learned my addr 172.21.15.139:0/4112235206 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:35.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.314+0000 7f39b9d9b640 1 --2- 172.21.15.139:0/4112235206 >> v2:172.21.15.150:3300/0 conn(0x7f39bc15cb20 0x7f39bc0cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.314+0000 7f39bad9d640 1 -- 172.21.15.139:0/4112235206 >> v2:172.21.15.150:3300/0 conn(0x7f39bc15cb20 msgr2=0x7f39bc0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.315+0000 7f39bad9d640 1 --2- 172.21.15.139:0/4112235206 >> v2:172.21.15.150:3300/0 conn(0x7f39bc15cb20 0x7f39bc0cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.315+0000 7f39bad9d640 1 -- 172.21.15.139:0/4112235206 >> v1:172.21.15.150:6789/0 conn(0x7f39bc0afc00 legacy=0x7f39bc06fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.315+0000 7f39bad9d640 1 -- 172.21.15.139:0/4112235206 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0ccef0 con 0x7f39bc0cfeb0 2024-04-25T10:02:35.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.315+0000 7f39b9d9b640 1 --2- 172.21.15.139:0/4112235206 >> v2:172.21.15.150:3300/0 conn(0x7f39bc15cb20 0x7f39bc0cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:35.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.315+0000 7f39bad9d640 1 --2- 172.21.15.139:0/4112235206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc0cfeb0 0x7f39bc06e370 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f39bc15f3d0 tx=0x7f39b402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed8a97d895e42c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.315+0000 7f39b959a640 1 -- 172.21.15.139:0/4112235206 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4036070 con 0x7f39bc0cfeb0 2024-04-25T10:02:35.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.315+0000 7f39b959a640 1 -- 172.21.15.139:0/4112235206 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b4031600 con 0x7f39bc0cfeb0 2024-04-25T10:02:35.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.315+0000 7f39b959a640 1 -- 172.21.15.139:0/4112235206 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b40319d0 con 0x7f39bc0cfeb0 2024-04-25T10:02:35.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.316+0000 7f39c085d640 1 -- 172.21.15.139:0/4112235206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc0cfeb0 msgr2=0x7f39bc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.316+0000 7f39c085d640 1 --2- 172.21.15.139:0/4112235206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc0cfeb0 0x7f39bc06e370 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f39bc15f3d0 tx=0x7f39b402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.316+0000 7f39c085d640 1 -- 172.21.15.139:0/4112235206 shutdown_connections 2024-04-25T10:02:35.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.316+0000 7f39c085d640 1 --2- 172.21.15.139:0/4112235206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc0cfeb0 0x7f39bc06e370 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.316+0000 7f39c085d640 1 --2- 172.21.15.139:0/4112235206 >> v2:172.21.15.150:3300/0 conn(0x7f39bc15cb20 0x7f39bc0cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.316+0000 7f39c085d640 1 -- 172.21.15.139:0/4112235206 >> 172.21.15.139:0/4112235206 conn(0x7f39bc0b2230 msgr2=0x7f39bc0b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.316+0000 7f39c085d640 1 -- 172.21.15.139:0/4112235206 shutdown_connections 2024-04-25T10:02:35.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.316+0000 7f39c085d640 1 -- 172.21.15.139:0/4112235206 wait complete. 2024-04-25T10:02:35.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.317+0000 7f39c085d640 1 Processor -- start 2024-04-25T10:02:35.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.318+0000 7f39c085d640 1 -- start start 2024-04-25T10:02:35.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.318+0000 7f39c085d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc0afc00 0x7f39bc0c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.318+0000 7f39c085d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39bc0cfeb0 0x7f39bc1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.318+0000 7f39c085d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39bc15cb20 0x7f39bc0ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.318+0000 7f39c085d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0cd4b0 con 0x7f39bc0afc00 2024-04-25T10:02:35.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.318+0000 7f39c085d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0affe0 con 0x7f39bc0cfeb0 2024-04-25T10:02:35.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.318+0000 7f39c085d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0cdcb0 con 0x7f39bc15cb20 2024-04-25T10:02:35.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.318+0000 7f39ba59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc0afc00 0x7f39bc0c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.318+0000 7f39ba59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc0afc00 0x7f39bc0c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41264/0 (socket says 172.21.15.139:41264) 2024-04-25T10:02:35.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.318+0000 7f39ba59c640 1 -- 172.21.15.139:0/3231045646 learned_addr learned my addr 172.21.15.139:0/3231045646 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:35.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39bad9d640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39bc15cb20 0x7f39bc0ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39ba59c640 1 -- 172.21.15.139:0/3231045646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39bc15cb20 msgr2=0x7f39bc0ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39ba59c640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39bc15cb20 0x7f39bc0ae380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39ba59c640 1 -- 172.21.15.139:0/3231045646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39bc0cfeb0 msgr2=0x7f39bc1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39b9d9b640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39bc0cfeb0 0x7f39bc1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39ba59c640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39bc0cfeb0 0x7f39bc1573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39ba59c640 1 -- 172.21.15.139:0/3231045646 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0af340 con 0x7f39bc0afc00 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39ba59c640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc0afc00 0x7f39bc0c7d40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00b3f0 tx=0x7f39ac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39aaffd640 1 -- 172.21.15.139:0/3231045646 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac015020 con 0x7f39bc0afc00 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39aaffd640 1 -- 172.21.15.139:0/3231045646 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac011830 con 0x7f39bc0afc00 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39c085d640 1 -- 172.21.15.139:0/3231045646 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc0c5820 con 0x7f39bc0afc00 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39aaffd640 1 -- 172.21.15.139:0/3231045646 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac011c20 con 0x7f39bc0afc00 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.319+0000 7f39c085d640 1 -- 172.21.15.139:0/3231045646 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc0c5de0 con 0x7f39bc0afc00 2024-04-25T10:02:35.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.321+0000 7f39aaffd640 1 -- 172.21.15.139:0/3231045646 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39ac018020 con 0x7f39bc0afc00 2024-04-25T10:02:35.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.321+0000 7f39aaffd640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f399c06b4d0 0x7f399c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.321+0000 7f39aaffd640 1 -- 172.21.15.139:0/3231045646 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f39ac089fd0 con 0x7f39bc0afc00 2024-04-25T10:02:35.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.321+0000 7f39b9d9b640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f399c06b4d0 0x7f399c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.321+0000 7f39b9d9b640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f399c06b4d0 0x7f399c06d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0c7680 tx=0x7f39b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.322+0000 7f39c3567740 1 Processor -- start 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.322+0000 7f39c3567740 1 -- start start 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39c3567740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 0x55b86bc59320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39c3567740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc56ef0 0x55b86bc59860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39c3567740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc29770 0x55b86bc2cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39c3567740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86bb92da0 con 0x55b86bc56b10 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39c3567740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86bb3bb80 con 0x55b86bc29770 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39c3567740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b86bb8ee40 con 0x55b86bc56ef0 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39ba59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 0x55b86bc59320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39ba59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 0x55b86bc59320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41268/0 (socket says 172.21.15.139:41268) 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39ba59c640 1 -- 172.21.15.139:0/2959478375 learned_addr learned my addr 172.21.15.139:0/2959478375 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39ba59c640 1 -- 172.21.15.139:0/2959478375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc56ef0 msgr2=0x55b86bc59860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39ba59c640 1 --2- 172.21.15.139:0/2959478375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc56ef0 0x55b86bc59860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39bad9d640 1 --2- 172.21.15.139:0/2959478375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc29770 0x55b86bc2cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39ba59c640 1 -- 172.21.15.139:0/2959478375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc29770 msgr2=0x55b86bc2cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39ba59c640 1 --2- 172.21.15.139:0/2959478375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc29770 0x55b86bc2cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39ba59c640 1 -- 172.21.15.139:0/2959478375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b86bb45f10 con 0x55b86bc56b10 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39bad9d640 1 --2- 172.21.15.139:0/2959478375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc29770 0x55b86bc2cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f39ba59c640 1 --2- 172.21.15.139:0/2959478375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 0x55b86bc59320 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009370 tx=0x7f39ac058700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a224dde5be0107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.323+0000 7f399bfff640 1 -- 172.21.15.139:0/2959478375 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0568c0 con 0x55b86bc56b10 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.324+0000 7f399bfff640 1 -- 172.21.15.139:0/2959478375 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac054c30 con 0x55b86bc56b10 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.324+0000 7f399bfff640 1 -- 172.21.15.139:0/2959478375 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac057c50 con 0x55b86bc56b10 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.324+0000 7f39c3567740 1 -- 172.21.15.139:0/2959478375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 msgr2=0x55b86bc59320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.324+0000 7f39c3567740 1 --2- 172.21.15.139:0/2959478375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 0x55b86bc59320 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009370 tx=0x7f39ac058700 comp rx=0 tx=0).stop 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.324+0000 7f39c3567740 1 -- 172.21.15.139:0/2959478375 shutdown_connections 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.324+0000 7f39c3567740 1 --2- 172.21.15.139:0/2959478375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc29770 0x55b86bc2cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.324+0000 7f39c3567740 1 --2- 172.21.15.139:0/2959478375 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc56ef0 0x55b86bc59860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.324+0000 7f39c3567740 1 --2- 172.21.15.139:0/2959478375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 0x55b86bc59320 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.324+0000 7f39c3567740 1 -- 172.21.15.139:0/2959478375 >> 172.21.15.139:0/2959478375 conn(0x55b86bc2b9e0 msgr2=0x55b86bc74320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.324+0000 7f39c3567740 1 -- 172.21.15.139:0/2959478375 shutdown_connections 2024-04-25T10:02:35.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.324+0000 7f39c3567740 1 -- 172.21.15.139:0/2959478375 wait complete. 2024-04-25T10:02:35.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.325+0000 7f39c3567740 1 -- start start 2024-04-25T10:02:35.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.325+0000 7f39c3567740 1 Processor -- start 2024-04-25T10:02:35.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39c3567740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 0x55b86bc7c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39c3567740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56ef0 0x55b86bc77230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39c3567740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc5b900 0x55b86bc5bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39c3567740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86bc2eb90 con 0x55b86bc56b10 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39c3567740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86bb314b0 con 0x55b86bc56ef0 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39c3567740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b86bc2ee70 con 0x55b86bc5b900 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39c3567740 10 client.0 _ll_register_callbacks cb 0x55b86bb71180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39ba59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 0x55b86bc7c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39ba59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 0x55b86bc7c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41280/0 (socket says 172.21.15.139:41280) 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39ba59c640 1 -- 172.21.15.139:0/1679055440 learned_addr learned my addr 172.21.15.139:0/1679055440 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39ba59c640 1 -- 172.21.15.139:0/1679055440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc5b900 msgr2=0x55b86bc5bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39ba59c640 1 --2- 172.21.15.139:0/1679055440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc5b900 0x55b86bc5bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39ba59c640 1 -- 172.21.15.139:0/1679055440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56ef0 msgr2=0x55b86bc77230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39ba59c640 1 --2- 172.21.15.139:0/1679055440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56ef0 0x55b86bc77230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39ba59c640 1 -- 172.21.15.139:0/1679055440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b86b9717d0 con 0x55b86bc56b10 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39ba59c640 1 --2- 172.21.15.139:0/1679055440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 0x55b86bc7c480 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f39ac058550 tx=0x7f39ac0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea8a5b0498af046 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39997fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:02:35.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.326+0000 7f39997fa640 1 -- 172.21.15.139:0/1679055440 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac085070 con 0x55b86bc56b10 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39997fa640 1 -- 172.21.15.139:0/1679055440 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f39ac057c10 con 0x55b86bc56b10 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39c3567740 1 -- 172.21.15.139:0/1679055440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39ac054c30 con 0x55b86bc56b10 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39997fa640 1 -- 172.21.15.139:0/1679055440 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39ac058390 con 0x55b86bc56b10 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39c3567740 10 client.8943 fetch_fsmap learned FSMap version 75 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39c3567740 1 -- 172.21.15.139:0/1679055440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55b86b8a73f0 con 0x55b86bc56b10 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39997fa640 1 -- 172.21.15.139:0/1679055440 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac054c30 con 0x55b86bc56b10 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39997fa640 1 -- 172.21.15.139:0/1679055440 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f39ac054310 con 0x55b86bc56b10 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39c3567740 10 client.8943 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39c3567740 10 client.8943 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39c3567740 20 client.8943 populate_metadata read hostname 'smithi139' 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39c3567740 1 --2- 172.21.15.139:0/1679055440 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b86bc847c0 0x55b86bc84c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39c3567740 4 client.8943 mds_command: new command op to 8808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39c3567740 1 -- 172.21.15.139:0/1679055440 --> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b86bba02e0 con 0x55b86bc847c0 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39b9d9b640 1 --2- 172.21.15.139:0/1679055440 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b86bc847c0 0x55b86bc84c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39997fa640 10 client.8943 ms_handle_connect on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.327+0000 7f39b9d9b640 1 --2- 172.21.15.139:0/1679055440 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b86bc847c0 0x55b86bc84c00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfdbe28b9a804486 server_cookie=cb4a2942b69055bc in_seq=0 out_seq=0 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39997fa640 1 -- 172.21.15.139:0/1679055440 <== mds.0 v2:172.21.15.139:6835/359626718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b86bba02e0 con 0x55b86bc847c0 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39997fa640 10 client.8943 handle_command_reply: tid=0 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39c3567740 1 client.8943 shutdown 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39c3567740 1 -- 172.21.15.139:0/1679055440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 msgr2=0x55b86bc7c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39c3567740 1 --2- 172.21.15.139:0/1679055440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 0x55b86bc7c480 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f39ac058550 tx=0x7f39ac0092e0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39c3567740 1 -- 172.21.15.139:0/1679055440 shutdown_connections 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39c3567740 1 --2- 172.21.15.139:0/1679055440 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b86bc847c0 0x55b86bc84c00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39c3567740 1 --2- 172.21.15.139:0/1679055440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc5b900 0x55b86bc5bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39997fa640 0 client.8943 ms_handle_reset on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39c3567740 1 --2- 172.21.15.139:0/1679055440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56ef0 0x55b86bc77230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39c3567740 1 --2- 172.21.15.139:0/1679055440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56b10 0x55b86bc7c480 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39c3567740 1 -- 172.21.15.139:0/1679055440 >> 172.21.15.139:0/1679055440 conn(0x55b86bc29770 msgr2=0x55b86bc2e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39c3567740 1 -- 172.21.15.139:0/1679055440 shutdown_connections 2024-04-25T10:02:35.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39c3567740 1 -- 172.21.15.139:0/1679055440 wait complete. 2024-04-25T10:02:35.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.328+0000 7f39c3567740 20 client.8943 trim_cache size 0 max 16384 2024-04-25T10:02:35.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.343+0000 7f39c3567740 1 Processor -- start 2024-04-25T10:02:35.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.343+0000 7f39c3567740 1 -- start start 2024-04-25T10:02:35.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39c3567740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56b10 0x55b86bc5b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39c3567740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56ef0 0x55b86bc86da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39c3567740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc5b900 0x55b86bc24b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39c3567740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc200690 con 0x55b86bc56ef0 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39c3567740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc1ff860 con 0x55b86bc56b10 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39c3567740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0c7170 con 0x55b86bc5b900 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39b9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56ef0 0x55b86bc86da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39b9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56ef0 0x55b86bc86da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41286/0 (socket says 172.21.15.139:41286) 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39b9d9b640 1 -- 172.21.15.139:0/886620273 learned_addr learned my addr 172.21.15.139:0/886620273 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39bad9d640 1 --2- 172.21.15.139:0/886620273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc5b900 0x55b86bc24b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39ba59c640 1 --2- 172.21.15.139:0/886620273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56b10 0x55b86bc5b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39b9d9b640 1 -- 172.21.15.139:0/886620273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc5b900 msgr2=0x55b86bc24b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39b9d9b640 1 --2- 172.21.15.139:0/886620273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc5b900 0x55b86bc24b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39b9d9b640 1 -- 172.21.15.139:0/886620273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56b10 msgr2=0x55b86bc5b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.344+0000 7f39b9d9b640 1 --2- 172.21.15.139:0/886620273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56b10 0x55b86bc5b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f39b9d9b640 1 -- 172.21.15.139:0/886620273 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b86b8670f0 con 0x55b86bc56ef0 2024-04-25T10:02:35.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f39b9d9b640 1 --2- 172.21.15.139:0/886620273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56ef0 0x55b86bc86da0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f39bc156940 tx=0x7f39b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9e4902af212818 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f399b7fe640 1 -- 172.21.15.139:0/886620273 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0021330 con 0x55b86bc56ef0 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f399b7fe640 1 -- 172.21.15.139:0/886620273 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b0025030 con 0x55b86bc56ef0 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f399b7fe640 1 -- 172.21.15.139:0/886620273 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0021710 con 0x55b86bc56ef0 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f39c3567740 1 -- 172.21.15.139:0/886620273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56ef0 msgr2=0x55b86bc86da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f39c3567740 1 --2- 172.21.15.139:0/886620273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56ef0 0x55b86bc86da0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f39bc156940 tx=0x7f39b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f39c3567740 1 -- 172.21.15.139:0/886620273 shutdown_connections 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f39c3567740 1 --2- 172.21.15.139:0/886620273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc56ef0 0x55b86bc86da0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f39c3567740 1 --2- 172.21.15.139:0/886620273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc5b900 0x55b86bc24b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f39c3567740 1 --2- 172.21.15.139:0/886620273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56b10 0x55b86bc5b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f39c3567740 1 -- 172.21.15.139:0/886620273 >> 172.21.15.139:0/886620273 conn(0x55b86bc29770 msgr2=0x55b86bc77b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f39c3567740 1 -- 172.21.15.139:0/886620273 shutdown_connections 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.345+0000 7f39c3567740 1 -- 172.21.15.139:0/886620273 wait complete. 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.346+0000 7f39c3567740 1 -- start start 2024-04-25T10:02:35.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.346+0000 7f39c3567740 1 Processor -- start 2024-04-25T10:02:35.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39c3567740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56b10 0x55b86bc5bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39c3567740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc56ef0 0x55b86bcdd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39c3567740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc5b900 0x55b86bc2e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39c3567740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86bc30a00 con 0x55b86bc5b900 2024-04-25T10:02:35.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39c3567740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc1f7f20 con 0x55b86bc56b10 2024-04-25T10:02:35.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39c3567740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b86bc307b0 con 0x55b86bc56ef0 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39c3567740 10 client.0 _ll_register_callbacks cb 0x7f39ac058100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39bad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc5b900 0x55b86bc2e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39bad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc5b900 0x55b86bc2e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41292/0 (socket says 172.21.15.139:41292) 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39bad9d640 1 -- 172.21.15.139:0/1145903027 learned_addr learned my addr 172.21.15.139:0/1145903027 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39b9d9b640 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc56ef0 0x55b86bcdd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39ba59c640 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56b10 0x55b86bc5bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39bad9d640 1 -- 172.21.15.139:0/1145903027 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc56ef0 msgr2=0x55b86bcdd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39bad9d640 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc56ef0 0x55b86bcdd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39bad9d640 1 -- 172.21.15.139:0/1145903027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56b10 msgr2=0x55b86bc5bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39bad9d640 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56b10 0x55b86bc5bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.347+0000 7f39bad9d640 1 -- 172.21.15.139:0/1145903027 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b86bc73df0 con 0x55b86bc5b900 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39bad9d640 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc5b900 0x55b86bc2e1b0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009f40 tx=0x7f39b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc650969a908c514 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39997fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:02:35.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39997fa640 1 -- 172.21.15.139:0/1145903027 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4036040 con 0x55b86bc5b900 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39997fa640 1 -- 172.21.15.139:0/1145903027 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f39b4031de0 con 0x55b86bc5b900 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39c3567740 1 -- 172.21.15.139:0/1145903027 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39b0025030 con 0x55b86bc5b900 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39997fa640 1 -- 172.21.15.139:0/1145903027 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39b404d050 con 0x55b86bc5b900 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39c3567740 10 client.8949 fetch_fsmap learned FSMap version 75 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39c3567740 1 -- 172.21.15.139:0/1145903027 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55b86bc59760 con 0x55b86bc5b900 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39997fa640 1 -- 172.21.15.139:0/1145903027 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b0025030 con 0x55b86bc5b900 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39997fa640 1 -- 172.21.15.139:0/1145903027 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f39b4050020 con 0x55b86bc5b900 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39c3567740 10 client.8949 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39c3567740 10 client.8949 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39c3567740 20 client.8949 populate_metadata read hostname 'smithi139' 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39c3567740 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b86bc2e6f0 0x55b86bc55340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39c3567740 4 client.8949 mds_command: new command op to 8808 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39c3567740 1 -- 172.21.15.139:0/1145903027 --> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b86bb9a630 con 0x55b86bc2e6f0 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.348+0000 7f39ba59c640 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b86bc2e6f0 0x55b86bc55340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39997fa640 10 client.8949 ms_handle_connect on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39ba59c640 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b86bc2e6f0 0x55b86bc55340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f660445c7af7d48 server_cookie=1150dd1ce28614cc in_seq=0 out_seq=0 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39997fa640 1 -- 172.21.15.139:0/1145903027 <== mds.0 v2:172.21.15.139:6835/359626718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b86bb9a630 con 0x55b86bc2e6f0 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39997fa640 10 client.8949 handle_command_reply: tid=0 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39c3567740 1 client.8949 shutdown 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39c3567740 1 -- 172.21.15.139:0/1145903027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc5b900 msgr2=0x55b86bc2e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39c3567740 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc5b900 0x55b86bc2e1b0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009f40 tx=0x7f39b4004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39c3567740 1 -- 172.21.15.139:0/1145903027 shutdown_connections 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39c3567740 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.139:6835/359626718,v1:172.21.15.139:6837/359626718] conn(0x55b86bc2e6f0 0x55b86bc55340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39c3567740 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b86bc5b900 0x55b86bc2e1b0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39c3567740 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86bc56ef0 0x55b86bcdd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39c3567740 1 --2- 172.21.15.139:0/1145903027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86bc56b10 0x55b86bc5bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39997fa640 0 client.8949 ms_handle_reset on v2:172.21.15.139:6835/359626718 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39c3567740 1 -- 172.21.15.139:0/1145903027 >> 172.21.15.139:0/1145903027 conn(0x55b86bc29770 msgr2=0x55b86bc77ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39c3567740 1 -- 172.21.15.139:0/1145903027 shutdown_connections 2024-04-25T10:02:35.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39c3567740 1 -- 172.21.15.139:0/1145903027 wait complete. 2024-04-25T10:02:35.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.349+0000 7f39c3567740 20 client.8949 trim_cache size 0 max 16384 2024-04-25T10:02:35.361 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:02:35.361 INFO:teuthology.orchestra.run.smithi139.stdout: "status": "no active scrubs running", 2024-04-25T10:02:35.361 INFO:teuthology.orchestra.run.smithi139.stdout: "scrubs": {} 2024-04-25T10:02:35.361 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:02:35.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.350+0000 7f39c085d640 1 -- 172.21.15.139:0/3231045646 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f399c06b4d0 msgr2=0x7f399c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.350+0000 7f39c085d640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f399c06b4d0 0x7f399c06d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0c7680 tx=0x7f39b0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:35.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.350+0000 7f39c085d640 1 -- 172.21.15.139:0/3231045646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc0afc00 msgr2=0x7f39bc0c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.350+0000 7f39c085d640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc0afc00 0x7f39bc0c7d40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00b3f0 tx=0x7f39ac00b8b0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.350+0000 7f39c085d640 1 -- 172.21.15.139:0/3231045646 shutdown_connections 2024-04-25T10:02:35.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.350+0000 7f39c085d640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f399c06b4d0 0x7f399c06d990 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.350+0000 7f39c085d640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39bc15cb20 0x7f39bc0ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.350+0000 7f39c085d640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39bc0cfeb0 0x7f39bc1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.350+0000 7f39c085d640 1 --2- 172.21.15.139:0/3231045646 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39bc0afc00 0x7f39bc0c7d40 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.350+0000 7f39c085d640 1 -- 172.21.15.139:0/3231045646 >> 172.21.15.139:0/3231045646 conn(0x7f39bc0b2230 msgr2=0x7f39bc073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.350+0000 7f39c085d640 1 -- 172.21.15.139:0/3231045646 shutdown_connections 2024-04-25T10:02:35.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.350+0000 7f39c085d640 1 -- 172.21.15.139:0/3231045646 wait complete. 2024-04-25T10:02:35.366 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:02:35.467 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:02:35.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.559+0000 7f7dadd5c640 1 Processor -- start 2024-04-25T10:02:35.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.559+0000 7f7dadd5c640 1 -- start start 2024-04-25T10:02:35.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.560+0000 7f7dadd5c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da815dbf0 0x7f7da81617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.560+0000 7f7dadd5c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7da80b9370 0x7f7da80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.560+0000 7f7dadd5c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da8161dc0 con 0x7f7da80cec80 2024-04-25T10:02:35.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.560+0000 7f7dadd5c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da80d1ee0 con 0x7f7da80b9370 2024-04-25T10:02:35.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.560+0000 7f7dadd5c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da81620a0 con 0x7f7da815dbf0 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.560+0000 7f7da7fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f7da80cec80 0x7f7da806c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53270/0 (socket says 172.21.15.139:53270) 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.560+0000 7f7da7fff640 1 -- 172.21.15.139:0/3510582361 learned_addr learned my addr 172.21.15.139:0/3510582361 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130224072 0 0) 0x7f7da8161dc0 con 0x7f7da80cec80 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da6ffd640 1 --2- 172.21.15.139:0/3510582361 >> v2:172.21.15.150:3300/0 conn(0x7f7da80b9370 0x7f7da80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da77fe640 1 --2- 172.21.15.139:0/3510582361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da815dbf0 0x7f7da81617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d90003520 con 0x7f7da80cec80 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1420212164 0 0) 0x7f7d90003520 con 0x7f7da80cec80 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da815dbf0 msgr2=0x7f7da81617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 --2- 172.21.15.139:0/3510582361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da815dbf0 0x7f7da81617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 >> v2:172.21.15.150:3300/0 conn(0x7f7da80b9370 msgr2=0x7f7da80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 --2- 172.21.15.139:0/3510582361 >> v2:172.21.15.150:3300/0 conn(0x7f7da80b9370 0x7f7da80ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da8162380 con 0x7f7da80cec80 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f7da00019f0 con 0x7f7da80cec80 2024-04-25T10:02:35.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 >> v1:172.21.15.139:6789/0 conn(0x7f7da80cec80 legacy=0x7f7da806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 --2- 172.21.15.139:0/3510582361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d900039b0 0x7f7d90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 --2- 172.21.15.139:0/3510582361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d90004800 0x7f7d90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 --2- 172.21.15.139:0/3510582361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da815dbf0 0x7f7d900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80d1ee0 con 0x7f7d900039b0 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da81620a0 con 0x7f7d90004800 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9000b9c0 con 0x7f7da815dbf0 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da77fe640 1 --2- 172.21.15.139:0/3510582361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d900039b0 0x7f7d90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da7fff640 1 --2- 172.21.15.139:0/3510582361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da815dbf0 0x7f7d900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da6ffd640 1 --2- 172.21.15.139:0/3510582361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d90004800 0x7f7d90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da77fe640 1 -- 172.21.15.139:0/3510582361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da815dbf0 msgr2=0x7f7d900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da77fe640 1 --2- 172.21.15.139:0/3510582361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da815dbf0 0x7f7d900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da77fe640 1 -- 172.21.15.139:0/3510582361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d90004800 msgr2=0x7f7d90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da77fe640 1 --2- 172.21.15.139:0/3510582361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d90004800 0x7f7d90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.561+0000 7f7da77fe640 1 -- 172.21.15.139:0/3510582361 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d90003520 con 0x7f7d900039b0 2024-04-25T10:02:35.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.562+0000 7f7da77fe640 1 --2- 172.21.15.139:0/3510582361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d900039b0 0x7f7d90003dd0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009ac0 tx=0x7f7d90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60099fb4afbb19d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.562+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98036070 con 0x7f7d900039b0 2024-04-25T10:02:35.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.562+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d98031400 con 0x7f7d900039b0 2024-04-25T10:02:35.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.562+0000 7f7da67fc640 1 -- 172.21.15.139:0/3510582361 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98009690 con 0x7f7d900039b0 2024-04-25T10:02:35.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.563+0000 7f7dadd5c640 1 -- 172.21.15.139:0/3510582361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d900039b0 msgr2=0x7f7d90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.563+0000 7f7dadd5c640 1 --2- 172.21.15.139:0/3510582361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d900039b0 0x7f7d90003dd0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009ac0 tx=0x7f7d90005830 comp rx=0 tx=0).stop 2024-04-25T10:02:35.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.563+0000 7f7da7fff640 1 -- 172.21.15.139:0/3510582361 reap_dead start 2024-04-25T10:02:35.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.563+0000 7f7dadd5c640 1 -- 172.21.15.139:0/3510582361 shutdown_connections 2024-04-25T10:02:35.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.563+0000 7f7dadd5c640 1 -- 172.21.15.139:0/3510582361 >> 172.21.15.139:0/3510582361 conn(0x7f7da80b5480 msgr2=0x7f7da80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.563+0000 7f7dadd5c640 1 -- 172.21.15.139:0/3510582361 shutdown_connections 2024-04-25T10:02:35.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.563+0000 7f7dadd5c640 1 -- 172.21.15.139:0/3510582361 wait complete. 2024-04-25T10:02:35.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.564+0000 7f7dadd5c640 1 Processor -- start 2024-04-25T10:02:35.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.564+0000 7f7dadd5c640 1 -- start start 2024-04-25T10:02:35.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7dadd5c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d900039b0 0x7f7da814be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7dadd5c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da81607f0 0x7f7da8159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7dadd5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7da80b7dd0 0x7f7da80b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7dadd5c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da815dfd0 con 0x7f7da80b7dd0 2024-04-25T10:02:35.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7dadd5c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80b97d0 con 0x7f7d900039b0 2024-04-25T10:02:35.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7dadd5c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da814c3d0 con 0x7f7da81607f0 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7da7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7da80b7dd0 0x7f7da80b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7da7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7da80b7dd0 0x7f7da80b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41310/0 (socket says 172.21.15.139:41310) 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7da7fff640 1 -- 172.21.15.139:0/1966499430 learned_addr learned my addr 172.21.15.139:0/1966499430 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7da77fe640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d900039b0 0x7f7da814be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7da6ffd640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da81607f0 0x7f7da8159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7da7fff640 1 -- 172.21.15.139:0/1966499430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da81607f0 msgr2=0x7f7da8159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7da7fff640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da81607f0 0x7f7da8159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7da7fff640 1 -- 172.21.15.139:0/1966499430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d900039b0 msgr2=0x7f7da814be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7da7fff640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d900039b0 0x7f7da814be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7da7fff640 1 -- 172.21.15.139:0/1966499430 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da815a3b0 con 0x7f7da80b7dd0 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7da6ffd640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da81607f0 0x7f7da8159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7da7fff640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7da80b7dd0 0x7f7da80b8260 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f7da80d14a0 tx=0x7f7da000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7d8ffff640 1 -- 172.21.15.139:0/1966499430 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da000ee20 con 0x7f7da80b7dd0 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7d8ffff640 1 -- 172.21.15.139:0/1966499430 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da0018070 con 0x7f7da80b7dd0 2024-04-25T10:02:35.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7dadd5c640 1 -- 172.21.15.139:0/1966499430 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da815a690 con 0x7f7da80b7dd0 2024-04-25T10:02:35.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7d8ffff640 1 -- 172.21.15.139:0/1966499430 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0013660 con 0x7f7da80b7dd0 2024-04-25T10:02:35.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.565+0000 7f7dadd5c640 1 -- 172.21.15.139:0/1966499430 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da815ac50 con 0x7f7da80b7dd0 2024-04-25T10:02:35.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.567+0000 7f7dadd5c640 1 -- 172.21.15.139:0/1966499430 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da8074520 con 0x7f7da80b7dd0 2024-04-25T10:02:35.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.567+0000 7f7d8ffff640 1 -- 172.21.15.139:0/1966499430 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7da0013800 con 0x7f7da80b7dd0 2024-04-25T10:02:35.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.567+0000 7f7d8ffff640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7d9406b4d0 0x7f7d9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.567+0000 7f7d8ffff640 1 -- 172.21.15.139:0/1966499430 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f7da008a7a0 con 0x7f7da80b7dd0 2024-04-25T10:02:35.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.567+0000 7f7da77fe640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7d9406b4d0 0x7f7d9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.568+0000 7f7da77fe640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7d9406b4d0 0x7f7d9406d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7d98033000 tx=0x7f7d98008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.571+0000 7f7d8ffff640 1 -- 172.21.15.139:0/1966499430 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7da0059400 con 0x7f7da80b7dd0 2024-04-25T10:02:35.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.774+0000 7f7dadd5c640 1 -- 172.21.15.139:0/1966499430 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7da80d1680 con 0x7f7da80b7dd0 2024-04-25T10:02:35.780 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T10:02:35.775+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:02:35.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.776+0000 7f7d8ffff640 1 -- 172.21.15.139:0/1966499430 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7da0058d00 con 0x7f7da80b7dd0 2024-04-25T10:02:35.782 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:02:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.778+0000 7f7dadd5c640 1 -- 172.21.15.139:0/1966499430 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7d9406b4d0 msgr2=0x7f7d9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.778+0000 7f7dadd5c640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7d9406b4d0 0x7f7d9406d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7d98033000 tx=0x7f7d98008030 comp rx=0 tx=0).stop 2024-04-25T10:02:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.778+0000 7f7dadd5c640 1 -- 172.21.15.139:0/1966499430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7da80b7dd0 msgr2=0x7f7da80b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.778+0000 7f7dadd5c640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7da80b7dd0 0x7f7da80b8260 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f7da80d14a0 tx=0x7f7da000e530 comp rx=0 tx=0).stop 2024-04-25T10:02:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.778+0000 7f7dadd5c640 1 -- 172.21.15.139:0/1966499430 shutdown_connections 2024-04-25T10:02:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.778+0000 7f7dadd5c640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7da80b7dd0 0x7f7da80b8260 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.778+0000 7f7dadd5c640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7d9406b4d0 0x7f7d9406d990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.778+0000 7f7dadd5c640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da81607f0 0x7f7da8159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.778+0000 7f7dadd5c640 1 --2- 172.21.15.139:0/1966499430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d900039b0 0x7f7da814be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.778+0000 7f7dadd5c640 1 -- 172.21.15.139:0/1966499430 >> 172.21.15.139:0/1966499430 conn(0x7f7da80b5480 msgr2=0x7f7da80c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.779+0000 7f7dadd5c640 1 -- 172.21.15.139:0/1966499430 shutdown_connections 2024-04-25T10:02:35.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.779+0000 7f7dadd5c640 1 -- 172.21.15.139:0/1966499430 wait complete. 2024-04-25T10:02:35.786 INFO:tasks.ceph.mds.c.smithi139.stderr: -1158> 2024-04-25T10:02:05.043+0000 7f23fb033640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:02:35.786 INFO:tasks.ceph.mds.c.smithi139.stderr: -1155> 2024-04-25T10:02:05.043+0000 7f23fb033640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T10:02:35.786 INFO:tasks.ceph.mds.c.smithi139.stderr: -1120> 2024-04-25T10:02:05.044+0000 7f23fb033640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:02:35.786 INFO:tasks.ceph.mds.c.smithi139.stderr: -1117> 2024-04-25T10:02:05.044+0000 7f23fb033640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T10:02:35.786 INFO:tasks.ceph.mds.c.smithi139.stderr: -1000> 2024-04-25T10:02:05.517+0000 7f23ff03b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:02:35.797 DEBUG:teuthology.orchestra.run.smithi139:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-25T10:02:35.811 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-04-25T10:02:35.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.831+0000 7efd45179280 1 -- start start 2024-04-25T10:02:35.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.831+0000 7efd45179280 1 Processor -- start 2024-04-25T10:02:35.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd45179280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350fe8a000 0x56350f117080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd45179280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350f0a7c00 0x56350f117b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd45179280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350fe8a800 0x56350f117600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd45179280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350fe90480 con 0x56350fe8a800 2024-04-25T10:02:35.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd45179280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350fe90600 con 0x56350fe8a000 2024-04-25T10:02:35.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd45179280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56350fe90780 con 0x56350f0a7c00 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd44128640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350fe8a800 0x56350f117600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd44128640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350fe8a800 0x56350f117600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41348/0 (socket says 172.21.15.139:41348) 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd44128640 1 -- 172.21.15.139:0/4148559434 learned_addr learned my addr 172.21.15.139:0/4148559434 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd43126640 1 --2- 172.21.15.139:0/4148559434 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350f0a7c00 0x56350f117b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd43927640 1 --2- 172.21.15.139:0/4148559434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350fe8a000 0x56350f117080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd44128640 1 -- 172.21.15.139:0/4148559434 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350f0a7c00 msgr2=0x56350f117b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd44128640 1 --2- 172.21.15.139:0/4148559434 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350f0a7c00 0x56350f117b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd44128640 1 -- 172.21.15.139:0/4148559434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350fe8a000 msgr2=0x56350f117080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd44128640 1 --2- 172.21.15.139:0/4148559434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350fe8a000 0x56350f117080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd44128640 1 -- 172.21.15.139:0/4148559434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56350f0dc780 con 0x56350fe8a800 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd44128640 1 --2- 172.21.15.139:0/4148559434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350fe8a800 0x56350f117600 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x56350fec92c0 tx=0x56350f21c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c5d905d7419040 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd41122640 1 -- 172.21.15.139:0/4148559434 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56350f0ccd00 con 0x56350fe8a800 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd41122640 1 -- 172.21.15.139:0/4148559434 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56350f0d2700 con 0x56350fe8a800 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd41122640 1 -- 172.21.15.139:0/4148559434 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56350f0cc680 con 0x56350fe8a800 2024-04-25T10:02:35.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd45179280 1 -- 172.21.15.139:0/4148559434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56350f0dc000 con 0x56350fe8a800 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd45179280 1 -- 172.21.15.139:0/4148559434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56350f0dda40 con 0x56350fe8a800 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.832+0000 7efd45179280 4 waiting for MDS map... 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.833+0000 7efd41122640 1 -- 172.21.15.139:0/4148559434 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x56350ff84800 con 0x56350fe8a800 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.833+0000 7efd41122640 1 -- 172.21.15.139:0/4148559434 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 76) v1 ==== 2723+0+0 (secure 0 0 0) 0x56350f101180 con 0x56350fe8a800 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.833+0000 7efd45179280 4 Got MDS map 76 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.833+0000 7efd45179280 10 main: main 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.833+0000 7efd45179280 4 main: connecting to RADOS... 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.833+0000 7efd45179280 1 Processor -- start 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.833+0000 7efd45179280 1 -- start start 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd45179280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350ff76800 0x56350f116b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd45179280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350ff76c00 0x56350f118100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd45179280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff77000 0x56350f118680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd45179280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350f108180 con 0x56350ff77000 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd45179280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350fe90f00 con 0x56350ff76800 2024-04-25T10:02:35.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd45179280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56350fe91080 con 0x56350ff76c00 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd43126640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff77000 0x56350f118680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd43126640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff77000 0x56350f118680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41352/0 (socket says 172.21.15.139:41352) 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd43126640 1 -- 172.21.15.139:0/350043497 learned_addr learned my addr 172.21.15.139:0/350043497 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd43927640 1 --2- 172.21.15.139:0/350043497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350ff76c00 0x56350f118100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd43126640 1 --2- 172.21.15.139:0/350043497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350ff76800 0x56350f116b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd43126640 1 -- 172.21.15.139:0/350043497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350ff76c00 msgr2=0x56350f118100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd43126640 1 --2- 172.21.15.139:0/350043497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350ff76c00 0x56350f118100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd43126640 1 -- 172.21.15.139:0/350043497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350ff76800 msgr2=0x56350f116b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd43126640 1 --2- 172.21.15.139:0/350043497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350ff76800 0x56350f116b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd43126640 1 -- 172.21.15.139:0/350043497 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56350f0dd860 con 0x56350ff77000 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.834+0000 7efd43126640 1 --2- 172.21.15.139:0/350043497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff77000 0x56350f118680 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x56350ffe20f0 tx=0x56350fe38e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e5d49d923e7040 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd3f11e640 1 -- 172.21.15.139:0/350043497 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56350f0ccb60 con 0x56350ff77000 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd3f11e640 1 -- 172.21.15.139:0/350043497 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56350f0d2fc0 con 0x56350ff77000 2024-04-25T10:02:35.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd3f11e640 1 -- 172.21.15.139:0/350043497 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56350f0ccd00 con 0x56350ff77000 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd45179280 1 -- 172.21.15.139:0/350043497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff77000 msgr2=0x56350f118680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd45179280 1 --2- 172.21.15.139:0/350043497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff77000 0x56350f118680 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x56350ffe20f0 tx=0x56350fe38e40 comp rx=0 tx=0).stop 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd45179280 1 -- 172.21.15.139:0/350043497 shutdown_connections 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd45179280 1 --2- 172.21.15.139:0/350043497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff77000 0x56350f118680 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd45179280 1 --2- 172.21.15.139:0/350043497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350ff76c00 0x56350f118100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd45179280 1 --2- 172.21.15.139:0/350043497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350ff76800 0x56350f116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd45179280 1 -- 172.21.15.139:0/350043497 >> 172.21.15.139:0/350043497 conn(0x56350fe8a400 msgr2=0x56350f1a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd45179280 1 -- 172.21.15.139:0/350043497 shutdown_connections 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd45179280 1 -- 172.21.15.139:0/350043497 wait complete. 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.835+0000 7efd45179280 1 Processor -- start 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd45179280 1 -- start start 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd45179280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff76800 0x56350f116b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd45179280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350ff76c00 0x56350f118100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd45179280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350ff77000 0x56350f118680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd45179280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350fe91380 con 0x56350ff76800 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd45179280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350fe91200 con 0x56350ff77000 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd45179280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56350fe91500 con 0x56350ff76c00 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd43126640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff76800 0x56350f116b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd43126640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff76800 0x56350f116b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41362/0 (socket says 172.21.15.139:41362) 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd43126640 1 -- 172.21.15.139:0/3912179428 learned_addr learned my addr 172.21.15.139:0/3912179428 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd43126640 1 -- 172.21.15.139:0/3912179428 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350ff76c00 msgr2=0x56350f118100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd43126640 1 --2- 172.21.15.139:0/3912179428 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350ff76c00 0x56350f118100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd43126640 1 -- 172.21.15.139:0/3912179428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350ff77000 msgr2=0x56350f118680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd43126640 1 --2- 172.21.15.139:0/3912179428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350ff77000 0x56350f118680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.836+0000 7efd43126640 1 -- 172.21.15.139:0/3912179428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56350f0dc780 con 0x56350ff76800 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.837+0000 7efd43126640 1 --2- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff76800 0x56350f116b00 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x56350ffe33b0 tx=0x56350fe2b1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.837+0000 7efd3d91b640 1 -- 172.21.15.139:0/3912179428 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56350f0cc680 con 0x56350ff76800 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.837+0000 7efd3d91b640 1 -- 172.21.15.139:0/3912179428 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56350f0d2e00 con 0x56350ff76800 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.837+0000 7efd3d91b640 1 -- 172.21.15.139:0/3912179428 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56350f0cc4e0 con 0x56350ff76800 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.837+0000 7efd45179280 1 -- 172.21.15.139:0/3912179428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56350f0dc000 con 0x56350ff76800 2024-04-25T10:02:35.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.837+0000 7efd45179280 1 -- 172.21.15.139:0/3912179428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56350f0dd680 con 0x56350ff76800 2024-04-25T10:02:35.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.837+0000 7efd45179280 4 main: resolving pool 15 2024-04-25T10:02:35.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.838+0000 7efd3d91b640 1 -- 172.21.15.139:0/3912179428 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x56350fea4c00 con 0x56350ff76800 2024-04-25T10:02:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.838+0000 7efd3d91b640 1 --2- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x56350ff77c00 0x56350f119700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.838+0000 7efd3d91b640 1 -- 172.21.15.139:0/3912179428 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x56350fe4b600 con 0x56350ff76800 2024-04-25T10:02:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.838+0000 7efd45179280 4 main: creating IoCtx.. 2024-04-25T10:02:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.838+0000 7efd43927640 1 --2- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x56350ff77c00 0x56350f119700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.838+0000 7efd45179280 1 --2- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x56350ff77400 0x56350f118c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.838+0000 7efd45179280 1 -- 172.21.15.139:0/3912179428 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v9 -- 0x56350ff77800 con 0x56350ff77400 2024-04-25T10:02:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.838+0000 7efd43126640 1 --2- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x56350ff77400 0x56350f118c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.839+0000 7efd43927640 1 --2- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x56350ff77c00 0x56350f119700 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x56350f194a50 tx=0x56350ff93f80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.839+0000 7efd43126640 1 --2- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x56350ff77400 0x56350f118c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd43126640 1 -- 172.21.15.139:0/3912179428 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x56350fe26000 con 0x56350ff77400 2024-04-25T10:02:35.851 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:02:35.851 INFO:teuthology.orchestra.run.smithi139.stdout: "0": { 2024-04-25T10:02:35.851 INFO:teuthology.orchestra.run.smithi139.stdout: "data": { 2024-04-25T10:02:35.851 INFO:teuthology.orchestra.run.smithi139.stdout: "version": 5, 2024-04-25T10:02:35.851 INFO:teuthology.orchestra.run.smithi139.stdout: "inotable": { 2024-04-25T10:02:35.851 INFO:teuthology.orchestra.run.smithi139.stdout: "projected_free": [ 2024-04-25T10:02:35.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:35.851 INFO:teuthology.orchestra.run.smithi139.stdout: "start": 1099511627779, 2024-04-25T10:02:35.851 INFO:teuthology.orchestra.run.smithi139.stdout: "len": 1099511627773 2024-04-25T10:02:35.851 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:35.851 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-04-25T10:02:35.852 INFO:teuthology.orchestra.run.smithi139.stdout: "free": [ 2024-04-25T10:02:35.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:35.852 INFO:teuthology.orchestra.run.smithi139.stdout: "start": 1099511627779, 2024-04-25T10:02:35.852 INFO:teuthology.orchestra.run.smithi139.stdout: "len": 1099511627773 2024-04-25T10:02:35.852 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:35.852 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-04-25T10:02:35.852 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:35.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:35.852 INFO:teuthology.orchestra.run.smithi139.stdout: "result": 0 2024-04-25T10:02:35.852 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:35.852 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:02:35.852 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:35.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 -- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x56350ff77400 msgr2=0x56350f118c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 --2- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x56350ff77400 0x56350f118c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 -- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x56350ff77c00 msgr2=0x56350f119700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 --2- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x56350ff77c00 0x56350f119700 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x56350f194a50 tx=0x56350ff93f80 comp rx=0 tx=0).stop 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 -- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff76800 msgr2=0x56350f116b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 --2- 172.21.15.139:0/3912179428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350ff76800 0x56350f116b00 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x56350ffe33b0 tx=0x56350fe2b1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd43927640 1 -- 172.21.15.139:0/3912179428 reap_dead start 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 -- 172.21.15.139:0/3912179428 shutdown_connections 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 -- 172.21.15.139:0/3912179428 >> 172.21.15.139:0/3912179428 conn(0x56350fe8a400 msgr2=0x56350f1a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 -- 172.21.15.139:0/3912179428 shutdown_connections 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 -- 172.21.15.139:0/3912179428 wait complete. 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 -- 172.21.15.139:0/4148559434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350fe8a800 msgr2=0x56350f117600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 --2- 172.21.15.139:0/4148559434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350fe8a800 0x56350f117600 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x56350fec92c0 tx=0x56350f21c000 comp rx=0 tx=0).stop 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 -- 172.21.15.139:0/4148559434 shutdown_connections 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 --2- 172.21.15.139:0/4148559434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56350fe8a800 0x56350f117600 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 --2- 172.21.15.139:0/4148559434 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56350f0a7c00 0x56350f117b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 --2- 172.21.15.139:0/4148559434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56350fe8a000 0x56350f117080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 -- 172.21.15.139:0/4148559434 >> 172.21.15.139:0/4148559434 conn(0x56350f0a7000 msgr2=0x56350f1a1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 -- 172.21.15.139:0/4148559434 shutdown_connections 2024-04-25T10:02:35.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:35.841+0000 7efd45179280 1 -- 172.21.15.139:0/4148559434 wait complete. 2024-04-25T10:02:35.856 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.059067, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-25T10:02:35.856 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:02:35.856 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:35.945 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:35.987 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:02:35.987 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:02:35.987 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:36.005 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:02:36.005 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T10:02:36.005 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:36.045 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:36.045 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:02:36.045 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:36.061 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:02:36.061 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T10:02:36.061 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:36.102 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:36.102 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:36.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.195+0000 7f200c389640 1 Processor -- start 2024-04-25T10:02:36.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.195+0000 7f200c389640 1 -- start start 2024-04-25T10:02:36.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f200c389640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f200415c9c0 0x7f20040ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f200c389640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f20040ad570 0x7f200406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f200c389640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20040af590 con 0x7f200406c1f0 2024-04-25T10:02:36.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f200c389640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20040af870 con 0x7f20040ad570 2024-04-25T10:02:36.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f200c389640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200415c530 con 0x7f200415c9c0 2024-04-25T10:02:36.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f200a125640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f200406c1f0 0x7f200415fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53292/0 (socket says 172.21.15.139:53292) 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f200a125640 1 -- 172.21.15.139:0/1562020707 learned_addr learned my addr 172.21.15.139:0/1562020707 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915578371 0 0) 0x7f20040af590 con 0x7f200406c1f0 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ff0003520 con 0x7f200406c1f0 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009924640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f200415c9c0 0x7f20040ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f200a926640 1 --2- 172.21.15.139:0/1562020707 >> v2:172.21.15.150:3300/0 conn(0x7f20040ad570 0x7f200406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3257642971 0 0) 0x7f1ff0003520 con 0x7f200406c1f0 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f200415c9c0 msgr2=0x7f20040ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009123640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f200415c9c0 0x7f20040ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 >> v2:172.21.15.150:3300/0 conn(0x7f20040ad570 msgr2=0x7f200406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009123640 1 --2- 172.21.15.139:0/1562020707 >> v2:172.21.15.150:3300/0 conn(0x7f20040ad570 0x7f200406ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200415bee0 con 0x7f200406c1f0 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f2000001670 con 0x7f200406c1f0 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f200a926640 1 --2- 172.21.15.139:0/1562020707 >> v2:172.21.15.150:3300/0 conn(0x7f20040ad570 0x7f200406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:36.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009924640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f200415c9c0 0x7f20040ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:36.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 >> v1:172.21.15.139:6789/0 conn(0x7f200406c1f0 legacy=0x7f200415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009123640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ff00039b0 0x7f1ff0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.196+0000 7f2009123640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ff0004800 0x7f1ff0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f2009123640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f200415c9c0 0x7f1ff00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20040af870 con 0x7f1ff00039b0 2024-04-25T10:02:36.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200415c530 con 0x7f1ff0004800 2024-04-25T10:02:36.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff000baf0 con 0x7f200415c9c0 2024-04-25T10:02:36.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f200a125640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ff00039b0 0x7f1ff0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f200a926640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f200415c9c0 0x7f1ff00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f2009924640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ff0004800 0x7f1ff0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f200a125640 1 -- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f200415c9c0 msgr2=0x7f1ff00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f200a125640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f200415c9c0 0x7f1ff00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f200a125640 1 -- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ff0004800 msgr2=0x7f1ff0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f200a125640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ff0004800 0x7f1ff0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f200a125640 1 -- 172.21.15.139:0/1562020707 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff0003520 con 0x7f1ff00039b0 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f200a926640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f200415c9c0 0x7f1ff00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.197+0000 7f2009924640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ff0004800 0x7f1ff0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.198+0000 7f200a125640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ff00039b0 0x7f1ff0003dd0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f200000c210 tx=0x7f1ff0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4d76cf5231ebc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.198+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000007d30 con 0x7f1ff00039b0 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.198+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20000058d0 con 0x7f1ff00039b0 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.198+0000 7f2009123640 1 -- 172.21.15.139:0/1562020707 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000005ce0 con 0x7f1ff00039b0 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.198+0000 7f200c389640 1 -- 172.21.15.139:0/1562020707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ff00039b0 msgr2=0x7f1ff0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.198+0000 7f200c389640 1 --2- 172.21.15.139:0/1562020707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ff00039b0 0x7f1ff0003dd0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f200000c210 tx=0x7f1ff0005830 comp rx=0 tx=0).stop 2024-04-25T10:02:36.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.198+0000 7f200a926640 1 -- 172.21.15.139:0/1562020707 reap_dead start 2024-04-25T10:02:36.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.198+0000 7f200c389640 1 -- 172.21.15.139:0/1562020707 shutdown_connections 2024-04-25T10:02:36.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.198+0000 7f200c389640 1 -- 172.21.15.139:0/1562020707 >> 172.21.15.139:0/1562020707 conn(0x7f20040b23d0 msgr2=0x7f20040cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.198+0000 7f200c389640 1 -- 172.21.15.139:0/1562020707 shutdown_connections 2024-04-25T10:02:36.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.198+0000 7f200c389640 1 -- 172.21.15.139:0/1562020707 wait complete. 2024-04-25T10:02:36.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.199+0000 7f200c389640 1 Processor -- start 2024-04-25T10:02:36.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.199+0000 7f200c389640 1 -- start start 2024-04-25T10:02:36.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.200+0000 7f200c389640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ff00039b0 0x7f200413b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.200+0000 7f200c389640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f200414d580 0x7f2004152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.200+0000 7f200c389640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2004152cb0 0x7f20041533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.200+0000 7f200c389640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200406f7b0 con 0x7f200414d580 2024-04-25T10:02:36.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.200+0000 7f200c389640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200406f630 con 0x7f1ff00039b0 2024-04-25T10:02:36.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.200+0000 7f200c389640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20040af160 con 0x7f2004152cb0 2024-04-25T10:02:36.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.200+0000 7f2009924640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f200414d580 0x7f2004152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.200+0000 7f2009924640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f200414d580 0x7f2004152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41380/0 (socket says 172.21.15.139:41380) 2024-04-25T10:02:36.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.200+0000 7f2009924640 1 -- 172.21.15.139:0/4270386785 learned_addr learned my addr 172.21.15.139:0/4270386785 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:36.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.200+0000 7f200a125640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ff00039b0 0x7f200413b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.200+0000 7f200a926640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2004152cb0 0x7f20041533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.201+0000 7f2009924640 1 -- 172.21.15.139:0/4270386785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2004152cb0 msgr2=0x7f20041533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.201+0000 7f2009924640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2004152cb0 0x7f20041533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.201+0000 7f2009924640 1 -- 172.21.15.139:0/4270386785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ff00039b0 msgr2=0x7f200413b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.201+0000 7f2009924640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ff00039b0 0x7f200413b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.201+0000 7f2009924640 1 -- 172.21.15.139:0/4270386785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200413e2a0 con 0x7f200414d580 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.201+0000 7f2009924640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f200414d580 0x7f2004152770 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4010890 tx=0x7f1ff4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.201+0000 7f1ffaffd640 1 -- 172.21.15.139:0/4270386785 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff401a020 con 0x7f200414d580 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.201+0000 7f1ffaffd640 1 -- 172.21.15.139:0/4270386785 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff400eb00 con 0x7f200414d580 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.201+0000 7f1ffaffd640 1 -- 172.21.15.139:0/4270386785 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400f040 con 0x7f200414d580 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.201+0000 7f200c389640 1 -- 172.21.15.139:0/4270386785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20040cc760 con 0x7f200414d580 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.201+0000 7f200c389640 1 -- 172.21.15.139:0/4270386785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200413b9d0 con 0x7f200414d580 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.202+0000 7f200c389640 1 -- 172.21.15.139:0/4270386785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20040cc550 con 0x7f200414d580 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.202+0000 7f1ffaffd640 1 -- 172.21.15.139:0/4270386785 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ff401d020 con 0x7f200414d580 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.203+0000 7f1ffaffd640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1fe806b4d0 0x7f1fe806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.203+0000 7f200a125640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1fe806b4d0 0x7f1fe806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.203+0000 7f1ffaffd640 1 -- 172.21.15.139:0/4270386785 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f1ff408ab40 con 0x7f200414d580 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.203+0000 7f200a125640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1fe806b4d0 0x7f1fe806d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2000033000 tx=0x7f2000009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.208+0000 7f1ffaffd640 1 -- 172.21.15.139:0/4270386785 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ff4024020 con 0x7f200414d580 2024-04-25T10:02:36.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.411+0000 7f200c389640 1 -- 172.21.15.139:0/4270386785 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2004074520 con 0x7f200414d580 2024-04-25T10:02:36.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.411+0000 7f1ffaffd640 1 -- 172.21.15.139:0/4270386785 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4152 (secure 0 0 0) 0x7f1ff4059100 con 0x7f200414d580 2024-04-25T10:02:36.418 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:36.418 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":76,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":76,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:15.497940+0000","modified":"2024-04-25T10:02:35.776771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:02:36.419 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 76 2024-04-25T10:02:36.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.413+0000 7f200c389640 1 -- 172.21.15.139:0/4270386785 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1fe806b4d0 msgr2=0x7f1fe806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.413+0000 7f200c389640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1fe806b4d0 0x7f1fe806d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2000033000 tx=0x7f2000009030 comp rx=0 tx=0).stop 2024-04-25T10:02:36.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.413+0000 7f200c389640 1 -- 172.21.15.139:0/4270386785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f200414d580 msgr2=0x7f2004152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.413+0000 7f200c389640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f200414d580 0x7f2004152770 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4010890 tx=0x7f1ff4010d50 comp rx=0 tx=0).stop 2024-04-25T10:02:36.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.413+0000 7f200c389640 1 -- 172.21.15.139:0/4270386785 shutdown_connections 2024-04-25T10:02:36.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.413+0000 7f200c389640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f200414d580 0x7f2004152770 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.413+0000 7f200c389640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1fe806b4d0 0x7f1fe806d990 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.413+0000 7f200c389640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2004152cb0 0x7f20041533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.413+0000 7f200c389640 1 --2- 172.21.15.139:0/4270386785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ff00039b0 0x7f200413b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.413+0000 7f200c389640 1 -- 172.21.15.139:0/4270386785 >> 172.21.15.139:0/4270386785 conn(0x7f20040b23d0 msgr2=0x7f20040b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.413+0000 7f200c389640 1 -- 172.21.15.139:0/4270386785 shutdown_connections 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.413+0000 7f200c389640 1 -- 172.21.15.139:0/4270386785 wait complete. 2024-04-25T10:02:36.430 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:36.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.522+0000 7f568d638640 1 Processor -- start 2024-04-25T10:02:36.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.522+0000 7f568d638640 1 -- start start 2024-04-25T10:02:36.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f568d638640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f56880b6af0 0x7f568806c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f568d638640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f568806c5b0 0x7f56880d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f568d638640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56880cf110 con 0x7f56881611e0 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f568d638640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880d1fa0 con 0x7f568806c5b0 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f568d638640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880d2560 con 0x7f56880b6af0 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f56877fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f568806c5b0 0x7f56880d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f5686ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f56881611e0 0x7f56880cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37266/0 (socket says 172.21.15.139:37266) 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f5686ffd640 1 -- 172.21.15.139:0/2179292568 learned_addr learned my addr 172.21.15.139:0/2179292568 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f56867fc640 1 --2- 172.21.15.139:0/2179292568 >> v2:172.21.15.150:3300/0 conn(0x7f56880b6af0 0x7f568806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f56877fe640 1 -- 172.21.15.139:0/2179292568 >> v2:172.21.15.150:3300/0 conn(0x7f56880b6af0 msgr2=0x7f568806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f56877fe640 1 --2- 172.21.15.139:0/2179292568 >> v2:172.21.15.150:3300/0 conn(0x7f56880b6af0 0x7f568806c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f56877fe640 1 -- 172.21.15.139:0/2179292568 >> v1:172.21.15.139:6789/0 conn(0x7f56881611e0 legacy=0x7f56880cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f56877fe640 1 -- 172.21.15.139:0/2179292568 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880b6f50 con 0x7f568806c5b0 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.523+0000 7f5685ffb640 1 -- 172.21.15.139:0/2179292568 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337360350 0 0) 0x7f56880cf110 con 0x7f56881611e0 2024-04-25T10:02:36.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.524+0000 7f56877fe640 1 --2- 172.21.15.139:0/2179292568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f568806c5b0 0x7f56880d29f0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f56880b3410 tx=0x7f568002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1438c4e26529275e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.524+0000 7f5685ffb640 1 -- 172.21.15.139:0/2179292568 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680007740 con 0x7f568806c5b0 2024-04-25T10:02:36.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.524+0000 7f5685ffb640 1 -- 172.21.15.139:0/2179292568 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56800078e0 con 0x7f568806c5b0 2024-04-25T10:02:36.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.524+0000 7f5685ffb640 1 -- 172.21.15.139:0/2179292568 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680007cb0 con 0x7f568806c5b0 2024-04-25T10:02:36.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.525+0000 7f568d638640 1 -- 172.21.15.139:0/2179292568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f568806c5b0 msgr2=0x7f56880d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.525+0000 7f568d638640 1 --2- 172.21.15.139:0/2179292568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f568806c5b0 0x7f56880d29f0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f56880b3410 tx=0x7f568002a680 comp rx=0 tx=0).stop 2024-04-25T10:02:36.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.525+0000 7f568d638640 1 -- 172.21.15.139:0/2179292568 shutdown_connections 2024-04-25T10:02:36.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.525+0000 7f568d638640 1 --2- 172.21.15.139:0/2179292568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f568806c5b0 0x7f56880d29f0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.525+0000 7f568d638640 1 --2- 172.21.15.139:0/2179292568 >> v2:172.21.15.150:3300/0 conn(0x7f56880b6af0 0x7f568806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.525+0000 7f568d638640 1 -- 172.21.15.139:0/2179292568 >> 172.21.15.139:0/2179292568 conn(0x7f56880b5b00 msgr2=0x7f56880d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.525+0000 7f568d638640 1 -- 172.21.15.139:0/2179292568 shutdown_connections 2024-04-25T10:02:36.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.525+0000 7f568d638640 1 -- 172.21.15.139:0/2179292568 wait complete. 2024-04-25T10:02:36.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.526+0000 7f568d638640 1 Processor -- start 2024-04-25T10:02:36.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.526+0000 7f568d638640 1 -- start start 2024-04-25T10:02:36.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f568d638640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f568806c5b0 0x7f56880c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f568d638640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56880b6af0 0x7f56881581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f568d638640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56881611e0 0x7f5688147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f568d638640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880b2a50 con 0x7f56881611e0 2024-04-25T10:02:36.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f568d638640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880b2770 con 0x7f568806c5b0 2024-04-25T10:02:36.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f568d638640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56880b2d30 con 0x7f56880b6af0 2024-04-25T10:02:36.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f56877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56881611e0 0x7f5688147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f56877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56881611e0 0x7f5688147910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42240/0 (socket says 172.21.15.139:42240) 2024-04-25T10:02:36.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f56877fe640 1 -- 172.21.15.139:0/1003410514 learned_addr learned my addr 172.21.15.139:0/1003410514 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:36.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f5686ffd640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f568806c5b0 0x7f56880c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f56867fc640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56880b6af0 0x7f56881581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f56877fe640 1 -- 172.21.15.139:0/1003410514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56880b6af0 msgr2=0x7f56881581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f56877fe640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56880b6af0 0x7f56881581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f56877fe640 1 -- 172.21.15.139:0/1003410514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f568806c5b0 msgr2=0x7f56880c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f56877fe640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f568806c5b0 0x7f56880c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f56877fe640 1 -- 172.21.15.139:0/1003410514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880c83a0 con 0x7f56881611e0 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f5686ffd640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f568806c5b0 0x7f56880c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f56867fc640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56880b6af0 0x7f56881581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.527+0000 7f56877fe640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56881611e0 0x7f5688147910 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f568002ac70 tx=0x7f5680004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.528+0000 7f566f7fe640 1 -- 172.21.15.139:0/1003410514 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680040070 con 0x7f56881611e0 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.528+0000 7f566f7fe640 1 -- 172.21.15.139:0/1003410514 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5680031c40 con 0x7f56881611e0 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.528+0000 7f566f7fe640 1 -- 172.21.15.139:0/1003410514 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680009450 con 0x7f56881611e0 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.528+0000 7f568d638640 1 -- 172.21.15.139:0/1003410514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56880c8680 con 0x7f56881611e0 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.528+0000 7f568d638640 1 -- 172.21.15.139:0/1003410514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56880c8c40 con 0x7f56881611e0 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.529+0000 7f568d638640 1 -- 172.21.15.139:0/1003410514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5688074520 con 0x7f56881611e0 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.529+0000 7f566f7fe640 1 -- 172.21.15.139:0/1003410514 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5680039040 con 0x7f56881611e0 2024-04-25T10:02:36.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.529+0000 7f566f7fe640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f567406b4d0 0x7f567406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.530+0000 7f566f7fe640 1 -- 172.21.15.139:0/1003410514 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f56800ab380 con 0x7f56881611e0 2024-04-25T10:02:36.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.530+0000 7f5686ffd640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f567406b4d0 0x7f567406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.530+0000 7f5686ffd640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f567406b4d0 0x7f567406d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5678000f60 tx=0x7f5678007ca0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.534+0000 7f566f7fe640 1 -- 172.21.15.139:0/1003410514 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5680079f90 con 0x7f56881611e0 2024-04-25T10:02:36.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.703+0000 7f568d638640 1 -- 172.21.15.139:0/1003410514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f568806ceb0 con 0x7f56881611e0 2024-04-25T10:02:36.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.704+0000 7f566f7fe640 1 -- 172.21.15.139:0/1003410514 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17291 (secure 0 0 0) 0x7f5680079890 con 0x7f56881611e0 2024-04-25T10:02:36.710 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:36.710 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":109,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:02:35.776788+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:13.695577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:14.460042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2a","mappings":[{"from":1,"to":7}]},{"pgid":"16.2f","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6835/359626718":"2024-04-26T10:02:35.776758+0000","172.21.15.139:6837/359626718":"2024-04-26T10:02:35.776758+0000","172.21.15.139:6837/4074693991":"2024-04-26T10:01:59.507353+0000","172.21.15.139:6835/4074693991":"2024-04-26T10:01:59.507353+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:36.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.707+0000 7f568d638640 1 -- 172.21.15.139:0/1003410514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f567406b4d0 msgr2=0x7f567406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.707+0000 7f568d638640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f567406b4d0 0x7f567406d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5678000f60 tx=0x7f5678007ca0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.707+0000 7f568d638640 1 -- 172.21.15.139:0/1003410514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56881611e0 msgr2=0x7f5688147910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.707+0000 7f568d638640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56881611e0 0x7f5688147910 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f568002ac70 tx=0x7f5680004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:36.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.707+0000 7f568d638640 1 -- 172.21.15.139:0/1003410514 shutdown_connections 2024-04-25T10:02:36.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.707+0000 7f568d638640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56881611e0 0x7f5688147910 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.707+0000 7f568d638640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f567406b4d0 0x7f567406d990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.707+0000 7f568d638640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56880b6af0 0x7f56881581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.707+0000 7f568d638640 1 --2- 172.21.15.139:0/1003410514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f568806c5b0 0x7f56880c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.707+0000 7f568d638640 1 -- 172.21.15.139:0/1003410514 >> 172.21.15.139:0/1003410514 conn(0x7f56880b5b00 msgr2=0x7f568815c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.707+0000 7f568d638640 1 -- 172.21.15.139:0/1003410514 shutdown_connections 2024-04-25T10:02:36.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.707+0000 7f568d638640 1 -- 172.21.15.139:0/1003410514 wait complete. 2024-04-25T10:02:36.724 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:13.695577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:14.460042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:36.724 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:36.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.819+0000 7fe3b8bb7640 1 Processor -- start 2024-04-25T10:02:36.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.819+0000 7fe3b8bb7640 1 -- start start 2024-04-25T10:02:36.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.820+0000 7fe3b8bb7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3b4144090 0x7fe3b40b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.820+0000 7fe3b8bb7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe3b40cc9e0 0x7fe3b406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.820+0000 7fe3b8bb7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b40b2a50 con 0x7fe3b40cce20 2024-04-25T10:02:36.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.820+0000 7fe3b8bb7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b415b090 con 0x7fe3b40cc9e0 2024-04-25T10:02:36.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.820+0000 7fe3b8bb7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b415b370 con 0x7fe3b4144090 2024-04-25T10:02:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.821+0000 7fe3b259c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3b4144090 0x7fe3b40b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.821+0000 7fe3b1d9b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe3b40cc9e0 0x7fe3b406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.821+0000 7fe3b1d9b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe3b40cc9e0 0x7fe3b406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:48674/0 (socket says 172.21.15.139:48674) 2024-04-25T10:02:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.821+0000 7fe3b259c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3b4144090 0x7fe3b40b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:59654/0 (socket says 172.21.15.139:59654) 2024-04-25T10:02:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.821+0000 7fe3b1d9b640 1 -- 172.21.15.139:0/3960338788 learned_addr learned my addr 172.21.15.139:0/3960338788 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.821+0000 7fe3b259c640 1 -- 172.21.15.139:0/3960338788 >> v2:172.21.15.150:3300/0 conn(0x7fe3b40cc9e0 msgr2=0x7fe3b406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.821+0000 7fe3b259c640 1 --2- 172.21.15.139:0/3960338788 >> v2:172.21.15.150:3300/0 conn(0x7fe3b40cc9e0 0x7fe3b406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.821+0000 7fe3b259c640 1 -- 172.21.15.139:0/3960338788 >> v1:172.21.15.150:6789/0 conn(0x7fe3b40cce20 legacy=0x7fe3b415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.821+0000 7fe3b259c640 1 -- 172.21.15.139:0/3960338788 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b415b650 con 0x7fe3b4144090 2024-04-25T10:02:36.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.821+0000 7fe3b1d9b640 1 --2- 172.21.15.139:0/3960338788 >> v2:172.21.15.150:3300/0 conn(0x7fe3b40cc9e0 0x7fe3b406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.822+0000 7fe3b259c640 1 --2- 172.21.15.139:0/3960338788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3b4144090 0x7fe3b40b31a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80097a0 tx=0x7fe3a802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9fbd0437e608028 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.822+0000 7fe3b159a640 1 -- 172.21.15.139:0/3960338788 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8037070 con 0x7fe3b4144090 2024-04-25T10:02:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.822+0000 7fe3b159a640 1 -- 172.21.15.139:0/3960338788 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a802ae50 con 0x7fe3b4144090 2024-04-25T10:02:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.822+0000 7fe3b159a640 1 -- 172.21.15.139:0/3960338788 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8032660 con 0x7fe3b4144090 2024-04-25T10:02:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.822+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/3960338788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3b4144090 msgr2=0x7fe3b40b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.822+0000 7fe3b8bb7640 1 --2- 172.21.15.139:0/3960338788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3b4144090 0x7fe3b40b31a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80097a0 tx=0x7fe3a802a100 comp rx=0 tx=0).stop 2024-04-25T10:02:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.822+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/3960338788 shutdown_connections 2024-04-25T10:02:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.822+0000 7fe3b8bb7640 1 --2- 172.21.15.139:0/3960338788 >> v2:172.21.15.150:3300/0 conn(0x7fe3b40cc9e0 0x7fe3b406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.822+0000 7fe3b8bb7640 1 --2- 172.21.15.139:0/3960338788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3b4144090 0x7fe3b40b31a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.822+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/3960338788 >> 172.21.15.139:0/3960338788 conn(0x7fe3b4067000 msgr2=0x7fe3b406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.822+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/3960338788 shutdown_connections 2024-04-25T10:02:36.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.823+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/3960338788 wait complete. 2024-04-25T10:02:36.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.824+0000 7fe3b8bb7640 1 Processor -- start 2024-04-25T10:02:36.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.824+0000 7fe3b8bb7640 1 -- start start 2024-04-25T10:02:36.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.824+0000 7fe3b8bb7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3b40cc9e0 0x7fe3b40ae390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.824+0000 7fe3b8bb7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3b40cce20 0x7fe3b4132e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.824+0000 7fe3b8bb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3b4144090 0x7fe3b41333a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.824+0000 7fe3b8bb7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b4144470 con 0x7fe3b4144090 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.824+0000 7fe3b8bb7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b415bc10 con 0x7fe3b40cce20 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.824+0000 7fe3b8bb7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b415ad80 con 0x7fe3b40cc9e0 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b2d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3b4144090 0x7fe3b41333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b2d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3b4144090 0x7fe3b41333a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42252/0 (socket says 172.21.15.139:42252) 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b2d9d640 1 -- 172.21.15.139:0/4049737777 learned_addr learned my addr 172.21.15.139:0/4049737777 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b259c640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3b40cc9e0 0x7fe3b40ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b1d9b640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3b40cce20 0x7fe3b4132e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b2d9d640 1 -- 172.21.15.139:0/4049737777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3b40cc9e0 msgr2=0x7fe3b40ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b2d9d640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3b40cc9e0 0x7fe3b40ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b2d9d640 1 -- 172.21.15.139:0/4049737777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3b40cce20 msgr2=0x7fe3b4132e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b2d9d640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3b40cce20 0x7fe3b4132e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b2d9d640 1 -- 172.21.15.139:0/4049737777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4130cf0 con 0x7fe3b4144090 2024-04-25T10:02:36.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b2d9d640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3b4144090 0x7fe3b41333a0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c6b0 tx=0x7fe3ac00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3a2ffd640 1 -- 172.21.15.139:0/4049737777 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac003b70 con 0x7fe3b4144090 2024-04-25T10:02:36.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3a2ffd640 1 -- 172.21.15.139:0/4049737777 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac003d30 con 0x7fe3b4144090 2024-04-25T10:02:36.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3a2ffd640 1 -- 172.21.15.139:0/4049737777 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac005310 con 0x7fe3b4144090 2024-04-25T10:02:36.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/4049737777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b41338e0 con 0x7fe3b4144090 2024-04-25T10:02:36.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.825+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/4049737777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b4130f90 con 0x7fe3b4144090 2024-04-25T10:02:36.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.826+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/4049737777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b4074520 con 0x7fe3b4144090 2024-04-25T10:02:36.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.827+0000 7fe3a2ffd640 1 -- 172.21.15.139:0/4049737777 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3ac0054b0 con 0x7fe3b4144090 2024-04-25T10:02:36.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.827+0000 7fe3a2ffd640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe39406b500 0x7fe39406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.827+0000 7fe3a2ffd640 1 -- 172.21.15.139:0/4049737777 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fe3ac08a280 con 0x7fe3b4144090 2024-04-25T10:02:36.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.827+0000 7fe3b259c640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe39406b500 0x7fe39406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.828+0000 7fe3b259c640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe39406b500 0x7fe39406d9c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8034000 tx=0x7fe3a802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:36.831+0000 7fe3a2ffd640 1 -- 172.21.15.139:0/4049737777 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3b40b2a50 con 0x7fe3b4144090 2024-04-25T10:02:37.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.002+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/4049737777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3b4073990 con 0x7fe3b4144090 2024-04-25T10:02:37.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.002+0000 7fe3a2ffd640 1 -- 172.21.15.139:0/4049737777 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3ac058ea0 con 0x7fe3b4144090 2024-04-25T10:02:37.008 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:02:37.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.004+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/4049737777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe39406b500 msgr2=0x7fe39406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.004+0000 7fe3b8bb7640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe39406b500 0x7fe39406d9c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8034000 tx=0x7fe3a802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:37.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.004+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/4049737777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3b4144090 msgr2=0x7fe3b41333a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.004+0000 7fe3b8bb7640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3b4144090 0x7fe3b41333a0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c6b0 tx=0x7fe3ac00cb70 comp rx=0 tx=0).stop 2024-04-25T10:02:37.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.004+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/4049737777 shutdown_connections 2024-04-25T10:02:37.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.004+0000 7fe3b8bb7640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3b4144090 0x7fe3b41333a0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.004+0000 7fe3b8bb7640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe39406b500 0x7fe39406d9c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.004+0000 7fe3b8bb7640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3b40cce20 0x7fe3b4132e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.004+0000 7fe3b8bb7640 1 --2- 172.21.15.139:0/4049737777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3b40cc9e0 0x7fe3b40ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.004+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/4049737777 >> 172.21.15.139:0/4049737777 conn(0x7fe3b4067000 msgr2=0x7fe3b406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.004+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/4049737777 shutdown_connections 2024-04-25T10:02:37.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.004+0000 7fe3b8bb7640 1 -- 172.21.15.139:0/4049737777 wait complete. 2024-04-25T10:02:37.020 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:37.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.115+0000 7fcb3a81f640 1 Processor -- start 2024-04-25T10:02:37.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.115+0000 7fcb3a81f640 1 -- start start 2024-04-25T10:02:37.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb3a81f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb3415dc90 0x7fcb340ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb3a81f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcb3406f300 0x7fcb340b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb3a81f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb340cf980 con 0x7fcb340cf3e0 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb3a81f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3406ab60 con 0x7fcb3406f300 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb3a81f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3406ae40 con 0x7fcb3415dc90 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb38dbc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcb3406f300 0x7fcb340b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb38dbc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcb3406f300 0x7fcb340b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42256/0 (socket says 172.21.15.139:42256) 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb38dbc640 1 -- 172.21.15.139:0/922246213 learned_addr learned my addr 172.21.15.139:0/922246213 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb33fff640 1 --2- 172.21.15.139:0/922246213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb3415dc90 0x7fcb340ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb38dbc640 1 -- 172.21.15.139:0/922246213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb3415dc90 msgr2=0x7fcb340ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb38dbc640 1 --2- 172.21.15.139:0/922246213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb3415dc90 0x7fcb340ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb38dbc640 1 -- 172.21.15.139:0/922246213 >> v1:172.21.15.150:6789/0 conn(0x7fcb340cf3e0 legacy=0x7fcb3406ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.116+0000 7fcb38dbc640 1 -- 172.21.15.139:0/922246213 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3406b120 con 0x7fcb3406f300 2024-04-25T10:02:37.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.117+0000 7fcb38dbc640 1 --2- 172.21.15.139:0/922246213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb3406f300 0x7fcb340b2db0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009e50 tx=0x7fcb2c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc7f318c133db8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.117+0000 7fcb32ffd640 1 -- 172.21.15.139:0/922246213 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c036070 con 0x7fcb3406f300 2024-04-25T10:02:37.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.117+0000 7fcb32ffd640 1 -- 172.21.15.139:0/922246213 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2c031600 con 0x7fcb3406f300 2024-04-25T10:02:37.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.117+0000 7fcb32ffd640 1 -- 172.21.15.139:0/922246213 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c0319d0 con 0x7fcb3406f300 2024-04-25T10:02:37.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.118+0000 7fcb3a81f640 1 -- 172.21.15.139:0/922246213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb3406f300 msgr2=0x7fcb340b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.118+0000 7fcb3a81f640 1 --2- 172.21.15.139:0/922246213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb3406f300 0x7fcb340b2db0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009e50 tx=0x7fcb2c02a710 comp rx=0 tx=0).stop 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.118+0000 7fcb3a81f640 1 -- 172.21.15.139:0/922246213 shutdown_connections 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.118+0000 7fcb3a81f640 1 --2- 172.21.15.139:0/922246213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb3406f300 0x7fcb340b2db0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.118+0000 7fcb3a81f640 1 --2- 172.21.15.139:0/922246213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb3415dc90 0x7fcb340ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.118+0000 7fcb3a81f640 1 -- 172.21.15.139:0/922246213 >> 172.21.15.139:0/922246213 conn(0x7fcb340cfd80 msgr2=0x7fcb340d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.118+0000 7fcb3a81f640 1 -- 172.21.15.139:0/922246213 shutdown_connections 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.118+0000 7fcb3a81f640 1 -- 172.21.15.139:0/922246213 wait complete. 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.119+0000 7fcb3a81f640 1 Processor -- start 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.119+0000 7fcb3a81f640 1 -- start start 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb3a81f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb3406f300 0x7fcb34157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb3a81f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb340cf3e0 0x7fcb34160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb3a81f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb3415dc90 0x7fcb34145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb3a81f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3406b920 con 0x7fcb3406f300 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb3a81f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34072620 con 0x7fcb340cf3e0 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb3a81f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340b32f0 con 0x7fcb3415dc90 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb33fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb3406f300 0x7fcb34157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb33fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb3406f300 0x7fcb34157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42260/0 (socket says 172.21.15.139:42260) 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb33fff640 1 -- 172.21.15.139:0/871456594 learned_addr learned my addr 172.21.15.139:0/871456594 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb337fe640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb340cf3e0 0x7fcb34160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb38dbc640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb3415dc90 0x7fcb34145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb33fff640 1 -- 172.21.15.139:0/871456594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb3415dc90 msgr2=0x7fcb34145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb33fff640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb3415dc90 0x7fcb34145980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb33fff640 1 -- 172.21.15.139:0/871456594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb340cf3e0 msgr2=0x7fcb34160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb33fff640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb340cf3e0 0x7fcb34160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb33fff640 1 -- 172.21.15.139:0/871456594 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34146240 con 0x7fcb3406f300 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb33fff640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb3406f300 0x7fcb34157e60 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400d5f0 tx=0x7fcb2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb30ff9640 1 -- 172.21.15.139:0/871456594 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240125f0 con 0x7fcb3406f300 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb30ff9640 1 -- 172.21.15.139:0/871456594 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb24019070 con 0x7fcb3406f300 2024-04-25T10:02:37.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb30ff9640 1 -- 172.21.15.139:0/871456594 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240129d0 con 0x7fcb3406f300 2024-04-25T10:02:37.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb3a81f640 1 -- 172.21.15.139:0/871456594 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb341584d0 con 0x7fcb3406f300 2024-04-25T10:02:37.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.120+0000 7fcb3a81f640 1 -- 172.21.15.139:0/871456594 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3415e1a0 con 0x7fcb3406f300 2024-04-25T10:02:37.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.121+0000 7fcb3a81f640 1 -- 172.21.15.139:0/871456594 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb34074520 con 0x7fcb3406f300 2024-04-25T10:02:37.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.122+0000 7fcb30ff9640 1 -- 172.21.15.139:0/871456594 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb24012b70 con 0x7fcb3406f300 2024-04-25T10:02:37.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.122+0000 7fcb30ff9640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcb2006b500 0x7fcb2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.122+0000 7fcb337fe640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcb2006b500 0x7fcb2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.122+0000 7fcb30ff9640 1 -- 172.21.15.139:0/871456594 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fcb2408a7a0 con 0x7fcb3406f300 2024-04-25T10:02:37.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.123+0000 7fcb337fe640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcb2006b500 0x7fcb2006d9c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcb3414c8c0 tx=0x7fcb28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.126+0000 7fcb30ff9640 1 -- 172.21.15.139:0/871456594 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb34146240 con 0x7fcb3406f300 2024-04-25T10:02:37.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.297+0000 7fcb3a81f640 1 -- 172.21.15.139:0/871456594 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb340cbfc0 con 0x7fcb3406f300 2024-04-25T10:02:37.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.297+0000 7fcb30ff9640 1 -- 172.21.15.139:0/871456594 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcb24059440 con 0x7fcb3406f300 2024-04-25T10:02:37.302 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T10:02:37.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.299+0000 7fcb3a81f640 1 -- 172.21.15.139:0/871456594 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcb2006b500 msgr2=0x7fcb2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.299+0000 7fcb3a81f640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcb2006b500 0x7fcb2006d9c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcb3414c8c0 tx=0x7fcb28009040 comp rx=0 tx=0).stop 2024-04-25T10:02:37.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.299+0000 7fcb3a81f640 1 -- 172.21.15.139:0/871456594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb3406f300 msgr2=0x7fcb34157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.299+0000 7fcb3a81f640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb3406f300 0x7fcb34157e60 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400d5f0 tx=0x7fcb2400dab0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.299+0000 7fcb3a81f640 1 -- 172.21.15.139:0/871456594 shutdown_connections 2024-04-25T10:02:37.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.299+0000 7fcb3a81f640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcb2006b500 0x7fcb2006d9c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.299+0000 7fcb3a81f640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb3415dc90 0x7fcb34145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.299+0000 7fcb3a81f640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb340cf3e0 0x7fcb34160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.299+0000 7fcb3a81f640 1 --2- 172.21.15.139:0/871456594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb3406f300 0x7fcb34157e60 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.299+0000 7fcb3a81f640 1 -- 172.21.15.139:0/871456594 >> 172.21.15.139:0/871456594 conn(0x7fcb340cfd80 msgr2=0x7fcb340b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.299+0000 7fcb3a81f640 1 -- 172.21.15.139:0/871456594 shutdown_connections 2024-04-25T10:02:37.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.299+0000 7fcb3a81f640 1 -- 172.21.15.139:0/871456594 wait complete. 2024-04-25T10:02:37.315 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:37.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.409+0000 7f586848f640 1 Processor -- start 2024-04-25T10:02:37.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.409+0000 7f586848f640 1 -- start start 2024-04-25T10:02:37.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.410+0000 7f586848f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f586015a870 0x7f58600b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.410+0000 7f586848f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58600b4190 0x7f58600d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.410+0000 7f586848f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58600b45f0 con 0x7f5860070260 2024-04-25T10:02:37.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.410+0000 7f586848f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58600d1020 con 0x7f58600b4190 2024-04-25T10:02:37.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.410+0000 7f586848f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58600b5190 con 0x7f586015a870 2024-04-25T10:02:37.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.410+0000 7f5865a2a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58600b4190 0x7f58600d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.410+0000 7f5866a2c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5860070260 0x7f58600ce610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37274/0 (socket says 172.21.15.139:37274) 2024-04-25T10:02:37.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.411+0000 7f5865a2a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58600b4190 0x7f58600d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42268/0 (socket says 172.21.15.139:42268) 2024-04-25T10:02:37.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.411+0000 7f5866a2c640 1 -- 172.21.15.139:0/2881602490 learned_addr learned my addr 172.21.15.139:0/2881602490 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:37.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.411+0000 7f586622b640 1 --2- 172.21.15.139:0/2881602490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f586015a870 0x7f58600b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.411+0000 7f5865a2a640 1 -- 172.21.15.139:0/2881602490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f586015a870 msgr2=0x7f58600b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.411+0000 7f5865a2a640 1 --2- 172.21.15.139:0/2881602490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f586015a870 0x7f58600b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.411+0000 7f5865a2a640 1 -- 172.21.15.139:0/2881602490 >> v1:172.21.15.139:6789/0 conn(0x7f5860070260 legacy=0x7f58600ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.411+0000 7f5865a2a640 1 -- 172.21.15.139:0/2881602490 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58600d0d40 con 0x7f58600b4190 2024-04-25T10:02:37.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.411+0000 7f5865229640 1 -- 172.21.15.139:0/2881602490 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061803348 0 0) 0x7f58600b45f0 con 0x7f5860070260 2024-04-25T10:02:37.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.412+0000 7f5865a2a640 1 --2- 172.21.15.139:0/2881602490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58600b4190 0x7f58600d05f0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f585000ba40 tx=0x7f585002c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61556c6ed96a8ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.412+0000 7f5865229640 1 -- 172.21.15.139:0/2881602490 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850031710 con 0x7f58600b4190 2024-04-25T10:02:37.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.412+0000 7f5865229640 1 -- 172.21.15.139:0/2881602490 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58500318b0 con 0x7f58600b4190 2024-04-25T10:02:37.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.412+0000 7f5865229640 1 -- 172.21.15.139:0/2881602490 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850031c80 con 0x7f58600b4190 2024-04-25T10:02:37.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.412+0000 7f586848f640 1 -- 172.21.15.139:0/2881602490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58600b4190 msgr2=0x7f58600d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.412+0000 7f586848f640 1 --2- 172.21.15.139:0/2881602490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58600b4190 0x7f58600d05f0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f585000ba40 tx=0x7f585002c720 comp rx=0 tx=0).stop 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.412+0000 7f586848f640 1 -- 172.21.15.139:0/2881602490 shutdown_connections 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.412+0000 7f586848f640 1 --2- 172.21.15.139:0/2881602490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58600b4190 0x7f58600d05f0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.412+0000 7f586848f640 1 --2- 172.21.15.139:0/2881602490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f586015a870 0x7f58600b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.412+0000 7f586848f640 1 -- 172.21.15.139:0/2881602490 >> 172.21.15.139:0/2881602490 conn(0x7f58600b23f0 msgr2=0x7f58600b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.412+0000 7f586848f640 1 -- 172.21.15.139:0/2881602490 shutdown_connections 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.412+0000 7f586848f640 1 -- 172.21.15.139:0/2881602490 wait complete. 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.413+0000 7f586848f640 1 Processor -- start 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.413+0000 7f586848f640 1 -- start start 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.414+0000 7f586848f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5860070260 0x7f58600c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.414+0000 7f586848f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58600b4190 0x7f586015b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.414+0000 7f586848f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586015a870 0x7f58601eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.414+0000 7f586848f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58600706f0 con 0x7f58600b4190 2024-04-25T10:02:37.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.414+0000 7f586848f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58600b4e20 con 0x7f586015a870 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.414+0000 7f586848f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5860070870 con 0x7f5860070260 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.414+0000 7f5865a2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58600b4190 0x7f586015b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.414+0000 7f5865a2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58600b4190 0x7f586015b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42272/0 (socket says 172.21.15.139:42272) 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.414+0000 7f5865a2a640 1 -- 172.21.15.139:0/3413759439 learned_addr learned my addr 172.21.15.139:0/3413759439 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.414+0000 7f586622b640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5860070260 0x7f58600c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.414+0000 7f5866a2c640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586015a870 0x7f58601eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.415+0000 7f5865a2a640 1 -- 172.21.15.139:0/3413759439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5860070260 msgr2=0x7f58600c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.415+0000 7f5865a2a640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5860070260 0x7f58600c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.415+0000 7f5865a2a640 1 -- 172.21.15.139:0/3413759439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586015a870 msgr2=0x7f58601eefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.415+0000 7f5865a2a640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586015a870 0x7f58601eefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.415+0000 7f5865a2a640 1 -- 172.21.15.139:0/3413759439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586015beb0 con 0x7f58600b4190 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.415+0000 7f5865a2a640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58600b4190 0x7f586015b660 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f585002cd10 tx=0x7f58500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.415+0000 7f5856ffd640 1 -- 172.21.15.139:0/3413759439 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850033c40 con 0x7f58600b4190 2024-04-25T10:02:37.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.415+0000 7f5856ffd640 1 -- 172.21.15.139:0/3413759439 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5850033de0 con 0x7f58600b4190 2024-04-25T10:02:37.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.415+0000 7f586848f640 1 -- 172.21.15.139:0/3413759439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586015c190 con 0x7f58600b4190 2024-04-25T10:02:37.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.415+0000 7f5856ffd640 1 -- 172.21.15.139:0/3413759439 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585003e660 con 0x7f58600b4190 2024-04-25T10:02:37.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.415+0000 7f586848f640 1 -- 172.21.15.139:0/3413759439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586015c750 con 0x7f58600b4190 2024-04-25T10:02:37.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.416+0000 7f586848f640 1 -- 172.21.15.139:0/3413759439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5860074520 con 0x7f58600b4190 2024-04-25T10:02:37.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.417+0000 7f5856ffd640 1 -- 172.21.15.139:0/3413759439 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f585003b040 con 0x7f58600b4190 2024-04-25T10:02:37.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.417+0000 7f5856ffd640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f584406b4d0 0x7f584406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.417+0000 7f5856ffd640 1 -- 172.21.15.139:0/3413759439 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f58500ad840 con 0x7f58600b4190 2024-04-25T10:02:37.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.417+0000 7f586622b640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f584406b4d0 0x7f584406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.417+0000 7f586622b640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f584406b4d0 0x7f584406d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f585c011340 tx=0x7f585c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.421+0000 7f5856ffd640 1 -- 172.21.15.139:0/3413759439 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5850042030 con 0x7f58600b4190 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.590+0000 7f586848f640 1 -- 172.21.15.139:0/3413759439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58600ccf60 con 0x7f58600b4190 2024-04-25T10:02:37.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.590+0000 7f5856ffd640 1 -- 172.21.15.139:0/3413759439 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7f585007be90 con 0x7f58600b4190 2024-04-25T10:02:37.595 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T10:02:37.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.592+0000 7f586848f640 1 -- 172.21.15.139:0/3413759439 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f584406b4d0 msgr2=0x7f584406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.592+0000 7f586848f640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f584406b4d0 0x7f584406d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f585c011340 tx=0x7f585c00b040 comp rx=0 tx=0).stop 2024-04-25T10:02:37.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.592+0000 7f586848f640 1 -- 172.21.15.139:0/3413759439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58600b4190 msgr2=0x7f586015b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.592+0000 7f586848f640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58600b4190 0x7f586015b660 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f585002cd10 tx=0x7f58500046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.592+0000 7f586848f640 1 -- 172.21.15.139:0/3413759439 shutdown_connections 2024-04-25T10:02:37.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.592+0000 7f586848f640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58600b4190 0x7f586015b660 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.592+0000 7f586848f640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f584406b4d0 0x7f584406d990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.592+0000 7f586848f640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586015a870 0x7f58601eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.592+0000 7f586848f640 1 --2- 172.21.15.139:0/3413759439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5860070260 0x7f58600c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.592+0000 7f586848f640 1 -- 172.21.15.139:0/3413759439 >> 172.21.15.139:0/3413759439 conn(0x7f58600b23f0 msgr2=0x7f58600cd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.592+0000 7f586848f640 1 -- 172.21.15.139:0/3413759439 shutdown_connections 2024-04-25T10:02:37.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.592+0000 7f586848f640 1 -- 172.21.15.139:0/3413759439 wait complete. 2024-04-25T10:02:37.608 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:37.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.700+0000 7fbfe7922640 1 Processor -- start 2024-04-25T10:02:37.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.700+0000 7fbfe7922640 1 -- start start 2024-04-25T10:02:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfe7922640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe0161850 0x7fbfe00b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfe7922640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe00cf250 con 0x7fbfe00d1080 2024-04-25T10:02:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfe7922640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe00ce9b0 con 0x7fbfe015dc60 2024-04-25T10:02:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfe7922640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00d3730 con 0x7fbfe0161850 2024-04-25T10:02:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfe4ebd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbfe00d1080 0x7fbfe00d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37290/0 (socket says 172.21.15.139:37290) 2024-04-25T10:02:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfe4ebd640 1 -- 172.21.15.139:0/1454102537 learned_addr learned my addr 172.21.15.139:0/1454102537 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfe5ebf640 1 --2- 172.21.15.139:0/1454102537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe0161850 0x7fbfe00b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfd7fff640 1 -- 172.21.15.139:0/1454102537 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 538381396 0 0) 0x7fbfd0001110 con 0x7fbfe00d1080 2024-04-25T10:02:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfd7fff640 1 -- 172.21.15.139:0/1454102537 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfcc003520 con 0x7fbfe00d1080 2024-04-25T10:02:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfe5ebf640 1 -- 172.21.15.139:0/1454102537 >> v1:172.21.15.150:6789/0 conn(0x7fbfe015dc60 legacy=0x7fbfe00d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfd7fff640 1 -- 172.21.15.139:0/1454102537 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201896134 0 0) 0x7fbfe00ce9b0 con 0x7fbfe015dc60 2024-04-25T10:02:37.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfe5ebf640 1 -- 172.21.15.139:0/1454102537 >> v1:172.21.15.139:6789/0 conn(0x7fbfe00d1080 legacy=0x7fbfe00d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.701+0000 7fbfe5ebf640 1 -- 172.21.15.139:0/1454102537 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe00cec90 con 0x7fbfe0161850 2024-04-25T10:02:37.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.702+0000 7fbfe5ebf640 1 --2- 172.21.15.139:0/1454102537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe0161850 0x7fbfe00b6c10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009dc0 tx=0x7fbfd802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a009ea0fd5a64854 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.702+0000 7fbfd7fff640 1 -- 172.21.15.139:0/1454102537 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8007740 con 0x7fbfe0161850 2024-04-25T10:02:37.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.702+0000 7fbfd7fff640 1 -- 172.21.15.139:0/1454102537 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd80078e0 con 0x7fbfe0161850 2024-04-25T10:02:37.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.702+0000 7fbfd7fff640 1 -- 172.21.15.139:0/1454102537 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8007c90 con 0x7fbfe0161850 2024-04-25T10:02:37.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.703+0000 7fbfe7922640 1 -- 172.21.15.139:0/1454102537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe0161850 msgr2=0x7fbfe00b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.703+0000 7fbfe7922640 1 --2- 172.21.15.139:0/1454102537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe0161850 0x7fbfe00b6c10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009dc0 tx=0x7fbfd802a6d0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.703+0000 7fbfe7922640 1 -- 172.21.15.139:0/1454102537 shutdown_connections 2024-04-25T10:02:37.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.703+0000 7fbfe7922640 1 --2- 172.21.15.139:0/1454102537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe0161850 0x7fbfe00b6c10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.703+0000 7fbfe7922640 1 -- 172.21.15.139:0/1454102537 >> 172.21.15.139:0/1454102537 conn(0x7fbfe00b5910 msgr2=0x7fbfe00b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.703+0000 7fbfe7922640 1 -- 172.21.15.139:0/1454102537 shutdown_connections 2024-04-25T10:02:37.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.703+0000 7fbfe7922640 1 -- 172.21.15.139:0/1454102537 wait complete. 2024-04-25T10:02:37.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.704+0000 7fbfe7922640 1 Processor -- start 2024-04-25T10:02:37.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.704+0000 7fbfe7922640 1 -- start start 2024-04-25T10:02:37.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe7922640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe00d1080 0x7fbfe014a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe7922640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe015dc60 0x7fbfe0153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe7922640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfe0161850 0x7fbfe0153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe7922640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe015e040 con 0x7fbfe015dc60 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe7922640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00d1fc0 con 0x7fbfe0161850 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe7922640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe015fd70 con 0x7fbfe00d1080 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe4ebd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe015dc60 0x7fbfe0153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe4ebd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe015dc60 0x7fbfe0153850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42274/0 (socket says 172.21.15.139:42274) 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe4ebd640 1 -- 172.21.15.139:0/3570925324 learned_addr learned my addr 172.21.15.139:0/3570925324 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe56be640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe00d1080 0x7fbfe014a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe5ebf640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfe0161850 0x7fbfe0153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe4ebd640 1 -- 172.21.15.139:0/3570925324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe00d1080 msgr2=0x7fbfe014a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe4ebd640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe00d1080 0x7fbfe014a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe4ebd640 1 -- 172.21.15.139:0/3570925324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfe0161850 msgr2=0x7fbfe0153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe4ebd640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfe0161850 0x7fbfe0153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe4ebd640 1 -- 172.21.15.139:0/3570925324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe014b3e0 con 0x7fbfe015dc60 2024-04-25T10:02:37.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe5ebf640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfe0161850 0x7fbfe0153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:37.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.705+0000 7fbfe4ebd640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe015dc60 0x7fbfe0153850 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000c730 tx=0x7fbfd000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.706+0000 7fbfd5ffb640 1 -- 172.21.15.139:0/3570925324 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0017070 con 0x7fbfe015dc60 2024-04-25T10:02:37.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.706+0000 7fbfd5ffb640 1 -- 172.21.15.139:0/3570925324 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd0005ad0 con 0x7fbfe015dc60 2024-04-25T10:02:37.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.706+0000 7fbfe7922640 1 -- 172.21.15.139:0/3570925324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe015f440 con 0x7fbfe015dc60 2024-04-25T10:02:37.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.706+0000 7fbfd5ffb640 1 -- 172.21.15.139:0/3570925324 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0007e20 con 0x7fbfe015dc60 2024-04-25T10:02:37.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.706+0000 7fbfe7922640 1 -- 172.21.15.139:0/3570925324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe015fa00 con 0x7fbfe015dc60 2024-04-25T10:02:37.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.707+0000 7fbfe7922640 1 -- 172.21.15.139:0/3570925324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe0074520 con 0x7fbfe015dc60 2024-04-25T10:02:37.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.707+0000 7fbfd5ffb640 1 -- 172.21.15.139:0/3570925324 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfd001a070 con 0x7fbfe015dc60 2024-04-25T10:02:37.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.708+0000 7fbfd5ffb640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfc406b4d0 0x7fbfc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.708+0000 7fbfd5ffb640 1 -- 172.21.15.139:0/3570925324 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fbfd00898a0 con 0x7fbfe015dc60 2024-04-25T10:02:37.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.708+0000 7fbfe56be640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfc406b4d0 0x7fbfc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.708+0000 7fbfe56be640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfc406b4d0 0x7fbfc406d990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc000f60 tx=0x7fbfdc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.712+0000 7fbfd5ffb640 1 -- 172.21.15.139:0/3570925324 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfe00cf250 con 0x7fbfe015dc60 2024-04-25T10:02:37.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.921+0000 7fbfe7922640 1 -- 172.21.15.139:0/3570925324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfe00d16f0 con 0x7fbfe015dc60 2024-04-25T10:02:37.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.921+0000 7fbfd5ffb640 1 -- 172.21.15.139:0/3570925324 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fbfd0058420 con 0x7fbfe015dc60 2024-04-25T10:02:37.926 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:37.926 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8961,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2441954626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2441954626},{"type":"v1","addr":"172.21.15.139:6837","nonce":2441954626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":76,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:15.497940+0000","modified":"2024-04-25T10:02:35.776771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:02:37.927 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 77 2024-04-25T10:02:37.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.923+0000 7fbfe7922640 1 -- 172.21.15.139:0/3570925324 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfc406b4d0 msgr2=0x7fbfc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.923+0000 7fbfe7922640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfc406b4d0 0x7fbfc406d990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc000f60 tx=0x7fbfdc009040 comp rx=0 tx=0).stop 2024-04-25T10:02:37.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.923+0000 7fbfe7922640 1 -- 172.21.15.139:0/3570925324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe015dc60 msgr2=0x7fbfe0153850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.923+0000 7fbfe7922640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe015dc60 0x7fbfe0153850 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000c730 tx=0x7fbfd000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.923+0000 7fbfe7922640 1 -- 172.21.15.139:0/3570925324 shutdown_connections 2024-04-25T10:02:37.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.923+0000 7fbfe7922640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe015dc60 0x7fbfe0153850 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.923+0000 7fbfe7922640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfc406b4d0 0x7fbfc406d990 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.923+0000 7fbfe7922640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfe0161850 0x7fbfe0153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.923+0000 7fbfe7922640 1 --2- 172.21.15.139:0/3570925324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe00d1080 0x7fbfe014a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.923+0000 7fbfe7922640 1 -- 172.21.15.139:0/3570925324 >> 172.21.15.139:0/3570925324 conn(0x7fbfe00b5910 msgr2=0x7fbfe00b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.924+0000 7fbfe7922640 1 -- 172.21.15.139:0/3570925324 shutdown_connections 2024-04-25T10:02:37.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:37.924+0000 7fbfe7922640 1 -- 172.21.15.139:0/3570925324 wait complete. 2024-04-25T10:02:37.939 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.031+0000 7fed5f5a5640 1 Processor -- start 2024-04-25T10:02:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.032+0000 7fed5f5a5640 1 -- start start 2024-04-25T10:02:38.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.032+0000 7fed5f5a5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580b0750 0x7fed580b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.032+0000 7fed5f5a5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fed580d0730 0x7fed580cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.032+0000 7fed5f5a5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed580b1070 con 0x7fed580d0b90 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.032+0000 7fed5f5a5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed580ccb70 con 0x7fed580d0730 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.032+0000 7fed5f5a5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed580cf970 con 0x7fed580b0750 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed5db42640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fed580d0b90 0x7fed580cc380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37296/0 (socket says 172.21.15.139:37296) 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed5db42640 1 -- 172.21.15.139:0/1279975954 learned_addr learned my addr 172.21.15.139:0/1279975954 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090760767 0 0) 0x7fed580b1070 con 0x7fed580d0b90 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed44003520 con 0x7fed580d0b90 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed5d341640 1 --2- 172.21.15.139:0/1279975954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580b0750 0x7fed580b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed5cb40640 1 --2- 172.21.15.139:0/1279975954 >> v2:172.21.15.150:3300/0 conn(0x7fed580d0730 0x7fed580cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2810220013 0 0) 0x7fed44003520 con 0x7fed580d0b90 2024-04-25T10:02:38.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580b0750 msgr2=0x7fed580b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 --2- 172.21.15.139:0/1279975954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580b0750 0x7fed580b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 >> v2:172.21.15.150:3300/0 conn(0x7fed580d0730 msgr2=0x7fed580cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 --2- 172.21.15.139:0/1279975954 >> v2:172.21.15.150:3300/0 conn(0x7fed580d0730 0x7fed580cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed580cfc50 con 0x7fed580d0b90 2024-04-25T10:02:38.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fed500019f0 con 0x7fed580d0b90 2024-04-25T10:02:38.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 >> v1:172.21.15.139:6789/0 conn(0x7fed580d0b90 legacy=0x7fed580cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 --2- 172.21.15.139:0/1279975954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed440039b0 0x7fed44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 --2- 172.21.15.139:0/1279975954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580b0750 0x7fed44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 --2- 172.21.15.139:0/1279975954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed44004d40 0x7fed440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed580ccb70 con 0x7fed44004d40 2024-04-25T10:02:38.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed580cf970 con 0x7fed440039b0 2024-04-25T10:02:38.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4400b9c0 con 0x7fed580b0750 2024-04-25T10:02:38.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed5db42640 1 --2- 172.21.15.139:0/1279975954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed44004d40 0x7fed440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed5d341640 1 --2- 172.21.15.139:0/1279975954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed440039b0 0x7fed44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.033+0000 7fed5cb40640 1 --2- 172.21.15.139:0/1279975954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580b0750 0x7fed44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.034+0000 7fed5db42640 1 -- 172.21.15.139:0/1279975954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580b0750 msgr2=0x7fed44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.034+0000 7fed5db42640 1 --2- 172.21.15.139:0/1279975954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580b0750 0x7fed44004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.034+0000 7fed5db42640 1 -- 172.21.15.139:0/1279975954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed440039b0 msgr2=0x7fed44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.034+0000 7fed5db42640 1 --2- 172.21.15.139:0/1279975954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed440039b0 0x7fed44003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.034+0000 7fed5db42640 1 -- 172.21.15.139:0/1279975954 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed44003520 con 0x7fed44004d40 2024-04-25T10:02:38.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.034+0000 7fed5db42640 1 --2- 172.21.15.139:0/1279975954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed44004d40 0x7fed440051c0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fed5000a890 tx=0x7fed44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9481fe1d72c5683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.034+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5000a4e0 con 0x7fed44004d40 2024-04-25T10:02:38.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.034+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5002f070 con 0x7fed44004d40 2024-04-25T10:02:38.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.034+0000 7fed4ffff640 1 -- 172.21.15.139:0/1279975954 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5002bcf0 con 0x7fed44004d40 2024-04-25T10:02:38.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.035+0000 7fed5f5a5640 1 -- 172.21.15.139:0/1279975954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed44004d40 msgr2=0x7fed440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.035+0000 7fed5f5a5640 1 --2- 172.21.15.139:0/1279975954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed44004d40 0x7fed440051c0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fed5000a890 tx=0x7fed44005830 comp rx=0 tx=0).stop 2024-04-25T10:02:38.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.035+0000 7fed5db42640 1 -- 172.21.15.139:0/1279975954 reap_dead start 2024-04-25T10:02:38.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.035+0000 7fed5f5a5640 1 -- 172.21.15.139:0/1279975954 shutdown_connections 2024-04-25T10:02:38.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.035+0000 7fed5f5a5640 1 -- 172.21.15.139:0/1279975954 >> 172.21.15.139:0/1279975954 conn(0x7fed580b3380 msgr2=0x7fed580b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.035+0000 7fed5f5a5640 1 -- 172.21.15.139:0/1279975954 shutdown_connections 2024-04-25T10:02:38.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.035+0000 7fed5f5a5640 1 -- 172.21.15.139:0/1279975954 wait complete. 2024-04-25T10:02:38.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.036+0000 7fed5f5a5640 1 Processor -- start 2024-04-25T10:02:38.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.036+0000 7fed5f5a5640 1 -- start start 2024-04-25T10:02:38.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5f5a5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580cad00 0x7fed58151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5f5a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed580cb0e0 0x7fed58147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5f5a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed58150db0 0x7fed58147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5f5a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5815bcc0 con 0x7fed58150db0 2024-04-25T10:02:38.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5f5a5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed580d0210 con 0x7fed580cb0e0 2024-04-25T10:02:38.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5f5a5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5815be40 con 0x7fed580cad00 2024-04-25T10:02:38.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5db42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed58150db0 0x7fed58147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5db42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed58150db0 0x7fed58147c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42284/0 (socket says 172.21.15.139:42284) 2024-04-25T10:02:38.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5db42640 1 -- 172.21.15.139:0/3988411994 learned_addr learned my addr 172.21.15.139:0/3988411994 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:38.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5d341640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580cad00 0x7fed58151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5db42640 1 -- 172.21.15.139:0/3988411994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580cad00 msgr2=0x7fed58151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5db42640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580cad00 0x7fed58151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5db42640 1 -- 172.21.15.139:0/3988411994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed580cb0e0 msgr2=0x7fed58147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5db42640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed580cb0e0 0x7fed58147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5db42640 1 -- 172.21.15.139:0/3988411994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed58154e70 con 0x7fed58150db0 2024-04-25T10:02:38.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed5db42640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed58150db0 0x7fed58147c90 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fed5000a890 tx=0x7fed50001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.037+0000 7fed4dffb640 1 -- 172.21.15.139:0/3988411994 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed50002040 con 0x7fed58150db0 2024-04-25T10:02:38.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.038+0000 7fed4dffb640 1 -- 172.21.15.139:0/3988411994 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5000a4c0 con 0x7fed58150db0 2024-04-25T10:02:38.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.038+0000 7fed4dffb640 1 -- 172.21.15.139:0/3988411994 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed500346b0 con 0x7fed58150db0 2024-04-25T10:02:38.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.038+0000 7fed5f5a5640 1 -- 172.21.15.139:0/3988411994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed581f4d10 con 0x7fed58150db0 2024-04-25T10:02:38.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.038+0000 7fed5f5a5640 1 -- 172.21.15.139:0/3988411994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed581f52d0 con 0x7fed58150db0 2024-04-25T10:02:38.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.039+0000 7fed5f5a5640 1 -- 172.21.15.139:0/3988411994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed58074520 con 0x7fed58150db0 2024-04-25T10:02:38.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.039+0000 7fed4dffb640 1 -- 172.21.15.139:0/3988411994 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed50034850 con 0x7fed58150db0 2024-04-25T10:02:38.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.040+0000 7fed4dffb640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed3c06b4d0 0x7fed3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.040+0000 7fed4dffb640 1 -- 172.21.15.139:0/3988411994 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fed500aa5e0 con 0x7fed58150db0 2024-04-25T10:02:38.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.040+0000 7fed5d341640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed3c06b4d0 0x7fed3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.040+0000 7fed5d341640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed3c06b4d0 0x7fed3c06d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fed48009980 tx=0x7fed48008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.044+0000 7fed4dffb640 1 -- 172.21.15.139:0/3988411994 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed50045060 con 0x7fed58150db0 2024-04-25T10:02:38.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.214+0000 7fed5f5a5640 1 -- 172.21.15.139:0/3988411994 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed580b6d70 con 0x7fed58150db0 2024-04-25T10:02:38.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.215+0000 7fed4dffb640 1 -- 172.21.15.139:0/3988411994 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17291 (secure 0 0 0) 0x7fed500791f0 con 0x7fed58150db0 2024-04-25T10:02:38.220 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:38.220 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":109,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:02:35.776788+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:13.695577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:14.460042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2a","mappings":[{"from":1,"to":7}]},{"pgid":"16.2f","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6835/359626718":"2024-04-26T10:02:35.776758+0000","172.21.15.139:6837/359626718":"2024-04-26T10:02:35.776758+0000","172.21.15.139:6837/4074693991":"2024-04-26T10:01:59.507353+0000","172.21.15.139:6835/4074693991":"2024-04-26T10:01:59.507353+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:38.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.217+0000 7fed5f5a5640 1 -- 172.21.15.139:0/3988411994 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed3c06b4d0 msgr2=0x7fed3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.217+0000 7fed5f5a5640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed3c06b4d0 0x7fed3c06d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fed48009980 tx=0x7fed48008040 comp rx=0 tx=0).stop 2024-04-25T10:02:38.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.217+0000 7fed5f5a5640 1 -- 172.21.15.139:0/3988411994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed58150db0 msgr2=0x7fed58147c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.217+0000 7fed5f5a5640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed58150db0 0x7fed58147c90 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fed5000a890 tx=0x7fed50001360 comp rx=0 tx=0).stop 2024-04-25T10:02:38.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.217+0000 7fed5f5a5640 1 -- 172.21.15.139:0/3988411994 shutdown_connections 2024-04-25T10:02:38.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.217+0000 7fed5f5a5640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed58150db0 0x7fed58147c90 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.217+0000 7fed5f5a5640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fed3c06b4d0 0x7fed3c06d990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.217+0000 7fed5f5a5640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed580cb0e0 0x7fed58147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.217+0000 7fed5f5a5640 1 --2- 172.21.15.139:0/3988411994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed580cad00 0x7fed58151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.217+0000 7fed5f5a5640 1 -- 172.21.15.139:0/3988411994 >> 172.21.15.139:0/3988411994 conn(0x7fed580b3380 msgr2=0x7fed580b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.217+0000 7fed5f5a5640 1 -- 172.21.15.139:0/3988411994 shutdown_connections 2024-04-25T10:02:38.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.217+0000 7fed5f5a5640 1 -- 172.21.15.139:0/3988411994 wait complete. 2024-04-25T10:02:38.233 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:02:38.233 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:38.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.326+0000 7f31e086b640 1 Processor -- start 2024-04-25T10:02:38.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.326+0000 7f31e086b640 1 -- start start 2024-04-25T10:02:38.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31e086b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f31dc147660 0x7f31dc06f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31e086b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc072620 0x7f31dc0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31e086b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31dc0b26b0 con 0x7f31dc0715e0 2024-04-25T10:02:38.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31e086b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31dc0b3ad0 con 0x7f31dc147660 2024-04-25T10:02:38.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31e086b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc06e050 con 0x7f31dc072620 2024-04-25T10:02:38.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31d3fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc072620 0x7f31dc0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31d3fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc072620 0x7f31dc0b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:59736/0 (socket says 172.21.15.139:59736) 2024-04-25T10:02:38.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31d3fff640 1 -- 172.21.15.139:0/2679463967 learned_addr learned my addr 172.21.15.139:0/2679463967 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:38.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31da59c640 1 --2- 172.21.15.139:0/2679463967 >> v2:172.21.15.150:3300/0 conn(0x7f31dc147660 0x7f31dc06f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31d3fff640 1 -- 172.21.15.139:0/2679463967 >> v2:172.21.15.150:3300/0 conn(0x7f31dc147660 msgr2=0x7f31dc06f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31d3fff640 1 --2- 172.21.15.139:0/2679463967 >> v2:172.21.15.150:3300/0 conn(0x7f31dc147660 0x7f31dc06f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31d3fff640 1 -- 172.21.15.139:0/2679463967 >> v1:172.21.15.150:6789/0 conn(0x7f31dc0715e0 legacy=0x7f31dc161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.327+0000 7f31d3fff640 1 -- 172.21.15.139:0/2679463967 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc06e330 con 0x7f31dc072620 2024-04-25T10:02:38.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.328+0000 7f31d3fff640 1 --2- 172.21.15.139:0/2679463967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc072620 0x7f31dc0b3590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009870 tx=0x7f31c402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a74c4dd8e7e74486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.328+0000 7f31d9d9b640 1 -- 172.21.15.139:0/2679463967 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4037070 con 0x7f31dc072620 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.328+0000 7f31d9d9b640 1 -- 172.21.15.139:0/2679463967 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c4032440 con 0x7f31dc072620 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.328+0000 7f31d9d9b640 1 -- 172.21.15.139:0/2679463967 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4032810 con 0x7f31dc072620 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.329+0000 7f31e086b640 1 -- 172.21.15.139:0/2679463967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc072620 msgr2=0x7f31dc0b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.329+0000 7f31e086b640 1 --2- 172.21.15.139:0/2679463967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc072620 0x7f31dc0b3590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009870 tx=0x7f31c402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.329+0000 7f31e086b640 1 -- 172.21.15.139:0/2679463967 shutdown_connections 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.329+0000 7f31e086b640 1 --2- 172.21.15.139:0/2679463967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc072620 0x7f31dc0b3590 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.329+0000 7f31e086b640 1 --2- 172.21.15.139:0/2679463967 >> v2:172.21.15.150:3300/0 conn(0x7f31dc147660 0x7f31dc06f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.329+0000 7f31e086b640 1 -- 172.21.15.139:0/2679463967 >> 172.21.15.139:0/2679463967 conn(0x7f31dc0b17a0 msgr2=0x7f31dc0b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.329+0000 7f31e086b640 1 -- 172.21.15.139:0/2679463967 shutdown_connections 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.329+0000 7f31e086b640 1 -- 172.21.15.139:0/2679463967 wait complete. 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.330+0000 7f31e086b640 1 Processor -- start 2024-04-25T10:02:38.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.330+0000 7f31e086b640 1 -- start start 2024-04-25T10:02:38.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31e086b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc0715e0 0x7f31dc13f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31e086b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc072620 0x7f31dc13a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31e086b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc147660 0x7f31dc13a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31e086b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc06ebd0 con 0x7f31dc147660 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31e086b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc06e8f0 con 0x7f31dc072620 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31e086b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc071a10 con 0x7f31dc0715e0 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31dad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc147660 0x7f31dc13a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31d3fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc072620 0x7f31dc13a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31da59c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc0715e0 0x7f31dc13f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31dad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc147660 0x7f31dc13a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42298/0 (socket says 172.21.15.139:42298) 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31dad9d640 1 -- 172.21.15.139:0/1730398579 learned_addr learned my addr 172.21.15.139:0/1730398579 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31d3fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc072620 0x7f31dc13a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:48740/0 (socket says 172.21.15.139:48740) 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31dad9d640 1 -- 172.21.15.139:0/1730398579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc0715e0 msgr2=0x7f31dc13f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31dad9d640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc0715e0 0x7f31dc13f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31dad9d640 1 -- 172.21.15.139:0/1730398579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc072620 msgr2=0x7f31dc13a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31dad9d640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc072620 0x7f31dc13a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31dad9d640 1 -- 172.21.15.139:0/1730398579 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc13b090 con 0x7f31dc147660 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.331+0000 7f31da59c640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc0715e0 0x7f31dc13f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.332+0000 7f31dad9d640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc147660 0x7f31dc13a7d0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f31d400c7a0 tx=0x7f31d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.332+0000 7f31d2ffd640 1 -- 172.21.15.139:0/1730398579 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d4003b70 con 0x7f31dc147660 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.332+0000 7f31d2ffd640 1 -- 172.21.15.139:0/1730398579 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d4003d30 con 0x7f31dc147660 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.332+0000 7f31e086b640 1 -- 172.21.15.139:0/1730398579 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31dc13b370 con 0x7f31dc147660 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.332+0000 7f31d2ffd640 1 -- 172.21.15.139:0/1730398579 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d400aa30 con 0x7f31dc147660 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.332+0000 7f31e086b640 1 -- 172.21.15.139:0/1730398579 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31dc13b930 con 0x7f31dc147660 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.333+0000 7f31e086b640 1 -- 172.21.15.139:0/1730398579 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31dc06d5c0 con 0x7f31dc147660 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.333+0000 7f31d2ffd640 1 -- 172.21.15.139:0/1730398579 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f31d400abd0 con 0x7f31dc147660 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.334+0000 7f31d2ffd640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f31bc06b4d0 0x7f31bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.334+0000 7f31da59c640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f31bc06b4d0 0x7f31bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.334+0000 7f31d2ffd640 1 -- 172.21.15.139:0/1730398579 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f31d408a160 con 0x7f31dc147660 2024-04-25T10:02:38.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.334+0000 7f31da59c640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f31bc06b4d0 0x7f31bc06d990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009e90 tx=0x7f31cc009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.338+0000 7f31d2ffd640 1 -- 172.21.15.139:0/1730398579 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f31dc0b26b0 con 0x7f31dc147660 2024-04-25T10:02:38.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.541+0000 7f31e086b640 1 -- 172.21.15.139:0/1730398579 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31dc074520 con 0x7f31dc147660 2024-04-25T10:02:38.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.542+0000 7f31d2ffd640 1 -- 172.21.15.139:0/1730398579 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f31d40587b0 con 0x7f31dc147660 2024-04-25T10:02:38.547 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:38.547 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8961,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2441954626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2441954626},{"type":"v1","addr":"172.21.15.139:6837","nonce":2441954626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":76,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:15.497940+0000","modified":"2024-04-25T10:02:35.776771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:02:38.547 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 77 2024-04-25T10:02:38.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.544+0000 7f31e086b640 1 -- 172.21.15.139:0/1730398579 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f31bc06b4d0 msgr2=0x7f31bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.544+0000 7f31e086b640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f31bc06b4d0 0x7f31bc06d990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009e90 tx=0x7f31cc009190 comp rx=0 tx=0).stop 2024-04-25T10:02:38.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.544+0000 7f31e086b640 1 -- 172.21.15.139:0/1730398579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc147660 msgr2=0x7f31dc13a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.544+0000 7f31e086b640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc147660 0x7f31dc13a7d0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f31d400c7a0 tx=0x7f31d400cc60 comp rx=0 tx=0).stop 2024-04-25T10:02:38.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.544+0000 7f31e086b640 1 -- 172.21.15.139:0/1730398579 shutdown_connections 2024-04-25T10:02:38.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.544+0000 7f31e086b640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31dc147660 0x7f31dc13a7d0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.544+0000 7f31e086b640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f31bc06b4d0 0x7f31bc06d990 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.544+0000 7f31e086b640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31dc072620 0x7f31dc13a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.544+0000 7f31e086b640 1 --2- 172.21.15.139:0/1730398579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31dc0715e0 0x7f31dc13f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.544+0000 7f31e086b640 1 -- 172.21.15.139:0/1730398579 >> 172.21.15.139:0/1730398579 conn(0x7f31dc0b17a0 msgr2=0x7f31dc0af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.544+0000 7f31e086b640 1 -- 172.21.15.139:0/1730398579 shutdown_connections 2024-04-25T10:02:38.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.544+0000 7f31e086b640 1 -- 172.21.15.139:0/1730398579 wait complete. 2024-04-25T10:02:38.560 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:38.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.653+0000 7f669afce640 1 Processor -- start 2024-04-25T10:02:38.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.653+0000 7f669afce640 1 -- start start 2024-04-25T10:02:38.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.653+0000 7f669afce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669406e750 0x7f66940721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.653+0000 7f669afce640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66940b4d20 con 0x7f669415b7b0 2024-04-25T10:02:38.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f669afce640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66940b4a40 con 0x7f66940b4510 2024-04-25T10:02:38.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f669afce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6694071710 con 0x7f669406e750 2024-04-25T10:02:38.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f669956b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f669415b7b0 0x7f66940cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37306/0 (socket says 172.21.15.139:37306) 2024-04-25T10:02:38.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f669956b640 1 -- 172.21.15.139:0/24830043 learned_addr learned my addr 172.21.15.139:0/24830043 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:38.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f6698d6a640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669406e750 0x7f66940721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821974825 0 0) 0x7f66940b4d20 con 0x7f669415b7b0 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6680003520 con 0x7f669415b7b0 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1185598932 0 0) 0x7f6680003520 con 0x7f669415b7b0 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669406e750 msgr2=0x7f66940721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669406e750 0x7f66940721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 >> v1:172.21.15.150:6789/0 conn(0x7f66940b4510 legacy=0x7f669406eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66940719f0 con 0x7f669415b7b0 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f667c001670 con 0x7f669415b7b0 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f6698d6a640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669406e750 0x7f66940721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 >> v1:172.21.15.139:6789/0 conn(0x7f669415b7b0 legacy=0x7f66940cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66800039e0 0x7f6680003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6680004390 0x7f6680004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669406e750 0x7f6680005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6694071710 con 0x7f66800039e0 2024-04-25T10:02:38.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668000ba10 con 0x7f6680004390 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668000bc10 con 0x7f669406e750 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f6698d6a640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66800039e0 0x7f6680003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f6693fff640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6680004390 0x7f6680004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.654+0000 7f669956b640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669406e750 0x7f6680005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.655+0000 7f6698d6a640 1 -- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669406e750 msgr2=0x7f6680005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.655+0000 7f6698d6a640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669406e750 0x7f6680005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.655+0000 7f6698d6a640 1 -- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6680004390 msgr2=0x7f6680004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.655+0000 7f6698d6a640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6680004390 0x7f6680004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.655+0000 7f6698d6a640 1 -- 172.21.15.139:0/24830043 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6680003520 con 0x7f66800039e0 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.655+0000 7f6693fff640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6680004390 0x7f6680004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.655+0000 7f6698d6a640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66800039e0 0x7f6680003dc0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f668c010700 tx=0x7f6680005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c382b8ee6c3e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.655+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c03a070 con 0x7f66800039e0 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.655+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c02dd20 con 0x7f66800039e0 2024-04-25T10:02:38.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.655+0000 7f66937fe640 1 -- 172.21.15.139:0/24830043 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c010350 con 0x7f66800039e0 2024-04-25T10:02:38.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.656+0000 7f669afce640 1 -- 172.21.15.139:0/24830043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66800039e0 msgr2=0x7f6680003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.656+0000 7f669afce640 1 --2- 172.21.15.139:0/24830043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66800039e0 0x7f6680003dc0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f668c010700 tx=0x7f6680005830 comp rx=0 tx=0).stop 2024-04-25T10:02:38.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.656+0000 7f669956b640 1 -- 172.21.15.139:0/24830043 reap_dead start 2024-04-25T10:02:38.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.656+0000 7f669afce640 1 -- 172.21.15.139:0/24830043 shutdown_connections 2024-04-25T10:02:38.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.656+0000 7f669afce640 1 -- 172.21.15.139:0/24830043 >> 172.21.15.139:0/24830043 conn(0x7f66940b17a0 msgr2=0x7f66940b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.656+0000 7f669afce640 1 -- 172.21.15.139:0/24830043 shutdown_connections 2024-04-25T10:02:38.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.656+0000 7f669afce640 1 -- 172.21.15.139:0/24830043 wait complete. 2024-04-25T10:02:38.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.657+0000 7f669afce640 1 Processor -- start 2024-04-25T10:02:38.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.657+0000 7f669afce640 1 -- start start 2024-04-25T10:02:38.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669afce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66800039e0 0x7f6694147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669afce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669415d300 0x7f669415d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669afce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6694157360 0x7f66941577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669afce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66941552e0 con 0x7f6694157360 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669afce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6694155160 con 0x7f66800039e0 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669afce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669415bc40 con 0x7f669415d300 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669956b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6694157360 0x7f66941577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669956b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6694157360 0x7f66941577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42322/0 (socket says 172.21.15.139:42322) 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669956b640 1 -- 172.21.15.139:0/3758309385 learned_addr learned my addr 172.21.15.139:0/3758309385 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669956b640 1 -- 172.21.15.139:0/3758309385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669415d300 msgr2=0x7f669415d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f6693fff640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669415d300 0x7f669415d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669956b640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669415d300 0x7f669415d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669956b640 1 -- 172.21.15.139:0/3758309385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66800039e0 msgr2=0x7f6694147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f6698d6a640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66800039e0 0x7f6694147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669956b640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66800039e0 0x7f6694147db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669956b640 1 -- 172.21.15.139:0/3758309385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6694155670 con 0x7f6694157360 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f6693fff640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669415d300 0x7f669415d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.658+0000 7f669956b640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6694157360 0x7f66941577e0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f669415bb90 tx=0x7f667c00e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.659+0000 7f66917fa640 1 -- 172.21.15.139:0/3758309385 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c010510 con 0x7f6694157360 2024-04-25T10:02:38.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.659+0000 7f66917fa640 1 -- 172.21.15.139:0/3758309385 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c0106d0 con 0x7f6694157360 2024-04-25T10:02:38.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.659+0000 7f669afce640 1 -- 172.21.15.139:0/3758309385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6694155950 con 0x7f6694157360 2024-04-25T10:02:38.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.659+0000 7f66917fa640 1 -- 172.21.15.139:0/3758309385 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c010ac0 con 0x7f6694157360 2024-04-25T10:02:38.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.659+0000 7f669afce640 1 -- 172.21.15.139:0/3758309385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6694155f10 con 0x7f6694157360 2024-04-25T10:02:38.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.660+0000 7f669afce640 1 -- 172.21.15.139:0/3758309385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6694074520 con 0x7f6694157360 2024-04-25T10:02:38.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.660+0000 7f66917fa640 1 -- 172.21.15.139:0/3758309385 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f667c01b020 con 0x7f6694157360 2024-04-25T10:02:38.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.660+0000 7f66917fa640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f667406b500 0x7f667406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.660+0000 7f66917fa640 1 -- 172.21.15.139:0/3758309385 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f667c0898f0 con 0x7f6694157360 2024-04-25T10:02:38.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.660+0000 7f6698d6a640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f667406b500 0x7f667406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.661+0000 7f6698d6a640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f667406b500 0x7f667406d9c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f668c037000 tx=0x7f668c02f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.665+0000 7f66917fa640 1 -- 172.21.15.139:0/3758309385 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f667c058500 con 0x7f6694157360 2024-04-25T10:02:38.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.871+0000 7f669afce640 1 -- 172.21.15.139:0/3758309385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f669406fb20 con 0x7f6694157360 2024-04-25T10:02:38.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.871+0000 7f66917fa640 1 -- 172.21.15.139:0/3758309385 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f667c057e00 con 0x7f6694157360 2024-04-25T10:02:38.876 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:38.876 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8961,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2441954626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2441954626},{"type":"v1","addr":"172.21.15.139:6837","nonce":2441954626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":76,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:15.497940+0000","modified":"2024-04-25T10:02:35.776771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:02:38.877 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 77 2024-04-25T10:02:38.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.873+0000 7f669afce640 1 -- 172.21.15.139:0/3758309385 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f667406b500 msgr2=0x7f667406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.873+0000 7f669afce640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f667406b500 0x7f667406d9c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f668c037000 tx=0x7f668c02f000 comp rx=0 tx=0).stop 2024-04-25T10:02:38.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.873+0000 7f669afce640 1 -- 172.21.15.139:0/3758309385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6694157360 msgr2=0x7f66941577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.873+0000 7f669afce640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6694157360 0x7f66941577e0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f669415bb90 tx=0x7f667c00e890 comp rx=0 tx=0).stop 2024-04-25T10:02:38.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.873+0000 7f669afce640 1 -- 172.21.15.139:0/3758309385 shutdown_connections 2024-04-25T10:02:38.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.873+0000 7f669afce640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6694157360 0x7f66941577e0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.873+0000 7f669afce640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f667406b500 0x7f667406d9c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.873+0000 7f669afce640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f669415d300 0x7f669415d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.873+0000 7f669afce640 1 --2- 172.21.15.139:0/3758309385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66800039e0 0x7f6694147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.873+0000 7f669afce640 1 -- 172.21.15.139:0/3758309385 >> 172.21.15.139:0/3758309385 conn(0x7f66940b17a0 msgr2=0x7f66941530b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.873+0000 7f669afce640 1 -- 172.21.15.139:0/3758309385 shutdown_connections 2024-04-25T10:02:38.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.873+0000 7f669afce640 1 -- 172.21.15.139:0/3758309385 wait complete. 2024-04-25T10:02:38.889 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:38.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.983+0000 7fa1552e2640 1 Processor -- start 2024-04-25T10:02:38.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.983+0000 7fa1552e2640 1 -- start start 2024-04-25T10:02:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa1552e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa15015dc60 0x7fa1500b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa1552e2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa150161850 0x7fa1500ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa1552e2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1500cf250 con 0x7fa1500d0970 2024-04-25T10:02:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa1552e2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1500ce780 con 0x7fa150161850 2024-04-25T10:02:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa1552e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500d3730 con 0x7fa15015dc60 2024-04-25T10:02:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa14effd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa15015dc60 0x7fa1500b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa1467fc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa150161850 0x7fa1500ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa14effd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa15015dc60 0x7fa1500b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:59766/0 (socket says 172.21.15.139:59766) 2024-04-25T10:02:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa14effd640 1 -- 172.21.15.139:0/1982457024 learned_addr learned my addr 172.21.15.139:0/1982457024 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa1467fc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa150161850 0x7fa1500ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:48784/0 (socket says 172.21.15.139:48784) 2024-04-25T10:02:38.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa14effd640 1 -- 172.21.15.139:0/1982457024 >> v2:172.21.15.150:3300/0 conn(0x7fa150161850 msgr2=0x7fa1500ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa14effd640 1 --2- 172.21.15.139:0/1982457024 >> v2:172.21.15.150:3300/0 conn(0x7fa150161850 0x7fa1500ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa14effd640 1 -- 172.21.15.139:0/1982457024 >> v1:172.21.15.150:6789/0 conn(0x7fa1500d0970 legacy=0x7fa1500d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.984+0000 7fa14effd640 1 -- 172.21.15.139:0/1982457024 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1500ce9b0 con 0x7fa15015dc60 2024-04-25T10:02:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.985+0000 7fa14effd640 1 --2- 172.21.15.139:0/1982457024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa15015dc60 0x7fa1500b6c10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa140009870 tx=0x7fa14002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8335a520613e26e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.985+0000 7fa14e7fc640 1 -- 172.21.15.139:0/1982457024 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140037070 con 0x7fa15015dc60 2024-04-25T10:02:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.985+0000 7fa14e7fc640 1 -- 172.21.15.139:0/1982457024 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa140032440 con 0x7fa15015dc60 2024-04-25T10:02:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.985+0000 7fa14e7fc640 1 -- 172.21.15.139:0/1982457024 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140032810 con 0x7fa15015dc60 2024-04-25T10:02:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.986+0000 7fa1552e2640 1 -- 172.21.15.139:0/1982457024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa15015dc60 msgr2=0x7fa1500b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.986+0000 7fa1552e2640 1 --2- 172.21.15.139:0/1982457024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa15015dc60 0x7fa1500b6c10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa140009870 tx=0x7fa14002a270 comp rx=0 tx=0).stop 2024-04-25T10:02:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.986+0000 7fa1552e2640 1 -- 172.21.15.139:0/1982457024 shutdown_connections 2024-04-25T10:02:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.986+0000 7fa1552e2640 1 --2- 172.21.15.139:0/1982457024 >> v2:172.21.15.150:3300/0 conn(0x7fa150161850 0x7fa1500ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.986+0000 7fa1552e2640 1 --2- 172.21.15.139:0/1982457024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa15015dc60 0x7fa1500b6c10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.986+0000 7fa1552e2640 1 -- 172.21.15.139:0/1982457024 >> 172.21.15.139:0/1982457024 conn(0x7fa1500b5910 msgr2=0x7fa1500b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.986+0000 7fa1552e2640 1 -- 172.21.15.139:0/1982457024 shutdown_connections 2024-04-25T10:02:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.986+0000 7fa1552e2640 1 -- 172.21.15.139:0/1982457024 wait complete. 2024-04-25T10:02:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.987+0000 7fa1552e2640 1 Processor -- start 2024-04-25T10:02:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.987+0000 7fa1552e2640 1 -- start start 2024-04-25T10:02:38.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1552e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1500d0970 0x7fa15006c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1552e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa15015dc60 0x7fa1500cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1552e2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa150161850 0x7fa150152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1552e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500d1fc0 con 0x7fa15015dc60 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1552e2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500cef70 con 0x7fa150161850 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1552e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15015e040 con 0x7fa1500d0970 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1467fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa15015dc60 0x7fa1500cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1467fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa15015dc60 0x7fa1500cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42336/0 (socket says 172.21.15.139:42336) 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1467fc640 1 -- 172.21.15.139:0/1768871093 learned_addr learned my addr 172.21.15.139:0/1768871093 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa14effd640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1500d0970 0x7fa15006c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa14f7fe640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa150161850 0x7fa150152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1467fc640 1 -- 172.21.15.139:0/1768871093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1500d0970 msgr2=0x7fa15006c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1467fc640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1500d0970 0x7fa15006c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1467fc640 1 -- 172.21.15.139:0/1768871093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa150161850 msgr2=0x7fa150152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1467fc640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa150161850 0x7fa150152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1467fc640 1 -- 172.21.15.139:0/1768871093 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1500ccc00 con 0x7fa15015dc60 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa14effd640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1500d0970 0x7fa15006c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa1467fc640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa15015dc60 0x7fa1500cc1b0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fa13800e870 tx=0x7fa13800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa147fff640 1 -- 172.21.15.139:0/1768871093 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138017070 con 0x7fa15015dc60 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.988+0000 7fa147fff640 1 -- 172.21.15.139:0/1768871093 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13800c940 con 0x7fa15015dc60 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.989+0000 7fa1552e2640 1 -- 172.21.15.139:0/1768871093 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1500ccee0 con 0x7fa15015dc60 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.989+0000 7fa147fff640 1 -- 172.21.15.139:0/1768871093 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13800cd30 con 0x7fa15015dc60 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.989+0000 7fa1552e2640 1 -- 172.21.15.139:0/1768871093 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1500cd4a0 con 0x7fa15015dc60 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.990+0000 7fa1552e2640 1 -- 172.21.15.139:0/1768871093 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1500cd980 con 0x7fa15015dc60 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.990+0000 7fa147fff640 1 -- 172.21.15.139:0/1768871093 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa13801a070 con 0x7fa15015dc60 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.990+0000 7fa147fff640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa13006b4d0 0x7fa13006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.991+0000 7fa14effd640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa13006b4d0 0x7fa13006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.991+0000 7fa147fff640 1 -- 172.21.15.139:0/1768871093 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fa13808a5b0 con 0x7fa15015dc60 2024-04-25T10:02:38.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.991+0000 7fa14effd640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa13006b4d0 0x7fa13006d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa140009870 tx=0x7fa14002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:38.995+0000 7fa147fff640 1 -- 172.21.15.139:0/1768871093 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1500cd980 con 0x7fa15015dc60 2024-04-25T10:02:39.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.169+0000 7fa1552e2640 1 -- 172.21.15.139:0/1768871093 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa150074520 con 0x7fa15015dc60 2024-04-25T10:02:39.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.170+0000 7fa147fff640 1 -- 172.21.15.139:0/1768871093 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17291 (secure 0 0 0) 0x7fa138058c00 con 0x7fa15015dc60 2024-04-25T10:02:39.175 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:39.175 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":109,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:02:35.776788+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:13.695577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:14.460042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2a","mappings":[{"from":1,"to":7}]},{"pgid":"16.2f","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6835/359626718":"2024-04-26T10:02:35.776758+0000","172.21.15.139:6837/359626718":"2024-04-26T10:02:35.776758+0000","172.21.15.139:6837/4074693991":"2024-04-26T10:01:59.507353+0000","172.21.15.139:6835/4074693991":"2024-04-26T10:01:59.507353+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:39.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.172+0000 7fa1552e2640 1 -- 172.21.15.139:0/1768871093 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa13006b4d0 msgr2=0x7fa13006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.172+0000 7fa1552e2640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa13006b4d0 0x7fa13006d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa140009870 tx=0x7fa14002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:39.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.172+0000 7fa1552e2640 1 -- 172.21.15.139:0/1768871093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa15015dc60 msgr2=0x7fa1500cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.172+0000 7fa1552e2640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa15015dc60 0x7fa1500cc1b0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fa13800e870 tx=0x7fa13800ed30 comp rx=0 tx=0).stop 2024-04-25T10:02:39.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.172+0000 7fa1552e2640 1 -- 172.21.15.139:0/1768871093 shutdown_connections 2024-04-25T10:02:39.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.172+0000 7fa1552e2640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa15015dc60 0x7fa1500cc1b0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.172+0000 7fa1552e2640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa13006b4d0 0x7fa13006d990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.172+0000 7fa1552e2640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa150161850 0x7fa150152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.172+0000 7fa1552e2640 1 --2- 172.21.15.139:0/1768871093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1500d0970 0x7fa15006c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.172+0000 7fa1552e2640 1 -- 172.21.15.139:0/1768871093 >> 172.21.15.139:0/1768871093 conn(0x7fa1500b5910 msgr2=0x7fa1500cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.172+0000 7fa1552e2640 1 -- 172.21.15.139:0/1768871093 shutdown_connections 2024-04-25T10:02:39.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.172+0000 7fa1552e2640 1 -- 172.21.15.139:0/1768871093 wait complete. 2024-04-25T10:02:39.188 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:02:39.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.281+0000 7f0067c1f640 1 Processor -- start 2024-04-25T10:02:39.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.281+0000 7f0067c1f640 1 -- start start 2024-04-25T10:02:39.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.282+0000 7f0067c1f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f00600afc80 0x7f00600b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.282+0000 7f0067c1f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f006006b700 0x7f006006ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.282+0000 7f0067c1f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00600c6cc0 con 0x7f00600af8a0 2024-04-25T10:02:39.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.282+0000 7f0067c1f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00600c6fa0 con 0x7f00600afc80 2024-04-25T10:02:39.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.282+0000 7f0067c1f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00600c7280 con 0x7f006006b700 2024-04-25T10:02:39.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.282+0000 7f00651ba640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f00600afc80 0x7f00600b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.282+0000 7f00659bb640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f00600af8a0 0x7f00600cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37308/0 (socket says 172.21.15.139:37308) 2024-04-25T10:02:39.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.282+0000 7f00659bb640 1 -- 172.21.15.139:0/647460335 learned_addr learned my addr 172.21.15.139:0/647460335 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:39.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.283+0000 7f00651ba640 1 -- 172.21.15.139:0/647460335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f006006b700 msgr2=0x7f006006ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.283+0000 7f00651ba640 1 --2- 172.21.15.139:0/647460335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f006006b700 0x7f006006ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.283+0000 7f00651ba640 1 -- 172.21.15.139:0/647460335 >> v1:172.21.15.139:6789/0 conn(0x7f00600af8a0 legacy=0x7f00600cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.283+0000 7f00651ba640 1 -- 172.21.15.139:0/647460335 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00600cd090 con 0x7f00600afc80 2024-04-25T10:02:39.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.283+0000 7f00649b9640 1 -- 172.21.15.139:0/647460335 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541306960 0 0) 0x7f00600c6cc0 con 0x7f00600af8a0 2024-04-25T10:02:39.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.283+0000 7f00651ba640 1 --2- 172.21.15.139:0/647460335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00600afc80 0x7f00600b2d50 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f00500097a0 tx=0x7f005002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3d8b8dbe3619c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.283+0000 7f00649b9640 1 -- 172.21.15.139:0/647460335 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050037070 con 0x7f00600afc80 2024-04-25T10:02:39.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.283+0000 7f00649b9640 1 -- 172.21.15.139:0/647460335 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0050032440 con 0x7f00600afc80 2024-04-25T10:02:39.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.283+0000 7f00649b9640 1 -- 172.21.15.139:0/647460335 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050032810 con 0x7f00600afc80 2024-04-25T10:02:39.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.284+0000 7f0067c1f640 1 -- 172.21.15.139:0/647460335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00600afc80 msgr2=0x7f00600b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.284+0000 7f0067c1f640 1 --2- 172.21.15.139:0/647460335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00600afc80 0x7f00600b2d50 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f00500097a0 tx=0x7f005002a180 comp rx=0 tx=0).stop 2024-04-25T10:02:39.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.284+0000 7f0067c1f640 1 -- 172.21.15.139:0/647460335 shutdown_connections 2024-04-25T10:02:39.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.284+0000 7f0067c1f640 1 --2- 172.21.15.139:0/647460335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f006006b700 0x7f006006ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.284+0000 7f0067c1f640 1 --2- 172.21.15.139:0/647460335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00600afc80 0x7f00600b2d50 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.284+0000 7f0067c1f640 1 -- 172.21.15.139:0/647460335 >> 172.21.15.139:0/647460335 conn(0x7f00600b2230 msgr2=0x7f00600b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.284+0000 7f0067c1f640 1 -- 172.21.15.139:0/647460335 shutdown_connections 2024-04-25T10:02:39.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.284+0000 7f0067c1f640 1 -- 172.21.15.139:0/647460335 wait complete. 2024-04-25T10:02:39.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.285+0000 7f0067c1f640 1 Processor -- start 2024-04-25T10:02:39.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.285+0000 7f0067c1f640 1 -- start start 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f0067c1f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f006006b700 0x7f0060148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f0067c1f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00600af8a0 0x7f00600b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f0067c1f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00600afc80 0x7f00600b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f0067c1f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006006b220 con 0x7f00600af8a0 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f0067c1f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006006b0a0 con 0x7f00600afc80 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f0067c1f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006006b3a0 con 0x7f006006b700 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f00651ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00600af8a0 0x7f00600b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f00651ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00600af8a0 0x7f00600b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42360/0 (socket says 172.21.15.139:42360) 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f00651ba640 1 -- 172.21.15.139:0/677644891 learned_addr learned my addr 172.21.15.139:0/677644891 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f00659bb640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f006006b700 0x7f0060148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f00661bc640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00600afc80 0x7f00600b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f00651ba640 1 -- 172.21.15.139:0/677644891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f006006b700 msgr2=0x7f0060148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f00651ba640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f006006b700 0x7f0060148330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f00651ba640 1 -- 172.21.15.139:0/677644891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00600afc80 msgr2=0x7f00600b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f00651ba640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00600afc80 0x7f00600b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f00651ba640 1 -- 172.21.15.139:0/677644891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060153760 con 0x7f00600af8a0 2024-04-25T10:02:39.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f00659bb640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f006006b700 0x7f0060148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:39.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.286+0000 7f00651ba640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00600af8a0 0x7f00600b4f20 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f0050035040 tx=0x7f00500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.287+0000 7f00567fc640 1 -- 172.21.15.139:0/677644891 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050037040 con 0x7f00600af8a0 2024-04-25T10:02:39.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.287+0000 7f00567fc640 1 -- 172.21.15.139:0/677644891 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0050004250 con 0x7f00600af8a0 2024-04-25T10:02:39.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.287+0000 7f0067c1f640 1 -- 172.21.15.139:0/677644891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0060153a40 con 0x7f00600af8a0 2024-04-25T10:02:39.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.287+0000 7f00567fc640 1 -- 172.21.15.139:0/677644891 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050032e20 con 0x7f00600af8a0 2024-04-25T10:02:39.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.287+0000 7f0067c1f640 1 -- 172.21.15.139:0/677644891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0060154000 con 0x7f00600af8a0 2024-04-25T10:02:39.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.288+0000 7f0067c1f640 1 -- 172.21.15.139:0/677644891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0060074520 con 0x7f00600af8a0 2024-04-25T10:02:39.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.288+0000 7f00567fc640 1 -- 172.21.15.139:0/677644891 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0050030770 con 0x7f00600af8a0 2024-04-25T10:02:39.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.288+0000 7f00567fc640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f004406b4d0 0x7f004406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.288+0000 7f00567fc640 1 -- 172.21.15.139:0/677644891 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7f00500ab720 con 0x7f00600af8a0 2024-04-25T10:02:39.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.288+0000 7f00659bb640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f004406b4d0 0x7f004406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.289+0000 7f00659bb640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f004406b4d0 0x7f004406d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f005c007fd0 tx=0x7f005c005b70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.293+0000 7f00567fc640 1 -- 172.21.15.139:0/677644891 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f005007a450 con 0x7f00600af8a0 2024-04-25T10:02:39.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.498+0000 7f0067c1f640 1 -- 172.21.15.139:0/677644891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f00600ada30 con 0x7f00600af8a0 2024-04-25T10:02:39.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.911+0000 7f00567fc640 1 -- 172.21.15.139:0/677644891 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0050079d50 con 0x7f00600af8a0 2024-04-25T10:02:39.916 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:02:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.914+0000 7f0067c1f640 1 -- 172.21.15.139:0/677644891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f004406b4d0 msgr2=0x7f004406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.914+0000 7f0067c1f640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f004406b4d0 0x7f004406d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f005c007fd0 tx=0x7f005c005b70 comp rx=0 tx=0).stop 2024-04-25T10:02:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.914+0000 7f0067c1f640 1 -- 172.21.15.139:0/677644891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00600af8a0 msgr2=0x7f00600b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.914+0000 7f0067c1f640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00600af8a0 0x7f00600b4f20 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f0050035040 tx=0x7f00500046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.914+0000 7f0067c1f640 1 -- 172.21.15.139:0/677644891 shutdown_connections 2024-04-25T10:02:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.914+0000 7f0067c1f640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00600af8a0 0x7f00600b4f20 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.914+0000 7f0067c1f640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f004406b4d0 0x7f004406d990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.914+0000 7f0067c1f640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00600afc80 0x7f00600b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.914+0000 7f0067c1f640 1 --2- 172.21.15.139:0/677644891 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f006006b700 0x7f0060148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.914+0000 7f0067c1f640 1 -- 172.21.15.139:0/677644891 >> 172.21.15.139:0/677644891 conn(0x7f00600b2230 msgr2=0x7f006015c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.914+0000 7f0067c1f640 1 -- 172.21.15.139:0/677644891 shutdown_connections 2024-04-25T10:02:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:39.914+0000 7f0067c1f640 1 -- 172.21.15.139:0/677644891 wait complete. 2024-04-25T10:02:39.931 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:02:40.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.028+0000 7fc3d02ac640 1 Processor -- start 2024-04-25T10:02:40.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.028+0000 7fc3d02ac640 1 -- start start 2024-04-25T10:02:40.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3d02ac640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc3c80d0a80 0x7fc3c81618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3d02ac640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc3c809ad40 0x7fc3c8161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3d02ac640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c8099ca0 con 0x7fc3c80cd610 2024-04-25T10:02:40.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3d02ac640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c8099a50 con 0x7fc3c80d0a80 2024-04-25T10:02:40.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3d02ac640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c8099f80 con 0x7fc3c809ad40 2024-04-25T10:02:40.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3cd847640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc3c80d0a80 0x7fc3c81618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3cd847640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc3c80d0a80 0x7fc3c81618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42364/0 (socket says 172.21.15.139:42364) 2024-04-25T10:02:40.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3cd847640 1 -- 172.21.15.139:0/3432721432 learned_addr learned my addr 172.21.15.139:0/3432721432 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:40.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3ce849640 1 --2- 172.21.15.139:0/3432721432 >> v2:172.21.15.150:3300/0 conn(0x7fc3c809ad40 0x7fc3c8161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3cd847640 1 -- 172.21.15.139:0/3432721432 >> v2:172.21.15.150:3300/0 conn(0x7fc3c809ad40 msgr2=0x7fc3c8161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3cd847640 1 --2- 172.21.15.139:0/3432721432 >> v2:172.21.15.150:3300/0 conn(0x7fc3c809ad40 0x7fc3c8161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3cd847640 1 -- 172.21.15.139:0/3432721432 >> v1:172.21.15.150:6789/0 conn(0x7fc3c80cd610 legacy=0x7fc3c80d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3cd847640 1 -- 172.21.15.139:0/3432721432 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c809a260 con 0x7fc3c80d0a80 2024-04-25T10:02:40.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.029+0000 7fc3ce849640 1 --2- 172.21.15.139:0/3432721432 >> v2:172.21.15.150:3300/0 conn(0x7fc3c809ad40 0x7fc3c8161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:40.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.030+0000 7fc3cd847640 1 --2- 172.21.15.139:0/3432721432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3c80d0a80 0x7fc3c81618c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80c51e0 tx=0x7fc3b802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293be421110d8222 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.030+0000 7fc3cd046640 1 -- 172.21.15.139:0/3432721432 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8037070 con 0x7fc3c80d0a80 2024-04-25T10:02:40.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.030+0000 7fc3cd046640 1 -- 172.21.15.139:0/3432721432 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b8032440 con 0x7fc3c80d0a80 2024-04-25T10:02:40.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.030+0000 7fc3cd046640 1 -- 172.21.15.139:0/3432721432 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8032810 con 0x7fc3c80d0a80 2024-04-25T10:02:40.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.030+0000 7fc3d02ac640 1 -- 172.21.15.139:0/3432721432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3c80d0a80 msgr2=0x7fc3c81618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.030+0000 7fc3d02ac640 1 --2- 172.21.15.139:0/3432721432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3c80d0a80 0x7fc3c81618c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80c51e0 tx=0x7fc3b802a2c0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.030+0000 7fc3d02ac640 1 -- 172.21.15.139:0/3432721432 shutdown_connections 2024-04-25T10:02:40.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.030+0000 7fc3d02ac640 1 --2- 172.21.15.139:0/3432721432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3c80d0a80 0x7fc3c81618c0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.030+0000 7fc3d02ac640 1 --2- 172.21.15.139:0/3432721432 >> v2:172.21.15.150:3300/0 conn(0x7fc3c809ad40 0x7fc3c8161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.030+0000 7fc3d02ac640 1 -- 172.21.15.139:0/3432721432 >> 172.21.15.139:0/3432721432 conn(0x7fc3c80b31f0 msgr2=0x7fc3c80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.031+0000 7fc3d02ac640 1 -- 172.21.15.139:0/3432721432 shutdown_connections 2024-04-25T10:02:40.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.031+0000 7fc3d02ac640 1 -- 172.21.15.139:0/3432721432 wait complete. 2024-04-25T10:02:40.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.032+0000 7fc3d02ac640 1 Processor -- start 2024-04-25T10:02:40.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.032+0000 7fc3d02ac640 1 -- start start 2024-04-25T10:02:40.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.032+0000 7fc3d02ac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3c809ad40 0x7fc3c80c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.032+0000 7fc3d02ac640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3c80cd610 0x7fc3c80b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.032+0000 7fc3d02ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3c80d0a80 0x7fc3c814ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.032+0000 7fc3d02ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80cd9f0 con 0x7fc3c80d0a80 2024-04-25T10:02:40.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.032+0000 7fc3d02ac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c809a820 con 0x7fc3c809ad40 2024-04-25T10:02:40.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.032+0000 7fc3d02ac640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80cdb70 con 0x7fc3c80cd610 2024-04-25T10:02:40.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.032+0000 7fc3ce849640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3c80d0a80 0x7fc3c814ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.032+0000 7fc3ce849640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3c80d0a80 0x7fc3c814ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42380/0 (socket says 172.21.15.139:42380) 2024-04-25T10:02:40.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.032+0000 7fc3ce849640 1 -- 172.21.15.139:0/1261245636 learned_addr learned my addr 172.21.15.139:0/1261245636 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:40.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.033+0000 7fc3ce849640 1 -- 172.21.15.139:0/1261245636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3c80cd610 msgr2=0x7fc3c80b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.033+0000 7fc3ce849640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3c80cd610 0x7fc3c80b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.033+0000 7fc3ce849640 1 -- 172.21.15.139:0/1261245636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3c809ad40 msgr2=0x7fc3c80c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.033+0000 7fc3ce048640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3c809ad40 0x7fc3c80c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.033+0000 7fc3ce849640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3c809ad40 0x7fc3c80c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.033+0000 7fc3ce849640 1 -- 172.21.15.139:0/1261245636 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c80b4d00 con 0x7fc3c80d0a80 2024-04-25T10:02:40.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.033+0000 7fc3ce849640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3c80d0a80 0x7fc3c814ba10 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fc3c000cef0 tx=0x7fc3c000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.033+0000 7fc3beffd640 1 -- 172.21.15.139:0/1261245636 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c001a070 con 0x7fc3c80d0a80 2024-04-25T10:02:40.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.033+0000 7fc3beffd640 1 -- 172.21.15.139:0/1261245636 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c000d860 con 0x7fc3c80d0a80 2024-04-25T10:02:40.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.033+0000 7fc3d02ac640 1 -- 172.21.15.139:0/1261245636 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c80b4f80 con 0x7fc3c80d0a80 2024-04-25T10:02:40.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.033+0000 7fc3beffd640 1 -- 172.21.15.139:0/1261245636 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c000dc50 con 0x7fc3c80d0a80 2024-04-25T10:02:40.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.033+0000 7fc3d02ac640 1 -- 172.21.15.139:0/1261245636 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c80b5540 con 0x7fc3c80d0a80 2024-04-25T10:02:40.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.034+0000 7fc3d02ac640 1 -- 172.21.15.139:0/1261245636 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c80cd170 con 0x7fc3c80d0a80 2024-04-25T10:02:40.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.034+0000 7fc3beffd640 1 -- 172.21.15.139:0/1261245636 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3c001d070 con 0x7fc3c80d0a80 2024-04-25T10:02:40.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.035+0000 7fc3beffd640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.035+0000 7fc3beffd640 1 -- 172.21.15.139:0/1261245636 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7fc3c008acb0 con 0x7fc3c80d0a80 2024-04-25T10:02:40.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.035+0000 7fc3ce048640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.035+0000 7fc3ce048640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8066ef0 tx=0x7fc3c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.039+0000 7fc3beffd640 1 -- 172.21.15.139:0/1261245636 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3c80cd170 con 0x7fc3c80d0a80 2024-04-25T10:02:40.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.243+0000 7fc3d02ac640 1 -- 172.21.15.139:0/1261245636 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc3c8074520 con 0x7fc3c80d0a80 2024-04-25T10:02:40.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.915+0000 7fc3beffd640 1 -- 172.21.15.139:0/1261245636 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc3c80b4d00 con 0x7fc3c80d0a80 2024-04-25T10:02:40.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.918+0000 7fc3d02ac640 1 -- 172.21.15.139:0/1261245636 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc3ac06b4d0 msgr2=0x7fc3ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.918+0000 7fc3d02ac640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8066ef0 tx=0x7fc3c4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:40.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.918+0000 7fc3d02ac640 1 -- 172.21.15.139:0/1261245636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3c80d0a80 msgr2=0x7fc3c814ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.918+0000 7fc3d02ac640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3c80d0a80 0x7fc3c814ba10 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fc3c000cef0 tx=0x7fc3c000a710 comp rx=0 tx=0).stop 2024-04-25T10:02:40.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.919+0000 7fc3d02ac640 1 -- 172.21.15.139:0/1261245636 shutdown_connections 2024-04-25T10:02:40.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.919+0000 7fc3d02ac640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3c80d0a80 0x7fc3c814ba10 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.919+0000 7fc3d02ac640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.919+0000 7fc3d02ac640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3c80cd610 0x7fc3c80b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.919+0000 7fc3d02ac640 1 --2- 172.21.15.139:0/1261245636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3c809ad40 0x7fc3c80c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.919+0000 7fc3d02ac640 1 -- 172.21.15.139:0/1261245636 >> 172.21.15.139:0/1261245636 conn(0x7fc3c80b31f0 msgr2=0x7fc3c80b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.919+0000 7fc3d02ac640 1 -- 172.21.15.139:0/1261245636 shutdown_connections 2024-04-25T10:02:40.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:40.919+0000 7fc3d02ac640 1 -- 172.21.15.139:0/1261245636 wait complete. 2024-04-25T10:02:40.936 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:02:41.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.035+0000 7ffb043a9640 1 Processor -- start 2024-04-25T10:02:41.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.035+0000 7ffb043a9640 1 -- start start 2024-04-25T10:02:41.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb043a9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ffafc0cd610 0x7ffafc0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb043a9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffafc09ad40 0x7ffafc0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb043a9640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffafc099ca0 con 0x7ffafc1618c0 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb043a9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc099a50 con 0x7ffafc09ad40 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb043a9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc099f80 con 0x7ffafc0cd610 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb01944640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffafc09ad40 0x7ffafc0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb01944640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffafc09ad40 0x7ffafc0d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42392/0 (socket says 172.21.15.139:42392) 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb01944640 1 -- 172.21.15.139:0/706437777 learned_addr learned my addr 172.21.15.139:0/706437777 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb01944640 1 -- 172.21.15.139:0/706437777 >> v2:172.21.15.150:3300/0 conn(0x7ffafc0cd610 msgr2=0x7ffafc0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb02145640 1 --2- 172.21.15.139:0/706437777 >> v2:172.21.15.150:3300/0 conn(0x7ffafc0cd610 0x7ffafc0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb01944640 1 --2- 172.21.15.139:0/706437777 >> v2:172.21.15.150:3300/0 conn(0x7ffafc0cd610 0x7ffafc0d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb01944640 1 -- 172.21.15.139:0/706437777 >> v1:172.21.15.150:6789/0 conn(0x7ffafc1618c0 legacy=0x7ffafc06be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.036+0000 7ffb01944640 1 -- 172.21.15.139:0/706437777 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc09a260 con 0x7ffafc09ad40 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.037+0000 7ffb01944640 1 --2- 172.21.15.139:0/706437777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc09ad40 0x7ffafc0d08b0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7ffafc06f210 tx=0x7ffaec02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84941be99d1ae188 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.037+0000 7ffb01143640 1 -- 172.21.15.139:0/706437777 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec037070 con 0x7ffafc09ad40 2024-04-25T10:02:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.037+0000 7ffb01143640 1 -- 172.21.15.139:0/706437777 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaec032440 con 0x7ffafc09ad40 2024-04-25T10:02:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.037+0000 7ffb01143640 1 -- 172.21.15.139:0/706437777 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec032810 con 0x7ffafc09ad40 2024-04-25T10:02:41.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.037+0000 7ffb043a9640 1 -- 172.21.15.139:0/706437777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc09ad40 msgr2=0x7ffafc0d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.037+0000 7ffb043a9640 1 --2- 172.21.15.139:0/706437777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc09ad40 0x7ffafc0d08b0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7ffafc06f210 tx=0x7ffaec02a2c0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.038+0000 7ffb043a9640 1 -- 172.21.15.139:0/706437777 shutdown_connections 2024-04-25T10:02:41.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.038+0000 7ffb043a9640 1 --2- 172.21.15.139:0/706437777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc09ad40 0x7ffafc0d08b0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.038+0000 7ffb043a9640 1 --2- 172.21.15.139:0/706437777 >> v2:172.21.15.150:3300/0 conn(0x7ffafc0cd610 0x7ffafc0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.038+0000 7ffb043a9640 1 -- 172.21.15.139:0/706437777 >> 172.21.15.139:0/706437777 conn(0x7ffafc0b31f0 msgr2=0x7ffafc0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.038+0000 7ffb043a9640 1 -- 172.21.15.139:0/706437777 shutdown_connections 2024-04-25T10:02:41.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.038+0000 7ffb043a9640 1 -- 172.21.15.139:0/706437777 wait complete. 2024-04-25T10:02:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.039+0000 7ffb043a9640 1 Processor -- start 2024-04-25T10:02:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.039+0000 7ffb043a9640 1 -- start start 2024-04-25T10:02:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.039+0000 7ffb043a9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffafc09ad40 0x7ffafc1f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.039+0000 7ffb043a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc0cd610 0x7ffafc0c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.039+0000 7ffb043a9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffafc1618c0 0x7ffafc0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.039+0000 7ffb043a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0cd9f0 con 0x7ffafc0cd610 2024-04-25T10:02:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.039+0000 7ffb043a9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc09a820 con 0x7ffafc09ad40 2024-04-25T10:02:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.039+0000 7ffb043a9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0cdb70 con 0x7ffafc1618c0 2024-04-25T10:02:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb01944640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc0cd610 0x7ffafc0c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb01944640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc0cd610 0x7ffafc0c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42404/0 (socket says 172.21.15.139:42404) 2024-04-25T10:02:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb01944640 1 -- 172.21.15.139:0/3794062873 learned_addr learned my addr 172.21.15.139:0/3794062873 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb02145640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffafc09ad40 0x7ffafc1f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb02946640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffafc1618c0 0x7ffafc0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb01944640 1 -- 172.21.15.139:0/3794062873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffafc1618c0 msgr2=0x7ffafc0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb01944640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffafc1618c0 0x7ffafc0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb01944640 1 -- 172.21.15.139:0/3794062873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffafc09ad40 msgr2=0x7ffafc1f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb01944640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffafc09ad40 0x7ffafc1f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb01944640 1 -- 172.21.15.139:0/3794062873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc0b5e50 con 0x7ffafc0cd610 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb02145640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffafc09ad40 0x7ffafc1f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb01944640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc0cd610 0x7ffafc0c6db0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7ffaec000c00 tx=0x7ffaec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffaf2ffd640 1 -- 172.21.15.139:0/3794062873 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec041020 con 0x7ffafc0cd610 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffaf2ffd640 1 -- 172.21.15.139:0/3794062873 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaec030970 con 0x7ffafc0cd610 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffaf2ffd640 1 -- 172.21.15.139:0/3794062873 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec037040 con 0x7ffafc0cd610 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb043a9640 1 -- 172.21.15.139:0/3794062873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffafc0b6130 con 0x7ffafc0cd610 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.040+0000 7ffb043a9640 1 -- 172.21.15.139:0/3794062873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffafc0b66f0 con 0x7ffafc0cd610 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.041+0000 7ffb043a9640 1 -- 172.21.15.139:0/3794062873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffafc074520 con 0x7ffafc0cd610 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.042+0000 7ffaf2ffd640 1 -- 172.21.15.139:0/3794062873 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffaec044020 con 0x7ffafc0cd610 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.042+0000 7ffaf2ffd640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffae006b4d0 0x7ffae006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.042+0000 7ffaf2ffd640 1 -- 172.21.15.139:0/3794062873 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6331+0+0 (secure 0 0 0) 0x7ffaec0aae70 con 0x7ffafc0cd610 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.042+0000 7ffb02145640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffae006b4d0 0x7ffae006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.043+0000 7ffb02145640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffae006b4d0 0x7ffae006d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8008890 tx=0x7ffaf8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.047+0000 7ffaf2ffd640 1 -- 172.21.15.139:0/3794062873 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffaec04b030 con 0x7ffafc0cd610 2024-04-25T10:02:41.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.217+0000 7ffb043a9640 1 -- 172.21.15.139:0/3794062873 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffafc0b3f70 con 0x7ffafc0cd610 2024-04-25T10:02:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.920+0000 7ffaf2ffd640 1 -- 172.21.15.139:0/3794062873 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) v1 ==== 181+0+0 (secure 0 0 0) 0x7ffaec079a80 con 0x7ffafc0cd610 2024-04-25T10:02:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:02:41.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.924+0000 7ffb043a9640 1 -- 172.21.15.139:0/3794062873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffae006b4d0 msgr2=0x7ffae006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.924+0000 7ffb043a9640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffae006b4d0 0x7ffae006d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8008890 tx=0x7ffaf8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:41.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.924+0000 7ffb043a9640 1 -- 172.21.15.139:0/3794062873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc0cd610 msgr2=0x7ffafc0c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.924+0000 7ffb043a9640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc0cd610 0x7ffafc0c6db0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7ffaec000c00 tx=0x7ffaec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.924+0000 7ffb043a9640 1 -- 172.21.15.139:0/3794062873 shutdown_connections 2024-04-25T10:02:41.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.924+0000 7ffb043a9640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc0cd610 0x7ffafc0c6db0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.924+0000 7ffb043a9640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ffae006b4d0 0x7ffae006d990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.924+0000 7ffb043a9640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffafc1618c0 0x7ffafc0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.924+0000 7ffb043a9640 1 --2- 172.21.15.139:0/3794062873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffafc09ad40 0x7ffafc1f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.924+0000 7ffb043a9640 1 -- 172.21.15.139:0/3794062873 >> 172.21.15.139:0/3794062873 conn(0x7ffafc0b31f0 msgr2=0x7ffafc0b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.924+0000 7ffb043a9640 1 -- 172.21.15.139:0/3794062873 shutdown_connections 2024-04-25T10:02:41.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:41.924+0000 7ffb043a9640 1 -- 172.21.15.139:0/3794062873 wait complete. 2024-04-25T10:02:41.941 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:02:42.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.039+0000 7f584916a640 1 Processor -- start 2024-04-25T10:02:42.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.039+0000 7f584916a640 1 -- start start 2024-04-25T10:02:42.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584916a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440d0a80 0x7f58441618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584916a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5844099ca0 con 0x7f584409ad40 2024-04-25T10:02:42.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584916a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5844099f80 con 0x7f58440cd610 2024-04-25T10:02:42.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584916a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5844099a50 con 0x7f58440d0a80 2024-04-25T10:02:42.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584359e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f584409ad40 0x7f58440cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37324/0 (socket says 172.21.15.139:37324) 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584359e640 1 -- 172.21.15.139:0/4260418399 learned_addr learned my addr 172.21.15.139:0/4260418399 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256671771 0 0) 0x7f5844099ca0 con 0x7f584409ad40 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5830003520 con 0x7f584409ad40 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f583bfff640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440d0a80 0x7f58441618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1922057713 0 0) 0x7f5830003520 con 0x7f584409ad40 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440d0a80 msgr2=0x7f58441618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440d0a80 0x7f58441618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 >> v1:172.21.15.150:6789/0 conn(0x7f58440cd610 legacy=0x7f58440d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584409a260 con 0x7f584409ad40 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f583c0019f0 con 0x7f584409ad40 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 >> v1:172.21.15.139:6789/0 conn(0x7f584409ad40 legacy=0x7f58440cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300039e0 0x7f5830003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440d0a80 0x7f5830004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5830004d40 0x7f58300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5844099a50 con 0x7f58300039e0 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583000bb30 con 0x7f5830004d40 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583000bd30 con 0x7f58440d0a80 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.040+0000 7f5842d9d640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300039e0 0x7f5830003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.041+0000 7f583bfff640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440d0a80 0x7f5830004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.041+0000 7f584359e640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5830004d40 0x7f58300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.041+0000 7f5842d9d640 1 -- 172.21.15.139:0/4260418399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440d0a80 msgr2=0x7f5830004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.041+0000 7f5842d9d640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440d0a80 0x7f5830004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.041+0000 7f5842d9d640 1 -- 172.21.15.139:0/4260418399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5830004d40 msgr2=0x7f58300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.041+0000 7f5842d9d640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5830004d40 0x7f58300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.041+0000 7f5842d9d640 1 -- 172.21.15.139:0/4260418399 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5830003520 con 0x7f58300039e0 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.041+0000 7f584359e640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5830004d40 0x7f58300051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.041+0000 7f5842d9d640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300039e0 0x7f5830003e00 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f5834001720 tx=0x7f5830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7847edb19718df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.042+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834001370 con 0x7f58300039e0 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.042+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58340052d0 con 0x7f58300039e0 2024-04-25T10:02:42.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.042+0000 7f584259c640 1 -- 172.21.15.139:0/4260418399 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583402abc0 con 0x7f58300039e0 2024-04-25T10:02:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.042+0000 7f584916a640 1 -- 172.21.15.139:0/4260418399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300039e0 msgr2=0x7f5830003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.042+0000 7f584916a640 1 --2- 172.21.15.139:0/4260418399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300039e0 0x7f5830003e00 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f5834001720 tx=0x7f5830005830 comp rx=0 tx=0).stop 2024-04-25T10:02:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.042+0000 7f584359e640 1 -- 172.21.15.139:0/4260418399 reap_dead start 2024-04-25T10:02:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.042+0000 7f584916a640 1 -- 172.21.15.139:0/4260418399 shutdown_connections 2024-04-25T10:02:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.042+0000 7f584916a640 1 -- 172.21.15.139:0/4260418399 >> 172.21.15.139:0/4260418399 conn(0x7f58440b31f0 msgr2=0x7f58440b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.042+0000 7f584916a640 1 -- 172.21.15.139:0/4260418399 shutdown_connections 2024-04-25T10:02:42.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.042+0000 7f584916a640 1 -- 172.21.15.139:0/4260418399 wait complete. 2024-04-25T10:02:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.043+0000 7f584916a640 1 Processor -- start 2024-04-25T10:02:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.043+0000 7f584916a640 1 -- start start 2024-04-25T10:02:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584916a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58300039e0 0x7f58440b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584916a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440b5980 0x7f5844156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584916a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f584414d9c0 0x7f58440b6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584916a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58440cd9f0 con 0x7f584414d9c0 2024-04-25T10:02:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584916a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584409a820 con 0x7f58300039e0 2024-04-25T10:02:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584916a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58440cdb70 con 0x7f58440b5980 2024-04-25T10:02:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f584414d9c0 0x7f58440b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f584414d9c0 0x7f58440b6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42432/0 (socket says 172.21.15.139:42432) 2024-04-25T10:02:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584359e640 1 -- 172.21.15.139:0/4283754780 learned_addr learned my addr 172.21.15.139:0/4283754780 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:42.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f583bfff640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440b5980 0x7f5844156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f5842d9d640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58300039e0 0x7f58440b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584359e640 1 -- 172.21.15.139:0/4283754780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440b5980 msgr2=0x7f5844156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584359e640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440b5980 0x7f5844156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584359e640 1 -- 172.21.15.139:0/4283754780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58300039e0 msgr2=0x7f58440b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584359e640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58300039e0 0x7f58440b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f584359e640 1 -- 172.21.15.139:0/4283754780 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58440b48e0 con 0x7f584414d9c0 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.044+0000 7f583bfff640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440b5980 0x7f5844156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.045+0000 7f584359e640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f584414d9c0 0x7f58440b6910 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f58440725c0 tx=0x7f583c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.045+0000 7f583b7fe640 1 -- 172.21.15.139:0/4283754780 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c00eb30 con 0x7f584414d9c0 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.045+0000 7f583b7fe640 1 -- 172.21.15.139:0/4283754780 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c018070 con 0x7f584414d9c0 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.045+0000 7f584916a640 1 -- 172.21.15.139:0/4283754780 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58440b4bc0 con 0x7f584414d9c0 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.045+0000 7f583b7fe640 1 -- 172.21.15.139:0/4283754780 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c013420 con 0x7f584414d9c0 2024-04-25T10:02:42.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.045+0000 7f584916a640 1 -- 172.21.15.139:0/4283754780 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58440b5180 con 0x7f584414d9c0 2024-04-25T10:02:42.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.046+0000 7f584916a640 1 -- 172.21.15.139:0/4283754780 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58440b5d60 con 0x7f584414d9c0 2024-04-25T10:02:42.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.046+0000 7f583b7fe640 1 -- 172.21.15.139:0/4283754780 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f583c0135c0 con 0x7f584414d9c0 2024-04-25T10:02:42.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.047+0000 7f583b7fe640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582406b500 0x7f582406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.047+0000 7f583b7fe640 1 -- 172.21.15.139:0/4283754780 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5893+0+0 (secure 0 0 0) 0x7f583c08a580 con 0x7f584414d9c0 2024-04-25T10:02:42.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.047+0000 7f5842d9d640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582406b500 0x7f582406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.047+0000 7f5842d9d640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582406b500 0x7f582406d9c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5834032000 tx=0x7f583402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.051+0000 7f583b7fe640 1 -- 172.21.15.139:0/4283754780 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f583c0593d0 con 0x7f584414d9c0 2024-04-25T10:02:42.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.223+0000 7f584916a640 1 -- 172.21.15.139:0/4283754780 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5844074520 con 0x7f584414d9c0 2024-04-25T10:02:42.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.927+0000 7f583b7fe640 1 -- 172.21.15.139:0/4283754780 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) v1 ==== 169+0+0 (secure 0 0 0) 0x7f583c05f0d0 con 0x7f584414d9c0 2024-04-25T10:02:42.932 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' removed 2024-04-25T10:02:42.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.930+0000 7f584916a640 1 -- 172.21.15.139:0/4283754780 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582406b500 msgr2=0x7f582406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.930+0000 7f584916a640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582406b500 0x7f582406d9c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5834032000 tx=0x7f583402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:42.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.930+0000 7f584916a640 1 -- 172.21.15.139:0/4283754780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f584414d9c0 msgr2=0x7f58440b6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.930+0000 7f584916a640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f584414d9c0 0x7f58440b6910 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f58440725c0 tx=0x7f583c00e530 comp rx=0 tx=0).stop 2024-04-25T10:02:42.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.930+0000 7f584916a640 1 -- 172.21.15.139:0/4283754780 shutdown_connections 2024-04-25T10:02:42.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.930+0000 7f584916a640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f584414d9c0 0x7f58440b6910 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.930+0000 7f584916a640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f582406b500 0x7f582406d9c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.930+0000 7f584916a640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58440b5980 0x7f5844156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.930+0000 7f584916a640 1 --2- 172.21.15.139:0/4283754780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58300039e0 0x7f58440b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.930+0000 7f584916a640 1 -- 172.21.15.139:0/4283754780 >> 172.21.15.139:0/4283754780 conn(0x7f58440b31f0 msgr2=0x7f58440cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:42.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.930+0000 7f584916a640 1 -- 172.21.15.139:0/4283754780 shutdown_connections 2024-04-25T10:02:42.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:42.930+0000 7f584916a640 1 -- 172.21.15.139:0/4283754780 wait complete. 2024-04-25T10:02:42.950 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:02:43.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.048+0000 7f45f8409640 1 Processor -- start 2024-04-25T10:02:43.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.048+0000 7f45f8409640 1 -- start start 2024-04-25T10:02:43.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f8409640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f45f015dbf0 0x7f45f01617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f8409640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f45f00cee50 0x7f45f00d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f8409640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f0161dc0 con 0x7f45f00b9370 2024-04-25T10:02:43.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f8409640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f00d1ee0 con 0x7f45f00cee50 2024-04-25T10:02:43.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f8409640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f01620a0 con 0x7f45f015dbf0 2024-04-25T10:02:43.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f69a6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f45f00cee50 0x7f45f00d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f69a6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f45f00cee50 0x7f45f00d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42438/0 (socket says 172.21.15.139:42438) 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f69a6640 1 -- 172.21.15.139:0/3715199792 learned_addr learned my addr 172.21.15.139:0/3715199792 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f61a5640 1 --2- 172.21.15.139:0/3715199792 >> v2:172.21.15.150:3300/0 conn(0x7f45f015dbf0 0x7f45f01617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f69a6640 1 -- 172.21.15.139:0/3715199792 >> v2:172.21.15.150:3300/0 conn(0x7f45f015dbf0 msgr2=0x7f45f01617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f69a6640 1 --2- 172.21.15.139:0/3715199792 >> v2:172.21.15.150:3300/0 conn(0x7f45f015dbf0 0x7f45f01617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f69a6640 1 -- 172.21.15.139:0/3715199792 >> v1:172.21.15.150:6789/0 conn(0x7f45f00b9370 legacy=0x7f45f00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.049+0000 7f45f69a6640 1 -- 172.21.15.139:0/3715199792 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f0162380 con 0x7f45f00cee50 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.050+0000 7f45f69a6640 1 --2- 172.21.15.139:0/3715199792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f00cee50 0x7f45f00d2370 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f45f006d330 tx=0x7f45ec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d179062b7ed9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.050+0000 7f45f51a3640 1 -- 172.21.15.139:0/3715199792 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec037070 con 0x7f45f00cee50 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.050+0000 7f45f51a3640 1 -- 172.21.15.139:0/3715199792 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec032440 con 0x7f45f00cee50 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.050+0000 7f45f51a3640 1 -- 172.21.15.139:0/3715199792 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec032810 con 0x7f45f00cee50 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.050+0000 7f45f8409640 1 -- 172.21.15.139:0/3715199792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f00cee50 msgr2=0x7f45f00d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.050+0000 7f45f8409640 1 --2- 172.21.15.139:0/3715199792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f00cee50 0x7f45f00d2370 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f45f006d330 tx=0x7f45ec02a230 comp rx=0 tx=0).stop 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.050+0000 7f45f8409640 1 -- 172.21.15.139:0/3715199792 shutdown_connections 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.050+0000 7f45f8409640 1 --2- 172.21.15.139:0/3715199792 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f00cee50 0x7f45f00d2370 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.050+0000 7f45f8409640 1 --2- 172.21.15.139:0/3715199792 >> v2:172.21.15.150:3300/0 conn(0x7f45f015dbf0 0x7f45f01617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.050+0000 7f45f8409640 1 -- 172.21.15.139:0/3715199792 >> 172.21.15.139:0/3715199792 conn(0x7f45f00b5480 msgr2=0x7f45f00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.051+0000 7f45f8409640 1 -- 172.21.15.139:0/3715199792 shutdown_connections 2024-04-25T10:02:43.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.051+0000 7f45f8409640 1 -- 172.21.15.139:0/3715199792 wait complete. 2024-04-25T10:02:43.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.052+0000 7f45f8409640 1 Processor -- start 2024-04-25T10:02:43.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.052+0000 7f45f8409640 1 -- start start 2024-04-25T10:02:43.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.052+0000 7f45f8409640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f00b9370 0x7f45f00cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.052+0000 7f45f8409640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45f00cee50 0x7f45f00cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.052+0000 7f45f8409640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f015dbf0 0x7f45f01603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.052+0000 7f45f8409640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f015dfd0 con 0x7f45f015dbf0 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.052+0000 7f45f8409640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f00b97d0 con 0x7f45f00b9370 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.052+0000 7f45f8409640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f0161190 con 0x7f45f00cee50 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.052+0000 7f45f69a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f015dbf0 0x7f45f01603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.052+0000 7f45f69a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f015dbf0 0x7f45f01603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42452/0 (socket says 172.21.15.139:42452) 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.053+0000 7f45f69a6640 1 -- 172.21.15.139:0/3310178295 learned_addr learned my addr 172.21.15.139:0/3310178295 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.053+0000 7f45f69a6640 1 -- 172.21.15.139:0/3310178295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45f00cee50 msgr2=0x7f45f00cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.053+0000 7f45f69a6640 1 --2- 172.21.15.139:0/3310178295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45f00cee50 0x7f45f00cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.053+0000 7f45f69a6640 1 -- 172.21.15.139:0/3310178295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f00b9370 msgr2=0x7f45f00cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.053+0000 7f45f69a6640 1 --2- 172.21.15.139:0/3310178295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f00b9370 0x7f45f00cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.053+0000 7f45f69a6640 1 -- 172.21.15.139:0/3310178295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f0159ae0 con 0x7f45f015dbf0 2024-04-25T10:02:43.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.053+0000 7f45f69a6640 1 --2- 172.21.15.139:0/3310178295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f015dbf0 0x7f45f01603f0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f45ec02a270 tx=0x7f45ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.053+0000 7f45e6ffd640 1 -- 172.21.15.139:0/3310178295 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec041020 con 0x7f45f015dbf0 2024-04-25T10:02:43.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.053+0000 7f45e6ffd640 1 -- 172.21.15.139:0/3310178295 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec0306c0 con 0x7f45f015dbf0 2024-04-25T10:02:43.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.053+0000 7f45f8409640 1 -- 172.21.15.139:0/3310178295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f0159dc0 con 0x7f45f015dbf0 2024-04-25T10:02:43.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.053+0000 7f45e6ffd640 1 -- 172.21.15.139:0/3310178295 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec037040 con 0x7f45f015dbf0 2024-04-25T10:02:43.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.053+0000 7f45f8409640 1 -- 172.21.15.139:0/3310178295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f014c0b0 con 0x7f45f015dbf0 2024-04-25T10:02:43.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.054+0000 7f45f8409640 1 -- 172.21.15.139:0/3310178295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f014c2e0 con 0x7f45f015dbf0 2024-04-25T10:02:43.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.054+0000 7f45e6ffd640 1 -- 172.21.15.139:0/3310178295 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45ec044020 con 0x7f45f015dbf0 2024-04-25T10:02:43.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.055+0000 7f45e6ffd640 1 --2- 172.21.15.139:0/3310178295 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f45d406b4d0 0x7f45d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.055+0000 7f45e6ffd640 1 -- 172.21.15.139:0/3310178295 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f45ec0aa950 con 0x7f45f015dbf0 2024-04-25T10:02:43.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.055+0000 7f45f61a5640 1 --2- 172.21.15.139:0/3310178295 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f45d406b4d0 0x7f45d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.055+0000 7f45f61a5640 1 --2- 172.21.15.139:0/3310178295 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f45d406b4d0 0x7f45d406d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f45e80070b0 tx=0x7f45e8007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.059+0000 7f45e6ffd640 1 -- 172.21.15.139:0/3310178295 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45ec04b030 con 0x7f45f015dbf0 2024-04-25T10:02:43.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.230+0000 7f45f8409640 1 -- 172.21.15.139:0/3310178295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f45f0074520 con 0x7f45f015dbf0 2024-04-25T10:02:43.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.236+0000 7f45e6ffd640 1 -- 172.21.15.139:0/3310178295 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f45ec079840 con 0x7f45f015dbf0 2024-04-25T10:02:43.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.238+0000 7f45f8409640 1 -- 172.21.15.139:0/3310178295 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f45d406b4d0 msgr2=0x7f45d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.238+0000 7f45f8409640 1 --2- 172.21.15.139:0/3310178295 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f45d406b4d0 0x7f45d406d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f45e80070b0 tx=0x7f45e8007040 comp rx=0 tx=0).stop 2024-04-25T10:02:43.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.238+0000 7f45f8409640 1 -- 172.21.15.139:0/3310178295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f015dbf0 msgr2=0x7f45f01603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.238+0000 7f45f8409640 1 --2- 172.21.15.139:0/3310178295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f015dbf0 0x7f45f01603f0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f45ec02a270 tx=0x7f45ec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.238+0000 7f45f8409640 1 -- 172.21.15.139:0/3310178295 shutdown_connections 2024-04-25T10:02:43.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.238+0000 7f45f8409640 1 --2- 172.21.15.139:0/3310178295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f015dbf0 0x7f45f01603f0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.238+0000 7f45f8409640 1 --2- 172.21.15.139:0/3310178295 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f45d406b4d0 0x7f45d406d990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.238+0000 7f45f8409640 1 --2- 172.21.15.139:0/3310178295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45f00cee50 0x7f45f00cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.238+0000 7f45f8409640 1 --2- 172.21.15.139:0/3310178295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f00b9370 0x7f45f00cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.238+0000 7f45f8409640 1 -- 172.21.15.139:0/3310178295 >> 172.21.15.139:0/3310178295 conn(0x7f45f00b5480 msgr2=0x7f45f00d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.238+0000 7f45f8409640 1 -- 172.21.15.139:0/3310178295 shutdown_connections 2024-04-25T10:02:43.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.238+0000 7f45f8409640 1 -- 172.21.15.139:0/3310178295 wait complete. 2024-04-25T10:02:43.253 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T10:02:43.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.346+0000 7fd209bc8640 1 Processor -- start 2024-04-25T10:02:43.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.346+0000 7fd209bc8640 1 -- start start 2024-04-25T10:02:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd209bc8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd204160e90 0x7fd2040afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd209bc8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd20406aff0 0x7fd204070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd209bc8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd20406b480 con 0x7fd20406ab60 2024-04-25T10:02:43.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd209bc8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2040d14a0 con 0x7fd204160e90 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd209bc8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040d0400 con 0x7fd20406aff0 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd2037fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd204160e90 0x7fd2040afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd2037fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd204160e90 0x7fd2040afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42456/0 (socket says 172.21.15.139:42456) 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd2037fe640 1 -- 172.21.15.139:0/3180347575 learned_addr learned my addr 172.21.15.139:0/3180347575 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd202ffd640 1 --1- 172.21.15.139:0/3180347575 >> v1:172.21.15.139:6789/0 conn(0x7fd20406ab60 0x7fd20406fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37330/0 (socket says 172.21.15.139:37330) 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd203fff640 1 --2- 172.21.15.139:0/3180347575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd20406aff0 0x7fd204070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd2037fe640 1 -- 172.21.15.139:0/3180347575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd20406aff0 msgr2=0x7fd204070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd2037fe640 1 --2- 172.21.15.139:0/3180347575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd20406aff0 0x7fd204070180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd2037fe640 1 -- 172.21.15.139:0/3180347575 >> v1:172.21.15.139:6789/0 conn(0x7fd20406ab60 legacy=0x7fd20406fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd2037fe640 1 -- 172.21.15.139:0/3180347575 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2040d06e0 con 0x7fd204160e90 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd2027fc640 1 -- 172.21.15.139:0/3180347575 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106985195 0 0) 0x7fd20406b480 con 0x7fd20406ab60 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.347+0000 7fd203fff640 1 --2- 172.21.15.139:0/3180347575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd20406aff0 0x7fd204070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.348+0000 7fd2037fe640 1 --2- 172.21.15.139:0/3180347575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd204160e90 0x7fd2040afac0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fd1f40098a0 tx=0x7fd1f402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a76380c0514152 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.348+0000 7fd2027fc640 1 -- 172.21.15.139:0/3180347575 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4037070 con 0x7fd204160e90 2024-04-25T10:02:43.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.348+0000 7fd2027fc640 1 -- 172.21.15.139:0/3180347575 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f4032440 con 0x7fd204160e90 2024-04-25T10:02:43.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.348+0000 7fd2027fc640 1 -- 172.21.15.139:0/3180347575 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4032810 con 0x7fd204160e90 2024-04-25T10:02:43.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.348+0000 7fd209bc8640 1 -- 172.21.15.139:0/3180347575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd204160e90 msgr2=0x7fd2040afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.348+0000 7fd209bc8640 1 --2- 172.21.15.139:0/3180347575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd204160e90 0x7fd2040afac0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fd1f40098a0 tx=0x7fd1f402a290 comp rx=0 tx=0).stop 2024-04-25T10:02:43.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.349+0000 7fd209bc8640 1 -- 172.21.15.139:0/3180347575 shutdown_connections 2024-04-25T10:02:43.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.349+0000 7fd209bc8640 1 --2- 172.21.15.139:0/3180347575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd20406aff0 0x7fd204070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.349+0000 7fd209bc8640 1 --2- 172.21.15.139:0/3180347575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd204160e90 0x7fd2040afac0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.349+0000 7fd209bc8640 1 -- 172.21.15.139:0/3180347575 >> 172.21.15.139:0/3180347575 conn(0x7fd2040b23f0 msgr2=0x7fd2040b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.349+0000 7fd209bc8640 1 -- 172.21.15.139:0/3180347575 shutdown_connections 2024-04-25T10:02:43.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.349+0000 7fd209bc8640 1 -- 172.21.15.139:0/3180347575 wait complete. 2024-04-25T10:02:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.350+0000 7fd209bc8640 1 Processor -- start 2024-04-25T10:02:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.350+0000 7fd209bc8640 1 -- start start 2024-04-25T10:02:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd209bc8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd20406ab60 0x7fd204149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd209bc8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20406aff0 0x7fd2041481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd209bc8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd204160e90 0x7fd204146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd209bc8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040d0f80 con 0x7fd20406ab60 2024-04-25T10:02:43.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd209bc8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040d0ca0 con 0x7fd20406aff0 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd209bc8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040b0000 con 0x7fd204160e90 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd2037fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd20406ab60 0x7fd204149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd2037fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd20406ab60 0x7fd204149580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42458/0 (socket says 172.21.15.139:42458) 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd2037fe640 1 -- 172.21.15.139:0/4176726739 learned_addr learned my addr 172.21.15.139:0/4176726739 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd203fff640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd204160e90 0x7fd204146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd202ffd640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20406aff0 0x7fd2041481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd2037fe640 1 -- 172.21.15.139:0/4176726739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd204160e90 msgr2=0x7fd204146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd2037fe640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd204160e90 0x7fd204146900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd2037fe640 1 -- 172.21.15.139:0/4176726739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20406aff0 msgr2=0x7fd2041481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd2037fe640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20406aff0 0x7fd2041481f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd2037fe640 1 -- 172.21.15.139:0/4176726739 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd204148730 con 0x7fd20406ab60 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd202ffd640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20406aff0 0x7fd2041481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd2037fe640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd20406ab60 0x7fd204149580 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4000c00 tx=0x7fd1f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd1ebfff640 1 -- 172.21.15.139:0/4176726739 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4041020 con 0x7fd20406ab60 2024-04-25T10:02:43.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd1ebfff640 1 -- 172.21.15.139:0/4176726739 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f40306c0 con 0x7fd20406ab60 2024-04-25T10:02:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd1ebfff640 1 -- 172.21.15.139:0/4176726739 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4037040 con 0x7fd20406ab60 2024-04-25T10:02:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd209bc8640 1 -- 172.21.15.139:0/4176726739 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2041471b0 con 0x7fd20406ab60 2024-04-25T10:02:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.351+0000 7fd209bc8640 1 -- 172.21.15.139:0/4176726739 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2041479f0 con 0x7fd20406ab60 2024-04-25T10:02:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.352+0000 7fd209bc8640 1 -- 172.21.15.139:0/4176726739 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd204074520 con 0x7fd20406ab60 2024-04-25T10:02:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.353+0000 7fd1ebfff640 1 -- 172.21.15.139:0/4176726739 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1f4044020 con 0x7fd20406ab60 2024-04-25T10:02:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.353+0000 7fd1ebfff640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd1f006b4d0 0x7fd1f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.353+0000 7fd1ebfff640 1 -- 172.21.15.139:0/4176726739 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd1f40aa990 con 0x7fd20406ab60 2024-04-25T10:02:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.353+0000 7fd202ffd640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd1f006b4d0 0x7fd1f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.354+0000 7fd202ffd640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd1f006b4d0 0x7fd1f006d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd2040ab220 tx=0x7fd1f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.357+0000 7fd1ebfff640 1 -- 172.21.15.139:0/4176726739 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1f404b030 con 0x7fd20406ab60 2024-04-25T10:02:43.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.593+0000 7fd209bc8640 1 -- 172.21.15.139:0/4176726739 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7fd2040728b0 con 0x7fd20406ab60 2024-04-25T10:02:43.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.933+0000 7fd1ebfff640 1 -- 172.21.15.139:0/4176726739 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v438) v1 ==== 146+0+0 (secure 0 0 0) 0x7fd2041471b0 con 0x7fd20406ab60 2024-04-25T10:02:43.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.935+0000 7fd209bc8640 1 -- 172.21.15.139:0/4176726739 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd1f006b4d0 msgr2=0x7fd1f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.935+0000 7fd209bc8640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd1f006b4d0 0x7fd1f006d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd2040ab220 tx=0x7fd1f8009040 comp rx=0 tx=0).stop 2024-04-25T10:02:43.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.935+0000 7fd209bc8640 1 -- 172.21.15.139:0/4176726739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd20406ab60 msgr2=0x7fd204149580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.935+0000 7fd209bc8640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd20406ab60 0x7fd204149580 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4000c00 tx=0x7fd1f40046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.936+0000 7fd209bc8640 1 -- 172.21.15.139:0/4176726739 shutdown_connections 2024-04-25T10:02:43.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.936+0000 7fd209bc8640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd1f006b4d0 0x7fd1f006d990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.936+0000 7fd209bc8640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd204160e90 0x7fd204146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.936+0000 7fd209bc8640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20406aff0 0x7fd2041481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.936+0000 7fd209bc8640 1 --2- 172.21.15.139:0/4176726739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd20406ab60 0x7fd204149580 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.936+0000 7fd209bc8640 1 -- 172.21.15.139:0/4176726739 >> 172.21.15.139:0/4176726739 conn(0x7fd2040b23f0 msgr2=0x7fd204161bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.936+0000 7fd209bc8640 1 -- 172.21.15.139:0/4176726739 shutdown_connections 2024-04-25T10:02:43.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:43.936+0000 7fd209bc8640 1 -- 172.21.15.139:0/4176726739 wait complete. 2024-04-25T10:02:43.952 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:02:43.952 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:02:43.953 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:44.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.045+0000 7f614bbc1640 1 Processor -- start 2024-04-25T10:02:44.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.045+0000 7f614bbc1640 1 -- start start 2024-04-25T10:02:44.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614bbc1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f614415ae40 0x7f61440b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614bbc1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61440cefe0 0x7f614406d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614bbc1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440ce220 con 0x7f61440b0210 2024-04-25T10:02:44.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614bbc1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61440b1070 con 0x7f614415ae40 2024-04-25T10:02:44.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614bbc1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61440cdff0 con 0x7f61440cefe0 2024-04-25T10:02:44.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614915c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f614415ae40 0x7f61440b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614915c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f614415ae40 0x7f61440b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42464/0 (socket says 172.21.15.139:42464) 2024-04-25T10:02:44.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614915c640 1 -- 172.21.15.139:0/734855932 learned_addr learned my addr 172.21.15.139:0/734855932 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:44.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614a15e640 1 --2- 172.21.15.139:0/734855932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61440cefe0 0x7f614406d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614915c640 1 -- 172.21.15.139:0/734855932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61440cefe0 msgr2=0x7f614406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614915c640 1 --2- 172.21.15.139:0/734855932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61440cefe0 0x7f614406d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614915c640 1 -- 172.21.15.139:0/734855932 >> v1:172.21.15.150:6789/0 conn(0x7f61440b0210 legacy=0x7f61440b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614915c640 1 -- 172.21.15.139:0/734855932 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61440ce500 con 0x7f614415ae40 2024-04-25T10:02:44.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.046+0000 7f614a15e640 1 --2- 172.21.15.139:0/734855932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61440cefe0 0x7f614406d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:44.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.047+0000 7f614915c640 1 --2- 172.21.15.139:0/734855932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f614415ae40 0x7f61440b1960 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f613400bb10 tx=0x7f613402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c2d9b15dba2374 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.047+0000 7f614895b640 1 -- 172.21.15.139:0/734855932 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134036020 con 0x7f614415ae40 2024-04-25T10:02:44.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.047+0000 7f614895b640 1 -- 172.21.15.139:0/734855932 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6134032820 con 0x7f614415ae40 2024-04-25T10:02:44.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.047+0000 7f614895b640 1 -- 172.21.15.139:0/734855932 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134032bf0 con 0x7f614415ae40 2024-04-25T10:02:44.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.048+0000 7f614bbc1640 1 -- 172.21.15.139:0/734855932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f614415ae40 msgr2=0x7f61440b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.048+0000 7f614bbc1640 1 --2- 172.21.15.139:0/734855932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f614415ae40 0x7f61440b1960 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f613400bb10 tx=0x7f613402c920 comp rx=0 tx=0).stop 2024-04-25T10:02:44.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.048+0000 7f614bbc1640 1 -- 172.21.15.139:0/734855932 shutdown_connections 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.048+0000 7f614bbc1640 1 --2- 172.21.15.139:0/734855932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61440cefe0 0x7f614406d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.048+0000 7f614bbc1640 1 --2- 172.21.15.139:0/734855932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f614415ae40 0x7f61440b1960 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.048+0000 7f614bbc1640 1 -- 172.21.15.139:0/734855932 >> 172.21.15.139:0/734855932 conn(0x7f6144066f20 msgr2=0x7f614406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.048+0000 7f614bbc1640 1 -- 172.21.15.139:0/734855932 shutdown_connections 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.048+0000 7f614bbc1640 1 -- 172.21.15.139:0/734855932 wait complete. 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.049+0000 7f614bbc1640 1 Processor -- start 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.049+0000 7f614bbc1640 1 -- start start 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614bbc1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61440b0210 0x7f61441490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614bbc1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61440cefe0 0x7f6144149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614bbc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f614415ae40 0x7f614406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614bbc1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614415b3e0 con 0x7f61440cefe0 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614bbc1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61440b1ea0 con 0x7f614415ae40 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614bbc1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61440ceb50 con 0x7f61440b0210 2024-04-25T10:02:44.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614915c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61440cefe0 0x7f6144149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614915c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61440cefe0 0x7f6144149b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42472/0 (socket says 172.21.15.139:42472) 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614915c640 1 -- 172.21.15.139:0/3671634842 learned_addr learned my addr 172.21.15.139:0/3671634842 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614a15e640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f614415ae40 0x7f614406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614995d640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61440b0210 0x7f61441490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614915c640 1 -- 172.21.15.139:0/3671634842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61440b0210 msgr2=0x7f61441490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614915c640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61440b0210 0x7f61441490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614915c640 1 -- 172.21.15.139:0/3671634842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f614415ae40 msgr2=0x7f614406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614915c640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f614415ae40 0x7f614406ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614915c640 1 -- 172.21.15.139:0/3671634842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6144155e70 con 0x7f61440cefe0 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614a15e640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f614415ae40 0x7f614406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f614915c640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61440cefe0 0x7f6144149b70 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f613402c960 tx=0x7f6134004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.050+0000 7f613a7fc640 1 -- 172.21.15.139:0/3671634842 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134042070 con 0x7f61440cefe0 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.051+0000 7f613a7fc640 1 -- 172.21.15.139:0/3671634842 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6134031cb0 con 0x7f61440cefe0 2024-04-25T10:02:44.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.051+0000 7f613a7fc640 1 -- 172.21.15.139:0/3671634842 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134036040 con 0x7f61440cefe0 2024-04-25T10:02:44.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.051+0000 7f614bbc1640 1 -- 172.21.15.139:0/3671634842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6144156150 con 0x7f61440cefe0 2024-04-25T10:02:44.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.051+0000 7f614bbc1640 1 -- 172.21.15.139:0/3671634842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6144156710 con 0x7f61440cefe0 2024-04-25T10:02:44.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.051+0000 7f614bbc1640 1 -- 172.21.15.139:0/3671634842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6144074520 con 0x7f61440cefe0 2024-04-25T10:02:44.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.052+0000 7f613a7fc640 1 -- 172.21.15.139:0/3671634842 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6134045070 con 0x7f61440cefe0 2024-04-25T10:02:44.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.052+0000 7f613a7fc640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f612806b500 0x7f612806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.052+0000 7f614995d640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f612806b500 0x7f612806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.052+0000 7f613a7fc640 1 -- 172.21.15.139:0/3671634842 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f61340aabd0 con 0x7f61440cefe0 2024-04-25T10:02:44.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.053+0000 7f614995d640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f612806b500 0x7f612806d9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6140007fd0 tx=0x7f6140009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.056+0000 7f613a7fc640 1 -- 172.21.15.139:0/3671634842 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f613402f070 con 0x7f61440cefe0 2024-04-25T10:02:44.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.228+0000 7f614bbc1640 1 -- 172.21.15.139:0/3671634842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61440afb10 con 0x7f61440cefe0 2024-04-25T10:02:44.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.229+0000 7f613a7fc640 1 -- 172.21.15.139:0/3671634842 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+13483 (secure 0 0 0) 0x7f6134079c90 con 0x7f61440cefe0 2024-04-25T10:02:44.234 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:44.234 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":111,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:02:42.921388+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6835/359626718":"2024-04-26T10:02:35.776758+0000","172.21.15.139:6837/359626718":"2024-04-26T10:02:35.776758+0000","172.21.15.139:6837/4074693991":"2024-04-26T10:01:59.507353+0000","172.21.15.139:6835/4074693991":"2024-04-26T10:01:59.507353+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:44.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.231+0000 7f614bbc1640 1 -- 172.21.15.139:0/3671634842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f612806b500 msgr2=0x7f612806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.231+0000 7f614bbc1640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f612806b500 0x7f612806d9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6140007fd0 tx=0x7f6140009040 comp rx=0 tx=0).stop 2024-04-25T10:02:44.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.231+0000 7f614bbc1640 1 -- 172.21.15.139:0/3671634842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61440cefe0 msgr2=0x7f6144149b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.231+0000 7f614bbc1640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61440cefe0 0x7f6144149b70 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f613402c960 tx=0x7f6134004730 comp rx=0 tx=0).stop 2024-04-25T10:02:44.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.231+0000 7f614bbc1640 1 -- 172.21.15.139:0/3671634842 shutdown_connections 2024-04-25T10:02:44.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.231+0000 7f614bbc1640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61440cefe0 0x7f6144149b70 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.231+0000 7f614bbc1640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f612806b500 0x7f612806d9c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.231+0000 7f614bbc1640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f614415ae40 0x7f614406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.231+0000 7f614bbc1640 1 --2- 172.21.15.139:0/3671634842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61440b0210 0x7f61441490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.231+0000 7f614bbc1640 1 -- 172.21.15.139:0/3671634842 >> 172.21.15.139:0/3671634842 conn(0x7f6144066f20 msgr2=0x7f61440b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.231+0000 7f614bbc1640 1 -- 172.21.15.139:0/3671634842 shutdown_connections 2024-04-25T10:02:44.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.231+0000 7f614bbc1640 1 -- 172.21.15.139:0/3671634842 wait complete. 2024-04-25T10:02:44.246 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:44.247 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:44.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.341+0000 7f3ce38a1640 1 Processor -- start 2024-04-25T10:02:44.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.342+0000 7f3ce38a1640 1 -- start start 2024-04-25T10:02:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.342+0000 7f3ce38a1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3cdc15dbf0 0x7f3cdc1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.342+0000 7f3ce38a1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cdc0b9370 0x7f3cdc0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.342+0000 7f3ce38a1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cdc161dc0 con 0x7f3cdc0cec80 2024-04-25T10:02:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.342+0000 7f3ce38a1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cdc0d1ee0 con 0x7f3cdc15dbf0 2024-04-25T10:02:44.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.342+0000 7f3ce38a1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc1620a0 con 0x7f3cdc0b9370 2024-04-25T10:02:44.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.343+0000 7f3ce0e3c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cdc0b9370 0x7f3cdc0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.343+0000 7f3ce163d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3cdc15dbf0 0x7f3cdc1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.343+0000 7f3ce0e3c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cdc0b9370 0x7f3cdc0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:59850/0 (socket says 172.21.15.139:59850) 2024-04-25T10:02:44.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.343+0000 7f3ce1e3e640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f3cdc0cec80 0x7f3cdc06c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:33240/0 (socket says 172.21.15.139:33240) 2024-04-25T10:02:44.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.343+0000 7f3ce0e3c640 1 -- 172.21.15.139:0/1543195210 learned_addr learned my addr 172.21.15.139:0/1543195210 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:44.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.343+0000 7f3cd3fff640 1 -- 172.21.15.139:0/1543195210 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143066761 0 0) 0x7f3cdc161dc0 con 0x7f3cdc0cec80 2024-04-25T10:02:44.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.343+0000 7f3cd3fff640 1 -- 172.21.15.139:0/1543195210 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cc8003520 con 0x7f3cdc0cec80 2024-04-25T10:02:44.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.344+0000 7f3ce163d640 1 -- 172.21.15.139:0/1543195210 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cdc0b9370 msgr2=0x7f3cdc0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.344+0000 7f3ce163d640 1 --2- 172.21.15.139:0/1543195210 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cdc0b9370 0x7f3cdc0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.344+0000 7f3ce163d640 1 -- 172.21.15.139:0/1543195210 >> v1:172.21.15.150:6789/0 conn(0x7f3cdc0cec80 legacy=0x7f3cdc06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.344+0000 7f3ce163d640 1 -- 172.21.15.139:0/1543195210 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc162380 con 0x7f3cdc15dbf0 2024-04-25T10:02:44.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.344+0000 7f3ce0e3c640 1 --2- 172.21.15.139:0/1543195210 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cdc0b9370 0x7f3cdc0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:44.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.344+0000 7f3ce163d640 1 --2- 172.21.15.139:0/1543195210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cdc15dbf0 0x7f3cdc1617e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80097a0 tx=0x7f3cd802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eac45edf59538e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.344+0000 7f3cd3fff640 1 -- 172.21.15.139:0/1543195210 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8037070 con 0x7f3cdc15dbf0 2024-04-25T10:02:44.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.344+0000 7f3cd3fff640 1 -- 172.21.15.139:0/1543195210 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8032440 con 0x7f3cdc15dbf0 2024-04-25T10:02:44.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.344+0000 7f3cd3fff640 1 -- 172.21.15.139:0/1543195210 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd80327f0 con 0x7f3cdc15dbf0 2024-04-25T10:02:44.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.345+0000 7f3ce38a1640 1 -- 172.21.15.139:0/1543195210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cdc15dbf0 msgr2=0x7f3cdc1617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.345+0000 7f3ce38a1640 1 --2- 172.21.15.139:0/1543195210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cdc15dbf0 0x7f3cdc1617e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80097a0 tx=0x7f3cd802a180 comp rx=0 tx=0).stop 2024-04-25T10:02:44.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.345+0000 7f3ce38a1640 1 -- 172.21.15.139:0/1543195210 shutdown_connections 2024-04-25T10:02:44.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.345+0000 7f3ce38a1640 1 --2- 172.21.15.139:0/1543195210 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cdc0b9370 0x7f3cdc0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.345+0000 7f3ce38a1640 1 --2- 172.21.15.139:0/1543195210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cdc15dbf0 0x7f3cdc1617e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.345+0000 7f3ce38a1640 1 -- 172.21.15.139:0/1543195210 >> 172.21.15.139:0/1543195210 conn(0x7f3cdc0b5480 msgr2=0x7f3cdc0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.345+0000 7f3ce38a1640 1 -- 172.21.15.139:0/1543195210 shutdown_connections 2024-04-25T10:02:44.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.345+0000 7f3ce38a1640 1 -- 172.21.15.139:0/1543195210 wait complete. 2024-04-25T10:02:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.346+0000 7f3ce38a1640 1 Processor -- start 2024-04-25T10:02:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.346+0000 7f3ce38a1640 1 -- start start 2024-04-25T10:02:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce38a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cdc0b9370 0x7f3cdc1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce38a1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cdc0cec80 0x7f3cdc0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce38a1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cdc15dbf0 0x7f3cdc0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce38a1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc15dfd0 con 0x7f3cdc0b9370 2024-04-25T10:02:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce38a1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc0b97d0 con 0x7f3cdc0cec80 2024-04-25T10:02:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce38a1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc155a30 con 0x7f3cdc15dbf0 2024-04-25T10:02:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce163d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cdc0b9370 0x7f3cdc1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce163d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cdc0b9370 0x7f3cdc1554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42486/0 (socket says 172.21.15.139:42486) 2024-04-25T10:02:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce163d640 1 -- 172.21.15.139:0/2804245871 learned_addr learned my addr 172.21.15.139:0/2804245871 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:44.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce1e3e640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cdc15dbf0 0x7f3cdc0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce0e3c640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cdc0cec80 0x7f3cdc0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce163d640 1 -- 172.21.15.139:0/2804245871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cdc15dbf0 msgr2=0x7f3cdc0b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce163d640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cdc15dbf0 0x7f3cdc0b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce163d640 1 -- 172.21.15.139:0/2804245871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cdc0cec80 msgr2=0x7f3cdc0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce163d640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cdc0cec80 0x7f3cdc0cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.347+0000 7f3ce163d640 1 -- 172.21.15.139:0/2804245871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc0ccae0 con 0x7f3cdc0b9370 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.348+0000 7f3ce163d640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cdc0b9370 0x7f3cdc1554f0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80097a0 tx=0x7f3cd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.348+0000 7f3cd1ffb640 1 -- 172.21.15.139:0/2804245871 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8041070 con 0x7f3cdc0b9370 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.348+0000 7f3cd1ffb640 1 -- 172.21.15.139:0/2804245871 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd803c870 con 0x7f3cdc0b9370 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.348+0000 7f3ce38a1640 1 -- 172.21.15.139:0/2804245871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cdc0ccdc0 con 0x7f3cdc0b9370 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.348+0000 7f3cd1ffb640 1 -- 172.21.15.139:0/2804245871 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8037040 con 0x7f3cdc0b9370 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.348+0000 7f3ce38a1640 1 -- 172.21.15.139:0/2804245871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cdc0cd380 con 0x7f3cdc0b9370 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.349+0000 7f3ce38a1640 1 -- 172.21.15.139:0/2804245871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cdc074520 con 0x7f3cdc0b9370 2024-04-25T10:02:44.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.349+0000 7f3cd1ffb640 1 -- 172.21.15.139:0/2804245871 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3cd8044070 con 0x7f3cdc0b9370 2024-04-25T10:02:44.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.350+0000 7f3cd1ffb640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3cc006b4d0 0x7f3cc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.350+0000 7f3cd1ffb640 1 -- 172.21.15.139:0/2804245871 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3cd80aa9f0 con 0x7f3cdc0b9370 2024-04-25T10:02:44.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.350+0000 7f3ce0e3c640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3cc006b4d0 0x7f3cc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.350+0000 7f3ce0e3c640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3cc006b4d0 0x7f3cc006d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0b5df0 tx=0x7f3ccc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.354+0000 7f3cd1ffb640 1 -- 172.21.15.139:0/2804245871 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3cd8079a20 con 0x7f3cdc0b9370 2024-04-25T10:02:44.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.523+0000 7f3ce38a1640 1 -- 172.21.15.139:0/2804245871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3cdc0d1680 con 0x7f3cdc0b9370 2024-04-25T10:02:44.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.523+0000 7f3cd1ffb640 1 -- 172.21.15.139:0/2804245871 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3cd8079320 con 0x7f3cdc0b9370 2024-04-25T10:02:44.528 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:02:44.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.525+0000 7f3ce38a1640 1 -- 172.21.15.139:0/2804245871 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3cc006b4d0 msgr2=0x7f3cc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.525+0000 7f3ce38a1640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3cc006b4d0 0x7f3cc006d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0b5df0 tx=0x7f3ccc008040 comp rx=0 tx=0).stop 2024-04-25T10:02:44.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.525+0000 7f3ce38a1640 1 -- 172.21.15.139:0/2804245871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cdc0b9370 msgr2=0x7f3cdc1554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.525+0000 7f3ce38a1640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cdc0b9370 0x7f3cdc1554f0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80097a0 tx=0x7f3cd80046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.525+0000 7f3ce38a1640 1 -- 172.21.15.139:0/2804245871 shutdown_connections 2024-04-25T10:02:44.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.525+0000 7f3ce38a1640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3cc006b4d0 0x7f3cc006d990 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.525+0000 7f3ce38a1640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cdc15dbf0 0x7f3cdc0b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.525+0000 7f3ce38a1640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cdc0cec80 0x7f3cdc0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.525+0000 7f3ce38a1640 1 --2- 172.21.15.139:0/2804245871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cdc0b9370 0x7f3cdc1554f0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.525+0000 7f3ce38a1640 1 -- 172.21.15.139:0/2804245871 >> 172.21.15.139:0/2804245871 conn(0x7f3cdc0b5480 msgr2=0x7f3cdc0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.526+0000 7f3ce38a1640 1 -- 172.21.15.139:0/2804245871 shutdown_connections 2024-04-25T10:02:44.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.526+0000 7f3ce38a1640 1 -- 172.21.15.139:0/2804245871 wait complete. 2024-04-25T10:02:44.541 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T10:02:44.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.634+0000 7fa614f70640 1 Processor -- start 2024-04-25T10:02:44.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.634+0000 7fa614f70640 1 -- start start 2024-04-25T10:02:44.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.635+0000 7fa614f70640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa6100cf360 0x7fa6100cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.635+0000 7fa614f70640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61015d1c0 con 0x7fa61015cde0 2024-04-25T10:02:44.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.635+0000 7fa614f70640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6100cd460 con 0x7fa6100cc2a0 2024-04-25T10:02:44.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.635+0000 7fa614f70640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6100cc720 con 0x7fa6100cf360 2024-04-25T10:02:44.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.635+0000 7fa60e59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa61015cde0 0x7fa61015d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37336/0 (socket says 172.21.15.139:37336) 2024-04-25T10:02:44.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.635+0000 7fa60e59c640 1 -- 172.21.15.139:0/3417666280 learned_addr learned my addr 172.21.15.139:0/3417666280 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:44.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.635+0000 7fa60ed9d640 1 --2- 172.21.15.139:0/3417666280 >> v2:172.21.15.150:3300/0 conn(0x7fa6100cf360 0x7fa6100cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.635+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099730350 0 0) 0x7fa61015d1c0 con 0x7fa61015cde0 2024-04-25T10:02:44.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.635+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5f8003520 con 0x7fa61015cde0 2024-04-25T10:02:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.635+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 138375354 0 0) 0x7fa5f8003520 con 0x7fa61015cde0 2024-04-25T10:02:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.635+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 >> v2:172.21.15.150:3300/0 conn(0x7fa6100cf360 msgr2=0x7fa6100cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.635+0000 7fa60d59a640 1 --2- 172.21.15.139:0/3417666280 >> v2:172.21.15.150:3300/0 conn(0x7fa6100cf360 0x7fa6100cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 >> v1:172.21.15.150:6789/0 conn(0x7fa6100cc2a0 legacy=0x7fa61009a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa610099bf0 con 0x7fa61015cde0 2024-04-25T10:02:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fa600001670 con 0x7fa61015cde0 2024-04-25T10:02:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 >> v1:172.21.15.139:6789/0 conn(0x7fa61015cde0 legacy=0x7fa61015d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60d59a640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5f80039b0 0x7fa5f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60d59a640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5f8004800 0x7fa5f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60d59a640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f80051c0 0x7fa5f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f800c030 con 0x7fa5f80039b0 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f800c1e0 con 0x7fa5f80051c0 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f800c3e0 con 0x7fa5f8004800 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60e59c640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5f80039b0 0x7fa5f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60ed9d640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f80051c0 0x7fa5f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60dd9b640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5f8004800 0x7fa5f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60e59c640 1 -- 172.21.15.139:0/3417666280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5f8004800 msgr2=0x7fa5f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60e59c640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5f8004800 0x7fa5f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60e59c640 1 -- 172.21.15.139:0/3417666280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f80051c0 msgr2=0x7fa5f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60e59c640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f80051c0 0x7fa5f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60e59c640 1 -- 172.21.15.139:0/3417666280 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8003520 con 0x7fa5f80039b0 2024-04-25T10:02:44.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60ed9d640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f80051c0 0x7fa5f800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:44.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60ed9d640 1 -- 172.21.15.139:0/3417666280 reap_dead start 2024-04-25T10:02:44.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.636+0000 7fa60dd9b640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5f8004800 0x7fa5f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:44.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.637+0000 7fa60e59c640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5f80039b0 0x7fa5f8003dd0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c240 tx=0x7fa5f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d73a609518a38f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.637+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600007cf0 con 0x7fa5f80039b0 2024-04-25T10:02:44.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.637+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa600005af0 con 0x7fa5f80039b0 2024-04-25T10:02:44.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.637+0000 7fa60d59a640 1 -- 172.21.15.139:0/3417666280 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600007070 con 0x7fa5f80039b0 2024-04-25T10:02:44.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.637+0000 7fa614f70640 1 -- 172.21.15.139:0/3417666280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5f80039b0 msgr2=0x7fa5f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.637+0000 7fa614f70640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5f80039b0 0x7fa5f8003dd0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c240 tx=0x7fa5f8005830 comp rx=0 tx=0).stop 2024-04-25T10:02:44.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.637+0000 7fa614f70640 1 -- 172.21.15.139:0/3417666280 shutdown_connections 2024-04-25T10:02:44.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.637+0000 7fa614f70640 1 --2- 172.21.15.139:0/3417666280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5f80039b0 0x7fa5f8003dd0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.637+0000 7fa614f70640 1 -- 172.21.15.139:0/3417666280 >> 172.21.15.139:0/3417666280 conn(0x7fa6100d1790 msgr2=0x7fa6100d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.637+0000 7fa614f70640 1 -- 172.21.15.139:0/3417666280 shutdown_connections 2024-04-25T10:02:44.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.637+0000 7fa614f70640 1 -- 172.21.15.139:0/3417666280 wait complete. 2024-04-25T10:02:44.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.639+0000 7fa614f70640 1 Processor -- start 2024-04-25T10:02:44.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.639+0000 7fa614f70640 1 -- start start 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.639+0000 7fa614f70640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f80039b0 0x7fa610070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.639+0000 7fa614f70640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6100b5d50 0x7fa610070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.639+0000 7fa614f70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa610159860 0x7fa610159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.639+0000 7fa614f70640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61015d790 con 0x7fa610159860 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.639+0000 7fa614f70640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61015d610 con 0x7fa5f80039b0 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.639+0000 7fa614f70640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61015d910 con 0x7fa6100b5d50 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa60ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa610159860 0x7fa610159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa60ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa610159860 0x7fa610159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42510/0 (socket says 172.21.15.139:42510) 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa60ed9d640 1 -- 172.21.15.139:0/832546515 learned_addr learned my addr 172.21.15.139:0/832546515 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa60dd9b640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6100b5d50 0x7fa610070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa60e59c640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f80039b0 0x7fa610070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa60ed9d640 1 -- 172.21.15.139:0/832546515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6100b5d50 msgr2=0x7fa610070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa60ed9d640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6100b5d50 0x7fa610070ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa60ed9d640 1 -- 172.21.15.139:0/832546515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f80039b0 msgr2=0x7fa610070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa60ed9d640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f80039b0 0x7fa610070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa60ed9d640 1 -- 172.21.15.139:0/832546515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100b4cb0 con 0x7fa610159860 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa60e59c640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f80039b0 0x7fa610070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa60ed9d640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa610159860 0x7fa610159d10 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fa6100b76e0 tx=0x7fa6080115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa5feffd640 1 -- 172.21.15.139:0/832546515 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608003990 con 0x7fa610159860 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa5feffd640 1 -- 172.21.15.139:0/832546515 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60801b070 con 0x7fa610159860 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa614f70640 1 -- 172.21.15.139:0/832546515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6100b4f90 con 0x7fa610159860 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa5feffd640 1 -- 172.21.15.139:0/832546515 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608003b90 con 0x7fa610159860 2024-04-25T10:02:44.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.640+0000 7fa614f70640 1 -- 172.21.15.139:0/832546515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6100b5550 con 0x7fa610159860 2024-04-25T10:02:44.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.641+0000 7fa614f70640 1 -- 172.21.15.139:0/832546515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa610074520 con 0x7fa610159860 2024-04-25T10:02:44.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.642+0000 7fa5feffd640 1 -- 172.21.15.139:0/832546515 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa608005d10 con 0x7fa610159860 2024-04-25T10:02:44.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.642+0000 7fa5feffd640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa5f006b500 0x7fa5f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.642+0000 7fa5feffd640 1 -- 172.21.15.139:0/832546515 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa60808a570 con 0x7fa610159860 2024-04-25T10:02:44.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.642+0000 7fa60e59c640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa5f006b500 0x7fa5f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.642+0000 7fa60e59c640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa5f006b500 0x7fa5f006d9c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa600032000 tx=0x7fa600009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.646+0000 7fa5feffd640 1 -- 172.21.15.139:0/832546515 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6080595a0 con 0x7fa610159860 2024-04-25T10:02:44.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.882+0000 7fa614f70640 1 -- 172.21.15.139:0/832546515 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7fa6100d25f0 con 0x7fa610159860 2024-04-25T10:02:44.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.935+0000 7fa5feffd640 1 -- 172.21.15.139:0/832546515 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v439) v1 ==== 147+0+0 (secure 0 0 0) 0x7fa608058ea0 con 0x7fa610159860 2024-04-25T10:02:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.937+0000 7fa614f70640 1 -- 172.21.15.139:0/832546515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa5f006b500 msgr2=0x7fa5f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.937+0000 7fa614f70640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa5f006b500 0x7fa5f006d9c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa600032000 tx=0x7fa600009030 comp rx=0 tx=0).stop 2024-04-25T10:02:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.937+0000 7fa614f70640 1 -- 172.21.15.139:0/832546515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa610159860 msgr2=0x7fa610159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.937+0000 7fa614f70640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa610159860 0x7fa610159d10 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fa6100b76e0 tx=0x7fa6080115a0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.937+0000 7fa614f70640 1 -- 172.21.15.139:0/832546515 shutdown_connections 2024-04-25T10:02:44.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.937+0000 7fa614f70640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa610159860 0x7fa610159d10 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.937+0000 7fa614f70640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa5f006b500 0x7fa5f006d9c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.937+0000 7fa614f70640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6100b5d50 0x7fa610070ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.937+0000 7fa614f70640 1 --2- 172.21.15.139:0/832546515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f80039b0 0x7fa610070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.937+0000 7fa614f70640 1 -- 172.21.15.139:0/832546515 >> 172.21.15.139:0/832546515 conn(0x7fa6100d1790 msgr2=0x7fa6100d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.937+0000 7fa614f70640 1 -- 172.21.15.139:0/832546515 shutdown_connections 2024-04-25T10:02:44.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:44.937+0000 7fa614f70640 1 -- 172.21.15.139:0/832546515 wait complete. 2024-04-25T10:02:44.953 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:02:45.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.047+0000 7fb7f3adb640 1 Processor -- start 2024-04-25T10:02:45.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.048+0000 7fb7f3adb640 1 -- start start 2024-04-25T10:02:45.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.048+0000 7fb7f3adb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb7ec15dbf0 0x7fb7ec1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.048+0000 7fb7f3adb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7ec0b9370 0x7fb7ec0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.048+0000 7fb7f3adb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb7ec0cec80 0x7fb7ec06c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.048+0000 7fb7f3adb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ec0d1ee0 con 0x7fb7ec0cec80 2024-04-25T10:02:45.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.048+0000 7fb7f3adb640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ec161dc0 con 0x7fb7ec15dbf0 2024-04-25T10:02:45.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.048+0000 7fb7f3adb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec1620a0 con 0x7fb7ec0b9370 2024-04-25T10:02:45.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.048+0000 7fb7f2078640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb7ec0cec80 0x7fb7ec06c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.049+0000 7fb7f2078640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb7ec0cec80 0x7fb7ec06c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42526/0 (socket says 172.21.15.139:42526) 2024-04-25T10:02:45.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.049+0000 7fb7f2078640 1 -- 172.21.15.139:0/681967721 learned_addr learned my addr 172.21.15.139:0/681967721 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:45.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.049+0000 7fb7f1076640 1 --2- 172.21.15.139:0/681967721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7ec0b9370 0x7fb7ec0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.049+0000 7fb7f1877640 1 --2- 172.21.15.139:0/681967721 >> v2:172.21.15.150:3300/0 conn(0x7fb7ec15dbf0 0x7fb7ec1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.049+0000 7fb7f2078640 1 -- 172.21.15.139:0/681967721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7ec0b9370 msgr2=0x7fb7ec0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.049+0000 7fb7f2078640 1 --2- 172.21.15.139:0/681967721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7ec0b9370 0x7fb7ec0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.049+0000 7fb7f2078640 1 -- 172.21.15.139:0/681967721 >> v2:172.21.15.150:3300/0 conn(0x7fb7ec15dbf0 msgr2=0x7fb7ec1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.049+0000 7fb7f2078640 1 --2- 172.21.15.139:0/681967721 >> v2:172.21.15.150:3300/0 conn(0x7fb7ec15dbf0 0x7fb7ec1617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.049+0000 7fb7f2078640 1 -- 172.21.15.139:0/681967721 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec162380 con 0x7fb7ec0cec80 2024-04-25T10:02:45.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.049+0000 7fb7f2078640 1 --2- 172.21.15.139:0/681967721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7ec0cec80 0x7fb7ec06c210 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec06d330 tx=0x7fb7e402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd7a042ad5a0f63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.050+0000 7fb7f0875640 1 -- 172.21.15.139:0/681967721 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4036070 con 0x7fb7ec0cec80 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.050+0000 7fb7f0875640 1 -- 172.21.15.139:0/681967721 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e4031600 con 0x7fb7ec0cec80 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.050+0000 7fb7f0875640 1 -- 172.21.15.139:0/681967721 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40319d0 con 0x7fb7ec0cec80 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.050+0000 7fb7f3adb640 1 -- 172.21.15.139:0/681967721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7ec0cec80 msgr2=0x7fb7ec06c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.050+0000 7fb7f3adb640 1 --2- 172.21.15.139:0/681967721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7ec0cec80 0x7fb7ec06c210 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec06d330 tx=0x7fb7e402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.050+0000 7fb7f3adb640 1 -- 172.21.15.139:0/681967721 shutdown_connections 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.050+0000 7fb7f3adb640 1 --2- 172.21.15.139:0/681967721 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7ec0cec80 0x7fb7ec06c210 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.050+0000 7fb7f3adb640 1 --2- 172.21.15.139:0/681967721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7ec0b9370 0x7fb7ec0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.050+0000 7fb7f3adb640 1 --2- 172.21.15.139:0/681967721 >> v2:172.21.15.150:3300/0 conn(0x7fb7ec15dbf0 0x7fb7ec1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.050+0000 7fb7f3adb640 1 -- 172.21.15.139:0/681967721 >> 172.21.15.139:0/681967721 conn(0x7fb7ec0b5480 msgr2=0x7fb7ec0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.050+0000 7fb7f3adb640 1 -- 172.21.15.139:0/681967721 shutdown_connections 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.050+0000 7fb7f3adb640 1 -- 172.21.15.139:0/681967721 wait complete. 2024-04-25T10:02:45.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.051+0000 7fb7f3adb640 1 Processor -- start 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.051+0000 7fb7f3adb640 1 -- start start 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f3adb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7ec0b9370 0x7fb7ec160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f3adb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7ec0cec80 0x7fb7ec0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f3adb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7ec15dbf0 0x7fb7ec15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f3adb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec15dfd0 con 0x7fb7ec0b9370 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f3adb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec0b97d0 con 0x7fb7ec0cec80 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f3adb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec15fbe0 con 0x7fb7ec15dbf0 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f1877640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7ec0b9370 0x7fb7ec160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f1877640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7ec0b9370 0x7fb7ec160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42542/0 (socket says 172.21.15.139:42542) 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f1877640 1 -- 172.21.15.139:0/1663850824 learned_addr learned my addr 172.21.15.139:0/1663850824 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f2078640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7ec15dbf0 0x7fb7ec15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f1076640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7ec0cec80 0x7fb7ec0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f1877640 1 -- 172.21.15.139:0/1663850824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7ec15dbf0 msgr2=0x7fb7ec15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.052+0000 7fb7f1877640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7ec15dbf0 0x7fb7ec15b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.053+0000 7fb7f1877640 1 -- 172.21.15.139:0/1663850824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7ec0cec80 msgr2=0x7fb7ec0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.053+0000 7fb7f1877640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7ec0cec80 0x7fb7ec0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.053+0000 7fb7f1877640 1 -- 172.21.15.139:0/1663850824 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec161280 con 0x7fb7ec0b9370 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.053+0000 7fb7f1877640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7ec0b9370 0x7fb7ec160bf0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00d800 tx=0x7fb7dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.053+0000 7fb7e27fc640 1 -- 172.21.15.139:0/1663850824 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00b6c0 con 0x7fb7ec0b9370 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.053+0000 7fb7e27fc640 1 -- 172.21.15.139:0/1663850824 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7dc019070 con 0x7fb7ec0b9370 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.053+0000 7fb7f3adb640 1 -- 172.21.15.139:0/1663850824 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ec15ee20 con 0x7fb7ec0b9370 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.053+0000 7fb7e27fc640 1 -- 172.21.15.139:0/1663850824 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00ba80 con 0x7fb7ec0b9370 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.053+0000 7fb7f3adb640 1 -- 172.21.15.139:0/1663850824 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ec15f6c0 con 0x7fb7ec0b9370 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.054+0000 7fb7f3adb640 1 -- 172.21.15.139:0/1663850824 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ec074520 con 0x7fb7ec0b9370 2024-04-25T10:02:45.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.054+0000 7fb7e27fc640 1 -- 172.21.15.139:0/1663850824 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7dc00bc20 con 0x7fb7ec0b9370 2024-04-25T10:02:45.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.055+0000 7fb7e27fc640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb7d006b4d0 0x7fb7d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.055+0000 7fb7e27fc640 1 -- 172.21.15.139:0/1663850824 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb7dc08a530 con 0x7fb7ec0b9370 2024-04-25T10:02:45.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.055+0000 7fb7f1076640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb7d006b4d0 0x7fb7d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.055+0000 7fb7f1076640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb7d006b4d0 0x7fb7d006d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8006fd0 tx=0x7fb7e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.059+0000 7fb7e27fc640 1 -- 172.21.15.139:0/1663850824 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7dc01c360 con 0x7fb7ec0b9370 2024-04-25T10:02:45.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.229+0000 7fb7f3adb640 1 -- 172.21.15.139:0/1663850824 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb7ec0d1680 con 0x7fb7ec0b9370 2024-04-25T10:02:45.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.233+0000 7fb7e27fc640 1 -- 172.21.15.139:0/1663850824 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb7dc059560 con 0x7fb7ec0b9370 2024-04-25T10:02:45.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.234+0000 7fb7f3adb640 1 -- 172.21.15.139:0/1663850824 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb7d006b4d0 msgr2=0x7fb7d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.234+0000 7fb7f3adb640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb7d006b4d0 0x7fb7d006d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8006fd0 tx=0x7fb7e8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:45.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.235+0000 7fb7f3adb640 1 -- 172.21.15.139:0/1663850824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7ec0b9370 msgr2=0x7fb7ec160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.235+0000 7fb7f3adb640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7ec0b9370 0x7fb7ec160bf0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00d800 tx=0x7fb7dc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.235+0000 7fb7f3adb640 1 -- 172.21.15.139:0/1663850824 shutdown_connections 2024-04-25T10:02:45.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.235+0000 7fb7f3adb640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb7d006b4d0 0x7fb7d006d990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.235+0000 7fb7f3adb640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7ec15dbf0 0x7fb7ec15b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.235+0000 7fb7f3adb640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7ec0cec80 0x7fb7ec0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.235+0000 7fb7f3adb640 1 --2- 172.21.15.139:0/1663850824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7ec0b9370 0x7fb7ec160bf0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.235+0000 7fb7f3adb640 1 -- 172.21.15.139:0/1663850824 >> 172.21.15.139:0/1663850824 conn(0x7fb7ec0b5480 msgr2=0x7fb7ec0b6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.235+0000 7fb7f3adb640 1 -- 172.21.15.139:0/1663850824 shutdown_connections 2024-04-25T10:02:45.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.235+0000 7fb7f3adb640 1 -- 172.21.15.139:0/1663850824 wait complete. 2024-04-25T10:02:45.250 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:02:45.250 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:45.293 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:45.342 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:02:45.342 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:02:45.342 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:45.357 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:02:45.357 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T10:02:45.357 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:45.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:45.375 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:02:45.376 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:45.412 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:02:45.412 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T10:02:45.412 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:45.432 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:45.432 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:02:45.432 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:45.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.545+0000 7fc9664f5640 1 Processor -- start 2024-04-25T10:02:45.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.545+0000 7fc9664f5640 1 -- start start 2024-04-25T10:02:45.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.546+0000 7fc9664f5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc9600d1080 0x7fc9600b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.546+0000 7fc9664f5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc960161850 0x7fc9600ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.546+0000 7fc9664f5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9600cf250 con 0x7fc96015dc60 2024-04-25T10:02:45.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.546+0000 7fc9664f5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9600ce780 con 0x7fc960161850 2024-04-25T10:02:45.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.546+0000 7fc9664f5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9600d3730 con 0x7fc9600d1080 2024-04-25T10:02:45.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.546+0000 7fc964a92640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc960161850 0x7fc9600ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.546+0000 7fc95ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc96015dc60 0x7fc9600d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37344/0 (socket says 172.21.15.139:37344) 2024-04-25T10:02:45.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.546+0000 7fc95ffff640 1 -- 172.21.15.139:0/3221380088 learned_addr learned my addr 172.21.15.139:0/3221380088 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:45.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.547+0000 7fc964a92640 1 -- 172.21.15.139:0/3221380088 >> v2:172.21.15.150:3300/0 conn(0x7fc9600d1080 msgr2=0x7fc9600b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.547+0000 7fc964a92640 1 --2- 172.21.15.139:0/3221380088 >> v2:172.21.15.150:3300/0 conn(0x7fc9600d1080 0x7fc9600b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.547+0000 7fc964a92640 1 -- 172.21.15.139:0/3221380088 >> v1:172.21.15.139:6789/0 conn(0x7fc96015dc60 legacy=0x7fc9600d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.547+0000 7fc95effd640 1 -- 172.21.15.139:0/3221380088 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2478784066 0 0) 0x7fc950001110 con 0x7fc96015dc60 2024-04-25T10:02:45.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.547+0000 7fc964a92640 1 -- 172.21.15.139:0/3221380088 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9600ce9b0 con 0x7fc960161850 2024-04-25T10:02:45.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.547+0000 7fc964a92640 1 --2- 172.21.15.139:0/3221380088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc960161850 0x7fc9600ce240 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc960068750 tx=0x7fc95802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbecbb1f7632038f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.547+0000 7fc95effd640 1 -- 172.21.15.139:0/3221380088 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958007740 con 0x7fc960161850 2024-04-25T10:02:45.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.547+0000 7fc95effd640 1 -- 172.21.15.139:0/3221380088 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9580078e0 con 0x7fc960161850 2024-04-25T10:02:45.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.547+0000 7fc95effd640 1 -- 172.21.15.139:0/3221380088 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958007cb0 con 0x7fc960161850 2024-04-25T10:02:45.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.548+0000 7fc9664f5640 1 -- 172.21.15.139:0/3221380088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc960161850 msgr2=0x7fc9600ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.548+0000 7fc9664f5640 1 --2- 172.21.15.139:0/3221380088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc960161850 0x7fc9600ce240 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc960068750 tx=0x7fc95802a680 comp rx=0 tx=0).stop 2024-04-25T10:02:45.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.548+0000 7fc9664f5640 1 -- 172.21.15.139:0/3221380088 shutdown_connections 2024-04-25T10:02:45.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.548+0000 7fc9664f5640 1 --2- 172.21.15.139:0/3221380088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc960161850 0x7fc9600ce240 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.548+0000 7fc9664f5640 1 --2- 172.21.15.139:0/3221380088 >> v2:172.21.15.150:3300/0 conn(0x7fc9600d1080 0x7fc9600b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.548+0000 7fc9664f5640 1 -- 172.21.15.139:0/3221380088 >> 172.21.15.139:0/3221380088 conn(0x7fc9600b5910 msgr2=0x7fc9600b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.548+0000 7fc9664f5640 1 -- 172.21.15.139:0/3221380088 shutdown_connections 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.548+0000 7fc9664f5640 1 -- 172.21.15.139:0/3221380088 wait complete. 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.549+0000 7fc9664f5640 1 Processor -- start 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.549+0000 7fc9664f5640 1 -- start start 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc9664f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9600d1080 0x7fc96014e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc9664f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc96015dc60 0x7fc960157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc9664f5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc960161850 0x7fc9600b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc9664f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600d1fc0 con 0x7fc96015dc60 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc9664f5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600cef70 con 0x7fc960161850 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc9664f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96015e040 con 0x7fc9600d1080 2024-04-25T10:02:45.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc95f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc96015dc60 0x7fc960157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc95f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc96015dc60 0x7fc960157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42560/0 (socket says 172.21.15.139:42560) 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc95f7fe640 1 -- 172.21.15.139:0/2135126922 learned_addr learned my addr 172.21.15.139:0/2135126922 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc95ffff640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9600d1080 0x7fc96014e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc964a92640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc960161850 0x7fc9600b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc95f7fe640 1 -- 172.21.15.139:0/2135126922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9600d1080 msgr2=0x7fc96014e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc95f7fe640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9600d1080 0x7fc96014e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc95f7fe640 1 -- 172.21.15.139:0/2135126922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc960161850 msgr2=0x7fc9600b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc95f7fe640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc960161850 0x7fc9600b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.550+0000 7fc95f7fe640 1 -- 172.21.15.139:0/2135126922 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9601f4480 con 0x7fc96015dc60 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.551+0000 7fc95f7fe640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc96015dc60 0x7fc960157cd0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fc95400b410 tx=0x7fc95400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.551+0000 7fc95cff9640 1 -- 172.21.15.139:0/2135126922 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954015020 con 0x7fc96015dc60 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.551+0000 7fc95cff9640 1 -- 172.21.15.139:0/2135126922 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc954011850 con 0x7fc96015dc60 2024-04-25T10:02:45.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.551+0000 7fc9664f5640 1 -- 172.21.15.139:0/2135126922 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9600cc050 con 0x7fc96015dc60 2024-04-25T10:02:45.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.551+0000 7fc95cff9640 1 -- 172.21.15.139:0/2135126922 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954011c40 con 0x7fc96015dc60 2024-04-25T10:02:45.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.551+0000 7fc9664f5640 1 -- 172.21.15.139:0/2135126922 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9600cc610 con 0x7fc96015dc60 2024-04-25T10:02:45.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.552+0000 7fc9664f5640 1 -- 172.21.15.139:0/2135126922 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc960074520 con 0x7fc96015dc60 2024-04-25T10:02:45.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.552+0000 7fc95cff9640 1 -- 172.21.15.139:0/2135126922 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc954018020 con 0x7fc96015dc60 2024-04-25T10:02:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.552+0000 7fc95cff9640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc94c06b500 0x7fc94c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.553+0000 7fc95cff9640 1 -- 172.21.15.139:0/2135126922 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fc954089f50 con 0x7fc96015dc60 2024-04-25T10:02:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.553+0000 7fc95ffff640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc94c06b500 0x7fc94c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.553+0000 7fc95ffff640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc94c06b500 0x7fc94c06d9c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc950000f60 tx=0x7fc950007970 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.557+0000 7fc95cff9640 1 -- 172.21.15.139:0/2135126922 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc95401f020 con 0x7fc96015dc60 2024-04-25T10:02:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.764+0000 7fc9664f5640 1 -- 172.21.15.139:0/2135126922 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9600d16f0 con 0x7fc96015dc60 2024-04-25T10:02:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.765+0000 7fc95cff9640 1 -- 172.21.15.139:0/2135126922 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+3711 (secure 0 0 0) 0x7fc9540172c0 con 0x7fc96015dc60 2024-04-25T10:02:45.770 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:45.770 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":79,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2040356156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2040356156},{"type":"v1","addr":"172.21.15.150:6835","nonce":2040356156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3203930336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3203930336},{"type":"v1","addr":"172.21.15.150:6834","nonce":3203930336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/1977212328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1977212328},{"type":"v1","addr":"172.21.15.139:6836","nonce":1977212328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8961,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/2441954626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":2441954626},{"type":"v1","addr":"172.21.15.139:6837","nonce":2441954626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[]} 2024-04-25T10:02:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 79 2024-04-25T10:02:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.767+0000 7fc9664f5640 1 -- 172.21.15.139:0/2135126922 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc94c06b500 msgr2=0x7fc94c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.767+0000 7fc9664f5640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc94c06b500 0x7fc94c06d9c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc950000f60 tx=0x7fc950007970 comp rx=0 tx=0).stop 2024-04-25T10:02:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.767+0000 7fc9664f5640 1 -- 172.21.15.139:0/2135126922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc96015dc60 msgr2=0x7fc960157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.767+0000 7fc9664f5640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc96015dc60 0x7fc960157cd0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fc95400b410 tx=0x7fc95400b8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.767+0000 7fc9664f5640 1 -- 172.21.15.139:0/2135126922 shutdown_connections 2024-04-25T10:02:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.767+0000 7fc9664f5640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc96015dc60 0x7fc960157cd0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.767+0000 7fc9664f5640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc94c06b500 0x7fc94c06d9c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.767+0000 7fc9664f5640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc960161850 0x7fc9600b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.767+0000 7fc9664f5640 1 --2- 172.21.15.139:0/2135126922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9600d1080 0x7fc96014e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.767+0000 7fc9664f5640 1 -- 172.21.15.139:0/2135126922 >> 172.21.15.139:0/2135126922 conn(0x7fc9600b5910 msgr2=0x7fc960161c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.767+0000 7fc9664f5640 1 -- 172.21.15.139:0/2135126922 shutdown_connections 2024-04-25T10:02:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.767+0000 7fc9664f5640 1 -- 172.21.15.139:0/2135126922 wait complete. 2024-04-25T10:02:45.783 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:02:45.783 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:02:45.783 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:02:45.783 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:45.784 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:02:45.784 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:02:45.784 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:02:45.784 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:45.784 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:02:45.784 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:02:45.785 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:02:45.785 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:45.785 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:02:45.785 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:02:45.785 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:02:45.785 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:45.792 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:02:45.793 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:02:45.794 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:02:45.794 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:02:45.797 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:02:45.797 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:02:45.798 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:02:45.798 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:02:45.835 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:02:45.835 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:45.835 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:02:45.836 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:45.840 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:02:45.840 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:45.841 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:02:45.841 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:45.841 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:02:45.907 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T10:02:45.907 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-04-25T10:02:45.917 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2024-04-25T10:02:45.917 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-04-25T10:02:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.940+0000 7f3ea2113640 1 Processor -- start 2024-04-25T10:02:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.940+0000 7f3ea2113640 1 -- start start 2024-04-25T10:02:45.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.941+0000 7f3ea2113640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3e9c1617e0 0x7f3e9c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.941+0000 7f3ea2113640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3e9c0cec80 0x7f3e9c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.941+0000 7f3ea2113640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c161dc0 con 0x7f3e9c15dbf0 2024-04-25T10:02:45.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.941+0000 7f3ea2113640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c0cf0d0 con 0x7f3e9c1617e0 2024-04-25T10:02:45.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.941+0000 7f3ea2113640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c0d1ee0 con 0x7f3e9c0cec80 2024-04-25T10:02:45.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.941+0000 7f3e9b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3e9c15dbf0 0x7f3e9c0b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37356/0 (socket says 172.21.15.139:37356) 2024-04-25T10:02:45.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.941+0000 7f3e9b7fe640 1 -- 172.21.15.139:0/3947665337 learned_addr learned my addr 172.21.15.139:0/3947665337 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:45.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.941+0000 7f3e9affd640 1 --2- 172.21.15.139:0/3947665337 >> v2:172.21.15.139:3300/0 conn(0x7f3e9c1617e0 0x7f3e9c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.941+0000 7f3e9bfff640 1 --2- 172.21.15.139:0/3947665337 >> v2:172.21.15.150:3300/0 conn(0x7f3e9c0cec80 0x7f3e9c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.942+0000 7f3e9affd640 1 -- 172.21.15.139:0/3947665337 >> v2:172.21.15.150:3300/0 conn(0x7f3e9c0cec80 msgr2=0x7f3e9c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.942+0000 7f3e9affd640 1 --2- 172.21.15.139:0/3947665337 >> v2:172.21.15.150:3300/0 conn(0x7f3e9c0cec80 0x7f3e9c0d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.942+0000 7f3e9affd640 1 -- 172.21.15.139:0/3947665337 >> v1:172.21.15.139:6789/0 conn(0x7f3e9c15dbf0 legacy=0x7f3e9c0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.942+0000 7f3e9affd640 1 -- 172.21.15.139:0/3947665337 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c1620a0 con 0x7f3e9c1617e0 2024-04-25T10:02:45.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.943+0000 7f3e9affd640 1 --2- 172.21.15.139:0/3947665337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c1617e0 0x7f3e9c0ce740 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c08ce00 tx=0x7f3e9002a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ecb4a65edf30f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.943+0000 7f3e9a7fc640 1 -- 172.21.15.139:0/3947665337 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90037070 con 0x7f3e9c1617e0 2024-04-25T10:02:45.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.943+0000 7f3e9a7fc640 1 -- 172.21.15.139:0/3947665337 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e90032440 con 0x7f3e9c1617e0 2024-04-25T10:02:45.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.943+0000 7f3e9a7fc640 1 -- 172.21.15.139:0/3947665337 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90032810 con 0x7f3e9c1617e0 2024-04-25T10:02:45.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.944+0000 7f3ea2113640 1 -- 172.21.15.139:0/3947665337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c1617e0 msgr2=0x7f3e9c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.944+0000 7f3ea2113640 1 --2- 172.21.15.139:0/3947665337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c1617e0 0x7f3e9c0ce740 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c08ce00 tx=0x7f3e9002a370 comp rx=0 tx=0).stop 2024-04-25T10:02:45.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.944+0000 7f3ea2113640 1 -- 172.21.15.139:0/3947665337 shutdown_connections 2024-04-25T10:02:45.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.944+0000 7f3ea2113640 1 --2- 172.21.15.139:0/3947665337 >> v2:172.21.15.150:3300/0 conn(0x7f3e9c0cec80 0x7f3e9c0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.944+0000 7f3ea2113640 1 --2- 172.21.15.139:0/3947665337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c1617e0 0x7f3e9c0ce740 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.944+0000 7f3ea2113640 1 -- 172.21.15.139:0/3947665337 >> 172.21.15.139:0/3947665337 conn(0x7f3e9c0b5480 msgr2=0x7f3e9c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.944+0000 7f3ea2113640 1 -- 172.21.15.139:0/3947665337 shutdown_connections 2024-04-25T10:02:45.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.944+0000 7f3ea2113640 1 -- 172.21.15.139:0/3947665337 wait complete. 2024-04-25T10:02:45.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.945+0000 7f3ea2113640 1 Processor -- start 2024-04-25T10:02:45.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.945+0000 7f3ea2113640 1 -- start start 2024-04-25T10:02:45.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3ea2113640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e9c0cec80 0x7f3e9c154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3ea2113640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c15dbf0 0x7f3e9c15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3ea2113640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e9c15f230 0x7f3e9c0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3ea2113640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c15dfd0 con 0x7f3e9c15dbf0 2024-04-25T10:02:45.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3ea2113640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c162660 con 0x7f3e9c0cec80 2024-04-25T10:02:45.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3ea2113640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c0c9dc0 con 0x7f3e9c15f230 2024-04-25T10:02:45.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3e9affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c15dbf0 0x7f3e9c15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3e9affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c15dbf0 0x7f3e9c15ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42620/0 (socket says 172.21.15.139:42620) 2024-04-25T10:02:45.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3e9affd640 1 -- 172.21.15.139:0/748220879 learned_addr learned my addr 172.21.15.139:0/748220879 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:45.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3e9b7fe640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e9c0cec80 0x7f3e9c154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3e9bfff640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e9c15f230 0x7f3e9c0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3e9b7fe640 1 -- 172.21.15.139:0/748220879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e9c15f230 msgr2=0x7f3e9c0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3e9b7fe640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e9c15f230 0x7f3e9c0ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3e9b7fe640 1 -- 172.21.15.139:0/748220879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c15dbf0 msgr2=0x7f3e9c15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.946+0000 7f3e9b7fe640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c15dbf0 0x7f3e9c15ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.947+0000 7f3e9b7fe640 1 -- 172.21.15.139:0/748220879 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c15f860 con 0x7f3e9c0cec80 2024-04-25T10:02:45.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.947+0000 7f3e9affd640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c15dbf0 0x7f3e9c15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:45.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.947+0000 7f3e9bfff640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e9c15f230 0x7f3e9c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:45.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.947+0000 7f3e9b7fe640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e9c0cec80 0x7f3e9c154d90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c000f60 tx=0x7f3e8c00c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.949+0000 7f3e83fff640 1 -- 172.21.15.139:0/748220879 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c017070 con 0x7f3e9c0cec80 2024-04-25T10:02:45.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.949+0000 7f3e83fff640 1 -- 172.21.15.139:0/748220879 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c005ce0 con 0x7f3e9c0cec80 2024-04-25T10:02:45.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.949+0000 7f3ea2113640 1 -- 172.21.15.139:0/748220879 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c15fb40 con 0x7f3e9c0cec80 2024-04-25T10:02:45.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.950+0000 7f3e83fff640 1 -- 172.21.15.139:0/748220879 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c007310 con 0x7f3e9c0cec80 2024-04-25T10:02:45.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.950+0000 7f3ea2113640 1 -- 172.21.15.139:0/748220879 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c160100 con 0x7f3e9c0cec80 2024-04-25T10:02:45.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.950+0000 7f3e83fff640 1 -- 172.21.15.139:0/748220879 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e8c01a070 con 0x7f3e9c0cec80 2024-04-25T10:02:45.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.951+0000 7f3e83fff640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e8806b4d0 0x7f3e8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.951+0000 7f3e83fff640 1 -- 172.21.15.139:0/748220879 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3e8c0897c0 con 0x7f3e9c0cec80 2024-04-25T10:02:45.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.951+0000 7f3e9affd640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e8806b4d0 0x7f3e8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.951+0000 7f3e9affd640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e8806b4d0 0x7f3e8806d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3e9002a3b0 tx=0x7f3e9002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.952+0000 7f3ea2113640 1 -- 172.21.15.139:0/748220879 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c074520 con 0x7f3e9c0cec80 2024-04-25T10:02:45.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:45.956+0000 7f3e83fff640 1 -- 172.21.15.139:0/748220879 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e8c058070 con 0x7f3e9c0cec80 2024-04-25T10:02:46.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.139+0000 7f3ea2113640 1 -- 172.21.15.139:0/748220879 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3e9c152d10 con 0x7f3e9c0cec80 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.953+0000 7f3e83fff640 1 -- 172.21.15.139:0/748220879 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v116) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3e8c05c110 con 0x7f3e9c0cec80 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi139.stderr: removed all blocklist entries 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.956+0000 7f3ea2113640 1 -- 172.21.15.139:0/748220879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e8806b4d0 msgr2=0x7f3e8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.956+0000 7f3ea2113640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e8806b4d0 0x7f3e8806d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3e9002a3b0 tx=0x7f3e9002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.956+0000 7f3ea2113640 1 -- 172.21.15.139:0/748220879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e9c0cec80 msgr2=0x7f3e9c154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.956+0000 7f3ea2113640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e9c0cec80 0x7f3e9c154d90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c000f60 tx=0x7f3e8c00c760 comp rx=0 tx=0).stop 2024-04-25T10:02:46.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.956+0000 7f3ea2113640 1 -- 172.21.15.139:0/748220879 shutdown_connections 2024-04-25T10:02:46.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.956+0000 7f3ea2113640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e9c15dbf0 0x7f3e9c15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.956+0000 7f3ea2113640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e8806b4d0 0x7f3e8806d990 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.956+0000 7f3ea2113640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e9c15f230 0x7f3e9c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.956+0000 7f3ea2113640 1 --2- 172.21.15.139:0/748220879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e9c0cec80 0x7f3e9c154d90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.956+0000 7f3ea2113640 1 -- 172.21.15.139:0/748220879 >> 172.21.15.139:0/748220879 conn(0x7f3e9c0b5480 msgr2=0x7f3e9c158670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.957+0000 7f3ea2113640 1 -- 172.21.15.139:0/748220879 shutdown_connections 2024-04-25T10:02:46.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:46.957+0000 7f3ea2113640 1 -- 172.21.15.139:0/748220879 wait complete. 2024-04-25T10:02:46.975 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:02:47.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.073+0000 7ff094a15640 1 Processor -- start 2024-04-25T10:02:47.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.073+0000 7ff094a15640 1 -- start start 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff094a15640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff09015efb0 0x7ff0900d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff094a15640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff0900cf900 0x7ff09006c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff094a15640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0900cfe70 con 0x7ff09006cd80 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff094a15640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0900d0a80 con 0x7ff0900cf900 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff094a15640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900d0150 con 0x7ff09015efb0 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff08ed9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff09006cd80 0x7ff090099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35460/0 (socket says 172.21.15.139:35460) 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff08ed9d640 1 -- 172.21.15.139:0/3618984130 learned_addr learned my addr 172.21.15.139:0/3618984130 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff087fff640 1 --2- 172.21.15.139:0/3618984130 >> v2:172.21.15.150:3300/0 conn(0x7ff0900cf900 0x7ff09006c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff08e59c640 1 --2- 172.21.15.139:0/3618984130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff09015efb0 0x7ff0900d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff08dd9b640 1 -- 172.21.15.139:0/3618984130 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029244393 0 0) 0x7ff0900cfe70 con 0x7ff09006cd80 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff08dd9b640 1 -- 172.21.15.139:0/3618984130 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff07c003520 con 0x7ff09006cd80 2024-04-25T10:02:47.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff087fff640 1 -- 172.21.15.139:0/3618984130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff09015efb0 msgr2=0x7ff0900d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff087fff640 1 --2- 172.21.15.139:0/3618984130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff09015efb0 0x7ff0900d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff087fff640 1 -- 172.21.15.139:0/3618984130 >> v1:172.21.15.139:6789/0 conn(0x7ff09006cd80 legacy=0x7ff090099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff08dd9b640 1 -- 172.21.15.139:0/3618984130 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2491424891 0 0) 0x7ff07c003520 con 0x7ff09006cd80 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff087fff640 1 -- 172.21.15.139:0/3618984130 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0900d0430 con 0x7ff0900cf900 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.074+0000 7ff08e59c640 1 --2- 172.21.15.139:0/3618984130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff09015efb0 0x7ff0900d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.075+0000 7ff087fff640 1 --2- 172.21.15.139:0/3618984130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0900cf900 0x7ff09006c840 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff0780097a0 tx=0x7ff07802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d927d0c4283f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.075+0000 7ff08dd9b640 1 -- 172.21.15.139:0/3618984130 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078037070 con 0x7ff0900cf900 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.075+0000 7ff08dd9b640 1 -- 172.21.15.139:0/3618984130 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07802ab10 con 0x7ff0900cf900 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.075+0000 7ff08dd9b640 1 -- 172.21.15.139:0/3618984130 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0880019f0 con 0x7ff0900cf900 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.076+0000 7ff094a15640 1 -- 172.21.15.139:0/3618984130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0900cf900 msgr2=0x7ff09006c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.076+0000 7ff094a15640 1 --2- 172.21.15.139:0/3618984130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0900cf900 0x7ff09006c840 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff0780097a0 tx=0x7ff07802a080 comp rx=0 tx=0).stop 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.076+0000 7ff094a15640 1 -- 172.21.15.139:0/3618984130 shutdown_connections 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.076+0000 7ff094a15640 1 --2- 172.21.15.139:0/3618984130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0900cf900 0x7ff09006c840 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.076+0000 7ff094a15640 1 --2- 172.21.15.139:0/3618984130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff09015efb0 0x7ff0900d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.076+0000 7ff094a15640 1 -- 172.21.15.139:0/3618984130 >> 172.21.15.139:0/3618984130 conn(0x7ff0900d1790 msgr2=0x7ff0900d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.076+0000 7ff094a15640 1 -- 172.21.15.139:0/3618984130 shutdown_connections 2024-04-25T10:02:47.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.076+0000 7ff094a15640 1 -- 172.21.15.139:0/3618984130 wait complete. 2024-04-25T10:02:47.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.077+0000 7ff094a15640 1 Processor -- start 2024-04-25T10:02:47.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.077+0000 7ff094a15640 1 -- start start 2024-04-25T10:02:47.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff094a15640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff09006cd80 0x7ff09014dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff094a15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0900cf900 0x7ff090159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff094a15640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff09015efb0 0x7ff090161200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff094a15640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09009a2e0 con 0x7ff0900cf900 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff094a15640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09009a160 con 0x7ff09015efb0 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff094a15640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900cbde0 con 0x7ff09006cd80 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff087fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0900cf900 0x7ff090159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff087fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0900cf900 0x7ff090159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53930/0 (socket says 172.21.15.139:53930) 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff087fff640 1 -- 172.21.15.139:0/3393627548 learned_addr learned my addr 172.21.15.139:0/3393627548 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff08ed9d640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff09015efb0 0x7ff090161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff08e59c640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff09006cd80 0x7ff09014dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff087fff640 1 -- 172.21.15.139:0/3393627548 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff09006cd80 msgr2=0x7ff09014dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff087fff640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff09006cd80 0x7ff09014dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff087fff640 1 -- 172.21.15.139:0/3393627548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff09015efb0 msgr2=0x7ff090161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff087fff640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff09015efb0 0x7ff090161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.078+0000 7ff087fff640 1 -- 172.21.15.139:0/3393627548 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0900b45e0 con 0x7ff0900cf900 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.079+0000 7ff087fff640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0900cf900 0x7ff090159460 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff07802a670 tx=0x7ff0780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.079+0000 7ff086ffd640 1 -- 172.21.15.139:0/3393627548 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078042020 con 0x7ff0900cf900 2024-04-25T10:02:47.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.079+0000 7ff086ffd640 1 -- 172.21.15.139:0/3393627548 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff078003a40 con 0x7ff0900cf900 2024-04-25T10:02:47.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.079+0000 7ff094a15640 1 -- 172.21.15.139:0/3393627548 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0900b48c0 con 0x7ff0900cf900 2024-04-25T10:02:47.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.079+0000 7ff086ffd640 1 -- 172.21.15.139:0/3393627548 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078037040 con 0x7ff0900cf900 2024-04-25T10:02:47.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.079+0000 7ff094a15640 1 -- 172.21.15.139:0/3393627548 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0900b4e80 con 0x7ff0900cf900 2024-04-25T10:02:47.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.080+0000 7ff094a15640 1 -- 172.21.15.139:0/3393627548 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff090074520 con 0x7ff0900cf900 2024-04-25T10:02:47.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.080+0000 7ff086ffd640 1 -- 172.21.15.139:0/3393627548 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff078045020 con 0x7ff0900cf900 2024-04-25T10:02:47.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.080+0000 7ff086ffd640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff07006b500 0x7ff07006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.081+0000 7ff086ffd640 1 -- 172.21.15.139:0/3393627548 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff0780aa570 con 0x7ff0900cf900 2024-04-25T10:02:47.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.081+0000 7ff08e59c640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff07006b500 0x7ff07006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.081+0000 7ff08e59c640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff07006b500 0x7ff07006d9c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff080009950 tx=0x7ff080008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.085+0000 7ff086ffd640 1 -- 172.21.15.139:0/3393627548 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff07804c030 con 0x7ff0900cf900 2024-04-25T10:02:47.313 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:47.316 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:47.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.326+0000 7ff094a15640 1 -- 172.21.15.139:0/3393627548 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff09009ae50 con 0x7ff0900cf900 2024-04-25T10:02:47.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.326+0000 7ff086ffd640 1 -- 172.21.15.139:0/3393627548 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff078030340 con 0x7ff0900cf900 2024-04-25T10:02:47.331 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:47.331 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:02:47.331 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_dump": [ 2024-04-25T10:02:47.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.a", 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypm0xdaHhAA6nupxCTQhucepW2iEWNUoQ==", 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.b", 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmB9GDKRAA/lmU4OiKO5dFgayYx09GQg==", 2024-04-25T10:02:47.332 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.333 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:02:47.333 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T10:02:47.333 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T10:02:47.333 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.333 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.333 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.c", 2024-04-25T10:02:47.333 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmvYgaJRAA/YIE6ralp2WuTK993D7H0Q==", 2024-04-25T10:02:47.333 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.333 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mds.d", 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmg49WMBAAlPY9ZSeOoV296xSdl/m1gg==", 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.334 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.0", 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDVJypmG7TpDhAAo9CKuZBw8BsQgB2F+BR5gw==", 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.1", 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDYJypmgR62DBAAotfnvS47WDunP9Ic03gA6Q==", 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.335 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.2", 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDbJypmzyqGChAA6wrfIdqpiLm3RO36oorihQ==", 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.336 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.3", 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDeJypmt72cCBAAgcLrFC3MUU7wlFySAfLWig==", 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.4", 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDkJypmxJLpDBAAvROZ9AvYdWMqOALEKIAFCw==", 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:02:47.337 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.5", 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDnJypmTPXEChAA20BghY39PGw6zFyu9yWbBQ==", 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.6", 2024-04-25T10:02:47.338 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDqJypmaBoYCRAAy9eHgUI/WTkE2XdT1RKLsw==", 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "osd.7", 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDtJypmViJlCBAANJSDWVHVbWR03Y1SMDH++Q==", 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile osd", 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.339 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.340 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.0", 2024-04-25T10:02:47.340 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg==", 2024-04-25T10:02:47.340 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.340 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:02:47.340 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow r", 2024-04-25T10:02:47.340 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:02:47.340 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.1", 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypmOncYARAA6gYjZbRDkhqFX0mpSDbH5Q==", 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.2", 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmQs2wORAArzyV70nGDX4486LlFjM70Q==", 2024-04-25T10:02:47.341 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.3", 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDSJypm90x9BhAA41rZPs0O5mwmuHy1bkPCvw==", 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow", 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow r", 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow rw", 2024-04-25T10:02:47.342 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow rwx" 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "client.admin", 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmOWQcAxAA/ZVXEHYJQZf9UtgZ5g25Ag==", 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": "allow *", 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow *", 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.x", 2024-04-25T10:02:47.343 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmt+RVGxAAi1djVoqnoYCDMzJ9A0dmNA==", 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: "entity": "mgr.y", 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "AQDRJypmuJ4bFxAAw6a53AgThU/ut+Mk5j0gqQ==", 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": { 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": "allow *", 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: "mon": "allow profile mgr", 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": "allow *" 2024-04-25T10:02:47.345 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.345 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:47.345 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-04-25T10:02:47.345 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:02:47.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.328+0000 7ff094a15640 1 -- 172.21.15.139:0/3393627548 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff07006b500 msgr2=0x7ff07006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.328+0000 7ff094a15640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff07006b500 0x7ff07006d9c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff080009950 tx=0x7ff080008040 comp rx=0 tx=0).stop 2024-04-25T10:02:47.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.328+0000 7ff094a15640 1 -- 172.21.15.139:0/3393627548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0900cf900 msgr2=0x7ff090159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.328+0000 7ff094a15640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0900cf900 0x7ff090159460 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff07802a670 tx=0x7ff0780046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.329+0000 7ff094a15640 1 -- 172.21.15.139:0/3393627548 shutdown_connections 2024-04-25T10:02:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.329+0000 7ff094a15640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0900cf900 0x7ff090159460 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.329+0000 7ff094a15640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff07006b500 0x7ff07006d9c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.329+0000 7ff094a15640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff09015efb0 0x7ff090161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.329+0000 7ff094a15640 1 --2- 172.21.15.139:0/3393627548 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff09006cd80 0x7ff09014dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.329+0000 7ff094a15640 1 -- 172.21.15.139:0/3393627548 >> 172.21.15.139:0/3393627548 conn(0x7ff0900d1790 msgr2=0x7ff090070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.329+0000 7ff094a15640 1 -- 172.21.15.139:0/3393627548 shutdown_connections 2024-04-25T10:02:47.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.329+0000 7ff094a15640 1 -- 172.21.15.139:0/3393627548 wait complete. 2024-04-25T10:02:47.348 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:47.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.479+0000 7f147282a640 1 Processor -- start 2024-04-25T10:02:47.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.479+0000 7f147282a640 1 -- start start 2024-04-25T10:02:47.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.479+0000 7f147282a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f146c161850 0x7f146c0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.479+0000 7f147282a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146c0cf250 con 0x7f146c15dc60 2024-04-25T10:02:47.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.479+0000 7f147282a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146c0ce9b0 con 0x7f146c0d1080 2024-04-25T10:02:47.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.479+0000 7f147282a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146c0d3730 con 0x7f146c161850 2024-04-25T10:02:47.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.480+0000 7f1470dc7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f146c161850 0x7f146c0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.480+0000 7f146bfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f146c15dc60 0x7f146c0d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35468/0 (socket says 172.21.15.139:35468) 2024-04-25T10:02:47.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.480+0000 7f146bfff640 1 -- 172.21.15.139:0/2873287963 learned_addr learned my addr 172.21.15.139:0/2873287963 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:47.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.480+0000 7f1470dc7640 1 -- 172.21.15.139:0/2873287963 >> v1:172.21.15.150:6789/0 conn(0x7f146c0d1080 legacy=0x7f146c0d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.480+0000 7f1470dc7640 1 -- 172.21.15.139:0/2873287963 >> v1:172.21.15.139:6789/0 conn(0x7f146c15dc60 legacy=0x7f146c0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.480+0000 7f146affd640 1 -- 172.21.15.139:0/2873287963 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271154655 0 0) 0x7f145c001110 con 0x7f146c15dc60 2024-04-25T10:02:47.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.480+0000 7f1470dc7640 1 -- 172.21.15.139:0/2873287963 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0cec90 con 0x7f146c161850 2024-04-25T10:02:47.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.481+0000 7f1470dc7640 1 --2- 172.21.15.139:0/2873287963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f146c161850 0x7f146c0b6c10 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f1464009d90 tx=0x7f146402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae22d36ea8456940 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.481+0000 7f146affd640 1 -- 172.21.15.139:0/2873287963 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1464007740 con 0x7f146c161850 2024-04-25T10:02:47.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.481+0000 7f146affd640 1 -- 172.21.15.139:0/2873287963 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14640078e0 con 0x7f146c161850 2024-04-25T10:02:47.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.481+0000 7f146affd640 1 -- 172.21.15.139:0/2873287963 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1464007cb0 con 0x7f146c161850 2024-04-25T10:02:47.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.481+0000 7f147282a640 1 -- 172.21.15.139:0/2873287963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f146c161850 msgr2=0x7f146c0b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.481+0000 7f147282a640 1 --2- 172.21.15.139:0/2873287963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f146c161850 0x7f146c0b6c10 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f1464009d90 tx=0x7f146402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.481+0000 7f147282a640 1 -- 172.21.15.139:0/2873287963 shutdown_connections 2024-04-25T10:02:47.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.481+0000 7f147282a640 1 --2- 172.21.15.139:0/2873287963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f146c161850 0x7f146c0b6c10 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.481+0000 7f147282a640 1 -- 172.21.15.139:0/2873287963 >> 172.21.15.139:0/2873287963 conn(0x7f146c0b5910 msgr2=0x7f146c0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.481+0000 7f147282a640 1 -- 172.21.15.139:0/2873287963 shutdown_connections 2024-04-25T10:02:47.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.481+0000 7f147282a640 1 -- 172.21.15.139:0/2873287963 wait complete. 2024-04-25T10:02:47.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.482+0000 7f147282a640 1 Processor -- start 2024-04-25T10:02:47.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.483+0000 7f147282a640 1 -- start start 2024-04-25T10:02:47.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.483+0000 7f147282a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f146c0d1080 0x7f146c0d4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.483+0000 7f147282a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f146c15dc60 0x7f146c160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.483+0000 7f147282a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f146c161850 0x7f146c14d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.483+0000 7f147282a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c15e040 con 0x7f146c161850 2024-04-25T10:02:47.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.483+0000 7f147282a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0d1fc0 con 0x7f146c0d1080 2024-04-25T10:02:47.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.483+0000 7f147282a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c14dce0 con 0x7f146c15dc60 2024-04-25T10:02:47.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.483+0000 7f1470dc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f146c161850 0x7f146c14d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.483+0000 7f1470dc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f146c161850 0x7f146c14d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53960/0 (socket says 172.21.15.139:53960) 2024-04-25T10:02:47.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.483+0000 7f1470dc7640 1 -- 172.21.15.139:0/161085318 learned_addr learned my addr 172.21.15.139:0/161085318 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:47.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f1470dc7640 1 -- 172.21.15.139:0/161085318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f146c15dc60 msgr2=0x7f146c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f1470dc7640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f146c15dc60 0x7f146c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f1470dc7640 1 -- 172.21.15.139:0/161085318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f146c0d1080 msgr2=0x7f146c0d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f146bfff640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f146c0d1080 0x7f146c0d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f1470dc7640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f146c0d1080 0x7f146c0d4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f1470dc7640 1 -- 172.21.15.139:0/161085318 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c14e050 con 0x7f146c161850 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f146bfff640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f146c0d1080 0x7f146c0d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f1470dc7640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f146c161850 0x7f146c14d7a0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f1464009d90 tx=0x7f1464004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f1468ff9640 1 -- 172.21.15.139:0/161085318 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146403f070 con 0x7f146c161850 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f1468ff9640 1 -- 172.21.15.139:0/161085318 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1464031c90 con 0x7f146c161850 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f1468ff9640 1 -- 172.21.15.139:0/161085318 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1464043630 con 0x7f146c161850 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f147282a640 1 -- 172.21.15.139:0/161085318 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c14e330 con 0x7f146c161850 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.484+0000 7f147282a640 1 -- 172.21.15.139:0/161085318 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c160090 con 0x7f146c161850 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.485+0000 7f147282a640 1 -- 172.21.15.139:0/161085318 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c1623a0 con 0x7f146c161850 2024-04-25T10:02:47.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.485+0000 7f1468ff9640 1 -- 172.21.15.139:0/161085318 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f146403b410 con 0x7f146c161850 2024-04-25T10:02:47.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.486+0000 7f1468ff9640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f145806b4d0 0x7f145806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.486+0000 7f1468ff9640 1 -- 172.21.15.139:0/161085318 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f14640ab2b0 con 0x7f146c161850 2024-04-25T10:02:47.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.486+0000 7f146bfff640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f145806b4d0 0x7f145806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.486+0000 7f146bfff640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f145806b4d0 0x7f145806d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f145c0013b0 tx=0x7f145c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.490+0000 7f1468ff9640 1 -- 172.21.15.139:0/161085318 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f146c1623a0 con 0x7f146c161850 2024-04-25T10:02:47.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.664+0000 7f147282a640 1 -- 172.21.15.139:0/161085318 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f146c067420 con 0x7f146c161850 2024-04-25T10:02:47.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.665+0000 7f1468ff9640 1 -- 172.21.15.139:0/161085318 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+13222 (secure 0 0 0) 0x7f146c067420 con 0x7f146c161850 2024-04-25T10:02:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:47.670 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":116,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:02:46.947842+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:47.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.667+0000 7f147282a640 1 -- 172.21.15.139:0/161085318 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f145806b4d0 msgr2=0x7f145806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.667+0000 7f147282a640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f145806b4d0 0x7f145806d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f145c0013b0 tx=0x7f145c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:47.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.667+0000 7f147282a640 1 -- 172.21.15.139:0/161085318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f146c161850 msgr2=0x7f146c14d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.667+0000 7f147282a640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f146c161850 0x7f146c14d7a0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f1464009d90 tx=0x7f1464004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:47.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.667+0000 7f147282a640 1 -- 172.21.15.139:0/161085318 shutdown_connections 2024-04-25T10:02:47.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.667+0000 7f147282a640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f146c161850 0x7f146c14d7a0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.667+0000 7f147282a640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f145806b4d0 0x7f145806d990 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.667+0000 7f147282a640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f146c15dc60 0x7f146c160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.667+0000 7f147282a640 1 --2- 172.21.15.139:0/161085318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f146c0d1080 0x7f146c0d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.667+0000 7f147282a640 1 -- 172.21.15.139:0/161085318 >> 172.21.15.139:0/161085318 conn(0x7f146c0b5910 msgr2=0x7f146c0d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.667+0000 7f147282a640 1 -- 172.21.15.139:0/161085318 shutdown_connections 2024-04-25T10:02:47.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.667+0000 7f147282a640 1 -- 172.21.15.139:0/161085318 wait complete. 2024-04-25T10:02:47.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:47.683 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:47.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.778+0000 7f98e5d1d640 1 Processor -- start 2024-04-25T10:02:47.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.778+0000 7f98e5d1d640 1 -- start start 2024-04-25T10:02:47.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.778+0000 7f98e5d1d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f98e00cee50 0x7f98e0168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.778+0000 7f98e5d1d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e0161dc0 con 0x7f98e015dbf0 2024-04-25T10:02:47.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.778+0000 7f98e5d1d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e01620a0 con 0x7f98e01617e0 2024-04-25T10:02:47.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.778+0000 7f98e5d1d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e00d1ee0 con 0x7f98e00cee50 2024-04-25T10:02:47.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.779+0000 7f98dffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f98e00cee50 0x7f98e0168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.779+0000 7f98dffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f98e00cee50 0x7f98e0168b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53972/0 (socket says 172.21.15.139:53972) 2024-04-25T10:02:47.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.779+0000 7f98dffff640 1 -- 172.21.15.139:0/1754607022 learned_addr learned my addr 172.21.15.139:0/1754607022 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:47.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.779+0000 7f98dffff640 1 -- 172.21.15.139:0/1754607022 >> v1:172.21.15.150:6789/0 conn(0x7f98e01617e0 legacy=0x7f98e00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.779+0000 7f98dffff640 1 -- 172.21.15.139:0/1754607022 >> v1:172.21.15.139:6789/0 conn(0x7f98e015dbf0 legacy=0x7f98e00b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.779+0000 7f98dffff640 1 -- 172.21.15.139:0/1754607022 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e0162380 con 0x7f98e00cee50 2024-04-25T10:02:47.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.779+0000 7f98de7fc640 1 -- 172.21.15.139:0/1754607022 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631360295 0 0) 0x7f98e0161dc0 con 0x7f98e015dbf0 2024-04-25T10:02:47.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.779+0000 7f98dffff640 1 --2- 172.21.15.139:0/1754607022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98e00cee50 0x7f98e0168b10 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009e50 tx=0x7f98d802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84aad7b145c43fca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.780+0000 7f98de7fc640 1 -- 172.21.15.139:0/1754607022 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8036070 con 0x7f98e00cee50 2024-04-25T10:02:47.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.780+0000 7f98de7fc640 1 -- 172.21.15.139:0/1754607022 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d8031600 con 0x7f98e00cee50 2024-04-25T10:02:47.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.780+0000 7f98de7fc640 1 -- 172.21.15.139:0/1754607022 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80319d0 con 0x7f98e00cee50 2024-04-25T10:02:47.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.780+0000 7f98e5d1d640 1 -- 172.21.15.139:0/1754607022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98e00cee50 msgr2=0x7f98e0168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.780+0000 7f98e5d1d640 1 --2- 172.21.15.139:0/1754607022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98e00cee50 0x7f98e0168b10 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009e50 tx=0x7f98d802a730 comp rx=0 tx=0).stop 2024-04-25T10:02:47.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.780+0000 7f98e5d1d640 1 -- 172.21.15.139:0/1754607022 shutdown_connections 2024-04-25T10:02:47.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.780+0000 7f98e5d1d640 1 --2- 172.21.15.139:0/1754607022 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98e00cee50 0x7f98e0168b10 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.780+0000 7f98e5d1d640 1 -- 172.21.15.139:0/1754607022 >> 172.21.15.139:0/1754607022 conn(0x7f98e00b5480 msgr2=0x7f98e00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.780+0000 7f98e5d1d640 1 -- 172.21.15.139:0/1754607022 shutdown_connections 2024-04-25T10:02:47.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.780+0000 7f98e5d1d640 1 -- 172.21.15.139:0/1754607022 wait complete. 2024-04-25T10:02:47.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.781+0000 7f98e5d1d640 1 Processor -- start 2024-04-25T10:02:47.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.782+0000 7f98e5d1d640 1 -- start start 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.782+0000 7f98e5d1d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e00cee50 0x7f98e00c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.782+0000 7f98e5d1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98e015dbf0 0x7f98e00cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.782+0000 7f98e5d1d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98e01617e0 0x7f98e013f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.782+0000 7f98e5d1d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e015dfd0 con 0x7f98e015dbf0 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.782+0000 7f98e5d1d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00d3380 con 0x7f98e00cee50 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.782+0000 7f98e5d1d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0151fc0 con 0x7f98e01617e0 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.782+0000 7f98deffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98e015dbf0 0x7f98e00cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.782+0000 7f98deffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98e015dbf0 0x7f98e00cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53976/0 (socket says 172.21.15.139:53976) 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.782+0000 7f98deffd640 1 -- 172.21.15.139:0/4117073603 learned_addr learned my addr 172.21.15.139:0/4117073603 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.782+0000 7f98df7fe640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e00cee50 0x7f98e00c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.782+0000 7f98dffff640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98e01617e0 0x7f98e013f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.783+0000 7f98deffd640 1 -- 172.21.15.139:0/4117073603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98e01617e0 msgr2=0x7f98e013f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.783+0000 7f98deffd640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98e01617e0 0x7f98e013f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.783+0000 7f98deffd640 1 -- 172.21.15.139:0/4117073603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e00cee50 msgr2=0x7f98e00c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.783+0000 7f98deffd640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e00cee50 0x7f98e00c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.783+0000 7f98deffd640 1 -- 172.21.15.139:0/4117073603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e00c93b0 con 0x7f98e015dbf0 2024-04-25T10:02:47.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.783+0000 7f98deffd640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98e015dbf0 0x7f98e00cb450 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f98d400c430 tx=0x7f98d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.783+0000 7f98c7fff640 1 -- 172.21.15.139:0/4117073603 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4017070 con 0x7f98e015dbf0 2024-04-25T10:02:47.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.783+0000 7f98c7fff640 1 -- 172.21.15.139:0/4117073603 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d40052d0 con 0x7f98e015dbf0 2024-04-25T10:02:47.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.783+0000 7f98e5d1d640 1 -- 172.21.15.139:0/4117073603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e00cabe0 con 0x7f98e015dbf0 2024-04-25T10:02:47.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.783+0000 7f98c7fff640 1 -- 172.21.15.139:0/4117073603 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4003830 con 0x7f98e015dbf0 2024-04-25T10:02:47.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.783+0000 7f98e5d1d640 1 -- 172.21.15.139:0/4117073603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e013bf80 con 0x7f98e015dbf0 2024-04-25T10:02:47.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.784+0000 7f98e5d1d640 1 -- 172.21.15.139:0/4117073603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e0074520 con 0x7f98e015dbf0 2024-04-25T10:02:47.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.785+0000 7f98c7fff640 1 -- 172.21.15.139:0/4117073603 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98d401a070 con 0x7f98e015dbf0 2024-04-25T10:02:47.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.785+0000 7f98c7fff640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f98cc06b4d0 0x7f98cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.785+0000 7f98c7fff640 1 -- 172.21.15.139:0/4117073603 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f98d4089310 con 0x7f98e015dbf0 2024-04-25T10:02:47.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.785+0000 7f98df7fe640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f98cc06b4d0 0x7f98cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.785+0000 7f98df7fe640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f98cc06b4d0 0x7f98cc06d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f98d0000f60 tx=0x7f98d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.789+0000 7f98c7fff640 1 -- 172.21.15.139:0/4117073603 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98d4057d00 con 0x7f98e015dbf0 2024-04-25T10:02:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.961+0000 7f98e5d1d640 1 -- 172.21.15.139:0/4117073603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98e00b7530 con 0x7f98e015dbf0 2024-04-25T10:02:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.961+0000 7f98c7fff640 1 -- 172.21.15.139:0/4117073603 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98d4087e20 con 0x7f98e015dbf0 2024-04-25T10:02:47.966 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:02:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.963+0000 7f98e5d1d640 1 -- 172.21.15.139:0/4117073603 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f98cc06b4d0 msgr2=0x7f98cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.963+0000 7f98e5d1d640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f98cc06b4d0 0x7f98cc06d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f98d0000f60 tx=0x7f98d0009040 comp rx=0 tx=0).stop 2024-04-25T10:02:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.963+0000 7f98e5d1d640 1 -- 172.21.15.139:0/4117073603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98e015dbf0 msgr2=0x7f98e00cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.963+0000 7f98e5d1d640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98e015dbf0 0x7f98e00cb450 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f98d400c430 tx=0x7f98d400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.963+0000 7f98e5d1d640 1 -- 172.21.15.139:0/4117073603 shutdown_connections 2024-04-25T10:02:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.963+0000 7f98e5d1d640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98e015dbf0 0x7f98e00cb450 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.963+0000 7f98e5d1d640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f98cc06b4d0 0x7f98cc06d990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.963+0000 7f98e5d1d640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98e01617e0 0x7f98e013f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.963+0000 7f98e5d1d640 1 --2- 172.21.15.139:0/4117073603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98e00cee50 0x7f98e00c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.963+0000 7f98e5d1d640 1 -- 172.21.15.139:0/4117073603 >> 172.21.15.139:0/4117073603 conn(0x7f98e00b5480 msgr2=0x7f98e00ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.963+0000 7f98e5d1d640 1 -- 172.21.15.139:0/4117073603 shutdown_connections 2024-04-25T10:02:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:47.963+0000 7f98e5d1d640 1 -- 172.21.15.139:0/4117073603 wait complete. 2024-04-25T10:02:47.979 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:02:48.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.072+0000 7f2925734640 1 Processor -- start 2024-04-25T10:02:48.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.072+0000 7f2925734640 1 -- start start 2024-04-25T10:02:48.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.073+0000 7f2925734640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f29200b6af0 0x7f292006c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.073+0000 7f2925734640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f292006c5b0 0x7f29200d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.073+0000 7f2925734640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29200cf110 con 0x7f29201611e0 2024-04-25T10:02:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.073+0000 7f2925734640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29200d1fa0 con 0x7f29200b6af0 2024-04-25T10:02:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.073+0000 7f2925734640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200d2560 con 0x7f292006c5b0 2024-04-25T10:02:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.073+0000 7f291effd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f29201611e0 0x7f29200cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35488/0 (socket says 172.21.15.139:35488) 2024-04-25T10:02:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.073+0000 7f291effd640 1 -- 172.21.15.139:0/3220391668 learned_addr learned my addr 172.21.15.139:0/3220391668 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.073+0000 7f2917fff640 1 --2- 172.21.15.139:0/3220391668 >> v2:172.21.15.150:3300/0 conn(0x7f29200b6af0 0x7f292006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.073+0000 7f291f7fe640 1 --2- 172.21.15.139:0/3220391668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f292006c5b0 0x7f29200d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.073+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745947688 0 0) 0x7f29200cf110 con 0x7f29201611e0 2024-04-25T10:02:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.073+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f290c003520 con 0x7f29201611e0 2024-04-25T10:02:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 753199563 0 0) 0x7f290c003520 con 0x7f29201611e0 2024-04-25T10:02:48.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f292006c5b0 msgr2=0x7f29200d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 --2- 172.21.15.139:0/3220391668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f292006c5b0 0x7f29200d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 >> v2:172.21.15.150:3300/0 conn(0x7f29200b6af0 msgr2=0x7f292006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 --2- 172.21.15.139:0/3220391668 >> v2:172.21.15.150:3300/0 conn(0x7f29200b6af0 0x7f292006c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29200b6f50 con 0x7f29201611e0 2024-04-25T10:02:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f2910001670 con 0x7f29201611e0 2024-04-25T10:02:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 >> v1:172.21.15.139:6789/0 conn(0x7f29201611e0 legacy=0x7f29200cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 --2- 172.21.15.139:0/3220391668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f290c0039b0 0x7f290c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 --2- 172.21.15.139:0/3220391668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f290c004800 0x7f290c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 --2- 172.21.15.139:0/3220391668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f292006c5b0 0x7f290c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200d1fa0 con 0x7f290c004800 2024-04-25T10:02:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200d2560 con 0x7f290c0039b0 2024-04-25T10:02:48.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c00b9c0 con 0x7f292006c5b0 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f2917fff640 1 --2- 172.21.15.139:0/3220391668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f290c004800 0x7f290c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291effd640 1 --2- 172.21.15.139:0/3220391668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f290c0039b0 0x7f290c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f291f7fe640 1 --2- 172.21.15.139:0/3220391668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f292006c5b0 0x7f290c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f2917fff640 1 -- 172.21.15.139:0/3220391668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f292006c5b0 msgr2=0x7f290c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f2917fff640 1 --2- 172.21.15.139:0/3220391668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f292006c5b0 0x7f290c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f2917fff640 1 -- 172.21.15.139:0/3220391668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f290c0039b0 msgr2=0x7f290c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f2917fff640 1 --2- 172.21.15.139:0/3220391668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f290c0039b0 0x7f290c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.074+0000 7f2917fff640 1 -- 172.21.15.139:0/3220391668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c003520 con 0x7f290c004800 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.075+0000 7f2917fff640 1 --2- 172.21.15.139:0/3220391668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f290c004800 0x7f290c004c80 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f2908009ac0 tx=0x7f290c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4125b74b222d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.075+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908036070 con 0x7f290c004800 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.075+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908031400 con 0x7f290c004800 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.075+0000 7f291e7fc640 1 -- 172.21.15.139:0/3220391668 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908009690 con 0x7f290c004800 2024-04-25T10:02:48.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.076+0000 7f2925734640 1 -- 172.21.15.139:0/3220391668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f290c004800 msgr2=0x7f290c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.076+0000 7f2925734640 1 --2- 172.21.15.139:0/3220391668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f290c004800 0x7f290c004c80 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f2908009ac0 tx=0x7f290c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:48.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.076+0000 7f291f7fe640 1 -- 172.21.15.139:0/3220391668 reap_dead start 2024-04-25T10:02:48.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.076+0000 7f2925734640 1 -- 172.21.15.139:0/3220391668 shutdown_connections 2024-04-25T10:02:48.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.076+0000 7f2925734640 1 -- 172.21.15.139:0/3220391668 >> 172.21.15.139:0/3220391668 conn(0x7f29200b5b00 msgr2=0x7f29200d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.076+0000 7f2925734640 1 -- 172.21.15.139:0/3220391668 shutdown_connections 2024-04-25T10:02:48.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.076+0000 7f2925734640 1 -- 172.21.15.139:0/3220391668 wait complete. 2024-04-25T10:02:48.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.077+0000 7f2925734640 1 Processor -- start 2024-04-25T10:02:48.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.077+0000 7f2925734640 1 -- start start 2024-04-25T10:02:48.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.077+0000 7f2925734640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f290c004800 0x7f292015ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.077+0000 7f2925734640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f292015d3b0 0x7f29200c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.077+0000 7f2925734640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29200c9c50 0x7f29200cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.077+0000 7f2925734640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200b2a50 con 0x7f29200c9c50 2024-04-25T10:02:48.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.077+0000 7f2925734640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200b2770 con 0x7f292015d3b0 2024-04-25T10:02:48.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.077+0000 7f2925734640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200b2d30 con 0x7f290c004800 2024-04-25T10:02:48.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f291f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29200c9c50 0x7f29200cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f291f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29200c9c50 0x7f29200cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54006/0 (socket says 172.21.15.139:54006) 2024-04-25T10:02:48.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f291f7fe640 1 -- 172.21.15.139:0/1201003094 learned_addr learned my addr 172.21.15.139:0/1201003094 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:48.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f2917fff640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f292015d3b0 0x7f29200c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f291effd640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f290c004800 0x7f292015ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f291f7fe640 1 -- 172.21.15.139:0/1201003094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f290c004800 msgr2=0x7f292015ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f291f7fe640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f290c004800 0x7f292015ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f291f7fe640 1 -- 172.21.15.139:0/1201003094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f292015d3b0 msgr2=0x7f29200c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f291f7fe640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f292015d3b0 0x7f29200c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f291f7fe640 1 -- 172.21.15.139:0/1201003094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29200c8670 con 0x7f29200c9c50 2024-04-25T10:02:48.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f291f7fe640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29200c9c50 0x7f29200cb640 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f29200b65b0 tx=0x7f291800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f29177fe640 1 -- 172.21.15.139:0/1201003094 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291800fe10 con 0x7f29200c9c50 2024-04-25T10:02:48.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f29177fe640 1 -- 172.21.15.139:0/1201003094 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2918015430 con 0x7f29200c9c50 2024-04-25T10:02:48.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f2925734640 1 -- 172.21.15.139:0/1201003094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29200c8950 con 0x7f29200c9c50 2024-04-25T10:02:48.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f29177fe640 1 -- 172.21.15.139:0/1201003094 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918015820 con 0x7f29200c9c50 2024-04-25T10:02:48.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.078+0000 7f2925734640 1 -- 172.21.15.139:0/1201003094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29200c8f10 con 0x7f29200c9c50 2024-04-25T10:02:48.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.079+0000 7f2925734640 1 -- 172.21.15.139:0/1201003094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2920074520 con 0x7f29200c9c50 2024-04-25T10:02:48.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.080+0000 7f29177fe640 1 -- 172.21.15.139:0/1201003094 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29180159c0 con 0x7f29200c9c50 2024-04-25T10:02:48.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.080+0000 7f29177fe640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f290006b4d0 0x7f290006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.080+0000 7f29177fe640 1 -- 172.21.15.139:0/1201003094 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f291808ad80 con 0x7f29200c9c50 2024-04-25T10:02:48.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.080+0000 7f291effd640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f290006b4d0 0x7f290006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.081+0000 7f291effd640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f290006b4d0 0x7f290006d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2910000fb0 tx=0x7f2910009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.084+0000 7f29177fe640 1 -- 172.21.15.139:0/1201003094 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2918019030 con 0x7f29200c9c50 2024-04-25T10:02:48.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.258+0000 7f2925734640 1 -- 172.21.15.139:0/1201003094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f292006ceb0 con 0x7f29200c9c50 2024-04-25T10:02:48.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.258+0000 7f29177fe640 1 -- 172.21.15.139:0/1201003094 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v116) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2918010040 con 0x7f29200c9c50 2024-04-25T10:02:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:48.263 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-04-25T10:02:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-04-25T10:02:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: "poolnum": 1, 2024-04-25T10:02:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: "poolname": ".mgr" 2024-04-25T10:02:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T10:02:48.264 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-04-25T10:02:48.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.260+0000 7f2925734640 1 -- 172.21.15.139:0/1201003094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f290006b4d0 msgr2=0x7f290006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.260+0000 7f2925734640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f290006b4d0 0x7f290006d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2910000fb0 tx=0x7f2910009040 comp rx=0 tx=0).stop 2024-04-25T10:02:48.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.260+0000 7f2925734640 1 -- 172.21.15.139:0/1201003094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29200c9c50 msgr2=0x7f29200cb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.260+0000 7f2925734640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29200c9c50 0x7f29200cb640 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f29200b65b0 tx=0x7f291800f580 comp rx=0 tx=0).stop 2024-04-25T10:02:48.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.260+0000 7f2925734640 1 -- 172.21.15.139:0/1201003094 shutdown_connections 2024-04-25T10:02:48.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.260+0000 7f2925734640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29200c9c50 0x7f29200cb640 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.260+0000 7f2925734640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f290006b4d0 0x7f290006d990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.260+0000 7f2925734640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f292015d3b0 0x7f29200c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.260+0000 7f2925734640 1 --2- 172.21.15.139:0/1201003094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f290c004800 0x7f292015ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.260+0000 7f2925734640 1 -- 172.21.15.139:0/1201003094 >> 172.21.15.139:0/1201003094 conn(0x7f29200b5b00 msgr2=0x7f29201615c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.260+0000 7f2925734640 1 -- 172.21.15.139:0/1201003094 shutdown_connections 2024-04-25T10:02:48.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.260+0000 7f2925734640 1 -- 172.21.15.139:0/1201003094 wait complete. 2024-04-25T10:02:48.278 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:02:48.278 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:02:48.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.375+0000 7fea7e709640 1 Processor -- start 2024-04-25T10:02:48.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.375+0000 7fea7e709640 1 -- start start 2024-04-25T10:02:48.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea7e709640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fea780afc00 0x7fea7806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea7e709640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fea780d0080 0x7fea7806e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea7e709640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea780ccef0 con 0x7fea7815cb20 2024-04-25T10:02:48.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea7e709640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea780703c0 con 0x7fea780d0080 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea7e709640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea780cd820 con 0x7fea780afc00 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea7cca6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fea780d0080 0x7fea7806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea777fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fea7815cb20 0x7fea780cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35504/0 (socket says 172.21.15.139:35504) 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea7cca6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fea780d0080 0x7fea7806e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54014/0 (socket says 172.21.15.139:54014) 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea777fe640 1 -- 172.21.15.139:0/2667519778 learned_addr learned my addr 172.21.15.139:0/2667519778 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea77fff640 1 --2- 172.21.15.139:0/2667519778 >> v2:172.21.15.150:3300/0 conn(0x7fea780afc00 0x7fea7806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea7cca6640 1 -- 172.21.15.139:0/2667519778 >> v2:172.21.15.150:3300/0 conn(0x7fea780afc00 msgr2=0x7fea7806fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea7cca6640 1 --2- 172.21.15.139:0/2667519778 >> v2:172.21.15.150:3300/0 conn(0x7fea780afc00 0x7fea7806fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea7cca6640 1 -- 172.21.15.139:0/2667519778 >> v1:172.21.15.139:6789/0 conn(0x7fea7815cb20 legacy=0x7fea780cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.376+0000 7fea7cca6640 1 -- 172.21.15.139:0/2667519778 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780cd1d0 con 0x7fea780d0080 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.377+0000 7fea7cca6640 1 --2- 172.21.15.139:0/2667519778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea780d0080 0x7fea7806e370 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fea78067330 tx=0x7fea7002a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6456a2ccd65b63c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.377+0000 7fea76ffd640 1 -- 172.21.15.139:0/2667519778 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70007740 con 0x7fea780d0080 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.377+0000 7fea76ffd640 1 -- 172.21.15.139:0/2667519778 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea700078e0 con 0x7fea780d0080 2024-04-25T10:02:48.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.377+0000 7fea76ffd640 1 -- 172.21.15.139:0/2667519778 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70007cb0 con 0x7fea780d0080 2024-04-25T10:02:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.377+0000 7fea7e709640 1 -- 172.21.15.139:0/2667519778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea780d0080 msgr2=0x7fea7806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.377+0000 7fea7e709640 1 --2- 172.21.15.139:0/2667519778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea780d0080 0x7fea7806e370 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fea78067330 tx=0x7fea7002a560 comp rx=0 tx=0).stop 2024-04-25T10:02:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.377+0000 7fea7e709640 1 -- 172.21.15.139:0/2667519778 shutdown_connections 2024-04-25T10:02:48.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.377+0000 7fea7e709640 1 --2- 172.21.15.139:0/2667519778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea780d0080 0x7fea7806e370 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.377+0000 7fea7e709640 1 --2- 172.21.15.139:0/2667519778 >> v2:172.21.15.150:3300/0 conn(0x7fea780afc00 0x7fea7806fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.377+0000 7fea7e709640 1 -- 172.21.15.139:0/2667519778 >> 172.21.15.139:0/2667519778 conn(0x7fea780b2230 msgr2=0x7fea780b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.377+0000 7fea7e709640 1 -- 172.21.15.139:0/2667519778 shutdown_connections 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.378+0000 7fea7e709640 1 -- 172.21.15.139:0/2667519778 wait complete. 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.379+0000 7fea7e709640 1 Processor -- start 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.379+0000 7fea7e709640 1 -- start start 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.379+0000 7fea7e709640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea780afc00 0x7fea780b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.379+0000 7fea7e709640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea780d0080 0x7fea78146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.379+0000 7fea7e709640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea7815cb20 0x7fea780b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.379+0000 7fea7e709640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea78070060 con 0x7fea780d0080 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.379+0000 7fea7e709640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea780affe0 con 0x7fea780afc00 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.379+0000 7fea7e709640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea780cdcb0 con 0x7fea7815cb20 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea777fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea780d0080 0x7fea78146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea777fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea780d0080 0x7fea78146ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54026/0 (socket says 172.21.15.139:54026) 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea777fe640 1 -- 172.21.15.139:0/12143941 learned_addr learned my addr 172.21.15.139:0/12143941 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea77fff640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea780afc00 0x7fea780b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea7cca6640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea7815cb20 0x7fea780b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea777fe640 1 -- 172.21.15.139:0/12143941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea7815cb20 msgr2=0x7fea780b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea777fe640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea7815cb20 0x7fea780b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea777fe640 1 -- 172.21.15.139:0/12143941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea780afc00 msgr2=0x7fea780b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea777fe640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea780afc00 0x7fea780b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea777fe640 1 -- 172.21.15.139:0/12143941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780b3c00 con 0x7fea780d0080 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea777fe640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea780d0080 0x7fea78146ff0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00c430 tx=0x7fea6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea74ff9640 1 -- 172.21.15.139:0/12143941 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c017070 con 0x7fea780d0080 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea74ff9640 1 -- 172.21.15.139:0/12143941 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c005ce0 con 0x7fea780d0080 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea74ff9640 1 -- 172.21.15.139:0/12143941 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0072f0 con 0x7fea780d0080 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea7e709640 1 -- 172.21.15.139:0/12143941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea780b3ee0 con 0x7fea780d0080 2024-04-25T10:02:48.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.380+0000 7fea7e709640 1 -- 172.21.15.139:0/12143941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea780b44a0 con 0x7fea780d0080 2024-04-25T10:02:48.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.381+0000 7fea7e709640 1 -- 172.21.15.139:0/12143941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea78074520 con 0x7fea780d0080 2024-04-25T10:02:48.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.382+0000 7fea74ff9640 1 -- 172.21.15.139:0/12143941 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea6c01a070 con 0x7fea780d0080 2024-04-25T10:02:48.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.382+0000 7fea74ff9640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fea6406b500 0x7fea6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.382+0000 7fea74ff9640 1 -- 172.21.15.139:0/12143941 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fea6c0893d0 con 0x7fea780d0080 2024-04-25T10:02:48.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.382+0000 7fea77fff640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fea6406b500 0x7fea6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.383+0000 7fea77fff640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fea6406b500 0x7fea6406d9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fea68009a30 tx=0x7fea68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.386+0000 7fea74ff9640 1 -- 172.21.15.139:0/12143941 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea6c0584a0 con 0x7fea780d0080 2024-04-25T10:02:48.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.555+0000 7fea7e709640 1 -- 172.21.15.139:0/12143941 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fea780ce680 con 0x7fea780d0080 2024-04-25T10:02:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.964+0000 7fea74ff9640 1 -- 172.21.15.139:0/12143941 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v117) v1 ==== 138+0+0 (secure 0 0 0) 0x7fea6c057da0 con 0x7fea780d0080 2024-04-25T10:02:48.970 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' created 2024-04-25T10:02:48.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.967+0000 7fea7e709640 1 -- 172.21.15.139:0/12143941 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fea6406b500 msgr2=0x7fea6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.967+0000 7fea7e709640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fea6406b500 0x7fea6406d9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fea68009a30 tx=0x7fea68008040 comp rx=0 tx=0).stop 2024-04-25T10:02:48.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.967+0000 7fea7e709640 1 -- 172.21.15.139:0/12143941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea780d0080 msgr2=0x7fea78146ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.967+0000 7fea7e709640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea780d0080 0x7fea78146ff0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00c430 tx=0x7fea6c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.967+0000 7fea7e709640 1 -- 172.21.15.139:0/12143941 shutdown_connections 2024-04-25T10:02:48.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.967+0000 7fea7e709640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea780d0080 0x7fea78146ff0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.967+0000 7fea7e709640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fea6406b500 0x7fea6406d9c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.967+0000 7fea7e709640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea7815cb20 0x7fea780b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.967+0000 7fea7e709640 1 --2- 172.21.15.139:0/12143941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea780afc00 0x7fea780b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.967+0000 7fea7e709640 1 -- 172.21.15.139:0/12143941 >> 172.21.15.139:0/12143941 conn(0x7fea780b2230 msgr2=0x7fea780cac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.968+0000 7fea7e709640 1 -- 172.21.15.139:0/12143941 shutdown_connections 2024-04-25T10:02:48.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:48.968+0000 7fea7e709640 1 -- 172.21.15.139:0/12143941 wait complete. 2024-04-25T10:02:48.988 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:02:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.089+0000 7feb0d66f640 1 Processor -- start 2024-04-25T10:02:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.089+0000 7feb0d66f640 1 -- start start 2024-04-25T10:02:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feb0d66f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb081617e0 0x7feb080ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feb0d66f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb08161dc0 con 0x7feb080cec80 2024-04-25T10:02:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feb0d66f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb081620a0 con 0x7feb0815dbf0 2024-04-25T10:02:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feb0d66f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb080d1ee0 con 0x7feb081617e0 2024-04-25T10:02:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feb077fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7feb080cec80 0x7feb080d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35514/0 (socket says 172.21.15.139:35514) 2024-04-25T10:02:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feb077fe640 1 -- 172.21.15.139:0/1966201959 learned_addr learned my addr 172.21.15.139:0/1966201959 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feafffff640 1 --2- 172.21.15.139:0/1966201959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb081617e0 0x7feb080ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feb067fc640 1 -- 172.21.15.139:0/1966201959 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531823296 0 0) 0x7feb08161dc0 con 0x7feb080cec80 2024-04-25T10:02:49.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feb067fc640 1 -- 172.21.15.139:0/1966201959 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaf4003520 con 0x7feb080cec80 2024-04-25T10:02:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feb067fc640 1 -- 172.21.15.139:0/1966201959 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252571602 0 0) 0x7feb081620a0 con 0x7feb0815dbf0 2024-04-25T10:02:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feb067fc640 1 -- 172.21.15.139:0/1966201959 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb08161dc0 con 0x7feb0815dbf0 2024-04-25T10:02:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feafffff640 1 -- 172.21.15.139:0/1966201959 >> v1:172.21.15.150:6789/0 conn(0x7feb0815dbf0 legacy=0x7feb080b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feafffff640 1 -- 172.21.15.139:0/1966201959 >> v1:172.21.15.139:6789/0 conn(0x7feb080cec80 legacy=0x7feb080d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.090+0000 7feafffff640 1 -- 172.21.15.139:0/1966201959 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb08162380 con 0x7feb081617e0 2024-04-25T10:02:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.091+0000 7feafffff640 1 --2- 172.21.15.139:0/1966201959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb081617e0 0x7feb080ce740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feaf00097d0 tx=0x7feaf002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5875783baea9f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.091+0000 7feb067fc640 1 -- 172.21.15.139:0/1966201959 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf0037070 con 0x7feb081617e0 2024-04-25T10:02:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.091+0000 7feb067fc640 1 -- 172.21.15.139:0/1966201959 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf0032440 con 0x7feb081617e0 2024-04-25T10:02:49.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.091+0000 7feb067fc640 1 -- 172.21.15.139:0/1966201959 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00327f0 con 0x7feb081617e0 2024-04-25T10:02:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.092+0000 7feb0d66f640 1 -- 172.21.15.139:0/1966201959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb081617e0 msgr2=0x7feb080ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.092+0000 7feb0d66f640 1 --2- 172.21.15.139:0/1966201959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb081617e0 0x7feb080ce740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feaf00097d0 tx=0x7feaf002a180 comp rx=0 tx=0).stop 2024-04-25T10:02:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.092+0000 7feb0d66f640 1 -- 172.21.15.139:0/1966201959 shutdown_connections 2024-04-25T10:02:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.092+0000 7feb0d66f640 1 --2- 172.21.15.139:0/1966201959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb081617e0 0x7feb080ce740 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.092+0000 7feb0d66f640 1 -- 172.21.15.139:0/1966201959 >> 172.21.15.139:0/1966201959 conn(0x7feb080b5480 msgr2=0x7feb080b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.092+0000 7feb0d66f640 1 -- 172.21.15.139:0/1966201959 shutdown_connections 2024-04-25T10:02:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.092+0000 7feb0d66f640 1 -- 172.21.15.139:0/1966201959 wait complete. 2024-04-25T10:02:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.093+0000 7feb0d66f640 1 Processor -- start 2024-04-25T10:02:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.093+0000 7feb0d66f640 1 -- start start 2024-04-25T10:02:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb0d66f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb080cec80 0x7feb081564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb0d66f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb0815dbf0 0x7feb08159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb0d66f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb081617e0 0x7feb0814bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb0d66f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0815dfd0 con 0x7feb080cec80 2024-04-25T10:02:49.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb0d66f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb080d2a80 con 0x7feb0815dbf0 2024-04-25T10:02:49.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb0d66f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb08159e80 con 0x7feb081617e0 2024-04-25T10:02:49.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb06ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb080cec80 0x7feb081564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb06ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb080cec80 0x7feb081564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54034/0 (socket says 172.21.15.139:54034) 2024-04-25T10:02:49.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb06ffd640 1 -- 172.21.15.139:0/2775673094 learned_addr learned my addr 172.21.15.139:0/2775673094 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:49.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feafffff640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb0815dbf0 0x7feb08159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb077fe640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb081617e0 0x7feb0814bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb06ffd640 1 -- 172.21.15.139:0/2775673094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb081617e0 msgr2=0x7feb0814bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb06ffd640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb081617e0 0x7feb0814bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb06ffd640 1 -- 172.21.15.139:0/2775673094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb0815dbf0 msgr2=0x7feb08159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb06ffd640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb0815dbf0 0x7feb08159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb06ffd640 1 -- 172.21.15.139:0/2775673094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0815a0b0 con 0x7feb080cec80 2024-04-25T10:02:49.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feafffff640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb0815dbf0 0x7feb08159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:49.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb077fe640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb081617e0 0x7feb0814bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:49.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb06ffd640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb080cec80 0x7feb081564b0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7feaf800c500 tx=0x7feaf800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feaff7fe640 1 -- 172.21.15.139:0/2775673094 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80052d0 con 0x7feb080cec80 2024-04-25T10:02:49.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feaff7fe640 1 -- 172.21.15.139:0/2775673094 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf8005490 con 0x7feb080cec80 2024-04-25T10:02:49.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb0d66f640 1 -- 172.21.15.139:0/2775673094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0815a390 con 0x7feb080cec80 2024-04-25T10:02:49.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feaff7fe640 1 -- 172.21.15.139:0/2775673094 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf8012870 con 0x7feb080cec80 2024-04-25T10:02:49.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.094+0000 7feb0d66f640 1 -- 172.21.15.139:0/2775673094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0815a950 con 0x7feb080cec80 2024-04-25T10:02:49.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.095+0000 7feb0d66f640 1 -- 172.21.15.139:0/2775673094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0815ab90 con 0x7feb080cec80 2024-04-25T10:02:49.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.096+0000 7feaff7fe640 1 -- 172.21.15.139:0/2775673094 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feaf8012a10 con 0x7feb080cec80 2024-04-25T10:02:49.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.096+0000 7feaff7fe640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feae806b4d0 0x7feae806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.096+0000 7feafffff640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feae806b4d0 0x7feae806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.096+0000 7feaff7fe640 1 -- 172.21.15.139:0/2775673094 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5619+0+0 (secure 0 0 0) 0x7feaf8089eb0 con 0x7feb080cec80 2024-04-25T10:02:49.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.097+0000 7feafffff640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feae806b4d0 0x7feae806d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feaf0034000 tx=0x7feaf002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.101+0000 7feaff7fe640 1 -- 172.21.15.139:0/2775673094 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb0815ab90 con 0x7feb080cec80 2024-04-25T10:02:49.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.273+0000 7feb0d66f640 1 -- 172.21.15.139:0/2775673094 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7feb08074520 con 0x7feb080cec80 2024-04-25T10:02:49.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.972+0000 7feaff7fe640 1 -- 172.21.15.139:0/2775673094 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v118) v1 ==== 170+0+0 (secure 0 0 0) 0x7feaf8058d90 con 0x7feb080cec80 2024-04-25T10:02:49.977 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' created 2024-04-25T10:02:49.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.975+0000 7feb0d66f640 1 -- 172.21.15.139:0/2775673094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feae806b4d0 msgr2=0x7feae806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.975+0000 7feb0d66f640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feae806b4d0 0x7feae806d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feaf0034000 tx=0x7feaf002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:49.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.975+0000 7feb0d66f640 1 -- 172.21.15.139:0/2775673094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb080cec80 msgr2=0x7feb081564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.975+0000 7feb0d66f640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb080cec80 0x7feb081564b0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7feaf800c500 tx=0x7feaf800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.975+0000 7feb0d66f640 1 -- 172.21.15.139:0/2775673094 shutdown_connections 2024-04-25T10:02:49.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.975+0000 7feb0d66f640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feae806b4d0 0x7feae806d990 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.975+0000 7feb0d66f640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb081617e0 0x7feb0814bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.975+0000 7feb0d66f640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb0815dbf0 0x7feb08159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.975+0000 7feb0d66f640 1 --2- 172.21.15.139:0/2775673094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb080cec80 0x7feb081564b0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.975+0000 7feb0d66f640 1 -- 172.21.15.139:0/2775673094 >> 172.21.15.139:0/2775673094 conn(0x7feb080b5480 msgr2=0x7feb080d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:49.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.976+0000 7feb0d66f640 1 -- 172.21.15.139:0/2775673094 shutdown_connections 2024-04-25T10:02:49.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:49.976+0000 7feb0d66f640 1 -- 172.21.15.139:0/2775673094 wait complete. 2024-04-25T10:02:50.004 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:02:50.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.101+0000 7fcb16e41640 1 Processor -- start 2024-04-25T10:02:50.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.101+0000 7fcb16e41640 1 -- start start 2024-04-25T10:02:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb16e41640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcb100b31a0 0x7fcb1006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb16e41640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcb1006e870 0x7fcb1015e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb16e41640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb100b2a50 con 0x7fcb10144090 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb16e41640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb100b3610 con 0x7fcb1006e870 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb16e41640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb1006eca0 con 0x7fcb100b31a0 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb153de640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcb1006e870 0x7fcb1015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb153de640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcb1006e870 0x7fcb1015e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54048/0 (socket says 172.21.15.139:54048) 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb153de640 1 -- 172.21.15.139:0/2959434390 learned_addr learned my addr 172.21.15.139:0/2959434390 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb0ffff640 1 --2- 172.21.15.139:0/2959434390 >> v2:172.21.15.150:3300/0 conn(0x7fcb100b31a0 0x7fcb1006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb153de640 1 -- 172.21.15.139:0/2959434390 >> v2:172.21.15.150:3300/0 conn(0x7fcb100b31a0 msgr2=0x7fcb1006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb153de640 1 --2- 172.21.15.139:0/2959434390 >> v2:172.21.15.150:3300/0 conn(0x7fcb100b31a0 0x7fcb1006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb153de640 1 -- 172.21.15.139:0/2959434390 >> v1:172.21.15.150:6789/0 conn(0x7fcb10144090 legacy=0x7fcb100cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.102+0000 7fcb153de640 1 -- 172.21.15.139:0/2959434390 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb100cd0f0 con 0x7fcb1006e870 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.103+0000 7fcb153de640 1 --2- 172.21.15.139:0/2959434390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb1006e870 0x7fcb1015e610 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fcb100b3790 tx=0x7fcb0802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d0f14341dc58ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.103+0000 7fcb0f7fe640 1 -- 172.21.15.139:0/2959434390 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08036070 con 0x7fcb1006e870 2024-04-25T10:02:50.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.103+0000 7fcb0f7fe640 1 -- 172.21.15.139:0/2959434390 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb08031600 con 0x7fcb1006e870 2024-04-25T10:02:50.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.103+0000 7fcb0f7fe640 1 -- 172.21.15.139:0/2959434390 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb080319d0 con 0x7fcb1006e870 2024-04-25T10:02:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.104+0000 7fcb16e41640 1 -- 172.21.15.139:0/2959434390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb1006e870 msgr2=0x7fcb1015e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.104+0000 7fcb16e41640 1 --2- 172.21.15.139:0/2959434390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb1006e870 0x7fcb1015e610 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fcb100b3790 tx=0x7fcb0802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.104+0000 7fcb16e41640 1 -- 172.21.15.139:0/2959434390 shutdown_connections 2024-04-25T10:02:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.104+0000 7fcb16e41640 1 --2- 172.21.15.139:0/2959434390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb1006e870 0x7fcb1015e610 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.104+0000 7fcb16e41640 1 --2- 172.21.15.139:0/2959434390 >> v2:172.21.15.150:3300/0 conn(0x7fcb100b31a0 0x7fcb1006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.104+0000 7fcb16e41640 1 -- 172.21.15.139:0/2959434390 >> 172.21.15.139:0/2959434390 conn(0x7fcb10067000 msgr2=0x7fcb1006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.104+0000 7fcb16e41640 1 -- 172.21.15.139:0/2959434390 shutdown_connections 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.104+0000 7fcb16e41640 1 -- 172.21.15.139:0/2959434390 wait complete. 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.105+0000 7fcb16e41640 1 Processor -- start 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.105+0000 7fcb16e41640 1 -- start start 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb16e41640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb1006e870 0x7fcb1015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb16e41640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb100b31a0 0x7fcb1015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb16e41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10144090 0x7fcb10156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb16e41640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1015b650 con 0x7fcb10144090 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb16e41640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1015b370 con 0x7fcb100b31a0 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb16e41640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1015b930 con 0x7fcb1006e870 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb153de640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10144090 0x7fcb10156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb153de640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10144090 0x7fcb10156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54054/0 (socket says 172.21.15.139:54054) 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb153de640 1 -- 172.21.15.139:0/4015067170 learned_addr learned my addr 172.21.15.139:0/4015067170 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb0ffff640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb100b31a0 0x7fcb1015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb14bdd640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb1006e870 0x7fcb1015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb153de640 1 -- 172.21.15.139:0/4015067170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb1006e870 msgr2=0x7fcb1015a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb153de640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb1006e870 0x7fcb1015a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb153de640 1 -- 172.21.15.139:0/4015067170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb100b31a0 msgr2=0x7fcb1015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb153de640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb100b31a0 0x7fcb1015be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb153de640 1 -- 172.21.15.139:0/4015067170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1015c5d0 con 0x7fcb10144090 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb0ffff640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb100b31a0 0x7fcb1015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb153de640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10144090 0x7fcb10156590 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009f10 tx=0x7fcb08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb0d7fa640 1 -- 172.21.15.139:0/4015067170 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08036050 con 0x7fcb10144090 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb0d7fa640 1 -- 172.21.15.139:0/4015067170 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb08042070 con 0x7fcb10144090 2024-04-25T10:02:50.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb0d7fa640 1 -- 172.21.15.139:0/4015067170 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08009510 con 0x7fcb10144090 2024-04-25T10:02:50.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb16e41640 1 -- 172.21.15.139:0/4015067170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1015c8b0 con 0x7fcb10144090 2024-04-25T10:02:50.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.106+0000 7fcb16e41640 1 -- 172.21.15.139:0/4015067170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb10139240 con 0x7fcb10144090 2024-04-25T10:02:50.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.108+0000 7fcb16e41640 1 -- 172.21.15.139:0/4015067170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb10074520 con 0x7fcb10144090 2024-04-25T10:02:50.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.108+0000 7fcb0d7fa640 1 -- 172.21.15.139:0/4015067170 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb08007c50 con 0x7fcb10144090 2024-04-25T10:02:50.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.109+0000 7fcb0d7fa640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcaf406b4d0 0x7fcaf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.109+0000 7fcb14bdd640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcaf406b4d0 0x7fcaf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.109+0000 7fcb0d7fa640 1 -- 172.21.15.139:0/4015067170 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fcb080ab2b0 con 0x7fcb10144090 2024-04-25T10:02:50.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.109+0000 7fcb14bdd640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcaf406b4d0 0x7fcaf406d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcb04001640 tx=0x7fcb04009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.113+0000 7fcb0d7fa640 1 -- 172.21.15.139:0/4015067170 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb0807a000 con 0x7fcb10144090 2024-04-25T10:02:50.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.318+0000 7fcb16e41640 1 -- 172.21.15.139:0/4015067170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcb1015bc00 con 0x7fcb10144090 2024-04-25T10:02:50.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.983+0000 7fcb0d7fa640 1 -- 172.21.15.139:0/4015067170 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v81) v1 ==== 174+0+0 (secure 0 0 0) 0x7fcb08079900 con 0x7fcb10144090 2024-04-25T10:02:50.989 INFO:teuthology.orchestra.run.smithi139.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T10:02:50.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.988+0000 7fcb16e41640 1 -- 172.21.15.139:0/4015067170 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcaf406b4d0 msgr2=0x7fcaf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.988+0000 7fcb16e41640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcaf406b4d0 0x7fcaf406d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcb04001640 tx=0x7fcb04009040 comp rx=0 tx=0).stop 2024-04-25T10:02:50.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.988+0000 7fcb16e41640 1 -- 172.21.15.139:0/4015067170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10144090 msgr2=0x7fcb10156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.988+0000 7fcb16e41640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10144090 0x7fcb10156590 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009f10 tx=0x7fcb08004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:50.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.989+0000 7fcb16e41640 1 -- 172.21.15.139:0/4015067170 shutdown_connections 2024-04-25T10:02:50.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.989+0000 7fcb16e41640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10144090 0x7fcb10156590 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.989+0000 7fcb16e41640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcaf406b4d0 0x7fcaf406d990 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.989+0000 7fcb16e41640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb100b31a0 0x7fcb1015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.989+0000 7fcb16e41640 1 --2- 172.21.15.139:0/4015067170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb1006e870 0x7fcb1015a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.989+0000 7fcb16e41640 1 -- 172.21.15.139:0/4015067170 >> 172.21.15.139:0/4015067170 conn(0x7fcb10067000 msgr2=0x7fcb100711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.990+0000 7fcb16e41640 1 -- 172.21.15.139:0/4015067170 shutdown_connections 2024-04-25T10:02:50.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:50.990+0000 7fcb16e41640 1 -- 172.21.15.139:0/4015067170 wait complete. 2024-04-25T10:02:51.016 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:51.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.110+0000 7f579697e640 1 Processor -- start 2024-04-25T10:02:51.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.110+0000 7f579697e640 1 -- start start 2024-04-25T10:02:51.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.111+0000 7f579697e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5790158260 0x7f579015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.111+0000 7f579697e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f579006da20 0x7f57900b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.111+0000 7f579697e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57900b3e10 con 0x7f57900b3600 2024-04-25T10:02:51.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.111+0000 7f579697e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57900b3a50 con 0x7f5790158260 2024-04-25T10:02:51.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.111+0000 7f579697e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57901615c0 con 0x7f579006da20 2024-04-25T10:02:51.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.111+0000 7f578ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5790158260 0x7f579015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.111+0000 7f5794f1b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f57900b3600 0x7f579006def0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35518/0 (socket says 172.21.15.139:35518) 2024-04-25T10:02:51.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.111+0000 7f5794f1b640 1 -- 172.21.15.139:0/3114391314 learned_addr learned my addr 172.21.15.139:0/3114391314 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:51.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.111+0000 7f578f7fe640 1 --2- 172.21.15.139:0/3114391314 >> v2:172.21.15.150:3300/0 conn(0x7f579006da20 0x7f57900b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.111+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691666321 0 0) 0x7f5788001490 con 0x7f57900b3600 2024-04-25T10:02:51.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.111+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5778003520 con 0x7f57900b3600 2024-04-25T10:02:51.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3034709680 0 0) 0x7f5778003520 con 0x7f57900b3600 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 >> v2:172.21.15.150:3300/0 conn(0x7f579006da20 msgr2=0x7f57900b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 --2- 172.21.15.139:0/3114391314 >> v2:172.21.15.150:3300/0 conn(0x7f579006da20 0x7f57900b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 >> v2:172.21.15.139:3300/0 conn(0x7f5790158260 msgr2=0x7f579015df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 --2- 172.21.15.139:0/3114391314 >> v2:172.21.15.139:3300/0 conn(0x7f5790158260 0x7f579015df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57901618a0 con 0x7f57900b3600 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f5788001bc0 con 0x7f57900b3600 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 >> v1:172.21.15.139:6789/0 conn(0x7f57900b3600 legacy=0x7f579006def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 --2- 172.21.15.139:0/3114391314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57780039b0 0x7f5778003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 --2- 172.21.15.139:0/3114391314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5778004800 0x7f5778004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 --2- 172.21.15.139:0/3114391314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 0x7f577800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57900b3a50 con 0x7f57780051c0 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57901615c0 con 0x7f57780039b0 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577800bf50 con 0x7f5778004800 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f5794f1b640 1 --2- 172.21.15.139:0/3114391314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 0x7f577800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578ffff640 1 --2- 172.21.15.139:0/3114391314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57780039b0 0x7f5778003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f578f7fe640 1 --2- 172.21.15.139:0/3114391314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5778004800 0x7f5778004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f5794f1b640 1 -- 172.21.15.139:0/3114391314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5778004800 msgr2=0x7f5778004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f5794f1b640 1 --2- 172.21.15.139:0/3114391314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5778004800 0x7f5778004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f5794f1b640 1 -- 172.21.15.139:0/3114391314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57780039b0 msgr2=0x7f5778003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f5794f1b640 1 --2- 172.21.15.139:0/3114391314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57780039b0 0x7f5778003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.112+0000 7f5794f1b640 1 -- 172.21.15.139:0/3114391314 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5778003520 con 0x7f57780051c0 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.113+0000 7f5794f1b640 1 -- 172.21.15.139:0/3114391314 reap_dead start 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.113+0000 7f5794f1b640 1 --2- 172.21.15.139:0/3114391314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 0x7f577800b9c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f578800aa60 tx=0x7f5778005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8cd9c35227b7ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.113+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578800a6b0 con 0x7f57780051c0 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.113+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578802f070 con 0x7f57780051c0 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.113+0000 7f578effd640 1 -- 172.21.15.139:0/3114391314 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788034470 con 0x7f57780051c0 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.113+0000 7f579697e640 1 -- 172.21.15.139:0/3114391314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 msgr2=0x7f577800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.113+0000 7f579697e640 1 --2- 172.21.15.139:0/3114391314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 0x7f577800b9c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f578800aa60 tx=0x7f5778005830 comp rx=0 tx=0).stop 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.114+0000 7f579697e640 1 -- 172.21.15.139:0/3114391314 shutdown_connections 2024-04-25T10:02:51.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.114+0000 7f579697e640 1 --2- 172.21.15.139:0/3114391314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 0x7f577800b9c0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.114+0000 7f579697e640 1 -- 172.21.15.139:0/3114391314 >> 172.21.15.139:0/3114391314 conn(0x7f57900b23d0 msgr2=0x7f57900cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.114+0000 7f579697e640 1 -- 172.21.15.139:0/3114391314 shutdown_connections 2024-04-25T10:02:51.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.114+0000 7f579697e640 1 -- 172.21.15.139:0/3114391314 wait complete. 2024-04-25T10:02:51.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.115+0000 7f579697e640 1 Processor -- start 2024-04-25T10:02:51.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.115+0000 7f579697e640 1 -- start start 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f579697e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 0x7f57900cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f579697e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57900cee80 0x7f57901497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f579697e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57900cceb0 0x7f57900cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f579697e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790162140 con 0x7f57780051c0 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f579697e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790161e60 con 0x7f57900cceb0 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f579697e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5790162420 con 0x7f57900cee80 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 0x7f57900cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 0x7f57900cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54078/0 (socket says 172.21.15.139:54078) 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578ffff640 1 -- 172.21.15.139:0/4224413100 learned_addr learned my addr 172.21.15.139:0/4224413100 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578f7fe640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57900cee80 0x7f57901497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f5794f1b640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57900cceb0 0x7f57900cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578ffff640 1 -- 172.21.15.139:0/4224413100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57900cee80 msgr2=0x7f57901497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578ffff640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57900cee80 0x7f57901497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578ffff640 1 -- 172.21.15.139:0/4224413100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57900cceb0 msgr2=0x7f57900cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578ffff640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57900cceb0 0x7f57900cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578ffff640 1 -- 172.21.15.139:0/4224413100 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57900ccb20 con 0x7f57780051c0 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578ffff640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 0x7f57900cc5e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f5780011940 tx=0x7f5780011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578cff9640 1 -- 172.21.15.139:0/4224413100 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578000fa90 con 0x7f57780051c0 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578cff9640 1 -- 172.21.15.139:0/4224413100 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578000fc30 con 0x7f57780051c0 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f579697e640 1 -- 172.21.15.139:0/4224413100 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5790157160 con 0x7f57780051c0 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f578cff9640 1 -- 172.21.15.139:0/4224413100 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780014440 con 0x7f57780051c0 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.116+0000 7f579697e640 1 -- 172.21.15.139:0/4224413100 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5790159a70 con 0x7f57780051c0 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.117+0000 7f579697e640 1 -- 172.21.15.139:0/4224413100 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5790074520 con 0x7f57780051c0 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.118+0000 7f578cff9640 1 -- 172.21.15.139:0/4224413100 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5780012040 con 0x7f57780051c0 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.118+0000 7f578cff9640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f577c06b500 0x7f577c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.118+0000 7f578cff9640 1 -- 172.21.15.139:0/4224413100 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f578008a7c0 con 0x7f57780051c0 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.118+0000 7f578f7fe640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f577c06b500 0x7f577c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.119+0000 7f578f7fe640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f577c06b500 0x7f577c06d9c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f57901578c0 tx=0x7f5784008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.123+0000 7f578cff9640 1 -- 172.21.15.139:0/4224413100 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5780019030 con 0x7f57780051c0 2024-04-25T10:02:51.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.293+0000 7f579697e640 1 -- 172.21.15.139:0/4224413100 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57900d0aa0 con 0x7f57780051c0 2024-04-25T10:02:51.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.294+0000 7f578cff9640 1 -- 172.21.15.139:0/4224413100 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+17013 (secure 0 0 0) 0x7f578005de20 con 0x7f57780051c0 2024-04-25T10:02:51.299 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:51.299 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":119,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:02:50.319555+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:48.574891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:49.293101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:51.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.296+0000 7f579697e640 1 -- 172.21.15.139:0/4224413100 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f577c06b500 msgr2=0x7f577c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.296+0000 7f579697e640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f577c06b500 0x7f577c06d9c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f57901578c0 tx=0x7f5784008040 comp rx=0 tx=0).stop 2024-04-25T10:02:51.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.296+0000 7f579697e640 1 -- 172.21.15.139:0/4224413100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 msgr2=0x7f57900cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.296+0000 7f579697e640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 0x7f57900cc5e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f5780011940 tx=0x7f5780011e00 comp rx=0 tx=0).stop 2024-04-25T10:02:51.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.296+0000 7f579697e640 1 -- 172.21.15.139:0/4224413100 shutdown_connections 2024-04-25T10:02:51.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.296+0000 7f579697e640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f577c06b500 0x7f577c06d9c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.296+0000 7f579697e640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57900cceb0 0x7f57900cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.296+0000 7f579697e640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57900cee80 0x7f57901497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.296+0000 7f579697e640 1 --2- 172.21.15.139:0/4224413100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f57780051c0 0x7f57900cc5e0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.296+0000 7f579697e640 1 -- 172.21.15.139:0/4224413100 >> 172.21.15.139:0/4224413100 conn(0x7f57900b23d0 msgr2=0x7f5790160a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.297+0000 7f579697e640 1 -- 172.21.15.139:0/4224413100 shutdown_connections 2024-04-25T10:02:51.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.297+0000 7f579697e640 1 -- 172.21.15.139:0/4224413100 wait complete. 2024-04-25T10:02:51.312 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:51.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.405+0000 7f2d7fe2a640 1 Processor -- start 2024-04-25T10:02:51.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.405+0000 7f2d7fe2a640 1 -- start start 2024-04-25T10:02:51.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7fe2a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d78160e90 0x7f2d780afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7fe2a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2d7806afc0 0x7f2d78070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7fe2a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d780d0400 con 0x7f2d7806ab60 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7fe2a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d780d14a0 con 0x7f2d7806afc0 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7fe2a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780d06e0 con 0x7f2d78160e90 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7e3c7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2d7806afc0 0x7f2d78070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7d3c5640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2d7806ab60 0x7f2d7806fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35520/0 (socket says 172.21.15.139:35520) 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7d3c5640 1 -- 172.21.15.139:0/4001842928 learned_addr learned my addr 172.21.15.139:0/4001842928 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7dbc6640 1 --2- 172.21.15.139:0/4001842928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d78160e90 0x7f2d780afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7e3c7640 1 -- 172.21.15.139:0/4001842928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d78160e90 msgr2=0x7f2d780afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7e3c7640 1 --2- 172.21.15.139:0/4001842928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d78160e90 0x7f2d780afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7e3c7640 1 -- 172.21.15.139:0/4001842928 >> v1:172.21.15.139:6789/0 conn(0x7f2d7806ab60 legacy=0x7f2d7806fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7e3c7640 1 -- 172.21.15.139:0/4001842928 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d780d09c0 con 0x7f2d7806afc0 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7cbc4640 1 -- 172.21.15.139:0/4001842928 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298330431 0 0) 0x7f2d780d0400 con 0x7f2d7806ab60 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.406+0000 7f2d7dbc6640 1 --2- 172.21.15.139:0/4001842928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d78160e90 0x7f2d780afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.407+0000 7f2d7e3c7640 1 --2- 172.21.15.139:0/4001842928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7806afc0 0x7f2d78070180 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009d90 tx=0x7f2d7002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49c0a2c0e5341c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.407+0000 7f2d7cbc4640 1 -- 172.21.15.139:0/4001842928 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70007740 con 0x7f2d7806afc0 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.407+0000 7f2d7cbc4640 1 -- 172.21.15.139:0/4001842928 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d700078e0 con 0x7f2d7806afc0 2024-04-25T10:02:51.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.407+0000 7f2d7cbc4640 1 -- 172.21.15.139:0/4001842928 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70007cb0 con 0x7f2d7806afc0 2024-04-25T10:02:51.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.407+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4001842928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7806afc0 msgr2=0x7f2d78070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.407+0000 7f2d7fe2a640 1 --2- 172.21.15.139:0/4001842928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7806afc0 0x7f2d78070180 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009d90 tx=0x7f2d7002a660 comp rx=0 tx=0).stop 2024-04-25T10:02:51.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.407+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4001842928 shutdown_connections 2024-04-25T10:02:51.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.407+0000 7f2d7fe2a640 1 --2- 172.21.15.139:0/4001842928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7806afc0 0x7f2d78070180 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.407+0000 7f2d7fe2a640 1 --2- 172.21.15.139:0/4001842928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d78160e90 0x7f2d780afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.407+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4001842928 >> 172.21.15.139:0/4001842928 conn(0x7f2d780b23f0 msgr2=0x7f2d780b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.408+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4001842928 shutdown_connections 2024-04-25T10:02:51.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.408+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4001842928 wait complete. 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.409+0000 7f2d7fe2a640 1 Processor -- start 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.409+0000 7f2d7fe2a640 1 -- start start 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.409+0000 7f2d7fe2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7806ab60 0x7f2d78149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.409+0000 7f2d7fe2a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d7806afc0 0x7f2d780b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.409+0000 7f2d7fe2a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d78160e90 0x7f2d78149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.409+0000 7f2d7fe2a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780b0000 con 0x7f2d7806ab60 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.409+0000 7f2d7fe2a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780d0f80 con 0x7f2d7806afc0 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.409+0000 7f2d7fe2a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7806b410 con 0x7f2d78160e90 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.409+0000 7f2d7dbc6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7806ab60 0x7f2d78149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.409+0000 7f2d7dbc6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7806ab60 0x7f2d78149160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54094/0 (socket says 172.21.15.139:54094) 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.409+0000 7f2d7dbc6640 1 -- 172.21.15.139:0/4119069695 learned_addr learned my addr 172.21.15.139:0/4119069695 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.410+0000 7f2d7dbc6640 1 -- 172.21.15.139:0/4119069695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d78160e90 msgr2=0x7f2d78149910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.410+0000 7f2d7dbc6640 1 --2- 172.21.15.139:0/4119069695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d78160e90 0x7f2d78149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.410+0000 7f2d7dbc6640 1 -- 172.21.15.139:0/4119069695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d7806afc0 msgr2=0x7f2d780b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.410+0000 7f2d7dbc6640 1 --2- 172.21.15.139:0/4119069695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d7806afc0 0x7f2d780b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.410+0000 7f2d7dbc6640 1 -- 172.21.15.139:0/4119069695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d780ca520 con 0x7f2d7806ab60 2024-04-25T10:02:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.410+0000 7f2d7dbc6640 1 --2- 172.21.15.139:0/4119069695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7806ab60 0x7f2d78149160 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400e810 tx=0x7f2d7400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.410+0000 7f2d6e7fc640 1 -- 172.21.15.139:0/4119069695 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400f040 con 0x7f2d7806ab60 2024-04-25T10:02:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.410+0000 7f2d6e7fc640 1 -- 172.21.15.139:0/4119069695 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d74004030 con 0x7f2d7806ab60 2024-04-25T10:02:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.410+0000 7f2d6e7fc640 1 -- 172.21.15.139:0/4119069695 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74004420 con 0x7f2d7806ab60 2024-04-25T10:02:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.410+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4119069695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d780cc850 con 0x7f2d7806ab60 2024-04-25T10:02:51.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.410+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4119069695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d780cc570 con 0x7f2d7806ab60 2024-04-25T10:02:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.411+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4119069695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d78074520 con 0x7f2d7806ab60 2024-04-25T10:02:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.411+0000 7f2d6e7fc640 1 -- 172.21.15.139:0/4119069695 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d7400c6d0 con 0x7f2d7806ab60 2024-04-25T10:02:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.412+0000 7f2d6e7fc640 1 --2- 172.21.15.139:0/4119069695 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d5c06b4d0 0x7f2d5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.412+0000 7f2d6e7fc640 1 -- 172.21.15.139:0/4119069695 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d7408a0b0 con 0x7f2d7806ab60 2024-04-25T10:02:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.412+0000 7f2d7d3c5640 1 --2- 172.21.15.139:0/4119069695 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d5c06b4d0 0x7f2d5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.412+0000 7f2d7d3c5640 1 --2- 172.21.15.139:0/4119069695 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d5c06b4d0 0x7f2d5c06d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2d68007fd0 tx=0x7f2d68007960 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.416+0000 7f2d6e7fc640 1 -- 172.21.15.139:0/4119069695 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d74017020 con 0x7f2d7806ab60 2024-04-25T10:02:51.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.589+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4119069695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d780728b0 con 0x7f2d7806ab60 2024-04-25T10:02:51.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.590+0000 7f2d6e7fc640 1 -- 172.21.15.139:0/4119069695 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+17013 (secure 0 0 0) 0x7f2d74058e30 con 0x7f2d7806ab60 2024-04-25T10:02:51.595 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:51.595 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":119,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:02:50.319555+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:48.574891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:49.293101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:51.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.592+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4119069695 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d5c06b4d0 msgr2=0x7f2d5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.592+0000 7f2d7fe2a640 1 --2- 172.21.15.139:0/4119069695 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d5c06b4d0 0x7f2d5c06d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2d68007fd0 tx=0x7f2d68007960 comp rx=0 tx=0).stop 2024-04-25T10:02:51.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.592+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4119069695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7806ab60 msgr2=0x7f2d78149160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.592+0000 7f2d7fe2a640 1 --2- 172.21.15.139:0/4119069695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7806ab60 0x7f2d78149160 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400e810 tx=0x7f2d7400ecd0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.592+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4119069695 shutdown_connections 2024-04-25T10:02:51.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.592+0000 7f2d7fe2a640 1 --2- 172.21.15.139:0/4119069695 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2d5c06b4d0 0x7f2d5c06d990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.592+0000 7f2d7fe2a640 1 --2- 172.21.15.139:0/4119069695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d78160e90 0x7f2d78149910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.592+0000 7f2d7fe2a640 1 --2- 172.21.15.139:0/4119069695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d7806afc0 0x7f2d780b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.592+0000 7f2d7fe2a640 1 --2- 172.21.15.139:0/4119069695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d7806ab60 0x7f2d78149160 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.592+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4119069695 >> 172.21.15.139:0/4119069695 conn(0x7f2d780b23f0 msgr2=0x7f2d780cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.592+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4119069695 shutdown_connections 2024-04-25T10:02:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.592+0000 7f2d7fe2a640 1 -- 172.21.15.139:0/4119069695 wait complete. 2024-04-25T10:02:51.608 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:02:51.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.703+0000 7f1bb86cb640 1 Processor -- start 2024-04-25T10:02:51.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.703+0000 7f1bb86cb640 1 -- start start 2024-04-25T10:02:51.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb86cb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1bb00cd610 0x7f1bb00d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb86cb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1bb0161fd0 0x7f1bb00d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb86cb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb0099ca0 con 0x7f1bb009ad40 2024-04-25T10:02:51.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb86cb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb0099a50 con 0x7f1bb00cd610 2024-04-25T10:02:51.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb86cb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb0099f80 con 0x7f1bb0161fd0 2024-04-25T10:02:51.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb6467640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1bb00cd610 0x7f1bb00d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb6467640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1bb00cd610 0x7f1bb00d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54108/0 (socket says 172.21.15.139:54108) 2024-04-25T10:02:51.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb6467640 1 -- 172.21.15.139:0/2232324428 learned_addr learned my addr 172.21.15.139:0/2232324428 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:51.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb6c68640 1 --2- 172.21.15.139:0/2232324428 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1bb0161fd0 0x7f1bb00d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb6467640 1 -- 172.21.15.139:0/2232324428 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1bb0161fd0 msgr2=0x7f1bb00d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb6467640 1 --2- 172.21.15.139:0/2232324428 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1bb0161fd0 0x7f1bb00d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb6467640 1 -- 172.21.15.139:0/2232324428 >> v1:172.21.15.150:6789/0 conn(0x7f1bb009ad40 legacy=0x7f1bb01618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.704+0000 7f1bb6467640 1 -- 172.21.15.139:0/2232324428 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb009a260 con 0x7f1bb00cd610 2024-04-25T10:02:51.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.705+0000 7f1bb6467640 1 --2- 172.21.15.139:0/2232324428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb00cd610 0x7f1bb00d0370 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009870 tx=0x7f1bac02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6a191f0189928a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.705+0000 7f1bb5465640 1 -- 172.21.15.139:0/2232324428 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac037070 con 0x7f1bb00cd610 2024-04-25T10:02:51.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.705+0000 7f1bb5465640 1 -- 172.21.15.139:0/2232324428 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac032440 con 0x7f1bb00cd610 2024-04-25T10:02:51.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.705+0000 7f1bb5465640 1 -- 172.21.15.139:0/2232324428 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac032810 con 0x7f1bb00cd610 2024-04-25T10:02:51.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.706+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2232324428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb00cd610 msgr2=0x7f1bb00d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.706+0000 7f1bb86cb640 1 --2- 172.21.15.139:0/2232324428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb00cd610 0x7f1bb00d0370 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009870 tx=0x7f1bac02a2a0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.706+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2232324428 shutdown_connections 2024-04-25T10:02:51.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.706+0000 7f1bb86cb640 1 --2- 172.21.15.139:0/2232324428 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1bb0161fd0 0x7f1bb00d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.706+0000 7f1bb86cb640 1 --2- 172.21.15.139:0/2232324428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb00cd610 0x7f1bb00d0370 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.706+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2232324428 >> 172.21.15.139:0/2232324428 conn(0x7f1bb00b31f0 msgr2=0x7f1bb00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.706+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2232324428 shutdown_connections 2024-04-25T10:02:51.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.706+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2232324428 wait complete. 2024-04-25T10:02:51.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.707+0000 7f1bb86cb640 1 Processor -- start 2024-04-25T10:02:51.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.707+0000 7f1bb86cb640 1 -- start start 2024-04-25T10:02:51.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb86cb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bb009ad40 0x7f1bb014ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb86cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb00cd610 0x7f1bb006bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb86cb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1bb0161fd0 0x7f1bb006c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb86cb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb00cd9f0 con 0x7f1bb00cd610 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb86cb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb009a820 con 0x7f1bb009ad40 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb86cb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb00cdb70 con 0x7f1bb0161fd0 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb5c66640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb00cd610 0x7f1bb006bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb5c66640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb00cd610 0x7f1bb006bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54112/0 (socket says 172.21.15.139:54112) 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb5c66640 1 -- 172.21.15.139:0/2901518228 learned_addr learned my addr 172.21.15.139:0/2901518228 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb6467640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bb009ad40 0x7f1bb014ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb6c68640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1bb0161fd0 0x7f1bb006c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb5c66640 1 -- 172.21.15.139:0/2901518228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1bb0161fd0 msgr2=0x7f1bb006c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb5c66640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1bb0161fd0 0x7f1bb006c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb5c66640 1 -- 172.21.15.139:0/2901518228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bb009ad40 msgr2=0x7f1bb014ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb5c66640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bb009ad40 0x7f1bb014ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb5c66640 1 -- 172.21.15.139:0/2901518228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb014b530 con 0x7f1bb00cd610 2024-04-25T10:02:51.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb6467640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bb009ad40 0x7f1bb014ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:51.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb6c68640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1bb0161fd0 0x7f1bb006c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:51.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb5c66640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb00cd610 0x7f1bb006bc70 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c430 tx=0x7f1ba000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1ba6ffd640 1 -- 172.21.15.139:0/2901518228 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0011790 con 0x7f1bb00cd610 2024-04-25T10:02:51.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1ba6ffd640 1 -- 172.21.15.139:0/2901518228 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba0017070 con 0x7f1bb00cd610 2024-04-25T10:02:51.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1ba6ffd640 1 -- 172.21.15.139:0/2901518228 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0011bb0 con 0x7f1bb00cd610 2024-04-25T10:02:51.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2901518228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb014b810 con 0x7f1bb00cd610 2024-04-25T10:02:51.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.708+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2901518228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb014ef80 con 0x7f1bb00cd610 2024-04-25T10:02:51.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.709+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2901518228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb0074520 con 0x7f1bb00cd610 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.710+0000 7f1ba6ffd640 1 -- 172.21.15.139:0/2901518228 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ba001c460 con 0x7f1bb00cd610 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.710+0000 7f1ba6ffd640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b9406b4d0 0x7f1b9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.710+0000 7f1bb6467640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b9406b4d0 0x7f1b9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.710+0000 7f1ba6ffd640 1 -- 172.21.15.139:0/2901518228 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1ba008a200 con 0x7f1bb00cd610 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.711+0000 7f1bb6467640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b9406b4d0 0x7f1b9406d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1bac034000 tx=0x7f1bac02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.715+0000 7f1ba6ffd640 1 -- 172.21.15.139:0/2901518228 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ba0058ef0 con 0x7f1bb00cd610 2024-04-25T10:02:51.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.920+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2901518228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1bb00b3f70 con 0x7f1bb00cd610 2024-04-25T10:02:51.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.986+0000 7f1ba6ffd640 1 -- 172.21.15.139:0/2901518228 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v82) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1ba00587f0 con 0x7f1bb00cd610 2024-04-25T10:02:51.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.988+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2901518228 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b9406b4d0 msgr2=0x7f1b9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.988+0000 7f1bb86cb640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b9406b4d0 0x7f1b9406d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1bac034000 tx=0x7f1bac02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:51.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.988+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2901518228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb00cd610 msgr2=0x7f1bb006bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.989+0000 7f1bb86cb640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb00cd610 0x7f1bb006bc70 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c430 tx=0x7f1ba000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.989+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2901518228 shutdown_connections 2024-04-25T10:02:51.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.989+0000 7f1bb86cb640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1bb00cd610 0x7f1bb006bc70 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.989+0000 7f1bb86cb640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1b9406b4d0 0x7f1b9406d990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.989+0000 7f1bb86cb640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1bb0161fd0 0x7f1bb006c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.989+0000 7f1bb86cb640 1 --2- 172.21.15.139:0/2901518228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bb009ad40 0x7f1bb014ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.989+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2901518228 >> 172.21.15.139:0/2901518228 conn(0x7f1bb00b31f0 msgr2=0x7f1bb00b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.989+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2901518228 shutdown_connections 2024-04-25T10:02:51.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:51.989+0000 7f1bb86cb640 1 -- 172.21.15.139:0/2901518228 wait complete. 2024-04-25T10:02:52.004 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:02:52.005 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:02:52.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.101+0000 7f84f96dd640 1 Processor -- start 2024-04-25T10:02:52.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.101+0000 7f84f96dd640 1 -- start start 2024-04-25T10:02:52.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.102+0000 7f84f96dd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f84f41617e0 0x7f84f40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.102+0000 7f84f96dd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f4161dc0 con 0x7f84f40cec80 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.102+0000 7f84f96dd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f41620a0 con 0x7f84f415dbf0 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.102+0000 7f84f96dd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f40cf0d0 con 0x7f84f41617e0 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.102+0000 7f84f37fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f84f40cec80 0x7f84f40d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35530/0 (socket says 172.21.15.139:35530) 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.102+0000 7f84f37fe640 1 -- 172.21.15.139:0/836837335 learned_addr learned my addr 172.21.15.139:0/836837335 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.102+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682659946 0 0) 0x7f84f4161dc0 con 0x7f84f40cec80 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.102+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84dc003520 con 0x7f84f40cec80 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.102+0000 7f84f27fc640 1 --2- 172.21.15.139:0/836837335 >> v2:172.21.15.150:3300/0 conn(0x7f84f41617e0 0x7f84f40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 79361367 0 0) 0x7f84dc003520 con 0x7f84f40cec80 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 >> v2:172.21.15.150:3300/0 conn(0x7f84f41617e0 msgr2=0x7f84f40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 --2- 172.21.15.139:0/836837335 >> v2:172.21.15.150:3300/0 conn(0x7f84f41617e0 0x7f84f40ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 >> v1:172.21.15.150:6789/0 conn(0x7f84f415dbf0 legacy=0x7f84f40b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f4162380 con 0x7f84f40cec80 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f84ec0019f0 con 0x7f84f40cec80 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 >> v1:172.21.15.139:6789/0 conn(0x7f84f40cec80 legacy=0x7f84f40d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 --2- 172.21.15.139:0/836837335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84dc0039b0 0x7f84dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 --2- 172.21.15.139:0/836837335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84dc004800 0x7f84dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 --2- 172.21.15.139:0/836837335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84dc0052e0 0x7f84dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f40cf0d0 con 0x7f84dc0039b0 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc00c1b0 con 0x7f84dc0052e0 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc00c3b0 con 0x7f84dc004800 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f2ffd640 1 --2- 172.21.15.139:0/836837335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84dc0039b0 0x7f84dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f37fe640 1 --2- 172.21.15.139:0/836837335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84dc0052e0 0x7f84dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f27fc640 1 --2- 172.21.15.139:0/836837335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84dc004800 0x7f84dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f2ffd640 1 -- 172.21.15.139:0/836837335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84dc004800 msgr2=0x7f84dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f2ffd640 1 --2- 172.21.15.139:0/836837335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84dc004800 0x7f84dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f2ffd640 1 -- 172.21.15.139:0/836837335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84dc0052e0 msgr2=0x7f84dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f2ffd640 1 --2- 172.21.15.139:0/836837335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84dc0052e0 0x7f84dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f2ffd640 1 -- 172.21.15.139:0/836837335 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc003520 con 0x7f84dc0039b0 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f37fe640 1 --2- 172.21.15.139:0/836837335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84dc0052e0 0x7f84dc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.103+0000 7f84f37fe640 1 -- 172.21.15.139:0/836837335 reap_dead start 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.104+0000 7f84f2ffd640 1 --2- 172.21.15.139:0/836837335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84dc0039b0 0x7f84dc003dd0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f84e4001720 tx=0x7f84dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffc5f45a3b5212b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.104+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4001370 con 0x7f84dc0039b0 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.104+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e40052d0 con 0x7f84dc0039b0 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.104+0000 7f84f1ffb640 1 -- 172.21.15.139:0/836837335 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e402ab90 con 0x7f84dc0039b0 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.104+0000 7f84f96dd640 1 -- 172.21.15.139:0/836837335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84dc0039b0 msgr2=0x7f84dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.104+0000 7f84f96dd640 1 --2- 172.21.15.139:0/836837335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84dc0039b0 0x7f84dc003dd0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f84e4001720 tx=0x7f84dc005830 comp rx=0 tx=0).stop 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.104+0000 7f84f96dd640 1 -- 172.21.15.139:0/836837335 shutdown_connections 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.104+0000 7f84f96dd640 1 --2- 172.21.15.139:0/836837335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84dc0039b0 0x7f84dc003dd0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.104+0000 7f84f96dd640 1 -- 172.21.15.139:0/836837335 >> 172.21.15.139:0/836837335 conn(0x7f84f40b5480 msgr2=0x7f84f40b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.104+0000 7f84f96dd640 1 -- 172.21.15.139:0/836837335 shutdown_connections 2024-04-25T10:02:52.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.104+0000 7f84f96dd640 1 -- 172.21.15.139:0/836837335 wait complete. 2024-04-25T10:02:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.106+0000 7f84f96dd640 1 Processor -- start 2024-04-25T10:02:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.106+0000 7f84f96dd640 1 -- start start 2024-04-25T10:02:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.106+0000 7f84f96dd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84dc0039b0 0x7f84f40c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.106+0000 7f84f96dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f415eff0 0x7f84f4156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.106+0000 7f84f96dd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84f4157500 0x7f84f41529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f96dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f415dfd0 con 0x7f84f415eff0 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f96dd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f40d1ee0 con 0x7f84dc0039b0 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f96dd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f415e2e0 con 0x7f84f4157500 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f415eff0 0x7f84f4156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f415eff0 0x7f84f4156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54126/0 (socket says 172.21.15.139:54126) 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f27fc640 1 -- 172.21.15.139:0/4154755016 learned_addr learned my addr 172.21.15.139:0/4154755016 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f37fe640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84f4157500 0x7f84f41529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f2ffd640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84dc0039b0 0x7f84f40c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f27fc640 1 -- 172.21.15.139:0/4154755016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84f4157500 msgr2=0x7f84f41529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f27fc640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84f4157500 0x7f84f41529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f27fc640 1 -- 172.21.15.139:0/4154755016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84dc0039b0 msgr2=0x7f84f40c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f27fc640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84dc0039b0 0x7f84f40c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f27fc640 1 -- 172.21.15.139:0/4154755016 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f41536b0 con 0x7f84f415eff0 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f2ffd640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84dc0039b0 0x7f84f40c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f27fc640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f415eff0 0x7f84f4156fc0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f84e800f600 tx=0x7f84e800fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84db7fe640 1 -- 172.21.15.139:0/4154755016 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8019020 con 0x7f84f415eff0 2024-04-25T10:02:52.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84db7fe640 1 -- 172.21.15.139:0/4154755016 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e8014870 con 0x7f84f415eff0 2024-04-25T10:02:52.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f96dd640 1 -- 172.21.15.139:0/4154755016 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f415e530 con 0x7f84f415eff0 2024-04-25T10:02:52.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84db7fe640 1 -- 172.21.15.139:0/4154755016 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8014c20 con 0x7f84f415eff0 2024-04-25T10:02:52.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.107+0000 7f84f96dd640 1 -- 172.21.15.139:0/4154755016 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f40c98e0 con 0x7f84f415eff0 2024-04-25T10:02:52.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.108+0000 7f84f96dd640 1 -- 172.21.15.139:0/4154755016 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f40cf630 con 0x7f84f415eff0 2024-04-25T10:02:52.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.109+0000 7f84db7fe640 1 -- 172.21.15.139:0/4154755016 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84e801c020 con 0x7f84f415eff0 2024-04-25T10:02:52.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.109+0000 7f84db7fe640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f84e006b4d0 0x7f84e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.109+0000 7f84db7fe640 1 -- 172.21.15.139:0/4154755016 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f84e808aae0 con 0x7f84f415eff0 2024-04-25T10:02:52.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.109+0000 7f84f2ffd640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f84e006b4d0 0x7f84e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.109+0000 7f84f2ffd640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f84e006b4d0 0x7f84e006d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f84e4032000 tx=0x7f84e402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.113+0000 7f84db7fe640 1 -- 172.21.15.139:0/4154755016 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84f40cf630 con 0x7f84f415eff0 2024-04-25T10:02:52.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.319+0000 7f84f96dd640 1 -- 172.21.15.139:0/4154755016 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f84f4157910 con 0x7f84f415eff0 2024-04-25T10:02:52.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.987+0000 7f84db7fe640 1 -- 172.21.15.139:0/4154755016 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v83) v1 ==== 122+0+0 (secure 0 0 0) 0x7f84f4157910 con 0x7f84f415eff0 2024-04-25T10:02:52.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.989+0000 7f84f96dd640 1 -- 172.21.15.139:0/4154755016 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f84e006b4d0 msgr2=0x7f84e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.989+0000 7f84f96dd640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f84e006b4d0 0x7f84e006d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f84e4032000 tx=0x7f84e402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:52.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.990+0000 7f84f96dd640 1 -- 172.21.15.139:0/4154755016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f415eff0 msgr2=0x7f84f4156fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.990+0000 7f84f96dd640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f415eff0 0x7f84f4156fc0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f84e800f600 tx=0x7f84e800fac0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.990+0000 7f84f96dd640 1 -- 172.21.15.139:0/4154755016 shutdown_connections 2024-04-25T10:02:52.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.990+0000 7f84f96dd640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84f415eff0 0x7f84f4156fc0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.990+0000 7f84f96dd640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f84e006b4d0 0x7f84e006d990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.990+0000 7f84f96dd640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84f4157500 0x7f84f41529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.990+0000 7f84f96dd640 1 --2- 172.21.15.139:0/4154755016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84dc0039b0 0x7f84f40c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.990+0000 7f84f96dd640 1 -- 172.21.15.139:0/4154755016 >> 172.21.15.139:0/4154755016 conn(0x7f84f40b5480 msgr2=0x7f84f41583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.990+0000 7f84f96dd640 1 -- 172.21.15.139:0/4154755016 shutdown_connections 2024-04-25T10:02:52.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:52.990+0000 7f84f96dd640 1 -- 172.21.15.139:0/4154755016 wait complete. 2024-04-25T10:02:53.006 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:02:53.006 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:53.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.100+0000 7f9ee7ffe640 1 Processor -- start 2024-04-25T10:02:53.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.100+0000 7f9ee7ffe640 1 -- start start 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.101+0000 7f9ee7ffe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9ee006eda0 0x7f9ee006ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.101+0000 7f9ee7ffe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ee006b0a0 0x7f9ee015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.101+0000 7f9ee7ffe640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee00cd030 con 0x7f9ee015b980 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.101+0000 7f9ee7ffe640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee0070580 con 0x7f9ee006eda0 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.101+0000 7f9ee7ffe640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0070860 con 0x7f9ee006b0a0 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.101+0000 7f9ee5d9a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9ee015b980 0x7f9ee006fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35532/0 (socket says 172.21.15.139:35532) 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.101+0000 7f9ee5d9a640 1 -- 172.21.15.139:0/3469009947 learned_addr learned my addr 172.21.15.139:0/3469009947 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.101+0000 7f9ee659b640 1 --2- 172.21.15.139:0/3469009947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ee006b0a0 0x7f9ee015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.101+0000 7f9ee5599640 1 --2- 172.21.15.139:0/3469009947 >> v2:172.21.15.150:3300/0 conn(0x7f9ee006eda0 0x7f9ee006ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.101+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980953627 0 0) 0x7f9ee00cd030 con 0x7f9ee015b980 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.101+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ecc003520 con 0x7f9ee015b980 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4218081310 0 0) 0x7f9ecc003520 con 0x7f9ee015b980 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ee006b0a0 msgr2=0x7f9ee015bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 --2- 172.21.15.139:0/3469009947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ee006b0a0 0x7f9ee015bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 >> v2:172.21.15.150:3300/0 conn(0x7f9ee006eda0 msgr2=0x7f9ee006ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 --2- 172.21.15.139:0/3469009947 >> v2:172.21.15.150:3300/0 conn(0x7f9ee006eda0 0x7f9ee006ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee0070b40 con 0x7f9ee015b980 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f9edc001670 con 0x7f9ee015b980 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 >> v1:172.21.15.139:6789/0 conn(0x7f9ee015b980 legacy=0x7f9ee006fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 --2- 172.21.15.139:0/3469009947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ecc0039b0 0x7f9ecc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 --2- 172.21.15.139:0/3469009947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ee006b0a0 0x7f9ecc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 --2- 172.21.15.139:0/3469009947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ecc004d40 0x7f9ecc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0070580 con 0x7f9ecc004d40 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0070860 con 0x7f9ecc0039b0 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc00b9c0 con 0x7f9ee006b0a0 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee659b640 1 --2- 172.21.15.139:0/3469009947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ecc004d40 0x7f9ecc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee5d9a640 1 --2- 172.21.15.139:0/3469009947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ecc0039b0 0x7f9ecc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee5599640 1 --2- 172.21.15.139:0/3469009947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ee006b0a0 0x7f9ecc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee659b640 1 -- 172.21.15.139:0/3469009947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ee006b0a0 msgr2=0x7f9ecc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee659b640 1 --2- 172.21.15.139:0/3469009947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ee006b0a0 0x7f9ecc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee659b640 1 -- 172.21.15.139:0/3469009947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ecc0039b0 msgr2=0x7f9ecc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee659b640 1 --2- 172.21.15.139:0/3469009947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ecc0039b0 0x7f9ecc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.102+0000 7f9ee659b640 1 -- 172.21.15.139:0/3469009947 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc003520 con 0x7f9ecc004d40 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.103+0000 7f9ee659b640 1 --2- 172.21.15.139:0/3469009947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ecc004d40 0x7f9ecc0051c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009fc0 tx=0x7f9ecc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270838107d11519b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.103+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8009c10 con 0x7f9ecc004d40 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.103+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed8036070 con 0x7f9ecc004d40 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.103+0000 7f9ee4d98640 1 -- 172.21.15.139:0/3469009947 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8031880 con 0x7f9ecc004d40 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.103+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/3469009947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ecc004d40 msgr2=0x7f9ecc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.103+0000 7f9ee7ffe640 1 --2- 172.21.15.139:0/3469009947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ecc004d40 0x7f9ecc0051c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009fc0 tx=0x7f9ecc005830 comp rx=0 tx=0).stop 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.104+0000 7f9ee659b640 1 -- 172.21.15.139:0/3469009947 reap_dead start 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.104+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/3469009947 shutdown_connections 2024-04-25T10:02:53.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.104+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/3469009947 >> 172.21.15.139:0/3469009947 conn(0x7f9ee006e280 msgr2=0x7f9ee006e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.104+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/3469009947 shutdown_connections 2024-04-25T10:02:53.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.104+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/3469009947 wait complete. 2024-04-25T10:02:53.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.105+0000 7f9ee7ffe640 1 Processor -- start 2024-04-25T10:02:53.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.105+0000 7f9ee7ffe640 1 -- start start 2024-04-25T10:02:53.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee7ffe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ecc004d40 0x7f9ee0159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee7ffe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee0159890 0x7f9ee0150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee7ffe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ee0156b40 0x7f9ee0156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee7ffe640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee015bee0 con 0x7f9ecc004d40 2024-04-25T10:02:53.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee7ffe640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee015bd60 con 0x7f9ee0159890 2024-04-25T10:02:53.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee7ffe640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee015c060 con 0x7f9ee0156b40 2024-04-25T10:02:53.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee5d9a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ecc004d40 0x7f9ee0159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee5d9a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ecc004d40 0x7f9ee0159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54150/0 (socket says 172.21.15.139:54150) 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee5d9a640 1 -- 172.21.15.139:0/787332597 learned_addr learned my addr 172.21.15.139:0/787332597 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee5d9a640 1 -- 172.21.15.139:0/787332597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ee0156b40 msgr2=0x7f9ee0156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee5d9a640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ee0156b40 0x7f9ee0156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee5599640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee0159890 0x7f9ee0150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee5d9a640 1 -- 172.21.15.139:0/787332597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee0159890 msgr2=0x7f9ee0150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee5d9a640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee0159890 0x7f9ee0150380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee5d9a640 1 -- 172.21.15.139:0/787332597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee01f47c0 con 0x7f9ecc004d40 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee5d9a640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ecc004d40 0x7f9ee0159e10 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0072c0 tx=0x7f9edc007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ed67fc640 1 -- 172.21.15.139:0/787332597 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc01a070 con 0x7f9ecc004d40 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ed67fc640 1 -- 172.21.15.139:0/787332597 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc00f8e0 con 0x7f9ecc004d40 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/787332597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee01f4aa0 con 0x7f9ecc004d40 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ed67fc640 1 -- 172.21.15.139:0/787332597 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00fcd0 con 0x7f9ecc004d40 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.106+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/787332597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee01f5060 con 0x7f9ecc004d40 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.107+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/787332597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee0074520 con 0x7f9ecc004d40 2024-04-25T10:02:53.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.108+0000 7f9ed67fc640 1 -- 172.21.15.139:0/787332597 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9edc01d070 con 0x7f9ecc004d40 2024-04-25T10:02:53.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.108+0000 7f9ed67fc640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9ec406b4d0 0x7f9ec406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.108+0000 7f9ed67fc640 1 -- 172.21.15.139:0/787332597 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9edc089d50 con 0x7f9ecc004d40 2024-04-25T10:02:53.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.108+0000 7f9ee5599640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9ec406b4d0 0x7f9ec406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.109+0000 7f9ee5599640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9ec406b4d0 0x7f9ec406d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9ee01f4510 tx=0x7f9ed0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.112+0000 7f9ed67fc640 1 -- 172.21.15.139:0/787332597 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9edc08d030 con 0x7f9ecc004d40 2024-04-25T10:02:53.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.319+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/787332597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ee01508c0 con 0x7f9ecc004d40 2024-04-25T10:02:53.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.320+0000 7f9ed67fc640 1 -- 172.21.15.139:0/787332597 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f9edc058a40 con 0x7f9ecc004d40 2024-04-25T10:02:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:53.325 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":83,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3478297010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3478297010},{"type":"v1","addr":"172.21.15.139:6836","nonce":3478297010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:02:52.320205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9132},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9132":{"gid":9132,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/1377607596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":1377607596},{"type":"v1","addr":"172.21.15.150:6835","nonce":1377607596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:53.325 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 83 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.322+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/787332597 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9ec406b4d0 msgr2=0x7f9ec406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.322+0000 7f9ee7ffe640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9ec406b4d0 0x7f9ec406d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9ee01f4510 tx=0x7f9ed0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.322+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/787332597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ecc004d40 msgr2=0x7f9ee0159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.322+0000 7f9ee7ffe640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ecc004d40 0x7f9ee0159e10 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0072c0 tx=0x7f9edc007780 comp rx=0 tx=0).stop 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.322+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/787332597 shutdown_connections 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.322+0000 7f9ee7ffe640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f9ec406b4d0 0x7f9ec406d990 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.322+0000 7f9ee7ffe640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ee0156b40 0x7f9ee0156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.322+0000 7f9ee7ffe640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ee0159890 0x7f9ee0150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.322+0000 7f9ee7ffe640 1 --2- 172.21.15.139:0/787332597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ecc004d40 0x7f9ee0159e10 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.322+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/787332597 >> 172.21.15.139:0/787332597 conn(0x7f9ee006e280 msgr2=0x7f9ee00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.322+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/787332597 shutdown_connections 2024-04-25T10:02:53.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.322+0000 7f9ee7ffe640 1 -- 172.21.15.139:0/787332597 wait complete. 2024-04-25T10:02:53.338 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:53.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.430+0000 7fa4b6e88640 1 Processor -- start 2024-04-25T10:02:53.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.430+0000 7fa4b6e88640 1 -- start start 2024-04-25T10:02:53.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.431+0000 7fa4b6e88640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa4b015aa00 0x7fa4b00b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.431+0000 7fa4b6e88640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa4b00b5110 0x7fa4b015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.431+0000 7fa4b6e88640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b015e500 con 0x7fa4b0072280 2024-04-25T10:02:53.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.431+0000 7fa4b6e88640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4b00b5c20 con 0x7fa4b00b5110 2024-04-25T10:02:53.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.431+0000 7fa4b6e88640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4b00b45d0 con 0x7fa4b015aa00 2024-04-25T10:02:53.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.431+0000 7fa4affff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa4b00b5110 0x7fa4b015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.431+0000 7fa4b5425640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa4b0072280 0x7fa4b00ce610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35542/0 (socket says 172.21.15.139:35542) 2024-04-25T10:02:53.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.431+0000 7fa4b5425640 1 -- 172.21.15.139:0/2041172750 learned_addr learned my addr 172.21.15.139:0/2041172750 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:53.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.432+0000 7fa4b4c24640 1 --2- 172.21.15.139:0/2041172750 >> v2:172.21.15.150:3300/0 conn(0x7fa4b015aa00 0x7fa4b00b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.432+0000 7fa4affff640 1 -- 172.21.15.139:0/2041172750 >> v2:172.21.15.150:3300/0 conn(0x7fa4b015aa00 msgr2=0x7fa4b00b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.432+0000 7fa4affff640 1 --2- 172.21.15.139:0/2041172750 >> v2:172.21.15.150:3300/0 conn(0x7fa4b015aa00 0x7fa4b00b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.432+0000 7fa4affff640 1 -- 172.21.15.139:0/2041172750 >> v1:172.21.15.139:6789/0 conn(0x7fa4b0072280 legacy=0x7fa4b00ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.432+0000 7fa4affff640 1 -- 172.21.15.139:0/2041172750 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b00b5660 con 0x7fa4b00b5110 2024-04-25T10:02:53.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.432+0000 7fa4affff640 1 --2- 172.21.15.139:0/2041172750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b00b5110 0x7fa4b015ecc0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00097a0 tx=0x7fa4a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41227e777ae5907 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.432+0000 7fa4af7fe640 1 -- 172.21.15.139:0/2041172750 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0037070 con 0x7fa4b00b5110 2024-04-25T10:02:53.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.433+0000 7fa4af7fe640 1 -- 172.21.15.139:0/2041172750 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a0032440 con 0x7fa4b00b5110 2024-04-25T10:02:53.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.433+0000 7fa4af7fe640 1 -- 172.21.15.139:0/2041172750 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0032810 con 0x7fa4b00b5110 2024-04-25T10:02:53.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.433+0000 7fa4b6e88640 1 -- 172.21.15.139:0/2041172750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b00b5110 msgr2=0x7fa4b015ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.433+0000 7fa4b6e88640 1 --2- 172.21.15.139:0/2041172750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b00b5110 0x7fa4b015ecc0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00097a0 tx=0x7fa4a002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.433+0000 7fa4b6e88640 1 -- 172.21.15.139:0/2041172750 shutdown_connections 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.433+0000 7fa4b6e88640 1 --2- 172.21.15.139:0/2041172750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b00b5110 0x7fa4b015ecc0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.433+0000 7fa4b6e88640 1 --2- 172.21.15.139:0/2041172750 >> v2:172.21.15.150:3300/0 conn(0x7fa4b015aa00 0x7fa4b00b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.433+0000 7fa4b6e88640 1 -- 172.21.15.139:0/2041172750 >> 172.21.15.139:0/2041172750 conn(0x7fa4b006e280 msgr2=0x7fa4b006e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.433+0000 7fa4b6e88640 1 -- 172.21.15.139:0/2041172750 shutdown_connections 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.433+0000 7fa4b6e88640 1 -- 172.21.15.139:0/2041172750 wait complete. 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.434+0000 7fa4b6e88640 1 Processor -- start 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.434+0000 7fa4b6e88640 1 -- start start 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.435+0000 7fa4b6e88640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b0072280 0x7fa4b00ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.435+0000 7fa4b6e88640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4b00b5110 0x7fa4b01519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.435+0000 7fa4b6e88640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b01424f0 0x7fa4b0151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.435+0000 7fa4b6e88640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b015f200 con 0x7fa4b01424f0 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.435+0000 7fa4b6e88640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b015e7a0 con 0x7fa4b0072280 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.435+0000 7fa4b6e88640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b0072730 con 0x7fa4b00b5110 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.435+0000 7fa4b5425640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b01424f0 0x7fa4b0151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.435+0000 7fa4b5425640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b01424f0 0x7fa4b0151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54172/0 (socket says 172.21.15.139:54172) 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.435+0000 7fa4b5425640 1 -- 172.21.15.139:0/183378314 learned_addr learned my addr 172.21.15.139:0/183378314 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.435+0000 7fa4b4c24640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b0072280 0x7fa4b00ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.435+0000 7fa4affff640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4b00b5110 0x7fa4b01519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.436+0000 7fa4b5425640 1 -- 172.21.15.139:0/183378314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4b00b5110 msgr2=0x7fa4b01519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.436+0000 7fa4b5425640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4b00b5110 0x7fa4b01519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.436+0000 7fa4b5425640 1 -- 172.21.15.139:0/183378314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b0072280 msgr2=0x7fa4b00ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.436+0000 7fa4b5425640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b0072280 0x7fa4b00ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.436+0000 7fa4b5425640 1 -- 172.21.15.139:0/183378314 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b014fb80 con 0x7fa4b01424f0 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.436+0000 7fa4b5425640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b01424f0 0x7fa4b0151f20 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa49800c420 tx=0x7fa49800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.436+0000 7fa4ad7fa640 1 -- 172.21.15.139:0/183378314 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498017070 con 0x7fa4b01424f0 2024-04-25T10:02:53.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.436+0000 7fa4ad7fa640 1 -- 172.21.15.139:0/183378314 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa498005ce0 con 0x7fa4b01424f0 2024-04-25T10:02:53.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.436+0000 7fa4b6e88640 1 -- 172.21.15.139:0/183378314 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b00cc200 con 0x7fa4b01424f0 2024-04-25T10:02:53.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.436+0000 7fa4ad7fa640 1 -- 172.21.15.139:0/183378314 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4980072d0 con 0x7fa4b01424f0 2024-04-25T10:02:53.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.436+0000 7fa4b6e88640 1 -- 172.21.15.139:0/183378314 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b0150100 con 0x7fa4b01424f0 2024-04-25T10:02:53.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.437+0000 7fa4b6e88640 1 -- 172.21.15.139:0/183378314 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b0074520 con 0x7fa4b01424f0 2024-04-25T10:02:53.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.437+0000 7fa4ad7fa640 1 -- 172.21.15.139:0/183378314 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa49801a070 con 0x7fa4b01424f0 2024-04-25T10:02:53.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.438+0000 7fa4ad7fa640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa49006b4d0 0x7fa49006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.438+0000 7fa4ad7fa640 1 -- 172.21.15.139:0/183378314 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa498089530 con 0x7fa4b01424f0 2024-04-25T10:02:53.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.438+0000 7fa4b4c24640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa49006b4d0 0x7fa49006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.438+0000 7fa4b4c24640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa49006b4d0 0x7fa49006d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8003aa0 tx=0x7fa4a8009830 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.442+0000 7fa4ad7fa640 1 -- 172.21.15.139:0/183378314 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa498057c60 con 0x7fa4b01424f0 2024-04-25T10:02:53.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.614+0000 7fa4b6e88640 1 -- 172.21.15.139:0/183378314 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4b006f180 con 0x7fa4b01424f0 2024-04-25T10:02:53.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.615+0000 7fa4ad7fa640 1 -- 172.21.15.139:0/183378314 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+17000 (secure 0 0 0) 0x7fa49805bd00 con 0x7fa4b01424f0 2024-04-25T10:02:53.620 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:53.620 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":120,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:02:51.983848+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:48.574891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:49.293101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:53.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.617+0000 7fa4b6e88640 1 -- 172.21.15.139:0/183378314 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa49006b4d0 msgr2=0x7fa49006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.617+0000 7fa4b6e88640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa49006b4d0 0x7fa49006d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8003aa0 tx=0x7fa4a8009830 comp rx=0 tx=0).stop 2024-04-25T10:02:53.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.617+0000 7fa4b6e88640 1 -- 172.21.15.139:0/183378314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b01424f0 msgr2=0x7fa4b0151f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.617+0000 7fa4b6e88640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b01424f0 0x7fa4b0151f20 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa49800c420 tx=0x7fa49800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.617+0000 7fa4b6e88640 1 -- 172.21.15.139:0/183378314 shutdown_connections 2024-04-25T10:02:53.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.617+0000 7fa4b6e88640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4b01424f0 0x7fa4b0151f20 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.617+0000 7fa4b6e88640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa49006b4d0 0x7fa49006d990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.617+0000 7fa4b6e88640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4b00b5110 0x7fa4b01519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.617+0000 7fa4b6e88640 1 --2- 172.21.15.139:0/183378314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b0072280 0x7fa4b00ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.617+0000 7fa4b6e88640 1 -- 172.21.15.139:0/183378314 >> 172.21.15.139:0/183378314 conn(0x7fa4b006e280 msgr2=0x7fa4b00b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.617+0000 7fa4b6e88640 1 -- 172.21.15.139:0/183378314 shutdown_connections 2024-04-25T10:02:53.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.618+0000 7fa4b6e88640 1 -- 172.21.15.139:0/183378314 wait complete. 2024-04-25T10:02:53.633 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:02:53.633 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:53.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.725+0000 7fd7ffd0a640 1 Processor -- start 2024-04-25T10:02:53.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.725+0000 7fd7ffd0a640 1 -- start start 2024-04-25T10:02:53.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.726+0000 7fd7ffd0a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd7f815dc60 0x7fd7f80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.726+0000 7fd7ffd0a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd7f8161850 0x7fd7f80ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.726+0000 7fd7ffd0a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f80cf250 con 0x7fd7f80d0970 2024-04-25T10:02:53.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.726+0000 7fd7ffd0a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f80ce780 con 0x7fd7f815dc60 2024-04-25T10:02:53.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.726+0000 7fd7ffd0a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f80d3730 con 0x7fd7f8161850 2024-04-25T10:02:53.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.726+0000 7fd7fdaa6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd7f815dc60 0x7fd7f80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.726+0000 7fd7fe2a7640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd7f80d0970 0x7fd7f80d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35544/0 (socket says 172.21.15.139:35544) 2024-04-25T10:02:53.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.726+0000 7fd7fe2a7640 1 -- 172.21.15.139:0/2518383558 learned_addr learned my addr 172.21.15.139:0/2518383558 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:53.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.727+0000 7fd7fdaa6640 1 -- 172.21.15.139:0/2518383558 >> v2:172.21.15.150:3300/0 conn(0x7fd7f8161850 msgr2=0x7fd7f80ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.727+0000 7fd7fdaa6640 1 --2- 172.21.15.139:0/2518383558 >> v2:172.21.15.150:3300/0 conn(0x7fd7f8161850 0x7fd7f80ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.727+0000 7fd7fdaa6640 1 -- 172.21.15.139:0/2518383558 >> v1:172.21.15.139:6789/0 conn(0x7fd7f80d0970 legacy=0x7fd7f80d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.727+0000 7fd7fdaa6640 1 -- 172.21.15.139:0/2518383558 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80ce9b0 con 0x7fd7f815dc60 2024-04-25T10:02:53.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.727+0000 7fd7fcaa4640 1 -- 172.21.15.139:0/2518383558 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330170841 0 0) 0x7fd7f0001110 con 0x7fd7f80d0970 2024-04-25T10:02:53.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.727+0000 7fd7fdaa6640 1 --2- 172.21.15.139:0/2518383558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7f815dc60 0x7fd7f80b6c10 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8068750 tx=0x7fd7f402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea269e357256ab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.727+0000 7fd7fcaa4640 1 -- 172.21.15.139:0/2518383558 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4007740 con 0x7fd7f815dc60 2024-04-25T10:02:53.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.728+0000 7fd7fcaa4640 1 -- 172.21.15.139:0/2518383558 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f40078e0 con 0x7fd7f815dc60 2024-04-25T10:02:53.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.728+0000 7fd7fcaa4640 1 -- 172.21.15.139:0/2518383558 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4007cb0 con 0x7fd7f815dc60 2024-04-25T10:02:53.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.728+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/2518383558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7f815dc60 msgr2=0x7fd7f80b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.728+0000 7fd7ffd0a640 1 --2- 172.21.15.139:0/2518383558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7f815dc60 0x7fd7f80b6c10 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8068750 tx=0x7fd7f402a680 comp rx=0 tx=0).stop 2024-04-25T10:02:53.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.728+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/2518383558 shutdown_connections 2024-04-25T10:02:53.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.728+0000 7fd7ffd0a640 1 --2- 172.21.15.139:0/2518383558 >> v2:172.21.15.150:3300/0 conn(0x7fd7f8161850 0x7fd7f80ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.728+0000 7fd7ffd0a640 1 --2- 172.21.15.139:0/2518383558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7f815dc60 0x7fd7f80b6c10 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.728+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/2518383558 >> 172.21.15.139:0/2518383558 conn(0x7fd7f80b5910 msgr2=0x7fd7f80b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.728+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/2518383558 shutdown_connections 2024-04-25T10:02:53.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.728+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/2518383558 wait complete. 2024-04-25T10:02:53.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.729+0000 7fd7ffd0a640 1 Processor -- start 2024-04-25T10:02:53.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.729+0000 7fd7ffd0a640 1 -- start start 2024-04-25T10:02:53.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7ffd0a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f80d0970 0x7fd7f81f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7ffd0a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f815dc60 0x7fd7f80c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7ffd0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7f8161850 0x7fd7f80cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7ffd0a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80d1fc0 con 0x7fd7f8161850 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7ffd0a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80cef70 con 0x7fd7f815dc60 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7ffd0a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f815e040 con 0x7fd7f80d0970 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7fe2a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7f8161850 0x7fd7f80cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7fe2a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7f8161850 0x7fd7f80cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54196/0 (socket says 172.21.15.139:54196) 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7fe2a7640 1 -- 172.21.15.139:0/3192123169 learned_addr learned my addr 172.21.15.139:0/3192123169 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7fe2a7640 1 -- 172.21.15.139:0/3192123169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f80d0970 msgr2=0x7fd7f81f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7fe2a7640 1 --2- 172.21.15.139:0/3192123169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f80d0970 0x7fd7f81f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7fe2a7640 1 -- 172.21.15.139:0/3192123169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f815dc60 msgr2=0x7fd7f80c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7fe2a7640 1 --2- 172.21.15.139:0/3192123169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f815dc60 0x7fd7f80c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.730+0000 7fd7fe2a7640 1 -- 172.21.15.139:0/3192123169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80c8e80 con 0x7fd7f8161850 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.731+0000 7fd7fe2a7640 1 --2- 172.21.15.139:0/3192123169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7f8161850 0x7fd7f80cab90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0001eb0 tx=0x7fd7f000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.731+0000 7fd7ee7fc640 1 -- 172.21.15.139:0/3192123169 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f00074e0 con 0x7fd7f8161850 2024-04-25T10:02:53.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.731+0000 7fd7ee7fc640 1 -- 172.21.15.139:0/3192123169 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f00047e0 con 0x7fd7f8161850 2024-04-25T10:02:53.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.731+0000 7fd7ee7fc640 1 -- 172.21.15.139:0/3192123169 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0004bb0 con 0x7fd7f8161850 2024-04-25T10:02:53.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.731+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/3192123169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f80c9160 con 0x7fd7f8161850 2024-04-25T10:02:53.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.731+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/3192123169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f80c9720 con 0x7fd7f8161850 2024-04-25T10:02:53.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.732+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/3192123169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f8074520 con 0x7fd7f8161850 2024-04-25T10:02:53.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.732+0000 7fd7ee7fc640 1 -- 172.21.15.139:0/3192123169 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7f0011070 con 0x7fd7f8161850 2024-04-25T10:02:53.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.732+0000 7fd7ee7fc640 1 --2- 172.21.15.139:0/3192123169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7dc06b4d0 0x7fd7dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.733+0000 7fd7ee7fc640 1 -- 172.21.15.139:0/3192123169 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7f0089fe0 con 0x7fd7f8161850 2024-04-25T10:02:53.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.733+0000 7fd7fdaa6640 1 --2- 172.21.15.139:0/3192123169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7dc06b4d0 0x7fd7dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.733+0000 7fd7fdaa6640 1 --2- 172.21.15.139:0/3192123169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7dc06b4d0 0x7fd7dc06d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4034000 tx=0x7fd7f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.737+0000 7fd7ee7fc640 1 -- 172.21.15.139:0/3192123169 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7f80cf250 con 0x7fd7f8161850 2024-04-25T10:02:53.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.902+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/3192123169 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7f815cce0 con 0x7fd7dc06b4d0 2024-04-25T10:02:53.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.908+0000 7fd7ee7fc640 1 -- 172.21.15.139:0/3192123169 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7fd7f815cce0 con 0x7fd7dc06b4d0 2024-04-25T10:02:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:53.917 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T10:02:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.911+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/3192123169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7dc06b4d0 msgr2=0x7fd7dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.911+0000 7fd7ffd0a640 1 --2- 172.21.15.139:0/3192123169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7dc06b4d0 0x7fd7dc06d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4034000 tx=0x7fd7f402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.911+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/3192123169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7f8161850 msgr2=0x7fd7f80cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.911+0000 7fd7ffd0a640 1 --2- 172.21.15.139:0/3192123169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7f8161850 0x7fd7f80cab90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0001eb0 tx=0x7fd7f000c870 comp rx=0 tx=0).stop 2024-04-25T10:02:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.911+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/3192123169 shutdown_connections 2024-04-25T10:02:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.911+0000 7fd7ffd0a640 1 --2- 172.21.15.139:0/3192123169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7f8161850 0x7fd7f80cab90 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.911+0000 7fd7ffd0a640 1 --2- 172.21.15.139:0/3192123169 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd7dc06b4d0 0x7fd7dc06d990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.911+0000 7fd7ffd0a640 1 --2- 172.21.15.139:0/3192123169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f815dc60 0x7fd7f80c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.911+0000 7fd7ffd0a640 1 --2- 172.21.15.139:0/3192123169 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f80d0970 0x7fd7f81f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.911+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/3192123169 >> 172.21.15.139:0/3192123169 conn(0x7fd7f80b5910 msgr2=0x7fd7f80b71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.911+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/3192123169 shutdown_connections 2024-04-25T10:02:53.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:53.911+0000 7fd7ffd0a640 1 -- 172.21.15.139:0/3192123169 wait complete. 2024-04-25T10:02:53.926 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-04-25T10:02:53.100569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262392,"kb_used_data":12904,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712328,"statfs":{"total":773094113280,"available":772825423872,"internally_reserved":0,"allocated":13213696,"data_stored":6769436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.134420"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063424+0000","last_change":"2024-04-25T10:02:50.995986+0000","last_active":"2024-04-25T10:02:52.063424+0000","last_peered":"2024-04-25T10:02:52.063424+0000","last_clean":"2024-04-25T10:02:52.063424+0000","last_became_active":"2024-04-25T10:02:50.995689+0000","last_became_peered":"2024-04-25T10:02:50.995689+0000","last_unstale":"2024-04-25T10:02:52.063424+0000","last_undegraded":"2024-04-25T10:02:52.063424+0000","last_fullsized":"2024-04-25T10:02:52.063424+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:43.530691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063221+0000","last_change":"2024-04-25T10:02:50.998670+0000","last_active":"2024-04-25T10:02:52.063221+0000","last_peered":"2024-04-25T10:02:52.063221+0000","last_clean":"2024-04-25T10:02:52.063221+0000","last_became_active":"2024-04-25T10:02:50.998408+0000","last_became_peered":"2024-04-25T10:02:50.998408+0000","last_unstale":"2024-04-25T10:02:52.063221+0000","last_undegraded":"2024-04-25T10:02:52.063221+0000","last_fullsized":"2024-04-25T10:02:52.063221+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:28.359044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063471+0000","last_change":"2024-04-25T10:02:50.997926+0000","last_active":"2024-04-25T10:02:52.063471+0000","last_peered":"2024-04-25T10:02:52.063471+0000","last_clean":"2024-04-25T10:02:52.063471+0000","last_became_active":"2024-04-25T10:02:50.995445+0000","last_became_peered":"2024-04-25T10:02:50.995445+0000","last_unstale":"2024-04-25T10:02:52.063471+0000","last_undegraded":"2024-04-25T10:02:52.063471+0000","last_fullsized":"2024-04-25T10:02:52.063471+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:20.548894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063476+0000","last_change":"2024-04-25T10:02:51.008205+0000","last_active":"2024-04-25T10:02:52.063476+0000","last_peered":"2024-04-25T10:02:52.063476+0000","last_clean":"2024-04-25T10:02:52.063476+0000","last_became_active":"2024-04-25T10:02:51.007934+0000","last_became_peered":"2024-04-25T10:02:51.007934+0000","last_unstale":"2024-04-25T10:02:52.063476+0000","last_undegraded":"2024-04-25T10:02:52.063476+0000","last_fullsized":"2024-04-25T10:02:52.063476+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:03.428029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063075+0000","last_change":"2024-04-25T10:02:50.995413+0000","last_active":"2024-04-25T10:02:52.063075+0000","last_peered":"2024-04-25T10:02:52.063075+0000","last_clean":"2024-04-25T10:02:52.063075+0000","last_became_active":"2024-04-25T10:02:50.995072+0000","last_became_peered":"2024-04-25T10:02:50.995072+0000","last_unstale":"2024-04-25T10:02:52.063075+0000","last_undegraded":"2024-04-25T10:02:52.063075+0000","last_fullsized":"2024-04-25T10:02:52.063075+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.049687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996093+0000","last_change":"2024-04-25T10:02:50.990984+0000","last_active":"2024-04-25T10:02:51.996093+0000","last_peered":"2024-04-25T10:02:51.996093+0000","last_clean":"2024-04-25T10:02:51.996093+0000","last_became_active":"2024-04-25T10:02:50.990729+0000","last_became_peered":"2024-04-25T10:02:50.990729+0000","last_unstale":"2024-04-25T10:02:51.996093+0000","last_undegraded":"2024-04-25T10:02:51.996093+0000","last_fullsized":"2024-04-25T10:02:51.996093+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:52.920085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996030+0000","last_change":"2024-04-25T10:02:50.997201+0000","last_active":"2024-04-25T10:02:51.996030+0000","last_peered":"2024-04-25T10:02:51.996030+0000","last_clean":"2024-04-25T10:02:51.996030+0000","last_became_active":"2024-04-25T10:02:50.996899+0000","last_became_peered":"2024-04-25T10:02:50.996899+0000","last_unstale":"2024-04-25T10:02:51.996030+0000","last_undegraded":"2024-04-25T10:02:51.996030+0000","last_fullsized":"2024-04-25T10:02:51.996030+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:50.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062525+0000","last_change":"2024-04-25T10:02:50.988718+0000","last_active":"2024-04-25T10:02:52.062525+0000","last_peered":"2024-04-25T10:02:52.062525+0000","last_clean":"2024-04-25T10:02:52.062525+0000","last_became_active":"2024-04-25T10:02:50.988322+0000","last_became_peered":"2024-04-25T10:02:50.988322+0000","last_unstale":"2024-04-25T10:02:52.062525+0000","last_undegraded":"2024-04-25T10:02:52.062525+0000","last_fullsized":"2024-04-25T10:02:52.062525+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:15.550123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063839+0000","last_change":"2024-04-25T10:02:51.006987+0000","last_active":"2024-04-25T10:02:52.063839+0000","last_peered":"2024-04-25T10:02:52.063839+0000","last_clean":"2024-04-25T10:02:52.063839+0000","last_became_active":"2024-04-25T10:02:51.006639+0000","last_became_peered":"2024-04-25T10:02:51.006639+0000","last_unstale":"2024-04-25T10:02:52.063839+0000","last_undegraded":"2024-04-25T10:02:52.063839+0000","last_fullsized":"2024-04-25T10:02:52.063839+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:59.868522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989284+0000","last_change":"2024-04-25T10:02:50.989951+0000","last_active":"2024-04-25T10:02:51.989284+0000","last_peered":"2024-04-25T10:02:51.989284+0000","last_clean":"2024-04-25T10:02:51.989284+0000","last_became_active":"2024-04-25T10:02:50.989651+0000","last_became_peered":"2024-04-25T10:02:50.989651+0000","last_unstale":"2024-04-25T10:02:51.989284+0000","last_undegraded":"2024-04-25T10:02:51.989284+0000","last_fullsized":"2024-04-25T10:02:51.989284+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:17.989782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063305+0000","last_change":"2024-04-25T10:02:50.997270+0000","last_active":"2024-04-25T10:02:52.063305+0000","last_peered":"2024-04-25T10:02:52.063305+0000","last_clean":"2024-04-25T10:02:52.063305+0000","last_became_active":"2024-04-25T10:02:50.997004+0000","last_became_peered":"2024-04-25T10:02:50.997004+0000","last_unstale":"2024-04-25T10:02:52.063305+0000","last_undegraded":"2024-04-25T10:02:52.063305+0000","last_fullsized":"2024-04-25T10:02:52.063305+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:42.494181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996441+0000","last_change":"2024-04-25T10:02:50.994299+0000","last_active":"2024-04-25T10:02:51.996441+0000","last_peered":"2024-04-25T10:02:51.996441+0000","last_clean":"2024-04-25T10:02:51.996441+0000","last_became_active":"2024-04-25T10:02:50.993997+0000","last_became_peered":"2024-04-25T10:02:50.993997+0000","last_unstale":"2024-04-25T10:02:51.996441+0000","last_undegraded":"2024-04-25T10:02:51.996441+0000","last_fullsized":"2024-04-25T10:02:51.996441+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:48.776004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989281+0000","last_change":"2024-04-25T10:02:50.995444+0000","last_active":"2024-04-25T10:02:51.989281+0000","last_peered":"2024-04-25T10:02:51.989281+0000","last_clean":"2024-04-25T10:02:51.989281+0000","last_became_active":"2024-04-25T10:02:50.995182+0000","last_became_peered":"2024-04-25T10:02:50.995182+0000","last_unstale":"2024-04-25T10:02:51.989281+0000","last_undegraded":"2024-04-25T10:02:51.989281+0000","last_fullsized":"2024-04-25T10:02:51.989281+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:32.458571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063171+0000","last_change":"2024-04-25T10:02:50.997195+0000","last_active":"2024-04-25T10:02:52.063171+0000","last_peered":"2024-04-25T10:02:52.063171+0000","last_clean":"2024-04-25T10:02:52.063171+0000","last_became_active":"2024-04-25T10:02:50.996889+0000","last_became_peered":"2024-04-25T10:02:50.996889+0000","last_unstale":"2024-04-25T10:02:52.063171+0000","last_undegraded":"2024-04-25T10:02:52.063171+0000","last_fullsized":"2024-04-25T10:02:52.063171+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:22.666439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062856+0000","last_change":"2024-04-25T10:02:51.004810+0000","last_active":"2024-04-25T10:02:52.062856+0000","last_peered":"2024-04-25T10:02:52.062856+0000","last_clean":"2024-04-25T10:02:52.062856+0000","last_became_active":"2024-04-25T10:02:51.004545+0000","last_became_peered":"2024-04-25T10:02:51.004545+0000","last_unstale":"2024-04-25T10:02:52.062856+0000","last_undegraded":"2024-04-25T10:02:52.062856+0000","last_fullsized":"2024-04-25T10:02:52.062856+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:08.588569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.995974+0000","last_change":"2024-04-25T10:02:50.987317+0000","last_active":"2024-04-25T10:02:51.995974+0000","last_peered":"2024-04-25T10:02:51.995974+0000","last_clean":"2024-04-25T10:02:51.995974+0000","last_became_active":"2024-04-25T10:02:50.987040+0000","last_became_peered":"2024-04-25T10:02:50.987040+0000","last_unstale":"2024-04-25T10:02:51.995974+0000","last_undegraded":"2024-04-25T10:02:51.995974+0000","last_fullsized":"2024-04-25T10:02:51.995974+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:13.916753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989271+0000","last_change":"2024-04-25T10:02:51.004140+0000","last_active":"2024-04-25T10:02:51.989271+0000","last_peered":"2024-04-25T10:02:51.989271+0000","last_clean":"2024-04-25T10:02:51.989271+0000","last_became_active":"2024-04-25T10:02:51.003868+0000","last_became_peered":"2024-04-25T10:02:51.003868+0000","last_unstale":"2024-04-25T10:02:51.989271+0000","last_undegraded":"2024-04-25T10:02:51.989271+0000","last_fullsized":"2024-04-25T10:02:51.989271+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:08.588569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063358+0000","last_change":"2024-04-25T10:02:50.997881+0000","last_active":"2024-04-25T10:02:52.063358+0000","last_peered":"2024-04-25T10:02:52.063358+0000","last_clean":"2024-04-25T10:02:52.063358+0000","last_became_active":"2024-04-25T10:02:50.997604+0000","last_became_peered":"2024-04-25T10:02:50.997604+0000","last_unstale":"2024-04-25T10:02:52.063358+0000","last_undegraded":"2024-04-25T10:02:52.063358+0000","last_fullsized":"2024-04-25T10:02:52.063358+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:59.868522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994458+0000","last_change":"2024-04-25T10:02:50.997944+0000","last_active":"2024-04-25T10:02:51.994458+0000","last_peered":"2024-04-25T10:02:51.994458+0000","last_clean":"2024-04-25T10:02:51.994458+0000","last_became_active":"2024-04-25T10:02:50.997496+0000","last_became_peered":"2024-04-25T10:02:50.997496+0000","last_unstale":"2024-04-25T10:02:51.994458+0000","last_undegraded":"2024-04-25T10:02:51.994458+0000","last_fullsized":"2024-04-25T10:02:51.994458+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.049687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062541+0000","last_change":"2024-04-25T10:02:50.987510+0000","last_active":"2024-04-25T10:02:52.062541+0000","last_peered":"2024-04-25T10:02:52.062541+0000","last_clean":"2024-04-25T10:02:52.062541+0000","last_became_active":"2024-04-25T10:02:50.987222+0000","last_became_peered":"2024-04-25T10:02:50.987222+0000","last_unstale":"2024-04-25T10:02:52.062541+0000","last_undegraded":"2024-04-25T10:02:52.062541+0000","last_fullsized":"2024-04-25T10:02:52.062541+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:57.363879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.064041+0000","last_change":"2024-04-25T10:02:51.004739+0000","last_active":"2024-04-25T10:02:52.064041+0000","last_peered":"2024-04-25T10:02:52.064041+0000","last_clean":"2024-04-25T10:02:52.064041+0000","last_became_active":"2024-04-25T10:02:51.004461+0000","last_became_peered":"2024-04-25T10:02:51.004461+0000","last_unstale":"2024-04-25T10:02:52.064041+0000","last_undegraded":"2024-04-25T10:02:52.064041+0000","last_fullsized":"2024-04-25T10:02:52.064041+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:12.813563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063243+0000","last_change":"2024-04-25T10:02:51.005417+0000","last_active":"2024-04-25T10:02:52.063243+0000","last_peered":"2024-04-25T10:02:52.063243+0000","last_clean":"2024-04-25T10:02:52.063243+0000","last_became_active":"2024-04-25T10:02:51.005127+0000","last_became_peered":"2024-04-25T10:02:51.005127+0000","last_unstale":"2024-04-25T10:02:52.063243+0000","last_undegraded":"2024-04-25T10:02:52.063243+0000","last_fullsized":"2024-04-25T10:02:52.063243+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:17.989782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063963+0000","last_change":"2024-04-25T10:02:51.007076+0000","last_active":"2024-04-25T10:02:52.063963+0000","last_peered":"2024-04-25T10:02:52.063963+0000","last_clean":"2024-04-25T10:02:52.063963+0000","last_became_active":"2024-04-25T10:02:51.006789+0000","last_became_peered":"2024-04-25T10:02:51.006789+0000","last_unstale":"2024-04-25T10:02:52.063963+0000","last_undegraded":"2024-04-25T10:02:52.063963+0000","last_fullsized":"2024-04-25T10:02:52.063963+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:31.011518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990082+0000","last_change":"2024-04-25T10:02:51.004073+0000","last_active":"2024-04-25T10:02:51.990082+0000","last_peered":"2024-04-25T10:02:51.990082+0000","last_clean":"2024-04-25T10:02:51.990082+0000","last_became_active":"2024-04-25T10:02:51.003786+0000","last_became_peered":"2024-04-25T10:02:51.003786+0000","last_unstale":"2024-04-25T10:02:51.990082+0000","last_undegraded":"2024-04-25T10:02:51.990082+0000","last_fullsized":"2024-04-25T10:02:51.990082+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:59.749858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990512+0000","last_change":"2024-04-25T10:02:51.006118+0000","last_active":"2024-04-25T10:02:51.990512+0000","last_peered":"2024-04-25T10:02:51.990512+0000","last_clean":"2024-04-25T10:02:51.990512+0000","last_became_active":"2024-04-25T10:02:51.005094+0000","last_became_peered":"2024-04-25T10:02:51.005094+0000","last_unstale":"2024-04-25T10:02:51.990512+0000","last_undegraded":"2024-04-25T10:02:51.990512+0000","last_fullsized":"2024-04-25T10:02:51.990512+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:04.656853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062733+0000","last_change":"2024-04-25T10:02:50.992175+0000","last_active":"2024-04-25T10:02:52.062733+0000","last_peered":"2024-04-25T10:02:52.062733+0000","last_clean":"2024-04-25T10:02:52.062733+0000","last_became_active":"2024-04-25T10:02:50.991872+0000","last_became_peered":"2024-04-25T10:02:50.991872+0000","last_unstale":"2024-04-25T10:02:52.062733+0000","last_undegraded":"2024-04-25T10:02:52.062733+0000","last_fullsized":"2024-04-25T10:02:52.062733+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:03.225652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994465+0000","last_change":"2024-04-25T10:02:50.997048+0000","last_active":"2024-04-25T10:02:51.994465+0000","last_peered":"2024-04-25T10:02:51.994465+0000","last_clean":"2024-04-25T10:02:51.994465+0000","last_became_active":"2024-04-25T10:02:50.996776+0000","last_became_peered":"2024-04-25T10:02:50.996776+0000","last_unstale":"2024-04-25T10:02:51.994465+0000","last_undegraded":"2024-04-25T10:02:51.994465+0000","last_fullsized":"2024-04-25T10:02:51.994465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:17.989782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996768+0000","last_change":"2024-04-25T10:02:51.002291+0000","last_active":"2024-04-25T10:02:51.996768+0000","last_peered":"2024-04-25T10:02:51.996768+0000","last_clean":"2024-04-25T10:02:51.996768+0000","last_became_active":"2024-04-25T10:02:51.002021+0000","last_became_peered":"2024-04-25T10:02:51.002021+0000","last_unstale":"2024-04-25T10:02:51.996768+0000","last_undegraded":"2024-04-25T10:02:51.996768+0000","last_fullsized":"2024-04-25T10:02:51.996768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:16.553269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062801+0000","last_change":"2024-04-25T10:02:50.985488+0000","last_active":"2024-04-25T10:02:52.062801+0000","last_peered":"2024-04-25T10:02:52.062801+0000","last_clean":"2024-04-25T10:02:52.062801+0000","last_became_active":"2024-04-25T10:02:50.985184+0000","last_became_peered":"2024-04-25T10:02:50.985184+0000","last_unstale":"2024-04-25T10:02:52.062801+0000","last_undegraded":"2024-04-25T10:02:52.062801+0000","last_fullsized":"2024-04-25T10:02:52.062801+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:00.911520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063216+0000","last_change":"2024-04-25T10:02:50.998971+0000","last_active":"2024-04-25T10:02:52.063216+0000","last_peered":"2024-04-25T10:02:52.063216+0000","last_clean":"2024-04-25T10:02:52.063216+0000","last_became_active":"2024-04-25T10:02:50.998706+0000","last_became_peered":"2024-04-25T10:02:50.998706+0000","last_unstale":"2024-04-25T10:02:52.063216+0000","last_undegraded":"2024-04-25T10:02:52.063216+0000","last_fullsized":"2024-04-25T10:02:52.063216+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:49.104042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989877+0000","last_change":"2024-04-25T10:02:51.001919+0000","last_active":"2024-04-25T10:02:51.989877+0000","last_peered":"2024-04-25T10:02:51.989877+0000","last_clean":"2024-04-25T10:02:51.989877+0000","last_became_active":"2024-04-25T10:02:51.001532+0000","last_became_peered":"2024-04-25T10:02:51.001532+0000","last_unstale":"2024-04-25T10:02:51.989877+0000","last_undegraded":"2024-04-25T10:02:51.989877+0000","last_fullsized":"2024-04-25T10:02:51.989877+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:42.494181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989730+0000","last_change":"2024-04-25T10:02:51.003757+0000","last_active":"2024-04-25T10:02:51.989730+0000","last_peered":"2024-04-25T10:02:51.989730+0000","last_clean":"2024-04-25T10:02:51.989730+0000","last_became_active":"2024-04-25T10:02:51.003434+0000","last_became_peered":"2024-04-25T10:02:51.003434+0000","last_unstale":"2024-04-25T10:02:51.989730+0000","last_undegraded":"2024-04-25T10:02:51.989730+0000","last_fullsized":"2024-04-25T10:02:51.989730+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:17.989782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989318+0000","last_change":"2024-04-25T10:02:50.988133+0000","last_active":"2024-04-25T10:02:51.989318+0000","last_peered":"2024-04-25T10:02:51.989318+0000","last_clean":"2024-04-25T10:02:51.989318+0000","last_became_active":"2024-04-25T10:02:50.987681+0000","last_became_peered":"2024-04-25T10:02:50.987681+0000","last_unstale":"2024-04-25T10:02:51.989318+0000","last_undegraded":"2024-04-25T10:02:51.989318+0000","last_fullsized":"2024-04-25T10:02:51.989318+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:42.494181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"119'4","reported_seq":18,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994230+0000","last_change":"2024-04-25T10:02:49.997081+0000","last_active":"2024-04-25T10:02:51.994230+0000","last_peered":"2024-04-25T10:02:51.994230+0000","last_clean":"2024-04-25T10:02:51.994230+0000","last_became_active":"2024-04-25T10:02:49.996786+0000","last_became_peered":"2024-04-25T10:02:49.996786+0000","last_unstale":"2024-04-25T10:02:51.994230+0000","last_undegraded":"2024-04-25T10:02:51.994230+0000","last_fullsized":"2024-04-25T10:02:51.994230+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.201776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996380+0000","last_change":"2024-04-25T10:02:50.990258+0000","last_active":"2024-04-25T10:02:51.996380+0000","last_peered":"2024-04-25T10:02:51.996380+0000","last_clean":"2024-04-25T10:02:51.996380+0000","last_became_active":"2024-04-25T10:02:50.989960+0000","last_became_peered":"2024-04-25T10:02:50.989960+0000","last_unstale":"2024-04-25T10:02:51.996380+0000","last_undegraded":"2024-04-25T10:02:51.996380+0000","last_fullsized":"2024-04-25T10:02:51.996380+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:52.987232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989216+0000","last_change":"2024-04-25T10:02:49.981186+0000","last_active":"2024-04-25T10:02:51.989216+0000","last_peered":"2024-04-25T10:02:51.989216+0000","last_clean":"2024-04-25T10:02:51.989216+0000","last_became_active":"2024-04-25T10:02:49.980876+0000","last_became_peered":"2024-04-25T10:02:49.980876+0000","last_unstale":"2024-04-25T10:02:51.989216+0000","last_undegraded":"2024-04-25T10:02:51.989216+0000","last_fullsized":"2024-04-25T10:02:51.989216+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:23.806266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990211+0000","last_change":"2024-04-25T10:02:51.002808+0000","last_active":"2024-04-25T10:02:51.990211+0000","last_peered":"2024-04-25T10:02:51.990211+0000","last_clean":"2024-04-25T10:02:51.990211+0000","last_became_active":"2024-04-25T10:02:51.001989+0000","last_became_peered":"2024-04-25T10:02:51.001989+0000","last_unstale":"2024-04-25T10:02:51.990211+0000","last_undegraded":"2024-04-25T10:02:51.990211+0000","last_fullsized":"2024-04-25T10:02:51.990211+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:27.223456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063807+0000","last_change":"2024-04-25T10:02:49.993341+0000","last_active":"2024-04-25T10:02:52.063807+0000","last_peered":"2024-04-25T10:02:52.063807+0000","last_clean":"2024-04-25T10:02:52.063807+0000","last_became_active":"2024-04-25T10:02:49.992880+0000","last_became_peered":"2024-04-25T10:02:49.992880+0000","last_unstale":"2024-04-25T10:02:52.063807+0000","last_undegraded":"2024-04-25T10:02:52.063807+0000","last_fullsized":"2024-04-25T10:02:52.063807+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:07.744516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063974+0000","last_change":"2024-04-25T10:02:50.999516+0000","last_active":"2024-04-25T10:02:52.063974+0000","last_peered":"2024-04-25T10:02:52.063974+0000","last_clean":"2024-04-25T10:02:52.063974+0000","last_became_active":"2024-04-25T10:02:50.999228+0000","last_became_peered":"2024-04-25T10:02:50.999228+0000","last_unstale":"2024-04-25T10:02:52.063974+0000","last_undegraded":"2024-04-25T10:02:52.063974+0000","last_fullsized":"2024-04-25T10:02:52.063974+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:42.323390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989196+0000","last_change":"2024-04-25T10:02:49.987633+0000","last_active":"2024-04-25T10:02:51.989196+0000","last_peered":"2024-04-25T10:02:51.989196+0000","last_clean":"2024-04-25T10:02:51.989196+0000","last_became_active":"2024-04-25T10:02:49.987375+0000","last_became_peered":"2024-04-25T10:02:49.987375+0000","last_unstale":"2024-04-25T10:02:51.989196+0000","last_undegraded":"2024-04-25T10:02:51.989196+0000","last_fullsized":"2024-04-25T10:02:51.989196+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:59.622317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989239+0000","last_change":"2024-04-25T10:02:50.995905+0000","last_active":"2024-04-25T10:02:51.989239+0000","last_peered":"2024-04-25T10:02:51.989239+0000","last_clean":"2024-04-25T10:02:51.989239+0000","last_became_active":"2024-04-25T10:02:50.995657+0000","last_became_peered":"2024-04-25T10:02:50.995657+0000","last_unstale":"2024-04-25T10:02:51.989239+0000","last_undegraded":"2024-04-25T10:02:51.989239+0000","last_fullsized":"2024-04-25T10:02:51.989239+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:32.421018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063078+0000","last_change":"2024-04-25T10:02:49.992479+0000","last_active":"2024-04-25T10:02:52.063078+0000","last_peered":"2024-04-25T10:02:52.063078+0000","last_clean":"2024-04-25T10:02:52.063078+0000","last_became_active":"2024-04-25T10:02:49.992191+0000","last_became_peered":"2024-04-25T10:02:49.992191+0000","last_unstale":"2024-04-25T10:02:52.063078+0000","last_undegraded":"2024-04-25T10:02:52.063078+0000","last_fullsized":"2024-04-25T10:02:52.063078+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:05.609417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062902+0000","last_change":"2024-04-25T10:02:50.998341+0000","last_active":"2024-04-25T10:02:52.062902+0000","last_peered":"2024-04-25T10:02:52.062902+0000","last_clean":"2024-04-25T10:02:52.062902+0000","last_became_active":"2024-04-25T10:02:50.998071+0000","last_became_peered":"2024-04-25T10:02:50.998071+0000","last_unstale":"2024-04-25T10:02:52.062902+0000","last_undegraded":"2024-04-25T10:02:52.062902+0000","last_fullsized":"2024-04-25T10:02:52.062902+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:32.458571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996086+0000","last_change":"2024-04-25T10:02:49.994519+0000","last_active":"2024-04-25T10:02:51.996086+0000","last_peered":"2024-04-25T10:02:51.996086+0000","last_clean":"2024-04-25T10:02:51.996086+0000","last_became_active":"2024-04-25T10:02:49.993878+0000","last_became_peered":"2024-04-25T10:02:49.993878+0000","last_unstale":"2024-04-25T10:02:51.996086+0000","last_undegraded":"2024-04-25T10:02:51.996086+0000","last_fullsized":"2024-04-25T10:02:51.996086+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:12.836756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063323+0000","last_change":"2024-04-25T10:02:50.992382+0000","last_active":"2024-04-25T10:02:52.063323+0000","last_peered":"2024-04-25T10:02:52.063323+0000","last_clean":"2024-04-25T10:02:52.063323+0000","last_became_active":"2024-04-25T10:02:50.992125+0000","last_became_peered":"2024-04-25T10:02:50.992125+0000","last_unstale":"2024-04-25T10:02:52.063323+0000","last_undegraded":"2024-04-25T10:02:52.063323+0000","last_fullsized":"2024-04-25T10:02:52.063323+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:47.908802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989818+0000","last_change":"2024-04-25T10:02:49.994358+0000","last_active":"2024-04-25T10:02:51.989818+0000","last_peered":"2024-04-25T10:02:51.989818+0000","last_clean":"2024-04-25T10:02:51.989818+0000","last_became_active":"2024-04-25T10:02:49.993749+0000","last_became_peered":"2024-04-25T10:02:49.993749+0000","last_unstale":"2024-04-25T10:02:51.989818+0000","last_undegraded":"2024-04-25T10:02:51.989818+0000","last_fullsized":"2024-04-25T10:02:51.989818+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:05.609417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989259+0000","last_change":"2024-04-25T10:02:50.990005+0000","last_active":"2024-04-25T10:02:51.989259+0000","last_peered":"2024-04-25T10:02:51.989259+0000","last_clean":"2024-04-25T10:02:51.989259+0000","last_became_active":"2024-04-25T10:02:50.989730+0000","last_became_peered":"2024-04-25T10:02:50.989730+0000","last_unstale":"2024-04-25T10:02:51.989259+0000","last_undegraded":"2024-04-25T10:02:51.989259+0000","last_fullsized":"2024-04-25T10:02:51.989259+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:57.671494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996221+0000","last_change":"2024-04-25T10:02:49.993593+0000","last_active":"2024-04-25T10:02:51.996221+0000","last_peered":"2024-04-25T10:02:51.996221+0000","last_clean":"2024-04-25T10:02:51.996221+0000","last_became_active":"2024-04-25T10:02:49.993217+0000","last_became_peered":"2024-04-25T10:02:49.993217+0000","last_unstale":"2024-04-25T10:02:51.996221+0000","last_undegraded":"2024-04-25T10:02:51.996221+0000","last_fullsized":"2024-04-25T10:02:51.996221+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:21.277770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994301+0000","last_change":"2024-04-25T10:02:50.998732+0000","last_active":"2024-04-25T10:02:51.994301+0000","last_peered":"2024-04-25T10:02:51.994301+0000","last_clean":"2024-04-25T10:02:51.994301+0000","last_became_active":"2024-04-25T10:02:50.998472+0000","last_became_peered":"2024-04-25T10:02:50.998472+0000","last_unstale":"2024-04-25T10:02:51.994301+0000","last_undegraded":"2024-04-25T10:02:51.994301+0000","last_fullsized":"2024-04-25T10:02:51.994301+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:08.588569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994405+0000","last_change":"2024-04-25T10:02:49.996405+0000","last_active":"2024-04-25T10:02:51.994405+0000","last_peered":"2024-04-25T10:02:51.994405+0000","last_clean":"2024-04-25T10:02:51.994405+0000","last_became_active":"2024-04-25T10:02:49.996065+0000","last_became_peered":"2024-04-25T10:02:49.996065+0000","last_unstale":"2024-04-25T10:02:51.994405+0000","last_undegraded":"2024-04-25T10:02:51.994405+0000","last_fullsized":"2024-04-25T10:02:51.994405+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:20.468891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990256+0000","last_change":"2024-04-25T10:02:51.007375+0000","last_active":"2024-04-25T10:02:51.990256+0000","last_peered":"2024-04-25T10:02:51.990256+0000","last_clean":"2024-04-25T10:02:51.990256+0000","last_became_active":"2024-04-25T10:02:51.007128+0000","last_became_peered":"2024-04-25T10:02:51.007128+0000","last_unstale":"2024-04-25T10:02:51.990256+0000","last_undegraded":"2024-04-25T10:02:51.990256+0000","last_fullsized":"2024-04-25T10:02:51.990256+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:13.464999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990308+0000","last_change":"2024-04-25T10:02:49.989926+0000","last_active":"2024-04-25T10:02:51.990308+0000","last_peered":"2024-04-25T10:02:51.990308+0000","last_clean":"2024-04-25T10:02:51.990308+0000","last_became_active":"2024-04-25T10:02:49.989602+0000","last_became_peered":"2024-04-25T10:02:49.989602+0000","last_unstale":"2024-04-25T10:02:51.990308+0000","last_undegraded":"2024-04-25T10:02:51.990308+0000","last_fullsized":"2024-04-25T10:02:51.990308+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:56.075749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063489+0000","last_change":"2024-04-25T10:02:50.995011+0000","last_active":"2024-04-25T10:02:52.063489+0000","last_peered":"2024-04-25T10:02:52.063489+0000","last_clean":"2024-04-25T10:02:52.063489+0000","last_became_active":"2024-04-25T10:02:50.994735+0000","last_became_peered":"2024-04-25T10:02:50.994735+0000","last_unstale":"2024-04-25T10:02:52.063489+0000","last_undegraded":"2024-04-25T10:02:52.063489+0000","last_fullsized":"2024-04-25T10:02:52.063489+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:31.011518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"119'3","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990422+0000","last_change":"2024-04-25T10:02:49.993333+0000","last_active":"2024-04-25T10:02:51.990422+0000","last_peered":"2024-04-25T10:02:51.990422+0000","last_clean":"2024-04-25T10:02:51.990422+0000","last_became_active":"2024-04-25T10:02:49.993041+0000","last_became_peered":"2024-04-25T10:02:49.993041+0000","last_unstale":"2024-04-25T10:02:51.990422+0000","last_undegraded":"2024-04-25T10:02:51.990422+0000","last_fullsized":"2024-04-25T10:02:51.990422+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:44.159091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990571+0000","last_change":"2024-04-25T10:02:50.998617+0000","last_active":"2024-04-25T10:02:51.990571+0000","last_peered":"2024-04-25T10:02:51.990571+0000","last_clean":"2024-04-25T10:02:51.990571+0000","last_became_active":"2024-04-25T10:02:50.998280+0000","last_became_peered":"2024-04-25T10:02:50.998280+0000","last_unstale":"2024-04-25T10:02:51.990571+0000","last_undegraded":"2024-04-25T10:02:51.990571+0000","last_fullsized":"2024-04-25T10:02:51.990571+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:53.414855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"119'6","reported_seq":20,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063139+0000","last_change":"2024-04-25T10:02:49.990625+0000","last_active":"2024-04-25T10:02:52.063139+0000","last_peered":"2024-04-25T10:02:52.063139+0000","last_clean":"2024-04-25T10:02:52.063139+0000","last_became_active":"2024-04-25T10:02:49.990320+0000","last_became_peered":"2024-04-25T10:02:49.990320+0000","last_unstale":"2024-04-25T10:02:52.063139+0000","last_undegraded":"2024-04-25T10:02:52.063139+0000","last_fullsized":"2024-04-25T10:02:52.063139+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:07.581852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994430+0000","last_change":"2024-04-25T10:02:50.995103+0000","last_active":"2024-04-25T10:02:51.994430+0000","last_peered":"2024-04-25T10:02:51.994430+0000","last_clean":"2024-04-25T10:02:51.994430+0000","last_became_active":"2024-04-25T10:02:50.993645+0000","last_became_peered":"2024-04-25T10:02:50.993645+0000","last_unstale":"2024-04-25T10:02:51.994430+0000","last_undegraded":"2024-04-25T10:02:51.994430+0000","last_fullsized":"2024-04-25T10:02:51.994430+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:32.458571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063076+0000","last_change":"2024-04-25T10:02:49.995923+0000","last_active":"2024-04-25T10:02:52.063076+0000","last_peered":"2024-04-25T10:02:52.063076+0000","last_clean":"2024-04-25T10:02:52.063076+0000","last_became_active":"2024-04-25T10:02:49.995634+0000","last_became_peered":"2024-04-25T10:02:49.995634+0000","last_unstale":"2024-04-25T10:02:52.063076+0000","last_undegraded":"2024-04-25T10:02:52.063076+0000","last_fullsized":"2024-04-25T10:02:52.063076+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:16.983066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989239+0000","last_change":"2024-04-25T10:02:50.998371+0000","last_active":"2024-04-25T10:02:51.989239+0000","last_peered":"2024-04-25T10:02:51.989239+0000","last_clean":"2024-04-25T10:02:51.989239+0000","last_became_active":"2024-04-25T10:02:50.998127+0000","last_became_peered":"2024-04-25T10:02:50.998127+0000","last_unstale":"2024-04-25T10:02:51.989239+0000","last_undegraded":"2024-04-25T10:02:51.989239+0000","last_fullsized":"2024-04-25T10:02:51.989239+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:42.230799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063410+0000","last_change":"2024-04-25T10:02:49.991026+0000","last_active":"2024-04-25T10:02:52.063410+0000","last_peered":"2024-04-25T10:02:52.063410+0000","last_clean":"2024-04-25T10:02:52.063410+0000","last_became_active":"2024-04-25T10:02:49.990598+0000","last_became_peered":"2024-04-25T10:02:49.990598+0000","last_unstale":"2024-04-25T10:02:52.063410+0000","last_undegraded":"2024-04-25T10:02:52.063410+0000","last_fullsized":"2024-04-25T10:02:52.063410+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:41.487464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062526+0000","last_change":"2024-04-25T10:02:50.999478+0000","last_active":"2024-04-25T10:02:52.062526+0000","last_peered":"2024-04-25T10:02:52.062526+0000","last_clean":"2024-04-25T10:02:52.062526+0000","last_became_active":"2024-04-25T10:02:50.999182+0000","last_became_peered":"2024-04-25T10:02:50.999182+0000","last_unstale":"2024-04-25T10:02:52.062526+0000","last_undegraded":"2024-04-25T10:02:52.062526+0000","last_fullsized":"2024-04-25T10:02:52.062526+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:04.825603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062840+0000","last_change":"2024-04-25T10:02:49.990146+0000","last_active":"2024-04-25T10:02:52.062840+0000","last_peered":"2024-04-25T10:02:52.062840+0000","last_clean":"2024-04-25T10:02:52.062840+0000","last_became_active":"2024-04-25T10:02:49.989601+0000","last_became_peered":"2024-04-25T10:02:49.989601+0000","last_unstale":"2024-04-25T10:02:52.062840+0000","last_undegraded":"2024-04-25T10:02:52.062840+0000","last_fullsized":"2024-04-25T10:02:52.062840+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:31.451854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990079+0000","last_change":"2024-04-25T10:02:51.002186+0000","last_active":"2024-04-25T10:02:51.990079+0000","last_peered":"2024-04-25T10:02:51.990079+0000","last_clean":"2024-04-25T10:02:51.990079+0000","last_became_active":"2024-04-25T10:02:51.001901+0000","last_became_peered":"2024-04-25T10:02:51.001901+0000","last_unstale":"2024-04-25T10:02:51.990079+0000","last_undegraded":"2024-04-25T10:02:51.990079+0000","last_fullsized":"2024-04-25T10:02:51.990079+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:11.029946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989367+0000","last_change":"2024-04-25T10:02:49.991065+0000","last_active":"2024-04-25T10:02:51.989367+0000","last_peered":"2024-04-25T10:02:51.989367+0000","last_clean":"2024-04-25T10:02:51.989367+0000","last_became_active":"2024-04-25T10:02:49.990673+0000","last_became_peered":"2024-04-25T10:02:49.990673+0000","last_unstale":"2024-04-25T10:02:51.989367+0000","last_undegraded":"2024-04-25T10:02:51.989367+0000","last_fullsized":"2024-04-25T10:02:51.989367+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:48.354977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063480+0000","last_change":"2024-04-25T10:02:51.003430+0000","last_active":"2024-04-25T10:02:52.063480+0000","last_peered":"2024-04-25T10:02:52.063480+0000","last_clean":"2024-04-25T10:02:52.063480+0000","last_became_active":"2024-04-25T10:02:51.003106+0000","last_became_peered":"2024-04-25T10:02:51.003106+0000","last_unstale":"2024-04-25T10:02:52.063480+0000","last_undegraded":"2024-04-25T10:02:52.063480+0000","last_fullsized":"2024-04-25T10:02:52.063480+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:11.029946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062863+0000","last_change":"2024-04-25T10:02:49.987877+0000","last_active":"2024-04-25T10:02:52.062863+0000","last_peered":"2024-04-25T10:02:52.062863+0000","last_clean":"2024-04-25T10:02:52.062863+0000","last_became_active":"2024-04-25T10:02:49.987609+0000","last_became_peered":"2024-04-25T10:02:49.987609+0000","last_unstale":"2024-04-25T10:02:52.062863+0000","last_undegraded":"2024-04-25T10:02:52.062863+0000","last_fullsized":"2024-04-25T10:02:52.062863+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:30.591432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996371+0000","last_change":"2024-04-25T10:02:50.984814+0000","last_active":"2024-04-25T10:02:51.996371+0000","last_peered":"2024-04-25T10:02:51.996371+0000","last_clean":"2024-04-25T10:02:51.996371+0000","last_became_active":"2024-04-25T10:02:50.984546+0000","last_became_peered":"2024-04-25T10:02:50.984546+0000","last_unstale":"2024-04-25T10:02:51.996371+0000","last_undegraded":"2024-04-25T10:02:51.996371+0000","last_fullsized":"2024-04-25T10:02:51.996371+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:42.368731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063915+0000","last_change":"2024-04-25T10:02:49.990902+0000","last_active":"2024-04-25T10:02:52.063915+0000","last_peered":"2024-04-25T10:02:52.063915+0000","last_clean":"2024-04-25T10:02:52.063915+0000","last_became_active":"2024-04-25T10:02:49.990559+0000","last_became_peered":"2024-04-25T10:02:49.990559+0000","last_unstale":"2024-04-25T10:02:52.063915+0000","last_undegraded":"2024-04-25T10:02:52.063915+0000","last_fullsized":"2024-04-25T10:02:52.063915+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:15.147862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990617+0000","last_change":"2024-04-25T10:02:50.995715+0000","last_active":"2024-04-25T10:02:51.990617+0000","last_peered":"2024-04-25T10:02:51.990617+0000","last_clean":"2024-04-25T10:02:51.990617+0000","last_became_active":"2024-04-25T10:02:50.995430+0000","last_became_peered":"2024-04-25T10:02:50.995430+0000","last_unstale":"2024-04-25T10:02:51.990617+0000","last_undegraded":"2024-04-25T10:02:51.990617+0000","last_fullsized":"2024-04-25T10:02:51.990617+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:14.143890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063500+0000","last_change":"2024-04-25T10:02:49.993025+0000","last_active":"2024-04-25T10:02:52.063500+0000","last_peered":"2024-04-25T10:02:52.063500+0000","last_clean":"2024-04-25T10:02:52.063500+0000","last_became_active":"2024-04-25T10:02:49.992731+0000","last_became_peered":"2024-04-25T10:02:49.992731+0000","last_unstale":"2024-04-25T10:02:52.063500+0000","last_undegraded":"2024-04-25T10:02:52.063500+0000","last_fullsized":"2024-04-25T10:02:52.063500+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:13.137173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.995990+0000","last_change":"2024-04-25T10:02:50.999319+0000","last_active":"2024-04-25T10:02:51.995990+0000","last_peered":"2024-04-25T10:02:51.995990+0000","last_clean":"2024-04-25T10:02:51.995990+0000","last_became_active":"2024-04-25T10:02:50.999074+0000","last_became_peered":"2024-04-25T10:02:50.999074+0000","last_unstale":"2024-04-25T10:02:51.995990+0000","last_undegraded":"2024-04-25T10:02:51.995990+0000","last_fullsized":"2024-04-25T10:02:51.995990+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:43.256506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996097+0000","last_change":"2024-04-25T10:02:49.995258+0000","last_active":"2024-04-25T10:02:51.996097+0000","last_peered":"2024-04-25T10:02:51.996097+0000","last_clean":"2024-04-25T10:02:51.996097+0000","last_became_active":"2024-04-25T10:02:49.994296+0000","last_became_peered":"2024-04-25T10:02:49.994296+0000","last_unstale":"2024-04-25T10:02:51.996097+0000","last_undegraded":"2024-04-25T10:02:51.996097+0000","last_fullsized":"2024-04-25T10:02:51.996097+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:05.794611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062616+0000","last_change":"2024-04-25T10:02:51.003659+0000","last_active":"2024-04-25T10:02:52.062616+0000","last_peered":"2024-04-25T10:02:52.062616+0000","last_clean":"2024-04-25T10:02:52.062616+0000","last_became_active":"2024-04-25T10:02:51.003384+0000","last_became_peered":"2024-04-25T10:02:51.003384+0000","last_unstale":"2024-04-25T10:02:52.062616+0000","last_undegraded":"2024-04-25T10:02:52.062616+0000","last_fullsized":"2024-04-25T10:02:52.062616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:53.904384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063337+0000","last_change":"2024-04-25T10:02:49.993830+0000","last_active":"2024-04-25T10:02:52.063337+0000","last_peered":"2024-04-25T10:02:52.063337+0000","last_clean":"2024-04-25T10:02:52.063337+0000","last_became_active":"2024-04-25T10:02:49.993548+0000","last_became_peered":"2024-04-25T10:02:49.993548+0000","last_unstale":"2024-04-25T10:02:52.063337+0000","last_undegraded":"2024-04-25T10:02:52.063337+0000","last_fullsized":"2024-04-25T10:02:52.063337+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:52.408138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063335+0000","last_change":"2024-04-25T10:02:50.996477+0000","last_active":"2024-04-25T10:02:52.063335+0000","last_peered":"2024-04-25T10:02:52.063335+0000","last_clean":"2024-04-25T10:02:52.063335+0000","last_became_active":"2024-04-25T10:02:50.995869+0000","last_became_peered":"2024-04-25T10:02:50.995869+0000","last_unstale":"2024-04-25T10:02:52.063335+0000","last_undegraded":"2024-04-25T10:02:52.063335+0000","last_fullsized":"2024-04-25T10:02:52.063335+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:03.428029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062936+0000","last_change":"2024-04-25T10:02:49.989803+0000","last_active":"2024-04-25T10:02:52.062936+0000","last_peered":"2024-04-25T10:02:52.062936+0000","last_clean":"2024-04-25T10:02:52.062936+0000","last_became_active":"2024-04-25T10:02:49.989240+0000","last_became_peered":"2024-04-25T10:02:49.989240+0000","last_unstale":"2024-04-25T10:02:52.062936+0000","last_undegraded":"2024-04-25T10:02:52.062936+0000","last_fullsized":"2024-04-25T10:02:52.062936+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:58.042970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989983+0000","last_change":"2024-04-25T10:02:50.999171+0000","last_active":"2024-04-25T10:02:51.989983+0000","last_peered":"2024-04-25T10:02:51.989983+0000","last_clean":"2024-04-25T10:02:51.989983+0000","last_became_active":"2024-04-25T10:02:50.998886+0000","last_became_peered":"2024-04-25T10:02:50.998886+0000","last_unstale":"2024-04-25T10:02:51.989983+0000","last_undegraded":"2024-04-25T10:02:51.989983+0000","last_fullsized":"2024-04-25T10:02:51.989983+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:43.530691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063065+0000","last_change":"2024-04-25T10:02:49.994508+0000","last_active":"2024-04-25T10:02:52.063065+0000","last_peered":"2024-04-25T10:02:52.063065+0000","last_clean":"2024-04-25T10:02:52.063065+0000","last_became_active":"2024-04-25T10:02:49.994233+0000","last_became_peered":"2024-04-25T10:02:49.994233+0000","last_unstale":"2024-04-25T10:02:52.063065+0000","last_undegraded":"2024-04-25T10:02:52.063065+0000","last_fullsized":"2024-04-25T10:02:52.063065+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:12.458282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996580+0000","last_change":"2024-04-25T10:02:50.987277+0000","last_active":"2024-04-25T10:02:51.996580+0000","last_peered":"2024-04-25T10:02:51.996580+0000","last_clean":"2024-04-25T10:02:51.996580+0000","last_became_active":"2024-04-25T10:02:50.986974+0000","last_became_peered":"2024-04-25T10:02:50.986974+0000","last_unstale":"2024-04-25T10:02:51.996580+0000","last_undegraded":"2024-04-25T10:02:51.996580+0000","last_fullsized":"2024-04-25T10:02:51.996580+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.024928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996432+0000","last_change":"2024-04-25T10:02:49.995093+0000","last_active":"2024-04-25T10:02:51.996432+0000","last_peered":"2024-04-25T10:02:51.996432+0000","last_clean":"2024-04-25T10:02:51.996432+0000","last_became_active":"2024-04-25T10:02:49.994761+0000","last_became_peered":"2024-04-25T10:02:49.994761+0000","last_unstale":"2024-04-25T10:02:51.996432+0000","last_undegraded":"2024-04-25T10:02:51.996432+0000","last_fullsized":"2024-04-25T10:02:51.996432+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:09.732865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989283+0000","last_change":"2024-04-25T10:02:50.995211+0000","last_active":"2024-04-25T10:02:51.989283+0000","last_peered":"2024-04-25T10:02:51.989283+0000","last_clean":"2024-04-25T10:02:51.989283+0000","last_became_active":"2024-04-25T10:02:50.994953+0000","last_became_peered":"2024-04-25T10:02:50.994953+0000","last_unstale":"2024-04-25T10:02:51.989283+0000","last_undegraded":"2024-04-25T10:02:51.989283+0000","last_fullsized":"2024-04-25T10:02:51.989283+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.049687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"120'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.995706+0000","last_change":"2024-04-25T10:02:49.995142+0000","last_active":"2024-04-25T10:02:51.995706+0000","last_peered":"2024-04-25T10:02:51.995706+0000","last_clean":"2024-04-25T10:02:51.995706+0000","last_became_active":"2024-04-25T10:02:49.994845+0000","last_became_peered":"2024-04-25T10:02:49.994845+0000","last_unstale":"2024-04-25T10:02:51.995706+0000","last_undegraded":"2024-04-25T10:02:51.995706+0000","last_fullsized":"2024-04-25T10:02:51.995706+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:03.283380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063905+0000","last_change":"2024-04-25T10:02:51.008017+0000","last_active":"2024-04-25T10:02:52.063905+0000","last_peered":"2024-04-25T10:02:52.063905+0000","last_clean":"2024-04-25T10:02:52.063905+0000","last_became_active":"2024-04-25T10:02:51.007535+0000","last_became_peered":"2024-04-25T10:02:51.007535+0000","last_unstale":"2024-04-25T10:02:52.063905+0000","last_undegraded":"2024-04-25T10:02:52.063905+0000","last_fullsized":"2024-04-25T10:02:52.063905+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:20.548894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994528+0000","last_change":"2024-04-25T10:02:49.995914+0000","last_active":"2024-04-25T10:02:51.994528+0000","last_peered":"2024-04-25T10:02:51.994528+0000","last_clean":"2024-04-25T10:02:51.994528+0000","last_became_active":"2024-04-25T10:02:49.995574+0000","last_became_peered":"2024-04-25T10:02:49.995574+0000","last_unstale":"2024-04-25T10:02:51.994528+0000","last_undegraded":"2024-04-25T10:02:51.994528+0000","last_fullsized":"2024-04-25T10:02:51.994528+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:21.599505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063402+0000","last_change":"2024-04-25T10:02:51.001795+0000","last_active":"2024-04-25T10:02:52.063402+0000","last_peered":"2024-04-25T10:02:52.063402+0000","last_clean":"2024-04-25T10:02:52.063402+0000","last_became_active":"2024-04-25T10:02:51.001462+0000","last_became_peered":"2024-04-25T10:02:51.001462+0000","last_unstale":"2024-04-25T10:02:52.063402+0000","last_undegraded":"2024-04-25T10:02:52.063402+0000","last_fullsized":"2024-04-25T10:02:52.063402+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:09.590707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996663+0000","last_change":"2024-04-25T10:02:49.992886+0000","last_active":"2024-04-25T10:02:51.996663+0000","last_peered":"2024-04-25T10:02:51.996663+0000","last_clean":"2024-04-25T10:02:51.996663+0000","last_became_active":"2024-04-25T10:02:49.992587+0000","last_became_peered":"2024-04-25T10:02:49.992587+0000","last_unstale":"2024-04-25T10:02:51.996663+0000","last_undegraded":"2024-04-25T10:02:51.996663+0000","last_fullsized":"2024-04-25T10:02:51.996663+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:45.537085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996159+0000","last_change":"2024-04-25T10:02:50.993580+0000","last_active":"2024-04-25T10:02:51.996159+0000","last_peered":"2024-04-25T10:02:51.996159+0000","last_clean":"2024-04-25T10:02:51.996159+0000","last_became_active":"2024-04-25T10:02:50.993284+0000","last_became_peered":"2024-04-25T10:02:50.993284+0000","last_unstale":"2024-04-25T10:02:51.996159+0000","last_undegraded":"2024-04-25T10:02:51.996159+0000","last_fullsized":"2024-04-25T10:02:51.996159+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:41.124412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063601+0000","last_change":"2024-04-25T10:02:49.989201+0000","last_active":"2024-04-25T10:02:52.063601+0000","last_peered":"2024-04-25T10:02:52.063601+0000","last_clean":"2024-04-25T10:02:52.063601+0000","last_became_active":"2024-04-25T10:02:49.988837+0000","last_became_peered":"2024-04-25T10:02:49.988837+0000","last_unstale":"2024-04-25T10:02:52.063601+0000","last_undegraded":"2024-04-25T10:02:52.063601+0000","last_fullsized":"2024-04-25T10:02:52.063601+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.201776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062748+0000","last_change":"2024-04-25T10:02:50.985343+0000","last_active":"2024-04-25T10:02:52.062748+0000","last_peered":"2024-04-25T10:02:52.062748+0000","last_clean":"2024-04-25T10:02:52.062748+0000","last_became_active":"2024-04-25T10:02:50.985077+0000","last_became_peered":"2024-04-25T10:02:50.985077+0000","last_unstale":"2024-04-25T10:02:52.062748+0000","last_undegraded":"2024-04-25T10:02:52.062748+0000","last_fullsized":"2024-04-25T10:02:52.062748+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:11.709040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063367+0000","last_change":"2024-04-25T10:02:49.991022+0000","last_active":"2024-04-25T10:02:52.063367+0000","last_peered":"2024-04-25T10:02:52.063367+0000","last_clean":"2024-04-25T10:02:52.063367+0000","last_became_active":"2024-04-25T10:02:49.990751+0000","last_became_peered":"2024-04-25T10:02:49.990751+0000","last_unstale":"2024-04-25T10:02:52.063367+0000","last_undegraded":"2024-04-25T10:02:52.063367+0000","last_fullsized":"2024-04-25T10:02:52.063367+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:44.159091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063223+0000","last_change":"2024-04-25T10:02:51.001087+0000","last_active":"2024-04-25T10:02:52.063223+0000","last_peered":"2024-04-25T10:02:52.063223+0000","last_clean":"2024-04-25T10:02:52.063223+0000","last_became_active":"2024-04-25T10:02:51.000508+0000","last_became_peered":"2024-04-25T10:02:51.000508+0000","last_unstale":"2024-04-25T10:02:52.063223+0000","last_undegraded":"2024-04-25T10:02:52.063223+0000","last_fullsized":"2024-04-25T10:02:52.063223+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:47.908802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996063+0000","last_change":"2024-04-25T10:02:49.993764+0000","last_active":"2024-04-25T10:02:51.996063+0000","last_peered":"2024-04-25T10:02:51.996063+0000","last_clean":"2024-04-25T10:02:51.996063+0000","last_became_active":"2024-04-25T10:02:49.993250+0000","last_became_peered":"2024-04-25T10:02:49.993250+0000","last_unstale":"2024-04-25T10:02:51.996063+0000","last_undegraded":"2024-04-25T10:02:51.996063+0000","last_fullsized":"2024-04-25T10:02:51.996063+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:59.534880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989402+0000","last_change":"2024-04-25T10:02:50.997672+0000","last_active":"2024-04-25T10:02:51.989402+0000","last_peered":"2024-04-25T10:02:51.989402+0000","last_clean":"2024-04-25T10:02:51.989402+0000","last_became_active":"2024-04-25T10:02:50.997401+0000","last_became_peered":"2024-04-25T10:02:50.997401+0000","last_unstale":"2024-04-25T10:02:51.989402+0000","last_undegraded":"2024-04-25T10:02:51.989402+0000","last_fullsized":"2024-04-25T10:02:51.989402+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:43.530691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063485+0000","last_change":"2024-04-25T10:02:49.991498+0000","last_active":"2024-04-25T10:02:52.063485+0000","last_peered":"2024-04-25T10:02:52.063485+0000","last_clean":"2024-04-25T10:02:52.063485+0000","last_became_active":"2024-04-25T10:02:49.990815+0000","last_became_peered":"2024-04-25T10:02:49.990815+0000","last_unstale":"2024-04-25T10:02:52.063485+0000","last_undegraded":"2024-04-25T10:02:52.063485+0000","last_fullsized":"2024-04-25T10:02:52.063485+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:03.650136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.064058+0000","last_change":"2024-04-25T10:02:51.005215+0000","last_active":"2024-04-25T10:02:52.064058+0000","last_peered":"2024-04-25T10:02:52.064058+0000","last_clean":"2024-04-25T10:02:52.064058+0000","last_became_active":"2024-04-25T10:02:51.004932+0000","last_became_peered":"2024-04-25T10:02:51.004932+0000","last_unstale":"2024-04-25T10:02:52.064058+0000","last_undegraded":"2024-04-25T10:02:52.064058+0000","last_fullsized":"2024-04-25T10:02:52.064058+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:38.716882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"119'8","reported_seq":22,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062602+0000","last_change":"2024-04-25T10:02:49.992124+0000","last_active":"2024-04-25T10:02:52.062602+0000","last_peered":"2024-04-25T10:02:52.062602+0000","last_clean":"2024-04-25T10:02:52.062602+0000","last_became_active":"2024-04-25T10:02:49.991903+0000","last_became_peered":"2024-04-25T10:02:49.991903+0000","last_unstale":"2024-04-25T10:02:52.062602+0000","last_undegraded":"2024-04-25T10:02:52.062602+0000","last_fullsized":"2024-04-25T10:02:52.062602+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:56.664777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":355,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063603+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T10:02:52.063603+0000","last_peered":"2024-04-25T10:02:52.063603+0000","last_clean":"2024-04-25T10:02:52.063603+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T10:02:52.063603+0000","last_undegraded":"2024-04-25T10:02:52.063603+0000","last_fullsized":"2024-04-25T10:02:52.063603+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574970,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2105344,"data_stored":1289129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":698567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1503232,"data_stored":698145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1507328,"data_stored":699597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1511424,"data_stored":699631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1495040,"data_stored":697985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":698007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574970,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2093056,"data_stored":1288375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:53.931 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:54.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.059+0000 7f8ee1d22640 1 Processor -- start 2024-04-25T10:02:54.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.059+0000 7f8ee1d22640 1 -- start start 2024-04-25T10:02:54.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8ee1d22640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8edc15ae40 0x7f8edc0b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8ee1d22640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0cefe0 0x7f8edc06d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8ee1d22640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8edc0ce220 con 0x7f8edc0b0210 2024-04-25T10:02:54.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8ee1d22640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8edc0b1070 con 0x7f8edc15ae40 2024-04-25T10:02:54.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8ee1d22640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0cdff0 con 0x7f8edc0cefe0 2024-04-25T10:02:54.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edb7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8edc0b0210 0x7f8edc0b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35554/0 (socket says 172.21.15.139:35554) 2024-04-25T10:02:54.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edb7fe640 1 -- 172.21.15.139:0/1384575009 learned_addr learned my addr 172.21.15.139:0/1384575009 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:54.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edbfff640 1 --2- 172.21.15.139:0/1384575009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0cefe0 0x7f8edc06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8ed3fff640 1 --2- 172.21.15.139:0/1384575009 >> v2:172.21.15.150:3300/0 conn(0x7f8edc15ae40 0x7f8edc0b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217098928 0 0) 0x7f8edc0ce220 con 0x7f8edc0b0210 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ec8003520 con 0x7f8edc0b0210 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3323194249 0 0) 0x7f8ec8003520 con 0x7f8edc0b0210 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0cefe0 msgr2=0x7f8edc06d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 --2- 172.21.15.139:0/1384575009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0cefe0 0x7f8edc06d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 >> v2:172.21.15.150:3300/0 conn(0x7f8edc15ae40 msgr2=0x7f8edc0b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 --2- 172.21.15.139:0/1384575009 >> v2:172.21.15.150:3300/0 conn(0x7f8edc15ae40 0x7f8edc0b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc0ce500 con 0x7f8edc0b0210 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f8ecc001670 con 0x7f8edc0b0210 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 >> v1:172.21.15.139:6789/0 conn(0x7f8edc0b0210 legacy=0x7f8edc0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 --2- 172.21.15.139:0/1384575009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0cefe0 0x7f8ec80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 --2- 172.21.15.139:0/1384575009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ec8003f30 0x7f8ec8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 --2- 172.21.15.139:0/1384575009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ec80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0b1070 con 0x7f8ec8003f30 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec800b9c0 con 0x7f8ec8004d40 2024-04-25T10:02:54.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec800bb40 con 0x7f8edc0cefe0 2024-04-25T10:02:54.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.060+0000 7f8ed3fff640 1 --2- 172.21.15.139:0/1384575009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ec8003f30 0x7f8ec8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.061+0000 7f8edb7fe640 1 --2- 172.21.15.139:0/1384575009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0cefe0 0x7f8ec80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.061+0000 7f8edbfff640 1 --2- 172.21.15.139:0/1384575009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ec80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.061+0000 7f8ed3fff640 1 -- 172.21.15.139:0/1384575009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0cefe0 msgr2=0x7f8ec80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.061+0000 7f8ed3fff640 1 --2- 172.21.15.139:0/1384575009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0cefe0 0x7f8ec80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.061+0000 7f8ed3fff640 1 -- 172.21.15.139:0/1384575009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 msgr2=0x7f8ec80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.061+0000 7f8ed3fff640 1 --2- 172.21.15.139:0/1384575009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8004d40 0x7f8ec80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.061+0000 7f8ed3fff640 1 -- 172.21.15.139:0/1384575009 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec8003520 con 0x7f8ec8003f30 2024-04-25T10:02:54.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.061+0000 7f8ed3fff640 1 --2- 172.21.15.139:0/1384575009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ec8003f30 0x7f8ec8004800 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009ac0 tx=0x7f8ec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f508d778dad50175 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.061+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4036070 con 0x7f8ec8003f30 2024-04-25T10:02:54.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.061+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec4031400 con 0x7f8ec8003f30 2024-04-25T10:02:54.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.061+0000 7f8edaffd640 1 -- 172.21.15.139:0/1384575009 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4009690 con 0x7f8ec8003f30 2024-04-25T10:02:54.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.062+0000 7f8ee1d22640 1 -- 172.21.15.139:0/1384575009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ec8003f30 msgr2=0x7f8ec8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.062+0000 7f8ee1d22640 1 --2- 172.21.15.139:0/1384575009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ec8003f30 0x7f8ec8004800 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009ac0 tx=0x7f8ec8005830 comp rx=0 tx=0).stop 2024-04-25T10:02:54.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.062+0000 7f8edbfff640 1 -- 172.21.15.139:0/1384575009 reap_dead start 2024-04-25T10:02:54.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.062+0000 7f8ee1d22640 1 -- 172.21.15.139:0/1384575009 shutdown_connections 2024-04-25T10:02:54.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.062+0000 7f8ee1d22640 1 -- 172.21.15.139:0/1384575009 >> 172.21.15.139:0/1384575009 conn(0x7f8edc066f20 msgr2=0x7f8edc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.062+0000 7f8ee1d22640 1 -- 172.21.15.139:0/1384575009 shutdown_connections 2024-04-25T10:02:54.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.062+0000 7f8ee1d22640 1 -- 172.21.15.139:0/1384575009 wait complete. 2024-04-25T10:02:54.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.063+0000 7f8ee1d22640 1 Processor -- start 2024-04-25T10:02:54.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.063+0000 7f8ee1d22640 1 -- start start 2024-04-25T10:02:54.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ee1d22640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8003f30 0x7f8edc1440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ee1d22640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8edc0ae090 0x7f8edc0ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ee1d22640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0ac7b0 0x7f8edc153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ee1d22640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc15b3e0 con 0x7f8edc0ae090 2024-04-25T10:02:54.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ee1d22640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0b1ea0 con 0x7f8ec8003f30 2024-04-25T10:02:54.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ee1d22640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0ceb50 con 0x7f8edc0ac7b0 2024-04-25T10:02:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ed3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8edc0ae090 0x7f8edc0ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ed3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8edc0ae090 0x7f8edc0ac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54226/0 (socket says 172.21.15.139:54226) 2024-04-25T10:02:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ed3fff640 1 -- 172.21.15.139:0/2196941718 learned_addr learned my addr 172.21.15.139:0/2196941718 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8edb7fe640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8003f30 0x7f8edc1440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8edbfff640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0ac7b0 0x7f8edc153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ed3fff640 1 -- 172.21.15.139:0/2196941718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0ac7b0 msgr2=0x7f8edc153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ed3fff640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0ac7b0 0x7f8edc153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ed3fff640 1 -- 172.21.15.139:0/2196941718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8003f30 msgr2=0x7f8edc1440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ed3fff640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8003f30 0x7f8edc1440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ed3fff640 1 -- 172.21.15.139:0/2196941718 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc1f1b30 con 0x7f8edc0ae090 2024-04-25T10:02:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8edb7fe640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8003f30 0x7f8edc1440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.064+0000 7f8ed3fff640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8edc0ae090 0x7f8edc0ac270 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4034040 tx=0x7f8ec4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.065+0000 7f8ed8ff9640 1 -- 172.21.15.139:0/2196941718 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4040020 con 0x7f8edc0ae090 2024-04-25T10:02:54.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.065+0000 7f8ed8ff9640 1 -- 172.21.15.139:0/2196941718 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec4003c60 con 0x7f8edc0ae090 2024-04-25T10:02:54.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.065+0000 7f8ed8ff9640 1 -- 172.21.15.139:0/2196941718 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4036040 con 0x7f8edc0ae090 2024-04-25T10:02:54.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.065+0000 7f8ee1d22640 1 -- 172.21.15.139:0/2196941718 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8edc1f1da0 con 0x7f8edc0ae090 2024-04-25T10:02:54.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.065+0000 7f8ee1d22640 1 -- 172.21.15.139:0/2196941718 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc1f2360 con 0x7f8edc0ae090 2024-04-25T10:02:54.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.066+0000 7f8ee1d22640 1 -- 172.21.15.139:0/2196941718 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8edc074520 con 0x7f8edc0ae090 2024-04-25T10:02:54.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.066+0000 7f8ed8ff9640 1 -- 172.21.15.139:0/2196941718 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ec4043020 con 0x7f8edc0ae090 2024-04-25T10:02:54.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.066+0000 7f8ed8ff9640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ebc06b4d0 0x7f8ebc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.067+0000 7f8ed8ff9640 1 -- 172.21.15.139:0/2196941718 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ec40aa590 con 0x7f8edc0ae090 2024-04-25T10:02:54.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.067+0000 7f8edb7fe640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ebc06b4d0 0x7f8ebc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.067+0000 7f8edb7fe640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ebc06b4d0 0x7f8ebc06d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc001050 tx=0x7f8ecc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.071+0000 7f8ed8ff9640 1 -- 172.21.15.139:0/2196941718 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ec404a070 con 0x7f8edc0ae090 2024-04-25T10:02:54.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.236+0000 7f8ee1d22640 1 -- 172.21.15.139:0/2196941718 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8edc0b0b20 con 0x7f8ebc06b4d0 2024-04-25T10:02:54.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.242+0000 7f8ed8ff9640 1 -- 172.21.15.139:0/2196941718 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7f8edc0b0b20 con 0x7f8ebc06b4d0 2024-04-25T10:02:54.249 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:54.251 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T10:02:54.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.245+0000 7f8ee1d22640 1 -- 172.21.15.139:0/2196941718 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ebc06b4d0 msgr2=0x7f8ebc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.245+0000 7f8ee1d22640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ebc06b4d0 0x7f8ebc06d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc001050 tx=0x7f8ecc009040 comp rx=0 tx=0).stop 2024-04-25T10:02:54.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.245+0000 7f8ee1d22640 1 -- 172.21.15.139:0/2196941718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8edc0ae090 msgr2=0x7f8edc0ac270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.245+0000 7f8ee1d22640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8edc0ae090 0x7f8edc0ac270 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4034040 tx=0x7f8ec4004730 comp rx=0 tx=0).stop 2024-04-25T10:02:54.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.245+0000 7f8ee1d22640 1 -- 172.21.15.139:0/2196941718 shutdown_connections 2024-04-25T10:02:54.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.245+0000 7f8ee1d22640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8edc0ae090 0x7f8edc0ac270 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.245+0000 7f8ee1d22640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ebc06b4d0 0x7f8ebc06d990 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.245+0000 7f8ee1d22640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8edc0ac7b0 0x7f8edc153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.245+0000 7f8ee1d22640 1 --2- 172.21.15.139:0/2196941718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ec8003f30 0x7f8edc1440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.245+0000 7f8ee1d22640 1 -- 172.21.15.139:0/2196941718 >> 172.21.15.139:0/2196941718 conn(0x7f8edc066f20 msgr2=0x7f8edc073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.245+0000 7f8ee1d22640 1 -- 172.21.15.139:0/2196941718 shutdown_connections 2024-04-25T10:02:54.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.245+0000 7f8ee1d22640 1 -- 172.21.15.139:0/2196941718 wait complete. 2024-04-25T10:02:54.260 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-04-25T10:02:53.100569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262392,"kb_used_data":12904,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712328,"statfs":{"total":773094113280,"available":772825423872,"internally_reserved":0,"allocated":13213696,"data_stored":6769436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.134420"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063424+0000","last_change":"2024-04-25T10:02:50.995986+0000","last_active":"2024-04-25T10:02:52.063424+0000","last_peered":"2024-04-25T10:02:52.063424+0000","last_clean":"2024-04-25T10:02:52.063424+0000","last_became_active":"2024-04-25T10:02:50.995689+0000","last_became_peered":"2024-04-25T10:02:50.995689+0000","last_unstale":"2024-04-25T10:02:52.063424+0000","last_undegraded":"2024-04-25T10:02:52.063424+0000","last_fullsized":"2024-04-25T10:02:52.063424+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:43.530691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063221+0000","last_change":"2024-04-25T10:02:50.998670+0000","last_active":"2024-04-25T10:02:52.063221+0000","last_peered":"2024-04-25T10:02:52.063221+0000","last_clean":"2024-04-25T10:02:52.063221+0000","last_became_active":"2024-04-25T10:02:50.998408+0000","last_became_peered":"2024-04-25T10:02:50.998408+0000","last_unstale":"2024-04-25T10:02:52.063221+0000","last_undegraded":"2024-04-25T10:02:52.063221+0000","last_fullsized":"2024-04-25T10:02:52.063221+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:28.359044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063471+0000","last_change":"2024-04-25T10:02:50.997926+0000","last_active":"2024-04-25T10:02:52.063471+0000","last_peered":"2024-04-25T10:02:52.063471+0000","last_clean":"2024-04-25T10:02:52.063471+0000","last_became_active":"2024-04-25T10:02:50.995445+0000","last_became_peered":"2024-04-25T10:02:50.995445+0000","last_unstale":"2024-04-25T10:02:52.063471+0000","last_undegraded":"2024-04-25T10:02:52.063471+0000","last_fullsized":"2024-04-25T10:02:52.063471+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:20.548894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063476+0000","last_change":"2024-04-25T10:02:51.008205+0000","last_active":"2024-04-25T10:02:52.063476+0000","last_peered":"2024-04-25T10:02:52.063476+0000","last_clean":"2024-04-25T10:02:52.063476+0000","last_became_active":"2024-04-25T10:02:51.007934+0000","last_became_peered":"2024-04-25T10:02:51.007934+0000","last_unstale":"2024-04-25T10:02:52.063476+0000","last_undegraded":"2024-04-25T10:02:52.063476+0000","last_fullsized":"2024-04-25T10:02:52.063476+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:03.428029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063075+0000","last_change":"2024-04-25T10:02:50.995413+0000","last_active":"2024-04-25T10:02:52.063075+0000","last_peered":"2024-04-25T10:02:52.063075+0000","last_clean":"2024-04-25T10:02:52.063075+0000","last_became_active":"2024-04-25T10:02:50.995072+0000","last_became_peered":"2024-04-25T10:02:50.995072+0000","last_unstale":"2024-04-25T10:02:52.063075+0000","last_undegraded":"2024-04-25T10:02:52.063075+0000","last_fullsized":"2024-04-25T10:02:52.063075+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.049687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996093+0000","last_change":"2024-04-25T10:02:50.990984+0000","last_active":"2024-04-25T10:02:51.996093+0000","last_peered":"2024-04-25T10:02:51.996093+0000","last_clean":"2024-04-25T10:02:51.996093+0000","last_became_active":"2024-04-25T10:02:50.990729+0000","last_became_peered":"2024-04-25T10:02:50.990729+0000","last_unstale":"2024-04-25T10:02:51.996093+0000","last_undegraded":"2024-04-25T10:02:51.996093+0000","last_fullsized":"2024-04-25T10:02:51.996093+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:52.920085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996030+0000","last_change":"2024-04-25T10:02:50.997201+0000","last_active":"2024-04-25T10:02:51.996030+0000","last_peered":"2024-04-25T10:02:51.996030+0000","last_clean":"2024-04-25T10:02:51.996030+0000","last_became_active":"2024-04-25T10:02:50.996899+0000","last_became_peered":"2024-04-25T10:02:50.996899+0000","last_unstale":"2024-04-25T10:02:51.996030+0000","last_undegraded":"2024-04-25T10:02:51.996030+0000","last_fullsized":"2024-04-25T10:02:51.996030+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:50.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062525+0000","last_change":"2024-04-25T10:02:50.988718+0000","last_active":"2024-04-25T10:02:52.062525+0000","last_peered":"2024-04-25T10:02:52.062525+0000","last_clean":"2024-04-25T10:02:52.062525+0000","last_became_active":"2024-04-25T10:02:50.988322+0000","last_became_peered":"2024-04-25T10:02:50.988322+0000","last_unstale":"2024-04-25T10:02:52.062525+0000","last_undegraded":"2024-04-25T10:02:52.062525+0000","last_fullsized":"2024-04-25T10:02:52.062525+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:15.550123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063839+0000","last_change":"2024-04-25T10:02:51.006987+0000","last_active":"2024-04-25T10:02:52.063839+0000","last_peered":"2024-04-25T10:02:52.063839+0000","last_clean":"2024-04-25T10:02:52.063839+0000","last_became_active":"2024-04-25T10:02:51.006639+0000","last_became_peered":"2024-04-25T10:02:51.006639+0000","last_unstale":"2024-04-25T10:02:52.063839+0000","last_undegraded":"2024-04-25T10:02:52.063839+0000","last_fullsized":"2024-04-25T10:02:52.063839+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:59.868522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989284+0000","last_change":"2024-04-25T10:02:50.989951+0000","last_active":"2024-04-25T10:02:51.989284+0000","last_peered":"2024-04-25T10:02:51.989284+0000","last_clean":"2024-04-25T10:02:51.989284+0000","last_became_active":"2024-04-25T10:02:50.989651+0000","last_became_peered":"2024-04-25T10:02:50.989651+0000","last_unstale":"2024-04-25T10:02:51.989284+0000","last_undegraded":"2024-04-25T10:02:51.989284+0000","last_fullsized":"2024-04-25T10:02:51.989284+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:17.989782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063305+0000","last_change":"2024-04-25T10:02:50.997270+0000","last_active":"2024-04-25T10:02:52.063305+0000","last_peered":"2024-04-25T10:02:52.063305+0000","last_clean":"2024-04-25T10:02:52.063305+0000","last_became_active":"2024-04-25T10:02:50.997004+0000","last_became_peered":"2024-04-25T10:02:50.997004+0000","last_unstale":"2024-04-25T10:02:52.063305+0000","last_undegraded":"2024-04-25T10:02:52.063305+0000","last_fullsized":"2024-04-25T10:02:52.063305+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:42.494181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996441+0000","last_change":"2024-04-25T10:02:50.994299+0000","last_active":"2024-04-25T10:02:51.996441+0000","last_peered":"2024-04-25T10:02:51.996441+0000","last_clean":"2024-04-25T10:02:51.996441+0000","last_became_active":"2024-04-25T10:02:50.993997+0000","last_became_peered":"2024-04-25T10:02:50.993997+0000","last_unstale":"2024-04-25T10:02:51.996441+0000","last_undegraded":"2024-04-25T10:02:51.996441+0000","last_fullsized":"2024-04-25T10:02:51.996441+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:48.776004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989281+0000","last_change":"2024-04-25T10:02:50.995444+0000","last_active":"2024-04-25T10:02:51.989281+0000","last_peered":"2024-04-25T10:02:51.989281+0000","last_clean":"2024-04-25T10:02:51.989281+0000","last_became_active":"2024-04-25T10:02:50.995182+0000","last_became_peered":"2024-04-25T10:02:50.995182+0000","last_unstale":"2024-04-25T10:02:51.989281+0000","last_undegraded":"2024-04-25T10:02:51.989281+0000","last_fullsized":"2024-04-25T10:02:51.989281+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:32.458571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063171+0000","last_change":"2024-04-25T10:02:50.997195+0000","last_active":"2024-04-25T10:02:52.063171+0000","last_peered":"2024-04-25T10:02:52.063171+0000","last_clean":"2024-04-25T10:02:52.063171+0000","last_became_active":"2024-04-25T10:02:50.996889+0000","last_became_peered":"2024-04-25T10:02:50.996889+0000","last_unstale":"2024-04-25T10:02:52.063171+0000","last_undegraded":"2024-04-25T10:02:52.063171+0000","last_fullsized":"2024-04-25T10:02:52.063171+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:22.666439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062856+0000","last_change":"2024-04-25T10:02:51.004810+0000","last_active":"2024-04-25T10:02:52.062856+0000","last_peered":"2024-04-25T10:02:52.062856+0000","last_clean":"2024-04-25T10:02:52.062856+0000","last_became_active":"2024-04-25T10:02:51.004545+0000","last_became_peered":"2024-04-25T10:02:51.004545+0000","last_unstale":"2024-04-25T10:02:52.062856+0000","last_undegraded":"2024-04-25T10:02:52.062856+0000","last_fullsized":"2024-04-25T10:02:52.062856+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:08.588569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.995974+0000","last_change":"2024-04-25T10:02:50.987317+0000","last_active":"2024-04-25T10:02:51.995974+0000","last_peered":"2024-04-25T10:02:51.995974+0000","last_clean":"2024-04-25T10:02:51.995974+0000","last_became_active":"2024-04-25T10:02:50.987040+0000","last_became_peered":"2024-04-25T10:02:50.987040+0000","last_unstale":"2024-04-25T10:02:51.995974+0000","last_undegraded":"2024-04-25T10:02:51.995974+0000","last_fullsized":"2024-04-25T10:02:51.995974+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:13.916753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989271+0000","last_change":"2024-04-25T10:02:51.004140+0000","last_active":"2024-04-25T10:02:51.989271+0000","last_peered":"2024-04-25T10:02:51.989271+0000","last_clean":"2024-04-25T10:02:51.989271+0000","last_became_active":"2024-04-25T10:02:51.003868+0000","last_became_peered":"2024-04-25T10:02:51.003868+0000","last_unstale":"2024-04-25T10:02:51.989271+0000","last_undegraded":"2024-04-25T10:02:51.989271+0000","last_fullsized":"2024-04-25T10:02:51.989271+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:08.588569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063358+0000","last_change":"2024-04-25T10:02:50.997881+0000","last_active":"2024-04-25T10:02:52.063358+0000","last_peered":"2024-04-25T10:02:52.063358+0000","last_clean":"2024-04-25T10:02:52.063358+0000","last_became_active":"2024-04-25T10:02:50.997604+0000","last_became_peered":"2024-04-25T10:02:50.997604+0000","last_unstale":"2024-04-25T10:02:52.063358+0000","last_undegraded":"2024-04-25T10:02:52.063358+0000","last_fullsized":"2024-04-25T10:02:52.063358+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:59.868522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994458+0000","last_change":"2024-04-25T10:02:50.997944+0000","last_active":"2024-04-25T10:02:51.994458+0000","last_peered":"2024-04-25T10:02:51.994458+0000","last_clean":"2024-04-25T10:02:51.994458+0000","last_became_active":"2024-04-25T10:02:50.997496+0000","last_became_peered":"2024-04-25T10:02:50.997496+0000","last_unstale":"2024-04-25T10:02:51.994458+0000","last_undegraded":"2024-04-25T10:02:51.994458+0000","last_fullsized":"2024-04-25T10:02:51.994458+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.049687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062541+0000","last_change":"2024-04-25T10:02:50.987510+0000","last_active":"2024-04-25T10:02:52.062541+0000","last_peered":"2024-04-25T10:02:52.062541+0000","last_clean":"2024-04-25T10:02:52.062541+0000","last_became_active":"2024-04-25T10:02:50.987222+0000","last_became_peered":"2024-04-25T10:02:50.987222+0000","last_unstale":"2024-04-25T10:02:52.062541+0000","last_undegraded":"2024-04-25T10:02:52.062541+0000","last_fullsized":"2024-04-25T10:02:52.062541+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:57.363879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.064041+0000","last_change":"2024-04-25T10:02:51.004739+0000","last_active":"2024-04-25T10:02:52.064041+0000","last_peered":"2024-04-25T10:02:52.064041+0000","last_clean":"2024-04-25T10:02:52.064041+0000","last_became_active":"2024-04-25T10:02:51.004461+0000","last_became_peered":"2024-04-25T10:02:51.004461+0000","last_unstale":"2024-04-25T10:02:52.064041+0000","last_undegraded":"2024-04-25T10:02:52.064041+0000","last_fullsized":"2024-04-25T10:02:52.064041+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:12.813563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063243+0000","last_change":"2024-04-25T10:02:51.005417+0000","last_active":"2024-04-25T10:02:52.063243+0000","last_peered":"2024-04-25T10:02:52.063243+0000","last_clean":"2024-04-25T10:02:52.063243+0000","last_became_active":"2024-04-25T10:02:51.005127+0000","last_became_peered":"2024-04-25T10:02:51.005127+0000","last_unstale":"2024-04-25T10:02:52.063243+0000","last_undegraded":"2024-04-25T10:02:52.063243+0000","last_fullsized":"2024-04-25T10:02:52.063243+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:17.989782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063963+0000","last_change":"2024-04-25T10:02:51.007076+0000","last_active":"2024-04-25T10:02:52.063963+0000","last_peered":"2024-04-25T10:02:52.063963+0000","last_clean":"2024-04-25T10:02:52.063963+0000","last_became_active":"2024-04-25T10:02:51.006789+0000","last_became_peered":"2024-04-25T10:02:51.006789+0000","last_unstale":"2024-04-25T10:02:52.063963+0000","last_undegraded":"2024-04-25T10:02:52.063963+0000","last_fullsized":"2024-04-25T10:02:52.063963+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:31.011518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990082+0000","last_change":"2024-04-25T10:02:51.004073+0000","last_active":"2024-04-25T10:02:51.990082+0000","last_peered":"2024-04-25T10:02:51.990082+0000","last_clean":"2024-04-25T10:02:51.990082+0000","last_became_active":"2024-04-25T10:02:51.003786+0000","last_became_peered":"2024-04-25T10:02:51.003786+0000","last_unstale":"2024-04-25T10:02:51.990082+0000","last_undegraded":"2024-04-25T10:02:51.990082+0000","last_fullsized":"2024-04-25T10:02:51.990082+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:59.749858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990512+0000","last_change":"2024-04-25T10:02:51.006118+0000","last_active":"2024-04-25T10:02:51.990512+0000","last_peered":"2024-04-25T10:02:51.990512+0000","last_clean":"2024-04-25T10:02:51.990512+0000","last_became_active":"2024-04-25T10:02:51.005094+0000","last_became_peered":"2024-04-25T10:02:51.005094+0000","last_unstale":"2024-04-25T10:02:51.990512+0000","last_undegraded":"2024-04-25T10:02:51.990512+0000","last_fullsized":"2024-04-25T10:02:51.990512+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:04.656853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062733+0000","last_change":"2024-04-25T10:02:50.992175+0000","last_active":"2024-04-25T10:02:52.062733+0000","last_peered":"2024-04-25T10:02:52.062733+0000","last_clean":"2024-04-25T10:02:52.062733+0000","last_became_active":"2024-04-25T10:02:50.991872+0000","last_became_peered":"2024-04-25T10:02:50.991872+0000","last_unstale":"2024-04-25T10:02:52.062733+0000","last_undegraded":"2024-04-25T10:02:52.062733+0000","last_fullsized":"2024-04-25T10:02:52.062733+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:03.225652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994465+0000","last_change":"2024-04-25T10:02:50.997048+0000","last_active":"2024-04-25T10:02:51.994465+0000","last_peered":"2024-04-25T10:02:51.994465+0000","last_clean":"2024-04-25T10:02:51.994465+0000","last_became_active":"2024-04-25T10:02:50.996776+0000","last_became_peered":"2024-04-25T10:02:50.996776+0000","last_unstale":"2024-04-25T10:02:51.994465+0000","last_undegraded":"2024-04-25T10:02:51.994465+0000","last_fullsized":"2024-04-25T10:02:51.994465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:17.989782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996768+0000","last_change":"2024-04-25T10:02:51.002291+0000","last_active":"2024-04-25T10:02:51.996768+0000","last_peered":"2024-04-25T10:02:51.996768+0000","last_clean":"2024-04-25T10:02:51.996768+0000","last_became_active":"2024-04-25T10:02:51.002021+0000","last_became_peered":"2024-04-25T10:02:51.002021+0000","last_unstale":"2024-04-25T10:02:51.996768+0000","last_undegraded":"2024-04-25T10:02:51.996768+0000","last_fullsized":"2024-04-25T10:02:51.996768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:16.553269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062801+0000","last_change":"2024-04-25T10:02:50.985488+0000","last_active":"2024-04-25T10:02:52.062801+0000","last_peered":"2024-04-25T10:02:52.062801+0000","last_clean":"2024-04-25T10:02:52.062801+0000","last_became_active":"2024-04-25T10:02:50.985184+0000","last_became_peered":"2024-04-25T10:02:50.985184+0000","last_unstale":"2024-04-25T10:02:52.062801+0000","last_undegraded":"2024-04-25T10:02:52.062801+0000","last_fullsized":"2024-04-25T10:02:52.062801+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:00.911520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063216+0000","last_change":"2024-04-25T10:02:50.998971+0000","last_active":"2024-04-25T10:02:52.063216+0000","last_peered":"2024-04-25T10:02:52.063216+0000","last_clean":"2024-04-25T10:02:52.063216+0000","last_became_active":"2024-04-25T10:02:50.998706+0000","last_became_peered":"2024-04-25T10:02:50.998706+0000","last_unstale":"2024-04-25T10:02:52.063216+0000","last_undegraded":"2024-04-25T10:02:52.063216+0000","last_fullsized":"2024-04-25T10:02:52.063216+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:49.104042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989877+0000","last_change":"2024-04-25T10:02:51.001919+0000","last_active":"2024-04-25T10:02:51.989877+0000","last_peered":"2024-04-25T10:02:51.989877+0000","last_clean":"2024-04-25T10:02:51.989877+0000","last_became_active":"2024-04-25T10:02:51.001532+0000","last_became_peered":"2024-04-25T10:02:51.001532+0000","last_unstale":"2024-04-25T10:02:51.989877+0000","last_undegraded":"2024-04-25T10:02:51.989877+0000","last_fullsized":"2024-04-25T10:02:51.989877+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:42.494181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989730+0000","last_change":"2024-04-25T10:02:51.003757+0000","last_active":"2024-04-25T10:02:51.989730+0000","last_peered":"2024-04-25T10:02:51.989730+0000","last_clean":"2024-04-25T10:02:51.989730+0000","last_became_active":"2024-04-25T10:02:51.003434+0000","last_became_peered":"2024-04-25T10:02:51.003434+0000","last_unstale":"2024-04-25T10:02:51.989730+0000","last_undegraded":"2024-04-25T10:02:51.989730+0000","last_fullsized":"2024-04-25T10:02:51.989730+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:17.989782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989318+0000","last_change":"2024-04-25T10:02:50.988133+0000","last_active":"2024-04-25T10:02:51.989318+0000","last_peered":"2024-04-25T10:02:51.989318+0000","last_clean":"2024-04-25T10:02:51.989318+0000","last_became_active":"2024-04-25T10:02:50.987681+0000","last_became_peered":"2024-04-25T10:02:50.987681+0000","last_unstale":"2024-04-25T10:02:51.989318+0000","last_undegraded":"2024-04-25T10:02:51.989318+0000","last_fullsized":"2024-04-25T10:02:51.989318+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:42.494181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"119'4","reported_seq":18,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994230+0000","last_change":"2024-04-25T10:02:49.997081+0000","last_active":"2024-04-25T10:02:51.994230+0000","last_peered":"2024-04-25T10:02:51.994230+0000","last_clean":"2024-04-25T10:02:51.994230+0000","last_became_active":"2024-04-25T10:02:49.996786+0000","last_became_peered":"2024-04-25T10:02:49.996786+0000","last_unstale":"2024-04-25T10:02:51.994230+0000","last_undegraded":"2024-04-25T10:02:51.994230+0000","last_fullsized":"2024-04-25T10:02:51.994230+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.201776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996380+0000","last_change":"2024-04-25T10:02:50.990258+0000","last_active":"2024-04-25T10:02:51.996380+0000","last_peered":"2024-04-25T10:02:51.996380+0000","last_clean":"2024-04-25T10:02:51.996380+0000","last_became_active":"2024-04-25T10:02:50.989960+0000","last_became_peered":"2024-04-25T10:02:50.989960+0000","last_unstale":"2024-04-25T10:02:51.996380+0000","last_undegraded":"2024-04-25T10:02:51.996380+0000","last_fullsized":"2024-04-25T10:02:51.996380+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:52.987232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989216+0000","last_change":"2024-04-25T10:02:49.981186+0000","last_active":"2024-04-25T10:02:51.989216+0000","last_peered":"2024-04-25T10:02:51.989216+0000","last_clean":"2024-04-25T10:02:51.989216+0000","last_became_active":"2024-04-25T10:02:49.980876+0000","last_became_peered":"2024-04-25T10:02:49.980876+0000","last_unstale":"2024-04-25T10:02:51.989216+0000","last_undegraded":"2024-04-25T10:02:51.989216+0000","last_fullsized":"2024-04-25T10:02:51.989216+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:23.806266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990211+0000","last_change":"2024-04-25T10:02:51.002808+0000","last_active":"2024-04-25T10:02:51.990211+0000","last_peered":"2024-04-25T10:02:51.990211+0000","last_clean":"2024-04-25T10:02:51.990211+0000","last_became_active":"2024-04-25T10:02:51.001989+0000","last_became_peered":"2024-04-25T10:02:51.001989+0000","last_unstale":"2024-04-25T10:02:51.990211+0000","last_undegraded":"2024-04-25T10:02:51.990211+0000","last_fullsized":"2024-04-25T10:02:51.990211+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:27.223456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063807+0000","last_change":"2024-04-25T10:02:49.993341+0000","last_active":"2024-04-25T10:02:52.063807+0000","last_peered":"2024-04-25T10:02:52.063807+0000","last_clean":"2024-04-25T10:02:52.063807+0000","last_became_active":"2024-04-25T10:02:49.992880+0000","last_became_peered":"2024-04-25T10:02:49.992880+0000","last_unstale":"2024-04-25T10:02:52.063807+0000","last_undegraded":"2024-04-25T10:02:52.063807+0000","last_fullsized":"2024-04-25T10:02:52.063807+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:07.744516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063974+0000","last_change":"2024-04-25T10:02:50.999516+0000","last_active":"2024-04-25T10:02:52.063974+0000","last_peered":"2024-04-25T10:02:52.063974+0000","last_clean":"2024-04-25T10:02:52.063974+0000","last_became_active":"2024-04-25T10:02:50.999228+0000","last_became_peered":"2024-04-25T10:02:50.999228+0000","last_unstale":"2024-04-25T10:02:52.063974+0000","last_undegraded":"2024-04-25T10:02:52.063974+0000","last_fullsized":"2024-04-25T10:02:52.063974+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:42.323390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989196+0000","last_change":"2024-04-25T10:02:49.987633+0000","last_active":"2024-04-25T10:02:51.989196+0000","last_peered":"2024-04-25T10:02:51.989196+0000","last_clean":"2024-04-25T10:02:51.989196+0000","last_became_active":"2024-04-25T10:02:49.987375+0000","last_became_peered":"2024-04-25T10:02:49.987375+0000","last_unstale":"2024-04-25T10:02:51.989196+0000","last_undegraded":"2024-04-25T10:02:51.989196+0000","last_fullsized":"2024-04-25T10:02:51.989196+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:59.622317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989239+0000","last_change":"2024-04-25T10:02:50.995905+0000","last_active":"2024-04-25T10:02:51.989239+0000","last_peered":"2024-04-25T10:02:51.989239+0000","last_clean":"2024-04-25T10:02:51.989239+0000","last_became_active":"2024-04-25T10:02:50.995657+0000","last_became_peered":"2024-04-25T10:02:50.995657+0000","last_unstale":"2024-04-25T10:02:51.989239+0000","last_undegraded":"2024-04-25T10:02:51.989239+0000","last_fullsized":"2024-04-25T10:02:51.989239+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:32.421018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063078+0000","last_change":"2024-04-25T10:02:49.992479+0000","last_active":"2024-04-25T10:02:52.063078+0000","last_peered":"2024-04-25T10:02:52.063078+0000","last_clean":"2024-04-25T10:02:52.063078+0000","last_became_active":"2024-04-25T10:02:49.992191+0000","last_became_peered":"2024-04-25T10:02:49.992191+0000","last_unstale":"2024-04-25T10:02:52.063078+0000","last_undegraded":"2024-04-25T10:02:52.063078+0000","last_fullsized":"2024-04-25T10:02:52.063078+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:05.609417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062902+0000","last_change":"2024-04-25T10:02:50.998341+0000","last_active":"2024-04-25T10:02:52.062902+0000","last_peered":"2024-04-25T10:02:52.062902+0000","last_clean":"2024-04-25T10:02:52.062902+0000","last_became_active":"2024-04-25T10:02:50.998071+0000","last_became_peered":"2024-04-25T10:02:50.998071+0000","last_unstale":"2024-04-25T10:02:52.062902+0000","last_undegraded":"2024-04-25T10:02:52.062902+0000","last_fullsized":"2024-04-25T10:02:52.062902+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:32.458571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996086+0000","last_change":"2024-04-25T10:02:49.994519+0000","last_active":"2024-04-25T10:02:51.996086+0000","last_peered":"2024-04-25T10:02:51.996086+0000","last_clean":"2024-04-25T10:02:51.996086+0000","last_became_active":"2024-04-25T10:02:49.993878+0000","last_became_peered":"2024-04-25T10:02:49.993878+0000","last_unstale":"2024-04-25T10:02:51.996086+0000","last_undegraded":"2024-04-25T10:02:51.996086+0000","last_fullsized":"2024-04-25T10:02:51.996086+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:12.836756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063323+0000","last_change":"2024-04-25T10:02:50.992382+0000","last_active":"2024-04-25T10:02:52.063323+0000","last_peered":"2024-04-25T10:02:52.063323+0000","last_clean":"2024-04-25T10:02:52.063323+0000","last_became_active":"2024-04-25T10:02:50.992125+0000","last_became_peered":"2024-04-25T10:02:50.992125+0000","last_unstale":"2024-04-25T10:02:52.063323+0000","last_undegraded":"2024-04-25T10:02:52.063323+0000","last_fullsized":"2024-04-25T10:02:52.063323+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:47.908802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989818+0000","last_change":"2024-04-25T10:02:49.994358+0000","last_active":"2024-04-25T10:02:51.989818+0000","last_peered":"2024-04-25T10:02:51.989818+0000","last_clean":"2024-04-25T10:02:51.989818+0000","last_became_active":"2024-04-25T10:02:49.993749+0000","last_became_peered":"2024-04-25T10:02:49.993749+0000","last_unstale":"2024-04-25T10:02:51.989818+0000","last_undegraded":"2024-04-25T10:02:51.989818+0000","last_fullsized":"2024-04-25T10:02:51.989818+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:05.609417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989259+0000","last_change":"2024-04-25T10:02:50.990005+0000","last_active":"2024-04-25T10:02:51.989259+0000","last_peered":"2024-04-25T10:02:51.989259+0000","last_clean":"2024-04-25T10:02:51.989259+0000","last_became_active":"2024-04-25T10:02:50.989730+0000","last_became_peered":"2024-04-25T10:02:50.989730+0000","last_unstale":"2024-04-25T10:02:51.989259+0000","last_undegraded":"2024-04-25T10:02:51.989259+0000","last_fullsized":"2024-04-25T10:02:51.989259+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:57.671494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996221+0000","last_change":"2024-04-25T10:02:49.993593+0000","last_active":"2024-04-25T10:02:51.996221+0000","last_peered":"2024-04-25T10:02:51.996221+0000","last_clean":"2024-04-25T10:02:51.996221+0000","last_became_active":"2024-04-25T10:02:49.993217+0000","last_became_peered":"2024-04-25T10:02:49.993217+0000","last_unstale":"2024-04-25T10:02:51.996221+0000","last_undegraded":"2024-04-25T10:02:51.996221+0000","last_fullsized":"2024-04-25T10:02:51.996221+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:21.277770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994301+0000","last_change":"2024-04-25T10:02:50.998732+0000","last_active":"2024-04-25T10:02:51.994301+0000","last_peered":"2024-04-25T10:02:51.994301+0000","last_clean":"2024-04-25T10:02:51.994301+0000","last_became_active":"2024-04-25T10:02:50.998472+0000","last_became_peered":"2024-04-25T10:02:50.998472+0000","last_unstale":"2024-04-25T10:02:51.994301+0000","last_undegraded":"2024-04-25T10:02:51.994301+0000","last_fullsized":"2024-04-25T10:02:51.994301+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:08.588569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994405+0000","last_change":"2024-04-25T10:02:49.996405+0000","last_active":"2024-04-25T10:02:51.994405+0000","last_peered":"2024-04-25T10:02:51.994405+0000","last_clean":"2024-04-25T10:02:51.994405+0000","last_became_active":"2024-04-25T10:02:49.996065+0000","last_became_peered":"2024-04-25T10:02:49.996065+0000","last_unstale":"2024-04-25T10:02:51.994405+0000","last_undegraded":"2024-04-25T10:02:51.994405+0000","last_fullsized":"2024-04-25T10:02:51.994405+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:20.468891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990256+0000","last_change":"2024-04-25T10:02:51.007375+0000","last_active":"2024-04-25T10:02:51.990256+0000","last_peered":"2024-04-25T10:02:51.990256+0000","last_clean":"2024-04-25T10:02:51.990256+0000","last_became_active":"2024-04-25T10:02:51.007128+0000","last_became_peered":"2024-04-25T10:02:51.007128+0000","last_unstale":"2024-04-25T10:02:51.990256+0000","last_undegraded":"2024-04-25T10:02:51.990256+0000","last_fullsized":"2024-04-25T10:02:51.990256+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:13.464999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990308+0000","last_change":"2024-04-25T10:02:49.989926+0000","last_active":"2024-04-25T10:02:51.990308+0000","last_peered":"2024-04-25T10:02:51.990308+0000","last_clean":"2024-04-25T10:02:51.990308+0000","last_became_active":"2024-04-25T10:02:49.989602+0000","last_became_peered":"2024-04-25T10:02:49.989602+0000","last_unstale":"2024-04-25T10:02:51.990308+0000","last_undegraded":"2024-04-25T10:02:51.990308+0000","last_fullsized":"2024-04-25T10:02:51.990308+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:56.075749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063489+0000","last_change":"2024-04-25T10:02:50.995011+0000","last_active":"2024-04-25T10:02:52.063489+0000","last_peered":"2024-04-25T10:02:52.063489+0000","last_clean":"2024-04-25T10:02:52.063489+0000","last_became_active":"2024-04-25T10:02:50.994735+0000","last_became_peered":"2024-04-25T10:02:50.994735+0000","last_unstale":"2024-04-25T10:02:52.063489+0000","last_undegraded":"2024-04-25T10:02:52.063489+0000","last_fullsized":"2024-04-25T10:02:52.063489+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:31.011518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"119'3","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990422+0000","last_change":"2024-04-25T10:02:49.993333+0000","last_active":"2024-04-25T10:02:51.990422+0000","last_peered":"2024-04-25T10:02:51.990422+0000","last_clean":"2024-04-25T10:02:51.990422+0000","last_became_active":"2024-04-25T10:02:49.993041+0000","last_became_peered":"2024-04-25T10:02:49.993041+0000","last_unstale":"2024-04-25T10:02:51.990422+0000","last_undegraded":"2024-04-25T10:02:51.990422+0000","last_fullsized":"2024-04-25T10:02:51.990422+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:44.159091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990571+0000","last_change":"2024-04-25T10:02:50.998617+0000","last_active":"2024-04-25T10:02:51.990571+0000","last_peered":"2024-04-25T10:02:51.990571+0000","last_clean":"2024-04-25T10:02:51.990571+0000","last_became_active":"2024-04-25T10:02:50.998280+0000","last_became_peered":"2024-04-25T10:02:50.998280+0000","last_unstale":"2024-04-25T10:02:51.990571+0000","last_undegraded":"2024-04-25T10:02:51.990571+0000","last_fullsized":"2024-04-25T10:02:51.990571+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:53.414855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"119'6","reported_seq":20,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063139+0000","last_change":"2024-04-25T10:02:49.990625+0000","last_active":"2024-04-25T10:02:52.063139+0000","last_peered":"2024-04-25T10:02:52.063139+0000","last_clean":"2024-04-25T10:02:52.063139+0000","last_became_active":"2024-04-25T10:02:49.990320+0000","last_became_peered":"2024-04-25T10:02:49.990320+0000","last_unstale":"2024-04-25T10:02:52.063139+0000","last_undegraded":"2024-04-25T10:02:52.063139+0000","last_fullsized":"2024-04-25T10:02:52.063139+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:07.581852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994430+0000","last_change":"2024-04-25T10:02:50.995103+0000","last_active":"2024-04-25T10:02:51.994430+0000","last_peered":"2024-04-25T10:02:51.994430+0000","last_clean":"2024-04-25T10:02:51.994430+0000","last_became_active":"2024-04-25T10:02:50.993645+0000","last_became_peered":"2024-04-25T10:02:50.993645+0000","last_unstale":"2024-04-25T10:02:51.994430+0000","last_undegraded":"2024-04-25T10:02:51.994430+0000","last_fullsized":"2024-04-25T10:02:51.994430+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:32.458571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063076+0000","last_change":"2024-04-25T10:02:49.995923+0000","last_active":"2024-04-25T10:02:52.063076+0000","last_peered":"2024-04-25T10:02:52.063076+0000","last_clean":"2024-04-25T10:02:52.063076+0000","last_became_active":"2024-04-25T10:02:49.995634+0000","last_became_peered":"2024-04-25T10:02:49.995634+0000","last_unstale":"2024-04-25T10:02:52.063076+0000","last_undegraded":"2024-04-25T10:02:52.063076+0000","last_fullsized":"2024-04-25T10:02:52.063076+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:16.983066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989239+0000","last_change":"2024-04-25T10:02:50.998371+0000","last_active":"2024-04-25T10:02:51.989239+0000","last_peered":"2024-04-25T10:02:51.989239+0000","last_clean":"2024-04-25T10:02:51.989239+0000","last_became_active":"2024-04-25T10:02:50.998127+0000","last_became_peered":"2024-04-25T10:02:50.998127+0000","last_unstale":"2024-04-25T10:02:51.989239+0000","last_undegraded":"2024-04-25T10:02:51.989239+0000","last_fullsized":"2024-04-25T10:02:51.989239+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:42.230799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063410+0000","last_change":"2024-04-25T10:02:49.991026+0000","last_active":"2024-04-25T10:02:52.063410+0000","last_peered":"2024-04-25T10:02:52.063410+0000","last_clean":"2024-04-25T10:02:52.063410+0000","last_became_active":"2024-04-25T10:02:49.990598+0000","last_became_peered":"2024-04-25T10:02:49.990598+0000","last_unstale":"2024-04-25T10:02:52.063410+0000","last_undegraded":"2024-04-25T10:02:52.063410+0000","last_fullsized":"2024-04-25T10:02:52.063410+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:41.487464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062526+0000","last_change":"2024-04-25T10:02:50.999478+0000","last_active":"2024-04-25T10:02:52.062526+0000","last_peered":"2024-04-25T10:02:52.062526+0000","last_clean":"2024-04-25T10:02:52.062526+0000","last_became_active":"2024-04-25T10:02:50.999182+0000","last_became_peered":"2024-04-25T10:02:50.999182+0000","last_unstale":"2024-04-25T10:02:52.062526+0000","last_undegraded":"2024-04-25T10:02:52.062526+0000","last_fullsized":"2024-04-25T10:02:52.062526+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:04.825603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062840+0000","last_change":"2024-04-25T10:02:49.990146+0000","last_active":"2024-04-25T10:02:52.062840+0000","last_peered":"2024-04-25T10:02:52.062840+0000","last_clean":"2024-04-25T10:02:52.062840+0000","last_became_active":"2024-04-25T10:02:49.989601+0000","last_became_peered":"2024-04-25T10:02:49.989601+0000","last_unstale":"2024-04-25T10:02:52.062840+0000","last_undegraded":"2024-04-25T10:02:52.062840+0000","last_fullsized":"2024-04-25T10:02:52.062840+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:31.451854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990079+0000","last_change":"2024-04-25T10:02:51.002186+0000","last_active":"2024-04-25T10:02:51.990079+0000","last_peered":"2024-04-25T10:02:51.990079+0000","last_clean":"2024-04-25T10:02:51.990079+0000","last_became_active":"2024-04-25T10:02:51.001901+0000","last_became_peered":"2024-04-25T10:02:51.001901+0000","last_unstale":"2024-04-25T10:02:51.990079+0000","last_undegraded":"2024-04-25T10:02:51.990079+0000","last_fullsized":"2024-04-25T10:02:51.990079+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:11.029946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989367+0000","last_change":"2024-04-25T10:02:49.991065+0000","last_active":"2024-04-25T10:02:51.989367+0000","last_peered":"2024-04-25T10:02:51.989367+0000","last_clean":"2024-04-25T10:02:51.989367+0000","last_became_active":"2024-04-25T10:02:49.990673+0000","last_became_peered":"2024-04-25T10:02:49.990673+0000","last_unstale":"2024-04-25T10:02:51.989367+0000","last_undegraded":"2024-04-25T10:02:51.989367+0000","last_fullsized":"2024-04-25T10:02:51.989367+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:48.354977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063480+0000","last_change":"2024-04-25T10:02:51.003430+0000","last_active":"2024-04-25T10:02:52.063480+0000","last_peered":"2024-04-25T10:02:52.063480+0000","last_clean":"2024-04-25T10:02:52.063480+0000","last_became_active":"2024-04-25T10:02:51.003106+0000","last_became_peered":"2024-04-25T10:02:51.003106+0000","last_unstale":"2024-04-25T10:02:52.063480+0000","last_undegraded":"2024-04-25T10:02:52.063480+0000","last_fullsized":"2024-04-25T10:02:52.063480+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:11.029946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062863+0000","last_change":"2024-04-25T10:02:49.987877+0000","last_active":"2024-04-25T10:02:52.062863+0000","last_peered":"2024-04-25T10:02:52.062863+0000","last_clean":"2024-04-25T10:02:52.062863+0000","last_became_active":"2024-04-25T10:02:49.987609+0000","last_became_peered":"2024-04-25T10:02:49.987609+0000","last_unstale":"2024-04-25T10:02:52.062863+0000","last_undegraded":"2024-04-25T10:02:52.062863+0000","last_fullsized":"2024-04-25T10:02:52.062863+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:30.591432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996371+0000","last_change":"2024-04-25T10:02:50.984814+0000","last_active":"2024-04-25T10:02:51.996371+0000","last_peered":"2024-04-25T10:02:51.996371+0000","last_clean":"2024-04-25T10:02:51.996371+0000","last_became_active":"2024-04-25T10:02:50.984546+0000","last_became_peered":"2024-04-25T10:02:50.984546+0000","last_unstale":"2024-04-25T10:02:51.996371+0000","last_undegraded":"2024-04-25T10:02:51.996371+0000","last_fullsized":"2024-04-25T10:02:51.996371+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:42.368731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063915+0000","last_change":"2024-04-25T10:02:49.990902+0000","last_active":"2024-04-25T10:02:52.063915+0000","last_peered":"2024-04-25T10:02:52.063915+0000","last_clean":"2024-04-25T10:02:52.063915+0000","last_became_active":"2024-04-25T10:02:49.990559+0000","last_became_peered":"2024-04-25T10:02:49.990559+0000","last_unstale":"2024-04-25T10:02:52.063915+0000","last_undegraded":"2024-04-25T10:02:52.063915+0000","last_fullsized":"2024-04-25T10:02:52.063915+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:15.147862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.990617+0000","last_change":"2024-04-25T10:02:50.995715+0000","last_active":"2024-04-25T10:02:51.990617+0000","last_peered":"2024-04-25T10:02:51.990617+0000","last_clean":"2024-04-25T10:02:51.990617+0000","last_became_active":"2024-04-25T10:02:50.995430+0000","last_became_peered":"2024-04-25T10:02:50.995430+0000","last_unstale":"2024-04-25T10:02:51.990617+0000","last_undegraded":"2024-04-25T10:02:51.990617+0000","last_fullsized":"2024-04-25T10:02:51.990617+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:14.143890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063500+0000","last_change":"2024-04-25T10:02:49.993025+0000","last_active":"2024-04-25T10:02:52.063500+0000","last_peered":"2024-04-25T10:02:52.063500+0000","last_clean":"2024-04-25T10:02:52.063500+0000","last_became_active":"2024-04-25T10:02:49.992731+0000","last_became_peered":"2024-04-25T10:02:49.992731+0000","last_unstale":"2024-04-25T10:02:52.063500+0000","last_undegraded":"2024-04-25T10:02:52.063500+0000","last_fullsized":"2024-04-25T10:02:52.063500+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:13.137173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.995990+0000","last_change":"2024-04-25T10:02:50.999319+0000","last_active":"2024-04-25T10:02:51.995990+0000","last_peered":"2024-04-25T10:02:51.995990+0000","last_clean":"2024-04-25T10:02:51.995990+0000","last_became_active":"2024-04-25T10:02:50.999074+0000","last_became_peered":"2024-04-25T10:02:50.999074+0000","last_unstale":"2024-04-25T10:02:51.995990+0000","last_undegraded":"2024-04-25T10:02:51.995990+0000","last_fullsized":"2024-04-25T10:02:51.995990+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:43.256506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996097+0000","last_change":"2024-04-25T10:02:49.995258+0000","last_active":"2024-04-25T10:02:51.996097+0000","last_peered":"2024-04-25T10:02:51.996097+0000","last_clean":"2024-04-25T10:02:51.996097+0000","last_became_active":"2024-04-25T10:02:49.994296+0000","last_became_peered":"2024-04-25T10:02:49.994296+0000","last_unstale":"2024-04-25T10:02:51.996097+0000","last_undegraded":"2024-04-25T10:02:51.996097+0000","last_fullsized":"2024-04-25T10:02:51.996097+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:05.794611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062616+0000","last_change":"2024-04-25T10:02:51.003659+0000","last_active":"2024-04-25T10:02:52.062616+0000","last_peered":"2024-04-25T10:02:52.062616+0000","last_clean":"2024-04-25T10:02:52.062616+0000","last_became_active":"2024-04-25T10:02:51.003384+0000","last_became_peered":"2024-04-25T10:02:51.003384+0000","last_unstale":"2024-04-25T10:02:52.062616+0000","last_undegraded":"2024-04-25T10:02:52.062616+0000","last_fullsized":"2024-04-25T10:02:52.062616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:53.904384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063337+0000","last_change":"2024-04-25T10:02:49.993830+0000","last_active":"2024-04-25T10:02:52.063337+0000","last_peered":"2024-04-25T10:02:52.063337+0000","last_clean":"2024-04-25T10:02:52.063337+0000","last_became_active":"2024-04-25T10:02:49.993548+0000","last_became_peered":"2024-04-25T10:02:49.993548+0000","last_unstale":"2024-04-25T10:02:52.063337+0000","last_undegraded":"2024-04-25T10:02:52.063337+0000","last_fullsized":"2024-04-25T10:02:52.063337+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:52.408138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063335+0000","last_change":"2024-04-25T10:02:50.996477+0000","last_active":"2024-04-25T10:02:52.063335+0000","last_peered":"2024-04-25T10:02:52.063335+0000","last_clean":"2024-04-25T10:02:52.063335+0000","last_became_active":"2024-04-25T10:02:50.995869+0000","last_became_peered":"2024-04-25T10:02:50.995869+0000","last_unstale":"2024-04-25T10:02:52.063335+0000","last_undegraded":"2024-04-25T10:02:52.063335+0000","last_fullsized":"2024-04-25T10:02:52.063335+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:03.428029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062936+0000","last_change":"2024-04-25T10:02:49.989803+0000","last_active":"2024-04-25T10:02:52.062936+0000","last_peered":"2024-04-25T10:02:52.062936+0000","last_clean":"2024-04-25T10:02:52.062936+0000","last_became_active":"2024-04-25T10:02:49.989240+0000","last_became_peered":"2024-04-25T10:02:49.989240+0000","last_unstale":"2024-04-25T10:02:52.062936+0000","last_undegraded":"2024-04-25T10:02:52.062936+0000","last_fullsized":"2024-04-25T10:02:52.062936+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:58.042970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989983+0000","last_change":"2024-04-25T10:02:50.999171+0000","last_active":"2024-04-25T10:02:51.989983+0000","last_peered":"2024-04-25T10:02:51.989983+0000","last_clean":"2024-04-25T10:02:51.989983+0000","last_became_active":"2024-04-25T10:02:50.998886+0000","last_became_peered":"2024-04-25T10:02:50.998886+0000","last_unstale":"2024-04-25T10:02:51.989983+0000","last_undegraded":"2024-04-25T10:02:51.989983+0000","last_fullsized":"2024-04-25T10:02:51.989983+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:43.530691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063065+0000","last_change":"2024-04-25T10:02:49.994508+0000","last_active":"2024-04-25T10:02:52.063065+0000","last_peered":"2024-04-25T10:02:52.063065+0000","last_clean":"2024-04-25T10:02:52.063065+0000","last_became_active":"2024-04-25T10:02:49.994233+0000","last_became_peered":"2024-04-25T10:02:49.994233+0000","last_unstale":"2024-04-25T10:02:52.063065+0000","last_undegraded":"2024-04-25T10:02:52.063065+0000","last_fullsized":"2024-04-25T10:02:52.063065+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:12.458282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996580+0000","last_change":"2024-04-25T10:02:50.987277+0000","last_active":"2024-04-25T10:02:51.996580+0000","last_peered":"2024-04-25T10:02:51.996580+0000","last_clean":"2024-04-25T10:02:51.996580+0000","last_became_active":"2024-04-25T10:02:50.986974+0000","last_became_peered":"2024-04-25T10:02:50.986974+0000","last_unstale":"2024-04-25T10:02:51.996580+0000","last_undegraded":"2024-04-25T10:02:51.996580+0000","last_fullsized":"2024-04-25T10:02:51.996580+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.024928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996432+0000","last_change":"2024-04-25T10:02:49.995093+0000","last_active":"2024-04-25T10:02:51.996432+0000","last_peered":"2024-04-25T10:02:51.996432+0000","last_clean":"2024-04-25T10:02:51.996432+0000","last_became_active":"2024-04-25T10:02:49.994761+0000","last_became_peered":"2024-04-25T10:02:49.994761+0000","last_unstale":"2024-04-25T10:02:51.996432+0000","last_undegraded":"2024-04-25T10:02:51.996432+0000","last_fullsized":"2024-04-25T10:02:51.996432+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:09.732865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989283+0000","last_change":"2024-04-25T10:02:50.995211+0000","last_active":"2024-04-25T10:02:51.989283+0000","last_peered":"2024-04-25T10:02:51.989283+0000","last_clean":"2024-04-25T10:02:51.989283+0000","last_became_active":"2024-04-25T10:02:50.994953+0000","last_became_peered":"2024-04-25T10:02:50.994953+0000","last_unstale":"2024-04-25T10:02:51.989283+0000","last_undegraded":"2024-04-25T10:02:51.989283+0000","last_fullsized":"2024-04-25T10:02:51.989283+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.049687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"120'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.995706+0000","last_change":"2024-04-25T10:02:49.995142+0000","last_active":"2024-04-25T10:02:51.995706+0000","last_peered":"2024-04-25T10:02:51.995706+0000","last_clean":"2024-04-25T10:02:51.995706+0000","last_became_active":"2024-04-25T10:02:49.994845+0000","last_became_peered":"2024-04-25T10:02:49.994845+0000","last_unstale":"2024-04-25T10:02:51.995706+0000","last_undegraded":"2024-04-25T10:02:51.995706+0000","last_fullsized":"2024-04-25T10:02:51.995706+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:03.283380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063905+0000","last_change":"2024-04-25T10:02:51.008017+0000","last_active":"2024-04-25T10:02:52.063905+0000","last_peered":"2024-04-25T10:02:52.063905+0000","last_clean":"2024-04-25T10:02:52.063905+0000","last_became_active":"2024-04-25T10:02:51.007535+0000","last_became_peered":"2024-04-25T10:02:51.007535+0000","last_unstale":"2024-04-25T10:02:52.063905+0000","last_undegraded":"2024-04-25T10:02:52.063905+0000","last_fullsized":"2024-04-25T10:02:52.063905+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:20.548894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.994528+0000","last_change":"2024-04-25T10:02:49.995914+0000","last_active":"2024-04-25T10:02:51.994528+0000","last_peered":"2024-04-25T10:02:51.994528+0000","last_clean":"2024-04-25T10:02:51.994528+0000","last_became_active":"2024-04-25T10:02:49.995574+0000","last_became_peered":"2024-04-25T10:02:49.995574+0000","last_unstale":"2024-04-25T10:02:51.994528+0000","last_undegraded":"2024-04-25T10:02:51.994528+0000","last_fullsized":"2024-04-25T10:02:51.994528+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:21.599505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063402+0000","last_change":"2024-04-25T10:02:51.001795+0000","last_active":"2024-04-25T10:02:52.063402+0000","last_peered":"2024-04-25T10:02:52.063402+0000","last_clean":"2024-04-25T10:02:52.063402+0000","last_became_active":"2024-04-25T10:02:51.001462+0000","last_became_peered":"2024-04-25T10:02:51.001462+0000","last_unstale":"2024-04-25T10:02:52.063402+0000","last_undegraded":"2024-04-25T10:02:52.063402+0000","last_fullsized":"2024-04-25T10:02:52.063402+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:09.590707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996663+0000","last_change":"2024-04-25T10:02:49.992886+0000","last_active":"2024-04-25T10:02:51.996663+0000","last_peered":"2024-04-25T10:02:51.996663+0000","last_clean":"2024-04-25T10:02:51.996663+0000","last_became_active":"2024-04-25T10:02:49.992587+0000","last_became_peered":"2024-04-25T10:02:49.992587+0000","last_unstale":"2024-04-25T10:02:51.996663+0000","last_undegraded":"2024-04-25T10:02:51.996663+0000","last_fullsized":"2024-04-25T10:02:51.996663+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:45.537085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996159+0000","last_change":"2024-04-25T10:02:50.993580+0000","last_active":"2024-04-25T10:02:51.996159+0000","last_peered":"2024-04-25T10:02:51.996159+0000","last_clean":"2024-04-25T10:02:51.996159+0000","last_became_active":"2024-04-25T10:02:50.993284+0000","last_became_peered":"2024-04-25T10:02:50.993284+0000","last_unstale":"2024-04-25T10:02:51.996159+0000","last_undegraded":"2024-04-25T10:02:51.996159+0000","last_fullsized":"2024-04-25T10:02:51.996159+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:41.124412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063601+0000","last_change":"2024-04-25T10:02:49.989201+0000","last_active":"2024-04-25T10:02:52.063601+0000","last_peered":"2024-04-25T10:02:52.063601+0000","last_clean":"2024-04-25T10:02:52.063601+0000","last_became_active":"2024-04-25T10:02:49.988837+0000","last_became_peered":"2024-04-25T10:02:49.988837+0000","last_unstale":"2024-04-25T10:02:52.063601+0000","last_undegraded":"2024-04-25T10:02:52.063601+0000","last_fullsized":"2024-04-25T10:02:52.063601+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.201776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062748+0000","last_change":"2024-04-25T10:02:50.985343+0000","last_active":"2024-04-25T10:02:52.062748+0000","last_peered":"2024-04-25T10:02:52.062748+0000","last_clean":"2024-04-25T10:02:52.062748+0000","last_became_active":"2024-04-25T10:02:50.985077+0000","last_became_peered":"2024-04-25T10:02:50.985077+0000","last_unstale":"2024-04-25T10:02:52.062748+0000","last_undegraded":"2024-04-25T10:02:52.062748+0000","last_fullsized":"2024-04-25T10:02:52.062748+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:11.709040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063367+0000","last_change":"2024-04-25T10:02:49.991022+0000","last_active":"2024-04-25T10:02:52.063367+0000","last_peered":"2024-04-25T10:02:52.063367+0000","last_clean":"2024-04-25T10:02:52.063367+0000","last_became_active":"2024-04-25T10:02:49.990751+0000","last_became_peered":"2024-04-25T10:02:49.990751+0000","last_unstale":"2024-04-25T10:02:52.063367+0000","last_undegraded":"2024-04-25T10:02:52.063367+0000","last_fullsized":"2024-04-25T10:02:52.063367+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:44.159091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063223+0000","last_change":"2024-04-25T10:02:51.001087+0000","last_active":"2024-04-25T10:02:52.063223+0000","last_peered":"2024-04-25T10:02:52.063223+0000","last_clean":"2024-04-25T10:02:52.063223+0000","last_became_active":"2024-04-25T10:02:51.000508+0000","last_became_peered":"2024-04-25T10:02:51.000508+0000","last_unstale":"2024-04-25T10:02:52.063223+0000","last_undegraded":"2024-04-25T10:02:52.063223+0000","last_fullsized":"2024-04-25T10:02:52.063223+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:47.908802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.996063+0000","last_change":"2024-04-25T10:02:49.993764+0000","last_active":"2024-04-25T10:02:51.996063+0000","last_peered":"2024-04-25T10:02:51.996063+0000","last_clean":"2024-04-25T10:02:51.996063+0000","last_became_active":"2024-04-25T10:02:49.993250+0000","last_became_peered":"2024-04-25T10:02:49.993250+0000","last_unstale":"2024-04-25T10:02:51.996063+0000","last_undegraded":"2024-04-25T10:02:51.996063+0000","last_fullsized":"2024-04-25T10:02:51.996063+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:59.534880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:51.989402+0000","last_change":"2024-04-25T10:02:50.997672+0000","last_active":"2024-04-25T10:02:51.989402+0000","last_peered":"2024-04-25T10:02:51.989402+0000","last_clean":"2024-04-25T10:02:51.989402+0000","last_became_active":"2024-04-25T10:02:50.997401+0000","last_became_peered":"2024-04-25T10:02:50.997401+0000","last_unstale":"2024-04-25T10:02:51.989402+0000","last_undegraded":"2024-04-25T10:02:51.989402+0000","last_fullsized":"2024-04-25T10:02:51.989402+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:43.530691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063485+0000","last_change":"2024-04-25T10:02:49.991498+0000","last_active":"2024-04-25T10:02:52.063485+0000","last_peered":"2024-04-25T10:02:52.063485+0000","last_clean":"2024-04-25T10:02:52.063485+0000","last_became_active":"2024-04-25T10:02:49.990815+0000","last_became_peered":"2024-04-25T10:02:49.990815+0000","last_unstale":"2024-04-25T10:02:52.063485+0000","last_undegraded":"2024-04-25T10:02:52.063485+0000","last_fullsized":"2024-04-25T10:02:52.063485+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:03.650136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.064058+0000","last_change":"2024-04-25T10:02:51.005215+0000","last_active":"2024-04-25T10:02:52.064058+0000","last_peered":"2024-04-25T10:02:52.064058+0000","last_clean":"2024-04-25T10:02:52.064058+0000","last_became_active":"2024-04-25T10:02:51.004932+0000","last_became_peered":"2024-04-25T10:02:51.004932+0000","last_unstale":"2024-04-25T10:02:52.064058+0000","last_undegraded":"2024-04-25T10:02:52.064058+0000","last_fullsized":"2024-04-25T10:02:52.064058+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:49.966130+0000","last_clean_scrub_stamp":"2024-04-25T10:02:49.966130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:38.716882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"119'8","reported_seq":22,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.062602+0000","last_change":"2024-04-25T10:02:49.992124+0000","last_active":"2024-04-25T10:02:52.062602+0000","last_peered":"2024-04-25T10:02:52.062602+0000","last_clean":"2024-04-25T10:02:52.062602+0000","last_became_active":"2024-04-25T10:02:49.991903+0000","last_became_peered":"2024-04-25T10:02:49.991903+0000","last_unstale":"2024-04-25T10:02:52.062602+0000","last_undegraded":"2024-04-25T10:02:52.062602+0000","last_fullsized":"2024-04-25T10:02:52.062602+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.959413+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.959413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:56.664777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":355,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:02:52.063603+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T10:02:52.063603+0000","last_peered":"2024-04-25T10:02:52.063603+0000","last_clean":"2024-04-25T10:02:52.063603+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T10:02:52.063603+0000","last_undegraded":"2024-04-25T10:02:52.063603+0000","last_fullsized":"2024-04-25T10:02:52.063603+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574970,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2105344,"data_stored":1289129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":698567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1503232,"data_stored":698145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1507328,"data_stored":699597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1511424,"data_stored":699631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1495040,"data_stored":697985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":698007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574970,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2093056,"data_stored":1288375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:54.264 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:02:54.264 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:02:54.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.400+0000 7f8c3dbe0640 1 Processor -- start 2024-04-25T10:02:54.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.400+0000 7f8c3dbe0640 1 -- start start 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c3dbe0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8c380afc80 0x7f8c380b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c3dbe0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8c3806b700 0x7f8c3806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c3dbe0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c380c6cc0 con 0x7f8c380af8a0 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c3dbe0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c380c6fa0 con 0x7f8c3806b700 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c3dbe0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c380c7280 con 0x7f8c380afc80 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c37fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8c3806b700 0x7f8c3806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c377fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8c380af8a0 0x7f8c380cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35566/0 (socket says 172.21.15.139:35566) 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c377fe640 1 -- 172.21.15.139:0/111758837 learned_addr learned my addr 172.21.15.139:0/111758837 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c36ffd640 1 --2- 172.21.15.139:0/111758837 >> v2:172.21.15.150:3300/0 conn(0x7f8c380afc80 0x7f8c380b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c37fff640 1 -- 172.21.15.139:0/111758837 >> v2:172.21.15.150:3300/0 conn(0x7f8c380afc80 msgr2=0x7f8c380b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c37fff640 1 --2- 172.21.15.139:0/111758837 >> v2:172.21.15.150:3300/0 conn(0x7f8c380afc80 0x7f8c380b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c37fff640 1 -- 172.21.15.139:0/111758837 >> v1:172.21.15.139:6789/0 conn(0x7f8c380af8a0 legacy=0x7f8c380cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c37fff640 1 -- 172.21.15.139:0/111758837 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c380cd090 con 0x7f8c3806b700 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.401+0000 7f8c36ffd640 1 --2- 172.21.15.139:0/111758837 >> v2:172.21.15.150:3300/0 conn(0x7f8c380afc80 0x7f8c380b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.402+0000 7f8c37fff640 1 --2- 172.21.15.139:0/111758837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c3806b700 0x7f8c3806ab60 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f8c3806f650 tx=0x7f8c3002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfef81870a99522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.402+0000 7f8c367fc640 1 -- 172.21.15.139:0/111758837 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30007740 con 0x7f8c3806b700 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.402+0000 7f8c367fc640 1 -- 172.21.15.139:0/111758837 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c300078e0 con 0x7f8c3806b700 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.402+0000 7f8c367fc640 1 -- 172.21.15.139:0/111758837 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30007cb0 con 0x7f8c3806b700 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.402+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/111758837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c3806b700 msgr2=0x7f8c3806ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.402+0000 7f8c3dbe0640 1 --2- 172.21.15.139:0/111758837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c3806b700 0x7f8c3806ab60 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f8c3806f650 tx=0x7f8c3002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.402+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/111758837 shutdown_connections 2024-04-25T10:02:54.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.402+0000 7f8c3dbe0640 1 --2- 172.21.15.139:0/111758837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c3806b700 0x7f8c3806ab60 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.402+0000 7f8c3dbe0640 1 --2- 172.21.15.139:0/111758837 >> v2:172.21.15.150:3300/0 conn(0x7f8c380afc80 0x7f8c380b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.402+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/111758837 >> 172.21.15.139:0/111758837 conn(0x7f8c380b2230 msgr2=0x7f8c380b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.402+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/111758837 shutdown_connections 2024-04-25T10:02:54.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.402+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/111758837 wait complete. 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.403+0000 7f8c3dbe0640 1 Processor -- start 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.404+0000 7f8c3dbe0640 1 -- start start 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.404+0000 7f8c3dbe0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3806b700 0x7f8c38160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.404+0000 7f8c3dbe0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c380af8a0 0x7f8c3815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.404+0000 7f8c3dbe0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c380afc80 0x7f8c380c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.404+0000 7f8c3dbe0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3806b220 con 0x7f8c380af8a0 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.404+0000 7f8c3dbe0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3806b0a0 con 0x7f8c3806b700 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.404+0000 7f8c3dbe0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3806b3a0 con 0x7f8c380afc80 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.404+0000 7f8c36ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c380af8a0 0x7f8c3815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c36ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c380af8a0 0x7f8c3815a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54240/0 (socket says 172.21.15.139:54240) 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c36ffd640 1 -- 172.21.15.139:0/2254818720 learned_addr learned my addr 172.21.15.139:0/2254818720 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c36ffd640 1 -- 172.21.15.139:0/2254818720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c380afc80 msgr2=0x7f8c380c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c36ffd640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c380afc80 0x7f8c380c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c36ffd640 1 -- 172.21.15.139:0/2254818720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3806b700 msgr2=0x7f8c38160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c377fe640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3806b700 0x7f8c38160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c36ffd640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3806b700 0x7f8c38160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c36ffd640 1 -- 172.21.15.139:0/2254818720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c381611f0 con 0x7f8c380af8a0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c36ffd640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c380af8a0 0x7f8c3815a8a0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00c7f0 tx=0x7f8c2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c1ffff640 1 -- 172.21.15.139:0/2254818720 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c015070 con 0x7f8c380af8a0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c1ffff640 1 -- 172.21.15.139:0/2254818720 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2c007820 con 0x7f8c380af8a0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/2254818720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c381f40c0 con 0x7f8c380af8a0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c1ffff640 1 -- 172.21.15.139:0/2254818720 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c007c10 con 0x7f8c380af8a0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.405+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/2254818720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c381601d0 con 0x7f8c380af8a0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.406+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/2254818720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c38074520 con 0x7f8c380af8a0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.407+0000 7f8c1ffff640 1 -- 172.21.15.139:0/2254818720 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c2c018070 con 0x7f8c380af8a0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.407+0000 7f8c1ffff640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c2406b4d0 0x7f8c2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.407+0000 7f8c377fe640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c2406b4d0 0x7f8c2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.407+0000 7f8c1ffff640 1 -- 172.21.15.139:0/2254818720 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c2c08a400 con 0x7f8c380af8a0 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.408+0000 7f8c377fe640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c2406b4d0 0x7f8c2406d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8c28000f60 tx=0x7f8c28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.411+0000 7f8c1ffff640 1 -- 172.21.15.139:0/2254818720 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c2c059180 con 0x7f8c380af8a0 2024-04-25T10:02:54.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.648+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/2254818720 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8c380cef90 con 0x7f8c380af8a0 2024-04-25T10:02:54.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.650+0000 7f8c1ffff640 1 -- 172.21.15.139:0/2254818720 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8c2c058a80 con 0x7f8c380af8a0 2024-04-25T10:02:54.655 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2024-04-25T10:02:54.655 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDRJypmz7RHNBAAc+ENNC8jQLwloOtyNMEjNg== 2024-04-25T10:02:54.655 INFO:teuthology.orchestra.run.smithi139.stdout: caps mds = "allow" 2024-04-25T10:02:54.655 INFO:teuthology.orchestra.run.smithi139.stdout: caps mon = "allow r" 2024-04-25T10:02:54.655 INFO:teuthology.orchestra.run.smithi139.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:02:54.656 INFO:teuthology.orchestra.run.smithi139.stderr:updated caps for client.0 2024-04-25T10:02:54.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.652+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/2254818720 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c2406b4d0 msgr2=0x7f8c2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.652+0000 7f8c3dbe0640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c2406b4d0 0x7f8c2406d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8c28000f60 tx=0x7f8c28009040 comp rx=0 tx=0).stop 2024-04-25T10:02:54.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.652+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/2254818720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c380af8a0 msgr2=0x7f8c3815a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.652+0000 7f8c3dbe0640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c380af8a0 0x7f8c3815a8a0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00c7f0 tx=0x7f8c2c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.652+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/2254818720 shutdown_connections 2024-04-25T10:02:54.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.652+0000 7f8c3dbe0640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c380af8a0 0x7f8c3815a8a0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.653+0000 7f8c3dbe0640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8c2406b4d0 0x7f8c2406d990 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.653+0000 7f8c3dbe0640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c380afc80 0x7f8c380c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.653+0000 7f8c3dbe0640 1 --2- 172.21.15.139:0/2254818720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c3806b700 0x7f8c38160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.653+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/2254818720 >> 172.21.15.139:0/2254818720 conn(0x7f8c380b2230 msgr2=0x7f8c3815d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.653+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/2254818720 shutdown_connections 2024-04-25T10:02:54.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.653+0000 7f8c3dbe0640 1 -- 172.21.15.139:0/2254818720 wait complete. 2024-04-25T10:02:54.668 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:54.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.760+0000 7fec342a1640 1 Processor -- start 2024-04-25T10:02:54.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.760+0000 7fec342a1640 1 -- start start 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.761+0000 7fec342a1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fec2c15efb0 0x7fec2c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.761+0000 7fec342a1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fec2c099a50 0x7fec2c099e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.761+0000 7fec342a1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2c0cfe70 con 0x7fec2c0cf900 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.761+0000 7fec342a1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2c0d0a80 con 0x7fec2c099a50 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.761+0000 7fec342a1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2c09a3c0 con 0x7fec2c15efb0 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.761+0000 7fec3283e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fec2c099a50 0x7fec2c099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.761+0000 7fec3183c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fec2c0cf900 0x7fec2c06c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35572/0 (socket says 172.21.15.139:35572) 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.761+0000 7fec3183c640 1 -- 172.21.15.139:0/1749205289 learned_addr learned my addr 172.21.15.139:0/1749205289 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.762+0000 7fec3283e640 1 -- 172.21.15.139:0/1749205289 >> v2:172.21.15.150:3300/0 conn(0x7fec2c15efb0 msgr2=0x7fec2c0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.762+0000 7fec3203d640 1 --2- 172.21.15.139:0/1749205289 >> v2:172.21.15.150:3300/0 conn(0x7fec2c15efb0 0x7fec2c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.762+0000 7fec3103b640 1 -- 172.21.15.139:0/1749205289 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069600586 0 0) 0x7fec2c0cfe70 con 0x7fec2c0cf900 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.762+0000 7fec3283e640 1 --2- 172.21.15.139:0/1749205289 >> v2:172.21.15.150:3300/0 conn(0x7fec2c15efb0 0x7fec2c0d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.762+0000 7fec3283e640 1 -- 172.21.15.139:0/1749205289 >> v1:172.21.15.139:6789/0 conn(0x7fec2c0cf900 legacy=0x7fec2c06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.762+0000 7fec3283e640 1 -- 172.21.15.139:0/1749205289 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c0d0150 con 0x7fec2c099a50 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.762+0000 7fec3203d640 1 --2- 172.21.15.139:0/1749205289 >> v2:172.21.15.150:3300/0 conn(0x7fec2c15efb0 0x7fec2c0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.762+0000 7fec3283e640 1 --2- 172.21.15.139:0/1749205289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec2c099a50 0x7fec2c099e80 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fec2c072c10 tx=0x7fec2402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ead4f78cdcde2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.762+0000 7fec3103b640 1 -- 172.21.15.139:0/1749205289 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24007740 con 0x7fec2c099a50 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.762+0000 7fec3103b640 1 -- 172.21.15.139:0/1749205289 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec240078e0 con 0x7fec2c099a50 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.762+0000 7fec3103b640 1 -- 172.21.15.139:0/1749205289 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24007cb0 con 0x7fec2c099a50 2024-04-25T10:02:54.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.763+0000 7fec342a1640 1 -- 172.21.15.139:0/1749205289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec2c099a50 msgr2=0x7fec2c099e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.763+0000 7fec342a1640 1 --2- 172.21.15.139:0/1749205289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec2c099a50 0x7fec2c099e80 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fec2c072c10 tx=0x7fec2402a5f0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.763+0000 7fec342a1640 1 -- 172.21.15.139:0/1749205289 shutdown_connections 2024-04-25T10:02:54.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.763+0000 7fec342a1640 1 --2- 172.21.15.139:0/1749205289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec2c099a50 0x7fec2c099e80 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.763+0000 7fec342a1640 1 --2- 172.21.15.139:0/1749205289 >> v2:172.21.15.150:3300/0 conn(0x7fec2c15efb0 0x7fec2c0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.763+0000 7fec342a1640 1 -- 172.21.15.139:0/1749205289 >> 172.21.15.139:0/1749205289 conn(0x7fec2c0d1790 msgr2=0x7fec2c0d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.763+0000 7fec342a1640 1 -- 172.21.15.139:0/1749205289 shutdown_connections 2024-04-25T10:02:54.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.763+0000 7fec342a1640 1 -- 172.21.15.139:0/1749205289 wait complete. 2024-04-25T10:02:54.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.764+0000 7fec342a1640 1 Processor -- start 2024-04-25T10:02:54.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.764+0000 7fec342a1640 1 -- start start 2024-04-25T10:02:54.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec342a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec2c099a50 0x7fec2c15f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec342a1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec2c0cf900 0x7fec2c1f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec342a1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec2c15efb0 0x7fec2c0c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec342a1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c06cf50 con 0x7fec2c099a50 2024-04-25T10:02:54.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec342a1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0d0710 con 0x7fec2c0cf900 2024-04-25T10:02:54.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec342a1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0cbde0 con 0x7fec2c15efb0 2024-04-25T10:02:54.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec3203d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec2c099a50 0x7fec2c15f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec3203d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec2c099a50 0x7fec2c15f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54258/0 (socket says 172.21.15.139:54258) 2024-04-25T10:02:54.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec3203d640 1 -- 172.21.15.139:0/3631102705 learned_addr learned my addr 172.21.15.139:0/3631102705 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:54.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec3203d640 1 -- 172.21.15.139:0/3631102705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec2c15efb0 msgr2=0x7fec2c0c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec3283e640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec2c15efb0 0x7fec2c0c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec3203d640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec2c15efb0 0x7fec2c0c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec3183c640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec2c0cf900 0x7fec2c1f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec3203d640 1 -- 172.21.15.139:0/3631102705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec2c0cf900 msgr2=0x7fec2c1f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec3203d640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec2c0cf900 0x7fec2c1f4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.765+0000 7fec3203d640 1 -- 172.21.15.139:0/3631102705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c15fa20 con 0x7fec2c099a50 2024-04-25T10:02:54.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.766+0000 7fec3203d640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec2c099a50 0x7fec2c15f390 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fec2800c7f0 tx=0x7fec2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.766+0000 7fec22ffd640 1 -- 172.21.15.139:0/3631102705 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28015070 con 0x7fec2c099a50 2024-04-25T10:02:54.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.766+0000 7fec22ffd640 1 -- 172.21.15.139:0/3631102705 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec28007820 con 0x7fec2c099a50 2024-04-25T10:02:54.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.766+0000 7fec22ffd640 1 -- 172.21.15.139:0/3631102705 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28007bd0 con 0x7fec2c099a50 2024-04-25T10:02:54.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.766+0000 7fec342a1640 1 -- 172.21.15.139:0/3631102705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2c15fd00 con 0x7fec2c099a50 2024-04-25T10:02:54.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.766+0000 7fec342a1640 1 -- 172.21.15.139:0/3631102705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c0c5360 con 0x7fec2c099a50 2024-04-25T10:02:54.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.767+0000 7fec342a1640 1 -- 172.21.15.139:0/3631102705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2c074520 con 0x7fec2c099a50 2024-04-25T10:02:54.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.767+0000 7fec22ffd640 1 -- 172.21.15.139:0/3631102705 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec28018070 con 0x7fec2c099a50 2024-04-25T10:02:54.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.768+0000 7fec22ffd640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fec1006b4d0 0x7fec1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.768+0000 7fec22ffd640 1 -- 172.21.15.139:0/3631102705 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fec2808a500 con 0x7fec2c099a50 2024-04-25T10:02:54.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.768+0000 7fec3183c640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fec1006b4d0 0x7fec1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.768+0000 7fec3183c640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fec1006b4d0 0x7fec1006d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fec1c007fd0 tx=0x7fec1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.772+0000 7fec22ffd640 1 -- 172.21.15.139:0/3631102705 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec280591f0 con 0x7fec2c099a50 2024-04-25T10:02:54.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.980+0000 7fec342a1640 1 -- 172.21.15.139:0/3631102705 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec2c09ae50 con 0x7fec2c099a50 2024-04-25T10:02:54.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.981+0000 7fec22ffd640 1 -- 172.21.15.139:0/3631102705 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fec28059e20 con 0x7fec2c099a50 2024-04-25T10:02:54.986 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:54.986 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":83,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/3478297010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3478297010},{"type":"v1","addr":"172.21.15.139:6836","nonce":3478297010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:02:52.320205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9132},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9132":{"gid":9132,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/1377607596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":1377607596},{"type":"v1","addr":"172.21.15.150:6835","nonce":1377607596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:54.986 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 83 2024-04-25T10:02:54.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.983+0000 7fec342a1640 1 -- 172.21.15.139:0/3631102705 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fec1006b4d0 msgr2=0x7fec1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.983+0000 7fec342a1640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fec1006b4d0 0x7fec1006d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fec1c007fd0 tx=0x7fec1c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:54.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.983+0000 7fec342a1640 1 -- 172.21.15.139:0/3631102705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec2c099a50 msgr2=0x7fec2c15f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.983+0000 7fec342a1640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec2c099a50 0x7fec2c15f390 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fec2800c7f0 tx=0x7fec2800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.983+0000 7fec342a1640 1 -- 172.21.15.139:0/3631102705 shutdown_connections 2024-04-25T10:02:54.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.983+0000 7fec342a1640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fec1006b4d0 0x7fec1006d990 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.983+0000 7fec342a1640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec2c15efb0 0x7fec2c0c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.983+0000 7fec342a1640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec2c0cf900 0x7fec2c1f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.983+0000 7fec342a1640 1 --2- 172.21.15.139:0/3631102705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec2c099a50 0x7fec2c15f390 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.983+0000 7fec342a1640 1 -- 172.21.15.139:0/3631102705 >> 172.21.15.139:0/3631102705 conn(0x7fec2c0d1790 msgr2=0x7fec2c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.983+0000 7fec342a1640 1 -- 172.21.15.139:0/3631102705 shutdown_connections 2024-04-25T10:02:54.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:54.983+0000 7fec342a1640 1 -- 172.21.15.139:0/3631102705 wait complete. 2024-04-25T10:02:55.401 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:50.319874+0000', 'modified': '2024-04-25T10:02:52.320205+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9132}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9132': {'gid': 9132, 'name': 'd', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6835/1377607596', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6833', 'nonce': 1377607596}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 1377607596}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:02:55.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:55.402 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:02:55.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.497+0000 7f77b1e38640 1 Processor -- start 2024-04-25T10:02:55.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.497+0000 7f77b1e38640 1 -- start start 2024-04-25T10:02:55.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77b1e38640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f77ac15dbf0 0x7f77ac1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77b1e38640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77ac0cee50 0x7f77ac0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77b1e38640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77ac161dc0 con 0x7f77ac0b9370 2024-04-25T10:02:55.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77b1e38640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77ac0d1ee0 con 0x7f77ac15dbf0 2024-04-25T10:02:55.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77b1e38640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac1620a0 con 0x7f77ac0cee50 2024-04-25T10:02:55.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77ab7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f77ac15dbf0 0x7f77ac1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77ab7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f77ac15dbf0 0x7f77ac1617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54264/0 (socket says 172.21.15.139:54264) 2024-04-25T10:02:55.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77ab7fe640 1 -- 172.21.15.139:0/323489190 learned_addr learned my addr 172.21.15.139:0/323489190 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:55.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77abfff640 1 --2- 172.21.15.139:0/323489190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77ac0cee50 0x7f77ac0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77ab7fe640 1 -- 172.21.15.139:0/323489190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77ac0cee50 msgr2=0x7f77ac0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77ab7fe640 1 --2- 172.21.15.139:0/323489190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77ac0cee50 0x7f77ac0d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77ab7fe640 1 -- 172.21.15.139:0/323489190 >> v1:172.21.15.150:6789/0 conn(0x7f77ac0b9370 legacy=0x7f77ac0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.498+0000 7f77ab7fe640 1 -- 172.21.15.139:0/323489190 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac162380 con 0x7f77ac15dbf0 2024-04-25T10:02:55.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.499+0000 7f77ab7fe640 1 --2- 172.21.15.139:0/323489190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ac15dbf0 0x7f77ac1617e0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009870 tx=0x7f77a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a254c01a4ae61a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.499+0000 7f77aa7fc640 1 -- 172.21.15.139:0/323489190 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0037070 con 0x7f77ac15dbf0 2024-04-25T10:02:55.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.499+0000 7f77aa7fc640 1 -- 172.21.15.139:0/323489190 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a0032440 con 0x7f77ac15dbf0 2024-04-25T10:02:55.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.499+0000 7f77aa7fc640 1 -- 172.21.15.139:0/323489190 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0032810 con 0x7f77ac15dbf0 2024-04-25T10:02:55.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.500+0000 7f77b1e38640 1 -- 172.21.15.139:0/323489190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ac15dbf0 msgr2=0x7f77ac1617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.500+0000 7f77b1e38640 1 --2- 172.21.15.139:0/323489190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ac15dbf0 0x7f77ac1617e0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009870 tx=0x7f77a002a270 comp rx=0 tx=0).stop 2024-04-25T10:02:55.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.500+0000 7f77b1e38640 1 -- 172.21.15.139:0/323489190 shutdown_connections 2024-04-25T10:02:55.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.500+0000 7f77b1e38640 1 --2- 172.21.15.139:0/323489190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77ac0cee50 0x7f77ac0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.500+0000 7f77b1e38640 1 --2- 172.21.15.139:0/323489190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ac15dbf0 0x7f77ac1617e0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.500+0000 7f77b1e38640 1 -- 172.21.15.139:0/323489190 >> 172.21.15.139:0/323489190 conn(0x7f77ac0b5480 msgr2=0x7f77ac0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.500+0000 7f77b1e38640 1 -- 172.21.15.139:0/323489190 shutdown_connections 2024-04-25T10:02:55.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.500+0000 7f77b1e38640 1 -- 172.21.15.139:0/323489190 wait complete. 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.501+0000 7f77b1e38640 1 Processor -- start 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.501+0000 7f77b1e38640 1 -- start start 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77b1e38640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77ac0b9370 0x7f77ac0aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77b1e38640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ac0cee50 0x7f77ac1f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77b1e38640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77ac15dbf0 0x7f77ac15cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77b1e38640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac15dfd0 con 0x7f77ac0cee50 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77b1e38640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac0b97d0 con 0x7f77ac0b9370 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77b1e38640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac15be70 con 0x7f77ac15dbf0 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ac0cee50 0x7f77ac1f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ac0cee50 0x7f77ac1f2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54280/0 (socket says 172.21.15.139:54280) 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77aaffd640 1 -- 172.21.15.139:0/1891836456 learned_addr learned my addr 172.21.15.139:0/1891836456 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77ab7fe640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77ac0b9370 0x7f77ac0aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77abfff640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77ac15dbf0 0x7f77ac15cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77aaffd640 1 -- 172.21.15.139:0/1891836456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77ac15dbf0 msgr2=0x7f77ac15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77aaffd640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77ac15dbf0 0x7f77ac15cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77aaffd640 1 -- 172.21.15.139:0/1891836456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77ac0b9370 msgr2=0x7f77ac0aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77aaffd640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77ac0b9370 0x7f77ac0aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77aaffd640 1 -- 172.21.15.139:0/1891836456 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac15c150 con 0x7f77ac0cee50 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77aaffd640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ac0cee50 0x7f77ac1f2e60 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c3c0 tx=0x7f779c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f7793fff640 1 -- 172.21.15.139:0/1891836456 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c0052d0 con 0x7f77ac0cee50 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f7793fff640 1 -- 172.21.15.139:0/1891836456 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c005490 con 0x7f77ac0cee50 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77b1e38640 1 -- 172.21.15.139:0/1891836456 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ac15c490 con 0x7f77ac0cee50 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f7793fff640 1 -- 172.21.15.139:0/1891836456 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c007260 con 0x7f77ac0cee50 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.502+0000 7f77b1e38640 1 -- 172.21.15.139:0/1891836456 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ac0ab210 con 0x7f77ac0cee50 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.504+0000 7f7793fff640 1 -- 172.21.15.139:0/1891836456 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f779c007400 con 0x7f77ac0cee50 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.504+0000 7f7793fff640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f779806b4d0 0x7f779806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.505+0000 7f77ab7fe640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f779806b4d0 0x7f779806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.505+0000 7f7793fff640 1 -- 172.21.15.139:0/1891836456 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f779c089860 con 0x7f77ac0cee50 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.505+0000 7f77ab7fe640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f779806b4d0 0x7f779806d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f77a0034000 tx=0x7f77a002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.505+0000 7f77b4c15740 1 Processor -- start 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.505+0000 7f77b4c15740 1 -- start start 2024-04-25T10:02:55.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.506+0000 7f77b4c15740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66cf230 0x5600f66cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.506+0000 7f77b4c15740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 0x5600f66ed020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.506+0000 7f77b4c15740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66a4f10 0x5600f66eb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.506+0000 7f77b4c15740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f6607a10 con 0x5600f66ecad0 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.506+0000 7f77b4c15740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f65b0b30 con 0x5600f66a4f10 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.506+0000 7f77b4c15740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600f6603c50 con 0x5600f66cf230 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.506+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 0x5600f66ed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.506+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 0x5600f66ed020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54282/0 (socket says 172.21.15.139:54282) 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.506+0000 7f77aaffd640 1 -- 172.21.15.139:0/3903329429 learned_addr learned my addr 172.21.15.139:0/3903329429 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.507+0000 7f77aaffd640 1 -- 172.21.15.139:0/3903329429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66cf230 msgr2=0x5600f66cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.507+0000 7f77aaffd640 1 --2- 172.21.15.139:0/3903329429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66cf230 0x5600f66cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.507+0000 7f77abfff640 1 --2- 172.21.15.139:0/3903329429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66a4f10 0x5600f66eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.507+0000 7f77aaffd640 1 -- 172.21.15.139:0/3903329429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66a4f10 msgr2=0x5600f66eb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.507+0000 7f77aaffd640 1 --2- 172.21.15.139:0/3903329429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66a4f10 0x5600f66eb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.507+0000 7f77aaffd640 1 -- 172.21.15.139:0/3903329429 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600f65c5e10 con 0x5600f66ecad0 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.507+0000 7f77aaffd640 1 --2- 172.21.15.139:0/3903329429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 0x5600f66ed020 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f779c05ff90 tx=0x7f779c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e43fb39e93f1a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.507+0000 7f77917fa640 1 -- 172.21.15.139:0/3903329429 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c056160 con 0x5600f66ecad0 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.507+0000 7f77917fa640 1 -- 172.21.15.139:0/3903329429 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c05b070 con 0x5600f66ecad0 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.507+0000 7f77917fa640 1 -- 172.21.15.139:0/3903329429 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c0544d0 con 0x5600f66ecad0 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.508+0000 7f77b4c15740 1 -- 172.21.15.139:0/3903329429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 msgr2=0x5600f66ed020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.508+0000 7f77b4c15740 1 --2- 172.21.15.139:0/3903329429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 0x5600f66ed020 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f779c05ff90 tx=0x7f779c00a000 comp rx=0 tx=0).stop 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.508+0000 7f77b4c15740 1 -- 172.21.15.139:0/3903329429 shutdown_connections 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.508+0000 7f77b4c15740 1 --2- 172.21.15.139:0/3903329429 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 0x5600f66ed020 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.508+0000 7f77b4c15740 1 --2- 172.21.15.139:0/3903329429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66a4f10 0x5600f66eb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.508+0000 7f77b4c15740 1 --2- 172.21.15.139:0/3903329429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66cf230 0x5600f66cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.508+0000 7f77b4c15740 1 -- 172.21.15.139:0/3903329429 >> 172.21.15.139:0/3903329429 conn(0x5600f66a1840 msgr2=0x5600f66ea040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.508+0000 7f77b4c15740 1 -- 172.21.15.139:0/3903329429 shutdown_connections 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.508+0000 7f77b4c15740 1 -- 172.21.15.139:0/3903329429 wait complete. 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.508+0000 7f77b4c15740 1 -- start start 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.508+0000 7f77b4c15740 1 Processor -- start 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77b4c15740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66cf230 0x5600f66d1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77b4c15740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 0x5600f66d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77b4c15740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66ba6c0 0x5600f66f9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77b4c15740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f66a5cb0 con 0x5600f66ecad0 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77b4c15740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f65c40c0 con 0x5600f66cf230 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77b4c15740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600f669f770 con 0x5600f66ba6c0 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77b4c15740 10 client.0 _ll_register_callbacks cb 0x5600f65b7ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 0x5600f66d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 0x5600f66d21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54294/0 (socket says 172.21.15.139:54294) 2024-04-25T10:02:55.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77aaffd640 1 -- 172.21.15.139:0/1740489942 learned_addr learned my addr 172.21.15.139:0/1740489942 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:55.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77aaffd640 1 -- 172.21.15.139:0/1740489942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66ba6c0 msgr2=0x5600f66f9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77aaffd640 1 --2- 172.21.15.139:0/1740489942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66ba6c0 0x5600f66f9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77aaffd640 1 -- 172.21.15.139:0/1740489942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66cf230 msgr2=0x5600f66d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77aaffd640 1 --2- 172.21.15.139:0/1740489942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66cf230 0x5600f66d1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.509+0000 7f77aaffd640 1 -- 172.21.15.139:0/1740489942 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600f631c2d0 con 0x5600f66ecad0 2024-04-25T10:02:55.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f77aaffd640 1 --2- 172.21.15.139:0/1740489942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 0x5600f66d21a0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f779c087ad0 tx=0x7f779c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c0988457bb4f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f778a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:02:55.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f778a7fc640 1 -- 172.21.15.139:0/1740489942 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c056220 con 0x5600f66ecad0 2024-04-25T10:02:55.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f778a7fc640 1 -- 172.21.15.139:0/1740489942 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f779c0572b0 con 0x5600f66ecad0 2024-04-25T10:02:55.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f77b4c15740 1 -- 172.21.15.139:0/1740489942 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f779c05b070 con 0x5600f66ecad0 2024-04-25T10:02:55.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f778a7fc640 1 -- 172.21.15.139:0/1740489942 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f779c0574b0 con 0x5600f66ecad0 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f77b4c15740 10 client.9279 fetch_fsmap learned FSMap version 83 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f77b4c15740 1 -- 172.21.15.139:0/1740489942 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x5600f63e66b0 con 0x5600f66ecad0 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f778a7fc640 1 -- 172.21.15.139:0/1740489942 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c05b070 con 0x5600f66ecad0 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f778a7fc640 1 -- 172.21.15.139:0/1740489942 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f779c06fb40 con 0x5600f66ecad0 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f77b4c15740 10 client.9279 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f77b4c15740 10 client.9279 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f77b4c15740 20 client.9279 populate_metadata read hostname 'smithi139' 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f77b4c15740 1 --2- 172.21.15.139:0/1740489942 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x5600f66bb910 0x5600f66bbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f77b4c15740 4 client.9279 mds_command: new command op to 9132 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f77b4c15740 1 -- 172.21.15.139:0/1740489942 --> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600f6666920 con 0x5600f66bb910 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.510+0000 7f77ab7fe640 1 --2- 172.21.15.139:0/1740489942 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x5600f66bb910 0x5600f66bbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.511+0000 7f778a7fc640 10 client.9279 ms_handle_connect on v2:172.21.15.150:6833/1377607596 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.511+0000 7f77ab7fe640 1 --2- 172.21.15.139:0/1740489942 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x5600f66bb910 0x5600f66bbcf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5573eccc05b03538 server_cookie=7d6204c566c83a8 in_seq=0 out_seq=0 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.511+0000 7f778a7fc640 1 -- 172.21.15.139:0/1740489942 <== mds.0 v2:172.21.15.150:6833/1377607596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600f6666920 con 0x5600f66bb910 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.511+0000 7f778a7fc640 10 client.9279 handle_command_reply: tid=0 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.511+0000 7f77b4c15740 1 client.9279 shutdown 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.512+0000 7f77b4c15740 1 -- 172.21.15.139:0/1740489942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 msgr2=0x5600f66d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.512+0000 7f77b4c15740 1 --2- 172.21.15.139:0/1740489942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 0x5600f66d21a0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f779c087ad0 tx=0x7f779c000bc0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.512+0000 7f77b4c15740 1 -- 172.21.15.139:0/1740489942 shutdown_connections 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.512+0000 7f77b4c15740 1 --2- 172.21.15.139:0/1740489942 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x5600f66bb910 0x5600f66bbcf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.512+0000 7f77b4c15740 1 --2- 172.21.15.139:0/1740489942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ecad0 0x5600f66d21a0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.512+0000 7f77b4c15740 1 --2- 172.21.15.139:0/1740489942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66ba6c0 0x5600f66f9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.512+0000 7f778a7fc640 0 client.9279 ms_handle_reset on v2:172.21.15.150:6833/1377607596 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.512+0000 7f77b4c15740 1 --2- 172.21.15.139:0/1740489942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66cf230 0x5600f66d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.512+0000 7f77b4c15740 1 -- 172.21.15.139:0/1740489942 >> 172.21.15.139:0/1740489942 conn(0x5600f66a4f10 msgr2=0x5600f66ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.512+0000 7f77b4c15740 1 -- 172.21.15.139:0/1740489942 shutdown_connections 2024-04-25T10:02:55.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.512+0000 7f77b4c15740 1 -- 172.21.15.139:0/1740489942 wait complete. 2024-04-25T10:02:55.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.512+0000 7f77b4c15740 20 client.9279 trim_cache size 0 max 16384 2024-04-25T10:02:55.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.527+0000 7f77b4c15740 1 Processor -- start 2024-04-25T10:02:55.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.527+0000 7f77b4c15740 1 -- start start 2024-04-25T10:02:55.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77b4c15740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66ba6c0 0x5600f66f9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77b4c15740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66cf230 0x5600f66ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77b4c15740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66ecad0 0x5600f66fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77b4c15740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac1ff670 con 0x5600f66cf230 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77b4c15740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac15a670 con 0x5600f66ecad0 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77b4c15740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac1ff070 con 0x5600f66ba6c0 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66cf230 0x5600f66ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77aaffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66cf230 0x5600f66ce1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54306/0 (socket says 172.21.15.139:54306) 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77aaffd640 1 -- 172.21.15.139:0/4179869076 learned_addr learned my addr 172.21.15.139:0/4179869076 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77aaffd640 1 -- 172.21.15.139:0/4179869076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66ba6c0 msgr2=0x5600f66f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77aaffd640 1 --2- 172.21.15.139:0/4179869076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66ba6c0 0x5600f66f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77aaffd640 1 -- 172.21.15.139:0/4179869076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66ecad0 msgr2=0x5600f66fcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77aaffd640 1 --2- 172.21.15.139:0/4179869076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66ecad0 0x5600f66fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77aaffd640 1 -- 172.21.15.139:0/4179869076 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600f66cee70 con 0x5600f66cf230 2024-04-25T10:02:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.528+0000 7f77aaffd640 1 --2- 172.21.15.139:0/4179869076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66cf230 0x5600f66ce1e0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f779c057df0 tx=0x7f779c083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c62d627282a37d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.529+0000 7f7790ff9640 1 -- 172.21.15.139:0/4179869076 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c056220 con 0x5600f66cf230 2024-04-25T10:02:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.529+0000 7f7790ff9640 1 -- 172.21.15.139:0/4179869076 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c05b070 con 0x5600f66cf230 2024-04-25T10:02:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.529+0000 7f7790ff9640 1 -- 172.21.15.139:0/4179869076 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c0544d0 con 0x5600f66cf230 2024-04-25T10:02:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.529+0000 7f77b4c15740 1 -- 172.21.15.139:0/4179869076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66cf230 msgr2=0x5600f66ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.529+0000 7f77b4c15740 1 --2- 172.21.15.139:0/4179869076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66cf230 0x5600f66ce1e0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f779c057df0 tx=0x7f779c083040 comp rx=0 tx=0).stop 2024-04-25T10:02:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.529+0000 7f77b4c15740 1 -- 172.21.15.139:0/4179869076 shutdown_connections 2024-04-25T10:02:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.529+0000 7f77b4c15740 1 --2- 172.21.15.139:0/4179869076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66cf230 0x5600f66ce1e0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.529+0000 7f77b4c15740 1 --2- 172.21.15.139:0/4179869076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66ecad0 0x5600f66fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.529+0000 7f77b4c15740 1 --2- 172.21.15.139:0/4179869076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66ba6c0 0x5600f66f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.529+0000 7f77b4c15740 1 -- 172.21.15.139:0/4179869076 >> 172.21.15.139:0/4179869076 conn(0x5600f66a4f10 msgr2=0x5600f66bbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.529+0000 7f77b4c15740 1 -- 172.21.15.139:0/4179869076 shutdown_connections 2024-04-25T10:02:55.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.529+0000 7f77b4c15740 1 -- 172.21.15.139:0/4179869076 wait complete. 2024-04-25T10:02:55.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.530+0000 7f77b4c15740 1 -- start start 2024-04-25T10:02:55.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.530+0000 7f77b4c15740 1 Processor -- start 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.530+0000 7f77b4c15740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ba6c0 0x5600f669efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.530+0000 7f77b4c15740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66cf230 0x5600f66f9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.530+0000 7f77b4c15740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66ecad0 0x5600f66a4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77b4c15740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f66a5cb0 con 0x5600f66ba6c0 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77b4c15740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600f65c40c0 con 0x5600f66cf230 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77b4c15740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600f66bd120 con 0x5600f66ecad0 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77b4c15740 10 client.0 _ll_register_callbacks cb 0x5600f669fc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77ab7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ba6c0 0x5600f669efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77ab7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ba6c0 0x5600f669efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54310/0 (socket says 172.21.15.139:54310) 2024-04-25T10:02:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77ab7fe640 1 -- 172.21.15.139:0/1365167664 learned_addr learned my addr 172.21.15.139:0/1365167664 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77ab7fe640 1 -- 172.21.15.139:0/1365167664 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66ecad0 msgr2=0x5600f66a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77ab7fe640 1 --2- 172.21.15.139:0/1365167664 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66ecad0 0x5600f66a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77ab7fe640 1 -- 172.21.15.139:0/1365167664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66cf230 msgr2=0x5600f66f9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77ab7fe640 1 --2- 172.21.15.139:0/1365167664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66cf230 0x5600f66f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77ab7fe640 1 -- 172.21.15.139:0/1365167664 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600f669f4f0 con 0x5600f66ba6c0 2024-04-25T10:02:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77ab7fe640 1 --2- 172.21.15.139:0/1365167664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ba6c0 0x5600f669efb0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009870 tx=0x7f77a00095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea0fd8b3c4a1b15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f778a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:02:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f778a7fc640 1 -- 172.21.15.139:0/1365167664 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0051070 con 0x5600f66ba6c0 2024-04-25T10:02:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f778a7fc640 1 -- 172.21.15.139:0/1365167664 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f77a0032cd0 con 0x5600f66ba6c0 2024-04-25T10:02:55.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77b4c15740 1 -- 172.21.15.139:0/1365167664 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f779c05b070 con 0x5600f66ba6c0 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f778a7fc640 1 -- 172.21.15.139:0/1365167664 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77a005b050 con 0x5600f66ba6c0 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77b4c15740 10 client.9285 fetch_fsmap learned FSMap version 83 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77b4c15740 1 -- 172.21.15.139:0/1365167664 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x5600f66e03f0 con 0x5600f66ba6c0 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f778a7fc640 1 -- 172.21.15.139:0/1365167664 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c05b070 con 0x5600f66ba6c0 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f778a7fc640 1 -- 172.21.15.139:0/1365167664 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f77a005e020 con 0x5600f66ba6c0 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.531+0000 7f77b4c15740 10 client.9285 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.532+0000 7f77b4c15740 10 client.9285 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.532+0000 7f77b4c15740 20 client.9285 populate_metadata read hostname 'smithi139' 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.532+0000 7f77b4c15740 1 --2- 172.21.15.139:0/1365167664 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x5600f6737220 0x5600f674e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.532+0000 7f77b4c15740 4 client.9285 mds_command: new command op to 9132 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.532+0000 7f77b4c15740 1 -- 172.21.15.139:0/1365167664 --> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5600f66e5290 con 0x5600f6737220 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.532+0000 7f77aaffd640 1 --2- 172.21.15.139:0/1365167664 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x5600f6737220 0x5600f674e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.532+0000 7f778a7fc640 10 client.9285 ms_handle_connect on v2:172.21.15.150:6833/1377607596 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.532+0000 7f77aaffd640 1 --2- 172.21.15.139:0/1365167664 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x5600f6737220 0x5600f674e300 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a212b8586dd4823 server_cookie=ced01dc24766865a in_seq=0 out_seq=0 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.532+0000 7f778a7fc640 1 -- 172.21.15.139:0/1365167664 <== mds.0 v2:172.21.15.150:6833/1377607596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5600f66e5290 con 0x5600f6737220 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.532+0000 7f778a7fc640 10 client.9285 handle_command_reply: tid=0 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.532+0000 7f77b4c15740 1 client.9285 shutdown 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b4c15740 1 -- 172.21.15.139:0/1365167664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ba6c0 msgr2=0x5600f669efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b4c15740 1 --2- 172.21.15.139:0/1365167664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ba6c0 0x5600f669efb0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009870 tx=0x7f77a00095a0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b4c15740 1 -- 172.21.15.139:0/1365167664 shutdown_connections 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b4c15740 1 --2- 172.21.15.139:0/1365167664 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x5600f6737220 0x5600f674e300 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b4c15740 1 --2- 172.21.15.139:0/1365167664 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5600f66ecad0 0x5600f66a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f778a7fc640 0 client.9285 ms_handle_reset on v2:172.21.15.150:6833/1377607596 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b4c15740 1 --2- 172.21.15.139:0/1365167664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600f66cf230 0x5600f66f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b4c15740 1 --2- 172.21.15.139:0/1365167664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5600f66ba6c0 0x5600f669efb0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b4c15740 1 -- 172.21.15.139:0/1365167664 >> 172.21.15.139:0/1365167664 conn(0x5600f66a4f10 msgr2=0x5600f66f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b4c15740 1 -- 172.21.15.139:0/1365167664 shutdown_connections 2024-04-25T10:02:55.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b4c15740 1 -- 172.21.15.139:0/1365167664 wait complete. 2024-04-25T10:02:55.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b4c15740 20 client.9285 trim_cache size 0 max 16384 2024-04-25T10:02:55.545 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:02:55.545 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T10:02:55.545 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T10:02:55.545 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 9132, 2024-04-25T10:02:55.545 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 4.5488571090000001, 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 83, 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 120, 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 120, 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 9.6181702869999999 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b1e38640 1 -- 172.21.15.139:0/1891836456 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f779806b4d0 msgr2=0x7f779806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b1e38640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f779806b4d0 0x7f779806d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f77a0034000 tx=0x7f77a002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b1e38640 1 -- 172.21.15.139:0/1891836456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ac0cee50 msgr2=0x7f77ac1f2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b1e38640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ac0cee50 0x7f77ac1f2e60 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c3c0 tx=0x7f779c00c880 comp rx=0 tx=0).stop 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b1e38640 1 -- 172.21.15.139:0/1891836456 shutdown_connections 2024-04-25T10:02:55.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b1e38640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77ac0cee50 0x7f77ac1f2e60 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b1e38640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f779806b4d0 0x7f779806d990 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b1e38640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77ac15dbf0 0x7f77ac15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b1e38640 1 --2- 172.21.15.139:0/1891836456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77ac0b9370 0x7f77ac0aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b1e38640 1 -- 172.21.15.139:0/1891836456 >> 172.21.15.139:0/1891836456 conn(0x7f77ac0b5480 msgr2=0x7f77ac0cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b1e38640 1 -- 172.21.15.139:0/1891836456 shutdown_connections 2024-04-25T10:02:55.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.533+0000 7f77b1e38640 1 -- 172.21.15.139:0/1891836456 wait complete. 2024-04-25T10:02:55.549 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:55.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.641+0000 7feb6355c640 1 Processor -- start 2024-04-25T10:02:55.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.641+0000 7feb6355c640 1 -- start start 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb6355c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feb5c06ab60 0x7feb5c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb6355c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feb5c0b36f0 0x7feb5c0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb6355c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5c0d0400 con 0x7feb5c160e90 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb6355c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5c0d14a0 con 0x7feb5c06ab60 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb6355c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5c0d06e0 con 0x7feb5c0b36f0 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb60af7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feb5c06ab60 0x7feb5c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb612f8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7feb5c160e90 0x7feb5c0afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35576/0 (socket says 172.21.15.139:35576) 2024-04-25T10:02:55.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb612f8640 1 -- 172.21.15.139:0/855750670 learned_addr learned my addr 172.21.15.139:0/855750670 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:55.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb61af9640 1 --2- 172.21.15.139:0/855750670 >> v2:172.21.15.150:3300/0 conn(0x7feb5c0b36f0 0x7feb5c0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb60af7640 1 -- 172.21.15.139:0/855750670 >> v2:172.21.15.150:3300/0 conn(0x7feb5c0b36f0 msgr2=0x7feb5c0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb60af7640 1 --2- 172.21.15.139:0/855750670 >> v2:172.21.15.150:3300/0 conn(0x7feb5c0b36f0 0x7feb5c0b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb60af7640 1 -- 172.21.15.139:0/855750670 >> v1:172.21.15.139:6789/0 conn(0x7feb5c160e90 legacy=0x7feb5c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb53fff640 1 -- 172.21.15.139:0/855750670 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2996912610 0 0) 0x7feb5c0d0400 con 0x7feb5c160e90 2024-04-25T10:02:55.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.642+0000 7feb60af7640 1 -- 172.21.15.139:0/855750670 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c0d09c0 con 0x7feb5c06ab60 2024-04-25T10:02:55.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.643+0000 7feb60af7640 1 --2- 172.21.15.139:0/855750670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5c06ab60 0x7feb5c06aff0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0097a0 tx=0x7feb4c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e358f61a3daa2cda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.643+0000 7feb53fff640 1 -- 172.21.15.139:0/855750670 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c037070 con 0x7feb5c06ab60 2024-04-25T10:02:55.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.643+0000 7feb53fff640 1 -- 172.21.15.139:0/855750670 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c02ae50 con 0x7feb5c06ab60 2024-04-25T10:02:55.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.643+0000 7feb53fff640 1 -- 172.21.15.139:0/855750670 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c032660 con 0x7feb5c06ab60 2024-04-25T10:02:55.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.644+0000 7feb6355c640 1 -- 172.21.15.139:0/855750670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5c06ab60 msgr2=0x7feb5c06aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.644+0000 7feb6355c640 1 --2- 172.21.15.139:0/855750670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5c06ab60 0x7feb5c06aff0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0097a0 tx=0x7feb4c02a140 comp rx=0 tx=0).stop 2024-04-25T10:02:55.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.644+0000 7feb6355c640 1 -- 172.21.15.139:0/855750670 shutdown_connections 2024-04-25T10:02:55.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.644+0000 7feb6355c640 1 --2- 172.21.15.139:0/855750670 >> v2:172.21.15.150:3300/0 conn(0x7feb5c0b36f0 0x7feb5c0b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.644+0000 7feb6355c640 1 --2- 172.21.15.139:0/855750670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5c06ab60 0x7feb5c06aff0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.644+0000 7feb6355c640 1 -- 172.21.15.139:0/855750670 >> 172.21.15.139:0/855750670 conn(0x7feb5c0b23f0 msgr2=0x7feb5c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.644+0000 7feb6355c640 1 -- 172.21.15.139:0/855750670 shutdown_connections 2024-04-25T10:02:55.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.644+0000 7feb6355c640 1 -- 172.21.15.139:0/855750670 wait complete. 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.645+0000 7feb6355c640 1 Processor -- start 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.645+0000 7feb6355c640 1 -- start start 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb6355c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb5c06ab60 0x7feb5c15a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb6355c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5c0b36f0 0x7feb5c157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb6355c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb5c160e90 0x7feb5c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb6355c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c161270 con 0x7feb5c0b36f0 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb6355c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c0d0f80 con 0x7feb5c06ab60 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb6355c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1593e0 con 0x7feb5c160e90 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb60af7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5c0b36f0 0x7feb5c157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb60af7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5c0b36f0 0x7feb5c157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54328/0 (socket says 172.21.15.139:54328) 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb60af7640 1 -- 172.21.15.139:0/3522178596 learned_addr learned my addr 172.21.15.139:0/3522178596 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb61af9640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb5c160e90 0x7feb5c0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb612f8640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb5c06ab60 0x7feb5c15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb60af7640 1 -- 172.21.15.139:0/3522178596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb5c160e90 msgr2=0x7feb5c0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb60af7640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb5c160e90 0x7feb5c0cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb60af7640 1 -- 172.21.15.139:0/3522178596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb5c06ab60 msgr2=0x7feb5c15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb60af7640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb5c06ab60 0x7feb5c15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb60af7640 1 -- 172.21.15.139:0/3522178596 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c159660 con 0x7feb5c0b36f0 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb612f8640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb5c06ab60 0x7feb5c15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb61af9640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb5c160e90 0x7feb5c0cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb60af7640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5c0b36f0 0x7feb5c157980 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0097a0 tx=0x7feb4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb51ffb640 1 -- 172.21.15.139:0/3522178596 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c037040 con 0x7feb5c0b36f0 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb51ffb640 1 -- 172.21.15.139:0/3522178596 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c004250 con 0x7feb5c0b36f0 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.646+0000 7feb6355c640 1 -- 172.21.15.139:0/3522178596 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5c159940 con 0x7feb5c0b36f0 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.647+0000 7feb51ffb640 1 -- 172.21.15.139:0/3522178596 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c003610 con 0x7feb5c0b36f0 2024-04-25T10:02:55.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.647+0000 7feb6355c640 1 -- 172.21.15.139:0/3522178596 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c159f00 con 0x7feb5c0b36f0 2024-04-25T10:02:55.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.648+0000 7feb6355c640 1 -- 172.21.15.139:0/3522178596 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5c074520 con 0x7feb5c0b36f0 2024-04-25T10:02:55.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.648+0000 7feb51ffb640 1 -- 172.21.15.139:0/3522178596 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb4c0037b0 con 0x7feb5c0b36f0 2024-04-25T10:02:55.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.648+0000 7feb51ffb640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb4006b4d0 0x7feb4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.649+0000 7feb51ffb640 1 -- 172.21.15.139:0/3522178596 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feb4c0ab1b0 con 0x7feb5c0b36f0 2024-04-25T10:02:55.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.649+0000 7feb612f8640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb4006b4d0 0x7feb4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.649+0000 7feb612f8640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb4006b4d0 0x7feb4006d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7feb580012e0 tx=0x7feb58005bb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.653+0000 7feb51ffb640 1 -- 172.21.15.139:0/3522178596 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb4c07a050 con 0x7feb5c0b36f0 2024-04-25T10:02:55.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.862+0000 7feb6355c640 1 -- 172.21.15.139:0/3522178596 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feb5c073c20 con 0x7feb5c0b36f0 2024-04-25T10:02:55.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.862+0000 7feb51ffb640 1 -- 172.21.15.139:0/3522178596 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) v1 ==== 53+0+73 (secure 0 0 0) 0x7feb4c079950 con 0x7feb5c0b36f0 2024-04-25T10:02:55.867 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:02:55.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.864+0000 7feb6355c640 1 -- 172.21.15.139:0/3522178596 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb4006b4d0 msgr2=0x7feb4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.864+0000 7feb6355c640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb4006b4d0 0x7feb4006d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7feb580012e0 tx=0x7feb58005bb0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.864+0000 7feb6355c640 1 -- 172.21.15.139:0/3522178596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5c0b36f0 msgr2=0x7feb5c157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.864+0000 7feb6355c640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5c0b36f0 0x7feb5c157980 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0097a0 tx=0x7feb4c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.864+0000 7feb6355c640 1 -- 172.21.15.139:0/3522178596 shutdown_connections 2024-04-25T10:02:55.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.864+0000 7feb6355c640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5c0b36f0 0x7feb5c157980 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.864+0000 7feb6355c640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feb4006b4d0 0x7feb4006d990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.864+0000 7feb6355c640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb5c160e90 0x7feb5c0cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.864+0000 7feb6355c640 1 --2- 172.21.15.139:0/3522178596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb5c06ab60 0x7feb5c15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.864+0000 7feb6355c640 1 -- 172.21.15.139:0/3522178596 >> 172.21.15.139:0/3522178596 conn(0x7feb5c0b23f0 msgr2=0x7feb5c06af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.864+0000 7feb6355c640 1 -- 172.21.15.139:0/3522178596 shutdown_connections 2024-04-25T10:02:55.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:55.864+0000 7feb6355c640 1 -- 172.21.15.139:0/3522178596 wait complete. 2024-04-25T10:02:55.880 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:02:55.880 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:55.880 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T10:02:55.880 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:02:55.880 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:55.880 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:02:55.880 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:02:55.880 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:55.880 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:55.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:55.881 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T10:02:55.896 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:55.896 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:55.896 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:55.896 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:55.896 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:55.896 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:55.896 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:55.896 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42123sec preferred_lft 42123sec 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:55.898 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:55.898 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T10:02:55.898 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:55.898 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:55.898 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:55.898 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T10:02:55.898 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:55.898 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:55.898 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:55.898 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T10:02:55.898 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:55.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:55.899 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:55.959 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:55.960 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T10:02:56.016 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T10:02:56.016 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:56.016 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:56.016 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:56.016 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:56.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:56.017 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T10:02:56.017 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T10:02:56.017 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:56.017 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:56.017 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:56.017 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T10:02:56.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:56.100 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:02:56.155 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:56.156 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T10:02:56.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:56.218 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:56.273 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:56.273 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:56.273 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:56.327 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:02:56.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:56.327 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:02:56.408 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:02:56.408 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:02:56.429 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:02:56.429 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:02:56.429 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T10:02:56.429 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T10:02:56.429 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T10:02:56.430 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:02:56.430 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:02:56.430 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:02:56.430 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:02:56.430 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:02:56.431 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:02:56.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:56.431 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:56.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:56.496 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:56.559 INFO:teuthology.orchestra.run.smithi139.stdout:client9297 2024-04-25T10:02:56.559 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:02:56.559 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:56.616 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:02:56.616 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:02:56.616 DEBUG:teuthology.orchestra.run.smithi139:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:02:56.616 DEBUG:teuthology.orchestra.run.smithi139:> parent = os.path.join(mount_point, "parent") 2024-04-25T10:02:56.616 DEBUG:teuthology.orchestra.run.smithi139:> os.mkdir(parent) 2024-04-25T10:02:56.616 DEBUG:teuthology.orchestra.run.smithi139:> flushed = os.path.join(parent, "flushed") 2024-04-25T10:02:56.616 DEBUG:teuthology.orchestra.run.smithi139:> os.mkdir(flushed) 2024-04-25T10:02:56.616 DEBUG:teuthology.orchestra.run.smithi139:> for f in ["alpha", "bravo", "charlie"]: 2024-04-25T10:02:56.616 DEBUG:teuthology.orchestra.run.smithi139:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-25T10:02:56.616 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:02:56.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:56.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:56.737 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:02:56.798 INFO:teuthology.orchestra.run.smithi139.stdout:./ 2024-04-25T10:02:56.798 INFO:teuthology.orchestra.run.smithi139.stdout:./parent 2024-04-25T10:02:56.798 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/flushed 2024-04-25T10:02:56.798 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/flushed/charlie 2024-04-25T10:02:56.798 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/flushed/alpha 2024-04-25T10:02:56.798 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/flushed/bravo 2024-04-25T10:02:56.799 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:02:56.799 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:02:56.800 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:02:56.800 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:02:56.800 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:02:56.800 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:02:56.803 INFO:tasks.ceph.mds.d.smithi150.stderr:2024-04-25T10:02:56.799+0000 7f1fc0ae9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:02:56.904 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T10:02:56.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:56.914 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:02:56.914 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:02:56.914 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:02:56.915 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:02:56.915 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:02:56.915 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:02:57.019 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627776 2024-04-25T10:02:57.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:57.028 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:02:57.028 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:02:57.028 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:02:57.028 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:02:57.028 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:02:57.028 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:02:57.133 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627777 2024-04-25T10:02:57.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:57.142 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:02:57.142 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:02:57.142 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:02:57.142 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:02:57.143 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:02:57.143 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:02:57.246 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627780 2024-04-25T10:02:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:57.256 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:02:57.256 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:02:57.256 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:02:57.256 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:02:57.256 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:02:57.256 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:02:57.361 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627778 2024-04-25T10:02:57.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:57.370 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:02:57.370 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:02:57.370 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:02:57.370 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:02:57.370 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T10:02:57.370 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:02:57.475 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627779 2024-04-25T10:02:57.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:57.484 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:02:57.484 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:57.540 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:02:57.540 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:57.596 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:57.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.728+0000 7f56a6df4640 1 Processor -- start 2024-04-25T10:02:57.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.728+0000 7f56a6df4640 1 -- start start 2024-04-25T10:02:57.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.729+0000 7f56a6df4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f56a0160e90 0x7f56a00afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.729+0000 7f56a6df4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56a006aff0 0x7f56a0070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.729+0000 7f56a6df4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a006b480 con 0x7f56a006ab60 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.729+0000 7f56a6df4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a00d14a0 con 0x7f56a0160e90 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.729+0000 7f56a6df4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a00d0400 con 0x7f56a006aff0 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.729+0000 7f56a5391640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56a006aff0 0x7f56a0070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.729+0000 7f56a4b90640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f56a0160e90 0x7f56a00afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.729+0000 7f569ffff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f56a006ab60 0x7f56a006fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:38916/0 (socket says 172.21.15.139:38916) 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.729+0000 7f56a5391640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56a006aff0 0x7f56a0070180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:35060/0 (socket says 172.21.15.139:35060) 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.729+0000 7f56a4b90640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f56a0160e90 0x7f56a00afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:32844/0 (socket says 172.21.15.139:32844) 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.729+0000 7f569ffff640 1 -- 172.21.15.139:0/2941102715 learned_addr learned my addr 172.21.15.139:0/2941102715 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.730+0000 7f56a5391640 1 -- 172.21.15.139:0/2941102715 >> v2:172.21.15.150:3300/0 conn(0x7f56a0160e90 msgr2=0x7f56a00afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.730+0000 7f56a5391640 1 --2- 172.21.15.139:0/2941102715 >> v2:172.21.15.150:3300/0 conn(0x7f56a0160e90 0x7f56a00afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.730+0000 7f56a5391640 1 -- 172.21.15.139:0/2941102715 >> v1:172.21.15.150:6789/0 conn(0x7f56a006ab60 legacy=0x7f56a006fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.730+0000 7f56a5391640 1 -- 172.21.15.139:0/2941102715 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a00d06e0 con 0x7f56a006aff0 2024-04-25T10:02:57.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.730+0000 7f56a4b90640 1 --2- 172.21.15.139:0/2941102715 >> v2:172.21.15.150:3300/0 conn(0x7f56a0160e90 0x7f56a00afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:57.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.731+0000 7f56a5391640 1 --2- 172.21.15.139:0/2941102715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56a006aff0 0x7f56a0070180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5698009dc0 tx=0x7f569802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b0e787339d34fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.731+0000 7f569f7fe640 1 -- 172.21.15.139:0/2941102715 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569802ae60 con 0x7f56a006aff0 2024-04-25T10:02:57.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.731+0000 7f569f7fe640 1 -- 172.21.15.139:0/2941102715 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5698007740 con 0x7f56a006aff0 2024-04-25T10:02:57.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.731+0000 7f569f7fe640 1 -- 172.21.15.139:0/2941102715 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698007b10 con 0x7f56a006aff0 2024-04-25T10:02:57.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.731+0000 7f56a6df4640 1 -- 172.21.15.139:0/2941102715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56a006aff0 msgr2=0x7f56a0070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.731+0000 7f56a6df4640 1 --2- 172.21.15.139:0/2941102715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56a006aff0 0x7f56a0070180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5698009dc0 tx=0x7f569802a680 comp rx=0 tx=0).stop 2024-04-25T10:02:57.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.731+0000 7f56a6df4640 1 -- 172.21.15.139:0/2941102715 shutdown_connections 2024-04-25T10:02:57.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.731+0000 7f56a6df4640 1 --2- 172.21.15.139:0/2941102715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56a006aff0 0x7f56a0070180 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.731+0000 7f56a6df4640 1 --2- 172.21.15.139:0/2941102715 >> v2:172.21.15.150:3300/0 conn(0x7f56a0160e90 0x7f56a00afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.731+0000 7f56a6df4640 1 -- 172.21.15.139:0/2941102715 >> 172.21.15.139:0/2941102715 conn(0x7f56a00b23f0 msgr2=0x7f56a00b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.731+0000 7f56a6df4640 1 -- 172.21.15.139:0/2941102715 shutdown_connections 2024-04-25T10:02:57.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.731+0000 7f56a6df4640 1 -- 172.21.15.139:0/2941102715 wait complete. 2024-04-25T10:02:57.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.732+0000 7f56a6df4640 1 Processor -- start 2024-04-25T10:02:57.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.732+0000 7f56a6df4640 1 -- start start 2024-04-25T10:02:57.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.733+0000 7f56a6df4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56a006ab60 0x7f56a014d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.733+0000 7f56a6df4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56a006aff0 0x7f56a0158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.733+0000 7f56a6df4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56a0160e90 0x7f56a0158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.733+0000 7f56a6df4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a00d0f80 con 0x7f56a0160e90 2024-04-25T10:02:57.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.733+0000 7f56a6df4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a00d0ca0 con 0x7f56a006ab60 2024-04-25T10:02:57.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.733+0000 7f56a6df4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a00b0000 con 0x7f56a006aff0 2024-04-25T10:02:57.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.733+0000 7f56a5391640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56a0160e90 0x7f56a0158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.733+0000 7f56a5391640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56a0160e90 0x7f56a0158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45712/0 (socket says 172.21.15.139:45712) 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.733+0000 7f56a5391640 1 -- 172.21.15.139:0/1193014916 learned_addr learned my addr 172.21.15.139:0/1193014916 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.733+0000 7f56a4b90640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56a006ab60 0x7f56a014d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.734+0000 7f569ffff640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56a006aff0 0x7f56a0158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.734+0000 7f56a5391640 1 -- 172.21.15.139:0/1193014916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56a006aff0 msgr2=0x7f56a0158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.734+0000 7f56a5391640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56a006aff0 0x7f56a0158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.734+0000 7f56a5391640 1 -- 172.21.15.139:0/1193014916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56a006ab60 msgr2=0x7f56a014d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.734+0000 7f56a5391640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56a006ab60 0x7f56a014d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.734+0000 7f56a5391640 1 -- 172.21.15.139:0/1193014916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a014d670 con 0x7f56a0160e90 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.734+0000 7f56a5391640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56a0160e90 0x7f56a0158640 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f5698009ee0 tx=0x7f5698004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.734+0000 7f569d7fa640 1 -- 172.21.15.139:0/1193014916 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698040070 con 0x7f56a0160e90 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.734+0000 7f569d7fa640 1 -- 172.21.15.139:0/1193014916 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56980319c0 con 0x7f56a0160e90 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.734+0000 7f56a6df4640 1 -- 172.21.15.139:0/1193014916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a00cb610 con 0x7f56a0160e90 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.734+0000 7f569d7fa640 1 -- 172.21.15.139:0/1193014916 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698031b90 con 0x7f56a0160e90 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.734+0000 7f56a6df4640 1 -- 172.21.15.139:0/1193014916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a00cbbd0 con 0x7f56a0160e90 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.735+0000 7f56a6df4640 1 -- 172.21.15.139:0/1193014916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a0074520 con 0x7f56a0160e90 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.736+0000 7f569d7fa640 1 -- 172.21.15.139:0/1193014916 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5698039040 con 0x7f56a0160e90 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.736+0000 7f569d7fa640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f568c06b4d0 0x7f568c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.736+0000 7f569d7fa640 1 -- 172.21.15.139:0/1193014916 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56980ab860 con 0x7f56a0160e90 2024-04-25T10:02:57.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.736+0000 7f56a4b90640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f568c06b4d0 0x7f568c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.737+0000 7f56a4b90640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f568c06b4d0 0x7f568c06d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5690009560 tx=0x7f5690009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.741+0000 7f569d7fa640 1 -- 172.21.15.139:0/1193014916 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56a014d670 con 0x7f56a0160e90 2024-04-25T10:02:57.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.914+0000 7f56a6df4640 1 -- 172.21.15.139:0/1193014916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56a00728b0 con 0x7f56a0160e90 2024-04-25T10:02:57.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.915+0000 7f569d7fa640 1 -- 172.21.15.139:0/1193014916 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+17000 (secure 0 0 0) 0x7f5698079f90 con 0x7f56a0160e90 2024-04-25T10:02:57.920 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:02:57.920 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":120,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:02:51.983848+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:48.574891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:49.293101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:57.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.917+0000 7f56a6df4640 1 -- 172.21.15.139:0/1193014916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f568c06b4d0 msgr2=0x7f568c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.917+0000 7f56a6df4640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f568c06b4d0 0x7f568c06d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5690009560 tx=0x7f5690009190 comp rx=0 tx=0).stop 2024-04-25T10:02:57.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.917+0000 7f56a6df4640 1 -- 172.21.15.139:0/1193014916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56a0160e90 msgr2=0x7f56a0158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.917+0000 7f56a6df4640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56a0160e90 0x7f56a0158640 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f5698009ee0 tx=0x7f5698004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:57.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.917+0000 7f56a6df4640 1 -- 172.21.15.139:0/1193014916 shutdown_connections 2024-04-25T10:02:57.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.917+0000 7f56a6df4640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56a0160e90 0x7f56a0158640 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.917+0000 7f56a6df4640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f568c06b4d0 0x7f568c06d990 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.917+0000 7f56a6df4640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56a006aff0 0x7f56a0158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.917+0000 7f56a6df4640 1 --2- 172.21.15.139:0/1193014916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f56a006ab60 0x7f56a014d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.917+0000 7f56a6df4640 1 -- 172.21.15.139:0/1193014916 >> 172.21.15.139:0/1193014916 conn(0x7f56a00b23f0 msgr2=0x7f56a00b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.917+0000 7f56a6df4640 1 -- 172.21.15.139:0/1193014916 shutdown_connections 2024-04-25T10:02:57.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:57.917+0000 7f56a6df4640 1 -- 172.21.15.139:0/1193014916 wait complete. 2024-04-25T10:02:57.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:48.574891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:49.293101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:57.933 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:58.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.031+0000 7fab2d05d640 1 Processor -- start 2024-04-25T10:02:58.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.031+0000 7fab2d05d640 1 -- start start 2024-04-25T10:02:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.032+0000 7fab2d05d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fab280719d0 0x7fab280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.032+0000 7fab2d05d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab280b1100 con 0x7fab280712f0 2024-04-25T10:02:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.032+0000 7fab2d05d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab280ce2c0 con 0x7fab28145250 2024-04-25T10:02:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.032+0000 7fab2d05d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab280ceed0 con 0x7fab280719d0 2024-04-25T10:02:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.032+0000 7fab2659c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fab280712f0 0x7fab280b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48654/0 (socket says 172.21.15.139:48654) 2024-04-25T10:02:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.032+0000 7fab2659c640 1 -- 172.21.15.139:0/3810098127 learned_addr learned my addr 172.21.15.139:0/3810098127 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.032+0000 7fab2759e640 1 --2- 172.21.15.139:0/3810098127 >> v2:172.21.15.139:3300/0 conn(0x7fab280719d0 0x7fab280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.033+0000 7fab25d9b640 1 -- 172.21.15.139:0/3810098127 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328805753 0 0) 0x7fab280b1100 con 0x7fab280712f0 2024-04-25T10:02:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.033+0000 7fab25d9b640 1 -- 172.21.15.139:0/3810098127 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab10003520 con 0x7fab280712f0 2024-04-25T10:02:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.033+0000 7fab2759e640 1 -- 172.21.15.139:0/3810098127 >> v1:172.21.15.150:6789/0 conn(0x7fab28145250 legacy=0x7fab28070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.033+0000 7fab2759e640 1 -- 172.21.15.139:0/3810098127 >> v1:172.21.15.139:6789/0 conn(0x7fab280712f0 legacy=0x7fab280b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.033+0000 7fab2759e640 1 -- 172.21.15.139:0/3810098127 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab280ce5a0 con 0x7fab280719d0 2024-04-25T10:02:58.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.033+0000 7fab2759e640 1 --2- 172.21.15.139:0/3810098127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab280719d0 0x7fab280b4d90 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fab20009d90 tx=0x7fab2002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b8b51e981548f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.033+0000 7fab25d9b640 1 -- 172.21.15.139:0/3810098127 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20031600 con 0x7fab280719d0 2024-04-25T10:02:58.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.033+0000 7fab25d9b640 1 -- 172.21.15.139:0/3810098127 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab200317a0 con 0x7fab280719d0 2024-04-25T10:02:58.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.034+0000 7fab25d9b640 1 -- 172.21.15.139:0/3810098127 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20031b70 con 0x7fab280719d0 2024-04-25T10:02:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.034+0000 7fab2d05d640 1 -- 172.21.15.139:0/3810098127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab280719d0 msgr2=0x7fab280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.034+0000 7fab2d05d640 1 --2- 172.21.15.139:0/3810098127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab280719d0 0x7fab280b4d90 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fab20009d90 tx=0x7fab2002a610 comp rx=0 tx=0).stop 2024-04-25T10:02:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.034+0000 7fab2d05d640 1 -- 172.21.15.139:0/3810098127 shutdown_connections 2024-04-25T10:02:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.034+0000 7fab2d05d640 1 --2- 172.21.15.139:0/3810098127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab280719d0 0x7fab280b4d90 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.034+0000 7fab2d05d640 1 -- 172.21.15.139:0/3810098127 >> 172.21.15.139:0/3810098127 conn(0x7fab2806f460 msgr2=0x7fab2806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.034+0000 7fab2d05d640 1 -- 172.21.15.139:0/3810098127 shutdown_connections 2024-04-25T10:02:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.034+0000 7fab2d05d640 1 -- 172.21.15.139:0/3810098127 wait complete. 2024-04-25T10:02:58.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.036+0000 7fab2d05d640 1 Processor -- start 2024-04-25T10:02:58.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.036+0000 7fab2d05d640 1 -- start start 2024-04-25T10:02:58.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.036+0000 7fab2d05d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab280712f0 0x7fab2815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.036+0000 7fab2d05d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab280719d0 0x7fab2815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.036+0000 7fab2d05d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28145250 0x7fab281f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.036+0000 7fab2d05d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab28145630 con 0x7fab280719d0 2024-04-25T10:02:58.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.036+0000 7fab2d05d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab280ceb60 con 0x7fab28145250 2024-04-25T10:02:58.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.036+0000 7fab2d05d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab280d11a0 con 0x7fab280712f0 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.036+0000 7fab2659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab280719d0 0x7fab2815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.036+0000 7fab2659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab280719d0 0x7fab2815b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45716/0 (socket says 172.21.15.139:45716) 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.036+0000 7fab2659c640 1 -- 172.21.15.139:0/3392404974 learned_addr learned my addr 172.21.15.139:0/3392404974 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab2759e640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28145250 0x7fab281f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab2659c640 1 -- 172.21.15.139:0/3392404974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab280712f0 msgr2=0x7fab2815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab2659c640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab280712f0 0x7fab2815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab2659c640 1 -- 172.21.15.139:0/3392404974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28145250 msgr2=0x7fab281f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab2659c640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28145250 0x7fab281f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab2659c640 1 -- 172.21.15.139:0/3392404974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab281f2970 con 0x7fab280719d0 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab2759e640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28145250 0x7fab281f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab2659c640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab280719d0 0x7fab2815b3a0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c460 tx=0x7fab1c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab0f7fe640 1 -- 172.21.15.139:0/3392404974 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c017070 con 0x7fab280719d0 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab0f7fe640 1 -- 172.21.15.139:0/3392404974 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c005ce0 con 0x7fab280719d0 2024-04-25T10:02:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab2d05d640 1 -- 172.21.15.139:0/3392404974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2815a900 con 0x7fab280719d0 2024-04-25T10:02:58.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab0f7fe640 1 -- 172.21.15.139:0/3392404974 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c007290 con 0x7fab280719d0 2024-04-25T10:02:58.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.037+0000 7fab2d05d640 1 -- 172.21.15.139:0/3392404974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab281f1950 con 0x7fab280719d0 2024-04-25T10:02:58.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.038+0000 7fab2d05d640 1 -- 172.21.15.139:0/3392404974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab28074520 con 0x7fab280719d0 2024-04-25T10:02:58.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.038+0000 7fab0f7fe640 1 -- 172.21.15.139:0/3392404974 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab1c01a070 con 0x7fab280719d0 2024-04-25T10:02:58.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.039+0000 7fab0f7fe640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fab1406b4d0 0x7fab1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.039+0000 7fab0f7fe640 1 -- 172.21.15.139:0/3392404974 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab1c089740 con 0x7fab280719d0 2024-04-25T10:02:58.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.039+0000 7fab26d9d640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fab1406b4d0 0x7fab1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.039+0000 7fab26d9d640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fab1406b4d0 0x7fab1406d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fab18006fd0 tx=0x7fab18006bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.043+0000 7fab0f7fe640 1 -- 172.21.15.139:0/3392404974 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab1c058430 con 0x7fab280719d0 2024-04-25T10:02:58.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.212+0000 7fab2d05d640 1 -- 172.21.15.139:0/3392404974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab280701f0 con 0x7fab280719d0 2024-04-25T10:02:58.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.213+0000 7fab0f7fe640 1 -- 172.21.15.139:0/3392404974 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab1c057d30 con 0x7fab280719d0 2024-04-25T10:02:58.217 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:02:58.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.214+0000 7fab2d05d640 1 -- 172.21.15.139:0/3392404974 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fab1406b4d0 msgr2=0x7fab1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.214+0000 7fab2d05d640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fab1406b4d0 0x7fab1406d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fab18006fd0 tx=0x7fab18006bc0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.214+0000 7fab2d05d640 1 -- 172.21.15.139:0/3392404974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab280719d0 msgr2=0x7fab2815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.214+0000 7fab2d05d640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab280719d0 0x7fab2815b3a0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c460 tx=0x7fab1c00c920 comp rx=0 tx=0).stop 2024-04-25T10:02:58.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.214+0000 7fab2d05d640 1 -- 172.21.15.139:0/3392404974 shutdown_connections 2024-04-25T10:02:58.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.214+0000 7fab2d05d640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab280719d0 0x7fab2815b3a0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.214+0000 7fab2d05d640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fab1406b4d0 0x7fab1406d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.214+0000 7fab2d05d640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab28145250 0x7fab281f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.214+0000 7fab2d05d640 1 --2- 172.21.15.139:0/3392404974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab280712f0 0x7fab2815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.214+0000 7fab2d05d640 1 -- 172.21.15.139:0/3392404974 >> 172.21.15.139:0/3392404974 conn(0x7fab2806f460 msgr2=0x7fab280b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.215+0000 7fab2d05d640 1 -- 172.21.15.139:0/3392404974 shutdown_connections 2024-04-25T10:02:58.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.215+0000 7fab2d05d640 1 -- 172.21.15.139:0/3392404974 wait complete. 2024-04-25T10:02:58.230 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:58.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.326+0000 7f319c989640 1 Processor -- start 2024-04-25T10:02:58.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.326+0000 7f319c989640 1 -- start start 2024-04-25T10:02:58.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f319c989640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198145230 0x7f319815be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f319c989640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f319815c3d0 0x7f31980711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f319c989640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3198071700 0x7f319806d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f319c989640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3198071b50 con 0x7f319815c3d0 2024-04-25T10:02:58.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f319c989640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319815c960 con 0x7f3198071700 2024-04-25T10:02:58.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f319c989640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319815cc40 con 0x7f3198145230 2024-04-25T10:02:58.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f318ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f319815c3d0 0x7f31980711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f318ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f319815c3d0 0x7f31980711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45722/0 (socket says 172.21.15.139:45722) 2024-04-25T10:02:58.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f318ffff640 1 -- 172.21.15.139:0/624800938 learned_addr learned my addr 172.21.15.139:0/624800938 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:58.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f319659c640 1 --2- 172.21.15.139:0/624800938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198145230 0x7f319815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f318ffff640 1 -- 172.21.15.139:0/624800938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198145230 msgr2=0x7f319815be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f318ffff640 1 --2- 172.21.15.139:0/624800938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198145230 0x7f319815be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f3196d9d640 1 --2- 172.21.15.139:0/624800938 >> v2:172.21.15.150:3300/0 conn(0x7f3198071700 0x7f319806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f318ffff640 1 -- 172.21.15.139:0/624800938 >> v2:172.21.15.150:3300/0 conn(0x7f3198071700 msgr2=0x7f319806d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f318ffff640 1 --2- 172.21.15.139:0/624800938 >> v2:172.21.15.150:3300/0 conn(0x7f3198071700 0x7f319806d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f318ffff640 1 -- 172.21.15.139:0/624800938 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319815d230 con 0x7f319815c3d0 2024-04-25T10:02:58.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.327+0000 7f3196d9d640 1 --2- 172.21.15.139:0/624800938 >> v2:172.21.15.150:3300/0 conn(0x7f3198071700 0x7f319806d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:58.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.328+0000 7f318ffff640 1 --2- 172.21.15.139:0/624800938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f319815c3d0 0x7f31980711c0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f319806f320 tx=0x7f318002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00d8404ce7a80d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.328+0000 7f3195d9b640 1 -- 172.21.15.139:0/624800938 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180037070 con 0x7f319815c3d0 2024-04-25T10:02:58.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.328+0000 7f3195d9b640 1 -- 172.21.15.139:0/624800938 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3180032440 con 0x7f319815c3d0 2024-04-25T10:02:58.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.328+0000 7f3195d9b640 1 -- 172.21.15.139:0/624800938 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180032810 con 0x7f319815c3d0 2024-04-25T10:02:58.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.329+0000 7f319c989640 1 -- 172.21.15.139:0/624800938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f319815c3d0 msgr2=0x7f31980711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.329+0000 7f319c989640 1 --2- 172.21.15.139:0/624800938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f319815c3d0 0x7f31980711c0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f319806f320 tx=0x7f318002a140 comp rx=0 tx=0).stop 2024-04-25T10:02:58.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.329+0000 7f319c989640 1 -- 172.21.15.139:0/624800938 shutdown_connections 2024-04-25T10:02:58.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.329+0000 7f319c989640 1 --2- 172.21.15.139:0/624800938 >> v2:172.21.15.150:3300/0 conn(0x7f3198071700 0x7f319806d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.329+0000 7f319c989640 1 --2- 172.21.15.139:0/624800938 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f319815c3d0 0x7f31980711c0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.329+0000 7f319c989640 1 --2- 172.21.15.139:0/624800938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198145230 0x7f319815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.329+0000 7f319c989640 1 -- 172.21.15.139:0/624800938 >> 172.21.15.139:0/624800938 conn(0x7f3198067000 msgr2=0x7f319806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.329+0000 7f319c989640 1 -- 172.21.15.139:0/624800938 shutdown_connections 2024-04-25T10:02:58.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.329+0000 7f319c989640 1 -- 172.21.15.139:0/624800938 wait complete. 2024-04-25T10:02:58.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.330+0000 7f319c989640 1 Processor -- start 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.330+0000 7f319c989640 1 -- start start 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319c989640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198071700 0x7f319814c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319c989640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198145230 0x7f319814b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319c989640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319815c3d0 0x7f31980ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319c989640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319806d760 con 0x7f3198071700 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319c989640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319806e4b0 con 0x7f319815c3d0 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319c989640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319806d8e0 con 0x7f3198145230 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198071700 0x7f319814c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198071700 0x7f319814c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45730/0 (socket says 172.21.15.139:45730) 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319659c640 1 -- 172.21.15.139:0/3746967990 learned_addr learned my addr 172.21.15.139:0/3746967990 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f3196d9d640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319815c3d0 0x7f31980ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f318ffff640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198145230 0x7f319814b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319659c640 1 -- 172.21.15.139:0/3746967990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198145230 msgr2=0x7f319814b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319659c640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198145230 0x7f319814b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319659c640 1 -- 172.21.15.139:0/3746967990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319815c3d0 msgr2=0x7f31980ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319659c640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319815c3d0 0x7f31980ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319659c640 1 -- 172.21.15.139:0/3746967990 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319814c700 con 0x7f3198071700 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f319659c640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198071700 0x7f319814c1c0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f318800c7f0 tx=0x7f318800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.331+0000 7f318effd640 1 -- 172.21.15.139:0/3746967990 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188015070 con 0x7f3198071700 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.332+0000 7f318effd640 1 -- 172.21.15.139:0/3746967990 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3188004240 con 0x7f3198071700 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.332+0000 7f319c989640 1 -- 172.21.15.139:0/3746967990 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3198151ff0 con 0x7f3198071700 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.332+0000 7f318effd640 1 -- 172.21.15.139:0/3746967990 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188007620 con 0x7f3198071700 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.332+0000 7f319c989640 1 -- 172.21.15.139:0/3746967990 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31980af790 con 0x7f3198071700 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.333+0000 7f319c989640 1 -- 172.21.15.139:0/3746967990 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319806baf0 con 0x7f3198071700 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.333+0000 7f318effd640 1 -- 172.21.15.139:0/3746967990 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f31880079d0 con 0x7f3198071700 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.333+0000 7f318effd640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f317806b4d0 0x7f317806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.333+0000 7f318effd640 1 -- 172.21.15.139:0/3746967990 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f318808a810 con 0x7f3198071700 2024-04-25T10:02:58.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.333+0000 7f318ffff640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f317806b4d0 0x7f317806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.334+0000 7f318ffff640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f317806b4d0 0x7f317806d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3180034000 tx=0x7f318002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.338+0000 7f318effd640 1 -- 172.21.15.139:0/3746967990 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3188059500 con 0x7f3198071700 2024-04-25T10:02:58.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.508+0000 7f319c989640 1 -- 172.21.15.139:0/3746967990 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31980b1af0 con 0x7f3198071700 2024-04-25T10:02:58.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.509+0000 7f318effd640 1 -- 172.21.15.139:0/3746967990 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3188058e00 con 0x7f3198071700 2024-04-25T10:02:58.514 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T10:02:58.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.511+0000 7f319c989640 1 -- 172.21.15.139:0/3746967990 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f317806b4d0 msgr2=0x7f317806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.511+0000 7f319c989640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f317806b4d0 0x7f317806d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3180034000 tx=0x7f318002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:58.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.511+0000 7f319c989640 1 -- 172.21.15.139:0/3746967990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198071700 msgr2=0x7f319814c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.511+0000 7f319c989640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198071700 0x7f319814c1c0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f318800c7f0 tx=0x7f318800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.511+0000 7f319c989640 1 -- 172.21.15.139:0/3746967990 shutdown_connections 2024-04-25T10:02:58.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.511+0000 7f319c989640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f317806b4d0 0x7f317806d990 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.511+0000 7f319c989640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319815c3d0 0x7f31980ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.511+0000 7f319c989640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198145230 0x7f319814b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.511+0000 7f319c989640 1 --2- 172.21.15.139:0/3746967990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3198071700 0x7f319814c1c0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.511+0000 7f319c989640 1 -- 172.21.15.139:0/3746967990 >> 172.21.15.139:0/3746967990 conn(0x7f3198067000 msgr2=0x7f3198142000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.511+0000 7f319c989640 1 -- 172.21.15.139:0/3746967990 shutdown_connections 2024-04-25T10:02:58.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.511+0000 7f319c989640 1 -- 172.21.15.139:0/3746967990 wait complete. 2024-04-25T10:02:58.527 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:58.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.622+0000 7feae1c4a640 1 Processor -- start 2024-04-25T10:02:58.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.622+0000 7feae1c4a640 1 -- start start 2024-04-25T10:02:58.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feae1c4a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feadc150ed0 0x7feadc0b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feae1c4a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feadc0cdd20 0x7feadc071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feae1c4a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc15a480 con 0x7feadc0b4740 2024-04-25T10:02:58.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feae1c4a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15a760 con 0x7feadc0cdd20 2024-04-25T10:02:58.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feadbfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7feadc0b4740 0x7feadc06d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48656/0 (socket says 172.21.15.139:48656) 2024-04-25T10:02:58.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feae1c4a640 1 -- 172.21.15.139:0/3835875657 --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15aa40 con 0x7feadc150ed0 2024-04-25T10:02:58.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feadbfff640 1 -- 172.21.15.139:0/3835875657 learned_addr learned my addr 172.21.15.139:0/3835875657 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:58.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feadaffd640 1 --2- 172.21.15.139:0/3835875657 >> v2:172.21.15.139:3300/0 conn(0x7feadc0cdd20 0x7feadc071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feadb7fe640 1 --2- 172.21.15.139:0/3835875657 >> v2:172.21.15.150:3300/0 conn(0x7feadc150ed0 0x7feadc0b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feadaffd640 1 -- 172.21.15.139:0/3835875657 >> v2:172.21.15.150:3300/0 conn(0x7feadc150ed0 msgr2=0x7feadc0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feadaffd640 1 --2- 172.21.15.139:0/3835875657 >> v2:172.21.15.150:3300/0 conn(0x7feadc150ed0 0x7feadc0b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feadaffd640 1 -- 172.21.15.139:0/3835875657 >> v1:172.21.15.139:6789/0 conn(0x7feadc0b4740 legacy=0x7feadc06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feadaffd640 1 -- 172.21.15.139:0/3835875657 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc15ad20 con 0x7feadc0cdd20 2024-04-25T10:02:58.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.623+0000 7feada7fc640 1 -- 172.21.15.139:0/3835875657 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647939214 0 0) 0x7feadc15a480 con 0x7feadc0b4740 2024-04-25T10:02:58.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.624+0000 7feadaffd640 1 --2- 172.21.15.139:0/3835875657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feadc0cdd20 0x7feadc071a50 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fead00097a0 tx=0x7fead002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b572803f8abc582a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.624+0000 7feada7fc640 1 -- 172.21.15.139:0/3835875657 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead0037070 con 0x7feadc0cdd20 2024-04-25T10:02:58.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.624+0000 7feada7fc640 1 -- 172.21.15.139:0/3835875657 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead002ae50 con 0x7feadc0cdd20 2024-04-25T10:02:58.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.624+0000 7feada7fc640 1 -- 172.21.15.139:0/3835875657 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead0032660 con 0x7feadc0cdd20 2024-04-25T10:02:58.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.625+0000 7feae1c4a640 1 -- 172.21.15.139:0/3835875657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feadc0cdd20 msgr2=0x7feadc071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.625+0000 7feae1c4a640 1 --2- 172.21.15.139:0/3835875657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feadc0cdd20 0x7feadc071a50 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fead00097a0 tx=0x7fead002a140 comp rx=0 tx=0).stop 2024-04-25T10:02:58.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.625+0000 7feae1c4a640 1 -- 172.21.15.139:0/3835875657 shutdown_connections 2024-04-25T10:02:58.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.625+0000 7feae1c4a640 1 --2- 172.21.15.139:0/3835875657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feadc0cdd20 0x7feadc071a50 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.625+0000 7feae1c4a640 1 --2- 172.21.15.139:0/3835875657 >> v2:172.21.15.150:3300/0 conn(0x7feadc150ed0 0x7feadc0b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.625+0000 7feae1c4a640 1 -- 172.21.15.139:0/3835875657 >> 172.21.15.139:0/3835875657 conn(0x7feadc067000 msgr2=0x7feadc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.625+0000 7feae1c4a640 1 -- 172.21.15.139:0/3835875657 shutdown_connections 2024-04-25T10:02:58.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.625+0000 7feae1c4a640 1 -- 172.21.15.139:0/3835875657 wait complete. 2024-04-25T10:02:58.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.626+0000 7feae1c4a640 1 Processor -- start 2024-04-25T10:02:58.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.626+0000 7feae1c4a640 1 -- start start 2024-04-25T10:02:58.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feae1c4a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feadc0b4740 0x7feadc130780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feae1c4a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feadc0cdd20 0x7feadc144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feae1c4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feadc150ed0 0x7feadc159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feae1c4a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc1512b0 con 0x7feadc150ed0 2024-04-25T10:02:58.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feae1c4a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc0b3960 con 0x7feadc0b4740 2024-04-25T10:02:58.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feae1c4a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc071f90 con 0x7feadc0cdd20 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feadbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feadc150ed0 0x7feadc159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feadbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feadc150ed0 0x7feadc159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45744/0 (socket says 172.21.15.139:45744) 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feadbfff640 1 -- 172.21.15.139:0/3522530098 learned_addr learned my addr 172.21.15.139:0/3522530098 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feadb7fe640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feadc0b4740 0x7feadc130780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feadaffd640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feadc0cdd20 0x7feadc144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feadbfff640 1 -- 172.21.15.139:0/3522530098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feadc0cdd20 msgr2=0x7feadc144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feadbfff640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feadc0cdd20 0x7feadc144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feadbfff640 1 -- 172.21.15.139:0/3522530098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feadc0b4740 msgr2=0x7feadc130780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feadbfff640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feadc0b4740 0x7feadc130780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feadbfff640 1 -- 172.21.15.139:0/3522530098 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc14ce30 con 0x7feadc150ed0 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feadbfff640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feadc150ed0 0x7feadc159cb0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fead400c780 tx=0x7fead400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feac3fff640 1 -- 172.21.15.139:0/3522530098 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4018070 con 0x7feadc150ed0 2024-04-25T10:02:58.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feac3fff640 1 -- 172.21.15.139:0/3522530098 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead400a640 con 0x7feadc150ed0 2024-04-25T10:02:58.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feae1c4a640 1 -- 172.21.15.139:0/3522530098 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feadc14d110 con 0x7feadc150ed0 2024-04-25T10:02:58.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feac3fff640 1 -- 172.21.15.139:0/3522530098 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead400aab0 con 0x7feadc150ed0 2024-04-25T10:02:58.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.627+0000 7feae1c4a640 1 -- 172.21.15.139:0/3522530098 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feadc14d6d0 con 0x7feadc150ed0 2024-04-25T10:02:58.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.628+0000 7feae1c4a640 1 -- 172.21.15.139:0/3522530098 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feadc074520 con 0x7feadc150ed0 2024-04-25T10:02:58.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.629+0000 7feac3fff640 1 -- 172.21.15.139:0/3522530098 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fead401b070 con 0x7feadc150ed0 2024-04-25T10:02:58.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.629+0000 7feac3fff640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feac806b4d0 0x7feac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.629+0000 7feac3fff640 1 -- 172.21.15.139:0/3522530098 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fead4089900 con 0x7feadc150ed0 2024-04-25T10:02:58.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.629+0000 7feadb7fe640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feac806b4d0 0x7feac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.630+0000 7feadb7fe640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feac806b4d0 0x7feac806d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7feacc0079a0 tx=0x7feacc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.634+0000 7feac3fff640 1 -- 172.21.15.139:0/3522530098 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fead4057fa0 con 0x7feadc150ed0 2024-04-25T10:02:58.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.803+0000 7feae1c4a640 1 -- 172.21.15.139:0/3522530098 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feadc0b1590 con 0x7feadc150ed0 2024-04-25T10:02:58.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.803+0000 7feac3fff640 1 -- 172.21.15.139:0/3522530098 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) v1 ==== 100+0+11 (secure 0 0 0) 0x7fead405c040 con 0x7feadc150ed0 2024-04-25T10:02:58.808 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T10:02:58.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.805+0000 7feae1c4a640 1 -- 172.21.15.139:0/3522530098 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feac806b4d0 msgr2=0x7feac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.805+0000 7feae1c4a640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feac806b4d0 0x7feac806d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7feacc0079a0 tx=0x7feacc008040 comp rx=0 tx=0).stop 2024-04-25T10:02:58.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.805+0000 7feae1c4a640 1 -- 172.21.15.139:0/3522530098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feadc150ed0 msgr2=0x7feadc159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.805+0000 7feae1c4a640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feadc150ed0 0x7feadc159cb0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fead400c780 tx=0x7fead400cc40 comp rx=0 tx=0).stop 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.805+0000 7feae1c4a640 1 -- 172.21.15.139:0/3522530098 shutdown_connections 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.805+0000 7feae1c4a640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feadc150ed0 0x7feadc159cb0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.805+0000 7feae1c4a640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7feac806b4d0 0x7feac806d990 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.805+0000 7feae1c4a640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feadc0cdd20 0x7feadc144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.805+0000 7feae1c4a640 1 --2- 172.21.15.139:0/3522530098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feadc0b4740 0x7feadc130780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.805+0000 7feae1c4a640 1 -- 172.21.15.139:0/3522530098 >> 172.21.15.139:0/3522530098 conn(0x7feadc067000 msgr2=0x7feadc1588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.805+0000 7feae1c4a640 1 -- 172.21.15.139:0/3522530098 shutdown_connections 2024-04-25T10:02:58.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.805+0000 7feae1c4a640 1 -- 172.21.15.139:0/3522530098 wait complete. 2024-04-25T10:02:58.821 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:58.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.914+0000 7f99f970a640 1 Processor -- start 2024-04-25T10:02:58.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.914+0000 7f99f970a640 1 -- start start 2024-04-25T10:02:58.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f970a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99f415dbf0 0x7f99f41617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f970a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99f40b9370 0x7f99f40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f970a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f4161dc0 con 0x7f99f40cec80 2024-04-25T10:02:58.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f970a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f40d1ee0 con 0x7f99f415dbf0 2024-04-25T10:02:58.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f970a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f41620a0 con 0x7f99f40b9370 2024-04-25T10:02:58.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f2ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99f415dbf0 0x7f99f41617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f2ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99f415dbf0 0x7f99f41617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45746/0 (socket says 172.21.15.139:45746) 2024-04-25T10:02:58.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f2ffd640 1 -- 172.21.15.139:0/1366213689 learned_addr learned my addr 172.21.15.139:0/1366213689 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:58.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f27fc640 1 --2- 172.21.15.139:0/1366213689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99f40b9370 0x7f99f40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f2ffd640 1 -- 172.21.15.139:0/1366213689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99f40b9370 msgr2=0x7f99f40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/1366213689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99f40b9370 0x7f99f40ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f2ffd640 1 -- 172.21.15.139:0/1366213689 >> v1:172.21.15.150:6789/0 conn(0x7f99f40cec80 legacy=0x7f99f406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.915+0000 7f99f2ffd640 1 -- 172.21.15.139:0/1366213689 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f4162380 con 0x7f99f415dbf0 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.916+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/1366213689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f415dbf0 0x7f99f41617e0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009870 tx=0x7f99e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7b528dace80915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.916+0000 7f99f1ffb640 1 -- 172.21.15.139:0/1366213689 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4037070 con 0x7f99f415dbf0 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.916+0000 7f99f1ffb640 1 -- 172.21.15.139:0/1366213689 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e4032440 con 0x7f99f415dbf0 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.916+0000 7f99f1ffb640 1 -- 172.21.15.139:0/1366213689 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4032810 con 0x7f99f415dbf0 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.916+0000 7f99f970a640 1 -- 172.21.15.139:0/1366213689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f415dbf0 msgr2=0x7f99f41617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.916+0000 7f99f970a640 1 --2- 172.21.15.139:0/1366213689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f415dbf0 0x7f99f41617e0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009870 tx=0x7f99e402a210 comp rx=0 tx=0).stop 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.916+0000 7f99f970a640 1 -- 172.21.15.139:0/1366213689 shutdown_connections 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.916+0000 7f99f970a640 1 --2- 172.21.15.139:0/1366213689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99f40b9370 0x7f99f40ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.916+0000 7f99f970a640 1 --2- 172.21.15.139:0/1366213689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f415dbf0 0x7f99f41617e0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.916+0000 7f99f970a640 1 -- 172.21.15.139:0/1366213689 >> 172.21.15.139:0/1366213689 conn(0x7f99f40b5480 msgr2=0x7f99f40b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.917+0000 7f99f970a640 1 -- 172.21.15.139:0/1366213689 shutdown_connections 2024-04-25T10:02:58.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.917+0000 7f99f970a640 1 -- 172.21.15.139:0/1366213689 wait complete. 2024-04-25T10:02:58.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.918+0000 7f99f970a640 1 Processor -- start 2024-04-25T10:02:58.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.918+0000 7f99f970a640 1 -- start start 2024-04-25T10:02:58.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.918+0000 7f99f970a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f40b9370 0x7f99f40abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.918+0000 7f99f970a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99f40cec80 0x7f99f40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.918+0000 7f99f970a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99f415dbf0 0x7f99f414ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.918+0000 7f99f970a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f415dfd0 con 0x7f99f40b9370 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.918+0000 7f99f970a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40b97d0 con 0x7f99f415dbf0 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.918+0000 7f99f970a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40ab6d0 con 0x7f99f40cec80 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f40b9370 0x7f99f40abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f40b9370 0x7f99f40abb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45748/0 (socket says 172.21.15.139:45748) 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f2ffd640 1 -- 172.21.15.139:0/2331384296 learned_addr learned my addr 172.21.15.139:0/2331384296 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f37fe640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99f415dbf0 0x7f99f414ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f27fc640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99f40cec80 0x7f99f40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f2ffd640 1 -- 172.21.15.139:0/2331384296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99f40cec80 msgr2=0x7f99f40b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99f40cec80 0x7f99f40b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f2ffd640 1 -- 172.21.15.139:0/2331384296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99f415dbf0 msgr2=0x7f99f414ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99f415dbf0 0x7f99f414ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f2ffd640 1 -- 172.21.15.139:0/2331384296 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f40aaac0 con 0x7f99f40b9370 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f2ffd640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f40b9370 0x7f99f40abb60 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009870 tx=0x7f99e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99db7fe640 1 -- 172.21.15.139:0/2331384296 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4041070 con 0x7f99f40b9370 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99db7fe640 1 -- 172.21.15.139:0/2331384296 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e403c890 con 0x7f99f40b9370 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f970a640 1 -- 172.21.15.139:0/2331384296 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f40aada0 con 0x7f99f40b9370 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99db7fe640 1 -- 172.21.15.139:0/2331384296 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4037040 con 0x7f99f40b9370 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.919+0000 7f99f970a640 1 -- 172.21.15.139:0/2331384296 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f40ab360 con 0x7f99f40b9370 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.920+0000 7f99f970a640 1 -- 172.21.15.139:0/2331384296 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f4074520 con 0x7f99f40b9370 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.921+0000 7f99db7fe640 1 -- 172.21.15.139:0/2331384296 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99e4044070 con 0x7f99f40b9370 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.921+0000 7f99db7fe640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f99e006b4d0 0x7f99e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.921+0000 7f99db7fe640 1 -- 172.21.15.139:0/2331384296 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99e40aad60 con 0x7f99f40b9370 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.921+0000 7f99f27fc640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f99e006b4d0 0x7f99e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.922+0000 7f99f27fc640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f99e006b4d0 0x7f99e006d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f99f40b5d60 tx=0x7f99e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:58.925+0000 7f99db7fe640 1 -- 172.21.15.139:0/2331384296 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99e40795b0 con 0x7f99f40b9370 2024-04-25T10:02:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.112+0000 7f99f970a640 1 -- 172.21.15.139:0/2331384296 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f99f40b7e70 con 0x7f99f40b9370 2024-04-25T10:02:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.113+0000 7f99db7fe640 1 -- 172.21.15.139:0/2331384296 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v120) v1 ==== 80+0+0 (secure 0 0 0) 0x7f99e407d650 con 0x7f99f40b9370 2024-04-25T10:02:59.117 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T10:02:59.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.115+0000 7f99f970a640 1 -- 172.21.15.139:0/2331384296 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f99e006b4d0 msgr2=0x7f99e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.115+0000 7f99f970a640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f99e006b4d0 0x7f99e006d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f99f40b5d60 tx=0x7f99e8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.115+0000 7f99f970a640 1 -- 172.21.15.139:0/2331384296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f40b9370 msgr2=0x7f99f40abb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.115+0000 7f99f970a640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f40b9370 0x7f99f40abb60 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009870 tx=0x7f99e40046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.115+0000 7f99f970a640 1 -- 172.21.15.139:0/2331384296 shutdown_connections 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.115+0000 7f99f970a640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f99e006b4d0 0x7f99e006d990 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.115+0000 7f99f970a640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99f415dbf0 0x7f99f414ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.115+0000 7f99f970a640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99f40cec80 0x7f99f40b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.115+0000 7f99f970a640 1 --2- 172.21.15.139:0/2331384296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99f40b9370 0x7f99f40abb60 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.115+0000 7f99f970a640 1 -- 172.21.15.139:0/2331384296 >> 172.21.15.139:0/2331384296 conn(0x7f99f40b5480 msgr2=0x7f99f40d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.115+0000 7f99f970a640 1 -- 172.21.15.139:0/2331384296 shutdown_connections 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.115+0000 7f99f970a640 1 -- 172.21.15.139:0/2331384296 wait complete. 2024-04-25T10:02:59.130 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:02:59.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:59.131 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:59.180 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:02:59.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:59.181 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:02:59.203 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:02:59.204 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T10:02:59.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.338+0000 7fb3d5fb0640 1 Processor -- start 2024-04-25T10:02:59.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.338+0000 7fb3d5fb0640 1 -- start start 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.339+0000 7fb3d5fb0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb3d00d0a80 0x7fb3d01618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.339+0000 7fb3d5fb0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d009ad40 0x7fb3d0161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.339+0000 7fb3d5fb0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d0099ca0 con 0x7fb3d00cd610 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.339+0000 7fb3d5fb0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d0099a50 con 0x7fb3d00d0a80 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.339+0000 7fb3d5fb0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0099f80 con 0x7fb3d009ad40 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.339+0000 7fb3cf7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb3d00cd610 0x7fb3d00d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48672/0 (socket says 172.21.15.139:48672) 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.339+0000 7fb3cf7fe640 1 -- 172.21.15.139:0/622068965 learned_addr learned my addr 172.21.15.139:0/622068965 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.339+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338437741 0 0) 0x7fb3d0099ca0 con 0x7fb3d00cd610 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.339+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b8003520 con 0x7fb3d00cd610 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3cffff640 1 --2- 172.21.15.139:0/622068965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d009ad40 0x7fb3d0161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ceffd640 1 --2- 172.21.15.139:0/622068965 >> v2:172.21.15.150:3300/0 conn(0x7fb3d00d0a80 0x7fb3d01618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2240167706 0 0) 0x7fb3b8003520 con 0x7fb3d00cd610 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d009ad40 msgr2=0x7fb3d0161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 --2- 172.21.15.139:0/622068965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d009ad40 0x7fb3d0161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 >> v2:172.21.15.150:3300/0 conn(0x7fb3d00d0a80 msgr2=0x7fb3d01618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 --2- 172.21.15.139:0/622068965 >> v2:172.21.15.150:3300/0 conn(0x7fb3d00d0a80 0x7fb3d01618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d009a260 con 0x7fb3d00cd610 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fb3c0001670 con 0x7fb3d00cd610 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 >> v1:172.21.15.139:6789/0 conn(0x7fb3d00cd610 legacy=0x7fb3d00d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 --2- 172.21.15.139:0/622068965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b80039b0 0x7fb3b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 --2- 172.21.15.139:0/622068965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3b8004800 0x7fb3b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 --2- 172.21.15.139:0/622068965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d009ad40 0x7fb3b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0099a50 con 0x7fb3b80039b0 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0099f80 con 0x7fb3b8004800 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b800b9c0 con 0x7fb3d009ad40 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/622068965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b80039b0 0x7fb3b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3cffff640 1 --2- 172.21.15.139:0/622068965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d009ad40 0x7fb3b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3ceffd640 1 --2- 172.21.15.139:0/622068965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3b8004800 0x7fb3b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3cf7fe640 1 -- 172.21.15.139:0/622068965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d009ad40 msgr2=0x7fb3b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/622068965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d009ad40 0x7fb3b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3cf7fe640 1 -- 172.21.15.139:0/622068965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3b8004800 msgr2=0x7fb3b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/622068965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3b8004800 0x7fb3b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.340+0000 7fb3cf7fe640 1 -- 172.21.15.139:0/622068965 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b8003520 con 0x7fb3b80039b0 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.341+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/622068965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b80039b0 0x7fb3b8003dd0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0001050 tx=0x7fb3b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d81cbba12c6ca56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.341+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c002e070 con 0x7fb3b80039b0 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.341+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c0005af0 con 0x7fb3b80039b0 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.341+0000 7fb3ce7fc640 1 -- 172.21.15.139:0/622068965 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0007d70 con 0x7fb3b80039b0 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.342+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/622068965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b80039b0 msgr2=0x7fb3b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.342+0000 7fb3d5fb0640 1 --2- 172.21.15.139:0/622068965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b80039b0 0x7fb3b8003dd0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0001050 tx=0x7fb3b8005830 comp rx=0 tx=0).stop 2024-04-25T10:02:59.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.342+0000 7fb3cffff640 1 -- 172.21.15.139:0/622068965 reap_dead start 2024-04-25T10:02:59.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.342+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/622068965 shutdown_connections 2024-04-25T10:02:59.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.342+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/622068965 >> 172.21.15.139:0/622068965 conn(0x7fb3d00b31f0 msgr2=0x7fb3d00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.342+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/622068965 shutdown_connections 2024-04-25T10:02:59.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.342+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/622068965 wait complete. 2024-04-25T10:02:59.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.343+0000 7fb3d5fb0640 1 Processor -- start 2024-04-25T10:02:59.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.343+0000 7fb3d5fb0640 1 -- start start 2024-04-25T10:02:59.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3d5fb0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3b80039b0 0x7fb3d01f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3d5fb0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d00b6540 0x7fb3d01f0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3d5fb0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3d00b69c0 0x7fb3d01f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3d5fb0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00cd9f0 con 0x7fb3d00b69c0 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3d5fb0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d009a820 con 0x7fb3b80039b0 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3d5fb0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00cdb70 con 0x7fb3d00b6540 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3d00b69c0 0x7fb3d01f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3d00b69c0 0x7fb3d01f0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45768/0 (socket says 172.21.15.139:45768) 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3cffff640 1 -- 172.21.15.139:0/890555680 learned_addr learned my addr 172.21.15.139:0/890555680 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3b80039b0 0x7fb3d01f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3ceffd640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d00b6540 0x7fb3d01f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3cffff640 1 -- 172.21.15.139:0/890555680 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d00b6540 msgr2=0x7fb3d01f0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3cffff640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d00b6540 0x7fb3d01f0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3cffff640 1 -- 172.21.15.139:0/890555680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3b80039b0 msgr2=0x7fb3d01f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3cffff640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3b80039b0 0x7fb3d01f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3cffff640 1 -- 172.21.15.139:0/890555680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d00b5800 con 0x7fb3d00b69c0 2024-04-25T10:02:59.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3cffff640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3d00b69c0 0x7fb3d01f0b40 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0161820 tx=0x7fb3c800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3b7fff640 1 -- 172.21.15.139:0/890555680 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800fd10 con 0x7fb3d00b69c0 2024-04-25T10:02:59.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3b7fff640 1 -- 172.21.15.139:0/890555680 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c801d070 con 0x7fb3d00b69c0 2024-04-25T10:02:59.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/890555680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d00b4760 con 0x7fb3d00b69c0 2024-04-25T10:02:59.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3b7fff640 1 -- 172.21.15.139:0/890555680 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8015670 con 0x7fb3d00b69c0 2024-04-25T10:02:59.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.344+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/890555680 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d00b4d20 con 0x7fb3d00b69c0 2024-04-25T10:02:59.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.346+0000 7fb3b7fff640 1 -- 172.21.15.139:0/890555680 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3c8015810 con 0x7fb3d00b69c0 2024-04-25T10:02:59.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.347+0000 7fb3b7fff640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb3bc06b4d0 0x7fb3bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.347+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb3bc06b4d0 0x7fb3bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.347+0000 7fb3b7fff640 1 -- 172.21.15.139:0/890555680 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3c808ab70 con 0x7fb3d00b69c0 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.347+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb3bc06b4d0 0x7fb3bc06d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0033000 tx=0x7fb3c0009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.348+0000 7fb3d8d94740 1 Processor -- start 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.348+0000 7fb3d8d94740 1 -- start start 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.348+0000 7fb3d8d94740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8054b10 0x55fea8057320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.348+0000 7fb3d8d94740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054ef0 0x55fea8057860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.348+0000 7fb3d8d94740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8027770 0x55fea802af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.348+0000 7fb3d8d94740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fea7f90da0 con 0x55fea8054ef0 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.348+0000 7fb3d8d94740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fea7f39b80 con 0x55fea8027770 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.348+0000 7fb3d8d94740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fea7f8ce40 con 0x55fea8054b10 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3ceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054ef0 0x55fea8057860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3ceffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054ef0 0x55fea8057860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45782/0 (socket says 172.21.15.139:45782) 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3ceffd640 1 -- 172.21.15.139:0/3797887628 learned_addr learned my addr 172.21.15.139:0/3797887628 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:59.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3cffff640 1 --2- 172.21.15.139:0/3797887628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8027770 0x55fea802af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/3797887628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8054b10 0x55fea8057320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3ceffd640 1 -- 172.21.15.139:0/3797887628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8054b10 msgr2=0x55fea8057320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3ceffd640 1 --2- 172.21.15.139:0/3797887628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8054b10 0x55fea8057320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3ceffd640 1 -- 172.21.15.139:0/3797887628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8027770 msgr2=0x55fea802af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3ceffd640 1 --2- 172.21.15.139:0/3797887628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8027770 0x55fea802af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3ceffd640 1 -- 172.21.15.139:0/3797887628 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fea7f43f10 con 0x55fea8054ef0 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3cffff640 1 --2- 172.21.15.139:0/3797887628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8027770 0x55fea802af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3ceffd640 1 --2- 172.21.15.139:0/3797887628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054ef0 0x55fea8057860 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009bb0 tx=0x7fb3c4014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e444f36b4be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3b57fa640 1 -- 172.21.15.139:0/3797887628 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4022070 con 0x55fea8054ef0 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3b57fa640 1 -- 172.21.15.139:0/3797887628 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c401d440 con 0x55fea8054ef0 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.349+0000 7fb3b57fa640 1 -- 172.21.15.139:0/3797887628 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c401d7f0 con 0x55fea8054ef0 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.350+0000 7fb3d8d94740 1 -- 172.21.15.139:0/3797887628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054ef0 msgr2=0x55fea8057860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.350+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/3797887628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054ef0 0x55fea8057860 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009bb0 tx=0x7fb3c4014780 comp rx=0 tx=0).stop 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.350+0000 7fb3d8d94740 1 -- 172.21.15.139:0/3797887628 shutdown_connections 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.350+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/3797887628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054ef0 0x55fea8057860 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.350+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/3797887628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8027770 0x55fea802af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.350+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/3797887628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8054b10 0x55fea8057320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.350+0000 7fb3d8d94740 1 -- 172.21.15.139:0/3797887628 >> 172.21.15.139:0/3797887628 conn(0x55fea80299e0 msgr2=0x55fea8072320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.350+0000 7fb3d8d94740 1 -- 172.21.15.139:0/3797887628 shutdown_connections 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.350+0000 7fb3d8d94740 1 -- 172.21.15.139:0/3797887628 wait complete. 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.350+0000 7fb3d8d94740 1 -- start start 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3d8d94740 1 Processor -- start 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3d8d94740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8054b10 0x55fea8079410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3d8d94740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8054ef0 0x55fea8044cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3d8d94740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8053dc0 0x55fea80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3d8d94740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fea802cb90 con 0x55fea8053dc0 2024-04-25T10:02:59.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3d8d94740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fea7f2f4b0 con 0x55fea8054b10 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3d8d94740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fea802ce70 con 0x55fea8054ef0 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3d8d94740 10 client.0 _ll_register_callbacks cb 0x55fea7f6f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8053dc0 0x55fea80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8053dc0 0x55fea80769f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45798/0 (socket says 172.21.15.139:45798) 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3cffff640 1 -- 172.21.15.139:0/4237231149 learned_addr learned my addr 172.21.15.139:0/4237231149 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3cf7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8054b10 0x55fea8079410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.351+0000 7fb3ceffd640 1 --2- 172.21.15.139:0/4237231149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8054ef0 0x55fea8044cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3cffff640 1 -- 172.21.15.139:0/4237231149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8054ef0 msgr2=0x55fea8044cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3cffff640 1 --2- 172.21.15.139:0/4237231149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8054ef0 0x55fea8044cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3cffff640 1 -- 172.21.15.139:0/4237231149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8054b10 msgr2=0x55fea8079410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3cffff640 1 --2- 172.21.15.139:0/4237231149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8054b10 0x55fea8079410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3cffff640 1 -- 172.21.15.139:0/4237231149 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fea7d6f7d0 con 0x55fea8053dc0 2024-04-25T10:02:59.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3cffff640 1 --2- 172.21.15.139:0/4237231149 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8053dc0 0x55fea80769f0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8061340 tx=0x7fb3c8059160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719d70fbe85559a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3ae7fc640 1 -- 172.21.15.139:0/4237231149 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8019020 con 0x55fea8053dc0 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3ae7fc640 1 -- 172.21.15.139:0/4237231149 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3c80613e0 con 0x55fea8053dc0 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3d8d94740 1 -- 172.21.15.139:0/4237231149 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3c401d440 con 0x55fea8053dc0 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3ae7fc640 1 -- 172.21.15.139:0/4237231149 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3c8061870 con 0x55fea8053dc0 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3d8d94740 10 client.9342 fetch_fsmap learned FSMap version 83 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3d8d94740 1 -- 172.21.15.139:0/4237231149 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x55fea7ca53f0 con 0x55fea8053dc0 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3ae7fc640 1 -- 172.21.15.139:0/4237231149 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c401d440 con 0x55fea8053dc0 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3ae7fc640 1 -- 172.21.15.139:0/4237231149 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb3c80585b0 con 0x55fea8053dc0 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3d8d94740 10 client.9342 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3d8d94740 10 client.9342 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3d8d94740 20 client.9342 populate_metadata read hostname 'smithi139' 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/4237231149 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x55fea8053210 0x55fea80535f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3d8d94740 4 client.9342 mds_command: new command op to 9132 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.352+0000 7fb3d8d94740 1 -- 172.21.15.139:0/4237231149 --> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fea7f9e2e0 con 0x55fea8053210 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.353+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/4237231149 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x55fea8053210 0x55fea80535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.353+0000 7fb3ae7fc640 10 client.9342 ms_handle_connect on v2:172.21.15.150:6833/1377607596 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.353+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/4237231149 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x55fea8053210 0x55fea80535f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eb0af8c88e9aff5 server_cookie=fde8c0977fd908e0 in_seq=0 out_seq=0 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.353+0000 7fb3ae7fc640 1 -- 172.21.15.139:0/4237231149 <== mds.0 v2:172.21.15.150:6833/1377607596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fea7f9e2e0 con 0x55fea8053210 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.353+0000 7fb3ae7fc640 10 client.9342 handle_command_reply: tid=0 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.353+0000 7fb3d8d94740 1 client.9342 shutdown 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.354+0000 7fb3d8d94740 1 -- 172.21.15.139:0/4237231149 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8053dc0 msgr2=0x55fea80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.354+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/4237231149 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8053dc0 0x55fea80769f0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8061340 tx=0x7fb3c8059160 comp rx=0 tx=0).stop 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.354+0000 7fb3d8d94740 1 -- 172.21.15.139:0/4237231149 shutdown_connections 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.354+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/4237231149 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x55fea8053210 0x55fea80535f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.354+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/4237231149 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8053dc0 0x55fea80769f0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.354+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/4237231149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8054ef0 0x55fea8044cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.354+0000 7fb3ae7fc640 0 client.9342 ms_handle_reset on v2:172.21.15.150:6833/1377607596 2024-04-25T10:02:59.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.354+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/4237231149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8054b10 0x55fea8079410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.354+0000 7fb3d8d94740 1 -- 172.21.15.139:0/4237231149 >> 172.21.15.139:0/4237231149 conn(0x55fea8027770 msgr2=0x55fea8028420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.354+0000 7fb3d8d94740 1 -- 172.21.15.139:0/4237231149 shutdown_connections 2024-04-25T10:02:59.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.354+0000 7fb3d8d94740 1 -- 172.21.15.139:0/4237231149 wait complete. 2024-04-25T10:02:59.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.354+0000 7fb3d8d94740 20 client.9342 trim_cache size 0 max 16384 2024-04-25T10:02:59.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.367+0000 7fb3d8d94740 1 Processor -- start 2024-04-25T10:02:59.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.367+0000 7fb3d8d94740 1 -- start start 2024-04-25T10:02:59.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.367+0000 7fb3d8d94740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8053210 0x55fea8094d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3d8d94740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8053dc0 0x55fea8044cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3d8d94740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 0x55fea80768b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3d8d94740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0200a70 con 0x55fea8054b10 2024-04-25T10:02:59.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3d8d94740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00c61b0 con 0x55fea8053210 2024-04-25T10:02:59.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3d8d94740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d01f7700 con 0x55fea8053dc0 2024-04-25T10:02:59.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 0x55fea80768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 0x55fea80768b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45814/0 (socket says 172.21.15.139:45814) 2024-04-25T10:02:59.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3cffff640 1 -- 172.21.15.139:0/2670003574 learned_addr learned my addr 172.21.15.139:0/2670003574 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3cffff640 1 -- 172.21.15.139:0/2670003574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8053dc0 msgr2=0x55fea8044cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3cffff640 1 --2- 172.21.15.139:0/2670003574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8053dc0 0x55fea8044cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3cffff640 1 -- 172.21.15.139:0/2670003574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8053210 msgr2=0x55fea8094d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3cffff640 1 --2- 172.21.15.139:0/2670003574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8053210 0x55fea8094d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3cffff640 1 -- 172.21.15.139:0/2670003574 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fea8053960 con 0x55fea8054b10 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3cffff640 1 --2- 172.21.15.139:0/2670003574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 0x55fea80768b0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80590f0 tx=0x7fb3c805d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f8a37457864fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3b4ff9640 1 -- 172.21.15.139:0/2670003574 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8074070 con 0x55fea8054b10 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3b4ff9640 1 -- 172.21.15.139:0/2670003574 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c80613e0 con 0x55fea8054b10 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.368+0000 7fb3b4ff9640 1 -- 172.21.15.139:0/2670003574 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8019040 con 0x55fea8054b10 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.369+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2670003574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 msgr2=0x55fea80768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.369+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/2670003574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 0x55fea80768b0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80590f0 tx=0x7fb3c805d000 comp rx=0 tx=0).stop 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.369+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2670003574 shutdown_connections 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.369+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/2670003574 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 0x55fea80768b0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.369+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/2670003574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8053dc0 0x55fea8044cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.369+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/2670003574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8053210 0x55fea8094d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.369+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2670003574 >> 172.21.15.139:0/2670003574 conn(0x55fea8027770 msgr2=0x55fea808bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.369+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2670003574 shutdown_connections 2024-04-25T10:02:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.369+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2670003574 wait complete. 2024-04-25T10:02:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.369+0000 7fb3d8d94740 1 -- start start 2024-04-25T10:02:59.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.369+0000 7fb3d8d94740 1 Processor -- start 2024-04-25T10:02:59.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3d8d94740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8053210 0x55fea8067cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3d8d94740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8053dc0 0x55fea8078900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3d8d94740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 0x55fea8072320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3d8d94740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fea7f524b0 con 0x55fea8054b10 2024-04-25T10:02:59.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3d8d94740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00c5be0 con 0x55fea8053210 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3d8d94740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fea8072f80 con 0x55fea8053dc0 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3d8d94740 10 client.0 _ll_register_callbacks cb 0x7fb3c4009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 0x55fea8072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 0x55fea8072320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45828/0 (socket says 172.21.15.139:45828) 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3cffff640 1 -- 172.21.15.139:0/2706963935 learned_addr learned my addr 172.21.15.139:0/2706963935 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3ceffd640 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8053dc0 0x55fea8078900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8053210 0x55fea8067cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3cffff640 1 -- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8053dc0 msgr2=0x55fea8078900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3cffff640 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8053dc0 0x55fea8078900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3cffff640 1 -- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8053210 msgr2=0x55fea8067cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.370+0000 7fb3cffff640 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8053210 0x55fea8067cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3cffff640 1 -- 172.21.15.139:0/2706963935 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fea7c650f0 con 0x55fea8054b10 2024-04-25T10:02:59.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8053210 0x55fea8067cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3cffff640 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 0x55fea8072320 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8088f10 tx=0x7fb3c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a4c8cb14655750 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3ae7fc640 1 -- 172.21.15.139:0/2706963935 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8019070 con 0x55fea8054b10 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3ae7fc640 1 -- 172.21.15.139:0/2706963935 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3c8023ca0 con 0x55fea8054b10 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2706963935 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3c80613e0 con 0x55fea8054b10 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3ae7fc640 1 -- 172.21.15.139:0/2706963935 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3c8074040 con 0x55fea8054b10 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3d8d94740 10 client.9348 fetch_fsmap learned FSMap version 83 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2706963935 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x55fea8078e40 con 0x55fea8054b10 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3ae7fc640 1 -- 172.21.15.139:0/2706963935 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c80613e0 con 0x55fea8054b10 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3ae7fc640 1 -- 172.21.15.139:0/2706963935 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb3c807f030 con 0x55fea8054b10 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3d8d94740 10 client.9348 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3d8d94740 10 client.9348 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3d8d94740 20 client.9348 populate_metadata read hostname 'smithi139' 2024-04-25T10:02:59.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x55fea8054ef0 0x55fea8072860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3d8d94740 4 client.9348 mds_command: new command op to 9132 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:02:59.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2706963935 --> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fea7f98630 con 0x55fea8054ef0 2024-04-25T10:02:59.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.371+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x55fea8054ef0 0x55fea8072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.372+0000 7fb3cf7fe640 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x55fea8054ef0 0x55fea8072860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=490809fcb97069ff server_cookie=2b35502bc5701eb1 in_seq=0 out_seq=0 2024-04-25T10:02:59.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.372+0000 7fb3ae7fc640 10 client.9348 ms_handle_connect on v2:172.21.15.150:6833/1377607596 2024-04-25T10:02:59.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.381+0000 7fb3ae7fc640 1 -- 172.21.15.139:0/2706963935 <== mds.0 v2:172.21.15.150:6833/1377607596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fea7f98630 con 0x55fea8054ef0 2024-04-25T10:02:59.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.381+0000 7fb3ae7fc640 10 client.9348 handle_command_reply: tid=0 2024-04-25T10:02:59.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.381+0000 7fb3d8d94740 1 client.9348 shutdown 2024-04-25T10:02:59.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.381+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2706963935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 msgr2=0x55fea8072320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.381+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 0x55fea8072320 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8088f10 tx=0x7fb3c8004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:59.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.381+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2706963935 shutdown_connections 2024-04-25T10:02:59.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:6833/1377607596,v1:172.21.15.150:6835/1377607596] conn(0x55fea8054ef0 0x55fea8072860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55fea8054b10 0x55fea8072320 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3ae7fc640 0 client.9348 ms_handle_reset on v2:172.21.15.150:6833/1377607596 2024-04-25T10:02:59.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55fea8053dc0 0x55fea8078900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d8d94740 1 --2- 172.21.15.139:0/2706963935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fea8053210 0x55fea8067cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2706963935 >> 172.21.15.139:0/2706963935 conn(0x55fea8027770 msgr2=0x55fea8044bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2706963935 shutdown_connections 2024-04-25T10:02:59.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d8d94740 1 -- 172.21.15.139:0/2706963935 wait complete. 2024-04-25T10:02:59.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d8d94740 20 client.9348 trim_cache size 0 max 16384 2024-04-25T10:02:59.388 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:02:59.388 INFO:teuthology.orchestra.run.smithi139.stdout: "message": "", 2024-04-25T10:02:59.388 INFO:teuthology.orchestra.run.smithi139.stdout: "return_code": 0 2024-04-25T10:02:59.388 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:02:59.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/890555680 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb3bc06b4d0 msgr2=0x7fb3bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d5fb0640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb3bc06b4d0 0x7fb3bc06d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0033000 tx=0x7fb3c0009030 comp rx=0 tx=0).stop 2024-04-25T10:02:59.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/890555680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3d00b69c0 msgr2=0x7fb3d01f0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d5fb0640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3d00b69c0 0x7fb3d01f0b40 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0161820 tx=0x7fb3c800f1b0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/890555680 shutdown_connections 2024-04-25T10:02:59.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d5fb0640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3d00b69c0 0x7fb3d01f0b40 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d5fb0640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb3bc06b4d0 0x7fb3bc06d990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d5fb0640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3d00b6540 0x7fb3d01f0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d5fb0640 1 --2- 172.21.15.139:0/890555680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3b80039b0 0x7fb3d01f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/890555680 >> 172.21.15.139:0/890555680 conn(0x7fb3d00b31f0 msgr2=0x7fb3d00b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/890555680 shutdown_connections 2024-04-25T10:02:59.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.382+0000 7fb3d5fb0640 1 -- 172.21.15.139:0/890555680 wait complete. 2024-04-25T10:02:59.398 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:59.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.490+0000 7f3029b5c640 1 Processor -- start 2024-04-25T10:02:59.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.490+0000 7f3029b5c640 1 -- start start 2024-04-25T10:02:59.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f3029b5c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30240cf900 0x7f30240d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f3029b5c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3024099a50 0x7f3024099ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f3029b5c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30240cfe70 con 0x7f302415efb0 2024-04-25T10:02:59.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f3029b5c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30240d0a80 con 0x7f3024099a50 2024-04-25T10:02:59.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f3029b5c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30240d0150 con 0x7f30240cf900 2024-04-25T10:02:59.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f3023fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3024099a50 0x7f3024099ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f30237fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f302415efb0 0x7f302406c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48682/0 (socket says 172.21.15.139:48682) 2024-04-25T10:02:59.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f30237fe640 1 -- 172.21.15.139:0/1666567062 learned_addr learned my addr 172.21.15.139:0/1666567062 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:02:59.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f3022ffd640 1 --2- 172.21.15.139:0/1666567062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30240cf900 0x7f30240d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f3023fff640 1 -- 172.21.15.139:0/1666567062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30240cf900 msgr2=0x7f30240d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f3023fff640 1 --2- 172.21.15.139:0/1666567062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30240cf900 0x7f30240d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f3023fff640 1 -- 172.21.15.139:0/1666567062 >> v1:172.21.15.139:6789/0 conn(0x7f302415efb0 legacy=0x7f302406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f3023fff640 1 -- 172.21.15.139:0/1666567062 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30240d0430 con 0x7f3024099a50 2024-04-25T10:02:59.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.491+0000 7f30227fc640 1 -- 172.21.15.139:0/1666567062 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122135281 0 0) 0x7f30240cfe70 con 0x7f302415efb0 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.492+0000 7f3023fff640 1 --2- 172.21.15.139:0/1666567062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3024099a50 0x7f3024099ee0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f301c009d90 tx=0x7f301c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0fe876b48f2c6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.492+0000 7f30227fc640 1 -- 172.21.15.139:0/1666567062 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c031600 con 0x7f3024099a50 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.492+0000 7f30227fc640 1 -- 172.21.15.139:0/1666567062 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301c0317a0 con 0x7f3024099a50 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.492+0000 7f30227fc640 1 -- 172.21.15.139:0/1666567062 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c031b70 con 0x7f3024099a50 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.492+0000 7f3029b5c640 1 -- 172.21.15.139:0/1666567062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3024099a50 msgr2=0x7f3024099ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.492+0000 7f3029b5c640 1 --2- 172.21.15.139:0/1666567062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3024099a50 0x7f3024099ee0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f301c009d90 tx=0x7f301c02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.493+0000 7f3029b5c640 1 -- 172.21.15.139:0/1666567062 shutdown_connections 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.493+0000 7f3029b5c640 1 --2- 172.21.15.139:0/1666567062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3024099a50 0x7f3024099ee0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.493+0000 7f3029b5c640 1 --2- 172.21.15.139:0/1666567062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30240cf900 0x7f30240d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.493+0000 7f3029b5c640 1 -- 172.21.15.139:0/1666567062 >> 172.21.15.139:0/1666567062 conn(0x7f30240d1790 msgr2=0x7f30240d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.493+0000 7f3029b5c640 1 -- 172.21.15.139:0/1666567062 shutdown_connections 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.493+0000 7f3029b5c640 1 -- 172.21.15.139:0/1666567062 wait complete. 2024-04-25T10:02:59.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.494+0000 7f3029b5c640 1 Processor -- start 2024-04-25T10:02:59.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.494+0000 7f3029b5c640 1 -- start start 2024-04-25T10:02:59.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f3029b5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3024099a50 0x7f30241f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f3029b5c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30240cf900 0x7f30241f7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f3029b5c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f302415efb0 0x7f30241f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f3029b5c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30240cbde0 con 0x7f3024099a50 2024-04-25T10:02:59.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f3029b5c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302406cf50 con 0x7f302415efb0 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f3029b5c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30240cbf60 con 0x7f30240cf900 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f30237fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3024099a50 0x7f30241f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f30237fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3024099a50 0x7f30241f6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45844/0 (socket says 172.21.15.139:45844) 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f30237fe640 1 -- 172.21.15.139:0/710665214 learned_addr learned my addr 172.21.15.139:0/710665214 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f3022ffd640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30240cf900 0x7f30241f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f3023fff640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f302415efb0 0x7f30241f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f30237fe640 1 -- 172.21.15.139:0/710665214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30240cf900 msgr2=0x7f30241f7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f30237fe640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30240cf900 0x7f30241f7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f30237fe640 1 -- 172.21.15.139:0/710665214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f302415efb0 msgr2=0x7f30241f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f30237fe640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f302415efb0 0x7f30241f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f30237fe640 1 -- 172.21.15.139:0/710665214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30240b64f0 con 0x7f3024099a50 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f30237fe640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3024099a50 0x7f30241f6dc0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f301400c430 tx=0x7f301400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f300bfff640 1 -- 172.21.15.139:0/710665214 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30140074e0 con 0x7f3024099a50 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f300bfff640 1 -- 172.21.15.139:0/710665214 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30140047e0 con 0x7f3024099a50 2024-04-25T10:02:59.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f3029b5c640 1 -- 172.21.15.139:0/710665214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30240b67d0 con 0x7f3024099a50 2024-04-25T10:02:59.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f300bfff640 1 -- 172.21.15.139:0/710665214 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3014004bd0 con 0x7f3024099a50 2024-04-25T10:02:59.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.495+0000 7f3029b5c640 1 -- 172.21.15.139:0/710665214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30240b6d90 con 0x7f3024099a50 2024-04-25T10:02:59.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.496+0000 7f3029b5c640 1 -- 172.21.15.139:0/710665214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3024074520 con 0x7f3024099a50 2024-04-25T10:02:59.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.497+0000 7f300bfff640 1 -- 172.21.15.139:0/710665214 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3014011070 con 0x7f3024099a50 2024-04-25T10:02:59.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.497+0000 7f300bfff640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f301006b4d0 0x7f301006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.497+0000 7f300bfff640 1 -- 172.21.15.139:0/710665214 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3014089ec0 con 0x7f3024099a50 2024-04-25T10:02:59.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.497+0000 7f3022ffd640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f301006b4d0 0x7f301006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.498+0000 7f3022ffd640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f301006b4d0 0x7f301006d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3018006fd0 tx=0x7f3018008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.501+0000 7f300bfff640 1 -- 172.21.15.139:0/710665214 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3014058560 con 0x7f3024099a50 2024-04-25T10:02:59.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.705+0000 7f3029b5c640 1 -- 172.21.15.139:0/710665214 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f302409ae50 con 0x7f3024099a50 2024-04-25T10:02:59.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.706+0000 7f300bfff640 1 -- 172.21.15.139:0/710665214 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) v1 ==== 53+0+73 (secure 0 0 0) 0x7f301405c600 con 0x7f3024099a50 2024-04-25T10:02:59.711 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:02:59.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.708+0000 7f3029b5c640 1 -- 172.21.15.139:0/710665214 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f301006b4d0 msgr2=0x7f301006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.708+0000 7f3029b5c640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f301006b4d0 0x7f301006d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3018006fd0 tx=0x7f3018008040 comp rx=0 tx=0).stop 2024-04-25T10:02:59.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.708+0000 7f3029b5c640 1 -- 172.21.15.139:0/710665214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3024099a50 msgr2=0x7f30241f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.708+0000 7f3029b5c640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3024099a50 0x7f30241f6dc0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f301400c430 tx=0x7f301400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.708+0000 7f3029b5c640 1 -- 172.21.15.139:0/710665214 shutdown_connections 2024-04-25T10:02:59.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.708+0000 7f3029b5c640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f301006b4d0 0x7f301006d990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.708+0000 7f3029b5c640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f302415efb0 0x7f30241f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.708+0000 7f3029b5c640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30240cf900 0x7f30241f7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.708+0000 7f3029b5c640 1 --2- 172.21.15.139:0/710665214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3024099a50 0x7f30241f6dc0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.708+0000 7f3029b5c640 1 -- 172.21.15.139:0/710665214 >> 172.21.15.139:0/710665214 conn(0x7f30240d1790 msgr2=0x7f3024156b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.708+0000 7f3029b5c640 1 -- 172.21.15.139:0/710665214 shutdown_connections 2024-04-25T10:02:59.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:02:59.708+0000 7f3029b5c640 1 -- 172.21.15.139:0/710665214 wait complete. 2024-04-25T10:02:59.723 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:02:59.724 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:59.724 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T10:02:59.724 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:02:59.724 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:59.724 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:02:59.724 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:02:59.724 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:59.724 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:59.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:59.724 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T10:02:59.740 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:59.740 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:59.740 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:59.740 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:59.740 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:59.740 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:59.740 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:59.740 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:59.740 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T10:02:59.740 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42119sec preferred_lft 42119sec 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:59.741 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:59.743 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:59.802 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T10:02:59.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:59.804 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T10:02:59.859 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T10:02:59.860 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:59.860 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:59.860 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:59.860 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:59.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:59.860 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T10:02:59.860 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T10:02:59.861 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:59.861 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:59.861 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:59.861 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T10:02:59.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:59.944 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T10:02:59.999 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:02:59.999 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:03:00.000 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:00.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:00.000 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T10:03:00.064 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:00.064 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:00.119 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:00.119 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:00.120 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:00.174 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:03:00.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:00.174 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:03:00.255 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:03:00.255 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:03:00.262 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:03:00.263 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:03:00.263 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T10:03:00.263 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T10:03:00.263 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T10:03:00.263 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:03:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:03:00.263 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:03:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:03:00.264 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:03:00.265 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:03:00.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:00.265 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:00.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:00.329 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:03:00.386 INFO:teuthology.orchestra.run.smithi139.stdout:client6986 2024-04-25T10:03:00.387 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:00.387 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:00.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:00.442 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-25T10:03:00.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:00.503 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-25T10:03:00.582 INFO:teuthology.orchestra.run.smithi139.stderr:8+0 records in 2024-04-25T10:03:00.583 INFO:teuthology.orchestra.run.smithi139.stderr:8+0 records out 2024-04-25T10:03:00.583 INFO:teuthology.orchestra.run.smithi139.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.024358 s, 344 MB/s 2024-04-25T10:03:00.583 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:00.583 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:00.583 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:00.583 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:00.583 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:03:00.583 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:00.687 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627781 2024-04-25T10:03:00.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:00.695 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:00.696 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:00.696 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:00.696 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:00.696 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:03:00.696 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:00.800 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627782 2024-04-25T10:03:00.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:00.809 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:00.809 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:00.865 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:00.865 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:00.920 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:01.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.054+0000 7fc38d8f7640 1 Processor -- start 2024-04-25T10:03:01.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.054+0000 7fc38d8f7640 1 -- start start 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc38d8f7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc38815cde0 0x7fc38815d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc38d8f7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc38815db50 0x7fc3880cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc38d8f7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38815d1c0 con 0x7fc3880cc2a0 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc38d8f7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3880cc720 con 0x7fc38815db50 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc38d8f7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880cd460 con 0x7fc38815cde0 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc3877fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc38815db50 0x7fc3880cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc3877fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc38815db50 0x7fc3880cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45852/0 (socket says 172.21.15.139:45852) 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc3877fe640 1 -- 172.21.15.139:0/3054972094 learned_addr learned my addr 172.21.15.139:0/3054972094 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc386ffd640 1 --2- 172.21.15.139:0/3054972094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc38815cde0 0x7fc38815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc3877fe640 1 -- 172.21.15.139:0/3054972094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc38815cde0 msgr2=0x7fc38815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc3877fe640 1 --2- 172.21.15.139:0/3054972094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc38815cde0 0x7fc38815d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc3877fe640 1 -- 172.21.15.139:0/3054972094 >> v1:172.21.15.150:6789/0 conn(0x7fc3880cc2a0 legacy=0x7fc38809a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.055+0000 7fc3877fe640 1 -- 172.21.15.139:0/3054972094 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388099bf0 con 0x7fc38815db50 2024-04-25T10:03:01.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.056+0000 7fc3877fe640 1 --2- 172.21.15.139:0/3054972094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc38815db50 0x7fc3880cf810 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fc380009e50 tx=0x7fc38002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf13e6d78ae465b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.056+0000 7fc385ffb640 1 -- 172.21.15.139:0/3054972094 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380036070 con 0x7fc38815db50 2024-04-25T10:03:01.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.056+0000 7fc385ffb640 1 -- 172.21.15.139:0/3054972094 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc380031600 con 0x7fc38815db50 2024-04-25T10:03:01.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.056+0000 7fc385ffb640 1 -- 172.21.15.139:0/3054972094 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3800319d0 con 0x7fc38815db50 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.057+0000 7fc38d8f7640 1 -- 172.21.15.139:0/3054972094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc38815db50 msgr2=0x7fc3880cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.057+0000 7fc38d8f7640 1 --2- 172.21.15.139:0/3054972094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc38815db50 0x7fc3880cf810 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fc380009e50 tx=0x7fc38002a710 comp rx=0 tx=0).stop 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.057+0000 7fc38d8f7640 1 -- 172.21.15.139:0/3054972094 shutdown_connections 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.057+0000 7fc38d8f7640 1 --2- 172.21.15.139:0/3054972094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc38815db50 0x7fc3880cf810 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.057+0000 7fc38d8f7640 1 --2- 172.21.15.139:0/3054972094 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc38815cde0 0x7fc38815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.057+0000 7fc38d8f7640 1 -- 172.21.15.139:0/3054972094 >> 172.21.15.139:0/3054972094 conn(0x7fc3880d1790 msgr2=0x7fc3880d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.057+0000 7fc38d8f7640 1 -- 172.21.15.139:0/3054972094 shutdown_connections 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.057+0000 7fc38d8f7640 1 -- 172.21.15.139:0/3054972094 wait complete. 2024-04-25T10:03:01.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.058+0000 7fc38d8f7640 1 Processor -- start 2024-04-25T10:03:01.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.058+0000 7fc38d8f7640 1 -- start start 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc38d8f7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3880cc2a0 0x7fc38806bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc38d8f7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc38815cde0 0x7fc38806c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc38d8f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc38815db50 0x7fc388144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc38d8f7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880cf4e0 con 0x7fc38815db50 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc38d8f7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880cf360 con 0x7fc38815cde0 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc38d8f7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3881f3ad0 con 0x7fc3880cc2a0 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc3877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc38815db50 0x7fc388144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc3877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc38815db50 0x7fc388144770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45856/0 (socket says 172.21.15.139:45856) 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc3877fe640 1 -- 172.21.15.139:0/4080522122 learned_addr learned my addr 172.21.15.139:0/4080522122 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc386ffd640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3880cc2a0 0x7fc38806bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc3867fc640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc38815cde0 0x7fc38806c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc3877fe640 1 -- 172.21.15.139:0/4080522122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3880cc2a0 msgr2=0x7fc38806bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc3877fe640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3880cc2a0 0x7fc38806bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc3877fe640 1 -- 172.21.15.139:0/4080522122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc38815cde0 msgr2=0x7fc38806c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc3877fe640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc38815cde0 0x7fc38806c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.059+0000 7fc3877fe640 1 -- 172.21.15.139:0/4080522122 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3881e4850 con 0x7fc38815db50 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.060+0000 7fc3877fe640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc38815db50 0x7fc388144770 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fc380034040 tx=0x7fc380004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.060+0000 7fc36ffff640 1 -- 172.21.15.139:0/4080522122 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380036050 con 0x7fc38815db50 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.060+0000 7fc36ffff640 1 -- 172.21.15.139:0/4080522122 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc380042070 con 0x7fc38815db50 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.060+0000 7fc38d8f7640 1 -- 172.21.15.139:0/4080522122 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3881e4b30 con 0x7fc38815db50 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.060+0000 7fc36ffff640 1 -- 172.21.15.139:0/4080522122 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380009510 con 0x7fc38815db50 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.060+0000 7fc38d8f7640 1 -- 172.21.15.139:0/4080522122 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3881e50f0 con 0x7fc38815db50 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.061+0000 7fc38d8f7640 1 -- 172.21.15.139:0/4080522122 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc388074520 con 0x7fc38815db50 2024-04-25T10:03:01.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.061+0000 7fc36ffff640 1 -- 172.21.15.139:0/4080522122 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc380007c50 con 0x7fc38815db50 2024-04-25T10:03:01.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.062+0000 7fc36ffff640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc37406b4d0 0x7fc37406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.062+0000 7fc386ffd640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc37406b4d0 0x7fc37406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.062+0000 7fc36ffff640 1 -- 172.21.15.139:0/4080522122 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc3800ab160 con 0x7fc38815db50 2024-04-25T10:03:01.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.062+0000 7fc386ffd640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc37406b4d0 0x7fc37406d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc378006fd0 tx=0x7fc378008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.066+0000 7fc36ffff640 1 -- 172.21.15.139:0/4080522122 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc380079800 con 0x7fc38815db50 2024-04-25T10:03:01.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.236+0000 7fc38d8f7640 1 -- 172.21.15.139:0/4080522122 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3880d25f0 con 0x7fc38815db50 2024-04-25T10:03:01.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.238+0000 7fc36ffff640 1 -- 172.21.15.139:0/4080522122 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+17000 (secure 0 0 0) 0x7fc38007d8a0 con 0x7fc38815db50 2024-04-25T10:03:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:01.243 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":120,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:02:51.983848+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:48.574891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:49.293101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:01.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.239+0000 7fc38d8f7640 1 -- 172.21.15.139:0/4080522122 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc37406b4d0 msgr2=0x7fc37406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.239+0000 7fc38d8f7640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc37406b4d0 0x7fc37406d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc378006fd0 tx=0x7fc378008040 comp rx=0 tx=0).stop 2024-04-25T10:03:01.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.239+0000 7fc38d8f7640 1 -- 172.21.15.139:0/4080522122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc38815db50 msgr2=0x7fc388144770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.239+0000 7fc38d8f7640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc38815db50 0x7fc388144770 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fc380034040 tx=0x7fc380004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:01.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.240+0000 7fc38d8f7640 1 -- 172.21.15.139:0/4080522122 shutdown_connections 2024-04-25T10:03:01.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.240+0000 7fc38d8f7640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc38815db50 0x7fc388144770 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.240+0000 7fc38d8f7640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc37406b4d0 0x7fc37406d990 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.240+0000 7fc38d8f7640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc38815cde0 0x7fc38806c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.240+0000 7fc38d8f7640 1 --2- 172.21.15.139:0/4080522122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc3880cc2a0 0x7fc38806bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.240+0000 7fc38d8f7640 1 -- 172.21.15.139:0/4080522122 >> 172.21.15.139:0/4080522122 conn(0x7fc3880d1790 msgr2=0x7fc388070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.240+0000 7fc38d8f7640 1 -- 172.21.15.139:0/4080522122 shutdown_connections 2024-04-25T10:03:01.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.240+0000 7fc38d8f7640 1 -- 172.21.15.139:0/4080522122 wait complete. 2024-04-25T10:03:01.255 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:48.574891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:49.293101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:01.256 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:01.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.351+0000 7f43308b6640 1 Processor -- start 2024-04-25T10:03:01.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.351+0000 7f43308b6640 1 -- start start 2024-04-25T10:03:01.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f43308b6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432c0d0080 0x7f432c1664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f43308b6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432c0ccef0 con 0x7f432c15cb20 2024-04-25T10:03:01.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f43308b6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432c0cd1d0 con 0x7f432c0afc00 2024-04-25T10:03:01.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f43308b6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0703c0 con 0x7f432c0d0080 2024-04-25T10:03:01.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f4329d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f432c15cb20 0x7f432c0cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48686/0 (socket says 172.21.15.139:48686) 2024-04-25T10:03:01.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f4329d9b640 1 -- 172.21.15.139:0/1537813067 learned_addr learned my addr 172.21.15.139:0/1537813067 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:01.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f432ad9d640 1 --2- 172.21.15.139:0/1537813067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432c0d0080 0x7f432c1664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f432959a640 1 -- 172.21.15.139:0/1537813067 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790469184 0 0) 0x7f432c0ccef0 con 0x7f432c15cb20 2024-04-25T10:03:01.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f432959a640 1 -- 172.21.15.139:0/1537813067 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4314003520 con 0x7f432c15cb20 2024-04-25T10:03:01.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f432959a640 1 -- 172.21.15.139:0/1537813067 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527900197 0 0) 0x7f432c0cd1d0 con 0x7f432c0afc00 2024-04-25T10:03:01.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f432959a640 1 -- 172.21.15.139:0/1537813067 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f432c0ccef0 con 0x7f432c0afc00 2024-04-25T10:03:01.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f432ad9d640 1 -- 172.21.15.139:0/1537813067 >> v1:172.21.15.150:6789/0 conn(0x7f432c0afc00 legacy=0x7f432c06fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f432ad9d640 1 -- 172.21.15.139:0/1537813067 >> v1:172.21.15.139:6789/0 conn(0x7f432c15cb20 legacy=0x7f432c0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.352+0000 7f432ad9d640 1 -- 172.21.15.139:0/1537813067 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c0cd4b0 con 0x7f432c0d0080 2024-04-25T10:03:01.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.353+0000 7f432ad9d640 1 --2- 172.21.15.139:0/1537813067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432c0d0080 0x7f432c1664d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4324009dc0 tx=0x7f432402a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7251542fa99861bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.353+0000 7f432959a640 1 -- 172.21.15.139:0/1537813067 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324007740 con 0x7f432c0d0080 2024-04-25T10:03:01.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.353+0000 7f432959a640 1 -- 172.21.15.139:0/1537813067 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43240078e0 con 0x7f432c0d0080 2024-04-25T10:03:01.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.353+0000 7f432959a640 1 -- 172.21.15.139:0/1537813067 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324007cb0 con 0x7f432c0d0080 2024-04-25T10:03:01.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.354+0000 7f43308b6640 1 -- 172.21.15.139:0/1537813067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432c0d0080 msgr2=0x7f432c1664d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.354+0000 7f43308b6640 1 --2- 172.21.15.139:0/1537813067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432c0d0080 0x7f432c1664d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4324009dc0 tx=0x7f432402a5b0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.354+0000 7f43308b6640 1 -- 172.21.15.139:0/1537813067 shutdown_connections 2024-04-25T10:03:01.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.354+0000 7f43308b6640 1 --2- 172.21.15.139:0/1537813067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432c0d0080 0x7f432c1664d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.354+0000 7f43308b6640 1 -- 172.21.15.139:0/1537813067 >> 172.21.15.139:0/1537813067 conn(0x7f432c0b2230 msgr2=0x7f432c0b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.354+0000 7f43308b6640 1 -- 172.21.15.139:0/1537813067 shutdown_connections 2024-04-25T10:03:01.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.354+0000 7f43308b6640 1 -- 172.21.15.139:0/1537813067 wait complete. 2024-04-25T10:03:01.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.355+0000 7f43308b6640 1 Processor -- start 2024-04-25T10:03:01.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.355+0000 7f43308b6640 1 -- start start 2024-04-25T10:03:01.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f43308b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f432c0afc00 0x7f432c1507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f43308b6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432c0d0080 0x7f432c0b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f43308b6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f432c15cb20 0x7f432c14af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f43308b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0affe0 con 0x7f432c0afc00 2024-04-25T10:03:01.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f43308b6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0cd820 con 0x7f432c15cb20 2024-04-25T10:03:01.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f43308b6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c166a10 con 0x7f432c0d0080 2024-04-25T10:03:01.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f432a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f432c0afc00 0x7f432c1507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f432a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f432c0afc00 0x7f432c1507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45862/0 (socket says 172.21.15.139:45862) 2024-04-25T10:03:01.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f432a59c640 1 -- 172.21.15.139:0/3649543777 learned_addr learned my addr 172.21.15.139:0/3649543777 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:01.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f432ad9d640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f432c15cb20 0x7f432c14af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f4329d9b640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432c0d0080 0x7f432c0b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f432a59c640 1 -- 172.21.15.139:0/3649543777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432c0d0080 msgr2=0x7f432c0b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f432a59c640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432c0d0080 0x7f432c0b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f432a59c640 1 -- 172.21.15.139:0/3649543777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f432c15cb20 msgr2=0x7f432c14af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f432a59c640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f432c15cb20 0x7f432c14af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f432a59c640 1 -- 172.21.15.139:0/3649543777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c151020 con 0x7f432c0afc00 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f432ad9d640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f432c15cb20 0x7f432c14af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f4329d9b640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432c0d0080 0x7f432c0b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.356+0000 7f432a59c640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f432c0afc00 0x7f432c1507d0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c430 tx=0x7f431c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.357+0000 7f431affd640 1 -- 172.21.15.139:0/3649543777 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c017070 con 0x7f432c0afc00 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.357+0000 7f431affd640 1 -- 172.21.15.139:0/3649543777 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c0052f0 con 0x7f432c0afc00 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.357+0000 7f43308b6640 1 -- 172.21.15.139:0/3649543777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432c0ad640 con 0x7f432c0afc00 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.357+0000 7f431affd640 1 -- 172.21.15.139:0/3649543777 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c003830 con 0x7f432c0afc00 2024-04-25T10:03:01.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.357+0000 7f43308b6640 1 -- 172.21.15.139:0/3649543777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432c150d10 con 0x7f432c0afc00 2024-04-25T10:03:01.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.358+0000 7f43308b6640 1 -- 172.21.15.139:0/3649543777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432c074520 con 0x7f432c0afc00 2024-04-25T10:03:01.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.358+0000 7f431affd640 1 -- 172.21.15.139:0/3649543777 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f431c01a070 con 0x7f432c0afc00 2024-04-25T10:03:01.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.358+0000 7f431affd640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f430c06b4d0 0x7f430c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.358+0000 7f4329d9b640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f430c06b4d0 0x7f430c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.358+0000 7f431affd640 1 -- 172.21.15.139:0/3649543777 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f431c0896d0 con 0x7f432c0afc00 2024-04-25T10:03:01.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.359+0000 7f4329d9b640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f430c06b4d0 0x7f430c06d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f432c0af960 tx=0x7f4320009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.362+0000 7f431affd640 1 -- 172.21.15.139:0/3649543777 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f431c058450 con 0x7f432c0afc00 2024-04-25T10:03:01.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.534+0000 7f43308b6640 1 -- 172.21.15.139:0/3649543777 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f432c151b30 con 0x7f432c0afc00 2024-04-25T10:03:01.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.535+0000 7f431affd640 1 -- 172.21.15.139:0/3649543777 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7f431c057d50 con 0x7f432c0afc00 2024-04-25T10:03:01.540 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:03:01.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.537+0000 7f43308b6640 1 -- 172.21.15.139:0/3649543777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f430c06b4d0 msgr2=0x7f430c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.537+0000 7f43308b6640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f430c06b4d0 0x7f430c06d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f432c0af960 tx=0x7f4320009040 comp rx=0 tx=0).stop 2024-04-25T10:03:01.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.537+0000 7f43308b6640 1 -- 172.21.15.139:0/3649543777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f432c0afc00 msgr2=0x7f432c1507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.537+0000 7f43308b6640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f432c0afc00 0x7f432c1507d0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c430 tx=0x7f431c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.537+0000 7f43308b6640 1 -- 172.21.15.139:0/3649543777 shutdown_connections 2024-04-25T10:03:01.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.537+0000 7f43308b6640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f430c06b4d0 0x7f430c06d990 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.537+0000 7f43308b6640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f432c15cb20 0x7f432c14af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.537+0000 7f43308b6640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432c0d0080 0x7f432c0b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.537+0000 7f43308b6640 1 --2- 172.21.15.139:0/3649543777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f432c0afc00 0x7f432c1507d0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.537+0000 7f43308b6640 1 -- 172.21.15.139:0/3649543777 >> 172.21.15.139:0/3649543777 conn(0x7f432c0b2230 msgr2=0x7f432c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.537+0000 7f43308b6640 1 -- 172.21.15.139:0/3649543777 shutdown_connections 2024-04-25T10:03:01.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.537+0000 7f43308b6640 1 -- 172.21.15.139:0/3649543777 wait complete. 2024-04-25T10:03:01.552 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:01.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.647+0000 7f2389c4d640 1 Processor -- start 2024-04-25T10:03:01.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.647+0000 7f2389c4d640 1 -- start start 2024-04-25T10:03:01.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.648+0000 7f2389c4d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f238415dbf0 0x7f23841617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.648+0000 7f2389c4d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f23840b9370 0x7f23840ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.648+0000 7f2389c4d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2384161dc0 con 0x7f23840cec80 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.648+0000 7f2389c4d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23840d1ee0 con 0x7f23840b9370 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.648+0000 7f2389c4d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23841620a0 con 0x7f238415dbf0 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.649+0000 7f23837fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f238415dbf0 0x7f23841617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.649+0000 7f2382ffd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f23840b9370 0x7f23840ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.649+0000 7f2383fff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f23840cec80 0x7f238406c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:38968/0 (socket says 172.21.15.139:38968) 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.649+0000 7f2383fff640 1 -- 172.21.15.139:0/1674155732 learned_addr learned my addr 172.21.15.139:0/1674155732 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.649+0000 7f23837fe640 1 --2- 172.21.15.139:0/1674155732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f238415dbf0 0x7f23841617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:35228/0 (socket says 172.21.15.139:35228) 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.649+0000 7f2382ffd640 1 -- 172.21.15.139:0/1674155732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f238415dbf0 msgr2=0x7f23841617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.649+0000 7f2382ffd640 1 --2- 172.21.15.139:0/1674155732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f238415dbf0 0x7f23841617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.649+0000 7f2382ffd640 1 -- 172.21.15.139:0/1674155732 >> v1:172.21.15.150:6789/0 conn(0x7f23840cec80 legacy=0x7f238406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.649+0000 7f2382ffd640 1 -- 172.21.15.139:0/1674155732 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2384162380 con 0x7f23840b9370 2024-04-25T10:03:01.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.649+0000 7f23837fe640 1 --2- 172.21.15.139:0/1674155732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f238415dbf0 0x7f23841617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:01.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.650+0000 7f2382ffd640 1 --2- 172.21.15.139:0/1674155732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23840b9370 0x7f23840ce740 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f23780097a0 tx=0x7f237802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d54252f895b748f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.650+0000 7f23827fc640 1 -- 172.21.15.139:0/1674155732 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378037070 con 0x7f23840b9370 2024-04-25T10:03:01.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.650+0000 7f23827fc640 1 -- 172.21.15.139:0/1674155732 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2378032440 con 0x7f23840b9370 2024-04-25T10:03:01.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.650+0000 7f23827fc640 1 -- 172.21.15.139:0/1674155732 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378032810 con 0x7f23840b9370 2024-04-25T10:03:01.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.650+0000 7f2389c4d640 1 -- 172.21.15.139:0/1674155732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23840b9370 msgr2=0x7f23840ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.650+0000 7f2389c4d640 1 --2- 172.21.15.139:0/1674155732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23840b9370 0x7f23840ce740 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f23780097a0 tx=0x7f237802a180 comp rx=0 tx=0).stop 2024-04-25T10:03:01.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.650+0000 7f2389c4d640 1 -- 172.21.15.139:0/1674155732 shutdown_connections 2024-04-25T10:03:01.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.650+0000 7f2389c4d640 1 --2- 172.21.15.139:0/1674155732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23840b9370 0x7f23840ce740 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.650+0000 7f2389c4d640 1 --2- 172.21.15.139:0/1674155732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f238415dbf0 0x7f23841617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.650+0000 7f2389c4d640 1 -- 172.21.15.139:0/1674155732 >> 172.21.15.139:0/1674155732 conn(0x7f23840b5480 msgr2=0x7f23840b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.650+0000 7f2389c4d640 1 -- 172.21.15.139:0/1674155732 shutdown_connections 2024-04-25T10:03:01.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.651+0000 7f2389c4d640 1 -- 172.21.15.139:0/1674155732 wait complete. 2024-04-25T10:03:01.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.652+0000 7f2389c4d640 1 Processor -- start 2024-04-25T10:03:01.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.652+0000 7f2389c4d640 1 -- start start 2024-04-25T10:03:01.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.652+0000 7f2389c4d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23840b9370 0x7f23840c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.652+0000 7f2389c4d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23840cec80 0x7f238415c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.652+0000 7f2389c4d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f238415dbf0 0x7f23840aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.652+0000 7f2389c4d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238415dfd0 con 0x7f23840cec80 2024-04-25T10:03:01.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.652+0000 7f2389c4d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23840b97d0 con 0x7f23840b9370 2024-04-25T10:03:01.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.652+0000 7f2389c4d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238415b840 con 0x7f238415dbf0 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23840cec80 0x7f238415c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f2382ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23840cec80 0x7f238415c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45878/0 (socket says 172.21.15.139:45878) 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f2382ffd640 1 -- 172.21.15.139:0/2164052728 learned_addr learned my addr 172.21.15.139:0/2164052728 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f23837fe640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23840b9370 0x7f23840c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f2383fff640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f238415dbf0 0x7f23840aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f2382ffd640 1 -- 172.21.15.139:0/2164052728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f238415dbf0 msgr2=0x7f23840aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f2382ffd640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f238415dbf0 0x7f23840aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f2382ffd640 1 -- 172.21.15.139:0/2164052728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23840b9370 msgr2=0x7f23840c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f2382ffd640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23840b9370 0x7f23840c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f2382ffd640 1 -- 172.21.15.139:0/2164052728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238414bb60 con 0x7f23840cec80 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f2382ffd640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23840cec80 0x7f238415c570 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f23780097a0 tx=0x7f23780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f236bfff640 1 -- 172.21.15.139:0/2164052728 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378041070 con 0x7f23840cec80 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f236bfff640 1 -- 172.21.15.139:0/2164052728 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237803c870 con 0x7f23840cec80 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f2389c4d640 1 -- 172.21.15.139:0/2164052728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238415ba90 con 0x7f23840cec80 2024-04-25T10:03:01.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f236bfff640 1 -- 172.21.15.139:0/2164052728 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378037040 con 0x7f23840cec80 2024-04-25T10:03:01.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.653+0000 7f2389c4d640 1 -- 172.21.15.139:0/2164052728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238415c050 con 0x7f23840cec80 2024-04-25T10:03:01.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.654+0000 7f2389c4d640 1 -- 172.21.15.139:0/2164052728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23840d3280 con 0x7f23840cec80 2024-04-25T10:03:01.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.655+0000 7f236bfff640 1 -- 172.21.15.139:0/2164052728 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2378044070 con 0x7f23840cec80 2024-04-25T10:03:01.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.655+0000 7f236bfff640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f237006b4d0 0x7f237006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.655+0000 7f236bfff640 1 -- 172.21.15.139:0/2164052728 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f23780aaf90 con 0x7f23840cec80 2024-04-25T10:03:01.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.655+0000 7f23837fe640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f237006b4d0 0x7f237006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.656+0000 7f23837fe640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f237006b4d0 0x7f237006d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2374009560 tx=0x7f2374009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.659+0000 7f236bfff640 1 -- 172.21.15.139:0/2164052728 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f23840d3280 con 0x7f23840cec80 2024-04-25T10:03:01.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.829+0000 7f2389c4d640 1 -- 172.21.15.139:0/2164052728 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23840cf630 con 0x7f23840cec80 2024-04-25T10:03:01.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.829+0000 7f236bfff640 1 -- 172.21.15.139:0/2164052728 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23840cf630 con 0x7f23840cec80 2024-04-25T10:03:01.834 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T10:03:01.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.831+0000 7f2389c4d640 1 -- 172.21.15.139:0/2164052728 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f237006b4d0 msgr2=0x7f237006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.831+0000 7f2389c4d640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f237006b4d0 0x7f237006d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2374009560 tx=0x7f2374009190 comp rx=0 tx=0).stop 2024-04-25T10:03:01.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.831+0000 7f2389c4d640 1 -- 172.21.15.139:0/2164052728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23840cec80 msgr2=0x7f238415c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.831+0000 7f2389c4d640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23840cec80 0x7f238415c570 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f23780097a0 tx=0x7f23780046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.831+0000 7f2389c4d640 1 -- 172.21.15.139:0/2164052728 shutdown_connections 2024-04-25T10:03:01.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.831+0000 7f2389c4d640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23840cec80 0x7f238415c570 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.831+0000 7f2389c4d640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f237006b4d0 0x7f237006d990 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.831+0000 7f2389c4d640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f238415dbf0 0x7f23840aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.831+0000 7f2389c4d640 1 --2- 172.21.15.139:0/2164052728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23840b9370 0x7f23840c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.831+0000 7f2389c4d640 1 -- 172.21.15.139:0/2164052728 >> 172.21.15.139:0/2164052728 conn(0x7f23840b5480 msgr2=0x7f23840b7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.831+0000 7f2389c4d640 1 -- 172.21.15.139:0/2164052728 shutdown_connections 2024-04-25T10:03:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.831+0000 7f2389c4d640 1 -- 172.21.15.139:0/2164052728 wait complete. 2024-04-25T10:03:01.847 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.941+0000 7f7c823ce640 1 Processor -- start 2024-04-25T10:03:01.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.941+0000 7f7c823ce640 1 -- start start 2024-04-25T10:03:01.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.942+0000 7f7c823ce640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7c7c0d0a80 0x7f7c7c1618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.942+0000 7f7c823ce640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7c7c09ad40 0x7f7c7c161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.942+0000 7f7c823ce640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c7c099ca0 con 0x7f7c7c0cd610 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.942+0000 7f7c823ce640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c7c099a50 con 0x7f7c7c09ad40 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.942+0000 7f7c823ce640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c7c099f80 con 0x7f7c7c0d0a80 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.942+0000 7f7c7bfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f7c7c0cd610 0x7f7c7c0d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48696/0 (socket says 172.21.15.139:48696) 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.942+0000 7f7c8096b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7c7c09ad40 0x7f7c7c161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.942+0000 7f7c7bfff640 1 -- 172.21.15.139:0/1627015345 learned_addr learned my addr 172.21.15.139:0/1627015345 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.942+0000 7f7c7b7fe640 1 --2- 172.21.15.139:0/1627015345 >> v2:172.21.15.150:3300/0 conn(0x7f7c7c0d0a80 0x7f7c7c1618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.943+0000 7f7c8096b640 1 -- 172.21.15.139:0/1627015345 >> v2:172.21.15.150:3300/0 conn(0x7f7c7c0d0a80 msgr2=0x7f7c7c1618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.943+0000 7f7c8096b640 1 --2- 172.21.15.139:0/1627015345 >> v2:172.21.15.150:3300/0 conn(0x7f7c7c0d0a80 0x7f7c7c1618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.943+0000 7f7c8096b640 1 -- 172.21.15.139:0/1627015345 >> v1:172.21.15.139:6789/0 conn(0x7f7c7c0cd610 legacy=0x7f7c7c0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.943+0000 7f7c7affd640 1 -- 172.21.15.139:0/1627015345 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963560125 0 0) 0x7f7c7c099ca0 con 0x7f7c7c0cd610 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.943+0000 7f7c8096b640 1 -- 172.21.15.139:0/1627015345 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c09a260 con 0x7f7c7c09ad40 2024-04-25T10:03:01.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.943+0000 7f7c7b7fe640 1 --2- 172.21.15.139:0/1627015345 >> v2:172.21.15.150:3300/0 conn(0x7f7c7c0d0a80 0x7f7c7c1618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.943+0000 7f7c8096b640 1 --2- 172.21.15.139:0/1627015345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c7c09ad40 0x7f7c7c161e00 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0725c0 tx=0x7f7c7402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92a6afa3b04eda1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.943+0000 7f7c7affd640 1 -- 172.21.15.139:0/1627015345 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74007740 con 0x7f7c7c09ad40 2024-04-25T10:03:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.943+0000 7f7c7affd640 1 -- 172.21.15.139:0/1627015345 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c740078e0 con 0x7f7c7c09ad40 2024-04-25T10:03:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.943+0000 7f7c7affd640 1 -- 172.21.15.139:0/1627015345 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74007cb0 con 0x7f7c7c09ad40 2024-04-25T10:03:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.944+0000 7f7c823ce640 1 -- 172.21.15.139:0/1627015345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c7c09ad40 msgr2=0x7f7c7c161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.944+0000 7f7c823ce640 1 --2- 172.21.15.139:0/1627015345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c7c09ad40 0x7f7c7c161e00 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0725c0 tx=0x7f7c7402a710 comp rx=0 tx=0).stop 2024-04-25T10:03:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.944+0000 7f7c823ce640 1 -- 172.21.15.139:0/1627015345 shutdown_connections 2024-04-25T10:03:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.944+0000 7f7c823ce640 1 --2- 172.21.15.139:0/1627015345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c7c09ad40 0x7f7c7c161e00 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.944+0000 7f7c823ce640 1 --2- 172.21.15.139:0/1627015345 >> v2:172.21.15.150:3300/0 conn(0x7f7c7c0d0a80 0x7f7c7c1618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.944+0000 7f7c823ce640 1 -- 172.21.15.139:0/1627015345 >> 172.21.15.139:0/1627015345 conn(0x7f7c7c0b31f0 msgr2=0x7f7c7c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.944+0000 7f7c823ce640 1 -- 172.21.15.139:0/1627015345 shutdown_connections 2024-04-25T10:03:01.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.944+0000 7f7c823ce640 1 -- 172.21.15.139:0/1627015345 wait complete. 2024-04-25T10:03:01.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.945+0000 7f7c823ce640 1 Processor -- start 2024-04-25T10:03:01.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.945+0000 7f7c823ce640 1 -- start start 2024-04-25T10:03:01.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c823ce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c7c09ad40 0x7f7c7c15ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c823ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c7c0cd610 0x7f7c7c15ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c823ce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c7c0d0a80 0x7f7c7c15d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c823ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0cd9f0 con 0x7f7c7c0cd610 2024-04-25T10:03:01.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c823ce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c09a820 con 0x7f7c7c09ad40 2024-04-25T10:03:01.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c823ce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0cdb70 con 0x7f7c7c0d0a80 2024-04-25T10:03:01.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c7b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c7c0cd610 0x7f7c7c15ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c7b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c7c0cd610 0x7f7c7c15ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45884/0 (socket says 172.21.15.139:45884) 2024-04-25T10:03:01.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c7b7fe640 1 -- 172.21.15.139:0/3523823778 learned_addr learned my addr 172.21.15.139:0/3523823778 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:01.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c7b7fe640 1 -- 172.21.15.139:0/3523823778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c7c0d0a80 msgr2=0x7f7c7c15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c7b7fe640 1 --2- 172.21.15.139:0/3523823778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c7c0d0a80 0x7f7c7c15d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c7b7fe640 1 -- 172.21.15.139:0/3523823778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c7c09ad40 msgr2=0x7f7c7c15ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c7b7fe640 1 --2- 172.21.15.139:0/3523823778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c7c09ad40 0x7f7c7c15ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c7b7fe640 1 -- 172.21.15.139:0/3523823778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c1f36a0 con 0x7f7c7c0cd610 2024-04-25T10:03:01.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.946+0000 7f7c7b7fe640 1 --2- 172.21.15.139:0/3523823778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c7c0cd610 0x7f7c7c15ce80 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c820 tx=0x7f7c7000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.947+0000 7f7c78ff9640 1 -- 172.21.15.139:0/3523823778 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70015070 con 0x7f7c7c0cd610 2024-04-25T10:03:01.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.947+0000 7f7c78ff9640 1 -- 172.21.15.139:0/3523823778 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c70007a30 con 0x7f7c7c0cd610 2024-04-25T10:03:01.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.947+0000 7f7c823ce640 1 -- 172.21.15.139:0/3523823778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7c1f3980 con 0x7f7c7c0cd610 2024-04-25T10:03:01.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.947+0000 7f7c78ff9640 1 -- 172.21.15.139:0/3523823778 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70007de0 con 0x7f7c7c0cd610 2024-04-25T10:03:01.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.947+0000 7f7c823ce640 1 -- 172.21.15.139:0/3523823778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7c1f3f40 con 0x7f7c7c0cd610 2024-04-25T10:03:01.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.948+0000 7f7c823ce640 1 -- 172.21.15.139:0/3523823778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c7c074520 con 0x7f7c7c0cd610 2024-04-25T10:03:01.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.948+0000 7f7c78ff9640 1 -- 172.21.15.139:0/3523823778 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c70018070 con 0x7f7c7c0cd610 2024-04-25T10:03:01.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.948+0000 7f7c78ff9640 1 --2- 172.21.15.139:0/3523823778 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7c6806b4d0 0x7f7c6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.948+0000 7f7c78ff9640 1 -- 172.21.15.139:0/3523823778 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7c7008a2c0 con 0x7f7c7c0cd610 2024-04-25T10:03:01.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.949+0000 7f7c7bfff640 1 --2- 172.21.15.139:0/3523823778 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7c6806b4d0 0x7f7c6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.949+0000 7f7c7bfff640 1 --2- 172.21.15.139:0/3523823778 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7c6806b4d0 0x7f7c6806d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c007fd0 tx=0x7f7c6c007920 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:01.953+0000 7f7c78ff9640 1 -- 172.21.15.139:0/3523823778 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c70058fb0 con 0x7f7c7c0cd610 2024-04-25T10:03:02.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.124+0000 7f7c823ce640 1 -- 172.21.15.139:0/3523823778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7c7c070e30 con 0x7f7c7c0cd610 2024-04-25T10:03:02.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.125+0000 7f7c78ff9640 1 -- 172.21.15.139:0/3523823778 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7c700588b0 con 0x7f7c7c0cd610 2024-04-25T10:03:02.130 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T10:03:02.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.127+0000 7f7c823ce640 1 -- 172.21.15.139:0/3523823778 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7c6806b4d0 msgr2=0x7f7c6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.127+0000 7f7c823ce640 1 --2- 172.21.15.139:0/3523823778 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7c6806b4d0 0x7f7c6806d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c007fd0 tx=0x7f7c6c007920 comp rx=0 tx=0).stop 2024-04-25T10:03:02.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.127+0000 7f7c823ce640 1 -- 172.21.15.139:0/3523823778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c7c0cd610 msgr2=0x7f7c7c15ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.127+0000 7f7c823ce640 1 --2- 172.21.15.139:0/3523823778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c7c0cd610 0x7f7c7c15ce80 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c820 tx=0x7f7c7000cce0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.127+0000 7f7c823ce640 1 -- 172.21.15.139:0/3523823778 shutdown_connections 2024-04-25T10:03:02.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.127+0000 7f7c823ce640 1 --2- 172.21.15.139:0/3523823778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c7c0cd610 0x7f7c7c15ce80 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.127+0000 7f7c823ce640 1 --2- 172.21.15.139:0/3523823778 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7c6806b4d0 0x7f7c6806d990 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.127+0000 7f7c823ce640 1 --2- 172.21.15.139:0/3523823778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c7c0d0a80 0x7f7c7c15d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.127+0000 7f7c823ce640 1 --2- 172.21.15.139:0/3523823778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c7c09ad40 0x7f7c7c15ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.127+0000 7f7c823ce640 1 -- 172.21.15.139:0/3523823778 >> 172.21.15.139:0/3523823778 conn(0x7f7c7c0b31f0 msgr2=0x7f7c7c15fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.127+0000 7f7c823ce640 1 -- 172.21.15.139:0/3523823778 shutdown_connections 2024-04-25T10:03:02.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.127+0000 7f7c823ce640 1 -- 172.21.15.139:0/3523823778 wait complete. 2024-04-25T10:03:02.142 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:02.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.236+0000 7f80dc914640 1 Processor -- start 2024-04-25T10:03:02.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.236+0000 7f80dc914640 1 -- start start 2024-04-25T10:03:02.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80dc914640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f80d80d0a80 0x7f80d81618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80dc914640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d8099ca0 con 0x7f80d80cd610 2024-04-25T10:03:02.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80dc914640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d8099f80 con 0x7f80d809ad40 2024-04-25T10:03:02.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80dc914640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d8099a50 con 0x7f80d80d0a80 2024-04-25T10:03:02.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d659c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f80d80cd610 0x7f80d80d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48704/0 (socket says 172.21.15.139:48704) 2024-04-25T10:03:02.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d659c640 1 -- 172.21.15.139:0/1227031259 learned_addr learned my addr 172.21.15.139:0/1227031259 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:02.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035581458 0 0) 0x7f80d8099ca0 con 0x7f80d80cd610 2024-04-25T10:03:02.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c4003520 con 0x7f80d80cd610 2024-04-25T10:03:02.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80cffff640 1 --2- 172.21.15.139:0/1227031259 >> v2:172.21.15.150:3300/0 conn(0x7f80d80d0a80 0x7f80d81618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2433344720 0 0) 0x7f80c4003520 con 0x7f80d80cd610 2024-04-25T10:03:02.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 >> v2:172.21.15.150:3300/0 conn(0x7f80d80d0a80 msgr2=0x7f80d81618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 --2- 172.21.15.139:0/1227031259 >> v2:172.21.15.150:3300/0 conn(0x7f80d80d0a80 0x7f80d81618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 >> v1:172.21.15.150:6789/0 conn(0x7f80d809ad40 legacy=0x7f80d80cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d809a260 con 0x7f80d80cd610 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f80c8001670 con 0x7f80d80cd610 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 >> v1:172.21.15.139:6789/0 conn(0x7f80d80cd610 legacy=0x7f80d80d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 --2- 172.21.15.139:0/1227031259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 0x7f80c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 --2- 172.21.15.139:0/1227031259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80c4004800 0x7f80c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 --2- 172.21.15.139:0/1227031259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80c40051c0 0x7f80c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8099a50 con 0x7f80c40039b0 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c400c070 con 0x7f80c40051c0 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.237+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c400c270 con 0x7f80c4004800 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.238+0000 7f80d659c640 1 --2- 172.21.15.139:0/1227031259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 0x7f80c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.238+0000 7f80d6d9d640 1 --2- 172.21.15.139:0/1227031259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80c40051c0 0x7f80c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.238+0000 7f80cffff640 1 --2- 172.21.15.139:0/1227031259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80c4004800 0x7f80c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.238+0000 7f80d659c640 1 -- 172.21.15.139:0/1227031259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80c4004800 msgr2=0x7f80c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.238+0000 7f80d659c640 1 --2- 172.21.15.139:0/1227031259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80c4004800 0x7f80c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.238+0000 7f80d659c640 1 -- 172.21.15.139:0/1227031259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80c40051c0 msgr2=0x7f80c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.238+0000 7f80d659c640 1 --2- 172.21.15.139:0/1227031259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80c40051c0 0x7f80c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.238+0000 7f80d6d9d640 1 -- 172.21.15.139:0/1227031259 reap_dead start 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.238+0000 7f80d659c640 1 -- 172.21.15.139:0/1227031259 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c4003520 con 0x7f80c40039b0 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.238+0000 7f80d659c640 1 --2- 172.21.15.139:0/1227031259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 0x7f80c4003dd0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f80c800c290 tx=0x7f80c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327002b9e5c6aecd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.239+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8007d30 con 0x7f80c40039b0 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.239+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c80058d0 con 0x7f80c40039b0 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.239+0000 7f80d5d9b640 1 -- 172.21.15.139:0/1227031259 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8005ce0 con 0x7f80c40039b0 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.239+0000 7f80dc914640 1 -- 172.21.15.139:0/1227031259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 msgr2=0x7f80c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.239+0000 7f80dc914640 1 --2- 172.21.15.139:0/1227031259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 0x7f80c4003dd0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f80c800c290 tx=0x7f80c4005830 comp rx=0 tx=0).stop 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.239+0000 7f80dc914640 1 -- 172.21.15.139:0/1227031259 shutdown_connections 2024-04-25T10:03:02.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.239+0000 7f80dc914640 1 --2- 172.21.15.139:0/1227031259 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 0x7f80c4003dd0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.239+0000 7f80dc914640 1 -- 172.21.15.139:0/1227031259 >> 172.21.15.139:0/1227031259 conn(0x7f80d80b31f0 msgr2=0x7f80d80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.239+0000 7f80dc914640 1 -- 172.21.15.139:0/1227031259 shutdown_connections 2024-04-25T10:03:02.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.239+0000 7f80dc914640 1 -- 172.21.15.139:0/1227031259 wait complete. 2024-04-25T10:03:02.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.240+0000 7f80dc914640 1 Processor -- start 2024-04-25T10:03:02.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.241+0000 7f80dc914640 1 -- start start 2024-04-25T10:03:02.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.241+0000 7f80dc914640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 0x7f80d814daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.241+0000 7f80dc914640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80d80b66f0 0x7f80d80b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.241+0000 7f80dc914640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80d80b4300 0x7f80d80b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.241+0000 7f80dc914640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80cd9f0 con 0x7f80c40039b0 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.241+0000 7f80dc914640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d809a820 con 0x7f80d80b66f0 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.241+0000 7f80dc914640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80cdb70 con 0x7f80d80b4300 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.241+0000 7f80d659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 0x7f80d814daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.241+0000 7f80d659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 0x7f80d814daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45902/0 (socket says 172.21.15.139:45902) 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.241+0000 7f80d659c640 1 -- 172.21.15.139:0/2719659916 learned_addr learned my addr 172.21.15.139:0/2719659916 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.242+0000 7f80d659c640 1 -- 172.21.15.139:0/2719659916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80d80b4300 msgr2=0x7f80d80b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.242+0000 7f80d659c640 1 --2- 172.21.15.139:0/2719659916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80d80b4300 0x7f80d80b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.242+0000 7f80d659c640 1 -- 172.21.15.139:0/2719659916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80d80b66f0 msgr2=0x7f80d80b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.242+0000 7f80d659c640 1 --2- 172.21.15.139:0/2719659916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80d80b66f0 0x7f80d80b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.242+0000 7f80d659c640 1 -- 172.21.15.139:0/2719659916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d80b4ff0 con 0x7f80c40039b0 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.242+0000 7f80d659c640 1 --2- 172.21.15.139:0/2719659916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 0x7f80d814daf0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f80c800c290 tx=0x7f80c8000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.242+0000 7f80ceffd640 1 -- 172.21.15.139:0/2719659916 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8001c80 con 0x7f80c40039b0 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.242+0000 7f80ceffd640 1 -- 172.21.15.139:0/2719659916 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c8041070 con 0x7f80c40039b0 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.242+0000 7f80dc914640 1 -- 172.21.15.139:0/2719659916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d80b52d0 con 0x7f80c40039b0 2024-04-25T10:03:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.242+0000 7f80ceffd640 1 -- 172.21.15.139:0/2719659916 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8005e30 con 0x7f80c40039b0 2024-04-25T10:03:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.242+0000 7f80dc914640 1 -- 172.21.15.139:0/2719659916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d80b5890 con 0x7f80c40039b0 2024-04-25T10:03:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.243+0000 7f80dc914640 1 -- 172.21.15.139:0/2719659916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d8074520 con 0x7f80c40039b0 2024-04-25T10:03:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.243+0000 7f80ceffd640 1 -- 172.21.15.139:0/2719659916 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80c803b070 con 0x7f80c40039b0 2024-04-25T10:03:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.244+0000 7f80ceffd640 1 --2- 172.21.15.139:0/2719659916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f80b806b4d0 0x7f80b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.244+0000 7f80ceffd640 1 -- 172.21.15.139:0/2719659916 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80c80a9e10 con 0x7f80c40039b0 2024-04-25T10:03:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.244+0000 7f80cffff640 1 --2- 172.21.15.139:0/2719659916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f80b806b4d0 0x7f80b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.244+0000 7f80cffff640 1 --2- 172.21.15.139:0/2719659916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f80b806b4d0 0x7f80b806d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f80d81611d0 tx=0x7f80c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.248+0000 7f80ceffd640 1 -- 172.21.15.139:0/2719659916 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80c8078a70 con 0x7f80c40039b0 2024-04-25T10:03:02.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.428+0000 7f80dc914640 1 -- 172.21.15.139:0/2719659916 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f80d80b3f70 con 0x7f80c40039b0 2024-04-25T10:03:02.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.428+0000 7f80ceffd640 1 -- 172.21.15.139:0/2719659916 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v120) v1 ==== 80+0+0 (secure 0 0 0) 0x7f80c8078370 con 0x7f80c40039b0 2024-04-25T10:03:02.433 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-04-25T10:03:02.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.430+0000 7f80dc914640 1 -- 172.21.15.139:0/2719659916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f80b806b4d0 msgr2=0x7f80b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.430+0000 7f80dc914640 1 --2- 172.21.15.139:0/2719659916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f80b806b4d0 0x7f80b806d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f80d81611d0 tx=0x7f80c0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:02.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.430+0000 7f80dc914640 1 -- 172.21.15.139:0/2719659916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 msgr2=0x7f80d814daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.430+0000 7f80dc914640 1 --2- 172.21.15.139:0/2719659916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 0x7f80d814daf0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f80c800c290 tx=0x7f80c8000fe0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.430+0000 7f80dc914640 1 -- 172.21.15.139:0/2719659916 shutdown_connections 2024-04-25T10:03:02.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.430+0000 7f80dc914640 1 --2- 172.21.15.139:0/2719659916 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f80b806b4d0 0x7f80b806d990 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.430+0000 7f80dc914640 1 --2- 172.21.15.139:0/2719659916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80d80b4300 0x7f80d80b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.430+0000 7f80dc914640 1 --2- 172.21.15.139:0/2719659916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80d80b66f0 0x7f80d80b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.430+0000 7f80dc914640 1 --2- 172.21.15.139:0/2719659916 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80c40039b0 0x7f80d814daf0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.430+0000 7f80dc914640 1 -- 172.21.15.139:0/2719659916 >> 172.21.15.139:0/2719659916 conn(0x7f80d80b31f0 msgr2=0x7f80d80cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.430+0000 7f80dc914640 1 -- 172.21.15.139:0/2719659916 shutdown_connections 2024-04-25T10:03:02.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.430+0000 7f80dc914640 1 -- 172.21.15.139:0/2719659916 wait complete. 2024-04-25T10:03:02.446 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:03:02.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:02.446 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:02.514 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:03:02.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:02.514 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:03:02.539 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:02.539 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:02.598 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:03:02.598 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:03:02.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:02.598 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:03:02.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:03:02.653 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:03:02.654 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:03:02.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.787+0000 7f7610109640 1 Processor -- start 2024-04-25T10:03:02.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.787+0000 7f7610109640 1 -- start start 2024-04-25T10:03:02.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f7610109640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76080d0a80 0x7f76081618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f7610109640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f760809ad40 0x7f7608161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f7610109640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7608099ca0 con 0x7f76080cd610 2024-04-25T10:03:02.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f7610109640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7608099a50 con 0x7f760809ad40 2024-04-25T10:03:02.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f7610109640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7608099f80 con 0x7f76080d0a80 2024-04-25T10:03:02.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f760e6a6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f760809ad40 0x7f7608161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f760dea5640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f76080cd610 0x7f76080d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48712/0 (socket says 172.21.15.139:48712) 2024-04-25T10:03:02.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f760dea5640 1 -- 172.21.15.139:0/4142814573 learned_addr learned my addr 172.21.15.139:0/4142814573 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:02.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f760d6a4640 1 --2- 172.21.15.139:0/4142814573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76080d0a80 0x7f76081618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f760e6a6640 1 -- 172.21.15.139:0/4142814573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76080d0a80 msgr2=0x7f76081618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f760e6a6640 1 --2- 172.21.15.139:0/4142814573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76080d0a80 0x7f76081618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f760e6a6640 1 -- 172.21.15.139:0/4142814573 >> v1:172.21.15.139:6789/0 conn(0x7f76080cd610 legacy=0x7f76080d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f760e6a6640 1 -- 172.21.15.139:0/4142814573 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760809a260 con 0x7f760809ad40 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.788+0000 7f760cea3640 1 -- 172.21.15.139:0/4142814573 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418504120 0 0) 0x7f7608099ca0 con 0x7f76080cd610 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.789+0000 7f760e6a6640 1 --2- 172.21.15.139:0/4142814573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760809ad40 0x7f7608161e00 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f7600009d90 tx=0x7f760002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56eae74e544567f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.789+0000 7f760cea3640 1 -- 172.21.15.139:0/4142814573 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600007740 con 0x7f760809ad40 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.789+0000 7f760cea3640 1 -- 172.21.15.139:0/4142814573 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76000078e0 con 0x7f760809ad40 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.789+0000 7f760cea3640 1 -- 172.21.15.139:0/4142814573 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600007cb0 con 0x7f760809ad40 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.789+0000 7f7610109640 1 -- 172.21.15.139:0/4142814573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760809ad40 msgr2=0x7f7608161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.789+0000 7f7610109640 1 --2- 172.21.15.139:0/4142814573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760809ad40 0x7f7608161e00 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f7600009d90 tx=0x7f760002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.789+0000 7f7610109640 1 -- 172.21.15.139:0/4142814573 shutdown_connections 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.789+0000 7f7610109640 1 --2- 172.21.15.139:0/4142814573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760809ad40 0x7f7608161e00 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.789+0000 7f7610109640 1 --2- 172.21.15.139:0/4142814573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76080d0a80 0x7f76081618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.790+0000 7f7610109640 1 -- 172.21.15.139:0/4142814573 >> 172.21.15.139:0/4142814573 conn(0x7f76080b31f0 msgr2=0x7f76080b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.790+0000 7f7610109640 1 -- 172.21.15.139:0/4142814573 shutdown_connections 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.790+0000 7f7610109640 1 -- 172.21.15.139:0/4142814573 wait complete. 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.791+0000 7f7610109640 1 Processor -- start 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.791+0000 7f7610109640 1 -- start start 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.791+0000 7f7610109640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f760809ad40 0x7f76080c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.791+0000 7f7610109640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76080cd610 0x7f76080c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.791+0000 7f7610109640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76080d0a80 0x7f760809ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.791+0000 7f7610109640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76080cd9f0 con 0x7f76080d0a80 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.791+0000 7f7610109640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760809a820 con 0x7f76080cd610 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.791+0000 7f7610109640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76080cdb70 con 0x7f760809ad40 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f760e6a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76080d0a80 0x7f760809ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f760e6a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76080d0a80 0x7f760809ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45918/0 (socket says 172.21.15.139:45918) 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f760e6a6640 1 -- 172.21.15.139:0/4207195421 learned_addr learned my addr 172.21.15.139:0/4207195421 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f760dea5640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f760809ad40 0x7f76080c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f760d6a4640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76080cd610 0x7f76080c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f760e6a6640 1 -- 172.21.15.139:0/4207195421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f760809ad40 msgr2=0x7f76080c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f760e6a6640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f760809ad40 0x7f76080c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f760e6a6640 1 -- 172.21.15.139:0/4207195421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76080cd610 msgr2=0x7f76080c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f760e6a6640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76080cd610 0x7f76080c5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f760e6a6640 1 -- 172.21.15.139:0/4207195421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760809d980 con 0x7f76080d0a80 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f760e6a6640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76080d0a80 0x7f760809ce30 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f7600009d90 tx=0x7f7600004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f75fe7fc640 1 -- 172.21.15.139:0/4207195421 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600040070 con 0x7f76080d0a80 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f75fe7fc640 1 -- 172.21.15.139:0/4207195421 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7600031bc0 con 0x7f76080d0a80 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f75fe7fc640 1 -- 172.21.15.139:0/4207195421 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760003c420 con 0x7f76080d0a80 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f7610109640 1 -- 172.21.15.139:0/4207195421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760809dc60 con 0x7f76080d0a80 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.792+0000 7f7610109640 1 -- 172.21.15.139:0/4207195421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760809e220 con 0x7f76080d0a80 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.793+0000 7f7610109640 1 -- 172.21.15.139:0/4207195421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760806fb10 con 0x7f76080d0a80 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.794+0000 7f75fe7fc640 1 -- 172.21.15.139:0/4207195421 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7600039040 con 0x7f76080d0a80 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.794+0000 7f75fe7fc640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f75ec06b4d0 0x7f75ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.794+0000 7f75fe7fc640 1 -- 172.21.15.139:0/4207195421 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f76000ab800 con 0x7f76080d0a80 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.794+0000 7f760dea5640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f75ec06b4d0 0x7f75ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.795+0000 7f760dea5640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f75ec06b4d0 0x7f75ec06d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7604000f60 tx=0x7f7604007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:02.798+0000 7f75fe7fc640 1 -- 172.21.15.139:0/4207195421 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f760806fb10 con 0x7f76080d0a80 2024-04-25T10:03:03.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.004+0000 7f7610109640 1 -- 172.21.15.139:0/4207195421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7608074520 con 0x7f76080d0a80 2024-04-25T10:03:03.009 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T10:03:03.005+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:03:03.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.006+0000 7f75fe7fc640 1 -- 172.21.15.139:0/4207195421 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7600079ea0 con 0x7f76080d0a80 2024-04-25T10:03:03.011 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:03:03.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.009+0000 7f7610109640 1 -- 172.21.15.139:0/4207195421 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f75ec06b4d0 msgr2=0x7f75ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.009+0000 7f7610109640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f75ec06b4d0 0x7f75ec06d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7604000f60 tx=0x7f7604007d50 comp rx=0 tx=0).stop 2024-04-25T10:03:03.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.009+0000 7f7610109640 1 -- 172.21.15.139:0/4207195421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76080d0a80 msgr2=0x7f760809ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.009+0000 7f7610109640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76080d0a80 0x7f760809ce30 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f7600009d90 tx=0x7f7600004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:03.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.009+0000 7f7610109640 1 -- 172.21.15.139:0/4207195421 shutdown_connections 2024-04-25T10:03:03.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.009+0000 7f7610109640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f76080d0a80 0x7f760809ce30 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.009+0000 7f7610109640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f75ec06b4d0 0x7f75ec06d990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.009+0000 7f7610109640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76080cd610 0x7f76080c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.009+0000 7f7610109640 1 --2- 172.21.15.139:0/4207195421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f760809ad40 0x7f76080c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.009+0000 7f7610109640 1 -- 172.21.15.139:0/4207195421 >> 172.21.15.139:0/4207195421 conn(0x7f76080b31f0 msgr2=0x7f760806f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.009+0000 7f7610109640 1 -- 172.21.15.139:0/4207195421 shutdown_connections 2024-04-25T10:03:03.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.009+0000 7f7610109640 1 -- 172.21.15.139:0/4207195421 wait complete. 2024-04-25T10:03:03.021 INFO:tasks.ceph.mds.d.smithi150.stderr: -5207> 2024-04-25T10:02:56.799+0000 7f1fc0ae9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:03.026 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:03:03.029 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:03:03.053 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2024-04-25T10:03:03.067 INFO:teuthology.orchestra.run.smithi139.stdout:[global] 2024-04-25T10:03:03.067 INFO:teuthology.orchestra.run.smithi139.stdout: chdir = "" 2024-04-25T10:03:03.067 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:03:03.067 INFO:teuthology.orchestra.run.smithi139.stdout: auth supported = cephx 2024-04-25T10:03:03.067 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.067 INFO:teuthology.orchestra.run.smithi139.stdout: filestore xattr use omap = true 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: mon clock drift allowed = 1.000 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: osd crush chooseleaf type = 0 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: auth debug = true 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on old message = true 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on bug = true 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: mon pg warn max object skew = 0 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.068 INFO:teuthology.orchestra.run.smithi139.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default size = 2 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow primary affinity = true 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow pg remap = true 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on no sortbitwise = false 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on too few osds = false 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi139.stdout: mon_allow_pool_size_one = true 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: osd default data pool replay window = 5 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: mon allow pool delete = true 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: mon cluster log file level = debug 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: debug asserts on shutdown = true 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: mon health detail to clog = false 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: mon host = "172.21.15.139,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on skipped message = False 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: fsid = 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T10:03:03.070 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout:[osd] 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: osd journal size = 100 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub load threshold = 5.0 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub max interval = 600 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: osd recover clone overlap = true 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: osd recovery max chunk = 1048576 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug shutdown = true 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug op order = true 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug verify stray on activate = true 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug trim objects = true 2024-04-25T10:03:03.071 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: osd open classes on start = true 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug pg log writeout = true 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: osd map max advance = 10 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: journal zero on create = true 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: filestore apply finisher threads = 3 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: bdev debug aio = true 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug misdirected ops = true 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore block size = 96636764160 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore fsck on mount = True 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluefs = 20 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluestore = 20 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: debug osd = 20 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: debug rocksdb = 10 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd full ratio = 0.9 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: osd objectstore = bluestore 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.073 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr] 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: debug mgr = 20 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: mgr/telemetry/nag = false 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout:[mon] 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: debug paxos = 20 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-04-25T10:03:03.074 INFO:teuthology.orchestra.run.smithi139.stdout: mon data avail warn = 5 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: mon mgr mkfs grace = 240 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd reporter subtree level = osd 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd prime pg temp = true 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:03:03.075 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: # 1m isn't quite enough 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: mon op complaint time = 120 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout:[client] 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: rgw cache enabled = true 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable ops log = true 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable usage log = true 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:03:03.076 INFO:teuthology.orchestra.run.smithi139.stdout: client mount timeout = 600 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout: debug client = 20 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = "" 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.a] 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.b] 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.c] 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2024-04-25T10:03:03.077 INFO:teuthology.orchestra.run.smithi139.stdout:[mds] 2024-04-25T10:03:03.078 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds = 20 2024-04-25T10:03:03.078 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds balancer = 20 2024-04-25T10:03:03.078 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:03.078 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug frag = True 2024-04-25T10:03:03.078 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug scatterstat = True 2024-04-25T10:03:03.078 INFO:teuthology.orchestra.run.smithi139.stdout: mds op complaint time = 180 2024-04-25T10:03:03.078 INFO:teuthology.orchestra.run.smithi139.stdout: mds verify scatter = False 2024-04-25T10:03:03.078 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-04-25T10:03:03.078 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-04-25T10:03:03.078 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-04-25T10:03:03.079 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-04-25T10:03:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2024-04-25T10:03:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:03:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2024-04-25T10:03:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2024-04-25T10:03:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.079 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2024-04-25T10:03:03.080 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.081 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.139,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on skipped message = False 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2024-04-25T10:03:03.082 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug trim objects = true 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:03:03.083 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 20 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 20 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:03.084 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 10 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:03.085 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.086 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: mon op complaint time = 120 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.087 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: client mount timeout = 600 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2024-04-25T10:03:03.088 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout:[mds] 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds = 20 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds balancer = 20 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug frag = True 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug scatterstat = True 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: mds op complaint time = 180 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: mds verify scatter = False 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2024-04-25T10:03:03.089 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2024-04-25T10:03:03.092 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:03:03.133 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:03:03.171 INFO:teuthology.orchestra.run.smithi139.stdout:[global] 2024-04-25T10:03:03.171 INFO:teuthology.orchestra.run.smithi139.stdout: chdir = "" 2024-04-25T10:03:03.171 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:03:03.171 INFO:teuthology.orchestra.run.smithi139.stdout: auth supported = cephx 2024-04-25T10:03:03.171 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.171 INFO:teuthology.orchestra.run.smithi139.stdout: filestore xattr use omap = true 2024-04-25T10:03:03.171 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: mon clock drift allowed = 1.000 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: osd crush chooseleaf type = 0 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: auth debug = true 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on old message = true 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on bug = true 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: mon pg warn max object skew = 0 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:03:03.172 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default size = 2 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow primary affinity = true 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow pg remap = true 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on no sortbitwise = false 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on too few osds = false 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: mon_allow_pool_size_one = true 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi139.stdout: osd default data pool replay window = 5 2024-04-25T10:03:03.174 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.174 INFO:teuthology.orchestra.run.smithi139.stdout: mon allow pool delete = true 2024-04-25T10:03:03.174 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.174 INFO:teuthology.orchestra.run.smithi139.stdout: mon cluster log file level = debug 2024-04-25T10:03:03.174 INFO:teuthology.orchestra.run.smithi139.stdout: debug asserts on shutdown = true 2024-04-25T10:03:03.174 INFO:teuthology.orchestra.run.smithi139.stdout: mon health detail to clog = false 2024-04-25T10:03:03.174 INFO:teuthology.orchestra.run.smithi139.stdout: mon host = "172.21.15.139,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2024-04-25T10:03:03.174 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on skipped message = False 2024-04-25T10:03:03.174 INFO:teuthology.orchestra.run.smithi139.stdout: fsid = 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T10:03:03.174 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.174 INFO:teuthology.orchestra.run.smithi139.stdout:[osd] 2024-04-25T10:03:03.174 INFO:teuthology.orchestra.run.smithi139.stdout: osd journal size = 100 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub load threshold = 5.0 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub max interval = 600 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: osd recover clone overlap = true 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: osd recovery max chunk = 1048576 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug shutdown = true 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug op order = true 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug verify stray on activate = true 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug trim objects = true 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: osd open classes on start = true 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug pg log writeout = true 2024-04-25T10:03:03.175 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: osd map max advance = 10 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: journal zero on create = true 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: filestore apply finisher threads = 3 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: bdev debug aio = true 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug misdirected ops = true 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore block size = 96636764160 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore fsck on mount = True 2024-04-25T10:03:03.176 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluefs = 20 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluestore = 20 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: debug osd = 20 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: debug rocksdb = 10 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd full ratio = 0.9 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: osd objectstore = bluestore 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr] 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: debug mgr = 20 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-04-25T10:03:03.177 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: mgr/telemetry/nag = false 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout:[mon] 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: debug paxos = 20 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: mon data avail warn = 5 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: mon mgr mkfs grace = 240 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd reporter subtree level = osd 2024-04-25T10:03:03.178 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd prime pg temp = true 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: # 1m isn't quite enough 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout: mon op complaint time = 120 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout:[client] 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout: rgw cache enabled = true 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable ops log = true 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable usage log = true 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout: client mount timeout = 600 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout: debug client = 20 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = "" 2024-04-25T10:03:03.180 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.a] 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.b] 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.c] 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout:[mds] 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds = 20 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds balancer = 20 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug frag = True 2024-04-25T10:03:03.181 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug scatterstat = False 2024-04-25T10:03:03.182 INFO:teuthology.orchestra.run.smithi139.stdout: mds op complaint time = 180 2024-04-25T10:03:03.182 INFO:teuthology.orchestra.run.smithi139.stdout: mds verify scatter = False 2024-04-25T10:03:03.182 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-04-25T10:03:03.182 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-04-25T10:03:03.182 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-04-25T10:03:03.182 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-04-25T10:03:03.182 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2024-04-25T10:03:03.182 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.183 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:03:03.184 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.139,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2024-04-25T10:03:03.185 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on skipped message = False 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug trim objects = true 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.187 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 20 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 20 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 10 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:03:03.188 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2024-04-25T10:03:03.189 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.190 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: mon op complaint time = 120 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:03:03.191 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout: client mount timeout = 600 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout:[mds] 2024-04-25T10:03:03.192 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds = 20 2024-04-25T10:03:03.193 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds balancer = 20 2024-04-25T10:03:03.193 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:03.193 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug frag = True 2024-04-25T10:03:03.193 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug scatterstat = False 2024-04-25T10:03:03.193 INFO:teuthology.orchestra.run.smithi150.stdout: mds op complaint time = 180 2024-04-25T10:03:03.193 INFO:teuthology.orchestra.run.smithi150.stdout: mds verify scatter = False 2024-04-25T10:03:03.193 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2024-04-25T10:03:03.193 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2024-04-25T10:03:03.193 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2024-04-25T10:03:03.197 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-25T10:03:03.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.272+0000 7f2b34120f40 1 Processor -- start 2024-04-25T10:03:03.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.272+0000 7f2b34120f40 1 -- start start 2024-04-25T10:03:03.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.272+0000 7f2b34120f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b200022a0 0x562cd7eaac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.272+0000 7f2b34120f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562cd7dcc2a0 0x562cd7ea30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.272+0000 7f2b34120f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 0x562cd7e97c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.272+0000 7f2b34120f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd7e15b40 con 0x562cd7e977b0 2024-04-25T10:03:03.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.272+0000 7f2b34120f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd7d74df0 con 0x7f2b200022a0 2024-04-25T10:03:03.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.272+0000 7f2b34120f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cd7e15de0 con 0x562cd7dcc2a0 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.272+0000 7f2b31b07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 0x562cd7e97c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.272+0000 7f2b31b07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 0x562cd7e97c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45930/0 (socket says 172.21.15.139:45930) 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b31b07640 1 -- 172.21.15.139:0/3191234371 learned_addr learned my addr 172.21.15.139:0/3191234371 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b31306640 1 --2- 172.21.15.139:0/3191234371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b200022a0 0x562cd7eaac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b30b05640 1 --2- 172.21.15.139:0/3191234371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562cd7dcc2a0 0x562cd7ea30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b31b07640 1 -- 172.21.15.139:0/3191234371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562cd7dcc2a0 msgr2=0x562cd7ea30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b31b07640 1 --2- 172.21.15.139:0/3191234371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562cd7dcc2a0 0x562cd7ea30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b31b07640 1 -- 172.21.15.139:0/3191234371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b200022a0 msgr2=0x562cd7eaac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b31b07640 1 --2- 172.21.15.139:0/3191234371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b200022a0 0x562cd7eaac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b31b07640 1 -- 172.21.15.139:0/3191234371 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cd7cfac50 con 0x562cd7e977b0 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b30b05640 1 --2- 172.21.15.139:0/3191234371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562cd7dcc2a0 0x562cd7ea30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b31b07640 1 --2- 172.21.15.139:0/3191234371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 0x562cd7e97c30 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00a5a0 tx=0x7f2b2c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8edcb78a3df8d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b2b7fe640 1 -- 172.21.15.139:0/3191234371 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c017030 con 0x562cd7e977b0 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b2b7fe640 1 -- 172.21.15.139:0/3191234371 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c012670 con 0x562cd7e977b0 2024-04-25T10:03:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b2b7fe640 1 -- 172.21.15.139:0/3191234371 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c012840 con 0x562cd7e977b0 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b34120f40 1 -- 172.21.15.139:0/3191234371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 msgr2=0x562cd7e97c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b34120f40 1 --2- 172.21.15.139:0/3191234371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 0x562cd7e97c30 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00a5a0 tx=0x7f2b2c005d80 comp rx=0 tx=0).stop 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b34120f40 1 -- 172.21.15.139:0/3191234371 shutdown_connections 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b34120f40 1 --2- 172.21.15.139:0/3191234371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 0x562cd7e97c30 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b34120f40 1 --2- 172.21.15.139:0/3191234371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562cd7dcc2a0 0x562cd7ea30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b34120f40 1 --2- 172.21.15.139:0/3191234371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b200022a0 0x562cd7eaac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.273+0000 7f2b34120f40 1 -- 172.21.15.139:0/3191234371 >> 172.21.15.139:0/3191234371 conn(0x562cd7dcdbe0 msgr2=0x562cd7dcf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.274+0000 7f2b34120f40 1 -- 172.21.15.139:0/3191234371 shutdown_connections 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.274+0000 7f2b34120f40 1 -- 172.21.15.139:0/3191234371 wait complete. 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.274+0000 7f2b34120f40 1 Processor -- start 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.274+0000 7f2b34120f40 1 -- start start 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b34120f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd7dcc2a0 0x562cd7e8da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b34120f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 0x562cd7ea03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b34120f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b200022a0 0x562cd7ea08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b34120f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd7dcfcd0 con 0x562cd7e977b0 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b34120f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd7e8e720 con 0x562cd7dcc2a0 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b34120f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cd7dcff70 con 0x7f2b200022a0 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b30b05640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 0x562cd7ea03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b30b05640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 0x562cd7ea03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45940/0 (socket says 172.21.15.139:45940) 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b30b05640 1 -- 172.21.15.139:0/2821128207 learned_addr learned my addr 172.21.15.139:0/2821128207 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b31306640 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd7dcc2a0 0x562cd7e8da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b31b07640 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b200022a0 0x562cd7ea08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b30b05640 1 -- 172.21.15.139:0/2821128207 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b200022a0 msgr2=0x562cd7ea08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b30b05640 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b200022a0 0x562cd7ea08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b30b05640 1 -- 172.21.15.139:0/2821128207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd7dcc2a0 msgr2=0x562cd7e8da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b30b05640 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562cd7dcc2a0 0x562cd7e8da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b30b05640 1 -- 172.21.15.139:0/2821128207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cd7eab4a0 con 0x562cd7e977b0 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b30b05640 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 0x562cd7ea03a0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009f20 tx=0x7f2b1c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b2a7fc640 1 -- 172.21.15.139:0/2821128207 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c01a020 con 0x562cd7e977b0 2024-04-25T10:03:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b2a7fc640 1 -- 172.21.15.139:0/2821128207 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c016440 con 0x562cd7e977b0 2024-04-25T10:03:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b34120f40 1 -- 172.21.15.139:0/2821128207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562cd7e995f0 con 0x562cd7e977b0 2024-04-25T10:03:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.275+0000 7f2b2a7fc640 1 -- 172.21.15.139:0/2821128207 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c0167f0 con 0x562cd7e977b0 2024-04-25T10:03:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.276+0000 7f2b34120f40 1 -- 172.21.15.139:0/2821128207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562cd7e99bb0 con 0x562cd7e977b0 2024-04-25T10:03:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.277+0000 7f2b2a7fc640 1 -- 172.21.15.139:0/2821128207 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b1c01d020 con 0x562cd7e977b0 2024-04-25T10:03:03.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.277+0000 7f2b2a7fc640 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b14069eb0 0x7f2b1406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.277+0000 7f2b2a7fc640 1 -- 172.21.15.139:0/2821128207 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2b1c089620 con 0x562cd7e977b0 2024-04-25T10:03:03.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.277+0000 7f2b31306640 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b14069eb0 0x7f2b1406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.277+0000 7f2b34120f40 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x562cd7e8e8d0 0x562cd7e8f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.277+0000 7f2b34120f40 1 -- 172.21.15.139:0/2821128207 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v9 -- 0x562cd7e8fcb0 con 0x562cd7e8e8d0 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.277+0000 7f2b31b07640 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x562cd7e8e8d0 0x562cd7e8f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.278+0000 7f2b31306640 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b14069eb0 0x7f2b1406c370 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2b24056000 tx=0x7f2b24050000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.278+0000 7f2b31b07640 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x562cd7e8e8d0 0x562cd7e8f770 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.279+0000 7f2b31b07640 1 -- 172.21.15.139:0/2821128207 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v121'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2b2c012bc0 con 0x562cd7e8e8d0 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.279+0000 7f2b34120f40 1 -- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x562cd7e8e8d0 msgr2=0x562cd7e8f770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.279+0000 7f2b34120f40 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x562cd7e8e8d0 0x562cd7e8f770 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.279+0000 7f2b34120f40 1 -- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b14069eb0 msgr2=0x7f2b1406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.279+0000 7f2b34120f40 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f2b14069eb0 0x7f2b1406c370 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2b24056000 tx=0x7f2b24050000 comp rx=0 tx=0).stop 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.279+0000 7f2b34120f40 1 -- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 msgr2=0x562cd7ea03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.279+0000 7f2b34120f40 1 --2- 172.21.15.139:0/2821128207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562cd7e977b0 0x562cd7ea03a0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009f20 tx=0x7f2b1c009f50 comp rx=0 tx=0).stop 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.279+0000 7f2b31b07640 1 -- 172.21.15.139:0/2821128207 reap_dead start 2024-04-25T10:03:03.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.279+0000 7f2b34120f40 1 -- 172.21.15.139:0/2821128207 shutdown_connections 2024-04-25T10:03:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.279+0000 7f2b34120f40 1 -- 172.21.15.139:0/2821128207 >> 172.21.15.139:0/2821128207 conn(0x562cd7dcdbe0 msgr2=0x562cd7e9fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.279+0000 7f2b34120f40 1 -- 172.21.15.139:0/2821128207 shutdown_connections 2024-04-25T10:03:03.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.279+0000 7f2b34120f40 1 -- 172.21.15.139:0/2821128207 wait complete. 2024-04-25T10:03:03.289 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:03:03.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.421+0000 7f34b0d90640 1 Processor -- start 2024-04-25T10:03:03.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.422+0000 7f34b0d90640 1 -- start start 2024-04-25T10:03:03.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.422+0000 7f34b0d90640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34ac15dbf0 0x7f34ac1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.422+0000 7f34b0d90640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f34ac0b9370 0x7f34ac0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.422+0000 7f34b0d90640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ac161dc0 con 0x7f34ac0cec80 2024-04-25T10:03:03.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.422+0000 7f34b0d90640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ac0cf0d0 con 0x7f34ac0b9370 2024-04-25T10:03:03.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.422+0000 7f34b0d90640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ac0d1ee0 con 0x7f34ac15dbf0 2024-04-25T10:03:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.422+0000 7f34a3fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f34ac0b9370 0x7f34ac0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.423+0000 7f34a3fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f34ac0b9370 0x7f34ac0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45956/0 (socket says 172.21.15.139:45956) 2024-04-25T10:03:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.423+0000 7f34a3fff640 1 -- 172.21.15.139:0/3683751153 learned_addr learned my addr 172.21.15.139:0/3683751153 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.423+0000 7f34aa59c640 1 --2- 172.21.15.139:0/3683751153 >> v2:172.21.15.150:3300/0 conn(0x7f34ac15dbf0 0x7f34ac1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.423+0000 7f34a3fff640 1 -- 172.21.15.139:0/3683751153 >> v2:172.21.15.150:3300/0 conn(0x7f34ac15dbf0 msgr2=0x7f34ac1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.423+0000 7f34a3fff640 1 --2- 172.21.15.139:0/3683751153 >> v2:172.21.15.150:3300/0 conn(0x7f34ac15dbf0 0x7f34ac1617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.423+0000 7f34a3fff640 1 -- 172.21.15.139:0/3683751153 >> v1:172.21.15.150:6789/0 conn(0x7f34ac0cec80 legacy=0x7f34ac06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.423+0000 7f34a3fff640 1 -- 172.21.15.139:0/3683751153 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac1620a0 con 0x7f34ac0b9370 2024-04-25T10:03:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.423+0000 7f34a3fff640 1 --2- 172.21.15.139:0/3683751153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34ac0b9370 0x7f34ac0ce740 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f34ac06d330 tx=0x7f349402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970d88da96f3c525 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.424+0000 7f34a9d9b640 1 -- 172.21.15.139:0/3683751153 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494037070 con 0x7f34ac0b9370 2024-04-25T10:03:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.424+0000 7f34a9d9b640 1 -- 172.21.15.139:0/3683751153 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3494032440 con 0x7f34ac0b9370 2024-04-25T10:03:03.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.424+0000 7f34a9d9b640 1 -- 172.21.15.139:0/3683751153 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494032810 con 0x7f34ac0b9370 2024-04-25T10:03:03.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.424+0000 7f34b0d90640 1 -- 172.21.15.139:0/3683751153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34ac0b9370 msgr2=0x7f34ac0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.424+0000 7f34b0d90640 1 --2- 172.21.15.139:0/3683751153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34ac0b9370 0x7f34ac0ce740 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f34ac06d330 tx=0x7f349402a270 comp rx=0 tx=0).stop 2024-04-25T10:03:03.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.424+0000 7f34b0d90640 1 -- 172.21.15.139:0/3683751153 shutdown_connections 2024-04-25T10:03:03.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.424+0000 7f34b0d90640 1 --2- 172.21.15.139:0/3683751153 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34ac0b9370 0x7f34ac0ce740 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.424+0000 7f34b0d90640 1 --2- 172.21.15.139:0/3683751153 >> v2:172.21.15.150:3300/0 conn(0x7f34ac15dbf0 0x7f34ac1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.424+0000 7f34b0d90640 1 -- 172.21.15.139:0/3683751153 >> 172.21.15.139:0/3683751153 conn(0x7f34ac0b5480 msgr2=0x7f34ac0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.424+0000 7f34b0d90640 1 -- 172.21.15.139:0/3683751153 shutdown_connections 2024-04-25T10:03:03.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.424+0000 7f34b0d90640 1 -- 172.21.15.139:0/3683751153 wait complete. 2024-04-25T10:03:03.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.425+0000 7f34b0d90640 1 Processor -- start 2024-04-25T10:03:03.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.426+0000 7f34b0d90640 1 -- start start 2024-04-25T10:03:03.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.426+0000 7f34b0d90640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34ac0b9370 0x7f34ac1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.426+0000 7f34b0d90640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34ac0cec80 0x7f34ac1519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.426+0000 7f34b0d90640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34ac15dbf0 0x7f34ac0c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.426+0000 7f34b0d90640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac15dfd0 con 0x7f34ac0b9370 2024-04-25T10:03:03.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.426+0000 7f34b0d90640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac162660 con 0x7f34ac15dbf0 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.426+0000 7f34b0d90640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac153e90 con 0x7f34ac0cec80 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.426+0000 7f34aa59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34ac0b9370 0x7f34ac1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.426+0000 7f34aa59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34ac0b9370 0x7f34ac1529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45962/0 (socket says 172.21.15.139:45962) 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.426+0000 7f34aa59c640 1 -- 172.21.15.139:0/3301179733 learned_addr learned my addr 172.21.15.139:0/3301179733 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.426+0000 7f34a3fff640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34ac0cec80 0x7f34ac1519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.426+0000 7f34aad9d640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34ac15dbf0 0x7f34ac0c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.427+0000 7f34aa59c640 1 -- 172.21.15.139:0/3301179733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34ac0cec80 msgr2=0x7f34ac1519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.427+0000 7f34aa59c640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34ac0cec80 0x7f34ac1519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.427+0000 7f34aa59c640 1 -- 172.21.15.139:0/3301179733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34ac15dbf0 msgr2=0x7f34ac0c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.427+0000 7f34aa59c640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34ac15dbf0 0x7f34ac0c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.427+0000 7f34aa59c640 1 -- 172.21.15.139:0/3301179733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac1f7670 con 0x7f34ac0b9370 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.427+0000 7f34aa59c640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34ac0b9370 0x7f34ac1529d0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f349c00e870 tx=0x7f349c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.427+0000 7f34a2ffd640 1 -- 172.21.15.139:0/3301179733 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c017070 con 0x7f34ac0b9370 2024-04-25T10:03:03.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.427+0000 7f34a2ffd640 1 -- 172.21.15.139:0/3301179733 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349c00c940 con 0x7f34ac0b9370 2024-04-25T10:03:03.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.427+0000 7f34b0d90640 1 -- 172.21.15.139:0/3301179733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ac1f7950 con 0x7f34ac0b9370 2024-04-25T10:03:03.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.427+0000 7f34a2ffd640 1 -- 172.21.15.139:0/3301179733 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c00cd30 con 0x7f34ac0b9370 2024-04-25T10:03:03.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.427+0000 7f34b0d90640 1 -- 172.21.15.139:0/3301179733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ac1f7f10 con 0x7f34ac0b9370 2024-04-25T10:03:03.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.428+0000 7f34b0d90640 1 -- 172.21.15.139:0/3301179733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ac074520 con 0x7f34ac0b9370 2024-04-25T10:03:03.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.428+0000 7f34a2ffd640 1 -- 172.21.15.139:0/3301179733 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f349c01a070 con 0x7f34ac0b9370 2024-04-25T10:03:03.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.429+0000 7f34a2ffd640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f348c06b4d0 0x7f348c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.429+0000 7f34a2ffd640 1 -- 172.21.15.139:0/3301179733 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f349c08a5d0 con 0x7f34ac0b9370 2024-04-25T10:03:03.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.429+0000 7f34a3fff640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f348c06b4d0 0x7f348c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.429+0000 7f34a3fff640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f348c06b4d0 0x7f348c06d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3494034000 tx=0x7f349402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.433+0000 7f34a2ffd640 1 -- 172.21.15.139:0/3301179733 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f349c059300 con 0x7f34ac0b9370 2024-04-25T10:03:03.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:03.640+0000 7f34b0d90640 1 -- 172.21.15.139:0/3301179733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f34ac161a20 con 0x7f34ac0b9370 2024-04-25T10:03:04.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.036+0000 7f34a2ffd640 1 -- 172.21.15.139:0/3301179733 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v85) v1 ==== 162+0+0 (secure 0 0 0) 0x7f349c058c00 con 0x7f34ac0b9370 2024-04-25T10:03:04.041 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:03:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.039+0000 7f34b0d90640 1 -- 172.21.15.139:0/3301179733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f348c06b4d0 msgr2=0x7f348c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.039+0000 7f34b0d90640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f348c06b4d0 0x7f348c06d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3494034000 tx=0x7f349402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.039+0000 7f34b0d90640 1 -- 172.21.15.139:0/3301179733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34ac0b9370 msgr2=0x7f34ac1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.039+0000 7f34b0d90640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34ac0b9370 0x7f34ac1529d0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f349c00e870 tx=0x7f349c00ed30 comp rx=0 tx=0).stop 2024-04-25T10:03:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.039+0000 7f34b0d90640 1 -- 172.21.15.139:0/3301179733 shutdown_connections 2024-04-25T10:03:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.039+0000 7f34b0d90640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f348c06b4d0 0x7f348c06d990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.039+0000 7f34b0d90640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34ac15dbf0 0x7f34ac0c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.039+0000 7f34b0d90640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34ac0cec80 0x7f34ac1519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.039+0000 7f34b0d90640 1 --2- 172.21.15.139:0/3301179733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34ac0b9370 0x7f34ac1529d0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.039+0000 7f34b0d90640 1 -- 172.21.15.139:0/3301179733 >> 172.21.15.139:0/3301179733 conn(0x7f34ac0b5480 msgr2=0x7f34ac0d2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.039+0000 7f34b0d90640 1 -- 172.21.15.139:0/3301179733 shutdown_connections 2024-04-25T10:03:04.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.039+0000 7f34b0d90640 1 -- 172.21.15.139:0/3301179733 wait complete. 2024-04-25T10:03:04.057 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:04.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.149+0000 7f4df365d640 1 Processor -- start 2024-04-25T10:03:04.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.149+0000 7f4df365d640 1 -- start start 2024-04-25T10:03:04.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.150+0000 7f4df365d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dec15efb0 0x7f4dec0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.150+0000 7f4df365d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dec0cfe70 con 0x7f4dec0cf900 2024-04-25T10:03:04.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.150+0000 7f4df365d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dec0d0150 con 0x7f4dec099a50 2024-04-25T10:03:04.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.150+0000 7f4df365d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0d0a80 con 0x7f4dec15efb0 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.150+0000 7f4df0bf8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4dec0cf900 0x7f4dec06c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:48722/0 (socket says 172.21.15.139:48722) 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.150+0000 7f4df0bf8640 1 -- 172.21.15.139:0/894211570 learned_addr learned my addr 172.21.15.139:0/894211570 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.150+0000 7f4df13f9640 1 --2- 172.21.15.139:0/894211570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dec15efb0 0x7f4dec0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.150+0000 7f4de3fff640 1 -- 172.21.15.139:0/894211570 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644198488 0 0) 0x7f4dec0cfe70 con 0x7f4dec0cf900 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.151+0000 7f4de3fff640 1 -- 172.21.15.139:0/894211570 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dd8003520 con 0x7f4dec0cf900 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.151+0000 7f4de3fff640 1 -- 172.21.15.139:0/894211570 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363517355 0 0) 0x7f4dec0d0150 con 0x7f4dec099a50 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.151+0000 7f4de3fff640 1 -- 172.21.15.139:0/894211570 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dec0cfe70 con 0x7f4dec099a50 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.151+0000 7f4df13f9640 1 -- 172.21.15.139:0/894211570 >> v1:172.21.15.150:6789/0 conn(0x7f4dec099a50 legacy=0x7f4dec0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.151+0000 7f4df13f9640 1 -- 172.21.15.139:0/894211570 >> v1:172.21.15.139:6789/0 conn(0x7f4dec0cf900 legacy=0x7f4dec06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.151+0000 7f4de3fff640 1 -- 172.21.15.139:0/894211570 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1357180619 0 0) 0x7f4dd8003520 con 0x7f4dec0cf900 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.151+0000 7f4df13f9640 1 -- 172.21.15.139:0/894211570 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec0d0430 con 0x7f4dec15efb0 2024-04-25T10:03:04.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.151+0000 7f4df13f9640 1 --2- 172.21.15.139:0/894211570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dec15efb0 0x7f4dec0d0f10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4de80097d0 tx=0x7f4de802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650804994ead4f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.152+0000 7f4de3fff640 1 -- 172.21.15.139:0/894211570 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8037070 con 0x7f4dec15efb0 2024-04-25T10:03:04.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.152+0000 7f4de3fff640 1 -- 172.21.15.139:0/894211570 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de802ab90 con 0x7f4dec15efb0 2024-04-25T10:03:04.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.152+0000 7f4de3fff640 1 -- 172.21.15.139:0/894211570 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc001670 con 0x7f4dec15efb0 2024-04-25T10:03:04.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.152+0000 7f4df365d640 1 -- 172.21.15.139:0/894211570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dec15efb0 msgr2=0x7f4dec0d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.152+0000 7f4df365d640 1 --2- 172.21.15.139:0/894211570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dec15efb0 0x7f4dec0d0f10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4de80097d0 tx=0x7f4de802a100 comp rx=0 tx=0).stop 2024-04-25T10:03:04.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.152+0000 7f4df365d640 1 -- 172.21.15.139:0/894211570 shutdown_connections 2024-04-25T10:03:04.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.152+0000 7f4df365d640 1 --2- 172.21.15.139:0/894211570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dec15efb0 0x7f4dec0d0f10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.152+0000 7f4df365d640 1 -- 172.21.15.139:0/894211570 >> 172.21.15.139:0/894211570 conn(0x7f4dec0d1790 msgr2=0x7f4dec0d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.152+0000 7f4df365d640 1 -- 172.21.15.139:0/894211570 shutdown_connections 2024-04-25T10:03:04.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.152+0000 7f4df365d640 1 -- 172.21.15.139:0/894211570 wait complete. 2024-04-25T10:03:04.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.153+0000 7f4df365d640 1 Processor -- start 2024-04-25T10:03:04.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.153+0000 7f4df365d640 1 -- start start 2024-04-25T10:03:04.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.154+0000 7f4df365d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dec099a50 0x7f4dec161960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.154+0000 7f4df365d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec0cf900 0x7f4dec154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.154+0000 7f4df365d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dec15efb0 0x7f4dec154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.154+0000 7f4df365d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec099ee0 con 0x7f4dec0cf900 2024-04-25T10:03:04.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.154+0000 7f4df365d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec06cf50 con 0x7f4dec099a50 2024-04-25T10:03:04.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.154+0000 7f4df365d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec09a060 con 0x7f4dec15efb0 2024-04-25T10:03:04.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.154+0000 7f4df0bf8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec0cf900 0x7f4dec154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.154+0000 7f4df0bf8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec0cf900 0x7f4dec154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45972/0 (socket says 172.21.15.139:45972) 2024-04-25T10:03:04.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.154+0000 7f4df0bf8640 1 -- 172.21.15.139:0/2041095086 learned_addr learned my addr 172.21.15.139:0/2041095086 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:04.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.154+0000 7f4df13f9640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dec099a50 0x7f4dec161960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.155+0000 7f4df1bfa640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dec15efb0 0x7f4dec154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.155+0000 7f4df0bf8640 1 -- 172.21.15.139:0/2041095086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dec15efb0 msgr2=0x7f4dec154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.155+0000 7f4df0bf8640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dec15efb0 0x7f4dec154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.155+0000 7f4df0bf8640 1 -- 172.21.15.139:0/2041095086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dec099a50 msgr2=0x7f4dec161960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.155+0000 7f4df0bf8640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dec099a50 0x7f4dec161960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.155+0000 7f4df0bf8640 1 -- 172.21.15.139:0/2041095086 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec154cd0 con 0x7f4dec0cf900 2024-04-25T10:03:04.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.155+0000 7f4df0bf8640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec0cf900 0x7f4dec154250 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc005880 tx=0x7f4ddc00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.155+0000 7f4de1ffb640 1 -- 172.21.15.139:0/2041095086 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc017070 con 0x7f4dec0cf900 2024-04-25T10:03:04.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.155+0000 7f4de1ffb640 1 -- 172.21.15.139:0/2041095086 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ddc005ad0 con 0x7f4dec0cf900 2024-04-25T10:03:04.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.155+0000 7f4df365d640 1 -- 172.21.15.139:0/2041095086 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec1600c0 con 0x7f4dec0cf900 2024-04-25T10:03:04.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.155+0000 7f4de1ffb640 1 -- 172.21.15.139:0/2041095086 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc007070 con 0x7f4dec0cf900 2024-04-25T10:03:04.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.155+0000 7f4df365d640 1 -- 172.21.15.139:0/2041095086 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dec160960 con 0x7f4dec0cf900 2024-04-25T10:03:04.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.156+0000 7f4df365d640 1 -- 172.21.15.139:0/2041095086 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dec074520 con 0x7f4dec0cf900 2024-04-25T10:03:04.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.156+0000 7f4de1ffb640 1 -- 172.21.15.139:0/2041095086 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ddc01a070 con 0x7f4dec0cf900 2024-04-25T10:03:04.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.157+0000 7f4de1ffb640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4dd006b4d0 0x7f4dd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.157+0000 7f4de1ffb640 1 -- 172.21.15.139:0/2041095086 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4ddc089700 con 0x7f4dec0cf900 2024-04-25T10:03:04.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.157+0000 7f4df13f9640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4dd006b4d0 0x7f4dd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.157+0000 7f4df13f9640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4dd006b4d0 0x7f4dd006d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4de8034000 tx=0x7f4de802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.161+0000 7f4de1ffb640 1 -- 172.21.15.139:0/2041095086 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4dec154cd0 con 0x7f4dec0cf900 2024-04-25T10:03:04.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.365+0000 7f4df365d640 1 -- 172.21.15.139:0/2041095086 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dec09ae50 con 0x7f4dec0cf900 2024-04-25T10:03:04.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.366+0000 7f4de1ffb640 1 -- 172.21.15.139:0/2041095086 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f4ddc057de0 con 0x7f4dec0cf900 2024-04-25T10:03:04.371 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:04.371 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":86,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":7690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3195407462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3195407462},{"type":"v1","addr":"172.21.15.150:6835","nonce":3195407462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:03:04.039907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9117},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"a","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6836/3478297010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3478297010},{"type":"v1","addr":"172.21.15.139:6836","nonce":3478297010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:03:04.371 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 86 2024-04-25T10:03:04.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.368+0000 7f4df365d640 1 -- 172.21.15.139:0/2041095086 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4dd006b4d0 msgr2=0x7f4dd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.368+0000 7f4df365d640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4dd006b4d0 0x7f4dd006d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4de8034000 tx=0x7f4de802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:04.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.368+0000 7f4df365d640 1 -- 172.21.15.139:0/2041095086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec0cf900 msgr2=0x7f4dec154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.368+0000 7f4df365d640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec0cf900 0x7f4dec154250 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc005880 tx=0x7f4ddc00cbe0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.368+0000 7f4df365d640 1 -- 172.21.15.139:0/2041095086 shutdown_connections 2024-04-25T10:03:04.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.368+0000 7f4df365d640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4dec0cf900 0x7f4dec154250 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.368+0000 7f4df365d640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4dd006b4d0 0x7f4dd006d990 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.368+0000 7f4df365d640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dec15efb0 0x7f4dec154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.368+0000 7f4df365d640 1 --2- 172.21.15.139:0/2041095086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dec099a50 0x7f4dec161960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.368+0000 7f4df365d640 1 -- 172.21.15.139:0/2041095086 >> 172.21.15.139:0/2041095086 conn(0x7f4dec0d1790 msgr2=0x7f4dec0ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.368+0000 7f4df365d640 1 -- 172.21.15.139:0/2041095086 shutdown_connections 2024-04-25T10:03:04.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:04.368+0000 7f4df365d640 1 -- 172.21.15.139:0/2041095086 wait complete. 2024-04-25T10:03:04.786 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:50.319874+0000', 'modified': '2024-04-25T10:03:04.039907+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9117}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9117': {'gid': 9117, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.139:6836/3478297010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 3478297010}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 3478297010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:03:04.786 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9117:up:replay 2024-04-25T10:03:05.787 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:05.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.892+0000 7fe48e729640 1 Processor -- start 2024-04-25T10:03:05.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.892+0000 7fe48e729640 1 -- start start 2024-04-25T10:03:05.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe48e729640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4880b0750 0x7fe4880b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe48e729640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe4880d0730 0x7fe4880cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe48e729640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4880b1070 con 0x7fe4880d0b90 2024-04-25T10:03:05.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe48e729640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4880ccb70 con 0x7fe4880d0730 2024-04-25T10:03:05.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe48e729640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4880cf970 con 0x7fe4880b0750 2024-04-25T10:03:05.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe4877fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe4880d0730 0x7fe4880cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe4877fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe4880d0730 0x7fe4880cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45974/0 (socket says 172.21.15.139:45974) 2024-04-25T10:03:05.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe4877fe640 1 -- 172.21.15.139:0/161474328 learned_addr learned my addr 172.21.15.139:0/161474328 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:05.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe487fff640 1 --2- 172.21.15.139:0/161474328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4880b0750 0x7fe4880b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe4877fe640 1 -- 172.21.15.139:0/161474328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4880b0750 msgr2=0x7fe4880b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe4877fe640 1 --2- 172.21.15.139:0/161474328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4880b0750 0x7fe4880b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe4877fe640 1 -- 172.21.15.139:0/161474328 >> v1:172.21.15.139:6789/0 conn(0x7fe4880d0b90 legacy=0x7fe4880cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe4877fe640 1 -- 172.21.15.139:0/161474328 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4880cfc50 con 0x7fe4880d0730 2024-04-25T10:03:05.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.893+0000 7fe486ffd640 1 -- 172.21.15.139:0/161474328 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085042705 0 0) 0x7fe4880b1070 con 0x7fe4880d0b90 2024-04-25T10:03:05.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.894+0000 7fe4877fe640 1 --2- 172.21.15.139:0/161474328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4880d0730 0x7fe4880cbe40 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fe47c009870 tx=0x7fe47c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed525041b918646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.894+0000 7fe486ffd640 1 -- 172.21.15.139:0/161474328 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c037070 con 0x7fe4880d0730 2024-04-25T10:03:05.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.894+0000 7fe486ffd640 1 -- 172.21.15.139:0/161474328 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c032440 con 0x7fe4880d0730 2024-04-25T10:03:05.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.894+0000 7fe486ffd640 1 -- 172.21.15.139:0/161474328 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c032810 con 0x7fe4880d0730 2024-04-25T10:03:05.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.894+0000 7fe48e729640 1 -- 172.21.15.139:0/161474328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4880d0730 msgr2=0x7fe4880cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.894+0000 7fe48e729640 1 --2- 172.21.15.139:0/161474328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4880d0730 0x7fe4880cbe40 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fe47c009870 tx=0x7fe47c02a270 comp rx=0 tx=0).stop 2024-04-25T10:03:05.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.895+0000 7fe48e729640 1 -- 172.21.15.139:0/161474328 shutdown_connections 2024-04-25T10:03:05.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.895+0000 7fe48e729640 1 --2- 172.21.15.139:0/161474328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4880d0730 0x7fe4880cbe40 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.895+0000 7fe48e729640 1 --2- 172.21.15.139:0/161474328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4880b0750 0x7fe4880b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.895+0000 7fe48e729640 1 -- 172.21.15.139:0/161474328 >> 172.21.15.139:0/161474328 conn(0x7fe4880b3380 msgr2=0x7fe4880b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.895+0000 7fe48e729640 1 -- 172.21.15.139:0/161474328 shutdown_connections 2024-04-25T10:03:05.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.895+0000 7fe48e729640 1 -- 172.21.15.139:0/161474328 wait complete. 2024-04-25T10:03:05.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.896+0000 7fe48e729640 1 Processor -- start 2024-04-25T10:03:05.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.896+0000 7fe48e729640 1 -- start start 2024-04-25T10:03:05.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48e729640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4880b0750 0x7fe48814ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48e729640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4880d0730 0x7fe48815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48e729640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48815c530 0x7fe48815db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48e729640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48815bcc0 con 0x7fe48815c530 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48e729640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4880d0210 con 0x7fe4880d0730 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48e729640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48815be40 con 0x7fe4880b0750 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48ccc6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48815c530 0x7fe48815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48ccc6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48815c530 0x7fe48815db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45986/0 (socket says 172.21.15.139:45986) 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48ccc6640 1 -- 172.21.15.139:0/120671334 learned_addr learned my addr 172.21.15.139:0/120671334 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe487fff640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4880b0750 0x7fe48814ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe4877fe640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4880d0730 0x7fe48815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48ccc6640 1 -- 172.21.15.139:0/120671334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4880b0750 msgr2=0x7fe48814ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48ccc6640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4880b0750 0x7fe48814ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48ccc6640 1 -- 172.21.15.139:0/120671334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4880d0730 msgr2=0x7fe48815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48ccc6640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4880d0730 0x7fe48815bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48ccc6640 1 -- 172.21.15.139:0/120671334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48815cab0 con 0x7fe48815c530 2024-04-25T10:03:05.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe4877fe640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4880d0730 0x7fe48815bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:05.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe48ccc6640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48815c530 0x7fe48815db70 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c760 tx=0x7fe48000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.897+0000 7fe484ff9640 1 -- 172.21.15.139:0/120671334 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480003da0 con 0x7fe48815c530 2024-04-25T10:03:05.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.898+0000 7fe484ff9640 1 -- 172.21.15.139:0/120671334 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe480005070 con 0x7fe48815c530 2024-04-25T10:03:05.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.898+0000 7fe48e729640 1 -- 172.21.15.139:0/120671334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48815cd90 con 0x7fe48815c530 2024-04-25T10:03:05.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.898+0000 7fe484ff9640 1 -- 172.21.15.139:0/120671334 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480005460 con 0x7fe48815c530 2024-04-25T10:03:05.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.898+0000 7fe48e729640 1 -- 172.21.15.139:0/120671334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48815d350 con 0x7fe48815c530 2024-04-25T10:03:05.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.898+0000 7fe48e729640 1 -- 172.21.15.139:0/120671334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe48815d720 con 0x7fe48815c530 2024-04-25T10:03:05.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.899+0000 7fe484ff9640 1 -- 172.21.15.139:0/120671334 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe480005600 con 0x7fe48815c530 2024-04-25T10:03:05.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.899+0000 7fe484ff9640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe47406b500 0x7fe47406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.899+0000 7fe484ff9640 1 -- 172.21.15.139:0/120671334 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe48008a250 con 0x7fe48815c530 2024-04-25T10:03:05.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.899+0000 7fe487fff640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe47406b500 0x7fe47406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.900+0000 7fe487fff640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe47406b500 0x7fe47406d9c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe4880cb4c0 tx=0x7fe478008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:05.904+0000 7fe484ff9640 1 -- 172.21.15.139:0/120671334 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe480058ef0 con 0x7fe48815c530 2024-04-25T10:03:06.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.117+0000 7fe48e729640 1 -- 172.21.15.139:0/120671334 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe488074520 con 0x7fe48815c530 2024-04-25T10:03:06.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.117+0000 7fe484ff9640 1 -- 172.21.15.139:0/120671334 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4951 (secure 0 0 0) 0x7fe4800587f0 con 0x7fe48815c530 2024-04-25T10:03:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:06.123 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":88,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":7690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3195407462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3195407462},{"type":"v1","addr":"172.21.15.150:6835","nonce":3195407462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:03:05.047538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9117},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"a","rank":0,"incarnation":86,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.139:6836/3478297010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3478297010},{"type":"v1","addr":"172.21.15.139:6836","nonce":3478297010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:03:06.123 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 88 2024-04-25T10:03:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.120+0000 7fe48e729640 1 -- 172.21.15.139:0/120671334 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe47406b500 msgr2=0x7fe47406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.120+0000 7fe48e729640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe47406b500 0x7fe47406d9c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe4880cb4c0 tx=0x7fe478008040 comp rx=0 tx=0).stop 2024-04-25T10:03:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.120+0000 7fe48e729640 1 -- 172.21.15.139:0/120671334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48815c530 msgr2=0x7fe48815db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.120+0000 7fe48e729640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48815c530 0x7fe48815db70 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c760 tx=0x7fe48000cc20 comp rx=0 tx=0).stop 2024-04-25T10:03:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.120+0000 7fe48e729640 1 -- 172.21.15.139:0/120671334 shutdown_connections 2024-04-25T10:03:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.120+0000 7fe48e729640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe48815c530 0x7fe48815db70 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.120+0000 7fe48e729640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe47406b500 0x7fe47406d9c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.120+0000 7fe48e729640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4880d0730 0x7fe48815bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.120+0000 7fe48e729640 1 --2- 172.21.15.139:0/120671334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4880b0750 0x7fe48814ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.120+0000 7fe48e729640 1 -- 172.21.15.139:0/120671334 >> 172.21.15.139:0/120671334 conn(0x7fe4880b3380 msgr2=0x7fe4880b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.120+0000 7fe48e729640 1 -- 172.21.15.139:0/120671334 shutdown_connections 2024-04-25T10:03:06.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:06.120+0000 7fe48e729640 1 -- 172.21.15.139:0/120671334 wait complete. 2024-04-25T10:03:06.538 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:50.319874+0000', 'modified': '2024-04-25T10:03:05.047538+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9117}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9117': {'gid': 9117, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.139:6836/3478297010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 3478297010}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 3478297010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:03:06.538 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9117:up:rejoin 2024-04-25T10:03:07.539 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:07.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.643+0000 7f1409b3b640 1 Processor -- start 2024-04-25T10:03:07.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.643+0000 7f1409b3b640 1 -- start start 2024-04-25T10:03:07.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f1409b3b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1404158260 0x7f140415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f1409b3b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14040b37d0 0x7f140406ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f1409b3b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14040b3e10 con 0x7f140406da20 2024-04-25T10:03:07.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f1409b3b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14041615c0 con 0x7f1404158260 2024-04-25T10:03:07.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f1409b3b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14041618a0 con 0x7f14040b37d0 2024-04-25T10:03:07.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f1402ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f140406da20 0x7f14040b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34132/0 (socket says 172.21.15.139:34132) 2024-04-25T10:03:07.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f1402ffd640 1 -- 172.21.15.139:0/1581905748 learned_addr learned my addr 172.21.15.139:0/1581905748 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:07.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f14037fe640 1 --2- 172.21.15.139:0/1581905748 >> v2:172.21.15.150:3300/0 conn(0x7f1404158260 0x7f140415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f1403fff640 1 --2- 172.21.15.139:0/1581905748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14040b37d0 0x7f140406ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f14027fc640 1 -- 172.21.15.139:0/1581905748 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479708001 0 0) 0x7f13f4001110 con 0x7f140406da20 2024-04-25T10:03:07.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f14027fc640 1 -- 172.21.15.139:0/1581905748 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13f0003520 con 0x7f140406da20 2024-04-25T10:03:07.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f14037fe640 1 -- 172.21.15.139:0/1581905748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14040b37d0 msgr2=0x7f140406ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f14027fc640 1 -- 172.21.15.139:0/1581905748 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1140203865 0 0) 0x7f13f0003520 con 0x7f140406da20 2024-04-25T10:03:07.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f14037fe640 1 --2- 172.21.15.139:0/1581905748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14040b37d0 0x7f140406ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f14037fe640 1 -- 172.21.15.139:0/1581905748 >> v1:172.21.15.139:6789/0 conn(0x7f140406da20 legacy=0x7f14040b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f14037fe640 1 -- 172.21.15.139:0/1581905748 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1404161b80 con 0x7f1404158260 2024-04-25T10:03:07.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.644+0000 7f1403fff640 1 --2- 172.21.15.139:0/1581905748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14040b37d0 0x7f140406ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:07.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.645+0000 7f14037fe640 1 --2- 172.21.15.139:0/1581905748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1404158260 0x7f140415df80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0097a0 tx=0x7f13ec02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa910b7b86d5a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.645+0000 7f14027fc640 1 -- 172.21.15.139:0/1581905748 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec037070 con 0x7f1404158260 2024-04-25T10:03:07.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.645+0000 7f14027fc640 1 -- 172.21.15.139:0/1581905748 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec02aba0 con 0x7f1404158260 2024-04-25T10:03:07.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.645+0000 7f14027fc640 1 -- 172.21.15.139:0/1581905748 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4001840 con 0x7f1404158260 2024-04-25T10:03:07.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.646+0000 7f1409b3b640 1 -- 172.21.15.139:0/1581905748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1404158260 msgr2=0x7f140415df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.646+0000 7f1409b3b640 1 --2- 172.21.15.139:0/1581905748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1404158260 0x7f140415df80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0097a0 tx=0x7f13ec02a110 comp rx=0 tx=0).stop 2024-04-25T10:03:07.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.646+0000 7f1409b3b640 1 -- 172.21.15.139:0/1581905748 shutdown_connections 2024-04-25T10:03:07.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.646+0000 7f1409b3b640 1 --2- 172.21.15.139:0/1581905748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14040b37d0 0x7f140406ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.646+0000 7f1409b3b640 1 --2- 172.21.15.139:0/1581905748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1404158260 0x7f140415df80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.646+0000 7f1409b3b640 1 -- 172.21.15.139:0/1581905748 >> 172.21.15.139:0/1581905748 conn(0x7f14040b23d0 msgr2=0x7f14040cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.646+0000 7f1409b3b640 1 -- 172.21.15.139:0/1581905748 shutdown_connections 2024-04-25T10:03:07.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.646+0000 7f1409b3b640 1 -- 172.21.15.139:0/1581905748 wait complete. 2024-04-25T10:03:07.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.647+0000 7f1409b3b640 1 Processor -- start 2024-04-25T10:03:07.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.647+0000 7f1409b3b640 1 -- start start 2024-04-25T10:03:07.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.648+0000 7f1409b3b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f140406da20 0x7f1404145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.648+0000 7f1409b3b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14040b37d0 0x7f14041460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.648+0000 7f1409b3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1404158260 0x7f1404142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.648+0000 7f1409b3b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1404162420 con 0x7f1404158260 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.648+0000 7f1409b3b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1404162140 con 0x7f140406da20 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.648+0000 7f1409b3b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140406b2e0 con 0x7f14040b37d0 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.648+0000 7f1403fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1404158260 0x7f1404142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.648+0000 7f1403fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1404158260 0x7f1404142ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52290/0 (socket says 172.21.15.139:52290) 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.648+0000 7f1403fff640 1 -- 172.21.15.139:0/747220208 learned_addr learned my addr 172.21.15.139:0/747220208 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.649+0000 7f14037fe640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f140406da20 0x7f1404145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.649+0000 7f1403fff640 1 -- 172.21.15.139:0/747220208 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14040b37d0 msgr2=0x7f14041460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.649+0000 7f1403fff640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14040b37d0 0x7f14041460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.649+0000 7f1403fff640 1 -- 172.21.15.139:0/747220208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f140406da20 msgr2=0x7f1404145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.649+0000 7f1403fff640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f140406da20 0x7f1404145b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.649+0000 7f1403fff640 1 -- 172.21.15.139:0/747220208 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140415f870 con 0x7f1404158260 2024-04-25T10:03:07.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.649+0000 7f1403fff640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1404158260 0x7f1404142ea0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00efc0 tx=0x7f13fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.649+0000 7f13fbfff640 1 -- 172.21.15.139:0/747220208 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc018030 con 0x7f1404158260 2024-04-25T10:03:07.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.649+0000 7f13fbfff640 1 -- 172.21.15.139:0/747220208 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc00cdf0 con 0x7f1404158260 2024-04-25T10:03:07.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.649+0000 7f1409b3b640 1 -- 172.21.15.139:0/747220208 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140415fb10 con 0x7f1404158260 2024-04-25T10:03:07.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.649+0000 7f13fbfff640 1 -- 172.21.15.139:0/747220208 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc003bb0 con 0x7f1404158260 2024-04-25T10:03:07.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.649+0000 7f1409b3b640 1 -- 172.21.15.139:0/747220208 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14041600d0 con 0x7f1404158260 2024-04-25T10:03:07.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.650+0000 7f1409b3b640 1 -- 172.21.15.139:0/747220208 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1404157100 con 0x7f1404158260 2024-04-25T10:03:07.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.650+0000 7f13fbfff640 1 -- 172.21.15.139:0/747220208 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13fc01b070 con 0x7f1404158260 2024-04-25T10:03:07.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.651+0000 7f13fbfff640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f13e406b500 0x7f13e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.651+0000 7f14037fe640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f13e406b500 0x7f13e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.651+0000 7f13fbfff640 1 -- 172.21.15.139:0/747220208 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f13fc08ae60 con 0x7f1404158260 2024-04-25T10:03:07.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.651+0000 7f14037fe640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f13e406b500 0x7f13e406d9c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f13ec034000 tx=0x7f13ec02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.655+0000 7f13fbfff640 1 -- 172.21.15.139:0/747220208 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1404157100 con 0x7f1404158260 2024-04-25T10:03:07.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.858+0000 7f1409b3b640 1 -- 172.21.15.139:0/747220208 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14040cc2c0 con 0x7f1404158260 2024-04-25T10:03:07.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.859+0000 7f13fbfff640 1 -- 172.21.15.139:0/747220208 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f13fc010040 con 0x7f1404158260 2024-04-25T10:03:07.864 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:07.864 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":7690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3195407462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3195407462},{"type":"v1","addr":"172.21.15.150:6835","nonce":3195407462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:03:06.057564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9117},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":8,"addr":"172.21.15.139:6836/3478297010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3478297010},{"type":"v1","addr":"172.21.15.139:6836","nonce":3478297010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:03:07.864 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 89 2024-04-25T10:03:07.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.861+0000 7f1409b3b640 1 -- 172.21.15.139:0/747220208 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f13e406b500 msgr2=0x7f13e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.861+0000 7f1409b3b640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f13e406b500 0x7f13e406d9c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f13ec034000 tx=0x7f13ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:07.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.861+0000 7f1409b3b640 1 -- 172.21.15.139:0/747220208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1404158260 msgr2=0x7f1404142ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.861+0000 7f1409b3b640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1404158260 0x7f1404142ea0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00efc0 tx=0x7f13fc00c490 comp rx=0 tx=0).stop 2024-04-25T10:03:07.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.861+0000 7f1409b3b640 1 -- 172.21.15.139:0/747220208 shutdown_connections 2024-04-25T10:03:07.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.861+0000 7f1409b3b640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1404158260 0x7f1404142ea0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.861+0000 7f1409b3b640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f13e406b500 0x7f13e406d9c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.861+0000 7f1409b3b640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f14040b37d0 0x7f14041460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.861+0000 7f1409b3b640 1 --2- 172.21.15.139:0/747220208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f140406da20 0x7f1404145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.861+0000 7f1409b3b640 1 -- 172.21.15.139:0/747220208 >> 172.21.15.139:0/747220208 conn(0x7f14040b23d0 msgr2=0x7f14040b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.861+0000 7f1409b3b640 1 -- 172.21.15.139:0/747220208 shutdown_connections 2024-04-25T10:03:07.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:07.861+0000 7f1409b3b640 1 -- 172.21.15.139:0/747220208 wait complete. 2024-04-25T10:03:08.279 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:50.319874+0000', 'modified': '2024-04-25T10:03:06.057564+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9117}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9117': {'gid': 9117, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.139:6836/3478297010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 3478297010}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 3478297010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:03:08.279 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:03:08.279 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:03:08.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.374+0000 7fe14d7f9640 1 Processor -- start 2024-04-25T10:03:08.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.374+0000 7fe14d7f9640 1 -- start start 2024-04-25T10:03:08.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.375+0000 7fe14d7f9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe14815b880 0x7fe14806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.375+0000 7fe14d7f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe148072db0 0x7fe1480cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.375+0000 7fe14d7f9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14815bc60 con 0x7fe14815f3a0 2024-04-25T10:03:08.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.375+0000 7fe14d7f9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe148070cd0 con 0x7fe14815b880 2024-04-25T10:03:08.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.375+0000 7fe14d7f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1480cffa0 con 0x7fe148072db0 2024-04-25T10:03:08.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.375+0000 7fe146ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe14815b880 0x7fe14806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.375+0000 7fe1467fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe14815f3a0 0x7fe1480726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34148/0 (socket says 172.21.15.139:34148) 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.375+0000 7fe1467fc640 1 -- 172.21.15.139:0/2863675567 learned_addr learned my addr 172.21.15.139:0/2863675567 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.375+0000 7fe146ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe14815b880 0x7fe14806fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52296/0 (socket says 172.21.15.139:52296) 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.376+0000 7fe1477fe640 1 --2- 172.21.15.139:0/2863675567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe148072db0 0x7fe1480cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.376+0000 7fe146ffd640 1 -- 172.21.15.139:0/2863675567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe148072db0 msgr2=0x7fe1480cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.376+0000 7fe145ffb640 1 -- 172.21.15.139:0/2863675567 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745251124 0 0) 0x7fe14815bc60 con 0x7fe14815f3a0 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.376+0000 7fe146ffd640 1 --2- 172.21.15.139:0/2863675567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe148072db0 0x7fe1480cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.376+0000 7fe146ffd640 1 -- 172.21.15.139:0/2863675567 >> v1:172.21.15.139:6789/0 conn(0x7fe14815f3a0 legacy=0x7fe1480726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.376+0000 7fe146ffd640 1 -- 172.21.15.139:0/2863675567 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480cf6b0 con 0x7fe14815b880 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.376+0000 7fe1477fe640 1 --2- 172.21.15.139:0/2863675567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe148072db0 0x7fe1480cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.376+0000 7fe146ffd640 1 --2- 172.21.15.139:0/2863675567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe14815b880 0x7fe14806fa90 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fe1480d0120 tx=0x7fe13802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d82cdf3d8477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.376+0000 7fe145ffb640 1 -- 172.21.15.139:0/2863675567 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13802ae60 con 0x7fe14815b880 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.377+0000 7fe145ffb640 1 -- 172.21.15.139:0/2863675567 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe138007740 con 0x7fe14815b880 2024-04-25T10:03:08.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.377+0000 7fe145ffb640 1 -- 172.21.15.139:0/2863675567 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138007b10 con 0x7fe14815b880 2024-04-25T10:03:08.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.377+0000 7fe14d7f9640 1 -- 172.21.15.139:0/2863675567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe14815b880 msgr2=0x7fe14806fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.377+0000 7fe14d7f9640 1 --2- 172.21.15.139:0/2863675567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe14815b880 0x7fe14806fa90 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fe1480d0120 tx=0x7fe13802a640 comp rx=0 tx=0).stop 2024-04-25T10:03:08.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.377+0000 7fe14d7f9640 1 -- 172.21.15.139:0/2863675567 shutdown_connections 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.377+0000 7fe14d7f9640 1 --2- 172.21.15.139:0/2863675567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe148072db0 0x7fe1480cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.377+0000 7fe14d7f9640 1 --2- 172.21.15.139:0/2863675567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe14815b880 0x7fe14806fa90 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.377+0000 7fe14d7f9640 1 -- 172.21.15.139:0/2863675567 >> 172.21.15.139:0/2863675567 conn(0x7fe1480b23f0 msgr2=0x7fe1480b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.377+0000 7fe14d7f9640 1 -- 172.21.15.139:0/2863675567 shutdown_connections 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.377+0000 7fe14d7f9640 1 -- 172.21.15.139:0/2863675567 wait complete. 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.378+0000 7fe14d7f9640 1 Processor -- start 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.378+0000 7fe14d7f9640 1 -- start start 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.379+0000 7fe14d7f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe148072db0 0x7fe1481506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.379+0000 7fe14d7f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe14815b880 0x7fe1480b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.379+0000 7fe14d7f9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe14815f3a0 0x7fe14815cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.379+0000 7fe14d7f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1480cfc30 con 0x7fe148072db0 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.379+0000 7fe14d7f9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1480cf950 con 0x7fe14815f3a0 2024-04-25T10:03:08.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.379+0000 7fe14d7f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe148070610 con 0x7fe14815b880 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.379+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe148072db0 0x7fe1481506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.379+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe148072db0 0x7fe1481506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52302/0 (socket says 172.21.15.139:52302) 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.379+0000 7fe146ffd640 1 -- 172.21.15.139:0/902451842 learned_addr learned my addr 172.21.15.139:0/902451842 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.379+0000 7fe1477fe640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe14815f3a0 0x7fe14815cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.379+0000 7fe1467fc640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe14815b880 0x7fe1480b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.380+0000 7fe146ffd640 1 -- 172.21.15.139:0/902451842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe14815b880 msgr2=0x7fe1480b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.380+0000 7fe146ffd640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe14815b880 0x7fe1480b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.380+0000 7fe146ffd640 1 -- 172.21.15.139:0/902451842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe14815f3a0 msgr2=0x7fe14815cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.380+0000 7fe146ffd640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe14815f3a0 0x7fe14815cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.380+0000 7fe146ffd640 1 -- 172.21.15.139:0/902451842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480c9ba0 con 0x7fe148072db0 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.380+0000 7fe1467fc640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe14815b880 0x7fe1480b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.380+0000 7fe146ffd640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe148072db0 0x7fe1481506c0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fe13802ac30 tx=0x7fe138004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.380+0000 7fe12f7fe640 1 -- 172.21.15.139:0/902451842 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138040070 con 0x7fe148072db0 2024-04-25T10:03:08.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.380+0000 7fe12f7fe640 1 -- 172.21.15.139:0/902451842 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe138031d40 con 0x7fe148072db0 2024-04-25T10:03:08.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.380+0000 7fe14d7f9640 1 -- 172.21.15.139:0/902451842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1480c9e80 con 0x7fe148072db0 2024-04-25T10:03:08.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.380+0000 7fe12f7fe640 1 -- 172.21.15.139:0/902451842 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138009450 con 0x7fe148072db0 2024-04-25T10:03:08.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.380+0000 7fe14d7f9640 1 -- 172.21.15.139:0/902451842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1480ca440 con 0x7fe148072db0 2024-04-25T10:03:08.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.382+0000 7fe12f7fe640 1 -- 172.21.15.139:0/902451842 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe138039040 con 0x7fe148072db0 2024-04-25T10:03:08.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.382+0000 7fe12f7fe640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe13406b4d0 0x7fe13406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.382+0000 7fe12f7fe640 1 -- 172.21.15.139:0/902451842 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe1380ab780 con 0x7fe148072db0 2024-04-25T10:03:08.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.382+0000 7fe1467fc640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe13406b4d0 0x7fe13406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.383+0000 7fe1467fc640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe13406b4d0 0x7fe13406d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe1481f72f0 tx=0x7fe13c007ca0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.383+0000 7fe1505dc740 1 Processor -- start 2024-04-25T10:03:08.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.383+0000 7fe1505dc740 1 -- start start 2024-04-25T10:03:08.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe1505dc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07d25610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe1505dc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d42ad0 0x560b07d43020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe1505dc740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07cfaf10 0x560b07d41190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe1505dc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b07c5da10 con 0x560b07d25230 2024-04-25T10:03:08.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe1505dc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b07c06b30 con 0x560b07d42ad0 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe1505dc740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b07c59c50 con 0x560b07cfaf10 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07d25610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07d25610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52310/0 (socket says 172.21.15.139:52310) 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe146ffd640 1 -- 172.21.15.139:0/513068021 learned_addr learned my addr 172.21.15.139:0/513068021 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe1467fc640 1 --2- 172.21.15.139:0/513068021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d42ad0 0x560b07d43020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe1477fe640 1 --2- 172.21.15.139:0/513068021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07cfaf10 0x560b07d41190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe146ffd640 1 -- 172.21.15.139:0/513068021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07cfaf10 msgr2=0x560b07d41190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe146ffd640 1 --2- 172.21.15.139:0/513068021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07cfaf10 0x560b07d41190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe146ffd640 1 -- 172.21.15.139:0/513068021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d42ad0 msgr2=0x560b07d43020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe146ffd640 1 --2- 172.21.15.139:0/513068021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d42ad0 0x560b07d43020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe146ffd640 1 -- 172.21.15.139:0/513068021 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b07c1be10 con 0x560b07d25230 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.384+0000 7fe146ffd640 1 --2- 172.21.15.139:0/513068021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07d25610 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fe1380820b0 tx=0x7fe138079830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a027a7c899dc3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.385+0000 7fe12cff9640 1 -- 172.21.15.139:0/513068021 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138091030 con 0x560b07d25230 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.385+0000 7fe12cff9640 1 -- 172.21.15.139:0/513068021 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13807e020 con 0x560b07d25230 2024-04-25T10:03:08.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.385+0000 7fe12cff9640 1 -- 172.21.15.139:0/513068021 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1380825e0 con 0x560b07d25230 2024-04-25T10:03:08.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.385+0000 7fe1505dc740 1 -- 172.21.15.139:0/513068021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 msgr2=0x560b07d25610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.385+0000 7fe1505dc740 1 --2- 172.21.15.139:0/513068021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07d25610 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fe1380820b0 tx=0x7fe138079830 comp rx=0 tx=0).stop 2024-04-25T10:03:08.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.385+0000 7fe1505dc740 1 -- 172.21.15.139:0/513068021 shutdown_connections 2024-04-25T10:03:08.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.385+0000 7fe1505dc740 1 --2- 172.21.15.139:0/513068021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07cfaf10 0x560b07d41190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.385+0000 7fe1505dc740 1 --2- 172.21.15.139:0/513068021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d42ad0 0x560b07d43020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.385+0000 7fe1505dc740 1 --2- 172.21.15.139:0/513068021 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07d25610 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.385+0000 7fe1505dc740 1 -- 172.21.15.139:0/513068021 >> 172.21.15.139:0/513068021 conn(0x560b07cf7840 msgr2=0x560b07d40040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.385+0000 7fe1505dc740 1 -- 172.21.15.139:0/513068021 shutdown_connections 2024-04-25T10:03:08.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.385+0000 7fe1505dc740 1 -- 172.21.15.139:0/513068021 wait complete. 2024-04-25T10:03:08.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.386+0000 7fe1505dc740 1 -- start start 2024-04-25T10:03:08.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.386+0000 7fe1505dc740 1 Processor -- start 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1505dc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d25230 0x560b07d439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1505dc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d42ad0 0x560b07d25ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1505dc740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d504b0 0x560b07d28860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1505dc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b07cfbcb0 con 0x560b07d42ad0 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1505dc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b07c1a0c0 con 0x560b07d25230 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1505dc740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b07cf5770 con 0x560b07d504b0 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1505dc740 10 client.0 _ll_register_callbacks cb 0x560b07c0dec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1467fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d42ad0 0x560b07d25ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1467fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d42ad0 0x560b07d25ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52326/0 (socket says 172.21.15.139:52326) 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1467fc640 1 -- 172.21.15.139:0/1583121869 learned_addr learned my addr 172.21.15.139:0/1583121869 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe146ffd640 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d25230 0x560b07d439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1477fe640 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d504b0 0x560b07d28860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1467fc640 1 -- 172.21.15.139:0/1583121869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d504b0 msgr2=0x560b07d28860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1467fc640 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d504b0 0x560b07d28860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1467fc640 1 -- 172.21.15.139:0/1583121869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d25230 msgr2=0x560b07d439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1467fc640 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d25230 0x560b07d439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1467fc640 1 -- 172.21.15.139:0/1583121869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b079722d0 con 0x560b07d42ad0 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1467fc640 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d42ad0 0x560b07d25ef0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fe13c001d50 tx=0x7fe13c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ae24dbf935ff0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe125ffb640 1 -- 172.21.15.139:0/1583121869 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c027060 con 0x560b07d42ad0 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe125ffb640 1 -- 172.21.15.139:0/1583121869 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe13c010040 con 0x560b07d42ad0 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.387+0000 7fe1505dc740 1 -- 172.21.15.139:0/1583121869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe13807e020 con 0x560b07d42ad0 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe125ffb640 1 -- 172.21.15.139:0/1583121869 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe13c015e50 con 0x560b07d42ad0 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe1505dc740 10 client.9450 fetch_fsmap learned FSMap version 89 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe1505dc740 1 -- 172.21.15.139:0/1583121869 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x560b07a3c6b0 con 0x560b07d42ad0 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe125ffb640 1 -- 172.21.15.139:0/1583121869 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13807e020 con 0x560b07d42ad0 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe125ffb640 1 -- 172.21.15.139:0/1583121869 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe13c013420 con 0x560b07d42ad0 2024-04-25T10:03:08.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe1505dc740 10 client.9450 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe1505dc740 10 client.9450 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe1505dc740 20 client.9450 populate_metadata read hostname 'smithi139' 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe1505dc740 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x560b07d376a0 0x560b07d37a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe1505dc740 4 client.9450 mds_command: new command op to 9117 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe1505dc740 1 -- 172.21.15.139:0/1583121869 --> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b07cbc920 con 0x560b07d376a0 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe146ffd640 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x560b07d376a0 0x560b07d37a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe146ffd640 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x560b07d376a0 0x560b07d37a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806ee2a75e0b488e server_cookie=5a6800375a2dd9e0 in_seq=0 out_seq=0 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.388+0000 7fe125ffb640 10 client.9450 ms_handle_connect on v2:172.21.15.139:6834/3478297010 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe125ffb640 1 -- 172.21.15.139:0/1583121869 <== mds.0 v2:172.21.15.139:6834/3478297010 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560b07cbc920 con 0x560b07d376a0 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe125ffb640 10 client.9450 handle_command_reply: tid=0 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe1505dc740 1 client.9450 shutdown 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe1505dc740 1 -- 172.21.15.139:0/1583121869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d42ad0 msgr2=0x560b07d25ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe1505dc740 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d42ad0 0x560b07d25ef0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fe13c001d50 tx=0x7fe13c011000 comp rx=0 tx=0).stop 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe1505dc740 1 -- 172.21.15.139:0/1583121869 shutdown_connections 2024-04-25T10:03:08.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe1505dc740 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x560b07d376a0 0x560b07d37a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe1505dc740 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d42ad0 0x560b07d25ef0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe1505dc740 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d504b0 0x560b07d28860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe1505dc740 1 --2- 172.21.15.139:0/1583121869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d25230 0x560b07d439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe125ffb640 0 client.9450 ms_handle_reset on v2:172.21.15.139:6834/3478297010 2024-04-25T10:03:08.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe1505dc740 1 -- 172.21.15.139:0/1583121869 >> 172.21.15.139:0/1583121869 conn(0x560b07cfaf10 msgr2=0x560b07cf97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe1505dc740 1 -- 172.21.15.139:0/1583121869 shutdown_connections 2024-04-25T10:03:08.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe1505dc740 1 -- 172.21.15.139:0/1583121869 wait complete. 2024-04-25T10:03:08.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.389+0000 7fe1505dc740 20 client.9450 trim_cache size 0 max 16384 2024-04-25T10:03:08.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.404+0000 7fe1505dc740 1 Processor -- start 2024-04-25T10:03:08.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe1505dc740 1 -- start start 2024-04-25T10:03:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe1505dc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07cf96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe1505dc740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d42ad0 0x560b07cfb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe1505dc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d504b0 0x560b07cfb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe1505dc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481ff390 con 0x560b07d25230 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe1505dc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481fe790 con 0x560b07d504b0 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe1505dc740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481fed90 con 0x560b07d42ad0 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07cf96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07cf96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52332/0 (socket says 172.21.15.139:52332) 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe146ffd640 1 -- 172.21.15.139:0/2885562964 learned_addr learned my addr 172.21.15.139:0/2885562964 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe146ffd640 1 -- 172.21.15.139:0/2885562964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d42ad0 msgr2=0x560b07cfb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe146ffd640 1 --2- 172.21.15.139:0/2885562964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d42ad0 0x560b07cfb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe146ffd640 1 -- 172.21.15.139:0/2885562964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d504b0 msgr2=0x560b07cfb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe146ffd640 1 --2- 172.21.15.139:0/2885562964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d504b0 0x560b07cfb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.405+0000 7fe146ffd640 1 -- 172.21.15.139:0/2885562964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b07d39320 con 0x560b07d25230 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe146ffd640 1 --2- 172.21.15.139:0/2885562964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07cf96c0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fe1380820b0 tx=0x7fe138080000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b1b45b2aaa733f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe127fff640 1 -- 172.21.15.139:0/2885562964 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138091040 con 0x560b07d25230 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe127fff640 1 -- 172.21.15.139:0/2885562964 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe138077f70 con 0x560b07d25230 2024-04-25T10:03:08.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe127fff640 1 -- 172.21.15.139:0/2885562964 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138076540 con 0x560b07d25230 2024-04-25T10:03:08.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe1505dc740 1 -- 172.21.15.139:0/2885562964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 msgr2=0x560b07cf96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe1505dc740 1 --2- 172.21.15.139:0/2885562964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07cf96c0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fe1380820b0 tx=0x7fe138080000 comp rx=0 tx=0).stop 2024-04-25T10:03:08.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe1505dc740 1 -- 172.21.15.139:0/2885562964 shutdown_connections 2024-04-25T10:03:08.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe1505dc740 1 --2- 172.21.15.139:0/2885562964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d504b0 0x560b07cfb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe1505dc740 1 --2- 172.21.15.139:0/2885562964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d42ad0 0x560b07cfb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe1505dc740 1 --2- 172.21.15.139:0/2885562964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07cf96c0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe1505dc740 1 -- 172.21.15.139:0/2885562964 >> 172.21.15.139:0/2885562964 conn(0x560b07cfaf10 msgr2=0x560b07cfa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe1505dc740 1 -- 172.21.15.139:0/2885562964 shutdown_connections 2024-04-25T10:03:08.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.406+0000 7fe1505dc740 1 -- 172.21.15.139:0/2885562964 wait complete. 2024-04-25T10:03:08.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.407+0000 7fe1505dc740 1 -- start start 2024-04-25T10:03:08.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.407+0000 7fe1505dc740 1 Processor -- start 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe1505dc740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07d4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe1505dc740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d42ad0 0x560b07cfb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe1505dc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d504b0 0x560b07cfb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe1505dc740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b07d16e50 con 0x560b07d25230 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe1505dc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b07d50d10 con 0x560b07d504b0 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe1505dc740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b07d39050 con 0x560b07d42ad0 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe1505dc740 10 client.0 _ll_register_callbacks cb 0x560b07cf5c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07d4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe146ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07d4f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52344/0 (socket says 172.21.15.139:52344) 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe146ffd640 1 -- 172.21.15.139:0/225171784 learned_addr learned my addr 172.21.15.139:0/225171784 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe1467fc640 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d42ad0 0x560b07cfb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe1477fe640 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d504b0 0x560b07cfb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe146ffd640 1 -- 172.21.15.139:0/225171784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d42ad0 msgr2=0x560b07cfb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe146ffd640 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d42ad0 0x560b07cfb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe146ffd640 1 -- 172.21.15.139:0/225171784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d504b0 msgr2=0x560b07cfb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe146ffd640 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d504b0 0x560b07cfb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe146ffd640 1 -- 172.21.15.139:0/225171784 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b07d359d0 con 0x560b07d25230 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe146ffd640 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07d4f090 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fe138079e60 tx=0x7fe138009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2aa74557ef45e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe125ffb640 1 -- 172.21.15.139:0/225171784 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138091040 con 0x560b07d25230 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe125ffb640 1 -- 172.21.15.139:0/225171784 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe1380800f0 con 0x560b07d25230 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe1505dc740 1 -- 172.21.15.139:0/225171784 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe138077f70 con 0x560b07d25230 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe125ffb640 1 -- 172.21.15.139:0/225171784 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe138096050 con 0x560b07d25230 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe1505dc740 10 client.9456 fetch_fsmap learned FSMap version 89 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe1505dc740 1 -- 172.21.15.139:0/225171784 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x560b07dae710 con 0x560b07d25230 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.408+0000 7fe125ffb640 1 -- 172.21.15.139:0/225171784 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe138077f70 con 0x560b07d25230 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe125ffb640 1 -- 172.21.15.139:0/225171784 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe13807a130 con 0x560b07d25230 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 10 client.9456 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 10 client.9456 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 20 client.9456 populate_metadata read hostname 'smithi139' 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x560b07dae2d0 0x560b07d59a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 4 client.9456 mds_command: new command op to 9117 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 -- 172.21.15.139:0/225171784 --> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560b07c6af50 con 0x560b07dae2d0 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1467fc640 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x560b07dae2d0 0x560b07d59a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe125ffb640 10 client.9456 ms_handle_connect on v2:172.21.15.139:6834/3478297010 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1467fc640 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x560b07dae2d0 0x560b07d59a30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc5546b69403b79 server_cookie=4225153c189721ca in_seq=0 out_seq=0 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe125ffb640 1 -- 172.21.15.139:0/225171784 <== mds.0 v2:172.21.15.139:6834/3478297010 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x560b07c6af50 con 0x560b07dae2d0 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe125ffb640 10 client.9456 handle_command_reply: tid=0 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 client.9456 shutdown 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 -- 172.21.15.139:0/225171784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 msgr2=0x560b07d4f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07d4f090 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fe138079e60 tx=0x7fe138009750 comp rx=0 tx=0).stop 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 -- 172.21.15.139:0/225171784 shutdown_connections 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x560b07dae2d0 0x560b07d59a30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b07d504b0 0x560b07cfb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560b07d42ad0 0x560b07cfb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe125ffb640 0 client.9456 ms_handle_reset on v2:172.21.15.139:6834/3478297010 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 --2- 172.21.15.139:0/225171784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560b07d25230 0x560b07d4f090 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 -- 172.21.15.139:0/225171784 >> 172.21.15.139:0/225171784 conn(0x560b07cfaf10 msgr2=0x560b07cf0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 -- 172.21.15.139:0/225171784 shutdown_connections 2024-04-25T10:03:08.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 1 -- 172.21.15.139:0/225171784 wait complete. 2024-04-25T10:03:08.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.409+0000 7fe1505dc740 20 client.9456 trim_cache size 0 max 16384 2024-04-25T10:03:08.421 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:03:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T10:03:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T10:03:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 9117, 2024-04-25T10:03:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T10:03:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T10:03:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T10:03:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 4.3667249330000004, 2024-04-25T10:03:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 89, 2024-04-25T10:03:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 121, 2024-04-25T10:03:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 121, 2024-04-25T10:03:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 22.505353396 2024-04-25T10:03:08.422 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:03:08.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.410+0000 7fe14d7f9640 1 -- 172.21.15.139:0/902451842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe13406b4d0 msgr2=0x7fe13406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.410+0000 7fe14d7f9640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe13406b4d0 0x7fe13406d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe1481f72f0 tx=0x7fe13c007ca0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.410+0000 7fe14d7f9640 1 -- 172.21.15.139:0/902451842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe148072db0 msgr2=0x7fe1481506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.410+0000 7fe14d7f9640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe148072db0 0x7fe1481506c0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fe13802ac30 tx=0x7fe138004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:08.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.410+0000 7fe14d7f9640 1 -- 172.21.15.139:0/902451842 shutdown_connections 2024-04-25T10:03:08.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.410+0000 7fe14d7f9640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe13406b4d0 0x7fe13406d990 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.410+0000 7fe14d7f9640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe14815f3a0 0x7fe14815cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.410+0000 7fe14d7f9640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe14815b880 0x7fe1480b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.410+0000 7fe14d7f9640 1 --2- 172.21.15.139:0/902451842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe148072db0 0x7fe1481506c0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.410+0000 7fe14d7f9640 1 -- 172.21.15.139:0/902451842 >> 172.21.15.139:0/902451842 conn(0x7fe1480b23f0 msgr2=0x7fe1480d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.410+0000 7fe14d7f9640 1 -- 172.21.15.139:0/902451842 shutdown_connections 2024-04-25T10:03:08.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.410+0000 7fe14d7f9640 1 -- 172.21.15.139:0/902451842 wait complete. 2024-04-25T10:03:08.425 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:03:08.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.518+0000 7f166a9f9640 1 Processor -- start 2024-04-25T10:03:08.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.518+0000 7f166a9f9640 1 -- start start 2024-04-25T10:03:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f166a9f9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f166415efb0 0x7f16640d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f166a9f9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1664099a50 0x7f1664099e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f166a9f9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16640cfe70 con 0x7f16640cf900 2024-04-25T10:03:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f166a9f9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16640d0a80 con 0x7f1664099a50 2024-04-25T10:03:08.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f166a9f9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166409a3c0 con 0x7f166415efb0 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f1668f96640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1664099a50 0x7f1664099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f1668f96640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1664099a50 0x7f1664099e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52350/0 (socket says 172.21.15.139:52350) 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f1668f96640 1 -- 172.21.15.139:0/1068025427 learned_addr learned my addr 172.21.15.139:0/1068025427 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f16637fe640 1 --1- 172.21.15.139:0/1068025427 >> v1:172.21.15.139:6789/0 conn(0x7f16640cf900 0x7f166406c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34152/0 (socket says 172.21.15.139:34152) 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f1663fff640 1 --2- 172.21.15.139:0/1068025427 >> v2:172.21.15.150:3300/0 conn(0x7f166415efb0 0x7f16640d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f1668f96640 1 -- 172.21.15.139:0/1068025427 >> v2:172.21.15.150:3300/0 conn(0x7f166415efb0 msgr2=0x7f16640d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f1668f96640 1 --2- 172.21.15.139:0/1068025427 >> v2:172.21.15.150:3300/0 conn(0x7f166415efb0 0x7f16640d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f1668f96640 1 -- 172.21.15.139:0/1068025427 >> v1:172.21.15.139:6789/0 conn(0x7f16640cf900 legacy=0x7f166406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f1662ffd640 1 -- 172.21.15.139:0/1068025427 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119092494 0 0) 0x7f16640cfe70 con 0x7f16640cf900 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.519+0000 7f1668f96640 1 -- 172.21.15.139:0/1068025427 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16640d0150 con 0x7f1664099a50 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.520+0000 7f1668f96640 1 --2- 172.21.15.139:0/1068025427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1664099a50 0x7f1664099e80 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f1664072c10 tx=0x7f165c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8e8117f74596a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.520+0000 7f1662ffd640 1 -- 172.21.15.139:0/1068025427 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c036070 con 0x7f1664099a50 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.520+0000 7f1662ffd640 1 -- 172.21.15.139:0/1068025427 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c031600 con 0x7f1664099a50 2024-04-25T10:03:08.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.520+0000 7f1662ffd640 1 -- 172.21.15.139:0/1068025427 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c0319d0 con 0x7f1664099a50 2024-04-25T10:03:08.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.520+0000 7f166a9f9640 1 -- 172.21.15.139:0/1068025427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1664099a50 msgr2=0x7f1664099e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.520+0000 7f166a9f9640 1 --2- 172.21.15.139:0/1068025427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1664099a50 0x7f1664099e80 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f1664072c10 tx=0x7f165c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.520+0000 7f166a9f9640 1 -- 172.21.15.139:0/1068025427 shutdown_connections 2024-04-25T10:03:08.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.520+0000 7f166a9f9640 1 --2- 172.21.15.139:0/1068025427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1664099a50 0x7f1664099e80 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.520+0000 7f166a9f9640 1 --2- 172.21.15.139:0/1068025427 >> v2:172.21.15.150:3300/0 conn(0x7f166415efb0 0x7f16640d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.520+0000 7f166a9f9640 1 -- 172.21.15.139:0/1068025427 >> 172.21.15.139:0/1068025427 conn(0x7f16640d1790 msgr2=0x7f16640d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.520+0000 7f166a9f9640 1 -- 172.21.15.139:0/1068025427 shutdown_connections 2024-04-25T10:03:08.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.520+0000 7f166a9f9640 1 -- 172.21.15.139:0/1068025427 wait complete. 2024-04-25T10:03:08.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.522+0000 7f166a9f9640 1 Processor -- start 2024-04-25T10:03:08.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.522+0000 7f166a9f9640 1 -- start start 2024-04-25T10:03:08.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.522+0000 7f166a9f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1664099a50 0x7f166414ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.522+0000 7f166a9f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16640cf900 0x7f16640c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.522+0000 7f166a9f9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f166415efb0 0x7f16640ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.522+0000 7f166a9f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166406cf50 con 0x7f16640cf900 2024-04-25T10:03:08.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.522+0000 7f166a9f9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16640d0710 con 0x7f166415efb0 2024-04-25T10:03:08.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.522+0000 7f166a9f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16640cbde0 con 0x7f1664099a50 2024-04-25T10:03:08.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f16637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16640cf900 0x7f16640c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f16637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16640cf900 0x7f16640c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52364/0 (socket says 172.21.15.139:52364) 2024-04-25T10:03:08.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f16637fe640 1 -- 172.21.15.139:0/3321605539 learned_addr learned my addr 172.21.15.139:0/3321605539 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:08.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f1668f96640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f166415efb0 0x7f16640ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f1663fff640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1664099a50 0x7f166414ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f16637fe640 1 -- 172.21.15.139:0/3321605539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1664099a50 msgr2=0x7f166414ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f16637fe640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1664099a50 0x7f166414ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f16637fe640 1 -- 172.21.15.139:0/3321605539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f166415efb0 msgr2=0x7f16640ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f16637fe640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f166415efb0 0x7f16640ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f16637fe640 1 -- 172.21.15.139:0/3321605539 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16640b67a0 con 0x7f16640cf900 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f1663fff640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1664099a50 0x7f166414ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f16637fe640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16640cf900 0x7f16640c7950 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f165800c4d0 tx=0x7f165800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f1660ff9640 1 -- 172.21.15.139:0/3321605539 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658018070 con 0x7f16640cf900 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f1660ff9640 1 -- 172.21.15.139:0/3321605539 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1658007500 con 0x7f16640cf900 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f166a9f9640 1 -- 172.21.15.139:0/3321605539 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16640b6a80 con 0x7f16640cf900 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f1660ff9640 1 -- 172.21.15.139:0/3321605539 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658004a20 con 0x7f16640cf900 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.523+0000 7f166a9f9640 1 -- 172.21.15.139:0/3321605539 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16640b7040 con 0x7f16640cf900 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.524+0000 7f166a9f9640 1 -- 172.21.15.139:0/3321605539 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1664074520 con 0x7f16640cf900 2024-04-25T10:03:08.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.525+0000 7f1660ff9640 1 -- 172.21.15.139:0/3321605539 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f165801b070 con 0x7f16640cf900 2024-04-25T10:03:08.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.525+0000 7f1660ff9640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f165006b4d0 0x7f165006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.525+0000 7f1660ff9640 1 -- 172.21.15.139:0/3321605539 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1658089680 con 0x7f16640cf900 2024-04-25T10:03:08.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.525+0000 7f1663fff640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f165006b4d0 0x7f165006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.526+0000 7f1663fff640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f165006b4d0 0x7f165006d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1654006fd0 tx=0x7f1654008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.529+0000 7f1660ff9640 1 -- 172.21.15.139:0/3321605539 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1658057cd0 con 0x7f16640cf900 2024-04-25T10:03:08.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.735+0000 7f166a9f9640 1 -- 172.21.15.139:0/3321605539 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f166409ae50 con 0x7f16640cf900 2024-04-25T10:03:08.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.735+0000 7f1660ff9640 1 -- 172.21.15.139:0/3321605539 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) v1 ==== 53+0+73 (secure 0 0 0) 0x7f165805bd70 con 0x7f16640cf900 2024-04-25T10:03:08.740 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:03:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.737+0000 7f166a9f9640 1 -- 172.21.15.139:0/3321605539 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f165006b4d0 msgr2=0x7f165006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.737+0000 7f166a9f9640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f165006b4d0 0x7f165006d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1654006fd0 tx=0x7f1654008040 comp rx=0 tx=0).stop 2024-04-25T10:03:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.737+0000 7f166a9f9640 1 -- 172.21.15.139:0/3321605539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16640cf900 msgr2=0x7f16640c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.737+0000 7f166a9f9640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16640cf900 0x7f16640c7950 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f165800c4d0 tx=0x7f165800c990 comp rx=0 tx=0).stop 2024-04-25T10:03:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.737+0000 7f166a9f9640 1 -- 172.21.15.139:0/3321605539 shutdown_connections 2024-04-25T10:03:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.737+0000 7f166a9f9640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f16640cf900 0x7f16640c7950 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.737+0000 7f166a9f9640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f165006b4d0 0x7f165006d990 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.737+0000 7f166a9f9640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f166415efb0 0x7f16640ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.737+0000 7f166a9f9640 1 --2- 172.21.15.139:0/3321605539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1664099a50 0x7f166414ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.737+0000 7f166a9f9640 1 -- 172.21.15.139:0/3321605539 >> 172.21.15.139:0/3321605539 conn(0x7f16640d1790 msgr2=0x7f16640708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.737+0000 7f166a9f9640 1 -- 172.21.15.139:0/3321605539 shutdown_connections 2024-04-25T10:03:08.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:08.737+0000 7f166a9f9640 1 -- 172.21.15.139:0/3321605539 wait complete. 2024-04-25T10:03:08.753 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:03:08.753 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:03:08.753 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T10:03:08.753 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:03:08.753 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:08.753 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:03:08.753 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:03:08.753 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:03:08.753 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:08.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:08.753 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T10:03:08.768 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:08.769 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42110sec preferred_lft 42110sec 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:08.770 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:08.771 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:08.771 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T10:03:08.771 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:08.771 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:08.771 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:08.771 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T10:03:08.771 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:08.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:08.771 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:03:08.830 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T10:03:08.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:08.832 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T10:03:08.887 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T10:03:08.887 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:03:08.888 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:03:08.888 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:03:08.888 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:03:08.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:08.888 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T10:03:08.888 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T10:03:08.888 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:03:08.888 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:03:08.888 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:03:08.888 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T10:03:08.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:08.973 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T10:03:09.028 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:03:09.028 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:03:09.028 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:09.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:09.029 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T10:03:09.091 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:09.091 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:09.147 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:09.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:09.147 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:09.202 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:03:09.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:09.202 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:03:09.283 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:03:09.283 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:03:09.290 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:03:09.290 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:03:09.291 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T10:03:09.291 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T10:03:09.291 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T10:03:09.291 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:03:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:03:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:03:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:03:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:03:09.293 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:03:09.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:09.293 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:09.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:09.357 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:03:09.412 INFO:teuthology.orchestra.run.smithi139.stdout:client7738 2024-04-25T10:03:09.412 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:09.413 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:09.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:09.468 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:03:09.525 INFO:teuthology.orchestra.run.smithi139.stdout:./ 2024-04-25T10:03:09.525 INFO:teuthology.orchestra.run.smithi139.stdout:./parent 2024-04-25T10:03:09.525 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/flushed 2024-04-25T10:03:09.525 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/flushed/charlie 2024-04-25T10:03:09.525 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/flushed/alpha 2024-04-25T10:03:09.526 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/unflushed 2024-04-25T10:03:09.526 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/unflushed/jfile 2024-04-25T10:03:09.526 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:09.526 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:09.526 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:09.526 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:09.526 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:03:09.526 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:09.629 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T10:03:09.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:09.638 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:09.638 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:09.638 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:09.638 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:09.638 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:03:09.638 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:09.743 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627776 2024-04-25T10:03:09.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:09.752 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:09.752 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:09.752 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:09.752 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:09.752 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:03:09.752 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:09.855 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627777 2024-04-25T10:03:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:09.864 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:09.864 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:09.864 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:09.864 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:09.864 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:03:09.864 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:09.968 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627780 2024-04-25T10:03:09.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:09.977 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:09.977 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:09.977 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:09.977 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:09.977 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:03:09.977 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:10.084 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627778 2024-04-25T10:03:10.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:10.093 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:10.093 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:10.093 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:10.093 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:10.093 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:03:10.093 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:10.197 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627781 2024-04-25T10:03:10.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:10.207 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:10.207 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:10.207 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:10.207 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:10.207 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:03:10.207 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:10.312 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627782 2024-04-25T10:03:10.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:10.321 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:10.321 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:10.375 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:10.375 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:10.430 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:10.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.562+0000 7faa8d86b640 1 Processor -- start 2024-04-25T10:03:10.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.562+0000 7faa8d86b640 1 -- start start 2024-04-25T10:03:10.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa8d86b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faa8815bdc0 0x7faa8815c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa8d86b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa880b1740 0x7faa880b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa8d86b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8815c880 con 0x7faa88155e30 2024-04-25T10:03:10.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa8d86b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8815d690 con 0x7faa8815bdc0 2024-04-25T10:03:10.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa8d86b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8806d6f0 con 0x7faa880b1740 2024-04-25T10:03:10.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa86ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7faa88155e30 0x7faa8815cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34162/0 (socket says 172.21.15.139:34162) 2024-04-25T10:03:10.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa867fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faa8815bdc0 0x7faa8815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa86ffd640 1 -- 172.21.15.139:0/1833032470 learned_addr learned my addr 172.21.15.139:0/1833032470 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:10.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa877fe640 1 --2- 172.21.15.139:0/1833032470 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa880b1740 0x7faa880b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa867fc640 1 -- 172.21.15.139:0/1833032470 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa880b1740 msgr2=0x7faa880b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa867fc640 1 --2- 172.21.15.139:0/1833032470 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa880b1740 0x7faa880b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa867fc640 1 -- 172.21.15.139:0/1833032470 >> v1:172.21.15.139:6789/0 conn(0x7faa88155e30 legacy=0x7faa8815cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.563+0000 7faa867fc640 1 -- 172.21.15.139:0/1833032470 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8815cb60 con 0x7faa8815bdc0 2024-04-25T10:03:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.564+0000 7faa867fc640 1 --2- 172.21.15.139:0/1833032470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa8815bdc0 0x7faa8815c240 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0097a0 tx=0x7faa7c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfad16f4fb142c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.564+0000 7faa85ffb640 1 -- 172.21.15.139:0/1833032470 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c037070 con 0x7faa8815bdc0 2024-04-25T10:03:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.564+0000 7faa85ffb640 1 -- 172.21.15.139:0/1833032470 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c032440 con 0x7faa8815bdc0 2024-04-25T10:03:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.564+0000 7faa85ffb640 1 -- 172.21.15.139:0/1833032470 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c032810 con 0x7faa8815bdc0 2024-04-25T10:03:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.565+0000 7faa8d86b640 1 -- 172.21.15.139:0/1833032470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa8815bdc0 msgr2=0x7faa8815c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.565+0000 7faa8d86b640 1 --2- 172.21.15.139:0/1833032470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa8815bdc0 0x7faa8815c240 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0097a0 tx=0x7faa7c02a180 comp rx=0 tx=0).stop 2024-04-25T10:03:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.565+0000 7faa8d86b640 1 -- 172.21.15.139:0/1833032470 shutdown_connections 2024-04-25T10:03:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.565+0000 7faa8d86b640 1 --2- 172.21.15.139:0/1833032470 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa880b1740 0x7faa880b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.565+0000 7faa8d86b640 1 --2- 172.21.15.139:0/1833032470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa8815bdc0 0x7faa8815c240 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.565+0000 7faa8d86b640 1 -- 172.21.15.139:0/1833032470 >> 172.21.15.139:0/1833032470 conn(0x7faa88066f20 msgr2=0x7faa8806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.565+0000 7faa8d86b640 1 -- 172.21.15.139:0/1833032470 shutdown_connections 2024-04-25T10:03:10.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.565+0000 7faa8d86b640 1 -- 172.21.15.139:0/1833032470 wait complete. 2024-04-25T10:03:10.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.566+0000 7faa8d86b640 1 Processor -- start 2024-04-25T10:03:10.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.566+0000 7faa8d86b640 1 -- start start 2024-04-25T10:03:10.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa8d86b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa880b1740 0x7faa88149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa8d86b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa88155e30 0x7faa880afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa8d86b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa8815bdc0 0x7faa8815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa8d86b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa880af0e0 con 0x7faa880b1740 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa8d86b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa88156210 con 0x7faa88155e30 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa8d86b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa880af260 con 0x7faa8815bdc0 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa86ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa880b1740 0x7faa88149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa86ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa880b1740 0x7faa88149a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52378/0 (socket says 172.21.15.139:52378) 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa86ffd640 1 -- 172.21.15.139:0/1619691755 learned_addr learned my addr 172.21.15.139:0/1619691755 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa86ffd640 1 -- 172.21.15.139:0/1619691755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa8815bdc0 msgr2=0x7faa8815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa86ffd640 1 --2- 172.21.15.139:0/1619691755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa8815bdc0 0x7faa8815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa86ffd640 1 -- 172.21.15.139:0/1619691755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa88155e30 msgr2=0x7faa880afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa86ffd640 1 --2- 172.21.15.139:0/1619691755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa88155e30 0x7faa880afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.567+0000 7faa86ffd640 1 -- 172.21.15.139:0/1619691755 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa881e6fa0 con 0x7faa880b1740 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.568+0000 7faa86ffd640 1 --2- 172.21.15.139:0/1619691755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa880b1740 0x7faa88149a70 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7faa7800c400 tx=0x7faa7800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.568+0000 7faa6ffff640 1 -- 172.21.15.139:0/1619691755 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78017070 con 0x7faa880b1740 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.568+0000 7faa6ffff640 1 -- 172.21.15.139:0/1619691755 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa78005ce0 con 0x7faa880b1740 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.568+0000 7faa8d86b640 1 -- 172.21.15.139:0/1619691755 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa881e7280 con 0x7faa880b1740 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.568+0000 7faa6ffff640 1 -- 172.21.15.139:0/1619691755 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa780072f0 con 0x7faa880b1740 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.568+0000 7faa8d86b640 1 -- 172.21.15.139:0/1619691755 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa881e7840 con 0x7faa880b1740 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.569+0000 7faa8d86b640 1 -- 172.21.15.139:0/1619691755 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa881e7c70 con 0x7faa880b1740 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.569+0000 7faa6ffff640 1 -- 172.21.15.139:0/1619691755 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa7801a070 con 0x7faa880b1740 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.569+0000 7faa6ffff640 1 --2- 172.21.15.139:0/1619691755 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faa7406b4d0 0x7faa7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.570+0000 7faa6ffff640 1 -- 172.21.15.139:0/1619691755 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7faa780897a0 con 0x7faa880b1740 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.570+0000 7faa867fc640 1 --2- 172.21.15.139:0/1619691755 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faa7406b4d0 0x7faa7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.570+0000 7faa867fc640 1 --2- 172.21.15.139:0/1619691755 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faa7406b4d0 0x7faa7406d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7faa7c034000 tx=0x7faa7c02c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.574+0000 7faa6ffff640 1 -- 172.21.15.139:0/1619691755 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa780584d0 con 0x7faa880b1740 2024-04-25T10:03:10.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.745+0000 7faa8d86b640 1 -- 172.21.15.139:0/1619691755 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa8806b830 con 0x7faa880b1740 2024-04-25T10:03:10.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.746+0000 7faa6ffff640 1 -- 172.21.15.139:0/1619691755 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17131 (secure 0 0 0) 0x7faa78057dd0 con 0x7faa880b1740 2024-04-25T10:03:10.751 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:10.751 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":121,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:03:03.005946+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:48.574891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:49.293101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:6835/1377607596":"2024-04-26T10:03:03.005923+0000","172.21.15.150:6833/1377607596":"2024-04-26T10:03:03.005923+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:10.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.748+0000 7faa8d86b640 1 -- 172.21.15.139:0/1619691755 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faa7406b4d0 msgr2=0x7faa7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.748+0000 7faa8d86b640 1 --2- 172.21.15.139:0/1619691755 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faa7406b4d0 0x7faa7406d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7faa7c034000 tx=0x7faa7c02c000 comp rx=0 tx=0).stop 2024-04-25T10:03:10.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.748+0000 7faa8d86b640 1 -- 172.21.15.139:0/1619691755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa880b1740 msgr2=0x7faa88149a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.748+0000 7faa8d86b640 1 --2- 172.21.15.139:0/1619691755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa880b1740 0x7faa88149a70 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7faa7800c400 tx=0x7faa7800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.748+0000 7faa8d86b640 1 -- 172.21.15.139:0/1619691755 shutdown_connections 2024-04-25T10:03:10.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.748+0000 7faa8d86b640 1 --2- 172.21.15.139:0/1619691755 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7faa7406b4d0 0x7faa7406d990 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.748+0000 7faa8d86b640 1 --2- 172.21.15.139:0/1619691755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa8815bdc0 0x7faa8815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.748+0000 7faa8d86b640 1 --2- 172.21.15.139:0/1619691755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa88155e30 0x7faa880afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.748+0000 7faa8d86b640 1 --2- 172.21.15.139:0/1619691755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa880b1740 0x7faa88149a70 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.748+0000 7faa8d86b640 1 -- 172.21.15.139:0/1619691755 >> 172.21.15.139:0/1619691755 conn(0x7faa88066f20 msgr2=0x7faa8815eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.748+0000 7faa8d86b640 1 -- 172.21.15.139:0/1619691755 shutdown_connections 2024-04-25T10:03:10.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.748+0000 7faa8d86b640 1 -- 172.21.15.139:0/1619691755 wait complete. 2024-04-25T10:03:10.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:48.574891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:49.293101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:10.764 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:10.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.859+0000 7f4dbb676640 1 Processor -- start 2024-04-25T10:03:10.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.860+0000 7f4dbb676640 1 -- start start 2024-04-25T10:03:10.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.860+0000 7f4dbb676640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db415dbf0 0x7f4db41617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.860+0000 7f4dbb676640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4db40cee50 0x7f4db40d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.860+0000 7f4dbb676640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db4161dc0 con 0x7f4db40b9370 2024-04-25T10:03:10.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.860+0000 7f4dbb676640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db40d1ee0 con 0x7f4db40cee50 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.860+0000 7f4dbb676640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41620a0 con 0x7f4db415dbf0 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.861+0000 7f4db9c13640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4db40cee50 0x7f4db40d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.861+0000 7f4db9c13640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4db40cee50 0x7f4db40d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52392/0 (socket says 172.21.15.139:52392) 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.861+0000 7f4db9c13640 1 -- 172.21.15.139:0/1132394516 learned_addr learned my addr 172.21.15.139:0/1132394516 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.861+0000 7f4db9c13640 1 -- 172.21.15.139:0/1132394516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db415dbf0 msgr2=0x7f4db41617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.861+0000 7f4db9c13640 1 --2- 172.21.15.139:0/1132394516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db415dbf0 0x7f4db41617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.861+0000 7f4db9c13640 1 -- 172.21.15.139:0/1132394516 >> v1:172.21.15.150:6789/0 conn(0x7f4db40b9370 legacy=0x7f4db40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.861+0000 7f4db9c13640 1 -- 172.21.15.139:0/1132394516 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db4162380 con 0x7f4db40cee50 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.861+0000 7f4db9c13640 1 --2- 172.21.15.139:0/1132394516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40cee50 0x7f4db40d2370 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009e50 tx=0x7f4dac02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ff9302581d3e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.862+0000 7f4dabfff640 1 -- 172.21.15.139:0/1132394516 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac036070 con 0x7f4db40cee50 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.862+0000 7f4dabfff640 1 -- 172.21.15.139:0/1132394516 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac031600 con 0x7f4db40cee50 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.862+0000 7f4dabfff640 1 -- 172.21.15.139:0/1132394516 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac0319d0 con 0x7f4db40cee50 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.862+0000 7f4dbb676640 1 -- 172.21.15.139:0/1132394516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40cee50 msgr2=0x7f4db40d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.862+0000 7f4dbb676640 1 --2- 172.21.15.139:0/1132394516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40cee50 0x7f4db40d2370 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009e50 tx=0x7f4dac02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.862+0000 7f4dbb676640 1 -- 172.21.15.139:0/1132394516 shutdown_connections 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.862+0000 7f4dbb676640 1 --2- 172.21.15.139:0/1132394516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40cee50 0x7f4db40d2370 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.862+0000 7f4dbb676640 1 --2- 172.21.15.139:0/1132394516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db415dbf0 0x7f4db41617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.862+0000 7f4dbb676640 1 -- 172.21.15.139:0/1132394516 >> 172.21.15.139:0/1132394516 conn(0x7f4db40b5480 msgr2=0x7f4db40b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.862+0000 7f4dbb676640 1 -- 172.21.15.139:0/1132394516 shutdown_connections 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.862+0000 7f4dbb676640 1 -- 172.21.15.139:0/1132394516 wait complete. 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.863+0000 7f4dbb676640 1 Processor -- start 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.863+0000 7f4dbb676640 1 -- start start 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.864+0000 7f4dbb676640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db40b9370 0x7f4db415d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.864+0000 7f4dbb676640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40cee50 0x7f4db4147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.864+0000 7f4dbb676640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db415dbf0 0x7f4db41e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.864+0000 7f4dbb676640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db415dfd0 con 0x7f4db40cee50 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.864+0000 7f4dbb676640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40b97d0 con 0x7f4db415dbf0 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.864+0000 7f4dbb676640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4160be0 con 0x7f4db40b9370 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.864+0000 7f4db8c11640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40cee50 0x7f4db4147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.864+0000 7f4db8c11640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40cee50 0x7f4db4147fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52402/0 (socket says 172.21.15.139:52402) 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.864+0000 7f4db8c11640 1 -- 172.21.15.139:0/1617929289 learned_addr learned my addr 172.21.15.139:0/1617929289 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.864+0000 7f4db9c13640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db415dbf0 0x7f4db41e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.864+0000 7f4db9412640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db40b9370 0x7f4db415d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.865+0000 7f4db8c11640 1 -- 172.21.15.139:0/1617929289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db40b9370 msgr2=0x7f4db415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.865+0000 7f4db8c11640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db40b9370 0x7f4db415d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.865+0000 7f4db8c11640 1 -- 172.21.15.139:0/1617929289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db415dbf0 msgr2=0x7f4db41e9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.865+0000 7f4db8c11640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db415dbf0 0x7f4db41e9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.865+0000 7f4db8c11640 1 -- 172.21.15.139:0/1617929289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db4148520 con 0x7f4db40cee50 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.865+0000 7f4db9412640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db40b9370 0x7f4db415d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.865+0000 7f4db8c11640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40cee50 0x7f4db4147fe0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f4db000c430 tx=0x7f4db000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.865+0000 7f4da9ffb640 1 -- 172.21.15.139:0/1617929289 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0011790 con 0x7f4db40cee50 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.865+0000 7f4da9ffb640 1 -- 172.21.15.139:0/1617929289 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db0017070 con 0x7f4db40cee50 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.865+0000 7f4da9ffb640 1 -- 172.21.15.139:0/1617929289 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0011b70 con 0x7f4db40cee50 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.865+0000 7f4dbb676640 1 -- 172.21.15.139:0/1617929289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db415ffd0 con 0x7f4db40cee50 2024-04-25T10:03:10.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.865+0000 7f4dbb676640 1 -- 172.21.15.139:0/1617929289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db4160590 con 0x7f4db40cee50 2024-04-25T10:03:10.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.866+0000 7f4dbb676640 1 -- 172.21.15.139:0/1617929289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db40cdf90 con 0x7f4db40cee50 2024-04-25T10:03:10.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.866+0000 7f4da9ffb640 1 -- 172.21.15.139:0/1617929289 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4db0011d10 con 0x7f4db40cee50 2024-04-25T10:03:10.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.867+0000 7f4da9ffb640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d9806b4d0 0x7f4d9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.867+0000 7f4db9412640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d9806b4d0 0x7f4d9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.867+0000 7f4da9ffb640 1 -- 172.21.15.139:0/1617929289 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4db008a180 con 0x7f4db40cee50 2024-04-25T10:03:10.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.867+0000 7f4db9412640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d9806b4d0 0x7f4d9806d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009890 tx=0x7f4da4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:10.871+0000 7f4da9ffb640 1 -- 172.21.15.139:0/1617929289 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4db40cdf90 con 0x7f4db40cee50 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.041+0000 7f4dbb676640 1 -- 172.21.15.139:0/1617929289 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4db4074520 con 0x7f4db40cee50 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.041+0000 7f4da9ffb640 1 -- 172.21.15.139:0/1617929289 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4db4148520 con 0x7f4db40cee50 2024-04-25T10:03:11.046 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.043+0000 7f4dbb676640 1 -- 172.21.15.139:0/1617929289 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d9806b4d0 msgr2=0x7f4d9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.043+0000 7f4dbb676640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d9806b4d0 0x7f4d9806d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009890 tx=0x7f4da4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.043+0000 7f4dbb676640 1 -- 172.21.15.139:0/1617929289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40cee50 msgr2=0x7f4db4147fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.043+0000 7f4dbb676640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40cee50 0x7f4db4147fe0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f4db000c430 tx=0x7f4db000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.043+0000 7f4dbb676640 1 -- 172.21.15.139:0/1617929289 shutdown_connections 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.043+0000 7f4dbb676640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4db40cee50 0x7f4db4147fe0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.043+0000 7f4dbb676640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f4d9806b4d0 0x7f4d9806d990 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.043+0000 7f4dbb676640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4db415dbf0 0x7f4db41e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.043+0000 7f4dbb676640 1 --2- 172.21.15.139:0/1617929289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4db40b9370 0x7f4db415d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.043+0000 7f4dbb676640 1 -- 172.21.15.139:0/1617929289 >> 172.21.15.139:0/1617929289 conn(0x7f4db40b5480 msgr2=0x7f4db40cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.043+0000 7f4dbb676640 1 -- 172.21.15.139:0/1617929289 shutdown_connections 2024-04-25T10:03:11.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.043+0000 7f4dbb676640 1 -- 172.21.15.139:0/1617929289 wait complete. 2024-04-25T10:03:11.059 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:11.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.154+0000 7f1552bd5640 1 Processor -- start 2024-04-25T10:03:11.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.154+0000 7f1552bd5640 1 -- start start 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.155+0000 7f1552bd5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f154c150ed0 0x7f154c15da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.155+0000 7f1552bd5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c15a480 con 0x7f154c15dfb0 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.155+0000 7f1552bd5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c15a760 con 0x7f154c0cdd20 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.155+0000 7f1552bd5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154c15aa40 con 0x7f154c150ed0 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.155+0000 7f1551172640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f154c15dfb0 0x7f154c0711c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34170/0 (socket says 172.21.15.139:34170) 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.155+0000 7f1551172640 1 -- 172.21.15.139:0/3788201781 learned_addr learned my addr 172.21.15.139:0/3788201781 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.155+0000 7f154bfff640 1 --2- 172.21.15.139:0/3788201781 >> v2:172.21.15.150:3300/0 conn(0x7f154c150ed0 0x7f154c15da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.155+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121317846 0 0) 0x7f154c15a480 con 0x7f154c15dfb0 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.155+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1534003520 con 0x7f154c15dfb0 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1707598098 0 0) 0x7f1534003520 con 0x7f154c15dfb0 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 >> v2:172.21.15.150:3300/0 conn(0x7f154c150ed0 msgr2=0x7f154c15da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 --2- 172.21.15.139:0/3788201781 >> v2:172.21.15.150:3300/0 conn(0x7f154c150ed0 0x7f154c15da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 >> v1:172.21.15.150:6789/0 conn(0x7f154c0cdd20 legacy=0x7f154c0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c15ad20 con 0x7f154c15dfb0 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f15440019f0 con 0x7f154c15dfb0 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 >> v1:172.21.15.139:6789/0 conn(0x7f154c15dfb0 legacy=0x7f154c0711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 --2- 172.21.15.139:0/3788201781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15340039b0 0x7f1534003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 --2- 172.21.15.139:0/3788201781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534004800 0x7f1534004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 --2- 172.21.15.139:0/3788201781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340051c0 0x7f153400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c15aa40 con 0x7f15340039b0 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153400c160 con 0x7f15340051c0 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153400c360 con 0x7f1534004800 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f1550971640 1 --2- 172.21.15.139:0/3788201781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15340039b0 0x7f1534003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f154bfff640 1 --2- 172.21.15.139:0/3788201781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534004800 0x7f1534004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f1551172640 1 --2- 172.21.15.139:0/3788201781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340051c0 0x7f153400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f1550971640 1 -- 172.21.15.139:0/3788201781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534004800 msgr2=0x7f1534004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f1550971640 1 --2- 172.21.15.139:0/3788201781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534004800 0x7f1534004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f1550971640 1 -- 172.21.15.139:0/3788201781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340051c0 msgr2=0x7f153400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f1550971640 1 --2- 172.21.15.139:0/3788201781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340051c0 0x7f153400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f1550971640 1 -- 172.21.15.139:0/3788201781 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1534003520 con 0x7f15340039b0 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f1551172640 1 --2- 172.21.15.139:0/3788201781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340051c0 0x7f153400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.156+0000 7f1551172640 1 -- 172.21.15.139:0/3788201781 reap_dead start 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.157+0000 7f1550971640 1 --2- 172.21.15.139:0/3788201781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15340039b0 0x7f1534003dd0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f153c001750 tx=0x7f1534005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08dfd5d11b485c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.157+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c001370 con 0x7f15340039b0 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.157+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c0052d0 con 0x7f15340039b0 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.157+0000 7f154b7fe640 1 -- 172.21.15.139:0/3788201781 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c02abc0 con 0x7f15340039b0 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.157+0000 7f1552bd5640 1 -- 172.21.15.139:0/3788201781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15340039b0 msgr2=0x7f1534003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.157+0000 7f1552bd5640 1 --2- 172.21.15.139:0/3788201781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15340039b0 0x7f1534003dd0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f153c001750 tx=0x7f1534005830 comp rx=0 tx=0).stop 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.157+0000 7f1552bd5640 1 -- 172.21.15.139:0/3788201781 shutdown_connections 2024-04-25T10:03:11.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.157+0000 7f1552bd5640 1 --2- 172.21.15.139:0/3788201781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f15340039b0 0x7f1534003dd0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.157+0000 7f1552bd5640 1 -- 172.21.15.139:0/3788201781 >> 172.21.15.139:0/3788201781 conn(0x7f154c067000 msgr2=0x7f154c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.157+0000 7f1552bd5640 1 -- 172.21.15.139:0/3788201781 shutdown_connections 2024-04-25T10:03:11.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.157+0000 7f1552bd5640 1 -- 172.21.15.139:0/3788201781 wait complete. 2024-04-25T10:03:11.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.159+0000 7f1552bd5640 1 Processor -- start 2024-04-25T10:03:11.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.159+0000 7f1552bd5640 1 -- start start 2024-04-25T10:03:11.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.159+0000 7f1552bd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15340039b0 0x7f154c152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.159+0000 7f1552bd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f154c1514e0 0x7f154c151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.159+0000 7f1552bd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f154c1e5d70 0x7f154c1e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.159+0000 7f1552bd5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0b3960 con 0x7f154c1514e0 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.159+0000 7f1552bd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c151360 con 0x7f154c1e5d70 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.159+0000 7f1552bd5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c15da70 con 0x7f15340039b0 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f154bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f154c1514e0 0x7f154c151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f154bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f154c1514e0 0x7f154c151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52422/0 (socket says 172.21.15.139:52422) 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f154bfff640 1 -- 172.21.15.139:0/4258602766 learned_addr learned my addr 172.21.15.139:0/4258602766 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f1551172640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f154c1e5d70 0x7f154c1e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f1550971640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15340039b0 0x7f154c152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f154bfff640 1 -- 172.21.15.139:0/4258602766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15340039b0 msgr2=0x7f154c152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f154bfff640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15340039b0 0x7f154c152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f154bfff640 1 -- 172.21.15.139:0/4258602766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f154c1e5d70 msgr2=0x7f154c1e61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f154bfff640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f154c1e5d70 0x7f154c1e61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f154bfff640 1 -- 172.21.15.139:0/4258602766 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c1581b0 con 0x7f154c1514e0 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f1551172640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f154c1e5d70 0x7f154c1e61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f154bfff640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f154c1514e0 0x7f154c151940 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f1540011730 tx=0x7f1540011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f15497fa640 1 -- 172.21.15.139:0/4258602766 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154000f9c0 con 0x7f154c1514e0 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f15497fa640 1 -- 172.21.15.139:0/4258602766 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154000fb60 con 0x7f154c1514e0 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f1552bd5640 1 -- 172.21.15.139:0/4258602766 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c151f80 con 0x7f154c1514e0 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f15497fa640 1 -- 172.21.15.139:0/4258602766 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540015460 con 0x7f154c1514e0 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.160+0000 7f1552bd5640 1 -- 172.21.15.139:0/4258602766 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c1460c0 con 0x7f154c1514e0 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.161+0000 7f1552bd5640 1 -- 172.21.15.139:0/4258602766 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c1e6730 con 0x7f154c1514e0 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.161+0000 7f15497fa640 1 -- 172.21.15.139:0/4258602766 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1540015600 con 0x7f154c1514e0 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.162+0000 7f15497fa640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f153806b4d0 0x7f153806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.162+0000 7f15497fa640 1 -- 172.21.15.139:0/4258602766 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f154008acb0 con 0x7f154c1514e0 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.162+0000 7f1550971640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f153806b4d0 0x7f153806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.162+0000 7f1550971640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f153806b4d0 0x7f153806d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f153c032000 tx=0x7f153c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.166+0000 7f15497fa640 1 -- 172.21.15.139:0/4258602766 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1540019030 con 0x7f154c1514e0 2024-04-25T10:03:11.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.334+0000 7f1552bd5640 1 -- 172.21.15.139:0/4258602766 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f154c074520 con 0x7f154c1514e0 2024-04-25T10:03:11.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.335+0000 7f15497fa640 1 -- 172.21.15.139:0/4258602766 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f15400599e0 con 0x7f154c1514e0 2024-04-25T10:03:11.340 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T10:03:11.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.337+0000 7f1552bd5640 1 -- 172.21.15.139:0/4258602766 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f153806b4d0 msgr2=0x7f153806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.337+0000 7f1552bd5640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f153806b4d0 0x7f153806d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f153c032000 tx=0x7f153c02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:11.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.337+0000 7f1552bd5640 1 -- 172.21.15.139:0/4258602766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f154c1514e0 msgr2=0x7f154c151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.337+0000 7f1552bd5640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f154c1514e0 0x7f154c151940 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f1540011730 tx=0x7f1540011bf0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.337+0000 7f1552bd5640 1 -- 172.21.15.139:0/4258602766 shutdown_connections 2024-04-25T10:03:11.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.337+0000 7f1552bd5640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f154c1514e0 0x7f154c151940 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.337+0000 7f1552bd5640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f153806b4d0 0x7f153806d990 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.337+0000 7f1552bd5640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f154c1e5d70 0x7f154c1e61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.337+0000 7f1552bd5640 1 --2- 172.21.15.139:0/4258602766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15340039b0 0x7f154c152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.337+0000 7f1552bd5640 1 -- 172.21.15.139:0/4258602766 >> 172.21.15.139:0/4258602766 conn(0x7f154c067000 msgr2=0x7f154c15b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.337+0000 7f1552bd5640 1 -- 172.21.15.139:0/4258602766 shutdown_connections 2024-04-25T10:03:11.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.337+0000 7f1552bd5640 1 -- 172.21.15.139:0/4258602766 wait complete. 2024-04-25T10:03:11.352 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:11.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.449+0000 7ff0c9b14640 1 Processor -- start 2024-04-25T10:03:11.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.449+0000 7ff0c9b14640 1 -- start start 2024-04-25T10:03:11.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c9b14640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c415dd90 0x7ff0c40ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c9b14640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff0c40cf3e0 0x7ff0c406ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c9b14640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff0c406f130 0x7ff0c40cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c9b14640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c406ab60 con 0x7ff0c40cf3e0 2024-04-25T10:03:11.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c9b14640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c40cf940 con 0x7ff0c406f130 2024-04-25T10:03:11.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c9b14640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c406adb0 con 0x7ff0c415dd90 2024-04-25T10:03:11.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c2ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff0c40cf3e0 0x7ff0c406ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c2ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff0c40cf3e0 0x7ff0c406ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52424/0 (socket says 172.21.15.139:52424) 2024-04-25T10:03:11.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c2ffd640 1 -- 172.21.15.139:0/2739336743 learned_addr learned my addr 172.21.15.139:0/2739336743 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:11.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c3fff640 1 --2- 172.21.15.139:0/2739336743 >> v2:172.21.15.150:3300/0 conn(0x7ff0c406f130 0x7ff0c40cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c37fe640 1 --2- 172.21.15.139:0/2739336743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c415dd90 0x7ff0c40ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c2ffd640 1 -- 172.21.15.139:0/2739336743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c415dd90 msgr2=0x7ff0c40ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c2ffd640 1 --2- 172.21.15.139:0/2739336743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c415dd90 0x7ff0c40ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c2ffd640 1 -- 172.21.15.139:0/2739336743 >> v2:172.21.15.150:3300/0 conn(0x7ff0c406f130 msgr2=0x7ff0c40cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c2ffd640 1 --2- 172.21.15.139:0/2739336743 >> v2:172.21.15.150:3300/0 conn(0x7ff0c406f130 0x7ff0c40cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.450+0000 7ff0c2ffd640 1 -- 172.21.15.139:0/2739336743 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c406b090 con 0x7ff0c40cf3e0 2024-04-25T10:03:11.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.451+0000 7ff0c2ffd640 1 --2- 172.21.15.139:0/2739336743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0c40cf3e0 0x7ff0c406ebf0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4072dc0 tx=0x7ff0b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c46faaf3ece0cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.451+0000 7ff0c27fc640 1 -- 172.21.15.139:0/2739336743 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8037070 con 0x7ff0c40cf3e0 2024-04-25T10:03:11.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.451+0000 7ff0c27fc640 1 -- 172.21.15.139:0/2739336743 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b8032440 con 0x7ff0c40cf3e0 2024-04-25T10:03:11.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.451+0000 7ff0c27fc640 1 -- 172.21.15.139:0/2739336743 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8032810 con 0x7ff0c40cf3e0 2024-04-25T10:03:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.452+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2739336743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0c40cf3e0 msgr2=0x7ff0c406ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.452+0000 7ff0c9b14640 1 --2- 172.21.15.139:0/2739336743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0c40cf3e0 0x7ff0c406ebf0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4072dc0 tx=0x7ff0b802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.452+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2739336743 shutdown_connections 2024-04-25T10:03:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.452+0000 7ff0c9b14640 1 --2- 172.21.15.139:0/2739336743 >> v2:172.21.15.150:3300/0 conn(0x7ff0c406f130 0x7ff0c40cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.452+0000 7ff0c9b14640 1 --2- 172.21.15.139:0/2739336743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0c40cf3e0 0x7ff0c406ebf0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.452+0000 7ff0c9b14640 1 --2- 172.21.15.139:0/2739336743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c415dd90 0x7ff0c40ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.452+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2739336743 >> 172.21.15.139:0/2739336743 conn(0x7ff0c40cfd80 msgr2=0x7ff0c40d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.452+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2739336743 shutdown_connections 2024-04-25T10:03:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.452+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2739336743 wait complete. 2024-04-25T10:03:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.453+0000 7ff0c9b14640 1 Processor -- start 2024-04-25T10:03:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.453+0000 7ff0c9b14640 1 -- start start 2024-04-25T10:03:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c9b14640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c406f130 0x7ff0c4146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c9b14640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c40cf3e0 0x7ff0c41e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c9b14640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0c415dd90 0x7ff0c41e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c9b14640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4072620 con 0x7ff0c415dd90 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c9b14640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c406f5c0 con 0x7ff0c40cf3e0 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c9b14640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c406b890 con 0x7ff0c406f130 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0c415dd90 0x7ff0c41e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0c415dd90 0x7ff0c41e8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52430/0 (socket says 172.21.15.139:52430) 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c3fff640 1 -- 172.21.15.139:0/2685139162 learned_addr learned my addr 172.21.15.139:0/2685139162 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c37fe640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c406f130 0x7ff0c4146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c2ffd640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c40cf3e0 0x7ff0c41e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c3fff640 1 -- 172.21.15.139:0/2685139162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c406f130 msgr2=0x7ff0c4146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c3fff640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c406f130 0x7ff0c4146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c3fff640 1 -- 172.21.15.139:0/2685139162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c40cf3e0 msgr2=0x7ff0c41e88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c3fff640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c40cf3e0 0x7ff0c41e88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c3fff640 1 -- 172.21.15.139:0/2685139162 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4147850 con 0x7ff0c415dd90 2024-04-25T10:03:11.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.454+0000 7ff0c3fff640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0c415dd90 0x7ff0c41e8e00 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc007d10 tx=0x7ff0bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.455+0000 7ff0abfff640 1 -- 172.21.15.139:0/2685139162 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc014420 con 0x7ff0c415dd90 2024-04-25T10:03:11.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.455+0000 7ff0abfff640 1 -- 172.21.15.139:0/2685139162 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc019070 con 0x7ff0c415dd90 2024-04-25T10:03:11.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.455+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2685139162 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c4147b30 con 0x7ff0c415dd90 2024-04-25T10:03:11.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.455+0000 7ff0abfff640 1 -- 172.21.15.139:0/2685139162 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc014820 con 0x7ff0c415dd90 2024-04-25T10:03:11.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.455+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2685139162 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c41480f0 con 0x7ff0c415dd90 2024-04-25T10:03:11.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.456+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2685139162 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c4074520 con 0x7ff0c415dd90 2024-04-25T10:03:11.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.456+0000 7ff0abfff640 1 -- 172.21.15.139:0/2685139162 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0bc0149c0 con 0x7ff0c415dd90 2024-04-25T10:03:11.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.456+0000 7ff0abfff640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff0b006b4d0 0x7ff0b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.457+0000 7ff0abfff640 1 -- 172.21.15.139:0/2685139162 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff0bc08a9a0 con 0x7ff0c415dd90 2024-04-25T10:03:11.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.457+0000 7ff0c37fe640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff0b006b4d0 0x7ff0b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.457+0000 7ff0c37fe640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff0b006b4d0 0x7ff0b006d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40079a0 tx=0x7ff0b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.461+0000 7ff0abfff640 1 -- 172.21.15.139:0/2685139162 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0bc059080 con 0x7ff0c415dd90 2024-04-25T10:03:11.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.629+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2685139162 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0c41609a0 con 0x7ff0c415dd90 2024-04-25T10:03:11.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.630+0000 7ff0abfff640 1 -- 172.21.15.139:0/2685139162 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0bc05d120 con 0x7ff0c415dd90 2024-04-25T10:03:11.635 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T10:03:11.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.632+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2685139162 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff0b006b4d0 msgr2=0x7ff0b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.632+0000 7ff0c9b14640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff0b006b4d0 0x7ff0b006d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40079a0 tx=0x7ff0b4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:11.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.632+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2685139162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0c415dd90 msgr2=0x7ff0c41e8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.632+0000 7ff0c9b14640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0c415dd90 0x7ff0c41e8e00 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc007d10 tx=0x7ff0bc00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.632+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2685139162 shutdown_connections 2024-04-25T10:03:11.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.632+0000 7ff0c9b14640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0c415dd90 0x7ff0c41e8e00 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.632+0000 7ff0c9b14640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff0b006b4d0 0x7ff0b006d990 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.632+0000 7ff0c9b14640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c40cf3e0 0x7ff0c41e88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.632+0000 7ff0c9b14640 1 --2- 172.21.15.139:0/2685139162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c406f130 0x7ff0c4146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.632+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2685139162 >> 172.21.15.139:0/2685139162 conn(0x7ff0c40cfd80 msgr2=0x7ff0c415c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.632+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2685139162 shutdown_connections 2024-04-25T10:03:11.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.632+0000 7ff0c9b14640 1 -- 172.21.15.139:0/2685139162 wait complete. 2024-04-25T10:03:11.647 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:11.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.741+0000 7f416b5a5640 1 Processor -- start 2024-04-25T10:03:11.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.741+0000 7f416b5a5640 1 -- start start 2024-04-25T10:03:11.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.742+0000 7f416b5a5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4164144090 0x7f41640b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.742+0000 7f416b5a5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f416406ea40 0x7f416415e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.742+0000 7f416b5a5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41640b2a50 con 0x7f41640cc9e0 2024-04-25T10:03:11.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.742+0000 7f416b5a5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416415b090 con 0x7f4164144090 2024-04-25T10:03:11.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.742+0000 7f416b5a5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416415b370 con 0x7f416406ea40 2024-04-25T10:03:11.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.742+0000 7f4169341640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4164144090 0x7f41640b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.742+0000 7f4169341640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4164144090 0x7f41640b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52440/0 (socket says 172.21.15.139:52440) 2024-04-25T10:03:11.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.742+0000 7f4169341640 1 -- 172.21.15.139:0/1257985613 learned_addr learned my addr 172.21.15.139:0/1257985613 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:11.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.742+0000 7f4169341640 1 -- 172.21.15.139:0/1257985613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f416406ea40 msgr2=0x7f416415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.742+0000 7f4169341640 1 --2- 172.21.15.139:0/1257985613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f416406ea40 0x7f416415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.742+0000 7f4169341640 1 -- 172.21.15.139:0/1257985613 >> v1:172.21.15.150:6789/0 conn(0x7f41640cc9e0 legacy=0x7f416406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.742+0000 7f4169341640 1 -- 172.21.15.139:0/1257985613 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416415b650 con 0x7f4164144090 2024-04-25T10:03:11.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.743+0000 7f4169341640 1 --2- 172.21.15.139:0/1257985613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4164144090 0x7f41640b31a0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f416000bb10 tx=0x7f416002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f378a87d1f401c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.743+0000 7f415bfff640 1 -- 172.21.15.139:0/1257985613 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160032710 con 0x7f4164144090 2024-04-25T10:03:11.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.743+0000 7f415bfff640 1 -- 172.21.15.139:0/1257985613 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41600328b0 con 0x7f4164144090 2024-04-25T10:03:11.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.743+0000 7f415bfff640 1 -- 172.21.15.139:0/1257985613 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160032ca0 con 0x7f4164144090 2024-04-25T10:03:11.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.743+0000 7f416b5a5640 1 -- 172.21.15.139:0/1257985613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4164144090 msgr2=0x7f41640b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.743+0000 7f416b5a5640 1 --2- 172.21.15.139:0/1257985613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4164144090 0x7f41640b31a0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f416000bb10 tx=0x7f416002c850 comp rx=0 tx=0).stop 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.743+0000 7f416b5a5640 1 -- 172.21.15.139:0/1257985613 shutdown_connections 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.743+0000 7f416b5a5640 1 --2- 172.21.15.139:0/1257985613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f416406ea40 0x7f416415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.743+0000 7f416b5a5640 1 --2- 172.21.15.139:0/1257985613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4164144090 0x7f41640b31a0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.743+0000 7f416b5a5640 1 -- 172.21.15.139:0/1257985613 >> 172.21.15.139:0/1257985613 conn(0x7f4164067000 msgr2=0x7f416406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.744+0000 7f416b5a5640 1 -- 172.21.15.139:0/1257985613 shutdown_connections 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.744+0000 7f416b5a5640 1 -- 172.21.15.139:0/1257985613 wait complete. 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.745+0000 7f416b5a5640 1 Processor -- start 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.745+0000 7f416b5a5640 1 -- start start 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.745+0000 7f416b5a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f416406ea40 0x7f41640a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.745+0000 7f416b5a5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41640cc9e0 0x7f41641ef620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.745+0000 7f416b5a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4164144090 0x7f416415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.745+0000 7f416b5a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4164144470 con 0x7f416406ea40 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.745+0000 7f416b5a5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416415bc10 con 0x7f4164144090 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.745+0000 7f416b5a5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416415ad80 con 0x7f41640cc9e0 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.745+0000 7f4169341640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f416406ea40 0x7f41640a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.745+0000 7f4169341640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f416406ea40 0x7f41640a84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52454/0 (socket says 172.21.15.139:52454) 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.745+0000 7f4169341640 1 -- 172.21.15.139:0/297091754 learned_addr learned my addr 172.21.15.139:0/297091754 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f4169341640 1 -- 172.21.15.139:0/297091754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41640cc9e0 msgr2=0x7f41641ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f4169b42640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4164144090 0x7f416415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f4168b40640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41640cc9e0 0x7f41641ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f4169341640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41640cc9e0 0x7f41641ef620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f4169341640 1 -- 172.21.15.139:0/297091754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4164144090 msgr2=0x7f416415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f4169341640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4164144090 0x7f416415a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f4169341640 1 -- 172.21.15.139:0/297091754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41640a8a90 con 0x7f416406ea40 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f4169341640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f416406ea40 0x7f41640a84b0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f416000b2c0 tx=0x7f41600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f4159ffb640 1 -- 172.21.15.139:0/297091754 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160044070 con 0x7f416406ea40 2024-04-25T10:03:11.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f4159ffb640 1 -- 172.21.15.139:0/297091754 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4160003810 con 0x7f416406ea40 2024-04-25T10:03:11.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f416b5a5640 1 -- 172.21.15.139:0/297091754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416413e4c0 con 0x7f416406ea40 2024-04-25T10:03:11.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f4159ffb640 1 -- 172.21.15.139:0/297091754 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160003c00 con 0x7f416406ea40 2024-04-25T10:03:11.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.746+0000 7f416b5a5640 1 -- 172.21.15.139:0/297091754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416413ea80 con 0x7f416406ea40 2024-04-25T10:03:11.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.747+0000 7f416b5a5640 1 -- 172.21.15.139:0/297091754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41640cdf50 con 0x7f416406ea40 2024-04-25T10:03:11.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.748+0000 7f4159ffb640 1 -- 172.21.15.139:0/297091754 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4160047070 con 0x7f416406ea40 2024-04-25T10:03:11.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.748+0000 7f4159ffb640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f414806b4d0 0x7f414806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.748+0000 7f4159ffb640 1 -- 172.21.15.139:0/297091754 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f41600ad950 con 0x7f416406ea40 2024-04-25T10:03:11.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.748+0000 7f4168b40640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f414806b4d0 0x7f414806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.749+0000 7f4168b40640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f414806b4d0 0x7f414806d990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f415400a730 tx=0x7f4154008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.752+0000 7f4159ffb640 1 -- 172.21.15.139:0/297091754 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41640cdf50 con 0x7f416406ea40 2024-04-25T10:03:11.803 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T10:03:11.798+0000 7fb5f3191640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:11.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.933+0000 7f416b5a5640 1 -- 172.21.15.139:0/297091754 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4164074520 con 0x7f416406ea40 2024-04-25T10:03:11.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.934+0000 7f4159ffb640 1 -- 172.21.15.139:0/297091754 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v121) v1 ==== 80+0+124 (secure 0 0 0) 0x7f41640b2a50 con 0x7f416406ea40 2024-04-25T10:03:11.939 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.150:6835/1377607596 2024-04-26T10:03:03.005923+0000 2024-04-25T10:03:11.939 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.150:6833/1377607596 2024-04-26T10:03:03.005923+0000 2024-04-25T10:03:11.939 INFO:teuthology.orchestra.run.smithi139.stderr:listed 2 entries 2024-04-25T10:03:11.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.936+0000 7f416b5a5640 1 -- 172.21.15.139:0/297091754 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f414806b4d0 msgr2=0x7f414806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.936+0000 7f416b5a5640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f414806b4d0 0x7f414806d990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f415400a730 tx=0x7f4154008040 comp rx=0 tx=0).stop 2024-04-25T10:03:11.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.936+0000 7f416b5a5640 1 -- 172.21.15.139:0/297091754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f416406ea40 msgr2=0x7f41640a84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.936+0000 7f416b5a5640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f416406ea40 0x7f41640a84b0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f416000b2c0 tx=0x7f41600046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.936+0000 7f416b5a5640 1 -- 172.21.15.139:0/297091754 shutdown_connections 2024-04-25T10:03:11.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.936+0000 7f416b5a5640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f414806b4d0 0x7f414806d990 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.936+0000 7f416b5a5640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4164144090 0x7f416415a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.936+0000 7f416b5a5640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41640cc9e0 0x7f41641ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.936+0000 7f416b5a5640 1 --2- 172.21.15.139:0/297091754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f416406ea40 0x7f41640a84b0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.936+0000 7f416b5a5640 1 -- 172.21.15.139:0/297091754 >> 172.21.15.139:0/297091754 conn(0x7f4164067000 msgr2=0x7f416406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.936+0000 7f416b5a5640 1 -- 172.21.15.139:0/297091754 shutdown_connections 2024-04-25T10:03:11.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:11.936+0000 7f416b5a5640 1 -- 172.21.15.139:0/297091754 wait complete. 2024-04-25T10:03:11.951 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:03:11.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:11.952 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:11.978 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:03:11.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:11.978 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:03:12.033 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:12.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.166+0000 7f225665d640 1 Processor -- start 2024-04-25T10:03:12.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.166+0000 7f225665d640 1 -- start start 2024-04-25T10:03:12.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.166+0000 7f225665d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2250160e90 0x7f22500afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.166+0000 7f225665d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f225006ab60 0x7f225006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.166+0000 7f225665d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22500d0400 con 0x7f225006fa70 2024-04-25T10:03:12.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.166+0000 7f225665d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22500d14a0 con 0x7f2250160e90 2024-04-25T10:03:12.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.166+0000 7f225665d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22500d06e0 con 0x7f225006ab60 2024-04-25T10:03:12.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.167+0000 7f224ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2250160e90 0x7f22500afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.167+0000 7f224ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2250160e90 0x7f22500afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52462/0 (socket says 172.21.15.139:52462) 2024-04-25T10:03:12.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.167+0000 7f224ffff640 1 -- 172.21.15.139:0/4125712197 learned_addr learned my addr 172.21.15.139:0/4125712197 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:12.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.167+0000 7f224f7fe640 1 --2- 172.21.15.139:0/4125712197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f225006ab60 0x7f225006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.167+0000 7f224ffff640 1 -- 172.21.15.139:0/4125712197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f225006ab60 msgr2=0x7f225006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.167+0000 7f224ffff640 1 --2- 172.21.15.139:0/4125712197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f225006ab60 0x7f225006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.167+0000 7f224ffff640 1 -- 172.21.15.139:0/4125712197 >> v1:172.21.15.150:6789/0 conn(0x7f225006fa70 legacy=0x7f225006fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.167+0000 7f224ffff640 1 -- 172.21.15.139:0/4125712197 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22500d09c0 con 0x7f2250160e90 2024-04-25T10:03:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.168+0000 7f224ffff640 1 --2- 172.21.15.139:0/4125712197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2250160e90 0x7f22500afac0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f2240009870 tx=0x7f224002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c6d1e4dce3a4e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.168+0000 7f224effd640 1 -- 172.21.15.139:0/4125712197 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240037070 con 0x7f2250160e90 2024-04-25T10:03:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.168+0000 7f224effd640 1 -- 172.21.15.139:0/4125712197 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2240032440 con 0x7f2250160e90 2024-04-25T10:03:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.168+0000 7f224effd640 1 -- 172.21.15.139:0/4125712197 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240032810 con 0x7f2250160e90 2024-04-25T10:03:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.168+0000 7f225665d640 1 -- 172.21.15.139:0/4125712197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2250160e90 msgr2=0x7f22500afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.168+0000 7f225665d640 1 --2- 172.21.15.139:0/4125712197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2250160e90 0x7f22500afac0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f2240009870 tx=0x7f224002a270 comp rx=0 tx=0).stop 2024-04-25T10:03:12.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.168+0000 7f225665d640 1 -- 172.21.15.139:0/4125712197 shutdown_connections 2024-04-25T10:03:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.168+0000 7f225665d640 1 --2- 172.21.15.139:0/4125712197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f225006ab60 0x7f225006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.168+0000 7f225665d640 1 --2- 172.21.15.139:0/4125712197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2250160e90 0x7f22500afac0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.168+0000 7f225665d640 1 -- 172.21.15.139:0/4125712197 >> 172.21.15.139:0/4125712197 conn(0x7f22500b23f0 msgr2=0x7f22500b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.168+0000 7f225665d640 1 -- 172.21.15.139:0/4125712197 shutdown_connections 2024-04-25T10:03:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.168+0000 7f225665d640 1 -- 172.21.15.139:0/4125712197 wait complete. 2024-04-25T10:03:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.169+0000 7f225665d640 1 Processor -- start 2024-04-25T10:03:12.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.170+0000 7f225665d640 1 -- start start 2024-04-25T10:03:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.170+0000 7f225665d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f225006ab60 0x7f225014a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.170+0000 7f225665d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f225006fa70 0x7f22501f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.170+0000 7f225665d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2250160e90 0x7f22501f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.170+0000 7f225665d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500b0000 con 0x7f225006fa70 2024-04-25T10:03:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.170+0000 7f225665d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500d0f80 con 0x7f225006ab60 2024-04-25T10:03:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.170+0000 7f225665d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2250161270 con 0x7f2250160e90 2024-04-25T10:03:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.170+0000 7f224f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f225006fa70 0x7f22501f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.170+0000 7f224f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f225006fa70 0x7f22501f6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52478/0 (socket says 172.21.15.139:52478) 2024-04-25T10:03:12.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.170+0000 7f224f7fe640 1 -- 172.21.15.139:0/1736193808 learned_addr learned my addr 172.21.15.139:0/1736193808 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f224ffff640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f225006ab60 0x7f225014a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f2254bfa640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2250160e90 0x7f22501f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f224f7fe640 1 -- 172.21.15.139:0/1736193808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2250160e90 msgr2=0x7f22501f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f224f7fe640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2250160e90 0x7f22501f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f224f7fe640 1 -- 172.21.15.139:0/1736193808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f225006ab60 msgr2=0x7f225014a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f224f7fe640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f225006ab60 0x7f225014a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f224f7fe640 1 -- 172.21.15.139:0/1736193808 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2250146c80 con 0x7f225006fa70 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f224ffff640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f225006ab60 0x7f225014a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f2254bfa640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2250160e90 0x7f22501f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f224f7fe640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f225006fa70 0x7f22501f6e10 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f224400d830 tx=0x7f224400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f224cff9640 1 -- 172.21.15.139:0/1736193808 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224400b6c0 con 0x7f225006fa70 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f224cff9640 1 -- 172.21.15.139:0/1736193808 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2244019070 con 0x7f225006fa70 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f225665d640 1 -- 172.21.15.139:0/1736193808 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2250147020 con 0x7f225006fa70 2024-04-25T10:03:12.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f224cff9640 1 -- 172.21.15.139:0/1736193808 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224400ba80 con 0x7f225006fa70 2024-04-25T10:03:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.171+0000 7f225665d640 1 -- 172.21.15.139:0/1736193808 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22501475e0 con 0x7f225006fa70 2024-04-25T10:03:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.172+0000 7f225665d640 1 -- 172.21.15.139:0/1736193808 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2250074520 con 0x7f225006fa70 2024-04-25T10:03:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.172+0000 7f224cff9640 1 -- 172.21.15.139:0/1736193808 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f224401e460 con 0x7f225006fa70 2024-04-25T10:03:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.173+0000 7f224cff9640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f223c06b4d0 0x7f223c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.173+0000 7f224cff9640 1 -- 172.21.15.139:0/1736193808 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f224408aa90 con 0x7f225006fa70 2024-04-25T10:03:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.173+0000 7f224ffff640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f223c06b4d0 0x7f223c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.173+0000 7f224ffff640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f223c06b4d0 0x7f223c06d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2240034000 tx=0x7f224002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.177+0000 7f224cff9640 1 -- 172.21.15.139:0/1736193808 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2244059730 con 0x7f225006fa70 2024-04-25T10:03:12.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.384+0000 7f225665d640 1 -- 172.21.15.139:0/1736193808 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2250073020 con 0x7f225006fa70 2024-04-25T10:03:12.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.384+0000 7f224cff9640 1 -- 172.21.15.139:0/1736193808 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f2244059030 con 0x7f225006fa70 2024-04-25T10:03:12.389 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:12.389 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":7690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3195407462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3195407462},{"type":"v1","addr":"172.21.15.150:6835","nonce":3195407462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:03:06.057564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9117},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":8,"addr":"172.21.15.139:6836/3478297010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3478297010},{"type":"v1","addr":"172.21.15.139:6836","nonce":3478297010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:03:12.390 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 89 2024-04-25T10:03:12.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.386+0000 7f225665d640 1 -- 172.21.15.139:0/1736193808 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f223c06b4d0 msgr2=0x7f223c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.386+0000 7f225665d640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f223c06b4d0 0x7f223c06d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2240034000 tx=0x7f224002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:12.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.386+0000 7f225665d640 1 -- 172.21.15.139:0/1736193808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f225006fa70 msgr2=0x7f22501f6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.386+0000 7f225665d640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f225006fa70 0x7f22501f6e10 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f224400d830 tx=0x7f224400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.386+0000 7f225665d640 1 -- 172.21.15.139:0/1736193808 shutdown_connections 2024-04-25T10:03:12.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.386+0000 7f225665d640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f225006fa70 0x7f22501f6e10 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.386+0000 7f225665d640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f223c06b4d0 0x7f223c06d990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.386+0000 7f225665d640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2250160e90 0x7f22501f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.386+0000 7f225665d640 1 --2- 172.21.15.139:0/1736193808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f225006ab60 0x7f225014a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.386+0000 7f225665d640 1 -- 172.21.15.139:0/1736193808 >> 172.21.15.139:0/1736193808 conn(0x7f22500b23f0 msgr2=0x7f22500ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.387+0000 7f225665d640 1 -- 172.21.15.139:0/1736193808 shutdown_connections 2024-04-25T10:03:12.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.387+0000 7f225665d640 1 -- 172.21.15.139:0/1736193808 wait complete. 2024-04-25T10:03:12.403 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-04-25T10:03:12.517 INFO:teuthology.orchestra.run.smithi139.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-25T10:03:12.517 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-25T10:03:12.518 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000003.00000000 scrub_tag 2024-04-25T10:03:12.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.554+0000 7ff128559f40 1 Processor -- start 2024-04-25T10:03:12.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.555+0000 7ff128559f40 1 -- start start 2024-04-25T10:03:12.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.555+0000 7ff128559f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8b0a580 0x55edd8b320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.555+0000 7ff128559f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55edd8bde350 0x55edd8bcf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.555+0000 7ff128559f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd8be0220 0x55edd8bcf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.555+0000 7ff128559f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edd8b50b00 con 0x55edd8b0a580 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.555+0000 7ff128559f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edd8aaedf0 con 0x55edd8be0220 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.555+0000 7ff128559f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edd8b50c80 con 0x55edd8bde350 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.555+0000 7ff12605a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8b0a580 0x55edd8b320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.555+0000 7ff12605a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8b0a580 0x55edd8b320c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52506/0 (socket says 172.21.15.139:52506) 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.555+0000 7ff12605a640 1 -- 172.21.15.139:0/1812275936 learned_addr learned my addr 172.21.15.139:0/1812275936 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.555+0000 7ff12605a640 1 -- 172.21.15.139:0/1812275936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55edd8bde350 msgr2=0x55edd8bcf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.555+0000 7ff12605a640 1 --2- 172.21.15.139:0/1812275936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55edd8bde350 0x55edd8bcf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff12605a640 1 -- 172.21.15.139:0/1812275936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd8be0220 msgr2=0x55edd8bcf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff12605a640 1 --2- 172.21.15.139:0/1812275936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd8be0220 0x55edd8bcf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff12605a640 1 -- 172.21.15.139:0/1812275936 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edd8a34c50 con 0x55edd8b0a580 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff12605a640 1 --2- 172.21.15.139:0/1812275936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8b0a580 0x55edd8b320c0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff11800a3f0 tx=0x7ff11800a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31ae9ca6b6dad7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff124857640 1 -- 172.21.15.139:0/1812275936 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118010070 con 0x55edd8b0a580 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff124857640 1 -- 172.21.15.139:0/1812275936 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11800c8f0 con 0x55edd8b0a580 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff124857640 1 -- 172.21.15.139:0/1812275936 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11800cce0 con 0x55edd8b0a580 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff128559f40 1 -- 172.21.15.139:0/1812275936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8b0a580 msgr2=0x55edd8b320c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff128559f40 1 --2- 172.21.15.139:0/1812275936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8b0a580 0x55edd8b320c0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff11800a3f0 tx=0x7ff11800a8b0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff128559f40 1 -- 172.21.15.139:0/1812275936 shutdown_connections 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff128559f40 1 --2- 172.21.15.139:0/1812275936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd8be0220 0x55edd8bcf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff128559f40 1 --2- 172.21.15.139:0/1812275936 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55edd8bde350 0x55edd8bcf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff128559f40 1 --2- 172.21.15.139:0/1812275936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8b0a580 0x55edd8b320c0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff128559f40 1 -- 172.21.15.139:0/1812275936 >> 172.21.15.139:0/1812275936 conn(0x55edd8b07be0 msgr2=0x55edd8bdf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff128559f40 1 -- 172.21.15.139:0/1812275936 shutdown_connections 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.556+0000 7ff128559f40 1 -- 172.21.15.139:0/1812275936 wait complete. 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.557+0000 7ff128559f40 1 Processor -- start 2024-04-25T10:03:12.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.557+0000 7ff128559f40 1 -- start start 2024-04-25T10:03:12.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff128559f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55edd8b0a580 0x55edd8bcec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff128559f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd8bde350 0x55edd8bc9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff128559f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8be0220 0x55edd8bcaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff128559f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edd8bd2a30 con 0x55edd8be0220 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff128559f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edd8bd66a0 con 0x55edd8bde350 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff128559f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edd8b31640 con 0x55edd8b0a580 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff12685b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8be0220 0x55edd8bcaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff12685b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8be0220 0x55edd8bcaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52520/0 (socket says 172.21.15.139:52520) 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff12685b640 1 -- 172.21.15.139:0/2563033733 learned_addr learned my addr 172.21.15.139:0/2563033733 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff125859640 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd8bde350 0x55edd8bc9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff12605a640 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55edd8b0a580 0x55edd8bcec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff12685b640 1 -- 172.21.15.139:0/2563033733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55edd8b0a580 msgr2=0x55edd8bcec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff12685b640 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55edd8b0a580 0x55edd8bcec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff12685b640 1 -- 172.21.15.139:0/2563033733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd8bde350 msgr2=0x55edd8bc9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff12685b640 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd8bde350 0x55edd8bc9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff12685b640 1 -- 172.21.15.139:0/2563033733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edd8bcc040 con 0x55edd8be0220 2024-04-25T10:03:12.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff12685b640 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8be0220 0x55edd8bcaa00 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff120005ee0 tx=0x7ff120008430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff1177fe640 1 -- 172.21.15.139:0/2563033733 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120010440 con 0x55edd8be0220 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff1177fe640 1 -- 172.21.15.139:0/2563033733 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff120015070 con 0x55edd8be0220 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff1177fe640 1 -- 172.21.15.139:0/2563033733 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120010800 con 0x55edd8be0220 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff128559f40 1 -- 172.21.15.139:0/2563033733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55edd8bcc770 con 0x55edd8be0220 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.558+0000 7ff128559f40 1 -- 172.21.15.139:0/2563033733 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edd8bcf190 con 0x55edd8be0220 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.560+0000 7ff1177fe640 1 -- 172.21.15.139:0/2563033733 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1200109a0 con 0x55edd8be0220 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.560+0000 7ff1177fe640 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff108069eb0 0x7ff10806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.560+0000 7ff12605a640 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff108069eb0 0x7ff10806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.560+0000 7ff1177fe640 1 -- 172.21.15.139:0/2563033733 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff120088c90 con 0x55edd8be0220 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.560+0000 7ff128559f40 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55edd8bcbb10 0x55edd8bd4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.560+0000 7ff128559f40 1 -- 172.21.15.139:0/2563033733 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:1 18.32 18:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) v9 -- 0x55edd8bd5380 con 0x55edd8bcbb10 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.560+0000 7ff125859640 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55edd8bcbb10 0x55edd8bd4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff12605a640 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff108069eb0 0x7ff10806c370 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff11800a3f0 tx=0x7ff118002000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff125859640 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55edd8bcbb10 0x55edd8bd4e40 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff125859640 1 -- 172.21.15.139:0/2563033733 <== osd.3 v2:172.21.15.139:6824/1040931291 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff11c050050 con 0x55edd8bcbb10 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:error getting xattr cephfs_data/10000000003.00000000/scrub_tag: (61) No data available 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff128559f40 1 -- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55edd8bcbb10 msgr2=0x55edd8bd4e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff128559f40 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55edd8bcbb10 0x55edd8bd4e40 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff128559f40 1 -- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff108069eb0 msgr2=0x7ff10806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff128559f40 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff108069eb0 0x7ff10806c370 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff11800a3f0 tx=0x7ff118002000 comp rx=0 tx=0).stop 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff128559f40 1 -- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8be0220 msgr2=0x55edd8bcaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff128559f40 1 --2- 172.21.15.139:0/2563033733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55edd8be0220 0x55edd8bcaa00 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff120005ee0 tx=0x7ff120008430 comp rx=0 tx=0).stop 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff12685b640 1 -- 172.21.15.139:0/2563033733 reap_dead start 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff128559f40 1 -- 172.21.15.139:0/2563033733 shutdown_connections 2024-04-25T10:03:12.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff128559f40 1 -- 172.21.15.139:0/2563033733 >> 172.21.15.139:0/2563033733 conn(0x55edd8b07be0 msgr2=0x55edd8bd2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff128559f40 1 -- 172.21.15.139:0/2563033733 shutdown_connections 2024-04-25T10:03:12.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.561+0000 7ff128559f40 1 -- 172.21.15.139:0/2563033733 wait complete. 2024-04-25T10:03:12.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:03:12.572 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T10:03:12.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.651+0000 7f005451bf40 1 Processor -- start 2024-04-25T10:03:12.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.651+0000 7f005451bf40 1 -- start start 2024-04-25T10:03:12.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.651+0000 7f005451bf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5582da588e70 0x5582da65ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005451bf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5582da5cfab0 0x5582da65b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005451bf40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da640e50 0x5582da587fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005451bf40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582da52ddf0 con 0x5582da640e50 2024-04-25T10:03:12.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005451bf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582da531d50 con 0x5582da5cfab0 2024-04-25T10:03:12.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005451bf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582da644580 con 0x5582da588e70 2024-04-25T10:03:12.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005281d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da640e50 0x5582da587fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005281d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da640e50 0x5582da587fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52532/0 (socket says 172.21.15.139:52532) 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005281d640 1 -- 172.21.15.139:0/1042831381 learned_addr learned my addr 172.21.15.139:0/1042831381 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005201c640 1 --2- 172.21.15.139:0/1042831381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5582da588e70 0x5582da65ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005281d640 1 -- 172.21.15.139:0/1042831381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5582da588e70 msgr2=0x5582da65ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005281d640 1 --2- 172.21.15.139:0/1042831381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5582da588e70 0x5582da65ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005281d640 1 -- 172.21.15.139:0/1042831381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5582da5cfab0 msgr2=0x5582da65b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005181b640 1 --2- 172.21.15.139:0/1042831381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5582da5cfab0 0x5582da65b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005281d640 1 --2- 172.21.15.139:0/1042831381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5582da5cfab0 0x5582da65b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005281d640 1 -- 172.21.15.139:0/1042831381 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582da638520 con 0x5582da640e50 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f005281d640 1 --2- 172.21.15.139:0/1042831381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da640e50 0x5582da587fe0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f004c006530 tx=0x7f004c0069f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5851b1cf5716b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f0050819640 1 -- 172.21.15.139:0/1042831381 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c0062e0 con 0x5582da640e50 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f0050819640 1 -- 172.21.15.139:0/1042831381 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c003670 con 0x5582da640e50 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.652+0000 7f0050819640 1 -- 172.21.15.139:0/1042831381 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c003a60 con 0x5582da640e50 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.653+0000 7f005451bf40 1 -- 172.21.15.139:0/1042831381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da640e50 msgr2=0x5582da587fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.653+0000 7f005451bf40 1 --2- 172.21.15.139:0/1042831381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da640e50 0x5582da587fe0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f004c006530 tx=0x7f004c0069f0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.653+0000 7f005451bf40 1 -- 172.21.15.139:0/1042831381 shutdown_connections 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.653+0000 7f005451bf40 1 --2- 172.21.15.139:0/1042831381 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da640e50 0x5582da587fe0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.653+0000 7f005451bf40 1 --2- 172.21.15.139:0/1042831381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5582da5cfab0 0x5582da65b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.653+0000 7f005451bf40 1 --2- 172.21.15.139:0/1042831381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5582da588e70 0x5582da65ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.653+0000 7f005451bf40 1 -- 172.21.15.139:0/1042831381 >> 172.21.15.139:0/1042831381 conn(0x5582da586be0 msgr2=0x5582da643b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.653+0000 7f005451bf40 1 -- 172.21.15.139:0/1042831381 shutdown_connections 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.653+0000 7f005451bf40 1 -- 172.21.15.139:0/1042831381 wait complete. 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.653+0000 7f005451bf40 1 Processor -- start 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.653+0000 7f005451bf40 1 -- start start 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005451bf40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5582da588e70 0x5582da5b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005451bf40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da5cfab0 0x5582da63e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005451bf40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5582da640e50 0x5582da587940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005451bf40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582da640220 con 0x5582da5cfab0 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005451bf40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582da65fac0 con 0x5582da640e50 2024-04-25T10:03:12.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005451bf40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582da640500 con 0x5582da588e70 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005181b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da5cfab0 0x5582da63e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005181b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da5cfab0 0x5582da63e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52538/0 (socket says 172.21.15.139:52538) 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005181b640 1 -- 172.21.15.139:0/791950062 learned_addr learned my addr 172.21.15.139:0/791950062 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005281d640 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5582da640e50 0x5582da587940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005201c640 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5582da588e70 0x5582da5b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005181b640 1 -- 172.21.15.139:0/791950062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5582da588e70 msgr2=0x5582da5b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005181b640 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5582da588e70 0x5582da5b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005181b640 1 -- 172.21.15.139:0/791950062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5582da640e50 msgr2=0x5582da587940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005181b640 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5582da640e50 0x5582da587940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005181b640 1 -- 172.21.15.139:0/791950062 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582da5b6680 con 0x5582da5cfab0 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005281d640 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5582da640e50 0x5582da587940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005201c640 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5582da588e70 0x5582da5b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.654+0000 7f005181b640 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da5cfab0 0x5582da63e460 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f004800cea0 tx=0x7f004800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.655+0000 7f00437fe640 1 -- 172.21.15.139:0/791950062 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048010650 con 0x5582da5cfab0 2024-04-25T10:03:12.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.655+0000 7f00437fe640 1 -- 172.21.15.139:0/791950062 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00480107f0 con 0x5582da5cfab0 2024-04-25T10:03:12.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.655+0000 7f00437fe640 1 -- 172.21.15.139:0/791950062 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048010be0 con 0x5582da5cfab0 2024-04-25T10:03:12.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.655+0000 7f005451bf40 1 -- 172.21.15.139:0/791950062 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5582da65b9c0 con 0x5582da5cfab0 2024-04-25T10:03:12.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.655+0000 7f005451bf40 1 -- 172.21.15.139:0/791950062 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582da63e9a0 con 0x5582da5cfab0 2024-04-25T10:03:12.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.656+0000 7f00437fe640 1 -- 172.21.15.139:0/791950062 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f004801b470 con 0x5582da5cfab0 2024-04-25T10:03:12.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.656+0000 7f00437fe640 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0034069eb0 0x7f003406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.656+0000 7f00437fe640 1 -- 172.21.15.139:0/791950062 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0048089280 con 0x5582da5cfab0 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.656+0000 7f005201c640 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0034069eb0 0x7f003406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.657+0000 7f005451bf40 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x5582da6430a0 0x5582da5b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.657+0000 7f005451bf40 1 -- 172.21.15.139:0/791950062 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) v9 -- 0x5582da659ec0 con 0x5582da6430a0 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.657+0000 7f005281d640 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x5582da6430a0 0x5582da5b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.657+0000 7f005201c640 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0034069eb0 0x7f003406c370 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0044056000 tx=0x7f0044050000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.657+0000 7f005281d640 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x5582da6430a0 0x5582da5b5a60 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.657+0000 7f005281d640 1 -- 172.21.15.139:0/791950062 <== osd.2 v2:172.21.15.139:6800/1982336775 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f004c00f030 con 0x5582da6430a0 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.657+0000 7f005451bf40 1 -- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x5582da6430a0 msgr2=0x5582da5b5a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.657+0000 7f005451bf40 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x5582da6430a0 0x5582da5b5a60 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.657+0000 7f005451bf40 1 -- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0034069eb0 msgr2=0x7f003406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.657+0000 7f005451bf40 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0034069eb0 0x7f003406c370 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0044056000 tx=0x7f0044050000 comp rx=0 tx=0).stop 2024-04-25T10:03:12.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.657+0000 7f005451bf40 1 -- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da5cfab0 msgr2=0x5582da63e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.657+0000 7f005451bf40 1 --2- 172.21.15.139:0/791950062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5582da5cfab0 0x5582da63e460 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f004800cea0 tx=0x7f004800cf90 comp rx=0 tx=0).stop 2024-04-25T10:03:12.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.658+0000 7f005281d640 1 -- 172.21.15.139:0/791950062 reap_dead start 2024-04-25T10:03:12.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.658+0000 7f005451bf40 1 -- 172.21.15.139:0/791950062 shutdown_connections 2024-04-25T10:03:12.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.658+0000 7f005451bf40 1 -- 172.21.15.139:0/791950062 >> 172.21.15.139:0/791950062 conn(0x5582da586be0 msgr2=0x5582da65b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.658+0000 7f005451bf40 1 -- 172.21.15.139:0/791950062 shutdown_connections 2024-04-25T10:03:12.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.658+0000 7f005451bf40 1 -- 172.21.15.139:0/791950062 wait complete. 2024-04-25T10:03:12.668 INFO:teuthology.orchestra.run.smithi139.stdout:mytag123 2024-04-25T10:03:12.668 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-25T10:03:12.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.746+0000 7f4908354f40 1 Processor -- start 2024-04-25T10:03:12.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.746+0000 7f4908354f40 1 -- start start 2024-04-25T10:03:12.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.746+0000 7f4908354f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555e4457580 0x5555e45245b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.746+0000 7f4908354f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5555e452b350 0x5555e45273c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.746+0000 7f4908354f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e452d220 0x5555e4522b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4908354f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e43fbdf0 con 0x5555e452d220 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4908354f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e43ffd50 con 0x5555e4457580 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4908354f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555e449c8b0 con 0x5555e452b350 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4906656640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e452d220 0x5555e4522b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4906656640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e452d220 0x5555e4522b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52550/0 (socket says 172.21.15.139:52550) 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4906656640 1 -- 172.21.15.139:0/1660420241 learned_addr learned my addr 172.21.15.139:0/1660420241 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4905e55640 1 --2- 172.21.15.139:0/1660420241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555e4457580 0x5555e45245b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4905654640 1 --2- 172.21.15.139:0/1660420241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5555e452b350 0x5555e45273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4906656640 1 -- 172.21.15.139:0/1660420241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5555e452b350 msgr2=0x5555e45273c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4906656640 1 --2- 172.21.15.139:0/1660420241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5555e452b350 0x5555e45273c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4906656640 1 -- 172.21.15.139:0/1660420241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555e4457580 msgr2=0x5555e45245b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4906656640 1 --2- 172.21.15.139:0/1660420241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555e4457580 0x5555e45245b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4906656640 1 -- 172.21.15.139:0/1660420241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555e4527ec0 con 0x5555e452d220 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4905654640 1 --2- 172.21.15.139:0/1660420241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5555e452b350 0x5555e45273c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f4906656640 1 --2- 172.21.15.139:0/1660420241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e452d220 0x5555e4522b60 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f49000015f0 tx=0x7f4900001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d77d72cb7c14e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f48f7fff640 1 -- 172.21.15.139:0/1660420241 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900016070 con 0x5555e452d220 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f48f7fff640 1 -- 172.21.15.139:0/1660420241 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490000f710 con 0x5555e452d220 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.747+0000 7f48f7fff640 1 -- 172.21.15.139:0/1660420241 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490000fb00 con 0x5555e452d220 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.748+0000 7f4908354f40 1 -- 172.21.15.139:0/1660420241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e452d220 msgr2=0x5555e4522b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.748+0000 7f4908354f40 1 --2- 172.21.15.139:0/1660420241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e452d220 0x5555e4522b60 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f49000015f0 tx=0x7f4900001ab0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.748+0000 7f4908354f40 1 -- 172.21.15.139:0/1660420241 shutdown_connections 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.748+0000 7f4908354f40 1 --2- 172.21.15.139:0/1660420241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e452d220 0x5555e4522b60 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.748+0000 7f4908354f40 1 --2- 172.21.15.139:0/1660420241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5555e452b350 0x5555e45273c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.748+0000 7f4908354f40 1 --2- 172.21.15.139:0/1660420241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555e4457580 0x5555e45245b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.748+0000 7f4908354f40 1 -- 172.21.15.139:0/1660420241 >> 172.21.15.139:0/1660420241 conn(0x5555e4454be0 msgr2=0x5555e449e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.748+0000 7f4908354f40 1 -- 172.21.15.139:0/1660420241 shutdown_connections 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.748+0000 7f4908354f40 1 -- 172.21.15.139:0/1660420241 wait complete. 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.748+0000 7f4908354f40 1 Processor -- start 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.748+0000 7f4908354f40 1 -- start start 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4908354f40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e4457580 0x5555e45c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4908354f40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5555e452b350 0x5555e45bfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4908354f40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555e452d220 0x5555e451e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4908354f40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e45236d0 con 0x5555e4457580 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4908354f40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e4456f70 con 0x5555e452d220 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4908354f40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555e4523850 con 0x5555e452b350 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4905e55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e4457580 0x5555e45c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4905e55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e4457580 0x5555e45c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52562/0 (socket says 172.21.15.139:52562) 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4905e55640 1 -- 172.21.15.139:0/3963996880 learned_addr learned my addr 172.21.15.139:0/3963996880 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4906656640 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555e452d220 0x5555e451e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4905654640 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5555e452b350 0x5555e45bfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4905e55640 1 -- 172.21.15.139:0/3963996880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5555e452b350 msgr2=0x5555e45bfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4905e55640 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5555e452b350 0x5555e45bfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4905e55640 1 -- 172.21.15.139:0/3963996880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555e452d220 msgr2=0x5555e451e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4905e55640 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555e452d220 0x5555e451e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4905e55640 1 -- 172.21.15.139:0/3963996880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555e4521600 con 0x5555e4457580 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.749+0000 7f4906656640 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5555e452d220 0x5555e451e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.750+0000 7f4905e55640 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e4457580 0x5555e45c3cd0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f48f8057040 tx=0x7f48f8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.750+0000 7f48f6ffd640 1 -- 172.21.15.139:0/3963996880 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8066020 con 0x5555e4457580 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.750+0000 7f48f6ffd640 1 -- 172.21.15.139:0/3963996880 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f8054c40 con 0x5555e4457580 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.750+0000 7f4908354f40 1 -- 172.21.15.139:0/3963996880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5555e449df10 con 0x5555e4457580 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.750+0000 7f48f6ffd640 1 -- 172.21.15.139:0/3963996880 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8059040 con 0x5555e4457580 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.750+0000 7f4908354f40 1 -- 172.21.15.139:0/3963996880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555e447ef00 con 0x5555e4457580 2024-04-25T10:03:12.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.751+0000 7f48f6ffd640 1 -- 172.21.15.139:0/3963996880 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48f8069020 con 0x5555e4457580 2024-04-25T10:03:12.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.751+0000 7f48f6ffd640 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f48e8069eb0 0x7f48e806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.752+0000 7f4905654640 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f48e8069eb0 0x7f48e806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.752+0000 7f48f6ffd640 1 -- 172.21.15.139:0/3963996880 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f48f80cd000 con 0x5555e4457580 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.752+0000 7f4908354f40 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5555e4520dc0 0x5555e447f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.752+0000 7f4908354f40 1 -- 172.21.15.139:0/3963996880 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) v9 -- 0x5555e447fc40 con 0x5555e4520dc0 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.752+0000 7f4906656640 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5555e4520dc0 0x5555e447f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.752+0000 7f4905654640 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f48e8069eb0 0x7f48e806c370 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x5555e4484890 tx=0x7f48fc001dd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.752+0000 7f4906656640 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5555e4520dc0 0x5555e447f700 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.752+0000 7f4906656640 1 -- 172.21.15.139:0/3963996880 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f490000f070 con 0x5555e4520dc0 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.752+0000 7f4908354f40 1 -- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5555e4520dc0 msgr2=0x5555e447f700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.752+0000 7f4908354f40 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5555e4520dc0 0x5555e447f700 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.753+0000 7f4908354f40 1 -- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f48e8069eb0 msgr2=0x7f48e806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.753+0000 7f4908354f40 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f48e8069eb0 0x7f48e806c370 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x5555e4484890 tx=0x7f48fc001dd0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.753+0000 7f4908354f40 1 -- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e4457580 msgr2=0x5555e45c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.753+0000 7f4908354f40 1 --2- 172.21.15.139:0/3963996880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5555e4457580 0x5555e45c3cd0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f48f8057040 tx=0x7f48f8002fb0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.753+0000 7f4906656640 1 -- 172.21.15.139:0/3963996880 reap_dead start 2024-04-25T10:03:12.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.753+0000 7f4908354f40 1 -- 172.21.15.139:0/3963996880 shutdown_connections 2024-04-25T10:03:12.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.753+0000 7f4908354f40 1 -- 172.21.15.139:0/3963996880 >> 172.21.15.139:0/3963996880 conn(0x5555e4454be0 msgr2=0x5555e449d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.753+0000 7f4908354f40 1 -- 172.21.15.139:0/3963996880 shutdown_connections 2024-04-25T10:03:12.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.753+0000 7f4908354f40 1 -- 172.21.15.139:0/3963996880 wait complete. 2024-04-25T10:03:12.763 INFO:teuthology.orchestra.run.smithi139.stdout:mytag123 2024-04-25T10:03:12.764 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000006.00000000 scrub_tag 2024-04-25T10:03:12.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.842+0000 7f7af931ff40 1 Processor -- start 2024-04-25T10:03:12.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.842+0000 7f7af931ff40 1 -- start start 2024-04-25T10:03:12.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.842+0000 7f7af931ff40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639dc7e43b0 0x5639dc8b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.842+0000 7f7af931ff40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639dc829430 0x5639dc8b7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.842+0000 7f7af931ff40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc8b3530 0x5639dc8b7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.842+0000 7f7af931ff40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dc7e47c0 con 0x5639dc8b3530 2024-04-25T10:03:12.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.842+0000 7f7af931ff40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dc788df0 con 0x5639dc829430 2024-04-25T10:03:12.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.842+0000 7f7af931ff40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639dc829b40 con 0x5639dc7e43b0 2024-04-25T10:03:12.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af6b9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc8b3530 0x5639dc8b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af6b9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc8b3530 0x5639dc8b7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52584/0 (socket says 172.21.15.139:52584) 2024-04-25T10:03:12.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af6b9b640 1 -- 172.21.15.139:0/1567881732 learned_addr learned my addr 172.21.15.139:0/1567881732 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af6b9b640 1 -- 172.21.15.139:0/1567881732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639dc7e43b0 msgr2=0x5639dc8b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af6b9b640 1 --2- 172.21.15.139:0/1567881732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639dc7e43b0 0x5639dc8b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af6b9b640 1 -- 172.21.15.139:0/1567881732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639dc829430 msgr2=0x5639dc8b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af5b99640 1 --2- 172.21.15.139:0/1567881732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639dc829430 0x5639dc8b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af6b9b640 1 --2- 172.21.15.139:0/1567881732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639dc829430 0x5639dc8b7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af6b9b640 1 -- 172.21.15.139:0/1567881732 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639dc70ec50 con 0x5639dc8b3530 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af6b9b640 1 --2- 172.21.15.139:0/1567881732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc8b3530 0x5639dc8b7690 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f7af0001e70 tx=0x7f7af000b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca00697a9a20ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af4b97640 1 -- 172.21.15.139:0/1567881732 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0008070 con 0x5639dc8b3530 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af4b97640 1 -- 172.21.15.139:0/1567881732 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af000be10 con 0x5639dc8b3530 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.843+0000 7f7af4b97640 1 -- 172.21.15.139:0/1567881732 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af000d740 con 0x5639dc8b3530 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.844+0000 7f7af931ff40 1 -- 172.21.15.139:0/1567881732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc8b3530 msgr2=0x5639dc8b7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.844+0000 7f7af931ff40 1 --2- 172.21.15.139:0/1567881732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc8b3530 0x5639dc8b7690 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f7af0001e70 tx=0x7f7af000b6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.844+0000 7f7af931ff40 1 -- 172.21.15.139:0/1567881732 shutdown_connections 2024-04-25T10:03:12.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.844+0000 7f7af931ff40 1 --2- 172.21.15.139:0/1567881732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc8b3530 0x5639dc8b7690 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.844+0000 7f7af931ff40 1 --2- 172.21.15.139:0/1567881732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639dc829430 0x5639dc8b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.844+0000 7f7af931ff40 1 --2- 172.21.15.139:0/1567881732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639dc7e43b0 0x5639dc8b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.844+0000 7f7af931ff40 1 -- 172.21.15.139:0/1567881732 >> 172.21.15.139:0/1567881732 conn(0x5639dc7e1be0 msgr2=0x5639dc8b8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.844+0000 7f7af931ff40 1 -- 172.21.15.139:0/1567881732 shutdown_connections 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.844+0000 7f7af931ff40 1 -- 172.21.15.139:0/1567881732 wait complete. 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.844+0000 7f7af931ff40 1 Processor -- start 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.844+0000 7f7af931ff40 1 -- start start 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af931ff40 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639dc7e43b0 0x5639dc826180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af931ff40 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc829430 0x5639dc823ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af931ff40 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639dc8b3530 0x5639dc8b8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af931ff40 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dc810960 con 0x5639dc829430 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af931ff40 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639dc8b2700 con 0x5639dc8b3530 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af931ff40 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639dc810c40 con 0x5639dc7e43b0 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af5b99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc829430 0x5639dc823ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af5b99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc829430 0x5639dc823ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52592/0 (socket says 172.21.15.139:52592) 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af5b99640 1 -- 172.21.15.139:0/2061387163 learned_addr learned my addr 172.21.15.139:0/2061387163 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af6b9b640 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639dc8b3530 0x5639dc8b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af639a640 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639dc7e43b0 0x5639dc826180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af5b99640 1 -- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639dc7e43b0 msgr2=0x5639dc826180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af5b99640 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639dc7e43b0 0x5639dc826180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af5b99640 1 -- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639dc8b3530 msgr2=0x5639dc8b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af5b99640 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639dc8b3530 0x5639dc8b8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af5b99640 1 -- 172.21.15.139:0/2061387163 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639dc827300 con 0x5639dc829430 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af6b9b640 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639dc8b3530 0x5639dc8b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af639a640 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5639dc7e43b0 0x5639dc826180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.845+0000 7f7af5b99640 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc829430 0x5639dc823ca0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f7aec007e00 tx=0x7f7aec006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.846+0000 7f7ae77fe640 1 -- 172.21.15.139:0/2061387163 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec017070 con 0x5639dc829430 2024-04-25T10:03:12.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.846+0000 7f7ae77fe640 1 -- 172.21.15.139:0/2061387163 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec010710 con 0x5639dc829430 2024-04-25T10:03:12.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.846+0000 7f7af931ff40 1 -- 172.21.15.139:0/2061387163 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5639dc8b8ea0 con 0x5639dc829430 2024-04-25T10:03:12.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.846+0000 7f7ae77fe640 1 -- 172.21.15.139:0/2061387163 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec006e60 con 0x5639dc829430 2024-04-25T10:03:12.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.846+0000 7f7af931ff40 1 -- 172.21.15.139:0/2061387163 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639dc8b1fa0 con 0x5639dc829430 2024-04-25T10:03:12.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.847+0000 7f7ae77fe640 1 -- 172.21.15.139:0/2061387163 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7aec01a070 con 0x5639dc829430 2024-04-25T10:03:12.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.847+0000 7f7ae77fe640 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ad8069eb0 0x7f7ad806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.847+0000 7f7ae77fe640 1 -- 172.21.15.139:0/2061387163 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7aec0882a0 con 0x5639dc829430 2024-04-25T10:03:12.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.847+0000 7f7af639a640 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ad8069eb0 0x7f7ad806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.847+0000 7f7af931ff40 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x5639dc8b7700 0x5639dc7dfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.848+0000 7f7af931ff40 1 -- 172.21.15.139:0/2061387163 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:1 18.25 18:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) v9 -- 0x5639dc95b830 con 0x5639dc8b7700 2024-04-25T10:03:12.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.848+0000 7f7af6b9b640 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x5639dc8b7700 0x5639dc7dfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.848+0000 7f7af639a640 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ad8069eb0 0x7f7ad806c370 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8056000 tx=0x7f7ae8059040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.848+0000 7f7af6b9b640 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x5639dc8b7700 0x5639dc7dfbc0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.848+0000 7f7af6b9b640 1 -- 172.21.15.139:0/2061387163 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (12) out=12b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f7af000da00 con 0x5639dc8b7700 2024-04-25T10:03:12.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.848+0000 7f7af931ff40 1 -- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x5639dc8b7700 msgr2=0x5639dc7dfbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.848+0000 7f7af931ff40 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x5639dc8b7700 0x5639dc7dfbc0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.848+0000 7f7af931ff40 1 -- 172.21.15.139:0/2061387163 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ad8069eb0 msgr2=0x7f7ad806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.848+0000 7f7af931ff40 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f7ad8069eb0 0x7f7ad806c370 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8056000 tx=0x7f7ae8059040 comp rx=0 tx=0).stop 2024-04-25T10:03:12.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.849+0000 7f7af931ff40 1 -- 172.21.15.139:0/2061387163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc829430 msgr2=0x5639dc823ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.849+0000 7f7af931ff40 1 --2- 172.21.15.139:0/2061387163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639dc829430 0x5639dc823ca0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f7aec007e00 tx=0x7f7aec006750 comp rx=0 tx=0).stop 2024-04-25T10:03:12.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.849+0000 7f7af6b9b640 1 -- 172.21.15.139:0/2061387163 reap_dead start 2024-04-25T10:03:12.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.849+0000 7f7af931ff40 1 -- 172.21.15.139:0/2061387163 shutdown_connections 2024-04-25T10:03:12.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.849+0000 7f7af931ff40 1 -- 172.21.15.139:0/2061387163 >> 172.21.15.139:0/2061387163 conn(0x5639dc7e1be0 msgr2=0x5639dc7e3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.849+0000 7f7af931ff40 1 -- 172.21.15.139:0/2061387163 shutdown_connections 2024-04-25T10:03:12.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.849+0000 7f7af931ff40 1 -- 172.21.15.139:0/2061387163 wait complete. 2024-04-25T10:03:12.859 INFO:teuthology.orchestra.run.smithi139.stdout:mytag123 2024-04-25T10:03:12.859 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:03:12.859 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T10:03:12.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.994+0000 7f38ad89e640 1 Processor -- start 2024-04-25T10:03:12.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.994+0000 7f38ad89e640 1 -- start start 2024-04-25T10:03:13.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38ad89e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a81617e0 0x7f38a80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38ad89e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f38a80cec80 0x7f38a80d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38ad89e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a8161dc0 con 0x7f38a815dbf0 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38ad89e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a80d1ee0 con 0x7f38a80cec80 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38ad89e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a81620a0 con 0x7f38a81617e0 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a6ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f38a815dbf0 0x7f38a80b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34204/0 (socket says 172.21.15.139:34204) 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a6ffd640 1 -- 172.21.15.139:0/881228398 learned_addr learned my addr 172.21.15.139:0/881228398 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a77fe640 1 --2- 172.21.15.139:0/881228398 >> v2:172.21.15.150:3300/0 conn(0x7f38a80cec80 0x7f38a80d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a67fc640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a81617e0 0x7f38a80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493170327 0 0) 0x7f38a8161dc0 con 0x7f38a815dbf0 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3890003520 con 0x7f38a815dbf0 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3975269308 0 0) 0x7f3890003520 con 0x7f38a815dbf0 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a81617e0 msgr2=0x7f38a80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a81617e0 0x7f38a80ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 >> v2:172.21.15.150:3300/0 conn(0x7f38a80cec80 msgr2=0x7f38a80d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 --2- 172.21.15.139:0/881228398 >> v2:172.21.15.150:3300/0 conn(0x7f38a80cec80 0x7f38a80d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a8162380 con 0x7f38a815dbf0 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f3898001670 con 0x7f38a815dbf0 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 >> v1:172.21.15.139:6789/0 conn(0x7f38a815dbf0 legacy=0x7f38a80b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a81617e0 0x7f38900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3890003f30 0x7f3890004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3890004d40 0x7f38900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80d1ee0 con 0x7f3890004d40 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a81620a0 con 0x7f3890003f30 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.995+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389000ba10 con 0x7f38a81617e0 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.996+0000 7f38a77fe640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3890004d40 0x7f38900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.996+0000 7f38a67fc640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3890003f30 0x7f3890004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.996+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a81617e0 0x7f38900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.996+0000 7f38a77fe640 1 -- 172.21.15.139:0/881228398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a81617e0 msgr2=0x7f38900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.996+0000 7f38a77fe640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a81617e0 0x7f38900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.996+0000 7f38a77fe640 1 -- 172.21.15.139:0/881228398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3890003f30 msgr2=0x7f3890004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.996+0000 7f38a77fe640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3890003f30 0x7f3890004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.996+0000 7f38a77fe640 1 -- 172.21.15.139:0/881228398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3890003520 con 0x7f3890004d40 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.996+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38a81617e0 0x7f38900039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.996+0000 7f38a77fe640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3890004d40 0x7f38900051c0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f38a000e170 tx=0x7f3890005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237cb75193dc59c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.997+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0034020 con 0x7f3890004d40 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.997+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a0030430 con 0x7f3890004d40 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.997+0000 7f38a5ffb640 1 -- 172.21.15.139:0/881228398 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0009c10 con 0x7f3890004d40 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.997+0000 7f38ad89e640 1 -- 172.21.15.139:0/881228398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3890004d40 msgr2=0x7f38900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.997+0000 7f38ad89e640 1 --2- 172.21.15.139:0/881228398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3890004d40 0x7f38900051c0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f38a000e170 tx=0x7f3890005830 comp rx=0 tx=0).stop 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.997+0000 7f38a77fe640 1 -- 172.21.15.139:0/881228398 reap_dead start 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.997+0000 7f38ad89e640 1 -- 172.21.15.139:0/881228398 shutdown_connections 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.997+0000 7f38ad89e640 1 -- 172.21.15.139:0/881228398 >> 172.21.15.139:0/881228398 conn(0x7f38a80b5480 msgr2=0x7f38a80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.997+0000 7f38ad89e640 1 -- 172.21.15.139:0/881228398 shutdown_connections 2024-04-25T10:03:13.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.997+0000 7f38ad89e640 1 -- 172.21.15.139:0/881228398 wait complete. 2024-04-25T10:03:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.998+0000 7f38ad89e640 1 Processor -- start 2024-04-25T10:03:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.999+0000 7f38ad89e640 1 -- start start 2024-04-25T10:03:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.999+0000 7f38ad89e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3890004d40 0x7f38a80c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.999+0000 7f38ad89e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38a81f6c80 0x7f38a81552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.999+0000 7f38ad89e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38a81604b0 0x7f38a8160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.999+0000 7f38ad89e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a815dfd0 con 0x7f38a81604b0 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.999+0000 7f38ad89e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80ce740 con 0x7f38a81f6c80 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.999+0000 7f38ad89e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a8150590 con 0x7f3890004d40 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.999+0000 7f38a77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38a81604b0 0x7f38a8160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.999+0000 7f38a77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38a81604b0 0x7f38a8160930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52610/0 (socket says 172.21.15.139:52610) 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:12.999+0000 7f38a77fe640 1 -- 172.21.15.139:0/2286148909 learned_addr learned my addr 172.21.15.139:0/2286148909 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f38a77fe640 1 -- 172.21.15.139:0/2286148909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3890004d40 msgr2=0x7f38a80c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f38a67fc640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38a81f6c80 0x7f38a81552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3890004d40 0x7f38a80c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f38a77fe640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3890004d40 0x7f38a80c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f38a77fe640 1 -- 172.21.15.139:0/2286148909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38a81f6c80 msgr2=0x7f38a81552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f38a77fe640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38a81f6c80 0x7f38a81552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f38a77fe640 1 -- 172.21.15.139:0/2286148909 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a81f71f0 con 0x7f38a81604b0 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f38a77fe640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38a81604b0 0x7f38a8160930 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f38a000e170 tx=0x7f38a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f388ffff640 1 -- 172.21.15.139:0/2286148909 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0034040 con 0x7f38a81604b0 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f388ffff640 1 -- 172.21.15.139:0/2286148909 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a0003990 con 0x7f38a81604b0 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f38ad89e640 1 -- 172.21.15.139:0/2286148909 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a81f74d0 con 0x7f38a81604b0 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f388ffff640 1 -- 172.21.15.139:0/2286148909 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0009c10 con 0x7f38a81604b0 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.000+0000 7f38ad89e640 1 -- 172.21.15.139:0/2286148909 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a81f7a90 con 0x7f38a81604b0 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.001+0000 7f388ffff640 1 -- 172.21.15.139:0/2286148909 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38a0037020 con 0x7f38a81604b0 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.002+0000 7f388ffff640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f389406b4d0 0x7f389406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.002+0000 7f388ffff640 1 -- 172.21.15.139:0/2286148909 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f38a00ab400 con 0x7f38a81604b0 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.002+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f389406b4d0 0x7f389406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.002+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f389406b4d0 0x7f389406d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3898001eb0 tx=0x7f3898001b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.003+0000 7f38b05a3740 1 Processor -- start 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.003+0000 7f38b05a3740 1 -- start start 2024-04-25T10:03:13.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38b05a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceaf1b10 0x563eceaf4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38b05a3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 0x563eceaf4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38b05a3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceac4770 0x563eceac7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38b05a3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ecea2dda0 con 0x563eceaf1ef0 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38b05a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ece9d6b80 con 0x563eceaf1b10 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38b05a3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ecea29e40 con 0x563eceac4770 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38a67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 0x563eceaf4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38a67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 0x563eceaf4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52626/0 (socket says 172.21.15.139:52626) 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38a67fc640 1 -- 172.21.15.139:0/1264719546 learned_addr learned my addr 172.21.15.139:0/1264719546 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/1264719546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceaf1b10 0x563eceaf4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38a77fe640 1 --2- 172.21.15.139:0/1264719546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceac4770 0x563eceac7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38a67fc640 1 -- 172.21.15.139:0/1264719546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceac4770 msgr2=0x563eceac7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38a67fc640 1 --2- 172.21.15.139:0/1264719546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceac4770 0x563eceac7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38a67fc640 1 -- 172.21.15.139:0/1264719546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceaf1b10 msgr2=0x563eceaf4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38a67fc640 1 --2- 172.21.15.139:0/1264719546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceaf1b10 0x563eceaf4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38a67fc640 1 -- 172.21.15.139:0/1264719546 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ece9e0f10 con 0x563eceaf1ef0 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f38a67fc640 1 --2- 172.21.15.139:0/1264719546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 0x563eceaf4860 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f389c009a50 tx=0x7f389c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdeda0637fc71ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f388d7fa640 1 -- 172.21.15.139:0/1264719546 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c01b070 con 0x563eceaf1ef0 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f388d7fa640 1 -- 172.21.15.139:0/1264719546 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c013b60 con 0x563eceaf1ef0 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.004+0000 7f388d7fa640 1 -- 172.21.15.139:0/1264719546 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c016460 con 0x563eceaf1ef0 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.005+0000 7f38b05a3740 1 -- 172.21.15.139:0/1264719546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 msgr2=0x563eceaf4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.005+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1264719546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 0x563eceaf4860 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f389c009a50 tx=0x7f389c009190 comp rx=0 tx=0).stop 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.005+0000 7f38b05a3740 1 -- 172.21.15.139:0/1264719546 shutdown_connections 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.005+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1264719546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 0x563eceaf4860 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.005+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1264719546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceac4770 0x563eceac7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.005+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1264719546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceaf1b10 0x563eceaf4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.005+0000 7f38b05a3740 1 -- 172.21.15.139:0/1264719546 >> 172.21.15.139:0/1264719546 conn(0x563eceac69e0 msgr2=0x563eceb0f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.005+0000 7f38b05a3740 1 -- 172.21.15.139:0/1264719546 shutdown_connections 2024-04-25T10:03:13.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.005+0000 7f38b05a3740 1 -- 172.21.15.139:0/1264719546 wait complete. 2024-04-25T10:03:13.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.006+0000 7f38b05a3740 1 -- start start 2024-04-25T10:03:13.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.006+0000 7f38b05a3740 1 Processor -- start 2024-04-25T10:03:13.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.006+0000 7f38b05a3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 0x563eceaea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.006+0000 7f38b05a3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceaf1ef0 0x563eceb0ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.006+0000 7f38b05a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceb0b2d0 0x563eceb11690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.006+0000 7f38b05a3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eceac9b90 con 0x563eceaf1b10 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.006+0000 7f38b05a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ece9cc4b0 con 0x563eceb0b2d0 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.006+0000 7f38b05a3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563eceac9e70 con 0x563eceaf1ef0 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38b05a3740 10 client.0 _ll_register_callbacks cb 0x563ecea0c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38a6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 0x563eceaea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38a6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 0x563eceaea3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52632/0 (socket says 172.21.15.139:52632) 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38a6ffd640 1 -- 172.21.15.139:0/1615871695 learned_addr learned my addr 172.21.15.139:0/1615871695 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38a67fc640 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceaf1ef0 0x563eceb0ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38a77fe640 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceb0b2d0 0x563eceb11690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38a6ffd640 1 -- 172.21.15.139:0/1615871695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceaf1ef0 msgr2=0x563eceb0ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceaf1ef0 0x563eceb0ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38a6ffd640 1 -- 172.21.15.139:0/1615871695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceb0b2d0 msgr2=0x563eceb11690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceb0b2d0 0x563eceb11690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38a6ffd640 1 -- 172.21.15.139:0/1615871695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ece80c7d0 con 0x563eceaf1b10 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38a67fc640 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceaf1ef0 0x563eceb0ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 0x563eceaea3c0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f389800ff20 tx=0x7f3898009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac77de39319a1a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38867fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38867fc640 1 -- 172.21.15.139:0/1615871695 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898028060 con 0x563eceaf1b10 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38867fc640 1 -- 172.21.15.139:0/1615871695 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3898023c70 con 0x563eceaf1b10 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38b05a3740 1 -- 172.21.15.139:0/1615871695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f389c013b60 con 0x563eceaf1b10 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38867fc640 1 -- 172.21.15.139:0/1615871695 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3898010040 con 0x563eceaf1b10 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38b05a3740 10 client.9555 fetch_fsmap learned FSMap version 89 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38b05a3740 1 -- 172.21.15.139:0/1615871695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x563ece7423f0 con 0x563eceaf1b10 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38867fc640 1 -- 172.21.15.139:0/1615871695 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c013b60 con 0x563eceaf1b10 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38867fc640 1 -- 172.21.15.139:0/1615871695 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3898013070 con 0x563eceaf1b10 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38b05a3740 10 client.9555 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:03:13.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38b05a3740 10 client.9555 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.007+0000 7f38b05a3740 20 client.9555 populate_metadata read hostname 'smithi139' 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.008+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x563eceb0bbd0 0x563eceb0bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.008+0000 7f38b05a3740 4 client.9555 mds_command: new command op to 9117 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.008+0000 7f38b05a3740 1 -- 172.21.15.139:0/1615871695 --> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ecea3b2e0 con 0x563eceb0bbd0 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.008+0000 7f38a67fc640 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x563eceb0bbd0 0x563eceb0bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.008+0000 7f38a67fc640 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x563eceb0bbd0 0x563eceb0bff0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1211deb25e5b067 server_cookie=16ff0be028cda9a3 in_seq=0 out_seq=0 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.008+0000 7f38867fc640 10 client.9555 ms_handle_connect on v2:172.21.15.139:6834/3478297010 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.008+0000 7f38867fc640 1 -- 172.21.15.139:0/1615871695 <== mds.0 v2:172.21.15.139:6834/3478297010 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ecea3b2e0 con 0x563eceb0bbd0 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.008+0000 7f38867fc640 10 client.9555 handle_command_reply: tid=0 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.008+0000 7f38b05a3740 1 client.9555 shutdown 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.009+0000 7f38b05a3740 1 -- 172.21.15.139:0/1615871695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 msgr2=0x563eceaea3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.009+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 0x563eceaea3c0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f389800ff20 tx=0x7f3898009070 comp rx=0 tx=0).stop 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.009+0000 7f38b05a3740 1 -- 172.21.15.139:0/1615871695 shutdown_connections 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.009+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x563eceb0bbd0 0x563eceb0bff0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.009+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceb0b2d0 0x563eceb11690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.009+0000 7f38867fc640 0 client.9555 ms_handle_reset on v2:172.21.15.139:6834/3478297010 2024-04-25T10:03:13.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.009+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceaf1ef0 0x563eceb0ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.009+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1615871695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 0x563eceaea3c0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.009+0000 7f38b05a3740 1 -- 172.21.15.139:0/1615871695 >> 172.21.15.139:0/1615871695 conn(0x563eceac4770 msgr2=0x563eceaf0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.009+0000 7f38b05a3740 1 -- 172.21.15.139:0/1615871695 shutdown_connections 2024-04-25T10:03:13.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.009+0000 7f38b05a3740 1 -- 172.21.15.139:0/1615871695 wait complete. 2024-04-25T10:03:13.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.009+0000 7f38b05a3740 20 client.9555 trim_cache size 0 max 16384 2024-04-25T10:03:13.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.022+0000 7f38b05a3740 1 Processor -- start 2024-04-25T10:03:13.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.022+0000 7f38b05a3740 1 -- start start 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38b05a3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceaf1b10 0x563eceb31c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38b05a3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 0x563eceb12280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38b05a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceb0f4b0 0x563eceb127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38b05a3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a8200be0 con 0x563eceaf1ef0 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38b05a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a81ffde0 con 0x563eceb0f4b0 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38b05a3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a81ff1e0 con 0x563eceaf1b10 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 0x563eceb12280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 0x563eceb12280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52640/0 (socket says 172.21.15.139:52640) 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a67fc640 1 -- 172.21.15.139:0/1634376512 learned_addr learned my addr 172.21.15.139:0/1634376512 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a77fe640 1 --2- 172.21.15.139:0/1634376512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceb0f4b0 0x563eceb127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/1634376512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceaf1b10 0x563eceb31c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a67fc640 1 -- 172.21.15.139:0/1634376512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceaf1b10 msgr2=0x563eceb31c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a67fc640 1 --2- 172.21.15.139:0/1634376512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceaf1b10 0x563eceb31c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a67fc640 1 -- 172.21.15.139:0/1634376512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceb0f4b0 msgr2=0x563eceb127c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a67fc640 1 --2- 172.21.15.139:0/1634376512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceb0f4b0 0x563eceb127c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a67fc640 1 -- 172.21.15.139:0/1634376512 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563eceaea730 con 0x563eceaf1ef0 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/1634376512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceaf1b10 0x563eceb31c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a77fe640 1 --2- 172.21.15.139:0/1634376512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceb0f4b0 0x563eceb127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f38a67fc640 1 --2- 172.21.15.139:0/1634376512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 0x563eceb12280 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f389c01a000 tx=0x7f389c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbd614d158a31b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f388cff9640 1 -- 172.21.15.139:0/1634376512 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c01b040 con 0x563eceaf1ef0 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f388cff9640 1 -- 172.21.15.139:0/1634376512 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c004250 con 0x563eceaf1ef0 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.023+0000 7f388cff9640 1 -- 172.21.15.139:0/1634376512 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c003610 con 0x563eceaf1ef0 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.024+0000 7f38b05a3740 1 -- 172.21.15.139:0/1634376512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 msgr2=0x563eceb12280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.024+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1634376512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 0x563eceb12280 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f389c01a000 tx=0x7f389c018000 comp rx=0 tx=0).stop 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.024+0000 7f38b05a3740 1 -- 172.21.15.139:0/1634376512 shutdown_connections 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.024+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1634376512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1ef0 0x563eceb12280 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.024+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1634376512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceb0f4b0 0x563eceb127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.024+0000 7f38b05a3740 1 --2- 172.21.15.139:0/1634376512 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceaf1b10 0x563eceb31c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.024+0000 7f38b05a3740 1 -- 172.21.15.139:0/1634376512 >> 172.21.15.139:0/1634376512 conn(0x563eceac4770 msgr2=0x563eceaca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.024+0000 7f38b05a3740 1 -- 172.21.15.139:0/1634376512 shutdown_connections 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.024+0000 7f38b05a3740 1 -- 172.21.15.139:0/1634376512 wait complete. 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.024+0000 7f38b05a3740 1 -- start start 2024-04-25T10:03:13.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.024+0000 7f38b05a3740 1 Processor -- start 2024-04-25T10:03:13.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.025+0000 7f38b05a3740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 0x563eceb79a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.025+0000 7f38b05a3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceaf1ef0 0x563eceb12280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.025+0000 7f38b05a3740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceb0f4b0 0x563eceb127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.025+0000 7f38b05a3740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eceaca0b0 con 0x563eceaf1b10 2024-04-25T10:03:13.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.025+0000 7f38b05a3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a81ff7e0 con 0x563eceaf1ef0 2024-04-25T10:03:13.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.025+0000 7f38b05a3740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563eceaf8350 con 0x563eceb0f4b0 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.025+0000 7f38b05a3740 10 client.0 _ll_register_callbacks cb 0x7f38980013b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.025+0000 7f38a6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 0x563eceb79a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.025+0000 7f38a6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 0x563eceb79a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52652/0 (socket says 172.21.15.139:52652) 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.025+0000 7f38a6ffd640 1 -- 172.21.15.139:0/988285675 learned_addr learned my addr 172.21.15.139:0/988285675 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.025+0000 7f38a77fe640 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceb0f4b0 0x563eceb127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.025+0000 7f38a67fc640 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceaf1ef0 0x563eceb12280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38a6ffd640 1 -- 172.21.15.139:0/988285675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceb0f4b0 msgr2=0x563eceb127c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceb0f4b0 0x563eceb127c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38a6ffd640 1 -- 172.21.15.139:0/988285675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceaf1ef0 msgr2=0x563eceb12280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceaf1ef0 0x563eceb12280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38a6ffd640 1 -- 172.21.15.139:0/988285675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ece7020f0 con 0x563eceaf1b10 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38a6ffd640 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 0x563eceb79a60 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f3898001970 tx=0x7f3898000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ddaebb8c80dafe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38867fc640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:03:13.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38867fc640 1 -- 172.21.15.139:0/988285675 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389800fce0 con 0x563eceaf1b10 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38867fc640 1 -- 172.21.15.139:0/988285675 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f389801f070 con 0x563eceaf1b10 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38b05a3740 1 -- 172.21.15.139:0/988285675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f389c004250 con 0x563eceaf1b10 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38867fc640 1 -- 172.21.15.139:0/988285675 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f389803d050 con 0x563eceaf1b10 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38b05a3740 10 client.9561 fetch_fsmap learned FSMap version 89 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38b05a3740 1 -- 172.21.15.139:0/988285675 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x563eceaf7df0 con 0x563eceaf1b10 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38867fc640 1 -- 172.21.15.139:0/988285675 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c004250 con 0x563eceaf1b10 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38867fc640 1 -- 172.21.15.139:0/988285675 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3898040020 con 0x563eceaf1b10 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38b05a3740 10 client.9561 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38b05a3740 10 client.9561 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38b05a3740 20 client.9561 populate_metadata read hostname 'smithi139' 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38b05a3740 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x563eceb13c70 0x563eceaf7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38b05a3740 4 client.9561 mds_command: new command op to 9117 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38b05a3740 1 -- 172.21.15.139:0/988285675 --> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563ecea35630 con 0x563eceb13c70 2024-04-25T10:03:13.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.026+0000 7f38a67fc640 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x563eceb13c70 0x563eceaf7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.027+0000 7f38a67fc640 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x563eceb13c70 0x563eceaf7120 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d14d164ef2d9274 server_cookie=c963864c24811508 in_seq=0 out_seq=0 2024-04-25T10:03:13.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.027+0000 7f38867fc640 10 client.9561 ms_handle_connect on v2:172.21.15.139:6834/3478297010 2024-04-25T10:03:13.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38867fc640 1 -- 172.21.15.139:0/988285675 <== mds.0 v2:172.21.15.139:6834/3478297010 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563ecea35630 con 0x563eceb13c70 2024-04-25T10:03:13.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38867fc640 10 client.9561 handle_command_reply: tid=0 2024-04-25T10:03:13.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38b05a3740 1 client.9561 shutdown 2024-04-25T10:03:13.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38b05a3740 1 -- 172.21.15.139:0/988285675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 msgr2=0x563eceb79a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38b05a3740 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 0x563eceb79a60 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f3898001970 tx=0x7f3898000fe0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38b05a3740 1 -- 172.21.15.139:0/988285675 shutdown_connections 2024-04-25T10:03:13.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38b05a3740 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.139:6834/3478297010,v1:172.21.15.139:6836/3478297010] conn(0x563eceb13c70 0x563eceaf7120 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38b05a3740 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563eceb0f4b0 0x563eceb127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38b05a3740 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563eceaf1ef0 0x563eceb12280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38867fc640 0 client.9561 ms_handle_reset on v2:172.21.15.139:6834/3478297010 2024-04-25T10:03:13.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38b05a3740 1 --2- 172.21.15.139:0/988285675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x563eceaf1b10 0x563eceb79a60 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38b05a3740 1 -- 172.21.15.139:0/988285675 >> 172.21.15.139:0/988285675 conn(0x563eceac4770 msgr2=0x563eceb31810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38b05a3740 1 -- 172.21.15.139:0/988285675 shutdown_connections 2024-04-25T10:03:13.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38b05a3740 1 -- 172.21.15.139:0/988285675 wait complete. 2024-04-25T10:03:13.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.037+0000 7f38b05a3740 20 client.9561 trim_cache size 0 max 16384 2024-04-25T10:03:13.043 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:03:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: "message": "", 2024-04-25T10:03:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: "return_code": 0 2024-04-25T10:03:13.044 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:03:13.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.038+0000 7f38ad89e640 1 -- 172.21.15.139:0/2286148909 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f389406b4d0 msgr2=0x7f389406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.038+0000 7f38ad89e640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f389406b4d0 0x7f389406d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3898001eb0 tx=0x7f3898001b50 comp rx=0 tx=0).stop 2024-04-25T10:03:13.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.038+0000 7f38ad89e640 1 -- 172.21.15.139:0/2286148909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38a81604b0 msgr2=0x7f38a8160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.038+0000 7f38ad89e640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38a81604b0 0x7f38a8160930 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f38a000e170 tx=0x7f38a0009750 comp rx=0 tx=0).stop 2024-04-25T10:03:13.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.038+0000 7f38ad89e640 1 -- 172.21.15.139:0/2286148909 shutdown_connections 2024-04-25T10:03:13.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.038+0000 7f38ad89e640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38a81604b0 0x7f38a8160930 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.038+0000 7f38ad89e640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f389406b4d0 0x7f389406d990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.038+0000 7f38ad89e640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38a81f6c80 0x7f38a81552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.038+0000 7f38ad89e640 1 --2- 172.21.15.139:0/2286148909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3890004d40 0x7f38a80c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.038+0000 7f38ad89e640 1 -- 172.21.15.139:0/2286148909 >> 172.21.15.139:0/2286148909 conn(0x7f38a80b5480 msgr2=0x7f38a80d2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.038+0000 7f38ad89e640 1 -- 172.21.15.139:0/2286148909 shutdown_connections 2024-04-25T10:03:13.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.038+0000 7f38ad89e640 1 -- 172.21.15.139:0/2286148909 wait complete. 2024-04-25T10:03:13.053 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:03:13.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.147+0000 7fceff1b6640 1 Processor -- start 2024-04-25T10:03:13.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.147+0000 7fceff1b6640 1 -- start start 2024-04-25T10:03:13.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.148+0000 7fceff1b6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcef80b5620 0x7fcef80b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.148+0000 7fceff1b6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef80b5a20 con 0x7fcef8070970 2024-04-25T10:03:13.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.148+0000 7fceff1b6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef80d0d40 con 0x7fcef815a870 2024-04-25T10:03:13.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.148+0000 7fceff1b6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef80d1020 con 0x7fcef80b5620 2024-04-25T10:03:13.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.148+0000 7fcefd753640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcef80b5620 0x7fcef80b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.148+0000 7fcefd753640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcef80b5620 0x7fcef80b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52668/0 (socket says 172.21.15.139:52668) 2024-04-25T10:03:13.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.148+0000 7fcefd753640 1 -- 172.21.15.139:0/471055038 learned_addr learned my addr 172.21.15.139:0/471055038 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.149+0000 7fcefd753640 1 -- 172.21.15.139:0/471055038 >> v1:172.21.15.150:6789/0 conn(0x7fcef815a870 legacy=0x7fcef8070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.149+0000 7fcefd753640 1 -- 172.21.15.139:0/471055038 >> v1:172.21.15.139:6789/0 conn(0x7fcef8070970 legacy=0x7fcef80ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.149+0000 7fcef77fe640 1 -- 172.21.15.139:0/471055038 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967804409 0 0) 0x7fcef80b5a20 con 0x7fcef8070970 2024-04-25T10:03:13.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.149+0000 7fcefd753640 1 -- 172.21.15.139:0/471055038 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef80b4b40 con 0x7fcef80b5620 2024-04-25T10:03:13.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.149+0000 7fcefd753640 1 --2- 172.21.15.139:0/471055038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef80b5620 0x7fcef80b4190 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009e50 tx=0x7fcef002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a40ce68205c34d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.149+0000 7fcef77fe640 1 -- 172.21.15.139:0/471055038 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0036070 con 0x7fcef80b5620 2024-04-25T10:03:13.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.150+0000 7fcef77fe640 1 -- 172.21.15.139:0/471055038 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0031600 con 0x7fcef80b5620 2024-04-25T10:03:13.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.150+0000 7fcef77fe640 1 -- 172.21.15.139:0/471055038 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef00319d0 con 0x7fcef80b5620 2024-04-25T10:03:13.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.150+0000 7fceff1b6640 1 -- 172.21.15.139:0/471055038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef80b5620 msgr2=0x7fcef80b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.150+0000 7fceff1b6640 1 --2- 172.21.15.139:0/471055038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef80b5620 0x7fcef80b4190 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009e50 tx=0x7fcef002a6d0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.150+0000 7fceff1b6640 1 -- 172.21.15.139:0/471055038 shutdown_connections 2024-04-25T10:03:13.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.150+0000 7fceff1b6640 1 --2- 172.21.15.139:0/471055038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef80b5620 0x7fcef80b4190 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.150+0000 7fceff1b6640 1 -- 172.21.15.139:0/471055038 >> 172.21.15.139:0/471055038 conn(0x7fcef80b23f0 msgr2=0x7fcef80b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.150+0000 7fceff1b6640 1 -- 172.21.15.139:0/471055038 shutdown_connections 2024-04-25T10:03:13.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.150+0000 7fceff1b6640 1 -- 172.21.15.139:0/471055038 wait complete. 2024-04-25T10:03:13.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.151+0000 7fceff1b6640 1 Processor -- start 2024-04-25T10:03:13.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.151+0000 7fceff1b6640 1 -- start start 2024-04-25T10:03:13.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fceff1b6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcef8070970 0x7fcef814b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fceff1b6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcef80b5620 0x7fcef81f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fceff1b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef815a870 0x7fcef81e5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fceff1b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef80d05f0 con 0x7fcef815a870 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fceff1b6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef80b5190 con 0x7fcef80b5620 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fceff1b6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef80d0770 con 0x7fcef8070970 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fcefd753640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef815a870 0x7fcef81e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fcefd753640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef815a870 0x7fcef81e5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52678/0 (socket says 172.21.15.139:52678) 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fcefd753640 1 -- 172.21.15.139:0/485227567 learned_addr learned my addr 172.21.15.139:0/485227567 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fcefd753640 1 -- 172.21.15.139:0/485227567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcef8070970 msgr2=0x7fcef814b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fcefd753640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcef8070970 0x7fcef814b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fcefd753640 1 -- 172.21.15.139:0/485227567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcef80b5620 msgr2=0x7fcef81f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fcef7fff640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcef80b5620 0x7fcef81f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fcefd753640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcef80b5620 0x7fcef81f24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.152+0000 7fcefd753640 1 -- 172.21.15.139:0/485227567 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef81e3d10 con 0x7fcef815a870 2024-04-25T10:03:13.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.153+0000 7fcefd753640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef815a870 0x7fcef81e5050 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009e50 tx=0x7fcef0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.153+0000 7fcef57fa640 1 -- 172.21.15.139:0/485227567 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0041020 con 0x7fcef815a870 2024-04-25T10:03:13.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.153+0000 7fcef57fa640 1 -- 172.21.15.139:0/485227567 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0007780 con 0x7fcef815a870 2024-04-25T10:03:13.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.153+0000 7fceff1b6640 1 -- 172.21.15.139:0/485227567 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef81e3fb0 con 0x7fcef815a870 2024-04-25T10:03:13.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.153+0000 7fcef57fa640 1 -- 172.21.15.139:0/485227567 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0036040 con 0x7fcef815a870 2024-04-25T10:03:13.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.153+0000 7fceff1b6640 1 -- 172.21.15.139:0/485227567 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef81e4570 con 0x7fcef815a870 2024-04-25T10:03:13.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.154+0000 7fceff1b6640 1 -- 172.21.15.139:0/485227567 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef80b21a0 con 0x7fcef815a870 2024-04-25T10:03:13.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.154+0000 7fcef57fa640 1 -- 172.21.15.139:0/485227567 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcef0044020 con 0x7fcef815a870 2024-04-25T10:03:13.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.155+0000 7fcef57fa640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcee406b4d0 0x7fcee406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.155+0000 7fcef57fa640 1 -- 172.21.15.139:0/485227567 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcef00ab140 con 0x7fcef815a870 2024-04-25T10:03:13.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.155+0000 7fcefcf52640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcee406b4d0 0x7fcee406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.155+0000 7fcefcf52640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcee406b4d0 0x7fcee406d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fcef81f4860 tx=0x7fcee8006ba0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.159+0000 7fcef57fa640 1 -- 172.21.15.139:0/485227567 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcef80b21a0 con 0x7fcef815a870 2024-04-25T10:03:13.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.363+0000 7fceff1b6640 1 -- 172.21.15.139:0/485227567 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcef80cf820 con 0x7fcef815a870 2024-04-25T10:03:13.368 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T10:03:13.363+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:03:13.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.367+0000 7fcef57fa640 1 -- 172.21.15.139:0/485227567 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcef004b030 con 0x7fcef815a870 2024-04-25T10:03:13.371 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:03:13.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.369+0000 7fceff1b6640 1 -- 172.21.15.139:0/485227567 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcee406b4d0 msgr2=0x7fcee406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.369+0000 7fceff1b6640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcee406b4d0 0x7fcee406d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fcef81f4860 tx=0x7fcee8006ba0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.369+0000 7fceff1b6640 1 -- 172.21.15.139:0/485227567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef815a870 msgr2=0x7fcef81e5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.369+0000 7fceff1b6640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef815a870 0x7fcef81e5050 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009e50 tx=0x7fcef0004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:13.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.369+0000 7fceff1b6640 1 -- 172.21.15.139:0/485227567 shutdown_connections 2024-04-25T10:03:13.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.369+0000 7fceff1b6640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef815a870 0x7fcef81e5050 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.369+0000 7fceff1b6640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcee406b4d0 0x7fcee406d990 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.369+0000 7fceff1b6640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcef80b5620 0x7fcef81f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.369+0000 7fceff1b6640 1 --2- 172.21.15.139:0/485227567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcef8070970 0x7fcef814b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.369+0000 7fceff1b6640 1 -- 172.21.15.139:0/485227567 >> 172.21.15.139:0/485227567 conn(0x7fcef80b23f0 msgr2=0x7fcef80b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.369+0000 7fceff1b6640 1 -- 172.21.15.139:0/485227567 shutdown_connections 2024-04-25T10:03:13.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.369+0000 7fceff1b6640 1 -- 172.21.15.139:0/485227567 wait complete. 2024-04-25T10:03:13.377 INFO:tasks.ceph.mds.a.smithi139.stderr: -935> 2024-04-25T10:03:11.798+0000 7fb5f3191640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:13.386 DEBUG:teuthology.orchestra.run.smithi139:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-25T10:03:13.402 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T10:03:13.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.423+0000 7fe101e24280 1 -- start start 2024-04-25T10:03:13.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.423+0000 7fe101e24280 1 Processor -- start 2024-04-25T10:03:13.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.423+0000 7fe101e24280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cda4e000 0x55d3cccf7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.423+0000 7fe101e24280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3ccc87c00 0x55d3cccf7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.423+0000 7fe101e24280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda4e800 0x55d3cccf7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.423+0000 7fe101e24280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cda50480 con 0x55d3cda4e800 2024-04-25T10:03:13.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.423+0000 7fe101e24280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cda50600 con 0x55d3ccc87c00 2024-04-25T10:03:13.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.423+0000 7fe101e24280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3cda50780 con 0x55d3cda4e000 2024-04-25T10:03:13.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.423+0000 7fe100dd1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda4e800 0x55d3cccf7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.423+0000 7fe100dd1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda4e800 0x55d3cccf7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52716/0 (socket says 172.21.15.139:52716) 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.423+0000 7fe100dd1640 1 -- 172.21.15.139:0/2363211794 learned_addr learned my addr 172.21.15.139:0/2363211794 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe100dd1640 1 -- 172.21.15.139:0/2363211794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cda4e000 msgr2=0x55d3cccf7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe100dd1640 1 --2- 172.21.15.139:0/2363211794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cda4e000 0x55d3cccf7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe100dd1640 1 -- 172.21.15.139:0/2363211794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3ccc87c00 msgr2=0x55d3cccf7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe100dd1640 1 --2- 172.21.15.139:0/2363211794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3ccc87c00 0x55d3cccf7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe100dd1640 1 -- 172.21.15.139:0/2363211794 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3cccbc780 con 0x55d3cda4e800 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe100dd1640 1 --2- 172.21.15.139:0/2363211794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda4e800 0x55d3cccf7600 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x55d3cdab14a0 tx=0x55d3cda196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5852c741e465f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe0fddcb640 1 -- 172.21.15.139:0/2363211794 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d3cccacd00 con 0x55d3cda4e800 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe0fddcb640 1 -- 172.21.15.139:0/2363211794 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d3cccb2fc0 con 0x55d3cda4e800 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe0fddcb640 1 -- 172.21.15.139:0/2363211794 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d3cccac680 con 0x55d3cda4e800 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe101e24280 1 -- 172.21.15.139:0/2363211794 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3cccbc000 con 0x55d3cda4e800 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe101e24280 1 -- 172.21.15.139:0/2363211794 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55d3cccbd680 con 0x55d3cda4e800 2024-04-25T10:03:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe101e24280 4 waiting for MDS map... 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe0fddcb640 1 -- 172.21.15.139:0/2363211794 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x55d3cdb56400 con 0x55d3cda4e800 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe0fddcb640 1 -- 172.21.15.139:0/2363211794 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 90) v1 ==== 2723+0+0 (secure 0 0 0) 0x55d3ccce1180 con 0x55d3cda4e800 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe101e24280 4 Got MDS map 90 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe101e24280 10 main: JournalTool::main 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.424+0000 7fe101e24280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.425+0000 7fe101e24280 1 Processor -- start 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.425+0000 7fe101e24280 1 -- start start 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.425+0000 7fe101e24280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 0x55d3cccf6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.425+0000 7fe101e24280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb58c00 0x55d3cccf8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.425+0000 7fe101e24280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cdb59000 0x55d3cccf8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.425+0000 7fe101e24280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ccce8180 con 0x55d3cdb58800 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.425+0000 7fe101e24280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cda50f00 con 0x55d3cdb58c00 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.425+0000 7fe101e24280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3cda51080 con 0x55d3cdb59000 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.425+0000 7fe0ffdcf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 0x55d3cccf6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.425+0000 7fe0ffdcf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 0x55d3cccf6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52720/0 (socket says 172.21.15.139:52720) 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.425+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/783372989 learned_addr learned my addr 172.21.15.139:0/783372989 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe0ffdcf640 1 --2- 172.21.15.139:0/783372989 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cdb59000 0x55d3cccf8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe1005d0640 1 --2- 172.21.15.139:0/783372989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb58c00 0x55d3cccf8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/783372989 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cdb59000 msgr2=0x55d3cccf8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe0ffdcf640 1 --2- 172.21.15.139:0/783372989 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cdb59000 0x55d3cccf8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/783372989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb58c00 msgr2=0x55d3cccf8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe0ffdcf640 1 --2- 172.21.15.139:0/783372989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb58c00 0x55d3cccf8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/783372989 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3cccbd860 con 0x55d3cdb58800 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe0ffdcf640 1 --2- 172.21.15.139:0/783372989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 0x55d3cccf6b00 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x55d3cdb66900 tx=0x55d3cdbbe060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d8376e2cde99ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe0fbdc7640 1 -- 172.21.15.139:0/783372989 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d3cccacd00 con 0x55d3cdb58800 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe0fbdc7640 1 -- 172.21.15.139:0/783372989 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d3cccb2e00 con 0x55d3cdb58800 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe0fbdc7640 1 -- 172.21.15.139:0/783372989 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d3cccac680 con 0x55d3cdb58800 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe101e24280 1 -- 172.21.15.139:0/783372989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 msgr2=0x55d3cccf6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe101e24280 1 --2- 172.21.15.139:0/783372989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 0x55d3cccf6b00 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x55d3cdb66900 tx=0x55d3cdbbe060 comp rx=0 tx=0).stop 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe101e24280 1 -- 172.21.15.139:0/783372989 shutdown_connections 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe101e24280 1 --2- 172.21.15.139:0/783372989 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cdb59000 0x55d3cccf8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe101e24280 1 --2- 172.21.15.139:0/783372989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb58c00 0x55d3cccf8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe101e24280 1 --2- 172.21.15.139:0/783372989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 0x55d3cccf6b00 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.426+0000 7fe101e24280 1 -- 172.21.15.139:0/783372989 >> 172.21.15.139:0/783372989 conn(0x55d3cda4e400 msgr2=0x55d3ccd81800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.427+0000 7fe101e24280 1 -- 172.21.15.139:0/783372989 shutdown_connections 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.427+0000 7fe101e24280 1 -- 172.21.15.139:0/783372989 wait complete. 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.427+0000 7fe101e24280 1 Processor -- start 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.427+0000 7fe101e24280 1 -- start start 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe101e24280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 0x55d3cccf6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe101e24280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb58c00 0x55d3cccf8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe101e24280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cdb59000 0x55d3cccf8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe101e24280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cda51380 con 0x55d3cdb58800 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe101e24280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cda51200 con 0x55d3cdb58c00 2024-04-25T10:03:13.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe101e24280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3cda51500 con 0x55d3cdb59000 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0ffdcf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 0x55d3cccf6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0ffdcf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 0x55d3cccf6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52726/0 (socket says 172.21.15.139:52726) 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/2520623342 learned_addr learned my addr 172.21.15.139:0/2520623342 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe1005d0640 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb58c00 0x55d3cccf8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0ffdcf640 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cdb59000 0x55d3cccf8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/2520623342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cdb59000 msgr2=0x55d3cccf8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0ffdcf640 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cdb59000 0x55d3cccf8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/2520623342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb58c00 msgr2=0x55d3cccf8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0ffdcf640 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb58c00 0x55d3cccf8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/2520623342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3cccbc780 con 0x55d3cdb58800 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0ffdcf640 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 0x55d3cccf6b00 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x55d3cdb675c0 tx=0x55d3cdbbe000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0fa5c4640 1 -- 172.21.15.139:0/2520623342 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d3cccac820 con 0x55d3cdb58800 2024-04-25T10:03:13.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0fa5c4640 1 -- 172.21.15.139:0/2520623342 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d3cccb2c40 con 0x55d3cdb58800 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe101e24280 1 -- 172.21.15.139:0/2520623342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d3cccbc000 con 0x55d3cdb58800 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe0fa5c4640 1 -- 172.21.15.139:0/2520623342 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d3cccacd00 con 0x55d3cdb58800 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe101e24280 1 -- 172.21.15.139:0/2520623342 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3cccbda40 con 0x55d3cdb58800 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.428+0000 7fe101e24280 4 main: JournalTool: resolving pool 17 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.429+0000 7fe0fa5c4640 1 -- 172.21.15.139:0/2520623342 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55d3ccde8c00 con 0x55d3cdb58800 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.430+0000 7fe0fa5c4640 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55d3cda88800 0x55d3cccf9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.430+0000 7fe0fa5c4640 1 -- 172.21.15.139:0/2520623342 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x55d3ccde6a00 con 0x55d3cdb58800 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.430+0000 7fe101e24280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.430+0000 7fe101e24280 4 main: Executing for rank 0 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.430+0000 7fe1005d0640 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55d3cda88800 0x55d3cccf9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.430+0000 7fe1005d0640 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55d3cda88800 0x55d3cccf9700 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x55d3ccd74a50 tx=0x55d3cdbbe240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.430+0000 7fe101e24280 1 -- start start 2024-04-25T10:03:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.430+0000 7fe101e24280 1 Processor -- start 2024-04-25T10:03:13.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe101e24280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb59800 0x55d3cccf8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe101e24280 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda88c00 0x55d3cccf9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe101e24280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cda89000 0x55d3cd052000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe101e24280 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cda51980 con 0x55d3cda88c00 2024-04-25T10:03:13.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe101e24280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cda51b00 con 0x55d3cdb59800 2024-04-25T10:03:13.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe101e24280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3cda51c80 con 0x55d3cda89000 2024-04-25T10:03:13.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe1005d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda88c00 0x55d3cccf9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe1005d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda88c00 0x55d3cccf9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52742/0 (socket says 172.21.15.139:52742) 2024-04-25T10:03:13.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe1005d0640 1 -- 172.21.15.139:0/423946490 learned_addr learned my addr 172.21.15.139:0/423946490 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe100dd1640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb59800 0x55d3cccf8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe1005d0640 1 -- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cda89000 msgr2=0x55d3cd052000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe1005d0640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cda89000 0x55d3cd052000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe1005d0640 1 -- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb59800 msgr2=0x55d3cccf8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe1005d0640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb59800 0x55d3cccf8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.431+0000 7fe1005d0640 1 -- 172.21.15.139:0/423946490 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3cccbdc20 con 0x55d3cda88c00 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe1005d0640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda88c00 0x55d3cccf9180 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x55d3cdab0e40 tx=0x55d3cdbbe300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9de0e2b3b72d4dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe0f75be640 1 -- 172.21.15.139:0/423946490 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d3cdc036c0 con 0x55d3cda88c00 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe0f75be640 1 -- 172.21.15.139:0/423946490 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d3cccb2700 con 0x55d3cda88c00 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe0f75be640 1 -- 172.21.15.139:0/423946490 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d3cdc03520 con 0x55d3cda88c00 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3cccbd4a0 con 0x55d3cda88c00 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55d3cccbc960 con 0x55d3cda88c00 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe101e24280 4 waiting for MDS map... 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe0f75be640 1 -- 172.21.15.139:0/423946490 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x55d3cd056200 con 0x55d3cda88c00 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe0f75be640 1 -- 172.21.15.139:0/423946490 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 90) v1 ==== 2723+0+0 (secure 0 0 0) 0x55d3cd05c000 con 0x55d3cda88c00 2024-04-25T10:03:13.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe101e24280 4 Got MDS map 90 2024-04-25T10:03:13.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe101e24280 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55d3cda89400 0x55d3cd052580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55d3cda89800 con 0x55d3cda89400 2024-04-25T10:03:13.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe100dd1640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55d3cda89400 0x55d3cd052580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.432+0000 7fe100dd1640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55d3cda89400 0x55d3cd052580 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=d7021a433a0f8f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.437+0000 7fe100dd1640 1 -- 172.21.15.139:0/423946490 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v122'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55d3cda06000 con 0x55d3cda89400 2024-04-25T10:03:13.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.437+0000 7fe101e24280 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55d3cda89c00 0x55d3cd052b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.437+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55d3cd059400 con 0x55d3cda89c00 2024-04-25T10:03:13.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.437+0000 7fe0ffdcf640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55d3cda89c00 0x55d3cd052b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.437+0000 7fe0ffdcf640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55d3cda89c00 0x55d3cd052b00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=8031cf738b9060ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/423946490 <== osd.3 v2:172.21.15.139:6824/1040931291 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v122'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55d3cda06500 con 0x55d3cda89c00 2024-04-25T10:03:13.447 INFO:teuthology.orchestra.run.smithi139.stdout:writing EResetJournal entry 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55d3cd059800 con 0x55d3cda89c00 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe101e24280 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55d3cd059c00 0x55d3cd053080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55d3cd082000 con 0x55d3cd059c00 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe101e24280 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55d3cd082400 0x55d3cd053600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55d3cd082800 con 0x55d3cd082400 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe101e24280 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55d3cd082c00 0x55d3cd053b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55d3cd083000 con 0x55d3cd082c00 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55d3cd083400 con 0x55d3cd082400 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55d3cd083800 con 0x55d3cda89400 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe1005d0640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55d3cd082400 0x55d3cd053600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe100dd1640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55d3cd059c00 0x55d3cd053080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.440+0000 7fe0ffdcf640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55d3cd082c00 0x55d3cd053b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.441+0000 7fe100dd1640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55d3cd059c00 0x55d3cd053080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=2c484d54cc7948a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.441+0000 7fe0ffdcf640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55d3cd082c00 0x55d3cd053b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=b08b1fc6b97c8528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.441+0000 7fe1005d0640 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55d3cd082400 0x55d3cd053600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=b2b4e55039a53f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.441+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/423946490 <== osd.3 v2:172.21.15.139:6824/1040931291 2 ==== osd_op_reply(3 200.00000001 [delete] v122'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55d3cda06500 con 0x55d3cda89c00 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.441+0000 7fe100dd1640 1 -- 172.21.15.139:0/423946490 <== osd.1 v2:172.21.15.139:6814/315640110 2 ==== osd_op_reply(8 200.00000006 [delete] v122'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55d3cda06000 con 0x55d3cda89400 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.442+0000 7fe1005d0640 1 -- 172.21.15.139:0/423946490 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== osd_op_reply(7 200.00000005 [delete] v122'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55d3cda06780 con 0x55d3cd082400 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.442+0000 7fe1005d0640 1 -- 172.21.15.139:0/423946490 <== osd.0 v2:172.21.15.139:6808/1167038101 2 ==== osd_op_reply(5 200.00000003 [delete] v122'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55d3cda06780 con 0x55d3cd082400 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.442+0000 7fe100dd1640 1 -- 172.21.15.139:0/423946490 <== osd.4 v2:172.21.15.150:6800/2810531857 1 ==== osd_op_reply(4 200.00000002 [delete] v122'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55d3cda06000 con 0x55d3cd059c00 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.442+0000 7fe0f5dbb640 1 -- 172.21.15.139:0/423946490 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55d3cdb58400 con 0x55d3cda89c00 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.444+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/423946490 <== osd.3 v2:172.21.15.139:6824/1040931291 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v122'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55d3cda06500 con 0x55d3cda89c00 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.444+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/423946490 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(6 200.00000004 [delete] v122'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55d3cda06500 con 0x55d3cd082c00 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.444+0000 7fe101e24280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.444+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55d3cd082400 msgr2=0x55d3cd053600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.444+0000 7fe101e24280 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55d3cd082400 0x55d3cd053600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.444+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55d3cda89400 msgr2=0x55d3cd052580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.444+0000 7fe101e24280 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55d3cda89400 0x55d3cd052580 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.444+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55d3cda89c00 msgr2=0x55d3cd052b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.444+0000 7fe101e24280 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55d3cda89c00 0x55d3cd052b00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.444+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55d3cd059c00 msgr2=0x55d3cd053080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55d3cd059c00 0x55d3cd053080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/423946490 reap_dead start 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55d3cd082c00 msgr2=0x55d3cd053b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55d3cd082c00 0x55d3cd053b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe0ffdcf640 1 -- 172.21.15.139:0/423946490 reap_dead start 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda88c00 msgr2=0x55d3cccf9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda88c00 0x55d3cccf9180 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x55d3cdab0e40 tx=0x55d3cdbbe300 comp rx=0 tx=0).stop 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 shutdown_connections 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/423946490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda88c00 0x55d3cccf9180 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 >> 172.21.15.139:0/423946490 conn(0x55d3cdb59400 msgr2=0x55d3cd048000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 shutdown_connections 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/423946490 wait complete. 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/2520623342 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55d3cda88800 msgr2=0x55d3cccf9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55d3cda88800 0x55d3cccf9700 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x55d3ccd74a50 tx=0x55d3cdbbe240 comp rx=0 tx=0).stop 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/2520623342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 msgr2=0x55d3cccf6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 0x55d3cccf6b00 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x55d3cdb675c0 tx=0x55d3cdbbe000 comp rx=0 tx=0).stop 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/2520623342 shutdown_connections 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55d3cda88800 0x55d3cccf9700 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cdb59000 0x55d3cccf8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3cdb58c00 0x55d3cccf8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/2520623342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cdb58800 0x55d3cccf6b00 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/2520623342 >> 172.21.15.139:0/2520623342 conn(0x55d3cda4e400 msgr2=0x55d3ccd81800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/2520623342 shutdown_connections 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/2520623342 wait complete. 2024-04-25T10:03:13.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/2363211794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda4e800 msgr2=0x55d3cccf7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/2363211794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda4e800 0x55d3cccf7600 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x55d3cdab14a0 tx=0x55d3cda196e0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/2363211794 shutdown_connections 2024-04-25T10:03:13.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/2363211794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d3cda4e800 0x55d3cccf7600 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/2363211794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3ccc87c00 0x55d3cccf7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 --2- 172.21.15.139:0/2363211794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d3cda4e000 0x55d3cccf7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.445+0000 7fe101e24280 1 -- 172.21.15.139:0/2363211794 >> 172.21.15.139:0/2363211794 conn(0x55d3ccc87000 msgr2=0x55d3ccd81000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.446+0000 7fe101e24280 1 -- 172.21.15.139:0/2363211794 shutdown_connections 2024-04-25T10:03:13.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.446+0000 7fe101e24280 1 -- 172.21.15.139:0/2363211794 wait complete. 2024-04-25T10:03:13.457 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.071062, result: writing EResetJournal entry 2024-04-25T10:03:13.457 DEBUG:teuthology.orchestra.run.smithi139:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T10:03:13.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.531+0000 7f6cd401c740 1 -- start start 2024-04-25T10:03:13.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.531+0000 7f6cd401c740 1 Processor -- start 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd401c740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeb9a92c00 0x55eeb9b02580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd401c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeb9a93800 0x55eeb9b02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd401c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeb9a93c00 0x55eeb9b03080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd401c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeba86e600 con 0x55eeb9a93c00 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd401c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeba86e480 con 0x55eeb9a93800 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd401c740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eeba86e300 con 0x55eeb9a92c00 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd2532640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeb9a93c00 0x55eeb9b03080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd2532640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeb9a93c00 0x55eeb9b03080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52766/0 (socket says 172.21.15.139:52766) 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd2532640 1 -- 172.21.15.139:0/2100464757 learned_addr learned my addr 172.21.15.139:0/2100464757 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd1d31640 1 --2- 172.21.15.139:0/2100464757 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeb9a92c00 0x55eeb9b02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd1530640 1 --2- 172.21.15.139:0/2100464757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeb9a93800 0x55eeb9b02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd2532640 1 -- 172.21.15.139:0/2100464757 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeb9a92c00 msgr2=0x55eeb9b02580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd2532640 1 --2- 172.21.15.139:0/2100464757 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeb9a92c00 0x55eeb9b02580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd2532640 1 -- 172.21.15.139:0/2100464757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeb9a93800 msgr2=0x55eeb9b02b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd2532640 1 --2- 172.21.15.139:0/2100464757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeb9a93800 0x55eeb9b02b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.532+0000 7f6cd2532640 1 -- 172.21.15.139:0/2100464757 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeb9abf680 con 0x55eeb9a93c00 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6cd1530640 1 --2- 172.21.15.139:0/2100464757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeb9a93800 0x55eeb9b02b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6cd2532640 1 --2- 172.21.15.139:0/2100464757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeb9a93c00 0x55eeb9b03080 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x55eeba8a5ce0 tx=0x55eeb9b6e240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c288964022f59a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6ccf52c640 1 -- 172.21.15.139:0/2100464757 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eeb9ab8680 con 0x55eeb9a93c00 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6ccf52c640 1 -- 172.21.15.139:0/2100464757 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eeb9adc540 con 0x55eeb9a93c00 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6ccf52c640 1 -- 172.21.15.139:0/2100464757 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eeb9ab8b60 con 0x55eeb9a93c00 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6cd401c740 1 -- 172.21.15.139:0/2100464757 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eeb9abf4a0 con 0x55eeb9a93c00 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6cd401c740 1 -- 172.21.15.139:0/2100464757 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55eeb9abfa40 con 0x55eeb9a93c00 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6cd401c740 4 waiting for MDS map... 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6ccf52c640 1 -- 172.21.15.139:0/2100464757 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x55eeba962600 con 0x55eeb9a93c00 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6ccf52c640 1 -- 172.21.15.139:0/2100464757 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 90) v1 ==== 2723+0+0 (secure 0 0 0) 0x55eeb9aeb180 con 0x55eeb9a93c00 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6cd401c740 4 Got MDS map 90 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6cd401c740 4 datascan.main: Using metadata pool output 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.533+0000 7f6cd401c740 4 datascan.main: connecting to RADOS... 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.534+0000 7f6cd401c740 1 Processor -- start 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.534+0000 7f6cd401c740 1 -- start start 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.534+0000 7f6cd401c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f000 0x55eeb9b03600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.534+0000 7f6cd401c740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeba95f400 0x55eeb9b03b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.534+0000 7f6cd401c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeba95f800 0x55eeb9b04100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.534+0000 7f6cd401c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeba86e180 con 0x55eeba95f000 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.534+0000 7f6cd401c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb9af2480 con 0x55eeba95f800 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.534+0000 7f6cd401c740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eeb9af2900 con 0x55eeba95f400 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.534+0000 7f6cd1530640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f000 0x55eeb9b03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.534+0000 7f6cd1530640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f000 0x55eeb9b03600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52768/0 (socket says 172.21.15.139:52768) 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.534+0000 7f6cd1530640 1 -- 172.21.15.139:0/393118183 learned_addr learned my addr 172.21.15.139:0/393118183 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.534+0000 7f6cd1d31640 1 --2- 172.21.15.139:0/393118183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeba95f400 0x55eeb9b03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd1530640 1 --2- 172.21.15.139:0/393118183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeba95f800 0x55eeb9b04100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd1530640 1 -- 172.21.15.139:0/393118183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeba95f400 msgr2=0x55eeb9b03b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd1530640 1 --2- 172.21.15.139:0/393118183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeba95f400 0x55eeb9b03b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd1530640 1 -- 172.21.15.139:0/393118183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeba95f800 msgr2=0x55eeb9b04100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd1530640 1 --2- 172.21.15.139:0/393118183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeba95f800 0x55eeb9b04100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd1530640 1 -- 172.21.15.139:0/393118183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeb9abf860 con 0x55eeba95f000 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd1530640 1 --2- 172.21.15.139:0/393118183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f000 0x55eeb9b03600 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x55eeba977170 tx=0x55eeba982c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90b7f4cdf7b04da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6ccd528640 1 -- 172.21.15.139:0/393118183 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eeb9ab8680 con 0x55eeba95f000 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6ccd528640 1 -- 172.21.15.139:0/393118183 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eeb9adcc40 con 0x55eeba95f000 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6ccd528640 1 -- 172.21.15.139:0/393118183 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eeb9ab8b60 con 0x55eeba95f000 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd401c740 1 -- 172.21.15.139:0/393118183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f000 msgr2=0x55eeb9b03600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd401c740 1 --2- 172.21.15.139:0/393118183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f000 0x55eeb9b03600 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x55eeba977170 tx=0x55eeba982c60 comp rx=0 tx=0).stop 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd401c740 1 -- 172.21.15.139:0/393118183 shutdown_connections 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd401c740 1 --2- 172.21.15.139:0/393118183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeba95f800 0x55eeb9b04100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd401c740 1 --2- 172.21.15.139:0/393118183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeba95f400 0x55eeb9b03b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd401c740 1 --2- 172.21.15.139:0/393118183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f000 0x55eeb9b03600 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd401c740 1 -- 172.21.15.139:0/393118183 >> 172.21.15.139:0/393118183 conn(0x55eeba95ec00 msgr2=0x55eeb9b78800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd401c740 1 -- 172.21.15.139:0/393118183 shutdown_connections 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.535+0000 7f6cd401c740 1 -- 172.21.15.139:0/393118183 wait complete. 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.536+0000 7f6cd401c740 1 Processor -- start 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.536+0000 7f6cd401c740 1 -- start start 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.536+0000 7f6cd401c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeba95f000 0x55eeb9b05180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.536+0000 7f6cd401c740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f400 0x55eeb9b04c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.536+0000 7f6cd401c740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeba95f800 0x55eeb9b04680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.536+0000 7f6cd401c740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeba86f380 con 0x55eeba95f400 2024-04-25T10:03:13.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.536+0000 7f6cd401c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeba86e180 con 0x55eeba95f000 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.536+0000 7f6cd401c740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eeba86f500 con 0x55eeba95f800 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd1d31640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f400 0x55eeb9b04c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd1d31640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f400 0x55eeb9b04c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52772/0 (socket says 172.21.15.139:52772) 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 learned_addr learned my addr 172.21.15.139:0/3373344014 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd1530640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeba95f000 0x55eeb9b05180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd1530640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeba95f800 0x55eeb9b04680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeba95f800 msgr2=0x55eeb9b04680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd1d31640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeba95f800 0x55eeb9b04680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeba95f000 msgr2=0x55eeb9b05180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd1d31640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeba95f000 0x55eeb9b05180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeb9abf680 con 0x55eeba95f400 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd1d31640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f400 0x55eeb9b04c00 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x55eeba8a51a0 tx=0x55eeba8257a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6ccbd25640 1 -- 172.21.15.139:0/3373344014 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eeb9ab89c0 con 0x55eeba95f400 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6ccbd25640 1 -- 172.21.15.139:0/3373344014 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eeb9adca80 con 0x55eeba95f400 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eeb9abf4a0 con 0x55eeba95f400 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6ccbd25640 1 -- 172.21.15.139:0/3373344014 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eeb9ab8680 con 0x55eeba95f400 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eeb9abfa40 con 0x55eeba95f400 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.537+0000 7f6cd401c740 4 datascan.init: resolving metadata pool 17 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.538+0000 7f6ccbd25640 1 -- 172.21.15.139:0/3373344014 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55eeba9c8c00 con 0x55eeba95f400 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.539+0000 7f6ccbd25640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55eeba882800 0x55eeb9b05700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.539+0000 7f6ccbd25640 1 -- 172.21.15.139:0/3373344014 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x55eeba9c6c00 con 0x55eeba95f400 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.539+0000 7f6cd401c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.539+0000 7f6cd401c740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.539+0000 7f6cd401c740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.539+0000 7f6cd1530640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55eeba882800 0x55eeb9b05700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.539+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55eeba882c00 0x55eeb9b04100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.539+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba883000 con 0x55eeba882c00 2024-04-25T10:03:13.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.539+0000 7f6cd1530640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55eeba882c00 0x55eeb9b04100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.539+0000 7f6cd1530640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55eeba882800 0x55eeb9b05700 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x55eeba9772c0 tx=0x55eeb9b6e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.539+0000 7f6cd1530640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55eeba882c00 0x55eeb9b04100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.540+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== osd_op_reply(1 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeba882c00 2024-04-25T10:03:13.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.540+0000 7f6cccd27640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55eeb9e08000 0x55eeb9e0e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.540+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e08400 con 0x55eeb9e08000 2024-04-25T10:03:13.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.540+0000 7f6cd2532640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55eeb9e08000 0x55eeb9e0e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.540+0000 7f6cd2532640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55eeb9e08000 0x55eeb9e0e000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.542+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.5 v2:172.21.15.150:6824/2184499879 1 ==== osd_op_reply(2 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeb9e08000 2024-04-25T10:03:13.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.542+0000 7f6ccd528640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55eeb9e08c00 0x55eeb9e0e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.542+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e09000 con 0x55eeb9e08c00 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.543+0000 7f6cd1d31640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55eeb9e08c00 0x55eeb9e0e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.543+0000 7f6cd1d31640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55eeb9e08c00 0x55eeb9e0e580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.543+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(3 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeb9e08c00 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.543+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e08800 con 0x55eeba882c00 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.544+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.0 v2:172.21.15.139:6808/1167038101 2 ==== osd_op_reply(4 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeba882c00 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.544+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e09400 con 0x55eeba882c00 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.544+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.0 v2:172.21.15.139:6808/1167038101 3 ==== osd_op_reply(5 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeba882c00 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.544+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e08800 con 0x55eeb9e08000 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.544+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.5 v2:172.21.15.150:6824/2184499879 2 ==== osd_op_reply(6 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeb9e08000 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.544+0000 7f6ccd528640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55eeb9e09400 0x55eeb9e0eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.544+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e09800 con 0x55eeb9e09400 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.544+0000 7f6cd1530640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55eeb9e09400 0x55eeb9e0eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.545+0000 7f6cd1530640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55eeb9e09400 0x55eeb9e0eb00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.545+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(7 [pgnls start_epoch 122 out=81b] v120'17 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09400 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.545+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:8 18.18 18:1ad8d55b:::10000000006.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba883400 con 0x55eeb9e09400 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.545+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.1 v2:172.21.15.139:6814/315640110 2 ==== osd_op_reply(8 10000000006.00000001 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09400 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.545+0000 7f6cd401c740 10 datascan.operator(): handling object 1099511627782.1 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.545+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:9 18.25 18:a5feb6f3:::10000000006.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba883800 con 0x55eeb9e08c00 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.547+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.6 v2:172.21.15.150:6809/3369628189 2 ==== osd_op_reply(9 10000000006.00000000 [call] v122'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeb9e08c00 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.547+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55eeba883c00 0x55eeb9b03b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.547+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2c000 con 0x55eeba883c00 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.547+0000 7f6cd2532640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55eeba883c00 0x55eeb9b03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.547+0000 7f6cd2532640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55eeba883c00 0x55eeb9b03b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.547+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 1 ==== osd_op_reply(10 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.547+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e09c00 con 0x55eeba882c00 2024-04-25T10:03:13.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.547+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.0 v2:172.21.15.139:6808/1167038101 4 ==== osd_op_reply(11 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeba882c00 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.547+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba882400 con 0x55eeba882c00 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.548+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.0 v2:172.21.15.139:6808/1167038101 5 ==== osd_op_reply(12 [pgnls start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeba882c00 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.548+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e08400 con 0x55eeb9e08c00 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.548+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.6 v2:172.21.15.150:6809/3369628189 3 ==== osd_op_reply(13 [pgnls start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeb9e08c00 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.548+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba883000 con 0x55eeba882c00 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.548+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.0 v2:172.21.15.139:6808/1167038101 6 ==== osd_op_reply(14 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeba882c00 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.548+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e08800 con 0x55eeb9e08000 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.549+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.5 v2:172.21.15.150:6824/2184499879 3 ==== osd_op_reply(15 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeb9e08000 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.549+0000 7f6cccd27640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x55eeba883400 0x55eeb9e0f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.549+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba882400 con 0x55eeba883400 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.549+0000 7f6cd1d31640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x55eeba883400 0x55eeb9e0f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.549+0000 7f6cd1d31640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x55eeba883400 0x55eeb9e0f600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.549+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.2 v2:172.21.15.139:6800/1982336775 1 ==== osd_op_reply(16 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeba883400 2024-04-25T10:03:13.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.549+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba95e800 con 0x55eeb9e09400 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.549+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.1 v2:172.21.15.139:6814/315640110 3 ==== osd_op_reply(17 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09400 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.549+0000 7f6cccd27640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55eeb9e09800 0x55eeb9e0fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.549+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e09c00 con 0x55eeb9e09800 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.550+0000 7f6cd1530640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55eeb9e09800 0x55eeb9e0fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.550+0000 7f6cd1530640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55eeb9e09800 0x55eeb9e0fb80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.550+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== osd_op_reply(18 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09800 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.550+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba95e400 con 0x55eeb9e09800 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.551+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.7 v2:172.21.15.150:6808/2122423536 2 ==== osd_op_reply(19 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09800 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.551+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba883800 con 0x55eeba883c00 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.551+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 2 ==== osd_op_reply(20 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.551+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2c000 con 0x55eeba883400 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.551+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.2 v2:172.21.15.139:6800/1982336775 2 ==== osd_op_reply(21 [pgnls start_epoch 122 out=81b] v121'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeba883400 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.551+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:22 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2c400 con 0x55eeba883400 2024-04-25T10:03:13.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.551+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.2 v2:172.21.15.139:6800/1982336775 3 ==== osd_op_reply(22 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeba883400 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.551+0000 7f6cd401c740 10 datascan.operator(): handling object 1099511627778.0 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.551+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:23 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2c800 con 0x55eeba883400 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.553+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.2 v2:172.21.15.139:6800/1982336775 4 ==== osd_op_reply(23 10000000002.00000000 [call] v122'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeba883400 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.553+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:24 18.32 18:4c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2cc00 con 0x55eeba883c00 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.553+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 3 ==== osd_op_reply(24 [pgnls start_epoch 122 out=81b] v120'16 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.553+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:25 18.32 18:4def72d4:::10000000003.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2d000 con 0x55eeba883c00 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.553+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 4 ==== osd_op_reply(25 10000000003.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.553+0000 7f6cd401c740 10 datascan.operator(): handling object 1099511627779.0 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.553+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:26 18.32 18:4def72d4:::10000000003.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2d400 con 0x55eeba883c00 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.554+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 5 ==== osd_op_reply(26 10000000003.00000000 [call] v122'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.554+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:27 18.a 18:50000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e40000 con 0x55eeba883c00 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.554+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 6 ==== osd_op_reply(27 [pgnls start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.554+0000 7f6cccd27640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55eeba882400 0x55eeb9e10100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.554+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:28 18.2a 18:54000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e09000 con 0x55eeba882400 2024-04-25T10:03:13.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.555+0000 7f6cd2532640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55eeba882400 0x55eeb9e10100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.555+0000 7f6cd2532640 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55eeba882400 0x55eeb9e10100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.555+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.4 v2:172.21.15.150:6800/2810531857 1 ==== osd_op_reply(28 [pgnls start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba882400 2024-04-25T10:03:13.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.555+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:29 18.1a 18:58000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2d800 con 0x55eeba883c00 2024-04-25T10:03:13.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.555+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 7 ==== osd_op_reply(29 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.556+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:30 18.3a 18:5c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e08400 con 0x55eeba882c00 2024-04-25T10:03:13.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.556+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.0 v2:172.21.15.139:6808/1167038101 7 ==== osd_op_reply(30 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeba882c00 2024-04-25T10:03:13.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.556+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:31 18.6 18:60000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2dc00 con 0x55eeb9e09800 2024-04-25T10:03:13.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.556+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.7 v2:172.21.15.150:6808/2122423536 3 ==== osd_op_reply(31 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09800 2024-04-25T10:03:13.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.556+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:32 18.26 18:64000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba871400 con 0x55eeba883400 2024-04-25T10:03:13.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.556+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.2 v2:172.21.15.139:6800/1982336775 5 ==== osd_op_reply(32 [pgnls start_epoch 122 out=49b] v120'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeba883400 2024-04-25T10:03:13.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.556+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:33 18.16 18:68000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba883000 con 0x55eeba883c00 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.557+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 8 ==== osd_op_reply(33 [pgnls start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.557+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:34 18.36 18:6c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e40000 con 0x55eeb9e09400 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.557+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.1 v2:172.21.15.139:6814/315640110 4 ==== osd_op_reply(34 [pgnls start_epoch 122 out=49b] v120'32 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09400 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.557+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:35 18.e 18:70000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e40000 con 0x55eeba882c00 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.557+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.0 v2:172.21.15.139:6808/1167038101 8 ==== osd_op_reply(35 [pgnls start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeba882c00 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.557+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:36 18.2e 18:74000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2cc00 con 0x55eeba883c00 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.557+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 9 ==== osd_op_reply(36 [pgnls start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.557+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:37 18.1e 18:78000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2dc00 con 0x55eeb9e08000 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.558+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.5 v2:172.21.15.150:6824/2184499879 4 ==== osd_op_reply(37 [pgnls start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeb9e08000 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.558+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:38 18.3e 18:7c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2d000 con 0x55eeba883c00 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.558+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 10 ==== osd_op_reply(38 [pgnls start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.558+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:39 18.1 18:80000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e09c00 con 0x55eeba883c00 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.558+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 11 ==== osd_op_reply(39 [pgnls start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.558+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:40 18.21 18:84000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2d400 con 0x55eeb9e08000 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.558+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.5 v2:172.21.15.150:6824/2184499879 5 ==== osd_op_reply(40 [pgnls start_epoch 122 out=49b] v120'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeb9e08000 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.558+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:41 18.11 18:88000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e08400 con 0x55eeb9e09400 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.559+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.1 v2:172.21.15.139:6814/315640110 5 ==== osd_op_reply(41 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09400 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.559+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:42 18.31 18:8c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba883800 con 0x55eeba882400 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.559+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.4 v2:172.21.15.150:6800/2810531857 2 ==== osd_op_reply(42 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba882400 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.559+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba95e800 con 0x55eeba883c00 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.559+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 12 ==== osd_op_reply(43 [pgnls start_epoch 122 out=49b] v120'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.559+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e08800 con 0x55eeb9e09800 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.559+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.7 v2:172.21.15.150:6808/2122423536 4 ==== osd_op_reply(44 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09800 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.559+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba95e400 con 0x55eeba882400 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.560+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.4 v2:172.21.15.150:6800/2810531857 3 ==== osd_op_reply(45 [pgnls start_epoch 122 out=49b] v120'21 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba882400 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.560+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:46 18.39 18:9c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e09000 con 0x55eeba882c00 2024-04-25T10:03:13.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.560+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.0 v2:172.21.15.139:6808/1167038101 9 ==== osd_op_reply(46 [pgnls start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeba882c00 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.560+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:47 18.5 18:a0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e41c00 con 0x55eeb9e09400 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.560+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.1 v2:172.21.15.139:6814/315640110 6 ==== osd_op_reply(47 [pgnls start_epoch 122 out=81b] v121'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09400 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.560+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:48 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e40400 con 0x55eeb9e09400 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.561+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.1 v2:172.21.15.139:6814/315640110 7 ==== osd_op_reply(48 10000000004.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09400 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.561+0000 7f6cd401c740 10 datascan.operator(): handling object 1099511627780.0 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.561+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:49 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e40800 con 0x55eeb9e09400 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.562+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.1 v2:172.21.15.139:6814/315640110 8 ==== osd_op_reply(49 10000000004.00000000 [call] v122'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09400 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.562+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:50 18.25 18:a4000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e40c00 con 0x55eeb9e08c00 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.562+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.6 v2:172.21.15.150:6809/3369628189 4 ==== osd_op_reply(50 [pgnls start_epoch 122 out=81b] v122'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeb9e08c00 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.562+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:51 18.25 18:a5feb6f3:::10000000006.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e41000 con 0x55eeb9e08c00 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.562+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.6 v2:172.21.15.150:6809/3369628189 5 ==== osd_op_reply(51 10000000006.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeb9e08c00 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.562+0000 7f6cd401c740 10 datascan.operator(): handling object 1099511627782.0 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.562+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:52 18.25 18:a5feb6f3:::10000000006.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e41400 con 0x55eeb9e08c00 2024-04-25T10:03:13.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.564+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.6 v2:172.21.15.150:6809/3369628189 6 ==== osd_op_reply(52 10000000006.00000000 [call] v122'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeb9e08c00 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.564+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e41800 con 0x55eeb9e08000 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.564+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.5 v2:172.21.15.150:6824/2184499879 6 ==== osd_op_reply(53 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeb9e08000 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.564+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e09c00 con 0x55eeba882400 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.564+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.4 v2:172.21.15.150:6800/2810531857 4 ==== osd_op_reply(54 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba882400 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.564+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e40c00 con 0x55eeba882400 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.565+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.4 v2:172.21.15.150:6800/2810531857 5 ==== osd_op_reply(55 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba882400 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.565+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2d800 con 0x55eeb9e08c00 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.565+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.6 v2:172.21.15.150:6809/3369628189 7 ==== osd_op_reply(56 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeb9e08c00 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.565+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e08400 con 0x55eeba882c00 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.565+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.0 v2:172.21.15.139:6808/1167038101 10 ==== osd_op_reply(57 [pgnls start_epoch 122 out=49b] v120'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeba882c00 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.565+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2cc00 con 0x55eeba883c00 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.565+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.3 v2:172.21.15.139:6824/1040931291 13 ==== osd_op_reply(58 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba883c00 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.565+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e08800 con 0x55eeb9e09400 2024-04-25T10:03:13.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.566+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.1 v2:172.21.15.139:6814/315640110 9 ==== osd_op_reply(59 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09400 2024-04-25T10:03:13.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.566+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2dc00 con 0x55eeba883400 2024-04-25T10:03:13.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.566+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.2 v2:172.21.15.139:6800/1982336775 6 ==== osd_op_reply(60 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeba883400 2024-04-25T10:03:13.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.566+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e09000 con 0x55eeb9e09800 2024-04-25T10:03:13.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.566+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.7 v2:172.21.15.150:6808/2122423536 5 ==== osd_op_reply(61 [pgnls start_epoch 122 out=49b] v120'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09800 2024-04-25T10:03:13.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.567+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2d000 con 0x55eeb9e09400 2024-04-25T10:03:13.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.567+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.1 v2:172.21.15.139:6814/315640110 10 ==== osd_op_reply(62 [pgnls start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09400 2024-04-25T10:03:13.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.567+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba871400 con 0x55eeb9e08000 2024-04-25T10:03:13.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.567+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.5 v2:172.21.15.150:6824/2184499879 7 ==== osd_op_reply(63 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeb9e08000 2024-04-25T10:03:13.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.567+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeba883000 con 0x55eeb9e09800 2024-04-25T10:03:13.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.568+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.7 v2:172.21.15.150:6808/2122423536 6 ==== osd_op_reply(64 [pgnls start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09800 2024-04-25T10:03:13.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.568+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2c000 con 0x55eeb9e09400 2024-04-25T10:03:13.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.568+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.1 v2:172.21.15.139:6814/315640110 11 ==== osd_op_reply(65 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09400 2024-04-25T10:03:13.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.568+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e41800 con 0x55eeba882400 2024-04-25T10:03:13.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.568+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.4 v2:172.21.15.150:6800/2810531857 6 ==== osd_op_reply(66 [pgnls start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba882400 2024-04-25T10:03:13.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.568+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2c400 con 0x55eeba883400 2024-04-25T10:03:13.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.569+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 <== osd.2 v2:172.21.15.139:6800/1982336775 7 ==== osd_op_reply(67 [pgnls start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80780 con 0x55eeba883400 2024-04-25T10:03:13.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.569+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e40c00 con 0x55eeb9e08000 2024-04-25T10:03:13.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.569+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.5 v2:172.21.15.150:6824/2184499879 8 ==== osd_op_reply(68 [pgnls start_epoch 122 out=49b] v120'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeb9e08000 2024-04-25T10:03:13.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.569+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2c800 con 0x55eeb9e08000 2024-04-25T10:03:13.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.569+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.5 v2:172.21.15.150:6824/2184499879 9 ==== osd_op_reply(69 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeb9e08000 2024-04-25T10:03:13.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.569+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2d400 con 0x55eeb9e09800 2024-04-25T10:03:13.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.569+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.7 v2:172.21.15.150:6808/2122423536 7 ==== osd_op_reply(70 [pgnls start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09800 2024-04-25T10:03:13.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.569+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e41000 con 0x55eeb9e08000 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.570+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.5 v2:172.21.15.150:6824/2184499879 10 ==== osd_op_reply(71 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeb9e08000 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.570+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e2cc00 con 0x55eeb9e09400 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.570+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.1 v2:172.21.15.139:6814/315640110 12 ==== osd_op_reply(72 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09400 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.570+0000 7f6ccd528640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e40000 con 0x55eeb9e09800 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd1530640 1 -- 172.21.15.139:0/3373344014 <== osd.7 v2:172.21.15.150:6808/2122423536 8 ==== osd_op_reply(73 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80000 con 0x55eeb9e09800 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cccd27640 1 -- 172.21.15.139:0/3373344014 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55eeb9e09c00 con 0x55eeba882400 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd2532640 1 -- 172.21.15.139:0/3373344014 <== osd.4 v2:172.21.15.150:6800/2810531857 7 ==== osd_op_reply(74 [pgnls start_epoch 122 out=49b] v120'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eeb9b80500 con 0x55eeba882400 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55eeba882c00 msgr2=0x55eeb9b04100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x55eeba882c00 0x55eeb9b04100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55eeb9e09400 msgr2=0x55eeb9e0eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x55eeb9e09400 0x55eeb9e0eb00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x55eeba883400 msgr2=0x55eeb9e0f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x55eeba883400 0x55eeb9e0f600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55eeba883c00 msgr2=0x55eeb9b03b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x55eeba883c00 0x55eeb9b03b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 reap_dead start 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55eeba882400 msgr2=0x55eeb9e10100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x55eeba882400 0x55eeb9e10100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55eeb9e08000 msgr2=0x55eeb9e0e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x55eeb9e08000 0x55eeb9e0e000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd1d31640 1 -- 172.21.15.139:0/3373344014 reap_dead start 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55eeb9e08c00 msgr2=0x55eeb9e0e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55eeb9e08c00 0x55eeb9e0e580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55eeb9e09800 msgr2=0x55eeb9e0fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55eeb9e09800 0x55eeb9e0fb80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55eeba882800 msgr2=0x55eeb9b05700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55eeba882800 0x55eeb9b05700 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x55eeba9772c0 tx=0x55eeb9b6e000 comp rx=0 tx=0).stop 2024-04-25T10:03:13.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f400 msgr2=0x55eeb9b04c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f400 0x55eeb9b04c00 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x55eeba8a51a0 tx=0x55eeba8257a0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 shutdown_connections 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x55eeb9e09800 0x55eeb9e0fb80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x55eeb9e08c00 0x55eeb9e0e580 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeba95f400 0x55eeb9b04c00 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 --2- 172.21.15.139:0/3373344014 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x55eeba882800 0x55eeb9b05700 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 >> 172.21.15.139:0/3373344014 conn(0x55eeba95ec00 msgr2=0x55eeb9b78800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 shutdown_connections 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.571+0000 7f6cd401c740 1 -- 172.21.15.139:0/3373344014 wait complete. 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.572+0000 7f6cd401c740 1 -- 172.21.15.139:0/2100464757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeb9a93c00 msgr2=0x55eeb9b03080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.572+0000 7f6cd401c740 1 --2- 172.21.15.139:0/2100464757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeb9a93c00 0x55eeb9b03080 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x55eeba8a5ce0 tx=0x55eeb9b6e240 comp rx=0 tx=0).stop 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.572+0000 7f6cd401c740 1 -- 172.21.15.139:0/2100464757 shutdown_connections 2024-04-25T10:03:13.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.572+0000 7f6cd401c740 1 --2- 172.21.15.139:0/2100464757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55eeb9a93c00 0x55eeb9b03080 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.572+0000 7f6cd401c740 1 --2- 172.21.15.139:0/2100464757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eeb9a93800 0x55eeb9b02b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.572+0000 7f6cd401c740 1 --2- 172.21.15.139:0/2100464757 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eeb9a92c00 0x55eeb9b02580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.572+0000 7f6cd401c740 1 -- 172.21.15.139:0/2100464757 >> 172.21.15.139:0/2100464757 conn(0x55eeb9a93000 msgr2=0x55eeb9b79000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.572+0000 7f6cd401c740 1 -- 172.21.15.139:0/2100464757 shutdown_connections 2024-04-25T10:03:13.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.572+0000 7f6cd401c740 1 -- 172.21.15.139:0/2100464757 wait complete. 2024-04-25T10:03:13.587 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.129921, result: 2024-04-25T10:03:13.587 DEBUG:teuthology.orchestra.run.smithi139:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-25T10:03:13.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.659+0000 7f6ba3419740 1 -- start start 2024-04-25T10:03:13.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.659+0000 7f6ba3419740 1 Processor -- start 2024-04-25T10:03:13.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.659+0000 7f6ba3419740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612be0fec00 0x5612be16e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.659+0000 7f6ba3419740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612be0ff800 0x5612be16eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.659+0000 7f6ba3419740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be0ffc00 0x5612be16f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.659+0000 7f6ba3419740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612befae180 con 0x5612be0ffc00 2024-04-25T10:03:13.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.659+0000 7f6ba3419740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612befae300 con 0x5612be0ff800 2024-04-25T10:03:13.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.659+0000 7f6ba3419740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612befae480 con 0x5612be0fec00 2024-04-25T10:03:13.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba197d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be0ffc00 0x5612be16f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba197d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be0ffc00 0x5612be16f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52794/0 (socket says 172.21.15.139:52794) 2024-04-25T10:03:13.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba197d640 1 -- 172.21.15.139:0/3984103254 learned_addr learned my addr 172.21.15.139:0/3984103254 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba117c640 1 --2- 172.21.15.139:0/3984103254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612be0fec00 0x5612be16e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba197d640 1 -- 172.21.15.139:0/3984103254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612be0fec00 msgr2=0x5612be16e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba197d640 1 --2- 172.21.15.139:0/3984103254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612be0fec00 0x5612be16e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba197d640 1 -- 172.21.15.139:0/3984103254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612be0ff800 msgr2=0x5612be16eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba197d640 1 --2- 172.21.15.139:0/3984103254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612be0ff800 0x5612be16eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba197d640 1 -- 172.21.15.139:0/3984103254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612be12b680 con 0x5612be0ffc00 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba197d640 1 --2- 172.21.15.139:0/3984103254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be0ffc00 0x5612be16f080 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x5612beeff200 tx=0x5612bee913e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcec0971e54b8c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6b9e977640 1 -- 172.21.15.139:0/3984103254 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5612be1249c0 con 0x5612be0ffc00 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6b9e977640 1 -- 172.21.15.139:0/3984103254 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5612be148e00 con 0x5612be0ffc00 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6b9e977640 1 -- 172.21.15.139:0/3984103254 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5612be124680 con 0x5612be0ffc00 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba3419740 1 -- 172.21.15.139:0/3984103254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612be12b4a0 con 0x5612be0ffc00 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba3419740 1 -- 172.21.15.139:0/3984103254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5612be12ba40 con 0x5612be0ffc00 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba3419740 4 waiting for MDS map... 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6b9e977640 1 -- 172.21.15.139:0/3984103254 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x5612beedc000 con 0x5612be0ffc00 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6b9e977640 1 -- 172.21.15.139:0/3984103254 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 90) v1 ==== 2723+0+0 (secure 0 0 0) 0x5612be157180 con 0x5612be0ffc00 2024-04-25T10:03:13.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.660+0000 7f6ba3419740 4 Got MDS map 90 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.661+0000 7f6ba3419740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.661+0000 7f6ba3419740 4 datascan.main: Using metadata pool output 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.661+0000 7f6ba3419740 4 datascan.main: connecting to RADOS... 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.661+0000 7f6ba3419740 1 Processor -- start 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.661+0000 7f6ba3419740 1 -- start start 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.661+0000 7f6ba3419740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be271c00 0x5612be16f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba3419740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612bf01e000 0x5612be16fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba3419740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612bf01e400 0x5612be170100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba3419740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612befae600 con 0x5612be271c00 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba3419740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612befae780 con 0x5612bf01e000 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba3419740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612befae900 con 0x5612bf01e400 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba097b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be271c00 0x5612be16f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba097b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be271c00 0x5612be16f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52810/0 (socket says 172.21.15.139:52810) 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba097b640 1 -- 172.21.15.139:0/144622247 learned_addr learned my addr 172.21.15.139:0/144622247 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba117c640 1 --2- 172.21.15.139:0/144622247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612bf01e000 0x5612be16fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba097b640 1 --2- 172.21.15.139:0/144622247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612bf01e400 0x5612be170100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba097b640 1 -- 172.21.15.139:0/144622247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612bf01e400 msgr2=0x5612be170100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba097b640 1 --2- 172.21.15.139:0/144622247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612bf01e400 0x5612be170100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba097b640 1 -- 172.21.15.139:0/144622247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612bf01e000 msgr2=0x5612be16fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba097b640 1 --2- 172.21.15.139:0/144622247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612bf01e000 0x5612be16fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba097b640 1 -- 172.21.15.139:0/144622247 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612be12b860 con 0x5612be271c00 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba117c640 1 --2- 172.21.15.139:0/144622247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612bf01e000 0x5612be16fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba097b640 1 --2- 172.21.15.139:0/144622247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be271c00 0x5612be16f600 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x5612befd2810 tx=0x5612bf02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d459c4c2d1a613ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6b9c973640 1 -- 172.21.15.139:0/144622247 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5612be1249c0 con 0x5612be271c00 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6b9c973640 1 -- 172.21.15.139:0/144622247 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5612be148a80 con 0x5612be271c00 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6b9c973640 1 -- 172.21.15.139:0/144622247 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5612be124680 con 0x5612be271c00 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.662+0000 7f6ba3419740 1 -- 172.21.15.139:0/144622247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be271c00 msgr2=0x5612be16f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.663+0000 7f6ba3419740 1 --2- 172.21.15.139:0/144622247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be271c00 0x5612be16f600 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x5612befd2810 tx=0x5612bf02c000 comp rx=0 tx=0).stop 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.663+0000 7f6ba3419740 1 -- 172.21.15.139:0/144622247 shutdown_connections 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.663+0000 7f6ba3419740 1 --2- 172.21.15.139:0/144622247 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612bf01e400 0x5612be170100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.663+0000 7f6ba3419740 1 --2- 172.21.15.139:0/144622247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612bf01e000 0x5612be16fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.663+0000 7f6ba3419740 1 --2- 172.21.15.139:0/144622247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be271c00 0x5612be16f600 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.663+0000 7f6ba3419740 1 -- 172.21.15.139:0/144622247 >> 172.21.15.139:0/144622247 conn(0x5612be271800 msgr2=0x5612be1f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.663+0000 7f6ba3419740 1 -- 172.21.15.139:0/144622247 shutdown_connections 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.663+0000 7f6ba3419740 1 -- 172.21.15.139:0/144622247 wait complete. 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.663+0000 7f6ba3419740 1 Processor -- start 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.663+0000 7f6ba3419740 1 -- start start 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba3419740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612be271800 0x5612be171180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba3419740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612bf01f000 0x5612be170c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba3419740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612bf01ec00 0x5612be170680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba3419740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612befafc80 con 0x5612bf01f000 2024-04-25T10:03:13.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba3419740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612befafb00 con 0x5612be271800 2024-04-25T10:03:13.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba3419740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612befaf980 con 0x5612bf01ec00 2024-04-25T10:03:13.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba117c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612bf01f000 0x5612be170c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba117c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612bf01f000 0x5612be170c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52816/0 (socket says 172.21.15.139:52816) 2024-04-25T10:03:13.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 learned_addr learned my addr 172.21.15.139:0/1686632287 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba097b640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612bf01ec00 0x5612be170680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba097b640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612be271800 0x5612be171180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612bf01ec00 msgr2=0x5612be170680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba117c640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612bf01ec00 0x5612be170680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612be271800 msgr2=0x5612be171180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba117c640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612be271800 0x5612be171180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612be12b2c0 con 0x5612bf01f000 2024-04-25T10:03:13.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba117c640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612bf01f000 0x5612be170c00 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x5612befd3b00 tx=0x5612bf02cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6b9b170640 1 -- 172.21.15.139:0/1686632287 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5612be1249c0 con 0x5612bf01f000 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6b9b170640 1 -- 172.21.15.139:0/1686632287 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5612be148540 con 0x5612bf01f000 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5612be12b0e0 con 0x5612bf01f000 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6b9b170640 1 -- 172.21.15.139:0/1686632287 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5612be124680 con 0x5612bf01f000 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612be12a3c0 con 0x5612bf01f000 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.664+0000 7f6ba3419740 4 datascan.init: resolving metadata pool 17 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.666+0000 7f6b9b170640 1 -- 172.21.15.139:0/1686632287 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x5612bf030c00 con 0x5612bf01f000 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.666+0000 7f6b9b170640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x5612bf046c00 0x5612be171700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.666+0000 7f6b9b170640 1 -- 172.21.15.139:0/1686632287 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x5612bf02f000 con 0x5612bf01f000 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.666+0000 7f6ba3419740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.666+0000 7f6ba3419740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.666+0000 7f6ba3419740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.666+0000 7f6ba097b640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x5612bf046c00 0x5612be171700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.666+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x5612bf01e800 0x5612be170100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.666+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf01e400 con 0x5612bf01e800 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.666+0000 7f6ba097b640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x5612bf01e800 0x5612be170100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.666+0000 7f6ba097b640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x5612bf046c00 0x5612be171700 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x5612befd2960 tx=0x5612bf02c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.667+0000 7f6ba097b640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x5612bf01e800 0x5612be170100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.667+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.667+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x5612bf01e000 0x5612be16fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.667+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047000 con 0x5612bf01e000 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.667+0000 7f6ba197d640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x5612bf01e000 0x5612be16fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.668+0000 7f6ba197d640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x5612bf01e000 0x5612be16fb80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.668+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5612bee7e500 con 0x5612bf01e000 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.668+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x5612bf047400 0x5612be16f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.668+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047800 con 0x5612bf047400 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.668+0000 7f6ba117c640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x5612bf047400 0x5612be16f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.668+0000 7f6ba117c640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x5612bf047400 0x5612be16f600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.669+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.669+0000 7f6b9c172640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x5612be488000 0x5612be48e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.669+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488400 con 0x5612be488000 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.669+0000 7f6ba097b640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x5612be488000 0x5612be48e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.669+0000 7f6ba097b640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x5612be488000 0x5612be48e000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.669+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.670+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488c00 con 0x5612bf01e000 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.670+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.6 v2:172.21.15.150:6809/3369628189 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612bf01e000 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.670+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488800 con 0x5612bf047400 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.670+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.670+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489000 con 0x5612bf047400 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.670+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.670+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489800 con 0x5612be488000 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.671+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.671+0000 7f6b9c973640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5612be489400 0x5612be48e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.671+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488400 con 0x5612be489400 2024-04-25T10:03:13.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.671+0000 7f6ba197d640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5612be489400 0x5612be48e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.671+0000 7f6ba197d640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5612be489400 0x5612be48e580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.671+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 122 out=49b] v120'17 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489400 2024-04-25T10:03:13.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.671+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489c00 con 0x5612bf01e800 2024-04-25T10:03:13.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.671+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.672+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf01fc00 con 0x5612bf047400 2024-04-25T10:03:13.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.672+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.672+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488800 con 0x5612bf047400 2024-04-25T10:03:13.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.672+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.672+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf01e400 con 0x5612bf01e000 2024-04-25T10:03:13.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.672+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.6 v2:172.21.15.150:6809/3369628189 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612bf01e000 2024-04-25T10:03:13.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.672+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489000 con 0x5612bf047400 2024-04-25T10:03:13.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.673+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.673+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488c00 con 0x5612be488000 2024-04-25T10:03:13.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.673+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.673+0000 7f6b9c172640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x5612bf01fc00 0x5612be48eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.673+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047800 con 0x5612bf01fc00 2024-04-25T10:03:13.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.673+0000 7f6ba117c640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x5612bf01fc00 0x5612be48eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.673+0000 7f6ba117c640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x5612bf01fc00 0x5612be48eb00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.673+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.2 v2:172.21.15.139:6800/1982336775 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf01fc00 2024-04-25T10:03:13.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.673+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf01e400 con 0x5612be489400 2024-04-25T10:03:13.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.674+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.1 v2:172.21.15.139:6814/315640110 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489400 2024-04-25T10:03:13.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.674+0000 7f6b9c172640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x5612be488400 0x5612be48f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.674+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489800 con 0x5612be488400 2024-04-25T10:03:13.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.674+0000 7f6ba097b640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x5612be488400 0x5612be48f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.674+0000 7f6ba097b640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x5612be488400 0x5612be48f080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.675+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488400 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.675+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047000 con 0x5612be488400 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.675+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.7 v2:172.21.15.150:6808/2122423536 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488400 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.675+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a0000 con 0x5612bf01e800 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.675+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.675+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be271400 con 0x5612bf01fc00 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.675+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.2 v2:172.21.15.139:6800/1982336775 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 122 out=81b] v122'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf01fc00 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.675+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047c00 con 0x5612bf047400 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.676+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.676+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488800 con 0x5612be488000 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.676+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.676+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a0400 con 0x5612bf01e000 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.676+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.6 v2:172.21.15.150:6809/3369628189 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612bf01e000 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.676+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489000 con 0x5612bf047400 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.677+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.677+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a0800 con 0x5612bf047400 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.677+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.677+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489c00 con 0x5612be488000 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.677+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.677+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a0c00 con 0x5612be489400 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.677+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.1 v2:172.21.15.139:6814/315640110 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 122 out=49b] v120'17 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489400 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.677+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488c00 con 0x5612bf01e800 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.678+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.678+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a1000 con 0x5612bf047400 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.678+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.678+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be271400 con 0x5612bf047400 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.678+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.678+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a1400 con 0x5612bf01e000 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.678+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.6 v2:172.21.15.150:6809/3369628189 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612bf01e000 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.678+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a0000 con 0x5612bf047400 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.678+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.678+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a1c00 con 0x5612be488000 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.679+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.679+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a1800 con 0x5612bf01fc00 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.679+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.2 v2:172.21.15.139:6800/1982336775 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf01fc00 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.679+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a0800 con 0x5612be489400 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.679+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.1 v2:172.21.15.139:6814/315640110 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489400 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.679+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047800 con 0x5612be488400 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.680+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.7 v2:172.21.15.150:6808/2122423536 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488400 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.680+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a1000 con 0x5612be488400 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.680+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.7 v2:172.21.15.150:6808/2122423536 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488400 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.680+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047c00 con 0x5612bf01e800 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.680+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.680+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a0000 con 0x5612bf01fc00 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.680+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.2 v2:172.21.15.139:6800/1982336775 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 122 out=49b] v122'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf01fc00 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.680+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047000 con 0x5612bf01e800 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.681+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 122 out=81b] v122'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.681+0000 7f6ba3419740 20 datascan.forall_objects: OSD matched oid 10000000003.00000000 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.681+0000 7f6ba3419740 10 datascan.operator(): handling object 10000000003.0 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.681+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:42 18.32 18:4def72d4:::10000000003.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a1000 con 0x5612bf01e800 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.681+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 7 ==== osd_op_reply(42 10000000003.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.681+0000 7f6ba3419740 10 datascan.inject_with_backtrace: inode: 0x0x10000000003 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.681+0000 7f6ba3419740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.681+0000 7f6ba3419740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.681+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:43 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047800 con 0x5612bf047400 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.681+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 13 ==== osd_op_reply(43 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.681+0000 7f6ba3419740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.681+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:44 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488800 con 0x5612bf01e800 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 8 ==== osd_op_reply(44 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+585 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba3419740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba3419740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba3419740 20 datascan.get_frag_of: frag is * 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:45 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489c00 con 0x5612bf047400 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 14 ==== osd_op_reply(45 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba3419740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba3419740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488c00 con 0x5612bf047400 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 15 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba3419740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba3419740 20 datascan.read_dentry: bravo_head not found in result 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba3419740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000003 at 0x0x10000000001/bravo with size=5 bytes 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.682+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:47 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4ac000 con 0x5612bf047400 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.683+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 16 ==== osd_op_reply(47 10000000001.00000000 [omap-set-vals] v122'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.683+0000 7f6ba3419740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x10000000003 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.683+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:48 18.a 18:50000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4ac400 con 0x5612bf01e800 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.684+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.684+0000 7f6b9c973640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x5612be489800 0x5612be48f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.684+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:49 18.2a 18:54000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b2000 con 0x5612be489800 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.684+0000 7f6ba197d640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x5612be489800 0x5612be48f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.684+0000 7f6ba197d640 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x5612be489800 0x5612be48f600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.684+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.4 v2:172.21.15.150:6800/2810531857 1 ==== osd_op_reply(49 [pgnls-filter start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489800 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.684+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:50 18.1a 18:58000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4a1800 con 0x5612bf01e800 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.685+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 10 ==== osd_op_reply(50 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.685+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:51 18.3a 18:5c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b2400 con 0x5612bf047400 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.685+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 17 ==== osd_op_reply(51 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.685+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:52 18.6 18:60000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be271400 con 0x5612be488400 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.685+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.7 v2:172.21.15.150:6808/2122423536 5 ==== osd_op_reply(52 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488400 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.685+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:53 18.26 18:64000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b2800 con 0x5612bf01fc00 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.685+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.2 v2:172.21.15.139:6800/1982336775 5 ==== osd_op_reply(53 [pgnls-filter start_epoch 122 out=49b] v120'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf01fc00 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.685+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:54 18.16 18:68000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489000 con 0x5612bf01e800 2024-04-25T10:03:13.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.686+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 11 ==== osd_op_reply(54 [pgnls-filter start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.686+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:55 18.36 18:6c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b2c00 con 0x5612be489400 2024-04-25T10:03:13.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.686+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.1 v2:172.21.15.139:6814/315640110 5 ==== osd_op_reply(55 [pgnls-filter start_epoch 122 out=49b] v120'32 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489400 2024-04-25T10:03:13.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.686+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf01e400 con 0x5612bf047400 2024-04-25T10:03:13.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.686+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.686+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b3000 con 0x5612bf01e800 2024-04-25T10:03:13.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.686+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 12 ==== osd_op_reply(57 [pgnls-filter start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.686+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4ac000 con 0x5612be488000 2024-04-25T10:03:13.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.687+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.687+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b3400 con 0x5612bf01e800 2024-04-25T10:03:13.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.687+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 13 ==== osd_op_reply(59 [pgnls-filter start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.687+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4ac400 con 0x5612bf01e800 2024-04-25T10:03:13.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.687+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 14 ==== osd_op_reply(60 [pgnls-filter start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.687+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b3800 con 0x5612be488000 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.688+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 122 out=49b] v120'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.688+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489c00 con 0x5612be489400 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.688+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.1 v2:172.21.15.139:6814/315640110 6 ==== osd_op_reply(62 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489400 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.688+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b2400 con 0x5612be489800 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.688+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.4 v2:172.21.15.150:6800/2810531857 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489800 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.688+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488c00 con 0x5612bf01e800 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.688+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 15 ==== osd_op_reply(64 [pgnls-filter start_epoch 122 out=49b] v120'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.688+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b2800 con 0x5612be488400 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.689+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.7 v2:172.21.15.150:6808/2122423536 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488400 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.689+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488800 con 0x5612be489800 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.689+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.4 v2:172.21.15.150:6800/2810531857 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 122 out=49b] v120'21 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489800 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.689+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b2000 con 0x5612bf047400 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.689+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.689+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047800 con 0x5612be489400 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.690+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.1 v2:172.21.15.139:6814/315640110 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 122 out=49b] v122'12 uv12 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489400 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.690+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b3c00 con 0x5612bf01e000 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.690+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.6 v2:172.21.15.150:6809/3369628189 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 122 out=49b] v122'17 uv17 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612bf01e000 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.690+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047c00 con 0x5612be488000 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.690+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.690+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489000 con 0x5612be489800 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.691+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.4 v2:172.21.15.150:6800/2810531857 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489800 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.691+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047000 con 0x5612be489800 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.691+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.4 v2:172.21.15.150:6800/2810531857 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489800 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.691+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488c00 con 0x5612bf01e000 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.691+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.6 v2:172.21.15.150:6809/3369628189 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612bf01e000 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.691+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489c00 con 0x5612bf047400 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.692+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.0 v2:172.21.15.139:6808/1167038101 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 122 out=49b] v120'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf047400 2024-04-25T10:03:13.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.692+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b3000 con 0x5612bf01e800 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.692+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.3 v2:172.21.15.139:6824/1040931291 16 ==== osd_op_reply(75 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612bf01e800 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.692+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be488800 con 0x5612be489400 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.692+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.1 v2:172.21.15.139:6814/315640110 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489400 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.692+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b3400 con 0x5612bf01fc00 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.692+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.2 v2:172.21.15.139:6800/1982336775 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf01fc00 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.692+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be489000 con 0x5612be488400 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.693+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.7 v2:172.21.15.150:6808/2122423536 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 122 out=49b] v120'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488400 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.693+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b3800 con 0x5612be489400 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.693+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.1 v2:172.21.15.139:6814/315640110 9 ==== osd_op_reply(79 [pgnls-filter start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489400 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.693+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b2c00 con 0x5612be488000 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.694+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.694+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4ac000 con 0x5612be488400 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.694+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.7 v2:172.21.15.150:6808/2122423536 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488400 2024-04-25T10:03:13.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.694+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b2400 con 0x5612be489400 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.694+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.1 v2:172.21.15.139:6814/315640110 10 ==== osd_op_reply(82 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489400 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.694+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4ac400 con 0x5612be489800 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.695+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.4 v2:172.21.15.150:6800/2810531857 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489800 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.695+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4b3c00 con 0x5612bf01fc00 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.695+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 <== osd.2 v2:172.21.15.139:6800/1982336775 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e780 con 0x5612bf01fc00 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.695+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be271400 con 0x5612be488000 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.695+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 122 out=49b] v120'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.695+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047800 con 0x5612be488000 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.696+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.696+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4c4000 con 0x5612be488400 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.696+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.7 v2:172.21.15.150:6808/2122423536 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488400 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.696+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4c4000 con 0x5612be488000 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.696+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.5 v2:172.21.15.150:6824/2184499879 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488000 2024-04-25T10:03:13.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.696+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4c4400 con 0x5612be489400 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.696+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.1 v2:172.21.15.139:6814/315640110 11 ==== osd_op_reply(89 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489400 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.696+0000 7f6b9c172640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612bf047c00 con 0x5612be488400 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba097b640 1 -- 172.21.15.139:0/1686632287 <== osd.7 v2:172.21.15.150:6808/2122423536 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e000 con 0x5612be488400 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6b9c973640 1 -- 172.21.15.139:0/1686632287 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5612be4c4800 con 0x5612be489800 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba197d640 1 -- 172.21.15.139:0/1686632287 <== osd.4 v2:172.21.15.150:6800/2810531857 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 122 out=49b] v120'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5612bee7e500 con 0x5612be489800 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x5612bf047400 msgr2=0x5612be16f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x5612bf047400 0x5612be16f600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5612be489400 msgr2=0x5612be48e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x5612be489400 0x5612be48e580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x5612bf01fc00 msgr2=0x5612be48eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x5612bf01fc00 0x5612be48eb00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x5612bf01e800 msgr2=0x5612be170100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x5612bf01e800 0x5612be170100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x5612be489800 msgr2=0x5612be48f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x5612be489800 0x5612be48f600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x5612be488000 msgr2=0x5612be48e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 reap_dead start 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x5612be488000 0x5612be48e000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 reap_dead start 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x5612bf01e000 msgr2=0x5612be16fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x5612bf01e000 0x5612be16fb80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x5612be488400 msgr2=0x5612be48f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x5612be488400 0x5612be48f080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x5612bf046c00 msgr2=0x5612be171700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x5612bf046c00 0x5612be171700 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x5612befd2960 tx=0x5612bf02c000 comp rx=0 tx=0).stop 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612bf01f000 msgr2=0x5612be170c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 --2- 172.21.15.139:0/1686632287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612bf01f000 0x5612be170c00 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x5612befd3b00 tx=0x5612bf02cde0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba117c640 1 -- 172.21.15.139:0/1686632287 reap_dead start 2024-04-25T10:03:13.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 shutdown_connections 2024-04-25T10:03:13.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.697+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 >> 172.21.15.139:0/1686632287 conn(0x5612be271c00 msgr2=0x5612be1f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.698+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 shutdown_connections 2024-04-25T10:03:13.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.698+0000 7f6ba3419740 1 -- 172.21.15.139:0/1686632287 wait complete. 2024-04-25T10:03:13.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.698+0000 7f6ba3419740 1 -- 172.21.15.139:0/3984103254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be0ffc00 msgr2=0x5612be16f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.698+0000 7f6ba3419740 1 --2- 172.21.15.139:0/3984103254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be0ffc00 0x5612be16f080 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x5612beeff200 tx=0x5612bee913e0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.698+0000 7f6ba3419740 1 -- 172.21.15.139:0/3984103254 shutdown_connections 2024-04-25T10:03:13.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.698+0000 7f6ba3419740 1 --2- 172.21.15.139:0/3984103254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612be0ffc00 0x5612be16f080 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.698+0000 7f6ba3419740 1 --2- 172.21.15.139:0/3984103254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612be0ff800 0x5612be16eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.698+0000 7f6ba3419740 1 --2- 172.21.15.139:0/3984103254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5612be0fec00 0x5612be16e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.698+0000 7f6ba3419740 1 -- 172.21.15.139:0/3984103254 >> 172.21.15.139:0/3984103254 conn(0x5612be0ff000 msgr2=0x5612be1f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.698+0000 7f6ba3419740 1 -- 172.21.15.139:0/3984103254 shutdown_connections 2024-04-25T10:03:13.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.698+0000 7f6ba3419740 1 -- 172.21.15.139:0/3984103254 wait complete. 2024-04-25T10:03:13.731 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.143526, result: 2024-04-25T10:03:13.731 DEBUG:teuthology.orchestra.run.smithi139:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T10:03:13.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc0a33a9740 1 -- start start 2024-04-25T10:03:13.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc0a33a9740 1 Processor -- start 2024-04-25T10:03:13.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc0a33a9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8a172c00 0x558d8a1e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc0a33a9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8a173800 0x558d8a1e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc0a33a9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8a173c00 0x558d8a1e3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc0a33a9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8af4e600 con 0x558d8a173800 2024-04-25T10:03:13.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc0a33a9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8af4e480 con 0x558d8a172c00 2024-04-25T10:03:13.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc0a33a9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d8af4e300 con 0x558d8a173c00 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc09f84d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8a173800 0x558d8a1e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc09f84d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8a173800 0x558d8a1e2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52822/0 (socket says 172.21.15.139:52822) 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc09f84d640 1 -- 172.21.15.139:0/3303395465 learned_addr learned my addr 172.21.15.139:0/3303395465 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc0a004e640 1 --2- 172.21.15.139:0/3303395465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8a172c00 0x558d8a1e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.788+0000 7fc0a084f640 1 --2- 172.21.15.139:0/3303395465 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8a173c00 0x558d8a1e3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc09f84d640 1 -- 172.21.15.139:0/3303395465 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8a173c00 msgr2=0x558d8a1e3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3303395465 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8a173c00 0x558d8a1e3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc09f84d640 1 -- 172.21.15.139:0/3303395465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8a172c00 msgr2=0x558d8a1e2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3303395465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8a172c00 0x558d8a1e2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc09f84d640 1 -- 172.21.15.139:0/3303395465 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d8a19f680 con 0x558d8a173800 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3303395465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8a173800 0x558d8a1e2b00 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x558d8af89c50 tx=0x558d8a24e2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3760c414064fa5e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc09d849640 1 -- 172.21.15.139:0/3303395465 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558d8a198680 con 0x558d8a173800 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc09d849640 1 -- 172.21.15.139:0/3303395465 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558d8a1bcc40 con 0x558d8a173800 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc09d849640 1 -- 172.21.15.139:0/3303395465 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558d8a198b60 con 0x558d8a173800 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3303395465 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d8a19f860 con 0x558d8a173800 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3303395465 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x558d8a19f2c0 con 0x558d8a173800 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc0a33a9740 4 waiting for MDS map... 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc09d849640 1 -- 172.21.15.139:0/3303395465 <== mon.0 v2:172.21.15.139:3300/0 4 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x558d8b044600 con 0x558d8a173800 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc09d849640 1 -- 172.21.15.139:0/3303395465 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 90) v1 ==== 2723+0+0 (secure 0 0 0) 0x558d8a1cb180 con 0x558d8a173800 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.789+0000 7fc0a33a9740 4 Got MDS map 90 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.790+0000 7fc0a33a9740 4 datascan.main: Using metadata pool output 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.790+0000 7fc0a33a9740 4 datascan.main: connecting to RADOS... 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.790+0000 7fc0a33a9740 1 Processor -- start 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.790+0000 7fc0a33a9740 1 -- start start 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc0a33a9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8b02d000 0x558d8a1e3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc0a33a9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8b02d400 0x558d8a1e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc0a33a9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d800 0x558d8a1e4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc0a33a9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8af4e180 con 0x558d8b02d800 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc0a33a9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8a1d2480 con 0x558d8b02d400 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc0a33a9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d8a1d2900 con 0x558d8b02d000 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc09f84d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d800 0x558d8a1e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc09f84d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d800 0x558d8a1e4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52838/0 (socket says 172.21.15.139:52838) 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc09f84d640 1 -- 172.21.15.139:0/719099463 learned_addr learned my addr 172.21.15.139:0/719099463 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc0a084f640 1 --2- 172.21.15.139:0/719099463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8b02d000 0x558d8a1e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc0a004e640 1 --2- 172.21.15.139:0/719099463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8b02d400 0x558d8a1e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc09f84d640 1 -- 172.21.15.139:0/719099463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8b02d000 msgr2=0x558d8a1e3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc09f84d640 1 --2- 172.21.15.139:0/719099463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8b02d000 0x558d8a1e3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc09f84d640 1 -- 172.21.15.139:0/719099463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8b02d400 msgr2=0x558d8a1e3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc09f84d640 1 --2- 172.21.15.139:0/719099463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8b02d400 0x558d8a1e3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc09f84d640 1 -- 172.21.15.139:0/719099463 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d8a19f4a0 con 0x558d8b02d800 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc0a004e640 1 --2- 172.21.15.139:0/719099463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8b02d400 0x558d8a1e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc09f84d640 1 --2- 172.21.15.139:0/719099463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d800 0x558d8a1e4100 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x558d8b031020 tx=0x558d8aef6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b31347c8a153f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc09b845640 1 -- 172.21.15.139:0/719099463 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558d8a198820 con 0x558d8b02d800 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc09b845640 1 -- 172.21.15.139:0/719099463 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558d8a1bca80 con 0x558d8b02d800 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.791+0000 7fc09b845640 1 -- 172.21.15.139:0/719099463 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558d8a198680 con 0x558d8b02d800 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.792+0000 7fc0a33a9740 1 -- 172.21.15.139:0/719099463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d800 msgr2=0x558d8a1e4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.792+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/719099463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d800 0x558d8a1e4100 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x558d8b031020 tx=0x558d8aef6cc0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.792+0000 7fc0a33a9740 1 -- 172.21.15.139:0/719099463 shutdown_connections 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.792+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/719099463 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d800 0x558d8a1e4100 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.792+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/719099463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8b02d400 0x558d8a1e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.792+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/719099463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8b02d000 0x558d8a1e3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.792+0000 7fc0a33a9740 1 -- 172.21.15.139:0/719099463 >> 172.21.15.139:0/719099463 conn(0x558d8b02cc00 msgr2=0x558d8a258800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.792+0000 7fc0a33a9740 1 -- 172.21.15.139:0/719099463 shutdown_connections 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.792+0000 7fc0a33a9740 1 -- 172.21.15.139:0/719099463 wait complete. 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.792+0000 7fc0a33a9740 1 Processor -- start 2024-04-25T10:03:13.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.792+0000 7fc0a33a9740 1 -- start start 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a33a9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d000 0x558d8a1e5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a33a9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8b02d400 0x558d8a1e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a33a9740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8b02d800 0x558d8a1e4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a33a9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8af4f800 con 0x558d8b02d000 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a33a9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8af4f680 con 0x558d8b02d400 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a33a9740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d8af4e180 con 0x558d8b02d800 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a084f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d000 0x558d8a1e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a084f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d000 0x558d8a1e5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52844/0 (socket says 172.21.15.139:52844) 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 learned_addr learned my addr 172.21.15.139:0/3976959046 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a004e640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8b02d400 0x558d8a1e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8b02d800 msgr2=0x558d8a1e4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a084f640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8b02d800 0x558d8a1e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8b02d400 msgr2=0x558d8a1e4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a084f640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8b02d400 0x558d8a1e4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d8a19f680 con 0x558d8b02d000 2024-04-25T10:03:13.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc0a084f640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d000 0x558d8a1e5180 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x558d8b031e00 tx=0x558d8b056e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.793+0000 7fc09a042640 1 -- 172.21.15.139:0/3976959046 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558d8a198820 con 0x558d8b02d000 2024-04-25T10:03:13.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.794+0000 7fc09a042640 1 -- 172.21.15.139:0/3976959046 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558d8a1bc1c0 con 0x558d8b02d000 2024-04-25T10:03:13.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.794+0000 7fc09a042640 1 -- 172.21.15.139:0/3976959046 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558d8a198680 con 0x558d8b02d000 2024-04-25T10:03:13.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.794+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558d8a19f860 con 0x558d8b02d000 2024-04-25T10:03:13.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.794+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d8a19f2c0 con 0x558d8b02d000 2024-04-25T10:03:13.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.794+0000 7fc0a33a9740 4 datascan.init: resolving metadata pool 17 2024-04-25T10:03:13.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.795+0000 7fc09a042640 1 -- 172.21.15.139:0/3976959046 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x558d8b0a6c00 con 0x558d8b02d000 2024-04-25T10:03:13.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.795+0000 7fc09a042640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x558d8b0b8800 0x558d8a1e5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.795+0000 7fc09a042640 1 -- 172.21.15.139:0/3976959046 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x558d8b0b6800 con 0x558d8b02d000 2024-04-25T10:03:13.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.795+0000 7fc0a33a9740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.795+0000 7fc0a33a9740 4 datascan.main: resolving metadata pool 17 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.795+0000 7fc0a004e640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x558d8b0b8800 0x558d8a1e5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.795+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x558d8b0b8c00 0x558d8a1e4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.795+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9000 con 0x558d8b0b8c00 2024-04-25T10:03:13.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.795+0000 7fc0a004e640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x558d8b0b8800 0x558d8a1e5700 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x558d8af061e0 tx=0x558d8af057a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.795+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x558d8b0b8c00 0x558d8a1e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.796+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x558d8b0b8c00 0x558d8a1e4100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.796+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== osd_op_reply(1 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.796+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x558d8b0b9000 0x558d8a4e8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.796+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b02c800 con 0x558d8b0b9000 2024-04-25T10:03:13.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.796+0000 7fc0a084f640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x558d8b0b9000 0x558d8a4e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.797+0000 7fc0a084f640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x558d8b0b9000 0x558d8a4e8000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.797+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 1 ==== osd_op_reply(2 [pgnls start_epoch 122 out=177b] v121'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.797+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b02c800 con 0x558d8b0b8c00 2024-04-25T10:03:13.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.797+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 2 ==== osd_op_reply(3 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.797+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x558d8b02c800 0x558d8a4e8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.797+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b02c400 con 0x558d8b02c800 2024-04-25T10:03:13.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.797+0000 7fc0a004e640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x558d8b02c800 0x558d8a4e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.797+0000 7fc0a004e640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x558d8b02c800 0x558d8a4e8580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.798+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== osd_op_reply(4 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.798+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b02c400 con 0x558d8b02c800 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.798+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 2 ==== osd_op_reply(5 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.798+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b02c400 con 0x558d8b0b9000 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.798+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 2 ==== osd_op_reply(6 [pgnls start_epoch 122 out=149b] v122'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.798+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b02c400 con 0x558d8b0b8c00 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.799+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 3 ==== osd_op_reply(7 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.799+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x558d8b02c400 0x558d8a4e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.799+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6000 con 0x558d8b02c400 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.799+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x558d8b02c400 0x558d8a4e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.799+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x558d8b02c400 0x558d8a4e8b00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.799+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== osd_op_reply(8 [pgnls start_epoch 122 out=73b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x558d8a260000 con 0x558d8b02c400 2024-04-25T10:03:13.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.799+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x558d8a4f6000 0x558d8a4e9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.799+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6400 con 0x558d8a4f6000 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.800+0000 7fc0a084f640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x558d8a4f6000 0x558d8a4e9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.800+0000 7fc0a084f640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x558d8a4f6000 0x558d8a4e9080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.800+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.4 v2:172.21.15.150:6800/2810531857 1 ==== osd_op_reply(9 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260500 con 0x558d8a4f6000 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.800+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6400 con 0x558d8b0b9000 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.800+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 3 ==== osd_op_reply(10 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.800+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6400 con 0x558d8b0b8c00 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.801+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 4 ==== osd_op_reply(11 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.801+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6400 con 0x558d8b02c800 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.801+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 3 ==== osd_op_reply(12 [pgnls start_epoch 122 out=49b] v122'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.801+0000 7fc0a004e640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x558d8a4f6400 0x558d8a4ea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.801+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8af51000 con 0x558d8a4f6400 2024-04-25T10:03:13.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.801+0000 7fc0a004e640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x558d8a4f6400 0x558d8a4ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.801+0000 7fc0a004e640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x558d8a4f6400 0x558d8a4ea100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.802+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== osd_op_reply(13 [pgnls start_epoch 122 out=77b] v121'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x558d8a260780 con 0x558d8a4f6400 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.802+0000 7fc0a004e640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x558d8af51000 0x558d8a4ea680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.802+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8af51400 con 0x558d8af51000 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.802+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x558d8af51000 0x558d8a4ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.802+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x558d8af51000 0x558d8a4ea680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.803+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.5 v2:172.21.15.150:6824/2184499879 1 ==== osd_op_reply(14 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8af51000 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.803+0000 7fc09f84d640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x558d8af51400 0x558d8a4e9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.803+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8af51400 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.803+0000 7fc0a084f640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x558d8af51400 0x558d8a4e9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.803+0000 7fc0a084f640 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x558d8af51400 0x558d8a4e9600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.803+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.2 v2:172.21.15.139:6800/1982336775 1 ==== osd_op_reply(15 [pgnls start_epoch 122 out=73b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x558d8a260500 con 0x558d8af51400 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.803+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8b02c400 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.803+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.1 v2:172.21.15.139:6814/315640110 2 ==== osd_op_reply(16 [pgnls start_epoch 122 out=49b] v122'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8b02c400 2024-04-25T10:03:13.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.803+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8b02c800 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.804+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 4 ==== osd_op_reply(17 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.804+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8b0b9000 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.804+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 4 ==== osd_op_reply(18 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.804+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8b0b9000 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.804+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 5 ==== osd_op_reply(19 [pgnls start_epoch 122 out=81b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.804+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8af51000 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.804+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.5 v2:172.21.15.150:6824/2184499879 2 ==== osd_op_reply(20 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8af51000 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.804+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8a4f6400 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.805+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.6 v2:172.21.15.150:6809/3369628189 2 ==== osd_op_reply(21 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260780 con 0x558d8a4f6400 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.805+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8af51000 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.805+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.5 v2:172.21.15.150:6824/2184499879 3 ==== osd_op_reply(22 [pgnls start_epoch 122 out=97b] v121'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x558d8a260000 con 0x558d8af51000 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.805+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8b0b8c00 2024-04-25T10:03:13.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.805+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 5 ==== osd_op_reply(23 [pgnls start_epoch 122 out=74b] v119'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.805+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8b0b8c00 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.806+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 6 ==== osd_op_reply(24 [pgnls start_epoch 122 out=73b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.806+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8a4f6000 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.806+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.4 v2:172.21.15.150:6800/2810531857 2 ==== osd_op_reply(25 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260500 con 0x558d8a4f6000 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.806+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8b0b9000 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.806+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 6 ==== osd_op_reply(26 [pgnls start_epoch 122 out=98b] v121'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.806+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8b02c800 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.806+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 5 ==== osd_op_reply(27 [pgnls start_epoch 122 out=76b] v122'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.806+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8a4f6000 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.807+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.4 v2:172.21.15.150:6800/2810531857 3 ==== osd_op_reply(28 [pgnls start_epoch 122 out=49b] v122'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260500 con 0x558d8a4f6000 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.807+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8b02c800 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.807+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 6 ==== osd_op_reply(29 [pgnls start_epoch 122 out=105b] v122'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.807+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8a4f6400 2024-04-25T10:03:13.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.807+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.6 v2:172.21.15.150:6809/3369628189 3 ==== osd_op_reply(30 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260780 con 0x558d8a4f6400 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.807+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8b02c400 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.808+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.1 v2:172.21.15.139:6814/315640110 3 ==== osd_op_reply(31 [pgnls start_epoch 122 out=73b] v122'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x558d8a260000 con 0x558d8b02c400 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.808+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f6800 con 0x558d8a4f6400 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.808+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.6 v2:172.21.15.150:6809/3369628189 4 ==== osd_op_reply(32 [pgnls start_epoch 122 out=101b] v122'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x558d8a260780 con 0x558d8a4f6400 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.808+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.808+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b0b9000 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.808+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.808+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.808+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9800 con 0x558d8b0b9000 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.808+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.808+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.808+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9c00 con 0x558d8b0b9000 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1169 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f7000 con 0x558d8b0b9000 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f7400 con 0x558d8b0b9000 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f7800 con 0x558d8b0b9000 2024-04-25T10:03:13.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.809+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f7c00 con 0x558d8b02c400 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.810+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.1 v2:172.21.15.139:6814/315640110 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260000 con 0x558d8b02c400 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.810+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.810+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.810+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.810+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:40 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a4f7000 con 0x558d8af51400 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.810+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.2 v2:172.21.15.139:6800/1982336775 2 ==== osd_op_reply(40 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260500 con 0x558d8af51400 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.810+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 10000000005.00000000 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.810+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:41 17.9 17:9258b67d:::10000000005.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b0b9000 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.810+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 13 ==== osd_op_reply(41 10000000005.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+584 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.810+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T10:03:13.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.810+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9800 con 0x558d8af51000 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.811+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.5 v2:172.21.15.150:6824/2184499879 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260000 con 0x558d8af51000 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.811+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.811+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.811+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.811+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.811+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.811+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9c00 con 0x558d8b0b8c00 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.811+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.811+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.811+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.811+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.811+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510000 con 0x558d8b0b9000 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:03:13.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510400 con 0x558d8b02c800 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510800 con 0x558d8b02c800 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1667 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 20 datascan.scan_links: injected ino detected 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510c00 con 0x558d8a4f6400 2024-04-25T10:03:13.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.812+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.6 v2:172.21.15.150:6809/3369628189 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+585 (crc 0 0 0) 0x558d8a260780 con 0x558d8a4f6400 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.813+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a511000 con 0x558d8b0b8c00 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.813+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 8 ==== osd_op_reply(48 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.813+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a511000 con 0x558d8b0b9000 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.813+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 15 ==== osd_op_reply(49 [pgnls start_epoch 122 out=177b] v121'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.813+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a511000 con 0x558d8b0b8c00 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.813+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 9 ==== osd_op_reply(50 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.813+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a511000 con 0x558d8b02c800 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.814+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 9 ==== osd_op_reply(51 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.814+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b02c800 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.814+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 10 ==== osd_op_reply(52 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.814+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b0b9000 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.814+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 16 ==== osd_op_reply(53 [pgnls start_epoch 122 out=149b] v122'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.814+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b0b8c00 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.815+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 10 ==== osd_op_reply(54 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.815+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b02c400 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.815+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.1 v2:172.21.15.139:6814/315640110 5 ==== osd_op_reply(55 [pgnls start_epoch 122 out=73b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x558d8a260000 con 0x558d8b02c400 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.815+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8a4f6000 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.815+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.4 v2:172.21.15.150:6800/2810531857 4 ==== osd_op_reply(56 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260500 con 0x558d8a4f6000 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.815+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b0b9000 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.816+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 17 ==== osd_op_reply(57 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.816+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b0b8c00 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.816+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 11 ==== osd_op_reply(58 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.816+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b02c800 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.816+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 11 ==== osd_op_reply(59 [pgnls start_epoch 122 out=49b] v122'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.816+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8a4f6400 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.816+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.6 v2:172.21.15.150:6809/3369628189 6 ==== osd_op_reply(60 [pgnls start_epoch 122 out=77b] v121'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x558d8a260780 con 0x558d8a4f6400 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.816+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8af51000 2024-04-25T10:03:13.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.817+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.5 v2:172.21.15.150:6824/2184499879 5 ==== osd_op_reply(61 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8af51000 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.817+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8af51400 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.817+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.2 v2:172.21.15.139:6800/1982336775 3 ==== osd_op_reply(62 [pgnls start_epoch 122 out=73b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x558d8a260500 con 0x558d8af51400 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.817+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b02c400 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.818+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.1 v2:172.21.15.139:6814/315640110 6 ==== osd_op_reply(63 [pgnls start_epoch 122 out=49b] v122'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8b02c400 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.818+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b02c800 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.818+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 12 ==== osd_op_reply(64 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.818+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b0b9000 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.818+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 18 ==== osd_op_reply(65 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.818+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b0b9000 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.818+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 19 ==== osd_op_reply(66 [pgnls start_epoch 122 out=81b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.818+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8af51000 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.819+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.5 v2:172.21.15.150:6824/2184499879 6 ==== osd_op_reply(67 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260000 con 0x558d8af51000 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.819+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8a4f6400 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.819+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.6 v2:172.21.15.150:6809/3369628189 7 ==== osd_op_reply(68 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260780 con 0x558d8a4f6400 2024-04-25T10:03:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.819+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8af51000 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.819+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.5 v2:172.21.15.150:6824/2184499879 7 ==== osd_op_reply(69 [pgnls start_epoch 122 out=97b] v121'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x558d8a260000 con 0x558d8af51000 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.819+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b0b8c00 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 12 ==== osd_op_reply(70 [pgnls start_epoch 122 out=74b] v119'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b0b8c00 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 13 ==== osd_op_reply(71 [pgnls start_epoch 122 out=73b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8a4f6000 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.4 v2:172.21.15.150:6800/2810531857 5 ==== osd_op_reply(72 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260500 con 0x558d8a4f6000 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b0b9000 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 20 ==== osd_op_reply(73 [pgnls start_epoch 122 out=98b] v121'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b02c800 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 13 ==== osd_op_reply(74 [pgnls start_epoch 122 out=76b] v122'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8a4f6000 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.4 v2:172.21.15.150:6800/2810531857 6 ==== osd_op_reply(75 [pgnls start_epoch 122 out=49b] v122'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260500 con 0x558d8a4f6000 2024-04-25T10:03:13.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b02c800 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 14 ==== osd_op_reply(76 [pgnls start_epoch 122 out=105b] v122'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8a4f6400 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.6 v2:172.21.15.150:6809/3369628189 8 ==== osd_op_reply(77 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558d8a260780 con 0x558d8a4f6400 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.820+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b02c400 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.821+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.1 v2:172.21.15.139:6814/315640110 7 ==== osd_op_reply(78 [pgnls start_epoch 122 out=73b] v122'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x558d8a260000 con 0x558d8b02c400 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.821+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8a4f6400 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.821+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.6 v2:172.21.15.150:6809/3369628189 9 ==== osd_op_reply(79 [pgnls start_epoch 122 out=101b] v122'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x558d8a260780 con 0x558d8a4f6400 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.821+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.821+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a511400 con 0x558d8b0b9000 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.821+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.821+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.821+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a511800 con 0x558d8b0b9000 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.821+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.821+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.821+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a511c00 con 0x558d8b0b9000 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1169 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510400 con 0x558d8b0b9000 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510800 con 0x558d8b0b9000 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:03:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510c00 con 0x558d8b0b9000 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.822+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a511000 con 0x558d8b02c400 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.823+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.1 v2:172.21.15.139:6814/315640110 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260000 con 0x558d8b02c400 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.823+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.823+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.823+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.823+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- osd_op(unknown.0.0:87 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510000 con 0x558d8af51400 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.823+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.2 v2:172.21.15.139:6800/1982336775 4 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260500 con 0x558d8af51400 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.823+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 10000000005.00000000 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.823+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:88 17.9 17:9258b67d:::10000000005.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a511400 con 0x558d8b0b9000 2024-04-25T10:03:13.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.823+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 27 ==== osd_op_reply(88 10000000005.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+584 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.823+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.823+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a511800 con 0x558d8af51000 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.824+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.5 v2:172.21.15.150:6824/2184499879 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260000 con 0x558d8af51000 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.824+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.824+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.824+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.824+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.824+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.824+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a511c00 con 0x558d8b0b8c00 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.824+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.824+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.824+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:03:13.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.824+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.824+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510400 con 0x558d8b0b9000 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510800 con 0x558d8b02c800 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510c00 con 0x558d8b02c800 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1667 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 20 datascan.scan_links: injected ino detected 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 20 datascan.scan_links: adding 0x10000000003 for future processing to fix dnfirst 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:03:13.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.825+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510800 con 0x558d8a4f6400 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.826+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.6 v2:172.21.15.150:6809/3369628189 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+585 (crc 0 0 0) 0x558d8a260780 con 0x558d8a4f6400 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.826+0000 7fc0a33a9740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.826+0000 7fc0a33a9740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.826+0000 7fc0a33a9740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.826+0000 7fc0a33a9740 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.826+0000 7fc0a33a9740 10 datascan.scan_links: handling injected_ino 0x10000000003 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.826+0000 7fc0a33a9740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.826+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8a510c00 con 0x558d8b02c800 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.826+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.826+0000 7fc0a33a9740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.826+0000 7fc0a33a9740 20 datascan.scan_links: first is now 2 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.826+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9400 con 0x558d8b02c800 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.827+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 <== osd.0 v2:172.21.15.139:6808/1167038101 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v122'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x558d8a260780 con 0x558d8b02c800 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.827+0000 7fc0a33a9740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x10000000003 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.827+0000 7fc0a33a9740 10 datascan.scan_links: updating inotable 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.827+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9800 con 0x558d8b0b9000 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a084f640 1 -- 172.21.15.139:0/3976959046 <== osd.3 v2:172.21.15.139:6824/1040931291 29 ==== osd_op_reply(97 mds0_inotable [read 0~34 out=34b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x558d8a260500 con 0x558d8b0b9000 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 10 datascan.scan_links: updating snaptable 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- osd_op(unknown.0.0:98 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558d8b0b9c00 con 0x558d8b0b8c00 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc09f84d640 1 -- 172.21.15.139:0/3976959046 <== osd.7 v2:172.21.15.150:6808/2122423536 15 ==== osd_op_reply(98 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x558d8a260000 con 0x558d8b0b8c00 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x558d8b02c800 msgr2=0x558d8a4e8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x558d8b02c800 0x558d8a4e8580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x558d8b02c400 msgr2=0x558d8a4e8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x558d8b02c400 0x558d8a4e8b00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x558d8af51400 msgr2=0x558d8a4e9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x558d8af51400 0x558d8a4e9600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x558d8b0b9000 msgr2=0x558d8a4e8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x558d8b0b9000 0x558d8a4e8000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x558d8a4f6000 msgr2=0x558d8a4e9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x558d8a4f6000 0x558d8a4e9080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x558d8af51000 msgr2=0x558d8a4ea680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x558d8af51000 0x558d8a4ea680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x558d8a4f6400 msgr2=0x558d8a4ea100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x558d8a4f6400 0x558d8a4ea100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 reap_dead start 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x558d8b0b8c00 msgr2=0x558d8a1e4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x558d8b0b8c00 0x558d8a1e4100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a004e640 1 -- 172.21.15.139:0/3976959046 reap_dead start 2024-04-25T10:03:13.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x558d8b0b8800 msgr2=0x558d8a1e5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x558d8b0b8800 0x558d8a1e5700 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x558d8af061e0 tx=0x558d8af057a0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d000 msgr2=0x558d8a1e5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d000 0x558d8a1e5180 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x558d8b031e00 tx=0x558d8b056e40 comp rx=0 tx=0).stop 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 shutdown_connections 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x558d8b0b8c00 0x558d8a1e4100 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x558d8b0b8800 0x558d8a1e5700 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3976959046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8b02d000 0x558d8a1e5180 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 >> 172.21.15.139:0/3976959046 conn(0x558d8b02cc00 msgr2=0x558d8a258800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 shutdown_connections 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.828+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3976959046 wait complete. 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.829+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3303395465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8a173800 msgr2=0x558d8a1e2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.829+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3303395465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8a173800 0x558d8a1e2b00 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x558d8af89c50 tx=0x558d8a24e2a0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.829+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3303395465 shutdown_connections 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.829+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3303395465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558d8a173800 0x558d8a1e2b00 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.829+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3303395465 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558d8a173c00 0x558d8a1e3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.829+0000 7fc0a33a9740 1 --2- 172.21.15.139:0/3303395465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d8a172c00 0x558d8a1e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.829+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3303395465 >> 172.21.15.139:0/3303395465 conn(0x558d8a173000 msgr2=0x558d8a259000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.829+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3303395465 shutdown_connections 2024-04-25T10:03:13.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:13.829+0000 7fc0a33a9740 1 -- 172.21.15.139:0/3303395465 wait complete. 2024-04-25T10:03:13.861 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.129557, result: 2024-04-25T10:03:13.861 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:03:13.903 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:03:13.941 INFO:teuthology.orchestra.run.smithi139.stdout:[global] 2024-04-25T10:03:13.941 INFO:teuthology.orchestra.run.smithi139.stdout: chdir = "" 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: auth supported = cephx 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: filestore xattr use omap = true 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: mon clock drift allowed = 1.000 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: osd crush chooseleaf type = 0 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: auth debug = true 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on old message = true 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on bug = true 2024-04-25T10:03:13.942 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: mon pg warn max object skew = 0 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default size = 2 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow primary affinity = true 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow pg remap = true 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:03:13.943 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on no sortbitwise = false 2024-04-25T10:03:13.944 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:03:13.944 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on too few osds = false 2024-04-25T10:03:13.944 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:03:13.944 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:03:13.944 INFO:teuthology.orchestra.run.smithi139.stdout: mon_allow_pool_size_one = true 2024-04-25T10:03:13.944 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.944 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:03:13.944 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.944 INFO:teuthology.orchestra.run.smithi139.stdout: osd default data pool replay window = 5 2024-04-25T10:03:13.944 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.944 INFO:teuthology.orchestra.run.smithi139.stdout: mon allow pool delete = true 2024-04-25T10:03:13.944 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout: mon cluster log file level = debug 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout: debug asserts on shutdown = true 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout: mon health detail to clog = false 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout: mon host = "172.21.15.139,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on skipped message = False 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout: fsid = 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout:[osd] 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout: osd journal size = 100 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub load threshold = 5.0 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub max interval = 600 2024-04-25T10:03:13.945 INFO:teuthology.orchestra.run.smithi139.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: osd recover clone overlap = true 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: osd recovery max chunk = 1048576 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug shutdown = true 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug op order = true 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug verify stray on activate = true 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug trim objects = true 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: osd open classes on start = true 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug pg log writeout = true 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.946 INFO:teuthology.orchestra.run.smithi139.stdout: osd map max advance = 10 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: journal zero on create = true 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: filestore apply finisher threads = 3 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: bdev debug aio = true 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug misdirected ops = true 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore block size = 96636764160 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore fsck on mount = True 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluefs = 20 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluestore = 20 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: debug osd = 20 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: debug rocksdb = 10 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd full ratio = 0.9 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: osd objectstore = bluestore 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr] 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: debug mgr = 20 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: mgr/telemetry/nag = false 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout:[mon] 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: debug paxos = 20 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: mon data avail warn = 5 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: mon mgr mkfs grace = 240 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd reporter subtree level = osd 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd prime pg temp = true 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.949 INFO:teuthology.orchestra.run.smithi139.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: # 1m isn't quite enough 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: mon op complaint time = 120 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi139.stdout:[client] 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout: rgw cache enabled = true 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable ops log = true 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable usage log = true 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout: client mount timeout = 600 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout: debug client = 20 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = "" 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.a] 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.b] 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.c] 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout:[mds] 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds = 20 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds balancer = 20 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug frag = True 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug scatterstat = False 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout: mds op complaint time = 180 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout: mds verify scatter = True 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-04-25T10:03:13.952 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-04-25T10:03:13.953 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-04-25T10:03:13.953 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-04-25T10:03:13.953 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2024-04-25T10:03:13.953 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:03:13.953 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2024-04-25T10:03:13.953 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.953 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2024-04-25T10:03:13.953 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:03:13.954 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:03:13.955 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.139,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on skipped message = False 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2024-04-25T10:03:13.956 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug trim objects = true 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2024-04-25T10:03:13.957 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 20 2024-04-25T10:03:13.958 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 20 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 10 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-25T10:03:13.959 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2024-04-25T10:03:13.960 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: mon op complaint time = 120 2024-04-25T10:03:13.961 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout: client mount timeout = 600 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2024-04-25T10:03:13.962 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout:[mds] 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds = 20 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds balancer = 20 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug frag = True 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug scatterstat = False 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout: mds op complaint time = 180 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout: mds verify scatter = True 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2024-04-25T10:03:13.963 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2024-04-25T10:03:13.965 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:03:14.007 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:03:14.044 INFO:teuthology.orchestra.run.smithi139.stdout:[global] 2024-04-25T10:03:14.044 INFO:teuthology.orchestra.run.smithi139.stdout: chdir = "" 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: auth supported = cephx 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: filestore xattr use omap = true 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: mon clock drift allowed = 1.000 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: osd crush chooseleaf type = 0 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: auth debug = true 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on old message = true 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on bug = true 2024-04-25T10:03:14.045 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.046 INFO:teuthology.orchestra.run.smithi139.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:03:14.046 INFO:teuthology.orchestra.run.smithi139.stdout: mon pg warn max object skew = 0 2024-04-25T10:03:14.046 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.046 INFO:teuthology.orchestra.run.smithi139.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:03:14.046 INFO:teuthology.orchestra.run.smithi139.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:03:14.046 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.046 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default size = 2 2024-04-25T10:03:14.046 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.046 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow primary affinity = true 2024-04-25T10:03:14.046 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow pg remap = true 2024-04-25T10:03:14.046 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:03:14.046 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on no sortbitwise = false 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on too few osds = false 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: mon_allow_pool_size_one = true 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: osd default data pool replay window = 5 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: mon allow pool delete = true 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.047 INFO:teuthology.orchestra.run.smithi139.stdout: mon cluster log file level = debug 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: debug asserts on shutdown = true 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: mon health detail to clog = false 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: mon host = "172.21.15.139,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on skipped message = False 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: fsid = 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout:[osd] 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: osd journal size = 100 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub load threshold = 5.0 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub max interval = 600 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: osd recover clone overlap = true 2024-04-25T10:03:14.048 INFO:teuthology.orchestra.run.smithi139.stdout: osd recovery max chunk = 1048576 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug shutdown = true 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug op order = true 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug verify stray on activate = true 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug trim objects = true 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: osd open classes on start = true 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug pg log writeout = true 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: osd map max advance = 10 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.049 INFO:teuthology.orchestra.run.smithi139.stdout: journal zero on create = true 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: filestore apply finisher threads = 3 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: bdev debug aio = true 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug misdirected ops = true 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore block size = 96636764160 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore fsck on mount = True 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluefs = 20 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluestore = 20 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: debug osd = 20 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: debug rocksdb = 10 2024-04-25T10:03:14.050 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd full ratio = 0.9 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: osd objectstore = bluestore 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr] 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: debug mgr = 20 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:14.051 INFO:teuthology.orchestra.run.smithi139.stdout: mgr/telemetry/nag = false 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout:[mon] 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: debug paxos = 20 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: mon data avail warn = 5 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: mon mgr mkfs grace = 240 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd reporter subtree level = osd 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd prime pg temp = true 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:03:14.052 INFO:teuthology.orchestra.run.smithi139.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: # 1m isn't quite enough 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: mon op complaint time = 120 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout:[client] 2024-04-25T10:03:14.053 INFO:teuthology.orchestra.run.smithi139.stdout: rgw cache enabled = true 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable ops log = true 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable usage log = true 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout: client mount timeout = 600 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout: debug client = 20 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = "" 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.a] 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.b] 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.c] 2024-04-25T10:03:14.054 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout:[mds] 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds = 20 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds balancer = 20 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug frag = True 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug scatterstat = True 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout: mds op complaint time = 180 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout: mds verify scatter = True 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-04-25T10:03:14.055 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-04-25T10:03:14.056 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-04-25T10:03:14.056 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-04-25T10:03:14.056 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2024-04-25T10:03:14.056 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:03:14.056 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2024-04-25T10:03:14.056 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.057 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2024-04-25T10:03:14.058 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.139,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on skipped message = False 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 922739a8-edcf-4ded-8206-dbdb23e9fea4 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.059 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug trim objects = true 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2024-04-25T10:03:14.060 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2024-04-25T10:03:14.061 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 20 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 20 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 10 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2024-04-25T10:03:14.062 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:03:14.063 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: mon op complaint time = 120 2024-04-25T10:03:14.064 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout: client mount timeout = 600 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2024-04-25T10:03:14.065 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout:[mds] 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds = 20 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds balancer = 20 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug frag = True 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug scatterstat = True 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout: mds op complaint time = 180 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout: mds verify scatter = True 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2024-04-25T10:03:14.066 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2024-04-25T10:03:14.068 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:03:14.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.201+0000 7f3056bcf640 1 Processor -- start 2024-04-25T10:03:14.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.201+0000 7f3056bcf640 1 -- start start 2024-04-25T10:03:14.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.202+0000 7f3056bcf640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3050150ed0 0x7f30500b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.202+0000 7f3056bcf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f30500cdd20 0x7f3050071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.202+0000 7f3056bcf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30500b4740 0x7f3050071f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.202+0000 7f3056bcf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305015a480 con 0x7f30500cdd20 2024-04-25T10:03:14.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.202+0000 7f3056bcf640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305015a760 con 0x7f3050150ed0 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.202+0000 7f3056bcf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305015aa40 con 0x7f30500b4740 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.202+0000 7f304ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f30500cdd20 0x7f3050071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.202+0000 7f304ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f30500cdd20 0x7f3050071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52848/0 (socket says 172.21.15.139:52848) 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.202+0000 7f304ffff640 1 -- 172.21.15.139:0/81906486 learned_addr learned my addr 172.21.15.139:0/81906486 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.203+0000 7f305496b640 1 --2- 172.21.15.139:0/81906486 >> v2:172.21.15.150:3300/0 conn(0x7f3050150ed0 0x7f30500b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.203+0000 7f305516c640 1 --2- 172.21.15.139:0/81906486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30500b4740 0x7f3050071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.203+0000 7f304ffff640 1 -- 172.21.15.139:0/81906486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30500b4740 msgr2=0x7f3050071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.203+0000 7f304ffff640 1 --2- 172.21.15.139:0/81906486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30500b4740 0x7f3050071f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.203+0000 7f304ffff640 1 -- 172.21.15.139:0/81906486 >> v2:172.21.15.150:3300/0 conn(0x7f3050150ed0 msgr2=0x7f30500b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.203+0000 7f304ffff640 1 --2- 172.21.15.139:0/81906486 >> v2:172.21.15.150:3300/0 conn(0x7f3050150ed0 0x7f30500b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.203+0000 7f304ffff640 1 -- 172.21.15.139:0/81906486 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305015ad20 con 0x7f30500cdd20 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.203+0000 7f305496b640 1 --2- 172.21.15.139:0/81906486 >> v2:172.21.15.150:3300/0 conn(0x7f3050150ed0 0x7f30500b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:14.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.203+0000 7f304ffff640 1 --2- 172.21.15.139:0/81906486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30500cdd20 0x7f3050071a50 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f305015a970 tx=0x7f304402b270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2120fc7074b6eda7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.204+0000 7f304f7fe640 1 -- 172.21.15.139:0/81906486 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044038070 con 0x7f30500cdd20 2024-04-25T10:03:14.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.204+0000 7f304f7fe640 1 -- 172.21.15.139:0/81906486 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3044033440 con 0x7f30500cdd20 2024-04-25T10:03:14.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.204+0000 7f304f7fe640 1 -- 172.21.15.139:0/81906486 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044033810 con 0x7f30500cdd20 2024-04-25T10:03:14.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.204+0000 7f3056bcf640 1 -- 172.21.15.139:0/81906486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30500cdd20 msgr2=0x7f3050071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.204+0000 7f3056bcf640 1 --2- 172.21.15.139:0/81906486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30500cdd20 0x7f3050071a50 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f305015a970 tx=0x7f304402b270 comp rx=0 tx=0).stop 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.204+0000 7f3056bcf640 1 -- 172.21.15.139:0/81906486 shutdown_connections 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.204+0000 7f3056bcf640 1 --2- 172.21.15.139:0/81906486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30500b4740 0x7f3050071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.204+0000 7f3056bcf640 1 --2- 172.21.15.139:0/81906486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f30500cdd20 0x7f3050071a50 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.204+0000 7f3056bcf640 1 --2- 172.21.15.139:0/81906486 >> v2:172.21.15.150:3300/0 conn(0x7f3050150ed0 0x7f30500b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.204+0000 7f3056bcf640 1 -- 172.21.15.139:0/81906486 >> 172.21.15.139:0/81906486 conn(0x7f3050067000 msgr2=0x7f305006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.204+0000 7f3056bcf640 1 -- 172.21.15.139:0/81906486 shutdown_connections 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.204+0000 7f3056bcf640 1 -- 172.21.15.139:0/81906486 wait complete. 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.205+0000 7f3056bcf640 1 Processor -- start 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.205+0000 7f3056bcf640 1 -- start start 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.206+0000 7f3056bcf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30500b4740 0x7f3050144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.206+0000 7f3056bcf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30500cdd20 0x7f305014a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.206+0000 7f3056bcf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3050150ed0 0x7f30500affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.206+0000 7f3056bcf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30501512b0 con 0x7f3050150ed0 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.206+0000 7f3056bcf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30500b3960 con 0x7f30500b4740 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.206+0000 7f3056bcf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30500711c0 con 0x7f30500cdd20 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.206+0000 7f305516c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3050150ed0 0x7f30500affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.206+0000 7f305516c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3050150ed0 0x7f30500affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52854/0 (socket says 172.21.15.139:52854) 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.206+0000 7f305516c640 1 -- 172.21.15.139:0/4221665887 learned_addr learned my addr 172.21.15.139:0/4221665887 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f305496b640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30500b4740 0x7f3050144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f305516c640 1 -- 172.21.15.139:0/4221665887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30500cdd20 msgr2=0x7f305014a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f304ffff640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30500cdd20 0x7f305014a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f305516c640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30500cdd20 0x7f305014a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f305516c640 1 -- 172.21.15.139:0/4221665887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30500b4740 msgr2=0x7f3050144ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f305516c640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30500b4740 0x7f3050144ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f305516c640 1 -- 172.21.15.139:0/4221665887 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3050145b80 con 0x7f3050150ed0 2024-04-25T10:03:14.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f304ffff640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30500cdd20 0x7f305014a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:14.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f305496b640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30500b4740 0x7f3050144ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:14.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f305516c640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3050150ed0 0x7f30500affb0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f3048007e30 tx=0x7f304800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f304d7fa640 1 -- 172.21.15.139:0/4221665887 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048014420 con 0x7f3050150ed0 2024-04-25T10:03:14.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f304d7fa640 1 -- 172.21.15.139:0/4221665887 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3048019070 con 0x7f3050150ed0 2024-04-25T10:03:14.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f3056bcf640 1 -- 172.21.15.139:0/4221665887 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3050145e60 con 0x7f3050150ed0 2024-04-25T10:03:14.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f304d7fa640 1 -- 172.21.15.139:0/4221665887 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048014800 con 0x7f3050150ed0 2024-04-25T10:03:14.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.207+0000 7f3056bcf640 1 -- 172.21.15.139:0/4221665887 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3050146420 con 0x7f3050150ed0 2024-04-25T10:03:14.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.208+0000 7f3056bcf640 1 -- 172.21.15.139:0/4221665887 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3050074520 con 0x7f3050150ed0 2024-04-25T10:03:14.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.208+0000 7f304d7fa640 1 -- 172.21.15.139:0/4221665887 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30480149a0 con 0x7f3050150ed0 2024-04-25T10:03:14.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.209+0000 7f304d7fa640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f303c06b4d0 0x7f303c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.209+0000 7f304d7fa640 1 -- 172.21.15.139:0/4221665887 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f304808acc0 con 0x7f3050150ed0 2024-04-25T10:03:14.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.209+0000 7f305496b640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f303c06b4d0 0x7f303c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.209+0000 7f305496b640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f303c06b4d0 0x7f303c06d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3040006fd0 tx=0x7f3040008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.213+0000 7f304d7fa640 1 -- 172.21.15.139:0/4221665887 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3048059900 con 0x7f3050150ed0 2024-04-25T10:03:14.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:14.425+0000 7f3056bcf640 1 -- 172.21.15.139:0/4221665887 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f305006f150 con 0x7f3050150ed0 2024-04-25T10:03:15.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.374+0000 7f304d7fa640 1 -- 172.21.15.139:0/4221665887 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v92) v1 ==== 162+0+0 (secure 0 0 0) 0x7f3048059200 con 0x7f3050150ed0 2024-04-25T10:03:15.380 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:03:15.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.378+0000 7f3056bcf640 1 -- 172.21.15.139:0/4221665887 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f303c06b4d0 msgr2=0x7f303c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.378+0000 7f3056bcf640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f303c06b4d0 0x7f303c06d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3040006fd0 tx=0x7f3040008040 comp rx=0 tx=0).stop 2024-04-25T10:03:15.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.378+0000 7f3056bcf640 1 -- 172.21.15.139:0/4221665887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3050150ed0 msgr2=0x7f30500affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.378+0000 7f3056bcf640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3050150ed0 0x7f30500affb0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f3048007e30 tx=0x7f304800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.378+0000 7f3056bcf640 1 -- 172.21.15.139:0/4221665887 shutdown_connections 2024-04-25T10:03:15.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.378+0000 7f3056bcf640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3050150ed0 0x7f30500affb0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.378+0000 7f3056bcf640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f303c06b4d0 0x7f303c06d990 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.378+0000 7f3056bcf640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30500cdd20 0x7f305014a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.378+0000 7f3056bcf640 1 --2- 172.21.15.139:0/4221665887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30500b4740 0x7f3050144ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.378+0000 7f3056bcf640 1 -- 172.21.15.139:0/4221665887 >> 172.21.15.139:0/4221665887 conn(0x7f3050067000 msgr2=0x7f30500b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.378+0000 7f3056bcf640 1 -- 172.21.15.139:0/4221665887 shutdown_connections 2024-04-25T10:03:15.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.378+0000 7f3056bcf640 1 -- 172.21.15.139:0/4221665887 wait complete. 2024-04-25T10:03:15.399 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:15.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.496+0000 7ff5e9e8f640 1 Processor -- start 2024-04-25T10:03:15.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.496+0000 7ff5e9e8f640 1 -- start start 2024-04-25T10:03:15.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.497+0000 7ff5e9e8f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40b0210 0x7ff5e40b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.497+0000 7ff5e9e8f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff5e415ae40 0x7ff5e40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.497+0000 7ff5e9e8f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff5e40cefe0 0x7ff5e406d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.497+0000 7ff5e9e8f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e40b1070 con 0x7ff5e40cefe0 2024-04-25T10:03:15.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.497+0000 7ff5e9e8f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e40cdff0 con 0x7ff5e415ae40 2024-04-25T10:03:15.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.497+0000 7ff5e9e8f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e40b1ea0 con 0x7ff5e40b0210 2024-04-25T10:03:15.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.497+0000 7ff5e3fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff5e40cefe0 0x7ff5e406d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.497+0000 7ff5e3fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff5e40cefe0 0x7ff5e406d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52856/0 (socket says 172.21.15.139:52856) 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.497+0000 7ff5e3fff640 1 -- 172.21.15.139:0/907413030 learned_addr learned my addr 172.21.15.139:0/907413030 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.497+0000 7ff5daffd640 1 --2- 172.21.15.139:0/907413030 >> v2:172.21.15.150:3300/0 conn(0x7ff5e415ae40 0x7ff5e40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.497+0000 7ff5e37fe640 1 --2- 172.21.15.139:0/907413030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40b0210 0x7ff5e40b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.497+0000 7ff5e3fff640 1 -- 172.21.15.139:0/907413030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40b0210 msgr2=0x7ff5e40b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.498+0000 7ff5e3fff640 1 --2- 172.21.15.139:0/907413030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40b0210 0x7ff5e40b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.498+0000 7ff5e3fff640 1 -- 172.21.15.139:0/907413030 >> v2:172.21.15.150:3300/0 conn(0x7ff5e415ae40 msgr2=0x7ff5e40b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.498+0000 7ff5e3fff640 1 --2- 172.21.15.139:0/907413030 >> v2:172.21.15.150:3300/0 conn(0x7ff5e415ae40 0x7ff5e40b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.498+0000 7ff5e3fff640 1 -- 172.21.15.139:0/907413030 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e40b0b30 con 0x7ff5e40cefe0 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.498+0000 7ff5daffd640 1 --2- 172.21.15.139:0/907413030 >> v2:172.21.15.150:3300/0 conn(0x7ff5e415ae40 0x7ff5e40b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.498+0000 7ff5e3fff640 1 --2- 172.21.15.139:0/907413030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40cefe0 0x7ff5e406d200 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7ff5e406f360 tx=0x7ff5dc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d20c31e4f52ba80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.498+0000 7ff5e2ffd640 1 -- 172.21.15.139:0/907413030 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc036070 con 0x7ff5e40cefe0 2024-04-25T10:03:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.498+0000 7ff5e2ffd640 1 -- 172.21.15.139:0/907413030 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5dc031600 con 0x7ff5e40cefe0 2024-04-25T10:03:15.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.498+0000 7ff5e2ffd640 1 -- 172.21.15.139:0/907413030 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc0319d0 con 0x7ff5e40cefe0 2024-04-25T10:03:15.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.499+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/907413030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40cefe0 msgr2=0x7ff5e406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.499+0000 7ff5e9e8f640 1 --2- 172.21.15.139:0/907413030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40cefe0 0x7ff5e406d200 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7ff5e406f360 tx=0x7ff5dc02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.499+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/907413030 shutdown_connections 2024-04-25T10:03:15.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.499+0000 7ff5e9e8f640 1 --2- 172.21.15.139:0/907413030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40cefe0 0x7ff5e406d200 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.499+0000 7ff5e9e8f640 1 --2- 172.21.15.139:0/907413030 >> v2:172.21.15.150:3300/0 conn(0x7ff5e415ae40 0x7ff5e40b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.499+0000 7ff5e9e8f640 1 --2- 172.21.15.139:0/907413030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40b0210 0x7ff5e40b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.499+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/907413030 >> 172.21.15.139:0/907413030 conn(0x7ff5e4066f20 msgr2=0x7ff5e406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.499+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/907413030 shutdown_connections 2024-04-25T10:03:15.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.499+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/907413030 wait complete. 2024-04-25T10:03:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.500+0000 7ff5e9e8f640 1 Processor -- start 2024-04-25T10:03:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.501+0000 7ff5e9e8f640 1 -- start start 2024-04-25T10:03:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.501+0000 7ff5e9e8f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40b0210 0x7ff5e4156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.501+0000 7ff5e9e8f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40cefe0 0x7ff5e4165580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.501+0000 7ff5e9e8f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e415ae40 0x7ff5e415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.501+0000 7ff5e9e8f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e40ceb50 con 0x7ff5e40cefe0 2024-04-25T10:03:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.501+0000 7ff5e9e8f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e415b3e0 con 0x7ff5e415ae40 2024-04-25T10:03:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.501+0000 7ff5e9e8f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e40ce500 con 0x7ff5e40b0210 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.501+0000 7ff5daffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40cefe0 0x7ff5e4165580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5daffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40cefe0 0x7ff5e4165580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52860/0 (socket says 172.21.15.139:52860) 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5daffd640 1 -- 172.21.15.139:0/1305421167 learned_addr learned my addr 172.21.15.139:0/1305421167 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5e3fff640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e415ae40 0x7ff5e415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5e37fe640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40b0210 0x7ff5e4156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5daffd640 1 -- 172.21.15.139:0/1305421167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40b0210 msgr2=0x7ff5e4156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5daffd640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40b0210 0x7ff5e4156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5daffd640 1 -- 172.21.15.139:0/1305421167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e415ae40 msgr2=0x7ff5e415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5daffd640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e415ae40 0x7ff5e415ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5daffd640 1 -- 172.21.15.139:0/1305421167 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e4154f10 con 0x7ff5e40cefe0 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5daffd640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40cefe0 0x7ff5e4165580 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00c7f0 tx=0x7ff5cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5e0ff9640 1 -- 172.21.15.139:0/1305421167 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc015070 con 0x7ff5e40cefe0 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5e0ff9640 1 -- 172.21.15.139:0/1305421167 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5cc007820 con 0x7ff5e40cefe0 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5e0ff9640 1 -- 172.21.15.139:0/1305421167 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc007bd0 con 0x7ff5e40cefe0 2024-04-25T10:03:15.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/1305421167 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e41551f0 con 0x7ff5e40cefe0 2024-04-25T10:03:15.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.502+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/1305421167 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e41557b0 con 0x7ff5e40cefe0 2024-04-25T10:03:15.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.503+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/1305421167 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e4074520 con 0x7ff5e40cefe0 2024-04-25T10:03:15.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.503+0000 7ff5e0ff9640 1 -- 172.21.15.139:0/1305421167 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5cc018070 con 0x7ff5e40cefe0 2024-04-25T10:03:15.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.504+0000 7ff5e0ff9640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff5c406b4d0 0x7ff5c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.504+0000 7ff5e37fe640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff5c406b4d0 0x7ff5c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.504+0000 7ff5e0ff9640 1 -- 172.21.15.139:0/1305421167 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff5cc08a5e0 con 0x7ff5e40cefe0 2024-04-25T10:03:15.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.504+0000 7ff5e37fe640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff5c406b4d0 0x7ff5c406d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4006fd0 tx=0x7ff5d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.508+0000 7ff5e0ff9640 1 -- 172.21.15.139:0/1305421167 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5cc058aa0 con 0x7ff5e40cefe0 2024-04-25T10:03:15.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.715+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/1305421167 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5e40b2040 con 0x7ff5e40cefe0 2024-04-25T10:03:15.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.716+0000 7ff5e0ff9640 1 -- 172.21.15.139:0/1305421167 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4951 (secure 0 0 0) 0x7ff5cc05cb40 con 0x7ff5e40cefe0 2024-04-25T10:03:15.722 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:15.722 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":7690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3195407462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3195407462},{"type":"v1","addr":"172.21.15.150:6835","nonce":3195407462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:03:15.379948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":93,"state":"up:replay","state_seq":1,"addr":"172.21.15.139:6836/1898344850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1898344850},{"type":"v1","addr":"172.21.15.139:6836","nonce":1898344850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:03:15.722 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 93 2024-04-25T10:03:15.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.718+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/1305421167 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff5c406b4d0 msgr2=0x7ff5c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.718+0000 7ff5e9e8f640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff5c406b4d0 0x7ff5c406d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4006fd0 tx=0x7ff5d4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:15.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.718+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/1305421167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40cefe0 msgr2=0x7ff5e4165580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.718+0000 7ff5e9e8f640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40cefe0 0x7ff5e4165580 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00c7f0 tx=0x7ff5cc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.718+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/1305421167 shutdown_connections 2024-04-25T10:03:15.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.718+0000 7ff5e9e8f640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5e40cefe0 0x7ff5e4165580 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.718+0000 7ff5e9e8f640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff5c406b4d0 0x7ff5c406d990 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.718+0000 7ff5e9e8f640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5e415ae40 0x7ff5e415ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.718+0000 7ff5e9e8f640 1 --2- 172.21.15.139:0/1305421167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5e40b0210 0x7ff5e4156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.718+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/1305421167 >> 172.21.15.139:0/1305421167 conn(0x7ff5e4066f20 msgr2=0x7ff5e415e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.718+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/1305421167 shutdown_connections 2024-04-25T10:03:15.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:15.718+0000 7ff5e9e8f640 1 -- 172.21.15.139:0/1305421167 wait complete. 2024-04-25T10:03:16.137 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:50.319874+0000', 'modified': '2024-04-25T10:03:15.379948+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9573}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9573': {'gid': 9573, 'name': 'a', 'rank': 0, 'incarnation': 93, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.139:6836/1898344850', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1898344850}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 1898344850}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:03:16.137 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9573:up:replay 2024-04-25T10:03:17.138 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.240+0000 7fbb7702a640 1 Processor -- start 2024-04-25T10:03:17.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.240+0000 7fbb7702a640 1 -- start start 2024-04-25T10:03:17.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.241+0000 7fbb7702a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb70160e90 0x7fbb700afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.241+0000 7fbb7702a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbb7006ab60 0x7fbb7006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.241+0000 7fbb7702a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb700d0400 con 0x7fbb7006fa70 2024-04-25T10:03:17.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.241+0000 7fbb7702a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb700d14a0 con 0x7fbb7006ab60 2024-04-25T10:03:17.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.241+0000 7fbb7702a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7006b500 con 0x7fbb70160e90 2024-04-25T10:03:17.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.242+0000 7fbb6ffff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbb7006ab60 0x7fbb7006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.242+0000 7fbb74dc6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb70160e90 0x7fbb700afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.242+0000 7fbb6ffff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbb7006ab60 0x7fbb7006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:34114/0 (socket says 172.21.15.139:34114) 2024-04-25T10:03:17.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.242+0000 7fbb6ffff640 1 -- 172.21.15.139:0/1796049704 learned_addr learned my addr 172.21.15.139:0/1796049704 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:17.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.242+0000 7fbb74dc6640 1 -- 172.21.15.139:0/1796049704 >> v2:172.21.15.150:3300/0 conn(0x7fbb7006ab60 msgr2=0x7fbb7006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.242+0000 7fbb74dc6640 1 --2- 172.21.15.139:0/1796049704 >> v2:172.21.15.150:3300/0 conn(0x7fbb7006ab60 0x7fbb7006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.242+0000 7fbb74dc6640 1 -- 172.21.15.139:0/1796049704 >> v1:172.21.15.150:6789/0 conn(0x7fbb7006fa70 legacy=0x7fbb7006fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.242+0000 7fbb74dc6640 1 -- 172.21.15.139:0/1796049704 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb700d06e0 con 0x7fbb70160e90 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.242+0000 7fbb6ffff640 1 --2- 172.21.15.139:0/1796049704 >> v2:172.21.15.150:3300/0 conn(0x7fbb7006ab60 0x7fbb7006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.243+0000 7fbb74dc6640 1 --2- 172.21.15.139:0/1796049704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb70160e90 0x7fbb700afac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbb600097a0 tx=0x7fbb6002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae3bfd73900c1dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.243+0000 7fbb6f7fe640 1 -- 172.21.15.139:0/1796049704 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60037070 con 0x7fbb70160e90 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.243+0000 7fbb6f7fe640 1 -- 172.21.15.139:0/1796049704 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb60032440 con 0x7fbb70160e90 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.243+0000 7fbb6f7fe640 1 -- 172.21.15.139:0/1796049704 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60032810 con 0x7fbb70160e90 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.244+0000 7fbb7702a640 1 -- 172.21.15.139:0/1796049704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb70160e90 msgr2=0x7fbb700afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.244+0000 7fbb7702a640 1 --2- 172.21.15.139:0/1796049704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb70160e90 0x7fbb700afac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbb600097a0 tx=0x7fbb6002a180 comp rx=0 tx=0).stop 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.244+0000 7fbb7702a640 1 -- 172.21.15.139:0/1796049704 shutdown_connections 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.244+0000 7fbb7702a640 1 --2- 172.21.15.139:0/1796049704 >> v2:172.21.15.150:3300/0 conn(0x7fbb7006ab60 0x7fbb7006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.244+0000 7fbb7702a640 1 --2- 172.21.15.139:0/1796049704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb70160e90 0x7fbb700afac0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.244+0000 7fbb7702a640 1 -- 172.21.15.139:0/1796049704 >> 172.21.15.139:0/1796049704 conn(0x7fbb700b23f0 msgr2=0x7fbb700b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.244+0000 7fbb7702a640 1 -- 172.21.15.139:0/1796049704 shutdown_connections 2024-04-25T10:03:17.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.244+0000 7fbb7702a640 1 -- 172.21.15.139:0/1796049704 wait complete. 2024-04-25T10:03:17.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.245+0000 7fbb7702a640 1 Processor -- start 2024-04-25T10:03:17.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.245+0000 7fbb7702a640 1 -- start start 2024-04-25T10:03:17.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb7702a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb7006ab60 0x7fbb70149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb7702a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb7006fa70 0x7fbb701e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb7702a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb70160e90 0x7fbb701e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb7702a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700d0f80 con 0x7fbb70160e90 2024-04-25T10:03:17.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb7702a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700d0ca0 con 0x7fbb7006ab60 2024-04-25T10:03:17.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb7702a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700b0000 con 0x7fbb7006fa70 2024-04-25T10:03:17.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb755c7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb70160e90 0x7fbb701e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb755c7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb70160e90 0x7fbb701e7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39670/0 (socket says 172.21.15.139:39670) 2024-04-25T10:03:17.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb755c7640 1 -- 172.21.15.139:0/2056334821 learned_addr learned my addr 172.21.15.139:0/2056334821 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:17.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb74dc6640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb7006ab60 0x7fbb70149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb6ffff640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb7006fa70 0x7fbb701e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb755c7640 1 -- 172.21.15.139:0/2056334821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb7006fa70 msgr2=0x7fbb701e7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb755c7640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb7006fa70 0x7fbb701e7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb755c7640 1 -- 172.21.15.139:0/2056334821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb7006ab60 msgr2=0x7fbb70149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb755c7640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb7006ab60 0x7fbb70149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.246+0000 7fbb755c7640 1 -- 172.21.15.139:0/2056334821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb701e8720 con 0x7fbb70160e90 2024-04-25T10:03:17.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.247+0000 7fbb755c7640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb70160e90 0x7fbb701e7fe0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c7d0 tx=0x7fbb6800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.247+0000 7fbb6d7fa640 1 -- 172.21.15.139:0/2056334821 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68003b70 con 0x7fbb70160e90 2024-04-25T10:03:17.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.247+0000 7fbb6d7fa640 1 -- 172.21.15.139:0/2056334821 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb68003d30 con 0x7fbb70160e90 2024-04-25T10:03:17.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.247+0000 7fbb7702a640 1 -- 172.21.15.139:0/2056334821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb701e8a00 con 0x7fbb70160e90 2024-04-25T10:03:17.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.247+0000 7fbb6d7fa640 1 -- 172.21.15.139:0/2056334821 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6800aa90 con 0x7fbb70160e90 2024-04-25T10:03:17.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.247+0000 7fbb7702a640 1 -- 172.21.15.139:0/2056334821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb701f3d20 con 0x7fbb70160e90 2024-04-25T10:03:17.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.248+0000 7fbb7702a640 1 -- 172.21.15.139:0/2056334821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb70074520 con 0x7fbb70160e90 2024-04-25T10:03:17.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.248+0000 7fbb6d7fa640 1 -- 172.21.15.139:0/2056334821 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb6800ac30 con 0x7fbb70160e90 2024-04-25T10:03:17.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.248+0000 7fbb6d7fa640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb5c06b4d0 0x7fbb5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.249+0000 7fbb6d7fa640 1 -- 172.21.15.139:0/2056334821 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fbb6808a2a0 con 0x7fbb70160e90 2024-04-25T10:03:17.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.249+0000 7fbb74dc6640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb5c06b4d0 0x7fbb5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.249+0000 7fbb74dc6640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb5c06b4d0 0x7fbb5c06d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbb60034000 tx=0x7fbb6002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.253+0000 7fbb6d7fa640 1 -- 172.21.15.139:0/2056334821 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb68058800 con 0x7fbb70160e90 2024-04-25T10:03:17.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.460+0000 7fbb7702a640 1 -- 172.21.15.139:0/2056334821 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb700728b0 con 0x7fbb70160e90 2024-04-25T10:03:17.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.460+0000 7fbb6d7fa640 1 -- 172.21.15.139:0/2056334821 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4951 (secure 0 0 0) 0x7fbb6805c8a0 con 0x7fbb70160e90 2024-04-25T10:03:17.467 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:17.467 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":95,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":7690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3195407462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3195407462},{"type":"v1","addr":"172.21.15.150:6835","nonce":3195407462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:03:16.384927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":93,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.139:6836/1898344850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1898344850},{"type":"v1","addr":"172.21.15.139:6836","nonce":1898344850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:03:17.467 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 95 2024-04-25T10:03:17.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.462+0000 7fbb7702a640 1 -- 172.21.15.139:0/2056334821 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb5c06b4d0 msgr2=0x7fbb5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.462+0000 7fbb7702a640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb5c06b4d0 0x7fbb5c06d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbb60034000 tx=0x7fbb6002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.462+0000 7fbb7702a640 1 -- 172.21.15.139:0/2056334821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb70160e90 msgr2=0x7fbb701e7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.462+0000 7fbb7702a640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb70160e90 0x7fbb701e7fe0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c7d0 tx=0x7fbb6800cc90 comp rx=0 tx=0).stop 2024-04-25T10:03:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.463+0000 7fbb7702a640 1 -- 172.21.15.139:0/2056334821 shutdown_connections 2024-04-25T10:03:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.463+0000 7fbb7702a640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb70160e90 0x7fbb701e7fe0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.463+0000 7fbb7702a640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbb5c06b4d0 0x7fbb5c06d990 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.463+0000 7fbb7702a640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb7006fa70 0x7fbb701e7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.463+0000 7fbb7702a640 1 --2- 172.21.15.139:0/2056334821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb7006ab60 0x7fbb70149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.463+0000 7fbb7702a640 1 -- 172.21.15.139:0/2056334821 >> 172.21.15.139:0/2056334821 conn(0x7fbb700b23f0 msgr2=0x7fbb7006eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.463+0000 7fbb7702a640 1 -- 172.21.15.139:0/2056334821 shutdown_connections 2024-04-25T10:03:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:17.463+0000 7fbb7702a640 1 -- 172.21.15.139:0/2056334821 wait complete. 2024-04-25T10:03:17.686 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:17.689 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:17.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:50.319874+0000', 'modified': '2024-04-25T10:03:16.384927+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9573}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9573': {'gid': 9573, 'name': 'a', 'rank': 0, 'incarnation': 93, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.139:6836/1898344850', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1898344850}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 1898344850}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:03:17.882 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9573:up:rejoin 2024-04-25T10:03:18.883 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:18.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.987+0000 7fa41886b640 1 Processor -- start 2024-04-25T10:03:18.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.988+0000 7fa41886b640 1 -- start start 2024-04-25T10:03:18.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa41886b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa4141628d0 0x7fa414166660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa41886b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4141633c0 0x7fa4140d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa41886b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4140ce500 con 0x7fa4140ce030 2024-04-25T10:03:18.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa41886b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa414165c10 con 0x7fa4141628d0 2024-04-25T10:03:18.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa41886b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa414166ba0 con 0x7fa4141633c0 2024-04-25T10:03:18.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa412d9d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4141633c0 0x7fa4140d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa41259c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa4141628d0 0x7fa414166660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa411d9b640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fa4140ce030 0x7fa414162cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:49004/0 (socket says 172.21.15.139:49004) 2024-04-25T10:03:18.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa411d9b640 1 -- 172.21.15.139:0/2339886942 learned_addr learned my addr 172.21.15.139:0/2339886942 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:18.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa41259c640 1 -- 172.21.15.139:0/2339886942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4141633c0 msgr2=0x7fa4140d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa41259c640 1 --2- 172.21.15.139:0/2339886942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4141633c0 0x7fa4140d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa41259c640 1 -- 172.21.15.139:0/2339886942 >> v1:172.21.15.150:6789/0 conn(0x7fa4140ce030 legacy=0x7fa414162cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa41259c640 1 -- 172.21.15.139:0/2339886942 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4140ce7c0 con 0x7fa4141628d0 2024-04-25T10:03:18.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa41159a640 1 -- 172.21.15.139:0/2339886942 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185680896 0 0) 0x7fa4140ce500 con 0x7fa4140ce030 2024-04-25T10:03:18.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.989+0000 7fa412d9d640 1 --2- 172.21.15.139:0/2339886942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4141633c0 0x7fa4140d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:18.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.990+0000 7fa41259c640 1 --2- 172.21.15.139:0/2339886942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4141628d0 0x7fa414166660 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa4040097a0 tx=0x7fa40402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d04877edca54daef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.990+0000 7fa41159a640 1 -- 172.21.15.139:0/2339886942 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404037070 con 0x7fa4141628d0 2024-04-25T10:03:18.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.990+0000 7fa41159a640 1 -- 172.21.15.139:0/2339886942 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40402adb0 con 0x7fa4141628d0 2024-04-25T10:03:18.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.990+0000 7fa41159a640 1 -- 172.21.15.139:0/2339886942 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404032660 con 0x7fa4141628d0 2024-04-25T10:03:18.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.991+0000 7fa41886b640 1 -- 172.21.15.139:0/2339886942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4141628d0 msgr2=0x7fa414166660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.991+0000 7fa41886b640 1 --2- 172.21.15.139:0/2339886942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4141628d0 0x7fa414166660 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa4040097a0 tx=0x7fa40402a060 comp rx=0 tx=0).stop 2024-04-25T10:03:18.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.991+0000 7fa41886b640 1 -- 172.21.15.139:0/2339886942 shutdown_connections 2024-04-25T10:03:18.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.991+0000 7fa41886b640 1 --2- 172.21.15.139:0/2339886942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4141633c0 0x7fa4140d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.991+0000 7fa41886b640 1 --2- 172.21.15.139:0/2339886942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4141628d0 0x7fa414166660 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.991+0000 7fa41886b640 1 -- 172.21.15.139:0/2339886942 >> 172.21.15.139:0/2339886942 conn(0x7fa4140b5910 msgr2=0x7fa4140b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.991+0000 7fa41886b640 1 -- 172.21.15.139:0/2339886942 shutdown_connections 2024-04-25T10:03:18.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.991+0000 7fa41886b640 1 -- 172.21.15.139:0/2339886942 wait complete. 2024-04-25T10:03:18.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.992+0000 7fa41886b640 1 Processor -- start 2024-04-25T10:03:19.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.992+0000 7fa41886b640 1 -- start start 2024-04-25T10:03:19.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.993+0000 7fa41886b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4140ce030 0x7fa4140b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.993+0000 7fa41886b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4141628d0 0x7fa4140cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.993+0000 7fa41886b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4141633c0 0x7fa4140cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.993+0000 7fa41886b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140d0e60 con 0x7fa4140ce030 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.993+0000 7fa41886b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa414166140 con 0x7fa4141633c0 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.993+0000 7fa41886b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140d0fe0 con 0x7fa4141628d0 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.993+0000 7fa41259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4140ce030 0x7fa4140b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.993+0000 7fa41259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4140ce030 0x7fa4140b8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39678/0 (socket says 172.21.15.139:39678) 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.993+0000 7fa41259c640 1 -- 172.21.15.139:0/330299753 learned_addr learned my addr 172.21.15.139:0/330299753 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.993+0000 7fa411d9b640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4141628d0 0x7fa4140cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.993+0000 7fa412d9d640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4141633c0 0x7fa4140cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.994+0000 7fa41259c640 1 -- 172.21.15.139:0/330299753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4141628d0 msgr2=0x7fa4140cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.994+0000 7fa41259c640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4141628d0 0x7fa4140cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.994+0000 7fa41259c640 1 -- 172.21.15.139:0/330299753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4141633c0 msgr2=0x7fa4140cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.994+0000 7fa41259c640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4141633c0 0x7fa4140cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.994+0000 7fa41259c640 1 -- 172.21.15.139:0/330299753 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41415bec0 con 0x7fa4140ce030 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.994+0000 7fa41259c640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4140ce030 0x7fa4140b8e90 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fa404035040 tx=0x7fa4040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.994+0000 7fa40affd640 1 -- 172.21.15.139:0/330299753 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404037040 con 0x7fa4140ce030 2024-04-25T10:03:19.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.994+0000 7fa40affd640 1 -- 172.21.15.139:0/330299753 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa404004030 con 0x7fa4140ce030 2024-04-25T10:03:19.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.994+0000 7fa41886b640 1 -- 172.21.15.139:0/330299753 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41414baf0 con 0x7fa4140ce030 2024-04-25T10:03:19.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.994+0000 7fa40affd640 1 -- 172.21.15.139:0/330299753 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404004400 con 0x7fa4140ce030 2024-04-25T10:03:19.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.994+0000 7fa41886b640 1 -- 172.21.15.139:0/330299753 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41414c070 con 0x7fa4140ce030 2024-04-25T10:03:19.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.995+0000 7fa41886b640 1 -- 172.21.15.139:0/330299753 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa414074520 con 0x7fa4140ce030 2024-04-25T10:03:19.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.996+0000 7fa40affd640 1 -- 172.21.15.139:0/330299753 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa404003610 con 0x7fa4140ce030 2024-04-25T10:03:19.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.996+0000 7fa40affd640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa3f406b4d0 0x7fa3f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.996+0000 7fa40affd640 1 -- 172.21.15.139:0/330299753 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fa4040ab210 con 0x7fa4140ce030 2024-04-25T10:03:19.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.996+0000 7fa411d9b640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa3f406b4d0 0x7fa3f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:18.996+0000 7fa411d9b640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa3f406b4d0 0x7fa3f406d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa4140b93d0 tx=0x7fa3fc007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.000+0000 7fa40affd640 1 -- 172.21.15.139:0/330299753 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa41415bec0 con 0x7fa4140ce030 2024-04-25T10:03:19.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.207+0000 7fa41886b640 1 -- 172.21.15.139:0/330299753 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4140ca750 con 0x7fa4140ce030 2024-04-25T10:03:19.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.208+0000 7fa40affd640 1 -- 172.21.15.139:0/330299753 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4951 (secure 0 0 0) 0x7fa404079e50 con 0x7fa4140ce030 2024-04-25T10:03:19.214 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:19.214 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":7690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3195407462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3195407462},{"type":"v1","addr":"172.21.15.150:6835","nonce":3195407462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:03:17.394485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6836/1898344850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1898344850},{"type":"v1","addr":"172.21.15.139:6836","nonce":1898344850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:03:19.214 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 96 2024-04-25T10:03:19.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.210+0000 7fa41886b640 1 -- 172.21.15.139:0/330299753 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa3f406b4d0 msgr2=0x7fa3f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.210+0000 7fa41886b640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa3f406b4d0 0x7fa3f406d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa4140b93d0 tx=0x7fa3fc007b50 comp rx=0 tx=0).stop 2024-04-25T10:03:19.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.210+0000 7fa41886b640 1 -- 172.21.15.139:0/330299753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4140ce030 msgr2=0x7fa4140b8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.210+0000 7fa41886b640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4140ce030 0x7fa4140b8e90 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fa404035040 tx=0x7fa4040046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.210+0000 7fa41886b640 1 -- 172.21.15.139:0/330299753 shutdown_connections 2024-04-25T10:03:19.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.210+0000 7fa41886b640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa3f406b4d0 0x7fa3f406d990 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.210+0000 7fa41886b640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4141633c0 0x7fa4140cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.210+0000 7fa41886b640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4141628d0 0x7fa4140cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.210+0000 7fa41886b640 1 --2- 172.21.15.139:0/330299753 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4140ce030 0x7fa4140b8e90 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.210+0000 7fa41886b640 1 -- 172.21.15.139:0/330299753 >> 172.21.15.139:0/330299753 conn(0x7fa4140b5910 msgr2=0x7fa41406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.210+0000 7fa41886b640 1 -- 172.21.15.139:0/330299753 shutdown_connections 2024-04-25T10:03:19.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.210+0000 7fa41886b640 1 -- 172.21.15.139:0/330299753 wait complete. 2024-04-25T10:03:19.380 INFO:tasks.ceph.mds.a.smithi139.stderr:2024-04-25T10:03:19.375+0000 7f26626c5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:19.629 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:50.319874+0000', 'modified': '2024-04-25T10:03:17.394485+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9573}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9573': {'gid': 9573, 'name': 'a', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.139:6836/1898344850', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1898344850}, {'type': 'v1', 'addr': '172.21.15.139:6836', 'nonce': 1898344850}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:03:19.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:03:19.630 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:03:19.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.725+0000 7f83bfdce640 1 Processor -- start 2024-04-25T10:03:19.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.725+0000 7f83bfdce640 1 -- start start 2024-04-25T10:03:19.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bfdce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b8150ed0 0x7f83b80b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bfdce640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f83b80cdd20 0x7f83b8071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bfdce640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f83b80b4740 0x7f83b8071f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bfdce640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b815a480 con 0x7f83b80cdd20 2024-04-25T10:03:19.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bfdce640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b815a760 con 0x7f83b80b4740 2024-04-25T10:03:19.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bfdce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b815aa40 con 0x7f83b8150ed0 2024-04-25T10:03:19.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bd369640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f83b80cdd20 0x7f83b8071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bd369640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f83b80cdd20 0x7f83b8071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39690/0 (socket says 172.21.15.139:39690) 2024-04-25T10:03:19.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bd369640 1 -- 172.21.15.139:0/4152060146 learned_addr learned my addr 172.21.15.139:0/4152060146 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bd369640 1 -- 172.21.15.139:0/4152060146 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b8150ed0 msgr2=0x7f83b80b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bd369640 1 --2- 172.21.15.139:0/4152060146 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b8150ed0 0x7f83b80b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bd369640 1 -- 172.21.15.139:0/4152060146 >> v2:172.21.15.150:3300/0 conn(0x7f83b80b4740 msgr2=0x7f83b8071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83be36b640 1 --2- 172.21.15.139:0/4152060146 >> v2:172.21.15.150:3300/0 conn(0x7f83b80b4740 0x7f83b8071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bd369640 1 --2- 172.21.15.139:0/4152060146 >> v2:172.21.15.150:3300/0 conn(0x7f83b80b4740 0x7f83b8071f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.726+0000 7f83bd369640 1 -- 172.21.15.139:0/4152060146 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b815ad20 con 0x7f83b80cdd20 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.727+0000 7f83bd369640 1 --2- 172.21.15.139:0/4152060146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b80cdd20 0x7f83b8071a50 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f83b8154ad0 tx=0x7f83a802c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d715780c28b109d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.727+0000 7f83bcb68640 1 -- 172.21.15.139:0/4152060146 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a803a070 con 0x7f83b80cdd20 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.727+0000 7f83bcb68640 1 -- 172.21.15.139:0/4152060146 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a8032480 con 0x7f83b80cdd20 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.727+0000 7f83bcb68640 1 -- 172.21.15.139:0/4152060146 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8032650 con 0x7f83b80cdd20 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.728+0000 7f83bfdce640 1 -- 172.21.15.139:0/4152060146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b80cdd20 msgr2=0x7f83b8071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.728+0000 7f83bfdce640 1 --2- 172.21.15.139:0/4152060146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b80cdd20 0x7f83b8071a50 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f83b8154ad0 tx=0x7f83a802c5e0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.728+0000 7f83bfdce640 1 -- 172.21.15.139:0/4152060146 shutdown_connections 2024-04-25T10:03:19.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.728+0000 7f83bfdce640 1 --2- 172.21.15.139:0/4152060146 >> v2:172.21.15.150:3300/0 conn(0x7f83b80b4740 0x7f83b8071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.728+0000 7f83bfdce640 1 --2- 172.21.15.139:0/4152060146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b80cdd20 0x7f83b8071a50 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.728+0000 7f83bfdce640 1 --2- 172.21.15.139:0/4152060146 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b8150ed0 0x7f83b80b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.728+0000 7f83bfdce640 1 -- 172.21.15.139:0/4152060146 >> 172.21.15.139:0/4152060146 conn(0x7f83b8067000 msgr2=0x7f83b806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.728+0000 7f83bfdce640 1 -- 172.21.15.139:0/4152060146 shutdown_connections 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.728+0000 7f83bfdce640 1 -- 172.21.15.139:0/4152060146 wait complete. 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.729+0000 7f83bfdce640 1 Processor -- start 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.729+0000 7f83bfdce640 1 -- start start 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83bfdce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b80b4740 0x7f83b81eef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83bfdce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83b80cdd20 0x7f83b8159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83bfdce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b8150ed0 0x7f83b81596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83bfdce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b81512b0 con 0x7f83b8150ed0 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83bfdce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80b3960 con 0x7f83b80cdd20 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83bfdce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80711c0 con 0x7f83b80b4740 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83be36b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b8150ed0 0x7f83b81596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83be36b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b8150ed0 0x7f83b81596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39706/0 (socket says 172.21.15.139:39706) 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83be36b640 1 -- 172.21.15.139:0/4198945331 learned_addr learned my addr 172.21.15.139:0/4198945331 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83bdb6a640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b80b4740 0x7f83b81eef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83bd369640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83b80cdd20 0x7f83b8159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83be36b640 1 -- 172.21.15.139:0/4198945331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b80b4740 msgr2=0x7f83b81eef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83be36b640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b80b4740 0x7f83b81eef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83be36b640 1 -- 172.21.15.139:0/4198945331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83b80cdd20 msgr2=0x7f83b8159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83be36b640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83b80cdd20 0x7f83b8159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83be36b640 1 -- 172.21.15.139:0/4198945331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b81f2d10 con 0x7f83b8150ed0 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83bd369640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83b80cdd20 0x7f83b8159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83be36b640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b8150ed0 0x7f83b81596d0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f83b400cd40 tx=0x7f83b400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83ae7fc640 1 -- 172.21.15.139:0/4198945331 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4003b90 con 0x7f83b8150ed0 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83ae7fc640 1 -- 172.21.15.139:0/4198945331 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b4003d30 con 0x7f83b8150ed0 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83bfdce640 1 -- 172.21.15.139:0/4198945331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b81f0530 con 0x7f83b8150ed0 2024-04-25T10:03:19.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.730+0000 7f83ae7fc640 1 -- 172.21.15.139:0/4198945331 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4004240 con 0x7f83b8150ed0 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.731+0000 7f83bfdce640 1 -- 172.21.15.139:0/4198945331 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b81f32b0 con 0x7f83b8150ed0 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.732+0000 7f83ae7fc640 1 -- 172.21.15.139:0/4198945331 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83b40043e0 con 0x7f83b8150ed0 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.732+0000 7f83ae7fc640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f839c06b4d0 0x7f839c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.732+0000 7f83ae7fc640 1 -- 172.21.15.139:0/4198945331 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f83b408ae40 con 0x7f83b8150ed0 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.732+0000 7f83bdb6a640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f839c06b4d0 0x7f839c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.733+0000 7f83bdb6a640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f839c06b4d0 0x7f839c06d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f83b81524b0 tx=0x7f83b0009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.734+0000 7f83c2bac740 1 Processor -- start 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.734+0000 7f83c2bac740 1 -- start start 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.734+0000 7f83c2bac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8c7e0 0x555636f64e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.734+0000 7f83c2bac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8cbc0 0x555636f8f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.734+0000 7f83c2bac740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8f580 0x555636f62de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.734+0000 7f83c2bac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555636ec7a10 con 0x555636f8c7e0 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.734+0000 7f83c2bac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555636e70b30 con 0x555636f8cbc0 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.734+0000 7f83c2bac740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555636ec3c50 con 0x555636f8f580 2024-04-25T10:03:19.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.734+0000 7f83bdb6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8c7e0 0x555636f64e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.734+0000 7f83bdb6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8c7e0 0x555636f64e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39720/0 (socket says 172.21.15.139:39720) 2024-04-25T10:03:19.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.734+0000 7f83bdb6a640 1 -- 172.21.15.139:0/3224822201 learned_addr learned my addr 172.21.15.139:0/3224822201 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:19.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f83be36b640 1 --2- 172.21.15.139:0/3224822201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8f580 0x555636f62de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f83bd369640 1 --2- 172.21.15.139:0/3224822201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8cbc0 0x555636f8f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f83bdb6a640 1 -- 172.21.15.139:0/3224822201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8f580 msgr2=0x555636f62de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f83bdb6a640 1 --2- 172.21.15.139:0/3224822201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8f580 0x555636f62de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f83bdb6a640 1 -- 172.21.15.139:0/3224822201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8cbc0 msgr2=0x555636f8f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f83bdb6a640 1 --2- 172.21.15.139:0/3224822201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8cbc0 0x555636f8f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f83bdb6a640 1 -- 172.21.15.139:0/3224822201 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555636e85e10 con 0x555636f8c7e0 2024-04-25T10:03:19.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f83be36b640 1 --2- 172.21.15.139:0/3224822201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8f580 0x555636f62de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f83bd369640 1 --2- 172.21.15.139:0/3224822201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8cbc0 0x555636f8f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f83bdb6a640 1 --2- 172.21.15.139:0/3224822201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8c7e0 0x555636f64e40 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f83b000ebd0 tx=0x7f83b0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6833a774911570 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f839b7fe640 1 -- 172.21.15.139:0/3224822201 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b001b040 con 0x555636f8c7e0 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f839b7fe640 1 -- 172.21.15.139:0/3224822201 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b0024070 con 0x555636f8c7e0 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.735+0000 7f839b7fe640 1 -- 172.21.15.139:0/3224822201 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0015cc0 con 0x555636f8c7e0 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.736+0000 7f83c2bac740 1 -- 172.21.15.139:0/3224822201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8c7e0 msgr2=0x555636f64e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.736+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3224822201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8c7e0 0x555636f64e40 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f83b000ebd0 tx=0x7f83b0011000 comp rx=0 tx=0).stop 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.736+0000 7f83c2bac740 1 -- 172.21.15.139:0/3224822201 shutdown_connections 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.736+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3224822201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8f580 0x555636f62de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.736+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3224822201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8cbc0 0x555636f8f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.736+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3224822201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8c7e0 0x555636f64e40 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.736+0000 7f83c2bac740 1 -- 172.21.15.139:0/3224822201 >> 172.21.15.139:0/3224822201 conn(0x555636f61840 msgr2=0x555636faa040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.736+0000 7f83c2bac740 1 -- 172.21.15.139:0/3224822201 shutdown_connections 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.736+0000 7f83c2bac740 1 -- 172.21.15.139:0/3224822201 wait complete. 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.736+0000 7f83c2bac740 1 -- start start 2024-04-25T10:03:19.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.736+0000 7f83c2bac740 1 Processor -- start 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83c2bac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8cbc0 0x555636f8b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83c2bac740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8f580 0x555636fa37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83c2bac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f911b0 0x555636f92ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83c2bac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555636e840c0 con 0x555636f8cbc0 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83c2bac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555636fa7d40 con 0x555636f911b0 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83c2bac740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555636fa4200 con 0x555636f8f580 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83c2bac740 10 client.0 _ll_register_callbacks cb 0x555636e77ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83bdb6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8cbc0 0x555636f8b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83bdb6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8cbc0 0x555636f8b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39730/0 (socket says 172.21.15.139:39730) 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83bdb6a640 1 -- 172.21.15.139:0/3070133911 learned_addr learned my addr 172.21.15.139:0/3070133911 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83bdb6a640 1 -- 172.21.15.139:0/3070133911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8f580 msgr2=0x555636fa37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83bdb6a640 1 --2- 172.21.15.139:0/3070133911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8f580 0x555636fa37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83bdb6a640 1 -- 172.21.15.139:0/3070133911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f911b0 msgr2=0x555636f92ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83bdb6a640 1 --2- 172.21.15.139:0/3070133911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f911b0 0x555636f92ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.737+0000 7f83bdb6a640 1 -- 172.21.15.139:0/3070133911 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555636bdc2d0 con 0x555636f8cbc0 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f83bdb6a640 1 --2- 172.21.15.139:0/3070133911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8cbc0 0x555636f8b230 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f83b000a000 tx=0x7f83b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6146c14053594886 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f8398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f8398ff9640 1 -- 172.21.15.139:0/3070133911 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b00139e0 con 0x555636f8cbc0 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f8398ff9640 1 -- 172.21.15.139:0/3070133911 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f83b0013b80 con 0x555636f8cbc0 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f83c2bac740 1 -- 172.21.15.139:0/3070133911 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83b0024070 con 0x555636f8cbc0 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f8398ff9640 1 -- 172.21.15.139:0/3070133911 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83b0039050 con 0x555636f8cbc0 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f83c2bac740 10 client.9660 fetch_fsmap learned FSMap version 96 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f83c2bac740 1 -- 172.21.15.139:0/3070133911 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x555636ca66b0 con 0x555636f8cbc0 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f8398ff9640 1 -- 172.21.15.139:0/3070133911 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b0024070 con 0x555636f8cbc0 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f8398ff9640 1 -- 172.21.15.139:0/3070133911 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 96) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f83b003c020 con 0x555636f8cbc0 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f83c2bac740 10 client.9660 fetch_fsmap finished waiting for FSMap version 96 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f83c2bac740 10 client.9660 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f83c2bac740 20 client.9660 populate_metadata read hostname 'smithi139' 2024-04-25T10:03:19.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3070133911 >> [v2:172.21.15.139:6834/1898344850,v1:172.21.15.139:6836/1898344850] conn(0x555636f8dbb0 0x555636f8dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f83c2bac740 4 client.9660 mds_command: new command op to 9573 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f83c2bac740 1 -- 172.21.15.139:0/3070133911 --> [v2:172.21.15.139:6834/1898344850,v1:172.21.15.139:6836/1898344850] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555636f26920 con 0x555636f8dbb0 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.738+0000 7f83bd369640 1 --2- 172.21.15.139:0/3070133911 >> [v2:172.21.15.139:6834/1898344850,v1:172.21.15.139:6836/1898344850] conn(0x555636f8dbb0 0x555636f8dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f8398ff9640 10 client.9660 ms_handle_connect on v2:172.21.15.139:6834/1898344850 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f83bd369640 1 --2- 172.21.15.139:0/3070133911 >> [v2:172.21.15.139:6834/1898344850,v1:172.21.15.139:6836/1898344850] conn(0x555636f8dbb0 0x555636f8dfb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7839b1d3c6aea6d9 server_cookie=b88285251fdf77cb in_seq=0 out_seq=0 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f8398ff9640 1 -- 172.21.15.139:0/3070133911 <== mds.0 v2:172.21.15.139:6834/1898344850 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555636f26920 con 0x555636f8dbb0 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f8398ff9640 10 client.9660 handle_command_reply: tid=0 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f83c2bac740 1 client.9660 shutdown 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f83c2bac740 1 -- 172.21.15.139:0/3070133911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8cbc0 msgr2=0x555636f8b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3070133911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8cbc0 0x555636f8b230 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f83b000a000 tx=0x7f83b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f83c2bac740 1 -- 172.21.15.139:0/3070133911 shutdown_connections 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3070133911 >> [v2:172.21.15.139:6834/1898344850,v1:172.21.15.139:6836/1898344850] conn(0x555636f8dbb0 0x555636f8dfb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3070133911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f911b0 0x555636f92ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3070133911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8f580 0x555636fa37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f8398ff9640 0 client.9660 ms_handle_reset on v2:172.21.15.139:6834/1898344850 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3070133911 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8cbc0 0x555636f8b230 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.739+0000 7f83c2bac740 1 -- 172.21.15.139:0/3070133911 >> 172.21.15.139:0/3070133911 conn(0x555636f8c7e0 msgr2=0x555636f5ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.740+0000 7f83c2bac740 1 -- 172.21.15.139:0/3070133911 shutdown_connections 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.740+0000 7f83c2bac740 1 -- 172.21.15.139:0/3070133911 wait complete. 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.740+0000 7f83c2bac740 20 client.9660 trim_cache size 0 max 16384 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.755+0000 7f83c2bac740 1 Processor -- start 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.755+0000 7f83c2bac740 1 -- start start 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83c2bac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8cbc0 0x555636f9ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83c2bac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8f580 0x555636fa04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83c2bac740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f911b0 0x555636fad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83c2bac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b81fb4a0 con 0x555636f8f580 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83c2bac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b815bf60 con 0x555636f8cbc0 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83c2bac740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b815c560 con 0x555636f911b0 2024-04-25T10:03:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83bd369640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8f580 0x555636fa04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83bd369640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8f580 0x555636fa04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39742/0 (socket says 172.21.15.139:39742) 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83bd369640 1 -- 172.21.15.139:0/3155556581 learned_addr learned my addr 172.21.15.139:0/3155556581 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83bd369640 1 -- 172.21.15.139:0/3155556581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f911b0 msgr2=0x555636fad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83bd369640 1 --2- 172.21.15.139:0/3155556581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f911b0 0x555636fad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83bd369640 1 -- 172.21.15.139:0/3155556581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8cbc0 msgr2=0x555636f9ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.756+0000 7f83bd369640 1 --2- 172.21.15.139:0/3155556581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8cbc0 0x555636f9ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f83bd369640 1 -- 172.21.15.139:0/3155556581 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555636f93570 con 0x555636f8f580 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f83bd369640 1 --2- 172.21.15.139:0/3155556581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8f580 0x555636fa04f0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f83a8000c00 tx=0x7f83a8032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49dda7ab46e04acb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f839affd640 1 -- 172.21.15.139:0/3155556581 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8049030 con 0x555636f8f580 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f839affd640 1 -- 172.21.15.139:0/3155556581 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a80440e0 con 0x555636f8f580 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f839affd640 1 -- 172.21.15.139:0/3155556581 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a803a040 con 0x555636f8f580 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f83c2bac740 1 -- 172.21.15.139:0/3155556581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8f580 msgr2=0x555636fa04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3155556581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8f580 0x555636fa04f0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f83a8000c00 tx=0x7f83a8032000 comp rx=0 tx=0).stop 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f83c2bac740 1 -- 172.21.15.139:0/3155556581 shutdown_connections 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3155556581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f8f580 0x555636fa04f0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3155556581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f911b0 0x555636fad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f83c2bac740 1 --2- 172.21.15.139:0/3155556581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8cbc0 0x555636f9ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f83c2bac740 1 -- 172.21.15.139:0/3155556581 >> 172.21.15.139:0/3155556581 conn(0x555636f8c7e0 msgr2=0x555636f8b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f83c2bac740 1 -- 172.21.15.139:0/3155556581 shutdown_connections 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.757+0000 7f83c2bac740 1 -- 172.21.15.139:0/3155556581 wait complete. 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.758+0000 7f83c2bac740 1 -- start start 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.758+0000 7f83c2bac740 1 Processor -- start 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83c2bac740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8cbc0 0x555637002950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83c2bac740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8f580 0x555636fc8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83c2bac740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f911b0 0x555636fc8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83c2bac740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x555636f91630 con 0x555636f911b0 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83c2bac740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555636f924f0 con 0x555636f8f580 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83c2bac740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555636fa1ab0 con 0x555636f8cbc0 2024-04-25T10:03:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83c2bac740 10 client.0 _ll_register_callbacks cb 0x7f83b0009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83be36b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f911b0 0x555636fc8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83be36b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f911b0 0x555636fc8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39746/0 (socket says 172.21.15.139:39746) 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83be36b640 1 -- 172.21.15.139:0/1625822584 learned_addr learned my addr 172.21.15.139:0/1625822584 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83bdb6a640 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8cbc0 0x555637002950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83bd369640 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8f580 0x555636fc8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83be36b640 1 -- 172.21.15.139:0/1625822584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8cbc0 msgr2=0x555637002950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83be36b640 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8cbc0 0x555637002950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83be36b640 1 -- 172.21.15.139:0/1625822584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8f580 msgr2=0x555636fc8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83be36b640 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8f580 0x555636fc8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83be36b640 1 -- 172.21.15.139:0/1625822584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555636f8a510 con 0x555636f911b0 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f83be36b640 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f911b0 0x555636fc8750 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f83b4061560 tx=0x7f83b4059b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9a6344c36db513 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.759+0000 7f8398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f8398ff9640 1 -- 172.21.15.139:0/1625822584 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4061a90 con 0x555636f911b0 2024-04-25T10:03:19.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f8398ff9640 1 -- 172.21.15.139:0/1625822584 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f83b405d150 con 0x555636f911b0 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f83c2bac740 1 -- 172.21.15.139:0/1625822584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83a80440e0 con 0x555636f911b0 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f8398ff9640 1 -- 172.21.15.139:0/1625822584 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83b40559f0 con 0x555636f911b0 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f83c2bac740 10 client.9666 fetch_fsmap learned FSMap version 96 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f83c2bac740 1 -- 172.21.15.139:0/1625822584 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x555636fa2de0 con 0x555636f911b0 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f8398ff9640 1 -- 172.21.15.139:0/1625822584 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a80440e0 con 0x555636f911b0 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f8398ff9640 1 -- 172.21.15.139:0/1625822584 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 96) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f83b407a020 con 0x555636f911b0 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f83c2bac740 10 client.9666 fetch_fsmap finished waiting for FSMap version 96 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f83c2bac740 10 client.9666 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f83c2bac740 20 client.9666 populate_metadata read hostname 'smithi139' 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f83c2bac740 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.139:6834/1898344850,v1:172.21.15.139:6836/1898344850] conn(0x555636f64ca0 0x555636f9ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f83c2bac740 4 client.9666 mds_command: new command op to 9573 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f83c2bac740 1 -- 172.21.15.139:0/1625822584 --> [v2:172.21.15.139:6834/1898344850,v1:172.21.15.139:6836/1898344850] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555636ed4f50 con 0x555636f64ca0 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f83bdb6a640 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.139:6834/1898344850,v1:172.21.15.139:6836/1898344850] conn(0x555636f64ca0 0x555636f9ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f8398ff9640 10 client.9666 ms_handle_connect on v2:172.21.15.139:6834/1898344850 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.760+0000 7f83bdb6a640 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.139:6834/1898344850,v1:172.21.15.139:6836/1898344850] conn(0x555636f64ca0 0x555636f9ffb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9edca6d055275844 server_cookie=242fe9cefa7f2b41 in_seq=0 out_seq=0 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f8398ff9640 1 -- 172.21.15.139:0/1625822584 <== mds.0 v2:172.21.15.139:6834/1898344850 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x555636ed4f50 con 0x555636f64ca0 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f8398ff9640 10 client.9666 handle_command_reply: tid=0 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83c2bac740 1 client.9666 shutdown 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83c2bac740 1 -- 172.21.15.139:0/1625822584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f911b0 msgr2=0x555636fc8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83c2bac740 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f911b0 0x555636fc8750 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f83b4061560 tx=0x7f83b4059b40 comp rx=0 tx=0).stop 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83c2bac740 1 -- 172.21.15.139:0/1625822584 shutdown_connections 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83c2bac740 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.139:6834/1898344850,v1:172.21.15.139:6836/1898344850] conn(0x555636f64ca0 0x555636f9ffb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83c2bac740 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555636f911b0 0x555636fc8750 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f8398ff9640 0 client.9666 ms_handle_reset on v2:172.21.15.139:6834/1898344850 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83c2bac740 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555636f8f580 0x555636fc8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83c2bac740 1 --2- 172.21.15.139:0/1625822584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555636f8cbc0 0x555637002950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83c2bac740 1 -- 172.21.15.139:0/1625822584 >> 172.21.15.139:0/1625822584 conn(0x555636f8c7e0 msgr2=0x555636f8b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83c2bac740 1 -- 172.21.15.139:0/1625822584 shutdown_connections 2024-04-25T10:03:19.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83c2bac740 1 -- 172.21.15.139:0/1625822584 wait complete. 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83c2bac740 20 client.9666 trim_cache size 0 max 16384 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "922739a8-edcf-4ded-8206-dbdb23e9fea4", 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 9573, 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 4.3808648010000004, 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 96, 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 122, 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 122, 2024-04-25T10:03:19.794 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 6.3628032079999999 2024-04-25T10:03:19.794 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:03:19.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83bfdce640 1 -- 172.21.15.139:0/4198945331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f839c06b4d0 msgr2=0x7f839c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83bfdce640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f839c06b4d0 0x7f839c06d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f83b81524b0 tx=0x7f83b0009060 comp rx=0 tx=0).stop 2024-04-25T10:03:19.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83bfdce640 1 -- 172.21.15.139:0/4198945331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b8150ed0 msgr2=0x7f83b81596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.761+0000 7f83bfdce640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b8150ed0 0x7f83b81596d0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f83b400cd40 tx=0x7f83b400a660 comp rx=0 tx=0).stop 2024-04-25T10:03:19.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.762+0000 7f83bfdce640 1 -- 172.21.15.139:0/4198945331 shutdown_connections 2024-04-25T10:03:19.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.762+0000 7f83bfdce640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83b8150ed0 0x7f83b81596d0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.762+0000 7f83bfdce640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f839c06b4d0 0x7f839c06d990 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.762+0000 7f83bfdce640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83b80cdd20 0x7f83b8159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.762+0000 7f83bfdce640 1 --2- 172.21.15.139:0/4198945331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b80b4740 0x7f83b81eef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.762+0000 7f83bfdce640 1 -- 172.21.15.139:0/4198945331 >> 172.21.15.139:0/4198945331 conn(0x7f83b8067000 msgr2=0x7f83b81501e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.762+0000 7f83bfdce640 1 -- 172.21.15.139:0/4198945331 shutdown_connections 2024-04-25T10:03:19.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.762+0000 7f83bfdce640 1 -- 172.21.15.139:0/4198945331 wait complete. 2024-04-25T10:03:19.796 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:03:19.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.887+0000 7ff99f8b3640 1 Processor -- start 2024-04-25T10:03:19.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.888+0000 7ff99f8b3640 1 -- start start 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.888+0000 7ff99f8b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff99815dc60 0x7ff9980b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.888+0000 7ff99f8b3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff9980d1080 0x7ff9980ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.888+0000 7ff99f8b3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9980cf250 con 0x7ff998161850 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.888+0000 7ff99f8b3640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9980ce780 con 0x7ff9980d1080 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.888+0000 7ff99f8b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9980d3730 con 0x7ff99815dc60 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff99ce4e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff998161850 0x7ff9980d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53898/0 (socket says 172.21.15.139:53898) 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff99ce4e640 1 -- 172.21.15.139:0/3592294087 learned_addr learned my addr 172.21.15.139:0/3592294087 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff99de50640 1 --2- 172.21.15.139:0/3592294087 >> v2:172.21.15.150:3300/0 conn(0x7ff9980d1080 0x7ff9980ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff99d64f640 1 --2- 172.21.15.139:0/3592294087 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff99815dc60 0x7ff9980b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893516672 0 0) 0x7ff994001110 con 0x7ff998161850 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff984003520 con 0x7ff998161850 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 328517840 0 0) 0x7ff984003520 con 0x7ff998161850 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff99815dc60 msgr2=0x7ff9980b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 --2- 172.21.15.139:0/3592294087 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff99815dc60 0x7ff9980b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 >> v2:172.21.15.150:3300/0 conn(0x7ff9980d1080 msgr2=0x7ff9980ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 --2- 172.21.15.139:0/3592294087 >> v2:172.21.15.150:3300/0 conn(0x7ff9980d1080 0x7ff9980ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9980ce9b0 con 0x7ff998161850 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7ff994001840 con 0x7ff998161850 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 >> v1:172.21.15.139:6789/0 conn(0x7ff998161850 legacy=0x7ff9980d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 --2- 172.21.15.139:0/3592294087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9840039b0 0x7ff984003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 --2- 172.21.15.139:0/3592294087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff984004800 0x7ff984004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 --2- 172.21.15.139:0/3592294087 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff99815dc60 0x7ff9840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9980d3730 con 0x7ff984004800 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98400b9c0 con 0x7ff9840039b0 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98400bb40 con 0x7ff99815dc60 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff99ce4e640 1 --2- 172.21.15.139:0/3592294087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff984004800 0x7ff984004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff99d64f640 1 --2- 172.21.15.139:0/3592294087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9840039b0 0x7ff984003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.889+0000 7ff99de50640 1 --2- 172.21.15.139:0/3592294087 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff99815dc60 0x7ff9840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.890+0000 7ff99ce4e640 1 -- 172.21.15.139:0/3592294087 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff99815dc60 msgr2=0x7ff9840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.890+0000 7ff99ce4e640 1 --2- 172.21.15.139:0/3592294087 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff99815dc60 0x7ff9840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.890+0000 7ff99ce4e640 1 -- 172.21.15.139:0/3592294087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9840039b0 msgr2=0x7ff984003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.890+0000 7ff99ce4e640 1 --2- 172.21.15.139:0/3592294087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9840039b0 0x7ff984003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.890+0000 7ff99ce4e640 1 -- 172.21.15.139:0/3592294087 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff984003520 con 0x7ff984004800 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.890+0000 7ff99ce4e640 1 --2- 172.21.15.139:0/3592294087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff984004800 0x7ff984004c80 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c3b0 tx=0x7ff984005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c781dbe322895e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.890+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99400c0e0 con 0x7ff984004800 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.890+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff994034070 con 0x7ff984004800 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.890+0000 7ff98ffff640 1 -- 172.21.15.139:0/3592294087 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994005d10 con 0x7ff984004800 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.891+0000 7ff99f8b3640 1 -- 172.21.15.139:0/3592294087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff984004800 msgr2=0x7ff984004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.891+0000 7ff99f8b3640 1 --2- 172.21.15.139:0/3592294087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff984004800 0x7ff984004c80 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c3b0 tx=0x7ff984005830 comp rx=0 tx=0).stop 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.891+0000 7ff99de50640 1 -- 172.21.15.139:0/3592294087 reap_dead start 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.891+0000 7ff99f8b3640 1 -- 172.21.15.139:0/3592294087 shutdown_connections 2024-04-25T10:03:19.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.891+0000 7ff99f8b3640 1 -- 172.21.15.139:0/3592294087 >> 172.21.15.139:0/3592294087 conn(0x7ff9980b5910 msgr2=0x7ff9980b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.891+0000 7ff99f8b3640 1 -- 172.21.15.139:0/3592294087 shutdown_connections 2024-04-25T10:03:19.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.891+0000 7ff99f8b3640 1 -- 172.21.15.139:0/3592294087 wait complete. 2024-04-25T10:03:19.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.892+0000 7ff99f8b3640 1 Processor -- start 2024-04-25T10:03:19.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.892+0000 7ff99f8b3640 1 -- start start 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99f8b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff984004800 0x7ff998159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99f8b3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9980ce0b0 0x7ff99814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99f8b3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99815adf0 0x7ff99815b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99f8b3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9980d1fc0 con 0x7ff99815adf0 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99f8b3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9980cef70 con 0x7ff9980ce0b0 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99f8b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99815e040 con 0x7ff984004800 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99de50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99815adf0 0x7ff99815b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99de50640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99815adf0 0x7ff99815b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39768/0 (socket says 172.21.15.139:39768) 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99de50640 1 -- 172.21.15.139:0/963088895 learned_addr learned my addr 172.21.15.139:0/963088895 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99d64f640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff984004800 0x7ff998159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99ce4e640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9980ce0b0 0x7ff99814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99de50640 1 -- 172.21.15.139:0/963088895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff984004800 msgr2=0x7ff998159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99de50640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff984004800 0x7ff998159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99de50640 1 -- 172.21.15.139:0/963088895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9980ce0b0 msgr2=0x7ff99814dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99de50640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9980ce0b0 0x7ff99814dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99de50640 1 -- 172.21.15.139:0/963088895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9980cbcd0 con 0x7ff99815adf0 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99ce4e640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9980ce0b0 0x7ff99814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff99de50640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99815adf0 0x7ff99815b270 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7ff99808c630 tx=0x7ff99000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.893+0000 7ff98dffb640 1 -- 172.21.15.139:0/963088895 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990015430 con 0x7ff99815adf0 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.894+0000 7ff98dffb640 1 -- 172.21.15.139:0/963088895 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99001d070 con 0x7ff99815adf0 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.894+0000 7ff99f8b3640 1 -- 172.21.15.139:0/963088895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99815a040 con 0x7ff99815adf0 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.894+0000 7ff98dffb640 1 -- 172.21.15.139:0/963088895 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990015830 con 0x7ff99815adf0 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.894+0000 7ff99f8b3640 1 -- 172.21.15.139:0/963088895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9980cc250 con 0x7ff99815adf0 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.895+0000 7ff99f8b3640 1 -- 172.21.15.139:0/963088895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff998074520 con 0x7ff99815adf0 2024-04-25T10:03:19.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.895+0000 7ff98dffb640 1 -- 172.21.15.139:0/963088895 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9900159d0 con 0x7ff99815adf0 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.895+0000 7ff98dffb640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff97c06b4d0 0x7ff97c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.896+0000 7ff98dffb640 1 -- 172.21.15.139:0/963088895 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff99008ae40 con 0x7ff99815adf0 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.896+0000 7ff99d64f640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff97c06b4d0 0x7ff97c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.896+0000 7ff99d64f640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff97c06b4d0 0x7ff97c06d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff9880098a0 tx=0x7ff988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:19.900+0000 7ff98dffb640 1 -- 172.21.15.139:0/963088895 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff990010040 con 0x7ff99815adf0 2024-04-25T10:03:20.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.106+0000 7ff99f8b3640 1 -- 172.21.15.139:0/963088895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff9980d16f0 con 0x7ff99815adf0 2024-04-25T10:03:20.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.106+0000 7ff98dffb640 1 -- 172.21.15.139:0/963088895 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff9900599f0 con 0x7ff99815adf0 2024-04-25T10:03:20.112 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:03:20.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.108+0000 7ff99f8b3640 1 -- 172.21.15.139:0/963088895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff97c06b4d0 msgr2=0x7ff97c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.108+0000 7ff99f8b3640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff97c06b4d0 0x7ff97c06d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff9880098a0 tx=0x7ff988008040 comp rx=0 tx=0).stop 2024-04-25T10:03:20.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.108+0000 7ff99f8b3640 1 -- 172.21.15.139:0/963088895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99815adf0 msgr2=0x7ff99815b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.108+0000 7ff99f8b3640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99815adf0 0x7ff99815b270 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7ff99808c630 tx=0x7ff99000f580 comp rx=0 tx=0).stop 2024-04-25T10:03:20.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.108+0000 7ff99f8b3640 1 -- 172.21.15.139:0/963088895 shutdown_connections 2024-04-25T10:03:20.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.108+0000 7ff99f8b3640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99815adf0 0x7ff99815b270 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.108+0000 7ff99f8b3640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff97c06b4d0 0x7ff97c06d990 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.108+0000 7ff99f8b3640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9980ce0b0 0x7ff99814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.108+0000 7ff99f8b3640 1 --2- 172.21.15.139:0/963088895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff984004800 0x7ff998159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.108+0000 7ff99f8b3640 1 -- 172.21.15.139:0/963088895 >> 172.21.15.139:0/963088895 conn(0x7ff9980b5910 msgr2=0x7ff9980d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.108+0000 7ff99f8b3640 1 -- 172.21.15.139:0/963088895 shutdown_connections 2024-04-25T10:03:20.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20.108+0000 7ff99f8b3640 1 -- 172.21.15.139:0/963088895 wait complete. 2024-04-25T10:03:20.125 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:03:20.125 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:03:20.125 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-04-25T10:03:20.125 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:03:20.125 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:20.125 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:03:20.125 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:03:20.125 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:03:20.125 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:20.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:20.126 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-04-25T10:03:20.140 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:20.141 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42098sec preferred_lft 42098sec 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::74bc:28cf:b166:40c3/64 scope link noprefixroute 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::38d4:6ff:fe6b:56e/64 scope link 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:20.142 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:20.143 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 8a:65:ee:03:05:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:20.143 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::8865:eeff:fe03:56b/64 scope link 2024-04-25T10:03:20.143 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:20.143 INFO:teuthology.orchestra.run.smithi139.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:20.143 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether ce:d6:87:2a:5f:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:20.143 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ccd6:87ff:fe2a:5f18/64 scope link 2024-04-25T10:03:20.143 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:20.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:20.143 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:03:20.203 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T10:03:20.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:20.205 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-04-25T10:03:20.260 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-04-25T10:03:20.260 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:03:20.260 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:03:20.260 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:03:20.260 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:03:20.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:20.260 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec bash -c ' 2024-04-25T10:03:20.261 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-04-25T10:03:20.261 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:03:20.261 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:03:20.261 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:03:20.261 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-04-25T10:03:20.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:20.344 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T10:03:20.398 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:03:20.398 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:03:20.398 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:20.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:20.399 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-04-25T10:03:20.462 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:20.462 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:20.517 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:20.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:20.517 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:20.572 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:03:20.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:20.572 DEBUG:teuthology.orchestra.run.smithi139:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:03:20.652 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:03:20.652 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi139.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: resolved to: "172.21.15.139:3300,172.21.15.150:3300,172.21.15.150:3301" 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: trying mount with new device syntax: 0@922739a8-edcf-4ded-8206-dbdb23e9fea4.cephfs=/ 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi139.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.139:3300/172.21.15.150:3300/172.21.15.150:3301" will pass to kernel 2024-04-25T10:03:20.660 INFO:teuthology.orchestra.run.smithi139.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:03:20.660 INFO:teuthology.orchestra.run.smithi139.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:03:20.660 INFO:teuthology.orchestra.run.smithi139.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:03:20.660 INFO:teuthology.orchestra.run.smithi139.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:03:20.660 INFO:teuthology.orchestra.run.smithi139.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:03:20.663 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:03:20.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:20.663 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:20.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:20.727 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:03:20.783 INFO:teuthology.orchestra.run.smithi139.stdout:client7873 2024-04-25T10:03:20.783 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:20.784 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:20.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:20.839 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:03:20.897 INFO:teuthology.orchestra.run.smithi139.stdout:./ 2024-04-25T10:03:20.897 INFO:teuthology.orchestra.run.smithi139.stdout:./parent 2024-04-25T10:03:20.897 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/flushed 2024-04-25T10:03:20.897 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/flushed/charlie 2024-04-25T10:03:20.897 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/flushed/alpha 2024-04-25T10:03:20.897 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/flushed/bravo 2024-04-25T10:03:20.897 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/unflushed 2024-04-25T10:03:20.897 INFO:teuthology.orchestra.run.smithi139.stdout:./parent/unflushed/jfile 2024-04-25T10:03:20.897 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:20.898 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:20.898 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:20.898 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:20.898 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:03:20.898 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:21.001 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-04-25T10:03:21.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:21.010 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:21.010 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:21.010 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:21.010 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:21.010 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:03:21.010 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:21.115 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627776 2024-04-25T10:03:21.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:21.124 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:21.124 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:21.124 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:21.124 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:21.124 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:03:21.124 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:21.227 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627777 2024-04-25T10:03:21.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:21.236 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:21.236 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:21.236 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:21.236 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:21.236 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:03:21.236 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:21.341 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627780 2024-04-25T10:03:21.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:21.350 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:21.350 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:21.350 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:21.350 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:21.350 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:03:21.350 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:21.455 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627778 2024-04-25T10:03:21.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:21.464 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:21.464 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:21.465 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:21.465 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:21.465 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T10:03:21.465 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:21.569 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627779 2024-04-25T10:03:21.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:21.578 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:21.579 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:21.579 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:21.579 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:21.579 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:03:21.579 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:21.689 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627781 2024-04-25T10:03:21.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:21.698 DEBUG:teuthology.orchestra.run.smithi139:> stdin-killer -- python3 -c ' 2024-04-25T10:03:21.698 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-04-25T10:03:21.698 DEBUG:teuthology.orchestra.run.smithi139:> import stat 2024-04-25T10:03:21.698 DEBUG:teuthology.orchestra.run.smithi139:> 2024-04-25T10:03:21.698 DEBUG:teuthology.orchestra.run.smithi139:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:03:21.698 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-04-25T10:03:21.802 INFO:teuthology.orchestra.run.smithi139.stdout:1099511627782 2024-04-25T10:03:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:21.811 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:03:21.812 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:21.897 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:21.941 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:03:21.941 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:21.941 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:21.957 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:21.957 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:22.014 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:22.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.145+0000 7f16b4e15640 1 Processor -- start 2024-04-25T10:03:22.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.145+0000 7f16b4e15640 1 -- start start 2024-04-25T10:03:22.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16b4e15640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f16b015ae40 0x7f16b00b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16b4e15640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b00ce220 con 0x7f16b00cefe0 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16b4e15640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b00ce500 con 0x7f16b00b0210 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16b4e15640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b00b1070 con 0x7f16b015ae40 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16aed9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f16b00cefe0 0x7f16b006d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53902/0 (socket says 172.21.15.139:53902) 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16aed9d640 1 -- 172.21.15.139:0/3790740139 learned_addr learned my addr 172.21.15.139:0/3790740139 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16a7fff640 1 --2- 172.21.15.139:0/3790740139 >> v2:172.21.15.150:3300/0 conn(0x7f16b015ae40 0x7f16b00b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624585415 0 0) 0x7f16b00ce220 con 0x7f16b00cefe0 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f169c003440 con 0x7f16b00cefe0 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1409158627 0 0) 0x7f169c003440 con 0x7f16b00cefe0 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 >> v2:172.21.15.150:3300/0 conn(0x7f16b015ae40 msgr2=0x7f16b00b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 --2- 172.21.15.139:0/3790740139 >> v2:172.21.15.150:3300/0 conn(0x7f16b015ae40 0x7f16b00b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 >> v1:172.21.15.150:6789/0 conn(0x7f16b00b0210 legacy=0x7f16b00b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b00ce7e0 con 0x7f16b00cefe0 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f16a80019f0 con 0x7f16b00cefe0 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 >> v1:172.21.15.139:6789/0 conn(0x7f16b00cefe0 legacy=0x7f16b006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 --2- 172.21.15.139:0/3790740139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f169c003aa0 0x7f169c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 --2- 172.21.15.139:0/3790740139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f169c008850 0x7f169c00acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 --2- 172.21.15.139:0/3790740139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 0x7f169c00d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.146+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b00b1070 con 0x7f169c00b4c0 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.147+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c00e260 con 0x7f169c008850 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.147+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c00e460 con 0x7f169c003aa0 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.147+0000 7f16aed9d640 1 --2- 172.21.15.139:0/3790740139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 0x7f169c00d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.147+0000 7f16a7fff640 1 --2- 172.21.15.139:0/3790740139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f169c008850 0x7f169c00acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.147+0000 7f16ae59c640 1 --2- 172.21.15.139:0/3790740139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f169c003aa0 0x7f169c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.147+0000 7f16aed9d640 1 -- 172.21.15.139:0/3790740139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f169c003aa0 msgr2=0x7f169c003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.147+0000 7f16aed9d640 1 --2- 172.21.15.139:0/3790740139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f169c003aa0 0x7f169c003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.147+0000 7f16aed9d640 1 -- 172.21.15.139:0/3790740139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f169c008850 msgr2=0x7f169c00acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.147+0000 7f16aed9d640 1 --2- 172.21.15.139:0/3790740139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f169c008850 0x7f169c00acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.147+0000 7f16aed9d640 1 -- 172.21.15.139:0/3790740139 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c003440 con 0x7f169c00b4c0 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.147+0000 7f16aed9d640 1 -- 172.21.15.139:0/3790740139 reap_dead start 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.148+0000 7f16aed9d640 1 --2- 172.21.15.139:0/3790740139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 0x7f169c00d980 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f16a800a860 tx=0x7f169c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60e4d7b51cb0e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.148+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a800a480 con 0x7f169c00b4c0 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.148+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a802f070 con 0x7f169c00b4c0 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.148+0000 7f16add9b640 1 -- 172.21.15.139:0/3790740139 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a802bcc0 con 0x7f169c00b4c0 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.148+0000 7f16b4e15640 1 -- 172.21.15.139:0/3790740139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 msgr2=0x7f169c00d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.148+0000 7f16b4e15640 1 --2- 172.21.15.139:0/3790740139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 0x7f169c00d980 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f16a800a860 tx=0x7f169c003730 comp rx=0 tx=0).stop 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.148+0000 7f16b4e15640 1 -- 172.21.15.139:0/3790740139 shutdown_connections 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.148+0000 7f16b4e15640 1 --2- 172.21.15.139:0/3790740139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 0x7f169c00d980 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.148+0000 7f16b4e15640 1 -- 172.21.15.139:0/3790740139 >> 172.21.15.139:0/3790740139 conn(0x7f16b0066f20 msgr2=0x7f16b006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.148+0000 7f16b4e15640 1 -- 172.21.15.139:0/3790740139 shutdown_connections 2024-04-25T10:03:22.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.148+0000 7f16b4e15640 1 -- 172.21.15.139:0/3790740139 wait complete. 2024-04-25T10:03:22.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.150+0000 7f16b4e15640 1 Processor -- start 2024-04-25T10:03:22.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.150+0000 7f16b4e15640 1 -- start start 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.150+0000 7f16b4e15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 0x7f16b013f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.150+0000 7f16b4e15640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16b00a84b0 0x7f16b00a8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.150+0000 7f16b4e15640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16b0151670 0x7f16b0151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.150+0000 7f16b4e15640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b015b3e0 con 0x7f169c00b4c0 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.150+0000 7f16b4e15640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b00ceb50 con 0x7f16b0151670 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.150+0000 7f16b4e15640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b00cf470 con 0x7f16b00a84b0 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.150+0000 7f16ae59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 0x7f16b013f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.150+0000 7f16ae59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 0x7f16b013f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39784/0 (socket says 172.21.15.139:39784) 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.150+0000 7f16ae59c640 1 -- 172.21.15.139:0/4009285288 learned_addr learned my addr 172.21.15.139:0/4009285288 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16aed9d640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16b0151670 0x7f16b0151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16a7fff640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16b00a84b0 0x7f16b00a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16ae59c640 1 -- 172.21.15.139:0/4009285288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16b00a84b0 msgr2=0x7f16b00a8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16ae59c640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16b00a84b0 0x7f16b00a8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16ae59c640 1 -- 172.21.15.139:0/4009285288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16b0151670 msgr2=0x7f16b0151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16ae59c640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16b0151670 0x7f16b0151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16ae59c640 1 -- 172.21.15.139:0/4009285288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b0152460 con 0x7f169c00b4c0 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16ae59c640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 0x7f16b013f7a0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f16a000f5f0 tx=0x7f16a000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16a6ffd640 1 -- 172.21.15.139:0/4009285288 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a00052e0 con 0x7f169c00b4c0 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16a6ffd640 1 -- 172.21.15.139:0/4009285288 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a0018070 con 0x7f169c00b4c0 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16b4e15640 1 -- 172.21.15.139:0/4009285288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b0140150 con 0x7f169c00b4c0 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16a6ffd640 1 -- 172.21.15.139:0/4009285288 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0003810 con 0x7f169c00b4c0 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.151+0000 7f16b4e15640 1 -- 172.21.15.139:0/4009285288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b0140710 con 0x7f169c00b4c0 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.152+0000 7f16b4e15640 1 -- 172.21.15.139:0/4009285288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b006baf0 con 0x7f169c00b4c0 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.152+0000 7f16a6ffd640 1 -- 172.21.15.139:0/4009285288 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16a00039b0 con 0x7f169c00b4c0 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.153+0000 7f16a6ffd640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f169006b500 0x7f169006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.153+0000 7f16a6ffd640 1 -- 172.21.15.139:0/4009285288 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f16a008a1d0 con 0x7f169c00b4c0 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.153+0000 7f16a7fff640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f169006b500 0x7f169006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.153+0000 7f16a7fff640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f169006b500 0x7f169006d9c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f16b0157ab0 tx=0x7f1698008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.157+0000 7f16a6ffd640 1 -- 172.21.15.139:0/4009285288 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16a0058ea0 con 0x7f169c00b4c0 2024-04-25T10:03:22.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.330+0000 7f16b4e15640 1 -- 172.21.15.139:0/4009285288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16b0074520 con 0x7f169c00b4c0 2024-04-25T10:03:22.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.332+0000 7f16a6ffd640 1 -- 172.21.15.139:0/4009285288 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17263 (secure 0 0 0) 0x7f16a00587a0 con 0x7f169c00b4c0 2024-04-25T10:03:22.336 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:22.337 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":122,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:03:13.364415+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:48.574891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:49.293101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6836/3478297010":"2024-04-26T10:03:13.364392+0000","172.21.15.139:6834/3478297010":"2024-04-26T10:03:13.364392+0000","172.21.15.150:6835/1377607596":"2024-04-26T10:03:03.005923+0000","172.21.15.150:6833/1377607596":"2024-04-26T10:03:03.005923+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:22.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.334+0000 7f16b4e15640 1 -- 172.21.15.139:0/4009285288 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f169006b500 msgr2=0x7f169006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.334+0000 7f16b4e15640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f169006b500 0x7f169006d9c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f16b0157ab0 tx=0x7f1698008040 comp rx=0 tx=0).stop 2024-04-25T10:03:22.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.334+0000 7f16b4e15640 1 -- 172.21.15.139:0/4009285288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 msgr2=0x7f16b013f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.334+0000 7f16b4e15640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 0x7f16b013f7a0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f16a000f5f0 tx=0x7f16a000fab0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.334+0000 7f16b4e15640 1 -- 172.21.15.139:0/4009285288 shutdown_connections 2024-04-25T10:03:22.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.334+0000 7f16b4e15640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f169006b500 0x7f169006d9c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.334+0000 7f16b4e15640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16b0151670 0x7f16b0151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.334+0000 7f16b4e15640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16b00a84b0 0x7f16b00a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.334+0000 7f16b4e15640 1 --2- 172.21.15.139:0/4009285288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f169c00b4c0 0x7f16b013f7a0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.334+0000 7f16b4e15640 1 -- 172.21.15.139:0/4009285288 >> 172.21.15.139:0/4009285288 conn(0x7f16b0066f20 msgr2=0x7f16b006b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.334+0000 7f16b4e15640 1 -- 172.21.15.139:0/4009285288 shutdown_connections 2024-04-25T10:03:22.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.334+0000 7f16b4e15640 1 -- 172.21.15.139:0/4009285288 wait complete. 2024-04-25T10:03:22.349 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:48.574891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:49.293101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:22.350 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:22.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.447+0000 7fdaf3e6b640 1 Processor -- start 2024-04-25T10:03:22.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.447+0000 7fdaf3e6b640 1 -- start start 2024-04-25T10:03:22.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf3e6b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdaec0cd610 0x7fdaec0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf3e6b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdaec09ad40 0x7fdaec0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf3e6b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaec099ca0 con 0x7fdaec1618c0 2024-04-25T10:03:22.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf3e6b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaec099a50 con 0x7fdaec0cd610 2024-04-25T10:03:22.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf3e6b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaec099f80 con 0x7fdaec09ad40 2024-04-25T10:03:22.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf1c07640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdaec0cd610 0x7fdaec0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf1c07640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdaec0cd610 0x7fdaec0d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39786/0 (socket says 172.21.15.139:39786) 2024-04-25T10:03:22.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf1c07640 1 -- 172.21.15.139:0/2010166585 learned_addr learned my addr 172.21.15.139:0/2010166585 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:22.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf2408640 1 --1- 172.21.15.139:0/2010166585 >> v1:172.21.15.139:6789/0 conn(0x7fdaec1618c0 0x7fdaec06be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53916/0 (socket says 172.21.15.139:53916) 2024-04-25T10:03:22.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf1c07640 1 -- 172.21.15.139:0/2010166585 >> v2:172.21.15.150:3300/0 conn(0x7fdaec09ad40 msgr2=0x7fdaec0d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf1406640 1 --2- 172.21.15.139:0/2010166585 >> v2:172.21.15.150:3300/0 conn(0x7fdaec09ad40 0x7fdaec0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf0c05640 1 -- 172.21.15.139:0/2010166585 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807887050 0 0) 0x7fdaec099ca0 con 0x7fdaec1618c0 2024-04-25T10:03:22.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf1c07640 1 --2- 172.21.15.139:0/2010166585 >> v2:172.21.15.150:3300/0 conn(0x7fdaec09ad40 0x7fdaec0d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf1c07640 1 -- 172.21.15.139:0/2010166585 >> v1:172.21.15.139:6789/0 conn(0x7fdaec1618c0 legacy=0x7fdaec06be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.448+0000 7fdaf1c07640 1 -- 172.21.15.139:0/2010166585 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec09a260 con 0x7fdaec0cd610 2024-04-25T10:03:22.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.449+0000 7fdaf1c07640 1 --2- 172.21.15.139:0/2010166585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec0cd610 0x7fdaec0d0370 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0725c0 tx=0x7fdae802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8abdc85f392672 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.449+0000 7fdaf0c05640 1 -- 172.21.15.139:0/2010166585 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8037070 con 0x7fdaec0cd610 2024-04-25T10:03:22.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.449+0000 7fdaf0c05640 1 -- 172.21.15.139:0/2010166585 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae802ae50 con 0x7fdaec0cd610 2024-04-25T10:03:22.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.449+0000 7fdaf0c05640 1 -- 172.21.15.139:0/2010166585 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8032860 con 0x7fdaec0cd610 2024-04-25T10:03:22.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.449+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/2010166585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec0cd610 msgr2=0x7fdaec0d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.449+0000 7fdaf3e6b640 1 --2- 172.21.15.139:0/2010166585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec0cd610 0x7fdaec0d0370 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0725c0 tx=0x7fdae802a300 comp rx=0 tx=0).stop 2024-04-25T10:03:22.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.449+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/2010166585 shutdown_connections 2024-04-25T10:03:22.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.450+0000 7fdaf3e6b640 1 --2- 172.21.15.139:0/2010166585 >> v2:172.21.15.150:3300/0 conn(0x7fdaec09ad40 0x7fdaec0d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.450+0000 7fdaf3e6b640 1 --2- 172.21.15.139:0/2010166585 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec0cd610 0x7fdaec0d0370 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.450+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/2010166585 >> 172.21.15.139:0/2010166585 conn(0x7fdaec0b31f0 msgr2=0x7fdaec0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.450+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/2010166585 shutdown_connections 2024-04-25T10:03:22.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.450+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/2010166585 wait complete. 2024-04-25T10:03:22.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.451+0000 7fdaf3e6b640 1 Processor -- start 2024-04-25T10:03:22.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.451+0000 7fdaf3e6b640 1 -- start start 2024-04-25T10:03:22.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.451+0000 7fdaf3e6b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaec09ad40 0x7fdaec0c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.451+0000 7fdaf3e6b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec0cd610 0x7fdaec15ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.451+0000 7fdaf3e6b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdaec1618c0 0x7fdaec1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.451+0000 7fdaf3e6b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec0cd9f0 con 0x7fdaec0cd610 2024-04-25T10:03:22.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.451+0000 7fdaf3e6b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec09a820 con 0x7fdaec09ad40 2024-04-25T10:03:22.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.451+0000 7fdaf3e6b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec0cdb70 con 0x7fdaec1618c0 2024-04-25T10:03:22.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf1406640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec0cd610 0x7fdaec15ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf1406640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec0cd610 0x7fdaec15ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39788/0 (socket says 172.21.15.139:39788) 2024-04-25T10:03:22.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf1406640 1 -- 172.21.15.139:0/1934951637 learned_addr learned my addr 172.21.15.139:0/1934951637 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:22.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf2408640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdaec1618c0 0x7fdaec1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf1406640 1 -- 172.21.15.139:0/1934951637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdaec1618c0 msgr2=0x7fdaec1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf1c07640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaec09ad40 0x7fdaec0c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf1406640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdaec1618c0 0x7fdaec1604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf1406640 1 -- 172.21.15.139:0/1934951637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaec09ad40 msgr2=0x7fdaec0c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf1406640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaec09ad40 0x7fdaec0c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf1406640 1 -- 172.21.15.139:0/1934951637 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec15b5f0 con 0x7fdaec0cd610 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf1406640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec0cd610 0x7fdaec15ff70 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00c7f0 tx=0x7fdadc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdae27fc640 1 -- 172.21.15.139:0/1934951637 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc015070 con 0x7fdaec0cd610 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdae27fc640 1 -- 172.21.15.139:0/1934951637 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc004240 con 0x7fdaec0cd610 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdae27fc640 1 -- 172.21.15.139:0/1934951637 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc007620 con 0x7fdaec0cd610 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/1934951637 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaec15c370 con 0x7fdaec0cd610 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.452+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/1934951637 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaec15bb70 con 0x7fdaec0cd610 2024-04-25T10:03:22.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.453+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/1934951637 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaec074520 con 0x7fdaec0cd610 2024-04-25T10:03:22.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.454+0000 7fdae27fc640 1 -- 172.21.15.139:0/1934951637 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdadc0079d0 con 0x7fdaec0cd610 2024-04-25T10:03:22.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.454+0000 7fdae27fc640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdad006b4d0 0x7fdad006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.454+0000 7fdae27fc640 1 -- 172.21.15.139:0/1934951637 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fdadc08a410 con 0x7fdaec0cd610 2024-04-25T10:03:22.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.454+0000 7fdaf1c07640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdad006b4d0 0x7fdad006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.454+0000 7fdaf1c07640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdad006b4d0 0x7fdad006d990 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdae8034000 tx=0x7fdae802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.458+0000 7fdae27fc640 1 -- 172.21.15.139:0/1934951637 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdadc0590e0 con 0x7fdaec0cd610 2024-04-25T10:03:22.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.626+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/1934951637 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdaec0b3f70 con 0x7fdaec0cd610 2024-04-25T10:03:22.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.627+0000 7fdae27fc640 1 -- 172.21.15.139:0/1934951637 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdadc0589e0 con 0x7fdaec0cd610 2024-04-25T10:03:22.632 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:03:22.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.629+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/1934951637 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdad006b4d0 msgr2=0x7fdad006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.629+0000 7fdaf3e6b640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdad006b4d0 0x7fdad006d990 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdae8034000 tx=0x7fdae802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:22.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.629+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/1934951637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec0cd610 msgr2=0x7fdaec15ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.629+0000 7fdaf3e6b640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec0cd610 0x7fdaec15ff70 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00c7f0 tx=0x7fdadc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.629+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/1934951637 shutdown_connections 2024-04-25T10:03:22.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.629+0000 7fdaf3e6b640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdaec0cd610 0x7fdaec15ff70 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.629+0000 7fdaf3e6b640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdad006b4d0 0x7fdad006d990 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.629+0000 7fdaf3e6b640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdaec1618c0 0x7fdaec1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.629+0000 7fdaf3e6b640 1 --2- 172.21.15.139:0/1934951637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaec09ad40 0x7fdaec0c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.629+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/1934951637 >> 172.21.15.139:0/1934951637 conn(0x7fdaec0b31f0 msgr2=0x7fdaec070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.629+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/1934951637 shutdown_connections 2024-04-25T10:03:22.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.629+0000 7fdaf3e6b640 1 -- 172.21.15.139:0/1934951637 wait complete. 2024-04-25T10:03:22.645 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.739+0000 7fe043bd5640 1 Processor -- start 2024-04-25T10:03:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.740+0000 7fe043bd5640 1 -- start start 2024-04-25T10:03:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.740+0000 7fe043bd5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe03c071a50 0x7fe03c071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.741+0000 7fe043bd5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c06b980 con 0x7fe03c0726c0 2024-04-25T10:03:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.741+0000 7fe043bd5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c072370 con 0x7fe03c0cf0e0 2024-04-25T10:03:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.741+0000 7fe043bd5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03c15c500 con 0x7fe03c071a50 2024-04-25T10:03:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.741+0000 7fe041971640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe03c071a50 0x7fe03c071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.741+0000 7fe041170640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe03c0726c0 0x7fe03c06ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53920/0 (socket says 172.21.15.139:53920) 2024-04-25T10:03:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.741+0000 7fe041170640 1 -- 172.21.15.139:0/4136792993 learned_addr learned my addr 172.21.15.139:0/4136792993 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.741+0000 7fe041971640 1 --2- 172.21.15.139:0/4136792993 >> v2:172.21.15.139:3300/0 conn(0x7fe03c071a50 0x7fe03c071e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39802/0 (socket says 172.21.15.139:39802) 2024-04-25T10:03:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.741+0000 7fe041971640 1 -- 172.21.15.139:0/4136792993 >> v1:172.21.15.150:6789/0 conn(0x7fe03c0cf0e0 legacy=0x7fe03c06b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.741+0000 7fe041971640 1 -- 172.21.15.139:0/4136792993 >> v1:172.21.15.139:6789/0 conn(0x7fe03c0726c0 legacy=0x7fe03c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.741+0000 7fe04096f640 1 -- 172.21.15.139:0/4136792993 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139075772 0 0) 0x7fe03c06b980 con 0x7fe03c0726c0 2024-04-25T10:03:22.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.741+0000 7fe041971640 1 -- 172.21.15.139:0/4136792993 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c15c7e0 con 0x7fe03c071a50 2024-04-25T10:03:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.742+0000 7fe041971640 1 --2- 172.21.15.139:0/4136792993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe03c071a50 0x7fe03c071e30 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fe034009d90 tx=0x7fe03402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fdec936d7738dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.742+0000 7fe04096f640 1 -- 172.21.15.139:0/4136792993 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034007740 con 0x7fe03c071a50 2024-04-25T10:03:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.742+0000 7fe04096f640 1 -- 172.21.15.139:0/4136792993 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0340078e0 con 0x7fe03c071a50 2024-04-25T10:03:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.742+0000 7fe04096f640 1 -- 172.21.15.139:0/4136792993 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034007cb0 con 0x7fe03c071a50 2024-04-25T10:03:22.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.742+0000 7fe043bd5640 1 -- 172.21.15.139:0/4136792993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe03c071a50 msgr2=0x7fe03c071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.742+0000 7fe043bd5640 1 --2- 172.21.15.139:0/4136792993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe03c071a50 0x7fe03c071e30 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fe034009d90 tx=0x7fe03402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.742+0000 7fe043bd5640 1 -- 172.21.15.139:0/4136792993 shutdown_connections 2024-04-25T10:03:22.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.742+0000 7fe043bd5640 1 --2- 172.21.15.139:0/4136792993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe03c071a50 0x7fe03c071e30 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.742+0000 7fe043bd5640 1 -- 172.21.15.139:0/4136792993 >> 172.21.15.139:0/4136792993 conn(0x7fe03c0cfd80 msgr2=0x7fe03c0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.742+0000 7fe043bd5640 1 -- 172.21.15.139:0/4136792993 shutdown_connections 2024-04-25T10:03:22.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.742+0000 7fe043bd5640 1 -- 172.21.15.139:0/4136792993 wait complete. 2024-04-25T10:03:22.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.743+0000 7fe043bd5640 1 Processor -- start 2024-04-25T10:03:22.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.744+0000 7fe043bd5640 1 -- start start 2024-04-25T10:03:22.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.744+0000 7fe043bd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe03c071a50 0x7fe03c06f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.744+0000 7fe043bd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe03c0cf0e0 0x7fe03c154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.744+0000 7fe043bd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe03c06d7d0 0x7fe03c06dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.744+0000 7fe043bd5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c156280 con 0x7fe03c071a50 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.744+0000 7fe043bd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c15cda0 con 0x7fe03c06d7d0 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.744+0000 7fe043bd5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c156400 con 0x7fe03c0cf0e0 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.744+0000 7fe041971640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe03c071a50 0x7fe03c06f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.744+0000 7fe041971640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe03c071a50 0x7fe03c06f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39814/0 (socket says 172.21.15.139:39814) 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.744+0000 7fe041971640 1 -- 172.21.15.139:0/2138552279 learned_addr learned my addr 172.21.15.139:0/2138552279 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe041170640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe03c0cf0e0 0x7fe03c154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe041971640 1 -- 172.21.15.139:0/2138552279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe03c0cf0e0 msgr2=0x7fe03c154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe042172640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe03c06d7d0 0x7fe03c06dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe041971640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe03c0cf0e0 0x7fe03c154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe041971640 1 -- 172.21.15.139:0/2138552279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe03c06d7d0 msgr2=0x7fe03c06dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe041971640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe03c06d7d0 0x7fe03c06dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe041971640 1 -- 172.21.15.139:0/2138552279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c06e290 con 0x7fe03c071a50 2024-04-25T10:03:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe041170640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe03c0cf0e0 0x7fe03c154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe042172640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe03c06d7d0 0x7fe03c06dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe041971640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe03c071a50 0x7fe03c06f330 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fe034009d90 tx=0x7fe034004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe0327fc640 1 -- 172.21.15.139:0/2138552279 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034040070 con 0x7fe03c071a50 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe0327fc640 1 -- 172.21.15.139:0/2138552279 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe034031c40 con 0x7fe03c071a50 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe043bd5640 1 -- 172.21.15.139:0/2138552279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c06e570 con 0x7fe03c071a50 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe0327fc640 1 -- 172.21.15.139:0/2138552279 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03403c420 con 0x7fe03c071a50 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.745+0000 7fe043bd5640 1 -- 172.21.15.139:0/2138552279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c06eb30 con 0x7fe03c071a50 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.746+0000 7fe043bd5640 1 -- 172.21.15.139:0/2138552279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03c074520 con 0x7fe03c071a50 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.746+0000 7fe0327fc640 1 -- 172.21.15.139:0/2138552279 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe034039040 con 0x7fe03c071a50 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.747+0000 7fe0327fc640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe02006b4d0 0x7fe02006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.747+0000 7fe0327fc640 1 -- 172.21.15.139:0/2138552279 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fe0340abaf0 con 0x7fe03c071a50 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.747+0000 7fe041170640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe02006b4d0 0x7fe02006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.747+0000 7fe041170640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe02006b4d0 0x7fe02006d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0ccd10 tx=0x7fe02c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.751+0000 7fe0327fc640 1 -- 172.21.15.139:0/2138552279 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe03407a730 con 0x7fe03c071a50 2024-04-25T10:03:22.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.920+0000 7fe043bd5640 1 -- 172.21.15.139:0/2138552279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe03c067770 con 0x7fe03c071a50 2024-04-25T10:03:22.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.920+0000 7fe0327fc640 1 -- 172.21.15.139:0/2138552279 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe03407a030 con 0x7fe03c071a50 2024-04-25T10:03:22.925 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T10:03:22.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.922+0000 7fe043bd5640 1 -- 172.21.15.139:0/2138552279 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe02006b4d0 msgr2=0x7fe02006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.922+0000 7fe043bd5640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe02006b4d0 0x7fe02006d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0ccd10 tx=0x7fe02c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:22.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.922+0000 7fe043bd5640 1 -- 172.21.15.139:0/2138552279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe03c071a50 msgr2=0x7fe03c06f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.922+0000 7fe043bd5640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe03c071a50 0x7fe03c06f330 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fe034009d90 tx=0x7fe034004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.922+0000 7fe043bd5640 1 -- 172.21.15.139:0/2138552279 shutdown_connections 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.922+0000 7fe043bd5640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fe02006b4d0 0x7fe02006d990 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.922+0000 7fe043bd5640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe03c06d7d0 0x7fe03c06dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.922+0000 7fe043bd5640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe03c0cf0e0 0x7fe03c154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.922+0000 7fe043bd5640 1 --2- 172.21.15.139:0/2138552279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe03c071a50 0x7fe03c06f330 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.922+0000 7fe043bd5640 1 -- 172.21.15.139:0/2138552279 >> 172.21.15.139:0/2138552279 conn(0x7fe03c0cfd80 msgr2=0x7fe03c15dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.923+0000 7fe043bd5640 1 -- 172.21.15.139:0/2138552279 shutdown_connections 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:22.923+0000 7fe043bd5640 1 -- 172.21.15.139:0/2138552279 wait complete. 2024-04-25T10:03:22.938 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:23.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.036+0000 7fd64b615640 1 Processor -- start 2024-04-25T10:03:23.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.036+0000 7fd64b615640 1 -- start start 2024-04-25T10:03:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.037+0000 7fd64b615640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd6440af8a0 0x7fd6440b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.037+0000 7fd64b615640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd64406b700 0x7fd64406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.037+0000 7fd64b615640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6440c6cc0 con 0x7fd6440afc80 2024-04-25T10:03:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.037+0000 7fd64b615640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6440c6fa0 con 0x7fd6440af8a0 2024-04-25T10:03:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.037+0000 7fd64b615640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440c7280 con 0x7fd64406b700 2024-04-25T10:03:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.037+0000 7fd6493b1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd6440af8a0 0x7fd6440b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.037+0000 7fd6493b1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd6440af8a0 0x7fd6440b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39818/0 (socket says 172.21.15.139:39818) 2024-04-25T10:03:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.037+0000 7fd6493b1640 1 -- 172.21.15.139:0/124543184 learned_addr learned my addr 172.21.15.139:0/124543184 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:23.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.037+0000 7fd649bb2640 1 --2- 172.21.15.139:0/124543184 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd64406b700 0x7fd64406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.038+0000 7fd6493b1640 1 -- 172.21.15.139:0/124543184 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd64406b700 msgr2=0x7fd64406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.038+0000 7fd6493b1640 1 --2- 172.21.15.139:0/124543184 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd64406b700 0x7fd64406ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.038+0000 7fd6493b1640 1 -- 172.21.15.139:0/124543184 >> v1:172.21.15.150:6789/0 conn(0x7fd6440afc80 legacy=0x7fd6440cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.038+0000 7fd6493b1640 1 -- 172.21.15.139:0/124543184 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6440cd090 con 0x7fd6440af8a0 2024-04-25T10:03:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.038+0000 7fd6493b1640 1 --2- 172.21.15.139:0/124543184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440af8a0 0x7fd6440b2d50 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fd640009870 tx=0x7fd64002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db1736ba35c63cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.038+0000 7fd63bfff640 1 -- 172.21.15.139:0/124543184 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640037070 con 0x7fd6440af8a0 2024-04-25T10:03:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.038+0000 7fd63bfff640 1 -- 172.21.15.139:0/124543184 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd640032440 con 0x7fd6440af8a0 2024-04-25T10:03:23.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.038+0000 7fd63bfff640 1 -- 172.21.15.139:0/124543184 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640032810 con 0x7fd6440af8a0 2024-04-25T10:03:23.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.039+0000 7fd64b615640 1 -- 172.21.15.139:0/124543184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440af8a0 msgr2=0x7fd6440b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.039+0000 7fd64b615640 1 --2- 172.21.15.139:0/124543184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440af8a0 0x7fd6440b2d50 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fd640009870 tx=0x7fd64002a270 comp rx=0 tx=0).stop 2024-04-25T10:03:23.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.039+0000 7fd64b615640 1 -- 172.21.15.139:0/124543184 shutdown_connections 2024-04-25T10:03:23.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.039+0000 7fd64b615640 1 --2- 172.21.15.139:0/124543184 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd64406b700 0x7fd64406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.039+0000 7fd64b615640 1 --2- 172.21.15.139:0/124543184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440af8a0 0x7fd6440b2d50 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.039+0000 7fd64b615640 1 -- 172.21.15.139:0/124543184 >> 172.21.15.139:0/124543184 conn(0x7fd6440b2230 msgr2=0x7fd6440b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.039+0000 7fd64b615640 1 -- 172.21.15.139:0/124543184 shutdown_connections 2024-04-25T10:03:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.039+0000 7fd64b615640 1 -- 172.21.15.139:0/124543184 wait complete. 2024-04-25T10:03:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.040+0000 7fd64b615640 1 Processor -- start 2024-04-25T10:03:23.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.040+0000 7fd64b615640 1 -- start start 2024-04-25T10:03:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd64b615640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd64406b700 0x7fd644151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd64b615640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440af8a0 0x7fd64409ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd64b615640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6440afc80 0x7fd6440ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd64b615640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64406b220 con 0x7fd6440af8a0 2024-04-25T10:03:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd64b615640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64406b0a0 con 0x7fd64406b700 2024-04-25T10:03:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd64b615640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64406b3a0 con 0x7fd6440afc80 2024-04-25T10:03:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd648bb0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440af8a0 0x7fd64409ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd648bb0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440af8a0 0x7fd64409ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39826/0 (socket says 172.21.15.139:39826) 2024-04-25T10:03:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd648bb0640 1 -- 172.21.15.139:0/2351582213 learned_addr learned my addr 172.21.15.139:0/2351582213 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd6493b1640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd64406b700 0x7fd644151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd649bb2640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6440afc80 0x7fd6440ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd648bb0640 1 -- 172.21.15.139:0/2351582213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6440afc80 msgr2=0x7fd6440ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd648bb0640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6440afc80 0x7fd6440ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd648bb0640 1 -- 172.21.15.139:0/2351582213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd64406b700 msgr2=0x7fd644151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd648bb0640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd64406b700 0x7fd644151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd648bb0640 1 -- 172.21.15.139:0/2351582213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64409c8c0 con 0x7fd6440af8a0 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd6493b1640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd64406b700 0x7fd644151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.041+0000 7fd648bb0640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440af8a0 0x7fd64409ded0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fd63400c450 tx=0x7fd63400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.042+0000 7fd639ffb640 1 -- 172.21.15.139:0/2351582213 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340052d0 con 0x7fd6440af8a0 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.042+0000 7fd639ffb640 1 -- 172.21.15.139:0/2351582213 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634005470 con 0x7fd6440af8a0 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.042+0000 7fd64b615640 1 -- 172.21.15.139:0/2351582213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64409cba0 con 0x7fd6440af8a0 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.042+0000 7fd639ffb640 1 -- 172.21.15.139:0/2351582213 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634012870 con 0x7fd6440af8a0 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.042+0000 7fd64b615640 1 -- 172.21.15.139:0/2351582213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64409d160 con 0x7fd6440af8a0 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.043+0000 7fd64b615640 1 -- 172.21.15.139:0/2351582213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd644074520 con 0x7fd6440af8a0 2024-04-25T10:03:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.043+0000 7fd639ffb640 1 -- 172.21.15.139:0/2351582213 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd634012a10 con 0x7fd6440af8a0 2024-04-25T10:03:23.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.044+0000 7fd639ffb640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd62806b4d0 0x7fd62806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.044+0000 7fd6493b1640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd62806b4d0 0x7fd62806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.044+0000 7fd639ffb640 1 -- 172.21.15.139:0/2351582213 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd63408a080 con 0x7fd6440af8a0 2024-04-25T10:03:23.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.044+0000 7fd6493b1640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd62806b4d0 0x7fd62806d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd640034000 tx=0x7fd64002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.048+0000 7fd639ffb640 1 -- 172.21.15.139:0/2351582213 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd634058cc0 con 0x7fd6440af8a0 2024-04-25T10:03:23.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.218+0000 7fd64b615640 1 -- 172.21.15.139:0/2351582213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd6440b1be0 con 0x7fd6440af8a0 2024-04-25T10:03:23.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.218+0000 7fd639ffb640 1 -- 172.21.15.139:0/2351582213 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd6340585c0 con 0x7fd6440af8a0 2024-04-25T10:03:23.223 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T10:03:23.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.220+0000 7fd64b615640 1 -- 172.21.15.139:0/2351582213 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd62806b4d0 msgr2=0x7fd62806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.220+0000 7fd64b615640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd62806b4d0 0x7fd62806d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd640034000 tx=0x7fd64002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:23.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.220+0000 7fd64b615640 1 -- 172.21.15.139:0/2351582213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440af8a0 msgr2=0x7fd64409ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.220+0000 7fd64b615640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440af8a0 0x7fd64409ded0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fd63400c450 tx=0x7fd63400c910 comp rx=0 tx=0).stop 2024-04-25T10:03:23.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.220+0000 7fd64b615640 1 -- 172.21.15.139:0/2351582213 shutdown_connections 2024-04-25T10:03:23.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.220+0000 7fd64b615640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440af8a0 0x7fd64409ded0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.220+0000 7fd64b615640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd62806b4d0 0x7fd62806d990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.220+0000 7fd64b615640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6440afc80 0x7fd6440ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.220+0000 7fd64b615640 1 --2- 172.21.15.139:0/2351582213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd64406b700 0x7fd644151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.220+0000 7fd64b615640 1 -- 172.21.15.139:0/2351582213 >> 172.21.15.139:0/2351582213 conn(0x7fd6440b2230 msgr2=0x7fd6440aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.220+0000 7fd64b615640 1 -- 172.21.15.139:0/2351582213 shutdown_connections 2024-04-25T10:03:23.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.220+0000 7fd64b615640 1 -- 172.21.15.139:0/2351582213 wait complete. 2024-04-25T10:03:23.236 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:23.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.329+0000 7fe01c258640 1 Processor -- start 2024-04-25T10:03:23.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.329+0000 7fe01c258640 1 -- start start 2024-04-25T10:03:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.330+0000 7fe01c258640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe01415dc90 0x7fe0140ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.330+0000 7fe01c258640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01406f300 0x7fe0140b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.330+0000 7fe01c258640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0140cf980 con 0x7fe0140cf3e0 2024-04-25T10:03:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.330+0000 7fe01c258640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01406ab60 con 0x7fe01415dc90 2024-04-25T10:03:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.330+0000 7fe01c258640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01406ae40 con 0x7fe01406f300 2024-04-25T10:03:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.330+0000 7fe0197f3640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe0140cf3e0 0x7fe01406ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53934/0 (socket says 172.21.15.139:53934) 2024-04-25T10:03:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.330+0000 7fe0197f3640 1 -- 172.21.15.139:0/3116752981 learned_addr learned my addr 172.21.15.139:0/3116752981 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.330+0000 7fe019ff4640 1 --2- 172.21.15.139:0/3116752981 >> v2:172.21.15.150:3300/0 conn(0x7fe01415dc90 0x7fe0140ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.330+0000 7fe01a7f5640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01406f300 0x7fe0140b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.330+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331263377 0 0) 0x7fe0140cf980 con 0x7fe0140cf3e0 2024-04-25T10:03:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe000003520 con 0x7fe0140cf3e0 2024-04-25T10:03:23.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2072663235 0 0) 0x7fe000003520 con 0x7fe0140cf3e0 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01406f300 msgr2=0x7fe0140b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01406f300 0x7fe0140b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 >> v2:172.21.15.150:3300/0 conn(0x7fe01415dc90 msgr2=0x7fe0140ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 --2- 172.21.15.139:0/3116752981 >> v2:172.21.15.150:3300/0 conn(0x7fe01415dc90 0x7fe0140ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01406b120 con 0x7fe0140cf3e0 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fe004001690 con 0x7fe0140cf3e0 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 >> v1:172.21.15.139:6789/0 conn(0x7fe0140cf3e0 legacy=0x7fe01406ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01406f300 0x7fe0000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe000003f30 0x7fe000004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000004d40 0x7fe0000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01406ab60 con 0x7fe000003f30 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01406ae40 con 0x7fe000004d40 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00000b9c0 con 0x7fe01406f300 2024-04-25T10:03:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe0197f3640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe000003f30 0x7fe000004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe019ff4640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01406f300 0x7fe0000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe01a7f5640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000004d40 0x7fe0000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe0197f3640 1 -- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01406f300 msgr2=0x7fe0000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe0197f3640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01406f300 0x7fe0000039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe0197f3640 1 -- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000004d40 msgr2=0x7fe0000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe0197f3640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000004d40 0x7fe0000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe0197f3640 1 -- 172.21.15.139:0/3116752981 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe000003520 con 0x7fe000003f30 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe01a7f5640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000004d40 0x7fe0000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.331+0000 7fe019ff4640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe01406f300 0x7fe0000039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.332+0000 7fe0197f3640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe000003f30 0x7fe000004800 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fe004007fd0 tx=0x7fe000005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d80144f11869f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.332+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004007d50 con 0x7fe000003f30 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.332+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0040058d0 con 0x7fe000003f30 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.332+0000 7fe018ff2640 1 -- 172.21.15.139:0/3116752981 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004005d20 con 0x7fe000003f30 2024-04-25T10:03:23.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.333+0000 7fe01c258640 1 -- 172.21.15.139:0/3116752981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe000003f30 msgr2=0x7fe000004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.333+0000 7fe01c258640 1 --2- 172.21.15.139:0/3116752981 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe000003f30 0x7fe000004800 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fe004007fd0 tx=0x7fe000005830 comp rx=0 tx=0).stop 2024-04-25T10:03:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.333+0000 7fe01a7f5640 1 -- 172.21.15.139:0/3116752981 reap_dead start 2024-04-25T10:03:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.333+0000 7fe01c258640 1 -- 172.21.15.139:0/3116752981 shutdown_connections 2024-04-25T10:03:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.333+0000 7fe01c258640 1 -- 172.21.15.139:0/3116752981 >> 172.21.15.139:0/3116752981 conn(0x7fe0140cfd80 msgr2=0x7fe0140d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.333+0000 7fe01c258640 1 -- 172.21.15.139:0/3116752981 shutdown_connections 2024-04-25T10:03:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.333+0000 7fe01c258640 1 -- 172.21.15.139:0/3116752981 wait complete. 2024-04-25T10:03:23.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.334+0000 7fe01c258640 1 Processor -- start 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.334+0000 7fe01c258640 1 -- start start 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.334+0000 7fe01c258640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe000003f30 0x7fe014157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.334+0000 7fe01c258640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01414ace0 0x7fe01414b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.334+0000 7fe01c258640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0140b3b70 0x7fe0140b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.334+0000 7fe01c258640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01406b920 con 0x7fe01414ace0 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.334+0000 7fe01c258640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe014072620 con 0x7fe0140b3b70 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.334+0000 7fe01c258640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0140b2db0 con 0x7fe000003f30 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe0197f3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01414ace0 0x7fe01414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe0197f3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01414ace0 0x7fe01414b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39836/0 (socket says 172.21.15.139:39836) 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe0197f3640 1 -- 172.21.15.139:0/4274150232 learned_addr learned my addr 172.21.15.139:0/4274150232 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe01a7f5640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0140b3b70 0x7fe0140b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe019ff4640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe000003f30 0x7fe014157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe0197f3640 1 -- 172.21.15.139:0/4274150232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe000003f30 msgr2=0x7fe014157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe0197f3640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe000003f30 0x7fe014157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe0197f3640 1 -- 172.21.15.139:0/4274150232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0140b3b70 msgr2=0x7fe0140b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe0197f3640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0140b3b70 0x7fe0140b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe0197f3640 1 -- 172.21.15.139:0/4274150232 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01415e110 con 0x7fe01414ace0 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe01a7f5640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0140b3b70 0x7fe0140b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe0197f3640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01414ace0 0x7fe01414b530 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fe00400c2d0 tx=0x7fe0140cfb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe00a7fc640 1 -- 172.21.15.139:0/4274150232 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004038040 con 0x7fe01414ace0 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe00a7fc640 1 -- 172.21.15.139:0/4274150232 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0040047e0 con 0x7fe01414ace0 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe00a7fc640 1 -- 172.21.15.139:0/4274150232 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004004bb0 con 0x7fe01414ace0 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe01c258640 1 -- 172.21.15.139:0/4274150232 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01414b140 con 0x7fe01414ace0 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.335+0000 7fe01c258640 1 -- 172.21.15.139:0/4274150232 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe014158490 con 0x7fe01414ace0 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.336+0000 7fe01c258640 1 -- 172.21.15.139:0/4274150232 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe014074520 con 0x7fe01414ace0 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.337+0000 7fe00a7fc640 1 -- 172.21.15.139:0/4274150232 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe004007070 con 0x7fe01414ace0 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.337+0000 7fe00a7fc640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdff806b4d0 0x7fdff806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.337+0000 7fe019ff4640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdff806b4d0 0x7fdff806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.337+0000 7fe00a7fc640 1 -- 172.21.15.139:0/4274150232 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fe0040a9eb0 con 0x7fe01414ace0 2024-04-25T10:03:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.338+0000 7fe019ff4640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdff806b4d0 0x7fdff806d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe010009a90 tx=0x7fe010008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.341+0000 7fe00a7fc640 1 -- 172.21.15.139:0/4274150232 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe00404a020 con 0x7fe01414ace0 2024-04-25T10:03:23.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.524+0000 7fe01c258640 1 -- 172.21.15.139:0/4274150232 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe0140cd0e0 con 0x7fe01414ace0 2024-04-25T10:03:23.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.524+0000 7fe00a7fc640 1 -- 172.21.15.139:0/4274150232 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v122) v1 ==== 80+0+248 (secure 0 0 0) 0x7fe004078b80 con 0x7fe01414ace0 2024-04-25T10:03:23.529 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6836/3478297010 2024-04-26T10:03:13.364392+0000 2024-04-25T10:03:23.529 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139:6834/3478297010 2024-04-26T10:03:13.364392+0000 2024-04-25T10:03:23.530 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.150:6835/1377607596 2024-04-26T10:03:03.005923+0000 2024-04-25T10:03:23.530 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.150:6833/1377607596 2024-04-26T10:03:03.005923+0000 2024-04-25T10:03:23.530 INFO:teuthology.orchestra.run.smithi139.stderr:listed 4 entries 2024-04-25T10:03:23.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.526+0000 7fe01c258640 1 -- 172.21.15.139:0/4274150232 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdff806b4d0 msgr2=0x7fdff806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.526+0000 7fe01c258640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdff806b4d0 0x7fdff806d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe010009a90 tx=0x7fe010008040 comp rx=0 tx=0).stop 2024-04-25T10:03:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.526+0000 7fe01c258640 1 -- 172.21.15.139:0/4274150232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01414ace0 msgr2=0x7fe01414b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.526+0000 7fe01c258640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01414ace0 0x7fe01414b530 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fe00400c2d0 tx=0x7fe0140cfb50 comp rx=0 tx=0).stop 2024-04-25T10:03:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.526+0000 7fe01c258640 1 -- 172.21.15.139:0/4274150232 shutdown_connections 2024-04-25T10:03:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.526+0000 7fe01c258640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe01414ace0 0x7fe01414b530 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.526+0000 7fe01c258640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdff806b4d0 0x7fdff806d990 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.526+0000 7fe01c258640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0140b3b70 0x7fe0140b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.526+0000 7fe01c258640 1 --2- 172.21.15.139:0/4274150232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe000003f30 0x7fe014157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.526+0000 7fe01c258640 1 -- 172.21.15.139:0/4274150232 >> 172.21.15.139:0/4274150232 conn(0x7fe0140cfd80 msgr2=0x7fe0140b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.526+0000 7fe01c258640 1 -- 172.21.15.139:0/4274150232 shutdown_connections 2024-04-25T10:03:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.526+0000 7fe01c258640 1 -- 172.21.15.139:0/4274150232 wait complete. 2024-04-25T10:03:23.542 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:03:23.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:23.542 DEBUG:teuthology.orchestra.run.smithi139:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:23.568 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:03:23.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:23.569 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:03:23.623 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T10:03:23.623 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:23.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:03:23.642 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:23.642 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:23.678 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:03:23.679 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T10:03:23.679 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:23.700 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:03:23.700 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:23.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.811+0000 7f1be7310640 1 Processor -- start 2024-04-25T10:03:23.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.811+0000 7f1be7310640 1 -- start start 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be7310640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1be0155e30 0x7f1be015cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be7310640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1be015bdc0 0x7f1be015c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be7310640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1be00b1740 0x7f1be00b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be7310640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be006d6f0 con 0x7f1be00b1740 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be7310640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be015c880 con 0x7f1be015bdc0 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be7310640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be015cb60 con 0x7f1be0155e30 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be58ad640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1be00b1740 0x7f1be00b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be58ad640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1be00b1740 0x7f1be00b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39852/0 (socket says 172.21.15.139:39852) 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be58ad640 1 -- 172.21.15.139:0/1071703494 learned_addr learned my addr 172.21.15.139:0/1071703494 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be50ac640 1 --2- 172.21.15.139:0/1071703494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1be0155e30 0x7f1be015cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be48ab640 1 --2- 172.21.15.139:0/1071703494 >> v2:172.21.15.150:3300/0 conn(0x7f1be015bdc0 0x7f1be015c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be58ad640 1 -- 172.21.15.139:0/1071703494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1be0155e30 msgr2=0x7f1be015cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be58ad640 1 --2- 172.21.15.139:0/1071703494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1be0155e30 0x7f1be015cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be58ad640 1 -- 172.21.15.139:0/1071703494 >> v2:172.21.15.150:3300/0 conn(0x7f1be015bdc0 msgr2=0x7f1be015c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be58ad640 1 --2- 172.21.15.139:0/1071703494 >> v2:172.21.15.150:3300/0 conn(0x7f1be015bdc0 0x7f1be015c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.812+0000 7f1be58ad640 1 -- 172.21.15.139:0/1071703494 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be006d380 con 0x7f1be00b1740 2024-04-25T10:03:23.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.813+0000 7f1be58ad640 1 --2- 172.21.15.139:0/1071703494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be00b1740 0x7f1be00b1bd0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f1be00b2380 tx=0x7f1bd802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261eb5e7bbf6b5d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.813+0000 7f1bd7fff640 1 -- 172.21.15.139:0/1071703494 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8036070 con 0x7f1be00b1740 2024-04-25T10:03:23.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.813+0000 7f1bd7fff640 1 -- 172.21.15.139:0/1071703494 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd8031600 con 0x7f1be00b1740 2024-04-25T10:03:23.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.813+0000 7f1bd7fff640 1 -- 172.21.15.139:0/1071703494 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80319d0 con 0x7f1be00b1740 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.814+0000 7f1be7310640 1 -- 172.21.15.139:0/1071703494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be00b1740 msgr2=0x7f1be00b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.814+0000 7f1be7310640 1 --2- 172.21.15.139:0/1071703494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be00b1740 0x7f1be00b1bd0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f1be00b2380 tx=0x7f1bd802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.814+0000 7f1be7310640 1 -- 172.21.15.139:0/1071703494 shutdown_connections 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.814+0000 7f1be7310640 1 --2- 172.21.15.139:0/1071703494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be00b1740 0x7f1be00b1bd0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.814+0000 7f1be7310640 1 --2- 172.21.15.139:0/1071703494 >> v2:172.21.15.150:3300/0 conn(0x7f1be015bdc0 0x7f1be015c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.814+0000 7f1be7310640 1 --2- 172.21.15.139:0/1071703494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1be0155e30 0x7f1be015cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.814+0000 7f1be7310640 1 -- 172.21.15.139:0/1071703494 >> 172.21.15.139:0/1071703494 conn(0x7f1be0066f20 msgr2=0x7f1be006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.814+0000 7f1be7310640 1 -- 172.21.15.139:0/1071703494 shutdown_connections 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.814+0000 7f1be7310640 1 -- 172.21.15.139:0/1071703494 wait complete. 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.815+0000 7f1be7310640 1 Processor -- start 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.815+0000 7f1be7310640 1 -- start start 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be7310640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1be00b1740 0x7f1be0143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be7310640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1be0155e30 0x7f1be014f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be7310640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be015bdc0 0x7f1be015f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be7310640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00af260 con 0x7f1be015bdc0 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be7310640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00af0e0 con 0x7f1be0155e30 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be7310640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be015d4c0 con 0x7f1be00b1740 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be58ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be015bdc0 0x7f1be015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be58ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be015bdc0 0x7f1be015f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39856/0 (socket says 172.21.15.139:39856) 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be58ad640 1 -- 172.21.15.139:0/795003203 learned_addr learned my addr 172.21.15.139:0/795003203 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be48ab640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1be0155e30 0x7f1be014f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be50ac640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1be00b1740 0x7f1be0143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be58ad640 1 -- 172.21.15.139:0/795003203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1be00b1740 msgr2=0x7f1be0143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be58ad640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1be00b1740 0x7f1be0143060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be58ad640 1 -- 172.21.15.139:0/795003203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1be0155e30 msgr2=0x7f1be014f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be58ad640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1be0155e30 0x7f1be014f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be58ad640 1 -- 172.21.15.139:0/795003203 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be01f1530 con 0x7f1be015bdc0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be58ad640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be015bdc0 0x7f1be015f340 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f1bd802ac90 tx=0x7f1bd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1bd5ffb640 1 -- 172.21.15.139:0/795003203 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8041020 con 0x7f1be015bdc0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1bd5ffb640 1 -- 172.21.15.139:0/795003203 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd8007800 con 0x7f1be015bdc0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1be7310640 1 -- 172.21.15.139:0/795003203 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be0159c40 con 0x7f1be015bdc0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.816+0000 7f1bd5ffb640 1 -- 172.21.15.139:0/795003203 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8036040 con 0x7f1be015bdc0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.817+0000 7f1be7310640 1 -- 172.21.15.139:0/795003203 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be01f1a80 con 0x7f1be015bdc0 2024-04-25T10:03:23.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.818+0000 7f1be7310640 1 -- 172.21.15.139:0/795003203 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be0074520 con 0x7f1be015bdc0 2024-04-25T10:03:23.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.818+0000 7f1bd5ffb640 1 -- 172.21.15.139:0/795003203 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bd8044020 con 0x7f1be015bdc0 2024-04-25T10:03:23.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.818+0000 7f1bd5ffb640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1bc406b4d0 0x7f1bc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.819+0000 7f1bd5ffb640 1 -- 172.21.15.139:0/795003203 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f1bd80ab2c0 con 0x7f1be015bdc0 2024-04-25T10:03:23.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.819+0000 7f1be50ac640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1bc406b4d0 0x7f1bc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.819+0000 7f1be50ac640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1bc406b4d0 0x7f1bc406d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc006fd0 tx=0x7f1bdc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:23.823+0000 7f1bd5ffb640 1 -- 172.21.15.139:0/795003203 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1bd804b020 con 0x7f1be015bdc0 2024-04-25T10:03:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.030+0000 7f1be7310640 1 -- 172.21.15.139:0/795003203 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1be01435a0 con 0x7f1be015bdc0 2024-04-25T10:03:24.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.030+0000 7f1bd5ffb640 1 -- 172.21.15.139:0/795003203 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f1be0159c40 con 0x7f1be015bdc0 2024-04-25T10:03:24.035 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:24.035 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":7690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3195407462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3195407462},{"type":"v1","addr":"172.21.15.150:6835","nonce":3195407462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:03:17.394485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6836/1898344850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1898344850},{"type":"v1","addr":"172.21.15.139:6836","nonce":1898344850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:03:24.036 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 96 2024-04-25T10:03:24.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.032+0000 7f1be7310640 1 -- 172.21.15.139:0/795003203 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1bc406b4d0 msgr2=0x7f1bc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.032+0000 7f1be7310640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1bc406b4d0 0x7f1bc406d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc006fd0 tx=0x7f1bdc008040 comp rx=0 tx=0).stop 2024-04-25T10:03:24.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.032+0000 7f1be7310640 1 -- 172.21.15.139:0/795003203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be015bdc0 msgr2=0x7f1be015f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.032+0000 7f1be7310640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be015bdc0 0x7f1be015f340 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f1bd802ac90 tx=0x7f1bd8004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:24.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.033+0000 7f1be7310640 1 -- 172.21.15.139:0/795003203 shutdown_connections 2024-04-25T10:03:24.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.033+0000 7f1be7310640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1be015bdc0 0x7f1be015f340 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.033+0000 7f1be7310640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1bc406b4d0 0x7f1bc406d990 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.033+0000 7f1be7310640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1be0155e30 0x7f1be014f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.033+0000 7f1be7310640 1 --2- 172.21.15.139:0/795003203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1be00b1740 0x7f1be0143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.033+0000 7f1be7310640 1 -- 172.21.15.139:0/795003203 >> 172.21.15.139:0/795003203 conn(0x7f1be0066f20 msgr2=0x7f1be006da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.033+0000 7f1be7310640 1 -- 172.21.15.139:0/795003203 shutdown_connections 2024-04-25T10:03:24.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.033+0000 7f1be7310640 1 -- 172.21.15.139:0/795003203 wait complete. 2024-04-25T10:03:24.048 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:24.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.140+0000 7fec5430a640 1 Processor -- start 2024-04-25T10:03:24.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.140+0000 7fec5430a640 1 -- start start 2024-04-25T10:03:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec5430a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fec4c155e30 0x7fec4c15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec5430a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec4c0b1e50 0x7fec4c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec5430a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4c15c880 con 0x7fec4c15bdc0 2024-04-25T10:03:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec5430a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4c06d6f0 con 0x7fec4c155e30 2024-04-25T10:03:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec5430a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c15cb60 con 0x7fec4c0b1e50 2024-04-25T10:03:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec520a6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fec4c155e30 0x7fec4c15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec518a5640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fec4c15bdc0 0x7fec4c0b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53944/0 (socket says 172.21.15.139:53944) 2024-04-25T10:03:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec520a6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fec4c155e30 0x7fec4c15cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39860/0 (socket says 172.21.15.139:39860) 2024-04-25T10:03:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec518a5640 1 -- 172.21.15.139:0/824050954 learned_addr learned my addr 172.21.15.139:0/824050954 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:24.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec520a6640 1 -- 172.21.15.139:0/824050954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec4c0b1e50 msgr2=0x7fec4c15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec520a6640 1 --2- 172.21.15.139:0/824050954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec4c0b1e50 0x7fec4c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec520a6640 1 -- 172.21.15.139:0/824050954 >> v1:172.21.15.139:6789/0 conn(0x7fec4c15bdc0 legacy=0x7fec4c0b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec520a6640 1 -- 172.21.15.139:0/824050954 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c06d380 con 0x7fec4c155e30 2024-04-25T10:03:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.141+0000 7fec510a4640 1 -- 172.21.15.139:0/824050954 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749331658 0 0) 0x7fec4c15c880 con 0x7fec4c15bdc0 2024-04-25T10:03:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.142+0000 7fec520a6640 1 --2- 172.21.15.139:0/824050954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec4c155e30 0x7fec4c15cf80 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fec48009d90 tx=0x7fec4802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c2a816ba4c2bed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.142+0000 7fec510a4640 1 -- 172.21.15.139:0/824050954 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48007740 con 0x7fec4c155e30 2024-04-25T10:03:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.142+0000 7fec510a4640 1 -- 172.21.15.139:0/824050954 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec480078e0 con 0x7fec4c155e30 2024-04-25T10:03:24.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.142+0000 7fec510a4640 1 -- 172.21.15.139:0/824050954 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48007cb0 con 0x7fec4c155e30 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.143+0000 7fec5430a640 1 -- 172.21.15.139:0/824050954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec4c155e30 msgr2=0x7fec4c15cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.143+0000 7fec5430a640 1 --2- 172.21.15.139:0/824050954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec4c155e30 0x7fec4c15cf80 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fec48009d90 tx=0x7fec4802a680 comp rx=0 tx=0).stop 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.143+0000 7fec5430a640 1 -- 172.21.15.139:0/824050954 shutdown_connections 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.143+0000 7fec5430a640 1 --2- 172.21.15.139:0/824050954 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec4c0b1e50 0x7fec4c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.143+0000 7fec5430a640 1 --2- 172.21.15.139:0/824050954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec4c155e30 0x7fec4c15cf80 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.143+0000 7fec5430a640 1 -- 172.21.15.139:0/824050954 >> 172.21.15.139:0/824050954 conn(0x7fec4c066f20 msgr2=0x7fec4c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.143+0000 7fec5430a640 1 -- 172.21.15.139:0/824050954 shutdown_connections 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.143+0000 7fec5430a640 1 -- 172.21.15.139:0/824050954 wait complete. 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.144+0000 7fec5430a640 1 Processor -- start 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.144+0000 7fec5430a640 1 -- start start 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec5430a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec4c0b1e50 0x7fec4c0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec5430a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec4c155e30 0x7fec4c1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec5430a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec4c15bdc0 0x7fec4c0a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec5430a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c0af260 con 0x7fec4c0b1e50 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec5430a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c0af0e0 con 0x7fec4c155e30 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec5430a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c15d4c0 con 0x7fec4c15bdc0 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec520a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec4c0b1e50 0x7fec4c0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec520a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec4c0b1e50 0x7fec4c0ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39876/0 (socket says 172.21.15.139:39876) 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec520a6640 1 -- 172.21.15.139:0/2198636697 learned_addr learned my addr 172.21.15.139:0/2198636697 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec528a7640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec4c15bdc0 0x7fec4c0a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec518a5640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec4c155e30 0x7fec4c1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec520a6640 1 -- 172.21.15.139:0/2198636697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec4c15bdc0 msgr2=0x7fec4c0a86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec520a6640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec4c15bdc0 0x7fec4c0a86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec520a6640 1 -- 172.21.15.139:0/2198636697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec4c155e30 msgr2=0x7fec4c1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec520a6640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec4c155e30 0x7fec4c1528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec520a6640 1 -- 172.21.15.139:0/2198636697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c153360 con 0x7fec4c0b1e50 2024-04-25T10:03:24.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec528a7640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec4c15bdc0 0x7fec4c0a86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:24.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec518a5640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec4c155e30 0x7fec4c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:24.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.145+0000 7fec520a6640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec4c0b1e50 0x7fec4c0ade30 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fec48009d90 tx=0x7fec48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.146+0000 7fec42ffd640 1 -- 172.21.15.139:0/2198636697 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48040070 con 0x7fec4c0b1e50 2024-04-25T10:03:24.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.146+0000 7fec42ffd640 1 -- 172.21.15.139:0/2198636697 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec48031d00 con 0x7fec4c0b1e50 2024-04-25T10:03:24.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.146+0000 7fec5430a640 1 -- 172.21.15.139:0/2198636697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4c153640 con 0x7fec4c0b1e50 2024-04-25T10:03:24.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.146+0000 7fec42ffd640 1 -- 172.21.15.139:0/2198636697 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48045680 con 0x7fec4c0b1e50 2024-04-25T10:03:24.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.146+0000 7fec5430a640 1 -- 172.21.15.139:0/2198636697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4c157750 con 0x7fec4c0b1e50 2024-04-25T10:03:24.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.147+0000 7fec5430a640 1 -- 172.21.15.139:0/2198636697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4c074520 con 0x7fec4c0b1e50 2024-04-25T10:03:24.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.147+0000 7fec42ffd640 1 -- 172.21.15.139:0/2198636697 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec48039040 con 0x7fec4c0b1e50 2024-04-25T10:03:24.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.148+0000 7fec42ffd640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fec3006b4d0 0x7fec3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.148+0000 7fec518a5640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fec3006b4d0 0x7fec3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.148+0000 7fec42ffd640 1 -- 172.21.15.139:0/2198636697 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fec480abe60 con 0x7fec4c0b1e50 2024-04-25T10:03:24.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.148+0000 7fec518a5640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fec3006b4d0 0x7fec3006d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fec4c15f240 tx=0x7fec3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.152+0000 7fec42ffd640 1 -- 172.21.15.139:0/2198636697 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec4807ab30 con 0x7fec4c0b1e50 2024-04-25T10:03:24.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.322+0000 7fec5430a640 1 -- 172.21.15.139:0/2198636697 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec4c06afc0 con 0x7fec4c0b1e50 2024-04-25T10:03:24.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.323+0000 7fec42ffd640 1 -- 172.21.15.139:0/2198636697 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17263 (secure 0 0 0) 0x7fec4807a430 con 0x7fec4c0b1e50 2024-04-25T10:03:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:24.328 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":122,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:03:13.364415+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:48.574891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:49.293101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6836/3478297010":"2024-04-26T10:03:13.364392+0000","172.21.15.139:6834/3478297010":"2024-04-26T10:03:13.364392+0000","172.21.15.150:6835/1377607596":"2024-04-26T10:03:03.005923+0000","172.21.15.150:6833/1377607596":"2024-04-26T10:03:03.005923+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:24.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.325+0000 7fec5430a640 1 -- 172.21.15.139:0/2198636697 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fec3006b4d0 msgr2=0x7fec3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.325+0000 7fec5430a640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fec3006b4d0 0x7fec3006d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fec4c15f240 tx=0x7fec3c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:24.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.325+0000 7fec5430a640 1 -- 172.21.15.139:0/2198636697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec4c0b1e50 msgr2=0x7fec4c0ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.325+0000 7fec5430a640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec4c0b1e50 0x7fec4c0ade30 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fec48009d90 tx=0x7fec48004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:24.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.325+0000 7fec5430a640 1 -- 172.21.15.139:0/2198636697 shutdown_connections 2024-04-25T10:03:24.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.325+0000 7fec5430a640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fec3006b4d0 0x7fec3006d990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.325+0000 7fec5430a640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec4c15bdc0 0x7fec4c0a86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.325+0000 7fec5430a640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec4c155e30 0x7fec4c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.325+0000 7fec5430a640 1 --2- 172.21.15.139:0/2198636697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec4c0b1e50 0x7fec4c0ade30 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.325+0000 7fec5430a640 1 -- 172.21.15.139:0/2198636697 >> 172.21.15.139:0/2198636697 conn(0x7fec4c066f20 msgr2=0x7fec4c15cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.325+0000 7fec5430a640 1 -- 172.21.15.139:0/2198636697 shutdown_connections 2024-04-25T10:03:24.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.325+0000 7fec5430a640 1 -- 172.21.15.139:0/2198636697 wait complete. 2024-04-25T10:03:24.341 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:53:02.998776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:48.574891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:49.293101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:24.342 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:24.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.438+0000 7f39f46ee640 1 Processor -- start 2024-04-25T10:03:24.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.438+0000 7f39f46ee640 1 -- start start 2024-04-25T10:03:24.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f46ee640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f39ec1617e0 0x7f39ec0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f46ee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39ec0cec80 0x7f39ec0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f46ee640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ec161dc0 con 0x7f39ec15dbf0 2024-04-25T10:03:24.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f46ee640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ec0d1ee0 con 0x7f39ec1617e0 2024-04-25T10:03:24.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f46ee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec1620a0 con 0x7f39ec0cec80 2024-04-25T10:03:24.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f1c89640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f39ec1617e0 0x7f39ec0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f2c8b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39ec0cec80 0x7f39ec0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f2c8b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39ec0cec80 0x7f39ec0d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:52368/0 (socket says 172.21.15.139:52368) 2024-04-25T10:03:24.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f1c89640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f39ec1617e0 0x7f39ec0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:34370/0 (socket says 172.21.15.139:34370) 2024-04-25T10:03:24.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f2c8b640 1 -- 172.21.15.139:0/2857603295 learned_addr learned my addr 172.21.15.139:0/2857603295 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:24.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f2c8b640 1 -- 172.21.15.139:0/2857603295 >> v2:172.21.15.150:3300/0 conn(0x7f39ec1617e0 msgr2=0x7f39ec0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f2c8b640 1 --2- 172.21.15.139:0/2857603295 >> v2:172.21.15.150:3300/0 conn(0x7f39ec1617e0 0x7f39ec0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f2c8b640 1 -- 172.21.15.139:0/2857603295 >> v1:172.21.15.150:6789/0 conn(0x7f39ec15dbf0 legacy=0x7f39ec0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f2c8b640 1 -- 172.21.15.139:0/2857603295 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ec162380 con 0x7f39ec0cec80 2024-04-25T10:03:24.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.439+0000 7f39f1c89640 1 --2- 172.21.15.139:0/2857603295 >> v2:172.21.15.150:3300/0 conn(0x7f39ec1617e0 0x7f39ec0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:24.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.440+0000 7f39f2c8b640 1 --2- 172.21.15.139:0/2857603295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39ec0cec80 0x7f39ec0d2370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009e50 tx=0x7f39e402a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8ad1e033869e642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.440+0000 7f39f1488640 1 -- 172.21.15.139:0/2857603295 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4036070 con 0x7f39ec0cec80 2024-04-25T10:03:24.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.440+0000 7f39f1488640 1 -- 172.21.15.139:0/2857603295 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e4031600 con 0x7f39ec0cec80 2024-04-25T10:03:24.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.440+0000 7f39f1488640 1 -- 172.21.15.139:0/2857603295 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40319d0 con 0x7f39ec0cec80 2024-04-25T10:03:24.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.441+0000 7f39f46ee640 1 -- 172.21.15.139:0/2857603295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39ec0cec80 msgr2=0x7f39ec0d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.441+0000 7f39f46ee640 1 --2- 172.21.15.139:0/2857603295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39ec0cec80 0x7f39ec0d2370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009e50 tx=0x7f39e402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.441+0000 7f39f46ee640 1 -- 172.21.15.139:0/2857603295 shutdown_connections 2024-04-25T10:03:24.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.441+0000 7f39f46ee640 1 --2- 172.21.15.139:0/2857603295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39ec0cec80 0x7f39ec0d2370 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.441+0000 7f39f46ee640 1 --2- 172.21.15.139:0/2857603295 >> v2:172.21.15.150:3300/0 conn(0x7f39ec1617e0 0x7f39ec0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.441+0000 7f39f46ee640 1 -- 172.21.15.139:0/2857603295 >> 172.21.15.139:0/2857603295 conn(0x7f39ec0b5480 msgr2=0x7f39ec0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.441+0000 7f39f46ee640 1 -- 172.21.15.139:0/2857603295 shutdown_connections 2024-04-25T10:03:24.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.441+0000 7f39f46ee640 1 -- 172.21.15.139:0/2857603295 wait complete. 2024-04-25T10:03:24.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.442+0000 7f39f46ee640 1 Processor -- start 2024-04-25T10:03:24.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.442+0000 7f39f46ee640 1 -- start start 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f46ee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39ec0cec80 0x7f39ec0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f46ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39ec15dbf0 0x7f39ec159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f46ee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39ec1617e0 0x7f39ec0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f46ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec15dfd0 con 0x7f39ec15dbf0 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f46ee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec0d28b0 con 0x7f39ec0cec80 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f46ee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec1f5060 con 0x7f39ec1617e0 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f1c89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39ec15dbf0 0x7f39ec159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f1c89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39ec15dbf0 0x7f39ec159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39892/0 (socket says 172.21.15.139:39892) 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f1c89640 1 -- 172.21.15.139:0/1912187413 learned_addr learned my addr 172.21.15.139:0/1912187413 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f248a640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39ec0cec80 0x7f39ec0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f2c8b640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39ec1617e0 0x7f39ec0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f1c89640 1 -- 172.21.15.139:0/1912187413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39ec1617e0 msgr2=0x7f39ec0b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f1c89640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39ec1617e0 0x7f39ec0b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f1c89640 1 -- 172.21.15.139:0/1912187413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39ec0cec80 msgr2=0x7f39ec0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f1c89640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39ec0cec80 0x7f39ec0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.443+0000 7f39f1c89640 1 -- 172.21.15.139:0/1912187413 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ec1f52d0 con 0x7f39ec15dbf0 2024-04-25T10:03:24.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.444+0000 7f39f1c89640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39ec15dbf0 0x7f39ec159e30 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00e990 tx=0x7f39dc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.444+0000 7f39e2ffd640 1 -- 172.21.15.139:0/1912187413 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc017070 con 0x7f39ec15dbf0 2024-04-25T10:03:24.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.444+0000 7f39e2ffd640 1 -- 172.21.15.139:0/1912187413 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc00c9e0 con 0x7f39ec15dbf0 2024-04-25T10:03:24.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.444+0000 7f39e2ffd640 1 -- 172.21.15.139:0/1912187413 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00cdb0 con 0x7f39ec15dbf0 2024-04-25T10:03:24.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.444+0000 7f39f46ee640 1 -- 172.21.15.139:0/1912187413 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ec1f55b0 con 0x7f39ec15dbf0 2024-04-25T10:03:24.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.444+0000 7f39f46ee640 1 -- 172.21.15.139:0/1912187413 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ec15e2e0 con 0x7f39ec15dbf0 2024-04-25T10:03:24.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.445+0000 7f39f46ee640 1 -- 172.21.15.139:0/1912187413 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ec074520 con 0x7f39ec15dbf0 2024-04-25T10:03:24.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.445+0000 7f39e2ffd640 1 -- 172.21.15.139:0/1912187413 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39dc01a070 con 0x7f39ec15dbf0 2024-04-25T10:03:24.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.445+0000 7f39e2ffd640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f39d006b4d0 0x7f39d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.446+0000 7f39e2ffd640 1 -- 172.21.15.139:0/1912187413 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f39dc08a560 con 0x7f39ec15dbf0 2024-04-25T10:03:24.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.446+0000 7f39f248a640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f39d006b4d0 0x7f39d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.446+0000 7f39f248a640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f39d006b4d0 0x7f39d006d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f39ec155a60 tx=0x7f39e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.450+0000 7f39e2ffd640 1 -- 172.21.15.139:0/1912187413 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39dc058be0 con 0x7f39ec15dbf0 2024-04-25T10:03:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.622+0000 7f39f46ee640 1 -- 172.21.15.139:0/1912187413 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39ec0d1680 con 0x7f39ec15dbf0 2024-04-25T10:03:24.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.622+0000 7f39e2ffd640 1 -- 172.21.15.139:0/1912187413 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39dc05cc80 con 0x7f39ec15dbf0 2024-04-25T10:03:24.627 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 1 2024-04-25T10:03:24.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.624+0000 7f39f46ee640 1 -- 172.21.15.139:0/1912187413 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f39d006b4d0 msgr2=0x7f39d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.624+0000 7f39f46ee640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f39d006b4d0 0x7f39d006d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f39ec155a60 tx=0x7f39e8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:24.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.624+0000 7f39f46ee640 1 -- 172.21.15.139:0/1912187413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39ec15dbf0 msgr2=0x7f39ec159e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.624+0000 7f39f46ee640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39ec15dbf0 0x7f39ec159e30 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00e990 tx=0x7f39dc00ee50 comp rx=0 tx=0).stop 2024-04-25T10:03:24.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.624+0000 7f39f46ee640 1 -- 172.21.15.139:0/1912187413 shutdown_connections 2024-04-25T10:03:24.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.624+0000 7f39f46ee640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39ec15dbf0 0x7f39ec159e30 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.624+0000 7f39f46ee640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f39d006b4d0 0x7f39d006d990 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.624+0000 7f39f46ee640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39ec1617e0 0x7f39ec0b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.624+0000 7f39f46ee640 1 --2- 172.21.15.139:0/1912187413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39ec0cec80 0x7f39ec0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.624+0000 7f39f46ee640 1 -- 172.21.15.139:0/1912187413 >> 172.21.15.139:0/1912187413 conn(0x7f39ec0b5480 msgr2=0x7f39ec0d2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.624+0000 7f39f46ee640 1 -- 172.21.15.139:0/1912187413 shutdown_connections 2024-04-25T10:03:24.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.625+0000 7f39f46ee640 1 -- 172.21.15.139:0/1912187413 wait complete. 2024-04-25T10:03:24.640 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.735+0000 7fcd6c04d640 1 Processor -- start 2024-04-25T10:03:24.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.735+0000 7fcd6c04d640 1 -- start start 2024-04-25T10:03:24.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.736+0000 7fcd6c04d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcd6406ebf0 0x7fcd6406f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.736+0000 7fcd6c04d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd640b2db0 0x7fcd640b3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.736+0000 7fcd6c04d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd640cf980 con 0x7fcd6415dc90 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.736+0000 7fcd6c04d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6406ab60 con 0x7fcd6406ebf0 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.736+0000 7fcd6c04d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6406ae40 con 0x7fcd640b2db0 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.736+0000 7fcd69de9640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fcd6415dc90 0x7fcd640ceea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53958/0 (socket says 172.21.15.139:53958) 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.736+0000 7fcd69de9640 1 -- 172.21.15.139:0/3695363174 learned_addr learned my addr 172.21.15.139:0/3695363174 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.736+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562089736 0 0) 0x7fcd640cf980 con 0x7fcd6415dc90 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.736+0000 7fcd695e8640 1 --2- 172.21.15.139:0/3695363174 >> v2:172.21.15.150:3300/0 conn(0x7fcd6406ebf0 0x7fcd6406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd6a5ea640 1 --2- 172.21.15.139:0/3695363174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd640b2db0 0x7fcd640b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd50003520 con 0x7fcd6415dc90 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 961280665 0 0) 0x7fcd50003520 con 0x7fcd6415dc90 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd640b2db0 msgr2=0x7fcd640b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 --2- 172.21.15.139:0/3695363174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd640b2db0 0x7fcd640b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 >> v2:172.21.15.150:3300/0 conn(0x7fcd6406ebf0 msgr2=0x7fcd6406f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 --2- 172.21.15.139:0/3695363174 >> v2:172.21.15.150:3300/0 conn(0x7fcd6406ebf0 0x7fcd6406f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6406b120 con 0x7fcd6415dc90 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fcd5c001a10 con 0x7fcd6415dc90 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 >> v1:172.21.15.139:6789/0 conn(0x7fcd6415dc90 legacy=0x7fcd640ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 --2- 172.21.15.139:0/3695363174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd640b2db0 0x7fcd500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 --2- 172.21.15.139:0/3695363174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd50003f30 0x7fcd50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 --2- 172.21.15.139:0/3695363174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd50004d40 0x7fcd500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6406ab60 con 0x7fcd50003f30 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6406ae40 con 0x7fcd50004d40 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5000b9c0 con 0x7fcd640b2db0 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd695e8640 1 --2- 172.21.15.139:0/3695363174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd50003f30 0x7fcd50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd69de9640 1 --2- 172.21.15.139:0/3695363174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd640b2db0 0x7fcd500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd6a5ea640 1 --2- 172.21.15.139:0/3695363174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd50004d40 0x7fcd500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd695e8640 1 -- 172.21.15.139:0/3695363174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd640b2db0 msgr2=0x7fcd500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd695e8640 1 --2- 172.21.15.139:0/3695363174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd640b2db0 0x7fcd500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd695e8640 1 -- 172.21.15.139:0/3695363174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd50004d40 msgr2=0x7fcd500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd695e8640 1 --2- 172.21.15.139:0/3695363174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd50004d40 0x7fcd500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.737+0000 7fcd695e8640 1 -- 172.21.15.139:0/3695363174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50003520 con 0x7fcd50003f30 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.738+0000 7fcd695e8640 1 --2- 172.21.15.139:0/3695363174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd50003f30 0x7fcd50004800 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009ac0 tx=0x7fcd50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a50638c7e6652a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.738+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd54036070 con 0x7fcd50003f30 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.738+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd54031400 con 0x7fcd50003f30 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.738+0000 7fcd68de7640 1 -- 172.21.15.139:0/3695363174 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd54009690 con 0x7fcd50003f30 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.739+0000 7fcd6c04d640 1 -- 172.21.15.139:0/3695363174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd50003f30 msgr2=0x7fcd50004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.739+0000 7fcd6c04d640 1 --2- 172.21.15.139:0/3695363174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd50003f30 0x7fcd50004800 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009ac0 tx=0x7fcd50005830 comp rx=0 tx=0).stop 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.739+0000 7fcd6a5ea640 1 -- 172.21.15.139:0/3695363174 reap_dead start 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.739+0000 7fcd6c04d640 1 -- 172.21.15.139:0/3695363174 shutdown_connections 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.739+0000 7fcd6c04d640 1 -- 172.21.15.139:0/3695363174 >> 172.21.15.139:0/3695363174 conn(0x7fcd640cfd80 msgr2=0x7fcd640d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.739+0000 7fcd6c04d640 1 -- 172.21.15.139:0/3695363174 shutdown_connections 2024-04-25T10:03:24.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.739+0000 7fcd6c04d640 1 -- 172.21.15.139:0/3695363174 wait complete. 2024-04-25T10:03:24.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.740+0000 7fcd6c04d640 1 Processor -- start 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.740+0000 7fcd6c04d640 1 -- start start 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd6c04d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd50003f30 0x7fcd641f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd6c04d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6414f890 0x7fcd640afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd6c04d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd640aff60 0x7fcd640c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd6c04d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd64072620 con 0x7fcd6414f890 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd6c04d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6406f5c0 con 0x7fcd640aff60 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd6c04d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6406b920 con 0x7fcd50003f30 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd695e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6414f890 0x7fcd640afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd695e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6414f890 0x7fcd640afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39904/0 (socket says 172.21.15.139:39904) 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd695e8640 1 -- 172.21.15.139:0/2591411458 learned_addr learned my addr 172.21.15.139:0/2591411458 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd69de9640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd50003f30 0x7fcd641f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd6a5ea640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd640aff60 0x7fcd640c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd695e8640 1 -- 172.21.15.139:0/2591411458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd50003f30 msgr2=0x7fcd641f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd695e8640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd50003f30 0x7fcd641f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd695e8640 1 -- 172.21.15.139:0/2591411458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd640aff60 msgr2=0x7fcd640c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd695e8640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd640aff60 0x7fcd640c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd695e8640 1 -- 172.21.15.139:0/2591411458 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6415ea10 con 0x7fcd6414f890 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd6a5ea640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd640aff60 0x7fcd640c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd695e8640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6414f890 0x7fcd640afa20 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009be0 tx=0x7fcd54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd5a7fc640 1 -- 172.21.15.139:0/2591411458 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd54040020 con 0x7fcd6414f890 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd5a7fc640 1 -- 172.21.15.139:0/2591411458 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd54003c60 con 0x7fcd6414f890 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd6c04d640 1 -- 172.21.15.139:0/2591411458 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6415ecc0 con 0x7fcd6414f890 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd5a7fc640 1 -- 172.21.15.139:0/2591411458 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd54036040 con 0x7fcd6414f890 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.741+0000 7fcd6c04d640 1 -- 172.21.15.139:0/2591411458 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6415f280 con 0x7fcd6414f890 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.742+0000 7fcd6c04d640 1 -- 172.21.15.139:0/2591411458 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd64074520 con 0x7fcd6414f890 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.743+0000 7fcd5a7fc640 1 -- 172.21.15.139:0/2591411458 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd54043020 con 0x7fcd6414f890 2024-04-25T10:03:24.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.743+0000 7fcd5a7fc640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd4806b4d0 0x7fcd4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.743+0000 7fcd5a7fc640 1 -- 172.21.15.139:0/2591411458 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fcd540aa990 con 0x7fcd6414f890 2024-04-25T10:03:24.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.743+0000 7fcd69de9640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd4806b4d0 0x7fcd4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.744+0000 7fcd69de9640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd4806b4d0 0x7fcd4806d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcd64067de0 tx=0x7fcd5c006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.747+0000 7fcd5a7fc640 1 -- 172.21.15.139:0/2591411458 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd5404a030 con 0x7fcd6414f890 2024-04-25T10:03:24.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.917+0000 7fcd6c04d640 1 -- 172.21.15.139:0/2591411458 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd640b50e0 con 0x7fcd6414f890 2024-04-25T10:03:24.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.917+0000 7fcd5a7fc640 1 -- 172.21.15.139:0/2591411458 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd540044e0 con 0x7fcd6414f890 2024-04-25T10:03:24.922 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 32 2024-04-25T10:03:24.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.920+0000 7fcd6c04d640 1 -- 172.21.15.139:0/2591411458 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd4806b4d0 msgr2=0x7fcd4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.920+0000 7fcd6c04d640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd4806b4d0 0x7fcd4806d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcd64067de0 tx=0x7fcd5c006040 comp rx=0 tx=0).stop 2024-04-25T10:03:24.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.920+0000 7fcd6c04d640 1 -- 172.21.15.139:0/2591411458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6414f890 msgr2=0x7fcd640afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.920+0000 7fcd6c04d640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6414f890 0x7fcd640afa20 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009be0 tx=0x7fcd54004730 comp rx=0 tx=0).stop 2024-04-25T10:03:24.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.920+0000 7fcd6c04d640 1 -- 172.21.15.139:0/2591411458 shutdown_connections 2024-04-25T10:03:24.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.920+0000 7fcd6c04d640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6414f890 0x7fcd640afa20 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.920+0000 7fcd6c04d640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fcd4806b4d0 0x7fcd4806d990 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.920+0000 7fcd6c04d640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd640aff60 0x7fcd640c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.920+0000 7fcd6c04d640 1 --2- 172.21.15.139:0/2591411458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd50003f30 0x7fcd641f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.920+0000 7fcd6c04d640 1 -- 172.21.15.139:0/2591411458 >> 172.21.15.139:0/2591411458 conn(0x7fcd640cfd80 msgr2=0x7fcd640ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.920+0000 7fcd6c04d640 1 -- 172.21.15.139:0/2591411458 shutdown_connections 2024-04-25T10:03:24.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:24.920+0000 7fcd6c04d640 1 -- 172.21.15.139:0/2591411458 wait complete. 2024-04-25T10:03:24.935 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:25.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.031+0000 7f1ae5a6f640 1 Processor -- start 2024-04-25T10:03:25.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.031+0000 7f1ae5a6f640 1 -- start start 2024-04-25T10:03:25.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1ae5a6f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1ae015dbf0 0x7f1ae01617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1ae5a6f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ae00b9370 0x7f1ae00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1ae5a6f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae0161dc0 con 0x7f1ae00cec80 2024-04-25T10:03:25.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1ae5a6f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae00d1ee0 con 0x7f1ae015dbf0 2024-04-25T10:03:25.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1ae5a6f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae01620a0 con 0x7f1ae00b9370 2024-04-25T10:03:25.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1adeffd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ae00b9370 0x7f1ae00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1adf7fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1ae015dbf0 0x7f1ae01617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1adf7fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1ae015dbf0 0x7f1ae01617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:34416/0 (socket says 172.21.15.139:34416) 2024-04-25T10:03:25.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1adf7fe640 1 -- 172.21.15.139:0/1385303909 learned_addr learned my addr 172.21.15.139:0/1385303909 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:25.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1adf7fe640 1 -- 172.21.15.139:0/1385303909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ae00b9370 msgr2=0x7f1ae00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1adf7fe640 1 --2- 172.21.15.139:0/1385303909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ae00b9370 0x7f1ae00ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1adf7fe640 1 -- 172.21.15.139:0/1385303909 >> v1:172.21.15.150:6789/0 conn(0x7f1ae00cec80 legacy=0x7f1ae006c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1adf7fe640 1 -- 172.21.15.139:0/1385303909 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae0162380 con 0x7f1ae015dbf0 2024-04-25T10:03:25.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.032+0000 7f1adeffd640 1 --2- 172.21.15.139:0/1385303909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ae00b9370 0x7f1ae00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:25.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.033+0000 7f1adf7fe640 1 --2- 172.21.15.139:0/1385303909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ae015dbf0 0x7f1ae01617e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009870 tx=0x7f1ad002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b258123ea04a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.033+0000 7f1ade7fc640 1 -- 172.21.15.139:0/1385303909 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0037070 con 0x7f1ae015dbf0 2024-04-25T10:03:25.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.033+0000 7f1ade7fc640 1 -- 172.21.15.139:0/1385303909 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad0032440 con 0x7f1ae015dbf0 2024-04-25T10:03:25.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.033+0000 7f1ade7fc640 1 -- 172.21.15.139:0/1385303909 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0032810 con 0x7f1ae015dbf0 2024-04-25T10:03:25.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.034+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1385303909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ae015dbf0 msgr2=0x7f1ae01617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.034+0000 7f1ae5a6f640 1 --2- 172.21.15.139:0/1385303909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ae015dbf0 0x7f1ae01617e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009870 tx=0x7f1ad002a270 comp rx=0 tx=0).stop 2024-04-25T10:03:25.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.034+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1385303909 shutdown_connections 2024-04-25T10:03:25.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.034+0000 7f1ae5a6f640 1 --2- 172.21.15.139:0/1385303909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ae00b9370 0x7f1ae00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.034+0000 7f1ae5a6f640 1 --2- 172.21.15.139:0/1385303909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ae015dbf0 0x7f1ae01617e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.034+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1385303909 >> 172.21.15.139:0/1385303909 conn(0x7f1ae00b5480 msgr2=0x7f1ae00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.034+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1385303909 shutdown_connections 2024-04-25T10:03:25.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.034+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1385303909 wait complete. 2024-04-25T10:03:25.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.035+0000 7f1ae5a6f640 1 Processor -- start 2024-04-25T10:03:25.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.035+0000 7f1ae5a6f640 1 -- start start 2024-04-25T10:03:25.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1ae5a6f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae00b9370 0x7f1ae00b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1ae5a6f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ae00cec80 0x7f1ae0160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1ae5a6f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ae015dbf0 0x7f1ae0160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1ae5a6f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae015dfd0 con 0x7f1ae00b9370 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1ae5a6f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae00b97d0 con 0x7f1ae00cec80 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1ae5a6f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae015cf30 con 0x7f1ae015dbf0 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1adf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae00b9370 0x7f1ae00b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1adf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae00b9370 0x7f1ae00b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39920/0 (socket says 172.21.15.139:39920) 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1adf7fe640 1 -- 172.21.15.139:0/1974631577 learned_addr learned my addr 172.21.15.139:0/1974631577 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1adeffd640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ae00cec80 0x7f1ae0160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1adffff640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ae015dbf0 0x7f1ae0160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1adf7fe640 1 -- 172.21.15.139:0/1974631577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ae015dbf0 msgr2=0x7f1ae0160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1adf7fe640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ae015dbf0 0x7f1ae0160b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1adf7fe640 1 -- 172.21.15.139:0/1974631577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ae00cec80 msgr2=0x7f1ae0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1adf7fe640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ae00cec80 0x7f1ae0160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.036+0000 7f1adf7fe640 1 -- 172.21.15.139:0/1974631577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae015c970 con 0x7f1ae00b9370 2024-04-25T10:03:25.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.037+0000 7f1adf7fe640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae00b9370 0x7f1ae00b4c40 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0000c00 tx=0x7f1ad00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.037+0000 7f1ac7fff640 1 -- 172.21.15.139:0/1974631577 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0041070 con 0x7f1ae00b9370 2024-04-25T10:03:25.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.037+0000 7f1ac7fff640 1 -- 172.21.15.139:0/1974631577 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad003c890 con 0x7f1ae00b9370 2024-04-25T10:03:25.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.037+0000 7f1ac7fff640 1 -- 172.21.15.139:0/1974631577 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0037040 con 0x7f1ae00b9370 2024-04-25T10:03:25.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.037+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1974631577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae015cc50 con 0x7f1ae00b9370 2024-04-25T10:03:25.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.037+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1974631577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae00b5180 con 0x7f1ae00b9370 2024-04-25T10:03:25.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.038+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1974631577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae0074520 con 0x7f1ae00b9370 2024-04-25T10:03:25.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.038+0000 7f1ac7fff640 1 -- 172.21.15.139:0/1974631577 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ad0044070 con 0x7f1ae00b9370 2024-04-25T10:03:25.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.039+0000 7f1ac7fff640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1acc06b4d0 0x7f1acc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.039+0000 7f1ac7fff640 1 -- 172.21.15.139:0/1974631577 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f1ad00aacd0 con 0x7f1ae00b9370 2024-04-25T10:03:25.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.039+0000 7f1adeffd640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1acc06b4d0 0x7f1acc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.039+0000 7f1adeffd640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1acc06b4d0 0x7f1acc06d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1ae01592c0 tx=0x7f1ad4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.043+0000 7f1ac7fff640 1 -- 172.21.15.139:0/1974631577 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ad00799a0 con 0x7f1ae00b9370 2024-04-25T10:03:25.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.212+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1974631577 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ae00d1680 con 0x7f1ae00b9370 2024-04-25T10:03:25.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.213+0000 7f1ac7fff640 1 -- 172.21.15.139:0/1974631577 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1ad00792a0 con 0x7f1ae00b9370 2024-04-25T10:03:25.218 INFO:teuthology.orchestra.run.smithi139.stdout:pg_num: 64 2024-04-25T10:03:25.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.215+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1974631577 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1acc06b4d0 msgr2=0x7f1acc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.215+0000 7f1ae5a6f640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1acc06b4d0 0x7f1acc06d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1ae01592c0 tx=0x7f1ad4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:25.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.215+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1974631577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae00b9370 msgr2=0x7f1ae00b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.215+0000 7f1ae5a6f640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae00b9370 0x7f1ae00b4c40 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0000c00 tx=0x7f1ad00046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.215+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1974631577 shutdown_connections 2024-04-25T10:03:25.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.215+0000 7f1ae5a6f640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1acc06b4d0 0x7f1acc06d990 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.215+0000 7f1ae5a6f640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ae015dbf0 0x7f1ae0160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.215+0000 7f1ae5a6f640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ae00cec80 0x7f1ae0160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.215+0000 7f1ae5a6f640 1 --2- 172.21.15.139:0/1974631577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ae00b9370 0x7f1ae00b4c40 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.215+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1974631577 >> 172.21.15.139:0/1974631577 conn(0x7f1ae00b5480 msgr2=0x7f1ae00caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.215+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1974631577 shutdown_connections 2024-04-25T10:03:25.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.215+0000 7f1ae5a6f640 1 -- 172.21.15.139:0/1974631577 wait complete. 2024-04-25T10:03:25.230 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:25.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.322+0000 7f3029ae1640 1 Processor -- start 2024-04-25T10:03:25.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.322+0000 7f3029ae1640 1 -- start start 2024-04-25T10:03:25.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.323+0000 7f3029ae1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f302415ac40 0x7f302406e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.323+0000 7f3029ae1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f302406e850 0x7f30240b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.323+0000 7f3029ae1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30240af6d0 con 0x7f302415a860 2024-04-25T10:03:25.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.323+0000 7f3029ae1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30240b02e0 con 0x7f302415ac40 2024-04-25T10:03:25.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.323+0000 7f3029ae1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30240af9b0 con 0x7f302406e850 2024-04-25T10:03:25.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.323+0000 7f301bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f302415ac40 0x7f302406e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.323+0000 7f301bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f302415ac40 0x7f302406e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39934/0 (socket says 172.21.15.139:39934) 2024-04-25T10:03:25.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.323+0000 7f301bfff640 1 -- 172.21.15.139:0/2575821003 learned_addr learned my addr 172.21.15.139:0/2575821003 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:25.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.323+0000 7f301bfff640 1 -- 172.21.15.139:0/2575821003 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f302406e850 msgr2=0x7f30240b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.323+0000 7f301bfff640 1 --2- 172.21.15.139:0/2575821003 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f302406e850 0x7f30240b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.323+0000 7f301bfff640 1 -- 172.21.15.139:0/2575821003 >> v1:172.21.15.150:6789/0 conn(0x7f302415a860 legacy=0x7f30240cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.323+0000 7f301bfff640 1 -- 172.21.15.139:0/2575821003 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30240afc90 con 0x7f302415ac40 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.324+0000 7f301bfff640 1 --2- 172.21.15.139:0/2575821003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f302415ac40 0x7f302406e310 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f300c009870 tx=0x7f300c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcb174e15a256c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.324+0000 7f3022ffd640 1 -- 172.21.15.139:0/2575821003 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c037070 con 0x7f302415ac40 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.324+0000 7f3022ffd640 1 -- 172.21.15.139:0/2575821003 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c032440 con 0x7f302415ac40 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.324+0000 7f3022ffd640 1 -- 172.21.15.139:0/2575821003 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c032810 con 0x7f302415ac40 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.325+0000 7f3029ae1640 1 -- 172.21.15.139:0/2575821003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f302415ac40 msgr2=0x7f302406e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.325+0000 7f3029ae1640 1 --2- 172.21.15.139:0/2575821003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f302415ac40 0x7f302406e310 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f300c009870 tx=0x7f300c02a210 comp rx=0 tx=0).stop 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.325+0000 7f3029ae1640 1 -- 172.21.15.139:0/2575821003 shutdown_connections 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.325+0000 7f3029ae1640 1 --2- 172.21.15.139:0/2575821003 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f302406e850 0x7f30240b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.325+0000 7f3029ae1640 1 --2- 172.21.15.139:0/2575821003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f302415ac40 0x7f302406e310 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.325+0000 7f3029ae1640 1 -- 172.21.15.139:0/2575821003 >> 172.21.15.139:0/2575821003 conn(0x7f3024066f20 msgr2=0x7f302406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.325+0000 7f3029ae1640 1 -- 172.21.15.139:0/2575821003 shutdown_connections 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.325+0000 7f3029ae1640 1 -- 172.21.15.139:0/2575821003 wait complete. 2024-04-25T10:03:25.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.326+0000 7f3029ae1640 1 Processor -- start 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.326+0000 7f3029ae1640 1 -- start start 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3029ae1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f302406e850 0x7f3024144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3029ae1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f302415a860 0x7f3024140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3029ae1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f302415ac40 0x7f3024149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3029ae1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30240af2a0 con 0x7f302415ac40 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3029ae1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30240cd3c0 con 0x7f302415a860 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3029ae1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30240af420 con 0x7f302406e850 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3023fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f302415ac40 0x7f3024149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3023fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f302415ac40 0x7f3024149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39944/0 (socket says 172.21.15.139:39944) 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3023fff640 1 -- 172.21.15.139:0/2813362799 learned_addr learned my addr 172.21.15.139:0/2813362799 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f30237fe640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f302406e850 0x7f3024144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3023fff640 1 -- 172.21.15.139:0/2813362799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f302406e850 msgr2=0x7f3024144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f301bfff640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f302415a860 0x7f3024140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3023fff640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f302406e850 0x7f3024144c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3023fff640 1 -- 172.21.15.139:0/2813362799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f302415a860 msgr2=0x7f3024140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3023fff640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f302415a860 0x7f3024140e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3023fff640 1 -- 172.21.15.139:0/2813362799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30241584a0 con 0x7f302415ac40 2024-04-25T10:03:25.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3023fff640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f302415ac40 0x7f3024149b90 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f301c00b970 tx=0x7f301c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.327+0000 7f3020ff9640 1 -- 172.21.15.139:0/2813362799 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c019070 con 0x7f302415ac40 2024-04-25T10:03:25.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.328+0000 7f3020ff9640 1 -- 172.21.15.139:0/2813362799 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301c00cb60 con 0x7f302415ac40 2024-04-25T10:03:25.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.328+0000 7f3020ff9640 1 -- 172.21.15.139:0/2813362799 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c014440 con 0x7f302415ac40 2024-04-25T10:03:25.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.328+0000 7f3029ae1640 1 -- 172.21.15.139:0/2813362799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3024145880 con 0x7f302415ac40 2024-04-25T10:03:25.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.328+0000 7f3029ae1640 1 -- 172.21.15.139:0/2813362799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3024158a20 con 0x7f302415ac40 2024-04-25T10:03:25.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.328+0000 7f3029ae1640 1 -- 172.21.15.139:0/2813362799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3024074df0 con 0x7f302415ac40 2024-04-25T10:03:25.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.329+0000 7f3020ff9640 1 -- 172.21.15.139:0/2813362799 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f301c01c070 con 0x7f302415ac40 2024-04-25T10:03:25.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.329+0000 7f3020ff9640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f300406b4d0 0x7f300406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.329+0000 7f3020ff9640 1 -- 172.21.15.139:0/2813362799 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f301c08a0a0 con 0x7f302415ac40 2024-04-25T10:03:25.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.329+0000 7f30237fe640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f300406b4d0 0x7f300406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.330+0000 7f30237fe640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f300406b4d0 0x7f300406d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f30240b4120 tx=0x7f3014009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.334+0000 7f3020ff9640 1 -- 172.21.15.139:0/2813362799 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3024074df0 con 0x7f302415ac40 2024-04-25T10:03:25.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.541+0000 7f3029ae1640 1 -- 172.21.15.139:0/2813362799 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30240b2590 con 0x7f302415ac40 2024-04-25T10:03:25.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.541+0000 7f3020ff9640 1 -- 172.21.15.139:0/2813362799 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f301c058d70 con 0x7f302415ac40 2024-04-25T10:03:25.546 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:25.546 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":7690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3195407462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3195407462},{"type":"v1","addr":"172.21.15.150:6835","nonce":3195407462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:03:17.394485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6836/1898344850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1898344850},{"type":"v1","addr":"172.21.15.139:6836","nonce":1898344850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:03:25.547 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 96 2024-04-25T10:03:25.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.543+0000 7f3029ae1640 1 -- 172.21.15.139:0/2813362799 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f300406b4d0 msgr2=0x7f300406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.543+0000 7f3029ae1640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f300406b4d0 0x7f300406d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f30240b4120 tx=0x7f3014009040 comp rx=0 tx=0).stop 2024-04-25T10:03:25.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.543+0000 7f3029ae1640 1 -- 172.21.15.139:0/2813362799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f302415ac40 msgr2=0x7f3024149b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.543+0000 7f3029ae1640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f302415ac40 0x7f3024149b90 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f301c00b970 tx=0x7f301c00be30 comp rx=0 tx=0).stop 2024-04-25T10:03:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.543+0000 7f3029ae1640 1 -- 172.21.15.139:0/2813362799 shutdown_connections 2024-04-25T10:03:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.543+0000 7f3029ae1640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f302415ac40 0x7f3024149b90 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.543+0000 7f3029ae1640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f300406b4d0 0x7f300406d990 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.543+0000 7f3029ae1640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f302415a860 0x7f3024140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.543+0000 7f3029ae1640 1 --2- 172.21.15.139:0/2813362799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f302406e850 0x7f3024144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.543+0000 7f3029ae1640 1 -- 172.21.15.139:0/2813362799 >> 172.21.15.139:0/2813362799 conn(0x7f3024066f20 msgr2=0x7f3024150210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.543+0000 7f3029ae1640 1 -- 172.21.15.139:0/2813362799 shutdown_connections 2024-04-25T10:03:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.543+0000 7f3029ae1640 1 -- 172.21.15.139:0/2813362799 wait complete. 2024-04-25T10:03:25.559 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:25.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.650+0000 7f323cbf9640 1 Processor -- start 2024-04-25T10:03:25.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.650+0000 7f323cbf9640 1 -- start start 2024-04-25T10:03:25.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.651+0000 7f323cbf9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f323806da20 0x7f323815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.651+0000 7f323cbf9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32380b3e10 con 0x7f3238158260 2024-04-25T10:03:25.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.651+0000 7f323cbf9640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32381615c0 con 0x7f32380b37d0 2024-04-25T10:03:25.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.651+0000 7f323cbf9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32381618a0 con 0x7f323806da20 2024-04-25T10:03:25.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.651+0000 7f323659c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3238158260 0x7f32380b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53972/0 (socket says 172.21.15.139:53972) 2024-04-25T10:03:25.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.651+0000 7f323659c640 1 -- 172.21.15.139:0/319839815 learned_addr learned my addr 172.21.15.139:0/319839815 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:25.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.651+0000 7f3236d9d640 1 --2- 172.21.15.139:0/319839815 >> v2:172.21.15.150:3300/0 conn(0x7f323806da20 0x7f323815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.651+0000 7f323559a640 1 -- 172.21.15.139:0/319839815 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678462984 0 0) 0x7f3228001490 con 0x7f3238158260 2024-04-25T10:03:25.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.651+0000 7f323559a640 1 -- 172.21.15.139:0/319839815 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3220003520 con 0x7f3238158260 2024-04-25T10:03:25.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.652+0000 7f323559a640 1 -- 172.21.15.139:0/319839815 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1163831217 0 0) 0x7f3220003520 con 0x7f3238158260 2024-04-25T10:03:25.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.652+0000 7f3236d9d640 1 -- 172.21.15.139:0/319839815 >> v1:172.21.15.150:6789/0 conn(0x7f32380b37d0 legacy=0x7f323806ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.652+0000 7f3236d9d640 1 -- 172.21.15.139:0/319839815 >> v1:172.21.15.139:6789/0 conn(0x7f3238158260 legacy=0x7f32380b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.652+0000 7f3236d9d640 1 -- 172.21.15.139:0/319839815 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238161b80 con 0x7f323806da20 2024-04-25T10:03:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.653+0000 7f3236d9d640 1 --2- 172.21.15.139:0/319839815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323806da20 0x7f323815df80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f32300097a0 tx=0x7f323002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a993a3df951472 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.653+0000 7f323559a640 1 -- 172.21.15.139:0/319839815 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230037070 con 0x7f323806da20 2024-04-25T10:03:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.653+0000 7f323559a640 1 -- 172.21.15.139:0/319839815 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323002ac50 con 0x7f323806da20 2024-04-25T10:03:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.653+0000 7f323559a640 1 -- 172.21.15.139:0/319839815 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228001bc0 con 0x7f323806da20 2024-04-25T10:03:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.653+0000 7f323cbf9640 1 -- 172.21.15.139:0/319839815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323806da20 msgr2=0x7f323815df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.653+0000 7f323cbf9640 1 --2- 172.21.15.139:0/319839815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323806da20 0x7f323815df80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f32300097a0 tx=0x7f323002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.653+0000 7f323cbf9640 1 -- 172.21.15.139:0/319839815 shutdown_connections 2024-04-25T10:03:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.653+0000 7f323cbf9640 1 --2- 172.21.15.139:0/319839815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323806da20 0x7f323815df80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.653+0000 7f323cbf9640 1 -- 172.21.15.139:0/319839815 >> 172.21.15.139:0/319839815 conn(0x7f32380b23d0 msgr2=0x7f32380cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.653+0000 7f323cbf9640 1 -- 172.21.15.139:0/319839815 shutdown_connections 2024-04-25T10:03:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.653+0000 7f323cbf9640 1 -- 172.21.15.139:0/319839815 wait complete. 2024-04-25T10:03:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.655+0000 7f323cbf9640 1 Processor -- start 2024-04-25T10:03:25.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.655+0000 7f323cbf9640 1 -- start start 2024-04-25T10:03:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.655+0000 7f323cbf9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323806da20 0x7f3238142710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.655+0000 7f323cbf9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32380b37d0 0x7f3238160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.655+0000 7f323cbf9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3238158260 0x7f323814f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.655+0000 7f323cbf9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3238162420 con 0x7f3238158260 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.655+0000 7f323cbf9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3238162140 con 0x7f323806da20 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.655+0000 7f323cbf9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323806b4b0 con 0x7f32380b37d0 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.655+0000 7f3236d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3238158260 0x7f323814f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.655+0000 7f3236d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3238158260 0x7f323814f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39948/0 (socket says 172.21.15.139:39948) 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.655+0000 7f3236d9d640 1 -- 172.21.15.139:0/4193314395 learned_addr learned my addr 172.21.15.139:0/4193314395 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f323659c640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323806da20 0x7f3238142710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f3235d9b640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32380b37d0 0x7f3238160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f3236d9d640 1 -- 172.21.15.139:0/4193314395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32380b37d0 msgr2=0x7f3238160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f3236d9d640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32380b37d0 0x7f3238160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f3236d9d640 1 -- 172.21.15.139:0/4193314395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323806da20 msgr2=0x7f3238142710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f3236d9d640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323806da20 0x7f3238142710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f3236d9d640 1 -- 172.21.15.139:0/4193314395 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238155d60 con 0x7f3238158260 2024-04-25T10:03:25.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f323659c640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323806da20 0x7f3238142710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:25.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f3236d9d640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3238158260 0x7f323814f5b0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f3230035040 tx=0x7f32300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f3226ffd640 1 -- 172.21.15.139:0/4193314395 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230041070 con 0x7f3238158260 2024-04-25T10:03:25.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f323cbf9640 1 -- 172.21.15.139:0/4193314395 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3238154fa0 con 0x7f3238158260 2024-04-25T10:03:25.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f3226ffd640 1 -- 172.21.15.139:0/4193314395 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323003c460 con 0x7f3238158260 2024-04-25T10:03:25.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f3226ffd640 1 -- 172.21.15.139:0/4193314395 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230037040 con 0x7f3238158260 2024-04-25T10:03:25.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.656+0000 7f323cbf9640 1 -- 172.21.15.139:0/4193314395 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3238155560 con 0x7f3238158260 2024-04-25T10:03:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.657+0000 7f323cbf9640 1 -- 172.21.15.139:0/4193314395 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3238074520 con 0x7f3238158260 2024-04-25T10:03:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.658+0000 7f3226ffd640 1 -- 172.21.15.139:0/4193314395 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3230044070 con 0x7f3238158260 2024-04-25T10:03:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.658+0000 7f3226ffd640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f321806b500 0x7f321806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.658+0000 7f3226ffd640 1 -- 172.21.15.139:0/4193314395 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f32300aa760 con 0x7f3238158260 2024-04-25T10:03:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.658+0000 7f323659c640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f321806b500 0x7f321806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.658+0000 7f323659c640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f321806b500 0x7f321806d9c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3228005c00 tx=0x7f3228009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.662+0000 7f3226ffd640 1 -- 172.21.15.139:0/4193314395 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3230079320 con 0x7f3238158260 2024-04-25T10:03:25.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.833+0000 7f323cbf9640 1 -- 172.21.15.139:0/4193314395 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3238160f90 con 0x7f3238158260 2024-04-25T10:03:25.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.834+0000 7f3226ffd640 1 -- 172.21.15.139:0/4193314395 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17263 (secure 0 0 0) 0x7f3230078c20 con 0x7f3238158260 2024-04-25T10:03:25.839 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:25.840 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":122,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:03:13.364415+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:48.574891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:49.293101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6836/3478297010":"2024-04-26T10:03:13.364392+0000","172.21.15.139:6834/3478297010":"2024-04-26T10:03:13.364392+0000","172.21.15.150:6835/1377607596":"2024-04-26T10:03:03.005923+0000","172.21.15.150:6833/1377607596":"2024-04-26T10:03:03.005923+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:25.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.836+0000 7f323cbf9640 1 -- 172.21.15.139:0/4193314395 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f321806b500 msgr2=0x7f321806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.836+0000 7f323cbf9640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f321806b500 0x7f321806d9c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3228005c00 tx=0x7f3228009040 comp rx=0 tx=0).stop 2024-04-25T10:03:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.836+0000 7f323cbf9640 1 -- 172.21.15.139:0/4193314395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3238158260 msgr2=0x7f323814f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.836+0000 7f323cbf9640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3238158260 0x7f323814f5b0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f3230035040 tx=0x7f32300046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.836+0000 7f323cbf9640 1 -- 172.21.15.139:0/4193314395 shutdown_connections 2024-04-25T10:03:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.836+0000 7f323cbf9640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3238158260 0x7f323814f5b0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.836+0000 7f323cbf9640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f321806b500 0x7f321806d9c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.836+0000 7f323cbf9640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32380b37d0 0x7f3238160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.836+0000 7f323cbf9640 1 --2- 172.21.15.139:0/4193314395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323806da20 0x7f3238142710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.836+0000 7f323cbf9640 1 -- 172.21.15.139:0/4193314395 >> 172.21.15.139:0/4193314395 conn(0x7f32380b23d0 msgr2=0x7f323806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.836+0000 7f323cbf9640 1 -- 172.21.15.139:0/4193314395 shutdown_connections 2024-04-25T10:03:25.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.836+0000 7f323cbf9640 1 -- 172.21.15.139:0/4193314395 wait complete. 2024-04-25T10:03:25.852 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:03:25.852 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:25.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.944+0000 7f3e82458640 1 Processor -- start 2024-04-25T10:03:25.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.944+0000 7f3e82458640 1 -- start start 2024-04-25T10:03:25.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.944+0000 7f3e82458640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e7c160e90 0x7f3e7c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.944+0000 7f3e82458640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7c070890 con 0x7f3e7c06ab60 2024-04-25T10:03:25.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.944+0000 7f3e82458640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7c0d0400 con 0x7f3e7c06afc0 2024-04-25T10:03:25.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.944+0000 7f3e82458640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0d14a0 con 0x7f3e7c160e90 2024-04-25T10:03:25.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3e7c06ab60 0x7f3e7c06fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53980/0 (socket says 172.21.15.139:53980) 2024-04-25T10:03:25.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7b7fe640 1 -- 172.21.15.139:0/709447000 learned_addr learned my addr 172.21.15.139:0/709447000 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:25.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654850753 0 0) 0x7f3e7c070890 con 0x7f3e7c06ab60 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7bfff640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e7c160e90 0x7f3e7c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e64003520 con 0x7f3e7c06ab60 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1876055116 0 0) 0x7f3e64003520 con 0x7f3e7c06ab60 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e7c160e90 msgr2=0x7f3e7c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e7c160e90 0x7f3e7c0afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 >> v1:172.21.15.150:6789/0 conn(0x7f3e7c06afc0 legacy=0x7f3e7c070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c0d06e0 con 0x7f3e7c06ab60 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f3e70001670 con 0x7f3e7c06ab60 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 >> v1:172.21.15.139:6789/0 conn(0x7f3e7c06ab60 legacy=0x7f3e7c06fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e640039e0 0x7f3e64003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e64004390 0x7f3e64004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e7c160e90 0x7f3e640050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0d14a0 con 0x7f3e640039e0 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6400b9c0 con 0x7f3e64004390 2024-04-25T10:03:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6400bbc0 con 0x7f3e7c160e90 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e7bfff640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e640039e0 0x7f3e64003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.945+0000 7f3e809f5640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e7c160e90 0x7f3e640050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.946+0000 7f3e7b7fe640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e64004390 0x7f3e64004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.946+0000 7f3e7bfff640 1 -- 172.21.15.139:0/709447000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e7c160e90 msgr2=0x7f3e640050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.946+0000 7f3e7bfff640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e7c160e90 0x7f3e640050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.946+0000 7f3e7bfff640 1 -- 172.21.15.139:0/709447000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e64004390 msgr2=0x7f3e64004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.946+0000 7f3e7bfff640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e64004390 0x7f3e64004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.946+0000 7f3e7bfff640 1 -- 172.21.15.139:0/709447000 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e64003520 con 0x7f3e640039e0 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.946+0000 7f3e7b7fe640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e64004390 0x7f3e64004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.946+0000 7f3e7bfff640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e640039e0 0x7f3e64003dc0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009ac0 tx=0x7f3e64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9971d32b19f8745e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.947+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c036070 con 0x7f3e640039e0 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.947+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c031400 con 0x7f3e640039e0 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.947+0000 7f3e7affd640 1 -- 172.21.15.139:0/709447000 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c009690 con 0x7f3e640039e0 2024-04-25T10:03:25.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.947+0000 7f3e82458640 1 -- 172.21.15.139:0/709447000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e640039e0 msgr2=0x7f3e64003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.947+0000 7f3e82458640 1 --2- 172.21.15.139:0/709447000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e640039e0 0x7f3e64003dc0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009ac0 tx=0x7f3e64005830 comp rx=0 tx=0).stop 2024-04-25T10:03:25.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.947+0000 7f3e809f5640 1 -- 172.21.15.139:0/709447000 reap_dead start 2024-04-25T10:03:25.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.947+0000 7f3e82458640 1 -- 172.21.15.139:0/709447000 shutdown_connections 2024-04-25T10:03:25.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.947+0000 7f3e82458640 1 -- 172.21.15.139:0/709447000 >> 172.21.15.139:0/709447000 conn(0x7f3e7c0b23f0 msgr2=0x7f3e7c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.947+0000 7f3e82458640 1 -- 172.21.15.139:0/709447000 shutdown_connections 2024-04-25T10:03:25.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.947+0000 7f3e82458640 1 -- 172.21.15.139:0/709447000 wait complete. 2024-04-25T10:03:25.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.948+0000 7f3e82458640 1 Processor -- start 2024-04-25T10:03:25.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.948+0000 7f3e82458640 1 -- start start 2024-04-25T10:03:25.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.949+0000 7f3e82458640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e640039e0 0x7f3e7c14b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.949+0000 7f3e82458640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e7c158280 0x7f3e7c06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.949+0000 7f3e82458640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e7c158700 0x7f3e7c06ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.949+0000 7f3e82458640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0d0f80 con 0x7f3e7c158280 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.949+0000 7f3e82458640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0d0ca0 con 0x7f3e7c158700 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.949+0000 7f3e82458640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c06b410 con 0x7f3e640039e0 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.949+0000 7f3e7b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e7c158280 0x7f3e7c06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.949+0000 7f3e7b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e7c158280 0x7f3e7c06fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39964/0 (socket says 172.21.15.139:39964) 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.949+0000 7f3e7b7fe640 1 -- 172.21.15.139:0/179847029 learned_addr learned my addr 172.21.15.139:0/179847029 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.949+0000 7f3e809f5640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e7c158700 0x7f3e7c06ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.949+0000 7f3e7bfff640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e640039e0 0x7f3e7c14b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.950+0000 7f3e7b7fe640 1 -- 172.21.15.139:0/179847029 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e640039e0 msgr2=0x7f3e7c14b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.950+0000 7f3e7b7fe640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e640039e0 0x7f3e7c14b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.950+0000 7f3e7b7fe640 1 -- 172.21.15.139:0/179847029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e7c158700 msgr2=0x7f3e7c06ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.950+0000 7f3e7b7fe640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e7c158700 0x7f3e7c06ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.950+0000 7f3e7b7fe640 1 -- 172.21.15.139:0/179847029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c1569f0 con 0x7f3e7c158280 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.950+0000 7f3e7bfff640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e640039e0 0x7f3e7c14b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.950+0000 7f3e7b7fe640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e7c158280 0x7f3e7c06fa70 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f3e70001200 tx=0x7f3e7000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.950+0000 7f3e78ff9640 1 -- 172.21.15.139:0/179847029 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7000c660 con 0x7f3e7c158280 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.950+0000 7f3e78ff9640 1 -- 172.21.15.139:0/179847029 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e70018070 con 0x7f3e7c158280 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.950+0000 7f3e82458640 1 -- 172.21.15.139:0/179847029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c156cd0 con 0x7f3e7c158280 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.950+0000 7f3e78ff9640 1 -- 172.21.15.139:0/179847029 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7000ca00 con 0x7f3e7c158280 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.950+0000 7f3e82458640 1 -- 172.21.15.139:0/179847029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c157290 con 0x7f3e7c158280 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.951+0000 7f3e82458640 1 -- 172.21.15.139:0/179847029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c074520 con 0x7f3e7c158280 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.951+0000 7f3e78ff9640 1 -- 172.21.15.139:0/179847029 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e7000cba0 con 0x7f3e7c158280 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.952+0000 7f3e78ff9640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e6806b4d0 0x7f3e6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.952+0000 7f3e78ff9640 1 -- 172.21.15.139:0/179847029 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f3e7008a0f0 con 0x7f3e7c158280 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.952+0000 7f3e7bfff640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e6806b4d0 0x7f3e6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.952+0000 7f3e7bfff640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e6806b4d0 0x7f3e6806d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009ac0 tx=0x7f3e6c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:25.956+0000 7f3e78ff9640 1 -- 172.21.15.139:0/179847029 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e70058dc0 con 0x7f3e7c158280 2024-04-25T10:03:26.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.162+0000 7f3e82458640 1 -- 172.21.15.139:0/179847029 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e7c0728b0 con 0x7f3e7c158280 2024-04-25T10:03:26.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.162+0000 7f3e78ff9640 1 -- 172.21.15.139:0/179847029 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f3e700586c0 con 0x7f3e7c158280 2024-04-25T10:03:26.167 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:26.167 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":7690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3195407462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3195407462},{"type":"v1","addr":"172.21.15.150:6835","nonce":3195407462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:03:17.394485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6836/1898344850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1898344850},{"type":"v1","addr":"172.21.15.139:6836","nonce":1898344850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:03:26.167 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 96 2024-04-25T10:03:26.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.164+0000 7f3e82458640 1 -- 172.21.15.139:0/179847029 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e6806b4d0 msgr2=0x7f3e6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.164+0000 7f3e82458640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e6806b4d0 0x7f3e6806d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009ac0 tx=0x7f3e6c008030 comp rx=0 tx=0).stop 2024-04-25T10:03:26.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.164+0000 7f3e82458640 1 -- 172.21.15.139:0/179847029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e7c158280 msgr2=0x7f3e7c06fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.164+0000 7f3e82458640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e7c158280 0x7f3e7c06fa70 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f3e70001200 tx=0x7f3e7000ec10 comp rx=0 tx=0).stop 2024-04-25T10:03:26.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.164+0000 7f3e82458640 1 -- 172.21.15.139:0/179847029 shutdown_connections 2024-04-25T10:03:26.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.164+0000 7f3e82458640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e7c158280 0x7f3e7c06fa70 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.164+0000 7f3e82458640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f3e6806b4d0 0x7f3e6806d990 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.164+0000 7f3e82458640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e7c158700 0x7f3e7c06ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.164+0000 7f3e82458640 1 --2- 172.21.15.139:0/179847029 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e640039e0 0x7f3e7c14b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.164+0000 7f3e82458640 1 -- 172.21.15.139:0/179847029 >> 172.21.15.139:0/179847029 conn(0x7f3e7c0b23f0 msgr2=0x7f3e7c0b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.164+0000 7f3e82458640 1 -- 172.21.15.139:0/179847029 shutdown_connections 2024-04-25T10:03:26.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.164+0000 7f3e82458640 1 -- 172.21.15.139:0/179847029 wait complete. 2024-04-25T10:03:26.180 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:26.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.272+0000 7f0f2545d640 1 Processor -- start 2024-04-25T10:03:26.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.272+0000 7f0f2545d640 1 -- start start 2024-04-25T10:03:26.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f2545d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20161850 0x7f0f200b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f2545d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f200cf250 con 0x7f0f2015dc60 2024-04-25T10:03:26.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f2545d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f200ce9b0 con 0x7f0f200d1080 2024-04-25T10:03:26.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f2545d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200d3730 con 0x7f0f20161850 2024-04-25T10:03:26.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f1effd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0f2015dc60 0x7f0f200d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:53994/0 (socket says 172.21.15.139:53994) 2024-04-25T10:03:26.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f1effd640 1 -- 172.21.15.139:0/213299438 learned_addr learned my addr 172.21.15.139:0/213299438 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:26.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089082717 0 0) 0x7f0f10001110 con 0x7f0f2015dc60 2024-04-25T10:03:26.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f08003520 con 0x7f0f2015dc60 2024-04-25T10:03:26.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f1f7fe640 1 --2- 172.21.15.139:0/213299438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20161850 0x7f0f200b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1687914035 0 0) 0x7f0f08003520 con 0x7f0f2015dc60 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20161850 msgr2=0x7f0f200b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f1dffb640 1 --2- 172.21.15.139:0/213299438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20161850 0x7f0f200b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 >> v1:172.21.15.150:6789/0 conn(0x7f0f200d1080 legacy=0x7f0f200d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f200cec90 con 0x7f0f2015dc60 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.273+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f0f10001840 con 0x7f0f2015dc60 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 >> v1:172.21.15.139:6789/0 conn(0x7f0f2015dc60 legacy=0x7f0f200d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1dffb640 1 --2- 172.21.15.139:0/213299438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f080039e0 0x7f0f08003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1dffb640 1 --2- 172.21.15.139:0/213299438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f08004800 0x7f0f08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1dffb640 1 --2- 172.21.15.139:0/213299438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20161850 0x7f0f080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200d3730 con 0x7f0f080039e0 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0800ba70 con 0x7f0f08004800 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0800bc70 con 0x7f0f20161850 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1effd640 1 --2- 172.21.15.139:0/213299438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f080039e0 0x7f0f08003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1f7fe640 1 --2- 172.21.15.139:0/213299438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20161850 0x7f0f080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1e7fc640 1 --2- 172.21.15.139:0/213299438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f08004800 0x7f0f08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1effd640 1 -- 172.21.15.139:0/213299438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20161850 msgr2=0x7f0f080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1effd640 1 --2- 172.21.15.139:0/213299438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20161850 0x7f0f080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1effd640 1 -- 172.21.15.139:0/213299438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f08004800 msgr2=0x7f0f08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1effd640 1 --2- 172.21.15.139:0/213299438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f08004800 0x7f0f08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.274+0000 7f0f1effd640 1 -- 172.21.15.139:0/213299438 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f08003520 con 0x7f0f080039e0 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.275+0000 7f0f1effd640 1 --2- 172.21.15.139:0/213299438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f080039e0 0x7f0f08003e00 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000c490 tx=0x7f0f08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d48889954decb08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.275+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1000c0e0 con 0x7f0f080039e0 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.275+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f10034070 con 0x7f0f080039e0 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.275+0000 7f0f1dffb640 1 -- 172.21.15.139:0/213299438 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10005d30 con 0x7f0f080039e0 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.275+0000 7f0f2545d640 1 -- 172.21.15.139:0/213299438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f080039e0 msgr2=0x7f0f08003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.275+0000 7f0f2545d640 1 --2- 172.21.15.139:0/213299438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f080039e0 0x7f0f08003e00 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000c490 tx=0x7f0f08005830 comp rx=0 tx=0).stop 2024-04-25T10:03:26.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.275+0000 7f0f1f7fe640 1 -- 172.21.15.139:0/213299438 reap_dead start 2024-04-25T10:03:26.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.275+0000 7f0f2545d640 1 -- 172.21.15.139:0/213299438 shutdown_connections 2024-04-25T10:03:26.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.275+0000 7f0f2545d640 1 -- 172.21.15.139:0/213299438 >> 172.21.15.139:0/213299438 conn(0x7f0f200b5910 msgr2=0x7f0f200b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.275+0000 7f0f2545d640 1 -- 172.21.15.139:0/213299438 shutdown_connections 2024-04-25T10:03:26.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.275+0000 7f0f2545d640 1 -- 172.21.15.139:0/213299438 wait complete. 2024-04-25T10:03:26.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.277+0000 7f0f2545d640 1 Processor -- start 2024-04-25T10:03:26.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.277+0000 7f0f2545d640 1 -- start start 2024-04-25T10:03:26.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.277+0000 7f0f2545d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f080039e0 0x7f0f20152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.277+0000 7f0f2545d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f20155160 0x7f0f200ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.277+0000 7f0f2545d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20156fc0 0x7f0f20157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.277+0000 7f0f2545d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2015e040 con 0x7f0f20155160 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.277+0000 7f0f2545d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200d1fc0 con 0x7f0f080039e0 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.277+0000 7f0f2545d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201562e0 con 0x7f0f20156fc0 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f1e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f20155160 0x7f0f200ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f1e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f20155160 0x7f0f200ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39992/0 (socket says 172.21.15.139:39992) 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f1e7fc640 1 -- 172.21.15.139:0/3904993268 learned_addr learned my addr 172.21.15.139:0/3904993268 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f1effd640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f080039e0 0x7f0f20152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f1f7fe640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20156fc0 0x7f0f20157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f1e7fc640 1 -- 172.21.15.139:0/3904993268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20156fc0 msgr2=0x7f0f20157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f1e7fc640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20156fc0 0x7f0f20157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f1e7fc640 1 -- 172.21.15.139:0/3904993268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f080039e0 msgr2=0x7f0f20152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f1e7fc640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f080039e0 0x7f0f20152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f1e7fc640 1 -- 172.21.15.139:0/3904993268 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f201556d0 con 0x7f0f20155160 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f1e7fc640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f20155160 0x7f0f200ccef0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400f620 tx=0x7f0f1400fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f077fe640 1 -- 172.21.15.139:0/3904993268 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f140052e0 con 0x7f0f20155160 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f077fe640 1 -- 172.21.15.139:0/3904993268 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f14018070 con 0x7f0f20155160 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f2545d640 1 -- 172.21.15.139:0/3904993268 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f201559b0 con 0x7f0f20155160 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f077fe640 1 -- 172.21.15.139:0/3904993268 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14003810 con 0x7f0f20155160 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.278+0000 7f0f2545d640 1 -- 172.21.15.139:0/3904993268 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f20155f70 con 0x7f0f20155160 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.279+0000 7f0f2545d640 1 -- 172.21.15.139:0/3904993268 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f20074520 con 0x7f0f20155160 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.280+0000 7f0f077fe640 1 -- 172.21.15.139:0/3904993268 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f140039b0 con 0x7f0f20155160 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.280+0000 7f0f077fe640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0f0c06b500 0x7f0f0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.280+0000 7f0f077fe640 1 -- 172.21.15.139:0/3904993268 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f0f1408a1c0 con 0x7f0f20155160 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.280+0000 7f0f1effd640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0f0c06b500 0x7f0f0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.280+0000 7f0f1effd640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0f0c06b500 0x7f0f0c06d9c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0f10033000 tx=0x7f0f1002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.284+0000 7f0f077fe640 1 -- 172.21.15.139:0/3904993268 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f14058e00 con 0x7f0f20155160 2024-04-25T10:03:26.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.489+0000 7f0f2545d640 1 -- 172.21.15.139:0/3904993268 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f20160e10 con 0x7f0f20155160 2024-04-25T10:03:26.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.489+0000 7f0f077fe640 1 -- 172.21.15.139:0/3904993268 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f0f14058700 con 0x7f0f20155160 2024-04-25T10:03:26.494 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:26.494 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":7690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3195407462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3195407462},{"type":"v1","addr":"172.21.15.150:6835","nonce":3195407462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:50.319874+0000","modified":"2024-04-25T10:03:17.394485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":4,"addr":"172.21.15.139:6836/1898344850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1898344850},{"type":"v1","addr":"172.21.15.139:6836","nonce":1898344850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:03:26.495 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 96 2024-04-25T10:03:26.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.491+0000 7f0f2545d640 1 -- 172.21.15.139:0/3904993268 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0f0c06b500 msgr2=0x7f0f0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.491+0000 7f0f2545d640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0f0c06b500 0x7f0f0c06d9c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0f10033000 tx=0x7f0f1002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:26.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.491+0000 7f0f2545d640 1 -- 172.21.15.139:0/3904993268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f20155160 msgr2=0x7f0f200ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.491+0000 7f0f2545d640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f20155160 0x7f0f200ccef0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400f620 tx=0x7f0f1400fae0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.491+0000 7f0f2545d640 1 -- 172.21.15.139:0/3904993268 shutdown_connections 2024-04-25T10:03:26.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.491+0000 7f0f2545d640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f20155160 0x7f0f200ccef0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.491+0000 7f0f2545d640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0f0c06b500 0x7f0f0c06d9c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.491+0000 7f0f2545d640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0f20156fc0 0x7f0f20157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.491+0000 7f0f2545d640 1 --2- 172.21.15.139:0/3904993268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f080039e0 0x7f0f20152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.491+0000 7f0f2545d640 1 -- 172.21.15.139:0/3904993268 >> 172.21.15.139:0/3904993268 conn(0x7f0f200b5910 msgr2=0x7f0f20160360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.491+0000 7f0f2545d640 1 -- 172.21.15.139:0/3904993268 shutdown_connections 2024-04-25T10:03:26.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.491+0000 7f0f2545d640 1 -- 172.21.15.139:0/3904993268 wait complete. 2024-04-25T10:03:26.507 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:26.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.600+0000 7f725c134640 1 Processor -- start 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.600+0000 7f725c134640 1 -- start start 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.601+0000 7f725c134640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f725415cde0 0x7f725415d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.601+0000 7f725c134640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f72540cc2a0 0x7f725409a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.601+0000 7f725c134640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725415d1c0 con 0x7f725415db50 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.601+0000 7f725c134640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72540cc720 con 0x7f72540cc2a0 2024-04-25T10:03:26.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.601+0000 7f725c134640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72540cd460 con 0x7f725415cde0 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.601+0000 7f72596cf640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f72540cc2a0 0x7f725409a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.601+0000 7f7259ed0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f725415cde0 0x7f725415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.601+0000 7f7259ed0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f725415cde0 0x7f725415d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:51630/0 (socket says 172.21.15.139:51630) 2024-04-25T10:03:26.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.601+0000 7f72596cf640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f72540cc2a0 0x7f725409a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:46274/0 (socket says 172.21.15.139:46274) 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.601+0000 7f7259ed0640 1 -- 172.21.15.139:0/1259204397 learned_addr learned my addr 172.21.15.139:0/1259204397 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.601+0000 7f725a6d1640 1 --1- 172.21.15.139:0/1259204397 >> v1:172.21.15.150:6789/0 conn(0x7f725415db50 0x7f72540cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:41046/0 (socket says 172.21.15.139:41046) 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.602+0000 7f72596cf640 1 -- 172.21.15.139:0/1259204397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f725415cde0 msgr2=0x7f725415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.602+0000 7f72596cf640 1 --2- 172.21.15.139:0/1259204397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f725415cde0 0x7f725415d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.602+0000 7f72596cf640 1 -- 172.21.15.139:0/1259204397 >> v1:172.21.15.150:6789/0 conn(0x7f725415db50 legacy=0x7f72540cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.602+0000 7f72596cf640 1 -- 172.21.15.139:0/1259204397 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7254099bf0 con 0x7f72540cc2a0 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.602+0000 7f7258ece640 1 -- 172.21.15.139:0/1259204397 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909059922 0 0) 0x7f725415d1c0 con 0x7f725415db50 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.602+0000 7f7259ed0640 1 --2- 172.21.15.139:0/1259204397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f725415cde0 0x7f725415d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.602+0000 7f72596cf640 1 --2- 172.21.15.139:0/1259204397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72540cc2a0 0x7f725409a3f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f72440097a0 tx=0x7f724402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86223d654b52c353 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.603+0000 7f7258ece640 1 -- 172.21.15.139:0/1259204397 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7244037070 con 0x7f72540cc2a0 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.603+0000 7f7258ece640 1 -- 172.21.15.139:0/1259204397 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7244032440 con 0x7f72540cc2a0 2024-04-25T10:03:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.603+0000 7f7258ece640 1 -- 172.21.15.139:0/1259204397 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7244032810 con 0x7f72540cc2a0 2024-04-25T10:03:26.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.603+0000 7f725c134640 1 -- 172.21.15.139:0/1259204397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72540cc2a0 msgr2=0x7f725409a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.603+0000 7f725c134640 1 --2- 172.21.15.139:0/1259204397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72540cc2a0 0x7f725409a3f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f72440097a0 tx=0x7f724402a180 comp rx=0 tx=0).stop 2024-04-25T10:03:26.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.603+0000 7f725c134640 1 -- 172.21.15.139:0/1259204397 shutdown_connections 2024-04-25T10:03:26.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.603+0000 7f725c134640 1 --2- 172.21.15.139:0/1259204397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72540cc2a0 0x7f725409a3f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.603+0000 7f725c134640 1 --2- 172.21.15.139:0/1259204397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f725415cde0 0x7f725415d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.603+0000 7f725c134640 1 -- 172.21.15.139:0/1259204397 >> 172.21.15.139:0/1259204397 conn(0x7f72540d1790 msgr2=0x7f72540d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.603+0000 7f725c134640 1 -- 172.21.15.139:0/1259204397 shutdown_connections 2024-04-25T10:03:26.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.603+0000 7f725c134640 1 -- 172.21.15.139:0/1259204397 wait complete. 2024-04-25T10:03:26.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.604+0000 7f725c134640 1 Processor -- start 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.604+0000 7f725c134640 1 -- start start 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.605+0000 7f725c134640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72540cc2a0 0x7f7254151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.605+0000 7f725c134640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f725415cde0 0x7f725409d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.605+0000 7f725c134640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f725415db50 0x7f72540c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.605+0000 7f725c134640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540cf4e0 con 0x7f725415db50 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.605+0000 7f725c134640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540cf360 con 0x7f72540cc2a0 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.605+0000 7f725c134640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725409e420 con 0x7f725415cde0 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.605+0000 7f725a6d1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f725415db50 0x7f72540c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.605+0000 7f725a6d1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f725415db50 0x7f72540c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42646/0 (socket says 172.21.15.139:42646) 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.605+0000 7f725a6d1640 1 -- 172.21.15.139:0/701284241 learned_addr learned my addr 172.21.15.139:0/701284241 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.605+0000 7f72596cf640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f725415cde0 0x7f725409d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.605+0000 7f7259ed0640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72540cc2a0 0x7f7254151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.606+0000 7f725a6d1640 1 -- 172.21.15.139:0/701284241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f725415cde0 msgr2=0x7f725409d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.606+0000 7f725a6d1640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f725415cde0 0x7f725409d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.606+0000 7f725a6d1640 1 -- 172.21.15.139:0/701284241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72540cc2a0 msgr2=0x7f7254151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.606+0000 7f725a6d1640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72540cc2a0 0x7f7254151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.606+0000 7f725a6d1640 1 -- 172.21.15.139:0/701284241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725409e700 con 0x7f725415db50 2024-04-25T10:03:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.606+0000 7f725a6d1640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f725415db50 0x7f72540c6540 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c730 tx=0x7f724c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.606+0000 7f724a7fc640 1 -- 172.21.15.139:0/701284241 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c018070 con 0x7f725415db50 2024-04-25T10:03:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.606+0000 7f724a7fc640 1 -- 172.21.15.139:0/701284241 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c00ce40 con 0x7f725415db50 2024-04-25T10:03:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.606+0000 7f725c134640 1 -- 172.21.15.139:0/701284241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725409e9e0 con 0x7f725415db50 2024-04-25T10:03:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.606+0000 7f724a7fc640 1 -- 172.21.15.139:0/701284241 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c00a860 con 0x7f725415db50 2024-04-25T10:03:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.606+0000 7f725c134640 1 -- 172.21.15.139:0/701284241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725409efa0 con 0x7f725415db50 2024-04-25T10:03:26.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.607+0000 7f725c134640 1 -- 172.21.15.139:0/701284241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7254074520 con 0x7f725415db50 2024-04-25T10:03:26.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.607+0000 7f724a7fc640 1 -- 172.21.15.139:0/701284241 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f724c01b070 con 0x7f725415db50 2024-04-25T10:03:26.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.608+0000 7f724a7fc640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f723806b4d0 0x7f723806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.608+0000 7f724a7fc640 1 -- 172.21.15.139:0/701284241 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f724c0898b0 con 0x7f725415db50 2024-04-25T10:03:26.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.608+0000 7f7259ed0640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f723806b4d0 0x7f723806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.608+0000 7f7259ed0640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f723806b4d0 0x7f723806d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7250006ca0 tx=0x7f7250009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.612+0000 7f724a7fc640 1 -- 172.21.15.139:0/701284241 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f724c057f30 con 0x7f725415db50 2024-04-25T10:03:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.782+0000 7f725c134640 1 -- 172.21.15.139:0/701284241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72540d25f0 con 0x7f725415db50 2024-04-25T10:03:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.784+0000 7f724a7fc640 1 -- 172.21.15.139:0/701284241 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17263 (secure 0 0 0) 0x7f724c05bfd0 con 0x7f725415db50 2024-04-25T10:03:26.789 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:26.789 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":122,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:03:13.364415+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:48.574891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:49.293101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6836/3478297010":"2024-04-26T10:03:13.364392+0000","172.21.15.139:6834/3478297010":"2024-04-26T10:03:13.364392+0000","172.21.15.150:6835/1377607596":"2024-04-26T10:03:03.005923+0000","172.21.15.150:6833/1377607596":"2024-04-26T10:03:03.005923+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:26.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.786+0000 7f725c134640 1 -- 172.21.15.139:0/701284241 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f723806b4d0 msgr2=0x7f723806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.786+0000 7f725c134640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f723806b4d0 0x7f723806d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7250006ca0 tx=0x7f7250009240 comp rx=0 tx=0).stop 2024-04-25T10:03:26.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.786+0000 7f725c134640 1 -- 172.21.15.139:0/701284241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f725415db50 msgr2=0x7f72540c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.786+0000 7f725c134640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f725415db50 0x7f72540c6540 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c730 tx=0x7f724c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.786+0000 7f725c134640 1 -- 172.21.15.139:0/701284241 shutdown_connections 2024-04-25T10:03:26.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.786+0000 7f725c134640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f725415db50 0x7f72540c6540 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.786+0000 7f725c134640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f723806b4d0 0x7f723806d990 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.786+0000 7f725c134640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f725415cde0 0x7f725409d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.786+0000 7f725c134640 1 --2- 172.21.15.139:0/701284241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72540cc2a0 0x7f7254151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.786+0000 7f725c134640 1 -- 172.21.15.139:0/701284241 >> 172.21.15.139:0/701284241 conn(0x7f72540d1790 msgr2=0x7f725415b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.786+0000 7f725c134640 1 -- 172.21.15.139:0/701284241 shutdown_connections 2024-04-25T10:03:26.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.786+0000 7f725c134640 1 -- 172.21.15.139:0/701284241 wait complete. 2024-04-25T10:03:26.802 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:03:26.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.895+0000 7f693e3f4640 1 Processor -- start 2024-04-25T10:03:26.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.896+0000 7f693e3f4640 1 -- start start 2024-04-25T10:03:26.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.896+0000 7f693e3f4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f69380cd610 0x7f69380d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.896+0000 7f693e3f4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6938099ca0 con 0x7f693809ad40 2024-04-25T10:03:26.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.896+0000 7f693e3f4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6938099f80 con 0x7f6938161fd0 2024-04-25T10:03:26.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.896+0000 7f693e3f4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6938099a50 con 0x7f69380cd610 2024-04-25T10:03:26.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.896+0000 7f69377fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f693809ad40 0x7f69381618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34538/0 (socket says 172.21.15.139:34538) 2024-04-25T10:03:26.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.896+0000 7f69377fe640 1 -- 172.21.15.139:0/3953895997 learned_addr learned my addr 172.21.15.139:0/3953895997 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:26.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6937fff640 1 --2- 172.21.15.139:0/3953895997 >> v2:172.21.15.150:3300/0 conn(0x7f69380cd610 0x7f69380d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101467810 0 0) 0x7f6938099ca0 con 0x7f693809ad40 2024-04-25T10:03:26.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6920003520 con 0x7f693809ad40 2024-04-25T10:03:26.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 932259738 0 0) 0x7f6920003520 con 0x7f693809ad40 2024-04-25T10:03:26.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 >> v2:172.21.15.150:3300/0 conn(0x7f69380cd610 msgr2=0x7f69380d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 --2- 172.21.15.139:0/3953895997 >> v2:172.21.15.150:3300/0 conn(0x7f69380cd610 0x7f69380d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 >> v1:172.21.15.150:6789/0 conn(0x7f6938161fd0 legacy=0x7f69380cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693809a260 con 0x7f693809ad40 2024-04-25T10:03:26.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f692c001670 con 0x7f693809ad40 2024-04-25T10:03:26.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 >> v1:172.21.15.139:6789/0 conn(0x7f693809ad40 legacy=0x7f69381618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 --2- 172.21.15.139:0/3953895997 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69200039b0 0x7f6920003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 --2- 172.21.15.139:0/3953895997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69200043b0 0x7f6920004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 --2- 172.21.15.139:0/3953895997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 0x7f692000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6938099a50 con 0x7f6920005080 2024-04-25T10:03:26.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692000bf30 con 0x7f69200043b0 2024-04-25T10:03:26.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692000c130 con 0x7f69200039b0 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f693c991640 1 --2- 172.21.15.139:0/3953895997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 0x7f692000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f6937fff640 1 --2- 172.21.15.139:0/3953895997 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69200039b0 0x7f6920003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f69377fe640 1 --2- 172.21.15.139:0/3953895997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69200043b0 0x7f6920004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f693c991640 1 -- 172.21.15.139:0/3953895997 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69200039b0 msgr2=0x7f6920003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f693c991640 1 --2- 172.21.15.139:0/3953895997 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69200039b0 0x7f6920003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f693c991640 1 -- 172.21.15.139:0/3953895997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69200043b0 msgr2=0x7f6920004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f693c991640 1 --2- 172.21.15.139:0/3953895997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69200043b0 0x7f6920004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f693c991640 1 -- 172.21.15.139:0/3953895997 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6920003520 con 0x7f6920005080 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.897+0000 7f69377fe640 1 --2- 172.21.15.139:0/3953895997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69200043b0 0x7f6920004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.898+0000 7f693c991640 1 -- 172.21.15.139:0/3953895997 reap_dead start 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.898+0000 7f693c991640 1 --2- 172.21.15.139:0/3953895997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 0x7f692000b9c0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f6930001f30 tx=0x7f6920005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209e80cc09ba43ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.898+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930001b30 con 0x7f6920005080 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.898+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6930031600 con 0x7f6920005080 2024-04-25T10:03:26.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.898+0000 7f6936ffd640 1 -- 172.21.15.139:0/3953895997 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69300319f0 con 0x7f6920005080 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.899+0000 7f693e3f4640 1 -- 172.21.15.139:0/3953895997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 msgr2=0x7f692000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.899+0000 7f693e3f4640 1 --2- 172.21.15.139:0/3953895997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 0x7f692000b9c0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f6930001f30 tx=0x7f6920005830 comp rx=0 tx=0).stop 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.899+0000 7f693e3f4640 1 -- 172.21.15.139:0/3953895997 shutdown_connections 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.899+0000 7f693e3f4640 1 --2- 172.21.15.139:0/3953895997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 0x7f692000b9c0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.899+0000 7f693e3f4640 1 -- 172.21.15.139:0/3953895997 >> 172.21.15.139:0/3953895997 conn(0x7f69380b31f0 msgr2=0x7f69380b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.899+0000 7f693e3f4640 1 -- 172.21.15.139:0/3953895997 shutdown_connections 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.899+0000 7f693e3f4640 1 -- 172.21.15.139:0/3953895997 wait complete. 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.900+0000 7f693e3f4640 1 Processor -- start 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.900+0000 7f693e3f4640 1 -- start start 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f693e3f4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 0x7f693815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f693e3f4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69380c7dd0 0x7f69380cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f693e3f4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69380c8250 0x7f69380ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f693e3f4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69380cd9f0 con 0x7f6920005080 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f693e3f4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693809a820 con 0x7f69380c7dd0 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f693e3f4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69380cdb70 con 0x7f69380c8250 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f6937fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 0x7f693815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f6937fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 0x7f693815cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42656/0 (socket says 172.21.15.139:42656) 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f6937fff640 1 -- 172.21.15.139:0/965508380 learned_addr learned my addr 172.21.15.139:0/965508380 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f693c991640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69380c8250 0x7f69380ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f69377fe640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69380c7dd0 0x7f69380cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f6937fff640 1 -- 172.21.15.139:0/965508380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69380c8250 msgr2=0x7f69380ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f6937fff640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69380c8250 0x7f69380ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f6937fff640 1 -- 172.21.15.139:0/965508380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69380c7dd0 msgr2=0x7f69380cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f6937fff640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69380c7dd0 0x7f69380cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f6937fff640 1 -- 172.21.15.139:0/965508380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693809cc70 con 0x7f6920005080 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.901+0000 7f6937fff640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 0x7f693815cf30 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f69280118e0 tx=0x7f6928011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.902+0000 7f6934ff9640 1 -- 172.21.15.139:0/965508380 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692800faf0 con 0x7f6920005080 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.902+0000 7f6934ff9640 1 -- 172.21.15.139:0/965508380 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692800fc90 con 0x7f6920005080 2024-04-25T10:03:26.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.902+0000 7f693e3f4640 1 -- 172.21.15.139:0/965508380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693809cf50 con 0x7f6920005080 2024-04-25T10:03:26.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.902+0000 7f6934ff9640 1 -- 172.21.15.139:0/965508380 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928014440 con 0x7f6920005080 2024-04-25T10:03:26.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.902+0000 7f693e3f4640 1 -- 172.21.15.139:0/965508380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693809d510 con 0x7f6920005080 2024-04-25T10:03:26.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.903+0000 7f693e3f4640 1 -- 172.21.15.139:0/965508380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6938067dc0 con 0x7f6920005080 2024-04-25T10:03:26.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.903+0000 7f6934ff9640 1 -- 172.21.15.139:0/965508380 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6928012040 con 0x7f6920005080 2024-04-25T10:03:26.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.903+0000 7f6934ff9640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f692406b4d0 0x7f692406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.904+0000 7f6934ff9640 1 -- 172.21.15.139:0/965508380 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f692808aa70 con 0x7f6920005080 2024-04-25T10:03:26.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.904+0000 7f69377fe640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f692406b4d0 0x7f692406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.904+0000 7f69377fe640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f692406b4d0 0x7f692406d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f692c001eb0 tx=0x7f692c001e00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:26.908+0000 7f6934ff9640 1 -- 172.21.15.139:0/965508380 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6938067dc0 con 0x7f6920005080 2024-04-25T10:03:27.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.113+0000 7f693e3f4640 1 -- 172.21.15.139:0/965508380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6938074520 con 0x7f6920005080 2024-04-25T10:03:27.119 INFO:tasks.ceph.mon.a.smithi139.stderr:2024-04-25T10:03:27.114+0000 7f73a6828640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:03:27.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.115+0000 7f6934ff9640 1 -- 172.21.15.139:0/965508380 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6928019030 con 0x7f6920005080 2024-04-25T10:03:27.120 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:03:27.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.118+0000 7f693e3f4640 1 -- 172.21.15.139:0/965508380 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f692406b4d0 msgr2=0x7f692406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.118+0000 7f693e3f4640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f692406b4d0 0x7f692406d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f692c001eb0 tx=0x7f692c001e00 comp rx=0 tx=0).stop 2024-04-25T10:03:27.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.118+0000 7f693e3f4640 1 -- 172.21.15.139:0/965508380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 msgr2=0x7f693815cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.118+0000 7f693e3f4640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 0x7f693815cf30 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f69280118e0 tx=0x7f6928011da0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.118+0000 7f693e3f4640 1 -- 172.21.15.139:0/965508380 shutdown_connections 2024-04-25T10:03:27.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.118+0000 7f693e3f4640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f692406b4d0 0x7f692406d990 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.118+0000 7f693e3f4640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69380c8250 0x7f69380ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.118+0000 7f693e3f4640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69380c7dd0 0x7f69380cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.118+0000 7f693e3f4640 1 --2- 172.21.15.139:0/965508380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6920005080 0x7f693815cf30 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.118+0000 7f693e3f4640 1 -- 172.21.15.139:0/965508380 >> 172.21.15.139:0/965508380 conn(0x7f69380b31f0 msgr2=0x7f6938073840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:27.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.118+0000 7f693e3f4640 1 -- 172.21.15.139:0/965508380 shutdown_connections 2024-04-25T10:03:27.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.118+0000 7f693e3f4640 1 -- 172.21.15.139:0/965508380 wait complete. 2024-04-25T10:03:27.124 INFO:tasks.ceph.mds.a.smithi139.stderr: -1181> 2024-04-25T10:03:19.375+0000 7f26626c5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:27.135 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:03:27.148 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-04-25T10:03:27.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.229+0000 7fb2283c8640 1 Processor -- start 2024-04-25T10:03:27.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.229+0000 7fb2283c8640 1 -- start start 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb2283c8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb220144090 0x7fb2200b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb2283c8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2200b2a50 con 0x7fb2200cc9e0 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb2283c8640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22015b090 con 0x7fb22006ea40 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb2283c8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22015b370 con 0x7fb220144090 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb225963640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb2200cc9e0 0x7fb22006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34542/0 (socket says 172.21.15.139:34542) 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb225963640 1 -- 172.21.15.139:0/572440546 learned_addr learned my addr 172.21.15.139:0/572440546 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb226164640 1 --2- 172.21.15.139:0/572440546 >> v2:172.21.15.150:3300/0 conn(0x7fb220144090 0x7fb2200b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb225162640 1 -- 172.21.15.139:0/572440546 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250046660 0 0) 0x7fb210001110 con 0x7fb2200cc9e0 2024-04-25T10:03:27.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb225162640 1 -- 172.21.15.139:0/572440546 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb20c003520 con 0x7fb2200cc9e0 2024-04-25T10:03:27.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb225162640 1 -- 172.21.15.139:0/572440546 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282076092 0 0) 0x7fb22015b090 con 0x7fb22006ea40 2024-04-25T10:03:27.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb225162640 1 -- 172.21.15.139:0/572440546 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb210001110 con 0x7fb22006ea40 2024-04-25T10:03:27.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb226164640 1 -- 172.21.15.139:0/572440546 >> v1:172.21.15.150:6789/0 conn(0x7fb22006ea40 legacy=0x7fb22015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb226164640 1 -- 172.21.15.139:0/572440546 >> v1:172.21.15.139:6789/0 conn(0x7fb2200cc9e0 legacy=0x7fb22006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb225162640 1 -- 172.21.15.139:0/572440546 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3296888945 0 0) 0x7fb20c003520 con 0x7fb2200cc9e0 2024-04-25T10:03:27.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.230+0000 7fb226164640 1 -- 172.21.15.139:0/572440546 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22015b650 con 0x7fb220144090 2024-04-25T10:03:27.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.231+0000 7fb226164640 1 --2- 172.21.15.139:0/572440546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb220144090 0x7fb2200b31a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0097a0 tx=0x7fb21c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72d386d1ef242cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:27.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.231+0000 7fb225162640 1 -- 172.21.15.139:0/572440546 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c037070 con 0x7fb220144090 2024-04-25T10:03:27.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.231+0000 7fb225162640 1 -- 172.21.15.139:0/572440546 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c02abc0 con 0x7fb220144090 2024-04-25T10:03:27.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.231+0000 7fb225162640 1 -- 172.21.15.139:0/572440546 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210001840 con 0x7fb220144090 2024-04-25T10:03:27.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.232+0000 7fb2283c8640 1 -- 172.21.15.139:0/572440546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb220144090 msgr2=0x7fb2200b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.232+0000 7fb2283c8640 1 --2- 172.21.15.139:0/572440546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb220144090 0x7fb2200b31a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0097a0 tx=0x7fb21c02a130 comp rx=0 tx=0).stop 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.232+0000 7fb2283c8640 1 -- 172.21.15.139:0/572440546 shutdown_connections 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.232+0000 7fb2283c8640 1 --2- 172.21.15.139:0/572440546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb220144090 0x7fb2200b31a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.232+0000 7fb2283c8640 1 -- 172.21.15.139:0/572440546 >> 172.21.15.139:0/572440546 conn(0x7fb220067000 msgr2=0x7fb22006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.232+0000 7fb2283c8640 1 -- 172.21.15.139:0/572440546 shutdown_connections 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.232+0000 7fb2283c8640 1 -- 172.21.15.139:0/572440546 wait complete. 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.233+0000 7fb2283c8640 1 Processor -- start 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.233+0000 7fb2283c8640 1 -- start start 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb2283c8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb22006ea40 0x7fb22015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb2283c8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2200cc9e0 0x7fb2201407b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb2283c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb220144090 0x7fb2200b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb2283c8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb220144470 con 0x7fb220144090 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb2283c8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22015bc10 con 0x7fb2200cc9e0 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb2283c8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22015ad80 con 0x7fb22006ea40 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb226965640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb220144090 0x7fb2200b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb226965640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb220144090 0x7fb2200b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42700/0 (socket says 172.21.15.139:42700) 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb226965640 1 -- 172.21.15.139:0/3621989079 learned_addr learned my addr 172.21.15.139:0/3621989079 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb226164640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb22006ea40 0x7fb22015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb225963640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2200cc9e0 0x7fb2201407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb226965640 1 -- 172.21.15.139:0/3621989079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb22006ea40 msgr2=0x7fb22015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb226965640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb22006ea40 0x7fb22015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb226965640 1 -- 172.21.15.139:0/3621989079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2200cc9e0 msgr2=0x7fb2201407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb226965640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2200cc9e0 0x7fb2201407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb226965640 1 -- 172.21.15.139:0/3621989079 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22015a6e0 con 0x7fb220144090 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb226965640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb220144090 0x7fb2200b31a0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fb21800c6b0 tx=0x7fb21800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.234+0000 7fb216ffd640 1 -- 172.21.15.139:0/3621989079 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218003b70 con 0x7fb220144090 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.235+0000 7fb216ffd640 1 -- 172.21.15.139:0/3621989079 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb218003d10 con 0x7fb220144090 2024-04-25T10:03:27.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.235+0000 7fb216ffd640 1 -- 172.21.15.139:0/3621989079 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218005310 con 0x7fb220144090 2024-04-25T10:03:27.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.235+0000 7fb2283c8640 1 -- 172.21.15.139:0/3621989079 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22013e360 con 0x7fb220144090 2024-04-25T10:03:27.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.235+0000 7fb2283c8640 1 -- 172.21.15.139:0/3621989079 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22013e920 con 0x7fb220144090 2024-04-25T10:03:27.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.235+0000 7fb2283c8640 1 -- 172.21.15.139:0/3621989079 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb220074520 con 0x7fb220144090 2024-04-25T10:03:27.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.236+0000 7fb216ffd640 1 -- 172.21.15.139:0/3621989079 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2180054b0 con 0x7fb220144090 2024-04-25T10:03:27.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.236+0000 7fb216ffd640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb20406b4d0 0x7fb20406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.236+0000 7fb216ffd640 1 -- 172.21.15.139:0/3621989079 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6359+0+0 (secure 0 0 0) 0x7fb21808a240 con 0x7fb220144090 2024-04-25T10:03:27.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.236+0000 7fb226164640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb20406b4d0 0x7fb20406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.237+0000 7fb226164640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb20406b4d0 0x7fb20406d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb21c034000 tx=0x7fb21c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:27.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.240+0000 7fb216ffd640 1 -- 172.21.15.139:0/3621989079 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb218058760 con 0x7fb220144090 2024-04-25T10:03:27.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:27.448+0000 7fb2283c8640 1 -- 172.21.15.139:0/3621989079 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb2201f3370 con 0x7fb220144090 2024-04-25T10:03:28.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.116+0000 7fb216ffd640 1 -- 172.21.15.139:0/3621989079 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v98) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb21805c800 con 0x7fb220144090 2024-04-25T10:03:28.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.120+0000 7fb2283c8640 1 -- 172.21.15.139:0/3621989079 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb20406b4d0 msgr2=0x7fb20406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.120+0000 7fb2283c8640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb20406b4d0 0x7fb20406d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb21c034000 tx=0x7fb21c02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:28.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.120+0000 7fb2283c8640 1 -- 172.21.15.139:0/3621989079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb220144090 msgr2=0x7fb2200b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.120+0000 7fb2283c8640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb220144090 0x7fb2200b31a0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fb21800c6b0 tx=0x7fb21800cb70 comp rx=0 tx=0).stop 2024-04-25T10:03:28.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.120+0000 7fb2283c8640 1 -- 172.21.15.139:0/3621989079 shutdown_connections 2024-04-25T10:03:28.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.120+0000 7fb2283c8640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb220144090 0x7fb2200b31a0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.120+0000 7fb2283c8640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb20406b4d0 0x7fb20406d990 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.120+0000 7fb2283c8640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2200cc9e0 0x7fb2201407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.120+0000 7fb2283c8640 1 --2- 172.21.15.139:0/3621989079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb22006ea40 0x7fb22015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.120+0000 7fb2283c8640 1 -- 172.21.15.139:0/3621989079 >> 172.21.15.139:0/3621989079 conn(0x7fb220067000 msgr2=0x7fb22006d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:28.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.120+0000 7fb2283c8640 1 -- 172.21.15.139:0/3621989079 shutdown_connections 2024-04-25T10:03:28.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.120+0000 7fb2283c8640 1 -- 172.21.15.139:0/3621989079 wait complete. 2024-04-25T10:03:28.138 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:03:28.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.281+0000 7f367f091640 1 Processor -- start 2024-04-25T10:03:28.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.281+0000 7f367f091640 1 -- start start 2024-04-25T10:03:28.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367f091640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f367815a870 0x7f36780b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367f091640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3678070970 0x7f36780d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367f091640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36780b45f0 con 0x7f36780b4190 2024-04-25T10:03:28.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367f091640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36780d1020 con 0x7f367815a870 2024-04-25T10:03:28.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367f091640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36780b5190 con 0x7f3678070970 2024-04-25T10:03:28.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367ce2d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f367815a870 0x7f36780b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367ce2d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f367815a870 0x7f36780b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42716/0 (socket says 172.21.15.139:42716) 2024-04-25T10:03:28.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f3677fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f36780b4190 0x7f3678070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34556/0 (socket says 172.21.15.139:34556) 2024-04-25T10:03:28.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367ce2d640 1 -- 172.21.15.139:0/459037990 learned_addr learned my addr 172.21.15.139:0/459037990 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:28.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367d62e640 1 --2- 172.21.15.139:0/459037990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3678070970 0x7f36780d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367ce2d640 1 -- 172.21.15.139:0/459037990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3678070970 msgr2=0x7f36780d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367ce2d640 1 --2- 172.21.15.139:0/459037990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3678070970 0x7f36780d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367ce2d640 1 -- 172.21.15.139:0/459037990 >> v1:172.21.15.139:6789/0 conn(0x7f36780b4190 legacy=0x7f3678070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f36777fe640 1 -- 172.21.15.139:0/459037990 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685066217 0 0) 0x7f36780b45f0 con 0x7f36780b4190 2024-04-25T10:03:28.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.282+0000 7f367ce2d640 1 -- 172.21.15.139:0/459037990 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36780d0d40 con 0x7f367815a870 2024-04-25T10:03:28.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.283+0000 7f367ce2d640 1 --2- 172.21.15.139:0/459037990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f367815a870 0x7f36780b5620 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f3668009870 tx=0x7f366802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7051127573e331eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.283+0000 7f36777fe640 1 -- 172.21.15.139:0/459037990 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668037070 con 0x7f367815a870 2024-04-25T10:03:28.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.283+0000 7f36777fe640 1 -- 172.21.15.139:0/459037990 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3668032440 con 0x7f367815a870 2024-04-25T10:03:28.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.283+0000 7f36777fe640 1 -- 172.21.15.139:0/459037990 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668032810 con 0x7f367815a870 2024-04-25T10:03:28.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.283+0000 7f367f091640 1 -- 172.21.15.139:0/459037990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f367815a870 msgr2=0x7f36780b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.283+0000 7f367f091640 1 --2- 172.21.15.139:0/459037990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f367815a870 0x7f36780b5620 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f3668009870 tx=0x7f366802a130 comp rx=0 tx=0).stop 2024-04-25T10:03:28.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.284+0000 7f367f091640 1 -- 172.21.15.139:0/459037990 shutdown_connections 2024-04-25T10:03:28.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.284+0000 7f367f091640 1 --2- 172.21.15.139:0/459037990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3678070970 0x7f36780d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.284+0000 7f367f091640 1 --2- 172.21.15.139:0/459037990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f367815a870 0x7f36780b5620 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.284+0000 7f367f091640 1 -- 172.21.15.139:0/459037990 >> 172.21.15.139:0/459037990 conn(0x7f36780b23f0 msgr2=0x7f36780b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:28.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.284+0000 7f367f091640 1 -- 172.21.15.139:0/459037990 shutdown_connections 2024-04-25T10:03:28.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.284+0000 7f367f091640 1 -- 172.21.15.139:0/459037990 wait complete. 2024-04-25T10:03:28.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.285+0000 7f367f091640 1 Processor -- start 2024-04-25T10:03:28.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.285+0000 7f367f091640 1 -- start start 2024-04-25T10:03:28.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f367f091640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3678070970 0x7f3678141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f367f091640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36780b4190 0x7f367815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f367f091640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367815a870 0x7f367815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f367f091640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36780ce610 con 0x7f36780b4190 2024-04-25T10:03:28.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f367f091640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36780b4e20 con 0x7f367815a870 2024-04-25T10:03:28.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f367f091640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36780ce790 con 0x7f3678070970 2024-04-25T10:03:28.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f3677fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36780b4190 0x7f367815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f3677fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36780b4190 0x7f367815d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42728/0 (socket says 172.21.15.139:42728) 2024-04-25T10:03:28.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f3677fff640 1 -- 172.21.15.139:0/3817489818 learned_addr learned my addr 172.21.15.139:0/3817489818 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:28.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f367ce2d640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3678070970 0x7f3678141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f367d62e640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367815a870 0x7f367815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f3677fff640 1 -- 172.21.15.139:0/3817489818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3678070970 msgr2=0x7f3678141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f3677fff640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3678070970 0x7f3678141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f3677fff640 1 -- 172.21.15.139:0/3817489818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367815a870 msgr2=0x7f367815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f3677fff640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367815a870 0x7f367815ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f3677fff640 1 -- 172.21.15.139:0/3817489818 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3678148220 con 0x7f36780b4190 2024-04-25T10:03:28.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f367ce2d640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3678070970 0x7f3678141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:28.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f3677fff640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36780b4190 0x7f367815d430 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f366c00c4d0 tx=0x7f366c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f36757fa640 1 -- 172.21.15.139:0/3817489818 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c018070 con 0x7f36780b4190 2024-04-25T10:03:28.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f36757fa640 1 -- 172.21.15.139:0/3817489818 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c007500 con 0x7f36780b4190 2024-04-25T10:03:28.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f367f091640 1 -- 172.21.15.139:0/3817489818 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3678148500 con 0x7f36780b4190 2024-04-25T10:03:28.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f36757fa640 1 -- 172.21.15.139:0/3817489818 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c004a20 con 0x7f36780b4190 2024-04-25T10:03:28.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.286+0000 7f367f091640 1 -- 172.21.15.139:0/3817489818 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3678148ac0 con 0x7f36780b4190 2024-04-25T10:03:28.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.287+0000 7f367f091640 1 -- 172.21.15.139:0/3817489818 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3678074520 con 0x7f36780b4190 2024-04-25T10:03:28.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.288+0000 7f36757fa640 1 -- 172.21.15.139:0/3817489818 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f366c01b070 con 0x7f36780b4190 2024-04-25T10:03:28.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.288+0000 7f36757fa640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f366406b4d0 0x7f366406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.288+0000 7f36757fa640 1 -- 172.21.15.139:0/3817489818 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6359+0+0 (secure 0 0 0) 0x7f366c089750 con 0x7f36780b4190 2024-04-25T10:03:28.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.288+0000 7f367ce2d640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f366406b4d0 0x7f366406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.289+0000 7f367ce2d640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f366406b4d0 0x7f366406d990 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3668034000 tx=0x7f366802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.292+0000 7f36757fa640 1 -- 172.21.15.139:0/3817489818 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f366c0583d0 con 0x7f36780b4190 2024-04-25T10:03:28.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:28.464+0000 7f367f091640 1 -- 172.21.15.139:0/3817489818 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f36780b2ca0 con 0x7f36780b4190 2024-04-25T10:03:29.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.433+0000 7f36757fa640 1 -- 172.21.15.139:0/3817489818 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v124) v1 ==== 181+0+0 (secure 0 0 0) 0x7f366c05e0d0 con 0x7f36780b4190 2024-04-25T10:03:29.438 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:03:29.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.436+0000 7f367f091640 1 -- 172.21.15.139:0/3817489818 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f366406b4d0 msgr2=0x7f366406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.436+0000 7f367f091640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f366406b4d0 0x7f366406d990 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3668034000 tx=0x7f366802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:29.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.436+0000 7f367f091640 1 -- 172.21.15.139:0/3817489818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36780b4190 msgr2=0x7f367815d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.436+0000 7f367f091640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36780b4190 0x7f367815d430 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f366c00c4d0 tx=0x7f366c00c990 comp rx=0 tx=0).stop 2024-04-25T10:03:29.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.436+0000 7f367f091640 1 -- 172.21.15.139:0/3817489818 shutdown_connections 2024-04-25T10:03:29.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.436+0000 7f367f091640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36780b4190 0x7f367815d430 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.436+0000 7f367f091640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f366406b4d0 0x7f366406d990 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.436+0000 7f367f091640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367815a870 0x7f367815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.436+0000 7f367f091640 1 --2- 172.21.15.139:0/3817489818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3678070970 0x7f3678141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.436+0000 7f367f091640 1 -- 172.21.15.139:0/3817489818 >> 172.21.15.139:0/3817489818 conn(0x7f36780b23f0 msgr2=0x7f36780b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:29.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.437+0000 7f367f091640 1 -- 172.21.15.139:0/3817489818 shutdown_connections 2024-04-25T10:03:29.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.437+0000 7f367f091640 1 -- 172.21.15.139:0/3817489818 wait complete. 2024-04-25T10:03:29.455 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:03:29.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.558+0000 7ff32dbd5640 1 Processor -- start 2024-04-25T10:03:29.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.558+0000 7ff32dbd5640 1 -- start start 2024-04-25T10:03:29.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff32dbd5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff3280d0a80 0x7ff3281618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff32dbd5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff328099ca0 con 0x7ff32809ad40 2024-04-25T10:03:29.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff32dbd5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff328099f80 con 0x7ff3280cd610 2024-04-25T10:03:29.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff32dbd5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff328099a50 con 0x7ff3280d0a80 2024-04-25T10:03:29.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff327fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff32809ad40 0x7ff3280cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34560/0 (socket says 172.21.15.139:34560) 2024-04-25T10:03:29.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff327fff640 1 -- 172.21.15.139:0/1091114665 learned_addr learned my addr 172.21.15.139:0/1091114665 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152746290 0 0) 0x7ff328099ca0 con 0x7ff32809ad40 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff326ffd640 1 --2- 172.21.15.139:0/1091114665 >> v2:172.21.15.150:3300/0 conn(0x7ff3280d0a80 0x7ff3281618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff310003520 con 0x7ff32809ad40 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3628067499 0 0) 0x7ff310003520 con 0x7ff32809ad40 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 >> v2:172.21.15.150:3300/0 conn(0x7ff3280d0a80 msgr2=0x7ff3281618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff3267fc640 1 --2- 172.21.15.139:0/1091114665 >> v2:172.21.15.150:3300/0 conn(0x7ff3280d0a80 0x7ff3281618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 >> v1:172.21.15.150:6789/0 conn(0x7ff3280cd610 legacy=0x7ff3280d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32809a260 con 0x7ff32809ad40 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7ff3200019f0 con 0x7ff32809ad40 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.559+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 >> v1:172.21.15.139:6789/0 conn(0x7ff32809ad40 legacy=0x7ff3280cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff3267fc640 1 --2- 172.21.15.139:0/1091114665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3100039b0 0x7ff310003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff3267fc640 1 --2- 172.21.15.139:0/1091114665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff310004800 0x7ff310004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff3267fc640 1 --2- 172.21.15.139:0/1091114665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3100051c0 0x7ff31000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff328099a50 con 0x7ff3100039b0 2024-04-25T10:03:29.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31000c0d0 con 0x7ff310004800 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31000c2d0 con 0x7ff3100051c0 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff3277fe640 1 --2- 172.21.15.139:0/1091114665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3100039b0 0x7ff310003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff327fff640 1 --2- 172.21.15.139:0/1091114665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3100051c0 0x7ff31000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff326ffd640 1 --2- 172.21.15.139:0/1091114665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff310004800 0x7ff310004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff3277fe640 1 -- 172.21.15.139:0/1091114665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3100051c0 msgr2=0x7ff31000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff3277fe640 1 --2- 172.21.15.139:0/1091114665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3100051c0 0x7ff31000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff3277fe640 1 -- 172.21.15.139:0/1091114665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff310004800 msgr2=0x7ff310004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff3277fe640 1 --2- 172.21.15.139:0/1091114665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff310004800 0x7ff310004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff3277fe640 1 -- 172.21.15.139:0/1091114665 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff310003520 con 0x7ff3100039b0 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.560+0000 7ff327fff640 1 -- 172.21.15.139:0/1091114665 reap_dead start 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.561+0000 7ff3277fe640 1 --2- 172.21.15.139:0/1091114665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3100039b0 0x7ff310003db0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7ff318003db0 tx=0x7ff310005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527374ecea366eae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.561+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318037070 con 0x7ff3100039b0 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.561+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31802a9f0 con 0x7ff3100039b0 2024-04-25T10:03:29.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.561+0000 7ff3267fc640 1 -- 172.21.15.139:0/1091114665 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318003980 con 0x7ff3100039b0 2024-04-25T10:03:29.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.561+0000 7ff32dbd5640 1 -- 172.21.15.139:0/1091114665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3100039b0 msgr2=0x7ff310003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.561+0000 7ff32dbd5640 1 --2- 172.21.15.139:0/1091114665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3100039b0 0x7ff310003db0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7ff318003db0 tx=0x7ff310005830 comp rx=0 tx=0).stop 2024-04-25T10:03:29.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.561+0000 7ff32dbd5640 1 -- 172.21.15.139:0/1091114665 shutdown_connections 2024-04-25T10:03:29.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.561+0000 7ff32dbd5640 1 --2- 172.21.15.139:0/1091114665 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3100039b0 0x7ff310003db0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.561+0000 7ff32dbd5640 1 -- 172.21.15.139:0/1091114665 >> 172.21.15.139:0/1091114665 conn(0x7ff3280b31f0 msgr2=0x7ff3280b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:29.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.561+0000 7ff32dbd5640 1 -- 172.21.15.139:0/1091114665 shutdown_connections 2024-04-25T10:03:29.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.561+0000 7ff32dbd5640 1 -- 172.21.15.139:0/1091114665 wait complete. 2024-04-25T10:03:29.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.562+0000 7ff32dbd5640 1 Processor -- start 2024-04-25T10:03:29.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.563+0000 7ff32dbd5640 1 -- start start 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.563+0000 7ff32dbd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3100039b0 0x7ff32809e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.563+0000 7ff32dbd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3280b6e90 0x7ff32809c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.563+0000 7ff32dbd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3281f6b00 0x7ff3281f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.563+0000 7ff32dbd5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280cd9f0 con 0x7ff3280b6e90 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.563+0000 7ff32dbd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32809a820 con 0x7ff3100039b0 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.563+0000 7ff32dbd5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3280cdb70 con 0x7ff3281f6b00 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.563+0000 7ff326ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3280b6e90 0x7ff32809c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff326ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3280b6e90 0x7ff32809c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42754/0 (socket says 172.21.15.139:42754) 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff326ffd640 1 -- 172.21.15.139:0/522795076 learned_addr learned my addr 172.21.15.139:0/522795076 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff3277fe640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3100039b0 0x7ff32809e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff327fff640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3281f6b00 0x7ff3281f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff326ffd640 1 -- 172.21.15.139:0/522795076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3281f6b00 msgr2=0x7ff3281f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff326ffd640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3281f6b00 0x7ff3281f6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff326ffd640 1 -- 172.21.15.139:0/522795076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3100039b0 msgr2=0x7ff32809e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff326ffd640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3100039b0 0x7ff32809e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff326ffd640 1 -- 172.21.15.139:0/522795076 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32809c830 con 0x7ff3280b6e90 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff326ffd640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3280b6e90 0x7ff32809c2f0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00f600 tx=0x7ff31c00fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff30ffff640 1 -- 172.21.15.139:0/522795076 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c019020 con 0x7ff3280b6e90 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff30ffff640 1 -- 172.21.15.139:0/522795076 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c014870 con 0x7ff3280b6e90 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff32dbd5640 1 -- 172.21.15.139:0/522795076 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32809eed0 con 0x7ff3280b6e90 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff30ffff640 1 -- 172.21.15.139:0/522795076 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c014c20 con 0x7ff3280b6e90 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.564+0000 7ff32dbd5640 1 -- 172.21.15.139:0/522795076 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32815f900 con 0x7ff3280b6e90 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.565+0000 7ff32dbd5640 1 -- 172.21.15.139:0/522795076 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff328074520 con 0x7ff3280b6e90 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.566+0000 7ff30ffff640 1 -- 172.21.15.139:0/522795076 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff31c01c020 con 0x7ff3280b6e90 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.566+0000 7ff30ffff640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff31406b4d0 0x7ff31406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.566+0000 7ff30ffff640 1 -- 172.21.15.139:0/522795076 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5921+0+0 (secure 0 0 0) 0x7ff31c08a9d0 con 0x7ff3280b6e90 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.566+0000 7ff3277fe640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff31406b4d0 0x7ff31406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.566+0000 7ff3277fe640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff31406b4d0 0x7ff31406d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff318034000 tx=0x7ff31802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:29.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.570+0000 7ff30ffff640 1 -- 172.21.15.139:0/522795076 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff31c023020 con 0x7ff3280b6e90 2024-04-25T10:03:29.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:29.746+0000 7ff32dbd5640 1 -- 172.21.15.139:0/522795076 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff3280b3f70 con 0x7ff3280b6e90 2024-04-25T10:03:30.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.434+0000 7ff30ffff640 1 -- 172.21.15.139:0/522795076 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v125) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff31c059800 con 0x7ff3280b6e90 2024-04-25T10:03:30.439 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' removed 2024-04-25T10:03:30.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.437+0000 7ff32dbd5640 1 -- 172.21.15.139:0/522795076 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff31406b4d0 msgr2=0x7ff31406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.437+0000 7ff32dbd5640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff31406b4d0 0x7ff31406d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff318034000 tx=0x7ff31802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:30.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.438+0000 7ff32dbd5640 1 -- 172.21.15.139:0/522795076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3280b6e90 msgr2=0x7ff32809c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.438+0000 7ff32dbd5640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3280b6e90 0x7ff32809c2f0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00f600 tx=0x7ff31c00fac0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.438+0000 7ff32dbd5640 1 -- 172.21.15.139:0/522795076 shutdown_connections 2024-04-25T10:03:30.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.438+0000 7ff32dbd5640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3280b6e90 0x7ff32809c2f0 secure :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00f600 tx=0x7ff31c00fac0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.438+0000 7ff32dbd5640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff31406b4d0 0x7ff31406d990 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.438+0000 7ff32dbd5640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3281f6b00 0x7ff3281f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.438+0000 7ff32dbd5640 1 --2- 172.21.15.139:0/522795076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3100039b0 0x7ff32809e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.438+0000 7ff32dbd5640 1 -- 172.21.15.139:0/522795076 >> 172.21.15.139:0/522795076 conn(0x7ff3280b31f0 msgr2=0x7ff3280709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:30.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.438+0000 7ff32dbd5640 1 -- 172.21.15.139:0/522795076 shutdown_connections 2024-04-25T10:03:30.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.438+0000 7ff32dbd5640 1 -- 172.21.15.139:0/522795076 wait complete. 2024-04-25T10:03:30.456 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:03:30.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.552+0000 7f1d8eb17640 1 Processor -- start 2024-04-25T10:03:30.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.552+0000 7f1d8eb17640 1 -- start start 2024-04-25T10:03:30.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8eb17640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1d8815dc90 0x7f1d880ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8eb17640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806f300 0x7f1d880b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8eb17640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d880cf980 con 0x7f1d880cf3e0 2024-04-25T10:03:30.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8eb17640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8806ab60 con 0x7f1d8815dc90 2024-04-25T10:03:30.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8eb17640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8806ae40 con 0x7f1d8806f300 2024-04-25T10:03:30.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8d0b4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806f300 0x7f1d880b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8c8b3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1d8815dc90 0x7f1d880ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8c8b3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1d8815dc90 0x7f1d880ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:46400/0 (socket says 172.21.15.139:46400) 2024-04-25T10:03:30.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8d0b4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806f300 0x7f1d880b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:51726/0 (socket says 172.21.15.139:51726) 2024-04-25T10:03:30.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8c8b3640 1 -- 172.21.15.139:0/2617009607 learned_addr learned my addr 172.21.15.139:0/2617009607 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:30.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d87fff640 1 --1- 172.21.15.139:0/2617009607 >> v1:172.21.15.150:6789/0 conn(0x7f1d880cf3e0 0x7f1d8806ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:41102/0 (socket says 172.21.15.139:41102) 2024-04-25T10:03:30.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8c8b3640 1 -- 172.21.15.139:0/2617009607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806f300 msgr2=0x7f1d880b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:30.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8c8b3640 1 --2- 172.21.15.139:0/2617009607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806f300 0x7f1d880b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8c8b3640 1 -- 172.21.15.139:0/2617009607 >> v1:172.21.15.150:6789/0 conn(0x7f1d880cf3e0 legacy=0x7f1d8806ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.553+0000 7f1d8c8b3640 1 -- 172.21.15.139:0/2617009607 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8806b120 con 0x7f1d8815dc90 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.554+0000 7f1d8c8b3640 1 --2- 172.21.15.139:0/2617009607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d8815dc90 0x7f1d880ceea0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009870 tx=0x7f1d7c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40f4b06ffd05999a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.554+0000 7f1d877fe640 1 -- 172.21.15.139:0/2617009607 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c037070 con 0x7f1d8815dc90 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.554+0000 7f1d877fe640 1 -- 172.21.15.139:0/2617009607 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c032440 con 0x7f1d8815dc90 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.554+0000 7f1d877fe640 1 -- 172.21.15.139:0/2617009607 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c032810 con 0x7f1d8815dc90 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.555+0000 7f1d8eb17640 1 -- 172.21.15.139:0/2617009607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d8815dc90 msgr2=0x7f1d880ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.555+0000 7f1d8eb17640 1 --2- 172.21.15.139:0/2617009607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d8815dc90 0x7f1d880ceea0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009870 tx=0x7f1d7c02a270 comp rx=0 tx=0).stop 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.555+0000 7f1d8eb17640 1 -- 172.21.15.139:0/2617009607 shutdown_connections 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.555+0000 7f1d8eb17640 1 --2- 172.21.15.139:0/2617009607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806f300 0x7f1d880b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.555+0000 7f1d8eb17640 1 --2- 172.21.15.139:0/2617009607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d8815dc90 0x7f1d880ceea0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.555+0000 7f1d8eb17640 1 -- 172.21.15.139:0/2617009607 >> 172.21.15.139:0/2617009607 conn(0x7f1d880cfd80 msgr2=0x7f1d880d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.555+0000 7f1d8eb17640 1 -- 172.21.15.139:0/2617009607 shutdown_connections 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.555+0000 7f1d8eb17640 1 -- 172.21.15.139:0/2617009607 wait complete. 2024-04-25T10:03:30.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.556+0000 7f1d8eb17640 1 Processor -- start 2024-04-25T10:03:30.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.556+0000 7f1d8eb17640 1 -- start start 2024-04-25T10:03:30.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8eb17640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806f300 0x7f1d881e9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8eb17640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d880cf3e0 0x7f1d881e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8eb17640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8815dc90 0x7f1d88148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8eb17640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8806b920 con 0x7f1d8815dc90 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8eb17640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d88072620 con 0x7f1d880cf3e0 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8eb17640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880b32f0 con 0x7f1d8806f300 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8d0b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8815dc90 0x7f1d88148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8d0b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8815dc90 0x7f1d88148b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42764/0 (socket says 172.21.15.139:42764) 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8d0b4640 1 -- 172.21.15.139:0/4188994254 learned_addr learned my addr 172.21.15.139:0/4188994254 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d87fff640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d880cf3e0 0x7f1d881e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8c8b3640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806f300 0x7f1d881e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8d0b4640 1 -- 172.21.15.139:0/4188994254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806f300 msgr2=0x7f1d881e9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8d0b4640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806f300 0x7f1d881e9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8d0b4640 1 -- 172.21.15.139:0/4188994254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d880cf3e0 msgr2=0x7f1d881e6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8d0b4640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d880cf3e0 0x7f1d881e6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8d0b4640 1 -- 172.21.15.139:0/4188994254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d88147a90 con 0x7f1d8815dc90 2024-04-25T10:03:30.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.557+0000 7f1d8d0b4640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8815dc90 0x7f1d88148b30 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000efc0 tx=0x7f1d8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.558+0000 7f1d857fa640 1 -- 172.21.15.139:0/4188994254 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80018020 con 0x7f1d8815dc90 2024-04-25T10:03:30.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.558+0000 7f1d857fa640 1 -- 172.21.15.139:0/4188994254 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d80009b10 con 0x7f1d8815dc90 2024-04-25T10:03:30.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.558+0000 7f1d8eb17640 1 -- 172.21.15.139:0/4188994254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d88147d70 con 0x7f1d8815dc90 2024-04-25T10:03:30.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.558+0000 7f1d857fa640 1 -- 172.21.15.139:0/4188994254 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800146c0 con 0x7f1d8815dc90 2024-04-25T10:03:30.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.558+0000 7f1d8eb17640 1 -- 172.21.15.139:0/4188994254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d88148330 con 0x7f1d8815dc90 2024-04-25T10:03:30.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.559+0000 7f1d8eb17640 1 -- 172.21.15.139:0/4188994254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d88074520 con 0x7f1d8815dc90 2024-04-25T10:03:30.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.559+0000 7f1d857fa640 1 -- 172.21.15.139:0/4188994254 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d80005d10 con 0x7f1d8815dc90 2024-04-25T10:03:30.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.559+0000 7f1d857fa640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d6c06b500 0x7f1d6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.559+0000 7f1d857fa640 1 -- 172.21.15.139:0/4188994254 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f1d8008aac0 con 0x7f1d8815dc90 2024-04-25T10:03:30.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.560+0000 7f1d8c8b3640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d6c06b500 0x7f1d6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.560+0000 7f1d8c8b3640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d6c06b500 0x7f1d6c06d9c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c034000 tx=0x7f1d7c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.564+0000 7f1d857fa640 1 -- 172.21.15.139:0/4188994254 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d80022070 con 0x7f1d8815dc90 2024-04-25T10:03:30.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.732+0000 7f1d8eb17640 1 -- 172.21.15.139:0/4188994254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1d880b4e00 con 0x7f1d8815dc90 2024-04-25T10:03:30.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.736+0000 7f1d857fa640 1 -- 172.21.15.139:0/4188994254 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1d80010040 con 0x7f1d8815dc90 2024-04-25T10:03:30.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.738+0000 7f1d8eb17640 1 -- 172.21.15.139:0/4188994254 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d6c06b500 msgr2=0x7f1d6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.738+0000 7f1d8eb17640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d6c06b500 0x7f1d6c06d9c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c034000 tx=0x7f1d7c02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:30.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.738+0000 7f1d8eb17640 1 -- 172.21.15.139:0/4188994254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8815dc90 msgr2=0x7f1d88148b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.738+0000 7f1d8eb17640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8815dc90 0x7f1d88148b30 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000efc0 tx=0x7f1d8000c490 comp rx=0 tx=0).stop 2024-04-25T10:03:30.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.738+0000 7f1d8eb17640 1 -- 172.21.15.139:0/4188994254 shutdown_connections 2024-04-25T10:03:30.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.738+0000 7f1d8eb17640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d8815dc90 0x7f1d88148b30 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.738+0000 7f1d8eb17640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f1d6c06b500 0x7f1d6c06d9c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.738+0000 7f1d8eb17640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d880cf3e0 0x7f1d881e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.738+0000 7f1d8eb17640 1 --2- 172.21.15.139:0/4188994254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1d8806f300 0x7f1d881e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.738+0000 7f1d8eb17640 1 -- 172.21.15.139:0/4188994254 >> 172.21.15.139:0/4188994254 conn(0x7f1d880cfd80 msgr2=0x7f1d880b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:30.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.738+0000 7f1d8eb17640 1 -- 172.21.15.139:0/4188994254 shutdown_connections 2024-04-25T10:03:30.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.738+0000 7f1d8eb17640 1 -- 172.21.15.139:0/4188994254 wait complete. 2024-04-25T10:03:30.753 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T10:03:30.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.849+0000 7fef9b438640 1 Processor -- start 2024-04-25T10:03:30.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.849+0000 7fef9b438640 1 -- start start 2024-04-25T10:03:30.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef9b438640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fef9415b5e0 0x7fef9415df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef9b438640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fef940b2db0 0x7fef940b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef9b438640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef940b3b80 con 0x7fef940b3770 2024-04-25T10:03:30.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef9b438640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9406da20 con 0x7fef940b2db0 2024-04-25T10:03:30.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef9b438640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9406f860 con 0x7fef9415b5e0 2024-04-25T10:03:30.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef989d3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fef940b2db0 0x7fef940b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef999d5640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fef940b3770 0x7fef94161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34572/0 (socket says 172.21.15.139:34572) 2024-04-25T10:03:30.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef989d3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fef940b2db0 0x7fef940b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42780/0 (socket says 172.21.15.139:42780) 2024-04-25T10:03:30.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef999d5640 1 -- 172.21.15.139:0/1458357983 learned_addr learned my addr 172.21.15.139:0/1458357983 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:30.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef991d4640 1 --2- 172.21.15.139:0/1458357983 >> v2:172.21.15.150:3300/0 conn(0x7fef9415b5e0 0x7fef9415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef989d3640 1 -- 172.21.15.139:0/1458357983 >> v2:172.21.15.150:3300/0 conn(0x7fef9415b5e0 msgr2=0x7fef9415df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:30.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef989d3640 1 --2- 172.21.15.139:0/1458357983 >> v2:172.21.15.150:3300/0 conn(0x7fef9415b5e0 0x7fef9415df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef989d3640 1 -- 172.21.15.139:0/1458357983 >> v1:172.21.15.139:6789/0 conn(0x7fef940b3770 legacy=0x7fef94161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef989d3640 1 -- 172.21.15.139:0/1458357983 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9406dc50 con 0x7fef940b2db0 2024-04-25T10:03:30.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.850+0000 7fef8bfff640 1 -- 172.21.15.139:0/1458357983 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159790226 0 0) 0x7fef8c001490 con 0x7fef940b3770 2024-04-25T10:03:30.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.851+0000 7fef989d3640 1 --2- 172.21.15.139:0/1458357983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef940b2db0 0x7fef940b3230 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fef94066e00 tx=0x7fef8402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d109e0367a9d6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.851+0000 7fef8bfff640 1 -- 172.21.15.139:0/1458357983 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84037070 con 0x7fef940b2db0 2024-04-25T10:03:30.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.851+0000 7fef8bfff640 1 -- 172.21.15.139:0/1458357983 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84032440 con 0x7fef940b2db0 2024-04-25T10:03:30.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.851+0000 7fef8bfff640 1 -- 172.21.15.139:0/1458357983 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8402ae60 con 0x7fef940b2db0 2024-04-25T10:03:30.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.851+0000 7fef9b438640 1 -- 172.21.15.139:0/1458357983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef940b2db0 msgr2=0x7fef940b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.851+0000 7fef9b438640 1 --2- 172.21.15.139:0/1458357983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef940b2db0 0x7fef940b3230 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fef94066e00 tx=0x7fef8402a110 comp rx=0 tx=0).stop 2024-04-25T10:03:30.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.852+0000 7fef9b438640 1 -- 172.21.15.139:0/1458357983 shutdown_connections 2024-04-25T10:03:30.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.852+0000 7fef9b438640 1 --2- 172.21.15.139:0/1458357983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef940b2db0 0x7fef940b3230 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.852+0000 7fef9b438640 1 --2- 172.21.15.139:0/1458357983 >> v2:172.21.15.150:3300/0 conn(0x7fef9415b5e0 0x7fef9415df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.852+0000 7fef9b438640 1 -- 172.21.15.139:0/1458357983 >> 172.21.15.139:0/1458357983 conn(0x7fef940b23d0 msgr2=0x7fef940cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:30.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.852+0000 7fef9b438640 1 -- 172.21.15.139:0/1458357983 shutdown_connections 2024-04-25T10:03:30.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.852+0000 7fef9b438640 1 -- 172.21.15.139:0/1458357983 wait complete. 2024-04-25T10:03:30.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.853+0000 7fef9b438640 1 Processor -- start 2024-04-25T10:03:30.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.853+0000 7fef9b438640 1 -- start start 2024-04-25T10:03:30.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.853+0000 7fef9b438640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef940b2db0 0x7fef94149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.853+0000 7fef9b438640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef940b3770 0x7fef940ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.853+0000 7fef9b438640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9415b5e0 0x7fef940cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.853+0000 7fef9b438640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9406e4f0 con 0x7fef9415b5e0 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.853+0000 7fef9b438640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9406e210 con 0x7fef940b2db0 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.853+0000 7fef9b438640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9406e7d0 con 0x7fef940b3770 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef999d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9415b5e0 0x7fef940cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef999d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9415b5e0 0x7fef940cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42788/0 (socket says 172.21.15.139:42788) 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef999d5640 1 -- 172.21.15.139:0/663046854 learned_addr learned my addr 172.21.15.139:0/663046854 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef999d5640 1 -- 172.21.15.139:0/663046854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef940b3770 msgr2=0x7fef940ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef999d5640 1 --2- 172.21.15.139:0/663046854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef940b3770 0x7fef940ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef999d5640 1 -- 172.21.15.139:0/663046854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef940b2db0 msgr2=0x7fef94149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef999d5640 1 --2- 172.21.15.139:0/663046854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef940b2db0 0x7fef94149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef999d5640 1 -- 172.21.15.139:0/663046854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef94156f20 con 0x7fef9415b5e0 2024-04-25T10:03:30.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef999d5640 1 --2- 172.21.15.139:0/663046854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9415b5e0 0x7fef940cc730 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00c7f0 tx=0x7fef8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef89ffb640 1 -- 172.21.15.139:0/663046854 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c018070 con 0x7fef9415b5e0 2024-04-25T10:03:30.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef89ffb640 1 -- 172.21.15.139:0/663046854 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c00a5e0 con 0x7fef9415b5e0 2024-04-25T10:03:30.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef89ffb640 1 -- 172.21.15.139:0/663046854 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00a9d0 con 0x7fef9415b5e0 2024-04-25T10:03:30.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef9b438640 1 -- 172.21.15.139:0/663046854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef940ad760 con 0x7fef9415b5e0 2024-04-25T10:03:30.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.854+0000 7fef9b438640 1 -- 172.21.15.139:0/663046854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef940add20 con 0x7fef9415b5e0 2024-04-25T10:03:30.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.855+0000 7fef9b438640 1 -- 172.21.15.139:0/663046854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef940b4980 con 0x7fef9415b5e0 2024-04-25T10:03:30.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.856+0000 7fef89ffb640 1 -- 172.21.15.139:0/663046854 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef8c01b070 con 0x7fef9415b5e0 2024-04-25T10:03:30.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.856+0000 7fef89ffb640 1 --2- 172.21.15.139:0/663046854 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef7806b4d0 0x7fef7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.856+0000 7fef89ffb640 1 -- 172.21.15.139:0/663046854 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fef8c0896d0 con 0x7fef9415b5e0 2024-04-25T10:03:30.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.856+0000 7fef991d4640 1 --2- 172.21.15.139:0/663046854 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef7806b4d0 0x7fef7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.856+0000 7fef991d4640 1 --2- 172.21.15.139:0/663046854 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef7806b4d0 0x7fef7806d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fef90006fd0 tx=0x7fef90008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:30.860+0000 7fef89ffb640 1 -- 172.21.15.139:0/663046854 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef940b4980 con 0x7fef9415b5e0 2024-04-25T10:03:31.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.098+0000 7fef9b438640 1 -- 172.21.15.139:0/663046854 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7fef94074520 con 0x7fef9415b5e0 2024-04-25T10:03:31.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.447+0000 7fef89ffb640 1 -- 172.21.15.139:0/663046854 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v484) v1 ==== 144+0+0 (secure 0 0 0) 0x7fef94156f20 con 0x7fef9415b5e0 2024-04-25T10:03:31.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.449+0000 7fef9b438640 1 -- 172.21.15.139:0/663046854 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef7806b4d0 msgr2=0x7fef7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.449+0000 7fef9b438640 1 --2- 172.21.15.139:0/663046854 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef7806b4d0 0x7fef7806d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fef90006fd0 tx=0x7fef90008040 comp rx=0 tx=0).stop 2024-04-25T10:03:31.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.449+0000 7fef9b438640 1 -- 172.21.15.139:0/663046854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9415b5e0 msgr2=0x7fef940cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.449+0000 7fef9b438640 1 --2- 172.21.15.139:0/663046854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9415b5e0 0x7fef940cc730 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00c7f0 tx=0x7fef8c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.449+0000 7fef9b438640 1 -- 172.21.15.139:0/663046854 shutdown_connections 2024-04-25T10:03:31.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.449+0000 7fef9b438640 1 --2- 172.21.15.139:0/663046854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9415b5e0 0x7fef940cc730 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.449+0000 7fef9b438640 1 --2- 172.21.15.139:0/663046854 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef7806b4d0 0x7fef7806d990 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.449+0000 7fef9b438640 1 --2- 172.21.15.139:0/663046854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef940b3770 0x7fef940ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.449+0000 7fef9b438640 1 --2- 172.21.15.139:0/663046854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef940b2db0 0x7fef94149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.449+0000 7fef9b438640 1 -- 172.21.15.139:0/663046854 >> 172.21.15.139:0/663046854 conn(0x7fef940b23d0 msgr2=0x7fef94160550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:31.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.449+0000 7fef9b438640 1 -- 172.21.15.139:0/663046854 shutdown_connections 2024-04-25T10:03:31.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:31.449+0000 7fef9b438640 1 -- 172.21.15.139:0/663046854 wait complete. 2024-04-25T10:03:31.465 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:03:31.466 INFO:tasks.cephfs_test_runner: 2024-04-25T10:03:31.466 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:03:31.466 INFO:tasks.cephfs_test_runner:Ran 7 tests in 575.087s 2024-04-25T10:03:31.466 INFO:tasks.cephfs_test_runner: 2024-04-25T10:03:31.466 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:03:31.466 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:03:31.476 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T10:03:31.504 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T10:03:31.504 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:31.505 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:31.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:03:31.521 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T10:03:31.521 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:31.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15881 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:03:31.576 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T10:03:31.577 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:31.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:03:31.595 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T10:03:31.595 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:31.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056108,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16621 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:03:31.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:31.654 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T10:03:31.668 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:03:31.669 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:03:31.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:31.669 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:31.732 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:31.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:31.797 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2024-04-25T10:03:31.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:31.902 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-04-25T10:03:31.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:31.917 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2024-04-25T10:03:31.984 INFO:teuthology.orchestra.run.smithi139.stderr:Cannot find device "ceph-brx" 2024-04-25T10:03:31.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:03:31.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:31.986 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-25T10:03:32.004 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:03:32.005 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:03:32.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:32.005 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:03:32.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:32.073 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:03:32.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:32.141 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-04-25T10:03:32.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:32.241 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-04-25T10:03:32.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:32.259 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-04-25T10:03:32.326 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2024-04-25T10:03:32.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:03:32.327 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:03:32.337 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:03:32.337 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:32.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.430+0000 7f8f029ab640 1 Processor -- start 2024-04-25T10:03:32.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.431+0000 7f8f029ab640 1 -- start start 2024-04-25T10:03:32.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.431+0000 7f8f029ab640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8efc15ac40 0x7f8efc06e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.431+0000 7f8f029ab640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8efc06e850 0x7f8efc0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.431+0000 7f8f029ab640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8efc0af6d0 con 0x7f8efc15a860 2024-04-25T10:03:32.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.431+0000 7f8f029ab640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8efc0b02e0 con 0x7f8efc06e850 2024-04-25T10:03:32.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.431+0000 7f8f029ab640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc0af9b0 con 0x7f8efc15ac40 2024-04-25T10:03:32.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.432+0000 7f8f00f48640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8efc06e850 0x7f8efc0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.432+0000 7f8f00f48640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8efc06e850 0x7f8efc0b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42790/0 (socket says 172.21.15.139:42790) 2024-04-25T10:03:32.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.432+0000 7f8f00f48640 1 -- 172.21.15.139:0/3436411299 learned_addr learned my addr 172.21.15.139:0/3436411299 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:32.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.432+0000 7f8efb7fe640 1 --2- 172.21.15.139:0/3436411299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8efc15ac40 0x7f8efc06e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.432+0000 7f8f00f48640 1 -- 172.21.15.139:0/3436411299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8efc15ac40 msgr2=0x7f8efc06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:32.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.432+0000 7f8f00f48640 1 --2- 172.21.15.139:0/3436411299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8efc15ac40 0x7f8efc06e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.432+0000 7f8f00f48640 1 -- 172.21.15.139:0/3436411299 >> v1:172.21.15.150:6789/0 conn(0x7f8efc15a860 legacy=0x7f8efc0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.432+0000 7f8f00f48640 1 -- 172.21.15.139:0/3436411299 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc0afc90 con 0x7f8efc06e850 2024-04-25T10:03:32.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.432+0000 7f8f00f48640 1 --2- 172.21.15.139:0/3436411299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8efc06e850 0x7f8efc0b0770 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009e50 tx=0x7f8ef402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eacb5665ec19ce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.433+0000 7f8efaffd640 1 -- 172.21.15.139:0/3436411299 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4036070 con 0x7f8efc06e850 2024-04-25T10:03:32.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.433+0000 7f8efaffd640 1 -- 172.21.15.139:0/3436411299 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef4031600 con 0x7f8efc06e850 2024-04-25T10:03:32.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.433+0000 7f8efaffd640 1 -- 172.21.15.139:0/3436411299 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef40319d0 con 0x7f8efc06e850 2024-04-25T10:03:32.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.433+0000 7f8f029ab640 1 -- 172.21.15.139:0/3436411299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8efc06e850 msgr2=0x7f8efc0b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.433+0000 7f8f029ab640 1 --2- 172.21.15.139:0/3436411299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8efc06e850 0x7f8efc0b0770 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009e50 tx=0x7f8ef402a710 comp rx=0 tx=0).stop 2024-04-25T10:03:32.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.433+0000 7f8f029ab640 1 -- 172.21.15.139:0/3436411299 shutdown_connections 2024-04-25T10:03:32.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.433+0000 7f8f029ab640 1 --2- 172.21.15.139:0/3436411299 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8efc06e850 0x7f8efc0b0770 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.433+0000 7f8f029ab640 1 --2- 172.21.15.139:0/3436411299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8efc15ac40 0x7f8efc06e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.433+0000 7f8f029ab640 1 -- 172.21.15.139:0/3436411299 >> 172.21.15.139:0/3436411299 conn(0x7f8efc066f20 msgr2=0x7f8efc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.433+0000 7f8f029ab640 1 -- 172.21.15.139:0/3436411299 shutdown_connections 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.433+0000 7f8f029ab640 1 -- 172.21.15.139:0/3436411299 wait complete. 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.434+0000 7f8f029ab640 1 Processor -- start 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.435+0000 7f8f029ab640 1 -- start start 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.435+0000 7f8f029ab640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8efc06e850 0x7f8efc14d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.435+0000 7f8f029ab640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8efc15a860 0x7f8efc0ac980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.435+0000 7f8f029ab640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8efc15ac40 0x7f8efc06d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.435+0000 7f8f029ab640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc0af2a0 con 0x7f8efc06e850 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.435+0000 7f8f029ab640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc0cd3c0 con 0x7f8efc15ac40 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.435+0000 7f8f029ab640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc0af420 con 0x7f8efc15a860 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.435+0000 7f8efbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8efc06e850 0x7f8efc14d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.435+0000 7f8efbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8efc06e850 0x7f8efc14d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42806/0 (socket says 172.21.15.139:42806) 2024-04-25T10:03:32.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.435+0000 7f8efbfff640 1 -- 172.21.15.139:0/948044957 learned_addr learned my addr 172.21.15.139:0/948044957 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.435+0000 7f8efb7fe640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8efc15a860 0x7f8efc0ac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.435+0000 7f8f00f48640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8efc15ac40 0x7f8efc06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.436+0000 7f8efbfff640 1 -- 172.21.15.139:0/948044957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8efc15a860 msgr2=0x7f8efc0ac980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.436+0000 7f8efbfff640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8efc15a860 0x7f8efc0ac980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.436+0000 7f8efbfff640 1 -- 172.21.15.139:0/948044957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8efc15ac40 msgr2=0x7f8efc06d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.436+0000 7f8efbfff640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8efc15ac40 0x7f8efc06d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.436+0000 7f8efbfff640 1 -- 172.21.15.139:0/948044957 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc0ac590 con 0x7f8efc06e850 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.436+0000 7f8efbfff640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8efc06e850 0x7f8efc14d2d0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00c430 tx=0x7f8eec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.436+0000 7f8ef8ff9640 1 -- 172.21.15.139:0/948044957 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0052d0 con 0x7f8efc06e850 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.436+0000 7f8ef8ff9640 1 -- 172.21.15.139:0/948044957 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec005490 con 0x7f8efc06e850 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.436+0000 7f8f029ab640 1 -- 172.21.15.139:0/948044957 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8efc0c4a10 con 0x7f8efc06e850 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.436+0000 7f8ef8ff9640 1 -- 172.21.15.139:0/948044957 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec007260 con 0x7f8efc06e850 2024-04-25T10:03:32.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.436+0000 7f8f029ab640 1 -- 172.21.15.139:0/948044957 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8efc0c4fd0 con 0x7f8efc06e850 2024-04-25T10:03:32.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.437+0000 7f8f029ab640 1 -- 172.21.15.139:0/948044957 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8efc074520 con 0x7f8efc06e850 2024-04-25T10:03:32.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.437+0000 7f8ef8ff9640 1 -- 172.21.15.139:0/948044957 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8eec007400 con 0x7f8efc06e850 2024-04-25T10:03:32.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.438+0000 7f8ef8ff9640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ee806b4d0 0x7f8ee806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.438+0000 7f8ef8ff9640 1 -- 172.21.15.139:0/948044957 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f8eec089580 con 0x7f8efc06e850 2024-04-25T10:03:32.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.438+0000 7f8efb7fe640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ee806b4d0 0x7f8ee806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.438+0000 7f8efb7fe640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ee806b4d0 0x7f8ee806d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0ac910 tx=0x7f8ef0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.442+0000 7f8ef8ff9640 1 -- 172.21.15.139:0/948044957 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8eec057e70 con 0x7f8efc06e850 2024-04-25T10:03:32.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.610+0000 7f8f029ab640 1 -- 172.21.15.139:0/948044957 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8efc0b4aa0 con 0x7f8ee806b4d0 2024-04-25T10:03:32.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.611+0000 7f8ef8ff9640 1 -- 172.21.15.139:0/948044957 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13410 (secure 0 0 0) 0x7f8efc0b4aa0 con 0x7f8ee806b4d0 2024-04-25T10:03:32.616 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:32.616 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T10:03:32.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.613+0000 7f8f029ab640 1 -- 172.21.15.139:0/948044957 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ee806b4d0 msgr2=0x7f8ee806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.613+0000 7f8f029ab640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ee806b4d0 0x7f8ee806d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0ac910 tx=0x7f8ef0009190 comp rx=0 tx=0).stop 2024-04-25T10:03:32.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.613+0000 7f8f029ab640 1 -- 172.21.15.139:0/948044957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8efc06e850 msgr2=0x7f8efc14d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.613+0000 7f8f029ab640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8efc06e850 0x7f8efc14d2d0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00c430 tx=0x7f8eec00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.613+0000 7f8f029ab640 1 -- 172.21.15.139:0/948044957 shutdown_connections 2024-04-25T10:03:32.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.613+0000 7f8f029ab640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f8ee806b4d0 0x7f8ee806d990 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.613+0000 7f8f029ab640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8efc15ac40 0x7f8efc06d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.613+0000 7f8f029ab640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8efc15a860 0x7f8efc0ac980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.613+0000 7f8f029ab640 1 --2- 172.21.15.139:0/948044957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8efc06e850 0x7f8efc14d2d0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.613+0000 7f8f029ab640 1 -- 172.21.15.139:0/948044957 >> 172.21.15.139:0/948044957 conn(0x7f8efc066f20 msgr2=0x7f8efc0b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:32.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.613+0000 7f8f029ab640 1 -- 172.21.15.139:0/948044957 shutdown_connections 2024-04-25T10:03:32.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.613+0000 7f8f029ab640 1 -- 172.21.15.139:0/948044957 wait complete. 2024-04-25T10:03:32.629 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-04-25T10:03:31.108989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278956,"kb_used_data":29528,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695764,"statfs":{"total":773094113280,"available":772808462336,"internally_reserved":0,"allocated":30236672,"data_stored":23663242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-171,"num_write_kb":-8301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001434"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":360,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.444342+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T10:03:13.444342+0000","last_peered":"2024-04-25T10:03:13.444342+0000","last_clean":"2024-04-25T10:03:13.444342+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T10:03:13.444342+0000","last_undegraded":"2024-04-25T10:03:13.444342+0000","last_fullsized":"2024-04-25T10:03:13.444342+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":6176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":6324224,"data_stored":5496465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40864,"kb_used_data":9688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330976,"statfs":{"total":96636764160,"available":96594919424,"internally_reserved":0,"allocated":9920512,"data_stored":9100219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1527808,"data_stored":711177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574977,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":718784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1540096,"data_stored":712173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574978,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1523712,"data_stored":711022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":5588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":5722112,"data_stored":4905350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574977,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33260,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2129920,"data_stored":1308052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:32.629 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:32.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.721+0000 7f40a4ecb640 1 Processor -- start 2024-04-25T10:03:32.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.721+0000 7f40a4ecb640 1 -- start start 2024-04-25T10:03:32.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.722+0000 7f40a4ecb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a009a160 0x7f40a00d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.722+0000 7f40a4ecb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a00cfe70 con 0x7f40a00cf900 2024-04-25T10:03:32.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.722+0000 7f40a4ecb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a00d0150 con 0x7f40a015efb0 2024-04-25T10:03:32.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.722+0000 7f40a4ecb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00d0a80 con 0x7f40a009a160 2024-04-25T10:03:32.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.722+0000 7f409dd9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f40a00cf900 0x7f40a0099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34584/0 (socket says 172.21.15.139:34584) 2024-04-25T10:03:32.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.722+0000 7f409dd9b640 1 -- 172.21.15.139:0/216746398 learned_addr learned my addr 172.21.15.139:0/216746398 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:32.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.722+0000 7f409ed9d640 1 --2- 172.21.15.139:0/216746398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a009a160 0x7f40a00d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.723+0000 7f409d59a640 1 -- 172.21.15.139:0/216746398 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646758270 0 0) 0x7f40a00cfe70 con 0x7f40a00cf900 2024-04-25T10:03:32.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.723+0000 7f409d59a640 1 -- 172.21.15.139:0/216746398 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4088003520 con 0x7f40a00cf900 2024-04-25T10:03:32.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.723+0000 7f409d59a640 1 -- 172.21.15.139:0/216746398 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1343895147 0 0) 0x7f40a00d0150 con 0x7f40a015efb0 2024-04-25T10:03:32.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.723+0000 7f409d59a640 1 -- 172.21.15.139:0/216746398 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a00cfe70 con 0x7f40a015efb0 2024-04-25T10:03:32.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.723+0000 7f409ed9d640 1 -- 172.21.15.139:0/216746398 >> v1:172.21.15.150:6789/0 conn(0x7f40a015efb0 legacy=0x7f40a006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.723+0000 7f409ed9d640 1 -- 172.21.15.139:0/216746398 >> v1:172.21.15.139:6789/0 conn(0x7f40a00cf900 legacy=0x7f40a0099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.723+0000 7f409ed9d640 1 -- 172.21.15.139:0/216746398 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a00d0430 con 0x7f40a009a160 2024-04-25T10:03:32.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.723+0000 7f409ed9d640 1 --2- 172.21.15.139:0/216746398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a009a160 0x7f40a00d0f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40980097d0 tx=0x7f409802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdb58104b780b0d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.724+0000 7f409d59a640 1 -- 172.21.15.139:0/216746398 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098037070 con 0x7f40a009a160 2024-04-25T10:03:32.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.724+0000 7f409d59a640 1 -- 172.21.15.139:0/216746398 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098032440 con 0x7f40a009a160 2024-04-25T10:03:32.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.724+0000 7f409d59a640 1 -- 172.21.15.139:0/216746398 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098032810 con 0x7f40a009a160 2024-04-25T10:03:32.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.724+0000 7f40a4ecb640 1 -- 172.21.15.139:0/216746398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a009a160 msgr2=0x7f40a00d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.724+0000 7f40a4ecb640 1 --2- 172.21.15.139:0/216746398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a009a160 0x7f40a00d0f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40980097d0 tx=0x7f409802a180 comp rx=0 tx=0).stop 2024-04-25T10:03:32.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.724+0000 7f40a4ecb640 1 -- 172.21.15.139:0/216746398 shutdown_connections 2024-04-25T10:03:32.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.724+0000 7f40a4ecb640 1 --2- 172.21.15.139:0/216746398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a009a160 0x7f40a00d0f10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.724+0000 7f40a4ecb640 1 -- 172.21.15.139:0/216746398 >> 172.21.15.139:0/216746398 conn(0x7f40a00d1790 msgr2=0x7f40a00d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:32.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.724+0000 7f40a4ecb640 1 -- 172.21.15.139:0/216746398 shutdown_connections 2024-04-25T10:03:32.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.724+0000 7f40a4ecb640 1 -- 172.21.15.139:0/216746398 wait complete. 2024-04-25T10:03:32.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.726+0000 7f40a4ecb640 1 Processor -- start 2024-04-25T10:03:32.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.726+0000 7f40a4ecb640 1 -- start start 2024-04-25T10:03:32.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.726+0000 7f40a4ecb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a009a160 0x7f40a015dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.726+0000 7f40a4ecb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40a00cf900 0x7f40a01f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.726+0000 7f40a4ecb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a015efb0 0x7f40a01f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.726+0000 7f40a4ecb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00cbde0 con 0x7f40a00cf900 2024-04-25T10:03:32.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.726+0000 7f40a4ecb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a006cf50 con 0x7f40a015efb0 2024-04-25T10:03:32.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.726+0000 7f40a4ecb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00cbf60 con 0x7f40a009a160 2024-04-25T10:03:32.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.726+0000 7f409dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40a00cf900 0x7f40a01f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f409dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40a00cf900 0x7f40a01f44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42808/0 (socket says 172.21.15.139:42808) 2024-04-25T10:03:32.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f409dd9b640 1 -- 172.21.15.139:0/1253581480 learned_addr learned my addr 172.21.15.139:0/1253581480 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:32.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f409ed9d640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a015efb0 0x7f40a01f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f409e59c640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a009a160 0x7f40a015dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f409dd9b640 1 -- 172.21.15.139:0/1253581480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a009a160 msgr2=0x7f40a015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f409dd9b640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a009a160 0x7f40a015dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f409dd9b640 1 -- 172.21.15.139:0/1253581480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a015efb0 msgr2=0x7f40a01f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f409dd9b640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a015efb0 0x7f40a01f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f409dd9b640 1 -- 172.21.15.139:0/1253581480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a01f49e0 con 0x7f40a00cf900 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f409ed9d640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a015efb0 0x7f40a01f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f409dd9b640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40a00cf900 0x7f40a01f44a0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f409400c500 tx=0x7f409400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f408effd640 1 -- 172.21.15.139:0/1253581480 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094017070 con 0x7f40a00cf900 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f408effd640 1 -- 172.21.15.139:0/1253581480 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4094005ce0 con 0x7f40a00cf900 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f40a4ecb640 1 -- 172.21.15.139:0/1253581480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a00c5830 con 0x7f40a00cf900 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f408effd640 1 -- 172.21.15.139:0/1253581480 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940072f0 con 0x7f40a00cf900 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.727+0000 7f40a4ecb640 1 -- 172.21.15.139:0/1253581480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a00c5df0 con 0x7f40a00cf900 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.728+0000 7f40a4ecb640 1 -- 172.21.15.139:0/1253581480 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a0074520 con 0x7f40a00cf900 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.729+0000 7f408effd640 1 -- 172.21.15.139:0/1253581480 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f409401a070 con 0x7f40a00cf900 2024-04-25T10:03:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.729+0000 7f408effd640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f408006b4d0 0x7f408006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.729+0000 7f409e59c640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f408006b4d0 0x7f408006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.729+0000 7f408effd640 1 -- 172.21.15.139:0/1253581480 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f4094089390 con 0x7f40a00cf900 2024-04-25T10:03:32.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.729+0000 7f409e59c640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f408006b4d0 0x7f408006d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4090001cb0 tx=0x7f4090009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.733+0000 7f408effd640 1 -- 172.21.15.139:0/1253581480 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f40a01f49e0 con 0x7f40a00cf900 2024-04-25T10:03:32.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.900+0000 7f40a4ecb640 1 -- 172.21.15.139:0/1253581480 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40a009ae50 con 0x7f408006b4d0 2024-04-25T10:03:32.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.901+0000 7f408effd640 1 -- 172.21.15.139:0/1253581480 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13410 (secure 0 0 0) 0x7f40a009ae50 con 0x7f408006b4d0 2024-04-25T10:03:32.906 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:32.906 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T10:03:32.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.903+0000 7f40a4ecb640 1 -- 172.21.15.139:0/1253581480 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f408006b4d0 msgr2=0x7f408006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.903+0000 7f40a4ecb640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f408006b4d0 0x7f408006d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4090001cb0 tx=0x7f4090009040 comp rx=0 tx=0).stop 2024-04-25T10:03:32.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.903+0000 7f40a4ecb640 1 -- 172.21.15.139:0/1253581480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40a00cf900 msgr2=0x7f40a01f44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.903+0000 7f40a4ecb640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40a00cf900 0x7f40a01f44a0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f409400c500 tx=0x7f409400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.903+0000 7f40a4ecb640 1 -- 172.21.15.139:0/1253581480 shutdown_connections 2024-04-25T10:03:32.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.903+0000 7f40a4ecb640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40a00cf900 0x7f40a01f44a0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.903+0000 7f40a4ecb640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f408006b4d0 0x7f408006d990 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.903+0000 7f40a4ecb640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a015efb0 0x7f40a01f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.903+0000 7f40a4ecb640 1 --2- 172.21.15.139:0/1253581480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a009a160 0x7f40a015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.903+0000 7f40a4ecb640 1 -- 172.21.15.139:0/1253581480 >> 172.21.15.139:0/1253581480 conn(0x7f40a00d1790 msgr2=0x7f40a00caab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:32.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.903+0000 7f40a4ecb640 1 -- 172.21.15.139:0/1253581480 shutdown_connections 2024-04-25T10:03:32.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:32.903+0000 7f40a4ecb640 1 -- 172.21.15.139:0/1253581480 wait complete. 2024-04-25T10:03:32.918 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-04-25T10:03:31.108989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278956,"kb_used_data":29528,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695764,"statfs":{"total":773094113280,"available":772808462336,"internally_reserved":0,"allocated":30236672,"data_stored":23663242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-171,"num_write_kb":-8301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001434"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":360,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T10:03:13.444342+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T10:03:13.444342+0000","last_peered":"2024-04-25T10:03:13.444342+0000","last_clean":"2024-04-25T10:03:13.444342+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T10:03:13.444342+0000","last_undegraded":"2024-04-25T10:03:13.444342+0000","last_fullsized":"2024-04-25T10:03:13.444342+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":6176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":6324224,"data_stored":5496465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40864,"kb_used_data":9688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330976,"statfs":{"total":96636764160,"available":96594919424,"internally_reserved":0,"allocated":9920512,"data_stored":9100219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1527808,"data_stored":711177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574977,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":718784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1540096,"data_stored":712173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574978,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1523712,"data_stored":711022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":5588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":5722112,"data_stored":4905350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574977,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33260,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2129920,"data_stored":1308052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:32.919 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:03:32.919 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:33.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.011+0000 7fa169212640 1 Processor -- start 2024-04-25T10:03:33.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.011+0000 7fa169212640 1 -- start start 2024-04-25T10:03:33.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa169212640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa16415ba10 0x7fa1640ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa169212640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa1640cf210 0x7fa16415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa169212640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1640cd680 con 0x7fa1640cd220 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa169212640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16406e260 con 0x7fa1640cf210 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa169212640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16406e540 con 0x7fa16415ba10 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa16259c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa1640cd220 0x7fa1640ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34588/0 (socket says 172.21.15.139:34588) 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa16259c640 1 -- 172.21.15.139:0/2718120134 learned_addr learned my addr 172.21.15.139:0/2718120134 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127952136 0 0) 0x7fa1640cd680 con 0x7fa1640cd220 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa14c003520 con 0x7fa1640cd220 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa162d9d640 1 --2- 172.21.15.139:0/2718120134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa16415ba10 0x7fa1640ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa16359e640 1 --2- 172.21.15.139:0/2718120134 >> v2:172.21.15.150:3300/0 conn(0x7fa1640cf210 0x7fa16415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 369721362 0 0) 0x7fa14c003520 con 0x7fa1640cd220 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa16415ba10 msgr2=0x7fa1640ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa161d9b640 1 --2- 172.21.15.139:0/2718120134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa16415ba10 0x7fa1640ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 >> v2:172.21.15.150:3300/0 conn(0x7fa1640cf210 msgr2=0x7fa16415bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa161d9b640 1 --2- 172.21.15.139:0/2718120134 >> v2:172.21.15.150:3300/0 conn(0x7fa1640cf210 0x7fa16415bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16406e820 con 0x7fa1640cd220 2024-04-25T10:03:33.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7fa158001670 con 0x7fa1640cd220 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 >> v1:172.21.15.139:6789/0 conn(0x7fa1640cd220 legacy=0x7fa1640ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.012+0000 7fa161d9b640 1 --2- 172.21.15.139:0/2718120134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c0039b0 0x7fa14c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa161d9b640 1 --2- 172.21.15.139:0/2718120134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14c004800 0x7fa14c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa161d9b640 1 --2- 172.21.15.139:0/2718120134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa16415ba10 0x7fa14c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16406e260 con 0x7fa14c004800 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16406e540 con 0x7fa14c0039b0 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c00ba70 con 0x7fa16415ba10 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa16259c640 1 --2- 172.21.15.139:0/2718120134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14c004800 0x7fa14c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa162d9d640 1 --2- 172.21.15.139:0/2718120134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c0039b0 0x7fa14c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa16359e640 1 --2- 172.21.15.139:0/2718120134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa16415ba10 0x7fa14c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa16259c640 1 -- 172.21.15.139:0/2718120134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa16415ba10 msgr2=0x7fa14c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa16259c640 1 --2- 172.21.15.139:0/2718120134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa16415ba10 0x7fa14c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa16259c640 1 -- 172.21.15.139:0/2718120134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c0039b0 msgr2=0x7fa14c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa16259c640 1 --2- 172.21.15.139:0/2718120134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c0039b0 0x7fa14c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.013+0000 7fa16259c640 1 -- 172.21.15.139:0/2718120134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c003520 con 0x7fa14c004800 2024-04-25T10:03:33.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.014+0000 7fa16259c640 1 --2- 172.21.15.139:0/2718120134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14c004800 0x7fa14c004c80 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fa158001050 tx=0x7fa14c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26700737f01468ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.014+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158007d50 con 0x7fa14c004800 2024-04-25T10:03:33.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.014+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa158005ad0 con 0x7fa14c004800 2024-04-25T10:03:33.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.014+0000 7fa161d9b640 1 -- 172.21.15.139:0/2718120134 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158007070 con 0x7fa14c004800 2024-04-25T10:03:33.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.014+0000 7fa169212640 1 -- 172.21.15.139:0/2718120134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14c004800 msgr2=0x7fa14c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.014+0000 7fa169212640 1 --2- 172.21.15.139:0/2718120134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14c004800 0x7fa14c004c80 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fa158001050 tx=0x7fa14c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:33.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.014+0000 7fa16359e640 1 -- 172.21.15.139:0/2718120134 reap_dead start 2024-04-25T10:03:33.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.014+0000 7fa169212640 1 -- 172.21.15.139:0/2718120134 shutdown_connections 2024-04-25T10:03:33.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.014+0000 7fa169212640 1 -- 172.21.15.139:0/2718120134 >> 172.21.15.139:0/2718120134 conn(0x7fa1640b17a0 msgr2=0x7fa1640b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.014+0000 7fa169212640 1 -- 172.21.15.139:0/2718120134 shutdown_connections 2024-04-25T10:03:33.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.014+0000 7fa169212640 1 -- 172.21.15.139:0/2718120134 wait complete. 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.016+0000 7fa169212640 1 Processor -- start 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.016+0000 7fa169212640 1 -- start start 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.016+0000 7fa169212640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa14c004800 0x7fa16414c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.016+0000 7fa169212640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1641f44f0 0x7fa1641530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.016+0000 7fa169212640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa164157230 0x7fa1641576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.016+0000 7fa169212640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16406f300 con 0x7fa1641f44f0 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.016+0000 7fa169212640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16406ede0 con 0x7fa164157230 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.016+0000 7fa169212640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa164153920 con 0x7fa14c004800 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa16259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1641f44f0 0x7fa1641530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa16259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1641f44f0 0x7fa1641530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42814/0 (socket says 172.21.15.139:42814) 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa16259c640 1 -- 172.21.15.139:0/176619709 learned_addr learned my addr 172.21.15.139:0/176619709 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa162d9d640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa14c004800 0x7fa16414c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa16359e640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa164157230 0x7fa1641576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa16259c640 1 -- 172.21.15.139:0/176619709 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa14c004800 msgr2=0x7fa16414c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa16259c640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa14c004800 0x7fa16414c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa16259c640 1 -- 172.21.15.139:0/176619709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa164157230 msgr2=0x7fa1641576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa16259c640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa164157230 0x7fa1641576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa16259c640 1 -- 172.21.15.139:0/176619709 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa164153c00 con 0x7fa1641f44f0 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa16259c640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1641f44f0 0x7fa1641530d0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c210 tx=0x7fa16406e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa14b7fe640 1 -- 172.21.15.139:0/176619709 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158043070 con 0x7fa1641f44f0 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa14b7fe640 1 -- 172.21.15.139:0/176619709 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa158038040 con 0x7fa1641f44f0 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa169212640 1 -- 172.21.15.139:0/176619709 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa164153ee0 con 0x7fa1641f44f0 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa14b7fe640 1 -- 172.21.15.139:0/176619709 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158007070 con 0x7fa1641f44f0 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.017+0000 7fa169212640 1 -- 172.21.15.139:0/176619709 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1641544a0 con 0x7fa1641f44f0 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.018+0000 7fa169212640 1 -- 172.21.15.139:0/176619709 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa164074520 con 0x7fa1641f44f0 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.019+0000 7fa14b7fe640 1 -- 172.21.15.139:0/176619709 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa158007230 con 0x7fa1641f44f0 2024-04-25T10:03:33.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.019+0000 7fa14b7fe640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa15006b500 0x7fa15006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.019+0000 7fa14b7fe640 1 -- 172.21.15.139:0/176619709 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fa1580a9ae0 con 0x7fa1641f44f0 2024-04-25T10:03:33.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.019+0000 7fa162d9d640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa15006b500 0x7fa15006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.020+0000 7fa162d9d640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa15006b500 0x7fa15006d9c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa154009a60 tx=0x7fa154009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.023+0000 7fa14b7fe640 1 -- 172.21.15.139:0/176619709 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa158005aa0 con 0x7fa1641f44f0 2024-04-25T10:03:33.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.193+0000 7fa169212640 1 -- 172.21.15.139:0/176619709 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1641563a0 con 0x7fa15006b500 2024-04-25T10:03:33.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.194+0000 7fa14b7fe640 1 -- 172.21.15.139:0/176619709 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13515 (secure 0 0 0) 0x7fa1641563a0 con 0x7fa15006b500 2024-04-25T10:03:33.198 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:33.199 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T10:03:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.195+0000 7fa169212640 1 -- 172.21.15.139:0/176619709 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa15006b500 msgr2=0x7fa15006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.195+0000 7fa169212640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa15006b500 0x7fa15006d9c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa154009a60 tx=0x7fa154009190 comp rx=0 tx=0).stop 2024-04-25T10:03:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.195+0000 7fa169212640 1 -- 172.21.15.139:0/176619709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1641f44f0 msgr2=0x7fa1641530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.195+0000 7fa169212640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1641f44f0 0x7fa1641530d0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c210 tx=0x7fa16406e9f0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.196+0000 7fa169212640 1 -- 172.21.15.139:0/176619709 shutdown_connections 2024-04-25T10:03:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.196+0000 7fa169212640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1641f44f0 0x7fa1641530d0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.196+0000 7fa169212640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa15006b500 0x7fa15006d9c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.196+0000 7fa169212640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa164157230 0x7fa1641576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.196+0000 7fa169212640 1 --2- 172.21.15.139:0/176619709 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa14c004800 0x7fa16414c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.196+0000 7fa169212640 1 -- 172.21.15.139:0/176619709 >> 172.21.15.139:0/176619709 conn(0x7fa1640b17a0 msgr2=0x7fa16406d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.196+0000 7fa169212640 1 -- 172.21.15.139:0/176619709 shutdown_connections 2024-04-25T10:03:33.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.196+0000 7fa169212640 1 -- 172.21.15.139:0/176619709 wait complete. 2024-04-25T10:03:33.211 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-04-25T10:03:33.109338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262860,"kb_used_data":13384,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711860,"statfs":{"total":773094113280,"available":772824944640,"internally_reserved":0,"allocated":13705216,"data_stored":7023600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-171,"num_write_kb":-8301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001439"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":366,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-25T10:03:30.486560+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T10:03:30.486560+0000","last_peered":"2024-04-25T10:03:30.486560+0000","last_clean":"2024-04-25T10:03:30.486560+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T10:03:30.486560+0000","last_undegraded":"2024-04-25T10:03:30.486560+0000","last_fullsized":"2024-04-25T10:03:30.486560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574978,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2158592,"data_stored":1320726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574978,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574977,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574978,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574979,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574978,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33292,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338548,"statfs":{"total":96636764160,"available":96602673152,"internally_reserved":0,"allocated":2158592,"data_stored":1320726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:33.212 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:33.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.311+0000 7fa092ecd640 1 Processor -- start 2024-04-25T10:03:33.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.311+0000 7fa092ecd640 1 -- start start 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.312+0000 7fa092ecd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa08c072990 0x7fa08c0b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.312+0000 7fa092ecd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08c15e500 con 0x7fa08c15aa00 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.312+0000 7fa092ecd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08c0b5660 con 0x7fa08c0b5110 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.312+0000 7fa092ecd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa08c0b5c20 con 0x7fa08c072990 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.312+0000 7fa09146a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa08c072990 0x7fa08c0b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.312+0000 7fa090c69640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa08c15aa00 0x7fa08c15ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34596/0 (socket says 172.21.15.139:34596) 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.312+0000 7fa090c69640 1 -- 172.21.15.139:0/3147381428 learned_addr learned my addr 172.21.15.139:0/3147381428 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.313+0000 7fa09146a640 1 -- 172.21.15.139:0/3147381428 >> v1:172.21.15.150:6789/0 conn(0x7fa08c0b5110 legacy=0x7fa08c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.313+0000 7fa09146a640 1 -- 172.21.15.139:0/3147381428 >> v1:172.21.15.139:6789/0 conn(0x7fa08c15aa00 legacy=0x7fa08c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.313+0000 7fa08b7fe640 1 -- 172.21.15.139:0/3147381428 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355050817 0 0) 0x7fa08c15e500 con 0x7fa08c15aa00 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.313+0000 7fa09146a640 1 -- 172.21.15.139:0/3147381428 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c0b5940 con 0x7fa08c072990 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.313+0000 7fa09146a640 1 --2- 172.21.15.139:0/3147381428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa08c072990 0x7fa08c0b4090 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fa084009dc0 tx=0x7fa08402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1c01b9c0fdcbec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.313+0000 7fa08b7fe640 1 -- 172.21.15.139:0/3147381428 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084032770 con 0x7fa08c072990 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.314+0000 7fa08b7fe640 1 -- 172.21.15.139:0/3147381428 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa084038070 con 0x7fa08c072990 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.314+0000 7fa08b7fe640 1 -- 172.21.15.139:0/3147381428 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084032b30 con 0x7fa08c072990 2024-04-25T10:03:33.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.314+0000 7fa092ecd640 1 -- 172.21.15.139:0/3147381428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa08c072990 msgr2=0x7fa08c0b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.314+0000 7fa092ecd640 1 --2- 172.21.15.139:0/3147381428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa08c072990 0x7fa08c0b4090 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fa084009dc0 tx=0x7fa08402c8b0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.314+0000 7fa092ecd640 1 -- 172.21.15.139:0/3147381428 shutdown_connections 2024-04-25T10:03:33.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.314+0000 7fa092ecd640 1 --2- 172.21.15.139:0/3147381428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa08c072990 0x7fa08c0b4090 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.314+0000 7fa092ecd640 1 -- 172.21.15.139:0/3147381428 >> 172.21.15.139:0/3147381428 conn(0x7fa08c06e280 msgr2=0x7fa08c06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.314+0000 7fa092ecd640 1 -- 172.21.15.139:0/3147381428 shutdown_connections 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.314+0000 7fa092ecd640 1 -- 172.21.15.139:0/3147381428 wait complete. 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.315+0000 7fa092ecd640 1 Processor -- start 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.315+0000 7fa092ecd640 1 -- start start 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.316+0000 7fa092ecd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa08c072990 0x7fa08c0cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.316+0000 7fa092ecd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08c0b5110 0x7fa08c145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.316+0000 7fa092ecd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa08c1432f0 0x7fa08c0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.316+0000 7fa092ecd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c0ce610 con 0x7fa08c072990 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.316+0000 7fa092ecd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c0b45d0 con 0x7fa08c0b5110 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.316+0000 7fa092ecd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c0ce790 con 0x7fa08c1432f0 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.316+0000 7fa090c69640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa08c072990 0x7fa08c0cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.316+0000 7fa090c69640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa08c072990 0x7fa08c0cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42832/0 (socket says 172.21.15.139:42832) 2024-04-25T10:03:33.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.316+0000 7fa090c69640 1 -- 172.21.15.139:0/3046314122 learned_addr learned my addr 172.21.15.139:0/3046314122 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:33.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa09146a640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa08c1432f0 0x7fa08c0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa08bfff640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08c0b5110 0x7fa08c145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa090c69640 1 -- 172.21.15.139:0/3046314122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa08c1432f0 msgr2=0x7fa08c0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa090c69640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa08c1432f0 0x7fa08c0cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa090c69640 1 -- 172.21.15.139:0/3046314122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08c0b5110 msgr2=0x7fa08c145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa090c69640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08c0b5110 0x7fa08c145d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa090c69640 1 -- 172.21.15.139:0/3046314122 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c1462d0 con 0x7fa08c072990 2024-04-25T10:03:33.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa090c69640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa08c072990 0x7fa08c0cc0f0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fa08000c420 tx=0x7fa08000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa0897fa640 1 -- 172.21.15.139:0/3046314122 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0800074e0 con 0x7fa08c072990 2024-04-25T10:03:33.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa0897fa640 1 -- 172.21.15.139:0/3046314122 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0800047e0 con 0x7fa08c072990 2024-04-25T10:03:33.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa0897fa640 1 -- 172.21.15.139:0/3046314122 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080004c10 con 0x7fa08c072990 2024-04-25T10:03:33.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa092ecd640 1 -- 172.21.15.139:0/3046314122 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08c15c870 con 0x7fa08c072990 2024-04-25T10:03:33.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.317+0000 7fa092ecd640 1 -- 172.21.15.139:0/3046314122 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08c15b170 con 0x7fa08c072990 2024-04-25T10:03:33.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.318+0000 7fa092ecd640 1 -- 172.21.15.139:0/3046314122 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa08c074520 con 0x7fa08c072990 2024-04-25T10:03:33.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.319+0000 7fa0897fa640 1 -- 172.21.15.139:0/3046314122 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa080011070 con 0x7fa08c072990 2024-04-25T10:03:33.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.319+0000 7fa0897fa640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa07006b4d0 0x7fa07006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.319+0000 7fa0897fa640 1 -- 172.21.15.139:0/3046314122 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fa080089b00 con 0x7fa08c072990 2024-04-25T10:03:33.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.319+0000 7fa08bfff640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa07006b4d0 0x7fa07006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.320+0000 7fa08bfff640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa07006b4d0 0x7fa07006d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0cc630 tx=0x7fa078009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.323+0000 7fa0897fa640 1 -- 172.21.15.139:0/3046314122 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa08c1462d0 con 0x7fa08c072990 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.516+0000 7fa092ecd640 1 -- 172.21.15.139:0/3046314122 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa08c073b20 con 0x7fa08c072990 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.517+0000 7fa0897fa640 1 -- 172.21.15.139:0/3046314122 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+13617 (secure 0 0 0) 0x7fa080058480 con 0x7fa08c072990 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":125,"fsid":"922739a8-edcf-4ded-8206-dbdb23e9fea4","created":"2024-04-25T09:52:50.914537+0000","modified":"2024-04-25T10:03:30.433710+0000","last_up_change":"2024-04-25T09:53:02.029092+0000","last_in_change":"2024-04-25T09:52:53.019402+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:53:02.998776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4af65dcf-4971-4314-8195-7e9e848ba2e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6809","nonce":1167038101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6811","nonce":1167038101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6815","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6817","nonce":1167038101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1167038101},{"type":"v1","addr":"172.21.15.139:6813","nonce":1167038101}]},"public_addr":"172.21.15.139:6809/1167038101","cluster_addr":"172.21.15.139:6811/1167038101","heartbeat_back_addr":"172.21.15.139:6817/1167038101","heartbeat_front_addr":"172.21.15.139:6813/1167038101","state":["exists","up"]},{"osd":1,"uuid":"42562e83-3222-41c7-a37d-cbe67b25096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6816","nonce":315640110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6819","nonce":315640110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6823","nonce":315640110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":315640110},{"type":"v1","addr":"172.21.15.139:6821","nonce":315640110}]},"public_addr":"172.21.15.139:6816/315640110","cluster_addr":"172.21.15.139:6819/315640110","heartbeat_back_addr":"172.21.15.139:6823/315640110","heartbeat_front_addr":"172.21.15.139:6821/315640110","state":["exists","up"]},{"osd":2,"uuid":"69e1d487-9327-46e1-adf8-9e5b5a16a81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6801","nonce":1982336775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6803","nonce":1982336775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6807","nonce":1982336775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1982336775},{"type":"v1","addr":"172.21.15.139:6805","nonce":1982336775}]},"public_addr":"172.21.15.139:6801/1982336775","cluster_addr":"172.21.15.139:6803/1982336775","heartbeat_back_addr":"172.21.15.139:6807/1982336775","heartbeat_front_addr":"172.21.15.139:6805/1982336775","state":["exists","up"]},{"osd":3,"uuid":"b3e8830f-b9fe-4afc-8aeb-b4b1417ab368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6825","nonce":1040931291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6827","nonce":1040931291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6831","nonce":1040931291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1040931291},{"type":"v1","addr":"172.21.15.139:6829","nonce":1040931291}]},"public_addr":"172.21.15.139:6825/1040931291","cluster_addr":"172.21.15.139:6827/1040931291","heartbeat_back_addr":"172.21.15.139:6831/1040931291","heartbeat_front_addr":"172.21.15.139:6829/1040931291","state":["exists","up"]},{"osd":4,"uuid":"7bc3028f-8684-4634-91f8-fab6676f7c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6801","nonce":2810531857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6803","nonce":2810531857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6807","nonce":2810531857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":2810531857},{"type":"v1","addr":"172.21.15.150:6805","nonce":2810531857}]},"public_addr":"172.21.15.150:6801/2810531857","cluster_addr":"172.21.15.150:6803/2810531857","heartbeat_back_addr":"172.21.15.150:6807/2810531857","heartbeat_front_addr":"172.21.15.150:6805/2810531857","state":["exists","up"]},{"osd":5,"uuid":"19cf45c1-b7da-4aa8-9033-76a51df3ee24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6825","nonce":2184499879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6827","nonce":2184499879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6831","nonce":2184499879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2184499879},{"type":"v1","addr":"172.21.15.150:6829","nonce":2184499879}]},"public_addr":"172.21.15.150:6825/2184499879","cluster_addr":"172.21.15.150:6827/2184499879","heartbeat_back_addr":"172.21.15.150:6831/2184499879","heartbeat_front_addr":"172.21.15.150:6829/2184499879","state":["exists","up"]},{"osd":6,"uuid":"9aaf5762-215d-4714-ace8-980aeecb6330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6811","nonce":3369628189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6815","nonce":3369628189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6821","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6823","nonce":3369628189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":3369628189},{"type":"v1","addr":"172.21.15.150:6819","nonce":3369628189}]},"public_addr":"172.21.15.150:6811/3369628189","cluster_addr":"172.21.15.150:6815/3369628189","heartbeat_back_addr":"172.21.15.150:6823/3369628189","heartbeat_front_addr":"172.21.15.150:6819/3369628189","state":["exists","up"]},{"osd":7,"uuid":"62755a3e-bdec-4e96-8e7a-a608b17acac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6810","nonce":2122423536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6814","nonce":2122423536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6822","nonce":2122423536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2122423536},{"type":"v1","addr":"172.21.15.150:6818","nonce":2122423536}]},"public_addr":"172.21.15.150:6810/2122423536","cluster_addr":"172.21.15.150:6814/2122423536","heartbeat_back_addr":"172.21.15.150:6822/2122423536","heartbeat_front_addr":"172.21.15.150:6818/2122423536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.180517+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.165676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.183799+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.209106+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.224343+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.199035+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.206796+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:52:58.167656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6836/1898344850":"2024-04-26T10:03:27.114986+0000","172.21.15.139:6834/1898344850":"2024-04-26T10:03:27.114986+0000","172.21.15.139:6836/3478297010":"2024-04-26T10:03:13.364392+0000","172.21.15.139:6834/3478297010":"2024-04-26T10:03:13.364392+0000","172.21.15.150:6835/1377607596":"2024-04-26T10:03:03.005923+0000","172.21.15.150:6833/1377607596":"2024-04-26T10:03:03.005923+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.519+0000 7fa092ecd640 1 -- 172.21.15.139:0/3046314122 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa07006b4d0 msgr2=0x7fa07006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.519+0000 7fa092ecd640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa07006b4d0 0x7fa07006d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0cc630 tx=0x7fa078009040 comp rx=0 tx=0).stop 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.519+0000 7fa092ecd640 1 -- 172.21.15.139:0/3046314122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa08c072990 msgr2=0x7fa08c0cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.519+0000 7fa092ecd640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa08c072990 0x7fa08c0cc0f0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fa08000c420 tx=0x7fa08000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.519+0000 7fa092ecd640 1 -- 172.21.15.139:0/3046314122 shutdown_connections 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.519+0000 7fa092ecd640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fa07006b4d0 0x7fa07006d990 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.519+0000 7fa092ecd640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa08c1432f0 0x7fa08c0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.519+0000 7fa092ecd640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08c0b5110 0x7fa08c145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.519+0000 7fa092ecd640 1 --2- 172.21.15.139:0/3046314122 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa08c072990 0x7fa08c0cc0f0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.519+0000 7fa092ecd640 1 -- 172.21.15.139:0/3046314122 >> 172.21.15.139:0/3046314122 conn(0x7fa08c06e280 msgr2=0x7fa08c0b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.519+0000 7fa092ecd640 1 -- 172.21.15.139:0/3046314122 shutdown_connections 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:33.519+0000 7fa092ecd640 1 -- 172.21.15.139:0/3046314122 wait complete. 2024-04-25T10:03:34.536 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:03:34.536 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:03:34.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.633+0000 7ff771d6a640 1 Processor -- start 2024-04-25T10:03:34.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.633+0000 7ff771d6a640 1 -- start start 2024-04-25T10:03:34.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.634+0000 7ff771d6a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff76c144090 0x7ff76c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.634+0000 7ff771d6a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff76c0cc9e0 0x7ff76c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.634+0000 7ff771d6a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76c0b2a50 con 0x7ff76c0cce20 2024-04-25T10:03:34.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.634+0000 7ff771d6a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76c15b090 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.634+0000 7ff771d6a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c15b370 con 0x7ff76c144090 2024-04-25T10:03:34.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.634+0000 7ff76affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff76c0cc9e0 0x7ff76c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.634+0000 7ff76affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff76c0cc9e0 0x7ff76c06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42842/0 (socket says 172.21.15.139:42842) 2024-04-25T10:03:34.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.634+0000 7ff76affd640 1 -- 172.21.15.139:0/4026827537 learned_addr learned my addr 172.21.15.139:0/4026827537 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:34.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.635+0000 7ff76b7fe640 1 --2- 172.21.15.139:0/4026827537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff76c144090 0x7ff76c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.635+0000 7ff76affd640 1 -- 172.21.15.139:0/4026827537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff76c144090 msgr2=0x7ff76c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.635+0000 7ff76affd640 1 --2- 172.21.15.139:0/4026827537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff76c144090 0x7ff76c0b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.635+0000 7ff76affd640 1 -- 172.21.15.139:0/4026827537 >> v1:172.21.15.150:6789/0 conn(0x7ff76c0cce20 legacy=0x7ff76c15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.635+0000 7ff76affd640 1 -- 172.21.15.139:0/4026827537 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c15b650 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.635+0000 7ff76affd640 1 --2- 172.21.15.139:0/4026827537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff76c0cc9e0 0x7ff76c06e330 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7ff760009870 tx=0x7ff76002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8b380e1c28dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.636+0000 7ff76a7fc640 1 -- 172.21.15.139:0/4026827537 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff760037070 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.636+0000 7ff76a7fc640 1 -- 172.21.15.139:0/4026827537 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff760032440 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.636+0000 7ff76a7fc640 1 -- 172.21.15.139:0/4026827537 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff760032810 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.636+0000 7ff771d6a640 1 -- 172.21.15.139:0/4026827537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff76c0cc9e0 msgr2=0x7ff76c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.636+0000 7ff771d6a640 1 --2- 172.21.15.139:0/4026827537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff76c0cc9e0 0x7ff76c06e330 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7ff760009870 tx=0x7ff76002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.636+0000 7ff771d6a640 1 -- 172.21.15.139:0/4026827537 shutdown_connections 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.636+0000 7ff771d6a640 1 --2- 172.21.15.139:0/4026827537 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff76c0cc9e0 0x7ff76c06e330 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.636+0000 7ff771d6a640 1 --2- 172.21.15.139:0/4026827537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff76c144090 0x7ff76c0b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.636+0000 7ff771d6a640 1 -- 172.21.15.139:0/4026827537 >> 172.21.15.139:0/4026827537 conn(0x7ff76c067000 msgr2=0x7ff76c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.636+0000 7ff771d6a640 1 -- 172.21.15.139:0/4026827537 shutdown_connections 2024-04-25T10:03:34.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.636+0000 7ff771d6a640 1 -- 172.21.15.139:0/4026827537 wait complete. 2024-04-25T10:03:34.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.637+0000 7ff771d6a640 1 Processor -- start 2024-04-25T10:03:34.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.638+0000 7ff771d6a640 1 -- start start 2024-04-25T10:03:34.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.638+0000 7ff771d6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff76c0cc9e0 0x7ff76c1303a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.638+0000 7ff771d6a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff76c0cce20 0x7ff76c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.638+0000 7ff771d6a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff76c144090 0x7ff76c12e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.638+0000 7ff771d6a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c144470 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.638+0000 7ff771d6a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c15bc10 con 0x7ff76c0cce20 2024-04-25T10:03:34.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.638+0000 7ff771d6a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c15ad80 con 0x7ff76c144090 2024-04-25T10:03:34.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.638+0000 7ff76b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff76c0cc9e0 0x7ff76c1303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.638+0000 7ff76b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff76c0cc9e0 0x7ff76c1303a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42846/0 (socket says 172.21.15.139:42846) 2024-04-25T10:03:34.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.638+0000 7ff76b7fe640 1 -- 172.21.15.139:0/2210904828 learned_addr learned my addr 172.21.15.139:0/2210904828 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:34.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.638+0000 7ff76affd640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff76c0cce20 0x7ff76c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.638+0000 7ff76bfff640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff76c144090 0x7ff76c12e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.639+0000 7ff76b7fe640 1 -- 172.21.15.139:0/2210904828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff76c144090 msgr2=0x7ff76c12e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.639+0000 7ff76b7fe640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff76c144090 0x7ff76c12e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.639+0000 7ff76b7fe640 1 -- 172.21.15.139:0/2210904828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff76c0cce20 msgr2=0x7ff76c159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.639+0000 7ff76b7fe640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff76c0cce20 0x7ff76c159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.639+0000 7ff76b7fe640 1 -- 172.21.15.139:0/2210904828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c15a2d0 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.639+0000 7ff76b7fe640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff76c0cc9e0 0x7ff76c1303a0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00d620 tx=0x7ff75c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.639+0000 7ff753fff640 1 -- 172.21.15.139:0/2210904828 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c00b620 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.639+0000 7ff753fff640 1 -- 172.21.15.139:0/2210904828 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c019070 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.639+0000 7ff753fff640 1 -- 172.21.15.139:0/2210904828 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c00ba00 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.639+0000 7ff771d6a640 1 -- 172.21.15.139:0/2210904828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76c12f300 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.639+0000 7ff771d6a640 1 -- 172.21.15.139:0/2210904828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c12f8c0 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.640+0000 7ff771d6a640 1 -- 172.21.15.139:0/2210904828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff76c074520 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.640+0000 7ff753fff640 1 -- 172.21.15.139:0/2210904828 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff75c00bba0 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.641+0000 7ff753fff640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff75806b4d0 0x7ff75806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.641+0000 7ff753fff640 1 -- 172.21.15.139:0/2210904828 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ff75c08a370 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.641+0000 7ff76affd640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff75806b4d0 0x7ff75806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.641+0000 7ff753fff640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x7ff758071ab0 0x7ff758073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.641+0000 7ff753fff640 1 -- 172.21.15.139:0/2210904828 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff75c008e50 con 0x7ff758071ab0 2024-04-25T10:03:34.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.641+0000 7ff753fff640 1 -- 172.21.15.139:0/2210904828 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff75c091050 con 0x7ff76c0cc9e0 2024-04-25T10:03:34.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.641+0000 7ff76bfff640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x7ff758071ab0 0x7ff758073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.641+0000 7ff76affd640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff75806b4d0 0x7ff75806d990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff760034000 tx=0x7ff76002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.641+0000 7ff76bfff640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x7ff758071ab0 0x7ff758073ef0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.642+0000 7ff753fff640 1 -- 172.21.15.139:0/2210904828 <== osd.0 v2:172.21.15.139:6808/1167038101 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff75c008e50 con 0x7ff758071ab0 2024-04-25T10:03:34.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.655+0000 7ff771d6a640 1 -- 172.21.15.139:0/2210904828 --> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff76c12fe50 con 0x7ff758071ab0 2024-04-25T10:03:34.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.655+0000 7ff753fff640 1 -- 172.21.15.139:0/2210904828 <== osd.0 v2:172.21.15.139:6808/1167038101 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff76c12fe50 con 0x7ff758071ab0 2024-04-25T10:03:34.660 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:03:34.660 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:03:34.660 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:03:34.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.655+0000 7ff771d6a640 1 -- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x7ff758071ab0 msgr2=0x7ff758073ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.655+0000 7ff771d6a640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:6808/1167038101,v1:172.21.15.139:6809/1167038101] conn(0x7ff758071ab0 0x7ff758073ef0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.656+0000 7ff771d6a640 1 -- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff75806b4d0 msgr2=0x7ff75806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.656+0000 7ff771d6a640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff75806b4d0 0x7ff75806d990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff760034000 tx=0x7ff76002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:34.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.656+0000 7ff771d6a640 1 -- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff76c0cc9e0 msgr2=0x7ff76c1303a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.656+0000 7ff771d6a640 1 --2- 172.21.15.139:0/2210904828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff76c0cc9e0 0x7ff76c1303a0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00d620 tx=0x7ff75c00dae0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.656+0000 7ff76bfff640 1 -- 172.21.15.139:0/2210904828 reap_dead start 2024-04-25T10:03:34.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.656+0000 7ff771d6a640 1 -- 172.21.15.139:0/2210904828 shutdown_connections 2024-04-25T10:03:34.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.656+0000 7ff771d6a640 1 -- 172.21.15.139:0/2210904828 >> 172.21.15.139:0/2210904828 conn(0x7ff76c067000 msgr2=0x7ff76c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.656+0000 7ff771d6a640 1 -- 172.21.15.139:0/2210904828 shutdown_connections 2024-04-25T10:03:34.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.656+0000 7ff771d6a640 1 -- 172.21.15.139:0/2210904828 wait complete. 2024-04-25T10:03:34.671 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:03:34.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.764+0000 7f06644fc640 1 Processor -- start 2024-04-25T10:03:34.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.764+0000 7f06644fc640 1 -- start start 2024-04-25T10:03:34.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f06644fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f065c0cd610 0x7f065c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f06644fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f065c09ad40 0x7f065c0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f06644fc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c099ca0 con 0x7f065c1618c0 2024-04-25T10:03:34.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f06644fc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065c099a50 con 0x7f065c09ad40 2024-04-25T10:03:34.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f06644fc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c099f80 con 0x7f065c0cd610 2024-04-25T10:03:34.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f0661a97640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f065c09ad40 0x7f065c0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f0662a99640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f065c1618c0 0x7f065c06be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34598/0 (socket says 172.21.15.139:34598) 2024-04-25T10:03:34.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f0661a97640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f065c09ad40 0x7f065c0d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42850/0 (socket says 172.21.15.139:42850) 2024-04-25T10:03:34.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f0662a99640 1 -- 172.21.15.139:0/1840233249 learned_addr learned my addr 172.21.15.139:0/1840233249 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:34.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f0662298640 1 --2- 172.21.15.139:0/1840233249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f065c0cd610 0x7f065c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f0661a97640 1 -- 172.21.15.139:0/1840233249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f065c0cd610 msgr2=0x7f065c0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f0661a97640 1 --2- 172.21.15.139:0/1840233249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f065c0cd610 0x7f065c0d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f0661a97640 1 -- 172.21.15.139:0/1840233249 >> v1:172.21.15.139:6789/0 conn(0x7f065c1618c0 legacy=0x7f065c06be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f0661296640 1 -- 172.21.15.139:0/1840233249 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037742948 0 0) 0x7f065c099ca0 con 0x7f065c1618c0 2024-04-25T10:03:34.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.765+0000 7f0661a97640 1 -- 172.21.15.139:0/1840233249 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c09a260 con 0x7f065c09ad40 2024-04-25T10:03:34.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.766+0000 7f0661a97640 1 --2- 172.21.15.139:0/1840233249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f065c09ad40 0x7f065c0d08b0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f064c0097a0 tx=0x7f064c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a643a31dac07f6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.766+0000 7f0661296640 1 -- 172.21.15.139:0/1840233249 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c037070 con 0x7f065c09ad40 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.766+0000 7f0661296640 1 -- 172.21.15.139:0/1840233249 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c032440 con 0x7f065c09ad40 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.766+0000 7f0661296640 1 -- 172.21.15.139:0/1840233249 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c032810 con 0x7f065c09ad40 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.766+0000 7f06644fc640 1 -- 172.21.15.139:0/1840233249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f065c09ad40 msgr2=0x7f065c0d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.766+0000 7f06644fc640 1 --2- 172.21.15.139:0/1840233249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f065c09ad40 0x7f065c0d08b0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f064c0097a0 tx=0x7f064c02a210 comp rx=0 tx=0).stop 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.767+0000 7f06644fc640 1 -- 172.21.15.139:0/1840233249 shutdown_connections 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.767+0000 7f06644fc640 1 --2- 172.21.15.139:0/1840233249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f065c09ad40 0x7f065c0d08b0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.767+0000 7f06644fc640 1 --2- 172.21.15.139:0/1840233249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f065c0cd610 0x7f065c0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.767+0000 7f06644fc640 1 -- 172.21.15.139:0/1840233249 >> 172.21.15.139:0/1840233249 conn(0x7f065c0b31f0 msgr2=0x7f065c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.767+0000 7f06644fc640 1 -- 172.21.15.139:0/1840233249 shutdown_connections 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.767+0000 7f06644fc640 1 -- 172.21.15.139:0/1840233249 wait complete. 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.768+0000 7f06644fc640 1 Processor -- start 2024-04-25T10:03:34.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.768+0000 7f06644fc640 1 -- start start 2024-04-25T10:03:34.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.768+0000 7f06644fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f065c09ad40 0x7f065c1f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.768+0000 7f06644fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f065c0cd610 0x7f065c159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.768+0000 7f06644fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f065c1618c0 0x7f065c0c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.768+0000 7f06644fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0cd9f0 con 0x7f065c09ad40 2024-04-25T10:03:34.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.768+0000 7f06644fc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c09a820 con 0x7f065c1618c0 2024-04-25T10:03:34.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.768+0000 7f06644fc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0cdb70 con 0x7f065c0cd610 2024-04-25T10:03:34.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0662298640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f065c09ad40 0x7f065c1f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0662298640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f065c09ad40 0x7f065c1f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42864/0 (socket says 172.21.15.139:42864) 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0662298640 1 -- 172.21.15.139:0/3153804335 learned_addr learned my addr 172.21.15.139:0/3153804335 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0661a97640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f065c0cd610 0x7f065c159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0662a99640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f065c1618c0 0x7f065c0c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0662298640 1 -- 172.21.15.139:0/3153804335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f065c0cd610 msgr2=0x7f065c159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0662298640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f065c0cd610 0x7f065c159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0662298640 1 -- 172.21.15.139:0/3153804335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f065c1618c0 msgr2=0x7f065c0c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0662298640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f065c1618c0 0x7f065c0c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0662298640 1 -- 172.21.15.139:0/3153804335 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c0c52d0 con 0x7f065c09ad40 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0662298640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f065c09ad40 0x7f065c1f4370 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f065800e9b0 tx=0x7f065800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0652ffd640 1 -- 172.21.15.139:0/3153804335 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658017070 con 0x7f065c09ad40 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0652ffd640 1 -- 172.21.15.139:0/3153804335 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065800c9a0 con 0x7f065c09ad40 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f06644fc640 1 -- 172.21.15.139:0/3153804335 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c0c55b0 con 0x7f065c09ad40 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f0652ffd640 1 -- 172.21.15.139:0/3153804335 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065800cd90 con 0x7f065c09ad40 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.769+0000 7f06644fc640 1 -- 172.21.15.139:0/3153804335 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065c0c5b70 con 0x7f065c09ad40 2024-04-25T10:03:34.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.770+0000 7f06644fc640 1 -- 172.21.15.139:0/3153804335 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065c074520 con 0x7f065c09ad40 2024-04-25T10:03:34.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.771+0000 7f0652ffd640 1 -- 172.21.15.139:0/3153804335 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f065801a070 con 0x7f065c09ad40 2024-04-25T10:03:34.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.771+0000 7f0652ffd640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f064006b4d0 0x7f064006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.771+0000 7f0652ffd640 1 -- 172.21.15.139:0/3153804335 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f065808a440 con 0x7f065c09ad40 2024-04-25T10:03:34.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.771+0000 7f0661a97640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f064006b4d0 0x7f064006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.772+0000 7f0661a97640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f064006b4d0 0x7f064006d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f064c034000 tx=0x7f064c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.775+0000 7f0652ffd640 1 -- 172.21.15.139:0/3153804335 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0658059410 con 0x7f065c09ad40 2024-04-25T10:03:34.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.945+0000 7f06644fc640 1 -- 172.21.15.139:0/3153804335 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f065c0b3f70 con 0x7f064006b4d0 2024-04-25T10:03:34.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.945+0000 7f0652ffd640 1 -- 172.21.15.139:0/3153804335 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f065c0b3f70 con 0x7f064006b4d0 2024-04-25T10:03:34.950 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:03:34.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.947+0000 7f06644fc640 1 -- 172.21.15.139:0/3153804335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f064006b4d0 msgr2=0x7f064006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.947+0000 7f06644fc640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f064006b4d0 0x7f064006d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f064c034000 tx=0x7f064c02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:34.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.947+0000 7f06644fc640 1 -- 172.21.15.139:0/3153804335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f065c09ad40 msgr2=0x7f065c1f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.947+0000 7f06644fc640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f065c09ad40 0x7f065c1f4370 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f065800e9b0 tx=0x7f065800ee70 comp rx=0 tx=0).stop 2024-04-25T10:03:34.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.947+0000 7f06644fc640 1 -- 172.21.15.139:0/3153804335 shutdown_connections 2024-04-25T10:03:34.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.947+0000 7f06644fc640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f064006b4d0 0x7f064006d990 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.947+0000 7f06644fc640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f065c1618c0 0x7f065c0c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.947+0000 7f06644fc640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f065c0cd610 0x7f065c159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.947+0000 7f06644fc640 1 --2- 172.21.15.139:0/3153804335 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f065c09ad40 0x7f065c1f4370 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.947+0000 7f06644fc640 1 -- 172.21.15.139:0/3153804335 >> 172.21.15.139:0/3153804335 conn(0x7f065c0b31f0 msgr2=0x7f065c0ceb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.947+0000 7f06644fc640 1 -- 172.21.15.139:0/3153804335 shutdown_connections 2024-04-25T10:03:34.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:34.947+0000 7f06644fc640 1 -- 172.21.15.139:0/3153804335 wait complete. 2024-04-25T10:03:34.963 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:03:34.963 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:03:35.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.059+0000 7fbfd4e93640 1 Processor -- start 2024-04-25T10:03:35.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.059+0000 7fbfd4e93640 1 -- start start 2024-04-25T10:03:35.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.060+0000 7fbfd4e93640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbfd00712f0 0x7fbfd00b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.060+0000 7fbfd4e93640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd00719d0 0x7fbfd00b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.060+0000 7fbfd4e93640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd00b1100 con 0x7fbfd0145250 2024-04-25T10:03:35.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.060+0000 7fbfd4e93640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd00ceed0 con 0x7fbfd00712f0 2024-04-25T10:03:35.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.060+0000 7fbfd4e93640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd00ce2c0 con 0x7fbfd00719d0 2024-04-25T10:03:35.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.061+0000 7fbfced9d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd00719d0 0x7fbfd00b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.061+0000 7fbfcdd9b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbfd00712f0 0x7fbfd00b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.061+0000 7fbfcdd9b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbfd00712f0 0x7fbfd00b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:46506/0 (socket says 172.21.15.139:46506) 2024-04-25T10:03:35.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.061+0000 7fbfcdd9b640 1 -- 172.21.15.139:0/2940854873 learned_addr learned my addr 172.21.15.139:0/2940854873 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:35.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.061+0000 7fbfcdd9b640 1 -- 172.21.15.139:0/2940854873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd00719d0 msgr2=0x7fbfd00b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.061+0000 7fbfcdd9b640 1 --2- 172.21.15.139:0/2940854873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd00719d0 0x7fbfd00b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.061+0000 7fbfcdd9b640 1 -- 172.21.15.139:0/2940854873 >> v1:172.21.15.150:6789/0 conn(0x7fbfd0145250 legacy=0x7fbfd0070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.061+0000 7fbfcd59a640 1 -- 172.21.15.139:0/2940854873 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159372579 0 0) 0x7fbfd00b1100 con 0x7fbfd0145250 2024-04-25T10:03:35.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.061+0000 7fbfcdd9b640 1 -- 172.21.15.139:0/2940854873 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd00ce5a0 con 0x7fbfd00712f0 2024-04-25T10:03:35.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.061+0000 7fbfced9d640 1 --2- 172.21.15.139:0/2940854873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd00719d0 0x7fbfd00b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.062+0000 7fbfcdd9b640 1 --2- 172.21.15.139:0/2940854873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd00712f0 0x7fbfd00b4680 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009870 tx=0x7fbfc402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb1bf5cce4dfc160 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.062+0000 7fbfcd59a640 1 -- 172.21.15.139:0/2940854873 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4037070 con 0x7fbfd00712f0 2024-04-25T10:03:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.062+0000 7fbfcd59a640 1 -- 172.21.15.139:0/2940854873 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc4032440 con 0x7fbfd00712f0 2024-04-25T10:03:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.062+0000 7fbfcd59a640 1 -- 172.21.15.139:0/2940854873 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4032810 con 0x7fbfd00712f0 2024-04-25T10:03:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.062+0000 7fbfd4e93640 1 -- 172.21.15.139:0/2940854873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd00712f0 msgr2=0x7fbfd00b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.062+0000 7fbfd4e93640 1 --2- 172.21.15.139:0/2940854873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd00712f0 0x7fbfd00b4680 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009870 tx=0x7fbfc402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.063+0000 7fbfd4e93640 1 -- 172.21.15.139:0/2940854873 shutdown_connections 2024-04-25T10:03:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.063+0000 7fbfd4e93640 1 --2- 172.21.15.139:0/2940854873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd00719d0 0x7fbfd00b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.063+0000 7fbfd4e93640 1 --2- 172.21.15.139:0/2940854873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd00712f0 0x7fbfd00b4680 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.063+0000 7fbfd4e93640 1 -- 172.21.15.139:0/2940854873 >> 172.21.15.139:0/2940854873 conn(0x7fbfd006f460 msgr2=0x7fbfd006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.063+0000 7fbfd4e93640 1 -- 172.21.15.139:0/2940854873 shutdown_connections 2024-04-25T10:03:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.063+0000 7fbfd4e93640 1 -- 172.21.15.139:0/2940854873 wait complete. 2024-04-25T10:03:35.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.064+0000 7fbfd4e93640 1 Processor -- start 2024-04-25T10:03:35.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.064+0000 7fbfd4e93640 1 -- start start 2024-04-25T10:03:35.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfd4e93640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd00712f0 0x7fbfd01e1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfd4e93640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfd00719d0 0x7fbfd013e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfd4e93640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0145250 0x7fbfd013e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfd4e93640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0145630 con 0x7fbfd00719d0 2024-04-25T10:03:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfd4e93640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd00ceb60 con 0x7fbfd0145250 2024-04-25T10:03:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfd4e93640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd00d11a0 con 0x7fbfd00712f0 2024-04-25T10:03:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfcdd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfd00719d0 0x7fbfd013e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfcdd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfd00719d0 0x7fbfd013e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42872/0 (socket says 172.21.15.139:42872) 2024-04-25T10:03:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfcdd9b640 1 -- 172.21.15.139:0/1235195134 learned_addr learned my addr 172.21.15.139:0/1235195134 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfced9d640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0145250 0x7fbfd013e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfce59c640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd00712f0 0x7fbfd01e1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfcdd9b640 1 -- 172.21.15.139:0/1235195134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd00712f0 msgr2=0x7fbfd01e1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfcdd9b640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd00712f0 0x7fbfd01e1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfcdd9b640 1 -- 172.21.15.139:0/1235195134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0145250 msgr2=0x7fbfd013e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfcdd9b640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0145250 0x7fbfd013e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfcdd9b640 1 -- 172.21.15.139:0/1235195134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd013d2c0 con 0x7fbfd00719d0 2024-04-25T10:03:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfcdd9b640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfd00719d0 0x7fbfd013e280 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4035040 tx=0x7fbfc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfbeffd640 1 -- 172.21.15.139:0/1235195134 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4037040 con 0x7fbfd00719d0 2024-04-25T10:03:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfbeffd640 1 -- 172.21.15.139:0/1235195134 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc4004230 con 0x7fbfd00719d0 2024-04-25T10:03:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfd4e93640 1 -- 172.21.15.139:0/1235195134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd00b4e40 con 0x7fbfd00719d0 2024-04-25T10:03:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.065+0000 7fbfbeffd640 1 -- 172.21.15.139:0/1235195134 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4003610 con 0x7fbfd00719d0 2024-04-25T10:03:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.066+0000 7fbfd4e93640 1 -- 172.21.15.139:0/1235195134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd01e22d0 con 0x7fbfd00719d0 2024-04-25T10:03:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.066+0000 7fbfd4e93640 1 -- 172.21.15.139:0/1235195134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbfd013d5a0 con 0x7fbfd00719d0 2024-04-25T10:03:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.067+0000 7fbfbeffd640 1 -- 172.21.15.139:0/1235195134 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfc402ad10 con 0x7fbfd00719d0 2024-04-25T10:03:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.067+0000 7fbfbeffd640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfb006b4d0 0x7fbfb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.067+0000 7fbfbeffd640 1 -- 172.21.15.139:0/1235195134 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fbfc40aaa00 con 0x7fbfd00719d0 2024-04-25T10:03:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.067+0000 7fbfce59c640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfb006b4d0 0x7fbfb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.067+0000 7fbfbeffd640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x7fbfb0071ab0 0x7fbfb0073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.067+0000 7fbfbeffd640 1 -- 172.21.15.139:0/1235195134 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfc4008f00 con 0x7fbfb0071ab0 2024-04-25T10:03:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.067+0000 7fbfbeffd640 1 -- 172.21.15.139:0/1235195134 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfc40ace60 con 0x7fbfd00719d0 2024-04-25T10:03:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.068+0000 7fbfced9d640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x7fbfb0071ab0 0x7fbfb0073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.068+0000 7fbfce59c640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfb006b4d0 0x7fbfb006d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00012e0 tx=0x7fbfc0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.068+0000 7fbfced9d640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x7fbfb0071ab0 0x7fbfb0073ef0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.068+0000 7fbfbeffd640 1 -- 172.21.15.139:0/1235195134 <== osd.1 v2:172.21.15.139:6814/315640110 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbfc4008f00 con 0x7fbfb0071ab0 2024-04-25T10:03:35.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.081+0000 7fbfd4e93640 1 -- 172.21.15.139:0/1235195134 --> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbfd0074520 con 0x7fbfb0071ab0 2024-04-25T10:03:35.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.082+0000 7fbfbeffd640 1 -- 172.21.15.139:0/1235195134 <== osd.1 v2:172.21.15.139:6814/315640110 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbfd0074520 con 0x7fbfb0071ab0 2024-04-25T10:03:35.086 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:03:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:03:35.087 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:03:35.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.082+0000 7fbfd4e93640 1 -- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x7fbfb0071ab0 msgr2=0x7fbfb0073ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.082+0000 7fbfd4e93640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:6814/315640110,v1:172.21.15.139:6816/315640110] conn(0x7fbfb0071ab0 0x7fbfb0073ef0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.082+0000 7fbfd4e93640 1 -- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfb006b4d0 msgr2=0x7fbfb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.082+0000 7fbfd4e93640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbfb006b4d0 0x7fbfb006d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00012e0 tx=0x7fbfc0009040 comp rx=0 tx=0).stop 2024-04-25T10:03:35.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.082+0000 7fbfd4e93640 1 -- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfd00719d0 msgr2=0x7fbfd013e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.082+0000 7fbfd4e93640 1 --2- 172.21.15.139:0/1235195134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfd00719d0 0x7fbfd013e280 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4035040 tx=0x7fbfc40046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.082+0000 7fbfced9d640 1 -- 172.21.15.139:0/1235195134 reap_dead start 2024-04-25T10:03:35.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.082+0000 7fbfd4e93640 1 -- 172.21.15.139:0/1235195134 shutdown_connections 2024-04-25T10:03:35.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.082+0000 7fbfd4e93640 1 -- 172.21.15.139:0/1235195134 >> 172.21.15.139:0/1235195134 conn(0x7fbfd006f460 msgr2=0x7fbfd0143ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.082+0000 7fbfd4e93640 1 -- 172.21.15.139:0/1235195134 shutdown_connections 2024-04-25T10:03:35.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.082+0000 7fbfd4e93640 1 -- 172.21.15.139:0/1235195134 wait complete. 2024-04-25T10:03:35.097 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:03:35.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.191+0000 7ff8d154f640 1 Processor -- start 2024-04-25T10:03:35.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.191+0000 7ff8d154f640 1 -- start start 2024-04-25T10:03:35.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.191+0000 7ff8d154f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff8cc15dbf0 0x7ff8cc1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.191+0000 7ff8d154f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc0b9370 0x7ff8cc0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.191+0000 7ff8d154f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff8cc0cec80 0x7ff8cc06c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.191+0000 7ff8d154f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8cc0d1ee0 con 0x7ff8cc0cec80 2024-04-25T10:03:35.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.191+0000 7ff8d154f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8cc161dc0 con 0x7ff8cc15dbf0 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.191+0000 7ff8d154f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc1620a0 con 0x7ff8cc0b9370 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.192+0000 7ff8cb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff8cc0cec80 0x7ff8cc06c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.192+0000 7ff8cb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff8cc0cec80 0x7ff8cc06c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42882/0 (socket says 172.21.15.139:42882) 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.192+0000 7ff8cb7fe640 1 -- 172.21.15.139:0/2989812101 learned_addr learned my addr 172.21.15.139:0/2989812101 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.192+0000 7ff8ca7fc640 1 --2- 172.21.15.139:0/2989812101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc0b9370 0x7ff8cc0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.192+0000 7ff8caffd640 1 --2- 172.21.15.139:0/2989812101 >> v2:172.21.15.150:3300/0 conn(0x7ff8cc15dbf0 0x7ff8cc1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.192+0000 7ff8cb7fe640 1 -- 172.21.15.139:0/2989812101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc0b9370 msgr2=0x7ff8cc0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.192+0000 7ff8cb7fe640 1 --2- 172.21.15.139:0/2989812101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc0b9370 0x7ff8cc0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.192+0000 7ff8cb7fe640 1 -- 172.21.15.139:0/2989812101 >> v2:172.21.15.150:3300/0 conn(0x7ff8cc15dbf0 msgr2=0x7ff8cc1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.192+0000 7ff8cb7fe640 1 --2- 172.21.15.139:0/2989812101 >> v2:172.21.15.150:3300/0 conn(0x7ff8cc15dbf0 0x7ff8cc1617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.192+0000 7ff8cb7fe640 1 -- 172.21.15.139:0/2989812101 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc162380 con 0x7ff8cc0cec80 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.192+0000 7ff8ca7fc640 1 --2- 172.21.15.139:0/2989812101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc0b9370 0x7ff8cc0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.192+0000 7ff8caffd640 1 --2- 172.21.15.139:0/2989812101 >> v2:172.21.15.150:3300/0 conn(0x7ff8cc15dbf0 0x7ff8cc1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:35.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8cb7fe640 1 --2- 172.21.15.139:0/2989812101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8cc0cec80 0x7ff8cc06c210 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc06d330 tx=0x7ff8c402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68282cf8aadcd79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8c9ffb640 1 -- 172.21.15.139:0/2989812101 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4036070 con 0x7ff8cc0cec80 2024-04-25T10:03:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8c9ffb640 1 -- 172.21.15.139:0/2989812101 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c4031600 con 0x7ff8cc0cec80 2024-04-25T10:03:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8c9ffb640 1 -- 172.21.15.139:0/2989812101 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c40319d0 con 0x7ff8cc0cec80 2024-04-25T10:03:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8d154f640 1 -- 172.21.15.139:0/2989812101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8cc0cec80 msgr2=0x7ff8cc06c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8d154f640 1 --2- 172.21.15.139:0/2989812101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8cc0cec80 0x7ff8cc06c210 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc06d330 tx=0x7ff8c402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8d154f640 1 -- 172.21.15.139:0/2989812101 shutdown_connections 2024-04-25T10:03:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8d154f640 1 --2- 172.21.15.139:0/2989812101 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8cc0cec80 0x7ff8cc06c210 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8d154f640 1 --2- 172.21.15.139:0/2989812101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc0b9370 0x7ff8cc0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8d154f640 1 --2- 172.21.15.139:0/2989812101 >> v2:172.21.15.150:3300/0 conn(0x7ff8cc15dbf0 0x7ff8cc1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8d154f640 1 -- 172.21.15.139:0/2989812101 >> 172.21.15.139:0/2989812101 conn(0x7ff8cc0b5480 msgr2=0x7ff8cc0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8d154f640 1 -- 172.21.15.139:0/2989812101 shutdown_connections 2024-04-25T10:03:35.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.193+0000 7ff8d154f640 1 -- 172.21.15.139:0/2989812101 wait complete. 2024-04-25T10:03:35.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.195+0000 7ff8d154f640 1 Processor -- start 2024-04-25T10:03:35.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.195+0000 7ff8d154f640 1 -- start start 2024-04-25T10:03:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.195+0000 7ff8d154f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc0b9370 0x7ff8cc158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.195+0000 7ff8d154f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8cc0cec80 0x7ff8cc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.195+0000 7ff8d154f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8cc15dbf0 0x7ff8cc0ab220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.195+0000 7ff8d154f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc15dfd0 con 0x7ff8cc0cec80 2024-04-25T10:03:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.195+0000 7ff8d154f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0b97d0 con 0x7ff8cc15dbf0 2024-04-25T10:03:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.195+0000 7ff8d154f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc15cc90 con 0x7ff8cc0b9370 2024-04-25T10:03:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.195+0000 7ff8ca7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8cc0cec80 0x7ff8cc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.195+0000 7ff8ca7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8cc0cec80 0x7ff8cc15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42898/0 (socket says 172.21.15.139:42898) 2024-04-25T10:03:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.195+0000 7ff8ca7fc640 1 -- 172.21.15.139:0/2850743001 learned_addr learned my addr 172.21.15.139:0/2850743001 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8cb7fe640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8cc15dbf0 0x7ff8cc0ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8caffd640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc0b9370 0x7ff8cc158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8ca7fc640 1 -- 172.21.15.139:0/2850743001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc0b9370 msgr2=0x7ff8cc158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8ca7fc640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc0b9370 0x7ff8cc158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8ca7fc640 1 -- 172.21.15.139:0/2850743001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8cc15dbf0 msgr2=0x7ff8cc0ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8ca7fc640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8cc15dbf0 0x7ff8cc0ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8ca7fc640 1 -- 172.21.15.139:0/2850743001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc15c080 con 0x7ff8cc0cec80 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8caffd640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc0b9370 0x7ff8cc158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8ca7fc640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8cc0cec80 0x7ff8cc15b6a0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000d800 tx=0x7ff8c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8b37fe640 1 -- 172.21.15.139:0/2850743001 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0018070 con 0x7ff8cc0cec80 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8b37fe640 1 -- 172.21.15.139:0/2850743001 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c0004030 con 0x7ff8cc0cec80 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8b37fe640 1 -- 172.21.15.139:0/2850743001 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0004400 con 0x7ff8cc0cec80 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8d154f640 1 -- 172.21.15.139:0/2850743001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc15c360 con 0x7ff8cc0cec80 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.196+0000 7ff8d154f640 1 -- 172.21.15.139:0/2850743001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8cc15c920 con 0x7ff8cc0cec80 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.197+0000 7ff8d154f640 1 -- 172.21.15.139:0/2850743001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8cc074520 con 0x7ff8cc0cec80 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.197+0000 7ff8b37fe640 1 -- 172.21.15.139:0/2850743001 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8c00096a0 con 0x7ff8cc0cec80 2024-04-25T10:03:35.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.198+0000 7ff8b37fe640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8b806b4d0 0x7ff8b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.198+0000 7ff8b37fe640 1 -- 172.21.15.139:0/2850743001 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ff8c008a490 con 0x7ff8cc0cec80 2024-04-25T10:03:35.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.198+0000 7ff8caffd640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8b806b4d0 0x7ff8b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.198+0000 7ff8caffd640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8b806b4d0 0x7ff8b806d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0c9f70 tx=0x7ff8bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.202+0000 7ff8b37fe640 1 -- 172.21.15.139:0/2850743001 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8c0059460 con 0x7ff8cc0cec80 2024-04-25T10:03:35.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.371+0000 7ff8d154f640 1 -- 172.21.15.139:0/2850743001 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8cc0d1680 con 0x7ff8b806b4d0 2024-04-25T10:03:35.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.372+0000 7ff8b37fe640 1 -- 172.21.15.139:0/2850743001 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff8cc0d1680 con 0x7ff8b806b4d0 2024-04-25T10:03:35.377 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:03:35.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.373+0000 7ff8d154f640 1 -- 172.21.15.139:0/2850743001 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8b806b4d0 msgr2=0x7ff8b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.373+0000 7ff8d154f640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8b806b4d0 0x7ff8b806d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0c9f70 tx=0x7ff8bc008040 comp rx=0 tx=0).stop 2024-04-25T10:03:35.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.373+0000 7ff8d154f640 1 -- 172.21.15.139:0/2850743001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8cc0cec80 msgr2=0x7ff8cc15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.373+0000 7ff8d154f640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8cc0cec80 0x7ff8cc15b6a0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000d800 tx=0x7ff8c000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.373+0000 7ff8d154f640 1 -- 172.21.15.139:0/2850743001 shutdown_connections 2024-04-25T10:03:35.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.373+0000 7ff8d154f640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8cc0cec80 0x7ff8cc15b6a0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.373+0000 7ff8d154f640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7ff8b806b4d0 0x7ff8b806d990 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.373+0000 7ff8d154f640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8cc15dbf0 0x7ff8cc0ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.373+0000 7ff8d154f640 1 --2- 172.21.15.139:0/2850743001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc0b9370 0x7ff8cc158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.373+0000 7ff8d154f640 1 -- 172.21.15.139:0/2850743001 >> 172.21.15.139:0/2850743001 conn(0x7ff8cc0b5480 msgr2=0x7ff8cc0caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.374+0000 7ff8d154f640 1 -- 172.21.15.139:0/2850743001 shutdown_connections 2024-04-25T10:03:35.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.374+0000 7ff8d154f640 1 -- 172.21.15.139:0/2850743001 wait complete. 2024-04-25T10:03:35.389 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:03:35.389 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:03:35.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.489+0000 7f0b63676640 1 Processor -- start 2024-04-25T10:03:35.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.489+0000 7f0b63676640 1 -- start start 2024-04-25T10:03:35.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.489+0000 7f0b63676640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0b5c0cd610 0x7f0b5c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.489+0000 7f0b63676640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0b5c161fd0 0x7f0b5c0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.489+0000 7f0b63676640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5c099ca0 con 0x7f0b5c09ad40 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.489+0000 7f0b63676640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5c099a50 con 0x7f0b5c0cd610 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.489+0000 7f0b63676640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5c099f80 con 0x7f0b5c161fd0 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.490+0000 7f0b61412640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0b5c0cd610 0x7f0b5c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.490+0000 7f0b60c11640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0b5c09ad40 0x7f0b5c1618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34612/0 (socket says 172.21.15.139:34612) 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.490+0000 7f0b60c11640 1 -- 172.21.15.139:0/2561890836 learned_addr learned my addr 172.21.15.139:0/2561890836 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.490+0000 7f0b61c13640 1 --2- 172.21.15.139:0/2561890836 >> v2:172.21.15.150:3300/0 conn(0x7f0b5c161fd0 0x7f0b5c0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.490+0000 7f0b61412640 1 -- 172.21.15.139:0/2561890836 >> v2:172.21.15.150:3300/0 conn(0x7f0b5c161fd0 msgr2=0x7f0b5c0d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.490+0000 7f0b61412640 1 --2- 172.21.15.139:0/2561890836 >> v2:172.21.15.150:3300/0 conn(0x7f0b5c161fd0 0x7f0b5c0d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.490+0000 7f0b53fff640 1 -- 172.21.15.139:0/2561890836 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801263341 0 0) 0x7f0b5c099ca0 con 0x7f0b5c09ad40 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.490+0000 7f0b61412640 1 -- 172.21.15.139:0/2561890836 >> v1:172.21.15.139:6789/0 conn(0x7f0b5c09ad40 legacy=0x7f0b5c1618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.490+0000 7f0b61412640 1 -- 172.21.15.139:0/2561890836 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c09a260 con 0x7f0b5c0cd610 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.490+0000 7f0b61c13640 1 --2- 172.21.15.139:0/2561890836 >> v2:172.21.15.150:3300/0 conn(0x7f0b5c161fd0 0x7f0b5c0d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.491+0000 7f0b61412640 1 --2- 172.21.15.139:0/2561890836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b5c0cd610 0x7f0b5c0d0370 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0b42d0 tx=0x7f0b4c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cc00995d4877e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.491+0000 7f0b53fff640 1 -- 172.21.15.139:0/2561890836 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c037070 con 0x7f0b5c0cd610 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.491+0000 7f0b53fff640 1 -- 172.21.15.139:0/2561890836 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c032440 con 0x7f0b5c0cd610 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.491+0000 7f0b53fff640 1 -- 172.21.15.139:0/2561890836 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c032810 con 0x7f0b5c0cd610 2024-04-25T10:03:35.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.491+0000 7f0b63676640 1 -- 172.21.15.139:0/2561890836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b5c0cd610 msgr2=0x7f0b5c0d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.491+0000 7f0b63676640 1 --2- 172.21.15.139:0/2561890836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b5c0cd610 0x7f0b5c0d0370 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0b42d0 tx=0x7f0b4c02a230 comp rx=0 tx=0).stop 2024-04-25T10:03:35.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.491+0000 7f0b63676640 1 -- 172.21.15.139:0/2561890836 shutdown_connections 2024-04-25T10:03:35.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.491+0000 7f0b63676640 1 --2- 172.21.15.139:0/2561890836 >> v2:172.21.15.150:3300/0 conn(0x7f0b5c161fd0 0x7f0b5c0d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.491+0000 7f0b63676640 1 --2- 172.21.15.139:0/2561890836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b5c0cd610 0x7f0b5c0d0370 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.491+0000 7f0b63676640 1 -- 172.21.15.139:0/2561890836 >> 172.21.15.139:0/2561890836 conn(0x7f0b5c0b31f0 msgr2=0x7f0b5c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.491+0000 7f0b63676640 1 -- 172.21.15.139:0/2561890836 shutdown_connections 2024-04-25T10:03:35.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.491+0000 7f0b63676640 1 -- 172.21.15.139:0/2561890836 wait complete. 2024-04-25T10:03:35.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.493+0000 7f0b63676640 1 Processor -- start 2024-04-25T10:03:35.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.493+0000 7f0b63676640 1 -- start start 2024-04-25T10:03:35.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.493+0000 7f0b63676640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b5c09ad40 0x7f0b5c1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.493+0000 7f0b63676640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b5c0cd610 0x7f0b5c156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.493+0000 7f0b63676640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b5c161fd0 0x7f0b5c156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.493+0000 7f0b63676640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c0cd9f0 con 0x7f0b5c161fd0 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.493+0000 7f0b63676640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c09a820 con 0x7f0b5c09ad40 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.493+0000 7f0b63676640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c0cdb70 con 0x7f0b5c0cd610 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.493+0000 7f0b61c13640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b5c161fd0 0x7f0b5c156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.493+0000 7f0b61c13640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b5c161fd0 0x7f0b5c156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42924/0 (socket says 172.21.15.139:42924) 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.493+0000 7f0b61c13640 1 -- 172.21.15.139:0/2117007688 learned_addr learned my addr 172.21.15.139:0/2117007688 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b61412640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b5c09ad40 0x7f0b5c1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b61c13640 1 -- 172.21.15.139:0/2117007688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b5c0cd610 msgr2=0x7f0b5c156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b60c11640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b5c0cd610 0x7f0b5c156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b61c13640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b5c0cd610 0x7f0b5c156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b61c13640 1 -- 172.21.15.139:0/2117007688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b5c09ad40 msgr2=0x7f0b5c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b61c13640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b5c09ad40 0x7f0b5c1604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b61c13640 1 -- 172.21.15.139:0/2117007688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c0b4570 con 0x7f0b5c161fd0 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b61412640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b5c09ad40 0x7f0b5c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b60c11640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b5c0cd610 0x7f0b5c156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b61c13640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b5c161fd0 0x7f0b5c156840 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400cd80 tx=0x7f0b5400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b51ffb640 1 -- 172.21.15.139:0/2117007688 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5400f680 con 0x7f0b5c161fd0 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b51ffb640 1 -- 172.21.15.139:0/2117007688 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5400f840 con 0x7f0b5c161fd0 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b63676640 1 -- 172.21.15.139:0/2117007688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5c0b4850 con 0x7f0b5c161fd0 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b51ffb640 1 -- 172.21.15.139:0/2117007688 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5400fc30 con 0x7f0b5c161fd0 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.494+0000 7f0b63676640 1 -- 172.21.15.139:0/2117007688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c0b4e10 con 0x7f0b5c161fd0 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.495+0000 7f0b63676640 1 -- 172.21.15.139:0/2117007688 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b5c0b4320 con 0x7f0b5c161fd0 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.495+0000 7f0b51ffb640 1 -- 172.21.15.139:0/2117007688 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b540213f0 con 0x7f0b5c161fd0 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.496+0000 7f0b51ffb640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0b4006b4d0 0x7f0b4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.496+0000 7f0b51ffb640 1 -- 172.21.15.139:0/2117007688 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f0b5408ae90 con 0x7f0b5c161fd0 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.496+0000 7f0b61412640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0b4006b4d0 0x7f0b4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.496+0000 7f0b51ffb640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x7f0b40071ab0 0x7f0b40073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.496+0000 7f0b51ffb640 1 -- 172.21.15.139:0/2117007688 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b400745c0 con 0x7f0b40071ab0 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.496+0000 7f0b51ffb640 1 -- 172.21.15.139:0/2117007688 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b5408d590 con 0x7f0b5c161fd0 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.496+0000 7f0b60c11640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x7f0b40071ab0 0x7f0b40073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.496+0000 7f0b61412640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0b4006b4d0 0x7f0b4006d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c034000 tx=0x7f0b4c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.496+0000 7f0b60c11640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x7f0b40071ab0 0x7f0b40073ef0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.497+0000 7f0b51ffb640 1 -- 172.21.15.139:0/2117007688 <== osd.2 v2:172.21.15.139:6800/1982336775 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0b400745c0 con 0x7f0b40071ab0 2024-04-25T10:03:35.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.510+0000 7f0b63676640 1 -- 172.21.15.139:0/2117007688 --> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0b5c074520 con 0x7f0b40071ab0 2024-04-25T10:03:35.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.510+0000 7f0b51ffb640 1 -- 172.21.15.139:0/2117007688 <== osd.2 v2:172.21.15.139:6800/1982336775 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0b5c074520 con 0x7f0b40071ab0 2024-04-25T10:03:35.516 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:03:35.516 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:03:35.516 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:03:35.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.510+0000 7f0b63676640 1 -- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x7f0b40071ab0 msgr2=0x7f0b40073ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.510+0000 7f0b63676640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:6800/1982336775,v1:172.21.15.139:6801/1982336775] conn(0x7f0b40071ab0 0x7f0b40073ef0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.511+0000 7f0b63676640 1 -- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0b4006b4d0 msgr2=0x7f0b4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.511+0000 7f0b63676640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0b4006b4d0 0x7f0b4006d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c034000 tx=0x7f0b4c02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.511+0000 7f0b63676640 1 -- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b5c161fd0 msgr2=0x7f0b5c156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.511+0000 7f0b63676640 1 --2- 172.21.15.139:0/2117007688 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b5c161fd0 0x7f0b5c156840 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400cd80 tx=0x7f0b5400a650 comp rx=0 tx=0).stop 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.511+0000 7f0b61c13640 1 -- 172.21.15.139:0/2117007688 reap_dead start 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.511+0000 7f0b63676640 1 -- 172.21.15.139:0/2117007688 shutdown_connections 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.511+0000 7f0b63676640 1 -- 172.21.15.139:0/2117007688 >> 172.21.15.139:0/2117007688 conn(0x7f0b5c0b31f0 msgr2=0x7f0b5c06f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.511+0000 7f0b63676640 1 -- 172.21.15.139:0/2117007688 shutdown_connections 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.511+0000 7f0b63676640 1 -- 172.21.15.139:0/2117007688 wait complete. 2024-04-25T10:03:35.526 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:03:35.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.619+0000 7fd107add640 1 Processor -- start 2024-04-25T10:03:35.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.619+0000 7fd107add640 1 -- start start 2024-04-25T10:03:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd107add640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd10015f8c0 0x7fd10009a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd107add640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd10015fca0 0x7fd1000cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd107add640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd1000d0220 0x7fd10015b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd107add640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1000d06c0 con 0x7fd10015fca0 2024-04-25T10:03:35.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd107add640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd100099bf0 con 0x7fd10015f8c0 2024-04-25T10:03:35.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd107add640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd100099ed0 con 0x7fd1000d0220 2024-04-25T10:03:35.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd105078640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd10015fca0 0x7fd1000cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd105078640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd10015fca0 0x7fd1000cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42926/0 (socket says 172.21.15.139:42926) 2024-04-25T10:03:35.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd105078640 1 -- 172.21.15.139:0/575950019 learned_addr learned my addr 172.21.15.139:0/575950019 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:35.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd105078640 1 -- 172.21.15.139:0/575950019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd1000d0220 msgr2=0x7fd10015b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd105078640 1 --2- 172.21.15.139:0/575950019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd1000d0220 0x7fd10015b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd105879640 1 --2- 172.21.15.139:0/575950019 >> v2:172.21.15.150:3300/0 conn(0x7fd10015f8c0 0x7fd10009a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd105078640 1 -- 172.21.15.139:0/575950019 >> v2:172.21.15.150:3300/0 conn(0x7fd10015f8c0 msgr2=0x7fd10009a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd105078640 1 --2- 172.21.15.139:0/575950019 >> v2:172.21.15.150:3300/0 conn(0x7fd10015f8c0 0x7fd10009a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.620+0000 7fd105078640 1 -- 172.21.15.139:0/575950019 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10009a1b0 con 0x7fd10015fca0 2024-04-25T10:03:35.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.621+0000 7fd105078640 1 --2- 172.21.15.139:0/575950019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd10015fca0 0x7fd1000cfce0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fd10006f210 tx=0x7fd0f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f69fa6a2ffb427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.621+0000 7fd104877640 1 -- 172.21.15.139:0/575950019 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0037070 con 0x7fd10015fca0 2024-04-25T10:03:35.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.621+0000 7fd104877640 1 -- 172.21.15.139:0/575950019 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f0032440 con 0x7fd10015fca0 2024-04-25T10:03:35.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.621+0000 7fd104877640 1 -- 172.21.15.139:0/575950019 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0032810 con 0x7fd10015fca0 2024-04-25T10:03:35.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.622+0000 7fd107add640 1 -- 172.21.15.139:0/575950019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd10015fca0 msgr2=0x7fd1000cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.622+0000 7fd107add640 1 --2- 172.21.15.139:0/575950019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd10015fca0 0x7fd1000cfce0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fd10006f210 tx=0x7fd0f002a270 comp rx=0 tx=0).stop 2024-04-25T10:03:35.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.622+0000 7fd107add640 1 -- 172.21.15.139:0/575950019 shutdown_connections 2024-04-25T10:03:35.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.622+0000 7fd107add640 1 --2- 172.21.15.139:0/575950019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd1000d0220 0x7fd10015b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.622+0000 7fd107add640 1 --2- 172.21.15.139:0/575950019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd10015fca0 0x7fd1000cfce0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.622+0000 7fd107add640 1 --2- 172.21.15.139:0/575950019 >> v2:172.21.15.150:3300/0 conn(0x7fd10015f8c0 0x7fd10009a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.622+0000 7fd107add640 1 -- 172.21.15.139:0/575950019 >> 172.21.15.139:0/575950019 conn(0x7fd1000b31f0 msgr2=0x7fd1000b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.622+0000 7fd107add640 1 -- 172.21.15.139:0/575950019 shutdown_connections 2024-04-25T10:03:35.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.622+0000 7fd107add640 1 -- 172.21.15.139:0/575950019 wait complete. 2024-04-25T10:03:35.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.623+0000 7fd107add640 1 Processor -- start 2024-04-25T10:03:35.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.623+0000 7fd107add640 1 -- start start 2024-04-25T10:03:35.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd107add640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1000d0220 0x7fd10015e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd107add640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10015f8c0 0x7fd10015cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd107add640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd10015fca0 0x7fd10015d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd107add640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1001602d0 con 0x7fd1000d0220 2024-04-25T10:03:35.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd107add640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10009aef0 con 0x7fd10015f8c0 2024-04-25T10:03:35.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd107add640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd100160450 con 0x7fd10015fca0 2024-04-25T10:03:35.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd105879640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1000d0220 0x7fd10015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd105879640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1000d0220 0x7fd10015e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42932/0 (socket says 172.21.15.139:42932) 2024-04-25T10:03:35.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd105879640 1 -- 172.21.15.139:0/303006897 learned_addr learned my addr 172.21.15.139:0/303006897 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:35.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd105078640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10015f8c0 0x7fd10015cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd10607a640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd10015fca0 0x7fd10015d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd105879640 1 -- 172.21.15.139:0/303006897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd10015fca0 msgr2=0x7fd10015d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd105879640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd10015fca0 0x7fd10015d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd105879640 1 -- 172.21.15.139:0/303006897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10015f8c0 msgr2=0x7fd10015cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd105879640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10015f8c0 0x7fd10015cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd105879640 1 -- 172.21.15.139:0/303006897 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10006d900 con 0x7fd1000d0220 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd105879640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1000d0220 0x7fd10015e030 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00c7f0 tx=0x7fd0fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd0f67fc640 1 -- 172.21.15.139:0/303006897 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc015070 con 0x7fd1000d0220 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd0f67fc640 1 -- 172.21.15.139:0/303006897 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0fc004240 con 0x7fd1000d0220 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd107add640 1 -- 172.21.15.139:0/303006897 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10006ca00 con 0x7fd1000d0220 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd0f67fc640 1 -- 172.21.15.139:0/303006897 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc007620 con 0x7fd1000d0220 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.624+0000 7fd107add640 1 -- 172.21.15.139:0/303006897 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10006cfc0 con 0x7fd1000d0220 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.625+0000 7fd107add640 1 -- 172.21.15.139:0/303006897 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd100074520 con 0x7fd1000d0220 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.626+0000 7fd0f67fc640 1 -- 172.21.15.139:0/303006897 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0fc0077c0 con 0x7fd1000d0220 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.626+0000 7fd0f67fc640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd0e406b4d0 0x7fd0e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.626+0000 7fd0f67fc640 1 -- 172.21.15.139:0/303006897 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fd0fc08a3b0 con 0x7fd1000d0220 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.626+0000 7fd105078640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd0e406b4d0 0x7fd0e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.627+0000 7fd105078640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd0e406b4d0 0x7fd0e406d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0034000 tx=0x7fd0f002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.630+0000 7fd0f67fc640 1 -- 172.21.15.139:0/303006897 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0fc0592f0 con 0x7fd1000d0220 2024-04-25T10:03:35.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.799+0000 7fd107add640 1 -- 172.21.15.139:0/303006897 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1000cdc30 con 0x7fd0e406b4d0 2024-04-25T10:03:35.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.799+0000 7fd0f67fc640 1 -- 172.21.15.139:0/303006897 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd1000cdc30 con 0x7fd0e406b4d0 2024-04-25T10:03:35.804 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:03:35.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.801+0000 7fd107add640 1 -- 172.21.15.139:0/303006897 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd0e406b4d0 msgr2=0x7fd0e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.801+0000 7fd107add640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd0e406b4d0 0x7fd0e406d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0034000 tx=0x7fd0f002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:35.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.801+0000 7fd107add640 1 -- 172.21.15.139:0/303006897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1000d0220 msgr2=0x7fd10015e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.801+0000 7fd107add640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1000d0220 0x7fd10015e030 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00c7f0 tx=0x7fd0fc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.801+0000 7fd107add640 1 -- 172.21.15.139:0/303006897 shutdown_connections 2024-04-25T10:03:35.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.801+0000 7fd107add640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd0e406b4d0 0x7fd0e406d990 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.801+0000 7fd107add640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd10015fca0 0x7fd10015d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.801+0000 7fd107add640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10015f8c0 0x7fd10015cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.801+0000 7fd107add640 1 --2- 172.21.15.139:0/303006897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1000d0220 0x7fd10015e030 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.801+0000 7fd107add640 1 -- 172.21.15.139:0/303006897 >> 172.21.15.139:0/303006897 conn(0x7fd1000b31f0 msgr2=0x7fd10006ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.802+0000 7fd107add640 1 -- 172.21.15.139:0/303006897 shutdown_connections 2024-04-25T10:03:35.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.802+0000 7fd107add640 1 -- 172.21.15.139:0/303006897 wait complete. 2024-04-25T10:03:35.817 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:03:35.817 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:03:35.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.914+0000 7fb024132640 1 Processor -- start 2024-04-25T10:03:35.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.914+0000 7fb024132640 1 -- start start 2024-04-25T10:03:35.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb024132640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb01c0cd610 0x7fb01c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb024132640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01c09ad40 0x7fb01c0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb024132640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c099ca0 con 0x7fb01c1618c0 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb024132640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c099a50 con 0x7fb01c0cd610 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb024132640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c099f80 con 0x7fb01c09ad40 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb021ece640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb01c0cd610 0x7fb01c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb0226cf640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb01c1618c0 0x7fb01c06be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34626/0 (socket says 172.21.15.139:34626) 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb0226cf640 1 -- 172.21.15.139:0/2487438660 learned_addr learned my addr 172.21.15.139:0/2487438660 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb0216cd640 1 --2- 172.21.15.139:0/2487438660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01c09ad40 0x7fb01c0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb021ece640 1 -- 172.21.15.139:0/2487438660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01c09ad40 msgr2=0x7fb01c0d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb021ece640 1 --2- 172.21.15.139:0/2487438660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01c09ad40 0x7fb01c0d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb021ece640 1 -- 172.21.15.139:0/2487438660 >> v1:172.21.15.139:6789/0 conn(0x7fb01c1618c0 legacy=0x7fb01c06be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb021ece640 1 -- 172.21.15.139:0/2487438660 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c09a260 con 0x7fb01c0cd610 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb020ecc640 1 -- 172.21.15.139:0/2487438660 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175480788 0 0) 0x7fb01c099ca0 con 0x7fb01c1618c0 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.915+0000 7fb0216cd640 1 --2- 172.21.15.139:0/2487438660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01c09ad40 0x7fb01c0d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.916+0000 7fb021ece640 1 --2- 172.21.15.139:0/2487438660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c0cd610 0x7fb01c0d0370 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fb018009d90 tx=0x7fb01802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31225c1a4910e4dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.916+0000 7fb020ecc640 1 -- 172.21.15.139:0/2487438660 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018007740 con 0x7fb01c0cd610 2024-04-25T10:03:35.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.916+0000 7fb020ecc640 1 -- 172.21.15.139:0/2487438660 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0180078e0 con 0x7fb01c0cd610 2024-04-25T10:03:35.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.916+0000 7fb020ecc640 1 -- 172.21.15.139:0/2487438660 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018007cb0 con 0x7fb01c0cd610 2024-04-25T10:03:35.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.916+0000 7fb024132640 1 -- 172.21.15.139:0/2487438660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c0cd610 msgr2=0x7fb01c0d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.916+0000 7fb024132640 1 --2- 172.21.15.139:0/2487438660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c0cd610 0x7fb01c0d0370 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fb018009d90 tx=0x7fb01802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.916+0000 7fb024132640 1 -- 172.21.15.139:0/2487438660 shutdown_connections 2024-04-25T10:03:35.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.916+0000 7fb024132640 1 --2- 172.21.15.139:0/2487438660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01c09ad40 0x7fb01c0d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.916+0000 7fb024132640 1 --2- 172.21.15.139:0/2487438660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c0cd610 0x7fb01c0d0370 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.916+0000 7fb024132640 1 -- 172.21.15.139:0/2487438660 >> 172.21.15.139:0/2487438660 conn(0x7fb01c0b31f0 msgr2=0x7fb01c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.916+0000 7fb024132640 1 -- 172.21.15.139:0/2487438660 shutdown_connections 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.917+0000 7fb024132640 1 -- 172.21.15.139:0/2487438660 wait complete. 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.918+0000 7fb024132640 1 Processor -- start 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.918+0000 7fb024132640 1 -- start start 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.918+0000 7fb024132640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c09ad40 0x7fb01c159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.918+0000 7fb024132640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb01c0cd610 0x7fb01c0b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.918+0000 7fb024132640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01c1618c0 0x7fb01c15e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.918+0000 7fb024132640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0cd9f0 con 0x7fb01c09ad40 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.918+0000 7fb024132640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c09a820 con 0x7fb01c0cd610 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.918+0000 7fb024132640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0cdb70 con 0x7fb01c1618c0 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.918+0000 7fb021ece640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c09ad40 0x7fb01c159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.918+0000 7fb021ece640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c09ad40 0x7fb01c159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42954/0 (socket says 172.21.15.139:42954) 2024-04-25T10:03:35.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.918+0000 7fb021ece640 1 -- 172.21.15.139:0/3952324374 learned_addr learned my addr 172.21.15.139:0/3952324374 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:35.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.919+0000 7fb021ece640 1 -- 172.21.15.139:0/3952324374 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01c1618c0 msgr2=0x7fb01c15e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.919+0000 7fb021ece640 1 --2- 172.21.15.139:0/3952324374 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb01c1618c0 0x7fb01c15e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.919+0000 7fb021ece640 1 -- 172.21.15.139:0/3952324374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb01c0cd610 msgr2=0x7fb01c0b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.919+0000 7fb021ece640 1 --2- 172.21.15.139:0/3952324374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb01c0cd610 0x7fb01c0b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.919+0000 7fb021ece640 1 -- 172.21.15.139:0/3952324374 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c159ea0 con 0x7fb01c09ad40 2024-04-25T10:03:35.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.919+0000 7fb021ece640 1 --2- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c09ad40 0x7fb01c159960 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fb018009d90 tx=0x7fb018004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.919+0000 7fb0127fc640 1 -- 172.21.15.139:0/3952324374 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018040070 con 0x7fb01c09ad40 2024-04-25T10:03:35.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.919+0000 7fb0127fc640 1 -- 172.21.15.139:0/3952324374 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb018031c40 con 0x7fb01c09ad40 2024-04-25T10:03:35.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.919+0000 7fb024132640 1 -- 172.21.15.139:0/3952324374 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c0b5130 con 0x7fb01c09ad40 2024-04-25T10:03:35.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.919+0000 7fb0127fc640 1 -- 172.21.15.139:0/3952324374 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01803c420 con 0x7fb01c09ad40 2024-04-25T10:03:35.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.919+0000 7fb024132640 1 -- 172.21.15.139:0/3952324374 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c0b56f0 con 0x7fb01c09ad40 2024-04-25T10:03:35.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.920+0000 7fb024132640 1 -- 172.21.15.139:0/3952324374 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb01c0b5db0 con 0x7fb01c09ad40 2024-04-25T10:03:35.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.920+0000 7fb0127fc640 1 -- 172.21.15.139:0/3952324374 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb018039040 con 0x7fb01c09ad40 2024-04-25T10:03:35.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.921+0000 7fb0127fc640 1 --2- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb00006b500 0x7fb00006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.921+0000 7fb0127fc640 1 -- 172.21.15.139:0/3952324374 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fb0180ab780 con 0x7fb01c09ad40 2024-04-25T10:03:35.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.921+0000 7fb0216cd640 1 --2- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb00006b500 0x7fb00006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.921+0000 7fb0127fc640 1 --2- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x7fb000071ae0 0x7fb000073f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.921+0000 7fb0127fc640 1 -- 172.21.15.139:0/3952324374 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0000745f0 con 0x7fb000071ae0 2024-04-25T10:03:35.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.921+0000 7fb0127fc640 1 -- 172.21.15.139:0/3952324374 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0180b2050 con 0x7fb01c09ad40 2024-04-25T10:03:35.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.921+0000 7fb0226cf640 1 --2- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x7fb000071ae0 0x7fb000073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.921+0000 7fb0216cd640 1 --2- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb00006b500 0x7fb00006d9c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb00c006fd0 tx=0x7fb00c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.921+0000 7fb0226cf640 1 --2- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x7fb000071ae0 0x7fb000073f20 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.922+0000 7fb0127fc640 1 -- 172.21.15.139:0/3952324374 <== osd.3 v2:172.21.15.139:6824/1040931291 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb0000745f0 con 0x7fb000071ae0 2024-04-25T10:03:35.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.935+0000 7fb024132640 1 -- 172.21.15.139:0/3952324374 --> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb01c074520 con 0x7fb000071ae0 2024-04-25T10:03:35.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.935+0000 7fb0127fc640 1 -- 172.21.15.139:0/3952324374 <== osd.3 v2:172.21.15.139:6824/1040931291 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb01c074520 con 0x7fb000071ae0 2024-04-25T10:03:35.943 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:03:35.944 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:03:35.944 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:03:35.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.935+0000 7fb024132640 1 -- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x7fb000071ae0 msgr2=0x7fb000073f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.935+0000 7fb024132640 1 --2- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:6824/1040931291,v1:172.21.15.139:6825/1040931291] conn(0x7fb000071ae0 0x7fb000073f20 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.936+0000 7fb024132640 1 -- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb00006b500 msgr2=0x7fb00006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.936+0000 7fb024132640 1 --2- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb00006b500 0x7fb00006d9c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb00c006fd0 tx=0x7fb00c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:35.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.936+0000 7fb024132640 1 -- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c09ad40 msgr2=0x7fb01c159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.936+0000 7fb024132640 1 --2- 172.21.15.139:0/3952324374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb01c09ad40 0x7fb01c159960 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fb018009d90 tx=0x7fb018004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:35.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.936+0000 7fb0226cf640 1 -- 172.21.15.139:0/3952324374 reap_dead start 2024-04-25T10:03:35.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.936+0000 7fb024132640 1 -- 172.21.15.139:0/3952324374 shutdown_connections 2024-04-25T10:03:35.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.936+0000 7fb024132640 1 -- 172.21.15.139:0/3952324374 >> 172.21.15.139:0/3952324374 conn(0x7fb01c0b31f0 msgr2=0x7fb01c1601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.936+0000 7fb024132640 1 -- 172.21.15.139:0/3952324374 shutdown_connections 2024-04-25T10:03:35.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:35.936+0000 7fb024132640 1 -- 172.21.15.139:0/3952324374 wait complete. 2024-04-25T10:03:35.951 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:03:36.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.044+0000 7f39f5570640 1 Processor -- start 2024-04-25T10:03:36.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.044+0000 7f39f5570640 1 -- start start 2024-04-25T10:03:36.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.045+0000 7f39f5570640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f39f015da30 0x7f39f00704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.045+0000 7f39f5570640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39f0072d60 0x7f39f006ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.045+0000 7f39f5570640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f00731a0 con 0x7f39f006b0a0 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.045+0000 7f39f5570640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f00cd540 con 0x7f39f015da30 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.045+0000 7f39f5570640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00b3de0 con 0x7f39f0072d60 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.045+0000 7f39eeffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f39f015da30 0x7f39f00704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.045+0000 7f39eeffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f39f015da30 0x7f39f00704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42970/0 (socket says 172.21.15.139:42970) 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.045+0000 7f39eeffd640 1 -- 172.21.15.139:0/1680045597 learned_addr learned my addr 172.21.15.139:0/1680045597 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.046+0000 7f39ee7fc640 1 --2- 172.21.15.139:0/1680045597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39f0072d60 0x7f39f006ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.046+0000 7f39eeffd640 1 -- 172.21.15.139:0/1680045597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39f0072d60 msgr2=0x7f39f006ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.046+0000 7f39eeffd640 1 --2- 172.21.15.139:0/1680045597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39f0072d60 0x7f39f006ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.046+0000 7f39eeffd640 1 -- 172.21.15.139:0/1680045597 >> v1:172.21.15.150:6789/0 conn(0x7f39f006b0a0 legacy=0x7f39f00d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.046+0000 7f39eeffd640 1 -- 172.21.15.139:0/1680045597 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f015de10 con 0x7f39f015da30 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.046+0000 7f39eeffd640 1 --2- 172.21.15.139:0/1680045597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39f015da30 0x7f39f00704a0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009e50 tx=0x7f39e002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207bdf0cc6f283bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.046+0000 7f39edffb640 1 -- 172.21.15.139:0/1680045597 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0036070 con 0x7f39f015da30 2024-04-25T10:03:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.046+0000 7f39edffb640 1 -- 172.21.15.139:0/1680045597 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e0031600 con 0x7f39f015da30 2024-04-25T10:03:36.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.046+0000 7f39edffb640 1 -- 172.21.15.139:0/1680045597 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e00319d0 con 0x7f39f015da30 2024-04-25T10:03:36.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.047+0000 7f39f5570640 1 -- 172.21.15.139:0/1680045597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39f015da30 msgr2=0x7f39f00704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.047+0000 7f39f5570640 1 --2- 172.21.15.139:0/1680045597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39f015da30 0x7f39f00704a0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009e50 tx=0x7f39e002a710 comp rx=0 tx=0).stop 2024-04-25T10:03:36.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.047+0000 7f39f5570640 1 -- 172.21.15.139:0/1680045597 shutdown_connections 2024-04-25T10:03:36.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.047+0000 7f39f5570640 1 --2- 172.21.15.139:0/1680045597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39f0072d60 0x7f39f006ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.047+0000 7f39f5570640 1 --2- 172.21.15.139:0/1680045597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39f015da30 0x7f39f00704a0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.047+0000 7f39f5570640 1 -- 172.21.15.139:0/1680045597 >> 172.21.15.139:0/1680045597 conn(0x7f39f00b23f0 msgr2=0x7f39f00b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.047+0000 7f39f5570640 1 -- 172.21.15.139:0/1680045597 shutdown_connections 2024-04-25T10:03:36.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.047+0000 7f39f5570640 1 -- 172.21.15.139:0/1680045597 wait complete. 2024-04-25T10:03:36.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.048+0000 7f39f5570640 1 Processor -- start 2024-04-25T10:03:36.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.048+0000 7f39f5570640 1 -- start start 2024-04-25T10:03:36.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.049+0000 7f39f5570640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39f006b0a0 0x7f39f015c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.049+0000 7f39f5570640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39f0072d60 0x7f39f01561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.049+0000 7f39f5570640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39f015da30 0x7f39f015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.049+0000 7f39f5570640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00b42a0 con 0x7f39f006b0a0 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.049+0000 7f39f5570640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00b3b00 con 0x7f39f015da30 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.049+0000 7f39f5570640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00b4580 con 0x7f39f0072d60 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.049+0000 7f39eeffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39f006b0a0 0x7f39f015c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.049+0000 7f39eeffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39f006b0a0 0x7f39f015c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42974/0 (socket says 172.21.15.139:42974) 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.049+0000 7f39eeffd640 1 -- 172.21.15.139:0/621593698 learned_addr learned my addr 172.21.15.139:0/621593698 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.049+0000 7f39ee7fc640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39f0072d60 0x7f39f01561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.049+0000 7f39ef7fe640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39f015da30 0x7f39f015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.050+0000 7f39eeffd640 1 -- 172.21.15.139:0/621593698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39f0072d60 msgr2=0x7f39f01561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.050+0000 7f39eeffd640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39f0072d60 0x7f39f01561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.050+0000 7f39eeffd640 1 -- 172.21.15.139:0/621593698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39f015da30 msgr2=0x7f39f015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.050+0000 7f39eeffd640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39f015da30 0x7f39f015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.050+0000 7f39eeffd640 1 -- 172.21.15.139:0/621593698 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f0154750 con 0x7f39f006b0a0 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.050+0000 7f39eeffd640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39f006b0a0 0x7f39f015c4d0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f39e0034040 tx=0x7f39e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.050+0000 7f39d77fe640 1 -- 172.21.15.139:0/621593698 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0036050 con 0x7f39f006b0a0 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.050+0000 7f39d77fe640 1 -- 172.21.15.139:0/621593698 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e0042070 con 0x7f39f006b0a0 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.050+0000 7f39f5570640 1 -- 172.21.15.139:0/621593698 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f015afc0 con 0x7f39f006b0a0 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.050+0000 7f39d77fe640 1 -- 172.21.15.139:0/621593698 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0009510 con 0x7f39f006b0a0 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.050+0000 7f39f5570640 1 -- 172.21.15.139:0/621593698 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f015b580 con 0x7f39f006b0a0 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.051+0000 7f39f5570640 1 -- 172.21.15.139:0/621593698 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f0074520 con 0x7f39f006b0a0 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.051+0000 7f39d77fe640 1 -- 172.21.15.139:0/621593698 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39e0007c50 con 0x7f39f006b0a0 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.052+0000 7f39d77fe640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f39dc06b4d0 0x7f39dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.052+0000 7f39d77fe640 1 -- 172.21.15.139:0/621593698 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f39e00aaec0 con 0x7f39f006b0a0 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.052+0000 7f39ee7fc640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f39dc06b4d0 0x7f39dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.052+0000 7f39ee7fc640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f39dc06b4d0 0x7f39dc06d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f39f00d12a0 tx=0x7f39e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.056+0000 7f39d77fe640 1 -- 172.21.15.139:0/621593698 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39e00797b0 con 0x7f39f006b0a0 2024-04-25T10:03:36.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.225+0000 7f39f5570640 1 -- 172.21.15.139:0/621593698 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f39f006f010 con 0x7f39dc06b4d0 2024-04-25T10:03:36.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.225+0000 7f39d77fe640 1 -- 172.21.15.139:0/621593698 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f39f006f010 con 0x7f39dc06b4d0 2024-04-25T10:03:36.230 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:03:36.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.227+0000 7f39f5570640 1 -- 172.21.15.139:0/621593698 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f39dc06b4d0 msgr2=0x7f39dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.227+0000 7f39f5570640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f39dc06b4d0 0x7f39dc06d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f39f00d12a0 tx=0x7f39e4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:36.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.227+0000 7f39f5570640 1 -- 172.21.15.139:0/621593698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39f006b0a0 msgr2=0x7f39f015c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.227+0000 7f39f5570640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39f006b0a0 0x7f39f015c4d0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f39e0034040 tx=0x7f39e0004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:36.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.227+0000 7f39f5570640 1 -- 172.21.15.139:0/621593698 shutdown_connections 2024-04-25T10:03:36.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.227+0000 7f39f5570640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f39dc06b4d0 0x7f39dc06d990 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.227+0000 7f39f5570640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39f015da30 0x7f39f015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.227+0000 7f39f5570640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39f0072d60 0x7f39f01561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.227+0000 7f39f5570640 1 --2- 172.21.15.139:0/621593698 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39f006b0a0 0x7f39f015c4d0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.227+0000 7f39f5570640 1 -- 172.21.15.139:0/621593698 >> 172.21.15.139:0/621593698 conn(0x7f39f00b23f0 msgr2=0x7f39f00cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.227+0000 7f39f5570640 1 -- 172.21.15.139:0/621593698 shutdown_connections 2024-04-25T10:03:36.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.227+0000 7f39f5570640 1 -- 172.21.15.139:0/621593698 wait complete. 2024-04-25T10:03:36.244 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:03:36.244 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:03:36.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.340+0000 7f83d7d3b640 1 Processor -- start 2024-04-25T10:03:36.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.340+0000 7f83d7d3b640 1 -- start start 2024-04-25T10:03:36.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.341+0000 7f83d7d3b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d0144090 0x7f83d00b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.341+0000 7f83d7d3b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f83d00cc9e0 0x7f83d006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.341+0000 7f83d7d3b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d00b2a50 con 0x7f83d00cce20 2024-04-25T10:03:36.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.341+0000 7f83d7d3b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d015b090 con 0x7f83d00cc9e0 2024-04-25T10:03:36.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.341+0000 7f83d7d3b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015b370 con 0x7f83d0144090 2024-04-25T10:03:36.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.341+0000 7f83d52d6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f83d00cc9e0 0x7f83d006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.341+0000 7f83d62d8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f83d00cce20 0x7f83d015e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:34630/0 (socket says 172.21.15.139:34630) 2024-04-25T10:03:36.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.341+0000 7f83d62d8640 1 -- 172.21.15.139:0/1594471895 learned_addr learned my addr 172.21.15.139:0/1594471895 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:36.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.342+0000 7f83d5ad7640 1 --2- 172.21.15.139:0/1594471895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d0144090 0x7f83d00b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.342+0000 7f83d52d6640 1 -- 172.21.15.139:0/1594471895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d0144090 msgr2=0x7f83d00b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.342+0000 7f83d52d6640 1 --2- 172.21.15.139:0/1594471895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d0144090 0x7f83d00b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.342+0000 7f83d52d6640 1 -- 172.21.15.139:0/1594471895 >> v1:172.21.15.139:6789/0 conn(0x7f83d00cce20 legacy=0x7f83d015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.342+0000 7f83d52d6640 1 -- 172.21.15.139:0/1594471895 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d015b650 con 0x7f83d00cc9e0 2024-04-25T10:03:36.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.342+0000 7f83d4ad5640 1 -- 172.21.15.139:0/1594471895 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764393705 0 0) 0x7f83c0001110 con 0x7f83d00cce20 2024-04-25T10:03:36.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.342+0000 7f83d52d6640 1 --2- 172.21.15.139:0/1594471895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00cc9e0 0x7f83d006e330 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f83c40097a0 tx=0x7f83c402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb970292600be33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.343+0000 7f83d4ad5640 1 -- 172.21.15.139:0/1594471895 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4037070 con 0x7f83d00cc9e0 2024-04-25T10:03:36.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.343+0000 7f83d4ad5640 1 -- 172.21.15.139:0/1594471895 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c402ae50 con 0x7f83d00cc9e0 2024-04-25T10:03:36.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.343+0000 7f83d4ad5640 1 -- 172.21.15.139:0/1594471895 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4032660 con 0x7f83d00cc9e0 2024-04-25T10:03:36.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.343+0000 7f83d7d3b640 1 -- 172.21.15.139:0/1594471895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00cc9e0 msgr2=0x7f83d006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.343+0000 7f83d7d3b640 1 --2- 172.21.15.139:0/1594471895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00cc9e0 0x7f83d006e330 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f83c40097a0 tx=0x7f83c402a100 comp rx=0 tx=0).stop 2024-04-25T10:03:36.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.343+0000 7f83d7d3b640 1 -- 172.21.15.139:0/1594471895 shutdown_connections 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.343+0000 7f83d7d3b640 1 --2- 172.21.15.139:0/1594471895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00cc9e0 0x7f83d006e330 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.343+0000 7f83d7d3b640 1 --2- 172.21.15.139:0/1594471895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d0144090 0x7f83d00b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.343+0000 7f83d7d3b640 1 -- 172.21.15.139:0/1594471895 >> 172.21.15.139:0/1594471895 conn(0x7f83d0067000 msgr2=0x7f83d006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.343+0000 7f83d7d3b640 1 -- 172.21.15.139:0/1594471895 shutdown_connections 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.343+0000 7f83d7d3b640 1 -- 172.21.15.139:0/1594471895 wait complete. 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.344+0000 7f83d7d3b640 1 Processor -- start 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.344+0000 7f83d7d3b640 1 -- start start 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d7d3b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d00cc9e0 0x7f83d015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d7d3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00cce20 0x7f83d012b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d7d3b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83d0144090 0x7f83d012bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d7d3b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d0144470 con 0x7f83d00cce20 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d7d3b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015bc10 con 0x7f83d0144090 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d7d3b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015ad80 con 0x7f83d00cc9e0 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d52d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00cce20 0x7f83d012b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d52d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00cce20 0x7f83d012b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43002/0 (socket says 172.21.15.139:43002) 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d52d6640 1 -- 172.21.15.139:0/368841132 learned_addr learned my addr 172.21.15.139:0/368841132 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d5ad7640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d00cc9e0 0x7f83d015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d62d8640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83d0144090 0x7f83d012bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d52d6640 1 -- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d00cc9e0 msgr2=0x7f83d015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d52d6640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d00cc9e0 0x7f83d015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d52d6640 1 -- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83d0144090 msgr2=0x7f83d012bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d52d6640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83d0144090 0x7f83d012bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d52d6640 1 -- 172.21.15.139:0/368841132 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d00ae5f0 con 0x7f83d00cce20 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d5ad7640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83d00cc9e0 0x7f83d015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.345+0000 7f83d62d8640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83d0144090 0x7f83d012bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.346+0000 7f83d52d6640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00cce20 0x7f83d012b7a0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f83c4035040 tx=0x7f83c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.346+0000 7f83ce7fc640 1 -- 172.21.15.139:0/368841132 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4041070 con 0x7f83d00cce20 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.346+0000 7f83ce7fc640 1 -- 172.21.15.139:0/368841132 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c403c670 con 0x7f83d00cce20 2024-04-25T10:03:36.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.346+0000 7f83ce7fc640 1 -- 172.21.15.139:0/368841132 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4037040 con 0x7f83d00cce20 2024-04-25T10:03:36.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.346+0000 7f83d7d3b640 1 -- 172.21.15.139:0/368841132 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d015c930 con 0x7f83d00cce20 2024-04-25T10:03:36.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.346+0000 7f83d7d3b640 1 -- 172.21.15.139:0/368841132 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d015cef0 con 0x7f83d00cce20 2024-04-25T10:03:36.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.347+0000 7f83d7d3b640 1 -- 172.21.15.139:0/368841132 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83d0074520 con 0x7f83d00cce20 2024-04-25T10:03:36.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.347+0000 7f83ce7fc640 1 -- 172.21.15.139:0/368841132 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83c403a070 con 0x7f83d00cce20 2024-04-25T10:03:36.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.348+0000 7f83ce7fc640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f83bc06b4d0 0x7f83bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.348+0000 7f83ce7fc640 1 -- 172.21.15.139:0/368841132 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f83c40aa3d0 con 0x7f83d00cce20 2024-04-25T10:03:36.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.348+0000 7f83d5ad7640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f83bc06b4d0 0x7f83bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.348+0000 7f83ce7fc640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x7f83bc071ab0 0x7f83bc073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.348+0000 7f83ce7fc640 1 -- 172.21.15.139:0/368841132 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83c4008eb0 con 0x7f83bc071ab0 2024-04-25T10:03:36.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.348+0000 7f83ce7fc640 1 -- 172.21.15.139:0/368841132 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83c40b1050 con 0x7f83d00cce20 2024-04-25T10:03:36.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.348+0000 7f83d62d8640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x7f83bc071ab0 0x7f83bc073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.348+0000 7f83d5ad7640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f83bc06b4d0 0x7f83bc06d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009f50 tx=0x7f83c800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.348+0000 7f83d62d8640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x7f83bc071ab0 0x7f83bc073ef0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.349+0000 7f83ce7fc640 1 -- 172.21.15.139:0/368841132 <== osd.4 v2:172.21.15.150:6800/2810531857 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f83c4008eb0 con 0x7f83bc071ab0 2024-04-25T10:03:36.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.362+0000 7f83d7d3b640 1 -- 172.21.15.139:0/368841132 --> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f83d0141320 con 0x7f83bc071ab0 2024-04-25T10:03:36.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.363+0000 7f83ce7fc640 1 -- 172.21.15.139:0/368841132 <== osd.4 v2:172.21.15.150:6800/2810531857 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f83d0141320 con 0x7f83bc071ab0 2024-04-25T10:03:36.368 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:03:36.368 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:03:36.368 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:03:36.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.363+0000 7f83d7d3b640 1 -- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x7f83bc071ab0 msgr2=0x7f83bc073ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.363+0000 7f83d7d3b640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.150:6800/2810531857,v1:172.21.15.150:6801/2810531857] conn(0x7f83bc071ab0 0x7f83bc073ef0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.363+0000 7f83d7d3b640 1 -- 172.21.15.139:0/368841132 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f83bc06b4d0 msgr2=0x7f83bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.363+0000 7f83d7d3b640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f83bc06b4d0 0x7f83bc06d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009f50 tx=0x7f83c800b040 comp rx=0 tx=0).stop 2024-04-25T10:03:36.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.363+0000 7f83d7d3b640 1 -- 172.21.15.139:0/368841132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00cce20 msgr2=0x7f83d012b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.363+0000 7f83d7d3b640 1 --2- 172.21.15.139:0/368841132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83d00cce20 0x7f83d012b7a0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f83c4035040 tx=0x7f83c40046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.363+0000 7f83d62d8640 1 -- 172.21.15.139:0/368841132 reap_dead start 2024-04-25T10:03:36.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.363+0000 7f83d7d3b640 1 -- 172.21.15.139:0/368841132 shutdown_connections 2024-04-25T10:03:36.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.363+0000 7f83d7d3b640 1 -- 172.21.15.139:0/368841132 >> 172.21.15.139:0/368841132 conn(0x7f83d0067000 msgr2=0x7f83d00af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.363+0000 7f83d7d3b640 1 -- 172.21.15.139:0/368841132 shutdown_connections 2024-04-25T10:03:36.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.363+0000 7f83d7d3b640 1 -- 172.21.15.139:0/368841132 wait complete. 2024-04-25T10:03:36.378 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:03:36.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.472+0000 7fbf6a9b0640 1 Processor -- start 2024-04-25T10:03:36.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.472+0000 7fbf6a9b0640 1 -- start start 2024-04-25T10:03:36.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf6a9b0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbf641617e0 0x7fbf640ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf6a9b0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbf640cec80 0x7fbf640d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf6a9b0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf64161dc0 con 0x7fbf6415dbf0 2024-04-25T10:03:36.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf6a9b0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf640cf0d0 con 0x7fbf640cec80 2024-04-25T10:03:36.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf6a9b0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf640d1ee0 con 0x7fbf641617e0 2024-04-25T10:03:36.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf63fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbf6415dbf0 0x7fbf640b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42760/0 (socket says 172.21.15.139:42760) 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf68f4d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbf640cec80 0x7fbf640d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf63fff640 1 -- 172.21.15.139:0/3968464776 learned_addr learned my addr 172.21.15.139:0/3968464776 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf637fe640 1 --2- 172.21.15.139:0/3968464776 >> v2:172.21.15.150:3300/0 conn(0x7fbf641617e0 0x7fbf640ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf68f4d640 1 -- 172.21.15.139:0/3968464776 >> v2:172.21.15.150:3300/0 conn(0x7fbf641617e0 msgr2=0x7fbf640ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf68f4d640 1 --2- 172.21.15.139:0/3968464776 >> v2:172.21.15.150:3300/0 conn(0x7fbf641617e0 0x7fbf640ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf68f4d640 1 -- 172.21.15.139:0/3968464776 >> v1:172.21.15.139:6789/0 conn(0x7fbf6415dbf0 legacy=0x7fbf640b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf68f4d640 1 -- 172.21.15.139:0/3968464776 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf641620a0 con 0x7fbf640cec80 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.473+0000 7fbf62ffd640 1 -- 172.21.15.139:0/3968464776 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929919122 0 0) 0x7fbf64161dc0 con 0x7fbf6415dbf0 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.474+0000 7fbf68f4d640 1 --2- 172.21.15.139:0/3968464776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf640cec80 0x7fbf640d2370 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fbf6408c420 tx=0x7fbf5c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daedb0faea490a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.474+0000 7fbf62ffd640 1 -- 172.21.15.139:0/3968464776 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c007740 con 0x7fbf640cec80 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.474+0000 7fbf62ffd640 1 -- 172.21.15.139:0/3968464776 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c0078e0 con 0x7fbf640cec80 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.474+0000 7fbf62ffd640 1 -- 172.21.15.139:0/3968464776 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c007cb0 con 0x7fbf640cec80 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.474+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/3968464776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf640cec80 msgr2=0x7fbf640d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.474+0000 7fbf6a9b0640 1 --2- 172.21.15.139:0/3968464776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf640cec80 0x7fbf640d2370 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fbf6408c420 tx=0x7fbf5c02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.474+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/3968464776 shutdown_connections 2024-04-25T10:03:36.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.474+0000 7fbf6a9b0640 1 --2- 172.21.15.139:0/3968464776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf640cec80 0x7fbf640d2370 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.474+0000 7fbf6a9b0640 1 --2- 172.21.15.139:0/3968464776 >> v2:172.21.15.150:3300/0 conn(0x7fbf641617e0 0x7fbf640ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.475+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/3968464776 >> 172.21.15.139:0/3968464776 conn(0x7fbf640b5480 msgr2=0x7fbf640b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.475+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/3968464776 shutdown_connections 2024-04-25T10:03:36.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.475+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/3968464776 wait complete. 2024-04-25T10:03:36.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.476+0000 7fbf6a9b0640 1 Processor -- start 2024-04-25T10:03:36.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.476+0000 7fbf6a9b0640 1 -- start start 2024-04-25T10:03:36.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf6a9b0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf640cec80 0x7fbf6414dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf6a9b0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf6415dbf0 0x7fbf64150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf6a9b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf641617e0 0x7fbf641506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf6a9b0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6415dfd0 con 0x7fbf640cec80 2024-04-25T10:03:36.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf6a9b0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf64162660 con 0x7fbf6415dbf0 2024-04-25T10:03:36.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf6a9b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf641518f0 con 0x7fbf641617e0 2024-04-25T10:03:36.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf63fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf640cec80 0x7fbf6414dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf63fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf640cec80 0x7fbf6414dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49016/0 (socket says 172.21.15.139:49016) 2024-04-25T10:03:36.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf63fff640 1 -- 172.21.15.139:0/2046008924 learned_addr learned my addr 172.21.15.139:0/2046008924 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:36.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf637fe640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf6415dbf0 0x7fbf64150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf68f4d640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf641617e0 0x7fbf641506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf63fff640 1 -- 172.21.15.139:0/2046008924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf641617e0 msgr2=0x7fbf641506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf63fff640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf641617e0 0x7fbf641506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf63fff640 1 -- 172.21.15.139:0/2046008924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf6415dbf0 msgr2=0x7fbf64150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf63fff640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf6415dbf0 0x7fbf64150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf63fff640 1 -- 172.21.15.139:0/2046008924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf64150ce0 con 0x7fbf640cec80 2024-04-25T10:03:36.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf68f4d640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf641617e0 0x7fbf641506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:36.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf63fff640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf640cec80 0x7fbf6414dd90 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400c4d0 tx=0x7fbf5400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf60ff9640 1 -- 172.21.15.139:0/2046008924 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54018070 con 0x7fbf640cec80 2024-04-25T10:03:36.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf60ff9640 1 -- 172.21.15.139:0/2046008924 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf54007500 con 0x7fbf640cec80 2024-04-25T10:03:36.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf60ff9640 1 -- 172.21.15.139:0/2046008924 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54004a20 con 0x7fbf640cec80 2024-04-25T10:03:36.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/2046008924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf64150fc0 con 0x7fbf640cec80 2024-04-25T10:03:36.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.477+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/2046008924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf64151580 con 0x7fbf640cec80 2024-04-25T10:03:36.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.478+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/2046008924 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf641611e0 con 0x7fbf640cec80 2024-04-25T10:03:36.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.479+0000 7fbf60ff9640 1 -- 172.21.15.139:0/2046008924 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf5401b070 con 0x7fbf640cec80 2024-04-25T10:03:36.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.479+0000 7fbf60ff9640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf5006b4d0 0x7fbf5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.479+0000 7fbf60ff9640 1 -- 172.21.15.139:0/2046008924 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fbf54089390 con 0x7fbf640cec80 2024-04-25T10:03:36.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.479+0000 7fbf637fe640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf5006b4d0 0x7fbf5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.480+0000 7fbf637fe640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf5006b4d0 0x7fbf5006d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbf640b8c50 tx=0x7fbf58009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.483+0000 7fbf60ff9640 1 -- 172.21.15.139:0/2046008924 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf54057c80 con 0x7fbf640cec80 2024-04-25T10:03:36.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.654+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/2046008924 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf64074520 con 0x7fbf5006b4d0 2024-04-25T10:03:36.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.654+0000 7fbf60ff9640 1 -- 172.21.15.139:0/2046008924 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbf64074520 con 0x7fbf5006b4d0 2024-04-25T10:03:36.659 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:03:36.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.656+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/2046008924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf5006b4d0 msgr2=0x7fbf5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.656+0000 7fbf6a9b0640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf5006b4d0 0x7fbf5006d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbf640b8c50 tx=0x7fbf58009190 comp rx=0 tx=0).stop 2024-04-25T10:03:36.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.656+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/2046008924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf640cec80 msgr2=0x7fbf6414dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.656+0000 7fbf6a9b0640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf640cec80 0x7fbf6414dd90 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400c4d0 tx=0x7fbf5400c990 comp rx=0 tx=0).stop 2024-04-25T10:03:36.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.656+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/2046008924 shutdown_connections 2024-04-25T10:03:36.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.656+0000 7fbf6a9b0640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf5006b4d0 0x7fbf5006d990 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.656+0000 7fbf6a9b0640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf641617e0 0x7fbf641506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.656+0000 7fbf6a9b0640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf6415dbf0 0x7fbf64150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.656+0000 7fbf6a9b0640 1 --2- 172.21.15.139:0/2046008924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf640cec80 0x7fbf6414dd90 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.656+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/2046008924 >> 172.21.15.139:0/2046008924 conn(0x7fbf640b5480 msgr2=0x7fbf640b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.656+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/2046008924 shutdown_connections 2024-04-25T10:03:36.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.656+0000 7fbf6a9b0640 1 -- 172.21.15.139:0/2046008924 wait complete. 2024-04-25T10:03:36.672 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:03:36.672 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:03:36.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.767+0000 7fb9bde15640 1 Processor -- start 2024-04-25T10:03:36.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.767+0000 7fb9bde15640 1 -- start start 2024-04-25T10:03:36.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.768+0000 7fb9bde15640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb9b806ebf0 0x7fb9b806f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.768+0000 7fb9bde15640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9b80b2db0 0x7fb9b80b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.768+0000 7fb9bde15640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b80cf940 con 0x7fb9b815dd90 2024-04-25T10:03:36.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.768+0000 7fb9bde15640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b806ab60 con 0x7fb9b806ebf0 2024-04-25T10:03:36.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.768+0000 7fb9bde15640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b806adb0 con 0x7fb9b80b2db0 2024-04-25T10:03:36.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.768+0000 7fb9b6ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb9b806ebf0 0x7fb9b806f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.768+0000 7fb9b77fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb9b815dd90 0x7fb9b80ceea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42764/0 (socket says 172.21.15.139:42764) 2024-04-25T10:03:36.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.768+0000 7fb9b6ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb9b806ebf0 0x7fb9b806f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49024/0 (socket says 172.21.15.139:49024) 2024-04-25T10:03:36.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.768+0000 7fb9b77fe640 1 -- 172.21.15.139:0/4041084352 learned_addr learned my addr 172.21.15.139:0/4041084352 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:36.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.769+0000 7fb9b7fff640 1 --2- 172.21.15.139:0/4041084352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9b80b2db0 0x7fb9b80b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.769+0000 7fb9b6ffd640 1 -- 172.21.15.139:0/4041084352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9b80b2db0 msgr2=0x7fb9b80b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.769+0000 7fb9b6ffd640 1 --2- 172.21.15.139:0/4041084352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9b80b2db0 0x7fb9b80b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.769+0000 7fb9b6ffd640 1 -- 172.21.15.139:0/4041084352 >> v1:172.21.15.139:6789/0 conn(0x7fb9b815dd90 legacy=0x7fb9b80ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.769+0000 7fb9b6ffd640 1 -- 172.21.15.139:0/4041084352 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b806b090 con 0x7fb9b806ebf0 2024-04-25T10:03:36.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.769+0000 7fb9b67fc640 1 -- 172.21.15.139:0/4041084352 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966896638 0 0) 0x7fb9b80cf940 con 0x7fb9b815dd90 2024-04-25T10:03:36.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.769+0000 7fb9b7fff640 1 --2- 172.21.15.139:0/4041084352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9b80b2db0 0x7fb9b80b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:36.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.769+0000 7fb9b6ffd640 1 --2- 172.21.15.139:0/4041084352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9b806ebf0 0x7fb9b806f060 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0097a0 tx=0x7fb9ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d795a07fe2bd07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.769+0000 7fb9b67fc640 1 -- 172.21.15.139:0/4041084352 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac037070 con 0x7fb9b806ebf0 2024-04-25T10:03:36.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.769+0000 7fb9b67fc640 1 -- 172.21.15.139:0/4041084352 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac032440 con 0x7fb9b806ebf0 2024-04-25T10:03:36.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.769+0000 7fb9b67fc640 1 -- 172.21.15.139:0/4041084352 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac032810 con 0x7fb9b806ebf0 2024-04-25T10:03:36.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.770+0000 7fb9bde15640 1 -- 172.21.15.139:0/4041084352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9b806ebf0 msgr2=0x7fb9b806f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.770+0000 7fb9bde15640 1 --2- 172.21.15.139:0/4041084352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9b806ebf0 0x7fb9b806f060 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0097a0 tx=0x7fb9ac02a180 comp rx=0 tx=0).stop 2024-04-25T10:03:36.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.770+0000 7fb9bde15640 1 -- 172.21.15.139:0/4041084352 shutdown_connections 2024-04-25T10:03:36.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.770+0000 7fb9bde15640 1 --2- 172.21.15.139:0/4041084352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9b80b2db0 0x7fb9b80b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.770+0000 7fb9bde15640 1 --2- 172.21.15.139:0/4041084352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9b806ebf0 0x7fb9b806f060 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.770+0000 7fb9bde15640 1 -- 172.21.15.139:0/4041084352 >> 172.21.15.139:0/4041084352 conn(0x7fb9b80cfd80 msgr2=0x7fb9b80d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.770+0000 7fb9bde15640 1 -- 172.21.15.139:0/4041084352 shutdown_connections 2024-04-25T10:03:36.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.770+0000 7fb9bde15640 1 -- 172.21.15.139:0/4041084352 wait complete. 2024-04-25T10:03:36.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.771+0000 7fb9bde15640 1 Processor -- start 2024-04-25T10:03:36.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.771+0000 7fb9bde15640 1 -- start start 2024-04-25T10:03:36.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9bde15640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9b806ebf0 0x7fb9b80b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9bde15640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9b80b2db0 0x7fb9b81f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9bde15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9b815dd90 0x7fb9b8159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9bde15640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b8072620 con 0x7fb9b815dd90 2024-04-25T10:03:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9bde15640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b806f5a0 con 0x7fb9b806ebf0 2024-04-25T10:03:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9bde15640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b80cf5b0 con 0x7fb9b80b2db0 2024-04-25T10:03:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9b7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9b815dd90 0x7fb9b8159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9b7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9b815dd90 0x7fb9b8159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49040/0 (socket says 172.21.15.139:49040) 2024-04-25T10:03:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9b7fff640 1 -- 172.21.15.139:0/1595587228 learned_addr learned my addr 172.21.15.139:0/1595587228 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9b77fe640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9b806ebf0 0x7fb9b80b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9b6ffd640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9b80b2db0 0x7fb9b81f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9b7fff640 1 -- 172.21.15.139:0/1595587228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9b80b2db0 msgr2=0x7fb9b81f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9b7fff640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9b80b2db0 0x7fb9b81f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9b7fff640 1 -- 172.21.15.139:0/1595587228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9b806ebf0 msgr2=0x7fb9b80b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9b7fff640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9b806ebf0 0x7fb9b80b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9b7fff640 1 -- 172.21.15.139:0/1595587228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b81f4900 con 0x7fb9b815dd90 2024-04-25T10:03:36.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.772+0000 7fb9b6ffd640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9b80b2db0 0x7fb9b81f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:36.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.773+0000 7fb9b7fff640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9b815dd90 0x7fb9b8159110 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000ce70 tx=0x7fb9b000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.773+0000 7fb99ffff640 1 -- 172.21.15.139:0/1595587228 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b001a070 con 0x7fb9b815dd90 2024-04-25T10:03:36.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.773+0000 7fb99ffff640 1 -- 172.21.15.139:0/1595587228 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b000d840 con 0x7fb9b815dd90 2024-04-25T10:03:36.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.773+0000 7fb9bde15640 1 -- 172.21.15.139:0/1595587228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b80b0ab0 con 0x7fb9b815dd90 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.773+0000 7fb99ffff640 1 -- 172.21.15.139:0/1595587228 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b000dc30 con 0x7fb9b815dd90 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.773+0000 7fb9bde15640 1 -- 172.21.15.139:0/1595587228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b81f3bc0 con 0x7fb9b815dd90 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.774+0000 7fb9bde15640 1 -- 172.21.15.139:0/1595587228 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9b8074520 con 0x7fb9b815dd90 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.774+0000 7fb99ffff640 1 -- 172.21.15.139:0/1595587228 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9b001d070 con 0x7fb9b815dd90 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.774+0000 7fb99ffff640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb9a406b4d0 0x7fb9a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.775+0000 7fb99ffff640 1 -- 172.21.15.139:0/1595587228 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fb9b008a8c0 con 0x7fb9b815dd90 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.775+0000 7fb9b77fe640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb9a406b4d0 0x7fb9a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.775+0000 7fb99ffff640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x7fb9a4071ab0 0x7fb9a4073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.775+0000 7fb99ffff640 1 -- 172.21.15.139:0/1595587228 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9a40745c0 con 0x7fb9a4071ab0 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.775+0000 7fb99ffff640 1 -- 172.21.15.139:0/1595587228 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9b008f2c0 con 0x7fb9b815dd90 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.775+0000 7fb9b77fe640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb9a406b4d0 0x7fb9a406d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8000f60 tx=0x7fb9a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.775+0000 7fb9b6ffd640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x7fb9a4071ab0 0x7fb9a4073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.775+0000 7fb9b6ffd640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x7fb9a4071ab0 0x7fb9a4073ef0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.776+0000 7fb99ffff640 1 -- 172.21.15.139:0/1595587228 <== osd.5 v2:172.21.15.150:6824/2184499879 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb9a40745c0 con 0x7fb9a4071ab0 2024-04-25T10:03:36.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.789+0000 7fb9bde15640 1 -- 172.21.15.139:0/1595587228 --> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb9b81f36b0 con 0x7fb9a4071ab0 2024-04-25T10:03:36.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.790+0000 7fb99ffff640 1 -- 172.21.15.139:0/1595587228 <== osd.5 v2:172.21.15.150:6824/2184499879 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb9b81f36b0 con 0x7fb9a4071ab0 2024-04-25T10:03:36.795 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:03:36.795 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:03:36.795 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:03:36.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.790+0000 7fb9bde15640 1 -- 172.21.15.139:0/1595587228 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x7fb9a4071ab0 msgr2=0x7fb9a4073ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.790+0000 7fb9bde15640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.150:6824/2184499879,v1:172.21.15.150:6825/2184499879] conn(0x7fb9a4071ab0 0x7fb9a4073ef0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.791+0000 7fb9bde15640 1 -- 172.21.15.139:0/1595587228 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb9a406b4d0 msgr2=0x7fb9a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.791+0000 7fb9bde15640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fb9a406b4d0 0x7fb9a406d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8000f60 tx=0x7fb9a8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:36.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.791+0000 7fb9bde15640 1 -- 172.21.15.139:0/1595587228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9b815dd90 msgr2=0x7fb9b8159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.791+0000 7fb9bde15640 1 --2- 172.21.15.139:0/1595587228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9b815dd90 0x7fb9b8159110 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000ce70 tx=0x7fb9b000a710 comp rx=0 tx=0).stop 2024-04-25T10:03:36.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.791+0000 7fb9b7fff640 1 -- 172.21.15.139:0/1595587228 reap_dead start 2024-04-25T10:03:36.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.791+0000 7fb9bde15640 1 -- 172.21.15.139:0/1595587228 shutdown_connections 2024-04-25T10:03:36.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.791+0000 7fb9bde15640 1 -- 172.21.15.139:0/1595587228 >> 172.21.15.139:0/1595587228 conn(0x7fb9b80cfd80 msgr2=0x7fb9b80cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.791+0000 7fb9bde15640 1 -- 172.21.15.139:0/1595587228 shutdown_connections 2024-04-25T10:03:36.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.791+0000 7fb9bde15640 1 -- 172.21.15.139:0/1595587228 wait complete. 2024-04-25T10:03:36.806 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:03:36.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.899+0000 7fe01c922640 1 Processor -- start 2024-04-25T10:03:36.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.899+0000 7fe01c922640 1 -- start start 2024-04-25T10:03:36.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.900+0000 7fe01c922640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe0180af8a0 0x7fe0180b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.900+0000 7fe01c922640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0180c6cc0 con 0x7fe0180afc80 2024-04-25T10:03:36.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.900+0000 7fe01c922640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0180c6fa0 con 0x7fe01806b700 2024-04-25T10:03:36.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.900+0000 7fe01c922640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0180c7280 con 0x7fe0180af8a0 2024-04-25T10:03:36.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.900+0000 7fe01659c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe0180af8a0 0x7fe0180b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.900+0000 7fe00ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe0180afc80 0x7fe0180cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42780/0 (socket says 172.21.15.139:42780) 2024-04-25T10:03:36.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.900+0000 7fe00ffff640 1 -- 172.21.15.139:0/1330894895 learned_addr learned my addr 172.21.15.139:0/1330894895 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:36.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.900+0000 7fe01659c640 1 -- 172.21.15.139:0/1330894895 >> v1:172.21.15.150:6789/0 conn(0x7fe01806b700 legacy=0x7fe01806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.900+0000 7fe01659c640 1 -- 172.21.15.139:0/1330894895 >> v1:172.21.15.139:6789/0 conn(0x7fe0180afc80 legacy=0x7fe0180cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.900+0000 7fe01659c640 1 -- 172.21.15.139:0/1330894895 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0180cd090 con 0x7fe0180af8a0 2024-04-25T10:03:36.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.901+0000 7fe01659c640 1 --2- 172.21.15.139:0/1330894895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0180af8a0 0x7fe0180b2d50 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fe0080097a0 tx=0x7fe00802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31301795d420c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.901+0000 7fe015d9b640 1 -- 172.21.15.139:0/1330894895 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008037070 con 0x7fe0180af8a0 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.901+0000 7fe015d9b640 1 -- 172.21.15.139:0/1330894895 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00802ae50 con 0x7fe0180af8a0 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.901+0000 7fe015d9b640 1 -- 172.21.15.139:0/1330894895 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008032660 con 0x7fe0180af8a0 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.902+0000 7fe01c922640 1 -- 172.21.15.139:0/1330894895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0180af8a0 msgr2=0x7fe0180b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.902+0000 7fe01c922640 1 --2- 172.21.15.139:0/1330894895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0180af8a0 0x7fe0180b2d50 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fe0080097a0 tx=0x7fe00802a140 comp rx=0 tx=0).stop 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.902+0000 7fe01c922640 1 -- 172.21.15.139:0/1330894895 shutdown_connections 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.902+0000 7fe01c922640 1 --2- 172.21.15.139:0/1330894895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0180af8a0 0x7fe0180b2d50 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.902+0000 7fe01c922640 1 -- 172.21.15.139:0/1330894895 >> 172.21.15.139:0/1330894895 conn(0x7fe0180b2230 msgr2=0x7fe0180b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.902+0000 7fe01c922640 1 -- 172.21.15.139:0/1330894895 shutdown_connections 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.902+0000 7fe01c922640 1 -- 172.21.15.139:0/1330894895 wait complete. 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.903+0000 7fe01c922640 1 Processor -- start 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.903+0000 7fe01c922640 1 -- start start 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe01c922640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01806b700 0x7fe01814d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe01c922640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0180af8a0 0x7fe0180b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe01c922640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0180afc80 0x7fe01809d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe01c922640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01806b3f0 con 0x7fe0180af8a0 2024-04-25T10:03:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe01c922640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01806b270 con 0x7fe01806b700 2024-04-25T10:03:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe01c922640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180c54a0 con 0x7fe0180afc80 2024-04-25T10:03:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe00ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0180af8a0 0x7fe0180b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe00ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0180af8a0 0x7fe0180b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49050/0 (socket says 172.21.15.139:49050) 2024-04-25T10:03:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe00ffff640 1 -- 172.21.15.139:0/3228957422 learned_addr learned my addr 172.21.15.139:0/3228957422 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe01659c640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01806b700 0x7fe01814d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe016d9d640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0180afc80 0x7fe01809d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe00ffff640 1 -- 172.21.15.139:0/3228957422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0180afc80 msgr2=0x7fe01809d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe00ffff640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0180afc80 0x7fe01809d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe00ffff640 1 -- 172.21.15.139:0/3228957422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01806b700 msgr2=0x7fe01814d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe00ffff640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01806b700 0x7fe01814d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe00ffff640 1 -- 172.21.15.139:0/3228957422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0180b5600 con 0x7fe0180af8a0 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe01659c640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01806b700 0x7fe01814d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe00ffff640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0180af8a0 0x7fe0180b4cf0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fe00000c4b0 tx=0x7fe00000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe00effd640 1 -- 172.21.15.139:0/3228957422 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000017070 con 0x7fe0180af8a0 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe00effd640 1 -- 172.21.15.139:0/3228957422 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe000005ce0 con 0x7fe0180af8a0 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe01c922640 1 -- 172.21.15.139:0/3228957422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0180b58e0 con 0x7fe0180af8a0 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe00effd640 1 -- 172.21.15.139:0/3228957422 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000007270 con 0x7fe0180af8a0 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.904+0000 7fe01c922640 1 -- 172.21.15.139:0/3228957422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01814de80 con 0x7fe0180af8a0 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.905+0000 7fe01c922640 1 -- 172.21.15.139:0/3228957422 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe018074520 con 0x7fe0180af8a0 2024-04-25T10:03:36.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.906+0000 7fe00effd640 1 -- 172.21.15.139:0/3228957422 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe00001a070 con 0x7fe0180af8a0 2024-04-25T10:03:36.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.906+0000 7fe00effd640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdff806b4d0 0x7fdff806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.906+0000 7fe00effd640 1 -- 172.21.15.139:0/3228957422 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe0000892d0 con 0x7fe0180af8a0 2024-04-25T10:03:36.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.906+0000 7fe01659c640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdff806b4d0 0x7fdff806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.907+0000 7fe01659c640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdff806b4d0 0x7fdff806d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe008034000 tx=0x7fe00802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:36.910+0000 7fe00effd640 1 -- 172.21.15.139:0/3228957422 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe000057c50 con 0x7fe0180af8a0 2024-04-25T10:03:37.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.081+0000 7fe01c922640 1 -- 172.21.15.139:0/3228957422 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0180c6840 con 0x7fdff806b4d0 2024-04-25T10:03:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.082+0000 7fe00effd640 1 -- 172.21.15.139:0/3228957422 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe0180c6840 con 0x7fdff806b4d0 2024-04-25T10:03:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:03:37.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.083+0000 7fe01c922640 1 -- 172.21.15.139:0/3228957422 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdff806b4d0 msgr2=0x7fdff806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.083+0000 7fe01c922640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdff806b4d0 0x7fdff806d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe008034000 tx=0x7fe00802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:37.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.083+0000 7fe01c922640 1 -- 172.21.15.139:0/3228957422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0180af8a0 msgr2=0x7fe0180b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.083+0000 7fe01c922640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0180af8a0 0x7fe0180b4cf0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fe00000c4b0 tx=0x7fe00000c970 comp rx=0 tx=0).stop 2024-04-25T10:03:37.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.084+0000 7fe01c922640 1 -- 172.21.15.139:0/3228957422 shutdown_connections 2024-04-25T10:03:37.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.084+0000 7fe01c922640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0180af8a0 0x7fe0180b4cf0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.084+0000 7fe01c922640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fdff806b4d0 0x7fdff806d990 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.084+0000 7fe01c922640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0180afc80 0x7fe01809d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.084+0000 7fe01c922640 1 --2- 172.21.15.139:0/3228957422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe01806b700 0x7fe01814d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.084+0000 7fe01c922640 1 -- 172.21.15.139:0/3228957422 >> 172.21.15.139:0/3228957422 conn(0x7fe0180b2230 msgr2=0x7fe0180c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.084+0000 7fe01c922640 1 -- 172.21.15.139:0/3228957422 shutdown_connections 2024-04-25T10:03:37.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.084+0000 7fe01c922640 1 -- 172.21.15.139:0/3228957422 wait complete. 2024-04-25T10:03:37.099 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:03:37.100 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:03:37.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.196+0000 7f0cd881c640 1 Processor -- start 2024-04-25T10:03:37.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.196+0000 7f0cd881c640 1 -- start start 2024-04-25T10:03:37.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.196+0000 7f0cd881c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0cd415cb20 0x7f0cd40cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.196+0000 7f0cd881c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0cd40cfeb0 0x7f0cd406e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.196+0000 7f0cd881c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd40ccef0 con 0x7f0cd40afc00 2024-04-25T10:03:37.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.196+0000 7f0cd881c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd40d0320 con 0x7f0cd40cfeb0 2024-04-25T10:03:37.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.196+0000 7f0cd881c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40703c0 con 0x7f0cd415cb20 2024-04-25T10:03:37.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.197+0000 7f0ccbfff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0cd415cb20 0x7f0cd40cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.197+0000 7f0cd2d9d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0cd40cfeb0 0x7f0cd406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.197+0000 7f0ccbfff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0cd415cb20 0x7f0cd40cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:54678/0 (socket says 172.21.15.139:54678) 2024-04-25T10:03:37.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.197+0000 7f0ccbfff640 1 -- 172.21.15.139:0/1162717154 learned_addr learned my addr 172.21.15.139:0/1162717154 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:37.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.197+0000 7f0cd2d9d640 1 -- 172.21.15.139:0/1162717154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0cd415cb20 msgr2=0x7f0cd40cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.197+0000 7f0cd2d9d640 1 --2- 172.21.15.139:0/1162717154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0cd415cb20 0x7f0cd40cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.197+0000 7f0cd2d9d640 1 -- 172.21.15.139:0/1162717154 >> v1:172.21.15.150:6789/0 conn(0x7f0cd40afc00 legacy=0x7f0cd406fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.197+0000 7f0cd2d9d640 1 -- 172.21.15.139:0/1162717154 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40cd1d0 con 0x7f0cd40cfeb0 2024-04-25T10:03:37.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.197+0000 7f0ccbfff640 1 --2- 172.21.15.139:0/1162717154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0cd415cb20 0x7f0cd40cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:37.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.198+0000 7f0cd2d9d640 1 --2- 172.21.15.139:0/1162717154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cd40cfeb0 0x7f0cd406e370 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009d90 tx=0x7f0ccc02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f37a51cb793a9335 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.198+0000 7f0cd1d9b640 1 -- 172.21.15.139:0/1162717154 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc031600 con 0x7f0cd40cfeb0 2024-04-25T10:03:37.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.198+0000 7f0cd1d9b640 1 -- 172.21.15.139:0/1162717154 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc0317a0 con 0x7f0cd40cfeb0 2024-04-25T10:03:37.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.198+0000 7f0cd1d9b640 1 -- 172.21.15.139:0/1162717154 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc031b70 con 0x7f0cd40cfeb0 2024-04-25T10:03:37.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.199+0000 7f0cd881c640 1 -- 172.21.15.139:0/1162717154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cd40cfeb0 msgr2=0x7f0cd406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.199+0000 7f0cd881c640 1 --2- 172.21.15.139:0/1162717154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cd40cfeb0 0x7f0cd406e370 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009d90 tx=0x7f0ccc02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.199+0000 7f0cd881c640 1 -- 172.21.15.139:0/1162717154 shutdown_connections 2024-04-25T10:03:37.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.199+0000 7f0cd881c640 1 --2- 172.21.15.139:0/1162717154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0cd415cb20 0x7f0cd40cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.199+0000 7f0cd881c640 1 --2- 172.21.15.139:0/1162717154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cd40cfeb0 0x7f0cd406e370 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.199+0000 7f0cd881c640 1 -- 172.21.15.139:0/1162717154 >> 172.21.15.139:0/1162717154 conn(0x7f0cd40b2230 msgr2=0x7f0cd40b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.199+0000 7f0cd881c640 1 -- 172.21.15.139:0/1162717154 shutdown_connections 2024-04-25T10:03:37.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.199+0000 7f0cd881c640 1 -- 172.21.15.139:0/1162717154 wait complete. 2024-04-25T10:03:37.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.200+0000 7f0cd881c640 1 Processor -- start 2024-04-25T10:03:37.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.200+0000 7f0cd881c640 1 -- start start 2024-04-25T10:03:37.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd881c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0cd40afc00 0x7f0cd415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd881c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cd40cfeb0 0x7f0cd415a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd881c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cd415cb20 0x7f0cd40c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd881c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40affe0 con 0x7f0cd415cb20 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd881c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40cd820 con 0x7f0cd40cfeb0 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd881c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40cdcb0 con 0x7f0cd40afc00 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd2d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cd415cb20 0x7f0cd40c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd2d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cd415cb20 0x7f0cd40c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49052/0 (socket says 172.21.15.139:49052) 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd2d9d640 1 -- 172.21.15.139:0/4229284729 learned_addr learned my addr 172.21.15.139:0/4229284729 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd259c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0cd40afc00 0x7f0cd415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0ccbfff640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cd40cfeb0 0x7f0cd415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd2d9d640 1 -- 172.21.15.139:0/4229284729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0cd40afc00 msgr2=0x7f0cd415e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd2d9d640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0cd40afc00 0x7f0cd415e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd2d9d640 1 -- 172.21.15.139:0/4229284729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cd40cfeb0 msgr2=0x7f0cd415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd2d9d640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cd40cfeb0 0x7f0cd415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd2d9d640 1 -- 172.21.15.139:0/4229284729 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd415bc60 con 0x7f0cd415cb20 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0cd2d9d640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cd415cb20 0x7f0cd40c76d0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009eb0 tx=0x7f0ccc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.201+0000 7f0ccaffd640 1 -- 172.21.15.139:0/4229284729 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc03f070 con 0x7f0cd415cb20 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.202+0000 7f0ccaffd640 1 -- 172.21.15.139:0/4229284729 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc030c00 con 0x7f0cd415cb20 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.202+0000 7f0ccaffd640 1 -- 172.21.15.139:0/4229284729 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc009450 con 0x7f0cd415cb20 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.202+0000 7f0cd881c640 1 -- 172.21.15.139:0/4229284729 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd40c8200 con 0x7f0cd415cb20 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.202+0000 7f0cd881c640 1 -- 172.21.15.139:0/4229284729 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd415a000 con 0x7f0cd415cb20 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.202+0000 7f0cd881c640 1 -- 172.21.15.139:0/4229284729 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0cd4074520 con 0x7f0cd415cb20 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.203+0000 7f0ccaffd640 1 -- 172.21.15.139:0/4229284729 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ccc03b410 con 0x7f0cd415cb20 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.203+0000 7f0ccaffd640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0cb406b4d0 0x7f0cb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.203+0000 7f0cd259c640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0cb406b4d0 0x7f0cb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.203+0000 7f0ccaffd640 1 -- 172.21.15.139:0/4229284729 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f0ccc0ab240 con 0x7f0cd415cb20 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.203+0000 7f0ccaffd640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x7f0cb4071ab0 0x7f0cb4073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.203+0000 7f0ccaffd640 1 -- 172.21.15.139:0/4229284729 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ccc0091a0 con 0x7f0cb4071ab0 2024-04-25T10:03:37.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.203+0000 7f0ccaffd640 1 -- 172.21.15.139:0/4229284729 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ccc0b2050 con 0x7f0cd415cb20 2024-04-25T10:03:37.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.204+0000 7f0ccbfff640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x7f0cb4071ab0 0x7f0cb4073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.204+0000 7f0cd259c640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0cb406b4d0 0x7f0cb406d990 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40016e0 tx=0x7f0cc4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.204+0000 7f0ccbfff640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x7f0cb4071ab0 0x7f0cb4073ef0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.205+0000 7f0ccaffd640 1 -- 172.21.15.139:0/4229284729 <== osd.6 v2:172.21.15.150:6809/3369628189 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0ccc0091a0 con 0x7f0cb4071ab0 2024-04-25T10:03:37.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.217+0000 7f0cd881c640 1 -- 172.21.15.139:0/4229284729 --> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0cd40ceed0 con 0x7f0cb4071ab0 2024-04-25T10:03:37.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.218+0000 7f0ccaffd640 1 -- 172.21.15.139:0/4229284729 <== osd.6 v2:172.21.15.150:6809/3369628189 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0cd40ceed0 con 0x7f0cb4071ab0 2024-04-25T10:03:37.224 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:03:37.224 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:03:37.224 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:03:37.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.218+0000 7f0cd881c640 1 -- 172.21.15.139:0/4229284729 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x7f0cb4071ab0 msgr2=0x7f0cb4073ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.218+0000 7f0cd881c640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.150:6809/3369628189,v1:172.21.15.150:6811/3369628189] conn(0x7f0cb4071ab0 0x7f0cb4073ef0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.219+0000 7f0cd881c640 1 -- 172.21.15.139:0/4229284729 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0cb406b4d0 msgr2=0x7f0cb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.219+0000 7f0cd881c640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0cb406b4d0 0x7f0cb406d990 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40016e0 tx=0x7f0cc4009040 comp rx=0 tx=0).stop 2024-04-25T10:03:37.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.219+0000 7f0cd881c640 1 -- 172.21.15.139:0/4229284729 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cd415cb20 msgr2=0x7f0cd40c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.219+0000 7f0cd881c640 1 --2- 172.21.15.139:0/4229284729 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0cd415cb20 0x7f0cd40c76d0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009eb0 tx=0x7f0ccc004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:37.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.219+0000 7f0cd2d9d640 1 -- 172.21.15.139:0/4229284729 reap_dead start 2024-04-25T10:03:37.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.219+0000 7f0cd881c640 1 -- 172.21.15.139:0/4229284729 shutdown_connections 2024-04-25T10:03:37.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.219+0000 7f0cd881c640 1 -- 172.21.15.139:0/4229284729 >> 172.21.15.139:0/4229284729 conn(0x7f0cd40b2230 msgr2=0x7f0cd40d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.219+0000 7f0cd881c640 1 -- 172.21.15.139:0/4229284729 shutdown_connections 2024-04-25T10:03:37.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.219+0000 7f0cd881c640 1 -- 172.21.15.139:0/4229284729 wait complete. 2024-04-25T10:03:37.234 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:03:37.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.327+0000 7fbf223c4640 1 Processor -- start 2024-04-25T10:03:37.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.327+0000 7fbf223c4640 1 -- start start 2024-04-25T10:03:37.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf223c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c0af8a0 0x7fbf1c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf223c4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbf1c0afc80 0x7fbf1c0cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf223c4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1c0c6cc0 con 0x7fbf1c06b700 2024-04-25T10:03:37.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf223c4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1c0c6fa0 con 0x7fbf1c0afc80 2024-04-25T10:03:37.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf223c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c0c7280 con 0x7fbf1c0af8a0 2024-04-25T10:03:37.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf20961640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbf1c06b700 0x7fbf1c06ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42796/0 (socket says 172.21.15.139:42796) 2024-04-25T10:03:37.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf20961640 1 -- 172.21.15.139:0/2177360494 learned_addr learned my addr 172.21.15.139:0/2177360494 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:37.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf1bfff640 1 --2- 172.21.15.139:0/2177360494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c0af8a0 0x7fbf1c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf1b7fe640 1 --2- 172.21.15.139:0/2177360494 >> v2:172.21.15.150:3300/0 conn(0x7fbf1c0afc80 0x7fbf1c0cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf1affd640 1 -- 172.21.15.139:0/2177360494 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425356099 0 0) 0x7fbf1c0c6cc0 con 0x7fbf1c06b700 2024-04-25T10:03:37.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf1affd640 1 -- 172.21.15.139:0/2177360494 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf04003520 con 0x7fbf1c06b700 2024-04-25T10:03:37.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf1b7fe640 1 -- 172.21.15.139:0/2177360494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c0af8a0 msgr2=0x7fbf1c0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf1affd640 1 -- 172.21.15.139:0/2177360494 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4282720816 0 0) 0x7fbf04003520 con 0x7fbf1c06b700 2024-04-25T10:03:37.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf1b7fe640 1 --2- 172.21.15.139:0/2177360494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c0af8a0 0x7fbf1c0b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf1b7fe640 1 -- 172.21.15.139:0/2177360494 >> v1:172.21.15.139:6789/0 conn(0x7fbf1c06b700 legacy=0x7fbf1c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf1b7fe640 1 -- 172.21.15.139:0/2177360494 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c0cd090 con 0x7fbf1c0afc80 2024-04-25T10:03:37.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.328+0000 7fbf1bfff640 1 --2- 172.21.15.139:0/2177360494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c0af8a0 0x7fbf1c0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:37.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.329+0000 7fbf1b7fe640 1 --2- 172.21.15.139:0/2177360494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf1c0afc80 0x7fbf1c0cd890 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbf100097a0 tx=0x7fbf1002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2cc5de3e321e690 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.330+0000 7fbf1affd640 1 -- 172.21.15.139:0/2177360494 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10037070 con 0x7fbf1c0afc80 2024-04-25T10:03:37.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.330+0000 7fbf1affd640 1 -- 172.21.15.139:0/2177360494 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1002aba0 con 0x7fbf1c0afc80 2024-04-25T10:03:37.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.330+0000 7fbf1affd640 1 -- 172.21.15.139:0/2177360494 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf140019f0 con 0x7fbf1c0afc80 2024-04-25T10:03:37.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.330+0000 7fbf223c4640 1 -- 172.21.15.139:0/2177360494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf1c0afc80 msgr2=0x7fbf1c0cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.330+0000 7fbf223c4640 1 --2- 172.21.15.139:0/2177360494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf1c0afc80 0x7fbf1c0cd890 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbf100097a0 tx=0x7fbf1002a110 comp rx=0 tx=0).stop 2024-04-25T10:03:37.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.330+0000 7fbf223c4640 1 -- 172.21.15.139:0/2177360494 shutdown_connections 2024-04-25T10:03:37.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.330+0000 7fbf223c4640 1 --2- 172.21.15.139:0/2177360494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf1c0afc80 0x7fbf1c0cd890 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.330+0000 7fbf223c4640 1 --2- 172.21.15.139:0/2177360494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c0af8a0 0x7fbf1c0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.330+0000 7fbf223c4640 1 -- 172.21.15.139:0/2177360494 >> 172.21.15.139:0/2177360494 conn(0x7fbf1c0b2230 msgr2=0x7fbf1c0b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.330+0000 7fbf223c4640 1 -- 172.21.15.139:0/2177360494 shutdown_connections 2024-04-25T10:03:37.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.330+0000 7fbf223c4640 1 -- 172.21.15.139:0/2177360494 wait complete. 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.331+0000 7fbf223c4640 1 Processor -- start 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.331+0000 7fbf223c4640 1 -- start start 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf223c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf1c06b700 0x7fbf1c160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf223c4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf1c0af8a0 0x7fbf1c15e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf223c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c0afc80 0x7fbf1c14b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf223c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c06b270 con 0x7fbf1c06b700 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf223c4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c0cddd0 con 0x7fbf1c0af8a0 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf223c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c06b3f0 con 0x7fbf1c0afc80 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf1bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf1c06b700 0x7fbf1c160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf1bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf1c06b700 0x7fbf1c160810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49056/0 (socket says 172.21.15.139:49056) 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf1bfff640 1 -- 172.21.15.139:0/2297753355 learned_addr learned my addr 172.21.15.139:0/2297753355 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf1bfff640 1 -- 172.21.15.139:0/2297753355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c0afc80 msgr2=0x7fbf1c14b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf1bfff640 1 --2- 172.21.15.139:0/2297753355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c0afc80 0x7fbf1c14b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf1bfff640 1 -- 172.21.15.139:0/2297753355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf1c0af8a0 msgr2=0x7fbf1c15e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.332+0000 7fbf1bfff640 1 --2- 172.21.15.139:0/2297753355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf1c0af8a0 0x7fbf1c15e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.333+0000 7fbf1bfff640 1 -- 172.21.15.139:0/2297753355 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c160df0 con 0x7fbf1c06b700 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.333+0000 7fbf1bfff640 1 --2- 172.21.15.139:0/2297753355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf1c06b700 0x7fbf1c160810 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00ebe0 tx=0x7fbf0c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.333+0000 7fbf18ff9640 1 -- 172.21.15.139:0/2297753355 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c017070 con 0x7fbf1c06b700 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.333+0000 7fbf18ff9640 1 -- 172.21.15.139:0/2297753355 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c00cc30 con 0x7fbf1c06b700 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.333+0000 7fbf223c4640 1 -- 172.21.15.139:0/2297753355 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1c1610d0 con 0x7fbf1c06b700 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.333+0000 7fbf18ff9640 1 -- 172.21.15.139:0/2297753355 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c013470 con 0x7fbf1c06b700 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.333+0000 7fbf223c4640 1 -- 172.21.15.139:0/2297753355 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c161690 con 0x7fbf1c06b700 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.334+0000 7fbf223c4640 1 -- 172.21.15.139:0/2297753355 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1c074520 con 0x7fbf1c06b700 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.334+0000 7fbf18ff9640 1 -- 172.21.15.139:0/2297753355 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf0c01a070 con 0x7fbf1c06b700 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.335+0000 7fbf18ff9640 1 --2- 172.21.15.139:0/2297753355 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf0806b500 0x7fbf0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.335+0000 7fbf18ff9640 1 -- 172.21.15.139:0/2297753355 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fbf0c08a4c0 con 0x7fbf1c06b700 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.335+0000 7fbf1b7fe640 1 --2- 172.21.15.139:0/2297753355 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf0806b500 0x7fbf0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.335+0000 7fbf1b7fe640 1 --2- 172.21.15.139:0/2297753355 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf0806b500 0x7fbf0806d9c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbf10034000 tx=0x7fbf1002c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.339+0000 7fbf18ff9640 1 -- 172.21.15.139:0/2297753355 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf0c059370 con 0x7fbf1c06b700 2024-04-25T10:03:37.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.511+0000 7fbf223c4640 1 -- 172.21.15.139:0/2297753355 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf1c15d7b0 con 0x7fbf0806b500 2024-04-25T10:03:37.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.511+0000 7fbf18ff9640 1 -- 172.21.15.139:0/2297753355 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbf1c15d7b0 con 0x7fbf0806b500 2024-04-25T10:03:37.516 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:03:37.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.513+0000 7fbf223c4640 1 -- 172.21.15.139:0/2297753355 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf0806b500 msgr2=0x7fbf0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.513+0000 7fbf223c4640 1 --2- 172.21.15.139:0/2297753355 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf0806b500 0x7fbf0806d9c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbf10034000 tx=0x7fbf1002c000 comp rx=0 tx=0).stop 2024-04-25T10:03:37.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.513+0000 7fbf223c4640 1 -- 172.21.15.139:0/2297753355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf1c06b700 msgr2=0x7fbf1c160810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.513+0000 7fbf223c4640 1 --2- 172.21.15.139:0/2297753355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf1c06b700 0x7fbf1c160810 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00ebe0 tx=0x7fbf0c00c520 comp rx=0 tx=0).stop 2024-04-25T10:03:37.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.513+0000 7fbf223c4640 1 -- 172.21.15.139:0/2297753355 shutdown_connections 2024-04-25T10:03:37.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.513+0000 7fbf223c4640 1 --2- 172.21.15.139:0/2297753355 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fbf0806b500 0x7fbf0806d9c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.513+0000 7fbf223c4640 1 --2- 172.21.15.139:0/2297753355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf1c0afc80 0x7fbf1c14b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.513+0000 7fbf223c4640 1 --2- 172.21.15.139:0/2297753355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf1c0af8a0 0x7fbf1c15e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.513+0000 7fbf223c4640 1 --2- 172.21.15.139:0/2297753355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf1c06b700 0x7fbf1c160810 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.513+0000 7fbf223c4640 1 -- 172.21.15.139:0/2297753355 >> 172.21.15.139:0/2297753355 conn(0x7fbf1c0b2230 msgr2=0x7fbf1c0738a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.514+0000 7fbf223c4640 1 -- 172.21.15.139:0/2297753355 shutdown_connections 2024-04-25T10:03:37.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.514+0000 7fbf223c4640 1 -- 172.21.15.139:0/2297753355 wait complete. 2024-04-25T10:03:37.529 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:03:37.530 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:03:37.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.627+0000 7f0fdd84a640 1 Processor -- start 2024-04-25T10:03:37.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.627+0000 7f0fdd84a640 1 -- start start 2024-04-25T10:03:37.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.627+0000 7f0fdd84a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fd815dbf0 0x7f0fd81617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.627+0000 7f0fdd84a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0fd80b9370 0x7f0fd80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fdd84a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd8161dc0 con 0x7f0fd80cec80 2024-04-25T10:03:37.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fcffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0fd80b9370 0x7f0fd80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fd77fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0fd80cec80 0x7f0fd806c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42804/0 (socket says 172.21.15.139:42804) 2024-04-25T10:03:37.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fdd84a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd80d1ee0 con 0x7f0fd80b9370 2024-04-25T10:03:37.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fd77fe640 1 -- 172.21.15.139:0/2210132576 learned_addr learned my addr 172.21.15.139:0/2210132576 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:37.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fdd84a640 1 -- 172.21.15.139:0/2210132576 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd81620a0 con 0x7f0fd815dbf0 2024-04-25T10:03:37.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fd6ffd640 1 --2- 172.21.15.139:0/2210132576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fd815dbf0 0x7f0fd81617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fcffff640 1 -- 172.21.15.139:0/2210132576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fd815dbf0 msgr2=0x7f0fd81617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fcffff640 1 --2- 172.21.15.139:0/2210132576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fd815dbf0 0x7f0fd81617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fcffff640 1 -- 172.21.15.139:0/2210132576 >> v1:172.21.15.139:6789/0 conn(0x7f0fd80cec80 legacy=0x7f0fd806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fcffff640 1 -- 172.21.15.139:0/2210132576 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd8162380 con 0x7f0fd80b9370 2024-04-25T10:03:37.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fd67fc640 1 -- 172.21.15.139:0/2210132576 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578592397 0 0) 0x7f0fd8161dc0 con 0x7f0fd80cec80 2024-04-25T10:03:37.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.628+0000 7f0fd6ffd640 1 --2- 172.21.15.139:0/2210132576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fd815dbf0 0x7f0fd81617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:37.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.629+0000 7f0fcffff640 1 --2- 172.21.15.139:0/2210132576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fd80b9370 0x7f0fd80ce740 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00097a0 tx=0x7f0fc002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc66d045524f3728 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.629+0000 7f0fd67fc640 1 -- 172.21.15.139:0/2210132576 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0037070 con 0x7f0fd80b9370 2024-04-25T10:03:37.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.629+0000 7f0fd67fc640 1 -- 172.21.15.139:0/2210132576 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc0032440 con 0x7f0fd80b9370 2024-04-25T10:03:37.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.629+0000 7f0fd67fc640 1 -- 172.21.15.139:0/2210132576 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0032810 con 0x7f0fd80b9370 2024-04-25T10:03:37.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.629+0000 7f0fdd84a640 1 -- 172.21.15.139:0/2210132576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fd80b9370 msgr2=0x7f0fd80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.629+0000 7f0fdd84a640 1 --2- 172.21.15.139:0/2210132576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fd80b9370 0x7f0fd80ce740 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00097a0 tx=0x7f0fc002a180 comp rx=0 tx=0).stop 2024-04-25T10:03:37.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.629+0000 7f0fdd84a640 1 -- 172.21.15.139:0/2210132576 shutdown_connections 2024-04-25T10:03:37.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.629+0000 7f0fdd84a640 1 --2- 172.21.15.139:0/2210132576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fd80b9370 0x7f0fd80ce740 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.629+0000 7f0fdd84a640 1 --2- 172.21.15.139:0/2210132576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fd815dbf0 0x7f0fd81617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.629+0000 7f0fdd84a640 1 -- 172.21.15.139:0/2210132576 >> 172.21.15.139:0/2210132576 conn(0x7f0fd80b5480 msgr2=0x7f0fd80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.630+0000 7f0fdd84a640 1 -- 172.21.15.139:0/2210132576 shutdown_connections 2024-04-25T10:03:37.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.630+0000 7f0fdd84a640 1 -- 172.21.15.139:0/2210132576 wait complete. 2024-04-25T10:03:37.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.631+0000 7f0fdd84a640 1 Processor -- start 2024-04-25T10:03:37.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.631+0000 7f0fdd84a640 1 -- start start 2024-04-25T10:03:37.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.631+0000 7f0fdd84a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fd80b9370 0x7f0fd815c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.631+0000 7f0fdd84a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fd80cec80 0x7f0fd81f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.631+0000 7f0fdd84a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fd815dbf0 0x7f0fd80b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.631+0000 7f0fdd84a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd815dfd0 con 0x7f0fd80b9370 2024-04-25T10:03:37.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.631+0000 7f0fdd84a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80b97d0 con 0x7f0fd80cec80 2024-04-25T10:03:37.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.631+0000 7f0fdd84a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd81f4440 con 0x7f0fd815dbf0 2024-04-25T10:03:37.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.631+0000 7f0fd6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fd80b9370 0x7f0fd815c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.631+0000 7f0fd6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fd80b9370 0x7f0fd815c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49074/0 (socket says 172.21.15.139:49074) 2024-04-25T10:03:37.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fd6ffd640 1 -- 172.21.15.139:0/4016997491 learned_addr learned my addr 172.21.15.139:0/4016997491 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:37.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fcffff640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fd80cec80 0x7f0fd81f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fd77fe640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fd815dbf0 0x7f0fd80b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fd6ffd640 1 -- 172.21.15.139:0/4016997491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fd815dbf0 msgr2=0x7f0fd80b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fd6ffd640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fd815dbf0 0x7f0fd80b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fd6ffd640 1 -- 172.21.15.139:0/4016997491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fd80cec80 msgr2=0x7f0fd81f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fd6ffd640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fd80cec80 0x7f0fd81f3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fd6ffd640 1 -- 172.21.15.139:0/4016997491 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd80b5270 con 0x7f0fd80b9370 2024-04-25T10:03:37.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fd6ffd640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fd80b9370 0x7f0fd815c410 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800c850 tx=0x7f0fc800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fd4ff9640 1 -- 172.21.15.139:0/4016997491 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8015070 con 0x7f0fd80b9370 2024-04-25T10:03:37.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fd4ff9640 1 -- 172.21.15.139:0/4016997491 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc8007a30 con 0x7f0fd80b9370 2024-04-25T10:03:37.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fdd84a640 1 -- 172.21.15.139:0/4016997491 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd815cb10 con 0x7f0fd80b9370 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fd4ff9640 1 -- 172.21.15.139:0/4016997491 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800cea0 con 0x7f0fd80b9370 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.632+0000 7f0fdd84a640 1 -- 172.21.15.139:0/4016997491 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd815d0d0 con 0x7f0fd80b9370 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.633+0000 7f0fdd84a640 1 -- 172.21.15.139:0/4016997491 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0fd8074520 con 0x7f0fd80b9370 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.634+0000 7f0fd4ff9640 1 -- 172.21.15.139:0/4016997491 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fc8018070 con 0x7f0fd80b9370 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.634+0000 7f0fd4ff9640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fb806b4d0 0x7f0fb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.634+0000 7f0fd4ff9640 1 -- 172.21.15.139:0/4016997491 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f0fc808a0d0 con 0x7f0fd80b9370 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.634+0000 7f0fcffff640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fb806b4d0 0x7f0fb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.634+0000 7f0fd4ff9640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x7f0fb8071ab0 0x7f0fb8073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.634+0000 7f0fd4ff9640 1 -- 172.21.15.139:0/4016997491 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fc8009fa0 con 0x7f0fb8071ab0 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.634+0000 7f0fd4ff9640 1 -- 172.21.15.139:0/4016997491 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fc8091050 con 0x7f0fd80b9370 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.634+0000 7f0fd77fe640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x7f0fb8071ab0 0x7f0fb8073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.634+0000 7f0fcffff640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fb806b4d0 0x7f0fb806d990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0034000 tx=0x7f0fc002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.635+0000 7f0fd77fe640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x7f0fb8071ab0 0x7f0fb8073ef0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.636+0000 7f0fd4ff9640 1 -- 172.21.15.139:0/4016997491 <== osd.7 v2:172.21.15.150:6808/2122423536 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0fc8009fa0 con 0x7f0fb8071ab0 2024-04-25T10:03:37.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.649+0000 7f0fdd84a640 1 -- 172.21.15.139:0/4016997491 --> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0fd81f47c0 con 0x7f0fb8071ab0 2024-04-25T10:03:37.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.649+0000 7f0fd4ff9640 1 -- 172.21.15.139:0/4016997491 <== osd.7 v2:172.21.15.150:6808/2122423536 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0fd81f47c0 con 0x7f0fb8071ab0 2024-04-25T10:03:37.654 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T10:03:37.654 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:03:37.654 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T10:03:37.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.650+0000 7f0fdd84a640 1 -- 172.21.15.139:0/4016997491 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x7f0fb8071ab0 msgr2=0x7f0fb8073ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.650+0000 7f0fdd84a640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.150:6808/2122423536,v1:172.21.15.150:6810/2122423536] conn(0x7f0fb8071ab0 0x7f0fb8073ef0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.650+0000 7f0fdd84a640 1 -- 172.21.15.139:0/4016997491 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fb806b4d0 msgr2=0x7f0fb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.650+0000 7f0fdd84a640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f0fb806b4d0 0x7f0fb806d990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0034000 tx=0x7f0fc002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:37.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.650+0000 7f0fdd84a640 1 -- 172.21.15.139:0/4016997491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fd80b9370 msgr2=0x7f0fd815c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.650+0000 7f0fdd84a640 1 --2- 172.21.15.139:0/4016997491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0fd80b9370 0x7f0fd815c410 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800c850 tx=0x7f0fc800cc50 comp rx=0 tx=0).stop 2024-04-25T10:03:37.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.650+0000 7f0fd77fe640 1 -- 172.21.15.139:0/4016997491 reap_dead start 2024-04-25T10:03:37.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.650+0000 7f0fdd84a640 1 -- 172.21.15.139:0/4016997491 shutdown_connections 2024-04-25T10:03:37.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.650+0000 7f0fdd84a640 1 -- 172.21.15.139:0/4016997491 >> 172.21.15.139:0/4016997491 conn(0x7f0fd80b5480 msgr2=0x7f0fd8158170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.650+0000 7f0fdd84a640 1 -- 172.21.15.139:0/4016997491 shutdown_connections 2024-04-25T10:03:37.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.650+0000 7f0fdd84a640 1 -- 172.21.15.139:0/4016997491 wait complete. 2024-04-25T10:03:37.665 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:03:37.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.758+0000 7f3436ec4640 1 Processor -- start 2024-04-25T10:03:37.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.758+0000 7f3436ec4640 1 -- start start 2024-04-25T10:03:37.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3436ec4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34301617e0 0x7f34300ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3436ec4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 0x7f34300d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3436ec4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3430161dc0 con 0x7f343015dbf0 2024-04-25T10:03:37.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3436ec4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34300d1ee0 con 0x7f34301617e0 2024-04-25T10:03:37.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3436ec4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34301620a0 con 0x7f34300cec80 2024-04-25T10:03:37.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3435461640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 0x7f34300d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3427fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34301617e0 0x7f34300ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3435461640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 0x7f34300d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:54736/0 (socket says 172.21.15.139:54736) 2024-04-25T10:03:37.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3427fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34301617e0 0x7f34300ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:41434/0 (socket says 172.21.15.139:41434) 2024-04-25T10:03:37.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3435461640 1 -- 172.21.15.139:0/451841249 learned_addr learned my addr 172.21.15.139:0/451841249 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:37.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3427fff640 1 -- 172.21.15.139:0/451841249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 msgr2=0x7f34300d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3427fff640 1 --2- 172.21.15.139:0/451841249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 0x7f34300d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3427fff640 1 -- 172.21.15.139:0/451841249 >> v1:172.21.15.150:6789/0 conn(0x7f343015dbf0 legacy=0x7f34300b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3427fff640 1 -- 172.21.15.139:0/451841249 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3430162380 con 0x7f34301617e0 2024-04-25T10:03:37.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.759+0000 7f3435461640 1 --2- 172.21.15.139:0/451841249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 0x7f34300d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:37.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.760+0000 7f3427fff640 1 --2- 172.21.15.139:0/451841249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34301617e0 0x7f34300ce740 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f34200097a0 tx=0x7f342002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26e3e8b1de28d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.760+0000 7f34277fe640 1 -- 172.21.15.139:0/451841249 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420037070 con 0x7f34301617e0 2024-04-25T10:03:37.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.760+0000 7f34277fe640 1 -- 172.21.15.139:0/451841249 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342002ae50 con 0x7f34301617e0 2024-04-25T10:03:37.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.760+0000 7f34277fe640 1 -- 172.21.15.139:0/451841249 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420032660 con 0x7f34301617e0 2024-04-25T10:03:37.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.761+0000 7f3436ec4640 1 -- 172.21.15.139:0/451841249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34301617e0 msgr2=0x7f34300ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.761+0000 7f3436ec4640 1 --2- 172.21.15.139:0/451841249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34301617e0 0x7f34300ce740 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f34200097a0 tx=0x7f342002a100 comp rx=0 tx=0).stop 2024-04-25T10:03:37.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.761+0000 7f3436ec4640 1 -- 172.21.15.139:0/451841249 shutdown_connections 2024-04-25T10:03:37.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.761+0000 7f3436ec4640 1 --2- 172.21.15.139:0/451841249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 0x7f34300d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.761+0000 7f3436ec4640 1 --2- 172.21.15.139:0/451841249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34301617e0 0x7f34300ce740 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.761+0000 7f3436ec4640 1 -- 172.21.15.139:0/451841249 >> 172.21.15.139:0/451841249 conn(0x7f34300b5480 msgr2=0x7f34300b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.761+0000 7f3436ec4640 1 -- 172.21.15.139:0/451841249 shutdown_connections 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.761+0000 7f3436ec4640 1 -- 172.21.15.139:0/451841249 wait complete. 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.762+0000 7f3436ec4640 1 Processor -- start 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.762+0000 7f3436ec4640 1 -- start start 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3436ec4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 0x7f34300ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3436ec4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f343015dbf0 0x7f34300c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3436ec4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34301617e0 0x7f34300cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3436ec4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343015dfd0 con 0x7f34301617e0 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3436ec4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34300d28b0 con 0x7f343015dbf0 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3436ec4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34300cd260 con 0x7f34300cec80 2024-04-25T10:03:37.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3435461640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34301617e0 0x7f34300cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3435461640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34301617e0 0x7f34300cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49090/0 (socket says 172.21.15.139:49090) 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3435461640 1 -- 172.21.15.139:0/2143230144 learned_addr learned my addr 172.21.15.139:0/2143230144 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3434c60640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 0x7f34300ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3427fff640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f343015dbf0 0x7f34300c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3435461640 1 -- 172.21.15.139:0/2143230144 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 msgr2=0x7f34300ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3435461640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 0x7f34300ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3435461640 1 -- 172.21.15.139:0/2143230144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f343015dbf0 msgr2=0x7f34300c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3435461640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f343015dbf0 0x7f34300c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3435461640 1 -- 172.21.15.139:0/2143230144 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34300cbc80 con 0x7f34301617e0 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3427fff640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f343015dbf0 0x7f34300c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.763+0000 7f3434c60640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 0x7f34300ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.764+0000 7f3435461640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34301617e0 0x7f34300cb380 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f342800efc0 tx=0x7f342800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.764+0000 7f34257fa640 1 -- 172.21.15.139:0/2143230144 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342800cbe0 con 0x7f34301617e0 2024-04-25T10:03:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.764+0000 7f34257fa640 1 -- 172.21.15.139:0/2143230144 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342800cd80 con 0x7f34301617e0 2024-04-25T10:03:37.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.764+0000 7f3436ec4640 1 -- 172.21.15.139:0/2143230144 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34300cbf60 con 0x7f34301617e0 2024-04-25T10:03:37.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.764+0000 7f34257fa640 1 -- 172.21.15.139:0/2143230144 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428003bb0 con 0x7f34301617e0 2024-04-25T10:03:37.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.764+0000 7f3436ec4640 1 -- 172.21.15.139:0/2143230144 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34300cc520 con 0x7f34301617e0 2024-04-25T10:03:37.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.765+0000 7f3436ec4640 1 -- 172.21.15.139:0/2143230144 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3430067420 con 0x7f34301617e0 2024-04-25T10:03:37.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.765+0000 7f34257fa640 1 -- 172.21.15.139:0/2143230144 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f342801b030 con 0x7f34301617e0 2024-04-25T10:03:37.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.765+0000 7f34257fa640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f341c06b4d0 0x7f341c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.766+0000 7f34257fa640 1 -- 172.21.15.139:0/2143230144 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f342808a6a0 con 0x7f34301617e0 2024-04-25T10:03:37.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.766+0000 7f3434c60640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f341c06b4d0 0x7f341c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.766+0000 7f3434c60640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f341c06b4d0 0x7f341c06d990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f342c001680 tx=0x7f342c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.770+0000 7f34257fa640 1 -- 172.21.15.139:0/2143230144 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3430067420 con 0x7f34301617e0 2024-04-25T10:03:37.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.939+0000 7f3436ec4640 1 -- 172.21.15.139:0/2143230144 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f34300b5860 con 0x7f341c06b4d0 2024-04-25T10:03:37.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.940+0000 7f34257fa640 1 -- 172.21.15.139:0/2143230144 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f34300b5860 con 0x7f341c06b4d0 2024-04-25T10:03:37.945 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:03:37.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.941+0000 7f3436ec4640 1 -- 172.21.15.139:0/2143230144 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f341c06b4d0 msgr2=0x7f341c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.941+0000 7f3436ec4640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f341c06b4d0 0x7f341c06d990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f342c001680 tx=0x7f342c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:37.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.941+0000 7f3436ec4640 1 -- 172.21.15.139:0/2143230144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34301617e0 msgr2=0x7f34300cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.941+0000 7f3436ec4640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34301617e0 0x7f34300cb380 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f342800efc0 tx=0x7f342800c490 comp rx=0 tx=0).stop 2024-04-25T10:03:37.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.942+0000 7f3436ec4640 1 -- 172.21.15.139:0/2143230144 shutdown_connections 2024-04-25T10:03:37.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.942+0000 7f3436ec4640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f34301617e0 0x7f34300cb380 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.942+0000 7f3436ec4640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f341c06b4d0 0x7f341c06d990 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.942+0000 7f3436ec4640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f343015dbf0 0x7f34300c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.942+0000 7f3436ec4640 1 --2- 172.21.15.139:0/2143230144 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34300cec80 0x7f34300ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.942+0000 7f3436ec4640 1 -- 172.21.15.139:0/2143230144 >> 172.21.15.139:0/2143230144 conn(0x7f34300b5480 msgr2=0x7f34300d2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.942+0000 7f3436ec4640 1 -- 172.21.15.139:0/2143230144 shutdown_connections 2024-04-25T10:03:37.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:37.942+0000 7f3436ec4640 1 -- 172.21.15.139:0/2143230144 wait complete. 2024-04-25T10:03:37.958 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:38.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.050+0000 7fc109437640 1 Processor -- start 2024-04-25T10:03:38.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.050+0000 7fc109437640 1 -- start start 2024-04-25T10:03:38.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.051+0000 7fc109437640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc10406c780 0x7fc1040d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.051+0000 7fc109437640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1040cf110 con 0x7fc1040b6af0 2024-04-25T10:03:38.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.051+0000 7fc109437640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1040b6f50 con 0x7fc1041611e0 2024-04-25T10:03:38.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.051+0000 7fc109437640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1040d1fa0 con 0x7fc10406c780 2024-04-25T10:03:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.051+0000 7fc1027fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc1040b6af0 0x7fc10406c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:42814/0 (socket says 172.21.15.139:42814) 2024-04-25T10:03:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.051+0000 7fc1037fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc10406c780 0x7fc1040d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.051+0000 7fc1027fc640 1 -- 172.21.15.139:0/146050264 learned_addr learned my addr 172.21.15.139:0/146050264 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.051+0000 7fc1037fe640 1 -- 172.21.15.139:0/146050264 >> v1:172.21.15.150:6789/0 conn(0x7fc1041611e0 legacy=0x7fc1040cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.051+0000 7fc1037fe640 1 -- 172.21.15.139:0/146050264 >> v1:172.21.15.139:6789/0 conn(0x7fc1040b6af0 legacy=0x7fc10406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.051+0000 7fc1037fe640 1 -- 172.21.15.139:0/146050264 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1040d4740 con 0x7fc10406c780 2024-04-25T10:03:38.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.052+0000 7fc1037fe640 1 --2- 172.21.15.139:0/146050264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10406c780 0x7fc1040d29f0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009d90 tx=0x7fc0fc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8701e859009dd0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.052+0000 7fc101ffb640 1 -- 172.21.15.139:0/146050264 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc007740 con 0x7fc10406c780 2024-04-25T10:03:38.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.052+0000 7fc101ffb640 1 -- 172.21.15.139:0/146050264 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc0078e0 con 0x7fc10406c780 2024-04-25T10:03:38.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.052+0000 7fc101ffb640 1 -- 172.21.15.139:0/146050264 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc007cb0 con 0x7fc10406c780 2024-04-25T10:03:38.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.052+0000 7fc109437640 1 -- 172.21.15.139:0/146050264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10406c780 msgr2=0x7fc1040d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.052+0000 7fc109437640 1 --2- 172.21.15.139:0/146050264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10406c780 0x7fc1040d29f0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009d90 tx=0x7fc0fc02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.052+0000 7fc109437640 1 -- 172.21.15.139:0/146050264 shutdown_connections 2024-04-25T10:03:38.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.052+0000 7fc109437640 1 --2- 172.21.15.139:0/146050264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10406c780 0x7fc1040d29f0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.052+0000 7fc109437640 1 -- 172.21.15.139:0/146050264 >> 172.21.15.139:0/146050264 conn(0x7fc1040b5b00 msgr2=0x7fc1040d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:38.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.052+0000 7fc109437640 1 -- 172.21.15.139:0/146050264 shutdown_connections 2024-04-25T10:03:38.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.052+0000 7fc109437640 1 -- 172.21.15.139:0/146050264 wait complete. 2024-04-25T10:03:38.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.054+0000 7fc109437640 1 Processor -- start 2024-04-25T10:03:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.054+0000 7fc109437640 1 -- start start 2024-04-25T10:03:38.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.054+0000 7fc109437640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10406c780 0x7fc10415fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.054+0000 7fc109437640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1040b6af0 0x7fc104156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.054+0000 7fc109437640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1041611e0 0x7fc1040caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.054+0000 7fc109437640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040b2a50 con 0x7fc10406c780 2024-04-25T10:03:38.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.054+0000 7fc109437640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040d2560 con 0x7fc1040b6af0 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.054+0000 7fc109437640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040b2d30 con 0x7fc1041611e0 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc102ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10406c780 0x7fc10415fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc102ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10406c780 0x7fc10415fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49118/0 (socket says 172.21.15.139:49118) 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc102ffd640 1 -- 172.21.15.139:0/2165443174 learned_addr learned my addr 172.21.15.139:0/2165443174 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc1037fe640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1041611e0 0x7fc1040caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc1027fc640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1040b6af0 0x7fc104156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc102ffd640 1 -- 172.21.15.139:0/2165443174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1041611e0 msgr2=0x7fc1040caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc102ffd640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1041611e0 0x7fc1040caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc102ffd640 1 -- 172.21.15.139:0/2165443174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1040b6af0 msgr2=0x7fc104156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc102ffd640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1040b6af0 0x7fc104156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc102ffd640 1 -- 172.21.15.139:0/2165443174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1040c9320 con 0x7fc10406c780 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc1027fc640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1040b6af0 0x7fc104156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc102ffd640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10406c780 0x7fc10415fa20 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c450 tx=0x7fc0f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc0eb7fe640 1 -- 172.21.15.139:0/2165443174 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40052d0 con 0x7fc10406c780 2024-04-25T10:03:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc0eb7fe640 1 -- 172.21.15.139:0/2165443174 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f4005470 con 0x7fc10406c780 2024-04-25T10:03:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc109437640 1 -- 172.21.15.139:0/2165443174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc104156550 con 0x7fc10406c780 2024-04-25T10:03:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc0eb7fe640 1 -- 172.21.15.139:0/2165443174 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4012870 con 0x7fc10406c780 2024-04-25T10:03:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.055+0000 7fc109437640 1 -- 172.21.15.139:0/2165443174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1040c9870 con 0x7fc10406c780 2024-04-25T10:03:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.056+0000 7fc109437640 1 -- 172.21.15.139:0/2165443174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc104074520 con 0x7fc10406c780 2024-04-25T10:03:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.057+0000 7fc0eb7fe640 1 -- 172.21.15.139:0/2165443174 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0f4012a10 con 0x7fc10406c780 2024-04-25T10:03:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.057+0000 7fc0eb7fe640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0f006b4d0 0x7fc0f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.057+0000 7fc0eb7fe640 1 -- 172.21.15.139:0/2165443174 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc0f4089cf0 con 0x7fc10406c780 2024-04-25T10:03:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.057+0000 7fc1027fc640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0f006b4d0 0x7fc0f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.057+0000 7fc1027fc640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0f006b4d0 0x7fc0f006d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc1041f7180 tx=0x7fc0f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.061+0000 7fc0eb7fe640 1 -- 172.21.15.139:0/2165443174 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0f408e070 con 0x7fc10406c780 2024-04-25T10:03:38.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.228+0000 7fc109437640 1 -- 172.21.15.139:0/2165443174 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc10406ceb0 con 0x7fc0f006b4d0 2024-04-25T10:03:38.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.229+0000 7fc0eb7fe640 1 -- 172.21.15.139:0/2165443174 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13515 (secure 0 0 0) 0x7fc10406ceb0 con 0x7fc0f006b4d0 2024-04-25T10:03:38.234 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:38.234 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T10:03:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.231+0000 7fc109437640 1 -- 172.21.15.139:0/2165443174 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0f006b4d0 msgr2=0x7fc0f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.231+0000 7fc109437640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0f006b4d0 0x7fc0f006d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc1041f7180 tx=0x7fc0f8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.231+0000 7fc109437640 1 -- 172.21.15.139:0/2165443174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10406c780 msgr2=0x7fc10415fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.231+0000 7fc109437640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10406c780 0x7fc10415fa20 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c450 tx=0x7fc0f400c910 comp rx=0 tx=0).stop 2024-04-25T10:03:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.231+0000 7fc109437640 1 -- 172.21.15.139:0/2165443174 shutdown_connections 2024-04-25T10:03:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.231+0000 7fc109437640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fc0f006b4d0 0x7fc0f006d990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.231+0000 7fc109437640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1041611e0 0x7fc1040caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.231+0000 7fc109437640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1040b6af0 0x7fc104156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.231+0000 7fc109437640 1 --2- 172.21.15.139:0/2165443174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10406c780 0x7fc10415fa20 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.231+0000 7fc109437640 1 -- 172.21.15.139:0/2165443174 >> 172.21.15.139:0/2165443174 conn(0x7fc1040b5b00 msgr2=0x7fc10406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.231+0000 7fc109437640 1 -- 172.21.15.139:0/2165443174 shutdown_connections 2024-04-25T10:03:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:38.231+0000 7fc109437640 1 -- 172.21.15.139:0/2165443174 wait complete. 2024-04-25T10:03:38.246 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-04-25T10:03:37.109938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262860,"kb_used_data":13384,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711860,"statfs":{"total":773094113280,"available":772824944640,"internally_reserved":0,"allocated":13705216,"data_stored":7023600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390540,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-174,"num_write_kb":-8302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001601"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":366,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-25T10:03:30.486560+0000","last_change":"2024-04-25T09:53:04.105575+0000","last_active":"2024-04-25T10:03:30.486560+0000","last_peered":"2024-04-25T10:03:30.486560+0000","last_clean":"2024-04-25T10:03:30.486560+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T10:03:30.486560+0000","last_undegraded":"2024-04-25T10:03:30.486560+0000","last_fullsized":"2024-04-25T10:03:30.486560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:53:03.030957+0000","last_clean_scrub_stamp":"2024-04-25T09:53:03.030957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:20.377105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574978,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2158592,"data_stored":1320726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574978,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574977,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574978,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574979,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574978,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33292,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338548,"statfs":{"total":96636764160,"available":96602673152,"internally_reserved":0,"allocated":2158592,"data_stored":1320726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:38.249 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T09:53:03.030957+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=9, tm_min=53, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=3, tm_sec=33, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:03:38.250 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T10:03:47.843 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:47.847 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:58.251 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:58.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.356+0000 7f0161be0640 1 Processor -- start 2024-04-25T10:03:58.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.356+0000 7f0161be0640 1 -- start start 2024-04-25T10:03:58.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.357+0000 7f0161be0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c158260 0x7f015c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.357+0000 7f0161be0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015c0b3e10 con 0x7f015c0b37d0 2024-04-25T10:03:58.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.357+0000 7f0161be0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015c1615c0 con 0x7f015c06da20 2024-04-25T10:03:58.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.357+0000 7f0161be0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c1618a0 con 0x7f015c158260 2024-04-25T10:03:58.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.357+0000 7f015bfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f015c0b37d0 0x7f015c06ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:40466/0 (socket says 172.21.15.139:40466) 2024-04-25T10:03:58.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.357+0000 7f015bfff640 1 -- 172.21.15.139:0/3407078667 learned_addr learned my addr 172.21.15.139:0/3407078667 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:58.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.357+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302750887 0 0) 0x7f0154001490 con 0x7f015c0b37d0 2024-04-25T10:03:58.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.357+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0144003520 con 0x7f015c0b37d0 2024-04-25T10:03:58.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015b7fe640 1 --2- 172.21.15.139:0/3407078667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c158260 0x7f015c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 971141889 0 0) 0x7f0144003520 con 0x7f015c0b37d0 2024-04-25T10:03:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c158260 msgr2=0x7f015c15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 --2- 172.21.15.139:0/3407078667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c158260 0x7f015c15df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 >> v1:172.21.15.150:6789/0 conn(0x7f015c06da20 legacy=0x7f015c0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c161b80 con 0x7f015c0b37d0 2024-04-25T10:03:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1636775792 0 0) 0x7f0154001bc0 con 0x7f015c0b37d0 2024-04-25T10:03:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 >> v1:172.21.15.139:6789/0 conn(0x7f015c0b37d0 legacy=0x7f015c06ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 --2- 172.21.15.139:0/3407078667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c158260 0x7f01440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 --2- 172.21.15.139:0/3407078667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144003f60 0x7f0144004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 --2- 172.21.15.139:0/3407078667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0144004d40 0x7f01440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c1618a0 con 0x7f0144004d40 2024-04-25T10:03:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014400baa0 con 0x7f0144003f60 2024-04-25T10:03:58.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014400bca0 con 0x7f015c158260 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015bfff640 1 --2- 172.21.15.139:0/3407078667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0144004d40 0x7f01440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015b7fe640 1 --2- 172.21.15.139:0/3407078667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c158260 0x7f01440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015affd640 1 --2- 172.21.15.139:0/3407078667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144003f60 0x7f0144004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015bfff640 1 -- 172.21.15.139:0/3407078667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c158260 msgr2=0x7f01440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015bfff640 1 --2- 172.21.15.139:0/3407078667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c158260 0x7f01440039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015bfff640 1 -- 172.21.15.139:0/3407078667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144003f60 msgr2=0x7f0144004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015bfff640 1 --2- 172.21.15.139:0/3407078667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144003f60 0x7f0144004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.358+0000 7f015bfff640 1 -- 172.21.15.139:0/3407078667 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0144003520 con 0x7f0144004d40 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.359+0000 7f015bfff640 1 --2- 172.21.15.139:0/3407078667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0144004d40 0x7f01440051c0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f015c161520 tx=0x7f0144005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617b92c3e9d4233f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.359+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015400a650 con 0x7f0144004d40 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.359+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015402f070 con 0x7f0144004d40 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.359+0000 7f015a7fc640 1 -- 172.21.15.139:0/3407078667 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154034470 con 0x7f0144004d40 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.360+0000 7f0161be0640 1 -- 172.21.15.139:0/3407078667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0144004d40 msgr2=0x7f01440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.360+0000 7f0161be0640 1 --2- 172.21.15.139:0/3407078667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0144004d40 0x7f01440051c0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f015c161520 tx=0x7f0144005830 comp rx=0 tx=0).stop 2024-04-25T10:03:58.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.360+0000 7f015bfff640 1 -- 172.21.15.139:0/3407078667 reap_dead start 2024-04-25T10:03:58.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.360+0000 7f0161be0640 1 -- 172.21.15.139:0/3407078667 shutdown_connections 2024-04-25T10:03:58.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.360+0000 7f0161be0640 1 -- 172.21.15.139:0/3407078667 >> 172.21.15.139:0/3407078667 conn(0x7f015c0b23d0 msgr2=0x7f015c0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.360+0000 7f0161be0640 1 -- 172.21.15.139:0/3407078667 shutdown_connections 2024-04-25T10:03:58.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.360+0000 7f0161be0640 1 -- 172.21.15.139:0/3407078667 wait complete. 2024-04-25T10:03:58.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.361+0000 7f0161be0640 1 Processor -- start 2024-04-25T10:03:58.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.361+0000 7f0161be0640 1 -- start start 2024-04-25T10:03:58.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f0161be0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144004d40 0x7f015c1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f0161be0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c145940 0x7f015c156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f0161be0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f015c156e80 0x7f015c0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f0161be0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c162420 con 0x7f015c156e80 2024-04-25T10:03:58.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f0161be0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c162140 con 0x7f0144004d40 2024-04-25T10:03:58.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f0161be0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c15df80 con 0x7f015c145940 2024-04-25T10:03:58.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f015bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f015c156e80 0x7f015c0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f015bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f015c156e80 0x7f015c0ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54412/0 (socket says 172.21.15.139:54412) 2024-04-25T10:03:58.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f015bfff640 1 -- 172.21.15.139:0/1359619298 learned_addr learned my addr 172.21.15.139:0/1359619298 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:58.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f015b7fe640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144004d40 0x7f015c1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f015affd640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c145940 0x7f015c156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f015bfff640 1 -- 172.21.15.139:0/1359619298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c145940 msgr2=0x7f015c156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f015bfff640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c145940 0x7f015c156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f015bfff640 1 -- 172.21.15.139:0/1359619298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144004d40 msgr2=0x7f015c1595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f015bfff640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144004d40 0x7f015c1595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.362+0000 7f015bfff640 1 -- 172.21.15.139:0/1359619298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c158830 con 0x7f015c156e80 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.363+0000 7f015bfff640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f015c156e80 0x7f015c0ae700 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f0154000f80 tx=0x7f015402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.363+0000 7f0143fff640 1 -- 172.21.15.139:0/1359619298 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015400a650 con 0x7f015c156e80 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.363+0000 7f0143fff640 1 -- 172.21.15.139:0/1359619298 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0154004d60 con 0x7f015c156e80 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.363+0000 7f0161be0640 1 -- 172.21.15.139:0/1359619298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c158b10 con 0x7f015c156e80 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.363+0000 7f0143fff640 1 -- 172.21.15.139:0/1359619298 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154034670 con 0x7f015c156e80 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.363+0000 7f0161be0640 1 -- 172.21.15.139:0/1359619298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c1590d0 con 0x7f015c156e80 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.364+0000 7f0161be0640 1 -- 172.21.15.139:0/1359619298 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c145d20 con 0x7f015c156e80 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.364+0000 7f0143fff640 1 -- 172.21.15.139:0/1359619298 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0154034810 con 0x7f015c156e80 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.365+0000 7f0143fff640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f014806b4d0 0x7f014806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.365+0000 7f0143fff640 1 -- 172.21.15.139:0/1359619298 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f01540a9ff0 con 0x7f015c156e80 2024-04-25T10:03:58.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.365+0000 7f015b7fe640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f014806b4d0 0x7f014806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.365+0000 7f015b7fe640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f014806b4d0 0x7f014806d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f014c009bc0 tx=0x7f014c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.369+0000 7f0143fff640 1 -- 172.21.15.139:0/1359619298 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f015c0b3e10 con 0x7f015c156e80 2024-04-25T10:03:58.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.536+0000 7f0161be0640 1 -- 172.21.15.139:0/1359619298 --> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f015c157300 con 0x7f014806b4d0 2024-04-25T10:03:58.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.537+0000 7f0143fff640 1 -- 172.21.15.139:0/1359619298 <== mgr.4099 v2:172.21.15.139:6832/839667050 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13379 (secure 0 0 0) 0x7f015c157300 con 0x7f014806b4d0 2024-04-25T10:03:58.542 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:58.542 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.539+0000 7f0161be0640 1 -- 172.21.15.139:0/1359619298 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f014806b4d0 msgr2=0x7f014806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.539+0000 7f0161be0640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f014806b4d0 0x7f014806d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f014c009bc0 tx=0x7f014c009190 comp rx=0 tx=0).stop 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.539+0000 7f0161be0640 1 -- 172.21.15.139:0/1359619298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f015c156e80 msgr2=0x7f015c0ae700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.539+0000 7f0161be0640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f015c156e80 0x7f015c0ae700 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f0154000f80 tx=0x7f015402d070 comp rx=0 tx=0).stop 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.539+0000 7f0161be0640 1 -- 172.21.15.139:0/1359619298 shutdown_connections 2024-04-25T10:03:58.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.539+0000 7f0161be0640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f015c156e80 0x7f015c0ae700 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.539+0000 7f0161be0640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7f014806b4d0 0x7f014806d990 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.539+0000 7f0161be0640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f015c145940 0x7f015c156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.539+0000 7f0161be0640 1 --2- 172.21.15.139:0/1359619298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144004d40 0x7f015c1595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.539+0000 7f0161be0640 1 -- 172.21.15.139:0/1359619298 >> 172.21.15.139:0/1359619298 conn(0x7f015c0b23d0 msgr2=0x7f015c0b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.539+0000 7f0161be0640 1 -- 172.21.15.139:0/1359619298 shutdown_connections 2024-04-25T10:03:58.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.539+0000 7f0161be0640 1 -- 172.21.15.139:0/1359619298 wait complete. 2024-04-25T10:03:58.555 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-04-25T10:03:57.114020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262840,"kb_used_data":13384,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711880,"statfs":{"total":773094113280,"available":772824965120,"internally_reserved":0,"allocated":13705216,"data_stored":7023600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002524"},"pg_stats":[{"pgid":"1.0","version":"125'119","reported_seq":377,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.827721+0000","last_change":"2024-04-25T10:03:38.827753+0000","last_active":"2024-04-25T10:03:38.827721+0000","last_peered":"2024-04-25T10:03:38.827721+0000","last_clean":"2024-04-25T10:03:38.827721+0000","last_became_active":"2024-04-25T09:53:04.104699+0000","last_became_peered":"2024-04-25T09:53:04.104699+0000","last_unstale":"2024-04-25T10:03:38.827721+0000","last_undegraded":"2024-04-25T10:03:38.827721+0000","last_fullsized":"2024-04-25T10:03:38.827721+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"125'119","last_scrub_stamp":"2024-04-25T10:03:38.827625+0000","last_deep_scrub":"125'119","last_deep_scrub_stamp":"2024-04-25T10:03:38.827625+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.827625+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:59.601077+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33284,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2158592,"data_stored":1320726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32712,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":730358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33284,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2158592,"data_stored":1320726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:58.556 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:03:58.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.646+0000 7fd64955c640 1 Processor -- start 2024-04-25T10:03:58.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.646+0000 7fd64955c640 1 -- start start 2024-04-25T10:03:58.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd64955c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd6440d0a80 0x7fd6441618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd64955c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd64409ad40 0x7fd644161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd64955c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd644099ca0 con 0x7fd6440cd610 2024-04-25T10:03:58.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd64955c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd644099a50 con 0x7fd6440d0a80 2024-04-25T10:03:58.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd64955c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd644099f80 con 0x7fd64409ad40 2024-04-25T10:03:58.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd6427fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd6440d0a80 0x7fd6441618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd642ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd6440cd610 0x7fd6440d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:40474/0 (socket says 172.21.15.139:40474) 2024-04-25T10:03:58.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd642ffd640 1 -- 172.21.15.139:0/2515680055 learned_addr learned my addr 172.21.15.139:0/2515680055 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-04-25T10:03:58.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd6427fc640 1 -- 172.21.15.139:0/2515680055 >> v2:172.21.15.150:3300/0 conn(0x7fd64409ad40 msgr2=0x7fd644161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd6427fc640 1 --2- 172.21.15.139:0/2515680055 >> v2:172.21.15.150:3300/0 conn(0x7fd64409ad40 0x7fd644161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd6427fc640 1 -- 172.21.15.139:0/2515680055 >> v1:172.21.15.139:6789/0 conn(0x7fd6440cd610 legacy=0x7fd6440d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd641ffb640 1 -- 172.21.15.139:0/2515680055 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880184922 0 0) 0x7fd644099ca0 con 0x7fd6440cd610 2024-04-25T10:03:58.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.647+0000 7fd6427fc640 1 -- 172.21.15.139:0/2515680055 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64409a260 con 0x7fd6440d0a80 2024-04-25T10:03:58.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.648+0000 7fd6427fc640 1 --2- 172.21.15.139:0/2515680055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440d0a80 0x7fd6441618c0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fd6440c51e0 tx=0x7fd63802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9843d298b9046a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.648+0000 7fd641ffb640 1 -- 172.21.15.139:0/2515680055 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638037070 con 0x7fd6440d0a80 2024-04-25T10:03:58.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.648+0000 7fd641ffb640 1 -- 172.21.15.139:0/2515680055 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd638032440 con 0x7fd6440d0a80 2024-04-25T10:03:58.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.648+0000 7fd641ffb640 1 -- 172.21.15.139:0/2515680055 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638032810 con 0x7fd6440d0a80 2024-04-25T10:03:58.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.649+0000 7fd64955c640 1 -- 172.21.15.139:0/2515680055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440d0a80 msgr2=0x7fd6441618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.649+0000 7fd64955c640 1 --2- 172.21.15.139:0/2515680055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440d0a80 0x7fd6441618c0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fd6440c51e0 tx=0x7fd63802a230 comp rx=0 tx=0).stop 2024-04-25T10:03:58.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.649+0000 7fd64955c640 1 -- 172.21.15.139:0/2515680055 shutdown_connections 2024-04-25T10:03:58.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.649+0000 7fd64955c640 1 --2- 172.21.15.139:0/2515680055 >> v2:172.21.15.150:3300/0 conn(0x7fd64409ad40 0x7fd644161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.649+0000 7fd64955c640 1 --2- 172.21.15.139:0/2515680055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6440d0a80 0x7fd6441618c0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.649+0000 7fd64955c640 1 -- 172.21.15.139:0/2515680055 >> 172.21.15.139:0/2515680055 conn(0x7fd6440b31f0 msgr2=0x7fd6440b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.649+0000 7fd64955c640 1 -- 172.21.15.139:0/2515680055 shutdown_connections 2024-04-25T10:03:58.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.649+0000 7fd64955c640 1 -- 172.21.15.139:0/2515680055 wait complete. 2024-04-25T10:03:58.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.650+0000 7fd64955c640 1 Processor -- start 2024-04-25T10:03:58.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.650+0000 7fd64955c640 1 -- start start 2024-04-25T10:03:58.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd64955c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd64409ad40 0x7fd64415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd64955c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6440cd610 0x7fd6441e7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd64955c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6440d0a80 0x7fd6441e7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd64955c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440cd9f0 con 0x7fd64409ad40 2024-04-25T10:03:58.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd64955c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64409a820 con 0x7fd6440cd610 2024-04-25T10:03:58.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd64955c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440cdb70 con 0x7fd6440d0a80 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd642ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd64409ad40 0x7fd64415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd642ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd64409ad40 0x7fd64415a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54434/0 (socket says 172.21.15.139:54434) 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd642ffd640 1 -- 172.21.15.139:0/4266369828 learned_addr learned my addr 172.21.15.139:0/4266369828 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd6437fe640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6440d0a80 0x7fd6441e7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd6427fc640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6440cd610 0x7fd6441e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd642ffd640 1 -- 172.21.15.139:0/4266369828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6440d0a80 msgr2=0x7fd6441e7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd642ffd640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6440d0a80 0x7fd6441e7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd642ffd640 1 -- 172.21.15.139:0/4266369828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6440cd610 msgr2=0x7fd6441e7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd642ffd640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6440cd610 0x7fd6441e7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd642ffd640 1 -- 172.21.15.139:0/4266369828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6441e8410 con 0x7fd64409ad40 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.651+0000 7fd642ffd640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd64409ad40 0x7fd64415a900 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fd63400c8c0 tx=0x7fd63400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.652+0000 7fd62b7fe640 1 -- 172.21.15.139:0/4266369828 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634018070 con 0x7fd64409ad40 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.652+0000 7fd62b7fe640 1 -- 172.21.15.139:0/4266369828 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63400a640 con 0x7fd64409ad40 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.652+0000 7fd64955c640 1 -- 172.21.15.139:0/4266369828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64415ae40 con 0x7fd64409ad40 2024-04-25T10:03:58.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.652+0000 7fd62b7fe640 1 -- 172.21.15.139:0/4266369828 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63400aa30 con 0x7fd64409ad40 2024-04-25T10:03:58.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.652+0000 7fd64955c640 1 -- 172.21.15.139:0/4266369828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6440b4cd0 con 0x7fd64409ad40 2024-04-25T10:03:58.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.653+0000 7fd64955c640 1 -- 172.21.15.139:0/4266369828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd644074520 con 0x7fd64409ad40 2024-04-25T10:03:58.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.653+0000 7fd62b7fe640 1 -- 172.21.15.139:0/4266369828 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd63401b070 con 0x7fd64409ad40 2024-04-25T10:03:58.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.653+0000 7fd62b7fe640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd63006b4d0 0x7fd63006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.654+0000 7fd6427fc640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd63006b4d0 0x7fd63006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.654+0000 7fd62b7fe640 1 -- 172.21.15.139:0/4266369828 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fd634089720 con 0x7fd64409ad40 2024-04-25T10:03:58.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.654+0000 7fd6427fc640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd63006b4d0 0x7fd63006d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd638034000 tx=0x7fd63802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.658+0000 7fd62b7fe640 1 -- 172.21.15.139:0/4266369828 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6340586f0 con 0x7fd64409ad40 2024-04-25T10:03:58.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.827+0000 7fd64955c640 1 -- 172.21.15.139:0/4266369828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fd6440b3f70 con 0x7fd64409ad40 2024-04-25T10:03:58.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.830+0000 7fd62b7fe640 1 -- 172.21.15.139:0/4266369828 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd634057ff0 con 0x7fd64409ad40 2024-04-25T10:03:58.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.832+0000 7fd62b7fe640 1 -- 172.21.15.139:0/4266369828 <== mon.0 v2:172.21.15.139:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd63405c140 con 0x7fd64409ad40 2024-04-25T10:03:58.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.833+0000 7fd64955c640 1 -- 172.21.15.139:0/4266369828 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd63006b4d0 msgr2=0x7fd63006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.833+0000 7fd64955c640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd63006b4d0 0x7fd63006d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd638034000 tx=0x7fd63802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:58.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.833+0000 7fd64955c640 1 -- 172.21.15.139:0/4266369828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd64409ad40 msgr2=0x7fd64415a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.833+0000 7fd64955c640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd64409ad40 0x7fd64415a900 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fd63400c8c0 tx=0x7fd63400cd80 comp rx=0 tx=0).stop 2024-04-25T10:03:58.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.833+0000 7fd64955c640 1 -- 172.21.15.139:0/4266369828 shutdown_connections 2024-04-25T10:03:58.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.833+0000 7fd64955c640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fd63006b4d0 0x7fd63006d990 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.833+0000 7fd64955c640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6440d0a80 0x7fd6441e7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.833+0000 7fd64955c640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6440cd610 0x7fd6441e7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.833+0000 7fd64955c640 1 --2- 172.21.15.139:0/4266369828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd64409ad40 0x7fd64415a900 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.833+0000 7fd64955c640 1 -- 172.21.15.139:0/4266369828 >> 172.21.15.139:0/4266369828 conn(0x7fd6440b31f0 msgr2=0x7fd6440ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.833+0000 7fd64955c640 1 -- 172.21.15.139:0/4266369828 shutdown_connections 2024-04-25T10:03:58.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.833+0000 7fd64955c640 1 -- 172.21.15.139:0/4266369828 wait complete. 2024-04-25T10:03:58.850 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:03:58.850 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:58.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.941+0000 7fef9e85d640 1 Processor -- start 2024-04-25T10:03:58.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.941+0000 7fef9e85d640 1 -- start start 2024-04-25T10:03:58.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef9e85d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fef980b0210 0x7fef980b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef9e85d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef9815ae40 0x7fef980b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef9e85d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef980b0b30 con 0x7fef980cefe0 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef9e85d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef980b1070 con 0x7fef980b0210 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef9e85d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef980cdff0 con 0x7fef9815ae40 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef977fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef9815ae40 0x7fef980b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef97fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fef980b0210 0x7fef980b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef977fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef9815ae40 0x7fef980b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.139:47552/0 (socket says 172.21.15.139:47552) 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef97fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fef980b0210 0x7fef980b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.139:59234/0 (socket says 172.21.15.139:59234) 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef9cdfa640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fef980cefe0 0x7fef9806d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.139:38368/0 (socket says 172.21.15.139:38368) 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef977fe640 1 -- 172.21.15.139:0/1604874969 learned_addr learned my addr 172.21.15.139:0/1604874969 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef977fe640 1 -- 172.21.15.139:0/1604874969 >> v2:172.21.15.150:3300/0 conn(0x7fef980b0210 msgr2=0x7fef980b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef977fe640 1 --2- 172.21.15.139:0/1604874969 >> v2:172.21.15.150:3300/0 conn(0x7fef980b0210 0x7fef980b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef977fe640 1 -- 172.21.15.139:0/1604874969 >> v1:172.21.15.150:6789/0 conn(0x7fef980cefe0 legacy=0x7fef9806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef977fe640 1 -- 172.21.15.139:0/1604874969 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef980ce220 con 0x7fef9815ae40 2024-04-25T10:03:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.942+0000 7fef97fff640 1 --2- 172.21.15.139:0/1604874969 >> v2:172.21.15.150:3300/0 conn(0x7fef980b0210 0x7fef980b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:58.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.943+0000 7fef977fe640 1 --2- 172.21.15.139:0/1604874969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef9815ae40 0x7fef980b05f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef8800bb10 tx=0x7fef8802c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b52cc85a49e674 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.943+0000 7fef96ffd640 1 -- 172.21.15.139:0/1604874969 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88036020 con 0x7fef9815ae40 2024-04-25T10:03:58.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.943+0000 7fef96ffd640 1 -- 172.21.15.139:0/1604874969 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fef88032820 con 0x7fef9815ae40 2024-04-25T10:03:58.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.943+0000 7fef96ffd640 1 -- 172.21.15.139:0/1604874969 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88032c10 con 0x7fef9815ae40 2024-04-25T10:03:58.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.944+0000 7fef9e85d640 1 -- 172.21.15.139:0/1604874969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef9815ae40 msgr2=0x7fef980b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.944+0000 7fef9e85d640 1 --2- 172.21.15.139:0/1604874969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef9815ae40 0x7fef980b05f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef8800bb10 tx=0x7fef8802c920 comp rx=0 tx=0).stop 2024-04-25T10:03:58.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.944+0000 7fef9e85d640 1 -- 172.21.15.139:0/1604874969 shutdown_connections 2024-04-25T10:03:58.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.944+0000 7fef9e85d640 1 --2- 172.21.15.139:0/1604874969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef9815ae40 0x7fef980b05f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.944+0000 7fef9e85d640 1 --2- 172.21.15.139:0/1604874969 >> v2:172.21.15.150:3300/0 conn(0x7fef980b0210 0x7fef980b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.944+0000 7fef9e85d640 1 -- 172.21.15.139:0/1604874969 >> 172.21.15.139:0/1604874969 conn(0x7fef98066f20 msgr2=0x7fef9806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.944+0000 7fef9e85d640 1 -- 172.21.15.139:0/1604874969 shutdown_connections 2024-04-25T10:03:58.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.944+0000 7fef9e85d640 1 -- 172.21.15.139:0/1604874969 wait complete. 2024-04-25T10:03:58.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.945+0000 7fef9e85d640 1 Processor -- start 2024-04-25T10:03:58.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.945+0000 7fef9e85d640 1 -- start start 2024-04-25T10:03:58.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef9e85d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef980b0210 0x7fef9813f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef9e85d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef980cefe0 0x7fef98156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef9e85d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef9815ae40 0x7fef981572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef9e85d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9815b3e0 con 0x7fef980cefe0 2024-04-25T10:03:58.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef9e85d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef980b1ea0 con 0x7fef9815ae40 2024-04-25T10:03:58.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef9e85d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef980ceb50 con 0x7fef980b0210 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef977fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef980cefe0 0x7fef98156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef977fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef980cefe0 0x7fef98156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54440/0 (socket says 172.21.15.139:54440) 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef977fe640 1 -- 172.21.15.139:0/3125693490 learned_addr learned my addr 172.21.15.139:0/3125693490 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef9cdfa640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef9815ae40 0x7fef981572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef97fff640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef980b0210 0x7fef9813f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef977fe640 1 -- 172.21.15.139:0/3125693490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef980b0210 msgr2=0x7fef9813f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef977fe640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef980b0210 0x7fef9813f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef977fe640 1 -- 172.21.15.139:0/3125693490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef9815ae40 msgr2=0x7fef981572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef977fe640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef9815ae40 0x7fef981572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.946+0000 7fef977fe640 1 -- 172.21.15.139:0/3125693490 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef98154ce0 con 0x7fef980cefe0 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.947+0000 7fef977fe640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef980cefe0 0x7fef98156d70 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fef8802c960 tx=0x7fef880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.947+0000 7fef94ff9640 1 -- 172.21.15.139:0/3125693490 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88036040 con 0x7fef980cefe0 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.947+0000 7fef94ff9640 1 -- 172.21.15.139:0/3125693490 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fef8803bce0 con 0x7fef980cefe0 2024-04-25T10:03:58.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.947+0000 7fef9e85d640 1 -- 172.21.15.139:0/3125693490 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef98154fc0 con 0x7fef980cefe0 2024-04-25T10:03:58.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.947+0000 7fef94ff9640 1 -- 172.21.15.139:0/3125693490 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88045650 con 0x7fef980cefe0 2024-04-25T10:03:58.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.947+0000 7fef9e85d640 1 -- 172.21.15.139:0/3125693490 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef98155580 con 0x7fef980cefe0 2024-04-25T10:03:58.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.948+0000 7fef9e85d640 1 -- 172.21.15.139:0/3125693490 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef98074520 con 0x7fef980cefe0 2024-04-25T10:03:58.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.948+0000 7fef94ff9640 1 -- 172.21.15.139:0/3125693490 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef8804c060 con 0x7fef980cefe0 2024-04-25T10:03:58.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.949+0000 7fef94ff9640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef8406b4d0 0x7fef8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.949+0000 7fef94ff9640 1 -- 172.21.15.139:0/3125693490 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fef880ad120 con 0x7fef980cefe0 2024-04-25T10:03:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.949+0000 7fef97fff640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef8406b4d0 0x7fef8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.949+0000 7fef97fff640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef8406b4d0 0x7fef8406d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009560 tx=0x7fef8c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:58.953+0000 7fef94ff9640 1 -- 172.21.15.139:0/3125693490 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef88040070 con 0x7fef980cefe0 2024-04-25T10:03:59.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.157+0000 7fef9e85d640 1 -- 172.21.15.139:0/3125693490 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef980cca70 con 0x7fef980cefe0 2024-04-25T10:03:59.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.158+0000 7fef94ff9640 1 -- 172.21.15.139:0/3125693490 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fef8802f070 con 0x7fef980cefe0 2024-04-25T10:03:59.164 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T10:03:59.164 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":98,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/535562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":535562290},{"type":"v1","addr":"172.21.15.150:6834","nonce":535562290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":7690,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3195407462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3195407462},{"type":"v1","addr":"172.21.15.150:6835","nonce":3195407462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9114,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3437700493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6835","nonce":3437700493},{"type":"v1","addr":"172.21.15.139:6837","nonce":3437700493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9774,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6836/2946921152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2946921152},{"type":"v1","addr":"172.21.15.139:6836","nonce":2946921152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[]} 2024-04-25T10:03:59.164 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 98 2024-04-25T10:03:59.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.160+0000 7fef9e85d640 1 -- 172.21.15.139:0/3125693490 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef8406b4d0 msgr2=0x7fef8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.160+0000 7fef9e85d640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef8406b4d0 0x7fef8406d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009560 tx=0x7fef8c009190 comp rx=0 tx=0).stop 2024-04-25T10:03:59.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.160+0000 7fef9e85d640 1 -- 172.21.15.139:0/3125693490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef980cefe0 msgr2=0x7fef98156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.160+0000 7fef9e85d640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef980cefe0 0x7fef98156d70 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fef8802c960 tx=0x7fef880046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.160+0000 7fef9e85d640 1 -- 172.21.15.139:0/3125693490 shutdown_connections 2024-04-25T10:03:59.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.160+0000 7fef9e85d640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef980cefe0 0x7fef98156d70 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.160+0000 7fef9e85d640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.139:6832/839667050,v1:172.21.15.139:6833/839667050] conn(0x7fef8406b4d0 0x7fef8406d990 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.160+0000 7fef9e85d640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef9815ae40 0x7fef981572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.160+0000 7fef9e85d640 1 --2- 172.21.15.139:0/3125693490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef980b0210 0x7fef9813f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.160+0000 7fef9e85d640 1 -- 172.21.15.139:0/3125693490 >> 172.21.15.139:0/3125693490 conn(0x7fef98066f20 msgr2=0x7fef9815a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.160+0000 7fef9e85d640 1 -- 172.21.15.139:0/3125693490 shutdown_connections 2024-04-25T10:03:59.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T10:03:59.160+0000 7fef9e85d640 1 -- 172.21.15.139:0/3125693490 wait complete. 2024-04-25T10:03:59.177 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:03:59.177 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:03:59.178 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:03:59.178 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.186 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:03:59.187 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:03:59.187 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.236 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:03:59.236 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:03:59.236 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.246 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:03:59.246 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:03:59.246 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.296 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:03:59.296 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:03:59.296 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:03:59.296 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.324 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:03:59.324 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:03:59.324 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.398 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:03:59.398 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:03:59.398 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.468 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:03:59.469 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:03:59.469 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.536 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:03:59.537 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:03:59.537 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.572 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:03:59.573 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:03:59.573 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.641 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:03:59.642 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:03:59.642 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.709 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:03:59.709 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:03:59.709 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.777 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:03:59.777 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:03:59.778 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.803 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:03:59.803 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:03:59.803 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.845 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:03:59.845 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:03:59.845 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:03:59.845 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.856 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:03:59.856 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:03:59.856 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.896 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:03:59.896 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:03:59.896 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:59.947 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:03:59.947 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:03:59.947 DEBUG:teuthology.orchestra.run.smithi139:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-25T10:03:59.984 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:03:59.984 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:04:00.221 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:04:00.222 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:04:00.279 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:04:00.279 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:04:00.327 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:04:00.327 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:04:00.386 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T10:04:00.386 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:04:00.559 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T10:04:00.560 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:04:00.671 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T10:04:00.672 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:04:00.722 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T10:04:00.722 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:04:00.787 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:04:00.788 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032/data/mon.a.tgz 2024-04-25T10:04:00.789 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-04-25T10:04:00.806 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.L5HzdJszsG 2024-04-25T10:04:00.807 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.L5HzdJszsG 2024-04-25T10:04:00.955 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.L5HzdJszsG 2024-04-25T10:04:01.038 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.L5HzdJszsG is 398KB 2024-04-25T10:04:01.107 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.L5HzdJszsG 2024-04-25T10:04:01.122 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032/data/mon.b.tgz 2024-04-25T10:04:01.122 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-04-25T10:04:01.140 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.DBCDvHbYfK 2024-04-25T10:04:01.140 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.DBCDvHbYfK 2024-04-25T10:04:01.300 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.DBCDvHbYfK 2024-04-25T10:04:01.383 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.DBCDvHbYfK is 402KB 2024-04-25T10:04:01.480 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.DBCDvHbYfK 2024-04-25T10:04:01.494 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032/data/mon.c.tgz 2024-04-25T10:04:01.494 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-04-25T10:04:01.548 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.ozQs3G5Mfl 2024-04-25T10:04:01.548 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ozQs3G5Mfl 2024-04-25T10:04:01.683 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.ozQs3G5Mfl 2024-04-25T10:04:01.763 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.ozQs3G5Mfl is 402KB 2024-04-25T10:04:01.860 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.ozQs3G5Mfl 2024-04-25T10:04:01.875 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:04:01.875 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:04:01.877 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:04:01.956 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:04:01.992 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:04:01.992 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:04:01.994 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032/remote/smithi139/crash 2024-04-25T10:04:01.995 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:04:02.020 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032/remote/smithi150/crash 2024-04-25T10:04:02.021 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:04:02.055 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:04:02.056 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:02.065 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:02.143 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:02.167 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:02.191 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:04:02.191 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:04:02.210 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:04:02.230 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63232.log 2024-04-25T10:04:02.230 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65507.log 2024-04-25T10:04:02.231 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69487.log 2024-04-25T10:04:02.288 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34491.log 2024-04-25T10:04:02.288 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:04:02.288 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34491.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:04:02.288 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34491.log.gz 2024-04-25T10:04:02.288 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:04:02.288 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:04:02.288 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34742.log 2024-04-25T10:04:02.288 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:04:02.288 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34742.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:04:02.288 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.5.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34742.log.gz 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34782.log 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34782.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34782.log.gz 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34662.log 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-mgr.x.log: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40658.log 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34662.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34662.log.gz 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr: 92.3%gzip -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr: 93.7%/var/log/ceph/ceph-client.1.40658.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40658.log.gz 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34702.log 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34531.log 2024-04-25T10:04:02.289 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34571.log 2024-04-25T10:04:02.290 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34702.log.gz 2024-04-25T10:04:02.290 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34531.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34531.log.gz 2024-04-25T10:04:02.290 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:04:02.290 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34571.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:04:02.290 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34571.log.gz 2024-04-25T10:04:02.290 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mds.d.log: 94.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:04:02.290 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41047.log 2024-04-25T10:04:02.290 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34615.log 2024-04-25T10:04:02.290 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.3.41047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41047.log.gz 2024-04-25T10:04:02.305 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.34615.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34615.log.gz 2024-04-25T10:04:02.446 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:04:02.527 INFO:teuthology.orchestra.run.smithi150.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:04:02.601 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:04:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73789.log 2024-04-25T10:04:02.626 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63232.log.gz 2024-04-25T10:04:02.626 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69487.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.71847.log 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65507.log.gz 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69403.log 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73789.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69487.log.gz 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.71847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40644.log 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73789.log.gz 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65549.log 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71847.log.gz 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40644.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69403.log.gz 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77616.log 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7%/var/log/ceph/ceph-client.admin.65549.log: -- replaced with /var/log/ceph/ceph-client.admin.40644.log.gz 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54538.log 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65549.log.gz 2024-04-25T10:04:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.77616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59834.log 2024-04-25T10:04:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46693.log 2024-04-25T10:04:02.628 INFO:teuthology.orchestra.run.smithi139.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.77616.log.gz 2024-04-25T10:04:02.628 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54538.log.gz 2024-04-25T10:04:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53874.log 2024-04-25T10:04:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65430.log 2024-04-25T10:04:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53874.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46693.log.gz 2024-04-25T10:04:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62920.log 2024-04-25T10:04:02.628 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.59834.log.gz 2024-04-25T10:04:02.628 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54934.log 2024-04-25T10:04:02.628 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65430.log.gz 2024-04-25T10:04:02.629 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62920.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53874.log.gz 2024-04-25T10:04:02.629 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77738.log 2024-04-25T10:04:02.629 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54934.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62920.log.gz 2024-04-25T10:04:02.629 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52343.log 2024-04-25T10:04:02.629 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.77738.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54934.log.gz 2024-04-25T10:04:02.629 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41918.log 2024-04-25T10:04:02.629 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52343.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77738.log.gz 2024-04-25T10:04:02.629 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61537.log 2024-04-25T10:04:02.629 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52343.log.gz 2024-04-25T10:04:02.629 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68039.log 2024-04-25T10:04:02.629 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67510.log 2024-04-25T10:04:02.630 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41918.log.gz 2024-04-25T10:04:02.630 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54454.log 2024-04-25T10:04:02.630 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66289.log 2024-04-25T10:04:02.630 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61537.log.gz 2024-04-25T10:04:02.630 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54454.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67510.log.gz 2024-04-25T10:04:02.630 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61987.log 2024-04-25T10:04:02.630 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.68039.log.gz 2024-04-25T10:04:02.630 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66289.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54454.log.gz 2024-04-25T10:04:02.630 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48106.log 2024-04-25T10:04:02.630 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70136.log 2024-04-25T10:04:02.630 INFO:teuthology.orchestra.run.smithi139.stderr: 91.3%/var/log/ceph/ceph-client.admin.48106.log: -- replaced with /var/log/ceph/ceph-client.admin.66289.log.gz 2024-04-25T10:04:02.631 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61065.log 2024-04-25T10:04:02.631 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48106.log.gz 2024-04-25T10:04:02.631 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59158.log 2024-04-25T10:04:02.631 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.61987.log.gz 2024-04-25T10:04:02.631 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.61065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59624.log 2024-04-25T10:04:02.631 INFO:teuthology.orchestra.run.smithi139.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61065.log.gz 2024-04-25T10:04:02.631 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70136.log.gz 2024-04-25T10:04:02.631 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47764.log 2024-04-25T10:04:02.631 INFO:teuthology.orchestra.run.smithi139.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.59158.log.gz 2024-04-25T10:04:02.631 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63419.log 2024-04-25T10:04:02.631 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47764.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59624.log.gzgzip 2024-04-25T10:04:02.631 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53150.log 2024-04-25T10:04:02.632 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63419.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.47764.log.gz -5 2024-04-25T10:04:02.632 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73495.log 2024-04-25T10:04:02.632 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63419.log.gz 2024-04-25T10:04:02.632 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48814.log 2024-04-25T10:04:02.632 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73495.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53150.log.gz 2024-04-25T10:04:02.632 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70282.log 2024-04-25T10:04:02.632 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48814.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73495.log.gz 2024-04-25T10:04:02.632 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75575.log 2024-04-25T10:04:02.632 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48814.log.gz 2024-04-25T10:04:02.632 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67638.log 2024-04-25T10:04:02.632 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70282.log: /var/log/ceph/ceph-client.admin.75575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71763.log 2024-04-25T10:04:02.632 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70282.log.gz 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70198.log 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75575.log.gz 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.71763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47113.log 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70198.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.67638.log.gz 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49987.log 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi139.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.71763.log.gz 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41688.log 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70198.log.gz 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47113.log.gz/var/log/ceph/ceph-client.admin.49987.log: 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60391.log 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49987.log.gz 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64054.log 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61621.log 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43026.log 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60391.log.gz 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64054.log.gz 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34671.log 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71805.log 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61621.log.gz 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.34671.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43026.log.gz 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi139.stderr: 92.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.41688.log.gz -5 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.80214.log 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.49945.log 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34671.log.gz 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.71805.log: /var/log/ceph/ceph-client.admin.80214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77761.log 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71805.log.gz 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57193.log 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80214.log.gz 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49945.log: /var/log/ceph/ceph-client.admin.77761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61394.log 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49945.log.gz 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40518.log 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61394.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57193.log.gz 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47806.log 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.77761.log.gz 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61394.log.gz 2024-04-25T10:04:02.635 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46147.log 2024-04-25T10:04:02.636 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40518.log.gz 2024-04-25T10:04:02.636 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43902.log 2024-04-25T10:04:02.636 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56401.log 2024-04-25T10:04:02.636 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46147.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47806.log.gz 2024-04-25T10:04:02.636 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43902.log: /var/log/ceph/ceph-client.0.76971.log 2024-04-25T10:04:02.636 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46147.log.gz 2024-04-25T10:04:02.636 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55981.log 2024-04-25T10:04:02.636 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43902.log.gz 2024-04-25T10:04:02.636 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.76971.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.56401.log.gz -5 2024-04-25T10:04:02.636 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73747.log 2024-04-25T10:04:02.636 INFO:teuthology.orchestra.run.smithi139.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76971.log.gz 2024-04-25T10:04:02.636 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51996.log 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55981.log.gz 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50239.log 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48478.log 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73747.log.gz 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50239.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.51996.log.gz 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72681.log 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42270.log 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72681.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48478.log.gz 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50239.log.gz 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64012.log 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73026.log 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0%/var/log/ceph/ceph-client.admin.64012.log: -- replaced with /var/log/ceph/ceph-client.admin.42270.log.gz 2024-04-25T10:04:02.637 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80298.log 2024-04-25T10:04:02.638 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.72681.log.gz 2024-04-25T10:04:02.638 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64012.log.gz 2024-04-25T10:04:02.638 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68297.log 2024-04-25T10:04:02.638 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71133.log 2024-04-25T10:04:02.638 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.73026.log.gz 2024-04-25T10:04:02.638 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68297.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80298.log.gz 2024-04-25T10:04:02.638 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58108.log 2024-04-25T10:04:02.638 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68297.log.gz 2024-04-25T10:04:02.638 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.71133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64688.log 2024-04-25T10:04:02.638 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34763.log 2024-04-25T10:04:02.638 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71133.log.gz 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52206.log 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58108.log.gz 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64688.log.gz 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.34763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42312.log 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52206.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34763.log.gz 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78481.log 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40602.log 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52206.log.gz 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42312.log.gz 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.78481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48436.log 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78481.log.gz 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57043.log 2024-04-25T10:04:02.639 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5/var/log/ceph/ceph-client.admin.48436.log: --verbose 89.5% -- /var/log/ceph/ceph.log -- replaced with /var/log/ceph/ceph-client.admin.40602.log.gz 2024-04-25T10:04:02.640 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.640 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63130.log 2024-04-25T10:04:02.640 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48436.log.gz 2024-04-25T10:04:02.640 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66128.log 2024-04-25T10:04:02.640 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63130.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57043.log.gz 2024-04-25T10:04:02.640 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42810.log 2024-04-25T10:04:02.640 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.66128.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63130.log.gz 2024-04-25T10:04:02.640 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64268.log 2024-04-25T10:04:02.640 INFO:teuthology.orchestra.run.smithi139.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66128.log.gz 2024-04-25T10:04:02.640 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48394.log 2024-04-25T10:04:02.640 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42810.log.gz 2024-04-25T10:04:02.640 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43566.log 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78933.log 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64268.log.gz 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43566.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48394.log.gz 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80676.log 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43566.log.gz 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.78933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71195.log 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80676.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.78933.log.gz 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76784.log 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.71195.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80676.log.gz 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48022.log 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71195.log.gz 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.76784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56868.log 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48022.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76784.log.gz 2024-04-25T10:04:02.642 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64562.log 2024-04-25T10:04:02.642 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56868.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48022.log.gzgzip 2024-04-25T10:04:02.642 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65815.log 2024-04-25T10:04:02.642 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64562.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.56868.log.gz 2024-04-25T10:04:02.642 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42558.log 2024-04-25T10:04:02.642 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65676.log 2024-04-25T10:04:02.642 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64562.log.gz 2024-04-25T10:04:02.642 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42558.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.44616.log 2024-04-25T10:04:02.642 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65815.log.gz 2024-04-25T10:04:02.642 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65676.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42558.log.gz 2024-04-25T10:04:02.642 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63629.log 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44616.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65676.log.gz 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57409.log 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44616.log.gz 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80382.log 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40560.log 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63629.log.gz 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66822.log 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40560.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80382.log.gzgzip 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68711.log 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57409.log.gz 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66822.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40560.log.gz 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:04:02.643 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68711.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66822.log.gz 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75617.log 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.2.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68711.log.gz -5 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.77505.log 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.75617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60265.log 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9%/var/log/ceph/ceph-client.admin.77505.log: -- replaced with /var/log/ceph/ceph-client.admin.75617.log.gz 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63046.log 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49384.log 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.77505.log.gz 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60265.log.gz 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72553.log 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63046.log.gz 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40834.log 2024-04-25T10:04:02.644 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72553.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49384.log.gzgzip 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64520.log 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72553.log.gz 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40164.log 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0%/var/log/ceph/ceph-client.admin.64520.log: -- replaced with /var/log/ceph/ceph-client.admin.40834.log.gz 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63671.log 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43368.log 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64520.log.gz 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40164.log.gz/var/log/ceph/ceph-client.admin.63671.log: 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55937.log 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42684.log 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63671.log.gz 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55937.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43368.log.gz 2024-04-25T10:04:02.645 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73411.log 2024-04-25T10:04:02.646 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55937.log.gz 2024-04-25T10:04:02.646 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51455.log 2024-04-25T10:04:02.646 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78294.log 2024-04-25T10:04:02.646 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42684.log.gz 2024-04-25T10:04:02.646 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51455.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73411.log.gz 2024-04-25T10:04:02.646 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53786.log 2024-04-25T10:04:02.646 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51455.log.gz 2024-04-25T10:04:02.646 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.78294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70450.log 2024-04-25T10:04:02.646 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50029.log 2024-04-25T10:04:02.646 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.78294.log.gz 2024-04-25T10:04:02.646 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59200.log 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70450.log.gz 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50029.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53786.log.gz 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44910.log 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50029.log.gz 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55672.log 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59200.log: /var/log/ceph/ceph-client.admin.44910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75449.log 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59200.log.gz 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44910.log.gz 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42044.log 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55672.log: /var/log/ceph/ceph-client.admin.75449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74698.log 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55672.log.gz 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42044.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75449.log.gz 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77385.log 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42044.log.gz 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63970.log 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.77385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73663.log 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74698.log.gz 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63970.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77385.log.gz 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79362.log 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63970.log.gz 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52164.log/var/log/ceph/ceph-client.admin.73663.log: 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50974.log 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73663.log.gz 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52164.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79362.log.gz 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57888.log 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52164.log.gz 2024-04-25T10:04:02.649 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.50974.log: /var/log/ceph/ceph-client.admin.48352.log 2024-04-25T10:04:02.649 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57888.log: gzip -5 --verbose 88.9% -- /var/log/ceph/ceph-client.admin.43034.log -- replaced with /var/log/ceph/ceph-client.admin.50974.log.gz 2024-04-25T10:04:02.649 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.649 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48352.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57888.log.gz 2024-04-25T10:04:02.649 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52631.log 2024-04-25T10:04:02.649 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48352.log.gz 2024-04-25T10:04:02.649 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49594.log 2024-04-25T10:04:02.649 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70324.log 2024-04-25T10:04:02.649 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43034.log.gz 2024-04-25T10:04:02.649 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49594.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52631.log.gz 2024-04-25T10:04:02.649 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46987.log 2024-04-25T10:04:02.649 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49594.log.gz 2024-04-25T10:04:02.650 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43703.log 2024-04-25T10:04:02.650 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46987.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:04:02.650 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70324.log.gz 2024-04-25T10:04:02.650 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43703.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46987.log.gz 2024-04-25T10:04:02.650 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74274.log 2024-04-25T10:04:02.650 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43703.log.gz 2024-04-25T10:04:02.650 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58936.log 2024-04-25T10:04:02.650 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72853.log 2024-04-25T10:04:02.650 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58936.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74274.log.gz 2024-04-25T10:04:02.650 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71889.log 2024-04-25T10:04:02.650 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58936.log.gz 2024-04-25T10:04:02.650 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49341.log 2024-04-25T10:04:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72853.log: /var/log/ceph/ceph-client.admin.71889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68879.log 2024-04-25T10:04:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49341.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71889.log.gz 2024-04-25T10:04:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67552.log 2024-04-25T10:04:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73327.log 2024-04-25T10:04:02.651 INFO:teuthology.orchestra.run.smithi139.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.72853.log.gz 2024-04-25T10:04:02.651 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5%/var/log/ceph/ceph-client.admin.68879.log: -- replaced with /var/log/ceph/ceph-client.admin.49341.log.gz 2024-04-25T10:04:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67552.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68879.log.gz 2024-04-25T10:04:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40730.log 2024-04-25T10:04:02.651 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67552.log.gz 2024-04-25T10:04:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49901.log 2024-04-25T10:04:02.651 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73327.log: /var/log/ceph/ceph-client.admin.40730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74740.log 2024-04-25T10:04:02.651 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73327.log.gz 2024-04-25T10:04:02.652 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40730.log.gz 2024-04-25T10:04:02.652 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49901.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:04:02.652 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49468.log 2024-04-25T10:04:02.652 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49901.log.gz 2024-04-25T10:04:02.652 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34399.log 2024-04-25T10:04:02.652 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74740.log: /var/log/ceph/ceph-client.admin.49468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80424.log 2024-04-25T10:04:02.652 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74740.log.gz 2024-04-25T10:04:02.652 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49468.log.gz 2024-04-25T10:04:02.652 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.34399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73579.log 2024-04-25T10:04:02.652 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43370.log 2024-04-25T10:04:02.652 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34399.log.gz 2024-04-25T10:04:02.652 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73579.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80424.log.gz 2024-04-25T10:04:02.653 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60812.log 2024-04-25T10:04:02.653 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73579.log.gz 2024-04-25T10:04:02.653 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61471.log 2024-04-25T10:04:02.653 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60812.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43370.log.gz 2024-04-25T10:04:02.653 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59540.log 2024-04-25T10:04:02.653 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65304.log 2024-04-25T10:04:02.653 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47722.log 2024-04-25T10:04:02.653 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60812.log.gz 2024-04-25T10:04:02.653 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59540.log.gz 2024-04-25T10:04:02.653 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49722.log 2024-04-25T10:04:02.653 INFO:teuthology.orchestra.run.smithi139.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61471.log.gz 2024-04-25T10:04:02.653 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62836.log 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65304.log.gz 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57301.log 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49722.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47722.log.gz 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65771.log 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49722.log.gz 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57301.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62836.log.gz 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41172.log 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57301.log.gz 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71218.log 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65771.log.gz 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80818.log 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.71218.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41172.log.gz 2024-04-25T10:04:02.654 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47071.log 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80818.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.71218.log.gzgzip 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56317.log 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47071.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80818.log.gz 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74400.log 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47071.log.gz 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44238.log 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74400.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56317.log.gz 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64880.log 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74400.log.gz 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55103.log 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64880.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44238.log.gz -5 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.74572.log 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55103.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64880.log.gz 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75760.log 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74572.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55103.log.gz 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52896.log 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.75760.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74572.log.gz 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50890.log 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75760.log.gz 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48646.log 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62095.log 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52896.log.gz 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48646.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50890.log.gz 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43697.log 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48646.log.gz 2024-04-25T10:04:02.657 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66223.log 2024-04-25T10:04:02.657 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51051.log 2024-04-25T10:04:02.657 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66223.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43697.log.gz 2024-04-25T10:04:02.657 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.62095.log.gz 2024-04-25T10:04:02.657 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44658.log 2024-04-25T10:04:02.657 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54790.log 2024-04-25T10:04:02.657 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44658.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.66223.log.gz 2024-04-25T10:04:02.657 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51051.log.gz 2024-04-25T10:04:02.657 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57343.log 2024-04-25T10:04:02.657 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54790.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44658.log.gz 2024-04-25T10:04:02.657 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63461.log 2024-04-25T10:04:02.657 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54790.log.gz 2024-04-25T10:04:02.658 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60770.log 2024-04-25T10:04:02.658 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55714.log 2024-04-25T10:04:02.658 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60770.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63461.log.gz 2024-04-25T10:04:02.658 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53659.log 2024-04-25T10:04:02.658 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57343.log.gz 2024-04-25T10:04:02.658 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60770.log.gz 2024-04-25T10:04:02.658 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48730.log 2024-04-25T10:04:02.658 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53659.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:04:02.658 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55714.log.gz 2024-04-25T10:04:02.658 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46357.log 2024-04-25T10:04:02.658 INFO:teuthology.orchestra.run.smithi139.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.53659.log.gz 2024-04-25T10:04:02.658 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.3.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48730.log.gz 2024-04-25T10:04:02.659 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40962.log 2024-04-25T10:04:02.659 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46357.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28368.log 2024-04-25T10:04:02.659 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40962.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46357.log.gz 2024-04-25T10:04:02.659 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46945.log 2024-04-25T10:04:02.659 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44952.log 2024-04-25T10:04:02.659 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph.tmp-client.admin.28368.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40962.log.gz 2024-04-25T10:04:02.659 INFO:teuthology.orchestra.run.smithi139.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28368.log.gz 2024-04-25T10:04:02.659 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46819.log 2024-04-25T10:04:02.659 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44952.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46945.log.gz 2024-04-25T10:04:02.659 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47938.log 2024-04-25T10:04:02.659 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67130.log 2024-04-25T10:04:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46819.log: /var/log/ceph/ceph-client.admin.47938.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44952.log.gz 2024-04-25T10:04:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69089.log 2024-04-25T10:04:02.660 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46819.log.gz 2024-04-25T10:04:02.660 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47938.log.gz 2024-04-25T10:04:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57846.log 2024-04-25T10:04:02.660 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67130.log.gz 2024-04-25T10:04:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57667.log 2024-04-25T10:04:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69089.log: /var/log/ceph/ceph-client.admin.57846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58276.log 2024-04-25T10:04:02.660 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69089.log.gz 2024-04-25T10:04:02.660 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57846.log.gz 2024-04-25T10:04:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80172.log 2024-04-25T10:04:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66657.log 2024-04-25T10:04:02.661 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57667.log.gz 2024-04-25T10:04:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58276.log: /var/log/ceph/ceph-client.admin.80172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53192.log 2024-04-25T10:04:02.661 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58276.log.gz 2024-04-25T10:04:02.661 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.80172.log.gz 2024-04-25T10:04:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43701.log 2024-04-25T10:04:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51809.log 2024-04-25T10:04:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43701.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53192.log.gz 2024-04-25T10:04:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52812.log 2024-04-25T10:04:02.661 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.66657.log.gz 2024-04-25T10:04:02.661 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43701.log.gz 2024-04-25T10:04:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46063.log 2024-04-25T10:04:02.661 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54192.log 2024-04-25T10:04:02.662 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52812.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51809.log.gz 2024-04-25T10:04:02.662 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66864.log 2024-04-25T10:04:02.662 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52812.log.gz 2024-04-25T10:04:02.662 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46063.log.gz 2024-04-25T10:04:02.662 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71461.log 2024-04-25T10:04:02.662 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54192.log: /var/log/ceph/ceph-client.admin.66864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41046.log 2024-04-25T10:04:02.662 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54192.log.gz 2024-04-25T10:04:02.662 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.71461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71461.log.gz 2024-04-25T10:04:02.662 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41130.log 2024-04-25T10:04:02.662 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.66864.log.gz 2024-04-25T10:04:02.662 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49678.log 2024-04-25T10:04:02.662 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68560.log 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41046.log.gz 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi139.stderr: 87.3%/var/log/ceph/ceph-client.admin.49678.log: -- replaced with /var/log/ceph/ceph-client.admin.41130.log.gz 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58402.log 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49678.log.gz 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71721.log 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68560.log: /var/log/ceph/ceph-client.admin.58402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55758.log 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68560.log.gz 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.71721.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58402.log.gz 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60631.log 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55758.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71721.log.gz 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67596.log 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55758.log.gz 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43033.log 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60631.log: /var/log/ceph/ceph-client.admin.67596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40774.log 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60631.log.gz 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43033.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67596.log.gz 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47434.log 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43033.log.gz 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51304.log 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40774.log: /var/log/ceph/ceph-client.admin.47434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58502.log 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40774.log.gz 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51304.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47434.log.gz 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46861.log 2024-04-25T10:04:02.664 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58318.log 2024-04-25T10:04:02.665 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51304.log.gz 2024-04-25T10:04:02.665 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58502.log: /var/log/ceph/ceph-client.admin.46861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68518.log 2024-04-25T10:04:02.665 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58502.log.gz 2024-04-25T10:04:02.665 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58318.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46861.log.gz 2024-04-25T10:04:02.665 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58841.log 2024-04-25T10:04:02.665 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58318.log.gz 2024-04-25T10:04:02.665 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78028.log 2024-04-25T10:04:02.665 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68518.log: /var/log/ceph/ceph-client.admin.58841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54748.log 2024-04-25T10:04:02.665 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68518.log.gz 2024-04-25T10:04:02.665 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58841.log.gz 2024-04-25T10:04:02.665 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.78028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55630.log 2024-04-25T10:04:02.666 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77827.log 2024-04-25T10:04:02.666 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54748.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.78028.log.gz 2024-04-25T10:04:02.666 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55630.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54748.log.gz 2024-04-25T10:04:02.666 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55895.log 2024-04-25T10:04:02.666 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68837.log 2024-04-25T10:04:02.666 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.77827.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55630.log.gz 2024-04-25T10:04:02.666 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69150.log 2024-04-25T10:04:02.666 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68837.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55895.log.gz 2024-04-25T10:04:02.666 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80718.log 2024-04-25T10:04:02.666 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68837.log.gz 2024-04-25T10:04:02.666 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48226.log 2024-04-25T10:04:02.666 INFO:teuthology.orchestra.run.smithi139.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.77827.log.gz 2024-04-25T10:04:02.667 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80718.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69150.log.gz 2024-04-25T10:04:02.667 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56275.log 2024-04-25T10:04:02.667 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48226.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80718.log.gz 2024-04-25T10:04:02.667 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:04:02.667 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62878.log 2024-04-25T10:04:02.667 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48226.log.gz 2024-04-25T10:04:02.667 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56275.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52248.log 2024-04-25T10:04:02.667 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56275.log.gz 2024-04-25T10:04:02.667 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41214.log 2024-04-25T10:04:02.667 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62878.log: /var/log/ceph/ceph-client.admin.52248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68921.log 2024-04-25T10:04:02.667 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62878.log.gz 2024-04-25T10:04:02.667 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41214.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52248.log.gz 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61795.log 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79591.log 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr:gzip/var/log/ceph/ceph-client.admin.61795.log: 91.1% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68921.log.gz -5 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.66756.log 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41214.log.gz 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79591.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61795.log.gz 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40686.log 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66756.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79591.log.gz 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58150.log 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70745.log 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40686.log.gz 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58150.log: gzip -5 --verbose -- 91.2% -- replaced with /var/log/ceph/ceph-client.admin.66756.log.gz 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr: /var/log/ceph/ceph-client.admin.44280.log 2024-04-25T10:04:02.668 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.70745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70745.log.gz 2024-04-25T10:04:02.669 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58150.log.gz 2024-04-25T10:04:02.669 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79866.log 2024-04-25T10:04:02.669 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59456.log 2024-04-25T10:04:02.669 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44280.log.gz 2024-04-25T10:04:02.669 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58589.log 2024-04-25T10:04:02.669 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72167.log 2024-04-25T10:04:02.669 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59456.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79866.log.gz 2024-04-25T10:04:02.669 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67973.log 2024-04-25T10:04:02.669 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59456.log.gz 2024-04-25T10:04:02.669 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69957.log 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58589.log.gz 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54664.log 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.72167.log.gz 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77259.log 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69957.log: gzip/var/log/ceph/ceph-client.admin.54664.log: -5 --verbose 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67973.log.gz 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi139.stderr: -- /var/log/ceph/ceph-client.admin.47638.log 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3%/var/log/ceph/ceph-client.admin.77259.log: -- replaced with /var/log/ceph/ceph-client.admin.69957.log.gz 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54664.log.gz 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62617.log 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77259.log.gz 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41876.log 2024-04-25T10:04:02.671 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43706.log 2024-04-25T10:04:02.671 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47638.log.gz 2024-04-25T10:04:02.671 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62617.log.gz 2024-04-25T10:04:02.671 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41876.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:04:02.671 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:04:02.671 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43706.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41876.log.gz 2024-04-25T10:04:02.671 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58024.log 2024-04-25T10:04:02.671 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43706.log.gz 2024-04-25T10:04:02.671 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62390.log 2024-04-25T10:04:02.671 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.58024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53787.log 2024-04-25T10:04:02.671 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.62390.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58024.log.gz 2024-04-25T10:04:02.671 INFO:teuthology.orchestra.run.smithi139.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62390.log.gz 2024-04-25T10:04:02.672 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57930.log 2024-04-25T10:04:02.672 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79194.log 2024-04-25T10:04:02.672 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68123.log 2024-04-25T10:04:02.672 INFO:teuthology.orchestra.run.smithi139.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.53787.log.gz 2024-04-25T10:04:02.672 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57930.log: /var/log/ceph/ceph-client.admin.79194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64310.log 2024-04-25T10:04:02.672 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68123.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57930.log.gz 2024-04-25T10:04:02.672 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79194.log.gz 2024-04-25T10:04:02.672 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73705.log 2024-04-25T10:04:02.672 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64310.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68123.log.gz 2024-04-25T10:04:02.672 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59328.log 2024-04-25T10:04:02.672 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64310.log.gz 2024-04-25T10:04:02.673 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57601.log 2024-04-25T10:04:02.673 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73705.log.gz/var/log/ceph/ceph-client.admin.59328.log: 2024-04-25T10:04:02.673 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59328.log.gz 2024-04-25T10:04:02.673 INFO:teuthology.orchestra.run.smithi139.stderr: /var/log/ceph/ceph-client.admin.50071.log 2024-04-25T10:04:02.673 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80046.log 2024-04-25T10:04:02.673 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49426.log 2024-04-25T10:04:02.673 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57601.log.gz 2024-04-25T10:04:02.673 INFO:teuthology.orchestra.run.smithi139.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50071.log.gz 2024-04-25T10:04:02.673 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60552.log 2024-04-25T10:04:02.673 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80046.log.gz 2024-04-25T10:04:02.673 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47476.log 2024-04-25T10:04:02.674 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49426.log.gz/var/log/ceph/ceph-client.admin.60552.log: 2024-04-25T10:04:02.674 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68476.log 2024-04-25T10:04:02.674 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60552.log.gz 2024-04-25T10:04:02.674 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:04:02.674 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47476.log: /var/log/ceph/ceph-client.admin.68476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68669.log 2024-04-25T10:04:02.674 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.0.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47476.log.gz 2024-04-25T10:04:02.674 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68476.log.gz 2024-04-25T10:04:02.674 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64184.log 2024-04-25T10:04:02.674 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80634.log 2024-04-25T10:04:02.674 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68669.log: /var/log/ceph/ceph-client.admin.64184.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68669.log.gz 2024-04-25T10:04:02.674 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56359.log 2024-04-25T10:04:02.675 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64184.log.gz 2024-04-25T10:04:02.675 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42432.log 2024-04-25T10:04:02.675 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56359.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80634.log.gz 2024-04-25T10:04:02.675 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56359.log.gz 2024-04-25T10:04:02.675 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78807.log 2024-04-25T10:04:02.675 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44868.log 2024-04-25T10:04:02.675 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42432.log: /var/log/ceph/ceph-client.admin.78807.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42432.log.gz 2024-04-25T10:04:02.675 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78807.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55019.log 2024-04-25T10:04:02.675 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.675 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72765.log 2024-04-25T10:04:02.675 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44868.log: /var/log/ceph/ceph-client.admin.55019.log: 88.0% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44868.log.gz 2024-04-25T10:04:02.676 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55019.log.gz 2024-04-25T10:04:02.676 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79026.log 2024-04-25T10:04:02.676 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68231.log 2024-04-25T10:04:02.676 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72765.log: /var/log/ceph/ceph-client.admin.79026.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79026.log.gz 2024-04-25T10:04:02.676 INFO:teuthology.orchestra.run.smithi139.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.72765.log.gz 2024-04-25T10:04:02.676 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60349.log 2024-04-25T10:04:02.676 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59242.log 2024-04-25T10:04:02.676 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60349.log: 91.2% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60349.log.gz 2024-04-25T10:04:02.676 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68231.log.gz 2024-04-25T10:04:02.676 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47350.log 2024-04-25T10:04:02.676 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51497.log 2024-04-25T10:04:02.676 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59242.log: /var/log/ceph/ceph-client.admin.47350.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59242.log.gz 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47350.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61729.log 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55229.log 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61729.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51497.log.gz 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54067.log 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55229.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61729.log.gz 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55229.log.gz 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73003.log 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79633.log 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi139.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.54067.log.gz/var/log/ceph/ceph-client.admin.73003.log: 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41960.log 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79633.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73003.log.gz 2024-04-25T10:04:02.678 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54622.log 2024-04-25T10:04:02.678 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79633.log.gz 2024-04-25T10:04:02.678 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51193.log 2024-04-25T10:04:02.678 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54622.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41960.log.gz 2024-04-25T10:04:02.678 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54622.log.gz 2024-04-25T10:04:02.678 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42474.log 2024-04-25T10:04:02.678 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70052.log 2024-04-25T10:04:02.678 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51193.log: /var/log/ceph/ceph-client.admin.42474.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51193.log.gz 2024-04-25T10:04:02.678 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42474.log.gz 2024-04-25T10:04:02.678 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51749.log 2024-04-25T10:04:02.678 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78765.log 2024-04-25T10:04:02.678 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70052.log: /var/log/ceph/ceph-client.admin.51749.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70052.log.gz 2024-04-25T10:04:02.679 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.51749.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.58234.log 2024-04-25T10:04:02.679 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.679 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73200.log 2024-04-25T10:04:02.679 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.78765.log: /var/log/ceph/ceph-client.admin.58234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60575.log 2024-04-25T10:04:02.679 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78765.log.gz 2024-04-25T10:04:02.679 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73200.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58234.log.gz 2024-04-25T10:04:02.679 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73200.log.gz 2024-04-25T10:04:02.679 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73369.log 2024-04-25T10:04:02.679 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.60575.log: -- /var/log/ceph/ceph-client.admin.74530.log 2024-04-25T10:04:02.679 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73369.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.60575.log.gz 2024-04-25T10:04:02.679 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54580.log 2024-04-25T10:04:02.680 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74530.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73369.log.gz 2024-04-25T10:04:02.680 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67088.log 2024-04-25T10:04:02.680 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74530.log.gz 2024-04-25T10:04:02.680 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54580.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54580.log.gz 2024-04-25T10:04:02.680 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47392.log 2024-04-25T10:04:02.680 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46609.log 2024-04-25T10:04:02.680 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67088.log.gz 2024-04-25T10:04:02.680 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80088.log 2024-04-25T10:04:02.680 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2%/var/log/ceph/ceph-client.admin.46609.log: -- replaced with /var/log/ceph/ceph-client.admin.47392.log.gz 2024-04-25T10:04:02.680 INFO:teuthology.orchestra.run.smithi139.stderr:gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46609.log.gz 2024-04-25T10:04:02.680 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58547.log 2024-04-25T10:04:02.681 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52761.log 2024-04-25T10:04:02.681 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80088.log: /var/log/ceph/ceph-client.admin.58547.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80088.log.gz 2024-04-25T10:04:02.681 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58547.log.gz 2024-04-25T10:04:02.681 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79110.log 2024-04-25T10:04:02.681 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44060.log 2024-04-25T10:04:02.681 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52761.log.gz 2024-04-25T10:04:02.681 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75491.log 2024-04-25T10:04:02.681 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79110.log.gz 2024-04-25T10:04:02.681 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44062.log 2024-04-25T10:04:02.681 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44060.log.gz 2024-04-25T10:04:02.681 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.75491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72401.log 2024-04-25T10:04:02.681 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44062.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75491.log.gz 2024-04-25T10:04:02.682 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54234.log 2024-04-25T10:04:02.682 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44062.log.gz 2024-04-25T10:04:02.682 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66962.log 2024-04-25T10:04:02.682 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54234.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54234.log.gz 2024-04-25T10:04:02.682 INFO:teuthology.orchestra.run.smithi139.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.72401.log.gz 2024-04-25T10:04:02.682 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55588.log 2024-04-25T10:04:02.682 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46651.log 2024-04-25T10:04:02.682 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66962.log: /var/log/ceph/ceph-client.admin.55588.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66962.log.gz 2024-04-25T10:04:02.682 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58192.log 2024-04-25T10:04:02.682 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55588.log.gz 2024-04-25T10:04:02.682 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72080.log 2024-04-25T10:04:02.682 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58192.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46651.log.gz 2024-04-25T10:04:02.683 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58192.log.gz 2024-04-25T10:04:02.683 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47596.log 2024-04-25T10:04:02.683 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53108.log 2024-04-25T10:04:02.683 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72080.log: /var/log/ceph/ceph-client.admin.47596.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72080.log.gz 2024-04-25T10:04:02.683 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47596.log.gz 2024-04-25T10:04:02.683 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64814.log 2024-04-25T10:04:02.683 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51267.log 2024-04-25T10:04:02.683 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64814.log: /var/log/ceph/ceph-client.admin.53108.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53108.log.gz 2024-04-25T10:04:02.683 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70492.log 2024-04-25T10:04:02.683 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64814.log.gz 2024-04-25T10:04:02.683 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61226.log 2024-04-25T10:04:02.683 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70492.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51267.log.gz 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64096.log 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70492.log.gz 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40433.log 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64096.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61226.log.gz 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51413.log 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64096.log.gz 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56233.log 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51413.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40433.log.gz 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51413.log.gz 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43351.log 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41088.log 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56233.log: /var/log/ceph/ceph-client.admin.43351.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56233.log.gz 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43351.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61310.log 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69787.log 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41088.log: /var/log/ceph/ceph-client.admin.61310.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41088.log.gz 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61310.log.gz 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55271.log 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44700.log 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69787.log.gz 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70558.log 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44700.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55271.log.gz 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44700.log.gz 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47197.log 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73158.log 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70558.log.gz 2024-04-25T10:04:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46189.log 2024-04-25T10:04:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73158.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47197.log.gz 2024-04-25T10:04:02.686 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73158.log.gz 2024-04-25T10:04:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74614.log 2024-04-25T10:04:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76537.log 2024-04-25T10:04:02.686 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46189.log.gz 2024-04-25T10:04:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74656.log 2024-04-25T10:04:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.76537.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74614.log.gz 2024-04-25T10:04:02.686 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76537.log.gz 2024-04-25T10:04:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42086.log 2024-04-25T10:04:02.686 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55187.log 2024-04-25T10:04:02.686 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74656.log.gz 2024-04-25T10:04:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50305.log 2024-04-25T10:04:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55187.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42086.log.gz 2024-04-25T10:04:02.687 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55187.log.gz 2024-04-25T10:04:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43031.log 2024-04-25T10:04:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63928.log 2024-04-25T10:04:02.687 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50305.log.gz 2024-04-25T10:04:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43029.log 2024-04-25T10:04:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63928.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43031.log.gz 2024-04-25T10:04:02.687 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63928.log.gz 2024-04-25T10:04:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73957.log 2024-04-25T10:04:02.687 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48064.log 2024-04-25T10:04:02.687 INFO:teuthology.orchestra.run.smithi139.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43029.log.gz 2024-04-25T10:04:02.688 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46021.log 2024-04-25T10:04:02.688 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3%/var/log/ceph/ceph-client.admin.48064.log: -- replaced with /var/log/ceph/ceph-client.admin.73957.log.gz 2024-04-25T10:04:02.688 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48064.log.gz 2024-04-25T10:04:02.688 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72639.log 2024-04-25T10:04:02.688 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65572.log 2024-04-25T10:04:02.688 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46021.log.gz 2024-04-25T10:04:02.688 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72639.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:04:02.688 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65572.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72639.log.gz 2024-04-25T10:04:02.688 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65572.log.gz 2024-04-25T10:04:02.688 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69047.log 2024-04-25T10:04:02.688 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58631.log 2024-04-25T10:04:02.688 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69047.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69047.log.gz 2024-04-25T10:04:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47308.log 2024-04-25T10:04:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58631.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58631.log.gz 2024-04-25T10:04:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77869.log 2024-04-25T10:04:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47308.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47308.log.gz 2024-04-25T10:04:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51623.log 2024-04-25T10:04:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.77869.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.77869.log.gz 2024-04-25T10:04:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67893.log 2024-04-25T10:04:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51623.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51623.log.gz 2024-04-25T10:04:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70240.log 2024-04-25T10:04:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.67893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67893.log.gz 2024-04-25T10:04:02.689 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54276.log 2024-04-25T10:04:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70240.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70240.log.gz 2024-04-25T10:04:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52423.log 2024-04-25T10:04:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54276.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54276.log.gz 2024-04-25T10:04:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63088.log 2024-04-25T10:04:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52423.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52423.log.gz 2024-04-25T10:04:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64394.log 2024-04-25T10:04:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63088.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63088.log.gz 2024-04-25T10:04:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54024.log 2024-04-25T10:04:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64394.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64394.log.gz 2024-04-25T10:04:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63274.log 2024-04-25T10:04:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54024.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54024.log.gz 2024-04-25T10:04:02.690 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50155.log 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63274.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63274.log.gz 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45430.log 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50155.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50155.log.gz 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63334.log 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.45430.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45430.log.gz 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55546.log 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63334.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63334.log.gz 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73831.log 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55546.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55546.log.gz 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42726.log 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73831.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73831.log.gz 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79549.log 2024-04-25T10:04:02.691 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42726.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42726.log.gz 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44406.log 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79549.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79549.log.gz 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79278.log 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44406.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44406.log.gz 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46231.log 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79278.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79278.log.gz 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72499.log 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46231.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46231.log.gz 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54496.log 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72499.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72499.log.gz 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73621.log 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54496.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54496.log.gz 2024-04-25T10:04:02.692 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74488.log 2024-04-25T10:04:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73621.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73621.log.gz 2024-04-25T10:04:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53487.log 2024-04-25T10:04:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74488.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74488.log.gz 2024-04-25T10:04:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67208.log 2024-04-25T10:04:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.53487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53487.log.gz 2024-04-25T10:04:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44364.log 2024-04-25T10:04:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67208.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67208.log.gz 2024-04-25T10:04:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67270.log 2024-04-25T10:04:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44364.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44364.log.gz 2024-04-25T10:04:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69445.log 2024-04-25T10:04:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67270.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67270.log.gz 2024-04-25T10:04:02.693 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42600.log 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69445.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69445.log.gz 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45116.log 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42600.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42600.log.gz 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80466.log 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.45116.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45116.log.gz 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70930.log 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80466.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80466.log.gz 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49859.log 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70930.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70930.log.gz 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59414.log 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49859.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49859.log.gz 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52982.log 2024-04-25T10:04:02.694 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59414.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59414.log.gz 2024-04-25T10:04:02.695 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69871.log 2024-04-25T10:04:02.695 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52982.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52982.log.gz 2024-04-25T10:04:02.695 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70366.log 2024-04-25T10:04:02.695 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69871.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69871.log.gz 2024-04-25T10:04:02.695 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51851.log 2024-04-25T10:04:02.695 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70366.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70366.log.gz 2024-04-25T10:04:02.695 INFO:teuthology.orchestra.run.smithi139.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76632.log 2024-04-25T10:04:02.695 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.695 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40920.log 2024-04-25T10:04:02.695 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.76632.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51851.log.gz 2024-04-25T10:04:02.695 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76632.log.gz 2024-04-25T10:04:02.695 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70972.log 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75203.log 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40920.log.gz 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70888.log 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.75203.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70972.log.gz 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75203.log.gz 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74824.log 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79993.log 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70888.log.gz 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55802.log 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79993.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74824.log.gz 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79068.log 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79993.log.gz 2024-04-25T10:04:02.696 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55802.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55802.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44490.log 2024-04-25T10:04:02.697 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.697 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67704.log 2024-04-25T10:04:02.697 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79068.log.gz 2024-04-25T10:04:02.697 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76292.log 2024-04-25T10:04:02.697 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44490.log.gz 2024-04-25T10:04:02.697 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77462.log 2024-04-25T10:04:02.697 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.76292.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67704.log.gz 2024-04-25T10:04:02.697 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76250.log 2024-04-25T10:04:02.697 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76292.log.gz 2024-04-25T10:04:02.697 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42002.log 2024-04-25T10:04:02.697 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.77462.log: gzip -5 --verbose -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77462.log.gz 2024-04-25T10:04:02.697 INFO:teuthology.orchestra.run.smithi139.stderr: /var/log/ceph/ceph-client.admin.52547.log 2024-04-25T10:04:02.698 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.76250.log: /var/log/ceph/ceph-client.admin.42002.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76250.log.gz 2024-04-25T10:04:02.698 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42002.log.gz 2024-04-25T10:04:02.698 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56825.log 2024-04-25T10:04:02.698 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59284.log 2024-04-25T10:04:02.698 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52547.log.gz 2024-04-25T10:04:02.698 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44574.log 2024-04-25T10:04:02.698 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59284.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56825.log.gz 2024-04-25T10:04:02.698 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59284.log.gz 2024-04-25T10:04:02.698 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45891.log 2024-04-25T10:04:02.698 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52589.log 2024-04-25T10:04:02.698 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.2.45891.log: /var/log/ceph/ceph-client.admin.44574.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45891.log.gz 2024-04-25T10:04:02.698 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44574.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79908.log 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74782.log 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52589.log.gz 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49817.log 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74782.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79908.log.gz 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74782.log.gz 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64436.log 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76166.log 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49817.log.gz 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46483.log 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.76166.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64436.log.gz 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44532.log 2024-04-25T10:04:02.699 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76166.log.gz 2024-04-25T10:04:02.700 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51911.log 2024-04-25T10:04:02.700 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80256.log 2024-04-25T10:04:02.700 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46483.log.gz 2024-04-25T10:04:02.700 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44532.log: /var/log/ceph/ceph-client.admin.51911.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44532.log.gz 2024-04-25T10:04:02.700 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51911.log.gz 2024-04-25T10:04:02.700 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52717.log 2024-04-25T10:04:02.700 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40876.log 2024-04-25T10:04:02.700 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80256.log.gz 2024-04-25T10:04:02.700 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71014.log 2024-04-25T10:04:02.700 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40876.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52717.log.gz 2024-04-25T10:04:02.700 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50492.log 2024-04-25T10:04:02.700 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40876.log.gz 2024-04-25T10:04:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.71014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59372.log 2024-04-25T10:04:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.50492.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50492.log.gz 2024-04-25T10:04:02.701 INFO:teuthology.orchestra.run.smithi139.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71014.log.gz 2024-04-25T10:04:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65262.log 2024-04-25T10:04:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74444.log 2024-04-25T10:04:02.701 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59372.log.gz 2024-04-25T10:04:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69663.log 2024-04-25T10:04:02.701 INFO:teuthology.orchestra.run.smithi139.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65262.log.gz 2024-04-25T10:04:02.701 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74444.log: 91.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52505.log 2024-04-25T10:04:02.701 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:04:02.701 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74444.log.gz 2024-04-25T10:04:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43364.log 2024-04-25T10:04:02.702 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0%/var/log/ceph/ceph-client.admin.52505.log: -- replaced with /var/log/ceph/ceph-client.admin.69663.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49766.log 2024-04-25T10:04:02.702 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.702 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52505.log.gz 2024-04-25T10:04:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74358.log 2024-04-25T10:04:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43364.log: /var/log/ceph/ceph-client.admin.49766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56023.log 2024-04-25T10:04:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74358.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43364.log.gz 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49766.log.gz 2024-04-25T10:04:02.702 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74358.log.gz 2024-04-25T10:04:02.702 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47896.log 2024-04-25T10:04:02.702 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51113.log 2024-04-25T10:04:02.702 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56023.log.gz 2024-04-25T10:04:02.703 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58715.log 2024-04-25T10:04:02.703 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47896.log.gz 2024-04-25T10:04:02.703 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78849.log 2024-04-25T10:04:02.703 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51113.log.gz 2024-04-25T10:04:02.703 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42354.log 2024-04-25T10:04:02.703 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42768.log 2024-04-25T10:04:02.703 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.78849.log: 88.3%/var/log/ceph/ceph-client.admin.42354.log: -- replaced with /var/log/ceph/ceph-client.admin.58715.log.gz 2024-04-25T10:04:02.703 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78849.log.gz 2024-04-25T10:04:02.703 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42354.log.gz 2024-04-25T10:04:02.703 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61268.log 2024-04-25T10:04:02.703 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76208.log 2024-04-25T10:04:02.703 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61268.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42768.log.gz 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43362.log 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2%/var/log/ceph/ceph-client.admin.76208.log: -- replaced with /var/log/ceph/ceph-client.admin.61268.log.gz 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66441.log 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76208.log.gz 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70094.log 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43362.log.gz 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49024.log 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66441.log: /var/log/ceph/ceph-client.admin.70094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51665.log 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49024.log: 89.1% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66441.log.gz 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70094.log.gz 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49024.log.gz 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46315.log 2024-04-25T10:04:02.704 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62575.log 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51665.log.gz 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48898.log 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46315.log.gz 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76718.log 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62575.log.gz 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55145.log 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79404.log 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.76718.log: 89.0%/var/log/ceph/ceph-client.admin.55145.log: -- replaced with /var/log/ceph/ceph-client.admin.48898.log.gz 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55145.log.gz 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63172.log 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76718.log.gz 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46441.log 2024-04-25T10:04:02.705 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63172.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79404.log.gz 2024-04-25T10:04:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62743.log 2024-04-25T10:04:02.706 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7%/var/log/ceph/ceph-client.admin.46441.log: -- replaced with /var/log/ceph/ceph-client.admin.63172.log.gz 2024-04-25T10:04:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72318.log 2024-04-25T10:04:02.706 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46441.log.gz 2024-04-25T10:04:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40352.log/var/log/ceph/ceph-client.admin.62743.log: 2024-04-25T10:04:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63587.log 2024-04-25T10:04:02.706 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62743.log.gz 2024-04-25T10:04:02.706 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72318.log.gz 2024-04-25T10:04:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71274.log 2024-04-25T10:04:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44061.log 2024-04-25T10:04:02.706 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63587.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40352.log.gz 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.71274.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63587.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50806.log 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71274.log.gz 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44322.log 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44061.log: /var/log/ceph/ceph-client.admin.50806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43367.log 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44322.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44061.log.gz 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50806.log.gz 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr:gzip 88.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.43027.log 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44322.log.gz 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47518.log 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1%/var/log/ceph/ceph-client.admin.43027.log: -- replaced with /var/log/ceph/ceph-client.admin.43367.log.gz 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64352.log 2024-04-25T10:04:02.707 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43027.log.gz 2024-04-25T10:04:02.708 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.47518.log: -- /var/log/ceph/ceph-client.admin.47029.log 2024-04-25T10:04:02.708 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64352.log: 89.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.47518.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.46273.log 2024-04-25T10:04:02.708 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.708 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64352.log.gz 2024-04-25T10:04:02.708 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62659.log 2024-04-25T10:04:02.708 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79446.log 2024-04-25T10:04:02.708 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47029.log.gz 2024-04-25T10:04:02.708 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62659.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.43366.log -- replaced with /var/log/ceph/ceph-client.admin.46273.log.gz 2024-04-25T10:04:02.708 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.708 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63766.log 2024-04-25T10:04:02.708 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62659.log.gz 2024-04-25T10:04:02.709 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79446.log: /var/log/ceph/ceph-client.admin.43366.log: 88.8% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43366.log.gz -- replaced with /var/log/ceph/ceph-client.admin.79446.log.gz 2024-04-25T10:04:02.709 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.709 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77301.log 2024-04-25T10:04:02.709 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51230.log 2024-04-25T10:04:02.709 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.77301.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63766.log.gz 2024-04-25T10:04:02.709 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79506.log 2024-04-25T10:04:02.709 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77301.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60684.log 2024-04-25T10:04:02.709 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.709 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51230.log: /var/log/ceph/ceph-client.admin.79506.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.51230.log.gz 2024-04-25T10:04:02.709 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79506.log.gz 2024-04-25T10:04:02.709 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55464.log 2024-04-25T10:04:02.709 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69583.log 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55464.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60684.log.gz 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59116.log 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55464.log.gz/var/log/ceph/ceph-client.admin.69583.log: 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60087.log 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69583.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54706.log 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69193.log 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.60087.log: 88.9% 0.0% -- replaced with /var/log/ceph/ceph-client.0.60087.log.gz 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59116.log.gz 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42642.log 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54706.log.gz 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49066.log 2024-04-25T10:04:02.710 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69193.log: /var/log/ceph/ceph-client.admin.42642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54874.log 2024-04-25T10:04:02.711 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49066.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42642.log.gz 2024-04-25T10:04:02.711 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69193.log.gz 2024-04-25T10:04:02.711 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49066.log.gz 2024-04-25T10:04:02.711 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53958.log 2024-04-25T10:04:02.711 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74214.log 2024-04-25T10:04:02.711 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54874.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58360.log 2024-04-25T10:04:02.711 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.711 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53958.log: /var/log/ceph/ceph-client.admin.74214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60510.log 2024-04-25T10:04:02.711 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58360.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74214.log.gz 2024-04-25T10:04:02.711 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58360.log.gz 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53958.log.gz 2024-04-25T10:04:02.711 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.711 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79152.log 2024-04-25T10:04:02.712 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65067.log 2024-04-25T10:04:02.712 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79152.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60510.log.gz 2024-04-25T10:04:02.712 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62161.log 2024-04-25T10:04:02.712 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79152.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43025.log 2024-04-25T10:04:02.712 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.712 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.65067.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65067.log.gz 2024-04-25T10:04:02.712 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77579.log 2024-04-25T10:04:02.712 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62161.log: /var/log/ceph/ceph-client.admin.43025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57127.log 2024-04-25T10:04:02.712 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.77579.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62161.log.gz 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43025.log.gz 2024-04-25T10:04:02.712 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.712 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77579.log.gz 2024-04-25T10:04:02.712 INFO:teuthology.orchestra.run.smithi139.stderr: -- /var/log/ceph/ceph-client.admin.58799.log 2024-04-25T10:04:02.713 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75533.log 2024-04-25T10:04:02.713 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48310.log 2024-04-25T10:04:02.713 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57127.log.gz 2024-04-25T10:04:02.713 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58799.log.gz 2024-04-25T10:04:02.713 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.75533.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75533.log.gz 2024-04-25T10:04:02.713 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69913.log 2024-04-25T10:04:02.713 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44784.log 2024-04-25T10:04:02.713 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48310.log.gz 2024-04-25T10:04:02.713 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71966.log 2024-04-25T10:04:02.713 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69913.log: /var/log/ceph/ceph-client.admin.44784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68795.log 2024-04-25T10:04:02.713 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0%/var/log/ceph/ceph-client.admin.71966.log: -- replaced with /var/log/ceph/ceph-client.admin.44784.log.gz 2024-04-25T10:04:02.714 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71966.log.gz 2024-04-25T10:04:02.714 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57559.log 2024-04-25T10:04:02.714 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68795.log: -- replaced with /var/log/ceph/ceph-client.admin.69913.log.gz 2024-04-25T10:04:02.714 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74950.log 2024-04-25T10:04:02.714 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68795.log.gz 2024-04-25T10:04:02.714 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62203.log 2024-04-25T10:04:02.714 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57559.log.gz 2024-04-25T10:04:02.714 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43698.log 2024-04-25T10:04:02.714 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64772.log 2024-04-25T10:04:02.714 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.74950.log.gz 2024-04-25T10:04:02.714 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62203.log.gz 2024-04-25T10:04:02.714 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65609.log 2024-04-25T10:04:02.715 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79236.log 2024-04-25T10:04:02.715 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43698.log.gz 2024-04-25T10:04:02.715 INFO:teuthology.orchestra.run.smithi139.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64772.log.gz 2024-04-25T10:04:02.715 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60878.log 2024-04-25T10:04:02.715 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56509.log 2024-04-25T10:04:02.715 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.65609.log.gz 2024-04-25T10:04:02.715 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79236.log.gz 2024-04-25T10:04:02.715 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54360.log 2024-04-25T10:04:02.715 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47980.log 2024-04-25T10:04:02.715 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60878.log.gz 2024-04-25T10:04:02.715 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54360.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56509.log.gz 2024-04-25T10:04:02.715 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54360.log.gz 2024-04-25T10:04:02.716 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.64730.log 2024-04-25T10:04:02.716 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72595.log 2024-04-25T10:04:02.716 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47980.log.gz 2024-04-25T10:04:02.716 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66483.log 2024-04-25T10:04:02.716 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64730.log.gz 2024-04-25T10:04:02.716 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54832.log 2024-04-25T10:04:02.716 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50113.log 2024-04-25T10:04:02.716 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72595.log.gz/var/log/ceph/ceph-client.admin.66483.log: 2024-04-25T10:04:02.716 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54832.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54832.log.gz 2024-04-25T10:04:02.716 INFO:teuthology.orchestra.run.smithi139.stderr:gzip 91.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.63846.log 2024-04-25T10:04:02.716 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66483.log.gz 2024-04-25T10:04:02.716 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43369.log 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68753.log 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50113.log.gz/var/log/ceph/ceph-client.admin.63846.log: 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43369.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63846.log.gz 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75016.log 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43369.log.gz 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68753.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74132.log 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68753.log.gz 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.75016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50848.log 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44826.log 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75016.log.gz 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3%/var/log/ceph/ceph-client.admin.50848.log: -- replaced with /var/log/ceph/ceph-client.admin.74132.log.gz 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60307.log 2024-04-25T10:04:02.717 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50848.log.gz 2024-04-25T10:04:02.718 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48982.log 2024-04-25T10:04:02.718 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78891.log 2024-04-25T10:04:02.718 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44826.log.gz 2024-04-25T10:04:02.718 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60307.log: /var/log/ceph/ceph-client.admin.48982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44742.log 2024-04-25T10:04:02.718 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60307.log.gz/var/log/ceph/ceph-client.admin.78891.log: 2024-04-25T10:04:02.718 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48982.log.gz 2024-04-25T10:04:02.718 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67395.log 2024-04-25T10:04:02.718 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78891.log.gz 2024-04-25T10:04:02.718 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56065.log 2024-04-25T10:04:02.718 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55853.log 2024-04-25T10:04:02.718 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44742.log.gz 2024-04-25T10:04:02.718 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67395.log: /var/log/ceph/ceph-client.admin.56065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42186.log 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55853.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56065.log.gz 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76124.log 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55853.log.gz 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42186.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42186.log.gz 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80130.log 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.76124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42144.log 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% 92.2% -- replaced with /var/log/ceph/ceph-client.admin.67395.log.gz 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76124.log.gz 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66375.log 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80130.log.gz 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80340.log 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42144.log.gz 2024-04-25T10:04:02.719 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48268.log 2024-04-25T10:04:02.720 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77217.log 2024-04-25T10:04:02.720 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80340.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.66375.log.gz 2024-04-25T10:04:02.720 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48268.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80340.log.gz 2024-04-25T10:04:02.720 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65923.log 89.0% 2024-04-25T10:04:02.720 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48268.log.gz 2024-04-25T10:04:02.720 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.77217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53234.log 2024-04-25T10:04:02.720 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77217.log.gz 2024-04-25T10:04:02.720 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54318.log 2024-04-25T10:04:02.720 INFO:teuthology.orchestra.run.smithi139.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.65923.log.gz 2024-04-25T10:04:02.720 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67004.log 2024-04-25T10:04:02.720 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77672.log 2024-04-25T10:04:02.720 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54318.log: 90.9%/var/log/ceph/ceph-client.admin.67004.log: -- replaced with /var/log/ceph/ceph-client.admin.53234.log.gz 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54318.log.gz 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52940.log 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67004.log.gz 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.77672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62962.log 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49256.log 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52940.log: 91.3%/var/log/ceph/ceph-client.admin.62962.log: -- replaced with /var/log/ceph/ceph-client.admin.77672.log.gz 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52940.log.gz 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62962.log.gz 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48940.log 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44448.log 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49256.log.gz 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58066.log 2024-04-25T10:04:02.721 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48940.log.gz 2024-04-25T10:04:02.722 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49146.log 2024-04-25T10:04:02.722 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44448.log.gz 2024-04-25T10:04:02.722 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41280.log 2024-04-25T10:04:02.722 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59666.log 2024-04-25T10:04:02.722 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7%/var/log/ceph/ceph-client.admin.49146.log: -- replaced with /var/log/ceph/ceph-client.admin.58066.log.gz 2024-04-25T10:04:02.722 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76590.log 2024-04-25T10:04:02.722 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49146.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41280.log.gz 2024-04-25T10:04:02.722 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.722 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44059.log 2024-04-25T10:04:02.722 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59666.log.gz/var/log/ceph/ceph-client.admin.76590.log: 2024-04-25T10:04:02.722 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69829.log 2024-04-25T10:04:02.722 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76590.log.gz 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80550.log/var/log/ceph/ceph-client.admin.44059.log: 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69829.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77343.log 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44059.log.gz 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69829.log.gz 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47680.log 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.80550.log: /var/log/ceph/ceph-client.admin.77343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60726.log 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47680.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80550.log.gz 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.77343.log.gz 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47680.log.gz 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59582.log 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60726.log: gzip -5 --verbose -- 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60726.log.gz 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr: /var/log/ceph/ceph-client.0.56696.log 2024-04-25T10:04:02.723 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62466.log 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2%/var/log/ceph/ceph-client.0.56696.log: -- replaced with /var/log/ceph/ceph-client.admin.59582.log.gz 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56696.log.gz 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54977.log 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67456.log 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40256.log 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62466.log.gz 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67456.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54977.log.gz 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61837.log 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57235.log 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67456.log.gz 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46525.log 89.2% 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40256.log.gz 2024-04-25T10:04:02.724 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58444.log 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57235.log: /var/log/ceph/ceph-client.admin.46525.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61837.log.gz 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46525.log.gz 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57235.log.gz 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72252.log 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79675.log 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72252.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58444.log.gz 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59708.log 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42852.log 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59708.log: 91.2%/var/log/ceph/ceph-client.admin.79675.log: -- replaced with /var/log/ceph/ceph-client.admin.72252.log.gz 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59708.log.gz -- replaced with /var/log/ceph/ceph-client.admin.79675.log.gz 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63377.log 2024-04-25T10:04:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73453.log 2024-04-25T10:04:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63377.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42852.log.gz 2024-04-25T10:04:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56977.log 2024-04-25T10:04:02.726 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63377.log.gz/var/log/ceph/ceph-client.admin.73453.log: 2024-04-25T10:04:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51156.log 2024-04-25T10:04:02.726 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73453.log.gz 2024-04-25T10:04:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72079.log 2024-04-25T10:04:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56977.log: /var/log/ceph/ceph-client.admin.51156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57762.log 2024-04-25T10:04:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72079.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51156.log.gz 2024-04-25T10:04:02.726 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56977.log.gz 2024-04-25T10:04:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71056.log 2024-04-25T10:04:02.726 INFO:teuthology.orchestra.run.smithi139.stderr: 90.9%/var/log/ceph/ceph-client.admin.57762.log: -- replaced with /var/log/ceph/ceph-client.admin.72079.log.gz 2024-04-25T10:04:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57762.log.gz 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr: /var/log/ceph/ceph-client.admin.56921.log 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.71056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77542.log 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71056.log.gz 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78082.log 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5%/var/log/ceph/ceph-client.admin.77542.log: -- replaced with /var/log/ceph/ceph-client.admin.56921.log.gz 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73285.log 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77542.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65857.log 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.78082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49299.log 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73285.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78082.log.gz 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74908.log 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73285.log.gz 2024-04-25T10:04:02.727 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49299.log: 88.5% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49299.log.gz -- replaced with /var/log/ceph/ceph-client.admin.65857.log.gz 2024-04-25T10:04:02.728 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.728 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59498.log 2024-04-25T10:04:02.728 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59750.log 2024-04-25T10:04:02.728 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59498.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74908.log.gz 2024-04-25T10:04:02.728 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80508.log 2024-04-25T10:04:02.728 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61352.log 2024-04-25T10:04:02.728 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59498.log.gz 2024-04-25T10:04:02.728 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59750.log: /var/log/ceph/ceph-client.admin.80508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51707.log 2024-04-25T10:04:02.728 INFO:teuthology.orchestra.run.smithi139.stderr: 87.5%/var/log/ceph/ceph-client.admin.61352.log: -- replaced with /var/log/ceph/ceph-client.admin.80508.log.gz 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59750.log.gz 2024-04-25T10:04:02.728 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.728 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61352.log.gz 2024-04-25T10:04:02.728 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40475.log 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62701.log 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51707.log: /var/log/ceph/ceph-client.admin.40475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47155.log 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40475.log.gz 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62701.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51707.log.gz 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43704.log 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62701.log.gz 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51539.log 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47155.log.gz 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43972.log 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr: 87.5%/var/log/ceph/ceph-client.admin.51539.log: -- replaced with /var/log/ceph/ceph-client.admin.43704.log.gz 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64140.log 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51539.log.gz 2024-04-25T10:04:02.729 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43922.log 2024-04-25T10:04:02.730 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43972.log: /var/log/ceph/ceph-client.admin.64140.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43972.log.gz 2024-04-25T10:04:02.730 INFO:teuthology.orchestra.run.smithi139.stderr: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53024.log 2024-04-25T10:04:02.730 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64140.log.gz 2024-04-25T10:04:02.730 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42516.log 2024-04-25T10:04:02.730 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43922.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76402.log 2024-04-25T10:04:02.730 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.730 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53024.log: /var/log/ceph/ceph-client.admin.42516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70001.log 2024-04-25T10:04:02.730 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.76402.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53024.log.gz 2024-04-25T10:04:02.730 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42516.log.gz 2024-04-25T10:04:02.730 INFO:teuthology.orchestra.run.smithi139.stderr:gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76402.log.gz 2024-04-25T10:04:02.730 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56149.log 2024-04-25T10:04:02.730 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46777.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70001.log.gz 2024-04-25T10:04:02.731 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.731 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52038.log 2024-04-25T10:04:02.731 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56149.log.gz/var/log/ceph/ceph-client.admin.46777.log: 2024-04-25T10:04:02.731 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77911.log 2024-04-25T10:04:02.731 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46777.log.gz 2024-04-25T10:04:02.731 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67353.log 2024-04-25T10:04:02.731 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52038.log: /var/log/ceph/ceph-client.admin.77911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72341.log 2024-04-25T10:04:02.731 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67353.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52038.log.gz 2024-04-25T10:04:02.731 INFO:teuthology.orchestra.run.smithi139.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.77911.log.gz 2024-04-25T10:04:02.731 INFO:teuthology.orchestra.run.smithi139.stderr:gzip 92.4% -- replaced with /var/log/ceph/ceph-client.admin.67353.log.gz 2024-04-25T10:04:02.731 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65729.log 2024-04-25T10:04:02.732 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72341.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:04:02.732 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65729.log: 90.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75694.log 2024-04-25T10:04:02.732 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72341.log.gz 2024-04-25T10:04:02.732 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65729.log.gz 2024-04-25T10:04:02.732 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63004.log 2024-04-25T10:04:02.732 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.75694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58757.log 2024-04-25T10:04:02.732 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63004.log: 91.1% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75694.log.gz 2024-04-25T10:04:02.732 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63004.log.gz 2024-04-25T10:04:02.732 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56191.log 2024-04-25T10:04:02.732 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52080.log 2024-04-25T10:04:02.732 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58757.log.gz 2024-04-25T10:04:02.732 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56191.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78124.log 2024-04-25T10:04:02.733 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56191.log.gz 2024-04-25T10:04:02.733 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52080.log: gzip -5 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52080.log.gz 2024-04-25T10:04:02.733 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43032.log 2024-04-25T10:04:02.733 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.78124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69005.log 2024-04-25T10:04:02.733 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43032.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78124.log.gz 2024-04-25T10:04:02.733 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75947.log 2024-04-25T10:04:02.733 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43032.log.gz/var/log/ceph/ceph-client.admin.69005.log: 2024-04-25T10:04:02.733 INFO:teuthology.orchestra.run.smithi139.stderr:gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69005.log.gz 2024-04-25T10:04:02.733 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66591.log 2024-04-25T10:04:02.733 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.75947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43705.log 2024-04-25T10:04:02.733 INFO:teuthology.orchestra.run.smithi139.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75947.log.gz 2024-04-25T10:04:02.733 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78186.log 2024-04-25T10:04:02.734 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43705.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43705.log.gz 2024-04-25T10:04:02.734 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.66591.log.gz 2024-04-25T10:04:02.734 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68189.log 2024-04-25T10:04:02.734 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.78186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43971.log 2024-04-25T10:04:02.734 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78186.log.gz 2024-04-25T10:04:02.734 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68189.log: gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68189.log.gz 2024-04-25T10:04:02.734 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57804.log 2024-04-25T10:04:02.734 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80778.log 2024-04-25T10:04:02.734 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7%/var/log/ceph/ceph-client.admin.57804.log: -- replaced with /var/log/ceph/ceph-client.admin.43971.log.gz 2024-04-25T10:04:02.734 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73915.log 2024-04-25T10:04:02.734 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3%/var/log/ceph/ceph-client.admin.80778.log: -- replaced with /var/log/ceph/ceph-client.admin.57804.log.gz 2024-04-25T10:04:02.735 INFO:teuthology.orchestra.run.smithi139.stderr:gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80778.log.gz 2024-04-25T10:04:02.735 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.76676.log 2024-04-25T10:04:02.735 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46567.log 2024-04-25T10:04:02.735 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.76676.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73915.log.gz 2024-04-25T10:04:02.735 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76676.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53300.log 2024-04-25T10:04:02.735 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.735 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56443.log 2024-04-25T10:04:02.735 INFO:teuthology.orchestra.run.smithi139.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46567.log.gz 2024-04-25T10:04:02.735 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46399.log 2024-04-25T10:04:02.735 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56443.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53300.log.gz 2024-04-25T10:04:02.735 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80592.log 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.56443.log.gz/var/log/ceph/ceph-client.admin.46399.log: 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58673.log 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2%/var/log/ceph/ceph-client.admin.80592.log: -- replaced with /var/log/ceph/ceph-client.admin.46399.log.gz 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59792.log 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80592.log.gz 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69361.log 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58673.log: /var/log/ceph/ceph-client.admin.59792.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58673.log.gz 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59792.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68963.log 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr:gzip/var/log/ceph/ceph-client.admin.69361.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56107.log 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69361.log.gz 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48856.log 2024-04-25T10:04:02.736 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56107.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68963.log.gz 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65388.log 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56107.log.gz 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79950.log 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65388.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48856.log.gz 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65388.log.gz 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69745.log 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48772.log 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69745.log: /var/log/ceph/ceph-client.admin.79950.log: 89.0% 89.8% -- replaced with /var/log/ceph/ceph-client.admin.79950.log.gz 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69745.log.gz 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55384.log 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67046.log 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48772.log.gz 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64478.log 2024-04-25T10:04:02.737 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67046.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55384.log.gz 2024-04-25T10:04:02.738 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59900.log 2024-04-25T10:04:02.738 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67046.log.gz 2024-04-25T10:04:02.738 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74316.log 2024-04-25T10:04:02.738 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59900.log: /var/log/ceph/ceph-client.admin.64478.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59900.log.gz 87.0% -- replaced with /var/log/ceph/ceph-client.admin.64478.log.gz 2024-04-25T10:04:02.738 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.738 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74866.log 2024-04-25T10:04:02.738 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63545.log 2024-04-25T10:04:02.738 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74316.log.gz 2024-04-25T10:04:02.738 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73537.log 2024-04-25T10:04:02.738 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63545.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74866.log.gz 2024-04-25T10:04:02.738 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46105.log 2024-04-25T10:04:02.739 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63545.log.gz 2024-04-25T10:04:02.739 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74052.log 2024-04-25T10:04:02.739 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46105.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73537.log.gz 2024-04-25T10:04:02.739 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76481.log 2024-04-25T10:04:02.739 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46105.log.gz 2024-04-25T10:04:02.739 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49510.log 2024-04-25T10:04:02.739 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.74052.log: /var/log/ceph/ceph-client.admin.76481.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74052.log.gz 2024-04-25T10:04:02.739 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51954.log 2024-04-25T10:04:02.739 INFO:teuthology.orchestra.run.smithi139.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.76481.log.gz 2024-04-25T10:04:02.739 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72766.log 2024-04-25T10:04:02.739 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51954.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49510.log.gz 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41004.log 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51954.log.gz 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65346.log 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41004.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.72766.log.gz 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41004.log.gz 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53712.log 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48562.log 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53712.log: /var/log/ceph/ceph-client.admin.65346.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65346.log.gz 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53712.log.gz 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53066.log 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48562.log: -- /var/log/ceph/ceph-client.admin.73242.log 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48562.log.gz 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62053.log 2024-04-25T10:04:02.741 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73242.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53066.log.gz 2024-04-25T10:04:02.741 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70408.log 2024-04-25T10:04:02.741 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73242.log.gz 2024-04-25T10:04:02.741 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53616.log 2024-04-25T10:04:02.741 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.70408.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62053.log.gz 2024-04-25T10:04:02.741 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70408.log.gz 2024-04-25T10:04:02.741 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67293.log 2024-04-25T10:04:02.741 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78228.log 2024-04-25T10:04:02.741 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53616.log: /var/log/ceph/ceph-client.admin.67293.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53616.log.gz 2024-04-25T10:04:02.741 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52854.log 2024-04-25T10:04:02.741 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.78228.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.67293.log.gz 2024-04-25T10:04:02.741 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.78228.log.gz 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51581.log 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55061.log 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52854.log.gz/var/log/ceph/ceph-client.admin.51581.log: 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69319.log 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5%/var/log/ceph/ceph-client.admin.55061.log: -- replaced with /var/log/ceph/ceph-client.admin.51581.log.gz 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55061.log.gz 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72443.log 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48184.log 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72443.log: /var/log/ceph/ceph-client.admin.69319.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69319.log.gz 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi139.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.72443.log.gz 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64226.log 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46903.log 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64226.log: /var/log/ceph/ceph-client.admin.48184.log: 88.9% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48184.log.gz 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64226.log.gz 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52673.log 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49636.log 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46903.log.gz 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61903.log 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49636.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52673.log.gz 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60433.log 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49636.log.gz 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61687.log 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60433.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60433.log.gz 2024-04-25T10:04:02.744 INFO:teuthology.orchestra.run.smithi139.stderr: 90.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79320.log 2024-04-25T10:04:02.744 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61903.log.gz/var/log/ceph/ceph-client.admin.61687.log: 2024-04-25T10:04:02.744 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64646.log 2024-04-25T10:04:02.744 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61687.log.gz 2024-04-25T10:04:02.744 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63503.log 2024-04-25T10:04:02.744 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.79320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69235.log 2024-04-25T10:04:02.744 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79320.log.gz 2024-04-25T10:04:02.744 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64646.log: /var/log/ceph/ceph-client.admin.63503.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64646.log.gz 2024-04-25T10:04:02.744 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63503.log.gz 2024-04-25T10:04:02.744 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52122.log 2024-04-25T10:04:02.744 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48688.log 2024-04-25T10:04:02.744 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69235.log.gz 2024-04-25T10:04:02.745 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49552.log 2024-04-25T10:04:02.745 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48688.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52122.log.gz 2024-04-25T10:04:02.745 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48688.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64604.log 2024-04-25T10:04:02.745 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:02.745 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45580.log 2024-04-25T10:04:02.745 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64604.log: /var/log/ceph/ceph-client.admin.49552.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49552.log.gz 2024-04-25T10:04:02.745 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64604.log.gz 2024-04-25T10:04:02.745 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46735.log 2024-04-25T10:04:02.745 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.45580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73873.log 2024-04-25T10:04:02.745 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4%/var/log/ceph/ceph-client.admin.46735.log: -- replaced with /var/log/ceph/ceph-client.admin.45580.log.gz 2024-04-25T10:04:02.745 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59016.log 2024-04-25T10:04:02.745 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46735.log.gz 2024-04-25T10:04:02.746 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.73873.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73873.log.gz 2024-04-25T10:04:02.746 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57493.log 2024-04-25T10:04:02.746 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69277.log 2024-04-25T10:04:02.746 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57493.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59016.log.gz 2024-04-25T10:04:02.746 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72937.log 2024-04-25T10:04:02.746 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.69277.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.57493.log.gz 2024-04-25T10:04:02.746 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69277.log.gz 2024-04-25T10:04:02.746 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80860.log 2024-04-25T10:04:02.746 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66549.log 2024-04-25T10:04:02.746 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.72937.log: /var/log/ceph/ceph-client.admin.80860.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80860.log.gz 2024-04-25T10:04:02.746 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.72937.log.gz 2024-04-25T10:04:02.746 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68392.log 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48520.log 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66549.log.gz 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50197.log 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48520.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68392.log.gz 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42228.log 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48520.log.gz 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68434.log 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42228.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50197.log.gz 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42228.log.gzgzip 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50932.log 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48604.log 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50932.log: /var/log/ceph/ceph-client.admin.68434.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68434.log.gz 2024-04-25T10:04:02.747 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50932.log.gz 2024-04-25T10:04:02.748 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48604.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48604.log.gz 2024-04-25T10:04:02.748 INFO:teuthology.orchestra.run.smithi139.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:04:03.301 INFO:teuthology.orchestra.run.smithi139.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:04:03.492 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:04:03.500 INFO:teuthology.orchestra.run.smithi139.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:04:03.545 INFO:teuthology.orchestra.run.smithi150.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:04:03.545 INFO:teuthology.orchestra.run.smithi150.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:04:03.563 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:04:03.565 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-25T10:04:03.565 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m1.316s 2024-04-25T10:04:03.565 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m5.669s 2024-04-25T10:04:03.565 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.238s 2024-04-25T10:04:03.631 INFO:teuthology.orchestra.run.smithi139.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:04:03.729 INFO:teuthology.orchestra.run.smithi139.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:04:03.837 INFO:teuthology.orchestra.run.smithi139.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:04:03.838 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T10:04:03.839 INFO:teuthology.orchestra.run.smithi139.stderr:real 0m1.617s 2024-04-25T10:04:03.839 INFO:teuthology.orchestra.run.smithi139.stderr:user 0m7.016s 2024-04-25T10:04:03.839 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m0.645s 2024-04-25T10:04:03.839 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:04:03.839 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032/remote/smithi139/log 2024-04-25T10:04:03.840 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:04:05.457 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032/remote/smithi150/log 2024-04-25T10:04:05.458 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:04:06.306 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:04:06.316 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:04:06.316 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:04:06.319 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:04:06.404 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:04:06.413 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:04:06.413 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:04:06.416 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:04:06.429 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:04:06.432 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:04:06.433 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:04:06.433 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-04-25T10:04:06.434 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 +1387us[+1399us] +/- 63ms 2024-04-25T10:04:06.434 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -1181us[-1169us] +/- 58ms 2024-04-25T10:04:06.434 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 -110us[ -110us] +/- 79ms 2024-04-25T10:04:06.434 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:04:06.437 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:04:06.438 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-25T10:04:06.438 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 +1384us[+1406us] +/- 63ms 2024-04-25T10:04:06.438 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -1199us[-1199us] +/- 58ms 2024-04-25T10:04:06.438 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -109us[ -109us] +/- 79ms 2024-04-25T10:04:06.438 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:04:06.439 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:04:06.448 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:04:06.449 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:04:06.467 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:04:06.560 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:04:06.629 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:04:06.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:06.658 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:04:06.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:06.687 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:04:06.698 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:04:06.717 INFO:teuthology.task.internal:Duration was 1074.862519 seconds 2024-04-25T10:04:06.718 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:04:06.751 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:04:06.752 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:04:06.754 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:04:06.790 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:04:06.792 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:04:07.107 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:04:07.107 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T10:04:07.108 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:04:07.168 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T10:04:07.168 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:04:07.194 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:04:07.194 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:04:07.211 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:04:07.261 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:04:07.261 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:04:07.276 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:04:07.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:04:07.414 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:04:07.415 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:04:07.438 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:04:07.462 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:04:07.473 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:04:07.481 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:04:07.500 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-04-25T10:04:07.523 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-04-25T10:04:07.536 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:04:07.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:07.567 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:04:07.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:07.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:04:07.600 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:04:07.600 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032/remote/smithi139 2024-04-25T10:04:07.601 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:04:07.641 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673032/remote/smithi150 2024-04-25T10:04:07.642 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:04:07.674 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:04:07.674 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:04:07.678 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:04:07.723 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:04:07.735 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:04:07.735 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:04:07.760 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:04:07.760 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:04:07.763 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:04:07.774 INFO:teuthology.orchestra.run.smithi139.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:04 /home/ubuntu/cephtest 2024-04-25T10:04:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:04 /home/ubuntu/cephtest 2024-04-25T10:04:07.779 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:04:07.793 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:04:07.863 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} duration: 1074.8625185489655 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T10:04:07.864 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:04:07.963 INFO:teuthology.run:pass